2024-03-06T07:07:59.794 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-06T07:07:59.794 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} --archive /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049 --verbose --owner scheduled_leonidus@teuthology --name leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/orig.config.yaml 2024-03-06T07:07:59.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-06T07:07:59.922 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049 branch: wip-lusov-qdb description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} email: null first_in_suite: false job_id: '7583049' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - \(MDS_INSUFFICIENT_STANDBY\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(MDS_DAMAGE\) - \(FS_DEGRADED\) sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: disabled: true install: ceph: flavor: default sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 5072 sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcrt7eHGyP45YOFjqvGwfgQruR9VE8zc0jHwe2MqzBnRuLZJfuLrloiAoLCPGAIdrYHw1ZthpXAFtdi5L8irSc= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTmalFYrhoj0FNk44EdEqJPl48O2hb4I1sAh3U3CBL03a7DTQ+SCm7f6Ki9qDFzw+00R9Y1L051xGFBKzR/JZw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_failover teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-06_07:00:27 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-06T07:07:59.923 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa; will attempt to use it 2024-03-06T07:07:59.923 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks 2024-03-06T07:07:59.923 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-06T07:07:59.925 INFO:teuthology.task.internal:Checking packages... 2024-03-06T07:07:59.949 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd' 2024-03-06T07:07:59.949 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-06T07:07:59.949 INFO:teuthology.packaging:ref: None 2024-03-06T07:07:59.949 INFO:teuthology.packaging:tag: None 2024-03-06T07:07:59.949 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-06T07:07:59.949 INFO:teuthology.packaging:sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:07:59.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-06T07:08:00.115 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739.g1bc250c7 2024-03-06T07:08:00.116 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-06T07:08:00.122 INFO:teuthology.task.internal:no buildpackages task found 2024-03-06T07:08:00.123 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-06T07:08:00.169 INFO:teuthology.task.internal:Saving configuration 2024-03-06T07:08:00.186 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-06T07:08:00.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:08:00.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:55.667680', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcrt7eHGyP45YOFjqvGwfgQruR9VE8zc0jHwe2MqzBnRuLZJfuLrloiAoLCPGAIdrYHw1ZthpXAFtdi5L8irSc='} 2024-03-06T07:08:00.244 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:55.669376', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTmalFYrhoj0FNk44EdEqJPl48O2hb4I1sAh3U3CBL03a7DTQ+SCm7f6Ki9qDFzw+00R9Y1L051xGFBKzR/JZw='} 2024-03-06T07:08:00.244 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-06T07:08:00.251 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-06T07:08:00.251 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-03-06T07:08:00.251 INFO:teuthology.run_tasks:Running task console_log... 2024-03-06T07:08:00.348 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbb4150f0d0>, signals=[15]) 2024-03-06T07:08:00.349 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-06T07:08:00.354 INFO:teuthology.task.internal:Opening connections... 2024-03-06T07:08:00.354 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:08:00.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:08:00.479 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:08:00.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:08:00.566 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-06T07:08:00.573 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-03-06T07:08:00.590 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-03-06T07:08:00.590 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="9" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="9" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el9" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:LOGO="fedora-logo-icon" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-06T07:08:00.647 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2024-03-06T07:08:00.648 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-06T07:08:00.648 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-06T07:08:00.648 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-06T07:08:00.648 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-03-06T07:08:00.672 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2024-03-06T07:08:00.689 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2024-03-06T07:08:00.689 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2024-03-06T07:08:00.745 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream" 2024-03-06T07:08:00.745 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9" 2024-03-06T07:08:00.745 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-06T07:08:00.746 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-06T07:08:00.747 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2024-03-06T07:08:00.772 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-06T07:08:00.781 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-06T07:08:00.844 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-06T07:08:00.844 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-03-06T07:08:00.847 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2024-03-06T07:08:00.863 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-06T07:08:00.877 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-06T07:08:00.878 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-03-06T07:08:00.905 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2024-03-06T07:08:01.000 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-06T07:08:01.045 INFO:teuthology.run_tasks:Running task kernel... 2024-03-06T07:08:01.060 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-06T07:08:01.060 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-03-06T07:08:01.060 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-03-06T07:08:01.060 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-06T07:08:01.061 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-06T07:08:01.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:08:01.077 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-06T07:08:01.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:08:01.079 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-06T07:08:01.134 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-361.el9.x86_64 2024-03-06T07:08:01.135 INFO:teuthology.task.kernel:Running kernel on smithi163: 5.14.0-361.el9.x86_64 2024-03-06T07:08:01.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum install -y kernel 2024-03-06T07:08:01.137 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-361.el9.x86_64 2024-03-06T07:08:01.137 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.14.0-361.el9.x86_64 2024-03-06T07:08:01.137 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2024-03-06T07:08:03.469 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.7 MB 00:00 2024-03-06T07:08:03.785 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.7 MB 00:00 2024-03-06T07:08:04.911 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-06T07:08:05.427 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-06T07:08:05.702 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-06T07:08:09.319 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-03-06T07:08:15.787 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.1 MB 00:02 2024-03-06T07:08:16.058 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 821 kB/s | 8.0 MB 00:10 2024-03-06T07:08:17.546 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 15 kB 00:00 2024-03-06T07:08:18.034 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-06T07:08:18.941 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-06T07:08:22.185 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.1 MB 00:00 2024-03-06T07:08:23.777 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-06T07:08:24.022 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-06T07:08:26.346 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-06T07:08:26.346 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-06T07:08:26.442 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-06T07:08:26.447 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout:Install 4 Packages 2024-03-06T07:08:26.448 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:08:26.449 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 100 M 2024-03-06T07:08:26.449 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 123 M 2024-03-06T07:08:26.449 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-03-06T07:08:29.211 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 3.9 MB/s | 21 MB 00:05 2024-03-06T07:08:29.265 INFO:teuthology.orchestra.run.smithi017.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 2.2 MB/s | 6.0 MB 00:02 2024-03-06T07:08:32.573 INFO:teuthology.orchestra.run.smithi017.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 3.4 MB/s | 21 MB 00:06 2024-03-06T07:08:34.169 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-06T07:08:34.357 INFO:teuthology.orchestra.run.smithi017.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 6.6 MB/s | 33 MB 00:05 2024-03-06T07:08:35.340 INFO:teuthology.orchestra.run.smithi017.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 4.4 MB/s | 39 MB 00:08 2024-03-06T07:08:35.340 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:08:35.340 INFO:teuthology.orchestra.run.smithi017.stdout:Total 11 MB/s | 100 MB 00:08 2024-03-06T07:08:36.267 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-03-06T07:08:36.392 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-06T07:08:36.392 INFO:teuthology.orchestra.run.smithi163.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-06T07:08:36.478 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: Package Architecture Version Repository Size 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-03-06T07:08:36.488 INFO:teuthology.orchestra.run.smithi163.stdout:================================================================================ 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi163.stdout:Install 4 Packages 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 100 M 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi163.stdout:Installed size: 123 M 2024-03-06T07:08:36.489 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-03-06T07:08:36.731 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-03-06T07:08:36.731 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-03-06T07:08:37.762 INFO:teuthology.orchestra.run.smithi163.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 5.4 MB/s | 6.0 MB 00:01 2024-03-06T07:08:38.125 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-03-06T07:08:38.125 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-03-06T07:08:38.753 INFO:teuthology.orchestra.run.smithi163.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 10 MB/s | 21 MB 00:02 2024-03-06T07:08:40.328 INFO:teuthology.orchestra.run.smithi163.stdout:(3/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 11 MB/s | 39 MB 00:03 2024-03-06T07:08:40.620 INFO:teuthology.orchestra.run.smithi163.stdout:(4/4): kernel-modules-core-5.14.0-427.el9.x86_6 12 MB/s | 33 MB 00:02 2024-03-06T07:08:40.621 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:08:40.621 INFO:teuthology.orchestra.run.smithi163.stdout:Total 24 MB/s | 100 MB 00:04 2024-03-06T07:08:40.985 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-03-06T07:08:41.510 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-03-06T07:08:41.665 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:08:41.799 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:41.893 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-03-06T07:08:41.893 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-03-06T07:08:43.503 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-03-06T07:08:43.503 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-03-06T07:08:44.710 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:44.893 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:46.206 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-03-06T07:08:46.864 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:08:47.004 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:49.799 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:08:49.991 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:50.027 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:50.155 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:08:54.710 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:08:54.842 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:08:55.320 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:08:59.622 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:26.140 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:26.140 INFO:teuthology.orchestra.run.smithi017.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-06T07:09:26.140 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:09:26.227 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:09:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-06T07:09:27.230 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:27.231 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:09:27.231 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-03-06T07:09:27.744 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:27.897 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:27.898 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:27.898 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-06T07:09:27.898 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-06T07:09:27.898 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-06T07:09:27.898 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-06T07:09:27.898 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-03-06T07:09:27.899 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:28.025 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:28.025 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:28.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.06 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 68.el9 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Group : Unspecified 2024-03-06T07:09:28.063 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 8264254 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : builder@centos.org 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : CentOS 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2024-03-06T07:09:28.064 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:09:28.065 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2024-03-06T07:09:28.065 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-06T07:09:28.065 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 5.14.0-427.el9.x86_64 2024-03-06T07:09:28.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-06T07:09:28.706 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-03-06T07:09:28.974 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:28.974 INFO:teuthology.orchestra.run.smithi163.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-06T07:09:28.974 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:09:29.060 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:29.470 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:29.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-06T07:09:29.470 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-06T07:09:29.471 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-06T07:09:29.914 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-06T07:09:29.914 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:09:29.914 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:09:29.915 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-03-06T07:09:30.160 INFO:teuthology.orchestra.run.smithi017.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-06T07:09:30.187 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-03-06T07:09:30.198 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-06T07:09:30.213 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.25RLzpV8bn 2024-03-06T07:09:30.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.25RLzpV8bn 2024-03-06T07:09:30.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.25RLzpV8bn 2024-03-06T07:09:30.423 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:30.432 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.25RLzpV8bn is 6KB 2024-03-06T07:09:30.481 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.25RLzpV8bn 2024-03-06T07:09:30.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2024-03-06T07:09:30.573 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-06T07:09:30.573 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-06T07:09:30.573 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-06T07:09:30.573 INFO:teuthology.orchestra.run.smithi017.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-06T07:09:30.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-06T07:09:30.580 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:30.581 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:30.582 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-06T07:09:30.582 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-06T07:09:30.582 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-06T07:09:30.582 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-03-06T07:09:30.582 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=None, version=distro) 2024-03-06T07:09:30.582 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q kernel | sort -rV | head -n 1 2024-03-06T07:09:30.704 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-06T07:09:30.704 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-06T07:09:30.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2024-03-06T07:09:30.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.06 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 68.el9 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2024-03-06T07:09:30.748 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Group : Unspecified 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 8264254 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : builder@centos.org 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : CentOS 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-06T07:09:30.749 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:09:30.750 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2024-03-06T07:09:30.751 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-06T07:09:30.751 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 5.14.0-427.el9.x86_64 2024-03-06T07:09:30.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-06T07:09:31.481 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2024-03-06T07:09:32.938 INFO:teuthology.orchestra.run.smithi163.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-06T07:09:32.959 INFO:teuthology.orchestra.run.smithi163.stderr:done 2024-03-06T07:09:32.961 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-03-06T07:09:32.976 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.my55ARnELq 2024-03-06T07:09:32.977 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.my55ARnELq 2024-03-06T07:09:33.046 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.my55ARnELq 2024-03-06T07:09:33.189 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.my55ARnELq is 6KB 2024-03-06T07:09:33.239 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.my55ARnELq 2024-03-06T07:09:33.254 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2024-03-06T07:09:33.320 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-06T07:09:33.320 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-06T07:09:33.321 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-06T07:09:33.321 INFO:teuthology.orchestra.run.smithi163.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-06T07:09:33.322 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-06T07:09:33.463 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2024-03-06T07:10:00.756 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-06T07:10:00.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:10:00.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:03.468 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-06T07:10:03.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:10:03.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:19.196 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-03-06T07:10:22.012 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-06T07:10:28.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:10:28.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:31.016 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:10:31.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:34.076 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-06T07:10:46.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:10:46.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:10:49.148 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-06T07:11:04.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:11:04.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:11:22.688 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2024-03-06T07:11:28.208 DEBUG:teuthology.orchestra.remote:timed out 2024-03-06T07:11:40.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:11:40.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:11:40.506 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-06T07:11:40.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:11:40.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:11:40.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:11:40.912 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-06T07:11:40.913 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-06T07:11:40.927 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-427.el9.x86_64 2024-03-06T07:11:40.927 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-06T07:11:40.927 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-06T07:11:40.927 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-06T07:11:41.117 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-06T07:11:41.570 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:11:41.570 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-427.el9.x86_64"... 2024-03-06T07:11:41.570 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2024-03-06T07:11:41.584 INFO:teuthology.orchestra.run.smithi163.stdout:5.14.0-427.el9.x86_64 2024-03-06T07:11:41.585 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-06T07:11:41.585 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-06T07:11:41.585 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-06T07:11:41.929 DEBUG:teuthology.parallel:result is None 2024-03-06T07:11:42.585 DEBUG:teuthology.parallel:result is None 2024-03-06T07:11:42.586 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-06T07:11:42.593 INFO:teuthology.task.internal:Creating test directory... 2024-03-06T07:11:42.594 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-06T07:11:42.596 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-06T07:11:42.611 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-06T07:11:42.645 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-06T07:11:42.695 INFO:teuthology.task.internal:Creating archive directory... 2024-03-06T07:11:42.696 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-06T07:11:42.698 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-06T07:11:42.731 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-06T07:11:42.745 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-06T07:11:42.746 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-06T07:11:42.772 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-06T07:11:42.845 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:42.865 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:42.896 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:42.904 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-06T07:11:42.906 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-06T07:11:42.913 INFO:teuthology.task.internal:Configuring sudo... 2024-03-06T07:11:42.913 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-06T07:11:42.915 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-06T07:11:42.943 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-06T07:11:42.953 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-06T07:11:42.953 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-06T07:11:42.990 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-06T07:11:43.005 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:43.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:43.142 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:43.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:43.274 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:11:43.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-06T07:11:43.347 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:43.387 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-06T07:11:43.443 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:43.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-06T07:11:43.567 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:11:43.567 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-06T07:11:43.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-03-06T07:11:43.636 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2024-03-06T07:11:43.675 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:11:43.715 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:11:44.234 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-06T07:11:44.243 INFO:teuthology.task.internal:Starting timer... 2024-03-06T07:11:44.243 INFO:teuthology.run_tasks:Running task pcp... 2024-03-06T07:11:44.291 INFO:teuthology.run_tasks:Running task selinux... 2024-03-06T07:11:44.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2024-03-06T07:11:44.391 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2024-03-06T07:11:44.392 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2024-03-06T07:11:44.462 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: 2024-03-06T07:11:44.464 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-06T07:11:44.464 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2024-03-06T07:11:44.484 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2024-03-06T07:11:44.485 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2024-03-06T07:11:44.504 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2024-03-06T07:11:44.505 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2024-03-06T07:11:44.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:11:44.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:11:44.557 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:11:44.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:11:44.584 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-06T07:11:44.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2024-03-06T07:11:44.636 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2024-03-06T07:11:44.672 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-06T07:11:44.684 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-06T07:11:44.703 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-06T07:11:44.705 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-06T07:16:02.878 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2024-03-06T07:16:02.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:16:02.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:16:02.975 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-06T07:16:03.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-06T07:16:03.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:16:03.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-06T07:16:03.157 DEBUG:teuthology.orchestra.run.smithi163:> true 2024-03-06T07:16:03.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2024-03-06T07:16:03.235 INFO:teuthology.run_tasks:Running task clock... 2024-03-06T07:16:03.246 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-06T07:16:03.246 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-06T07:16:03.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:16:03.249 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-06T07:16:03.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:16:03.279 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-06T07:16:03.296 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-06T07:16:03.305 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-06T07:16:03.322 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-06T07:16:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2024-03-06T07:16:03.339 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2024-03-06T07:16:03.352 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2024-03-06T07:16:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-06T07:16:03.366 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2024-03-06T07:16:03.371 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-06T07:16:03.382 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-06T07:16:03.399 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-06T07:16:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.429 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.451 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:16:03.455 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:16:03.455 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-06T07:16:03.456 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.456 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.456 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.456 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:16:03.456 INFO:teuthology.run_tasks:Running task install... 2024-03-06T07:16:03.466 DEBUG:teuthology.task.install:project ceph 2024-03-06T07:16:03.467 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}} 2024-03-06T07:16:03.467 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'} 2024-03-06T07:16:03.467 INFO:teuthology.task.install:Using flavor: default 2024-03-06T07:16:03.473 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-06T07:16:03.474 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-06T07:16:03.474 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-06T07:16:03.474 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:16:03.476 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'tag': None, 'wait_for_package': False} 2024-03-06T07:16:03.477 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:16:03.627 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-06T07:16:03.627 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-06T07:16:03.630 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/ 2024-03-06T07:16:03.631 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1739.g1bc250c7 2024-03-06T07:16:03.903 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-06T07:16:03.903 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:16:03.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-06T07:16:03.910 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-06T07:16:03.910 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:16:03.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-06T07:16:03.933 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-06T07:16:03.933 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-06T07:16:03.939 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-06T07:16:03.939 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-06T07:16:04.003 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-06T07:16:04.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-06T07:16:04.084 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-06T07:16:04.085 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-06T07:16:04.150 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2024-03-06T07:16:04.151 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2024-03-06T07:16:04.152 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2024-03-06T07:16:04.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2024-03-06T07:16:04.393 INFO:teuthology.orchestra.run.smithi163.stdout:74 files removed 2024-03-06T07:16:04.398 INFO:teuthology.orchestra.run.smithi017.stdout:74 files removed 2024-03-06T07:16:04.420 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-06T07:16:04.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-06T07:16:05.838 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 76 kB/s | 83 kB 00:01 2024-03-06T07:16:05.845 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 75 kB/s | 83 kB 00:01 2024-03-06T07:16:06.640 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-06T07:16:06.662 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 13 kB/s | 10 kB 00:00 2024-03-06T07:16:07.388 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-03-06T07:16:07.407 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-03-06T07:16:08.023 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-06T07:16:08.208 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-03-06T07:16:09.667 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-06T07:16:09.699 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-06T07:16:10.532 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-06T07:16:10.664 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-06T07:16:12.526 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-06T07:16:15.290 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-03-06T07:16:16.626 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-03-06T07:16:18.177 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-03-06T07:16:19.662 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-06T07:16:21.532 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-03-06T07:16:23.058 INFO:teuthology.orchestra.run.smithi163.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 15 kB 00:00 2024-03-06T07:16:24.862 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-06T07:16:25.642 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 21 MB 00:02 2024-03-06T07:16:27.525 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:27.526 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:27.541 INFO:teuthology.orchestra.run.smithi017.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-06T07:16:27.542 INFO:teuthology.orchestra.run.smithi017.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-06T07:16:27.543 INFO:teuthology.orchestra.run.smithi017.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-06T07:16:27.544 INFO:teuthology.orchestra.run.smithi017.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:27.547 INFO:teuthology.orchestra.run.smithi017.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:27.550 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-06T07:16:27.551 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-06T07:16:27.637 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-03-06T07:16:27.666 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-06T07:16:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-03-06T07:16:27.666 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-06T07:16:27.666 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-03-06T07:16:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-06T07:16:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-06T07:16:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2024-03-06T07:16:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-06T07:16:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-06T07:16:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-06T07:16:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-06T07:16:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-06T07:16:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-06T07:16:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-06T07:16:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-06T07:16:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-06T07:16:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-06T07:16:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:Installing weak dependencies: 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:16:27.680 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-03-06T07:16:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================ 2024-03-06T07:16:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:Install 150 Packages 2024-03-06T07:16:27.681 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 26 Packages 2024-03-06T07:16:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:16:27.682 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 224 M 2024-03-06T07:16:27.682 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-03-06T07:16:28.571 INFO:teuthology.orchestra.run.smithi017.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-03-06T07:16:29.288 INFO:teuthology.orchestra.run.smithi017.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 1.2 MB/s | 888 kB 00:00 2024-03-06T07:16:29.372 INFO:teuthology.orchestra.run.smithi017.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 1.7 MB/s | 145 kB 00:00 2024-03-06T07:16:30.089 INFO:teuthology.orchestra.run.smithi017.stdout:(4/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 3.2 MB/s | 2.3 MB 00:00 2024-03-06T07:16:30.489 INFO:teuthology.orchestra.run.smithi017.stdout:(5/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-03-06T07:16:30.559 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-06T07:16:31.489 INFO:teuthology.orchestra.run.smithi017.stdout:(6/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 1.6 MB/s | 5.3 MB 00:03 2024-03-06T07:16:31.681 INFO:teuthology.orchestra.run.smithi017.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-03-06T07:16:33.096 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:33.096 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-06T07:16:33.110 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-06T07:16:33.111 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-06T07:16:33.112 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-06T07:16:33.113 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:33.115 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-06T07:16:33.119 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-06T07:16:33.119 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-06T07:16:33.200 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2024-03-06T07:16:33.227 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-06T07:16:33.227 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2024-03-06T07:16:33.227 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-06T07:16:33.227 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2024-03-06T07:16:33.227 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 6.6 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.3 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 888 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 145 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 1.5 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 138 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 1.8 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 7.4 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 49 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 9.8 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 47 M 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 264 k 2024-03-06T07:16:33.228 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 748 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 220 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 27 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 18 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 31 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 747 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 127 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:3.0.7-27.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-06T07:16:33.229 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 162 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 321 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 298 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 99 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 85 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 171 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager x86_64 1:1.46.0-1.el9 CentOS-BaseOS 2.3 M 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 CentOS-BaseOS 1.8 M 2024-03-06T07:16:33.230 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 CentOS-BaseOS 41 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 CentOS-BaseOS 247 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.4 M 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 3.1 M 2024-03-06T07:16:33.231 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: openssl x86_64 1:3.0.7-27.el9 CentOS-BaseOS 1.2 M 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs x86_64 1:3.0.7-27.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-06T07:16:33.232 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 20 M 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 31 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 2.3 M 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 248 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 4.7 M 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 17 M 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1739.g1bc250c7.el9 ceph-noarch 15 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 25 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-06T07:16:33.233 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 163 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 501 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 5.1 M 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-06T07:16:33.234 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-06T07:16:33.235 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 45 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:19.0.0-1739.g1bc250c7.el9 ceph 131 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-06T07:16:33.236 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-06T07:16:33.237 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-06T07:16:33.238 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-06T07:16:33.239 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-06T07:16:33.240 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout:Installing weak dependencies: 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout:============================================================================================ 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout:Install 150 Packages 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 26 Packages 2024-03-06T07:16:33.241 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:16:33.242 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 224 M 2024-03-06T07:16:33.242 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2024-03-06T07:16:34.201 INFO:teuthology.orchestra.run.smithi163.stdout:(1/176): ceph-19.0.0-1739.g1bc250c7.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-06T07:16:36.151 INFO:teuthology.orchestra.run.smithi163.stdout:(2/176): ceph-fuse-19.0.0-1739.g1bc250c7.el9.x8 456 kB/s | 888 kB 00:01 2024-03-06T07:16:36.476 INFO:teuthology.orchestra.run.smithi163.stdout:(3/176): ceph-immutable-object-cache-19.0.0-173 447 kB/s | 145 kB 00:00 2024-03-06T07:16:37.839 INFO:teuthology.orchestra.run.smithi017.stdout:(8/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el9 1.6 MB/s | 9.8 MB 00:06 2024-03-06T07:16:37.939 INFO:teuthology.orchestra.run.smithi017.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 252 kB/s | 25 kB 00:00 2024-03-06T07:16:38.943 INFO:teuthology.orchestra.run.smithi163.stdout:(4/176): ceph-base-19.0.0-1739.g1bc250c7.el9.x8 1.0 MB/s | 5.3 MB 00:05 2024-03-06T07:16:41.310 INFO:teuthology.orchestra.run.smithi163.stdout:(5/176): ceph-mds-19.0.0-1739.g1bc250c7.el9.x86 489 kB/s | 2.3 MB 00:04 2024-03-06T07:16:41.427 INFO:teuthology.orchestra.run.smithi163.stdout:(6/176): ceph-mgr-19.0.0-1739.g1bc250c7.el9.x86 616 kB/s | 1.5 MB 00:02 2024-03-06T07:16:47.910 INFO:teuthology.orchestra.run.smithi163.stdout:(7/176): ceph-mon-19.0.0-1739.g1bc250c7.el9.x86 727 kB/s | 4.7 MB 00:06 2024-03-06T07:16:53.022 INFO:teuthology.orchestra.run.smithi017.stdout:(10/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 800 kB/s | 17 MB 00:21 2024-03-06T07:16:53.555 INFO:teuthology.orchestra.run.smithi017.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 413 kB/s | 220 kB 00:00 2024-03-06T07:16:53.664 INFO:teuthology.orchestra.run.smithi017.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 250 kB/s | 27 kB 00:00 2024-03-06T07:16:53.756 INFO:teuthology.orchestra.run.smithi017.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 341 kB/s | 31 kB 00:00 2024-03-06T07:16:55.223 INFO:teuthology.orchestra.run.smithi017.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 509 kB/s | 747 kB 00:01 2024-03-06T07:16:55.556 INFO:teuthology.orchestra.run.smithi017.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 490 kB/s | 163 kB 00:00 2024-03-06T07:16:55.806 INFO:teuthology.orchestra.run.smithi017.stdout:(16/176): librados-devel-19.0.0-1739.g1bc250c7. 507 kB/s | 127 kB 00:00 2024-03-06T07:16:57.185 INFO:teuthology.orchestra.run.smithi017.stdout:(17/176): libradosstriper1-19.0.0-1739.g1bc250c 365 kB/s | 501 kB 00:01 2024-03-06T07:16:57.759 INFO:teuthology.orchestra.run.smithi163.stdout:(8/176): ceph-common-19.0.0-1739.g1bc250c7.el9. 873 kB/s | 20 MB 00:23 2024-03-06T07:16:57.875 INFO:teuthology.orchestra.run.smithi163.stdout:(9/176): ceph-selinux-19.0.0-1739.g1bc250c7.el9 216 kB/s | 25 kB 00:00 2024-03-06T07:17:00.942 INFO:teuthology.orchestra.run.smithi163.stdout:(10/176): ceph-radosgw-19.0.0-1739.g1bc250c7.el 768 kB/s | 9.8 MB 00:13 2024-03-06T07:17:01.106 INFO:teuthology.orchestra.run.smithi017.stdout:(18/176): ceph-common-19.0.0-1739.g1bc250c7.el9 636 kB/s | 20 MB 00:32 2024-03-06T07:17:01.207 INFO:teuthology.orchestra.run.smithi017.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 452 kB/s | 45 kB 00:00 2024-03-06T07:17:01.334 INFO:teuthology.orchestra.run.smithi163.stdout:(11/176): cephfs-mirror-19.0.0-1739.g1bc250c7.e 563 kB/s | 220 kB 00:00 2024-03-06T07:17:01.426 INFO:teuthology.orchestra.run.smithi163.stdout:(12/176): cephfs-shell-19.0.0-1739.g1bc250c7.el 295 kB/s | 27 kB 00:00 2024-03-06T07:17:01.473 INFO:teuthology.orchestra.run.smithi017.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 493 kB/s | 131 kB 00:00 2024-03-06T07:17:01.518 INFO:teuthology.orchestra.run.smithi163.stdout:(13/176): libcephfs-devel-19.0.0-1739.g1bc250c7 341 kB/s | 31 kB 00:00 2024-03-06T07:17:01.815 INFO:teuthology.orchestra.run.smithi017.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 473 kB/s | 162 kB 00:00 2024-03-06T07:17:02.391 INFO:teuthology.orchestra.run.smithi017.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 558 kB/s | 321 kB 00:00 2024-03-06T07:17:02.560 INFO:teuthology.orchestra.run.smithi163.stdout:(14/176): libcephfs2-19.0.0-1739.g1bc250c7.el9. 717 kB/s | 747 kB 00:01 2024-03-06T07:17:02.860 INFO:teuthology.orchestra.run.smithi163.stdout:(15/176): libcephsqlite-19.0.0-1739.g1bc250c7.e 545 kB/s | 163 kB 00:00 2024-03-06T07:17:02.874 INFO:teuthology.orchestra.run.smithi017.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 616 kB/s | 298 kB 00:00 2024-03-06T07:17:03.042 INFO:teuthology.orchestra.run.smithi017.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 596 kB/s | 99 kB 00:00 2024-03-06T07:17:03.052 INFO:teuthology.orchestra.run.smithi163.stdout:(16/176): ceph-osd-19.0.0-1739.g1bc250c7.el9.x8 796 kB/s | 17 MB 00:21 2024-03-06T07:17:03.152 INFO:teuthology.orchestra.run.smithi163.stdout:(17/176): librados-devel-19.0.0-1739.g1bc250c7. 434 kB/s | 127 kB 00:00 2024-03-06T07:17:03.225 INFO:teuthology.orchestra.run.smithi017.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 465 kB/s | 85 kB 00:00 2024-03-06T07:17:04.044 INFO:teuthology.orchestra.run.smithi163.stdout:(18/176): libradosstriper1-19.0.0-1739.g1bc250c 506 kB/s | 501 kB 00:00 2024-03-06T07:17:04.152 INFO:teuthology.orchestra.run.smithi163.stdout:(19/176): python3-ceph-argparse-19.0.0-1739.g1b 417 kB/s | 45 kB 00:00 2024-03-06T07:17:04.403 INFO:teuthology.orchestra.run.smithi163.stdout:(20/176): python3-ceph-common-19.0.0-1739.g1bc2 525 kB/s | 131 kB 00:00 2024-03-06T07:17:04.736 INFO:teuthology.orchestra.run.smithi163.stdout:(21/176): python3-cephfs-19.0.0-1739.g1bc250c7. 485 kB/s | 162 kB 00:00 2024-03-06T07:17:05.220 INFO:teuthology.orchestra.run.smithi163.stdout:(22/176): python3-rados-19.0.0-1739.g1bc250c7.e 664 kB/s | 321 kB 00:00 2024-03-06T07:17:05.804 INFO:teuthology.orchestra.run.smithi163.stdout:(23/176): python3-rbd-19.0.0-1739.g1bc250c7.el9 510 kB/s | 298 kB 00:00 2024-03-06T07:17:06.045 INFO:teuthology.orchestra.run.smithi163.stdout:(24/176): python3-rgw-19.0.0-1739.g1bc250c7.el9 410 kB/s | 99 kB 00:00 2024-03-06T07:17:06.304 INFO:teuthology.orchestra.run.smithi163.stdout:(25/176): rbd-fuse-19.0.0-1739.g1bc250c7.el9.x8 330 kB/s | 85 kB 00:00 2024-03-06T07:17:06.367 INFO:teuthology.orchestra.run.smithi017.stdout:(26/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 571 kB/s | 5.1 MB 00:09 2024-03-06T07:17:06.608 INFO:teuthology.orchestra.run.smithi017.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 707 kB/s | 171 kB 00:00 2024-03-06T07:17:06.709 INFO:teuthology.orchestra.run.smithi017.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 314 kB/s | 31 kB 00:00 2024-03-06T07:17:07.242 INFO:teuthology.orchestra.run.smithi017.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 259 kB/s | 138 kB 00:00 2024-03-06T07:17:07.584 INFO:teuthology.orchestra.run.smithi017.stdout:(30/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 726 kB/s | 3.1 MB 00:04 2024-03-06T07:17:09.568 INFO:teuthology.orchestra.run.smithi017.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 784 kB/s | 1.8 MB 00:02 2024-03-06T07:17:09.637 INFO:teuthology.orchestra.run.smithi163.stdout:(26/176): librgw2-19.0.0-1739.g1bc250c7.el9.x86 808 kB/s | 5.1 MB 00:06 2024-03-06T07:17:09.896 INFO:teuthology.orchestra.run.smithi163.stdout:(27/176): rbd-nbd-19.0.0-1739.g1bc250c7.el9.x86 661 kB/s | 171 kB 00:00 2024-03-06T07:17:09.976 INFO:teuthology.orchestra.run.smithi017.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 609 kB/s | 248 kB 00:00 2024-03-06T07:17:10.004 INFO:teuthology.orchestra.run.smithi163.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1739.g 290 kB/s | 31 kB 00:00 2024-03-06T07:17:10.077 INFO:teuthology.orchestra.run.smithi017.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 494 kB/s | 49 kB 00:00 2024-03-06T07:17:10.177 INFO:teuthology.orchestra.run.smithi017.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 153 kB/s | 15 kB 00:00 2024-03-06T07:17:10.271 INFO:teuthology.orchestra.run.smithi163.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1739.g1bc250c 518 kB/s | 138 kB 00:00 2024-03-06T07:17:10.594 INFO:teuthology.orchestra.run.smithi017.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 633 kB/s | 264 kB 00:00 2024-03-06T07:17:11.561 INFO:teuthology.orchestra.run.smithi017.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 774 kB/s | 748 kB 00:00 2024-03-06T07:17:11.661 INFO:teuthology.orchestra.run.smithi017.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 183 kB/s | 18 kB 00:00 2024-03-06T07:17:11.903 INFO:teuthology.orchestra.run.smithi017.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 662 kB 00:00 2024-03-06T07:17:11.962 INFO:teuthology.orchestra.run.smithi017.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 920 kB 00:00 2024-03-06T07:17:11.995 INFO:teuthology.orchestra.run.smithi017.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-06T07:17:12.029 INFO:teuthology.orchestra.run.smithi017.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 615 kB/s | 20 kB 00:00 2024-03-06T07:17:12.071 INFO:teuthology.orchestra.run.smithi017.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-06T07:17:12.104 INFO:teuthology.orchestra.run.smithi017.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 953 kB/s | 32 kB 00:00 2024-03-06T07:17:12.196 INFO:teuthology.orchestra.run.smithi017.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-06T07:17:12.238 INFO:teuthology.orchestra.run.smithi017.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 412 kB/s | 17 kB 00:00 2024-03-06T07:17:12.272 INFO:teuthology.orchestra.run.smithi017.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 303 kB/s | 10 kB 00:00 2024-03-06T07:17:12.314 INFO:teuthology.orchestra.run.smithi017.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-06T07:17:12.347 INFO:teuthology.orchestra.run.smithi017.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-06T07:17:12.406 INFO:teuthology.orchestra.run.smithi017.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-06T07:17:12.448 INFO:teuthology.orchestra.run.smithi017.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-06T07:17:12.481 INFO:teuthology.orchestra.run.smithi017.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 734 kB/s | 24 kB 00:00 2024-03-06T07:17:12.515 INFO:teuthology.orchestra.run.smithi017.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-06T07:17:12.557 INFO:teuthology.orchestra.run.smithi017.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-06T07:17:12.591 INFO:teuthology.orchestra.run.smithi017.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2024-03-06T07:17:12.616 INFO:teuthology.orchestra.run.smithi017.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 1.3 MB/s | 34 kB 00:00 2024-03-06T07:17:12.741 INFO:teuthology.orchestra.run.smithi017.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-06T07:17:12.850 INFO:teuthology.orchestra.run.smithi017.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-03-06T07:17:12.856 INFO:teuthology.orchestra.run.smithi163.stdout:(30/176): ceph-mgr-dashboard-19.0.0-1739.g1bc25 706 kB/s | 1.8 MB 00:02 2024-03-06T07:17:12.934 INFO:teuthology.orchestra.run.smithi017.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-03-06T07:17:12.963 INFO:teuthology.orchestra.run.smithi163.stdout:(31/176): rbd-mirror-19.0.0-1739.g1bc250c7.el9. 475 kB/s | 3.1 MB 00:06 2024-03-06T07:17:13.001 INFO:teuthology.orchestra.run.smithi017.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-06T07:17:13.034 INFO:teuthology.orchestra.run.smithi017.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.9 MB/s | 134 kB 00:00 2024-03-06T07:17:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-03-06T07:17:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-03-06T07:17:13.318 INFO:teuthology.orchestra.run.smithi017.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-03-06T07:17:13.477 INFO:teuthology.orchestra.run.smithi017.stdout:(64/176): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-06T07:17:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 5.3 MB/s | 227 kB 00:00 2024-03-06T07:17:13.530 INFO:teuthology.orchestra.run.smithi163.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1739.g1b 439 kB/s | 248 kB 00:00 2024-03-06T07:17:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 734 kB/s | 24 kB 00:00 2024-03-06T07:17:13.586 INFO:teuthology.orchestra.run.smithi017.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 545 kB/s | 18 kB 00:00 2024-03-06T07:17:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:(68/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-06T07:17:13.662 INFO:teuthology.orchestra.run.smithi017.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 955 kB/s | 32 kB 00:00 2024-03-06T07:17:13.705 INFO:teuthology.orchestra.run.smithi163.stdout:(33/176): ceph-mgr-rook-19.0.0-1739.g1bc250c7.e 282 kB/s | 49 kB 00:00 2024-03-06T07:17:13.804 INFO:teuthology.orchestra.run.smithi017.stdout:(70/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-06T07:17:13.806 INFO:teuthology.orchestra.run.smithi163.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1739.g1 153 kB/s | 15 kB 00:00 2024-03-06T07:17:13.846 INFO:teuthology.orchestra.run.smithi017.stdout:(71/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.1 MB/s | 217 kB 00:00 2024-03-06T07:17:13.880 INFO:teuthology.orchestra.run.smithi017.stdout:(72/176): python3-oauthlib-3.1.1-5.el9.noarch.r 5.1 MB/s | 174 kB 00:00 2024-03-06T07:17:13.913 INFO:teuthology.orchestra.run.smithi017.stdout:(73/176): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:13.947 INFO:teuthology.orchestra.run.smithi017.stdout:(74/176): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-06T07:17:13.989 INFO:teuthology.orchestra.run.smithi017.stdout:(75/176): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-06T07:17:14.022 INFO:teuthology.orchestra.run.smithi017.stdout:(76/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.9 MB/s | 132 kB 00:00 2024-03-06T07:17:14.055 INFO:teuthology.orchestra.run.smithi017.stdout:(77/176): python3-pyasn1-modules-0.4.8-6.el9.no 6.2 MB/s | 211 kB 00:00 2024-03-06T07:17:14.089 INFO:teuthology.orchestra.run.smithi017.stdout:(78/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-06T07:17:14.123 INFO:teuthology.orchestra.run.smithi017.stdout:(79/176): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-06T07:17:14.489 INFO:teuthology.orchestra.run.smithi163.stdout:(35/176): ceph-volume-19.0.0-1739.g1bc250c7.el9 386 kB/s | 264 kB 00:00 2024-03-06T07:17:14.548 INFO:teuthology.orchestra.run.smithi017.stdout:(80/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-06T07:17:14.591 INFO:teuthology.orchestra.run.smithi017.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 932 kB/s | 39 kB 00:00 2024-03-06T07:17:14.658 INFO:teuthology.orchestra.run.smithi017.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 603 kB/s | 41 kB 00:00 2024-03-06T07:17:14.699 INFO:teuthology.orchestra.run.smithi017.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 5.2 MB/s | 215 kB 00:00 2024-03-06T07:17:14.733 INFO:teuthology.orchestra.run.smithi017.stdout:(84/176): qatlib-service-23.11.0-1.el9.x86_64.r 1.0 MB/s | 34 kB 00:00 2024-03-06T07:17:14.766 INFO:teuthology.orchestra.run.smithi017.stdout:(85/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-03-06T07:17:14.808 INFO:teuthology.orchestra.run.smithi017.stdout:(86/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-06T07:17:14.842 INFO:teuthology.orchestra.run.smithi017.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 1.6 MB/s | 56 kB 00:00 2024-03-06T07:17:14.884 INFO:teuthology.orchestra.run.smithi017.stdout:(88/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-03-06T07:17:14.917 INFO:teuthology.orchestra.run.smithi017.stdout:(89/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-06T07:17:14.951 INFO:teuthology.orchestra.run.smithi017.stdout:(90/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-06T07:17:14.985 INFO:teuthology.orchestra.run.smithi017.stdout:(91/176): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-03-06T07:17:15.018 INFO:teuthology.orchestra.run.smithi017.stdout:(92/176): jq-1.6-16.el9.x86_64.rpm 5.5 MB/s | 186 kB 00:00 2024-03-06T07:17:15.077 INFO:teuthology.orchestra.run.smithi017.stdout:(93/176): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-06T07:17:15.112 INFO:teuthology.orchestra.run.smithi017.stdout:(94/176): libquadmath-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 194 kB 00:00 2024-03-06T07:17:15.161 INFO:teuthology.orchestra.run.smithi017.stdout:(95/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-06T07:17:15.203 INFO:teuthology.orchestra.run.smithi017.stdout:(96/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.7 MB/s | 241 kB 00:00 2024-03-06T07:17:15.270 INFO:teuthology.orchestra.run.smithi017.stdout:(97/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-06T07:17:15.303 INFO:teuthology.orchestra.run.smithi017.stdout:(98/176): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-06T07:17:15.354 INFO:teuthology.orchestra.run.smithi017.stdout:(99/176): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-06T07:17:15.387 INFO:teuthology.orchestra.run.smithi017.stdout:(100/176): python3-requests-2.25.1-8.el9.noarch 3.3 MB/s | 113 kB 00:00 2024-03-06T07:17:15.446 INFO:teuthology.orchestra.run.smithi017.stdout:(101/176): python3-urllib3-1.26.5-5.el9.noarch. 3.1 MB/s | 187 kB 00:00 2024-03-06T07:17:15.596 INFO:teuthology.orchestra.run.smithi017.stdout:(102/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.6 MB/s | 245 kB 00:00 2024-03-06T07:17:15.638 INFO:teuthology.orchestra.run.smithi017.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 536 kB/s | 22 kB 00:00 2024-03-06T07:17:15.747 INFO:teuthology.orchestra.run.smithi017.stdout:(104/176): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-03-06T07:17:16.122 INFO:teuthology.orchestra.run.smithi017.stdout:(105/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-03-06T07:17:16.214 INFO:teuthology.orchestra.run.smithi017.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-06T07:17:16.289 INFO:teuthology.orchestra.run.smithi017.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 260 kB/s | 19 kB 00:00 2024-03-06T07:17:16.481 INFO:teuthology.orchestra.run.smithi163.stdout:(36/176): cephadm-19.0.0-1739.g1bc250c7.el9.noa 376 kB/s | 748 kB 00:01 2024-03-06T07:17:16.556 INFO:teuthology.orchestra.run.smithi017.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-03-06T07:17:16.581 INFO:teuthology.orchestra.run.smithi163.stdout:(37/176): cephfs-top-19.0.0-1739.g1bc250c7.el9. 183 kB/s | 18 kB 00:00 2024-03-06T07:17:16.623 INFO:teuthology.orchestra.run.smithi017.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-03-06T07:17:16.698 INFO:teuthology.orchestra.run.smithi017.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-03-06T07:17:16.723 INFO:teuthology.orchestra.run.smithi163.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-06T07:17:16.774 INFO:teuthology.orchestra.run.smithi017.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-03-06T07:17:16.774 INFO:teuthology.orchestra.run.smithi163.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-06T07:17:16.807 INFO:teuthology.orchestra.run.smithi163.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-06T07:17:16.874 INFO:teuthology.orchestra.run.smithi163.stdout:(41/176): elfutils-libelf-devel-0.190-2.el9.x86 306 kB/s | 20 kB 00:00 2024-03-06T07:17:16.882 INFO:teuthology.orchestra.run.smithi017.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-06T07:17:16.924 INFO:teuthology.orchestra.run.smithi163.stdout:(42/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-06T07:17:16.966 INFO:teuthology.orchestra.run.smithi163.stdout:(43/176): flexiblas-3.0.4-8.el9.x86_64.rpm 760 kB/s | 32 kB 00:00 2024-03-06T07:17:16.982 INFO:teuthology.orchestra.run.smithi017.stdout:(113/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-03-06T07:17:17.075 INFO:teuthology.orchestra.run.smithi163.stdout:(44/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-06T07:17:17.091 INFO:teuthology.orchestra.run.smithi017.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-03-06T07:17:17.109 INFO:teuthology.orchestra.run.smithi163.stdout:(45/176): flexiblas-openblas-openmp-3.0.4-8.el9 518 kB/s | 17 kB 00:00 2024-03-06T07:17:17.142 INFO:teuthology.orchestra.run.smithi163.stdout:(46/176): libaio-devel-0.3.111-13.el9.x86_64.rp 302 kB/s | 10 kB 00:00 2024-03-06T07:17:17.175 INFO:teuthology.orchestra.run.smithi017.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-03-06T07:17:17.184 INFO:teuthology.orchestra.run.smithi163.stdout:(47/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-06T07:17:17.218 INFO:teuthology.orchestra.run.smithi163.stdout:(48/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-06T07:17:17.242 INFO:teuthology.orchestra.run.smithi017.stdout:(116/176): python3-cachetools-4.2.4-1.el9.noarc 483 kB/s | 32 kB 00:00 2024-03-06T07:17:17.276 INFO:teuthology.orchestra.run.smithi163.stdout:(49/176): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-06T07:17:17.317 INFO:teuthology.orchestra.run.smithi017.stdout:(117/176): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-06T07:17:17.326 INFO:teuthology.orchestra.run.smithi163.stdout:(50/176): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-06T07:17:17.393 INFO:teuthology.orchestra.run.smithi163.stdout:(51/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-03-06T07:17:17.469 INFO:teuthology.orchestra.run.smithi163.stdout:(52/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 618 kB/s | 46 kB 00:00 2024-03-06T07:17:17.536 INFO:teuthology.orchestra.run.smithi163.stdout:(53/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-06T07:17:17.600 INFO:teuthology.orchestra.run.smithi017.stdout:(118/176): python3-cheroot-8.6.0-4.el9.noarch.r 606 kB/s | 172 kB 00:00 2024-03-06T07:17:17.684 INFO:teuthology.orchestra.run.smithi017.stdout:(119/176): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-03-06T07:17:17.694 INFO:teuthology.orchestra.run.smithi163.stdout:(54/176): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-06T07:17:17.736 INFO:teuthology.orchestra.run.smithi163.stdout:(55/176): openblas-0.3.21-2.el9.x86_64.rpm 819 kB/s | 34 kB 00:00 2024-03-06T07:17:17.810 INFO:teuthology.orchestra.run.smithi017.stdout:(120/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-03-06T07:17:17.901 INFO:teuthology.orchestra.run.smithi017.stdout:(121/176): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-03-06T07:17:17.928 INFO:teuthology.orchestra.run.smithi163.stdout:(56/176): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-06T07:17:18.052 INFO:teuthology.orchestra.run.smithi017.stdout:(122/176): python3-google-auth-2.28.0-1.el9.noa 1.3 MB/s | 203 kB 00:00 2024-03-06T07:17:18.062 INFO:teuthology.orchestra.run.smithi163.stdout:(57/176): openssl-devel-3.0.7-27.el9.x86_64.rpm 22 MB/s | 3.0 MB 00:00 2024-03-06T07:17:18.146 INFO:teuthology.orchestra.run.smithi163.stdout:(58/176): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-03-06T07:17:18.194 INFO:teuthology.orchestra.run.smithi017.stdout:(123/176): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-03-06T07:17:18.195 INFO:teuthology.orchestra.run.smithi163.stdout:(59/176): postgresql-contrib-13.11-1.el9.x86_64 16 MB/s | 811 kB 00:00 2024-03-06T07:17:18.237 INFO:teuthology.orchestra.run.smithi163.stdout:(60/176): postgresql-private-libs-13.11-1.el9.x 3.1 MB/s | 134 kB 00:00 2024-03-06T07:17:18.277 INFO:teuthology.orchestra.run.smithi017.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-03-06T07:17:18.353 INFO:teuthology.orchestra.run.smithi017.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-06T07:17:18.403 INFO:teuthology.orchestra.run.smithi017.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-06T07:17:18.421 INFO:teuthology.orchestra.run.smithi163.stdout:(61/176): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-03-06T07:17:18.470 INFO:teuthology.orchestra.run.smithi017.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-03-06T07:17:18.471 INFO:teuthology.orchestra.run.smithi163.stdout:(62/176): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-03-06T07:17:18.514 INFO:teuthology.orchestra.run.smithi163.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-03-06T07:17:18.528 INFO:teuthology.orchestra.run.smithi017.stdout:(128/176): python3-jaraco-functools-3.5.0-2.el9 334 kB/s | 19 kB 00:00 2024-03-06T07:17:18.612 INFO:teuthology.orchestra.run.smithi017.stdout:(129/176): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-03-06T07:17:18.672 INFO:teuthology.orchestra.run.smithi163.stdout:(64/176): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-06T07:17:18.704 INFO:teuthology.orchestra.run.smithi017.stdout:(130/176): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-06T07:17:18.714 INFO:teuthology.orchestra.run.smithi163.stdout:(65/176): python3-jinja2-2.11.3-5.el9.noarch.rp 5.3 MB/s | 227 kB 00:00 2024-03-06T07:17:18.739 INFO:teuthology.orchestra.run.smithi163.stdout:(66/176): python3-jsonpatch-1.21-16.el9.noarch. 984 kB/s | 24 kB 00:00 2024-03-06T07:17:18.773 INFO:teuthology.orchestra.run.smithi163.stdout:(67/176): python3-jsonpointer-2.0-4.el9.noarch. 542 kB/s | 18 kB 00:00 2024-03-06T07:17:18.813 INFO:teuthology.orchestra.run.smithi017.stdout:(131/176): ceph-mgr-diskprediction-local-19.0.0 676 kB/s | 7.4 MB 00:11 2024-03-06T07:17:18.815 INFO:teuthology.orchestra.run.smithi163.stdout:(68/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-06T07:17:18.838 INFO:teuthology.orchestra.run.smithi017.stdout:(132/176): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-03-06T07:17:18.848 INFO:teuthology.orchestra.run.smithi163.stdout:(69/176): python3-markupsafe-1.1.1-12.el9.x86_6 951 kB/s | 32 kB 00:00 2024-03-06T07:17:18.922 INFO:teuthology.orchestra.run.smithi017.stdout:(133/176): python3-natsort-7.1.1-5.el9.noarch.r 693 kB/s | 58 kB 00:00 2024-03-06T07:17:18.974 INFO:teuthology.orchestra.run.smithi163.stdout:(70/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-03-06T07:17:18.997 INFO:teuthology.orchestra.run.smithi017.stdout:(134/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-06T07:17:19.008 INFO:teuthology.orchestra.run.smithi163.stdout:(71/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 6.4 MB/s | 217 kB 00:00 2024-03-06T07:17:19.047 INFO:teuthology.orchestra.run.smithi017.stdout:(135/176): python3-more-itertools-8.12.0-2.el9. 337 kB/s | 79 kB 00:00 2024-03-06T07:17:19.049 INFO:teuthology.orchestra.run.smithi163.stdout:(72/176): python3-oauthlib-3.1.1-5.el9.noarch.r 4.1 MB/s | 174 kB 00:00 2024-03-06T07:17:19.073 INFO:teuthology.orchestra.run.smithi017.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-03-06T07:17:19.083 INFO:teuthology.orchestra.run.smithi163.stdout:(73/176): python3-packaging-20.9-5.el9.noarch.r 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:19.117 INFO:teuthology.orchestra.run.smithi163.stdout:(74/176): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-06T07:17:19.131 INFO:teuthology.orchestra.run.smithi017.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 485 kB/s | 28 kB 00:00 2024-03-06T07:17:19.150 INFO:teuthology.orchestra.run.smithi163.stdout:(75/176): python3-protobuf-3.14.0-13.el9.noarch 7.0 MB/s | 238 kB 00:00 2024-03-06T07:17:19.156 INFO:teuthology.orchestra.run.smithi017.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 826 kB/s | 90 kB 00:00 2024-03-06T07:17:19.190 INFO:teuthology.orchestra.run.smithi017.stdout:(139/176): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-03-06T07:17:19.192 INFO:teuthology.orchestra.run.smithi163.stdout:(76/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-03-06T07:17:19.226 INFO:teuthology.orchestra.run.smithi163.stdout:(77/176): python3-pyasn1-modules-0.4.8-6.el9.no 6.1 MB/s | 211 kB 00:00 2024-03-06T07:17:19.257 INFO:teuthology.orchestra.run.smithi017.stdout:(140/176): python3-rsa-4.9-2.el9.noarch.rpm 887 kB/s | 59 kB 00:00 2024-03-06T07:17:19.259 INFO:teuthology.orchestra.run.smithi163.stdout:(78/176): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-06T07:17:19.282 INFO:teuthology.orchestra.run.smithi017.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-03-06T07:17:19.293 INFO:teuthology.orchestra.run.smithi163.stdout:(79/176): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-06T07:17:19.332 INFO:teuthology.orchestra.run.smithi017.stdout:(142/176): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-06T07:17:19.366 INFO:teuthology.orchestra.run.smithi017.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 611 kB/s | 51 kB 00:00 2024-03-06T07:17:19.424 INFO:teuthology.orchestra.run.smithi017.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-06T07:17:19.466 INFO:teuthology.orchestra.run.smithi017.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-03-06T07:17:19.517 INFO:teuthology.orchestra.run.smithi017.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-03-06T07:17:19.550 INFO:teuthology.orchestra.run.smithi017.stdout:(147/176): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-03-06T07:17:19.584 INFO:teuthology.orchestra.run.smithi017.stdout:(148/176): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-03-06T07:17:19.677 INFO:teuthology.orchestra.run.smithi163.stdout:(80/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-06T07:17:19.710 INFO:teuthology.orchestra.run.smithi163.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-06T07:17:19.726 INFO:teuthology.orchestra.run.smithi017.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-03-06T07:17:19.744 INFO:teuthology.orchestra.run.smithi163.stdout:(82/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.2 MB/s | 41 kB 00:00 2024-03-06T07:17:19.794 INFO:teuthology.orchestra.run.smithi163.stdout:(83/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-03-06T07:17:19.828 INFO:teuthology.orchestra.run.smithi163.stdout:(84/176): qatlib-service-23.11.0-1.el9.x86_64.r 1.0 MB/s | 34 kB 00:00 2024-03-06T07:17:19.862 INFO:teuthology.orchestra.run.smithi163.stdout:(85/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-03-06T07:17:19.903 INFO:teuthology.orchestra.run.smithi163.stdout:(86/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-03-06T07:17:19.937 INFO:teuthology.orchestra.run.smithi163.stdout:(87/176): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 56 kB 00:00 2024-03-06T07:17:19.979 INFO:teuthology.orchestra.run.smithi163.stdout:(88/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-03-06T07:17:20.021 INFO:teuthology.orchestra.run.smithi163.stdout:(89/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-06T07:17:20.063 INFO:teuthology.orchestra.run.smithi163.stdout:(90/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-06T07:17:20.096 INFO:teuthology.orchestra.run.smithi163.stdout:(91/176): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-03-06T07:17:20.130 INFO:teuthology.orchestra.run.smithi163.stdout:(92/176): jq-1.6-16.el9.x86_64.rpm 5.4 MB/s | 186 kB 00:00 2024-03-06T07:17:20.180 INFO:teuthology.orchestra.run.smithi163.stdout:(93/176): libgfortran-11.4.1-3.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-06T07:17:20.222 INFO:teuthology.orchestra.run.smithi163.stdout:(94/176): libquadmath-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 194 kB 00:00 2024-03-06T07:17:20.264 INFO:teuthology.orchestra.run.smithi163.stdout:(95/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-03-06T07:17:20.306 INFO:teuthology.orchestra.run.smithi163.stdout:(96/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-03-06T07:17:20.365 INFO:teuthology.orchestra.run.smithi163.stdout:(97/176): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-06T07:17:20.424 INFO:teuthology.orchestra.run.smithi163.stdout:(98/176): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 103 kB 00:00 2024-03-06T07:17:20.466 INFO:teuthology.orchestra.run.smithi163.stdout:(99/176): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-06T07:17:20.507 INFO:teuthology.orchestra.run.smithi163.stdout:(100/176): python3-requests-2.25.1-8.el9.noarch 2.7 MB/s | 113 kB 00:00 2024-03-06T07:17:20.549 INFO:teuthology.orchestra.run.smithi163.stdout:(101/176): python3-urllib3-1.26.5-5.el9.noarch. 4.4 MB/s | 187 kB 00:00 2024-03-06T07:17:20.758 INFO:teuthology.orchestra.run.smithi163.stdout:(102/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.1 MB/s | 245 kB 00:00 2024-03-06T07:17:20.850 INFO:teuthology.orchestra.run.smithi163.stdout:(103/176): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-03-06T07:17:21.000 INFO:teuthology.orchestra.run.smithi163.stdout:(104/176): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-03-06T07:17:21.526 INFO:teuthology.orchestra.run.smithi163.stdout:(105/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-06T07:17:21.626 INFO:teuthology.orchestra.run.smithi163.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-03-06T07:17:21.726 INFO:teuthology.orchestra.run.smithi163.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-03-06T07:17:22.068 INFO:teuthology.orchestra.run.smithi163.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-03-06T07:17:22.168 INFO:teuthology.orchestra.run.smithi163.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-03-06T07:17:22.260 INFO:teuthology.orchestra.run.smithi163.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-03-06T07:17:22.352 INFO:teuthology.orchestra.run.smithi163.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 736 kB/s | 67 kB 00:00 2024-03-06T07:17:22.444 INFO:teuthology.orchestra.run.smithi163.stdout:(112/176): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-06T07:17:22.553 INFO:teuthology.orchestra.run.smithi163.stdout:(113/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-03-06T07:17:22.653 INFO:teuthology.orchestra.run.smithi163.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-03-06T07:17:22.737 INFO:teuthology.orchestra.run.smithi163.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 523 kB/s | 43 kB 00:00 2024-03-06T07:17:22.820 INFO:teuthology.orchestra.run.smithi163.stdout:(116/176): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-03-06T07:17:22.921 INFO:teuthology.orchestra.run.smithi163.stdout:(117/176): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-03-06T07:17:23.021 INFO:teuthology.orchestra.run.smithi163.stdout:(118/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-06T07:17:23.121 INFO:teuthology.orchestra.run.smithi163.stdout:(119/176): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-06T07:17:23.238 INFO:teuthology.orchestra.run.smithi163.stdout:(120/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-03-06T07:17:23.330 INFO:teuthology.orchestra.run.smithi163.stdout:(121/176): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-03-06T07:17:23.422 INFO:teuthology.orchestra.run.smithi163.stdout:(122/176): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-06T07:17:23.451 INFO:teuthology.orchestra.run.smithi017.stdout:(150/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 862 kB/s | 3.1 MB 00:03 2024-03-06T07:17:23.476 INFO:teuthology.orchestra.run.smithi017.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.7 MB/s | 68 kB 00:00 2024-03-06T07:17:23.601 INFO:teuthology.orchestra.run.smithi017.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 18 MB/s | 2.3 MB 00:00 2024-03-06T07:17:23.677 INFO:teuthology.orchestra.run.smithi017.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-06T07:17:23.702 INFO:teuthology.orchestra.run.smithi017.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-06T07:17:23.744 INFO:teuthology.orchestra.run.smithi017.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-06T07:17:23.777 INFO:teuthology.orchestra.run.smithi017.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:23.803 INFO:teuthology.orchestra.run.smithi017.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.4 MB/s | 35 kB 00:00 2024-03-06T07:17:23.861 INFO:teuthology.orchestra.run.smithi017.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-03-06T07:17:23.903 INFO:teuthology.orchestra.run.smithi017.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-03-06T07:17:23.962 INFO:teuthology.orchestra.run.smithi017.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-03-06T07:17:24.021 INFO:teuthology.orchestra.run.smithi017.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 7.3 MB/s | 433 kB 00:00 2024-03-06T07:17:24.046 INFO:teuthology.orchestra.run.smithi017.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 882 kB/s | 22 kB 00:00 2024-03-06T07:17:24.088 INFO:teuthology.orchestra.run.smithi017.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-03-06T07:17:24.098 INFO:teuthology.orchestra.run.smithi163.stdout:(123/176): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-03-06T07:17:24.121 INFO:teuthology.orchestra.run.smithi017.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 154 kB 00:00 2024-03-06T07:17:24.155 INFO:teuthology.orchestra.run.smithi017.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.9 MB/s | 135 kB 00:00 2024-03-06T07:17:24.180 INFO:teuthology.orchestra.run.smithi017.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 2.5 MB/s | 63 kB 00:00 2024-03-06T07:17:24.189 INFO:teuthology.orchestra.run.smithi163.stdout:(124/176): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-06T07:17:24.205 INFO:teuthology.orchestra.run.smithi017.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 28 kB 00:00 2024-03-06T07:17:24.297 INFO:teuthology.orchestra.run.smithi017.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2024-03-06T07:17:24.298 INFO:teuthology.orchestra.run.smithi163.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-06T07:17:24.414 INFO:teuthology.orchestra.run.smithi017.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 18 MB/s | 2.2 MB 00:00 2024-03-06T07:17:24.423 INFO:teuthology.orchestra.run.smithi163.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-03-06T07:17:24.551 INFO:teuthology.orchestra.run.smithi163.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-03-06T07:17:24.606 INFO:teuthology.orchestra.run.smithi017.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-06T07:17:24.649 INFO:teuthology.orchestra.run.smithi163.stdout:(128/176): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-03-06T07:17:24.665 INFO:teuthology.orchestra.run.smithi017.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 7.4 MB/s | 438 kB 00:00 2024-03-06T07:17:24.707 INFO:teuthology.orchestra.run.smithi017.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 720 kB/s | 30 kB 00:00 2024-03-06T07:17:24.766 INFO:teuthology.orchestra.run.smithi163.stdout:(129/176): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-03-06T07:17:24.774 INFO:teuthology.orchestra.run.smithi017.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-06T07:17:24.924 INFO:teuthology.orchestra.run.smithi017.stdout:(174/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2024-03-06T07:17:25.366 INFO:teuthology.orchestra.run.smithi163.stdout:(130/176): python3-kubernetes-26.1.0-2.el9.noar 1.7 MB/s | 1.0 MB 00:00 2024-03-06T07:17:25.491 INFO:teuthology.orchestra.run.smithi163.stdout:(131/176): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-03-06T07:17:25.583 INFO:teuthology.orchestra.run.smithi163.stdout:(132/176): python3-more-itertools-8.12.0-2.el9. 862 kB/s | 79 kB 00:00 2024-03-06T07:17:25.717 INFO:teuthology.orchestra.run.smithi163.stdout:(133/176): python3-natsort-7.1.1-5.el9.noarch.r 433 kB/s | 58 kB 00:00 2024-03-06T07:17:25.825 INFO:teuthology.orchestra.run.smithi163.stdout:(134/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-06T07:17:25.832 INFO:teuthology.orchestra.run.smithi017.stdout:(175/176): librados2-19.0.0-1739.g1bc250c7.el9. 556 kB/s | 3.4 MB 00:06 2024-03-06T07:17:25.917 INFO:teuthology.orchestra.run.smithi163.stdout:(135/176): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-03-06T07:17:26.009 INFO:teuthology.orchestra.run.smithi163.stdout:(136/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 982 kB/s | 90 kB 00:00 2024-03-06T07:17:26.101 INFO:teuthology.orchestra.run.smithi163.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-03-06T07:17:26.193 INFO:teuthology.orchestra.run.smithi163.stdout:(138/176): python3-repoze-lru-0.7-16.el9.noarch 336 kB/s | 31 kB 00:00 2024-03-06T07:17:26.302 INFO:teuthology.orchestra.run.smithi163.stdout:(139/176): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-06T07:17:26.394 INFO:teuthology.orchestra.run.smithi163.stdout:(140/176): python3-rsa-4.9-2.el9.noarch.rpm 646 kB/s | 59 kB 00:00 2024-03-06T07:17:26.486 INFO:teuthology.orchestra.run.smithi163.stdout:(141/176): python3-tempora-5.0.0-2.el9.noarch.r 391 kB/s | 36 kB 00:00 2024-03-06T07:17:26.578 INFO:teuthology.orchestra.run.smithi163.stdout:(142/176): python3-typing-extensions-4.4.0-2.el 557 kB/s | 51 kB 00:00 2024-03-06T07:17:26.670 INFO:teuthology.orchestra.run.smithi163.stdout:(143/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-03-06T07:17:26.762 INFO:teuthology.orchestra.run.smithi163.stdout:(144/176): python3-websocket-client-1.2.3-2.el9 980 kB/s | 90 kB 00:00 2024-03-06T07:17:26.862 INFO:teuthology.orchestra.run.smithi163.stdout:(145/176): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-06T07:17:26.954 INFO:teuthology.orchestra.run.smithi163.stdout:(146/176): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-03-06T07:17:27.054 INFO:teuthology.orchestra.run.smithi163.stdout:(147/176): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-06T07:17:27.180 INFO:teuthology.orchestra.run.smithi163.stdout:(148/176): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-03-06T07:17:28.455 INFO:teuthology.orchestra.run.smithi163.stdout:(149/176): ceph-mgr-diskprediction-local-19.0.0 486 kB/s | 7.4 MB 00:15 2024-03-06T07:17:32.355 INFO:teuthology.orchestra.run.smithi163.stdout:(150/176): librbd1-19.0.0-1739.g1bc250c7.el9.x8 823 kB/s | 3.1 MB 00:03 2024-03-06T07:17:32.388 INFO:teuthology.orchestra.run.smithi163.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-03-06T07:17:32.472 INFO:teuthology.orchestra.run.smithi163.stdout:(152/176): NetworkManager-1.46.0-1.el9.x86_64.r 27 MB/s | 2.3 MB 00:00 2024-03-06T07:17:32.555 INFO:teuthology.orchestra.run.smithi163.stdout:(153/176): NetworkManager-libnm-1.46.0-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-03-06T07:17:32.581 INFO:teuthology.orchestra.run.smithi163.stdout:(154/176): NetworkManager-team-1.46.0-1.el9.x86 1.6 MB/s | 41 kB 00:00 2024-03-06T07:17:32.623 INFO:teuthology.orchestra.run.smithi163.stdout:(155/176): NetworkManager-tui-1.46.0-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-03-06T07:17:32.656 INFO:teuthology.orchestra.run.smithi163.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-03-06T07:17:32.690 INFO:teuthology.orchestra.run.smithi163.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-03-06T07:17:32.732 INFO:teuthology.orchestra.run.smithi163.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-03-06T07:17:32.774 INFO:teuthology.orchestra.run.smithi163.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-03-06T07:17:32.824 INFO:teuthology.orchestra.run.smithi163.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-03-06T07:17:32.866 INFO:teuthology.orchestra.run.smithi163.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 10 MB/s | 433 kB 00:00 2024-03-06T07:17:32.900 INFO:teuthology.orchestra.run.smithi163.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 22 kB 00:00 2024-03-06T07:17:32.933 INFO:teuthology.orchestra.run.smithi163.stdout:(163/176): libblkid-2.37.4-18.el9.x86_64.rpm 3.1 MB/s | 107 kB 00:00 2024-03-06T07:17:32.975 INFO:teuthology.orchestra.run.smithi163.stdout:(164/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 154 kB 00:00 2024-03-06T07:17:33.009 INFO:teuthology.orchestra.run.smithi163.stdout:(165/176): libmount-2.37.4-18.el9.x86_64.rpm 3.9 MB/s | 135 kB 00:00 2024-03-06T07:17:33.042 INFO:teuthology.orchestra.run.smithi163.stdout:(166/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.8 MB/s | 63 kB 00:00 2024-03-06T07:17:33.076 INFO:teuthology.orchestra.run.smithi163.stdout:(167/176): libuuid-2.37.4-18.el9.x86_64.rpm 825 kB/s | 28 kB 00:00 2024-03-06T07:17:33.135 INFO:teuthology.orchestra.run.smithi163.stdout:(168/176): openssl-3.0.7-27.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-03-06T07:17:33.218 INFO:teuthology.orchestra.run.smithi163.stdout:(169/176): openssl-libs-3.0.7-27.el9.x86_64.rpm 26 MB/s | 2.2 MB 00:00 2024-03-06T07:17:33.319 INFO:teuthology.orchestra.run.smithi163.stdout:(170/176): util-linux-2.37.4-18.el9.x86_64.rpm 22 MB/s | 2.2 MB 00:00 2024-03-06T07:17:33.377 INFO:teuthology.orchestra.run.smithi163.stdout:(171/176): util-linux-core-2.37.4-18.el9.x86_64 7.4 MB/s | 438 kB 00:00 2024-03-06T07:17:33.435 INFO:teuthology.orchestra.run.smithi163.stdout:(172/176): util-linux-user-2.37.4-18.el9.x86_64 515 kB/s | 30 kB 00:00 2024-03-06T07:17:33.494 INFO:teuthology.orchestra.run.smithi163.stdout:(173/176): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2024-03-06T07:17:33.578 INFO:teuthology.orchestra.run.smithi163.stdout:(174/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-03-06T07:17:34.637 INFO:teuthology.orchestra.run.smithi163.stdout:(175/176): librados2-19.0.0-1739.g1bc250c7.el9. 466 kB/s | 3.4 MB 00:07 2024-03-06T07:18:03.899 INFO:teuthology.orchestra.run.smithi163.stdout:(176/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 735 kB/s | 47 MB 01:06 2024-03-06T07:18:03.903 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:18:03.903 INFO:teuthology.orchestra.run.smithi163.stdout:Total 2.5 MB/s | 224 MB 01:30 2024-03-06T07:18:03.950 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2024-03-06T07:18:04.259 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2024-03-06T07:18:04.260 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2024-03-06T07:18:04.779 INFO:teuthology.orchestra.run.smithi017.stdout:(176/176): ceph-test-19.0.0-1739.g1bc250c7.el9. 559 kB/s | 47 MB 01:26 2024-03-06T07:18:04.783 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-03-06T07:18:04.783 INFO:teuthology.orchestra.run.smithi017.stdout:Total 2.3 MB/s | 224 MB 01:37 2024-03-06T07:18:04.822 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-03-06T07:18:05.155 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-03-06T07:18:05.155 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-03-06T07:18:06.256 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2024-03-06T07:18:06.256 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2024-03-06T07:18:07.186 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-03-06T07:18:07.187 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-03-06T07:18:13.213 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2024-03-06T07:18:13.311 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-06T07:18:13.346 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-06T07:18:13.392 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:13.419 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:13.471 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-06T07:18:13.510 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-06T07:18:13.563 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-06T07:18:13.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-06T07:18:13.639 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-06T07:18:13.692 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-06T07:18:14.089 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-06T07:18:14.123 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:14.245 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:14.364 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:14.400 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:18:14.454 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-03-06T07:18:14.550 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/202 2024-03-06T07:18:14.554 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-06T07:18:14.585 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-03-06T07:18:14.623 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:14.626 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-06T07:18:14.644 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-03-06T07:18:14.679 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:14.710 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-03-06T07:18:14.758 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-03-06T07:18:14.765 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:14.788 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:14.819 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-03-06T07:18:14.845 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:14.860 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-03-06T07:18:14.884 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-06T07:18:14.885 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-03-06T07:18:14.907 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:14.922 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:14.934 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-03-06T07:18:15.116 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:15.147 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:15.178 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-06T07:18:15.281 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-03-06T07:18:15.336 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:15.389 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-06T07:18:15.460 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:18:15.539 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:15.607 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:15.612 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:18:15.726 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:15.734 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:18:15.784 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:15.898 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-03-06T07:18:15.957 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-03-06T07:18:15.983 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:16.003 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:16.088 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:18:16.108 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-06T07:18:16.117 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:16.181 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 17/202 2024-03-06T07:18:16.200 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-06T07:18:16.229 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-03-06T07:18:16.239 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-06T07:18:16.252 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-03-06T07:18:16.307 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-06T07:18:16.361 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-06T07:18:16.393 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-06T07:18:16.437 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-06T07:18:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:16.477 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-06T07:18:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-03-06T07:18:16.523 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-03-06T07:18:16.541 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-06T07:18:16.576 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-06T07:18:16.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-06T07:18:16.661 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-06T07:18:16.697 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-06T07:18:16.703 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-03-06T07:18:16.766 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-06T07:18:16.802 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-06T07:18:16.840 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:16.845 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-06T07:18:16.874 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-06T07:18:16.900 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/202 2024-03-06T07:18:16.942 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-06T07:18:16.989 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-06T07:18:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:17.063 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-06T07:18:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:17.197 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-06T07:18:17.250 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-06T07:18:17.292 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/202 2024-03-06T07:18:17.402 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-03-06T07:18:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-03-06T07:18:17.657 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-03-06T07:18:17.728 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-03-06T07:18:17.780 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-03-06T07:18:17.781 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-06T07:18:17.810 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-03-06T07:18:17.846 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-03-06T07:18:17.862 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-06T07:18:17.885 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-03-06T07:18:17.957 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-06T07:18:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-03-06T07:18:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-03-06T07:18:18.007 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:18.015 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-03-06T07:18:18.054 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-06T07:18:18.071 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-03-06T07:18:18.092 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-06T07:18:18.098 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-03-06T07:18:18.133 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-06T07:18:18.170 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-03-06T07:18:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 39/202 2024-03-06T07:18:18.229 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-06T07:18:18.264 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 40/202 2024-03-06T07:18:18.284 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-06T07:18:18.292 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 41/202 2024-03-06T07:18:18.365 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-03-06T07:18:18.407 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-03-06T07:18:18.483 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-03-06T07:18:18.557 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-06T07:18:18.749 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-06T07:18:18.755 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-03-06T07:18:18.810 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-03-06T07:18:18.828 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-06T07:18:18.873 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-06T07:18:19.005 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-06T07:18:19.051 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-06T07:18:19.107 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-06T07:18:19.178 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-06T07:18:19.210 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-06T07:18:19.247 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-06T07:18:19.294 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-06T07:18:19.330 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-06T07:18:19.373 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-06T07:18:19.378 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-03-06T07:18:19.424 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-06T07:18:19.457 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-03-06T07:18:19.537 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-03-06T07:18:19.558 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-06T07:18:19.599 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:19.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-06T07:18:19.606 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-03-06T07:18:19.634 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-06T07:18:19.646 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-03-06T07:18:19.676 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-06T07:18:19.684 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 52/202 2024-03-06T07:18:19.716 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-03-06T07:18:19.778 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-06T07:18:19.818 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-06T07:18:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-03-06T07:18:19.858 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-06T07:18:19.884 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-06T07:18:19.900 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-03-06T07:18:19.908 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-06T07:18:19.986 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-06T07:18:20.111 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-06T07:18:20.178 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-03-06T07:18:20.216 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-06T07:18:20.259 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-06T07:18:20.316 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-06T07:18:20.372 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-06T07:18:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-03-06T07:18:20.469 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-03-06T07:18:20.513 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-03-06T07:18:20.557 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-06T07:18:20.605 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-06T07:18:20.652 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-03-06T07:18:20.699 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 61/202 2024-03-06T07:18:20.743 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-06T07:18:20.755 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-03-06T07:18:20.803 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-06T07:18:20.830 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-03-06T07:18:20.836 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-06T07:18:20.866 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-03-06T07:18:20.868 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-06T07:18:20.903 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-03-06T07:18:20.903 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-06T07:18:20.935 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-06T07:18:20.950 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-03-06T07:18:20.962 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-06T07:18:20.986 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-03-06T07:18:20.997 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-06T07:18:21.029 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-03-06T07:18:21.036 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-06T07:18:21.068 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-06T07:18:21.071 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-03-06T07:18:21.106 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-06T07:18:21.183 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-06T07:18:21.212 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-03-06T07:18:21.216 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-06T07:18:21.250 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-06T07:18:21.291 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-06T07:18:21.327 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-03-06T07:18:21.333 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-06T07:18:21.357 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-03-06T07:18:21.399 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-03-06T07:18:21.425 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-06T07:18:21.497 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-03-06T07:18:21.513 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-06T07:18:21.532 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 75/202 2024-03-06T07:18:21.556 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-06T07:18:21.572 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-03-06T07:18:21.596 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-06T07:18:21.597 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-03-06T07:18:21.622 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-03-06T07:18:21.644 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-06T07:18:21.701 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-03-06T07:18:21.751 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-06T07:18:21.807 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-06T07:18:21.827 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-03-06T07:18:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-03-06T07:18:22.181 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-03-06T07:18:22.340 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-03-06T07:18:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-03-06T07:18:22.540 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-06T07:18:22.587 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-06T07:18:22.612 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-03-06T07:18:22.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-06T07:18:22.659 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-03-06T07:18:22.670 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-06T07:18:22.759 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-06T07:18:22.797 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-06T07:18:22.804 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-03-06T07:18:22.850 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-03-06T07:18:22.882 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-03-06T07:18:22.914 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-03-06T07:18:22.949 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-03-06T07:18:22.975 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-06T07:18:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-03-06T07:18:23.004 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-06T07:18:23.007 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-03-06T07:18:23.042 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-03-06T07:18:23.081 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-06T07:18:23.081 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-03-06T07:18:23.113 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-03-06T07:18:23.143 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-03-06T07:18:23.222 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-03-06T07:18:23.253 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-03-06T07:18:23.286 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-03-06T07:18:23.328 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-03-06T07:18:23.362 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-03-06T07:18:23.454 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-03-06T07:18:23.544 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-03-06T07:18:23.595 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-03-06T07:18:23.624 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-03-06T07:18:23.672 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-03-06T07:18:23.728 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-06T07:18:23.773 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-03-06T07:18:23.827 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-03-06T07:18:23.844 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-06T07:18:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-03-06T07:18:24.631 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-03-06T07:18:24.659 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-03-06T07:18:24.708 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-03-06T07:18:24.799 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-03-06T07:18:24.835 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-03-06T07:18:25.021 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-03-06T07:18:25.049 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-03-06T07:18:25.073 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-06T07:18:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-03-06T07:18:25.150 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-06T07:18:25.178 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-06T07:18:25.488 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-06T07:18:25.516 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:25.607 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:25.640 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-06T07:18:25.682 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-06T07:18:25.805 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-03-06T07:18:25.916 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-03-06T07:18:26.136 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-06T07:18:26.161 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:26.232 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:26.254 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-06T07:18:27.202 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-03-06T07:18:27.295 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-03-06T07:18:27.321 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-03-06T07:18:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-03-06T07:18:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:27.767 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 125/202 2024-03-06T07:18:27.798 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 126/202 2024-03-06T07:18:27.834 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-03-06T07:18:28.312 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-03-06T07:18:28.337 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:28.409 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 129/202 2024-03-06T07:18:28.431 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 130/202 2024-03-06T07:18:28.467 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:28.510 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:29.050 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:29.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:29.572 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:29.572 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-06T07:18:29.572 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-06T07:18:29.573 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:29.623 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:30.819 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:31.384 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 131/202 2024-03-06T07:18:31.423 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:31.507 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 132/202 2024-03-06T07:18:31.508 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-06T07:18:31.508 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-06T07:18:31.508 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:31.549 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:41.208 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:41.208 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2024-03-06T07:18:41.208 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /root 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2024-03-06T07:18:41.209 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:41.374 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:41.389 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:41.389 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:41.389 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-06T07:18:41.389 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:42.903 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:42.928 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:42.928 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:42.928 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-06T07:18:42.928 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:42.929 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:42.929 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:43.154 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:43.176 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:43.621 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:43.643 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 133/202 2024-03-06T07:18:43.647 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /root 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2024-03-06T07:18:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:43.731 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-06T07:18:43.812 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:43.826 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 134/202 2024-03-06T07:18:43.826 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:43.826 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-06T07:18:43.826 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:44.174 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:44.221 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:44.253 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-06T07:18:44.321 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-06T07:18:44.353 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:44.580 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:44.612 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:45.383 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 135/202 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-06T07:18:45.408 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:45.572 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:45.604 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:45.642 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 136/202 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-06T07:18:45.664 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:45.705 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:45.868 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-06T07:18:45.898 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:45.921 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:45.921 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:45.921 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-06T07:18:45.921 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:45.922 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:45.922 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:45.966 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:45.976 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:46.028 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-06T07:18:46.126 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:46.147 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 137/202 2024-03-06T07:18:46.147 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:46.147 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-06T07:18:46.148 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:46.148 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-06T07:18:46.148 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:46.245 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-03-06T07:18:46.638 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:46.660 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 139/202 2024-03-06T07:18:46.683 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 140/202 2024-03-06T07:18:46.746 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 141/202 2024-03-06T07:18:46.775 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:46.900 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-06T07:18:46.935 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:46.957 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:46.958 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:46.958 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-06T07:18:46.958 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:46.958 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:46.958 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:47.002 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:47.014 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 142/202 2024-03-06T07:18:47.023 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:47.023 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:47.023 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-06T07:18:47.023 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:47.042 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:47.078 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:47.100 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:47.412 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:47.434 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-06T07:18:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:47.435 INFO:teuthology.orchestra.run.smithi163.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:47.435 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:18:48.056 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 143/202 2024-03-06T07:18:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 144/202 2024-03-06T07:18:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 145/202 2024-03-06T07:18:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 146/202 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-06T07:18:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:48.445 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:48.455 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 147/202 2024-03-06T07:18:48.505 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 148/202 2024-03-06T07:18:49.418 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-03-06T07:18:49.456 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 150/202 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-06T07:18:49.478 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:49.523 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:49.544 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 151/202 2024-03-06T07:18:49.544 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:49.544 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-06T07:18:49.544 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:49.599 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 152/202 2024-03-06T07:18:49.620 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:49.620 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-06T07:18:49.620 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:49.620 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-06T07:18:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:49.953 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:49.974 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:18:49.975 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-06T07:18:49.975 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-06T07:18:49.975 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:49.975 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-06T07:18:49.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:18:51.859 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-06T07:18:51.900 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-06T07:18:51.930 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-06T07:18:51.957 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-06T07:18:52.152 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-06T07:18:52.276 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-06T07:18:52.316 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-06T07:18:52.370 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-06T07:18:52.425 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-06T07:18:52.501 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-06T07:18:52.552 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-06T07:18:52.595 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-06T07:18:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-06T07:18:52.681 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-06T07:18:52.766 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-06T07:18:52.826 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-06T07:18:52.863 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-06T07:18:53.475 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:53.512 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:53.532 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:54.352 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-06T07:18:54.416 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-06T07:18:54.458 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-06T07:18:54.553 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-06T07:18:54.587 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-06T07:18:54.612 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-06T07:18:54.638 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-06T07:18:54.673 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-06T07:18:54.673 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:54.688 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 154/202 2024-03-06T07:18:54.729 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 155/202 2024-03-06T07:18:54.759 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 156/202 2024-03-06T07:18:54.766 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:54.787 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-03-06T07:18:54.793 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-06T07:18:54.824 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-06T07:18:54.824 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:54.900 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:54.927 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-06T07:18:54.969 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-06T07:18:54.990 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 158/202 2024-03-06T07:18:55.002 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-06T07:18:55.035 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-06T07:18:55.070 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-06T07:18:55.104 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-06T07:18:55.106 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-03-06T07:18:55.151 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-06T07:18:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 160/202 2024-03-06T07:18:55.192 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-06T07:18:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-03-06T07:18:55.217 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-06T07:18:55.242 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-06T07:18:55.268 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:18:55.268 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-03-06T07:18:55.302 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-06T07:18:55.308 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:55.308 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:55.346 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-03-06T07:18:55.395 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-03-06T07:18:55.430 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-03-06T07:18:55.461 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 166/202 2024-03-06T07:18:55.491 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 167/202 2024-03-06T07:18:55.496 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:55.497 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:55.568 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 168/202 2024-03-06T07:18:55.588 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:55.620 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:18:55.623 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-03-06T07:18:55.646 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-06T07:18:55.664 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-03-06T07:18:55.671 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-06T07:18:55.697 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-06T07:18:55.763 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:18:56.248 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:56.287 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:56.306 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-03-06T07:18:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 172/202 2024-03-06T07:18:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 173/202 2024-03-06T07:18:57.293 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-03-06T07:18:57.389 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-03-06T07:18:57.421 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-03-06T07:18:57.446 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-03-06T07:18:57.472 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-03-06T07:18:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-03-06T07:18:57.498 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:57.583 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-03-06T07:18:57.611 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-03-06T07:18:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-03-06T07:18:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:57.726 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-03-06T07:18:57.761 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-03-06T07:18:57.794 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-03-06T07:18:57.819 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-03-06T07:18:57.853 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-03-06T07:18:57.887 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-03-06T07:18:57.921 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-03-06T07:18:57.968 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-03-06T07:18:58.000 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-03-06T07:18:58.025 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-03-06T07:18:58.051 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-03-06T07:18:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:18:58.102 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-03-06T07:18:58.108 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:58.108 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:58.288 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-03-06T07:18:58.288 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:58.354 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-03-06T07:18:58.386 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:18:58.413 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-03-06T07:18:58.438 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-03-06T07:18:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-03-06T07:18:58.532 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-06T07:19:00.767 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-06T07:19:00.768 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-06T07:19:00.769 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-06T07:19:00.787 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-06T07:19:00.788 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-06T07:19:00.789 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-06T07:19:00.790 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-06T07:19:00.791 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-06T07:19:00.792 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-06T07:19:00.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-06T07:19:00.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-06T07:19:00.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-06T07:19:00.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-06T07:19:00.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-06T07:19:00.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-06T07:19:00.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-06T07:19:03.155 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 1/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 3/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 4/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1739.g1bc25 5/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 6/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 7/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 8/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 9/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 10/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 11/202 2024-03-06T07:19:03.156 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 12/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 13/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 14/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_ 15/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 16/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 17/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_6 18/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86 19/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 20/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el 21/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9. 22/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_6 23/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 24/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 25/202 2024-03-06T07:19:03.157 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 26/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 27/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 28/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 29/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7. 30/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noa 31/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.n 32/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc 33/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el 34/202 2024-03-06T07:19:03.174 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 35/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.e 36/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 37/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 38/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 39/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-03-06T07:19:03.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 59/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-03-06T07:19:03.176 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-03-06T07:19:03.177 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-03-06T07:19:03.178 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-03-06T07:19:03.179 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-03-06T07:19:03.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-03-06T07:19:03.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-03-06T07:19:03.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 151/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 153/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-03-06T07:19:03.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-03-06T07:19:03.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 189/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 191/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-03-06T07:19:03.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-03-06T07:19:03.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-03-06T07:19:03.436 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-06T07:19:03.436 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:03.436 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2024-03-06T07:19:03.436 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: acl-2.3.1-4.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.437 INFO:teuthology.orchestra.run.smithi163.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.2-8.el9.noarch 2024-03-06T07:19:03.438 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.7.4-5.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.439 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.440 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.4-9.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.6-16.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.441 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.442 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: lua-5.4.4-4.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-06T07:19:03.443 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-13.11-1.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-06T07:19:03.444 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-06T07:19:03.445 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-06T07:19:03.446 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ply-3.11-14.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-06T07:19:03.447 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-06T07:19:03.448 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: re2-1:20211101-3.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-06T07:19:03.449 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-06T07:19:03.450 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-06T07:19:03.450 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:03.450 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2024-03-06T07:19:04.104 DEBUG:teuthology.parallel:result is None 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: acl-2.3.1-4.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-06T07:19:05.858 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-6.7.0-2.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:05.859 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: automake-1.16.2-8.el9.noarch 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: bison-3.7.4-5.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.860 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.861 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-shell-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top-2:19.0.0-1739.g1bc250c7.el9.noarch 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: flex-2.6.4-9.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-06T07:19:05.862 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-16.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.863 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: lua-5.4.4-4.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-06T07:19:05.864 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-13.11-1.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.865 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-06T07:19:05.866 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-06T07:19:05.867 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-06T07:19:05.868 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply-3.11-14.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.869 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-06T07:19:05.870 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:19.0.0-1739.g1bc250c7.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: re2-1:20211101-3.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-06T07:19:05.871 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-06T07:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-06T07:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-06T07:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-06T07:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-06T07:19:05.872 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:05.873 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-03-06T07:19:06.454 DEBUG:teuthology.parallel:result is None 2024-03-06T07:19:06.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:19:06.596 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-06T07:19:06.618 INFO:teuthology.orchestra.run.smithi017.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:06.619 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:06.619 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-06T07:19:06.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-06T07:19:06.758 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-06T07:19:06.783 INFO:teuthology.orchestra.run.smithi163.stdout:19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:06.784 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739.g1bc250c7.el9 2024-03-06T07:19:06.784 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739.g1bc250c7 is installed. 2024-03-06T07:19:06.786 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-06T07:19:06.786 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:06.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-06T07:19:06.817 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:06.817 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-06T07:19:06.857 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-06T07:19:06.857 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:06.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-03-06T07:19:06.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-06T07:19:06.953 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:06.953 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2024-03-06T07:19:06.982 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-06T07:19:07.051 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-06T07:19:07.051 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:07.051 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-06T07:19:07.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-06T07:19:07.147 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:07.147 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-06T07:19:07.176 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-06T07:19:07.245 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-06T07:19:07.245 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:07.245 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-03-06T07:19:07.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-06T07:19:07.339 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:07.339 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2024-03-06T07:19:07.367 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-06T07:19:07.435 INFO:teuthology.run_tasks:Running task ceph... 2024-03-06T07:19:07.520 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-06T07:19:07.521 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2024-03-06T07:19:07.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2024-03-06T07:19:07.550 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-06T07:19:07.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-06T07:19:07.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-06T07:19:07.618 INFO:tasks.ceph:Creating extra log directories... 2024-03-06T07:19:07.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-06T07:19:07.658 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-06T07:19:07.691 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-06T07:19:07.692 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-06T07:19:07.692 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-06T07:19:07.692 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:07.692 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-06T07:19:07.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:07.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:07.883 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:07.883 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-06T07:19:07.898 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:08.017 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:08.049 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-06T07:19:08.050 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-06T07:19:08.050 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049', 'branch': 'wip-lusov-qdb', 'description': 'fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover}', 'email': None, 'first_in_suite': False, 'job_id': '7583049', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'ceph-fuse': {'disabled': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 5072, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'sleep_before_teardown': 0, 'subset': '246/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi017.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcrt7eHGyP45YOFjqvGwfgQruR9VE8zc0jHwe2MqzBnRuLZJfuLrloiAoLCPGAIdrYHw1ZthpXAFtdi5L8irSc=', 'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTmalFYrhoj0FNk44EdEqJPl48O2hb4I1sAh3U3CBL03a7DTQ+SCm7f6Ki9qDFzw+00R9Y1L051xGFBKzR/JZw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', '\\(MDS_INSUFFICIENT_STANDBY\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(MDS_DAMAGE\\)', '\\(FS_DEGRADED\\)'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'cluster': 'ceph'}}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_failover']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-06_07:00:27', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-06T07:19:08.050 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-06T07:19:08.053 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-06T07:19:08.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2024-03-06T07:19:08.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2024-03-06T07:19:08.166 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:08.167 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-03-06T07:19:08.187 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-06T07:19:08.187 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:08.242 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-06 07:18:58.848466331 +0000 2024-03-06T07:19:08.243 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-06 07:15:40.468926529 +0000 2024-03-06T07:19:08.243 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-06 07:15:40.468926529 +0000 2024-03-06T07:19:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-06 07:15:40.468926529 +0000 2024-03-06T07:19:08.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-06T07:19:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-06T07:19:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-06T07:19:08.307 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 8.2593e-05 s, 6.2 MB/s 2024-03-06T07:19:08.308 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-06T07:19:08.363 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-06 07:18:58.849466322 +0000 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-06 07:15:40.799923425 +0000 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-06 07:15:40.799923425 +0000 2024-03-06T07:19:08.418 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-06 07:15:40.799923425 +0000 2024-03-06T07:19:08.418 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-06T07:19:08.484 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-06T07:19:08.484 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-06T07:19:08.484 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 9.5442e-05 s, 5.4 MB/s 2024-03-06T07:19:08.485 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-06T07:19:08.541 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-06 07:18:58.849466322 +0000 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-06 07:15:41.138920245 +0000 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-06 07:15:41.138920245 +0000 2024-03-06T07:19:08.596 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-06 07:15:41.138920245 +0000 2024-03-06T07:19:08.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-06T07:19:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-06T07:19:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-06T07:19:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 8.7685e-05 s, 5.8 MB/s 2024-03-06T07:19:08.662 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-06T07:19:08.718 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-03-06T07:19:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-06 07:18:58.849466322 +0000 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-06 07:15:41.497916878 +0000 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-06 07:15:41.497916878 +0000 2024-03-06T07:19:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-03-06 07:15:41.497916878 +0000 2024-03-06T07:19:08.772 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-06T07:19:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-06T07:19:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-06T07:19:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 7.5265e-05 s, 6.8 MB/s 2024-03-06T07:19:08.837 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-06T07:19:08.892 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:08.892 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:08.893 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2024-03-06T07:19:08.909 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-06T07:19:08.909 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-06 07:18:56.113598628 +0000 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-06 07:15:38.744871482 +0000 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-06 07:15:38.744871482 +0000 2024-03-06T07:19:08.966 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-06 07:15:38.744871482 +0000 2024-03-06T07:19:08.967 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-06T07:19:09.033 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-06T07:19:09.033 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-06T07:19:09.033 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000282244 s, 1.8 MB/s 2024-03-06T07:19:09.035 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-06T07:19:09.092 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-06 07:18:56.114598607 +0000 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-06 07:15:39.083864412 +0000 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-06 07:15:39.083864412 +0000 2024-03-06T07:19:09.149 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-06 07:15:39.083864412 +0000 2024-03-06T07:19:09.150 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-06T07:19:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-06T07:19:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-06T07:19:09.215 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000218347 s, 2.3 MB/s 2024-03-06T07:19:09.217 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-06T07:19:09.273 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-06 07:18:56.114598607 +0000 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-06 07:15:39.415857487 +0000 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-06 07:15:39.415857487 +0000 2024-03-06T07:19:09.330 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-06 07:15:39.415857487 +0000 2024-03-06T07:19:09.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-06T07:19:09.396 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-06T07:19:09.396 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-06T07:19:09.396 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000209415 s, 2.4 MB/s 2024-03-06T07:19:09.397 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-06T07:19:09.453 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2024-03-06T07:19:09.509 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-06T07:19:09.509 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-06T07:19:09.509 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-06T07:19:09.509 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-06T07:19:09.510 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2024-03-06T07:19:09.510 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2024-03-06 07:18:56.115598587 +0000 2024-03-06T07:19:09.510 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2024-03-06 07:15:39.971845891 +0000 2024-03-06T07:19:09.510 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2024-03-06 07:15:39.971845891 +0000 2024-03-06T07:19:09.510 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: 2024-03-06 07:15:39.971845891 +0000 2024-03-06T07:19:09.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-06T07:19:09.575 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2024-03-06T07:19:09.576 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2024-03-06T07:19:09.576 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000169129 s, 3.0 MB/s 2024-03-06T07:19:09.577 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-06T07:19:09.632 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:09.632 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-06T07:19:09.632 INFO:tasks.ceph:Generating config... 2024-03-06T07:19:09.633 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-06T07:19:09.633 INFO:tasks.ceph:[client] debug client = 20 2024-03-06T07:19:09.634 INFO:tasks.ceph:[client] debug ms = 1 2024-03-06T07:19:09.634 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-06T07:19:09.634 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-06T07:19:09.634 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-06T07:19:09.634 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-06T07:19:09.635 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-06T07:19:09.635 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-06T07:19:09.635 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-06T07:19:09.635 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-06T07:19:09.636 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-06T07:19:09.636 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-06T07:19:09.636 INFO:tasks.ceph:Setting up mon.a... 2024-03-06T07:19:09.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:09.695 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2024-03-06T07:19:09.698 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-06T07:19:09.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:09.771 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.b', '172.21.15.163'), ('mon.c', '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]')] 2024-03-06T07:19:09.772 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-06T07:19:09.772 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:09.773 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-06T07:19:09.827 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --add b 172.21.15.163 --addv c '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:09.913 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:09.913 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid 5fd8a41a-f84b-4e64-ac3e-5a25cd35578f 2024-03-06T07:19:09.913 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-03-06T07:19:09.913 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 5fd8a41a-f84b-4e64-ac3e-5a25cd35578f 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2024-03-06T07:19:09.912562+0000 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-03-06T07:19:09.912562+0000 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 17 (quincy) 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] mon.c 2024-03-06T07:19:09.914 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-06T07:19:09.915 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-06T07:19:09.930 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5fd8a41a-f84b-4e64-ac3e-5a25cd35578f... 2024-03-06T07:19:09.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-06T07:19:09.973 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-06T07:19:10.021 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2024-03-06T07:19:10.022 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2024-03-06T07:19:10.023 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2024-03-06T07:19:10.024 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on skipped message = False 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = 5fd8a41a-f84b-4e64-ac3e-5a25cd35578f 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock profile = high_recovery_ops 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2024-03-06T07:19:10.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug trim objects = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2024-03-06T07:19:10.026 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2024-03-06T07:19:10.027 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:10.028 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-06T07:19:10.029 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: mon op complaint time = 120 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:10.030 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 600 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 900 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 900 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2024-03-06T07:19:10.031 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds = 20 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds balancer = 20 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug frag = True 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug scatterstat = True 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: mds op complaint time = 180 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: mds verify scatter = True 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2024-03-06T07:19:10.032 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 900 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 900 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2024-03-06T07:19:10.033 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.034 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.035 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.17,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = 5fd8a41a-f84b-4e64-ac3e-5a25cd35578f 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2024-03-06T07:19:10.036 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug trim objects = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2024-03-06T07:19:10.037 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore compression mode = aggressive 2024-03-06T07:19:10.038 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 20 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 20 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 10 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-06T07:19:10.039 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2024-03-06T07:19:10.040 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.041 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2024-03-06T07:19:10.042 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds balancer = 20 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2024-03-06T07:19:10.043 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2024-03-06T07:19:10.044 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2024-03-06T07:19:10.045 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-06T07:19:10.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-06T07:19:10.141 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-06T07:19:10.141 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:10.141 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-06T07:19:10.155 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:10.155 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-06T07:19:10.211 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:19:10.211 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:10.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-06T07:19:10.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:10.289 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:10.289 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:10.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:19:10.343 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:10.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-06T07:19:10.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-06T07:19:10.383 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:10.383 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:10.438 INFO:tasks.ceph:Setting up mon nodes... 2024-03-06T07:19:10.439 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-06T07:19:10.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-06T07:19:10.493 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-06T07:19:10.495 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-06T07:19:10.557 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-06T07:19:10.559 INFO:tasks.ceph:Setting up mds nodes... 2024-03-06T07:19:10.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-06T07:19:10.613 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-06T07:19:10.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-06T07:19:10.642 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-06T07:19:10.736 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-06T07:19:10.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-06T07:19:10.765 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-06T07:19:10.820 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-06T07:19:10.822 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-06T07:19:10.849 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-06T07:19:10.943 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-06T07:19:10.945 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-06T07:19:10.972 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-06T07:19:10.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-06T07:19:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-06T07:19:11.030 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-06T07:19:11.072 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-06T07:19:11.087 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-06T07:19:11.087 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-06T07:19:11.088 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:11.116 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:11.116 INFO:tasks.ceph:role: osd.0 2024-03-06T07:19:11.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:19:11.116 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-06T07:19:11.182 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:11.183 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:11.215 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-06T07:19:11.217 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-06T07:19:11.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:11.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:11.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:11.427 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:11.428 INFO:tasks.ceph:role: osd.1 2024-03-06T07:19:11.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:19:11.428 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:11.493 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:11.494 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:11.494 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:11.494 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:11.528 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-06T07:19:11.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-06T07:19:11.528 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:11.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:11.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:11.736 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:11.736 INFO:tasks.ceph:role: osd.2 2024-03-06T07:19:11.737 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:19:11.737 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-06T07:19:11.802 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:11.802 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:11.802 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:11.802 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:11.802 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:11.803 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:11.803 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:11.803 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:11.803 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:11.803 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:11.834 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-06T07:19:11.836 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-06T07:19:11.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:11.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:11.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:12.041 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:12.041 INFO:tasks.ceph:role: osd.3 2024-03-06T07:19:12.042 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:19:12.042 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-06T07:19:12.107 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:12.108 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:12.140 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-06T07:19:12.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-06T07:19:12.142 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:12.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:12.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-06T07:19:12.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:12.385+0000 7f5d289bd640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-06T07:19:15.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-06T07:19:15.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:15.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-06T07:19:15.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-06T07:19:15.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:15.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:15.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-06T07:19:15.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:15.363+0000 7f371a3e1640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-06T07:19:18.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-06T07:19:18.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.329+0000 7f7c45211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-06T07:19:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.330+0000 7f7c45211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-06T07:19:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.330+0000 7f7c45211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.330+0000 7f7c45211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.330+0000 7f7c45211640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-06T07:19:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:18.330+0000 7f7c45211640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-06T07:19:21.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-06T07:19:21.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.313+0000 7f852260c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-06T07:19:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.313+0000 7f852260c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-06T07:19:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.313+0000 7f852260c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:21.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.313+0000 7f852260c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.313+0000 7f852260c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-06T07:19:21.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:21.314+0000 7f852260c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-06T07:19:24.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-06T07:19:24.204 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:24.235 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:24.236 INFO:tasks.ceph:role: osd.4 2024-03-06T07:19:24.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:19:24.236 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:24.313 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:24.314 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:24.314 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:24.824 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-03-06T07:19:24.825 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-03-06T07:19:24.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:24.943 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:24.979 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:25.046 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:25.047 INFO:tasks.ceph:role: osd.5 2024-03-06T07:19:25.047 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:19:25.047 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-06T07:19:25.122 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:25.122 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:25.122 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:25.122 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:25.122 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:25.123 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:25.123 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:25.123 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:25.123 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:25.123 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:25.576 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-03-06T07:19:25.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-03-06T07:19:25.579 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:25.669 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:25.700 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:25.767 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:25.767 INFO:tasks.ceph:role: osd.6 2024-03-06T07:19:25.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:19:25.767 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-06T07:19:25.840 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:25.841 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:26.332 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-03-06T07:19:26.334 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-03-06T07:19:26.334 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:26.474 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:26.506 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:26.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-06T07:19:26.574 INFO:tasks.ceph:role: osd.7 2024-03-06T07:19:26.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:19:26.574 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-06T07:19:26.647 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-06T07:19:26.648 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-06T07:19:27.173 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2024-03-06T07:19:27.175 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2024-03-06T07:19:27.175 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:27.310 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:27.343 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.439+0000 7f2253211640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.440+0000 7f2253211640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.440+0000 7f2253211640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.440+0000 7f2253211640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.440+0000 7f2253211640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-06T07:19:27.443 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:27.440+0000 7f2253211640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-06T07:19:30.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-06T07:19:30.327 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.415+0000 7f9874266640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.415+0000 7f9874266640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.415+0000 7f9874266640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.415+0000 7f9874266640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.415+0000 7f9874266640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-06T07:19:30.418 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:30.416+0000 7f9874266640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-06T07:19:33.264 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-06T07:19:33.294 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:33.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.382+0000 7ff999020640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-06T07:19:33.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.382+0000 7ff999020640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-06T07:19:33.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.382+0000 7ff999020640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:33.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.382+0000 7ff999020640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:33.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.382+0000 7ff999020640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-06T07:19:33.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:33.383+0000 7ff999020640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-06T07:19:36.230 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-06T07:19:36.258 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-06T07:19:36.345 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:36.343+0000 7fd4ff611640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-06T07:19:38.051 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:38.054 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:19:39.196 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-06T07:19:39.224 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-06T07:19:39.224 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-06T07:19:39.256 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-06T07:19:39.328 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.328 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-06T07:19:39.400 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.400 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-06T07:19:39.470 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-06T07:19:39.541 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.541 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-06T07:19:39.611 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:39.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-06T07:19:39.682 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.682 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-06T07:19:39.710 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.710 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-06T07:19:39.778 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.779 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-06T07:19:39.846 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.846 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-06T07:19:39.914 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-06T07:19:39.980 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:39.980 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-06T07:19:40.048 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:40.048 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-06T07:19:40.114 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:40.114 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-06T07:19:40.132 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:40.132 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-06T07:19:40.169 INFO:tasks.ceph:Adding keys to all mons... 2024-03-06T07:19:40.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-06T07:19:40.175 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-06T07:19:40.213 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.y] 2024-03-06T07:19:40.213 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ== 2024-03-06T07:19:40.214 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-03-06T07:19:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A== 2024-03-06T07:19:40.214 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2024-03-06T07:19:40.214 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ== 2024-03-06T07:19:40.214 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g== 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA== 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw== 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA== 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2024-03-06T07:19:40.215 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og== 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w== 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d] 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA== 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ== 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2024-03-06T07:19:40.216 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ== 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.6] 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q== 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.7] 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww== 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout:[client.1] 2024-03-06T07:19:40.217 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw== 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ== 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A== 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ== 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g== 2024-03-06T07:19:40.235 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ== 2024-03-06T07:19:40.236 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ== 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q== 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww== 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2024-03-06T07:19:40.237 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw== 2024-03-06T07:19:40.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:40.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:40.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.332 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.390 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.452 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.508 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.562 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.615 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.666 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:40.669 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-06T07:19:40.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.723 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.781 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-06T07:19:40.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.838 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.945 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:40.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:41.000 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-06T07:19:41.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:41.054 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:41.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:41.111 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-06T07:19:41.167 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-06T07:19:41.167 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-06T07:19:41.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:41.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-06T07:19:41.316 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-06T07:19:41.344 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:41.441 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-06T07:19:41.468 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-06T07:19:41.535 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-06T07:19:41.632 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-06T07:19:41.659 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:41.661 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-06T07:19:41.713 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-06T07:19:41.714 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-06T07:19:41.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-06T07:19:41.717 INFO:tasks.ceph.mon.a:Started 2024-03-06T07:19:41.717 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-06T07:19:41.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-06T07:19:41.756 INFO:tasks.ceph.mon.b:Started 2024-03-06T07:19:41.757 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-06T07:19:41.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-06T07:19:41.759 INFO:tasks.ceph.mon.c:Started 2024-03-06T07:19:41.759 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-06T07:19:41.760 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-06T07:19:41.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-06T07:19:41.763 INFO:tasks.ceph.mgr.y:Started 2024-03-06T07:19:41.763 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-06T07:19:41.763 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-06T07:19:41.766 INFO:tasks.ceph.mgr.x:Started 2024-03-06T07:19:41.766 INFO:tasks.ceph:Setting crush tunables to default 2024-03-06T07:19:41.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2024-03-06T07:19:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.867+0000 7fce889b5640 1 Processor -- start 2024-03-06T07:19:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.867+0000 7fce889b5640 1 -- start start 2024-03-06T07:19:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce889b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce889b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce889b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840c7760 con 0x7fce84152920 2024-03-06T07:19:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce889b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce841587b0 con 0x7fce840c8a00 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce889b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce840c7a40 con 0x7fce84159460 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38106/0 (socket says 172.21.15.17:38106) 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 learned_addr learned my addr 172.21.15.17:0/3439240593 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce8259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce84152920 0x7fce84155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35874/0 (socket says 172.21.15.17:35874) 2024-03-06T07:19:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:41.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:41.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:41.868+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:42.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-06T07:19:42.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-06T07:19:42.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:42.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:42.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.069+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-06T07:19:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:42.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:42.470+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-06T07:19:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.271+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.272+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:43.272+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-06T07:19:44.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 msgr2=0x7fce840c5330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:19:44.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce84159460 0x7fce840c5330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 msgr2=0x7fce84158f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.17:3300/0 conn(0x7fce840c8a00 0x7fce84158f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 -- 172.21.15.17:0/3439240593 >> v1:172.21.15.17:6789/0 conn(0x7fce84152920 legacy=0x7fce84155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 -- 172.21.15.17:0/3439240593 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840c7d20 con 0x7fce640036c0 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce7bfff640 1 -- 172.21.15.17:0/3439240593 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce841587b0 con 0x7fce64007350 2024-03-06T07:19:44.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:44.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:44.869+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-06T07:19:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.070+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.070+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.070+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.070+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.070+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-06T07:19:45.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.471+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:45.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.471+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:45.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.471+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:45.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.471+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:45.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:45.471+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-06T07:19:46.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:46.273+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:46.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:46.273+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-06T07:19:46.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:46.273+0000 7fce81d9b640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-06T07:19:46.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:46.273+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-06T07:19:46.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:46.273+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-06T07:19:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.094+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791373342 0 0) 0x7fce840c7d20 con 0x7fce640036c0 2024-03-06T07:19:47.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.094+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce70003520 con 0x7fce640036c0 2024-03-06T07:19:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.099+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690561305 0 0) 0x7fce70003520 con 0x7fce640036c0 2024-03-06T07:19:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.099+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 msgr2=0x7fce640097f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:19:47.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.099+0000 7fce8159a640 1 --2- 172.21.15.17:0/3439240593 >> v2:172.21.15.163:3300/0 conn(0x7fce64007350 0x7fce640097f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.099+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6400af00 con 0x7fce640036c0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fce740016f0 con 0x7fce640036c0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 >> v1:172.21.15.163:6789/0 conn(0x7fce640036c0 legacy=0x7fce64005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 reap_dead start 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce70004800 0x7fce70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 0x7fce7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce841587b0 con 0x7fce700053c0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7000c280 con 0x7fce70003a00 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7000c480 con 0x7fce70004800 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 0x7fce7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce81d9b640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce70004800 0x7fce70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce8259c640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce70004800 msgr2=0x7fce70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce70004800 0x7fce70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 msgr2=0x7fce70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.100+0000 7fce82d9d640 1 -- 172.21.15.17:0/3439240593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70003520 con 0x7fce700053c0 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.101+0000 7fce82d9d640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 0x7fce7000b9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fce64009e50 tx=0x7fce70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5963c345e555b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.102+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c03a070 con 0x7fce700053c0 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.102+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c035410 con 0x7fce700053c0 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.102+0000 7fce8159a640 1 -- 172.21.15.17:0/3439240593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c003dd0 con 0x7fce700053c0 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 -- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 msgr2=0x7fce7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 0x7fce7000b9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fce64009e50 tx=0x7fce70005830 comp rx=0 tx=0).stop 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 -- 172.21.15.17:0/3439240593 shutdown_connections 2024-03-06T07:19:47.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce700053c0 0x7fce7000b9c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce70004800 0x7fce70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 --2- 172.21.15.17:0/3439240593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce70003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 -- 172.21.15.17:0/3439240593 >> 172.21.15.17:0/3439240593 conn(0x7fce840ac560 msgr2=0x7fce840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 -- 172.21.15.17:0/3439240593 shutdown_connections 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.103+0000 7fce889b5640 1 -- 172.21.15.17:0/3439240593 wait complete. 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 Processor -- start 2024-03-06T07:19:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 -- start start 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce700053c0 0x7fce8414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840c7390 con 0x7fce70004800 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8406d030 con 0x7fce70003a00 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.105+0000 7fce889b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce840c7510 con 0x7fce700053c0 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce8259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce8259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50888/0 (socket says 172.21.15.17:50888) 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38194/0 (socket says 172.21.15.17:38194) 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce8259c640 1 -- 172.21.15.17:0/2770934636 learned_addr learned my addr 172.21.15.17:0/2770934636 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce82d9d640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce700053c0 0x7fce8414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 -- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce700053c0 msgr2=0x7fce8414ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce700053c0 0x7fce8414ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 -- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 msgr2=0x7fce8414ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 -- 172.21.15.17:0/2770934636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce841ea580 con 0x7fce70004800 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce8259c640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:47.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.106+0000 7fce81d9b640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0169f0 tx=0x7fce6c016df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c027020 con 0x7fce70004800 2024-03-06T07:19:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c015cf0 con 0x7fce70004800 2024-03-06T07:19:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00f040 con 0x7fce70004800 2024-03-06T07:19:47.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce840c1a80 con 0x7fce70004800 2024-03-06T07:19:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce841ea800 con 0x7fce70004800 2024-03-06T07:19:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fce6c02a020 con 0x7fce70004800 2024-03-06T07:19:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.108+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fce6c023bf0 con 0x7fce70004800 2024-03-06T07:19:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.109+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8404f330 con 0x7fce70004800 2024-03-06T07:19:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.112+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fce6c031030 con 0x7fce70004800 2024-03-06T07:19:47.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.181+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fce8406cda0 con 0x7fce70004800 2024-03-06T07:19:47.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.183+0000 7fce7affd640 1 -- 172.21.15.17:0/2770934636 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fce841ea580 con 0x7fce70004800 2024-03-06T07:19:47.185 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 msgr2=0x7fce8414d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0169f0 tx=0x7fce6c016df0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 shutdown_connections 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce700053c0 0x7fce8414ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce70004800 0x7fce8414d370 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 --2- 172.21.15.17:0/2770934636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce70003a00 0x7fce8414ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 >> 172.21.15.17:0/2770934636 conn(0x7fce840ac560 msgr2=0x7fce84151e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 shutdown_connections 2024-03-06T07:19:47.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:47.184+0000 7fce889b5640 1 -- 172.21.15.17:0/2770934636 wait complete. 2024-03-06T07:19:47.197 INFO:tasks.ceph:check_enable_crimson: False 2024-03-06T07:19:47.198 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-06T07:19:47.198 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:47.198 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-06T07:19:47.228 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:47.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-06T07:19:47.295 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:47.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-06T07:19:47.362 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:19:47.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-06T07:19:47.429 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:47.429 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-06T07:19:47.457 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:47.457 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-06T07:19:47.525 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:47.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-06T07:19:47.561 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:47.559+0000 7fe5ceafa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-06T07:19:47.586 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:47.584+0000 7f1a9be2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-06T07:19:47.588 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:19:47.588 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-06T07:19:47.618 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:47.616+0000 7fe5ceafa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-06T07:19:47.641 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:47.640+0000 7f1a9be2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-06T07:19:47.644 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new bdfaea27-aff4-4c67-9a67-1b61c249c50e 0 2024-03-06T07:19:47.785 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.783+0000 7f5596864640 1 Processor -- start 2024-03-06T07:19:47.786 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.783+0000 7f5596864640 1 -- start start 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f5596864640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55901527a0 0x7f55900c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f5596864640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f55900c5350 0x7f559006d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f5596864640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55900c6500 con 0x7f55900c48c0 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f5596864640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55900c3dc0 con 0x7f55900c5350 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f5596864640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5590155450 con 0x7f55901527a0 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f558ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55901527a0 0x7f55900c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f558ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55901527a0 0x7f55900c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51162/0 (socket says 172.21.15.163:51162) 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f558ffff640 1 -- 172.21.15.163:0/3044031369 learned_addr learned my addr 172.21.15.163:0/3044031369 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.784+0000 7f558f7fe640 1 --1- 172.21.15.163:0/3044031369 >> v1:172.21.15.163:6789/0 conn(0x7f55900c48c0 0x7f55901558e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40352/0 (socket says 172.21.15.163:40352) 2024-03-06T07:19:47.789 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f5594e01640 1 --2- 172.21.15.163:0/3044031369 >> v2:172.21.15.17:3300/0 conn(0x7f55900c5350 0x7f559006d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558ffff640 1 -- 172.21.15.163:0/3044031369 >> v2:172.21.15.17:3300/0 conn(0x7f55900c5350 msgr2=0x7f559006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558ffff640 1 --2- 172.21.15.163:0/3044031369 >> v2:172.21.15.17:3300/0 conn(0x7f55900c5350 0x7f559006d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558ffff640 1 -- 172.21.15.163:0/3044031369 >> v1:172.21.15.163:6789/0 conn(0x7f55900c48c0 legacy=0x7f55901558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558ffff640 1 -- 172.21.15.163:0/3044031369 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55900c5c60 con 0x7f55901527a0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558ffff640 1 --2- 172.21.15.163:0/3044031369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55901527a0 0x7f55900c4e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f55900c7560 tx=0x7f558402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65ea585a611f158 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.785+0000 7f558effd640 1 -- 172.21.15.163:0/3044031369 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584035020 con 0x7f55901527a0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f558effd640 1 -- 172.21.15.163:0/3044031369 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584031870 con 0x7f55901527a0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f558effd640 1 -- 172.21.15.163:0/3044031369 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584031c40 con 0x7f55901527a0 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 -- 172.21.15.163:0/3044031369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55901527a0 msgr2=0x7f55900c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 --2- 172.21.15.163:0/3044031369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55901527a0 0x7f55900c4e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f55900c7560 tx=0x7f558402b910 comp rx=0 tx=0).stop 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 -- 172.21.15.163:0/3044031369 shutdown_connections 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 --2- 172.21.15.163:0/3044031369 >> v2:172.21.15.17:3300/0 conn(0x7f55900c5350 0x7f559006d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 --2- 172.21.15.163:0/3044031369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55901527a0 0x7f55900c4e10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 -- 172.21.15.163:0/3044031369 >> 172.21.15.163:0/3044031369 conn(0x7f55900c6710 msgr2=0x7f55900c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.790 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 -- 172.21.15.163:0/3044031369 shutdown_connections 2024-03-06T07:19:47.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.786+0000 7f5596864640 1 -- 172.21.15.163:0/3044031369 wait complete. 2024-03-06T07:19:47.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.787+0000 7f5596864640 1 Processor -- start 2024-03-06T07:19:47.791 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.787+0000 7f5596864640 1 -- start start 2024-03-06T07:19:47.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 0x7f55901eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.793 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55901527a0 0x7f55901eba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5590152b80 con 0x7f55901527a0 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55900c6220 con 0x7f55900c48c0 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5596864640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55900c58d0 con 0x7f55900c5350 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 0x7f55901eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51164/0 (socket says 172.21.15.163:51164) 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 -- 172.21.15.163:0/1790620046 learned_addr learned my addr 172.21.15.163:0/1790620046 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:47.794 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f5594e01640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55901527a0 0x7f55901eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 -- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 msgr2=0x7f55901eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 0x7f55901eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 -- 172.21.15.163:0/1790620046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55901527a0 msgr2=0x7f55901eba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55901527a0 0x7f55901eba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558ffff640 1 -- 172.21.15.163:0/1790620046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559014a7c0 con 0x7f55900c48c0 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.788+0000 7f558f7fe640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 0x7f55901eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558ffff640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f558402b950 tx=0x7f55840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.795 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584040070 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584030d00 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584035040 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559014aaa0 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5590072a80 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f558404b060 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.789+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f558404d680 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.790+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5590069460 con 0x7f55900c48c0 2024-03-06T07:19:47.796 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.794+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5590069460 con 0x7f55900c48c0 2024-03-06T07:19:47.868 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.866+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bdfaea27-aff4-4c67-9a67-1b61c249c50e", "id": 0} v 0) v1 -- 0x7f559004f330 con 0x7f55900c48c0 2024-03-06T07:19:47.871 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.869+0000 7f558cff9640 1 -- 172.21.15.163:0/1790620046 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bdfaea27-aff4-4c67-9a67-1b61c249c50e", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f558403ab60 con 0x7f55900c48c0 2024-03-06T07:19:47.871 INFO:teuthology.orchestra.run.smithi163.stdout:0 2024-03-06T07:19:47.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 msgr2=0x7f5590072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f558402b950 tx=0x7f55840046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 shutdown_connections 2024-03-06T07:19:47.873 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55901527a0 0x7f55901eba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55900c5350 0x7f55901eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 --2- 172.21.15.163:0/1790620046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55900c48c0 0x7f5590072540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 >> 172.21.15.163:0/1790620046 conn(0x7f55900c6710 msgr2=0x7f55900bfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 shutdown_connections 2024-03-06T07:19:47.874 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.870+0000 7f5596864640 1 -- 172.21.15.163:0/1790620046 wait complete. 2024-03-06T07:19:47.882 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 685f40a3-4f8f-4cf6-97a2-824b13a8c8af 1 2024-03-06T07:19:47.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.981+0000 7fef5fc42640 1 Processor -- start 2024-03-06T07:19:47.983 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5fc42640 1 -- start start 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5fc42640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fef58070bb0 0x7fef5814f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5fc42640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58151f70 con 0x7fef580acb80 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5fc42640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58152e60 con 0x7fef5806c240 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5fc42640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5806ee00 con 0x7fef58070bb0 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5d9de640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fef5806c240 0x7fef58152750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40368/0 (socket says 172.21.15.163:40368) 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5d9de640 1 -- 172.21.15.163:0/2954153752 learned_addr learned my addr 172.21.15.163:0/2954153752 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.982+0000 7fef5e1df640 1 --2- 172.21.15.163:0/2954153752 >> v2:172.21.15.17:3300/0 conn(0x7fef58070bb0 0x7fef5814f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694630577 0 0) 0x7fef58152e60 con 0x7fef5806c240 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef44003520 con 0x7fef5806c240 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385240856 0 0) 0x7fef58151f70 con 0x7fef580acb80 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58152e60 con 0x7fef580acb80 2024-03-06T07:19:47.985 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5e1df640 1 -- 172.21.15.163:0/2954153752 >> v1:172.21.15.163:6789/0 conn(0x7fef5806c240 legacy=0x7fef58152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5e1df640 1 -- 172.21.15.163:0/2954153752 >> v1:172.21.15.17:6789/0 conn(0x7fef580acb80 legacy=0x7fef5814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.986 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.983+0000 7fef5e1df640 1 -- 172.21.15.163:0/2954153752 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5806f0e0 con 0x7fef58070bb0 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5e1df640 1 --2- 172.21.15.163:0/2954153752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef58070bb0 0x7fef5814f810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fef50009d90 tx=0x7fef5002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b532b961d5898362 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5003b070 con 0x7fef58070bb0 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef50036440 con 0x7fef58070bb0 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5c9dc640 1 -- 172.21.15.163:0/2954153752 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef50036810 con 0x7fef58070bb0 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 -- 172.21.15.163:0/2954153752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef58070bb0 msgr2=0x7fef5814f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 --2- 172.21.15.163:0/2954153752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef58070bb0 0x7fef5814f810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fef50009d90 tx=0x7fef5002e0c0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 -- 172.21.15.163:0/2954153752 shutdown_connections 2024-03-06T07:19:47.987 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 --2- 172.21.15.163:0/2954153752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef58070bb0 0x7fef5814f810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 -- 172.21.15.163:0/2954153752 >> 172.21.15.163:0/2954153752 conn(0x7fef580c3b40 msgr2=0x7fef580c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:47.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 -- 172.21.15.163:0/2954153752 shutdown_connections 2024-03-06T07:19:47.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.984+0000 7fef5fc42640 1 -- 172.21.15.163:0/2954153752 wait complete. 2024-03-06T07:19:47.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.985+0000 7fef5fc42640 1 Processor -- start 2024-03-06T07:19:47.988 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 -- start start 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 0x7fef58154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 0x7fef581e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5806c620 con 0x7fef5806c240 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58152230 con 0x7fef580acb80 2024-03-06T07:19:47.990 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5fc42640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5806c7a0 con 0x7fef58070bb0 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5e1df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5d1dd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 0x7fef581e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5e1df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51180/0 (socket says 172.21.15.163:51180) 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5e1df640 1 -- 172.21.15.163:0/1082205342 learned_addr learned my addr 172.21.15.163:0/1082205342 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.986+0000 7fef5d9de640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 0x7fef58154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 -- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 msgr2=0x7fef581e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 0x7fef581e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 -- 172.21.15.163:0/1082205342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 msgr2=0x7fef58154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 0x7fef58154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 -- 172.21.15.163:0/1082205342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580c14f0 con 0x7fef580acb80 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5d1dd640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 0x7fef581e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5d9de640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 0x7fef58154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5e1df640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef50009eb0 tx=0x7fef50004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5003b040 con 0x7fef580acb80 2024-03-06T07:19:47.991 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef50044070 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef580abf70 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5003f680 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef58154ac0 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fef5003f820 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.987+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fef500483c0 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.988+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5804f330 con 0x7fef580acb80 2024-03-06T07:19:47.992 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:47.990+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fef50049460 con 0x7fef580acb80 2024-03-06T07:19:48.063 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.061+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "685f40a3-4f8f-4cf6-97a2-824b13a8c8af", "id": 1} v 0) v1 -- 0x7fef58072040 con 0x7fef580acb80 2024-03-06T07:19:48.066 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.064+0000 7fef4e7fc640 1 -- 172.21.15.163:0/1082205342 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "685f40a3-4f8f-4cf6-97a2-824b13a8c8af", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef5003fc50 con 0x7fef580acb80 2024-03-06T07:19:48.066 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 msgr2=0x7fef581e9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef50009eb0 tx=0x7fef50004a40 comp rx=0 tx=0).stop 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 shutdown_connections 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef580acb80 0x7fef581e9bf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef58070bb0 0x7fef581e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 --2- 172.21.15.163:0/1082205342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef5806c240 0x7fef58154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 >> 172.21.15.163:0/1082205342 conn(0x7fef580c3b40 msgr2=0x7fef5806dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 shutdown_connections 2024-03-06T07:19:48.068 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.065+0000 7fef5fc42640 1 -- 172.21.15.163:0/1082205342 wait complete. 2024-03-06T07:19:48.078 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 19c31cbd-e62a-420a-8e92-af8cb68e07ea 2 2024-03-06T07:19:48.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.178+0000 7fb362356640 1 Processor -- start 2024-03-06T07:19:48.180 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.178+0000 7fb362356640 1 -- start start 2024-03-06T07:19:48.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb362356640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb35c152860 0x7fb35c155a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb362356640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c0c7060 con 0x7fb35c06c340 2024-03-06T07:19:48.181 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb362356640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c0c7340 con 0x7fb35c0a6cc0 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb362356640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c0c2b10 con 0x7fb35c152860 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35b7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb35c0a6cc0 0x7fb35c155370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40380/0 (socket says 172.21.15.163:40380) 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35b7fe640 1 -- 172.21.15.163:0/2633705347 learned_addr learned my addr 172.21.15.163:0/2633705347 (peer_addr_for_me v1:172.21.15.163:0/0) 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb3608f3640 1 --2- 172.21.15.163:0/2633705347 >> v2:172.21.15.17:3300/0 conn(0x7fb35c152860 0x7fb35c155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159124122 0 0) 0x7fb35c0c7340 con 0x7fb35c0a6cc0 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb348003520 con 0x7fb35c0a6cc0 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690848702 0 0) 0x7fb35c0c7060 con 0x7fb35c06c340 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb35c0c7340 con 0x7fb35c06c340 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb3608f3640 1 -- 172.21.15.163:0/2633705347 >> v1:172.21.15.163:6789/0 conn(0x7fb35c0a6cc0 legacy=0x7fb35c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb3608f3640 1 -- 172.21.15.163:0/2633705347 >> v1:172.21.15.17:6789/0 conn(0x7fb35c06c340 legacy=0x7fb35c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.179+0000 7fb3608f3640 1 -- 172.21.15.163:0/2633705347 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c0c27a0 con 0x7fb35c152860 2024-03-06T07:19:48.182 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.180+0000 7fb3608f3640 1 --2- 172.21.15.163:0/2633705347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c155a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c2c90 tx=0x7fb35402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ff72519b591927 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.180+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354039070 con 0x7fb35c152860 2024-03-06T07:19:48.183 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.180+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35402cbc0 con 0x7fb35c152860 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.180+0000 7fb35affd640 1 -- 172.21.15.163:0/2633705347 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354034440 con 0x7fb35c152860 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 -- 172.21.15.163:0/2633705347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 msgr2=0x7fb35c155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 --2- 172.21.15.163:0/2633705347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c155a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb35c0c2c90 tx=0x7fb35402be10 comp rx=0 tx=0).stop 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 -- 172.21.15.163:0/2633705347 shutdown_connections 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 --2- 172.21.15.163:0/2633705347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c155a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 -- 172.21.15.163:0/2633705347 >> 172.21.15.163:0/2633705347 conn(0x7fb35c0a9540 msgr2=0x7fb35c0a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 -- 172.21.15.163:0/2633705347 shutdown_connections 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.181+0000 7fb362356640 1 -- 172.21.15.163:0/2633705347 wait complete. 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.182+0000 7fb362356640 1 Processor -- start 2024-03-06T07:19:48.184 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.182+0000 7fb362356640 1 -- start start 2024-03-06T07:19:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 0x7fb35c072880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 0x7fb35c144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c155fc0 con 0x7fb35c152860 2024-03-06T07:19:48.186 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0c2440 con 0x7fb35c0a6cc0 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c14f710 con 0x7fb35c06c340 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 0x7fb35c144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 0x7fb35c072880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 0x7fb35c144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51192/0 (socket says 172.21.15.163:51192) 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35b7fe640 1 -- 172.21.15.163:0/42736701 learned_addr learned my addr 172.21.15.163:0/42736701 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb3608f3640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 -- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 msgr2=0x7fb35c144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 0x7fb35c144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 -- 172.21.15.163:0/42736701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 msgr2=0x7fb35c145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 -- 172.21.15.163:0/42736701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c151c30 con 0x7fb35c06c340 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb3608f3640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb35bfff640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 0x7fb35c072880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c3c0 tx=0x7fb34400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.187 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344017070 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3440052f0 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344003830 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c151f10 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.183+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c072dc0 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.184+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb34401a070 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.184+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb34401c440 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.184+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c04f330 con 0x7fb35c06c340 2024-03-06T07:19:48.188 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.184+0000 7f1a9be2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-06T07:19:48.189 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.187+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb344021030 con 0x7fb35c06c340 2024-03-06T07:19:48.195 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.193+0000 7fe5ceafa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-06T07:19:48.241 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.240+0000 7f1a9be2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-06T07:19:48.249 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.248+0000 7fe5ceafa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-06T07:19:48.260 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.258+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "19c31cbd-e62a-420a-8e92-af8cb68e07ea", "id": 2} v 0) v1 -- 0x7fb35c14e9f0 con 0x7fb35c06c340 2024-03-06T07:19:48.263 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.261+0000 7fb358ff9640 1 -- 172.21.15.163:0/42736701 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19c31cbd-e62a-420a-8e92-af8cb68e07ea", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb344011670 con 0x7fb35c06c340 2024-03-06T07:19:48.263 INFO:teuthology.orchestra.run.smithi163.stdout:2 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 msgr2=0x7fb35c072880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 0x7fb35c072880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb34400c3c0 tx=0x7fb34400c880 comp rx=0 tx=0).stop 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 shutdown_connections 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb35c152860 0x7fb35c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb35c0a6cc0 0x7fb35c144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 --2- 172.21.15.163:0/42736701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb35c06c340 0x7fb35c072880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 >> 172.21.15.163:0/42736701 conn(0x7fb35c0a9540 msgr2=0x7fb35c0a8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 shutdown_connections 2024-03-06T07:19:48.265 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.262+0000 7fb362356640 1 -- 172.21.15.163:0/42736701 wait complete. 2024-03-06T07:19:48.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 476f7afe-1b29-400e-af39-a61efe750209 3 2024-03-06T07:19:48.288 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.286+0000 7f1a9be2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-06T07:19:48.298 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.296+0000 7fe5ceafa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-06T07:19:48.343 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.341+0000 7f1a9be2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-06T07:19:48.355 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.353+0000 7fe5ceafa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-06T07:19:48.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.375+0000 7f9bbd3bc640 1 Processor -- start 2024-03-06T07:19:48.377 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.375+0000 7f9bbd3bc640 1 -- start start 2024-03-06T07:19:48.379 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bbd3bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.379 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bbd3bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bb80c6d50 0x7f9bb8151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.379 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bbd3bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb81554f0 con 0x7f9bb8152420 2024-03-06T07:19:48.379 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bbd3bc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb8155b00 con 0x7f9bb80c6d50 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bbd3bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80c71e0 con 0x7f9bb80c39c0 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bb6ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bb80c6d50 0x7f9bb8151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bb80c6d50 0x7f9bb8151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51204/0 (socket says 172.21.15.163:51204) 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bb6ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34112/0 (socket says 172.21.15.163:34112) 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 -- 172.21.15.163:0/409744891 learned_addr learned my addr 172.21.15.163:0/409744891 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 -- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 msgr2=0x7f9bb8155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 -- 172.21.15.163:0/409744891 >> v1:172.21.15.17:6789/0 conn(0x7f9bb8152420 legacy=0x7f9bb806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9baffff640 1 -- 172.21.15.163:0/409744891 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8155790 con 0x7f9bb80c6d50 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.376+0000 7f9bb6ffd640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.377+0000 7f9baffff640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80c6d50 0x7f9bb8151ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8155c80 tx=0x7f9ba002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f9e831aa10c2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.380 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.377+0000 7f9bb67fc640 1 -- 172.21.15.163:0/409744891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0035020 con 0x7f9bb80c6d50 2024-03-06T07:19:48.381 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.377+0000 7f9bb67fc640 1 -- 172.21.15.163:0/409744891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba0031640 con 0x7f9bb80c6d50 2024-03-06T07:19:48.381 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.377+0000 7f9bb67fc640 1 -- 172.21.15.163:0/409744891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba002d040 con 0x7f9bb80c6d50 2024-03-06T07:19:48.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80c6d50 msgr2=0x7f9bb8151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.384 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80c6d50 0x7f9bb8151ee0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8155c80 tx=0x7f9ba002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/409744891 shutdown_connections 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80c6d50 0x7f9bb8151ee0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/409744891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb80c39c0 0x7f9bb8155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/409744891 >> 172.21.15.163:0/409744891 conn(0x7f9bb80a94d0 msgr2=0x7f9bb80a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/409744891 shutdown_connections 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.378+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/409744891 wait complete. 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.379+0000 7f9bbd3bc640 1 Processor -- start 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.379+0000 7f9bbd3bc640 1 -- start start 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 0x7f9bb81ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.385 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 0x7f9bb81dbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb806c960 con 0x7f9bb80c39c0 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb806c7e0 con 0x7f9bb80714b0 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bbd3bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb80a6fe0 con 0x7f9bb8152420 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9baffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 0x7f9bb81dbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9baffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34126/0 (socket says 172.21.15.163:34126) 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9baffff640 1 -- 172.21.15.163:0/350209654 learned_addr learned my addr 172.21.15.163:0/350209654 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb6ffd640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 0x7f9bb81ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 -- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 msgr2=0x7f9bb80abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.386 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.387 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 -- 172.21.15.163:0/350209654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 msgr2=0x7f9bb81ddd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 0x7f9bb81ddd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 -- 172.21.15.163:0/350209654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8151290 con 0x7f9bb80714b0 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9baffff640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb6ffd640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 0x7f9bb81ddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.380+0000 7f9bb77fe640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 0x7f9bb81dbe70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8152820 tx=0x7f9bb000cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.388 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0003b70 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb0003d10 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb8150fb0 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0005310 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb814ed00 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9bb00054b0 con 0x7f9bb80714b0 2024-03-06T07:19:48.389 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.381+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9bb00128b0 con 0x7f9bb80714b0 2024-03-06T07:19:48.390 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.382+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb804f330 con 0x7f9bb80714b0 2024-03-06T07:19:48.391 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.385+0000 7f1a9be2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-06T07:19:48.391 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.384+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9bb0012420 con 0x7f9bb80714b0 2024-03-06T07:19:48.400 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.399+0000 7fe5ceafa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-06T07:19:48.458 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.457+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "476f7afe-1b29-400e-af39-a61efe750209", "id": 3} v 0) v1 -- 0x7f9bb80c43e0 con 0x7f9bb80714b0 2024-03-06T07:19:48.461 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.460+0000 7f9baf7fe640 1 -- 172.21.15.163:0/350209654 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "476f7afe-1b29-400e-af39-a61efe750209", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bb00126c0 con 0x7f9bb80714b0 2024-03-06T07:19:48.461 INFO:teuthology.orchestra.run.smithi163.stdout:3 2024-03-06T07:19:48.462 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.460+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 msgr2=0x7f9bb81dbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.460+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 0x7f9bb81dbe70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8152820 tx=0x7f9bb000cb20 comp rx=0 tx=0).stop 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.460+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 shutdown_connections 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.460+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bb80714b0 0x7f9bb81dbe70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.461+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bb8152420 0x7f9bb80abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.461+0000 7f9bbd3bc640 1 --2- 172.21.15.163:0/350209654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9bb80c39c0 0x7f9bb81ddd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.461+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 >> 172.21.15.163:0/350209654 conn(0x7f9bb80a94d0 msgr2=0x7f9bb80c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.461+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 shutdown_connections 2024-03-06T07:19:48.463 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.461+0000 7f9bbd3bc640 1 -- 172.21.15.163:0/350209654 wait complete. 2024-03-06T07:19:48.473 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 04f85f0d-7f27-43d7-a6c1-54d2b957e248 4 2024-03-06T07:19:48.506 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.505+0000 7f1a9be2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-06T07:19:48.522 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.520+0000 7fe5ceafa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-06T07:19:48.578 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.577+0000 7f1a9be2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:48.596 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.594+0000 7fe5ceafa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:48.614 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.613+0000 7f718f5bc640 1 Processor -- start 2024-03-06T07:19:48.615 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.613+0000 7f718f5bc640 1 -- start start 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.613+0000 7f718f5bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 0x7f718806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718f5bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718f5bc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7188155200 con 0x7f7188152740 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718f5bc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7188155530 con 0x7f71880c1010 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718f5bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71881560d0 con 0x7f718806c310 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718cb57640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 0x7f718806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718cb57640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 0x7f718806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51216/0 (socket says 172.21.15.163:51216) 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34128/0 (socket says 172.21.15.163:34128) 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718cb57640 1 -- 172.21.15.163:0/3579243787 learned_addr learned my addr 172.21.15.163:0/3579243787 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718d358640 1 --1- 172.21.15.163:0/3579243787 >> v1:172.21.15.163:6789/0 conn(0x7f7188152740 0x7f71881559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40394/0 (socket says 172.21.15.163:40394) 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 -- 172.21.15.163:0/3579243787 >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 msgr2=0x7f718806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 --2- 172.21.15.163:0/3579243787 >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 0x7f718806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.617 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 -- 172.21.15.163:0/3579243787 >> v1:172.21.15.163:6789/0 conn(0x7f7188152740 legacy=0x7f71881559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.614+0000 7f718db59640 1 -- 172.21.15.163:0/3579243787 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188152b20 con 0x7f718806c310 2024-03-06T07:19:48.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f718db59640 1 --2- 172.21.15.163:0/3579243787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71881556b0 tx=0x7f718002be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3319e2cb065eed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f717ffff640 1 -- 172.21.15.163:0/3579243787 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180039070 con 0x7f718806c310 2024-03-06T07:19:48.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f717ffff640 1 -- 172.21.15.163:0/3579243787 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718002cbb0 con 0x7f718806c310 2024-03-06T07:19:48.618 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f717ffff640 1 -- 172.21.15.163:0/3579243787 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180034440 con 0x7f718806c310 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f718f5bc640 1 -- 172.21.15.163:0/3579243787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 msgr2=0x7f71880c03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.615+0000 7f718f5bc640 1 --2- 172.21.15.163:0/3579243787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71881556b0 tx=0x7f718002be00 comp rx=0 tx=0).stop 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 -- 172.21.15.163:0/3579243787 shutdown_connections 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 --2- 172.21.15.163:0/3579243787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f71880c03e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 --2- 172.21.15.163:0/3579243787 >> v2:172.21.15.163:3300/0 conn(0x7f71880c1010 0x7f718806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 -- 172.21.15.163:0/3579243787 >> 172.21.15.163:0/3579243787 conn(0x7f71880c3ab0 msgr2=0x7f71880c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 -- 172.21.15.163:0/3579243787 shutdown_connections 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.616+0000 7f718f5bc640 1 -- 172.21.15.163:0/3579243787 wait complete. 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 Processor -- start 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 -- start start 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f718814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71880c1010 0x7f718814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.621 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880ad0a0 con 0x7f71880c1010 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7188072e90 con 0x7f7188152740 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718f5bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c0920 con 0x7f718806c310 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.617+0000 7f718db59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f718814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718db59640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51222/0 (socket says 172.21.15.163:51222) 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718db59640 1 -- 172.21.15.163:0/2914294117 learned_addr learned my addr 172.21.15.163:0/2914294117 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718cb57640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71880c1010 0x7f718814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 -- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 msgr2=0x7f71881eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 -- 172.21.15.163:0/2914294117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71880c1010 msgr2=0x7f718814c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71880c1010 0x7f718814c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 -- 172.21.15.163:0/2914294117 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71881eeb10 con 0x7f718806c310 2024-03-06T07:19:48.622 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718db59640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718d358640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f718814a520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f717800c390 tx=0x7f717800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780074e0 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71780047e0 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7188148080 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178004bd0 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f717801a020 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.618+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718814aa60 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.619+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f71780124e0 con 0x7f718806c310 2024-03-06T07:19:48.623 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.619+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71881ef010 con 0x7f718806c310 2024-03-06T07:19:48.624 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.621+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f71881ef010 con 0x7f718806c310 2024-03-06T07:19:48.626 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.624+0000 7f1a9be2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-06T07:19:48.646 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.644+0000 7fe5ceafa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-06T07:19:48.680 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.678+0000 7f1a9be2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-06T07:19:48.695 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.694+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "04f85f0d-7f27-43d7-a6c1-54d2b957e248", "id": 4} v 0) v1 -- 0x7f71881eab90 con 0x7f718806c310 2024-03-06T07:19:48.698 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.697+0000 7f717dffb640 1 -- 172.21.15.163:0/2914294117 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04f85f0d-7f27-43d7-a6c1-54d2b957e248", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7178021070 con 0x7f718806c310 2024-03-06T07:19:48.699 INFO:teuthology.orchestra.run.smithi163.stdout:4 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 msgr2=0x7f718814a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f718814a520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f717800c390 tx=0x7f717800c790 comp rx=0 tx=0).stop 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 shutdown_connections 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7188152740 0x7f71881eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71880c1010 0x7f718814c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 --2- 172.21.15.163:0/2914294117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f718806c310 0x7f718814a520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 >> 172.21.15.163:0/2914294117 conn(0x7f71880c3ab0 msgr2=0x7f71880c3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 shutdown_connections 2024-03-06T07:19:48.700 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.698+0000 7f718f5bc640 1 -- 172.21.15.163:0/2914294117 wait complete. 2024-03-06T07:19:48.701 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.698+0000 7fe5ceafa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-06T07:19:48.710 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new dd5c8061-c103-41cf-9c03-5446a836d0af 5 2024-03-06T07:19:48.729 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:48.727+0000 7f1a9be2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-06T07:19:48.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:48.748+0000 7fe5ceafa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-06T07:19:48.850 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.848+0000 7f340fdb5640 1 Processor -- start 2024-03-06T07:19:48.850 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.848+0000 7f340fdb5640 1 -- start start 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340fdb5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3408070ba0 0x7f34080ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340fdb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340fdb5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34081563b0 con 0x7f340814f420 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340fdb5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340806c7c0 con 0x7f3408070ba0 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340fdb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34081560d0 con 0x7f3408155b80 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34134/0 (socket says 172.21.15.163:34134) 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 -- 172.21.15.163:0/847640233 learned_addr learned my addr 172.21.15.163:0/847640233 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340d350640 1 --2- 172.21.15.163:0/847640233 >> v2:172.21.15.17:3300/0 conn(0x7f3408070ba0 0x7f34080ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 -- 172.21.15.163:0/847640233 >> v2:172.21.15.17:3300/0 conn(0x7f3408070ba0 msgr2=0x7f34080ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:48.852 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 --2- 172.21.15.163:0/847640233 >> v2:172.21.15.17:3300/0 conn(0x7f3408070ba0 0x7f34080ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 -- 172.21.15.163:0/847640233 >> v1:172.21.15.17:6789/0 conn(0x7f340814f420 legacy=0x7f3408155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.849+0000 7f340e352640 1 -- 172.21.15.163:0/847640233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340806c200 con 0x7f3408155b80 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.850+0000 7f340e352640 1 --2- 172.21.15.163:0/847640233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f340000d150 tx=0x7f340002e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527ab8c224060947 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.850+0000 7f340cb4f640 1 -- 172.21.15.163:0/847640233 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340002e9e0 con 0x7f3408155b80 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.850+0000 7f340cb4f640 1 -- 172.21.15.163:0/847640233 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340002eb80 con 0x7f3408155b80 2024-03-06T07:19:48.853 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.850+0000 7f340cb4f640 1 -- 172.21.15.163:0/847640233 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400009600 con 0x7f3408155b80 2024-03-06T07:19:48.860 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 -- 172.21.15.163:0/847640233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 msgr2=0x7f34080c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 --2- 172.21.15.163:0/847640233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f340000d150 tx=0x7f340002e040 comp rx=0 tx=0).stop 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 -- 172.21.15.163:0/847640233 shutdown_connections 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 --2- 172.21.15.163:0/847640233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3408155b80 0x7f34080c26a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 --2- 172.21.15.163:0/847640233 >> v2:172.21.15.17:3300/0 conn(0x7f3408070ba0 0x7f34080ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 -- 172.21.15.163:0/847640233 >> 172.21.15.163:0/847640233 conn(0x7f34080c3ab0 msgr2=0x7f34080c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 -- 172.21.15.163:0/847640233 shutdown_connections 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.851+0000 7f340fdb5640 1 -- 172.21.15.163:0/847640233 wait complete. 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 Processor -- start 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 -- start start 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 0x7f340806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3408155b80 0x7f34081e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340814f980 con 0x7f3408155b80 2024-03-06T07:19:48.861 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340814f800 con 0x7f3408070ba0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.852+0000 7f340fdb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340814fb00 con 0x7f340814f420 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340d350640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 0x7f340806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340d350640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34148/0 (socket says 172.21.15.163:34148) 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340d350640 1 -- 172.21.15.163:0/3857044093 learned_addr learned my addr 172.21.15.163:0/3857044093 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340e352640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3408155b80 0x7f34081e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 -- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 msgr2=0x7f34081e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 -- 172.21.15.163:0/3857044093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3408155b80 msgr2=0x7f34081e7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3408155b80 0x7f34081e7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 -- 172.21.15.163:0/3857044093 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080c6a30 con 0x7f3408070ba0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340d350640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340db51640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 0x7f340806dde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3408156000 tx=0x7f33f800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:48.862 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8017070 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f80052d0 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34080c6d10 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8003850 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.853+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34080c15e0 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.854+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f33f801a070 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.854+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f33f8007610 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.854+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340804f330 con 0x7f3408070ba0 2024-03-06T07:19:48.863 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.856+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f33f80067d0 con 0x7f3408070ba0 2024-03-06T07:19:48.930 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.928+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd5c8061-c103-41cf-9c03-5446a836d0af", "id": 5} v 0) v1 -- 0x7f340806b4a0 con 0x7f3408070ba0 2024-03-06T07:19:48.933 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.931+0000 7f33fe7fc640 1 -- 172.21.15.163:0/3857044093 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd5c8061-c103-41cf-9c03-5446a836d0af", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f33f80069b0 con 0x7f3408070ba0 2024-03-06T07:19:48.933 INFO:teuthology.orchestra.run.smithi163.stdout:5 2024-03-06T07:19:48.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 msgr2=0x7f340806dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:48.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 0x7f340806dde0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3408156000 tx=0x7f33f800c820 comp rx=0 tx=0).stop 2024-03-06T07:19:48.934 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 shutdown_connections 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3408155b80 0x7f34081e7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f340814f420 0x7f34081e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 --2- 172.21.15.163:0/3857044093 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3408070ba0 0x7f340806dde0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 >> 172.21.15.163:0/3857044093 conn(0x7f34080c3ab0 msgr2=0x7f34080c3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 shutdown_connections 2024-03-06T07:19:48.935 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:48.932+0000 7f340fdb5640 1 -- 172.21.15.163:0/3857044093 wait complete. 2024-03-06T07:19:48.944 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 46156205-4bb9-4cdf-bd12-9aa85d03da13 6 2024-03-06T07:19:49.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.044+0000 7fefe0dd7640 1 Processor -- start 2024-03-06T07:19:49.046 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.044+0000 7fefe0dd7640 1 -- start start 2024-03-06T07:19:49.047 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefe0dd7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefdc0c1010 0x7fefdc06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefe0dd7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0ad150 con 0x7fefdc14f1c0 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefe0dd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc155ee0 con 0x7fefdc072960 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefe0dd7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc071f30 con 0x7fefdc0c1010 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefdc0c1010 0x7fefdc06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefdc0c1010 0x7fefdc06c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51232/0 (socket says 172.21.15.163:51232) 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 -- 172.21.15.163:0/2326728625 learned_addr learned my addr 172.21.15.163:0/2326728625 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 -- 172.21.15.163:0/2326728625 >> v1:172.21.15.163:6789/0 conn(0x7fefdc072960 legacy=0x7fefdc0c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 -- 172.21.15.163:0/2326728625 >> v1:172.21.15.17:6789/0 conn(0x7fefdc14f1c0 legacy=0x7fefdc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.045+0000 7fefd1d9b640 1 -- 172.21.15.163:0/2326728625 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc072160 con 0x7fefdc0c1010 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefd1d9b640 1 --2- 172.21.15.163:0/2326728625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc0c1010 0x7fefdc06c310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefc40098a0 tx=0x7fefc402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93cc13cca721c1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefd9d9b640 1 -- 172.21.15.163:0/2326728625 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc402f840 con 0x7fefdc0c1010 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefd9d9b640 1 -- 172.21.15.163:0/2326728625 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc402f9e0 con 0x7fefdc0c1010 2024-03-06T07:19:49.048 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefd9d9b640 1 -- 172.21.15.163:0/2326728625 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc402fdb0 con 0x7fefdc0c1010 2024-03-06T07:19:49.049 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2326728625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc0c1010 msgr2=0x7fefdc06c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2326728625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc0c1010 0x7fefdc06c310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fefc40098a0 tx=0x7fefc402ba20 comp rx=0 tx=0).stop 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2326728625 shutdown_connections 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2326728625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc0c1010 0x7fefdc06c310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2326728625 >> 172.21.15.163:0/2326728625 conn(0x7fefdc0c3ab0 msgr2=0x7fefdc0c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2326728625 shutdown_connections 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.046+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2326728625 wait complete. 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 Processor -- start 2024-03-06T07:19:49.050 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 -- start start 2024-03-06T07:19:49.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.052 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 0x7fefdc145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 0x7fefdc0c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc14f5a0 con 0x7fefdc0c1010 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc155bf0 con 0x7fefdc14f1c0 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.048+0000 7fefe0dd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc072e10 con 0x7fefdc072960 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefdad9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 0x7fefdc0c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34160/0 (socket says 172.21.15.163:34160) 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefd1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 0x7fefdc145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 -- 172.21.15.163:0/2333704157 learned_addr learned my addr 172.21.15.163:0/2333704157 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 -- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 msgr2=0x7fefdc0c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 0x7fefdc0c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 -- 172.21.15.163:0/2333704157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 msgr2=0x7fefdc145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 0x7fefdc145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.053 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 -- 172.21.15.163:0/2333704157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc146540 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefdad9d640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 0x7fefdc0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefd1d9b640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 0x7fefdc145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefda59c640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c3f0 tx=0x7fefcc00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0052d0 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc005490 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc007260 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc146820 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.049+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc147030 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.050+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fefcc007400 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.050+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fefcc011590 con 0x7fefdc072960 2024-03-06T07:19:49.054 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.050+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc04f330 con 0x7fefdc072960 2024-03-06T07:19:49.055 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.052+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fefcc003a00 con 0x7fefdc072960 2024-03-06T07:19:49.120 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.118+0000 7f1a9be2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-06T07:19:49.127 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.125+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "46156205-4bb9-4cdf-bd12-9aa85d03da13", "id": 6} v 0) v1 -- 0x7fefdc0acf30 con 0x7fefdc072960 2024-03-06T07:19:49.135 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.133+0000 7fefd37fe640 1 -- 172.21.15.163:0/2333704157 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46156205-4bb9-4cdf-bd12-9aa85d03da13", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fefcc003be0 con 0x7fefdc072960 2024-03-06T07:19:49.135 INFO:teuthology.orchestra.run.smithi163.stdout:6 2024-03-06T07:19:49.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 msgr2=0x7fefdc1ec020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c3f0 tx=0x7fefcc00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 shutdown_connections 2024-03-06T07:19:49.136 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc14f1c0 0x7fefdc0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefdc0c1010 0x7fefdc145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 --2- 172.21.15.163:0/2333704157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefdc072960 0x7fefdc1ec020 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 >> 172.21.15.163:0/2333704157 conn(0x7fefdc0c3ab0 msgr2=0x7fefdc0c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 shutdown_connections 2024-03-06T07:19:49.137 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.134+0000 7fefe0dd7640 1 -- 172.21.15.163:0/2333704157 wait complete. 2024-03-06T07:19:49.147 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 05b429af-7fbc-4a72-89c2-b4091bb1cfde 7 2024-03-06T07:19:49.154 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.152+0000 7fe5ceafa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-06T07:19:49.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.246+0000 7fb00d92c640 1 Processor -- start 2024-03-06T07:19:49.248 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.246+0000 7fb00d92c640 1 -- start start 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb008072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 0x7fb0080c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0081569b0 con 0x7fb008072970 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb008156be0 con 0x7fb008150330 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb00d92c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb008156ec0 con 0x7fb008152ea0 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb006ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.250 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb008072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb006ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51246/0 (socket says 172.21.15.163:51246) 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb006ffd640 1 -- 172.21.15.163:0/1102801176 learned_addr learned my addr 172.21.15.163:0/1102801176 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0077fe640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 0x7fb0080c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 -- 172.21.15.163:0/1102801176 >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 msgr2=0x7fb008156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 -- 172.21.15.163:0/1102801176 >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 msgr2=0x7fb0080c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 0x7fb0080c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0067fc640 1 -- 172.21.15.163:0/1102801176 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0081571a0 con 0x7fb008152ea0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb006ffd640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.247+0000 7fb0077fe640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 0x7fb0080c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.248+0000 7fb0067fc640 1 --2- 172.21.15.163:0/1102801176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb008072430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb008150cc0 tx=0x7faff802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3393203bb206bd8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.248+0000 7fb005ffb640 1 -- 172.21.15.163:0/1102801176 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8035020 con 0x7fb008152ea0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.248+0000 7fb005ffb640 1 -- 172.21.15.163:0/1102801176 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8031830 con 0x7fb008152ea0 2024-03-06T07:19:49.251 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.248+0000 7fb005ffb640 1 -- 172.21.15.163:0/1102801176 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031c00 con 0x7fb008152ea0 2024-03-06T07:19:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 -- 172.21.15.163:0/1102801176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 msgr2=0x7fb008072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.254 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 --2- 172.21.15.163:0/1102801176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb008072430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb008150cc0 tx=0x7faff802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 -- 172.21.15.163:0/1102801176 shutdown_connections 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.17:3300/0 conn(0x7fb008072970 0x7fb0080c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 --2- 172.21.15.163:0/1102801176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb008072430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 --2- 172.21.15.163:0/1102801176 >> v2:172.21.15.163:3300/0 conn(0x7fb008150330 0x7fb008156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 -- 172.21.15.163:0/1102801176 >> 172.21.15.163:0/1102801176 conn(0x7fb0080aa5c0 msgr2=0x7fb0080aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 -- 172.21.15.163:0/1102801176 shutdown_connections 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.249+0000 7fb00d92c640 1 -- 172.21.15.163:0/1102801176 wait complete. 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.250+0000 7fb00d92c640 1 Processor -- start 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.250+0000 7fb00d92c640 1 -- start start 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb008150330 0x7fb0081e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008150890 con 0x7fb008150330 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008150710 con 0x7fb008072970 2024-03-06T07:19:49.255 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb00d92c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c3e10 con 0x7fb008152ea0 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb0077fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:34166/0 (socket says 172.21.15.163:34166) 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:51248/0 (socket says 172.21.15.163:51248) 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb0077fe640 1 -- 172.21.15.163:0/853261624 learned_addr learned my addr 172.21.15.163:0/853261624 (peer_addr_for_me v2:172.21.15.163:0/0) 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb0067fc640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb008150330 0x7fb0081e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 -- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 msgr2=0x7fb0081e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 -- 172.21.15.163:0/853261624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb008150330 msgr2=0x7fb0081e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb008150330 0x7fb0081e8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb006ffd640 1 -- 172.21.15.163:0/853261624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00814c410 con 0x7fb008072970 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.251+0000 7fb0077fe640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fb006ffd640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faffc010c90 tx=0x7faffc00b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.256 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc01e070 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00bdb0 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00814b520 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc004230 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080c7bc0 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faffc021070 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.252+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7faffc003d40 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.253+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00804f330 con 0x7fb008072970 2024-03-06T07:19:49.257 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.255+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faffc0093d0 con 0x7fb008072970 2024-03-06T07:19:49.260 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.258+0000 7f1a9be2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-06T07:19:49.297 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.295+0000 7fe5ceafa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-06T07:19:49.329 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.327+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "05b429af-7fbc-4a72-89c2-b4091bb1cfde", "id": 7} v 0) v1 -- 0x7fb008070a60 con 0x7fb008072970 2024-03-06T07:19:49.337 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.335+0000 7fafeffff640 1 -- 172.21.15.163:0/853261624 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05b429af-7fbc-4a72-89c2-b4091bb1cfde", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7faffc003610 con 0x7fb008072970 2024-03-06T07:19:49.337 INFO:teuthology.orchestra.run.smithi163.stdout:7 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 msgr2=0x7fb0080c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faffc010c90 tx=0x7faffc00b510 comp rx=0 tx=0).stop 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 shutdown_connections 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb008152ea0 0x7fb0081e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb008150330 0x7fb0081e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 --2- 172.21.15.163:0/853261624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb008072970 0x7fb0080c7680 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 >> 172.21.15.163:0/853261624 conn(0x7fb0080aa5c0 msgr2=0x7fb00814f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 shutdown_connections 2024-03-06T07:19:49.339 INFO:teuthology.orchestra.run.smithi163.stderr:2024-03-06T07:19:49.336+0000 7fb00d92c640 1 -- 172.21.15.163:0/853261624 wait complete. 2024-03-06T07:19:49.348 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-06T07:19:49.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-06T07:19:49.351 INFO:tasks.ceph.osd.0:Started 2024-03-06T07:19:49.351 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-06T07:19:49.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-06T07:19:49.353 INFO:tasks.ceph.osd.1:Started 2024-03-06T07:19:49.353 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-06T07:19:49.353 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-06T07:19:49.355 INFO:tasks.ceph.osd.2:Started 2024-03-06T07:19:49.355 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-06T07:19:49.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-06T07:19:49.357 INFO:tasks.ceph.osd.3:Started 2024-03-06T07:19:49.358 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-06T07:19:49.358 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-06T07:19:49.385 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.384+0000 7f1a9be2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-06T07:19:49.391 INFO:tasks.ceph.osd.4:Started 2024-03-06T07:19:49.391 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-06T07:19:49.391 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-06T07:19:49.393 INFO:tasks.ceph.osd.5:Started 2024-03-06T07:19:49.393 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-06T07:19:49.393 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-06T07:19:49.395 INFO:tasks.ceph.osd.6:Started 2024-03-06T07:19:49.395 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-06T07:19:49.395 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-06T07:19:49.398 INFO:tasks.ceph.osd.7:Started 2024-03-06T07:19:49.398 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:49.423 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.421+0000 7fe5ceafa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-06T07:19:49.454 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.453+0000 7f1a9be2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-06T07:19:49.480 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.478+0000 7fe5ceafa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-06T07:19:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.510+0000 7f6424683640 1 Processor -- start 2024-03-06T07:19:49.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 -- start start 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f641c153ea0 0x7f641c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c0c5b40 0x7f641c0c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641c1576e0 con 0x7f641c0c2ae0 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641c0c42e0 con 0x7f641c153ea0 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.511+0000 7f6424683640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0c45c0 con 0x7f641c0c5b40 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f641c153ea0 0x7f641c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f641c153ea0 0x7f641c1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38210/0 (socket says 172.21.15.17:38210) 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 -- 172.21.15.17:0/2195119166 learned_addr learned my addr 172.21.15.17:0/2195119166 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f6421c1e640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c0c5b40 0x7f641c0c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 -- 172.21.15.17:0/2195119166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c0c5b40 msgr2=0x7f641c0c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c0c5b40 0x7f641c0c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 -- 172.21.15.17:0/2195119166 >> v1:172.21.15.163:6789/0 conn(0x7f641c0c2ae0 legacy=0x7f641c06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642241f640 1 -- 172.21.15.17:0/2195119166 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c0c48a0 con 0x7f641c153ea0 2024-03-06T07:19:49.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.512+0000 7f642141d640 1 -- 172.21.15.17:0/2195119166 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222470039 0 0) 0x7f641c1576e0 con 0x7f641c0c2ae0 2024-03-06T07:19:49.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f642241f640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c153ea0 0x7f641c1571a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9d22f6627c8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f642141d640 1 -- 172.21.15.17:0/2195119166 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c035020 con 0x7f641c153ea0 2024-03-06T07:19:49.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f642141d640 1 -- 172.21.15.17:0/2195119166 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c031640 con 0x7f641c153ea0 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f642141d640 1 -- 172.21.15.17:0/2195119166 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c02d040 con 0x7f641c153ea0 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f6424683640 1 -- 172.21.15.17:0/2195119166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c153ea0 msgr2=0x7f641c1571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.513+0000 7f6424683640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c153ea0 0x7f641c1571a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f640c009870 tx=0x7f640c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 -- 172.21.15.17:0/2195119166 shutdown_connections 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c0c5b40 0x7f641c0c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 --2- 172.21.15.17:0/2195119166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c153ea0 0x7f641c1571a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 -- 172.21.15.17:0/2195119166 >> 172.21.15.17:0/2195119166 conn(0x7f641c0c7800 msgr2=0x7f641c0c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 -- 172.21.15.17:0/2195119166 shutdown_connections 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.514+0000 7f6424683640 1 -- 172.21.15.17:0/2195119166 wait complete. 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 Processor -- start 2024-03-06T07:19:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 -- start start 2024-03-06T07:19:49.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f641c0c5b40 0x7f641c141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c153ea0 0x7f641c141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0c3f60 con 0x7f641c0c2ae0 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c06d9e0 con 0x7f641c0c5b40 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f6424683640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c154280 con 0x7f641c153ea0 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f642241f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f642241f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38216/0 (socket says 172.21.15.17:38216) 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.515+0000 7f642241f640 1 -- 172.21.15.17:0/2514084844 learned_addr learned my addr 172.21.15.17:0/2514084844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6421c1e640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f641c0c5b40 0x7f641c141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6422c20640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c153ea0 0x7f641c141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 -- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c153ea0 msgr2=0x7f641c141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c153ea0 0x7f641c141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 -- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f641c0c5b40 msgr2=0x7f641c141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f641c0c5b40 0x7f641c141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 -- 172.21.15.17:0/2514084844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c153560 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f642241f640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c040070 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c0399f0 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c0c0cd0 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c035040 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c1eb550 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f640c04b060 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.516+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f640c03aaf0 con 0x7f641c0c2ae0 2024-03-06T07:19:49.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.517+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c0c5370 con 0x7f641c0c2ae0 2024-03-06T07:19:49.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.519+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f640c039e60 con 0x7f641c0c2ae0 2024-03-06T07:19:49.575 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.573+0000 7f1a9be2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:49.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.581+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f641c04f330 con 0x7f641c0c2ae0 2024-03-06T07:19:49.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.582+0000 7f6412ffd640 1 -- 172.21.15.17:0/2514084844 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f640c053090 con 0x7f641c0c2ae0 2024-03-06T07:19:49.583 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:19:49.329550+0000","last_up_change":"0.000000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:49.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.582+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 msgr2=0x7f641c1dca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.582+0000 7f6424683640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 shutdown_connections 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f641c153ea0 0x7f641c141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f641c0c5b40 0x7f641c141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 --2- 172.21.15.17:0/2514084844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f641c0c2ae0 0x7f641c1dca40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 >> 172.21.15.17:0/2514084844 conn(0x7f641c0c7800 msgr2=0x7f641c0c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 shutdown_connections 2024-03-06T07:19:49.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:49.583+0000 7f6424683640 1 -- 172.21.15.17:0/2514084844 wait complete. 2024-03-06T07:19:49.594 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-06T07:19:49.595 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-06T07:19:49.598 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.597+0000 7fe5ceafa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:49.674 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.673+0000 7f1a9be2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-06T07:19:49.693 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.691+0000 7fe5ceafa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-06T07:19:49.836 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.835+0000 7f1a9be2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:49.850 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.848+0000 7fe5ceafa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-06T07:19:49.881 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:49.879+0000 7f1a9be2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:49.892 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:49.890+0000 7fe5ceafa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-06T07:19:49.957 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-06T07:19:49.955+0000 7f5607019640 -1 Falling back to public interface 2024-03-06T07:19:49.958 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:49.955+0000 7f65d0811640 -1 Falling back to public interface 2024-03-06T07:19:49.958 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:49.955+0000 7f65cf28e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-06T07:19:49.958 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:49.955+0000 7f65cf28e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-03-06T07:19:49.978 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-06T07:19:49.976+0000 7f0ede60c640 -1 Falling back to public interface 2024-03-06T07:19:49.982 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-06T07:19:49.980+0000 7fec9ede1640 -1 Falling back to public interface 2024-03-06T07:19:49.989 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-03-06T07:19:49.987+0000 7fefef41a640 -1 Falling back to public interface 2024-03-06T07:19:49.994 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-03-06T07:19:49.992+0000 7f37c9a0e640 -1 Falling back to public interface 2024-03-06T07:19:50.014 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-03-06T07:19:50.012+0000 7f9d1f81f640 -1 Falling back to public interface 2024-03-06T07:19:50.022 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-03-06T07:19:50.020+0000 7f8b8eab5640 -1 Falling back to public interface 2024-03-06T07:19:50.048 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:50.047+0000 7f1a9be2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-06T07:19:50.050 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:50.049+0000 7fe5ceafa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-06T07:19:50.400 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:50.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.491+0000 7f63da15e640 1 Processor -- start 2024-03-06T07:19:50.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.491+0000 7f63da15e640 1 -- start start 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.492+0000 7f63da15e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.492+0000 7f63da15e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 0x7f63d40c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.492+0000 7f63da15e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d4155700 con 0x7f63d4155190 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.492+0000 7f63da15e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40c5f50 con 0x7f63d40c7c90 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.492+0000 7f63da15e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41559e0 con 0x7f63d4152b60 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d2ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 0x7f63d40c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:45040/0 (socket says 172.21.15.17:45040) 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d2ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 0x7f63d40c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:44160/0 (socket says 172.21.15.17:44160) 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 -- 172.21.15.17:0/134336421 learned_addr learned my addr 172.21.15.17:0/134336421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:50.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 -- 172.21.15.17:0/134336421 >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 msgr2=0x7f63d40c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:50.496 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 --2- 172.21.15.17:0/134336421 >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 0x7f63d40c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.496 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 -- 172.21.15.17:0/134336421 >> v1:172.21.15.163:6789/0 conn(0x7f63d4155190 legacy=0x7f63d40c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:50.496 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.493+0000 7f63d37fe640 1 -- 172.21.15.17:0/134336421 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4155cc0 con 0x7f63d4152b60 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.494+0000 7f63d37fe640 1 --2- 172.21.15.17:0/134336421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009870 tx=0x7f63c402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb50a90b0c11644 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.494+0000 7f63d27fc640 1 -- 172.21.15.17:0/134336421 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4035020 con 0x7f63d4152b60 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.494+0000 7f63d27fc640 1 -- 172.21.15.17:0/134336421 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4031640 con 0x7f63d4152b60 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.494+0000 7f63d27fc640 1 -- 172.21.15.17:0/134336421 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c402d040 con 0x7f63d4152b60 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 -- 172.21.15.17:0/134336421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 msgr2=0x7f63d40c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 --2- 172.21.15.17:0/134336421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009870 tx=0x7f63c402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 -- 172.21.15.17:0/134336421 shutdown_connections 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 --2- 172.21.15.17:0/134336421 >> v2:172.21.15.163:3300/0 conn(0x7f63d40c7c90 0x7f63d40c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 --2- 172.21.15.17:0/134336421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4152b60 0x7f63d40c7750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.497 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 -- 172.21.15.17:0/134336421 >> 172.21.15.17:0/134336421 conn(0x7f63d40cadc0 msgr2=0x7f63d40cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:50.498 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 -- 172.21.15.17:0/134336421 shutdown_connections 2024-03-06T07:19:50.498 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.495+0000 7f63da15e640 1 -- 172.21.15.17:0/134336421 wait complete. 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.496+0000 7f63da15e640 1 Processor -- start 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.496+0000 7f63da15e640 1 -- start start 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 0x7f63d41ee540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63d4152b60 0x7f63d4148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40be150 con 0x7f63d4152b60 2024-03-06T07:19:50.504 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40ca5b0 con 0x7f63d40c7c90 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63da15e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40be2d0 con 0x7f63d4155190 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63d4152b60 0x7f63d4148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d3fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:45042/0 (socket says 172.21.15.17:45042) 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 0x7f63d41ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d3fff640 1 -- 172.21.15.17:0/788643790 learned_addr learned my addr 172.21.15.17:0/788643790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 -- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 msgr2=0x7f63d4149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 -- 172.21.15.17:0/788643790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63d4152b60 msgr2=0x7f63d4148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63d4152b60 0x7f63d4148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d37fe640 1 -- 172.21.15.17:0/788643790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d41eefc0 con 0x7f63d40c7c90 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.497+0000 7f63d3fff640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63d37fe640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 0x7f63d41ee540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63c4033000 tx=0x7f63c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:50.505 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c403aa60 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c403ac20 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d41585e0 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40443f0 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d40c3dc0 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f63c404b060 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.498+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f63c4043a70 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.499+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d404f330 con 0x7f63d40c7c90 2024-03-06T07:19:50.506 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.501+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f63c403f070 con 0x7f63d40c7c90 2024-03-06T07:19:50.568 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.566+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63d40c88a0 con 0x7f63d40c7c90 2024-03-06T07:19:50.568 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.567+0000 7f63bbfff640 1 -- 172.21.15.17:0/788643790 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f63c4043c70 con 0x7f63d40c7c90 2024-03-06T07:19:50.569 INFO:teuthology.misc.health.smithi017.stdout: 2024-03-06T07:19:50.569 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":10,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:19:49.329550+0000","last_up_change":"0.000000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 msgr2=0x7f63d41ee540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 0x7f63d41ee540 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63c4033000 tx=0x7f63c40046c0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 shutdown_connections 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d4155190 0x7f63d4149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63d4152b60 0x7f63d4148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 --2- 172.21.15.17:0/788643790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d40c7c90 0x7f63d41ee540 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 >> 172.21.15.17:0/788643790 conn(0x7f63d40cadc0 msgr2=0x7f63d4154450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 shutdown_connections 2024-03-06T07:19:50.570 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:50.568+0000 7f63da15e640 1 -- 172.21.15.17:0/788643790 wait complete. 2024-03-06T07:19:50.579 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-06T07:19:50.874 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:50.872+0000 7fe5ceafa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-06T07:19:50.919 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:50.917+0000 7f1a9be2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-06T07:19:51.153 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:51.151+0000 7fe5ceafa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-06T07:19:51.204 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:51.202+0000 7fe5ceafa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-06T07:19:51.208 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:51.206+0000 7f1a9be2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-06T07:19:51.261 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:51.258+0000 7f1a9be2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-06T07:19:51.318 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:51.316+0000 7fe5ceafa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-06T07:19:51.382 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-06T07:19:51.379+0000 7f1a9be2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-06T07:19:52.349 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:52.347+0000 7fe56ff5a640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-06T07:19:52.349 INFO:tasks.ceph.mgr.x.smithi163.stderr:2024-03-06T07:19:52.347+0000 7fe56ff5a640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-06T07:19:53.444 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-03-06T07:19:53.442+0000 7fefef41a640 -1 osd.5 0 log_to_monitors true 2024-03-06T07:19:53.468 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-06T07:19:53.465+0000 7f5607019640 -1 osd.0 0 log_to_monitors true 2024-03-06T07:19:53.473 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:53.471+0000 7f65d0811640 -1 osd.2 0 log_to_monitors true 2024-03-06T07:19:53.486 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-03-06T07:19:53.484+0000 7f37c9a0e640 -1 osd.6 0 log_to_monitors true 2024-03-06T07:19:53.502 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-06T07:19:53.499+0000 7f0ede60c640 -1 osd.3 0 log_to_monitors true 2024-03-06T07:19:53.526 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-06T07:19:53.523+0000 7fec9ede1640 -1 osd.1 0 log_to_monitors true 2024-03-06T07:19:53.532 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-03-06T07:19:53.531+0000 7f9d1f81f640 -1 osd.7 0 log_to_monitors true 2024-03-06T07:19:53.544 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-03-06T07:19:53.543+0000 7f8b8eab5640 -1 osd.4 0 log_to_monitors true 2024-03-06T07:19:56.320 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-03-06T07:19:56.318+0000 7fefead61640 -1 osd.5 0 waiting for initial osdmap 2024-03-06T07:19:56.325 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-06T07:19:56.323+0000 7fec9ad77640 -1 osd.1 0 waiting for initial osdmap 2024-03-06T07:19:56.334 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-06T07:19:56.331+0000 7f0ed9fd3640 -1 osd.3 0 waiting for initial osdmap 2024-03-06T07:19:56.335 INFO:tasks.ceph.osd.5.smithi163.stderr:2024-03-06T07:19:56.332+0000 7fefe2b81640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.343 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-06T07:19:56.341+0000 7fec92b97640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.346 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-06T07:19:56.344+0000 7f0ed1df3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.350 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:56.348+0000 7f65cc275640 -1 osd.2 0 waiting for initial osdmap 2024-03-06T07:19:56.358 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-06T07:19:56.356+0000 7f65c4095640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.361 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-06T07:19:56.358+0000 7f56028ed640 -1 osd.0 0 waiting for initial osdmap 2024-03-06T07:19:56.367 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-03-06T07:19:56.365+0000 7f8b8b25f640 -1 osd.4 0 waiting for initial osdmap 2024-03-06T07:19:56.376 INFO:tasks.ceph.osd.4.smithi163.stderr:2024-03-06T07:19:56.375+0000 7f8b8286b640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.377 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-03-06T07:19:56.375+0000 7f9d1b980640 -1 osd.7 0 waiting for initial osdmap 2024-03-06T07:19:56.378 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-03-06T07:19:56.376+0000 7f37c5475640 -1 osd.6 0 waiting for initial osdmap 2024-03-06T07:19:56.380 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-06T07:19:56.378+0000 7f55fa70d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.389 INFO:tasks.ceph.osd.6.smithi163.stderr:2024-03-06T07:19:56.388+0000 7f37bd295640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:56.390 INFO:tasks.ceph.osd.7.smithi163.stderr:2024-03-06T07:19:56.388+0000 7f9d12f8c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-06T07:19:57.386 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:57.482 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 Processor -- start 2024-03-06T07:19:57.483 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 -- start start 2024-03-06T07:19:57.485 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b140c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.485 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9b140c8b30 0x7f9b14158520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.485 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1406ca00 con 0x7f9b14158a60 2024-03-06T07:19:57.485 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1406aff0 con 0x7f9b140c8b30 2024-03-06T07:19:57.485 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.480+0000 7f9b19402640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1406c4f0 con 0x7f9b140c71d0 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b137fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9b14158a60 0x7f9b140c94e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49194/0 (socket says 172.21.15.17:49194) 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b137fe640 1 -- 172.21.15.17:0/1510165471 learned_addr learned my addr 172.21.15.17:0/1510165471 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b0bfff640 1 --2- 172.21.15.17:0/1510165471 >> v2:172.21.15.163:3300/0 conn(0x7f9b140c8b30 0x7f9b14158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b140c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834128880 0 0) 0x7f9b1406ca00 con 0x7f9b14158a60 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b00003520 con 0x7f9b14158a60 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93282903 0 0) 0x7f9b00003520 con 0x7f9b14158a60 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 msgr2=0x7f9b140c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b140c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 >> v2:172.21.15.163:3300/0 conn(0x7f9b140c8b30 msgr2=0x7f9b14158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 --2- 172.21.15.17:0/1510165471 >> v2:172.21.15.163:3300/0 conn(0x7f9b140c8b30 0x7f9b14158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1406b240 con 0x7f9b14158a60 2024-03-06T07:19:57.486 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9b0c0019f0 con 0x7f9b14158a60 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 >> v1:172.21.15.17:6789/0 conn(0x7f9b14158a60 legacy=0x7f9b140c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b00004d40 0x7f9b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1406aff0 con 0x7f9b000039b0 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1406c4f0 con 0x7f9b00004d40 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0000b9c0 con 0x7f9b140c71d0 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b0bfff640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.481+0000 7f9b137fe640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b00004d40 0x7f9b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 -- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 msgr2=0x7f9b00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b140c71d0 0x7f9b00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 -- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b00004d40 msgr2=0x7f9b000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b00004d40 0x7f9b000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.487 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 -- 172.21.15.17:0/1510165471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b00003520 con 0x7f9b000039b0 2024-03-06T07:19:57.488 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.482+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b00003dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ac0 tx=0x7f9b00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8a2019185c6ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.488 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04039070 con 0x7f9b000039b0 2024-03-06T07:19:57.488 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0402c6b0 con 0x7f9b000039b0 2024-03-06T07:19:57.488 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b127fc640 1 -- 172.21.15.17:0/1510165471 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04009690 con 0x7f9b000039b0 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 -- 172.21.15.17:0/1510165471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 msgr2=0x7f9b00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 --2- 172.21.15.17:0/1510165471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b00003dd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ac0 tx=0x7f9b00005830 comp rx=0 tx=0).stop 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b137fe640 1 -- 172.21.15.17:0/1510165471 reap_dead start 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 -- 172.21.15.17:0/1510165471 shutdown_connections 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 -- 172.21.15.17:0/1510165471 >> 172.21.15.17:0/1510165471 conn(0x7f9b140ac4e0 msgr2=0x7f9b140ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 -- 172.21.15.17:0/1510165471 shutdown_connections 2024-03-06T07:19:57.491 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.483+0000 7f9b19402640 1 -- 172.21.15.17:0/1510165471 wait complete. 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.484+0000 7f9b19402640 1 Processor -- start 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.484+0000 7f9b19402640 1 -- start start 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14157ff0 0x7f9b1414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b1414bfe0 0x7f9b141e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1406bae0 con 0x7f9b000039b0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1406b800 con 0x7f9b14157ff0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b19402640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140c5b80 con 0x7f9b1414bfe0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38912/0 (socket says 172.21.15.17:38912) 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 -- 172.21.15.17:0/34259123 learned_addr learned my addr 172.21.15.17:0/34259123 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b0bfff640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14157ff0 0x7f9b1414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b137fe640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b1414bfe0 0x7f9b141e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.492 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 -- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b1414bfe0 msgr2=0x7f9b141e17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b1414bfe0 0x7f9b141e17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 -- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14157ff0 msgr2=0x7f9b1414baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14157ff0 0x7f9b1414baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.485+0000 7f9b12ffd640 1 -- 172.21.15.17:0/34259123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140c0a30 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b12ffd640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ac0 tx=0x7f9b04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04039050 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04045070 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b140c0d10 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04003a50 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.486+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b140be460 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.487+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1404f330 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.487+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f9b04003bf0 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.488+0000 7f9b0b7fe640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 0x7f9af406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.488+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9b040abe40 con 0x7f9b000039b0 2024-03-06T07:19:57.493 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.488+0000 7f9b0bfff640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 0x7f9af406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.494 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.488+0000 7f9b0bfff640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 0x7f9af406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b141583f0 tx=0x7f9afc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.495 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.493+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b1404f330 con 0x7f9b000039b0 2024-03-06T07:19:57.676 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.674+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b1414c3f0 con 0x7f9b000039b0 2024-03-06T07:19:57.677 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.674+0000 7f9b0b7fe640 1 -- 172.21.15.17:0/34259123 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10909 (secure 0 0 0) 0x7f9b1414c3f0 con 0x7f9b000039b0 2024-03-06T07:19:57.677 INFO:teuthology.misc.health.smithi017.stdout: 2024-03-06T07:19:57.677 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":13,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:19:57.335777+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 msgr2=0x7f9af406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 0x7f9af406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b141583f0 tx=0x7f9afc008040 comp rx=0 tx=0).stop 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 msgr2=0x7f9b14155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ac0 tx=0x7f9b04004730 comp rx=0 tx=0).stop 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 shutdown_connections 2024-03-06T07:19:57.679 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9af406b440 0x7f9af406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b1414bfe0 0x7f9b141e17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b14157ff0 0x7f9b1414baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 --2- 172.21.15.17:0/34259123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b000039b0 0x7f9b14155230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 >> 172.21.15.17:0/34259123 conn(0x7f9b140ac4e0 msgr2=0x7f9b140ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 shutdown_connections 2024-03-06T07:19:57.680 INFO:teuthology.misc.health.smithi017.stderr:2024-03-06T07:19:57.676+0000 7f9b19402640 1 -- 172.21.15.17:0/34259123 wait complete. 2024-03-06T07:19:57.688 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-06T07:19:57.689 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-06T07:19:57.689 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:19:57.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:19:57.692 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:19:57.692 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:19:57.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:19:57.694 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:19:57.694 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:19:57.694 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:19:57.697 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:19:57.697 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:19:57.697 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:19:57.699 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:19:57.699 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-06T07:19:57.702 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:19:57.768 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:19:57.768 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:19:57.782 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:19:57.783 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:19:57.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.805+0000 7fc03d8df640 1 Processor -- start 2024-03-06T07:19:57.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.805+0000 7fc03d8df640 1 -- start start 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc03d8df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc038152c20 0x7fc03815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc03d8df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc038072440 con 0x7fc0380c3150 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc03d8df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0380c2b10 con 0x7fc03814f650 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc03d8df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc038152140 con 0x7fc038152c20 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc02ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc0380c3150 0x7fc0380c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49220/0 (socket says 172.21.15.17:49220) 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc02ffff640 1 -- 172.21.15.17:0/2690581489 learned_addr learned my addr 172.21.15.17:0/2690581489 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0377fe640 1 --2- 172.21.15.17:0/2690581489 >> v2:172.21.15.17:3300/0 conn(0x7fc038152c20 0x7fc03815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136575358 0 0) 0x7fc020001110 con 0x7fc0380c3150 2024-03-06T07:19:57.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc024005730 con 0x7fc0380c3150 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0377fe640 1 -- 172.21.15.17:0/2690581489 >> v1:172.21.15.163:6789/0 conn(0x7fc03814f650 legacy=0x7fc038071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544012645 0 0) 0x7fc0380c2b10 con 0x7fc03814f650 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0377fe640 1 -- 172.21.15.17:0/2690581489 >> v1:172.21.15.17:6789/0 conn(0x7fc0380c3150 legacy=0x7fc0380c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.806+0000 7fc0377fe640 1 -- 172.21.15.17:0/2690581489 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038152420 con 0x7fc038152c20 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.807+0000 7fc0377fe640 1 --2- 172.21.15.17:0/2690581489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc038152c20 0x7fc03815d820 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d25d355a8152cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.807+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030039070 con 0x7fc038152c20 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.807+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03002cc60 con 0x7fc038152c20 2024-03-06T07:19:57.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.807+0000 7fc0367fc640 1 -- 172.21.15.17:0/2690581489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030034660 con 0x7fc038152c20 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 -- 172.21.15.17:0/2690581489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc038152c20 msgr2=0x7fc03815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 --2- 172.21.15.17:0/2690581489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc038152c20 0x7fc03815d820 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc030009d90 tx=0x7fc03002beb0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 -- 172.21.15.17:0/2690581489 shutdown_connections 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 --2- 172.21.15.17:0/2690581489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc038152c20 0x7fc03815d820 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 -- 172.21.15.17:0/2690581489 >> 172.21.15.17:0/2690581489 conn(0x7fc0380c4290 msgr2=0x7fc0380c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:57.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 -- 172.21.15.17:0/2690581489 shutdown_connections 2024-03-06T07:19:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.808+0000 7fc03d8df640 1 -- 172.21.15.17:0/2690581489 wait complete. 2024-03-06T07:19:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.809+0000 7fc03d8df640 1 Processor -- start 2024-03-06T07:19:57.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.809+0000 7fc03d8df640 1 -- start start 2024-03-06T07:19:57.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 0x7fc0381e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 0x7fc0381e8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038072110 con 0x7fc0380c3150 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc038071f90 con 0x7fc038152c20 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc03d8df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0380c3f40 con 0x7fc03814f650 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc036ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc036ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38964/0 (socket says 172.21.15.17:38964) 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc036ffd640 1 -- 172.21.15.17:0/808799479 learned_addr learned my addr 172.21.15.17:0/808799479 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc02ffff640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 0x7fc0381e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc0377fe640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 0x7fc0381e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc036ffd640 1 -- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 msgr2=0x7fc0381e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.810+0000 7fc036ffd640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 0x7fc0381e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc036ffd640 1 -- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 msgr2=0x7fc0381e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc036ffd640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 0x7fc0381e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc036ffd640 1 -- 172.21.15.17:0/808799479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03814ff40 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc02ffff640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 0x7fc0381e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc0377fe640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 0x7fc0381e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc036ffd640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c490 tx=0x7fc02800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028005480 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc038150220 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028005480 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280035f0 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.811+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0381507e0 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.812+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03804f330 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.812+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fc028003790 con 0x7fc0380c3150 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.813+0000 7fc034ff9640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 0x7fc01806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:57.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.813+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fc028089450 con 0x7fc0380c3150 2024-03-06T07:19:57.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.813+0000 7fc02ffff640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 0x7fc01806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:57.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.814+0000 7fc02ffff640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 0x7fc01806db00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc020007fd0 tx=0x7fc020009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:57.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.817+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc028058890 con 0x7fc0380c3150 2024-03-06T07:19:58.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.999+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc038068460 con 0x7fc0380c3150 2024-03-06T07:19:58.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:57.999+0000 7fc034ff9640 1 -- 172.21.15.17:0/808799479 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10909 (secure 0 0 0) 0x7fc028058190 con 0x7fc0380c3150 2024-03-06T07:19:58.002 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:58.002 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":13,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:19:57.335777+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:19:58.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 msgr2=0x7fc01806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 0x7fc01806db00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc020007fd0 tx=0x7fc020009040 comp rx=0 tx=0).stop 2024-03-06T07:19:58.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 msgr2=0x7fc0380c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c490 tx=0x7fc02800c950 comp rx=0 tx=0).stop 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 shutdown_connections 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc01806b640 0x7fc01806db00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc038152c20 0x7fc0381e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc03814f650 0x7fc0381e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 --2- 172.21.15.17:0/808799479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0380c3150 0x7fc0380c6640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 >> 172.21.15.17:0/808799479 conn(0x7fc0380c4290 msgr2=0x7fc03814a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 shutdown_connections 2024-03-06T07:19:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.001+0000 7fc03d8df640 1 -- 172.21.15.17:0/808799479 wait complete. 2024-03-06T07:19:58.015 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-06T07:19:58.073 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:19:58.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.171+0000 7f020fcdf640 1 Processor -- start 2024-03-06T07:19:58.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.171+0000 7f020fcdf640 1 -- start start 2024-03-06T07:19:58.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020fcdf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02080c88c0 0x7f0208155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020fcdf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02080c73d0 con 0x7f0208158b60 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020fcdf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02080be2e0 con 0x7f020814f360 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020fcdf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0208159700 con 0x7f02080c88c0 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020e27c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0208158b60 0x7f0208158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49232/0 (socket says 172.21.15.17:49232) 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020e27c640 1 -- 172.21.15.17:0/3731205365 learned_addr learned my addr 172.21.15.17:0/3731205365 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020d27a640 1 --2- 172.21.15.17:0/3731205365 >> v2:172.21.15.163:3300/0 conn(0x7f02080c88c0 0x7f0208155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985741339 0 0) 0x7f02080c73d0 con 0x7f0208158b60 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.172+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f0003520 con 0x7f0208158b60 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1191306460 0 0) 0x7f01f0003520 con 0x7f0208158b60 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 >> v2:172.21.15.163:3300/0 conn(0x7f02080c88c0 msgr2=0x7f0208155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 --2- 172.21.15.17:0/3731205365 >> v2:172.21.15.163:3300/0 conn(0x7f02080c88c0 0x7f0208155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 >> v1:172.21.15.163:6789/0 conn(0x7f020814f360 legacy=0x7f02080c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02080c7670 con 0x7f0208158b60 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0200001670 con 0x7f0208158b60 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020d27a640 1 --2- 172.21.15.17:0/3731205365 >> v2:172.21.15.163:3300/0 conn(0x7f02080c88c0 0x7f0208155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 >> v1:172.21.15.17:6789/0 conn(0x7f0208158b60 legacy=0x7f0208158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f01f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0208159700 con 0x7f01f00039b0 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000c160 con 0x7f01f00051c0 2024-03-06T07:19:58.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f01f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000c360 con 0x7f01f0004800 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020e27c640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020d27a640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 -- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f0004800 msgr2=0x7f01f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 -- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00051c0 msgr2=0x7f01f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020da7b640 1 -- 172.21.15.17:0/3731205365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0003520 con 0x7f01f00039b0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.173+0000 7f020e27c640 1 -- 172.21.15.17:0/3731205365 reap_dead start 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020da7b640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f01f0003dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001e50 tx=0x7f01f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc48ac08dd0dc80c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8001a70 con 0x7f01f00039b0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8003b70 con 0x7f01f00039b0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020ca79640 1 -- 172.21.15.17:0/3731205365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80050d0 con 0x7f01f00039b0 2024-03-06T07:19:58.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020fcdf640 1 -- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 msgr2=0x7f01f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020fcdf640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f01f0003dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f01f8001e50 tx=0x7f01f0005830 comp rx=0 tx=0).stop 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020fcdf640 1 -- 172.21.15.17:0/3731205365 shutdown_connections 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020fcdf640 1 --2- 172.21.15.17:0/3731205365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f01f0003dd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.174+0000 7f020fcdf640 1 -- 172.21.15.17:0/3731205365 >> 172.21.15.17:0/3731205365 conn(0x7f02080ac570 msgr2=0x7f02080ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.175+0000 7f020fcdf640 1 -- 172.21.15.17:0/3731205365 shutdown_connections 2024-03-06T07:19:58.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.175+0000 7f020fcdf640 1 -- 172.21.15.17:0/3731205365 wait complete. 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 Processor -- start 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 -- start start 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02081ee310 0x7f02080beec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 0x7f02080bf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02080c7c30 con 0x7f01f00039b0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0208155c10 con 0x7f02081ee790 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020fcdf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0208155d90 con 0x7f02081ee310 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020da7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020da7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38990/0 (socket says 172.21.15.17:38990) 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020da7b640 1 -- 172.21.15.17:0/898049845 learned_addr learned my addr 172.21.15.17:0/898049845 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020d27a640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02081ee310 0x7f02080beec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.176+0000 7f020e27c640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 0x7f02080bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 -- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02081ee310 msgr2=0x7f02080beec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02081ee310 0x7f02080beec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 -- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 msgr2=0x7f02080bf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 0x7f02080bf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 -- 172.21.15.17:0/898049845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02081eefa0 con 0x7f01f00039b0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020e27c640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 0x7f02080bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020da7b640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f80 tx=0x7f01f80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8007db0 con 0x7f01f00039b0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8003970 con 0x7f01f00039b0 2024-03-06T07:19:58.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02080c0c90 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8006d70 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.177+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02081eecc0 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.178+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020804f330 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.178+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f01f802f440 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.179+0000 7f02067fc640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 0x7f01f406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.179+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f01f80ab200 con 0x7f01f00039b0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.179+0000 7f020d27a640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 0x7f01f406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.179+0000 7f020d27a640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 0x7f01f406d900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f02080bfea0 tx=0x7f01fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:19:58.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.183+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f01f8033020 con 0x7f01f00039b0 2024-03-06T07:19:58.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.406+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02080c9510 con 0x7f01f00039b0 2024-03-06T07:19:58.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.407+0000 7f02067fc640 1 -- 172.21.15.17:0/898049845 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f01f807a6c0 con 0x7f01f00039b0 2024-03-06T07:19:58.409 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:19:58.410 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-06T07:19:58.410 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-03-06T07:19:58.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 msgr2=0x7f01f406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 0x7f01f406d900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f02080bfea0 tx=0x7f01fc008040 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 msgr2=0x7f020814bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f01f8000f80 tx=0x7f01f80014a0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 shutdown_connections 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01f406b440 0x7f01f406d900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02081ee790 0x7f02080bf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02081ee310 0x7f02080beec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 --2- 172.21.15.17:0/898049845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f00039b0 0x7f020814bda0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 >> 172.21.15.17:0/898049845 conn(0x7f02080ac570 msgr2=0x7f02080c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 shutdown_connections 2024-03-06T07:19:58.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:19:58.409+0000 7f020fcdf640 1 -- 172.21.15.17:0/898049845 wait complete. 2024-03-06T07:20:00.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:00.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.525+0000 7fd1e2ac0640 1 Processor -- start 2024-03-06T07:20:00.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.525+0000 7fd1e2ac0640 1 -- start start 2024-03-06T07:20:00.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e2ac0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 0x7fd1dc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e2ac0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0c28d0 con 0x7fd1dc0c1aa0 2024-03-06T07:20:00.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e2ac0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0c2bb0 con 0x7fd1dc155a40 2024-03-06T07:20:00.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e2ac0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0c2680 con 0x7fd1dc14f790 2024-03-06T07:20:00.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1dbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1dc0c1aa0 0x7fd1dc0c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59934/0 (socket says 172.21.15.17:59934) 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1dbfff640 1 -- 172.21.15.17:0/2474157424 learned_addr learned my addr 172.21.15.17:0/2474157424 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e085c640 1 --2- 172.21.15.17:0/2474157424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 0x7fd1dc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1db7fe640 1 -- 172.21.15.17:0/2474157424 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859069731 0 0) 0x7fd1dc0c28d0 con 0x7fd1dc0c1aa0 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1db7fe640 1 -- 172.21.15.17:0/2474157424 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1c4003520 con 0x7fd1dc0c1aa0 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e085c640 1 -- 172.21.15.17:0/2474157424 >> v1:172.21.15.163:6789/0 conn(0x7fd1dc155a40 legacy=0x7fd1dc155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e085c640 1 -- 172.21.15.17:0/2474157424 >> v1:172.21.15.17:6789/0 conn(0x7fd1dc0c1aa0 legacy=0x7fd1dc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.526+0000 7fd1e085c640 1 -- 172.21.15.17:0/2474157424 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0c2e90 con 0x7fd1dc14f790 2024-03-06T07:20:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.527+0000 7fd1e085c640 1 --2- 172.21.15.17:0/2474157424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 0x7fd1dc1529d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009dc0 tx=0x7fd1d402bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8bc74d15644ca56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.527+0000 7fd1db7fe640 1 -- 172.21.15.17:0/2474157424 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4039070 con 0x7fd1dc14f790 2024-03-06T07:20:00.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.527+0000 7fd1db7fe640 1 -- 172.21.15.17:0/2474157424 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d402ccd0 con 0x7fd1dc14f790 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.527+0000 7fd1db7fe640 1 -- 172.21.15.17:0/2474157424 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4034640 con 0x7fd1dc14f790 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/2474157424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 msgr2=0x7fd1dc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/2474157424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 0x7fd1dc1529d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009dc0 tx=0x7fd1d402bf20 comp rx=0 tx=0).stop 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/2474157424 shutdown_connections 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/2474157424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc14f790 0x7fd1dc1529d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/2474157424 >> 172.21.15.17:0/2474157424 conn(0x7fd1dc0c6710 msgr2=0x7fd1dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/2474157424 shutdown_connections 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.528+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/2474157424 wait complete. 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.529+0000 7fd1e2ac0640 1 Processor -- start 2024-03-06T07:20:00.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.529+0000 7fd1e2ac0640 1 -- start start 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc0c1aa0 0x7fd1dc0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc155a40 0x7fd1dc14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0c5e30 con 0x7fd1dc14f790 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0c0d60 con 0x7fd1dc0c1aa0 2024-03-06T07:20:00.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e2ac0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0c5fb0 con 0x7fd1dc155a40 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52376/0 (socket says 172.21.15.17:52376) 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 -- 172.21.15.17:0/405173481 learned_addr learned my addr 172.21.15.17:0/405173481 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e085c640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc0c1aa0 0x7fd1dc0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1e105d640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc155a40 0x7fd1dc14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 -- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc155a40 msgr2=0x7fd1dc14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc155a40 0x7fd1dc14ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 -- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc0c1aa0 msgr2=0x7fd1dc0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc0c1aa0 0x7fd1dc0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 -- 172.21.15.17:0/405173481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc1dca20 con 0x7fd1dc14f790 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1dbfff640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c500 tx=0x7fd1d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.530+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0017070 con 0x7fd1dc14f790 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.531+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d0005ce0 con 0x7fd1dc14f790 2024-03-06T07:20:00.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.531+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc1e9fd0 con 0x7fd1dc14f790 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.531+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0007330 con 0x7fd1dc14f790 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.531+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc1ea590 con 0x7fd1dc14f790 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.532+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc04f330 con 0x7fd1dc14f790 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.532+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fd1d001a070 con 0x7fd1dc14f790 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.533+0000 7fd1d97fa640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 0x7fd1c806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.533+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd1d0089110 con 0x7fd1dc14f790 2024-03-06T07:20:00.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.533+0000 7fd1e085c640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 0x7fd1c806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.533+0000 7fd1e085c640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 0x7fd1c806d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4036000 tx=0x7fd1d4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.537+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1d0057ec0 con 0x7fd1dc14f790 2024-03-06T07:20:00.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.760+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1dc0ab5c0 con 0x7fd1dc14f790 2024-03-06T07:20:00.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.760+0000 7fd1d97fa640 1 -- 172.21.15.17:0/405173481 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7fd1d005bf60 con 0x7fd1dc14f790 2024-03-06T07:20:00.763 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:00.763 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-06T07:20:00.763 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2024-03-06T07:20:00.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.762+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 msgr2=0x7fd1c806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.762+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 0x7fd1c806d900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4036000 tx=0x7fd1d4030000 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.762+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 msgr2=0x7fd1dc14ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.762+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c500 tx=0x7fd1d000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 shutdown_connections 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1c806b440 0x7fd1c806d900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc155a40 0x7fd1dc14ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1dc14f790 0x7fd1dc14ea40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 --2- 172.21.15.17:0/405173481 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc0c1aa0 0x7fd1dc0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 >> 172.21.15.17:0/405173481 conn(0x7fd1dc0c6710 msgr2=0x7fd1dc0c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 shutdown_connections 2024-03-06T07:20:00.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.763+0000 7fd1e2ac0640 1 -- 172.21.15.17:0/405173481 wait complete. 2024-03-06T07:20:00.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.874+0000 7fbd165d6640 1 Processor -- start 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.874+0000 7fbd165d6640 1 -- start start 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd165d6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 0x7fbd101566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd165d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd100c6000 0x7fbd10156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd165d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd10153a60 con 0x7fbd10150580 2024-03-06T07:20:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd165d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd10153d40 con 0x7fbd100c6000 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd165d6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd1006c2c0 con 0x7fbd100c1320 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd14b73640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd100c6000 0x7fbd10156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd0ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbd10150580 0x7fbd1006dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59950/0 (socket says 172.21.15.17:59950) 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd0ffff640 1 -- 172.21.15.17:0/4132000738 learned_addr learned my addr 172.21.15.17:0/4132000738 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd0f7fe640 1 --2- 172.21.15.17:0/4132000738 >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 0x7fbd101566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd14b73640 1 -- 172.21.15.17:0/4132000738 >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 msgr2=0x7fbd101566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd14b73640 1 --2- 172.21.15.17:0/4132000738 >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 0x7fbd101566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd14b73640 1 -- 172.21.15.17:0/4132000738 >> v1:172.21.15.17:6789/0 conn(0x7fbd10150580 legacy=0x7fbd1006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd14b73640 1 -- 172.21.15.17:0/4132000738 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1006c5a0 con 0x7fbd100c6000 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.875+0000 7fbd0f7fe640 1 --2- 172.21.15.17:0/4132000738 >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 0x7fbd101566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd14b73640 1 --2- 172.21.15.17:0/4132000738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd100c6000 0x7fbd10156bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd100c7730 tx=0x7fbd0802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0118947be723a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd0effd640 1 -- 172.21.15.17:0/4132000738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08039070 con 0x7fbd100c6000 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd0effd640 1 -- 172.21.15.17:0/4132000738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0802cbf0 con 0x7fbd100c6000 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd0effd640 1 -- 172.21.15.17:0/4132000738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08034440 con 0x7fbd100c6000 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 -- 172.21.15.17:0/4132000738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd100c6000 msgr2=0x7fbd10156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 --2- 172.21.15.17:0/4132000738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd100c6000 0x7fbd10156bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd100c7730 tx=0x7fbd0802be40 comp rx=0 tx=0).stop 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 -- 172.21.15.17:0/4132000738 shutdown_connections 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 --2- 172.21.15.17:0/4132000738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd100c6000 0x7fbd10156bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 --2- 172.21.15.17:0/4132000738 >> v2:172.21.15.163:3300/0 conn(0x7fbd100c1320 0x7fbd101566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 -- 172.21.15.17:0/4132000738 >> 172.21.15.17:0/4132000738 conn(0x7fbd100c6710 msgr2=0x7fbd100c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.876+0000 7fbd165d6640 1 -- 172.21.15.17:0/4132000738 shutdown_connections 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.877+0000 7fbd165d6640 1 -- 172.21.15.17:0/4132000738 wait complete. 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 Processor -- start 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 -- start start 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd100c1320 0x7fbd100a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd100c6000 0x7fbd101541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1006e5a0 con 0x7fbd10150580 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1006e420 con 0x7fbd100c6000 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd165d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100c0c40 con 0x7fbd100c1320 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd14b73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.878+0000 7fbd14b73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52402/0 (socket says 172.21.15.17:52402) 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 -- 172.21.15.17:0/1411686645 learned_addr learned my addr 172.21.15.17:0/1411686645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd0f7fe640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd100c6000 0x7fbd101541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd0ffff640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd100c1320 0x7fbd100a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 -- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd100c1320 msgr2=0x7fbd100a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd100c1320 0x7fbd100a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 -- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd100c6000 msgr2=0x7fbd101541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd100c6000 0x7fbd101541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 -- 172.21.15.17:0/1411686645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd101e93b0 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd14b73640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009e80 tx=0x7fbd08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08039050 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd08045070 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd10156270 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0803e830 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.879+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd10150d30 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.880+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1004f330 con 0x7fbd10150580 2024-03-06T07:20:00.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.881+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fbd0803e9d0 con 0x7fbd10150580 2024-03-06T07:20:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.881+0000 7fbd0cff9640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 0x7fbcfc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.881+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd080ace40 con 0x7fbd10150580 2024-03-06T07:20:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.881+0000 7fbd0ffff640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 0x7fbcfc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.882+0000 7fbd0ffff640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 0x7fbcfc06d930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000f60 tx=0x7fbd04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:00.885+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd08032e20 con 0x7fbd10150580 2024-03-06T07:20:01.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.067+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd100c5de0 con 0x7fbd10150580 2024-03-06T07:20:01.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.068+0000 7fbd0cff9640 1 -- 172.21.15.17:0/1411686645 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12884 (secure 0 0 0) 0x7fbd0807c2a0 con 0x7fbd10150580 2024-03-06T07:20:01.070 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:01.071 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":16,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:00.388381+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:01.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 msgr2=0x7fbcfc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 0x7fbcfc06d930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000f60 tx=0x7fbd04009040 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 msgr2=0x7fbd10146930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009e80 tx=0x7fbd08004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 shutdown_connections 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcfc06b470 0x7fbcfc06d930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd10150580 0x7fbd10146930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd100c6000 0x7fbd101541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 --2- 172.21.15.17:0/1411686645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd100c1320 0x7fbd100a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 >> 172.21.15.17:0/1411686645 conn(0x7fbd100c6710 msgr2=0x7fbd100c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 shutdown_connections 2024-03-06T07:20:01.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.070+0000 7fbd165d6640 1 -- 172.21.15.17:0/1411686645 wait complete. 2024-03-06T07:20:01.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:20:01.084 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:01.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.186+0000 7fd32ffff640 1 Processor -- start 2024-03-06T07:20:01.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.186+0000 7fd32ffff640 1 -- start start 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd330155d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3300c5910 0x7fd33006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32ffff640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3300c5130 con 0x7fd33006c270 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32ffff640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33006ce90 con 0x7fd3300c5910 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32ffff640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c53d0 con 0x7fd33006bbf0 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3300c5910 0x7fd33006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd33006c270 0x7fd3300c86b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59960/0 (socket says 172.21.15.17:59960) 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32effd640 1 -- 172.21.15.17:0/3181086332 learned_addr learned my addr 172.21.15.17:0/3181086332 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32e7fc640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd330155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32f7fe640 1 -- 172.21.15.17:0/3181086332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 msgr2=0x7fd330155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32f7fe640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd330155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32f7fe640 1 -- 172.21.15.17:0/3181086332 >> v1:172.21.15.17:6789/0 conn(0x7fd33006c270 legacy=0x7fd3300c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32f7fe640 1 -- 172.21.15.17:0/3181086332 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300c56b0 con 0x7fd3300c5910 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32dffb640 1 -- 172.21.15.17:0/3181086332 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66555208 0 0) 0x7fd3300c5130 con 0x7fd33006c270 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.187+0000 7fd32e7fc640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd330155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.188+0000 7fd32f7fe640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd33006c650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d90 tx=0x7fd32002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51b0ffc9303497 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.188+0000 7fd32dffb640 1 -- 172.21.15.17:0/3181086332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320039070 con 0x7fd3300c5910 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.188+0000 7fd32dffb640 1 -- 172.21.15.17:0/3181086332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32002cc50 con 0x7fd3300c5910 2024-03-06T07:20:01.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.188+0000 7fd32dffb640 1 -- 172.21.15.17:0/3181086332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320034440 con 0x7fd3300c5910 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 -- 172.21.15.17:0/3181086332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 msgr2=0x7fd33006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd33006c650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d90 tx=0x7fd32002bea0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 -- 172.21.15.17:0/3181086332 shutdown_connections 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd33006c650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 --2- 172.21.15.17:0/3181086332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd330155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 -- 172.21.15.17:0/3181086332 >> 172.21.15.17:0/3181086332 conn(0x7fd3300ca7b0 msgr2=0x7fd3300cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 -- 172.21.15.17:0/3181086332 shutdown_connections 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.189+0000 7fd32ffff640 1 -- 172.21.15.17:0/3181086332 wait complete. 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.190+0000 7fd32ffff640 1 Processor -- start 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.190+0000 7fd32ffff640 1 -- start start 2024-03-06T07:20:01.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd3300aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd33006c270 0x7fd3301edfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301557f0 con 0x7fd3300c5910 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330155510 con 0x7fd33006c270 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32ffff640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300c8dc0 con 0x7fd33006bbf0 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52420/0 (socket says 172.21.15.17:52420) 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 -- 172.21.15.17:0/2812915376 learned_addr learned my addr 172.21.15.17:0/2812915376 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32e7fc640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd33006c270 0x7fd3301edfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32effd640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd3300aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 -- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 msgr2=0x7fd3300aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd3300aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 -- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd33006c270 msgr2=0x7fd3301edfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd33006c270 0x7fd3301edfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.191+0000 7fd32f7fe640 1 -- 172.21.15.17:0/2812915376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd330151f40 con 0x7fd3300c5910 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd32f7fe640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d90 tx=0x7fd320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320044020 con 0x7fd3300c5910 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32003a040 con 0x7fd3300c5910 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd330151130 con 0x7fd3300c5910 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320039050 con 0x7fd3300c5910 2024-03-06T07:20:01.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.192+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33014e010 con 0x7fd3300c5910 2024-03-06T07:20:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.193+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3300693c0 con 0x7fd3300c5910 2024-03-06T07:20:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.193+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fd320003990 con 0x7fd3300c5910 2024-03-06T07:20:01.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.194+0000 7fd327fff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 0x7fd31006d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.194+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3200ac000 con 0x7fd3300c5910 2024-03-06T07:20:01.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.194+0000 7fd32effd640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 0x7fd31006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.194+0000 7fd32effd640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 0x7fd31006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd328007fd0 tx=0x7fd328007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.198+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd32004e060 con 0x7fd3300c5910 2024-03-06T07:20:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.378+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd33006afc0 con 0x7fd3300c5910 2024-03-06T07:20:01.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.378+0000 7fd327fff640 1 -- 172.21.15.17:0/2812915376 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd32007acf0 con 0x7fd3300c5910 2024-03-06T07:20:01.381 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 msgr2=0x7fd31006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 0x7fd31006d900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd328007fd0 tx=0x7fd328007de0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 msgr2=0x7fd3301ee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd320009d90 tx=0x7fd320004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 shutdown_connections 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31006b440 0x7fd31006d900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3300c5910 0x7fd3301ee520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd33006c270 0x7fd3301edfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 --2- 172.21.15.17:0/2812915376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd33006bbf0 0x7fd3300aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 >> 172.21.15.17:0/2812915376 conn(0x7fd3300ca7b0 msgr2=0x7fd3300cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 shutdown_connections 2024-03-06T07:20:01.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.381+0000 7fd32ffff640 1 -- 172.21.15.17:0/2812915376 wait complete. 2024-03-06T07:20:01.394 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:20:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 Processor -- start 2024-03-06T07:20:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 -- start start 2024-03-06T07:20:01.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe68c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c155110 con 0x7fe68c158db0 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0c8f40 con 0x7fe68c0c88b0 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.493+0000 7fe6923ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1559e0 con 0x7fe68c0c5330 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe690949640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe68c158db0 0x7fe68c15fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59976/0 (socket says 172.21.15.17:59976) 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe690949640 1 -- 172.21.15.17:0/3139016317 learned_addr learned my addr 172.21.15.17:0/3139016317 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68bfff640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe68c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862021640 0 0) 0x7fe684001490 con 0x7fe68c158db0 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003520 con 0x7fe68c158db0 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154587541 0 0) 0x7fe674003520 con 0x7fe68c158db0 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 msgr2=0x7fe68c0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe68c0c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 >> v1:172.21.15.163:6789/0 conn(0x7fe68c0c88b0 legacy=0x7fe68c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c155390 con 0x7fe68c158db0 2024-03-06T07:20:01.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe684001bc0 con 0x7fe68c158db0 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 >> v1:172.21.15.17:6789/0 conn(0x7fe68c158db0 legacy=0x7fe68c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6740039e0 0x7fe674003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe674004800 0x7fe674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe674005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1559e0 con 0x7fe6740039e0 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400baf0 con 0x7fe674004800 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bcf0 con 0x7fe68c0c5330 2024-03-06T07:20:01.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68bfff640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6740039e0 0x7fe674003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe68b7fe640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe674004800 0x7fe674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.494+0000 7fe690949640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe674005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 -- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 msgr2=0x7fe674005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe674005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 -- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe674004800 msgr2=0x7fe674004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe674004800 0x7fe674004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 -- 172.21.15.17:0/3139016317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003520 con 0x7fe6740039e0 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe690949640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe68c0c5330 0x7fe674005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:01.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68bfff640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6740039e0 0x7fe674003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ac0 tx=0x7fe674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2501aa240c8c43aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c039070 con 0x7fe6740039e0 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c02c6b0 con 0x7fe6740039e0 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.495+0000 7fe68affd640 1 -- 172.21.15.17:0/3139016317 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c009690 con 0x7fe6740039e0 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 -- 172.21.15.17:0/3139016317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6740039e0 msgr2=0x7fe674003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 --2- 172.21.15.17:0/3139016317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6740039e0 0x7fe674003e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009ac0 tx=0x7fe674005830 comp rx=0 tx=0).stop 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe690949640 1 -- 172.21.15.17:0/3139016317 reap_dead start 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 -- 172.21.15.17:0/3139016317 shutdown_connections 2024-03-06T07:20:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 -- 172.21.15.17:0/3139016317 >> 172.21.15.17:0/3139016317 conn(0x7fe68c0ac560 msgr2=0x7fe68c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 -- 172.21.15.17:0/3139016317 shutdown_connections 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.496+0000 7fe6923ac640 1 -- 172.21.15.17:0/3139016317 wait complete. 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.497+0000 7fe6923ac640 1 Processor -- start 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.497+0000 7fe6923ac640 1 -- start start 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6740039e0 0x7fe68c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c14be20 0x7fe68c14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1529d0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c06d090 con 0x7fe68c14be20 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe6923ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c152b50 con 0x7fe6740039e0 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52436/0 (socket says 172.21.15.17:52436) 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 -- 172.21.15.17:0/104409860 learned_addr learned my addr 172.21.15.17:0/104409860 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68bfff640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6740039e0 0x7fe68c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe690949640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c14be20 0x7fe68c14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 -- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6740039e0 msgr2=0x7fe68c157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6740039e0 0x7fe68c157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 -- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c14be20 msgr2=0x7fe68c14b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c14be20 0x7fe68c14b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 -- 172.21.15.17:0/104409860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c14e640 con 0x7fe68c14cbe0 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.498+0000 7fe68b7fe640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f820 tx=0x7fe68000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.499+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6800050d0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.499+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680018070 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.499+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c14e920 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.499+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680005490 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.499+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c158150 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.500+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c04f330 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.500+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fe68000d6d0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.500+0000 7fe688ff9640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 0x7fe67806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.500+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe680089a90 con 0x7fe68c14cbe0 2024-03-06T07:20:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.500+0000 7fe68bfff640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 0x7fe67806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.501+0000 7fe68bfff640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 0x7fe67806d900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe67c036000 tx=0x7fe67c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.505+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe680058dd0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.686+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe68c06cdc0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.686+0000 7fe688ff9640 1 -- 172.21.15.17:0/104409860 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe6800586d0 con 0x7fe68c14cbe0 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:20:01.689 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:20:01.690 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:20:01.690 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:20:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 msgr2=0x7fe67806d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 0x7fe67806d900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe67c036000 tx=0x7fe67c030000 comp rx=0 tx=0).stop 2024-03-06T07:20:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 msgr2=0x7fe68c14abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f820 tx=0x7fe68000fce0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 shutdown_connections 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe67806b440 0x7fe67806d900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe68c14be20 0x7fe68c14b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe68c14cbe0 0x7fe68c14abc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 --2- 172.21.15.17:0/104409860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6740039e0 0x7fe68c157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.688+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 >> 172.21.15.17:0/104409860 conn(0x7fe68c0ac560 msgr2=0x7fe68c0c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.689+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 shutdown_connections 2024-03-06T07:20:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.689+0000 7fe6923ac640 1 -- 172.21.15.17:0/104409860 wait complete. 2024-03-06T07:20:01.704 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:20:01.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:20:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 Processor -- start 2024-03-06T07:20:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 -- start start 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f469806c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f469806c610 0x7f46980aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469814f8b0 con 0x7f46980c23f0 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46980c3680 con 0x7f469806c610 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f469dda2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980c2a70 con 0x7f469814f4d0 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f4697fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f469806c610 0x7f46980aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f4697fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f469806c610 0x7f46980aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52444/0 (socket says 172.21.15.17:52444) 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.809+0000 7f4697fff640 1 -- 172.21.15.17:0/1557320095 learned_addr learned my addr 172.21.15.17:0/1557320095 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.810+0000 7f4697fff640 1 -- 172.21.15.17:0/1557320095 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 msgr2=0x7f469806c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.810+0000 7f4697fff640 1 --2- 172.21.15.17:0/1557320095 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f469806c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.810+0000 7f4697fff640 1 -- 172.21.15.17:0/1557320095 >> v1:172.21.15.163:6789/0 conn(0x7f46980c23f0 legacy=0x7f4698155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.810+0000 7f4697fff640 1 -- 172.21.15.17:0/1557320095 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980c2d50 con 0x7f469806c610 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.810+0000 7f4697fff640 1 --2- 172.21.15.17:0/1557320095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46980aab80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc3c2f7b35daf02 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f46977fe640 1 -- 172.21.15.17:0/1557320095 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840317c0 con 0x7f469806c610 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f46977fe640 1 -- 172.21.15.17:0/1557320095 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684037070 con 0x7f469806c610 2024-03-06T07:20:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f46977fe640 1 -- 172.21.15.17:0/1557320095 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684031b80 con 0x7f469806c610 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 -- 172.21.15.17:0/1557320095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 msgr2=0x7f46980aab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 --2- 172.21.15.17:0/1557320095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46980aab80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4684009870 tx=0x7f468402b860 comp rx=0 tx=0).stop 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 -- 172.21.15.17:0/1557320095 shutdown_connections 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 --2- 172.21.15.17:0/1557320095 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46980aab80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 --2- 172.21.15.17:0/1557320095 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f469806c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 -- 172.21.15.17:0/1557320095 >> 172.21.15.17:0/1557320095 conn(0x7f46980c3fe0 msgr2=0x7f46980c43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 -- 172.21.15.17:0/1557320095 shutdown_connections 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.811+0000 7f469dda2640 1 -- 172.21.15.17:0/1557320095 wait complete. 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.812+0000 7f469dda2640 1 Processor -- start 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.812+0000 7f469dda2640 1 -- start start 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46980c23f0 0x7f469814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f4698151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980ab620 con 0x7f469806c610 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980c3310 con 0x7f46980c23f0 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469dda2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980ab0c0 con 0x7f469814f4d0 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52452/0 (socket says 172.21.15.17:52452) 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 -- 172.21.15.17:0/1785882225 learned_addr learned my addr 172.21.15.17:0/1785882225 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f4697fff640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46980c23f0 0x7f469814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469d5a1640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f4698151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 -- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 msgr2=0x7f4698151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f4698151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 -- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46980c23f0 msgr2=0x7f469814b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46980c23f0 0x7f469814b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469cda0640 1 -- 172.21.15.17:0/1785882225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469814a680 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.813+0000 7f469d5a1640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f4698151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f469cda0640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b410 tx=0x7f468c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c015020 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c011850 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46980a9c80 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c011c40 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.814+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469814b860 con 0x7f469806c610 2024-03-06T07:20:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.815+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469814ad00 con 0x7f469806c610 2024-03-06T07:20:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.815+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f468c018020 con 0x7f469806c610 2024-03-06T07:20:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.816+0000 7f46957fa640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 0x7f467c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.816+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f468c089ce0 con 0x7f469806c610 2024-03-06T07:20:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.816+0000 7f4697fff640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 0x7f467c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:01.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.816+0000 7f4697fff640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 0x7f467c06d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4684036000 tx=0x7f468403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:01.820+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f469814ad00 con 0x7f469806c610 2024-03-06T07:20:02.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.001+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46980693c0 con 0x7f469806c610 2024-03-06T07:20:02.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.693+0000 7f46957fa640 1 -- 172.21.15.17:0/1785882225 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f468c01f020 con 0x7f469806c610 2024-03-06T07:20:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.696+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 msgr2=0x7f467c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.696+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 0x7f467c06d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4684036000 tx=0x7f468403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.696+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 msgr2=0x7f46981ec690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.696+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f468c00b410 tx=0x7f468c00b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 shutdown_connections 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f467c06b500 0x7f467c06d9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469814f4d0 0x7f4698151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46980c23f0 0x7f469814b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 --2- 172.21.15.17:0/1785882225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f469806c610 0x7f46981ec690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 >> 172.21.15.17:0/1785882225 conn(0x7f46980c3fe0 msgr2=0x7f46980c43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 shutdown_connections 2024-03-06T07:20:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.697+0000 7f469dda2640 1 -- 172.21.15.17:0/1785882225 wait complete. 2024-03-06T07:20:02.716 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-06T07:20:02.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.827+0000 7fbd7bd5c640 1 Processor -- start 2024-03-06T07:20:02.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.827+0000 7fbd7bd5c640 1 -- start start 2024-03-06T07:20:02.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd7bd5c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd740b0070 0x7fbd74155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd7bd5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd7bd5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd740b0450 con 0x7fbd740c5380 2024-03-06T07:20:02.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd7bd5c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7406c020 con 0x7fbd740b0070 2024-03-06T07:20:02.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd7bd5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406cf20 con 0x7fbd7406ca70 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd792f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd79af8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd740b0070 0x7fbd74155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd792f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:41574/0 (socket says 172.21.15.17:41574) 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.828+0000 7fbd792f7640 1 -- 172.21.15.17:0/3981422919 learned_addr learned my addr 172.21.15.17:0/3981422919 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.829+0000 7fbd79af8640 1 -- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 msgr2=0x7fbd740c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.829+0000 7fbd79af8640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.829+0000 7fbd79af8640 1 -- 172.21.15.17:0/3981422919 >> v1:172.21.15.163:6789/0 conn(0x7fbd740c5380 legacy=0x7fbd7414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.829+0000 7fbd79af8640 1 -- 172.21.15.17:0/3981422919 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd740c77f0 con 0x7fbd740b0070 2024-03-06T07:20:02.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.829+0000 7fbd792f7640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:02.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd79af8640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd740b0070 0x7fbd74155c30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097a0 tx=0x7fbd7002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e6b0338ad9921e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd78af6640 1 -- 172.21.15.17:0/3981422919 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd700315a0 con 0x7fbd740b0070 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd78af6640 1 -- 172.21.15.17:0/3981422919 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd70037070 con 0x7fbd740b0070 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd78af6640 1 -- 172.21.15.17:0/3981422919 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031960 con 0x7fbd740b0070 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd740b0070 msgr2=0x7fbd74155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd740b0070 0x7fbd74155c30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbd700097a0 tx=0x7fbd7002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/3981422919 shutdown_connections 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7406ca70 0x7fbd740c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/3981422919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd740b0070 0x7fbd74155c30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/3981422919 >> 172.21.15.17:0/3981422919 conn(0x7fbd740ac590 msgr2=0x7fbd740ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:02.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/3981422919 shutdown_connections 2024-03-06T07:20:02.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.830+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/3981422919 wait complete. 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.831+0000 7fbd7bd5c640 1 Processor -- start 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 -- start start 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7406ca70 0x7fbd74157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd740c5380 0x7fbd7414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406c550 con 0x7fbd740b0070 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406c270 con 0x7fbd7406ca70 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7bd5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7406da80 con 0x7fbd740c5380 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd792f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd792f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52468/0 (socket says 172.21.15.17:52468) 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd792f7640 1 -- 172.21.15.17:0/941501531 learned_addr learned my addr 172.21.15.17:0/941501531 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd79af8640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7406ca70 0x7fbd74157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.832+0000 7fbd7a2f9640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd740c5380 0x7fbd7414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 -- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd740c5380 msgr2=0x7fbd7414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd740c5380 0x7fbd7414df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 -- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7406ca70 msgr2=0x7fbd74157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7406ca70 0x7fbd74157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 -- 172.21.15.17:0/941501531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd741eb4c0 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd792f7640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9e0 tx=0x7fbd6400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64017070 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400c9d0 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd74152fa0 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400cd80 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.833+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd740bddc0 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.834+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7404f330 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.835+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd6401a070 con 0x7fbd740b0070 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.835+0000 7fbd6a7fc640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:02.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.835+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbd6408a1e0 con 0x7fbd740b0070 2024-03-06T07:20:02.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.835+0000 7fbd79af8640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:02.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.836+0000 7fbd79af8640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 0x7fbd5806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036000 tx=0x7fbd7003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:02.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:02.839+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd640592d0 con 0x7fbd740b0070 2024-03-06T07:20:03.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.020+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fbd7406d860 con 0x7fbd740b0070 2024-03-06T07:20:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.700+0000 7fbd6a7fc640 1 -- 172.21.15.17:0/941501531 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fbd64058bd0 con 0x7fbd740b0070 2024-03-06T07:20:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:20:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 msgr2=0x7fbd5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 0x7fbd5806d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036000 tx=0x7fbd7003d040 comp rx=0 tx=0).stop 2024-03-06T07:20:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 msgr2=0x7fbd74151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9e0 tx=0x7fbd6400eea0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 shutdown_connections 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd5806b4d0 0x7fbd5806d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd740c5380 0x7fbd7414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd740b0070 0x7fbd74151410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.705+0000 7fbd7bd5c640 1 --2- 172.21.15.17:0/941501531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7406ca70 0x7fbd74157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.706+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 >> 172.21.15.17:0/941501531 conn(0x7fbd740ac590 msgr2=0x7fbd740c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:03.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.708+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 shutdown_connections 2024-03-06T07:20:03.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.710+0000 7fbd7bd5c640 1 -- 172.21.15.17:0/941501531 wait complete. 2024-03-06T07:20:03.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:20:03.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.842+0000 7fb2b7688640 1 Processor -- start 2024-03-06T07:20:03.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.842+0000 7fb2b7688640 1 -- start start 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b7688640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb2b006da40 0x7fb2b0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b7688640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b0156050 con 0x7fb2b00c7380 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b7688640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b0072630 con 0x7fb2b00c6780 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b7688640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b006de20 con 0x7fb2b006da40 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5424640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb2b006da40 0x7fb2b0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5c25640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb2b00c7380 0x7fb2b00c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59988/0 (socket says 172.21.15.17:59988) 2024-03-06T07:20:03.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5c25640 1 -- 172.21.15.17:0/843207898 learned_addr learned my addr 172.21.15.17:0/843207898 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5424640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb2b006da40 0x7fb2b0155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52482/0 (socket says 172.21.15.17:52482) 2024-03-06T07:20:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5424640 1 -- 172.21.15.17:0/843207898 >> v1:172.21.15.163:6789/0 conn(0x7fb2b00c6780 legacy=0x7fb2b0158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5424640 1 -- 172.21.15.17:0/843207898 >> v1:172.21.15.17:6789/0 conn(0x7fb2b00c7380 legacy=0x7fb2b00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2b5424640 1 -- 172.21.15.17:0/843207898 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0072910 con 0x7fb2b006da40 2024-03-06T07:20:03.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.843+0000 7fb2a7fff640 1 -- 172.21.15.17:0/843207898 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826818523 0 0) 0x7fb2ac001490 con 0x7fb2b00c7380 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.844+0000 7fb2b5424640 1 --2- 172.21.15.17:0/843207898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b006da40 0x7fb2b0155b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80097a0 tx=0x7fb2a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c94266112a1ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.844+0000 7fb2a7fff640 1 -- 172.21.15.17:0/843207898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8035020 con 0x7fb2b006da40 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.844+0000 7fb2a7fff640 1 -- 172.21.15.17:0/843207898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8031830 con 0x7fb2b006da40 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.844+0000 7fb2a7fff640 1 -- 172.21.15.17:0/843207898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8031c00 con 0x7fb2b006da40 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 -- 172.21.15.17:0/843207898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b006da40 msgr2=0x7fb2b0155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 --2- 172.21.15.17:0/843207898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b006da40 0x7fb2b0155b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80097a0 tx=0x7fb2a802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 -- 172.21.15.17:0/843207898 shutdown_connections 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 --2- 172.21.15.17:0/843207898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b006da40 0x7fb2b0155b10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 -- 172.21.15.17:0/843207898 >> 172.21.15.17:0/843207898 conn(0x7fb2b00aa630 msgr2=0x7fb2b00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:03.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 -- 172.21.15.17:0/843207898 shutdown_connections 2024-03-06T07:20:03.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.845+0000 7fb2b7688640 1 -- 172.21.15.17:0/843207898 wait complete. 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.846+0000 7fb2b7688640 1 Processor -- start 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.846+0000 7fb2b7688640 1 -- start start 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 0x7fb2b01de880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 0x7fb2b0152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b01594a0 con 0x7fb2b00c7380 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b0158890 con 0x7fb2b006da40 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b00c3990 con 0x7fb2b00c6780 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52490/0 (socket says 172.21.15.17:52490) 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 -- 172.21.15.17:0/1385639804 learned_addr learned my addr 172.21.15.17:0/1385639804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:03.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5424640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 0x7fb2b01de880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b4c23640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 0x7fb2b0152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 -- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 msgr2=0x7fb2b0152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 0x7fb2b0152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 -- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 msgr2=0x7fb2b01de880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 0x7fb2b01de880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 -- 172.21.15.17:0/1385639804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0153440 con 0x7fb2b00c7380 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b4c23640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 0x7fb2b0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5424640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 0x7fb2b01de880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b5c25640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac001730 tx=0x7fb2ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac018070 con 0x7fb2b00c7380 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac004d80 con 0x7fb2b00c7380 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b0151d30 con 0x7fb2b00c7380 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.847+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00a880 con 0x7fb2b00c7380 2024-03-06T07:20:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.848+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b0151a50 con 0x7fb2b00c7380 2024-03-06T07:20:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.849+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b004f330 con 0x7fb2b00c7380 2024-03-06T07:20:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.849+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2ac01b070 con 0x7fb2b00c7380 2024-03-06T07:20:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.849+0000 7fb2a5ffb640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.849+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb2ac089600 con 0x7fb2b00c7380 2024-03-06T07:20:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.849+0000 7fb2b5424640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.850+0000 7fb2b5424640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8036000 tx=0x7fb2a803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:03.854+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb2b0156050 con 0x7fb2b00c7380 2024-03-06T07:20:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.073+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb2b0152780 con 0x7fb2b00c7380 2024-03-06T07:20:04.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.702+0000 7fb2a5ffb640 1 -- 172.21.15.17:0/1385639804 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb2ac058570 con 0x7fb2b00c7380 2024-03-06T07:20:04.705 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2024-03-06T07:20:04.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.706+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 msgr2=0x7fb29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.706+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8036000 tx=0x7fb2a803c040 comp rx=0 tx=0).stop 2024-03-06T07:20:04.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.706+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 msgr2=0x7fb2b0151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.706+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac001730 tx=0x7fb2ac00cb70 comp rx=0 tx=0).stop 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 shutdown_connections 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2b00c7380 0x7fb2b0151180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2b00c6780 0x7fb2b0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 --2- 172.21.15.17:0/1385639804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2b006da40 0x7fb2b01de880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 >> 172.21.15.17:0/1385639804 conn(0x7fb2b00aa630 msgr2=0x7fb2b00a84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:04.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 shutdown_connections 2024-03-06T07:20:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.707+0000 7fb2b7688640 1 -- 172.21.15.17:0/1385639804 wait complete. 2024-03-06T07:20:04.739 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-06T07:20:04.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-06T07:20:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.846+0000 7fa492b93640 1 Processor -- start 2024-03-06T07:20:04.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.846+0000 7fa492b93640 1 -- start start 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa492b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa492b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c1566c0 0x7fa48c06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa492b93640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c0c8f60 con 0x7fa48c1551d0 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa492b93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c0c9240 con 0x7fa48c1566c0 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa492b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0c9520 con 0x7fa48c14f180 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c1566c0 0x7fa48c06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa48c1566c0 0x7fa48c06c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52492/0 (socket says 172.21.15.17:52492) 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 -- 172.21.15.17:0/3727477713 learned_addr learned my addr 172.21.15.17:0/3727477713 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:04.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa49092f640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 -- 172.21.15.17:0/3727477713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 msgr2=0x7fa48c0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c0c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 -- 172.21.15.17:0/3727477713 >> v1:172.21.15.163:6789/0 conn(0x7fa48c1551d0 legacy=0x7fa48c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.847+0000 7fa491130640 1 -- 172.21.15.17:0/3727477713 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0c9800 con 0x7fa48c1566c0 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.848+0000 7fa491130640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c06c990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa48402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf5086b7824d67f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.848+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3727477713 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48402cb00 con 0x7fa48c1566c0 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.848+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3727477713 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48402cca0 con 0x7fa48c1566c0 2024-03-06T07:20:04.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.848+0000 7fa48b7fe640 1 -- 172.21.15.17:0/3727477713 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484034610 con 0x7fa48c1566c0 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 -- 172.21.15.17:0/3727477713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 msgr2=0x7fa48c06c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c06c990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa48402bf60 comp rx=0 tx=0).stop 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 -- 172.21.15.17:0/3727477713 shutdown_connections 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c06c990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 --2- 172.21.15.17:0/3727477713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 -- 172.21.15.17:0/3727477713 >> 172.21.15.17:0/3727477713 conn(0x7fa48c072d20 msgr2=0x7fa48c0ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 -- 172.21.15.17:0/3727477713 shutdown_connections 2024-03-06T07:20:04.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.849+0000 7fa492b93640 1 -- 172.21.15.17:0/3727477713 wait complete. 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.850+0000 7fa492b93640 1 Processor -- start 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.850+0000 7fa492b93640 1 -- start start 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c1551d0 0x7fa48c147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c14f560 con 0x7fa48c1566c0 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c156b50 con 0x7fa48c1551d0 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa492b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c14f6e0 con 0x7fa48c14f180 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52502/0 (socket says 172.21.15.17:52502) 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 -- 172.21.15.17:0/3875594965 learned_addr learned my addr 172.21.15.17:0/3875594965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa48bfff640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c1551d0 0x7fa48c147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa49092f640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 -- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 msgr2=0x7fa48c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 -- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c1551d0 msgr2=0x7fa48c147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c1551d0 0x7fa48c147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 -- 172.21.15.17:0/3875594965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c1ee810 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.851+0000 7fa491130640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.852+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484043020 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.852+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48402ce50 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.852+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c1eeaf0 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.852+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484009510 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.852+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1585b0 con 0x7fa48c1566c0 2024-03-06T07:20:04.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.853+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c1587f0 con 0x7fa48c1566c0 2024-03-06T07:20:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.853+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa484046020 con 0x7fa48c1566c0 2024-03-06T07:20:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.853+0000 7fa4897fa640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.854+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4840ac670 con 0x7fa48c1566c0 2024-03-06T07:20:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.854+0000 7fa49092f640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.854+0000 7fa49092f640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 0x7fa47806d990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa47c006fd0 tx=0x7fa47c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:04.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:04.858+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa48404d030 con 0x7fa48c1566c0 2024-03-06T07:20:05.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.051+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa48c04f330 con 0x7fa48c1566c0 2024-03-06T07:20:05.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.719+0000 7fa4897fa640 1 -- 172.21.15.17:0/3875594965 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa48407af30 con 0x7fa48c1566c0 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 msgr2=0x7fa47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 0x7fa47806d990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa47c006fd0 tx=0x7fa47c008040 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 msgr2=0x7fa48c1484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa484004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 shutdown_connections 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa47806b4d0 0x7fa47806d990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa48c1566c0 0x7fa48c1484f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa48c1551d0 0x7fa48c147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 --2- 172.21.15.17:0/3875594965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa48c14f180 0x7fa48c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 >> 172.21.15.17:0/3875594965 conn(0x7fa48c072d20 msgr2=0x7fa48c0adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 shutdown_connections 2024-03-06T07:20:05.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.723+0000 7fa492b93640 1 -- 172.21.15.17:0/3875594965 wait complete. 2024-03-06T07:20:05.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:20:05.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.852+0000 7fbb7cf56640 1 Processor -- start 2024-03-06T07:20:05.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.852+0000 7fbb7cf56640 1 -- start start 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7cf56640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb780b01e0 0x7fbb78155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7cf56640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb781565b0 0x7fbb780c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7cf56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb780c7e00 con 0x7fbb780c5910 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7cf56640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780c7840 con 0x7fbb781565b0 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7cf56640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780c7b20 con 0x7fbb780b01e0 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb75d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbb781565b0 0x7fbb780c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb76d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbb780c5910 0x7fbb7806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59996/0 (socket says 172.21.15.17:59996) 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb76d9d640 1 -- 172.21.15.17:0/2605666540 learned_addr learned my addr 172.21.15.17:0/2605666540 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7659c640 1 --2- 172.21.15.17:0/2605666540 >> v2:172.21.15.163:3300/0 conn(0x7fbb780b01e0 0x7fbb78155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb75d9b640 1 -- 172.21.15.17:0/2605666540 >> v2:172.21.15.163:3300/0 conn(0x7fbb780b01e0 msgr2=0x7fbb78155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb75d9b640 1 --2- 172.21.15.17:0/2605666540 >> v2:172.21.15.163:3300/0 conn(0x7fbb780b01e0 0x7fbb78155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb75d9b640 1 -- 172.21.15.17:0/2605666540 >> v1:172.21.15.17:6789/0 conn(0x7fbb780c5910 legacy=0x7fbb7806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb7559a640 1 -- 172.21.15.17:0/2605666540 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125762569 0 0) 0x7fbb780c7e00 con 0x7fbb780c5910 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.853+0000 7fbb75d9b640 1 -- 172.21.15.17:0/2605666540 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7806b970 con 0x7fbb781565b0 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.854+0000 7fbb75d9b640 1 --2- 172.21.15.17:0/2605666540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb781565b0 0x7fbb780c53d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb78073630 tx=0x7fbb6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d38c52d0290d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.854+0000 7fbb7559a640 1 -- 172.21.15.17:0/2605666540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60035020 con 0x7fbb781565b0 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.854+0000 7fbb7559a640 1 -- 172.21.15.17:0/2605666540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb60031830 con 0x7fbb781565b0 2024-03-06T07:20:05.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.854+0000 7fbb7559a640 1 -- 172.21.15.17:0/2605666540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60031c00 con 0x7fbb781565b0 2024-03-06T07:20:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 -- 172.21.15.17:0/2605666540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb781565b0 msgr2=0x7fbb780c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/2605666540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb781565b0 0x7fbb780c53d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb78073630 tx=0x7fbb6002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 -- 172.21.15.17:0/2605666540 shutdown_connections 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/2605666540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb781565b0 0x7fbb780c53d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/2605666540 >> v2:172.21.15.163:3300/0 conn(0x7fbb780b01e0 0x7fbb78155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 -- 172.21.15.17:0/2605666540 >> 172.21.15.17:0/2605666540 conn(0x7fbb780ac600 msgr2=0x7fbb780aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 -- 172.21.15.17:0/2605666540 shutdown_connections 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.855+0000 7fbb7cf56640 1 -- 172.21.15.17:0/2605666540 wait complete. 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.856+0000 7fbb7cf56640 1 Processor -- start 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.856+0000 7fbb7cf56640 1 -- start start 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb780c5910 0x7fbb780c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781565b0 0x7fbb780c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780be3d0 con 0x7fbb780b01e0 2024-03-06T07:20:05.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780be250 con 0x7fbb781565b0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7cf56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780c7460 con 0x7fbb780c5910 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52512/0 (socket says 172.21.15.17:52512) 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 -- 172.21.15.17:0/333799709 learned_addr learned my addr 172.21.15.17:0/333799709 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb76d9d640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781565b0 0x7fbb780c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb75d9b640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb780c5910 0x7fbb780c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 -- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb780c5910 msgr2=0x7fbb780c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb780c5910 0x7fbb780c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 -- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781565b0 msgr2=0x7fbb780c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781565b0 0x7fbb780c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 -- 172.21.15.17:0/333799709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780c2bd0 con 0x7fbb780b01e0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb7659c640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800efc0 tx=0x7fbb6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.857+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68018020 con 0x7fbb780b01e0 2024-03-06T07:20:05.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.858+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68014480 con 0x7fbb780b01e0 2024-03-06T07:20:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.858+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68014850 con 0x7fbb780b01e0 2024-03-06T07:20:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.858+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb780ca690 con 0x7fbb780b01e0 2024-03-06T07:20:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.858+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb780cac50 con 0x7fbb780b01e0 2024-03-06T07:20:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.859+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7804f330 con 0x7fbb780b01e0 2024-03-06T07:20:05.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.859+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb68005d10 con 0x7fbb780b01e0 2024-03-06T07:20:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.859+0000 7fbb6effd640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 0x7fbb5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.859+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fbb6808ac50 con 0x7fbb780b01e0 2024-03-06T07:20:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.860+0000 7fbb75d9b640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 0x7fbb5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.860+0000 7fbb75d9b640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 0x7fbb5806d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb60036000 tx=0x7fbb60008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:05.864+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb68090020 con 0x7fbb780b01e0 2024-03-06T07:20:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:06.044+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbb78158100 con 0x7fbb780b01e0 2024-03-06T07:20:07.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.734+0000 7fbb6effd640 1 -- 172.21.15.17:0/333799709 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbb68022070 con 0x7fbb780b01e0 2024-03-06T07:20:07.737 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' created 2024-03-06T07:20:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 msgr2=0x7fbb5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 0x7fbb5806d990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb60036000 tx=0x7fbb60008030 comp rx=0 tx=0).stop 2024-03-06T07:20:07.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 msgr2=0x7fbb780be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800efc0 tx=0x7fbb6800c490 comp rx=0 tx=0).stop 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 shutdown_connections 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb5806b4d0 0x7fbb5806d990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb781565b0 0x7fbb780c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb780c5910 0x7fbb780c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 --2- 172.21.15.17:0/333799709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb780b01e0 0x7fbb780be8a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.739+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 >> 172.21.15.17:0/333799709 conn(0x7fbb780ac600 msgr2=0x7fbb781578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.740+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 shutdown_connections 2024-03-06T07:20:07.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.740+0000 7fbb7cf56640 1 -- 172.21.15.17:0/333799709 wait complete. 2024-03-06T07:20:07.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-06T07:20:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.868+0000 7f3296478640 1 Processor -- start 2024-03-06T07:20:07.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.868+0000 7f3296478640 1 -- start start 2024-03-06T07:20:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3296478640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3290152900 0x7f32900c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3296478640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32900c58c0 0x7f3290155900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3296478640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32900c9140 con 0x7f32900c8900 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3296478640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32900c6bc0 con 0x7f3290152900 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3296478640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32901588a0 con 0x7f32900c58c0 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f328ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3290152900 0x7f32900c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f328ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3290152900 0x7f32900c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52514/0 (socket says 172.21.15.17:52514) 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f328ffff640 1 -- 172.21.15.17:0/774653772 learned_addr learned my addr 172.21.15.17:0/774653772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.869+0000 7f3294a15640 1 --2- 172.21.15.17:0/774653772 >> v2:172.21.15.163:3300/0 conn(0x7f32900c58c0 0x7f3290155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328ffff640 1 -- 172.21.15.17:0/774653772 >> v2:172.21.15.163:3300/0 conn(0x7f32900c58c0 msgr2=0x7f3290155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328ffff640 1 --2- 172.21.15.17:0/774653772 >> v2:172.21.15.163:3300/0 conn(0x7f32900c58c0 0x7f3290155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328ffff640 1 -- 172.21.15.17:0/774653772 >> v1:172.21.15.163:6789/0 conn(0x7f32900c8900 legacy=0x7f32901551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328ffff640 1 -- 172.21.15.17:0/774653772 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3290158b80 con 0x7f3290152900 2024-03-06T07:20:07.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328ffff640 1 --2- 172.21.15.17:0/774653772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3290152900 0x7f32900c5380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32900c7310 tx=0x7f328002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f24dce83db3ead7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.870+0000 7f328effd640 1 -- 172.21.15.17:0/774653772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280035020 con 0x7f3290152900 2024-03-06T07:20:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f328effd640 1 -- 172.21.15.17:0/774653772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280031870 con 0x7f3290152900 2024-03-06T07:20:07.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f328effd640 1 -- 172.21.15.17:0/774653772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280031c40 con 0x7f3290152900 2024-03-06T07:20:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 -- 172.21.15.17:0/774653772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3290152900 msgr2=0x7f32900c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 --2- 172.21.15.17:0/774653772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3290152900 0x7f32900c5380 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32900c7310 tx=0x7f328002b910 comp rx=0 tx=0).stop 2024-03-06T07:20:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 -- 172.21.15.17:0/774653772 shutdown_connections 2024-03-06T07:20:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 --2- 172.21.15.17:0/774653772 >> v2:172.21.15.163:3300/0 conn(0x7f32900c58c0 0x7f3290155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 --2- 172.21.15.17:0/774653772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3290152900 0x7f32900c5380 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 -- 172.21.15.17:0/774653772 >> 172.21.15.17:0/774653772 conn(0x7f32900728d0 msgr2=0x7f3290072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:07.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 -- 172.21.15.17:0/774653772 shutdown_connections 2024-03-06T07:20:07.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.871+0000 7f3296478640 1 -- 172.21.15.17:0/774653772 wait complete. 2024-03-06T07:20:07.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.872+0000 7f3296478640 1 Processor -- start 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.872+0000 7f3296478640 1 -- start start 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32900c58c0 0x7f32900c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3290152900 0x7f3290147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329006cba0 con 0x7f32900c8900 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3290159660 con 0x7f3290152900 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3296478640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900c8e10 con 0x7f32900c58c0 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52520/0 (socket says 172.21.15.17:52520) 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 -- 172.21.15.17:0/2632927680 learned_addr learned my addr 172.21.15.17:0/2632927680 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f3294a15640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3290152900 0x7f3290147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328ffff640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32900c58c0 0x7f32900c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 -- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32900c58c0 msgr2=0x7f32900c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32900c58c0 0x7f32900c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 -- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3290152900 msgr2=0x7f3290147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3290152900 0x7f3290147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.873+0000 7f328f7fe640 1 -- 172.21.15.17:0/2632927680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900acee0 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f328f7fe640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f328400c400 tx=0x7f328400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32840052d0 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284005490 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32900ad1c0 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284007260 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.874+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32900c35f0 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.875+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32900681e0 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.875+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3284007400 con 0x7f32900c8900 2024-03-06T07:20:07.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.876+0000 7f328cff9640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 0x7f327c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:07.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.876+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f32840898f0 con 0x7f32900c8900 2024-03-06T07:20:07.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.876+0000 7f328ffff640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 0x7f327c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:07.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.876+0000 7f328ffff640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 0x7f327c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3280036000 tx=0x7f328003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:07.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:07.880+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f32900681e0 con 0x7f32900c8900 2024-03-06T07:20:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.060+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f329004f330 con 0x7f32900c8900 2024-03-06T07:20:08.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:08.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:08.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.738+0000 7f328cff9640 1 -- 172.21.15.17:0/2632927680 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f32900c9140 con 0x7f32900c8900 2024-03-06T07:20:08.741 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.741+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 msgr2=0x7f327c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.741+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 0x7f327c06d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3280036000 tx=0x7f328003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.742+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 msgr2=0x7f3290147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.742+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f328400c400 tx=0x7f328400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.742+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 shutdown_connections 2024-03-06T07:20:08.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.742+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f327c06b4d0 0x7f327c06d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.743+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3290152900 0x7f3290147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.743+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32900c8900 0x7f3290147770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.743+0000 7f3296478640 1 --2- 172.21.15.17:0/2632927680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32900c58c0 0x7f32900c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.743+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 >> 172.21.15.17:0/2632927680 conn(0x7f32900728d0 msgr2=0x7f3290155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:08.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.749+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 shutdown_connections 2024-03-06T07:20:08.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.750+0000 7f3296478640 1 -- 172.21.15.17:0/2632927680 wait complete. 2024-03-06T07:20:08.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-06T07:20:08.781 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:20:08.780+0000 7f2d3f13e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:20:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.880+0000 7fb4fa1b6640 1 Processor -- start 2024-03-06T07:20:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.880+0000 7fb4fa1b6640 1 -- start start 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4fa1b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40b01e0 0x7fb4f4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4fa1b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4f41565b0 0x7fb4f40c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4fa1b6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f40c7e00 con 0x7fb4f40c5910 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4fa1b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40c7840 con 0x7fb4f41565b0 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4fa1b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c7b20 con 0x7fb4f40b01e0 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4f41565b0 0x7fb4f40c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4f41565b0 0x7fb4f40c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52532/0 (socket says 172.21.15.17:52532) 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 -- 172.21.15.17:0/2743141723 learned_addr learned my addr 172.21.15.17:0/2743141723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f37fe640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40b01e0 0x7fb4f4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 -- 172.21.15.17:0/2743141723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40b01e0 msgr2=0x7fb4f4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40b01e0 0x7fb4f4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 -- 172.21.15.17:0/2743141723 >> v1:172.21.15.163:6789/0 conn(0x7fb4f40c5910 legacy=0x7fb4f406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.881+0000 7fb4f2ffd640 1 -- 172.21.15.17:0/2743141723 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f406b970 con 0x7fb4f41565b0 2024-03-06T07:20:08.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.882+0000 7fb4f2ffd640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f41565b0 0x7fb4f40c53d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009870 tx=0x7fb4e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a98746eb4090a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.882+0000 7fb4f27fc640 1 -- 172.21.15.17:0/2743141723 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8034070 con 0x7fb4f41565b0 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.882+0000 7fb4f27fc640 1 -- 172.21.15.17:0/2743141723 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8030840 con 0x7fb4f41565b0 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.882+0000 7fb4f27fc640 1 -- 172.21.15.17:0/2743141723 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8030c10 con 0x7fb4f41565b0 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2743141723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f41565b0 msgr2=0x7fb4f40c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f41565b0 0x7fb4f40c53d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009870 tx=0x7fb4e802b980 comp rx=0 tx=0).stop 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2743141723 shutdown_connections 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f41565b0 0x7fb4f40c53d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2743141723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40b01e0 0x7fb4f4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2743141723 >> 172.21.15.17:0/2743141723 conn(0x7fb4f40ac600 msgr2=0x7fb4f40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2743141723 shutdown_connections 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.883+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2743141723 wait complete. 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.884+0000 7fb4fa1b6640 1 Processor -- start 2024-03-06T07:20:08.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.884+0000 7fb4fa1b6640 1 -- start start 2024-03-06T07:20:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40c5910 0x7fb4f414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f41565b0 0x7fb4f41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40be3d0 con 0x7fb4f40b01e0 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40be250 con 0x7fb4f41565b0 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c7460 con 0x7fb4f40c5910 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52544/0 (socket says 172.21.15.17:52544) 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 -- 172.21.15.17:0/2362175667 learned_addr learned my addr 172.21.15.17:0/2362175667 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f2ffd640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40c5910 0x7fb4f414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f3fff640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f41565b0 0x7fb4f41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 -- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40c5910 msgr2=0x7fb4f414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40c5910 0x7fb4f414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 -- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f41565b0 msgr2=0x7fb4f41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f41565b0 0x7fb4f41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 -- 172.21.15.17:0/2362175667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f40cb340 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4f37fe640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e8a0 tx=0x7fb4e400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017070 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e400c940 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.885+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f41ebb50 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.886+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400cd30 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.886+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f41ebe30 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.886+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f404f330 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.887+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4e401a070 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.887+0000 7fb4dbfff640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.887+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb4e408a610 con 0x7fb4f40b01e0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.887+0000 7fb4f2ffd640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.888+0000 7fb4f2ffd640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 0x7fb4e006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8036000 tx=0x7fb4e803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:08.892+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb4e4058bb0 con 0x7fb4f40b01e0 2024-03-06T07:20:09.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.112+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb4f406cef0 con 0x7fb4f40b01e0 2024-03-06T07:20:09.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.749+0000 7fb4dbfff640 1 -- 172.21.15.17:0/2362175667 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb4e405cc50 con 0x7fb4f40b01e0 2024-03-06T07:20:09.752 INFO:teuthology.orchestra.run.smithi017.stderr:added data pool 4 to fsmap 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 msgr2=0x7fb4e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 0x7fb4e006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8036000 tx=0x7fb4e803b040 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 msgr2=0x7fb4f414d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e8a0 tx=0x7fb4e400ed60 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 shutdown_connections 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb4e006b4d0 0x7fb4e006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4f41565b0 0x7fb4f41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4f40c5910 0x7fb4f414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 --2- 172.21.15.17:0/2362175667 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4f40b01e0 0x7fb4f414d840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 >> 172.21.15.17:0/2362175667 conn(0x7fb4f40ac600 msgr2=0x7fb4f4157570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 shutdown_connections 2024-03-06T07:20:09.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.752+0000 7fb4fa1b6640 1 -- 172.21.15.17:0/2362175667 wait complete. 2024-03-06T07:20:09.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:09.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.880+0000 7f2225266640 1 Processor -- start 2024-03-06T07:20:09.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.880+0000 7f2225266640 1 -- start start 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f2225266640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f222006c240 0x7f22200c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f2225266640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2220159210 0x7f2220152470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f2225266640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2220159670 con 0x7f22201529b0 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f2225266640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22201553a0 con 0x7f2220159210 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f2225266640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2220155680 con 0x7f222006c240 2024-03-06T07:20:09.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f221e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2220159210 0x7f2220152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f221f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f22201529b0 0x7f22200c19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60000/0 (socket says 172.21.15.17:60000) 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f221f7fe640 1 -- 172.21.15.17:0/1056351083 learned_addr learned my addr 172.21.15.17:0/1056351083 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.881+0000 7f221effd640 1 --2- 172.21.15.17:0/1056351083 >> v2:172.21.15.163:3300/0 conn(0x7f222006c240 0x7f22200c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221e7fc640 1 -- 172.21.15.17:0/1056351083 >> v2:172.21.15.163:3300/0 conn(0x7f222006c240 msgr2=0x7f22200c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221e7fc640 1 --2- 172.21.15.17:0/1056351083 >> v2:172.21.15.163:3300/0 conn(0x7f222006c240 0x7f22200c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221e7fc640 1 -- 172.21.15.17:0/1056351083 >> v1:172.21.15.17:6789/0 conn(0x7f22201529b0 legacy=0x7f22200c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221e7fc640 1 -- 172.21.15.17:0/1056351083 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220155960 con 0x7f2220159210 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221dffb640 1 -- 172.21.15.17:0/1056351083 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999570977 0 0) 0x7f2220159670 con 0x7f22201529b0 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221e7fc640 1 --2- 172.21.15.17:0/1056351083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220152470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2220068900 tx=0x7f220802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f67f7d1c7e087c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221dffb640 1 -- 172.21.15.17:0/1056351083 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22080315a0 con 0x7f2220159210 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221dffb640 1 -- 172.21.15.17:0/1056351083 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2208037070 con 0x7f2220159210 2024-03-06T07:20:09.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.882+0000 7f221dffb640 1 -- 172.21.15.17:0/1056351083 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208031960 con 0x7f2220159210 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 -- 172.21.15.17:0/1056351083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 msgr2=0x7f2220152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 --2- 172.21.15.17:0/1056351083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220152470 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2220068900 tx=0x7f220802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 -- 172.21.15.17:0/1056351083 shutdown_connections 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 --2- 172.21.15.17:0/1056351083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220152470 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 --2- 172.21.15.17:0/1056351083 >> v2:172.21.15.163:3300/0 conn(0x7f222006c240 0x7f22200c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 -- 172.21.15.17:0/1056351083 >> 172.21.15.17:0/1056351083 conn(0x7f22200c3ad0 msgr2=0x7f22200c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 -- 172.21.15.17:0/1056351083 shutdown_connections 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.883+0000 7f2225266640 1 -- 172.21.15.17:0/1056351083 wait complete. 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.884+0000 7f2225266640 1 Processor -- start 2024-03-06T07:20:09.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.884+0000 7f2225266640 1 -- start start 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f222006c240 0x7f2220124450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22201529b0 0x7f22201ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2220152e40 con 0x7f2220159210 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222006d7c0 con 0x7f222006c240 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f2225266640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222006c620 con 0x7f22201529b0 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52570/0 (socket says 172.21.15.17:52570) 2024-03-06T07:20:09.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 -- 172.21.15.17:0/1906225366 learned_addr learned my addr 172.21.15.17:0/1906225366 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221effd640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f222006c240 0x7f2220124450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221e7fc640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22201529b0 0x7f22201ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 -- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22201529b0 msgr2=0x7f22201ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22201529b0 0x7f22201ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 -- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f222006c240 msgr2=0x7f2220124450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f222006c240 0x7f2220124450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.885+0000 7f221f7fe640 1 -- 172.21.15.17:0/1906225366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220120d90 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f221f7fe640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f221000c810 tx=0x7f221000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210018070 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221000a640 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22201eed50 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221000aa30 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.886+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2220121300 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.887+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222004f330 con 0x7f2220159210 2024-03-06T07:20:09.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.887+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f221001b070 con 0x7f2220159210 2024-03-06T07:20:09.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.888+0000 7f22177fe640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 0x7f220006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:09.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.888+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f22100899b0 con 0x7f2220159210 2024-03-06T07:20:09.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.888+0000 7f221effd640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 0x7f220006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:09.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.888+0000 7f221effd640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 0x7f220006d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22201520c0 tx=0x7f2218009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:09.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:09.892+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2210057f30 con 0x7f2220159210 2024-03-06T07:20:10.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.111+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f222006dc80 con 0x7f2220159210 2024-03-06T07:20:10.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.112+0000 7f22177fe640 1 -- 172.21.15.17:0/1906225366 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f221005b380 con 0x7f2220159210 2024-03-06T07:20:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:10.115 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:09.115148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:10.116 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-03-06T07:20:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 msgr2=0x7f220006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 0x7f220006d9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22201520c0 tx=0x7f2218009190 comp rx=0 tx=0).stop 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 msgr2=0x7f2220147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f221000c810 tx=0x7f221000ccd0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 shutdown_connections 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f220006b500 0x7f220006d9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2220159210 0x7f2220147520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22201529b0 0x7f22201ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 --2- 172.21.15.17:0/1906225366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f222006c240 0x7f2220124450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 >> 172.21.15.17:0/1906225366 conn(0x7f22200c3ad0 msgr2=0x7f22200c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 shutdown_connections 2024-03-06T07:20:10.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.114+0000 7f2225266640 1 -- 172.21.15.17:0/1906225366 wait complete. 2024-03-06T07:20:10.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.225+0000 7fea80113640 1 Processor -- start 2024-03-06T07:20:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.225+0000 7fea80113640 1 -- start start 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea80113640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea780c4a50 0x7fea78072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea80113640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780707c0 con 0x7fea78070c70 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea80113640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780c8220 con 0x7fea780c79c0 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea80113640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea78150a90 con 0x7fea780c4a50 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea7d6ae640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fea78070c70 0x7fea780c6880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60008/0 (socket says 172.21.15.17:60008) 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea7d6ae640 1 -- 172.21.15.17:0/3259749722 learned_addr learned my addr 172.21.15.17:0/3259749722 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.226+0000 7fea7deaf640 1 --2- 172.21.15.17:0/3259749722 >> v2:172.21.15.163:3300/0 conn(0x7fea780c4a50 0x7fea78072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677975145 0 0) 0x7fea780707c0 con 0x7fea78070c70 2024-03-06T07:20:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea60003520 con 0x7fea78070c70 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7deaf640 1 -- 172.21.15.17:0/3259749722 >> v1:172.21.15.163:6789/0 conn(0x7fea780c79c0 legacy=0x7fea780c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283908159 0 0) 0x7fea780c8220 con 0x7fea780c79c0 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7deaf640 1 -- 172.21.15.17:0/3259749722 >> v1:172.21.15.17:6789/0 conn(0x7fea78070c70 legacy=0x7fea780c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.227+0000 7fea7deaf640 1 -- 172.21.15.17:0/3259749722 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780a8280 con 0x7fea780c4a50 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea7deaf640 1 --2- 172.21.15.17:0/3259749722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c4a50 0x7fea78072570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea68009d90 tx=0x7fea6802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d39068fc0d7ab658 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68039070 con 0x7fea780c4a50 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6802cbd0 con 0x7fea780c4a50 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea7cead640 1 -- 172.21.15.17:0/3259749722 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68034440 con 0x7fea780c4a50 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 -- 172.21.15.17:0/3259749722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c4a50 msgr2=0x7fea78072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 --2- 172.21.15.17:0/3259749722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c4a50 0x7fea78072570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea68009d90 tx=0x7fea6802be20 comp rx=0 tx=0).stop 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 -- 172.21.15.17:0/3259749722 shutdown_connections 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 --2- 172.21.15.17:0/3259749722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c4a50 0x7fea78072570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 -- 172.21.15.17:0/3259749722 >> 172.21.15.17:0/3259749722 conn(0x7fea780aa6a0 msgr2=0x7fea780aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 -- 172.21.15.17:0/3259749722 shutdown_connections 2024-03-06T07:20:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.228+0000 7fea80113640 1 -- 172.21.15.17:0/3259749722 wait complete. 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.229+0000 7fea80113640 1 Processor -- start 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.229+0000 7fea80113640 1 -- start start 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c4a50 0x7fea7814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 0x7fea781e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780c4e30 con 0x7fea78070c70 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78072d40 con 0x7fea780c79c0 2024-03-06T07:20:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea80113640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea781571d0 con 0x7fea780c4a50 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52584/0 (socket says 172.21.15.17:52584) 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 -- 172.21.15.17:0/959334437 learned_addr learned my addr 172.21.15.17:0/959334437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7d6ae640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c4a50 0x7fea7814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7e6b0640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 0x7fea781e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 -- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c4a50 msgr2=0x7fea7814de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c4a50 0x7fea7814de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 -- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 msgr2=0x7fea781e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 0x7fea781e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7deaf640 1 -- 172.21.15.17:0/959334437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea781ed490 con 0x7fea78070c70 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.230+0000 7fea7e6b0640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 0x7fea781e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea7deaf640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea68009eb0 tx=0x7fea68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68039050 con 0x7fea78070c70 2024-03-06T07:20:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68045070 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea78151b20 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6803e830 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.231+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea781ec750 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.232+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea780aecc0 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.232+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea6803e9d0 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.232+0000 7fea767fc640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.233+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea680ad250 con 0x7fea78070c70 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.233+0000 7fea7d6ae640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.233+0000 7fea7d6ae640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 0x7fea6406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea780acb70 tx=0x7fea6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.237+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea780aecc0 con 0x7fea78070c70 2024-03-06T07:20:10.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.419+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea78152040 con 0x7fea78070c70 2024-03-06T07:20:10.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.420+0000 7fea767fc640 1 -- 172.21.15.17:0/959334437 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18527 (secure 0 0 0) 0x7fea6807bf40 con 0x7fea78070c70 2024-03-06T07:20:10.423 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:10.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:09.114944+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 msgr2=0x7fea6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 0x7fea6406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea780acb70 tx=0x7fea6c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 msgr2=0x7fea781ecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea68009eb0 tx=0x7fea68004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 shutdown_connections 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea6406b4d0 0x7fea6406d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c79c0 0x7fea781e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c4a50 0x7fea7814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 --2- 172.21.15.17:0/959334437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea78070c70 0x7fea781ecf50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 >> 172.21.15.17:0/959334437 conn(0x7fea780aa6a0 msgr2=0x7fea780c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 shutdown_connections 2024-03-06T07:20:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.422+0000 7fea80113640 1 -- 172.21.15.17:0/959334437 wait complete. 2024-03-06T07:20:10.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.534+0000 7fc8439e2640 1 Processor -- start 2024-03-06T07:20:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 -- start start 2024-03-06T07:20:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc83c155c20 0x7fc83c0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc83c0c17c0 0x7fc83c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83c06afc0 con 0x7fc83c152920 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c06b2a0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc8439e2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83c06b580 con 0x7fc83c155c20 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc841f7f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc83c0c17c0 0x7fc83c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc841f7f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc83c0c17c0 0x7fc83c15fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33202/0 (socket says 172.21.15.17:33202) 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc841f7f640 1 -- 172.21.15.17:0/2071397651 learned_addr learned my addr 172.21.15.17:0/2071397651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc84177e640 1 --1- 172.21.15.17:0/2071397651 >> v1:172.21.15.163:6789/0 conn(0x7fc83c152920 0x7fc83c0c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59720/0 (socket says 172.21.15.17:59720) 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.535+0000 7fc840f7d640 1 --2- 172.21.15.17:0/2071397651 >> v2:172.21.15.163:3300/0 conn(0x7fc83c155c20 0x7fc83c0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc841f7f640 1 -- 172.21.15.17:0/2071397651 >> v2:172.21.15.163:3300/0 conn(0x7fc83c155c20 msgr2=0x7fc83c0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc841f7f640 1 --2- 172.21.15.17:0/2071397651 >> v2:172.21.15.163:3300/0 conn(0x7fc83c155c20 0x7fc83c0c1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc841f7f640 1 -- 172.21.15.17:0/2071397651 >> v1:172.21.15.163:6789/0 conn(0x7fc83c152920 legacy=0x7fc83c0c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc833fff640 1 -- 172.21.15.17:0/2071397651 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244512464 0 0) 0x7fc83c06afc0 con 0x7fc83c152920 2024-03-06T07:20:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc841f7f640 1 -- 172.21.15.17:0/2071397651 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c06b860 con 0x7fc83c0c17c0 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc841f7f640 1 --2- 172.21.15.17:0/2071397651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c15fb20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc83c159830 tx=0x7fc83402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdad5da30b416c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.536+0000 7fc833fff640 1 -- 172.21.15.17:0/2071397651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83402cb20 con 0x7fc83c0c17c0 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc833fff640 1 -- 172.21.15.17:0/2071397651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83402ccc0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc833fff640 1 -- 172.21.15.17:0/2071397651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834034610 con 0x7fc83c0c17c0 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 -- 172.21.15.17:0/2071397651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 msgr2=0x7fc83c15fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2071397651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c15fb20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc83c159830 tx=0x7fc83402bf80 comp rx=0 tx=0).stop 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 -- 172.21.15.17:0/2071397651 shutdown_connections 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2071397651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c15fb20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2071397651 >> v2:172.21.15.163:3300/0 conn(0x7fc83c155c20 0x7fc83c0c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 -- 172.21.15.17:0/2071397651 >> 172.21.15.17:0/2071397651 conn(0x7fc83c0c4290 msgr2=0x7fc83c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 -- 172.21.15.17:0/2071397651 shutdown_connections 2024-03-06T07:20:10.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.537+0000 7fc8439e2640 1 -- 172.21.15.17:0/2071397651 wait complete. 2024-03-06T07:20:10.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.538+0000 7fc8439e2640 1 Processor -- start 2024-03-06T07:20:10.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.538+0000 7fc8439e2640 1 -- start start 2024-03-06T07:20:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83c152920 0x7fc83c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc83c155c20 0x7fc83c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0c28a0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c1560b0 con 0x7fc83c152920 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc8439e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83c0c2a20 con 0x7fc83c155c20 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33218/0 (socket says 172.21.15.17:33218) 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 -- 172.21.15.17:0/2027778750 learned_addr learned my addr 172.21.15.17:0/2027778750 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc841f7f640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc83c155c20 0x7fc83c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc840f7d640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83c152920 0x7fc83c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 -- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc83c155c20 msgr2=0x7fc83c158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc83c155c20 0x7fc83c158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 -- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83c152920 msgr2=0x7fc83c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83c152920 0x7fc83c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 -- 172.21.15.17:0/2027778750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c0aa8d0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.539+0000 7fc84177e640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c500 tx=0x7fc83800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.540+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838018070 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.540+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc838007580 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.540+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83c1eb950 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.540+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc838004a20 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.540+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83c072a70 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.541+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83c04f330 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.541+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc83801b070 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.541+0000 7fc831ffb640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.541+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc8380896c0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.541+0000 7fc840f7d640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.542+0000 7fc840f7d640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 0x7fc82006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc83c14ed60 tx=0x7fc82c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.546+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc838057c40 con 0x7fc83c0c17c0 2024-03-06T07:20:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.726+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc83c068460 con 0x7fc83c0c17c0 2024-03-06T07:20:10.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.727+0000 7fc831ffb640 1 -- 172.21.15.17:0/2027778750 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18527 (secure 0 0 0) 0x7fc83805bce0 con 0x7fc83c0c17c0 2024-03-06T07:20:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:10.730 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":24,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:09.114944+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 msgr2=0x7fc82006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 0x7fc82006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc83c14ed60 tx=0x7fc82c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 msgr2=0x7fc83c14df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c500 tx=0x7fc83800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 shutdown_connections 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc82006b500 0x7fc82006d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc83c155c20 0x7fc83c158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83c152920 0x7fc83c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 --2- 172.21.15.17:0/2027778750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc83c0c17c0 0x7fc83c14df80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 >> 172.21.15.17:0/2027778750 conn(0x7fc83c0c4290 msgr2=0x7fc83c0c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 shutdown_connections 2024-03-06T07:20:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.729+0000 7fc8439e2640 1 -- 172.21.15.17:0/2027778750 wait complete. 2024-03-06T07:20:10.757 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:10.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.857+0000 7fbc4ddd7640 1 Processor -- start 2024-03-06T07:20:10.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.857+0000 7fbc4ddd7640 1 -- start start 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc4ddd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc4814f360 0x7fbc48155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc4ddd7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbc480c88c0 0x7fbc480c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc4ddd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc480c73d0 con 0x7fbc480c5880 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc4ddd7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc480c8dd0 con 0x7fbc480c88c0 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc4ddd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480be2e0 con 0x7fbc4814f360 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbc480c88c0 0x7fbc480c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbc480c88c0 0x7fbc480c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33226/0 (socket says 172.21.15.17:33226) 2024-03-06T07:20:10.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 -- 172.21.15.17:0/1780984210 learned_addr learned my addr 172.21.15.17:0/1780984210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc477fe640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc4814f360 0x7fbc48155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 -- 172.21.15.17:0/1780984210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc4814f360 msgr2=0x7fbc48155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc4814f360 0x7fbc48155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 -- 172.21.15.17:0/1780984210 >> v1:172.21.15.163:6789/0 conn(0x7fbc480c5880 legacy=0x7fbc48158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.858+0000 7fbc46ffd640 1 -- 172.21.15.17:0/1780984210 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480c7670 con 0x7fbc480c88c0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc46ffd640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c88c0 0x7fbc480c5340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009870 tx=0x7fbc3c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ea46ba0bed704 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc467fc640 1 -- 172.21.15.17:0/1780984210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c035020 con 0x7fbc480c88c0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc467fc640 1 -- 172.21.15.17:0/1780984210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c031640 con 0x7fbc480c88c0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc467fc640 1 -- 172.21.15.17:0/1780984210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c02d040 con 0x7fbc480c88c0 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/1780984210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c88c0 msgr2=0x7fbc480c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c88c0 0x7fbc480c5340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c009870 tx=0x7fbc3c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/1780984210 shutdown_connections 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c88c0 0x7fbc480c5340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/1780984210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc4814f360 0x7fbc48155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/1780984210 >> 172.21.15.17:0/1780984210 conn(0x7fbc480ac570 msgr2=0x7fbc480ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:10.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/1780984210 shutdown_connections 2024-03-06T07:20:10.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.859+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/1780984210 wait complete. 2024-03-06T07:20:10.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.860+0000 7fbc4ddd7640 1 Processor -- start 2024-03-06T07:20:10.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 -- start start 2024-03-06T07:20:10.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc480c88c0 0x7fbc481eb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc4814f360 0x7fbc480c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48159270 con 0x7fbc480c5880 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480c7c30 con 0x7fbc4814f360 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc4ddd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481593f0 con 0x7fbc480c88c0 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33234/0 (socket says 172.21.15.17:33234) 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc477fe640 1 -- 172.21.15.17:0/2099494532 learned_addr learned my addr 172.21.15.17:0/2099494532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc46ffd640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc480c88c0 0x7fbc481eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.861+0000 7fbc47fff640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc4814f360 0x7fbc480c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 -- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc480c88c0 msgr2=0x7fbc481eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc480c88c0 0x7fbc481eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 -- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc4814f360 msgr2=0x7fbc480c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc4814f360 0x7fbc480c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 -- 172.21.15.17:0/2099494532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480c7e70 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc477fe640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e870 tx=0x7fbc3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38017070 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3800c940 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc480c1b30 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3800cd30 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.862+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc480c20b0 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.863+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc480c8590 con 0x7fbc480c5880 2024-03-06T07:20:10.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.863+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc3801a070 con 0x7fbc480c5880 2024-03-06T07:20:10.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.864+0000 7fbc2ffff640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 0x7fbc3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:10.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.864+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbc3808a370 con 0x7fbc480c5880 2024-03-06T07:20:10.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.864+0000 7fbc46ffd640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 0x7fbc3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:10.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.864+0000 7fbc46ffd640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 0x7fbc3406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c036000 tx=0x7fbc3c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:10.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:10.868+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc480c8590 con 0x7fbc480c5880 2024-03-06T07:20:11.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.090+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc480ca3c0 con 0x7fbc480c5880 2024-03-06T07:20:11.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.090+0000 7fbc2ffff640 1 -- 172.21.15.17:0/2099494532 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4976 (secure 0 0 0) 0x7fbc38058860 con 0x7fbc480c5880 2024-03-06T07:20:11.093 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:11.093 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:09.115148+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:11.093 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-03-06T07:20:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 msgr2=0x7fbc3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 0x7fbc3406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c036000 tx=0x7fbc3c03c040 comp rx=0 tx=0).stop 2024-03-06T07:20:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 msgr2=0x7fbc480add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800e870 tx=0x7fbc3800ed30 comp rx=0 tx=0).stop 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 shutdown_connections 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc3406b4d0 0x7fbc3406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc4814f360 0x7fbc480c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc480c88c0 0x7fbc481eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 --2- 172.21.15.17:0/2099494532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc480c5880 0x7fbc480add50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 >> 172.21.15.17:0/2099494532 conn(0x7fbc480ac570 msgr2=0x7fbc480c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.092+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 shutdown_connections 2024-03-06T07:20:11.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.093+0000 7fbc4ddd7640 1 -- 172.21.15.17:0/2099494532 wait complete. 2024-03-06T07:20:11.107 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.205+0000 7f3510965640 1 Processor -- start 2024-03-06T07:20:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.205+0000 7f3510965640 1 -- start start 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3510965640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 0x7f350c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3510965640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f350c159460 0x7f350c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3510965640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c0c7760 con 0x7f350c152920 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3510965640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c1587b0 con 0x7f350c159460 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3510965640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0c7a40 con 0x7f350c0c8a00 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f350c159460 0x7f350c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3503fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 0x7f350c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f350c159460 0x7f350c0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33250/0 (socket says 172.21.15.17:33250) 2024-03-06T07:20:11.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 -- 172.21.15.17:0/3321950762 learned_addr learned my addr 172.21.15.17:0/3321950762 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350a59c640 1 --1- 172.21.15.17:0/3321950762 >> v1:172.21.15.163:6789/0 conn(0x7f350c152920 0x7f350c155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59728/0 (socket says 172.21.15.17:59728) 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 -- 172.21.15.17:0/3321950762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 msgr2=0x7f350c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f3509d9b640 1 -- 172.21.15.17:0/3321950762 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370396706 0 0) 0x7f350c0c7760 con 0x7f350c152920 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 0x7f350c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 -- 172.21.15.17:0/3321950762 >> v1:172.21.15.163:6789/0 conn(0x7f350c152920 legacy=0x7f350c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.206+0000 7f350ad9d640 1 -- 172.21.15.17:0/3321950762 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0c7d20 con 0x7f350c159460 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.207+0000 7f3503fff640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 0x7f350c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.207+0000 7f350ad9d640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c159460 0x7f350c0c5330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3504009e50 tx=0x7f350402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28f74f6c6559cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.207+0000 7f3509d9b640 1 -- 172.21.15.17:0/3321950762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350402ca70 con 0x7f350c159460 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.207+0000 7f3509d9b640 1 -- 172.21.15.17:0/3321950762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350402cc10 con 0x7f350c159460 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.207+0000 7f3509d9b640 1 -- 172.21.15.17:0/3321950762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040343f0 con 0x7f350c159460 2024-03-06T07:20:11.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 -- 172.21.15.17:0/3321950762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c159460 msgr2=0x7f350c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c159460 0x7f350c0c5330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3504009e50 tx=0x7f350402bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 -- 172.21.15.17:0/3321950762 shutdown_connections 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c159460 0x7f350c0c5330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 --2- 172.21.15.17:0/3321950762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c0c8a00 0x7f350c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 -- 172.21.15.17:0/3321950762 >> 172.21.15.17:0/3321950762 conn(0x7f350c0ac560 msgr2=0x7f350c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 -- 172.21.15.17:0/3321950762 shutdown_connections 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.208+0000 7f3510965640 1 -- 172.21.15.17:0/3321950762 wait complete. 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.209+0000 7f3510965640 1 Processor -- start 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.209+0000 7f3510965640 1 -- start start 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 0x7f350c147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c1480d0 0x7f350c1ea9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0c7390 con 0x7f350c0c8a00 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c06d030 con 0x7f350c152920 2024-03-06T07:20:11.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3510965640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0c7510 con 0x7f350c1480d0 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33260/0 (socket says 172.21.15.17:33260) 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 -- 172.21.15.17:0/1176410635 learned_addr learned my addr 172.21.15.17:0/1176410635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3503fff640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 0x7f350c147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350ad9d640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c1480d0 0x7f350c1ea9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 -- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c1480d0 msgr2=0x7f350c1ea9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c1480d0 0x7f350c1ea9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 -- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 msgr2=0x7f350c147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 0x7f350c147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 -- 172.21.15.17:0/1176410635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0c31e0 con 0x7f350c0c8a00 2024-03-06T07:20:11.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f3503fff640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 0x7f350c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.210+0000 7f350a59c640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c4d0 tx=0x7f34fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.211+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc018070 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.211+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc007500 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.211+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c0c3e70 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.211+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc004a20 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.211+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c0c4430 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.212+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c04f330 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.212+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34fc01b070 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.212+0000 7f3502ffd640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 0x7f34ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.213+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f34fc089800 con 0x7f350c0c8a00 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.213+0000 7f3503fff640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 0x7f34ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.213+0000 7f3503fff640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 0x7f34ec06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009560 tx=0x7f34f4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.217+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34fc057ce0 con 0x7f350c0c8a00 2024-03-06T07:20:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.401+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c06cda0 con 0x7f350c0c8a00 2024-03-06T07:20:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.402+0000 7f3502ffd640 1 -- 172.21.15.17:0/1176410635 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f34fc05bd80 con 0x7f350c0c8a00 2024-03-06T07:20:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:11.406 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 msgr2=0x7f34ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 0x7f34ec06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009560 tx=0x7f34f4009190 comp rx=0 tx=0).stop 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 msgr2=0x7f350c0ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c4d0 tx=0x7f34fc00c990 comp rx=0 tx=0).stop 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 shutdown_connections 2024-03-06T07:20:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ec06b4d0 0x7f34ec06d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f350c1480d0 0x7f350c1ea9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f350c152920 0x7f350c147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 --2- 172.21.15.17:0/1176410635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c0c8a00 0x7f350c0ab750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 >> 172.21.15.17:0/1176410635 conn(0x7f350c0ac560 msgr2=0x7f350c06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 shutdown_connections 2024-03-06T07:20:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.404+0000 7f3510965640 1 -- 172.21.15.17:0/1176410635 wait complete. 2024-03-06T07:20:11.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:20:04.076240+0000', 'modified': '2024-03-06T07:20:09.115148+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2082625649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2082625649}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2082625649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4264, 'qdb_cluster': [4264]} 2024-03-06T07:20:11.821 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:20:11.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:20:11.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.922+0000 7f5f829c7640 1 Processor -- start 2024-03-06T07:20:11.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.922+0000 7f5f829c7640 1 -- start start 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f829c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f829c7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c0c8e30 con 0x7f5f7c0c64f0 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f829c7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c0c9110 con 0x7f5f7c152900 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f829c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c152ce0 con 0x7f5f7c0c8950 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f80f64640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5f7c0c64f0 0x7f5f7c0c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47650/0 (socket says 172.21.15.17:47650) 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f80f64640 1 -- 172.21.15.17:0/1629518226 learned_addr learned my addr 172.21.15.17:0/1629518226 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/1629518226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026459844 0 0) 0x7f5f7c0c8e30 con 0x7f5f7c0c64f0 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f68003520 con 0x7f5f7c0c64f0 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849982480 0 0) 0x7f5f7c0c9110 con 0x7f5f7c152900 2024-03-06T07:20:11.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f7c0c8e30 con 0x7f5f7c152900 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/1629518226 >> v1:172.21.15.163:6789/0 conn(0x7f5f7c152900 legacy=0x7f5f7c0bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/1629518226 >> v1:172.21.15.17:6789/0 conn(0x7f5f7c0c64f0 legacy=0x7f5f7c0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.923+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/1629518226 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0c93f0 con 0x7f5f7c0c8950 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.924+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/1629518226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c155240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097d0 tx=0x7f5f6c02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c334cffa73f9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.924+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c031770 con 0x7f5f7c0c8950 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.924+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c037070 con 0x7f5f7c0c8950 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.924+0000 7f5f7affd640 1 -- 172.21.15.17:0/1629518226 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c031b10 con 0x7f5f7c0c8950 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 -- 172.21.15.17:0/1629518226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 msgr2=0x7f5f7c155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1629518226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c155240 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0097d0 tx=0x7f5f6c02b810 comp rx=0 tx=0).stop 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 -- 172.21.15.17:0/1629518226 shutdown_connections 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1629518226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c155240 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 -- 172.21.15.17:0/1629518226 >> 172.21.15.17:0/1629518226 conn(0x7f5f7c0ac860 msgr2=0x7f5f7c0acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 -- 172.21.15.17:0/1629518226 shutdown_connections 2024-03-06T07:20:11.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.925+0000 7f5f829c7640 1 -- 172.21.15.17:0/1629518226 wait complete. 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.926+0000 7f5f829c7640 1 Processor -- start 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.926+0000 7f5f829c7640 1 -- start start 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f7c0c64f0 0x7f5f7c0c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c0000 con 0x7f5f7c152900 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c70b0 con 0x7f5f7c0c64f0 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0c0180 con 0x7f5f7c0c8950 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33264/0 (socket says 172.21.15.17:33264) 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 -- 172.21.15.17:0/1338646279 learned_addr learned my addr 172.21.15.17:0/1338646279 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f7c0c64f0 0x7f5f7c0c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 -- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 msgr2=0x7f5f7c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 -- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f7c0c64f0 msgr2=0x7f5f7c0c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f7c0c64f0 0x7f5f7c0c3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 -- 172.21.15.17:0/1338646279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c159260 con 0x7f5f7c152900 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f80f64640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400cb40 tx=0x7f5f7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f78ff9640 1 -- 172.21.15.17:0/1338646279 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74017070 con 0x7f5f7c152900 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f78ff9640 1 -- 172.21.15.17:0/1338646279 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7400acc0 con 0x7f5f7c152900 2024-03-06T07:20:11.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c159520 con 0x7f5f7c152900 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f78ff9640 1 -- 172.21.15.17:0/1338646279 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74012640 con 0x7f5f7c152900 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.927+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c14fd20 con 0x7f5f7c152900 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.929+0000 7f5f78ff9640 1 -- 172.21.15.17:0/1338646279 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f7401a070 con 0x7f5f7c152900 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.929+0000 7f5f78ff9640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 0x7f5f5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.929+0000 7f5f78ff9640 1 -- 172.21.15.17:0/1338646279 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5f7408a2f0 con 0x7f5f7c152900 2024-03-06T07:20:11.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.929+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 0x7f5f5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.930+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 0x7f5f5c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f640011e0 tx=0x7f5f64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.930+0000 7f5f855ac740 1 Processor -- start 2024-03-06T07:20:11.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.930+0000 7f5f855ac740 1 -- start start 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b1da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1e4b0 0x560b38b1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38a846b0 con 0x560b38b20330 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38a2da60 con 0x560b38b1e4b0 2024-03-06T07:20:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b38a80750 con 0x560b38b1ff50 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b1da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b1df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33276/0 (socket says 172.21.15.17:33276) 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/3309369463 learned_addr learned my addr 172.21.15.17:0/3309369463 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f80f64640 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1e4b0 0x560b38b1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 -- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1e4b0 msgr2=0x560b38b1a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1e4b0 0x560b38b1a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 -- 172.21.15.17:0/3309369463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 msgr2=0x560b38b1df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b1df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 -- 172.21.15.17:0/3309369463 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b38a4bf30 con 0x560b38b1ff50 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b1da30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400f5a0 tx=0x7f5f64012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a88ef59be12a813 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f71ffb640 1 -- 172.21.15.17:0/3309369463 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64025070 con 0x560b38b1ff50 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f71ffb640 1 -- 172.21.15.17:0/3309369463 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6401b050 con 0x560b38b1ff50 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.931+0000 7f5f71ffb640 1 -- 172.21.15.17:0/3309369463 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400fea0 con 0x560b38b1ff50 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 msgr2=0x560b38b1da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b1da30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400f5a0 tx=0x7f5f64012000 comp rx=0 tx=0).stop 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- 172.21.15.17:0/3309369463 shutdown_connections 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1e4b0 0x560b38b1a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b1df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3309369463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b1da30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- 172.21.15.17:0/3309369463 >> 172.21.15.17:0/3309369463 conn(0x560b38b1c7d0 msgr2=0x560b38b413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- 172.21.15.17:0/3309369463 shutdown_connections 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- 172.21.15.17:0/3309369463 wait complete. 2024-03-06T07:20:11.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.932+0000 7f5f855ac740 1 -- start start 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 Processor -- start 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b460a0 0x560b38b58a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38b42230 con 0x560b38b20330 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38a40ea0 con 0x560b38b460a0 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.933+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b38b41c70 con 0x560b38b1ff50 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f855ac740 10 client.0 _ll_register_callbacks cb 0x560b38a40870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33288/0 (socket says 172.21.15.17:33288) 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/2538541591 learned_addr learned my addr 172.21.15.17:0/2538541591 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f80f64640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b460a0 0x560b38b58a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 msgr2=0x560b38b5bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b5bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b460a0 msgr2=0x560b38b58a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b460a0 0x560b38b58a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/2538541591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b38a41270 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0098f0 tx=0x7f5f6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9a4a5fa4aa0349 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f5affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c032810 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5f6c0329b0 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f6401b050 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6c053050 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f855ac740 10 client.4404 fetch_fsmap learned FSMap version 7 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560b387992d0 con 0x560b38b20330 2024-03-06T07:20:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.934+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6401b050 con 0x560b38b20330 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 7) v1 ==== 3322+0+0 (secure 0 0 0) 0x7f5f6c032bb0 con 0x560b38b20330 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 10 client.4404 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 20 client.4404 populate_metadata read hostname 'smithi017' 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38b6d4f0 0x560b38b7b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 4 client.4404 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 --> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b38a91bf0 con 0x560b38b6d4f0 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38b6d4f0 0x560b38b7b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f5affd640 10 client.4404 ms_handle_connect on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.935+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38b6d4f0 0x560b38b7b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed7daf43f717e00 server_cookie=b24b0475ecfa702f in_seq=0 out_seq=0 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f5affd640 1 -- 172.21.15.17:0/2538541591 <== mds.0 v2:172.21.15.163:6835/2082625649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560b38a91bf0 con 0x560b38b6d4f0 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f5affd640 10 client.4404 handle_command_reply: tid=0 2024-03-06T07:20:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 client.4404 shutdown 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 msgr2=0x560b38b42600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0098f0 tx=0x7f5f6c004730 comp rx=0 tx=0).stop 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 shutdown_connections 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38b6d4f0 0x560b38b7b5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b460a0 0x560b38b58a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b42600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f5affd640 0 client.4404 ms_handle_reset on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 --2- 172.21.15.17:0/2538541591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b5bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 >> 172.21.15.17:0/2538541591 conn(0x560b38b1e4b0 msgr2=0x560b38b19510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 shutdown_connections 2024-03-06T07:20:11.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 1 -- 172.21.15.17:0/2538541591 wait complete. 2024-03-06T07:20:11.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.936+0000 7f5f855ac740 20 client.4404 trim_cache size 0 max 16384 2024-03-06T07:20:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 Processor -- start 2024-03-06T07:20:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 -- start start 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1ff50 0x560b38b805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 0x560b38b44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.953+0000 7f5f855ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c1f7a30 con 0x560b38b20330 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c1ef730 con 0x560b38b1ff50 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c156ec0 con 0x560b38b460a0 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33302/0 (socket says 172.21.15.17:33302) 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/3090159032 learned_addr learned my addr 172.21.15.17:0/3090159032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1ff50 0x560b38b805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f80f64640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 0x560b38b44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 msgr2=0x560b38b44ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 0x560b38b44ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1ff50 msgr2=0x560b38b805d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1ff50 0x560b38b805d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 -- 172.21.15.17:0/3090159032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b388636b0 con 0x560b38b20330 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f80f64640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 0x560b38b44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009720 tx=0x7f5f6c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36042aebbf2be6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f717fa640 1 -- 172.21.15.17:0/3090159032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004040 con 0x560b38b20330 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f717fa640 1 -- 172.21.15.17:0/3090159032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c04b070 con 0x560b38b20330 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.954+0000 7f5f717fa640 1 -- 172.21.15.17:0/3090159032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004400 con 0x560b38b20330 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- 172.21.15.17:0/3090159032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 msgr2=0x560b38b5f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009720 tx=0x7f5f6c042000 comp rx=0 tx=0).stop 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- 172.21.15.17:0/3090159032 shutdown_connections 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b460a0 0x560b38b44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b20330 0x560b38b5f510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 --2- 172.21.15.17:0/3090159032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b1ff50 0x560b38b805d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- 172.21.15.17:0/3090159032 >> 172.21.15.17:0/3090159032 conn(0x560b38b1e4b0 msgr2=0x560b38b1d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- 172.21.15.17:0/3090159032 shutdown_connections 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- 172.21.15.17:0/3090159032 wait complete. 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 -- start start 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.955+0000 7f5f855ac740 1 Processor -- start 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b43dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b20330 0x560b38bcc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38b47e00 con 0x560b38b460a0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b38b70660 con 0x560b38b20330 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b38b42eb0 con 0x560b38b1ff50 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f855ac740 10 client.0 _ll_register_callbacks cb 0x7f5f6c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f80f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f80f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33308/0 (socket says 172.21.15.17:33308) 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b43dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f80f64640 1 -- 172.21.15.17:0/536218461 learned_addr learned my addr 172.21.15.17:0/536218461 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b20330 0x560b38bcc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 -- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b20330 msgr2=0x560b38bcc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b20330 0x560b38bcc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 -- 172.21.15.17:0/536218461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 msgr2=0x560b38bc9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f7bfff640 1 -- 172.21.15.17:0/536218461 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b38b47a30 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.956+0000 7f5f80f64640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f7bfff640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b43dc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f64007fd0 tx=0x7f5f64001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c24a64ac3ba7f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 10 client.0 ms_handle_connect on v2:172.21.15.163:3301/0 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64025040 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f6c04b070 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mon.2 v2:172.21.15.163:3301/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5f64030830 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f64030a70 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 10 client.4346 fetch_fsmap learned FSMap version 7 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560b38bcc3d0 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mon.2 v2:172.21.15.163:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c04b070 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mon.2 v2:172.21.15.163:3301/0 5 ==== fsmap(e 7) v1 ==== 3322+0+0 (secure 0 0 0) 0x7f5f64030c10 con 0x560b38b1ff50 2024-03-06T07:20:11.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 10 client.4346 fetch_fsmap finished waiting for FSMap version 7 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 10 client.4346 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 20 client.4346 populate_metadata read hostname 'smithi017' 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38bcf4f0 0x560b38bcf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 4 client.4346 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 --> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560b38b5cfa0 con 0x560b38bcf4f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.957+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38bcf4f0 0x560b38bcf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f7b7fe640 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38bcf4f0 0x560b38bcf8d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9312909b039cca3 server_cookie=4286dec7befdc701 in_seq=0 out_seq=0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f5affd640 10 client.4346 ms_handle_connect on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f5affd640 1 -- 172.21.15.17:0/536218461 <== mds.0 v2:172.21.15.163:6835/2082625649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560b38b5cfa0 con 0x560b38bcf4f0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f5affd640 10 client.4346 handle_command_reply: tid=0 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 client.4346 shutdown 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 msgr2=0x560b38b43dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b43dc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5f64007fd0 tx=0x7f5f64001150 comp rx=0 tx=0).stop 2024-03-06T07:20:11.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 shutdown_connections 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x560b38bcf4f0 0x560b38bcf8d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b38b460a0 0x560b38bc9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560b38b20330 0x560b38bcc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f5affd640 0 client.4346 ms_handle_reset on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 --2- 172.21.15.17:0/536218461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560b38b1ff50 0x560b38b43dc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 >> 172.21.15.17:0/536218461 conn(0x560b38b1e4b0 msgr2=0x560b38b5f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 shutdown_connections 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 1 -- 172.21.15.17:0/536218461 wait complete. 2024-03-06T07:20:11.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.958+0000 7f5f855ac740 20 client.4346 trim_cache size 0 max 16384 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4264, 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 7.2549996630000004, 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 7, 2024-03-06T07:20:11.971 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 25, 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 24, 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14.176204542000001 2024-03-06T07:20:11.972 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 msgr2=0x7f5f5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 0x7f5f5c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f640011e0 tx=0x7f5f64009040 comp rx=0 tx=0).stop 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 msgr2=0x7f5f7c14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400cb40 tx=0x7f5f7400a060 comp rx=0 tx=0).stop 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 shutdown_connections 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5c06b4d0 0x7f5f5c06d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f7c152900 0x7f5f7c14f7e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f7c0c8950 0x7f5f7c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 --2- 172.21.15.17:0/1338646279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f7c0c64f0 0x7f5f7c0c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 >> 172.21.15.17:0/1338646279 conn(0x7f5f7c0ac860 msgr2=0x7f5f7c0bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 shutdown_connections 2024-03-06T07:20:11.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:11.959+0000 7f5f829c7640 1 -- 172.21.15.17:0/1338646279 wait complete. 2024-03-06T07:20:11.974 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:11.974 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:11.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:12.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.073+0000 7fab4ccd4640 1 Processor -- start 2024-03-06T07:20:12.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.073+0000 7fab4ccd4640 1 -- start start 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4ccd4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 0x7fab48155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4ccd4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab480c2620 0x7fab480c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4ccd4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4806afc0 con 0x7fab480c2190 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4ccd4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab48156160 con 0x7fab480c2620 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4ccd4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4806b2a0 con 0x7fab48152920 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab46d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab480c2620 0x7fab480c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab45d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fab480c2190 0x7fab480c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47652/0 (socket says 172.21.15.17:47652) 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab45d9b640 1 -- 172.21.15.17:0/3983257932 learned_addr learned my addr 172.21.15.17:0/3983257932 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4659c640 1 --2- 172.21.15.17:0/3983257932 >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 0x7fab48155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4559a640 1 -- 172.21.15.17:0/3983257932 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958372094 0 0) 0x7fab4806afc0 con 0x7fab480c2190 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.074+0000 7fab4559a640 1 -- 172.21.15.17:0/3983257932 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab34003520 con 0x7fab480c2190 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab46d9d640 1 -- 172.21.15.17:0/3983257932 >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 msgr2=0x7fab48155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab46d9d640 1 --2- 172.21.15.17:0/3983257932 >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 0x7fab48155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab46d9d640 1 -- 172.21.15.17:0/3983257932 >> v1:172.21.15.17:6789/0 conn(0x7fab480c2190 legacy=0x7fab480c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab46d9d640 1 -- 172.21.15.17:0/3983257932 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4806b580 con 0x7fab480c2620 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab4659c640 1 --2- 172.21.15.17:0/3983257932 >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 0x7fab48155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab46d9d640 1 --2- 172.21.15.17:0/3983257932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480c1990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab48070a90 tx=0x7fab3802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d558697dece356 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab4559a640 1 -- 172.21.15.17:0/3983257932 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38039070 con 0x7fab480c2620 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab4559a640 1 -- 172.21.15.17:0/3983257932 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3802cc80 con 0x7fab480c2620 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.075+0000 7fab4559a640 1 -- 172.21.15.17:0/3983257932 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38034660 con 0x7fab480c2620 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 -- 172.21.15.17:0/3983257932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 msgr2=0x7fab480c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/3983257932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480c1990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fab48070a90 tx=0x7fab3802bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 -- 172.21.15.17:0/3983257932 shutdown_connections 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/3983257932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480c1990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/3983257932 >> v2:172.21.15.163:3300/0 conn(0x7fab48152920 0x7fab48155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 -- 172.21.15.17:0/3983257932 >> 172.21.15.17:0/3983257932 conn(0x7fab480c4290 msgr2=0x7fab480c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 -- 172.21.15.17:0/3983257932 shutdown_connections 2024-03-06T07:20:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.076+0000 7fab4ccd4640 1 -- 172.21.15.17:0/3983257932 wait complete. 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.077+0000 7fab4ccd4640 1 Processor -- start 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.077+0000 7fab4ccd4640 1 -- start start 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab480c2190 0x7fab4814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab48152920 0x7fab481edf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480a7100 con 0x7fab480c2620 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c0d60 con 0x7fab48152920 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4ccd4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480a7280 con 0x7fab480c2190 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33320/0 (socket says 172.21.15.17:33320) 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 -- 172.21.15.17:0/10944871 learned_addr learned my addr 172.21.15.17:0/10944871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab46d9d640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab48152920 0x7fab481edf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab4659c640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab480c2190 0x7fab4814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 -- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab480c2190 msgr2=0x7fab4814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab480c2190 0x7fab4814f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 -- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab48152920 msgr2=0x7fab481edf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab48152920 0x7fab481edf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.078+0000 7fab45d9b640 1 -- 172.21.15.17:0/10944871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4814db00 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab45d9b640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c430 tx=0x7fab3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30017070 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab30005ce0 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab48156a10 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab300072f0 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.079+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4814eb50 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.080+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4804f330 con 0x7fab480c2620 2024-03-06T07:20:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.080+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab3001a070 con 0x7fab480c2620 2024-03-06T07:20:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.081+0000 7fab3effd640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 0x7fab2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.081+0000 7fab4659c640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 0x7fab2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.081+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fab30089920 con 0x7fab480c2620 2024-03-06T07:20:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.081+0000 7fab4659c640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 0x7fab2806d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab48152650 tx=0x7fab40009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.086+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab30057e00 con 0x7fab480c2620 2024-03-06T07:20:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.303+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fab48068460 con 0x7fab480c2620 2024-03-06T07:20:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.304+0000 7fab3effd640 1 -- 172.21.15.17:0/10944871 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fab3005bea0 con 0x7fab480c2620 2024-03-06T07:20:12.309 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 msgr2=0x7fab2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 0x7fab2806d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab48152650 tx=0x7fab40009190 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 msgr2=0x7fab480725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fab3000c430 tx=0x7fab3000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 shutdown_connections 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab2806b500 0x7fab2806d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab48152920 0x7fab481edf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab480c2620 0x7fab480725b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 --2- 172.21.15.17:0/10944871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab480c2190 0x7fab4814f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 >> 172.21.15.17:0/10944871 conn(0x7fab480c4290 msgr2=0x7fab48153820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 shutdown_connections 2024-03-06T07:20:12.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:12.306+0000 7fab4ccd4640 1 -- 172.21.15.17:0/10944871 wait complete. 2024-03-06T07:20:12.319 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-06T07:20:12.319 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:12.319 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:12.320 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-06T07:20:12.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.320 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-06T07:20:12.335 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:12.335 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:12.335 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:12.335 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:12.335 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42655sec preferred_lft 42655sec 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-03-06T07:20:12.336 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:12.337 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:12.337 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:12.337 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-06T07:20:12.337 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:12.338 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:12.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.504 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:12.529 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-06T07:20:12.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.531 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-06T07:20:12.602 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-06T07:20:12.602 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:12.603 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-06T07:20:12.603 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-06T07:20:12.603 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:12.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.603 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:12.603 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:12.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-06T07:20:12.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-06T07:20:12.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-06T07:20:12.604 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:12.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.860 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-06T07:20:12.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.875 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-03-06T07:20:12.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:12.931 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:12.931 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:12.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:12.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-06T07:20:12.931 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:13.015 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-06T07:20:13.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-06T07:20:13.016 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:13.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.165 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:13.165 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:13.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-06T07:20:13.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:13.165 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:13.208 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-06T07:20:13.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:13.208 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:13.263 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-06T07:20:13.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:13.263 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:13.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-06T07:20:13.387 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-06T07:20:13.444 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-06T07:20:13.444 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-06T07:20:13.445 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-06T07:20:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:13.447 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:13.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.447 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-06T07:20:13.499 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-06T07:20:13.499 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-06T07:20:13.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-06T07:20:13.569 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-06T07:20:13.625 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/selinux 2024-03-06T07:20:13.626 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysctl.service 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/nfs/rpc_pipefs 2024-03-06T07:20:13.627 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:13.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.629 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-06T07:20:13.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:2024-03-06T07:20:13.628+0000 7fcfcf842480 -1 init, newargv = 0x56420bcc4f90 newargc=15 2024-03-06T07:20:13.631 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[43188]: starting ceph client 2024-03-06T07:20:13.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[43188]: starting fuse 2024-03-06T07:20:13.648 INFO:teuthology.orchestra.run.smithi017.stdout:43 2024-03-06T07:20:13.648 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-06T07:20:13.648 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-03-06T07:20:13.648 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-03-06T07:20:13.648 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-03-06T07:20:13.648 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-03-06T07:20:13.648 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-06T07:20:13.649 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-03-06T07:20:13.650 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-06T07:20:13.651 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-06T07:20:13.651 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.admin")) 2024-03-06T07:20:13.651 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-03-06T07:20:13.770 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.admin.43188.asok 2024-03-06T07:20:13.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-06T07:20:13.780 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43188.asok 2024-03-06T07:20:13.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.780 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43188.asok status 2024-03-06T07:20:13.918 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:20:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-03-06T07:20:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd", 2024-03-06T07:20:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-1739-g1bc250c7 (1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd) squid (dev)", 2024-03-06T07:20:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "admin", 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "43188", 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4428, 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4428 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:20:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3867886274 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3867886274 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.4428 192.168.0.1:0/3867886274", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/3867886274", 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-03-06T07:20:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 7, 2024-03-06T07:20:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 25, 2024-03-06T07:20:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-03-06T07:20:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-03-06T07:20:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-03-06T07:20:13.921 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:20:13.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.927 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-03-06T07:20:13.927 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:13.942 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-03-06T07:20:13.943 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:13.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:13.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:14.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:14.010 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-06T07:20:14.076 INFO:teuthology.orchestra.run.smithi017.stdout:# file: . 2024-03-06T07:20:14.077 INFO:teuthology.orchestra.run.smithi017.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-06T07:20:14.077 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:14.077 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:14.077 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:14.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.271+0000 7f89d5556640 1 Processor -- start 2024-03-06T07:20:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.271+0000 7f89d5556640 1 -- start start 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89d5556640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89d0153990 0x7f89d0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89d5556640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 0x7f89d00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89d5556640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d0070aa0 con 0x7f89d01506b0 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89d5556640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d0156450 con 0x7f89d0153990 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89d5556640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0156a10 con 0x7f89d00acc00 2024-03-06T07:20:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89cf7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 0x7f89d00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89d0153990 0x7f89d0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89d0153990 0x7f89d0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56598/0 (socket says 172.21.15.17:56598) 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89cf7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 0x7f89d00c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54554/0 (socket says 172.21.15.17:54554) 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 -- 172.21.15.17:0/1666161294 learned_addr learned my addr 172.21.15.17:0/1666161294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ceffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f89d01506b0 0x7f89d00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59744/0 (socket says 172.21.15.17:59744) 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 -- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 msgr2=0x7f89d00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 --2- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 0x7f89d00c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.272+0000 7f89ce7fc640 1 -- 172.21.15.17:0/1666161294 >> v1:172.21.15.163:6789/0 conn(0x7f89d01506b0 legacy=0x7f89d00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.273+0000 7f89ce7fc640 1 -- 172.21.15.17:0/1666161294 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0070d80 con 0x7f89d0153990 2024-03-06T07:20:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.273+0000 7f89ce7fc640 1 --2- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d0153990 0x7f89d0156ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009870 tx=0x7f89b802b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4a1007c79d6d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.273+0000 7f89cdffb640 1 -- 172.21.15.17:0/1666161294 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8037070 con 0x7f89d0153990 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89cdffb640 1 -- 172.21.15.17:0/1666161294 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8004230 con 0x7f89d0153990 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89cdffb640 1 -- 172.21.15.17:0/1666161294 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8003610 con 0x7f89d0153990 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 -- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d0153990 msgr2=0x7f89d0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 --2- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d0153990 0x7f89d0156ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89b8009870 tx=0x7f89b802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 -- 172.21.15.17:0/1666161294 shutdown_connections 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 --2- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d00acc00 0x7f89d00c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 --2- 172.21.15.17:0/1666161294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d0153990 0x7f89d0156ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 -- 172.21.15.17:0/1666161294 >> 172.21.15.17:0/1666161294 conn(0x7f89d006b1a0 msgr2=0x7f89d006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 -- 172.21.15.17:0/1666161294 shutdown_connections 2024-03-06T07:20:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.274+0000 7f89d5556640 1 -- 172.21.15.17:0/1666161294 wait complete. 2024-03-06T07:20:14.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.275+0000 7f89d5556640 1 Processor -- start 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 -- start start 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d0153990 0x7f89d01499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d00ad1d0 con 0x7f89d01506b0 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d00ad050 con 0x7f89d00acc00 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89d5556640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d00ad350 con 0x7f89d0153990 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89ce7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89ce7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33328/0 (socket says 172.21.15.17:33328) 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89ce7fc640 1 -- 172.21.15.17:0/102399866 learned_addr learned my addr 172.21.15.17:0/102399866 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89ceffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56606/0 (socket says 172.21.15.17:56606) 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.276+0000 7f89cf7fe640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d0153990 0x7f89d01499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 -- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d0153990 msgr2=0x7f89d01499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d0153990 0x7f89d01499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 -- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 msgr2=0x7f89d00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 -- 172.21.15.17:0/102399866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d01ef410 con 0x7f89d01506b0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ceffd640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89ce7fc640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89b8035040 tx=0x7f89b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8042020 con 0x7f89d01506b0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8030510 con 0x7f89d01506b0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d01ee650 con 0x7f89d01506b0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8037040 con 0x7f89d01506b0 2024-03-06T07:20:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.277+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d01eec10 con 0x7f89d01506b0 2024-03-06T07:20:14.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.278+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d004f330 con 0x7f89d01506b0 2024-03-06T07:20:14.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.279+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89b8045020 con 0x7f89d01506b0 2024-03-06T07:20:14.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.279+0000 7f89c77fe640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 0x7f89b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.279+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f89b80abcd0 con 0x7f89d01506b0 2024-03-06T07:20:14.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.279+0000 7f89ceffd640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 0x7f89b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.280+0000 7f89ceffd640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 0x7f89b006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000f60 tx=0x7f89c8007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.283+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89b804c070 con 0x7f89d01506b0 2024-03-06T07:20:14.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.464+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89d00707e0 con 0x7f89d01506b0 2024-03-06T07:20:14.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.465+0000 7f89c77fe640 1 -- 172.21.15.17:0/102399866 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f89b807a1c0 con 0x7f89d01506b0 2024-03-06T07:20:14.467 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:14.467 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.466+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 msgr2=0x7f89b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.466+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 0x7f89b006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89c8000f60 tx=0x7f89c8007e40 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.466+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 msgr2=0x7f89d00c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.466+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89b8035040 tx=0x7f89b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 shutdown_connections 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f89b006b4d0 0x7f89b006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89d0153990 0x7f89d01499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89d01506b0 0x7f89d00c49f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 --2- 172.21.15.17:0/102399866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89d00acc00 0x7f89d00c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 >> 172.21.15.17:0/102399866 conn(0x7f89d006b1a0 msgr2=0x7f89d014f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 shutdown_connections 2024-03-06T07:20:14.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.467+0000 7f89d5556640 1 -- 172.21.15.17:0/102399866 wait complete. 2024-03-06T07:20:14.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:14.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.582+0000 7f44f1823640 1 Processor -- start 2024-03-06T07:20:14.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.582+0000 7f44f1823640 1 -- start start 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44f1823640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44ec155c30 0x7f44ec06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44f1823640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec0c5380 0x7f44ec0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44f1823640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec0b0450 con 0x7f44ec0b0070 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44f1823640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec06cfb0 con 0x7f44ec155c30 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44f1823640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec06c020 con 0x7f44ec0c5380 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44eaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f44ec0b0070 0x7f44ec0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47666/0 (socket says 172.21.15.17:47666) 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44eaffd640 1 -- 172.21.15.17:0/2557710531 learned_addr learned my addr 172.21.15.17:0/2557710531 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/2557710531 >> v2:172.21.15.17:3300/0 conn(0x7f44ec155c30 0x7f44ec06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44eb7fe640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec0c5380 0x7f44ec0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44ea7fc640 1 -- 172.21.15.17:0/2557710531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec0c5380 msgr2=0x7f44ec0c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec0c5380 0x7f44ec0c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.583+0000 7f44ea7fc640 1 -- 172.21.15.17:0/2557710531 >> v1:172.21.15.17:6789/0 conn(0x7f44ec0b0070 legacy=0x7f44ec0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44e9ffb640 1 -- 172.21.15.17:0/2557710531 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026441932 0 0) 0x7f44e4001110 con 0x7f44ec0b0070 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44ea7fc640 1 -- 172.21.15.17:0/2557710531 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec1560e0 con 0x7f44ec155c30 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec155c30 0x7f44ec06ca70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44d40097a0 tx=0x7f44d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a19b386524d2bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44e9ffb640 1 -- 172.21.15.17:0/2557710531 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40315a0 con 0x7f44ec155c30 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44e9ffb640 1 -- 172.21.15.17:0/2557710531 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4037070 con 0x7f44ec155c30 2024-03-06T07:20:14.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.584+0000 7f44e9ffb640 1 -- 172.21.15.17:0/2557710531 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4031960 con 0x7f44ec155c30 2024-03-06T07:20:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 -- 172.21.15.17:0/2557710531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec155c30 msgr2=0x7f44ec06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec155c30 0x7f44ec06ca70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44d40097a0 tx=0x7f44d402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 -- 172.21.15.17:0/2557710531 shutdown_connections 2024-03-06T07:20:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec0c5380 0x7f44ec0c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 --2- 172.21.15.17:0/2557710531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec155c30 0x7f44ec06ca70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 -- 172.21.15.17:0/2557710531 >> 172.21.15.17:0/2557710531 conn(0x7f44ec0ac590 msgr2=0x7f44ec0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 -- 172.21.15.17:0/2557710531 shutdown_connections 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.585+0000 7f44f1823640 1 -- 172.21.15.17:0/2557710531 wait complete. 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.586+0000 7f44f1823640 1 Processor -- start 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.586+0000 7f44f1823640 1 -- start start 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 0x7f44ec0c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec155c30 0x7f44ec0c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec06c270 con 0x7f44ec0b0070 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0c7ad0 con 0x7f44ec0c5380 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44f1823640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec06c550 con 0x7f44ec155c30 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33344/0 (socket says 172.21.15.17:33344) 2024-03-06T07:20:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 -- 172.21.15.17:0/3416793708 learned_addr learned my addr 172.21.15.17:0/3416793708 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 0x7f44ec0c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eb7fe640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec155c30 0x7f44ec0c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 -- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec155c30 msgr2=0x7f44ec0c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec155c30 0x7f44ec0c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 -- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 msgr2=0x7f44ec0c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 0x7f44ec0c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44eaffd640 1 -- 172.21.15.17:0/3416793708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec1570c0 con 0x7f44ec0b0070 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.587+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 0x7f44ec0c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44eaffd640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c460 tx=0x7f44e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4018070 con 0x7f44ec0b0070 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4007500 con 0x7f44ec0b0070 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec157360 con 0x7f44ec0b0070 2024-03-06T07:20:14.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004a20 con 0x7f44ec0b0070 2024-03-06T07:20:14.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.588+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec157920 con 0x7f44ec0b0070 2024-03-06T07:20:14.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.589+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec04f330 con 0x7f44ec0b0070 2024-03-06T07:20:14.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.589+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44e401b070 con 0x7f44ec0b0070 2024-03-06T07:20:14.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.590+0000 7f44e3fff640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.590+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f44e40896d0 con 0x7f44ec0b0070 2024-03-06T07:20:14.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.590+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.590+0000 7f44ea7fc640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44d4036000 tx=0x7f44d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.594+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44ec0b0450 con 0x7f44ec0b0070 2024-03-06T07:20:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.775+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44ec06d860 con 0x7f44ec0b0070 2024-03-06T07:20:14.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.775+0000 7f44e3fff640 1 -- 172.21.15.17:0/3416793708 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44e4057c50 con 0x7f44ec0b0070 2024-03-06T07:20:14.778 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 msgr2=0x7f44cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 0x7f44cc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44d4036000 tx=0x7f44d403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:14.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 msgr2=0x7f44ec158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c460 tx=0x7f44e400c920 comp rx=0 tx=0).stop 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 shutdown_connections 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44cc06b4d0 0x7f44cc06d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44ec155c30 0x7f44ec0c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44ec0c5380 0x7f44ec0c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 --2- 172.21.15.17:0/3416793708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ec0b0070 0x7f44ec158ec0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 >> 172.21.15.17:0/3416793708 conn(0x7f44ec0ac590 msgr2=0x7f44ec0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 shutdown_connections 2024-03-06T07:20:14.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.777+0000 7f44f1823640 1 -- 172.21.15.17:0/3416793708 wait complete. 2024-03-06T07:20:14.790 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:14.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.892+0000 7fb28cdd7640 1 Processor -- start 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.892+0000 7fb28cdd7640 1 -- start start 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.892+0000 7fb28cdd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.892+0000 7fb28cdd7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.892+0000 7fb28cdd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880c75e0 con 0x7fb2880c5910 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28cdd7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2880c7f10 con 0x7fb2880be150 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28cdd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880c78c0 con 0x7fb28814bcc0 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb27ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb27ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56620/0 (socket says 172.21.15.17:56620) 2024-03-06T07:20:14.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54584/0 (socket says 172.21.15.17:54584) 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb286d9d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb2880c5910 0x7fb288158a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59758/0 (socket says 172.21.15.17:59758) 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb27ffff640 1 -- 172.21.15.17:0/50798989 learned_addr learned my addr 172.21.15.17:0/50798989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 -- 172.21.15.17:0/50798989 >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 msgr2=0x7fb2880c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 --2- 172.21.15.17:0/50798989 >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 -- 172.21.15.17:0/50798989 >> v1:172.21.15.163:6789/0 conn(0x7fb2880c5910 legacy=0x7fb288158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb28659c640 1 -- 172.21.15.17:0/50798989 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880c7ba0 con 0x7fb28814bcc0 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb285d9b640 1 -- 172.21.15.17:0/50798989 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561884397 0 0) 0x7fb2880c75e0 con 0x7fb2880c5910 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.893+0000 7fb27ffff640 1 --2- 172.21.15.17:0/50798989 >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb28659c640 1 --2- 172.21.15.17:0/50798989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8097f46c6e023aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb285d9b640 1 -- 172.21.15.17:0/50798989 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278034070 con 0x7fb28814bcc0 2024-03-06T07:20:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb285d9b640 1 -- 172.21.15.17:0/50798989 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278030610 con 0x7fb28814bcc0 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb285d9b640 1 -- 172.21.15.17:0/50798989 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780309e0 con 0x7fb28814bcc0 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb28cdd7640 1 -- 172.21.15.17:0/50798989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 msgr2=0x7fb288155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.894+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/50798989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2780097a0 tx=0x7fb27802b960 comp rx=0 tx=0).stop 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 -- 172.21.15.17:0/50798989 shutdown_connections 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/50798989 >> v2:172.21.15.163:3300/0 conn(0x7fb2880be150 0x7fb2880c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/50798989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb28814bcc0 0x7fb288155a00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 -- 172.21.15.17:0/50798989 >> 172.21.15.17:0/50798989 conn(0x7fb2880ac600 msgr2=0x7fb2880aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 -- 172.21.15.17:0/50798989 shutdown_connections 2024-03-06T07:20:14.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.895+0000 7fb28cdd7640 1 -- 172.21.15.17:0/50798989 wait complete. 2024-03-06T07:20:14.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.896+0000 7fb28cdd7640 1 Processor -- start 2024-03-06T07:20:14.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.896+0000 7fb28cdd7640 1 -- start start 2024-03-06T07:20:14.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.896+0000 7fb28cdd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.896+0000 7fb28cdd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 0x7fb288151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb28814bcc0 0x7fb2880c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2881592a0 con 0x7fb2880c5910 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb288159120 con 0x7fb28814bcc0 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb288159420 con 0x7fb2880be150 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 0x7fb288151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 0x7fb288151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33358/0 (socket says 172.21.15.17:33358) 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27ffff640 1 -- 172.21.15.17:0/4116500702 learned_addr learned my addr 172.21.15.17:0/4116500702 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54592/0 (socket says 172.21.15.17:54592) 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb286d9d640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb28814bcc0 0x7fb2880c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 -- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb28814bcc0 msgr2=0x7fb2880c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb28814bcc0 0x7fb2880c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 -- 172.21.15.17:0/4116500702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 msgr2=0x7fb288151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:14.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 0x7fb288151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 -- 172.21.15.17:0/4116500702 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28814ade0 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28659c640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb278033000 tx=0x7fb2780046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278034040 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27803aaa0 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2880be760 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780443f0 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.897+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2880c4a50 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.898+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2880be550 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.899+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb27804b060 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.899+0000 7fb27effd640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 0x7fb26806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.899+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb2780ac150 con 0x7fb2880be150 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.899+0000 7fb27ffff640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 0x7fb26806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:14.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.900+0000 7fb27ffff640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 0x7fb26806d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb270009630 tx=0x7fb270009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:14.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:14.904+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb27803f070 con 0x7fb2880be150 2024-03-06T07:20:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.084+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb28804f330 con 0x7fb2880be150 2024-03-06T07:20:15.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.085+0000 7fb27effd640 1 -- 172.21.15.17:0/4116500702 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb27807a6b0 con 0x7fb2880be150 2024-03-06T07:20:15.087 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 msgr2=0x7fb26806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 0x7fb26806d9c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb270009630 tx=0x7fb270009240 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 msgr2=0x7fb2880c07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb278033000 tx=0x7fb2780046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 shutdown_connections 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb26806b500 0x7fb26806d9c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb28814bcc0 0x7fb2880c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2880c5910 0x7fb288151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 --2- 172.21.15.17:0/4116500702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2880be150 0x7fb2880c07b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 >> 172.21.15.17:0/4116500702 conn(0x7fb2880ac600 msgr2=0x7fb2880bd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 shutdown_connections 2024-03-06T07:20:15.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.087+0000 7fb28cdd7640 1 -- 172.21.15.17:0/4116500702 wait complete. 2024-03-06T07:20:15.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:15.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.202+0000 7f9b00676640 1 Processor -- start 2024-03-06T07:20:15.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.202+0000 7f9b00676640 1 -- start start 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9b00676640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9af814bcc0 0x7f9af8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9b00676640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80c75e0 con 0x7f9af8158a10 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9b00676640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80c78c0 con 0x7f9af80be150 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9b00676640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80c7f10 con 0x7f9af814bcc0 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afec13640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9af8158a10 0x7f9af8158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47670/0 (socket says 172.21.15.17:47670) 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afec13640 1 -- 172.21.15.17:0/4087898137 learned_addr learned my addr 172.21.15.17:0/4087898137 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afe412640 1 --2- 172.21.15.17:0/4087898137 >> v2:172.21.15.163:3300/0 conn(0x7f9af814bcc0 0x7f9af8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afd410640 1 -- 172.21.15.17:0/4087898137 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966975712 0 0) 0x7f9af80c75e0 con 0x7f9af8158a10 2024-03-06T07:20:15.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afd410640 1 -- 172.21.15.17:0/4087898137 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae4003520 con 0x7f9af8158a10 2024-03-06T07:20:15.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afe412640 1 -- 172.21.15.17:0/4087898137 >> v1:172.21.15.163:6789/0 conn(0x7f9af80be150 legacy=0x7f9af80c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afe412640 1 -- 172.21.15.17:0/4087898137 >> v1:172.21.15.17:6789/0 conn(0x7f9af8158a10 legacy=0x7f9af8158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.203+0000 7f9afe412640 1 -- 172.21.15.17:0/4087898137 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80c7ba0 con 0x7f9af814bcc0 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9afe412640 1 --2- 172.21.15.17:0/4087898137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af8155a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9af40097a0 tx=0x7f9af402b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f6c1c816800f23 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9afd410640 1 -- 172.21.15.17:0/4087898137 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af402f810 con 0x7f9af814bcc0 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9afd410640 1 -- 172.21.15.17:0/4087898137 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af402f9b0 con 0x7f9af814bcc0 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9afd410640 1 -- 172.21.15.17:0/4087898137 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af402fd80 con 0x7f9af814bcc0 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9b00676640 1 -- 172.21.15.17:0/4087898137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 msgr2=0x7f9af8155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.204+0000 7f9b00676640 1 --2- 172.21.15.17:0/4087898137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af8155a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9af40097a0 tx=0x7f9af402b9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.205+0000 7f9b00676640 1 -- 172.21.15.17:0/4087898137 shutdown_connections 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.205+0000 7f9b00676640 1 --2- 172.21.15.17:0/4087898137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af8155a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.205+0000 7f9b00676640 1 -- 172.21.15.17:0/4087898137 >> 172.21.15.17:0/4087898137 conn(0x7f9af80ac600 msgr2=0x7f9af80aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.205+0000 7f9b00676640 1 -- 172.21.15.17:0/4087898137 shutdown_connections 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.205+0000 7f9b00676640 1 -- 172.21.15.17:0/4087898137 wait complete. 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 Processor -- start 2024-03-06T07:20:15.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 -- start start 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af81ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8158a10 0x7f9af814a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80c5ae0 con 0x7f9af80be150 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af806d1d0 con 0x7f9af814bcc0 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9b00676640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80c5c60 con 0x7f9af8158a10 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9afe412640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9afe412640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33368/0 (socket says 172.21.15.17:33368) 2024-03-06T07:20:15.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.206+0000 7f9afe412640 1 -- 172.21.15.17:0/4034340978 learned_addr learned my addr 172.21.15.17:0/4034340978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afdc11640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af81ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afec13640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8158a10 0x7f9af814a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 -- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8158a10 msgr2=0x7f9af814a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8158a10 0x7f9af814a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 -- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 msgr2=0x7f9af81ee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af81ee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 -- 172.21.15.17:0/4034340978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af81eef20 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9afe412640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9af40098c0 tx=0x7f9af4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4047070 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4037830 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af8155030 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4037c20 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.207+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af814b5b0 con 0x7f9af80be150 2024-03-06T07:20:15.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.208+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af804f330 con 0x7f9af80be150 2024-03-06T07:20:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.208+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9af402d3a0 con 0x7f9af80be150 2024-03-06T07:20:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.209+0000 7f9aeeffd640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 0x7f9adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.209+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9af40ac090 con 0x7f9af80be150 2024-03-06T07:20:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.209+0000 7f9afdc11640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 0x7f9adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.210+0000 7f9afdc11640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 0x7f9adc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9af80c01c0 tx=0x7f9ae8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.213+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9af4033030 con 0x7f9af80be150 2024-03-06T07:20:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.393+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9af806cfb0 con 0x7f9af80be150 2024-03-06T07:20:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.393+0000 7f9aeeffd640 1 -- 172.21.15.17:0/4034340978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9af407a590 con 0x7f9af80be150 2024-03-06T07:20:15.395 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 msgr2=0x7f9adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 0x7f9adc06d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9af80c01c0 tx=0x7f9ae8009040 comp rx=0 tx=0).stop 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 msgr2=0x7f9af81ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9af40098c0 tx=0x7f9af4004730 comp rx=0 tx=0).stop 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 shutdown_connections 2024-03-06T07:20:15.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9adc06b4d0 0x7f9adc06d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8158a10 0x7f9af814a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af814bcc0 0x7f9af81ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 --2- 172.21.15.17:0/4034340978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9af80be150 0x7f9af81ee4a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 >> 172.21.15.17:0/4034340978 conn(0x7f9af80ac600 msgr2=0x7f9af80ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 shutdown_connections 2024-03-06T07:20:15.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.395+0000 7f9b00676640 1 -- 172.21.15.17:0/4034340978 wait complete. 2024-03-06T07:20:15.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.510+0000 7f0a1b93e640 1 Processor -- start 2024-03-06T07:20:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.510+0000 7f0a1b93e640 1 -- start start 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a1b93e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a140be150 0x7f0a14155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a1b93e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a14158a10 0x7f0a14158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a1b93e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140c75e0 con 0x7f0a1414bcc0 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a1b93e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140c7f10 con 0x7f0a140be150 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a1b93e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140c78c0 con 0x7f0a14158a10 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a140be150 0x7f0a14155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a140be150 0x7f0a14155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33376/0 (socket says 172.21.15.17:33376) 2024-03-06T07:20:15.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 -- 172.21.15.17:0/2220990645 learned_addr learned my addr 172.21.15.17:0/2220990645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a19edb640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a14158a10 0x7f0a14158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 -- 172.21.15.17:0/2220990645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a14158a10 msgr2=0x7f0a14158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a14158a10 0x7f0a14158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 -- 172.21.15.17:0/2220990645 >> v1:172.21.15.163:6789/0 conn(0x7f0a1414bcc0 legacy=0x7f0a140c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a18ed9640 1 -- 172.21.15.17:0/2220990645 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140c7ba0 con 0x7f0a140be150 2024-03-06T07:20:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.511+0000 7f0a0bfff640 1 -- 172.21.15.17:0/2220990645 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874291715 0 0) 0x7f0a140c75e0 con 0x7f0a1414bcc0 2024-03-06T07:20:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.512+0000 7f0a18ed9640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a140be150 0x7f0a14155a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009870 tx=0x7f0a0402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f092d4a92d47bcc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.512+0000 7f0a0bfff640 1 -- 172.21.15.17:0/2220990645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0402f840 con 0x7f0a140be150 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.512+0000 7f0a0bfff640 1 -- 172.21.15.17:0/2220990645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0402f9e0 con 0x7f0a140be150 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.512+0000 7f0a0bfff640 1 -- 172.21.15.17:0/2220990645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0402fdb0 con 0x7f0a140be150 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 -- 172.21.15.17:0/2220990645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a140be150 msgr2=0x7f0a14155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a140be150 0x7f0a14155a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009870 tx=0x7f0a0402ba20 comp rx=0 tx=0).stop 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 -- 172.21.15.17:0/2220990645 shutdown_connections 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a14158a10 0x7f0a14158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/2220990645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a140be150 0x7f0a14155a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 -- 172.21.15.17:0/2220990645 >> 172.21.15.17:0/2220990645 conn(0x7f0a140ac600 msgr2=0x7f0a140aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 -- 172.21.15.17:0/2220990645 shutdown_connections 2024-03-06T07:20:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.513+0000 7f0a1b93e640 1 -- 172.21.15.17:0/2220990645 wait complete. 2024-03-06T07:20:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.514+0000 7f0a1b93e640 1 Processor -- start 2024-03-06T07:20:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.514+0000 7f0a1b93e640 1 -- start start 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a140be150 0x7f0a140becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a1414bcc0 0x7f0a14154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140c5c60 con 0x7f0a14158a10 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140c5ae0 con 0x7f0a1414bcc0 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a141593c0 con 0x7f0a140be150 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33392/0 (socket says 172.21.15.17:33392) 2024-03-06T07:20:15.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 -- 172.21.15.17:0/802617139 learned_addr learned my addr 172.21.15.17:0/802617139 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a196da640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a140be150 0x7f0a140becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a18ed9640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a1414bcc0 0x7f0a14154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 -- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a140be150 msgr2=0x7f0a140becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a140be150 0x7f0a140becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 -- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a1414bcc0 msgr2=0x7f0a14154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a1414bcc0 0x7f0a14154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 -- 172.21.15.17:0/802617139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140bf420 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a19edb640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00efc0 tx=0x7f0a0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c018020 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c014480 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a141edf70 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.515+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c014870 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.516+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a141ee530 con 0x7f0a14158a10 2024-03-06T07:20:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.517+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1404f330 con 0x7f0a14158a10 2024-03-06T07:20:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.517+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a0c005d10 con 0x7f0a14158a10 2024-03-06T07:20:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.517+0000 7f0a09ffb640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.518+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a0c08af40 con 0x7f0a14158a10 2024-03-06T07:20:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.518+0000 7f0a196da640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.518+0000 7f0a196da640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 0x7f09f806d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000f60 tx=0x7f0a10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.522+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a0c022070 con 0x7f0a14158a10 2024-03-06T07:20:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.702+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0a140c83e0 con 0x7f0a14158a10 2024-03-06T07:20:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.703+0000 7f0a09ffb640 1 -- 172.21.15.17:0/802617139 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a0c010040 con 0x7f0a14158a10 2024-03-06T07:20:15.704 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 msgr2=0x7f09f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 0x7f09f806d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000f60 tx=0x7f0a10009040 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 msgr2=0x7f0a140c2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00efc0 tx=0x7f0a0c00c490 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 shutdown_connections 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f09f806b4d0 0x7f09f806d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a14158a10 0x7f0a140c2fb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a1414bcc0 0x7f0a14154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 --2- 172.21.15.17:0/802617139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a140be150 0x7f0a140becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 >> 172.21.15.17:0/802617139 conn(0x7f0a140ac600 msgr2=0x7f0a140c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 shutdown_connections 2024-03-06T07:20:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.705+0000 7f0a1b93e640 1 -- 172.21.15.17:0/802617139 wait complete. 2024-03-06T07:20:15.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.820+0000 7f9a1a3e2640 1 Processor -- start 2024-03-06T07:20:15.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.820+0000 7f9a1a3e2640 1 -- start start 2024-03-06T07:20:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1a3e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 0x7f9a14155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1a3e2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a140c64f0 0x7f9a140c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1a3e2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140c8e30 con 0x7f9a140c8950 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1a3e2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a14152ce0 con 0x7f9a140c64f0 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1a3e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c9110 con 0x7f9a14152900 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a137fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9a140c8950 0x7f9a140bf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47672/0 (socket says 172.21.15.17:47672) 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a137fe640 1 -- 172.21.15.17:0/1460280075 learned_addr learned my addr 172.21.15.17:0/1460280075 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a13fff640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 0x7f9a14155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1897f640 1 --2- 172.21.15.17:0/1460280075 >> v2:172.21.15.163:3300/0 conn(0x7f9a140c64f0 0x7f9a140c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a12ffd640 1 -- 172.21.15.17:0/1460280075 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160687579 0 0) 0x7f9a140c8e30 con 0x7f9a140c8950 2024-03-06T07:20:15.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a12ffd640 1 -- 172.21.15.17:0/1460280075 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99fc003520 con 0x7f9a140c8950 2024-03-06T07:20:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1897f640 1 -- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 msgr2=0x7f9a14155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1897f640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 0x7f9a14155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1897f640 1 -- 172.21.15.17:0/1460280075 >> v1:172.21.15.17:6789/0 conn(0x7f9a140c8950 legacy=0x7f9a140bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a1897f640 1 -- 172.21.15.17:0/1460280075 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140c93f0 con 0x7f9a140c64f0 2024-03-06T07:20:15.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.821+0000 7f9a13fff640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 0x7f9a14155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.822+0000 7f9a1897f640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a140c64f0 0x7f9a140c69a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009d90 tx=0x7f9a0c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15233ab7236c737 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.822+0000 7f9a12ffd640 1 -- 172.21.15.17:0/1460280075 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c039070 con 0x7f9a140c64f0 2024-03-06T07:20:15.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.822+0000 7f9a12ffd640 1 -- 172.21.15.17:0/1460280075 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c02cb30 con 0x7f9a140c64f0 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.822+0000 7f9a12ffd640 1 -- 172.21.15.17:0/1460280075 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c034440 con 0x7f9a140c64f0 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a140c64f0 msgr2=0x7f9a140c69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a140c64f0 0x7f9a140c69a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009d90 tx=0x7f9a0c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/1460280075 shutdown_connections 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a140c64f0 0x7f9a140c69a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/1460280075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a14152900 0x7f9a14155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/1460280075 >> 172.21.15.17:0/1460280075 conn(0x7f9a140ac860 msgr2=0x7f9a140acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/1460280075 shutdown_connections 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.823+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/1460280075 wait complete. 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.824+0000 7f9a1a3e2640 1 Processor -- start 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.824+0000 7f9a1a3e2640 1 -- start start 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 0x7f9a141eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a14152900 0x7f9a1406c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c0180 con 0x7f9a140c8950 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c0000 con 0x7f9a14152900 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1a3e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c55f0 con 0x7f9a140c64f0 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33396/0 (socket says 172.21.15.17:33396) 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 -- 172.21.15.17:0/496675296 learned_addr learned my addr 172.21.15.17:0/496675296 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a13fff640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 0x7f9a141eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a1897f640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a14152900 0x7f9a1406c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 -- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 msgr2=0x7f9a141eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 0x7f9a141eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 -- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a14152900 msgr2=0x7f9a1406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a14152900 0x7f9a1406c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.825+0000 7f9a137fe640 1 -- 172.21.15.17:0/496675296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a141569c0 con 0x7f9a140c8950 2024-03-06T07:20:15.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a13fff640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 0x7f9a141eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a137fe640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c460 tx=0x7f9a0800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08017070 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08005ce0 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1414b8b0 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08007290 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.826+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a14156c80 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.827+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1404f330 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.827+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a0801a070 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.828+0000 7f9a10ff9640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 0x7f9a0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.828+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9a08089710 con 0x7f9a140c8950 2024-03-06T07:20:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.828+0000 7f9a13fff640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 0x7f9a0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.829+0000 7f9a13fff640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 0x7f9a0006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009560 tx=0x7f9a04009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:15.832+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a08057c90 con 0x7f9a140c8950 2024-03-06T07:20:16.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.024+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a140c5f70 con 0x7f9a140c8950 2024-03-06T07:20:16.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.025+0000 7f9a10ff9640 1 -- 172.21.15.17:0/496675296 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9a0805bd30 con 0x7f9a140c8950 2024-03-06T07:20:16.026 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-06T07:20:16.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 msgr2=0x7f9a0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 0x7f9a0006d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009560 tx=0x7f9a04009190 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 msgr2=0x7f9a1414aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c460 tx=0x7f9a0800c920 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 shutdown_connections 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a0006b500 0x7f9a0006d9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a14152900 0x7f9a1406c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a140c8950 0x7f9a1414aea0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.026+0000 7f9a1a3e2640 1 --2- 172.21.15.17:0/496675296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a140c64f0 0x7f9a141eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.027+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 >> 172.21.15.17:0/496675296 conn(0x7f9a140ac860 msgr2=0x7f9a140acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.027+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 shutdown_connections 2024-03-06T07:20:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.027+0000 7f9a1a3e2640 1 -- 172.21.15.17:0/496675296 wait complete. 2024-03-06T07:20:16.039 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:16.039 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:16.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.194+0000 7f808cd0c640 1 Processor -- start 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.194+0000 7f808cd0c640 1 -- start start 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808cd0c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 0x7f8088155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808cd0c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80880c5330 0x7f80880c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808cd0c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8088156180 con 0x7f80880c8a00 2024-03-06T07:20:16.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808cd0c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80881587b0 con 0x7f80880c5330 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808cd0c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8088159630 con 0x7f8088152920 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80880c5330 0x7f80880c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80880c5330 0x7f80880c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33398/0 (socket says 172.21.15.17:33398) 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 -- 172.21.15.17:0/2745392722 learned_addr learned my addr 172.21.15.17:0/2745392722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 0x7f8088155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f807ffff640 1 --1- 172.21.15.17:0/2745392722 >> v1:172.21.15.17:6789/0 conn(0x7f80880c8a00 0x7f8088158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47688/0 (socket says 172.21.15.17:47688) 2024-03-06T07:20:16.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 -- 172.21.15.17:0/2745392722 >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 msgr2=0x7f8088155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 --2- 172.21.15.17:0/2745392722 >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 0x7f8088155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 -- 172.21.15.17:0/2745392722 >> v1:172.21.15.17:6789/0 conn(0x7f80880c8a00 legacy=0x7f8088158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8086d9d640 1 -- 172.21.15.17:0/2745392722 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880c7760 con 0x7f80880c5330 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f8085d9b640 1 -- 172.21.15.17:0/2745392722 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673358584 0 0) 0x7f8070001110 con 0x7f80880c8a00 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.195+0000 7f808659c640 1 --2- 172.21.15.17:0/2745392722 >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 0x7f8088155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.196+0000 7f8086d9d640 1 --2- 172.21.15.17:0/2745392722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80880c5330 0x7f80880c57a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f80880be490 tx=0x7f808002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e032e660de7a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.196+0000 7f8085d9b640 1 -- 172.21.15.17:0/2745392722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808002ca80 con 0x7f80880c5330 2024-03-06T07:20:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.196+0000 7f8085d9b640 1 -- 172.21.15.17:0/2745392722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808002cc20 con 0x7f80880c5330 2024-03-06T07:20:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.196+0000 7f8085d9b640 1 -- 172.21.15.17:0/2745392722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800343f0 con 0x7f80880c5330 2024-03-06T07:20:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 -- 172.21.15.17:0/2745392722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80880c5330 msgr2=0x7f80880c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 --2- 172.21.15.17:0/2745392722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80880c5330 0x7f80880c57a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f80880be490 tx=0x7f808002bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 -- 172.21.15.17:0/2745392722 shutdown_connections 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 --2- 172.21.15.17:0/2745392722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80880c5330 0x7f80880c57a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 --2- 172.21.15.17:0/2745392722 >> v2:172.21.15.163:3300/0 conn(0x7f8088152920 0x7f8088155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 -- 172.21.15.17:0/2745392722 >> 172.21.15.17:0/2745392722 conn(0x7f80880ac560 msgr2=0x7f80880ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 -- 172.21.15.17:0/2745392722 shutdown_connections 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.197+0000 7f808cd0c640 1 -- 172.21.15.17:0/2745392722 wait complete. 2024-03-06T07:20:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.198+0000 7f808cd0c640 1 Processor -- start 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.198+0000 7f808cd0c640 1 -- start start 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.198+0000 7f808cd0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80880c5330 0x7f808814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80880c8a00 0x7f8088147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8088158a00 con 0x7f8088152920 2024-03-06T07:20:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80880c7d20 con 0x7f80880c5330 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808806d030 con 0x7f80880c8a00 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33402/0 (socket says 172.21.15.17:33402) 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 -- 172.21.15.17:0/538966601 learned_addr learned my addr 172.21.15.17:0/538966601 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808659c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80880c5330 0x7f808814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f807ffff640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80880c8a00 0x7f8088147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 -- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80880c8a00 msgr2=0x7f8088147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80880c8a00 0x7f8088147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 -- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80880c5330 msgr2=0x7f808814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80880c5330 0x7f808814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 -- 172.21.15.17:0/538966601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880abe40 con 0x7f8088152920 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f8086d9d640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8080009f40 tx=0x7f8080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080043020 con 0x7f8088152920 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808003e450 con 0x7f8088152920 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80880ac120 con 0x7f8088152920 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080009510 con 0x7f8088152920 2024-03-06T07:20:16.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.199+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80881478c0 con 0x7f8088152920 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.200+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80880ac320 con 0x7f8088152920 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.201+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8080046020 con 0x7f8088152920 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.201+0000 7f807effd640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.201+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f80800acac0 con 0x7f8088152920 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.201+0000 7f808659c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.202+0000 7f808659c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8078009560 tx=0x7f8078009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.206+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f80880ac320 con 0x7f8088152920 2024-03-06T07:20:16.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.386+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f808804f330 con 0x7f8088152920 2024-03-06T07:20:16.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.387+0000 7f807effd640 1 -- 172.21.15.17:0/538966601 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f808004d030 con 0x7f8088152920 2024-03-06T07:20:16.389 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:16.389 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 msgr2=0x7f806806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 0x7f806806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8078009560 tx=0x7f8078009190 comp rx=0 tx=0).stop 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 msgr2=0x7f80880aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8080009f40 tx=0x7f8080004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 shutdown_connections 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f806806b4d0 0x7f806806d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8088152920 0x7f80880aae30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80880c8a00 0x7f8088147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 --2- 172.21.15.17:0/538966601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80880c5330 0x7f808814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 >> 172.21.15.17:0/538966601 conn(0x7f80880ac560 msgr2=0x7f808814ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 shutdown_connections 2024-03-06T07:20:16.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.389+0000 7f808cd0c640 1 -- 172.21.15.17:0/538966601 wait complete. 2024-03-06T07:20:16.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:16.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:16.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.505+0000 7f02b695c640 1 Processor -- start 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.505+0000 7f02b695c640 1 -- start start 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b695c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02b00b0070 0x7f02b0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b695c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b695c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b00b0450 con 0x7f02b00c5380 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b695c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b006c020 con 0x7f02b00b0070 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b695c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b006cf20 con 0x7f02b006ca70 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02affff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02b00b0070 0x7f02b0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02affff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02b00b0070 0x7f02b0155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33418/0 (socket says 172.21.15.17:33418) 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02affff640 1 -- 172.21.15.17:0/370596999 learned_addr learned my addr 172.21.15.17:0/370596999 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02b4ef9640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f02b00c5380 0x7f02b014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59786/0 (socket says 172.21.15.17:59786) 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.506+0000 7f02af7fe640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02affff640 1 -- 172.21.15.17:0/370596999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 msgr2=0x7f02b00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02affff640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b00c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02affff640 1 -- 172.21.15.17:0/370596999 >> v1:172.21.15.163:6789/0 conn(0x7f02b00c5380 legacy=0x7f02b014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02aeffd640 1 -- 172.21.15.17:0/370596999 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672885379 0 0) 0x7f02a8001490 con 0x7f02b00c5380 2024-03-06T07:20:16.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02affff640 1 -- 172.21.15.17:0/370596999 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00c77f0 con 0x7f02b00b0070 2024-03-06T07:20:16.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02af7fe640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.507+0000 7f02affff640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b0155c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009870 tx=0x7f02a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261689048aa1932b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02aeffd640 1 -- 172.21.15.17:0/370596999 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0035020 con 0x7f02b00b0070 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02aeffd640 1 -- 172.21.15.17:0/370596999 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0031640 con 0x7f02b00b0070 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02aeffd640 1 -- 172.21.15.17:0/370596999 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a002d040 con 0x7f02b00b0070 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 -- 172.21.15.17:0/370596999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 msgr2=0x7f02b0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b0155c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009870 tx=0x7f02a002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 -- 172.21.15.17:0/370596999 shutdown_connections 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 --2- 172.21.15.17:0/370596999 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b0155c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 -- 172.21.15.17:0/370596999 >> 172.21.15.17:0/370596999 conn(0x7f02b00ac590 msgr2=0x7f02b00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 -- 172.21.15.17:0/370596999 shutdown_connections 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.508+0000 7f02b695c640 1 -- 172.21.15.17:0/370596999 wait complete. 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.509+0000 7f02b695c640 1 Processor -- start 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.509+0000 7f02b695c640 1 -- start start 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b01502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 0x7f02b00c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b006c550 con 0x7f02b00b0070 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b006c270 con 0x7f02b00c5380 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b695c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b006da80 con 0x7f02b006ca70 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33432/0 (socket says 172.21.15.17:33432) 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02af7fe640 1 -- 172.21.15.17:0/23704406 learned_addr learned my addr 172.21.15.17:0/23704406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02b4ef9640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 0x7f02b00c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.510+0000 7f02affff640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b01502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 -- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 msgr2=0x7f02b01502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b01502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 -- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 msgr2=0x7f02b00c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 0x7f02b00c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 -- 172.21.15.17:0/23704406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0150b80 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02b4ef9640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 0x7f02b00c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02af7fe640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02a400e960 tx=0x7f02a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4017070 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a400c9a0 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a400cd70 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b0151500 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.511+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b01517c0 con 0x7f02b00b0070 2024-03-06T07:20:16.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.512+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b004f330 con 0x7f02b00b0070 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.512+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02a401a070 con 0x7f02b00b0070 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.513+0000 7f02acff9640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 0x7f029c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.513+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f02a408a230 con 0x7f02b00b0070 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.513+0000 7f02affff640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 0x7f029c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.513+0000 7f02affff640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 0x7f029c06d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02a0036000 tx=0x7f02a003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.517+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f02b0150b80 con 0x7f02b00b0070 2024-03-06T07:20:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.698+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02b00acc70 con 0x7f02b00b0070 2024-03-06T07:20:16.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.698+0000 7f02acff9640 1 -- 172.21.15.17:0/23704406 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02a40587b0 con 0x7f02b00b0070 2024-03-06T07:20:16.700 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 msgr2=0x7f029c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 0x7f029c06d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f02a0036000 tx=0x7f02a003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 msgr2=0x7f02b00c3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02a400e960 tx=0x7f02a400ee20 comp rx=0 tx=0).stop 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 shutdown_connections 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f029c06b500 0x7f029c06d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02b00c5380 0x7f02b00c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b00b0070 0x7f02b00c3cf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 --2- 172.21.15.17:0/23704406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02b006ca70 0x7f02b01502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 >> 172.21.15.17:0/23704406 conn(0x7f02b00ac590 msgr2=0x7f02b00ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 shutdown_connections 2024-03-06T07:20:16.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.700+0000 7f02b695c640 1 -- 172.21.15.17:0/23704406 wait complete. 2024-03-06T07:20:16.712 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.819+0000 7fdbd4fc7640 1 Processor -- start 2024-03-06T07:20:16.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.819+0000 7fdbd4fc7640 1 -- start start 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbd4fc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd0155c30 0x7fdbd006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbd4fc7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdbd00c5380 0x7fdbd00c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbd4fc7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00b0450 con 0x7fdbd00b0070 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbd4fc7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd006cfb0 con 0x7fdbd00c5380 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbd4fc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd006c020 con 0x7fdbd0155c30 2024-03-06T07:20:16.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdbd00c5380 0x7fdbd00c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdbd00c5380 0x7fdbd00c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33442/0 (socket says 172.21.15.17:33442) 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbce59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdbd00b0070 0x7fdbd00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47698/0 (socket says 172.21.15.17:47698) 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 -- 172.21.15.17:0/4132289474 learned_addr learned my addr 172.21.15.17:0/4132289474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbcdd9b640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd0155c30 0x7fdbd006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 -- 172.21.15.17:0/4132289474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd0155c30 msgr2=0x7fdbd006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd0155c30 0x7fdbd006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.820+0000 7fdbced9d640 1 -- 172.21.15.17:0/4132289474 >> v1:172.21.15.17:6789/0 conn(0x7fdbd00b0070 legacy=0x7fdbd00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.821+0000 7fdbced9d640 1 -- 172.21.15.17:0/4132289474 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd01560e0 con 0x7fdbd00c5380 2024-03-06T07:20:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.821+0000 7fdbced9d640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd00c5380 0x7fdbd00c57f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e50 tx=0x7fdbc002bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b929b911e17c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.821+0000 7fdbcd59a640 1 -- 172.21.15.17:0/4132289474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc002c840 con 0x7fdbd00c5380 2024-03-06T07:20:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.821+0000 7fdbcd59a640 1 -- 172.21.15.17:0/4132289474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc002c9e0 con 0x7fdbd00c5380 2024-03-06T07:20:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.821+0000 7fdbcd59a640 1 -- 172.21.15.17:0/4132289474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc002cdb0 con 0x7fdbd00c5380 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/4132289474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd00c5380 msgr2=0x7fdbd00c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd00c5380 0x7fdbd00c57f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e50 tx=0x7fdbc002bd20 comp rx=0 tx=0).stop 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/4132289474 shutdown_connections 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd00c5380 0x7fdbd00c57f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/4132289474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd0155c30 0x7fdbd006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/4132289474 >> 172.21.15.17:0/4132289474 conn(0x7fdbd00ac590 msgr2=0x7fdbd00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/4132289474 shutdown_connections 2024-03-06T07:20:16.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.822+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/4132289474 wait complete. 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.823+0000 7fdbd4fc7640 1 Processor -- start 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.823+0000 7fdbd4fc7640 1 -- start start 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd00b0070 0x7fdbd00c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd00c5380 0x7fdbd00af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd006c270 con 0x7fdbd0155c30 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c7ad0 con 0x7fdbd00c5380 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbd4fc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd006c550 con 0x7fdbd00b0070 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33446/0 (socket says 172.21.15.17:33446) 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 -- 172.21.15.17:0/725098959 learned_addr learned my addr 172.21.15.17:0/725098959 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbcdd9b640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd00c5380 0x7fdbd00af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbce59c640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd00b0070 0x7fdbd00c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 -- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd00b0070 msgr2=0x7fdbd00c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd00b0070 0x7fdbd00c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 -- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd00c5380 msgr2=0x7fdbd00af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd00c5380 0x7fdbd00af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.824+0000 7fdbced9d640 1 -- 172.21.15.17:0/725098959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0159550 con 0x7fdbd0155c30 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbced9d640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e50 tx=0x7fdbc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0043020 con 0x7fdbd0155c30 2024-03-06T07:20:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0007930 con 0x7fdbd0155c30 2024-03-06T07:20:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd01ee8a0 con 0x7fdbd0155c30 2024-03-06T07:20:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0009510 con 0x7fdbd0155c30 2024-03-06T07:20:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.825+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd00c4000 con 0x7fdbd0155c30 2024-03-06T07:20:16.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.826+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd004f330 con 0x7fdbd0155c30 2024-03-06T07:20:16.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.826+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbc0046020 con 0x7fdbd0155c30 2024-03-06T07:20:16.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.827+0000 7fdbc6ffd640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:16.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.827+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdbc00ac660 con 0x7fdbd0155c30 2024-03-06T07:20:16.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.827+0000 7fdbce59c640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:16.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.827+0000 7fdbce59c640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 0x7fdbb006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8000f60 tx=0x7fdbc8007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:16.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:16.831+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdbc004d030 con 0x7fdbd0155c30 2024-03-06T07:20:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.014+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdbd00af730 con 0x7fdbd0155c30 2024-03-06T07:20:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.014+0000 7fdbc6ffd640 1 -- 172.21.15.17:0/725098959 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdbc007abe0 con 0x7fdbd0155c30 2024-03-06T07:20:17.016 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 msgr2=0x7fdbb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 0x7fdbb006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8000f60 tx=0x7fdbc8007e20 comp rx=0 tx=0).stop 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 msgr2=0x7fdbd0156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009e50 tx=0x7fdbc0004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 shutdown_connections 2024-03-06T07:20:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbd0155c30 0x7fdbd0156820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbd00c5380 0x7fdbd00af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 --2- 172.21.15.17:0/725098959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbd00b0070 0x7fdbd00c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 >> 172.21.15.17:0/725098959 conn(0x7fdbd00ac590 msgr2=0x7fdbd00adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 shutdown_connections 2024-03-06T07:20:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.016+0000 7fdbd4fc7640 1 -- 172.21.15.17:0/725098959 wait complete. 2024-03-06T07:20:17.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 Processor -- start 2024-03-06T07:20:17.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 -- start start 2024-03-06T07:20:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72480b0070 0x7f7248155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 0x7f72480c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480b0450 con 0x7f72480c5380 2024-03-06T07:20:17.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724806c020 con 0x7f72480b0070 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.131+0000 7f724ef02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724806cf20 con 0x7f724806ca70 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72480b0070 0x7f7248155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724d49f640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f72480c5380 0x7f724814e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59790/0 (socket says 172.21.15.17:59790) 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724d49f640 1 -- 172.21.15.17:0/1388865680 learned_addr learned my addr 172.21.15.17:0/1388865680 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72480b0070 0x7f7248155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56726/0 (socket says 172.21.15.17:56726) 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f7247fff640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 0x7f72480c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 -- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 msgr2=0x7f72480c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 0x7f72480c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 -- 172.21.15.17:0/1388865680 >> v1:172.21.15.163:6789/0 conn(0x7f72480c5380 legacy=0x7f724814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f72477fe640 1 -- 172.21.15.17:0/1388865680 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152377226 0 0) 0x7f7240001490 con 0x7f72480c5380 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f724cc9e640 1 -- 172.21.15.17:0/1388865680 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480c77f0 con 0x7f72480b0070 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.132+0000 7f7247fff640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 0x7f72480c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724cc9e640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72480b0070 0x7f7248155c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f723c0097a0 tx=0x7f723c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a5a132115d4bd67 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f72477fe640 1 -- 172.21.15.17:0/1388865680 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0315a0 con 0x7f72480b0070 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f72477fe640 1 -- 172.21.15.17:0/1388865680 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c037070 con 0x7f72480b0070 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f72477fe640 1 -- 172.21.15.17:0/1388865680 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c031960 con 0x7f72480b0070 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 -- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72480b0070 msgr2=0x7f7248155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72480b0070 0x7f7248155c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f723c0097a0 tx=0x7f723c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 -- 172.21.15.17:0/1388865680 shutdown_connections 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f724806ca70 0x7f72480c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 --2- 172.21.15.17:0/1388865680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72480b0070 0x7f7248155c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.133+0000 7f724ef02640 1 -- 172.21.15.17:0/1388865680 >> 172.21.15.17:0/1388865680 conn(0x7f72480ac590 msgr2=0x7f72480ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.134+0000 7f724ef02640 1 -- 172.21.15.17:0/1388865680 shutdown_connections 2024-03-06T07:20:17.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.134+0000 7f724ef02640 1 -- 172.21.15.17:0/1388865680 wait complete. 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 Processor -- start 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 -- start start 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806ca70 0x7f7248154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 0x7f724814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724806c550 con 0x7f72480b0070 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724806c270 con 0x7f724806ca70 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f724ef02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724806da80 con 0x7f72480c5380 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.135+0000 7f7247fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33458/0 (socket says 172.21.15.17:33458) 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f724cc9e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806ca70 0x7f7248154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 -- 172.21.15.17:0/2385453342 learned_addr learned my addr 172.21.15.17:0/2385453342 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f724d49f640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 0x7f724814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 -- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 msgr2=0x7f724814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 0x7f724814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 -- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806ca70 msgr2=0x7f7248154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806ca70 0x7f7248154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 -- 172.21.15.17:0/2385453342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724814a980 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f724d49f640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 0x7f724814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f7247fff640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f723400e960 tx=0x7f723400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234017070 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723400c9a0 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724814eb10 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400cd70 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.136+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7248154da0 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.137+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724804f330 con 0x7f72480b0070 2024-03-06T07:20:17.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.138+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f723401a070 con 0x7f72480b0070 2024-03-06T07:20:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.138+0000 7f72457fa640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 0x7f722c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.138+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f723408a600 con 0x7f72480b0070 2024-03-06T07:20:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.138+0000 7f724cc9e640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 0x7f722c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.139+0000 7f724cc9e640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 0x7f722c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f723c036000 tx=0x7f723c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.142+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7234058b80 con 0x7f72480b0070 2024-03-06T07:20:17.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.321+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f724814e8e0 con 0x7f72480b0070 2024-03-06T07:20:17.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.321+0000 7f72457fa640 1 -- 172.21.15.17:0/2385453342 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f723405cc20 con 0x7f72480b0070 2024-03-06T07:20:17.323 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:17.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 msgr2=0x7f722c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 0x7f722c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f723c036000 tx=0x7f723c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:17.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 msgr2=0x7f7248149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f723400e960 tx=0x7f723400ee20 comp rx=0 tx=0).stop 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 shutdown_connections 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f722c06b4d0 0x7f722c06d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72480c5380 0x7f724814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72480b0070 0x7f7248149c00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 --2- 172.21.15.17:0/2385453342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f724806ca70 0x7f7248154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 >> 172.21.15.17:0/2385453342 conn(0x7f72480ac590 msgr2=0x7f7248151430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.323+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 shutdown_connections 2024-03-06T07:20:17.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.324+0000 7f724ef02640 1 -- 172.21.15.17:0/2385453342 wait complete. 2024-03-06T07:20:17.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:17.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.438+0000 7fbd8300e640 1 Processor -- start 2024-03-06T07:20:17.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.438+0000 7fbd8300e640 1 -- start start 2024-03-06T07:20:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd8300e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd8300e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd8300e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c156060 con 0x7fbd7c06d750 2024-03-06T07:20:17.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd8300e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c155b30 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd8300e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0a8430 con 0x7fbd7c06db30 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd815ab640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd7bfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd815ab640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54712/0 (socket says 172.21.15.17:54712) 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd815ab640 1 -- 172.21.15.17:0/331780896 learned_addr learned my addr 172.21.15.17:0/331780896 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.439+0000 7fbd7bfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56746/0 (socket says 172.21.15.17:56746) 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd815ab640 1 -- 172.21.15.17:0/331780896 >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 msgr2=0x7fbd7c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd815ab640 1 --2- 172.21.15.17:0/331780896 >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd815ab640 1 -- 172.21.15.17:0/331780896 >> v1:172.21.15.163:6789/0 conn(0x7fbd7c06d750 legacy=0x7fbd7c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd815ab640 1 -- 172.21.15.17:0/331780896 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c155d80 con 0x7fbd7c06db30 2024-03-06T07:20:17.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd7bfff640 1 --2- 172.21.15.17:0/331780896 >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.440+0000 7fbd815ab640 1 --2- 172.21.15.17:0/331780896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009870 tx=0x7fbd6c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53f97bdcf0a8bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd7b7fe640 1 -- 172.21.15.17:0/331780896 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0317c0 con 0x7fbd7c06db30 2024-03-06T07:20:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd7b7fe640 1 -- 172.21.15.17:0/331780896 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c037070 con 0x7fbd7c06db30 2024-03-06T07:20:17.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd7b7fe640 1 -- 172.21.15.17:0/331780896 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c031b80 con 0x7fbd7c06db30 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 -- 172.21.15.17:0/331780896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 msgr2=0x7fbd7c0c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 --2- 172.21.15.17:0/331780896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009870 tx=0x7fbd6c02b860 comp rx=0 tx=0).stop 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 -- 172.21.15.17:0/331780896 shutdown_connections 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 --2- 172.21.15.17:0/331780896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c06db30 0x7fbd7c0c1e90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 --2- 172.21.15.17:0/331780896 >> v2:172.21.15.163:3300/0 conn(0x7fbd7c0c3a70 0x7fbd7c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 -- 172.21.15.17:0/331780896 >> 172.21.15.17:0/331780896 conn(0x7fbd7c0c5560 msgr2=0x7fbd7c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 -- 172.21.15.17:0/331780896 shutdown_connections 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.441+0000 7fbd8300e640 1 -- 172.21.15.17:0/331780896 wait complete. 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.442+0000 7fbd8300e640 1 Processor -- start 2024-03-06T07:20:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.442+0000 7fbd8300e640 1 -- start start 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7c06d750 0x7fbd7c1ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 0x7fbd7c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0a7e50 con 0x7fbd7c06db30 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0a7cd0 con 0x7fbd7c06d750 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd8300e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0a7fd0 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd7bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33474/0 (socket says 172.21.15.17:33474) 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd7bfff640 1 -- 172.21.15.17:0/3280527413 learned_addr learned my addr 172.21.15.17:0/3280527413 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 0x7fbd7c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd80daa640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7c06d750 0x7fbd7c1ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 -- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7c06d750 msgr2=0x7fbd7c1ee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7c06d750 0x7fbd7c1ee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 -- 172.21.15.17:0/3280527413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 msgr2=0x7fbd7c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd815ab640 1 -- 172.21.15.17:0/3280527413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c1ee840 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.443+0000 7fbd7bfff640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd815ab640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 0x7fbd7c154900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c033000 tx=0x7fbd6c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c032650 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c032830 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c1eeb20 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c032c20 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.444+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c1ee540 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.445+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c04f330 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.445+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd6c04e440 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.446+0000 7fbd797fa640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.446+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbd6c0ac080 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.446+0000 7fbd80daa640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.446+0000 7fbd80daa640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0c7380 tx=0x7fbd74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.451+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd6c040070 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.631+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbd7c155010 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.632+0000 7fbd797fa640 1 -- 172.21.15.17:0/3280527413 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbd6c07a600 con 0x7fbd7c0c3a70 2024-03-06T07:20:17.634 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 msgr2=0x7fbd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0c7380 tx=0x7fbd74009040 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 msgr2=0x7fbd7c154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 0x7fbd7c154900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c033000 tx=0x7fbd6c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 shutdown_connections 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd7c0c3a70 0x7fbd7c154900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd7c06db30 0x7fbd7c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 --2- 172.21.15.17:0/3280527413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd7c06d750 0x7fbd7c1ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 >> 172.21.15.17:0/3280527413 conn(0x7fbd7c0c5560 msgr2=0x7fbd7c072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 shutdown_connections 2024-03-06T07:20:17.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.634+0000 7fbd8300e640 1 -- 172.21.15.17:0/3280527413 wait complete. 2024-03-06T07:20:17.646 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:17.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 Processor -- start 2024-03-06T07:20:17.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 -- start start 2024-03-06T07:20:17.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 0x7fdae00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae00733f0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae00c82c0 con 0x7fdae00724d0 2024-03-06T07:20:17.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae6bb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0072d70 con 0x7fdae0152b40 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae5152640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdae00a7ff0 0x7fdae015fb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47704/0 (socket says 172.21.15.17:47704) 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae5152640 1 -- 172.21.15.17:0/3206138450 learned_addr learned my addr 172.21.15.17:0/3206138450 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.747+0000 7fdae4951640 1 --2- 172.21.15.17:0/3206138450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 0x7fdae00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168790277 0 0) 0x7fdae00733f0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdac8003520 con 0x7fdae00a7ff0 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521427098 0 0) 0x7fdae00c82c0 con 0x7fdae00724d0 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdae4951640 1 -- 172.21.15.17:0/3206138450 >> v1:172.21.15.163:6789/0 conn(0x7fdae00724d0 legacy=0x7fdae01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdae4951640 1 -- 172.21.15.17:0/3206138450 >> v1:172.21.15.17:6789/0 conn(0x7fdae00a7ff0 legacy=0x7fdae015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.748+0000 7fdae4951640 1 -- 172.21.15.17:0/3206138450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae006fd30 con 0x7fdae0152b40 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae4951640 1 --2- 172.21.15.17:0/3206138450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 0x7fdae00c8810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdad00097d0 tx=0x7fdad002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d642c8a01fb3b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0035020 con 0x7fdae0152b40 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad0031800 con 0x7fdae0152b40 2024-03-06T07:20:17.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdadf7fe640 1 -- 172.21.15.17:0/3206138450 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0031bb0 con 0x7fdae0152b40 2024-03-06T07:20:17.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3206138450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 msgr2=0x7fdae00c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3206138450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 0x7fdae00c8810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdad00097d0 tx=0x7fdad002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3206138450 shutdown_connections 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3206138450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae0152b40 0x7fdae00c8810 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3206138450 >> 172.21.15.17:0/3206138450 conn(0x7fdae006b970 msgr2=0x7fdae00aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3206138450 shutdown_connections 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.749+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3206138450 wait complete. 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.750+0000 7fdae6bb5640 1 Processor -- start 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 -- start start 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae00724d0 0x7fdae00ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdae0152b40 0x7fdae01ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00c8d50 con 0x7fdae00a7ff0 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae0070490 con 0x7fdae0152b40 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdae6bb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00c8ed0 con 0x7fdae00724d0 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdadffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdadffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33490/0 (socket says 172.21.15.17:33490) 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.751+0000 7fdadffff640 1 -- 172.21.15.17:0/3107130876 learned_addr learned my addr 172.21.15.17:0/3107130876 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdae4951640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae00724d0 0x7fdae00ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdae5152640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdae0152b40 0x7fdae01ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 -- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae00724d0 msgr2=0x7fdae00ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae00724d0 0x7fdae00ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 -- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdae0152b40 msgr2=0x7fdae01ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdae0152b40 0x7fdae01ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 -- 172.21.15.17:0/3107130876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae014a0c0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadffff640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c410 tx=0x7fdad400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4017070 con 0x7fdae00a7ff0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad40035f0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae014d1d0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad40039e0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.752+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae0151220 con 0x7fdae00a7ff0 2024-03-06T07:20:17.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.753+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae00ac2f0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.754+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdad401a070 con 0x7fdae00a7ff0 2024-03-06T07:20:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.754+0000 7fdadd7fa640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 0x7fdacc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.754+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdad4089870 con 0x7fdae00a7ff0 2024-03-06T07:20:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.754+0000 7fdae4951640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 0x7fdacc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.755+0000 7fdae4951640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 0x7fdacc06d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036000 tx=0x7fdad003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.758+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdad4057df0 con 0x7fdae00a7ff0 2024-03-06T07:20:17.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.958+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdae004f330 con 0x7fdae00a7ff0 2024-03-06T07:20:17.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.958+0000 7fdadd7fa640 1 -- 172.21.15.17:0/3107130876 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdad405be90 con 0x7fdae00a7ff0 2024-03-06T07:20:17.961 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 msgr2=0x7fdacc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 0x7fdacc06d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036000 tx=0x7fdad003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 msgr2=0x7fdae01ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdad400c410 tx=0x7fdad400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 shutdown_connections 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdacc06b500 0x7fdacc06d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdae0152b40 0x7fdae01ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdae00a7ff0 0x7fdae01ee2c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 --2- 172.21.15.17:0/3107130876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdae00724d0 0x7fdae00ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 >> 172.21.15.17:0/3107130876 conn(0x7fdae006b970 msgr2=0x7fdae0151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 shutdown_connections 2024-03-06T07:20:17.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:17.961+0000 7fdae6bb5640 1 -- 172.21.15.17:0/3107130876 wait complete. 2024-03-06T07:20:17.973 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-03-06T07:20:17.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:17.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-06T07:20:17.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[43188]: fuse finished with error 0 and tester_r 0 2024-03-06T07:20:18.000 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:18.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:20:18.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:18.021 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-06T07:20:18.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.225+0000 7f2f26ef4640 1 Processor -- start 2024-03-06T07:20:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.225+0000 7f2f26ef4640 1 -- start start 2024-03-06T07:20:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f26ef4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f2014f360 0x7f2f20155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f26ef4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f20158b60 0x7f2f20158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f26ef4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f200c73d0 con 0x7f2f200c88c0 2024-03-06T07:20:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f26ef4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200c8dd0 con 0x7f2f2014f360 2024-03-06T07:20:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f26ef4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200be2e0 con 0x7f2f20158b60 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f24c90640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f2014f360 0x7f2f20155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f17fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2f200c88c0 0x7f2f200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47720/0 (socket says 172.21.15.17:47720) 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f17fff640 1 -- 172.21.15.17:0/672460893 learned_addr learned my addr 172.21.15.17:0/672460893 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f25491640 1 --2- 172.21.15.17:0/672460893 >> v2:172.21.15.163:3300/0 conn(0x7f2f20158b60 0x7f2f20158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f24c90640 1 -- 172.21.15.17:0/672460893 >> v2:172.21.15.163:3300/0 conn(0x7f2f20158b60 msgr2=0x7f2f20158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f24c90640 1 --2- 172.21.15.17:0/672460893 >> v2:172.21.15.163:3300/0 conn(0x7f2f20158b60 0x7f2f20158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f24c90640 1 -- 172.21.15.17:0/672460893 >> v1:172.21.15.17:6789/0 conn(0x7f2f200c88c0 legacy=0x7f2f200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f24c90640 1 -- 172.21.15.17:0/672460893 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200c7670 con 0x7f2f2014f360 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.226+0000 7f2f177fe640 1 -- 172.21.15.17:0/672460893 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648814778 0 0) 0x7f2f200c73d0 con 0x7f2f200c88c0 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f24c90640 1 --2- 172.21.15.17:0/672460893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f2014f360 0x7f2f20155c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f20152d60 tx=0x7f2f1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc41263481655b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f177fe640 1 -- 172.21.15.17:0/672460893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0315a0 con 0x7f2f2014f360 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f177fe640 1 -- 172.21.15.17:0/672460893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c037070 con 0x7f2f2014f360 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f177fe640 1 -- 172.21.15.17:0/672460893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c031960 con 0x7f2f2014f360 2024-03-06T07:20:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f26ef4640 1 -- 172.21.15.17:0/672460893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f2014f360 msgr2=0x7f2f20155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/672460893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f2014f360 0x7f2f20155c10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2f20152d60 tx=0x7f2f1c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f26ef4640 1 -- 172.21.15.17:0/672460893 shutdown_connections 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/672460893 >> v2:172.21.15.163:3300/0 conn(0x7f2f20158b60 0x7f2f20158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.227+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/672460893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f2014f360 0x7f2f20155c10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.228+0000 7f2f26ef4640 1 -- 172.21.15.17:0/672460893 >> 172.21.15.17:0/672460893 conn(0x7f2f200ac570 msgr2=0x7f2f200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.228+0000 7f2f26ef4640 1 -- 172.21.15.17:0/672460893 shutdown_connections 2024-03-06T07:20:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.228+0000 7f2f26ef4640 1 -- 172.21.15.17:0/672460893 wait complete. 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 Processor -- start 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 -- start start 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2014f360 0x7f2f2014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f20158b60 0x7f2f200bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c5a50 con 0x7f2f200c88c0 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c7c30 con 0x7f2f20158b60 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.229+0000 7f2f26ef4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200c5bd0 con 0x7f2f2014f360 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33516/0 (socket says 172.21.15.17:33516) 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 -- 172.21.15.17:0/2068498015 learned_addr learned my addr 172.21.15.17:0/2068498015 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f17fff640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2014f360 0x7f2f2014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f25491640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f20158b60 0x7f2f200bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 -- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2014f360 msgr2=0x7f2f2014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2014f360 0x7f2f2014f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 -- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f20158b60 msgr2=0x7f2f200bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f20158b60 0x7f2f200bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 -- 172.21.15.17:0/2068498015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200bfd60 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f24c90640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02be50 tx=0x7f2f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c032630 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c0327d0 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f201ebc60 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c032bc0 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.230+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f200be8d0 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.231+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200c9f60 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.232+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f1c04c020 con 0x7f2f200c88c0 2024-03-06T07:20:18.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.232+0000 7f2f157fa640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 0x7f2f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.232+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f1c02f020 con 0x7f2f200c88c0 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.232+0000 7f2f17fff640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 0x7f2f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.233+0000 7f2f17fff640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 0x7f2f0c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f200c1bb0 tx=0x7f2f10007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.236+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f200c9f60 con 0x7f2f200c88c0 2024-03-06T07:20:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.416+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f2004f330 con 0x7f2f200c88c0 2024-03-06T07:20:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.418+0000 7f2f157fa640 1 -- 172.21.15.17:0/2068498015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f2f1c040070 con 0x7f2f200c88c0 2024-03-06T07:20:18.419 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:18.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.419+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 msgr2=0x7f2f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.419+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 0x7f2f0c06d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2f200c1bb0 tx=0x7f2f10007de0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.419+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 msgr2=0x7f2f200c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.419+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02be50 tx=0x7f2f1c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 shutdown_connections 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f0c06b4d0 0x7f2f0c06d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f20158b60 0x7f2f200bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f2014f360 0x7f2f2014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 --2- 172.21.15.17:0/2068498015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f200c88c0 0x7f2f200c3c70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 >> 172.21.15.17:0/2068498015 conn(0x7f2f200ac570 msgr2=0x7f2f2014e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 shutdown_connections 2024-03-06T07:20:18.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.420+0000 7f2f26ef4640 1 -- 172.21.15.17:0/2068498015 wait complete. 2024-03-06T07:20:18.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:18.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.533+0000 7f2f3a60e640 1 Processor -- start 2024-03-06T07:20:18.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.533+0000 7f2f3a60e640 1 -- start start 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f3406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f340c1320 0x7f2f3406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f340c6000 0x7f2f341566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f34153a60 con 0x7f2f340c1320 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f34153d40 con 0x7f2f340c6000 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f3a60e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3406c2c0 con 0x7f2f34150580 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f340c1320 0x7f2f3406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f340c1320 0x7f2f3406e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33520/0 (socket says 172.21.15.17:33520) 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 -- 172.21.15.17:0/350318924 learned_addr learned my addr 172.21.15.17:0/350318924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f33fff640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f3406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f38bab640 1 --2- 172.21.15.17:0/350318924 >> v2:172.21.15.163:3300/0 conn(0x7f2f340c6000 0x7f2f341566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 -- 172.21.15.17:0/350318924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 msgr2=0x7f2f3406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f3406dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 -- 172.21.15.17:0/350318924 >> v2:172.21.15.163:3300/0 conn(0x7f2f340c6000 msgr2=0x7f2f341566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 --2- 172.21.15.17:0/350318924 >> v2:172.21.15.163:3300/0 conn(0x7f2f340c6000 0x7f2f341566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.534+0000 7f2f337fe640 1 -- 172.21.15.17:0/350318924 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3406c5a0 con 0x7f2f340c1320 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.535+0000 7f2f337fe640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c1320 0x7f2f3406e250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f34152fe0 tx=0x7f2f1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61723e2a20d72257 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.535+0000 7f2f32ffd640 1 -- 172.21.15.17:0/350318924 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c02e070 con 0x7f2f340c1320 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.535+0000 7f2f32ffd640 1 -- 172.21.15.17:0/350318924 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c037070 con 0x7f2f340c1320 2024-03-06T07:20:18.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.535+0000 7f2f32ffd640 1 -- 172.21.15.17:0/350318924 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c031a00 con 0x7f2f340c1320 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 -- 172.21.15.17:0/350318924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c1320 msgr2=0x7f2f3406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c1320 0x7f2f3406e250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2f34152fe0 tx=0x7f2f1c02b880 comp rx=0 tx=0).stop 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 -- 172.21.15.17:0/350318924 shutdown_connections 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/350318924 >> v2:172.21.15.163:3300/0 conn(0x7f2f340c6000 0x7f2f341566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c1320 0x7f2f3406e250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/350318924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f3406dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 -- 172.21.15.17:0/350318924 >> 172.21.15.17:0/350318924 conn(0x7f2f340c6710 msgr2=0x7f2f340c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 -- 172.21.15.17:0/350318924 shutdown_connections 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.536+0000 7f2f3a60e640 1 -- 172.21.15.17:0/350318924 wait complete. 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.537+0000 7f2f3a60e640 1 Processor -- start 2024-03-06T07:20:18.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.537+0000 7f2f3a60e640 1 -- start start 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f340c1320 0x7f2f340a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f34149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340c0dc0 con 0x7f2f340c6000 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340c0c40 con 0x7f2f340c1320 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f3a60e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34156bf0 con 0x7f2f34150580 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33532/0 (socket says 172.21.15.17:33532) 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f337fe640 1 -- 172.21.15.17:0/3520991565 learned_addr learned my addr 172.21.15.17:0/3520991565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f34149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f33fff640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f340c1320 0x7f2f340a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 -- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f340c1320 msgr2=0x7f2f340a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f340c1320 0x7f2f340a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 -- 172.21.15.17:0/3520991565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 msgr2=0x7f2f34149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f38bab640 1 -- 172.21.15.17:0/3520991565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f341e9250 con 0x7f2f34150580 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.538+0000 7f2f337fe640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f38bab640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f34149f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00eed0 tx=0x7f2f2c00b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c018020 con 0x7f2f34150580 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c014400 con 0x7f2f34150580 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f341e9530 con 0x7f2f34150580 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c009e60 con 0x7f2f34150580 2024-03-06T07:20:18.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.539+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f341e9af0 con 0x7f2f34150580 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.540+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3404f330 con 0x7f2f34150580 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.540+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f2c005d10 con 0x7f2f34150580 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.540+0000 7f2f30ff9640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.541+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f2c08ad60 con 0x7f2f34150580 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.541+0000 7f2f33fff640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.541+0000 7f2f33fff640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f340a8390 tx=0x7f2f24009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.545+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f2c022030 con 0x7f2f34150580 2024-03-06T07:20:18.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.726+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f340c4060 con 0x7f2f34150580 2024-03-06T07:20:18.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.728+0000 7f2f30ff9640 1 -- 172.21.15.17:0/3520991565 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f2f2c059240 con 0x7f2f34150580 2024-03-06T07:20:18.730 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:18.730 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 msgr2=0x7f2f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f340a8390 tx=0x7f2f24009190 comp rx=0 tx=0).stop 2024-03-06T07:20:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 msgr2=0x7f2f34149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f34149f50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00eed0 tx=0x7f2f2c00b480 comp rx=0 tx=0).stop 2024-03-06T07:20:18.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 shutdown_connections 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f34150580 0x7f2f34149f50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f340c6000 0x7f2f34149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 --2- 172.21.15.17:0/3520991565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f340c1320 0x7f2f340a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 >> 172.21.15.17:0/3520991565 conn(0x7f2f340c6710 msgr2=0x7f2f3414a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 shutdown_connections 2024-03-06T07:20:18.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.730+0000 7f2f3a60e640 1 -- 172.21.15.17:0/3520991565 wait complete. 2024-03-06T07:20:18.743 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:20:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.847+0000 7fd2a1e0e640 1 Processor -- start 2024-03-06T07:20:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.847+0000 7fd2a1e0e640 1 -- start start 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd2a1e0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c0b0070 0x7fd29c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd2a1e0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd29c06ca70 0x7fd29c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd2a1e0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c0b0450 con 0x7fd29c0c5380 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd2a1e0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c06c020 con 0x7fd29c06ca70 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd2a1e0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c0c77f0 con 0x7fd29c0b0070 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd29c06ca70 0x7fd29c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd29c0c5380 0x7fd29c14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47730/0 (socket says 172.21.15.17:47730) 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29bfff640 1 -- 172.21.15.17:0/4263824233 learned_addr learned my addr 172.21.15.17:0/4263824233 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29b7fe640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c0b0070 0x7fd29c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29affd640 1 -- 172.21.15.17:0/4263824233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c0b0070 msgr2=0x7fd29c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29affd640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c0b0070 0x7fd29c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29affd640 1 -- 172.21.15.17:0/4263824233 >> v1:172.21.15.17:6789/0 conn(0x7fd29c0c5380 legacy=0x7fd29c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.848+0000 7fd29affd640 1 -- 172.21.15.17:0/4263824233 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0c7ad0 con 0x7fd29c06ca70 2024-03-06T07:20:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd29affd640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c06ca70 0x7fd29c0c8900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2900097a0 tx=0x7fd29002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efca26cc424da4b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd29a7fc640 1 -- 172.21.15.17:0/4263824233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2900315a0 con 0x7fd29c06ca70 2024-03-06T07:20:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd29a7fc640 1 -- 172.21.15.17:0/4263824233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd290037070 con 0x7fd29c06ca70 2024-03-06T07:20:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd29a7fc640 1 -- 172.21.15.17:0/4263824233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290031960 con 0x7fd29c06ca70 2024-03-06T07:20:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/4263824233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c06ca70 msgr2=0x7fd29c0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.849+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c06ca70 0x7fd29c0c8900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2900097a0 tx=0x7fd29002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/4263824233 shutdown_connections 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c06ca70 0x7fd29c0c8900 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/4263824233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c0b0070 0x7fd29c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/4263824233 >> 172.21.15.17:0/4263824233 conn(0x7fd29c0ac590 msgr2=0x7fd29c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/4263824233 shutdown_connections 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.850+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/4263824233 wait complete. 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.851+0000 7fd2a1e0e640 1 Processor -- start 2024-03-06T07:20:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.851+0000 7fd2a1e0e640 1 -- start start 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.851+0000 7fd2a1e0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c06ca70 0x7fd29c1df740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd29c0b0070 0x7fd29c14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c06cef0 con 0x7fd29c0c5380 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c06c550 con 0x7fd29c0b0070 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c06da80 con 0x7fd29c06ca70 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33548/0 (socket says 172.21.15.17:33548) 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 -- 172.21.15.17:0/578253507 learned_addr learned my addr 172.21.15.17:0/578253507 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:18.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29b7fe640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c06ca70 0x7fd29c1df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29affd640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd29c0b0070 0x7fd29c14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 -- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c06ca70 msgr2=0x7fd29c1df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c06ca70 0x7fd29c1df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 -- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd29c0b0070 msgr2=0x7fd29c14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd29c0b0070 0x7fd29c14bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 -- 172.21.15.17:0/578253507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c1e0f10 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd29bfff640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd29400cab0 tx=0x7fd29400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29400ac30 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c14ca60 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294012400 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.852+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c1543b0 con 0x7fd29c0c5380 2024-03-06T07:20:18.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.853+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c04f330 con 0x7fd29c0c5380 2024-03-06T07:20:18.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.854+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd29401a070 con 0x7fd29c0c5380 2024-03-06T07:20:18.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.854+0000 7fd283fff640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:18.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.854+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd29408a040 con 0x7fd29c0c5380 2024-03-06T07:20:18.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.854+0000 7fd29b7fe640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:18.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.855+0000 7fd29b7fe640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 0x7fd28806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0c96f0 tx=0x7fd28c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:18.859+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd29c0b0450 con 0x7fd29c0c5380 2024-03-06T07:20:19.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.077+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd29c06d860 con 0x7fd29c0c5380 2024-03-06T07:20:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.776+0000 7fd283fff640 1 -- 172.21.15.17:0/578253507 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd2940585c0 con 0x7fd29c0c5380 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 msgr2=0x7fd28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 0x7fd28806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0c96f0 tx=0x7fd28c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 msgr2=0x7fd29c14c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd29400cab0 tx=0x7fd29400cf70 comp rx=0 tx=0).stop 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 shutdown_connections 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd28806b4d0 0x7fd28806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd29c0c5380 0x7fd29c14c520 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd29c0b0070 0x7fd29c14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 --2- 172.21.15.17:0/578253507 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd29c06ca70 0x7fd29c1df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 >> 172.21.15.17:0/578253507 conn(0x7fd29c0ac590 msgr2=0x7fd29c0c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 shutdown_connections 2024-03-06T07:20:19.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.779+0000 7fd2a1e0e640 1 -- 172.21.15.17:0/578253507 wait complete. 2024-03-06T07:20:19.794 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-06T07:20:19.794 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:20:19.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.903+0000 7fc5dd29d640 1 Processor -- start 2024-03-06T07:20:19.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.903+0000 7fc5dd29d640 1 -- start start 2024-03-06T07:20:19.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5dd29d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 0x7fc5d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5dd29d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5dd29d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d8156060 con 0x7fc5d80c3a70 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5dd29d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d8155b30 con 0x7fc5d806d750 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5dd29d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80a8430 con 0x7fc5d806db30 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d6ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 0x7fc5d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d77fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d67fc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc5d80c3a70 0x7fc5d8072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59818/0 (socket says 172.21.15.17:59818) 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d67fc640 1 -- 172.21.15.17:0/4175250572 learned_addr learned my addr 172.21.15.17:0/4175250572 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d5ffb640 1 -- 172.21.15.17:0/4175250572 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974901742 0 0) 0x7fc5d8156060 con 0x7fc5d80c3a70 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.904+0000 7fc5d5ffb640 1 -- 172.21.15.17:0/4175250572 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c4003520 con 0x7fc5d80c3a70 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d77fe640 1 -- 172.21.15.17:0/4175250572 >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 msgr2=0x7fc5d80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d77fe640 1 --2- 172.21.15.17:0/4175250572 >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 0x7fc5d80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d77fe640 1 -- 172.21.15.17:0/4175250572 >> v1:172.21.15.163:6789/0 conn(0x7fc5d80c3a70 legacy=0x7fc5d8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d77fe640 1 -- 172.21.15.17:0/4175250572 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8155d80 con 0x7fc5d806db30 2024-03-06T07:20:19.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d6ffd640 1 --2- 172.21.15.17:0/4175250572 >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 0x7fc5d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:19.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.905+0000 7fc5d77fe640 1 --2- 172.21.15.17:0/4175250572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80c1e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d90 tx=0x7fc5c802bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ff331996eaea7c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5d5ffb640 1 -- 172.21.15.17:0/4175250572 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8039070 con 0x7fc5d806db30 2024-03-06T07:20:19.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5d5ffb640 1 -- 172.21.15.17:0/4175250572 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c802cb30 con 0x7fc5d806db30 2024-03-06T07:20:19.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5d5ffb640 1 -- 172.21.15.17:0/4175250572 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8034440 con 0x7fc5d806db30 2024-03-06T07:20:19.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 -- 172.21.15.17:0/4175250572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 msgr2=0x7fc5d80c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/4175250572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80c1e90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009d90 tx=0x7fc5c802bd80 comp rx=0 tx=0).stop 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 -- 172.21.15.17:0/4175250572 shutdown_connections 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/4175250572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80c1e90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/4175250572 >> v2:172.21.15.163:3300/0 conn(0x7fc5d806d750 0x7fc5d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 -- 172.21.15.17:0/4175250572 >> 172.21.15.17:0/4175250572 conn(0x7fc5d80c5560 msgr2=0x7fc5d80c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 -- 172.21.15.17:0/4175250572 shutdown_connections 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.906+0000 7fc5dd29d640 1 -- 172.21.15.17:0/4175250572 wait complete. 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.907+0000 7fc5dd29d640 1 Processor -- start 2024-03-06T07:20:19.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.907+0000 7fc5dd29d640 1 -- start start 2024-03-06T07:20:19.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80c3a70 0x7fc5d814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80a7e50 con 0x7fc5d806d750 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80a7cd0 con 0x7fc5d80c3a70 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5dd29d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80a7fd0 con 0x7fc5d806db30 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33558/0 (socket says 172.21.15.17:33558) 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 -- 172.21.15.17:0/321516159 learned_addr learned my addr 172.21.15.17:0/321516159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d67fc640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d77fe640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80c3a70 0x7fc5d814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 -- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 msgr2=0x7fc5d80abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 -- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80c3a70 msgr2=0x7fc5d814b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80c3a70 0x7fc5d814b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.908+0000 7fc5d6ffd640 1 -- 172.21.15.17:0/321516159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d81ea800 con 0x7fc5d806d750 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5d6ffd640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000e960 tx=0x7fc5d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0017070 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d000ca00 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d000cdd0 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d81eaae0 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.909+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d81eb0a0 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.910+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8069460 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.910+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5d001a070 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.911+0000 7fc5cf7fe640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.911+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc5d008a6e0 con 0x7fc5d806d750 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.911+0000 7fc5d67fc640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:19.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.911+0000 7fc5d67fc640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 0x7fc5b806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80a88d0 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:19.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:19.915+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5d8069460 con 0x7fc5d806d750 2024-03-06T07:20:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.133+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc5d80a8920 con 0x7fc5d806d750 2024-03-06T07:20:20.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.777+0000 7fc5cf7fe640 1 -- 172.21.15.17:0/321516159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc5d80a8920 con 0x7fc5d806d750 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 msgr2=0x7fc5b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 0x7fc5b806d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80a88d0 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 msgr2=0x7fc5d806fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000e960 tx=0x7fc5d000ee20 comp rx=0 tx=0).stop 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 shutdown_connections 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5b806b4d0 0x7fc5b806d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5d80c3a70 0x7fc5d814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5d806db30 0x7fc5d80abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 --2- 172.21.15.17:0/321516159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5d806d750 0x7fc5d806fe60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 >> 172.21.15.17:0/321516159 conn(0x7fc5d80c5560 msgr2=0x7fc5d8072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 shutdown_connections 2024-03-06T07:20:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.779+0000 7fc5dd29d640 1 -- 172.21.15.17:0/321516159 wait complete. 2024-03-06T07:20:20.795 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:20:20.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:20.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.901+0000 7fc5ff3a1640 1 Processor -- start 2024-03-06T07:20:20.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.901+0000 7fc5ff3a1640 1 -- start start 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.901+0000 7fc5ff3a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f8158db0 0x7fc5f80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.901+0000 7fc5ff3a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c88b0 0x7fc5f815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.901+0000 7fc5ff3a1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f8155110 con 0x7fc5f80c5330 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5ff3a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f81559e0 con 0x7fc5f8158db0 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5ff3a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c8f40 con 0x7fc5f80c88b0 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f8158db0 0x7fc5f80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f8158db0 0x7fc5f80c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38004/0 (socket says 172.21.15.17:38004) 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 -- 172.21.15.17:0/1957989486 learned_addr learned my addr 172.21.15.17:0/1957989486 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fd93e640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c88b0 0x7fc5f815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 -- 172.21.15.17:0/1957989486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c88b0 msgr2=0x7fc5f815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:20.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c88b0 0x7fc5f815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 -- 172.21.15.17:0/1957989486 >> v1:172.21.15.163:6789/0 conn(0x7fc5f80c5330 legacy=0x7fc5f81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.902+0000 7fc5fc93c640 1 -- 172.21.15.17:0/1957989486 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8155390 con 0x7fc5f8158db0 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5fc93c640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f8158db0 0x7fc5f80c5710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009870 tx=0x7fc5e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6402aeac4861530 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5effff640 1 -- 172.21.15.17:0/1957989486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8035020 con 0x7fc5f8158db0 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5effff640 1 -- 172.21.15.17:0/1957989486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8031640 con 0x7fc5f8158db0 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5effff640 1 -- 172.21.15.17:0/1957989486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802d040 con 0x7fc5f8158db0 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/1957989486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f8158db0 msgr2=0x7fc5f80c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f8158db0 0x7fc5f80c5710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009870 tx=0x7fc5e802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/1957989486 shutdown_connections 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c88b0 0x7fc5f815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/1957989486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f8158db0 0x7fc5f80c5710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.903+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/1957989486 >> 172.21.15.17:0/1957989486 conn(0x7fc5f80ac560 msgr2=0x7fc5f80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.904+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/1957989486 shutdown_connections 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.904+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/1957989486 wait complete. 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 Processor -- start 2024-03-06T07:20:20.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 -- start start 2024-03-06T07:20:20.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c5330 0x7fc5f8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 0x7fc5f8148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f81529d0 con 0x7fc5f80c88b0 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f806d090 con 0x7fc5f80ac0d0 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5ff3a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8152b50 con 0x7fc5f80c5330 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5fc93c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5fc93c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38018/0 (socket says 172.21.15.17:38018) 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5fc93c640 1 -- 172.21.15.17:0/3455134806 learned_addr learned my addr 172.21.15.17:0/3455134806 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5fd13d640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c5330 0x7fc5f8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.905+0000 7fc5fd93e640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 0x7fc5f8148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 -- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c5330 msgr2=0x7fc5f8150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c5330 0x7fc5f8150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 -- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 msgr2=0x7fc5f8148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 0x7fc5f8148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 -- 172.21.15.17:0/3455134806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80c1070 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fd93e640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 0x7fc5f8148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5fc93c640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8040070 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80399f0 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f814d740 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8035040 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.906+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80abde0 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.907+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f804f330 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.908+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5e804b060 con 0x7fc5f80c88b0 2024-03-06T07:20:20.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.908+0000 7fc5edffb640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:20.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.908+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc5e80abc30 con 0x7fc5f80c88b0 2024-03-06T07:20:20.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.908+0000 7fc5fd13d640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:20.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.909+0000 7fc5fd13d640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4001f10 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:20.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:20.912+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5e807a800 con 0x7fc5f80c88b0 2024-03-06T07:20:21.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.132+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5f81501f0 con 0x7fc5f80c88b0 2024-03-06T07:20:21.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.133+0000 7fc5edffb640 1 -- 172.21.15.17:0/3455134806 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7fc5e807a100 con 0x7fc5f80c88b0 2024-03-06T07:20:21.134 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:21.135 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:21.135 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:21.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 msgr2=0x7fc5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4001f10 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 msgr2=0x7fc5f80ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000c00 tx=0x7fc5e80046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 shutdown_connections 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f80ac0d0 0x7fc5f8148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c88b0 0x7fc5f80ab8a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 --2- 172.21.15.17:0/3455134806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c5330 0x7fc5f8150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 >> 172.21.15.17:0/3455134806 conn(0x7fc5f80ac560 msgr2=0x7fc5f8153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 shutdown_connections 2024-03-06T07:20:21.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.135+0000 7fc5ff3a1640 1 -- 172.21.15.17:0/3455134806 wait complete. 2024-03-06T07:20:21.147 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:21.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 Processor -- start 2024-03-06T07:20:21.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 -- start start 2024-03-06T07:20:21.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4200159270 0x7f4200155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000c73d0 con 0x7f42000c88c0 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000be2e0 con 0x7f420014f360 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f42070fb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4200159700 con 0x7f4200159270 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f4205698640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4200159270 0x7f4200155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f41fffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f42000c88c0 0x7f4200158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58258/0 (socket says 172.21.15.17:58258) 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.247+0000 7f41fffff640 1 -- 172.21.15.17:0/3884998170 learned_addr learned my addr 172.21.15.17:0/3884998170 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.248+0000 7f4205698640 1 -- 172.21.15.17:0/3884998170 >> v1:172.21.15.163:6789/0 conn(0x7f420014f360 legacy=0x7f42000c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.248+0000 7f4205698640 1 -- 172.21.15.17:0/3884998170 >> v1:172.21.15.17:6789/0 conn(0x7f42000c88c0 legacy=0x7f4200158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.248+0000 7f41ff7fe640 1 -- 172.21.15.17:0/3884998170 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670039396 0 0) 0x7f42000c73d0 con 0x7f42000c88c0 2024-03-06T07:20:21.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.248+0000 7f4205698640 1 -- 172.21.15.17:0/3884998170 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000c7670 con 0x7f4200159270 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.248+0000 7f4205698640 1 --2- 172.21.15.17:0/3884998170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200155c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41f00097a0 tx=0x7f41f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03712c99e8cd6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f41ff7fe640 1 -- 172.21.15.17:0/3884998170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0035020 con 0x7f4200159270 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f41ff7fe640 1 -- 172.21.15.17:0/3884998170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0031830 con 0x7f4200159270 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f41ff7fe640 1 -- 172.21.15.17:0/3884998170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0031c00 con 0x7f4200159270 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 -- 172.21.15.17:0/3884998170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 msgr2=0x7f4200155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 --2- 172.21.15.17:0/3884998170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200155c10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41f00097a0 tx=0x7f41f002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 -- 172.21.15.17:0/3884998170 shutdown_connections 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 --2- 172.21.15.17:0/3884998170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200155c10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 -- 172.21.15.17:0/3884998170 >> 172.21.15.17:0/3884998170 conn(0x7f42000ac570 msgr2=0x7f42000ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 -- 172.21.15.17:0/3884998170 shutdown_connections 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.249+0000 7f42070fb640 1 -- 172.21.15.17:0/3884998170 wait complete. 2024-03-06T07:20:21.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.250+0000 7f42070fb640 1 Processor -- start 2024-03-06T07:20:21.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.250+0000 7f42070fb640 1 -- start start 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42000c88c0 0x7f42000becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 0x7f42001eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000c7c30 con 0x7f4200159270 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000c5a50 con 0x7f420014f360 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f42070fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42000c5bd0 con 0x7f42000c88c0 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f4205698640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f4205698640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38038/0 (socket says 172.21.15.17:38038) 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f4205698640 1 -- 172.21.15.17:0/1509999925 learned_addr learned my addr 172.21.15.17:0/1509999925 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f41fffff640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 0x7f42001eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.251+0000 7f4204e97640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42000c88c0 0x7f42000becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 -- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42000c88c0 msgr2=0x7f42000becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42000c88c0 0x7f42000becb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 -- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 msgr2=0x7f42001eb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 0x7f42001eb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 -- 172.21.15.17:0/1509999925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4200152740 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f41fffff640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 0x7f42001eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f4205698640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098c0 tx=0x7f41f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0040070 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f00398c0 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420014f7e0 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0035040 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.252+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420014fda0 con 0x7f4200159270 2024-03-06T07:20:21.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.253+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420004f330 con 0x7f4200159270 2024-03-06T07:20:21.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.253+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41f0038020 con 0x7f4200159270 2024-03-06T07:20:21.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.254+0000 7f41fd7fa640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 0x7f41ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.254+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f41f00abda0 con 0x7f4200159270 2024-03-06T07:20:21.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.254+0000 7f4204e97640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 0x7f41ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.254+0000 7f4204e97640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 0x7f41ec06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41f80012e0 tx=0x7f41f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.258+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41f007a320 con 0x7f4200159270 2024-03-06T07:20:21.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.438+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42000c8d50 con 0x7f4200159270 2024-03-06T07:20:21.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.439+0000 7f41fd7fa640 1 -- 172.21.15.17:0/1509999925 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f41f007e3c0 con 0x7f4200159270 2024-03-06T07:20:21.441 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:21.442 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:21.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.441+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 msgr2=0x7f41ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.441+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 0x7f41ec06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f41f80012e0 tx=0x7f41f8009040 comp rx=0 tx=0).stop 2024-03-06T07:20:21.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.441+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 msgr2=0x7f4200151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.441+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41f00098c0 tx=0x7f41f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 shutdown_connections 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41ec06b4d0 0x7f41ec06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4200159270 0x7f4200151d60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f420014f360 0x7f42001eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 --2- 172.21.15.17:0/1509999925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42000c88c0 0x7f42000becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 >> 172.21.15.17:0/1509999925 conn(0x7f42000ac570 msgr2=0x7f420014e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 shutdown_connections 2024-03-06T07:20:21.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.442+0000 7f42070fb640 1 -- 172.21.15.17:0/1509999925 wait complete. 2024-03-06T07:20:21.454 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:20:21.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.554+0000 7f0499a33640 1 Processor -- start 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.554+0000 7f0499a33640 1 -- start start 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0499a33640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f04941506b0 0x7f0494153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0499a33640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 0x7f04940702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0499a33640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0494070800 con 0x7f04940acc00 2024-03-06T07:20:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0499a33640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0494156450 con 0x7f04941506b0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0499a33640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0494156a10 con 0x7f0494156ea0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f04937fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04940acc00 0x7f04940c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58270/0 (socket says 172.21.15.17:58270) 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f04937fe640 1 -- 172.21.15.17:0/650589997 learned_addr learned my addr 172.21.15.17:0/650589997 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0492ffd640 1 --2- 172.21.15.17:0/650589997 >> v2:172.21.15.163:3300/0 conn(0x7f04941506b0 0x7f0494153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f04927fc640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 0x7f04940702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0491ffb640 1 -- 172.21.15.17:0/650589997 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590051699 0 0) 0x7f0494070800 con 0x7f04940acc00 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.555+0000 7f0491ffb640 1 -- 172.21.15.17:0/650589997 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f047c003520 con 0x7f04940acc00 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0492ffd640 1 -- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 msgr2=0x7f04940702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0492ffd640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 0x7f04940702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0492ffd640 1 -- 172.21.15.17:0/650589997 >> v1:172.21.15.17:6789/0 conn(0x7f04940acc00 legacy=0x7f04940c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0492ffd640 1 -- 172.21.15.17:0/650589997 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494070aa0 con 0x7f04941506b0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f04927fc640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 0x7f04940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0492ffd640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04941506b0 0x7f0494153990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04941565d0 tx=0x7f048c02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6448f65c536743b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.556+0000 7f0491ffb640 1 -- 172.21.15.17:0/650589997 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c035020 con 0x7f04941506b0 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0491ffb640 1 -- 172.21.15.17:0/650589997 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c031800 con 0x7f04941506b0 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0491ffb640 1 -- 172.21.15.17:0/650589997 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c031bb0 con 0x7f04941506b0 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 -- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04941506b0 msgr2=0x7f0494153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04941506b0 0x7f0494153990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04941565d0 tx=0x7f048c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 -- 172.21.15.17:0/650589997 shutdown_connections 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0494156ea0 0x7f04940702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 --2- 172.21.15.17:0/650589997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04941506b0 0x7f0494153990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 -- 172.21.15.17:0/650589997 >> 172.21.15.17:0/650589997 conn(0x7f049406b1a0 msgr2=0x7f049406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 -- 172.21.15.17:0/650589997 shutdown_connections 2024-03-06T07:20:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.557+0000 7f0499a33640 1 -- 172.21.15.17:0/650589997 wait complete. 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.558+0000 7f0499a33640 1 Processor -- start 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.558+0000 7f0499a33640 1 -- start start 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04941506b0 0x7f04940c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 0x7f04941ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c5bb0 con 0x7f04940acc00 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941566a0 con 0x7f0494156ea0 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0499a33640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04940c5d30 con 0x7f04941506b0 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38050/0 (socket says 172.21.15.17:38050) 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 -- 172.21.15.17:0/855069792 learned_addr learned my addr 172.21.15.17:0/855069792 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f04927fc640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04941506b0 0x7f04940c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f04937fe640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 0x7f04941ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 -- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04941506b0 msgr2=0x7f04940c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04941506b0 0x7f04940c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 -- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 msgr2=0x7f04941ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 0x7f04941ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f0492ffd640 1 -- 172.21.15.17:0/855069792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940c4610 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.559+0000 7f04937fe640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 0x7f04941ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f0492ffd640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c041070 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c031e30 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c035040 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04940c48f0 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.560+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04940c22a0 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.561+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049404f330 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.561+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f048c038020 con 0x7f04940acc00 2024-03-06T07:20:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.562+0000 7f047bfff640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 0x7f048006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.562+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f048c0abe90 con 0x7f04940acc00 2024-03-06T07:20:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.562+0000 7f04927fc640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 0x7f048006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.562+0000 7f04927fc640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 0x7f048006d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0488009560 tx=0x7f0488009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.566+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f048c07a410 con 0x7f04940acc00 2024-03-06T07:20:21.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.744+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04940700a0 con 0x7f048006b500 2024-03-06T07:20:21.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.753+0000 7f047bfff640 1 -- 172.21.15.17:0/855069792 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7f04940700a0 con 0x7f048006b500 2024-03-06T07:20:21.759 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:20:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.756+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 msgr2=0x7f048006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.756+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 0x7f048006d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0488009560 tx=0x7f0488009190 comp rx=0 tx=0).stop 2024-03-06T07:20:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.756+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 msgr2=0x7f04940aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.756+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 shutdown_connections 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f048006b500 0x7f048006d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0494156ea0 0x7f04941ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04941506b0 0x7f04940c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 --2- 172.21.15.17:0/855069792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04940acc00 0x7f04940aae20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 >> 172.21.15.17:0/855069792 conn(0x7f049406b1a0 msgr2=0x7f049406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 shutdown_connections 2024-03-06T07:20:21.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.757+0000 7f0499a33640 1 -- 172.21.15.17:0/855069792 wait complete. 2024-03-06T07:20:21.769 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-03-06T07:20:21.345200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230916,"kb_used_data":2272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743804,"statfs":{"total":773094113280,"available":772857655296,"internally_reserved":0,"allocated":2326528,"data_stored":2048878,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.602102"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772886+0000","last_change":"2024-03-06T07:20:03.728088+0000","last_active":"2024-03-06T07:20:10.772886+0000","last_peered":"2024-03-06T07:20:10.772886+0000","last_clean":"2024-03-06T07:20:10.772886+0000","last_became_active":"2024-03-06T07:20:03.727805+0000","last_became_peered":"2024-03-06T07:20:03.727805+0000","last_unstale":"2024-03-06T07:20:10.772886+0000","last_undegraded":"2024-03-06T07:20:10.772886+0000","last_fullsized":"2024-03-06T07:20:10.772886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744735+0000","last_change":"2024-03-06T07:20:04.732564+0000","last_active":"2024-03-06T07:20:10.744735+0000","last_peered":"2024-03-06T07:20:10.744735+0000","last_clean":"2024-03-06T07:20:10.744735+0000","last_became_active":"2024-03-06T07:20:04.732256+0000","last_became_peered":"2024-03-06T07:20:04.732256+0000","last_unstale":"2024-03-06T07:20:10.744735+0000","last_undegraded":"2024-03-06T07:20:10.744735+0000","last_fullsized":"2024-03-06T07:20:10.744735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744130+0000","last_change":"2024-03-06T07:20:09.527717+0000","last_active":"2024-03-06T07:20:10.744130+0000","last_peered":"2024-03-06T07:20:10.744130+0000","last_clean":"2024-03-06T07:20:10.744130+0000","last_became_active":"2024-03-06T07:20:09.527324+0000","last_became_peered":"2024-03-06T07:20:09.527324+0000","last_unstale":"2024-03-06T07:20:10.744130+0000","last_undegraded":"2024-03-06T07:20:10.744130+0000","last_fullsized":"2024-03-06T07:20:10.744130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:05.448069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744975+0000","last_change":"2024-03-06T07:20:03.719181+0000","last_active":"2024-03-06T07:20:10.744975+0000","last_peered":"2024-03-06T07:20:10.744975+0000","last_clean":"2024-03-06T07:20:10.744975+0000","last_became_active":"2024-03-06T07:20:03.718904+0000","last_became_peered":"2024-03-06T07:20:03.718904+0000","last_unstale":"2024-03-06T07:20:10.744975+0000","last_undegraded":"2024-03-06T07:20:10.744975+0000","last_fullsized":"2024-03-06T07:20:10.744975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772804+0000","last_change":"2024-03-06T07:20:04.731613+0000","last_active":"2024-03-06T07:20:10.772804+0000","last_peered":"2024-03-06T07:20:10.772804+0000","last_clean":"2024-03-06T07:20:10.772804+0000","last_became_active":"2024-03-06T07:20:04.731105+0000","last_became_peered":"2024-03-06T07:20:04.731105+0000","last_unstale":"2024-03-06T07:20:10.772804+0000","last_undegraded":"2024-03-06T07:20:10.772804+0000","last_fullsized":"2024-03-06T07:20:10.772804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744947+0000","last_change":"2024-03-06T07:20:08.761811+0000","last_active":"2024-03-06T07:20:10.744947+0000","last_peered":"2024-03-06T07:20:10.744947+0000","last_clean":"2024-03-06T07:20:10.744947+0000","last_became_active":"2024-03-06T07:20:08.761354+0000","last_became_peered":"2024-03-06T07:20:08.761354+0000","last_unstale":"2024-03-06T07:20:10.744947+0000","last_undegraded":"2024-03-06T07:20:10.744947+0000","last_fullsized":"2024-03-06T07:20:10.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:52.215187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273914+0000","last_change":"2024-03-06T07:20:03.728086+0000","last_active":"2024-03-06T07:20:11.273914+0000","last_peered":"2024-03-06T07:20:11.273914+0000","last_clean":"2024-03-06T07:20:11.273914+0000","last_became_active":"2024-03-06T07:20:03.727793+0000","last_became_peered":"2024-03-06T07:20:03.727793+0000","last_unstale":"2024-03-06T07:20:11.273914+0000","last_undegraded":"2024-03-06T07:20:11.273914+0000","last_fullsized":"2024-03-06T07:20:11.273914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273526+0000","last_change":"2024-03-06T07:20:04.732751+0000","last_active":"2024-03-06T07:20:11.273526+0000","last_peered":"2024-03-06T07:20:11.273526+0000","last_clean":"2024-03-06T07:20:11.273526+0000","last_became_active":"2024-03-06T07:20:04.731757+0000","last_became_peered":"2024-03-06T07:20:04.731757+0000","last_unstale":"2024-03-06T07:20:11.273526+0000","last_undegraded":"2024-03-06T07:20:11.273526+0000","last_fullsized":"2024-03-06T07:20:11.273526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746076+0000","last_change":"2024-03-06T07:20:08.756321+0000","last_active":"2024-03-06T07:20:10.746076+0000","last_peered":"2024-03-06T07:20:10.746076+0000","last_clean":"2024-03-06T07:20:10.746076+0000","last_became_active":"2024-03-06T07:20:08.755944+0000","last_became_peered":"2024-03-06T07:20:08.755944+0000","last_unstale":"2024-03-06T07:20:10.746076+0000","last_undegraded":"2024-03-06T07:20:10.746076+0000","last_fullsized":"2024-03-06T07:20:10.746076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273206+0000","last_change":"2024-03-06T07:20:03.730453+0000","last_active":"2024-03-06T07:20:11.273206+0000","last_peered":"2024-03-06T07:20:11.273206+0000","last_clean":"2024-03-06T07:20:11.273206+0000","last_became_active":"2024-03-06T07:20:03.730180+0000","last_became_peered":"2024-03-06T07:20:03.730180+0000","last_unstale":"2024-03-06T07:20:11.273206+0000","last_undegraded":"2024-03-06T07:20:11.273206+0000","last_fullsized":"2024-03-06T07:20:11.273206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:04.734206+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:04.733937+0000","last_became_peered":"2024-03-06T07:20:04.733937+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745261+0000","last_change":"2024-03-06T07:20:08.760335+0000","last_active":"2024-03-06T07:20:10.745261+0000","last_peered":"2024-03-06T07:20:10.745261+0000","last_clean":"2024-03-06T07:20:10.745261+0000","last_became_active":"2024-03-06T07:20:08.759945+0000","last_became_peered":"2024-03-06T07:20:08.759945+0000","last_unstale":"2024-03-06T07:20:10.745261+0000","last_undegraded":"2024-03-06T07:20:10.745261+0000","last_fullsized":"2024-03-06T07:20:10.745261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744803+0000","last_change":"2024-03-06T07:20:04.730615+0000","last_active":"2024-03-06T07:20:10.744803+0000","last_peered":"2024-03-06T07:20:10.744803+0000","last_clean":"2024-03-06T07:20:10.744803+0000","last_became_active":"2024-03-06T07:20:04.729978+0000","last_became_peered":"2024-03-06T07:20:04.729978+0000","last_unstale":"2024-03-06T07:20:10.744803+0000","last_undegraded":"2024-03-06T07:20:10.744803+0000","last_fullsized":"2024-03-06T07:20:10.744803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274021+0000","last_change":"2024-03-06T07:20:03.730180+0000","last_active":"2024-03-06T07:20:11.274021+0000","last_peered":"2024-03-06T07:20:11.274021+0000","last_clean":"2024-03-06T07:20:11.274021+0000","last_became_active":"2024-03-06T07:20:03.729812+0000","last_became_peered":"2024-03-06T07:20:03.729812+0000","last_unstale":"2024-03-06T07:20:11.274021+0000","last_undegraded":"2024-03-06T07:20:11.274021+0000","last_fullsized":"2024-03-06T07:20:11.274021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746544+0000","last_change":"2024-03-06T07:20:08.761949+0000","last_active":"2024-03-06T07:20:10.746544+0000","last_peered":"2024-03-06T07:20:10.746544+0000","last_clean":"2024-03-06T07:20:10.746544+0000","last_became_active":"2024-03-06T07:20:08.761590+0000","last_became_peered":"2024-03-06T07:20:08.761590+0000","last_unstale":"2024-03-06T07:20:10.746544+0000","last_undegraded":"2024-03-06T07:20:10.746544+0000","last_fullsized":"2024-03-06T07:20:10.746544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744471+0000","last_change":"2024-03-06T07:20:04.726070+0000","last_active":"2024-03-06T07:20:10.744471+0000","last_peered":"2024-03-06T07:20:10.744471+0000","last_clean":"2024-03-06T07:20:10.744471+0000","last_became_active":"2024-03-06T07:20:04.725563+0000","last_became_peered":"2024-03-06T07:20:04.725563+0000","last_unstale":"2024-03-06T07:20:10.744471+0000","last_undegraded":"2024-03-06T07:20:10.744471+0000","last_fullsized":"2024-03-06T07:20:10.744471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273584+0000","last_change":"2024-03-06T07:20:03.719632+0000","last_active":"2024-03-06T07:20:11.273584+0000","last_peered":"2024-03-06T07:20:11.273584+0000","last_clean":"2024-03-06T07:20:11.273584+0000","last_became_active":"2024-03-06T07:20:03.719328+0000","last_became_peered":"2024-03-06T07:20:03.719328+0000","last_unstale":"2024-03-06T07:20:11.273584+0000","last_undegraded":"2024-03-06T07:20:11.273584+0000","last_fullsized":"2024-03-06T07:20:11.273584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746162+0000","last_change":"2024-03-06T07:20:08.756550+0000","last_active":"2024-03-06T07:20:10.746162+0000","last_peered":"2024-03-06T07:20:10.746162+0000","last_clean":"2024-03-06T07:20:10.746162+0000","last_became_active":"2024-03-06T07:20:08.755838+0000","last_became_peered":"2024-03-06T07:20:08.755838+0000","last_unstale":"2024-03-06T07:20:10.746162+0000","last_undegraded":"2024-03-06T07:20:10.746162+0000","last_fullsized":"2024-03-06T07:20:10.746162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744264+0000","last_change":"2024-03-06T07:20:04.727861+0000","last_active":"2024-03-06T07:20:10.744264+0000","last_peered":"2024-03-06T07:20:10.744264+0000","last_clean":"2024-03-06T07:20:10.744264+0000","last_became_active":"2024-03-06T07:20:04.727579+0000","last_became_peered":"2024-03-06T07:20:04.727579+0000","last_unstale":"2024-03-06T07:20:10.744264+0000","last_undegraded":"2024-03-06T07:20:10.744264+0000","last_fullsized":"2024-03-06T07:20:10.744264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273057+0000","last_change":"2024-03-06T07:20:03.722227+0000","last_active":"2024-03-06T07:20:11.273057+0000","last_peered":"2024-03-06T07:20:11.273057+0000","last_clean":"2024-03-06T07:20:11.273057+0000","last_became_active":"2024-03-06T07:20:03.721962+0000","last_became_peered":"2024-03-06T07:20:03.721962+0000","last_unstale":"2024-03-06T07:20:11.273057+0000","last_undegraded":"2024-03-06T07:20:11.273057+0000","last_fullsized":"2024-03-06T07:20:11.273057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273518+0000","last_change":"2024-03-06T07:20:09.524648+0000","last_active":"2024-03-06T07:20:11.273518+0000","last_peered":"2024-03-06T07:20:11.273518+0000","last_clean":"2024-03-06T07:20:11.273518+0000","last_became_active":"2024-03-06T07:20:09.524267+0000","last_became_peered":"2024-03-06T07:20:09.524267+0000","last_unstale":"2024-03-06T07:20:11.273518+0000","last_undegraded":"2024-03-06T07:20:11.273518+0000","last_fullsized":"2024-03-06T07:20:11.273518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745823+0000","last_change":"2024-03-06T07:20:04.720167+0000","last_active":"2024-03-06T07:20:10.745823+0000","last_peered":"2024-03-06T07:20:10.745823+0000","last_clean":"2024-03-06T07:20:10.745823+0000","last_became_active":"2024-03-06T07:20:04.719907+0000","last_became_peered":"2024-03-06T07:20:04.719907+0000","last_unstale":"2024-03-06T07:20:10.745823+0000","last_undegraded":"2024-03-06T07:20:10.745823+0000","last_fullsized":"2024-03-06T07:20:10.745823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:45.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272765+0000","last_change":"2024-03-06T07:20:03.730868+0000","last_active":"2024-03-06T07:20:11.272765+0000","last_peered":"2024-03-06T07:20:11.272765+0000","last_clean":"2024-03-06T07:20:11.272765+0000","last_became_active":"2024-03-06T07:20:03.729771+0000","last_became_peered":"2024-03-06T07:20:03.729771+0000","last_unstale":"2024-03-06T07:20:11.272765+0000","last_undegraded":"2024-03-06T07:20:11.272765+0000","last_fullsized":"2024-03-06T07:20:11.272765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746511+0000","last_change":"2024-03-06T07:20:08.756023+0000","last_active":"2024-03-06T07:20:10.746511+0000","last_peered":"2024-03-06T07:20:10.746511+0000","last_clean":"2024-03-06T07:20:10.746511+0000","last_became_active":"2024-03-06T07:20:08.755638+0000","last_became_peered":"2024-03-06T07:20:08.755638+0000","last_unstale":"2024-03-06T07:20:10.746511+0000","last_undegraded":"2024-03-06T07:20:10.746511+0000","last_fullsized":"2024-03-06T07:20:10.746511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773057+0000","last_change":"2024-03-06T07:20:04.724646+0000","last_active":"2024-03-06T07:20:10.773057+0000","last_peered":"2024-03-06T07:20:10.773057+0000","last_clean":"2024-03-06T07:20:10.773057+0000","last_became_active":"2024-03-06T07:20:04.723034+0000","last_became_peered":"2024-03-06T07:20:04.723034+0000","last_unstale":"2024-03-06T07:20:10.773057+0000","last_undegraded":"2024-03-06T07:20:10.773057+0000","last_fullsized":"2024-03-06T07:20:10.773057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745736+0000","last_change":"2024-03-06T07:20:03.723209+0000","last_active":"2024-03-06T07:20:10.745736+0000","last_peered":"2024-03-06T07:20:10.745736+0000","last_clean":"2024-03-06T07:20:10.745736+0000","last_became_active":"2024-03-06T07:20:03.722898+0000","last_became_peered":"2024-03-06T07:20:03.722898+0000","last_unstale":"2024-03-06T07:20:10.745736+0000","last_undegraded":"2024-03-06T07:20:10.745736+0000","last_fullsized":"2024-03-06T07:20:10.745736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745558+0000","last_change":"2024-03-06T07:20:09.526554+0000","last_active":"2024-03-06T07:20:10.745558+0000","last_peered":"2024-03-06T07:20:10.745558+0000","last_clean":"2024-03-06T07:20:10.745558+0000","last_became_active":"2024-03-06T07:20:09.526194+0000","last_became_peered":"2024-03-06T07:20:09.526194+0000","last_unstale":"2024-03-06T07:20:10.745558+0000","last_undegraded":"2024-03-06T07:20:10.745558+0000","last_fullsized":"2024-03-06T07:20:10.745558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745015+0000","last_change":"2024-03-06T07:20:04.736741+0000","last_active":"2024-03-06T07:20:10.745015+0000","last_peered":"2024-03-06T07:20:10.745015+0000","last_clean":"2024-03-06T07:20:10.745015+0000","last_became_active":"2024-03-06T07:20:04.736427+0000","last_became_peered":"2024-03-06T07:20:04.736427+0000","last_unstale":"2024-03-06T07:20:10.745015+0000","last_undegraded":"2024-03-06T07:20:10.745015+0000","last_fullsized":"2024-03-06T07:20:10.745015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273597+0000","last_change":"2024-03-06T07:20:03.720361+0000","last_active":"2024-03-06T07:20:11.273597+0000","last_peered":"2024-03-06T07:20:11.273597+0000","last_clean":"2024-03-06T07:20:11.273597+0000","last_became_active":"2024-03-06T07:20:03.718595+0000","last_became_peered":"2024-03-06T07:20:03.718595+0000","last_unstale":"2024-03-06T07:20:11.273597+0000","last_undegraded":"2024-03-06T07:20:11.273597+0000","last_fullsized":"2024-03-06T07:20:11.273597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744052+0000","last_change":"2024-03-06T07:20:09.526923+0000","last_active":"2024-03-06T07:20:10.744052+0000","last_peered":"2024-03-06T07:20:10.744052+0000","last_clean":"2024-03-06T07:20:10.744052+0000","last_became_active":"2024-03-06T07:20:09.526539+0000","last_became_peered":"2024-03-06T07:20:09.526539+0000","last_unstale":"2024-03-06T07:20:10.744052+0000","last_undegraded":"2024-03-06T07:20:10.744052+0000","last_fullsized":"2024-03-06T07:20:10.744052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:30.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273347+0000","last_change":"2024-03-06T07:20:04.720682+0000","last_active":"2024-03-06T07:20:11.273347+0000","last_peered":"2024-03-06T07:20:11.273347+0000","last_clean":"2024-03-06T07:20:11.273347+0000","last_became_active":"2024-03-06T07:20:04.720404+0000","last_became_peered":"2024-03-06T07:20:04.720404+0000","last_unstale":"2024-03-06T07:20:11.273347+0000","last_undegraded":"2024-03-06T07:20:11.273347+0000","last_fullsized":"2024-03-06T07:20:11.273347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274076+0000","last_change":"2024-03-06T07:20:03.727473+0000","last_active":"2024-03-06T07:20:11.274076+0000","last_peered":"2024-03-06T07:20:11.274076+0000","last_clean":"2024-03-06T07:20:11.274076+0000","last_became_active":"2024-03-06T07:20:03.727177+0000","last_became_peered":"2024-03-06T07:20:03.727177+0000","last_unstale":"2024-03-06T07:20:11.274076+0000","last_undegraded":"2024-03-06T07:20:11.274076+0000","last_fullsized":"2024-03-06T07:20:11.274076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772695+0000","last_change":"2024-03-06T07:20:09.525617+0000","last_active":"2024-03-06T07:20:10.772695+0000","last_peered":"2024-03-06T07:20:10.772695+0000","last_clean":"2024-03-06T07:20:10.772695+0000","last_became_active":"2024-03-06T07:20:09.525228+0000","last_became_peered":"2024-03-06T07:20:09.525228+0000","last_unstale":"2024-03-06T07:20:10.772695+0000","last_undegraded":"2024-03-06T07:20:10.772695+0000","last_fullsized":"2024-03-06T07:20:10.772695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273528+0000","last_change":"2024-03-06T07:20:04.725517+0000","last_active":"2024-03-06T07:20:11.273528+0000","last_peered":"2024-03-06T07:20:11.273528+0000","last_clean":"2024-03-06T07:20:11.273528+0000","last_became_active":"2024-03-06T07:20:04.725226+0000","last_became_peered":"2024-03-06T07:20:04.725226+0000","last_unstale":"2024-03-06T07:20:11.273528+0000","last_undegraded":"2024-03-06T07:20:11.273528+0000","last_fullsized":"2024-03-06T07:20:11.273528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:03.721095+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:03.720730+0000","last_became_peered":"2024-03-06T07:20:03.720730+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273165+0000","last_change":"2024-03-06T07:20:08.763568+0000","last_active":"2024-03-06T07:20:11.273165+0000","last_peered":"2024-03-06T07:20:11.273165+0000","last_clean":"2024-03-06T07:20:11.273165+0000","last_became_active":"2024-03-06T07:20:08.763213+0000","last_became_peered":"2024-03-06T07:20:08.763213+0000","last_unstale":"2024-03-06T07:20:11.273165+0000","last_undegraded":"2024-03-06T07:20:11.273165+0000","last_fullsized":"2024-03-06T07:20:11.273165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746237+0000","last_change":"2024-03-06T07:20:04.715991+0000","last_active":"2024-03-06T07:20:10.746237+0000","last_peered":"2024-03-06T07:20:10.746237+0000","last_clean":"2024-03-06T07:20:10.746237+0000","last_became_active":"2024-03-06T07:20:04.715695+0000","last_became_peered":"2024-03-06T07:20:04.715695+0000","last_unstale":"2024-03-06T07:20:10.746237+0000","last_undegraded":"2024-03-06T07:20:10.746237+0000","last_fullsized":"2024-03-06T07:20:10.746237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:19.554698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274213+0000","last_change":"2024-03-06T07:20:03.724950+0000","last_active":"2024-03-06T07:20:11.274213+0000","last_peered":"2024-03-06T07:20:11.274213+0000","last_clean":"2024-03-06T07:20:11.274213+0000","last_became_active":"2024-03-06T07:20:03.724641+0000","last_became_peered":"2024-03-06T07:20:03.724641+0000","last_unstale":"2024-03-06T07:20:11.274213+0000","last_undegraded":"2024-03-06T07:20:11.274213+0000","last_fullsized":"2024-03-06T07:20:11.274213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273385+0000","last_change":"2024-03-06T07:20:08.762922+0000","last_active":"2024-03-06T07:20:11.273385+0000","last_peered":"2024-03-06T07:20:11.273385+0000","last_clean":"2024-03-06T07:20:11.273385+0000","last_became_active":"2024-03-06T07:20:08.762488+0000","last_became_peered":"2024-03-06T07:20:08.762488+0000","last_unstale":"2024-03-06T07:20:11.273385+0000","last_undegraded":"2024-03-06T07:20:11.273385+0000","last_fullsized":"2024-03-06T07:20:11.273385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273235+0000","last_change":"2024-03-06T07:20:04.729950+0000","last_active":"2024-03-06T07:20:11.273235+0000","last_peered":"2024-03-06T07:20:11.273235+0000","last_clean":"2024-03-06T07:20:11.273235+0000","last_became_active":"2024-03-06T07:20:04.729294+0000","last_became_peered":"2024-03-06T07:20:04.729294+0000","last_unstale":"2024-03-06T07:20:11.273235+0000","last_undegraded":"2024-03-06T07:20:11.273235+0000","last_fullsized":"2024-03-06T07:20:11.273235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877392+0000","last_change":"2024-03-06T07:20:03.717344+0000","last_active":"2024-03-06T07:20:10.877392+0000","last_peered":"2024-03-06T07:20:10.877392+0000","last_clean":"2024-03-06T07:20:10.877392+0000","last_became_active":"2024-03-06T07:20:03.717054+0000","last_became_peered":"2024-03-06T07:20:03.717054+0000","last_unstale":"2024-03-06T07:20:10.877392+0000","last_undegraded":"2024-03-06T07:20:10.877392+0000","last_fullsized":"2024-03-06T07:20:10.877392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273655+0000","last_change":"2024-03-06T07:20:08.763839+0000","last_active":"2024-03-06T07:20:11.273655+0000","last_peered":"2024-03-06T07:20:11.273655+0000","last_clean":"2024-03-06T07:20:11.273655+0000","last_became_active":"2024-03-06T07:20:08.763398+0000","last_became_peered":"2024-03-06T07:20:08.763398+0000","last_unstale":"2024-03-06T07:20:11.273655+0000","last_undegraded":"2024-03-06T07:20:11.273655+0000","last_fullsized":"2024-03-06T07:20:11.273655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772428+0000","last_change":"2024-03-06T07:20:04.717362+0000","last_active":"2024-03-06T07:20:10.772428+0000","last_peered":"2024-03-06T07:20:10.772428+0000","last_clean":"2024-03-06T07:20:10.772428+0000","last_became_active":"2024-03-06T07:20:04.717064+0000","last_became_peered":"2024-03-06T07:20:04.717064+0000","last_unstale":"2024-03-06T07:20:10.772428+0000","last_undegraded":"2024-03-06T07:20:10.772428+0000","last_fullsized":"2024-03-06T07:20:10.772428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:19.051158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272770+0000","last_change":"2024-03-06T07:20:03.730818+0000","last_active":"2024-03-06T07:20:11.272770+0000","last_peered":"2024-03-06T07:20:11.272770+0000","last_clean":"2024-03-06T07:20:11.272770+0000","last_became_active":"2024-03-06T07:20:03.730026+0000","last_became_peered":"2024-03-06T07:20:03.730026+0000","last_unstale":"2024-03-06T07:20:11.272770+0000","last_undegraded":"2024-03-06T07:20:11.272770+0000","last_fullsized":"2024-03-06T07:20:11.272770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773002+0000","last_change":"2024-03-06T07:20:09.524939+0000","last_active":"2024-03-06T07:20:10.773002+0000","last_peered":"2024-03-06T07:20:10.773002+0000","last_clean":"2024-03-06T07:20:10.773002+0000","last_became_active":"2024-03-06T07:20:09.524188+0000","last_became_peered":"2024-03-06T07:20:09.524188+0000","last_unstale":"2024-03-06T07:20:10.773002+0000","last_undegraded":"2024-03-06T07:20:10.773002+0000","last_fullsized":"2024-03-06T07:20:10.773002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744330+0000","last_change":"2024-03-06T07:20:04.733611+0000","last_active":"2024-03-06T07:20:10.744330+0000","last_peered":"2024-03-06T07:20:10.744330+0000","last_clean":"2024-03-06T07:20:10.744330+0000","last_became_active":"2024-03-06T07:20:04.733340+0000","last_became_peered":"2024-03-06T07:20:04.733340+0000","last_unstale":"2024-03-06T07:20:10.744330+0000","last_undegraded":"2024-03-06T07:20:10.744330+0000","last_fullsized":"2024-03-06T07:20:10.744330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273220+0000","last_change":"2024-03-06T07:20:03.720793+0000","last_active":"2024-03-06T07:20:11.273220+0000","last_peered":"2024-03-06T07:20:11.273220+0000","last_clean":"2024-03-06T07:20:11.273220+0000","last_became_active":"2024-03-06T07:20:03.720504+0000","last_became_peered":"2024-03-06T07:20:03.720504+0000","last_unstale":"2024-03-06T07:20:11.273220+0000","last_undegraded":"2024-03-06T07:20:11.273220+0000","last_fullsized":"2024-03-06T07:20:11.273220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272866+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:20:11.272866+0000","last_peered":"2024-03-06T07:20:11.272866+0000","last_clean":"2024-03-06T07:20:11.272866+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:20:11.272866+0000","last_undegraded":"2024-03-06T07:20:11.272866+0000","last_fullsized":"2024-03-06T07:20:11.272866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745518+0000","last_change":"2024-03-06T07:20:09.525305+0000","last_active":"2024-03-06T07:20:10.745518+0000","last_peered":"2024-03-06T07:20:10.745518+0000","last_clean":"2024-03-06T07:20:10.745518+0000","last_became_active":"2024-03-06T07:20:09.524930+0000","last_became_peered":"2024-03-06T07:20:09.524930+0000","last_unstale":"2024-03-06T07:20:10.745518+0000","last_undegraded":"2024-03-06T07:20:10.745518+0000","last_fullsized":"2024-03-06T07:20:10.745518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.731569+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.731301+0000","last_became_peered":"2024-03-06T07:20:04.731301+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877664+0000","last_change":"2024-03-06T07:20:03.716123+0000","last_active":"2024-03-06T07:20:10.877664+0000","last_peered":"2024-03-06T07:20:10.877664+0000","last_clean":"2024-03-06T07:20:10.877664+0000","last_became_active":"2024-03-06T07:20:03.715853+0000","last_became_peered":"2024-03-06T07:20:03.715853+0000","last_unstale":"2024-03-06T07:20:10.877664+0000","last_undegraded":"2024-03-06T07:20:10.877664+0000","last_fullsized":"2024-03-06T07:20:10.877664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744756+0000","last_change":"2024-03-06T07:20:09.524337+0000","last_active":"2024-03-06T07:20:10.744756+0000","last_peered":"2024-03-06T07:20:10.744756+0000","last_clean":"2024-03-06T07:20:10.744756+0000","last_became_active":"2024-03-06T07:20:09.523876+0000","last_became_peered":"2024-03-06T07:20:09.523876+0000","last_unstale":"2024-03-06T07:20:10.744756+0000","last_undegraded":"2024-03-06T07:20:10.744756+0000","last_fullsized":"2024-03-06T07:20:10.744756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273623+0000","last_change":"2024-03-06T07:20:04.736879+0000","last_active":"2024-03-06T07:20:11.273623+0000","last_peered":"2024-03-06T07:20:11.273623+0000","last_clean":"2024-03-06T07:20:11.273623+0000","last_became_active":"2024-03-06T07:20:04.736271+0000","last_became_peered":"2024-03-06T07:20:04.736271+0000","last_unstale":"2024-03-06T07:20:11.273623+0000","last_undegraded":"2024-03-06T07:20:11.273623+0000","last_fullsized":"2024-03-06T07:20:11.273623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273465+0000","last_change":"2024-03-06T07:20:03.721336+0000","last_active":"2024-03-06T07:20:11.273465+0000","last_peered":"2024-03-06T07:20:11.273465+0000","last_clean":"2024-03-06T07:20:11.273465+0000","last_became_active":"2024-03-06T07:20:03.718827+0000","last_became_peered":"2024-03-06T07:20:03.718827+0000","last_unstale":"2024-03-06T07:20:11.273465+0000","last_undegraded":"2024-03-06T07:20:11.273465+0000","last_fullsized":"2024-03-06T07:20:11.273465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772927+0000","last_change":"2024-03-06T07:20:08.756386+0000","last_active":"2024-03-06T07:20:10.772927+0000","last_peered":"2024-03-06T07:20:10.772927+0000","last_clean":"2024-03-06T07:20:10.772927+0000","last_became_active":"2024-03-06T07:20:08.755970+0000","last_became_peered":"2024-03-06T07:20:08.755970+0000","last_unstale":"2024-03-06T07:20:10.772927+0000","last_undegraded":"2024-03-06T07:20:10.772927+0000","last_fullsized":"2024-03-06T07:20:10.772927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273864+0000","last_change":"2024-03-06T07:20:04.737597+0000","last_active":"2024-03-06T07:20:11.273864+0000","last_peered":"2024-03-06T07:20:11.273864+0000","last_clean":"2024-03-06T07:20:11.273864+0000","last_became_active":"2024-03-06T07:20:04.735046+0000","last_became_peered":"2024-03-06T07:20:04.735046+0000","last_unstale":"2024-03-06T07:20:11.273864+0000","last_undegraded":"2024-03-06T07:20:11.273864+0000","last_fullsized":"2024-03-06T07:20:11.273864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:16.868161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877519+0000","last_change":"2024-03-06T07:20:03.712917+0000","last_active":"2024-03-06T07:20:10.877519+0000","last_peered":"2024-03-06T07:20:10.877519+0000","last_clean":"2024-03-06T07:20:10.877519+0000","last_became_active":"2024-03-06T07:20:03.712584+0000","last_became_peered":"2024-03-06T07:20:03.712584+0000","last_unstale":"2024-03-06T07:20:10.877519+0000","last_undegraded":"2024-03-06T07:20:10.877519+0000","last_fullsized":"2024-03-06T07:20:10.877519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272841+0000","last_change":"2024-03-06T07:20:09.525362+0000","last_active":"2024-03-06T07:20:11.272841+0000","last_peered":"2024-03-06T07:20:11.272841+0000","last_clean":"2024-03-06T07:20:11.272841+0000","last_became_active":"2024-03-06T07:20:09.525026+0000","last_became_peered":"2024-03-06T07:20:09.525026+0000","last_unstale":"2024-03-06T07:20:11.272841+0000","last_undegraded":"2024-03-06T07:20:11.272841+0000","last_fullsized":"2024-03-06T07:20:11.272841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:20.377540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273503+0000","last_change":"2024-03-06T07:20:04.730553+0000","last_active":"2024-03-06T07:20:11.273503+0000","last_peered":"2024-03-06T07:20:11.273503+0000","last_clean":"2024-03-06T07:20:11.273503+0000","last_became_active":"2024-03-06T07:20:04.730222+0000","last_became_peered":"2024-03-06T07:20:04.730222+0000","last_unstale":"2024-03-06T07:20:11.273503+0000","last_undegraded":"2024-03-06T07:20:11.273503+0000","last_fullsized":"2024-03-06T07:20:11.273503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746578+0000","last_change":"2024-03-06T07:20:03.718268+0000","last_active":"2024-03-06T07:20:10.746578+0000","last_peered":"2024-03-06T07:20:10.746578+0000","last_clean":"2024-03-06T07:20:10.746578+0000","last_became_active":"2024-03-06T07:20:03.717969+0000","last_became_peered":"2024-03-06T07:20:03.717969+0000","last_unstale":"2024-03-06T07:20:10.746578+0000","last_undegraded":"2024-03-06T07:20:10.746578+0000","last_fullsized":"2024-03-06T07:20:10.746578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745335+0000","last_change":"2024-03-06T07:20:08.756182+0000","last_active":"2024-03-06T07:20:10.745335+0000","last_peered":"2024-03-06T07:20:10.745335+0000","last_clean":"2024-03-06T07:20:10.745335+0000","last_became_active":"2024-03-06T07:20:08.755784+0000","last_became_peered":"2024-03-06T07:20:08.755784+0000","last_unstale":"2024-03-06T07:20:10.745335+0000","last_undegraded":"2024-03-06T07:20:10.745335+0000","last_fullsized":"2024-03-06T07:20:10.745335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273029+0000","last_change":"2024-03-06T07:20:04.729827+0000","last_active":"2024-03-06T07:20:11.273029+0000","last_peered":"2024-03-06T07:20:11.273029+0000","last_clean":"2024-03-06T07:20:11.273029+0000","last_became_active":"2024-03-06T07:20:04.729566+0000","last_became_peered":"2024-03-06T07:20:04.729566+0000","last_unstale":"2024-03-06T07:20:11.273029+0000","last_undegraded":"2024-03-06T07:20:11.273029+0000","last_fullsized":"2024-03-06T07:20:11.273029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877244+0000","last_change":"2024-03-06T07:20:03.723471+0000","last_active":"2024-03-06T07:20:10.877244+0000","last_peered":"2024-03-06T07:20:10.877244+0000","last_clean":"2024-03-06T07:20:10.877244+0000","last_became_active":"2024-03-06T07:20:03.723192+0000","last_became_peered":"2024-03-06T07:20:03.723192+0000","last_unstale":"2024-03-06T07:20:10.877244+0000","last_undegraded":"2024-03-06T07:20:10.877244+0000","last_fullsized":"2024-03-06T07:20:10.877244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772877+0000","last_change":"2024-03-06T07:20:08.760930+0000","last_active":"2024-03-06T07:20:10.772877+0000","last_peered":"2024-03-06T07:20:10.772877+0000","last_clean":"2024-03-06T07:20:10.772877+0000","last_became_active":"2024-03-06T07:20:08.760027+0000","last_became_peered":"2024-03-06T07:20:08.760027+0000","last_unstale":"2024-03-06T07:20:10.772877+0000","last_undegraded":"2024-03-06T07:20:10.772877+0000","last_fullsized":"2024-03-06T07:20:10.772877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744694+0000","last_change":"2024-03-06T07:20:04.717033+0000","last_active":"2024-03-06T07:20:10.744694+0000","last_peered":"2024-03-06T07:20:10.744694+0000","last_clean":"2024-03-06T07:20:10.744694+0000","last_became_active":"2024-03-06T07:20:04.716742+0000","last_became_peered":"2024-03-06T07:20:04.716742+0000","last_unstale":"2024-03-06T07:20:10.744694+0000","last_undegraded":"2024-03-06T07:20:10.744694+0000","last_fullsized":"2024-03-06T07:20:10.744694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745234+0000","last_change":"2024-03-06T07:20:03.717805+0000","last_active":"2024-03-06T07:20:10.745234+0000","last_peered":"2024-03-06T07:20:10.745234+0000","last_clean":"2024-03-06T07:20:10.745234+0000","last_became_active":"2024-03-06T07:20:03.717048+0000","last_became_peered":"2024-03-06T07:20:03.717048+0000","last_unstale":"2024-03-06T07:20:10.745234+0000","last_undegraded":"2024-03-06T07:20:10.745234+0000","last_fullsized":"2024-03-06T07:20:10.745234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744965+0000","last_change":"2024-03-06T07:20:09.525949+0000","last_active":"2024-03-06T07:20:10.744965+0000","last_peered":"2024-03-06T07:20:10.744965+0000","last_clean":"2024-03-06T07:20:10.744965+0000","last_became_active":"2024-03-06T07:20:09.525427+0000","last_became_peered":"2024-03-06T07:20:09.525427+0000","last_unstale":"2024-03-06T07:20:10.744965+0000","last_undegraded":"2024-03-06T07:20:10.744965+0000","last_fullsized":"2024-03-06T07:20:10.744965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:44.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273942+0000","last_change":"2024-03-06T07:20:04.732333+0000","last_active":"2024-03-06T07:20:11.273942+0000","last_peered":"2024-03-06T07:20:11.273942+0000","last_clean":"2024-03-06T07:20:11.273942+0000","last_became_active":"2024-03-06T07:20:04.732069+0000","last_became_peered":"2024-03-06T07:20:04.732069+0000","last_unstale":"2024-03-06T07:20:11.273942+0000","last_undegraded":"2024-03-06T07:20:11.273942+0000","last_fullsized":"2024-03-06T07:20:11.273942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274155+0000","last_change":"2024-03-06T07:20:03.719609+0000","last_active":"2024-03-06T07:20:11.274155+0000","last_peered":"2024-03-06T07:20:11.274155+0000","last_clean":"2024-03-06T07:20:11.274155+0000","last_became_active":"2024-03-06T07:20:03.719325+0000","last_became_peered":"2024-03-06T07:20:03.719325+0000","last_unstale":"2024-03-06T07:20:11.274155+0000","last_undegraded":"2024-03-06T07:20:11.274155+0000","last_fullsized":"2024-03-06T07:20:11.274155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744816+0000","last_change":"2024-03-06T07:20:09.524644+0000","last_active":"2024-03-06T07:20:10.744816+0000","last_peered":"2024-03-06T07:20:10.744816+0000","last_clean":"2024-03-06T07:20:10.744816+0000","last_became_active":"2024-03-06T07:20:09.524220+0000","last_became_peered":"2024-03-06T07:20:09.524220+0000","last_unstale":"2024-03-06T07:20:10.744816+0000","last_undegraded":"2024-03-06T07:20:10.744816+0000","last_fullsized":"2024-03-06T07:20:10.744816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:30.841732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772642+0000","last_change":"2024-03-06T07:20:04.735008+0000","last_active":"2024-03-06T07:20:10.772642+0000","last_peered":"2024-03-06T07:20:10.772642+0000","last_clean":"2024-03-06T07:20:10.772642+0000","last_became_active":"2024-03-06T07:20:04.734709+0000","last_became_peered":"2024-03-06T07:20:04.734709+0000","last_unstale":"2024-03-06T07:20:10.772642+0000","last_undegraded":"2024-03-06T07:20:10.772642+0000","last_fullsized":"2024-03-06T07:20:10.772642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773006+0000","last_change":"2024-03-06T07:20:03.722267+0000","last_active":"2024-03-06T07:20:10.773006+0000","last_peered":"2024-03-06T07:20:10.773006+0000","last_clean":"2024-03-06T07:20:10.773006+0000","last_became_active":"2024-03-06T07:20:03.721952+0000","last_became_peered":"2024-03-06T07:20:03.721952+0000","last_unstale":"2024-03-06T07:20:10.773006+0000","last_undegraded":"2024-03-06T07:20:10.773006+0000","last_fullsized":"2024-03-06T07:20:10.773006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273402+0000","last_change":"2024-03-06T07:20:09.524376+0000","last_active":"2024-03-06T07:20:11.273402+0000","last_peered":"2024-03-06T07:20:11.273402+0000","last_clean":"2024-03-06T07:20:11.273402+0000","last_became_active":"2024-03-06T07:20:09.524010+0000","last_became_peered":"2024-03-06T07:20:09.524010+0000","last_unstale":"2024-03-06T07:20:11.273402+0000","last_undegraded":"2024-03-06T07:20:11.273402+0000","last_fullsized":"2024-03-06T07:20:11.273402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:50.092001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273678+0000","last_change":"2024-03-06T07:20:04.734379+0000","last_active":"2024-03-06T07:20:11.273678+0000","last_peered":"2024-03-06T07:20:11.273678+0000","last_clean":"2024-03-06T07:20:11.273678+0000","last_became_active":"2024-03-06T07:20:04.733727+0000","last_became_peered":"2024-03-06T07:20:04.733727+0000","last_unstale":"2024-03-06T07:20:11.273678+0000","last_undegraded":"2024-03-06T07:20:11.273678+0000","last_fullsized":"2024-03-06T07:20:11.273678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:13.635762+0000","last_change":"2024-03-06T07:20:03.718296+0000","last_active":"2024-03-06T07:20:13.635762+0000","last_peered":"2024-03-06T07:20:13.635762+0000","last_clean":"2024-03-06T07:20:13.635762+0000","last_became_active":"2024-03-06T07:20:03.718015+0000","last_became_peered":"2024-03-06T07:20:03.718015+0000","last_unstale":"2024-03-06T07:20:13.635762+0000","last_undegraded":"2024-03-06T07:20:13.635762+0000","last_fullsized":"2024-03-06T07:20:13.635762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273542+0000","last_change":"2024-03-06T07:20:08.764548+0000","last_active":"2024-03-06T07:20:11.273542+0000","last_peered":"2024-03-06T07:20:11.273542+0000","last_clean":"2024-03-06T07:20:11.273542+0000","last_became_active":"2024-03-06T07:20:08.764175+0000","last_became_peered":"2024-03-06T07:20:08.764175+0000","last_unstale":"2024-03-06T07:20:11.273542+0000","last_undegraded":"2024-03-06T07:20:11.273542+0000","last_fullsized":"2024-03-06T07:20:11.273542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746370+0000","last_change":"2024-03-06T07:20:04.733069+0000","last_active":"2024-03-06T07:20:10.746370+0000","last_peered":"2024-03-06T07:20:10.746370+0000","last_clean":"2024-03-06T07:20:10.746370+0000","last_became_active":"2024-03-06T07:20:04.732656+0000","last_became_peered":"2024-03-06T07:20:04.732656+0000","last_unstale":"2024-03-06T07:20:10.746370+0000","last_undegraded":"2024-03-06T07:20:10.746370+0000","last_fullsized":"2024-03-06T07:20:10.746370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273433+0000","last_change":"2024-03-06T07:20:03.728079+0000","last_active":"2024-03-06T07:20:11.273433+0000","last_peered":"2024-03-06T07:20:11.273433+0000","last_clean":"2024-03-06T07:20:11.273433+0000","last_became_active":"2024-03-06T07:20:03.727628+0000","last_became_peered":"2024-03-06T07:20:03.727628+0000","last_unstale":"2024-03-06T07:20:11.273433+0000","last_undegraded":"2024-03-06T07:20:11.273433+0000","last_fullsized":"2024-03-06T07:20:11.273433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746197+0000","last_change":"2024-03-06T07:20:08.753762+0000","last_active":"2024-03-06T07:20:10.746197+0000","last_peered":"2024-03-06T07:20:10.746197+0000","last_clean":"2024-03-06T07:20:10.746197+0000","last_became_active":"2024-03-06T07:20:08.753331+0000","last_became_peered":"2024-03-06T07:20:08.753331+0000","last_unstale":"2024-03-06T07:20:10.746197+0000","last_undegraded":"2024-03-06T07:20:10.746197+0000","last_fullsized":"2024-03-06T07:20:10.746197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772817+0000","last_change":"2024-03-06T07:20:04.731376+0000","last_active":"2024-03-06T07:20:10.772817+0000","last_peered":"2024-03-06T07:20:10.772817+0000","last_clean":"2024-03-06T07:20:10.772817+0000","last_became_active":"2024-03-06T07:20:04.730731+0000","last_became_peered":"2024-03-06T07:20:04.730731+0000","last_unstale":"2024-03-06T07:20:10.772817+0000","last_undegraded":"2024-03-06T07:20:10.772817+0000","last_fullsized":"2024-03-06T07:20:10.772817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273372+0000","last_change":"2024-03-06T07:20:03.727603+0000","last_active":"2024-03-06T07:20:11.273372+0000","last_peered":"2024-03-06T07:20:11.273372+0000","last_clean":"2024-03-06T07:20:11.273372+0000","last_became_active":"2024-03-06T07:20:03.727238+0000","last_became_peered":"2024-03-06T07:20:03.727238+0000","last_unstale":"2024-03-06T07:20:11.273372+0000","last_undegraded":"2024-03-06T07:20:11.273372+0000","last_fullsized":"2024-03-06T07:20:11.273372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744238+0000","last_change":"2024-03-06T07:20:08.759817+0000","last_active":"2024-03-06T07:20:10.744238+0000","last_peered":"2024-03-06T07:20:10.744238+0000","last_clean":"2024-03-06T07:20:10.744238+0000","last_became_active":"2024-03-06T07:20:08.759430+0000","last_became_peered":"2024-03-06T07:20:08.759430+0000","last_unstale":"2024-03-06T07:20:10.744238+0000","last_undegraded":"2024-03-06T07:20:10.744238+0000","last_fullsized":"2024-03-06T07:20:10.744238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:59.350405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273280+0000","last_change":"2024-03-06T07:20:04.736807+0000","last_active":"2024-03-06T07:20:11.273280+0000","last_peered":"2024-03-06T07:20:11.273280+0000","last_clean":"2024-03-06T07:20:11.273280+0000","last_became_active":"2024-03-06T07:20:04.736552+0000","last_became_peered":"2024-03-06T07:20:04.736552+0000","last_unstale":"2024-03-06T07:20:11.273280+0000","last_undegraded":"2024-03-06T07:20:11.273280+0000","last_fullsized":"2024-03-06T07:20:11.273280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745647+0000","last_change":"2024-03-06T07:20:03.725869+0000","last_active":"2024-03-06T07:20:10.745647+0000","last_peered":"2024-03-06T07:20:10.745647+0000","last_clean":"2024-03-06T07:20:10.745647+0000","last_became_active":"2024-03-06T07:20:03.725563+0000","last_became_peered":"2024-03-06T07:20:03.725563+0000","last_unstale":"2024-03-06T07:20:10.745647+0000","last_undegraded":"2024-03-06T07:20:10.745647+0000","last_fullsized":"2024-03-06T07:20:10.745647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744191+0000","last_change":"2024-03-06T07:20:09.527243+0000","last_active":"2024-03-06T07:20:10.744191+0000","last_peered":"2024-03-06T07:20:10.744191+0000","last_clean":"2024-03-06T07:20:10.744191+0000","last_became_active":"2024-03-06T07:20:09.526607+0000","last_became_peered":"2024-03-06T07:20:09.526607+0000","last_unstale":"2024-03-06T07:20:10.744191+0000","last_undegraded":"2024-03-06T07:20:10.744191+0000","last_fullsized":"2024-03-06T07:20:10.744191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:34.603979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772826+0000","last_change":"2024-03-06T07:20:09.524593+0000","last_active":"2024-03-06T07:20:10.772826+0000","last_peered":"2024-03-06T07:20:10.772826+0000","last_clean":"2024-03-06T07:20:10.772826+0000","last_became_active":"2024-03-06T07:20:09.524244+0000","last_became_peered":"2024-03-06T07:20:09.524244+0000","last_unstale":"2024-03-06T07:20:10.772826+0000","last_undegraded":"2024-03-06T07:20:10.772826+0000","last_fullsized":"2024-03-06T07:20:10.772826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272939+0000","last_change":"2024-03-06T07:20:04.718433+0000","last_active":"2024-03-06T07:20:11.272939+0000","last_peered":"2024-03-06T07:20:11.272939+0000","last_clean":"2024-03-06T07:20:11.272939+0000","last_became_active":"2024-03-06T07:20:04.718154+0000","last_became_peered":"2024-03-06T07:20:04.718154+0000","last_unstale":"2024-03-06T07:20:11.272939+0000","last_undegraded":"2024-03-06T07:20:11.272939+0000","last_fullsized":"2024-03-06T07:20:11.272939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273133+0000","last_change":"2024-03-06T07:20:03.727685+0000","last_active":"2024-03-06T07:20:11.273133+0000","last_peered":"2024-03-06T07:20:11.273133+0000","last_clean":"2024-03-06T07:20:11.273133+0000","last_became_active":"2024-03-06T07:20:03.727363+0000","last_became_peered":"2024-03-06T07:20:03.727363+0000","last_unstale":"2024-03-06T07:20:11.273133+0000","last_undegraded":"2024-03-06T07:20:11.273133+0000","last_fullsized":"2024-03-06T07:20:11.273133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745586+0000","last_change":"2024-03-06T07:20:08.760387+0000","last_active":"2024-03-06T07:20:10.745586+0000","last_peered":"2024-03-06T07:20:10.745586+0000","last_clean":"2024-03-06T07:20:10.745586+0000","last_became_active":"2024-03-06T07:20:08.760017+0000","last_became_peered":"2024-03-06T07:20:08.760017+0000","last_unstale":"2024-03-06T07:20:10.745586+0000","last_undegraded":"2024-03-06T07:20:10.745586+0000","last_fullsized":"2024-03-06T07:20:10.745586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744172+0000","last_change":"2024-03-06T07:20:04.721795+0000","last_active":"2024-03-06T07:20:10.744172+0000","last_peered":"2024-03-06T07:20:10.744172+0000","last_clean":"2024-03-06T07:20:10.744172+0000","last_became_active":"2024-03-06T07:20:04.720723+0000","last_became_peered":"2024-03-06T07:20:04.720723+0000","last_unstale":"2024-03-06T07:20:10.744172+0000","last_undegraded":"2024-03-06T07:20:10.744172+0000","last_fullsized":"2024-03-06T07:20:10.744172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744343+0000","last_change":"2024-03-06T07:20:03.722350+0000","last_active":"2024-03-06T07:20:10.744343+0000","last_peered":"2024-03-06T07:20:10.744343+0000","last_clean":"2024-03-06T07:20:10.744343+0000","last_became_active":"2024-03-06T07:20:03.722059+0000","last_became_peered":"2024-03-06T07:20:03.722059+0000","last_unstale":"2024-03-06T07:20:10.744343+0000","last_undegraded":"2024-03-06T07:20:10.744343+0000","last_fullsized":"2024-03-06T07:20:10.744343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744977+0000","last_change":"2024-03-06T07:20:03.717393+0000","last_active":"2024-03-06T07:20:10.744977+0000","last_peered":"2024-03-06T07:20:10.744977+0000","last_clean":"2024-03-06T07:20:10.744977+0000","last_became_active":"2024-03-06T07:20:03.717051+0000","last_became_peered":"2024-03-06T07:20:03.717051+0000","last_unstale":"2024-03-06T07:20:10.744977+0000","last_undegraded":"2024-03-06T07:20:10.744977+0000","last_fullsized":"2024-03-06T07:20:10.744977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772980+0000","last_change":"2024-03-06T07:20:04.718195+0000","last_active":"2024-03-06T07:20:10.772980+0000","last_peered":"2024-03-06T07:20:10.772980+0000","last_clean":"2024-03-06T07:20:10.772980+0000","last_became_active":"2024-03-06T07:20:04.717447+0000","last_became_peered":"2024-03-06T07:20:04.717447+0000","last_unstale":"2024-03-06T07:20:10.772980+0000","last_undegraded":"2024-03-06T07:20:10.772980+0000","last_fullsized":"2024-03-06T07:20:10.772980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877565+0000","last_change":"2024-03-06T07:20:08.759429+0000","last_active":"2024-03-06T07:20:10.877565+0000","last_peered":"2024-03-06T07:20:10.877565+0000","last_clean":"2024-03-06T07:20:10.877565+0000","last_became_active":"2024-03-06T07:20:08.759038+0000","last_became_peered":"2024-03-06T07:20:08.759038+0000","last_unstale":"2024-03-06T07:20:10.877565+0000","last_undegraded":"2024-03-06T07:20:10.877565+0000","last_fullsized":"2024-03-06T07:20:10.877565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:33.125882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744524+0000","last_change":"2024-03-06T07:20:03.730392+0000","last_active":"2024-03-06T07:20:10.744524+0000","last_peered":"2024-03-06T07:20:10.744524+0000","last_clean":"2024-03-06T07:20:10.744524+0000","last_became_active":"2024-03-06T07:20:03.730088+0000","last_became_peered":"2024-03-06T07:20:03.730088+0000","last_unstale":"2024-03-06T07:20:10.744524+0000","last_undegraded":"2024-03-06T07:20:10.744524+0000","last_fullsized":"2024-03-06T07:20:10.744524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745946+0000","last_change":"2024-03-06T07:20:04.724861+0000","last_active":"2024-03-06T07:20:10.745946+0000","last_peered":"2024-03-06T07:20:10.745946+0000","last_clean":"2024-03-06T07:20:10.745946+0000","last_became_active":"2024-03-06T07:20:04.724584+0000","last_became_peered":"2024-03-06T07:20:04.724584+0000","last_unstale":"2024-03-06T07:20:10.745946+0000","last_undegraded":"2024-03-06T07:20:10.745946+0000","last_fullsized":"2024-03-06T07:20:10.745946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:05.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744887+0000","last_change":"2024-03-06T07:20:08.758137+0000","last_active":"2024-03-06T07:20:10.744887+0000","last_peered":"2024-03-06T07:20:10.744887+0000","last_clean":"2024-03-06T07:20:10.744887+0000","last_became_active":"2024-03-06T07:20:08.757711+0000","last_became_peered":"2024-03-06T07:20:08.757711+0000","last_unstale":"2024-03-06T07:20:10.744887+0000","last_undegraded":"2024-03-06T07:20:10.744887+0000","last_fullsized":"2024-03-06T07:20:10.744887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:47.463176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744591+0000","last_change":"2024-03-06T07:20:04.724223+0000","last_active":"2024-03-06T07:20:10.744591+0000","last_peered":"2024-03-06T07:20:10.744591+0000","last_clean":"2024-03-06T07:20:10.744591+0000","last_became_active":"2024-03-06T07:20:04.723946+0000","last_became_peered":"2024-03-06T07:20:04.723946+0000","last_unstale":"2024-03-06T07:20:10.744591+0000","last_undegraded":"2024-03-06T07:20:10.744591+0000","last_fullsized":"2024-03-06T07:20:10.744591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877317+0000","last_change":"2024-03-06T07:20:04.730151+0000","last_active":"2024-03-06T07:20:10.877317+0000","last_peered":"2024-03-06T07:20:10.877317+0000","last_clean":"2024-03-06T07:20:10.877317+0000","last_became_active":"2024-03-06T07:20:04.729835+0000","last_became_peered":"2024-03-06T07:20:04.729835+0000","last_unstale":"2024-03-06T07:20:10.877317+0000","last_undegraded":"2024-03-06T07:20:10.877317+0000","last_fullsized":"2024-03-06T07:20:10.877317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274150+0000","last_change":"2024-03-06T07:20:04.734333+0000","last_active":"2024-03-06T07:20:11.274150+0000","last_peered":"2024-03-06T07:20:11.274150+0000","last_clean":"2024-03-06T07:20:11.274150+0000","last_became_active":"2024-03-06T07:20:04.734075+0000","last_became_peered":"2024-03-06T07:20:04.734075+0000","last_unstale":"2024-03-06T07:20:11.274150+0000","last_undegraded":"2024-03-06T07:20:11.274150+0000","last_fullsized":"2024-03-06T07:20:11.274150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772954+0000","last_change":"2024-03-06T07:20:04.722013+0000","last_active":"2024-03-06T07:20:10.772954+0000","last_peered":"2024-03-06T07:20:10.772954+0000","last_clean":"2024-03-06T07:20:10.772954+0000","last_became_active":"2024-03-06T07:20:04.721746+0000","last_became_peered":"2024-03-06T07:20:04.721746+0000","last_unstale":"2024-03-06T07:20:10.772954+0000","last_undegraded":"2024-03-06T07:20:10.772954+0000","last_fullsized":"2024-03-06T07:20:10.772954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744882+0000","last_change":"2024-03-06T07:20:04.732536+0000","last_active":"2024-03-06T07:20:10.744882+0000","last_peered":"2024-03-06T07:20:10.744882+0000","last_clean":"2024-03-06T07:20:10.744882+0000","last_became_active":"2024-03-06T07:20:04.728058+0000","last_became_peered":"2024-03-06T07:20:04.728058+0000","last_unstale":"2024-03-06T07:20:10.744882+0000","last_undegraded":"2024-03-06T07:20:10.744882+0000","last_fullsized":"2024-03-06T07:20:10.744882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273227+0000","last_change":"2024-03-06T07:20:04.723910+0000","last_active":"2024-03-06T07:20:11.273227+0000","last_peered":"2024-03-06T07:20:11.273227+0000","last_clean":"2024-03-06T07:20:11.273227+0000","last_became_active":"2024-03-06T07:20:04.722877+0000","last_became_peered":"2024-03-06T07:20:04.722877+0000","last_unstale":"2024-03-06T07:20:11.273227+0000","last_undegraded":"2024-03-06T07:20:11.273227+0000","last_fullsized":"2024-03-06T07:20:11.273227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745929+0000","last_change":"2024-03-06T07:20:04.734964+0000","last_active":"2024-03-06T07:20:10.745929+0000","last_peered":"2024-03-06T07:20:10.745929+0000","last_clean":"2024-03-06T07:20:10.745929+0000","last_became_active":"2024-03-06T07:20:04.734459+0000","last_became_peered":"2024-03-06T07:20:04.734459+0000","last_unstale":"2024-03-06T07:20:10.745929+0000","last_undegraded":"2024-03-06T07:20:10.745929+0000","last_fullsized":"2024-03-06T07:20:10.745929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744920+0000","last_change":"2024-03-06T07:20:04.725185+0000","last_active":"2024-03-06T07:20:10.744920+0000","last_peered":"2024-03-06T07:20:10.744920+0000","last_clean":"2024-03-06T07:20:10.744920+0000","last_became_active":"2024-03-06T07:20:04.724888+0000","last_became_peered":"2024-03-06T07:20:04.724888+0000","last_unstale":"2024-03-06T07:20:10.744920+0000","last_undegraded":"2024-03-06T07:20:10.744920+0000","last_fullsized":"2024-03-06T07:20:10.744920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273048+0000","last_change":"2024-03-06T07:20:04.735180+0000","last_active":"2024-03-06T07:20:11.273048+0000","last_peered":"2024-03-06T07:20:11.273048+0000","last_clean":"2024-03-06T07:20:11.273048+0000","last_became_active":"2024-03-06T07:20:04.733178+0000","last_became_peered":"2024-03-06T07:20:04.733178+0000","last_unstale":"2024-03-06T07:20:11.273048+0000","last_undegraded":"2024-03-06T07:20:11.273048+0000","last_fullsized":"2024-03-06T07:20:11.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744251+0000","last_change":"2024-03-06T07:20:04.726353+0000","last_active":"2024-03-06T07:20:10.744251+0000","last_peered":"2024-03-06T07:20:10.744251+0000","last_clean":"2024-03-06T07:20:10.744251+0000","last_became_active":"2024-03-06T07:20:04.725518+0000","last_became_peered":"2024-03-06T07:20:04.725518+0000","last_unstale":"2024-03-06T07:20:10.744251+0000","last_undegraded":"2024-03-06T07:20:10.744251+0000","last_fullsized":"2024-03-06T07:20:10.744251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274062+0000","last_change":"2024-03-06T07:20:04.736206+0000","last_active":"2024-03-06T07:20:11.274062+0000","last_peered":"2024-03-06T07:20:11.274062+0000","last_clean":"2024-03-06T07:20:11.274062+0000","last_became_active":"2024-03-06T07:20:04.735949+0000","last_became_peered":"2024-03-06T07:20:04.735949+0000","last_unstale":"2024-03-06T07:20:11.274062+0000","last_undegraded":"2024-03-06T07:20:11.274062+0000","last_fullsized":"2024-03-06T07:20:11.274062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772746+0000","last_change":"2024-03-06T07:20:04.728579+0000","last_active":"2024-03-06T07:20:10.772746+0000","last_peered":"2024-03-06T07:20:10.772746+0000","last_clean":"2024-03-06T07:20:10.772746+0000","last_became_active":"2024-03-06T07:20:04.728300+0000","last_became_peered":"2024-03-06T07:20:04.728300+0000","last_unstale":"2024-03-06T07:20:10.772746+0000","last_undegraded":"2024-03-06T07:20:10.772746+0000","last_fullsized":"2024-03-06T07:20:10.772746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745423+0000","last_change":"2024-03-06T07:20:04.724410+0000","last_active":"2024-03-06T07:20:10.745423+0000","last_peered":"2024-03-06T07:20:10.745423+0000","last_clean":"2024-03-06T07:20:10.745423+0000","last_became_active":"2024-03-06T07:20:04.723916+0000","last_became_peered":"2024-03-06T07:20:04.723916+0000","last_unstale":"2024-03-06T07:20:10.745423+0000","last_undegraded":"2024-03-06T07:20:10.745423+0000","last_fullsized":"2024-03-06T07:20:10.745423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274000+0000","last_change":"2024-03-06T07:20:04.734607+0000","last_active":"2024-03-06T07:20:11.274000+0000","last_peered":"2024-03-06T07:20:11.274000+0000","last_clean":"2024-03-06T07:20:11.274000+0000","last_became_active":"2024-03-06T07:20:04.734081+0000","last_became_peered":"2024-03-06T07:20:04.734081+0000","last_unstale":"2024-03-06T07:20:11.274000+0000","last_undegraded":"2024-03-06T07:20:11.274000+0000","last_fullsized":"2024-03-06T07:20:11.274000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877491+0000","last_change":"2024-03-06T07:20:04.726465+0000","last_active":"2024-03-06T07:20:10.877491+0000","last_peered":"2024-03-06T07:20:10.877491+0000","last_clean":"2024-03-06T07:20:10.877491+0000","last_became_active":"2024-03-06T07:20:04.726114+0000","last_became_peered":"2024-03-06T07:20:04.726114+0000","last_unstale":"2024-03-06T07:20:10.877491+0000","last_undegraded":"2024-03-06T07:20:10.877491+0000","last_fullsized":"2024-03-06T07:20:10.877491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.738589+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.737924+0000","last_became_peered":"2024-03-06T07:20:04.737924+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272964+0000","last_change":"2024-03-06T07:20:04.726607+0000","last_active":"2024-03-06T07:20:11.272964+0000","last_peered":"2024-03-06T07:20:11.272964+0000","last_clean":"2024-03-06T07:20:11.272964+0000","last_became_active":"2024-03-06T07:20:04.726337+0000","last_became_peered":"2024-03-06T07:20:04.726337+0000","last_unstale":"2024-03-06T07:20:11.272964+0000","last_undegraded":"2024-03-06T07:20:11.272964+0000","last_fullsized":"2024-03-06T07:20:11.272964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273000+0000","last_change":"2024-03-06T07:20:04.719555+0000","last_active":"2024-03-06T07:20:11.273000+0000","last_peered":"2024-03-06T07:20:11.273000+0000","last_clean":"2024-03-06T07:20:11.273000+0000","last_became_active":"2024-03-06T07:20:04.719273+0000","last_became_peered":"2024-03-06T07:20:04.719273+0000","last_unstale":"2024-03-06T07:20:11.273000+0000","last_undegraded":"2024-03-06T07:20:11.273000+0000","last_fullsized":"2024-03-06T07:20:11.273000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772701+0000","last_change":"2024-03-06T07:20:04.724038+0000","last_active":"2024-03-06T07:20:10.772701+0000","last_peered":"2024-03-06T07:20:10.772701+0000","last_clean":"2024-03-06T07:20:10.772701+0000","last_became_active":"2024-03-06T07:20:04.723527+0000","last_became_peered":"2024-03-06T07:20:04.723527+0000","last_unstale":"2024-03-06T07:20:10.772701+0000","last_undegraded":"2024-03-06T07:20:10.772701+0000","last_fullsized":"2024-03-06T07:20:10.772701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:32.763183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744248+0000","last_change":"2024-03-06T07:20:04.730004+0000","last_active":"2024-03-06T07:20:10.744248+0000","last_peered":"2024-03-06T07:20:10.744248+0000","last_clean":"2024-03-06T07:20:10.744248+0000","last_became_active":"2024-03-06T07:20:04.729702+0000","last_became_peered":"2024-03-06T07:20:04.729702+0000","last_unstale":"2024-03-06T07:20:10.744248+0000","last_undegraded":"2024-03-06T07:20:10.744248+0000","last_fullsized":"2024-03-06T07:20:10.744248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746363+0000","last_change":"2024-03-06T07:20:04.729091+0000","last_active":"2024-03-06T07:20:10.746363+0000","last_peered":"2024-03-06T07:20:10.746363+0000","last_clean":"2024-03-06T07:20:10.746363+0000","last_became_active":"2024-03-06T07:20:04.728793+0000","last_became_peered":"2024-03-06T07:20:04.728793+0000","last_unstale":"2024-03-06T07:20:10.746363+0000","last_undegraded":"2024-03-06T07:20:10.746363+0000","last_fullsized":"2024-03-06T07:20:10.746363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745590+0000","last_change":"2024-03-06T07:20:04.730775+0000","last_active":"2024-03-06T07:20:10.745590+0000","last_peered":"2024-03-06T07:20:10.745590+0000","last_clean":"2024-03-06T07:20:10.745590+0000","last_became_active":"2024-03-06T07:20:04.730386+0000","last_became_peered":"2024-03-06T07:20:04.730386+0000","last_unstale":"2024-03-06T07:20:10.745590+0000","last_undegraded":"2024-03-06T07:20:10.745590+0000","last_fullsized":"2024-03-06T07:20:10.745590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745842+0000","last_change":"2024-03-06T07:20:04.730885+0000","last_active":"2024-03-06T07:20:10.745842+0000","last_peered":"2024-03-06T07:20:10.745842+0000","last_clean":"2024-03-06T07:20:10.745842+0000","last_became_active":"2024-03-06T07:20:04.730294+0000","last_became_peered":"2024-03-06T07:20:04.730294+0000","last_unstale":"2024-03-06T07:20:10.745842+0000","last_undegraded":"2024-03-06T07:20:10.745842+0000","last_fullsized":"2024-03-06T07:20:10.745842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273816+0000","last_change":"2024-03-06T07:20:04.737430+0000","last_active":"2024-03-06T07:20:11.273816+0000","last_peered":"2024-03-06T07:20:11.273816+0000","last_clean":"2024-03-06T07:20:11.273816+0000","last_became_active":"2024-03-06T07:20:04.734870+0000","last_became_peered":"2024-03-06T07:20:04.734870+0000","last_unstale":"2024-03-06T07:20:11.273816+0000","last_undegraded":"2024-03-06T07:20:11.273816+0000","last_fullsized":"2024-03-06T07:20:11.273816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744080+0000","last_change":"2024-03-06T07:20:04.732622+0000","last_active":"2024-03-06T07:20:10.744080+0000","last_peered":"2024-03-06T07:20:10.744080+0000","last_clean":"2024-03-06T07:20:10.744080+0000","last_became_active":"2024-03-06T07:20:04.732149+0000","last_became_peered":"2024-03-06T07:20:04.732149+0000","last_unstale":"2024-03-06T07:20:10.744080+0000","last_undegraded":"2024-03-06T07:20:10.744080+0000","last_fullsized":"2024-03-06T07:20:10.744080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272952+0000","last_change":"2024-03-06T07:20:04.733067+0000","last_active":"2024-03-06T07:20:11.272952+0000","last_peered":"2024-03-06T07:20:11.272952+0000","last_clean":"2024-03-06T07:20:11.272952+0000","last_became_active":"2024-03-06T07:20:04.732805+0000","last_became_peered":"2024-03-06T07:20:04.732805+0000","last_unstale":"2024-03-06T07:20:11.272952+0000","last_undegraded":"2024-03-06T07:20:11.272952+0000","last_fullsized":"2024-03-06T07:20:11.272952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273984+0000","last_change":"2024-03-06T07:20:04.729682+0000","last_active":"2024-03-06T07:20:11.273984+0000","last_peered":"2024-03-06T07:20:11.273984+0000","last_clean":"2024-03-06T07:20:11.273984+0000","last_became_active":"2024-03-06T07:20:04.729407+0000","last_became_peered":"2024-03-06T07:20:04.729407+0000","last_unstale":"2024-03-06T07:20:11.273984+0000","last_undegraded":"2024-03-06T07:20:11.273984+0000","last_fullsized":"2024-03-06T07:20:11.273984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273787+0000","last_change":"2024-03-06T07:20:04.737815+0000","last_active":"2024-03-06T07:20:11.273787+0000","last_peered":"2024-03-06T07:20:11.273787+0000","last_clean":"2024-03-06T07:20:11.273787+0000","last_became_active":"2024-03-06T07:20:04.735276+0000","last_became_peered":"2024-03-06T07:20:04.735276+0000","last_unstale":"2024-03-06T07:20:11.273787+0000","last_undegraded":"2024-03-06T07:20:11.273787+0000","last_fullsized":"2024-03-06T07:20:11.273787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273449+0000","last_change":"2024-03-06T07:20:04.737108+0000","last_active":"2024-03-06T07:20:11.273449+0000","last_peered":"2024-03-06T07:20:11.273449+0000","last_clean":"2024-03-06T07:20:11.273449+0000","last_became_active":"2024-03-06T07:20:04.736608+0000","last_became_peered":"2024-03-06T07:20:04.736608+0000","last_unstale":"2024-03-06T07:20:11.273449+0000","last_undegraded":"2024-03-06T07:20:11.273449+0000","last_fullsized":"2024-03-06T07:20:11.273449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772691+0000","last_change":"2024-03-06T07:20:04.720240+0000","last_active":"2024-03-06T07:20:10.772691+0000","last_peered":"2024-03-06T07:20:10.772691+0000","last_clean":"2024-03-06T07:20:10.772691+0000","last_became_active":"2024-03-06T07:20:04.719952+0000","last_became_peered":"2024-03-06T07:20:04.719952+0000","last_unstale":"2024-03-06T07:20:10.772691+0000","last_undegraded":"2024-03-06T07:20:10.772691+0000","last_fullsized":"2024-03-06T07:20:10.772691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:13.957015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273519+0000","last_change":"2024-03-06T07:20:04.719505+0000","last_active":"2024-03-06T07:20:11.273519+0000","last_peered":"2024-03-06T07:20:11.273519+0000","last_clean":"2024-03-06T07:20:11.273519+0000","last_became_active":"2024-03-06T07:20:04.718961+0000","last_became_peered":"2024-03-06T07:20:04.718961+0000","last_unstale":"2024-03-06T07:20:11.273519+0000","last_undegraded":"2024-03-06T07:20:11.273519+0000","last_fullsized":"2024-03-06T07:20:11.273519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273711+0000","last_change":"2024-03-06T07:20:04.737365+0000","last_active":"2024-03-06T07:20:11.273711+0000","last_peered":"2024-03-06T07:20:11.273711+0000","last_clean":"2024-03-06T07:20:11.273711+0000","last_became_active":"2024-03-06T07:20:04.734552+0000","last_became_peered":"2024-03-06T07:20:04.734552+0000","last_unstale":"2024-03-06T07:20:11.273711+0000","last_undegraded":"2024-03-06T07:20:11.273711+0000","last_fullsized":"2024-03-06T07:20:11.273711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":698343,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":344064,"data_stored":698399,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:21.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.909+0000 7f48c517a640 1 Processor -- start 2024-03-06T07:20:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.909+0000 7f48c517a640 1 -- start start 2024-03-06T07:20:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48c517a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48c517a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48c0158b60 0x7f48c0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48c517a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00c73d0 con 0x7f48c00c88c0 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48c517a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c00c8dd0 con 0x7f48c0158b60 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48c517a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00be2e0 con 0x7f48c014f360 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48b659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f48c00c88c0 0x7f48c00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58284/0 (socket says 172.21.15.17:58284) 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48b659c640 1 -- 172.21.15.17:0/3684371232 learned_addr learned my addr 172.21.15.17:0/3684371232 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bed9d640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3684371232 >> v2:172.21.15.163:3300/0 conn(0x7f48c0158b60 0x7f48c0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48be59c640 1 -- 172.21.15.17:0/3684371232 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874279493 0 0) 0x7f48c00c73d0 con 0x7f48c00c88c0 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48be59c640 1 -- 172.21.15.17:0/3684371232 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ac003520 con 0x7f48c00c88c0 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bf59e640 1 -- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 msgr2=0x7f48c0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bf59e640 1 -- 172.21.15.17:0/3684371232 >> v1:172.21.15.17:6789/0 conn(0x7f48c00c88c0 legacy=0x7f48c00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bf59e640 1 -- 172.21.15.17:0/3684371232 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00c7670 con 0x7f48c0158b60 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.910+0000 7f48bed9d640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.911+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0158b60 0x7f48c0158ff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009d90 tx=0x7f48b802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abec1803828bbac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.911+0000 7f48be59c640 1 -- 172.21.15.17:0/3684371232 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8039070 con 0x7f48c0158b60 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.911+0000 7f48be59c640 1 -- 172.21.15.17:0/3684371232 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b802cbc0 con 0x7f48c0158b60 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.911+0000 7f48be59c640 1 -- 172.21.15.17:0/3684371232 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8034440 con 0x7f48c0158b60 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 -- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0158b60 msgr2=0x7f48c0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0158b60 0x7f48c0158ff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009d90 tx=0x7f48b802be10 comp rx=0 tx=0).stop 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 -- 172.21.15.17:0/3684371232 shutdown_connections 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c0158b60 0x7f48c0158ff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 --2- 172.21.15.17:0/3684371232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 -- 172.21.15.17:0/3684371232 >> 172.21.15.17:0/3684371232 conn(0x7f48c00ac570 msgr2=0x7f48c00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 -- 172.21.15.17:0/3684371232 shutdown_connections 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.912+0000 7f48c517a640 1 -- 172.21.15.17:0/3684371232 wait complete. 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.913+0000 7f48c517a640 1 Processor -- start 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.913+0000 7f48c517a640 1 -- start start 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c00c88c0 0x7f48c006be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c01ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00c5a50 con 0x7f48c0158b60 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00c7c30 con 0x7f48c00c88c0 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00c5bd0 con 0x7f48c014f360 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38064/0 (socket says 172.21.15.17:38064) 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 -- 172.21.15.17:0/3285248354 learned_addr learned my addr 172.21.15.17:0/3285248354 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48b659c640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c01ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bed9d640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c00c88c0 0x7f48c006be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 -- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 msgr2=0x7f48c01ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c01ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 -- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c00c88c0 msgr2=0x7f48c006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c00c88c0 0x7f48c006be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 -- 172.21.15.17:0/3285248354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00afbd0 con 0x7f48c0158b60 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48bf59e640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009d90 tx=0x7f48b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8039050 con 0x7f48c0158b60 2024-03-06T07:20:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b8045070 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c00af610 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.914+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8009450 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.915+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01edda0 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.916+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c004f330 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.916+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48b803e830 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.916+0000 7f48b7fff640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 0x7f48a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.916+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48b80ad470 con 0x7f48c0158b60 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.916+0000 7f48bed9d640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 0x7f48a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.917+0000 7f48bed9d640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 0x7f48a006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009560 tx=0x7f48b0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:21.921+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f48b807bb40 con 0x7f48c0158b60 2024-03-06T07:20:22.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.098+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48c01edf70 con 0x7f48a006b4d0 2024-03-06T07:20:22.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.106+0000 7f48b7fff640 1 -- 172.21.15.17:0/3285248354 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7f48c01edf70 con 0x7f48a006b4d0 2024-03-06T07:20:22.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:22.113 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 msgr2=0x7f48a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 0x7f48a006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009560 tx=0x7f48b0009190 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 msgr2=0x7f48c01ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009d90 tx=0x7f48b8004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 shutdown_connections 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48a006b4d0 0x7f48a006d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48c0158b60 0x7f48c01ee800 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48c014f360 0x7f48c01ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 --2- 172.21.15.17:0/3285248354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48c00c88c0 0x7f48c006be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 >> 172.21.15.17:0/3285248354 conn(0x7f48c00ac570 msgr2=0x7f48c00c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 shutdown_connections 2024-03-06T07:20:22.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.109+0000 7f48c517a640 1 -- 172.21.15.17:0/3285248354 wait complete. 2024-03-06T07:20:22.121 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-03-06T07:20:21.345200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230916,"kb_used_data":2272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743804,"statfs":{"total":773094113280,"available":772857655296,"internally_reserved":0,"allocated":2326528,"data_stored":2048878,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.602102"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772886+0000","last_change":"2024-03-06T07:20:03.728088+0000","last_active":"2024-03-06T07:20:10.772886+0000","last_peered":"2024-03-06T07:20:10.772886+0000","last_clean":"2024-03-06T07:20:10.772886+0000","last_became_active":"2024-03-06T07:20:03.727805+0000","last_became_peered":"2024-03-06T07:20:03.727805+0000","last_unstale":"2024-03-06T07:20:10.772886+0000","last_undegraded":"2024-03-06T07:20:10.772886+0000","last_fullsized":"2024-03-06T07:20:10.772886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744735+0000","last_change":"2024-03-06T07:20:04.732564+0000","last_active":"2024-03-06T07:20:10.744735+0000","last_peered":"2024-03-06T07:20:10.744735+0000","last_clean":"2024-03-06T07:20:10.744735+0000","last_became_active":"2024-03-06T07:20:04.732256+0000","last_became_peered":"2024-03-06T07:20:04.732256+0000","last_unstale":"2024-03-06T07:20:10.744735+0000","last_undegraded":"2024-03-06T07:20:10.744735+0000","last_fullsized":"2024-03-06T07:20:10.744735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744130+0000","last_change":"2024-03-06T07:20:09.527717+0000","last_active":"2024-03-06T07:20:10.744130+0000","last_peered":"2024-03-06T07:20:10.744130+0000","last_clean":"2024-03-06T07:20:10.744130+0000","last_became_active":"2024-03-06T07:20:09.527324+0000","last_became_peered":"2024-03-06T07:20:09.527324+0000","last_unstale":"2024-03-06T07:20:10.744130+0000","last_undegraded":"2024-03-06T07:20:10.744130+0000","last_fullsized":"2024-03-06T07:20:10.744130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:05.448069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744975+0000","last_change":"2024-03-06T07:20:03.719181+0000","last_active":"2024-03-06T07:20:10.744975+0000","last_peered":"2024-03-06T07:20:10.744975+0000","last_clean":"2024-03-06T07:20:10.744975+0000","last_became_active":"2024-03-06T07:20:03.718904+0000","last_became_peered":"2024-03-06T07:20:03.718904+0000","last_unstale":"2024-03-06T07:20:10.744975+0000","last_undegraded":"2024-03-06T07:20:10.744975+0000","last_fullsized":"2024-03-06T07:20:10.744975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772804+0000","last_change":"2024-03-06T07:20:04.731613+0000","last_active":"2024-03-06T07:20:10.772804+0000","last_peered":"2024-03-06T07:20:10.772804+0000","last_clean":"2024-03-06T07:20:10.772804+0000","last_became_active":"2024-03-06T07:20:04.731105+0000","last_became_peered":"2024-03-06T07:20:04.731105+0000","last_unstale":"2024-03-06T07:20:10.772804+0000","last_undegraded":"2024-03-06T07:20:10.772804+0000","last_fullsized":"2024-03-06T07:20:10.772804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744947+0000","last_change":"2024-03-06T07:20:08.761811+0000","last_active":"2024-03-06T07:20:10.744947+0000","last_peered":"2024-03-06T07:20:10.744947+0000","last_clean":"2024-03-06T07:20:10.744947+0000","last_became_active":"2024-03-06T07:20:08.761354+0000","last_became_peered":"2024-03-06T07:20:08.761354+0000","last_unstale":"2024-03-06T07:20:10.744947+0000","last_undegraded":"2024-03-06T07:20:10.744947+0000","last_fullsized":"2024-03-06T07:20:10.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:52.215187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273914+0000","last_change":"2024-03-06T07:20:03.728086+0000","last_active":"2024-03-06T07:20:11.273914+0000","last_peered":"2024-03-06T07:20:11.273914+0000","last_clean":"2024-03-06T07:20:11.273914+0000","last_became_active":"2024-03-06T07:20:03.727793+0000","last_became_peered":"2024-03-06T07:20:03.727793+0000","last_unstale":"2024-03-06T07:20:11.273914+0000","last_undegraded":"2024-03-06T07:20:11.273914+0000","last_fullsized":"2024-03-06T07:20:11.273914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273526+0000","last_change":"2024-03-06T07:20:04.732751+0000","last_active":"2024-03-06T07:20:11.273526+0000","last_peered":"2024-03-06T07:20:11.273526+0000","last_clean":"2024-03-06T07:20:11.273526+0000","last_became_active":"2024-03-06T07:20:04.731757+0000","last_became_peered":"2024-03-06T07:20:04.731757+0000","last_unstale":"2024-03-06T07:20:11.273526+0000","last_undegraded":"2024-03-06T07:20:11.273526+0000","last_fullsized":"2024-03-06T07:20:11.273526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746076+0000","last_change":"2024-03-06T07:20:08.756321+0000","last_active":"2024-03-06T07:20:10.746076+0000","last_peered":"2024-03-06T07:20:10.746076+0000","last_clean":"2024-03-06T07:20:10.746076+0000","last_became_active":"2024-03-06T07:20:08.755944+0000","last_became_peered":"2024-03-06T07:20:08.755944+0000","last_unstale":"2024-03-06T07:20:10.746076+0000","last_undegraded":"2024-03-06T07:20:10.746076+0000","last_fullsized":"2024-03-06T07:20:10.746076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273206+0000","last_change":"2024-03-06T07:20:03.730453+0000","last_active":"2024-03-06T07:20:11.273206+0000","last_peered":"2024-03-06T07:20:11.273206+0000","last_clean":"2024-03-06T07:20:11.273206+0000","last_became_active":"2024-03-06T07:20:03.730180+0000","last_became_peered":"2024-03-06T07:20:03.730180+0000","last_unstale":"2024-03-06T07:20:11.273206+0000","last_undegraded":"2024-03-06T07:20:11.273206+0000","last_fullsized":"2024-03-06T07:20:11.273206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:04.734206+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:04.733937+0000","last_became_peered":"2024-03-06T07:20:04.733937+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745261+0000","last_change":"2024-03-06T07:20:08.760335+0000","last_active":"2024-03-06T07:20:10.745261+0000","last_peered":"2024-03-06T07:20:10.745261+0000","last_clean":"2024-03-06T07:20:10.745261+0000","last_became_active":"2024-03-06T07:20:08.759945+0000","last_became_peered":"2024-03-06T07:20:08.759945+0000","last_unstale":"2024-03-06T07:20:10.745261+0000","last_undegraded":"2024-03-06T07:20:10.745261+0000","last_fullsized":"2024-03-06T07:20:10.745261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744803+0000","last_change":"2024-03-06T07:20:04.730615+0000","last_active":"2024-03-06T07:20:10.744803+0000","last_peered":"2024-03-06T07:20:10.744803+0000","last_clean":"2024-03-06T07:20:10.744803+0000","last_became_active":"2024-03-06T07:20:04.729978+0000","last_became_peered":"2024-03-06T07:20:04.729978+0000","last_unstale":"2024-03-06T07:20:10.744803+0000","last_undegraded":"2024-03-06T07:20:10.744803+0000","last_fullsized":"2024-03-06T07:20:10.744803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274021+0000","last_change":"2024-03-06T07:20:03.730180+0000","last_active":"2024-03-06T07:20:11.274021+0000","last_peered":"2024-03-06T07:20:11.274021+0000","last_clean":"2024-03-06T07:20:11.274021+0000","last_became_active":"2024-03-06T07:20:03.729812+0000","last_became_peered":"2024-03-06T07:20:03.729812+0000","last_unstale":"2024-03-06T07:20:11.274021+0000","last_undegraded":"2024-03-06T07:20:11.274021+0000","last_fullsized":"2024-03-06T07:20:11.274021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746544+0000","last_change":"2024-03-06T07:20:08.761949+0000","last_active":"2024-03-06T07:20:10.746544+0000","last_peered":"2024-03-06T07:20:10.746544+0000","last_clean":"2024-03-06T07:20:10.746544+0000","last_became_active":"2024-03-06T07:20:08.761590+0000","last_became_peered":"2024-03-06T07:20:08.761590+0000","last_unstale":"2024-03-06T07:20:10.746544+0000","last_undegraded":"2024-03-06T07:20:10.746544+0000","last_fullsized":"2024-03-06T07:20:10.746544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744471+0000","last_change":"2024-03-06T07:20:04.726070+0000","last_active":"2024-03-06T07:20:10.744471+0000","last_peered":"2024-03-06T07:20:10.744471+0000","last_clean":"2024-03-06T07:20:10.744471+0000","last_became_active":"2024-03-06T07:20:04.725563+0000","last_became_peered":"2024-03-06T07:20:04.725563+0000","last_unstale":"2024-03-06T07:20:10.744471+0000","last_undegraded":"2024-03-06T07:20:10.744471+0000","last_fullsized":"2024-03-06T07:20:10.744471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273584+0000","last_change":"2024-03-06T07:20:03.719632+0000","last_active":"2024-03-06T07:20:11.273584+0000","last_peered":"2024-03-06T07:20:11.273584+0000","last_clean":"2024-03-06T07:20:11.273584+0000","last_became_active":"2024-03-06T07:20:03.719328+0000","last_became_peered":"2024-03-06T07:20:03.719328+0000","last_unstale":"2024-03-06T07:20:11.273584+0000","last_undegraded":"2024-03-06T07:20:11.273584+0000","last_fullsized":"2024-03-06T07:20:11.273584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746162+0000","last_change":"2024-03-06T07:20:08.756550+0000","last_active":"2024-03-06T07:20:10.746162+0000","last_peered":"2024-03-06T07:20:10.746162+0000","last_clean":"2024-03-06T07:20:10.746162+0000","last_became_active":"2024-03-06T07:20:08.755838+0000","last_became_peered":"2024-03-06T07:20:08.755838+0000","last_unstale":"2024-03-06T07:20:10.746162+0000","last_undegraded":"2024-03-06T07:20:10.746162+0000","last_fullsized":"2024-03-06T07:20:10.746162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744264+0000","last_change":"2024-03-06T07:20:04.727861+0000","last_active":"2024-03-06T07:20:10.744264+0000","last_peered":"2024-03-06T07:20:10.744264+0000","last_clean":"2024-03-06T07:20:10.744264+0000","last_became_active":"2024-03-06T07:20:04.727579+0000","last_became_peered":"2024-03-06T07:20:04.727579+0000","last_unstale":"2024-03-06T07:20:10.744264+0000","last_undegraded":"2024-03-06T07:20:10.744264+0000","last_fullsized":"2024-03-06T07:20:10.744264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273057+0000","last_change":"2024-03-06T07:20:03.722227+0000","last_active":"2024-03-06T07:20:11.273057+0000","last_peered":"2024-03-06T07:20:11.273057+0000","last_clean":"2024-03-06T07:20:11.273057+0000","last_became_active":"2024-03-06T07:20:03.721962+0000","last_became_peered":"2024-03-06T07:20:03.721962+0000","last_unstale":"2024-03-06T07:20:11.273057+0000","last_undegraded":"2024-03-06T07:20:11.273057+0000","last_fullsized":"2024-03-06T07:20:11.273057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273518+0000","last_change":"2024-03-06T07:20:09.524648+0000","last_active":"2024-03-06T07:20:11.273518+0000","last_peered":"2024-03-06T07:20:11.273518+0000","last_clean":"2024-03-06T07:20:11.273518+0000","last_became_active":"2024-03-06T07:20:09.524267+0000","last_became_peered":"2024-03-06T07:20:09.524267+0000","last_unstale":"2024-03-06T07:20:11.273518+0000","last_undegraded":"2024-03-06T07:20:11.273518+0000","last_fullsized":"2024-03-06T07:20:11.273518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745823+0000","last_change":"2024-03-06T07:20:04.720167+0000","last_active":"2024-03-06T07:20:10.745823+0000","last_peered":"2024-03-06T07:20:10.745823+0000","last_clean":"2024-03-06T07:20:10.745823+0000","last_became_active":"2024-03-06T07:20:04.719907+0000","last_became_peered":"2024-03-06T07:20:04.719907+0000","last_unstale":"2024-03-06T07:20:10.745823+0000","last_undegraded":"2024-03-06T07:20:10.745823+0000","last_fullsized":"2024-03-06T07:20:10.745823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:45.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272765+0000","last_change":"2024-03-06T07:20:03.730868+0000","last_active":"2024-03-06T07:20:11.272765+0000","last_peered":"2024-03-06T07:20:11.272765+0000","last_clean":"2024-03-06T07:20:11.272765+0000","last_became_active":"2024-03-06T07:20:03.729771+0000","last_became_peered":"2024-03-06T07:20:03.729771+0000","last_unstale":"2024-03-06T07:20:11.272765+0000","last_undegraded":"2024-03-06T07:20:11.272765+0000","last_fullsized":"2024-03-06T07:20:11.272765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746511+0000","last_change":"2024-03-06T07:20:08.756023+0000","last_active":"2024-03-06T07:20:10.746511+0000","last_peered":"2024-03-06T07:20:10.746511+0000","last_clean":"2024-03-06T07:20:10.746511+0000","last_became_active":"2024-03-06T07:20:08.755638+0000","last_became_peered":"2024-03-06T07:20:08.755638+0000","last_unstale":"2024-03-06T07:20:10.746511+0000","last_undegraded":"2024-03-06T07:20:10.746511+0000","last_fullsized":"2024-03-06T07:20:10.746511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773057+0000","last_change":"2024-03-06T07:20:04.724646+0000","last_active":"2024-03-06T07:20:10.773057+0000","last_peered":"2024-03-06T07:20:10.773057+0000","last_clean":"2024-03-06T07:20:10.773057+0000","last_became_active":"2024-03-06T07:20:04.723034+0000","last_became_peered":"2024-03-06T07:20:04.723034+0000","last_unstale":"2024-03-06T07:20:10.773057+0000","last_undegraded":"2024-03-06T07:20:10.773057+0000","last_fullsized":"2024-03-06T07:20:10.773057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745736+0000","last_change":"2024-03-06T07:20:03.723209+0000","last_active":"2024-03-06T07:20:10.745736+0000","last_peered":"2024-03-06T07:20:10.745736+0000","last_clean":"2024-03-06T07:20:10.745736+0000","last_became_active":"2024-03-06T07:20:03.722898+0000","last_became_peered":"2024-03-06T07:20:03.722898+0000","last_unstale":"2024-03-06T07:20:10.745736+0000","last_undegraded":"2024-03-06T07:20:10.745736+0000","last_fullsized":"2024-03-06T07:20:10.745736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745558+0000","last_change":"2024-03-06T07:20:09.526554+0000","last_active":"2024-03-06T07:20:10.745558+0000","last_peered":"2024-03-06T07:20:10.745558+0000","last_clean":"2024-03-06T07:20:10.745558+0000","last_became_active":"2024-03-06T07:20:09.526194+0000","last_became_peered":"2024-03-06T07:20:09.526194+0000","last_unstale":"2024-03-06T07:20:10.745558+0000","last_undegraded":"2024-03-06T07:20:10.745558+0000","last_fullsized":"2024-03-06T07:20:10.745558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745015+0000","last_change":"2024-03-06T07:20:04.736741+0000","last_active":"2024-03-06T07:20:10.745015+0000","last_peered":"2024-03-06T07:20:10.745015+0000","last_clean":"2024-03-06T07:20:10.745015+0000","last_became_active":"2024-03-06T07:20:04.736427+0000","last_became_peered":"2024-03-06T07:20:04.736427+0000","last_unstale":"2024-03-06T07:20:10.745015+0000","last_undegraded":"2024-03-06T07:20:10.745015+0000","last_fullsized":"2024-03-06T07:20:10.745015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273597+0000","last_change":"2024-03-06T07:20:03.720361+0000","last_active":"2024-03-06T07:20:11.273597+0000","last_peered":"2024-03-06T07:20:11.273597+0000","last_clean":"2024-03-06T07:20:11.273597+0000","last_became_active":"2024-03-06T07:20:03.718595+0000","last_became_peered":"2024-03-06T07:20:03.718595+0000","last_unstale":"2024-03-06T07:20:11.273597+0000","last_undegraded":"2024-03-06T07:20:11.273597+0000","last_fullsized":"2024-03-06T07:20:11.273597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744052+0000","last_change":"2024-03-06T07:20:09.526923+0000","last_active":"2024-03-06T07:20:10.744052+0000","last_peered":"2024-03-06T07:20:10.744052+0000","last_clean":"2024-03-06T07:20:10.744052+0000","last_became_active":"2024-03-06T07:20:09.526539+0000","last_became_peered":"2024-03-06T07:20:09.526539+0000","last_unstale":"2024-03-06T07:20:10.744052+0000","last_undegraded":"2024-03-06T07:20:10.744052+0000","last_fullsized":"2024-03-06T07:20:10.744052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:30.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273347+0000","last_change":"2024-03-06T07:20:04.720682+0000","last_active":"2024-03-06T07:20:11.273347+0000","last_peered":"2024-03-06T07:20:11.273347+0000","last_clean":"2024-03-06T07:20:11.273347+0000","last_became_active":"2024-03-06T07:20:04.720404+0000","last_became_peered":"2024-03-06T07:20:04.720404+0000","last_unstale":"2024-03-06T07:20:11.273347+0000","last_undegraded":"2024-03-06T07:20:11.273347+0000","last_fullsized":"2024-03-06T07:20:11.273347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274076+0000","last_change":"2024-03-06T07:20:03.727473+0000","last_active":"2024-03-06T07:20:11.274076+0000","last_peered":"2024-03-06T07:20:11.274076+0000","last_clean":"2024-03-06T07:20:11.274076+0000","last_became_active":"2024-03-06T07:20:03.727177+0000","last_became_peered":"2024-03-06T07:20:03.727177+0000","last_unstale":"2024-03-06T07:20:11.274076+0000","last_undegraded":"2024-03-06T07:20:11.274076+0000","last_fullsized":"2024-03-06T07:20:11.274076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772695+0000","last_change":"2024-03-06T07:20:09.525617+0000","last_active":"2024-03-06T07:20:10.772695+0000","last_peered":"2024-03-06T07:20:10.772695+0000","last_clean":"2024-03-06T07:20:10.772695+0000","last_became_active":"2024-03-06T07:20:09.525228+0000","last_became_peered":"2024-03-06T07:20:09.525228+0000","last_unstale":"2024-03-06T07:20:10.772695+0000","last_undegraded":"2024-03-06T07:20:10.772695+0000","last_fullsized":"2024-03-06T07:20:10.772695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273528+0000","last_change":"2024-03-06T07:20:04.725517+0000","last_active":"2024-03-06T07:20:11.273528+0000","last_peered":"2024-03-06T07:20:11.273528+0000","last_clean":"2024-03-06T07:20:11.273528+0000","last_became_active":"2024-03-06T07:20:04.725226+0000","last_became_peered":"2024-03-06T07:20:04.725226+0000","last_unstale":"2024-03-06T07:20:11.273528+0000","last_undegraded":"2024-03-06T07:20:11.273528+0000","last_fullsized":"2024-03-06T07:20:11.273528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:03.721095+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:03.720730+0000","last_became_peered":"2024-03-06T07:20:03.720730+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273165+0000","last_change":"2024-03-06T07:20:08.763568+0000","last_active":"2024-03-06T07:20:11.273165+0000","last_peered":"2024-03-06T07:20:11.273165+0000","last_clean":"2024-03-06T07:20:11.273165+0000","last_became_active":"2024-03-06T07:20:08.763213+0000","last_became_peered":"2024-03-06T07:20:08.763213+0000","last_unstale":"2024-03-06T07:20:11.273165+0000","last_undegraded":"2024-03-06T07:20:11.273165+0000","last_fullsized":"2024-03-06T07:20:11.273165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746237+0000","last_change":"2024-03-06T07:20:04.715991+0000","last_active":"2024-03-06T07:20:10.746237+0000","last_peered":"2024-03-06T07:20:10.746237+0000","last_clean":"2024-03-06T07:20:10.746237+0000","last_became_active":"2024-03-06T07:20:04.715695+0000","last_became_peered":"2024-03-06T07:20:04.715695+0000","last_unstale":"2024-03-06T07:20:10.746237+0000","last_undegraded":"2024-03-06T07:20:10.746237+0000","last_fullsized":"2024-03-06T07:20:10.746237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:19.554698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274213+0000","last_change":"2024-03-06T07:20:03.724950+0000","last_active":"2024-03-06T07:20:11.274213+0000","last_peered":"2024-03-06T07:20:11.274213+0000","last_clean":"2024-03-06T07:20:11.274213+0000","last_became_active":"2024-03-06T07:20:03.724641+0000","last_became_peered":"2024-03-06T07:20:03.724641+0000","last_unstale":"2024-03-06T07:20:11.274213+0000","last_undegraded":"2024-03-06T07:20:11.274213+0000","last_fullsized":"2024-03-06T07:20:11.274213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273385+0000","last_change":"2024-03-06T07:20:08.762922+0000","last_active":"2024-03-06T07:20:11.273385+0000","last_peered":"2024-03-06T07:20:11.273385+0000","last_clean":"2024-03-06T07:20:11.273385+0000","last_became_active":"2024-03-06T07:20:08.762488+0000","last_became_peered":"2024-03-06T07:20:08.762488+0000","last_unstale":"2024-03-06T07:20:11.273385+0000","last_undegraded":"2024-03-06T07:20:11.273385+0000","last_fullsized":"2024-03-06T07:20:11.273385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273235+0000","last_change":"2024-03-06T07:20:04.729950+0000","last_active":"2024-03-06T07:20:11.273235+0000","last_peered":"2024-03-06T07:20:11.273235+0000","last_clean":"2024-03-06T07:20:11.273235+0000","last_became_active":"2024-03-06T07:20:04.729294+0000","last_became_peered":"2024-03-06T07:20:04.729294+0000","last_unstale":"2024-03-06T07:20:11.273235+0000","last_undegraded":"2024-03-06T07:20:11.273235+0000","last_fullsized":"2024-03-06T07:20:11.273235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877392+0000","last_change":"2024-03-06T07:20:03.717344+0000","last_active":"2024-03-06T07:20:10.877392+0000","last_peered":"2024-03-06T07:20:10.877392+0000","last_clean":"2024-03-06T07:20:10.877392+0000","last_became_active":"2024-03-06T07:20:03.717054+0000","last_became_peered":"2024-03-06T07:20:03.717054+0000","last_unstale":"2024-03-06T07:20:10.877392+0000","last_undegraded":"2024-03-06T07:20:10.877392+0000","last_fullsized":"2024-03-06T07:20:10.877392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273655+0000","last_change":"2024-03-06T07:20:08.763839+0000","last_active":"2024-03-06T07:20:11.273655+0000","last_peered":"2024-03-06T07:20:11.273655+0000","last_clean":"2024-03-06T07:20:11.273655+0000","last_became_active":"2024-03-06T07:20:08.763398+0000","last_became_peered":"2024-03-06T07:20:08.763398+0000","last_unstale":"2024-03-06T07:20:11.273655+0000","last_undegraded":"2024-03-06T07:20:11.273655+0000","last_fullsized":"2024-03-06T07:20:11.273655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772428+0000","last_change":"2024-03-06T07:20:04.717362+0000","last_active":"2024-03-06T07:20:10.772428+0000","last_peered":"2024-03-06T07:20:10.772428+0000","last_clean":"2024-03-06T07:20:10.772428+0000","last_became_active":"2024-03-06T07:20:04.717064+0000","last_became_peered":"2024-03-06T07:20:04.717064+0000","last_unstale":"2024-03-06T07:20:10.772428+0000","last_undegraded":"2024-03-06T07:20:10.772428+0000","last_fullsized":"2024-03-06T07:20:10.772428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:19.051158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272770+0000","last_change":"2024-03-06T07:20:03.730818+0000","last_active":"2024-03-06T07:20:11.272770+0000","last_peered":"2024-03-06T07:20:11.272770+0000","last_clean":"2024-03-06T07:20:11.272770+0000","last_became_active":"2024-03-06T07:20:03.730026+0000","last_became_peered":"2024-03-06T07:20:03.730026+0000","last_unstale":"2024-03-06T07:20:11.272770+0000","last_undegraded":"2024-03-06T07:20:11.272770+0000","last_fullsized":"2024-03-06T07:20:11.272770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773002+0000","last_change":"2024-03-06T07:20:09.524939+0000","last_active":"2024-03-06T07:20:10.773002+0000","last_peered":"2024-03-06T07:20:10.773002+0000","last_clean":"2024-03-06T07:20:10.773002+0000","last_became_active":"2024-03-06T07:20:09.524188+0000","last_became_peered":"2024-03-06T07:20:09.524188+0000","last_unstale":"2024-03-06T07:20:10.773002+0000","last_undegraded":"2024-03-06T07:20:10.773002+0000","last_fullsized":"2024-03-06T07:20:10.773002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744330+0000","last_change":"2024-03-06T07:20:04.733611+0000","last_active":"2024-03-06T07:20:10.744330+0000","last_peered":"2024-03-06T07:20:10.744330+0000","last_clean":"2024-03-06T07:20:10.744330+0000","last_became_active":"2024-03-06T07:20:04.733340+0000","last_became_peered":"2024-03-06T07:20:04.733340+0000","last_unstale":"2024-03-06T07:20:10.744330+0000","last_undegraded":"2024-03-06T07:20:10.744330+0000","last_fullsized":"2024-03-06T07:20:10.744330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273220+0000","last_change":"2024-03-06T07:20:03.720793+0000","last_active":"2024-03-06T07:20:11.273220+0000","last_peered":"2024-03-06T07:20:11.273220+0000","last_clean":"2024-03-06T07:20:11.273220+0000","last_became_active":"2024-03-06T07:20:03.720504+0000","last_became_peered":"2024-03-06T07:20:03.720504+0000","last_unstale":"2024-03-06T07:20:11.273220+0000","last_undegraded":"2024-03-06T07:20:11.273220+0000","last_fullsized":"2024-03-06T07:20:11.273220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272866+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:20:11.272866+0000","last_peered":"2024-03-06T07:20:11.272866+0000","last_clean":"2024-03-06T07:20:11.272866+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:20:11.272866+0000","last_undegraded":"2024-03-06T07:20:11.272866+0000","last_fullsized":"2024-03-06T07:20:11.272866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745518+0000","last_change":"2024-03-06T07:20:09.525305+0000","last_active":"2024-03-06T07:20:10.745518+0000","last_peered":"2024-03-06T07:20:10.745518+0000","last_clean":"2024-03-06T07:20:10.745518+0000","last_became_active":"2024-03-06T07:20:09.524930+0000","last_became_peered":"2024-03-06T07:20:09.524930+0000","last_unstale":"2024-03-06T07:20:10.745518+0000","last_undegraded":"2024-03-06T07:20:10.745518+0000","last_fullsized":"2024-03-06T07:20:10.745518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.731569+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.731301+0000","last_became_peered":"2024-03-06T07:20:04.731301+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877664+0000","last_change":"2024-03-06T07:20:03.716123+0000","last_active":"2024-03-06T07:20:10.877664+0000","last_peered":"2024-03-06T07:20:10.877664+0000","last_clean":"2024-03-06T07:20:10.877664+0000","last_became_active":"2024-03-06T07:20:03.715853+0000","last_became_peered":"2024-03-06T07:20:03.715853+0000","last_unstale":"2024-03-06T07:20:10.877664+0000","last_undegraded":"2024-03-06T07:20:10.877664+0000","last_fullsized":"2024-03-06T07:20:10.877664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744756+0000","last_change":"2024-03-06T07:20:09.524337+0000","last_active":"2024-03-06T07:20:10.744756+0000","last_peered":"2024-03-06T07:20:10.744756+0000","last_clean":"2024-03-06T07:20:10.744756+0000","last_became_active":"2024-03-06T07:20:09.523876+0000","last_became_peered":"2024-03-06T07:20:09.523876+0000","last_unstale":"2024-03-06T07:20:10.744756+0000","last_undegraded":"2024-03-06T07:20:10.744756+0000","last_fullsized":"2024-03-06T07:20:10.744756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273623+0000","last_change":"2024-03-06T07:20:04.736879+0000","last_active":"2024-03-06T07:20:11.273623+0000","last_peered":"2024-03-06T07:20:11.273623+0000","last_clean":"2024-03-06T07:20:11.273623+0000","last_became_active":"2024-03-06T07:20:04.736271+0000","last_became_peered":"2024-03-06T07:20:04.736271+0000","last_unstale":"2024-03-06T07:20:11.273623+0000","last_undegraded":"2024-03-06T07:20:11.273623+0000","last_fullsized":"2024-03-06T07:20:11.273623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273465+0000","last_change":"2024-03-06T07:20:03.721336+0000","last_active":"2024-03-06T07:20:11.273465+0000","last_peered":"2024-03-06T07:20:11.273465+0000","last_clean":"2024-03-06T07:20:11.273465+0000","last_became_active":"2024-03-06T07:20:03.718827+0000","last_became_peered":"2024-03-06T07:20:03.718827+0000","last_unstale":"2024-03-06T07:20:11.273465+0000","last_undegraded":"2024-03-06T07:20:11.273465+0000","last_fullsized":"2024-03-06T07:20:11.273465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772927+0000","last_change":"2024-03-06T07:20:08.756386+0000","last_active":"2024-03-06T07:20:10.772927+0000","last_peered":"2024-03-06T07:20:10.772927+0000","last_clean":"2024-03-06T07:20:10.772927+0000","last_became_active":"2024-03-06T07:20:08.755970+0000","last_became_peered":"2024-03-06T07:20:08.755970+0000","last_unstale":"2024-03-06T07:20:10.772927+0000","last_undegraded":"2024-03-06T07:20:10.772927+0000","last_fullsized":"2024-03-06T07:20:10.772927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273864+0000","last_change":"2024-03-06T07:20:04.737597+0000","last_active":"2024-03-06T07:20:11.273864+0000","last_peered":"2024-03-06T07:20:11.273864+0000","last_clean":"2024-03-06T07:20:11.273864+0000","last_became_active":"2024-03-06T07:20:04.735046+0000","last_became_peered":"2024-03-06T07:20:04.735046+0000","last_unstale":"2024-03-06T07:20:11.273864+0000","last_undegraded":"2024-03-06T07:20:11.273864+0000","last_fullsized":"2024-03-06T07:20:11.273864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:16.868161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877519+0000","last_change":"2024-03-06T07:20:03.712917+0000","last_active":"2024-03-06T07:20:10.877519+0000","last_peered":"2024-03-06T07:20:10.877519+0000","last_clean":"2024-03-06T07:20:10.877519+0000","last_became_active":"2024-03-06T07:20:03.712584+0000","last_became_peered":"2024-03-06T07:20:03.712584+0000","last_unstale":"2024-03-06T07:20:10.877519+0000","last_undegraded":"2024-03-06T07:20:10.877519+0000","last_fullsized":"2024-03-06T07:20:10.877519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272841+0000","last_change":"2024-03-06T07:20:09.525362+0000","last_active":"2024-03-06T07:20:11.272841+0000","last_peered":"2024-03-06T07:20:11.272841+0000","last_clean":"2024-03-06T07:20:11.272841+0000","last_became_active":"2024-03-06T07:20:09.525026+0000","last_became_peered":"2024-03-06T07:20:09.525026+0000","last_unstale":"2024-03-06T07:20:11.272841+0000","last_undegraded":"2024-03-06T07:20:11.272841+0000","last_fullsized":"2024-03-06T07:20:11.272841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:20.377540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273503+0000","last_change":"2024-03-06T07:20:04.730553+0000","last_active":"2024-03-06T07:20:11.273503+0000","last_peered":"2024-03-06T07:20:11.273503+0000","last_clean":"2024-03-06T07:20:11.273503+0000","last_became_active":"2024-03-06T07:20:04.730222+0000","last_became_peered":"2024-03-06T07:20:04.730222+0000","last_unstale":"2024-03-06T07:20:11.273503+0000","last_undegraded":"2024-03-06T07:20:11.273503+0000","last_fullsized":"2024-03-06T07:20:11.273503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746578+0000","last_change":"2024-03-06T07:20:03.718268+0000","last_active":"2024-03-06T07:20:10.746578+0000","last_peered":"2024-03-06T07:20:10.746578+0000","last_clean":"2024-03-06T07:20:10.746578+0000","last_became_active":"2024-03-06T07:20:03.717969+0000","last_became_peered":"2024-03-06T07:20:03.717969+0000","last_unstale":"2024-03-06T07:20:10.746578+0000","last_undegraded":"2024-03-06T07:20:10.746578+0000","last_fullsized":"2024-03-06T07:20:10.746578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745335+0000","last_change":"2024-03-06T07:20:08.756182+0000","last_active":"2024-03-06T07:20:10.745335+0000","last_peered":"2024-03-06T07:20:10.745335+0000","last_clean":"2024-03-06T07:20:10.745335+0000","last_became_active":"2024-03-06T07:20:08.755784+0000","last_became_peered":"2024-03-06T07:20:08.755784+0000","last_unstale":"2024-03-06T07:20:10.745335+0000","last_undegraded":"2024-03-06T07:20:10.745335+0000","last_fullsized":"2024-03-06T07:20:10.745335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273029+0000","last_change":"2024-03-06T07:20:04.729827+0000","last_active":"2024-03-06T07:20:11.273029+0000","last_peered":"2024-03-06T07:20:11.273029+0000","last_clean":"2024-03-06T07:20:11.273029+0000","last_became_active":"2024-03-06T07:20:04.729566+0000","last_became_peered":"2024-03-06T07:20:04.729566+0000","last_unstale":"2024-03-06T07:20:11.273029+0000","last_undegraded":"2024-03-06T07:20:11.273029+0000","last_fullsized":"2024-03-06T07:20:11.273029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877244+0000","last_change":"2024-03-06T07:20:03.723471+0000","last_active":"2024-03-06T07:20:10.877244+0000","last_peered":"2024-03-06T07:20:10.877244+0000","last_clean":"2024-03-06T07:20:10.877244+0000","last_became_active":"2024-03-06T07:20:03.723192+0000","last_became_peered":"2024-03-06T07:20:03.723192+0000","last_unstale":"2024-03-06T07:20:10.877244+0000","last_undegraded":"2024-03-06T07:20:10.877244+0000","last_fullsized":"2024-03-06T07:20:10.877244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772877+0000","last_change":"2024-03-06T07:20:08.760930+0000","last_active":"2024-03-06T07:20:10.772877+0000","last_peered":"2024-03-06T07:20:10.772877+0000","last_clean":"2024-03-06T07:20:10.772877+0000","last_became_active":"2024-03-06T07:20:08.760027+0000","last_became_peered":"2024-03-06T07:20:08.760027+0000","last_unstale":"2024-03-06T07:20:10.772877+0000","last_undegraded":"2024-03-06T07:20:10.772877+0000","last_fullsized":"2024-03-06T07:20:10.772877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744694+0000","last_change":"2024-03-06T07:20:04.717033+0000","last_active":"2024-03-06T07:20:10.744694+0000","last_peered":"2024-03-06T07:20:10.744694+0000","last_clean":"2024-03-06T07:20:10.744694+0000","last_became_active":"2024-03-06T07:20:04.716742+0000","last_became_peered":"2024-03-06T07:20:04.716742+0000","last_unstale":"2024-03-06T07:20:10.744694+0000","last_undegraded":"2024-03-06T07:20:10.744694+0000","last_fullsized":"2024-03-06T07:20:10.744694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745234+0000","last_change":"2024-03-06T07:20:03.717805+0000","last_active":"2024-03-06T07:20:10.745234+0000","last_peered":"2024-03-06T07:20:10.745234+0000","last_clean":"2024-03-06T07:20:10.745234+0000","last_became_active":"2024-03-06T07:20:03.717048+0000","last_became_peered":"2024-03-06T07:20:03.717048+0000","last_unstale":"2024-03-06T07:20:10.745234+0000","last_undegraded":"2024-03-06T07:20:10.745234+0000","last_fullsized":"2024-03-06T07:20:10.745234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744965+0000","last_change":"2024-03-06T07:20:09.525949+0000","last_active":"2024-03-06T07:20:10.744965+0000","last_peered":"2024-03-06T07:20:10.744965+0000","last_clean":"2024-03-06T07:20:10.744965+0000","last_became_active":"2024-03-06T07:20:09.525427+0000","last_became_peered":"2024-03-06T07:20:09.525427+0000","last_unstale":"2024-03-06T07:20:10.744965+0000","last_undegraded":"2024-03-06T07:20:10.744965+0000","last_fullsized":"2024-03-06T07:20:10.744965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:44.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273942+0000","last_change":"2024-03-06T07:20:04.732333+0000","last_active":"2024-03-06T07:20:11.273942+0000","last_peered":"2024-03-06T07:20:11.273942+0000","last_clean":"2024-03-06T07:20:11.273942+0000","last_became_active":"2024-03-06T07:20:04.732069+0000","last_became_peered":"2024-03-06T07:20:04.732069+0000","last_unstale":"2024-03-06T07:20:11.273942+0000","last_undegraded":"2024-03-06T07:20:11.273942+0000","last_fullsized":"2024-03-06T07:20:11.273942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274155+0000","last_change":"2024-03-06T07:20:03.719609+0000","last_active":"2024-03-06T07:20:11.274155+0000","last_peered":"2024-03-06T07:20:11.274155+0000","last_clean":"2024-03-06T07:20:11.274155+0000","last_became_active":"2024-03-06T07:20:03.719325+0000","last_became_peered":"2024-03-06T07:20:03.719325+0000","last_unstale":"2024-03-06T07:20:11.274155+0000","last_undegraded":"2024-03-06T07:20:11.274155+0000","last_fullsized":"2024-03-06T07:20:11.274155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744816+0000","last_change":"2024-03-06T07:20:09.524644+0000","last_active":"2024-03-06T07:20:10.744816+0000","last_peered":"2024-03-06T07:20:10.744816+0000","last_clean":"2024-03-06T07:20:10.744816+0000","last_became_active":"2024-03-06T07:20:09.524220+0000","last_became_peered":"2024-03-06T07:20:09.524220+0000","last_unstale":"2024-03-06T07:20:10.744816+0000","last_undegraded":"2024-03-06T07:20:10.744816+0000","last_fullsized":"2024-03-06T07:20:10.744816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:30.841732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772642+0000","last_change":"2024-03-06T07:20:04.735008+0000","last_active":"2024-03-06T07:20:10.772642+0000","last_peered":"2024-03-06T07:20:10.772642+0000","last_clean":"2024-03-06T07:20:10.772642+0000","last_became_active":"2024-03-06T07:20:04.734709+0000","last_became_peered":"2024-03-06T07:20:04.734709+0000","last_unstale":"2024-03-06T07:20:10.772642+0000","last_undegraded":"2024-03-06T07:20:10.772642+0000","last_fullsized":"2024-03-06T07:20:10.772642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773006+0000","last_change":"2024-03-06T07:20:03.722267+0000","last_active":"2024-03-06T07:20:10.773006+0000","last_peered":"2024-03-06T07:20:10.773006+0000","last_clean":"2024-03-06T07:20:10.773006+0000","last_became_active":"2024-03-06T07:20:03.721952+0000","last_became_peered":"2024-03-06T07:20:03.721952+0000","last_unstale":"2024-03-06T07:20:10.773006+0000","last_undegraded":"2024-03-06T07:20:10.773006+0000","last_fullsized":"2024-03-06T07:20:10.773006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273402+0000","last_change":"2024-03-06T07:20:09.524376+0000","last_active":"2024-03-06T07:20:11.273402+0000","last_peered":"2024-03-06T07:20:11.273402+0000","last_clean":"2024-03-06T07:20:11.273402+0000","last_became_active":"2024-03-06T07:20:09.524010+0000","last_became_peered":"2024-03-06T07:20:09.524010+0000","last_unstale":"2024-03-06T07:20:11.273402+0000","last_undegraded":"2024-03-06T07:20:11.273402+0000","last_fullsized":"2024-03-06T07:20:11.273402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:50.092001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273678+0000","last_change":"2024-03-06T07:20:04.734379+0000","last_active":"2024-03-06T07:20:11.273678+0000","last_peered":"2024-03-06T07:20:11.273678+0000","last_clean":"2024-03-06T07:20:11.273678+0000","last_became_active":"2024-03-06T07:20:04.733727+0000","last_became_peered":"2024-03-06T07:20:04.733727+0000","last_unstale":"2024-03-06T07:20:11.273678+0000","last_undegraded":"2024-03-06T07:20:11.273678+0000","last_fullsized":"2024-03-06T07:20:11.273678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:13.635762+0000","last_change":"2024-03-06T07:20:03.718296+0000","last_active":"2024-03-06T07:20:13.635762+0000","last_peered":"2024-03-06T07:20:13.635762+0000","last_clean":"2024-03-06T07:20:13.635762+0000","last_became_active":"2024-03-06T07:20:03.718015+0000","last_became_peered":"2024-03-06T07:20:03.718015+0000","last_unstale":"2024-03-06T07:20:13.635762+0000","last_undegraded":"2024-03-06T07:20:13.635762+0000","last_fullsized":"2024-03-06T07:20:13.635762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273542+0000","last_change":"2024-03-06T07:20:08.764548+0000","last_active":"2024-03-06T07:20:11.273542+0000","last_peered":"2024-03-06T07:20:11.273542+0000","last_clean":"2024-03-06T07:20:11.273542+0000","last_became_active":"2024-03-06T07:20:08.764175+0000","last_became_peered":"2024-03-06T07:20:08.764175+0000","last_unstale":"2024-03-06T07:20:11.273542+0000","last_undegraded":"2024-03-06T07:20:11.273542+0000","last_fullsized":"2024-03-06T07:20:11.273542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746370+0000","last_change":"2024-03-06T07:20:04.733069+0000","last_active":"2024-03-06T07:20:10.746370+0000","last_peered":"2024-03-06T07:20:10.746370+0000","last_clean":"2024-03-06T07:20:10.746370+0000","last_became_active":"2024-03-06T07:20:04.732656+0000","last_became_peered":"2024-03-06T07:20:04.732656+0000","last_unstale":"2024-03-06T07:20:10.746370+0000","last_undegraded":"2024-03-06T07:20:10.746370+0000","last_fullsized":"2024-03-06T07:20:10.746370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273433+0000","last_change":"2024-03-06T07:20:03.728079+0000","last_active":"2024-03-06T07:20:11.273433+0000","last_peered":"2024-03-06T07:20:11.273433+0000","last_clean":"2024-03-06T07:20:11.273433+0000","last_became_active":"2024-03-06T07:20:03.727628+0000","last_became_peered":"2024-03-06T07:20:03.727628+0000","last_unstale":"2024-03-06T07:20:11.273433+0000","last_undegraded":"2024-03-06T07:20:11.273433+0000","last_fullsized":"2024-03-06T07:20:11.273433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746197+0000","last_change":"2024-03-06T07:20:08.753762+0000","last_active":"2024-03-06T07:20:10.746197+0000","last_peered":"2024-03-06T07:20:10.746197+0000","last_clean":"2024-03-06T07:20:10.746197+0000","last_became_active":"2024-03-06T07:20:08.753331+0000","last_became_peered":"2024-03-06T07:20:08.753331+0000","last_unstale":"2024-03-06T07:20:10.746197+0000","last_undegraded":"2024-03-06T07:20:10.746197+0000","last_fullsized":"2024-03-06T07:20:10.746197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772817+0000","last_change":"2024-03-06T07:20:04.731376+0000","last_active":"2024-03-06T07:20:10.772817+0000","last_peered":"2024-03-06T07:20:10.772817+0000","last_clean":"2024-03-06T07:20:10.772817+0000","last_became_active":"2024-03-06T07:20:04.730731+0000","last_became_peered":"2024-03-06T07:20:04.730731+0000","last_unstale":"2024-03-06T07:20:10.772817+0000","last_undegraded":"2024-03-06T07:20:10.772817+0000","last_fullsized":"2024-03-06T07:20:10.772817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273372+0000","last_change":"2024-03-06T07:20:03.727603+0000","last_active":"2024-03-06T07:20:11.273372+0000","last_peered":"2024-03-06T07:20:11.273372+0000","last_clean":"2024-03-06T07:20:11.273372+0000","last_became_active":"2024-03-06T07:20:03.727238+0000","last_became_peered":"2024-03-06T07:20:03.727238+0000","last_unstale":"2024-03-06T07:20:11.273372+0000","last_undegraded":"2024-03-06T07:20:11.273372+0000","last_fullsized":"2024-03-06T07:20:11.273372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744238+0000","last_change":"2024-03-06T07:20:08.759817+0000","last_active":"2024-03-06T07:20:10.744238+0000","last_peered":"2024-03-06T07:20:10.744238+0000","last_clean":"2024-03-06T07:20:10.744238+0000","last_became_active":"2024-03-06T07:20:08.759430+0000","last_became_peered":"2024-03-06T07:20:08.759430+0000","last_unstale":"2024-03-06T07:20:10.744238+0000","last_undegraded":"2024-03-06T07:20:10.744238+0000","last_fullsized":"2024-03-06T07:20:10.744238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:59.350405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273280+0000","last_change":"2024-03-06T07:20:04.736807+0000","last_active":"2024-03-06T07:20:11.273280+0000","last_peered":"2024-03-06T07:20:11.273280+0000","last_clean":"2024-03-06T07:20:11.273280+0000","last_became_active":"2024-03-06T07:20:04.736552+0000","last_became_peered":"2024-03-06T07:20:04.736552+0000","last_unstale":"2024-03-06T07:20:11.273280+0000","last_undegraded":"2024-03-06T07:20:11.273280+0000","last_fullsized":"2024-03-06T07:20:11.273280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745647+0000","last_change":"2024-03-06T07:20:03.725869+0000","last_active":"2024-03-06T07:20:10.745647+0000","last_peered":"2024-03-06T07:20:10.745647+0000","last_clean":"2024-03-06T07:20:10.745647+0000","last_became_active":"2024-03-06T07:20:03.725563+0000","last_became_peered":"2024-03-06T07:20:03.725563+0000","last_unstale":"2024-03-06T07:20:10.745647+0000","last_undegraded":"2024-03-06T07:20:10.745647+0000","last_fullsized":"2024-03-06T07:20:10.745647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744191+0000","last_change":"2024-03-06T07:20:09.527243+0000","last_active":"2024-03-06T07:20:10.744191+0000","last_peered":"2024-03-06T07:20:10.744191+0000","last_clean":"2024-03-06T07:20:10.744191+0000","last_became_active":"2024-03-06T07:20:09.526607+0000","last_became_peered":"2024-03-06T07:20:09.526607+0000","last_unstale":"2024-03-06T07:20:10.744191+0000","last_undegraded":"2024-03-06T07:20:10.744191+0000","last_fullsized":"2024-03-06T07:20:10.744191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:34.603979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772826+0000","last_change":"2024-03-06T07:20:09.524593+0000","last_active":"2024-03-06T07:20:10.772826+0000","last_peered":"2024-03-06T07:20:10.772826+0000","last_clean":"2024-03-06T07:20:10.772826+0000","last_became_active":"2024-03-06T07:20:09.524244+0000","last_became_peered":"2024-03-06T07:20:09.524244+0000","last_unstale":"2024-03-06T07:20:10.772826+0000","last_undegraded":"2024-03-06T07:20:10.772826+0000","last_fullsized":"2024-03-06T07:20:10.772826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272939+0000","last_change":"2024-03-06T07:20:04.718433+0000","last_active":"2024-03-06T07:20:11.272939+0000","last_peered":"2024-03-06T07:20:11.272939+0000","last_clean":"2024-03-06T07:20:11.272939+0000","last_became_active":"2024-03-06T07:20:04.718154+0000","last_became_peered":"2024-03-06T07:20:04.718154+0000","last_unstale":"2024-03-06T07:20:11.272939+0000","last_undegraded":"2024-03-06T07:20:11.272939+0000","last_fullsized":"2024-03-06T07:20:11.272939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273133+0000","last_change":"2024-03-06T07:20:03.727685+0000","last_active":"2024-03-06T07:20:11.273133+0000","last_peered":"2024-03-06T07:20:11.273133+0000","last_clean":"2024-03-06T07:20:11.273133+0000","last_became_active":"2024-03-06T07:20:03.727363+0000","last_became_peered":"2024-03-06T07:20:03.727363+0000","last_unstale":"2024-03-06T07:20:11.273133+0000","last_undegraded":"2024-03-06T07:20:11.273133+0000","last_fullsized":"2024-03-06T07:20:11.273133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745586+0000","last_change":"2024-03-06T07:20:08.760387+0000","last_active":"2024-03-06T07:20:10.745586+0000","last_peered":"2024-03-06T07:20:10.745586+0000","last_clean":"2024-03-06T07:20:10.745586+0000","last_became_active":"2024-03-06T07:20:08.760017+0000","last_became_peered":"2024-03-06T07:20:08.760017+0000","last_unstale":"2024-03-06T07:20:10.745586+0000","last_undegraded":"2024-03-06T07:20:10.745586+0000","last_fullsized":"2024-03-06T07:20:10.745586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744172+0000","last_change":"2024-03-06T07:20:04.721795+0000","last_active":"2024-03-06T07:20:10.744172+0000","last_peered":"2024-03-06T07:20:10.744172+0000","last_clean":"2024-03-06T07:20:10.744172+0000","last_became_active":"2024-03-06T07:20:04.720723+0000","last_became_peered":"2024-03-06T07:20:04.720723+0000","last_unstale":"2024-03-06T07:20:10.744172+0000","last_undegraded":"2024-03-06T07:20:10.744172+0000","last_fullsized":"2024-03-06T07:20:10.744172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744343+0000","last_change":"2024-03-06T07:20:03.722350+0000","last_active":"2024-03-06T07:20:10.744343+0000","last_peered":"2024-03-06T07:20:10.744343+0000","last_clean":"2024-03-06T07:20:10.744343+0000","last_became_active":"2024-03-06T07:20:03.722059+0000","last_became_peered":"2024-03-06T07:20:03.722059+0000","last_unstale":"2024-03-06T07:20:10.744343+0000","last_undegraded":"2024-03-06T07:20:10.744343+0000","last_fullsized":"2024-03-06T07:20:10.744343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744977+0000","last_change":"2024-03-06T07:20:03.717393+0000","last_active":"2024-03-06T07:20:10.744977+0000","last_peered":"2024-03-06T07:20:10.744977+0000","last_clean":"2024-03-06T07:20:10.744977+0000","last_became_active":"2024-03-06T07:20:03.717051+0000","last_became_peered":"2024-03-06T07:20:03.717051+0000","last_unstale":"2024-03-06T07:20:10.744977+0000","last_undegraded":"2024-03-06T07:20:10.744977+0000","last_fullsized":"2024-03-06T07:20:10.744977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772980+0000","last_change":"2024-03-06T07:20:04.718195+0000","last_active":"2024-03-06T07:20:10.772980+0000","last_peered":"2024-03-06T07:20:10.772980+0000","last_clean":"2024-03-06T07:20:10.772980+0000","last_became_active":"2024-03-06T07:20:04.717447+0000","last_became_peered":"2024-03-06T07:20:04.717447+0000","last_unstale":"2024-03-06T07:20:10.772980+0000","last_undegraded":"2024-03-06T07:20:10.772980+0000","last_fullsized":"2024-03-06T07:20:10.772980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877565+0000","last_change":"2024-03-06T07:20:08.759429+0000","last_active":"2024-03-06T07:20:10.877565+0000","last_peered":"2024-03-06T07:20:10.877565+0000","last_clean":"2024-03-06T07:20:10.877565+0000","last_became_active":"2024-03-06T07:20:08.759038+0000","last_became_peered":"2024-03-06T07:20:08.759038+0000","last_unstale":"2024-03-06T07:20:10.877565+0000","last_undegraded":"2024-03-06T07:20:10.877565+0000","last_fullsized":"2024-03-06T07:20:10.877565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:33.125882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744524+0000","last_change":"2024-03-06T07:20:03.730392+0000","last_active":"2024-03-06T07:20:10.744524+0000","last_peered":"2024-03-06T07:20:10.744524+0000","last_clean":"2024-03-06T07:20:10.744524+0000","last_became_active":"2024-03-06T07:20:03.730088+0000","last_became_peered":"2024-03-06T07:20:03.730088+0000","last_unstale":"2024-03-06T07:20:10.744524+0000","last_undegraded":"2024-03-06T07:20:10.744524+0000","last_fullsized":"2024-03-06T07:20:10.744524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745946+0000","last_change":"2024-03-06T07:20:04.724861+0000","last_active":"2024-03-06T07:20:10.745946+0000","last_peered":"2024-03-06T07:20:10.745946+0000","last_clean":"2024-03-06T07:20:10.745946+0000","last_became_active":"2024-03-06T07:20:04.724584+0000","last_became_peered":"2024-03-06T07:20:04.724584+0000","last_unstale":"2024-03-06T07:20:10.745946+0000","last_undegraded":"2024-03-06T07:20:10.745946+0000","last_fullsized":"2024-03-06T07:20:10.745946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:05.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744887+0000","last_change":"2024-03-06T07:20:08.758137+0000","last_active":"2024-03-06T07:20:10.744887+0000","last_peered":"2024-03-06T07:20:10.744887+0000","last_clean":"2024-03-06T07:20:10.744887+0000","last_became_active":"2024-03-06T07:20:08.757711+0000","last_became_peered":"2024-03-06T07:20:08.757711+0000","last_unstale":"2024-03-06T07:20:10.744887+0000","last_undegraded":"2024-03-06T07:20:10.744887+0000","last_fullsized":"2024-03-06T07:20:10.744887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:47.463176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744591+0000","last_change":"2024-03-06T07:20:04.724223+0000","last_active":"2024-03-06T07:20:10.744591+0000","last_peered":"2024-03-06T07:20:10.744591+0000","last_clean":"2024-03-06T07:20:10.744591+0000","last_became_active":"2024-03-06T07:20:04.723946+0000","last_became_peered":"2024-03-06T07:20:04.723946+0000","last_unstale":"2024-03-06T07:20:10.744591+0000","last_undegraded":"2024-03-06T07:20:10.744591+0000","last_fullsized":"2024-03-06T07:20:10.744591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877317+0000","last_change":"2024-03-06T07:20:04.730151+0000","last_active":"2024-03-06T07:20:10.877317+0000","last_peered":"2024-03-06T07:20:10.877317+0000","last_clean":"2024-03-06T07:20:10.877317+0000","last_became_active":"2024-03-06T07:20:04.729835+0000","last_became_peered":"2024-03-06T07:20:04.729835+0000","last_unstale":"2024-03-06T07:20:10.877317+0000","last_undegraded":"2024-03-06T07:20:10.877317+0000","last_fullsized":"2024-03-06T07:20:10.877317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274150+0000","last_change":"2024-03-06T07:20:04.734333+0000","last_active":"2024-03-06T07:20:11.274150+0000","last_peered":"2024-03-06T07:20:11.274150+0000","last_clean":"2024-03-06T07:20:11.274150+0000","last_became_active":"2024-03-06T07:20:04.734075+0000","last_became_peered":"2024-03-06T07:20:04.734075+0000","last_unstale":"2024-03-06T07:20:11.274150+0000","last_undegraded":"2024-03-06T07:20:11.274150+0000","last_fullsized":"2024-03-06T07:20:11.274150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772954+0000","last_change":"2024-03-06T07:20:04.722013+0000","last_active":"2024-03-06T07:20:10.772954+0000","last_peered":"2024-03-06T07:20:10.772954+0000","last_clean":"2024-03-06T07:20:10.772954+0000","last_became_active":"2024-03-06T07:20:04.721746+0000","last_became_peered":"2024-03-06T07:20:04.721746+0000","last_unstale":"2024-03-06T07:20:10.772954+0000","last_undegraded":"2024-03-06T07:20:10.772954+0000","last_fullsized":"2024-03-06T07:20:10.772954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744882+0000","last_change":"2024-03-06T07:20:04.732536+0000","last_active":"2024-03-06T07:20:10.744882+0000","last_peered":"2024-03-06T07:20:10.744882+0000","last_clean":"2024-03-06T07:20:10.744882+0000","last_became_active":"2024-03-06T07:20:04.728058+0000","last_became_peered":"2024-03-06T07:20:04.728058+0000","last_unstale":"2024-03-06T07:20:10.744882+0000","last_undegraded":"2024-03-06T07:20:10.744882+0000","last_fullsized":"2024-03-06T07:20:10.744882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273227+0000","last_change":"2024-03-06T07:20:04.723910+0000","last_active":"2024-03-06T07:20:11.273227+0000","last_peered":"2024-03-06T07:20:11.273227+0000","last_clean":"2024-03-06T07:20:11.273227+0000","last_became_active":"2024-03-06T07:20:04.722877+0000","last_became_peered":"2024-03-06T07:20:04.722877+0000","last_unstale":"2024-03-06T07:20:11.273227+0000","last_undegraded":"2024-03-06T07:20:11.273227+0000","last_fullsized":"2024-03-06T07:20:11.273227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745929+0000","last_change":"2024-03-06T07:20:04.734964+0000","last_active":"2024-03-06T07:20:10.745929+0000","last_peered":"2024-03-06T07:20:10.745929+0000","last_clean":"2024-03-06T07:20:10.745929+0000","last_became_active":"2024-03-06T07:20:04.734459+0000","last_became_peered":"2024-03-06T07:20:04.734459+0000","last_unstale":"2024-03-06T07:20:10.745929+0000","last_undegraded":"2024-03-06T07:20:10.745929+0000","last_fullsized":"2024-03-06T07:20:10.745929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744920+0000","last_change":"2024-03-06T07:20:04.725185+0000","last_active":"2024-03-06T07:20:10.744920+0000","last_peered":"2024-03-06T07:20:10.744920+0000","last_clean":"2024-03-06T07:20:10.744920+0000","last_became_active":"2024-03-06T07:20:04.724888+0000","last_became_peered":"2024-03-06T07:20:04.724888+0000","last_unstale":"2024-03-06T07:20:10.744920+0000","last_undegraded":"2024-03-06T07:20:10.744920+0000","last_fullsized":"2024-03-06T07:20:10.744920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273048+0000","last_change":"2024-03-06T07:20:04.735180+0000","last_active":"2024-03-06T07:20:11.273048+0000","last_peered":"2024-03-06T07:20:11.273048+0000","last_clean":"2024-03-06T07:20:11.273048+0000","last_became_active":"2024-03-06T07:20:04.733178+0000","last_became_peered":"2024-03-06T07:20:04.733178+0000","last_unstale":"2024-03-06T07:20:11.273048+0000","last_undegraded":"2024-03-06T07:20:11.273048+0000","last_fullsized":"2024-03-06T07:20:11.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744251+0000","last_change":"2024-03-06T07:20:04.726353+0000","last_active":"2024-03-06T07:20:10.744251+0000","last_peered":"2024-03-06T07:20:10.744251+0000","last_clean":"2024-03-06T07:20:10.744251+0000","last_became_active":"2024-03-06T07:20:04.725518+0000","last_became_peered":"2024-03-06T07:20:04.725518+0000","last_unstale":"2024-03-06T07:20:10.744251+0000","last_undegraded":"2024-03-06T07:20:10.744251+0000","last_fullsized":"2024-03-06T07:20:10.744251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274062+0000","last_change":"2024-03-06T07:20:04.736206+0000","last_active":"2024-03-06T07:20:11.274062+0000","last_peered":"2024-03-06T07:20:11.274062+0000","last_clean":"2024-03-06T07:20:11.274062+0000","last_became_active":"2024-03-06T07:20:04.735949+0000","last_became_peered":"2024-03-06T07:20:04.735949+0000","last_unstale":"2024-03-06T07:20:11.274062+0000","last_undegraded":"2024-03-06T07:20:11.274062+0000","last_fullsized":"2024-03-06T07:20:11.274062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772746+0000","last_change":"2024-03-06T07:20:04.728579+0000","last_active":"2024-03-06T07:20:10.772746+0000","last_peered":"2024-03-06T07:20:10.772746+0000","last_clean":"2024-03-06T07:20:10.772746+0000","last_became_active":"2024-03-06T07:20:04.728300+0000","last_became_peered":"2024-03-06T07:20:04.728300+0000","last_unstale":"2024-03-06T07:20:10.772746+0000","last_undegraded":"2024-03-06T07:20:10.772746+0000","last_fullsized":"2024-03-06T07:20:10.772746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745423+0000","last_change":"2024-03-06T07:20:04.724410+0000","last_active":"2024-03-06T07:20:10.745423+0000","last_peered":"2024-03-06T07:20:10.745423+0000","last_clean":"2024-03-06T07:20:10.745423+0000","last_became_active":"2024-03-06T07:20:04.723916+0000","last_became_peered":"2024-03-06T07:20:04.723916+0000","last_unstale":"2024-03-06T07:20:10.745423+0000","last_undegraded":"2024-03-06T07:20:10.745423+0000","last_fullsized":"2024-03-06T07:20:10.745423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274000+0000","last_change":"2024-03-06T07:20:04.734607+0000","last_active":"2024-03-06T07:20:11.274000+0000","last_peered":"2024-03-06T07:20:11.274000+0000","last_clean":"2024-03-06T07:20:11.274000+0000","last_became_active":"2024-03-06T07:20:04.734081+0000","last_became_peered":"2024-03-06T07:20:04.734081+0000","last_unstale":"2024-03-06T07:20:11.274000+0000","last_undegraded":"2024-03-06T07:20:11.274000+0000","last_fullsized":"2024-03-06T07:20:11.274000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877491+0000","last_change":"2024-03-06T07:20:04.726465+0000","last_active":"2024-03-06T07:20:10.877491+0000","last_peered":"2024-03-06T07:20:10.877491+0000","last_clean":"2024-03-06T07:20:10.877491+0000","last_became_active":"2024-03-06T07:20:04.726114+0000","last_became_peered":"2024-03-06T07:20:04.726114+0000","last_unstale":"2024-03-06T07:20:10.877491+0000","last_undegraded":"2024-03-06T07:20:10.877491+0000","last_fullsized":"2024-03-06T07:20:10.877491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.738589+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.737924+0000","last_became_peered":"2024-03-06T07:20:04.737924+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272964+0000","last_change":"2024-03-06T07:20:04.726607+0000","last_active":"2024-03-06T07:20:11.272964+0000","last_peered":"2024-03-06T07:20:11.272964+0000","last_clean":"2024-03-06T07:20:11.272964+0000","last_became_active":"2024-03-06T07:20:04.726337+0000","last_became_peered":"2024-03-06T07:20:04.726337+0000","last_unstale":"2024-03-06T07:20:11.272964+0000","last_undegraded":"2024-03-06T07:20:11.272964+0000","last_fullsized":"2024-03-06T07:20:11.272964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273000+0000","last_change":"2024-03-06T07:20:04.719555+0000","last_active":"2024-03-06T07:20:11.273000+0000","last_peered":"2024-03-06T07:20:11.273000+0000","last_clean":"2024-03-06T07:20:11.273000+0000","last_became_active":"2024-03-06T07:20:04.719273+0000","last_became_peered":"2024-03-06T07:20:04.719273+0000","last_unstale":"2024-03-06T07:20:11.273000+0000","last_undegraded":"2024-03-06T07:20:11.273000+0000","last_fullsized":"2024-03-06T07:20:11.273000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772701+0000","last_change":"2024-03-06T07:20:04.724038+0000","last_active":"2024-03-06T07:20:10.772701+0000","last_peered":"2024-03-06T07:20:10.772701+0000","last_clean":"2024-03-06T07:20:10.772701+0000","last_became_active":"2024-03-06T07:20:04.723527+0000","last_became_peered":"2024-03-06T07:20:04.723527+0000","last_unstale":"2024-03-06T07:20:10.772701+0000","last_undegraded":"2024-03-06T07:20:10.772701+0000","last_fullsized":"2024-03-06T07:20:10.772701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:32.763183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744248+0000","last_change":"2024-03-06T07:20:04.730004+0000","last_active":"2024-03-06T07:20:10.744248+0000","last_peered":"2024-03-06T07:20:10.744248+0000","last_clean":"2024-03-06T07:20:10.744248+0000","last_became_active":"2024-03-06T07:20:04.729702+0000","last_became_peered":"2024-03-06T07:20:04.729702+0000","last_unstale":"2024-03-06T07:20:10.744248+0000","last_undegraded":"2024-03-06T07:20:10.744248+0000","last_fullsized":"2024-03-06T07:20:10.744248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746363+0000","last_change":"2024-03-06T07:20:04.729091+0000","last_active":"2024-03-06T07:20:10.746363+0000","last_peered":"2024-03-06T07:20:10.746363+0000","last_clean":"2024-03-06T07:20:10.746363+0000","last_became_active":"2024-03-06T07:20:04.728793+0000","last_became_peered":"2024-03-06T07:20:04.728793+0000","last_unstale":"2024-03-06T07:20:10.746363+0000","last_undegraded":"2024-03-06T07:20:10.746363+0000","last_fullsized":"2024-03-06T07:20:10.746363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745590+0000","last_change":"2024-03-06T07:20:04.730775+0000","last_active":"2024-03-06T07:20:10.745590+0000","last_peered":"2024-03-06T07:20:10.745590+0000","last_clean":"2024-03-06T07:20:10.745590+0000","last_became_active":"2024-03-06T07:20:04.730386+0000","last_became_peered":"2024-03-06T07:20:04.730386+0000","last_unstale":"2024-03-06T07:20:10.745590+0000","last_undegraded":"2024-03-06T07:20:10.745590+0000","last_fullsized":"2024-03-06T07:20:10.745590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745842+0000","last_change":"2024-03-06T07:20:04.730885+0000","last_active":"2024-03-06T07:20:10.745842+0000","last_peered":"2024-03-06T07:20:10.745842+0000","last_clean":"2024-03-06T07:20:10.745842+0000","last_became_active":"2024-03-06T07:20:04.730294+0000","last_became_peered":"2024-03-06T07:20:04.730294+0000","last_unstale":"2024-03-06T07:20:10.745842+0000","last_undegraded":"2024-03-06T07:20:10.745842+0000","last_fullsized":"2024-03-06T07:20:10.745842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273816+0000","last_change":"2024-03-06T07:20:04.737430+0000","last_active":"2024-03-06T07:20:11.273816+0000","last_peered":"2024-03-06T07:20:11.273816+0000","last_clean":"2024-03-06T07:20:11.273816+0000","last_became_active":"2024-03-06T07:20:04.734870+0000","last_became_peered":"2024-03-06T07:20:04.734870+0000","last_unstale":"2024-03-06T07:20:11.273816+0000","last_undegraded":"2024-03-06T07:20:11.273816+0000","last_fullsized":"2024-03-06T07:20:11.273816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744080+0000","last_change":"2024-03-06T07:20:04.732622+0000","last_active":"2024-03-06T07:20:10.744080+0000","last_peered":"2024-03-06T07:20:10.744080+0000","last_clean":"2024-03-06T07:20:10.744080+0000","last_became_active":"2024-03-06T07:20:04.732149+0000","last_became_peered":"2024-03-06T07:20:04.732149+0000","last_unstale":"2024-03-06T07:20:10.744080+0000","last_undegraded":"2024-03-06T07:20:10.744080+0000","last_fullsized":"2024-03-06T07:20:10.744080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272952+0000","last_change":"2024-03-06T07:20:04.733067+0000","last_active":"2024-03-06T07:20:11.272952+0000","last_peered":"2024-03-06T07:20:11.272952+0000","last_clean":"2024-03-06T07:20:11.272952+0000","last_became_active":"2024-03-06T07:20:04.732805+0000","last_became_peered":"2024-03-06T07:20:04.732805+0000","last_unstale":"2024-03-06T07:20:11.272952+0000","last_undegraded":"2024-03-06T07:20:11.272952+0000","last_fullsized":"2024-03-06T07:20:11.272952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273984+0000","last_change":"2024-03-06T07:20:04.729682+0000","last_active":"2024-03-06T07:20:11.273984+0000","last_peered":"2024-03-06T07:20:11.273984+0000","last_clean":"2024-03-06T07:20:11.273984+0000","last_became_active":"2024-03-06T07:20:04.729407+0000","last_became_peered":"2024-03-06T07:20:04.729407+0000","last_unstale":"2024-03-06T07:20:11.273984+0000","last_undegraded":"2024-03-06T07:20:11.273984+0000","last_fullsized":"2024-03-06T07:20:11.273984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273787+0000","last_change":"2024-03-06T07:20:04.737815+0000","last_active":"2024-03-06T07:20:11.273787+0000","last_peered":"2024-03-06T07:20:11.273787+0000","last_clean":"2024-03-06T07:20:11.273787+0000","last_became_active":"2024-03-06T07:20:04.735276+0000","last_became_peered":"2024-03-06T07:20:04.735276+0000","last_unstale":"2024-03-06T07:20:11.273787+0000","last_undegraded":"2024-03-06T07:20:11.273787+0000","last_fullsized":"2024-03-06T07:20:11.273787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273449+0000","last_change":"2024-03-06T07:20:04.737108+0000","last_active":"2024-03-06T07:20:11.273449+0000","last_peered":"2024-03-06T07:20:11.273449+0000","last_clean":"2024-03-06T07:20:11.273449+0000","last_became_active":"2024-03-06T07:20:04.736608+0000","last_became_peered":"2024-03-06T07:20:04.736608+0000","last_unstale":"2024-03-06T07:20:11.273449+0000","last_undegraded":"2024-03-06T07:20:11.273449+0000","last_fullsized":"2024-03-06T07:20:11.273449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772691+0000","last_change":"2024-03-06T07:20:04.720240+0000","last_active":"2024-03-06T07:20:10.772691+0000","last_peered":"2024-03-06T07:20:10.772691+0000","last_clean":"2024-03-06T07:20:10.772691+0000","last_became_active":"2024-03-06T07:20:04.719952+0000","last_became_peered":"2024-03-06T07:20:04.719952+0000","last_unstale":"2024-03-06T07:20:10.772691+0000","last_undegraded":"2024-03-06T07:20:10.772691+0000","last_fullsized":"2024-03-06T07:20:10.772691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:13.957015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273519+0000","last_change":"2024-03-06T07:20:04.719505+0000","last_active":"2024-03-06T07:20:11.273519+0000","last_peered":"2024-03-06T07:20:11.273519+0000","last_clean":"2024-03-06T07:20:11.273519+0000","last_became_active":"2024-03-06T07:20:04.718961+0000","last_became_peered":"2024-03-06T07:20:04.718961+0000","last_unstale":"2024-03-06T07:20:11.273519+0000","last_undegraded":"2024-03-06T07:20:11.273519+0000","last_fullsized":"2024-03-06T07:20:11.273519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273711+0000","last_change":"2024-03-06T07:20:04.737365+0000","last_active":"2024-03-06T07:20:11.273711+0000","last_peered":"2024-03-06T07:20:11.273711+0000","last_clean":"2024-03-06T07:20:11.273711+0000","last_became_active":"2024-03-06T07:20:04.734552+0000","last_became_peered":"2024-03-06T07:20:04.734552+0000","last_unstale":"2024-03-06T07:20:11.273711+0000","last_undegraded":"2024-03-06T07:20:11.273711+0000","last_fullsized":"2024-03-06T07:20:11.273711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":698343,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":344064,"data_stored":698399,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:22.126 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:20:22.127 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-06T07:20:22.127 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-06T07:20:22.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-06T07:20:22.127 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-06T07:20:22.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.263+0000 7f1daa0c9640 1 Processor -- start 2024-03-06T07:20:22.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.263+0000 7f1daa0c9640 1 -- start start 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1daa0c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1daa0c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1da41586f0 0x7f1da4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1daa0c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da41591c0 con 0x7f1da40c5480 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1daa0c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da406df40 con 0x7f1da41586f0 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1daa0c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da406fdb0 con 0x7f1da4155ba0 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1da3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1da41586f0 0x7f1da4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1da2ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52204/0 (socket says 172.21.15.17:52204) 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1da2ffd640 1 -- 172.21.15.17:0/790265171 learned_addr learned my addr 172.21.15.17:0/790265171 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.264+0000 7f1da37fe640 1 --1- 172.21.15.17:0/790265171 >> v1:172.21.15.163:6789/0 conn(0x7f1da40c5480 0x7f1da406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57334/0 (socket says 172.21.15.17:57334) 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da3fff640 1 -- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 msgr2=0x7f1da40c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da3fff640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da3fff640 1 -- 172.21.15.17:0/790265171 >> v1:172.21.15.163:6789/0 conn(0x7f1da40c5480 legacy=0x7f1da406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da3fff640 1 -- 172.21.15.17:0/790265171 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da41594a0 con 0x7f1da41586f0 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da27fc640 1 -- 172.21.15.17:0/790265171 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253838532 0 0) 0x7f1da41591c0 con 0x7f1da40c5480 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da2ffd640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.265+0000 7f1da3fff640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da41586f0 0x7f1da4158b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009d90 tx=0x7f1d9c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c92077dfc126ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1da27fc640 1 -- 172.21.15.17:0/790265171 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c039070 con 0x7f1da41586f0 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1da27fc640 1 -- 172.21.15.17:0/790265171 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c02cbc0 con 0x7f1da41586f0 2024-03-06T07:20:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1da27fc640 1 -- 172.21.15.17:0/790265171 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c034440 con 0x7f1da41586f0 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 -- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da41586f0 msgr2=0x7f1da4158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da41586f0 0x7f1da4158b80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009d90 tx=0x7f1d9c02be10 comp rx=0 tx=0).stop 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 -- 172.21.15.17:0/790265171 shutdown_connections 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da41586f0 0x7f1da4158b80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/790265171 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da4155ba0 0x7f1da40c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 -- 172.21.15.17:0/790265171 >> 172.21.15.17:0/790265171 conn(0x7f1da406b1a0 msgr2=0x7f1da406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 -- 172.21.15.17:0/790265171 shutdown_connections 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.266+0000 7f1daa0c9640 1 -- 172.21.15.17:0/790265171 wait complete. 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.267+0000 7f1daa0c9640 1 Processor -- start 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.267+0000 7f1daa0c9640 1 -- start start 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da40c5480 0x7f1da4151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 0x7f1da4157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4155680 con 0x7f1da41586f0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4156000 con 0x7f1da4155ba0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1daa0c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da4152a20 con 0x7f1da40c5480 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38072/0 (socket says 172.21.15.17:38072) 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da3fff640 1 -- 172.21.15.17:0/1233715438 learned_addr learned my addr 172.21.15.17:0/1233715438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da2ffd640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 0x7f1da4157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da37fe640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da40c5480 0x7f1da4151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da2ffd640 1 -- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da40c5480 msgr2=0x7f1da4151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da2ffd640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da40c5480 0x7f1da4151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da2ffd640 1 -- 172.21.15.17:0/1233715438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 msgr2=0x7f1da4150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.268+0000 7f1da2ffd640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1da2ffd640 1 -- 172.21.15.17:0/1233715438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da41eebc0 con 0x7f1da4155ba0 2024-03-06T07:20:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1da3fff640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1da2ffd640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 0x7f1da4157570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800e900 tx=0x7f1d9800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800ca00 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9800cba0 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da41deb90 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d98004030 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.269+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da4157ab0 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.270+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da4069460 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.271+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d980041d0 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.271+0000 7f1d8bfff640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 0x7f1d9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.271+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1d9808ac90 con 0x7f1da4155ba0 2024-03-06T07:20:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.271+0000 7f1da37fe640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 0x7f1d9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.272+0000 7f1da37fe640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 0x7f1d9006d9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d94000f60 tx=0x7f1d94007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.275+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1da4069460 con 0x7f1da4155ba0 2024-03-06T07:20:22.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.505+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1da404f330 con 0x7f1da4155ba0 2024-03-06T07:20:22.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.508+0000 7f1d8bfff640 1 -- 172.21.15.17:0/1233715438 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171792 (secure 0 0 0) 0x7f1d98017020 con 0x7f1da4155ba0 2024-03-06T07:20:22.512 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 msgr2=0x7f1d9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 0x7f1d9006d9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d94000f60 tx=0x7f1d94007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 msgr2=0x7f1da4157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 0x7f1da4157570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800e900 tx=0x7f1d9800edc0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 shutdown_connections 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d9006b500 0x7f1d9006d9c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1da41586f0 0x7f1da4150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1da4155ba0 0x7f1da4157570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 --2- 172.21.15.17:0/1233715438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1da40c5480 0x7f1da4151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 >> 172.21.15.17:0/1233715438 conn(0x7f1da406b1a0 msgr2=0x7f1da40ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 shutdown_connections 2024-03-06T07:20:22.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.511+0000 7f1daa0c9640 1 -- 172.21.15.17:0/1233715438 wait complete. 2024-03-06T07:20:22.523 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1478765449},{"type":"v1","addr":"172.21.15.163:6833","nonce":1478765449}]},"active_addr":"172.21.15.163:6833/1478765449","active_change":"2024-03-06T07:19:51.331870+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":1330691185}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2244581720}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3479325411}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":1661329155}]}]} 2024-03-06T07:20:22.527 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-06T07:20:22.527 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-06T07:20:22.527 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:22.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.626+0000 7f648a8a4640 1 Processor -- start 2024-03-06T07:20:22.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.626+0000 7f648a8a4640 1 -- start start 2024-03-06T07:20:22.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f648a8a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64840c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f648a8a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64840c88b0 0x7f64841586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f648a8a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6484155110 con 0x7f64840c7dd0 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f648a8a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64841559e0 con 0x7f64840c88b0 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f648a8a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64840c8f40 con 0x7f64840c5330 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f64837fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64840c88b0 0x7f64841586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f6488e41640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f64840c7dd0 0x7f648406b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57344/0 (socket says 172.21.15.17:57344) 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f6483fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64840c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.627+0000 7f6488e41640 1 -- 172.21.15.17:0/1297948826 learned_addr learned my addr 172.21.15.17:0/1297948826 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.628+0000 7f64837fe640 1 -- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 msgr2=0x7f64840c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.628+0000 7f64837fe640 1 --2- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64840c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.628+0000 7f64837fe640 1 -- 172.21.15.17:0/1297948826 >> v1:172.21.15.163:6789/0 conn(0x7f64840c7dd0 legacy=0x7f648406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.628+0000 7f64837fe640 1 -- 172.21.15.17:0/1297948826 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484155390 con 0x7f64840c88b0 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.628+0000 7f64837fe640 1 --2- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c88b0 0x7f64841586a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64780097a0 tx=0x7f647802b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c33cec129c93434 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f6482ffd640 1 -- 172.21.15.17:0/1297948826 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478037070 con 0x7f64840c88b0 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f6482ffd640 1 -- 172.21.15.17:0/1297948826 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478030710 con 0x7f64840c88b0 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f6482ffd640 1 -- 172.21.15.17:0/1297948826 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647802be60 con 0x7f64840c88b0 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 -- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c88b0 msgr2=0x7f64841586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c88b0 0x7f64841586a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64780097a0 tx=0x7f647802b660 comp rx=0 tx=0).stop 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 -- 172.21.15.17:0/1297948826 shutdown_connections 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c88b0 0x7f64841586a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1297948826 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64840c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 -- 172.21.15.17:0/1297948826 >> 172.21.15.17:0/1297948826 conn(0x7f64840ac560 msgr2=0x7f64840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 -- 172.21.15.17:0/1297948826 shutdown_connections 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.629+0000 7f648a8a4640 1 -- 172.21.15.17:0/1297948826 wait complete. 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.630+0000 7f648a8a4640 1 Processor -- start 2024-03-06T07:20:22.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.630+0000 7f648a8a4640 1 -- start start 2024-03-06T07:20:22.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64841eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c7dd0 0x7f64841edc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64841529d0 con 0x7f64840c88b0 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648406d090 con 0x7f64840c7dd0 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f648a8a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6484152b50 con 0x7f64840c5330 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38076/0 (socket says 172.21.15.17:38076) 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 -- 172.21.15.17:0/1230847152 learned_addr learned my addr 172.21.15.17:0/1230847152 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f64837fe640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c7dd0 0x7f64841edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6483fff640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64841eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 -- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 msgr2=0x7f64841eae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64841eae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 -- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c7dd0 msgr2=0x7f64841edc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c7dd0 0x7f64841edc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.631+0000 7f6488e41640 1 -- 172.21.15.17:0/1230847152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648414ea20 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f6488e41640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f647c00cc10 tx=0x7f647c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c018070 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c004d80 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648414cec0 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00ad20 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.632+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648414d480 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.633+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648404f330 con 0x7f64840c88b0 2024-03-06T07:20:22.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.633+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f647c0070d0 con 0x7f64840c88b0 2024-03-06T07:20:22.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.633+0000 7f6480ff9640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 0x7f647006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.634+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f647c089ed0 con 0x7f64840c88b0 2024-03-06T07:20:22.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.634+0000 7f6483fff640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 0x7f647006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.634+0000 7f6483fff640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 0x7f647006d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b40 tx=0x7f6474009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.638+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6484155110 con 0x7f64840c88b0 2024-03-06T07:20:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.820+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f648406cdc0 con 0x7f64840c88b0 2024-03-06T07:20:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.821+0000 7f6480ff9640 1 -- 172.21.15.17:0/1230847152 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f647c0583d0 con 0x7f64840c88b0 2024-03-06T07:20:22.823 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:22.823 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 msgr2=0x7f647006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 0x7f647006d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6474009b40 tx=0x7f6474009190 comp rx=0 tx=0).stop 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 msgr2=0x7f648414b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f647c00cc10 tx=0x7f647c00a430 comp rx=0 tx=0).stop 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 shutdown_connections 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647006b4d0 0x7f647006d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64840c88b0 0x7f648414b2d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64840c7dd0 0x7f64841edc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 --2- 172.21.15.17:0/1230847152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64840c5330 0x7f64841eae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 >> 172.21.15.17:0/1230847152 conn(0x7f64840ac560 msgr2=0x7f6484152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 shutdown_connections 2024-03-06T07:20:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.823+0000 7f648a8a4640 1 -- 172.21.15.17:0/1230847152 wait complete. 2024-03-06T07:20:22.836 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-06T07:20:22.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.935+0000 7f28f41d7640 1 Processor -- start 2024-03-06T07:20:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.935+0000 7f28f41d7640 1 -- start start 2024-03-06T07:20:22.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f41d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28ec1506b0 0x7f28ec153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f41d7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28ec156ea0 0x7f28ec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f41d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28ec070800 con 0x7f28ec0acc00 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f41d7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec156450 con 0x7f28ec156ea0 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f41d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec156a10 con 0x7f28ec1506b0 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f1772640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28ec156ea0 0x7f28ec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f1772640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28ec156ea0 0x7f28ec0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38092/0 (socket says 172.21.15.17:38092) 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f1772640 1 -- 172.21.15.17:0/1911398346 learned_addr learned my addr 172.21.15.17:0/1911398346 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.936+0000 7f28f1f73640 1 --2- 172.21.15.17:0/1911398346 >> v2:172.21.15.163:3300/0 conn(0x7f28ec1506b0 0x7f28ec153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f1772640 1 -- 172.21.15.17:0/1911398346 >> v2:172.21.15.163:3300/0 conn(0x7f28ec1506b0 msgr2=0x7f28ec153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f1772640 1 --2- 172.21.15.17:0/1911398346 >> v2:172.21.15.163:3300/0 conn(0x7f28ec1506b0 0x7f28ec153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f1772640 1 -- 172.21.15.17:0/1911398346 >> v1:172.21.15.163:6789/0 conn(0x7f28ec0acc00 legacy=0x7f28ec0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f1772640 1 -- 172.21.15.17:0/1911398346 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec070aa0 con 0x7f28ec156ea0 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f1772640 1 --2- 172.21.15.17:0/1911398346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec156ea0 0x7f28ec0702c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1565d0 tx=0x7f28dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18316b0a48447ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.937+0000 7f28f0f71640 1 -- 172.21.15.17:0/1911398346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc034070 con 0x7f28ec156ea0 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f0f71640 1 -- 172.21.15.17:0/1911398346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc030860 con 0x7f28ec156ea0 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f0f71640 1 -- 172.21.15.17:0/1911398346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc030c30 con 0x7f28ec156ea0 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 -- 172.21.15.17:0/1911398346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec156ea0 msgr2=0x7f28ec0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1911398346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec156ea0 0x7f28ec0702c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28ec1565d0 tx=0x7f28dc02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 -- 172.21.15.17:0/1911398346 shutdown_connections 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1911398346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec156ea0 0x7f28ec0702c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1911398346 >> v2:172.21.15.163:3300/0 conn(0x7f28ec1506b0 0x7f28ec153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 -- 172.21.15.17:0/1911398346 >> 172.21.15.17:0/1911398346 conn(0x7f28ec06b1a0 msgr2=0x7f28ec06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 -- 172.21.15.17:0/1911398346 shutdown_connections 2024-03-06T07:20:22.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.938+0000 7f28f41d7640 1 -- 172.21.15.17:0/1911398346 wait complete. 2024-03-06T07:20:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.939+0000 7f28f41d7640 1 Processor -- start 2024-03-06T07:20:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.939+0000 7f28f41d7640 1 -- start start 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28ec0acc00 0x7f28ec1dff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28ec156ea0 0x7f28ec1ec550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0c5bb0 con 0x7f28ec1506b0 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec1566a0 con 0x7f28ec0acc00 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f41d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0c5d30 con 0x7f28ec156ea0 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38094/0 (socket says 172.21.15.17:38094) 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 -- 172.21.15.17:0/1729943326 learned_addr learned my addr 172.21.15.17:0/1729943326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1f73640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28ec0acc00 0x7f28ec1dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f2774640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28ec156ea0 0x7f28ec1ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 -- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28ec156ea0 msgr2=0x7f28ec1ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28ec156ea0 0x7f28ec1ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 -- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28ec0acc00 msgr2=0x7f28ec1dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28ec0acc00 0x7f28ec1dff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.940+0000 7f28f1772640 1 -- 172.21.15.17:0/1729943326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0ab0d0 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28f1772640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28dc000c00 tx=0x7f28dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc034040 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc039d90 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec0ab3b0 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc043610 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.941+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec0a8db0 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.942+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec04f330 con 0x7f28ec1506b0 2024-03-06T07:20:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.942+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28dc037070 con 0x7f28ec1506b0 2024-03-06T07:20:22.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.943+0000 7f28e27fc640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 0x7f28d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:22.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.943+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f28dc0ac4f0 con 0x7f28ec1506b0 2024-03-06T07:20:22.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.943+0000 7f28f1f73640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 0x7f28d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:22.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.943+0000 7f28f1f73640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 0x7f28d006d9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009b40 tx=0x7f28e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:22.947+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28dc03e070 con 0x7f28ec1506b0 2024-03-06T07:20:23.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.131+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ec156230 con 0x7f28ec1506b0 2024-03-06T07:20:23.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.132+0000 7f28e27fc640 1 -- 172.21.15.17:0/1729943326 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f28dc07a9e0 con 0x7f28ec1506b0 2024-03-06T07:20:23.134 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:23.135 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 msgr2=0x7f28d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 0x7f28d006d9c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009b40 tx=0x7f28e8009190 comp rx=0 tx=0).stop 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 msgr2=0x7f28ec1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f28dc000c00 tx=0x7f28dc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 shutdown_connections 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d006b500 0x7f28d006d9c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28ec156ea0 0x7f28ec1ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28ec1506b0 0x7f28ec1535c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 --2- 172.21.15.17:0/1729943326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28ec0acc00 0x7f28ec1dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 >> 172.21.15.17:0/1729943326 conn(0x7f28ec06b1a0 msgr2=0x7f28ec0c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 shutdown_connections 2024-03-06T07:20:23.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.134+0000 7f28f41d7640 1 -- 172.21.15.17:0/1729943326 wait complete. 2024-03-06T07:20:23.147 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-06T07:20:23.147 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-06T07:20:23.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-06T07:20:23.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-06T07:20:23.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-06T07:20:23.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-06T07:20:23.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-06T07:20:23.150 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.318+0000 7f24ef59e640 1 Processor -- start 2024-03-06T07:20:23.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.318+0000 7f24ef59e640 1 -- start start 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.319+0000 7f24ef59e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24f0147d70 0x7f24f01553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.319+0000 7f24ef59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 0x7f24f00ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24f0147d70 0x7f24f01553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24f0147d70 0x7f24f01553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59894/0 (socket says 172.21.15.17:59894) 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 -- 172.21.15.17:0/3000787796 learned_addr learned my addr 172.21.15.17:0/3000787796 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f00ab580 con 0x7f24f0151ce0 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f0072b80 con 0x7f24f00aa680 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f0073100 con 0x7f24f0147d70 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24edd9b640 1 --2- 172.21.15.17:0/3000787796 >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 0x7f24f00ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 -- 172.21.15.17:0/3000787796 >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 msgr2=0x7f24f00ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 --2- 172.21.15.17:0/3000787796 >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 0x7f24f00ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 -- 172.21.15.17:0/3000787796 >> v1:172.21.15.163:6789/0 conn(0x7f24f0151ce0 legacy=0x7f24f0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.320+0000 7f24ee59c640 1 -- 172.21.15.17:0/3000787796 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f0072e20 con 0x7f24f0147d70 2024-03-06T07:20:23.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.321+0000 7f24edd9b640 1 --2- 172.21.15.17:0/3000787796 >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 0x7f24f00ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:23.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7fbbf8b72640 1 Processor -- start 2024-03-06T07:20:23.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.321+0000 7f24ee59c640 1 --2- 172.21.15.17:0/3000787796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0147d70 0x7f24f01553d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24f00675f0 tx=0x7f24e002bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcffb32bfa9d245e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.321+0000 7f24ed59a640 1 -- 172.21.15.17:0/3000787796 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000b040 con 0x7f24f0147d70 2024-03-06T07:20:23.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.321+0000 7f24ed59a640 1 -- 172.21.15.17:0/3000787796 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0004230 con 0x7f24f0147d70 2024-03-06T07:20:23.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.321+0000 7f24ed59a640 1 -- 172.21.15.17:0/3000787796 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e002c690 con 0x7f24f0147d70 2024-03-06T07:20:23.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7f4c9759e640 1 Processor -- start 2024-03-06T07:20:23.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0147d70 msgr2=0x7f24f01553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7f24ef59e640 1 --2- 172.21.15.17:0/3000787796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0147d70 0x7f24f01553d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24f00675f0 tx=0x7f24e002bcf0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7f4c9759e640 1 -- start start 2024-03-06T07:20:23.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7fb958464640 1 Processor -- start 2024-03-06T07:20:23.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.322+0000 7fb958464640 1 -- start start 2024-03-06T07:20:23.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f62224eb640 1 Processor -- start 2024-03-06T07:20:23.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f62224eb640 1 -- start start 2024-03-06T07:20:23.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f4accd65640 1 Processor -- start 2024-03-06T07:20:23.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 shutdown_connections 2024-03-06T07:20:23.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 --2- 172.21.15.17:0/3000787796 >> v2:172.21.15.17:3300/0 conn(0x7f24f00aa680 0x7f24f00ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 --2- 172.21.15.17:0/3000787796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0147d70 0x7f24f01553d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 >> 172.21.15.17:0/3000787796 conn(0x7f24f00c5df0 msgr2=0x7f24f00c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 shutdown_connections 2024-03-06T07:20:23.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f24ef59e640 1 -- 172.21.15.17:0/3000787796 wait complete. 2024-03-06T07:20:23.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7fb958464640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb9501529a0 0x7fb95014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f4c9759e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c9000b0d0 0x7f4c9004ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f62224eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 0x7f621c155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f4c9759e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 0x7f4c900daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f62224eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0c6d70 0x7f621c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f62224eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621c0c71d0 con 0x7f621c1593c0 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f62224eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f621c158710 con 0x7f621c0bfe90 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f62224eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0c59d0 con 0x7f621c0c6d70 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f621bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 0x7f621c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f621bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 0x7f621c155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38128/0 (socket says 172.21.15.17:38128) 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f621bfff640 1 -- 172.21.15.17:0/1952572943 learned_addr learned my addr 172.21.15.17:0/1952572943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f4c9659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c9000b0d0 0x7f4c9004ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7f4accd65640 1 -- start start 2024-03-06T07:20:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f621b7fe640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0c6d70 0x7f621c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7fb956a01640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb9500c5eb0 0x7fb9500a5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57386/0 (socket says 172.21.15.17:57386) 2024-03-06T07:20:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7fb956a01640 1 -- 172.21.15.17:0/160576352 learned_addr learned my addr 172.21.15.17:0/160576352 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7fb956200640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb95014e9f0 0x7fb950151f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58290/0 (socket says 172.21.15.17:58290) 2024-03-06T07:20:23.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f4c9659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4c9000b0d0 0x7f4c9004ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38120/0 (socket says 172.21.15.17:38120) 2024-03-06T07:20:23.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f4c9659c640 1 -- 172.21.15.17:0/2085944407 learned_addr learned my addr 172.21.15.17:0/2085944407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.324+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436880073 0 0) 0x7f621c0c71d0 con 0x7f621c1593c0 2024-03-06T07:20:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900468d0 con 0x7f4c900d3c40 2024-03-06T07:20:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c90047660 con 0x7f4c9000b0d0 2024-03-06T07:20:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900d65d0 con 0x7f4c9004b1b0 2024-03-06T07:20:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9500c6570 con 0x7fb95014e9f0 2024-03-06T07:20:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6204003520 con 0x7f621c1593c0 2024-03-06T07:20:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c96d9d640 1 --2- 172.21.15.17:0/2085944407 >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 0x7f4c900daca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9500c6850 con 0x7fb9500c5eb0 2024-03-06T07:20:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb950154880 con 0x7fb9501529a0 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.323+0000 7fbbf8b72640 1 -- start start 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf8b72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 0x7fbbf40c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf8b72640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf406fdd0 con 0x7fbbf40724d0 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf8b72640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf40703b0 con 0x7fbbf4072a40 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf8b72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40c7190 con 0x7fbbf4159c00 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf259c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbbf4072a40 0x7fbbf4072e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57402/0 (socket says 172.21.15.17:57402) 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf259c640 1 -- 172.21.15.17:0/1522105282 learned_addr learned my addr 172.21.15.17:0/1522105282 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f9519847640 1 Processor -- start 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573758149 0 0) 0x7f6204003520 con 0x7f621c1593c0 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0c6d70 msgr2=0x7f621c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0c6d70 0x7f621c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 msgr2=0x7f621c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 --2- 172.21.15.17:0/1952572943 >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 0x7f621c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213936709 0 0) 0x7fb9500c6850 con 0x7fb9500c5eb0 2024-03-06T07:20:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb938003520 con 0x7fb9500c5eb0 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c0c5cb0 con 0x7f621c1593c0 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f62140019f0 con 0x7f621c1593c0 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 >> v1:172.21.15.163:6789/0 conn(0x7f621c1593c0 legacy=0x7f621c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467470245 0 0) 0x7fb9500c6570 con 0x7fb95014e9f0 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9559ff640 1 --2- 172.21.15.17:0/160576352 >> v2:172.21.15.17:3300/0 conn(0x7fb9501529a0 0x7fb95014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb938004a60 con 0x7fb95014e9f0 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040039b0 0x7f6204003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6204004800 0x7f6204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62040051c0 0x7f620400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c158710 con 0x7f62040039b0 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0c59d0 con 0x7f6204004800 2024-03-06T07:20:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620400c060 con 0x7f62040051c0 2024-03-06T07:20:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9659c640 1 -- 172.21.15.17:0/2085944407 >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 msgr2=0x7f4c900daca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9659c640 1 --2- 172.21.15.17:0/2085944407 >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 0x7f4c900daca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9659c640 1 -- 172.21.15.17:0/2085944407 >> v1:172.21.15.17:6789/0 conn(0x7f4c900d3c40 legacy=0x7f4c900d6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f4c9659c640 1 -- 172.21.15.17:0/2085944407 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90046bb0 con 0x7f4c9000b0d0 2024-03-06T07:20:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133466208 0 0) 0x7fb938003520 con 0x7fb9500c5eb0 2024-03-06T07:20:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 >> v2:172.21.15.17:3300/0 conn(0x7fb9501529a0 msgr2=0x7fb95014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4c96d9d640 1 --2- 172.21.15.17:0/2085944407 >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 0x7f4c900daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:23.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62040051c0 0x7f620400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f24ef59e640 1 Processor -- start 2024-03-06T07:20:23.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f24ef59e640 1 -- start start 2024-03-06T07:20:23.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f621bfff640 1 --2- 172.21.15.17:0/1952572943 >> v2:172.21.15.17:3300/0 conn(0x7f621c0bfe90 0x7f621c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:23.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 --2- 172.21.15.17:0/160576352 >> v2:172.21.15.17:3300/0 conn(0x7fb9501529a0 0x7fb95014edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7f9519847640 1 -- start start 2024-03-06T07:20:23.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 >> v1:172.21.15.17:6789/0 conn(0x7fb95014e9f0 legacy=0x7fb950151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95006d6f0 con 0x7fb9500c5eb0 2024-03-06T07:20:23.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb940001650 con 0x7fb9500c5eb0 2024-03-06T07:20:23.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f9519847640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95140c2620 0x7f951406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 >> v1:172.21.15.163:6789/0 conn(0x7fb9500c5eb0 legacy=0x7fb9500a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4accd65640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 0x7f4ac80c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4accd65640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ac80c6d80 0x7f4ac806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4accd65640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac815d080 con 0x7f4ac815cb80 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4accd65640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80a9960 con 0x7f4ac80c6d80 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f4accd65640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac815d360 con 0x7f4ac8156bf0 2024-03-06T07:20:23.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 0x7fb938003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb938006da0 0x7fb938009230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb938009770 0x7fb938009c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f9519847640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951415d080 con 0x7f9514156bf0 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950154880 con 0x7fb938009770 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93800a290 con 0x7fb938006da0 2024-03-06T07:20:23.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93800a490 con 0x7fb9380039b0 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.325+0000 7fbbf2d9d640 1 --2- 172.21.15.17:0/1522105282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 0x7fbbf40c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fbbf159a640 1 -- 172.21.15.17:0/1522105282 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079696960 0 0) 0x7fbbf40703b0 con 0x7fbbf4072a40 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fbbf159a640 1 -- 172.21.15.17:0/1522105282 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbdc003520 con 0x7fbbf4072a40 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fbbf2d9d640 1 -- 172.21.15.17:0/1522105282 >> v1:172.21.15.163:6789/0 conn(0x7fbbf4072a40 legacy=0x7fbbf4072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fbbf2d9d640 1 -- 172.21.15.17:0/1522105282 >> v1:172.21.15.17:6789/0 conn(0x7fbbf40724d0 legacy=0x7fbbf41594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7fbbf2d9d640 1 -- 172.21.15.17:0/1522105282 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40c73e0 con 0x7fbbf4159c00 2024-03-06T07:20:23.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f9519847640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951415d360 con 0x7f951415cb80 2024-03-06T07:20:23.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f9519847640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95140a9960 con 0x7f95140c2620 2024-03-06T07:20:23.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb9559ff640 1 --2- 172.21.15.17:0/160576352 >> v2:172.21.15.17:3300/0 conn(0x7fb9501529a0 0x7fb95014edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:23.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f4ac5d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4ac815cb80 0x7f4ac806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57420/0 (socket says 172.21.15.17:57420) 2024-03-06T07:20:23.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 -- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6204004800 msgr2=0x7f6204004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:20:23.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6204004800 0x7f6204004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 -- 172.21.15.17:0/1952572943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040039b0 msgr2=0x7f6204003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:20:23.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f4ac5d9b640 1 -- 172.21.15.17:0/381593258 learned_addr learned my addr 172.21.15.17:0/381593258 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:23.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040039b0 0x7f6204003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.326+0000 7f6220a88640 1 -- 172.21.15.17:0/1952572943 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204003520 con 0x7f62040051c0 2024-03-06T07:20:23.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f621b7fe640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0c6d70 0x7f621c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:23.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 0x7fb938003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f9512ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9514156bf0 0x7f95140c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58330/0 (socket says 172.21.15.17:58330) 2024-03-06T07:20:23.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 0x7f24f014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f9512ffd640 1 -- 172.21.15.17:0/763292280 learned_addr learned my addr 172.21.15.17:0/763292280 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:23.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0147d70 0x7f24f006e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb9559ff640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb938006da0 0x7fb938009230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f95137fe640 1 --2- 172.21.15.17:0/763292280 >> v2:172.21.15.163:3300/0 conn(0x7f95140c2620 0x7f951406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f6220a88640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62040051c0 0x7f620400b9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f621400c410 tx=0x7f6204005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fba4d42c18537 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f4ac6d9d640 1 --2- 172.21.15.17:0/381593258 >> v2:172.21.15.17:3300/0 conn(0x7f4ac80c6d80 0x7f4ac806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f4c9659c640 1 --2- 172.21.15.17:0/2085944407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9000b0d0 0x7f4c9004ac70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c98050b10 tx=0x7f4c98069930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861ffa2822431caf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f4ac659c640 1 --2- 172.21.15.17:0/381593258 >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 0x7f4ac80c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0151ce0 0x7f24f006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0072110 con 0x7f24f0147d70 2024-03-06T07:20:23.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f01517c0 con 0x7f24f0151ce0 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ef59e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0072290 con 0x7f24f00aa680 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ee59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 0x7f24f014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ee59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 0x7f24f014e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52268/0 (socket says 172.21.15.17:52268) 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7f24ee59c640 1 -- 172.21.15.17:0/1888605393 learned_addr learned my addr 172.21.15.17:0/1888605393 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956a01640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb938009770 0x7fb938009c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 -- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb938006da0 msgr2=0x7fb938009230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb938006da0 0x7fb938009230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 -- 172.21.15.17:0/160576352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb938009770 msgr2=0x7fb938009c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb938009770 0x7fb938009c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.327+0000 7fb956200640 1 -- 172.21.15.17:0/160576352 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95006d6f0 con 0x7fb9380039b0 2024-03-06T07:20:23.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fb956a01640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb938009770 0x7fb938009c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fb956a01640 1 -- 172.21.15.17:0/160576352 reap_dead start 2024-03-06T07:20:23.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400c0e0 con 0x7f62040051c0 2024-03-06T07:20:23.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24eed9d640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0151ce0 0x7f24f006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6214006d70 con 0x7f62040051c0 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fbbf2d9d640 1 --2- 172.21.15.17:0/1522105282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 0x7fbbf40c6a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009dc0 tx=0x7fbbec02be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cab8538c6c0c6c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f621affd640 1 -- 172.21.15.17:0/1952572943 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621402cd30 con 0x7f62040051c0 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 -- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0151ce0 msgr2=0x7f24f006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0151ce0 0x7f24f006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 -- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0147d70 msgr2=0x7f24f006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f0147d70 0x7f24f006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f014e970 con 0x7f24f00aa680 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24eed9d640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f0151ce0 0x7f24f006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117209732 0 0) 0x7f951415d360 con 0x7f951415cb80 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94fc003520 con 0x7f951415cb80 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052600948 0 0) 0x7f94fc003520 con 0x7f951415cb80 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 >> v2:172.21.15.163:3300/0 conn(0x7f95140c2620 msgr2=0x7f951406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 --2- 172.21.15.17:0/763292280 >> v2:172.21.15.163:3300/0 conn(0x7f95140c2620 0x7f951406df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 >> v1:172.21.15.17:6789/0 conn(0x7f9514156bf0 legacy=0x7f95140c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951415d640 con 0x7f951415cb80 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9508001650 con 0x7f951415cb80 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 >> v1:172.21.15.163:6789/0 conn(0x7f951415cb80 legacy=0x7f951406d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94fc003a10 0x7f94fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94fc0052e0 0x7f94fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00c180 con 0x7f94fc003a10 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00c330 con 0x7f94fc004800 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc00c530 con 0x7f94fc0052e0 2024-03-06T07:20:23.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac559a640 1 -- 172.21.15.17:0/381593258 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799887618 0 0) 0x7f4ac815d080 con 0x7f4ac815cb80 2024-03-06T07:20:23.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f24ee59c640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 0x7f24f014e430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036040 tx=0x7f24e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f95127fc640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f95137fe640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94fc0052e0 0x7f94fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fbbf159a640 1 -- 172.21.15.17:0/1522105282 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec039070 con 0x7fbbf4159c00 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fbbf159a640 1 -- 172.21.15.17:0/1522105282 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec02cbb0 con 0x7fbbf4159c00 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7fbbf159a640 1 -- 172.21.15.17:0/1522105282 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec034440 con 0x7fbbf4159c00 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac559a640 1 -- 172.21.15.17:0/381593258 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab0003520 con 0x7f4ac815cb80 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac6d9d640 1 -- 172.21.15.17:0/381593258 >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 msgr2=0x7f4ac80c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac6d9d640 1 --2- 172.21.15.17:0/381593258 >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 0x7f4ac80c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac6d9d640 1 -- 172.21.15.17:0/381593258 >> v1:172.21.15.163:6789/0 conn(0x7f4ac815cb80 legacy=0x7f4ac806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4ac6d9d640 1 -- 172.21.15.17:0/381593258 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac815d640 con 0x7f4ac80c6d80 2024-03-06T07:20:23.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0044030 con 0x7f24f00aa680 2024-03-06T07:20:23.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f578f57a640 1 Processor -- start 2024-03-06T07:20:23.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.328+0000 7f4c9559a640 1 -- 172.21.15.17:0/2085944407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980b4070 con 0x7f4c9000b0d0 2024-03-06T07:20:23.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f4c9559a640 1 -- 172.21.15.17:0/2085944407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c980a7b90 con 0x7f4c9000b0d0 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f4c9559a640 1 -- 172.21.15.17:0/2085944407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980b4070 con 0x7f4c9000b0d0 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f62224eb640 1 -- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62040051c0 msgr2=0x7f620400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f62224eb640 1 --2- 172.21.15.17:0/1952572943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62040051c0 0x7f620400b9c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f621400c410 tx=0x7f6204005830 comp rx=0 tx=0).stop 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f6220a88640 1 -- 172.21.15.17:0/1952572943 reap_dead start 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f62224eb640 1 -- 172.21.15.17:0/1952572943 shutdown_connections 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f62224eb640 1 -- 172.21.15.17:0/1952572943 >> 172.21.15.17:0/1952572943 conn(0x7f621c0acaa0 msgr2=0x7f621c0aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 -- 172.21.15.17:0/1522105282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 msgr2=0x7fbbf40c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 --2- 172.21.15.17:0/1522105282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 0x7fbbf40c6a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009dc0 tx=0x7fbbec02be00 comp rx=0 tx=0).stop 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 -- 172.21.15.17:0/1522105282 shutdown_connections 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 --2- 172.21.15.17:0/1522105282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf4159c00 0x7fbbf40c6a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 -- 172.21.15.17:0/1522105282 >> 172.21.15.17:0/1522105282 conn(0x7fbbf406b970 msgr2=0x7fbbf40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 -- 172.21.15.17:0/1522105282 shutdown_connections 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f4ac659c640 1 --2- 172.21.15.17:0/381593258 >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 0x7f4ac80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:23.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7fb956200640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 0x7fb938003d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c360 tx=0x7fb938004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9604307184c6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7fbbf8b72640 1 -- 172.21.15.17:0/1522105282 wait complete. 2024-03-06T07:20:23.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0003610 con 0x7f24f00aa680 2024-03-06T07:20:23.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0003a00 con 0x7f24f00aa680 2024-03-06T07:20:23.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f24ef59e640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f014ba00 con 0x7f24f00aa680 2024-03-06T07:20:23.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f24ef59e640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f01e9fd0 con 0x7f24f00aa680 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9000b0d0 msgr2=0x7f4c9004ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 --2- 172.21.15.17:0/2085944407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9000b0d0 0x7f4c9004ac70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c98050b10 tx=0x7f4c98069930 comp rx=0 tx=0).stop 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95127fc640 1 -- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94fc0052e0 msgr2=0x7f94fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95127fc640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94fc0052e0 0x7f94fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95127fc640 1 -- 172.21.15.17:0/763292280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94fc003a10 msgr2=0x7f94fc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95127fc640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94fc003a10 0x7f94fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95137fe640 1 -- 172.21.15.17:0/763292280 reap_dead start 2024-03-06T07:20:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f95127fc640 1 -- 172.21.15.17:0/763292280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951415d640 con 0x7f94fc004800 2024-03-06T07:20:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.329+0000 7f578f57a640 1 -- start start 2024-03-06T07:20:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f62224eb640 1 -- 172.21.15.17:0/1952572943 shutdown_connections 2024-03-06T07:20:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 shutdown_connections 2024-03-06T07:20:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578f57a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5788156bf0 0x7f57880c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578f57a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 0x7f578806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4ac6d9d640 1 --2- 172.21.15.17:0/381593258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac80c6d80 0x7f4ac806df70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ac806bf30 tx=0x7f4ac002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abf5aa226b2be28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 --2- 172.21.15.17:0/2085944407 >> v2:172.21.15.163:3300/0 conn(0x7f4c9004b1b0 0x7f4c900daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578f57a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578815d080 con 0x7f578815cb80 2024-03-06T07:20:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578f57a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57880a9960 con 0x7f5788156bf0 2024-03-06T07:20:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578f57a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578815d360 con 0x7f57880c6d80 2024-03-06T07:20:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4ac559a640 1 -- 172.21.15.17:0/381593258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0039070 con 0x7f4ac80c6d80 2024-03-06T07:20:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 --2- 172.21.15.17:0/2085944407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9000b0d0 0x7f4c9004ac70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 >> 172.21.15.17:0/2085944407 conn(0x7f4c90005e90 msgr2=0x7f4c900062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578db17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 0x7f578806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578db17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 0x7f578806df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52292/0 (socket says 172.21.15.17:52292) 2024-03-06T07:20:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578db17640 1 -- 172.21.15.17:0/553920670 learned_addr learned my addr 172.21.15.17:0/553920670 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578d316640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5788156bf0 0x7f57880c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f578cb15640 1 --1- 172.21.15.17:0/553920670 >> v1:172.21.15.17:6789/0 conn(0x7f578815cb80 0x7f578806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58342/0 (socket says 172.21.15.17:58342) 2024-03-06T07:20:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f62224eb640 1 -- 172.21.15.17:0/1952572943 wait complete. 2024-03-06T07:20:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 shutdown_connections 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f24ef59e640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24f014b300 con 0x7f24f00aa680 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f95127fc640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f951415cfb0 tx=0x7f94fc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d1f593175e9d34 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9508007c80 con 0x7f94fc004800 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9508030070 con 0x7f94fc004800 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.330+0000 7f9511ffb640 1 -- 172.21.15.17:0/763292280 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080074d0 con 0x7f94fc004800 2024-03-06T07:20:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f4ac559a640 1 -- 172.21.15.17:0/381593258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac002cc00 con 0x7f4ac80c6d80 2024-03-06T07:20:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24e0047070 con 0x7f24f00aa680 2024-03-06T07:20:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f4ac559a640 1 -- 172.21.15.17:0/381593258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0034440 con 0x7f4ac80c6d80 2024-03-06T07:20:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7fbbf8b72640 1 Processor -- start 2024-03-06T07:20:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f4c9759e640 1 -- 172.21.15.17:0/2085944407 wait complete. 2024-03-06T07:20:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480020c0 con 0x7fb9380039b0 2024-03-06T07:20:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948030070 con 0x7fb9380039b0 2024-03-06T07:20:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7fb9551fe640 1 -- 172.21.15.17:0/160576352 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94802cc60 con 0x7fb9380039b0 2024-03-06T07:20:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 -- 172.21.15.17:0/381593258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac80c6d80 msgr2=0x7f4ac806df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 msgr2=0x7fb938003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fb958464640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 0x7fb938003d90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb94800c360 tx=0x7fb938004820 comp rx=0 tx=0).stop 2024-03-06T07:20:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 --2- 172.21.15.17:0/381593258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac80c6d80 0x7f4ac806df70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ac806bf30 tx=0x7f4ac002be50 comp rx=0 tx=0).stop 2024-03-06T07:20:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f62224eb640 1 Processor -- start 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f9519847640 1 -- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 msgr2=0x7f94fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f9519847640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f951415cfb0 tx=0x7f94fc005830 comp rx=0 tx=0).stop 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7f9519847640 1 -- 172.21.15.17:0/763292280 shutdown_connections 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f9519847640 1 --2- 172.21.15.17:0/763292280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f94fc004c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f9519847640 1 -- 172.21.15.17:0/763292280 >> 172.21.15.17:0/763292280 conn(0x7f951406b1a0 msgr2=0x7f951406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f577ffff640 1 -- 172.21.15.17:0/553920670 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34369506 0 0) 0x7f578815d080 con 0x7f578815cb80 2024-03-06T07:20:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 -- 172.21.15.17:0/381593258 shutdown_connections 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 --2- 172.21.15.17:0/381593258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac80c6d80 0x7f4ac806df70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 --2- 172.21.15.17:0/381593258 >> v2:172.21.15.163:3300/0 conn(0x7f4ac8156bf0 0x7f4ac80c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 -- 172.21.15.17:0/381593258 >> 172.21.15.17:0/381593258 conn(0x7f4ac806b1a0 msgr2=0x7f4ac806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f24deffd640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24d006b5a0 0x7f24d006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f577ffff640 1 -- 172.21.15.17:0/553920670 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5774003520 con 0x7f578815cb80 2024-03-06T07:20:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 -- 172.21.15.17:0/381593258 shutdown_connections 2024-03-06T07:20:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f24edd9b640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24d006b5a0 0x7f24d006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 shutdown_connections 2024-03-06T07:20:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fb958464640 1 --2- 172.21.15.17:0/160576352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9380039b0 0x7fb938003d90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 >> 172.21.15.17:0/160576352 conn(0x7fb9500a8900 msgr2=0x7fb9500a8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f24e00ac640 con 0x7f24f00aa680 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.331+0000 7fbbf8b72640 1 -- start start 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fbbf8b72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf40724d0 0x7fbbf406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fbbf8b72640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 0x7fbbf40acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7fbbf8b72640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbf4159c00 0x7fbbf40ac140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf8b72640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40c2e20 con 0x7fbbf4159c00 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf8b72640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40c4c40 con 0x7fbbf4072a40 2024-03-06T07:20:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf8b72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40c2fa0 con 0x7fbbf40724d0 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f4accd65640 1 -- 172.21.15.17:0/381593258 wait complete. 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f578d316640 1 -- 172.21.15.17:0/553920670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 msgr2=0x7f578806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f578d316640 1 --2- 172.21.15.17:0/553920670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 0x7f578806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f578d316640 1 -- 172.21.15.17:0/553920670 >> v1:172.21.15.17:6789/0 conn(0x7f578815cb80 legacy=0x7f578806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f578d316640 1 -- 172.21.15.17:0/553920670 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578815d640 con 0x7f5788156bf0 2024-03-06T07:20:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f24deffd640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f24d0074d20 0x7f24d0077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24d0077830 con 0x7f24d0074d20 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24e00b3050 con 0x7f24f00aa680 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f24edd9b640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24d006b5a0 0x7f24d006da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f014efc0 tx=0x7f24e4005ad0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f4c9759e640 1 Processor -- start 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 0x7fbbf40acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf259c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf40724d0 0x7fbbf406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 0x7fbbf40acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59976/0 (socket says 172.21.15.17:59976) 2024-03-06T07:20:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 -- 172.21.15.17:0/3717583375 learned_addr learned my addr 172.21.15.17:0/3717583375 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f4c9759e640 1 -- start start 2024-03-06T07:20:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf2d9d640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbf4159c00 0x7fbbf40ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.332+0000 7f62224eb640 1 -- start start 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f62224eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040051c0 0x7f621c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 -- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf40724d0 msgr2=0x7fbbf406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf40724d0 0x7fbbf406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 -- 172.21.15.17:0/3717583375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbf4159c00 msgr2=0x7fbbf40ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbf4159c00 0x7fbbf40ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf406cc30 con 0x7fbbf4072a40 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf259c640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbbf40724d0 0x7fbbf406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f62224eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 0x7f621c150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f62224eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f621c0b01d0 0x7f621c0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 0x7f621c150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f621b7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 0x7f621c150490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52312/0 (socket says 172.21.15.17:52312) 2024-03-06T07:20:23.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7f621b7fe640 1 -- 172.21.15.17:0/1776869532 learned_addr learned my addr 172.21.15.17:0/1776869532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.333+0000 7fbbf1d9b640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 0x7fbbf40acf10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c3c0 tx=0x7fbbe800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621bfff640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040051c0 0x7f621c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f6220a88640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f621c0b01d0 0x7f621c0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0ca700 con 0x7f62040051c0 2024-03-06T07:20:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c158960 con 0x7f621c0b01d0 2024-03-06T07:20:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621c0ca880 con 0x7f621c0ad6b0 2024-03-06T07:20:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8017070 con 0x7fbbf4072a40 2024-03-06T07:20:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f9519847640 1 -- 172.21.15.17:0/763292280 shutdown_connections 2024-03-06T07:20:23.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f9519847640 1 -- 172.21.15.17:0/763292280 wait complete. 2024-03-06T07:20:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 shutdown_connections 2024-03-06T07:20:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fb958464640 1 -- 172.21.15.17:0/160576352 wait complete. 2024-03-06T07:20:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4accd65640 1 Processor -- start 2024-03-06T07:20:23.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe80052f0 con 0x7fbbf4072a40 2024-03-06T07:20:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4accd65640 1 -- start start 2024-03-06T07:20:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8003830 con 0x7fbbf4072a40 2024-03-06T07:20:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf41eff20 con 0x7fbbf4072a40 2024-03-06T07:20:23.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9759e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 0x7f4c90032460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf406d1b0 con 0x7fbbf4072a40 2024-03-06T07:20:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9004b1b0 0x7f4c900d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 0x7f4c90032460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 0x7f4c90032460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59984/0 (socket says 172.21.15.17:59984) 2024-03-06T07:20:23.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f24eed9d640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f24d0074d20 0x7f24d0077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9659c640 1 -- 172.21.15.17:0/3050509068 learned_addr learned my addr 172.21.15.17:0/3050509068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9759e640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900d3c40 0x7f4c90034a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90046360 con 0x7f4c9004b1b0 2024-03-06T07:20:23.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900461e0 con 0x7f4c9000b0d0 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90046e90 con 0x7f4c900d3c40 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c96d9d640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900d3c40 0x7f4c90034a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c95d9b640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9004b1b0 0x7f4c900d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621b7fe640 1 -- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f621c0b01d0 msgr2=0x7f621c0b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621b7fe640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f621c0b01d0 0x7f621c0b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621b7fe640 1 -- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040051c0 msgr2=0x7f621c14f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621b7fe640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62040051c0 0x7f621c14f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.334+0000 7f621b7fe640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c1f2070 con 0x7f621c0ad6b0 2024-03-06T07:20:23.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f621b7fe640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 0x7f621c150490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f620c00e540 tx=0x7f620c00e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbf404f330 con 0x7fbbf4072a40 2024-03-06T07:20:23.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 0x7f4ac80aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ac8156bf0 0x7f4ac8070c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c00f040 con 0x7f621c0ad6b0 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac815cb80 0x7f4ac80c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c2220 con 0x7f4ac815cb80 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8156fd0 con 0x7f4ac80c6d80 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4accd65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80c23a0 con 0x7f4ac8156bf0 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c004230 con 0x7f621c0ad6b0 2024-03-06T07:20:23.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 0x7f4ac80aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 0x7f4ac80aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59990/0 (socket says 172.21.15.17:59990) 2024-03-06T07:20:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4ac659c640 1 -- 172.21.15.17:0/4086526351 learned_addr learned my addr 172.21.15.17:0/4086526351 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7fb958464640 1 Processor -- start 2024-03-06T07:20:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f9519847640 1 Processor -- start 2024-03-06T07:20:23.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7fb958464640 1 -- start start 2024-03-06T07:20:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f24eed9d640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f24d0074d20 0x7f24d0077160 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbbe801a070 con 0x7fbbf4072a40 2024-03-06T07:20:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 -- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ac8156bf0 msgr2=0x7f4ac8070c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-06T07:20:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ac8156bf0 0x7f4ac8070c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 -- 172.21.15.17:0/4086526351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac815cb80 msgr2=0x7f4ac80c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ac815cb80 0x7f4ac80c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac81f2200 con 0x7f4ac80c6d80 2024-03-06T07:20:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c1f2310 con 0x7f621c0ad6b0 2024-03-06T07:20:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c1f28d0 con 0x7f621c0ad6b0 2024-03-06T07:20:23.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4ac659c640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 0x7f4ac80aa300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ebe0 tx=0x7f4abc00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 -- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900d3c40 msgr2=0x7f4c90034a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c900d3c40 0x7f4c90034a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 -- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9004b1b0 msgr2=0x7f4c900d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c9004b1b0 0x7f4c900d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900d8d00 con 0x7f4c9000b0d0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.335+0000 7f4c9659c640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 0x7f4c90032460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4c98064ac0 tx=0x7f4c98069a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980b4070 con 0x7f4c9000b0d0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9004a330 con 0x7f4c9000b0d0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90034220 con 0x7f4c9000b0d0 2024-03-06T07:20:23.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c980b0240 con 0x7f4c9000b0d0 2024-03-06T07:20:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.336+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980ba440 con 0x7f4c9000b0d0 2024-03-06T07:20:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc017070 con 0x7f4ac80c6d80 2024-03-06T07:20:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fbbe2ffd640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbbd406b4d0 0x7fbbd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00cc30 con 0x7f4ac80c6d80 2024-03-06T07:20:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbbe80896c0 con 0x7fbbf4072a40 2024-03-06T07:20:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb958464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9380039b0 0x7fb9500bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb958464640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9500bbb70 0x7fb9500c0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb958464640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 0x7fb95014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fbbe2ffd640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7fbbd4074bc0 0x7fbbd4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbe80054a0 con 0x7fbbd4074bc0 2024-03-06T07:20:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe8090050 con 0x7fbbf4072a40 2024-03-06T07:20:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb958464640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500c6290 con 0x7fb9380039b0 2024-03-06T07:20:23.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb956a01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 0x7fb95014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7f9519847640 1 -- start start 2024-03-06T07:20:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7f578d316640 1 --2- 172.21.15.17:0/553920670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5788156bf0 0x7f57880c6840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57840097a0 tx=0x7f578402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79457a9a9bd48672 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c003610 con 0x7f621c0ad6b0 2024-03-06T07:20:23.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c7c001030 con 0x7f4c9000b0d0 2024-03-06T07:20:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f577ffff640 1 -- 172.21.15.17:0/553920670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784031780 con 0x7f5788156bf0 2024-03-06T07:20:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fbbf259c640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbbd406b4d0 0x7fbbd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== osd.1 v2:172.21.15.17:6824/1161291487 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f24d0077830 con 0x7f24d0074d20 2024-03-06T07:20:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f577ffff640 1 -- 172.21.15.17:0/553920670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5784037070 con 0x7f5788156bf0 2024-03-06T07:20:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc013470 con 0x7f4ac80c6d80 2024-03-06T07:20:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f4accd65640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac81f24e0 con 0x7f4ac80c6d80 2024-03-06T07:20:23.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f577ffff640 1 -- 172.21.15.17:0/553920670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784031b20 con 0x7f5788156bf0 2024-03-06T07:20:23.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb956a01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 0x7fb95014cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60004/0 (socket says 172.21.15.17:60004) 2024-03-06T07:20:23.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fbbf2d9d640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7fbbd4074bc0 0x7fbbd4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f621c04f330 con 0x7f621c0ad6b0 2024-03-06T07:20:23.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.337+0000 7fb956a01640 1 -- 172.21.15.17:0/3460430587 learned_addr learned my addr 172.21.15.17:0/3460430587 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fb956200640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9380039b0 0x7fb9500bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95006dcb0 con 0x7fb95014caf0 2024-03-06T07:20:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7f4accd65640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac81f2aa0 con 0x7f4ac80c6d80 2024-03-06T07:20:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb950068e00 con 0x7fb9500bbb70 2024-03-06T07:20:23.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fbbf259c640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbbd406b4d0 0x7fbbd406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40011e0 tx=0x7fbbe4007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956200640 1 -- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9500bbb70 msgr2=0x7fb9500c0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.338+0000 7fb9559ff640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9500bbb70 0x7fb9500c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9519847640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f95140a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 -- 172.21.15.17:0/553920670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5788156bf0 msgr2=0x7f57880c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 --2- 172.21.15.17:0/553920670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5788156bf0 0x7f57880c6840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f57840097a0 tx=0x7f578402b820 comp rx=0 tx=0).stop 2024-03-06T07:20:23.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956200640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9500bbb70 0x7fb9500c0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9519847640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95141f2a00 0x7f951415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fbbf2d9d640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7fbbd4074bc0 0x7fbbd4077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956200640 1 -- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 msgr2=0x7fb95014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956200640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 0x7fb95014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f620c005990 con 0x7f621c0ad6b0 2024-03-06T07:20:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f95140a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9512ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f95140a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60014/0 (socket says 172.21.15.17:60014) 2024-03-06T07:20:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9512ffd640 1 -- 172.21.15.17:0/1914622446 learned_addr learned my addr 172.21.15.17:0/1914622446 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956200640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500c09c0 con 0x7fb9380039b0 2024-03-06T07:20:23.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f9519847640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9514150b00 0x7f951415b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4aa0001030 con 0x7f4ac80c6d80 2024-03-06T07:20:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c980c1020 con 0x7f4c9000b0d0 2024-03-06T07:20:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7fb956a01640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb95014caf0 0x7fb95014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:23.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 -- 172.21.15.17:0/553920670 shutdown_connections 2024-03-06T07:20:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 --2- 172.21.15.17:0/553920670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57880c6d80 0x7f578806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 --2- 172.21.15.17:0/553920670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5788156bf0 0x7f57880c6840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f578f57a640 1 -- 172.21.15.17:0/553920670 >> 172.21.15.17:0/553920670 conn(0x7f578806b1a0 msgr2=0x7f578806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.339+0000 7f95127fc640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95141f2a00 0x7f951415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7fb9559ff640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9500bbb70 0x7fb9500c0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6218ff9640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f620806b5d0 0x7f620806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7fb956200640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9380039b0 0x7fb9500bfbd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb948001b80 tx=0x7fb948001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9514150b00 0x7f951415b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95140c2220 con 0x7f9514150b00 2024-03-06T07:20:23.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9514156fd0 con 0x7f94fc004800 2024-03-06T07:20:23.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f621bfff640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f620806b5d0 0x7f620806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95140c23a0 con 0x7f95141f2a00 2024-03-06T07:20:23.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f4c86ffd640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7806b520 0x7f4c7806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f620c08a970 con 0x7f621c0ad6b0 2024-03-06T07:20:23.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4c98127320 con 0x7f4c9000b0d0 2024-03-06T07:20:23.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f4c95d9b640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7806b520 0x7f4c7806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6218ff9640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7f6208074d50 0x7f6208077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f620c004230 con 0x7f6208074d50 2024-03-06T07:20:23.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620c08ad60 con 0x7f621c0ad6b0 2024-03-06T07:20:23.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f578f57a640 1 -- 172.21.15.17:0/553920670 shutdown_connections 2024-03-06T07:20:23.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f4c86ffd640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f4c78074c90 0x7f4c780770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f578f57a640 1 -- 172.21.15.17:0/553920670 wait complete. 2024-03-06T07:20:23.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4abc01a070 con 0x7f4ac80c6d80 2024-03-06T07:20:23.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 -- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95141f2a00 msgr2=0x7f951415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95141f2a00 0x7f951415b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 -- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 msgr2=0x7f95140a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94fc004800 0x7f95140a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f6220a88640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7f6208074d50 0x7f6208077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480020c0 con 0x7fb9380039b0 2024-03-06T07:20:23.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aa806b5d0 0x7f4aa806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.340+0000 7f95137fe640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951415bca0 con 0x7f9514150b00 2024-03-06T07:20:23.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== osd.6 v2:172.21.15.163:6808/1305126403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbbe80054a0 con 0x7fbbd4074bc0 2024-03-06T07:20:23.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f95127fc640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95141f2a00 0x7f951415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4abc08a7b0 con 0x7f4ac80c6d80 2024-03-06T07:20:23.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ac5d9b640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aa806b5d0 0x7f4aa806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f6220a88640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7f6208074d50 0x7f6208077190 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4c95d9b640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7806b520 0x7f4c7806d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c88007fd0 tx=0x7f4c88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7f4aa8074d50 0x7f4aa8077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4abc0093e0 con 0x7f4aa8074d50 2024-03-06T07:20:23.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4abc091050 con 0x7f4ac80c6d80 2024-03-06T07:20:23.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f95137fe640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9514150b00 0x7f951415b760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95140c0d20 tx=0x7f950c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c98062a10 con 0x7f4c78074c90 2024-03-06T07:20:23.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4ac6d9d640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7f4aa8074d50 0x7f4aa8077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c9812e050 con 0x7f4c9000b0d0 2024-03-06T07:20:23.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c0046a0 con 0x7f9514150b00 2024-03-06T07:20:23.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f4c96d9d640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f4c78074c90 0x7f4c780770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c01a070 con 0x7f9514150b00 2024-03-06T07:20:23.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f4ac5d9b640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aa806b5d0 0x7f4aa806da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8070440 tx=0x7f4ab80078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95141ef490 con 0x7f9514150b00 2024-03-06T07:20:23.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95141efa30 con 0x7f9514150b00 2024-03-06T07:20:23.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f578f57a640 1 Processor -- start 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948044070 con 0x7fb9380039b0 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb950150b20 con 0x7fb9380039b0 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9501e6a50 con 0x7fb9380039b0 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480020c0 con 0x7fb9380039b0 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.343+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c015660 con 0x7f9514150b00 2024-03-06T07:20:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.342+0000 7f4c96d9d640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f4c78074c90 0x7f4c780770d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.343+0000 7f578f57a640 1 -- start start 2024-03-06T07:20:23.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.343+0000 7f4ac6d9d640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7f4aa8074d50 0x7f4aa8077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.343+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== osd.0 v2:172.21.15.17:6800/3406119864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4c98062a10 con 0x7f4c78074c90 2024-03-06T07:20:23.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb95004f330 con 0x7fb9380039b0 2024-03-06T07:20:23.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f951404f330 con 0x7f9514150b00 2024-03-06T07:20:23.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7f578f57a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 0x7f5788072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== osd.7 v2:172.21.15.163:6816/145909431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4abc0093e0 con 0x7f4aa8074d50 2024-03-06T07:20:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7f578f57a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5788156bf0 0x7f57880c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.341+0000 7f621bfff640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f620806b5d0 0x7f620806da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6210006cc0 tx=0x7f6210006bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.344+0000 7f578d316640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 0x7f5788072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578d316640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 0x7f5788072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60024/0 (socket says 172.21.15.17:60024) 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578cb15640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5788156bf0 0x7f57880c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578d316640 1 -- 172.21.15.17:0/1701251036 learned_addr learned my addr 172.21.15.17:0/1701251036 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578f57a640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578815cb80 0x7f578815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578f57a640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57880c2220 con 0x7f578815cb80 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578f57a640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788156fd0 con 0x7f57880c6d80 2024-03-06T07:20:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f578f57a640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57880c23a0 con 0x7f5788156bf0 2024-03-06T07:20:23.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== osd.3 v2:172.21.15.17:6816/3152244895 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f620c004230 con 0x7f6208074d50 2024-03-06T07:20:23.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f578db17640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578815cb80 0x7f578815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb948007270 con 0x7fb9380039b0 2024-03-06T07:20:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f950c015800 con 0x7f9514150b00 2024-03-06T07:20:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.345+0000 7f94fbfff640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f950006b5a0 0x7f950006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f950c08aac0 con 0x7f9514150b00 2024-03-06T07:20:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f94fbfff640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7f9500074c90 0x7f95000770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f950c01bec0 con 0x7f9500074c90 2024-03-06T07:20:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f9512ffd640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f950006b5a0 0x7f950006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f950c055210 con 0x7f9514150b00 2024-03-06T07:20:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7f95127fc640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7f9500074c90 0x7f95000770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.346+0000 7fb94effd640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb93c06b700 0x7fb93c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb9480ab550 con 0x7fb9380039b0 2024-03-06T07:20:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 -- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5788156bf0 msgr2=0x7f57880c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5788156bf0 0x7f57880c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 -- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 msgr2=0x7f5788072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 0x7f5788072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788072ee0 con 0x7f578815cb80 2024-03-06T07:20:23.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f9512ffd640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f950006b5a0 0x7f950006da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9504005880 tx=0x7f9504009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb94effd640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7fb93c074d80 0x7fb93c0771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb94800bf30 con 0x7fb93c074d80 2024-03-06T07:20:23.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f578db17640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578815cb80 0x7f578815b8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f578000efc0 tx=0x7f578000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb9559ff640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb93c06b700 0x7fb93c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb956a01640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7fb93c074d80 0x7fb93c0771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578000cd20 con 0x7f578815cb80 2024-03-06T07:20:23.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780003990 con 0x7f578815cb80 2024-03-06T07:20:23.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.347+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9480b2050 con 0x7fb9380039b0 2024-03-06T07:20:23.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f578d316640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57880c6d80 0x7f5788072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f95127fc640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7f9500074c90 0x7f95000770d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7fb956a01640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7fb93c074d80 0x7fb93c0771c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f578f57a640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57880c4710 con 0x7f578815cb80 2024-03-06T07:20:23.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7fb9559ff640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb93c06b700 0x7fb93c06dbc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9501e43a0 tx=0x7fb944008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f578f57a640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578815c1c0 con 0x7f578815cb80 2024-03-06T07:20:23.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.348+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== osd.5 v2:172.21.15.163:6800/3569452097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f950c01bec0 con 0x7f9500074c90 2024-03-06T07:20:23.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.349+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== osd.4 v2:172.21.15.163:6824/2660678220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb94800bf30 con 0x7fb93c074d80 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.349+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780003d80 con 0x7f578815cb80 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.351+0000 7f578cb15640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5788156bf0 0x7f57880c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.351+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5780005d10 con 0x7f578815cb80 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.351+0000 7f577dffb640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f576c06b5a0 0x7f576c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.352+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57880728a0 con 0x7f578815cb80 2024-03-06T07:20:23.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.352+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f578008aa60 con 0x7f578815cb80 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.352+0000 7f578d316640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f576c06b5a0 0x7f576c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.352+0000 7f577dffb640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f576c074ca0 0x7f576c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.352+0000 7f578cb15640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f576c074ca0 0x7f576c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.353+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5780011e50 con 0x7f576c074ca0 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.353+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5780055270 con 0x7f578815cb80 2024-03-06T07:20:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.353+0000 7f578cb15640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f576c074ca0 0x7f576c0770e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.354+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== osd.2 v2:172.21.15.17:6802/1741224553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5780011e50 con 0x7f576c074ca0 2024-03-06T07:20:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.355+0000 7f578d316640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f576c06b5a0 0x7f576c06da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5784034040 tx=0x7f5784008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.368+0000 7f24ef59e640 1 -- 172.21.15.17:0/1888605393 --> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f24f01ea210 con 0x7f24d0074d20 2024-03-06T07:20:23.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.369+0000 7f24deffd640 1 -- 172.21.15.17:0/1888605393 <== osd.1 v2:172.21.15.17:6824/1161291487 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f24f01ea210 con 0x7f24d0074d20 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.369+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f24d0074d20 msgr2=0x7f24d0077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f24d0074d20 0x7f24d0077160 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24d006b5a0 msgr2=0x7f24d006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24d006b5a0 0x7f24d006da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f014efc0 tx=0x7f24e4005ad0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 msgr2=0x7f24f014e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 --2- 172.21.15.17:0/1888605393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f00aa680 0x7f24f014e430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f24e0036040 tx=0x7f24e00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24eed9d640 1 -- 172.21.15.17:0/1888605393 reap_dead start 2024-03-06T07:20:23.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.370+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 shutdown_connections 2024-03-06T07:20:23.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.371+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 >> 172.21.15.17:0/1888605393 conn(0x7f24f00c5df0 msgr2=0x7f24f014fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.371+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 shutdown_connections 2024-03-06T07:20:23.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.371+0000 7f24dcff9640 1 -- 172.21.15.17:0/1888605393 wait complete. 2024-03-06T07:20:23.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.373+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 --> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbf406fb10 con 0x7fbbd4074bc0 2024-03-06T07:20:23.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.377+0000 7fbbe2ffd640 1 -- 172.21.15.17:0/3717583375 <== osd.6 v2:172.21.15.163:6808/1305126403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbbf406fb10 con 0x7fbbd4074bc0 2024-03-06T07:20:23.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.377+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 --> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c7c002dd0 con 0x7f4c78074c90 2024-03-06T07:20:23.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.378+0000 7f4c86ffd640 1 -- 172.21.15.17:0/3050509068 <== osd.0 v2:172.21.15.17:6800/3406119864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c7c002dd0 con 0x7f4c78074c90 2024-03-06T07:20:23.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.378+0000 7f62224eb640 1 -- 172.21.15.17:0/1776869532 --> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f621c0c5710 con 0x7f6208074d50 2024-03-06T07:20:23.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.379+0000 7f6218ff9640 1 -- 172.21.15.17:0/1776869532 <== osd.3 v2:172.21.15.17:6816/3152244895 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f621c0c5710 con 0x7f6208074d50 2024-03-06T07:20:23.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.379+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f4c78074c90 msgr2=0x7f4c780770d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.379+0000 7f4c9759e640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f4c78074c90 0x7f4c780770d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7f6208074d50 msgr2=0x7f6208077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7f6208074d50 0x7f6208077190 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f620806b5d0 msgr2=0x7f620806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f620806b5d0 0x7f620806da90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6210006cc0 tx=0x7f6210006bd0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 msgr2=0x7f621c150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f62027fc640 1 --2- 172.21.15.17:0/1776869532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f621c0ad6b0 0x7f621c150490 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f620c00e540 tx=0x7f620c00e940 comp rx=0 tx=0).stop 2024-03-06T07:20:23.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f6220a88640 1 -- 172.21.15.17:0/1776869532 reap_dead start 2024-03-06T07:20:23.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7fbbd4074bc0 msgr2=0x7fbbd4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7fbbd4074bc0 0x7fbbd4077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 --> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95141534d0 con 0x7f9500074c90 2024-03-06T07:20:23.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbbd406b4d0 msgr2=0x7fbbd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbbd406b4d0 0x7fbbd406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40011e0 tx=0x7fbbe4007d30 comp rx=0 tx=0).stop 2024-03-06T07:20:23.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 msgr2=0x7fbbf40acf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 --2- 172.21.15.17:0/3717583375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbbf4072a40 0x7fbbf40acf10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbbe800c3c0 tx=0x7fbbe800c880 comp rx=0 tx=0).stop 2024-03-06T07:20:23.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf2d9d640 1 -- 172.21.15.17:0/3717583375 reap_dead start 2024-03-06T07:20:23.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7806b520 msgr2=0x7f4c7806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 --> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4aa0002dd0 con 0x7f4aa8074d50 2024-03-06T07:20:23.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7f4c9759e640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7806b520 0x7f4c7806d9e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c88007fd0 tx=0x7f4c88009040 comp rx=0 tx=0).stop 2024-03-06T07:20:23.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 shutdown_connections 2024-03-06T07:20:23.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.380+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 >> 172.21.15.17:0/3717583375 conn(0x7fbbf406b970 msgr2=0x7fbbf4155760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 msgr2=0x7f4c90032460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 --2- 172.21.15.17:0/3050509068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c9000b0d0 0x7f4c90032460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4c98064ac0 tx=0x7f4c98069a40 comp rx=0 tx=0).stop 2024-03-06T07:20:23.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c96d9d640 1 -- 172.21.15.17:0/3050509068 reap_dead start 2024-03-06T07:20:23.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 shutdown_connections 2024-03-06T07:20:23.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 shutdown_connections 2024-03-06T07:20:23.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 >> 172.21.15.17:0/3050509068 conn(0x7f4c90005e90 msgr2=0x7f4c900d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 >> 172.21.15.17:0/1776869532 conn(0x7f621c0acaa0 msgr2=0x7f621c0aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 shutdown_connections 2024-03-06T07:20:23.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7fbbf8b72640 1 -- 172.21.15.17:0/3717583375 wait complete. 2024-03-06T07:20:23.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 shutdown_connections 2024-03-06T07:20:23.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 shutdown_connections 2024-03-06T07:20:23.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4c9759e640 1 -- 172.21.15.17:0/3050509068 wait complete. 2024-03-06T07:20:23.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f94fbfff640 1 -- 172.21.15.17:0/1914622446 <== osd.5 v2:172.21.15.163:6800/3569452097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f95141534d0 con 0x7f9500074c90 2024-03-06T07:20:23.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/4086526351 <== osd.7 v2:172.21.15.163:6816/145909431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4aa0002dd0 con 0x7f4aa8074d50 2024-03-06T07:20:23.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7f62027fc640 1 -- 172.21.15.17:0/1776869532 wait complete. 2024-03-06T07:20:23.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.381+0000 7fb958464640 1 -- 172.21.15.17:0/3460430587 --> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9500c3670 con 0x7fb93c074d80 2024-03-06T07:20:23.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7f4aa8074d50 msgr2=0x7f4aa8077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f4ab4ff9640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7f4aa8074d50 0x7f4aa8077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7fb94effd640 1 -- 172.21.15.17:0/3460430587 <== osd.4 v2:172.21.15.163:6824/2660678220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb9500c3670 con 0x7fb93c074d80 2024-03-06T07:20:23.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7f9500074c90 msgr2=0x7f95000770d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7f9500074c90 0x7f95000770d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f950006b5a0 msgr2=0x7f950006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f950006b5a0 0x7f950006da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9504005880 tx=0x7f9504009040 comp rx=0 tx=0).stop 2024-03-06T07:20:23.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9514150b00 msgr2=0x7f951415b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7fb93c074d80 msgr2=0x7fb93c0771c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7fb93c074d80 0x7fb93c0771c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f9519847640 1 --2- 172.21.15.17:0/1914622446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9514150b00 0x7f951415b760 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95140c0d20 tx=0x7f950c007700 comp rx=0 tx=0).stop 2024-03-06T07:20:23.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.382+0000 7f95137fe640 1 -- 172.21.15.17:0/1914622446 reap_dead start 2024-03-06T07:20:23.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb93c06b700 msgr2=0x7fb93c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb93c06b700 0x7fb93c06dbc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb9501e43a0 tx=0x7fb944008040 comp rx=0 tx=0).stop 2024-03-06T07:20:23.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 shutdown_connections 2024-03-06T07:20:23.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9380039b0 msgr2=0x7fb9500bfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb94cff9640 1 --2- 172.21.15.17:0/3460430587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9380039b0 0x7fb9500bfbd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb948001b80 tx=0x7fb948001af0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7fb956a01640 1 -- 172.21.15.17:0/3460430587 reap_dead start 2024-03-06T07:20:23.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.383+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 >> 172.21.15.17:0/1914622446 conn(0x7f951406b1a0 msgr2=0x7f95141543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 shutdown_connections 2024-03-06T07:20:23.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 shutdown_connections 2024-03-06T07:20:23.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 >> 172.21.15.17:0/3460430587 conn(0x7fb9500a8900 msgr2=0x7fb9500c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 shutdown_connections 2024-03-06T07:20:23.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7fb94cff9640 1 -- 172.21.15.17:0/3460430587 wait complete. 2024-03-06T07:20:23.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f9519847640 1 -- 172.21.15.17:0/1914622446 wait complete. 2024-03-06T07:20:23.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aa806b5d0 msgr2=0x7f4aa806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aa806b5d0 0x7f4aa806da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8070440 tx=0x7f4ab80078f0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 msgr2=0x7f4ac80aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 --2- 172.21.15.17:0/4086526351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ac80c6d80 0x7f4ac80aa300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ebe0 tx=0x7f4abc00c520 comp rx=0 tx=0).stop 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ac6d9d640 1 -- 172.21.15.17:0/4086526351 reap_dead start 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 shutdown_connections 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 >> 172.21.15.17:0/4086526351 conn(0x7f4ac806b1a0 msgr2=0x7f4ac80c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 shutdown_connections 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.384+0000 7f4ab4ff9640 1 -- 172.21.15.17:0/4086526351 wait complete. 2024-03-06T07:20:23.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.387+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 --> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57880ab480 con 0x7f576c074ca0 2024-03-06T07:20:23.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.387+0000 7f577dffb640 1 -- 172.21.15.17:0/1701251036 <== osd.2 v2:172.21.15.17:6802/1741224553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57880ab480 con 0x7f576c074ca0 2024-03-06T07:20:23.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.388+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f576c074ca0 msgr2=0x7f576c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.388+0000 7f57737fe640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f576c074ca0 0x7f576c0770e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.477 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:23.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f576c06b5a0 msgr2=0x7f576c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f576c06b5a0 0x7f576c06da60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5784034040 tx=0x7f5784008030 comp rx=0 tx=0).stop 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578815cb80 msgr2=0x7f578815b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 --2- 172.21.15.17:0/1701251036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f578815cb80 0x7f578815b8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f578000efc0 tx=0x7f578000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f578db17640 1 -- 172.21.15.17:0/1701251036 reap_dead start 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 shutdown_connections 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 >> 172.21.15.17:0/1701251036 conn(0x7f578806b1a0 msgr2=0x7f578806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 shutdown_connections 2024-03-06T07:20:23.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.390+0000 7f57737fe640 1 -- 172.21.15.17:0/1701251036 wait complete. 2024-03-06T07:20:23.479 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:23.480 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-06T07:20:23.481 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:23.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-06T07:20:23.482 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:23.482 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-06T07:20:23.483 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:23.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-06T07:20:23.484 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:23.484 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-06T07:20:23.485 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:23.485 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-06T07:20:23.486 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:23.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-06T07:20:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.689+0000 7f35b56af640 1 Processor -- start 2024-03-06T07:20:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.689+0000 7f35b56af640 1 -- start start 2024-03-06T07:20:23.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.689+0000 7f4cce3ac640 1 Processor -- start 2024-03-06T07:20:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f1dd1b87640 1 Processor -- start 2024-03-06T07:20:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f1dd1b87640 1 -- start start 2024-03-06T07:20:23.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f4cce3ac640 1 -- start start 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f35b56af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f35b56af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 0x7f35b015d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f35b56af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b01598c0 con 0x7f35b00acdd0 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f35b56af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b0159ba0 con 0x7f35b00c0320 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.690+0000 7f35b56af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0159e80 con 0x7f35b00c6d30 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35af7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 0x7f35b015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52368/0 (socket says 172.21.15.17:52368) 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35af7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 0x7f35b015d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60032/0 (socket says 172.21.15.17:60032) 2024-03-06T07:20:23.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3893852279 learned_addr learned my addr 172.21.15.17:0/3893852279 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f90cc3b6640 1 Processor -- start 2024-03-06T07:20:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f90cc3b6640 1 -- start start 2024-03-06T07:20:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cce3ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cc004dfe0 0x7f4cc004e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f1dd1b87640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3893852279 >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 msgr2=0x7f35b015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 --2- 172.21.15.17:0/3893852279 >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 0x7f35b015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3893852279 >> v1:172.21.15.163:6789/0 conn(0x7f35b00acdd0 legacy=0x7f35b0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3893852279 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b015a160 con 0x7f35b00c6d30 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cce3ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc004dad0 0x7f4cc004ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cce3ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc000f9c0 con 0x7f4cc00d7ad0 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cce3ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc004f240 con 0x7f4cc004dfe0 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cce3ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00da4a0 con 0x7f4cc004dad0 2024-03-06T07:20:23.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f5226cd4640 1 Processor -- start 2024-03-06T07:20:23.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cccba9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cc004dfe0 0x7f4cc004e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cccba9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cc004dfe0 0x7f4cc004e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60044/0 (socket says 172.21.15.17:60044) 2024-03-06T07:20:23.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f4cccba9640 1 -- 172.21.15.17:0/1795073830 learned_addr learned my addr 172.21.15.17:0/1795073830 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f1dd1b87640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1dc4006ef0 0x7f1dc40e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f1dd1b87640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc4031170 con 0x7f1dc40dbfe0 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f1dd1b87640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc404aa70 con 0x7f1dc4006ef0 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.691+0000 7f1dd1b87640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4031430 con 0x7f1dc400b0d0 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f14ebb79640 1 Processor -- start 2024-03-06T07:20:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52386/0 (socket says 172.21.15.17:52386) 2024-03-06T07:20:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 -- 172.21.15.17:0/2850802085 learned_addr learned my addr 172.21.15.17:0/2850802085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dd1386640 1 --2- 172.21.15.17:0/2850802085 >> v2:172.21.15.163:3300/0 conn(0x7f1dc4006ef0 0x7f1dc40e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f14ebb79640 1 -- start start 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f5226cd4640 1 -- start start 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f4ccdbab640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc004dad0 0x7f4cc004ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f4cccba9640 1 -- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc004dad0 msgr2=0x7f4cc004ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f4cccba9640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc004dad0 0x7f4cc004ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f4cccba9640 1 -- 172.21.15.17:0/1795073830 >> v1:172.21.15.163:6789/0 conn(0x7f4cc00d7ad0 legacy=0x7f4cc00071a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f4cccba9640 1 -- 172.21.15.17:0/1795073830 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00da780 con 0x7f4cc004dfe0 2024-03-06T07:20:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f5803038640 1 Processor -- start 2024-03-06T07:20:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f5803038640 1 -- start start 2024-03-06T07:20:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f90cc3b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90c414f150 0x7f90c4158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f35ae7fc640 1 --2- 172.21.15.17:0/3893852279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009e50 tx=0x7f35a802bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466fa82f7f5b909 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5226cd4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f522015cd60 0x7f522015d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5a49cdc640 1 Processor -- start 2024-03-06T07:20:23.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f4cccba9640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dfe0 0x7f4cc004e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4cc807ae30 tx=0x7f4cc8068d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703e78186984fbf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5226cd4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c6560 0x7f52200c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5a49cdc640 1 -- start start 2024-03-06T07:20:23.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5226cd4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f522015d830 con 0x7f52200c0e70 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5226cd4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522006cbf0 con 0x7f522015cd60 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5226cd4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522006ce40 con 0x7f52200c6560 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 -- 172.21.15.17:0/2850802085 >> v2:172.21.15.163:3300/0 conn(0x7f1dc4006ef0 msgr2=0x7f1dc40e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/2850802085 >> v2:172.21.15.163:3300/0 conn(0x7f1dc4006ef0 0x7f1dc40e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 -- 172.21.15.17:0/2850802085 >> v1:172.21.15.17:6789/0 conn(0x7f1dc40dbfe0 legacy=0x7f1dc403e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f1dc3fff640 1 -- 172.21.15.17:0/2850802085 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc404b0f0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f4cbffff640 1 -- 172.21.15.17:0/1795073830 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80b4070 con 0x7f4cc004dfe0 2024-03-06T07:20:23.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f4cbffff640 1 -- 172.21.15.17:0/1795073830 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80a7b70 con 0x7f4cc004dfe0 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5224a70640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f522015cd60 0x7f522015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5224a70640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f522015cd60 0x7f522015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38254/0 (socket says 172.21.15.17:38254) 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f90cc3b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40ab0c0 con 0x7f90c40a8ce0 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f90cc3b6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40ac0a0 con 0x7f90c40aac30 2024-03-06T07:20:23.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.692+0000 7f90cc3b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c4156150 con 0x7f90c414f150 2024-03-06T07:20:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f5224a70640 1 -- 172.21.15.17:0/3687590829 learned_addr learned my addr 172.21.15.17:0/3687590829 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f14ebb79640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f14ebb79640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14e40c6110 0x7f14e40c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f14ebb79640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e406eaa0 con 0x7f14e40c3850 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f14ebb79640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40c31a0 con 0x7f14e40c6110 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f14ebb79640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406ed80 con 0x7f14e4073360 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52392/0 (socket says 172.21.15.17:52392) 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9915640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f14e40c3850 0x7f14e4156a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57476/0 (socket says 172.21.15.17:57476) 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 -- 172.21.15.17:0/755535394 learned_addr learned my addr 172.21.15.17:0/755535394 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14ea116640 1 --2- 172.21.15.17:0/755535394 >> v2:172.21.15.163:3300/0 conn(0x7f14e40c6110 0x7f14e40c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/2850802085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009870 tx=0x7f1dbc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dfb719d75d42916 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5803038640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57fc06e540 0x7f57fc0abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f90ca152640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90c414f150 0x7f90c4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f90ca152640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90c414f150 0x7f90c4158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38250/0 (socket says 172.21.15.17:38250) 2024-03-06T07:20:23.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.693+0000 7f90ca152640 1 -- 172.21.15.17:0/4124124179 learned_addr learned my addr 172.21.15.17:0/4124124179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 -- 172.21.15.17:0/755535394 >> v2:172.21.15.163:3300/0 conn(0x7f14e40c6110 msgr2=0x7f14e40c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 --2- 172.21.15.17:0/755535394 >> v2:172.21.15.163:3300/0 conn(0x7f14e40c6110 0x7f14e40c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 -- 172.21.15.17:0/755535394 >> v1:172.21.15.163:6789/0 conn(0x7f14e40c3850 legacy=0x7f14e4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f14e9114640 1 -- 172.21.15.17:0/755535394 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40c5bf0 con 0x7f14e4073360 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 -- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dfe0 msgr2=0x7f4cc004e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dfe0 0x7f4cc004e7c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4cc807ae30 tx=0x7f4cc8068d50 comp rx=0 tx=0).stop 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f35adffb640 1 -- 172.21.15.17:0/3893852279 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a802cb20 con 0x7f35b00c6d30 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f35adffb640 1 -- 172.21.15.17:0/3893852279 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a802ccc0 con 0x7f35b00c6d30 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f35adffb640 1 -- 172.21.15.17:0/3893852279 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8034610 con 0x7f35b00c6d30 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5803038640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0c4e50 con 0x7f57fc0c47d0 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5803038640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc0c61f0 con 0x7f57fc151eb0 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5803038640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c5330 con 0x7f57fc06e540 2024-03-06T07:20:23.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5801835640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f57fc151eb0 0x7f57fc0c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57488/0 (socket says 172.21.15.17:57488) 2024-03-06T07:20:23.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5801835640 1 -- 172.21.15.17:0/3922726214 learned_addr learned my addr 172.21.15.17:0/3922726214 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:23.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5a49cdc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f1dc37fe640 1 -- 172.21.15.17:0/2850802085 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0317c0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 -- 172.21.15.17:0/1795073830 shutdown_connections 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc004dad0 0x7f4cc004ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 --2- 172.21.15.17:0/1795073830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dfe0 0x7f4cc004e7c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f4cce3ac640 1 -- 172.21.15.17:0/1795073830 >> 172.21.15.17:0/1795073830 conn(0x7f4cc004c650 msgr2=0x7f4cc004ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f1dc37fe640 1 -- 172.21.15.17:0/2850802085 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc037070 con 0x7f1dc400b0d0 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f1dc37fe640 1 -- 172.21.15.17:0/2850802085 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0317c0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f90ca152640 1 -- 172.21.15.17:0/4124124179 >> v1:172.21.15.163:6789/0 conn(0x7f90c40aac30 legacy=0x7f90c406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f90ca152640 1 -- 172.21.15.17:0/4124124179 >> v1:172.21.15.17:6789/0 conn(0x7f90c40a8ce0 legacy=0x7f90c4155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f90ca152640 1 -- 172.21.15.17:0/4124124179 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4155280 con 0x7f90c414f150 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5a48cda640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5a48cda640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60084/0 (socket says 172.21.15.17:60084) 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5a49cdc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c03f740 0x7f5a3c030ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a48cda640 1 -- 172.21.15.17:0/508078527 learned_addr learned my addr 172.21.15.17:0/508078527 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c031b20 con 0x7f5a3c049f40 2024-03-06T07:20:23.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198644392 0 0) 0x7f57fc0c61f0 con 0x7f57fc151eb0 2024-03-06T07:20:23.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e8005730 con 0x7f57fc151eb0 2024-03-06T07:20:23.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5802036640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57fc06e540 0x7f57fc0abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a43fff640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c03f740 0x7f5a3c030ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c00b800 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f14e9114640 1 --2- 172.21.15.17:0/755535394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009870 tx=0x7f14d802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d7ca896908b481 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a437fe640 1 -- 172.21.15.17:0/508078527 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318187652 0 0) 0x7f5a3c031b20 con 0x7f5a3c049f40 2024-03-06T07:20:23.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0cd690 con 0x7f5a3c03f740 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f14e8913640 1 -- 172.21.15.17:0/755535394 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80317c0 con 0x7f14e4073360 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f14e8913640 1 -- 172.21.15.17:0/755535394 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d8037070 con 0x7f14e4073360 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f14e8913640 1 -- 172.21.15.17:0/755535394 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8031b80 con 0x7f14e4073360 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150257230 0 0) 0x7f57e8005730 con 0x7f57fc151eb0 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57fc06e540 msgr2=0x7f57fc0abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57fc06e540 0x7f57fc0abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 >> v1:172.21.15.17:6789/0 conn(0x7f57fc0c47d0 legacy=0x7f57fc06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0c1910 con 0x7f57fc151eb0 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f57ec001670 con 0x7f57fc151eb0 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 >> v1:172.21.15.163:6789/0 conn(0x7f57fc151eb0 legacy=0x7f57fc0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e8005bf0 0x7f57e8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57e8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e80074f0 0x7f57e800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c5330 con 0x7f57e80074f0 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e800e410 con 0x7f57e8005bf0 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e800e610 con 0x7f57e8006a10 2024-03-06T07:20:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f1dd1b87640 1 -- 172.21.15.17:0/2850802085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 msgr2=0x7f1dc40069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/2850802085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009870 tx=0x7f1dbc02b860 comp rx=0 tx=0).stop 2024-03-06T07:20:23.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5801835640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57e8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5224a70640 1 -- 172.21.15.17:0/3687590829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c6560 msgr2=0x7f52200c6e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:20:23.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5224a70640 1 --2- 172.21.15.17:0/3687590829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c6560 0x7f52200c6e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5224a70640 1 -- 172.21.15.17:0/3687590829 >> v1:172.21.15.17:6789/0 conn(0x7f52200c0e70 legacy=0x7f52200c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.694+0000 7f5224a70640 1 -- 172.21.15.17:0/3687590829 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522006d120 con 0x7f522015cd60 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.695+0000 7f5224a70640 1 --2- 172.21.15.17:0/3687590829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f522015cd60 0x7f522015d140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f521c009870 tx=0x7f521c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff106e072313066 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f52177fe640 1 -- 172.21.15.17:0/3687590829 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c035020 con 0x7f522015cd60 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f1dd1b87640 1 -- 172.21.15.17:0/2850802085 shutdown_connections 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/2850802085 >> v2:172.21.15.163:3300/0 conn(0x7f1dc4006ef0 0x7f1dc40e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/2850802085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc400b0d0 0x7f1dc40069b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f1dd1b87640 1 -- 172.21.15.17:0/2850802085 >> 172.21.15.17:0/2850802085 conn(0x7f1dc4005e90 msgr2=0x7f1dc40062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5a437fe640 1 -- 172.21.15.17:0/508078527 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a34005730 con 0x7f5a3c049f40 2024-03-06T07:20:23.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f52177fe640 1 -- 172.21.15.17:0/3687590829 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c031870 con 0x7f522015cd60 2024-03-06T07:20:23.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f52177fe640 1 -- 172.21.15.17:0/3687590829 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c035020 con 0x7f522015cd60 2024-03-06T07:20:23.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f4cce3ac640 1 -- 172.21.15.17:0/1795073830 shutdown_connections 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f4cce3ac640 1 -- 172.21.15.17:0/1795073830 wait complete. 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f35b56af640 1 -- 172.21.15.17:0/3893852279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 msgr2=0x7f35b015ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f35b56af640 1 --2- 172.21.15.17:0/3893852279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009e50 tx=0x7f35a802bf80 comp rx=0 tx=0).stop 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f90ca152640 1 --2- 172.21.15.17:0/4124124179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c414f150 0x7f90c4158760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009e50 tx=0x7f90bc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de9f816df96e294 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5a48cda640 1 -- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c03f740 msgr2=0x7f5a3c030ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5a48cda640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c03f740 0x7f5a3c030ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5a48cda640 1 -- 172.21.15.17:0/508078527 >> v1:172.21.15.163:6789/0 conn(0x7f5a3c049f40 legacy=0x7f5a3c031410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5a48cda640 1 -- 172.21.15.17:0/508078527 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0d7720 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5802837640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e80074f0 0x7f57e800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5801835640 1 -- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e8005bf0 msgr2=0x7f57e8006010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:23.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f90c9150640 1 -- 172.21.15.17:0/4124124179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02cb10 con 0x7f90c414f150 2024-03-06T07:20:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f90c9150640 1 -- 172.21.15.17:0/4124124179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc02ccb0 con 0x7f90c414f150 2024-03-06T07:20:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5801835640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e8005bf0 0x7f57e8006010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 -- 172.21.15.17:0/755535394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 msgr2=0x7f14e415a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 --2- 172.21.15.17:0/755535394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009870 tx=0x7f14d802b860 comp rx=0 tx=0).stop 2024-03-06T07:20:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 -- 172.21.15.17:0/755535394 shutdown_connections 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 --2- 172.21.15.17:0/755535394 >> v2:172.21.15.163:3300/0 conn(0x7f14e40c6110 0x7f14e40c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 --2- 172.21.15.17:0/755535394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4073360 0x7f14e415a250 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009870 tx=0x7f14d802b860 comp rx=0 tx=0).stop 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f14ebb79640 1 -- 172.21.15.17:0/755535394 >> 172.21.15.17:0/755535394 conn(0x7f14e40a8900 msgr2=0x7f14e40a8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5801835640 1 -- 172.21.15.17:0/3922726214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e80074f0 msgr2=0x7f57e800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5801835640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e80074f0 0x7f57e800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.696+0000 7f5801835640 1 -- 172.21.15.17:0/3922726214 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0c1910 con 0x7f57e8006a10 2024-03-06T07:20:23.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f1dd1b87640 1 -- 172.21.15.17:0/2850802085 shutdown_connections 2024-03-06T07:20:23.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f14ebb79640 1 -- 172.21.15.17:0/755535394 shutdown_connections 2024-03-06T07:20:23.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f14ebb79640 1 -- 172.21.15.17:0/755535394 wait complete. 2024-03-06T07:20:23.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 -- 172.21.15.17:0/3687590829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f522015cd60 msgr2=0x7f522015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3687590829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f522015cd60 0x7f522015d140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f521c009870 tx=0x7f521c02b910 comp rx=0 tx=0).stop 2024-03-06T07:20:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5802837640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e80074f0 0x7f57e800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f90c9150640 1 -- 172.21.15.17:0/4124124179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02ce60 con 0x7f90c414f150 2024-03-06T07:20:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 -- 172.21.15.17:0/3687590829 shutdown_connections 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3687590829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c6560 0x7f52200c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3687590829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f522015cd60 0x7f522015d140 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 -- 172.21.15.17:0/3687590829 >> 172.21.15.17:0/3687590829 conn(0x7f52200c78f0 msgr2=0x7f52200c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f35b56af640 1 -- 172.21.15.17:0/3893852279 shutdown_connections 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f35b56af640 1 --2- 172.21.15.17:0/3893852279 >> v2:172.21.15.163:3300/0 conn(0x7f35b00c0320 0x7f35b015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f35b56af640 1 --2- 172.21.15.17:0/3893852279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b015ce20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f35b56af640 1 -- 172.21.15.17:0/3893852279 >> 172.21.15.17:0/3893852279 conn(0x7f35b00cae70 msgr2=0x7f35b00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f1dd1b87640 1 -- 172.21.15.17:0/2850802085 wait complete. 2024-03-06T07:20:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f4cce3ac640 1 Processor -- start 2024-03-06T07:20:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 -- 172.21.15.17:0/3687590829 shutdown_connections 2024-03-06T07:20:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5226cd4640 1 -- 172.21.15.17:0/3687590829 wait complete. 2024-03-06T07:20:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5a48cda640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a44050b10 tx=0x7f5a44069930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6767d8fc08371eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f90cc3b6640 1 -- 172.21.15.17:0/4124124179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c414f150 msgr2=0x7f90c4158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/4124124179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c414f150 0x7f90c4158760 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009e50 tx=0x7f90bc02bf70 comp rx=0 tx=0).stop 2024-03-06T07:20:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5801835640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57e8006e90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57ec007fd0 tx=0x7f57e8007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b89de1f82b5ee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f4cce3ac640 1 -- start start 2024-03-06T07:20:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5a437fe640 1 -- 172.21.15.17:0/508078527 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440b4070 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5a437fe640 1 -- 172.21.15.17:0/508078527 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a440a7b40 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f35b56af640 1 -- 172.21.15.17:0/3893852279 shutdown_connections 2024-03-06T07:20:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f35b56af640 1 -- 172.21.15.17:0/3893852279 wait complete. 2024-03-06T07:20:23.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5a437fe640 1 -- 172.21.15.17:0/508078527 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a440af440 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007d50 con 0x7f57e8006a10 2024-03-06T07:20:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.697+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec005ad0 con 0x7f57e8006a10 2024-03-06T07:20:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f5801034640 1 -- 172.21.15.17:0/3922726214 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007070 con 0x7f57e8006a10 2024-03-06T07:20:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f14ebb79640 1 Processor -- start 2024-03-06T07:20:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f14ebb79640 1 -- start start 2024-03-06T07:20:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f4cce3ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dad0 0x7f4cc0035350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f4cce3ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f5803038640 1 -- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 msgr2=0x7f57e8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f4cccba9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cccba9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38272/0 (socket says 172.21.15.17:38272) 2024-03-06T07:20:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cccba9640 1 -- 172.21.15.17:0/2988835635 learned_addr learned my addr 172.21.15.17:0/2988835635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cce3ac640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc00d7ad0 0x7f4cc01777e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f5803038640 1 --2- 172.21.15.17:0/3922726214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57e8006e90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f57ec007fd0 tx=0x7f57e8007a40 comp rx=0 tx=0).stop 2024-03-06T07:20:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cce3ac640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00db260 con 0x7f4cc004dfe0 2024-03-06T07:20:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cce3ac640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00dad40 con 0x7f4cc004dad0 2024-03-06T07:20:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 msgr2=0x7f5a3c0469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4cce3ac640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc004e4a0 con 0x7f4cc00d7ad0 2024-03-06T07:20:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5a44050b10 tx=0x7f5a44069930 comp rx=0 tx=0).stop 2024-03-06T07:20:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.698+0000 7f5802837640 1 -- 172.21.15.17:0/3922726214 reap_dead start 2024-03-06T07:20:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5803038640 1 -- 172.21.15.17:0/3922726214 shutdown_connections 2024-03-06T07:20:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5803038640 1 -- 172.21.15.17:0/3922726214 >> 172.21.15.17:0/3922726214 conn(0x7f57fc0a82b0 msgr2=0x7f57fc0a8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f1dd1b87640 1 Processor -- start 2024-03-06T07:20:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f1dd1b87640 1 -- start start 2024-03-06T07:20:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f4ccd3aa640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dad0 0x7f4cc0035350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5803038640 1 -- 172.21.15.17:0/3922726214 shutdown_connections 2024-03-06T07:20:23.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 shutdown_connections 2024-03-06T07:20:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c03f740 0x7f5a3c030ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c6110 0x7f14e41553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/508078527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0469c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4156140 0x7f14e4155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c4560 con 0x7f14e40c3850 2024-03-06T07:20:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c43e0 con 0x7f14e40c6110 2024-03-06T07:20:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ebb79640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c3c30 con 0x7f14e4156140 2024-03-06T07:20:23.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 >> 172.21.15.17:0/508078527 conn(0x7f5a3c048ac0 msgr2=0x7f5a3c048ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5226cd4640 1 Processor -- start 2024-03-06T07:20:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14e9915640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5226cd4640 1 -- start start 2024-03-06T07:20:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14e9915640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38282/0 (socket says 172.21.15.17:38282) 2024-03-06T07:20:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14e9915640 1 -- 172.21.15.17:0/2747682666 learned_addr learned my addr 172.21.15.17:0/2747682666 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14ea116640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4156140 0x7f14e4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f14e9114640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c6110 0x7f14e41553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f5803038640 1 -- 172.21.15.17:0/3922726214 wait complete. 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 shutdown_connections 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f5a49cdc640 1 -- 172.21.15.17:0/508078527 wait complete. 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 -- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc00d7ad0 msgr2=0x7f4cc01777e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc00d7ad0 0x7f4cc01777e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 -- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dad0 msgr2=0x7f4cc0035350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dad0 0x7f4cc0035350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc0033d40 con 0x7f4cc004dfe0 2024-03-06T07:20:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 -- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4156140 msgr2=0x7f14e4155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4156140 0x7f14e4155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 -- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c6110 msgr2=0x7f14e41553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c6110 0x7f14e41553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 -- 172.21.15.17:0/2747682666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40a3270 con 0x7f14e40c3850 2024-03-06T07:20:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 0x7f1dc40dffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14e9915640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14e000b440 tx=0x7f14e000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f4cccba9640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80696e0 tx=0x7f4cc8069930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 0x7f1dc41782c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 0x7f1dc416b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd0b85640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 0x7f1dc40dffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd0b85640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 0x7f1dc40dffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52442/0 (socket says 172.21.15.17:52442) 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1386640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 0x7f1dc416b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1386640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 0x7f1dc416b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60122/0 (socket says 172.21.15.17:60122) 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc40dc3c0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc40096a0 con 0x7f1dc40dbfe0 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd1b87640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc403eeb0 con 0x7f1dc4006ef0 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dd0b85640 1 -- 172.21.15.17:0/943790844 learned_addr learned my addr 172.21.15.17:0/943790844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 0x7f1dc41782c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.700+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0015020 con 0x7f14e40c3850 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0011890 con 0x7f14e40c3850 2024-03-06T07:20:23.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80bf020 con 0x7f4cc004dfe0 2024-03-06T07:20:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80ba730 con 0x7f4cc004dfe0 2024-03-06T07:20:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0011c80 con 0x7f14e40c3850 2024-03-06T07:20:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f4cce3ac640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc0035890 con 0x7f4cc004dfe0 2024-03-06T07:20:23.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f5226cd4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f35b56af640 1 Processor -- start 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f35b56af640 1 -- start start 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00acdd0 0x7f35b01fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b0153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015d8a0 con 0x7f35b00acdd0 2024-03-06T07:20:23.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0159570 con 0x7f35b00c0320 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35b56af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015da20 con 0x7f35b00c6d30 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60136/0 (socket says 172.21.15.17:60136) 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3359571013 learned_addr learned my addr 172.21.15.17:0/3359571013 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f4cce3ac640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc00e5070 con 0x7f4cc004dfe0 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35af7fe640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b0153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f5224a70640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f5803038640 1 Processor -- start 2024-03-06T07:20:23.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f5224a70640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52464/0 (socket says 172.21.15.17:52464) 2024-03-06T07:20:23.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f5224a70640 1 -- 172.21.15.17:0/3891973306 learned_addr learned my addr 172.21.15.17:0/3891973306 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 msgr2=0x7f35b0153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b0153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3359571013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00acdd0 msgr2=0x7f35b01fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f14ebb79640 1 -- 172.21.15.17:0/2747682666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e41f0130 con 0x7f14e40c3850 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00acdd0 0x7f35b01fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f35ae7fc640 1 -- 172.21.15.17:0/3359571013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00ac3e0 con 0x7f35b00c0320 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f14ebb79640 1 -- 172.21.15.17:0/2747682666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e40a9f60 con 0x7f14e40c3850 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f1dc3fff640 1 -- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 msgr2=0x7f1dc40dffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 0x7f1dc40dffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f1dc3fff640 1 -- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 msgr2=0x7f1dc416b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.701+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 0x7f1dc416b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f1dc3fff640 1 -- 172.21.15.17:0/943790844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4174340 con 0x7f1dc400b0d0 2024-03-06T07:20:23.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52200c6560 0x7f52200c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f90cc3b6640 1 -- 172.21.15.17:0/4124124179 shutdown_connections 2024-03-06T07:20:23.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/4124124179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c414f150 0x7f90c4158760 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.699+0000 7f90cc3b6640 1 -- 172.21.15.17:0/4124124179 >> 172.21.15.17:0/4124124179 conn(0x7f90c406b1a0 msgr2=0x7f90c406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:23.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f90cc3b6640 1 -- 172.21.15.17:0/4124124179 shutdown_connections 2024-03-06T07:20:23.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f522015cd60 0x7f52201e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f35ae7fc640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35a802c590 tx=0x7f35a8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200c1670 con 0x7f52200c6560 2024-03-06T07:20:23.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52200c14f0 con 0x7f522015cd60 2024-03-06T07:20:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52200c9990 con 0x7f52200c0e70 2024-03-06T07:20:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5225271640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f522015cd60 0x7f52201e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f90cc3b6640 1 -- 172.21.15.17:0/4124124179 wait complete. 2024-03-06T07:20:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5a49cdc640 1 Processor -- start 2024-03-06T07:20:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80b4040 con 0x7f4cc004dfe0 2024-03-06T07:20:23.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.702+0000 7f5a49cdc640 1 -- start start 2024-03-06T07:20:23.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5224a70640 1 -- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f522015cd60 msgr2=0x7f52201e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5224a70640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f522015cd60 0x7f52201e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5224a70640 1 -- 172.21.15.17:0/3891973306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52200c6560 msgr2=0x7f52200c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5224a70640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52200c6560 0x7f52200c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5224a70640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200c4410 con 0x7f52200c0e70 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8043020 con 0x7f35b00c0320 2024-03-06T07:20:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f1dc3fff640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 0x7f1dc41782c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc033000 tx=0x7f1dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f35b56af640 1 -- 172.21.15.17:0/3359571013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01fb7e0 con 0x7f35b00c0320 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f35b56af640 1 -- 172.21.15.17:0/3359571013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b00ae110 con 0x7f35b00c0320 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a803e450 con 0x7f35b00c0320 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 0x7f5a3c045ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0310e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c00b490 con 0x7f5a3c03f740 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c00b1b0 con 0x7f5a3c0cd2b0 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a49cdc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c00c220 con 0x7f5a3c049f40 2024-03-06T07:20:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e404f330 con 0x7f14e40c3850 2024-03-06T07:20:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52480/0 (socket says 172.21.15.17:52480) 2024-03-06T07:20:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 -- 172.21.15.17:0/1155689091 learned_addr learned my addr 172.21.15.17:0/1155689091 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f90cc3b6640 1 Processor -- start 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a494db640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.703+0000 7f5803038640 1 -- start start 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57fc152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 0x7f57fc0a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57fc1f2990 0x7f57fc1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0abd00 con 0x7f57fc1f2990 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0c1ed0 con 0x7f57fc072df0 2024-03-06T07:20:23.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5803038640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0abe80 con 0x7f57e8006a10 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a48cda640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 0x7f5a3c045ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 -- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 msgr2=0x7f5a3c0310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 -- 172.21.15.17:0/1155689091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 msgr2=0x7f5a3c045ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 0x7f5a3c045ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 -- 172.21.15.17:0/1155689091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0435c0 con 0x7f5a3c049f40 2024-03-06T07:20:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a48cda640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 0x7f5a3c045ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5224a70640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f521c0090b0 tx=0x7f521c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5802036640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57fc152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5802036640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57fc152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52482/0 (socket says 172.21.15.17:52482) 2024-03-06T07:20:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5802036640 1 -- 172.21.15.17:0/1696408090 learned_addr learned my addr 172.21.15.17:0/1696408090 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a494db640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14e0018020 con 0x7f14e40c3850 2024-03-06T07:20:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0317c0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4169130 con 0x7f1dc400b0d0 2024-03-06T07:20:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc416a380 con 0x7f1dc400b0d0 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0327e0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc032bf0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f5a43fff640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800f900 tx=0x7f5a3800fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38019070 con 0x7f5a3c049f40 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800dc50 con 0x7f5a3c049f40 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800b040 con 0x7f5a3c049f40 2024-03-06T07:20:23.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c16fcd0 con 0x7f5a3c049f40 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c16f9f0 con 0x7f5a3c049f40 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f14d67fc640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 0x7f14c806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f14e008a780 con 0x7f14e40c3850 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5802837640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57fc1f2990 0x7f57fc1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 0x7f57fc0a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 -- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 msgr2=0x7f57fc152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57fc152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 -- 172.21.15.17:0/1696408090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57fc1f2990 msgr2=0x7f57fc1f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57fc1f2990 0x7f57fc1f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f5801835640 1 -- 172.21.15.17:0/1696408090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0a5210 con 0x7f57fc072df0 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a803e840 con 0x7f35b00c0320 2024-03-06T07:20:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c003a20 con 0x7f52200c0e70 2024-03-06T07:20:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35a8046020 con 0x7f35b00c0320 2024-03-06T07:20:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f5801835640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 0x7f57fc0a3e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57ec035000 tx=0x7f57ec001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f35b56af640 1 -- 172.21.15.17:0/3359571013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3590005290 con 0x7f35b00c0320 2024-03-06T07:20:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f14e9114640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 0x7f14c806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec045070 con 0x7f57fc072df0 2024-03-06T07:20:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc00057d0 con 0x7f4cc004dfe0 2024-03-06T07:20:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.706+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c003bc0 con 0x7f52200c0e70 2024-03-06T07:20:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52200c46f0 con 0x7f52200c0e70 2024-03-06T07:20:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cc80c2020 con 0x7f4cc004dfe0 2024-03-06T07:20:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.704+0000 7f90cc3b6640 1 -- start start 2024-03-06T07:20:23.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c40a8ce0 0x7f90c4149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c40aac30 0x7f90c40c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4155520 con 0x7f90c40aac30 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40ab630 con 0x7f90c414f150 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.705+0000 7f90cc3b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40ab910 con 0x7f90c40a8ce0 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60162/0 (socket says 172.21.15.17:60162) 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 -- 172.21.15.17:0/1672472978 learned_addr learned my addr 172.21.15.17:0/1672472978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90c9951640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c40aac30 0x7f90c40c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 -- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c40a8ce0 msgr2=0x7f90c4149d20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-06T07:20:23.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c40a8ce0 0x7f90c4149d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 -- 172.21.15.17:0/1672472978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c40aac30 msgr2=0x7f90c40c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c40aac30 0x7f90c40c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 -- 172.21.15.17:0/1672472978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c4072140 con 0x7f90c414f150 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f90ca953640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c3c0 tx=0x7f90b400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc0a54f0 con 0x7f57fc072df0 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc1f3380 con 0x7f57fc072df0 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec011370 con 0x7f57fc072df0 2024-03-06T07:20:23.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec001630 con 0x7f57fc072df0 2024-03-06T07:20:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac005290 con 0x7f1dc400b0d0 2024-03-06T07:20:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c003a20 con 0x7f52200c0e70 2024-03-06T07:20:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52200af6c0 con 0x7f52200c0e70 2024-03-06T07:20:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f4cbdffb640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 0x7f4cac06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f35977fe640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 0x7f359c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f35a80acca0 con 0x7f35b00c0320 2024-03-06T07:20:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4cc8127b10 con 0x7f4cc004dfe0 2024-03-06T07:20:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.707+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c00bf50 con 0x7f5a3c049f40 2024-03-06T07:20:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a3801c070 con 0x7f5a3c049f40 2024-03-06T07:20:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f4ccd3aa640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 0x7f4cac06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e4005290 con 0x7f57fc072df0 2024-03-06T07:20:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f35aeffd640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 0x7f359c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f14e9114640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 0x7f14c806d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14d8036000 tx=0x7f14d803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5a417fa640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 0x7f5a2806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522004f330 con 0x7f52200c0e70 2024-03-06T07:20:23.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5a48cda640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 0x7f5a2806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.708+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a3808a1c0 con 0x7f5a3c049f40 2024-03-06T07:20:23.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f4ccd3aa640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 0x7f4cac06da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4007fd0 tx=0x7f4cc4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f35aeffd640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 0x7f359c06da90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000f60 tx=0x7f35a00079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f521c038020 con 0x7f52200c0e70 2024-03-06T07:20:23.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f5a48cda640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 0x7f5a2806da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007ac0 tx=0x7f5a440ab000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f52157fa640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 0x7f520c06dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f521c003820 con 0x7f52200c0e70 2024-03-06T07:20:23.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.709+0000 7f5217fff640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 0x7f520c06dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.710+0000 7f5217fff640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 0x7f520c06dd90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5220159d40 tx=0x7f5210006c40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.711+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14e001f020 con 0x7f14e40c3850 2024-03-06T07:20:23.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.712+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40052d0 con 0x7f90c414f150 2024-03-06T07:20:23.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.712+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b4005470 con 0x7f90c414f150 2024-03-06T07:20:23.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.712+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b4007240 con 0x7f90c414f150 2024-03-06T07:20:23.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.712+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35a804d030 con 0x7f35b00c0320 2024-03-06T07:20:23.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.712+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c41f7960 con 0x7f90c414f150 2024-03-06T07:20:23.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.713+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c41f7f20 con 0x7f90c414f150 2024-03-06T07:20:23.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.713+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1dbc04c060 con 0x7f1dc400b0d0 2024-03-06T07:20:23.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.713+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a38058d70 con 0x7f5a3c049f40 2024-03-06T07:20:23.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.713+0000 7f1dc17fa640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 0x7f1db806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.713+0000 7f1dd0b85640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 0x7f1db806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57ec007070 con 0x7f57fc072df0 2024-03-06T07:20:23.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f57f2ffd640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 0x7f57e006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f1dd0b85640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 0x7f1db806da10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0645f0 tx=0x7f1dcc068d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac005290 con 0x7f90c414f150 2024-03-06T07:20:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f57ec0abe70 con 0x7f57fc072df0 2024-03-06T07:20:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.714+0000 7f5802036640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 0x7f57e006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.715+0000 7f5802036640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 0x7f57e006d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006fd0 tx=0x7f57f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.717+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57ec005aa0 con 0x7f57fc072df0 2024-03-06T07:20:23.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.718+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f521c07a500 con 0x7f52200c0e70 2024-03-06T07:20:23.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.719+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90b40073e0 con 0x7f90c414f150 2024-03-06T07:20:23.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.719+0000 7f90baffd640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 0x7f90a806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:23.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.719+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f90b408a270 con 0x7f90c414f150 2024-03-06T07:20:23.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.720+0000 7f90ca152640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 0x7f90a806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.721+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1dbc04fdf0 con 0x7f1dc400b0d0 2024-03-06T07:20:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.721+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4cc80c9030 con 0x7f4cc004dfe0 2024-03-06T07:20:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.721+0000 7f90ca152640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 0x7f90a806da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90bc035000 tx=0x7f90bc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:23.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.722+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90b4058d90 con 0x7f90c414f150 2024-03-06T07:20:23.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:23.730+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1dbc040070 con 0x7f1dc400b0d0 2024-03-06T07:20:24.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.132+0000 7f35b56af640 1 -- 172.21.15.17:0/3359571013 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3590005580 con 0x7f35b00c0320 2024-03-06T07:20:24.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.133+0000 7f35977fe640 1 -- 172.21.15.17:0/3359571013 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35a807b850 con 0x7f35b00c0320 2024-03-06T07:20:24.134 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 msgr2=0x7f359c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 0x7f359c06da90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000f60 tx=0x7f35a00079d0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 msgr2=0x7f35b00c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35a802c590 tx=0x7f35a8004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 shutdown_connections 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f359c06b5d0 0x7f359c06da90 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000f60 tx=0x7f35a00079d0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35b00c6d30 0x7f35b0153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.136+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35b00c0320 0x7f35b00c4060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.137+0000 7f35957fa640 1 --2- 172.21.15.17:0/3359571013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35b00acdd0 0x7f35b01fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.137+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 >> 172.21.15.17:0/3359571013 conn(0x7f35b00cae70 msgr2=0x7f35b00cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.137+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 shutdown_connections 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.137+0000 7f35957fa640 1 -- 172.21.15.17:0/3359571013 wait complete. 2024-03-06T07:20:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.137+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5a3c0d4c00 con 0x7f5a3c049f40 2024-03-06T07:20:24.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.138+0000 7f5a417fa640 1 -- 172.21.15.17:0/1155689091 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5a38058670 con 0x7f5a3c049f40 2024-03-06T07:20:24.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.139+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f14e406e700 con 0x7f14e40c3850 2024-03-06T07:20:24.140 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:24.142 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.139+0000 7f14d67fc640 1 -- 172.21.15.17:0/2747682666 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14e000d040 con 0x7f14e40c3850 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 msgr2=0x7f5a2806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 0x7f5a2806da60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c007ac0 tx=0x7f5a440ab000 comp rx=0 tx=0).stop 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 msgr2=0x7f5a3c043080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800f900 tx=0x7f5a3800fdc0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 shutdown_connections 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.141+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a2806b5a0 0x7f5a2806da60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a3c0cd2b0 0x7f5a3c0310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a3c049f40 0x7f5a3c043080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 --2- 172.21.15.17:0/1155689091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a3c03f740 0x7f5a3c045ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 >> 172.21.15.17:0/1155689091 conn(0x7f5a3c048ac0 msgr2=0x7f5a3c0468c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 shutdown_connections 2024-03-06T07:20:24.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.142+0000 7f5a49cdc640 1 -- 172.21.15.17:0/1155689091 wait complete. 2024-03-06T07:20:24.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 msgr2=0x7f14c806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 0x7f14c806d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14d8036000 tx=0x7f14d803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 msgr2=0x7f14e40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14e000b440 tx=0x7f14e000b900 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 shutdown_connections 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b520 0x7f14c806d9e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.143+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4156140 0x7f14e4155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.144+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c6110 0x7f14e41553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.144+0000 7f14c7fff640 1 --2- 172.21.15.17:0/2747682666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e40c3850 0x7f14e40a3a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.144+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 >> 172.21.15.17:0/2747682666 conn(0x7f14e40a8900 msgr2=0x7f14e41e2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.144+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 shutdown_connections 2024-03-06T07:20:24.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.144+0000 7f14c7fff640 1 -- 172.21.15.17:0/2747682666 wait complete. 2024-03-06T07:20:24.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.150+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f90ac005850 con 0x7f90c414f150 2024-03-06T07:20:24.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.150+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f52200b0780 con 0x7f52200c0e70 2024-03-06T07:20:24.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.150+0000 7f90baffd640 1 -- 172.21.15.17:0/1672472978 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90b4058690 con 0x7f90c414f150 2024-03-06T07:20:24.153 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:24.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.151+0000 7f52157fa640 1 -- 172.21.15.17:0/3891973306 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f521c07e5a0 con 0x7f52200c0e70 2024-03-06T07:20:24.153 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.153+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1dac005850 con 0x7f1dc400b0d0 2024-03-06T07:20:24.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.153+0000 7f1dc17fa640 1 -- 172.21.15.17:0/943790844 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1dbc07ae60 con 0x7f1dc400b0d0 2024-03-06T07:20:24.155 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:24.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 msgr2=0x7f90a806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 0x7f90a806da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90bc035000 tx=0x7f90bc030000 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 msgr2=0x7f90c41fae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c3c0 tx=0x7f90b400c880 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 shutdown_connections 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b5a0 0x7f90a806da60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90c414f150 0x7f90c41fae30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c40aac30 0x7f90c40c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 --2- 172.21.15.17:0/1672472978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90c40a8ce0 0x7f90c4149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 >> 172.21.15.17:0/1672472978 conn(0x7f90c406b1a0 msgr2=0x7f90c4154700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 msgr2=0x7f520c06dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 shutdown_connections 2024-03-06T07:20:24.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 0x7f520c06dd90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5220159d40 tx=0x7f5210006c40 comp rx=0 tx=0).stop 2024-03-06T07:20:24.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f90cc3b6640 1 -- 172.21.15.17:0/1672472978 wait complete. 2024-03-06T07:20:24.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 msgr2=0x7f52200af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.154+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f521c0090b0 tx=0x7f521c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 shutdown_connections 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f520c06b8d0 0x7f520c06dd90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f522015cd60 0x7f52201e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52200c6560 0x7f52200c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 --2- 172.21.15.17:0/3891973306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52200c0e70 0x7f52200af180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 >> 172.21.15.17:0/3891973306 conn(0x7f52200c78f0 msgr2=0x7f52200c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 shutdown_connections 2024-03-06T07:20:24.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.155+0000 7f5226cd4640 1 -- 172.21.15.17:0/3891973306 wait complete. 2024-03-06T07:20:24.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.157+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 msgr2=0x7f1db806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.157+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 0x7f1db806da10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0645f0 tx=0x7f1dcc068d50 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.157+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 msgr2=0x7f1dc41782c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.157+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 0x7f1dc41782c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc033000 tx=0x7f1dbc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 shutdown_connections 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1db806b550 0x7f1db806da10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dc40dbfe0 0x7f1dc416b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dc400b0d0 0x7f1dc41782c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 --2- 172.21.15.17:0/943790844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dc4006ef0 0x7f1dc40dffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 >> 172.21.15.17:0/943790844 conn(0x7f1dc4005e90 msgr2=0x7f1dc4006270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 shutdown_connections 2024-03-06T07:20:24.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.158+0000 7f1dd1b87640 1 -- 172.21.15.17:0/943790844 wait complete. 2024-03-06T07:20:24.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-03-06T07:20:24.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-03-06T07:20:24.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.167+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f57e4005850 con 0x7f57fc072df0 2024-03-06T07:20:24.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.167+0000 7f57f2ffd640 1 -- 172.21.15.17:0/1696408090 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f57ec07a3d0 con 0x7f57fc072df0 2024-03-06T07:20:24.169 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 msgr2=0x7f57e006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 0x7f57e006d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006fd0 tx=0x7f57f8008040 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 msgr2=0x7f57fc0a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 0x7f57fc0a3e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57ec035000 tx=0x7f57ec001970 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 shutdown_connections 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57e006b520 0x7f57e006d9e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57fc1f2990 0x7f57fc1f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57fc072df0 0x7f57fc0a3e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 --2- 172.21.15.17:0/1696408090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e8006a10 0x7f57fc152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 >> 172.21.15.17:0/1696408090 conn(0x7f57fc0a82b0 msgr2=0x7f57fc150dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 shutdown_connections 2024-03-06T07:20:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.171+0000 7f5803038640 1 -- 172.21.15.17:0/1696408090 wait complete. 2024-03-06T07:20:24.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-03-06T07:20:24.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2024-03-06T07:20:24.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-03-06T07:20:24.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.180+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4cc000fb90 con 0x7f4cc004dfe0 2024-03-06T07:20:24.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.181+0000 7f4cbdffb640 1 -- 172.21.15.17:0/2988835635 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4cc80f6640 con 0x7f4cc004dfe0 2024-03-06T07:20:24.183 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:24.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 msgr2=0x7f4cac06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 0x7f4cac06da60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4007fd0 tx=0x7f4cc4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 msgr2=0x7f4cc00dd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80696e0 tx=0x7f4cc8069930 comp rx=0 tx=0).stop 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 shutdown_connections 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cac06b5a0 0x7f4cac06da60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cc00d7ad0 0x7f4cc01777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cc004dfe0 0x7f4cc00dd1e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 --2- 172.21.15.17:0/2988835635 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cc004dad0 0x7f4cc0035350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:24.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 >> 172.21.15.17:0/2988835635 conn(0x7f4cc004c650 msgr2=0x7f4cc0006770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:24.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 shutdown_connections 2024-03-06T07:20:24.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:24.183+0000 7f4cb37fe640 1 -- 172.21.15.17:0/2988835635 wait complete. 2024-03-06T07:20:24.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-03-06T07:20:25.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-06T07:20:25.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-06T07:20:25.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-06T07:20:25.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-06T07:20:25.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-06T07:20:25.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-06T07:20:25.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-06T07:20:25.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-06T07:20:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.327+0000 7feb3e21f640 1 Processor -- start 2024-03-06T07:20:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.327+0000 7feb3e21f640 1 -- start start 2024-03-06T07:20:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.328+0000 7f9b8eb5e640 1 Processor -- start 2024-03-06T07:20:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.328+0000 7fe0ea98a640 1 Processor -- start 2024-03-06T07:20:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.328+0000 7f9b8eb5e640 1 -- start start 2024-03-06T07:20:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.328+0000 7feb3e21f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb30004b00 0x7feb300d7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7f9b8eb5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b88151cc0 0x7f9b880c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7f9b8eb5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880aad60 con 0x7f9b880c20d0 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7f9b8eb5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880c2560 con 0x7f9b8806b6d0 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7f9b8eb5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880a95e0 con 0x7f9b88151cc0 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7f9b8c8fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b88151cc0 0x7f9b880c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7f9b8c8fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b88151cc0 0x7f9b880c27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38342/0 (socket says 172.21.15.17:38342) 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/4122079934 learned_addr learned my addr 172.21.15.17:0/4122079934 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/4122079934 >> v1:172.21.15.163:6789/0 conn(0x7f9b8806b6d0 legacy=0x7f9b8806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/4122079934 >> v1:172.21.15.17:6789/0 conn(0x7f9b880c20d0 legacy=0x7f9b880c2cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:20:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7fe0ea98a640 1 -- start start 2024-03-06T07:20:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0ea98a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0dc0dbea0 0x7fe0dc00d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3e21f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3e21f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb30007050 con 0x7feb300465a0 2024-03-06T07:20:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3e21f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb30006e00 con 0x7feb30004b00 2024-03-06T07:20:25.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0ea98a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0dc00d030 0x7fe0dc0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3e21f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb30007330 con 0x7feb300d3ab0 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52506/0 (socket says 172.21.15.17:52506) 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 -- 172.21.15.17:0/1479876447 learned_addr learned my addr 172.21.15.17:0/1479876447 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3d21d640 1 --2- 172.21.15.17:0/1479876447 >> v2:172.21.15.163:3300/0 conn(0x7feb30004b00 0x7feb300d7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 -- 172.21.15.17:0/1479876447 >> v2:172.21.15.163:3300/0 conn(0x7feb30004b00 msgr2=0x7feb300d7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 --2- 172.21.15.17:0/1479876447 >> v2:172.21.15.163:3300/0 conn(0x7feb30004b00 0x7feb300d7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 -- 172.21.15.17:0/1479876447 >> v1:172.21.15.17:6789/0 conn(0x7feb300465a0 legacy=0x7feb300d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.329+0000 7feb3ca1c640 1 -- 172.21.15.17:0/1479876447 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300d6510 con 0x7feb300d3ab0 2024-03-06T07:20:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7feb3ca1c640 1 --2- 172.21.15.17:0/1479876447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb38050b10 tx=0x7feb38069930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94777c758f3c7e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0ea98a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0dc280 con 0x7fe0dc040f20 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0ea98a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc009bf0 con 0x7fe0dc00d030 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0ea98a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc0dea10 con 0x7fe0dc0dbea0 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0e37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0dc00d030 0x7fe0dc0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0e37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0dc00d030 0x7fe0dc0409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38358/0 (socket says 172.21.15.17:38358) 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7fe0e37fe640 1 -- 172.21.15.17:0/2760470887 learned_addr learned my addr 172.21.15.17:0/2760470887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7fe0e3fff640 1 --2- 172.21.15.17:0/2760470887 >> v2:172.21.15.163:3300/0 conn(0x7fe0dc0dbea0 0x7fe0dc00d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7fe0e37fe640 1 -- 172.21.15.17:0/2760470887 >> v2:172.21.15.163:3300/0 conn(0x7fe0dc0dbea0 msgr2=0x7fe0dc00d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7fe0e37fe640 1 --2- 172.21.15.17:0/2760470887 >> v2:172.21.15.163:3300/0 conn(0x7fe0dc0dbea0 0x7fe0dc00d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7fe0e37fe640 1 -- 172.21.15.17:0/2760470887 >> v1:172.21.15.17:6789/0 conn(0x7fe0dc040f20 legacy=0x7fe0dc0e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7fe0e37fe640 1 -- 172.21.15.17:0/2760470887 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0decf0 con 0x7fe0dc00d030 2024-03-06T07:20:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.330+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/4122079934 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88155210 con 0x7f9b88151cc0 2024-03-06T07:20:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.331+0000 7feb2ffff640 1 -- 172.21.15.17:0/1479876447 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380b4070 con 0x7feb300d3ab0 2024-03-06T07:20:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.332+0000 7feb2ffff640 1 -- 172.21.15.17:0/1479876447 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380a7bb0 con 0x7feb300d3ab0 2024-03-06T07:20:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.332+0000 7feb2ffff640 1 -- 172.21.15.17:0/1479876447 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380af660 con 0x7feb300d3ab0 2024-03-06T07:20:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.332+0000 7f9b8c8fa640 1 --2- 172.21.15.17:0/4122079934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b880c27b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b880a8e20 tx=0x7f9b7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5574ef558fab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.332+0000 7feb3e21f640 1 -- 172.21.15.17:0/1479876447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 msgr2=0x7feb300d6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.332+0000 7feb3e21f640 1 --2- 172.21.15.17:0/1479876447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb38050b10 tx=0x7feb38069930 comp rx=0 tx=0).stop 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7feb3e21f640 1 -- 172.21.15.17:0/1479876447 shutdown_connections 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7feb3e21f640 1 --2- 172.21.15.17:0/1479876447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb300d6d10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7feb3e21f640 1 --2- 172.21.15.17:0/1479876447 >> v2:172.21.15.163:3300/0 conn(0x7feb30004b00 0x7feb300d7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7feb3e21f640 1 -- 172.21.15.17:0/1479876447 >> 172.21.15.17:0/1479876447 conn(0x7feb300062e0 msgr2=0x7feb300066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7f9b877fe640 1 -- 172.21.15.17:0/4122079934 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c034070 con 0x7f9b88151cc0 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7f9b877fe640 1 -- 172.21.15.17:0/4122079934 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c030840 con 0x7f9b88151cc0 2024-03-06T07:20:25.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7f9b877fe640 1 -- 172.21.15.17:0/4122079934 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c030c10 con 0x7f9b88151cc0 2024-03-06T07:20:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7fe0e37fe640 1 --2- 172.21.15.17:0/2760470887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0409e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc041480 tx=0x7fe0d802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cde375e8e940aea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7fe0e2ffd640 1 -- 172.21.15.17:0/2760470887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8038070 con 0x7fe0dc00d030 2024-03-06T07:20:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.333+0000 7fe0e2ffd640 1 -- 172.21.15.17:0/2760470887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d802f050 con 0x7fe0dc00d030 2024-03-06T07:20:25.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7fe0e2ffd640 1 -- 172.21.15.17:0/2760470887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8031970 con 0x7fe0dc00d030 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/4122079934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 msgr2=0x7f9b880c27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/4122079934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b880c27b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b880a8e20 tx=0x7f9b7c02b980 comp rx=0 tx=0).stop 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/4122079934 shutdown_connections 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/4122079934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b880c27b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/4122079934 >> 172.21.15.17:0/4122079934 conn(0x7f9b88067120 msgr2=0x7f9b8806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7feb3e21f640 1 -- 172.21.15.17:0/1479876447 shutdown_connections 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7feb3e21f640 1 -- 172.21.15.17:0/1479876447 wait complete. 2024-03-06T07:20:25.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7fe0ea98a640 1 -- 172.21.15.17:0/2760470887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 msgr2=0x7fe0dc0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.334+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/2760470887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0409e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc041480 tx=0x7fe0d802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/4122079934 shutdown_connections 2024-03-06T07:20:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.336+0000 7feb3e21f640 1 Processor -- start 2024-03-06T07:20:25.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.336+0000 7feb3e21f640 1 -- start start 2024-03-06T07:20:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/4122079934 wait complete. 2024-03-06T07:20:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7f9b8eb5e640 1 Processor -- start 2024-03-06T07:20:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7f9b8eb5e640 1 -- start start 2024-03-06T07:20:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8eb5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 -- 172.21.15.17:0/2760470887 shutdown_connections 2024-03-06T07:20:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8eb5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b88143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8eb5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b88154490 0x7f9b88152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/2760470887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0409e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8c8fa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8c8fa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52532/0 (socket says 172.21.15.17:52532) 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/809462478 learned_addr learned my addr 172.21.15.17:0/809462478 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8d0fb640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b88154490 0x7f9b88152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881520a0 con 0x7f9b88151cc0 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880a98c0 con 0x7f9b88154490 2024-03-06T07:20:25.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880c1a50 con 0x7f9b8806b6d0 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/2760470887 >> v2:172.21.15.163:3300/0 conn(0x7fe0dc0dbea0 0x7fe0dc00d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 -- 172.21.15.17:0/2760470887 >> 172.21.15.17:0/2760470887 conn(0x7fe0dc02f5c0 msgr2=0x7fe0dc02f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 -- 172.21.15.17:0/2760470887 shutdown_connections 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.335+0000 7fe0ea98a640 1 -- 172.21.15.17:0/2760470887 wait complete. 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.336+0000 7fe0ea98a640 1 Processor -- start 2024-03-06T07:20:25.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7fe0ea98a640 1 -- start start 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b88154490 msgr2=0x7f9b88152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8c8fa640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b88154490 0x7f9b88152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/809462478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 msgr2=0x7f9b88143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b87fff640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b88143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0dc040f20 0x7fe0dc046ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 0x7fe0dc04c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0e25e0 con 0x7fe0dc00d030 2024-03-06T07:20:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8c8fa640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b88143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7f9b8c8fa640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b88146580 con 0x7f9b8806b6d0 2024-03-06T07:20:25.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0df2b0 con 0x7fe0dc040f20 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 0x7feb3003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb30175140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300d7250 con 0x7feb30004b00 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300d3f10 con 0x7feb300465a0 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3e21f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb30009a60 con 0x7feb300d3ab0 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3ca1c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.337+0000 7feb3ca1c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60194/0 (socket says 172.21.15.17:60194) 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7feb3ca1c640 1 -- 172.21.15.17:0/71007353 learned_addr learned my addr 172.21.15.17:0/71007353 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7feb3da1e640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb30175140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7feb3d21d640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 0x7feb3003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7feb3ca1c640 1 -- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 msgr2=0x7feb30175140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7feb3ca1c640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb30175140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7feb3ca1c640 1 -- 172.21.15.17:0/71007353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 msgr2=0x7feb3003b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7feb3ca1c640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 0x7feb3003b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7feb3ca1c640 1 -- 172.21.15.17:0/71007353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb301785f0 con 0x7feb300465a0 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0ea98a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0e2760 con 0x7fe0dc0dbea0 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38366/0 (socket says 172.21.15.17:38366) 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0e3fff640 1 -- 172.21.15.17:0/962098931 learned_addr learned my addr 172.21.15.17:0/962098931 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.338+0000 7fe0e8f27640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 0x7fe0dc04c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 -- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 msgr2=0x7fe0dc04c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 0x7fe0dc04c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 -- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0dc040f20 msgr2=0x7fe0dc046ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0dc040f20 0x7fe0dc046ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 -- 172.21.15.17:0/962098931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc047870 con 0x7fe0dc00d030 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e8f27640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 0x7fe0dc04c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.339+0000 7fe0e3fff640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000d800 tx=0x7fe0d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7f9b8c8fa640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c034040 con 0x7f9b8806b6d0 2024-03-06T07:20:25.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c039d90 con 0x7f9b8806b6d0 2024-03-06T07:20:25.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0018020 con 0x7fe0dc00d030 2024-03-06T07:20:25.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7feb3d21d640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 0x7feb3003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:25.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc031160 con 0x7fe0dc00d030 2024-03-06T07:20:25.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc031400 con 0x7fe0dc00d030 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d00096c0 con 0x7fe0dc00d030 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.340+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0018020 con 0x7fe0dc00d030 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7feb3ca1c640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb38069df0 tx=0x7feb38069a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b88146860 con 0x7f9b8806b6d0 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c043630 con 0x7f9b8806b6d0 2024-03-06T07:20:25.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b881400a0 con 0x7f9b8806b6d0 2024-03-06T07:20:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380bf020 con 0x7feb300465a0 2024-03-06T07:20:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.341+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380ba670 con 0x7feb300465a0 2024-03-06T07:20:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.342+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb380b4040 con 0x7feb300465a0 2024-03-06T07:20:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.343+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d001b020 con 0x7fe0dc00d030 2024-03-06T07:20:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.343+0000 7fe0e0ff9640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 0x7fe0d406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.343+0000 7feb3e21f640 1 -- 172.21.15.17:0/71007353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb301788d0 con 0x7feb300465a0 2024-03-06T07:20:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.343+0000 7feb3e21f640 1 -- 172.21.15.17:0/71007353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb30174940 con 0x7feb300465a0 2024-03-06T07:20:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.344+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b7c04a060 con 0x7f9b8806b6d0 2024-03-06T07:20:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.344+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe0d000f040 con 0x7fe0dc00d030 2024-03-06T07:20:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.344+0000 7fe0e37fe640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 0x7fe0d406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.344+0000 7f9b857fa640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 0x7f9b6c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.344+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9b7c0ac530 con 0x7f9b8806b6d0 2024-03-06T07:20:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.345+0000 7f9b87fff640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 0x7f9b6c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.345+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c4005290 con 0x7fe0dc00d030 2024-03-06T07:20:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.345+0000 7fe0e37fe640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 0x7fe0d406ded0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8035000 tx=0x7fe0d8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.345+0000 7f9b87fff640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 0x7f9b6c06da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b8814f270 tx=0x7f9b74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.346+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb380c2020 con 0x7feb300465a0 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.347+0000 7feb2dffb640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 0x7feb1c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.347+0000 7feb3d21d640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 0x7feb1c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.348+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feb38127890 con 0x7feb300465a0 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.348+0000 7feb3d21d640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 0x7feb1c06da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb34006fd0 tx=0x7feb34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.348+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b880693c0 con 0x7f9b8806b6d0 2024-03-06T07:20:25.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.348+0000 7feb3e21f640 1 -- 172.21.15.17:0/71007353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb300057e0 con 0x7feb300465a0 2024-03-06T07:20:25.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.353+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb380c9020 con 0x7feb300465a0 2024-03-06T07:20:25.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.354+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0d0022020 con 0x7fe0dc00d030 2024-03-06T07:20:25.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.360+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b7c03e070 con 0x7f9b8806b6d0 2024-03-06T07:20:25.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.382+0000 7fc553284640 1 Processor -- start 2024-03-06T07:20:25.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.382+0000 7fc553284640 1 -- start start 2024-03-06T07:20:25.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.383+0000 7fc553284640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c0c1a00 0x7fc54c0c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.383+0000 7fc553284640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc54c06e730 0x7fc54c0a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.383+0000 7fc553284640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c154890 con 0x7fc54c072b30 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.383+0000 7fc553284640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c154b70 con 0x7fc54c06e730 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.383+0000 7fc553284640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c154e50 con 0x7fc54c0c1a00 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.384+0000 7fc551821640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc54c072b30 0x7fc54c0a9e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58398/0 (socket says 172.21.15.17:58398) 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.384+0000 7fc551821640 1 -- 172.21.15.17:0/2068469389 learned_addr learned my addr 172.21.15.17:0/2068469389 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.384+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947606198 0 0) 0x7fc54c154890 con 0x7fc54c072b30 2024-03-06T07:20:25.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.384+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc538003520 con 0x7fc54c072b30 2024-03-06T07:20:25.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.384+0000 7f3f916be640 1 Processor -- start 2024-03-06T07:20:25.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7f3f916be640 1 -- start start 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954105732 0 0) 0x7fc538003520 con 0x7fc54c072b30 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c0c1a00 msgr2=0x7fc54c0c2540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c0c1a00 0x7fc54c0c2540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 >> v2:172.21.15.163:3300/0 conn(0x7fc54c06e730 msgr2=0x7fc54c0a9930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 --2- 172.21.15.17:0/2068469389 >> v2:172.21.15.163:3300/0 conn(0x7fc54c06e730 0x7fc54c0a9930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c155130 con 0x7fc54c072b30 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc53c001670 con 0x7fc54c072b30 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 >> v1:172.21.15.17:6789/0 conn(0x7fc54c072b30 legacy=0x7fc54c0a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380039b0 0x7fc538003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc538004800 0x7fc538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5380053c0 0x7fc53800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c154b70 con 0x7fc5380039b0 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c154e50 con 0x7fc538004800 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.385+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53800c260 con 0x7fc5380053c0 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7fc551821640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5380053c0 0x7fc53800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7fc551020640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380039b0 0x7fc538003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7fc55081f640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc538004800 0x7fc538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f916be640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f84047490 0x7f3f84006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f916be640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 0x7f3f8400a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f916be640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8400bb40 con 0x7f3f840dbcd0 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f916be640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8400a410 con 0x7f3f840073d0 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f916be640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f84048050 con 0x7f3f84047490 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f84047490 0x7f3f84006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f84047490 0x7f3f84006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60234/0 (socket says 172.21.15.17:60234) 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 -- 172.21.15.17:0/1969339976 learned_addr learned my addr 172.21.15.17:0/1969339976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f90ebd640 1 --2- 172.21.15.17:0/1969339976 >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 0x7f3f8400a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 -- 172.21.15.17:0/1969339976 >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 msgr2=0x7f3f8400a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 --2- 172.21.15.17:0/1969339976 >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 0x7f3f8400a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 -- 172.21.15.17:0/1969339976 >> v1:172.21.15.17:6789/0 conn(0x7f3f840dbcd0 legacy=0x7f3f840df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.386+0000 7f3f8b7fe640 1 -- 172.21.15.17:0/1969339976 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8400a0e0 con 0x7f3f84047490 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.387+0000 7fc55081f640 1 -- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5380053c0 msgr2=0x7fc53800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.387+0000 7fc55081f640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5380053c0 0x7fc53800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.387+0000 7fc55081f640 1 -- 172.21.15.17:0/2068469389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380039b0 msgr2=0x7fc538003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.387+0000 7fc55081f640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5380039b0 0x7fc538003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.387+0000 7fc55081f640 1 -- 172.21.15.17:0/2068469389 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c155130 con 0x7fc538004800 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7fc55081f640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc538004800 0x7fc538004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5480097d0 tx=0x7fc538005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8515d59ca4a128cb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548037070 con 0x7fc538004800 2024-03-06T07:20:25.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548032600 con 0x7fc538004800 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc543fff640 1 -- 172.21.15.17:0/2068469389 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480093d0 con 0x7fc538004800 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc553284640 1 -- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc538004800 msgr2=0x7fc538004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc553284640 1 --2- 172.21.15.17:0/2068469389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc538004800 0x7fc538004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc5480097d0 tx=0x7fc538005830 comp rx=0 tx=0).stop 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7f3f90ebd640 1 --2- 172.21.15.17:0/1969339976 >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 0x7f3f8400a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7f3f8b7fe640 1 --2- 172.21.15.17:0/1969339976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f84006e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f84046bd0 tx=0x7f3f7802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0c32a7b972a8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7f3f8affd640 1 -- 172.21.15.17:0/1969339976 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78034070 con 0x7f3f84047490 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7f3f8affd640 1 -- 172.21.15.17:0/1969339976 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78030610 con 0x7f3f84047490 2024-03-06T07:20:25.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.388+0000 7f3f8affd640 1 -- 172.21.15.17:0/1969339976 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78030a20 con 0x7f3f84047490 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc551821640 1 -- 172.21.15.17:0/2068469389 reap_dead start 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc553284640 1 -- 172.21.15.17:0/2068469389 shutdown_connections 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7fc553284640 1 -- 172.21.15.17:0/2068469389 >> 172.21.15.17:0/2068469389 conn(0x7fc54c06b1a0 msgr2=0x7fc54c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7f3f916be640 1 -- 172.21.15.17:0/1969339976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 msgr2=0x7f3f84006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.389+0000 7f3f916be640 1 --2- 172.21.15.17:0/1969339976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f84006e90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3f84046bd0 tx=0x7f3f7802b960 comp rx=0 tx=0).stop 2024-03-06T07:20:25.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7f144b9eb640 1 Processor -- start 2024-03-06T07:20:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7fc553284640 1 -- 172.21.15.17:0/2068469389 shutdown_connections 2024-03-06T07:20:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7fc553284640 1 -- 172.21.15.17:0/2068469389 wait complete. 2024-03-06T07:20:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7f58f1ec2640 1 Processor -- start 2024-03-06T07:20:25.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f58f1ec2640 1 -- start start 2024-03-06T07:20:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fc553284640 1 Processor -- start 2024-03-06T07:20:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7f144b9eb640 1 -- start start 2024-03-06T07:20:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f144b9eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14440c7160 0x7f144415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f144b9eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f144415d6e0 0x7f14440c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7fa7d759e640 1 Processor -- start 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.390+0000 7fa7d759e640 1 -- start start 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fa7d759e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa7d003f730 0x7fa7d0030ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fa7d759e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d0031b10 con 0x7fa7d00cd2a0 2024-03-06T07:20:25.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fa7d759e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00cd680 con 0x7fa7d0049f30 2024-03-06T07:20:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fa7d759e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d000b7f0 con 0x7fa7d003f730 2024-03-06T07:20:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa7d00cd2a0 0x7fa7d00469b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58426/0 (socket says 172.21.15.17:58426) 2024-03-06T07:20:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58f1ec2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d659c640 1 -- 172.21.15.17:0/3625389533 learned_addr learned my addr 172.21.15.17:0/3625389533 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:25.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d5d9b640 1 --2- 172.21.15.17:0/3625389533 >> v2:172.21.15.17:3300/0 conn(0x7fa7d003f730 0x7fa7d0030ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f3f916be640 1 -- 172.21.15.17:0/1969339976 shutdown_connections 2024-03-06T07:20:25.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f3f916be640 1 --2- 172.21.15.17:0/1969339976 >> v2:172.21.15.17:3300/0 conn(0x7f3f840073d0 0x7f3f8400a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f3f916be640 1 --2- 172.21.15.17:0/1969339976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f84006e90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7f3f916be640 1 -- 172.21.15.17:0/1969339976 >> 172.21.15.17:0/1969339976 conn(0x7f3f84048240 msgr2=0x7f3f84048650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52552/0 (socket says 172.21.15.17:52552) 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1612918829 learned_addr learned my addr 172.21.15.17:0/1612918829 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58eaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58ec0c0fa0 0x7f58ec0c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58434/0 (socket says 172.21.15.17:58434) 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1612918829 >> v2:172.21.15.17:3300/0 conn(0x7f58ec06d990 0x7f58ec06de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.391+0000 7fc553284640 1 -- start start 2024-03-06T07:20:25.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f144b9eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144415db70 con 0x7f14440c6600 2024-03-06T07:20:25.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f144b9eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440c2ed0 con 0x7f144415d6e0 2024-03-06T07:20:25.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f144b9eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1444159800 con 0x7f14440c7160 2024-03-06T07:20:25.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ec070550 con 0x7f58ec0c0fa0 2024-03-06T07:20:25.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ec06e360 con 0x7f58ec06d990 2024-03-06T07:20:25.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fc553284640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 0x7fc54c144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fc553284640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 0x7fc54c145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc553284640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 0x7fc54c1f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc553284640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0aa580 con 0x7fc54c1f2750 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc553284640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1556f0 con 0x7fc54c141b30 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc553284640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c1de0 con 0x7fc54c072240 2024-03-06T07:20:25.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec070830 con 0x7f58ec072f00 2024-03-06T07:20:25.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551020640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 0x7fc54c144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551020640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 0x7fc54c144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60248/0 (socket says 172.21.15.17:60248) 2024-03-06T07:20:25.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551020640 1 -- 172.21.15.17:0/1796737862 learned_addr learned my addr 172.21.15.17:0/1796737862 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 0x7fc54c1f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f1448f86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f144415d6e0 0x7f14440c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f1448f86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f144415d6e0 0x7f14440c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38408/0 (socket says 172.21.15.17:38408) 2024-03-06T07:20:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f1448f86640 1 -- 172.21.15.17:0/3087305723 learned_addr learned my addr 172.21.15.17:0/3087305723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc55081f640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 0x7fc54c145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d559a640 1 -- 172.21.15.17:0/3625389533 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025751312 0 0) 0x7fa7d0031b10 con 0x7fa7d00cd2a0 2024-03-06T07:20:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d559a640 1 -- 172.21.15.17:0/3625389533 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c0005730 con 0x7fa7d00cd2a0 2024-03-06T07:20:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/3625389533 >> v1:172.21.15.163:6789/0 conn(0x7fa7d0049f30 legacy=0x7fa7d0031400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/3625389533 >> v1:172.21.15.17:6789/0 conn(0x7fa7d00cd2a0 legacy=0x7fa7d00469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.392+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/3625389533 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00d7710 con 0x7fa7d003f730 2024-03-06T07:20:25.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f3f916be640 1 -- 172.21.15.17:0/1969339976 shutdown_connections 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 -- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 msgr2=0x7fc54c145330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 0x7fc54c145330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 -- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 msgr2=0x7fc54c144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 0x7fc54c144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc551821640 1 -- 172.21.15.17:0/1796737862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c071790 con 0x7fc54c1f2750 2024-03-06T07:20:25.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7fc55081f640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 0x7fc54c145330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1612918829 >> v2:172.21.15.17:3300/0 conn(0x7f58ec06d990 msgr2=0x7f58ec06de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f58eb7fe640 1 --2- 172.21.15.17:0/1612918829 >> v2:172.21.15.17:3300/0 conn(0x7f58ec06d990 0x7f58ec06de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1612918829 >> v1:172.21.15.17:6789/0 conn(0x7f58ec0c0fa0 legacy=0x7f58ec0c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1612918829 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec070b10 con 0x7f58ec072f00 2024-03-06T07:20:25.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.393+0000 7f3f916be640 1 -- 172.21.15.17:0/1969339976 wait complete. 2024-03-06T07:20:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7fa7d5d9b640 1 --2- 172.21.15.17:0/3625389533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d0030ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800a8c0 tx=0x7fa7c802fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5082a4ae2143e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f1448f86640 1 -- 172.21.15.17:0/3087305723 >> v2:172.21.15.163:3300/0 conn(0x7f14440c7160 msgr2=0x7f144415a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-06T07:20:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f1448f86640 1 --2- 172.21.15.17:0/3087305723 >> v2:172.21.15.163:3300/0 conn(0x7f14440c7160 0x7f144415a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f1448f86640 1 -- 172.21.15.17:0/3087305723 >> v1:172.21.15.163:6789/0 conn(0x7f14440c6600 legacy=0x7f14440c4300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-06T07:20:25.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.394+0000 7f1448f86640 1 -- 172.21.15.17:0/3087305723 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444159ae0 con 0x7f144415d6e0 2024-03-06T07:20:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fa7d559a640 1 -- 172.21.15.17:0/3625389533 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c803d070 con 0x7fa7d003f730 2024-03-06T07:20:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fa7d559a640 1 -- 172.21.15.17:0/3625389533 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8030980 con 0x7fa7d003f730 2024-03-06T07:20:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fa7d559a640 1 -- 172.21.15.17:0/3625389533 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8030d50 con 0x7fa7d003f730 2024-03-06T07:20:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f58eb7fe640 1 --2- 172.21.15.17:0/1612918829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009870 tx=0x7f58dc02b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80efdfef04f057e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fc551821640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 0x7fc54c1f2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc54c072f10 tx=0x7fc53c007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f58ea7fc640 1 -- 172.21.15.17:0/1612918829 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc037070 con 0x7f58ec072f00 2024-03-06T07:20:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f58ea7fc640 1 -- 172.21.15.17:0/1612918829 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc030710 con 0x7f58ec072f00 2024-03-06T07:20:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f58ea7fc640 1 -- 172.21.15.17:0/1612918829 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc030ae0 con 0x7f58ec072f00 2024-03-06T07:20:25.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f3f916be640 1 Processor -- start 2024-03-06T07:20:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f1448f86640 1 --2- 172.21.15.17:0/3087305723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f144415d6e0 0x7f14440c60c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f144406c060 tx=0x7f143402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfa4930bdcfa8c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7f3f916be640 1 -- start start 2024-03-06T07:20:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c01a070 con 0x7fc54c1f2750 2024-03-06T07:20:25.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.395+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00f890 con 0x7fc54c1f2750 2024-03-06T07:20:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7fa7d759e640 1 -- 172.21.15.17:0/3625389533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 msgr2=0x7fa7d0030ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7fa7d759e640 1 --2- 172.21.15.17:0/3625389533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d0030ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800a8c0 tx=0x7fa7c802fbd0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00fc60 con 0x7fc54c1f2750 2024-03-06T07:20:25.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c071a70 con 0x7fc54c1f2750 2024-03-06T07:20:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7f143bfff640 1 -- 172.21.15.17:0/3087305723 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434035020 con 0x7f144415d6e0 2024-03-06T07:20:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 msgr2=0x7f58ec153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1612918829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009870 tx=0x7f58dc02b7d0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 shutdown_connections 2024-03-06T07:20:25.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1612918829 >> v2:172.21.15.17:3300/0 conn(0x7f58ec06d990 0x7f58ec06de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f916be640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1612918829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec072f00 0x7f58ec153990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fa7d759e640 1 -- 172.21.15.17:0/3625389533 shutdown_connections 2024-03-06T07:20:25.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fa7d759e640 1 --2- 172.21.15.17:0/3625389533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d0030ec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fa7d759e640 1 -- 172.21.15.17:0/3625389533 >> 172.21.15.17:0/3625389533 conn(0x7fa7d0048ab0 msgr2=0x7fa7d0048ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f916be640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f840d6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.396+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c141f10 con 0x7fc54c1f2750 2024-03-06T07:20:25.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f916be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 0x7f3f840d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 >> 172.21.15.17:0/1612918829 conn(0x7f58ec06b1a0 msgr2=0x7f58ec06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 shutdown_connections 2024-03-06T07:20:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c04f330 con 0x7fc54c1f2750 2024-03-06T07:20:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1612918829 wait complete. 2024-03-06T07:20:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f143bfff640 1 -- 172.21.15.17:0/3087305723 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434031800 con 0x7f144415d6e0 2024-03-06T07:20:25.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f143bfff640 1 -- 172.21.15.17:0/3087305723 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434031bd0 con 0x7f144415d6e0 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fa7d759e640 1 -- 172.21.15.17:0/3625389533 shutdown_connections 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7fa7d759e640 1 -- 172.21.15.17:0/3625389533 wait complete. 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 -- 172.21.15.17:0/3087305723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f144415d6e0 msgr2=0x7f14440c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 --2- 172.21.15.17:0/3087305723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f144415d6e0 0x7f14440c60c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f144406c060 tx=0x7f143402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 -- 172.21.15.17:0/3087305723 shutdown_connections 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 --2- 172.21.15.17:0/3087305723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f144415d6e0 0x7f14440c60c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 --2- 172.21.15.17:0/3087305723 >> v2:172.21.15.163:3300/0 conn(0x7f14440c7160 0x7f144415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f144b9eb640 1 -- 172.21.15.17:0/3087305723 >> 172.21.15.17:0/3087305723 conn(0x7f144406c700 msgr2=0x7f14440c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7fa7d759e640 1 Processor -- start 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f8bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52568/0 (socket says 172.21.15.17:52568) 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f8bfff640 1 -- 172.21.15.17:0/1070659096 learned_addr learned my addr 172.21.15.17:0/1070659096 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f90ebd640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 0x7f3f840d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.397+0000 7f3f8b7fe640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f840d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840dc0b0 con 0x7f3f840dbcd0 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84047d70 con 0x7f3f84047490 2024-03-06T07:20:25.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840dfbe0 con 0x7f3f840073d0 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f8bfff640 1 -- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 msgr2=0x7f3f840d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f8bfff640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f840d6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f8bfff640 1 -- 172.21.15.17:0/1070659096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 msgr2=0x7f3f840d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.398+0000 7f3f8bfff640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 0x7f3f840d67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7f3f8bfff640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8402e880 con 0x7f3f840073d0 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7f3f90ebd640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 0x7f3f840d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7fa7d759e640 1 -- start start 2024-03-06T07:20:25.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7f58f1ec2640 1 Processor -- start 2024-03-06T07:20:25.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7f3f8bfff640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c480 tx=0x7f3f8000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f144b9eb640 1 -- 172.21.15.17:0/3087305723 shutdown_connections 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80017070 con 0x7f3f840073d0 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f800052d0 con 0x7f3f840073d0 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84178290 con 0x7f3f840073d0 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f58f1ec2640 1 -- start start 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7fa7d759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d00d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d759e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc53c01d070 con 0x7fc54c1f2750 2024-03-06T07:20:25.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.399+0000 7fc541ffb640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 0x7fc53006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc53c089cc0 con 0x7fc54c1f2750 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d759e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7d00cd2a0 0x7fa7d00435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d759e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000b480 con 0x7fa7d003f730 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d759e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000b1a0 con 0x7fa7d0049f30 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d759e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000c210 con 0x7fa7d00cd2a0 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d5d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d6d9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7d00cd2a0 0x7fa7d00435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d5d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60270/0 (socket says 172.21.15.17:60270) 2024-03-06T07:20:25.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/1307630339 learned_addr learned my addr 172.21.15.17:0/1307630339 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.400+0000 7f144b9eb640 1 -- 172.21.15.17:0/3087305723 wait complete. 2024-03-06T07:20:25.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7f144b9eb640 1 Processor -- start 2024-03-06T07:20:25.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58f1ec2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f144b9eb640 1 -- start start 2024-03-06T07:20:25.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80003830 con 0x7f3f840073d0 2024-03-06T07:20:25.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58f1ec2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ec072f00 0x7f58ec155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.401+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f84178570 con 0x7f3f840073d0 2024-03-06T07:20:25.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58eb7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60274/0 (socket says 172.21.15.17:60274) 2024-03-06T07:20:25.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1943316980 learned_addr learned my addr 172.21.15.17:0/1943316980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.402+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec0c0fa0 0x7f58ec1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0c70d0 con 0x7f58ec072f00 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f58ebfff640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec0c0fa0 0x7f58ec1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec0c6f50 con 0x7f58ec06d990 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f8001a070 con 0x7f3f840073d0 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f144b9eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 0x7f14441e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f144b9eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec070df0 con 0x7f58ec0c0fa0 2024-03-06T07:20:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f144b9eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 0x7f14441e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7fc551020640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 0x7fc53006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8402ee30 con 0x7f3f840073d0 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f3f88ff9640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 0x7f3f7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7fa7d659c640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d00d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7d00cd2a0 msgr2=0x7fa7d00435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7fa7d5d9b640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7d00cd2a0 0x7fa7d00435b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/1307630339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 msgr2=0x7fa7d00d8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f8008e060 con 0x7f3f840073d0 2024-03-06T07:20:25.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f3f8b7fe640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 0x7f3f7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7d5d9b640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d00d8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7d5d9b640 1 -- 172.21.15.17:0/1307630339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00d36e0 con 0x7fa7d0049f30 2024-03-06T07:20:25.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7d659c640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d00d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f1448f86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f1448f86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38500/0 (socket says 172.21.15.17:38500) 2024-03-06T07:20:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f1448f86640 1 -- 172.21.15.17:0/2069123039 learned_addr learned my addr 172.21.15.17:0/2069123039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c4b90 con 0x7f14440c7160 2024-03-06T07:20:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.403+0000 7f58eaffd640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ec072f00 0x7f58ec155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f1449f88640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 0x7f14441e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec0c0fa0 msgr2=0x7f58ec1555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fc551020640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 0x7fc53006da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5440070b0 tx=0x7fc544010040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7d5d9b640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800a130 tx=0x7fa7c80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c4a10 con 0x7f144415d6e0 2024-03-06T07:20:25.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c7540 con 0x7f14440c6600 2024-03-06T07:20:25.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f58eb7fe640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec0c0fa0 0x7f58ec1555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1943316980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ec072f00 msgr2=0x7f58ec155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c803d040 con 0x7fa7d0049f30 2024-03-06T07:20:25.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f58eb7fe640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ec072f00 0x7f58ec155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f1449787640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 0x7f14441e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d002bcb0 con 0x7fa7d0049f30 2024-03-06T07:20:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00d93e0 con 0x7fa7d0049f30 2024-03-06T07:20:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.404+0000 7f58eb7fe640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec14dfc0 con 0x7f58ec06d990 2024-03-06T07:20:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8038b70 con 0x7fa7d0049f30 2024-03-06T07:20:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8043430 con 0x7fa7d0049f30 2024-03-06T07:20:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f1448f86640 1 -- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 msgr2=0x7f14441e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f58eb7fe640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0090b0 tx=0x7f58dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc041070 con 0x7f58ec06d990 2024-03-06T07:20:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc032a40 con 0x7f58ec06d990 2024-03-06T07:20:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.405+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc037040 con 0x7f58ec06d990 2024-03-06T07:20:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f3f8b7fe640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 0x7f3f7006d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f7803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec1ead60 con 0x7f58ec06d990 2024-03-06T07:20:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec156fe0 con 0x7f58ec06d990 2024-03-06T07:20:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f1448f86640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 0x7f14441e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f1448f86640 1 -- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 msgr2=0x7f14441e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f1448f86640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 0x7f14441e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f1448f86640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14441f2dc0 con 0x7f14440c7160 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7f1449787640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 0x7f14441e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.407+0000 7f1449f88640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 0x7f14441e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.406+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc005290 con 0x7fa7d0049f30 2024-03-06T07:20:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.407+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7c80435d0 con 0x7fa7d0049f30 2024-03-06T07:20:25.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.407+0000 7f1448f86640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14340090b0 tx=0x7f14340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.407+0000 7fa7c6ffd640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 0x7fa7b806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa7c80b0440 con 0x7fa7d0049f30 2024-03-06T07:20:25.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc53c058220 con 0x7fc54c1f2750 2024-03-06T07:20:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7fa7d659c640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 0x7fa7b806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec04f330 con 0x7f58ec06d990 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340395c0 con 0x7f14440c7160 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441f30a0 con 0x7f14440c7160 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143403ae50 con 0x7f14440c7160 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440a37c0 con 0x7f14440c7160 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.409+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340399c0 con 0x7f14440c7160 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.409+0000 7fa7d659c640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 0x7fa7b806d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000f60 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.408+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58dc044070 con 0x7f58ec06d990 2024-03-06T07:20:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.409+0000 7f58d3fff640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.409+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f58dc0abee0 con 0x7f58ec06d990 2024-03-06T07:20:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.410+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144404f330 con 0x7f14440c7160 2024-03-06T07:20:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.410+0000 7f58eaffd640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.410+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1434038020 con 0x7f14440c7160 2024-03-06T07:20:25.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.410+0000 7f58eaffd640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 0x7f58d806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0c42d0 tx=0x7f58e0007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.410+0000 7f1439ffb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 0x7f142806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.411+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f14340ac610 con 0x7f14440c7160 2024-03-06T07:20:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.411+0000 7f1449787640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 0x7f142806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.411+0000 7f1449787640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 0x7f142806da10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1440006fd0 tx=0x7f14400090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.412+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7c807efa0 con 0x7fa7d0049f30 2024-03-06T07:20:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.413+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f800580a0 con 0x7f3f840073d0 2024-03-06T07:20:25.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.417+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58dc04b070 con 0x7f58ec06d990 2024-03-06T07:20:25.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.420+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f143403f070 con 0x7f14440c7160 2024-03-06T07:20:25.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.751+0000 7feb3e21f640 1 -- 172.21.15.17:0/71007353 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7feb30032c40 con 0x7feb300465a0 2024-03-06T07:20:25.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.751+0000 7feb2dffb640 1 -- 172.21.15.17:0/71007353 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb380f63b0 con 0x7feb300465a0 2024-03-06T07:20:25.753 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:25.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.753+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe0c4005850 con 0x7fe0dc00d030 2024-03-06T07:20:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.753+0000 7fe0e0ff9640 1 -- 172.21.15.17:0/962098931 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0dc047870 con 0x7fe0dc00d030 2024-03-06T07:20:25.755 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 msgr2=0x7feb1c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 0x7feb1c06da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb34006fd0 tx=0x7feb34008040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 msgr2=0x7feb30177da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feb38069df0 tx=0x7feb38069a40 comp rx=0 tx=0).stop 2024-03-06T07:20:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 shutdown_connections 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b5a0 0x7feb1c06da60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb300d3ab0 0x7feb30175140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb300465a0 0x7feb30177da0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 --2- 172.21.15.17:0/71007353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30004b00 0x7feb3003b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 >> 172.21.15.17:0/71007353 conn(0x7feb300062e0 msgr2=0x7feb3002bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 shutdown_connections 2024-03-06T07:20:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.755+0000 7feb1b7fe640 1 -- 172.21.15.17:0/71007353 wait complete. 2024-03-06T07:20:25.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 msgr2=0x7fe0d406ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 0x7fe0d406ded0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8035000 tx=0x7fe0d8008030 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 msgr2=0x7fe0dc0442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000d800 tx=0x7fe0d000dcc0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 shutdown_connections 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.757+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0d406ba10 0x7fe0d406ded0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0dc0dbea0 0x7fe0dc04c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0dc040f20 0x7fe0dc046ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 --2- 172.21.15.17:0/962098931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0dc00d030 0x7fe0dc0442e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 >> 172.21.15.17:0/962098931 conn(0x7fe0dc02f5c0 msgr2=0x7fe0dc0059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 shutdown_connections 2024-03-06T07:20:25.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.758+0000 7fe0ea98a640 1 -- 172.21.15.17:0/962098931 wait complete. 2024-03-06T07:20:25.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-03-06T07:20:25.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.774+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9b88143d40 con 0x7f9b8806b6d0 2024-03-06T07:20:25.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.774+0000 7f9b857fa640 1 -- 172.21.15.17:0/809462478 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b7c07b0e0 con 0x7f9b8806b6d0 2024-03-06T07:20:25.776 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-06T07:20:25.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-03-06T07:20:25.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.777+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 msgr2=0x7f9b6c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.777+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 0x7f9b6c06da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9b8814f270 tx=0x7f9b74009040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.777+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 msgr2=0x7f9b881408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.777+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 shutdown_connections 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b6c06b5a0 0x7f9b6c06da60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b88154490 0x7f9b88152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88151cc0 0x7f9b88143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 --2- 172.21.15.17:0/809462478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b8806b6d0 0x7f9b881408a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 >> 172.21.15.17:0/809462478 conn(0x7f9b88067120 msgr2=0x7f9b8806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 shutdown_connections 2024-03-06T07:20:25.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.778+0000 7f9b8eb5e640 1 -- 172.21.15.17:0/809462478 wait complete. 2024-03-06T07:20:25.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-03-06T07:20:25.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.802+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f58ec0abf40 con 0x7f58ec06d990 2024-03-06T07:20:25.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.802+0000 7f58d3fff640 1 -- 172.21.15.17:0/1943316980 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58dc07a3c0 con 0x7f58ec06d990 2024-03-06T07:20:25.804 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 msgr2=0x7f58d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 0x7f58d806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0c42d0 tx=0x7f58e0007e00 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 msgr2=0x7f58ec13dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0090b0 tx=0x7f58dc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 shutdown_connections 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58ec0c0fa0 0x7f58ec1555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ec072f00 0x7f58ec155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 --2- 172.21.15.17:0/1943316980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58ec06d990 0x7f58ec13dbb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 >> 172.21.15.17:0/1943316980 conn(0x7f58ec06b1a0 msgr2=0x7f58ec151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 shutdown_connections 2024-03-06T07:20:25.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.805+0000 7f58f1ec2640 1 -- 172.21.15.17:0/1943316980 wait complete. 2024-03-06T07:20:25.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.807+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa7bc005580 con 0x7fa7d0049f30 2024-03-06T07:20:25.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.807+0000 7fa7c6ffd640 1 -- 172.21.15.17:0/1307630339 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7c807e8a0 con 0x7fa7d0049f30 2024-03-06T07:20:25.809 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:25.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 msgr2=0x7fa7b806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 0x7fa7b806d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000f60 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 msgr2=0x7fa7d0043070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800a130 tx=0x7fa7c80046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 shutdown_connections 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa7b806b520 0x7fa7b806d9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7d00cd2a0 0x7fa7d00435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7d0049f30 0x7fa7d0043070 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 --2- 172.21.15.17:0/1307630339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7d003f730 0x7fa7d00d8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 >> 172.21.15.17:0/1307630339 conn(0x7fa7d0048ab0 msgr2=0x7fa7d00d5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 shutdown_connections 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.811+0000 7fa7d759e640 1 -- 172.21.15.17:0/1307630339 wait complete. 2024-03-06T07:20:25.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.812+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc54c0a8f90 con 0x7fc54c1f2750 2024-03-06T07:20:25.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.812+0000 7fc541ffb640 1 -- 172.21.15.17:0/1796737862 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc53c05c2c0 con 0x7fc54c1f2750 2024-03-06T07:20:25.815 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:25.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 msgr2=0x7fc53006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 0x7fc53006da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc5440070b0 tx=0x7fc544010040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 msgr2=0x7fc54c1f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 0x7fc54c1f2bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc54c072f10 tx=0x7fc53c007660 comp rx=0 tx=0).stop 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 shutdown_connections 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc53006b5d0 0x7fc53006da90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc54c1f2750 0x7fc54c1f2bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc54c072240 0x7fc54c145330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 --2- 172.21.15.17:0/1796737862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc54c141b30 0x7fc54c144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.814+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 >> 172.21.15.17:0/1796737862 conn(0x7fc54c06b1a0 msgr2=0x7fc54c0c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.815+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 shutdown_connections 2024-03-06T07:20:25.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.817+0000 7fc553284640 1 -- 172.21.15.17:0/1796737862 wait complete. 2024-03-06T07:20:25.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-03-06T07:20:25.820 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:25.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-03-06T07:20:25.826 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:25.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-03-06T07:20:25.830 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.837+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3f84005380 con 0x7f3f840073d0 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.837+0000 7f3f88ff9640 1 -- 172.21.15.17:0/1070659096 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f800579a0 con 0x7f3f840073d0 2024-03-06T07:20:25.839 INFO:teuthology.orchestra.run.smithi017.stdout:55834574856 2024-03-06T07:20:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 msgr2=0x7f3f7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 0x7f3f7006d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3f78036000 tx=0x7f3f7803b040 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 msgr2=0x7f3f840e0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c480 tx=0x7f3f8000c880 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 shutdown_connections 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7006b500 0x7f3f7006d9c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f840dbcd0 0x7f3f840d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f84047490 0x7f3f840d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 --2- 172.21.15.17:0/1070659096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f840073d0 0x7f3f840e0bd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 >> 172.21.15.17:0/1070659096 conn(0x7f3f84048240 msgr2=0x7f3f84048620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 shutdown_connections 2024-03-06T07:20:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.840+0000 7f3f916be640 1 -- 172.21.15.17:0/1070659096 wait complete. 2024-03-06T07:20:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.842+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1444159430 con 0x7f14440c7160 2024-03-06T07:20:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.842+0000 7f1439ffb640 1 -- 172.21.15.17:0/2069123039 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f144415db70 con 0x7f14440c7160 2024-03-06T07:20:25.844 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 msgr2=0x7f142806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 0x7f142806da10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1440006fd0 tx=0x7f14400090e0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 msgr2=0x7f14440a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14340090b0 tx=0x7f14340046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 shutdown_connections 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142806b550 0x7f142806da10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144415d6e0 0x7f14441e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14440c7160 0x7f14440a9870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 --2- 172.21.15.17:0/2069123039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14440c6600 0x7f14441e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 >> 172.21.15.17:0/2069123039 conn(0x7f144406c700 msgr2=0x7f14440c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 shutdown_connections 2024-03-06T07:20:25.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:25.844+0000 7f144b9eb640 1 -- 172.21.15.17:0/2069123039 wait complete. 2024-03-06T07:20:25.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-06T07:20:25.854 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:25.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-06T07:20:25.857 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:26.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-06T07:20:26.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-06T07:20:26.796 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-06T07:20:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.890+0000 7fda8f9e9640 1 Processor -- start 2024-03-06T07:20:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.890+0000 7f14488eb640 1 Processor -- start 2024-03-06T07:20:26.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.890+0000 7f14488eb640 1 -- start start 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.890+0000 7fda8f9e9640 1 -- start start 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8f9e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 0x7fda881529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8f9e9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda881561a0 0x7fda880a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8f9e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda880a9990 con 0x7fda880aa430 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8f9e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda88155d10 con 0x7fda880c6ab0 2024-03-06T07:20:26.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8f9e9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda881556c0 con 0x7fda881561a0 2024-03-06T07:20:26.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f14488eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14440c5700 0x7f1444155260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8d785640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 0x7fda881529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8cf84640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda881561a0 0x7fda880a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f14488eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f14441592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f14488eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1444156230 con 0x7f14441527f0 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f14488eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1444159810 con 0x7f14440c5700 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f14488eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444155990 con 0x7f1444158e80 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f1441d9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14440c5700 0x7f1444155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f1441d9b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14440c5700 0x7f1444155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60282/0 (socket says 172.21.15.17:60282) 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8d785640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 0x7fda881529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38512/0 (socket says 172.21.15.17:38512) 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8d785640 1 -- 172.21.15.17:0/1678441836 learned_addr learned my addr 172.21.15.17:0/1678441836 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8cf84640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda881561a0 0x7fda880a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60278/0 (socket says 172.21.15.17:60278) 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda8df86640 1 --1- 172.21.15.17:0/1678441836 >> v1:172.21.15.163:6789/0 conn(0x7fda880aa430 0x7fda8814f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57520/0 (socket says 172.21.15.17:57520) 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f144259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f14441527f0 0x7f144406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58442/0 (socket says 172.21.15.17:58442) 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f1441d9b640 1 -- 172.21.15.17:0/3979536730 learned_addr learned my addr 172.21.15.17:0/3979536730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7f1442d9d640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f14441592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.891+0000 7fda7ffff640 1 -- 172.21.15.17:0/1678441836 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019843290 0 0) 0x7fda880a9990 con 0x7fda880aa430 2024-03-06T07:20:26.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda7ffff640 1 -- 172.21.15.17:0/1678441836 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda74003520 con 0x7fda880aa430 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda8cf84640 1 -- 172.21.15.17:0/1678441836 >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 msgr2=0x7fda881529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda8cf84640 1 --2- 172.21.15.17:0/1678441836 >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 0x7fda881529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7f1441d9b640 1 -- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 msgr2=0x7f14441592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7f1441d9b640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f14441592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7f1441d9b640 1 -- 172.21.15.17:0/3979536730 >> v1:172.21.15.17:6789/0 conn(0x7f14441527f0 legacy=0x7f144406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7f1441d9b640 1 -- 172.21.15.17:0/3979536730 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444155c70 con 0x7f14440c5700 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda8cf84640 1 -- 172.21.15.17:0/1678441836 >> v1:172.21.15.163:6789/0 conn(0x7fda880aa430 legacy=0x7fda8814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda8cf84640 1 -- 172.21.15.17:0/1678441836 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda881559a0 con 0x7fda881561a0 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.892+0000 7fda8cf84640 1 --2- 172.21.15.17:0/1678441836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda880a9ef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fda880c0b80 tx=0x7fda7802fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c8a0cca6474b78e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda7ffff640 1 -- 172.21.15.17:0/1678441836 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780307f0 con 0x7fda881561a0 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda7ffff640 1 -- 172.21.15.17:0/1678441836 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda78030990 con 0x7fda881561a0 2024-03-06T07:20:26.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda7ffff640 1 -- 172.21.15.17:0/1678441836 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78030d60 con 0x7fda881561a0 2024-03-06T07:20:26.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7f1441d9b640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f1444155260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f143802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc26088a1960c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7f144159a640 1 -- 172.21.15.17:0/3979536730 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438035020 con 0x7f14440c5700 2024-03-06T07:20:26.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7f144159a640 1 -- 172.21.15.17:0/3979536730 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1438031640 con 0x7f14440c5700 2024-03-06T07:20:26.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7f144159a640 1 -- 172.21.15.17:0/3979536730 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143802d040 con 0x7f14440c5700 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 -- 172.21.15.17:0/1678441836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 msgr2=0x7fda880a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/1678441836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda880a9ef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fda880c0b80 tx=0x7fda7802fc50 comp rx=0 tx=0).stop 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 -- 172.21.15.17:0/1678441836 shutdown_connections 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/1678441836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda880a9ef0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/1678441836 >> v2:172.21.15.17:3300/0 conn(0x7fda880c6ab0 0x7fda881529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.893+0000 7fda8f9e9640 1 -- 172.21.15.17:0/1678441836 >> 172.21.15.17:0/1678441836 conn(0x7fda8806b1a0 msgr2=0x7fda8806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7fda8f9e9640 1 -- 172.21.15.17:0/1678441836 shutdown_connections 2024-03-06T07:20:26.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7fda8f9e9640 1 -- 172.21.15.17:0/1678441836 wait complete. 2024-03-06T07:20:26.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 -- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 msgr2=0x7f1444155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f1444155260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1438009870 tx=0x7f143802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 -- 172.21.15.17:0/3979536730 shutdown_connections 2024-03-06T07:20:26.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f14441592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 --2- 172.21.15.17:0/3979536730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f1444155260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 -- 172.21.15.17:0/3979536730 >> 172.21.15.17:0/3979536730 conn(0x7f14440a95b0 msgr2=0x7f14440a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 -- 172.21.15.17:0/3979536730 shutdown_connections 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.894+0000 7f14488eb640 1 -- 172.21.15.17:0/3979536730 wait complete. 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.895+0000 7fda8f9e9640 1 Processor -- start 2024-03-06T07:20:26.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.895+0000 7fda8f9e9640 1 -- start start 2024-03-06T07:20:26.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.895+0000 7f14488eb640 1 Processor -- start 2024-03-06T07:20:26.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.895+0000 7f14488eb640 1 -- start start 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda88150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8814f820 con 0x7fda880c6ab0 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c2a90 con 0x7fda881561a0 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8f9e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda881506b0 con 0x7fda880aa430 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8cf84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8cf84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38526/0 (socket says 172.21.15.17:38526) 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8cf84640 1 -- 172.21.15.17:0/4219256855 learned_addr learned my addr 172.21.15.17:0/4219256855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52610/0 (socket says 172.21.15.17:52610) 2024-03-06T07:20:26.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8df86640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda88150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f144414d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 0x7f14441eb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f1444154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c2ca0 con 0x7f14441527f0 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444158960 con 0x7f14440c5700 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f144259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f144414d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 -- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 msgr2=0x7fda88150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda88150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 -- 172.21.15.17:0/4219256855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 msgr2=0x7fda881518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f1442d9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f1444154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f14488eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444152bd0 con 0x7f1444158e80 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f144259c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f144414d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60310/0 (socket says 172.21.15.17:60310) 2024-03-06T07:20:26.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f144259c640 1 -- 172.21.15.17:0/1652234617 learned_addr learned my addr 172.21.15.17:0/1652234617 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7f1441d9b640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 0x7f14441eb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8d785640 1 -- 172.21.15.17:0/4219256855 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda881e70b0 con 0x7fda880aa430 2024-03-06T07:20:26.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.896+0000 7fda8cf84640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:26.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda8d785640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda8400e850 tx=0x7fda8400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 -- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 msgr2=0x7f1444154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f1444154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 -- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 msgr2=0x7f144414d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f144414d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 -- 172.21.15.17:0/1652234617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144414ced0 con 0x7f14441527f0 2024-03-06T07:20:26.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda84017070 con 0x7fda880aa430 2024-03-06T07:20:26.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8400c940 con 0x7fda880aa430 2024-03-06T07:20:26.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8400ccf0 con 0x7fda880aa430 2024-03-06T07:20:26.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda881e7390 con 0x7fda880aa430 2024-03-06T07:20:26.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1441d9b640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 0x7f14441eb820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1438033000 tx=0x7f14380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88150340 con 0x7fda880aa430 2024-03-06T07:20:26.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438040070 con 0x7f14441527f0 2024-03-06T07:20:26.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14380399f0 con 0x7f14441527f0 2024-03-06T07:20:26.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144414cc10 con 0x7f14441527f0 2024-03-06T07:20:26.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1444155100 con 0x7f14441527f0 2024-03-06T07:20:26.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.897+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438035040 con 0x7f14441527f0 2024-03-06T07:20:26.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.898+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1444159630 con 0x7f14441527f0 2024-03-06T07:20:26.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8804f330 con 0x7fda880aa430 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda8401a070 con 0x7fda880aa430 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7fda7dffb640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 0x7fda6c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fda8408a6b0 con 0x7fda880aa430 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7fda8cf84640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 0x7fda6c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.899+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143804b060 con 0x7f14441527f0 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.900+0000 7f1432ffd640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.900+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f14380abd70 con 0x7f14441527f0 2024-03-06T07:20:26.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.900+0000 7fda8cf84640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 0x7fda6c06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda88072e50 tx=0x7fda78034000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.900+0000 7f144259c640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.900+0000 7f144259c640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 0x7f142406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1444150410 tx=0x7f1434007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.903+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda84058d10 con 0x7fda880aa430 2024-03-06T07:20:26.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.904+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1444159630 con 0x7f14441527f0 2024-03-06T07:20:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.930+0000 7f9cb8437640 1 Processor -- start 2024-03-06T07:20:26.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.930+0000 7f9cb8437640 1 -- start start 2024-03-06T07:20:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb8437640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cb00c8950 0x7f9cb00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb8437640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb0155130 con 0x7f9cb01558d0 2024-03-06T07:20:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb8437640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb00ad480 con 0x7f9cb00c69f0 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb8437640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb0155440 con 0x7f9cb00c8950 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb59d2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cb00c8950 0x7f9cb00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb59d2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9cb00c8950 0x7f9cb00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38538/0 (socket says 172.21.15.17:38538) 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb61d3640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9cb00c69f0 0x7f9cb00acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57524/0 (socket says 172.21.15.17:57524) 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.931+0000 7f9cb59d2640 1 -- 172.21.15.17:0/384610460 learned_addr learned my addr 172.21.15.17:0/384610460 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973733112 0 0) 0x7f9cb00ad480 con 0x7f9cb00c69f0 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c9c003520 con 0x7f9cb00c69f0 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb59d2640 1 -- 172.21.15.17:0/384610460 >> v1:172.21.15.163:6789/0 conn(0x7f9cb00c69f0 legacy=0x7f9cb00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb59d2640 1 -- 172.21.15.17:0/384610460 >> v1:172.21.15.17:6789/0 conn(0x7f9cb01558d0 legacy=0x7f9cb015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712792248 0 0) 0x7f9ca8001490 con 0x7f9cb01558d0 2024-03-06T07:20:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.932+0000 7f9cb59d2640 1 -- 172.21.15.17:0/384610460 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00c5d50 con 0x7f9cb00c8950 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb59d2640 1 --2- 172.21.15.17:0/384610460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb00c6dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009870 tx=0x7f9ca002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70f94b3e39ce797 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0034070 con 0x7f9cb00c8950 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0030860 con 0x7f9cb00c8950 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb51d1640 1 -- 172.21.15.17:0/384610460 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0030c30 con 0x7f9cb00c8950 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 -- 172.21.15.17:0/384610460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 msgr2=0x7f9cb00c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 --2- 172.21.15.17:0/384610460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb00c6dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009870 tx=0x7f9ca002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 -- 172.21.15.17:0/384610460 shutdown_connections 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 --2- 172.21.15.17:0/384610460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb00c6dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 -- 172.21.15.17:0/384610460 >> 172.21.15.17:0/384610460 conn(0x7f9cb00cae70 msgr2=0x7f9cb00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 -- 172.21.15.17:0/384610460 shutdown_connections 2024-03-06T07:20:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.933+0000 7f9cb8437640 1 -- 172.21.15.17:0/384610460 wait complete. 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 Processor -- start 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 -- start start 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cb00c69f0 0x7f9cb0154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cb01558d0 0x7f9cb01ed730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb0152a20 con 0x7f9cb00c8950 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb00c6310 con 0x7f9cb00c69f0 2024-03-06T07:20:26.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb8437640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb0152ba0 con 0x7f9cb01558d0 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb59d2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb59d2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38550/0 (socket says 172.21.15.17:38550) 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb59d2640 1 -- 172.21.15.17:0/2751063938 learned_addr learned my addr 172.21.15.17:0/2751063938 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.935+0000 7f9cb61d3640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cb00c69f0 0x7f9cb0154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb69d4640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cb01558d0 0x7f9cb01ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 -- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cb01558d0 msgr2=0x7f9cb01ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cb01558d0 0x7f9cb01ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 -- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cb00c69f0 msgr2=0x7f9cb0154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cb00c69f0 0x7f9cb0154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 -- 172.21.15.17:0/2751063938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0158950 con 0x7f9cb00c8950 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb59d2640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00090b0 tx=0x7f9ca0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0048070 con 0x7f9cb00c8950 2024-03-06T07:20:26.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0030660 con 0x7f9cb00c8950 2024-03-06T07:20:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0034040 con 0x7f9cb00c8950 2024-03-06T07:20:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb0158c30 con 0x7f9cb00c8950 2024-03-06T07:20:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.936+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb01591f0 con 0x7f9cb00c8950 2024-03-06T07:20:26.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.937+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb004f330 con 0x7f9cb00c8950 2024-03-06T07:20:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.938+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ca0037070 con 0x7f9cb00c8950 2024-03-06T07:20:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.938+0000 7f9ca6ffd640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.939+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9ca00ac170 con 0x7f9cb00c8950 2024-03-06T07:20:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.939+0000 7f9cb61d3640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:26.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.939+0000 7f9cb61d3640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 0x7f9c9406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0011e0 tx=0x7f9cac007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:26.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:26.947+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ca007a5d0 con 0x7f9cb00c8950 2024-03-06T07:20:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.092+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fda880c26b0 con 0x7fda880aa430 2024-03-06T07:20:27.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.092+0000 7fda7dffb640 1 -- 172.21.15.17:0/4219256855 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fda8405cdb0 con 0x7fda880aa430 2024-03-06T07:20:27.094 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 msgr2=0x7fda6c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 0x7fda6c06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fda88072e50 tx=0x7fda78034000 comp rx=0 tx=0).stop 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 msgr2=0x7fda8814e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fda8400e850 tx=0x7fda8400ed10 comp rx=0 tx=0).stop 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 shutdown_connections 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda6c06b520 0x7fda6c06d9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda881561a0 0x7fda88150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda880c6ab0 0x7fda881518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 --2- 172.21.15.17:0/4219256855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda880aa430 0x7fda8814e190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 >> 172.21.15.17:0/4219256855 conn(0x7fda8806b1a0 msgr2=0x7fda880ab1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f144404f330 con 0x7f14441527f0 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 shutdown_connections 2024-03-06T07:20:27.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7fda8f9e9640 1 -- 172.21.15.17:0/4219256855 wait complete. 2024-03-06T07:20:27.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.094+0000 7f1432ffd640 1 -- 172.21.15.17:0/1652234617 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14380b0070 con 0x7f14441527f0 2024-03-06T07:20:27.098 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 msgr2=0x7f142406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 0x7f142406d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1444150410 tx=0x7f1434007f10 comp rx=0 tx=0).stop 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 msgr2=0x7f14441eb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 0x7f14441eb820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1438033000 tx=0x7f14380046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 shutdown_connections 2024-03-06T07:20:27.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f142406b4d0 0x7f142406d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1444158e80 0x7f1444154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14441527f0 0x7f14441eb820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 --2- 172.21.15.17:0/1652234617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14440c5700 0x7f144414d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 >> 172.21.15.17:0/1652234617 conn(0x7f14440a95b0 msgr2=0x7f14440a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 shutdown_connections 2024-03-06T07:20:27.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.097+0000 7f14488eb640 1 -- 172.21.15.17:0/1652234617 wait complete. 2024-03-06T07:20:27.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-03-06T07:20:27.108 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:27.111 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-03-06T07:20:27.111 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:27.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.132+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9cb00c0390 con 0x7f9cb00c8950 2024-03-06T07:20:27.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.132+0000 7f9ca6ffd640 1 -- 172.21.15.17:0/2751063938 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ca007e670 con 0x7f9cb00c8950 2024-03-06T07:20:27.134 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 msgr2=0x7f9c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 0x7f9c9406d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0011e0 tx=0x7f9cac007d30 comp rx=0 tx=0).stop 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 msgr2=0x7f9cb01ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00090b0 tx=0x7f9ca0004730 comp rx=0 tx=0).stop 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 shutdown_connections 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c9406b4d0 0x7f9c9406d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cb01558d0 0x7f9cb01ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cb00c8950 0x7f9cb01ed1f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 --2- 172.21.15.17:0/2751063938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cb00c69f0 0x7f9cb0154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 >> 172.21.15.17:0/2751063938 conn(0x7f9cb00cae70 msgr2=0x7f9cb0150a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 shutdown_connections 2024-03-06T07:20:27.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.134+0000 7f9cb8437640 1 -- 172.21.15.17:0/2751063938 wait complete. 2024-03-06T07:20:27.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-06T07:20:27.146 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:27.146 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-06T07:20:27.146 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:27.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.286+0000 7efd336fa640 1 Processor -- start 2024-03-06T07:20:27.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.286+0000 7efd336fa640 1 -- start start 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd336fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd2c0c88c0 0x7efd2c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd336fa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2c0c73d0 con 0x7efd2c158b60 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd336fa640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2c0be2e0 con 0x7efd2c14f360 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd336fa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2c159700 con 0x7efd2c0c88c0 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd30c95640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd2c0c88c0 0x7efd2c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd31c97640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efd2c158b60 0x7efd2c158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58460/0 (socket says 172.21.15.17:58460) 2024-03-06T07:20:27.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd31c97640 1 -- 172.21.15.17:0/3922035243 learned_addr learned my addr 172.21.15.17:0/3922035243 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:27.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd30c95640 1 -- 172.21.15.17:0/3922035243 >> v1:172.21.15.163:6789/0 conn(0x7efd2c14f360 legacy=0x7efd2c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd30c95640 1 -- 172.21.15.17:0/3922035243 >> v1:172.21.15.17:6789/0 conn(0x7efd2c158b60 legacy=0x7efd2c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd2bfff640 1 -- 172.21.15.17:0/3922035243 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847503035 0 0) 0x7efd2c0be2e0 con 0x7efd2c14f360 2024-03-06T07:20:27.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.287+0000 7efd30c95640 1 -- 172.21.15.17:0/3922035243 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c0c7670 con 0x7efd2c0c88c0 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.288+0000 7efd30c95640 1 --2- 172.21.15.17:0/3922035243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c0c88c0 0x7efd2c155c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd200097a0 tx=0x7efd2002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd758acb6a0310da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.288+0000 7efd2bfff640 1 -- 172.21.15.17:0/3922035243 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20037070 con 0x7efd2c0c88c0 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.288+0000 7efd2bfff640 1 -- 172.21.15.17:0/3922035243 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20030710 con 0x7efd2c0c88c0 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.288+0000 7efd2bfff640 1 -- 172.21.15.17:0/3922035243 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20030ae0 con 0x7efd2c0c88c0 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 -- 172.21.15.17:0/3922035243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c0c88c0 msgr2=0x7efd2c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 --2- 172.21.15.17:0/3922035243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c0c88c0 0x7efd2c155c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd200097a0 tx=0x7efd2002b710 comp rx=0 tx=0).stop 2024-03-06T07:20:27.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 -- 172.21.15.17:0/3922035243 shutdown_connections 2024-03-06T07:20:27.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 --2- 172.21.15.17:0/3922035243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c0c88c0 0x7efd2c155c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 -- 172.21.15.17:0/3922035243 >> 172.21.15.17:0/3922035243 conn(0x7efd2c0ac570 msgr2=0x7efd2c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 -- 172.21.15.17:0/3922035243 shutdown_connections 2024-03-06T07:20:27.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.289+0000 7efd336fa640 1 -- 172.21.15.17:0/3922035243 wait complete. 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.290+0000 7efd336fa640 1 Processor -- start 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.290+0000 7efd336fa640 1 -- start start 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0c88c0 0x7efd2c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c158b60 0x7efd2c0c0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0c7c30 con 0x7efd2c14f360 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0c5a50 con 0x7efd2c158b60 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd336fa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c0c5bd0 con 0x7efd2c0c88c0 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38570/0 (socket says 172.21.15.17:38570) 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 -- 172.21.15.17:0/1258073640 learned_addr learned my addr 172.21.15.17:0/1258073640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd31496640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0c88c0 0x7efd2c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd31c97640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c158b60 0x7efd2c0c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 -- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0c88c0 msgr2=0x7efd2c157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0c88c0 0x7efd2c157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 -- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c158b60 msgr2=0x7efd2c0c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c158b60 0x7efd2c0c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 -- 172.21.15.17:0/1258073640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c1eb250 con 0x7efd2c14f360 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd30c95640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd200098c0 tx=0x7efd200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.291+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20042070 con 0x7efd2c14f360 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.292+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20032a40 con 0x7efd2c14f360 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.292+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c14b570 con 0x7efd2c14f360 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.292+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20037040 con 0x7efd2c14f360 2024-03-06T07:20:27.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.292+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c0be870 con 0x7efd2c14f360 2024-03-06T07:20:27.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.293+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c04f330 con 0x7efd2c14f360 2024-03-06T07:20:27.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.293+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd20045070 con 0x7efd2c14f360 2024-03-06T07:20:27.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.293+0000 7efd29ffb640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 0x7efd1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.294+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd200ac2f0 con 0x7efd2c14f360 2024-03-06T07:20:27.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.294+0000 7efd31496640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 0x7efd1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.294+0000 7efd31496640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 0x7efd1806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0012e0 tx=0x7efd1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.298+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd2007a8f0 con 0x7efd2c14f360 2024-03-06T07:20:27.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.474+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd2c0c8d50 con 0x7efd1806b4d0 2024-03-06T07:20:27.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.482+0000 7efd29ffb640 1 -- 172.21.15.17:0/1258073640 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7efd2c0c8d50 con 0x7efd1806b4d0 2024-03-06T07:20:27.487 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:27.489 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 msgr2=0x7efd1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 0x7efd1806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0012e0 tx=0x7efd1c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 msgr2=0x7efd2c0c0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd200098c0 tx=0x7efd200046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 shutdown_connections 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd1806b4d0 0x7efd1806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd2c158b60 0x7efd2c0c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd2c14f360 0x7efd2c0c0100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 --2- 172.21.15.17:0/1258073640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd2c0c88c0 0x7efd2c157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.485+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 >> 172.21.15.17:0/1258073640 conn(0x7efd2c0ac570 msgr2=0x7efd2c14bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.486+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 shutdown_connections 2024-03-06T07:20:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.486+0000 7efd336fa640 1 -- 172.21.15.17:0/1258073640 wait complete. 2024-03-06T07:20:27.498 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-03-06T07:20:27.346581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230916,"kb_used_data":2272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743804,"statfs":{"total":773094113280,"available":772857655296,"internally_reserved":0,"allocated":2326528,"data_stored":2049160,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002448"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772886+0000","last_change":"2024-03-06T07:20:03.728088+0000","last_active":"2024-03-06T07:20:10.772886+0000","last_peered":"2024-03-06T07:20:10.772886+0000","last_clean":"2024-03-06T07:20:10.772886+0000","last_became_active":"2024-03-06T07:20:03.727805+0000","last_became_peered":"2024-03-06T07:20:03.727805+0000","last_unstale":"2024-03-06T07:20:10.772886+0000","last_undegraded":"2024-03-06T07:20:10.772886+0000","last_fullsized":"2024-03-06T07:20:10.772886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744735+0000","last_change":"2024-03-06T07:20:04.732564+0000","last_active":"2024-03-06T07:20:10.744735+0000","last_peered":"2024-03-06T07:20:10.744735+0000","last_clean":"2024-03-06T07:20:10.744735+0000","last_became_active":"2024-03-06T07:20:04.732256+0000","last_became_peered":"2024-03-06T07:20:04.732256+0000","last_unstale":"2024-03-06T07:20:10.744735+0000","last_undegraded":"2024-03-06T07:20:10.744735+0000","last_fullsized":"2024-03-06T07:20:10.744735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744130+0000","last_change":"2024-03-06T07:20:09.527717+0000","last_active":"2024-03-06T07:20:10.744130+0000","last_peered":"2024-03-06T07:20:10.744130+0000","last_clean":"2024-03-06T07:20:10.744130+0000","last_became_active":"2024-03-06T07:20:09.527324+0000","last_became_peered":"2024-03-06T07:20:09.527324+0000","last_unstale":"2024-03-06T07:20:10.744130+0000","last_undegraded":"2024-03-06T07:20:10.744130+0000","last_fullsized":"2024-03-06T07:20:10.744130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:05.448069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744975+0000","last_change":"2024-03-06T07:20:03.719181+0000","last_active":"2024-03-06T07:20:10.744975+0000","last_peered":"2024-03-06T07:20:10.744975+0000","last_clean":"2024-03-06T07:20:10.744975+0000","last_became_active":"2024-03-06T07:20:03.718904+0000","last_became_peered":"2024-03-06T07:20:03.718904+0000","last_unstale":"2024-03-06T07:20:10.744975+0000","last_undegraded":"2024-03-06T07:20:10.744975+0000","last_fullsized":"2024-03-06T07:20:10.744975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772804+0000","last_change":"2024-03-06T07:20:04.731613+0000","last_active":"2024-03-06T07:20:10.772804+0000","last_peered":"2024-03-06T07:20:10.772804+0000","last_clean":"2024-03-06T07:20:10.772804+0000","last_became_active":"2024-03-06T07:20:04.731105+0000","last_became_peered":"2024-03-06T07:20:04.731105+0000","last_unstale":"2024-03-06T07:20:10.772804+0000","last_undegraded":"2024-03-06T07:20:10.772804+0000","last_fullsized":"2024-03-06T07:20:10.772804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744947+0000","last_change":"2024-03-06T07:20:08.761811+0000","last_active":"2024-03-06T07:20:10.744947+0000","last_peered":"2024-03-06T07:20:10.744947+0000","last_clean":"2024-03-06T07:20:10.744947+0000","last_became_active":"2024-03-06T07:20:08.761354+0000","last_became_peered":"2024-03-06T07:20:08.761354+0000","last_unstale":"2024-03-06T07:20:10.744947+0000","last_undegraded":"2024-03-06T07:20:10.744947+0000","last_fullsized":"2024-03-06T07:20:10.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:52.215187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273914+0000","last_change":"2024-03-06T07:20:03.728086+0000","last_active":"2024-03-06T07:20:11.273914+0000","last_peered":"2024-03-06T07:20:11.273914+0000","last_clean":"2024-03-06T07:20:11.273914+0000","last_became_active":"2024-03-06T07:20:03.727793+0000","last_became_peered":"2024-03-06T07:20:03.727793+0000","last_unstale":"2024-03-06T07:20:11.273914+0000","last_undegraded":"2024-03-06T07:20:11.273914+0000","last_fullsized":"2024-03-06T07:20:11.273914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273526+0000","last_change":"2024-03-06T07:20:04.732751+0000","last_active":"2024-03-06T07:20:11.273526+0000","last_peered":"2024-03-06T07:20:11.273526+0000","last_clean":"2024-03-06T07:20:11.273526+0000","last_became_active":"2024-03-06T07:20:04.731757+0000","last_became_peered":"2024-03-06T07:20:04.731757+0000","last_unstale":"2024-03-06T07:20:11.273526+0000","last_undegraded":"2024-03-06T07:20:11.273526+0000","last_fullsized":"2024-03-06T07:20:11.273526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746076+0000","last_change":"2024-03-06T07:20:08.756321+0000","last_active":"2024-03-06T07:20:10.746076+0000","last_peered":"2024-03-06T07:20:10.746076+0000","last_clean":"2024-03-06T07:20:10.746076+0000","last_became_active":"2024-03-06T07:20:08.755944+0000","last_became_peered":"2024-03-06T07:20:08.755944+0000","last_unstale":"2024-03-06T07:20:10.746076+0000","last_undegraded":"2024-03-06T07:20:10.746076+0000","last_fullsized":"2024-03-06T07:20:10.746076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273206+0000","last_change":"2024-03-06T07:20:03.730453+0000","last_active":"2024-03-06T07:20:11.273206+0000","last_peered":"2024-03-06T07:20:11.273206+0000","last_clean":"2024-03-06T07:20:11.273206+0000","last_became_active":"2024-03-06T07:20:03.730180+0000","last_became_peered":"2024-03-06T07:20:03.730180+0000","last_unstale":"2024-03-06T07:20:11.273206+0000","last_undegraded":"2024-03-06T07:20:11.273206+0000","last_fullsized":"2024-03-06T07:20:11.273206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:04.734206+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:04.733937+0000","last_became_peered":"2024-03-06T07:20:04.733937+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745261+0000","last_change":"2024-03-06T07:20:08.760335+0000","last_active":"2024-03-06T07:20:10.745261+0000","last_peered":"2024-03-06T07:20:10.745261+0000","last_clean":"2024-03-06T07:20:10.745261+0000","last_became_active":"2024-03-06T07:20:08.759945+0000","last_became_peered":"2024-03-06T07:20:08.759945+0000","last_unstale":"2024-03-06T07:20:10.745261+0000","last_undegraded":"2024-03-06T07:20:10.745261+0000","last_fullsized":"2024-03-06T07:20:10.745261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744803+0000","last_change":"2024-03-06T07:20:04.730615+0000","last_active":"2024-03-06T07:20:10.744803+0000","last_peered":"2024-03-06T07:20:10.744803+0000","last_clean":"2024-03-06T07:20:10.744803+0000","last_became_active":"2024-03-06T07:20:04.729978+0000","last_became_peered":"2024-03-06T07:20:04.729978+0000","last_unstale":"2024-03-06T07:20:10.744803+0000","last_undegraded":"2024-03-06T07:20:10.744803+0000","last_fullsized":"2024-03-06T07:20:10.744803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274021+0000","last_change":"2024-03-06T07:20:03.730180+0000","last_active":"2024-03-06T07:20:11.274021+0000","last_peered":"2024-03-06T07:20:11.274021+0000","last_clean":"2024-03-06T07:20:11.274021+0000","last_became_active":"2024-03-06T07:20:03.729812+0000","last_became_peered":"2024-03-06T07:20:03.729812+0000","last_unstale":"2024-03-06T07:20:11.274021+0000","last_undegraded":"2024-03-06T07:20:11.274021+0000","last_fullsized":"2024-03-06T07:20:11.274021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746544+0000","last_change":"2024-03-06T07:20:08.761949+0000","last_active":"2024-03-06T07:20:10.746544+0000","last_peered":"2024-03-06T07:20:10.746544+0000","last_clean":"2024-03-06T07:20:10.746544+0000","last_became_active":"2024-03-06T07:20:08.761590+0000","last_became_peered":"2024-03-06T07:20:08.761590+0000","last_unstale":"2024-03-06T07:20:10.746544+0000","last_undegraded":"2024-03-06T07:20:10.746544+0000","last_fullsized":"2024-03-06T07:20:10.746544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744471+0000","last_change":"2024-03-06T07:20:04.726070+0000","last_active":"2024-03-06T07:20:10.744471+0000","last_peered":"2024-03-06T07:20:10.744471+0000","last_clean":"2024-03-06T07:20:10.744471+0000","last_became_active":"2024-03-06T07:20:04.725563+0000","last_became_peered":"2024-03-06T07:20:04.725563+0000","last_unstale":"2024-03-06T07:20:10.744471+0000","last_undegraded":"2024-03-06T07:20:10.744471+0000","last_fullsized":"2024-03-06T07:20:10.744471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273584+0000","last_change":"2024-03-06T07:20:03.719632+0000","last_active":"2024-03-06T07:20:11.273584+0000","last_peered":"2024-03-06T07:20:11.273584+0000","last_clean":"2024-03-06T07:20:11.273584+0000","last_became_active":"2024-03-06T07:20:03.719328+0000","last_became_peered":"2024-03-06T07:20:03.719328+0000","last_unstale":"2024-03-06T07:20:11.273584+0000","last_undegraded":"2024-03-06T07:20:11.273584+0000","last_fullsized":"2024-03-06T07:20:11.273584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746162+0000","last_change":"2024-03-06T07:20:08.756550+0000","last_active":"2024-03-06T07:20:10.746162+0000","last_peered":"2024-03-06T07:20:10.746162+0000","last_clean":"2024-03-06T07:20:10.746162+0000","last_became_active":"2024-03-06T07:20:08.755838+0000","last_became_peered":"2024-03-06T07:20:08.755838+0000","last_unstale":"2024-03-06T07:20:10.746162+0000","last_undegraded":"2024-03-06T07:20:10.746162+0000","last_fullsized":"2024-03-06T07:20:10.746162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744264+0000","last_change":"2024-03-06T07:20:04.727861+0000","last_active":"2024-03-06T07:20:10.744264+0000","last_peered":"2024-03-06T07:20:10.744264+0000","last_clean":"2024-03-06T07:20:10.744264+0000","last_became_active":"2024-03-06T07:20:04.727579+0000","last_became_peered":"2024-03-06T07:20:04.727579+0000","last_unstale":"2024-03-06T07:20:10.744264+0000","last_undegraded":"2024-03-06T07:20:10.744264+0000","last_fullsized":"2024-03-06T07:20:10.744264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273057+0000","last_change":"2024-03-06T07:20:03.722227+0000","last_active":"2024-03-06T07:20:11.273057+0000","last_peered":"2024-03-06T07:20:11.273057+0000","last_clean":"2024-03-06T07:20:11.273057+0000","last_became_active":"2024-03-06T07:20:03.721962+0000","last_became_peered":"2024-03-06T07:20:03.721962+0000","last_unstale":"2024-03-06T07:20:11.273057+0000","last_undegraded":"2024-03-06T07:20:11.273057+0000","last_fullsized":"2024-03-06T07:20:11.273057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273518+0000","last_change":"2024-03-06T07:20:09.524648+0000","last_active":"2024-03-06T07:20:11.273518+0000","last_peered":"2024-03-06T07:20:11.273518+0000","last_clean":"2024-03-06T07:20:11.273518+0000","last_became_active":"2024-03-06T07:20:09.524267+0000","last_became_peered":"2024-03-06T07:20:09.524267+0000","last_unstale":"2024-03-06T07:20:11.273518+0000","last_undegraded":"2024-03-06T07:20:11.273518+0000","last_fullsized":"2024-03-06T07:20:11.273518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745823+0000","last_change":"2024-03-06T07:20:04.720167+0000","last_active":"2024-03-06T07:20:10.745823+0000","last_peered":"2024-03-06T07:20:10.745823+0000","last_clean":"2024-03-06T07:20:10.745823+0000","last_became_active":"2024-03-06T07:20:04.719907+0000","last_became_peered":"2024-03-06T07:20:04.719907+0000","last_unstale":"2024-03-06T07:20:10.745823+0000","last_undegraded":"2024-03-06T07:20:10.745823+0000","last_fullsized":"2024-03-06T07:20:10.745823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:45.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272765+0000","last_change":"2024-03-06T07:20:03.730868+0000","last_active":"2024-03-06T07:20:11.272765+0000","last_peered":"2024-03-06T07:20:11.272765+0000","last_clean":"2024-03-06T07:20:11.272765+0000","last_became_active":"2024-03-06T07:20:03.729771+0000","last_became_peered":"2024-03-06T07:20:03.729771+0000","last_unstale":"2024-03-06T07:20:11.272765+0000","last_undegraded":"2024-03-06T07:20:11.272765+0000","last_fullsized":"2024-03-06T07:20:11.272765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746511+0000","last_change":"2024-03-06T07:20:08.756023+0000","last_active":"2024-03-06T07:20:10.746511+0000","last_peered":"2024-03-06T07:20:10.746511+0000","last_clean":"2024-03-06T07:20:10.746511+0000","last_became_active":"2024-03-06T07:20:08.755638+0000","last_became_peered":"2024-03-06T07:20:08.755638+0000","last_unstale":"2024-03-06T07:20:10.746511+0000","last_undegraded":"2024-03-06T07:20:10.746511+0000","last_fullsized":"2024-03-06T07:20:10.746511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773057+0000","last_change":"2024-03-06T07:20:04.724646+0000","last_active":"2024-03-06T07:20:10.773057+0000","last_peered":"2024-03-06T07:20:10.773057+0000","last_clean":"2024-03-06T07:20:10.773057+0000","last_became_active":"2024-03-06T07:20:04.723034+0000","last_became_peered":"2024-03-06T07:20:04.723034+0000","last_unstale":"2024-03-06T07:20:10.773057+0000","last_undegraded":"2024-03-06T07:20:10.773057+0000","last_fullsized":"2024-03-06T07:20:10.773057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745736+0000","last_change":"2024-03-06T07:20:03.723209+0000","last_active":"2024-03-06T07:20:10.745736+0000","last_peered":"2024-03-06T07:20:10.745736+0000","last_clean":"2024-03-06T07:20:10.745736+0000","last_became_active":"2024-03-06T07:20:03.722898+0000","last_became_peered":"2024-03-06T07:20:03.722898+0000","last_unstale":"2024-03-06T07:20:10.745736+0000","last_undegraded":"2024-03-06T07:20:10.745736+0000","last_fullsized":"2024-03-06T07:20:10.745736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745558+0000","last_change":"2024-03-06T07:20:09.526554+0000","last_active":"2024-03-06T07:20:10.745558+0000","last_peered":"2024-03-06T07:20:10.745558+0000","last_clean":"2024-03-06T07:20:10.745558+0000","last_became_active":"2024-03-06T07:20:09.526194+0000","last_became_peered":"2024-03-06T07:20:09.526194+0000","last_unstale":"2024-03-06T07:20:10.745558+0000","last_undegraded":"2024-03-06T07:20:10.745558+0000","last_fullsized":"2024-03-06T07:20:10.745558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745015+0000","last_change":"2024-03-06T07:20:04.736741+0000","last_active":"2024-03-06T07:20:10.745015+0000","last_peered":"2024-03-06T07:20:10.745015+0000","last_clean":"2024-03-06T07:20:10.745015+0000","last_became_active":"2024-03-06T07:20:04.736427+0000","last_became_peered":"2024-03-06T07:20:04.736427+0000","last_unstale":"2024-03-06T07:20:10.745015+0000","last_undegraded":"2024-03-06T07:20:10.745015+0000","last_fullsized":"2024-03-06T07:20:10.745015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273597+0000","last_change":"2024-03-06T07:20:03.720361+0000","last_active":"2024-03-06T07:20:11.273597+0000","last_peered":"2024-03-06T07:20:11.273597+0000","last_clean":"2024-03-06T07:20:11.273597+0000","last_became_active":"2024-03-06T07:20:03.718595+0000","last_became_peered":"2024-03-06T07:20:03.718595+0000","last_unstale":"2024-03-06T07:20:11.273597+0000","last_undegraded":"2024-03-06T07:20:11.273597+0000","last_fullsized":"2024-03-06T07:20:11.273597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744052+0000","last_change":"2024-03-06T07:20:09.526923+0000","last_active":"2024-03-06T07:20:10.744052+0000","last_peered":"2024-03-06T07:20:10.744052+0000","last_clean":"2024-03-06T07:20:10.744052+0000","last_became_active":"2024-03-06T07:20:09.526539+0000","last_became_peered":"2024-03-06T07:20:09.526539+0000","last_unstale":"2024-03-06T07:20:10.744052+0000","last_undegraded":"2024-03-06T07:20:10.744052+0000","last_fullsized":"2024-03-06T07:20:10.744052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:30.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273347+0000","last_change":"2024-03-06T07:20:04.720682+0000","last_active":"2024-03-06T07:20:11.273347+0000","last_peered":"2024-03-06T07:20:11.273347+0000","last_clean":"2024-03-06T07:20:11.273347+0000","last_became_active":"2024-03-06T07:20:04.720404+0000","last_became_peered":"2024-03-06T07:20:04.720404+0000","last_unstale":"2024-03-06T07:20:11.273347+0000","last_undegraded":"2024-03-06T07:20:11.273347+0000","last_fullsized":"2024-03-06T07:20:11.273347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274076+0000","last_change":"2024-03-06T07:20:03.727473+0000","last_active":"2024-03-06T07:20:11.274076+0000","last_peered":"2024-03-06T07:20:11.274076+0000","last_clean":"2024-03-06T07:20:11.274076+0000","last_became_active":"2024-03-06T07:20:03.727177+0000","last_became_peered":"2024-03-06T07:20:03.727177+0000","last_unstale":"2024-03-06T07:20:11.274076+0000","last_undegraded":"2024-03-06T07:20:11.274076+0000","last_fullsized":"2024-03-06T07:20:11.274076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772695+0000","last_change":"2024-03-06T07:20:09.525617+0000","last_active":"2024-03-06T07:20:10.772695+0000","last_peered":"2024-03-06T07:20:10.772695+0000","last_clean":"2024-03-06T07:20:10.772695+0000","last_became_active":"2024-03-06T07:20:09.525228+0000","last_became_peered":"2024-03-06T07:20:09.525228+0000","last_unstale":"2024-03-06T07:20:10.772695+0000","last_undegraded":"2024-03-06T07:20:10.772695+0000","last_fullsized":"2024-03-06T07:20:10.772695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273528+0000","last_change":"2024-03-06T07:20:04.725517+0000","last_active":"2024-03-06T07:20:11.273528+0000","last_peered":"2024-03-06T07:20:11.273528+0000","last_clean":"2024-03-06T07:20:11.273528+0000","last_became_active":"2024-03-06T07:20:04.725226+0000","last_became_peered":"2024-03-06T07:20:04.725226+0000","last_unstale":"2024-03-06T07:20:11.273528+0000","last_undegraded":"2024-03-06T07:20:11.273528+0000","last_fullsized":"2024-03-06T07:20:11.273528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:03.721095+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:03.720730+0000","last_became_peered":"2024-03-06T07:20:03.720730+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273165+0000","last_change":"2024-03-06T07:20:08.763568+0000","last_active":"2024-03-06T07:20:11.273165+0000","last_peered":"2024-03-06T07:20:11.273165+0000","last_clean":"2024-03-06T07:20:11.273165+0000","last_became_active":"2024-03-06T07:20:08.763213+0000","last_became_peered":"2024-03-06T07:20:08.763213+0000","last_unstale":"2024-03-06T07:20:11.273165+0000","last_undegraded":"2024-03-06T07:20:11.273165+0000","last_fullsized":"2024-03-06T07:20:11.273165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746237+0000","last_change":"2024-03-06T07:20:04.715991+0000","last_active":"2024-03-06T07:20:10.746237+0000","last_peered":"2024-03-06T07:20:10.746237+0000","last_clean":"2024-03-06T07:20:10.746237+0000","last_became_active":"2024-03-06T07:20:04.715695+0000","last_became_peered":"2024-03-06T07:20:04.715695+0000","last_unstale":"2024-03-06T07:20:10.746237+0000","last_undegraded":"2024-03-06T07:20:10.746237+0000","last_fullsized":"2024-03-06T07:20:10.746237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:19.554698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274213+0000","last_change":"2024-03-06T07:20:03.724950+0000","last_active":"2024-03-06T07:20:11.274213+0000","last_peered":"2024-03-06T07:20:11.274213+0000","last_clean":"2024-03-06T07:20:11.274213+0000","last_became_active":"2024-03-06T07:20:03.724641+0000","last_became_peered":"2024-03-06T07:20:03.724641+0000","last_unstale":"2024-03-06T07:20:11.274213+0000","last_undegraded":"2024-03-06T07:20:11.274213+0000","last_fullsized":"2024-03-06T07:20:11.274213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273385+0000","last_change":"2024-03-06T07:20:08.762922+0000","last_active":"2024-03-06T07:20:11.273385+0000","last_peered":"2024-03-06T07:20:11.273385+0000","last_clean":"2024-03-06T07:20:11.273385+0000","last_became_active":"2024-03-06T07:20:08.762488+0000","last_became_peered":"2024-03-06T07:20:08.762488+0000","last_unstale":"2024-03-06T07:20:11.273385+0000","last_undegraded":"2024-03-06T07:20:11.273385+0000","last_fullsized":"2024-03-06T07:20:11.273385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273235+0000","last_change":"2024-03-06T07:20:04.729950+0000","last_active":"2024-03-06T07:20:11.273235+0000","last_peered":"2024-03-06T07:20:11.273235+0000","last_clean":"2024-03-06T07:20:11.273235+0000","last_became_active":"2024-03-06T07:20:04.729294+0000","last_became_peered":"2024-03-06T07:20:04.729294+0000","last_unstale":"2024-03-06T07:20:11.273235+0000","last_undegraded":"2024-03-06T07:20:11.273235+0000","last_fullsized":"2024-03-06T07:20:11.273235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877392+0000","last_change":"2024-03-06T07:20:03.717344+0000","last_active":"2024-03-06T07:20:10.877392+0000","last_peered":"2024-03-06T07:20:10.877392+0000","last_clean":"2024-03-06T07:20:10.877392+0000","last_became_active":"2024-03-06T07:20:03.717054+0000","last_became_peered":"2024-03-06T07:20:03.717054+0000","last_unstale":"2024-03-06T07:20:10.877392+0000","last_undegraded":"2024-03-06T07:20:10.877392+0000","last_fullsized":"2024-03-06T07:20:10.877392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273655+0000","last_change":"2024-03-06T07:20:08.763839+0000","last_active":"2024-03-06T07:20:11.273655+0000","last_peered":"2024-03-06T07:20:11.273655+0000","last_clean":"2024-03-06T07:20:11.273655+0000","last_became_active":"2024-03-06T07:20:08.763398+0000","last_became_peered":"2024-03-06T07:20:08.763398+0000","last_unstale":"2024-03-06T07:20:11.273655+0000","last_undegraded":"2024-03-06T07:20:11.273655+0000","last_fullsized":"2024-03-06T07:20:11.273655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772428+0000","last_change":"2024-03-06T07:20:04.717362+0000","last_active":"2024-03-06T07:20:10.772428+0000","last_peered":"2024-03-06T07:20:10.772428+0000","last_clean":"2024-03-06T07:20:10.772428+0000","last_became_active":"2024-03-06T07:20:04.717064+0000","last_became_peered":"2024-03-06T07:20:04.717064+0000","last_unstale":"2024-03-06T07:20:10.772428+0000","last_undegraded":"2024-03-06T07:20:10.772428+0000","last_fullsized":"2024-03-06T07:20:10.772428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:19.051158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272770+0000","last_change":"2024-03-06T07:20:03.730818+0000","last_active":"2024-03-06T07:20:11.272770+0000","last_peered":"2024-03-06T07:20:11.272770+0000","last_clean":"2024-03-06T07:20:11.272770+0000","last_became_active":"2024-03-06T07:20:03.730026+0000","last_became_peered":"2024-03-06T07:20:03.730026+0000","last_unstale":"2024-03-06T07:20:11.272770+0000","last_undegraded":"2024-03-06T07:20:11.272770+0000","last_fullsized":"2024-03-06T07:20:11.272770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773002+0000","last_change":"2024-03-06T07:20:09.524939+0000","last_active":"2024-03-06T07:20:10.773002+0000","last_peered":"2024-03-06T07:20:10.773002+0000","last_clean":"2024-03-06T07:20:10.773002+0000","last_became_active":"2024-03-06T07:20:09.524188+0000","last_became_peered":"2024-03-06T07:20:09.524188+0000","last_unstale":"2024-03-06T07:20:10.773002+0000","last_undegraded":"2024-03-06T07:20:10.773002+0000","last_fullsized":"2024-03-06T07:20:10.773002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744330+0000","last_change":"2024-03-06T07:20:04.733611+0000","last_active":"2024-03-06T07:20:10.744330+0000","last_peered":"2024-03-06T07:20:10.744330+0000","last_clean":"2024-03-06T07:20:10.744330+0000","last_became_active":"2024-03-06T07:20:04.733340+0000","last_became_peered":"2024-03-06T07:20:04.733340+0000","last_unstale":"2024-03-06T07:20:10.744330+0000","last_undegraded":"2024-03-06T07:20:10.744330+0000","last_fullsized":"2024-03-06T07:20:10.744330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273220+0000","last_change":"2024-03-06T07:20:03.720793+0000","last_active":"2024-03-06T07:20:11.273220+0000","last_peered":"2024-03-06T07:20:11.273220+0000","last_clean":"2024-03-06T07:20:11.273220+0000","last_became_active":"2024-03-06T07:20:03.720504+0000","last_became_peered":"2024-03-06T07:20:03.720504+0000","last_unstale":"2024-03-06T07:20:11.273220+0000","last_undegraded":"2024-03-06T07:20:11.273220+0000","last_fullsized":"2024-03-06T07:20:11.273220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272866+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:20:11.272866+0000","last_peered":"2024-03-06T07:20:11.272866+0000","last_clean":"2024-03-06T07:20:11.272866+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:20:11.272866+0000","last_undegraded":"2024-03-06T07:20:11.272866+0000","last_fullsized":"2024-03-06T07:20:11.272866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745518+0000","last_change":"2024-03-06T07:20:09.525305+0000","last_active":"2024-03-06T07:20:10.745518+0000","last_peered":"2024-03-06T07:20:10.745518+0000","last_clean":"2024-03-06T07:20:10.745518+0000","last_became_active":"2024-03-06T07:20:09.524930+0000","last_became_peered":"2024-03-06T07:20:09.524930+0000","last_unstale":"2024-03-06T07:20:10.745518+0000","last_undegraded":"2024-03-06T07:20:10.745518+0000","last_fullsized":"2024-03-06T07:20:10.745518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.731569+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.731301+0000","last_became_peered":"2024-03-06T07:20:04.731301+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877664+0000","last_change":"2024-03-06T07:20:03.716123+0000","last_active":"2024-03-06T07:20:10.877664+0000","last_peered":"2024-03-06T07:20:10.877664+0000","last_clean":"2024-03-06T07:20:10.877664+0000","last_became_active":"2024-03-06T07:20:03.715853+0000","last_became_peered":"2024-03-06T07:20:03.715853+0000","last_unstale":"2024-03-06T07:20:10.877664+0000","last_undegraded":"2024-03-06T07:20:10.877664+0000","last_fullsized":"2024-03-06T07:20:10.877664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744756+0000","last_change":"2024-03-06T07:20:09.524337+0000","last_active":"2024-03-06T07:20:10.744756+0000","last_peered":"2024-03-06T07:20:10.744756+0000","last_clean":"2024-03-06T07:20:10.744756+0000","last_became_active":"2024-03-06T07:20:09.523876+0000","last_became_peered":"2024-03-06T07:20:09.523876+0000","last_unstale":"2024-03-06T07:20:10.744756+0000","last_undegraded":"2024-03-06T07:20:10.744756+0000","last_fullsized":"2024-03-06T07:20:10.744756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273623+0000","last_change":"2024-03-06T07:20:04.736879+0000","last_active":"2024-03-06T07:20:11.273623+0000","last_peered":"2024-03-06T07:20:11.273623+0000","last_clean":"2024-03-06T07:20:11.273623+0000","last_became_active":"2024-03-06T07:20:04.736271+0000","last_became_peered":"2024-03-06T07:20:04.736271+0000","last_unstale":"2024-03-06T07:20:11.273623+0000","last_undegraded":"2024-03-06T07:20:11.273623+0000","last_fullsized":"2024-03-06T07:20:11.273623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273465+0000","last_change":"2024-03-06T07:20:03.721336+0000","last_active":"2024-03-06T07:20:11.273465+0000","last_peered":"2024-03-06T07:20:11.273465+0000","last_clean":"2024-03-06T07:20:11.273465+0000","last_became_active":"2024-03-06T07:20:03.718827+0000","last_became_peered":"2024-03-06T07:20:03.718827+0000","last_unstale":"2024-03-06T07:20:11.273465+0000","last_undegraded":"2024-03-06T07:20:11.273465+0000","last_fullsized":"2024-03-06T07:20:11.273465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772927+0000","last_change":"2024-03-06T07:20:08.756386+0000","last_active":"2024-03-06T07:20:10.772927+0000","last_peered":"2024-03-06T07:20:10.772927+0000","last_clean":"2024-03-06T07:20:10.772927+0000","last_became_active":"2024-03-06T07:20:08.755970+0000","last_became_peered":"2024-03-06T07:20:08.755970+0000","last_unstale":"2024-03-06T07:20:10.772927+0000","last_undegraded":"2024-03-06T07:20:10.772927+0000","last_fullsized":"2024-03-06T07:20:10.772927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273864+0000","last_change":"2024-03-06T07:20:04.737597+0000","last_active":"2024-03-06T07:20:11.273864+0000","last_peered":"2024-03-06T07:20:11.273864+0000","last_clean":"2024-03-06T07:20:11.273864+0000","last_became_active":"2024-03-06T07:20:04.735046+0000","last_became_peered":"2024-03-06T07:20:04.735046+0000","last_unstale":"2024-03-06T07:20:11.273864+0000","last_undegraded":"2024-03-06T07:20:11.273864+0000","last_fullsized":"2024-03-06T07:20:11.273864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:16.868161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877519+0000","last_change":"2024-03-06T07:20:03.712917+0000","last_active":"2024-03-06T07:20:10.877519+0000","last_peered":"2024-03-06T07:20:10.877519+0000","last_clean":"2024-03-06T07:20:10.877519+0000","last_became_active":"2024-03-06T07:20:03.712584+0000","last_became_peered":"2024-03-06T07:20:03.712584+0000","last_unstale":"2024-03-06T07:20:10.877519+0000","last_undegraded":"2024-03-06T07:20:10.877519+0000","last_fullsized":"2024-03-06T07:20:10.877519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272841+0000","last_change":"2024-03-06T07:20:09.525362+0000","last_active":"2024-03-06T07:20:11.272841+0000","last_peered":"2024-03-06T07:20:11.272841+0000","last_clean":"2024-03-06T07:20:11.272841+0000","last_became_active":"2024-03-06T07:20:09.525026+0000","last_became_peered":"2024-03-06T07:20:09.525026+0000","last_unstale":"2024-03-06T07:20:11.272841+0000","last_undegraded":"2024-03-06T07:20:11.272841+0000","last_fullsized":"2024-03-06T07:20:11.272841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:20.377540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273503+0000","last_change":"2024-03-06T07:20:04.730553+0000","last_active":"2024-03-06T07:20:11.273503+0000","last_peered":"2024-03-06T07:20:11.273503+0000","last_clean":"2024-03-06T07:20:11.273503+0000","last_became_active":"2024-03-06T07:20:04.730222+0000","last_became_peered":"2024-03-06T07:20:04.730222+0000","last_unstale":"2024-03-06T07:20:11.273503+0000","last_undegraded":"2024-03-06T07:20:11.273503+0000","last_fullsized":"2024-03-06T07:20:11.273503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746578+0000","last_change":"2024-03-06T07:20:03.718268+0000","last_active":"2024-03-06T07:20:10.746578+0000","last_peered":"2024-03-06T07:20:10.746578+0000","last_clean":"2024-03-06T07:20:10.746578+0000","last_became_active":"2024-03-06T07:20:03.717969+0000","last_became_peered":"2024-03-06T07:20:03.717969+0000","last_unstale":"2024-03-06T07:20:10.746578+0000","last_undegraded":"2024-03-06T07:20:10.746578+0000","last_fullsized":"2024-03-06T07:20:10.746578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745335+0000","last_change":"2024-03-06T07:20:08.756182+0000","last_active":"2024-03-06T07:20:10.745335+0000","last_peered":"2024-03-06T07:20:10.745335+0000","last_clean":"2024-03-06T07:20:10.745335+0000","last_became_active":"2024-03-06T07:20:08.755784+0000","last_became_peered":"2024-03-06T07:20:08.755784+0000","last_unstale":"2024-03-06T07:20:10.745335+0000","last_undegraded":"2024-03-06T07:20:10.745335+0000","last_fullsized":"2024-03-06T07:20:10.745335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273029+0000","last_change":"2024-03-06T07:20:04.729827+0000","last_active":"2024-03-06T07:20:11.273029+0000","last_peered":"2024-03-06T07:20:11.273029+0000","last_clean":"2024-03-06T07:20:11.273029+0000","last_became_active":"2024-03-06T07:20:04.729566+0000","last_became_peered":"2024-03-06T07:20:04.729566+0000","last_unstale":"2024-03-06T07:20:11.273029+0000","last_undegraded":"2024-03-06T07:20:11.273029+0000","last_fullsized":"2024-03-06T07:20:11.273029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877244+0000","last_change":"2024-03-06T07:20:03.723471+0000","last_active":"2024-03-06T07:20:10.877244+0000","last_peered":"2024-03-06T07:20:10.877244+0000","last_clean":"2024-03-06T07:20:10.877244+0000","last_became_active":"2024-03-06T07:20:03.723192+0000","last_became_peered":"2024-03-06T07:20:03.723192+0000","last_unstale":"2024-03-06T07:20:10.877244+0000","last_undegraded":"2024-03-06T07:20:10.877244+0000","last_fullsized":"2024-03-06T07:20:10.877244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772877+0000","last_change":"2024-03-06T07:20:08.760930+0000","last_active":"2024-03-06T07:20:10.772877+0000","last_peered":"2024-03-06T07:20:10.772877+0000","last_clean":"2024-03-06T07:20:10.772877+0000","last_became_active":"2024-03-06T07:20:08.760027+0000","last_became_peered":"2024-03-06T07:20:08.760027+0000","last_unstale":"2024-03-06T07:20:10.772877+0000","last_undegraded":"2024-03-06T07:20:10.772877+0000","last_fullsized":"2024-03-06T07:20:10.772877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744694+0000","last_change":"2024-03-06T07:20:04.717033+0000","last_active":"2024-03-06T07:20:10.744694+0000","last_peered":"2024-03-06T07:20:10.744694+0000","last_clean":"2024-03-06T07:20:10.744694+0000","last_became_active":"2024-03-06T07:20:04.716742+0000","last_became_peered":"2024-03-06T07:20:04.716742+0000","last_unstale":"2024-03-06T07:20:10.744694+0000","last_undegraded":"2024-03-06T07:20:10.744694+0000","last_fullsized":"2024-03-06T07:20:10.744694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745234+0000","last_change":"2024-03-06T07:20:03.717805+0000","last_active":"2024-03-06T07:20:10.745234+0000","last_peered":"2024-03-06T07:20:10.745234+0000","last_clean":"2024-03-06T07:20:10.745234+0000","last_became_active":"2024-03-06T07:20:03.717048+0000","last_became_peered":"2024-03-06T07:20:03.717048+0000","last_unstale":"2024-03-06T07:20:10.745234+0000","last_undegraded":"2024-03-06T07:20:10.745234+0000","last_fullsized":"2024-03-06T07:20:10.745234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744965+0000","last_change":"2024-03-06T07:20:09.525949+0000","last_active":"2024-03-06T07:20:10.744965+0000","last_peered":"2024-03-06T07:20:10.744965+0000","last_clean":"2024-03-06T07:20:10.744965+0000","last_became_active":"2024-03-06T07:20:09.525427+0000","last_became_peered":"2024-03-06T07:20:09.525427+0000","last_unstale":"2024-03-06T07:20:10.744965+0000","last_undegraded":"2024-03-06T07:20:10.744965+0000","last_fullsized":"2024-03-06T07:20:10.744965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:44.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273942+0000","last_change":"2024-03-06T07:20:04.732333+0000","last_active":"2024-03-06T07:20:11.273942+0000","last_peered":"2024-03-06T07:20:11.273942+0000","last_clean":"2024-03-06T07:20:11.273942+0000","last_became_active":"2024-03-06T07:20:04.732069+0000","last_became_peered":"2024-03-06T07:20:04.732069+0000","last_unstale":"2024-03-06T07:20:11.273942+0000","last_undegraded":"2024-03-06T07:20:11.273942+0000","last_fullsized":"2024-03-06T07:20:11.273942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274155+0000","last_change":"2024-03-06T07:20:03.719609+0000","last_active":"2024-03-06T07:20:11.274155+0000","last_peered":"2024-03-06T07:20:11.274155+0000","last_clean":"2024-03-06T07:20:11.274155+0000","last_became_active":"2024-03-06T07:20:03.719325+0000","last_became_peered":"2024-03-06T07:20:03.719325+0000","last_unstale":"2024-03-06T07:20:11.274155+0000","last_undegraded":"2024-03-06T07:20:11.274155+0000","last_fullsized":"2024-03-06T07:20:11.274155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744816+0000","last_change":"2024-03-06T07:20:09.524644+0000","last_active":"2024-03-06T07:20:10.744816+0000","last_peered":"2024-03-06T07:20:10.744816+0000","last_clean":"2024-03-06T07:20:10.744816+0000","last_became_active":"2024-03-06T07:20:09.524220+0000","last_became_peered":"2024-03-06T07:20:09.524220+0000","last_unstale":"2024-03-06T07:20:10.744816+0000","last_undegraded":"2024-03-06T07:20:10.744816+0000","last_fullsized":"2024-03-06T07:20:10.744816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:30.841732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772642+0000","last_change":"2024-03-06T07:20:04.735008+0000","last_active":"2024-03-06T07:20:10.772642+0000","last_peered":"2024-03-06T07:20:10.772642+0000","last_clean":"2024-03-06T07:20:10.772642+0000","last_became_active":"2024-03-06T07:20:04.734709+0000","last_became_peered":"2024-03-06T07:20:04.734709+0000","last_unstale":"2024-03-06T07:20:10.772642+0000","last_undegraded":"2024-03-06T07:20:10.772642+0000","last_fullsized":"2024-03-06T07:20:10.772642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773006+0000","last_change":"2024-03-06T07:20:03.722267+0000","last_active":"2024-03-06T07:20:10.773006+0000","last_peered":"2024-03-06T07:20:10.773006+0000","last_clean":"2024-03-06T07:20:10.773006+0000","last_became_active":"2024-03-06T07:20:03.721952+0000","last_became_peered":"2024-03-06T07:20:03.721952+0000","last_unstale":"2024-03-06T07:20:10.773006+0000","last_undegraded":"2024-03-06T07:20:10.773006+0000","last_fullsized":"2024-03-06T07:20:10.773006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273402+0000","last_change":"2024-03-06T07:20:09.524376+0000","last_active":"2024-03-06T07:20:11.273402+0000","last_peered":"2024-03-06T07:20:11.273402+0000","last_clean":"2024-03-06T07:20:11.273402+0000","last_became_active":"2024-03-06T07:20:09.524010+0000","last_became_peered":"2024-03-06T07:20:09.524010+0000","last_unstale":"2024-03-06T07:20:11.273402+0000","last_undegraded":"2024-03-06T07:20:11.273402+0000","last_fullsized":"2024-03-06T07:20:11.273402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:50.092001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273678+0000","last_change":"2024-03-06T07:20:04.734379+0000","last_active":"2024-03-06T07:20:11.273678+0000","last_peered":"2024-03-06T07:20:11.273678+0000","last_clean":"2024-03-06T07:20:11.273678+0000","last_became_active":"2024-03-06T07:20:04.733727+0000","last_became_peered":"2024-03-06T07:20:04.733727+0000","last_unstale":"2024-03-06T07:20:11.273678+0000","last_undegraded":"2024-03-06T07:20:11.273678+0000","last_fullsized":"2024-03-06T07:20:11.273678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:14.073340+0000","last_change":"2024-03-06T07:20:03.718296+0000","last_active":"2024-03-06T07:20:14.073340+0000","last_peered":"2024-03-06T07:20:14.073340+0000","last_clean":"2024-03-06T07:20:14.073340+0000","last_became_active":"2024-03-06T07:20:03.718015+0000","last_became_peered":"2024-03-06T07:20:03.718015+0000","last_unstale":"2024-03-06T07:20:14.073340+0000","last_undegraded":"2024-03-06T07:20:14.073340+0000","last_fullsized":"2024-03-06T07:20:14.073340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273542+0000","last_change":"2024-03-06T07:20:08.764548+0000","last_active":"2024-03-06T07:20:11.273542+0000","last_peered":"2024-03-06T07:20:11.273542+0000","last_clean":"2024-03-06T07:20:11.273542+0000","last_became_active":"2024-03-06T07:20:08.764175+0000","last_became_peered":"2024-03-06T07:20:08.764175+0000","last_unstale":"2024-03-06T07:20:11.273542+0000","last_undegraded":"2024-03-06T07:20:11.273542+0000","last_fullsized":"2024-03-06T07:20:11.273542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746370+0000","last_change":"2024-03-06T07:20:04.733069+0000","last_active":"2024-03-06T07:20:10.746370+0000","last_peered":"2024-03-06T07:20:10.746370+0000","last_clean":"2024-03-06T07:20:10.746370+0000","last_became_active":"2024-03-06T07:20:04.732656+0000","last_became_peered":"2024-03-06T07:20:04.732656+0000","last_unstale":"2024-03-06T07:20:10.746370+0000","last_undegraded":"2024-03-06T07:20:10.746370+0000","last_fullsized":"2024-03-06T07:20:10.746370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273433+0000","last_change":"2024-03-06T07:20:03.728079+0000","last_active":"2024-03-06T07:20:11.273433+0000","last_peered":"2024-03-06T07:20:11.273433+0000","last_clean":"2024-03-06T07:20:11.273433+0000","last_became_active":"2024-03-06T07:20:03.727628+0000","last_became_peered":"2024-03-06T07:20:03.727628+0000","last_unstale":"2024-03-06T07:20:11.273433+0000","last_undegraded":"2024-03-06T07:20:11.273433+0000","last_fullsized":"2024-03-06T07:20:11.273433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746197+0000","last_change":"2024-03-06T07:20:08.753762+0000","last_active":"2024-03-06T07:20:10.746197+0000","last_peered":"2024-03-06T07:20:10.746197+0000","last_clean":"2024-03-06T07:20:10.746197+0000","last_became_active":"2024-03-06T07:20:08.753331+0000","last_became_peered":"2024-03-06T07:20:08.753331+0000","last_unstale":"2024-03-06T07:20:10.746197+0000","last_undegraded":"2024-03-06T07:20:10.746197+0000","last_fullsized":"2024-03-06T07:20:10.746197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772817+0000","last_change":"2024-03-06T07:20:04.731376+0000","last_active":"2024-03-06T07:20:10.772817+0000","last_peered":"2024-03-06T07:20:10.772817+0000","last_clean":"2024-03-06T07:20:10.772817+0000","last_became_active":"2024-03-06T07:20:04.730731+0000","last_became_peered":"2024-03-06T07:20:04.730731+0000","last_unstale":"2024-03-06T07:20:10.772817+0000","last_undegraded":"2024-03-06T07:20:10.772817+0000","last_fullsized":"2024-03-06T07:20:10.772817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273372+0000","last_change":"2024-03-06T07:20:03.727603+0000","last_active":"2024-03-06T07:20:11.273372+0000","last_peered":"2024-03-06T07:20:11.273372+0000","last_clean":"2024-03-06T07:20:11.273372+0000","last_became_active":"2024-03-06T07:20:03.727238+0000","last_became_peered":"2024-03-06T07:20:03.727238+0000","last_unstale":"2024-03-06T07:20:11.273372+0000","last_undegraded":"2024-03-06T07:20:11.273372+0000","last_fullsized":"2024-03-06T07:20:11.273372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744238+0000","last_change":"2024-03-06T07:20:08.759817+0000","last_active":"2024-03-06T07:20:10.744238+0000","last_peered":"2024-03-06T07:20:10.744238+0000","last_clean":"2024-03-06T07:20:10.744238+0000","last_became_active":"2024-03-06T07:20:08.759430+0000","last_became_peered":"2024-03-06T07:20:08.759430+0000","last_unstale":"2024-03-06T07:20:10.744238+0000","last_undegraded":"2024-03-06T07:20:10.744238+0000","last_fullsized":"2024-03-06T07:20:10.744238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:59.350405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273280+0000","last_change":"2024-03-06T07:20:04.736807+0000","last_active":"2024-03-06T07:20:11.273280+0000","last_peered":"2024-03-06T07:20:11.273280+0000","last_clean":"2024-03-06T07:20:11.273280+0000","last_became_active":"2024-03-06T07:20:04.736552+0000","last_became_peered":"2024-03-06T07:20:04.736552+0000","last_unstale":"2024-03-06T07:20:11.273280+0000","last_undegraded":"2024-03-06T07:20:11.273280+0000","last_fullsized":"2024-03-06T07:20:11.273280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745647+0000","last_change":"2024-03-06T07:20:03.725869+0000","last_active":"2024-03-06T07:20:10.745647+0000","last_peered":"2024-03-06T07:20:10.745647+0000","last_clean":"2024-03-06T07:20:10.745647+0000","last_became_active":"2024-03-06T07:20:03.725563+0000","last_became_peered":"2024-03-06T07:20:03.725563+0000","last_unstale":"2024-03-06T07:20:10.745647+0000","last_undegraded":"2024-03-06T07:20:10.745647+0000","last_fullsized":"2024-03-06T07:20:10.745647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744191+0000","last_change":"2024-03-06T07:20:09.527243+0000","last_active":"2024-03-06T07:20:10.744191+0000","last_peered":"2024-03-06T07:20:10.744191+0000","last_clean":"2024-03-06T07:20:10.744191+0000","last_became_active":"2024-03-06T07:20:09.526607+0000","last_became_peered":"2024-03-06T07:20:09.526607+0000","last_unstale":"2024-03-06T07:20:10.744191+0000","last_undegraded":"2024-03-06T07:20:10.744191+0000","last_fullsized":"2024-03-06T07:20:10.744191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:34.603979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772826+0000","last_change":"2024-03-06T07:20:09.524593+0000","last_active":"2024-03-06T07:20:10.772826+0000","last_peered":"2024-03-06T07:20:10.772826+0000","last_clean":"2024-03-06T07:20:10.772826+0000","last_became_active":"2024-03-06T07:20:09.524244+0000","last_became_peered":"2024-03-06T07:20:09.524244+0000","last_unstale":"2024-03-06T07:20:10.772826+0000","last_undegraded":"2024-03-06T07:20:10.772826+0000","last_fullsized":"2024-03-06T07:20:10.772826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272939+0000","last_change":"2024-03-06T07:20:04.718433+0000","last_active":"2024-03-06T07:20:11.272939+0000","last_peered":"2024-03-06T07:20:11.272939+0000","last_clean":"2024-03-06T07:20:11.272939+0000","last_became_active":"2024-03-06T07:20:04.718154+0000","last_became_peered":"2024-03-06T07:20:04.718154+0000","last_unstale":"2024-03-06T07:20:11.272939+0000","last_undegraded":"2024-03-06T07:20:11.272939+0000","last_fullsized":"2024-03-06T07:20:11.272939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273133+0000","last_change":"2024-03-06T07:20:03.727685+0000","last_active":"2024-03-06T07:20:11.273133+0000","last_peered":"2024-03-06T07:20:11.273133+0000","last_clean":"2024-03-06T07:20:11.273133+0000","last_became_active":"2024-03-06T07:20:03.727363+0000","last_became_peered":"2024-03-06T07:20:03.727363+0000","last_unstale":"2024-03-06T07:20:11.273133+0000","last_undegraded":"2024-03-06T07:20:11.273133+0000","last_fullsized":"2024-03-06T07:20:11.273133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745586+0000","last_change":"2024-03-06T07:20:08.760387+0000","last_active":"2024-03-06T07:20:10.745586+0000","last_peered":"2024-03-06T07:20:10.745586+0000","last_clean":"2024-03-06T07:20:10.745586+0000","last_became_active":"2024-03-06T07:20:08.760017+0000","last_became_peered":"2024-03-06T07:20:08.760017+0000","last_unstale":"2024-03-06T07:20:10.745586+0000","last_undegraded":"2024-03-06T07:20:10.745586+0000","last_fullsized":"2024-03-06T07:20:10.745586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744172+0000","last_change":"2024-03-06T07:20:04.721795+0000","last_active":"2024-03-06T07:20:10.744172+0000","last_peered":"2024-03-06T07:20:10.744172+0000","last_clean":"2024-03-06T07:20:10.744172+0000","last_became_active":"2024-03-06T07:20:04.720723+0000","last_became_peered":"2024-03-06T07:20:04.720723+0000","last_unstale":"2024-03-06T07:20:10.744172+0000","last_undegraded":"2024-03-06T07:20:10.744172+0000","last_fullsized":"2024-03-06T07:20:10.744172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744343+0000","last_change":"2024-03-06T07:20:03.722350+0000","last_active":"2024-03-06T07:20:10.744343+0000","last_peered":"2024-03-06T07:20:10.744343+0000","last_clean":"2024-03-06T07:20:10.744343+0000","last_became_active":"2024-03-06T07:20:03.722059+0000","last_became_peered":"2024-03-06T07:20:03.722059+0000","last_unstale":"2024-03-06T07:20:10.744343+0000","last_undegraded":"2024-03-06T07:20:10.744343+0000","last_fullsized":"2024-03-06T07:20:10.744343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744977+0000","last_change":"2024-03-06T07:20:03.717393+0000","last_active":"2024-03-06T07:20:10.744977+0000","last_peered":"2024-03-06T07:20:10.744977+0000","last_clean":"2024-03-06T07:20:10.744977+0000","last_became_active":"2024-03-06T07:20:03.717051+0000","last_became_peered":"2024-03-06T07:20:03.717051+0000","last_unstale":"2024-03-06T07:20:10.744977+0000","last_undegraded":"2024-03-06T07:20:10.744977+0000","last_fullsized":"2024-03-06T07:20:10.744977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772980+0000","last_change":"2024-03-06T07:20:04.718195+0000","last_active":"2024-03-06T07:20:10.772980+0000","last_peered":"2024-03-06T07:20:10.772980+0000","last_clean":"2024-03-06T07:20:10.772980+0000","last_became_active":"2024-03-06T07:20:04.717447+0000","last_became_peered":"2024-03-06T07:20:04.717447+0000","last_unstale":"2024-03-06T07:20:10.772980+0000","last_undegraded":"2024-03-06T07:20:10.772980+0000","last_fullsized":"2024-03-06T07:20:10.772980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877565+0000","last_change":"2024-03-06T07:20:08.759429+0000","last_active":"2024-03-06T07:20:10.877565+0000","last_peered":"2024-03-06T07:20:10.877565+0000","last_clean":"2024-03-06T07:20:10.877565+0000","last_became_active":"2024-03-06T07:20:08.759038+0000","last_became_peered":"2024-03-06T07:20:08.759038+0000","last_unstale":"2024-03-06T07:20:10.877565+0000","last_undegraded":"2024-03-06T07:20:10.877565+0000","last_fullsized":"2024-03-06T07:20:10.877565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:33.125882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744524+0000","last_change":"2024-03-06T07:20:03.730392+0000","last_active":"2024-03-06T07:20:10.744524+0000","last_peered":"2024-03-06T07:20:10.744524+0000","last_clean":"2024-03-06T07:20:10.744524+0000","last_became_active":"2024-03-06T07:20:03.730088+0000","last_became_peered":"2024-03-06T07:20:03.730088+0000","last_unstale":"2024-03-06T07:20:10.744524+0000","last_undegraded":"2024-03-06T07:20:10.744524+0000","last_fullsized":"2024-03-06T07:20:10.744524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745946+0000","last_change":"2024-03-06T07:20:04.724861+0000","last_active":"2024-03-06T07:20:10.745946+0000","last_peered":"2024-03-06T07:20:10.745946+0000","last_clean":"2024-03-06T07:20:10.745946+0000","last_became_active":"2024-03-06T07:20:04.724584+0000","last_became_peered":"2024-03-06T07:20:04.724584+0000","last_unstale":"2024-03-06T07:20:10.745946+0000","last_undegraded":"2024-03-06T07:20:10.745946+0000","last_fullsized":"2024-03-06T07:20:10.745946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:05.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744887+0000","last_change":"2024-03-06T07:20:08.758137+0000","last_active":"2024-03-06T07:20:10.744887+0000","last_peered":"2024-03-06T07:20:10.744887+0000","last_clean":"2024-03-06T07:20:10.744887+0000","last_became_active":"2024-03-06T07:20:08.757711+0000","last_became_peered":"2024-03-06T07:20:08.757711+0000","last_unstale":"2024-03-06T07:20:10.744887+0000","last_undegraded":"2024-03-06T07:20:10.744887+0000","last_fullsized":"2024-03-06T07:20:10.744887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:47.463176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744591+0000","last_change":"2024-03-06T07:20:04.724223+0000","last_active":"2024-03-06T07:20:10.744591+0000","last_peered":"2024-03-06T07:20:10.744591+0000","last_clean":"2024-03-06T07:20:10.744591+0000","last_became_active":"2024-03-06T07:20:04.723946+0000","last_became_peered":"2024-03-06T07:20:04.723946+0000","last_unstale":"2024-03-06T07:20:10.744591+0000","last_undegraded":"2024-03-06T07:20:10.744591+0000","last_fullsized":"2024-03-06T07:20:10.744591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877317+0000","last_change":"2024-03-06T07:20:04.730151+0000","last_active":"2024-03-06T07:20:10.877317+0000","last_peered":"2024-03-06T07:20:10.877317+0000","last_clean":"2024-03-06T07:20:10.877317+0000","last_became_active":"2024-03-06T07:20:04.729835+0000","last_became_peered":"2024-03-06T07:20:04.729835+0000","last_unstale":"2024-03-06T07:20:10.877317+0000","last_undegraded":"2024-03-06T07:20:10.877317+0000","last_fullsized":"2024-03-06T07:20:10.877317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274150+0000","last_change":"2024-03-06T07:20:04.734333+0000","last_active":"2024-03-06T07:20:11.274150+0000","last_peered":"2024-03-06T07:20:11.274150+0000","last_clean":"2024-03-06T07:20:11.274150+0000","last_became_active":"2024-03-06T07:20:04.734075+0000","last_became_peered":"2024-03-06T07:20:04.734075+0000","last_unstale":"2024-03-06T07:20:11.274150+0000","last_undegraded":"2024-03-06T07:20:11.274150+0000","last_fullsized":"2024-03-06T07:20:11.274150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772954+0000","last_change":"2024-03-06T07:20:04.722013+0000","last_active":"2024-03-06T07:20:10.772954+0000","last_peered":"2024-03-06T07:20:10.772954+0000","last_clean":"2024-03-06T07:20:10.772954+0000","last_became_active":"2024-03-06T07:20:04.721746+0000","last_became_peered":"2024-03-06T07:20:04.721746+0000","last_unstale":"2024-03-06T07:20:10.772954+0000","last_undegraded":"2024-03-06T07:20:10.772954+0000","last_fullsized":"2024-03-06T07:20:10.772954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744882+0000","last_change":"2024-03-06T07:20:04.732536+0000","last_active":"2024-03-06T07:20:10.744882+0000","last_peered":"2024-03-06T07:20:10.744882+0000","last_clean":"2024-03-06T07:20:10.744882+0000","last_became_active":"2024-03-06T07:20:04.728058+0000","last_became_peered":"2024-03-06T07:20:04.728058+0000","last_unstale":"2024-03-06T07:20:10.744882+0000","last_undegraded":"2024-03-06T07:20:10.744882+0000","last_fullsized":"2024-03-06T07:20:10.744882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273227+0000","last_change":"2024-03-06T07:20:04.723910+0000","last_active":"2024-03-06T07:20:11.273227+0000","last_peered":"2024-03-06T07:20:11.273227+0000","last_clean":"2024-03-06T07:20:11.273227+0000","last_became_active":"2024-03-06T07:20:04.722877+0000","last_became_peered":"2024-03-06T07:20:04.722877+0000","last_unstale":"2024-03-06T07:20:11.273227+0000","last_undegraded":"2024-03-06T07:20:11.273227+0000","last_fullsized":"2024-03-06T07:20:11.273227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745929+0000","last_change":"2024-03-06T07:20:04.734964+0000","last_active":"2024-03-06T07:20:10.745929+0000","last_peered":"2024-03-06T07:20:10.745929+0000","last_clean":"2024-03-06T07:20:10.745929+0000","last_became_active":"2024-03-06T07:20:04.734459+0000","last_became_peered":"2024-03-06T07:20:04.734459+0000","last_unstale":"2024-03-06T07:20:10.745929+0000","last_undegraded":"2024-03-06T07:20:10.745929+0000","last_fullsized":"2024-03-06T07:20:10.745929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744920+0000","last_change":"2024-03-06T07:20:04.725185+0000","last_active":"2024-03-06T07:20:10.744920+0000","last_peered":"2024-03-06T07:20:10.744920+0000","last_clean":"2024-03-06T07:20:10.744920+0000","last_became_active":"2024-03-06T07:20:04.724888+0000","last_became_peered":"2024-03-06T07:20:04.724888+0000","last_unstale":"2024-03-06T07:20:10.744920+0000","last_undegraded":"2024-03-06T07:20:10.744920+0000","last_fullsized":"2024-03-06T07:20:10.744920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273048+0000","last_change":"2024-03-06T07:20:04.735180+0000","last_active":"2024-03-06T07:20:11.273048+0000","last_peered":"2024-03-06T07:20:11.273048+0000","last_clean":"2024-03-06T07:20:11.273048+0000","last_became_active":"2024-03-06T07:20:04.733178+0000","last_became_peered":"2024-03-06T07:20:04.733178+0000","last_unstale":"2024-03-06T07:20:11.273048+0000","last_undegraded":"2024-03-06T07:20:11.273048+0000","last_fullsized":"2024-03-06T07:20:11.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744251+0000","last_change":"2024-03-06T07:20:04.726353+0000","last_active":"2024-03-06T07:20:10.744251+0000","last_peered":"2024-03-06T07:20:10.744251+0000","last_clean":"2024-03-06T07:20:10.744251+0000","last_became_active":"2024-03-06T07:20:04.725518+0000","last_became_peered":"2024-03-06T07:20:04.725518+0000","last_unstale":"2024-03-06T07:20:10.744251+0000","last_undegraded":"2024-03-06T07:20:10.744251+0000","last_fullsized":"2024-03-06T07:20:10.744251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274062+0000","last_change":"2024-03-06T07:20:04.736206+0000","last_active":"2024-03-06T07:20:11.274062+0000","last_peered":"2024-03-06T07:20:11.274062+0000","last_clean":"2024-03-06T07:20:11.274062+0000","last_became_active":"2024-03-06T07:20:04.735949+0000","last_became_peered":"2024-03-06T07:20:04.735949+0000","last_unstale":"2024-03-06T07:20:11.274062+0000","last_undegraded":"2024-03-06T07:20:11.274062+0000","last_fullsized":"2024-03-06T07:20:11.274062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772746+0000","last_change":"2024-03-06T07:20:04.728579+0000","last_active":"2024-03-06T07:20:10.772746+0000","last_peered":"2024-03-06T07:20:10.772746+0000","last_clean":"2024-03-06T07:20:10.772746+0000","last_became_active":"2024-03-06T07:20:04.728300+0000","last_became_peered":"2024-03-06T07:20:04.728300+0000","last_unstale":"2024-03-06T07:20:10.772746+0000","last_undegraded":"2024-03-06T07:20:10.772746+0000","last_fullsized":"2024-03-06T07:20:10.772746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745423+0000","last_change":"2024-03-06T07:20:04.724410+0000","last_active":"2024-03-06T07:20:10.745423+0000","last_peered":"2024-03-06T07:20:10.745423+0000","last_clean":"2024-03-06T07:20:10.745423+0000","last_became_active":"2024-03-06T07:20:04.723916+0000","last_became_peered":"2024-03-06T07:20:04.723916+0000","last_unstale":"2024-03-06T07:20:10.745423+0000","last_undegraded":"2024-03-06T07:20:10.745423+0000","last_fullsized":"2024-03-06T07:20:10.745423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274000+0000","last_change":"2024-03-06T07:20:04.734607+0000","last_active":"2024-03-06T07:20:11.274000+0000","last_peered":"2024-03-06T07:20:11.274000+0000","last_clean":"2024-03-06T07:20:11.274000+0000","last_became_active":"2024-03-06T07:20:04.734081+0000","last_became_peered":"2024-03-06T07:20:04.734081+0000","last_unstale":"2024-03-06T07:20:11.274000+0000","last_undegraded":"2024-03-06T07:20:11.274000+0000","last_fullsized":"2024-03-06T07:20:11.274000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877491+0000","last_change":"2024-03-06T07:20:04.726465+0000","last_active":"2024-03-06T07:20:10.877491+0000","last_peered":"2024-03-06T07:20:10.877491+0000","last_clean":"2024-03-06T07:20:10.877491+0000","last_became_active":"2024-03-06T07:20:04.726114+0000","last_became_peered":"2024-03-06T07:20:04.726114+0000","last_unstale":"2024-03-06T07:20:10.877491+0000","last_undegraded":"2024-03-06T07:20:10.877491+0000","last_fullsized":"2024-03-06T07:20:10.877491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.738589+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.737924+0000","last_became_peered":"2024-03-06T07:20:04.737924+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272964+0000","last_change":"2024-03-06T07:20:04.726607+0000","last_active":"2024-03-06T07:20:11.272964+0000","last_peered":"2024-03-06T07:20:11.272964+0000","last_clean":"2024-03-06T07:20:11.272964+0000","last_became_active":"2024-03-06T07:20:04.726337+0000","last_became_peered":"2024-03-06T07:20:04.726337+0000","last_unstale":"2024-03-06T07:20:11.272964+0000","last_undegraded":"2024-03-06T07:20:11.272964+0000","last_fullsized":"2024-03-06T07:20:11.272964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273000+0000","last_change":"2024-03-06T07:20:04.719555+0000","last_active":"2024-03-06T07:20:11.273000+0000","last_peered":"2024-03-06T07:20:11.273000+0000","last_clean":"2024-03-06T07:20:11.273000+0000","last_became_active":"2024-03-06T07:20:04.719273+0000","last_became_peered":"2024-03-06T07:20:04.719273+0000","last_unstale":"2024-03-06T07:20:11.273000+0000","last_undegraded":"2024-03-06T07:20:11.273000+0000","last_fullsized":"2024-03-06T07:20:11.273000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772701+0000","last_change":"2024-03-06T07:20:04.724038+0000","last_active":"2024-03-06T07:20:10.772701+0000","last_peered":"2024-03-06T07:20:10.772701+0000","last_clean":"2024-03-06T07:20:10.772701+0000","last_became_active":"2024-03-06T07:20:04.723527+0000","last_became_peered":"2024-03-06T07:20:04.723527+0000","last_unstale":"2024-03-06T07:20:10.772701+0000","last_undegraded":"2024-03-06T07:20:10.772701+0000","last_fullsized":"2024-03-06T07:20:10.772701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:32.763183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744248+0000","last_change":"2024-03-06T07:20:04.730004+0000","last_active":"2024-03-06T07:20:10.744248+0000","last_peered":"2024-03-06T07:20:10.744248+0000","last_clean":"2024-03-06T07:20:10.744248+0000","last_became_active":"2024-03-06T07:20:04.729702+0000","last_became_peered":"2024-03-06T07:20:04.729702+0000","last_unstale":"2024-03-06T07:20:10.744248+0000","last_undegraded":"2024-03-06T07:20:10.744248+0000","last_fullsized":"2024-03-06T07:20:10.744248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746363+0000","last_change":"2024-03-06T07:20:04.729091+0000","last_active":"2024-03-06T07:20:10.746363+0000","last_peered":"2024-03-06T07:20:10.746363+0000","last_clean":"2024-03-06T07:20:10.746363+0000","last_became_active":"2024-03-06T07:20:04.728793+0000","last_became_peered":"2024-03-06T07:20:04.728793+0000","last_unstale":"2024-03-06T07:20:10.746363+0000","last_undegraded":"2024-03-06T07:20:10.746363+0000","last_fullsized":"2024-03-06T07:20:10.746363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745590+0000","last_change":"2024-03-06T07:20:04.730775+0000","last_active":"2024-03-06T07:20:10.745590+0000","last_peered":"2024-03-06T07:20:10.745590+0000","last_clean":"2024-03-06T07:20:10.745590+0000","last_became_active":"2024-03-06T07:20:04.730386+0000","last_became_peered":"2024-03-06T07:20:04.730386+0000","last_unstale":"2024-03-06T07:20:10.745590+0000","last_undegraded":"2024-03-06T07:20:10.745590+0000","last_fullsized":"2024-03-06T07:20:10.745590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745842+0000","last_change":"2024-03-06T07:20:04.730885+0000","last_active":"2024-03-06T07:20:10.745842+0000","last_peered":"2024-03-06T07:20:10.745842+0000","last_clean":"2024-03-06T07:20:10.745842+0000","last_became_active":"2024-03-06T07:20:04.730294+0000","last_became_peered":"2024-03-06T07:20:04.730294+0000","last_unstale":"2024-03-06T07:20:10.745842+0000","last_undegraded":"2024-03-06T07:20:10.745842+0000","last_fullsized":"2024-03-06T07:20:10.745842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273816+0000","last_change":"2024-03-06T07:20:04.737430+0000","last_active":"2024-03-06T07:20:11.273816+0000","last_peered":"2024-03-06T07:20:11.273816+0000","last_clean":"2024-03-06T07:20:11.273816+0000","last_became_active":"2024-03-06T07:20:04.734870+0000","last_became_peered":"2024-03-06T07:20:04.734870+0000","last_unstale":"2024-03-06T07:20:11.273816+0000","last_undegraded":"2024-03-06T07:20:11.273816+0000","last_fullsized":"2024-03-06T07:20:11.273816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744080+0000","last_change":"2024-03-06T07:20:04.732622+0000","last_active":"2024-03-06T07:20:10.744080+0000","last_peered":"2024-03-06T07:20:10.744080+0000","last_clean":"2024-03-06T07:20:10.744080+0000","last_became_active":"2024-03-06T07:20:04.732149+0000","last_became_peered":"2024-03-06T07:20:04.732149+0000","last_unstale":"2024-03-06T07:20:10.744080+0000","last_undegraded":"2024-03-06T07:20:10.744080+0000","last_fullsized":"2024-03-06T07:20:10.744080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272952+0000","last_change":"2024-03-06T07:20:04.733067+0000","last_active":"2024-03-06T07:20:11.272952+0000","last_peered":"2024-03-06T07:20:11.272952+0000","last_clean":"2024-03-06T07:20:11.272952+0000","last_became_active":"2024-03-06T07:20:04.732805+0000","last_became_peered":"2024-03-06T07:20:04.732805+0000","last_unstale":"2024-03-06T07:20:11.272952+0000","last_undegraded":"2024-03-06T07:20:11.272952+0000","last_fullsized":"2024-03-06T07:20:11.272952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273984+0000","last_change":"2024-03-06T07:20:04.729682+0000","last_active":"2024-03-06T07:20:11.273984+0000","last_peered":"2024-03-06T07:20:11.273984+0000","last_clean":"2024-03-06T07:20:11.273984+0000","last_became_active":"2024-03-06T07:20:04.729407+0000","last_became_peered":"2024-03-06T07:20:04.729407+0000","last_unstale":"2024-03-06T07:20:11.273984+0000","last_undegraded":"2024-03-06T07:20:11.273984+0000","last_fullsized":"2024-03-06T07:20:11.273984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273787+0000","last_change":"2024-03-06T07:20:04.737815+0000","last_active":"2024-03-06T07:20:11.273787+0000","last_peered":"2024-03-06T07:20:11.273787+0000","last_clean":"2024-03-06T07:20:11.273787+0000","last_became_active":"2024-03-06T07:20:04.735276+0000","last_became_peered":"2024-03-06T07:20:04.735276+0000","last_unstale":"2024-03-06T07:20:11.273787+0000","last_undegraded":"2024-03-06T07:20:11.273787+0000","last_fullsized":"2024-03-06T07:20:11.273787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273449+0000","last_change":"2024-03-06T07:20:04.737108+0000","last_active":"2024-03-06T07:20:11.273449+0000","last_peered":"2024-03-06T07:20:11.273449+0000","last_clean":"2024-03-06T07:20:11.273449+0000","last_became_active":"2024-03-06T07:20:04.736608+0000","last_became_peered":"2024-03-06T07:20:04.736608+0000","last_unstale":"2024-03-06T07:20:11.273449+0000","last_undegraded":"2024-03-06T07:20:11.273449+0000","last_fullsized":"2024-03-06T07:20:11.273449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772691+0000","last_change":"2024-03-06T07:20:04.720240+0000","last_active":"2024-03-06T07:20:10.772691+0000","last_peered":"2024-03-06T07:20:10.772691+0000","last_clean":"2024-03-06T07:20:10.772691+0000","last_became_active":"2024-03-06T07:20:04.719952+0000","last_became_peered":"2024-03-06T07:20:04.719952+0000","last_unstale":"2024-03-06T07:20:10.772691+0000","last_undegraded":"2024-03-06T07:20:10.772691+0000","last_fullsized":"2024-03-06T07:20:10.772691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:13.957015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273519+0000","last_change":"2024-03-06T07:20:04.719505+0000","last_active":"2024-03-06T07:20:11.273519+0000","last_peered":"2024-03-06T07:20:11.273519+0000","last_clean":"2024-03-06T07:20:11.273519+0000","last_became_active":"2024-03-06T07:20:04.718961+0000","last_became_peered":"2024-03-06T07:20:04.718961+0000","last_unstale":"2024-03-06T07:20:11.273519+0000","last_undegraded":"2024-03-06T07:20:11.273519+0000","last_fullsized":"2024-03-06T07:20:11.273519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273711+0000","last_change":"2024-03-06T07:20:04.737365+0000","last_active":"2024-03-06T07:20:11.273711+0000","last_peered":"2024-03-06T07:20:11.273711+0000","last_clean":"2024-03-06T07:20:11.273711+0000","last_became_active":"2024-03-06T07:20:04.734552+0000","last_became_peered":"2024-03-06T07:20:04.734552+0000","last_unstale":"2024-03-06T07:20:11.273711+0000","last_undegraded":"2024-03-06T07:20:11.273711+0000","last_fullsized":"2024-03-06T07:20:11.273711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":698343,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":344064,"data_stored":698399,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:27.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:20:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.638+0000 7faedff13640 1 Processor -- start 2024-03-06T07:20:27.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.638+0000 7faedff13640 1 -- start start 2024-03-06T07:20:27.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedff13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faed80c8a00 0x7faed8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedff13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8159460 0x7faed80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedff13640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80c7760 con 0x7faed8152920 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedff13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed81587b0 con 0x7faed80c8a00 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedff13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80c7a40 con 0x7faed8159460 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedd4ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faed80c8a00 0x7faed8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faeddcaf640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faed8152920 0x7faed8155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58466/0 (socket says 172.21.15.17:58466) 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faeddcaf640 1 -- 172.21.15.17:0/3645650857 learned_addr learned my addr 172.21.15.17:0/3645650857 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faede4b0640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8159460 0x7faed80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedd4ae640 1 -- 172.21.15.17:0/3645650857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8159460 msgr2=0x7faed80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedd4ae640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8159460 0x7faed80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedd4ae640 1 -- 172.21.15.17:0/3645650857 >> v1:172.21.15.17:6789/0 conn(0x7faed8152920 legacy=0x7faed8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedccad640 1 -- 172.21.15.17:0/3645650857 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823764763 0 0) 0x7faed80c7760 con 0x7faed8152920 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.639+0000 7faedd4ae640 1 -- 172.21.15.17:0/3645650857 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80c7d20 con 0x7faed80c8a00 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.640+0000 7faedd4ae640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed8158f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faecc0097a0 tx=0x7faecc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d8e84f11964811 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.640+0000 7faedccad640 1 -- 172.21.15.17:0/3645650857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc0315a0 con 0x7faed80c8a00 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.640+0000 7faedccad640 1 -- 172.21.15.17:0/3645650857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc037070 con 0x7faed80c8a00 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.640+0000 7faedccad640 1 -- 172.21.15.17:0/3645650857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc031960 con 0x7faed80c8a00 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 -- 172.21.15.17:0/3645650857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 msgr2=0x7faed8158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed8158f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faecc0097a0 tx=0x7faecc02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 -- 172.21.15.17:0/3645650857 shutdown_connections 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8159460 0x7faed80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 --2- 172.21.15.17:0/3645650857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed8158f20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 -- 172.21.15.17:0/3645650857 >> 172.21.15.17:0/3645650857 conn(0x7faed80ac560 msgr2=0x7faed80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 -- 172.21.15.17:0/3645650857 shutdown_connections 2024-03-06T07:20:27.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.641+0000 7faedff13640 1 -- 172.21.15.17:0/3645650857 wait complete. 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.642+0000 7faedff13640 1 Processor -- start 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.642+0000 7faedff13640 1 -- start start 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8152920 0x7faed8147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed8147c80 0x7faed80c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80c7390 con 0x7faed80c8a00 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed806d030 con 0x7faed8147c80 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80c7510 con 0x7faed8152920 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38584/0 (socket says 172.21.15.17:38584) 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 -- 172.21.15.17:0/841089295 learned_addr learned my addr 172.21.15.17:0/841089295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedd4ae640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8152920 0x7faed8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faede4b0640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed8147c80 0x7faed80c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 -- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8152920 msgr2=0x7faed8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8152920 0x7faed8147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 -- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed8147c80 msgr2=0x7faed80c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed8147c80 0x7faed80c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 -- 172.21.15.17:0/841089295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed81585d0 con 0x7faed80c8a00 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faeddcaf640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faed000c890 tx=0x7faed000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0018070 con 0x7faed80c8a00 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed000a640 con 0x7faed80c8a00 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed000aa30 con 0x7faed80c8a00 2024-03-06T07:20:27.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed81ebe30 con 0x7faed80c8a00 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.643+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed81ebb50 con 0x7faed80c8a00 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.644+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed804f330 con 0x7faed80c8a00 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.645+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faed001b070 con 0x7faed80c8a00 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.645+0000 7faed67fc640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 0x7faec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.645+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faed0089aa0 con 0x7faed80c8a00 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.645+0000 7faedd4ae640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 0x7faec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.646+0000 7faedd4ae640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 0x7faec406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faecc036000 tx=0x7faecc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.650+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faed81585d0 con 0x7faed80c8a00 2024-03-06T07:20:27.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.829+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faed80ac340 con 0x7faec406b4d0 2024-03-06T07:20:27.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.837+0000 7faed67fc640 1 -- 172.21.15.17:0/841089295 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332611 (secure 0 0 0) 0x7faed80ac340 con 0x7faec406b4d0 2024-03-06T07:20:27.842 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:27.844 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:20:27.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 msgr2=0x7faec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 0x7faec406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faecc036000 tx=0x7faecc03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 msgr2=0x7faed814e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faed000c890 tx=0x7faed000cd50 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 shutdown_connections 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faec406b4d0 0x7faec406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed8147c80 0x7faed80c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed8152920 0x7faed8147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 --2- 172.21.15.17:0/841089295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed80c8a00 0x7faed814e540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 >> 172.21.15.17:0/841089295 conn(0x7faed80ac560 msgr2=0x7faed80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 shutdown_connections 2024-03-06T07:20:27.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.840+0000 7faedff13640 1 -- 172.21.15.17:0/841089295 wait complete. 2024-03-06T07:20:27.853 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-03-06T07:20:27.346581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230916,"kb_used_data":2272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754743804,"statfs":{"total":773094113280,"available":772857655296,"internally_reserved":0,"allocated":2326528,"data_stored":2049160,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":234016308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002448"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772886+0000","last_change":"2024-03-06T07:20:03.728088+0000","last_active":"2024-03-06T07:20:10.772886+0000","last_peered":"2024-03-06T07:20:10.772886+0000","last_clean":"2024-03-06T07:20:10.772886+0000","last_became_active":"2024-03-06T07:20:03.727805+0000","last_became_peered":"2024-03-06T07:20:03.727805+0000","last_unstale":"2024-03-06T07:20:10.772886+0000","last_undegraded":"2024-03-06T07:20:10.772886+0000","last_fullsized":"2024-03-06T07:20:10.772886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744735+0000","last_change":"2024-03-06T07:20:04.732564+0000","last_active":"2024-03-06T07:20:10.744735+0000","last_peered":"2024-03-06T07:20:10.744735+0000","last_clean":"2024-03-06T07:20:10.744735+0000","last_became_active":"2024-03-06T07:20:04.732256+0000","last_became_peered":"2024-03-06T07:20:04.732256+0000","last_unstale":"2024-03-06T07:20:10.744735+0000","last_undegraded":"2024-03-06T07:20:10.744735+0000","last_fullsized":"2024-03-06T07:20:10.744735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744130+0000","last_change":"2024-03-06T07:20:09.527717+0000","last_active":"2024-03-06T07:20:10.744130+0000","last_peered":"2024-03-06T07:20:10.744130+0000","last_clean":"2024-03-06T07:20:10.744130+0000","last_became_active":"2024-03-06T07:20:09.527324+0000","last_became_peered":"2024-03-06T07:20:09.527324+0000","last_unstale":"2024-03-06T07:20:10.744130+0000","last_undegraded":"2024-03-06T07:20:10.744130+0000","last_fullsized":"2024-03-06T07:20:10.744130+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:33:05.448069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744975+0000","last_change":"2024-03-06T07:20:03.719181+0000","last_active":"2024-03-06T07:20:10.744975+0000","last_peered":"2024-03-06T07:20:10.744975+0000","last_clean":"2024-03-06T07:20:10.744975+0000","last_became_active":"2024-03-06T07:20:03.718904+0000","last_became_peered":"2024-03-06T07:20:03.718904+0000","last_unstale":"2024-03-06T07:20:10.744975+0000","last_undegraded":"2024-03-06T07:20:10.744975+0000","last_fullsized":"2024-03-06T07:20:10.744975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772804+0000","last_change":"2024-03-06T07:20:04.731613+0000","last_active":"2024-03-06T07:20:10.772804+0000","last_peered":"2024-03-06T07:20:10.772804+0000","last_clean":"2024-03-06T07:20:10.772804+0000","last_became_active":"2024-03-06T07:20:04.731105+0000","last_became_peered":"2024-03-06T07:20:04.731105+0000","last_unstale":"2024-03-06T07:20:10.772804+0000","last_undegraded":"2024-03-06T07:20:10.772804+0000","last_fullsized":"2024-03-06T07:20:10.772804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744947+0000","last_change":"2024-03-06T07:20:08.761811+0000","last_active":"2024-03-06T07:20:10.744947+0000","last_peered":"2024-03-06T07:20:10.744947+0000","last_clean":"2024-03-06T07:20:10.744947+0000","last_became_active":"2024-03-06T07:20:08.761354+0000","last_became_peered":"2024-03-06T07:20:08.761354+0000","last_unstale":"2024-03-06T07:20:10.744947+0000","last_undegraded":"2024-03-06T07:20:10.744947+0000","last_fullsized":"2024-03-06T07:20:10.744947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:52.215187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273914+0000","last_change":"2024-03-06T07:20:03.728086+0000","last_active":"2024-03-06T07:20:11.273914+0000","last_peered":"2024-03-06T07:20:11.273914+0000","last_clean":"2024-03-06T07:20:11.273914+0000","last_became_active":"2024-03-06T07:20:03.727793+0000","last_became_peered":"2024-03-06T07:20:03.727793+0000","last_unstale":"2024-03-06T07:20:11.273914+0000","last_undegraded":"2024-03-06T07:20:11.273914+0000","last_fullsized":"2024-03-06T07:20:11.273914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273526+0000","last_change":"2024-03-06T07:20:04.732751+0000","last_active":"2024-03-06T07:20:11.273526+0000","last_peered":"2024-03-06T07:20:11.273526+0000","last_clean":"2024-03-06T07:20:11.273526+0000","last_became_active":"2024-03-06T07:20:04.731757+0000","last_became_peered":"2024-03-06T07:20:04.731757+0000","last_unstale":"2024-03-06T07:20:11.273526+0000","last_undegraded":"2024-03-06T07:20:11.273526+0000","last_fullsized":"2024-03-06T07:20:11.273526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746076+0000","last_change":"2024-03-06T07:20:08.756321+0000","last_active":"2024-03-06T07:20:10.746076+0000","last_peered":"2024-03-06T07:20:10.746076+0000","last_clean":"2024-03-06T07:20:10.746076+0000","last_became_active":"2024-03-06T07:20:08.755944+0000","last_became_peered":"2024-03-06T07:20:08.755944+0000","last_unstale":"2024-03-06T07:20:10.746076+0000","last_undegraded":"2024-03-06T07:20:10.746076+0000","last_fullsized":"2024-03-06T07:20:10.746076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273206+0000","last_change":"2024-03-06T07:20:03.730453+0000","last_active":"2024-03-06T07:20:11.273206+0000","last_peered":"2024-03-06T07:20:11.273206+0000","last_clean":"2024-03-06T07:20:11.273206+0000","last_became_active":"2024-03-06T07:20:03.730180+0000","last_became_peered":"2024-03-06T07:20:03.730180+0000","last_unstale":"2024-03-06T07:20:11.273206+0000","last_undegraded":"2024-03-06T07:20:11.273206+0000","last_fullsized":"2024-03-06T07:20:11.273206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:04.734206+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:04.733937+0000","last_became_peered":"2024-03-06T07:20:04.733937+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745261+0000","last_change":"2024-03-06T07:20:08.760335+0000","last_active":"2024-03-06T07:20:10.745261+0000","last_peered":"2024-03-06T07:20:10.745261+0000","last_clean":"2024-03-06T07:20:10.745261+0000","last_became_active":"2024-03-06T07:20:08.759945+0000","last_became_peered":"2024-03-06T07:20:08.759945+0000","last_unstale":"2024-03-06T07:20:10.745261+0000","last_undegraded":"2024-03-06T07:20:10.745261+0000","last_fullsized":"2024-03-06T07:20:10.745261+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744803+0000","last_change":"2024-03-06T07:20:04.730615+0000","last_active":"2024-03-06T07:20:10.744803+0000","last_peered":"2024-03-06T07:20:10.744803+0000","last_clean":"2024-03-06T07:20:10.744803+0000","last_became_active":"2024-03-06T07:20:04.729978+0000","last_became_peered":"2024-03-06T07:20:04.729978+0000","last_unstale":"2024-03-06T07:20:10.744803+0000","last_undegraded":"2024-03-06T07:20:10.744803+0000","last_fullsized":"2024-03-06T07:20:10.744803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274021+0000","last_change":"2024-03-06T07:20:03.730180+0000","last_active":"2024-03-06T07:20:11.274021+0000","last_peered":"2024-03-06T07:20:11.274021+0000","last_clean":"2024-03-06T07:20:11.274021+0000","last_became_active":"2024-03-06T07:20:03.729812+0000","last_became_peered":"2024-03-06T07:20:03.729812+0000","last_unstale":"2024-03-06T07:20:11.274021+0000","last_undegraded":"2024-03-06T07:20:11.274021+0000","last_fullsized":"2024-03-06T07:20:11.274021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746544+0000","last_change":"2024-03-06T07:20:08.761949+0000","last_active":"2024-03-06T07:20:10.746544+0000","last_peered":"2024-03-06T07:20:10.746544+0000","last_clean":"2024-03-06T07:20:10.746544+0000","last_became_active":"2024-03-06T07:20:08.761590+0000","last_became_peered":"2024-03-06T07:20:08.761590+0000","last_unstale":"2024-03-06T07:20:10.746544+0000","last_undegraded":"2024-03-06T07:20:10.746544+0000","last_fullsized":"2024-03-06T07:20:10.746544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744471+0000","last_change":"2024-03-06T07:20:04.726070+0000","last_active":"2024-03-06T07:20:10.744471+0000","last_peered":"2024-03-06T07:20:10.744471+0000","last_clean":"2024-03-06T07:20:10.744471+0000","last_became_active":"2024-03-06T07:20:04.725563+0000","last_became_peered":"2024-03-06T07:20:04.725563+0000","last_unstale":"2024-03-06T07:20:10.744471+0000","last_undegraded":"2024-03-06T07:20:10.744471+0000","last_fullsized":"2024-03-06T07:20:10.744471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273584+0000","last_change":"2024-03-06T07:20:03.719632+0000","last_active":"2024-03-06T07:20:11.273584+0000","last_peered":"2024-03-06T07:20:11.273584+0000","last_clean":"2024-03-06T07:20:11.273584+0000","last_became_active":"2024-03-06T07:20:03.719328+0000","last_became_peered":"2024-03-06T07:20:03.719328+0000","last_unstale":"2024-03-06T07:20:11.273584+0000","last_undegraded":"2024-03-06T07:20:11.273584+0000","last_fullsized":"2024-03-06T07:20:11.273584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746162+0000","last_change":"2024-03-06T07:20:08.756550+0000","last_active":"2024-03-06T07:20:10.746162+0000","last_peered":"2024-03-06T07:20:10.746162+0000","last_clean":"2024-03-06T07:20:10.746162+0000","last_became_active":"2024-03-06T07:20:08.755838+0000","last_became_peered":"2024-03-06T07:20:08.755838+0000","last_unstale":"2024-03-06T07:20:10.746162+0000","last_undegraded":"2024-03-06T07:20:10.746162+0000","last_fullsized":"2024-03-06T07:20:10.746162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744264+0000","last_change":"2024-03-06T07:20:04.727861+0000","last_active":"2024-03-06T07:20:10.744264+0000","last_peered":"2024-03-06T07:20:10.744264+0000","last_clean":"2024-03-06T07:20:10.744264+0000","last_became_active":"2024-03-06T07:20:04.727579+0000","last_became_peered":"2024-03-06T07:20:04.727579+0000","last_unstale":"2024-03-06T07:20:10.744264+0000","last_undegraded":"2024-03-06T07:20:10.744264+0000","last_fullsized":"2024-03-06T07:20:10.744264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273057+0000","last_change":"2024-03-06T07:20:03.722227+0000","last_active":"2024-03-06T07:20:11.273057+0000","last_peered":"2024-03-06T07:20:11.273057+0000","last_clean":"2024-03-06T07:20:11.273057+0000","last_became_active":"2024-03-06T07:20:03.721962+0000","last_became_peered":"2024-03-06T07:20:03.721962+0000","last_unstale":"2024-03-06T07:20:11.273057+0000","last_undegraded":"2024-03-06T07:20:11.273057+0000","last_fullsized":"2024-03-06T07:20:11.273057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273518+0000","last_change":"2024-03-06T07:20:09.524648+0000","last_active":"2024-03-06T07:20:11.273518+0000","last_peered":"2024-03-06T07:20:11.273518+0000","last_clean":"2024-03-06T07:20:11.273518+0000","last_became_active":"2024-03-06T07:20:09.524267+0000","last_became_peered":"2024-03-06T07:20:09.524267+0000","last_unstale":"2024-03-06T07:20:11.273518+0000","last_undegraded":"2024-03-06T07:20:11.273518+0000","last_fullsized":"2024-03-06T07:20:11.273518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745823+0000","last_change":"2024-03-06T07:20:04.720167+0000","last_active":"2024-03-06T07:20:10.745823+0000","last_peered":"2024-03-06T07:20:10.745823+0000","last_clean":"2024-03-06T07:20:10.745823+0000","last_became_active":"2024-03-06T07:20:04.719907+0000","last_became_peered":"2024-03-06T07:20:04.719907+0000","last_unstale":"2024-03-06T07:20:10.745823+0000","last_undegraded":"2024-03-06T07:20:10.745823+0000","last_fullsized":"2024-03-06T07:20:10.745823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:45.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272765+0000","last_change":"2024-03-06T07:20:03.730868+0000","last_active":"2024-03-06T07:20:11.272765+0000","last_peered":"2024-03-06T07:20:11.272765+0000","last_clean":"2024-03-06T07:20:11.272765+0000","last_became_active":"2024-03-06T07:20:03.729771+0000","last_became_peered":"2024-03-06T07:20:03.729771+0000","last_unstale":"2024-03-06T07:20:11.272765+0000","last_undegraded":"2024-03-06T07:20:11.272765+0000","last_fullsized":"2024-03-06T07:20:11.272765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746511+0000","last_change":"2024-03-06T07:20:08.756023+0000","last_active":"2024-03-06T07:20:10.746511+0000","last_peered":"2024-03-06T07:20:10.746511+0000","last_clean":"2024-03-06T07:20:10.746511+0000","last_became_active":"2024-03-06T07:20:08.755638+0000","last_became_peered":"2024-03-06T07:20:08.755638+0000","last_unstale":"2024-03-06T07:20:10.746511+0000","last_undegraded":"2024-03-06T07:20:10.746511+0000","last_fullsized":"2024-03-06T07:20:10.746511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773057+0000","last_change":"2024-03-06T07:20:04.724646+0000","last_active":"2024-03-06T07:20:10.773057+0000","last_peered":"2024-03-06T07:20:10.773057+0000","last_clean":"2024-03-06T07:20:10.773057+0000","last_became_active":"2024-03-06T07:20:04.723034+0000","last_became_peered":"2024-03-06T07:20:04.723034+0000","last_unstale":"2024-03-06T07:20:10.773057+0000","last_undegraded":"2024-03-06T07:20:10.773057+0000","last_fullsized":"2024-03-06T07:20:10.773057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745736+0000","last_change":"2024-03-06T07:20:03.723209+0000","last_active":"2024-03-06T07:20:10.745736+0000","last_peered":"2024-03-06T07:20:10.745736+0000","last_clean":"2024-03-06T07:20:10.745736+0000","last_became_active":"2024-03-06T07:20:03.722898+0000","last_became_peered":"2024-03-06T07:20:03.722898+0000","last_unstale":"2024-03-06T07:20:10.745736+0000","last_undegraded":"2024-03-06T07:20:10.745736+0000","last_fullsized":"2024-03-06T07:20:10.745736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745558+0000","last_change":"2024-03-06T07:20:09.526554+0000","last_active":"2024-03-06T07:20:10.745558+0000","last_peered":"2024-03-06T07:20:10.745558+0000","last_clean":"2024-03-06T07:20:10.745558+0000","last_became_active":"2024-03-06T07:20:09.526194+0000","last_became_peered":"2024-03-06T07:20:09.526194+0000","last_unstale":"2024-03-06T07:20:10.745558+0000","last_undegraded":"2024-03-06T07:20:10.745558+0000","last_fullsized":"2024-03-06T07:20:10.745558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745015+0000","last_change":"2024-03-06T07:20:04.736741+0000","last_active":"2024-03-06T07:20:10.745015+0000","last_peered":"2024-03-06T07:20:10.745015+0000","last_clean":"2024-03-06T07:20:10.745015+0000","last_became_active":"2024-03-06T07:20:04.736427+0000","last_became_peered":"2024-03-06T07:20:04.736427+0000","last_unstale":"2024-03-06T07:20:10.745015+0000","last_undegraded":"2024-03-06T07:20:10.745015+0000","last_fullsized":"2024-03-06T07:20:10.745015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273597+0000","last_change":"2024-03-06T07:20:03.720361+0000","last_active":"2024-03-06T07:20:11.273597+0000","last_peered":"2024-03-06T07:20:11.273597+0000","last_clean":"2024-03-06T07:20:11.273597+0000","last_became_active":"2024-03-06T07:20:03.718595+0000","last_became_peered":"2024-03-06T07:20:03.718595+0000","last_unstale":"2024-03-06T07:20:11.273597+0000","last_undegraded":"2024-03-06T07:20:11.273597+0000","last_fullsized":"2024-03-06T07:20:11.273597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744052+0000","last_change":"2024-03-06T07:20:09.526923+0000","last_active":"2024-03-06T07:20:10.744052+0000","last_peered":"2024-03-06T07:20:10.744052+0000","last_clean":"2024-03-06T07:20:10.744052+0000","last_became_active":"2024-03-06T07:20:09.526539+0000","last_became_peered":"2024-03-06T07:20:09.526539+0000","last_unstale":"2024-03-06T07:20:10.744052+0000","last_undegraded":"2024-03-06T07:20:10.744052+0000","last_fullsized":"2024-03-06T07:20:10.744052+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:30.282312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273347+0000","last_change":"2024-03-06T07:20:04.720682+0000","last_active":"2024-03-06T07:20:11.273347+0000","last_peered":"2024-03-06T07:20:11.273347+0000","last_clean":"2024-03-06T07:20:11.273347+0000","last_became_active":"2024-03-06T07:20:04.720404+0000","last_became_peered":"2024-03-06T07:20:04.720404+0000","last_unstale":"2024-03-06T07:20:11.273347+0000","last_undegraded":"2024-03-06T07:20:11.273347+0000","last_fullsized":"2024-03-06T07:20:11.273347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274076+0000","last_change":"2024-03-06T07:20:03.727473+0000","last_active":"2024-03-06T07:20:11.274076+0000","last_peered":"2024-03-06T07:20:11.274076+0000","last_clean":"2024-03-06T07:20:11.274076+0000","last_became_active":"2024-03-06T07:20:03.727177+0000","last_became_peered":"2024-03-06T07:20:03.727177+0000","last_unstale":"2024-03-06T07:20:11.274076+0000","last_undegraded":"2024-03-06T07:20:11.274076+0000","last_fullsized":"2024-03-06T07:20:11.274076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772695+0000","last_change":"2024-03-06T07:20:09.525617+0000","last_active":"2024-03-06T07:20:10.772695+0000","last_peered":"2024-03-06T07:20:10.772695+0000","last_clean":"2024-03-06T07:20:10.772695+0000","last_became_active":"2024-03-06T07:20:09.525228+0000","last_became_peered":"2024-03-06T07:20:09.525228+0000","last_unstale":"2024-03-06T07:20:10.772695+0000","last_undegraded":"2024-03-06T07:20:10.772695+0000","last_fullsized":"2024-03-06T07:20:10.772695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273528+0000","last_change":"2024-03-06T07:20:04.725517+0000","last_active":"2024-03-06T07:20:11.273528+0000","last_peered":"2024-03-06T07:20:11.273528+0000","last_clean":"2024-03-06T07:20:11.273528+0000","last_became_active":"2024-03-06T07:20:04.725226+0000","last_became_peered":"2024-03-06T07:20:04.725226+0000","last_unstale":"2024-03-06T07:20:11.273528+0000","last_undegraded":"2024-03-06T07:20:11.273528+0000","last_fullsized":"2024-03-06T07:20:11.273528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273447+0000","last_change":"2024-03-06T07:20:03.721095+0000","last_active":"2024-03-06T07:20:11.273447+0000","last_peered":"2024-03-06T07:20:11.273447+0000","last_clean":"2024-03-06T07:20:11.273447+0000","last_became_active":"2024-03-06T07:20:03.720730+0000","last_became_peered":"2024-03-06T07:20:03.720730+0000","last_unstale":"2024-03-06T07:20:11.273447+0000","last_undegraded":"2024-03-06T07:20:11.273447+0000","last_fullsized":"2024-03-06T07:20:11.273447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273165+0000","last_change":"2024-03-06T07:20:08.763568+0000","last_active":"2024-03-06T07:20:11.273165+0000","last_peered":"2024-03-06T07:20:11.273165+0000","last_clean":"2024-03-06T07:20:11.273165+0000","last_became_active":"2024-03-06T07:20:08.763213+0000","last_became_peered":"2024-03-06T07:20:08.763213+0000","last_unstale":"2024-03-06T07:20:11.273165+0000","last_undegraded":"2024-03-06T07:20:11.273165+0000","last_fullsized":"2024-03-06T07:20:11.273165+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:50.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746237+0000","last_change":"2024-03-06T07:20:04.715991+0000","last_active":"2024-03-06T07:20:10.746237+0000","last_peered":"2024-03-06T07:20:10.746237+0000","last_clean":"2024-03-06T07:20:10.746237+0000","last_became_active":"2024-03-06T07:20:04.715695+0000","last_became_peered":"2024-03-06T07:20:04.715695+0000","last_unstale":"2024-03-06T07:20:10.746237+0000","last_undegraded":"2024-03-06T07:20:10.746237+0000","last_fullsized":"2024-03-06T07:20:10.746237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:19.554698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274213+0000","last_change":"2024-03-06T07:20:03.724950+0000","last_active":"2024-03-06T07:20:11.274213+0000","last_peered":"2024-03-06T07:20:11.274213+0000","last_clean":"2024-03-06T07:20:11.274213+0000","last_became_active":"2024-03-06T07:20:03.724641+0000","last_became_peered":"2024-03-06T07:20:03.724641+0000","last_unstale":"2024-03-06T07:20:11.274213+0000","last_undegraded":"2024-03-06T07:20:11.274213+0000","last_fullsized":"2024-03-06T07:20:11.274213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273385+0000","last_change":"2024-03-06T07:20:08.762922+0000","last_active":"2024-03-06T07:20:11.273385+0000","last_peered":"2024-03-06T07:20:11.273385+0000","last_clean":"2024-03-06T07:20:11.273385+0000","last_became_active":"2024-03-06T07:20:08.762488+0000","last_became_peered":"2024-03-06T07:20:08.762488+0000","last_unstale":"2024-03-06T07:20:11.273385+0000","last_undegraded":"2024-03-06T07:20:11.273385+0000","last_fullsized":"2024-03-06T07:20:11.273385+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273235+0000","last_change":"2024-03-06T07:20:04.729950+0000","last_active":"2024-03-06T07:20:11.273235+0000","last_peered":"2024-03-06T07:20:11.273235+0000","last_clean":"2024-03-06T07:20:11.273235+0000","last_became_active":"2024-03-06T07:20:04.729294+0000","last_became_peered":"2024-03-06T07:20:04.729294+0000","last_unstale":"2024-03-06T07:20:11.273235+0000","last_undegraded":"2024-03-06T07:20:11.273235+0000","last_fullsized":"2024-03-06T07:20:11.273235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877392+0000","last_change":"2024-03-06T07:20:03.717344+0000","last_active":"2024-03-06T07:20:10.877392+0000","last_peered":"2024-03-06T07:20:10.877392+0000","last_clean":"2024-03-06T07:20:10.877392+0000","last_became_active":"2024-03-06T07:20:03.717054+0000","last_became_peered":"2024-03-06T07:20:03.717054+0000","last_unstale":"2024-03-06T07:20:10.877392+0000","last_undegraded":"2024-03-06T07:20:10.877392+0000","last_fullsized":"2024-03-06T07:20:10.877392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273655+0000","last_change":"2024-03-06T07:20:08.763839+0000","last_active":"2024-03-06T07:20:11.273655+0000","last_peered":"2024-03-06T07:20:11.273655+0000","last_clean":"2024-03-06T07:20:11.273655+0000","last_became_active":"2024-03-06T07:20:08.763398+0000","last_became_peered":"2024-03-06T07:20:08.763398+0000","last_unstale":"2024-03-06T07:20:11.273655+0000","last_undegraded":"2024-03-06T07:20:11.273655+0000","last_fullsized":"2024-03-06T07:20:11.273655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772428+0000","last_change":"2024-03-06T07:20:04.717362+0000","last_active":"2024-03-06T07:20:10.772428+0000","last_peered":"2024-03-06T07:20:10.772428+0000","last_clean":"2024-03-06T07:20:10.772428+0000","last_became_active":"2024-03-06T07:20:04.717064+0000","last_became_peered":"2024-03-06T07:20:04.717064+0000","last_unstale":"2024-03-06T07:20:10.772428+0000","last_undegraded":"2024-03-06T07:20:10.772428+0000","last_fullsized":"2024-03-06T07:20:10.772428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:19.051158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272770+0000","last_change":"2024-03-06T07:20:03.730818+0000","last_active":"2024-03-06T07:20:11.272770+0000","last_peered":"2024-03-06T07:20:11.272770+0000","last_clean":"2024-03-06T07:20:11.272770+0000","last_became_active":"2024-03-06T07:20:03.730026+0000","last_became_peered":"2024-03-06T07:20:03.730026+0000","last_unstale":"2024-03-06T07:20:11.272770+0000","last_undegraded":"2024-03-06T07:20:11.272770+0000","last_fullsized":"2024-03-06T07:20:11.272770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773002+0000","last_change":"2024-03-06T07:20:09.524939+0000","last_active":"2024-03-06T07:20:10.773002+0000","last_peered":"2024-03-06T07:20:10.773002+0000","last_clean":"2024-03-06T07:20:10.773002+0000","last_became_active":"2024-03-06T07:20:09.524188+0000","last_became_peered":"2024-03-06T07:20:09.524188+0000","last_unstale":"2024-03-06T07:20:10.773002+0000","last_undegraded":"2024-03-06T07:20:10.773002+0000","last_fullsized":"2024-03-06T07:20:10.773002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:17:08.494931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744330+0000","last_change":"2024-03-06T07:20:04.733611+0000","last_active":"2024-03-06T07:20:10.744330+0000","last_peered":"2024-03-06T07:20:10.744330+0000","last_clean":"2024-03-06T07:20:10.744330+0000","last_became_active":"2024-03-06T07:20:04.733340+0000","last_became_peered":"2024-03-06T07:20:04.733340+0000","last_unstale":"2024-03-06T07:20:10.744330+0000","last_undegraded":"2024-03-06T07:20:10.744330+0000","last_fullsized":"2024-03-06T07:20:10.744330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273220+0000","last_change":"2024-03-06T07:20:03.720793+0000","last_active":"2024-03-06T07:20:11.273220+0000","last_peered":"2024-03-06T07:20:11.273220+0000","last_clean":"2024-03-06T07:20:11.273220+0000","last_became_active":"2024-03-06T07:20:03.720504+0000","last_became_peered":"2024-03-06T07:20:03.720504+0000","last_unstale":"2024-03-06T07:20:11.273220+0000","last_undegraded":"2024-03-06T07:20:11.273220+0000","last_fullsized":"2024-03-06T07:20:11.273220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272866+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:20:11.272866+0000","last_peered":"2024-03-06T07:20:11.272866+0000","last_clean":"2024-03-06T07:20:11.272866+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:20:11.272866+0000","last_undegraded":"2024-03-06T07:20:11.272866+0000","last_fullsized":"2024-03-06T07:20:11.272866+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745518+0000","last_change":"2024-03-06T07:20:09.525305+0000","last_active":"2024-03-06T07:20:10.745518+0000","last_peered":"2024-03-06T07:20:10.745518+0000","last_clean":"2024-03-06T07:20:10.745518+0000","last_became_active":"2024-03-06T07:20:09.524930+0000","last_became_peered":"2024-03-06T07:20:09.524930+0000","last_unstale":"2024-03-06T07:20:10.745518+0000","last_undegraded":"2024-03-06T07:20:10.745518+0000","last_fullsized":"2024-03-06T07:20:10.745518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.731569+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.731301+0000","last_became_peered":"2024-03-06T07:20:04.731301+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877664+0000","last_change":"2024-03-06T07:20:03.716123+0000","last_active":"2024-03-06T07:20:10.877664+0000","last_peered":"2024-03-06T07:20:10.877664+0000","last_clean":"2024-03-06T07:20:10.877664+0000","last_became_active":"2024-03-06T07:20:03.715853+0000","last_became_peered":"2024-03-06T07:20:03.715853+0000","last_unstale":"2024-03-06T07:20:10.877664+0000","last_undegraded":"2024-03-06T07:20:10.877664+0000","last_fullsized":"2024-03-06T07:20:10.877664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:03:31.379655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744756+0000","last_change":"2024-03-06T07:20:09.524337+0000","last_active":"2024-03-06T07:20:10.744756+0000","last_peered":"2024-03-06T07:20:10.744756+0000","last_clean":"2024-03-06T07:20:10.744756+0000","last_became_active":"2024-03-06T07:20:09.523876+0000","last_became_peered":"2024-03-06T07:20:09.523876+0000","last_unstale":"2024-03-06T07:20:10.744756+0000","last_undegraded":"2024-03-06T07:20:10.744756+0000","last_fullsized":"2024-03-06T07:20:10.744756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:10.906347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273623+0000","last_change":"2024-03-06T07:20:04.736879+0000","last_active":"2024-03-06T07:20:11.273623+0000","last_peered":"2024-03-06T07:20:11.273623+0000","last_clean":"2024-03-06T07:20:11.273623+0000","last_became_active":"2024-03-06T07:20:04.736271+0000","last_became_peered":"2024-03-06T07:20:04.736271+0000","last_unstale":"2024-03-06T07:20:11.273623+0000","last_undegraded":"2024-03-06T07:20:11.273623+0000","last_fullsized":"2024-03-06T07:20:11.273623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273465+0000","last_change":"2024-03-06T07:20:03.721336+0000","last_active":"2024-03-06T07:20:11.273465+0000","last_peered":"2024-03-06T07:20:11.273465+0000","last_clean":"2024-03-06T07:20:11.273465+0000","last_became_active":"2024-03-06T07:20:03.718827+0000","last_became_peered":"2024-03-06T07:20:03.718827+0000","last_unstale":"2024-03-06T07:20:11.273465+0000","last_undegraded":"2024-03-06T07:20:11.273465+0000","last_fullsized":"2024-03-06T07:20:11.273465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:18.536755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772927+0000","last_change":"2024-03-06T07:20:08.756386+0000","last_active":"2024-03-06T07:20:10.772927+0000","last_peered":"2024-03-06T07:20:10.772927+0000","last_clean":"2024-03-06T07:20:10.772927+0000","last_became_active":"2024-03-06T07:20:08.755970+0000","last_became_peered":"2024-03-06T07:20:08.755970+0000","last_unstale":"2024-03-06T07:20:10.772927+0000","last_undegraded":"2024-03-06T07:20:10.772927+0000","last_fullsized":"2024-03-06T07:20:10.772927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273864+0000","last_change":"2024-03-06T07:20:04.737597+0000","last_active":"2024-03-06T07:20:11.273864+0000","last_peered":"2024-03-06T07:20:11.273864+0000","last_clean":"2024-03-06T07:20:11.273864+0000","last_became_active":"2024-03-06T07:20:04.735046+0000","last_became_peered":"2024-03-06T07:20:04.735046+0000","last_unstale":"2024-03-06T07:20:11.273864+0000","last_undegraded":"2024-03-06T07:20:11.273864+0000","last_fullsized":"2024-03-06T07:20:11.273864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:16.868161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877519+0000","last_change":"2024-03-06T07:20:03.712917+0000","last_active":"2024-03-06T07:20:10.877519+0000","last_peered":"2024-03-06T07:20:10.877519+0000","last_clean":"2024-03-06T07:20:10.877519+0000","last_became_active":"2024-03-06T07:20:03.712584+0000","last_became_peered":"2024-03-06T07:20:03.712584+0000","last_unstale":"2024-03-06T07:20:10.877519+0000","last_undegraded":"2024-03-06T07:20:10.877519+0000","last_fullsized":"2024-03-06T07:20:10.877519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272841+0000","last_change":"2024-03-06T07:20:09.525362+0000","last_active":"2024-03-06T07:20:11.272841+0000","last_peered":"2024-03-06T07:20:11.272841+0000","last_clean":"2024-03-06T07:20:11.272841+0000","last_became_active":"2024-03-06T07:20:09.525026+0000","last_became_peered":"2024-03-06T07:20:09.525026+0000","last_unstale":"2024-03-06T07:20:11.272841+0000","last_undegraded":"2024-03-06T07:20:11.272841+0000","last_fullsized":"2024-03-06T07:20:11.272841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:55:20.377540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273503+0000","last_change":"2024-03-06T07:20:04.730553+0000","last_active":"2024-03-06T07:20:11.273503+0000","last_peered":"2024-03-06T07:20:11.273503+0000","last_clean":"2024-03-06T07:20:11.273503+0000","last_became_active":"2024-03-06T07:20:04.730222+0000","last_became_peered":"2024-03-06T07:20:04.730222+0000","last_unstale":"2024-03-06T07:20:11.273503+0000","last_undegraded":"2024-03-06T07:20:11.273503+0000","last_fullsized":"2024-03-06T07:20:11.273503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746578+0000","last_change":"2024-03-06T07:20:03.718268+0000","last_active":"2024-03-06T07:20:10.746578+0000","last_peered":"2024-03-06T07:20:10.746578+0000","last_clean":"2024-03-06T07:20:10.746578+0000","last_became_active":"2024-03-06T07:20:03.717969+0000","last_became_peered":"2024-03-06T07:20:03.717969+0000","last_unstale":"2024-03-06T07:20:10.746578+0000","last_undegraded":"2024-03-06T07:20:10.746578+0000","last_fullsized":"2024-03-06T07:20:10.746578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745335+0000","last_change":"2024-03-06T07:20:08.756182+0000","last_active":"2024-03-06T07:20:10.745335+0000","last_peered":"2024-03-06T07:20:10.745335+0000","last_clean":"2024-03-06T07:20:10.745335+0000","last_became_active":"2024-03-06T07:20:08.755784+0000","last_became_peered":"2024-03-06T07:20:08.755784+0000","last_unstale":"2024-03-06T07:20:10.745335+0000","last_undegraded":"2024-03-06T07:20:10.745335+0000","last_fullsized":"2024-03-06T07:20:10.745335+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:27.602522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273029+0000","last_change":"2024-03-06T07:20:04.729827+0000","last_active":"2024-03-06T07:20:11.273029+0000","last_peered":"2024-03-06T07:20:11.273029+0000","last_clean":"2024-03-06T07:20:11.273029+0000","last_became_active":"2024-03-06T07:20:04.729566+0000","last_became_peered":"2024-03-06T07:20:04.729566+0000","last_unstale":"2024-03-06T07:20:11.273029+0000","last_undegraded":"2024-03-06T07:20:11.273029+0000","last_fullsized":"2024-03-06T07:20:11.273029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877244+0000","last_change":"2024-03-06T07:20:03.723471+0000","last_active":"2024-03-06T07:20:10.877244+0000","last_peered":"2024-03-06T07:20:10.877244+0000","last_clean":"2024-03-06T07:20:10.877244+0000","last_became_active":"2024-03-06T07:20:03.723192+0000","last_became_peered":"2024-03-06T07:20:03.723192+0000","last_unstale":"2024-03-06T07:20:10.877244+0000","last_undegraded":"2024-03-06T07:20:10.877244+0000","last_fullsized":"2024-03-06T07:20:10.877244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772877+0000","last_change":"2024-03-06T07:20:08.760930+0000","last_active":"2024-03-06T07:20:10.772877+0000","last_peered":"2024-03-06T07:20:10.772877+0000","last_clean":"2024-03-06T07:20:10.772877+0000","last_became_active":"2024-03-06T07:20:08.760027+0000","last_became_peered":"2024-03-06T07:20:08.760027+0000","last_unstale":"2024-03-06T07:20:10.772877+0000","last_undegraded":"2024-03-06T07:20:10.772877+0000","last_fullsized":"2024-03-06T07:20:10.772877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744694+0000","last_change":"2024-03-06T07:20:04.717033+0000","last_active":"2024-03-06T07:20:10.744694+0000","last_peered":"2024-03-06T07:20:10.744694+0000","last_clean":"2024-03-06T07:20:10.744694+0000","last_became_active":"2024-03-06T07:20:04.716742+0000","last_became_peered":"2024-03-06T07:20:04.716742+0000","last_unstale":"2024-03-06T07:20:10.744694+0000","last_undegraded":"2024-03-06T07:20:10.744694+0000","last_fullsized":"2024-03-06T07:20:10.744694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745234+0000","last_change":"2024-03-06T07:20:03.717805+0000","last_active":"2024-03-06T07:20:10.745234+0000","last_peered":"2024-03-06T07:20:10.745234+0000","last_clean":"2024-03-06T07:20:10.745234+0000","last_became_active":"2024-03-06T07:20:03.717048+0000","last_became_peered":"2024-03-06T07:20:03.717048+0000","last_unstale":"2024-03-06T07:20:10.745234+0000","last_undegraded":"2024-03-06T07:20:10.745234+0000","last_fullsized":"2024-03-06T07:20:10.745234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744965+0000","last_change":"2024-03-06T07:20:09.525949+0000","last_active":"2024-03-06T07:20:10.744965+0000","last_peered":"2024-03-06T07:20:10.744965+0000","last_clean":"2024-03-06T07:20:10.744965+0000","last_became_active":"2024-03-06T07:20:09.525427+0000","last_became_peered":"2024-03-06T07:20:09.525427+0000","last_unstale":"2024-03-06T07:20:10.744965+0000","last_undegraded":"2024-03-06T07:20:10.744965+0000","last_fullsized":"2024-03-06T07:20:10.744965+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:41:44.763996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273942+0000","last_change":"2024-03-06T07:20:04.732333+0000","last_active":"2024-03-06T07:20:11.273942+0000","last_peered":"2024-03-06T07:20:11.273942+0000","last_clean":"2024-03-06T07:20:11.273942+0000","last_became_active":"2024-03-06T07:20:04.732069+0000","last_became_peered":"2024-03-06T07:20:04.732069+0000","last_unstale":"2024-03-06T07:20:11.273942+0000","last_undegraded":"2024-03-06T07:20:11.273942+0000","last_fullsized":"2024-03-06T07:20:11.273942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274155+0000","last_change":"2024-03-06T07:20:03.719609+0000","last_active":"2024-03-06T07:20:11.274155+0000","last_peered":"2024-03-06T07:20:11.274155+0000","last_clean":"2024-03-06T07:20:11.274155+0000","last_became_active":"2024-03-06T07:20:03.719325+0000","last_became_peered":"2024-03-06T07:20:03.719325+0000","last_unstale":"2024-03-06T07:20:11.274155+0000","last_undegraded":"2024-03-06T07:20:11.274155+0000","last_fullsized":"2024-03-06T07:20:11.274155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:13:18.549180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744816+0000","last_change":"2024-03-06T07:20:09.524644+0000","last_active":"2024-03-06T07:20:10.744816+0000","last_peered":"2024-03-06T07:20:10.744816+0000","last_clean":"2024-03-06T07:20:10.744816+0000","last_became_active":"2024-03-06T07:20:09.524220+0000","last_became_peered":"2024-03-06T07:20:09.524220+0000","last_unstale":"2024-03-06T07:20:10.744816+0000","last_undegraded":"2024-03-06T07:20:10.744816+0000","last_fullsized":"2024-03-06T07:20:10.744816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:30.841732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772642+0000","last_change":"2024-03-06T07:20:04.735008+0000","last_active":"2024-03-06T07:20:10.772642+0000","last_peered":"2024-03-06T07:20:10.772642+0000","last_clean":"2024-03-06T07:20:10.772642+0000","last_became_active":"2024-03-06T07:20:04.734709+0000","last_became_peered":"2024-03-06T07:20:04.734709+0000","last_unstale":"2024-03-06T07:20:10.772642+0000","last_undegraded":"2024-03-06T07:20:10.772642+0000","last_fullsized":"2024-03-06T07:20:10.772642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.773006+0000","last_change":"2024-03-06T07:20:03.722267+0000","last_active":"2024-03-06T07:20:10.773006+0000","last_peered":"2024-03-06T07:20:10.773006+0000","last_clean":"2024-03-06T07:20:10.773006+0000","last_became_active":"2024-03-06T07:20:03.721952+0000","last_became_peered":"2024-03-06T07:20:03.721952+0000","last_unstale":"2024-03-06T07:20:10.773006+0000","last_undegraded":"2024-03-06T07:20:10.773006+0000","last_fullsized":"2024-03-06T07:20:10.773006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273402+0000","last_change":"2024-03-06T07:20:09.524376+0000","last_active":"2024-03-06T07:20:11.273402+0000","last_peered":"2024-03-06T07:20:11.273402+0000","last_clean":"2024-03-06T07:20:11.273402+0000","last_became_active":"2024-03-06T07:20:09.524010+0000","last_became_peered":"2024-03-06T07:20:09.524010+0000","last_unstale":"2024-03-06T07:20:11.273402+0000","last_undegraded":"2024-03-06T07:20:11.273402+0000","last_fullsized":"2024-03-06T07:20:11.273402+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:50.092001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273678+0000","last_change":"2024-03-06T07:20:04.734379+0000","last_active":"2024-03-06T07:20:11.273678+0000","last_peered":"2024-03-06T07:20:11.273678+0000","last_clean":"2024-03-06T07:20:11.273678+0000","last_became_active":"2024-03-06T07:20:04.733727+0000","last_became_peered":"2024-03-06T07:20:04.733727+0000","last_unstale":"2024-03-06T07:20:11.273678+0000","last_undegraded":"2024-03-06T07:20:11.273678+0000","last_fullsized":"2024-03-06T07:20:11.273678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:14.073340+0000","last_change":"2024-03-06T07:20:03.718296+0000","last_active":"2024-03-06T07:20:14.073340+0000","last_peered":"2024-03-06T07:20:14.073340+0000","last_clean":"2024-03-06T07:20:14.073340+0000","last_became_active":"2024-03-06T07:20:03.718015+0000","last_became_peered":"2024-03-06T07:20:03.718015+0000","last_unstale":"2024-03-06T07:20:14.073340+0000","last_undegraded":"2024-03-06T07:20:14.073340+0000","last_fullsized":"2024-03-06T07:20:14.073340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:04.896733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273542+0000","last_change":"2024-03-06T07:20:08.764548+0000","last_active":"2024-03-06T07:20:11.273542+0000","last_peered":"2024-03-06T07:20:11.273542+0000","last_clean":"2024-03-06T07:20:11.273542+0000","last_became_active":"2024-03-06T07:20:08.764175+0000","last_became_peered":"2024-03-06T07:20:08.764175+0000","last_unstale":"2024-03-06T07:20:11.273542+0000","last_undegraded":"2024-03-06T07:20:11.273542+0000","last_fullsized":"2024-03-06T07:20:11.273542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:32.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746370+0000","last_change":"2024-03-06T07:20:04.733069+0000","last_active":"2024-03-06T07:20:10.746370+0000","last_peered":"2024-03-06T07:20:10.746370+0000","last_clean":"2024-03-06T07:20:10.746370+0000","last_became_active":"2024-03-06T07:20:04.732656+0000","last_became_peered":"2024-03-06T07:20:04.732656+0000","last_unstale":"2024-03-06T07:20:10.746370+0000","last_undegraded":"2024-03-06T07:20:10.746370+0000","last_fullsized":"2024-03-06T07:20:10.746370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273433+0000","last_change":"2024-03-06T07:20:03.728079+0000","last_active":"2024-03-06T07:20:11.273433+0000","last_peered":"2024-03-06T07:20:11.273433+0000","last_clean":"2024-03-06T07:20:11.273433+0000","last_became_active":"2024-03-06T07:20:03.727628+0000","last_became_peered":"2024-03-06T07:20:03.727628+0000","last_unstale":"2024-03-06T07:20:11.273433+0000","last_undegraded":"2024-03-06T07:20:11.273433+0000","last_fullsized":"2024-03-06T07:20:11.273433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:20.214171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746197+0000","last_change":"2024-03-06T07:20:08.753762+0000","last_active":"2024-03-06T07:20:10.746197+0000","last_peered":"2024-03-06T07:20:10.746197+0000","last_clean":"2024-03-06T07:20:10.746197+0000","last_became_active":"2024-03-06T07:20:08.753331+0000","last_became_peered":"2024-03-06T07:20:08.753331+0000","last_unstale":"2024-03-06T07:20:10.746197+0000","last_undegraded":"2024-03-06T07:20:10.746197+0000","last_fullsized":"2024-03-06T07:20:10.746197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:04.517972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772817+0000","last_change":"2024-03-06T07:20:04.731376+0000","last_active":"2024-03-06T07:20:10.772817+0000","last_peered":"2024-03-06T07:20:10.772817+0000","last_clean":"2024-03-06T07:20:10.772817+0000","last_became_active":"2024-03-06T07:20:04.730731+0000","last_became_peered":"2024-03-06T07:20:04.730731+0000","last_unstale":"2024-03-06T07:20:10.772817+0000","last_undegraded":"2024-03-06T07:20:10.772817+0000","last_fullsized":"2024-03-06T07:20:10.772817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:57.553850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273372+0000","last_change":"2024-03-06T07:20:03.727603+0000","last_active":"2024-03-06T07:20:11.273372+0000","last_peered":"2024-03-06T07:20:11.273372+0000","last_clean":"2024-03-06T07:20:11.273372+0000","last_became_active":"2024-03-06T07:20:03.727238+0000","last_became_peered":"2024-03-06T07:20:03.727238+0000","last_unstale":"2024-03-06T07:20:11.273372+0000","last_undegraded":"2024-03-06T07:20:11.273372+0000","last_fullsized":"2024-03-06T07:20:11.273372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:54:41.374880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744238+0000","last_change":"2024-03-06T07:20:08.759817+0000","last_active":"2024-03-06T07:20:10.744238+0000","last_peered":"2024-03-06T07:20:10.744238+0000","last_clean":"2024-03-06T07:20:10.744238+0000","last_became_active":"2024-03-06T07:20:08.759430+0000","last_became_peered":"2024-03-06T07:20:08.759430+0000","last_unstale":"2024-03-06T07:20:10.744238+0000","last_undegraded":"2024-03-06T07:20:10.744238+0000","last_fullsized":"2024-03-06T07:20:10.744238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:59.350405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273280+0000","last_change":"2024-03-06T07:20:04.736807+0000","last_active":"2024-03-06T07:20:11.273280+0000","last_peered":"2024-03-06T07:20:11.273280+0000","last_clean":"2024-03-06T07:20:11.273280+0000","last_became_active":"2024-03-06T07:20:04.736552+0000","last_became_peered":"2024-03-06T07:20:04.736552+0000","last_unstale":"2024-03-06T07:20:11.273280+0000","last_undegraded":"2024-03-06T07:20:11.273280+0000","last_fullsized":"2024-03-06T07:20:11.273280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745647+0000","last_change":"2024-03-06T07:20:03.725869+0000","last_active":"2024-03-06T07:20:10.745647+0000","last_peered":"2024-03-06T07:20:10.745647+0000","last_clean":"2024-03-06T07:20:10.745647+0000","last_became_active":"2024-03-06T07:20:03.725563+0000","last_became_peered":"2024-03-06T07:20:03.725563+0000","last_unstale":"2024-03-06T07:20:10.745647+0000","last_undegraded":"2024-03-06T07:20:10.745647+0000","last_fullsized":"2024-03-06T07:20:10.745647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:42.520164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744191+0000","last_change":"2024-03-06T07:20:09.527243+0000","last_active":"2024-03-06T07:20:10.744191+0000","last_peered":"2024-03-06T07:20:10.744191+0000","last_clean":"2024-03-06T07:20:10.744191+0000","last_became_active":"2024-03-06T07:20:09.526607+0000","last_became_peered":"2024-03-06T07:20:09.526607+0000","last_unstale":"2024-03-06T07:20:10.744191+0000","last_undegraded":"2024-03-06T07:20:10.744191+0000","last_fullsized":"2024-03-06T07:20:10.744191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:34.603979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772826+0000","last_change":"2024-03-06T07:20:09.524593+0000","last_active":"2024-03-06T07:20:10.772826+0000","last_peered":"2024-03-06T07:20:10.772826+0000","last_clean":"2024-03-06T07:20:10.772826+0000","last_became_active":"2024-03-06T07:20:09.524244+0000","last_became_peered":"2024-03-06T07:20:09.524244+0000","last_unstale":"2024-03-06T07:20:10.772826+0000","last_undegraded":"2024-03-06T07:20:10.772826+0000","last_fullsized":"2024-03-06T07:20:10.772826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:06.287810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272939+0000","last_change":"2024-03-06T07:20:04.718433+0000","last_active":"2024-03-06T07:20:11.272939+0000","last_peered":"2024-03-06T07:20:11.272939+0000","last_clean":"2024-03-06T07:20:11.272939+0000","last_became_active":"2024-03-06T07:20:04.718154+0000","last_became_peered":"2024-03-06T07:20:04.718154+0000","last_unstale":"2024-03-06T07:20:11.272939+0000","last_undegraded":"2024-03-06T07:20:11.272939+0000","last_fullsized":"2024-03-06T07:20:11.272939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273133+0000","last_change":"2024-03-06T07:20:03.727685+0000","last_active":"2024-03-06T07:20:11.273133+0000","last_peered":"2024-03-06T07:20:11.273133+0000","last_clean":"2024-03-06T07:20:11.273133+0000","last_became_active":"2024-03-06T07:20:03.727363+0000","last_became_peered":"2024-03-06T07:20:03.727363+0000","last_unstale":"2024-03-06T07:20:11.273133+0000","last_undegraded":"2024-03-06T07:20:11.273133+0000","last_fullsized":"2024-03-06T07:20:11.273133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:46.302315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745586+0000","last_change":"2024-03-06T07:20:08.760387+0000","last_active":"2024-03-06T07:20:10.745586+0000","last_peered":"2024-03-06T07:20:10.745586+0000","last_clean":"2024-03-06T07:20:10.745586+0000","last_became_active":"2024-03-06T07:20:08.760017+0000","last_became_peered":"2024-03-06T07:20:08.760017+0000","last_unstale":"2024-03-06T07:20:10.745586+0000","last_undegraded":"2024-03-06T07:20:10.745586+0000","last_fullsized":"2024-03-06T07:20:10.745586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:20.905249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744172+0000","last_change":"2024-03-06T07:20:04.721795+0000","last_active":"2024-03-06T07:20:10.744172+0000","last_peered":"2024-03-06T07:20:10.744172+0000","last_clean":"2024-03-06T07:20:10.744172+0000","last_became_active":"2024-03-06T07:20:04.720723+0000","last_became_peered":"2024-03-06T07:20:04.720723+0000","last_unstale":"2024-03-06T07:20:10.744172+0000","last_undegraded":"2024-03-06T07:20:10.744172+0000","last_fullsized":"2024-03-06T07:20:10.744172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744343+0000","last_change":"2024-03-06T07:20:03.722350+0000","last_active":"2024-03-06T07:20:10.744343+0000","last_peered":"2024-03-06T07:20:10.744343+0000","last_clean":"2024-03-06T07:20:10.744343+0000","last_became_active":"2024-03-06T07:20:03.722059+0000","last_became_peered":"2024-03-06T07:20:03.722059+0000","last_unstale":"2024-03-06T07:20:10.744343+0000","last_undegraded":"2024-03-06T07:20:10.744343+0000","last_fullsized":"2024-03-06T07:20:10.744343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:39.399954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744977+0000","last_change":"2024-03-06T07:20:03.717393+0000","last_active":"2024-03-06T07:20:10.744977+0000","last_peered":"2024-03-06T07:20:10.744977+0000","last_clean":"2024-03-06T07:20:10.744977+0000","last_became_active":"2024-03-06T07:20:03.717051+0000","last_became_peered":"2024-03-06T07:20:03.717051+0000","last_unstale":"2024-03-06T07:20:10.744977+0000","last_undegraded":"2024-03-06T07:20:10.744977+0000","last_fullsized":"2024-03-06T07:20:10.744977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:44.520135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772980+0000","last_change":"2024-03-06T07:20:04.718195+0000","last_active":"2024-03-06T07:20:10.772980+0000","last_peered":"2024-03-06T07:20:10.772980+0000","last_clean":"2024-03-06T07:20:10.772980+0000","last_became_active":"2024-03-06T07:20:04.717447+0000","last_became_peered":"2024-03-06T07:20:04.717447+0000","last_unstale":"2024-03-06T07:20:10.772980+0000","last_undegraded":"2024-03-06T07:20:10.772980+0000","last_fullsized":"2024-03-06T07:20:10.772980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877565+0000","last_change":"2024-03-06T07:20:08.759429+0000","last_active":"2024-03-06T07:20:10.877565+0000","last_peered":"2024-03-06T07:20:10.877565+0000","last_clean":"2024-03-06T07:20:10.877565+0000","last_became_active":"2024-03-06T07:20:08.759038+0000","last_became_peered":"2024-03-06T07:20:08.759038+0000","last_unstale":"2024-03-06T07:20:10.877565+0000","last_undegraded":"2024-03-06T07:20:10.877565+0000","last_fullsized":"2024-03-06T07:20:10.877565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:33.125882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744524+0000","last_change":"2024-03-06T07:20:03.730392+0000","last_active":"2024-03-06T07:20:10.744524+0000","last_peered":"2024-03-06T07:20:10.744524+0000","last_clean":"2024-03-06T07:20:10.744524+0000","last_became_active":"2024-03-06T07:20:03.730088+0000","last_became_peered":"2024-03-06T07:20:03.730088+0000","last_unstale":"2024-03-06T07:20:10.744524+0000","last_undegraded":"2024-03-06T07:20:10.744524+0000","last_fullsized":"2024-03-06T07:20:10.744524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:02.689592+0000","last_clean_scrub_stamp":"2024-03-06T07:20:02.689592+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:34:41.216196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745946+0000","last_change":"2024-03-06T07:20:04.724861+0000","last_active":"2024-03-06T07:20:10.745946+0000","last_peered":"2024-03-06T07:20:10.745946+0000","last_clean":"2024-03-06T07:20:10.745946+0000","last_became_active":"2024-03-06T07:20:04.724584+0000","last_became_peered":"2024-03-06T07:20:04.724584+0000","last_unstale":"2024-03-06T07:20:10.745946+0000","last_undegraded":"2024-03-06T07:20:10.745946+0000","last_fullsized":"2024-03-06T07:20:10.745946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:05.902251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744887+0000","last_change":"2024-03-06T07:20:08.758137+0000","last_active":"2024-03-06T07:20:10.744887+0000","last_peered":"2024-03-06T07:20:10.744887+0000","last_clean":"2024-03-06T07:20:10.744887+0000","last_became_active":"2024-03-06T07:20:08.757711+0000","last_became_peered":"2024-03-06T07:20:08.757711+0000","last_unstale":"2024-03-06T07:20:10.744887+0000","last_undegraded":"2024-03-06T07:20:10.744887+0000","last_fullsized":"2024-03-06T07:20:10.744887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:07.732199+0000","last_clean_scrub_stamp":"2024-03-06T07:20:07.732199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:53:47.463176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744591+0000","last_change":"2024-03-06T07:20:04.724223+0000","last_active":"2024-03-06T07:20:10.744591+0000","last_peered":"2024-03-06T07:20:10.744591+0000","last_clean":"2024-03-06T07:20:10.744591+0000","last_became_active":"2024-03-06T07:20:04.723946+0000","last_became_peered":"2024-03-06T07:20:04.723946+0000","last_unstale":"2024-03-06T07:20:10.744591+0000","last_undegraded":"2024-03-06T07:20:10.744591+0000","last_fullsized":"2024-03-06T07:20:10.744591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:16.554288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877317+0000","last_change":"2024-03-06T07:20:04.730151+0000","last_active":"2024-03-06T07:20:10.877317+0000","last_peered":"2024-03-06T07:20:10.877317+0000","last_clean":"2024-03-06T07:20:10.877317+0000","last_became_active":"2024-03-06T07:20:04.729835+0000","last_became_peered":"2024-03-06T07:20:04.729835+0000","last_unstale":"2024-03-06T07:20:10.877317+0000","last_undegraded":"2024-03-06T07:20:10.877317+0000","last_fullsized":"2024-03-06T07:20:10.877317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:19.542273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274150+0000","last_change":"2024-03-06T07:20:04.734333+0000","last_active":"2024-03-06T07:20:11.274150+0000","last_peered":"2024-03-06T07:20:11.274150+0000","last_clean":"2024-03-06T07:20:11.274150+0000","last_became_active":"2024-03-06T07:20:04.734075+0000","last_became_peered":"2024-03-06T07:20:04.734075+0000","last_unstale":"2024-03-06T07:20:11.274150+0000","last_undegraded":"2024-03-06T07:20:11.274150+0000","last_fullsized":"2024-03-06T07:20:11.274150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:28.221496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772954+0000","last_change":"2024-03-06T07:20:04.722013+0000","last_active":"2024-03-06T07:20:10.772954+0000","last_peered":"2024-03-06T07:20:10.772954+0000","last_clean":"2024-03-06T07:20:10.772954+0000","last_became_active":"2024-03-06T07:20:04.721746+0000","last_became_peered":"2024-03-06T07:20:04.721746+0000","last_unstale":"2024-03-06T07:20:10.772954+0000","last_undegraded":"2024-03-06T07:20:10.772954+0000","last_fullsized":"2024-03-06T07:20:10.772954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744882+0000","last_change":"2024-03-06T07:20:04.732536+0000","last_active":"2024-03-06T07:20:10.744882+0000","last_peered":"2024-03-06T07:20:10.744882+0000","last_clean":"2024-03-06T07:20:10.744882+0000","last_became_active":"2024-03-06T07:20:04.728058+0000","last_became_peered":"2024-03-06T07:20:04.728058+0000","last_unstale":"2024-03-06T07:20:10.744882+0000","last_undegraded":"2024-03-06T07:20:10.744882+0000","last_fullsized":"2024-03-06T07:20:10.744882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273227+0000","last_change":"2024-03-06T07:20:04.723910+0000","last_active":"2024-03-06T07:20:11.273227+0000","last_peered":"2024-03-06T07:20:11.273227+0000","last_clean":"2024-03-06T07:20:11.273227+0000","last_became_active":"2024-03-06T07:20:04.722877+0000","last_became_peered":"2024-03-06T07:20:04.722877+0000","last_unstale":"2024-03-06T07:20:11.273227+0000","last_undegraded":"2024-03-06T07:20:11.273227+0000","last_fullsized":"2024-03-06T07:20:11.273227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745929+0000","last_change":"2024-03-06T07:20:04.734964+0000","last_active":"2024-03-06T07:20:10.745929+0000","last_peered":"2024-03-06T07:20:10.745929+0000","last_clean":"2024-03-06T07:20:10.745929+0000","last_became_active":"2024-03-06T07:20:04.734459+0000","last_became_peered":"2024-03-06T07:20:04.734459+0000","last_unstale":"2024-03-06T07:20:10.745929+0000","last_undegraded":"2024-03-06T07:20:10.745929+0000","last_fullsized":"2024-03-06T07:20:10.745929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744920+0000","last_change":"2024-03-06T07:20:04.725185+0000","last_active":"2024-03-06T07:20:10.744920+0000","last_peered":"2024-03-06T07:20:10.744920+0000","last_clean":"2024-03-06T07:20:10.744920+0000","last_became_active":"2024-03-06T07:20:04.724888+0000","last_became_peered":"2024-03-06T07:20:04.724888+0000","last_unstale":"2024-03-06T07:20:10.744920+0000","last_undegraded":"2024-03-06T07:20:10.744920+0000","last_fullsized":"2024-03-06T07:20:10.744920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273048+0000","last_change":"2024-03-06T07:20:04.735180+0000","last_active":"2024-03-06T07:20:11.273048+0000","last_peered":"2024-03-06T07:20:11.273048+0000","last_clean":"2024-03-06T07:20:11.273048+0000","last_became_active":"2024-03-06T07:20:04.733178+0000","last_became_peered":"2024-03-06T07:20:04.733178+0000","last_unstale":"2024-03-06T07:20:11.273048+0000","last_undegraded":"2024-03-06T07:20:11.273048+0000","last_fullsized":"2024-03-06T07:20:11.273048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:09.156654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744251+0000","last_change":"2024-03-06T07:20:04.726353+0000","last_active":"2024-03-06T07:20:10.744251+0000","last_peered":"2024-03-06T07:20:10.744251+0000","last_clean":"2024-03-06T07:20:10.744251+0000","last_became_active":"2024-03-06T07:20:04.725518+0000","last_became_peered":"2024-03-06T07:20:04.725518+0000","last_unstale":"2024-03-06T07:20:10.744251+0000","last_undegraded":"2024-03-06T07:20:10.744251+0000","last_fullsized":"2024-03-06T07:20:10.744251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:15.136148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274062+0000","last_change":"2024-03-06T07:20:04.736206+0000","last_active":"2024-03-06T07:20:11.274062+0000","last_peered":"2024-03-06T07:20:11.274062+0000","last_clean":"2024-03-06T07:20:11.274062+0000","last_became_active":"2024-03-06T07:20:04.735949+0000","last_became_peered":"2024-03-06T07:20:04.735949+0000","last_unstale":"2024-03-06T07:20:11.274062+0000","last_undegraded":"2024-03-06T07:20:11.274062+0000","last_fullsized":"2024-03-06T07:20:11.274062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:13:37.128857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772746+0000","last_change":"2024-03-06T07:20:04.728579+0000","last_active":"2024-03-06T07:20:10.772746+0000","last_peered":"2024-03-06T07:20:10.772746+0000","last_clean":"2024-03-06T07:20:10.772746+0000","last_became_active":"2024-03-06T07:20:04.728300+0000","last_became_peered":"2024-03-06T07:20:04.728300+0000","last_unstale":"2024-03-06T07:20:10.772746+0000","last_undegraded":"2024-03-06T07:20:10.772746+0000","last_fullsized":"2024-03-06T07:20:10.772746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745423+0000","last_change":"2024-03-06T07:20:04.724410+0000","last_active":"2024-03-06T07:20:10.745423+0000","last_peered":"2024-03-06T07:20:10.745423+0000","last_clean":"2024-03-06T07:20:10.745423+0000","last_became_active":"2024-03-06T07:20:04.723916+0000","last_became_peered":"2024-03-06T07:20:04.723916+0000","last_unstale":"2024-03-06T07:20:10.745423+0000","last_undegraded":"2024-03-06T07:20:10.745423+0000","last_fullsized":"2024-03-06T07:20:10.745423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:16:09.685149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.274000+0000","last_change":"2024-03-06T07:20:04.734607+0000","last_active":"2024-03-06T07:20:11.274000+0000","last_peered":"2024-03-06T07:20:11.274000+0000","last_clean":"2024-03-06T07:20:11.274000+0000","last_became_active":"2024-03-06T07:20:04.734081+0000","last_became_peered":"2024-03-06T07:20:04.734081+0000","last_unstale":"2024-03-06T07:20:11.274000+0000","last_undegraded":"2024-03-06T07:20:11.274000+0000","last_fullsized":"2024-03-06T07:20:11.274000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:59:46.855071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.877491+0000","last_change":"2024-03-06T07:20:04.726465+0000","last_active":"2024-03-06T07:20:10.877491+0000","last_peered":"2024-03-06T07:20:10.877491+0000","last_clean":"2024-03-06T07:20:10.877491+0000","last_became_active":"2024-03-06T07:20:04.726114+0000","last_became_peered":"2024-03-06T07:20:04.726114+0000","last_unstale":"2024-03-06T07:20:10.877491+0000","last_undegraded":"2024-03-06T07:20:10.877491+0000","last_fullsized":"2024-03-06T07:20:10.877491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273756+0000","last_change":"2024-03-06T07:20:04.738589+0000","last_active":"2024-03-06T07:20:11.273756+0000","last_peered":"2024-03-06T07:20:11.273756+0000","last_clean":"2024-03-06T07:20:11.273756+0000","last_became_active":"2024-03-06T07:20:04.737924+0000","last_became_peered":"2024-03-06T07:20:04.737924+0000","last_unstale":"2024-03-06T07:20:11.273756+0000","last_undegraded":"2024-03-06T07:20:11.273756+0000","last_fullsized":"2024-03-06T07:20:11.273756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:06.869258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272964+0000","last_change":"2024-03-06T07:20:04.726607+0000","last_active":"2024-03-06T07:20:11.272964+0000","last_peered":"2024-03-06T07:20:11.272964+0000","last_clean":"2024-03-06T07:20:11.272964+0000","last_became_active":"2024-03-06T07:20:04.726337+0000","last_became_peered":"2024-03-06T07:20:04.726337+0000","last_unstale":"2024-03-06T07:20:11.272964+0000","last_undegraded":"2024-03-06T07:20:11.272964+0000","last_fullsized":"2024-03-06T07:20:11.272964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273000+0000","last_change":"2024-03-06T07:20:04.719555+0000","last_active":"2024-03-06T07:20:11.273000+0000","last_peered":"2024-03-06T07:20:11.273000+0000","last_clean":"2024-03-06T07:20:11.273000+0000","last_became_active":"2024-03-06T07:20:04.719273+0000","last_became_peered":"2024-03-06T07:20:04.719273+0000","last_unstale":"2024-03-06T07:20:11.273000+0000","last_undegraded":"2024-03-06T07:20:11.273000+0000","last_fullsized":"2024-03-06T07:20:11.273000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772701+0000","last_change":"2024-03-06T07:20:04.724038+0000","last_active":"2024-03-06T07:20:10.772701+0000","last_peered":"2024-03-06T07:20:10.772701+0000","last_clean":"2024-03-06T07:20:10.772701+0000","last_became_active":"2024-03-06T07:20:04.723527+0000","last_became_peered":"2024-03-06T07:20:04.723527+0000","last_unstale":"2024-03-06T07:20:10.772701+0000","last_undegraded":"2024-03-06T07:20:10.772701+0000","last_fullsized":"2024-03-06T07:20:10.772701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:32.763183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744248+0000","last_change":"2024-03-06T07:20:04.730004+0000","last_active":"2024-03-06T07:20:10.744248+0000","last_peered":"2024-03-06T07:20:10.744248+0000","last_clean":"2024-03-06T07:20:10.744248+0000","last_became_active":"2024-03-06T07:20:04.729702+0000","last_became_peered":"2024-03-06T07:20:04.729702+0000","last_unstale":"2024-03-06T07:20:10.744248+0000","last_undegraded":"2024-03-06T07:20:10.744248+0000","last_fullsized":"2024-03-06T07:20:10.744248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.746363+0000","last_change":"2024-03-06T07:20:04.729091+0000","last_active":"2024-03-06T07:20:10.746363+0000","last_peered":"2024-03-06T07:20:10.746363+0000","last_clean":"2024-03-06T07:20:10.746363+0000","last_became_active":"2024-03-06T07:20:04.728793+0000","last_became_peered":"2024-03-06T07:20:04.728793+0000","last_unstale":"2024-03-06T07:20:10.746363+0000","last_undegraded":"2024-03-06T07:20:10.746363+0000","last_fullsized":"2024-03-06T07:20:10.746363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:21.219689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745590+0000","last_change":"2024-03-06T07:20:04.730775+0000","last_active":"2024-03-06T07:20:10.745590+0000","last_peered":"2024-03-06T07:20:10.745590+0000","last_clean":"2024-03-06T07:20:10.745590+0000","last_became_active":"2024-03-06T07:20:04.730386+0000","last_became_peered":"2024-03-06T07:20:04.730386+0000","last_unstale":"2024-03-06T07:20:10.745590+0000","last_undegraded":"2024-03-06T07:20:10.745590+0000","last_fullsized":"2024-03-06T07:20:10.745590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:49.108722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.745842+0000","last_change":"2024-03-06T07:20:04.730885+0000","last_active":"2024-03-06T07:20:10.745842+0000","last_peered":"2024-03-06T07:20:10.745842+0000","last_clean":"2024-03-06T07:20:10.745842+0000","last_became_active":"2024-03-06T07:20:04.730294+0000","last_became_peered":"2024-03-06T07:20:04.730294+0000","last_unstale":"2024-03-06T07:20:10.745842+0000","last_undegraded":"2024-03-06T07:20:10.745842+0000","last_fullsized":"2024-03-06T07:20:10.745842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:39.048328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273816+0000","last_change":"2024-03-06T07:20:04.737430+0000","last_active":"2024-03-06T07:20:11.273816+0000","last_peered":"2024-03-06T07:20:11.273816+0000","last_clean":"2024-03-06T07:20:11.273816+0000","last_became_active":"2024-03-06T07:20:04.734870+0000","last_became_peered":"2024-03-06T07:20:04.734870+0000","last_unstale":"2024-03-06T07:20:11.273816+0000","last_undegraded":"2024-03-06T07:20:11.273816+0000","last_fullsized":"2024-03-06T07:20:11.273816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:48.178098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.744080+0000","last_change":"2024-03-06T07:20:04.732622+0000","last_active":"2024-03-06T07:20:10.744080+0000","last_peered":"2024-03-06T07:20:10.744080+0000","last_clean":"2024-03-06T07:20:10.744080+0000","last_became_active":"2024-03-06T07:20:04.732149+0000","last_became_peered":"2024-03-06T07:20:04.732149+0000","last_unstale":"2024-03-06T07:20:10.744080+0000","last_undegraded":"2024-03-06T07:20:10.744080+0000","last_fullsized":"2024-03-06T07:20:10.744080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:29.088793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.272952+0000","last_change":"2024-03-06T07:20:04.733067+0000","last_active":"2024-03-06T07:20:11.272952+0000","last_peered":"2024-03-06T07:20:11.272952+0000","last_clean":"2024-03-06T07:20:11.272952+0000","last_became_active":"2024-03-06T07:20:04.732805+0000","last_became_peered":"2024-03-06T07:20:04.732805+0000","last_unstale":"2024-03-06T07:20:11.272952+0000","last_undegraded":"2024-03-06T07:20:11.272952+0000","last_fullsized":"2024-03-06T07:20:11.272952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273984+0000","last_change":"2024-03-06T07:20:04.729682+0000","last_active":"2024-03-06T07:20:11.273984+0000","last_peered":"2024-03-06T07:20:11.273984+0000","last_clean":"2024-03-06T07:20:11.273984+0000","last_became_active":"2024-03-06T07:20:04.729407+0000","last_became_peered":"2024-03-06T07:20:04.729407+0000","last_unstale":"2024-03-06T07:20:11.273984+0000","last_undegraded":"2024-03-06T07:20:11.273984+0000","last_fullsized":"2024-03-06T07:20:11.273984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:45.266122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273787+0000","last_change":"2024-03-06T07:20:04.737815+0000","last_active":"2024-03-06T07:20:11.273787+0000","last_peered":"2024-03-06T07:20:11.273787+0000","last_clean":"2024-03-06T07:20:11.273787+0000","last_became_active":"2024-03-06T07:20:04.735276+0000","last_became_peered":"2024-03-06T07:20:04.735276+0000","last_unstale":"2024-03-06T07:20:11.273787+0000","last_undegraded":"2024-03-06T07:20:11.273787+0000","last_fullsized":"2024-03-06T07:20:11.273787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:23.565433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273449+0000","last_change":"2024-03-06T07:20:04.737108+0000","last_active":"2024-03-06T07:20:11.273449+0000","last_peered":"2024-03-06T07:20:11.273449+0000","last_clean":"2024-03-06T07:20:11.273449+0000","last_became_active":"2024-03-06T07:20:04.736608+0000","last_became_peered":"2024-03-06T07:20:04.736608+0000","last_unstale":"2024-03-06T07:20:11.273449+0000","last_undegraded":"2024-03-06T07:20:11.273449+0000","last_fullsized":"2024-03-06T07:20:11.273449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:13.997716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:10.772691+0000","last_change":"2024-03-06T07:20:04.720240+0000","last_active":"2024-03-06T07:20:10.772691+0000","last_peered":"2024-03-06T07:20:10.772691+0000","last_clean":"2024-03-06T07:20:10.772691+0000","last_became_active":"2024-03-06T07:20:04.719952+0000","last_became_peered":"2024-03-06T07:20:04.719952+0000","last_unstale":"2024-03-06T07:20:10.772691+0000","last_undegraded":"2024-03-06T07:20:10.772691+0000","last_fullsized":"2024-03-06T07:20:10.772691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:13.957015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273519+0000","last_change":"2024-03-06T07:20:04.719505+0000","last_active":"2024-03-06T07:20:11.273519+0000","last_peered":"2024-03-06T07:20:11.273519+0000","last_clean":"2024-03-06T07:20:11.273519+0000","last_became_active":"2024-03-06T07:20:04.718961+0000","last_became_peered":"2024-03-06T07:20:04.718961+0000","last_unstale":"2024-03-06T07:20:11.273519+0000","last_undegraded":"2024-03-06T07:20:11.273519+0000","last_fullsized":"2024-03-06T07:20:11.273519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:34.961922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-03-06T07:20:11.273711+0000","last_change":"2024-03-06T07:20:04.737365+0000","last_active":"2024-03-06T07:20:11.273711+0000","last_peered":"2024-03-06T07:20:11.273711+0000","last_clean":"2024-03-06T07:20:11.273711+0000","last_became_active":"2024-03-06T07:20:04.734552+0000","last_became_peered":"2024-03-06T07:20:04.734552+0000","last_unstale":"2024-03-06T07:20:11.273711+0000","last_undegraded":"2024-03-06T07:20:11.273711+0000","last_fullsized":"2024-03-06T07:20:11.273711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:20:03.695110+0000","last_clean_scrub_stamp":"2024-03-06T07:20:03.695110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:28.342654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":698343,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":344064,"data_stored":698399,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:20:27.858 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-06T07:20:27.858 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-06T07:20:27.858 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-06T07:20:27.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:20:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.992+0000 7eff13aa4640 1 Processor -- start 2024-03-06T07:20:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff13aa4640 1 -- start start 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff13aa4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff0c156ea0 0x7eff0c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff13aa4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c0ad310 con 0x7eff0c153990 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff13aa4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c1572a0 con 0x7eff0c1506b0 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff13aa4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c156450 con 0x7eff0c156ea0 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff1103f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7eff0c153990 0x7eff0c0acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58480/0 (socket says 172.21.15.17:58480) 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.993+0000 7eff12041640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff0c156ea0 0x7eff0c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff1103f640 1 -- 172.21.15.17:0/3679379817 learned_addr learned my addr 172.21.15.17:0/3679379817 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120674198 0 0) 0x7eff0c0ad310 con 0x7eff0c153990 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef8003520 con 0x7eff0c153990 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff12041640 1 -- 172.21.15.17:0/3679379817 >> v1:172.21.15.163:6789/0 conn(0x7eff0c1506b0 legacy=0x7eff0c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff12041640 1 -- 172.21.15.17:0/3679379817 >> v1:172.21.15.17:6789/0 conn(0x7eff0c153990 legacy=0x7eff0c0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983235749 0 0) 0x7eff0c1572a0 con 0x7eff0c1506b0 2024-03-06T07:20:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.994+0000 7eff12041640 1 -- 172.21.15.17:0/3679379817 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c070aa0 con 0x7eff0c156ea0 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff12041640 1 --2- 172.21.15.17:0/3679379817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c54a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2656baa07692be85 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04039070 con 0x7eff0c156ea0 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0402ccf0 con 0x7eff0c156ea0 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff1083e640 1 -- 172.21.15.17:0/3679379817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04034660 con 0x7eff0c156ea0 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 -- 172.21.15.17:0/3679379817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 msgr2=0x7eff0c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 --2- 172.21.15.17:0/3679379817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c54a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0402bf40 comp rx=0 tx=0).stop 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 -- 172.21.15.17:0/3679379817 shutdown_connections 2024-03-06T07:20:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 --2- 172.21.15.17:0/3679379817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c54a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 -- 172.21.15.17:0/3679379817 >> 172.21.15.17:0/3679379817 conn(0x7eff0c06b1a0 msgr2=0x7eff0c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 -- 172.21.15.17:0/3679379817 shutdown_connections 2024-03-06T07:20:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.995+0000 7eff13aa4640 1 -- 172.21.15.17:0/3679379817 wait complete. 2024-03-06T07:20:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.996+0000 7eff13aa4640 1 Processor -- start 2024-03-06T07:20:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.996+0000 7eff13aa4640 1 -- start start 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c1506b0 0x7eff0c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c153990 0x7eff0c1e9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1566a0 con 0x7eff0c156ea0 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c156a10 con 0x7eff0c153990 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff13aa4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0c59e0 con 0x7eff0c1506b0 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff12041640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff12041640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38600/0 (socket says 172.21.15.17:38600) 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff12041640 1 -- 172.21.15.17:0/2270630716 learned_addr learned my addr 172.21.15.17:0/2270630716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff11840640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c1506b0 0x7eff0c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.997+0000 7eff1103f640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c153990 0x7eff0c1e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 -- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c1506b0 msgr2=0x7eff0c152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c1506b0 0x7eff0c152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 -- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c153990 msgr2=0x7eff0c1e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c153990 0x7eff0c1e9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 -- 172.21.15.17:0/2270630716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c1e8770 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff12041640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff04037040 tx=0x7eff04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04039050 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0403a040 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04009450 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c152920 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.998+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c1e8cf0 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.999+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c04f330 con 0x7eff0c156ea0 2024-03-06T07:20:28.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:27.999+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff04047020 con 0x7eff0c156ea0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.000+0000 7eff027fc640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 0x7efef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.000+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7eff040ac560 con 0x7eff0c156ea0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.000+0000 7eff11840640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 0x7efef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.000+0000 7eff11840640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 0x7efef006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff08000f60 tx=0x7eff08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.004+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff0404e020 con 0x7eff0c156ea0 2024-03-06T07:20:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.222+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7eff0c0700a0 con 0x7eff0c156ea0 2024-03-06T07:20:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.222+0000 7eff027fc640 1 -- 172.21.15.17:0/2270630716 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7eff0407aa60 con 0x7eff0c156ea0 2024-03-06T07:20:28.224 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:28.224 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:20:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 msgr2=0x7efef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 0x7efef006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff08000f60 tx=0x7eff08009040 comp rx=0 tx=0).stop 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 msgr2=0x7eff0c0c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff04037040 tx=0x7eff04004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 shutdown_connections 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b4d0 0x7efef006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c156ea0 0x7eff0c0c77e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c153990 0x7eff0c1e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 --2- 172.21.15.17:0/2270630716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c1506b0 0x7eff0c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 >> 172.21.15.17:0/2270630716 conn(0x7eff0c06b1a0 msgr2=0x7eff0c14d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 shutdown_connections 2024-03-06T07:20:28.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.224+0000 7eff13aa4640 1 -- 172.21.15.17:0/2270630716 wait complete. 2024-03-06T07:20:28.237 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-06T07:20:28.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.336+0000 7f475762a640 1 Processor -- start 2024-03-06T07:20:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.336+0000 7f475762a640 1 -- start start 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f475762a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4750158db0 0x7f47500c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f475762a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4750155110 con 0x7f47500c88b0 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f475762a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500c8f40 con 0x7f47500c5330 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f475762a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47501559e0 con 0x7f4750158db0 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f4755bc7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f47500c88b0 0x7f475015fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58496/0 (socket says 172.21.15.17:58496) 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f4755bc7640 1 -- 172.21.15.17:0/1042367006 learned_addr learned my addr 172.21.15.17:0/1042367006 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f4754bc5640 1 --2- 172.21.15.17:0/1042367006 >> v2:172.21.15.163:3300/0 conn(0x7f4750158db0 0x7f47500c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886781542 0 0) 0x7f4740001490 con 0x7f47500c88b0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4738003520 con 0x7f47500c88b0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780809978 0 0) 0x7f47500c8f40 con 0x7f47500c5330 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.337+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4740001490 con 0x7f47500c5330 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f4754bc5640 1 -- 172.21.15.17:0/1042367006 >> v1:172.21.15.163:6789/0 conn(0x7f47500c5330 legacy=0x7f47501586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f4754bc5640 1 -- 172.21.15.17:0/1042367006 >> v1:172.21.15.17:6789/0 conn(0x7f47500c88b0 legacy=0x7f475015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f4754bc5640 1 -- 172.21.15.17:0/1042367006 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750155390 con 0x7f4750158db0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f4754bc5640 1 --2- 172.21.15.17:0/1042367006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4750158db0 0x7f47500c5710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47440097a0 tx=0x7f474402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d58a358b77213d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744035020 con 0x7f4750158db0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.338+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744031830 con 0x7f4750158db0 2024-03-06T07:20:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f474ffff640 1 -- 172.21.15.17:0/1042367006 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744031c00 con 0x7f4750158db0 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 -- 172.21.15.17:0/1042367006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4750158db0 msgr2=0x7f47500c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 --2- 172.21.15.17:0/1042367006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4750158db0 0x7f47500c5710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47440097a0 tx=0x7f474402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 -- 172.21.15.17:0/1042367006 shutdown_connections 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 --2- 172.21.15.17:0/1042367006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4750158db0 0x7f47500c5710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 -- 172.21.15.17:0/1042367006 >> 172.21.15.17:0/1042367006 conn(0x7f47500ac560 msgr2=0x7f47500ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 -- 172.21.15.17:0/1042367006 shutdown_connections 2024-03-06T07:20:28.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.339+0000 7f475762a640 1 -- 172.21.15.17:0/1042367006 wait complete. 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.340+0000 7f475762a640 1 Processor -- start 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.340+0000 7f475762a640 1 -- start start 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47500c5330 0x7f47501514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47500c4980 0x7f47500c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47501529d0 con 0x7f47500c88b0 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475006d090 con 0x7f47500c5330 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f475762a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4750152b50 con 0x7f47500c4980 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38610/0 (socket says 172.21.15.17:38610) 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 -- 172.21.15.17:0/3990129655 learned_addr learned my addr 172.21.15.17:0/3990129655 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f47553c6640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47500c5330 0x7f47501514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4755bc7640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47500c4980 0x7f47500c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 -- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47500c4980 msgr2=0x7f47500c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47500c4980 0x7f47500c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 -- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47500c5330 msgr2=0x7f47501514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47500c5330 0x7f47501514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 -- 172.21.15.17:0/3990129655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47501eb460 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.341+0000 7f4754bc5640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47440098c0 tx=0x7f47440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.342+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744040070 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.342+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474403e070 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.342+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750158080 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.342+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744035040 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.342+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47501519f0 con 0x7f47500c88b0 2024-03-06T07:20:28.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.343+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475004f330 con 0x7f47500c88b0 2024-03-06T07:20:28.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.343+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4744038020 con 0x7f47500c88b0 2024-03-06T07:20:28.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.344+0000 7f474dffb640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 0x7f473c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.344+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f47440ab7c0 con 0x7f47500c88b0 2024-03-06T07:20:28.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.344+0000 7f47553c6640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 0x7f473c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.344+0000 7f47553c6640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 0x7f473c06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4748007fd0 tx=0x7f4748009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.348+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4744079d40 con 0x7f47500c88b0 2024-03-06T07:20:28.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.528+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f475006cdc0 con 0x7f47500c88b0 2024-03-06T07:20:28.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.529+0000 7f474dffb640 1 -- 172.21.15.17:0/3990129655 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f474407dde0 con 0x7f47500c88b0 2024-03-06T07:20:28.532 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:28.532 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 msgr2=0x7f473c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 0x7f473c06d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4748007fd0 tx=0x7f4748009040 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 msgr2=0x7f47500c09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47440098c0 tx=0x7f47440046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 shutdown_connections 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f473c06b4d0 0x7f473c06d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47500c4980 0x7f47500c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47500c88b0 0x7f47500c09f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 --2- 172.21.15.17:0/3990129655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47500c5330 0x7f47501514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 >> 172.21.15.17:0/3990129655 conn(0x7f47500ac560 msgr2=0x7f475014f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 shutdown_connections 2024-03-06T07:20:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.531+0000 7f475762a640 1 -- 172.21.15.17:0/3990129655 wait complete. 2024-03-06T07:20:28.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:28.545 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:28.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.647+0000 7f5312dac640 1 Processor -- start 2024-03-06T07:20:28.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.647+0000 7f5312dac640 1 -- start start 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5312dac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f530c14bcc0 0x7f530c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5312dac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f530c158a10 0x7f530c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5312dac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0c75e0 con 0x7f530c0be150 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5312dac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c0c7f10 con 0x7f530c158a10 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5312dac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c0c78c0 con 0x7f530c14bcc0 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f530c158a10 0x7f530c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f530c158a10 0x7f530c158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38614/0 (socket says 172.21.15.17:38614) 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 -- 172.21.15.17:0/2175554528 learned_addr learned my addr 172.21.15.17:0/2175554528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5310b48640 1 --2- 172.21.15.17:0/2175554528 >> v2:172.21.15.163:3300/0 conn(0x7f530c14bcc0 0x7f530c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 -- 172.21.15.17:0/2175554528 >> v2:172.21.15.163:3300/0 conn(0x7f530c14bcc0 msgr2=0x7f530c155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 --2- 172.21.15.17:0/2175554528 >> v2:172.21.15.163:3300/0 conn(0x7f530c14bcc0 0x7f530c155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 -- 172.21.15.17:0/2175554528 >> v1:172.21.15.163:6789/0 conn(0x7f530c0be150 legacy=0x7f530c0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.648+0000 7f5311349640 1 -- 172.21.15.17:0/2175554528 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0c7ba0 con 0x7f530c158a10 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5311349640 1 --2- 172.21.15.17:0/2175554528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c158a10 0x7f530c158e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f530c08c180 tx=0x7f530402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46258631fa85af01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f530b7fe640 1 -- 172.21.15.17:0/2175554528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530402cbb0 con 0x7f530c158a10 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f530b7fe640 1 -- 172.21.15.17:0/2175554528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530402cd50 con 0x7f530c158a10 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f530b7fe640 1 -- 172.21.15.17:0/2175554528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304034610 con 0x7f530c158a10 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 -- 172.21.15.17:0/2175554528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c158a10 msgr2=0x7f530c158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 --2- 172.21.15.17:0/2175554528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c158a10 0x7f530c158e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f530c08c180 tx=0x7f530402c010 comp rx=0 tx=0).stop 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 -- 172.21.15.17:0/2175554528 shutdown_connections 2024-03-06T07:20:28.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 --2- 172.21.15.17:0/2175554528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c158a10 0x7f530c158e80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 --2- 172.21.15.17:0/2175554528 >> v2:172.21.15.163:3300/0 conn(0x7f530c14bcc0 0x7f530c155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 -- 172.21.15.17:0/2175554528 >> 172.21.15.17:0/2175554528 conn(0x7f530c0ac600 msgr2=0x7f530c0aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 -- 172.21.15.17:0/2175554528 shutdown_connections 2024-03-06T07:20:28.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.649+0000 7f5312dac640 1 -- 172.21.15.17:0/2175554528 wait complete. 2024-03-06T07:20:28.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.650+0000 7f5312dac640 1 Processor -- start 2024-03-06T07:20:28.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 -- start start 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f530c14bcc0 0x7f530c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f530c158a10 0x7f530c0c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c5c60 con 0x7f530c0be150 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c5ae0 con 0x7f530c158a10 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5312dac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c1593c0 con 0x7f530c14bcc0 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5310b48640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5310b48640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38616/0 (socket says 172.21.15.17:38616) 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.651+0000 7f5310b48640 1 -- 172.21.15.17:0/1751246020 learned_addr learned my addr 172.21.15.17:0/1751246020 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5311349640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f530c158a10 0x7f530c0c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f530bfff640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f530c14bcc0 0x7f530c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 -- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f530c14bcc0 msgr2=0x7f530c154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f530c14bcc0 0x7f530c154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 -- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f530c158a10 msgr2=0x7f530c0c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f530c158a10 0x7f530c0c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 -- 172.21.15.17:0/1751246020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0c14b0 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5310b48640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f530000e9e0 tx=0x7f530000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000c9d0 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c1ebab0 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000cdc0 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.652+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c1eafd0 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.653+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c04f330 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.654+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f530001a070 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.654+0000 7f53097fa640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 0x7f52f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.654+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f530008a490 con 0x7f530c0be150 2024-03-06T07:20:28.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.654+0000 7f530bfff640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 0x7f52f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.655+0000 7f530bfff640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 0x7f52f006d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f530c1550d0 tx=0x7f52f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.658+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5300058a10 con 0x7f530c0be150 2024-03-06T07:20:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.840+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f530c0c83e0 con 0x7f530c0be150 2024-03-06T07:20:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.840+0000 7f53097fa640 1 -- 172.21.15.17:0/1751246020 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f530005cab0 con 0x7f530c0be150 2024-03-06T07:20:28.842 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 msgr2=0x7f52f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 0x7f52f006d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f530c1550d0 tx=0x7f52f8009040 comp rx=0 tx=0).stop 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 msgr2=0x7f530c0c0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f530000e9e0 tx=0x7f530000eea0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 shutdown_connections 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52f006b4d0 0x7f52f006d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f530c158a10 0x7f530c0c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f530c14bcc0 0x7f530c154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 --2- 172.21.15.17:0/1751246020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f530c0be150 0x7f530c0c0e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 >> 172.21.15.17:0/1751246020 conn(0x7f530c0ac600 msgr2=0x7f530c0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 shutdown_connections 2024-03-06T07:20:28.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.842+0000 7f5312dac640 1 -- 172.21.15.17:0/1751246020 wait complete. 2024-03-06T07:20:28.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:28.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.958+0000 7f5957c57640 1 Processor -- start 2024-03-06T07:20:28.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.958+0000 7f5957c57640 1 -- start start 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f5957c57640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59501529f0 0x7f59500c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f5957c57640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59500c5370 0x7f59500c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f5957c57640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950152dd0 con 0x7f5950155530 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f5957c57640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5950159710 con 0x7f59500c5370 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f5957c57640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500c68b0 con 0x7f59501529f0 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59500c5370 0x7f59500c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f59500c5370 0x7f59500c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38630/0 (socket says 172.21.15.17:38630) 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59559f3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59501529f0 0x7f59500c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 -- 172.21.15.17:0/3646810487 learned_addr learned my addr 172.21.15.17:0/3646810487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59561f4640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5950155530 0x7f59501559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57574/0 (socket says 172.21.15.17:57574) 2024-03-06T07:20:28.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59549f1640 1 -- 172.21.15.17:0/3646810487 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458276896 0 0) 0x7f5950152dd0 con 0x7f5950155530 2024-03-06T07:20:28.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 -- 172.21.15.17:0/3646810487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59501529f0 msgr2=0x7f59500c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:28.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 --2- 172.21.15.17:0/3646810487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59501529f0 0x7f59500c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 -- 172.21.15.17:0/3646810487 >> v1:172.21.15.163:6789/0 conn(0x7f5950155530 legacy=0x7f59501559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.959+0000 7f59551f2640 1 -- 172.21.15.17:0/3646810487 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950158b00 con 0x7f59500c5370 2024-03-06T07:20:28.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.960+0000 7f59551f2640 1 --2- 172.21.15.17:0/3646810487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500c57d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5940009870 tx=0x7f594002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958bb2ab36d1cd04 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.960+0000 7f59549f1640 1 -- 172.21.15.17:0/3646810487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940035020 con 0x7f59500c5370 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.960+0000 7f59549f1640 1 -- 172.21.15.17:0/3646810487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940031640 con 0x7f59500c5370 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.960+0000 7f59549f1640 1 -- 172.21.15.17:0/3646810487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594002d040 con 0x7f59500c5370 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 -- 172.21.15.17:0/3646810487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 msgr2=0x7f59500c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 --2- 172.21.15.17:0/3646810487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500c57d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5940009870 tx=0x7f594002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 -- 172.21.15.17:0/3646810487 shutdown_connections 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 --2- 172.21.15.17:0/3646810487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500c57d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 --2- 172.21.15.17:0/3646810487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59501529f0 0x7f59500c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 -- 172.21.15.17:0/3646810487 >> 172.21.15.17:0/3646810487 conn(0x7f59500ac580 msgr2=0x7f59500ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 -- 172.21.15.17:0/3646810487 shutdown_connections 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.961+0000 7f5957c57640 1 -- 172.21.15.17:0/3646810487 wait complete. 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.962+0000 7f5957c57640 1 Processor -- start 2024-03-06T07:20:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.962+0000 7f5957c57640 1 -- start start 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 0x7f5950149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5950155530 0x7f595014a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501593a0 con 0x7f59500c5370 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59501590c0 con 0x7f59501529f0 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59501560d0 con 0x7f5950155530 2024-03-06T07:20:28.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38638/0 (socket says 172.21.15.17:38638) 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 -- 172.21.15.17:0/3607302672 learned_addr learned my addr 172.21.15.17:0/3607302672 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59551f2640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 0x7f5950149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59561f4640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5950155530 0x7f595014a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 -- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5950155530 msgr2=0x7f595014a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5950155530 0x7f595014a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 -- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 msgr2=0x7f5950149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 0x7f5950149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 -- 172.21.15.17:0/3607302672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950158070 con 0x7f59500c5370 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59551f2640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 0x7f5950149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59559f3640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e9e0 tx=0x7f594c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c017070 con 0x7f59500c5370 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00c9d0 con 0x7f59500c5370 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950145dc0 con 0x7f59500c5370 2024-03-06T07:20:28.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00cdc0 con 0x7f59500c5370 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.963+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500abb60 con 0x7f59500c5370 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.964+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950157940 con 0x7f59500c5370 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.965+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f594c01a070 con 0x7f59500c5370 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.965+0000 7f59467fc640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.965+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f594c08a8f0 con 0x7f59500c5370 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.965+0000 7f59551f2640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:28.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.966+0000 7f59551f2640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 0x7f593406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5940036000 tx=0x7f594003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:28.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:28.969+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f594c0594c0 con 0x7f59500c5370 2024-03-06T07:20:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.147+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f595004f330 con 0x7f59500c5370 2024-03-06T07:20:29.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.148+0000 7f59467fc640 1 -- 172.21.15.17:0/3607302672 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f594c058dc0 con 0x7f59500c5370 2024-03-06T07:20:29.150 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 msgr2=0x7f593406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 0x7f593406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5940036000 tx=0x7f594003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 msgr2=0x7f59500ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e9e0 tx=0x7f594c00eea0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 shutdown_connections 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f593406b4d0 0x7f593406d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5950155530 0x7f595014a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59501529f0 0x7f5950149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.149+0000 7f5957c57640 1 --2- 172.21.15.17:0/3607302672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59500c5370 0x7f59500ab620 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.150+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 >> 172.21.15.17:0/3607302672 conn(0x7f59500ac580 msgr2=0x7f59500c9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.150+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 shutdown_connections 2024-03-06T07:20:29.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.150+0000 7f5957c57640 1 -- 172.21.15.17:0/3607302672 wait complete. 2024-03-06T07:20:29.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:29.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 Processor -- start 2024-03-06T07:20:29.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 -- start start 2024-03-06T07:20:29.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 0x7f48580c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f485806c900 0x7f48580a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48580c3ef0 con 0x7f48580ab170 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48581556d0 con 0x7f485806c900 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.265+0000 7f485d822640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48581559b0 con 0x7f48580c5990 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f485806c900 0x7f48580a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f485806c900 0x7f48580a9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38652/0 (socket says 172.21.15.17:38652) 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f4856ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 0x7f48580c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 -- 172.21.15.17:0/3003914560 learned_addr learned my addr 172.21.15.17:0/3003914560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48567fc640 1 -- 172.21.15.17:0/3003914560 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693976096 0 0) 0x7f48580c3ef0 con 0x7f48580ab170 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 -- 172.21.15.17:0/3003914560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 msgr2=0x7f48580c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 0x7f48580c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 -- 172.21.15.17:0/3003914560 >> v1:172.21.15.163:6789/0 conn(0x7f48580ab170 legacy=0x7f485806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f48577fe640 1 -- 172.21.15.17:0/3003914560 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858155c90 con 0x7f485806c900 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.266+0000 7f4856ffd640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 0x7f48580c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f48577fe640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f485806c900 0x7f48580a9fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4850009e50 tx=0x7f485002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fe5e4afbca6986 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f48567fc640 1 -- 172.21.15.17:0/3003914560 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485002cb00 con 0x7f485806c900 2024-03-06T07:20:29.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f48567fc640 1 -- 172.21.15.17:0/3003914560 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485002cca0 con 0x7f485806c900 2024-03-06T07:20:29.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f48567fc640 1 -- 172.21.15.17:0/3003914560 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850034610 con 0x7f485806c900 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 -- 172.21.15.17:0/3003914560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f485806c900 msgr2=0x7f48580a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f485806c900 0x7f48580a9fa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4850009e50 tx=0x7f485002bf60 comp rx=0 tx=0).stop 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 -- 172.21.15.17:0/3003914560 shutdown_connections 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f485806c900 0x7f48580a9fa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 --2- 172.21.15.17:0/3003914560 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48580c5990 0x7f48580c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.267+0000 7f485d822640 1 -- 172.21.15.17:0/3003914560 >> 172.21.15.17:0/3003914560 conn(0x7f48580c6710 msgr2=0x7f48580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.268+0000 7f485d822640 1 -- 172.21.15.17:0/3003914560 shutdown_connections 2024-03-06T07:20:29.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.268+0000 7f485d822640 1 -- 172.21.15.17:0/3003914560 wait complete. 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 Processor -- start 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 -- start start 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485806c900 0x7f48581dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48580ab170 0x7f4858150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858156610 con 0x7f48580c5990 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858156490 con 0x7f48580ab170 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f485d822640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48580c3af0 con 0x7f485806c900 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f48577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f48577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38658/0 (socket says 172.21.15.17:38658) 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.269+0000 7f48577fe640 1 -- 172.21.15.17:0/3052689976 learned_addr learned my addr 172.21.15.17:0/3052689976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f484ffff640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48580ab170 0x7f4858150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f4856ffd640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485806c900 0x7f48581dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 -- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485806c900 msgr2=0x7f48581dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485806c900 0x7f48581dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 -- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48580ab170 msgr2=0x7f4858150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48580ab170 0x7f4858150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 -- 172.21.15.17:0/3052689976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4858150550 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f48577fe640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4850036040 tx=0x7f4850004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850043030 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485002ce50 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48581eea70 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850009510 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.270+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48581eed50 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.271+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485804f330 con 0x7f48580c5990 2024-03-06T07:20:29.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.271+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4850046070 con 0x7f48580c5990 2024-03-06T07:20:29.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.272+0000 7f4854ff9640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 0x7f483806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.272+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48500ac610 con 0x7f48580c5990 2024-03-06T07:20:29.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.272+0000 7f4856ffd640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 0x7f483806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.272+0000 7f4856ffd640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 0x7f483806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48581dd4f0 tx=0x7f4848009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.276+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f485004d070 con 0x7f48580c5990 2024-03-06T07:20:29.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.460+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48580c1740 con 0x7f48580c5990 2024-03-06T07:20:29.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.460+0000 7f4854ff9640 1 -- 172.21.15.17:0/3052689976 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f485007ab90 con 0x7f48580c5990 2024-03-06T07:20:29.462 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:29.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 msgr2=0x7f483806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 0x7f483806d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48581dd4f0 tx=0x7f4848009190 comp rx=0 tx=0).stop 2024-03-06T07:20:29.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 msgr2=0x7f4858151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4850036040 tx=0x7f4850004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 shutdown_connections 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f483806b4d0 0x7f483806d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48580c5990 0x7f4858151250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48580ab170 0x7f4858150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 --2- 172.21.15.17:0/3052689976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f485806c900 0x7f48581dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 >> 172.21.15.17:0/3052689976 conn(0x7f48580c6710 msgr2=0x7f48580a8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 shutdown_connections 2024-03-06T07:20:29.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.462+0000 7f485d822640 1 -- 172.21.15.17:0/3052689976 wait complete. 2024-03-06T07:20:29.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:29.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.578+0000 7fc786057640 1 Processor -- start 2024-03-06T07:20:29.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.578+0000 7fc786057640 1 -- start start 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc786057640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc780155a00 0x7fc780159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc786057640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc780155e70 con 0x7fc7800be150 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc786057640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7800c75e0 con 0x7fc78014bcc0 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc786057640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800c7f10 con 0x7fc780155a00 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc7800be150 0x7fc780158a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58498/0 (socket says 172.21.15.17:58498) 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77effd640 1 -- 172.21.15.17:0/517883249 learned_addr learned my addr 172.21.15.17:0/517883249 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77ffff640 1 --2- 172.21.15.17:0/517883249 >> v2:172.21.15.163:3300/0 conn(0x7fc780155a00 0x7fc780159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77e7fc640 1 -- 172.21.15.17:0/517883249 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034313560 0 0) 0x7fc780155e70 con 0x7fc7800be150 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77e7fc640 1 -- 172.21.15.17:0/517883249 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc768003520 con 0x7fc7800be150 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77ffff640 1 -- 172.21.15.17:0/517883249 >> v1:172.21.15.163:6789/0 conn(0x7fc78014bcc0 legacy=0x7fc7800c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77ffff640 1 -- 172.21.15.17:0/517883249 >> v1:172.21.15.17:6789/0 conn(0x7fc7800be150 legacy=0x7fc780158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.579+0000 7fc77ffff640 1 -- 172.21.15.17:0/517883249 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800c78c0 con 0x7fc780155a00 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.580+0000 7fc77ffff640 1 --2- 172.21.15.17:0/517883249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc780155a00 0x7fc780159120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc778009d90 tx=0x7fc77802bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787ecb0d36905e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.580+0000 7fc77e7fc640 1 -- 172.21.15.17:0/517883249 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778039070 con 0x7fc780155a00 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.580+0000 7fc77e7fc640 1 -- 172.21.15.17:0/517883249 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77802cd80 con 0x7fc780155a00 2024-03-06T07:20:29.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.580+0000 7fc77e7fc640 1 -- 172.21.15.17:0/517883249 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778034640 con 0x7fc780155a00 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 -- 172.21.15.17:0/517883249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc780155a00 msgr2=0x7fc780159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 --2- 172.21.15.17:0/517883249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc780155a00 0x7fc780159120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc778009d90 tx=0x7fc77802bfd0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 -- 172.21.15.17:0/517883249 shutdown_connections 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 --2- 172.21.15.17:0/517883249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc780155a00 0x7fc780159120 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 -- 172.21.15.17:0/517883249 >> 172.21.15.17:0/517883249 conn(0x7fc7800ac600 msgr2=0x7fc7800aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 -- 172.21.15.17:0/517883249 shutdown_connections 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.581+0000 7fc786057640 1 -- 172.21.15.17:0/517883249 wait complete. 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.582+0000 7fc786057640 1 Processor -- start 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.582+0000 7fc786057640 1 -- start start 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800be150 0x7fc78014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014bcc0 0x7fc780147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78006d1d0 con 0x7fc780155a00 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c5ae0 con 0x7fc7800be150 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc786057640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c5c60 con 0x7fc78014bcc0 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38668/0 (socket says 172.21.15.17:38668) 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 -- 172.21.15.17:0/3048584375 learned_addr learned my addr 172.21.15.17:0/3048584375 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:29.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77f7fe640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800be150 0x7fc78014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77effd640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014bcc0 0x7fc780147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 -- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014bcc0 msgr2=0x7fc780147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014bcc0 0x7fc780147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 -- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800be150 msgr2=0x7fc78014d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800be150 0x7fc78014d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.583+0000 7fc77ffff640 1 -- 172.21.15.17:0/3048584375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800bf2f0 con 0x7fc780155a00 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc77ffff640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc778037040 tx=0x7fc778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778044020 con 0x7fc780155a00 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77803a040 con 0x7fc780155a00 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78014e4d0 con 0x7fc780155a00 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778039050 con 0x7fc780155a00 2024-03-06T07:20:29.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.584+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780148a70 con 0x7fc780155a00 2024-03-06T07:20:29.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.585+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78004f330 con 0x7fc780155a00 2024-03-06T07:20:29.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.585+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc778003b90 con 0x7fc780155a00 2024-03-06T07:20:29.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.586+0000 7fc767fff640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.586+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7780ac5f0 con 0x7fc780155a00 2024-03-06T07:20:29.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.586+0000 7fc77f7fe640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.586+0000 7fc77f7fe640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc770001780 tx=0x7fc770009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.590+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc77807ab70 con 0x7fc780155a00 2024-03-06T07:20:29.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.773+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc7800c83e0 con 0x7fc780155a00 2024-03-06T07:20:29.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.773+0000 7fc767fff640 1 -- 172.21.15.17:0/3048584375 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc77807ec10 con 0x7fc780155a00 2024-03-06T07:20:29.775 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 msgr2=0x7fc76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc770001780 tx=0x7fc770009040 comp rx=0 tx=0).stop 2024-03-06T07:20:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 msgr2=0x7fc780148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc778037040 tx=0x7fc778004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:29.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 shutdown_connections 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780155a00 0x7fc780148530 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014bcc0 0x7fc780147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 --2- 172.21.15.17:0/3048584375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800be150 0x7fc78014d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 >> 172.21.15.17:0/3048584375 conn(0x7fc7800ac600 msgr2=0x7fc7800ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 shutdown_connections 2024-03-06T07:20:29.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.775+0000 7fc786057640 1 -- 172.21.15.17:0/3048584375 wait complete. 2024-03-06T07:20:29.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:29.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.888+0000 7f655e4c0640 1 Processor -- start 2024-03-06T07:20:29.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.888+0000 7f655e4c0640 1 -- start start 2024-03-06T07:20:29.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f655e4c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65580c1990 0x7f6558160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f655e4c0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65580c1dc0 con 0x7f6558155c20 2024-03-06T07:20:29.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f655e4c0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655806afc0 con 0x7f6558152920 2024-03-06T07:20:29.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f655e4c0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655806b2a0 con 0x7f65580c1990 2024-03-06T07:20:29.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f655ca5d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65580c1990 0x7f6558160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f65577fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6558155c20 0x7f65580c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58500/0 (socket says 172.21.15.17:58500) 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.889+0000 7f65577fe640 1 -- 172.21.15.17:0/2835149730 learned_addr learned my addr 172.21.15.17:0/2835149730 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f655ca5d640 1 -- 172.21.15.17:0/2835149730 >> v1:172.21.15.163:6789/0 conn(0x7f6558152920 legacy=0x7f65580c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f655ca5d640 1 -- 172.21.15.17:0/2835149730 >> v1:172.21.15.17:6789/0 conn(0x7f6558155c20 legacy=0x7f65580c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f655ca5d640 1 -- 172.21.15.17:0/2835149730 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655806b580 con 0x7f65580c1990 2024-03-06T07:20:29.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f6556ffd640 1 -- 172.21.15.17:0/2835149730 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463968461 0 0) 0x7f65580c1dc0 con 0x7f6558155c20 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f655ca5d640 1 --2- 172.21.15.17:0/2835149730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f6558160b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6550009d90 tx=0x7f655002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5778d0681bbce4c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f6556ffd640 1 -- 172.21.15.17:0/2835149730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550039070 con 0x7f65580c1990 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f6556ffd640 1 -- 172.21.15.17:0/2835149730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655002cd00 con 0x7f65580c1990 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.890+0000 7f6556ffd640 1 -- 172.21.15.17:0/2835149730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550034660 con 0x7f65580c1990 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 -- 172.21.15.17:0/2835149730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 msgr2=0x7f6558160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 --2- 172.21.15.17:0/2835149730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f6558160b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6550009d90 tx=0x7f655002bf50 comp rx=0 tx=0).stop 2024-03-06T07:20:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 -- 172.21.15.17:0/2835149730 shutdown_connections 2024-03-06T07:20:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 --2- 172.21.15.17:0/2835149730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f6558160b30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 -- 172.21.15.17:0/2835149730 >> 172.21.15.17:0/2835149730 conn(0x7f65580c4290 msgr2=0x7f65580c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 -- 172.21.15.17:0/2835149730 shutdown_connections 2024-03-06T07:20:29.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.891+0000 7f655e4c0640 1 -- 172.21.15.17:0/2835149730 wait complete. 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.892+0000 7f655e4c0640 1 Processor -- start 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.892+0000 7f655e4c0640 1 -- start start 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 0x7f65581540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6558155c20 0x7f65581545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65581560b0 con 0x7f65580c1990 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c0d60 con 0x7f6558152920 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655e4c0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c28a0 con 0x7f6558155c20 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38684/0 (socket says 172.21.15.17:38684) 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 -- 172.21.15.17:0/3956881632 learned_addr learned my addr 172.21.15.17:0/3956881632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f655ca5d640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6558155c20 0x7f65581545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f65577fe640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 0x7f65581540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 -- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6558155c20 msgr2=0x7f65581545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6558155c20 0x7f65581545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 -- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 msgr2=0x7f65581540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 0x7f65581540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 -- 172.21.15.17:0/3956881632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558155260 con 0x7f65580c1990 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f65577fe640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 0x7f65581540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.893+0000 7f6557fff640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f654800c500 tx=0x7f654800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.894+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65480052d0 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.894+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548005470 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.894+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6558154f80 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.894+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548012850 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.894+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65581eb920 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.895+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655804f330 con 0x7f65580c1990 2024-03-06T07:20:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.895+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65480129f0 con 0x7f65580c1990 2024-03-06T07:20:29.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.896+0000 7f6554ff9640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 0x7f654406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:29.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.896+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f654808a040 con 0x7f65580c1990 2024-03-06T07:20:29.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.896+0000 7f65577fe640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 0x7f654406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:29.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.896+0000 7f65577fe640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 0x7f654406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fd0 tx=0x7f654c007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:29.900+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6548058b90 con 0x7f65580c1990 2024-03-06T07:20:30.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.118+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6558068460 con 0x7f65580c1990 2024-03-06T07:20:30.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.118+0000 7f6554ff9640 1 -- 172.21.15.17:0/3956881632 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f6548058490 con 0x7f65580c1990 2024-03-06T07:20:30.120 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:30.120 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:30.121 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 msgr2=0x7f654406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 0x7f654406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f654c007fd0 tx=0x7f654c007dc0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 msgr2=0x7f65580ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f654800c500 tx=0x7f654800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 shutdown_connections 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f654406b4d0 0x7f654406d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6558155c20 0x7f65581545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6558152920 0x7f65581540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 --2- 172.21.15.17:0/3956881632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65580c1990 0x7f65580ac9d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 >> 172.21.15.17:0/3956881632 conn(0x7f65580c4290 msgr2=0x7f655814dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 shutdown_connections 2024-03-06T07:20:30.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.120+0000 7f655e4c0640 1 -- 172.21.15.17:0/3956881632 wait complete. 2024-03-06T07:20:30.133 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.232+0000 7f3d2e77a640 1 Processor -- start 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.232+0000 7f3d2e77a640 1 -- start start 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2e77a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 0x7f3d280c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2e77a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d28155110 con 0x7f3d28158db0 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2e77a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d280c8f40 con 0x7f3d280c88b0 2024-03-06T07:20:30.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2e77a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d281559e0 con 0x7f3d280c5330 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2cd17640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3d28158db0 0x7f3d2815fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58512/0 (socket says 172.21.15.17:58512) 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d2cd17640 1 -- 172.21.15.17:0/2435213451 learned_addr learned my addr 172.21.15.17:0/2435213451 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d27fff640 1 --2- 172.21.15.17:0/2435213451 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 0x7f3d280c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226730767 0 0) 0x7f3d20001490 con 0x7f3d28158db0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d10003520 con 0x7f3d28158db0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147936922 0 0) 0x7f3d280c8f40 con 0x7f3d280c88b0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d20001490 con 0x7f3d280c88b0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d27fff640 1 -- 172.21.15.17:0/2435213451 >> v1:172.21.15.163:6789/0 conn(0x7f3d280c88b0 legacy=0x7f3d281586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d27fff640 1 -- 172.21.15.17:0/2435213451 >> v1:172.21.15.17:6789/0 conn(0x7f3d28158db0 legacy=0x7f3d2815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.233+0000 7f3d27fff640 1 -- 172.21.15.17:0/2435213451 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d28155390 con 0x7f3d280c5330 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.234+0000 7f3d27fff640 1 --2- 172.21.15.17:0/2435213451 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 0x7f3d280c5710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097d0 tx=0x7f3d1c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc8b97e08f48b61 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.234+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c035020 con 0x7f3d280c5330 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.234+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c031800 con 0x7f3d280c5330 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.234+0000 7f3d26ffd640 1 -- 172.21.15.17:0/2435213451 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c031bd0 con 0x7f3d280c5330 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2435213451 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 msgr2=0x7f3d280c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2435213451 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 0x7f3d280c5710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0097d0 tx=0x7f3d1c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2435213451 shutdown_connections 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2435213451 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d280c5330 0x7f3d280c5710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2435213451 >> 172.21.15.17:0/2435213451 conn(0x7f3d280ac560 msgr2=0x7f3d280ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2435213451 shutdown_connections 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.235+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2435213451 wait complete. 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.236+0000 7f3d2e77a640 1 Processor -- start 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.236+0000 7f3d2e77a640 1 -- start start 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.236+0000 7f3d2e77a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d280c5330 0x7f3d281516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2814bb10 0x7f3d281567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d281529d0 con 0x7f3d280c88b0 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2806d090 con 0x7f3d280c5330 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d28152b50 con 0x7f3d2814bb10 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38686/0 (socket says 172.21.15.17:38686) 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 -- 172.21.15.17:0/2077253428 learned_addr learned my addr 172.21.15.17:0/2077253428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d27fff640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d280c5330 0x7f3d281516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2cd17640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2814bb10 0x7f3d281567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 -- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2814bb10 msgr2=0x7f3d281567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2814bb10 0x7f3d281567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 -- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d280c5330 msgr2=0x7f3d281516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d280c5330 0x7f3d281516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 -- 172.21.15.17:0/2077253428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d281eb6e0 con 0x7f3d280c88b0 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d277fe640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c420 tx=0x7f3d1800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18017070 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d180052d0 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d28158300 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d18003830 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.237+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d28151c10 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.238+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d281ebbe0 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.239+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d1801a070 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.239+0000 7f3d24ff9640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 0x7f3d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.239+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d180897d0 con 0x7f3d280c88b0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.239+0000 7f3d27fff640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 0x7f3d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.240+0000 7f3d27fff640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 0x7f3d1406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c036000 tx=0x7f3d1c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.244+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d281ebbe0 con 0x7f3d280c88b0 2024-03-06T07:20:30.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.426+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d280bde30 con 0x7f3d280c88b0 2024-03-06T07:20:30.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.427+0000 7f3d24ff9640 1 -- 172.21.15.17:0/2077253428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f3d18057d70 con 0x7f3d280c88b0 2024-03-06T07:20:30.429 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:30.430 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 msgr2=0x7f3d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 0x7f3d1406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c036000 tx=0x7f3d1c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 msgr2=0x7f3d28156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c420 tx=0x7f3d1800c8e0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 shutdown_connections 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d1406b4d0 0x7f3d1406d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2814bb10 0x7f3d281567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d280c88b0 0x7f3d28156290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 --2- 172.21.15.17:0/2077253428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d280c5330 0x7f3d281516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.429+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 >> 172.21.15.17:0/2077253428 conn(0x7f3d280ac560 msgr2=0x7f3d280ca150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.430+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 shutdown_connections 2024-03-06T07:20:30.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.430+0000 7f3d2e77a640 1 -- 172.21.15.17:0/2077253428 wait complete. 2024-03-06T07:20:30.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:30.444 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.546+0000 7f2b30c64640 1 Processor -- start 2024-03-06T07:20:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.546+0000 7f2b30c64640 1 -- start start 2024-03-06T07:20:30.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06b3a0 0x7f2b2c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c06fde0 con 0x7f2b2c06afc0 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c156fc0 con 0x7f2b2c06b3a0 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b30c64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06b820 con 0x7f2b2c152f90 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45486/0 (socket says 172.21.15.17:45486) 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 -- 172.21.15.17:0/2885923584 learned_addr learned my addr 172.21.15.17:0/2885923584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b29d9b640 1 --2- 172.21.15.17:0/2885923584 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06b3a0 0x7f2b2c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2ad9d640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 -- 172.21.15.17:0/2885923584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 msgr2=0x7f2b2c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 -- 172.21.15.17:0/2885923584 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06b3a0 msgr2=0x7f2b2c06f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/2885923584 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06b3a0 0x7f2b2c06f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.547+0000 7f2b2a59c640 1 -- 172.21.15.17:0/2885923584 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0c7430 con 0x7f2b2c06afc0 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.548+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aa430 tx=0x7f2b2402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab64482fe0744 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.548+0000 7f2b2959a640 1 -- 172.21.15.17:0/2885923584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2402e070 con 0x7f2b2c06afc0 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.548+0000 7f2b2959a640 1 -- 172.21.15.17:0/2885923584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24037070 con 0x7f2b2c06afc0 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.548+0000 7f2b2959a640 1 -- 172.21.15.17:0/2885923584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24031a00 con 0x7f2b2c06afc0 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 -- 172.21.15.17:0/2885923584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 msgr2=0x7f2b2c0c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aa430 tx=0x7f2b2402b880 comp rx=0 tx=0).stop 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 -- 172.21.15.17:0/2885923584 shutdown_connections 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 --2- 172.21.15.17:0/2885923584 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06b3a0 0x7f2b2c06f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 --2- 172.21.15.17:0/2885923584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c6da0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 -- 172.21.15.17:0/2885923584 >> 172.21.15.17:0/2885923584 conn(0x7f2b2c0c4220 msgr2=0x7f2b2c0c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 -- 172.21.15.17:0/2885923584 shutdown_connections 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.549+0000 7f2b30c64640 1 -- 172.21.15.17:0/2885923584 wait complete. 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.550+0000 7f2b30c64640 1 Processor -- start 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.550+0000 7f2b30c64640 1 -- start start 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06b3a0 0x7f2b2c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c156970 con 0x7f2b2c06afc0 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c156690 con 0x7f2b2c06b3a0 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b30c64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c156c50 con 0x7f2b2c152f90 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45502/0 (socket says 172.21.15.17:45502) 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 -- 172.21.15.17:0/444328314 learned_addr learned my addr 172.21.15.17:0/444328314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2ad9d640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b29d9b640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06b3a0 0x7f2b2c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 -- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 msgr2=0x7f2b2c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 -- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06b3a0 msgr2=0x7f2b2c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06b3a0 0x7f2b2c154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.551+0000 7f2b2a59c640 1 -- 172.21.15.17:0/444328314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c1ec0f0 con 0x7f2b2c06afc0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b2a59c640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b2402b8c0 tx=0x7f2b240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24032670 con 0x7f2b2c06afc0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24032830 con 0x7f2b2c06afc0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c156300 con 0x7f2b2c06afc0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24032c20 con 0x7f2b2c06afc0 2024-03-06T07:20:30.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.552+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c156020 con 0x7f2b2c06afc0 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.553+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c04f330 con 0x7f2b2c06afc0 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.553+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b240453f0 con 0x7f2b2c06afc0 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.554+0000 7f2b22ffd640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 0x7f2b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.554+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2b240ac0f0 con 0x7f2b2c06afc0 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.554+0000 7f2b29d9b640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 0x7f2b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.554+0000 7f2b29d9b640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 0x7f2b0c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0a9440 tx=0x7f2b14009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.558+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b24040070 con 0x7f2b2c06afc0 2024-03-06T07:20:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.737+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b2c154130 con 0x7f2b2c06afc0 2024-03-06T07:20:30.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.737+0000 7f2b22ffd640 1 -- 172.21.15.17:0/444328314 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b2407a670 con 0x7f2b2c06afc0 2024-03-06T07:20:30.739 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 msgr2=0x7f2b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 0x7f2b0c06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0a9440 tx=0x7f2b14009190 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 msgr2=0x7f2b2c0c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b2402b8c0 tx=0x7f2b240046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 shutdown_connections 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b0c06b4d0 0x7f2b0c06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c152f90 0x7f2b2c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06b3a0 0x7f2b2c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 --2- 172.21.15.17:0/444328314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b2c06afc0 0x7f2b2c0c3410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 >> 172.21.15.17:0/444328314 conn(0x7f2b2c0c4220 msgr2=0x7f2b2c14f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 shutdown_connections 2024-03-06T07:20:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.739+0000 7f2b30c64640 1 -- 172.21.15.17:0/444328314 wait complete. 2024-03-06T07:20:30.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.856+0000 7fa62860e640 1 Processor -- start 2024-03-06T07:20:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.856+0000 7fa62860e640 1 -- start start 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa62860e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa6200c26c0 0x7fa620155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa62860e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6200670c0 con 0x7fa6200c0470 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa62860e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa62006b0f0 con 0x7fa6200c2aa0 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa62860e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa62006b740 con 0x7fa6200c26c0 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa626bab640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa6200c0470 0x7fa620152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35716/0 (socket says 172.21.15.17:35716) 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa626bab640 1 -- 172.21.15.17:0/2895769263 learned_addr learned my addr 172.21.15.17:0/2895769263 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:30.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6263aa640 1 --2- 172.21.15.17:0/2895769263 >> v2:172.21.15.163:3300/0 conn(0x7fa6200c26c0 0x7fa620155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6253a8640 1 -- 172.21.15.17:0/2895769263 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372742235 0 0) 0x7fa6200670c0 con 0x7fa6200c0470 2024-03-06T07:20:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6253a8640 1 -- 172.21.15.17:0/2895769263 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa608003520 con 0x7fa6200c0470 2024-03-06T07:20:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6263aa640 1 -- 172.21.15.17:0/2895769263 >> v1:172.21.15.163:6789/0 conn(0x7fa6200c2aa0 legacy=0x7fa62006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6263aa640 1 -- 172.21.15.17:0/2895769263 >> v1:172.21.15.17:6789/0 conn(0x7fa6200c0470 legacy=0x7fa620152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.857+0000 7fa6263aa640 1 -- 172.21.15.17:0/2895769263 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62006b3d0 con 0x7fa6200c26c0 2024-03-06T07:20:30.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.858+0000 7fa6263aa640 1 --2- 172.21.15.17:0/2895769263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620155bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6180097a0 tx=0x7fa61802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11df78df8573a317 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.858+0000 7fa6253a8640 1 -- 172.21.15.17:0/2895769263 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6180315a0 con 0x7fa6200c26c0 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.858+0000 7fa6253a8640 1 -- 172.21.15.17:0/2895769263 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa618037070 con 0x7fa6200c26c0 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.858+0000 7fa6253a8640 1 -- 172.21.15.17:0/2895769263 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618031940 con 0x7fa6200c26c0 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 -- 172.21.15.17:0/2895769263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 msgr2=0x7fa620155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 --2- 172.21.15.17:0/2895769263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620155bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa6180097a0 tx=0x7fa61802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 -- 172.21.15.17:0/2895769263 shutdown_connections 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 --2- 172.21.15.17:0/2895769263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620155bd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 -- 172.21.15.17:0/2895769263 >> 172.21.15.17:0/2895769263 conn(0x7fa6200c3b40 msgr2=0x7fa6200c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 -- 172.21.15.17:0/2895769263 shutdown_connections 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.859+0000 7fa62860e640 1 -- 172.21.15.17:0/2895769263 wait complete. 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.860+0000 7fa62860e640 1 Processor -- start 2024-03-06T07:20:30.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.860+0000 7fa62860e640 1 -- start start 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 0x7fa6200c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6200c20b0 con 0x7fa6200c2aa0 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6200c1f30 con 0x7fa6200c26c0 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa62860e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa620066d00 con 0x7fa6200c0470 2024-03-06T07:20:30.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa6263aa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 0x7fa6200c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa625ba9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45514/0 (socket says 172.21.15.17:45514) 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 -- 172.21.15.17:0/1789458571 learned_addr learned my addr 172.21.15.17:0/1789458571 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa625ba9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59386/0 (socket says 172.21.15.17:59386) 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa6263aa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 0x7fa6200c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54346/0 (socket says 172.21.15.17:54346) 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 -- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 msgr2=0x7fa6200c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 0x7fa6200c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 -- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 msgr2=0x7fa620148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 -- 172.21.15.17:0/1789458571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620149f10 con 0x7fa6200c2aa0 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa625ba9640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:30.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa626bab640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa61000cb40 tx=0x7fa61000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.861+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610017070 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.862+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61000aca0 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.862+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62014a1b0 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.862+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa610012620 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.862+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62014a770 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.862+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62004f330 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.863+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa61001a070 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.863+0000 7fa61effd640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 0x7fa60c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.863+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa61008a300 con 0x7fa6200c2aa0 2024-03-06T07:20:30.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.863+0000 7fa6263aa640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 0x7fa60c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:30.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.864+0000 7fa6263aa640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 0x7fa60c06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa618036000 tx=0x7fa618008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:30.868+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa6100587e0 con 0x7fa6200c2aa0 2024-03-06T07:20:31.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.048+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6200c0c40 con 0x7fa6200c2aa0 2024-03-06T07:20:31.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.048+0000 7fa61effd640 1 -- 172.21.15.17:0/1789458571 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa61005c880 con 0x7fa6200c2aa0 2024-03-06T07:20:31.050 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:31.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 msgr2=0x7fa60c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 0x7fa60c06d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa618036000 tx=0x7fa618008030 comp rx=0 tx=0).stop 2024-03-06T07:20:31.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 msgr2=0x7fa620147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa61000cb40 tx=0x7fa61000a060 comp rx=0 tx=0).stop 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 shutdown_connections 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60c06b4d0 0x7fa60c06d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6200c2aa0 0x7fa620147760 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6200c26c0 0x7fa620148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 --2- 172.21.15.17:0/1789458571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6200c0470 0x7fa6200c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 >> 172.21.15.17:0/1789458571 conn(0x7fa6200c3b40 msgr2=0x7fa6200c1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 shutdown_connections 2024-03-06T07:20:31.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.050+0000 7fa62860e640 1 -- 172.21.15.17:0/1789458571 wait complete. 2024-03-06T07:20:31.063 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:31.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.165+0000 7f629697a640 1 Processor -- start 2024-03-06T07:20:31.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.165+0000 7f629697a640 1 -- start start 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f629697a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 0x7f6290152d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f629697a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6290155f70 con 0x7f629006d800 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f629697a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6290156250 con 0x7f62900a5eb0 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f629697a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c49a0 con 0x7f6290152920 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f629006d800 0x7f62900bfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35732/0 (socket says 172.21.15.17:35732) 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628f7fe640 1 -- 172.21.15.17:0/1541651532 learned_addr learned my addr 172.21.15.17:0/1541651532 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628ffff640 1 --2- 172.21.15.17:0/1541651532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 0x7f6290152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585035548 0 0) 0x7f6290155f70 con 0x7f629006d800 2024-03-06T07:20:31.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f627c003520 con 0x7f629006d800 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973457790 0 0) 0x7f6290156250 con 0x7f62900a5eb0 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6290155f70 con 0x7f62900a5eb0 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628ffff640 1 -- 172.21.15.17:0/1541651532 >> v1:172.21.15.163:6789/0 conn(0x7f62900a5eb0 legacy=0x7f62900c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628ffff640 1 -- 172.21.15.17:0/1541651532 >> v1:172.21.15.17:6789/0 conn(0x7f629006d800 legacy=0x7f62900bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.166+0000 7f628ffff640 1 -- 172.21.15.17:0/1541651532 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62900c4c80 con 0x7f6290152920 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.167+0000 7f628ffff640 1 --2- 172.21.15.17:0/1541651532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 0x7f6290152d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62780097d0 tx=0x7f627802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5f29cc0fe6ceee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.167+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278035020 con 0x7f6290152920 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.167+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278031800 con 0x7f6290152920 2024-03-06T07:20:31.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.167+0000 7f628effd640 1 -- 172.21.15.17:0/1541651532 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278031bb0 con 0x7f6290152920 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 -- 172.21.15.17:0/1541651532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 msgr2=0x7f6290152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 --2- 172.21.15.17:0/1541651532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 0x7f6290152d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62780097d0 tx=0x7f627802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 -- 172.21.15.17:0/1541651532 shutdown_connections 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 --2- 172.21.15.17:0/1541651532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6290152920 0x7f6290152d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 -- 172.21.15.17:0/1541651532 >> 172.21.15.17:0/1541651532 conn(0x7f629006cb60 msgr2=0x7f62900a86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 -- 172.21.15.17:0/1541651532 shutdown_connections 2024-03-06T07:20:31.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.168+0000 7f629697a640 1 -- 172.21.15.17:0/1541651532 wait complete. 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.169+0000 7f629697a640 1 Processor -- start 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.169+0000 7f629697a640 1 -- start start 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 0x7f629014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 0x7f629014b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c72c0 con 0x7f629006d800 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c7140 con 0x7f6290152920 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f629697a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900a6340 con 0x7f62900a5eb0 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45518/0 (socket says 172.21.15.17:45518) 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 -- 172.21.15.17:0/1922940149 learned_addr learned my addr 172.21.15.17:0/1922940149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628f7fe640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 0x7f629014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f6294f17640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 0x7f629014b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 -- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 msgr2=0x7f629014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 0x7f629014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 -- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 msgr2=0x7f629014b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 0x7f629014b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 -- 172.21.15.17:0/1922940149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629014e050 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628f7fe640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 0x7f629014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f6294f17640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 0x7f629014b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.170+0000 7f628ffff640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f627802b8e0 tx=0x7f62780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.171+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278035040 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.171+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627803ad00 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.171+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62900c36c0 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.171+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780466c0 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.171+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62900c3960 con 0x7f629006d800 2024-03-06T07:20:31.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.172+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6290156250 con 0x7f629006d800 2024-03-06T07:20:31.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.172+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6278038020 con 0x7f629006d800 2024-03-06T07:20:31.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.172+0000 7f628cff9640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.172+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f62780ac610 con 0x7f629006d800 2024-03-06T07:20:31.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.173+0000 7f628f7fe640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.173+0000 7f628f7fe640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.177+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6290156250 con 0x7f629006d800 2024-03-06T07:20:31.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.356+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f629014b100 con 0x7f629006d800 2024-03-06T07:20:31.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.356+0000 7f628cff9640 1 -- 172.21.15.17:0/1922940149 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f629014b100 con 0x7f629006d800 2024-03-06T07:20:31.358 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 msgr2=0x7f627006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6280007fd0 tx=0x7f6280009040 comp rx=0 tx=0).stop 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 msgr2=0x7f629014c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f627802b8e0 tx=0x7f62780046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 shutdown_connections 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6290152920 0x7f629014b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62900a5eb0 0x7f629014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 --2- 172.21.15.17:0/1922940149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f629006d800 0x7f629014c850 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 >> 172.21.15.17:0/1922940149 conn(0x7f629006cb60 msgr2=0x7f62900709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 shutdown_connections 2024-03-06T07:20:31.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.358+0000 7f629697a640 1 -- 172.21.15.17:0/1922940149 wait complete. 2024-03-06T07:20:31.370 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:31.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.473+0000 7f24a4ab5640 1 Processor -- start 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.473+0000 7f24a4ab5640 1 -- start start 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f24a4ab5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24a00c2aa0 0x7f24a0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f24a4ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a0152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f24a4ab5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a00670c0 con 0x7f24a00c26c0 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f24a4ab5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a006b740 con 0x7f24a00c2aa0 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f24a4ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a006b0f0 con 0x7f24a00c0470 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f249e59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f24a00c26c0 0x7f24a006bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35734/0 (socket says 172.21.15.17:35734) 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f249e59c640 1 -- 172.21.15.17:0/1919607494 learned_addr learned my addr 172.21.15.17:0/1919607494 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f2497fff640 1 --2- 172.21.15.17:0/1919607494 >> v2:172.21.15.163:3300/0 conn(0x7f24a00c2aa0 0x7f24a0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f249ed9d640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f249dd9b640 1 -- 172.21.15.17:0/1919607494 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962837346 0 0) 0x7f24a00670c0 con 0x7f24a00c26c0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f2497fff640 1 -- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 msgr2=0x7f24a0152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f2497fff640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a0152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f2497fff640 1 -- 172.21.15.17:0/1919607494 >> v1:172.21.15.17:6789/0 conn(0x7f24a00c26c0 legacy=0x7f24a006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.474+0000 7f2497fff640 1 -- 172.21.15.17:0/1919607494 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a006b3d0 con 0x7f24a00c2aa0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.475+0000 7f2497fff640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a0155bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24880097a0 tx=0x7f248802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae100a3c85e2692 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.475+0000 7f249dd9b640 1 -- 172.21.15.17:0/1919607494 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488039070 con 0x7f24a00c2aa0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.475+0000 7f249dd9b640 1 -- 172.21.15.17:0/1919607494 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24880316f0 con 0x7f24a00c2aa0 2024-03-06T07:20:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.475+0000 7f249dd9b640 1 -- 172.21.15.17:0/1919607494 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488031ac0 con 0x7f24a00c2aa0 2024-03-06T07:20:31.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 -- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 msgr2=0x7f24a0155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a0155bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24880097a0 tx=0x7f248802b790 comp rx=0 tx=0).stop 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 -- 172.21.15.17:0/1919607494 shutdown_connections 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a0152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/1919607494 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a0155bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 -- 172.21.15.17:0/1919607494 >> 172.21.15.17:0/1919607494 conn(0x7f24a00c3b40 msgr2=0x7f24a00c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 -- 172.21.15.17:0/1919607494 shutdown_connections 2024-03-06T07:20:31.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.476+0000 7f24a4ab5640 1 -- 172.21.15.17:0/1919607494 wait complete. 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.477+0000 7f24a4ab5640 1 Processor -- start 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.477+0000 7f24a4ab5640 1 -- start start 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a01ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00c20b0 con 0x7f24a00c26c0 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00c1f30 con 0x7f24a00c2aa0 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0066d00 con 0x7f24a00c0470 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45534/0 (socket says 172.21.15.17:45534) 2024-03-06T07:20:31.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 -- 172.21.15.17:0/2936115488 learned_addr learned my addr 172.21.15.17:0/2936115488 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f249e59c640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f249ed9d640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a01ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 -- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 msgr2=0x7f24a014c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a014c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 -- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 msgr2=0x7f24a01ead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a01ead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 -- 172.21.15.17:0/2936115488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a01e7ee0 con 0x7f24a00c26c0 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f249ed9d640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a01ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2497fff640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2488037040 tx=0x7f24880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488043070 con 0x7f24a00c26c0 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488034680 con 0x7f24a00c26c0 2024-03-06T07:20:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a00ac310 con 0x7f24a00c26c0 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.478+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488039040 con 0x7f24a00c26c0 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.479+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a00aa300 con 0x7f24a00c26c0 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.480+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a00ac830 con 0x7f24a00c26c0 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.480+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2488046070 con 0x7f24a00c26c0 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.480+0000 7f2496ffd640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.480+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f24880abec0 con 0x7f24a00c26c0 2024-03-06T07:20:31.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.480+0000 7f249e59c640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.481+0000 7f249e59c640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 0x7f248006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2490000f60 tx=0x7f2490007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.484+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24a00ac830 con 0x7f24a00c26c0 2024-03-06T07:20:31.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.665+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f24a004f330 con 0x7f24a00c26c0 2024-03-06T07:20:31.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.665+0000 7f2496ffd640 1 -- 172.21.15.17:0/2936115488 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f248807a560 con 0x7f24a00c26c0 2024-03-06T07:20:31.668 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 msgr2=0x7f248006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 0x7f248006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2490000f60 tx=0x7f2490007d50 comp rx=0 tx=0).stop 2024-03-06T07:20:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 msgr2=0x7f24a0142eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2488037040 tx=0x7f24880046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 shutdown_connections 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f248006b4d0 0x7f248006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24a00c2aa0 0x7f24a01ead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24a00c26c0 0x7f24a0142eb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 --2- 172.21.15.17:0/2936115488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24a00c0470 0x7f24a014c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 >> 172.21.15.17:0/2936115488 conn(0x7f24a00c3b40 msgr2=0x7f24a0152120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 shutdown_connections 2024-03-06T07:20:31.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.667+0000 7f24a4ab5640 1 -- 172.21.15.17:0/2936115488 wait complete. 2024-03-06T07:20:31.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:31.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.780+0000 7fe3ed6e0640 1 Processor -- start 2024-03-06T07:20:31.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.780+0000 7fe3ed6e0640 1 -- start start 2024-03-06T07:20:31.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3ed6e0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe3e806d9f0 0x7fe3e80a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3ed6e0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3710 0x7fe3e81558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3ed6e0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e8158800 con 0x7fe3e80c3280 2024-03-06T07:20:31.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3ed6e0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e8158ae0 con 0x7fe3e806d9f0 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3ed6e0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8158dc0 con 0x7fe3e80c3710 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe3e806d9f0 0x7fe3e80a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe3e80c3280 0x7fe3e81551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35750/0 (socket says 172.21.15.17:35750) 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e67fc640 1 -- 172.21.15.17:0/858702161 learned_addr learned my addr 172.21.15.17:0/858702161 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e77fe640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3710 0x7fe3e81558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e6ffd640 1 -- 172.21.15.17:0/858702161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3710 msgr2=0x7fe3e81558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e6ffd640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3710 0x7fe3e81558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e6ffd640 1 -- 172.21.15.17:0/858702161 >> v1:172.21.15.17:6789/0 conn(0x7fe3e80c3280 legacy=0x7fe3e81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.781+0000 7fe3e6ffd640 1 -- 172.21.15.17:0/858702161 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e81590a0 con 0x7fe3e806d9f0 2024-03-06T07:20:31.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.782+0000 7fe3e6ffd640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e806d9f0 0x7fe3e80a7cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097a0 tx=0x7fe3e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40436c73142b203 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.782+0000 7fe3e5ffb640 1 -- 172.21.15.17:0/858702161 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00315a0 con 0x7fe3e806d9f0 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.782+0000 7fe3e5ffb640 1 -- 172.21.15.17:0/858702161 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0037070 con 0x7fe3e806d9f0 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.782+0000 7fe3e5ffb640 1 -- 172.21.15.17:0/858702161 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0031960 con 0x7fe3e806d9f0 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/858702161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e806d9f0 msgr2=0x7fe3e80a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e806d9f0 0x7fe3e80a7cd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00097a0 tx=0x7fe3e002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/858702161 shutdown_connections 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3710 0x7fe3e81558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/858702161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e806d9f0 0x7fe3e80a7cd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/858702161 >> 172.21.15.17:0/858702161 conn(0x7fe3e80aa630 msgr2=0x7fe3e80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/858702161 shutdown_connections 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.783+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/858702161 wait complete. 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.784+0000 7fe3ed6e0640 1 Processor -- start 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.784+0000 7fe3ed6e0640 1 -- start start 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e806d9f0 0x7fe3e81ee400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3280 0x7fe3e814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e806ddd0 con 0x7fe3e80c3710 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8155e30 con 0x7fe3e806d9f0 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3ed6e0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e806df50 con 0x7fe3e80c3280 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e6ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e806d9f0 0x7fe3e81ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45544/0 (socket says 172.21.15.17:45544) 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e67fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3280 0x7fe3e814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 -- 172.21.15.17:0/3694354255 learned_addr learned my addr 172.21.15.17:0/3694354255 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 -- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3280 msgr2=0x7fe3e814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3280 0x7fe3e814f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 -- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e806d9f0 msgr2=0x7fe3e81ee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e806d9f0 0x7fe3e81ee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 -- 172.21.15.17:0/3694354255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e81584a0 con 0x7fe3e80c3710 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3e77fe640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800efc0 tx=0x7fe3d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.785+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8018020 con 0x7fe3e80c3710 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.786+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8009b10 con 0x7fe3e80c3710 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.786+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e80ab6c0 con 0x7fe3e80c3710 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.786+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d80146c0 con 0x7fe3e80c3710 2024-03-06T07:20:31.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.786+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e80ab960 con 0x7fe3e80c3710 2024-03-06T07:20:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.787+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e804f330 con 0x7fe3e80c3710 2024-03-06T07:20:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.787+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3d8005d10 con 0x7fe3e80c3710 2024-03-06T07:20:31.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.787+0000 7fe3df7fe640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 0x7fe3c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.788+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3d808b0b0 con 0x7fe3e80c3710 2024-03-06T07:20:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.788+0000 7fe3e6ffd640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 0x7fe3c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:31.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.788+0000 7fe3e6ffd640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 0x7fe3c806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0036000 tx=0x7fe3e003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:31.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:31.792+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3d8022030 con 0x7fe3e80c3710 2024-03-06T07:20:32.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.016+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e80ab160 con 0x7fe3e80c3710 2024-03-06T07:20:32.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.016+0000 7fe3df7fe640 1 -- 172.21.15.17:0/3694354255 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7fe3e81584a0 con 0x7fe3e80c3710 2024-03-06T07:20:32.019 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:32.019 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:32.020 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 msgr2=0x7fe3c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 0x7fe3c806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0036000 tx=0x7fe3e003d040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 msgr2=0x7fe3e81e1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800efc0 tx=0x7fe3d800c490 comp rx=0 tx=0).stop 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 shutdown_connections 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3c806b4d0 0x7fe3c806d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e80c3710 0x7fe3e81e1a60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3e80c3280 0x7fe3e814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 --2- 172.21.15.17:0/3694354255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3e806d9f0 0x7fe3e81ee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.018+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 >> 172.21.15.17:0/3694354255 conn(0x7fe3e80aa630 msgr2=0x7fe3e80c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.019+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 shutdown_connections 2024-03-06T07:20:32.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.019+0000 7fe3ed6e0640 1 -- 172.21.15.17:0/3694354255 wait complete. 2024-03-06T07:20:32.031 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:32.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.130+0000 7fb866b19640 1 Processor -- start 2024-03-06T07:20:32.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.130+0000 7fb866b19640 1 -- start start 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb866b19640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb860155c80 0x7fb86006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb866b19640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8600720a0 0x7fb86006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb866b19640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8600c1ae0 con 0x7fb860152520 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb866b19640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8600c1dc0 con 0x7fb8600720a0 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb866b19640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb86006e450 con 0x7fb860155c80 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8600720a0 0x7fb86006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb8600720a0 0x7fb86006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45552/0 (socket says 172.21.15.17:45552) 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 -- 172.21.15.17:0/3756927489 learned_addr learned my addr 172.21.15.17:0/3756927489 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb85ffff640 1 --2- 172.21.15.17:0/3756927489 >> v2:172.21.15.163:3300/0 conn(0x7fb860155c80 0x7fb86006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 -- 172.21.15.17:0/3756927489 >> v2:172.21.15.163:3300/0 conn(0x7fb860155c80 msgr2=0x7fb86006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 --2- 172.21.15.17:0/3756927489 >> v2:172.21.15.163:3300/0 conn(0x7fb860155c80 0x7fb86006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 -- 172.21.15.17:0/3756927489 >> v1:172.21.15.163:6789/0 conn(0x7fb860152520 legacy=0x7fb8600c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.131+0000 7fb8650b6640 1 -- 172.21.15.17:0/3756927489 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86006e730 con 0x7fb8600720a0 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.132+0000 7fb8650b6640 1 --2- 172.21.15.17:0/3756927489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8600720a0 0x7fb86006afc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb860073750 tx=0x7fb85002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212136c992a39807 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.132+0000 7fb85f7fe640 1 -- 172.21.15.17:0/3756927489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850034070 con 0x7fb8600720a0 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.132+0000 7fb85f7fe640 1 -- 172.21.15.17:0/3756927489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850030860 con 0x7fb8600720a0 2024-03-06T07:20:32.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.132+0000 7fb85f7fe640 1 -- 172.21.15.17:0/3756927489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850030c30 con 0x7fb8600720a0 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 -- 172.21.15.17:0/3756927489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8600720a0 msgr2=0x7fb86006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 --2- 172.21.15.17:0/3756927489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8600720a0 0x7fb86006afc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb860073750 tx=0x7fb85002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 -- 172.21.15.17:0/3756927489 shutdown_connections 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 --2- 172.21.15.17:0/3756927489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8600720a0 0x7fb86006afc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 --2- 172.21.15.17:0/3756927489 >> v2:172.21.15.163:3300/0 conn(0x7fb860155c80 0x7fb86006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 -- 172.21.15.17:0/3756927489 >> 172.21.15.17:0/3756927489 conn(0x7fb8600c76b0 msgr2=0x7fb8600c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 -- 172.21.15.17:0/3756927489 shutdown_connections 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.133+0000 7fb866b19640 1 -- 172.21.15.17:0/3756927489 wait complete. 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.134+0000 7fb866b19640 1 Processor -- start 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.134+0000 7fb866b19640 1 -- start start 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8600720a0 0x7fb8601ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb860152520 0x7fb8601de640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860152a80 con 0x7fb860155c80 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb860152900 con 0x7fb8600720a0 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb866b19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86006b500 con 0x7fb860152520 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45554/0 (socket says 172.21.15.17:45554) 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb860152520 0x7fb8601de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 -- 172.21.15.17:0/1073709400 learned_addr learned my addr 172.21.15.17:0/1073709400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8648b5640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8600720a0 0x7fb8601ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 -- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb860152520 msgr2=0x7fb8601de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb860152520 0x7fb8601de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 -- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8600720a0 msgr2=0x7fb8601ede30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8600720a0 0x7fb8601ede30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 -- 172.21.15.17:0/1073709400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86014fb90 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.135+0000 7fb8650b6640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb85002b9e0 tx=0x7fb8500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.136+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850034040 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.136+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850039d90 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.136+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850043610 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.136+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8601ee670 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.136+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8601ee370 con 0x7fb860155c80 2024-03-06T07:20:32.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.137+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004f330 con 0x7fb860155c80 2024-03-06T07:20:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.137+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb850037070 con 0x7fb860155c80 2024-03-06T07:20:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.137+0000 7fb85d7fa640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 0x7fb84c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.137+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb8500ac450 con 0x7fb860155c80 2024-03-06T07:20:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.138+0000 7fb8648b5640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 0x7fb84c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.138+0000 7fb8648b5640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 0x7fb84c06d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8600c6d50 tx=0x7fb858009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.142+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb85003e070 con 0x7fb860155c80 2024-03-06T07:20:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.323+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb860074700 con 0x7fb860155c80 2024-03-06T07:20:32.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.324+0000 7fb85d7fa640 1 -- 172.21.15.17:0/1073709400 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fb85007a9d0 con 0x7fb860155c80 2024-03-06T07:20:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:32.326 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 msgr2=0x7fb84c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 0x7fb84c06d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8600c6d50 tx=0x7fb858009040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 msgr2=0x7fb8601deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb85002b9e0 tx=0x7fb8500046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 shutdown_connections 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb84c06b500 0x7fb84c06d9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb860155c80 0x7fb8601deb80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb860152520 0x7fb8601de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 --2- 172.21.15.17:0/1073709400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8600720a0 0x7fb8601ede30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 >> 172.21.15.17:0/1073709400 conn(0x7fb8600c76b0 msgr2=0x7fb8600c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 shutdown_connections 2024-03-06T07:20:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.326+0000 7fb866b19640 1 -- 172.21.15.17:0/1073709400 wait complete. 2024-03-06T07:20:32.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.438+0000 7efd4eff7640 1 Processor -- start 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.438+0000 7efd4eff7640 1 -- start start 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4eff7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efd481506b0 0x7efd48153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4eff7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd48156ea0 0x7efd480702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4eff7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48070800 con 0x7efd480acc00 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4eff7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48156450 con 0x7efd48156ea0 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4eff7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd48156a10 con 0x7efd481506b0 2024-03-06T07:20:32.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd47fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd48156ea0 0x7efd480702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd47fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd48156ea0 0x7efd480702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45564/0 (socket says 172.21.15.17:45564) 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd47fff640 1 -- 172.21.15.17:0/319717320 learned_addr learned my addr 172.21.15.17:0/319717320 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.439+0000 7efd4cd93640 1 --2- 172.21.15.17:0/319717320 >> v2:172.21.15.163:3300/0 conn(0x7efd481506b0 0x7efd48153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd47fff640 1 -- 172.21.15.17:0/319717320 >> v2:172.21.15.163:3300/0 conn(0x7efd481506b0 msgr2=0x7efd48153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd47fff640 1 --2- 172.21.15.17:0/319717320 >> v2:172.21.15.163:3300/0 conn(0x7efd481506b0 0x7efd48153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd47fff640 1 -- 172.21.15.17:0/319717320 >> v1:172.21.15.163:6789/0 conn(0x7efd480acc00 legacy=0x7efd480c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd47fff640 1 -- 172.21.15.17:0/319717320 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48070aa0 con 0x7efd48156ea0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd47fff640 1 --2- 172.21.15.17:0/319717320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd48156ea0 0x7efd480702c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd481565d0 tx=0x7efd3c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e53bb135d283d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.440+0000 7efd477fe640 1 -- 172.21.15.17:0/319717320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c034070 con 0x7efd48156ea0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd477fe640 1 -- 172.21.15.17:0/319717320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c030860 con 0x7efd48156ea0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd477fe640 1 -- 172.21.15.17:0/319717320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c030c30 con 0x7efd48156ea0 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 -- 172.21.15.17:0/319717320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd48156ea0 msgr2=0x7efd480702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 --2- 172.21.15.17:0/319717320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd48156ea0 0x7efd480702c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd481565d0 tx=0x7efd3c02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 -- 172.21.15.17:0/319717320 shutdown_connections 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 --2- 172.21.15.17:0/319717320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd48156ea0 0x7efd480702c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 --2- 172.21.15.17:0/319717320 >> v2:172.21.15.163:3300/0 conn(0x7efd481506b0 0x7efd48153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 -- 172.21.15.17:0/319717320 >> 172.21.15.17:0/319717320 conn(0x7efd4806b1a0 msgr2=0x7efd4806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 -- 172.21.15.17:0/319717320 shutdown_connections 2024-03-06T07:20:32.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.441+0000 7efd4eff7640 1 -- 172.21.15.17:0/319717320 wait complete. 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.442+0000 7efd4eff7640 1 Processor -- start 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.442+0000 7efd4eff7640 1 -- start start 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd481506b0 0x7efd481535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd48156ea0 0x7efd481ec550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c5bb0 con 0x7efd480acc00 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd481566a0 con 0x7efd481506b0 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4eff7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c5d30 con 0x7efd48156ea0 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45578/0 (socket says 172.21.15.17:45578) 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 -- 172.21.15.17:0/1647256020 learned_addr learned my addr 172.21.15.17:0/1647256020 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd47fff640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd481506b0 0x7efd481535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4d594640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd48156ea0 0x7efd481ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 -- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd48156ea0 msgr2=0x7efd481ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd48156ea0 0x7efd481ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 -- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd481506b0 msgr2=0x7efd481535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd481506b0 0x7efd481535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.443+0000 7efd4cd93640 1 -- 172.21.15.17:0/1647256020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480ab0d0 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd4cd93640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd40009db0 tx=0x7efd4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40018020 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40014480 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd480ab3b0 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40014850 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.444+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd480a8db0 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.445+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4804f330 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.445+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd40005d10 con 0x7efd480acc00 2024-03-06T07:20:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.446+0000 7efd457fa640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:32.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.446+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd4008aea0 con 0x7efd480acc00 2024-03-06T07:20:32.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.446+0000 7efd47fff640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:32.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.446+0000 7efd47fff640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036000 tx=0x7efd3c03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:32.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.450+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd40022070 con 0x7efd480acc00 2024-03-06T07:20:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.668+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd48156230 con 0x7efd480acc00 2024-03-06T07:20:32.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.668+0000 7efd457fa640 1 -- 172.21.15.17:0/1647256020 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7efd40010040 con 0x7efd480acc00 2024-03-06T07:20:32.670 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:32.670 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:32.671 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:32.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 msgr2=0x7efd3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036000 tx=0x7efd3c03b040 comp rx=0 tx=0).stop 2024-03-06T07:20:32.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 msgr2=0x7efd481dff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:32.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd40009db0 tx=0x7efd4000c490 comp rx=0 tx=0).stop 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 shutdown_connections 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd48156ea0 0x7efd481ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd481506b0 0x7efd481535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 --2- 172.21.15.17:0/1647256020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd480acc00 0x7efd481dff80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 >> 172.21.15.17:0/1647256020 conn(0x7efd4806b1a0 msgr2=0x7efd480c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 shutdown_connections 2024-03-06T07:20:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:32.670+0000 7efd4eff7640 1 -- 172.21.15.17:0/1647256020 wait complete. 2024-03-06T07:20:33.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:20:04.076240+0000', 'modified': '2024-03-06T07:20:20.134434+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2082625649', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2082625649}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2082625649}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4264, 'qdb_cluster': [4264]} 2024-03-06T07:20:33.085 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:20:33.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:20:33.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.187+0000 7f683cf1e640 1 Processor -- start 2024-03-06T07:20:33.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.187+0000 7f683cf1e640 1 -- start start 2024-03-06T07:20:33.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f683cf1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f683806ff10 0x7f6838153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f683cf1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380a8390 con 0x7f68380c7170 2024-03-06T07:20:33.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f683cf1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380730f0 con 0x7f68380c8f20 2024-03-06T07:20:33.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f683cf1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380c7620 con 0x7f683806ff10 2024-03-06T07:20:33.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f683659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f683806ff10 0x7f6838153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f6835d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68380c7170 0x7f68380c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35754/0 (socket says 172.21.15.17:35754) 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.188+0000 7f6835d9b640 1 -- 172.21.15.17:0/311166358 learned_addr learned my addr 172.21.15.17:0/311166358 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.189+0000 7f683659c640 1 -- 172.21.15.17:0/311166358 >> v1:172.21.15.163:6789/0 conn(0x7f68380c8f20 legacy=0x7f683815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.189+0000 7f683659c640 1 -- 172.21.15.17:0/311166358 >> v1:172.21.15.17:6789/0 conn(0x7f68380c7170 legacy=0x7f68380c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.189+0000 7f683659c640 1 -- 172.21.15.17:0/311166358 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380733d0 con 0x7f683806ff10 2024-03-06T07:20:33.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.189+0000 7f683559a640 1 -- 172.21.15.17:0/311166358 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411162135 0 0) 0x7f68380a8390 con 0x7f68380c7170 2024-03-06T07:20:33.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.189+0000 7f683659c640 1 --2- 172.21.15.17:0/311166358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f683806ff10 0x7f6838153c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f68380c77a0 tx=0x7f683002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4529d74de7463271 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683559a640 1 -- 172.21.15.17:0/311166358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68300315a0 con 0x7f683806ff10 2024-03-06T07:20:33.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683559a640 1 -- 172.21.15.17:0/311166358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830037070 con 0x7f683806ff10 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683559a640 1 -- 172.21.15.17:0/311166358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830031960 con 0x7f683806ff10 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 -- 172.21.15.17:0/311166358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f683806ff10 msgr2=0x7f6838153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 --2- 172.21.15.17:0/311166358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f683806ff10 0x7f6838153c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f68380c77a0 tx=0x7f683002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 -- 172.21.15.17:0/311166358 shutdown_connections 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 --2- 172.21.15.17:0/311166358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f683806ff10 0x7f6838153c20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 -- 172.21.15.17:0/311166358 >> 172.21.15.17:0/311166358 conn(0x7f683806b970 msgr2=0x7f68380aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 -- 172.21.15.17:0/311166358 shutdown_connections 2024-03-06T07:20:33.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.190+0000 7f683cf1e640 1 -- 172.21.15.17:0/311166358 wait complete. 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.191+0000 7f683cf1e640 1 Processor -- start 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.191+0000 7f683cf1e640 1 -- start start 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f683806ff10 0x7f68380c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68380c7170 0x7f68381e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838070470 con 0x7f68380c8f20 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380702f0 con 0x7f68380c7170 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683cf1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c6860 con 0x7f683806ff10 2024-03-06T07:20:33.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45588/0 (socket says 172.21.15.17:45588) 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 -- 172.21.15.17:0/3514983698 learned_addr learned my addr 172.21.15.17:0/3514983698 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f683659c640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f683806ff10 0x7f68380c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6835d9b640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68380c7170 0x7f68381e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 -- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f683806ff10 msgr2=0x7f68380c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f683806ff10 0x7f68380c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 -- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68380c7170 msgr2=0x7f68381e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68380c7170 0x7f68381e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.192+0000 7f6836d9d640 1 -- 172.21.15.17:0/3514983698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380c2240 con 0x7f68380c8f20 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f6836d9d640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f682800c6e0 tx=0x7f682800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f682effd640 1 -- 172.21.15.17:0/3514983698 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828003b70 con 0x7f68380c8f20 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f682effd640 1 -- 172.21.15.17:0/3514983698 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6828003d30 con 0x7f68380c8f20 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f682effd640 1 -- 172.21.15.17:0/3514983698 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828005310 con 0x7f68380c8f20 2024-03-06T07:20:33.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380be840 con 0x7f68380c8f20 2024-03-06T07:20:33.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.193+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380be560 con 0x7f68380c8f20 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.194+0000 7f682effd640 1 -- 172.21.15.17:0/3514983698 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68280054b0 con 0x7f68380c8f20 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.195+0000 7f682effd640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 0x7f681806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.195+0000 7f682effd640 1 -- 172.21.15.17:0/3514983698 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f682808a240 con 0x7f68380c8f20 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.195+0000 7f683659c640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 0x7f681806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.195+0000 7f683659c640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 0x7f681806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6830036000 tx=0x7f6830008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 Processor -- start 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 -- start start 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d9152a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d9152f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d90b96b0 con 0x5632d91534b0 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d9062a60 con 0x5632d9155330 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.196+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d90b5750 con 0x5632d9154f50 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45602/0 (socket says 172.21.15.17:45602) 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 -- 172.21.15.17:0/1456113290 learned_addr learned my addr 172.21.15.17:0/1456113290 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6835d9b640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d9152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f683659c640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d9152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 -- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 msgr2=0x5632d9152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d9152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 -- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 msgr2=0x5632d9152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d9152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 -- 172.21.15.17:0/1456113290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d9080f30 con 0x5632d91534b0 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6835d9b640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d9152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6836d9d640 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6828060850 tx=0x7f6828011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7df0bffa9d37b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6817fff640 1 -- 172.21.15.17:0/1456113290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280701d0 con 0x5632d91534b0 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6817fff640 1 -- 172.21.15.17:0/1456113290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280567c0 con 0x5632d91534b0 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.197+0000 7f6817fff640 1 -- 172.21.15.17:0/1456113290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828054d90 con 0x5632d91534b0 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- 172.21.15.17:0/1456113290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 msgr2=0x5632d914f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6828060850 tx=0x7f6828011000 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- 172.21.15.17:0/1456113290 shutdown_connections 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91534b0 0x5632d914f680 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d9152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 --2- 172.21.15.17:0/1456113290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d9152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- 172.21.15.17:0/1456113290 >> 172.21.15.17:0/1456113290 conn(0x5632d91517d0 msgr2=0x5632d91763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- 172.21.15.17:0/1456113290 shutdown_connections 2024-03-06T07:20:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- 172.21.15.17:0/1456113290 wait complete. 2024-03-06T07:20:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.198+0000 7f683fbe5740 1 -- start start 2024-03-06T07:20:33.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 Processor -- start 2024-03-06T07:20:33.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 0x5632d9178c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9171f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d9177230 con 0x5632d9155330 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d9075ea0 con 0x5632d9154f50 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d9176c70 con 0x5632d91a4ad0 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683fbe5740 10 client.0 _ll_register_callbacks cb 0x5632d9075870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f6835d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f6835d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45614/0 (socket says 172.21.15.17:45614) 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f6835d9b640 1 -- 172.21.15.17:0/186544681 learned_addr learned my addr 172.21.15.17:0/186544681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.199+0000 7f683659c640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 0x5632d9178c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6836d9d640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9171f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 -- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 msgr2=0x5632d9171f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9171f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 -- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 msgr2=0x5632d9178c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 0x5632d9178c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 -- 172.21.15.17:0/186544681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d9076270 con 0x5632d9155330 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f683659c640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 0x5632d9178c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f6835d9b640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6820004e80 tx=0x7f6820001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63d5fa195b117e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f68157fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820001cf0 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6820010040 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68280567c0 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6820015650 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f683fbe5740 10 client.4842 fetch_fsmap learned FSMap version 9 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5632d8dce2d0 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.200+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280567c0 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 9) v1 ==== 3322+0+0 (secure 0 0 0) 0x7f68200157f0 con 0x5632d9155330 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 10 client.4842 fetch_fsmap finished waiting for FSMap version 9 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 10 client.4842 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:20:33.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 20 client.4842 populate_metadata read hostname 'smithi017' 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d91781b0 0x5632d91b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 4 client.4842 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 --> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632d90c6bf0 con 0x5632d91781b0 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683659c640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d91781b0 0x5632d91b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f683659c640 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d91781b0 0x5632d91b0110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00f3781add011b7 server_cookie=280f4997671478af in_seq=0 out_seq=0 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.201+0000 7f68157fa640 10 client.4842 ms_handle_connect on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f68157fa640 1 -- 172.21.15.17:0/186544681 <== mds.0 v2:172.21.15.163:6835/2082625649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5632d90c6bf0 con 0x5632d91781b0 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f68157fa640 10 client.4842 handle_command_reply: tid=0 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 client.4842 shutdown 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 msgr2=0x5632d917aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6820004e80 tx=0x7f6820001680 comp rx=0 tx=0).stop 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 shutdown_connections 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d91781b0 0x5632d91b0110 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9171f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f68157fa640 0 client.4842 ms_handle_reset on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9155330 0x5632d917aa30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 --2- 172.21.15.17:0/186544681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9154f50 0x5632d9178c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 >> 172.21.15.17:0/186544681 conn(0x5632d91534b0 msgr2=0x5632d914e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 shutdown_connections 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 1 -- 172.21.15.17:0/186544681 wait complete. 2024-03-06T07:20:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.202+0000 7f683fbe5740 20 client.4842 trim_cache size 0 max 16384 2024-03-06T07:20:33.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 Processor -- start 2024-03-06T07:20:33.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 -- start start 2024-03-06T07:20:33.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f5a20 con 0x5632d91a4ad0 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68381ec8e0 con 0x5632d9155330 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.219+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68381f5480 con 0x5632d9154f50 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f683659c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f683659c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:54452/0 (socket says 172.21.15.17:54452) 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45624/0 (socket says 172.21.15.17:45624) 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f683659c640 1 -- 172.21.15.17:0/4274194256 learned_addr learned my addr 172.21.15.17:0/4274194256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6835d9b640 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 -- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 msgr2=0x5632d918e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 -- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 msgr2=0x5632d914e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 -- 172.21.15.17:0/4274194256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632d8e986b0 con 0x5632d91a4ad0 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f683659c640 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f6836d9d640 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f682800a430 tx=0x7f6828058620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b68228d96011d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f68177fe640 1 -- 172.21.15.17:0/4274194256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68280549c0 con 0x5632d91a4ad0 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f68177fe640 1 -- 172.21.15.17:0/4274194256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280701d0 con 0x5632d91a4ad0 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.220+0000 7f68177fe640 1 -- 172.21.15.17:0/4274194256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828056a20 con 0x5632d91a4ad0 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- 172.21.15.17:0/4274194256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 msgr2=0x5632d914ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f682800a430 tx=0x7f6828058620 comp rx=0 tx=0).stop 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- 172.21.15.17:0/4274194256 shutdown_connections 2024-03-06T07:20:33.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d91a4ad0 0x5632d914ea50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 --2- 172.21.15.17:0/4274194256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d9154f50 0x5632d918e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- 172.21.15.17:0/4274194256 >> 172.21.15.17:0/4274194256 conn(0x5632d91534b0 msgr2=0x5632d9171d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- 172.21.15.17:0/4274194256 shutdown_connections 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- 172.21.15.17:0/4274194256 wait complete. 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 -- start start 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.221+0000 7f683fbe5740 1 Processor -- start 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9202ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d9179ee0 con 0x5632d9154f50 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632d917b400 con 0x5632d9155330 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632d9177600 con 0x5632d91a4ad0 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683fbe5740 10 client.0 _ll_register_callbacks cb 0x7f6820004ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:20:33.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45630/0 (socket says 172.21.15.17:45630) 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f683659c640 1 -- 172.21.15.17:0/2019866056 learned_addr learned my addr 172.21.15.17:0/2019866056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f6835d9b640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.222+0000 7f6836d9d640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9202ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 -- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 msgr2=0x5632d9202ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9202ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 -- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 msgr2=0x5632d914f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 -- 172.21.15.17:0/2019866056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632d9209280 con 0x5632d9154f50 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f6835d9b640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f6836d9d640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9202ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683659c640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6830009860 tx=0x7f6830004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df05de87f8956091 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:20:33.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683005d020 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6830059230 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68280701d0 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6830067050 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 10 client.4848 fetch_fsmap learned FSMap version 9 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5632d916efe0 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280701d0 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 9) v1 ==== 3322+0+0 (secure 0 0 0) 0x7f683006a020 con 0x5632d9154f50 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 10 client.4848 fetch_fsmap finished waiting for FSMap version 9 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 10 client.4848 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 20 client.4848 populate_metadata read hostname 'smithi017' 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d919da20 0x5632d919de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 4 client.4848 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.223+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 --> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5632d90c0f40 con 0x5632d919da20 2024-03-06T07:20:33.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f6835d9b640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d919da20 0x5632d919de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f6835d9b640 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d919da20 0x5632d919de00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ace9ff8db9902e7 server_cookie=f2e7914b364d6d9f in_seq=0 out_seq=0 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f68157fa640 10 client.4848 ms_handle_connect on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f68157fa640 1 -- 172.21.15.17:0/2019866056 <== mds.0 v2:172.21.15.163:6835/2082625649 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5632d90c0f40 con 0x5632d919da20 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f68157fa640 10 client.4848 handle_command_reply: tid=0 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 client.4848 shutdown 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 msgr2=0x5632d91733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6830009860 tx=0x7f6830004730 comp rx=0 tx=0).stop 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 shutdown_connections 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:6835/2082625649,v1:172.21.15.163:6837/2082625649] conn(0x5632d919da20 0x5632d919de00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5632d91a4ad0 0x5632d9202ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5632d9155330 0x5632d914f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f68157fa640 0 client.4848 ms_handle_reset on v2:172.21.15.163:6835/2082625649 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 --2- 172.21.15.17:0/2019866056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5632d9154f50 0x5632d91733e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 >> 172.21.15.17:0/2019866056 conn(0x5632d91534b0 msgr2=0x5632d9171b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 shutdown_connections 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.224+0000 7f683fbe5740 1 -- 172.21.15.17:0/2019866056 wait complete. 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683fbe5740 20 client.4848 trim_cache size 0 max 16384 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4264, 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:20:33.234 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 28.520258091999999, 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 9, 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 25, 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 25, 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 35.441465057999999 2024-03-06T07:20:33.235 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 msgr2=0x7f681806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 0x7f681806d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6830036000 tx=0x7f6830008030 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 msgr2=0x7f6838155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f682800c6e0 tx=0x7f682800cba0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 shutdown_connections 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f681806b4d0 0x7f681806d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68380c8f20 0x7f6838155fd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68380c7170 0x7f68381e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 --2- 172.21.15.17:0/3514983698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f683806ff10 0x7f68380c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 >> 172.21.15.17:0/3514983698 conn(0x7f683806b970 msgr2=0x7f68380723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 shutdown_connections 2024-03-06T07:20:33.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.225+0000 7f683cf1e640 1 -- 172.21.15.17:0/3514983698 wait complete. 2024-03-06T07:20:33.238 INFO:teuthology.run_tasks:Running task kclient... 2024-03-06T07:20:33.250 INFO:tasks.kclient:Mounting kernel clients... 2024-03-06T07:20:33.250 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-06T07:20:33.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.251 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-06T07:20:33.265 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-06T07:20:33.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-06T07:20:33.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-06T07:20:33.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.429 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-06T07:20:33.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.445 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-03-06T07:20:33.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:20:33.516 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2024-03-06T07:20:33.517 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-06T07:20:33.517 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:33.517 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:33.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:33.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.618+0000 7f90b3ab5640 1 Processor -- start 2024-03-06T07:20:33.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 -- start start 2024-03-06T07:20:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90ac14f360 0x7f90ac155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c88c0 0x7f90ac0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac0c73d0 con 0x7f90ac0c5880 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ac0c8dd0 con 0x7f90ac14f360 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b3ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0be2e0 con 0x7f90ac0c88c0 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.619+0000 7f90b1851640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90ac14f360 0x7f90ac155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90ac14f360 0x7f90ac155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45636/0 (socket says 172.21.15.17:45636) 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 -- 172.21.15.17:0/4039617811 learned_addr learned my addr 172.21.15.17:0/4039617811 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1050640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c88c0 0x7f90ac0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 -- 172.21.15.17:0/4039617811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c88c0 msgr2=0x7f90ac0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c88c0 0x7f90ac0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 -- 172.21.15.17:0/4039617811 >> v1:172.21.15.17:6789/0 conn(0x7f90ac0c5880 legacy=0x7f90ac158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.620+0000 7f90b1851640 1 -- 172.21.15.17:0/4039617811 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0c7670 con 0x7f90ac14f360 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b1851640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac155c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009870 tx=0x7f90a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32805bc84848712 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b084f640 1 -- 172.21.15.17:0/4039617811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8035020 con 0x7f90ac14f360 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b084f640 1 -- 172.21.15.17:0/4039617811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a8031640 con 0x7f90ac14f360 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b084f640 1 -- 172.21.15.17:0/4039617811 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a802d040 con 0x7f90ac14f360 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 -- 172.21.15.17:0/4039617811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 msgr2=0x7f90ac155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac155c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009870 tx=0x7f90a802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 -- 172.21.15.17:0/4039617811 shutdown_connections 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c88c0 0x7f90ac0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/4039617811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac155c10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 -- 172.21.15.17:0/4039617811 >> 172.21.15.17:0/4039617811 conn(0x7f90ac0ac570 msgr2=0x7f90ac0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 -- 172.21.15.17:0/4039617811 shutdown_connections 2024-03-06T07:20:33.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.621+0000 7f90b3ab5640 1 -- 172.21.15.17:0/4039617811 wait complete. 2024-03-06T07:20:33.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.622+0000 7f90b3ab5640 1 Processor -- start 2024-03-06T07:20:33.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.622+0000 7f90b3ab5640 1 -- start start 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c5880 0x7f90ac1ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90ac0c88c0 0x7f90ac14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac159270 con 0x7f90ac14f360 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0c7c30 con 0x7f90ac0c88c0 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b3ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac1593f0 con 0x7f90ac0c5880 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b2052640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b2052640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45650/0 (socket says 172.21.15.17:45650) 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b2052640 1 -- 172.21.15.17:0/1294993955 learned_addr learned my addr 172.21.15.17:0/1294993955 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b1851640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c5880 0x7f90ac1ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.623+0000 7f90b1050640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90ac0c88c0 0x7f90ac14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 -- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c5880 msgr2=0x7f90ac1ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c5880 0x7f90ac1ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 -- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90ac0c88c0 msgr2=0x7f90ac14bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90ac0c88c0 0x7f90ac14bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 -- 172.21.15.17:0/1294993955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac1eee60 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b2052640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c740 tx=0x7f90a400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4016020 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a400ce50 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4003970 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac1ef140 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.624+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac0c1370 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.625+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac153550 con 0x7f90ac14f360 2024-03-06T07:20:33.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.625+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90a4003b10 con 0x7f90ac14f360 2024-03-06T07:20:33.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.626+0000 7f90a27fc640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 0x7f909006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:33.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.626+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f90a408a000 con 0x7f90ac14f360 2024-03-06T07:20:33.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.626+0000 7f90b1851640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 0x7f909006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:33.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.626+0000 7f90b1851640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 0x7f909006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90a8036000 tx=0x7f90a803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:33.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.630+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90ac153550 con 0x7f90ac14f360 2024-03-06T07:20:33.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.851+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90ac04f330 con 0x7f90ac14f360 2024-03-06T07:20:33.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.851+0000 7f90a27fc640 1 -- 172.21.15.17:0/1294993955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f90a4020020 con 0x7f90ac14f360 2024-03-06T07:20:33.853 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:33.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 msgr2=0x7f909006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 0x7f909006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f90a8036000 tx=0x7f90a803c040 comp rx=0 tx=0).stop 2024-03-06T07:20:33.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 msgr2=0x7f90ac0bf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:33.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c740 tx=0x7f90a400cc00 comp rx=0 tx=0).stop 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 shutdown_connections 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f909006b4d0 0x7f909006d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90ac14f360 0x7f90ac0bf1e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90ac0c88c0 0x7f90ac14bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 --2- 172.21.15.17:0/1294993955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90ac0c5880 0x7f90ac1ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 >> 172.21.15.17:0/1294993955 conn(0x7f90ac0ac570 msgr2=0x7f90ac14cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 shutdown_connections 2024-03-06T07:20:33.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:33.853+0000 7f90b3ab5640 1 -- 172.21.15.17:0/1294993955 wait complete. 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:33.866 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-06T07:20:33.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.867 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-06T07:20:33.881 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:33.882 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42633sec preferred_lft 42633sec 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link noprefixroute 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:33.883 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:33.884 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:33.884 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-06T07:20:33.884 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:33.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.884 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:33.884 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:33.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:33.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:33.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-03-06T07:20:33.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:33.885 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:33.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:33.991 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:34.016 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-06T07:20:34.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.018 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-06T07:20:34.074 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-06T07:20:34.074 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:34.074 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-06T07:20:34.074 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-06T07:20:34.074 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:34.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.075 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:34.075 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:34.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-06T07:20:34.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-06T07:20:34.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-06T07:20:34.076 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:34.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.170 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-06T07:20:34.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.185 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-03-06T07:20:34.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.240 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:34.240 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:34.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-06T07:20:34.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-06T07:20:34.240 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:34.323 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-06T07:20:34.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-06T07:20:34.323 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:34.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.449 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec bash -c ' 2024-03-06T07:20:34.449 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-06T07:20:34.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-06T07:20:34.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:34.449 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-06T07:20:34.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:34.491 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:34.547 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-06T07:20:34.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:34.548 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:34.601 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-06T07:20:34.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.602 DEBUG:teuthology.orchestra.run.smithi017:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:34.685 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:34.686 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:34.794 INFO:teuthology.orchestra.run.smithi017.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:34.794 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: resolved to: "172.21.15.17:3300,172.21.15.163:3300,172.21.15.163:3301" 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: trying mount with new device syntax: 0@5fd8a41a-f84b-4e64-ac3e-5a25cd35578f.cephfs=/ 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.17:3300/172.21.15.163:3300/172.21.15.163:3301" will pass to kernel 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout: You just mounted a file system that supports labels which does not 2024-03-06T07:20:34.795 INFO:teuthology.orchestra.run.smithi017.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-06T07:20:34.796 INFO:teuthology.orchestra.run.smithi017.stdout: applications will generate AVC messages and not be allowed access to 2024-03-06T07:20:34.796 INFO:teuthology.orchestra.run.smithi017.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-06T07:20:34.797 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-06T07:20:34.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-06T07:20:34.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.824 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:34.881 INFO:teuthology.orchestra.run.smithi017.stdout:client4858 2024-03-06T07:20:34.881 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:34.881 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:34.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.937 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:34.993 INFO:teuthology.orchestra.run.smithi017.stdout:client4858 2024-03-06T07:20:34.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:34.994 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:35.049 INFO:teuthology.orchestra.run.smithi017.stdout:client4858 2024-03-06T07:20:35.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.049 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5fd8a41a-f84b-4e64-ac3e-5a25cd35578f.client4858/status') 2024-03-06T07:20:35.116 INFO:teuthology.orchestra.run.smithi017.stdout:instance: client.4858 (3)172.21.15.17:0/910523366 2024-03-06T07:20:35.116 INFO:teuthology.orchestra.run.smithi017.stdout:blocklisted: false 2024-03-06T07:20:35.116 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records in 2024-03-06T07:20:35.117 INFO:teuthology.orchestra.run.smithi017.stderr:0+1 records out 2024-03-06T07:20:35.117 INFO:teuthology.orchestra.run.smithi017.stderr:69 bytes copied, 2.1397e-05 s, 3.2 MB/s 2024-03-06T07:20:35.118 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-06T07:20:35.118 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-06T07:20:35.118 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:35.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-06T07:20:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.258+0000 7f3e25c8f640 1 Processor -- start 2024-03-06T07:20:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.258+0000 7f3e25c8f640 1 -- start start 2024-03-06T07:20:35.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.258+0000 7f3e25c8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e20152920 0x7f3e20155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e25c8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e200c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e25c8f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e20156180 con 0x7f3e200c8a00 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e25c8f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e201587b0 con 0x7f3e20152920 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e25c8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20159630 con 0x7f3e200c5330 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3e200c8a00 0x7f3e20158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35766/0 (socket says 172.21.15.17:35766) 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1effd640 1 -- 172.21.15.17:0/2969768708 learned_addr learned my addr 172.21.15.17:0/2969768708 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/2969768708 >> v2:172.21.15.163:3300/0 conn(0x7f3e20152920 0x7f3e20155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e200c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1e7fc640 1 -- 172.21.15.17:0/2969768708 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799992608 0 0) 0x7f3e10001110 con 0x7f3e200c8a00 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1e7fc640 1 -- 172.21.15.17:0/2969768708 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e08003520 con 0x7f3e200c8a00 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1f7fe640 1 -- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 msgr2=0x7f3e200c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e200c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1f7fe640 1 -- 172.21.15.17:0/2969768708 >> v1:172.21.15.17:6789/0 conn(0x7f3e200c8a00 legacy=0x7f3e20158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.259+0000 7f3e1f7fe640 1 -- 172.21.15.17:0/2969768708 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200c7760 con 0x7f3e20152920 2024-03-06T07:20:35.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20155c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e140097a0 tx=0x7f3e1402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dcc81d0838d010d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e1e7fc640 1 -- 172.21.15.17:0/2969768708 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140315a0 con 0x7f3e20152920 2024-03-06T07:20:35.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e1e7fc640 1 -- 172.21.15.17:0/2969768708 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14037070 con 0x7f3e20152920 2024-03-06T07:20:35.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e1e7fc640 1 -- 172.21.15.17:0/2969768708 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14031960 con 0x7f3e20152920 2024-03-06T07:20:35.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e25c8f640 1 -- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 msgr2=0x7f3e20155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.260+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20155c40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3e140097a0 tx=0x7f3e1402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 -- 172.21.15.17:0/2969768708 shutdown_connections 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e200c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/2969768708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20155c40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 -- 172.21.15.17:0/2969768708 >> 172.21.15.17:0/2969768708 conn(0x7f3e200ac560 msgr2=0x7f3e200ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 -- 172.21.15.17:0/2969768708 shutdown_connections 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.261+0000 7f3e25c8f640 1 -- 172.21.15.17:0/2969768708 wait complete. 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.262+0000 7f3e25c8f640 1 Processor -- start 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.262+0000 7f3e25c8f640 1 -- start start 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e2014b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20149380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20158a00 con 0x7f3e200c8a00 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c7d20 con 0x7f3e20152920 2024-03-06T07:20:35.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2006d030 con 0x7f3e200c5330 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45664/0 (socket says 172.21.15.17:45664) 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1effd640 1 -- 172.21.15.17:0/255524188 learned_addr learned my addr 172.21.15.17:0/255524188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e2014b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20149380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 -- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 msgr2=0x7f3e2014b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e2014b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 -- 172.21.15.17:0/255524188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 msgr2=0x7f3e20148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 -- 172.21.15.17:0/255524188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e201565b0 con 0x7f3e20152920 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1effd640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20149380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800eba0 tx=0x7f3e1800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18018020 con 0x7f3e20152920 2024-03-06T07:20:35.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1800cd20 con 0x7f3e20152920 2024-03-06T07:20:35.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e20145a90 con 0x7f3e20152920 2024-03-06T07:20:35.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180146c0 con 0x7f3e20152920 2024-03-06T07:20:35.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.263+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e20146b10 con 0x7f3e20152920 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.265+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e20152df0 con 0x7f3e20152920 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.265+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e18005d10 con 0x7f3e20152920 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.265+0000 7f3e07fff640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 0x7f3e0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.265+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3e1808aff0 con 0x7f3e20152920 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.265+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 0x7f3e0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:35.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.266+0000 7f3e1f7fe640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 0x7f3e0c06d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e14036000 tx=0x7f3e1403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:35.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.270+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3e20152df0 con 0x7f3e20152920 2024-03-06T07:20:35.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.490+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3e2004f330 con 0x7f3e20152920 2024-03-06T07:20:35.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.490+0000 7f3e07fff640 1 -- 172.21.15.17:0/255524188 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3e18022030 con 0x7f3e20152920 2024-03-06T07:20:35.492 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-06T07:20:35.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 msgr2=0x7f3e0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 0x7f3e0c06d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3e14036000 tx=0x7f3e1403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:35.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 msgr2=0x7f3e20149380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20149380 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800eba0 tx=0x7f3e1800c490 comp rx=0 tx=0).stop 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 shutdown_connections 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b500 0x7f3e0c06d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e20152920 0x7f3e20149380 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e200c8a00 0x7f3e20148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 --2- 172.21.15.17:0/255524188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e200c5330 0x7f3e2014b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.492+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 >> 172.21.15.17:0/255524188 conn(0x7f3e200ac560 msgr2=0x7f3e200caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.493+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 shutdown_connections 2024-03-06T07:20:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:35.493+0000 7f3e25c8f640 1 -- 172.21.15.17:0/255524188 wait complete. 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-06T07:20:35.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-06T07:20:35.506 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-06T07:20:35.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.506 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:35.522 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42658sec preferred_lft 42658sec 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2024-03-06T07:20:35.523 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2024-03-06T07:20:35.524 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-06T07:20:35.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2024-03-06T07:20:35.524 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-06T07:20:35.525 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-06T07:20:35.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.700 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-06T07:20:35.727 INFO:teuthology.orchestra.run.smithi163.stdout:1 2024-03-06T07:20:35.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.729 DEBUG:teuthology.orchestra.run.smithi163:> route 2024-03-06T07:20:35.823 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2024-03-06T07:20:35.824 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-06T07:20:35.824 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-06T07:20:35.824 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-06T07:20:35.824 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-06T07:20:35.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:35.824 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-06T07:20:35.824 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-06T07:20:35.824 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-06T07:20:35.825 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-06T07:20:35.825 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-06T07:20:35.825 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-06T07:20:36.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.027 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-06T07:20:36.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.043 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2024-03-06T07:20:36.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.098 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-06T07:20:36.098 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-06T07:20:36.098 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-06T07:20:36.098 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-06T07:20:36.098 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-06T07:20:36.184 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-06T07:20:36.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.184 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-06T07:20:36.185 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-06T07:20:36.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.291 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2024-03-06T07:20:36.291 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2024-03-06T07:20:36.291 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2024-03-06T07:20:36.292 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2024-03-06T07:20:36.292 DEBUG:teuthology.orchestra.run.smithi163:> ') 2024-03-06T07:20:36.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:36.335 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:36.390 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-06T07:20:36.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-06T07:20:36.390 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:36.446 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-06T07:20:36.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.446 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:36.532 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:36.532 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:36.671 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-06T07:20:36.671 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-06T07:20:36.671 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.17:3300,172.21.15.163:3300,172.21.15.163:3301" 2024-03-06T07:20:36.671 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with new device syntax: 1@5fd8a41a-f84b-4e64-ac3e-5a25cd35578f.cephfs=/ 2024-03-06T07:20:36.671 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.17:3300/172.21.15.163:3300/172.21.15.163:3301" will pass to kernel 2024-03-06T07:20:36.672 INFO:teuthology.orchestra.run.smithi163.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-06T07:20:36.672 INFO:teuthology.orchestra.run.smithi163.stdout: You just mounted a file system that supports labels which does not 2024-03-06T07:20:36.672 INFO:teuthology.orchestra.run.smithi163.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-06T07:20:36.672 INFO:teuthology.orchestra.run.smithi163.stdout: applications will generate AVC messages and not be allowed access to 2024-03-06T07:20:36.672 INFO:teuthology.orchestra.run.smithi163.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-06T07:20:36.674 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-06T07:20:36.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.674 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-06T07:20:36.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.702 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:36.759 INFO:teuthology.orchestra.run.smithi163.stdout:client4866 2024-03-06T07:20:36.759 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:20:36.760 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:36.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.816 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:36.873 INFO:teuthology.orchestra.run.smithi163.stdout:client4866 2024-03-06T07:20:36.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.874 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-06T07:20:36.932 INFO:teuthology.orchestra.run.smithi163.stdout:client4866 2024-03-06T07:20:36.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:36.932 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5fd8a41a-f84b-4e64-ac3e-5a25cd35578f.client4866/status') 2024-03-06T07:20:36.998 INFO:teuthology.orchestra.run.smithi163.stdout:instance: client.4866 (3)172.21.15.163:0/3975363408 2024-03-06T07:20:36.998 INFO:teuthology.orchestra.run.smithi163.stdout:blocklisted: false 2024-03-06T07:20:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records in 2024-03-06T07:20:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:0+1 records out 2024-03-06T07:20:36.998 INFO:teuthology.orchestra.run.smithi163.stderr:71 bytes copied, 2.3108e-05 s, 3.1 MB/s 2024-03-06T07:20:36.999 INFO:teuthology.run_tasks:Running task exec... 2024-03-06T07:20:37.010 INFO:teuthology.task.exec:Executing custom commands... 2024-03-06T07:20:37.010 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:20:37.010 DEBUG:teuthology.orchestra.run.smithi017:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2024-03-06T07:20:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 Processor -- start 2024-03-06T07:20:37.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 -- start start 2024-03-06T07:20:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 0x7fb6b00c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 0x7fb6b015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b0155270 con 0x7fb6b00c6610 2024-03-06T07:20:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b00c6c90 con 0x7fb6b00c8f10 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.123+0000 7fb6b6881640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0155530 con 0x7fb6b00c54a0 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6af7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb6b00c6610 0x7fb6b00c8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35770/0 (socket says 172.21.15.17:35770) 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6af7fe640 1 -- 172.21.15.17:0/58130597 learned_addr learned my addr 172.21.15.17:0/58130597 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6affff640 1 --2- 172.21.15.17:0/58130597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 0x7fb6b00c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6b4e1e640 1 --2- 172.21.15.17:0/58130597 >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 0x7fb6b015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6aeffd640 1 -- 172.21.15.17:0/58130597 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911983777 0 0) 0x7fb6b0155270 con 0x7fb6b00c6610 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6aeffd640 1 -- 172.21.15.17:0/58130597 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb698003520 con 0x7fb6b00c6610 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6affff640 1 -- 172.21.15.17:0/58130597 >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 msgr2=0x7fb6b015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6affff640 1 --2- 172.21.15.17:0/58130597 >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 0x7fb6b015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6affff640 1 -- 172.21.15.17:0/58130597 >> v1:172.21.15.17:6789/0 conn(0x7fb6b00c6610 legacy=0x7fb6b00c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6affff640 1 -- 172.21.15.17:0/58130597 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b0155810 con 0x7fb6b00c54a0 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.124+0000 7fb6b4e1e640 1 --2- 172.21.15.17:0/58130597 >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 0x7fb6b015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6affff640 1 --2- 172.21.15.17:0/58130597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 0x7fb6b00c5880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00097a0 tx=0x7fb6a002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c4c12dcfbddc72 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6aeffd640 1 -- 172.21.15.17:0/58130597 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0038070 con 0x7fb6b00c54a0 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6aeffd640 1 -- 172.21.15.17:0/58130597 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0031700 con 0x7fb6b00c54a0 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6aeffd640 1 -- 172.21.15.17:0/58130597 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0031ab0 con 0x7fb6b00c54a0 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6b6881640 1 -- 172.21.15.17:0/58130597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 msgr2=0x7fb6b00c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.125+0000 7fb6b6881640 1 --2- 172.21.15.17:0/58130597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 0x7fb6b00c5880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00097a0 tx=0x7fb6a002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 -- 172.21.15.17:0/58130597 shutdown_connections 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 --2- 172.21.15.17:0/58130597 >> v2:172.21.15.163:3300/0 conn(0x7fb6b00c8f10 0x7fb6b015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 --2- 172.21.15.17:0/58130597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b00c54a0 0x7fb6b00c5880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 -- 172.21.15.17:0/58130597 >> 172.21.15.17:0/58130597 conn(0x7fb6b00ca900 msgr2=0x7fb6b00cad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 -- 172.21.15.17:0/58130597 shutdown_connections 2024-03-06T07:20:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.126+0000 7fb6b6881640 1 -- 172.21.15.17:0/58130597 wait complete. 2024-03-06T07:20:37.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.127+0000 7fb6b6881640 1 Processor -- start 2024-03-06T07:20:37.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.127+0000 7fb6b6881640 1 -- start start 2024-03-06T07:20:37.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 0x7fb6b014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b01e0210 0x7fb6b01e0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00c60d0 con 0x7fb6b00c6610 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0152be0 con 0x7fb6b00c54a0 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b00c6250 con 0x7fb6b01e0210 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6af7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45678/0 (socket says 172.21.15.17:45678) 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b4e1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b01e0210 0x7fb6b01e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 0x7fb6b014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6af7fe640 1 -- 172.21.15.17:0/1497972974 learned_addr learned my addr 172.21.15.17:0/1497972974 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 -- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b01e0210 msgr2=0x7fb6b01e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b01e0210 0x7fb6b01e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 -- 172.21.15.17:0/1497972974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 msgr2=0x7fb6b00c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 -- 172.21.15.17:0/1497972974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b01eede0 con 0x7fb6b00c54a0 2024-03-06T07:20:37.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6af7fe640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6affff640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 0x7fb6b014f4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6b01556b0 tx=0x7fb6a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0042030 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a003d670 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.128+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b0159220 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.129+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0038040 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.129+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b00c0940 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.130+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b004f330 con 0x7fb6b00c54a0 2024-03-06T07:20:37.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.130+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6a0045070 con 0x7fb6b00c54a0 2024-03-06T07:20:37.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.130+0000 7fb6acff9640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 0x7fb69c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.130+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb6a00abaf0 con 0x7fb6b00c54a0 2024-03-06T07:20:37.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.131+0000 7fb6af7fe640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 0x7fb69c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.131+0000 7fb6af7fe640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 0x7fb69c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00bda00 tx=0x7fb6a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.139+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6a0030070 con 0x7fb6b00c54a0 2024-03-06T07:20:37.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.319+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) v1 -- 0x7fb6b0155270 con 0x7fb6b00c54a0 2024-03-06T07:20:37.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.323+0000 7fb6acff9640 1 -- 172.21.15.17:0/1497972974 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) v1 ==== 128+0+0 (secure 0 0 0) 0x7fb6a007a7e0 con 0x7fb6b00c54a0 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 msgr2=0x7fb69c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 0x7fb69c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00bda00 tx=0x7fb6a4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 msgr2=0x7fb6b014f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 0x7fb6b014f4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb6b01556b0 tx=0x7fb6a00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 shutdown_connections 2024-03-06T07:20:37.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb69c06b4d0 0x7fb69c06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6b01e0210 0x7fb6b01e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6b00c6610 0x7fb6b00c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 --2- 172.21.15.17:0/1497972974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6b00c54a0 0x7fb6b014f4c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 >> 172.21.15.17:0/1497972974 conn(0x7fb6b00ca900 msgr2=0x7fb6b00be430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 shutdown_connections 2024-03-06T07:20:37.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.325+0000 7fb6b6881640 1 -- 172.21.15.17:0/1497972974 wait complete. 2024-03-06T07:20:37.337 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-06T07:20:37.354 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:37.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.453+0000 7fb04e51e640 1 Processor -- start 2024-03-06T07:20:37.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.453+0000 7fb04e51e640 1 -- start start 2024-03-06T07:20:37.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04e51e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0480c8a00 0x7fb048158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04e51e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb048159460 0x7fb0480c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04e51e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0480c7760 con 0x7fb048152920 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04e51e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0481587b0 con 0x7fb0480c8a00 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04e51e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480c7a40 con 0x7fb048159460 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0480c8a00 0x7fb048158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0480c8a00 0x7fb048158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45684/0 (socket says 172.21.15.17:45684) 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 -- 172.21.15.17:0/3259882345 learned_addr learned my addr 172.21.15.17:0/3259882345 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb04cabb640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb048159460 0x7fb0480c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 -- 172.21.15.17:0/3259882345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb048159460 msgr2=0x7fb0480c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:37.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb048159460 0x7fb0480c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 -- 172.21.15.17:0/3259882345 >> v1:172.21.15.163:6789/0 conn(0x7fb048152920 legacy=0x7fb048155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.454+0000 7fb0477fe640 1 -- 172.21.15.17:0/3259882345 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480c7d20 con 0x7fb0480c8a00 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb0477fe640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048158f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb030009870 tx=0x7fb03002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5c9c3d40c5c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb046ffd640 1 -- 172.21.15.17:0/3259882345 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030035020 con 0x7fb0480c8a00 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb046ffd640 1 -- 172.21.15.17:0/3259882345 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030031640 con 0x7fb0480c8a00 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb046ffd640 1 -- 172.21.15.17:0/3259882345 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03002d040 con 0x7fb0480c8a00 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb04e51e640 1 -- 172.21.15.17:0/3259882345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 msgr2=0x7fb048158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.455+0000 7fb04e51e640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048158f20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb030009870 tx=0x7fb03002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 -- 172.21.15.17:0/3259882345 shutdown_connections 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb048159460 0x7fb0480c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 --2- 172.21.15.17:0/3259882345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048158f20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 -- 172.21.15.17:0/3259882345 >> 172.21.15.17:0/3259882345 conn(0x7fb0480ac560 msgr2=0x7fb0480ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 -- 172.21.15.17:0/3259882345 shutdown_connections 2024-03-06T07:20:37.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.456+0000 7fb04e51e640 1 -- 172.21.15.17:0/3259882345 wait complete. 2024-03-06T07:20:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 Processor -- start 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 -- start start 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb048152920 0x7fb0480c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0481eea70 0x7fb048149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480c7390 con 0x7fb0480c8a00 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04806d030 con 0x7fb048152920 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.457+0000 7fb04e51e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480c7510 con 0x7fb0481eea70 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45698/0 (socket says 172.21.15.17:45698) 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 -- 172.21.15.17:0/71191136 learned_addr learned my addr 172.21.15.17:0/71191136 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb04cabb640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0481eea70 0x7fb048149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb0477fe640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb048152920 0x7fb0480c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 -- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0481eea70 msgr2=0x7fb048149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0481eea70 0x7fb048149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 -- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb048152920 msgr2=0x7fb0480c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb048152920 0x7fb0480c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 -- 172.21.15.17:0/71191136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480c2860 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb047fff640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c400 tx=0x7fb03800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0380052d0 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038005490 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0480be610 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038007260 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.458+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0480c2340 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.459+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04804f330 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.460+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb038007400 con 0x7fb0480c8a00 2024-03-06T07:20:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.460+0000 7fb044ff9640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.460+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb038089950 con 0x7fb0480c8a00 2024-03-06T07:20:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.460+0000 7fb0477fe640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.461+0000 7fb0477fe640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 0x7fb02806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb030036000 tx=0x7fb03003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.464+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb038057ed0 con 0x7fb0480c8a00 2024-03-06T07:20:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.646+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb04806cda0 con 0x7fb0480c8a00 2024-03-06T07:20:37.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.647+0000 7fb044ff9640 1 -- 172.21.15.17:0/71191136 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fb03805bf70 con 0x7fb0480c8a00 2024-03-06T07:20:37.650 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:37.650 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 msgr2=0x7fb02806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 0x7fb02806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb030036000 tx=0x7fb03003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 msgr2=0x7fb048147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c400 tx=0x7fb03800c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 shutdown_connections 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb02806b4d0 0x7fb02806d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0481eea70 0x7fb048149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb048152920 0x7fb0480c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 --2- 172.21.15.17:0/71191136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0480c8a00 0x7fb048147d70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 >> 172.21.15.17:0/71191136 conn(0x7fb0480ac560 msgr2=0x7fb0480c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 shutdown_connections 2024-03-06T07:20:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.650+0000 7fb04e51e640 1 -- 172.21.15.17:0/71191136 wait complete. 2024-03-06T07:20:37.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:37.664 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:37.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.766+0000 7faf37381640 1 Processor -- start 2024-03-06T07:20:37.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 -- start start 2024-03-06T07:20:37.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf300b0070 0x7faf30155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300c5380 0x7faf300c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf300b0450 con 0x7faf3006ca70 2024-03-06T07:20:37.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3006c020 con 0x7faf300b0070 2024-03-06T07:20:37.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.767+0000 7faf37381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3006cf20 con 0x7faf300c5380 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3511d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf300b0070 0x7faf30155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3491c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faf3006ca70 0x7faf300c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35782/0 (socket says 172.21.15.17:35782) 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3491c640 1 -- 172.21.15.17:0/1517748344 learned_addr learned my addr 172.21.15.17:0/1517748344 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3591e640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300c5380 0x7faf300c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3511d640 1 -- 172.21.15.17:0/1517748344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300c5380 msgr2=0x7faf300c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3511d640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300c5380 0x7faf300c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3511d640 1 -- 172.21.15.17:0/1517748344 >> v1:172.21.15.17:6789/0 conn(0x7faf3006ca70 legacy=0x7faf300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.768+0000 7faf3511d640 1 -- 172.21.15.17:0/1517748344 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300c77f0 con 0x7faf300b0070 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf3511d640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300b0070 0x7faf30155c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097a0 tx=0x7faf2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cb4bf9caa8b180 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf27fff640 1 -- 172.21.15.17:0/1517748344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0315a0 con 0x7faf300b0070 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf27fff640 1 -- 172.21.15.17:0/1517748344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c037070 con 0x7faf300b0070 2024-03-06T07:20:37.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf27fff640 1 -- 172.21.15.17:0/1517748344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c031960 con 0x7faf300b0070 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 -- 172.21.15.17:0/1517748344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300b0070 msgr2=0x7faf30155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300b0070 0x7faf30155c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097a0 tx=0x7faf2c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 -- 172.21.15.17:0/1517748344 shutdown_connections 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300c5380 0x7faf300c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 --2- 172.21.15.17:0/1517748344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300b0070 0x7faf30155c30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 -- 172.21.15.17:0/1517748344 >> 172.21.15.17:0/1517748344 conn(0x7faf300ac590 msgr2=0x7faf300ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 -- 172.21.15.17:0/1517748344 shutdown_connections 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.769+0000 7faf37381640 1 -- 172.21.15.17:0/1517748344 wait complete. 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.770+0000 7faf37381640 1 Processor -- start 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.770+0000 7faf37381640 1 -- start start 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 0x7faf301ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 0x7faf3014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3006c550 con 0x7faf300c5380 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3006c270 con 0x7faf3006ca70 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf37381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3006da80 con 0x7faf300b0070 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45720/0 (socket says 172.21.15.17:45720) 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 -- 172.21.15.17:0/1982223843 learned_addr learned my addr 172.21.15.17:0/1982223843 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:37.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3511d640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 0x7faf301ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3491c640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 0x7faf3014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 -- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 msgr2=0x7faf3014ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 0x7faf3014ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 -- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 msgr2=0x7faf301ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 0x7faf301ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.771+0000 7faf3591e640 1 -- 172.21.15.17:0/1982223843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300addc0 con 0x7faf300c5380 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf3491c640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 0x7faf3014ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf3511d640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 0x7faf301ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf3591e640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28018020 con 0x7faf300c5380 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28014480 con 0x7faf300c5380 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf301546c0 con 0x7faf300c5380 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28014870 con 0x7faf300c5380 2024-03-06T07:20:37.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.772+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf30153900 con 0x7faf300c5380 2024-03-06T07:20:37.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.773+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf300678a0 con 0x7faf300c5380 2024-03-06T07:20:37.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.774+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf28005d10 con 0x7faf300c5380 2024-03-06T07:20:37.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.774+0000 7faf25ffb640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 0x7faf1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:37.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.774+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faf2808b2f0 con 0x7faf300c5380 2024-03-06T07:20:37.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.774+0000 7faf3511d640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 0x7faf1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:37.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.775+0000 7faf3511d640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 0x7faf1406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf2c036000 tx=0x7faf2c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:37.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.778+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf300678a0 con 0x7faf300c5380 2024-03-06T07:20:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.958+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf300ad6f0 con 0x7faf300c5380 2024-03-06T07:20:37.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.958+0000 7faf25ffb640 1 -- 172.21.15.17:0/1982223843 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf300ad6f0 con 0x7faf300c5380 2024-03-06T07:20:37.960 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 msgr2=0x7faf1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 0x7faf1406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf2c036000 tx=0x7faf2c03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 msgr2=0x7faf3014bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faf2800efc0 tx=0x7faf2800c490 comp rx=0 tx=0).stop 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 shutdown_connections 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf1406b4d0 0x7faf1406d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf300c5380 0x7faf3014bf40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf300b0070 0x7faf3014ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 --2- 172.21.15.17:0/1982223843 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3006ca70 0x7faf301ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:37.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 >> 172.21.15.17:0/1982223843 conn(0x7faf300ac590 msgr2=0x7faf300c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:37.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 shutdown_connections 2024-03-06T07:20:37.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:37.960+0000 7faf37381640 1 -- 172.21.15.17:0/1982223843 wait complete. 2024-03-06T07:20:37.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:38.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 Processor -- start 2024-03-06T07:20:38.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 -- start start 2024-03-06T07:20:38.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f410c1527c0 0x7f410c0c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f410c155e10 0x7f410c06c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c152ba0 con 0x7f410c0c88c0 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c0c9390 con 0x7f410c155e10 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.076+0000 7f4111c15640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c0c8dd0 con 0x7f410c1527c0 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f410c155e10 0x7f410c06c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f410c0c88c0 0x7f410c158710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35792/0 (socket says 172.21.15.17:35792) 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410bfff640 1 -- 172.21.15.17:0/3362899455 learned_addr learned my addr 172.21.15.17:0/3362899455 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410b7fe640 1 --2- 172.21.15.17:0/3362899455 >> v2:172.21.15.163:3300/0 conn(0x7f410c1527c0 0x7f410c0c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410affd640 1 -- 172.21.15.17:0/3362899455 >> v2:172.21.15.163:3300/0 conn(0x7f410c1527c0 msgr2=0x7f410c0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410affd640 1 --2- 172.21.15.17:0/3362899455 >> v2:172.21.15.163:3300/0 conn(0x7f410c1527c0 0x7f410c0c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410affd640 1 -- 172.21.15.17:0/3362899455 >> v1:172.21.15.17:6789/0 conn(0x7f410c0c88c0 legacy=0x7f410c158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410affd640 1 -- 172.21.15.17:0/3362899455 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0c90b0 con 0x7f410c155e10 2024-03-06T07:20:38.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.077+0000 7f410a7fc640 1 -- 172.21.15.17:0/3362899455 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203575145 0 0) 0x7f4104001490 con 0x7f410c0c88c0 2024-03-06T07:20:38.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f410affd640 1 --2- 172.21.15.17:0/3362899455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c06c020 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f410c0cba50 tx=0x7f410002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc329cec8cf1d438 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f410a7fc640 1 -- 172.21.15.17:0/3362899455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41000315a0 con 0x7f410c155e10 2024-03-06T07:20:38.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f410a7fc640 1 -- 172.21.15.17:0/3362899455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4100037070 con 0x7f410c155e10 2024-03-06T07:20:38.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f410a7fc640 1 -- 172.21.15.17:0/3362899455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100031960 con 0x7f410c155e10 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 -- 172.21.15.17:0/3362899455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 msgr2=0x7f410c06c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 --2- 172.21.15.17:0/3362899455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c06c020 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f410c0cba50 tx=0x7f410002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 -- 172.21.15.17:0/3362899455 shutdown_connections 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 --2- 172.21.15.17:0/3362899455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c06c020 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 --2- 172.21.15.17:0/3362899455 >> v2:172.21.15.163:3300/0 conn(0x7f410c1527c0 0x7f410c0c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 -- 172.21.15.17:0/3362899455 >> 172.21.15.17:0/3362899455 conn(0x7f410c0ac550 msgr2=0x7f410c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.078+0000 7f4111c15640 1 -- 172.21.15.17:0/3362899455 shutdown_connections 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.079+0000 7f4111c15640 1 -- 172.21.15.17:0/3362899455 wait complete. 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 Processor -- start 2024-03-06T07:20:38.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 -- start start 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c88c0 0x7f410c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c1527c0 0x7f410c154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06d590 con 0x7f410c155e10 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c155610 con 0x7f410c1527c0 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.080+0000 7f4111c15640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c06da10 con 0x7f410c0c88c0 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45740/0 (socket says 172.21.15.17:45740) 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 -- 172.21.15.17:0/4017290817 learned_addr learned my addr 172.21.15.17:0/4017290817 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410b7fe640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c88c0 0x7f410c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410affd640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c1527c0 0x7f410c154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 -- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c88c0 msgr2=0x7f410c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c88c0 0x7f410c158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 -- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c1527c0 msgr2=0x7f410c154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c1527c0 0x7f410c154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 -- 172.21.15.17:0/4017290817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c1eec10 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f410bfff640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f410400c820 tx=0x7f410400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104018070 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410400a5e0 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c0c3e60 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410400a9d0 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.081+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c154fe0 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.082+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c04f330 con 0x7f410c155e10 2024-03-06T07:20:38.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.083+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f410401b070 con 0x7f410c155e10 2024-03-06T07:20:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.083+0000 7f40f3fff640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 0x7f40f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.083+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f41040899e0 con 0x7f410c155e10 2024-03-06T07:20:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.083+0000 7f410b7fe640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 0x7f40f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.084+0000 7f410b7fe640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 0x7f40f806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009b40 tx=0x7f40fc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.087+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f410c152ba0 con 0x7f410c155e10 2024-03-06T07:20:38.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:38.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:20:38.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.270+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f410c06d7f0 con 0x7f410c155e10 2024-03-06T07:20:38.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.271+0000 7f40f3fff640 1 -- 172.21.15.17:0/4017290817 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4104057f60 con 0x7f410c155e10 2024-03-06T07:20:38.272 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 msgr2=0x7f40f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 0x7f40f806d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009b40 tx=0x7f40fc009190 comp rx=0 tx=0).stop 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 msgr2=0x7f410c148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f410400c820 tx=0x7f410400cce0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 shutdown_connections 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40f806b4d0 0x7f40f806d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c155e10 0x7f410c148d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c1527c0 0x7f410c154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 --2- 172.21.15.17:0/4017290817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c88c0 0x7f410c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 >> 172.21.15.17:0/4017290817 conn(0x7f410c0ac550 msgr2=0x7f410c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 shutdown_connections 2024-03-06T07:20:38.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.273+0000 7f4111c15640 1 -- 172.21.15.17:0/4017290817 wait complete. 2024-03-06T07:20:38.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.428+0000 7f47386e1640 1 Processor -- start 2024-03-06T07:20:38.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.428+0000 7f47386e1640 1 -- start start 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f47386e1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47301529f0 0x7f47300c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f47386e1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47300c5370 0x7f47300c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f47386e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4730152dd0 con 0x7f4730155530 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f47386e1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4730159710 con 0x7f47300c5370 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f47386e1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300c68b0 con 0x7f47301529f0 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47300c5370 0x7f47300c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4736c7e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4730155530 0x7f47301559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35808/0 (socket says 172.21.15.17:35808) 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47300c5370 0x7f47300c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45748/0 (socket says 172.21.15.17:45748) 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4736c7e640 1 -- 172.21.15.17:0/2135544898 learned_addr learned my addr 172.21.15.17:0/2135544898 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f473647d640 1 --2- 172.21.15.17:0/2135544898 >> v2:172.21.15.163:3300/0 conn(0x7f47301529f0 0x7f47300c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 -- 172.21.15.17:0/2135544898 >> v2:172.21.15.163:3300/0 conn(0x7f47301529f0 msgr2=0x7f47300c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 --2- 172.21.15.17:0/2135544898 >> v2:172.21.15.163:3300/0 conn(0x7f47301529f0 0x7f47300c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 -- 172.21.15.17:0/2135544898 >> v1:172.21.15.17:6789/0 conn(0x7f4730155530 legacy=0x7f47301559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f4735c7c640 1 -- 172.21.15.17:0/2135544898 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730158b00 con 0x7f47300c5370 2024-03-06T07:20:38.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.429+0000 7f473547b640 1 -- 172.21.15.17:0/2135544898 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289634728 0 0) 0x7f4730152dd0 con 0x7f4730155530 2024-03-06T07:20:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.430+0000 7f4735c7c640 1 --2- 172.21.15.17:0/2135544898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f47300c57d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47300cba90 tx=0x7f472402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ac8b586731f8a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.430+0000 7f473547b640 1 -- 172.21.15.17:0/2135544898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240315a0 con 0x7f47300c5370 2024-03-06T07:20:38.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.430+0000 7f473547b640 1 -- 172.21.15.17:0/2135544898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4724037070 con 0x7f47300c5370 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.430+0000 7f473547b640 1 -- 172.21.15.17:0/2135544898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724031960 con 0x7f47300c5370 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 -- 172.21.15.17:0/2135544898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 msgr2=0x7f47300c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 --2- 172.21.15.17:0/2135544898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f47300c57d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f47300cba90 tx=0x7f472402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 -- 172.21.15.17:0/2135544898 shutdown_connections 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 --2- 172.21.15.17:0/2135544898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f47300c57d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 --2- 172.21.15.17:0/2135544898 >> v2:172.21.15.163:3300/0 conn(0x7f47301529f0 0x7f47300c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 -- 172.21.15.17:0/2135544898 >> 172.21.15.17:0/2135544898 conn(0x7f47300ac580 msgr2=0x7f47300ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 -- 172.21.15.17:0/2135544898 shutdown_connections 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.431+0000 7f47386e1640 1 -- 172.21.15.17:0/2135544898 wait complete. 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.432+0000 7f47386e1640 1 Processor -- start 2024-03-06T07:20:38.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.432+0000 7f47386e1640 1 -- start start 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 0x7f473014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4730155530 0x7f473014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301593a0 con 0x7f47300c5370 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47301590c0 con 0x7f47301529f0 2024-03-06T07:20:38.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47301560d0 con 0x7f4730155530 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45762/0 (socket says 172.21.15.17:45762) 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 -- 172.21.15.17:0/3110487053 learned_addr learned my addr 172.21.15.17:0/3110487053 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f4736c7e640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4730155530 0x7f473014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f4735c7c640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 0x7f473014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 -- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4730155530 msgr2=0x7f473014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4730155530 0x7f473014be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 -- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 msgr2=0x7f473014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 0x7f473014b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 -- 172.21.15.17:0/3110487053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4730148f30 con 0x7f47300c5370 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f4735c7c640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 0x7f473014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f473647d640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9b0 tx=0x7f472800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4728017070 con 0x7f47300c5370 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472800c9a0 con 0x7f47300c5370 2024-03-06T07:20:38.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800cd90 con 0x7f47300c5370 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473014c3c0 con 0x7f47300c5370 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.433+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4730154c40 con 0x7f47300c5370 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.434+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4730149730 con 0x7f47300c5370 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.435+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f472801a070 con 0x7f47300c5370 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.435+0000 7f472effd640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 0x7f471c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.435+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f472808a7b0 con 0x7f47300c5370 2024-03-06T07:20:38.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.435+0000 7f4735c7c640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 0x7f471c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.436+0000 7f4735c7c640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 0x7f471c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4724036000 tx=0x7f472403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.439+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4728058d30 con 0x7f47300c5370 2024-03-06T07:20:38.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.622+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f473004f330 con 0x7f47300c5370 2024-03-06T07:20:38.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.622+0000 7f472effd640 1 -- 172.21.15.17:0/3110487053 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f472805cdd0 con 0x7f47300c5370 2024-03-06T07:20:38.624 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 msgr2=0x7f471c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 0x7f471c06d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4724036000 tx=0x7f472403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 msgr2=0x7f4730154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f472800e9b0 tx=0x7f472800ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 shutdown_connections 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471c06b4d0 0x7f471c06d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4730155530 0x7f473014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47301529f0 0x7f473014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 --2- 172.21.15.17:0/3110487053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47300c5370 0x7f4730154700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 >> 172.21.15.17:0/3110487053 conn(0x7f47300ac580 msgr2=0x7f47301559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 shutdown_connections 2024-03-06T07:20:38.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.624+0000 7f47386e1640 1 -- 172.21.15.17:0/3110487053 wait complete. 2024-03-06T07:20:38.637 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:38.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.739+0000 7efc26264640 1 Processor -- start 2024-03-06T07:20:38.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.739+0000 7efc26264640 1 -- start start 2024-03-06T07:20:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc26264640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc201529f0 0x7efc200c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc26264640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 0x7efc200c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc26264640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc20152dd0 con 0x7efc20155530 2024-03-06T07:20:38.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc26264640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc20159710 con 0x7efc201529f0 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc26264640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200c68b0 con 0x7efc200c5370 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc201529f0 0x7efc200c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc201529f0 0x7efc200c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45770/0 (socket says 172.21.15.17:45770) 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 -- 172.21.15.17:0/364473917 learned_addr learned my addr 172.21.15.17:0/364473917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1f7fe640 1 --2- 172.21.15.17:0/364473917 >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 0x7efc200c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 -- 172.21.15.17:0/364473917 >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 msgr2=0x7efc200c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 --2- 172.21.15.17:0/364473917 >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 0x7efc200c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 -- 172.21.15.17:0/364473917 >> v1:172.21.15.163:6789/0 conn(0x7efc20155530 legacy=0x7efc201559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1ffff640 1 -- 172.21.15.17:0/364473917 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20158b00 con 0x7efc201529f0 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1effd640 1 -- 172.21.15.17:0/364473917 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193812315 0 0) 0x7efc20152dd0 con 0x7efc20155530 2024-03-06T07:20:38.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.740+0000 7efc1f7fe640 1 --2- 172.21.15.17:0/364473917 >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 0x7efc200c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.741+0000 7efc1ffff640 1 --2- 172.21.15.17:0/364473917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc201529f0 0x7efc200c88f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc2008c9b0 tx=0x7efc1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a3b9ed0b795076 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.741+0000 7efc1effd640 1 -- 172.21.15.17:0/364473917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10035020 con 0x7efc201529f0 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.741+0000 7efc1effd640 1 -- 172.21.15.17:0/364473917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10031870 con 0x7efc201529f0 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.741+0000 7efc1effd640 1 -- 172.21.15.17:0/364473917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10031c40 con 0x7efc201529f0 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 -- 172.21.15.17:0/364473917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc201529f0 msgr2=0x7efc200c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 --2- 172.21.15.17:0/364473917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc201529f0 0x7efc200c88f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc2008c9b0 tx=0x7efc1002b910 comp rx=0 tx=0).stop 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 -- 172.21.15.17:0/364473917 shutdown_connections 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 --2- 172.21.15.17:0/364473917 >> v2:172.21.15.163:3300/0 conn(0x7efc200c5370 0x7efc200c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 --2- 172.21.15.17:0/364473917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc201529f0 0x7efc200c88f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 -- 172.21.15.17:0/364473917 >> 172.21.15.17:0/364473917 conn(0x7efc200ac580 msgr2=0x7efc200ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 -- 172.21.15.17:0/364473917 shutdown_connections 2024-03-06T07:20:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.742+0000 7efc26264640 1 -- 172.21.15.17:0/364473917 wait complete. 2024-03-06T07:20:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.743+0000 7efc26264640 1 Processor -- start 2024-03-06T07:20:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.743+0000 7efc26264640 1 -- start start 2024-03-06T07:20:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201529f0 0x7efc2014be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc20155530 0x7efc2014c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201593a0 con 0x7efc200c5370 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc201590c0 con 0x7efc201529f0 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc26264640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc201560d0 con 0x7efc20155530 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45782/0 (socket says 172.21.15.17:45782) 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 -- 172.21.15.17:0/3717005312 learned_addr learned my addr 172.21.15.17:0/3717005312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1f7fe640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201529f0 0x7efc2014be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc24801640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc20155530 0x7efc2014c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 -- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc20155530 msgr2=0x7efc2014c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc20155530 0x7efc2014c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 -- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201529f0 msgr2=0x7efc2014be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201529f0 0x7efc2014be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 -- 172.21.15.17:0/3717005312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20157b40 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1ffff640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc100090b0 tx=0x7efc100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.744+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10041070 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.745+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1003ad00 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.745+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc20157e20 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.745+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10035040 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.745+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc201547c0 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.746+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2004f330 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.746+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc10038020 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.746+0000 7efc1cff9640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 0x7efc0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.746+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efc100ac170 con 0x7efc200c5370 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.746+0000 7efc1f7fe640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 0x7efc0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:38.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.747+0000 7efc1f7fe640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 0x7efc0006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc08009560 tx=0x7efc08009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.751+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc1007a6f0 con 0x7efc200c5370 2024-03-06T07:20:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.933+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efc20157ff0 con 0x7efc200c5370 2024-03-06T07:20:38.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.933+0000 7efc1cff9640 1 -- 172.21.15.17:0/3717005312 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7efc1007e790 con 0x7efc200c5370 2024-03-06T07:20:38.935 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:38.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 msgr2=0x7efc0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 0x7efc0006d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efc08009560 tx=0x7efc08009190 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 msgr2=0x7efc20154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc100090b0 tx=0x7efc100046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 shutdown_connections 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0006b4d0 0x7efc0006d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc20155530 0x7efc2014c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201529f0 0x7efc2014be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 --2- 172.21.15.17:0/3717005312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc200c5370 0x7efc20154280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.935+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 >> 172.21.15.17:0/3717005312 conn(0x7efc200ac580 msgr2=0x7efc20150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.936+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 shutdown_connections 2024-03-06T07:20:38.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:38.936+0000 7efc26264640 1 -- 172.21.15.17:0/3717005312 wait complete. 2024-03-06T07:20:38.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.048+0000 7eff13a0f640 1 Processor -- start 2024-03-06T07:20:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.048+0000 7eff13a0f640 1 -- start start 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff13a0f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff0c152920 0x7eff0c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff13a0f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c156180 con 0x7eff0c0c8a00 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff13a0f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c159630 con 0x7eff0c0c5330 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff13a0f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c1587b0 con 0x7eff0c152920 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff10faa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7eff0c0c8a00 0x7eff0c158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35810/0 (socket says 172.21.15.17:35810) 2024-03-06T07:20:39.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff10faa640 1 -- 172.21.15.17:0/357340325 learned_addr learned my addr 172.21.15.17:0/357340325 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff117ab640 1 --2- 172.21.15.17:0/357340325 >> v2:172.21.15.163:3300/0 conn(0x7eff0c152920 0x7eff0c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229942976 0 0) 0x7efefc001110 con 0x7eff0c0c8a00 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef8003520 con 0x7eff0c0c8a00 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756860149 0 0) 0x7eff0c159630 con 0x7eff0c0c5330 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efefc001110 con 0x7eff0c0c5330 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff117ab640 1 -- 172.21.15.17:0/357340325 >> v1:172.21.15.163:6789/0 conn(0x7eff0c0c5330 legacy=0x7eff0c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff117ab640 1 -- 172.21.15.17:0/357340325 >> v1:172.21.15.17:6789/0 conn(0x7eff0c0c8a00 legacy=0x7eff0c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.049+0000 7eff117ab640 1 -- 172.21.15.17:0/357340325 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0c7760 con 0x7eff0c152920 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.050+0000 7eff117ab640 1 --2- 172.21.15.17:0/357340325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c155c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff080097a0 tx=0x7eff0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10b2ecf86330cfc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.050+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08035020 con 0x7eff0c152920 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.050+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08031830 con 0x7eff0c152920 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.050+0000 7eff03fff640 1 -- 172.21.15.17:0/357340325 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08031c00 con 0x7eff0c152920 2024-03-06T07:20:39.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 -- 172.21.15.17:0/357340325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 msgr2=0x7eff0c155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 --2- 172.21.15.17:0/357340325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c155c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7eff080097a0 tx=0x7eff0802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 -- 172.21.15.17:0/357340325 shutdown_connections 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 --2- 172.21.15.17:0/357340325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c155c40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 -- 172.21.15.17:0/357340325 >> 172.21.15.17:0/357340325 conn(0x7eff0c0ac560 msgr2=0x7eff0c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 -- 172.21.15.17:0/357340325 shutdown_connections 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.051+0000 7eff13a0f640 1 -- 172.21.15.17:0/357340325 wait complete. 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.052+0000 7eff13a0f640 1 Processor -- start 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.052+0000 7eff13a0f640 1 -- start start 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c0c8a00 0x7eff0c1eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c158a00 con 0x7eff0c0c5330 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0c7d20 con 0x7eff0c152920 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff13a0f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c06d030 con 0x7eff0c0c8a00 2024-03-06T07:20:39.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45792/0 (socket says 172.21.15.17:45792) 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 -- 172.21.15.17:0/4066673127 learned_addr learned my addr 172.21.15.17:0/4066673127 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff11fac640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff10faa640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c0c8a00 0x7eff0c1eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 -- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c0c8a00 msgr2=0x7eff0c1eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c0c8a00 0x7eff0c1eeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 -- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 msgr2=0x7eff0c153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 -- 172.21.15.17:0/4066673127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0c49f0 con 0x7eff0c0c5330 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff117ab640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff0802bee0 tx=0x7eff080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.053+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08035040 con 0x7eff0c0c5330 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.054+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0803b920 con 0x7eff0c0c5330 2024-03-06T07:20:39.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.054+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0803bd10 con 0x7eff0c0c5330 2024-03-06T07:20:39.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.054+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c0c4430 con 0x7eff0c0c5330 2024-03-06T07:20:39.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.054+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c1eb030 con 0x7eff0c0c5330 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.055+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c067910 con 0x7eff0c0c5330 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.055+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff08038020 con 0x7eff0c0c5330 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.056+0000 7eff01ffb640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 0x7efef006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.056+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7eff080ac410 con 0x7eff0c0c5330 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.056+0000 7eff10faa640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 0x7efef006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.056+0000 7eff10faa640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 0x7efef006d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cb980 tx=0x7efefc007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.060+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff0c067910 con 0x7eff0c0c5330 2024-03-06T07:20:39.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.241+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff0c04f330 con 0x7eff0c0c5330 2024-03-06T07:20:39.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.242+0000 7eff01ffb640 1 -- 172.21.15.17:0/4066673127 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7eff0807a990 con 0x7eff0c0c5330 2024-03-06T07:20:39.244 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:39.244 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 msgr2=0x7efef006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 0x7efef006d9c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cb980 tx=0x7efefc007d30 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 msgr2=0x7eff0c155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff0802bee0 tx=0x7eff080046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 shutdown_connections 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efef006b500 0x7efef006d9c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff0c152920 0x7eff0c153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff0c0c8a00 0x7eff0c1eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 --2- 172.21.15.17:0/4066673127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff0c0c5330 0x7eff0c155960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 >> 172.21.15.17:0/4066673127 conn(0x7eff0c0ac560 msgr2=0x7eff0c14ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 shutdown_connections 2024-03-06T07:20:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.244+0000 7eff13a0f640 1 -- 172.21.15.17:0/4066673127 wait complete. 2024-03-06T07:20:39.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:39.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:39.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.359+0000 7f74eb5b6640 1 Processor -- start 2024-03-06T07:20:39.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.359+0000 7f74eb5b6640 1 -- start start 2024-03-06T07:20:39.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74eb5b6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f74e40ab170 0x7f74e40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74eb5b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e40c3ef0 con 0x7f74e40c5990 2024-03-06T07:20:39.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74eb5b6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74e41556d0 con 0x7f74e406c900 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74eb5b6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e41559b0 con 0x7f74e40ab170 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74e9352640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f74e40c5990 0x7f74e406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35814/0 (socket says 172.21.15.17:35814) 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74e9352640 1 -- 172.21.15.17:0/1872393505 learned_addr learned my addr 172.21.15.17:0/1872393505 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.360+0000 7f74e9b53640 1 --2- 172.21.15.17:0/1872393505 >> v2:172.21.15.163:3300/0 conn(0x7f74e40ab170 0x7f74e40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890465892 0 0) 0x7f74e40c3ef0 con 0x7f74e40c5990 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74cc003520 con 0x7f74e40c5990 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298614141 0 0) 0x7f74e41556d0 con 0x7f74e406c900 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74e40c3ef0 con 0x7f74e406c900 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e9b53640 1 -- 172.21.15.17:0/1872393505 >> v1:172.21.15.163:6789/0 conn(0x7f74e406c900 legacy=0x7f74e40a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e9b53640 1 -- 172.21.15.17:0/1872393505 >> v1:172.21.15.17:6789/0 conn(0x7f74e40c5990 legacy=0x7f74e406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e9b53640 1 -- 172.21.15.17:0/1872393505 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e4155c90 con 0x7f74e40ab170 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.361+0000 7f74e9b53640 1 --2- 172.21.15.17:0/1872393505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e40ab170 0x7f74e40c0ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097d0 tx=0x7f74dc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c7c54231b2d253 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc034070 con 0x7f74e40ab170 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc030610 con 0x7f74e40ab170 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74e3fff640 1 -- 172.21.15.17:0/1872393505 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0309e0 con 0x7f74e40ab170 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 -- 172.21.15.17:0/1872393505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e40ab170 msgr2=0x7f74e40c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/1872393505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e40ab170 0x7f74e40c0ab0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097d0 tx=0x7f74dc02b960 comp rx=0 tx=0).stop 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 -- 172.21.15.17:0/1872393505 shutdown_connections 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/1872393505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e40ab170 0x7f74e40c0ab0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 -- 172.21.15.17:0/1872393505 >> 172.21.15.17:0/1872393505 conn(0x7f74e40c6710 msgr2=0x7f74e40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 -- 172.21.15.17:0/1872393505 shutdown_connections 2024-03-06T07:20:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.362+0000 7f74eb5b6640 1 -- 172.21.15.17:0/1872393505 wait complete. 2024-03-06T07:20:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.363+0000 7f74eb5b6640 1 Processor -- start 2024-03-06T07:20:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.363+0000 7f74eb5b6640 1 -- start start 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e406c900 0x7f74e414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74e40ab170 0x7f74e414b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4156610 con 0x7f74e40c5990 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4156490 con 0x7f74e406c900 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74eb5b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40c3af0 con 0x7f74e40ab170 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45802/0 (socket says 172.21.15.17:45802) 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 -- 172.21.15.17:0/3888524316 learned_addr learned my addr 172.21.15.17:0/3888524316 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e8b51640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74e40ab170 0x7f74e414b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9352640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e406c900 0x7f74e414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 -- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74e40ab170 msgr2=0x7f74e414b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74e40ab170 0x7f74e414b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 -- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e406c900 msgr2=0x7f74e414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e406c900 0x7f74e414a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.364+0000 7f74e9b53640 1 -- 172.21.15.17:0/3888524316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e40c2330 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74e9b53640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097d0 tx=0x7f74dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc034040 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc030410 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc006e10 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e40c2610 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.365+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e4153d20 con 0x7f74e40c5990 2024-03-06T07:20:39.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.366+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e404f330 con 0x7f74e40c5990 2024-03-06T07:20:39.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.367+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74dc037070 con 0x7f74e40c5990 2024-03-06T07:20:39.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.367+0000 7f74e1ffb640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 0x7f74d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.367+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f74dc0ac2d0 con 0x7f74e40c5990 2024-03-06T07:20:39.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.367+0000 7f74e9352640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 0x7f74d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.367+0000 7f74e9352640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 0x7f74d006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74d40012e0 tx=0x7f74d4001a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.371+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f74dc07a7b0 con 0x7f74e40c5990 2024-03-06T07:20:39.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.550+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74e40c1740 con 0x7f74e40c5990 2024-03-06T07:20:39.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.550+0000 7f74e1ffb640 1 -- 172.21.15.17:0/3888524316 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74dc07e850 con 0x7f74e40c5990 2024-03-06T07:20:39.552 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 msgr2=0x7f74d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 0x7f74d006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74d40012e0 tx=0x7f74d4001a10 comp rx=0 tx=0).stop 2024-03-06T07:20:39.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 msgr2=0x7f74e4144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0097d0 tx=0x7f74dc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 shutdown_connections 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f74d006b4d0 0x7f74d006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74e40c5990 0x7f74e4144780 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74e40ab170 0x7f74e414b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 --2- 172.21.15.17:0/3888524316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74e406c900 0x7f74e414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.552+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 >> 172.21.15.17:0/3888524316 conn(0x7f74e40c6710 msgr2=0x7f74e414ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.553+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 shutdown_connections 2024-03-06T07:20:39.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.553+0000 7f74eb5b6640 1 -- 172.21.15.17:0/3888524316 wait complete. 2024-03-06T07:20:39.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.668+0000 7f15f5a95640 1 Processor -- start 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.668+0000 7f15f5a95640 1 -- start start 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15f5a95640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f006c900 0x7f15f00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15f5a95640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15f00ab170 0x7f15f00a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15f5a95640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f00c3ef0 con 0x7f15f00c5990 2024-03-06T07:20:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15f5a95640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f01556d0 con 0x7f15f00ab170 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15f5a95640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f01559b0 con 0x7f15f006c900 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15effff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15f00ab170 0x7f15f00a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15ef7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f15f00c5990 0x7f15f006c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35816/0 (socket says 172.21.15.17:35816) 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15ef7fe640 1 -- 172.21.15.17:0/1204917641 learned_addr learned my addr 172.21.15.17:0/1204917641 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15eeffd640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f006c900 0x7f15f00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15effff640 1 -- 172.21.15.17:0/1204917641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f006c900 msgr2=0x7f15f00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15effff640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f006c900 0x7f15f00c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15effff640 1 -- 172.21.15.17:0/1204917641 >> v1:172.21.15.17:6789/0 conn(0x7f15f00c5990 legacy=0x7f15f006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15ee7fc640 1 -- 172.21.15.17:0/1204917641 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668293079 0 0) 0x7f15f00c3ef0 con 0x7f15f00c5990 2024-03-06T07:20:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.669+0000 7f15effff640 1 -- 172.21.15.17:0/1204917641 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0155c90 con 0x7f15f00ab170 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.670+0000 7f15effff640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00ab170 0x7f15f00a9fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009d90 tx=0x7f15e802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99531fef5b61328c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.670+0000 7f15ee7fc640 1 -- 172.21.15.17:0/1204917641 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8039070 con 0x7f15f00ab170 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.670+0000 7f15ee7fc640 1 -- 172.21.15.17:0/1204917641 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e802cc50 con 0x7f15f00ab170 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.670+0000 7f15ee7fc640 1 -- 172.21.15.17:0/1204917641 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8034440 con 0x7f15f00ab170 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 -- 172.21.15.17:0/1204917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00ab170 msgr2=0x7f15f00a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00ab170 0x7f15f00a9fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009d90 tx=0x7f15e802bea0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 -- 172.21.15.17:0/1204917641 shutdown_connections 2024-03-06T07:20:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00ab170 0x7f15f00a9fa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 --2- 172.21.15.17:0/1204917641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f006c900 0x7f15f00c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 -- 172.21.15.17:0/1204917641 >> 172.21.15.17:0/1204917641 conn(0x7f15f00c6710 msgr2=0x7f15f00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 -- 172.21.15.17:0/1204917641 shutdown_connections 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.671+0000 7f15f5a95640 1 -- 172.21.15.17:0/1204917641 wait complete. 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.672+0000 7f15f5a95640 1 Processor -- start 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.672+0000 7f15f5a95640 1 -- start start 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15f006c900 0x7f15f0070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 0x7f15f00a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0156610 con 0x7f15f00c5990 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0156490 con 0x7f15f006c900 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00c3af0 con 0x7f15f00ab170 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45814/0 (socket says 172.21.15.17:45814) 2024-03-06T07:20:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 -- 172.21.15.17:0/3319658569 learned_addr learned my addr 172.21.15.17:0/3319658569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15eeffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 0x7f15f00a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15ef7fe640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15f006c900 0x7f15f0070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 -- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 msgr2=0x7f15f00a7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 0x7f15f00a7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 -- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15f006c900 msgr2=0x7f15f0070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15f006c900 0x7f15f0070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 -- 172.21.15.17:0/3319658569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0151fc0 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15eeffd640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 0x7f15f00a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15effff640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009d90 tx=0x7f15e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8039050 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8045070 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8009450 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.673+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f01ee500 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.674+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f00a80d0 con 0x7f15f00c5990 2024-03-06T07:20:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.675+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f004f330 con 0x7f15f00c5990 2024-03-06T07:20:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.675+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15e803e610 con 0x7f15f00c5990 2024-03-06T07:20:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.675+0000 7f15d7fff640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 0x7f15dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.676+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15e80ac7a0 con 0x7f15f00c5990 2024-03-06T07:20:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.676+0000 7f15ef7fe640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 0x7f15dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.676+0000 7f15ef7fe640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 0x7f15dc06d9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15e0007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.680+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15e804e020 con 0x7f15f00c5990 2024-03-06T07:20:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.863+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15f00c12d0 con 0x7f15f00c5990 2024-03-06T07:20:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.863+0000 7f15d7fff640 1 -- 172.21.15.17:0/3319658569 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15f0151fc0 con 0x7f15f00c5990 2024-03-06T07:20:39.865 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 msgr2=0x7f15dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 0x7f15dc06d9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007fd0 tx=0x7f15e0007de0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 msgr2=0x7f15f01eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009d90 tx=0x7f15e8004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 shutdown_connections 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15dc06b500 0x7f15dc06d9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15f00c5990 0x7f15f01eac50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15f00ab170 0x7f15f00a7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 --2- 172.21.15.17:0/3319658569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15f006c900 0x7f15f0070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.865+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 >> 172.21.15.17:0/3319658569 conn(0x7f15f00c6710 msgr2=0x7f15f014b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.866+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 shutdown_connections 2024-03-06T07:20:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.866+0000 7f15f5a95640 1 -- 172.21.15.17:0/3319658569 wait complete. 2024-03-06T07:20:39.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:39.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.981+0000 7fda4bb3e640 1 Processor -- start 2024-03-06T07:20:39.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.981+0000 7fda4bb3e640 1 -- start start 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4bb3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 0x7fda44155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4bb3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440c75e0 con 0x7fda44158a10 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4bb3e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440c78c0 con 0x7fda4414bcc0 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4bb3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c7f10 con 0x7fda440be150 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4a0db640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fda44158a10 0x7fda44158e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35830/0 (socket says 172.21.15.17:35830) 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda4a0db640 1 -- 172.21.15.17:0/532920145 learned_addr learned my addr 172.21.15.17:0/532920145 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda490d9640 1 --2- 172.21.15.17:0/532920145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 0x7fda44155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237154457 0 0) 0x7fda440c75e0 con 0x7fda44158a10 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda30003520 con 0x7fda44158a10 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79734526 0 0) 0x7fda440c78c0 con 0x7fda4414bcc0 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda440c75e0 con 0x7fda4414bcc0 2024-03-06T07:20:39.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda490d9640 1 -- 172.21.15.17:0/532920145 >> v1:172.21.15.163:6789/0 conn(0x7fda4414bcc0 legacy=0x7fda440c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda490d9640 1 -- 172.21.15.17:0/532920145 >> v1:172.21.15.17:6789/0 conn(0x7fda44158a10 legacy=0x7fda44158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.982+0000 7fda490d9640 1 -- 172.21.15.17:0/532920145 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440c7ba0 con 0x7fda440be150 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.983+0000 7fda490d9640 1 --2- 172.21.15.17:0/532920145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 0x7fda44155a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda340097d0 tx=0x7fda3402b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8298c21dec1cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.983+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34034070 con 0x7fda440be150 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.983+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda34030880 con 0x7fda440be150 2024-03-06T07:20:39.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.983+0000 7fda488d8640 1 -- 172.21.15.17:0/532920145 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34030c30 con 0x7fda440be150 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 -- 172.21.15.17:0/532920145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 msgr2=0x7fda44155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/532920145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 0x7fda44155a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda340097d0 tx=0x7fda3402b9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 -- 172.21.15.17:0/532920145 shutdown_connections 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/532920145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda440be150 0x7fda44155a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 -- 172.21.15.17:0/532920145 >> 172.21.15.17:0/532920145 conn(0x7fda440ac600 msgr2=0x7fda440aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 -- 172.21.15.17:0/532920145 shutdown_connections 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.984+0000 7fda4bb3e640 1 -- 172.21.15.17:0/532920145 wait complete. 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.985+0000 7fda4bb3e640 1 Processor -- start 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.985+0000 7fda4bb3e640 1 -- start start 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda440be150 0x7fda440c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda4414bcc0 0x7fda440c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c5ae0 con 0x7fda44158a10 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4406d1d0 con 0x7fda440be150 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4bb3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c5c60 con 0x7fda4414bcc0 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda490d9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda4414bcc0 0x7fda440c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45822/0 (socket says 172.21.15.17:45822) 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda498da640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda440be150 0x7fda440c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 -- 172.21.15.17:0/125707241 learned_addr learned my addr 172.21.15.17:0/125707241 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 -- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda4414bcc0 msgr2=0x7fda440c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda4414bcc0 0x7fda440c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 -- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda440be150 msgr2=0x7fda440c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda440be150 0x7fda440c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 -- 172.21.15.17:0/125707241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440c48c0 con 0x7fda44158a10 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda4a0db640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00cad0 tx=0x7fda3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c017070 con 0x7fda44158a10 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.986+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c00ac50 con 0x7fda44158a10 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.987+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda441eae60 con 0x7fda44158a10 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.987+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c012400 con 0x7fda44158a10 2024-03-06T07:20:39.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.987+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda441eb420 con 0x7fda44158a10 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.987+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda441eac50 con 0x7fda44158a10 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.988+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda3c01a070 con 0x7fda44158a10 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.988+0000 7fda3a7fc640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 0x7fda2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.988+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fda3c08a230 con 0x7fda44158a10 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.989+0000 7fda498da640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 0x7fda2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:39.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.989+0000 7fda498da640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 0x7fda2806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda40007fd0 tx=0x7fda40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:39.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:39.993+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda3c058730 con 0x7fda44158a10 2024-03-06T07:20:40.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.172+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda4404f330 con 0x7fda44158a10 2024-03-06T07:20:40.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.173+0000 7fda3a7fc640 1 -- 172.21.15.17:0/125707241 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda3c05c7d0 con 0x7fda44158a10 2024-03-06T07:20:40.174 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:40.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.174+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 msgr2=0x7fda2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.174+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 0x7fda2806d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fda40007fd0 tx=0x7fda40009040 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.174+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 msgr2=0x7fda440c2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.174+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00cad0 tx=0x7fda3c00cf90 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 shutdown_connections 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda2806b4d0 0x7fda2806d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda44158a10 0x7fda440c2060 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda4414bcc0 0x7fda440c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 --2- 172.21.15.17:0/125707241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda440be150 0x7fda440c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 >> 172.21.15.17:0/125707241 conn(0x7fda440ac600 msgr2=0x7fda440c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 shutdown_connections 2024-03-06T07:20:40.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.175+0000 7fda4bb3e640 1 -- 172.21.15.17:0/125707241 wait complete. 2024-03-06T07:20:40.187 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:40.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.289+0000 7f51c3e88640 1 Processor -- start 2024-03-06T07:20:40.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.289+0000 7f51c3e88640 1 -- start start 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c3e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c3e88640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51bc0c5380 0x7f51bc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c3e88640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc0b0450 con 0x7f51bc06ca70 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c3e88640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc06c020 con 0x7f51bc0c5380 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c3e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0c77f0 con 0x7f51bc0b0070 2024-03-06T07:20:40.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c2425640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51bc0c5380 0x7f51bc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c2425640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51bc0c5380 0x7f51bc0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45828/0 (socket says 172.21.15.17:45828) 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c2425640 1 -- 172.21.15.17:0/3529682985 learned_addr learned my addr 172.21.15.17:0/3529682985 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.290+0000 7f51c1c24640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.291+0000 7f51c2425640 1 -- 172.21.15.17:0/3529682985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 msgr2=0x7f51bc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.291+0000 7f51c2425640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.291+0000 7f51c2425640 1 -- 172.21.15.17:0/3529682985 >> v1:172.21.15.163:6789/0 conn(0x7f51bc06ca70 legacy=0x7f51bc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.291+0000 7f51c2425640 1 -- 172.21.15.17:0/3529682985 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0c7ad0 con 0x7f51bc0c5380 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.291+0000 7f51c2425640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c57d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009e50 tx=0x7f51ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d70b70af0415318 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c0c22640 1 -- 172.21.15.17:0/3529682985 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac02ca70 con 0x7f51bc0c5380 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c0c22640 1 -- 172.21.15.17:0/3529682985 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac02cc10 con 0x7f51bc0c5380 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c0c22640 1 -- 172.21.15.17:0/3529682985 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac0343f0 con 0x7f51bc0c5380 2024-03-06T07:20:40.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 -- 172.21.15.17:0/3529682985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 msgr2=0x7f51bc0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c57d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009e50 tx=0x7f51ac02bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 -- 172.21.15.17:0/3529682985 shutdown_connections 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c57d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 --2- 172.21.15.17:0/3529682985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 -- 172.21.15.17:0/3529682985 >> 172.21.15.17:0/3529682985 conn(0x7f51bc0ac590 msgr2=0x7f51bc0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 -- 172.21.15.17:0/3529682985 shutdown_connections 2024-03-06T07:20:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.292+0000 7f51c3e88640 1 -- 172.21.15.17:0/3529682985 wait complete. 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.293+0000 7f51c3e88640 1 Processor -- start 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.293+0000 7f51c3e88640 1 -- start start 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51bc06ca70 0x7f51bc14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc06cef0 con 0x7f51bc0c5380 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc06c550 con 0x7f51bc06ca70 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c3e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc06da80 con 0x7f51bc0b0070 2024-03-06T07:20:40.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45830/0 (socket says 172.21.15.17:45830) 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 -- 172.21.15.17:0/2228157048 learned_addr learned my addr 172.21.15.17:0/2228157048 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c1c24640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51bc06ca70 0x7f51bc14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c1423640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 -- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 msgr2=0x7f51bc0c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc0c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 -- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51bc06ca70 msgr2=0x7f51bc14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51bc06ca70 0x7f51bc14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.294+0000 7f51c2425640 1 -- 172.21.15.17:0/2228157048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc1ea870 con 0x7f51bc0c5380 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51c2425640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009e50 tx=0x7f51ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac043020 con 0x7f51bc0c5380 2024-03-06T07:20:40.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac03e450 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc1eab50 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac009510 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.295+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc1eb110 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.296+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51bc04f330 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.296+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51ac046020 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.297+0000 7f51ba7fc640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 0x7f51a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.297+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51ac0acc00 con 0x7f51bc0c5380 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.297+0000 7f51c1c24640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 0x7f51a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.297+0000 7f51c1c24640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 0x7f51a806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009b40 tx=0x7f51b4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.301+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51ac04d030 con 0x7f51bc0c5380 2024-03-06T07:20:40.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.480+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51bc06d860 con 0x7f51bc0c5380 2024-03-06T07:20:40.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.481+0000 7f51ba7fc640 1 -- 172.21.15.17:0/2228157048 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51ac07b050 con 0x7f51bc0c5380 2024-03-06T07:20:40.483 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:40.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 msgr2=0x7f51a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 0x7f51a806d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009b40 tx=0x7f51b4009190 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 msgr2=0x7f51bc0c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009e50 tx=0x7f51ac004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 shutdown_connections 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51a806b4d0 0x7f51a806d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51bc0c5380 0x7f51bc0c4260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51bc0b0070 0x7f51bc0c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 --2- 172.21.15.17:0/2228157048 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51bc06ca70 0x7f51bc14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 >> 172.21.15.17:0/2228157048 conn(0x7f51bc0ac590 msgr2=0x7f51bc0c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 shutdown_connections 2024-03-06T07:20:40.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.483+0000 7f51c3e88640 1 -- 172.21.15.17:0/2228157048 wait complete. 2024-03-06T07:20:40.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.596+0000 7fa1e262a640 1 Processor -- start 2024-03-06T07:20:40.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.596+0000 7fa1e262a640 1 -- start start 2024-03-06T07:20:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e262a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1dc14f360 0x7fa1dc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e262a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1dc0c88c0 0x7fa1dc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e262a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0c73d0 con 0x7fa1dc0c5880 2024-03-06T07:20:40.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e262a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c8dd0 con 0x7fa1dc14f360 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e262a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0be2e0 con 0x7fa1dc0c88c0 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1dc14f360 0x7fa1dc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1dc14f360 0x7fa1dc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43608/0 (socket says 172.21.15.17:43608) 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 -- 172.21.15.17:0/1097132645 learned_addr learned my addr 172.21.15.17:0/1097132645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1e0bc7640 1 --1- 172.21.15.17:0/1097132645 >> v1:172.21.15.17:6789/0 conn(0x7fa1dc0c5880 0x7fa1dc158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41302/0 (socket says 172.21.15.17:41302) 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1db7fe640 1 --2- 172.21.15.17:0/1097132645 >> v2:172.21.15.163:3300/0 conn(0x7fa1dc0c88c0 0x7fa1dc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 -- 172.21.15.17:0/1097132645 >> v2:172.21.15.163:3300/0 conn(0x7fa1dc0c88c0 msgr2=0x7fa1dc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 --2- 172.21.15.17:0/1097132645 >> v2:172.21.15.163:3300/0 conn(0x7fa1dc0c88c0 0x7fa1dc0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 -- 172.21.15.17:0/1097132645 >> v1:172.21.15.17:6789/0 conn(0x7fa1dc0c5880 legacy=0x7fa1dc158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1dbfff640 1 -- 172.21.15.17:0/1097132645 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0c7670 con 0x7fa1dc14f360 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.597+0000 7fa1daffd640 1 -- 172.21.15.17:0/1097132645 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441981954 0 0) 0x7fa1dc0c73d0 con 0x7fa1dc0c5880 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.598+0000 7fa1dbfff640 1 --2- 172.21.15.17:0/1097132645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc155c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0c84c0 tx=0x7fa1cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b603eaf69e6ad6f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.598+0000 7fa1daffd640 1 -- 172.21.15.17:0/1097132645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc035020 con 0x7fa1dc14f360 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.598+0000 7fa1daffd640 1 -- 172.21.15.17:0/1097132645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc031870 con 0x7fa1dc14f360 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.598+0000 7fa1daffd640 1 -- 172.21.15.17:0/1097132645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc031c40 con 0x7fa1dc14f360 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 -- 172.21.15.17:0/1097132645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 msgr2=0x7fa1dc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 --2- 172.21.15.17:0/1097132645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc155c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0c84c0 tx=0x7fa1cc02b910 comp rx=0 tx=0).stop 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 -- 172.21.15.17:0/1097132645 shutdown_connections 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 --2- 172.21.15.17:0/1097132645 >> v2:172.21.15.163:3300/0 conn(0x7fa1dc0c88c0 0x7fa1dc0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 --2- 172.21.15.17:0/1097132645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc155c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 -- 172.21.15.17:0/1097132645 >> 172.21.15.17:0/1097132645 conn(0x7fa1dc0ac570 msgr2=0x7fa1dc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 -- 172.21.15.17:0/1097132645 shutdown_connections 2024-03-06T07:20:40.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.599+0000 7fa1e262a640 1 -- 172.21.15.17:0/1097132645 wait complete. 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.600+0000 7fa1e262a640 1 Processor -- start 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.600+0000 7fa1e262a640 1 -- start start 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1dc0c5880 0x7fa1dc1ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1dc0c88c0 0x7fa1dc1eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc159270 con 0x7fa1dc14f360 2024-03-06T07:20:40.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c7c30 con 0x7fa1dc0c88c0 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc1593f0 con 0x7fa1dc0c5880 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43610/0 (socket says 172.21.15.17:43610) 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 -- 172.21.15.17:0/2062247210 learned_addr learned my addr 172.21.15.17:0/2062247210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1dbfff640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1dc0c5880 0x7fa1dc1ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1db7fe640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1dc0c88c0 0x7fa1dc1eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 -- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1dc0c5880 msgr2=0x7fa1dc1ebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1dc0c5880 0x7fa1dc1ebbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 -- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1dc0c88c0 msgr2=0x7fa1dc1eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1dc0c88c0 0x7fa1dc1eb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 -- 172.21.15.17:0/2062247210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc1eb690 con 0x7fa1dc14f360 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e0bc7640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c7a0 tx=0x7fa1d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4018070 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d400a640 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc157f40 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400aa70 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.601+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc0bf6c0 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.602+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc04f330 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.603+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1d401b070 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.603+0000 7fa1d8ff9640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.603+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa1d4089a50 con 0x7fa1dc14f360 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.603+0000 7fa1dbfff640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.604+0000 7fa1dbfff640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc036000 tx=0x7fa1cc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.607+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1dc1eb690 con 0x7fa1dc14f360 2024-03-06T07:20:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.793+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1dc06cda0 con 0x7fa1dc14f360 2024-03-06T07:20:40.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.794+0000 7fa1d8ff9640 1 -- 172.21.15.17:0/2062247210 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fa1d4057f30 con 0x7fa1dc14f360 2024-03-06T07:20:40.796 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:40.796 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:40.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 msgr2=0x7fa1bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc036000 tx=0x7fa1cc008030 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 msgr2=0x7fa1dc0c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c7a0 tx=0x7fa1d400cc60 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 shutdown_connections 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.796+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1dc14f360 0x7fa1dc0c0e50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1dc0c88c0 0x7fa1dc1eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 --2- 172.21.15.17:0/2062247210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1dc0c5880 0x7fa1dc1ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 >> 172.21.15.17:0/2062247210 conn(0x7fa1dc0ac570 msgr2=0x7fa1dc06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 shutdown_connections 2024-03-06T07:20:40.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.797+0000 7fa1e262a640 1 -- 172.21.15.17:0/2062247210 wait complete. 2024-03-06T07:20:40.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:40.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.912+0000 7f830e94b640 1 Processor -- start 2024-03-06T07:20:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.912+0000 7f830e94b640 1 -- start start 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830e94b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830e94b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f830806db30 0x7f83080c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830e94b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8308156060 con 0x7f83080c3a70 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830e94b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8308155b30 con 0x7f830806db30 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830e94b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080a8430 con 0x7f830806d750 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f830806db30 0x7f83080c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f830806db30 0x7f83080c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43620/0 (socket says 172.21.15.17:43620) 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 -- 172.21.15.17:0/3939340970 learned_addr learned my addr 172.21.15.17:0/3939340970 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f8307fff640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 -- 172.21.15.17:0/3939340970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 msgr2=0x7f83080732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 -- 172.21.15.17:0/3939340970 >> v1:172.21.15.163:6789/0 conn(0x7f83080c3a70 legacy=0x7f8308072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.913+0000 7f830cee8640 1 -- 172.21.15.17:0/3939340970 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308155d80 con 0x7f830806db30 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.914+0000 7f830cee8640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080c1e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8300009e50 tx=0x7f830002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935410f99afef749 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.914+0000 7f8306ffd640 1 -- 172.21.15.17:0/3939340970 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830002c9e0 con 0x7f830806db30 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.914+0000 7f8306ffd640 1 -- 172.21.15.17:0/3939340970 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830002cb80 con 0x7f830806db30 2024-03-06T07:20:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.914+0000 7f8306ffd640 1 -- 172.21.15.17:0/3939340970 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83000343f0 con 0x7f830806db30 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 -- 172.21.15.17:0/3939340970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 msgr2=0x7f83080c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080c1e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8300009e50 tx=0x7f830002be40 comp rx=0 tx=0).stop 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 -- 172.21.15.17:0/3939340970 shutdown_connections 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080c1e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 --2- 172.21.15.17:0/3939340970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 -- 172.21.15.17:0/3939340970 >> 172.21.15.17:0/3939340970 conn(0x7f83080c5560 msgr2=0x7f83080c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 -- 172.21.15.17:0/3939340970 shutdown_connections 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.915+0000 7f830e94b640 1 -- 172.21.15.17:0/3939340970 wait complete. 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.916+0000 7f830e94b640 1 Processor -- start 2024-03-06T07:20:40.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.916+0000 7f830e94b640 1 -- start start 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83080c3a70 0x7f83080add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080a7e50 con 0x7f830806db30 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83080a7cd0 con 0x7f83080c3a70 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830e94b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83080a7fd0 con 0x7f830806d750 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43626/0 (socket says 172.21.15.17:43626) 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 -- 172.21.15.17:0/1826485172 learned_addr learned my addr 172.21.15.17:0/1826485172 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f830cee8640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83080c3a70 0x7f83080add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f8307fff640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 -- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 msgr2=0x7f83080a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 -- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83080c3a70 msgr2=0x7f83080add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83080c3a70 0x7f83080add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.917+0000 7f83077fe640 1 -- 172.21.15.17:0/1826485172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83081542e0 con 0x7f830806db30 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f83077fe640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c440 tx=0x7f82f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00052d0 con 0x7f830806db30 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0005470 con 0x7f830806db30 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081545c0 con 0x7f830806db30 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0007240 con 0x7f830806db30 2024-03-06T07:20:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.918+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8308154b80 con 0x7f830806db30 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.919+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830804f330 con 0x7f830806db30 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.919+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82f00073e0 con 0x7f830806db30 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.920+0000 7f8304ff9640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 0x7f82e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.920+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f82f00898b0 con 0x7f830806db30 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.920+0000 7f8307fff640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 0x7f82e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.920+0000 7f8307fff640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 0x7f82e806d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009aa0 tx=0x7f82f8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:40.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:40.924+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f82f0057da0 con 0x7f830806db30 2024-03-06T07:20:41.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.103+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83080aa170 con 0x7f830806db30 2024-03-06T07:20:41.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.103+0000 7f8304ff9640 1 -- 172.21.15.17:0/1826485172 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82f005be40 con 0x7f830806db30 2024-03-06T07:20:41.105 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 msgr2=0x7f82e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 0x7f82e806d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009aa0 tx=0x7f82f8009190 comp rx=0 tx=0).stop 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 msgr2=0x7f83080ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c440 tx=0x7f82f0001e70 comp rx=0 tx=0).stop 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 shutdown_connections 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f82e806b500 0x7f82e806d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83080c3a70 0x7f83080add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f830806db30 0x7f83080ad7d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 --2- 172.21.15.17:0/1826485172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f830806d750 0x7f83080a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 >> 172.21.15.17:0/1826485172 conn(0x7f83080c5560 msgr2=0x7f83080a8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 shutdown_connections 2024-03-06T07:20:41.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.105+0000 7f830e94b640 1 -- 172.21.15.17:0/1826485172 wait complete. 2024-03-06T07:20:41.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:41.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.220+0000 7f17d54ed640 1 Processor -- start 2024-03-06T07:20:41.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.220+0000 7f17d54ed640 1 -- start start 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17d54ed640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17d00be150 0x7f17d0155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17d54ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00c75e0 con 0x7f17d014bcc0 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17d54ed640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00c78c0 con 0x7f17d0158a10 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17d54ed640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00c7f10 con 0x7f17d00be150 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ceffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f17d014bcc0 0x7f17d00c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41318/0 (socket says 172.21.15.17:41318) 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ceffd640 1 -- 172.21.15.17:0/1039634927 learned_addr learned my addr 172.21.15.17:0/1039634927 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17c7fff640 1 --2- 172.21.15.17:0/1039634927 >> v2:172.21.15.163:3300/0 conn(0x7f17d00be150 0x7f17d0155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935760741 0 0) 0x7f17d00c75e0 con 0x7f17d014bcc0 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17bc003520 con 0x7f17d014bcc0 2024-03-06T07:20:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892522908 0 0) 0x7f17d00c78c0 con 0x7f17d0158a10 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d00c75e0 con 0x7f17d0158a10 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17c7fff640 1 -- 172.21.15.17:0/1039634927 >> v1:172.21.15.163:6789/0 conn(0x7f17d0158a10 legacy=0x7f17d0158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17c7fff640 1 -- 172.21.15.17:0/1039634927 >> v1:172.21.15.17:6789/0 conn(0x7f17d014bcc0 legacy=0x7f17d00c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.221+0000 7f17c7fff640 1 -- 172.21.15.17:0/1039634927 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00c7ba0 con 0x7f17d00be150 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.222+0000 7f17c7fff640 1 --2- 172.21.15.17:0/1039634927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d00be150 0x7f17d0155a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17b80097a0 tx=0x7f17b802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de03f32e32bcb3a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.222+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802f810 con 0x7f17d00be150 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.222+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b802f9b0 con 0x7f17d00be150 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.222+0000 7f17ce7fc640 1 -- 172.21.15.17:0/1039634927 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b802fd80 con 0x7f17d00be150 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 -- 172.21.15.17:0/1039634927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d00be150 msgr2=0x7f17d0155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 --2- 172.21.15.17:0/1039634927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d00be150 0x7f17d0155a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17b80097a0 tx=0x7f17b802b9f0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 -- 172.21.15.17:0/1039634927 shutdown_connections 2024-03-06T07:20:41.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 --2- 172.21.15.17:0/1039634927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d00be150 0x7f17d0155a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 -- 172.21.15.17:0/1039634927 >> 172.21.15.17:0/1039634927 conn(0x7f17d00ac600 msgr2=0x7f17d00aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 -- 172.21.15.17:0/1039634927 shutdown_connections 2024-03-06T07:20:41.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.223+0000 7f17d54ed640 1 -- 172.21.15.17:0/1039634927 wait complete. 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.224+0000 7f17d54ed640 1 Processor -- start 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.224+0000 7f17d54ed640 1 -- start start 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d014bcc0 0x7f17d0147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d0158a10 0x7f17d0154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00c5ae0 con 0x7f17d00be150 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d006d1d0 con 0x7f17d014bcc0 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00c5c60 con 0x7f17d0158a10 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43628/0 (socket says 172.21.15.17:43628) 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 -- 172.21.15.17:0/2583461995 learned_addr learned my addr 172.21.15.17:0/2583461995 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17c7fff640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d014bcc0 0x7f17d0147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17cf7fe640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d0158a10 0x7f17d0154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 -- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d0158a10 msgr2=0x7f17d0154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d0158a10 0x7f17d0154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 -- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d014bcc0 msgr2=0x7f17d0147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d014bcc0 0x7f17d0147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 -- 172.21.15.17:0/2583461995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d0152560 con 0x7f17d00be150 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17ceffd640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c500 tx=0x7f17c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0017070 con 0x7f17d00be150 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0005ce0 con 0x7f17d00be150 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00072f0 con 0x7f17d00be150 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d00c43a0 con 0x7f17d00be150 2024-03-06T07:20:41.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.225+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d01545c0 con 0x7f17d00be150 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.226+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d004f330 con 0x7f17d00be150 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.227+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17c001a070 con 0x7f17d00be150 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.227+0000 7f17c77fe640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 0x7f17b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.227+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f17c0089810 con 0x7f17d00be150 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.227+0000 7f17c7fff640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 0x7f17b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.228+0000 7f17c7fff640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 0x7f17b006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17b8036000 tx=0x7f17b8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.231+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f17c0057d90 con 0x7f17d00be150 2024-03-06T07:20:41.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.410+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17d00c83e0 con 0x7f17d00be150 2024-03-06T07:20:41.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.411+0000 7f17c77fe640 1 -- 172.21.15.17:0/2583461995 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17c0087e20 con 0x7f17d00be150 2024-03-06T07:20:41.413 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 msgr2=0x7f17b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 0x7f17b006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f17b8036000 tx=0x7f17b8008030 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 msgr2=0x7f17d0151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c500 tx=0x7f17c000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 shutdown_connections 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f17b006b4d0 0x7f17b006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d0158a10 0x7f17d0154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d014bcc0 0x7f17d0147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 --2- 172.21.15.17:0/2583461995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17d00be150 0x7f17d0151f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 >> 172.21.15.17:0/2583461995 conn(0x7f17d00ac600 msgr2=0x7f17d00abe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 shutdown_connections 2024-03-06T07:20:41.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.413+0000 7f17d54ed640 1 -- 172.21.15.17:0/2583461995 wait complete. 2024-03-06T07:20:41.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.530+0000 7f4725439640 1 Processor -- start 2024-03-06T07:20:41.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.530+0000 7f4725439640 1 -- start start 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f4725439640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 0x7f4720155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f4725439640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47200c5380 0x7f47200c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f4725439640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47200b0450 con 0x7f472006ca70 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f4725439640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472006c020 con 0x7f47200c5380 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f4725439640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472006cf20 con 0x7f47200b0070 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47200c5380 0x7f47200c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47200c5380 0x7f47200c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43632/0 (socket says 172.21.15.17:43632) 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 -- 172.21.15.17:0/1338763545 learned_addr learned my addr 172.21.15.17:0/1338763545 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:41.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471effd640 1 --2- 172.21.15.17:0/1338763545 >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 0x7f4720155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 -- 172.21.15.17:0/1338763545 >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 msgr2=0x7f4720155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 --2- 172.21.15.17:0/1338763545 >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 0x7f4720155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 -- 172.21.15.17:0/1338763545 >> v1:172.21.15.163:6789/0 conn(0x7f472006ca70 legacy=0x7f47200c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471f7fe640 1 -- 172.21.15.17:0/1338763545 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200c77f0 con 0x7f47200c5380 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.531+0000 7f471effd640 1 --2- 172.21.15.17:0/1338763545 >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 0x7f4720155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.532+0000 7f471f7fe640 1 --2- 172.21.15.17:0/1338763545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200c5380 0x7f47200c5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47200aa3d0 tx=0x7f471002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52d1d33b4ff7830 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.532+0000 7f471dffb640 1 -- 172.21.15.17:0/1338763545 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471002ca80 con 0x7f47200c5380 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.532+0000 7f471dffb640 1 -- 172.21.15.17:0/1338763545 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471002cc20 con 0x7f47200c5380 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.532+0000 7f471dffb640 1 -- 172.21.15.17:0/1338763545 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100343f0 con 0x7f47200c5380 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 -- 172.21.15.17:0/1338763545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200c5380 msgr2=0x7f47200c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 --2- 172.21.15.17:0/1338763545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200c5380 0x7f47200c5790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f47200aa3d0 tx=0x7f471002bee0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 -- 172.21.15.17:0/1338763545 shutdown_connections 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 --2- 172.21.15.17:0/1338763545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200c5380 0x7f47200c5790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 --2- 172.21.15.17:0/1338763545 >> v2:172.21.15.163:3300/0 conn(0x7f47200b0070 0x7f4720155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 -- 172.21.15.17:0/1338763545 >> 172.21.15.17:0/1338763545 conn(0x7f47200ac590 msgr2=0x7f47200ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 -- 172.21.15.17:0/1338763545 shutdown_connections 2024-03-06T07:20:41.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.533+0000 7f4725439640 1 -- 172.21.15.17:0/1338763545 wait complete. 2024-03-06T07:20:41.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.534+0000 7f4725439640 1 Processor -- start 2024-03-06T07:20:41.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.534+0000 7f4725439640 1 -- start start 2024-03-06T07:20:41.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f472006ca70 0x7f4720154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47200c5380 0x7f472014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472006c550 con 0x7f47200b0070 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472006c270 con 0x7f47200c5380 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f4725439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472006da80 con 0x7f472006ca70 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43648/0 (socket says 172.21.15.17:43648) 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 -- 172.21.15.17:0/491898165 learned_addr learned my addr 172.21.15.17:0/491898165 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471effd640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f472006ca70 0x7f4720154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471f7fe640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47200c5380 0x7f472014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 -- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f472006ca70 msgr2=0x7f4720154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f472006ca70 0x7f4720154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 -- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47200c5380 msgr2=0x7f472014dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47200c5380 0x7f472014dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 -- 172.21.15.17:0/491898165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200c4310 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f471e7fc640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f470800c3c0 tx=0x7f470800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.535+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080052d0 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.536+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708005490 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.536+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47200c45f0 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.536+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708007260 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.536+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47200c4bb0 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.537+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472004f330 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.537+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4708007400 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.537+0000 7f47177fe640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.537+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4708089970 con 0x7f47200b0070 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.537+0000 7f471effd640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.538+0000 7f471effd640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 0x7f470006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f472006bef0 tx=0x7f4718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.542+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f47200b0450 con 0x7f47200b0070 2024-03-06T07:20:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.724+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f472006d860 con 0x7f47200b0070 2024-03-06T07:20:41.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.724+0000 7f47177fe640 1 -- 172.21.15.17:0/491898165 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f470805ce20 con 0x7f47200b0070 2024-03-06T07:20:41.726 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 msgr2=0x7f470006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 0x7f470006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f472006bef0 tx=0x7f4718008040 comp rx=0 tx=0).stop 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 msgr2=0x7f47201590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f470800c3c0 tx=0x7f470800c880 comp rx=0 tx=0).stop 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 shutdown_connections 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47200c5380 0x7f472014dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47200b0070 0x7f47201590e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 --2- 172.21.15.17:0/491898165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f472006ca70 0x7f4720154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 >> 172.21.15.17:0/491898165 conn(0x7f47200ac590 msgr2=0x7f47200ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 shutdown_connections 2024-03-06T07:20:41.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.726+0000 7f4725439640 1 -- 172.21.15.17:0/491898165 wait complete. 2024-03-06T07:20:41.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.841+0000 7f20d7839640 1 Processor -- start 2024-03-06T07:20:41.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.841+0000 7f20d7839640 1 -- start start 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d7839640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20d01529f0 0x7f20d00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d7839640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d00c57d0 0x7f20d0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d7839640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d0152dd0 con 0x7f20d00c5370 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d7839640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d0159710 con 0x7f20d00c57d0 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d7839640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d0156180 con 0x7f20d01529f0 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d00c57d0 0x7f20d0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d00c57d0 0x7f20d0155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43658/0 (socket says 172.21.15.17:43658) 2024-03-06T07:20:41.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 -- 172.21.15.17:0/2994116420 learned_addr learned my addr 172.21.15.17:0/2994116420 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d55d5640 1 --2- 172.21.15.17:0/2994116420 >> v2:172.21.15.163:3300/0 conn(0x7f20d01529f0 0x7f20d00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 -- 172.21.15.17:0/2994116420 >> v2:172.21.15.163:3300/0 conn(0x7f20d01529f0 msgr2=0x7f20d00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 --2- 172.21.15.17:0/2994116420 >> v2:172.21.15.163:3300/0 conn(0x7f20d01529f0 0x7f20d00c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 -- 172.21.15.17:0/2994116420 >> v1:172.21.15.163:6789/0 conn(0x7f20d00c5370 legacy=0x7f20d0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.842+0000 7f20d5dd6640 1 -- 172.21.15.17:0/2994116420 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d0158b00 con 0x7f20d00c57d0 2024-03-06T07:20:41.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.843+0000 7f20d5dd6640 1 --2- 172.21.15.17:0/2994116420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d0155c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20d00cba90 tx=0x7f20c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5bb83cabccc4d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.843+0000 7f20cffff640 1 -- 172.21.15.17:0/2994116420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0035020 con 0x7f20d00c57d0 2024-03-06T07:20:41.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.843+0000 7f20cffff640 1 -- 172.21.15.17:0/2994116420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c0031870 con 0x7f20d00c57d0 2024-03-06T07:20:41.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.843+0000 7f20cffff640 1 -- 172.21.15.17:0/2994116420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c0031c40 con 0x7f20d00c57d0 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 -- 172.21.15.17:0/2994116420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 msgr2=0x7f20d0155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 --2- 172.21.15.17:0/2994116420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d0155c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20d00cba90 tx=0x7f20c002b910 comp rx=0 tx=0).stop 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 -- 172.21.15.17:0/2994116420 shutdown_connections 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 --2- 172.21.15.17:0/2994116420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d0155c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 --2- 172.21.15.17:0/2994116420 >> v2:172.21.15.163:3300/0 conn(0x7f20d01529f0 0x7f20d00c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 -- 172.21.15.17:0/2994116420 >> 172.21.15.17:0/2994116420 conn(0x7f20d00ac580 msgr2=0x7f20d00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 -- 172.21.15.17:0/2994116420 shutdown_connections 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.844+0000 7f20d7839640 1 -- 172.21.15.17:0/2994116420 wait complete. 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.845+0000 7f20d7839640 1 Processor -- start 2024-03-06T07:20:41.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.845+0000 7f20d7839640 1 -- start start 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d00c5370 0x7f20d0154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d01529f0 0x7f20d01eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01590c0 con 0x7f20d00c57d0 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c68b0 con 0x7f20d00c5370 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01593a0 con 0x7f20d01529f0 2024-03-06T07:20:41.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43660/0 (socket says 172.21.15.17:43660) 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 -- 172.21.15.17:0/1910099950 learned_addr learned my addr 172.21.15.17:0/1910099950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d55d5640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d00c5370 0x7f20d0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d5dd6640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d01529f0 0x7f20d01eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 -- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d01529f0 msgr2=0x7f20d01eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d01529f0 0x7f20d01eb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 -- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d00c5370 msgr2=0x7f20d0154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d00c5370 0x7f20d0154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 -- 172.21.15.17:0/1910099950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d01ee030 con 0x7f20d00c57d0 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d4dd4640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c400 tx=0x7f20c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40052d0 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4005490 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d01ee2f0 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.846+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007260 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.847+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d01ee8b0 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.848+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d004f330 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.848+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20c4007400 con 0x7f20d00c57d0 2024-03-06T07:20:41.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.848+0000 7f20cdffb640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:41.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.848+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f20c40899a0 con 0x7f20d00c57d0 2024-03-06T07:20:41.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.848+0000 7f20d55d5640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:41.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.849+0000 7f20d55d5640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 0x7f20bc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20d0152320 tx=0x7f20c800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:41.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:41.852+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20c4057e90 con 0x7f20d00c57d0 2024-03-06T07:20:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.032+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f20d00c9830 con 0x7f20d00c57d0 2024-03-06T07:20:42.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.033+0000 7f20cdffb640 1 -- 172.21.15.17:0/1910099950 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f20c405bf30 con 0x7f20d00c57d0 2024-03-06T07:20:42.034 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.034+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 msgr2=0x7f20bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.034+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 0x7f20bc06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20d0152320 tx=0x7f20c800b040 comp rx=0 tx=0).stop 2024-03-06T07:20:42.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.034+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 msgr2=0x7f20d01eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.034+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20c400c400 tx=0x7f20c400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 shutdown_connections 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20bc06b4d0 0x7f20bc06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d01529f0 0x7f20d01eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d00c57d0 0x7f20d01eada0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 --2- 172.21.15.17:0/1910099950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d00c5370 0x7f20d0154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 >> 172.21.15.17:0/1910099950 conn(0x7f20d00ac580 msgr2=0x7f20d0151610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 shutdown_connections 2024-03-06T07:20:42.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.035+0000 7f20d7839640 1 -- 172.21.15.17:0/1910099950 wait complete. 2024-03-06T07:20:42.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:42.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.147+0000 7fbce7683640 1 Processor -- start 2024-03-06T07:20:42.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.147+0000 7fbce7683640 1 -- start start 2024-03-06T07:20:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce7683640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbce006d9f0 0x7fbce00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce7683640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbce00c3280 0x7fbce00c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce7683640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce0158800 con 0x7fbce01551e0 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce7683640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce0158ae0 con 0x7fbce006d9f0 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce7683640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce0158dc0 con 0x7fbce00c3280 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbce006d9f0 0x7fbce00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbce006d9f0 0x7fbce00a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43670/0 (socket says 172.21.15.17:43670) 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 -- 172.21.15.17:0/2148791502 learned_addr learned my addr 172.21.15.17:0/2148791502 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce4c1e640 1 --2- 172.21.15.17:0/2148791502 >> v2:172.21.15.163:3300/0 conn(0x7fbce00c3280 0x7fbce00c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 -- 172.21.15.17:0/2148791502 >> v2:172.21.15.163:3300/0 conn(0x7fbce00c3280 msgr2=0x7fbce00c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:42.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 --2- 172.21.15.17:0/2148791502 >> v2:172.21.15.163:3300/0 conn(0x7fbce00c3280 0x7fbce00c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 -- 172.21.15.17:0/2148791502 >> v1:172.21.15.163:6789/0 conn(0x7fbce01551e0 legacy=0x7fbce01555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.148+0000 7fbce541f640 1 -- 172.21.15.17:0/2148791502 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce01590a0 con 0x7fbce006d9f0 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbce541f640 1 --2- 172.21.15.17:0/2148791502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce006d9f0 0x7fbce00a7cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbce00721a0 tx=0x7fbcdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4739d7162fc9be04 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbcd7fff640 1 -- 172.21.15.17:0/2148791502 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc035020 con 0x7fbce006d9f0 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbcd7fff640 1 -- 172.21.15.17:0/2148791502 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcdc031870 con 0x7fbce006d9f0 2024-03-06T07:20:42.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbcd7fff640 1 -- 172.21.15.17:0/2148791502 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc031c40 con 0x7fbce006d9f0 2024-03-06T07:20:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbce7683640 1 -- 172.21.15.17:0/2148791502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce006d9f0 msgr2=0x7fbce00a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.149+0000 7fbce7683640 1 --2- 172.21.15.17:0/2148791502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce006d9f0 0x7fbce00a7cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbce00721a0 tx=0x7fbcdc02b910 comp rx=0 tx=0).stop 2024-03-06T07:20:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 -- 172.21.15.17:0/2148791502 shutdown_connections 2024-03-06T07:20:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 --2- 172.21.15.17:0/2148791502 >> v2:172.21.15.163:3300/0 conn(0x7fbce00c3280 0x7fbce00c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 --2- 172.21.15.17:0/2148791502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce006d9f0 0x7fbce00a7cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 -- 172.21.15.17:0/2148791502 >> 172.21.15.17:0/2148791502 conn(0x7fbce00aa630 msgr2=0x7fbce00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 -- 172.21.15.17:0/2148791502 shutdown_connections 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.150+0000 7fbce7683640 1 -- 172.21.15.17:0/2148791502 wait complete. 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 Processor -- start 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 -- start start 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbce006d9f0 0x7fbce0157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbce00c3280 0x7fbce014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce006df50 con 0x7fbce01551e0 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce006ddd0 con 0x7fbce006d9f0 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce7683640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00a8210 con 0x7fbce00c3280 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.151+0000 7fbce5c20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43686/0 (socket says 172.21.15.17:43686) 2024-03-06T07:20:42.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 -- 172.21.15.17:0/2001118518 learned_addr learned my addr 172.21.15.17:0/2001118518 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce4c1e640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbce00c3280 0x7fbce014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce541f640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbce006d9f0 0x7fbce0157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 -- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbce00c3280 msgr2=0x7fbce014eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbce00c3280 0x7fbce014eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 -- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbce006d9f0 msgr2=0x7fbce0157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbce006d9f0 0x7fbce0157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 -- 172.21.15.17:0/2001118518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce01512c0 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce5c20640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c760 tx=0x7fbcd800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003b70 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8003d30 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce0070210 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8005310 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.152+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce01ee010 con 0x7fbce01551e0 2024-03-06T07:20:42.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.153+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce004f330 con 0x7fbce01551e0 2024-03-06T07:20:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.154+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcd80054b0 con 0x7fbce01551e0 2024-03-06T07:20:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.154+0000 7fbcd5ffb640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.154+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbcd808a1e0 con 0x7fbce01551e0 2024-03-06T07:20:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.154+0000 7fbce541f640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.155+0000 7fbce541f640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 0x7fbcc406d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc036000 tx=0x7fbcdc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.159+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcd8058760 con 0x7fbce01551e0 2024-03-06T07:20:42.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.340+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbce0073240 con 0x7fbce01551e0 2024-03-06T07:20:42.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.342+0000 7fbcd5ffb640 1 -- 172.21.15.17:0/2001118518 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fbcd805c800 con 0x7fbce01551e0 2024-03-06T07:20:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:42.344 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 msgr2=0x7fbcc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 0x7fbcc406d9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc036000 tx=0x7fbcdc03c040 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 msgr2=0x7fbce014f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c760 tx=0x7fbcd800cc20 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 shutdown_connections 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbcc406b500 0x7fbcc406d9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbce01551e0 0x7fbce014f000 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbce00c3280 0x7fbce014eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 --2- 172.21.15.17:0/2001118518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbce006d9f0 0x7fbce0157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 >> 172.21.15.17:0/2001118518 conn(0x7fbce00aa630 msgr2=0x7fbce00c6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 shutdown_connections 2024-03-06T07:20:42.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.344+0000 7fbce7683640 1 -- 172.21.15.17:0/2001118518 wait complete. 2024-03-06T07:20:42.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:42.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 Processor -- start 2024-03-06T07:20:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 -- start start 2024-03-06T07:20:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f387c155e10 0x7f387c06c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f387c0c88c0 0x7f387c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c152ba0 con 0x7f387c1527c0 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0c9390 con 0x7f387c155e10 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.460+0000 7f3884313640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c0c8dd0 con 0x7f387c0c88c0 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f387c155e10 0x7f387c06c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f387c155e10 0x7f387c06c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43696/0 (socket says 172.21.15.17:43696) 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 -- 172.21.15.17:0/2512998389 learned_addr learned my addr 172.21.15.17:0/2512998389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38828b0640 1 --2- 172.21.15.17:0/2512998389 >> v2:172.21.15.163:3300/0 conn(0x7f387c0c88c0 0x7f387c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 -- 172.21.15.17:0/2512998389 >> v2:172.21.15.163:3300/0 conn(0x7f387c0c88c0 msgr2=0x7f387c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 --2- 172.21.15.17:0/2512998389 >> v2:172.21.15.163:3300/0 conn(0x7f387c0c88c0 0x7f387c06c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 -- 172.21.15.17:0/2512998389 >> v1:172.21.15.163:6789/0 conn(0x7f387c1527c0 legacy=0x7f387c0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.461+0000 7f38818ae640 1 -- 172.21.15.17:0/2512998389 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0c90b0 con 0x7f387c155e10 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f38818ae640 1 --2- 172.21.15.17:0/2512998389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c155e10 0x7f387c06c020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f387c0c63e0 tx=0x7f386c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6066f0d28e883b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f38810ad640 1 -- 172.21.15.17:0/2512998389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c035020 con 0x7f387c155e10 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f38810ad640 1 -- 172.21.15.17:0/2512998389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c031870 con 0x7f387c155e10 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f38810ad640 1 -- 172.21.15.17:0/2512998389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c031c40 con 0x7f387c155e10 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 -- 172.21.15.17:0/2512998389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c155e10 msgr2=0x7f387c06c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 --2- 172.21.15.17:0/2512998389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c155e10 0x7f387c06c020 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f387c0c63e0 tx=0x7f386c02b910 comp rx=0 tx=0).stop 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 -- 172.21.15.17:0/2512998389 shutdown_connections 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 --2- 172.21.15.17:0/2512998389 >> v2:172.21.15.163:3300/0 conn(0x7f387c0c88c0 0x7f387c06c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 --2- 172.21.15.17:0/2512998389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c155e10 0x7f387c06c020 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 -- 172.21.15.17:0/2512998389 >> 172.21.15.17:0/2512998389 conn(0x7f387c0ac550 msgr2=0x7f387c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 -- 172.21.15.17:0/2512998389 shutdown_connections 2024-03-06T07:20:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.462+0000 7f3884313640 1 -- 172.21.15.17:0/2512998389 wait complete. 2024-03-06T07:20:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.463+0000 7f3884313640 1 Processor -- start 2024-03-06T07:20:42.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 -- start start 2024-03-06T07:20:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c0c88c0 0x7f387c1ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c155e10 0x7f387c1ee690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c06d590 con 0x7f387c1527c0 2024-03-06T07:20:42.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c155610 con 0x7f387c0c88c0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f3884313640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c7ae0 con 0x7f387c155e10 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f38818ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f38818ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43708/0 (socket says 172.21.15.17:43708) 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f38818ae640 1 -- 172.21.15.17:0/1899951381 learned_addr learned my addr 172.21.15.17:0/1899951381 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f38820af640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c0c88c0 0x7f387c1ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.464+0000 7f38828b0640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c155e10 0x7f387c1ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 -- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c155e10 msgr2=0x7f387c1ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c155e10 0x7f387c1ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 -- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c0c88c0 msgr2=0x7f387c1ec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c0c88c0 0x7f387c1ec320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 -- 172.21.15.17:0/1899951381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0aa4e0 con 0x7f387c1527c0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f38818ae640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c035040 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386c030d00 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0456a0 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c0c4460 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.465+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c0c2620 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.466+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c04f330 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.466+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f386c038020 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.467+0000 7f3872ffd640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.467+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f386c0ac690 con 0x7f387c1527c0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.467+0000 7f38820af640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.467+0000 7f38820af640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 0x7f386006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f387c1e8cf0 tx=0x7f3878009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.471+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f386c07ac10 con 0x7f387c1527c0 2024-03-06T07:20:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.649+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f387c0b0900 con 0x7f387c1527c0 2024-03-06T07:20:42.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.649+0000 7f3872ffd640 1 -- 172.21.15.17:0/1899951381 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f386c07ecb0 con 0x7f387c1527c0 2024-03-06T07:20:42.652 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 msgr2=0x7f386006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 0x7f386006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f387c1e8cf0 tx=0x7f3878009040 comp rx=0 tx=0).stop 2024-03-06T07:20:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 msgr2=0x7f387c0c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 shutdown_connections 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386006b4d0 0x7f386006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c155e10 0x7f387c1ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c1527c0 0x7f387c0c2e20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 --2- 172.21.15.17:0/1899951381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c0c88c0 0x7f387c1ec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 >> 172.21.15.17:0/1899951381 conn(0x7f387c0ac550 msgr2=0x7f387c1518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 shutdown_connections 2024-03-06T07:20:42.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.651+0000 7f3884313640 1 -- 172.21.15.17:0/1899951381 wait complete. 2024-03-06T07:20:42.664 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.766+0000 7fda13ec9640 1 Processor -- start 2024-03-06T07:20:42.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 -- start start 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 0x7fda0c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda0c155530 0x7fda0c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0c159710 con 0x7fda0c155530 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0c0c68b0 con 0x7fda0c1529f0 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.767+0000 7fda13ec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c152dd0 con 0x7fda0c0c5370 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda0c155530 0x7fda0c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fda0c155530 0x7fda0c1559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43718/0 (socket says 172.21.15.17:43718) 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 -- 172.21.15.17:0/1342241644 learned_addr learned my addr 172.21.15.17:0/1342241644 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda11c65640 1 --2- 172.21.15.17:0/1342241644 >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 0x7fda0c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda11464640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 -- 172.21.15.17:0/1342241644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 msgr2=0x7fda0c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 -- 172.21.15.17:0/1342241644 >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 msgr2=0x7fda0c0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 --2- 172.21.15.17:0/1342241644 >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 0x7fda0c0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda12466640 1 -- 172.21.15.17:0/1342241644 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c158b00 con 0x7fda0c155530 2024-03-06T07:20:42.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.768+0000 7fda11c65640 1 --2- 172.21.15.17:0/1342241644 >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 0x7fda0c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda12466640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c155530 0x7fda0c1559c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0cba90 tx=0x7fda0402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80446489133979b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda10c63640 1 -- 172.21.15.17:0/1342241644 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0402c9e0 con 0x7fda0c155530 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda10c63640 1 -- 172.21.15.17:0/1342241644 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0402cb80 con 0x7fda0c155530 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda10c63640 1 -- 172.21.15.17:0/1342241644 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040343f0 con 0x7fda0c155530 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 -- 172.21.15.17:0/1342241644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c155530 msgr2=0x7fda0c1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c155530 0x7fda0c1559c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0cba90 tx=0x7fda0402be40 comp rx=0 tx=0).stop 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 -- 172.21.15.17:0/1342241644 shutdown_connections 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c155530 0x7fda0c1559c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 --2- 172.21.15.17:0/1342241644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 --2- 172.21.15.17:0/1342241644 >> v2:172.21.15.163:3300/0 conn(0x7fda0c1529f0 0x7fda0c0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.769+0000 7fda13ec9640 1 -- 172.21.15.17:0/1342241644 >> 172.21.15.17:0/1342241644 conn(0x7fda0c0ac580 msgr2=0x7fda0c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.770+0000 7fda13ec9640 1 -- 172.21.15.17:0/1342241644 shutdown_connections 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.770+0000 7fda13ec9640 1 -- 172.21.15.17:0/1342241644 wait complete. 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 Processor -- start 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 -- start start 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c155530 0x7fda0c1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c1593a0 con 0x7fda0c1529f0 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c1590c0 con 0x7fda0c155530 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda13ec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c158740 con 0x7fda0c0c5370 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda11464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda11464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43730/0 (socket says 172.21.15.17:43730) 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda11464640 1 -- 172.21.15.17:0/2887461761 learned_addr learned my addr 172.21.15.17:0/2887461761 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.771+0000 7fda11c65640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda12466640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c155530 0x7fda0c1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 -- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 msgr2=0x7fda0c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 -- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c155530 msgr2=0x7fda0c1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c155530 0x7fda0c1e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 -- 172.21.15.17:0/2887461761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c157100 con 0x7fda0c1529f0 2024-03-06T07:20:42.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda11464640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e890 tx=0x7fd9fc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc017070 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc00c940 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c0bf520 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cd10 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.772+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c0bfae0 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.773+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c04f330 con 0x7fda0c1529f0 2024-03-06T07:20:42.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.774+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9fc01a070 con 0x7fda0c1529f0 2024-03-06T07:20:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.774+0000 7fda027fc640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 0x7fd9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.774+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd9fc08a7b0 con 0x7fda0c1529f0 2024-03-06T07:20:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.774+0000 7fda11c65640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 0x7fd9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.775+0000 7fda11c65640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 0x7fd9f006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda08009560 tx=0x7fda08009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.778+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd9fc058ca0 con 0x7fda0c1529f0 2024-03-06T07:20:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.960+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda0c0c9d80 con 0x7fda0c1529f0 2024-03-06T07:20:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.960+0000 7fda027fc640 1 -- 172.21.15.17:0/2887461761 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9fc05cd40 con 0x7fda0c1529f0 2024-03-06T07:20:42.962 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 msgr2=0x7fd9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 0x7fd9f006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fda08009560 tx=0x7fda08009190 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 msgr2=0x7fda0c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e890 tx=0x7fd9fc00ed50 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 shutdown_connections 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9f006b4d0 0x7fd9f006d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c155530 0x7fda0c1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda0c1529f0 0x7fda0c14eb00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 --2- 172.21.15.17:0/2887461761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c0c5370 0x7fda0c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 >> 172.21.15.17:0/2887461761 conn(0x7fda0c0ac580 msgr2=0x7fda0c06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 shutdown_connections 2024-03-06T07:20:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:42.962+0000 7fda13ec9640 1 -- 172.21.15.17:0/2887461761 wait complete. 2024-03-06T07:20:42.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.077+0000 7f5c72793640 1 Processor -- start 2024-03-06T07:20:43.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.077+0000 7f5c72793640 1 -- start start 2024-03-06T07:20:43.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c72793640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c6c1528d0 0x7f5c6c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c72793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 0x7f5c6c155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c72793640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c152cb0 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c72793640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0c38b0 con 0x7f5c6c1528d0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c72793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1594f0 con 0x7f5c6c072a10 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c70d30640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5c6c0c67f0 0x7f5c6c155760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41328/0 (socket says 172.21.15.17:41328) 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c70d30640 1 -- 172.21.15.17:0/3818997761 learned_addr learned my addr 172.21.15.17:0/3818997761 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6b7fe640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 0x7f5c6c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3818997761 >> v2:172.21.15.163:3300/0 conn(0x7f5c6c1528d0 0x7f5c6c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6affd640 1 -- 172.21.15.17:0/3818997761 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399758681 0 0) 0x7f5c6c152cb0 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6affd640 1 -- 172.21.15.17:0/3818997761 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c54003520 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6bfff640 1 -- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 msgr2=0x7f5c6c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 0x7f5c6c155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6bfff640 1 -- 172.21.15.17:0/3818997761 >> v1:172.21.15.17:6789/0 conn(0x7f5c6c0c67f0 legacy=0x7f5c6c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6bfff640 1 -- 172.21.15.17:0/3818997761 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0a7f70 con 0x7f5c6c1528d0 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.078+0000 7f5c6b7fe640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 0x7f5c6c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.079+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c1528d0 0x7f5c6c0724d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c600097a0 tx=0x7f5c6002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866002d9793874c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.079+0000 7f5c6affd640 1 -- 172.21.15.17:0/3818997761 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60034070 con 0x7f5c6c1528d0 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.079+0000 7f5c6affd640 1 -- 172.21.15.17:0/3818997761 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60030610 con 0x7f5c6c1528d0 2024-03-06T07:20:43.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.079+0000 7f5c6affd640 1 -- 172.21.15.17:0/3818997761 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c600309c0 con 0x7f5c6c1528d0 2024-03-06T07:20:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 -- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c1528d0 msgr2=0x7f5c6c0724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c1528d0 0x7f5c6c0724d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5c600097a0 tx=0x7f5c6002b960 comp rx=0 tx=0).stop 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 -- 172.21.15.17:0/3818997761 shutdown_connections 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c072a10 0x7f5c6c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 --2- 172.21.15.17:0/3818997761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c1528d0 0x7f5c6c0724d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 -- 172.21.15.17:0/3818997761 >> 172.21.15.17:0/3818997761 conn(0x7f5c6c0aa6a0 msgr2=0x7f5c6c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 -- 172.21.15.17:0/3818997761 shutdown_connections 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.080+0000 7f5c72793640 1 -- 172.21.15.17:0/3818997761 wait complete. 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.081+0000 7f5c72793640 1 Processor -- start 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.081+0000 7f5c72793640 1 -- start start 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c072a10 0x7f5c6c0ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1528d0 0x7f5c6c1e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c1591a0 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0a8250 con 0x7f5c6c072a10 2024-03-06T07:20:43.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0c4ef0 con 0x7f5c6c1528d0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43734/0 (socket says 172.21.15.17:43734) 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 -- 172.21.15.17:0/3473926155 learned_addr learned my addr 172.21.15.17:0/3473926155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c072a10 0x7f5c6c0ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c70d30640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1528d0 0x7f5c6c1e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 -- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1528d0 msgr2=0x7f5c6c1e7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1528d0 0x7f5c6c1e7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 -- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c072a10 msgr2=0x7f5c6c0ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c072a10 0x7f5c6c0ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 -- 172.21.15.17:0/3473926155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c1ee800 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c6b7fe640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00ebe0 tx=0x7f5c5c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c017070 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c00cc30 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c1eeae0 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c013470 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.082+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c14bb60 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.084+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c04f330 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.084+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c5c01a070 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.084+0000 7f5c68ff9640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 0x7f5c5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.084+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c5c08a6f0 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.084+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 0x7f5c5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.085+0000 7f5c6bfff640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 0x7f5c5806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c60036000 tx=0x7f5c6003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.088+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5c5c058c70 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.266+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c6c0c7210 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.267+0000 7f5c68ff9640 1 -- 172.21.15.17:0/3473926155 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c5c05cd10 con 0x7f5c6c0c67f0 2024-03-06T07:20:43.268 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 msgr2=0x7f5c5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 0x7f5c5806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c60036000 tx=0x7f5c6003c040 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 msgr2=0x7f5c6c154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00ebe0 tx=0x7f5c5c00c520 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 shutdown_connections 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c5806b4d0 0x7f5c5806d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1528d0 0x7f5c6c1e7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c6c0c67f0 0x7f5c6c154840 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 --2- 172.21.15.17:0/3473926155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c072a10 0x7f5c6c0ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 >> 172.21.15.17:0/3473926155 conn(0x7f5c6c0aa6a0 msgr2=0x7f5c6c0c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 shutdown_connections 2024-03-06T07:20:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.269+0000 7f5c72793640 1 -- 172.21.15.17:0/3473926155 wait complete. 2024-03-06T07:20:43.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:43.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.384+0000 7fcda75ac640 1 Processor -- start 2024-03-06T07:20:43.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.384+0000 7fcda75ac640 1 -- start start 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda75ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcda006c340 0x7fcda0152860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda75ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda00a6cc0 0x7fcda014f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda75ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda00c7060 con 0x7fcda014f540 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda75ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda00c2b10 con 0x7fcda006c340 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda75ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda00c2440 con 0x7fcda00a6cc0 2024-03-06T07:20:43.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcda006c340 0x7fcda0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcda006c340 0x7fcda0152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43742/0 (socket says 172.21.15.17:43742) 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 -- 172.21.15.17:0/2368230546 learned_addr learned my addr 172.21.15.17:0/2368230546 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda4b47640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda00a6cc0 0x7fcda014f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 -- 172.21.15.17:0/2368230546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda00a6cc0 msgr2=0x7fcda014f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda00a6cc0 0x7fcda014f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 -- 172.21.15.17:0/2368230546 >> v1:172.21.15.163:6789/0 conn(0x7fcda014f540 legacy=0x7fcda0155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.385+0000 7fcda5348640 1 -- 172.21.15.17:0/2368230546 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00c7340 con 0x7fcda006c340 2024-03-06T07:20:43.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.386+0000 7fcda5348640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0152860 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009e50 tx=0x7fcd9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b01ca28b8f9d565 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.386+0000 7fcd9ffff640 1 -- 172.21.15.17:0/2368230546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9002c9e0 con 0x7fcda006c340 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.386+0000 7fcd9ffff640 1 -- 172.21.15.17:0/2368230546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9002cb80 con 0x7fcda006c340 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.386+0000 7fcd9ffff640 1 -- 172.21.15.17:0/2368230546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd900343f0 con 0x7fcda006c340 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 -- 172.21.15.17:0/2368230546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 msgr2=0x7fcda0152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0152860 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009e50 tx=0x7fcd9002be40 comp rx=0 tx=0).stop 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 -- 172.21.15.17:0/2368230546 shutdown_connections 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda00a6cc0 0x7fcda014f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2368230546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0152860 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 -- 172.21.15.17:0/2368230546 >> 172.21.15.17:0/2368230546 conn(0x7fcda00a9540 msgr2=0x7fcda00a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 -- 172.21.15.17:0/2368230546 shutdown_connections 2024-03-06T07:20:43.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.387+0000 7fcda75ac640 1 -- 172.21.15.17:0/2368230546 wait complete. 2024-03-06T07:20:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.388+0000 7fcda75ac640 1 Processor -- start 2024-03-06T07:20:43.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.388+0000 7fcda75ac640 1 -- start start 2024-03-06T07:20:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcda00a6cc0 0x7fcda00a8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda014f540 0x7fcda0146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda006c8a0 con 0x7fcda006c340 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda006c720 con 0x7fcda00a6cc0 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda75ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda0155a80 con 0x7fcda014f540 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43750/0 (socket says 172.21.15.17:43750) 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 -- 172.21.15.17:0/2644065221 learned_addr learned my addr 172.21.15.17:0/2644065221 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda4b47640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcda00a6cc0 0x7fcda00a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5b49640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda014f540 0x7fcda0146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 -- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda014f540 msgr2=0x7fcda0146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda014f540 0x7fcda0146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 -- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcda00a6cc0 msgr2=0x7fcda00a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcda00a6cc0 0x7fcda00a8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 -- 172.21.15.17:0/2644065221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0153c00 con 0x7fcda006c340 2024-03-06T07:20:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.389+0000 7fcda5348640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd90036040 tx=0x7fcd90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.390+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90043020 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.390+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9003e450 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.390+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda0153ee0 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.390+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9003e840 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.390+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda00a8980 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.391+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda004f330 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.391+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd90046020 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.392+0000 7fcd9dffb640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.392+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcd900acc50 con 0x7fcda006c340 2024-03-06T07:20:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.392+0000 7fcda4b47640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.392+0000 7fcda4b47640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcda00c1a00 tx=0x7fcd94009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.396+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcd9004d030 con 0x7fcda006c340 2024-03-06T07:20:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.576+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcda0149510 con 0x7fcda006c340 2024-03-06T07:20:43.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.577+0000 7fcd9dffb640 1 -- 172.21.15.17:0/2644065221 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcd9007b1d0 con 0x7fcda006c340 2024-03-06T07:20:43.579 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 msgr2=0x7fcd8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcda00c1a00 tx=0x7fcd94009190 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 msgr2=0x7fcda0154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcd90036040 tx=0x7fcd90004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 shutdown_connections 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcd8c06b4d0 0x7fcd8c06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcda014f540 0x7fcda0146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcda00a6cc0 0x7fcda00a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 --2- 172.21.15.17:0/2644065221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcda006c340 0x7fcda0154820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 >> 172.21.15.17:0/2644065221 conn(0x7fcda00a9540 msgr2=0x7fcda006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 shutdown_connections 2024-03-06T07:20:43.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.579+0000 7fcda75ac640 1 -- 172.21.15.17:0/2644065221 wait complete. 2024-03-06T07:20:43.604 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. 2024-03-06T07:20:43.604 INFO:tasks.cephfs_test_runner:test_join_fs_drop (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:20:43.605 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.705+0000 7f3222d8a640 1 Processor -- start 2024-03-06T07:20:43.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.705+0000 7f3222d8a640 1 -- start start 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3222d8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f321c0c3710 0x7f321c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3222d8a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c158800 con 0x7f321c06d9f0 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3222d8a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c158ae0 con 0x7f321c0c3280 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3222d8a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c158dc0 con 0x7f321c0c3710 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f321c0c3710 0x7f321c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f321c0c3710 0x7f321c1558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43758/0 (socket says 172.21.15.17:43758) 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3220b26640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f321c06d9f0 0x7f321c0a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41342/0 (socket says 172.21.15.17:41342) 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 -- 172.21.15.17:0/614929027 learned_addr learned my addr 172.21.15.17:0/614929027 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:43.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 -- 172.21.15.17:0/614929027 >> v1:172.21.15.163:6789/0 conn(0x7f321c0c3280 legacy=0x7f321c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 -- 172.21.15.17:0/614929027 >> v1:172.21.15.17:6789/0 conn(0x7f321c06d9f0 legacy=0x7f321c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f321b7fe640 1 -- 172.21.15.17:0/614929027 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260441841 0 0) 0x7f321c158ae0 con 0x7f321c0c3280 2024-03-06T07:20:43.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.706+0000 7f3221327640 1 -- 172.21.15.17:0/614929027 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c1590a0 con 0x7f321c0c3710 2024-03-06T07:20:43.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.707+0000 7f3221327640 1 --2- 172.21.15.17:0/614929027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c0c3710 0x7f321c1558f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e50 tx=0x7f321402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122232803dbaaee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.707+0000 7f321b7fe640 1 -- 172.21.15.17:0/614929027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321402cb20 con 0x7f321c0c3710 2024-03-06T07:20:43.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.707+0000 7f321b7fe640 1 -- 172.21.15.17:0/614929027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321402ccc0 con 0x7f321c0c3710 2024-03-06T07:20:43.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.707+0000 7f321b7fe640 1 -- 172.21.15.17:0/614929027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214034610 con 0x7f321c0c3710 2024-03-06T07:20:43.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 -- 172.21.15.17:0/614929027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c0c3710 msgr2=0x7f321c1558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 --2- 172.21.15.17:0/614929027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c0c3710 0x7f321c1558f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3214009e50 tx=0x7f321402bf80 comp rx=0 tx=0).stop 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 -- 172.21.15.17:0/614929027 shutdown_connections 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 --2- 172.21.15.17:0/614929027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c0c3710 0x7f321c1558f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 -- 172.21.15.17:0/614929027 >> 172.21.15.17:0/614929027 conn(0x7f321c0aa630 msgr2=0x7f321c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 -- 172.21.15.17:0/614929027 shutdown_connections 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.708+0000 7f3222d8a640 1 -- 172.21.15.17:0/614929027 wait complete. 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.709+0000 7f3222d8a640 1 Processor -- start 2024-03-06T07:20:43.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.709+0000 7f3222d8a640 1 -- start start 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f321c0c3280 0x7f321c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f321c0c3710 0x7f321c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c06ddd0 con 0x7f321c06d9f0 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c155e30 con 0x7f321c0c3710 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3222d8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c06df50 con 0x7f321c0c3280 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43766/0 (socket says 172.21.15.17:43766) 2024-03-06T07:20:43.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 -- 172.21.15.17:0/37280468 learned_addr learned my addr 172.21.15.17:0/37280468 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3221327640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f321c0c3710 0x7f321c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f321bfff640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f321c0c3280 0x7f321c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 -- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f321c0c3280 msgr2=0x7f321c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f321c0c3280 0x7f321c06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 -- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f321c0c3710 msgr2=0x7f321c14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f321c0c3710 0x7f321c14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 -- 172.21.15.17:0/37280468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0ab6c0 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f3220b26640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f321000c400 tx=0x7f321000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.710+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32100074e0 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.711+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3210007680 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.711+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210004a00 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.711+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0ab9a0 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.711+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c0acdb0 con 0x7f321c06d9f0 2024-03-06T07:20:43.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.712+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c04f330 con 0x7f321c06d9f0 2024-03-06T07:20:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.712+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f321001a020 con 0x7f321c06d9f0 2024-03-06T07:20:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.712+0000 7f32197fa640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 0x7f320006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:43.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.713+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3210089be0 con 0x7f321c06d9f0 2024-03-06T07:20:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.713+0000 7f321bfff640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 0x7f320006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.713+0000 7f321bfff640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 0x7f320006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f321c1e2540 tx=0x7f3208009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:43.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.717+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f32100580e0 con 0x7f321c06d9f0 2024-03-06T07:20:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.899+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f321c073240 con 0x7f321c06d9f0 2024-03-06T07:20:43.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.901+0000 7f32197fa640 1 -- 172.21.15.17:0/37280468 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f321005c180 con 0x7f321c06d9f0 2024-03-06T07:20:43.902 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:43.902 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:43.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.902+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 msgr2=0x7f320006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.902+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 0x7f320006d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f321c1e2540 tx=0x7f3208009040 comp rx=0 tx=0).stop 2024-03-06T07:20:43.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.902+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 msgr2=0x7f321c0ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.902+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f321000c400 tx=0x7f321000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 shutdown_connections 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f320006b4d0 0x7f320006d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f321c0c3710 0x7f321c14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f321c0c3280 0x7f321c06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 --2- 172.21.15.17:0/37280468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f321c06d9f0 0x7f321c0ac870 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 >> 172.21.15.17:0/37280468 conn(0x7f321c0aa630 msgr2=0x7f321c0c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 shutdown_connections 2024-03-06T07:20:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:43.903+0000 7f3222d8a640 1 -- 172.21.15.17:0/37280468 wait complete. 2024-03-06T07:20:43.916 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:43.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:44.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.019+0000 7f526cc9a640 1 Processor -- start 2024-03-06T07:20:44.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.019+0000 7f526cc9a640 1 -- start start 2024-03-06T07:20:44.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526cc9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f5268155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526cc9a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52680c2aa0 0x7f526806bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526cc9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526806c110 con 0x7f52680c0470 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526cc9a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526806b740 con 0x7f52680c2aa0 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526cc9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680670c0 con 0x7f52680c26c0 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52680c2aa0 0x7f526806bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52680c2aa0 0x7f526806bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43770/0 (socket says 172.21.15.17:43770) 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5266d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52680c0470 0x7f5268152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41348/0 (socket says 172.21.15.17:41348) 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 -- 172.21.15.17:0/347200834 learned_addr learned my addr 172.21.15.17:0/347200834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526659c640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f5268155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 -- 172.21.15.17:0/347200834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 msgr2=0x7f5268155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f5268155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 -- 172.21.15.17:0/347200834 >> v1:172.21.15.17:6789/0 conn(0x7f52680c0470 legacy=0x7f5268152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f5265d9b640 1 -- 172.21.15.17:0/347200834 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526806b0f0 con 0x7f52680c2aa0 2024-03-06T07:20:44.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.020+0000 7f526559a640 1 -- 172.21.15.17:0/347200834 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627397441 0 0) 0x7f526806c110 con 0x7f52680c0470 2024-03-06T07:20:44.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.021+0000 7f5265d9b640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526806bbd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f525c009870 tx=0x7f525c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0aa38662c4db9c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.021+0000 7f526559a640 1 -- 172.21.15.17:0/347200834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0317c0 con 0x7f52680c2aa0 2024-03-06T07:20:44.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.021+0000 7f526559a640 1 -- 172.21.15.17:0/347200834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c037070 con 0x7f52680c2aa0 2024-03-06T07:20:44.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.021+0000 7f526559a640 1 -- 172.21.15.17:0/347200834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c031b80 con 0x7f52680c2aa0 2024-03-06T07:20:44.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 -- 172.21.15.17:0/347200834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 msgr2=0x7f526806bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526806bbd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f525c009870 tx=0x7f525c02b860 comp rx=0 tx=0).stop 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 -- 172.21.15.17:0/347200834 shutdown_connections 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526806bbd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 --2- 172.21.15.17:0/347200834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f5268155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 -- 172.21.15.17:0/347200834 >> 172.21.15.17:0/347200834 conn(0x7f52680c3b40 msgr2=0x7f52680c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 -- 172.21.15.17:0/347200834 shutdown_connections 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.022+0000 7f526cc9a640 1 -- 172.21.15.17:0/347200834 wait complete. 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.023+0000 7f526cc9a640 1 Processor -- start 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.023+0000 7f526cc9a640 1 -- start start 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52680c0470 0x7f5268151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f52680ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c1f30 con 0x7f52680c2aa0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c0900 con 0x7f52680c0470 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52680c20b0 con 0x7f52680c26c0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43778/0 (socket says 172.21.15.17:43778) 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 -- 172.21.15.17:0/1041317024 learned_addr learned my addr 172.21.15.17:0/1041317024 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526659c640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52680c0470 0x7f5268151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5265d9b640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f52680ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 -- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 msgr2=0x7f52680ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f52680ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 -- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52680c0470 msgr2=0x7f5268151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52680c0470 0x7f5268151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 -- 172.21.15.17:0/1041317024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680aa5b0 con 0x7f52680c2aa0 2024-03-06T07:20:44.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5266d9d640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f526000c7d0 tx=0x7f526000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260018070 con 0x7f52680c2aa0 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000a640 con 0x7f52680c2aa0 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52680abb40 con 0x7f52680c2aa0 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000aa70 con 0x7f52680c2aa0 2024-03-06T07:20:44.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.024+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52680accc0 con 0x7f52680c2aa0 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.025+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52680ab950 con 0x7f52680c2aa0 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.026+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f526001b070 con 0x7f52680c2aa0 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.026+0000 7f5256ffd640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 0x7f524806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.026+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5260089a20 con 0x7f52680c2aa0 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.026+0000 7f526659c640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 0x7f524806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.027+0000 7f526659c640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 0x7f524806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5268153e40 tx=0x7f5258009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.030+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5260057fa0 con 0x7f52680c2aa0 2024-03-06T07:20:44.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.209+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f526804f330 con 0x7f52680c2aa0 2024-03-06T07:20:44.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.210+0000 7f5256ffd640 1 -- 172.21.15.17:0/1041317024 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f526005c040 con 0x7f52680c2aa0 2024-03-06T07:20:44.212 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:44.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 msgr2=0x7f524806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 0x7f524806d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5268153e40 tx=0x7f5258009190 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 msgr2=0x7f526813a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f526000c7d0 tx=0x7f526000cc90 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 shutdown_connections 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f524806b4d0 0x7f524806d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52680c2aa0 0x7f526813a720 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52680c26c0 0x7f52680ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 --2- 172.21.15.17:0/1041317024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52680c0470 0x7f5268151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 >> 172.21.15.17:0/1041317024 conn(0x7f52680c3b40 msgr2=0x7f526806da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 shutdown_connections 2024-03-06T07:20:44.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.212+0000 7f526cc9a640 1 -- 172.21.15.17:0/1041317024 wait complete. 2024-03-06T07:20:44.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:44.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.327+0000 7f014d38c640 1 Processor -- start 2024-03-06T07:20:44.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.327+0000 7f014d38c640 1 -- start start 2024-03-06T07:20:44.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f014d38c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01480c8ff0 0x7f01480c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f014d38c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01480b04b0 con 0x7f014806bff0 2024-03-06T07:20:44.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f014d38c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014806d210 con 0x7f01480b00d0 2024-03-06T07:20:44.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f014d38c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0148156160 con 0x7f01480c8ff0 2024-03-06T07:20:44.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f01467fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f014806bff0 0x7f01480c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41356/0 (socket says 172.21.15.17:41356) 2024-03-06T07:20:44.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f01467fc640 1 -- 172.21.15.17:0/1030293091 learned_addr learned my addr 172.21.15.17:0/1030293091 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:44.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.328+0000 7f01477fe640 1 --2- 172.21.15.17:0/1030293091 >> v2:172.21.15.17:3300/0 conn(0x7f01480c8ff0 0x7f01480c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f01477fe640 1 -- 172.21.15.17:0/1030293091 >> v1:172.21.15.163:6789/0 conn(0x7f01480b00d0 legacy=0x7f014806b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f01477fe640 1 -- 172.21.15.17:0/1030293091 >> v1:172.21.15.17:6789/0 conn(0x7f014806bff0 legacy=0x7f01480c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f01477fe640 1 -- 172.21.15.17:0/1030293091 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0148155e80 con 0x7f01480c8ff0 2024-03-06T07:20:44.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f01477fe640 1 --2- 172.21.15.17:0/1030293091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01480c7920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f01481562e0 tx=0x7f013802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913a9af5783ecbfa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f0145ffb640 1 -- 172.21.15.17:0/1030293091 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138039070 con 0x7f01480c8ff0 2024-03-06T07:20:44.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f0145ffb640 1 -- 172.21.15.17:0/1030293091 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013802cce0 con 0x7f01480c8ff0 2024-03-06T07:20:44.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.329+0000 7f0145ffb640 1 -- 172.21.15.17:0/1030293091 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138034660 con 0x7f01480c8ff0 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 -- 172.21.15.17:0/1030293091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 msgr2=0x7f01480c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 --2- 172.21.15.17:0/1030293091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01480c7920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f01481562e0 tx=0x7f013802bf30 comp rx=0 tx=0).stop 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 -- 172.21.15.17:0/1030293091 shutdown_connections 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 --2- 172.21.15.17:0/1030293091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01480c7920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 -- 172.21.15.17:0/1030293091 >> 172.21.15.17:0/1030293091 conn(0x7f01480ac810 msgr2=0x7f01480acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 -- 172.21.15.17:0/1030293091 shutdown_connections 2024-03-06T07:20:44.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.330+0000 7f014d38c640 1 -- 172.21.15.17:0/1030293091 wait complete. 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.331+0000 7f014d38c640 1 Processor -- start 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.331+0000 7f014d38c640 1 -- start start 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014806bff0 0x7f01480c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01480b00d0 0x7f01480c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0148152c50 con 0x7f01480c8ff0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01480c5710 con 0x7f01480b00d0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f014d38c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01480c5ba0 con 0x7f014806bff0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43794/0 (socket says 172.21.15.17:43794) 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 -- 172.21.15.17:0/3058700617 learned_addr learned my addr 172.21.15.17:0/3058700617 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01467fc640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01480b00d0 0x7f01480c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f0146ffd640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014806bff0 0x7f01480c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 -- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014806bff0 msgr2=0x7f01480c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014806bff0 0x7f01480c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 -- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01480b00d0 msgr2=0x7f01480c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01480b00d0 0x7f01480c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 -- 172.21.15.17:0/3058700617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01481eeaa0 con 0x7f01480c8ff0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.332+0000 7f01477fe640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f013802c540 tx=0x7f0138004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.333+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138039050 con 0x7f01480c8ff0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.333+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0138045070 con 0x7f01480c8ff0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.333+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01480c49c0 con 0x7f01480c8ff0 2024-03-06T07:20:44.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.333+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013803e830 con 0x7f01480c8ff0 2024-03-06T07:20:44.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.333+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01481e2110 con 0x7f01480c8ff0 2024-03-06T07:20:44.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.334+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014804f330 con 0x7f01480c8ff0 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.334+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f013803e9d0 con 0x7f01480c8ff0 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.334+0000 7f013f7fe640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 0x7f012806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.334+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f01380ad200 con 0x7f01480c8ff0 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.335+0000 7f0146ffd640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 0x7f012806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.335+0000 7f0146ffd640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 0x7f012806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01480aac80 tx=0x7f0140009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.339+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f013807b8a0 con 0x7f01480c8ff0 2024-03-06T07:20:44.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.520+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f014806ce50 con 0x7f01480c8ff0 2024-03-06T07:20:44.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.520+0000 7f013f7fe640 1 -- 172.21.15.17:0/3058700617 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f013807f940 con 0x7f01480c8ff0 2024-03-06T07:20:44.522 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 msgr2=0x7f012806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 0x7f012806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01480aac80 tx=0x7f0140009040 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 msgr2=0x7f01481e1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f013802c540 tx=0x7f0138004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 shutdown_connections 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f012806b4d0 0x7f012806d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01480c8ff0 0x7f01481e1bd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01480b00d0 0x7f01480c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 --2- 172.21.15.17:0/3058700617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014806bff0 0x7f01480c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 >> 172.21.15.17:0/3058700617 conn(0x7f01480ac810 msgr2=0x7f01480ab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 shutdown_connections 2024-03-06T07:20:44.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.523+0000 7f014d38c640 1 -- 172.21.15.17:0/3058700617 wait complete. 2024-03-06T07:20:44.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.638+0000 7f5cfabac640 1 Processor -- start 2024-03-06T07:20:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.638+0000 7f5cfabac640 1 -- start start 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cfabac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 0x7f5cf40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cfabac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf40c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cfabac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf4152dd0 con 0x7f5cf4155530 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cfabac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf4159710 con 0x7f5cf41529f0 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cfabac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40c68b0 con 0x7f5cf40c5370 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf3fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf40c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf9149640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5cf4155530 0x7f5cf41559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:51930/0 (socket says 172.21.15.17:51930) 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf8948640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 0x7f5cf40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf9149640 1 -- 172.21.15.17:0/3037108469 learned_addr learned my addr 172.21.15.17:0/3037108469 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf37fe640 1 -- 172.21.15.17:0/3037108469 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154298385 0 0) 0x7f5cf4152dd0 con 0x7f5cf4155530 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf37fe640 1 -- 172.21.15.17:0/3037108469 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce0003520 con 0x7f5cf4155530 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf3fff640 1 -- 172.21.15.17:0/3037108469 >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 msgr2=0x7f5cf40c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf3fff640 1 --2- 172.21.15.17:0/3037108469 >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 0x7f5cf40c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf3fff640 1 -- 172.21.15.17:0/3037108469 >> v1:172.21.15.163:6789/0 conn(0x7f5cf4155530 legacy=0x7f5cf41559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf3fff640 1 -- 172.21.15.17:0/3037108469 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4158b00 con 0x7f5cf40c5370 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.639+0000 7f5cf8948640 1 --2- 172.21.15.17:0/3037108469 >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 0x7f5cf40c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.640+0000 7f5cf3fff640 1 --2- 172.21.15.17:0/3037108469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf40c57d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097a0 tx=0x7f5ce402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714a2bd9b3e2b5d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.640+0000 7f5cf37fe640 1 -- 172.21.15.17:0/3037108469 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40315a0 con 0x7f5cf40c5370 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.640+0000 7f5cf37fe640 1 -- 172.21.15.17:0/3037108469 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce4037070 con 0x7f5cf40c5370 2024-03-06T07:20:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.640+0000 7f5cf37fe640 1 -- 172.21.15.17:0/3037108469 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4031940 con 0x7f5cf40c5370 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 -- 172.21.15.17:0/3037108469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 msgr2=0x7f5cf40c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 --2- 172.21.15.17:0/3037108469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf40c57d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097a0 tx=0x7f5ce402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 -- 172.21.15.17:0/3037108469 shutdown_connections 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 --2- 172.21.15.17:0/3037108469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf40c57d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 --2- 172.21.15.17:0/3037108469 >> v2:172.21.15.163:3300/0 conn(0x7f5cf41529f0 0x7f5cf40c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 -- 172.21.15.17:0/3037108469 >> 172.21.15.17:0/3037108469 conn(0x7f5cf40ac580 msgr2=0x7f5cf40ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 -- 172.21.15.17:0/3037108469 shutdown_connections 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.641+0000 7f5cfabac640 1 -- 172.21.15.17:0/3037108469 wait complete. 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.642+0000 7f5cfabac640 1 Processor -- start 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.642+0000 7f5cfabac640 1 -- start start 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41593a0 con 0x7f5cf4155530 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41590c0 con 0x7f5cf41529f0 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cfabac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf41560d0 con 0x7f5cf40c5370 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf9149640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf8948640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:54552/0 (socket says 172.21.15.17:54552) 2024-03-06T07:20:44.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 -- 172.21.15.17:0/4184050927 learned_addr learned my addr 172.21.15.17:0/4184050927 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf9149640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43806/0 (socket says 172.21.15.17:43806) 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 -- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 msgr2=0x7f5cf414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 -- 172.21.15.17:0/4184050927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 msgr2=0x7f5cf41ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 -- 172.21.15.17:0/4184050927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40bf3c0 con 0x7f5cf41529f0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf9149640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.643+0000 7f5cf3fff640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097a0 tx=0x7f5ce40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.644+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4032430 con 0x7f5cf41529f0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.644+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40325d0 con 0x7f5cf41529f0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.644+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40add70 con 0x7f5cf41529f0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.644+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4032980 con 0x7f5cf41529f0 2024-03-06T07:20:44.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.644+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40be070 con 0x7f5cf41529f0 2024-03-06T07:20:44.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.645+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf404f330 con 0x7f5cf41529f0 2024-03-06T07:20:44.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.645+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ce4032b20 con 0x7f5cf41529f0 2024-03-06T07:20:44.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.645+0000 7f5cf17fa640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 0x7f5cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.645+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5ce402f030 con 0x7f5cf41529f0 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.646+0000 7f5cf8948640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 0x7f5cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.646+0000 7f5cf8948640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 0x7f5cd406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009bd0 tx=0x7f5cec00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.650+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ce4040070 con 0x7f5cf41529f0 2024-03-06T07:20:44.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.833+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5cf4151cf0 con 0x7f5cf41529f0 2024-03-06T07:20:44.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.834+0000 7f5cf17fa640 1 -- 172.21.15.17:0/4184050927 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ce407a660 con 0x7f5cf41529f0 2024-03-06T07:20:44.835 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 msgr2=0x7f5cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 0x7f5cd406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009bd0 tx=0x7f5cec00b040 comp rx=0 tx=0).stop 2024-03-06T07:20:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 msgr2=0x7f5cf414f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40097a0 tx=0x7f5ce40046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 shutdown_connections 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5cd406b4d0 0x7f5cd406d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5cf4155530 0x7f5cf41ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5cf41529f0 0x7f5cf414f2b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 --2- 172.21.15.17:0/4184050927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5cf40c5370 0x7f5cf414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.835+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 >> 172.21.15.17:0/4184050927 conn(0x7f5cf40ac580 msgr2=0x7f5cf40ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.836+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 shutdown_connections 2024-03-06T07:20:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.836+0000 7f5cfabac640 1 -- 172.21.15.17:0/4184050927 wait complete. 2024-03-06T07:20:44.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:44.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.950+0000 7ffa5fcc2640 1 Processor -- start 2024-03-06T07:20:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.950+0000 7ffa5fcc2640 1 -- start start 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5fcc2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa580c5370 0x7ffa580c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5fcc2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 0x7ffa581559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5fcc2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa58152dd0 con 0x7ffa581529f0 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5fcc2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa58159710 con 0x7ffa580c5370 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5fcc2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580c68b0 con 0x7ffa58155530 2024-03-06T07:20:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5da5e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffa581529f0 0x7ffa580c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41360/0 (socket says 172.21.15.17:41360) 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5da5e640 1 -- 172.21.15.17:0/318703789 learned_addr learned my addr 172.21.15.17:0/318703789 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5d25d640 1 --2- 172.21.15.17:0/318703789 >> v2:172.21.15.163:3300/0 conn(0x7ffa580c5370 0x7ffa580c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5e25f640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 0x7ffa581559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5ca5c640 1 -- 172.21.15.17:0/318703789 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278403345 0 0) 0x7ffa58152dd0 con 0x7ffa581529f0 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5ca5c640 1 -- 172.21.15.17:0/318703789 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa44003520 con 0x7ffa581529f0 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5d25d640 1 -- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 msgr2=0x7ffa581559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5d25d640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 0x7ffa581559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5d25d640 1 -- 172.21.15.17:0/318703789 >> v1:172.21.15.17:6789/0 conn(0x7ffa581529f0 legacy=0x7ffa580c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5d25d640 1 -- 172.21.15.17:0/318703789 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58158b00 con 0x7ffa580c5370 2024-03-06T07:20:44.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.951+0000 7ffa5e25f640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 0x7ffa581559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:44.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.952+0000 7ffa5d25d640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa580c57d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa480097a0 tx=0x7ffa4802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfebea01b2f37ca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.952+0000 7ffa5ca5c640 1 -- 172.21.15.17:0/318703789 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa480315a0 con 0x7ffa580c5370 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.952+0000 7ffa5ca5c640 1 -- 172.21.15.17:0/318703789 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa48037070 con 0x7ffa580c5370 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.952+0000 7ffa5ca5c640 1 -- 172.21.15.17:0/318703789 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa48031960 con 0x7ffa580c5370 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 msgr2=0x7ffa580c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa580c57d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa480097a0 tx=0x7ffa4802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/318703789 shutdown_connections 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa58155530 0x7ffa581559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/318703789 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa580c57d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/318703789 >> 172.21.15.17:0/318703789 conn(0x7ffa580ac580 msgr2=0x7ffa580ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/318703789 shutdown_connections 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.953+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/318703789 wait complete. 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.954+0000 7ffa5fcc2640 1 Processor -- start 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.954+0000 7ffa5fcc2640 1 -- start start 2024-03-06T07:20:44.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa5814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 0x7ffa58157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa581590c0 con 0x7ffa58155530 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa580c9000 con 0x7ffa580c5370 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa581593a0 con 0x7ffa581529f0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43818/0 (socket says 172.21.15.17:43818) 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 -- 172.21.15.17:0/3518632543 learned_addr learned my addr 172.21.15.17:0/3518632543 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5d25d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 0x7ffa58157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5da5e640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa5814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 -- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 msgr2=0x7ffa58157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 0x7ffa58157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 -- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 msgr2=0x7ffa5814b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa5814b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 -- 172.21.15.17:0/3518632543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa581eb4a0 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5d25d640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 0x7ffa58157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5e25f640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000efc0 tx=0x7ffa50009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50018030 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5000cde0 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5814e560 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50003bb0 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.955+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5814b720 con 0x7ffa58155530 2024-03-06T07:20:44.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.956+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5804f330 con 0x7ffa58155530 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.957+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa5001b070 con 0x7ffa58155530 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.957+0000 7ffa4e7fc640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 0x7ffa3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.957+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa5008adb0 con 0x7ffa58155530 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.957+0000 7ffa5da5e640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 0x7ffa3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:44.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.958+0000 7ffa5da5e640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 0x7ffa3c06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005880 tx=0x7ffa54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:44.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:44.962+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa50010040 con 0x7ffa58155530 2024-03-06T07:20:45.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.142+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffa581514d0 con 0x7ffa58155530 2024-03-06T07:20:45.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.142+0000 7ffa4e7fc640 1 -- 172.21.15.17:0/3518632543 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffa50059330 con 0x7ffa58155530 2024-03-06T07:20:45.144 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 msgr2=0x7ffa3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 0x7ffa3c06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffa54005880 tx=0x7ffa54009040 comp rx=0 tx=0).stop 2024-03-06T07:20:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 msgr2=0x7ffa581edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000efc0 tx=0x7ffa50009f90 comp rx=0 tx=0).stop 2024-03-06T07:20:45.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 shutdown_connections 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa3c06b500 0x7ffa3c06d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa58155530 0x7ffa581edcb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa581529f0 0x7ffa58157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 --2- 172.21.15.17:0/3518632543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa580c5370 0x7ffa5814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 >> 172.21.15.17:0/3518632543 conn(0x7ffa580ac580 msgr2=0x7ffa58073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 shutdown_connections 2024-03-06T07:20:45.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.144+0000 7ffa5fcc2640 1 -- 172.21.15.17:0/3518632543 wait complete. 2024-03-06T07:20:45.157 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop' 2024-03-06T07:20:45.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.257+0000 7fa224f25640 1 Processor -- start 2024-03-06T07:20:45.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.257+0000 7fa224f25640 1 -- start start 2024-03-06T07:20:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa224f25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 0x7fa2200c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa224f25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa220155110 con 0x7fa220158db0 2024-03-06T07:20:45.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa224f25640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2200c8f40 con 0x7fa2200c88b0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa224f25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201559e0 con 0x7fa2200c5330 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21ed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa220158db0 0x7fa22015fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41374/0 (socket says 172.21.15.17:41374) 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21ed9d640 1 -- 172.21.15.17:0/1956467135 learned_addr learned my addr 172.21.15.17:0/1956467135 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21e59c640 1 --2- 172.21.15.17:0/1956467135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 0x7fa2200c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777584516 0 0) 0x7fa218001490 con 0x7fa220158db0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa20c003520 con 0x7fa220158db0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271724924 0 0) 0x7fa2200c8f40 con 0x7fa2200c88b0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa218001490 con 0x7fa2200c88b0 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21e59c640 1 -- 172.21.15.17:0/1956467135 >> v1:172.21.15.163:6789/0 conn(0x7fa2200c88b0 legacy=0x7fa2201586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21e59c640 1 -- 172.21.15.17:0/1956467135 >> v1:172.21.15.17:6789/0 conn(0x7fa220158db0 legacy=0x7fa22015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.258+0000 7fa21e59c640 1 -- 172.21.15.17:0/1956467135 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220155390 con 0x7fa2200c5330 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa21e59c640 1 --2- 172.21.15.17:0/1956467135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 0x7fa2200c5710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2100097d0 tx=0x7fa21002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd883dba00fbdbf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210035020 con 0x7fa2200c5330 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210031800 con 0x7fa2200c5330 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa21dd9b640 1 -- 172.21.15.17:0/1956467135 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210031bd0 con 0x7fa2200c5330 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa224f25640 1 -- 172.21.15.17:0/1956467135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 msgr2=0x7fa2200c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.259+0000 7fa224f25640 1 --2- 172.21.15.17:0/1956467135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 0x7fa2200c5710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2100097d0 tx=0x7fa21002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.260+0000 7fa224f25640 1 -- 172.21.15.17:0/1956467135 shutdown_connections 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.260+0000 7fa224f25640 1 --2- 172.21.15.17:0/1956467135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c5330 0x7fa2200c5710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.260+0000 7fa224f25640 1 -- 172.21.15.17:0/1956467135 >> 172.21.15.17:0/1956467135 conn(0x7fa2200ac560 msgr2=0x7fa2200ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:45.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.260+0000 7fa224f25640 1 -- 172.21.15.17:0/1956467135 shutdown_connections 2024-03-06T07:20:45.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.260+0000 7fa224f25640 1 -- 172.21.15.17:0/1956467135 wait complete. 2024-03-06T07:20:45.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 Processor -- start 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 -- start start 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2200c88b0 0x7fa220157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c2370 0x7fa22014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201529d0 con 0x7fa2200c5330 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22006d090 con 0x7fa2200c88b0 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.261+0000 7fa224f25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa220152b50 con 0x7fa2200c2370 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43830/0 (socket says 172.21.15.17:43830) 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 -- 172.21.15.17:0/3471259824 learned_addr learned my addr 172.21.15.17:0/3471259824 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21ed9d640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c2370 0x7fa22014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa217fff640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2200c88b0 0x7fa220157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 -- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c2370 msgr2=0x7fa22014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c2370 0x7fa22014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 -- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2200c88b0 msgr2=0x7fa220157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2200c88b0 0x7fa220157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 -- 172.21.15.17:0/3471259824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22014efd0 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa21e59c640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa2100098f0 tx=0x7fa2100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210041070 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210031e30 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22014cda0 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210035040 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.262+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22014ecf0 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.263+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa220067910 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.264+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa21004c060 con 0x7fa2200c5330 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.264+0000 7fa216ffd640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 0x7fa20006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:45.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.264+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa2100abdd0 con 0x7fa2200c5330 2024-03-06T07:20:45.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.264+0000 7fa217fff640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 0x7fa20006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:45.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.265+0000 7fa217fff640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 0x7fa20006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:45.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.268+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa220067910 con 0x7fa2200c5330 2024-03-06T07:20:45.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:45.520+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]} v 0) v1 -- 0x7fa22004f330 con 0x7fa2200c5330 2024-03-06T07:20:46.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.353+0000 7fa216ffd640 1 -- 172.21.15.17:0/3471259824 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]}]=0 v58) v1 ==== 146+0+0 (secure 0 0 0) 0x7fa2100ad0d0 con 0x7fa2200c5330 2024-03-06T07:20:46.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 msgr2=0x7fa20006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 0x7fa20006d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208009040 comp rx=0 tx=0).stop 2024-03-06T07:20:46.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 msgr2=0x7fa2201eb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa2100098f0 tx=0x7fa2100046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 shutdown_connections 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa20006b4d0 0x7fa20006d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2200c2370 0x7fa22014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa2200c88b0 0x7fa220157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 --2- 172.21.15.17:0/3471259824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2200c5330 0x7fa2201eb720 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 >> 172.21.15.17:0/3471259824 conn(0x7fa2200ac560 msgr2=0x7fa2200c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 shutdown_connections 2024-03-06T07:20:46.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.356+0000 7fa224f25640 1 -- 172.21.15.17:0/3471259824 wait complete. 2024-03-06T07:20:46.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:20:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.480+0000 7ff8bdfbc640 1 Processor -- start 2024-03-06T07:20:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.480+0000 7ff8bdfbc640 1 -- start start 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8bdfbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8b81529d0 0x7ff8b80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8bdfbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff8b8072cd0 0x7ff8b8155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8bdfbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b8152db0 con 0x7ff8b80c57e0 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8bdfbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b80c6dd0 con 0x7ff8b8072cd0 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8bdfbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b80c6940 con 0x7ff8b81529d0 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff8b8072cd0 0x7ff8b8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8affff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff8b80c57e0 0x7ff8b8155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41384/0 (socket says 172.21.15.17:41384) 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8affff640 1 -- 172.21.15.17:0/106125997 learned_addr learned my addr 172.21.15.17:0/106125997 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b77fe640 1 --2- 172.21.15.17:0/106125997 >> v2:172.21.15.163:3300/0 conn(0x7ff8b81529d0 0x7ff8b80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b7fff640 1 -- 172.21.15.17:0/106125997 >> v2:172.21.15.163:3300/0 conn(0x7ff8b81529d0 msgr2=0x7ff8b80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:46.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b7fff640 1 --2- 172.21.15.17:0/106125997 >> v2:172.21.15.163:3300/0 conn(0x7ff8b81529d0 0x7ff8b80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b7fff640 1 -- 172.21.15.17:0/106125997 >> v1:172.21.15.17:6789/0 conn(0x7ff8b80c57e0 legacy=0x7ff8b8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b7fff640 1 -- 172.21.15.17:0/106125997 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b81590f0 con 0x7ff8b8072cd0 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.481+0000 7ff8b6ffd640 1 -- 172.21.15.17:0/106125997 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364870637 0 0) 0x7ff8a0001110 con 0x7ff8b80c57e0 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.482+0000 7ff8b7fff640 1 --2- 172.21.15.17:0/106125997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b8155830 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80c98d0 tx=0x7ff8b002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53380dd8bd4b67b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.482+0000 7ff8b6ffd640 1 -- 172.21.15.17:0/106125997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0039070 con 0x7ff8b8072cd0 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.482+0000 7ff8b6ffd640 1 -- 172.21.15.17:0/106125997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b002cb60 con 0x7ff8b8072cd0 2024-03-06T07:20:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.482+0000 7ff8b6ffd640 1 -- 172.21.15.17:0/106125997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0034440 con 0x7ff8b8072cd0 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/106125997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 msgr2=0x7ff8b8155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/106125997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b8155830 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80c98d0 tx=0x7ff8b002bdb0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/106125997 shutdown_connections 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/106125997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b8155830 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/106125997 >> v2:172.21.15.163:3300/0 conn(0x7ff8b81529d0 0x7ff8b80c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/106125997 >> 172.21.15.17:0/106125997 conn(0x7ff8b80aa630 msgr2=0x7ff8b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/106125997 shutdown_connections 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.483+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/106125997 wait complete. 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.484+0000 7ff8bdfbc640 1 Processor -- start 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.484+0000 7ff8bdfbc640 1 -- start start 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 0x7ff8b80abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b81529d0 0x7ff8b80ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8072720 con 0x7ff8b8072cd0 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8155d70 con 0x7ff8b81529d0 2024-03-06T07:20:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8159370 con 0x7ff8b80c57e0 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43846/0 (socket says 172.21.15.17:43846) 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 -- 172.21.15.17:0/3008299910 learned_addr learned my addr 172.21.15.17:0/3008299910 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b7fff640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b81529d0 0x7ff8b80ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8affff640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 0x7ff8b80abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 -- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 msgr2=0x7ff8b80abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 0x7ff8b80abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 -- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b81529d0 msgr2=0x7ff8b80ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b81529d0 0x7ff8b80ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 -- 172.21.15.17:0/3008299910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b80ae730 con 0x7ff8b8072cd0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8affff640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 0x7ff8b80abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b77fe640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800e870 tx=0x7ff8a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8017070 con 0x7ff8b8072cd0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a800c940 con 0x7ff8b8072cd0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b80aca40 con 0x7ff8b8072cd0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800cd30 con 0x7ff8b8072cd0 2024-03-06T07:20:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.485+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b81eb620 con 0x7ff8b8072cd0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.487+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b804f330 con 0x7ff8b8072cd0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.487+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8a801a070 con 0x7ff8b8072cd0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.487+0000 7ff8b4ff9640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 0x7ff89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.488+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff8a808a2c0 con 0x7ff8b8072cd0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.488+0000 7ff8affff640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 0x7ff89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.488+0000 7ff8affff640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 0x7ff89806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8b81569d0 tx=0x7ff8a0007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.492+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8a8058840 con 0x7ff8b8072cd0 2024-03-06T07:20:46.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.672+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8b80ae220 con 0x7ff8b8072cd0 2024-03-06T07:20:46.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.675+0000 7ff8b4ff9640 1 -- 172.21.15.17:0/3008299910 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8a805c8e0 con 0x7ff8b8072cd0 2024-03-06T07:20:46.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.677+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 msgr2=0x7ff89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.677+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 0x7ff89806d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8b81569d0 tx=0x7ff8a0007ca0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.677+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 msgr2=0x7ff8b81540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.677+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800e870 tx=0x7ff8a800ed30 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 shutdown_connections 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff89806b4d0 0x7ff89806d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8b81529d0 0x7ff8b80ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8b80c57e0 0x7ff8b80abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 --2- 172.21.15.17:0/3008299910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8b8072cd0 0x7ff8b81540a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 >> 172.21.15.17:0/3008299910 conn(0x7ff8b80aa630 msgr2=0x7ff8b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 shutdown_connections 2024-03-06T07:20:46.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:46.678+0000 7ff8bdfbc640 1 -- 172.21.15.17:0/3008299910 wait complete. 2024-03-06T07:20:46.690 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:20:46.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:20:46.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:20:46.775 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:20:46.775 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:46.775 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:46.790 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:46.790 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:46.845 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:20:46.845 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:46.900 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 Processor -- start 2024-03-06T07:20:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 -- start start 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6490153ea0 0x7f64901571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64901576e0 con 0x7f649006df00 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900c42e0 con 0x7f64900c5b40 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f6496046640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900c45c0 con 0x7f6490153ea0 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f648f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6490153ea0 0x7f64901571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f648ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f649006df00 0x7f649006e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41394/0 (socket says 172.21.15.17:41394) 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.040+0000 7f648ffff640 1 -- 172.21.15.17:0/2244788180 learned_addr learned my addr 172.21.15.17:0/2244788180 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.041+0000 7f648f7fe640 1 -- 172.21.15.17:0/2244788180 >> v1:172.21.15.163:6789/0 conn(0x7f64900c5b40 legacy=0x7f64900c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.041+0000 7f648f7fe640 1 -- 172.21.15.17:0/2244788180 >> v1:172.21.15.17:6789/0 conn(0x7f649006df00 legacy=0x7f649006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.041+0000 7f648f7fe640 1 -- 172.21.15.17:0/2244788180 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900c48a0 con 0x7f6490153ea0 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.041+0000 7f648f7fe640 1 --2- 172.21.15.17:0/2244788180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6490153ea0 0x7f64901571a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64800097a0 tx=0x7f648002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1772c1a0004131f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f648e7fc640 1 -- 172.21.15.17:0/2244788180 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480035020 con 0x7f6490153ea0 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f648e7fc640 1 -- 172.21.15.17:0/2244788180 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480031830 con 0x7f6490153ea0 2024-03-06T07:20:47.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f648e7fc640 1 -- 172.21.15.17:0/2244788180 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480031c00 con 0x7f6490153ea0 2024-03-06T07:20:47.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 -- 172.21.15.17:0/2244788180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6490153ea0 msgr2=0x7f64901571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 --2- 172.21.15.17:0/2244788180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6490153ea0 0x7f64901571a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64800097a0 tx=0x7f648002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 -- 172.21.15.17:0/2244788180 shutdown_connections 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 --2- 172.21.15.17:0/2244788180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6490153ea0 0x7f64901571a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 -- 172.21.15.17:0/2244788180 >> 172.21.15.17:0/2244788180 conn(0x7f64900c7800 msgr2=0x7f64900c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 -- 172.21.15.17:0/2244788180 shutdown_connections 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.042+0000 7f6496046640 1 -- 172.21.15.17:0/2244788180 wait complete. 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.043+0000 7f6496046640 1 Processor -- start 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.043+0000 7f6496046640 1 -- start start 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64900c5b40 0x7f64901ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 0x7f64900c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64900c3f60 con 0x7f649006df00 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649006d9e0 con 0x7f6490153ea0 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f6496046640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64900c2cb0 con 0x7f64900c5b40 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43862/0 (socket says 172.21.15.17:43862) 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 -- 172.21.15.17:0/1665051312 learned_addr learned my addr 172.21.15.17:0/1665051312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648effd640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64900c5b40 0x7f64901ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648ffff640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 0x7f64900c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 -- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64900c5b40 msgr2=0x7f64901ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64900c5b40 0x7f64901ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 -- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 msgr2=0x7f64900c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 0x7f64900c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648f7fe640 1 -- 172.21.15.17:0/1665051312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64901dffb0 con 0x7f649006df00 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.044+0000 7f648ffff640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 0x7f64900c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f648f7fe640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6480034040 tx=0x7f64800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480040070 con 0x7f649006df00 2024-03-06T07:20:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480039a20 con 0x7f649006df00 2024-03-06T07:20:47.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64901e0290 con 0x7f649006df00 2024-03-06T07:20:47.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480035040 con 0x7f649006df00 2024-03-06T07:20:47.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.045+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64900a7840 con 0x7f649006df00 2024-03-06T07:20:47.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.046+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649004f330 con 0x7f649006df00 2024-03-06T07:20:47.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.046+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6480038020 con 0x7f649006df00 2024-03-06T07:20:47.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.047+0000 7f6477fff640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 0x7f647c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.047+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f64800abeb0 con 0x7f649006df00 2024-03-06T07:20:47.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.047+0000 7f648effd640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 0x7f647c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.047+0000 7f648effd640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 0x7f647c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64900c1d00 tx=0x7f6484009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.051+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f648007aa80 con 0x7f649006df00 2024-03-06T07:20:47.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.232+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64900c58b0 con 0x7f649006df00 2024-03-06T07:20:47.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.233+0000 7f6477fff640 1 -- 172.21.15.17:0/1665051312 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f648007a380 con 0x7f649006df00 2024-03-06T07:20:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:47.235 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 msgr2=0x7f647c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 0x7f647c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64900c1d00 tx=0x7f6484009040 comp rx=0 tx=0).stop 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 msgr2=0x7f64900a8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6480034040 tx=0x7f64800046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 shutdown_connections 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f647c06b4d0 0x7f647c06d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6490153ea0 0x7f64900c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64900c5b40 0x7f64901ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 --2- 172.21.15.17:0/1665051312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f649006df00 0x7f64900a8040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 >> 172.21.15.17:0/1665051312 conn(0x7f64900c7800 msgr2=0x7f64901519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 shutdown_connections 2024-03-06T07:20:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.235+0000 7f6496046640 1 -- 172.21.15.17:0/1665051312 wait complete. 2024-03-06T07:20:47.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:47.248 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:47.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.351+0000 7ff93ab13640 1 Processor -- start 2024-03-06T07:20:47.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.351+0000 7ff93ab13640 1 -- start start 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff93ab13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 0x7ff934072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff93ab13640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340c62a0 con 0x7ff9340c4740 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff93ab13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340c4c00 con 0x7ff9340a8290 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff93ab13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c4ee0 con 0x7ff9341539c0 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff933fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff9340c4740 0x7ff9340c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41410/0 (socket says 172.21.15.17:41410) 2024-03-06T07:20:47.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff933fff640 1 -- 172.21.15.17:0/1024468349 learned_addr learned my addr 172.21.15.17:0/1024468349 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9390b0640 1 --2- 172.21.15.17:0/1024468349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 0x7ff934072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750786612 0 0) 0x7ff9340c62a0 con 0x7ff9340c4740 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff924003520 con 0x7ff9340c4740 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103598869 0 0) 0x7ff9340c4c00 con 0x7ff9340a8290 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9340c62a0 con 0x7ff9340a8290 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1223495376 0 0) 0x7ff924003520 con 0x7ff9340c4740 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9390b0640 1 -- 172.21.15.17:0/1024468349 >> v1:172.21.15.163:6789/0 conn(0x7ff9340a8290 legacy=0x7ff9340701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9390b0640 1 -- 172.21.15.17:0/1024468349 >> v1:172.21.15.17:6789/0 conn(0x7ff9340c4740 legacy=0x7ff9340c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.352+0000 7ff9390b0640 1 -- 172.21.15.17:0/1024468349 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93406f9a0 con 0x7ff9341539c0 2024-03-06T07:20:47.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.353+0000 7ff9390b0640 1 --2- 172.21.15.17:0/1024468349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 0x7ff934072680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009dc0 tx=0x7ff92c02bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f1488971dda954 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.353+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c039070 con 0x7ff9341539c0 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.353+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c02c7f0 con 0x7ff9341539c0 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.353+0000 7ff9337fe640 1 -- 172.21.15.17:0/1024468349 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920001670 con 0x7ff9341539c0 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 -- 172.21.15.17:0/1024468349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 msgr2=0x7ff934072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 --2- 172.21.15.17:0/1024468349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 0x7ff934072680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009dc0 tx=0x7ff92c02bcc0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 -- 172.21.15.17:0/1024468349 shutdown_connections 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 --2- 172.21.15.17:0/1024468349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9341539c0 0x7ff934072680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 -- 172.21.15.17:0/1024468349 >> 172.21.15.17:0/1024468349 conn(0x7ff9340c5560 msgr2=0x7ff9340c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 -- 172.21.15.17:0/1024468349 shutdown_connections 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.354+0000 7ff93ab13640 1 -- 172.21.15.17:0/1024468349 wait complete. 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.355+0000 7ff93ab13640 1 Processor -- start 2024-03-06T07:20:47.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.355+0000 7ff93ab13640 1 -- start start 2024-03-06T07:20:47.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340a8290 0x7ff9341ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340c4740 0x7ff934155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff934072bc0 con 0x7ff9341539c0 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff934153e50 con 0x7ff9340c4740 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff93ab13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff934072d40 con 0x7ff9340a8290 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43864/0 (socket says 172.21.15.17:43864) 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 -- 172.21.15.17:0/3035571436 learned_addr learned my addr 172.21.15.17:0/3035571436 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff933fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340c4740 0x7ff934155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9388af640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340a8290 0x7ff9341ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 -- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340a8290 msgr2=0x7ff9341ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340a8290 0x7ff9341ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 -- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340c4740 msgr2=0x7ff934155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340c4740 0x7ff934155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.356+0000 7ff9390b0640 1 -- 172.21.15.17:0/3035571436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340aba20 con 0x7ff9341539c0 2024-03-06T07:20:47.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff9390b0640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009ee0 tx=0x7ff92c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c039050 con 0x7ff9341539c0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c045070 con 0x7ff9341539c0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93414bd70 con 0x7ff9341539c0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c009450 con 0x7ff9341539c0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.357+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934151fc0 con 0x7ff9341539c0 2024-03-06T07:20:47.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.358+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9340abbf0 con 0x7ff9341539c0 2024-03-06T07:20:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.358+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff92c040690 con 0x7ff9341539c0 2024-03-06T07:20:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.359+0000 7ff9317fa640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.359+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff92c0ac490 con 0x7ff9341539c0 2024-03-06T07:20:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.359+0000 7ff9388af640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.359+0000 7ff9388af640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 0x7ff91806d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9280011e0 tx=0x7ff928009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.363+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff92c04e020 con 0x7ff9341539c0 2024-03-06T07:20:47.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.543+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff93404f330 con 0x7ff9341539c0 2024-03-06T07:20:47.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.544+0000 7ff9317fa640 1 -- 172.21.15.17:0/3035571436 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff92c02ce20 con 0x7ff9341539c0 2024-03-06T07:20:47.545 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 msgr2=0x7ff91806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 0x7ff91806d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff9280011e0 tx=0x7ff928009040 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 msgr2=0x7ff934156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009ee0 tx=0x7ff92c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 shutdown_connections 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff91806b500 0x7ff91806d9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9341539c0 0x7ff934156250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9340c4740 0x7ff934155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 --2- 172.21.15.17:0/3035571436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9340a8290 0x7ff9341ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.545+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 >> 172.21.15.17:0/3035571436 conn(0x7ff9340c5560 msgr2=0x7ff9340c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.546+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 shutdown_connections 2024-03-06T07:20:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.546+0000 7ff93ab13640 1 -- 172.21.15.17:0/3035571436 wait complete. 2024-03-06T07:20:47.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.660+0000 7fb42c75e640 1 Processor -- start 2024-03-06T07:20:47.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.660+0000 7fb42c75e640 1 -- start start 2024-03-06T07:20:47.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb42c75e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4240a8290 0x7fb4241539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb42c75e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240c62a0 con 0x7fb4240c4740 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb42c75e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240c4c00 con 0x7fb4240c69a0 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb42c75e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb424153f00 con 0x7fb4240a8290 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb429cf9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb4240c4740 0x7fb4240701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41420/0 (socket says 172.21.15.17:41420) 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb429cf9640 1 -- 172.21.15.17:0/2826094957 learned_addr learned my addr 172.21.15.17:0/2826094957 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb42a4fa640 1 --2- 172.21.15.17:0/2826094957 >> v2:172.21.15.163:3300/0 conn(0x7fb4240a8290 0x7fb4241539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.661+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161729807 0 0) 0x7fb4240c62a0 con 0x7fb4240c4740 2024-03-06T07:20:47.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c003520 con 0x7fb4240c4740 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082677945 0 0) 0x7fb4240c4c00 con 0x7fb4240c69a0 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4240c62a0 con 0x7fb4240c69a0 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb42a4fa640 1 -- 172.21.15.17:0/2826094957 >> v1:172.21.15.163:6789/0 conn(0x7fb4240c69a0 legacy=0x7fb424072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb42a4fa640 1 -- 172.21.15.17:0/2826094957 >> v1:172.21.15.17:6789/0 conn(0x7fb4240c4740 legacy=0x7fb4240701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408454914 0 0) 0x7fb40c003520 con 0x7fb4240c4740 2024-03-06T07:20:47.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.662+0000 7fb42a4fa640 1 -- 172.21.15.17:0/2826094957 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240c4ee0 con 0x7fb4240a8290 2024-03-06T07:20:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42a4fa640 1 --2- 172.21.15.17:0/2826094957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241539c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097a0 tx=0x7fb41c02b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4785db052bc62e68 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c037070 con 0x7fb4240a8290 2024-03-06T07:20:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c030510 con 0x7fb4240a8290 2024-03-06T07:20:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb4294f8640 1 -- 172.21.15.17:0/2826094957 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418001670 con 0x7fb4240a8290 2024-03-06T07:20:47.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 -- 172.21.15.17:0/2826094957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 msgr2=0x7fb4241539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 --2- 172.21.15.17:0/2826094957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241539c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0097a0 tx=0x7fb41c02b710 comp rx=0 tx=0).stop 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 -- 172.21.15.17:0/2826094957 shutdown_connections 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 --2- 172.21.15.17:0/2826094957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241539c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 -- 172.21.15.17:0/2826094957 >> 172.21.15.17:0/2826094957 conn(0x7fb4240c5560 msgr2=0x7fb4240c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 -- 172.21.15.17:0/2826094957 shutdown_connections 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.663+0000 7fb42c75e640 1 -- 172.21.15.17:0/2826094957 wait complete. 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.664+0000 7fb42c75e640 1 Processor -- start 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.664+0000 7fb42c75e640 1 -- start start 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4240c69a0 0x7fb4241ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424072d90 con 0x7fb4240c4740 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42406fc80 con 0x7fb4240a8290 2024-03-06T07:20:47.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42c75e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb424072f10 con 0x7fb4240c69a0 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb429cf9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb429cf9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43866/0 (socket says 172.21.15.17:43866) 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb429cf9640 1 -- 172.21.15.17:0/437364563 learned_addr learned my addr 172.21.15.17:0/437364563 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42a4fa640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb42acfb640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4240c69a0 0x7fb4241ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb429cf9640 1 -- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4240c69a0 msgr2=0x7fb4241ec7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.665+0000 7fb429cf9640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4240c69a0 0x7fb4241ec7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb429cf9640 1 -- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 msgr2=0x7fb4241514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb429cf9640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb429cf9640 1 -- 172.21.15.17:0/437364563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42414d320 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb429cf9640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c750 tx=0x7fb41800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418005ad0 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb418005c70 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42414d600 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4180072b0 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.666+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb424151a20 con 0x7fb4240c4740 2024-03-06T07:20:47.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.667+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb42404f330 con 0x7fb4240c4740 2024-03-06T07:20:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.667+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb418007450 con 0x7fb4240c4740 2024-03-06T07:20:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.668+0000 7fb422ffd640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 0x7fb41006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.668+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb41808a1f0 con 0x7fb4240c4740 2024-03-06T07:20:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.668+0000 7fb42a4fa640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 0x7fb41006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.668+0000 7fb42a4fa640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 0x7fb41006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.672+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb418005e20 con 0x7fb4240c4740 2024-03-06T07:20:47.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.850+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4241539c0 con 0x7fb4240c4740 2024-03-06T07:20:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.851+0000 7fb422ffd640 1 -- 172.21.15.17:0/437364563 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb418058830 con 0x7fb4240c4740 2024-03-06T07:20:47.853 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 msgr2=0x7fb41006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 0x7fb41006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb41c034000 tx=0x7fb41c008030 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 msgr2=0x7fb4241ec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb41800c750 tx=0x7fb41800cc10 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 shutdown_connections 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41006b4d0 0x7fb41006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4240c69a0 0x7fb4241ec7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4240c4740 0x7fb4241ec280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 --2- 172.21.15.17:0/437364563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4240a8290 0x7fb4241514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 >> 172.21.15.17:0/437364563 conn(0x7fb4240c5560 msgr2=0x7fb4240a9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 shutdown_connections 2024-03-06T07:20:47.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.853+0000 7fb42c75e640 1 -- 172.21.15.17:0/437364563 wait complete. 2024-03-06T07:20:47.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:47.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 Processor -- start 2024-03-06T07:20:47.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 -- start start 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac90155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac90158a10 0x7fac90158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900c75e0 con 0x7fac9014bcc0 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900c7f10 con 0x7fac90158a10 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.968+0000 7fac981e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c78c0 con 0x7fac900be150 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac90158a10 0x7fac90158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac90158a10 0x7fac90158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43876/0 (socket says 172.21.15.17:43876) 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 -- 172.21.15.17:0/3583651303 learned_addr learned my addr 172.21.15.17:0/3583651303 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac95784640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac90155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 -- 172.21.15.17:0/3583651303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 msgr2=0x7fac90155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac90155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 -- 172.21.15.17:0/3583651303 >> v1:172.21.15.163:6789/0 conn(0x7fac9014bcc0 legacy=0x7fac900c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.969+0000 7fac96786640 1 -- 172.21.15.17:0/3583651303 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900c7ba0 con 0x7fac90158a10 2024-03-06T07:20:47.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac96786640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90158e20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac8802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fdc10977225070 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac94f83640 1 -- 172.21.15.17:0/3583651303 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8802cb90 con 0x7fac90158a10 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac94f83640 1 -- 172.21.15.17:0/3583651303 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8802cd30 con 0x7fac90158a10 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac94f83640 1 -- 172.21.15.17:0/3583651303 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88034610 con 0x7fac90158a10 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 -- 172.21.15.17:0/3583651303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 msgr2=0x7fac90158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90158e20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac8802bff0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 -- 172.21.15.17:0/3583651303 shutdown_connections 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90158e20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 --2- 172.21.15.17:0/3583651303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac90155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 -- 172.21.15.17:0/3583651303 >> 172.21.15.17:0/3583651303 conn(0x7fac900ac600 msgr2=0x7fac900aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 -- 172.21.15.17:0/3583651303 shutdown_connections 2024-03-06T07:20:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.970+0000 7fac981e9640 1 -- 172.21.15.17:0/3583651303 wait complete. 2024-03-06T07:20:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.971+0000 7fac981e9640 1 Processor -- start 2024-03-06T07:20:47.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 -- start start 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac900c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9014bcc0 0x7fac901eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c5c60 con 0x7fac90158a10 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c5ae0 con 0x7fac9014bcc0 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac981e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90159360 con 0x7fac900be150 2024-03-06T07:20:47.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac96786640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac96786640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43880/0 (socket says 172.21.15.17:43880) 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac96786640 1 -- 172.21.15.17:0/2466875445 learned_addr learned my addr 172.21.15.17:0/2466875445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac95f85640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac900c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.972+0000 7fac95784640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9014bcc0 0x7fac901eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 -- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 msgr2=0x7fac900c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac900c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 -- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9014bcc0 msgr2=0x7fac901eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9014bcc0 0x7fac901eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 -- 172.21.15.17:0/2466875445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac901df220 con 0x7fac90158a10 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac96786640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88043020 con 0x7fac90158a10 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8803e450 con 0x7fac90158a10 2024-03-06T07:20:47.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac901eeef0 con 0x7fac90158a10 2024-03-06T07:20:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88009510 con 0x7fac90158a10 2024-03-06T07:20:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.973+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9014ae80 con 0x7fac90158a10 2024-03-06T07:20:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.974+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9004f330 con 0x7fac90158a10 2024-03-06T07:20:47.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.975+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac88005d10 con 0x7fac90158a10 2024-03-06T07:20:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.975+0000 7fac867fc640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 0x7fac7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.975+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fac880acc90 con 0x7fac90158a10 2024-03-06T07:20:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.975+0000 7fac95f85640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 0x7fac7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:47.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.976+0000 7fac95f85640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 0x7fac7406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fac901e8d10 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:47.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:47.979+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fac8807b210 con 0x7fac90158a10 2024-03-06T07:20:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.159+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac900bdf30 con 0x7fac90158a10 2024-03-06T07:20:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.159+0000 7fac867fc640 1 -- 172.21.15.17:0/2466875445 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac8807f2b0 con 0x7fac90158a10 2024-03-06T07:20:48.161 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 msgr2=0x7fac7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 0x7fac7406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fac901e8d10 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 msgr2=0x7fac90157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fac88009e50 tx=0x7fac88004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 shutdown_connections 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac7406b4d0 0x7fac7406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac90158a10 0x7fac90157740 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9014bcc0 0x7fac901eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 --2- 172.21.15.17:0/2466875445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac900be150 0x7fac900c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 >> 172.21.15.17:0/2466875445 conn(0x7fac900ac600 msgr2=0x7fac900c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 shutdown_connections 2024-03-06T07:20:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.161+0000 7fac981e9640 1 -- 172.21.15.17:0/2466875445 wait complete. 2024-03-06T07:20:48.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.276+0000 7fea7595e640 1 Processor -- start 2024-03-06T07:20:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.276+0000 7fea7595e640 1 -- start start 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea7595e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea7006d750 0x7fea700732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea7595e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea700c3a70 0x7fea70072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea7595e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea70156060 con 0x7fea7006db30 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea7595e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea70155b30 con 0x7fea700c3a70 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea7595e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea700a8430 con 0x7fea7006d750 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea667fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea700c3a70 0x7fea70072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea667fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea700c3a70 0x7fea70072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43884/0 (socket says 172.21.15.17:43884) 2024-03-06T07:20:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea667fc640 1 -- 172.21.15.17:0/3569596422 learned_addr learned my addr 172.21.15.17:0/3569596422 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.277+0000 7fea6effd640 1 --2- 172.21.15.17:0/3569596422 >> v2:172.21.15.163:3300/0 conn(0x7fea7006d750 0x7fea700732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea667fc640 1 -- 172.21.15.17:0/3569596422 >> v2:172.21.15.163:3300/0 conn(0x7fea7006d750 msgr2=0x7fea700732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea667fc640 1 --2- 172.21.15.17:0/3569596422 >> v2:172.21.15.163:3300/0 conn(0x7fea7006d750 0x7fea700732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea667fc640 1 -- 172.21.15.17:0/3569596422 >> v1:172.21.15.163:6789/0 conn(0x7fea7006db30 legacy=0x7fea700c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea667fc640 1 -- 172.21.15.17:0/3569596422 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70155d80 con 0x7fea700c3a70 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea6e7fc640 1 -- 172.21.15.17:0/3569596422 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305997567 0 0) 0x7fea70156060 con 0x7fea7006db30 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea667fc640 1 --2- 172.21.15.17:0/3569596422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea700c3a70 0x7fea70072680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea700c5440 tx=0x7fea5802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d639fc6f9917182 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea6e7fc640 1 -- 172.21.15.17:0/3569596422 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5802e070 con 0x7fea700c3a70 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea6e7fc640 1 -- 172.21.15.17:0/3569596422 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea58037070 con 0x7fea700c3a70 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.278+0000 7fea6e7fc640 1 -- 172.21.15.17:0/3569596422 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58031a00 con 0x7fea700c3a70 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 -- 172.21.15.17:0/3569596422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea700c3a70 msgr2=0x7fea70072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 --2- 172.21.15.17:0/3569596422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea700c3a70 0x7fea70072680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fea700c5440 tx=0x7fea5802b880 comp rx=0 tx=0).stop 2024-03-06T07:20:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 -- 172.21.15.17:0/3569596422 shutdown_connections 2024-03-06T07:20:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 --2- 172.21.15.17:0/3569596422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea700c3a70 0x7fea70072680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 --2- 172.21.15.17:0/3569596422 >> v2:172.21.15.163:3300/0 conn(0x7fea7006d750 0x7fea700732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 -- 172.21.15.17:0/3569596422 >> 172.21.15.17:0/3569596422 conn(0x7fea700c5560 msgr2=0x7fea700c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 -- 172.21.15.17:0/3569596422 shutdown_connections 2024-03-06T07:20:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.279+0000 7fea7595e640 1 -- 172.21.15.17:0/3569596422 wait complete. 2024-03-06T07:20:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.280+0000 7fea7595e640 1 Processor -- start 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.280+0000 7fea7595e640 1 -- start start 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea7006db30 0x7fea700a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 0x7fea7014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea70072bc0 con 0x7fea7006d750 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea700c27b0 con 0x7fea700c3a70 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea7595e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea70072d40 con 0x7fea7006db30 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43898/0 (socket says 172.21.15.17:43898) 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 -- 172.21.15.17:0/2758940605 learned_addr learned my addr 172.21.15.17:0/2758940605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea667fc640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea7006db30 0x7fea700a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6f7fe640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 0x7fea7014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 -- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea7006db30 msgr2=0x7fea700a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea7006db30 0x7fea700a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 -- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 msgr2=0x7fea7014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 0x7fea7014d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 -- 172.21.15.17:0/2758940605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70153ad0 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6f7fe640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 0x7fea7014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.281+0000 7fea6effd640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea6000e870 tx=0x7fea6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.282+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60017070 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.282+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6000c940 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.282+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea70153db0 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.282+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6000cd30 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.282+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea701571b0 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.283+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7004f330 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.283+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea6001a070 con 0x7fea7006d750 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.284+0000 7fea6cff9640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 0x7fea5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.284+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea6008a6c0 con 0x7fea7006d750 2024-03-06T07:20:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.284+0000 7fea667fc640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 0x7fea5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.284+0000 7fea667fc640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 0x7fea5006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea58036000 tx=0x7fea5803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.288+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea60058ba0 con 0x7fea7006d750 2024-03-06T07:20:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.467+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fea700c71a0 con 0x7fea7006d750 2024-03-06T07:20:48.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.467+0000 7fea6cff9640 1 -- 172.21.15.17:0/2758940605 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fea6005cc40 con 0x7fea7006d750 2024-03-06T07:20:48.469 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:48.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 msgr2=0x7fea5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 0x7fea5006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea58036000 tx=0x7fea5803d040 comp rx=0 tx=0).stop 2024-03-06T07:20:48.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 msgr2=0x7fea70154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea6000e870 tx=0x7fea6000ed30 comp rx=0 tx=0).stop 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 shutdown_connections 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5006b4d0 0x7fea5006d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea700c3a70 0x7fea7014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea7006db30 0x7fea700a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.469+0000 7fea7595e640 1 --2- 172.21.15.17:0/2758940605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7006d750 0x7fea70154b10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.470+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 >> 172.21.15.17:0/2758940605 conn(0x7fea700c5560 msgr2=0x7fea701512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.470+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 shutdown_connections 2024-03-06T07:20:48.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.470+0000 7fea7595e640 1 -- 172.21.15.17:0/2758940605 wait complete. 2024-03-06T07:20:48.482 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.582+0000 7f6cec102640 1 Processor -- start 2024-03-06T07:20:48.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 -- start start 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 0x7f6ce4155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 0x7f6ce4152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce40c7200 con 0x7f6ce40c6d70 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce4158710 con 0x7f6ce40bfe90 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.583+0000 7f6cec102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4159590 con 0x7f6ce41522d0 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce9e9e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 0x7f6ce4155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6cea69f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 0x7f6ce4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce969d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6ce40c6d70 0x7f6ce4158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:52008/0 (socket says 172.21.15.17:52008) 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce969d640 1 -- 172.21.15.17:0/275444838 learned_addr learned my addr 172.21.15.17:0/275444838 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:48.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce8e9c640 1 -- 172.21.15.17:0/275444838 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222649651 0 0) 0x7f6cd4001110 con 0x7f6ce40c6d70 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce8e9c640 1 -- 172.21.15.17:0/275444838 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd0003520 con 0x7f6ce40c6d70 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6cea69f640 1 -- 172.21.15.17:0/275444838 >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 msgr2=0x7f6ce4155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6cea69f640 1 --2- 172.21.15.17:0/275444838 >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 0x7f6ce4155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6cea69f640 1 -- 172.21.15.17:0/275444838 >> v1:172.21.15.163:6789/0 conn(0x7f6ce40c6d70 legacy=0x7f6ce4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6cea69f640 1 -- 172.21.15.17:0/275444838 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40c59d0 con 0x7f6ce41522d0 2024-03-06T07:20:48.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.584+0000 7f6ce9e9e640 1 --2- 172.21.15.17:0/275444838 >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 0x7f6ce4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:48.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6cea69f640 1 --2- 172.21.15.17:0/275444838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 0x7f6ce4152740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=286718cfef8048de server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6ce8e9c640 1 -- 172.21.15.17:0/275444838 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc039070 con 0x7f6ce41522d0 2024-03-06T07:20:48.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6ce8e9c640 1 -- 172.21.15.17:0/275444838 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc02cbc0 con 0x7f6ce41522d0 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6ce8e9c640 1 -- 172.21.15.17:0/275444838 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc034440 con 0x7f6ce41522d0 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6cec102640 1 -- 172.21.15.17:0/275444838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 msgr2=0x7f6ce4152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.585+0000 7f6cec102640 1 --2- 172.21.15.17:0/275444838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 0x7f6ce4152740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc02be10 comp rx=0 tx=0).stop 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 -- 172.21.15.17:0/275444838 shutdown_connections 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 --2- 172.21.15.17:0/275444838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce41522d0 0x7f6ce4152740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 --2- 172.21.15.17:0/275444838 >> v2:172.21.15.163:3300/0 conn(0x7f6ce40bfe90 0x7f6ce4155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 -- 172.21.15.17:0/275444838 >> 172.21.15.17:0/275444838 conn(0x7f6ce40acaa0 msgr2=0x7f6ce40aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 -- 172.21.15.17:0/275444838 shutdown_connections 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.586+0000 7f6cec102640 1 -- 172.21.15.17:0/275444838 wait complete. 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 Processor -- start 2024-03-06T07:20:48.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 -- start start 2024-03-06T07:20:48.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce40bfe90 0x7f6ce41ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce41522d0 0x7f6ce41e1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce4158960 con 0x7f6ce40c6d70 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40c5f90 con 0x7f6ce41522d0 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6cec102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40ca700 con 0x7f6ce40bfe90 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.587+0000 7f6ce969d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43904/0 (socket says 172.21.15.17:43904) 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce9e9e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce40bfe90 0x7f6ce41ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 -- 172.21.15.17:0/1863876773 learned_addr learned my addr 172.21.15.17:0/1863876773 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cea69f640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce41522d0 0x7f6ce41e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 -- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce40bfe90 msgr2=0x7f6ce41ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce40bfe90 0x7f6ce41ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 -- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce41522d0 msgr2=0x7f6ce41e1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce41522d0 0x7f6ce41e1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 -- 172.21.15.17:0/1863876773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce41e2240 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6ce969d640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c730 tx=0x7f6cd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4017070 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4005ad0 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce41eea70 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4007e20 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.588+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce41ee770 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.589+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce404f330 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.590+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cd401a070 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.590+0000 7f6cda7fc640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 0x7f6cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.590+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6cd4089840 con 0x7f6ce40c6d70 2024-03-06T07:20:48.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.590+0000 7f6ce9e9e640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 0x7f6cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:48.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.591+0000 7f6ce9e9e640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 0x7f6cc806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009560 tx=0x7f6ce0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:48.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.594+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ce40c7200 con 0x7f6ce40c6d70 2024-03-06T07:20:48.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.790+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ce40c53d0 con 0x7f6ce40c6d70 2024-03-06T07:20:48.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.790+0000 7f6cda7fc640 1 -- 172.21.15.17:0/1863876773 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6cd4057dc0 con 0x7f6ce40c6d70 2024-03-06T07:20:48.792 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-06T07:20:48.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 msgr2=0x7f6cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 0x7f6cc806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009560 tx=0x7f6ce0009190 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 msgr2=0x7f6ce40ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c730 tx=0x7f6cd400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 shutdown_connections 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6cc806b4d0 0x7f6cc806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ce41522d0 0x7f6ce41e1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ce40c6d70 0x7f6ce40ad660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 --2- 172.21.15.17:0/1863876773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ce40bfe90 0x7f6ce41ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 >> 172.21.15.17:0/1863876773 conn(0x7f6ce40acaa0 msgr2=0x7f6ce4156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 shutdown_connections 2024-03-06T07:20:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:48.792+0000 7f6cec102640 1 -- 172.21.15.17:0/1863876773 wait complete. 2024-03-06T07:20:48.805 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-06T07:20:48.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:48.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-06T07:20:48.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:20:48.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:48.833 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-06T07:20:48.888 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:20:48.889 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:48.904 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:20:48.905 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:48.960 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:20:48.960 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:20:49.014 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.113+0000 7f6c6a3a0640 1 Processor -- start 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.113+0000 7f6c6a3a0640 1 -- start start 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6a3a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6c64153990 0x7f6c64156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6a3a0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c640acc00 0x7f6c640c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6a3a0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c64070aa0 con 0x7f6c641506b0 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6a3a0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c64156450 con 0x7f6c640acc00 2024-03-06T07:20:49.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6a3a0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c64156a10 con 0x7f6c64153990 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c63fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c641506b0 0x7f6c640702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41428/0 (socket says 172.21.15.17:41428) 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c63fff640 1 -- 172.21.15.17:0/3926364673 learned_addr learned my addr 172.21.15.17:0/3926364673 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c6893d640 1 --2- 172.21.15.17:0/3926364673 >> v2:172.21.15.17:3300/0 conn(0x7f6c640acc00 0x7f6c640c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.114+0000 7f6c637fe640 1 --2- 172.21.15.17:0/3926364673 >> v2:172.21.15.163:3300/0 conn(0x7f6c64153990 0x7f6c64156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c6893d640 1 -- 172.21.15.17:0/3926364673 >> v2:172.21.15.163:3300/0 conn(0x7f6c64153990 msgr2=0x7f6c64156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c6893d640 1 --2- 172.21.15.17:0/3926364673 >> v2:172.21.15.163:3300/0 conn(0x7f6c64153990 0x7f6c64156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c6893d640 1 -- 172.21.15.17:0/3926364673 >> v1:172.21.15.17:6789/0 conn(0x7f6c641506b0 legacy=0x7f6c640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c6893d640 1 -- 172.21.15.17:0/3926364673 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64070d80 con 0x7f6c640acc00 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c62ffd640 1 -- 172.21.15.17:0/3926364673 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540790556 0 0) 0x7f6c64070aa0 con 0x7f6c641506b0 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c6893d640 1 --2- 172.21.15.17:0/3926364673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c54a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c64150a90 tx=0x7f6c5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b6358a42136faa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c62ffd640 1 -- 172.21.15.17:0/3926364673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c039070 con 0x7f6c640acc00 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c62ffd640 1 -- 172.21.15.17:0/3926364673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5c02cc80 con 0x7f6c640acc00 2024-03-06T07:20:49.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.115+0000 7f6c62ffd640 1 -- 172.21.15.17:0/3926364673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c034660 con 0x7f6c640acc00 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/3926364673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 msgr2=0x7f6c640c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/3926364673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c54a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6c64150a90 tx=0x7f6c5c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/3926364673 shutdown_connections 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/3926364673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c54a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/3926364673 >> v2:172.21.15.163:3300/0 conn(0x7f6c64153990 0x7f6c64156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/3926364673 >> 172.21.15.17:0/3926364673 conn(0x7f6c6406b1a0 msgr2=0x7f6c6406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/3926364673 shutdown_connections 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.116+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/3926364673 wait complete. 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.117+0000 7f6c6a3a0640 1 Processor -- start 2024-03-06T07:20:49.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.117+0000 7f6c6a3a0640 1 -- start start 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c641506b0 0x7f6c64150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c64153990 0x7f6c641517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c5b60 con 0x7f6c640acc00 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c59e0 con 0x7f6c641506b0 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6a3a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c640c5ce0 con 0x7f6c64153990 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43922/0 (socket says 172.21.15.17:43922) 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 -- 172.21.15.17:0/821776219 learned_addr learned my addr 172.21.15.17:0/821776219 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c637fe640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c641506b0 0x7f6c64150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c6893d640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c64153990 0x7f6c641517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 -- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c64153990 msgr2=0x7f6c641517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c64153990 0x7f6c641517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 -- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c641506b0 msgr2=0x7f6c64150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c641506b0 0x7f6c64150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.118+0000 7f6c63fff640 1 -- 172.21.15.17:0/821776219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64152980 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c63fff640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c500 tx=0x7f6c5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54018070 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c54007580 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c641e9880 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54004a20 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.119+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c640c7d20 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.120+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6406bab0 con 0x7f6c640acc00 2024-03-06T07:20:49.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.120+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c5401b070 con 0x7f6c640acc00 2024-03-06T07:20:49.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.121+0000 7f6c60ff9640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 0x7f6c5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.121+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6c54089730 con 0x7f6c640acc00 2024-03-06T07:20:49.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.121+0000 7f6c637fe640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 0x7f6c5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.121+0000 7f6c637fe640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 0x7f6c5006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ad890 tx=0x7f6c58009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.125+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c6406bab0 con 0x7f6c640acc00 2024-03-06T07:20:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.306+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c6404f330 con 0x7f6c640acc00 2024-03-06T07:20:49.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.307+0000 7f6c60ff9640 1 -- 172.21.15.17:0/821776219 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7f6c54057cb0 con 0x7f6c640acc00 2024-03-06T07:20:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:49.309 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 msgr2=0x7f6c5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 0x7f6c5006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ad890 tx=0x7f6c58009190 comp rx=0 tx=0).stop 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 msgr2=0x7f6c640c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400c500 tx=0x7f6c5400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 shutdown_connections 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c5006b4d0 0x7f6c5006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c64153990 0x7f6c641517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c641506b0 0x7f6c64150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 --2- 172.21.15.17:0/821776219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c640acc00 0x7f6c640c77e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 >> 172.21.15.17:0/821776219 conn(0x7f6c6406b1a0 msgr2=0x7f6c64152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 shutdown_connections 2024-03-06T07:20:49.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.309+0000 7f6c6a3a0640 1 -- 172.21.15.17:0/821776219 wait complete. 2024-03-06T07:20:49.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:49.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.425+0000 7f406a5ac640 1 Processor -- start 2024-03-06T07:20:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.425+0000 7f406a5ac640 1 -- start start 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f406a5ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40640c1820 0x7f406406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f406a5ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640c2790 con 0x7f406406c1f0 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f406a5ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40640c1540 con 0x7f406414ef70 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f406a5ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40640c3090 con 0x7f40640c1820 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f4068b49640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40640c1820 0x7f406406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f40637fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f406406c1f0 0x7f4064154c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41444/0 (socket says 172.21.15.17:41444) 2024-03-06T07:20:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f40637fe640 1 -- 172.21.15.17:0/658422111 learned_addr learned my addr 172.21.15.17:0/658422111 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f4068b49640 1 -- 172.21.15.17:0/658422111 >> v1:172.21.15.163:6789/0 conn(0x7f406414ef70 legacy=0x7f40640c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f4068b49640 1 -- 172.21.15.17:0/658422111 >> v1:172.21.15.17:6789/0 conn(0x7f406406c1f0 legacy=0x7f4064154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.426+0000 7f4062ffd640 1 -- 172.21.15.17:0/658422111 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2617853958 0 0) 0x7f40640c2790 con 0x7f406406c1f0 2024-03-06T07:20:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.427+0000 7f4068b49640 1 -- 172.21.15.17:0/658422111 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640c24b0 con 0x7f40640c1820 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.427+0000 7f4068b49640 1 --2- 172.21.15.17:0/658422111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40640c1820 0x7f406406cad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f405c00aea0 tx=0x7f405c02cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5c97895e4f8a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.427+0000 7f4062ffd640 1 -- 172.21.15.17:0/658422111 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c02db60 con 0x7f40640c1820 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.427+0000 7f4062ffd640 1 -- 172.21.15.17:0/658422111 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c02dd00 con 0x7f40640c1820 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.427+0000 7f4062ffd640 1 -- 172.21.15.17:0/658422111 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c008660 con 0x7f40640c1820 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 -- 172.21.15.17:0/658422111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40640c1820 msgr2=0x7f406406cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 --2- 172.21.15.17:0/658422111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40640c1820 0x7f406406cad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f405c00aea0 tx=0x7f405c02cfc0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 -- 172.21.15.17:0/658422111 shutdown_connections 2024-03-06T07:20:49.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 --2- 172.21.15.17:0/658422111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40640c1820 0x7f406406cad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 -- 172.21.15.17:0/658422111 >> 172.21.15.17:0/658422111 conn(0x7f40640c6710 msgr2=0x7f40640c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 -- 172.21.15.17:0/658422111 shutdown_connections 2024-03-06T07:20:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.428+0000 7f406a5ac640 1 -- 172.21.15.17:0/658422111 wait complete. 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.429+0000 7f406a5ac640 1 Processor -- start 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.429+0000 7f406a5ac640 1 -- start start 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406406c1f0 0x7f4064071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40640c1820 0x7f40641497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40640c1c70 con 0x7f406414ef70 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406414f350 con 0x7f406406c1f0 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f406a5ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4064155340 con 0x7f40640c1820 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43942/0 (socket says 172.21.15.17:43942) 2024-03-06T07:20:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 -- 172.21.15.17:0/1121074592 learned_addr learned my addr 172.21.15.17:0/1121074592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4063fff640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406406c1f0 0x7f4064071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f40637fe640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40640c1820 0x7f40641497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 -- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40640c1820 msgr2=0x7f40641497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40640c1820 0x7f40641497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 -- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406406c1f0 msgr2=0x7f4064071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406406c1f0 0x7f4064071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.430+0000 7f4068b49640 1 -- 172.21.15.17:0/1121074592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640a88b0 con 0x7f406414ef70 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f4068b49640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f405c00aea0 tx=0x7f405c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c044020 con 0x7f406414ef70 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c040690 con 0x7f406414ef70 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40641ebf70 con 0x7f406414ef70 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c040a80 con 0x7f406414ef70 2024-03-06T07:20:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.431+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40641ec530 con 0x7f406414ef70 2024-03-06T07:20:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.432+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406404f330 con 0x7f406414ef70 2024-03-06T07:20:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.432+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f405c047020 con 0x7f406414ef70 2024-03-06T07:20:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.433+0000 7f4060ff9640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 0x7f404406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.433+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f405c0ad340 con 0x7f406414ef70 2024-03-06T07:20:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.433+0000 7f4063fff640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 0x7f404406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.433+0000 7f4063fff640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 0x7f404406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f404c000f60 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.437+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f405c04e030 con 0x7f406414ef70 2024-03-06T07:20:49.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.619+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40641de790 con 0x7f406414ef70 2024-03-06T07:20:49.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.619+0000 7f4060ff9640 1 -- 172.21.15.17:0/1121074592 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f405c07b8c0 con 0x7f406414ef70 2024-03-06T07:20:49.621 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 msgr2=0x7f404406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 0x7f404406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f404c000f60 tx=0x7f404c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 msgr2=0x7f4064149d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f405c00aea0 tx=0x7f405c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:49.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 shutdown_connections 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f404406b4d0 0x7f404406d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f406414ef70 0x7f4064149d30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40640c1820 0x7f40641497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 --2- 172.21.15.17:0/1121074592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406406c1f0 0x7f4064071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 >> 172.21.15.17:0/1121074592 conn(0x7f40640c6710 msgr2=0x7f40640c3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 shutdown_connections 2024-03-06T07:20:49.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.621+0000 7f406a5ac640 1 -- 172.21.15.17:0/1121074592 wait complete. 2024-03-06T07:20:49.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.736+0000 7fa09a95c640 1 Processor -- start 2024-03-06T07:20:49.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.736+0000 7fa09a95c640 1 -- start start 2024-03-06T07:20:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa09a95c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0940a8290 0x7fa0941539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa09a95c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0940c4740 0x7fa0940701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa09a95c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940c62a0 con 0x7fa0940c69a0 2024-03-06T07:20:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa09a95c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa094153f00 con 0x7fa0940c4740 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa09a95c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940706e0 con 0x7fa0940a8290 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa0937fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0940c4740 0x7fa0940701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa098ef9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa0940c69a0 0x7fa094072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41458/0 (socket says 172.21.15.17:41458) 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa098ef9640 1 -- 172.21.15.17:0/3589587285 learned_addr learned my addr 172.21.15.17:0/3589587285 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa093fff640 1 --2- 172.21.15.17:0/3589587285 >> v2:172.21.15.163:3300/0 conn(0x7fa0940a8290 0x7fa0941539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa0937fe640 1 -- 172.21.15.17:0/3589587285 >> v2:172.21.15.163:3300/0 conn(0x7fa0940a8290 msgr2=0x7fa0941539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa0937fe640 1 --2- 172.21.15.17:0/3589587285 >> v2:172.21.15.163:3300/0 conn(0x7fa0940a8290 0x7fa0941539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa0937fe640 1 -- 172.21.15.17:0/3589587285 >> v1:172.21.15.17:6789/0 conn(0x7fa0940c69a0 legacy=0x7fa094072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa0937fe640 1 -- 172.21.15.17:0/3589587285 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940c4c00 con 0x7fa0940c4740 2024-03-06T07:20:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.737+0000 7fa092ffd640 1 -- 172.21.15.17:0/3589587285 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745789397 0 0) 0x7fa0940c62a0 con 0x7fa0940c69a0 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.738+0000 7fa0937fe640 1 --2- 172.21.15.17:0/3589587285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c4740 0x7fa0940701a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0940735b0 tx=0x7fa07c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f186fc0fe407256 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.738+0000 7fa092ffd640 1 -- 172.21.15.17:0/3589587285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c038070 con 0x7fa0940c4740 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.738+0000 7fa092ffd640 1 -- 172.21.15.17:0/3589587285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c031710 con 0x7fa0940c4740 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.738+0000 7fa092ffd640 1 -- 172.21.15.17:0/3589587285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c031ae0 con 0x7fa0940c4740 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 -- 172.21.15.17:0/3589587285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c4740 msgr2=0x7fa0940701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3589587285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c4740 0x7fa0940701a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa0940735b0 tx=0x7fa07c02b7b0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 -- 172.21.15.17:0/3589587285 shutdown_connections 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3589587285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c4740 0x7fa0940701a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3589587285 >> v2:172.21.15.163:3300/0 conn(0x7fa0940a8290 0x7fa0941539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 -- 172.21.15.17:0/3589587285 >> 172.21.15.17:0/3589587285 conn(0x7fa0940c5560 msgr2=0x7fa0940c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 -- 172.21.15.17:0/3589587285 shutdown_connections 2024-03-06T07:20:49.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.739+0000 7fa09a95c640 1 -- 172.21.15.17:0/3589587285 wait complete. 2024-03-06T07:20:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.740+0000 7fa09a95c640 1 Processor -- start 2024-03-06T07:20:49.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.740+0000 7fa09a95c640 1 -- start start 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0940a8290 0x7fa0941dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0940c4740 0x7fa0941e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09406fc80 con 0x7fa0940c69a0 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09406f9a0 con 0x7fa0940a8290 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa09a95c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa094072d90 con 0x7fa0940c4740 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43952/0 (socket says 172.21.15.17:43952) 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 -- 172.21.15.17:0/3046977209 learned_addr learned my addr 172.21.15.17:0/3046977209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa0937fe640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0940c4740 0x7fa0941e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa093fff640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0940a8290 0x7fa0941dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 -- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0940c4740 msgr2=0x7fa0941e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0940c4740 0x7fa0941e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 -- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0940a8290 msgr2=0x7fa0941dcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0940a8290 0x7fa0941dcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 -- 172.21.15.17:0/3046977209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0941dda30 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.741+0000 7fa098ef9640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00cad0 tx=0x7fa08c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.742+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00ac30 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.742+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c00adf0 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.742+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0941ddd10 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.742+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c012640 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.742+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0941dc7c0 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.743+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09404f330 con 0x7fa0940c69a0 2024-03-06T07:20:49.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.743+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa08c0127e0 con 0x7fa0940c69a0 2024-03-06T07:20:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.744+0000 7fa090ff9640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 0x7fa074071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.744+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa08c08a9b0 con 0x7fa0940c69a0 2024-03-06T07:20:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.744+0000 7fa093fff640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 0x7fa074071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:49.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.744+0000 7fa093fff640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 0x7fa074071dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa084009b40 tx=0x7fa084009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:49.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.748+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa08c058f30 con 0x7fa0940c69a0 2024-03-06T07:20:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.927+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0940c8480 con 0x7fa0940c69a0 2024-03-06T07:20:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.928+0000 7fa090ff9640 1 -- 172.21.15.17:0/3046977209 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa08c05cfd0 con 0x7fa0940c69a0 2024-03-06T07:20:49.929 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.929+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 msgr2=0x7fa074071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.929+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 0x7fa074071dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa084009b40 tx=0x7fa084009190 comp rx=0 tx=0).stop 2024-03-06T07:20:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.929+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 msgr2=0x7fa0940adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.929+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00cad0 tx=0x7fa08c00cf90 comp rx=0 tx=0).stop 2024-03-06T07:20:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 shutdown_connections 2024-03-06T07:20:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa07406f910 0x7fa074071dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0940c69a0 0x7fa0940adce0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0940c4740 0x7fa0941e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 --2- 172.21.15.17:0/3046977209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0940a8290 0x7fa0941dcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 >> 172.21.15.17:0/3046977209 conn(0x7fa0940c5560 msgr2=0x7fa0940a9180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 shutdown_connections 2024-03-06T07:20:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:49.930+0000 7fa09a95c640 1 -- 172.21.15.17:0/3046977209 wait complete. 2024-03-06T07:20:49.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:50.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.045+0000 7faa6aa66640 1 Processor -- start 2024-03-06T07:20:50.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.045+0000 7faa6aa66640 1 -- start start 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa6aa66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 0x7faa64155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa6aa66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa6406ca70 0x7faa640c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa6aa66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa640b0450 con 0x7faa640c5380 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa6aa66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6406c020 con 0x7faa6406ca70 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa6aa66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa640c77f0 con 0x7faa640b0070 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa6406ca70 0x7faa640c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa6406ca70 0x7faa640c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43968/0 (socket says 172.21.15.17:43968) 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 -- 172.21.15.17:0/3144114210 learned_addr learned my addr 172.21.15.17:0/3144114210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa68802640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 0x7faa64155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 -- 172.21.15.17:0/3144114210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 msgr2=0x7faa64155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 0x7faa64155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.046+0000 7faa63fff640 1 -- 172.21.15.17:0/3144114210 >> v1:172.21.15.163:6789/0 conn(0x7faa640c5380 legacy=0x7faa6414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa63fff640 1 -- 172.21.15.17:0/3144114210 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640c7ad0 con 0x7faa6406ca70 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa68802640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 0x7faa64155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa63fff640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa6406ca70 0x7faa640c8900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667507af80c4b892 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa637fe640 1 -- 172.21.15.17:0/3144114210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58035020 con 0x7faa6406ca70 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa637fe640 1 -- 172.21.15.17:0/3144114210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58031640 con 0x7faa6406ca70 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.047+0000 7faa637fe640 1 -- 172.21.15.17:0/3144114210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802d040 con 0x7faa6406ca70 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 -- 172.21.15.17:0/3144114210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa6406ca70 msgr2=0x7faa640c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa6406ca70 0x7faa640c8900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 -- 172.21.15.17:0/3144114210 shutdown_connections 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa6406ca70 0x7faa640c8900 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 --2- 172.21.15.17:0/3144114210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa640b0070 0x7faa64155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 -- 172.21.15.17:0/3144114210 >> 172.21.15.17:0/3144114210 conn(0x7faa640ac590 msgr2=0x7faa640ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 -- 172.21.15.17:0/3144114210 shutdown_connections 2024-03-06T07:20:50.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.048+0000 7faa6aa66640 1 -- 172.21.15.17:0/3144114210 wait complete. 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.049+0000 7faa6aa66640 1 Processor -- start 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.049+0000 7faa6aa66640 1 -- start start 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa6406ca70 0x7faa640c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa640c5380 0x7faa6414ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6406cef0 con 0x7faa640b0070 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6406c550 con 0x7faa640c5380 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa6aa66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6406da80 con 0x7faa6406ca70 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43984/0 (socket says 172.21.15.17:43984) 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 -- 172.21.15.17:0/527785079 learned_addr learned my addr 172.21.15.17:0/527785079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa68802640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa6406ca70 0x7faa640c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa69003640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa640c5380 0x7faa6414ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 -- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa6406ca70 msgr2=0x7faa640c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa6406ca70 0x7faa640c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 -- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa640c5380 msgr2=0x7faa6414ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.050+0000 7faa63fff640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa640c5380 0x7faa6414ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa63fff640 1 -- 172.21.15.17:0/527785079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640af6a0 con 0x7faa640b0070 2024-03-06T07:20:50.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa63fff640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58040070 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa580399f0 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa64158d40 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58035040 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.051+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa64159300 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.052+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6404f330 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.052+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa5804b060 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.053+0000 7faa617fa640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.053+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faa580abcc0 con 0x7faa640b0070 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.053+0000 7faa68802640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.053+0000 7faa68802640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 0x7faa5006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa54006fd0 tx=0x7faa54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.057+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa5807a240 con 0x7faa640b0070 2024-03-06T07:20:50.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.236+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa6406d860 con 0x7faa640b0070 2024-03-06T07:20:50.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.236+0000 7faa617fa640 1 -- 172.21.15.17:0/527785079 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa5807e2e0 con 0x7faa640b0070 2024-03-06T07:20:50.238 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 msgr2=0x7faa5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 0x7faa5006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa54006fd0 tx=0x7faa54008040 comp rx=0 tx=0).stop 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 msgr2=0x7faa641546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa580046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 shutdown_connections 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5006b4d0 0x7faa5006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa640c5380 0x7faa6414ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa640b0070 0x7faa641546c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 --2- 172.21.15.17:0/527785079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa6406ca70 0x7faa640c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 >> 172.21.15.17:0/527785079 conn(0x7faa640ac590 msgr2=0x7faa640ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 shutdown_connections 2024-03-06T07:20:50.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.238+0000 7faa6aa66640 1 -- 172.21.15.17:0/527785079 wait complete. 2024-03-06T07:20:50.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:50.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.353+0000 7fe5df481640 1 Processor -- start 2024-03-06T07:20:50.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.353+0000 7fe5df481640 1 -- start start 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5df481640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5d81529f0 0x7fe5d80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5df481640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d80c57d0 0x7fe5d8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5df481640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d8152dd0 con 0x7fe5d80c5370 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5df481640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d8159710 con 0x7fe5d81529f0 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5df481640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8156180 con 0x7fe5d80c57d0 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5dd21d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5d81529f0 0x7fe5d80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5dca1c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5d80c5370 0x7fe5d8155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41464/0 (socket says 172.21.15.17:41464) 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5dca1c640 1 -- 172.21.15.17:0/71923123 learned_addr learned my addr 172.21.15.17:0/71923123 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.354+0000 7fe5dda1e640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d80c57d0 0x7fe5d8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5dd21d640 1 -- 172.21.15.17:0/71923123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d80c57d0 msgr2=0x7fe5d8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5dd21d640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d80c57d0 0x7fe5d8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5dd21d640 1 -- 172.21.15.17:0/71923123 >> v1:172.21.15.17:6789/0 conn(0x7fe5d80c5370 legacy=0x7fe5d8155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5dd21d640 1 -- 172.21.15.17:0/71923123 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8158b00 con 0x7fe5d81529f0 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5cffff640 1 -- 172.21.15.17:0/71923123 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814172190 0 0) 0x7fe5d8152dd0 con 0x7fe5d80c5370 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5dd21d640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d81529f0 0x7fe5d80c88f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40097a0 tx=0x7fe5d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7eb13d9751ed337 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.355+0000 7fe5cffff640 1 -- 172.21.15.17:0/71923123 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40315a0 con 0x7fe5d81529f0 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5cffff640 1 -- 172.21.15.17:0/71923123 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4037070 con 0x7fe5d81529f0 2024-03-06T07:20:50.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5cffff640 1 -- 172.21.15.17:0/71923123 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4031960 con 0x7fe5d81529f0 2024-03-06T07:20:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 -- 172.21.15.17:0/71923123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d81529f0 msgr2=0x7fe5d80c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d81529f0 0x7fe5d80c88f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40097a0 tx=0x7fe5d402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 -- 172.21.15.17:0/71923123 shutdown_connections 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d80c57d0 0x7fe5d8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 --2- 172.21.15.17:0/71923123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d81529f0 0x7fe5d80c88f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 -- 172.21.15.17:0/71923123 >> 172.21.15.17:0/71923123 conn(0x7fe5d80ac580 msgr2=0x7fe5d80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 -- 172.21.15.17:0/71923123 shutdown_connections 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.356+0000 7fe5df481640 1 -- 172.21.15.17:0/71923123 wait complete. 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.357+0000 7fe5df481640 1 Processor -- start 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.357+0000 7fe5df481640 1 -- start start 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5d80c5370 0x7fe5d81497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d81529f0 0x7fe5d814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d81590c0 con 0x7fe5d80c57d0 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80c68b0 con 0x7fe5d80c5370 2024-03-06T07:20:50.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5df481640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d81593a0 con 0x7fe5d81529f0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44002/0 (socket says 172.21.15.17:44002) 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 -- 172.21.15.17:0/1513586893 learned_addr learned my addr 172.21.15.17:0/1513586893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dd21d640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5d80c5370 0x7fe5d81497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dda1e640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d81529f0 0x7fe5d814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 -- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d81529f0 msgr2=0x7fe5d814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d81529f0 0x7fe5d814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 -- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5d80c5370 msgr2=0x7fe5d81497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5d80c5370 0x7fe5d81497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.358+0000 7fe5dca1c640 1 -- 172.21.15.17:0/1513586893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8149ce0 con 0x7fe5d80c57d0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5dca1c640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c4d0 tx=0x7fe5c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8018070 con 0x7fe5d80c57d0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8007500 con 0x7fe5d80c57d0 2024-03-06T07:20:50.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004a20 con 0x7fe5d80c57d0 2024-03-06T07:20:50.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d81584f0 con 0x7fe5d80c57d0 2024-03-06T07:20:50.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.359+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d8156960 con 0x7fe5d80c57d0 2024-03-06T07:20:50.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.360+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d804f330 con 0x7fe5d80c57d0 2024-03-06T07:20:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.360+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5c801b070 con 0x7fe5d80c57d0 2024-03-06T07:20:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.361+0000 7fe5cdffb640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.361+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe5c8089710 con 0x7fe5d80c57d0 2024-03-06T07:20:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.361+0000 7fe5dd21d640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.361+0000 7fe5dd21d640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.366+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe5c8057c90 con 0x7fe5d80c57d0 2024-03-06T07:20:50.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.546+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe5d814fea0 con 0x7fe5d80c57d0 2024-03-06T07:20:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.546+0000 7fe5cdffb640 1 -- 172.21.15.17:0/1513586893 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe5c805bd30 con 0x7fe5d80c57d0 2024-03-06T07:20:50.548 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:50.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 msgr2=0x7fe5bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d4008030 comp rx=0 tx=0).stop 2024-03-06T07:20:50.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 msgr2=0x7fe5d8157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c4d0 tx=0x7fe5c800c990 comp rx=0 tx=0).stop 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 shutdown_connections 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5d81529f0 0x7fe5d814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5d80c57d0 0x7fe5d8157ca0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 --2- 172.21.15.17:0/1513586893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5d80c5370 0x7fe5d81497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 >> 172.21.15.17:0/1513586893 conn(0x7fe5d80ac580 msgr2=0x7fe5d80c97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 shutdown_connections 2024-03-06T07:20:50.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.548+0000 7fe5df481640 1 -- 172.21.15.17:0/1513586893 wait complete. 2024-03-06T07:20:50.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-06T07:20:50.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.661+0000 7f466d565640 1 Processor -- start 2024-03-06T07:20:50.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.661+0000 7f466d565640 1 -- start start 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f466d565640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 0x7f46680a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f466d565640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4668156600 0x7f46680a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f466d565640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4668153010 con 0x7f46681504c0 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f466d565640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4668153ee0 con 0x7f4668156600 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f466d565640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466806f390 con 0x7f46680c2130 2024-03-06T07:20:50.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f4666ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46681504c0 0x7f46680c4940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51972/0 (socket says 172.21.15.17:51972) 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f4666ffd640 1 -- 172.21.15.17:0/2316070876 learned_addr learned my addr 172.21.15.17:0/2316070876 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f46677fe640 1 --2- 172.21.15.17:0/2316070876 >> v2:172.21.15.163:3300/0 conn(0x7f4668156600 0x7f46680a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f46667fc640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 0x7f46680a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.662+0000 7f4665ffb640 1 -- 172.21.15.17:0/2316070876 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838799204 0 0) 0x7f4668153010 con 0x7f46681504c0 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f4665ffb640 1 -- 172.21.15.17:0/2316070876 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4654003520 con 0x7f46681504c0 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46677fe640 1 -- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 msgr2=0x7f46680a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46677fe640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 0x7f46680a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46677fe640 1 -- 172.21.15.17:0/2316070876 >> v1:172.21.15.17:6789/0 conn(0x7f46681504c0 legacy=0x7f46680c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46677fe640 1 -- 172.21.15.17:0/2316070876 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46681532d0 con 0x7f4668156600 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46667fc640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 0x7f46680a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.663+0000 7f46677fe640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4668156600 0x7f46680a99c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498ecafd5527762e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f4665ffb640 1 -- 172.21.15.17:0/2316070876 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658039070 con 0x7f4668156600 2024-03-06T07:20:50.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f4665ffb640 1 -- 172.21.15.17:0/2316070876 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465802cbc0 con 0x7f4668156600 2024-03-06T07:20:50.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f4665ffb640 1 -- 172.21.15.17:0/2316070876 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658034440 con 0x7f4668156600 2024-03-06T07:20:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 -- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4668156600 msgr2=0x7f46680a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4668156600 0x7f46680a99c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465802be10 comp rx=0 tx=0).stop 2024-03-06T07:20:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 -- 172.21.15.17:0/2316070876 shutdown_connections 2024-03-06T07:20:50.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4668156600 0x7f46680a99c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 --2- 172.21.15.17:0/2316070876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46680c2130 0x7f46680a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 -- 172.21.15.17:0/2316070876 >> 172.21.15.17:0/2316070876 conn(0x7f46680c7de0 msgr2=0x7f46680c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 -- 172.21.15.17:0/2316070876 shutdown_connections 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.664+0000 7f466d565640 1 -- 172.21.15.17:0/2316070876 wait complete. 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.665+0000 7f466d565640 1 Processor -- start 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 -- start start 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 0x7f46681ec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 0x7f46681e8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668153b70 con 0x7f46681504c0 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668153890 con 0x7f46680c2130 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f466d565640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46680c5050 con 0x7f4668156600 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f46667fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f46667fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33774/0 (socket says 172.21.15.17:33774) 2024-03-06T07:20:50.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f46667fc640 1 -- 172.21.15.17:0/1949997498 learned_addr learned my addr 172.21.15.17:0/1949997498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f4666ffd640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 0x7f46681ec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.666+0000 7f46677fe640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 0x7f46681e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 -- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 msgr2=0x7f46681e8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 0x7f46681e8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 -- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 msgr2=0x7f46681ec6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 0x7f46681ec6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 -- 172.21.15.17:0/1949997498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466814e060 con 0x7f46681504c0 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f4666ffd640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 0x7f46681ec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46677fe640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 0x7f46681e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f46667fc640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9b0 tx=0x7f465000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650017070 con 0x7f46681504c0 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465000ca00 con 0x7f46681504c0 2024-03-06T07:20:50.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466814e300 con 0x7f46681504c0 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465000cdf0 con 0x7f46681504c0 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.667+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4668155d30 con 0x7f46681504c0 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.668+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466804f330 con 0x7f46681504c0 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.668+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f465001a070 con 0x7f46681504c0 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.669+0000 7f465f7fe640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:50.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.669+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f465008a510 con 0x7f46681504c0 2024-03-06T07:20:50.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.669+0000 7f4666ffd640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:50.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.669+0000 7f4666ffd640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4660007fd0 tx=0x7f4660009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:50.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.673+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4650059060 con 0x7f46681504c0 2024-03-06T07:20:50.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.865+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f466806f050 con 0x7f46681504c0 2024-03-06T07:20:50.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.865+0000 7f465f7fe640 1 -- 172.21.15.17:0/1949997498 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4650058960 con 0x7f46681504c0 2024-03-06T07:20:50.867 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 msgr2=0x7f464806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4660007fd0 tx=0x7f4660009040 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 msgr2=0x7f46681557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9b0 tx=0x7f465000ee70 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 shutdown_connections 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4668156600 0x7f46681e8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46681504c0 0x7f46681557f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 --2- 172.21.15.17:0/1949997498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46680c2130 0x7f46681ec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:50.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 >> 172.21.15.17:0/1949997498 conn(0x7f46680c7de0 msgr2=0x7f46680c3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:50.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 shutdown_connections 2024-03-06T07:20:50.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:50.867+0000 7f466d565640 1 -- 172.21.15.17:0/1949997498 wait complete. 2024-03-06T07:20:50.879 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-06T07:20:50.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:50.880 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-06T07:20:50.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:20:50.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:20:50.909 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-06T07:20:50.982 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:20:50.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:51.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.082+0000 7f7992439640 1 Processor -- start 2024-03-06T07:20:51.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.082+0000 7f7992439640 1 -- start start 2024-03-06T07:20:51.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f7992439640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f798c06afc0 0x7f798c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f7992439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 0x7f798c153890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f7992439640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798c153dd0 con 0x7f798c0c20b0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f7992439640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798c0a9530 con 0x7f798c06afc0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f7992439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0a9810 con 0x7f798c0c25c0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f798c06afc0 0x7f798c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f798c06afc0 0x7f798c06b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33790/0 (socket says 172.21.15.17:33790) 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 -- 172.21.15.17:0/772423401 learned_addr learned my addr 172.21.15.17:0/772423401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f79909d6640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 0x7f798c153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 -- 172.21.15.17:0/772423401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 msgr2=0x7f798c153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 0x7f798c153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 -- 172.21.15.17:0/772423401 >> v1:172.21.15.163:6789/0 conn(0x7f798c0c20b0 legacy=0x7f798c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f798bfff640 1 -- 172.21.15.17:0/772423401 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0a9af0 con 0x7f798c06afc0 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.083+0000 7f79909d6640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 0x7f798c153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:51.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.084+0000 7f798bfff640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c06afc0 0x7f798c06b3a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7974009870 tx=0x7f797402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ff0930b36e295c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.084+0000 7f798affd640 1 -- 172.21.15.17:0/772423401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740317c0 con 0x7f798c06afc0 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.084+0000 7f798affd640 1 -- 172.21.15.17:0/772423401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974037070 con 0x7f798c06afc0 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.084+0000 7f798affd640 1 -- 172.21.15.17:0/772423401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974031b80 con 0x7f798c06afc0 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 -- 172.21.15.17:0/772423401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c06afc0 msgr2=0x7f798c06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c06afc0 0x7f798c06b3a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7974009870 tx=0x7f797402b860 comp rx=0 tx=0).stop 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 -- 172.21.15.17:0/772423401 shutdown_connections 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c25c0 0x7f798c153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 --2- 172.21.15.17:0/772423401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c06afc0 0x7f798c06b3a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 -- 172.21.15.17:0/772423401 >> 172.21.15.17:0/772423401 conn(0x7f798c0c76b0 msgr2=0x7f798c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 -- 172.21.15.17:0/772423401 shutdown_connections 2024-03-06T07:20:51.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.085+0000 7f7992439640 1 -- 172.21.15.17:0/772423401 wait complete. 2024-03-06T07:20:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.086+0000 7f7992439640 1 Processor -- start 2024-03-06T07:20:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.086+0000 7f7992439640 1 -- start start 2024-03-06T07:20:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c06afc0 0x7f798c148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c20b0 0x7f798c1ddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c150980 con 0x7f798c0c25c0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0aa0b0 con 0x7f798c06afc0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7992439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c150b00 con 0x7f798c0c20b0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f798b7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c20b0 0x7f798c1ddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33798/0 (socket says 172.21.15.17:33798) 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f798bfff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c06afc0 0x7f798c148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 -- 172.21.15.17:0/3209094551 learned_addr learned my addr 172.21.15.17:0/3209094551 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 -- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c20b0 msgr2=0x7f798c1ddfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c20b0 0x7f798c1ddfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 -- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c06afc0 msgr2=0x7f798c148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c06afc0 0x7f798c148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 -- 172.21.15.17:0/3209094551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c1de970 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f79909d6640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7984009fd0 tx=0x7f798400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.087+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798401a070 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.088+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7984003bb0 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.088+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798c1e9150 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.088+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004030 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.088+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798c1e9710 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.089+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798c04f330 con 0x7f798c0c25c0 2024-03-06T07:20:51.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.089+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f798401d070 con 0x7f798c0c25c0 2024-03-06T07:20:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.089+0000 7f7988ff9640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 0x7f796c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.090+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f798408a6e0 con 0x7f798c0c25c0 2024-03-06T07:20:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.090+0000 7f798bfff640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 0x7f796c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.090+0000 7f798bfff640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 0x7f796c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7974036000 tx=0x7f797403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.094+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7984010070 con 0x7f798c0c25c0 2024-03-06T07:20:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.312+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f798c0ad790 con 0x7f798c0c25c0 2024-03-06T07:20:51.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.313+0000 7f7988ff9640 1 -- 172.21.15.17:0/3209094551 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f7984058c60 con 0x7f798c0c25c0 2024-03-06T07:20:51.315 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:51.315 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:51.315 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 msgr2=0x7f796c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 0x7f796c06d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7974036000 tx=0x7f797403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 msgr2=0x7f798c147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7984009fd0 tx=0x7f798400d560 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 shutdown_connections 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796c06b4d0 0x7f796c06d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c0c25c0 0x7f798c147730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798c0c20b0 0x7f798c1ddfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 --2- 172.21.15.17:0/3209094551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f798c06afc0 0x7f798c148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 >> 172.21.15.17:0/3209094551 conn(0x7f798c0c76b0 msgr2=0x7f798c152570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 shutdown_connections 2024-03-06T07:20:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.315+0000 7f7992439640 1 -- 172.21.15.17:0/3209094551 wait complete. 2024-03-06T07:20:51.328 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.427+0000 7fe9518df640 1 Processor -- start 2024-03-06T07:20:51.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.427+0000 7fe9518df640 1 -- start start 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe9518df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe94c1506b0 0x7fe94c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe9518df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe94c0acc00 0x7fe94c0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe9518df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94c070aa0 con 0x7fe94c156ea0 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe9518df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c156450 con 0x7fe94c0acc00 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe9518df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94c156a10 con 0x7fe94c1506b0 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe94b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe94c0acc00 0x7fe94c0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe94a7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe94c156ea0 0x7fe94c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51986/0 (socket says 172.21.15.17:51986) 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe94a7fc640 1 -- 172.21.15.17:0/2240091443 learned_addr learned my addr 172.21.15.17:0/2240091443 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.428+0000 7fe94affd640 1 --2- 172.21.15.17:0/2240091443 >> v2:172.21.15.163:3300/0 conn(0x7fe94c1506b0 0x7fe94c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2240091443 >> v2:172.21.15.163:3300/0 conn(0x7fe94c1506b0 msgr2=0x7fe94c153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe94b7fe640 1 --2- 172.21.15.17:0/2240091443 >> v2:172.21.15.163:3300/0 conn(0x7fe94c1506b0 0x7fe94c153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2240091443 >> v1:172.21.15.17:6789/0 conn(0x7fe94c156ea0 legacy=0x7fe94c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2240091443 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c070d80 con 0x7fe94c0acc00 2024-03-06T07:20:51.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe949ffb640 1 -- 172.21.15.17:0/2240091443 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139623330 0 0) 0x7fe94c070aa0 con 0x7fe94c156ea0 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe94b7fe640 1 --2- 172.21.15.17:0/2240091443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0acc00 0x7fe94c0c54a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0c97e0 tx=0x7fe93c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922f6a31850e684d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe949ffb640 1 -- 172.21.15.17:0/2240091443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c039070 con 0x7fe94c0acc00 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe949ffb640 1 -- 172.21.15.17:0/2240091443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c02cc80 con 0x7fe94c0acc00 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.429+0000 7fe949ffb640 1 -- 172.21.15.17:0/2240091443 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c034660 con 0x7fe94c0acc00 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 -- 172.21.15.17:0/2240091443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0acc00 msgr2=0x7fe94c0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 --2- 172.21.15.17:0/2240091443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0acc00 0x7fe94c0c54a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0c97e0 tx=0x7fe93c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 -- 172.21.15.17:0/2240091443 shutdown_connections 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 --2- 172.21.15.17:0/2240091443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0acc00 0x7fe94c0c54a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 --2- 172.21.15.17:0/2240091443 >> v2:172.21.15.163:3300/0 conn(0x7fe94c1506b0 0x7fe94c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 -- 172.21.15.17:0/2240091443 >> 172.21.15.17:0/2240091443 conn(0x7fe94c06b1a0 msgr2=0x7fe94c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 -- 172.21.15.17:0/2240091443 shutdown_connections 2024-03-06T07:20:51.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.430+0000 7fe9518df640 1 -- 172.21.15.17:0/2240091443 wait complete. 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.431+0000 7fe9518df640 1 Processor -- start 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.431+0000 7fe9518df640 1 -- start start 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe94c0acc00 0x7fe94c1e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe94c1506b0 0x7fe94c0c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0ad1d0 con 0x7fe94c156ea0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0ad050 con 0x7fe94c0acc00 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe9518df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c0ad350 con 0x7fe94c1506b0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33814/0 (socket says 172.21.15.17:33814) 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2673863369 learned_addr learned my addr 172.21.15.17:0/2673863369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94affd640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe94c0acc00 0x7fe94c1e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94a7fc640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe94c1506b0 0x7fe94c0c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe94c1506b0 msgr2=0x7fe94c0c1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe94c1506b0 0x7fe94c0c1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe94c0acc00 msgr2=0x7fe94c1e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe94c0acc00 0x7fe94c1e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.432+0000 7fe94b7fe640 1 -- 172.21.15.17:0/2673863369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c1eec00 con 0x7fe94c156ea0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe94b7fe640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009e80 tx=0x7fe93c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c039050 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c045070 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c06c720 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c009450 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.433+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c06df10 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.434+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c04f330 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.434+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe93c03e850 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.435+0000 7fe933fff640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.435+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe93c0ad360 con 0x7fe94c156ea0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.435+0000 7fe94affd640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.435+0000 7fe94affd640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe944009b40 tx=0x7fe944009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.439+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe93c07ba00 con 0x7fe94c156ea0 2024-03-06T07:20:51.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.620+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe94c0700a0 con 0x7fe94c156ea0 2024-03-06T07:20:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.621+0000 7fe933fff640 1 -- 172.21.15.17:0/2673863369 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18514 (secure 0 0 0) 0x7fe93c07faa0 con 0x7fe94c156ea0 2024-03-06T07:20:51.623 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:51.623 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:10.741512+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 msgr2=0x7fe93806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe944009b40 tx=0x7fe944009190 comp rx=0 tx=0).stop 2024-03-06T07:20:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 msgr2=0x7fe94c1ea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009e80 tx=0x7fe93c004a40 comp rx=0 tx=0).stop 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 shutdown_connections 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c156ea0 0x7fe94c1ea6d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe94c1506b0 0x7fe94c0c1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 --2- 172.21.15.17:0/2673863369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe94c0acc00 0x7fe94c1e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 >> 172.21.15.17:0/2673863369 conn(0x7fe94c06b1a0 msgr2=0x7fe94c152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 shutdown_connections 2024-03-06T07:20:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.623+0000 7fe9518df640 1 -- 172.21.15.17:0/2673863369 wait complete. 2024-03-06T07:20:51.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:20:02.021234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:20:03.040453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-06T07:20:06.752436+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-06T07:20:51.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:20:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.740+0000 7f98a630c640 1 Processor -- start 2024-03-06T07:20:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.740+0000 7f98a630c640 1 -- start start 2024-03-06T07:20:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a630c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98a00b00d0 0x7f98a00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a630c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 0x7f98a006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a630c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a00b04b0 con 0x7f98a00c7210 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a630c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a0156160 con 0x7f98a00b00d0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a630c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00c5710 con 0x7f98a006b8e0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98a00b00d0 0x7f98a00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a48a9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f98a00c7210 0x7f98a00c76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52002/0 (socket says 172.21.15.17:52002) 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f98a48a9640 1 -- 172.21.15.17:0/2317663508 learned_addr learned my addr 172.21.15.17:0/2317663508 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989f7fe640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 0x7f98a006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989ffff640 1 -- 172.21.15.17:0/2317663508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 msgr2=0x7f98a006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989ffff640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 0x7f98a006bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989ffff640 1 -- 172.21.15.17:0/2317663508 >> v1:172.21.15.17:6789/0 conn(0x7f98a00c7210 legacy=0x7f98a00c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989ffff640 1 -- 172.21.15.17:0/2317663508 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a006d210 con 0x7f98a00b00d0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989effd640 1 -- 172.21.15.17:0/2317663508 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733416931 0 0) 0x7f98a00b04b0 con 0x7f98a00c7210 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.741+0000 7f989f7fe640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 0x7f98a006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.742+0000 7f989ffff640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00b00d0 0x7f98a00c8ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f98940097a0 tx=0x7f989402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f687348275864 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.742+0000 7f989effd640 1 -- 172.21.15.17:0/2317663508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894035020 con 0x7f98a00b00d0 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.742+0000 7f989effd640 1 -- 172.21.15.17:0/2317663508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9894031830 con 0x7f98a00b00d0 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.742+0000 7f989effd640 1 -- 172.21.15.17:0/2317663508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9894031c00 con 0x7f98a00b00d0 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 -- 172.21.15.17:0/2317663508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00b00d0 msgr2=0x7f98a00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00b00d0 0x7f98a00c8ff0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f98940097a0 tx=0x7f989402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 -- 172.21.15.17:0/2317663508 shutdown_connections 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a006b8e0 0x7f98a006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 --2- 172.21.15.17:0/2317663508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00b00d0 0x7f98a00c8ff0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 -- 172.21.15.17:0/2317663508 >> 172.21.15.17:0/2317663508 conn(0x7f98a00ac810 msgr2=0x7f98a00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 -- 172.21.15.17:0/2317663508 shutdown_connections 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.743+0000 7f98a630c640 1 -- 172.21.15.17:0/2317663508 wait complete. 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.744+0000 7f98a630c640 1 Processor -- start 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.744+0000 7f98a630c640 1 -- start start 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a006b8e0 0x7f98a00c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a00b00d0 0x7f98a00c0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a006c260 con 0x7f98a00c7210 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0152c50 con 0x7f98a006b8e0 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00c5ba0 con 0x7f98a00b00d0 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f989f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a00b00d0 0x7f98a00c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33834/0 (socket says 172.21.15.17:33834) 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 -- 172.21.15.17:0/2883259837 learned_addr learned my addr 172.21.15.17:0/2883259837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f989ffff640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a006b8e0 0x7f98a00c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 -- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a00b00d0 msgr2=0x7f98a00c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a00b00d0 0x7f98a00c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 -- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a006b8e0 msgr2=0x7f98a00c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a006b8e0 0x7f98a00c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 -- 172.21.15.17:0/2883259837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a01ebd30 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a48a9640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f989800c700 tx=0x7f989800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898018070 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989800ce10 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.745+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a00c4890 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.746+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800a860 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.746+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0159700 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.747+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a004f330 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.747+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f989801b070 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.747+0000 7f989cff9640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.747+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98980899b0 con 0x7f98a00c7210 2024-03-06T07:20:51.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.748+0000 7f989ffff640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.748+0000 7f989ffff640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 0x7f988c06d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9894036000 tx=0x7f989403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:51.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.752+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9898057e90 con 0x7f98a00c7210 2024-03-06T07:20:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.931+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98a01528f0 con 0x7f98a00c7210 2024-03-06T07:20:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.931+0000 7f989cff9640 1 -- 172.21.15.17:0/2883259837 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f989805bf30 con 0x7f98a00c7210 2024-03-06T07:20:51.934 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:20:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 msgr2=0x7f988c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 0x7f988c06d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9894036000 tx=0x7f989403d040 comp rx=0 tx=0).stop 2024-03-06T07:20:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 msgr2=0x7f98a00c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f989800c700 tx=0x7f989800cbc0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 shutdown_connections 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988c06b500 0x7f988c06d9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98a00c7210 0x7f98a00c0750 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a00b00d0 0x7f98a00c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 --2- 172.21.15.17:0/2883259837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a006b8e0 0x7f98a00c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 >> 172.21.15.17:0/2883259837 conn(0x7f98a00ac810 msgr2=0x7f98a00af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 shutdown_connections 2024-03-06T07:20:51.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:51.934+0000 7f98a630c640 1 -- 172.21.15.17:0/2883259837 wait complete. 2024-03-06T07:20:51.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:20:52.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.048+0000 7f3df857a640 1 Processor -- start 2024-03-06T07:20:52.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.048+0000 7f3df857a640 1 -- start start 2024-03-06T07:20:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df857a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df857a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df0152cb0 con 0x7f3df0072a10 2024-03-06T07:20:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df857a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df00c38b0 con 0x7f3df00c67f0 2024-03-06T07:20:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df857a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01594f0 con 0x7f3df01528d0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df5b15640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3df0072a10 0x7f3df0155220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52014/0 (socket says 172.21.15.17:52014) 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df5b15640 1 -- 172.21.15.17:0/2749266750 learned_addr learned my addr 172.21.15.17:0/2749266750 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.049+0000 7f3df6316640 1 --2- 172.21.15.17:0/2749266750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228665985 0 0) 0x7f3df0152cb0 con 0x7f3df0072a10 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd8003520 con 0x7f3df0072a10 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947715959 0 0) 0x7f3df00c38b0 con 0x7f3df00c67f0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df6316640 1 -- 172.21.15.17:0/2749266750 >> v1:172.21.15.163:6789/0 conn(0x7f3df00c67f0 legacy=0x7f3df015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df6316640 1 -- 172.21.15.17:0/2749266750 >> v1:172.21.15.17:6789/0 conn(0x7f3df0072a10 legacy=0x7f3df0155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.050+0000 7f3df6316640 1 -- 172.21.15.17:0/2749266750 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df00a7f70 con 0x7f3df01528d0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df6316640 1 --2- 172.21.15.17:0/2749266750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00724d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3de80097d0 tx=0x7f3de802b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4cbbe1c85469fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8034070 con 0x7f3df01528d0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de8030880 con 0x7f3df01528d0 2024-03-06T07:20:52.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df5314640 1 -- 172.21.15.17:0/2749266750 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8030c50 con 0x7f3df01528d0 2024-03-06T07:20:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 -- 172.21.15.17:0/2749266750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 msgr2=0x7f3df00724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 --2- 172.21.15.17:0/2749266750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00724d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3de80097d0 tx=0x7f3de802b9c0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 -- 172.21.15.17:0/2749266750 shutdown_connections 2024-03-06T07:20:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 --2- 172.21.15.17:0/2749266750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00724d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 -- 172.21.15.17:0/2749266750 >> 172.21.15.17:0/2749266750 conn(0x7f3df00aa6a0 msgr2=0x7f3df00aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 -- 172.21.15.17:0/2749266750 shutdown_connections 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.051+0000 7f3df857a640 1 -- 172.21.15.17:0/2749266750 wait complete. 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 Processor -- start 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 -- start start 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00c67f0 0x7f3df01eb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01591a0 con 0x7f3df0072a10 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00a8250 con 0x7f3df00c67f0 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df857a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00c4ef0 con 0x7f3df01528d0 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df6316640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df6316640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33836/0 (socket says 172.21.15.17:33836) 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df6316640 1 -- 172.21.15.17:0/1836887419 learned_addr learned my addr 172.21.15.17:0/1836887419 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df5b15640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00c67f0 0x7f3df01eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.053+0000 7f3df6b17640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 -- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 msgr2=0x7f3df00c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 -- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00c67f0 msgr2=0x7f3df01eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00c67f0 0x7f3df01eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 -- 172.21.15.17:0/1836887419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df01ede60 con 0x7f3df0072a10 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df6316640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3de8000c00 tx=0x7f3de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8049070 con 0x7f3df0072a10 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de803ab10 con 0x7f3df0072a10 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8034040 con 0x7f3df0072a10 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df01510e0 con 0x7f3df0072a10 2024-03-06T07:20:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.054+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df01516a0 con 0x7f3df0072a10 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.055+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df004f330 con 0x7f3df0072a10 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.056+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3de804c070 con 0x7f3df0072a10 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.056+0000 7f3deeffd640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 0x7f3ddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.056+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3de80ac570 con 0x7f3df0072a10 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.056+0000 7f3df5b15640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 0x7f3ddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.057+0000 7f3df5b15640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 0x7f3ddc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3df014c910 tx=0x7f3de4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.060+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3de803f070 con 0x7f3df0072a10 2024-03-06T07:20:52.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.238+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3df0073120 con 0x7f3df0072a10 2024-03-06T07:20:52.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.239+0000 7f3deeffd640 1 -- 172.21.15.17:0/1836887419 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3de807ac10 con 0x7f3df0072a10 2024-03-06T07:20:52.240 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:52.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 msgr2=0x7f3ddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 0x7f3ddc06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3df014c910 tx=0x7f3de4009040 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 msgr2=0x7f3df00aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3de8000c00 tx=0x7f3de80046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 shutdown_connections 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ddc06b4d0 0x7f3ddc06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3df01528d0 0x7f3df00c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3df00c67f0 0x7f3df01eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 --2- 172.21.15.17:0/1836887419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3df0072a10 0x7f3df00aba30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.240+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 >> 172.21.15.17:0/1836887419 conn(0x7f3df00aa6a0 msgr2=0x7f3df014ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.241+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 shutdown_connections 2024-03-06T07:20:52.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.241+0000 7f3df857a640 1 -- 172.21.15.17:0/1836887419 wait complete. 2024-03-06T07:20:52.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:20:52.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.355+0000 7fc4c2f65640 1 Processor -- start 2024-03-06T07:20:52.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.355+0000 7fc4c2f65640 1 -- start start 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c2f65640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c2f65640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4bc06cad0 0x7fc4bc154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c2f65640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4bc0c2790 con 0x7fc4bc0c1820 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c2f65640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc0c3090 con 0x7fc4bc14ef70 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c2f65640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4bc14f350 con 0x7fc4bc06cad0 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33850/0 (socket says 172.21.15.17:33850) 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 -- 172.21.15.17:0/355622155 learned_addr learned my addr 172.21.15.17:0/355622155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c1502640 1 --2- 172.21.15.17:0/355622155 >> v2:172.21.15.163:3300/0 conn(0x7fc4bc06cad0 0x7fc4bc154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 -- 172.21.15.17:0/355622155 >> v2:172.21.15.163:3300/0 conn(0x7fc4bc06cad0 msgr2=0x7fc4bc154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 --2- 172.21.15.17:0/355622155 >> v2:172.21.15.163:3300/0 conn(0x7fc4bc06cad0 0x7fc4bc154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 -- 172.21.15.17:0/355622155 >> v1:172.21.15.163:6789/0 conn(0x7fc4bc0c1820 legacy=0x7fc4bc0c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.356+0000 7fc4c0d01640 1 -- 172.21.15.17:0/355622155 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc06cf40 con 0x7fc4bc14ef70 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.357+0000 7fc4c0d01640 1 --2- 172.21.15.17:0/355622155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0c76b0 tx=0x7fc4b002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4077ee7001b298f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.357+0000 7fc4bb7fe640 1 -- 172.21.15.17:0/355622155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b003b070 con 0x7fc4bc14ef70 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.357+0000 7fc4bb7fe640 1 -- 172.21.15.17:0/355622155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b0036440 con 0x7fc4bc14ef70 2024-03-06T07:20:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.357+0000 7fc4bb7fe640 1 -- 172.21.15.17:0/355622155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0036810 con 0x7fc4bc14ef70 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 -- 172.21.15.17:0/355622155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 msgr2=0x7fc4bc06c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/355622155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0c76b0 tx=0x7fc4b002e0e0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 -- 172.21.15.17:0/355622155 shutdown_connections 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/355622155 >> v2:172.21.15.163:3300/0 conn(0x7fc4bc06cad0 0x7fc4bc154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/355622155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc06c1f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 -- 172.21.15.17:0/355622155 >> 172.21.15.17:0/355622155 conn(0x7fc4bc0c6710 msgr2=0x7fc4bc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 -- 172.21.15.17:0/355622155 shutdown_connections 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.358+0000 7fc4c2f65640 1 -- 172.21.15.17:0/355622155 wait complete. 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.359+0000 7fc4c2f65640 1 Processor -- start 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.359+0000 7fc4c2f65640 1 -- start start 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4bc06cad0 0x7fc4bc1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4bc0c1820 0x7fc4bc153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc155750 con 0x7fc4bc14ef70 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc0c24b0 con 0x7fc4bc0c1820 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c2f65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc155170 con 0x7fc4bc06cad0 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33858/0 (socket says 172.21.15.17:33858) 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 -- 172.21.15.17:0/3045295152 learned_addr learned my addr 172.21.15.17:0/3045295152 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c0d01640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4bc06cad0 0x7fc4bc1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4bbfff640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4bc0c1820 0x7fc4bc153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 -- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4bc06cad0 msgr2=0x7fc4bc1511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4bc06cad0 0x7fc4bc1511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 -- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4bc0c1820 msgr2=0x7fc4bc153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4bc0c1820 0x7fc4bc153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 -- 172.21.15.17:0/3045295152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc1e66f0 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.360+0000 7fc4c1502640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009dd0 tx=0x7fc4b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.361+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4018020 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.361+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4014480 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.361+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc1506c0 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.361+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4014870 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.361+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc153cc0 con 0x7fc4bc14ef70 2024-03-06T07:20:52.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.362+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc04f330 con 0x7fc4bc14ef70 2024-03-06T07:20:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.362+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4b4005d10 con 0x7fc4bc14ef70 2024-03-06T07:20:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.362+0000 7fc4b97fa640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.362+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4b408b290 con 0x7fc4bc14ef70 2024-03-06T07:20:52.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.362+0000 7fc4c0d01640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.363+0000 7fc4c0d01640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 0x7fc4a006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0038000 tx=0x7fc4b0032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.367+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc4b4022030 con 0x7fc4bc14ef70 2024-03-06T07:20:52.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.551+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4bc0c1110 con 0x7fc4bc14ef70 2024-03-06T07:20:52.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.551+0000 7fc4b97fa640 1 -- 172.21.15.17:0/3045295152 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4b4076e20 con 0x7fc4bc14ef70 2024-03-06T07:20:52.553 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 msgr2=0x7fc4a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 0x7fc4a006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0038000 tx=0x7fc4b0032000 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 msgr2=0x7fc4bc0a6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009dd0 tx=0x7fc4b400c490 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 shutdown_connections 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4a006b4d0 0x7fc4a006d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4bc14ef70 0x7fc4bc0a6d00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4bc0c1820 0x7fc4bc153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 --2- 172.21.15.17:0/3045295152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4bc06cad0 0x7fc4bc1511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 >> 172.21.15.17:0/3045295152 conn(0x7fc4bc0c6710 msgr2=0x7fc4bc0c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 shutdown_connections 2024-03-06T07:20:52.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.554+0000 7fc4c2f65640 1 -- 172.21.15.17:0/3045295152 wait complete. 2024-03-06T07:20:52.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-06T07:20:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.669+0000 7fe797f8a640 1 Processor -- start 2024-03-06T07:20:52.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.669+0000 7fe797f8a640 1 -- start start 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe797f8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7900b0070 0x7fe790155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe797f8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe79006ca70 0x7fe7900c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe797f8a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7900b0450 con 0x7fe7900c5380 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe797f8a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79006c020 con 0x7fe7900b0070 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe797f8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79006cf20 con 0x7fe79006ca70 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe795d26640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7900b0070 0x7fe790155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe796527640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7900c5380 0x7fe79014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52024/0 (socket says 172.21.15.17:52024) 2024-03-06T07:20:52.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe796527640 1 -- 172.21.15.17:0/3805191841 learned_addr learned my addr 172.21.15.17:0/3805191841 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe795525640 1 --2- 172.21.15.17:0/3805191841 >> v2:172.21.15.163:3300/0 conn(0x7fe79006ca70 0x7fe7900c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe795d26640 1 -- 172.21.15.17:0/3805191841 >> v2:172.21.15.163:3300/0 conn(0x7fe79006ca70 msgr2=0x7fe7900c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe795d26640 1 --2- 172.21.15.17:0/3805191841 >> v2:172.21.15.163:3300/0 conn(0x7fe79006ca70 0x7fe7900c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe795d26640 1 -- 172.21.15.17:0/3805191841 >> v1:172.21.15.17:6789/0 conn(0x7fe7900c5380 legacy=0x7fe79014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.670+0000 7fe794d24640 1 -- 172.21.15.17:0/3805191841 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226597923 0 0) 0x7fe780001490 con 0x7fe7900c5380 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.671+0000 7fe795d26640 1 -- 172.21.15.17:0/3805191841 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900c77f0 con 0x7fe7900b0070 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.671+0000 7fe795d26640 1 --2- 172.21.15.17:0/3805191841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7900b0070 0x7fe790155c30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe79006d4c0 tx=0x7fe78802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849f0243342143f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.671+0000 7fe794d24640 1 -- 172.21.15.17:0/3805191841 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880315a0 con 0x7fe7900b0070 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.671+0000 7fe794d24640 1 -- 172.21.15.17:0/3805191841 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788037070 con 0x7fe7900b0070 2024-03-06T07:20:52.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.671+0000 7fe794d24640 1 -- 172.21.15.17:0/3805191841 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788031960 con 0x7fe7900b0070 2024-03-06T07:20:52.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 -- 172.21.15.17:0/3805191841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7900b0070 msgr2=0x7fe790155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 --2- 172.21.15.17:0/3805191841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7900b0070 0x7fe790155c30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe79006d4c0 tx=0x7fe78802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:52.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 -- 172.21.15.17:0/3805191841 shutdown_connections 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 --2- 172.21.15.17:0/3805191841 >> v2:172.21.15.163:3300/0 conn(0x7fe79006ca70 0x7fe7900c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 --2- 172.21.15.17:0/3805191841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7900b0070 0x7fe790155c30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 -- 172.21.15.17:0/3805191841 >> 172.21.15.17:0/3805191841 conn(0x7fe7900ac590 msgr2=0x7fe7900ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 -- 172.21.15.17:0/3805191841 shutdown_connections 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.672+0000 7fe797f8a640 1 -- 172.21.15.17:0/3805191841 wait complete. 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.673+0000 7fe797f8a640 1 Processor -- start 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.673+0000 7fe797f8a640 1 -- start start 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7900b0070 0x7fe7900c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7900c5380 0x7fe790154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79006c550 con 0x7fe79006ca70 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79006c270 con 0x7fe7900b0070 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe797f8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79006da80 con 0x7fe7900c5380 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33878/0 (socket says 172.21.15.17:33878) 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 -- 172.21.15.17:0/871839682 learned_addr learned my addr 172.21.15.17:0/871839682 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795525640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7900b0070 0x7fe7900c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe796527640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7900c5380 0x7fe790154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 -- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7900c5380 msgr2=0x7fe790154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7900c5380 0x7fe790154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 -- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7900b0070 msgr2=0x7fe7900c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7900b0070 0x7fe7900c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 -- 172.21.15.17:0/871839682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79014e760 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.674+0000 7fe795d26640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788034040 tx=0x7fe7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.675+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788004230 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.675+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7880043d0 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.675+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7900c4200 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.675+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788003850 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.675+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900ca510 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.676+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79004f330 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.676+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7880039f0 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.677+0000 7fe78e7fc640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 0x7fe77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.677+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe78802f030 con 0x7fe79006ca70 2024-03-06T07:20:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.677+0000 7fe795525640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 0x7fe77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.677+0000 7fe795525640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 0x7fe77c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7900c8d70 tx=0x7fe784009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.681+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe788041070 con 0x7fe79006ca70 2024-03-06T07:20:52.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.863+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe79006d860 con 0x7fe79006ca70 2024-03-06T07:20:52.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.863+0000 7fe78e7fc640 1 -- 172.21.15.17:0/871839682 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe78807aad0 con 0x7fe79006ca70 2024-03-06T07:20:52.865 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:20:52.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 msgr2=0x7fe77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 0x7fe77c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe7900c8d70 tx=0x7fe784009190 comp rx=0 tx=0).stop 2024-03-06T07:20:52.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 msgr2=0x7fe7900c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe788034040 tx=0x7fe7880046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 shutdown_connections 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe77c06b4d0 0x7fe77c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7900c5380 0x7fe790154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7900b0070 0x7fe7900c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 --2- 172.21.15.17:0/871839682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe79006ca70 0x7fe7900c3460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 >> 172.21.15.17:0/871839682 conn(0x7fe7900ac590 msgr2=0x7fe790151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 shutdown_connections 2024-03-06T07:20:52.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.865+0000 7fe797f8a640 1 -- 172.21.15.17:0/871839682 wait complete. 2024-03-06T07:20:52.877 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.977+0000 7f96c6ac2640 1 Processor -- start 2024-03-06T07:20:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.977+0000 7f96c6ac2640 1 -- start start 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c6ac2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c6ac2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96c00c36e0 0x7f96c01558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c6ac2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c0158800 con 0x7f96c00c3280 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c6ac2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00c3b10 con 0x7f96c00c36e0 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c6ac2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0158ae0 con 0x7f96c006d9f0 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96bffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96c00c3280 0x7f96c01551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52028/0 (socket says 172.21.15.17:52028) 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96bffff640 1 -- 172.21.15.17:0/1392320783 learned_addr learned my addr 172.21.15.17:0/1392320783 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c505f640 1 --2- 172.21.15.17:0/1392320783 >> v2:172.21.15.163:3300/0 conn(0x7f96c00c36e0 0x7f96c01558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c485e640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96bf7fe640 1 -- 172.21.15.17:0/1392320783 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244460962 0 0) 0x7f96c0158800 con 0x7f96c00c3280 2024-03-06T07:20:52.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96bf7fe640 1 -- 172.21.15.17:0/1392320783 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a8003520 con 0x7f96c00c3280 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c505f640 1 -- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 msgr2=0x7f96c00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c505f640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c00a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c505f640 1 -- 172.21.15.17:0/1392320783 >> v1:172.21.15.17:6789/0 conn(0x7f96c00c3280 legacy=0x7f96c01551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.978+0000 7f96c505f640 1 -- 172.21.15.17:0/1392320783 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0158dc0 con 0x7f96c00c36e0 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.979+0000 7f96c505f640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c36e0 0x7f96c01558f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222dc61acd9b7984 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.979+0000 7f96bf7fe640 1 -- 172.21.15.17:0/1392320783 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00315a0 con 0x7f96c00c36e0 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.979+0000 7f96bf7fe640 1 -- 172.21.15.17:0/1392320783 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0037070 con 0x7f96c00c36e0 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.979+0000 7f96bf7fe640 1 -- 172.21.15.17:0/1392320783 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0031940 con 0x7f96c00c36e0 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 -- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c36e0 msgr2=0x7f96c01558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c36e0 0x7f96c01558f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f96b00097a0 tx=0x7f96b002b840 comp rx=0 tx=0).stop 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 -- 172.21.15.17:0/1392320783 shutdown_connections 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c36e0 0x7f96c01558f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/1392320783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 -- 172.21.15.17:0/1392320783 >> 172.21.15.17:0/1392320783 conn(0x7f96c00aa630 msgr2=0x7f96c00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:52.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 -- 172.21.15.17:0/1392320783 shutdown_connections 2024-03-06T07:20:52.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.980+0000 7f96c6ac2640 1 -- 172.21.15.17:0/1392320783 wait complete. 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.981+0000 7f96c6ac2640 1 Processor -- start 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.981+0000 7f96c6ac2640 1 -- start start 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c01ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c3280 0x7f96c0143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0155e30 con 0x7f96c00c36e0 2024-03-06T07:20:52.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0159380 con 0x7f96c00c3280 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c6ac2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c006ddd0 con 0x7f96c006d9f0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33892/0 (socket says 172.21.15.17:33892) 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 -- 172.21.15.17:0/79688914 learned_addr learned my addr 172.21.15.17:0/79688914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c485e640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c01ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96bffff640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c3280 0x7f96c0143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 -- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 msgr2=0x7f96c01ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c01ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 -- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c3280 msgr2=0x7f96c0143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c3280 0x7f96c0143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 -- 172.21.15.17:0/79688914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c0151600 con 0x7f96c00c36e0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.982+0000 7f96c505f640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96b002b880 tx=0x7f96b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.983+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0032430 con 0x7f96c00c36e0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.983+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b00325d0 con 0x7f96c00c36e0 2024-03-06T07:20:52.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.983+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c0144bf0 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.983+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0032980 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.983+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c01582e0 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.984+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c0158520 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.984+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96b0032b20 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.985+0000 7f96bd7fa640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 0x7f96ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.985+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f96b002f030 con 0x7f96c00c36e0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.985+0000 7f96c485e640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 0x7f96ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:52.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.985+0000 7f96c485e640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 0x7f96ac06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011340 tx=0x7f96b800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:52.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:52.989+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96c0158520 con 0x7f96c00c36e0 2024-03-06T07:20:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.206+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96c004f330 con 0x7f96c00c36e0 2024-03-06T07:20:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.207+0000 7f96bd7fa640 1 -- 172.21.15.17:0/79688914 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f96b0040070 con 0x7f96c00c36e0 2024-03-06T07:20:53.209 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:53.209 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:53.209 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 msgr2=0x7f96ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 0x7f96ac06d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f96b8011340 tx=0x7f96b800b040 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 msgr2=0x7f96c01442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f96b002b880 tx=0x7f96b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 shutdown_connections 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96ac06b4d0 0x7f96ac06d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96c00c36e0 0x7f96c01442e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00c3280 0x7f96c0143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 --2- 172.21.15.17:0/79688914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c006d9f0 0x7f96c01ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 >> 172.21.15.17:0/79688914 conn(0x7f96c00aa630 msgr2=0x7f96c0154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 shutdown_connections 2024-03-06T07:20:53.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.209+0000 7f96c6ac2640 1 -- 172.21.15.17:0/79688914 wait complete. 2024-03-06T07:20:53.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.322+0000 7f50f9464640 1 Processor -- start 2024-03-06T07:20:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.322+0000 7f50f9464640 1 -- start start 2024-03-06T07:20:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f9464640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50f4152920 0x7f50f4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f9464640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f406afc0 con 0x7f50f40c2190 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f9464640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f406b2a0 con 0x7f50f40c2620 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f9464640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f4156160 con 0x7f50f4152920 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50f4152920 0x7f50f4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f27fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50f40c2190 0x7f50f40c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52042/0 (socket says 172.21.15.17:52042) 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.323+0000 7f50f27fc640 1 -- 172.21.15.17:0/417282261 learned_addr learned my addr 172.21.15.17:0/417282261 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:53.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f2ffd640 1 -- 172.21.15.17:0/417282261 >> v1:172.21.15.163:6789/0 conn(0x7f50f40c2620 legacy=0x7f50f415fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f2ffd640 1 -- 172.21.15.17:0/417282261 >> v1:172.21.15.17:6789/0 conn(0x7f50f40c2190 legacy=0x7f50f40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f2ffd640 1 -- 172.21.15.17:0/417282261 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f406b580 con 0x7f50f4152920 2024-03-06T07:20:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f2ffd640 1 --2- 172.21.15.17:0/417282261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f4155c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0097a0 tx=0x7f50ec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f90998a3e5eb3fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f1ffb640 1 -- 172.21.15.17:0/417282261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec034070 con 0x7f50f4152920 2024-03-06T07:20:53.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f1ffb640 1 -- 172.21.15.17:0/417282261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec030610 con 0x7f50f4152920 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.324+0000 7f50f1ffb640 1 -- 172.21.15.17:0/417282261 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0309e0 con 0x7f50f4152920 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 -- 172.21.15.17:0/417282261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 msgr2=0x7f50f4155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 --2- 172.21.15.17:0/417282261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f4155c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0097a0 tx=0x7f50ec02b960 comp rx=0 tx=0).stop 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 -- 172.21.15.17:0/417282261 shutdown_connections 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 --2- 172.21.15.17:0/417282261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f4155c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 -- 172.21.15.17:0/417282261 >> 172.21.15.17:0/417282261 conn(0x7f50f40c4290 msgr2=0x7f50f40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 -- 172.21.15.17:0/417282261 shutdown_connections 2024-03-06T07:20:53.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.325+0000 7f50f9464640 1 -- 172.21.15.17:0/417282261 wait complete. 2024-03-06T07:20:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.326+0000 7f50f9464640 1 Processor -- start 2024-03-06T07:20:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.326+0000 7f50f9464640 1 -- start start 2024-03-06T07:20:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f40c2190 0x7f50f41eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f40c2620 0x7f50f4153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40a7100 con 0x7f50f4152920 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40c0d60 con 0x7f50f40c2190 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f9464640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40a7280 con 0x7f50f40c2620 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33910/0 (socket says 172.21.15.17:33910) 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 -- 172.21.15.17:0/3250370318 learned_addr learned my addr 172.21.15.17:0/3250370318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f27fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f40c2620 0x7f50f4153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f2ffd640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f40c2190 0x7f50f41eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 -- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f40c2620 msgr2=0x7f50f4153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f40c2620 0x7f50f4153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 -- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f40c2190 msgr2=0x7f50f41eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f40c2190 0x7f50f41eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 -- 172.21.15.17:0/3250370318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4156880 con 0x7f50f4152920 2024-03-06T07:20:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.327+0000 7f50f37fe640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c710 tx=0x7f50e400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.328+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4016020 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.328+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4006d50 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.328+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003bb0 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.328+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f4156b60 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.328+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f41eeb80 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.329+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4067ad0 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.329+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50e400a850 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.330+0000 7f50eb7fe640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.330+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f50e408a0f0 con 0x7f50f4152920 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.330+0000 7f50f2ffd640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.330+0000 7f50f2ffd640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 0x7f50d406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50ec036000 tx=0x7f50ec03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.334+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50f4067ad0 con 0x7f50f4152920 2024-03-06T07:20:53.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.525+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f4075300 con 0x7f50f4152920 2024-03-06T07:20:53.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.526+0000 7f50eb7fe640 1 -- 172.21.15.17:0/3250370318 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19043 (secure 0 0 0) 0x7f50f4075300 con 0x7f50f4152920 2024-03-06T07:20:53.528 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:53.528 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:53.373723+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 msgr2=0x7f50d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 0x7f50d406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50ec036000 tx=0x7f50ec03b040 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 msgr2=0x7f50f40a83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c710 tx=0x7f50e400cbd0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 shutdown_connections 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50d406b4d0 0x7f50d406d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50f4152920 0x7f50f40a83c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50f40c2620 0x7f50f4153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 --2- 172.21.15.17:0/3250370318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50f40c2190 0x7f50f41eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 >> 172.21.15.17:0/3250370318 conn(0x7f50f40c4290 msgr2=0x7f50f4150b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 shutdown_connections 2024-03-06T07:20:53.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.528+0000 7f50f9464640 1 -- 172.21.15.17:0/3250370318 wait complete. 2024-03-06T07:20:53.541 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:20:53.541 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.640+0000 7f15c2313640 1 Processor -- start 2024-03-06T07:20:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.640+0000 7f15c2313640 1 -- start start 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15c2313640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15bc06afc0 0x7f15bc06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15c2313640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15bc06b3a0 0x7f15bc152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15c2313640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc153940 con 0x7f15bc1534f0 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15c2313640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc06b830 con 0x7f15bc06afc0 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15c2313640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc150a20 con 0x7f15bc06b3a0 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15bbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15bc06afc0 0x7f15bc06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15bbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15bc06afc0 0x7f15bc06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33914/0 (socket says 172.21.15.17:33914) 2024-03-06T07:20:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15bbfff640 1 -- 172.21.15.17:0/1742366569 learned_addr learned my addr 172.21.15.17:0/1742366569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.641+0000 7f15bb7fe640 1 --2- 172.21.15.17:0/1742366569 >> v2:172.21.15.163:3300/0 conn(0x7f15bc06b3a0 0x7f15bc152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.642+0000 7f15bbfff640 1 -- 172.21.15.17:0/1742366569 >> v2:172.21.15.163:3300/0 conn(0x7f15bc06b3a0 msgr2=0x7f15bc152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.642+0000 7f15bbfff640 1 --2- 172.21.15.17:0/1742366569 >> v2:172.21.15.163:3300/0 conn(0x7f15bc06b3a0 0x7f15bc152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.642+0000 7f15bbfff640 1 -- 172.21.15.17:0/1742366569 >> v1:172.21.15.163:6789/0 conn(0x7f15bc1534f0 legacy=0x7f15bc0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.642+0000 7f15bbfff640 1 -- 172.21.15.17:0/1742366569 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0a70f0 con 0x7f15bc06afc0 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.642+0000 7f15bbfff640 1 --2- 172.21.15.17:0/1742366569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc06f100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009870 tx=0x7f15b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c607370e6f45b14f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15baffd640 1 -- 172.21.15.17:0/1742366569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b002e070 con 0x7f15bc06afc0 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15baffd640 1 -- 172.21.15.17:0/1742366569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0037070 con 0x7f15bc06afc0 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15baffd640 1 -- 172.21.15.17:0/1742366569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0031a00 con 0x7f15bc06afc0 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 -- 172.21.15.17:0/1742366569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 msgr2=0x7f15bc06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 --2- 172.21.15.17:0/1742366569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc06f100 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009870 tx=0x7f15b002b880 comp rx=0 tx=0).stop 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 -- 172.21.15.17:0/1742366569 shutdown_connections 2024-03-06T07:20:53.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 --2- 172.21.15.17:0/1742366569 >> v2:172.21.15.163:3300/0 conn(0x7f15bc06b3a0 0x7f15bc152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 --2- 172.21.15.17:0/1742366569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc06f100 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 -- 172.21.15.17:0/1742366569 >> 172.21.15.17:0/1742366569 conn(0x7f15bc0c4290 msgr2=0x7f15bc0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 -- 172.21.15.17:0/1742366569 shutdown_connections 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.643+0000 7f15c2313640 1 -- 172.21.15.17:0/1742366569 wait complete. 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.644+0000 7f15c2313640 1 Processor -- start 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 -- start start 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15bc06b3a0 0x7f15bc1e9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15bc1534f0 0x7f15bc154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc156ad0 con 0x7f15bc06afc0 2024-03-06T07:20:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc1565a0 con 0x7f15bc1534f0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c2313640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc156db0 con 0x7f15bc06b3a0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33922/0 (socket says 172.21.15.17:33922) 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15bbfff640 1 -- 172.21.15.17:0/3091176116 learned_addr learned my addr 172.21.15.17:0/3091176116 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15c08b0640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15bc1534f0 0x7f15bc154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.645+0000 7f15bb7fe640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15bc06b3a0 0x7f15bc1e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 -- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15bc06b3a0 msgr2=0x7f15bc1e9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15bc06b3a0 0x7f15bc1e9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 -- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15bc1534f0 msgr2=0x7f15bc154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15bc1534f0 0x7f15bc154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 -- 172.21.15.17:0/3091176116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc1e8c40 con 0x7f15bc06afc0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15bbfff640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15b0000c00 tx=0x7f15b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0032670 con 0x7f15bc06afc0 2024-03-06T07:20:53.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b0032810 con 0x7f15bc06afc0 2024-03-06T07:20:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b0032c00 con 0x7f15bc06afc0 2024-03-06T07:20:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc152ee0 con 0x7f15bc06afc0 2024-03-06T07:20:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.646+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc1ecf40 con 0x7f15bc06afc0 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.647+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc04f330 con 0x7f15bc06afc0 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.647+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15b004c060 con 0x7f15bc06afc0 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.648+0000 7f15b8ff9640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 0x7f15a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.648+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f15b00abb00 con 0x7f15bc06afc0 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.648+0000 7f15bb7fe640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 0x7f15a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.648+0000 7f15bb7fe640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 0x7f15a806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15bc146e10 tx=0x7f15ac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.652+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15b0040070 con 0x7f15bc06afc0 2024-03-06T07:20:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.873+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15bc068460 con 0x7f15bc06afc0 2024-03-06T07:20:53.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.873+0000 7f15b8ff9640 1 -- 172.21.15.17:0/3091176116 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f15b0079ff0 con 0x7f15bc06afc0 2024-03-06T07:20:53.875 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:53.875 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:53.876 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 msgr2=0x7f15a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 0x7f15a806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15bc146e10 tx=0x7f15ac009190 comp rx=0 tx=0).stop 2024-03-06T07:20:53.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 msgr2=0x7f15bc1555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15b0000c00 tx=0x7f15b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 shutdown_connections 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15a806b500 0x7f15a806d9c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15bc1534f0 0x7f15bc154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15bc06b3a0 0x7f15bc1e9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 --2- 172.21.15.17:0/3091176116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15bc06afc0 0x7f15bc1555e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 >> 172.21.15.17:0/3091176116 conn(0x7f15bc0c4290 msgr2=0x7f15bc073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 shutdown_connections 2024-03-06T07:20:53.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.876+0000 7f15c2313640 1 -- 172.21.15.17:0/3091176116 wait complete. 2024-03-06T07:20:53.888 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:20:53.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.988+0000 7f933a039640 1 Processor -- start 2024-03-06T07:20:53.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.988+0000 7f933a039640 1 -- start start 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f933a039640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f93341554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f933a039640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340aa660 con 0x7f93340aa180 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f933a039640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340aa940 con 0x7f93341559e0 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f933a039640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c4610 con 0x7f93341550c0 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f9333fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f93340aa180 0x7f93340c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52050/0 (socket says 172.21.15.17:52050) 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f9333fff640 1 -- 172.21.15.17:0/572119891 learned_addr learned my addr 172.21.15.17:0/572119891 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:53.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93337fe640 1 --2- 172.21.15.17:0/572119891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f93341554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926014368 0 0) 0x7f93340aa660 con 0x7f93340aa180 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c005730 con 0x7f93340aa180 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399495962 0 0) 0x7f93340aa940 con 0x7f93341559e0 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93340aa660 con 0x7f93341559e0 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93337fe640 1 -- 172.21.15.17:0/572119891 >> v1:172.21.15.163:6789/0 conn(0x7f93341559e0 legacy=0x7f933414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93337fe640 1 -- 172.21.15.17:0/572119891 >> v1:172.21.15.17:6789/0 conn(0x7f93340aa180 legacy=0x7f93340c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.989+0000 7f93337fe640 1 -- 172.21.15.17:0/572119891 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340c4860 con 0x7f93341550c0 2024-03-06T07:20:53.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.990+0000 7f93337fe640 1 --2- 172.21.15.17:0/572119891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f93341554a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93240097d0 tx=0x7f932402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4d0e48b29794e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:53.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.990+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324031770 con 0x7f93341550c0 2024-03-06T07:20:53.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.990+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324037070 con 0x7f93341550c0 2024-03-06T07:20:53.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.990+0000 7f93327fc640 1 -- 172.21.15.17:0/572119891 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324031b10 con 0x7f93341550c0 2024-03-06T07:20:53.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 -- 172.21.15.17:0/572119891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 msgr2=0x7f93341554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:53.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 --2- 172.21.15.17:0/572119891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f93341554a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f93240097d0 tx=0x7f932402b810 comp rx=0 tx=0).stop 2024-03-06T07:20:53.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 -- 172.21.15.17:0/572119891 shutdown_connections 2024-03-06T07:20:53.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 --2- 172.21.15.17:0/572119891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f93341554a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:53.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 -- 172.21.15.17:0/572119891 >> 172.21.15.17:0/572119891 conn(0x7f93340c6710 msgr2=0x7f93340c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:53.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 -- 172.21.15.17:0/572119891 shutdown_connections 2024-03-06T07:20:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.991+0000 7f933a039640 1 -- 172.21.15.17:0/572119891 wait complete. 2024-03-06T07:20:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.992+0000 7f933a039640 1 Processor -- start 2024-03-06T07:20:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.992+0000 7f933a039640 1 -- start start 2024-03-06T07:20:54.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f933414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93341559e0 0x7f933414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933414f310 con 0x7f93340aa180 2024-03-06T07:20:54.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c1520 con 0x7f93341559e0 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f933a039640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933414f490 con 0x7f93341550c0 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33932/0 (socket says 172.21.15.17:33932) 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 -- 172.21.15.17:0/85062596 learned_addr learned my addr 172.21.15.17:0/85062596 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f9332ffd640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f933414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f9333fff640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93341559e0 0x7f933414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 -- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 msgr2=0x7f933414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f933414a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 -- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93341559e0 msgr2=0x7f933414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93341559e0 0x7f933414ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 -- 172.21.15.17:0/85062596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933414f7a0 con 0x7f93340aa180 2024-03-06T07:20:54.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.993+0000 7f93337fe640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f93240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.994+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324031ed0 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.994+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9324032600 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.994+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933414fa80 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.994+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240329f0 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.994+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340a7250 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.995+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933404f330 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.995+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f932404c060 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.995+0000 7f931bfff640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 0x7f932006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.996+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f93240abc00 con 0x7f93340aa180 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.996+0000 7f9332ffd640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 0x7f932006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:53.996+0000 7f9332ffd640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 0x7f932006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9328007fd0 tx=0x7f9328009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.000+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f93240b2080 con 0x7f93340aa180 2024-03-06T07:20:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.219+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9334068450 con 0x7f93340aa180 2024-03-06T07:20:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.220+0000 7f931bfff640 1 -- 172.21.15.17:0/85062596 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4976 (secure 0 0 0) 0x7f9324040070 con 0x7f93340aa180 2024-03-06T07:20:54.222 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:54.222 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1540756516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1540756516},{"type":"v1","addr":"172.21.15.163:6836","nonce":1540756516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2429683368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2429683368},{"type":"v1","addr":"172.21.15.17:6834","nonce":2429683368}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/662249768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":662249768},{"type":"v1","addr":"172.21.15.17:6835","nonce":662249768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:20:04.076240+0000","modified":"2024-03-06T07:20:20.134434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2082625649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2082625649},{"type":"v1","addr":"172.21.15.163:6837","nonce":2082625649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4264,"qdb_cluster":[4264]},"id":1}]} 2024-03-06T07:20:54.223 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 9 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 msgr2=0x7f932006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 0x7f932006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9328007fd0 tx=0x7f9328009040 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 msgr2=0x7f933414bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9324000c00 tx=0x7f93240046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 shutdown_connections 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f932006b4d0 0x7f932006d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93341559e0 0x7f933414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93341550c0 0x7f933414a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 --2- 172.21.15.17:0/85062596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93340aa180 0x7f933414bd60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 >> 172.21.15.17:0/85062596 conn(0x7f93340c6710 msgr2=0x7f93340c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 shutdown_connections 2024-03-06T07:20:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.222+0000 7f933a039640 1 -- 172.21.15.17:0/85062596 wait complete. 2024-03-06T07:20:54.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:20:54.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.334+0000 7faf53e78640 1 Processor -- start 2024-03-06T07:20:54.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.334+0000 7faf53e78640 1 -- start start 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf53e78640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf4c06d9f0 0x7faf4c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf53e78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c0c3280 0x7faf4c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf53e78640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c158800 con 0x7faf4c1551e0 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf53e78640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c158ae0 con 0x7faf4c06d9f0 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf53e78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c158dc0 con 0x7faf4c0c3280 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51c14640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faf4c06d9f0 0x7faf4c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf52415640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faf4c1551e0 0x7faf4c155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52060/0 (socket says 172.21.15.17:52060) 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf52415640 1 -- 172.21.15.17:0/314251932 learned_addr learned my addr 172.21.15.17:0/314251932 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51413640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c0c3280 0x7faf4c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51c14640 1 -- 172.21.15.17:0/314251932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c0c3280 msgr2=0x7faf4c0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:54.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51c14640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c0c3280 0x7faf4c0c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51c14640 1 -- 172.21.15.17:0/314251932 >> v1:172.21.15.17:6789/0 conn(0x7faf4c1551e0 legacy=0x7faf4c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf51c14640 1 -- 172.21.15.17:0/314251932 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c1590a0 con 0x7faf4c06d9f0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.335+0000 7faf50c12640 1 -- 172.21.15.17:0/314251932 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352244645 0 0) 0x7faf4c158800 con 0x7faf4c1551e0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf51c14640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c0a7cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf480097a0 tx=0x7faf4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb91f3b687fceff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf50c12640 1 -- 172.21.15.17:0/314251932 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480315a0 con 0x7faf4c06d9f0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf50c12640 1 -- 172.21.15.17:0/314251932 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48037070 con 0x7faf4c06d9f0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf50c12640 1 -- 172.21.15.17:0/314251932 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48031960 con 0x7faf4c06d9f0 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf53e78640 1 -- 172.21.15.17:0/314251932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 msgr2=0x7faf4c0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.336+0000 7faf53e78640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c0a7cd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf480097a0 tx=0x7faf4802b840 comp rx=0 tx=0).stop 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 -- 172.21.15.17:0/314251932 shutdown_connections 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c0c3280 0x7faf4c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 --2- 172.21.15.17:0/314251932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c0a7cd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 -- 172.21.15.17:0/314251932 >> 172.21.15.17:0/314251932 conn(0x7faf4c0aa630 msgr2=0x7faf4c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 -- 172.21.15.17:0/314251932 shutdown_connections 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.337+0000 7faf53e78640 1 -- 172.21.15.17:0/314251932 wait complete. 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 Processor -- start 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 -- start start 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0c3280 0x7faf4c0a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 0x7faf4c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c06df50 con 0x7faf4c06d9f0 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c06ddd0 con 0x7faf4c0c3280 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf53e78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0a8210 con 0x7faf4c1551e0 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf51c14640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf51c14640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33952/0 (socket says 172.21.15.17:33952) 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.338+0000 7faf51c14640 1 -- 172.21.15.17:0/2586290868 learned_addr learned my addr 172.21.15.17:0/2586290868 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51413640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0c3280 0x7faf4c0a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf52415640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 0x7faf4c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 -- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 msgr2=0x7faf4c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 0x7faf4c1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 -- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0c3280 msgr2=0x7faf4c0a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0c3280 0x7faf4c0a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 -- 172.21.15.17:0/2586290868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c14d520 con 0x7faf4c06d9f0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf52415640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 0x7faf4c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf51c14640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf48034040 tx=0x7faf480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48032630 con 0x7faf4c06d9f0 2024-03-06T07:20:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480327f0 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c158170 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48032be0 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.339+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c158450 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.340+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c04f330 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.340+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf4804e430 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.341+0000 7faf427fc640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 0x7faf3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.341+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6822+0+0 (secure 0 0 0) 0x7faf4802f030 con 0x7faf4c06d9f0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.341+0000 7faf51413640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 0x7faf3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.342+0000 7faf51413640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 0x7faf3006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0c3860 tx=0x7faf3c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.345+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf48040070 con 0x7faf4c06d9f0 2024-03-06T07:20:54.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.526+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf4c1ee660 con 0x7faf4c06d9f0 2024-03-06T07:20:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.527+0000 7faf427fc640 1 -- 172.21.15.17:0/2586290868 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19043 (secure 0 0 0) 0x7faf4807ae50 con 0x7faf4c06d9f0 2024-03-06T07:20:54.530 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:20:54.530 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:53.373723+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:20:02.021234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-06T07:20:03.040453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-06T07:20:06.752436+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.11","mappings":[{"from":4,"to":6}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":3,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 msgr2=0x7faf3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 0x7faf3006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0c3860 tx=0x7faf3c009190 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 msgr2=0x7faf4c14d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faf48034040 tx=0x7faf480046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 shutdown_connections 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf3006b4d0 0x7faf3006d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf4c1551e0 0x7faf4c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0c3280 0x7faf4c0a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 --2- 172.21.15.17:0/2586290868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c06d9f0 0x7faf4c14d800 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 >> 172.21.15.17:0/2586290868 conn(0x7faf4c0aa630 msgr2=0x7faf4c151070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 shutdown_connections 2024-03-06T07:20:54.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.530+0000 7faf53e78640 1 -- 172.21.15.17:0/2586290868 wait complete. 2024-03-06T07:20:54.543 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:20:54.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.643+0000 7f238e6d4640 1 Processor -- start 2024-03-06T07:20:54.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.643+0000 7f238e6d4640 1 -- start start 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238e6d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f2388155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238e6d4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23880c8950 0x7f23880bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238e6d4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880c8e30 con 0x7f23880bfe30 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238e6d4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2388152ce0 con 0x7f23880c8950 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238e6d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c9110 con 0x7f2388152900 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238cc71640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f23880bfe30 0x7f23880c64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52074/0 (socket says 172.21.15.17:52074) 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f238cc71640 1 -- 172.21.15.17:0/1724226901 learned_addr learned my addr 172.21.15.17:0/1724226901 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f2387fff640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f2388155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f23877fe640 1 --2- 172.21.15.17:0/1724226901 >> v2:172.21.15.163:3300/0 conn(0x7f23880c8950 0x7f23880bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f2386ffd640 1 -- 172.21.15.17:0/1724226901 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176556898 0 0) 0x7f23880c8e30 con 0x7f23880bfe30 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f2386ffd640 1 -- 172.21.15.17:0/1724226901 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2370003520 con 0x7f23880bfe30 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f23877fe640 1 -- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 msgr2=0x7f2388155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f23877fe640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f2388155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f23877fe640 1 -- 172.21.15.17:0/1724226901 >> v1:172.21.15.17:6789/0 conn(0x7f23880bfe30 legacy=0x7f23880c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f23877fe640 1 -- 172.21.15.17:0/1724226901 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880c93f0 con 0x7f23880c8950 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.644+0000 7f2387fff640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f2388155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.645+0000 7f23877fe640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880c8950 0x7f23880bf8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097a0 tx=0x7f237c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09484da0aecb755 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.645+0000 7f2386ffd640 1 -- 172.21.15.17:0/1724226901 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c038070 con 0x7f23880c8950 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.645+0000 7f2386ffd640 1 -- 172.21.15.17:0/1724226901 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c031700 con 0x7f23880c8950 2024-03-06T07:20:54.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.645+0000 7f2386ffd640 1 -- 172.21.15.17:0/1724226901 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c031ab0 con 0x7f23880c8950 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 -- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880c8950 msgr2=0x7f23880bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880c8950 0x7f23880bf8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097a0 tx=0x7f237c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 -- 172.21.15.17:0/1724226901 shutdown_connections 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880c8950 0x7f23880bf8f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1724226901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f2388155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 -- 172.21.15.17:0/1724226901 >> 172.21.15.17:0/1724226901 conn(0x7f23880ac860 msgr2=0x7f23880acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 -- 172.21.15.17:0/1724226901 shutdown_connections 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.646+0000 7f238e6d4640 1 -- 172.21.15.17:0/1724226901 wait complete. 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.647+0000 7f238e6d4640 1 Processor -- start 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.647+0000 7f238e6d4640 1 -- start start 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880bfe30 0x7f2388154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f238814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c5770 con 0x7f23880c8950 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c55f0 con 0x7f23880bfe30 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238e6d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c6c00 con 0x7f2388152900 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33958/0 (socket says 172.21.15.17:33958) 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 -- 172.21.15.17:0/1194512050 learned_addr learned my addr 172.21.15.17:0/1194512050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f2387fff640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880bfe30 0x7f2388154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f238cc71640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f238814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 -- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 msgr2=0x7f238814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f238814c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 -- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880bfe30 msgr2=0x7f2388154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880bfe30 0x7f2388154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 -- 172.21.15.17:0/1194512050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388157050 con 0x7f23880c8950 2024-03-06T07:20:54.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.648+0000 7f23877fe640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f237c036040 tx=0x7f237c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.649+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c042020 con 0x7f23880c8950 2024-03-06T07:20:54.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.649+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c03d890 con 0x7f23880c8950 2024-03-06T07:20:54.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.649+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2388157330 con 0x7f23880c8950 2024-03-06T07:20:54.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.649+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c038040 con 0x7f23880c8950 2024-03-06T07:20:54.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.649+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388154870 con 0x7f23880c8950 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.650+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238804f330 con 0x7f23880c8950 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.650+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f237c045020 con 0x7f23880c8950 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.651+0000 7f2384ff9640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 0x7f237406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.651+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f237c0abe30 con 0x7f23880c8950 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.651+0000 7f2387fff640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 0x7f237406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.651+0000 7f2387fff640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 0x7f237406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2378009560 tx=0x7f2378009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.655+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f237c04c030 con 0x7f23880c8950 2024-03-06T07:20:54.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.872+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f238814d420 con 0x7f23880c8950 2024-03-06T07:20:54.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:20:54.872+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:20:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.874+0000 7f2384ff9640 1 -- 172.21.15.17:0/1194512050 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f237c030070 con 0x7f23880c8950 2024-03-06T07:20:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 msgr2=0x7f237406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 0x7f237406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2378009560 tx=0x7f2378009190 comp rx=0 tx=0).stop 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 msgr2=0x7f238814be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f237c036040 tx=0x7f237c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 shutdown_connections 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f237406b4d0 0x7f237406d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2388152900 0x7f238814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23880c8950 0x7f238814be90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 --2- 172.21.15.17:0/1194512050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23880bfe30 0x7f2388154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 >> 172.21.15.17:0/1194512050 conn(0x7f23880ac860 msgr2=0x7f23880acc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 shutdown_connections 2024-03-06T07:20:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.876+0000 7f238e6d4640 1 -- 172.21.15.17:0/1194512050 wait complete. 2024-03-06T07:20:54.884 INFO:tasks.ceph.mds.b.smithi163.stderr: -3528> 2024-03-06T07:20:08.780+0000 7f2d3f13e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:20:54.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:20:54.915 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:20:54.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.990+0000 7fdfa42b1640 1 Processor -- start 2024-03-06T07:20:54.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.990+0000 7fdfa42b1640 1 -- start start 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa42b1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 0x7fdf9c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa42b1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c156650 con 0x7fdf9c153030 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa42b1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c1568f0 con 0x7fdf9c0c6080 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa42b1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c06ec50 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa284e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdf9c153030 0x7fdf9c1534c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52090/0 (socket says 172.21.15.17:52090) 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa284e640 1 -- 172.21.15.17:0/2676982475 learned_addr learned my addr 172.21.15.17:0/2676982475 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:54.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa204d640 1 --2- 172.21.15.17:0/2676982475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 0x7fdf9c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597881228 0 0) 0x7fdf9c156650 con 0x7fdf9c153030 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf88003520 con 0x7fdf9c153030 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735086371 0 0) 0x7fdf9c1568f0 con 0x7fdf9c0c6080 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa204d640 1 -- 172.21.15.17:0/2676982475 >> v1:172.21.15.163:6789/0 conn(0x7fdf9c0c6080 legacy=0x7fdf9c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa204d640 1 -- 172.21.15.17:0/2676982475 >> v1:172.21.15.17:6789/0 conn(0x7fdf9c153030 legacy=0x7fdf9c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.991+0000 7fdfa204d640 1 -- 172.21.15.17:0/2676982475 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c156bd0 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.992+0000 7fdfa204d640 1 --2- 172.21.15.17:0/2676982475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 0x7fdf9c06f7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097d0 tx=0x7fdf9802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791d81779bc9992b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.992+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98035020 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.992+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98031890 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.992+0000 7fdfa104b640 1 -- 172.21.15.17:0/2676982475 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98031c60 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 -- 172.21.15.17:0/2676982475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 msgr2=0x7fdf9c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/2676982475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 0x7fdf9c06f7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097d0 tx=0x7fdf9802b930 comp rx=0 tx=0).stop 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 -- 172.21.15.17:0/2676982475 shutdown_connections 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/2676982475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c0c4e60 0x7fdf9c06f7d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 -- 172.21.15.17:0/2676982475 >> 172.21.15.17:0/2676982475 conn(0x7fdf9c0c5560 msgr2=0x7fdf9c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 -- 172.21.15.17:0/2676982475 shutdown_connections 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.993+0000 7fdfa42b1640 1 -- 172.21.15.17:0/2676982475 wait complete. 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.994+0000 7fdfa42b1640 1 Processor -- start 2024-03-06T07:20:54.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.994+0000 7fdfa42b1640 1 -- start start 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c0c6080 0x7fdf9c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c153030 0x7fdf9c14d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0c87d0 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c157190 con 0x7fdf9c0c6080 2024-03-06T07:20:54.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c072d90 con 0x7fdf9c153030 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33968/0 (socket says 172.21.15.17:33968) 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 -- 172.21.15.17:0/4245101149 learned_addr learned my addr 172.21.15.17:0/4245101149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa284e640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c153030 0x7fdf9c14d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa184c640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c0c6080 0x7fdf9c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 -- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c153030 msgr2=0x7fdf9c14d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c153030 0x7fdf9c14d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 -- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c0c6080 msgr2=0x7fdf9c14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c0c6080 0x7fdf9c14cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 -- 172.21.15.17:0/4245101149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c06f840 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa204d640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098f0 tx=0x7fdf980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98035040 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf98030ca0 con 0x7fdf9c0c4e60 2024-03-06T07:20:54.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c0ac1e0 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98004270 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.995+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c0abc30 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.996+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c04f330 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.997+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf98038020 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.997+0000 7fdf92ffd640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 0x7fdf8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.997+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6908+0+0 (secure 0 0 0) 0x7fdf980ac900 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.997+0000 7fdfa184c640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 0x7fdf8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:54.998+0000 7fdfa184c640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 0x7fdf8006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0c7380 tx=0x7fdf8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:55.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.001+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf9802d040 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.219+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf9c0c8330 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.879+0000 7fdf92ffd640 1 -- 172.21.15.17:0/4245101149 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf9807ac80 con 0x7fdf9c0c4e60 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 msgr2=0x7fdf8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 0x7fdf8006d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0c7380 tx=0x7fdf8c009040 comp rx=0 tx=0).stop 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 msgr2=0x7fdf9c0a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098f0 tx=0x7fdf980046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 shutdown_connections 2024-03-06T07:20:55.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf8006b4d0 0x7fdf8006d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf9c153030 0x7fdf9c14d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf9c0c6080 0x7fdf9c14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 --2- 172.21.15.17:0/4245101149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf9c0c4e60 0x7fdf9c0a9ea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 >> 172.21.15.17:0/4245101149 conn(0x7fdf9c0c5560 msgr2=0x7fdf9c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 shutdown_connections 2024-03-06T07:20:55.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:55.882+0000 7fdfa42b1640 1 -- 172.21.15.17:0/4245101149 wait complete. 2024-03-06T07:20:55.898 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:20:56.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.008+0000 7f7920925640 1 Processor -- start 2024-03-06T07:20:56.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.008+0000 7f7920925640 1 -- start start 2024-03-06T07:20:56.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7920925640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f791c0c88e0 0x7f791c0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7920925640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c06bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7920925640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791c0c5400 con 0x7f791c152870 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7920925640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f791c06d320 con 0x7f791c0c88e0 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7920925640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c06b8d0 con 0x7f791c0c7930 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7913fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f791c0c88e0 0x7f791c0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f791a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f791c152870 0x7f791c155bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52092/0 (socket says 172.21.15.17:52092) 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f791a59c640 1 -- 172.21.15.17:0/135013152 learned_addr learned my addr 172.21.15.17:0/135013152 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7913fff640 1 -- 172.21.15.17:0/135013152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 msgr2=0x7f791c06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7913fff640 1 --2- 172.21.15.17:0/135013152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c06bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7913fff640 1 -- 172.21.15.17:0/135013152 >> v1:172.21.15.17:6789/0 conn(0x7f791c152870 legacy=0x7f791c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7919d9b640 1 -- 172.21.15.17:0/135013152 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681071652 0 0) 0x7f791c0c5400 con 0x7f791c152870 2024-03-06T07:20:56.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.009+0000 7f7913fff640 1 -- 172.21.15.17:0/135013152 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0c56e0 con 0x7f791c0c88e0 2024-03-06T07:20:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7913fff640 1 --2- 172.21.15.17:0/135013152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c0c73f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79040097a0 tx=0x7f790402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ef15e46f5202c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7919d9b640 1 -- 172.21.15.17:0/135013152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79040315a0 con 0x7f791c0c88e0 2024-03-06T07:20:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7919d9b640 1 -- 172.21.15.17:0/135013152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7904037070 con 0x7f791c0c88e0 2024-03-06T07:20:56.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7919d9b640 1 -- 172.21.15.17:0/135013152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904031960 con 0x7f791c0c88e0 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7920925640 1 -- 172.21.15.17:0/135013152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 msgr2=0x7f791c0c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.010+0000 7f7920925640 1 --2- 172.21.15.17:0/135013152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c0c73f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79040097a0 tx=0x7f790402b840 comp rx=0 tx=0).stop 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 -- 172.21.15.17:0/135013152 shutdown_connections 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 --2- 172.21.15.17:0/135013152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c06bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 --2- 172.21.15.17:0/135013152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c0c73f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 -- 172.21.15.17:0/135013152 >> 172.21.15.17:0/135013152 conn(0x7f791c0ac5f0 msgr2=0x7f791c0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 -- 172.21.15.17:0/135013152 shutdown_connections 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.011+0000 7f7920925640 1 -- 172.21.15.17:0/135013152 wait complete. 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.012+0000 7f7920925640 1 Processor -- start 2024-03-06T07:20:56.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.012+0000 7f7920925640 1 -- start start 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c1ef830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 0x7f791c0af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c9470 con 0x7f791c0c88e0 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c5ca0 con 0x7f791c152870 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c0c95f0 con 0x7f791c0c7930 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33984/0 (socket says 172.21.15.17:33984) 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 -- 172.21.15.17:0/2018236953 learned_addr learned my addr 172.21.15.17:0/2018236953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f791a59c640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c1ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f791ad9d640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 0x7f791c0af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 -- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 msgr2=0x7f791c1ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c1ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 -- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 msgr2=0x7f791c0af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 0x7f791c0af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 -- 172.21.15.17:0/2018236953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c0ad8c0 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f791ad9d640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 0x7f791c0af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7913fff640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904032840 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79040329e0 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791c1eed70 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904032dd0 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.013+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791c159440 con 0x7f791c0c88e0 2024-03-06T07:20:56.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.014+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f791c075300 con 0x7f791c0c88e0 2024-03-06T07:20:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.015+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f790403b3f0 con 0x7f791c0c88e0 2024-03-06T07:20:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.015+0000 7f7912ffd640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.015+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6908+0+0 (secure 0 0 0) 0x7f790402f030 con 0x7f791c0c88e0 2024-03-06T07:20:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.015+0000 7f791a59c640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:56.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.016+0000 7f791a59c640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f790c000f60 tx=0x7f790c007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:56.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.020+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f791c075300 con 0x7f791c0c88e0 2024-03-06T07:20:56.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.202+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f791c04f330 con 0x7f791c0c88e0 2024-03-06T07:20:56.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.881+0000 7f7912ffd640 1 -- 172.21.15.17:0/2018236953 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7904040070 con 0x7f791c0c88e0 2024-03-06T07:20:56.883 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 msgr2=0x7f78fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f790c000f60 tx=0x7f790c007c00 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 msgr2=0x7f791c1f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040046c0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 shutdown_connections 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f791c152870 0x7f791c0af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f791c0c88e0 0x7f791c1f2cf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 --2- 172.21.15.17:0/2018236953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f791c0c7930 0x7f791c1ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:56.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.885+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 >> 172.21.15.17:0/2018236953 conn(0x7f791c0ac5f0 msgr2=0x7f791c0bf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:56.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.886+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 shutdown_connections 2024-03-06T07:20:56.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:56.886+0000 7f7920925640 1 -- 172.21.15.17:0/2018236953 wait complete. 2024-03-06T07:20:56.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.010+0000 7f5941b19640 1 Processor -- start 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.010+0000 7f5941b19640 1 -- start start 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f5941b19640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f593c0c2200 0x7f593c0c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f5941b19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c155b70 0x7f593c15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f5941b19640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593c1596c0 con 0x7f593c0a6cc0 2024-03-06T07:20:57.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f5941b19640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593c158ab0 con 0x7f593c0c2200 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f5941b19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c158d90 con 0x7f593c155b70 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f593c0c2200 0x7f593c0c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f593c0a6cc0 0x7f593c155460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52094/0 (socket says 172.21.15.17:52094) 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593affd640 1 -- 172.21.15.17:0/757706871 learned_addr learned my addr 172.21.15.17:0/757706871 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593bfff640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c155b70 0x7f593c15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593b7fe640 1 -- 172.21.15.17:0/757706871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c155b70 msgr2=0x7f593c15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593b7fe640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c155b70 0x7f593c15fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593b7fe640 1 -- 172.21.15.17:0/757706871 >> v1:172.21.15.17:6789/0 conn(0x7f593c0a6cc0 legacy=0x7f593c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593b7fe640 1 -- 172.21.15.17:0/757706871 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c159070 con 0x7f593c0c2200 2024-03-06T07:20:57.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.011+0000 7f593a7fc640 1 -- 172.21.15.17:0/757706871 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673987757 0 0) 0x7f593c1596c0 con 0x7f593c0a6cc0 2024-03-06T07:20:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.012+0000 7f593b7fe640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c0c2200 0x7f593c0c25e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d7df4e11addcd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.012+0000 7f593a7fc640 1 -- 172.21.15.17:0/757706871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c0315a0 con 0x7f593c0c2200 2024-03-06T07:20:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.012+0000 7f593a7fc640 1 -- 172.21.15.17:0/757706871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c037070 con 0x7f593c0c2200 2024-03-06T07:20:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.012+0000 7f593a7fc640 1 -- 172.21.15.17:0/757706871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c031960 con 0x7f593c0c2200 2024-03-06T07:20:57.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 -- 172.21.15.17:0/757706871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c0c2200 msgr2=0x7f593c0c25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c0c2200 0x7f593c0c25e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02b840 comp rx=0 tx=0).stop 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 -- 172.21.15.17:0/757706871 shutdown_connections 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c155b70 0x7f593c15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 --2- 172.21.15.17:0/757706871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c0c2200 0x7f593c0c25e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 -- 172.21.15.17:0/757706871 >> 172.21.15.17:0/757706871 conn(0x7f593c0c4300 msgr2=0x7f593c0c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 -- 172.21.15.17:0/757706871 shutdown_connections 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.013+0000 7f5941b19640 1 -- 172.21.15.17:0/757706871 wait complete. 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.014+0000 7f5941b19640 1 Processor -- start 2024-03-06T07:20:57.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.014+0000 7f5941b19640 1 -- start start 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f593c0a6cc0 0x7f593c151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c0c2200 0x7f593c1ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c073580 con 0x7f593c155b70 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c06be10 con 0x7f593c0a6cc0 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f5941b19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c158740 con 0x7f593c0c2200 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33992/0 (socket says 172.21.15.17:33992) 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 -- 172.21.15.17:0/1793776471 learned_addr learned my addr 172.21.15.17:0/1793776471 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593affd640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c0c2200 0x7f593c1ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593b7fe640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f593c0a6cc0 0x7f593c151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 -- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c0c2200 msgr2=0x7f593c1ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c0c2200 0x7f593c1ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 -- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f593c0a6cc0 msgr2=0x7f593c151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f593c0a6cc0 0x7f593c151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 -- 172.21.15.17:0/1793776471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c156550 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.015+0000 7f593bfff640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f593400cc30 tx=0x7f593400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.016+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593400ce40 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.016+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934003990 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.016+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c157f30 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.016+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934003d40 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.016+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c156830 con 0x7f593c155b70 2024-03-06T07:20:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.017+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593c04f330 con 0x7f593c155b70 2024-03-06T07:20:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.017+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5934003fa0 con 0x7f593c155b70 2024-03-06T07:20:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.017+0000 7f5923fff640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 0x7f592806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.018+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6470+0+0 (secure 0 0 0) 0x7f593408a940 con 0x7f593c155b70 2024-03-06T07:20:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.018+0000 7f593b7fe640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 0x7f592806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:57.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.018+0000 7f593b7fe640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 0x7f592806d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f592c036000 tx=0x7f592c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.022+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5934014030 con 0x7f593c155b70 2024-03-06T07:20:57.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.204+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f593c06bbf0 con 0x7f593c155b70 2024-03-06T07:20:57.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.886+0000 7f5923fff640 1 -- 172.21.15.17:0/1793776471 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5934058dd0 con 0x7f593c155b70 2024-03-06T07:20:57.888 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:20:57.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 msgr2=0x7f592806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 0x7f592806d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f592c036000 tx=0x7f592c03d040 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 msgr2=0x7f593c1ee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f593400cc30 tx=0x7f593400a5c0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 shutdown_connections 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f592806b4d0 0x7f592806d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f593c155b70 0x7f593c1ee730 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f593c0c2200 0x7f593c1ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 --2- 172.21.15.17:0/1793776471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f593c0a6cc0 0x7f593c151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:57.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.889+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 >> 172.21.15.17:0/1793776471 conn(0x7f593c0c4300 msgr2=0x7f593c0c25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:57.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.890+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 shutdown_connections 2024-03-06T07:20:57.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:57.890+0000 7f5941b19640 1 -- 172.21.15.17:0/1793776471 wait complete. 2024-03-06T07:20:57.907 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-06T07:20:58.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.024+0000 7f3313b93640 1 Processor -- start 2024-03-06T07:20:58.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 -- start start 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f330c152920 0x7f330c155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f330c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c159740 con 0x7f330c0c80d0 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330c1551d0 con 0x7f330c152920 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.025+0000 7f3313b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c8ab0 con 0x7f330c155eb0 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f3312130640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f330c0c80d0 0x7f330c0724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52110/0 (socket says 172.21.15.17:52110) 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f3312130640 1 -- 172.21.15.17:0/3860771152 learned_addr learned my addr 172.21.15.17:0/3860771152 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624532053 0 0) 0x7f330c159740 con 0x7f330c0c80d0 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f8003520 con 0x7f330c0c80d0 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331112e640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f330c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331192f640 1 --2- 172.21.15.17:0/3860771152 >> v2:172.21.15.163:3300/0 conn(0x7f330c152920 0x7f330c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2366278567 0 0) 0x7f32f8003520 con 0x7f330c0c80d0 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 msgr2=0x7f330c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f330c0c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 >> v2:172.21.15.163:3300/0 conn(0x7f330c152920 msgr2=0x7f330c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 --2- 172.21.15.17:0/3860771152 >> v2:172.21.15.163:3300/0 conn(0x7f330c152920 0x7f330c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0c8d90 con 0x7f330c0c80d0 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f33040019f0 con 0x7f330c0c80d0 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 >> v1:172.21.15.17:6789/0 conn(0x7f330c0c80d0 legacy=0x7f330c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f32f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f8004d40 0x7f32f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c1551d0 con 0x7f32f8004d40 2024-03-06T07:20:58.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c8ab0 con 0x7f32f80039b0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800b9f0 con 0x7f330c155eb0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f3312130640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f8004d40 0x7f32f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331192f640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.026+0000 7f331112e640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f32f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 -- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 msgr2=0x7f32f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c155eb0 0x7f32f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 -- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f80039b0 msgr2=0x7f32f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f80039b0 0x7f32f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 -- 172.21.15.17:0/3860771152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8003520 con 0x7f32f8004d40 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f3312130640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f8004d40 0x7f32f80051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33040013d0 tx=0x7f32f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d735da50c472f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330400a4e0 con 0x7f32f8004d40 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304034650 con 0x7f32f8004d40 2024-03-06T07:20:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.027+0000 7f331092d640 1 -- 172.21.15.17:0/3860771152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304034a40 con 0x7f32f8004d40 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 -- 172.21.15.17:0/3860771152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f8004d40 msgr2=0x7f32f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 --2- 172.21.15.17:0/3860771152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f8004d40 0x7f32f80051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33040013d0 tx=0x7f32f8005830 comp rx=0 tx=0).stop 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3312130640 1 -- 172.21.15.17:0/3860771152 reap_dead start 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 -- 172.21.15.17:0/3860771152 shutdown_connections 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 -- 172.21.15.17:0/3860771152 >> 172.21.15.17:0/3860771152 conn(0x7f330c0aa5c0 msgr2=0x7f330c0aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 -- 172.21.15.17:0/3860771152 shutdown_connections 2024-03-06T07:20:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.028+0000 7f3313b93640 1 -- 172.21.15.17:0/3860771152 wait complete. 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.029+0000 7f3313b93640 1 Processor -- start 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.029+0000 7f3313b93640 1 -- start start 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 0x7f330c1eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c14ee00 0x7f330c0ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c86c0 con 0x7f330c157290 2024-03-06T07:20:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c155450 con 0x7f32f8004d40 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f3313b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0c8840 con 0x7f330c14ee00 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33998/0 (socket says 172.21.15.17:33998) 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 -- 172.21.15.17:0/3542865043 learned_addr learned my addr 172.21.15.17:0/3542865043 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331192f640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 0x7f330c1eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 -- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c14ee00 msgr2=0x7f330c0ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c14ee00 0x7f330c0ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 -- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 msgr2=0x7f330c1eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 0x7f330c1eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 -- 172.21.15.17:0/3542865043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c157e10 con 0x7f330c157290 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331192f640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 0x7f330c1eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.030+0000 7f331112e640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32fc011920 tx=0x7f32fc011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.031+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00faf0 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.031+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc00fc90 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.031+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c158090 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.031+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc014440 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.031+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c1567b0 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.032+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c04f330 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.032+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32fc012040 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.032+0000 7f33027fc640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 0x7f32f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.033+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f32fc08a810 con 0x7f330c157290 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.033+0000 7f331192f640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 0x7f32f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:58.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.033+0000 7f331192f640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 0x7f32f006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f330c0adef0 tx=0x7f3308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:58.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.037+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f330c157e10 con 0x7f330c157290 2024-03-06T07:20:58.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.218+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f330c0c1fd0 con 0x7f330c157290 2024-03-06T07:20:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.887+0000 7f33027fc640 1 -- 172.21.15.17:0/3542865043 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f32fc019030 con 0x7f330c157290 2024-03-06T07:20:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' removed 2024-03-06T07:20:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 msgr2=0x7f32f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 0x7f32f006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f330c0adef0 tx=0x7f3308008040 comp rx=0 tx=0).stop 2024-03-06T07:20:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 msgr2=0x7f330c1578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32fc011920 tx=0x7f32fc011de0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 shutdown_connections 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32f006b4d0 0x7f32f006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f330c14ee00 0x7f330c0ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f330c157290 0x7f330c1578d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 --2- 172.21.15.17:0/3542865043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32f8004d40 0x7f330c1eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 >> 172.21.15.17:0/3542865043 conn(0x7f330c0aa5c0 msgr2=0x7f330c151f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 shutdown_connections 2024-03-06T07:20:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:58.891+0000 7f3313b93640 1 -- 172.21.15.17:0/3542865043 wait complete. 2024-03-06T07:20:58.908 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:20:58.908 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:20:58.908 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:20:58.908 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:58.908 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:20:58.908 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:20:58.909 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:20:58.909 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:58.909 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:20:58.909 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:20:58.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:20:58.909 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:58.910 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:20:58.910 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:20:58.910 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:20:58.910 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:20:58.918 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:20:58.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:20:58.919 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:20:58.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:20:58.920 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:20:58.921 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:20:58.921 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:20:58.922 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:20:58.961 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:20:58.961 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:58.962 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:20:58.962 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:58.963 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:20:58.963 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:58.964 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:20:58.964 DEBUG:teuthology.parallel:result is None 2024-03-06T07:20:58.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:20:59.044 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:20:59.044 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:20:59.046 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:20:59.046 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:20:59.047 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:20:59.046+0000 7f5ce8a7d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:20:59.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.075+0000 7f982fb8a640 1 Processor -- start 2024-03-06T07:20:59.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.075+0000 7f982fb8a640 1 -- start start 2024-03-06T07:20:59.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982fb8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 0x7f98280c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982fb8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982fb8a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9828155130 con 0x7f98280c8950 2024-03-06T07:20:59.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982fb8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9828155440 con 0x7f98280c69f0 2024-03-06T07:20:59.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982fb8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280ad480 con 0x7f98281558d0 2024-03-06T07:20:59.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982d926640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 0x7f98280c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982e127640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982d125640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f98280c8950 0x7f98280acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59470/0 (socket says 172.21.15.17:59470) 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982e127640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46372/0 (socket says 172.21.15.17:46372) 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982d125640 1 -- 172.21.15.17:0/755031825 learned_addr learned my addr 172.21.15.17:0/755031825 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.076+0000 7f982d926640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 0x7f98280c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:47326/0 (socket says 172.21.15.17:47326) 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982e127640 1 -- 172.21.15.17:0/755031825 >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 msgr2=0x7f98280c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982c924640 1 -- 172.21.15.17:0/755031825 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935651039 0 0) 0x7f9818001110 con 0x7f98280c8950 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982e127640 1 --2- 172.21.15.17:0/755031825 >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 0x7f98280c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982e127640 1 -- 172.21.15.17:0/755031825 >> v1:172.21.15.163:6789/0 conn(0x7f98280c8950 legacy=0x7f98280acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982e127640 1 -- 172.21.15.17:0/755031825 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280c5d50 con 0x7f98281558d0 2024-03-06T07:20:59.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.077+0000 7f982e127640 1 --2- 172.21.15.17:0/755031825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9820009d90 tx=0x7f982002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4421440f3772a5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982c924640 1 -- 172.21.15.17:0/755031825 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820039070 con 0x7f98281558d0 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982c924640 1 -- 172.21.15.17:0/755031825 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982002cbc0 con 0x7f98281558d0 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982c924640 1 -- 172.21.15.17:0/755031825 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820034440 con 0x7f98281558d0 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 -- 172.21.15.17:0/755031825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 msgr2=0x7f9828155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 --2- 172.21.15.17:0/755031825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9820009d90 tx=0x7f982002be10 comp rx=0 tx=0).stop 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 -- 172.21.15.17:0/755031825 shutdown_connections 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 --2- 172.21.15.17:0/755031825 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98281558d0 0x7f9828155d40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 --2- 172.21.15.17:0/755031825 >> v2:172.21.15.163:3300/0 conn(0x7f98280c69f0 0x7f98280c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 -- 172.21.15.17:0/755031825 >> 172.21.15.17:0/755031825 conn(0x7f98280cae70 msgr2=0x7f98280cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 -- 172.21.15.17:0/755031825 shutdown_connections 2024-03-06T07:20:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.078+0000 7f982fb8a640 1 -- 172.21.15.17:0/755031825 wait complete. 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.079+0000 7f982fb8a640 1 Processor -- start 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.079+0000 7f982fb8a640 1 -- start start 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 0x7f98281ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 0x7f98281ec410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828152a20 con 0x7f98281558d0 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280c6310 con 0x7f98280c8950 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982fb8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9828152ba0 con 0x7f98280c69f0 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982e127640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 0x7f98281ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982d926640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982d125640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 0x7f98281ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982d926640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46378/0 (socket says 172.21.15.17:46378) 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982d926640 1 -- 172.21.15.17:0/4000755625 learned_addr learned my addr 172.21.15.17:0/4000755625 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.080+0000 7f982e127640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 0x7f98281ec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34032/0 (socket says 172.21.15.17:34032) 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 -- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 msgr2=0x7f98280aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 -- 172.21.15.17:0/4000755625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 msgr2=0x7f98281ec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 0x7f98281ec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 -- 172.21.15.17:0/4000755625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98281e1660 con 0x7f98280c8950 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d926640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982d125640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 0x7f98281ebed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f981800c410 tx=0x7f981800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98180074e0 con 0x7f98280c8950 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180047e0 con 0x7f98280c8950 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982814f080 con 0x7f98280c8950 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9818004bd0 con 0x7f98280c8950 2024-03-06T07:20:59.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.081+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828156650 con 0x7f98280c8950 2024-03-06T07:20:59.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.082+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982804f330 con 0x7f98280c8950 2024-03-06T07:20:59.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.083+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9818011070 con 0x7f98280c8950 2024-03-06T07:20:59.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.083+0000 7f981e7fc640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 0x7f980c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:20:59.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.083+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9818089ab0 con 0x7f98280c8950 2024-03-06T07:20:59.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.083+0000 7f982d926640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 0x7f980c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:20:59.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.084+0000 7f982d926640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 0x7f980c06d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9824009f50 tx=0x7f9824009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:20:59.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.088+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9828155130 con 0x7f98280c8950 2024-03-06T07:20:59.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.280+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f98280ca650 con 0x7f98280c8950 2024-03-06T07:20:59.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.906+0000 7f981e7fc640 1 -- 172.21.15.17:0/4000755625 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9818058510 con 0x7f98280c8950 2024-03-06T07:20:59.908 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 msgr2=0x7f980c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 0x7f980c06d9c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9824009f50 tx=0x7f9824009190 comp rx=0 tx=0).stop 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 msgr2=0x7f98281ebed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 0x7f98281ebed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f981800c410 tx=0x7f981800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 shutdown_connections 2024-03-06T07:20:59.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f980c06b500 0x7f980c06d9c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98281558d0 0x7f98281ec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98280c8950 0x7f98281ebed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 --2- 172.21.15.17:0/4000755625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98280c69f0 0x7f98280aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.909+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 >> 172.21.15.17:0/4000755625 conn(0x7f98280cae70 msgr2=0x7f9828151790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.910+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 shutdown_connections 2024-03-06T07:20:59.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:20:59.910+0000 7f982fb8a640 1 -- 172.21.15.17:0/4000755625 wait complete. 2024-03-06T07:20:59.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:21:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.031+0000 7fd829a33640 1 Processor -- start 2024-03-06T07:21:00.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.031+0000 7fd829a33640 1 -- start start 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8240c5480 0x7fd824155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5860 0x7fd82406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8241586f0 0x7fd824158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82406df40 con 0x7fd8240c5480 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8240c5cc0 con 0x7fd8241586f0 2024-03-06T07:21:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd829a33640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8241591c0 con 0x7fd8240c5860 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd822ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8240c5480 0x7fd824155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd822ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8240c5480 0x7fd824155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34046/0 (socket says 172.21.15.17:34046) 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd822ffd640 1 -- 172.21.15.17:0/3716195766 learned_addr learned my addr 172.21.15.17:0/3716195766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd8237fe640 1 --2- 172.21.15.17:0/3716195766 >> v2:172.21.15.163:3300/0 conn(0x7fd8241586f0 0x7fd824158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.032+0000 7fd8227fc640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5860 0x7fd82406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 -- 172.21.15.17:0/3716195766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5860 msgr2=0x7fd82406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5860 0x7fd82406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 -- 172.21.15.17:0/3716195766 >> v2:172.21.15.163:3300/0 conn(0x7fd8241586f0 msgr2=0x7fd824158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3716195766 >> v2:172.21.15.163:3300/0 conn(0x7fd8241586f0 0x7fd824158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 -- 172.21.15.17:0/3716195766 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8241594a0 con 0x7fd8240c5480 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.033+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8240c5480 0x7fd824155ba0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd824071060 tx=0x7fd81c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818c76495323dc87 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd821ffb640 1 -- 172.21.15.17:0/3716195766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c02e070 con 0x7fd8240c5480 2024-03-06T07:21:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd821ffb640 1 -- 172.21.15.17:0/3716195766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c037070 con 0x7fd8240c5480 2024-03-06T07:21:00.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd821ffb640 1 -- 172.21.15.17:0/3716195766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c031a00 con 0x7fd8240c5480 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 -- 172.21.15.17:0/3716195766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8240c5480 msgr2=0x7fd824155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8240c5480 0x7fd824155ba0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd824071060 tx=0x7fd81c02b880 comp rx=0 tx=0).stop 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 -- 172.21.15.17:0/3716195766 shutdown_connections 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 --2- 172.21.15.17:0/3716195766 >> v2:172.21.15.163:3300/0 conn(0x7fd8241586f0 0x7fd824158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5860 0x7fd82406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 --2- 172.21.15.17:0/3716195766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8240c5480 0x7fd824155ba0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 -- 172.21.15.17:0/3716195766 >> 172.21.15.17:0/3716195766 conn(0x7fd82406b1a0 msgr2=0x7fd82406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 -- 172.21.15.17:0/3716195766 shutdown_connections 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.034+0000 7fd829a33640 1 -- 172.21.15.17:0/3716195766 wait complete. 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.035+0000 7fd829a33640 1 Processor -- start 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 -- start start 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5480 0x7fd8240c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240c5860 0x7fd824151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824152a20 con 0x7fd8241586f0 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824155680 con 0x7fd8240c5860 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd829a33640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd824155050 con 0x7fd8240c5480 2024-03-06T07:21:00.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd8237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd8237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34056/0 (socket says 172.21.15.17:34056) 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd8237fe640 1 -- 172.21.15.17:0/3591628082 learned_addr learned my addr 172.21.15.17:0/3591628082 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5480 0x7fd8240c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.036+0000 7fd8227fc640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240c5860 0x7fd824151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 -- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5480 msgr2=0x7fd8240c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5480 0x7fd8240c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 -- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240c5860 msgr2=0x7fd824151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240c5860 0x7fd824151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 -- 172.21.15.17:0/3591628082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240ab600 con 0x7fd8241586f0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd8237fe640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd814007cf0 tx=0x7fd81400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814014420 con 0x7fd8241586f0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814019070 con 0x7fd8241586f0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8240ab870 con 0x7fd8241586f0 2024-03-06T07:21:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814014820 con 0x7fd8241586f0 2024-03-06T07:21:00.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.037+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd824151f50 con 0x7fd8241586f0 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.038+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8241eecd0 con 0x7fd8241586f0 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.039+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8140149c0 con 0x7fd8241586f0 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.039+0000 7fd80bfff640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 0x7fd81006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.039+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd81408a5b0 con 0x7fd8241586f0 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.039+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 0x7fd81006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.040+0000 7fd822ffd640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 0x7fd81006d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd81c036000 tx=0x7fd81c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.044+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd814058fa0 con 0x7fd8241586f0 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.291+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd82404f330 con 0x7fd8241586f0 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.291+0000 7fd80bfff640 1 -- 172.21.15.17:0/3591628082 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6065 (secure 0 0 0) 0x7fd81405d040 con 0x7fd8241586f0 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:21:00.293 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:00.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.297 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.301 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:21:00.303 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mds", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehlxWGaBBAAYjDue2qPVl4I7wrjkYWUFA==", 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mds" 2024-03-06T07:21:00.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mgr", 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehlQGuaBBAAHcsvgFvV6iakO3xML5zoYg==", 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-osd", 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehl93GaBBAAKACjppDnTmYUbDVmsb+jCQ==", 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-osd" 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd", 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehlDHmaBBAAl3UBFaDme73Mp7GNGHjD0g==", 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehloX+aBBAAnf8/LXh1xBViajyiNTjR8g==", 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rgw", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATGehlUYaaBBAAJKEulomTqbSS1LfVkj1ZNw==", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:00.308 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:21:00.309 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:00.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 msgr2=0x7fd81006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 0x7fd81006d9c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd81c036000 tx=0x7fd81c03d040 comp rx=0 tx=0).stop 2024-03-06T07:21:00.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 msgr2=0x7fd8241eb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd814007cf0 tx=0x7fd81400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 shutdown_connections 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b500 0x7fd81006d9c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8241586f0 0x7fd8241eb0f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8240c5860 0x7fd824151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 --2- 172.21.15.17:0/3591628082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8240c5480 0x7fd8240c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 >> 172.21.15.17:0/3591628082 conn(0x7fd82406b1a0 msgr2=0x7fd8240c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 shutdown_connections 2024-03-06T07:21:00.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.293+0000 7fd829a33640 1 -- 172.21.15.17:0/3591628082 wait complete. 2024-03-06T07:21:00.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-06T07:21:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.413+0000 7f19b56c2640 1 Processor -- start 2024-03-06T07:21:00.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.413+0000 7f19b56c2640 1 -- start start 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19b56c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19b00c8950 0x7f19b00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19b56c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19b01558d0 0x7f19b0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19b56c2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b0155130 con 0x7f19b00c69f0 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19b56c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b0155440 con 0x7f19b00c8950 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19b56c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b00ad480 con 0x7f19b01558d0 2024-03-06T07:21:00.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19ae7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19b00c8950 0x7f19b00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19ae7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19b00c8950 0x7f19b00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34072/0 (socket says 172.21.15.17:34072) 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19ae7fc640 1 -- 172.21.15.17:0/480890436 learned_addr learned my addr 172.21.15.17:0/480890436 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.414+0000 7f19af7fe640 1 --2- 172.21.15.17:0/480890436 >> v2:172.21.15.163:3300/0 conn(0x7f19b01558d0 0x7f19b0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19ae7fc640 1 -- 172.21.15.17:0/480890436 >> v2:172.21.15.163:3300/0 conn(0x7f19b01558d0 msgr2=0x7f19b0155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19ae7fc640 1 --2- 172.21.15.17:0/480890436 >> v2:172.21.15.163:3300/0 conn(0x7f19b01558d0 0x7f19b0155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19ae7fc640 1 -- 172.21.15.17:0/480890436 >> v1:172.21.15.163:6789/0 conn(0x7f19b00c69f0 legacy=0x7f19b00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19ae7fc640 1 -- 172.21.15.17:0/480890436 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00c5d50 con 0x7f19b00c8950 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19ae7fc640 1 --2- 172.21.15.17:0/480890436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c8950 0x7f19b00c6dd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19b0074a60 tx=0x7f19a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d632122489c55840 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19adffb640 1 -- 172.21.15.17:0/480890436 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a402e070 con 0x7f19b00c8950 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.415+0000 7f19adffb640 1 -- 172.21.15.17:0/480890436 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4037070 con 0x7f19b00c8950 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19adffb640 1 -- 172.21.15.17:0/480890436 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4031a00 con 0x7f19b00c8950 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 -- 172.21.15.17:0/480890436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c8950 msgr2=0x7f19b00c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 --2- 172.21.15.17:0/480890436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c8950 0x7f19b00c6dd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19b0074a60 tx=0x7f19a402b880 comp rx=0 tx=0).stop 2024-03-06T07:21:00.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 -- 172.21.15.17:0/480890436 shutdown_connections 2024-03-06T07:21:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 --2- 172.21.15.17:0/480890436 >> v2:172.21.15.163:3300/0 conn(0x7f19b01558d0 0x7f19b0155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 --2- 172.21.15.17:0/480890436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c8950 0x7f19b00c6dd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 -- 172.21.15.17:0/480890436 >> 172.21.15.17:0/480890436 conn(0x7f19b00cae70 msgr2=0x7f19b00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 -- 172.21.15.17:0/480890436 shutdown_connections 2024-03-06T07:21:00.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.416+0000 7f19b56c2640 1 -- 172.21.15.17:0/480890436 wait complete. 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.417+0000 7f19b56c2640 1 Processor -- start 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.417+0000 7f19b56c2640 1 -- start start 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19b00c8950 0x7f19b0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 0x7f19b01ee490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0152a20 con 0x7f19b00c69f0 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00c6310 con 0x7f19b00c8950 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19b56c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0152ba0 con 0x7f19b01558d0 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34078/0 (socket says 172.21.15.17:34078) 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 -- 172.21.15.17:0/2841291304 learned_addr learned my addr 172.21.15.17:0/2841291304 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19ae7fc640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19b00c8950 0x7f19b0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19af7fe640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 0x7f19b01ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 -- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 msgr2=0x7f19b01ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 0x7f19b01ee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 -- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19b00c8950 msgr2=0x7f19b0157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19b00c8950 0x7f19b0157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19aeffd640 1 -- 172.21.15.17:0/2841291304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b0156770 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.418+0000 7f19af7fe640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 0x7f19b01ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19aeffd640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19a000b3f0 tx=0x7f19a000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0015020 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0011830 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011c20 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b01eef10 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.419+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b0158380 con 0x7f19b00c69f0 2024-03-06T07:21:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.420+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b004f330 con 0x7f19b00c69f0 2024-03-06T07:21:00.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.420+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19a0018020 con 0x7f19b00c69f0 2024-03-06T07:21:00.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.420+0000 7f19977fe640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 0x7f199c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.421+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f19a0089cb0 con 0x7f19b00c69f0 2024-03-06T07:21:00.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.421+0000 7f19ae7fc640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 0x7f199c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.421+0000 7f19ae7fc640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 0x7f199c06d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a4036000 tx=0x7f19a403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.425+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f19a001f030 con 0x7f19b00c69f0 2024-03-06T07:21:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.675+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f19b00ca650 con 0x7f19b00c69f0 2024-03-06T07:21:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.678+0000 7f19977fe640 1 -- 172.21.15.17:0/2841291304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19a0058cf0 con 0x7f19b00c69f0 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.679+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 msgr2=0x7f199c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.679+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 0x7f199c06d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a4036000 tx=0x7f19a403d040 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.679+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 msgr2=0x7f19b01df960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.679+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f19a000b3f0 tx=0x7f19a000b8b0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 shutdown_connections 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f199c06b4d0 0x7f199c06d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19b01558d0 0x7f19b01ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19b00c8950 0x7f19b0157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 --2- 172.21.15.17:0/2841291304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00c69f0 0x7f19b01df960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 >> 172.21.15.17:0/2841291304 conn(0x7f19b00cae70 msgr2=0x7f19b01517b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 shutdown_connections 2024-03-06T07:21:00.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.680+0000 7f19b56c2640 1 -- 172.21.15.17:0/2841291304 wait complete. 2024-03-06T07:21:00.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-06T07:21:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.793+0000 7fe8e273a640 1 Processor -- start 2024-03-06T07:21:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.793+0000 7fe8e273a640 1 -- start start 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 0x7fe8dc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 0x7fe8dc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc155440 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc155130 con 0x7fe8dc0ad2b0 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8e273a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c5d50 con 0x7fe8dc0c8950 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35424/0 (socket says 172.21.15.17:35424) 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.794+0000 7fe8dbfff640 1 -- 172.21.15.17:0/81341395 learned_addr learned my addr 172.21.15.17:0/81341395 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8e0cd7640 1 --2- 172.21.15.17:0/81341395 >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 0x7fe8dc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 -- 172.21.15.17:0/81341395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 msgr2=0x7fe8dc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8db7fe640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 0x7fe8dc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 0x7fe8dc0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 -- 172.21.15.17:0/81341395 >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 msgr2=0x7fe8dc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/81341395 >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 0x7fe8dc1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 -- 172.21.15.17:0/81341395 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0c6030 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8e0cd7640 1 --2- 172.21.15.17:0/81341395 >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 0x7fe8dc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8db7fe640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 0x7fe8dc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.795+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc159820 tx=0x7fe8cc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff30d535c1666b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8daffd640 1 -- 172.21.15.17:0/81341395 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc02e070 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8daffd640 1 -- 172.21.15.17:0/81341395 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc037070 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8daffd640 1 -- 172.21.15.17:0/81341395 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc031a00 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 -- 172.21.15.17:0/81341395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 msgr2=0x7fe8dc0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc159820 tx=0x7fe8cc02b880 comp rx=0 tx=0).stop 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 -- 172.21.15.17:0/81341395 shutdown_connections 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 --2- 172.21.15.17:0/81341395 >> v2:172.21.15.163:3300/0 conn(0x7fe8dc0ad2b0 0x7fe8dc1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0c8950 0x7fe8dc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 --2- 172.21.15.17:0/81341395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc0c6dd0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 -- 172.21.15.17:0/81341395 >> 172.21.15.17:0/81341395 conn(0x7fe8dc0cae70 msgr2=0x7fe8dc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 -- 172.21.15.17:0/81341395 shutdown_connections 2024-03-06T07:21:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.796+0000 7fe8e273a640 1 -- 172.21.15.17:0/81341395 wait complete. 2024-03-06T07:21:00.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.797+0000 7fe8e273a640 1 Processor -- start 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.797+0000 7fe8e273a640 1 -- start start 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0ad2b0 0x7fe8dc1ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc0c8950 0x7fe8dc154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc152ba0 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc152a20 con 0x7fe8dc0c8950 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e273a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c58e0 con 0x7fe8dc0ad2b0 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35438/0 (socket says 172.21.15.17:35438) 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 -- 172.21.15.17:0/3099112105 learned_addr learned my addr 172.21.15.17:0/3099112105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8e0cd7640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc0c8950 0x7fe8dc154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 -- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0ad2b0 msgr2=0x7fe8dc1ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0ad2b0 0x7fe8dc1ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 -- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc0c8950 msgr2=0x7fe8dc154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc0c8950 0x7fe8dc154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.798+0000 7fe8db7fe640 1 -- 172.21.15.17:0/3099112105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc148ff0 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8db7fe640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c820 tx=0x7fe8d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0015070 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0007a30 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc1463d0 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0007de0 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.799+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc154870 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.800+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc04f330 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.800+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.801+0000 7fe8d8ff9640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 0x7fe8c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.801+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe8d0089ee0 con 0x7fe8dc0c69f0 2024-03-06T07:21:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.801+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 0x7fe8c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.801+0000 7fe8dbfff640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 0x7fe8c806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc036000 tx=0x7fe8cc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:00.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:00.805+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8d0058fb0 con 0x7fe8dc0c69f0 2024-03-06T07:21:01.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.059+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe8dc0ca650 con 0x7fe8dc0c69f0 2024-03-06T07:21:01.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.060+0000 7fe8d8ff9640 1 -- 172.21.15.17:0/3099112105 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe8d00588b0 con 0x7fe8dc0c69f0 2024-03-06T07:21:01.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 msgr2=0x7fe8c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 0x7fe8c806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc036000 tx=0x7fe8cc030000 comp rx=0 tx=0).stop 2024-03-06T07:21:01.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 msgr2=0x7fe8dc1482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c820 tx=0x7fe8d000cce0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 shutdown_connections 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8c806b4d0 0x7fe8c806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8dc0c8950 0x7fe8dc154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8dc0c69f0 0x7fe8dc1482a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 --2- 172.21.15.17:0/3099112105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8dc0ad2b0 0x7fe8dc1ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 >> 172.21.15.17:0/3099112105 conn(0x7fe8dc0cae70 msgr2=0x7fe8dc0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 shutdown_connections 2024-03-06T07:21:01.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.062+0000 7fe8e273a640 1 -- 172.21.15.17:0/3099112105 wait complete. 2024-03-06T07:21:01.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-06T07:21:01.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.174+0000 7f564f593640 1 Processor -- start 2024-03-06T07:21:01.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 -- start start 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56480c69f0 0x7f56480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56480acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5648155130 con 0x7f56480ad2b0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5648155440 con 0x7f56480c69f0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564f593640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c5d50 con 0x7f56480c8950 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564db30640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f56480ad2b0 0x7f56481558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46396/0 (socket says 172.21.15.17:46396) 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.175+0000 7f564db30640 1 -- 172.21.15.17:0/3992739203 learned_addr learned my addr 172.21.15.17:0/3992739203 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 --2- 172.21.15.17:0/3992739203 >> v2:172.21.15.163:3300/0 conn(0x7f56480c69f0 0x7f56480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564cb2e640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56480acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591476824 0 0) 0x7f5640001490 con 0x7f56480ad2b0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5634003520 con 0x7f56480ad2b0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505077883 0 0) 0x7f5634003520 con 0x7f56480ad2b0 2024-03-06T07:21:01.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 msgr2=0x7f56480acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56480acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 >> v2:172.21.15.163:3300/0 conn(0x7f56480c69f0 msgr2=0x7f56480c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 --2- 172.21.15.17:0/3992739203 >> v2:172.21.15.163:3300/0 conn(0x7f56480c69f0 0x7f56480c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480c6030 con 0x7f56480ad2b0 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f5640001bc0 con 0x7f56480ad2b0 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 >> v1:172.21.15.17:6789/0 conn(0x7f56480ad2b0 legacy=0x7f56481558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f5634003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648155440 con 0x7f56340039b0 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c5d50 con 0x7f5634004800 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563400b9c0 con 0x7f56480c8950 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f5634003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564db30640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564cb2e640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 -- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 msgr2=0x7f56340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 -- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5634004800 msgr2=0x7f5634004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.176+0000 7f564d32f640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f564d32f640 1 -- 172.21.15.17:0/3992739203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5634003520 con 0x7f56340039b0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f564db30640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c8950 0x7f56340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f564cb2e640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f564d32f640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f5634003dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5644009af0 tx=0x7f5634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a17d9777f2ec886 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644039070 con 0x7f56340039b0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564402c710 con 0x7f56340039b0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.177+0000 7f563ffff640 1 -- 172.21.15.17:0/3992739203 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644009690 con 0x7f56340039b0 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 -- 172.21.15.17:0/3992739203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 msgr2=0x7f5634003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 --2- 172.21.15.17:0/3992739203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f5634003dd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5644009af0 tx=0x7f5634005830 comp rx=0 tx=0).stop 2024-03-06T07:21:01.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564db30640 1 -- 172.21.15.17:0/3992739203 reap_dead start 2024-03-06T07:21:01.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 -- 172.21.15.17:0/3992739203 shutdown_connections 2024-03-06T07:21:01.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 -- 172.21.15.17:0/3992739203 >> 172.21.15.17:0/3992739203 conn(0x7f56480cae70 msgr2=0x7f56480cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 -- 172.21.15.17:0/3992739203 shutdown_connections 2024-03-06T07:21:01.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.178+0000 7f564f593640 1 -- 172.21.15.17:0/3992739203 wait complete. 2024-03-06T07:21:01.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.179+0000 7f564f593640 1 Processor -- start 2024-03-06T07:21:01.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.179+0000 7f564f593640 1 -- start start 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 0x7f5648156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5648156dd0 0x7f56481edef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648152a20 con 0x7f56340039b0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648155fe0 con 0x7f5648158ac0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564f593640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5648152ba0 con 0x7f5648156dd0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35450/0 (socket says 172.21.15.17:35450) 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 -- 172.21.15.17:0/1265888677 learned_addr learned my addr 172.21.15.17:0/1265888677 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564db30640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5648156dd0 0x7f56481edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564cb2e640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 0x7f5648156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 -- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5648156dd0 msgr2=0x7f56481edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5648156dd0 0x7f56481edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 -- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 msgr2=0x7f5648156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 0x7f5648156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 -- 172.21.15.17:0/1265888677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648158fc0 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564cb2e640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 0x7f5648156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f564d32f640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5644009af0 tx=0x7f5644004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.180+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644042020 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.181+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564403a040 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.181+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5648157c50 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.181+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644039070 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.181+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564806c7c0 con 0x7f56340039b0 2024-03-06T07:21:01.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.182+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5648067910 con 0x7f56340039b0 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.182+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5644045020 con 0x7f56340039b0 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.183+0000 7f563dffb640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 0x7f562c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.183+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f56440ab640 con 0x7f56340039b0 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.183+0000 7f564cb2e640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 0x7f562c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.183+0000 7f564cb2e640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 0x7f562c06d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56480c7220 tx=0x7f5638008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.187+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5648067910 con 0x7f56340039b0 2024-03-06T07:21:01.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.436+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f564804f330 con 0x7f56340039b0 2024-03-06T07:21:01.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.438+0000 7f563dffb640 1 -- 172.21.15.17:0/1265888677 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f564404c030 con 0x7f56340039b0 2024-03-06T07:21:01.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 msgr2=0x7f562c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 0x7f562c06d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f56480c7220 tx=0x7f5638008040 comp rx=0 tx=0).stop 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 msgr2=0x7f564806c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5644009af0 tx=0x7f5644004730 comp rx=0 tx=0).stop 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 shutdown_connections 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f562c06b500 0x7f562c06d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5648156dd0 0x7f56481edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158ac0 0x7f5648156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 --2- 172.21.15.17:0/1265888677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56340039b0 0x7f564806c280 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 >> 172.21.15.17:0/1265888677 conn(0x7f56480cae70 msgr2=0x7f56480cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 shutdown_connections 2024-03-06T07:21:01.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.440+0000 7f564f593640 1 -- 172.21.15.17:0/1265888677 wait complete. 2024-03-06T07:21:01.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-06T07:21:01.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.554+0000 7f452fa78640 1 Processor -- start 2024-03-06T07:21:01.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.554+0000 7f452fa78640 1 -- start start 2024-03-06T07:21:01.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452fa78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45280c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452fa78640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45280c2b20 0x7f45280c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452fa78640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280c5140 con 0x7f45280c8e70 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452fa78640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45280c3790 con 0x7f45280c2b20 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452fa78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4528155bb0 con 0x7f45281526c0 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452e015640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f45280c8e70 0x7f45281587e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46412/0 (socket says 172.21.15.17:46412) 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452e015640 1 -- 172.21.15.17:0/1760997408 learned_addr learned my addr 172.21.15.17:0/1760997408 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452d814640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45280c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.555+0000 7f452d013640 1 --2- 172.21.15.17:0/1760997408 >> v2:172.21.15.163:3300/0 conn(0x7f45280c2b20 0x7f45280c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546519270 0 0) 0x7f45280c5140 con 0x7f45280c8e70 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003520 con 0x7f45280c8e70 2024-03-06T07:21:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 206510377 0 0) 0x7f4514003520 con 0x7f45280c8e70 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 msgr2=0x7f45280c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45280c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 >> v2:172.21.15.163:3300/0 conn(0x7f45280c2b20 msgr2=0x7f45280c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 --2- 172.21.15.17:0/1760997408 >> v2:172.21.15.163:3300/0 conn(0x7f45280c2b20 0x7f45280c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528155e90 con 0x7f45280c8e70 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f45200019f0 con 0x7f45280c8e70 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 >> v1:172.21.15.17:6789/0 conn(0x7f45280c8e70 legacy=0x7f45281587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4514003f30 0x7f4514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f45140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c3790 con 0x7f4514004d40 2024-03-06T07:21:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528155bb0 con 0x7f4514003f30 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451400ba10 con 0x7f45281526c0 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f45140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452d013640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4514003f30 0x7f4514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452d814640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 -- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 msgr2=0x7f45140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45281526c0 0x7f45140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 -- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4514003f30 msgr2=0x7f4514004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4514003f30 0x7f4514004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.556+0000 7f452e015640 1 -- 172.21.15.17:0/1760997408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4514003520 con 0x7f4514004d40 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.557+0000 7f452e015640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f45140051c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f452000a8d0 tx=0x7f4514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41976ddcf2fee11d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.557+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452000a4c0 con 0x7f4514004d40 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.557+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4520034650 con 0x7f4514004d40 2024-03-06T07:21:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.557+0000 7f452c812640 1 -- 172.21.15.17:0/1760997408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520034a80 con 0x7f4514004d40 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 -- 172.21.15.17:0/1760997408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 msgr2=0x7f45140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 --2- 172.21.15.17:0/1760997408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f45140051c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f452000a8d0 tx=0x7f4514005830 comp rx=0 tx=0).stop 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452e015640 1 -- 172.21.15.17:0/1760997408 reap_dead start 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 -- 172.21.15.17:0/1760997408 shutdown_connections 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 -- 172.21.15.17:0/1760997408 >> 172.21.15.17:0/1760997408 conn(0x7f45280aa810 msgr2=0x7f45280aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 -- 172.21.15.17:0/1760997408 shutdown_connections 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.558+0000 7f452fa78640 1 -- 172.21.15.17:0/1760997408 wait complete. 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.559+0000 7f452fa78640 1 Processor -- start 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.559+0000 7f452fa78640 1 -- start start 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45280a9e10 0x7f45281564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f452806d300 0x7f45281e7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c69b0 con 0x7f4514004d40 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528152aa0 con 0x7f45280a9e10 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45280c6b30 con 0x7f452806d300 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35474/0 (socket says 172.21.15.17:35474) 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 -- 172.21.15.17:0/168430951 learned_addr learned my addr 172.21.15.17:0/168430951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452e015640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f452806d300 0x7f45281e7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d013640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45280a9e10 0x7f45281564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 -- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f452806d300 msgr2=0x7f45281e7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f452806d300 0x7f45281e7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 -- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45280a9e10 msgr2=0x7f45281564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45280a9e10 0x7f45281564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 -- 172.21.15.17:0/168430951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45281eb530 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452d814640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45240119d0 tx=0x7f4524011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452400faf0 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452400fc90 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45280a7ee0 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.560+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524014440 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.561+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45281eb810 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.562+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45280a8400 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.562+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4524012040 con 0x7f4514004d40 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.562+0000 7f451e7fc640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 0x7f450c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.562+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f452408a6b0 con 0x7f4514004d40 2024-03-06T07:21:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.562+0000 7f452d013640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 0x7f450c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.563+0000 7f452d013640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 0x7f450c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f452814bdd0 tx=0x7f4518008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.567+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f45280a8400 con 0x7f4514004d40 2024-03-06T07:21:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.821+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f45280c8640 con 0x7f4514004d40 2024-03-06T07:21:01.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.823+0000 7f451e7fc640 1 -- 172.21.15.17:0/168430951 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f45280c8640 con 0x7f4514004d40 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 msgr2=0x7f450c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 0x7f450c06d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f452814bdd0 tx=0x7f4518008040 comp rx=0 tx=0).stop 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 msgr2=0x7f452814b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f45240119d0 tx=0x7f4524011e90 comp rx=0 tx=0).stop 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 shutdown_connections 2024-03-06T07:21:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f450c06b4d0 0x7f450c06d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f452806d300 0x7f45281e7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45280a9e10 0x7f45281564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 --2- 172.21.15.17:0/168430951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4514004d40 0x7f452814b870 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 >> 172.21.15.17:0/168430951 conn(0x7f45280aa810 msgr2=0x7f4528151080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 shutdown_connections 2024-03-06T07:21:01.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.825+0000 7f452fa78640 1 -- 172.21.15.17:0/168430951 wait complete. 2024-03-06T07:21:01.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-06T07:21:01.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.939+0000 7f1bd746b640 1 Processor -- start 2024-03-06T07:21:01.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.939+0000 7f1bd746b640 1 -- start start 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd746b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bd00bfe90 0x7f1bd0155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd746b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bd00c6d70 0x7f1bd0158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd746b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd00c7200 con 0x7f1bd01593c0 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd746b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd0158710 con 0x7f1bd00c6d70 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd746b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bd00c59d0 con 0x7f1bd00bfe90 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bd00c6d70 0x7f1bd0158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bd00c6d70 0x7f1bd0158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35476/0 (socket says 172.21.15.17:35476) 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 -- 172.21.15.17:0/4265428834 learned_addr learned my addr 172.21.15.17:0/4265428834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 -- 172.21.15.17:0/4265428834 >> v2:172.21.15.163:3300/0 conn(0x7f1bd00bfe90 msgr2=0x7f1bd0155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 --2- 172.21.15.17:0/4265428834 >> v2:172.21.15.163:3300/0 conn(0x7f1bd00bfe90 0x7f1bd0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 -- 172.21.15.17:0/4265428834 >> v1:172.21.15.17:6789/0 conn(0x7f1bd01593c0 legacy=0x7f1bd01522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.940+0000 7f1bd4a06640 1 -- 172.21.15.17:0/4265428834 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd00c5cb0 con 0x7f1bd00c6d70 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bd4a06640 1 --2- 172.21.15.17:0/4265428834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd0158e80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00c8530 tx=0x7f1bc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e881f607b67d45d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bc7fff640 1 -- 172.21.15.17:0/4265428834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0035020 con 0x7f1bd00c6d70 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bc7fff640 1 -- 172.21.15.17:0/4265428834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc0031870 con 0x7f1bd00c6d70 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bc7fff640 1 -- 172.21.15.17:0/4265428834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0031c40 con 0x7f1bd00c6d70 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bd746b640 1 -- 172.21.15.17:0/4265428834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 msgr2=0x7f1bd0158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.941+0000 7f1bd746b640 1 --2- 172.21.15.17:0/4265428834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd0158e80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00c8530 tx=0x7f1bc002b910 comp rx=0 tx=0).stop 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 -- 172.21.15.17:0/4265428834 shutdown_connections 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 --2- 172.21.15.17:0/4265428834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd0158e80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 --2- 172.21.15.17:0/4265428834 >> v2:172.21.15.163:3300/0 conn(0x7f1bd00bfe90 0x7f1bd0155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 -- 172.21.15.17:0/4265428834 >> 172.21.15.17:0/4265428834 conn(0x7f1bd00acaa0 msgr2=0x7f1bd00aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:01.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 -- 172.21.15.17:0/4265428834 shutdown_connections 2024-03-06T07:21:01.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.942+0000 7f1bd746b640 1 -- 172.21.15.17:0/4265428834 wait complete. 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 Processor -- start 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 -- start start 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 0x7f1bd014d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 0x7f1bd0152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00ca700 con 0x7f1bd00c6d70 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd0158960 con 0x7f1bd00bfe90 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd746b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd00ca880 con 0x7f1bd01eb170 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.943+0000 7f1bd4a06640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35482/0 (socket says 172.21.15.17:35482) 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 -- 172.21.15.17:0/2368451898 learned_addr learned my addr 172.21.15.17:0/2368451898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd5207640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 0x7f1bd014d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd5a08640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 0x7f1bd0152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 -- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 msgr2=0x7f1bd0152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 0x7f1bd0152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 -- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 msgr2=0x7f1bd014d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 0x7f1bd014d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:01.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 -- 172.21.15.17:0/2368451898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd014fe80 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd5207640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 0x7f1bd014d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd5a08640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 0x7f1bd0152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd4a06640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1bc002b950 tx=0x7f1bc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0040070 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc003a420 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd01ef340 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0035040 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.944+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd00ab710 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.945+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd004f330 con 0x7f1bd00c6d70 2024-03-06T07:21:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.946+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bc0038020 con 0x7f1bd00c6d70 2024-03-06T07:21:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.946+0000 7f1bc5ffb640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 0x7f1bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.946+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1bc00ac020 con 0x7f1bd00c6d70 2024-03-06T07:21:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.946+0000 7f1bd5207640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 0x7f1bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.947+0000 7f1bd5207640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 0x7f1bb406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006fd0 tx=0x7f1bcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:01.951+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1bc007b0f0 con 0x7f1bd00c6d70 2024-03-06T07:21:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.200+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1bd006bb90 con 0x7f1bd00c6d70 2024-03-06T07:21:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.202+0000 7f1bc5ffb640 1 -- 172.21.15.17:0/2368451898 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1bc007a9f0 con 0x7f1bd00c6d70 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 msgr2=0x7f1bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 0x7f1bb406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006fd0 tx=0x7f1bcc008040 comp rx=0 tx=0).stop 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 msgr2=0x7f1bd01e1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1bc002b950 tx=0x7f1bc00046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 shutdown_connections 2024-03-06T07:21:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bb406b4d0 0x7f1bb406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bd01eb170 0x7f1bd0152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bd00c6d70 0x7f1bd01e1ec0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 --2- 172.21.15.17:0/2368451898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bd00bfe90 0x7f1bd014d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 >> 172.21.15.17:0/2368451898 conn(0x7f1bd00acaa0 msgr2=0x7f1bd00c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 shutdown_connections 2024-03-06T07:21:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.204+0000 7f1bd746b640 1 -- 172.21.15.17:0/2368451898 wait complete. 2024-03-06T07:21:02.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-06T07:21:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.317+0000 7f89404c3640 1 Processor -- start 2024-03-06T07:21:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.317+0000 7f89404c3640 1 -- start start 2024-03-06T07:21:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f89404c3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8938152720 0x7f89381593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f89404c3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 0x7f89380c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f89404c3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89380733f0 con 0x7f89380c7d20 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f89404c3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8938155ca0 con 0x7f8938152720 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f89404c3640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89381561d0 con 0x7f893806f550 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8938152720 0x7f89381593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8938152720 0x7f89381593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35494/0 (socket says 172.21.15.17:35494) 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 -- 172.21.15.17:0/3995490455 learned_addr learned my addr 172.21.15.17:0/3995490455 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893da5e640 1 --2- 172.21.15.17:0/3995490455 >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 0x7f89380c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 -- 172.21.15.17:0/3995490455 >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 msgr2=0x7f89380c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 --2- 172.21.15.17:0/3995490455 >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 0x7f89380c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 -- 172.21.15.17:0/3995490455 >> v1:172.21.15.17:6789/0 conn(0x7f89380c7d20 legacy=0x7f893806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893e25f640 1 -- 172.21.15.17:0/3995490455 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938155ef0 con 0x7f8938152720 2024-03-06T07:21:02.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.318+0000 7f893da5e640 1 --2- 172.21.15.17:0/3995490455 >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 0x7f89380c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.319+0000 7f893e25f640 1 --2- 172.21.15.17:0/3995490455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f89381593c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f893806bd00 tx=0x7f893402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7461e5a5bd0e570 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.319+0000 7f893d25d640 1 -- 172.21.15.17:0/3995490455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934035020 con 0x7f8938152720 2024-03-06T07:21:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.319+0000 7f893d25d640 1 -- 172.21.15.17:0/3995490455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934031870 con 0x7f8938152720 2024-03-06T07:21:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.319+0000 7f893d25d640 1 -- 172.21.15.17:0/3995490455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934031c40 con 0x7f8938152720 2024-03-06T07:21:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 -- 172.21.15.17:0/3995490455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 msgr2=0x7f89381593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 --2- 172.21.15.17:0/3995490455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f89381593c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f893806bd00 tx=0x7f893402b910 comp rx=0 tx=0).stop 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 -- 172.21.15.17:0/3995490455 shutdown_connections 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 --2- 172.21.15.17:0/3995490455 >> v2:172.21.15.163:3300/0 conn(0x7f893806f550 0x7f89380c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 --2- 172.21.15.17:0/3995490455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f89381593c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 -- 172.21.15.17:0/3995490455 >> 172.21.15.17:0/3995490455 conn(0x7f893806ea40 msgr2=0x7f893806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 -- 172.21.15.17:0/3995490455 shutdown_connections 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.320+0000 7f89404c3640 1 -- 172.21.15.17:0/3995490455 wait complete. 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 Processor -- start 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 -- start start 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f893806f550 0x7f8938154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89380c7d20 0x7f8938148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8938158ea0 con 0x7f8938152720 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8938158bc0 con 0x7f89380c7d20 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.321+0000 7f89404c3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89380704a0 con 0x7f893806f550 2024-03-06T07:21:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35498/0 (socket says 172.21.15.17:35498) 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 -- 172.21.15.17:0/3339409273 learned_addr learned my addr 172.21.15.17:0/3339409273 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 -- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f893806f550 msgr2=0x7f8938154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f893806f550 0x7f8938154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 -- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89380c7d20 msgr2=0x7f8938148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89380c7d20 0x7f8938148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 -- 172.21.15.17:0/3339409273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380abca0 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f893ea60640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f893000c760 tx=0x7f893000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930003b70 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8930003d30 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89381ef0d0 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893000aa90 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.322+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89380ac1f0 con 0x7f8938152720 2024-03-06T07:21:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.323+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89380aa470 con 0x7f8938152720 2024-03-06T07:21:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.324+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f893000ac30 con 0x7f8938152720 2024-03-06T07:21:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.324+0000 7f892effd640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.324+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8930089f10 con 0x7f8938152720 2024-03-06T07:21:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.324+0000 7f893e25f640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.325+0000 7f893e25f640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8934036000 tx=0x7f8934030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.329+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89380aa470 con 0x7f8938152720 2024-03-06T07:21:02.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.577+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f893804f330 con 0x7f8938152720 2024-03-06T07:21:02.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.579+0000 7f892effd640 1 -- 172.21.15.17:0/3339409273 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8930058ec0 con 0x7f8938152720 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 msgr2=0x7f891c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8934036000 tx=0x7f8934030000 comp rx=0 tx=0).stop 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 msgr2=0x7f8938149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f893000c760 tx=0x7f893000cc20 comp rx=0 tx=0).stop 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 shutdown_connections 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8938152720 0x7f8938149240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89380c7d20 0x7f8938148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 --2- 172.21.15.17:0/3339409273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f893806f550 0x7f8938154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 >> 172.21.15.17:0/3339409273 conn(0x7f893806ea40 msgr2=0x7f8938072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 shutdown_connections 2024-03-06T07:21:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.582+0000 7f89404c3640 1 -- 172.21.15.17:0/3339409273 wait complete. 2024-03-06T07:21:02.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.694+0000 7f754a615640 1 Processor -- start 2024-03-06T07:21:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.694+0000 7f754a615640 1 -- start start 2024-03-06T07:21:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.695+0000 7f754a615640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 0x7f75440c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.695+0000 7f754a615640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 0x7f75440c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.695+0000 7f754a615640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754406c150 con 0x7f7544155c20 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.695+0000 7f754a615640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754406c480 con 0x7f754414f250 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.695+0000 7f754a615640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754414f630 con 0x7f75440c7350 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7543fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 0x7f75440c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7548bb2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 0x7f75440c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f75437fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7544155c20 0x7f754406c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:35700/0 (socket says 172.21.15.17:35700) 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f75437fe640 1 -- 172.21.15.17:0/131182914 learned_addr learned my addr 172.21.15.17:0/131182914 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7548bb2640 1 -- 172.21.15.17:0/131182914 >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 msgr2=0x7f75440c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7548bb2640 1 --2- 172.21.15.17:0/131182914 >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 0x7f75440c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7548bb2640 1 -- 172.21.15.17:0/131182914 >> v1:172.21.15.163:6789/0 conn(0x7f7544155c20 legacy=0x7f754406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7548bb2640 1 -- 172.21.15.17:0/131182914 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75440c5660 con 0x7f75440c7350 2024-03-06T07:21:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.696+0000 7f7543fff640 1 --2- 172.21.15.17:0/131182914 >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 0x7f75440c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.697+0000 7f7548bb2640 1 --2- 172.21.15.17:0/131182914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 0x7f75440c77e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f753c009d90 tx=0x7f753c02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5e0a773e8d9d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.697+0000 7f7542ffd640 1 -- 172.21.15.17:0/131182914 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c039070 con 0x7f75440c7350 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.697+0000 7f7542ffd640 1 -- 172.21.15.17:0/131182914 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c02cb30 con 0x7f75440c7350 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.697+0000 7f7542ffd640 1 -- 172.21.15.17:0/131182914 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c034440 con 0x7f75440c7350 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 -- 172.21.15.17:0/131182914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 msgr2=0x7f75440c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 --2- 172.21.15.17:0/131182914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 0x7f75440c77e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f753c009d90 tx=0x7f753c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:21:02.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 -- 172.21.15.17:0/131182914 shutdown_connections 2024-03-06T07:21:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 --2- 172.21.15.17:0/131182914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75440c7350 0x7f75440c77e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 --2- 172.21.15.17:0/131182914 >> v2:172.21.15.163:3300/0 conn(0x7f754414f250 0x7f75440c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 -- 172.21.15.17:0/131182914 >> 172.21.15.17:0/131182914 conn(0x7f75440ac4d0 msgr2=0x7f75440ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 -- 172.21.15.17:0/131182914 shutdown_connections 2024-03-06T07:21:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.698+0000 7f754a615640 1 -- 172.21.15.17:0/131182914 wait complete. 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.699+0000 7f754a615640 1 Processor -- start 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.699+0000 7f754a615640 1 -- start start 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754414f250 0x7f7544154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7544155c20 0x7f75441edd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75440c5c20 con 0x7f75440c7350 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754406daf0 con 0x7f7544155c20 2024-03-06T07:21:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75441560a0 con 0x7f754414f250 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35514/0 (socket says 172.21.15.17:35514) 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 -- 172.21.15.17:0/1000343849 learned_addr learned my addr 172.21.15.17:0/1000343849 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f75437fe640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754414f250 0x7f7544154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7548bb2640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7544155c20 0x7f75441edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 -- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754414f250 msgr2=0x7f7544154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754414f250 0x7f7544154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 -- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7544155c20 msgr2=0x7f75441edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7544155c20 0x7f75441edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 -- 172.21.15.17:0/1000343849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7544151f60 con 0x7f75440c7350 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7543fff640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f752c00ec10 tx=0x7f752c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c017070 con 0x7f75440c7350 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c00cc30 con 0x7f75440c7350 2024-03-06T07:21:02.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7544150460 con 0x7f75440c7350 2024-03-06T07:21:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c013470 con 0x7f75440c7350 2024-03-06T07:21:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.700+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7544154a80 con 0x7f75440c7350 2024-03-06T07:21:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.701+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754404f330 con 0x7f75440c7350 2024-03-06T07:21:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.702+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f752c01a070 con 0x7f75440c7350 2024-03-06T07:21:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.702+0000 7f7540ff9640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 0x7f752406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.702+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f752c089ec0 con 0x7f75440c7350 2024-03-06T07:21:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.702+0000 7f75437fe640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 0x7f752406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.703+0000 7f75437fe640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 0x7f752406d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75440c9560 tx=0x7f7534007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.706+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f752c0588b0 con 0x7f75440c7350 2024-03-06T07:21:02.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.891+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f754406d860 con 0x7f75440c7350 2024-03-06T07:21:02.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.891+0000 7f7540ff9640 1 -- 172.21.15.17:0/1000343849 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f752c05c950 con 0x7f75440c7350 2024-03-06T07:21:02.893 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:02.893 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":35,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:20:59.910629+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:02.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 msgr2=0x7f752406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 0x7f752406d9c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75440c9560 tx=0x7f7534007e80 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 msgr2=0x7f7544159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f752c00ec10 tx=0x7f752c00c520 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 shutdown_connections 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f752406b500 0x7f752406d9c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7544155c20 0x7f75441edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754414f250 0x7f7544154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 --2- 172.21.15.17:0/1000343849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75440c7350 0x7f7544159120 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 >> 172.21.15.17:0/1000343849 conn(0x7f75440ac4d0 msgr2=0x7f75440ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 shutdown_connections 2024-03-06T07:21:02.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:02.894+0000 7f754a615640 1 -- 172.21.15.17:0/1000343849 wait complete. 2024-03-06T07:21:02.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:02.907 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:03.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.009+0000 7f9204f8a640 1 Processor -- start 2024-03-06T07:21:03.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.009+0000 7f9204f8a640 1 -- start start 2024-03-06T07:21:03.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f9204f8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f920006b3a0 0x7f9200158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f9204f8a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920006b800 con 0x7f9200159160 2024-03-06T07:21:03.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f9204f8a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92000c5840 con 0x7f920006afc0 2024-03-06T07:21:03.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f9204f8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9200158790 con 0x7f920006b3a0 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9200159160 0x7f920015fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46448/0 (socket says 172.21.15.17:46448) 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fed9d640 1 -- 172.21.15.17:0/417510341 learned_addr learned my addr 172.21.15.17:0/417510341 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91f5d9b640 1 --2- 172.21.15.17:0/417510341 >> v2:172.21.15.163:3300/0 conn(0x7f920006b3a0 0x7f9200158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101276390 0 0) 0x7f920006b800 con 0x7f9200159160 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ec003520 con 0x7f9200159160 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979746374 0 0) 0x7f92000c5840 con 0x7f920006afc0 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920006b800 con 0x7f920006afc0 2024-03-06T07:21:03.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 584681724 0 0) 0x7f91ec003520 con 0x7f9200159160 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91f5d9b640 1 -- 172.21.15.17:0/417510341 >> v1:172.21.15.163:6789/0 conn(0x7f920006afc0 legacy=0x7f92000c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91f5d9b640 1 -- 172.21.15.17:0/417510341 >> v1:172.21.15.17:6789/0 conn(0x7f9200159160 legacy=0x7f920015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.010+0000 7f91f5d9b640 1 -- 172.21.15.17:0/417510341 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000c5b20 con 0x7f920006b3a0 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.011+0000 7f91f5d9b640 1 --2- 172.21.15.17:0/417510341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f920006b3a0 0x7f9200158c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91e80097a0 tx=0x7f91e802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6154a8d876aa3e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.011+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8037070 con 0x7f920006b3a0 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.011+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8030510 con 0x7f920006b3a0 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.011+0000 7f91fdd9b640 1 -- 172.21.15.17:0/417510341 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80019f0 con 0x7f920006b3a0 2024-03-06T07:21:03.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 -- 172.21.15.17:0/417510341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f920006b3a0 msgr2=0x7f9200158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 --2- 172.21.15.17:0/417510341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f920006b3a0 0x7f9200158c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91e80097a0 tx=0x7f91e802b790 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 -- 172.21.15.17:0/417510341 shutdown_connections 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 --2- 172.21.15.17:0/417510341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f920006b3a0 0x7f9200158c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 -- 172.21.15.17:0/417510341 >> 172.21.15.17:0/417510341 conn(0x7f92000c4220 msgr2=0x7f92000c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 -- 172.21.15.17:0/417510341 shutdown_connections 2024-03-06T07:21:03.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.012+0000 7f9204f8a640 1 -- 172.21.15.17:0/417510341 wait complete. 2024-03-06T07:21:03.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.013+0000 7f9204f8a640 1 Processor -- start 2024-03-06T07:21:03.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.013+0000 7f9204f8a640 1 -- start start 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f920006afc0 0x7f9200072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200159160 0x7f920014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92001595f0 con 0x7f920006b3a0 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000c60e0 con 0x7f9200159160 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f9204f8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9200155220 con 0x7f920006afc0 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f91f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f91f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35522/0 (socket says 172.21.15.17:35522) 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f91f5d9b640 1 -- 172.21.15.17:0/4145964779 learned_addr learned my addr 172.21.15.17:0/4145964779 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f91fe59c640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f920006afc0 0x7f9200072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.014+0000 7f91fed9d640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200159160 0x7f920014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 -- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f920006afc0 msgr2=0x7f9200072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f920006afc0 0x7f9200072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 -- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200159160 msgr2=0x7f920014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200159160 0x7f920014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 -- 172.21.15.17:0/4145964779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200157ae0 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f5d9b640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f91e802b7d0 tx=0x7f91e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8040020 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e8038040 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9200157dc0 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8037070 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.015+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001ee790 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.016+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920004f330 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.016+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91e8043020 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.017+0000 7f91f77fe640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 0x7f91e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.017+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f91e80ab300 con 0x7f920006b3a0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.017+0000 7f91fe59c640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 0x7f91e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.017+0000 7f91fe59c640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 0x7f91e006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91f00011e0 tx=0x7f91f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.021+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f91e804a030 con 0x7f920006b3a0 2024-03-06T07:21:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.200+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f920006be40 con 0x7f920006b3a0 2024-03-06T07:21:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.201+0000 7f91f77fe640 1 -- 172.21.15.17:0/4145964779 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91e8079d80 con 0x7f920006b3a0 2024-03-06T07:21:03.202 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:21:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 msgr2=0x7f91e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 0x7f91e006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f91f00011e0 tx=0x7f91f0009040 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 msgr2=0x7f92001eafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f91e802b7d0 tx=0x7f91e80046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 shutdown_connections 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91e006b4d0 0x7f91e006d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9200159160 0x7f920014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920006b3a0 0x7f92001eafd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 --2- 172.21.15.17:0/4145964779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f920006afc0 0x7f9200072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.202+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 >> 172.21.15.17:0/4145964779 conn(0x7f92000c4220 msgr2=0x7f9200151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.203+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 shutdown_connections 2024-03-06T07:21:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.203+0000 7f9204f8a640 1 -- 172.21.15.17:0/4145964779 wait complete. 2024-03-06T07:21:03.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:21:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.314+0000 7f4b7f4eb640 1 Processor -- start 2024-03-06T07:21:03.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.314+0000 7f4b7f4eb640 1 -- start start 2024-03-06T07:21:03.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7f4eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7f4eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b780c1aa0 0x7f4b780c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7f4eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b780c28d0 con 0x7f4b780c5c60 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7f4eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780c2680 con 0x7f4b780c1aa0 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7f4eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780c2bb0 con 0x7f4b7814f790 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b780c1aa0 0x7f4b780c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b780c1aa0 0x7f4b780c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35524/0 (socket says 172.21.15.17:35524) 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 -- 172.21.15.17:0/3564404004 learned_addr learned my addr 172.21.15.17:0/3564404004 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7d287640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 -- 172.21.15.17:0/3564404004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 msgr2=0x7f4b781529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 -- 172.21.15.17:0/3564404004 >> v1:172.21.15.163:6789/0 conn(0x7f4b780c5c60 legacy=0x7f4b78155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7ca86640 1 -- 172.21.15.17:0/3564404004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780c2e90 con 0x7f4b780c1aa0 2024-03-06T07:21:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.315+0000 7f4b7d287640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:03.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.316+0000 7f4b7ca86640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c1aa0 0x7f4b780c5720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009870 tx=0x7f4b6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b27979ddc086da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.316+0000 7f4b6ffff640 1 -- 172.21.15.17:0/3564404004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68035020 con 0x7f4b780c1aa0 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.316+0000 7f4b6ffff640 1 -- 172.21.15.17:0/3564404004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68031640 con 0x7f4b780c1aa0 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.316+0000 7f4b6ffff640 1 -- 172.21.15.17:0/3564404004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6802d040 con 0x7f4b780c1aa0 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/3564404004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c1aa0 msgr2=0x7f4b780c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c1aa0 0x7f4b780c5720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009870 tx=0x7f4b6802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/3564404004 shutdown_connections 2024-03-06T07:21:03.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c1aa0 0x7f4b780c5720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/3564404004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/3564404004 >> 172.21.15.17:0/3564404004 conn(0x7f4b780c6710 msgr2=0x7f4b780c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/3564404004 shutdown_connections 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.317+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/3564404004 wait complete. 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.318+0000 7f4b7f4eb640 1 Processor -- start 2024-03-06T07:21:03.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.318+0000 7f4b7f4eb640 1 -- start start 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b780c1aa0 0x7f4b780bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7806cc00 con 0x7f4b780c5c60 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780c0d60 con 0x7f4b780c1aa0 2024-03-06T07:21:03.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7806cd80 con 0x7f4b7814f790 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35530/0 (socket says 172.21.15.17:35530) 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 -- 172.21.15.17:0/797822300 learned_addr learned my addr 172.21.15.17:0/797822300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7d287640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b780c1aa0 0x7f4b780bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7da88640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 -- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 msgr2=0x7f4b781eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 -- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b780c1aa0 msgr2=0x7f4b780bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b780c1aa0 0x7f4b780bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 -- 172.21.15.17:0/797822300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b781deb30 con 0x7f4b780c5c60 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7ca86640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000c00 tx=0x7f4b680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6803ac60 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6803ae00 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b781dedd0 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.319+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68044610 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.320+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b781df390 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.320+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b781df5d0 con 0x7f4b780c5c60 2024-03-06T07:21:03.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.321+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b68038020 con 0x7f4b780c5c60 2024-03-06T07:21:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.321+0000 7f4b6dffb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.321+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4b680abda0 con 0x7f4b780c5c60 2024-03-06T07:21:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.322+0000 7f4b7d287640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.322+0000 7f4b7d287640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400a950 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.326+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b781df5d0 con 0x7f4b780c5c60 2024-03-06T07:21:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.508+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b78069460 con 0x7f4b780c5c60 2024-03-06T07:21:03.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.509+0000 7f4b6dffb640 1 -- 172.21.15.17:0/797822300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4b78069460 con 0x7f4b780c5c60 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:03.511 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:21:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 msgr2=0x7f4b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400a950 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-03-06T07:21:03.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 msgr2=0x7f4b78149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b68000c00 tx=0x7f4b680046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 shutdown_connections 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b5c06b4d0 0x7f4b5c06d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b7814f790 0x7f4b781eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b780c5c60 0x7f4b78149d60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 --2- 172.21.15.17:0/797822300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b780c1aa0 0x7f4b780bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 >> 172.21.15.17:0/797822300 conn(0x7f4b780c6710 msgr2=0x7f4b780c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 shutdown_connections 2024-03-06T07:21:03.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.511+0000 7f4b7f4eb640 1 -- 172.21.15.17:0/797822300 wait complete. 2024-03-06T07:21:03.523 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:21:03.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:21:03.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.626+0000 7fd2186b5640 1 Processor -- start 2024-03-06T07:21:03.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.626+0000 7fd2186b5640 1 -- start start 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd2186b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 0x7fd210153940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd2186b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd210072fc0 0x7fd2100724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd2186b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd210153e80 con 0x7fd210072a10 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd2186b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2100c6220 con 0x7fd210072fc0 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd2186b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd210070210 con 0x7fd2100c5e40 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd215c50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd210072fc0 0x7fd2100724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd215c50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd210072fc0 0x7fd2100724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35536/0 (socket says 172.21.15.17:35536) 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.627+0000 7fd215c50640 1 -- 172.21.15.17:0/559139490 learned_addr learned my addr 172.21.15.17:0/559139490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:03.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd216451640 1 --2- 172.21.15.17:0/559139490 >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 0x7fd210153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd215c50640 1 -- 172.21.15.17:0/559139490 >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 msgr2=0x7fd210153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd215c50640 1 --2- 172.21.15.17:0/559139490 >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 0x7fd210153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd215c50640 1 -- 172.21.15.17:0/559139490 >> v1:172.21.15.163:6789/0 conn(0x7fd210072a10 legacy=0x7fd2100c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd215c50640 1 -- 172.21.15.17:0/559139490 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100704d0 con 0x7fd210072fc0 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd216451640 1 --2- 172.21.15.17:0/559139490 >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 0x7fd210153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd215c50640 1 --2- 172.21.15.17:0/559139490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072fc0 0x7fd2100724d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd210073570 tx=0x7fd20402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7b35f4f43bbdb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.628+0000 7fd21544f640 1 -- 172.21.15.17:0/559139490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040317c0 con 0x7fd210072fc0 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd21544f640 1 -- 172.21.15.17:0/559139490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204037070 con 0x7fd210072fc0 2024-03-06T07:21:03.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd21544f640 1 -- 172.21.15.17:0/559139490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204031b80 con 0x7fd210072fc0 2024-03-06T07:21:03.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 -- 172.21.15.17:0/559139490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072fc0 msgr2=0x7fd2100724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 --2- 172.21.15.17:0/559139490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072fc0 0x7fd2100724d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd210073570 tx=0x7fd20402b860 comp rx=0 tx=0).stop 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 -- 172.21.15.17:0/559139490 shutdown_connections 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 --2- 172.21.15.17:0/559139490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072fc0 0x7fd2100724d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 --2- 172.21.15.17:0/559139490 >> v2:172.21.15.163:3300/0 conn(0x7fd2100c5e40 0x7fd210153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 -- 172.21.15.17:0/559139490 >> 172.21.15.17:0/559139490 conn(0x7fd2100c88f0 msgr2=0x7fd2100c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 -- 172.21.15.17:0/559139490 shutdown_connections 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.629+0000 7fd2186b5640 1 -- 172.21.15.17:0/559139490 wait complete. 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.630+0000 7fd2186b5640 1 Processor -- start 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.630+0000 7fd2186b5640 1 -- start start 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd210072fc0 0x7fd2100c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2100c5e40 0x7fd2101eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006fbd0 con 0x7fd210072a10 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006f8f0 con 0x7fd2100c5e40 2024-03-06T07:21:03.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd2186b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100c4de0 con 0x7fd210072fc0 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35538/0 (socket says 172.21.15.17:35538) 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 -- 172.21.15.17:0/188284597 learned_addr learned my addr 172.21.15.17:0/188284597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 -- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd210072fc0 msgr2=0x7fd2100c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd210072fc0 0x7fd2100c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 -- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2100c5e40 msgr2=0x7fd2101eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2100c5e40 0x7fd2101eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 -- 172.21.15.17:0/188284597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21014be00 con 0x7fd210072a10 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.631+0000 7fd216451640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ce30 tx=0x7fd20800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.632+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800f890 con 0x7fd210072a10 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.632+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20800fa30 con 0x7fd210072a10 2024-03-06T07:21:03.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.632+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2101eedd0 con 0x7fd210072a10 2024-03-06T07:21:03.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.632+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800fe00 con 0x7fd210072a10 2024-03-06T07:21:03.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.632+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21006c0d0 con 0x7fd210072a10 2024-03-06T07:21:03.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.633+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210153b70 con 0x7fd210072a10 2024-03-06T07:21:03.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.633+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd208018400 con 0x7fd210072a10 2024-03-06T07:21:03.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.634+0000 7fd20effd640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:03.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.634+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd20808adc0 con 0x7fd210072a10 2024-03-06T07:21:03.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.634+0000 7fd215c50640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:03.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.634+0000 7fd215c50640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd204036000 tx=0x7fd204030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:03.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.638+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd210153b70 con 0x7fd210072a10 2024-03-06T07:21:03.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:03.819+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd2100c3880 con 0x7fd210072a10 2024-03-06T07:21:04.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.067+0000 7fd20effd640 1 -- 172.21.15.17:0/188284597 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd208014030 con 0x7fd210072a10 2024-03-06T07:21:04.069 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 msgr2=0x7fd1fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd204036000 tx=0x7fd204030000 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 msgr2=0x7fd210156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd20800ce30 tx=0x7fd20800a650 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 shutdown_connections 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd1fc06b4d0 0x7fd1fc06d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2100c5e40 0x7fd2101eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd210072fc0 0x7fd2100c1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 --2- 172.21.15.17:0/188284597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd210072a10 0x7fd210156b60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 >> 172.21.15.17:0/188284597 conn(0x7fd2100c88f0 msgr2=0x7fd2100c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 shutdown_connections 2024-03-06T07:21:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.070+0000 7fd2186b5640 1 -- 172.21.15.17:0/188284597 wait complete. 2024-03-06T07:21:04.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:21:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.193+0000 7f9b3cb25640 1 Processor -- start 2024-03-06T07:21:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.193+0000 7f9b3cb25640 1 -- start start 2024-03-06T07:21:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b3cb25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b380c5370 0x7f9b380c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b3cb25640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 0x7f9b381559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b3cb25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b38152dd0 con 0x7f9b381529f0 2024-03-06T07:21:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b3cb25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b38159710 con 0x7f9b380c5370 2024-03-06T07:21:04.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b3cb25640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b380c68b0 con 0x7f9b38155530 2024-03-06T07:21:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b380c5370 0x7f9b380c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b380c5370 0x7f9b380c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35542/0 (socket says 172.21.15.17:35542) 2024-03-06T07:21:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 -- 172.21.15.17:0/4076386539 learned_addr learned my addr 172.21.15.17:0/4076386539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 -- 172.21.15.17:0/4076386539 >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 msgr2=0x7f9b381559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b36d9d640 1 --2- 172.21.15.17:0/4076386539 >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 0x7f9b381559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 --2- 172.21.15.17:0/4076386539 >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 0x7f9b381559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 -- 172.21.15.17:0/4076386539 >> v1:172.21.15.17:6789/0 conn(0x7f9b381529f0 legacy=0x7f9b380c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b2ffff640 1 -- 172.21.15.17:0/4076386539 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38158b00 con 0x7f9b380c5370 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.194+0000 7f9b36d9d640 1 --2- 172.21.15.17:0/4076386539 >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 0x7f9b381559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.195+0000 7f9b2ffff640 1 --2- 172.21.15.17:0/4076386539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b380c5370 0x7f9b380c57d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b380735c0 tx=0x7f9b2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9adb1b16e5dd47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.195+0000 7f9b35d9b640 1 -- 172.21.15.17:0/4076386539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035020 con 0x7f9b380c5370 2024-03-06T07:21:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.195+0000 7f9b35d9b640 1 -- 172.21.15.17:0/4076386539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b20031870 con 0x7f9b380c5370 2024-03-06T07:21:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.195+0000 7f9b35d9b640 1 -- 172.21.15.17:0/4076386539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20031c40 con 0x7f9b380c5370 2024-03-06T07:21:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 -- 172.21.15.17:0/4076386539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b380c5370 msgr2=0x7f9b380c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/4076386539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b380c5370 0x7f9b380c57d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9b380735c0 tx=0x7f9b2002b910 comp rx=0 tx=0).stop 2024-03-06T07:21:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 -- 172.21.15.17:0/4076386539 shutdown_connections 2024-03-06T07:21:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/4076386539 >> v2:172.21.15.163:3300/0 conn(0x7f9b38155530 0x7f9b381559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/4076386539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b380c5370 0x7f9b380c57d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 -- 172.21.15.17:0/4076386539 >> 172.21.15.17:0/4076386539 conn(0x7f9b380ac580 msgr2=0x7f9b380ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 -- 172.21.15.17:0/4076386539 shutdown_connections 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.196+0000 7f9b3cb25640 1 -- 172.21.15.17:0/4076386539 wait complete. 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 Processor -- start 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 -- start start 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b380c5370 0x7f9b3814e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 0x7f9b38151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381590c0 con 0x7f9b381529f0 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b380c9000 con 0x7f9b38155530 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.197+0000 7f9b3cb25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381593a0 con 0x7f9b380c5370 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35544/0 (socket says 172.21.15.17:35544) 2024-03-06T07:21:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 -- 172.21.15.17:0/2181145989 learned_addr learned my addr 172.21.15.17:0/2181145989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 -- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b380c5370 msgr2=0x7f9b3814e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b380c5370 0x7f9b3814e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b36d9d640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 0x7f9b38151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 -- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 msgr2=0x7f9b38151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 0x7f9b38151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 -- 172.21.15.17:0/2181145989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b381eeaf0 con 0x7f9b381529f0 2024-03-06T07:21:04.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b36d9d640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 0x7f9b38151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:04.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2ffff640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b2002b950 tx=0x7f9b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20040070 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2003a420 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b380bde70 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035040 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.198+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b381ee010 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.199+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3804f330 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.200+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b2004b060 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.200+0000 7f9b2effd640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.200+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f9b200ac070 con 0x7f9b381529f0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.200+0000 7f9b3659c640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.201+0000 7f9b3659c640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 0x7f9b1806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b28000f60 tx=0x7f9b280079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.205+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b2007afd0 con 0x7f9b381529f0 2024-03-06T07:21:04.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:04.386+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9b380c9b10 con 0x7f9b381529f0 2024-03-06T07:21:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.073+0000 7f9b2effd640 1 -- 172.21.15.17:0/2181145989 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9b2007a8d0 con 0x7f9b381529f0 2024-03-06T07:21:05.075 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:21:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.076+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 msgr2=0x7f9b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.076+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 0x7f9b1806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b28000f60 tx=0x7f9b280079d0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.076+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 msgr2=0x7f9b38151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.076+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9b2002b950 tx=0x7f9b200046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 shutdown_connections 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9b1806b4d0 0x7f9b1806d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b38155530 0x7f9b38151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b381529f0 0x7f9b38151100 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 --2- 172.21.15.17:0/2181145989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b380c5370 0x7f9b3814e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.077+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 >> 172.21.15.17:0/2181145989 conn(0x7f9b380ac580 msgr2=0x7f9b380ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.078+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 shutdown_connections 2024-03-06T07:21:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.079+0000 7f9b3cb25640 1 -- 172.21.15.17:0/2181145989 wait complete. 2024-03-06T07:21:05.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:21:05.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.214+0000 7f70c29e9640 1 Processor -- start 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.214+0000 7f70c29e9640 1 -- start start 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70c29e9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70bc0c3710 0x7f70bc1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70c29e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc158800 con 0x7f70bc0c3280 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70c29e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc158ae0 con 0x7f70bc06d9f0 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70c29e9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc158dc0 con 0x7f70bc0c3710 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70bb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f70bc0c3280 0x7f70bc1551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46458/0 (socket says 172.21.15.17:46458) 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70bb7fe640 1 -- 172.21.15.17:0/1988745018 learned_addr learned my addr 172.21.15.17:0/1988745018 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70c0f86640 1 --2- 172.21.15.17:0/1988745018 >> v2:172.21.15.163:3300/0 conn(0x7f70bc0c3710 0x7f70bc1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518608294 0 0) 0x7f70bc158800 con 0x7f70bc0c3280 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.215+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a8003520 con 0x7f70bc0c3280 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3921932549 0 0) 0x7f70a8003520 con 0x7f70bc0c3280 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 >> v2:172.21.15.163:3300/0 conn(0x7f70bc0c3710 msgr2=0x7f70bc1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 --2- 172.21.15.17:0/1988745018 >> v2:172.21.15.163:3300/0 conn(0x7f70bc0c3710 0x7f70bc1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 >> v1:172.21.15.163:6789/0 conn(0x7f70bc06d9f0 legacy=0x7f70bc0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc1590a0 con 0x7f70bc0c3280 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f70ac001670 con 0x7f70bc0c3280 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 >> v1:172.21.15.17:6789/0 conn(0x7f70bc0c3280 legacy=0x7f70bc1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a80039b0 0x7f70a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a8004800 0x7f70a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 0x7f70a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc158dc0 con 0x7f70a80051c0 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a800c160 con 0x7f70a80039b0 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a800c360 con 0x7f70a8004800 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 0x7f70a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70bbfff640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a80039b0 0x7f70a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70bb7fe640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a8004800 0x7f70a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 -- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a8004800 msgr2=0x7f70a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a8004800 0x7f70a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 -- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a80039b0 msgr2=0x7f70a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70a80039b0 0x7f70a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.216+0000 7f70c0f86640 1 -- 172.21.15.17:0/1988745018 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8003520 con 0x7f70a80051c0 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.217+0000 7f70c0f86640 1 -- 172.21.15.17:0/1988745018 reap_dead start 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.217+0000 7f70c0f86640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 0x7f70a800b9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009fc0 tx=0x7f70a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653b28c388f78318 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.217+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4009c10 con 0x7f70a80051c0 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.217+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b402cc60 con 0x7f70a80051c0 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.217+0000 7f70baffd640 1 -- 172.21.15.17:0/1988745018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4003bd0 con 0x7f70a80051c0 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 -- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 msgr2=0x7f70a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 0x7f70a800b9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009fc0 tx=0x7f70a8005830 comp rx=0 tx=0).stop 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 -- 172.21.15.17:0/1988745018 shutdown_connections 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 --2- 172.21.15.17:0/1988745018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70a80051c0 0x7f70a800b9c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 -- 172.21.15.17:0/1988745018 >> 172.21.15.17:0/1988745018 conn(0x7f70bc0aa630 msgr2=0x7f70bc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:05.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 -- 172.21.15.17:0/1988745018 shutdown_connections 2024-03-06T07:21:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.218+0000 7f70c29e9640 1 -- 172.21.15.17:0/1988745018 wait complete. 2024-03-06T07:21:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.219+0000 7f70c29e9640 1 Processor -- start 2024-03-06T07:21:05.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.219+0000 7f70c29e9640 1 -- start start 2024-03-06T07:21:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a80051c0 0x7f70bc14f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70bc149bd0 0x7f70bc148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc06df50 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc06ddd0 con 0x7f70bc149bd0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc0c6910 con 0x7f70a80051c0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35568/0 (socket says 172.21.15.17:35568) 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 -- 172.21.15.17:0/3818222952 learned_addr learned my addr 172.21.15.17:0/3818222952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bbfff640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a80051c0 0x7f70bc14f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 -- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a80051c0 msgr2=0x7f70bc14f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c0f86640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70bc149bd0 0x7f70bc148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a80051c0 0x7f70bc14f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 -- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70bc149bd0 msgr2=0x7f70bc148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70bc149bd0 0x7f70bc148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 -- 172.21.15.17:0/3818222952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc150090 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70bb7fe640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0072f0 tx=0x7f70ac0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00f960 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac018070 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc0ac0d0 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00fd60 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.220+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc0abc10 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.222+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc0abef0 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.222+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70ac01d460 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.222+0000 7f70b8ff9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 0x7f709c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:05.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.222+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f70ac08a340 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.223+0000 7f70bbfff640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 0x7f709c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.223+0000 7f70bbfff640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 0x7f709c06d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:05.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.227+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70bc0abef0 con 0x7f70bc0ac5b0 2024-03-06T07:21:05.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:05.444+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70bc0ac2a0 con 0x7f70bc0ac5b0 2024-03-06T07:21:06.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.085+0000 7f70b8ff9640 1 -- 172.21.15.17:0/3818222952 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f70bc150090 con 0x7f70bc0ac5b0 2024-03-06T07:21:06.087 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 5 and data pool 6 2024-03-06T07:21:06.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 msgr2=0x7f709c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 0x7f709c06d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:06.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 msgr2=0x7f70bc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0072f0 tx=0x7f70ac0077b0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 shutdown_connections 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709c06b500 0x7f709c06d9c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70bc149bd0 0x7f70bc148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70bc0ac5b0 0x7f70bc06f080 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 --2- 172.21.15.17:0/3818222952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70a80051c0 0x7f70bc14f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 >> 172.21.15.17:0/3818222952 conn(0x7f70bc0aa630 msgr2=0x7f70bc0723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 shutdown_connections 2024-03-06T07:21:06.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.100+0000 7f70c29e9640 1 -- 172.21.15.17:0/3818222952 wait complete. 2024-03-06T07:21:06.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:06.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.223+0000 7fd1b6a39640 1 Processor -- start 2024-03-06T07:21:06.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.223+0000 7fd1b6a39640 1 -- start start 2024-03-06T07:21:06.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b6a39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd1b0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b6a39640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0156180 con 0x7fd1b00c5330 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b6a39640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b0159630 con 0x7fd1b00c8a00 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b6a39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01587b0 con 0x7fd1b0152920 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b4fd6640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1b00c5330 0x7fd1b015fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46460/0 (socket says 172.21.15.17:46460) 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2876330776 learned_addr learned my addr 172.21.15.17:0/2876330776 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080787106 0 0) 0x7fd1a8001490 con 0x7fd1b00c5330 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1affff640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd1b0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.224+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd198003520 con 0x7fd1b00c5330 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730858467 0 0) 0x7fd198003520 con 0x7fd1b00c5330 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 msgr2=0x7fd1b0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd1b0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 >> v1:172.21.15.163:6789/0 conn(0x7fd1b00c8a00 legacy=0x7fd1b0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00c7760 con 0x7fd1b00c5330 2024-03-06T07:21:06.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd1a8001bc0 con 0x7fd1b00c5330 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 >> v1:172.21.15.17:6789/0 conn(0x7fd1b00c5330 legacy=0x7fd1b015fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd198004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01587b0 con 0x7fd1980039e0 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800baf0 con 0x7fd198004d40 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19800bcf0 con 0x7fd1b0152920 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1af7fe640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd198004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1b4fd6640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 -- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 msgr2=0x7fd198004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b0152920 0x7fd198004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 -- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd198004d40 msgr2=0x7fd1980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1affff640 1 -- 172.21.15.17:0/2876330776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198003520 con 0x7fd1980039e0 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.225+0000 7fd1b4fd6640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd198004d40 0x7fd1980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1affff640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009ac0 tx=0x7fd198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63815a65f4a801cc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4039070 con 0x7fd1980039e0 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a402c600 con 0x7fd1980039e0 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1aeffd640 1 -- 172.21.15.17:0/2876330776 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4009690 con 0x7fd1980039e0 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2876330776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1980039e0 msgr2=0x7fd198003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2876330776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1980039e0 0x7fd198003e00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009ac0 tx=0x7fd198005830 comp rx=0 tx=0).stop 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2876330776 reap_dead start 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.226+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2876330776 shutdown_connections 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.227+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2876330776 >> 172.21.15.17:0/2876330776 conn(0x7fd1b00ac560 msgr2=0x7fd1b00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.227+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2876330776 shutdown_connections 2024-03-06T07:21:06.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.227+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2876330776 wait complete. 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 Processor -- start 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 -- start start 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1980039e0 0x7fd1b0154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b00c0d00 0x7fd1b014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0158a00 con 0x7fd1b00c1160 2024-03-06T07:21:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00c7d20 con 0x7fd1980039e0 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.228+0000 7fd1b6a39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b006d030 con 0x7fd1b00c0d00 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35586/0 (socket says 172.21.15.17:35586) 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2492144980 learned_addr learned my addr 172.21.15.17:0/2492144980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1af7fe640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b00c0d00 0x7fd1b014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1affff640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1980039e0 0x7fd1b0154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b00c0d00 msgr2=0x7fd1b014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b00c0d00 0x7fd1b014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1980039e0 msgr2=0x7fd1b0154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1980039e0 0x7fd1b0154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 -- 172.21.15.17:0/2492144980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b0154970 con 0x7fd1b00c1160 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b4fd6640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80012e0 tx=0x7fd1a800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8017020 con 0x7fd1b00c1160 2024-03-06T07:21:06.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800eb90 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b0153df0 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8013420 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.229+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01543b0 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.230+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b004f330 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.231+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1a801a020 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.231+0000 7fd1acff9640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.231+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1a808a3d0 con 0x7fd1b00c1160 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.231+0000 7fd1affff640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.232+0000 7fd1affff640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036000 tx=0x7fd1a4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.235+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1a8021030 con 0x7fd1b00c1160 2024-03-06T07:21:06.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.417+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1b006cda0 con 0x7fd1b00c1160 2024-03-06T07:21:06.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.418+0000 7fd1acff9640 1 -- 172.21.15.17:0/2492144980 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16955 (secure 0 0 0) 0x7fd1b0156180 con 0x7fd1b00c1160 2024-03-06T07:21:06.420 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:06.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":38,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:05.446036+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 msgr2=0x7fd19c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036000 tx=0x7fd1a4030000 comp rx=0 tx=0).stop 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 msgr2=0x7fd1b014c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80012e0 tx=0x7fd1a800e530 comp rx=0 tx=0).stop 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 shutdown_connections 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1b00c1160 0x7fd1b014c110 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b00c0d00 0x7fd1b014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 --2- 172.21.15.17:0/2492144980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1980039e0 0x7fd1b0154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 >> 172.21.15.17:0/2492144980 conn(0x7fd1b00ac560 msgr2=0x7fd1b00aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 shutdown_connections 2024-03-06T07:21:06.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.420+0000 7fd1b6a39640 1 -- 172.21.15.17:0/2492144980 wait complete. 2024-03-06T07:21:06.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:06.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.533+0000 7f4ed0d56640 1 Processor -- start 2024-03-06T07:21:06.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.533+0000 7f4ed0d56640 1 -- start start 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.533+0000 7f4ed0d56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4ecc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.533+0000 7f4ed0d56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0c73d0 con 0x7f4ecc14f360 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ed0d56640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0be2e0 con 0x7f4ecc0c88c0 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ed0d56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc159700 con 0x7f4ecc159270 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4eca59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4ecc14f360 0x7f4ecc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46470/0 (socket says 172.21.15.17:46470) 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4eca59c640 1 -- 172.21.15.17:0/3953628162 learned_addr learned my addr 172.21.15.17:0/3953628162 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4ecc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835436380 0 0) 0x7f4ecc0c73d0 con 0x7f4ecc14f360 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb8003520 con 0x7f4ecc14f360 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997632692 0 0) 0x7f4eb8003520 con 0x7f4ecc14f360 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 msgr2=0x7f4ecc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4ecc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 >> v1:172.21.15.163:6789/0 conn(0x7f4ecc0c88c0 legacy=0x7f4ecc158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0c7670 con 0x7f4ecc14f360 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4ebc001670 con 0x7f4ecc14f360 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 >> v1:172.21.15.17:6789/0 conn(0x7f4ecc14f360 legacy=0x7f4ecc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4eb80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb8003f60 0x7f4eb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4eb8004d40 0x7f4eb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc159700 con 0x7f4eb8004d40 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800bb50 con 0x7f4eb8003f60 2024-03-06T07:21:06.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800bd50 con 0x7f4ecc159270 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4eb8004d40 0x7f4eb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4eca59c640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4eb80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.534+0000 7f4ec3fff640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb8003f60 0x7f4eb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 -- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 msgr2=0x7f4eb80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc159270 0x7f4eb80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 -- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb8003f60 msgr2=0x7f4eb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4eb8003f60 0x7f4eb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 -- 172.21.15.17:0/3953628162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8003520 con 0x7f4eb8004d40 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.535+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4eb8004d40 0x7f4eb80051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009fc0 tx=0x7f4eb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d825b58346cc30b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4039070 con 0x7f4eb8004d40 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec402cc20 con 0x7f4eb8004d40 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ec9d9b640 1 -- 172.21.15.17:0/3953628162 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4009c10 con 0x7f4eb8004d40 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 -- 172.21.15.17:0/3953628162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4eb8004d40 msgr2=0x7f4eb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/3953628162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4eb8004d40 0x7f4eb80051c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009fc0 tx=0x7f4eb8005830 comp rx=0 tx=0).stop 2024-03-06T07:21:06.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ecad9d640 1 -- 172.21.15.17:0/3953628162 reap_dead start 2024-03-06T07:21:06.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 -- 172.21.15.17:0/3953628162 shutdown_connections 2024-03-06T07:21:06.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 -- 172.21.15.17:0/3953628162 >> 172.21.15.17:0/3953628162 conn(0x7f4ecc0ac570 msgr2=0x7f4ecc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 -- 172.21.15.17:0/3953628162 shutdown_connections 2024-03-06T07:21:06.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.536+0000 7f4ed0d56640 1 -- 172.21.15.17:0/3953628162 wait complete. 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.537+0000 7f4ed0d56640 1 Processor -- start 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.537+0000 7f4ed0d56640 1 -- start start 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 0x7f4ecc1ee990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 0x7f4ecc157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c7c30 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c5a50 con 0x7f4ecc14bc50 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ed0d56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c5bd0 con 0x7f4ecc1ee5b0 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35598/0 (socket says 172.21.15.17:35598) 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 -- 172.21.15.17:0/1639072528 learned_addr learned my addr 172.21.15.17:0/1639072528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 0x7f4ecc157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4eca59c640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 0x7f4ecc1ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 -- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 msgr2=0x7f4ecc1ee990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 0x7f4ecc1ee990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 -- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 msgr2=0x7f4ecc157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 0x7f4ecc157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ec3fff640 1 -- 172.21.15.17:0/1639072528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc157790 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4ecad9d640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 0x7f4ecc157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.538+0000 7f4eca59c640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 0x7f4ecc1ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ec3fff640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400f820 tx=0x7f4eb400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40050d0 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4018070 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc1ef2f0 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4005490 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.539+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc154ad0 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.540+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc04f330 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.540+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4eb400d6d0 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.541+0000 7f4ec2ffd640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 0x7f4eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.541+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4eb408a0c0 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.541+0000 7f4eca59c640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 0x7f4eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.541+0000 7f4eca59c640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 0x7f4eac06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004840 tx=0x7f4ebc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.545+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4eb4058e40 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.730+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ecc0c8d50 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.731+0000 7f4ec2ffd640 1 -- 172.21.15.17:0/1639072528 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f4eb4058740 con 0x7f4ecc14c1a0 2024-03-06T07:21:06.733 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:06.733 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":38,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:05.446036+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 msgr2=0x7f4eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 0x7f4eac06d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc004840 tx=0x7f4ebc009040 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 msgr2=0x7f4ecc156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400f820 tx=0x7f4eb400fce0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 shutdown_connections 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eac06b4d0 0x7f4eac06d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ecc14bc50 0x7f4ecc157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ecc14c1a0 0x7f4ecc156b00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 --2- 172.21.15.17:0/1639072528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ecc1ee5b0 0x7f4ecc1ee990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 >> 172.21.15.17:0/1639072528 conn(0x7f4ecc0ac570 msgr2=0x7f4ecc14e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 shutdown_connections 2024-03-06T07:21:06.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.733+0000 7f4ed0d56640 1 -- 172.21.15.17:0/1639072528 wait complete. 2024-03-06T07:21:06.746 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:21:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.849+0000 7f7735e15640 1 Processor -- start 2024-03-06T07:21:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.849+0000 7f7735e15640 1 -- start start 2024-03-06T07:21:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f7735e15640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77300b0070 0x7f7730155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f7735e15640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300b0450 con 0x7f77300c5380 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f7735e15640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773006cf20 con 0x7f773006ca70 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f7735e15640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773006c020 con 0x7f77300b0070 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f772f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77300b0070 0x7f7730155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f772f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77300b0070 0x7f7730155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35612/0 (socket says 172.21.15.17:35612) 2024-03-06T07:21:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.850+0000 7f772f7fe640 1 -- 172.21.15.17:0/1849402643 learned_addr learned my addr 172.21.15.17:0/1849402643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.851+0000 7f772f7fe640 1 -- 172.21.15.17:0/1849402643 >> v1:172.21.15.163:6789/0 conn(0x7f773006ca70 legacy=0x7f77300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.851+0000 7f772f7fe640 1 -- 172.21.15.17:0/1849402643 >> v1:172.21.15.17:6789/0 conn(0x7f77300c5380 legacy=0x7f773014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.851+0000 7f772f7fe640 1 -- 172.21.15.17:0/1849402643 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300c77f0 con 0x7f77300b0070 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.851+0000 7f772f7fe640 1 --2- 172.21.15.17:0/1849402643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300b0070 0x7f7730155c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7720009870 tx=0x7f772002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957c0a09af0069c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.851+0000 7f772e7fc640 1 -- 172.21.15.17:0/1849402643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720034070 con 0x7f77300b0070 2024-03-06T07:21:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f772e7fc640 1 -- 172.21.15.17:0/1849402643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720030860 con 0x7f77300b0070 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f772e7fc640 1 -- 172.21.15.17:0/1849402643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720030c30 con 0x7f77300b0070 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 -- 172.21.15.17:0/1849402643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300b0070 msgr2=0x7f7730155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 --2- 172.21.15.17:0/1849402643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300b0070 0x7f7730155c30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7720009870 tx=0x7f772002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 -- 172.21.15.17:0/1849402643 shutdown_connections 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 --2- 172.21.15.17:0/1849402643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300b0070 0x7f7730155c30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 -- 172.21.15.17:0/1849402643 >> 172.21.15.17:0/1849402643 conn(0x7f77300ac590 msgr2=0x7f77300ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 -- 172.21.15.17:0/1849402643 shutdown_connections 2024-03-06T07:21:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.852+0000 7f7735e15640 1 -- 172.21.15.17:0/1849402643 wait complete. 2024-03-06T07:21:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.853+0000 7f7735e15640 1 Processor -- start 2024-03-06T07:21:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.853+0000 7f7735e15640 1 -- start start 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f773006ca70 0x7f7730154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77300b0070 0x7f77300c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773006c550 con 0x7f77300c5380 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773006c270 con 0x7f77300b0070 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f7735e15640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773006da80 con 0x7f773006ca70 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35626/0 (socket says 172.21.15.17:35626) 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 -- 172.21.15.17:0/3480432052 learned_addr learned my addr 172.21.15.17:0/3480432052 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772f7fe640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f773006ca70 0x7f7730154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772effd640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77300b0070 0x7f77300c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 -- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f773006ca70 msgr2=0x7f7730154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f773006ca70 0x7f7730154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 -- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77300b0070 msgr2=0x7f77300c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77300b0070 0x7f77300c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.854+0000 7f772ffff640 1 -- 172.21.15.17:0/3480432052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300c4a20 con 0x7f77300c5380 2024-03-06T07:21:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f772ffff640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f772800c7a0 tx=0x7f772800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003b70 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728003d30 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77300c27c0 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800aa90 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.855+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77301eeb70 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.856+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77300c2990 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.856+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f772800ac30 con 0x7f77300c5380 2024-03-06T07:21:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.856+0000 7f7717fff640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.857+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f772808a100 con 0x7f77300c5380 2024-03-06T07:21:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.857+0000 7f772f7fe640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.857+0000 7f772f7fe640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 0x7f771c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7720036000 tx=0x7f7720008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:06.861+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f77300b0450 con 0x7f77300c5380 2024-03-06T07:21:07.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:07.081+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f773004f330 con 0x7f77300c5380 2024-03-06T07:21:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.077+0000 7f7717fff640 1 -- 172.21.15.17:0/3480432052 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7728058830 con 0x7f77300c5380 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 msgr2=0x7f771c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 0x7f771c06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7720036000 tx=0x7f7720008030 comp rx=0 tx=0).stop 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 msgr2=0x7f7730157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f772800c7a0 tx=0x7f772800cc60 comp rx=0 tx=0).stop 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 shutdown_connections 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f771c06b4d0 0x7f771c06d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77300c5380 0x7f7730157af0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77300b0070 0x7f77300c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 --2- 172.21.15.17:0/3480432052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f773006ca70 0x7f7730154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 >> 172.21.15.17:0/3480432052 conn(0x7f77300ac590 msgr2=0x7f77300ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 shutdown_connections 2024-03-06T07:21:08.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.080+0000 7f7735e15640 1 -- 172.21.15.17:0/3480432052 wait complete. 2024-03-06T07:21:08.095 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:21:08.096 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:21:08.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.205+0000 7ff4cefa0640 1 Processor -- start 2024-03-06T07:21:08.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.205+0000 7ff4cefa0640 1 -- start start 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cefa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cefa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4c80c8d50 0x7ff4c806b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cefa0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c81552c0 con 0x7ff4c806afc0 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cefa0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c80c6d30 con 0x7ff4c80c8d50 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cefa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8155580 con 0x7ff4c8152a20 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4c80c8d50 0x7ff4c806b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4c80c8d50 0x7ff4c806b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35630/0 (socket says 172.21.15.17:35630) 2024-03-06T07:21:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 -- 172.21.15.17:0/1265657150 learned_addr learned my addr 172.21.15.17:0/1265657150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4cdf9e640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 -- 172.21.15.17:0/1265657150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 msgr2=0x7ff4c81592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c81592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 -- 172.21.15.17:0/1265657150 >> v1:172.21.15.163:6789/0 conn(0x7ff4c806afc0 legacy=0x7ff4c80c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ccf9c640 1 -- 172.21.15.17:0/1265657150 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3736540523 0 0) 0x7ff4c81552c0 con 0x7ff4c806afc0 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.206+0000 7ff4ce79f640 1 -- 172.21.15.17:0/1265657150 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8155860 con 0x7ff4c80c8d50 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4ce79f640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c80c8d50 0x7ff4c806b420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009e50 tx=0x7ff4c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d3e7c4bede39c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4ccf9c640 1 -- 172.21.15.17:0/1265657150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c002c9e0 con 0x7ff4c80c8d50 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4ccf9c640 1 -- 172.21.15.17:0/1265657150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c002cb80 con 0x7ff4c80c8d50 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4ccf9c640 1 -- 172.21.15.17:0/1265657150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00343f0 con 0x7ff4c80c8d50 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4cefa0640 1 -- 172.21.15.17:0/1265657150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c80c8d50 msgr2=0x7ff4c806b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.207+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c80c8d50 0x7ff4c806b420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009e50 tx=0x7ff4c002be40 comp rx=0 tx=0).stop 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 -- 172.21.15.17:0/1265657150 shutdown_connections 2024-03-06T07:21:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c80c8d50 0x7ff4c806b420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/1265657150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c81592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 -- 172.21.15.17:0/1265657150 >> 172.21.15.17:0/1265657150 conn(0x7ff4c80ca740 msgr2=0x7ff4c80cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 -- 172.21.15.17:0/1265657150 shutdown_connections 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.208+0000 7ff4cefa0640 1 -- 172.21.15.17:0/1265657150 wait complete. 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 Processor -- start 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 -- start start 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c806afc0 0x7ff4c814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c80c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8156060 con 0x7ff4c814d050 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8152e00 con 0x7ff4c806afc0 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.209+0000 7ff4cefa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8158770 con 0x7ff4c8152a20 2024-03-06T07:21:08.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35640/0 (socket says 172.21.15.17:35640) 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 -- 172.21.15.17:0/979295197 learned_addr learned my addr 172.21.15.17:0/979295197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4cdf9e640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c806afc0 0x7ff4c814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4cd79d640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c80c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 -- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 msgr2=0x7ff4c80c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c80c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 -- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c806afc0 msgr2=0x7ff4c814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c806afc0 0x7ff4c814eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 -- 172.21.15.17:0/979295197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80c20e0 con 0x7ff4c814d050 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4ce79f640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0036040 tx=0x7ff4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0043020 con 0x7ff4c814d050 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c003e450 con 0x7ff4c814d050 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c80c23c0 con 0x7ff4c814d050 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c003e840 con 0x7ff4c814d050 2024-03-06T07:21:08.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.210+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c80c3fc0 con 0x7ff4c814d050 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.211+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c80c28e0 con 0x7ff4c814d050 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.212+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4c0046020 con 0x7ff4c814d050 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.212+0000 7ff4be7fc640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 0x7ff4ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.212+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4c00ac990 con 0x7ff4c814d050 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.212+0000 7ff4cdf9e640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 0x7ff4ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.213+0000 7ff4cdf9e640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 0x7ff4ac06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009aa0 tx=0x7ff4c4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:08.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.216+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4c004d030 con 0x7ff4c814d050 2024-03-06T07:21:08.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:08.280 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:08.435+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff4c80684b0 con 0x7ff4c814d050 2024-03-06T07:21:09.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.087+0000 7ff4be7fc640 1 -- 172.21.15.17:0/979295197 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4c007b0c0 con 0x7ff4c814d050 2024-03-06T07:21:09.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 msgr2=0x7ff4ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 0x7ff4ac06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009aa0 tx=0x7ff4c4009190 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 msgr2=0x7ff4c8148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0036040 tx=0x7ff4c0004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 shutdown_connections 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4ac06b4d0 0x7ff4ac06d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4c814d050 0x7ff4c8148d80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4c8152a20 0x7ff4c80c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 --2- 172.21.15.17:0/979295197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4c806afc0 0x7ff4c814eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 >> 172.21.15.17:0/979295197 conn(0x7ff4c80ca740 msgr2=0x7ff4c80c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 shutdown_connections 2024-03-06T07:21:09.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.090+0000 7ff4cefa0640 1 -- 172.21.15.17:0/979295197 wait complete. 2024-03-06T07:21:09.103 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:21:09.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.206+0000 7fc8dfedf640 1 Processor -- start 2024-03-06T07:21:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.206+0000 7fc8dfedf640 1 -- start start 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8dfedf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8d814f360 0x7fc8d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8dfedf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8d8158b60 0x7fc8d8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8dfedf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d80c73d0 con 0x7fc8d80c88c0 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8dfedf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d80c8dd0 con 0x7fc8d814f360 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8dfedf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d80be2e0 con 0x7fc8d8158b60 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8ddc7b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8d814f360 0x7fc8d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8ddc7b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc8d814f360 0x7fc8d8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35656/0 (socket says 172.21.15.17:35656) 2024-03-06T07:21:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/1636459891 learned_addr learned my addr 172.21.15.17:0/1636459891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.207+0000 7fc8de47c640 1 --2- 172.21.15.17:0/1636459891 >> v2:172.21.15.163:3300/0 conn(0x7fc8d8158b60 0x7fc8d8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/1636459891 >> v2:172.21.15.163:3300/0 conn(0x7fc8d8158b60 msgr2=0x7fc8d8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8ddc7b640 1 --2- 172.21.15.17:0/1636459891 >> v2:172.21.15.163:3300/0 conn(0x7fc8d8158b60 0x7fc8d8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/1636459891 >> v1:172.21.15.17:6789/0 conn(0x7fc8d80c88c0 legacy=0x7fc8d80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/1636459891 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80c7670 con 0x7fc8d814f360 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8ddc7b640 1 --2- 172.21.15.17:0/1636459891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d814f360 0x7fc8d8155c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8152d60 tx=0x7fc8d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4489f6cb692c8494 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.208+0000 7fc8dcc79640 1 -- 172.21.15.17:0/1636459891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4035020 con 0x7fc8d814f360 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dcc79640 1 -- 172.21.15.17:0/1636459891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4031870 con 0x7fc8d814f360 2024-03-06T07:21:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dcc79640 1 -- 172.21.15.17:0/1636459891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031c40 con 0x7fc8d814f360 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 -- 172.21.15.17:0/1636459891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d814f360 msgr2=0x7fc8d8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/1636459891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d814f360 0x7fc8d8155c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8152d60 tx=0x7fc8d402b910 comp rx=0 tx=0).stop 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 -- 172.21.15.17:0/1636459891 shutdown_connections 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/1636459891 >> v2:172.21.15.163:3300/0 conn(0x7fc8d8158b60 0x7fc8d8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/1636459891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d814f360 0x7fc8d8155c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 -- 172.21.15.17:0/1636459891 >> 172.21.15.17:0/1636459891 conn(0x7fc8d80ac570 msgr2=0x7fc8d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 -- 172.21.15.17:0/1636459891 shutdown_connections 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.209+0000 7fc8dfedf640 1 -- 172.21.15.17:0/1636459891 wait complete. 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.210+0000 7fc8dfedf640 1 Processor -- start 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.210+0000 7fc8dfedf640 1 -- start start 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8d814f360 0x7fc8d81ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d8158b60 0x7fc8d81ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c5a50 con 0x7fc8d80c88c0 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c7c30 con 0x7fc8d8158b60 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dfedf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c5bd0 con 0x7fc8d814f360 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35664/0 (socket says 172.21.15.17:35664) 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/3478557247 learned_addr learned my addr 172.21.15.17:0/3478557247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8de47c640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d8158b60 0x7fc8d81ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8dd47a640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8d814f360 0x7fc8d81ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8d814f360 msgr2=0x7fc8d81ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8d814f360 0x7fc8d81ef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d8158b60 msgr2=0x7fc8d81ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d8158b60 0x7fc8d81ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.211+0000 7fc8ddc7b640 1 -- 172.21.15.17:0/3478557247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8158370 con 0x7fc8d80c88c0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8ddc7b640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4035040 con 0x7fc8d80c88c0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4030d00 con 0x7fc8d80c88c0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d8157ad0 con 0x7fc8d80c88c0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40456a0 con 0x7fc8d80c88c0 2024-03-06T07:21:09.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.212+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d8158090 con 0x7fc8d80c88c0 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.213+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d804f330 con 0x7fc8d80c88c0 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.213+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8d4038020 con 0x7fc8d80c88c0 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.213+0000 7fc8ce7fc640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 0x7fc8bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.214+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8d40ac560 con 0x7fc8d80c88c0 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.214+0000 7fc8dd47a640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 0x7fc8bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.214+0000 7fc8dd47a640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 0x7fc8bc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8d81efa20 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.218+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc8d407ac90 con 0x7fc8d80c88c0 2024-03-06T07:21:09.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.437+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8d80c1640 con 0x7fc8d80c88c0 2024-03-06T07:21:09.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.438+0000 7fc8ce7fc640 1 -- 172.21.15.17:0/3478557247 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fc8d407ed30 con 0x7fc8d80c88c0 2024-03-06T07:21:09.439 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:09.440 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:08.436696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2927346551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2927346551},{"type":"v1","addr":"172.21.15.163:6837","nonce":2927346551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5289,"qdb_cluster":[5289]},"id":2}]} 2024-03-06T07:21:09.440 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-06T07:21:09.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 msgr2=0x7fc8bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 0x7fc8bc06d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc8d81efa20 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2024-03-06T07:21:09.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 msgr2=0x7fc8d81e0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4000c00 tx=0x7fc8d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 shutdown_connections 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc8bc06b4d0 0x7fc8bc06d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8d8158b60 0x7fc8d81ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8d814f360 0x7fc8d81ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 --2- 172.21.15.17:0/3478557247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8d80c88c0 0x7fc8d81e0530 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.439+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 >> 172.21.15.17:0/3478557247 conn(0x7fc8d80ac570 msgr2=0x7fc8d80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.440+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 shutdown_connections 2024-03-06T07:21:09.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.440+0000 7fc8dfedf640 1 -- 172.21.15.17:0/3478557247 wait complete. 2024-03-06T07:21:09.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.551+0000 7f432f61c640 1 Processor -- start 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.551+0000 7f432f61c640 1 -- start start 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432f61c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 0x7f4328158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432f61c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280c7760 con 0x7f4328152920 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432f61c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280c7a40 con 0x7f4328159460 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432f61c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43281587b0 con 0x7f43280c8a00 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432d3b8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4328152920 0x7f4328155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46494/0 (socket says 172.21.15.17:46494) 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432d3b8640 1 -- 172.21.15.17:0/593177874 learned_addr learned my addr 172.21.15.17:0/593177874 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f432cbb7640 1 --2- 172.21.15.17:0/593177874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 0x7f4328158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044856898 0 0) 0x7f43280c7760 con 0x7f4328152920 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003520 con 0x7f4328152920 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.552+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571535833 0 0) 0x7f43280c7a40 con 0x7f4328159460 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.553+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43280c7760 con 0x7f4328159460 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.553+0000 7f432cbb7640 1 -- 172.21.15.17:0/593177874 >> v1:172.21.15.163:6789/0 conn(0x7f4328159460 legacy=0x7f43280c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.553+0000 7f432cbb7640 1 -- 172.21.15.17:0/593177874 >> v1:172.21.15.17:6789/0 conn(0x7f4328152920 legacy=0x7f4328155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.553+0000 7f432cbb7640 1 -- 172.21.15.17:0/593177874 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280c7d20 con 0x7f43280c8a00 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.553+0000 7f432cbb7640 1 --2- 172.21.15.17:0/593177874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 0x7f4328158f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f431c0097d0 tx=0x7f431c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150170174ec6c47e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c035020 con 0x7f43280c8a00 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c031800 con 0x7f43280c8a00 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f4327fff640 1 -- 172.21.15.17:0/593177874 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c031bb0 con 0x7f43280c8a00 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 -- 172.21.15.17:0/593177874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 msgr2=0x7f4328158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 --2- 172.21.15.17:0/593177874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 0x7f4328158f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f431c0097d0 tx=0x7f431c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 -- 172.21.15.17:0/593177874 shutdown_connections 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 --2- 172.21.15.17:0/593177874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c8a00 0x7f4328158f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 -- 172.21.15.17:0/593177874 >> 172.21.15.17:0/593177874 conn(0x7f43280ac560 msgr2=0x7f43280ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 -- 172.21.15.17:0/593177874 shutdown_connections 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.554+0000 7f432f61c640 1 -- 172.21.15.17:0/593177874 wait complete. 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.555+0000 7f432f61c640 1 Processor -- start 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.555+0000 7f432f61c640 1 -- start start 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43280c8a00 0x7f432814b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c4740 0x7f43280c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43280c5a40 con 0x7f4328152920 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432806d030 con 0x7f43280c8a00 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432f61c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280c5bc0 con 0x7f43280c4740 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35672/0 (socket says 172.21.15.17:35672) 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 -- 172.21.15.17:0/100247981 learned_addr learned my addr 172.21.15.17:0/100247981 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432d3b8640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43280c8a00 0x7f432814b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432dbb9640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c4740 0x7f43280c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 -- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c4740 msgr2=0x7f43280c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c4740 0x7f43280c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 -- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43280c8a00 msgr2=0x7f432814b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43280c8a00 0x7f432814b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.556+0000 7f432cbb7640 1 -- 172.21.15.17:0/100247981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280c3f50 con 0x7f4328152920 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f432cbb7640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f431c000c00 tx=0x7f431c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c041070 con 0x7f4328152920 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c031e30 con 0x7f4328152920 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4328155ee0 con 0x7f4328152920 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c035040 con 0x7f4328152920 2024-03-06T07:21:09.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.557+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280c3750 con 0x7f4328152920 2024-03-06T07:21:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.558+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432804f330 con 0x7f4328152920 2024-03-06T07:21:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.558+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f431c04c060 con 0x7f4328152920 2024-03-06T07:21:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.559+0000 7f4325ffb640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 0x7f431406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.559+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f431c0abbf0 con 0x7f4328152920 2024-03-06T07:21:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.559+0000 7f432d3b8640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 0x7f431406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.559+0000 7f432d3b8640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 0x7f431406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4320000f60 tx=0x7f4320007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.563+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f431c07a320 con 0x7f4328152920 2024-03-06T07:21:09.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.745+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f432806cda0 con 0x7f4328152920 2024-03-06T07:21:09.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.746+0000 7f4325ffb640 1 -- 172.21.15.17:0/100247981 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f431c07e3c0 con 0x7f4328152920 2024-03-06T07:21:09.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:09.748 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":39,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:07.075762+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 msgr2=0x7f431406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 0x7f431406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4320000f60 tx=0x7f4320007d70 comp rx=0 tx=0).stop 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 msgr2=0x7f4328157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f431c000c00 tx=0x7f431c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 shutdown_connections 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f431406b4d0 0x7f431406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43280c4740 0x7f43280c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4328152920 0x7f4328157c80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 --2- 172.21.15.17:0/100247981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43280c8a00 0x7f432814b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 >> 172.21.15.17:0/100247981 conn(0x7f43280ac560 msgr2=0x7f4328151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 shutdown_connections 2024-03-06T07:21:09.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.748+0000 7f432f61c640 1 -- 172.21.15.17:0/100247981 wait complete. 2024-03-06T07:21:09.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:21:09.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:09.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.861+0000 7ff810015640 1 Processor -- start 2024-03-06T07:21:09.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.861+0000 7ff810015640 1 -- start start 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff810015640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 0x7ff808155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff810015640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff808158b60 0x7ff808158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff810015640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8080c73d0 con 0x7ff80814f360 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff810015640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8080c8dd0 con 0x7ff808158b60 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff810015640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8080be2e0 con 0x7ff8080c88c0 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff808158b60 0x7ff808158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff808158b60 0x7ff808158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35686/0 (socket says 172.21.15.17:35686) 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 -- 172.21.15.17:0/2049137973 learned_addr learned my addr 172.21.15.17:0/2049137973 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80d5b0640 1 --2- 172.21.15.17:0/2049137973 >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 0x7ff808155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80cdaf640 1 -- 172.21.15.17:0/2049137973 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497806837 0 0) 0x7ff8080c73d0 con 0x7ff80814f360 2024-03-06T07:21:09.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 -- 172.21.15.17:0/2049137973 >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 msgr2=0x7ff808155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 --2- 172.21.15.17:0/2049137973 >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 0x7ff808155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 -- 172.21.15.17:0/2049137973 >> v1:172.21.15.163:6789/0 conn(0x7ff80814f360 legacy=0x7ff8080c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80e5b2640 1 -- 172.21.15.17:0/2049137973 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080c7670 con 0x7ff808158b60 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.862+0000 7ff80d5b0640 1 --2- 172.21.15.17:0/2049137973 >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 0x7ff808155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.863+0000 7ff80e5b2640 1 --2- 172.21.15.17:0/2049137973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff808158b60 0x7ff808158ff0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff8080c84c0 tx=0x7ff80002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb4bd72df33fdaf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.863+0000 7ff80cdaf640 1 -- 172.21.15.17:0/2049137973 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80002ca80 con 0x7ff808158b60 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.863+0000 7ff80cdaf640 1 -- 172.21.15.17:0/2049137973 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80002cc20 con 0x7ff808158b60 2024-03-06T07:21:09.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.863+0000 7ff80cdaf640 1 -- 172.21.15.17:0/2049137973 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000343f0 con 0x7ff808158b60 2024-03-06T07:21:09.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 -- 172.21.15.17:0/2049137973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff808158b60 msgr2=0x7ff808158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 --2- 172.21.15.17:0/2049137973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff808158b60 0x7ff808158ff0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff8080c84c0 tx=0x7ff80002bee0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 -- 172.21.15.17:0/2049137973 shutdown_connections 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 --2- 172.21.15.17:0/2049137973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff808158b60 0x7ff808158ff0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 --2- 172.21.15.17:0/2049137973 >> v2:172.21.15.163:3300/0 conn(0x7ff8080c88c0 0x7ff808155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 -- 172.21.15.17:0/2049137973 >> 172.21.15.17:0/2049137973 conn(0x7ff8080ac570 msgr2=0x7ff8080ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 -- 172.21.15.17:0/2049137973 shutdown_connections 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.864+0000 7ff810015640 1 -- 172.21.15.17:0/2049137973 wait complete. 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 Processor -- start 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 -- start start 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8080c88c0 0x7ff808151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff808158b60 0x7ff808150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080c5a50 con 0x7ff80814f360 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080c7c30 con 0x7ff808158b60 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.865+0000 7ff810015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8080c5bd0 con 0x7ff8080c88c0 2024-03-06T07:21:09.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35694/0 (socket says 172.21.15.17:35694) 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 -- 172.21.15.17:0/1716981161 learned_addr learned my addr 172.21.15.17:0/1716981161 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80ddb1640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8080c88c0 0x7ff808151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80e5b2640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff808158b60 0x7ff808150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 -- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8080c88c0 msgr2=0x7ff808151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8080c88c0 0x7ff808151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 -- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff808158b60 msgr2=0x7ff808150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff808158b60 0x7ff808150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 -- 172.21.15.17:0/1716981161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80814b940 con 0x7ff80814f360 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff80d5b0640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e900 tx=0x7ff7f800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8017070 con 0x7ff80814f360 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f800ca00 con 0x7ff80814f360 2024-03-06T07:21:09.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8081eb8a0 con 0x7ff80814f360 2024-03-06T07:21:09.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800cdf0 con 0x7ff80814f360 2024-03-06T07:21:09.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.866+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80814bc20 con 0x7ff80814f360 2024-03-06T07:21:09.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.867+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff808154fd0 con 0x7ff80814f360 2024-03-06T07:21:09.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.868+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7f801a070 con 0x7ff80814f360 2024-03-06T07:21:09.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.868+0000 7ff7fe7fc640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 0x7ff7ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:09.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.868+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7f808a210 con 0x7ff80814f360 2024-03-06T07:21:09.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.868+0000 7ff80ddb1640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 0x7ff7ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:09.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.869+0000 7ff80ddb1640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 0x7ff7ec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff804000f60 tx=0x7ff804009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:09.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:09.872+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff808154fd0 con 0x7ff80814f360 2024-03-06T07:21:10.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.048+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8080c1410 con 0x7ff7ec06b4d0 2024-03-06T07:21:10.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.054+0000 7ff7fe7fc640 1 -- 172.21.15.17:0/1716981161 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252003 (secure 0 0 0) 0x7ff804001420 con 0x7ff7ec06b4d0 2024-03-06T07:21:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 msgr2=0x7ff7ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 0x7ff7ec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff804000f60 tx=0x7ff804009040 comp rx=0 tx=0).stop 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 msgr2=0x7ff80814fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800e900 tx=0x7ff7f800edc0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 shutdown_connections 2024-03-06T07:21:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7ec06b4d0 0x7ff7ec06d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff808158b60 0x7ff808150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.057+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff80814f360 0x7ff80814fbd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.058+0000 7ff810015640 1 --2- 172.21.15.17:0/1716981161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8080c88c0 0x7ff808151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.058+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 >> 172.21.15.17:0/1716981161 conn(0x7ff8080ac570 msgr2=0x7ff8080c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.058+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 shutdown_connections 2024-03-06T07:21:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.058+0000 7ff810015640 1 -- 172.21.15.17:0/1716981161 wait complete. 2024-03-06T07:21:10.070 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-06T07:21:09.354633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236480,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738240,"statfs":{"total":773094113280,"available":772851957760,"internally_reserved":0,"allocated":3727360,"data_stored":2726676,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.284080"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078763+0000","last_change":"2024-03-06T07:21:06.099356+0000","last_active":"2024-03-06T07:21:07.078763+0000","last_peered":"2024-03-06T07:21:07.078763+0000","last_clean":"2024-03-06T07:21:07.078763+0000","last_became_active":"2024-03-06T07:21:06.096785+0000","last_became_peered":"2024-03-06T07:21:06.096785+0000","last_unstale":"2024-03-06T07:21:07.078763+0000","last_undegraded":"2024-03-06T07:21:07.078763+0000","last_fullsized":"2024-03-06T07:21:07.078763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:18.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078434+0000","last_change":"2024-03-06T07:21:06.089908+0000","last_active":"2024-03-06T07:21:07.078434+0000","last_peered":"2024-03-06T07:21:07.078434+0000","last_clean":"2024-03-06T07:21:07.078434+0000","last_became_active":"2024-03-06T07:21:06.089630+0000","last_became_peered":"2024-03-06T07:21:06.089630+0000","last_unstale":"2024-03-06T07:21:07.078434+0000","last_undegraded":"2024-03-06T07:21:07.078434+0000","last_fullsized":"2024-03-06T07:21:07.078434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:04.201769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.108109+0000","last_change":"2024-03-06T07:21:06.107183+0000","last_active":"2024-03-06T07:21:06.108109+0000","last_peered":"2024-03-06T07:21:06.108109+0000","last_clean":"2024-03-06T07:21:06.108109+0000","last_became_active":"2024-03-06T07:21:06.104951+0000","last_became_peered":"2024-03-06T07:21:06.104951+0000","last_unstale":"2024-03-06T07:21:06.108109+0000","last_undegraded":"2024-03-06T07:21:06.108109+0000","last_fullsized":"2024-03-06T07:21:06.108109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096676+0000","last_change":"2024-03-06T07:21:06.093481+0000","last_active":"2024-03-06T07:21:07.096676+0000","last_peered":"2024-03-06T07:21:07.096676+0000","last_clean":"2024-03-06T07:21:07.096676+0000","last_became_active":"2024-03-06T07:21:06.091728+0000","last_became_peered":"2024-03-06T07:21:06.091728+0000","last_unstale":"2024-03-06T07:21:07.096676+0000","last_undegraded":"2024-03-06T07:21:07.096676+0000","last_fullsized":"2024-03-06T07:21:07.096676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:55.490456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096499+0000","last_change":"2024-03-06T07:21:06.083513+0000","last_active":"2024-03-06T07:21:07.096499+0000","last_peered":"2024-03-06T07:21:07.096499+0000","last_clean":"2024-03-06T07:21:07.096499+0000","last_became_active":"2024-03-06T07:21:06.083154+0000","last_became_peered":"2024-03-06T07:21:06.083154+0000","last_unstale":"2024-03-06T07:21:07.096499+0000","last_undegraded":"2024-03-06T07:21:07.096499+0000","last_fullsized":"2024-03-06T07:21:07.096499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:15.049203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.093568+0000","last_change":"2024-03-06T07:21:06.090047+0000","last_active":"2024-03-06T07:21:06.093568+0000","last_peered":"2024-03-06T07:21:06.093568+0000","last_clean":"2024-03-06T07:21:06.093568+0000","last_became_active":"2024-03-06T07:21:06.089757+0000","last_became_peered":"2024-03-06T07:21:06.089757+0000","last_unstale":"2024-03-06T07:21:06.093568+0000","last_undegraded":"2024-03-06T07:21:06.093568+0000","last_fullsized":"2024-03-06T07:21:06.093568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098225+0000","last_change":"2024-03-06T07:21:06.105353+0000","last_active":"2024-03-06T07:21:07.098225+0000","last_peered":"2024-03-06T07:21:07.098225+0000","last_clean":"2024-03-06T07:21:07.098225+0000","last_became_active":"2024-03-06T07:21:06.105103+0000","last_became_peered":"2024-03-06T07:21:06.105103+0000","last_unstale":"2024-03-06T07:21:07.098225+0000","last_undegraded":"2024-03-06T07:21:07.098225+0000","last_fullsized":"2024-03-06T07:21:07.098225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:44.500260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078200+0000","last_change":"2024-03-06T07:21:06.106008+0000","last_active":"2024-03-06T07:21:07.078200+0000","last_peered":"2024-03-06T07:21:07.078200+0000","last_clean":"2024-03-06T07:21:07.078200+0000","last_became_active":"2024-03-06T07:21:06.105715+0000","last_became_peered":"2024-03-06T07:21:06.105715+0000","last_unstale":"2024-03-06T07:21:07.078200+0000","last_undegraded":"2024-03-06T07:21:07.078200+0000","last_fullsized":"2024-03-06T07:21:07.078200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:08.094311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107794+0000","last_change":"2024-03-06T07:21:06.104535+0000","last_active":"2024-03-06T07:21:06.107794+0000","last_peered":"2024-03-06T07:21:06.107794+0000","last_clean":"2024-03-06T07:21:06.107794+0000","last_became_active":"2024-03-06T07:21:06.104252+0000","last_became_peered":"2024-03-06T07:21:06.104252+0000","last_unstale":"2024-03-06T07:21:06.107794+0000","last_undegraded":"2024-03-06T07:21:06.107794+0000","last_fullsized":"2024-03-06T07:21:06.107794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:35.566633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098551+0000","last_change":"2024-03-06T07:21:06.103954+0000","last_active":"2024-03-06T07:21:07.098551+0000","last_peered":"2024-03-06T07:21:07.098551+0000","last_clean":"2024-03-06T07:21:07.098551+0000","last_became_active":"2024-03-06T07:21:06.102851+0000","last_became_peered":"2024-03-06T07:21:06.102851+0000","last_unstale":"2024-03-06T07:21:07.098551+0000","last_undegraded":"2024-03-06T07:21:07.098551+0000","last_fullsized":"2024-03-06T07:21:07.098551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.909624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098608+0000","last_change":"2024-03-06T07:21:06.106113+0000","last_active":"2024-03-06T07:21:07.098608+0000","last_peered":"2024-03-06T07:21:07.098608+0000","last_clean":"2024-03-06T07:21:07.098608+0000","last_became_active":"2024-03-06T07:21:06.105811+0000","last_became_peered":"2024-03-06T07:21:06.105811+0000","last_unstale":"2024-03-06T07:21:07.098608+0000","last_undegraded":"2024-03-06T07:21:07.098608+0000","last_fullsized":"2024-03-06T07:21:07.098608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:04.201769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.103786+0000","last_change":"2024-03-06T07:21:06.101617+0000","last_active":"2024-03-06T07:21:06.103786+0000","last_peered":"2024-03-06T07:21:06.103786+0000","last_clean":"2024-03-06T07:21:06.103786+0000","last_became_active":"2024-03-06T07:21:06.101315+0000","last_became_peered":"2024-03-06T07:21:06.101315+0000","last_unstale":"2024-03-06T07:21:06.103786+0000","last_undegraded":"2024-03-06T07:21:06.103786+0000","last_fullsized":"2024-03-06T07:21:06.103786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098288+0000","last_change":"2024-03-06T07:21:06.101712+0000","last_active":"2024-03-06T07:21:07.098288+0000","last_peered":"2024-03-06T07:21:07.098288+0000","last_clean":"2024-03-06T07:21:07.098288+0000","last_became_active":"2024-03-06T07:21:06.101439+0000","last_became_peered":"2024-03-06T07:21:06.101439+0000","last_unstale":"2024-03-06T07:21:07.098288+0000","last_undegraded":"2024-03-06T07:21:07.098288+0000","last_fullsized":"2024-03-06T07:21:07.098288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:19.714799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078859+0000","last_change":"2024-03-06T07:21:06.092741+0000","last_active":"2024-03-06T07:21:07.078859+0000","last_peered":"2024-03-06T07:21:07.078859+0000","last_clean":"2024-03-06T07:21:07.078859+0000","last_became_active":"2024-03-06T07:21:06.092475+0000","last_became_peered":"2024-03-06T07:21:06.092475+0000","last_unstale":"2024-03-06T07:21:07.078859+0000","last_undegraded":"2024-03-06T07:21:07.078859+0000","last_fullsized":"2024-03-06T07:21:07.078859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098249+0000","last_change":"2024-03-06T07:21:06.103816+0000","last_active":"2024-03-06T07:21:07.098249+0000","last_peered":"2024-03-06T07:21:07.098249+0000","last_clean":"2024-03-06T07:21:07.098249+0000","last_became_active":"2024-03-06T07:21:06.103310+0000","last_became_peered":"2024-03-06T07:21:06.103310+0000","last_unstale":"2024-03-06T07:21:07.098249+0000","last_undegraded":"2024-03-06T07:21:07.098249+0000","last_fullsized":"2024-03-06T07:21:07.098249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:50.247773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.110323+0000","last_change":"2024-03-06T07:21:06.105144+0000","last_active":"2024-03-06T07:21:06.110323+0000","last_peered":"2024-03-06T07:21:06.110323+0000","last_clean":"2024-03-06T07:21:06.110323+0000","last_became_active":"2024-03-06T07:21:06.104893+0000","last_became_peered":"2024-03-06T07:21:06.104893+0000","last_unstale":"2024-03-06T07:21:06.110323+0000","last_undegraded":"2024-03-06T07:21:06.110323+0000","last_fullsized":"2024-03-06T07:21:06.110323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:52.176507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104848+0000","last_change":"2024-03-06T07:21:06.100513+0000","last_active":"2024-03-06T07:21:06.104848+0000","last_peered":"2024-03-06T07:21:06.104848+0000","last_clean":"2024-03-06T07:21:06.104848+0000","last_became_active":"2024-03-06T07:21:06.100255+0000","last_became_peered":"2024-03-06T07:21:06.100255+0000","last_unstale":"2024-03-06T07:21:06.104848+0000","last_undegraded":"2024-03-06T07:21:06.104848+0000","last_fullsized":"2024-03-06T07:21:06.104848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:35.566633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079311+0000","last_change":"2024-03-06T07:21:06.095506+0000","last_active":"2024-03-06T07:21:07.079311+0000","last_peered":"2024-03-06T07:21:07.079311+0000","last_clean":"2024-03-06T07:21:07.079311+0000","last_became_active":"2024-03-06T07:21:06.095254+0000","last_became_peered":"2024-03-06T07:21:06.095254+0000","last_unstale":"2024-03-06T07:21:07.079311+0000","last_undegraded":"2024-03-06T07:21:07.079311+0000","last_fullsized":"2024-03-06T07:21:07.079311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096684+0000","last_change":"2024-03-06T07:21:06.087805+0000","last_active":"2024-03-06T07:21:07.096684+0000","last_peered":"2024-03-06T07:21:07.096684+0000","last_clean":"2024-03-06T07:21:07.096684+0000","last_became_active":"2024-03-06T07:21:06.087524+0000","last_became_peered":"2024-03-06T07:21:06.087524+0000","last_unstale":"2024-03-06T07:21:07.096684+0000","last_undegraded":"2024-03-06T07:21:07.096684+0000","last_fullsized":"2024-03-06T07:21:07.096684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:33.218026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078751+0000","last_change":"2024-03-06T07:21:06.093728+0000","last_active":"2024-03-06T07:21:07.078751+0000","last_peered":"2024-03-06T07:21:07.078751+0000","last_clean":"2024-03-06T07:21:07.078751+0000","last_became_active":"2024-03-06T07:21:06.093470+0000","last_became_peered":"2024-03-06T07:21:06.093470+0000","last_unstale":"2024-03-06T07:21:07.078751+0000","last_undegraded":"2024-03-06T07:21:07.078751+0000","last_fullsized":"2024-03-06T07:21:07.078751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098323+0000","last_change":"2024-03-06T07:21:06.098747+0000","last_active":"2024-03-06T07:21:07.098323+0000","last_peered":"2024-03-06T07:21:07.098323+0000","last_clean":"2024-03-06T07:21:07.098323+0000","last_became_active":"2024-03-06T07:21:06.098308+0000","last_became_peered":"2024-03-06T07:21:06.098308+0000","last_unstale":"2024-03-06T07:21:07.098323+0000","last_undegraded":"2024-03-06T07:21:07.098323+0000","last_fullsized":"2024-03-06T07:21:07.098323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:08.094311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079164+0000","last_change":"2024-03-06T07:21:06.090214+0000","last_active":"2024-03-06T07:21:07.079164+0000","last_peered":"2024-03-06T07:21:07.079164+0000","last_clean":"2024-03-06T07:21:07.079164+0000","last_became_active":"2024-03-06T07:21:06.089958+0000","last_became_peered":"2024-03-06T07:21:06.089958+0000","last_unstale":"2024-03-06T07:21:07.079164+0000","last_undegraded":"2024-03-06T07:21:07.079164+0000","last_fullsized":"2024-03-06T07:21:07.079164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:19.048287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097197+0000","last_change":"2024-03-06T07:21:06.092351+0000","last_active":"2024-03-06T07:21:06.097197+0000","last_peered":"2024-03-06T07:21:06.097197+0000","last_clean":"2024-03-06T07:21:06.097197+0000","last_became_active":"2024-03-06T07:21:06.092094+0000","last_became_peered":"2024-03-06T07:21:06.092094+0000","last_unstale":"2024-03-06T07:21:06.097197+0000","last_undegraded":"2024-03-06T07:21:06.097197+0000","last_fullsized":"2024-03-06T07:21:06.097197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096411+0000","last_change":"2024-03-06T07:21:06.088643+0000","last_active":"2024-03-06T07:21:07.096411+0000","last_peered":"2024-03-06T07:21:07.096411+0000","last_clean":"2024-03-06T07:21:07.096411+0000","last_became_active":"2024-03-06T07:21:06.088294+0000","last_became_peered":"2024-03-06T07:21:06.088294+0000","last_unstale":"2024-03-06T07:21:07.096411+0000","last_undegraded":"2024-03-06T07:21:07.096411+0000","last_fullsized":"2024-03-06T07:21:07.096411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:22.915440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079517+0000","last_change":"2024-03-06T07:21:06.093069+0000","last_active":"2024-03-06T07:21:07.079517+0000","last_peered":"2024-03-06T07:21:07.079517+0000","last_clean":"2024-03-06T07:21:07.079517+0000","last_became_active":"2024-03-06T07:21:06.092789+0000","last_became_peered":"2024-03-06T07:21:06.092789+0000","last_unstale":"2024-03-06T07:21:07.079517+0000","last_undegraded":"2024-03-06T07:21:07.079517+0000","last_fullsized":"2024-03-06T07:21:07.079517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104484+0000","last_change":"2024-03-06T07:21:06.102648+0000","last_active":"2024-03-06T07:21:06.104484+0000","last_peered":"2024-03-06T07:21:06.104484+0000","last_clean":"2024-03-06T07:21:06.104484+0000","last_became_active":"2024-03-06T07:21:06.102325+0000","last_became_peered":"2024-03-06T07:21:06.102325+0000","last_unstale":"2024-03-06T07:21:06.104484+0000","last_undegraded":"2024-03-06T07:21:06.104484+0000","last_fullsized":"2024-03-06T07:21:06.104484+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079027+0000","last_change":"2024-03-06T07:21:06.085000+0000","last_active":"2024-03-06T07:21:07.079027+0000","last_peered":"2024-03-06T07:21:07.079027+0000","last_clean":"2024-03-06T07:21:07.079027+0000","last_became_active":"2024-03-06T07:21:06.084697+0000","last_became_peered":"2024-03-06T07:21:06.084697+0000","last_unstale":"2024-03-06T07:21:07.079027+0000","last_undegraded":"2024-03-06T07:21:07.079027+0000","last_fullsized":"2024-03-06T07:21:07.079027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098593+0000","last_change":"2024-03-06T07:21:06.097795+0000","last_active":"2024-03-06T07:21:07.098593+0000","last_peered":"2024-03-06T07:21:07.098593+0000","last_clean":"2024-03-06T07:21:07.098593+0000","last_became_active":"2024-03-06T07:21:06.097486+0000","last_became_peered":"2024-03-06T07:21:06.097486+0000","last_unstale":"2024-03-06T07:21:07.098593+0000","last_undegraded":"2024-03-06T07:21:07.098593+0000","last_fullsized":"2024-03-06T07:21:07.098593+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:36.243435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078789+0000","last_change":"2024-03-06T07:21:06.086882+0000","last_active":"2024-03-06T07:21:07.078789+0000","last_peered":"2024-03-06T07:21:07.078789+0000","last_clean":"2024-03-06T07:21:07.078789+0000","last_became_active":"2024-03-06T07:21:06.086579+0000","last_became_peered":"2024-03-06T07:21:06.086579+0000","last_unstale":"2024-03-06T07:21:07.078789+0000","last_undegraded":"2024-03-06T07:21:07.078789+0000","last_fullsized":"2024-03-06T07:21:07.078789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079067+0000","last_change":"2024-03-06T07:21:06.087811+0000","last_active":"2024-03-06T07:21:07.079067+0000","last_peered":"2024-03-06T07:21:07.079067+0000","last_clean":"2024-03-06T07:21:07.079067+0000","last_became_active":"2024-03-06T07:21:06.087302+0000","last_became_peered":"2024-03-06T07:21:06.087302+0000","last_unstale":"2024-03-06T07:21:07.079067+0000","last_undegraded":"2024-03-06T07:21:07.079067+0000","last_fullsized":"2024-03-06T07:21:07.079067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.099001+0000","last_change":"2024-03-06T07:21:06.096001+0000","last_active":"2024-03-06T07:21:06.099001+0000","last_peered":"2024-03-06T07:21:06.099001+0000","last_clean":"2024-03-06T07:21:06.099001+0000","last_became_active":"2024-03-06T07:21:06.095734+0000","last_became_peered":"2024-03-06T07:21:06.095734+0000","last_unstale":"2024-03-06T07:21:06.099001+0000","last_undegraded":"2024-03-06T07:21:06.099001+0000","last_fullsized":"2024-03-06T07:21:06.099001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097220+0000","last_change":"2024-03-06T07:21:06.092028+0000","last_active":"2024-03-06T07:21:06.097220+0000","last_peered":"2024-03-06T07:21:06.097220+0000","last_clean":"2024-03-06T07:21:06.097220+0000","last_became_active":"2024-03-06T07:21:06.091600+0000","last_became_peered":"2024-03-06T07:21:06.091600+0000","last_unstale":"2024-03-06T07:21:06.097220+0000","last_undegraded":"2024-03-06T07:21:06.097220+0000","last_fullsized":"2024-03-06T07:21:06.097220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.105291+0000","last_change":"2024-03-06T07:21:06.100443+0000","last_active":"2024-03-06T07:21:06.105291+0000","last_peered":"2024-03-06T07:21:06.105291+0000","last_clean":"2024-03-06T07:21:06.105291+0000","last_became_active":"2024-03-06T07:21:06.100173+0000","last_became_peered":"2024-03-06T07:21:06.100173+0000","last_unstale":"2024-03-06T07:21:06.105291+0000","last_undegraded":"2024-03-06T07:21:06.105291+0000","last_fullsized":"2024-03-06T07:21:06.105291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079510+0000","last_change":"2024-03-06T07:21:05.091543+0000","last_active":"2024-03-06T07:21:07.079510+0000","last_peered":"2024-03-06T07:21:07.079510+0000","last_clean":"2024-03-06T07:21:07.079510+0000","last_became_active":"2024-03-06T07:21:05.091244+0000","last_became_peered":"2024-03-06T07:21:05.091244+0000","last_unstale":"2024-03-06T07:21:07.079510+0000","last_undegraded":"2024-03-06T07:21:07.079510+0000","last_fullsized":"2024-03-06T07:21:07.079510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:54.485855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.106492+0000","last_change":"2024-03-06T07:21:06.104326+0000","last_active":"2024-03-06T07:21:06.106492+0000","last_peered":"2024-03-06T07:21:06.106492+0000","last_clean":"2024-03-06T07:21:06.106492+0000","last_became_active":"2024-03-06T07:21:06.104060+0000","last_became_peered":"2024-03-06T07:21:06.104060+0000","last_unstale":"2024-03-06T07:21:06.106492+0000","last_undegraded":"2024-03-06T07:21:06.106492+0000","last_fullsized":"2024-03-06T07:21:06.106492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107490+0000","last_change":"2024-03-06T07:21:05.097659+0000","last_active":"2024-03-06T07:21:06.107490+0000","last_peered":"2024-03-06T07:21:06.107490+0000","last_clean":"2024-03-06T07:21:06.107490+0000","last_became_active":"2024-03-06T07:21:05.097315+0000","last_became_peered":"2024-03-06T07:21:05.097315+0000","last_unstale":"2024-03-06T07:21:06.107490+0000","last_undegraded":"2024-03-06T07:21:06.107490+0000","last_fullsized":"2024-03-06T07:21:06.107490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079138+0000","last_change":"2024-03-06T07:21:06.084799+0000","last_active":"2024-03-06T07:21:07.079138+0000","last_peered":"2024-03-06T07:21:07.079138+0000","last_clean":"2024-03-06T07:21:07.079138+0000","last_became_active":"2024-03-06T07:21:06.084507+0000","last_became_peered":"2024-03-06T07:21:06.084507+0000","last_unstale":"2024-03-06T07:21:07.079138+0000","last_undegraded":"2024-03-06T07:21:07.079138+0000","last_fullsized":"2024-03-06T07:21:07.079138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:11.227745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.106967+0000","last_change":"2024-03-06T07:21:05.093387+0000","last_active":"2024-03-06T07:21:06.106967+0000","last_peered":"2024-03-06T07:21:06.106967+0000","last_clean":"2024-03-06T07:21:06.106967+0000","last_became_active":"2024-03-06T07:21:05.093090+0000","last_became_peered":"2024-03-06T07:21:05.093090+0000","last_unstale":"2024-03-06T07:21:06.106967+0000","last_undegraded":"2024-03-06T07:21:06.106967+0000","last_fullsized":"2024-03-06T07:21:06.106967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078669+0000","last_change":"2024-03-06T07:21:06.088015+0000","last_active":"2024-03-06T07:21:07.078669+0000","last_peered":"2024-03-06T07:21:07.078669+0000","last_clean":"2024-03-06T07:21:07.078669+0000","last_became_active":"2024-03-06T07:21:06.087672+0000","last_became_peered":"2024-03-06T07:21:06.087672+0000","last_unstale":"2024-03-06T07:21:07.078669+0000","last_undegraded":"2024-03-06T07:21:07.078669+0000","last_fullsized":"2024-03-06T07:21:07.078669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:25.221930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079574+0000","last_change":"2024-03-06T07:21:05.085786+0000","last_active":"2024-03-06T07:21:07.079574+0000","last_peered":"2024-03-06T07:21:07.079574+0000","last_clean":"2024-03-06T07:21:07.079574+0000","last_became_active":"2024-03-06T07:21:05.085526+0000","last_became_peered":"2024-03-06T07:21:05.085526+0000","last_unstale":"2024-03-06T07:21:07.079574+0000","last_undegraded":"2024-03-06T07:21:07.079574+0000","last_fullsized":"2024-03-06T07:21:07.079574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:35.243708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.095659+0000","last_change":"2024-03-06T07:21:06.094087+0000","last_active":"2024-03-06T07:21:06.095659+0000","last_peered":"2024-03-06T07:21:06.095659+0000","last_clean":"2024-03-06T07:21:06.095659+0000","last_became_active":"2024-03-06T07:21:06.093130+0000","last_became_peered":"2024-03-06T07:21:06.093130+0000","last_unstale":"2024-03-06T07:21:06.095659+0000","last_undegraded":"2024-03-06T07:21:06.095659+0000","last_fullsized":"2024-03-06T07:21:06.095659+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098383+0000","last_change":"2024-03-06T07:21:05.092616+0000","last_active":"2024-03-06T07:21:07.098383+0000","last_peered":"2024-03-06T07:21:07.098383+0000","last_clean":"2024-03-06T07:21:07.098383+0000","last_became_active":"2024-03-06T07:21:05.092315+0000","last_became_peered":"2024-03-06T07:21:05.092315+0000","last_unstale":"2024-03-06T07:21:07.098383+0000","last_undegraded":"2024-03-06T07:21:07.098383+0000","last_fullsized":"2024-03-06T07:21:07.098383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:05.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098378+0000","last_change":"2024-03-06T07:21:06.101952+0000","last_active":"2024-03-06T07:21:07.098378+0000","last_peered":"2024-03-06T07:21:07.098378+0000","last_clean":"2024-03-06T07:21:07.098378+0000","last_became_active":"2024-03-06T07:21:06.101654+0000","last_became_peered":"2024-03-06T07:21:06.101654+0000","last_unstale":"2024-03-06T07:21:07.098378+0000","last_undegraded":"2024-03-06T07:21:07.098378+0000","last_fullsized":"2024-03-06T07:21:07.098378+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:19:29.695081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.110671+0000","last_change":"2024-03-06T07:21:05.087989+0000","last_active":"2024-03-06T07:21:06.110671+0000","last_peered":"2024-03-06T07:21:06.110671+0000","last_clean":"2024-03-06T07:21:06.110671+0000","last_became_active":"2024-03-06T07:21:05.087401+0000","last_became_peered":"2024-03-06T07:21:05.087401+0000","last_unstale":"2024-03-06T07:21:06.110671+0000","last_undegraded":"2024-03-06T07:21:06.110671+0000","last_fullsized":"2024-03-06T07:21:06.110671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079332+0000","last_change":"2024-03-06T07:21:06.100412+0000","last_active":"2024-03-06T07:21:07.079332+0000","last_peered":"2024-03-06T07:21:07.079332+0000","last_clean":"2024-03-06T07:21:07.079332+0000","last_became_active":"2024-03-06T07:21:06.100141+0000","last_became_peered":"2024-03-06T07:21:06.100141+0000","last_unstale":"2024-03-06T07:21:07.079332+0000","last_undegraded":"2024-03-06T07:21:07.079332+0000","last_fullsized":"2024-03-06T07:21:07.079332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:00.831001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.082919+0000","last_change":"2024-03-06T07:21:05.094485+0000","last_active":"2024-03-06T07:21:06.082919+0000","last_peered":"2024-03-06T07:21:06.082919+0000","last_clean":"2024-03-06T07:21:06.082919+0000","last_became_active":"2024-03-06T07:21:05.094192+0000","last_became_peered":"2024-03-06T07:21:05.094192+0000","last_unstale":"2024-03-06T07:21:06.082919+0000","last_undegraded":"2024-03-06T07:21:06.082919+0000","last_fullsized":"2024-03-06T07:21:06.082919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078409+0000","last_change":"2024-03-06T07:21:06.095557+0000","last_active":"2024-03-06T07:21:07.078409+0000","last_peered":"2024-03-06T07:21:07.078409+0000","last_clean":"2024-03-06T07:21:07.078409+0000","last_became_active":"2024-03-06T07:21:06.095274+0000","last_became_peered":"2024-03-06T07:21:06.095274+0000","last_unstale":"2024-03-06T07:21:07.078409+0000","last_undegraded":"2024-03-06T07:21:07.078409+0000","last_fullsized":"2024-03-06T07:21:07.078409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.909624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078569+0000","last_change":"2024-03-06T07:21:05.088582+0000","last_active":"2024-03-06T07:21:07.078569+0000","last_peered":"2024-03-06T07:21:07.078569+0000","last_clean":"2024-03-06T07:21:07.078569+0000","last_became_active":"2024-03-06T07:21:05.088307+0000","last_became_peered":"2024-03-06T07:21:05.088307+0000","last_unstale":"2024-03-06T07:21:07.078569+0000","last_undegraded":"2024-03-06T07:21:07.078569+0000","last_fullsized":"2024-03-06T07:21:07.078569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096683+0000","last_change":"2024-03-06T07:21:06.090692+0000","last_active":"2024-03-06T07:21:07.096683+0000","last_peered":"2024-03-06T07:21:07.096683+0000","last_clean":"2024-03-06T07:21:07.096683+0000","last_became_active":"2024-03-06T07:21:06.090397+0000","last_became_peered":"2024-03-06T07:21:06.090397+0000","last_unstale":"2024-03-06T07:21:07.096683+0000","last_undegraded":"2024-03-06T07:21:07.096683+0000","last_fullsized":"2024-03-06T07:21:07.096683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:29:39.575589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078719+0000","last_change":"2024-03-06T07:21:05.093205+0000","last_active":"2024-03-06T07:21:07.078719+0000","last_peered":"2024-03-06T07:21:07.078719+0000","last_clean":"2024-03-06T07:21:07.078719+0000","last_became_active":"2024-03-06T07:21:05.092925+0000","last_became_peered":"2024-03-06T07:21:05.092925+0000","last_unstale":"2024-03-06T07:21:07.078719+0000","last_undegraded":"2024-03-06T07:21:07.078719+0000","last_fullsized":"2024-03-06T07:21:07.078719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:51.171906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104895+0000","last_change":"2024-03-06T07:21:06.101181+0000","last_active":"2024-03-06T07:21:06.104895+0000","last_peered":"2024-03-06T07:21:06.104895+0000","last_clean":"2024-03-06T07:21:06.104895+0000","last_became_active":"2024-03-06T07:21:06.100923+0000","last_became_peered":"2024-03-06T07:21:06.100923+0000","last_unstale":"2024-03-06T07:21:06.104895+0000","last_undegraded":"2024-03-06T07:21:06.104895+0000","last_fullsized":"2024-03-06T07:21:06.104895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098672+0000","last_change":"2024-03-06T07:21:05.089625+0000","last_active":"2024-03-06T07:21:07.098672+0000","last_peered":"2024-03-06T07:21:07.098672+0000","last_clean":"2024-03-06T07:21:07.098672+0000","last_became_active":"2024-03-06T07:21:05.089370+0000","last_became_peered":"2024-03-06T07:21:05.089370+0000","last_unstale":"2024-03-06T07:21:07.098672+0000","last_undegraded":"2024-03-06T07:21:07.098672+0000","last_fullsized":"2024-03-06T07:21:07.098672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:30.147471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.098782+0000","last_change":"2024-03-06T07:21:06.098086+0000","last_active":"2024-03-06T07:21:06.098782+0000","last_peered":"2024-03-06T07:21:06.098782+0000","last_clean":"2024-03-06T07:21:06.098782+0000","last_became_active":"2024-03-06T07:21:06.097742+0000","last_became_peered":"2024-03-06T07:21:06.097742+0000","last_unstale":"2024-03-06T07:21:06.098782+0000","last_undegraded":"2024-03-06T07:21:06.098782+0000","last_fullsized":"2024-03-06T07:21:06.098782+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098699+0000","last_change":"2024-03-06T07:21:05.094453+0000","last_active":"2024-03-06T07:21:07.098699+0000","last_peered":"2024-03-06T07:21:07.098699+0000","last_clean":"2024-03-06T07:21:07.098699+0000","last_became_active":"2024-03-06T07:21:05.094163+0000","last_became_peered":"2024-03-06T07:21:05.094163+0000","last_unstale":"2024-03-06T07:21:07.098699+0000","last_undegraded":"2024-03-06T07:21:07.098699+0000","last_fullsized":"2024-03-06T07:21:07.098699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:34.562031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079470+0000","last_change":"2024-03-06T07:21:06.082936+0000","last_active":"2024-03-06T07:21:07.079470+0000","last_peered":"2024-03-06T07:21:07.079470+0000","last_clean":"2024-03-06T07:21:07.079470+0000","last_became_active":"2024-03-06T07:21:06.082654+0000","last_became_peered":"2024-03-06T07:21:06.082654+0000","last_unstale":"2024-03-06T07:21:07.079470+0000","last_undegraded":"2024-03-06T07:21:07.079470+0000","last_fullsized":"2024-03-06T07:21:07.079470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:34.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078517+0000","last_change":"2024-03-06T07:21:05.091330+0000","last_active":"2024-03-06T07:21:07.078517+0000","last_peered":"2024-03-06T07:21:07.078517+0000","last_clean":"2024-03-06T07:21:07.078517+0000","last_became_active":"2024-03-06T07:21:05.091070+0000","last_became_peered":"2024-03-06T07:21:05.091070+0000","last_unstale":"2024-03-06T07:21:07.078517+0000","last_undegraded":"2024-03-06T07:21:07.078517+0000","last_fullsized":"2024-03-06T07:21:07.078517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:08.208214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107909+0000","last_change":"2024-03-06T07:21:06.104641+0000","last_active":"2024-03-06T07:21:06.107909+0000","last_peered":"2024-03-06T07:21:06.107909+0000","last_clean":"2024-03-06T07:21:06.107909+0000","last_became_active":"2024-03-06T07:21:06.104341+0000","last_became_peered":"2024-03-06T07:21:06.104341+0000","last_unstale":"2024-03-06T07:21:06.107909+0000","last_undegraded":"2024-03-06T07:21:06.107909+0000","last_fullsized":"2024-03-06T07:21:06.107909+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:46.662722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078647+0000","last_change":"2024-03-06T07:21:05.093190+0000","last_active":"2024-03-06T07:21:07.078647+0000","last_peered":"2024-03-06T07:21:07.078647+0000","last_clean":"2024-03-06T07:21:07.078647+0000","last_became_active":"2024-03-06T07:21:05.092893+0000","last_became_peered":"2024-03-06T07:21:05.092893+0000","last_unstale":"2024-03-06T07:21:07.078647+0000","last_undegraded":"2024-03-06T07:21:07.078647+0000","last_fullsized":"2024-03-06T07:21:07.078647+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:10.223143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078247+0000","last_change":"2024-03-06T07:21:06.093572+0000","last_active":"2024-03-06T07:21:07.078247+0000","last_peered":"2024-03-06T07:21:07.078247+0000","last_clean":"2024-03-06T07:21:07.078247+0000","last_became_active":"2024-03-06T07:21:06.093315+0000","last_became_peered":"2024-03-06T07:21:06.093315+0000","last_unstale":"2024-03-06T07:21:07.078247+0000","last_undegraded":"2024-03-06T07:21:07.078247+0000","last_fullsized":"2024-03-06T07:21:07.078247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:19:29.695081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.082864+0000","last_change":"2024-03-06T07:21:05.094724+0000","last_active":"2024-03-06T07:21:06.082864+0000","last_peered":"2024-03-06T07:21:06.082864+0000","last_clean":"2024-03-06T07:21:06.082864+0000","last_became_active":"2024-03-06T07:21:05.094275+0000","last_became_peered":"2024-03-06T07:21:05.094275+0000","last_unstale":"2024-03-06T07:21:06.082864+0000","last_undegraded":"2024-03-06T07:21:06.082864+0000","last_fullsized":"2024-03-06T07:21:06.082864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:19.133399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.096888+0000","last_change":"2024-03-06T07:21:06.090830+0000","last_active":"2024-03-06T07:21:06.096888+0000","last_peered":"2024-03-06T07:21:06.096888+0000","last_clean":"2024-03-06T07:21:06.096888+0000","last_became_active":"2024-03-06T07:21:06.090592+0000","last_became_peered":"2024-03-06T07:21:06.090592+0000","last_unstale":"2024-03-06T07:21:06.096888+0000","last_undegraded":"2024-03-06T07:21:06.096888+0000","last_fullsized":"2024-03-06T07:21:06.096888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078898+0000","last_change":"2024-03-06T07:21:05.085103+0000","last_active":"2024-03-06T07:21:07.078898+0000","last_peered":"2024-03-06T07:21:07.078898+0000","last_clean":"2024-03-06T07:21:07.078898+0000","last_became_active":"2024-03-06T07:21:05.084837+0000","last_became_peered":"2024-03-06T07:21:05.084837+0000","last_unstale":"2024-03-06T07:21:07.078898+0000","last_undegraded":"2024-03-06T07:21:07.078898+0000","last_fullsized":"2024-03-06T07:21:07.078898+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:14.044602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078550+0000","last_change":"2024-03-06T07:21:06.098746+0000","last_active":"2024-03-06T07:21:07.078550+0000","last_peered":"2024-03-06T07:21:07.078550+0000","last_clean":"2024-03-06T07:21:07.078550+0000","last_became_active":"2024-03-06T07:21:06.098372+0000","last_became_peered":"2024-03-06T07:21:06.098372+0000","last_unstale":"2024-03-06T07:21:07.078550+0000","last_undegraded":"2024-03-06T07:21:07.078550+0000","last_fullsized":"2024-03-06T07:21:07.078550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:47.552521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.111669+0000","last_change":"2024-03-06T07:21:05.092851+0000","last_active":"2024-03-06T07:21:06.111669+0000","last_peered":"2024-03-06T07:21:06.111669+0000","last_clean":"2024-03-06T07:21:06.111669+0000","last_became_active":"2024-03-06T07:21:05.092584+0000","last_became_peered":"2024-03-06T07:21:05.092584+0000","last_unstale":"2024-03-06T07:21:06.111669+0000","last_undegraded":"2024-03-06T07:21:06.111669+0000","last_fullsized":"2024-03-06T07:21:06.111669+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:19.133399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078423+0000","last_change":"2024-03-06T07:21:06.096517+0000","last_active":"2024-03-06T07:21:07.078423+0000","last_peered":"2024-03-06T07:21:07.078423+0000","last_clean":"2024-03-06T07:21:07.078423+0000","last_became_active":"2024-03-06T07:21:06.096150+0000","last_became_peered":"2024-03-06T07:21:06.096150+0000","last_unstale":"2024-03-06T07:21:07.078423+0000","last_undegraded":"2024-03-06T07:21:07.078423+0000","last_fullsized":"2024-03-06T07:21:07.078423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:36.243435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098693+0000","last_change":"2024-03-06T07:21:05.090623+0000","last_active":"2024-03-06T07:21:07.098693+0000","last_peered":"2024-03-06T07:21:07.098693+0000","last_clean":"2024-03-06T07:21:07.098693+0000","last_became_active":"2024-03-06T07:21:05.090366+0000","last_became_peered":"2024-03-06T07:21:05.090366+0000","last_unstale":"2024-03-06T07:21:07.098693+0000","last_undegraded":"2024-03-06T07:21:07.098693+0000","last_fullsized":"2024-03-06T07:21:07.098693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:33.799299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098612+0000","last_change":"2024-03-06T07:21:06.102335+0000","last_active":"2024-03-06T07:21:07.098612+0000","last_peered":"2024-03-06T07:21:07.098612+0000","last_clean":"2024-03-06T07:21:07.098612+0000","last_became_active":"2024-03-06T07:21:06.102052+0000","last_became_peered":"2024-03-06T07:21:06.102052+0000","last_unstale":"2024-03-06T07:21:07.098612+0000","last_undegraded":"2024-03-06T07:21:07.098612+0000","last_fullsized":"2024-03-06T07:21:07.098612+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:06.978703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079222+0000","last_change":"2024-03-06T07:21:05.086098+0000","last_active":"2024-03-06T07:21:07.079222+0000","last_peered":"2024-03-06T07:21:07.079222+0000","last_clean":"2024-03-06T07:21:07.079222+0000","last_became_active":"2024-03-06T07:21:05.085188+0000","last_became_peered":"2024-03-06T07:21:05.085188+0000","last_unstale":"2024-03-06T07:21:07.079222+0000","last_undegraded":"2024-03-06T07:21:07.079222+0000","last_fullsized":"2024-03-06T07:21:07.079222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079015+0000","last_change":"2024-03-06T07:21:06.088754+0000","last_active":"2024-03-06T07:21:07.079015+0000","last_peered":"2024-03-06T07:21:07.079015+0000","last_clean":"2024-03-06T07:21:07.079015+0000","last_became_active":"2024-03-06T07:21:06.086597+0000","last_became_peered":"2024-03-06T07:21:06.086597+0000","last_unstale":"2024-03-06T07:21:07.079015+0000","last_undegraded":"2024-03-06T07:21:07.079015+0000","last_fullsized":"2024-03-06T07:21:07.079015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:09.212815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096695+0000","last_change":"2024-03-06T07:21:05.087302+0000","last_active":"2024-03-06T07:21:07.096695+0000","last_peered":"2024-03-06T07:21:07.096695+0000","last_clean":"2024-03-06T07:21:07.096695+0000","last_became_active":"2024-03-06T07:21:05.087023+0000","last_became_peered":"2024-03-06T07:21:05.087023+0000","last_unstale":"2024-03-06T07:21:07.096695+0000","last_undegraded":"2024-03-06T07:21:07.096695+0000","last_fullsized":"2024-03-06T07:21:07.096695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:55.682506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096791+0000","last_change":"2024-03-06T07:21:06.087099+0000","last_active":"2024-03-06T07:21:07.096791+0000","last_peered":"2024-03-06T07:21:07.096791+0000","last_clean":"2024-03-06T07:21:07.096791+0000","last_became_active":"2024-03-06T07:21:06.086793+0000","last_became_peered":"2024-03-06T07:21:06.086793+0000","last_unstale":"2024-03-06T07:21:07.096791+0000","last_undegraded":"2024-03-06T07:21:07.096791+0000","last_fullsized":"2024-03-06T07:21:07.096791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:20.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083092+0000","last_change":"2024-03-06T07:21:05.098028+0000","last_active":"2024-03-06T07:21:06.083092+0000","last_peered":"2024-03-06T07:21:06.083092+0000","last_clean":"2024-03-06T07:21:06.083092+0000","last_became_active":"2024-03-06T07:21:05.097634+0000","last_became_peered":"2024-03-06T07:21:05.097634+0000","last_unstale":"2024-03-06T07:21:06.083092+0000","last_undegraded":"2024-03-06T07:21:06.083092+0000","last_fullsized":"2024-03-06T07:21:06.083092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097161+0000","last_change":"2024-03-06T07:21:06.091645+0000","last_active":"2024-03-06T07:21:06.097161+0000","last_peered":"2024-03-06T07:21:06.097161+0000","last_clean":"2024-03-06T07:21:06.097161+0000","last_became_active":"2024-03-06T07:21:06.091407+0000","last_became_peered":"2024-03-06T07:21:06.091407+0000","last_unstale":"2024-03-06T07:21:06.097161+0000","last_undegraded":"2024-03-06T07:21:06.097161+0000","last_fullsized":"2024-03-06T07:21:06.097161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096526+0000","last_change":"2024-03-06T07:21:05.089793+0000","last_active":"2024-03-06T07:21:07.096526+0000","last_peered":"2024-03-06T07:21:07.096526+0000","last_clean":"2024-03-06T07:21:07.096526+0000","last_became_active":"2024-03-06T07:21:05.089518+0000","last_became_peered":"2024-03-06T07:21:05.089518+0000","last_unstale":"2024-03-06T07:21:07.096526+0000","last_undegraded":"2024-03-06T07:21:07.096526+0000","last_fullsized":"2024-03-06T07:21:07.096526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:27.173833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.099866+0000","last_change":"2024-03-06T07:21:06.098242+0000","last_active":"2024-03-06T07:21:06.099866+0000","last_peered":"2024-03-06T07:21:06.099866+0000","last_clean":"2024-03-06T07:21:06.099866+0000","last_became_active":"2024-03-06T07:21:06.097832+0000","last_became_peered":"2024-03-06T07:21:06.097832+0000","last_unstale":"2024-03-06T07:21:06.099866+0000","last_undegraded":"2024-03-06T07:21:06.099866+0000","last_fullsized":"2024-03-06T07:21:06.099866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096494+0000","last_change":"2024-03-06T07:21:05.083387+0000","last_active":"2024-03-06T07:21:07.096494+0000","last_peered":"2024-03-06T07:21:07.096494+0000","last_clean":"2024-03-06T07:21:07.096494+0000","last_became_active":"2024-03-06T07:21:05.083080+0000","last_became_peered":"2024-03-06T07:21:05.083080+0000","last_unstale":"2024-03-06T07:21:07.096494+0000","last_undegraded":"2024-03-06T07:21:07.096494+0000","last_fullsized":"2024-03-06T07:21:07.096494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:26.614413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098324+0000","last_change":"2024-03-06T07:21:06.103567+0000","last_active":"2024-03-06T07:21:07.098324+0000","last_peered":"2024-03-06T07:21:07.098324+0000","last_clean":"2024-03-06T07:21:07.098324+0000","last_became_active":"2024-03-06T07:21:06.102705+0000","last_became_peered":"2024-03-06T07:21:06.102705+0000","last_unstale":"2024-03-06T07:21:07.098324+0000","last_undegraded":"2024-03-06T07:21:07.098324+0000","last_fullsized":"2024-03-06T07:21:07.098324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:01.488069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083038+0000","last_change":"2024-03-06T07:21:05.094737+0000","last_active":"2024-03-06T07:21:06.083038+0000","last_peered":"2024-03-06T07:21:06.083038+0000","last_clean":"2024-03-06T07:21:06.083038+0000","last_became_active":"2024-03-06T07:21:05.094472+0000","last_became_peered":"2024-03-06T07:21:05.094472+0000","last_unstale":"2024-03-06T07:21:06.083038+0000","last_undegraded":"2024-03-06T07:21:06.083038+0000","last_fullsized":"2024-03-06T07:21:06.083038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078602+0000","last_change":"2024-03-06T07:21:06.086457+0000","last_active":"2024-03-06T07:21:07.078602+0000","last_peered":"2024-03-06T07:21:07.078602+0000","last_clean":"2024-03-06T07:21:07.078602+0000","last_became_active":"2024-03-06T07:21:06.086101+0000","last_became_peered":"2024-03-06T07:21:06.086101+0000","last_unstale":"2024-03-06T07:21:07.078602+0000","last_undegraded":"2024-03-06T07:21:07.078602+0000","last_fullsized":"2024-03-06T07:21:07.078602+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:23.191444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096399+0000","last_change":"2024-03-06T07:21:05.089122+0000","last_active":"2024-03-06T07:21:07.096399+0000","last_peered":"2024-03-06T07:21:07.096399+0000","last_clean":"2024-03-06T07:21:07.096399+0000","last_became_active":"2024-03-06T07:21:05.088851+0000","last_became_peered":"2024-03-06T07:21:05.088851+0000","last_unstale":"2024-03-06T07:21:07.096399+0000","last_undegraded":"2024-03-06T07:21:07.096399+0000","last_fullsized":"2024-03-06T07:21:07.096399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:01.780170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104517+0000","last_change":"2024-03-06T07:21:06.103520+0000","last_active":"2024-03-06T07:21:06.104517+0000","last_peered":"2024-03-06T07:21:06.104517+0000","last_clean":"2024-03-06T07:21:06.104517+0000","last_became_active":"2024-03-06T07:21:06.103253+0000","last_became_peered":"2024-03-06T07:21:06.103253+0000","last_unstale":"2024-03-06T07:21:06.104517+0000","last_undegraded":"2024-03-06T07:21:06.104517+0000","last_fullsized":"2024-03-06T07:21:06.104517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098794+0000","last_change":"2024-03-06T07:21:05.094399+0000","last_active":"2024-03-06T07:21:07.098794+0000","last_peered":"2024-03-06T07:21:07.098794+0000","last_clean":"2024-03-06T07:21:07.098794+0000","last_became_active":"2024-03-06T07:21:05.094135+0000","last_became_peered":"2024-03-06T07:21:05.094135+0000","last_unstale":"2024-03-06T07:21:07.098794+0000","last_undegraded":"2024-03-06T07:21:07.098794+0000","last_fullsized":"2024-03-06T07:21:07.098794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:33.012779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104876+0000","last_change":"2024-03-06T07:21:06.100883+0000","last_active":"2024-03-06T07:21:06.104876+0000","last_peered":"2024-03-06T07:21:06.104876+0000","last_clean":"2024-03-06T07:21:06.104876+0000","last_became_active":"2024-03-06T07:21:06.096861+0000","last_became_peered":"2024-03-06T07:21:06.096861+0000","last_unstale":"2024-03-06T07:21:06.104876+0000","last_undegraded":"2024-03-06T07:21:06.104876+0000","last_fullsized":"2024-03-06T07:21:06.104876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:46.662722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.097791+0000","last_change":"2024-03-06T07:21:05.088022+0000","last_active":"2024-03-06T07:21:07.097791+0000","last_peered":"2024-03-06T07:21:07.097791+0000","last_clean":"2024-03-06T07:21:07.097791+0000","last_became_active":"2024-03-06T07:21:05.087705+0000","last_became_peered":"2024-03-06T07:21:05.087705+0000","last_unstale":"2024-03-06T07:21:07.097791+0000","last_undegraded":"2024-03-06T07:21:07.097791+0000","last_fullsized":"2024-03-06T07:21:07.097791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:30:30.936080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078451+0000","last_change":"2024-03-06T07:21:06.096761+0000","last_active":"2024-03-06T07:21:07.078451+0000","last_peered":"2024-03-06T07:21:07.078451+0000","last_clean":"2024-03-06T07:21:07.078451+0000","last_became_active":"2024-03-06T07:21:06.096250+0000","last_became_peered":"2024-03-06T07:21:06.096250+0000","last_unstale":"2024-03-06T07:21:07.078451+0000","last_undegraded":"2024-03-06T07:21:07.078451+0000","last_fullsized":"2024-03-06T07:21:07.078451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:06.978703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.087871+0000","last_change":"2024-03-06T07:21:05.096211+0000","last_active":"2024-03-06T07:21:06.087871+0000","last_peered":"2024-03-06T07:21:06.087871+0000","last_clean":"2024-03-06T07:21:06.087871+0000","last_became_active":"2024-03-06T07:21:05.095930+0000","last_became_peered":"2024-03-06T07:21:05.095930+0000","last_unstale":"2024-03-06T07:21:06.087871+0000","last_undegraded":"2024-03-06T07:21:06.087871+0000","last_fullsized":"2024-03-06T07:21:06.087871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.095638+0000","last_change":"2024-03-06T07:21:06.093043+0000","last_active":"2024-03-06T07:21:06.095638+0000","last_peered":"2024-03-06T07:21:06.095638+0000","last_clean":"2024-03-06T07:21:06.095638+0000","last_became_active":"2024-03-06T07:21:06.092770+0000","last_became_peered":"2024-03-06T07:21:06.092770+0000","last_unstale":"2024-03-06T07:21:06.095638+0000","last_undegraded":"2024-03-06T07:21:06.095638+0000","last_fullsized":"2024-03-06T07:21:06.095638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.090989+0000","last_change":"2024-03-06T07:21:05.090269+0000","last_active":"2024-03-06T07:21:06.090989+0000","last_peered":"2024-03-06T07:21:06.090989+0000","last_clean":"2024-03-06T07:21:06.090989+0000","last_became_active":"2024-03-06T07:21:05.089761+0000","last_became_peered":"2024-03-06T07:21:05.089761+0000","last_unstale":"2024-03-06T07:21:06.090989+0000","last_undegraded":"2024-03-06T07:21:06.090989+0000","last_fullsized":"2024-03-06T07:21:06.090989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:35.243708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.093530+0000","last_change":"2024-03-06T07:21:06.090886+0000","last_active":"2024-03-06T07:21:06.093530+0000","last_peered":"2024-03-06T07:21:06.093530+0000","last_clean":"2024-03-06T07:21:06.093530+0000","last_became_active":"2024-03-06T07:21:06.090601+0000","last_became_peered":"2024-03-06T07:21:06.090601+0000","last_unstale":"2024-03-06T07:21:06.093530+0000","last_undegraded":"2024-03-06T07:21:06.093530+0000","last_fullsized":"2024-03-06T07:21:06.093530+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079116+0000","last_change":"2024-03-06T07:21:05.092926+0000","last_active":"2024-03-06T07:21:07.079116+0000","last_peered":"2024-03-06T07:21:07.079116+0000","last_clean":"2024-03-06T07:21:07.079116+0000","last_became_active":"2024-03-06T07:21:05.092335+0000","last_became_peered":"2024-03-06T07:21:05.092335+0000","last_unstale":"2024-03-06T07:21:07.079116+0000","last_undegraded":"2024-03-06T07:21:07.079116+0000","last_fullsized":"2024-03-06T07:21:07.079116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:21.910838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078990+0000","last_change":"2024-03-06T07:21:06.087824+0000","last_active":"2024-03-06T07:21:07.078990+0000","last_peered":"2024-03-06T07:21:07.078990+0000","last_clean":"2024-03-06T07:21:07.078990+0000","last_became_active":"2024-03-06T07:21:06.086550+0000","last_became_peered":"2024-03-06T07:21:06.086550+0000","last_unstale":"2024-03-06T07:21:07.078990+0000","last_undegraded":"2024-03-06T07:21:07.078990+0000","last_fullsized":"2024-03-06T07:21:07.078990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:52.176507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078744+0000","last_change":"2024-03-06T07:21:05.087524+0000","last_active":"2024-03-06T07:21:07.078744+0000","last_peered":"2024-03-06T07:21:07.078744+0000","last_clean":"2024-03-06T07:21:07.078744+0000","last_became_active":"2024-03-06T07:21:05.087235+0000","last_became_peered":"2024-03-06T07:21:05.087235+0000","last_unstale":"2024-03-06T07:21:07.078744+0000","last_undegraded":"2024-03-06T07:21:07.078744+0000","last_fullsized":"2024-03-06T07:21:07.078744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.102182+0000","last_change":"2024-03-06T07:21:06.096597+0000","last_active":"2024-03-06T07:21:06.102182+0000","last_peered":"2024-03-06T07:21:06.102182+0000","last_clean":"2024-03-06T07:21:06.102182+0000","last_became_active":"2024-03-06T07:21:06.096350+0000","last_became_peered":"2024-03-06T07:21:06.096350+0000","last_unstale":"2024-03-06T07:21:06.102182+0000","last_undegraded":"2024-03-06T07:21:06.102182+0000","last_fullsized":"2024-03-06T07:21:06.102182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078216+0000","last_change":"2024-03-06T07:21:05.095555+0000","last_active":"2024-03-06T07:21:07.078216+0000","last_peered":"2024-03-06T07:21:07.078216+0000","last_clean":"2024-03-06T07:21:07.078216+0000","last_became_active":"2024-03-06T07:21:05.095291+0000","last_became_peered":"2024-03-06T07:21:05.095291+0000","last_unstale":"2024-03-06T07:21:07.078216+0000","last_undegraded":"2024-03-06T07:21:07.078216+0000","last_fullsized":"2024-03-06T07:21:07.078216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:12.507293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098620+0000","last_change":"2024-03-06T07:21:06.104361+0000","last_active":"2024-03-06T07:21:07.098620+0000","last_peered":"2024-03-06T07:21:07.098620+0000","last_clean":"2024-03-06T07:21:07.098620+0000","last_became_active":"2024-03-06T07:21:06.104135+0000","last_became_peered":"2024-03-06T07:21:06.104135+0000","last_unstale":"2024-03-06T07:21:07.098620+0000","last_undegraded":"2024-03-06T07:21:07.098620+0000","last_fullsized":"2024-03-06T07:21:07.098620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:47.552521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.084026+0000","last_change":"2024-03-06T07:21:05.091940+0000","last_active":"2024-03-06T07:21:06.084026+0000","last_peered":"2024-03-06T07:21:06.084026+0000","last_clean":"2024-03-06T07:21:06.084026+0000","last_became_active":"2024-03-06T07:21:05.091650+0000","last_became_peered":"2024-03-06T07:21:05.091650+0000","last_unstale":"2024-03-06T07:21:06.084026+0000","last_undegraded":"2024-03-06T07:21:06.084026+0000","last_fullsized":"2024-03-06T07:21:06.084026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083076+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:06.083076+0000","last_peered":"2024-03-06T07:21:06.083076+0000","last_clean":"2024-03-06T07:21:06.083076+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:06.083076+0000","last_undegraded":"2024-03-06T07:21:06.083076+0000","last_fullsized":"2024-03-06T07:21:06.083076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":783017,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":191549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":197771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":192613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":191037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":191057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":781369,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:10.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:10.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.210+0000 7f6e5f039640 1 Processor -- start 2024-03-06T07:21:10.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.210+0000 7f6e5f039640 1 -- start start 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5f039640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e58156150 0x7f6e581529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5f039640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580c28d0 con 0x7f6e5814f790 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5f039640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580c2bb0 con 0x7f6e580c1aa0 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5f039640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580c2680 con 0x7f6e58156150 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e58156150 0x7f6e581529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6e58156150 0x7f6e581529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35702/0 (socket says 172.21.15.17:35702) 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 -- 172.21.15.17:0/2439442139 learned_addr learned my addr 172.21.15.17:0/2439442139 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 -- 172.21.15.17:0/2439442139 >> v1:172.21.15.163:6789/0 conn(0x7f6e580c1aa0 legacy=0x7f6e58155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 -- 172.21.15.17:0/2439442139 >> v1:172.21.15.17:6789/0 conn(0x7f6e5814f790 legacy=0x7f6e580c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.211+0000 7f6e5d5d6640 1 -- 172.21.15.17:0/2439442139 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580c2e90 con 0x7f6e58156150 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.212+0000 7f6e5d5d6640 1 --2- 172.21.15.17:0/2439442139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e58156150 0x7f6e581529d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6e580c2800 tx=0x7f6e4802dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0cadf516b26f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.212+0000 7f6e577fe640 1 -- 172.21.15.17:0/2439442139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4803b070 con 0x7f6e58156150 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.212+0000 7f6e577fe640 1 -- 172.21.15.17:0/2439442139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4802e9e0 con 0x7f6e58156150 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.212+0000 7f6e577fe640 1 -- 172.21.15.17:0/2439442139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4802edb0 con 0x7f6e58156150 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 -- 172.21.15.17:0/2439442139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e58156150 msgr2=0x7f6e581529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 --2- 172.21.15.17:0/2439442139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e58156150 0x7f6e581529d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6e580c2800 tx=0x7f6e4802dc30 comp rx=0 tx=0).stop 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 -- 172.21.15.17:0/2439442139 shutdown_connections 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 --2- 172.21.15.17:0/2439442139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e58156150 0x7f6e581529d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 -- 172.21.15.17:0/2439442139 >> 172.21.15.17:0/2439442139 conn(0x7f6e580c6710 msgr2=0x7f6e580c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 -- 172.21.15.17:0/2439442139 shutdown_connections 2024-03-06T07:21:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.213+0000 7f6e5f039640 1 -- 172.21.15.17:0/2439442139 wait complete. 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.214+0000 7f6e5f039640 1 Processor -- start 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.214+0000 7f6e5f039640 1 -- start start 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e5814f790 0x7f6e58154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 0x7f6e5814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c5e30 con 0x7f6e580c1aa0 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c0d60 con 0x7f6e58156150 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5f039640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c5fb0 con 0x7f6e5814f790 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35714/0 (socket says 172.21.15.17:35714) 2024-03-06T07:21:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 -- 172.21.15.17:0/318729662 learned_addr learned my addr 172.21.15.17:0/318729662 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5d5d6640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 0x7f6e5814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e57fff640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e5814f790 0x7f6e58154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 -- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e5814f790 msgr2=0x7f6e58154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e5814f790 0x7f6e58154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 -- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 msgr2=0x7f6e5814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 0x7f6e5814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 -- 172.21.15.17:0/318729662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5814d6c0 con 0x7f6e580c1aa0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5d5d6640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 0x7f6e5814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.215+0000 7f6e5cdd5640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000c870 tx=0x7f6e5000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.216+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50003b70 con 0x7f6e580c1aa0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.216+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50003d30 con 0x7f6e580c1aa0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.216+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5814d3e0 con 0x7f6e580c1aa0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.216+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000aa90 con 0x7f6e580c1aa0 2024-03-06T07:21:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.216+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e58154c50 con 0x7f6e580c1aa0 2024-03-06T07:21:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.217+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e58152410 con 0x7f6e580c1aa0 2024-03-06T07:21:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.217+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e5000ac30 con 0x7f6e580c1aa0 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.217+0000 7f6e557fa640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.217+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e5008a140 con 0x7f6e580c1aa0 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.218+0000 7f6e57fff640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.218+0000 7f6e57fff640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 0x7f6e4406d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073690 tx=0x7f6e4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.222+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6e58152410 con 0x7f6e580c1aa0 2024-03-06T07:21:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.398+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e5806c5b0 con 0x7f6e4406b500 2024-03-06T07:21:10.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.405+0000 7f6e557fa640 1 -- 172.21.15.17:0/318729662 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252003 (secure 0 0 0) 0x7f6e5806c5b0 con 0x7f6e4406b500 2024-03-06T07:21:10.409 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:10.410 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 msgr2=0x7f6e4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 0x7f6e4406d9c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073690 tx=0x7f6e4c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 msgr2=0x7f6e5814bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000c870 tx=0x7f6e5000cd30 comp rx=0 tx=0).stop 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 shutdown_connections 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6e4406b500 0x7f6e4406d9c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e58156150 0x7f6e5814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e5814f790 0x7f6e58154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 --2- 172.21.15.17:0/318729662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580c1aa0 0x7f6e5814bf10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 >> 172.21.15.17:0/318729662 conn(0x7f6e580c6710 msgr2=0x7f6e580c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 shutdown_connections 2024-03-06T07:21:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.408+0000 7f6e5f039640 1 -- 172.21.15.17:0/318729662 wait complete. 2024-03-06T07:21:10.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-06T07:21:09.354633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236480,"kb_used_data":3640,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738240,"statfs":{"total":773094113280,"available":772851957760,"internally_reserved":0,"allocated":3727360,"data_stored":2726676,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.284080"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078763+0000","last_change":"2024-03-06T07:21:06.099356+0000","last_active":"2024-03-06T07:21:07.078763+0000","last_peered":"2024-03-06T07:21:07.078763+0000","last_clean":"2024-03-06T07:21:07.078763+0000","last_became_active":"2024-03-06T07:21:06.096785+0000","last_became_peered":"2024-03-06T07:21:06.096785+0000","last_unstale":"2024-03-06T07:21:07.078763+0000","last_undegraded":"2024-03-06T07:21:07.078763+0000","last_fullsized":"2024-03-06T07:21:07.078763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:18.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078434+0000","last_change":"2024-03-06T07:21:06.089908+0000","last_active":"2024-03-06T07:21:07.078434+0000","last_peered":"2024-03-06T07:21:07.078434+0000","last_clean":"2024-03-06T07:21:07.078434+0000","last_became_active":"2024-03-06T07:21:06.089630+0000","last_became_peered":"2024-03-06T07:21:06.089630+0000","last_unstale":"2024-03-06T07:21:07.078434+0000","last_undegraded":"2024-03-06T07:21:07.078434+0000","last_fullsized":"2024-03-06T07:21:07.078434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:04.201769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.108109+0000","last_change":"2024-03-06T07:21:06.107183+0000","last_active":"2024-03-06T07:21:06.108109+0000","last_peered":"2024-03-06T07:21:06.108109+0000","last_clean":"2024-03-06T07:21:06.108109+0000","last_became_active":"2024-03-06T07:21:06.104951+0000","last_became_peered":"2024-03-06T07:21:06.104951+0000","last_unstale":"2024-03-06T07:21:06.108109+0000","last_undegraded":"2024-03-06T07:21:06.108109+0000","last_fullsized":"2024-03-06T07:21:06.108109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096676+0000","last_change":"2024-03-06T07:21:06.093481+0000","last_active":"2024-03-06T07:21:07.096676+0000","last_peered":"2024-03-06T07:21:07.096676+0000","last_clean":"2024-03-06T07:21:07.096676+0000","last_became_active":"2024-03-06T07:21:06.091728+0000","last_became_peered":"2024-03-06T07:21:06.091728+0000","last_unstale":"2024-03-06T07:21:07.096676+0000","last_undegraded":"2024-03-06T07:21:07.096676+0000","last_fullsized":"2024-03-06T07:21:07.096676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:55.490456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096499+0000","last_change":"2024-03-06T07:21:06.083513+0000","last_active":"2024-03-06T07:21:07.096499+0000","last_peered":"2024-03-06T07:21:07.096499+0000","last_clean":"2024-03-06T07:21:07.096499+0000","last_became_active":"2024-03-06T07:21:06.083154+0000","last_became_peered":"2024-03-06T07:21:06.083154+0000","last_unstale":"2024-03-06T07:21:07.096499+0000","last_undegraded":"2024-03-06T07:21:07.096499+0000","last_fullsized":"2024-03-06T07:21:07.096499+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:15.049203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.093568+0000","last_change":"2024-03-06T07:21:06.090047+0000","last_active":"2024-03-06T07:21:06.093568+0000","last_peered":"2024-03-06T07:21:06.093568+0000","last_clean":"2024-03-06T07:21:06.093568+0000","last_became_active":"2024-03-06T07:21:06.089757+0000","last_became_peered":"2024-03-06T07:21:06.089757+0000","last_unstale":"2024-03-06T07:21:06.093568+0000","last_undegraded":"2024-03-06T07:21:06.093568+0000","last_fullsized":"2024-03-06T07:21:06.093568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098225+0000","last_change":"2024-03-06T07:21:06.105353+0000","last_active":"2024-03-06T07:21:07.098225+0000","last_peered":"2024-03-06T07:21:07.098225+0000","last_clean":"2024-03-06T07:21:07.098225+0000","last_became_active":"2024-03-06T07:21:06.105103+0000","last_became_peered":"2024-03-06T07:21:06.105103+0000","last_unstale":"2024-03-06T07:21:07.098225+0000","last_undegraded":"2024-03-06T07:21:07.098225+0000","last_fullsized":"2024-03-06T07:21:07.098225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:44.500260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078200+0000","last_change":"2024-03-06T07:21:06.106008+0000","last_active":"2024-03-06T07:21:07.078200+0000","last_peered":"2024-03-06T07:21:07.078200+0000","last_clean":"2024-03-06T07:21:07.078200+0000","last_became_active":"2024-03-06T07:21:06.105715+0000","last_became_peered":"2024-03-06T07:21:06.105715+0000","last_unstale":"2024-03-06T07:21:07.078200+0000","last_undegraded":"2024-03-06T07:21:07.078200+0000","last_fullsized":"2024-03-06T07:21:07.078200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:08.094311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107794+0000","last_change":"2024-03-06T07:21:06.104535+0000","last_active":"2024-03-06T07:21:06.107794+0000","last_peered":"2024-03-06T07:21:06.107794+0000","last_clean":"2024-03-06T07:21:06.107794+0000","last_became_active":"2024-03-06T07:21:06.104252+0000","last_became_peered":"2024-03-06T07:21:06.104252+0000","last_unstale":"2024-03-06T07:21:06.107794+0000","last_undegraded":"2024-03-06T07:21:06.107794+0000","last_fullsized":"2024-03-06T07:21:06.107794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:35.566633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098551+0000","last_change":"2024-03-06T07:21:06.103954+0000","last_active":"2024-03-06T07:21:07.098551+0000","last_peered":"2024-03-06T07:21:07.098551+0000","last_clean":"2024-03-06T07:21:07.098551+0000","last_became_active":"2024-03-06T07:21:06.102851+0000","last_became_peered":"2024-03-06T07:21:06.102851+0000","last_unstale":"2024-03-06T07:21:07.098551+0000","last_undegraded":"2024-03-06T07:21:07.098551+0000","last_fullsized":"2024-03-06T07:21:07.098551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.909624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098608+0000","last_change":"2024-03-06T07:21:06.106113+0000","last_active":"2024-03-06T07:21:07.098608+0000","last_peered":"2024-03-06T07:21:07.098608+0000","last_clean":"2024-03-06T07:21:07.098608+0000","last_became_active":"2024-03-06T07:21:06.105811+0000","last_became_peered":"2024-03-06T07:21:06.105811+0000","last_unstale":"2024-03-06T07:21:07.098608+0000","last_undegraded":"2024-03-06T07:21:07.098608+0000","last_fullsized":"2024-03-06T07:21:07.098608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:04.201769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.103786+0000","last_change":"2024-03-06T07:21:06.101617+0000","last_active":"2024-03-06T07:21:06.103786+0000","last_peered":"2024-03-06T07:21:06.103786+0000","last_clean":"2024-03-06T07:21:06.103786+0000","last_became_active":"2024-03-06T07:21:06.101315+0000","last_became_peered":"2024-03-06T07:21:06.101315+0000","last_unstale":"2024-03-06T07:21:06.103786+0000","last_undegraded":"2024-03-06T07:21:06.103786+0000","last_fullsized":"2024-03-06T07:21:06.103786+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098288+0000","last_change":"2024-03-06T07:21:06.101712+0000","last_active":"2024-03-06T07:21:07.098288+0000","last_peered":"2024-03-06T07:21:07.098288+0000","last_clean":"2024-03-06T07:21:07.098288+0000","last_became_active":"2024-03-06T07:21:06.101439+0000","last_became_peered":"2024-03-06T07:21:06.101439+0000","last_unstale":"2024-03-06T07:21:07.098288+0000","last_undegraded":"2024-03-06T07:21:07.098288+0000","last_fullsized":"2024-03-06T07:21:07.098288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:19.714799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078859+0000","last_change":"2024-03-06T07:21:06.092741+0000","last_active":"2024-03-06T07:21:07.078859+0000","last_peered":"2024-03-06T07:21:07.078859+0000","last_clean":"2024-03-06T07:21:07.078859+0000","last_became_active":"2024-03-06T07:21:06.092475+0000","last_became_peered":"2024-03-06T07:21:06.092475+0000","last_unstale":"2024-03-06T07:21:07.078859+0000","last_undegraded":"2024-03-06T07:21:07.078859+0000","last_fullsized":"2024-03-06T07:21:07.078859+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098249+0000","last_change":"2024-03-06T07:21:06.103816+0000","last_active":"2024-03-06T07:21:07.098249+0000","last_peered":"2024-03-06T07:21:07.098249+0000","last_clean":"2024-03-06T07:21:07.098249+0000","last_became_active":"2024-03-06T07:21:06.103310+0000","last_became_peered":"2024-03-06T07:21:06.103310+0000","last_unstale":"2024-03-06T07:21:07.098249+0000","last_undegraded":"2024-03-06T07:21:07.098249+0000","last_fullsized":"2024-03-06T07:21:07.098249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:50.247773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.110323+0000","last_change":"2024-03-06T07:21:06.105144+0000","last_active":"2024-03-06T07:21:06.110323+0000","last_peered":"2024-03-06T07:21:06.110323+0000","last_clean":"2024-03-06T07:21:06.110323+0000","last_became_active":"2024-03-06T07:21:06.104893+0000","last_became_peered":"2024-03-06T07:21:06.104893+0000","last_unstale":"2024-03-06T07:21:06.110323+0000","last_undegraded":"2024-03-06T07:21:06.110323+0000","last_fullsized":"2024-03-06T07:21:06.110323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:52.176507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104848+0000","last_change":"2024-03-06T07:21:06.100513+0000","last_active":"2024-03-06T07:21:06.104848+0000","last_peered":"2024-03-06T07:21:06.104848+0000","last_clean":"2024-03-06T07:21:06.104848+0000","last_became_active":"2024-03-06T07:21:06.100255+0000","last_became_peered":"2024-03-06T07:21:06.100255+0000","last_unstale":"2024-03-06T07:21:06.104848+0000","last_undegraded":"2024-03-06T07:21:06.104848+0000","last_fullsized":"2024-03-06T07:21:06.104848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:35.566633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079311+0000","last_change":"2024-03-06T07:21:06.095506+0000","last_active":"2024-03-06T07:21:07.079311+0000","last_peered":"2024-03-06T07:21:07.079311+0000","last_clean":"2024-03-06T07:21:07.079311+0000","last_became_active":"2024-03-06T07:21:06.095254+0000","last_became_peered":"2024-03-06T07:21:06.095254+0000","last_unstale":"2024-03-06T07:21:07.079311+0000","last_undegraded":"2024-03-06T07:21:07.079311+0000","last_fullsized":"2024-03-06T07:21:07.079311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096684+0000","last_change":"2024-03-06T07:21:06.087805+0000","last_active":"2024-03-06T07:21:07.096684+0000","last_peered":"2024-03-06T07:21:07.096684+0000","last_clean":"2024-03-06T07:21:07.096684+0000","last_became_active":"2024-03-06T07:21:06.087524+0000","last_became_peered":"2024-03-06T07:21:06.087524+0000","last_unstale":"2024-03-06T07:21:07.096684+0000","last_undegraded":"2024-03-06T07:21:07.096684+0000","last_fullsized":"2024-03-06T07:21:07.096684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:33.218026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078751+0000","last_change":"2024-03-06T07:21:06.093728+0000","last_active":"2024-03-06T07:21:07.078751+0000","last_peered":"2024-03-06T07:21:07.078751+0000","last_clean":"2024-03-06T07:21:07.078751+0000","last_became_active":"2024-03-06T07:21:06.093470+0000","last_became_peered":"2024-03-06T07:21:06.093470+0000","last_unstale":"2024-03-06T07:21:07.078751+0000","last_undegraded":"2024-03-06T07:21:07.078751+0000","last_fullsized":"2024-03-06T07:21:07.078751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098323+0000","last_change":"2024-03-06T07:21:06.098747+0000","last_active":"2024-03-06T07:21:07.098323+0000","last_peered":"2024-03-06T07:21:07.098323+0000","last_clean":"2024-03-06T07:21:07.098323+0000","last_became_active":"2024-03-06T07:21:06.098308+0000","last_became_peered":"2024-03-06T07:21:06.098308+0000","last_unstale":"2024-03-06T07:21:07.098323+0000","last_undegraded":"2024-03-06T07:21:07.098323+0000","last_fullsized":"2024-03-06T07:21:07.098323+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:08.094311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079164+0000","last_change":"2024-03-06T07:21:06.090214+0000","last_active":"2024-03-06T07:21:07.079164+0000","last_peered":"2024-03-06T07:21:07.079164+0000","last_clean":"2024-03-06T07:21:07.079164+0000","last_became_active":"2024-03-06T07:21:06.089958+0000","last_became_peered":"2024-03-06T07:21:06.089958+0000","last_unstale":"2024-03-06T07:21:07.079164+0000","last_undegraded":"2024-03-06T07:21:07.079164+0000","last_fullsized":"2024-03-06T07:21:07.079164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:19.048287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097197+0000","last_change":"2024-03-06T07:21:06.092351+0000","last_active":"2024-03-06T07:21:06.097197+0000","last_peered":"2024-03-06T07:21:06.097197+0000","last_clean":"2024-03-06T07:21:06.097197+0000","last_became_active":"2024-03-06T07:21:06.092094+0000","last_became_peered":"2024-03-06T07:21:06.092094+0000","last_unstale":"2024-03-06T07:21:06.097197+0000","last_undegraded":"2024-03-06T07:21:06.097197+0000","last_fullsized":"2024-03-06T07:21:06.097197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096411+0000","last_change":"2024-03-06T07:21:06.088643+0000","last_active":"2024-03-06T07:21:07.096411+0000","last_peered":"2024-03-06T07:21:07.096411+0000","last_clean":"2024-03-06T07:21:07.096411+0000","last_became_active":"2024-03-06T07:21:06.088294+0000","last_became_peered":"2024-03-06T07:21:06.088294+0000","last_unstale":"2024-03-06T07:21:07.096411+0000","last_undegraded":"2024-03-06T07:21:07.096411+0000","last_fullsized":"2024-03-06T07:21:07.096411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:22.915440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079517+0000","last_change":"2024-03-06T07:21:06.093069+0000","last_active":"2024-03-06T07:21:07.079517+0000","last_peered":"2024-03-06T07:21:07.079517+0000","last_clean":"2024-03-06T07:21:07.079517+0000","last_became_active":"2024-03-06T07:21:06.092789+0000","last_became_peered":"2024-03-06T07:21:06.092789+0000","last_unstale":"2024-03-06T07:21:07.079517+0000","last_undegraded":"2024-03-06T07:21:07.079517+0000","last_fullsized":"2024-03-06T07:21:07.079517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104484+0000","last_change":"2024-03-06T07:21:06.102648+0000","last_active":"2024-03-06T07:21:06.104484+0000","last_peered":"2024-03-06T07:21:06.104484+0000","last_clean":"2024-03-06T07:21:06.104484+0000","last_became_active":"2024-03-06T07:21:06.102325+0000","last_became_peered":"2024-03-06T07:21:06.102325+0000","last_unstale":"2024-03-06T07:21:06.104484+0000","last_undegraded":"2024-03-06T07:21:06.104484+0000","last_fullsized":"2024-03-06T07:21:06.104484+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079027+0000","last_change":"2024-03-06T07:21:06.085000+0000","last_active":"2024-03-06T07:21:07.079027+0000","last_peered":"2024-03-06T07:21:07.079027+0000","last_clean":"2024-03-06T07:21:07.079027+0000","last_became_active":"2024-03-06T07:21:06.084697+0000","last_became_peered":"2024-03-06T07:21:06.084697+0000","last_unstale":"2024-03-06T07:21:07.079027+0000","last_undegraded":"2024-03-06T07:21:07.079027+0000","last_fullsized":"2024-03-06T07:21:07.079027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098593+0000","last_change":"2024-03-06T07:21:06.097795+0000","last_active":"2024-03-06T07:21:07.098593+0000","last_peered":"2024-03-06T07:21:07.098593+0000","last_clean":"2024-03-06T07:21:07.098593+0000","last_became_active":"2024-03-06T07:21:06.097486+0000","last_became_peered":"2024-03-06T07:21:06.097486+0000","last_unstale":"2024-03-06T07:21:07.098593+0000","last_undegraded":"2024-03-06T07:21:07.098593+0000","last_fullsized":"2024-03-06T07:21:07.098593+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:36.243435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078789+0000","last_change":"2024-03-06T07:21:06.086882+0000","last_active":"2024-03-06T07:21:07.078789+0000","last_peered":"2024-03-06T07:21:07.078789+0000","last_clean":"2024-03-06T07:21:07.078789+0000","last_became_active":"2024-03-06T07:21:06.086579+0000","last_became_peered":"2024-03-06T07:21:06.086579+0000","last_unstale":"2024-03-06T07:21:07.078789+0000","last_undegraded":"2024-03-06T07:21:07.078789+0000","last_fullsized":"2024-03-06T07:21:07.078789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079067+0000","last_change":"2024-03-06T07:21:06.087811+0000","last_active":"2024-03-06T07:21:07.079067+0000","last_peered":"2024-03-06T07:21:07.079067+0000","last_clean":"2024-03-06T07:21:07.079067+0000","last_became_active":"2024-03-06T07:21:06.087302+0000","last_became_peered":"2024-03-06T07:21:06.087302+0000","last_unstale":"2024-03-06T07:21:07.079067+0000","last_undegraded":"2024-03-06T07:21:07.079067+0000","last_fullsized":"2024-03-06T07:21:07.079067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.099001+0000","last_change":"2024-03-06T07:21:06.096001+0000","last_active":"2024-03-06T07:21:06.099001+0000","last_peered":"2024-03-06T07:21:06.099001+0000","last_clean":"2024-03-06T07:21:06.099001+0000","last_became_active":"2024-03-06T07:21:06.095734+0000","last_became_peered":"2024-03-06T07:21:06.095734+0000","last_unstale":"2024-03-06T07:21:06.099001+0000","last_undegraded":"2024-03-06T07:21:06.099001+0000","last_fullsized":"2024-03-06T07:21:06.099001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097220+0000","last_change":"2024-03-06T07:21:06.092028+0000","last_active":"2024-03-06T07:21:06.097220+0000","last_peered":"2024-03-06T07:21:06.097220+0000","last_clean":"2024-03-06T07:21:06.097220+0000","last_became_active":"2024-03-06T07:21:06.091600+0000","last_became_peered":"2024-03-06T07:21:06.091600+0000","last_unstale":"2024-03-06T07:21:06.097220+0000","last_undegraded":"2024-03-06T07:21:06.097220+0000","last_fullsized":"2024-03-06T07:21:06.097220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.105291+0000","last_change":"2024-03-06T07:21:06.100443+0000","last_active":"2024-03-06T07:21:06.105291+0000","last_peered":"2024-03-06T07:21:06.105291+0000","last_clean":"2024-03-06T07:21:06.105291+0000","last_became_active":"2024-03-06T07:21:06.100173+0000","last_became_peered":"2024-03-06T07:21:06.100173+0000","last_unstale":"2024-03-06T07:21:06.105291+0000","last_undegraded":"2024-03-06T07:21:06.105291+0000","last_fullsized":"2024-03-06T07:21:06.105291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079510+0000","last_change":"2024-03-06T07:21:05.091543+0000","last_active":"2024-03-06T07:21:07.079510+0000","last_peered":"2024-03-06T07:21:07.079510+0000","last_clean":"2024-03-06T07:21:07.079510+0000","last_became_active":"2024-03-06T07:21:05.091244+0000","last_became_peered":"2024-03-06T07:21:05.091244+0000","last_unstale":"2024-03-06T07:21:07.079510+0000","last_undegraded":"2024-03-06T07:21:07.079510+0000","last_fullsized":"2024-03-06T07:21:07.079510+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:20:54.485855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.106492+0000","last_change":"2024-03-06T07:21:06.104326+0000","last_active":"2024-03-06T07:21:06.106492+0000","last_peered":"2024-03-06T07:21:06.106492+0000","last_clean":"2024-03-06T07:21:06.106492+0000","last_became_active":"2024-03-06T07:21:06.104060+0000","last_became_peered":"2024-03-06T07:21:06.104060+0000","last_unstale":"2024-03-06T07:21:06.106492+0000","last_undegraded":"2024-03-06T07:21:06.106492+0000","last_fullsized":"2024-03-06T07:21:06.106492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:31.152072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107490+0000","last_change":"2024-03-06T07:21:05.097659+0000","last_active":"2024-03-06T07:21:06.107490+0000","last_peered":"2024-03-06T07:21:06.107490+0000","last_clean":"2024-03-06T07:21:06.107490+0000","last_became_active":"2024-03-06T07:21:05.097315+0000","last_became_peered":"2024-03-06T07:21:05.097315+0000","last_unstale":"2024-03-06T07:21:06.107490+0000","last_undegraded":"2024-03-06T07:21:06.107490+0000","last_fullsized":"2024-03-06T07:21:06.107490+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079138+0000","last_change":"2024-03-06T07:21:06.084799+0000","last_active":"2024-03-06T07:21:07.079138+0000","last_peered":"2024-03-06T07:21:07.079138+0000","last_clean":"2024-03-06T07:21:07.079138+0000","last_became_active":"2024-03-06T07:21:06.084507+0000","last_became_peered":"2024-03-06T07:21:06.084507+0000","last_unstale":"2024-03-06T07:21:07.079138+0000","last_undegraded":"2024-03-06T07:21:07.079138+0000","last_fullsized":"2024-03-06T07:21:07.079138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:11.227745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.106967+0000","last_change":"2024-03-06T07:21:05.093387+0000","last_active":"2024-03-06T07:21:06.106967+0000","last_peered":"2024-03-06T07:21:06.106967+0000","last_clean":"2024-03-06T07:21:06.106967+0000","last_became_active":"2024-03-06T07:21:05.093090+0000","last_became_peered":"2024-03-06T07:21:05.093090+0000","last_unstale":"2024-03-06T07:21:06.106967+0000","last_undegraded":"2024-03-06T07:21:06.106967+0000","last_fullsized":"2024-03-06T07:21:06.106967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078669+0000","last_change":"2024-03-06T07:21:06.088015+0000","last_active":"2024-03-06T07:21:07.078669+0000","last_peered":"2024-03-06T07:21:07.078669+0000","last_clean":"2024-03-06T07:21:07.078669+0000","last_became_active":"2024-03-06T07:21:06.087672+0000","last_became_peered":"2024-03-06T07:21:06.087672+0000","last_unstale":"2024-03-06T07:21:07.078669+0000","last_undegraded":"2024-03-06T07:21:07.078669+0000","last_fullsized":"2024-03-06T07:21:07.078669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:25.221930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079574+0000","last_change":"2024-03-06T07:21:05.085786+0000","last_active":"2024-03-06T07:21:07.079574+0000","last_peered":"2024-03-06T07:21:07.079574+0000","last_clean":"2024-03-06T07:21:07.079574+0000","last_became_active":"2024-03-06T07:21:05.085526+0000","last_became_peered":"2024-03-06T07:21:05.085526+0000","last_unstale":"2024-03-06T07:21:07.079574+0000","last_undegraded":"2024-03-06T07:21:07.079574+0000","last_fullsized":"2024-03-06T07:21:07.079574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:35.243708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.095659+0000","last_change":"2024-03-06T07:21:06.094087+0000","last_active":"2024-03-06T07:21:06.095659+0000","last_peered":"2024-03-06T07:21:06.095659+0000","last_clean":"2024-03-06T07:21:06.095659+0000","last_became_active":"2024-03-06T07:21:06.093130+0000","last_became_peered":"2024-03-06T07:21:06.093130+0000","last_unstale":"2024-03-06T07:21:06.095659+0000","last_undegraded":"2024-03-06T07:21:06.095659+0000","last_fullsized":"2024-03-06T07:21:06.095659+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:06.316849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098383+0000","last_change":"2024-03-06T07:21:05.092616+0000","last_active":"2024-03-06T07:21:07.098383+0000","last_peered":"2024-03-06T07:21:07.098383+0000","last_clean":"2024-03-06T07:21:07.098383+0000","last_became_active":"2024-03-06T07:21:05.092315+0000","last_became_peered":"2024-03-06T07:21:05.092315+0000","last_unstale":"2024-03-06T07:21:07.098383+0000","last_undegraded":"2024-03-06T07:21:07.098383+0000","last_fullsized":"2024-03-06T07:21:07.098383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:49:05.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098378+0000","last_change":"2024-03-06T07:21:06.101952+0000","last_active":"2024-03-06T07:21:07.098378+0000","last_peered":"2024-03-06T07:21:07.098378+0000","last_clean":"2024-03-06T07:21:07.098378+0000","last_became_active":"2024-03-06T07:21:06.101654+0000","last_became_peered":"2024-03-06T07:21:06.101654+0000","last_unstale":"2024-03-06T07:21:07.098378+0000","last_undegraded":"2024-03-06T07:21:07.098378+0000","last_fullsized":"2024-03-06T07:21:07.098378+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:19:29.695081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.110671+0000","last_change":"2024-03-06T07:21:05.087989+0000","last_active":"2024-03-06T07:21:06.110671+0000","last_peered":"2024-03-06T07:21:06.110671+0000","last_clean":"2024-03-06T07:21:06.110671+0000","last_became_active":"2024-03-06T07:21:05.087401+0000","last_became_peered":"2024-03-06T07:21:05.087401+0000","last_unstale":"2024-03-06T07:21:06.110671+0000","last_undegraded":"2024-03-06T07:21:06.110671+0000","last_fullsized":"2024-03-06T07:21:06.110671+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079332+0000","last_change":"2024-03-06T07:21:06.100412+0000","last_active":"2024-03-06T07:21:07.079332+0000","last_peered":"2024-03-06T07:21:07.079332+0000","last_clean":"2024-03-06T07:21:07.079332+0000","last_became_active":"2024-03-06T07:21:06.100141+0000","last_became_peered":"2024-03-06T07:21:06.100141+0000","last_unstale":"2024-03-06T07:21:07.079332+0000","last_undegraded":"2024-03-06T07:21:07.079332+0000","last_fullsized":"2024-03-06T07:21:07.079332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:00.831001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.082919+0000","last_change":"2024-03-06T07:21:05.094485+0000","last_active":"2024-03-06T07:21:06.082919+0000","last_peered":"2024-03-06T07:21:06.082919+0000","last_clean":"2024-03-06T07:21:06.082919+0000","last_became_active":"2024-03-06T07:21:05.094192+0000","last_became_peered":"2024-03-06T07:21:05.094192+0000","last_unstale":"2024-03-06T07:21:06.082919+0000","last_undegraded":"2024-03-06T07:21:06.082919+0000","last_fullsized":"2024-03-06T07:21:06.082919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078409+0000","last_change":"2024-03-06T07:21:06.095557+0000","last_active":"2024-03-06T07:21:07.078409+0000","last_peered":"2024-03-06T07:21:07.078409+0000","last_clean":"2024-03-06T07:21:07.078409+0000","last_became_active":"2024-03-06T07:21:06.095274+0000","last_became_peered":"2024-03-06T07:21:06.095274+0000","last_unstale":"2024-03-06T07:21:07.078409+0000","last_undegraded":"2024-03-06T07:21:07.078409+0000","last_fullsized":"2024-03-06T07:21:07.078409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:51.909624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078569+0000","last_change":"2024-03-06T07:21:05.088582+0000","last_active":"2024-03-06T07:21:07.078569+0000","last_peered":"2024-03-06T07:21:07.078569+0000","last_clean":"2024-03-06T07:21:07.078569+0000","last_became_active":"2024-03-06T07:21:05.088307+0000","last_became_peered":"2024-03-06T07:21:05.088307+0000","last_unstale":"2024-03-06T07:21:07.078569+0000","last_undegraded":"2024-03-06T07:21:07.078569+0000","last_fullsized":"2024-03-06T07:21:07.078569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096683+0000","last_change":"2024-03-06T07:21:06.090692+0000","last_active":"2024-03-06T07:21:07.096683+0000","last_peered":"2024-03-06T07:21:07.096683+0000","last_clean":"2024-03-06T07:21:07.096683+0000","last_became_active":"2024-03-06T07:21:06.090397+0000","last_became_peered":"2024-03-06T07:21:06.090397+0000","last_unstale":"2024-03-06T07:21:07.096683+0000","last_undegraded":"2024-03-06T07:21:07.096683+0000","last_fullsized":"2024-03-06T07:21:07.096683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:29:39.575589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078719+0000","last_change":"2024-03-06T07:21:05.093205+0000","last_active":"2024-03-06T07:21:07.078719+0000","last_peered":"2024-03-06T07:21:07.078719+0000","last_clean":"2024-03-06T07:21:07.078719+0000","last_became_active":"2024-03-06T07:21:05.092925+0000","last_became_peered":"2024-03-06T07:21:05.092925+0000","last_unstale":"2024-03-06T07:21:07.078719+0000","last_undegraded":"2024-03-06T07:21:07.078719+0000","last_fullsized":"2024-03-06T07:21:07.078719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:51.171906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104895+0000","last_change":"2024-03-06T07:21:06.101181+0000","last_active":"2024-03-06T07:21:06.104895+0000","last_peered":"2024-03-06T07:21:06.104895+0000","last_clean":"2024-03-06T07:21:06.104895+0000","last_became_active":"2024-03-06T07:21:06.100923+0000","last_became_peered":"2024-03-06T07:21:06.100923+0000","last_unstale":"2024-03-06T07:21:06.104895+0000","last_undegraded":"2024-03-06T07:21:06.104895+0000","last_fullsized":"2024-03-06T07:21:06.104895+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:13.511895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098672+0000","last_change":"2024-03-06T07:21:05.089625+0000","last_active":"2024-03-06T07:21:07.098672+0000","last_peered":"2024-03-06T07:21:07.098672+0000","last_clean":"2024-03-06T07:21:07.098672+0000","last_became_active":"2024-03-06T07:21:05.089370+0000","last_became_peered":"2024-03-06T07:21:05.089370+0000","last_unstale":"2024-03-06T07:21:07.098672+0000","last_undegraded":"2024-03-06T07:21:07.098672+0000","last_fullsized":"2024-03-06T07:21:07.098672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:30.147471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.098782+0000","last_change":"2024-03-06T07:21:06.098086+0000","last_active":"2024-03-06T07:21:06.098782+0000","last_peered":"2024-03-06T07:21:06.098782+0000","last_clean":"2024-03-06T07:21:06.098782+0000","last_became_active":"2024-03-06T07:21:06.097742+0000","last_became_peered":"2024-03-06T07:21:06.097742+0000","last_unstale":"2024-03-06T07:21:06.098782+0000","last_undegraded":"2024-03-06T07:21:06.098782+0000","last_fullsized":"2024-03-06T07:21:06.098782+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:34.803901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098699+0000","last_change":"2024-03-06T07:21:05.094453+0000","last_active":"2024-03-06T07:21:07.098699+0000","last_peered":"2024-03-06T07:21:07.098699+0000","last_clean":"2024-03-06T07:21:07.098699+0000","last_became_active":"2024-03-06T07:21:05.094163+0000","last_became_peered":"2024-03-06T07:21:05.094163+0000","last_unstale":"2024-03-06T07:21:07.098699+0000","last_undegraded":"2024-03-06T07:21:07.098699+0000","last_fullsized":"2024-03-06T07:21:07.098699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:10:34.562031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079470+0000","last_change":"2024-03-06T07:21:06.082936+0000","last_active":"2024-03-06T07:21:07.079470+0000","last_peered":"2024-03-06T07:21:07.079470+0000","last_clean":"2024-03-06T07:21:07.079470+0000","last_became_active":"2024-03-06T07:21:06.082654+0000","last_became_peered":"2024-03-06T07:21:06.082654+0000","last_unstale":"2024-03-06T07:21:07.079470+0000","last_undegraded":"2024-03-06T07:21:07.079470+0000","last_fullsized":"2024-03-06T07:21:07.079470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:34.337257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078517+0000","last_change":"2024-03-06T07:21:05.091330+0000","last_active":"2024-03-06T07:21:07.078517+0000","last_peered":"2024-03-06T07:21:07.078517+0000","last_clean":"2024-03-06T07:21:07.078517+0000","last_became_active":"2024-03-06T07:21:05.091070+0000","last_became_peered":"2024-03-06T07:21:05.091070+0000","last_unstale":"2024-03-06T07:21:07.078517+0000","last_undegraded":"2024-03-06T07:21:07.078517+0000","last_fullsized":"2024-03-06T07:21:07.078517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:08.208214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.107909+0000","last_change":"2024-03-06T07:21:06.104641+0000","last_active":"2024-03-06T07:21:06.107909+0000","last_peered":"2024-03-06T07:21:06.107909+0000","last_clean":"2024-03-06T07:21:06.107909+0000","last_became_active":"2024-03-06T07:21:06.104341+0000","last_became_peered":"2024-03-06T07:21:06.104341+0000","last_unstale":"2024-03-06T07:21:06.107909+0000","last_undegraded":"2024-03-06T07:21:06.107909+0000","last_fullsized":"2024-03-06T07:21:06.107909+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:46.662722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078647+0000","last_change":"2024-03-06T07:21:05.093190+0000","last_active":"2024-03-06T07:21:07.078647+0000","last_peered":"2024-03-06T07:21:07.078647+0000","last_clean":"2024-03-06T07:21:07.078647+0000","last_became_active":"2024-03-06T07:21:05.092893+0000","last_became_peered":"2024-03-06T07:21:05.092893+0000","last_unstale":"2024-03-06T07:21:07.078647+0000","last_undegraded":"2024-03-06T07:21:07.078647+0000","last_fullsized":"2024-03-06T07:21:07.078647+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:10.223143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078247+0000","last_change":"2024-03-06T07:21:06.093572+0000","last_active":"2024-03-06T07:21:07.078247+0000","last_peered":"2024-03-06T07:21:07.078247+0000","last_clean":"2024-03-06T07:21:07.078247+0000","last_became_active":"2024-03-06T07:21:06.093315+0000","last_became_peered":"2024-03-06T07:21:06.093315+0000","last_unstale":"2024-03-06T07:21:07.078247+0000","last_undegraded":"2024-03-06T07:21:07.078247+0000","last_fullsized":"2024-03-06T07:21:07.078247+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:19:29.695081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.082864+0000","last_change":"2024-03-06T07:21:05.094724+0000","last_active":"2024-03-06T07:21:06.082864+0000","last_peered":"2024-03-06T07:21:06.082864+0000","last_clean":"2024-03-06T07:21:06.082864+0000","last_became_active":"2024-03-06T07:21:05.094275+0000","last_became_peered":"2024-03-06T07:21:05.094275+0000","last_unstale":"2024-03-06T07:21:06.082864+0000","last_undegraded":"2024-03-06T07:21:06.082864+0000","last_fullsized":"2024-03-06T07:21:06.082864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:19.133399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.096888+0000","last_change":"2024-03-06T07:21:06.090830+0000","last_active":"2024-03-06T07:21:06.096888+0000","last_peered":"2024-03-06T07:21:06.096888+0000","last_clean":"2024-03-06T07:21:06.096888+0000","last_became_active":"2024-03-06T07:21:06.090592+0000","last_became_peered":"2024-03-06T07:21:06.090592+0000","last_unstale":"2024-03-06T07:21:06.096888+0000","last_undegraded":"2024-03-06T07:21:06.096888+0000","last_fullsized":"2024-03-06T07:21:06.096888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:36.248310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078898+0000","last_change":"2024-03-06T07:21:05.085103+0000","last_active":"2024-03-06T07:21:07.078898+0000","last_peered":"2024-03-06T07:21:07.078898+0000","last_clean":"2024-03-06T07:21:07.078898+0000","last_became_active":"2024-03-06T07:21:05.084837+0000","last_became_peered":"2024-03-06T07:21:05.084837+0000","last_unstale":"2024-03-06T07:21:07.078898+0000","last_undegraded":"2024-03-06T07:21:07.078898+0000","last_fullsized":"2024-03-06T07:21:07.078898+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:12:14.044602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078550+0000","last_change":"2024-03-06T07:21:06.098746+0000","last_active":"2024-03-06T07:21:07.078550+0000","last_peered":"2024-03-06T07:21:07.078550+0000","last_clean":"2024-03-06T07:21:07.078550+0000","last_became_active":"2024-03-06T07:21:06.098372+0000","last_became_peered":"2024-03-06T07:21:06.098372+0000","last_unstale":"2024-03-06T07:21:07.078550+0000","last_undegraded":"2024-03-06T07:21:07.078550+0000","last_fullsized":"2024-03-06T07:21:07.078550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:47.552521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.111669+0000","last_change":"2024-03-06T07:21:05.092851+0000","last_active":"2024-03-06T07:21:06.111669+0000","last_peered":"2024-03-06T07:21:06.111669+0000","last_clean":"2024-03-06T07:21:06.111669+0000","last_became_active":"2024-03-06T07:21:05.092584+0000","last_became_peered":"2024-03-06T07:21:05.092584+0000","last_unstale":"2024-03-06T07:21:06.111669+0000","last_undegraded":"2024-03-06T07:21:06.111669+0000","last_fullsized":"2024-03-06T07:21:06.111669+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:19.133399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078423+0000","last_change":"2024-03-06T07:21:06.096517+0000","last_active":"2024-03-06T07:21:07.078423+0000","last_peered":"2024-03-06T07:21:07.078423+0000","last_clean":"2024-03-06T07:21:07.078423+0000","last_became_active":"2024-03-06T07:21:06.096150+0000","last_became_peered":"2024-03-06T07:21:06.096150+0000","last_unstale":"2024-03-06T07:21:07.078423+0000","last_undegraded":"2024-03-06T07:21:07.078423+0000","last_fullsized":"2024-03-06T07:21:07.078423+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:56:36.243435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098693+0000","last_change":"2024-03-06T07:21:05.090623+0000","last_active":"2024-03-06T07:21:07.098693+0000","last_peered":"2024-03-06T07:21:07.098693+0000","last_clean":"2024-03-06T07:21:07.098693+0000","last_became_active":"2024-03-06T07:21:05.090366+0000","last_became_peered":"2024-03-06T07:21:05.090366+0000","last_unstale":"2024-03-06T07:21:07.098693+0000","last_undegraded":"2024-03-06T07:21:07.098693+0000","last_fullsized":"2024-03-06T07:21:07.098693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:13:33.799299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098612+0000","last_change":"2024-03-06T07:21:06.102335+0000","last_active":"2024-03-06T07:21:07.098612+0000","last_peered":"2024-03-06T07:21:07.098612+0000","last_clean":"2024-03-06T07:21:07.098612+0000","last_became_active":"2024-03-06T07:21:06.102052+0000","last_became_peered":"2024-03-06T07:21:06.102052+0000","last_unstale":"2024-03-06T07:21:07.098612+0000","last_undegraded":"2024-03-06T07:21:07.098612+0000","last_fullsized":"2024-03-06T07:21:07.098612+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:06.978703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079222+0000","last_change":"2024-03-06T07:21:05.086098+0000","last_active":"2024-03-06T07:21:07.079222+0000","last_peered":"2024-03-06T07:21:07.079222+0000","last_clean":"2024-03-06T07:21:07.079222+0000","last_became_active":"2024-03-06T07:21:05.085188+0000","last_became_peered":"2024-03-06T07:21:05.085188+0000","last_unstale":"2024-03-06T07:21:07.079222+0000","last_undegraded":"2024-03-06T07:21:07.079222+0000","last_fullsized":"2024-03-06T07:21:07.079222+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079015+0000","last_change":"2024-03-06T07:21:06.088754+0000","last_active":"2024-03-06T07:21:07.079015+0000","last_peered":"2024-03-06T07:21:07.079015+0000","last_clean":"2024-03-06T07:21:07.079015+0000","last_became_active":"2024-03-06T07:21:06.086597+0000","last_became_peered":"2024-03-06T07:21:06.086597+0000","last_unstale":"2024-03-06T07:21:07.079015+0000","last_undegraded":"2024-03-06T07:21:07.079015+0000","last_fullsized":"2024-03-06T07:21:07.079015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:09.212815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096695+0000","last_change":"2024-03-06T07:21:05.087302+0000","last_active":"2024-03-06T07:21:07.096695+0000","last_peered":"2024-03-06T07:21:07.096695+0000","last_clean":"2024-03-06T07:21:07.096695+0000","last_became_active":"2024-03-06T07:21:05.087023+0000","last_became_peered":"2024-03-06T07:21:05.087023+0000","last_unstale":"2024-03-06T07:21:07.096695+0000","last_undegraded":"2024-03-06T07:21:07.096695+0000","last_fullsized":"2024-03-06T07:21:07.096695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:55.682506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096791+0000","last_change":"2024-03-06T07:21:06.087099+0000","last_active":"2024-03-06T07:21:07.096791+0000","last_peered":"2024-03-06T07:21:07.096791+0000","last_clean":"2024-03-06T07:21:07.096791+0000","last_became_active":"2024-03-06T07:21:06.086793+0000","last_became_peered":"2024-03-06T07:21:06.086793+0000","last_unstale":"2024-03-06T07:21:07.096791+0000","last_undegraded":"2024-03-06T07:21:07.096791+0000","last_fullsized":"2024-03-06T07:21:07.096791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:20.138001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083092+0000","last_change":"2024-03-06T07:21:05.098028+0000","last_active":"2024-03-06T07:21:06.083092+0000","last_peered":"2024-03-06T07:21:06.083092+0000","last_clean":"2024-03-06T07:21:06.083092+0000","last_became_active":"2024-03-06T07:21:05.097634+0000","last_became_peered":"2024-03-06T07:21:05.097634+0000","last_unstale":"2024-03-06T07:21:06.083092+0000","last_undegraded":"2024-03-06T07:21:06.083092+0000","last_fullsized":"2024-03-06T07:21:06.083092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.097161+0000","last_change":"2024-03-06T07:21:06.091645+0000","last_active":"2024-03-06T07:21:06.097161+0000","last_peered":"2024-03-06T07:21:06.097161+0000","last_clean":"2024-03-06T07:21:06.097161+0000","last_became_active":"2024-03-06T07:21:06.091407+0000","last_became_peered":"2024-03-06T07:21:06.091407+0000","last_unstale":"2024-03-06T07:21:06.097161+0000","last_undegraded":"2024-03-06T07:21:06.097161+0000","last_fullsized":"2024-03-06T07:21:06.097161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096526+0000","last_change":"2024-03-06T07:21:05.089793+0000","last_active":"2024-03-06T07:21:07.096526+0000","last_peered":"2024-03-06T07:21:07.096526+0000","last_clean":"2024-03-06T07:21:07.096526+0000","last_became_active":"2024-03-06T07:21:05.089518+0000","last_became_peered":"2024-03-06T07:21:05.089518+0000","last_unstale":"2024-03-06T07:21:07.096526+0000","last_undegraded":"2024-03-06T07:21:07.096526+0000","last_fullsized":"2024-03-06T07:21:07.096526+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:02:27.173833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.099866+0000","last_change":"2024-03-06T07:21:06.098242+0000","last_active":"2024-03-06T07:21:06.099866+0000","last_peered":"2024-03-06T07:21:06.099866+0000","last_clean":"2024-03-06T07:21:06.099866+0000","last_became_active":"2024-03-06T07:21:06.097832+0000","last_became_peered":"2024-03-06T07:21:06.097832+0000","last_unstale":"2024-03-06T07:21:06.099866+0000","last_undegraded":"2024-03-06T07:21:06.099866+0000","last_fullsized":"2024-03-06T07:21:06.099866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096494+0000","last_change":"2024-03-06T07:21:05.083387+0000","last_active":"2024-03-06T07:21:07.096494+0000","last_peered":"2024-03-06T07:21:07.096494+0000","last_clean":"2024-03-06T07:21:07.096494+0000","last_became_active":"2024-03-06T07:21:05.083080+0000","last_became_peered":"2024-03-06T07:21:05.083080+0000","last_unstale":"2024-03-06T07:21:07.096494+0000","last_undegraded":"2024-03-06T07:21:07.096494+0000","last_fullsized":"2024-03-06T07:21:07.096494+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:33:26.614413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098324+0000","last_change":"2024-03-06T07:21:06.103567+0000","last_active":"2024-03-06T07:21:07.098324+0000","last_peered":"2024-03-06T07:21:07.098324+0000","last_clean":"2024-03-06T07:21:07.098324+0000","last_became_active":"2024-03-06T07:21:06.102705+0000","last_became_peered":"2024-03-06T07:21:06.102705+0000","last_unstale":"2024-03-06T07:21:07.098324+0000","last_undegraded":"2024-03-06T07:21:07.098324+0000","last_fullsized":"2024-03-06T07:21:07.098324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:01.488069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083038+0000","last_change":"2024-03-06T07:21:05.094737+0000","last_active":"2024-03-06T07:21:06.083038+0000","last_peered":"2024-03-06T07:21:06.083038+0000","last_clean":"2024-03-06T07:21:06.083038+0000","last_became_active":"2024-03-06T07:21:05.094472+0000","last_became_peered":"2024-03-06T07:21:05.094472+0000","last_unstale":"2024-03-06T07:21:06.083038+0000","last_undegraded":"2024-03-06T07:21:06.083038+0000","last_fullsized":"2024-03-06T07:21:06.083038+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078602+0000","last_change":"2024-03-06T07:21:06.086457+0000","last_active":"2024-03-06T07:21:07.078602+0000","last_peered":"2024-03-06T07:21:07.078602+0000","last_clean":"2024-03-06T07:21:07.078602+0000","last_became_active":"2024-03-06T07:21:06.086101+0000","last_became_peered":"2024-03-06T07:21:06.086101+0000","last_unstale":"2024-03-06T07:21:07.078602+0000","last_undegraded":"2024-03-06T07:21:07.078602+0000","last_fullsized":"2024-03-06T07:21:07.078602+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:23.191444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.096399+0000","last_change":"2024-03-06T07:21:05.089122+0000","last_active":"2024-03-06T07:21:07.096399+0000","last_peered":"2024-03-06T07:21:07.096399+0000","last_clean":"2024-03-06T07:21:07.096399+0000","last_became_active":"2024-03-06T07:21:05.088851+0000","last_became_peered":"2024-03-06T07:21:05.088851+0000","last_unstale":"2024-03-06T07:21:07.096399+0000","last_undegraded":"2024-03-06T07:21:07.096399+0000","last_fullsized":"2024-03-06T07:21:07.096399+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:01.780170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104517+0000","last_change":"2024-03-06T07:21:06.103520+0000","last_active":"2024-03-06T07:21:06.104517+0000","last_peered":"2024-03-06T07:21:06.104517+0000","last_clean":"2024-03-06T07:21:06.104517+0000","last_became_active":"2024-03-06T07:21:06.103253+0000","last_became_peered":"2024-03-06T07:21:06.103253+0000","last_unstale":"2024-03-06T07:21:06.104517+0000","last_undegraded":"2024-03-06T07:21:06.104517+0000","last_fullsized":"2024-03-06T07:21:06.104517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098794+0000","last_change":"2024-03-06T07:21:05.094399+0000","last_active":"2024-03-06T07:21:07.098794+0000","last_peered":"2024-03-06T07:21:07.098794+0000","last_clean":"2024-03-06T07:21:07.098794+0000","last_became_active":"2024-03-06T07:21:05.094135+0000","last_became_peered":"2024-03-06T07:21:05.094135+0000","last_unstale":"2024-03-06T07:21:07.098794+0000","last_undegraded":"2024-03-06T07:21:07.098794+0000","last_fullsized":"2024-03-06T07:21:07.098794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:33.012779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.104876+0000","last_change":"2024-03-06T07:21:06.100883+0000","last_active":"2024-03-06T07:21:06.104876+0000","last_peered":"2024-03-06T07:21:06.104876+0000","last_clean":"2024-03-06T07:21:06.104876+0000","last_became_active":"2024-03-06T07:21:06.096861+0000","last_became_peered":"2024-03-06T07:21:06.096861+0000","last_unstale":"2024-03-06T07:21:06.104876+0000","last_undegraded":"2024-03-06T07:21:06.104876+0000","last_fullsized":"2024-03-06T07:21:06.104876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:46.662722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.097791+0000","last_change":"2024-03-06T07:21:05.088022+0000","last_active":"2024-03-06T07:21:07.097791+0000","last_peered":"2024-03-06T07:21:07.097791+0000","last_clean":"2024-03-06T07:21:07.097791+0000","last_became_active":"2024-03-06T07:21:05.087705+0000","last_became_peered":"2024-03-06T07:21:05.087705+0000","last_unstale":"2024-03-06T07:21:07.097791+0000","last_undegraded":"2024-03-06T07:21:07.097791+0000","last_fullsized":"2024-03-06T07:21:07.097791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:30:30.936080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078451+0000","last_change":"2024-03-06T07:21:06.096761+0000","last_active":"2024-03-06T07:21:07.078451+0000","last_peered":"2024-03-06T07:21:07.078451+0000","last_clean":"2024-03-06T07:21:07.078451+0000","last_became_active":"2024-03-06T07:21:06.096250+0000","last_became_peered":"2024-03-06T07:21:06.096250+0000","last_unstale":"2024-03-06T07:21:07.078451+0000","last_undegraded":"2024-03-06T07:21:07.078451+0000","last_fullsized":"2024-03-06T07:21:07.078451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:06.978703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.087871+0000","last_change":"2024-03-06T07:21:05.096211+0000","last_active":"2024-03-06T07:21:06.087871+0000","last_peered":"2024-03-06T07:21:06.087871+0000","last_clean":"2024-03-06T07:21:06.087871+0000","last_became_active":"2024-03-06T07:21:05.095930+0000","last_became_peered":"2024-03-06T07:21:05.095930+0000","last_unstale":"2024-03-06T07:21:06.087871+0000","last_undegraded":"2024-03-06T07:21:06.087871+0000","last_fullsized":"2024-03-06T07:21:06.087871+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:06:23.053976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.095638+0000","last_change":"2024-03-06T07:21:06.093043+0000","last_active":"2024-03-06T07:21:06.095638+0000","last_peered":"2024-03-06T07:21:06.095638+0000","last_clean":"2024-03-06T07:21:06.095638+0000","last_became_active":"2024-03-06T07:21:06.092770+0000","last_became_peered":"2024-03-06T07:21:06.092770+0000","last_unstale":"2024-03-06T07:21:06.095638+0000","last_undegraded":"2024-03-06T07:21:06.095638+0000","last_fullsized":"2024-03-06T07:21:06.095638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.090989+0000","last_change":"2024-03-06T07:21:05.090269+0000","last_active":"2024-03-06T07:21:06.090989+0000","last_peered":"2024-03-06T07:21:06.090989+0000","last_clean":"2024-03-06T07:21:06.090989+0000","last_became_active":"2024-03-06T07:21:05.089761+0000","last_became_peered":"2024-03-06T07:21:05.089761+0000","last_unstale":"2024-03-06T07:21:06.090989+0000","last_undegraded":"2024-03-06T07:21:06.090989+0000","last_fullsized":"2024-03-06T07:21:06.090989+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:35.243708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.093530+0000","last_change":"2024-03-06T07:21:06.090886+0000","last_active":"2024-03-06T07:21:06.093530+0000","last_peered":"2024-03-06T07:21:06.093530+0000","last_clean":"2024-03-06T07:21:06.093530+0000","last_became_active":"2024-03-06T07:21:06.090601+0000","last_became_peered":"2024-03-06T07:21:06.090601+0000","last_unstale":"2024-03-06T07:21:06.093530+0000","last_undegraded":"2024-03-06T07:21:06.093530+0000","last_fullsized":"2024-03-06T07:21:06.093530+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:37.231627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.079116+0000","last_change":"2024-03-06T07:21:05.092926+0000","last_active":"2024-03-06T07:21:07.079116+0000","last_peered":"2024-03-06T07:21:07.079116+0000","last_clean":"2024-03-06T07:21:07.079116+0000","last_became_active":"2024-03-06T07:21:05.092335+0000","last_became_peered":"2024-03-06T07:21:05.092335+0000","last_unstale":"2024-03-06T07:21:07.079116+0000","last_undegraded":"2024-03-06T07:21:07.079116+0000","last_fullsized":"2024-03-06T07:21:07.079116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:15:21.910838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078990+0000","last_change":"2024-03-06T07:21:06.087824+0000","last_active":"2024-03-06T07:21:07.078990+0000","last_peered":"2024-03-06T07:21:07.078990+0000","last_clean":"2024-03-06T07:21:07.078990+0000","last_became_active":"2024-03-06T07:21:06.086550+0000","last_became_peered":"2024-03-06T07:21:06.086550+0000","last_unstale":"2024-03-06T07:21:07.078990+0000","last_undegraded":"2024-03-06T07:21:07.078990+0000","last_fullsized":"2024-03-06T07:21:07.078990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:52.176507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078744+0000","last_change":"2024-03-06T07:21:05.087524+0000","last_active":"2024-03-06T07:21:07.078744+0000","last_peered":"2024-03-06T07:21:07.078744+0000","last_clean":"2024-03-06T07:21:07.078744+0000","last_became_active":"2024-03-06T07:21:05.087235+0000","last_became_peered":"2024-03-06T07:21:05.087235+0000","last_unstale":"2024-03-06T07:21:07.078744+0000","last_undegraded":"2024-03-06T07:21:07.078744+0000","last_fullsized":"2024-03-06T07:21:07.078744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:38:15.373385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.102182+0000","last_change":"2024-03-06T07:21:06.096597+0000","last_active":"2024-03-06T07:21:06.102182+0000","last_peered":"2024-03-06T07:21:06.102182+0000","last_clean":"2024-03-06T07:21:06.102182+0000","last_became_active":"2024-03-06T07:21:06.096350+0000","last_became_peered":"2024-03-06T07:21:06.096350+0000","last_unstale":"2024-03-06T07:21:06.102182+0000","last_undegraded":"2024-03-06T07:21:06.102182+0000","last_fullsized":"2024-03-06T07:21:06.102182+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:08:34.017381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.078216+0000","last_change":"2024-03-06T07:21:05.095555+0000","last_active":"2024-03-06T07:21:07.078216+0000","last_peered":"2024-03-06T07:21:07.078216+0000","last_clean":"2024-03-06T07:21:07.078216+0000","last_became_active":"2024-03-06T07:21:05.095291+0000","last_became_peered":"2024-03-06T07:21:05.095291+0000","last_unstale":"2024-03-06T07:21:07.078216+0000","last_undegraded":"2024-03-06T07:21:07.078216+0000","last_fullsized":"2024-03-06T07:21:07.078216+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:12.507293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-06T07:21:07.098620+0000","last_change":"2024-03-06T07:21:06.104361+0000","last_active":"2024-03-06T07:21:07.098620+0000","last_peered":"2024-03-06T07:21:07.098620+0000","last_clean":"2024-03-06T07:21:07.098620+0000","last_became_active":"2024-03-06T07:21:06.104135+0000","last_became_peered":"2024-03-06T07:21:06.104135+0000","last_unstale":"2024-03-06T07:21:07.098620+0000","last_undegraded":"2024-03-06T07:21:07.098620+0000","last_fullsized":"2024-03-06T07:21:07.098620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:05.068901+0000","last_clean_scrub_stamp":"2024-03-06T07:21:05.068901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:47.552521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.084026+0000","last_change":"2024-03-06T07:21:05.091940+0000","last_active":"2024-03-06T07:21:06.084026+0000","last_peered":"2024-03-06T07:21:06.084026+0000","last_clean":"2024-03-06T07:21:06.084026+0000","last_became_active":"2024-03-06T07:21:05.091650+0000","last_became_peered":"2024-03-06T07:21:05.091650+0000","last_unstale":"2024-03-06T07:21:06.084026+0000","last_undegraded":"2024-03-06T07:21:06.084026+0000","last_fullsized":"2024-03-06T07:21:06.084026+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:04.064300+0000","last_clean_scrub_stamp":"2024-03-06T07:21:04.064300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:22.307527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-06T07:21:06.083076+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:06.083076+0000","last_peered":"2024-03-06T07:21:06.083076+0000","last_clean":"2024-03-06T07:21:06.083076+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:06.083076+0000","last_undegraded":"2024-03-06T07:21:06.083076+0000","last_fullsized":"2024-03-06T07:21:06.083076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":783017,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":191549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":197771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":198263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":192613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":191037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":191057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":507904,"data_stored":781369,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:10.424 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:21:10.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:21:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 Processor -- start 2024-03-06T07:21:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 -- start start 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07a4155bd0 0x7f07a4152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a40670c0 con 0x7f07a40c2aa0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a406b0f0 con 0x7f07a40c26c0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.567+0000 7f07a922a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a4156060 con 0x7f07a4155bd0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07a4155bd0 0x7f07a4152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07a4155bd0 0x7f07a4152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49290/0 (socket says 172.21.15.17:49290) 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 -- 172.21.15.17:0/1163829400 learned_addr learned my addr 172.21.15.17:0/1163829400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 -- 172.21.15.17:0/1163829400 >> v1:172.21.15.163:6789/0 conn(0x7f07a40c26c0 legacy=0x7f07a406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 -- 172.21.15.17:0/1163829400 >> v1:172.21.15.17:6789/0 conn(0x7f07a40c2aa0 legacy=0x7f07a40c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a1d9b640 1 -- 172.21.15.17:0/1163829400 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869818661 0 0) 0x7f07a406b0f0 con 0x7f07a40c26c0 2024-03-06T07:21:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.568+0000 7f07a359e640 1 -- 172.21.15.17:0/1163829400 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a406b3d0 con 0x7f07a4155bd0 2024-03-06T07:21:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a359e640 1 --2- 172.21.15.17:0/1163829400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a4152350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f079c009e50 tx=0x7f079c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151a310300a70b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a1d9b640 1 -- 172.21.15.17:0/1163829400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c02ca80 con 0x7f07a4155bd0 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a1d9b640 1 -- 172.21.15.17:0/1163829400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c02cc20 con 0x7f07a4155bd0 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a1d9b640 1 -- 172.21.15.17:0/1163829400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0343f0 con 0x7f07a4155bd0 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 -- 172.21.15.17:0/1163829400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 msgr2=0x7f07a4152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 --2- 172.21.15.17:0/1163829400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a4152350 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f079c009e50 tx=0x7f079c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 -- 172.21.15.17:0/1163829400 shutdown_connections 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 --2- 172.21.15.17:0/1163829400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a4152350 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 -- 172.21.15.17:0/1163829400 >> 172.21.15.17:0/1163829400 conn(0x7f07a40c3b40 msgr2=0x7f07a40c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 -- 172.21.15.17:0/1163829400 shutdown_connections 2024-03-06T07:21:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.569+0000 7f07a922a640 1 -- 172.21.15.17:0/1163829400 wait complete. 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 Processor -- start 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 -- start start 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07a40c26c0 0x7f07a414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a40c2aa0 0x7f07a40ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4066d00 con 0x7f07a4155bd0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a406b740 con 0x7f07a40c2aa0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.571+0000 7f07a922a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4066e80 con 0x7f07a40c26c0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49306/0 (socket says 172.21.15.17:49306) 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 -- 172.21.15.17:0/3077284748 learned_addr learned my addr 172.21.15.17:0/3077284748 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a259c640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a40c2aa0 0x7f07a40ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a2d9d640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07a40c26c0 0x7f07a414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 -- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07a40c26c0 msgr2=0x7f07a414cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07a40c26c0 0x7f07a414cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 -- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a40c2aa0 msgr2=0x7f07a40ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a40c2aa0 0x7f07a40ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 -- 172.21.15.17:0/3077284748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a414d770 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a359e640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f079c009f70 tx=0x7f079c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c043020 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c03e450 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a41eaac0 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c009510 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.572+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a414d010 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.573+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a406f260 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.574+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f079c046020 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.574+0000 7f078b7fe640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 0x7f079006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.574+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f079c0acaa0 con 0x7f07a4155bd0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.574+0000 7f07a2d9d640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 0x7f079006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.575+0000 7f07a2d9d640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 0x7f079006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ac6c0 tx=0x7f0794009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.578+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07a406f260 con 0x7f07a4155bd0 2024-03-06T07:21:10.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.830+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f07a404f330 con 0x7f07a4155bd0 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.831+0000 7f078b7fe640 1 -- 172.21.15.17:0/3077284748 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f079c04d030 con 0x7f07a4155bd0 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:10.833 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:21:10.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 msgr2=0x7f079006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 0x7f079006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ac6c0 tx=0x7f0794009040 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 msgr2=0x7f07a40ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f079c009f70 tx=0x7f079c004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 shutdown_connections 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079006b4d0 0x7f079006d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07a4155bd0 0x7f07a40ab6f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07a40c2aa0 0x7f07a40ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 --2- 172.21.15.17:0/3077284748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07a40c26c0 0x7f07a414cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 >> 172.21.15.17:0/3077284748 conn(0x7f07a40c3b40 msgr2=0x7f07a4150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 shutdown_connections 2024-03-06T07:21:10.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.834+0000 7f07a922a640 1 -- 172.21.15.17:0/3077284748 wait complete. 2024-03-06T07:21:10.846 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:10.910 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:21:10.908+0000 7f5ce2a71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:10.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.946+0000 7fdc9cd32640 1 Processor -- start 2024-03-06T07:21:10.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.946+0000 7fdc9cd32640 1 -- start start 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9cd32640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdc9814f250 0x7fdc980c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9cd32640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9cd32640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc9806c150 con 0x7fdc98155c20 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9cd32640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9806c480 con 0x7fdc980c7350 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9cd32640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9814f630 con 0x7fdc9814f250 2024-03-06T07:21:10.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc96d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9659c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdc9814f250 0x7fdc980c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc96d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49318/0 (socket says 172.21.15.17:49318) 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc95d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdc98155c20 0x7fdc9806c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60602/0 (socket says 172.21.15.17:60602) 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc96d9d640 1 -- 172.21.15.17:0/2549557381 learned_addr learned my addr 172.21.15.17:0/2549557381 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9659c640 1 -- 172.21.15.17:0/2549557381 >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 msgr2=0x7fdc980c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9659c640 1 --2- 172.21.15.17:0/2549557381 >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9659c640 1 -- 172.21.15.17:0/2549557381 >> v1:172.21.15.17:6789/0 conn(0x7fdc98155c20 legacy=0x7fdc9806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc9659c640 1 -- 172.21.15.17:0/2549557381 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980c5660 con 0x7fdc9814f250 2024-03-06T07:21:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.947+0000 7fdc96d9d640 1 --2- 172.21.15.17:0/2549557381 >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:10.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.948+0000 7fdc9659c640 1 --2- 172.21.15.17:0/2549557381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc980c8840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ad2d0 tx=0x7fdc8802b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2955787f2f5e7f68 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.948+0000 7fdc9559a640 1 -- 172.21.15.17:0/2549557381 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88038070 con 0x7fdc9814f250 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.948+0000 7fdc9559a640 1 -- 172.21.15.17:0/2549557381 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc88031710 con 0x7fdc9814f250 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.948+0000 7fdc9559a640 1 -- 172.21.15.17:0/2549557381 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88031ae0 con 0x7fdc9814f250 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2549557381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 msgr2=0x7fdc980c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2549557381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc980c8840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ad2d0 tx=0x7fdc8802b7b0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2549557381 shutdown_connections 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2549557381 >> v2:172.21.15.17:3300/0 conn(0x7fdc980c7350 0x7fdc980c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2549557381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc980c8840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2549557381 >> 172.21.15.17:0/2549557381 conn(0x7fdc980ac4d0 msgr2=0x7fdc980ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2549557381 shutdown_connections 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.949+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2549557381 wait complete. 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.950+0000 7fdc9cd32640 1 Processor -- start 2024-03-06T07:21:10.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.950+0000 7fdc9cd32640 1 -- start start 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc980c7350 0x7fdc981476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc98154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc980c5c20 con 0x7fdc98155c20 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9806daf0 con 0x7fdc9814f250 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981560a0 con 0x7fdc980c7350 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49332/0 (socket says 172.21.15.17:49332) 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 -- 172.21.15.17:0/2701461900 learned_addr learned my addr 172.21.15.17:0/2701461900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9659c640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc980c7350 0x7fdc981476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc95d9b640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc98154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 -- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc980c7350 msgr2=0x7fdc981476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc980c7350 0x7fdc981476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 -- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 msgr2=0x7fdc98154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc98154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 -- 172.21.15.17:0/2701461900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980c1a10 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc96d9d640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000efc0 tx=0x7fdc9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9000cbe0 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9000cd80 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc981ebe00 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.951+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90003bb0 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.952+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980c1fb0 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.952+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9804f330 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.953+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc9001b030 con 0x7fdc98155c20 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.953+0000 7fdc86ffd640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 0x7fdc7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.953+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdc9008a5f0 con 0x7fdc98155c20 2024-03-06T07:21:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.953+0000 7fdc9659c640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 0x7fdc7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.954+0000 7fdc9659c640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 0x7fdc7806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc88035000 tx=0x7fdc88008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:10.958+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdc90022020 con 0x7fdc98155c20 2024-03-06T07:21:11.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.174+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc9806d860 con 0x7fdc98155c20 2024-03-06T07:21:11.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.174+0000 7fdc86ffd640 1 -- 172.21.15.17:0/2701461900 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fdc90010040 con 0x7fdc98155c20 2024-03-06T07:21:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:11.177 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:08.436696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2927346551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2927346551},{"type":"v1","addr":"172.21.15.163:6837","nonce":2927346551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5289,"qdb_cluster":[5289]},"id":2}]} 2024-03-06T07:21:11.177 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 msgr2=0x7fdc7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 0x7fdc7806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc88035000 tx=0x7fdc88008030 comp rx=0 tx=0).stop 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 msgr2=0x7fdc980c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000efc0 tx=0x7fdc9000c490 comp rx=0 tx=0).stop 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 shutdown_connections 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdc7806b4d0 0x7fdc7806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc98155c20 0x7fdc980c3540 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc9814f250 0x7fdc98154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 --2- 172.21.15.17:0/2701461900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc980c7350 0x7fdc981476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 >> 172.21.15.17:0/2701461900 conn(0x7fdc980ac4d0 msgr2=0x7fdc9814ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 shutdown_connections 2024-03-06T07:21:11.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.176+0000 7fdc9cd32640 1 -- 172.21.15.17:0/2701461900 wait complete. 2024-03-06T07:21:11.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:05.446329+0000', 'modified': '2024-03-06T07:21:08.436696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5289}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5289': {'gid': 5289, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2927346551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2927346551}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2927346551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5289, 'qdb_cluster': [5289]} 2024-03-06T07:21:11.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:11.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:21:11.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.700+0000 7f61a8581640 1 Processor -- start 2024-03-06T07:21:11.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.700+0000 7f61a8581640 1 -- start start 2024-03-06T07:21:11.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a8581640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61a00c8950 0x7f61a00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a8581640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a01558d0 0x7f61a0155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a8581640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a0155130 con 0x7f61a00c69f0 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a8581640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a0155440 con 0x7f61a00c8950 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a8581640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00ad480 con 0x7f61a01558d0 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a5b1c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61a00c8950 0x7f61a00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a5b1c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61a00c8950 0x7f61a00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49344/0 (socket says 172.21.15.17:49344) 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a5b1c640 1 -- 172.21.15.17:0/875551452 learned_addr learned my addr 172.21.15.17:0/875551452 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.701+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a01558d0 0x7f61a0155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a5b1c640 1 -- 172.21.15.17:0/875551452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a01558d0 msgr2=0x7f61a0155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a01558d0 0x7f61a0155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a5b1c640 1 -- 172.21.15.17:0/875551452 >> v1:172.21.15.163:6789/0 conn(0x7f61a00c69f0 legacy=0x7f61a00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a5b1c640 1 -- 172.21.15.17:0/875551452 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00c5d50 con 0x7f61a00c8950 2024-03-06T07:21:11.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a531b640 1 -- 172.21.15.17:0/875551452 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457160496 0 0) 0x7f619c001110 con 0x7f61a00c69f0 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.702+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a00c8950 0x7f61a00c6dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6190009870 tx=0x7f619002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc6f85d789c53bb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a531b640 1 -- 172.21.15.17:0/875551452 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190035020 con 0x7f61a00c8950 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a531b640 1 -- 172.21.15.17:0/875551452 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6190031640 con 0x7f61a00c8950 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a531b640 1 -- 172.21.15.17:0/875551452 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619002d040 con 0x7f61a00c8950 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 -- 172.21.15.17:0/875551452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a00c8950 msgr2=0x7f61a00c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a00c8950 0x7f61a00c6dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6190009870 tx=0x7f619002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 -- 172.21.15.17:0/875551452 shutdown_connections 2024-03-06T07:21:11.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a01558d0 0x7f61a0155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 --2- 172.21.15.17:0/875551452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a00c8950 0x7f61a00c6dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 -- 172.21.15.17:0/875551452 >> 172.21.15.17:0/875551452 conn(0x7f61a00cae70 msgr2=0x7f61a00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 -- 172.21.15.17:0/875551452 shutdown_connections 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.703+0000 7f61a8581640 1 -- 172.21.15.17:0/875551452 wait complete. 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.704+0000 7f61a8581640 1 Processor -- start 2024-03-06T07:21:11.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.704+0000 7f61a8581640 1 -- start start 2024-03-06T07:21:11.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a00c69f0 0x7f61a014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61a00c8950 0x7f61a0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152a20 con 0x7f61a01558d0 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00c6310 con 0x7f61a00c8950 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a8581640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0152ba0 con 0x7f61a00c69f0 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49350/0 (socket says 172.21.15.17:49350) 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1546130681 learned_addr learned my addr 172.21.15.17:0/1546130681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61a00c8950 0x7f61a0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a631d640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a00c69f0 0x7f61a014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a00c69f0 msgr2=0x7f61a014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a00c69f0 0x7f61a014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61a00c8950 msgr2=0x7f61a0153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61a00c8950 0x7f61a0153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.705+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1546130681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a01ea5c0 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f619800efc0 tx=0x7f619800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f6196ffd640 1 -- 172.21.15.17:0/1546130681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198018020 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f6196ffd640 1 -- 172.21.15.17:0/1546130681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198014480 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a0154af0 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f6196ffd640 1 -- 172.21.15.17:0/1546130681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198014870 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.706+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a0158bc0 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.708+0000 7f6196ffd640 1 -- 172.21.15.17:0/1546130681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6198005d10 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.708+0000 7f6196ffd640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 0x7f618406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.708+0000 7f6196ffd640 1 -- 172.21.15.17:0/1546130681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f619808afc0 con 0x7f61a01558d0 2024-03-06T07:21:11.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.708+0000 7f61a631d640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 0x7f618406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.709+0000 7f61a631d640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 0x7f618406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f619c000f60 tx=0x7f619c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.709+0000 7f61ab16c740 1 Processor -- start 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.709+0000 7f61ab16c740 1 -- start start 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea012a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e9f796b0 con 0x55d2ea0134b0 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e9f22a60 con 0x55d2ea014f50 2024-03-06T07:21:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2e9f75750 con 0x55d2ea015330 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a631d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea012a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49362/0 (socket says 172.21.15.17:49362) 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1039606070 learned_addr learned my addr 172.21.15.17:0/1039606070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 msgr2=0x55d2ea012f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 msgr2=0x55d2ea012a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea012a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1039606070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e9f40f30 con 0x55d2ea0134b0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6198061820 tx=0x7f61980597b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ff971a9035ef0e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f6183fff640 1 -- 172.21.15.17:0/1039606070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198087070 con 0x55d2ea0134b0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f6183fff640 1 -- 172.21.15.17:0/1039606070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61980579f0 con 0x55d2ea0134b0 2024-03-06T07:21:11.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.710+0000 7f6183fff640 1 -- 172.21.15.17:0/1039606070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198071070 con 0x55d2ea0134b0 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 -- 172.21.15.17:0/1039606070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 msgr2=0x55d2ea00f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6198061820 tx=0x7f61980597b0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 -- 172.21.15.17:0/1039606070 shutdown_connections 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea0134b0 0x55d2ea00f680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1039606070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea012a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 -- 172.21.15.17:0/1039606070 >> 172.21.15.17:0/1039606070 conn(0x55d2ea0117d0 msgr2=0x55d2ea0363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 -- 172.21.15.17:0/1039606070 shutdown_connections 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.711+0000 7f61ab16c740 1 -- 172.21.15.17:0/1039606070 wait complete. 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.712+0000 7f61ab16c740 1 -- start start 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.712+0000 7f61ab16c740 1 Processor -- start 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea0618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea061e10 0x55d2ea030a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ea037230 con 0x55d2ea014f50 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e9f35ea0 con 0x55d2ea015330 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2ea036c70 con 0x55d2ea061e10 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 10 client.0 _ll_register_callbacks cb 0x55d2e9f35870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49378/0 (socket says 172.21.15.17:49378) 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 -- 172.21.15.17:0/248930028 learned_addr learned my addr 172.21.15.17:0/248930028 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea0618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea061e10 0x55d2ea030a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 -- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea061e10 msgr2=0x55d2ea030a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea061e10 0x55d2ea030a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 -- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 msgr2=0x55d2ea0618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea0618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 -- 172.21.15.17:0/248930028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e9f36270 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61a631d640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f619c014700 tx=0x7f619c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d749ff7519295f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c025020 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f619c013700 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.713+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61980579f0 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f619c02f050 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 10 client.5448 fetch_fsmap learned FSMap version 17 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55d2e9c8e2d0 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61980579f0 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f619c032020 con 0x55d2ea014f50 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 10 client.5448 fetch_fsmap finished waiting for FSMap version 17 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 10 client.5448 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 20 client.5448 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea04db00 0x55d2ea0353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 4 client.5448 mds_command: new command op to 5289 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 --> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2e9f86bf0 con 0x55d2ea04db00 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea04db00 0x55d2ea0353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61817fa640 10 client.5448 ms_handle_connect on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.714+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea04db00 0x55d2ea0353b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e80541b7c5bb16 server_cookie=1a8725a253536556 in_seq=0 out_seq=0 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61817fa640 1 -- 172.21.15.17:0/248930028 <== mds.0 v2:172.21.15.163:6835/2927346551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d2e9f86bf0 con 0x55d2ea04db00 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61817fa640 10 client.5448 handle_command_reply: tid=0 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 client.5448 shutdown 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 msgr2=0x55d2ea037a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f619c014700 tx=0x7f619c001080 comp rx=0 tx=0).stop 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 shutdown_connections 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea061e10 0x55d2ea030a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea04db00 0x55d2ea0353b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea0618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 --2- 172.21.15.17:0/248930028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea014f50 0x55d2ea037a20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61817fa640 0 client.5448 ms_handle_reset on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:11.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.715+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 >> 172.21.15.17:0/248930028 conn(0x55d2ea0134b0 msgr2=0x55d2ea00e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.716+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 shutdown_connections 2024-03-06T07:21:11.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.716+0000 7f61ab16c740 1 -- 172.21.15.17:0/248930028 wait complete. 2024-03-06T07:21:11.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.716+0000 7f61ab16c740 20 client.5448 trim_cache size 0 max 16384 2024-03-06T07:21:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.733+0000 7f61ab16c740 1 Processor -- start 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.733+0000 7f61ab16c740 1 -- start start 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea014f50 0x55d2ea05ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea012790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01f6ea0 con 0x55d2ea061e10 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01f62a0 con 0x55d2ea015330 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01f68a0 con 0x55d2ea014f50 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49390/0 (socket says 172.21.15.17:49390) 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1704515102 learned_addr learned my addr 172.21.15.17:0/1704515102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a631d640 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea014f50 0x55d2ea05ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea012790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea014f50 msgr2=0x55d2ea05ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea014f50 0x55d2ea05ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 msgr2=0x55d2ea012790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea012790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1704515102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e9d586b0 con 0x55d2ea061e10 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f619800ef90 tx=0x7f619805d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9621485aeda7e0d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61837fe640 1 -- 172.21.15.17:0/1704515102 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198087040 con 0x55d2ea061e10 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61837fe640 1 -- 172.21.15.17:0/1704515102 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61980579f0 con 0x55d2ea061e10 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.734+0000 7f61837fe640 1 -- 172.21.15.17:0/1704515102 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198071040 con 0x55d2ea061e10 2024-03-06T07:21:11.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 -- 172.21.15.17:0/1704515102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 msgr2=0x55d2ea012cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f619800ef90 tx=0x7f619805d000 comp rx=0 tx=0).stop 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 -- 172.21.15.17:0/1704515102 shutdown_connections 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea012cd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea015330 0x55d2ea012790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1704515102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea014f50 0x55d2ea05ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 -- 172.21.15.17:0/1704515102 >> 172.21.15.17:0/1704515102 conn(0x55d2ea0134b0 msgr2=0x55d2ea00e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 -- 172.21.15.17:0/1704515102 shutdown_connections 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.735+0000 7f61ab16c740 1 -- 172.21.15.17:0/1704515102 wait complete. 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 -- start start 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 Processor -- start 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ea055640 con 0x55d2ea061e10 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2ea054cd0 con 0x55d2ea014f50 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2ea037fb0 con 0x55d2ea015330 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.736+0000 7f61ab16c740 10 client.0 _ll_register_callbacks cb 0x7f619c001de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49404/0 (socket says 172.21.15.17:49404) 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1767933977 learned_addr learned my addr 172.21.15.17:0/1767933977 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a5b1c640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a631d640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 msgr2=0x55d2ea012b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 msgr2=0x55d2ea0c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea0c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 -- 172.21.15.17:0/1767933977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2ea035b20 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61a6b1e640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f619800c990 tx=0x7f6198013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab754d2e966caa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198087040 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6198079d50 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61980579f0 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6198080050 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61ab16c740 10 client.5454 fetch_fsmap learned FSMap version 17 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55d2ea04ec50 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61980579f0 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.737+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6198055440 con 0x55d2ea061e10 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 10 client.5454 fetch_fsmap finished waiting for FSMap version 17 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 10 client.5454 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 20 client.5454 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea0be0a0 0x55d2ea0be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 4 client.5454 mds_command: new command op to 5289 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 --> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d2e9f80f40 con 0x55d2ea0be0a0 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61a631d640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea0be0a0 0x55d2ea0be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61a631d640 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea0be0a0 0x55d2ea0be480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8432fe2e526cc7 server_cookie=2568d86e770fc51d in_seq=0 out_seq=0 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61817fa640 10 client.5454 ms_handle_connect on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61817fa640 1 -- 172.21.15.17:0/1767933977 <== mds.0 v2:172.21.15.163:6835/2927346551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d2e9f80f40 con 0x55d2ea0be0a0 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61817fa640 10 client.5454 handle_command_reply: tid=0 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 client.5454 shutdown 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 msgr2=0x55d2ea0355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f619800c990 tx=0x7f6198013000 comp rx=0 tx=0).stop 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 shutdown_connections 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d2ea061e10 0x55d2ea0355e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55d2ea015330 0x55d2ea012b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x55d2ea0be0a0 0x55d2ea0be480 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 --2- 172.21.15.17:0/1767933977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55d2ea014f50 0x55d2ea0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61817fa640 0 client.5454 ms_handle_reset on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.738+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 >> 172.21.15.17:0/1767933977 conn(0x55d2ea0134b0 msgr2=0x55d2ea061510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 shutdown_connections 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61ab16c740 1 -- 172.21.15.17:0/1767933977 wait complete. 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61ab16c740 20 client.5454 trim_cache size 0 max 16384 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5289, 2024-03-06T07:21:11.748 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 5.6528994460000002, 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 17, 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 39, 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 39, 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 12.691621792999999 2024-03-06T07:21:11.749 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 msgr2=0x7f618406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 0x7f618406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f619c000f60 tx=0x7f619c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 msgr2=0x7f61a00c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f619800efc0 tx=0x7f619800c490 comp rx=0 tx=0).stop 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 shutdown_connections 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f618406b4d0 0x7f618406d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a01558d0 0x7f61a00c4380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61a00c8950 0x7f61a0153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 --2- 172.21.15.17:0/1546130681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61a00c69f0 0x7f61a014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 >> 172.21.15.17:0/1546130681 conn(0x7f61a00cae70 msgr2=0x7f61a00acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 shutdown_connections 2024-03-06T07:21:11.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.739+0000 7f61a8581640 1 -- 172.21.15.17:0/1546130681 wait complete. 2024-03-06T07:21:11.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.851+0000 7f19771f6640 1 Processor -- start 2024-03-06T07:21:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.851+0000 7f19771f6640 1 -- start start 2024-03-06T07:21:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f19771f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1970072720 0x7f1970156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f19771f6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19701508c0 0x7f19700c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f19771f6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1970157340 con 0x7f197006f7b0 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f19771f6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19700c3740 con 0x7f19701508c0 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f19771f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1970156d80 con 0x7f1970072720 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19701508c0 0x7f19700c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19701508c0 0x7f19700c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49414/0 (socket says 172.21.15.17:49414) 2024-03-06T07:21:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 -- 172.21.15.17:0/3518449971 learned_addr learned my addr 172.21.15.17:0/3518449971 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1974f92640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1970072720 0x7f1970156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 -- 172.21.15.17:0/3518449971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1970072720 msgr2=0x7f1970156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1970072720 0x7f1970156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 -- 172.21.15.17:0/3518449971 >> v1:172.21.15.163:6789/0 conn(0x7f197006f7b0 legacy=0x7f19701501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.852+0000 7f1975793640 1 -- 172.21.15.17:0/3518449971 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970157060 con 0x7f19701508c0 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.853+0000 7f1975793640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19701508c0 0x7f19700c3bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1958009870 tx=0x7f195802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58940e6cde2f53f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.853+0000 7f196f7fe640 1 -- 172.21.15.17:0/3518449971 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958035020 con 0x7f19701508c0 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.853+0000 7f196f7fe640 1 -- 172.21.15.17:0/3518449971 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958031640 con 0x7f19701508c0 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.853+0000 7f196f7fe640 1 -- 172.21.15.17:0/3518449971 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195802d040 con 0x7f19701508c0 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 -- 172.21.15.17:0/3518449971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19701508c0 msgr2=0x7f19700c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19701508c0 0x7f19700c3bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1958009870 tx=0x7f195802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 -- 172.21.15.17:0/3518449971 shutdown_connections 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19701508c0 0x7f19700c3bd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 --2- 172.21.15.17:0/3518449971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1970072720 0x7f1970156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 -- 172.21.15.17:0/3518449971 >> 172.21.15.17:0/3518449971 conn(0x7f19700aa630 msgr2=0x7f19700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 -- 172.21.15.17:0/3518449971 shutdown_connections 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.854+0000 7f19771f6640 1 -- 172.21.15.17:0/3518449971 wait complete. 2024-03-06T07:21:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.855+0000 7f19771f6640 1 Processor -- start 2024-03-06T07:21:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.855+0000 7f19771f6640 1 -- start start 2024-03-06T07:21:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1970072720 0x7f19701479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19701508c0 0x7f1970146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970156af0 con 0x7f197006f7b0 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970156970 con 0x7f1970072720 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f19771f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1970072b00 con 0x7f19701508c0 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49424/0 (socket says 172.21.15.17:49424) 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 -- 172.21.15.17:0/3597728888 learned_addr learned my addr 172.21.15.17:0/3597728888 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:11.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f196ffff640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1970072720 0x7f19701479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1975793640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19701508c0 0x7f1970146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 -- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19701508c0 msgr2=0x7f1970146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19701508c0 0x7f1970146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 -- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1970072720 msgr2=0x7f19701479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1970072720 0x7f19701479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 -- 172.21.15.17:0/3597728888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970147740 con 0x7f197006f7b0 2024-03-06T07:21:11.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f1974f92640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9e0 tx=0x7f196800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968017070 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.856+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800c9d0 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.857+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700c8580 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.857+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800cdc0 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.857+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1970156130 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.858+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197004f330 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.858+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f196801a070 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.858+0000 7f196d7fa640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.859+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f196808a240 con 0x7f197006f7b0 2024-03-06T07:21:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.859+0000 7f196ffff640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:11.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.859+0000 7f196ffff640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19700a9e60 tx=0x7f1960009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:11.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:11.863+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1968058970 con 0x7f197006f7b0 2024-03-06T07:21:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.081+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1970154190 con 0x7f197006f7b0 2024-03-06T07:21:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.081+0000 7f196d7fa640 1 -- 172.21.15.17:0/3597728888 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f196805ca10 con 0x7f197006f7b0 2024-03-06T07:21:12.083 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:12.083 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:08.436696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2927346551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2927346551},{"type":"v1","addr":"172.21.15.163:6837","nonce":2927346551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5289,"qdb_cluster":[5289]},"id":2}]} 2024-03-06T07:21:12.084 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-06T07:21:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 msgr2=0x7f195006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19700a9e60 tx=0x7f1960009040 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 msgr2=0x7f19700c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f196800e9e0 tx=0x7f196800eea0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 shutdown_connections 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19701508c0 0x7f1970146a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1970072720 0x7f19701479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 --2- 172.21.15.17:0/3597728888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197006f7b0 0x7f19700c7860 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 >> 172.21.15.17:0/3597728888 conn(0x7f19700aa630 msgr2=0x7f19700c5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 shutdown_connections 2024-03-06T07:21:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.084+0000 7f19771f6640 1 -- 172.21.15.17:0/3597728888 wait complete. 2024-03-06T07:21:12.097 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 17, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:05.446329+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 17, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5289": { "addr": "172.21.15.163:6837/2927346551", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2927346551, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2927346551, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5289, "incarnation": 14, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:08.436696+0000", "qdb_cluster": [ 5289 ], "qdb_leader": 5289, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5289 } } } ], "standbys": [ { "addr": "172.21.15.17:6834/3720795082", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3720795082, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3720795082, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5233, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6836/1442407901", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1442407901, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1442407901, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5239, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1497992819", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1497992819, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1497992819, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5286, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:12.097 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:12.097 INFO:tasks.cephfs.test_failover:current = [{'gid': 5233, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5289, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2927346551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2927346551}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2927346551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5286, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1497992819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1497992819}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1497992819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 5239, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1442407901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1442407901}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1442407901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:12.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.a mds_join_fs cephfs 2024-03-06T07:21:12.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.200+0000 7f39c85f6640 1 Processor -- start 2024-03-06T07:21:12.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.200+0000 7f39c85f6640 1 -- start start 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c85f6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f39c00c7370 0x7f39c00c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c85f6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39c006c0b0 0x7f39c006c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c85f6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c0155260 con 0x7f39c01527f0 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c85f6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c00c8f70 con 0x7f39c006c0b0 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c85f6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c0155540 con 0x7f39c00c7370 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c6b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39c006c0b0 0x7f39c006c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c6b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39c006c0b0 0x7f39c006c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49440/0 (socket says 172.21.15.17:49440) 2024-03-06T07:21:12.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c6b93640 1 -- 172.21.15.17:0/2081496839 learned_addr learned my addr 172.21.15.17:0/2081496839 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c5b91640 1 --2- 172.21.15.17:0/2081496839 >> v2:172.21.15.163:3300/0 conn(0x7f39c00c7370 0x7f39c00c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c6b93640 1 -- 172.21.15.17:0/2081496839 >> v2:172.21.15.163:3300/0 conn(0x7f39c00c7370 msgr2=0x7f39c00c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.201+0000 7f39c6b93640 1 --2- 172.21.15.17:0/2081496839 >> v2:172.21.15.163:3300/0 conn(0x7f39c00c7370 0x7f39c00c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c6b93640 1 -- 172.21.15.17:0/2081496839 >> v1:172.21.15.163:6789/0 conn(0x7f39c01527f0 legacy=0x7f39c00c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c6b93640 1 -- 172.21.15.17:0/2081496839 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c0155820 con 0x7f39c006c0b0 2024-03-06T07:21:12.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c5390640 1 -- 172.21.15.17:0/2081496839 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108402932 0 0) 0x7f39c0155260 con 0x7f39c01527f0 2024-03-06T07:21:12.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c6b93640 1 --2- 172.21.15.17:0/2081496839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c006c4c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f39c00aa410 tx=0x7f39b002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45344b11c3b23e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c5390640 1 -- 172.21.15.17:0/2081496839 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b002c9f0 con 0x7f39c006c0b0 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c5390640 1 -- 172.21.15.17:0/2081496839 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b002cb90 con 0x7f39c006c0b0 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.202+0000 7f39c5390640 1 -- 172.21.15.17:0/2081496839 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b00343f0 con 0x7f39c006c0b0 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 -- 172.21.15.17:0/2081496839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 msgr2=0x7f39c006c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 --2- 172.21.15.17:0/2081496839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c006c4c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f39c00aa410 tx=0x7f39b002be50 comp rx=0 tx=0).stop 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 -- 172.21.15.17:0/2081496839 shutdown_connections 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 --2- 172.21.15.17:0/2081496839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c006c4c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 --2- 172.21.15.17:0/2081496839 >> v2:172.21.15.163:3300/0 conn(0x7f39c00c7370 0x7f39c00c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 -- 172.21.15.17:0/2081496839 >> 172.21.15.17:0/2081496839 conn(0x7f39c00ac4f0 msgr2=0x7f39c00ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 -- 172.21.15.17:0/2081496839 shutdown_connections 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.203+0000 7f39c85f6640 1 -- 172.21.15.17:0/2081496839 wait complete. 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.204+0000 7f39c85f6640 1 Processor -- start 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.204+0000 7f39c85f6640 1 -- start start 2024-03-06T07:21:12.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39c00c7370 0x7f39c0146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39c01527f0 0x7f39c01eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c006ca00 con 0x7f39c006c0b0 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00be1a0 con 0x7f39c00c7370 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c85f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c006cb80 con 0x7f39c01527f0 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49450/0 (socket says 172.21.15.17:49450) 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 -- 172.21.15.17:0/744013688 learned_addr learned my addr 172.21.15.17:0/744013688 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c5b91640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39c00c7370 0x7f39c0146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6b93640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39c01527f0 0x7f39c01eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 -- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39c01527f0 msgr2=0x7f39c01eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39c01527f0 0x7f39c01eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 -- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39c00c7370 msgr2=0x7f39c0146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39c00c7370 0x7f39c0146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 -- 172.21.15.17:0/744013688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00c1bd0 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39c6392640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c3e0 tx=0x7f39b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.205+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8017070 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.206+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80052d0 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.206+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c00c1eb0 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.206+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8003830 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.206+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c01473c0 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.206+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c004f330 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.207+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39b801a070 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.207+0000 7f39beffd640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 0x7f39ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.208+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f39b8089660 con 0x7f39c006c0b0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.208+0000 7f39c5b91640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 0x7f39ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.208+0000 7f39c5b91640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 0x7f39ac06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39c0154f00 tx=0x7f39b4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.212+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f39b8057d90 con 0x7f39c006c0b0 2024-03-06T07:21:12.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.394+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f39c00ac8d0 con 0x7f39c006c0b0 2024-03-06T07:21:12.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.397+0000 7f39beffd640 1 -- 172.21.15.17:0/744013688 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v12)=0 v12) v1 ==== 116+0+0 (secure 0 0 0) 0x7f39b8087e20 con 0x7f39c006c0b0 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 msgr2=0x7f39ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 0x7f39ac06d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39c0154f00 tx=0x7f39b4009190 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 msgr2=0x7f39c01eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c3e0 tx=0x7f39b800c8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 shutdown_connections 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f39ac06b4d0 0x7f39ac06d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f39c01527f0 0x7f39c01eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39c00c7370 0x7f39c0146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 --2- 172.21.15.17:0/744013688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39c006c0b0 0x7f39c01eeb20 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 >> 172.21.15.17:0/744013688 conn(0x7f39c00ac4f0 msgr2=0x7f39c0151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:12.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 shutdown_connections 2024-03-06T07:21:12.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.399+0000 7f39c85f6640 1 -- 172.21.15.17:0/744013688 wait complete. 2024-03-06T07:21:12.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.511+0000 7f62890b8640 1 Processor -- start 2024-03-06T07:21:12.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.511+0000 7f62890b8640 1 -- start start 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f62890b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f6284155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f62890b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840c73d0 con 0x7f628414f360 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f62890b8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62840be2e0 con 0x7f6284158b60 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f62890b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6284159700 con 0x7f62840c88c0 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f6282d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f628414f360 0x7f62840c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60616/0 (socket says 172.21.15.17:60616) 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f6282d9d640 1 -- 172.21.15.17:0/4192600303 learned_addr learned my addr 172.21.15.17:0/4192600303 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:12.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f628259c640 1 --2- 172.21.15.17:0/4192600303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f6284155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.512+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016430019 0 0) 0x7f62840c73d0 con 0x7f628414f360 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f626c003520 con 0x7f628414f360 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730619228 0 0) 0x7f62840be2e0 con 0x7f6284158b60 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62840c73d0 con 0x7f6284158b60 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f628259c640 1 -- 172.21.15.17:0/4192600303 >> v1:172.21.15.163:6789/0 conn(0x7f6284158b60 legacy=0x7f6284158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f628259c640 1 -- 172.21.15.17:0/4192600303 >> v1:172.21.15.17:6789/0 conn(0x7f628414f360 legacy=0x7f62840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f628259c640 1 -- 172.21.15.17:0/4192600303 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840c7670 con 0x7f62840c88c0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.513+0000 7f628259c640 1 --2- 172.21.15.17:0/4192600303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f6284155c10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62780097d0 tx=0x7f627802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20eb0654482d7558 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278035020 con 0x7f62840c88c0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278031800 con 0x7f62840c88c0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f6281d9b640 1 -- 172.21.15.17:0/4192600303 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278031bb0 con 0x7f62840c88c0 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 -- 172.21.15.17:0/4192600303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 msgr2=0x7f6284155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 --2- 172.21.15.17:0/4192600303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f6284155c10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f62780097d0 tx=0x7f627802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 -- 172.21.15.17:0/4192600303 shutdown_connections 2024-03-06T07:21:12.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 --2- 172.21.15.17:0/4192600303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f6284155c10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 -- 172.21.15.17:0/4192600303 >> 172.21.15.17:0/4192600303 conn(0x7f62840ac570 msgr2=0x7f62840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:12.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 -- 172.21.15.17:0/4192600303 shutdown_connections 2024-03-06T07:21:12.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.514+0000 7f62890b8640 1 -- 172.21.15.17:0/4192600303 wait complete. 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.515+0000 7f62890b8640 1 Processor -- start 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 -- start start 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f62841571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6284158b60 0x7f62841eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840c7c30 con 0x7f628414f360 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62840c5a50 con 0x7f6284158b60 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f62890b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62840c5bd0 con 0x7f62840c88c0 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f628259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f628259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49460/0 (socket says 172.21.15.17:49460) 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f628259c640 1 -- 172.21.15.17:0/3160345155 learned_addr learned my addr 172.21.15.17:0/3160345155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f6282d9d640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f62841571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.516+0000 7f628359e640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6284158b60 0x7f62841eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 -- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 msgr2=0x7f62841571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f62841571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 -- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6284158b60 msgr2=0x7f62841eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6284158b60 0x7f62841eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 -- 172.21.15.17:0/3160345155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62841ebe10 con 0x7f628414f360 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f628259c640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f627802b8e0 tx=0x7f62780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278040070 con 0x7f628414f360 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278031e30 con 0x7f628414f360 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62840ac2e0 con 0x7f628414f360 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278035040 con 0x7f628414f360 2024-03-06T07:21:12.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.517+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284148a80 con 0x7f628414f360 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.518+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6284156040 con 0x7f628414f360 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.518+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6278038020 con 0x7f628414f360 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.519+0000 7f626b7fe640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.519+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62780abcf0 con 0x7f628414f360 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.519+0000 7f6282d9d640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:12.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.519+0000 7f6282d9d640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f627c000f60 tx=0x7f627c007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:12.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.523+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6284156040 con 0x7f628414f360 2024-03-06T07:21:12.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.741+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f628404f330 con 0x7f628414f360 2024-03-06T07:21:12.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.741+0000 7f626b7fe640 1 -- 172.21.15.17:0/3160345155 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f62780b0070 con 0x7f628414f360 2024-03-06T07:21:12.743 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:12.743 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:08.436696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5289},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5289":{"gid":5289,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/2927346551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2927346551},{"type":"v1","addr":"172.21.15.163:6837","nonce":2927346551}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5289,"qdb_cluster":[5289]},"id":2}]} 2024-03-06T07:21:12.743 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 msgr2=0x7f627006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f627c000f60 tx=0x7f627c007d70 comp rx=0 tx=0).stop 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 msgr2=0x7f62841eb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f627802b8e0 tx=0x7f62780046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 shutdown_connections 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f627006b4d0 0x7f627006d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6284158b60 0x7f62841eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f628414f360 0x7f62841eb390 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 --2- 172.21.15.17:0/3160345155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62840c88c0 0x7f62841571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:12.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 >> 172.21.15.17:0/3160345155 conn(0x7f62840ac570 msgr2=0x7f62840ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:12.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 shutdown_connections 2024-03-06T07:21:12.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:12.743+0000 7f62890b8640 1 -- 172.21.15.17:0/3160345155 wait complete. 2024-03-06T07:21:13.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:05.446329+0000', 'modified': '2024-03-06T07:21:08.436696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5289}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5289': {'gid': 5289, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2927346551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2927346551}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2927346551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5289, 'qdb_cluster': [5289]} 2024-03-06T07:21:13.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:13.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:21:13.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.262+0000 7f826ac76640 1 Processor -- start 2024-03-06T07:21:13.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.262+0000 7f826ac76640 1 -- start start 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f826ac76640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82640724d0 0x7f82641551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f826ac76640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82640a7ff0 0x7f8264155730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f826ac76640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640733f0 con 0x7f8264152b40 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f826ac76640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8264072d70 con 0x7f82640a7ff0 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f826ac76640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640c82c0 con 0x7f82640724d0 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f8269213640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82640a7ff0 0x7f8264155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f8269213640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82640a7ff0 0x7f8264155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49476/0 (socket says 172.21.15.17:49476) 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f8269213640 1 -- 172.21.15.17:0/740984283 learned_addr learned my addr 172.21.15.17:0/740984283 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f8268a12640 1 --1- 172.21.15.17:0/740984283 >> v1:172.21.15.17:6789/0 conn(0x7f8264152b40 0x7f82640c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60622/0 (socket says 172.21.15.17:60622) 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.263+0000 7f8263fff640 1 --2- 172.21.15.17:0/740984283 >> v2:172.21.15.163:3300/0 conn(0x7f82640724d0 0x7f82641551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f8269213640 1 -- 172.21.15.17:0/740984283 >> v2:172.21.15.163:3300/0 conn(0x7f82640724d0 msgr2=0x7f82641551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f8269213640 1 --2- 172.21.15.17:0/740984283 >> v2:172.21.15.163:3300/0 conn(0x7f82640724d0 0x7f82641551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f8269213640 1 -- 172.21.15.17:0/740984283 >> v1:172.21.15.17:6789/0 conn(0x7f8264152b40 legacy=0x7f82640c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f8269213640 1 -- 172.21.15.17:0/740984283 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826406fd30 con 0x7f82640a7ff0 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f8269213640 1 --2- 172.21.15.17:0/740984283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f8264155730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82640c5330 tx=0x7f825c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be11a63bf953858 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.264+0000 7f82637fe640 1 -- 172.21.15.17:0/740984283 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c02ca80 con 0x7f82640a7ff0 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f82637fe640 1 -- 172.21.15.17:0/740984283 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c02cc20 con 0x7f82640a7ff0 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f82637fe640 1 -- 172.21.15.17:0/740984283 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0343f0 con 0x7f82640a7ff0 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 -- 172.21.15.17:0/740984283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 msgr2=0x7f8264155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 --2- 172.21.15.17:0/740984283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f8264155730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f82640c5330 tx=0x7f825c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 -- 172.21.15.17:0/740984283 shutdown_connections 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 --2- 172.21.15.17:0/740984283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f8264155730 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 --2- 172.21.15.17:0/740984283 >> v2:172.21.15.163:3300/0 conn(0x7f82640724d0 0x7f82641551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 -- 172.21.15.17:0/740984283 >> 172.21.15.17:0/740984283 conn(0x7f826406b970 msgr2=0x7f82640aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 -- 172.21.15.17:0/740984283 shutdown_connections 2024-03-06T07:21:13.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.265+0000 7f826ac76640 1 -- 172.21.15.17:0/740984283 wait complete. 2024-03-06T07:21:13.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.266+0000 7f826ac76640 1 Processor -- start 2024-03-06T07:21:13.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.266+0000 7f826ac76640 1 -- start start 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82640724d0 0x7f826414edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8264152b40 0x7f82640c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8264155df0 con 0x7f82640a7ff0 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8264155c70 con 0x7f8264152b40 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f826ac76640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8264070280 con 0x7f82640724d0 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49486/0 (socket says 172.21.15.17:49486) 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 -- 172.21.15.17:0/2832952212 learned_addr learned my addr 172.21.15.17:0/2832952212 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8268a12640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82640724d0 0x7f826414edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8269213640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8264152b40 0x7f82640c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 -- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82640724d0 msgr2=0x7f826414edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82640724d0 0x7f826414edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 -- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8264152b40 msgr2=0x7f82640c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8264152b40 0x7f82640c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.267+0000 7f8263fff640 1 -- 172.21.15.17:0/2832952212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82641ea8d0 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f8263fff640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9b0 tx=0x7f825800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f82617fa640 1 -- 172.21.15.17:0/2832952212 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258017070 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f82617fa640 1 -- 172.21.15.17:0/2832952212 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825800c9a0 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826414e270 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f82617fa640 1 -- 172.21.15.17:0/2832952212 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825800cd90 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.268+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826414e830 con 0x7f82640a7ff0 2024-03-06T07:21:13.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.269+0000 7f82617fa640 1 -- 172.21.15.17:0/2832952212 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f825801a070 con 0x7f82640a7ff0 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.269+0000 7f82617fa640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 0x7f825006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.270+0000 7f82617fa640 1 -- 172.21.15.17:0/2832952212 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f825808a2b0 con 0x7f82640a7ff0 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.270+0000 7f8268a12640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 0x7f825006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.270+0000 7f8268a12640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 0x7f825006d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8264159670 tx=0x7f8254007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 Processor -- start 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 -- start start 2024-03-06T07:21:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e7da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7e4b0 0x564c87e7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87de46b0 con 0x564c87e80330 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87d8da60 con 0x564c87e7e4b0 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c87de0750 con 0x564c87e7ff50 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49498/0 (socket says 172.21.15.17:49498) 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f8263fff640 1 -- 172.21.15.17:0/3473902670 learned_addr learned my addr 172.21.15.17:0/3473902670 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f8268a12640 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e7da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.271+0000 7f8269213640 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7e4b0 0x564c87e7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 -- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 msgr2=0x564c87e7da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e7da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 -- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7e4b0 msgr2=0x564c87e7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7e4b0 0x564c87e7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 -- 172.21.15.17:0/3473902670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c87dabf30 con 0x564c87e80330 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f8263fff640 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f825800cfd0 tx=0x7f8258058820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5638a035e697d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f824a7fc640 1 -- 172.21.15.17:0/3473902670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258021040 con 0x564c87e80330 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f824a7fc640 1 -- 172.21.15.17:0/3473902670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825808a070 con 0x564c87e80330 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f824a7fc640 1 -- 172.21.15.17:0/3473902670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258054fb0 con 0x564c87e80330 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 -- 172.21.15.17:0/3473902670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 msgr2=0x564c87e7df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f825800cfd0 tx=0x7f8258058820 comp rx=0 tx=0).stop 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 -- 172.21.15.17:0/3473902670 shutdown_connections 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7e4b0 0x564c87e7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87e7df70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 --2- 172.21.15.17:0/3473902670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e7da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.272+0000 7f826d85e740 1 -- 172.21.15.17:0/3473902670 >> 172.21.15.17:0/3473902670 conn(0x564c87e7c7d0 msgr2=0x564c87ea13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.273+0000 7f826d85e740 1 -- 172.21.15.17:0/3473902670 shutdown_connections 2024-03-06T07:21:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.273+0000 7f826d85e740 1 -- 172.21.15.17:0/3473902670 wait complete. 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.273+0000 7f826d85e740 1 -- start start 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.273+0000 7f826d85e740 1 Processor -- start 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e9ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ea3a70 0x564c87ecfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87ea2230 con 0x564c87e80330 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87da0ea0 con 0x564c87ea3a70 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c87ea1c70 con 0x564c87e7ff50 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f826d85e740 10 client.0 _ll_register_callbacks cb 0x564c87da0870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49512/0 (socket says 172.21.15.17:49512) 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 -- 172.21.15.17:0/4188235078 learned_addr learned my addr 172.21.15.17:0/4188235078 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8268a12640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e9ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8269213640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ea3a70 0x564c87ecfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 -- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 msgr2=0x564c87e9ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e9ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 -- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ea3a70 msgr2=0x564c87ecfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ea3a70 0x564c87ecfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.274+0000 7f8263fff640 1 -- 172.21.15.17:0/4188235078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c87da1270 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f8263fff640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8258054d50 tx=0x7f8258000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67406b4ecf89e258 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258087030 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8258088270 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f825808a070 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8258021040 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 10 client.5487 fetch_fsmap learned FSMap version 17 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x564c87af92d0 con 0x564c87e80330 2024-03-06T07:21:13.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825808a070 con 0x564c87e80330 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8258024060 con 0x564c87e80330 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 10 client.5487 fetch_fsmap finished waiting for FSMap version 17 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 10 client.5487 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 20 client.5487 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87ed0210 0x564c87ed05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 4 client.5487 mds_command: new command op to 5289 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 --> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c87df1bf0 con 0x564c87ed0210 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.275+0000 7f8268a12640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87ed0210 0x564c87ed05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f82437fe640 10 client.5487 ms_handle_connect on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f8268a12640 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87ed0210 0x564c87ed05f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745a151a3ad868c1 server_cookie=8b43313ca3a08b61 in_seq=0 out_seq=0 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f82437fe640 1 -- 172.21.15.17:0/4188235078 <== mds.0 v2:172.21.15.163:6835/2927346551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c87df1bf0 con 0x564c87ed0210 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f82437fe640 10 client.5487 handle_command_reply: tid=0 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f826d85e740 1 client.5487 shutdown 2024-03-06T07:21:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 msgr2=0x564c87ecc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.276+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8258054d50 tx=0x7f8258000bc0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 shutdown_connections 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87ed0210 0x564c87ed05f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ea3a70 0x564c87ecfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ecc820 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f82437fe640 0 client.5487 ms_handle_reset on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 --2- 172.21.15.17:0/4188235078 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87e9ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 >> 172.21.15.17:0/4188235078 conn(0x564c87e7e4b0 msgr2=0x564c87e7f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 shutdown_connections 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 1 -- 172.21.15.17:0/4188235078 wait complete. 2024-03-06T07:21:13.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.277+0000 7f826d85e740 20 client.5487 trim_cache size 0 max 16384 2024-03-06T07:21:13.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.293+0000 7f826d85e740 1 Processor -- start 2024-03-06T07:21:13.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.293+0000 7f826d85e740 1 -- start start 2024-03-06T07:21:13.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87ea71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 0x564c87ec9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82641f6e30 con 0x564c87e80330 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82641f6380 con 0x564c87ee0370 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82641f6830 con 0x564c87e7ff50 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49514/0 (socket says 172.21.15.17:49514) 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 -- 172.21.15.17:0/3748635068 learned_addr learned my addr 172.21.15.17:0/3748635068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8268a12640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87ea71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8269213640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 0x564c87ec9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 -- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 msgr2=0x564c87ea71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87ea71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 -- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 msgr2=0x564c87ec9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 0x564c87ec9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 -- 172.21.15.17:0/3748635068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c87ed0960 con 0x564c87e80330 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8269213640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 0x564c87ec9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.294+0000 7f8263fff640 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f82580588d0 tx=0x7f825806e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c38704049c23537 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f8249ffb640 1 -- 172.21.15.17:0/3748635068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258087070 con 0x564c87e80330 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f8249ffb640 1 -- 172.21.15.17:0/3748635068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825808a080 con 0x564c87e80330 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f8249ffb640 1 -- 172.21.15.17:0/3748635068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580703c0 con 0x564c87e80330 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 -- 172.21.15.17:0/3748635068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 msgr2=0x564c87ea0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f82580588d0 tx=0x7f825806e070 comp rx=0 tx=0).stop 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 -- 172.21.15.17:0/3748635068 shutdown_connections 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87ee0370 0x564c87ec9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87ea0ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 --2- 172.21.15.17:0/3748635068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87e7ff50 0x564c87ea71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 -- 172.21.15.17:0/3748635068 >> 172.21.15.17:0/3748635068 conn(0x564c87e7e4b0 msgr2=0x564c87ea3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 -- 172.21.15.17:0/3748635068 shutdown_connections 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.295+0000 7f826d85e740 1 -- 172.21.15.17:0/3748635068 wait complete. 2024-03-06T07:21:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.296+0000 7f826d85e740 1 -- start start 2024-03-06T07:21:13.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.296+0000 7f826d85e740 1 Processor -- start 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7ff50 0x564c87edfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87ee0370 0x564c87f29300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87ea2230 con 0x564c87e80330 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c87ea1c70 con 0x564c87e7ff50 2024-03-06T07:21:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c87da0ea0 con 0x564c87ee0370 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 10 client.0 _ll_register_callbacks cb 0x7f82580582a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49522/0 (socket says 172.21.15.17:49522) 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 -- 172.21.15.17:0/2701652834 learned_addr learned my addr 172.21.15.17:0/2701652834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8268a12640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7ff50 0x564c87edfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8269213640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87ee0370 0x564c87f29300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 -- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87ee0370 msgr2=0x564c87f29300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87ee0370 0x564c87f29300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 -- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7ff50 msgr2=0x564c87edfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7ff50 0x564c87edfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 -- 172.21.15.17:0/2701652834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c87bc36b0 con 0x564c87e80330 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f8263fff640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8258054d50 tx=0x7f82580096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4680c46645f6cb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f82437fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580703c0 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8258059bc0 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f825808a080 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8258059dc0 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 10 client.5493 fetch_fsmap learned FSMap version 17 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.297+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x564c87bc36b0 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825808a080 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 17) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8258057d90 con 0x564c87e80330 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 10 client.5493 fetch_fsmap finished waiting for FSMap version 17 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 10 client.5493 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 20 client.5493 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:13.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87e799f0 0x564c87f29840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 4 client.5493 mds_command: new command op to 5289 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 --> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c87debf40 con 0x564c87e799f0 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f8268a12640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87e799f0 0x564c87f29840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f82437fe640 10 client.5493 ms_handle_connect on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f8268a12640 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87e799f0 0x564c87f29840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c22c77b6e23414e server_cookie=4f05fa4ce6d118ea in_seq=0 out_seq=0 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f82437fe640 1 -- 172.21.15.17:0/2701652834 <== mds.0 v2:172.21.15.163:6835/2927346551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564c87debf40 con 0x564c87e799f0 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f82437fe640 10 client.5493 handle_command_reply: tid=0 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.298+0000 7f826d85e740 1 client.5493 shutdown 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 msgr2=0x564c87f2f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8258054d50 tx=0x7f82580096f0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 shutdown_connections 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564c87ee0370 0x564c87f29300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c87e80330 0x564c87f2f6b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:6835/2927346551,v1:172.21.15.163:6837/2927346551] conn(0x564c87e799f0 0x564c87f29840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 --2- 172.21.15.17:0/2701652834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564c87e7ff50 0x564c87edfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f82437fe640 0 client.5493 ms_handle_reset on v2:172.21.15.163:6835/2927346551 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 >> 172.21.15.17:0/2701652834 conn(0x564c87e7e4b0 msgr2=0x564c87ea3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 shutdown_connections 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 1 -- 172.21.15.17:0/2701652834 wait complete. 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826d85e740 20 client.5493 trim_cache size 0 max 16384 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:13.307 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5289, 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 7.2131141029999997, 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 17, 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 39, 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 39, 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14.251835969 2024-03-06T07:21:13.308 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 msgr2=0x7f825006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 0x7f825006d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8264159670 tx=0x7f8254007e80 comp rx=0 tx=0).stop 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 msgr2=0x7f82641ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9b0 tx=0x7f825800ee70 comp rx=0 tx=0).stop 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 shutdown_connections 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f825006b4d0 0x7f825006d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8264152b40 0x7f82640c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82640a7ff0 0x7f82641ee800 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 --2- 172.21.15.17:0/2832952212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82640724d0 0x7f826414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:13.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 >> 172.21.15.17:0/2832952212 conn(0x7f826406b970 msgr2=0x7f82641514d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:13.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 shutdown_connections 2024-03-06T07:21:13.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:13.299+0000 7f826ac76640 1 -- 172.21.15.17:0/2832952212 wait complete. 2024-03-06T07:21:13.311 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 17, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:05.446329+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 17, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5289": { "addr": "172.21.15.163:6837/2927346551", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2927346551, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2927346551, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5289, "incarnation": 14, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:08.436696+0000", "qdb_cluster": [ 5289 ], "qdb_leader": 5289, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5289 } } } ], "standbys": [ { "addr": "172.21.15.17:6834/3720795082", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3720795082, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3720795082, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5233, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6836/1442407901", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1442407901, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1442407901, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5239, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1497992819", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1497992819, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1497992819, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5286, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:13.313 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': 2, 'name': 'a', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:13.313 INFO:tasks.cephfs.test_failover:current = [{'gid': 5233, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}, {'gid': 5289, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/2927346551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2927346551}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2927346551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5286, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1497992819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1497992819}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1497992819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 5239, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1442407901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1442407901}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1442407901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:13.313 DEBUG:tasks.cephfs.test_failover:2 != -1 2024-03-06T07:21:13.313 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:21:15.108 INFO:tasks.ceph.mds.b.smithi163.stderr: -1535> 2024-03-06T07:20:59.046+0000 7f5ce8a7d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:21:15.110 INFO:tasks.ceph.mds.b.smithi163.stderr: -226> 2024-03-06T07:21:10.908+0000 7f5ce2a71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:15.135 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:21:18.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:18.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.442+0000 7f4e1ccaa640 1 Processor -- start 2024-03-06T07:21:18.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.442+0000 7f4e1ccaa640 1 -- start start 2024-03-06T07:21:18.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1ccaa640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e180c88c0 0x7f4e18155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1ccaa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e18158b60 0x7f4e18158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1ccaa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e180c73d0 con 0x7f4e1814f360 2024-03-06T07:21:18.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1ccaa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e180c8dd0 con 0x7f4e18158b60 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1ccaa640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e180be2e0 con 0x7f4e180c88c0 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e16d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e18158b60 0x7f4e18158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.443+0000 7f4e1659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e1814f360 0x7f4e180c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60634/0 (socket says 172.21.15.17:60634) 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e1659c640 1 -- 172.21.15.17:0/2492403060 learned_addr learned my addr 172.21.15.17:0/2492403060 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e15d9b640 1 --2- 172.21.15.17:0/2492403060 >> v2:172.21.15.163:3300/0 conn(0x7f4e180c88c0 0x7f4e18155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e16d9d640 1 -- 172.21.15.17:0/2492403060 >> v2:172.21.15.163:3300/0 conn(0x7f4e180c88c0 msgr2=0x7f4e18155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e16d9d640 1 --2- 172.21.15.17:0/2492403060 >> v2:172.21.15.163:3300/0 conn(0x7f4e180c88c0 0x7f4e18155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e16d9d640 1 -- 172.21.15.17:0/2492403060 >> v1:172.21.15.17:6789/0 conn(0x7f4e1814f360 legacy=0x7f4e180c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e16d9d640 1 -- 172.21.15.17:0/2492403060 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180c7670 con 0x7f4e18158b60 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.444+0000 7f4e1559a640 1 -- 172.21.15.17:0/2492403060 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629434328 0 0) 0x7f4e180c73d0 con 0x7f4e1814f360 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e16d9d640 1 --2- 172.21.15.17:0/2492403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e18158b60 0x7f4e18158ff0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e180c84c0 tx=0x7f4e1002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a082a62940f011a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1559a640 1 -- 172.21.15.17:0/2492403060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e100315a0 con 0x7f4e18158b60 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1559a640 1 -- 172.21.15.17:0/2492403060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10037070 con 0x7f4e18158b60 2024-03-06T07:21:18.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1559a640 1 -- 172.21.15.17:0/2492403060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10031960 con 0x7f4e18158b60 2024-03-06T07:21:18.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2492403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e18158b60 msgr2=0x7f4e18158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2492403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e18158b60 0x7f4e18158ff0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e180c84c0 tx=0x7f4e1002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:18.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2492403060 shutdown_connections 2024-03-06T07:21:18.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2492403060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e18158b60 0x7f4e18158ff0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2492403060 >> v2:172.21.15.163:3300/0 conn(0x7f4e180c88c0 0x7f4e18155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2492403060 >> 172.21.15.17:0/2492403060 conn(0x7f4e180ac570 msgr2=0x7f4e180ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2492403060 shutdown_connections 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.445+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2492403060 wait complete. 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 Processor -- start 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 -- start start 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 0x7f4e181ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 0x7f4e180bf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e180c5a50 con 0x7f4e1814f360 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e180c7c30 con 0x7f4e18158b60 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.447+0000 7f4e1ccaa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e180c5bd0 con 0x7f4e180c88c0 2024-03-06T07:21:18.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49528/0 (socket says 172.21.15.17:49528) 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 -- 172.21.15.17:0/2509904687 learned_addr learned my addr 172.21.15.17:0/2509904687 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e16d9d640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 0x7f4e180bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e1659c640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 0x7f4e181ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 -- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 msgr2=0x7f4e181ee3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 0x7f4e181ee3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 -- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 msgr2=0x7f4e180bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 0x7f4e180bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 -- 172.21.15.17:0/2509904687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1814f7b0 con 0x7f4e1814f360 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e16d9d640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 0x7f4e180bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e1659c640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 0x7f4e181ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e15d9b640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000d5c0 tx=0x7f4e0000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000125f0 con 0x7f4e1814f360 2024-03-06T07:21:18.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e00019070 con 0x7f4e1814f360 2024-03-06T07:21:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e000129d0 con 0x7f4e1814f360 2024-03-06T07:21:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e181eee20 con 0x7f4e1814f360 2024-03-06T07:21:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.448+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1814c4a0 con 0x7f4e1814f360 2024-03-06T07:21:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.449+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1804f330 con 0x7f4e1814f360 2024-03-06T07:21:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.450+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e00012b70 con 0x7f4e1814f360 2024-03-06T07:21:18.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.450+0000 7f4e0effd640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:18.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.450+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4e0008a720 con 0x7f4e1814f360 2024-03-06T07:21:18.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.450+0000 7f4e1659c640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:18.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.451+0000 7f4e1659c640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 0x7f4df806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e080012e0 tx=0x7f4e08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:18.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.454+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e00059450 con 0x7f4e1814f360 2024-03-06T07:21:18.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.672+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e180c5240 con 0x7f4e1814f360 2024-03-06T07:21:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.672+0000 7f4e0effd640 1 -- 172.21.15.17:0/2509904687 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f4e00058d50 con 0x7f4e1814f360 2024-03-06T07:21:18.674 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:18.674 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:18.113495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:18.674 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 22 2024-03-06T07:21:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 msgr2=0x7f4df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 0x7f4df806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4e080012e0 tx=0x7f4e08009040 comp rx=0 tx=0).stop 2024-03-06T07:21:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 msgr2=0x7f4e180bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:18.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000d5c0 tx=0x7f4e0000da80 comp rx=0 tx=0).stop 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 shutdown_connections 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e18158b60 0x7f4e180bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e1814f360 0x7f4e180bef00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 --2- 172.21.15.17:0/2509904687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e180c88c0 0x7f4e181ee3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 >> 172.21.15.17:0/2509904687 conn(0x7f4e180ac570 msgr2=0x7f4e1814e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 shutdown_connections 2024-03-06T07:21:18.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:18.674+0000 7f4e1ccaa640 1 -- 172.21.15.17:0/2509904687 wait complete. 2024-03-06T07:21:19.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:05.446329+0000', 'modified': '2024-03-06T07:21:18.113495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5233': {'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5233, 'qdb_cluster': [5233]} 2024-03-06T07:21:19.093 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:19.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:21:19.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.195+0000 7f0b9ca64640 1 Processor -- start 2024-03-06T07:21:19.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.195+0000 7f0b9ca64640 1 -- start start 2024-03-06T07:21:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9ca64640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b9814d000 0x7f0b980ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9ca64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b98156960 0x7f0b9806bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9ca64640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b980aa380 con 0x7f0b981564d0 2024-03-06T07:21:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9ca64640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b9814d3e0 con 0x7f0b9814d000 2024-03-06T07:21:19.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9ca64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980aba80 con 0x7f0b98156960 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b95d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0b981564d0 0x7f0b980c3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60640/0 (socket says 172.21.15.17:60640) 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b95d9b640 1 -- 172.21.15.17:0/3131297640 learned_addr learned my addr 172.21.15.17:0/3131297640 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b98156960 0x7f0b9806bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9659c640 1 --2- 172.21.15.17:0/3131297640 >> v2:172.21.15.163:3300/0 conn(0x7f0b9814d000 0x7f0b980ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420597373 0 0) 0x7f0b980aa380 con 0x7f0b981564d0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b80003520 con 0x7f0b981564d0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1488681689 0 0) 0x7f0b80003520 con 0x7f0b981564d0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b98156960 msgr2=0x7f0b9806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b98156960 0x7f0b9806bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 >> v2:172.21.15.163:3300/0 conn(0x7f0b9814d000 msgr2=0x7f0b980ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 --2- 172.21.15.17:0/3131297640 >> v2:172.21.15.163:3300/0 conn(0x7f0b9814d000 0x7f0b980ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.197+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b980c64f0 con 0x7f0b981564d0 2024-03-06T07:21:19.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0b8c001670 con 0x7f0b981564d0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 >> v1:172.21.15.17:6789/0 conn(0x7f0b981564d0 legacy=0x7f0b980c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800039b0 0x7f0b80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80004800 0x7f0b80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9814d3e0 con 0x7f0b800051c0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980aba80 con 0x7f0b800039b0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8000bfe0 con 0x7f0b80004800 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b98156960 0x7f0b9806bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9659c640 1 --2- 172.21.15.17:0/3131297640 >> v2:172.21.15.163:3300/0 conn(0x7f0b9814d000 0x7f0b980ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80004800 0x7f0b80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b9659c640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800039b0 0x7f0b80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 -- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80004800 msgr2=0x7f0b80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b80004800 0x7f0b80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 -- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800039b0 msgr2=0x7f0b80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b800039b0 0x7f0b80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.198+0000 7f0b96d9d640 1 -- 172.21.15.17:0/3131297640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80003520 con 0x7f0b800051c0 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b8000b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b98156d40 tx=0x7f0b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa132e5cfa6e30d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90009d40 con 0x7f0b800051c0 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9002ccf0 con 0x7f0b800051c0 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b9559a640 1 -- 172.21.15.17:0/3131297640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003bb0 con 0x7f0b800051c0 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b9ca64640 1 -- 172.21.15.17:0/3131297640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 msgr2=0x7f0b8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/3131297640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b8000b9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b98156d40 tx=0x7f0b80005830 comp rx=0 tx=0).stop 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.199+0000 7f0b96d9d640 1 -- 172.21.15.17:0/3131297640 reap_dead start 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.200+0000 7f0b9ca64640 1 -- 172.21.15.17:0/3131297640 shutdown_connections 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.200+0000 7f0b9ca64640 1 -- 172.21.15.17:0/3131297640 >> 172.21.15.17:0/3131297640 conn(0x7f0b980c3bb0 msgr2=0x7f0b980c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.200+0000 7f0b9ca64640 1 -- 172.21.15.17:0/3131297640 shutdown_connections 2024-03-06T07:21:19.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.200+0000 7f0b9ca64640 1 -- 172.21.15.17:0/3131297640 wait complete. 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 Processor -- start 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 -- start start 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 0x7f0b98147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 0x7f0b9814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980abd40 con 0x7f0b800051c0 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c6ab0 con 0x7f0b98151e20 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.201+0000 7f0b9ca64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b980c3940 con 0x7f0b9814d750 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49540/0 (socket says 172.21.15.17:49540) 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 -- 172.21.15.17:0/1823403933 learned_addr learned my addr 172.21.15.17:0/1823403933 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 0x7f0b9814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 0x7f0b98147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 -- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 msgr2=0x7f0b98147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 0x7f0b98147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 -- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 msgr2=0x7f0b9814a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 0x7f0b9814a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 -- 172.21.15.17:0/1823403933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9814dbd0 con 0x7f0b800051c0 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 0x7f0b9814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 0x7f0b98147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9659c640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b88011cd0 tx=0x7f0b8800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b86ffd640 1 -- 172.21.15.17:0/1823403933 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8800fd20 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b86ffd640 1 -- 172.21.15.17:0/1823403933 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b88004020 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b981502e0 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b86ffd640 1 -- 172.21.15.17:0/1823403933 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b88004410 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.202+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b98147780 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.204+0000 7f0b86ffd640 1 -- 172.21.15.17:0/1823403933 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b88012040 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.204+0000 7f0b86ffd640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.204+0000 7f0b86ffd640 1 -- 172.21.15.17:0/1823403933 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0b8808a9d0 con 0x7f0b800051c0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.205+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.205+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 0x7f0b7806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b9814ca10 tx=0x7f0b8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.205+0000 7f0b9f651740 1 Processor -- start 2024-03-06T07:21:19.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 -- start start 2024-03-06T07:21:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a06010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a06550 0x55f126a4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f12696a6b0 con 0x55f126a40420 2024-03-06T07:21:19.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126913a60 con 0x55f126a4f800 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126966750 con 0x55f126a06550 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49554/0 (socket says 172.21.15.17:49554) 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.206+0000 7f0b96d9d640 1 -- 172.21.15.17:0/1893421176 learned_addr learned my addr 172.21.15.17:0/1893421176 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a06550 0x55f126a4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b9659c640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a06010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 -- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a06550 msgr2=0x55f126a4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a06550 0x55f126a4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 -- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 msgr2=0x55f126a06010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a06010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 -- 172.21.15.17:0/1893421176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f126931f30 con 0x55f126a40420 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b9659c640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a06010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0b9002c720 tx=0x7f0b9002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342003c1732dd333 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b77fff640 1 -- 172.21.15.17:0/1893421176 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9002caf0 con 0x55f126a40420 2024-03-06T07:21:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b77fff640 1 -- 172.21.15.17:0/1893421176 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90045070 con 0x55f126a40420 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.207+0000 7f0b77fff640 1 -- 172.21.15.17:0/1893421176 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003990 con 0x55f126a40420 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- 172.21.15.17:0/1893421176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 msgr2=0x55f126a00680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0b9002c720 tx=0x7f0b9002e070 comp rx=0 tx=0).stop 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- 172.21.15.17:0/1893421176 shutdown_connections 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a40420 0x55f126a00680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a06550 0x55f126a4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 --2- 172.21.15.17:0/1893421176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a06010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- 172.21.15.17:0/1893421176 >> 172.21.15.17:0/1893421176 conn(0x55f126a027d0 msgr2=0x55f126a273c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- 172.21.15.17:0/1893421176 shutdown_connections 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- 172.21.15.17:0/1893421176 wait complete. 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 -- start start 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.208+0000 7f0b9f651740 1 Processor -- start 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 0x55f126a260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a44590 con 0x55f126a4f800 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126926ea0 con 0x55f126a5a800 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126a27db0 con 0x55f126a40420 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b9f651740 10 client.0 _ll_register_callbacks cb 0x55f126926870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b95d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b95d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49568/0 (socket says 172.21.15.17:49568) 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.209+0000 7f0b95d9b640 1 -- 172.21.15.17:0/2499465250 learned_addr learned my addr 172.21.15.17:0/2499465250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 0x55f126a260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 -- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 msgr2=0x55f126a012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 -- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 msgr2=0x55f126a260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 0x55f126a260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 -- 172.21.15.17:0/2499465250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f126927270 con 0x55f126a4f800 2024-03-06T07:21:19.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 0x55f126a260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0157c0 tx=0x7f0b8c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd114491bfc2a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b757fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c015e00 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0b8c014ae0 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b90045070 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b8c014ce0 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b9f651740 10 client.5517 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f12667f2d0 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.210+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90045070 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0b8c02d460 con 0x55f126a4f800 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 10 client.5517 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 10 client.5517 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:19.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 20 client.5517 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a4b770 0x55f126a65e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 4 client.5517 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f126977bf0 con 0x55f126a4b770 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a4b770 0x55f126a65e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a4b770 0x55f126a65e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7ce17d017f9add server_cookie=884385a570eaa9a8 in_seq=0 out_seq=0 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.211+0000 7f0b757fa640 10 client.5517 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b757fa640 1 -- 172.21.15.17:0/2499465250 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f126977bf0 con 0x55f126a4b770 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b757fa640 10 client.5517 handle_command_reply: tid=0 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 client.5517 shutdown 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 msgr2=0x55f126a46c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0157c0 tx=0x7f0b8c001a40 comp rx=0 tx=0).stop 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 shutdown_connections 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a4b770 0x55f126a65e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a5a800 0x55f126a260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a4f800 0x55f126a46c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2499465250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b757fa640 0 client.5517 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 >> 172.21.15.17:0/2499465250 conn(0x55f126a06550 msgr2=0x55f126a275c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 shutdown_connections 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 1 -- 172.21.15.17:0/2499465250 wait complete. 2024-03-06T07:21:19.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.212+0000 7f0b9f651740 20 client.5517 trim_cache size 0 max 16384 2024-03-06T07:21:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 Processor -- start 2024-03-06T07:21:19.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 -- start start 2024-03-06T07:21:19.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a40420 0x55f126a251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a4f800 0x55f126a49600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b981f3180 con 0x55f126a5a800 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98072bd0 con 0x55f126a40420 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.229+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b98145230 con 0x55f126a4f800 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49578/0 (socket says 172.21.15.17:49578) 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 -- 172.21.15.17:0/16980898 learned_addr learned my addr 172.21.15.17:0/16980898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a4f800 0x55f126a49600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b9659c640 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a40420 0x55f126a251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 -- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a4f800 msgr2=0x55f126a49600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a4f800 0x55f126a49600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 -- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a40420 msgr2=0x55f126a251c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a40420 0x55f126a251c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 -- 172.21.15.17:0/16980898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1267496b0 con 0x55f126a5a800 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009cd0 tx=0x7f0b90030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642f5c5bb1f533f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b777fe640 1 -- 172.21.15.17:0/16980898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9002ccf0 con 0x55f126a5a800 2024-03-06T07:21:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b777fe640 1 -- 172.21.15.17:0/16980898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90045040 con 0x55f126a5a800 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.230+0000 7f0b777fe640 1 -- 172.21.15.17:0/16980898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003b90 con 0x55f126a5a800 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 -- 172.21.15.17:0/16980898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 msgr2=0x55f126a49b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009cd0 tx=0x7f0b90030000 comp rx=0 tx=0).stop 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 -- 172.21.15.17:0/16980898 shutdown_connections 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a49b40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a4f800 0x55f126a49600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 --2- 172.21.15.17:0/16980898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a40420 0x55f126a251c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 -- 172.21.15.17:0/16980898 >> 172.21.15.17:0/16980898 conn(0x55f126a06550 msgr2=0x55f126a010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 -- 172.21.15.17:0/16980898 shutdown_connections 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.231+0000 7f0b9f651740 1 -- 172.21.15.17:0/16980898 wait complete. 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.232+0000 7f0b9f651740 1 -- start start 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.232+0000 7f0b9f651740 1 Processor -- start 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.232+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.232+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a57f10 con 0x55f126a5a800 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f126a4b5f0 con 0x55f126a4f800 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f126a490f0 con 0x55f126a40420 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 10 client.0 _ll_register_callbacks cb 0x7f0b8c015430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49594/0 (socket says 172.21.15.17:49594) 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 -- 172.21.15.17:0/2322679253 learned_addr learned my addr 172.21.15.17:0/2322679253 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 -- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 msgr2=0x55f126a388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 -- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 msgr2=0x55f126a426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 -- 172.21.15.17:0/2322679253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f126a3f100 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b95d9b640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b96d9d640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e2b0 tx=0x7f0b90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cbef17204d8e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b757fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004230 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0b900043d0 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b90045040 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.233+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b90067050 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 10 client.5523 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f126a6b250 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b90045040 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0b9006a020 con 0x55f126a5a800 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 10 client.5523 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 10 client.5523 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 20 client.5523 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a43120 0x55f126ab7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 4 client.5523 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f126971f40 con 0x55f126a43120 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a43120 0x55f126ab7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b757fa640 10 client.5523 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9659c640 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a43120 0x55f126ab7330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52adaa24c482817 server_cookie=f1cf0503d56c4954 in_seq=0 out_seq=0 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b757fa640 1 -- 172.21.15.17:0/2322679253 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f126971f40 con 0x55f126a43120 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b757fa640 10 client.5523 handle_command_reply: tid=0 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.234+0000 7f0b9f651740 1 client.5523 shutdown 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 msgr2=0x55f126a42be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e2b0 tx=0x7f0b90004ab0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 shutdown_connections 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f126a43120 0x55f126ab7330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f126a5a800 0x55f126a42be0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f126a4f800 0x55f126a426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 --2- 172.21.15.17:0/2322679253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f126a40420 0x55f126a388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b757fa640 0 client.5523 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 >> 172.21.15.17:0/2322679253 conn(0x55f126a06550 msgr2=0x55f126a251c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 shutdown_connections 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 1 -- 172.21.15.17:0/2322679253 wait complete. 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9f651740 20 client.5523 trim_cache size 0 max 16384 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:19.243 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5233, 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.1282793980000001, 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 22, 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 40, 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 40, 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20.190197518000002 2024-03-06T07:21:19.244 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 msgr2=0x7f0b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 0x7f0b7806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b9814ca10 tx=0x7f0b8c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 msgr2=0x7f0b98145840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0b88011cd0 tx=0x7f0b8800f610 comp rx=0 tx=0).stop 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 shutdown_connections 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b7806b4d0 0x7f0b7806d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b98151e20 0x7f0b9814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b9814d750 0x7f0b98147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 --2- 172.21.15.17:0/1823403933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b800051c0 0x7f0b98145840 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 >> 172.21.15.17:0/1823403933 conn(0x7f0b980c3bb0 msgr2=0x7f0b9806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 shutdown_connections 2024-03-06T07:21:19.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.235+0000 7f0b9ca64640 1 -- 172.21.15.17:0/1823403933 wait complete. 2024-03-06T07:21:19.248 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:05.446329+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5233": { "addr": "172.21.15.17:6834/3720795082", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3720795082, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3720795082, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5233, "incarnation": 19, "join_fscid": 2, "name": "a", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:18.113495+0000", "qdb_cluster": [ 5233 ], "qdb_leader": 5233, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5233 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1442407901", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1442407901, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1442407901, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5239, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1497992819", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1497992819, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1497992819, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5286, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/871521754", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 871521754, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 871521754, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 20, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5431, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:19.249 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': 2, 'name': 'a', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:19.249 INFO:tasks.cephfs.test_failover:current = [{'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5431, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/871521754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 871521754}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 871521754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 20}, {'gid': 5286, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1497992819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1497992819}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1497992819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 5239, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1442407901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1442407901}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1442407901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:19.249 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:21:19.249 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:21:19.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 Processor -- start 2024-03-06T07:21:19.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 -- start start 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff45c0b0070 0x7ff45c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff45c06ca70 0x7ff45c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45c0b0450 con 0x7ff45c0c5380 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c06c020 con 0x7ff45c06ca70 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff46290d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45c06cf20 con 0x7ff45c0b0070 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.351+0000 7ff45b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff45c06ca70 0x7ff45c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff460eaa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff45c0c5380 0x7ff45c14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60648/0 (socket says 172.21.15.17:60648) 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff460eaa640 1 -- 172.21.15.17:0/1180331757 learned_addr learned my addr 172.21.15.17:0/1180331757 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:19.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff45c06ca70 0x7ff45c0c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49598/0 (socket says 172.21.15.17:49598) 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45bfff640 1 --2- 172.21.15.17:0/1180331757 >> v2:172.21.15.163:3300/0 conn(0x7ff45c0b0070 0x7ff45c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45b7fe640 1 -- 172.21.15.17:0/1180331757 >> v2:172.21.15.163:3300/0 conn(0x7ff45c0b0070 msgr2=0x7ff45c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45b7fe640 1 --2- 172.21.15.17:0/1180331757 >> v2:172.21.15.163:3300/0 conn(0x7ff45c0b0070 0x7ff45c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45b7fe640 1 -- 172.21.15.17:0/1180331757 >> v1:172.21.15.17:6789/0 conn(0x7ff45c0c5380 legacy=0x7ff45c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45b7fe640 1 -- 172.21.15.17:0/1180331757 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0c77f0 con 0x7ff45c06ca70 2024-03-06T07:21:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.352+0000 7ff45affd640 1 -- 172.21.15.17:0/1180331757 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173287403 0 0) 0x7ff454001490 con 0x7ff45c0c5380 2024-03-06T07:21:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.353+0000 7ff45b7fe640 1 --2- 172.21.15.17:0/1180331757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c06ca70 0x7ff45c0c8900 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0aa3d0 tx=0x7ff45002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c1b19e93e3324c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.353+0000 7ff45affd640 1 -- 172.21.15.17:0/1180331757 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4500315a0 con 0x7ff45c06ca70 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.353+0000 7ff45affd640 1 -- 172.21.15.17:0/1180331757 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff450037070 con 0x7ff45c06ca70 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.353+0000 7ff45affd640 1 -- 172.21.15.17:0/1180331757 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450031960 con 0x7ff45c06ca70 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 -- 172.21.15.17:0/1180331757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c06ca70 msgr2=0x7ff45c0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 --2- 172.21.15.17:0/1180331757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c06ca70 0x7ff45c0c8900 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0aa3d0 tx=0x7ff45002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 -- 172.21.15.17:0/1180331757 shutdown_connections 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 --2- 172.21.15.17:0/1180331757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c06ca70 0x7ff45c0c8900 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 --2- 172.21.15.17:0/1180331757 >> v2:172.21.15.163:3300/0 conn(0x7ff45c0b0070 0x7ff45c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 -- 172.21.15.17:0/1180331757 >> 172.21.15.17:0/1180331757 conn(0x7ff45c0ac590 msgr2=0x7ff45c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 -- 172.21.15.17:0/1180331757 shutdown_connections 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.354+0000 7ff46290d640 1 -- 172.21.15.17:0/1180331757 wait complete. 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.355+0000 7ff46290d640 1 Processor -- start 2024-03-06T07:21:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.355+0000 7ff46290d640 1 -- start start 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 0x7ff45c14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0b0070 0x7ff45c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c06c550 con 0x7ff45c0c5380 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c06c270 con 0x7ff45c06ca70 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff46290d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c06da80 con 0x7ff45c0b0070 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff460eaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff460eaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49612/0 (socket says 172.21.15.17:49612) 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff460eaa640 1 -- 172.21.15.17:0/4223112839 learned_addr learned my addr 172.21.15.17:0/4223112839 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff45b7fe640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0b0070 0x7ff45c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.356+0000 7ff45bfff640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 0x7ff45c14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 -- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0b0070 msgr2=0x7ff45c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0b0070 0x7ff45c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 -- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 msgr2=0x7ff45c14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 0x7ff45c14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 -- 172.21.15.17:0/4223112839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0c2710 con 0x7ff45c0c5380 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff45bfff640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 0x7ff45c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff460eaa640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff45400c820 tx=0x7ff45400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454018070 con 0x7ff45c0c5380 2024-03-06T07:21:19.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45400a5e0 con 0x7ff45c0c5380 2024-03-06T07:21:19.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c0c29f0 con 0x7ff45c0c5380 2024-03-06T07:21:19.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45400a9d0 con 0x7ff45c0c5380 2024-03-06T07:21:19.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.357+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c0c1fc0 con 0x7ff45c0c5380 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.358+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff45401b070 con 0x7ff45c0c5380 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.358+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c04f330 con 0x7ff45c0c5380 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.359+0000 7ff458ff9640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 0x7ff44806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.359+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff454089980 con 0x7ff45c0c5380 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.359+0000 7ff45bfff640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 0x7ff44806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.360+0000 7ff45bfff640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 0x7ff44806d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009b90 tx=0x7ff44c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.363+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff45c0b0450 con 0x7ff45c0c5380 2024-03-06T07:21:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.583+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7ff45c06d860 con 0x7ff45c0c5380 2024-03-06T07:21:19.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.586+0000 7ff458ff9640 1 -- 172.21.15.17:0/4223112839 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v13) v1 ==== 96+0+0 (secure 0 0 0) 0x7ff454057f40 con 0x7ff45c0c5380 2024-03-06T07:21:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 msgr2=0x7ff44806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 0x7ff44806d9c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009b90 tx=0x7ff44c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 msgr2=0x7ff45c1ee150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff45400c820 tx=0x7ff45400cce0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 shutdown_connections 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff44806b500 0x7ff44806d9c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff45c0c5380 0x7ff45c1ee150 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff45c0b0070 0x7ff45c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 --2- 172.21.15.17:0/4223112839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff45c06ca70 0x7ff45c14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 >> 172.21.15.17:0/4223112839 conn(0x7ff45c0ac590 msgr2=0x7ff45c0ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 shutdown_connections 2024-03-06T07:21:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.588+0000 7ff46290d640 1 -- 172.21.15.17:0/4223112839 wait complete. 2024-03-06T07:21:19.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.700+0000 7f25ec148640 1 Processor -- start 2024-03-06T07:21:19.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25ec148640 1 -- start start 2024-03-06T07:21:19.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25ec148640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25e40c88c0 0x7f25e4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25ec148640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40c73d0 con 0x7f25e414f360 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25ec148640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e40be2e0 con 0x7f25e4158b60 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25ec148640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e4159700 con 0x7f25e40c88c0 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25e96e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25e40c88c0 0x7f25e4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25e9ee4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25e414f360 0x7f25e40c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60652/0 (socket says 172.21.15.17:60652) 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.701+0000 7f25e9ee4640 1 -- 172.21.15.17:0/1995975628 learned_addr learned my addr 172.21.15.17:0/1995975628 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.702+0000 7f25e96e3640 1 -- 172.21.15.17:0/1995975628 >> v1:172.21.15.163:6789/0 conn(0x7f25e4158b60 legacy=0x7f25e4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.702+0000 7f25e96e3640 1 -- 172.21.15.17:0/1995975628 >> v1:172.21.15.17:6789/0 conn(0x7f25e414f360 legacy=0x7f25e40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.702+0000 7f25e8ee2640 1 -- 172.21.15.17:0/1995975628 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189353823 0 0) 0x7f25e40c73d0 con 0x7f25e414f360 2024-03-06T07:21:19.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.702+0000 7f25e96e3640 1 -- 172.21.15.17:0/1995975628 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40c7670 con 0x7f25e40c88c0 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25e96e3640 1 --2- 172.21.15.17:0/1995975628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e40c88c0 0x7f25e4155c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25d40097a0 tx=0x7f25d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7dc3346b4221db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25e8ee2640 1 -- 172.21.15.17:0/1995975628 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4035020 con 0x7f25e40c88c0 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25e8ee2640 1 -- 172.21.15.17:0/1995975628 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4031830 con 0x7f25e40c88c0 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25e8ee2640 1 -- 172.21.15.17:0/1995975628 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4031c00 con 0x7f25e40c88c0 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 -- 172.21.15.17:0/1995975628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e40c88c0 msgr2=0x7f25e4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 --2- 172.21.15.17:0/1995975628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e40c88c0 0x7f25e4155c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f25d40097a0 tx=0x7f25d402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 -- 172.21.15.17:0/1995975628 shutdown_connections 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 --2- 172.21.15.17:0/1995975628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e40c88c0 0x7f25e4155c10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 -- 172.21.15.17:0/1995975628 >> 172.21.15.17:0/1995975628 conn(0x7f25e40ac570 msgr2=0x7f25e40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 -- 172.21.15.17:0/1995975628 shutdown_connections 2024-03-06T07:21:19.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.703+0000 7f25ec148640 1 -- 172.21.15.17:0/1995975628 wait complete. 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 Processor -- start 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 -- start start 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 0x7f25e41eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 0x7f25e4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c7c30 con 0x7f25e414f360 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c5a50 con 0x7f25e4158b60 2024-03-06T07:21:19.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25ec148640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40c5bd0 con 0x7f25e40c88c0 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25e96e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25e96e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49622/0 (socket says 172.21.15.17:49622) 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.705+0000 7f25e96e3640 1 -- 172.21.15.17:0/427642453 learned_addr learned my addr 172.21.15.17:0/427642453 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25ea6e5640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 0x7f25e4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 -- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 msgr2=0x7f25e41eb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e9ee4640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 0x7f25e41eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 0x7f25e41eb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 -- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 msgr2=0x7f25e4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 0x7f25e4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 -- 172.21.15.17:0/427642453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e414fe40 con 0x7f25e414f360 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25ea6e5640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 0x7f25e4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e9ee4640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 0x7f25e41eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25e96e3640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000c00 tx=0x7f25d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4035050 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4004230 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e4150120 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d403b740 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.706+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e41506e0 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.707+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e414ebc0 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.708+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25d4038020 con 0x7f25e414f360 2024-03-06T07:21:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.708+0000 7f25da7fc640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.708+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f25d40ac210 con 0x7f25e414f360 2024-03-06T07:21:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.708+0000 7f25e9ee4640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:19.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.709+0000 7f25e9ee4640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 0x7f25c806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000f60 tx=0x7f25e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:19.712+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25e414ebc0 con 0x7f25e414f360 2024-03-06T07:21:20.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.144+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e4069460 con 0x7f25e414f360 2024-03-06T07:21:20.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.144+0000 7f25da7fc640 1 -- 172.21.15.17:0/427642453 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f25e4069460 con 0x7f25e414f360 2024-03-06T07:21:20.147 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:20.147 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:18.113495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":2,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:20.147 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 22 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.146+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 msgr2=0x7f25c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.146+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 0x7f25c806d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25e0000f60 tx=0x7f25e0009040 comp rx=0 tx=0).stop 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.146+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 msgr2=0x7f25e414f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.146+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f25d4000c00 tx=0x7f25d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.146+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 shutdown_connections 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25c806b4d0 0x7f25c806d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25e4158b60 0x7f25e4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25e414f360 0x7f25e414f740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 --2- 172.21.15.17:0/427642453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25e40c88c0 0x7f25e41eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 >> 172.21.15.17:0/427642453 conn(0x7f25e40ac570 msgr2=0x7f25e414e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 shutdown_connections 2024-03-06T07:21:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.147+0000 7f25ec148640 1 -- 172.21.15.17:0/427642453 wait complete. 2024-03-06T07:21:20.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:05.446329+0000', 'modified': '2024-03-06T07:21:18.113495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5233': {'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5233, 'qdb_cluster': [5233]} 2024-03-06T07:21:20.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:20.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:21:20.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.664+0000 7f9f03423640 1 Processor -- start 2024-03-06T07:21:20.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.664+0000 7f9f03423640 1 -- start start 2024-03-06T07:21:20.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.665+0000 7f9f03423640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9efc0c69f0 0x7f9efc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.665+0000 7f9f03423640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9efc1558d0 0x7f9efc155d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.665+0000 7f9f03423640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc155130 con 0x7f9efc0c8950 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.665+0000 7f9f03423640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc155440 con 0x7f9efc1558d0 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.665+0000 7f9f03423640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9efc0ad480 con 0x7f9efc0c69f0 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f019c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9efc1558d0 0x7f9efc155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f009be640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9efc0c8950 0x7f9efc0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55506/0 (socket says 172.21.15.17:55506) 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f009be640 1 -- 172.21.15.17:0/3193974474 learned_addr learned my addr 172.21.15.17:0/3193974474 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f011bf640 1 --2- 172.21.15.17:0/3193974474 >> v2:172.21.15.163:3300/0 conn(0x7f9efc0c69f0 0x7f9efc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f019c0640 1 -- 172.21.15.17:0/3193974474 >> v2:172.21.15.163:3300/0 conn(0x7f9efc0c69f0 msgr2=0x7f9efc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f019c0640 1 --2- 172.21.15.17:0/3193974474 >> v2:172.21.15.163:3300/0 conn(0x7f9efc0c69f0 0x7f9efc0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f019c0640 1 -- 172.21.15.17:0/3193974474 >> v1:172.21.15.17:6789/0 conn(0x7f9efc0c8950 legacy=0x7f9efc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.666+0000 7f9f019c0640 1 -- 172.21.15.17:0/3193974474 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0c5d50 con 0x7f9efc1558d0 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f019c0640 1 --2- 172.21.15.17:0/3193974474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc155d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0aa430 tx=0x7f9ef402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286712836c455141 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9eeffff640 1 -- 172.21.15.17:0/3193974474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4039070 con 0x7f9efc1558d0 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9eeffff640 1 -- 172.21.15.17:0/3193974474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef402cbf0 con 0x7f9efc1558d0 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9eeffff640 1 -- 172.21.15.17:0/3193974474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4034440 con 0x7f9efc1558d0 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 -- 172.21.15.17:0/3193974474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 msgr2=0x7f9efc155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 --2- 172.21.15.17:0/3193974474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc155d00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0aa430 tx=0x7f9ef402be40 comp rx=0 tx=0).stop 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 -- 172.21.15.17:0/3193974474 shutdown_connections 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 --2- 172.21.15.17:0/3193974474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc155d00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 --2- 172.21.15.17:0/3193974474 >> v2:172.21.15.163:3300/0 conn(0x7f9efc0c69f0 0x7f9efc0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 -- 172.21.15.17:0/3193974474 >> 172.21.15.17:0/3193974474 conn(0x7f9efc0cae70 msgr2=0x7f9efc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 -- 172.21.15.17:0/3193974474 shutdown_connections 2024-03-06T07:21:20.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.667+0000 7f9f03423640 1 -- 172.21.15.17:0/3193974474 wait complete. 2024-03-06T07:21:20.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.668+0000 7f9f03423640 1 Processor -- start 2024-03-06T07:21:20.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.668+0000 7f9f03423640 1 -- start start 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 0x7f9efc14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 0x7f9efc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc152a20 con 0x7f9efc1558d0 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0c6310 con 0x7f9efc0c8950 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f03423640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc152ba0 con 0x7f9efc0c69f0 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f019c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f019c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52280/0 (socket says 172.21.15.17:52280) 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f019c0640 1 -- 172.21.15.17:0/2668604518 learned_addr learned my addr 172.21.15.17:0/2668604518 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f009be640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 0x7f9efc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.669+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 0x7f9efc14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 -- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 msgr2=0x7f9efc14cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 0x7f9efc14cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 -- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 msgr2=0x7f9efc158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 0x7f9efc158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 -- 172.21.15.17:0/2668604518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc1595a0 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f009be640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 0x7f9efc158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 0x7f9efc14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9ef402c450 tx=0x7f9ef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9eedffb640 1 -- 172.21.15.17:0/2668604518 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4039050 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9eedffb640 1 -- 172.21.15.17:0/2668604518 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4045070 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc0c3480 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9eedffb640 1 -- 172.21.15.17:0/2668604518 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef403e830 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.670+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc0c3a40 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.672+0000 7f9eedffb640 1 -- 172.21.15.17:0/2668604518 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ef403e9d0 con 0x7f9efc1558d0 2024-03-06T07:21:20.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.672+0000 7f9eedffb640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 0x7f9ee006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.672+0000 7f9eedffb640 1 -- 172.21.15.17:0/2668604518 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ef40ad330 con 0x7f9efc1558d0 2024-03-06T07:21:20.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.672+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 0x7f9ee006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.673+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 0x7f9ee006d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.673+0000 7f9f0600d740 1 Processor -- start 2024-03-06T07:21:20.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.673+0000 7f9f0600d740 1 -- start start 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768231330 0x55f76822ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f76822f4b0 0x55f76822b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7681956b0 con 0x55f768230f50 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76813ea60 con 0x55f76822f4b0 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f768191750 con 0x55f768231330 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52286/0 (socket says 172.21.15.17:52286) 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 -- 172.21.15.17:0/2340066472 learned_addr learned my addr 172.21.15.17:0/2340066472 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f009be640 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768231330 0x55f76822ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f76822f4b0 0x55f76822b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 -- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768231330 msgr2=0x55f76822ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768231330 0x55f76822ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 -- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f76822f4b0 msgr2=0x55f76822b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f76822f4b0 0x55f76822b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 -- 172.21.15.17:0/2340066472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76815cf30 con 0x55f768230f50 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000eba0 tx=0x7f9ef0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde000c183260c63 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9ee6ffd640 1 -- 172.21.15.17:0/2340066472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001a050 con 0x55f768230f50 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9ee6ffd640 1 -- 172.21.15.17:0/2340066472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0024070 con 0x55f768230f50 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.674+0000 7f9ee6ffd640 1 -- 172.21.15.17:0/2340066472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001f440 con 0x55f768230f50 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 -- 172.21.15.17:0/2340066472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 msgr2=0x55f76822ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000eba0 tx=0x7f9ef0011000 comp rx=0 tx=0).stop 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 -- 172.21.15.17:0/2340066472 shutdown_connections 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f76822f4b0 0x55f76822b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768231330 0x55f76822ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2340066472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76822ea30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 -- 172.21.15.17:0/2340066472 >> 172.21.15.17:0/2340066472 conn(0x55f76822d7d0 msgr2=0x55f7682523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 -- 172.21.15.17:0/2340066472 shutdown_connections 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.675+0000 7f9f0600d740 1 -- 172.21.15.17:0/2340066472 wait complete. 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.676+0000 7f9f0600d740 1 -- start start 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.676+0000 7f9f0600d740 1 Processor -- start 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f768270670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 0x55f768253840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f768253230 con 0x55f768230f50 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f768151ea0 con 0x55f768231330 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f768252c70 con 0x55f768255890 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f0600d740 10 client.0 _ll_register_callbacks cb 0x55f768151870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52290/0 (socket says 172.21.15.17:52290) 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 -- 172.21.15.17:0/2146888300 learned_addr learned my addr 172.21.15.17:0/2146888300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 0x55f768253840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f009be640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f768270670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 -- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 msgr2=0x55f768253840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 0x55f768253840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 -- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 msgr2=0x55f768270670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f768270670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f011bf640 1 -- 172.21.15.17:0/2146888300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f768152270 con 0x55f768230f50 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.677+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 0x55f768253840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000a230 tx=0x7f9ef0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdcf47ec6fd12a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:20.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef001fcc0 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ef0033480 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ef0024070 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef003b050 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 10 client.5556 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f767eaa2d0 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0024070 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f9ef003e020 con 0x55f768230f50 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 10 client.5556 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 10 client.5556 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 20 client.5556 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f768281350 0x55f76828c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 4 client.5556 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7681a2bf0 con 0x55f768281350 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.678+0000 7f9f009be640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f768281350 0x55f76828c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.679+0000 7f9edbfff640 10 client.5556 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.679+0000 7f9f009be640 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f768281350 0x55f76828c270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45651722ca4df0 server_cookie=55fbfec0ce38562 in_seq=0 out_seq=0 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9edbfff640 1 -- 172.21.15.17:0/2146888300 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f7681a2bf0 con 0x55f768281350 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9edbfff640 10 client.5556 handle_command_reply: tid=0 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 client.5556 shutdown 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 msgr2=0x55f76824e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000a230 tx=0x7f9ef0004730 comp rx=0 tx=0).stop 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 shutdown_connections 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f768281350 0x55f76828c270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768255890 0x55f768253840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9edbfff640 0 client.5556 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f768270670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2146888300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768230f50 0x55f76824e4f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 >> 172.21.15.17:0/2146888300 conn(0x55f76822f4b0 msgr2=0x55f76822f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 shutdown_connections 2024-03-06T07:21:20.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 1 -- 172.21.15.17:0/2146888300 wait complete. 2024-03-06T07:21:20.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.680+0000 7f9f0600d740 20 client.5556 trim_cache size 0 max 16384 2024-03-06T07:21:20.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.697+0000 7f9f0600d740 1 Processor -- start 2024-03-06T07:21:20.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.697+0000 7f9f0600d740 1 -- start start 2024-03-06T07:21:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.697+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.697+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768255890 0x55f76822bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f0600d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1f6e30 con 0x55f768231330 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1eed00 con 0x55f768255890 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1f6830 con 0x55f768230f50 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52300/0 (socket says 172.21.15.17:52300) 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 -- 172.21.15.17:0/749493064 learned_addr learned my addr 172.21.15.17:0/749493064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 -- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 msgr2=0x55f76822e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 -- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768255890 msgr2=0x55f76822bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768255890 0x55f76822bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 -- 172.21.15.17:0/749493064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f767f746b0 con 0x55f768231330 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9f009be640 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9efc152360 tx=0x7f9ef8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761dc9fd62fadd2e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9ee67fc640 1 -- 172.21.15.17:0/749493064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef801acb0 con 0x55f768231330 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9ee67fc640 1 -- 172.21.15.17:0/749493064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef801ae50 con 0x55f768231330 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.698+0000 7f9ee67fc640 1 -- 172.21.15.17:0/749493064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8007440 con 0x55f768231330 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 -- 172.21.15.17:0/749493064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 msgr2=0x55f76822b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9efc152360 tx=0x7f9ef8009040 comp rx=0 tx=0).stop 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 -- 172.21.15.17:0/749493064 shutdown_connections 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768255890 0x55f76822bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768231330 0x55f76822b870 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 --2- 172.21.15.17:0/749493064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 -- 172.21.15.17:0/749493064 >> 172.21.15.17:0/749493064 conn(0x55f76822f4b0 msgr2=0x55f76822f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 -- 172.21.15.17:0/749493064 shutdown_connections 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.699+0000 7f9f0600d740 1 -- 172.21.15.17:0/749493064 wait complete. 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.700+0000 7f9f0600d740 1 -- start start 2024-03-06T07:21:20.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.700+0000 7f9f0600d740 1 Processor -- start 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f76824eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f768255f30 con 0x55f768255890 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7682811d0 con 0x55f768231330 2024-03-06T07:21:20.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f768253d80 con 0x55f768230f50 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f0600d740 10 client.0 _ll_register_callbacks cb 0x7f9ef407b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52306/0 (socket says 172.21.15.17:52306) 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 -- 172.21.15.17:0/2524052592 learned_addr learned my addr 172.21.15.17:0/2524052592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f009be640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f76824eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 -- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 msgr2=0x55f76822ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 -- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 msgr2=0x55f76824eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f76824eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f019c0640 1 -- 172.21.15.17:0/2524052592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f768270070 con 0x55f768255890 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.701+0000 7f9f009be640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f76824eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f019c0640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9ef407b780 tx=0x7f9ef4032ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2147689f2b25e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:20.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4093070 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ef403ecc0 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ef801ae50 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ef407fbd0 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 10 client.5562 fetch_fsmap learned FSMap version 22 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f76822a9e0 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef801ae50 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f9ef40ab2f0 con 0x55f768255890 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 10 client.5562 fetch_fsmap finished waiting for FSMap version 22 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 10 client.5562 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 20 client.5562 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f7682e0050 0x55f7682e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 4 client.5562 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f76819cf40 con 0x55f7682e0050 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.702+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f7682e0050 0x55f7682e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9edbfff640 10 client.5562 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:20.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f011bf640 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f7682e0050 0x55f7682e0430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6738341018e4eebf server_cookie=fdf2f9e2be1f3b3e in_seq=0 out_seq=0 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9edbfff640 1 -- 172.21.15.17:0/2524052592 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f76819cf40 con 0x55f7682e0050 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9edbfff640 10 client.5562 handle_command_reply: tid=0 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 client.5562 shutdown 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 msgr2=0x55f7682d9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9ef407b780 tx=0x7f9ef4032ee0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 shutdown_connections 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x55f7682e0050 0x55f7682e0430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f768255890 0x55f7682d9cd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f768231330 0x55f76824eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 --2- 172.21.15.17:0/2524052592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f768230f50 0x55f76822ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 >> 172.21.15.17:0/2524052592 conn(0x55f76822f4b0 msgr2=0x55f76824e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9edbfff640 0 client.5562 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 shutdown_connections 2024-03-06T07:21:20.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 1 -- 172.21.15.17:0/2524052592 wait complete. 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.703+0000 7f9f0600d740 20 client.5562 trim_cache size 0 max 16384 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5233, 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 5.5967310159999997, 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 22, 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 40, 2024-03-06T07:21:20.711 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 40, 2024-03-06T07:21:20.712 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 21.658649543999999 2024-03-06T07:21:20.712 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:20.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 msgr2=0x7f9ee006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 0x7f9ee006d9c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0006fd0 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 msgr2=0x7f9efc14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9ef402c450 tx=0x7f9ef4004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 shutdown_connections 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ee006b500 0x7f9ee006d9c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9efc1558d0 0x7f9efc14ea90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9efc0c8950 0x7f9efc158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 --2- 172.21.15.17:0/2668604518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9efc0c69f0 0x7f9efc14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 >> 172.21.15.17:0/2668604518 conn(0x7f9efc0cae70 msgr2=0x7f9efc152e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 shutdown_connections 2024-03-06T07:21:20.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:20.704+0000 7f9f03423640 1 -- 172.21.15.17:0/2668604518 wait complete. 2024-03-06T07:21:20.717 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:05.446329+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5233": { "addr": "172.21.15.17:6834/3720795082", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3720795082, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3720795082, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5233, "incarnation": 19, "join_fscid": 2, "name": "a", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:18.113495+0000", "qdb_cluster": [ 5233 ], "qdb_leader": 5233, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5233 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1442407901", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1442407901, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1442407901, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5239, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1497992819", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1497992819, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1497992819, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5286, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/871521754", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 871521754, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 871521754, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 20, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5431, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:20.718 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:20.718 INFO:tasks.cephfs.test_failover:current = [{'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': 2, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5431, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/871521754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 871521754}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 871521754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 20}, {'gid': 5286, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1497992819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1497992819}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1497992819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 5239, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1442407901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1442407901}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1442407901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:20.718 DEBUG:tasks.cephfs.test_failover:-1 != 2 2024-03-06T07:21:20.718 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:21:20.912 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:21:20.910+0000 7f6151d5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:25.719 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:25.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.818+0000 7f0721c5c640 1 Processor -- start 2024-03-06T07:21:25.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f0721c5c640 1 -- start start 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f0721c5c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f071c0c88c0 0x7f071c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f0721c5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0c73d0 con 0x7f071c14f360 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f0721c5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0be2e0 con 0x7f071c158b60 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f0721c5c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c159700 con 0x7f071c0c88c0 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f071b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f071c14f360 0x7f071c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55512/0 (socket says 172.21.15.17:55512) 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.819+0000 7f071b7fe640 1 -- 172.21.15.17:0/1896712869 learned_addr learned my addr 172.21.15.17:0/1896712869 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1896712869 >> v2:172.21.15.163:3300/0 conn(0x7f071c0c88c0 0x7f071c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286294818 0 0) 0x7f071c0c73d0 con 0x7f071c14f360 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0708003520 con 0x7f071c14f360 2024-03-06T07:21:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016469081 0 0) 0x7f0708003520 con 0x7f071c14f360 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 >> v2:172.21.15.163:3300/0 conn(0x7f071c0c88c0 msgr2=0x7f071c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 --2- 172.21.15.17:0/1896712869 >> v2:172.21.15.163:3300/0 conn(0x7f071c0c88c0 0x7f071c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 >> v1:172.21.15.163:6789/0 conn(0x7f071c158b60 legacy=0x7f071c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0c7670 con 0x7f071c14f360 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f070c001670 con 0x7f071c14f360 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 >> v1:172.21.15.17:6789/0 conn(0x7f071c14f360 legacy=0x7f071c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 0x7f0708003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0708004800 0x7f0708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080051c0 0x7f070800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c159700 con 0x7f07080039b0 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070800c160 con 0x7f0708004800 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070800c360 con 0x7f07080051c0 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 0x7f0708003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0708004800 0x7f0708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071bfff640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080051c0 0x7f070800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 -- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080051c0 msgr2=0x7f070800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080051c0 0x7f070800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 -- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0708004800 msgr2=0x7f0708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0708004800 0x7f0708004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071b7fe640 1 -- 172.21.15.17:0/1896712869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708003520 con 0x7f07080039b0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f071bfff640 1 -- 172.21.15.17:0/1896712869 reap_dead start 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.820+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0708004800 0x7f0708004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.821+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 0x7f0708003dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c210 tx=0x7f0708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316f3093b0003b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.821+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c007d30 con 0x7f07080039b0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.821+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c030070 con 0x7f07080039b0 2024-03-06T07:21:25.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.821+0000 7f071affd640 1 -- 172.21.15.17:0/1896712869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c02cda0 con 0x7f07080039b0 2024-03-06T07:21:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 -- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 msgr2=0x7f0708003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 0x7f0708003dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f070c00c210 tx=0x7f0708005830 comp rx=0 tx=0).stop 2024-03-06T07:21:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 -- 172.21.15.17:0/1896712869 shutdown_connections 2024-03-06T07:21:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1896712869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07080039b0 0x7f0708003dd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 -- 172.21.15.17:0/1896712869 >> 172.21.15.17:0/1896712869 conn(0x7f071c0ac570 msgr2=0x7f071c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 -- 172.21.15.17:0/1896712869 shutdown_connections 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.822+0000 7f0721c5c640 1 -- 172.21.15.17:0/1896712869 wait complete. 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.823+0000 7f0721c5c640 1 Processor -- start 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.823+0000 7f0721c5c640 1 -- start start 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 0x7f071c0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 0x7f071c1ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0c7c30 con 0x7f071c0c1290 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c155c10 con 0x7f071c1eb4e0 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0721c5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c155d90 con 0x7f07080039b0 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52320/0 (socket says 172.21.15.17:52320) 2024-03-06T07:21:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 -- 172.21.15.17:0/1581848332 learned_addr learned my addr 172.21.15.17:0/1581848332 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 0x7f071c0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f071bfff640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 0x7f071c1ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 -- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 msgr2=0x7f071c0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 0x7f071c0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 -- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 msgr2=0x7f071c1ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 0x7f071c1ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f0712ffd640 1 -- 172.21.15.17:0/1581848332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c1ead40 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 0x7f071c0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.824+0000 7f071bfff640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 0x7f071c1ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0712ffd640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f07040114c0 tx=0x7f0704011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704015870 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704015a10 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704015dc0 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c0aead0 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.825+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1eafc0 con 0x7f071c0c1290 2024-03-06T07:21:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.826+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c0aee80 con 0x7f071c0c1290 2024-03-06T07:21:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.826+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0704013400 con 0x7f071c0c1290 2024-03-06T07:21:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.827+0000 7f0718ff9640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.827+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f070408ab80 con 0x7f071c0c1290 2024-03-06T07:21:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.827+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.827+0000 7f071b7fe640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f070c034000 tx=0x7f070c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:25.831+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f071c1ead40 con 0x7f071c0c1290 2024-03-06T07:21:26.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.053+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f071c04f330 con 0x7f071c0c1290 2024-03-06T07:21:26.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.054+0000 7f0718ff9640 1 -- 172.21.15.17:0/1581848332 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f0704019030 con 0x7f071c0c1290 2024-03-06T07:21:26.056 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:26.056 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:21.115466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:26.056 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 msgr2=0x7f06fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 0x7f06fc06d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f070c034000 tx=0x7f070c009030 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 msgr2=0x7f071c0af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f07040114c0 tx=0x7f0704011980 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 shutdown_connections 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06fc06b4d0 0x7f06fc06d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f071c1eb4e0 0x7f071c1ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f071c0c1290 0x7f071c0af610 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 --2- 172.21.15.17:0/1581848332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07080039b0 0x7f071c0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 >> 172.21.15.17:0/1581848332 conn(0x7f071c0ac570 msgr2=0x7f071c0c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 shutdown_connections 2024-03-06T07:21:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.056+0000 7f0721c5c640 1 -- 172.21.15.17:0/1581848332 wait complete. 2024-03-06T07:21:26.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:05.446329+0000', 'modified': '2024-03-06T07:21:21.115466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5233': {'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5233, 'qdb_cluster': [5233]} 2024-03-06T07:21:26.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:26.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:21:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.599+0000 7f51a470f640 1 Processor -- start 2024-03-06T07:21:26.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.599+0000 7f51a470f640 1 -- start start 2024-03-06T07:21:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a470f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f519c155120 0x7f519c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a470f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f519c1522d0 0x7f519c152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a470f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c159590 con 0x7f519c0bfe90 2024-03-06T07:21:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a470f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c158710 con 0x7f519c1522d0 2024-03-06T07:21:26.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a470f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0c59d0 con 0x7f519c155120 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f519c1522d0 0x7f519c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a24ab640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f519c0bfe90 0x7f519c158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55518/0 (socket says 172.21.15.17:55518) 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f519c1522d0 0x7f519c152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52334/0 (socket says 172.21.15.17:52334) 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a24ab640 1 -- 172.21.15.17:0/3038515076 learned_addr learned my addr 172.21.15.17:0/3038515076 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a1caa640 1 --2- 172.21.15.17:0/3038515076 >> v2:172.21.15.163:3300/0 conn(0x7f519c155120 0x7f519c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 -- 172.21.15.17:0/3038515076 >> v2:172.21.15.163:3300/0 conn(0x7f519c155120 msgr2=0x7f519c0c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 --2- 172.21.15.17:0/3038515076 >> v2:172.21.15.163:3300/0 conn(0x7f519c155120 0x7f519c0c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 -- 172.21.15.17:0/3038515076 >> v1:172.21.15.17:6789/0 conn(0x7f519c0bfe90 legacy=0x7f519c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.600+0000 7f51a2cac640 1 -- 172.21.15.17:0/3038515076 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0c5cb0 con 0x7f519c1522d0 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.601+0000 7f51a2cac640 1 --2- 172.21.15.17:0/3038515076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c1522d0 0x7f519c152740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9450609880ac5942 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.601+0000 7f51a14a9640 1 -- 172.21.15.17:0/3038515076 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940315a0 con 0x7f519c1522d0 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.601+0000 7f51a14a9640 1 -- 172.21.15.17:0/3038515076 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194037070 con 0x7f519c1522d0 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.601+0000 7f51a14a9640 1 -- 172.21.15.17:0/3038515076 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194031960 con 0x7f519c1522d0 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 -- 172.21.15.17:0/3038515076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c1522d0 msgr2=0x7f519c152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 --2- 172.21.15.17:0/3038515076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c1522d0 0x7f519c152740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b840 comp rx=0 tx=0).stop 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 -- 172.21.15.17:0/3038515076 shutdown_connections 2024-03-06T07:21:26.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 --2- 172.21.15.17:0/3038515076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c1522d0 0x7f519c152740 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 --2- 172.21.15.17:0/3038515076 >> v2:172.21.15.163:3300/0 conn(0x7f519c155120 0x7f519c0c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 -- 172.21.15.17:0/3038515076 >> 172.21.15.17:0/3038515076 conn(0x7f519c0acaa0 msgr2=0x7f519c0aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 -- 172.21.15.17:0/3038515076 shutdown_connections 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.602+0000 7f51a470f640 1 -- 172.21.15.17:0/3038515076 wait complete. 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.603+0000 7f51a470f640 1 Processor -- start 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.603+0000 7f51a470f640 1 -- start start 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f519c0bfe90 0x7f519c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 0x7f519c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ca700 con 0x7f519c155120 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c158960 con 0x7f519c0bfe90 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0ca880 con 0x7f519c1522d0 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52350/0 (socket says 172.21.15.17:52350) 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 -- 172.21.15.17:0/1468422116 learned_addr learned my addr 172.21.15.17:0/1468422116 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f519c0bfe90 0x7f519c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a1caa640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 0x7f519c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 -- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 msgr2=0x7f519c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 0x7f519c150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 -- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f519c0bfe90 msgr2=0x7f519c0afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f519c0bfe90 0x7f519c0afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 -- 172.21.15.17:0/1468422116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0b0ba0 con 0x7f519c155120 2024-03-06T07:21:26.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a1caa640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 0x7f519c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a2cac640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5194034040 tx=0x7f51940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f519affd640 1 -- 172.21.15.17:0/1468422116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032630 con 0x7f519c155120 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f519affd640 1 -- 172.21.15.17:0/1468422116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940327f0 con 0x7f519c155120 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.604+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0af2e0 con 0x7f519c155120 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.605+0000 7f519affd640 1 -- 172.21.15.17:0/1468422116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194032be0 con 0x7f519c155120 2024-03-06T07:21:26.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.605+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c150e40 con 0x7f519c155120 2024-03-06T07:21:26.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.606+0000 7f519affd640 1 -- 172.21.15.17:0/1468422116 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f519403b3f0 con 0x7f519c155120 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f519affd640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 0x7f518806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f519affd640 1 -- 172.21.15.17:0/1468422116 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f519402f030 con 0x7f519c155120 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 0x7f518806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f51a73cc740 1 Processor -- start 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f51a73cc740 1 -- start start 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.607+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 0x7f518806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f518c0012e0 tx=0x7f518c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039d81a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d824b0 0x560039d7e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039ce86b0 con 0x560039d84330 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039c91a60 con 0x560039d824b0 2024-03-06T07:21:26.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560039ce4750 con 0x560039d83f50 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52360/0 (socket says 172.21.15.17:52360) 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 -- 172.21.15.17:0/724804891 learned_addr learned my addr 172.21.15.17:0/724804891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a24ab640 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039d81a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a2cac640 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d824b0 0x560039d7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 -- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 msgr2=0x560039d81a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039d81a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 -- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d824b0 msgr2=0x560039d7e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d824b0 0x560039d7e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.608+0000 7f51a1caa640 1 -- 172.21.15.17:0/724804891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560039caff30 con 0x560039d84330 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a1caa640 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f519c0c3290 tx=0x7f5190009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4861aae0e6add8e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f5183fff640 1 -- 172.21.15.17:0/724804891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190017020 con 0x560039d84330 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f5183fff640 1 -- 172.21.15.17:0/724804891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000e950 con 0x560039d84330 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f5183fff640 1 -- 172.21.15.17:0/724804891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000f040 con 0x560039d84330 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 -- 172.21.15.17:0/724804891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 msgr2=0x560039d81f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f519c0c3290 tx=0x7f5190009d70 comp rx=0 tx=0).stop 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 -- 172.21.15.17:0/724804891 shutdown_connections 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d824b0 0x560039d7e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039d81f70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 --2- 172.21.15.17:0/724804891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039d81a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 -- 172.21.15.17:0/724804891 >> 172.21.15.17:0/724804891 conn(0x560039d807d0 msgr2=0x560039da53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 -- 172.21.15.17:0/724804891 shutdown_connections 2024-03-06T07:21:26.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.609+0000 7f51a73cc740 1 -- 172.21.15.17:0/724804891 wait complete. 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.610+0000 7f51a73cc740 1 -- start start 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.610+0000 7f51a73cc740 1 Processor -- start 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039da6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039da6230 con 0x560039da9a30 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039ca4ea0 con 0x560039d84330 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560039da5c70 con 0x560039d83f50 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a73cc740 10 client.0 _ll_register_callbacks cb 0x560039ca4870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.611+0000 7f51a2cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52370/0 (socket says 172.21.15.17:52370) 2024-03-06T07:21:26.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 -- 172.21.15.17:0/3128950869 learned_addr learned my addr 172.21.15.17:0/3128950869 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a1caa640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a24ab640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039da6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 -- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 msgr2=0x560039da6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039da6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 -- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 msgr2=0x560039da21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 -- 172.21.15.17:0/3128950869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560039ca5270 con 0x560039da9a30 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a2cac640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f519402be50 tx=0x7f519407ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88b634e8c1fa0dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940a7020 con 0x560039da9a30 2024-03-06T07:21:26.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5194082810 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f519000e950 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51940991a0 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a73cc740 10 client.5583 fetch_fsmap learned FSMap version 23 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.612+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5600399fd2d0 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000e950 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f519404c070 con 0x560039da9a30 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 10 client.5583 fetch_fsmap finished waiting for FSMap version 23 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 10 client.5583 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 20 client.5583 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dd4460 0x560039dd4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 4 client.5583 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:26.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560039cf5bf0 con 0x560039dd4460 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a24ab640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dd4460 0x560039dd4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51817fa640 10 client.5583 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.613+0000 7f51a24ab640 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dd4460 0x560039dd4840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d04671fa40d9bdb server_cookie=d7f42fcbe10ea2f9 in_seq=0 out_seq=0 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51817fa640 1 -- 172.21.15.17:0/3128950869 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560039cf5bf0 con 0x560039dd4460 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51817fa640 10 client.5583 handle_command_reply: tid=0 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 client.5583 shutdown 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 msgr2=0x560039da0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f519402be50 tx=0x7f519407ad10 comp rx=0 tx=0).stop 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 shutdown_connections 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dd4460 0x560039dd4840 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039da9a30 0x560039da0e30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51817fa640 0 client.5583 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 --2- 172.21.15.17:0/3128950869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039d83f50 0x560039da6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.614+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 >> 172.21.15.17:0/3128950869 conn(0x560039d824b0 msgr2=0x560039d7d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.615+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 shutdown_connections 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.615+0000 7f51a73cc740 1 -- 172.21.15.17:0/3128950869 wait complete. 2024-03-06T07:21:26.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.615+0000 7f51a73cc740 20 client.5583 trim_cache size 0 max 16384 2024-03-06T07:21:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.647+0000 7f51a73cc740 1 Processor -- start 2024-03-06T07:21:26.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.647+0000 7f51a73cc740 1 -- start start 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039da5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1f6fb0 con 0x560039d83f50 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c06cb60 con 0x560039d84330 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1f69b0 con 0x560039da9a30 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52376/0 (socket says 172.21.15.17:52376) 2024-03-06T07:21:26.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 -- 172.21.15.17:0/459426111 learned_addr learned my addr 172.21.15.17:0/459426111 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a1caa640 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a2cac640 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039da5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 -- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 msgr2=0x560039da5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039da5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 -- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 msgr2=0x560039da4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.648+0000 7f51a24ab640 1 -- 172.21.15.17:0/459426111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560039ac76b0 con 0x560039d83f50 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a24ab640 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f518c001660 tx=0x7f518c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7b2ddcb015c2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51837fe640 1 -- 172.21.15.17:0/459426111 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0281d0 con 0x560039d83f50 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51837fe640 1 -- 172.21.15.17:0/459426111 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c028880 con 0x560039d83f50 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51837fe640 1 -- 172.21.15.17:0/459426111 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c028a30 con 0x560039d83f50 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 -- 172.21.15.17:0/459426111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 msgr2=0x560039dd0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f518c001660 tx=0x7f518c011000 comp rx=0 tx=0).stop 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 -- 172.21.15.17:0/459426111 shutdown_connections 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039da5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d84330 0x560039da4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 --2- 172.21.15.17:0/459426111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d83f50 0x560039dd0b00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 -- 172.21.15.17:0/459426111 >> 172.21.15.17:0/459426111 conn(0x560039d824b0 msgr2=0x560039d7e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 -- 172.21.15.17:0/459426111 shutdown_connections 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.649+0000 7f51a73cc740 1 -- 172.21.15.17:0/459426111 wait complete. 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.650+0000 7f51a73cc740 1 -- start start 2024-03-06T07:21:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.650+0000 7f51a73cc740 1 Processor -- start 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d83f50 0x560039e33130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039e2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039da6230 con 0x560039d84330 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560039dd42e0 con 0x560039d83f50 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560039dd1a20 con 0x560039da9a30 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a73cc740 10 client.0 _ll_register_callbacks cb 0x7f5194079ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.651+0000 7f51a1caa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52388/0 (socket says 172.21.15.17:52388) 2024-03-06T07:21:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 -- 172.21.15.17:0/1726784977 learned_addr learned my addr 172.21.15.17:0/1726784977 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a2cac640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039e2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d83f50 0x560039e33130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 -- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 msgr2=0x560039e2ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039e2ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 -- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d83f50 msgr2=0x560039e33130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d83f50 0x560039e33130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 -- 172.21.15.17:0/1726784977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560039da5410 con 0x560039d84330 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a1caa640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5190016040 tx=0x7f5190011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b86942e52d775d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519002b060 con 0x560039d84330 2024-03-06T07:21:26.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5190004030 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f518c028880 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5190031050 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a73cc740 10 client.5589 fetch_fsmap learned FSMap version 23 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x560039da24a0 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.652+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c028880 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f5190034020 con 0x560039d84330 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 10 client.5589 fetch_fsmap finished waiting for FSMap version 23 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 10 client.5589 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 20 client.5589 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dcd6a0 0x560039dcda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 4 client.5589 mds_command: new command op to 5233 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 --> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560039ceff40 con 0x560039dcd6a0 2024-03-06T07:21:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dcd6a0 0x560039dcda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51817fa640 10 client.5589 ms_handle_connect on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a24ab640 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dcd6a0 0x560039dcda80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2572486bce189b48 server_cookie=fb0fac119799089 in_seq=0 out_seq=0 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51817fa640 1 -- 172.21.15.17:0/1726784977 <== mds.0 v2:172.21.15.17:6832/3720795082 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x560039ceff40 con 0x560039dcd6a0 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51817fa640 10 client.5589 handle_command_reply: tid=0 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 1 client.5589 shutdown 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 msgr2=0x560039dd0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.653+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5190016040 tx=0x7f5190011000 comp rx=0 tx=0).stop 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 shutdown_connections 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:6832/3720795082,v1:172.21.15.17:6834/3720795082] conn(0x560039dcd6a0 0x560039dcda80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560039da9a30 0x560039e2ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560039d84330 0x560039dd0e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 --2- 172.21.15.17:0/1726784977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560039d83f50 0x560039e33130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51817fa640 0 client.5589 ms_handle_reset on v2:172.21.15.17:6832/3720795082 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 >> 172.21.15.17:0/1726784977 conn(0x560039d824b0 msgr2=0x560039d7e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 shutdown_connections 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 1 -- 172.21.15.17:0/1726784977 wait complete. 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a73cc740 20 client.5589 trim_cache size 0 max 16384 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5233, 2024-03-06T07:21:26.663 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 11.547165736, 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 23, 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 40, 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 40, 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 27.609084352 2024-03-06T07:21:26.664 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 msgr2=0x7f518806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 0x7f518806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f518c0012e0 tx=0x7f518c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 msgr2=0x7f519c1ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5194034040 tx=0x7f51940046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 shutdown_connections 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f518806b4d0 0x7f518806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f519c155120 0x7f519c1ee380 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f519c1522d0 0x7f519c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 --2- 172.21.15.17:0/1468422116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f519c0bfe90 0x7f519c0afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 >> 172.21.15.17:0/1468422116 conn(0x7f519c0acaa0 msgr2=0x7f519c1570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 shutdown_connections 2024-03-06T07:21:26.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.654+0000 7f51a470f640 1 -- 172.21.15.17:0/1468422116 wait complete. 2024-03-06T07:21:26.667 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 23, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:05.446329+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 23, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5233": { "addr": "172.21.15.17:6834/3720795082", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3720795082, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3720795082, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5233, "incarnation": 19, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 40, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2024-03-06T07:21:21.115466+0000", "qdb_cluster": [ 5233 ], "qdb_leader": 5233, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5233 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1442407901", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1442407901, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1442407901, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 13, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5239, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1497992819", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1497992819, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1497992819, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 12, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5286, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/871521754", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 871521754, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 871521754, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 20, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5431, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:26.668 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:26.668 INFO:tasks.cephfs.test_failover:current = [{'gid': 5233, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.17:6834/3720795082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3720795082}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3720795082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5431, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/871521754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 871521754}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 871521754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 20}, {'gid': 5286, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1497992819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1497992819}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1497992819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 12}, {'gid': 5239, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1442407901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1442407901}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1442407901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 13}] 2024-03-06T07:21:26.669 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:21:26.669 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:21:26.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:26.715 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:26.770 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:21:26.770 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:21:26.770 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:26.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:21:26.785 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:21:26.785 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:26.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:21:26.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.924+0000 7fd849a8f640 1 Processor -- start 2024-03-06T07:21:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.924+0000 7fd849a8f640 1 -- start start 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd849a8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd844153990 0x7fd844156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd849a8f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8440acc00 0x7fd8440c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd849a8f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd844070aa0 con 0x7fd8441506b0 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd849a8f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd844156450 con 0x7fd8440acc00 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd849a8f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd844156a10 con 0x7fd844153990 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd843fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8440acc00 0x7fd8440c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd8437fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd8441506b0 0x7fd8440702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55532/0 (socket says 172.21.15.17:55532) 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd8437fe640 1 -- 172.21.15.17:0/3719231790 learned_addr learned my addr 172.21.15.17:0/3719231790 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd843fff640 1 -- 172.21.15.17:0/3719231790 >> v2:172.21.15.163:3300/0 conn(0x7fd844153990 msgr2=0x7fd844156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd843fff640 1 --2- 172.21.15.17:0/3719231790 >> v2:172.21.15.163:3300/0 conn(0x7fd844153990 0x7fd844156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd843fff640 1 -- 172.21.15.17:0/3719231790 >> v1:172.21.15.17:6789/0 conn(0x7fd8441506b0 legacy=0x7fd8440702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd843fff640 1 -- 172.21.15.17:0/3719231790 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844070d80 con 0x7fd8440acc00 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.925+0000 7fd8427fc640 1 -- 172.21.15.17:0/3719231790 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555310202 0 0) 0x7fd844070aa0 con 0x7fd8441506b0 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.926+0000 7fd843fff640 1 --2- 172.21.15.17:0/3719231790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8440acc00 0x7fd8440c54a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd844150a90 tx=0x7fd83c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1904c12caa3a3953 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.926+0000 7fd8427fc640 1 -- 172.21.15.17:0/3719231790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c035020 con 0x7fd8440acc00 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.926+0000 7fd8427fc640 1 -- 172.21.15.17:0/3719231790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c031830 con 0x7fd8440acc00 2024-03-06T07:21:26.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.926+0000 7fd8427fc640 1 -- 172.21.15.17:0/3719231790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c031c00 con 0x7fd8440acc00 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 -- 172.21.15.17:0/3719231790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8440acc00 msgr2=0x7fd8440c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 --2- 172.21.15.17:0/3719231790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8440acc00 0x7fd8440c54a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd844150a90 tx=0x7fd83c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 -- 172.21.15.17:0/3719231790 shutdown_connections 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 --2- 172.21.15.17:0/3719231790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8440acc00 0x7fd8440c54a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 --2- 172.21.15.17:0/3719231790 >> v2:172.21.15.163:3300/0 conn(0x7fd844153990 0x7fd844156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 -- 172.21.15.17:0/3719231790 >> 172.21.15.17:0/3719231790 conn(0x7fd84406b1a0 msgr2=0x7fd84406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 -- 172.21.15.17:0/3719231790 shutdown_connections 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.927+0000 7fd849a8f640 1 -- 172.21.15.17:0/3719231790 wait complete. 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.928+0000 7fd849a8f640 1 Processor -- start 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.928+0000 7fd849a8f640 1 -- start start 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 0x7fd8440c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8441506b0 0x7fd844145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440c5b60 con 0x7fd844153990 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440c59e0 con 0x7fd8440acc00 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd849a8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440c5ce0 con 0x7fd8441506b0 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52406/0 (socket says 172.21.15.17:52406) 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 -- 172.21.15.17:0/1003579737 learned_addr learned my addr 172.21.15.17:0/1003579737 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd8437fe640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 0x7fd8440c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd842ffd640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8441506b0 0x7fd844145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 -- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8441506b0 msgr2=0x7fd844145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8441506b0 0x7fd844145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 -- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 msgr2=0x7fd8440c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 0x7fd8440c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 -- 172.21.15.17:0/1003579737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8441467f0 con 0x7fd844153990 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd8437fe640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 0x7fd8440c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.929+0000 7fd843fff640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02bee0 tx=0x7fd83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.930+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c004230 con 0x7fd844153990 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.930+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c0043d0 con 0x7fd844153990 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.930+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd844148e50 con 0x7fd844153990 2024-03-06T07:21:26.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.930+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c03a920 con 0x7fd844153990 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.930+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd844149410 con 0x7fd844153990 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.931+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd844149610 con 0x7fd844153990 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.931+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd83c038020 con 0x7fd844153990 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.932+0000 7fd82bfff640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 0x7fd83006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.932+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd83c0ac620 con 0x7fd844153990 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.932+0000 7fd8437fe640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 0x7fd83006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:26.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.932+0000 7fd8437fe640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 0x7fd83006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8340012e0 tx=0x7fd834009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:26.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:26.936+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd844149610 con 0x7fd844153990 2024-03-06T07:21:27.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.317+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd84404f330 con 0x7fd844153990 2024-03-06T07:21:27.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.318+0000 7fd82bfff640 1 -- 172.21.15.17:0/1003579737 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4981 (secure 0 0 0) 0x7fd83c041070 con 0x7fd844153990 2024-03-06T07:21:27.320 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:27.320 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:21.115466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:27.320 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 msgr2=0x7fd83006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 0x7fd83006d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8340012e0 tx=0x7fd834009040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 msgr2=0x7fd844146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd83c02bee0 tx=0x7fd83c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 shutdown_connections 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd83006b4d0 0x7fd83006d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd844153990 0x7fd844146070 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8441506b0 0x7fd844145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 --2- 172.21.15.17:0/1003579737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8440acc00 0x7fd8440c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 >> 172.21.15.17:0/1003579737 conn(0x7fd84406b1a0 msgr2=0x7fd84414e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 shutdown_connections 2024-03-06T07:21:27.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.320+0000 7fd849a8f640 1 -- 172.21.15.17:0/1003579737 wait complete. 2024-03-06T07:21:27.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.434+0000 7fded25ac640 1 Processor -- start 2024-03-06T07:21:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.434+0000 7fded25ac640 1 -- start start 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded25ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdecc155ba0 0x7fdecc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded25ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 0x7fdecc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded25ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdecc1591c0 con 0x7fdecc0c5480 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded25ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdecc06df40 con 0x7fdecc155ba0 2024-03-06T07:21:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded25ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc06fdb0 con 0x7fdecc1586f0 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fdecb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdecc155ba0 0x7fdecc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fdecb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdecc155ba0 0x7fdecc0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52410/0 (socket says 172.21.15.17:52410) 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fdecb7fe640 1 -- 172.21.15.17:0/4194569316 learned_addr learned my addr 172.21.15.17:0/4194569316 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.435+0000 7fded0b49640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 0x7fdecc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.436+0000 7fdecb7fe640 1 -- 172.21.15.17:0/4194569316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 msgr2=0x7fdecc158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.436+0000 7fdecb7fe640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 0x7fdecc158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.436+0000 7fdecb7fe640 1 -- 172.21.15.17:0/4194569316 >> v1:172.21.15.163:6789/0 conn(0x7fdecc0c5480 legacy=0x7fdecc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.436+0000 7fdecb7fe640 1 -- 172.21.15.17:0/4194569316 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc1594a0 con 0x7fdecc155ba0 2024-03-06T07:21:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.436+0000 7fded0b49640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 0x7fdecc158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:27.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fdecb7fe640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc155ba0 0x7fdecc0c5860 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009870 tx=0x7fdec002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cae36e210a6018 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fdecaffd640 1 -- 172.21.15.17:0/4194569316 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0035020 con 0x7fdecc155ba0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fdecaffd640 1 -- 172.21.15.17:0/4194569316 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0031640 con 0x7fdecc155ba0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fdecaffd640 1 -- 172.21.15.17:0/4194569316 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec002d040 con 0x7fdecc155ba0 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 -- 172.21.15.17:0/4194569316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc155ba0 msgr2=0x7fdecc0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc155ba0 0x7fdecc0c5860 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009870 tx=0x7fdec002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 -- 172.21.15.17:0/4194569316 shutdown_connections 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc1586f0 0x7fdecc158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 --2- 172.21.15.17:0/4194569316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc155ba0 0x7fdecc0c5860 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.437+0000 7fded25ac640 1 -- 172.21.15.17:0/4194569316 >> 172.21.15.17:0/4194569316 conn(0x7fdecc06b1a0 msgr2=0x7fdecc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.438+0000 7fded25ac640 1 -- 172.21.15.17:0/4194569316 shutdown_connections 2024-03-06T07:21:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.438+0000 7fded25ac640 1 -- 172.21.15.17:0/4194569316 wait complete. 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 Processor -- start 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 -- start start 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc0c5480 0x7fdecc1ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdecc155ba0 0x7fdecc1ec110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc155680 con 0x7fdecc1586f0 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc156000 con 0x7fdecc155ba0 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded25ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc152a20 con 0x7fdecc0c5480 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.439+0000 7fded0b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52426/0 (socket says 172.21.15.17:52426) 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 -- 172.21.15.17:0/2617276148 learned_addr learned my addr 172.21.15.17:0/2617276148 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fdecbfff640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc0c5480 0x7fdecc1ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fdecb7fe640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdecc155ba0 0x7fdecc1ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 -- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc0c5480 msgr2=0x7fdecc1ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc0c5480 0x7fdecc1ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 -- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdecc155ba0 msgr2=0x7fdecc1ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdecc155ba0 0x7fdecc1ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 -- 172.21.15.17:0/2617276148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc1ec650 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded0b49640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d5d0 tx=0x7fdebc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc004030 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc0041d0 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc0c45a0 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc003610 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.440+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc1eeed0 con 0x7fdecc1586f0 2024-03-06T07:21:27.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.441+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc069460 con 0x7fdecc1586f0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.442+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdebc0037b0 con 0x7fdecc1586f0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.442+0000 7fdec8ff9640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 0x7fdeb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.442+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdebc08a460 con 0x7fdecc1586f0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.442+0000 7fdecbfff640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 0x7fdeb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.443+0000 7fdecbfff640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 0x7fdeb806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdec4001f10 tx=0x7fdec4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.446+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdecc069460 con 0x7fdecc1586f0 2024-03-06T07:21:27.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.627+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdecc075300 con 0x7fdecc1586f0 2024-03-06T07:21:27.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.629+0000 7fdec8ff9640 1 -- 172.21.15.17:0/2617276148 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17073 (secure 0 0 0) 0x7fdecc075300 con 0x7fdecc1586f0 2024-03-06T07:21:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:27.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:15.105391+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/2927346551":"2024-03-07T07:21:15.105361+0000","172.21.15.163:6835/2927346551":"2024-03-07T07:21:15.105361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 msgr2=0x7fdeb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 0x7fdeb806d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdec4001f10 tx=0x7fdec4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 msgr2=0x7fdecc1509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00d5d0 tx=0x7fdebc00da90 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 shutdown_connections 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdeb806b4d0 0x7fdeb806d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdecc1586f0 0x7fdecc1509d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdecc155ba0 0x7fdecc1ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 --2- 172.21.15.17:0/2617276148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdecc0c5480 0x7fdecc1ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 >> 172.21.15.17:0/2617276148 conn(0x7fdecc06b1a0 msgr2=0x7fdecc0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 shutdown_connections 2024-03-06T07:21:27.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.631+0000 7fded25ac640 1 -- 172.21.15.17:0/2617276148 wait complete. 2024-03-06T07:21:27.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:21:03.838881+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:21:04.405321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:27.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.760+0000 7f7616b55640 1 Processor -- start 2024-03-06T07:21:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f7616b55640 1 -- start start 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f7616b55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f761014e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f7616b55640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76100b0450 con 0x7f76100b0070 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f7616b55640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76101560e0 con 0x7f7610155c30 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f7616b55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761006c020 con 0x7f761006ca70 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f76148f1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f76100b0070 0x7f76100c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55540/0 (socket says 172.21.15.17:55540) 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.761+0000 7f76148f1640 1 -- 172.21.15.17:0/2544278372 learned_addr learned my addr 172.21.15.17:0/2544278372 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379768573 0 0) 0x7f7608001490 con 0x7f76100b0070 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f76150f2640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f761014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f8003520 con 0x7f76100b0070 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1687037605 0 0) 0x7f75f8003520 con 0x7f76100b0070 2024-03-06T07:21:27.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 msgr2=0x7f761014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f761014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 >> v1:172.21.15.163:6789/0 conn(0x7f7610155c30 legacy=0x7f76100c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76100c77f0 con 0x7f76100b0070 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7608001bc0 con 0x7f76100b0070 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 >> v1:172.21.15.17:6789/0 conn(0x7f76100b0070 legacy=0x7f76100c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f75f8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75f8003f80 0x7f75f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f8004d40 0x7f75f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761006c020 con 0x7f75f8003f80 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f800bb80 con 0x7f75f8004d40 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f800bd80 con 0x7f761006ca70 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75f8003f80 0x7f75f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f76148f1640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f75f8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f76150f2640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f8004d40 0x7f75f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 -- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 msgr2=0x7f75f8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f761006ca70 0x7f75f8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 -- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f8004d40 msgr2=0x7f75f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75f8004d40 0x7f75f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.762+0000 7f760ffff640 1 -- 172.21.15.17:0/2544278372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8003520 con 0x7f75f8003f80 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.763+0000 7f760ffff640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75f8003f80 0x7f75f8004800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7604001750 tx=0x7f75f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5c654f648da94f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.763+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604001370 con 0x7f75f8003f80 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.763+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76040052d0 con 0x7f75f8003f80 2024-03-06T07:21:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.763+0000 7f760f7fe640 1 -- 172.21.15.17:0/2544278372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604003830 con 0x7f75f8003f80 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 -- 172.21.15.17:0/2544278372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75f8003f80 msgr2=0x7f75f8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 --2- 172.21.15.17:0/2544278372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75f8003f80 0x7f75f8004800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7604001750 tx=0x7f75f8005830 comp rx=0 tx=0).stop 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f76150f2640 1 -- 172.21.15.17:0/2544278372 reap_dead start 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 -- 172.21.15.17:0/2544278372 shutdown_connections 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 -- 172.21.15.17:0/2544278372 >> 172.21.15.17:0/2544278372 conn(0x7f76100ac590 msgr2=0x7f76100ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 -- 172.21.15.17:0/2544278372 shutdown_connections 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.764+0000 7f7616b55640 1 -- 172.21.15.17:0/2544278372 wait complete. 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.765+0000 7f7616b55640 1 Processor -- start 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.765+0000 7f7616b55640 1 -- start start 2024-03-06T07:21:27.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75f8003f80 0x7f76100c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f761014b2f0 0x7f76100c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761006c550 con 0x7f76100c4570 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761006c270 con 0x7f761014b2f0 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f7616b55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f761006da80 con 0x7f75f8003f80 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52448/0 (socket says 172.21.15.17:52448) 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 -- 172.21.15.17:0/3735928677 learned_addr learned my addr 172.21.15.17:0/3735928677 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f76148f1640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75f8003f80 0x7f76100c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f76150f2640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f761014b2f0 0x7f76100c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 -- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75f8003f80 msgr2=0x7f76100c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75f8003f80 0x7f76100c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 -- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f761014b2f0 msgr2=0x7f76100c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f761014b2f0 0x7f76100c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.766+0000 7f760ffff640 1 -- 172.21.15.17:0/3735928677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76101eadd0 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f760ffff640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7604005040 tx=0x7f7604001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604041020 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7604005480 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604001370 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76100c4b00 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.767+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76100c1fc0 con 0x7f76100c4570 2024-03-06T07:21:27.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.768+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761014b700 con 0x7f76100c4570 2024-03-06T07:21:27.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.769+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7604044020 con 0x7f76100c4570 2024-03-06T07:21:27.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.769+0000 7f760d7fa640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:27.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.769+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f76040abae0 con 0x7f76100c4570 2024-03-06T07:21:27.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.770+0000 7f76148f1640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:27.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.770+0000 7f76148f1640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 0x7f75fc06d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7608001750 tx=0x7f7608006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:27.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.774+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f760404b030 con 0x7f76100c4570 2024-03-06T07:21:27.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.954+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f761004f330 con 0x7f76100c4570 2024-03-06T07:21:27.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.954+0000 7f760d7fa640 1 -- 172.21.15.17:0/3735928677 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f760407a810 con 0x7f76100c4570 2024-03-06T07:21:27.956 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:21:27.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 msgr2=0x7f75fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 0x7f75fc06d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7608001750 tx=0x7f7608006040 comp rx=0 tx=0).stop 2024-03-06T07:21:27.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 msgr2=0x7f761014adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7604005040 tx=0x7f7604001120 comp rx=0 tx=0).stop 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 shutdown_connections 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75fc06b4d0 0x7f75fc06d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f761014b2f0 0x7f76100c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76100c4570 0x7f761014adb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 --2- 172.21.15.17:0/3735928677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75f8003f80 0x7f76100c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 >> 172.21.15.17:0/3735928677 conn(0x7f76100ac590 msgr2=0x7f7610151430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 shutdown_connections 2024-03-06T07:21:27.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:27.956+0000 7f7616b55640 1 -- 172.21.15.17:0/3735928677 wait complete. 2024-03-06T07:21:27.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:21:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 Processor -- start 2024-03-06T07:21:28.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 -- start start 2024-03-06T07:21:28.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01cc1539c0 0x7f01cc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0c62a0 con 0x7f01cc0c4740 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0c4c00 con 0x7f01cc0a8290 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01d1266640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc0c4ee0 con 0x7f01cc1539c0 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.072+0000 7f01cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f01cc1539c0 0x7f01cc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01ca7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01cc0c4740 0x7f01cc0c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55542/0 (socket says 172.21.15.17:55542) 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01ca7fc640 1 -- 172.21.15.17:0/2826656792 learned_addr learned my addr 172.21.15.17:0/2826656792 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01cb7fe640 1 -- 172.21.15.17:0/2826656792 >> v1:172.21.15.163:6789/0 conn(0x7f01cc0a8290 legacy=0x7f01cc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01cb7fe640 1 -- 172.21.15.17:0/2826656792 >> v1:172.21.15.17:6789/0 conn(0x7f01cc0c4740 legacy=0x7f01cc0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01cb7fe640 1 -- 172.21.15.17:0/2826656792 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc06f9a0 con 0x7f01cc1539c0 2024-03-06T07:21:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.073+0000 7f01c9ffb640 1 -- 172.21.15.17:0/2826656792 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384116607 0 0) 0x7f01cc0c62a0 con 0x7f01cc0c4740 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01cb7fe640 1 --2- 172.21.15.17:0/2826656792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc1539c0 0x7f01cc072680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009d90 tx=0x7f01bc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582184ea20b5cc49 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01c9ffb640 1 -- 172.21.15.17:0/2826656792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc039070 con 0x7f01cc1539c0 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01c9ffb640 1 -- 172.21.15.17:0/2826656792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc02cb40 con 0x7f01cc1539c0 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01c9ffb640 1 -- 172.21.15.17:0/2826656792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc034440 con 0x7f01cc1539c0 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 -- 172.21.15.17:0/2826656792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc1539c0 msgr2=0x7f01cc072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 --2- 172.21.15.17:0/2826656792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc1539c0 0x7f01cc072680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009d90 tx=0x7f01bc02bd90 comp rx=0 tx=0).stop 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 -- 172.21.15.17:0/2826656792 shutdown_connections 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 --2- 172.21.15.17:0/2826656792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc1539c0 0x7f01cc072680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 -- 172.21.15.17:0/2826656792 >> 172.21.15.17:0/2826656792 conn(0x7f01cc0c5560 msgr2=0x7f01cc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 -- 172.21.15.17:0/2826656792 shutdown_connections 2024-03-06T07:21:28.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.074+0000 7f01d1266640 1 -- 172.21.15.17:0/2826656792 wait complete. 2024-03-06T07:21:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.075+0000 7f01d1266640 1 Processor -- start 2024-03-06T07:21:28.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 -- start start 2024-03-06T07:21:28.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc0a8290 0x7f01cc14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1539c0 0x7f01cc0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc072bc0 con 0x7f01cc0c4740 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc153e10 con 0x7f01cc0a8290 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01d1266640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc072d40 con 0x7f01cc1539c0 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52466/0 (socket says 172.21.15.17:52466) 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.076+0000 7f01ca7fc640 1 -- 172.21.15.17:0/3984916420 learned_addr learned my addr 172.21.15.17:0/3984916420 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01cb7fe640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1539c0 0x7f01cc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01caffd640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc0a8290 0x7f01cc14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 -- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1539c0 msgr2=0x7f01cc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1539c0 0x7f01cc0c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 -- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc0a8290 msgr2=0x7f01cc14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc0a8290 0x7f01cc14c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 -- 172.21.15.17:0/3984916420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc1de810 con 0x7f01cc0c4740 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01ca7fc640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c4d0 tx=0x7f01b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4018070 con 0x7f01cc0c4740 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b4007500 con 0x7f01cc0c4740 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc06dc40 con 0x7f01cc0c4740 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4004a20 con 0x7f01cc0c4740 2024-03-06T07:21:28.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.077+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc06d5b0 con 0x7f01cc0c4740 2024-03-06T07:21:28.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.078+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc04f330 con 0x7f01cc0c4740 2024-03-06T07:21:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.079+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01b401b070 con 0x7f01cc0c4740 2024-03-06T07:21:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.079+0000 7f01c37fe640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 0x7f01ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.079+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01b4089650 con 0x7f01cc0c4740 2024-03-06T07:21:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.079+0000 7f01caffd640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 0x7f01ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.080+0000 7f01caffd640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 0x7f01ac06d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0aead0 tx=0x7f01c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.083+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f01b4058380 con 0x7f01cc0c4740 2024-03-06T07:21:28.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.264+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01cc0735e0 con 0x7f01cc0c4740 2024-03-06T07:21:28.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.264+0000 7f01c37fe640 1 -- 172.21.15.17:0/3984916420 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f01b4057c80 con 0x7f01cc0c4740 2024-03-06T07:21:28.266 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 msgr2=0x7f01ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 0x7f01ac06d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0aead0 tx=0x7f01c4009040 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 msgr2=0x7f01cc14b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f01b400c4d0 tx=0x7f01b400c990 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 shutdown_connections 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01ac06b4d0 0x7f01ac06d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1539c0 0x7f01cc0c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01cc0c4740 0x7f01cc14b890 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 --2- 172.21.15.17:0/3984916420 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc0a8290 0x7f01cc14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 >> 172.21.15.17:0/3984916420 conn(0x7f01cc0c5560 msgr2=0x7f01cc1519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 shutdown_connections 2024-03-06T07:21:28.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.266+0000 7f01d1266640 1 -- 172.21.15.17:0/3984916420 wait complete. 2024-03-06T07:21:28.278 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:21:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 Processor -- start 2024-03-06T07:21:28.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 -- start start 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f04a80c57d0 0x7f04a8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a8152dd0 con 0x7f04a81529f0 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a8158b00 con 0x7f04a80c5370 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.382+0000 7f04ae6f4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a8159710 con 0x7f04a80c57d0 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04a81529f0 0x7f04a80c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55548/0 (socket says 172.21.15.17:55548) 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a7fff640 1 -- 172.21.15.17:0/3517903961 learned_addr learned my addr 172.21.15.17:0/3517903961 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> v2:172.21.15.163:3300/0 conn(0x7f04a80c57d0 0x7f04a8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922292628 0 0) 0x7f04a8152dd0 con 0x7f04a81529f0 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003520 con 0x7f04a81529f0 2024-03-06T07:21:28.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369769671 0 0) 0x7f0490003520 con 0x7f04a81529f0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 >> v2:172.21.15.163:3300/0 conn(0x7f04a80c57d0 msgr2=0x7f04a8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 --2- 172.21.15.17:0/3517903961 >> v2:172.21.15.163:3300/0 conn(0x7f04a80c57d0 0x7f04a8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 >> v1:172.21.15.163:6789/0 conn(0x7f04a80c5370 legacy=0x7f04a8155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a8158de0 con 0x7f04a81529f0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0498001670 con 0x7f04a81529f0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 >> v1:172.21.15.17:6789/0 conn(0x7f04a81529f0 legacy=0x7f04a80c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900039b0 0x7f0490003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0490004800 0x7f0490004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 0x7f049000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8159710 con 0x7f04900051c0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049000c160 con 0x7f0490004800 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049000c360 con 0x7f04900039b0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> v2:172.21.15.163:3300/0 conn(0x7f04a80c57d0 0x7f04a8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 0x7f049000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a7fff640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900039b0 0x7f0490003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.383+0000 7f04a77fe640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0490004800 0x7f0490004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 -- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900039b0 msgr2=0x7f0490003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900039b0 0x7f0490003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 -- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0490004800 msgr2=0x7f0490004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0490004800 0x7f0490004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 -- 172.21.15.17:0/3517903961 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490003520 con 0x7f04900051c0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 -- 172.21.15.17:0/3517903961 reap_dead start 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.384+0000 7f04acc91640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 0x7f049000b9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e350 tx=0x7f0490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d70971a6002216 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0009dc0 con 0x7f04900051c0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a002ce00 con 0x7f04900051c0 2024-03-06T07:21:28.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04a6ffd640 1 -- 172.21.15.17:0/3517903961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0003bd0 con 0x7f04900051c0 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 msgr2=0x7f049000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 0x7f049000b9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e350 tx=0x7f0490005830 comp rx=0 tx=0).stop 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3517903961 shutdown_connections 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3517903961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04900051c0 0x7f049000b9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3517903961 >> 172.21.15.17:0/3517903961 conn(0x7f04a80ac580 msgr2=0x7f04a80ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3517903961 shutdown_connections 2024-03-06T07:21:28.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.385+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3517903961 wait complete. 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.386+0000 7f04ae6f4640 1 Processor -- start 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.386+0000 7f04ae6f4640 1 -- start start 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900051c0 0x7f04a814d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a814d070 0x7f04a80aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a81593a0 con 0x7f04a80be030 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a80c68b0 con 0x7f04a814d070 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04ae6f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a8158740 con 0x7f04900051c0 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04acc91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04acc91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52488/0 (socket says 172.21.15.17:52488) 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04acc91640 1 -- 172.21.15.17:0/3416496299 learned_addr learned my addr 172.21.15.17:0/3416496299 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04a7fff640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900051c0 0x7f04a814d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04a77fe640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a814d070 0x7f04a80aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.387+0000 7f04acc91640 1 -- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900051c0 msgr2=0x7f04a814d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04acc91640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900051c0 0x7f04a814d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04acc91640 1 -- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a814d070 msgr2=0x7f04a80aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04acc91640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a814d070 0x7f04a80aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04acc91640 1 -- 172.21.15.17:0/3416496299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a80b0330 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04acc91640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f04a002c850 tx=0x7f04a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0009dc0 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0044070 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a80b0610 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00045e0 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.388+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a80af5e0 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.389+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a804f330 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.389+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04a003d040 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.390+0000 7f04a4ff9640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 0x7f049406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.390+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f04a00ac8b0 con 0x7f04a80be030 2024-03-06T07:21:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.390+0000 7f04a7fff640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 0x7f049406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.390+0000 7f04a7fff640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 0x7f049406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0498007d90 tx=0x7f0498000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.394+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04a007b540 con 0x7f04a80be030 2024-03-06T07:21:28.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.669+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04a806bff0 con 0x7f04a80be030 2024-03-06T07:21:28.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.669+0000 7f04a4ff9640 1 -- 172.21.15.17:0/3416496299 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04a007ae40 con 0x7f04a80be030 2024-03-06T07:21:28.671 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 msgr2=0x7f049406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 0x7f049406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0498007d90 tx=0x7f0498000ed0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 msgr2=0x7f04a81499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f04a002c850 tx=0x7f04a0009750 comp rx=0 tx=0).stop 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 shutdown_connections 2024-03-06T07:21:28.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049406b4d0 0x7f049406d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04a80be030 0x7f04a81499d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04a814d070 0x7f04a80aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 --2- 172.21.15.17:0/3416496299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04900051c0 0x7f04a814d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 >> 172.21.15.17:0/3416496299 conn(0x7f04a80ac580 msgr2=0x7f04a80ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 shutdown_connections 2024-03-06T07:21:28.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.671+0000 7f04ae6f4640 1 -- 172.21.15.17:0/3416496299 wait complete. 2024-03-06T07:21:28.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 Processor -- start 2024-03-06T07:21:28.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 -- start start 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcce81506b0 0x7fcce8153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce8070aa0 con 0x7fcce80acc00 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce8070d80 con 0x7fcce8156ea0 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fccec93a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce8156450 con 0x7fcce81506b0 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fcce659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcce81506b0 0x7fcce8153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fcce6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcce80acc00 0x7fcce80c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55564/0 (socket says 172.21.15.17:55564) 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.786+0000 7fcce6d9d640 1 -- 172.21.15.17:0/1661150559 learned_addr learned my addr 172.21.15.17:0/1661150559 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.787+0000 7fcce659c640 1 -- 172.21.15.17:0/1661150559 >> v1:172.21.15.163:6789/0 conn(0x7fcce8156ea0 legacy=0x7fcce80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.787+0000 7fcce659c640 1 -- 172.21.15.17:0/1661150559 >> v1:172.21.15.17:6789/0 conn(0x7fcce80acc00 legacy=0x7fcce80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.787+0000 7fcce5d9b640 1 -- 172.21.15.17:0/1661150559 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350324287 0 0) 0x7fcce8070aa0 con 0x7fcce80acc00 2024-03-06T07:21:28.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.787+0000 7fcce659c640 1 -- 172.21.15.17:0/1661150559 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce81566a0 con 0x7fcce81506b0 2024-03-06T07:21:28.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fcce659c640 1 --2- 172.21.15.17:0/1661150559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce8153990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccd80097a0 tx=0x7fccd802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd979451dec1153 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fcce5d9b640 1 -- 172.21.15.17:0/1661150559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8034070 con 0x7fcce81506b0 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fcce5d9b640 1 -- 172.21.15.17:0/1661150559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8030610 con 0x7fcce81506b0 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fcce5d9b640 1 -- 172.21.15.17:0/1661150559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80309e0 con 0x7fcce81506b0 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 -- 172.21.15.17:0/1661150559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 msgr2=0x7fcce8153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 --2- 172.21.15.17:0/1661150559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce8153990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccd80097a0 tx=0x7fccd802b960 comp rx=0 tx=0).stop 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 -- 172.21.15.17:0/1661150559 shutdown_connections 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 --2- 172.21.15.17:0/1661150559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce8153990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 -- 172.21.15.17:0/1661150559 >> 172.21.15.17:0/1661150559 conn(0x7fcce806b1a0 msgr2=0x7fcce806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:28.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 -- 172.21.15.17:0/1661150559 shutdown_connections 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.788+0000 7fccec93a640 1 -- 172.21.15.17:0/1661150559 wait complete. 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 Processor -- start 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 -- start start 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 0x7fcce814a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 0x7fcce81ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80ad050 con 0x7fcce81506b0 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce8156a10 con 0x7fcce80acc00 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccec93a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80ad1d0 con 0x7fcce8156ea0 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52512/0 (socket says 172.21.15.17:52512) 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.790+0000 7fccdffff640 1 -- 172.21.15.17:0/29151497 learned_addr learned my addr 172.21.15.17:0/29151497 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fcce659c640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 0x7fcce814a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fcce6d9d640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 0x7fcce81ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 -- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 msgr2=0x7fcce81ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 0x7fcce81ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 -- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 msgr2=0x7fcce814a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 0x7fcce814a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 -- 172.21.15.17:0/29151497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce814c3d0 con 0x7fcce81506b0 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fcce659c640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 0x7fcce814a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fcce6d9d640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 0x7fcce81ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdffff640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c3c0 tx=0x7fccd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0017070 con 0x7fcce81506b0 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd00052f0 con 0x7fcce81506b0 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce814c6b0 con 0x7fcce81506b0 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0003830 con 0x7fcce81506b0 2024-03-06T07:21:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.791+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce814cc70 con 0x7fcce81506b0 2024-03-06T07:21:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.793+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce804f330 con 0x7fcce81506b0 2024-03-06T07:21:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.793+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccd001a070 con 0x7fcce81506b0 2024-03-06T07:21:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.793+0000 7fccdeffd640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 0x7fccc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.793+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fccd0089730 con 0x7fcce81506b0 2024-03-06T07:21:28.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.793+0000 7fcce659c640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 0x7fccc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.794+0000 7fcce659c640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 0x7fccc806d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fccd8036000 tx=0x7fccd8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:28.798+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fccd00583d0 con 0x7fcce81506b0 2024-03-06T07:21:29.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.023+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce80700a0 con 0x7fcce81506b0 2024-03-06T07:21:29.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.024+0000 7fccdeffd640 1 -- 172.21.15.17:0/29151497 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4981 (secure 0 0 0) 0x7fccd005e0d0 con 0x7fcce81506b0 2024-03-06T07:21:29.028 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:29.028 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:21.115466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.025+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 msgr2=0x7fccc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.025+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 0x7fccc806d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fccd8036000 tx=0x7fccd8008030 comp rx=0 tx=0).stop 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.025+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 msgr2=0x7fcce80c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.025+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c3c0 tx=0x7fccd000c880 comp rx=0 tx=0).stop 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 shutdown_connections 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fccc806b4d0 0x7fccc806d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcce8156ea0 0x7fcce81ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcce81506b0 0x7fcce80c7910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 --2- 172.21.15.17:0/29151497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcce80acc00 0x7fcce814a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 >> 172.21.15.17:0/29151497 conn(0x7fcce806b1a0 msgr2=0x7fcce814ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 shutdown_connections 2024-03-06T07:21:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.026+0000 7fccec93a640 1 -- 172.21.15.17:0/29151497 wait complete. 2024-03-06T07:21:29.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:29.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.162+0000 7f9beb30d640 1 Processor -- start 2024-03-06T07:21:29.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.162+0000 7f9beb30d640 1 -- start start 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9beb30d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9be4073250 0x7f9be40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9beb30d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9be40c3950 0x7f9be415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9beb30d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40bdc20 con 0x7f9be4072e70 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9beb30d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be406c8e0 con 0x7f9be40c3950 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9beb30d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be406ce80 con 0x7f9be4073250 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be98aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9be40c3950 0x7f9be415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be90a9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9be4072e70 0x7f9be40bd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55580/0 (socket says 172.21.15.17:55580) 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be98aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9be40c3950 0x7f9be415fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52520/0 (socket says 172.21.15.17:52520) 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be90a9640 1 -- 172.21.15.17:0/753871088 learned_addr learned my addr 172.21.15.17:0/753871088 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:29.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be88a8640 1 --2- 172.21.15.17:0/753871088 >> v2:172.21.15.163:3300/0 conn(0x7f9be4073250 0x7f9be40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be98aa640 1 -- 172.21.15.17:0/753871088 >> v2:172.21.15.163:3300/0 conn(0x7f9be4073250 msgr2=0x7f9be40c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be98aa640 1 --2- 172.21.15.17:0/753871088 >> v2:172.21.15.163:3300/0 conn(0x7f9be4073250 0x7f9be40c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.163+0000 7f9be98aa640 1 -- 172.21.15.17:0/753871088 >> v1:172.21.15.17:6789/0 conn(0x7f9be4072e70 legacy=0x7f9be40bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be3fff640 1 -- 172.21.15.17:0/753871088 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933098216 0 0) 0x7f9be40bdc20 con 0x7f9be4072e70 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be98aa640 1 -- 172.21.15.17:0/753871088 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40c1b00 con 0x7f9be40c3950 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be98aa640 1 --2- 172.21.15.17:0/753871088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be40c3950 0x7f9be415fce0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9be40c7c00 tx=0x7f9bd402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85e46200927be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be3fff640 1 -- 172.21.15.17:0/753871088 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40315a0 con 0x7f9be40c3950 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be3fff640 1 -- 172.21.15.17:0/753871088 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4037070 con 0x7f9be40c3950 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.164+0000 7f9be3fff640 1 -- 172.21.15.17:0/753871088 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4031960 con 0x7f9be40c3950 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 -- 172.21.15.17:0/753871088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be40c3950 msgr2=0x7f9be415fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 --2- 172.21.15.17:0/753871088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be40c3950 0x7f9be415fce0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9be40c7c00 tx=0x7f9bd402b840 comp rx=0 tx=0).stop 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 -- 172.21.15.17:0/753871088 shutdown_connections 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 --2- 172.21.15.17:0/753871088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be40c3950 0x7f9be415fce0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 --2- 172.21.15.17:0/753871088 >> v2:172.21.15.163:3300/0 conn(0x7f9be4073250 0x7f9be40c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 -- 172.21.15.17:0/753871088 >> 172.21.15.17:0/753871088 conn(0x7f9be406b1a0 msgr2=0x7f9be406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 -- 172.21.15.17:0/753871088 shutdown_connections 2024-03-06T07:21:29.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.165+0000 7f9beb30d640 1 -- 172.21.15.17:0/753871088 wait complete. 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.166+0000 7f9beb30d640 1 Processor -- start 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.166+0000 7f9beb30d640 1 -- start start 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9be4073250 0x7f9be40abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 0x7f9be414f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c6760 con 0x7f9be4072e70 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40c4a00 con 0x7f9be40c3950 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9beb30d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41503e0 con 0x7f9be4073250 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52524/0 (socket says 172.21.15.17:52524) 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 -- 172.21.15.17:0/1735442353 learned_addr learned my addr 172.21.15.17:0/1735442353 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be98aa640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 0x7f9be414f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be88a8640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9be4073250 0x7f9be40abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 -- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9be4073250 msgr2=0x7f9be40abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9be4073250 0x7f9be40abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 -- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 msgr2=0x7f9be414f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 0x7f9be414f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 -- 172.21.15.17:0/1735442353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be41eb480 con 0x7f9be4072e70 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be98aa640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 0x7f9be414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.167+0000 7f9be90a9640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c8c0 tx=0x7f9bdc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.168+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc018070 con 0x7f9be4072e70 2024-03-06T07:21:29.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.168+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00a640 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.168+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be41506a0 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.168+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00aa30 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.168+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4071e70 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.169+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be404f330 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.169+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bdc01b070 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.170+0000 7f9be1ffb640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.170+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9bdc089a40 con 0x7f9be4072e70 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.170+0000 7f9be88a8640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.170+0000 7f9be88a8640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 0x7f9bd006d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9be40acc50 tx=0x7f9bd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.174+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9bdc0586d0 con 0x7f9be4072e70 2024-03-06T07:21:29.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.401+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9be40c6340 con 0x7f9be4072e70 2024-03-06T07:21:29.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.402+0000 7f9be1ffb640 1 -- 172.21.15.17:0/1735442353 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17073 (secure 0 0 0) 0x7f9bdc057fd0 con 0x7f9be4072e70 2024-03-06T07:21:29.403 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:29.404 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:15.105391+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/2927346551":"2024-03-07T07:21:15.105361+0000","172.21.15.163:6835/2927346551":"2024-03-07T07:21:15.105361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 msgr2=0x7f9bd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 0x7f9bd006d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9be40acc50 tx=0x7f9bd8008040 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 msgr2=0x7f9be4071930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00c8c0 tx=0x7f9bdc00cd80 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 shutdown_connections 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9be40c3950 0x7f9be414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9be4073250 0x7f9be40abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 --2- 172.21.15.17:0/1735442353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9be4072e70 0x7f9be4071930 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 >> 172.21.15.17:0/1735442353 conn(0x7f9be406b1a0 msgr2=0x7f9be40c1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 shutdown_connections 2024-03-06T07:21:29.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.404+0000 7f9beb30d640 1 -- 172.21.15.17:0/1735442353 wait complete. 2024-03-06T07:21:29.417 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:21:29.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:29.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.517+0000 7fe5e5feb640 1 Processor -- start 2024-03-06T07:21:29.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.517+0000 7fe5e5feb640 1 -- start start 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5e014f360 0x7fe5e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 0x7fe5e00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5e00c5880 0x7fe5e0158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00c8dd0 con 0x7fe5e014f360 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e00c73d0 con 0x7fe5e00c5880 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5e5feb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00be2e0 con 0x7fe5e00c88c0 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5df7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5e014f360 0x7fe5e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5df7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe5e014f360 0x7fe5e0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52538/0 (socket says 172.21.15.17:52538) 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.518+0000 7fe5df7fe640 1 -- 172.21.15.17:0/1407425438 learned_addr learned my addr 172.21.15.17:0/1407425438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5dffff640 1 --2- 172.21.15.17:0/1407425438 >> v2:172.21.15.163:3300/0 conn(0x7fe5e00c5880 0x7fe5e0158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5deffd640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 0x7fe5e00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 -- 172.21.15.17:0/1407425438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 msgr2=0x7fe5e00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 0x7fe5e00c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 -- 172.21.15.17:0/1407425438 >> v2:172.21.15.163:3300/0 conn(0x7fe5e00c5880 msgr2=0x7fe5e0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/1407425438 >> v2:172.21.15.163:3300/0 conn(0x7fe5e00c5880 0x7fe5e0158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 -- 172.21.15.17:0/1407425438 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00c7670 con 0x7fe5e014f360 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5deffd640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 0x7fe5e00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.519+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e014f360 0x7fe5e0155c10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe5e008cc10 tx=0x7fe5d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b649af806636a17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5de7fc640 1 -- 172.21.15.17:0/1407425438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002e070 con 0x7fe5e014f360 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5de7fc640 1 -- 172.21.15.17:0/1407425438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0037070 con 0x7fe5e014f360 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5de7fc640 1 -- 172.21.15.17:0/1407425438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0031a00 con 0x7fe5e014f360 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 -- 172.21.15.17:0/1407425438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e014f360 msgr2=0x7fe5e0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e014f360 0x7fe5e0155c10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe5e008cc10 tx=0x7fe5d002b880 comp rx=0 tx=0).stop 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 -- 172.21.15.17:0/1407425438 shutdown_connections 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/1407425438 >> v2:172.21.15.163:3300/0 conn(0x7fe5e00c5880 0x7fe5e0158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e00c88c0 0x7fe5e00c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/1407425438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e014f360 0x7fe5e0155c10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 -- 172.21.15.17:0/1407425438 >> 172.21.15.17:0/1407425438 conn(0x7fe5e00ac570 msgr2=0x7fe5e00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 -- 172.21.15.17:0/1407425438 shutdown_connections 2024-03-06T07:21:29.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.520+0000 7fe5e5feb640 1 -- 172.21.15.17:0/1407425438 wait complete. 2024-03-06T07:21:29.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.521+0000 7fe5e5feb640 1 Processor -- start 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.521+0000 7fe5e5feb640 1 -- start start 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e00c5880 0x7fe5e0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 0x7fe5e00bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e01590a0 con 0x7fe5e00c88c0 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00c7c30 con 0x7fe5e00c5880 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5e5feb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0159220 con 0x7fe5e014f360 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5deffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52544/0 (socket says 172.21.15.17:52544) 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5deffd640 1 -- 172.21.15.17:0/2697751658 learned_addr learned my addr 172.21.15.17:0/2697751658 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e00c5880 0x7fe5e0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.522+0000 7fe5dffff640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 0x7fe5e00bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 -- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 msgr2=0x7fe5e00bdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 0x7fe5e00bdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 -- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e00c5880 msgr2=0x7fe5e0154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e00c5880 0x7fe5e0154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 -- 172.21.15.17:0/2697751658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0158650 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5dffff640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 0x7fe5e00bdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5deffd640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c7f0 tx=0x7fe5d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4015070 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4007a30 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4007e00 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e01ebb80 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.523+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e0154f20 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.524+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e004f330 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.524+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5d4018070 con 0x7fe5e00c88c0 2024-03-06T07:21:29.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.525+0000 7fe5c7fff640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 0x7fe5cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.525+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe5d408a2d0 con 0x7fe5e00c88c0 2024-03-06T07:21:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.525+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 0x7fe5cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.525+0000 7fe5df7fe640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 0x7fe5cc06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0036000 tx=0x7fe5d003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.529+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe5d4059000 con 0x7fe5e00c88c0 2024-03-06T07:21:29.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.750+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5e006c710 con 0x7fe5e00c88c0 2024-03-06T07:21:29.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.751+0000 7fe5c7fff640 1 -- 172.21.15.17:0/2697751658 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4981 (secure 0 0 0) 0x7fe5d4058900 con 0x7fe5e00c88c0 2024-03-06T07:21:29.753 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:29.753 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:21.115466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:29.753 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-03-06T07:21:29.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 msgr2=0x7fe5cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 0x7fe5cc06d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0036000 tx=0x7fe5d003d040 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 msgr2=0x7fe5e00bd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400c7f0 tx=0x7fe5d400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 shutdown_connections 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe5cc06b4d0 0x7fe5cc06d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5e014f360 0x7fe5e00bdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5e00c88c0 0x7fe5e00bd7c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 --2- 172.21.15.17:0/2697751658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5e00c5880 0x7fe5e0154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.753+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 >> 172.21.15.17:0/2697751658 conn(0x7fe5e00ac570 msgr2=0x7fe5e00c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.754+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 shutdown_connections 2024-03-06T07:21:29.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.754+0000 7fe5e5feb640 1 -- 172.21.15.17:0/2697751658 wait complete. 2024-03-06T07:21:29.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:29.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.865+0000 7f1369a81640 1 Processor -- start 2024-03-06T07:21:29.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.865+0000 7f1369a81640 1 -- start start 2024-03-06T07:21:29.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1369a81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1364152920 0x7f1364155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1369a81640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 0x7f13640c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1369a81640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1364156180 con 0x7f13640c8a00 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1369a81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13641587b0 con 0x7f1364152920 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1369a81640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1364159630 con 0x7f13640c5330 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1364152920 0x7f1364155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1364152920 0x7f1364155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52546/0 (socket says 172.21.15.17:52546) 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 -- 172.21.15.17:0/3901701318 learned_addr learned my addr 172.21.15.17:0/3901701318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f1363fff640 1 --2- 172.21.15.17:0/3901701318 >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 0x7f13640c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 -- 172.21.15.17:0/3901701318 >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 msgr2=0x7f13640c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 --2- 172.21.15.17:0/3901701318 >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 0x7f13640c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.866+0000 7f13637fe640 1 -- 172.21.15.17:0/3901701318 >> v1:172.21.15.163:6789/0 conn(0x7f13640c8a00 legacy=0x7f1364158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:29.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.867+0000 7f13637fe640 1 -- 172.21.15.17:0/3901701318 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13640c7760 con 0x7f1364152920 2024-03-06T07:21:29.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.867+0000 7f1363fff640 1 --2- 172.21.15.17:0/3901701318 >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 0x7f13640c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.868+0000 7f13637fe640 1 --2- 172.21.15.17:0/3901701318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1364152920 0x7f1364155c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f13640cba60 tx=0x7f135402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d503ee79962e869c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.868+0000 7f13627fc640 1 -- 172.21.15.17:0/3901701318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135402e070 con 0x7f1364152920 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.868+0000 7f13627fc640 1 -- 172.21.15.17:0/3901701318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354037070 con 0x7f1364152920 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.868+0000 7f13627fc640 1 -- 172.21.15.17:0/3901701318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354031a00 con 0x7f1364152920 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 -- 172.21.15.17:0/3901701318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1364152920 msgr2=0x7f1364155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 --2- 172.21.15.17:0/3901701318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1364152920 0x7f1364155c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f13640cba60 tx=0x7f135402b880 comp rx=0 tx=0).stop 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 -- 172.21.15.17:0/3901701318 shutdown_connections 2024-03-06T07:21:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 --2- 172.21.15.17:0/3901701318 >> v2:172.21.15.163:3300/0 conn(0x7f13640c5330 0x7f13640c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 --2- 172.21.15.17:0/3901701318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1364152920 0x7f1364155c40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 -- 172.21.15.17:0/3901701318 >> 172.21.15.17:0/3901701318 conn(0x7f13640ac560 msgr2=0x7f13640ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 -- 172.21.15.17:0/3901701318 shutdown_connections 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.869+0000 7f1369a81640 1 -- 172.21.15.17:0/3901701318 wait complete. 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.870+0000 7f1369a81640 1 Processor -- start 2024-03-06T07:21:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.871+0000 7f1369a81640 1 -- start start 2024-03-06T07:21:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13640c5330 0x7f1364147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1364152920 0x7f13641df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364158a00 con 0x7f13640c8a00 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13640c7d20 con 0x7f13640c5330 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1369a81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136406d030 con 0x7f1364152920 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52548/0 (socket says 172.21.15.17:52548) 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 -- 172.21.15.17:0/2591880781 learned_addr learned my addr 172.21.15.17:0/2591880781 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f13637fe640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13640c5330 0x7f1364147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1363fff640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1364152920 0x7f13641df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 -- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1364152920 msgr2=0x7f13641df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1364152920 0x7f13641df8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 -- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13640c5330 msgr2=0x7f1364147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13640c5330 0x7f1364147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:29.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 -- 172.21.15.17:0/2591880781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13641de8c0 con 0x7f13640c8a00 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f1362ffd640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b420 tx=0x7f134c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c011830 con 0x7f13640c8a00 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.872+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c016070 con 0x7f13640c8a00 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.873+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1364156df0 con 0x7f13640c8a00 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.873+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c011c10 con 0x7f13640c8a00 2024-03-06T07:21:29.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.873+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641573b0 con 0x7f13640c8a00 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.874+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136404f330 con 0x7f13640c8a00 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.874+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f134c01b460 con 0x7f13640c8a00 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.874+0000 7f135bfff640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 0x7f134406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.875+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f134c089d90 con 0x7f13640c8a00 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.875+0000 7f13637fe640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 0x7f134406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:29.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.875+0000 7f13637fe640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 0x7f134406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1354036000 tx=0x7f135403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:29.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:29.879+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f134c020020 con 0x7f13640c8a00 2024-03-06T07:21:30.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.102+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f136406cda0 con 0x7f13640c8a00 2024-03-06T07:21:30.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.103+0000 7f135bfff640 1 -- 172.21.15.17:0/2591880781 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4981 (secure 0 0 0) 0x7f134c0583e0 con 0x7f13640c8a00 2024-03-06T07:21:30.105 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:30.105 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:05.446329+0000","modified":"2024-03-06T07:21:21.115466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5233":{"gid":5233,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.17:6834/3720795082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3720795082},{"type":"v1","addr":"172.21.15.17:6834","nonce":3720795082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5233,"qdb_cluster":[5233]},"id":2}]} 2024-03-06T07:21:30.105 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 23 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 msgr2=0x7f134406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 0x7f134406d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1354036000 tx=0x7f135403d040 comp rx=0 tx=0).stop 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 msgr2=0x7f1364157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b420 tx=0x7f134c00b8e0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 shutdown_connections 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f134406b4d0 0x7f134406d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1364152920 0x7f13641df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13640c8a00 0x7f1364157ba0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 --2- 172.21.15.17:0/2591880781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13640c5330 0x7f1364147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 >> 172.21.15.17:0/2591880781 conn(0x7f13640ac560 msgr2=0x7f13641e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 shutdown_connections 2024-03-06T07:21:30.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.105+0000 7f1369a81640 1 -- 172.21.15.17:0/2591880781 wait complete. 2024-03-06T07:21:30.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.218+0000 7ff53b2b8640 1 Processor -- start 2024-03-06T07:21:30.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.218+0000 7ff53b2b8640 1 -- start start 2024-03-06T07:21:30.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff53b2b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff534155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff53b2b8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff534158b60 0x7ff534158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff53b2b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340c73d0 con 0x7ff5340c88c0 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff53b2b8640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5340c8dd0 con 0x7ff534158b60 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff53b2b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340be2e0 con 0x7ff53414f360 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff538853640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff5340c88c0 0x7ff5340c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55592/0 (socket says 172.21.15.17:55592) 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff538853640 1 -- 172.21.15.17:0/933888466 learned_addr learned my addr 172.21.15.17:0/933888466 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff539855640 1 --2- 172.21.15.17:0/933888466 >> v2:172.21.15.163:3300/0 conn(0x7ff534158b60 0x7ff534158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755190821 0 0) 0x7ff5340c73d0 con 0x7ff5340c88c0 2024-03-06T07:21:30.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff539054640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff534155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.219+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520003520 con 0x7ff5340c88c0 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422657130 0 0) 0x7ff520003520 con 0x7ff5340c88c0 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 msgr2=0x7ff534155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff534155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 >> v2:172.21.15.163:3300/0 conn(0x7ff534158b60 msgr2=0x7ff534158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 --2- 172.21.15.17:0/933888466 >> v2:172.21.15.163:3300/0 conn(0x7ff534158b60 0x7ff534158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340c7670 con 0x7ff5340c88c0 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff524001670 con 0x7ff5340c88c0 2024-03-06T07:21:30.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 >> v1:172.21.15.17:6789/0 conn(0x7ff5340c88c0 legacy=0x7ff5340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200039b0 0x7ff520003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff520004800 0x7ff520004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340be2e0 con 0x7ff520004800 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52000b9c0 con 0x7ff5200039b0 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52000bb40 con 0x7ff53414f360 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff520004800 0x7ff520004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff539054640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200039b0 0x7ff520003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff539855640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 -- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 msgr2=0x7ff5200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff5200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 -- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200039b0 msgr2=0x7ff520003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5200039b0 0x7ff520003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff538853640 1 -- 172.21.15.17:0/933888466 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520003520 con 0x7ff520004800 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.220+0000 7ff539855640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53414f360 0x7ff5200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.221+0000 7ff538853640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff520004800 0x7ff520004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c210 tx=0x7ff520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc1f2f6af671d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.221+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524007d50 con 0x7ff520004800 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.221+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524030070 con 0x7ff520004800 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.221+0000 7ff52bfff640 1 -- 172.21.15.17:0/933888466 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52402cd40 con 0x7ff520004800 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 -- 172.21.15.17:0/933888466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff520004800 msgr2=0x7ff520004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/933888466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff520004800 0x7ff520004c80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c210 tx=0x7ff520005830 comp rx=0 tx=0).stop 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff539855640 1 -- 172.21.15.17:0/933888466 reap_dead start 2024-03-06T07:21:30.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 -- 172.21.15.17:0/933888466 shutdown_connections 2024-03-06T07:21:30.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 -- 172.21.15.17:0/933888466 >> 172.21.15.17:0/933888466 conn(0x7ff5340ac570 msgr2=0x7ff5340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 -- 172.21.15.17:0/933888466 shutdown_connections 2024-03-06T07:21:30.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.222+0000 7ff53b2b8640 1 -- 172.21.15.17:0/933888466 wait complete. 2024-03-06T07:21:30.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.223+0000 7ff53b2b8640 1 Processor -- start 2024-03-06T07:21:30.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.223+0000 7ff53b2b8640 1 -- start start 2024-03-06T07:21:30.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520004800 0x7ff5340ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff534149070 0x7ff5341494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c5a50 con 0x7ff5341ec2f0 2024-03-06T07:21:30.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c7c30 con 0x7ff520004800 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff53b2b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340c5bd0 con 0x7ff534149070 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff538853640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff538853640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52570/0 (socket says 172.21.15.17:52570) 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.224+0000 7ff538853640 1 -- 172.21.15.17:0/4106150880 learned_addr learned my addr 172.21.15.17:0/4106150880 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff539054640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520004800 0x7ff5340ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff539855640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff534149070 0x7ff5341494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 -- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff534149070 msgr2=0x7ff5341494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff534149070 0x7ff5341494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 -- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520004800 msgr2=0x7ff5340ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520004800 0x7ff5340ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 -- 172.21.15.17:0/4106150880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534149a30 con 0x7ff5341ec2f0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff538853640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c300 tx=0x7ff524001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524045070 con 0x7ff5341ec2f0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524004bf0 con 0x7ff5341ec2f0 2024-03-06T07:21:30.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5341570c0 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524001c90 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.225+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff534157680 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.226+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5340c5240 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.227+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff524007070 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.227+0000 7ff529ffb640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.227+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff5240ab4e0 con 0x7ff5341ec2f0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.228+0000 7ff539054640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.228+0000 7ff539054640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 0x7ff51806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099c0 tx=0x7ff530008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.235+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff5340c5240 con 0x7ff5341ec2f0 2024-03-06T07:21:30.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.416+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5340c66c0 con 0x7ff5341ec2f0 2024-03-06T07:21:30.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.417+0000 7ff529ffb640 1 -- 172.21.15.17:0/4106150880 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+17073 (secure 0 0 0) 0x7ff5340c66c0 con 0x7ff5341ec2f0 2024-03-06T07:21:30.419 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:30.419 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":40,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:15.105391+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:03.838881+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:04.405321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/2927346551":"2024-03-07T07:21:15.105361+0000","172.21.15.163:6835/2927346551":"2024-03-07T07:21:15.105361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 msgr2=0x7ff51806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 0x7ff51806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff5300099c0 tx=0x7ff530008040 comp rx=0 tx=0).stop 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 msgr2=0x7ff5341ec770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff52400c300 tx=0x7ff524001a40 comp rx=0 tx=0).stop 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 shutdown_connections 2024-03-06T07:21:30.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff51806b4d0 0x7ff51806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff534149070 0x7ff5341494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5341ec2f0 0x7ff5341ec770 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 --2- 172.21.15.17:0/4106150880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff520004800 0x7ff5340ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 >> 172.21.15.17:0/4106150880 conn(0x7ff5340ac570 msgr2=0x7ff5340ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 shutdown_connections 2024-03-06T07:21:30.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.419+0000 7ff53b2b8640 1 -- 172.21.15.17:0/4106150880 wait complete. 2024-03-06T07:21:30.432 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:21:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.533+0000 7fb6c5c81640 1 Processor -- start 2024-03-06T07:21:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.533+0000 7fb6c5c81640 1 -- start start 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6c5c81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6c5c81640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c00a8390 con 0x7fb6c00c7170 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6c5c81640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c00730f0 con 0x7fb6c006ff10 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6c5c81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00c7620 con 0x7fb6c00c8f20 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6beffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb6c00c7170 0x7fb6c00c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54230/0 (socket says 172.21.15.17:54230) 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6beffd640 1 -- 172.21.15.17:0/313251111 learned_addr learned my addr 172.21.15.17:0/313251111 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:30.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6bffff640 1 --2- 172.21.15.17:0/313251111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766977100 0 0) 0x7fb6c00a8390 con 0x7fb6c00c7170 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a8003520 con 0x7fb6c00c7170 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233847647 0 0) 0x7fb6c00730f0 con 0x7fb6c006ff10 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c00a8390 con 0x7fb6c006ff10 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744348381 0 0) 0x7fb6a8003520 con 0x7fb6c00c7170 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6bffff640 1 -- 172.21.15.17:0/313251111 >> v1:172.21.15.163:6789/0 conn(0x7fb6c006ff10 legacy=0x7fb6c0153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6bffff640 1 -- 172.21.15.17:0/313251111 >> v1:172.21.15.17:6789/0 conn(0x7fb6c00c7170 legacy=0x7fb6c00c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.534+0000 7fb6bffff640 1 -- 172.21.15.17:0/313251111 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00733d0 con 0x7fb6c00c8f20 2024-03-06T07:21:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6bffff640 1 --2- 172.21.15.17:0/313251111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c015ebb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009dc0 tx=0x7fb6b802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fcd5bc66753cf36 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8039070 con 0x7fb6c00c8f20 2024-03-06T07:21:30.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b802c7f0 con 0x7fb6c00c8f20 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6be7fc640 1 -- 172.21.15.17:0/313251111 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4001670 con 0x7fb6c00c8f20 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 -- 172.21.15.17:0/313251111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 msgr2=0x7fb6c015ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/313251111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c015ebb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009dc0 tx=0x7fb6b802bcc0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 -- 172.21.15.17:0/313251111 shutdown_connections 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/313251111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c015ebb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 -- 172.21.15.17:0/313251111 >> 172.21.15.17:0/313251111 conn(0x7fb6c006b970 msgr2=0x7fb6c00aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 -- 172.21.15.17:0/313251111 shutdown_connections 2024-03-06T07:21:30.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.536+0000 7fb6c5c81640 1 -- 172.21.15.17:0/313251111 wait complete. 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.537+0000 7fb6c5c81640 1 Processor -- start 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 -- start start 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c006ff10 0x7fb6c006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c01ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c01544b0 con 0x7fb6c00c7170 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c0154330 con 0x7fb6c006ff10 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6c5c81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00702f0 con 0x7fb6c00c8f20 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42004/0 (socket says 172.21.15.17:42004) 2024-03-06T07:21:30.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.538+0000 7fb6beffd640 1 -- 172.21.15.17:0/1883516047 learned_addr learned my addr 172.21.15.17:0/1883516047 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6bffff640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c01ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6bf7fe640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c006ff10 0x7fb6c006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 -- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 msgr2=0x7fb6c01ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c01ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 -- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c006ff10 msgr2=0x7fb6c006db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c006ff10 0x7fb6c006db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 -- 172.21.15.17:0/1883516047 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0154f20 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6beffd640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c780 tx=0x7fb6b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4017070 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4005af0 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c0156940 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4007070 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.539+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c0156f00 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.540+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c004f330 con 0x7fb6c00c7170 2024-03-06T07:21:30.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.540+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6b401a070 con 0x7fb6c00c7170 2024-03-06T07:21:30.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.541+0000 7fb6a7fff640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 0x7fb6ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.541+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb6b40898d0 con 0x7fb6c00c7170 2024-03-06T07:21:30.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.541+0000 7fb6bf7fe640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 0x7fb6ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.541+0000 7fb6bf7fe640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 0x7fb6ac06d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00011e0 tx=0x7fb6b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.545+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6b4058600 con 0x7fb6c00c7170 2024-03-06T07:21:30.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.846+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6c00678d0 con 0x7fb6c00c7170 2024-03-06T07:21:30.848 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:21:30.847+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:21:30.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.848+0000 7fb6a7fff640 1 -- 172.21.15.17:0/1883516047 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6b4057f00 con 0x7fb6c00c7170 2024-03-06T07:21:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 msgr2=0x7fb6ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 0x7fb6ac06d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00011e0 tx=0x7fb6b0009040 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 msgr2=0x7fb6c01ede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c780 tx=0x7fb6b400cc40 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 shutdown_connections 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6ac06b500 0x7fb6ac06d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6c00c8f20 0x7fb6c01ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6c00c7170 0x7fb6c01ede10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 --2- 172.21.15.17:0/1883516047 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6c006ff10 0x7fb6c006db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 >> 172.21.15.17:0/1883516047 conn(0x7fb6c006b970 msgr2=0x7fb6c0150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 shutdown_connections 2024-03-06T07:21:30.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.851+0000 7fb6c5c81640 1 -- 172.21.15.17:0/1883516047 wait complete. 2024-03-06T07:21:30.854 INFO:tasks.ceph.mds.a.smithi017.stderr: -367> 2024-03-06T07:21:20.910+0000 7f6151d5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:30.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:21:30.878 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:21:30.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.967+0000 7f85bf21c640 1 Processor -- start 2024-03-06T07:21:30.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 -- start start 2024-03-06T07:21:30.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85b80724d0 0x7f85b8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 0x7f85b8158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b80c2c00 con 0x7f85b8158c90 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b8155270 con 0x7f85b80724d0 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.968+0000 7f85bf21c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8155550 con 0x7f85b80c8370 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 0x7f85b8158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85bcfb8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85b80724d0 0x7f85b8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 0x7f85b8158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46030/0 (socket says 172.21.15.17:46030) 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85b7fff640 1 -- 172.21.15.17:0/3031275731 learned_addr learned my addr 172.21.15.17:0/3031275731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85bcfb8640 1 -- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 msgr2=0x7f85b8158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85bcfb8640 1 --2- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 0x7f85b8158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85bcfb8640 1 -- 172.21.15.17:0/3031275731 >> v1:172.21.15.163:6789/0 conn(0x7f85b8158c90 legacy=0x7f85b80728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.969+0000 7f85bcfb8640 1 -- 172.21.15.17:0/3031275731 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8155830 con 0x7f85b80724d0 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85bcfb8640 1 --2- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b80724d0 0x7f85b8155d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009d90 tx=0x7f85b002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c12683f48e7e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85b77fe640 1 -- 172.21.15.17:0/3031275731 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0039070 con 0x7f85b80724d0 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85b77fe640 1 -- 172.21.15.17:0/3031275731 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b002cbc0 con 0x7f85b80724d0 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85b77fe640 1 -- 172.21.15.17:0/3031275731 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0034440 con 0x7f85b80724d0 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85bf21c640 1 -- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b80724d0 msgr2=0x7f85b8155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.970+0000 7f85bf21c640 1 --2- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b80724d0 0x7f85b8155d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009d90 tx=0x7f85b002be10 comp rx=0 tx=0).stop 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 -- 172.21.15.17:0/3031275731 shutdown_connections 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 --2- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80c8370 0x7f85b8158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 --2- 172.21.15.17:0/3031275731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b80724d0 0x7f85b8155d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 -- 172.21.15.17:0/3031275731 >> 172.21.15.17:0/3031275731 conn(0x7f85b80aa5c0 msgr2=0x7f85b80aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 -- 172.21.15.17:0/3031275731 shutdown_connections 2024-03-06T07:21:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.971+0000 7f85bf21c640 1 -- 172.21.15.17:0/3031275731 wait complete. 2024-03-06T07:21:30.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.972+0000 7f85bf21c640 1 Processor -- start 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.972+0000 7f85bf21c640 1 -- start start 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80724d0 0x7f85b80a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b8158c90 0x7f85b81eea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80c2170 con 0x7f85b80c8370 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80c2880 con 0x7f85b8158c90 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80c22f0 con 0x7f85b80724d0 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42018/0 (socket says 172.21.15.17:42018) 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 -- 172.21.15.17:0/680700787 learned_addr learned my addr 172.21.15.17:0/680700787 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bcfb8640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80724d0 0x7f85b80a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bd7b9640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b8158c90 0x7f85b81eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 -- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80724d0 msgr2=0x7f85b80a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80724d0 0x7f85b80a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 -- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b8158c90 msgr2=0x7f85b81eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b8158c90 0x7f85b81eea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 -- 172.21.15.17:0/680700787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b81e1100 con 0x7f85b80c8370 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b7fff640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e840 tx=0x7f85ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017070 con 0x7f85b80c8370 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac00c8e0 con 0x7f85b80c8370 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b8157860 con 0x7f85b80c8370 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.973+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00ccd0 con 0x7f85b80c8370 2024-03-06T07:21:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.974+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b8157580 con 0x7f85b80c8370 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.974+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b804f330 con 0x7f85b80c8370 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.975+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85ac01a070 con 0x7f85b80c8370 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.975+0000 7f85b57fa640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 0x7f85a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.975+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f85ac08a640 con 0x7f85b80c8370 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.976+0000 7f85bcfb8640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 0x7f85a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.976+0000 7f85bcfb8640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 0x7f85a406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85b0036000 tx=0x7f85b0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:30.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:30.980+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f85ac059310 con 0x7f85b80c8370 2024-03-06T07:21:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.202+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f85b80c1f50 con 0x7f85b80c8370 2024-03-06T07:21:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.856+0000 7f85b57fa640 1 -- 172.21.15.17:0/680700787 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85ac058c10 con 0x7f85b80c8370 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 msgr2=0x7f85a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 0x7f85a406d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f85b0036000 tx=0x7f85b0030000 comp rx=0 tx=0).stop 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 msgr2=0x7f85b81ee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00e840 tx=0x7f85ac00ed00 comp rx=0 tx=0).stop 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 shutdown_connections 2024-03-06T07:21:31.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85a406b4d0 0x7f85a406d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85b8158c90 0x7f85b81eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85b80c8370 0x7f85b81ee550 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 --2- 172.21.15.17:0/680700787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85b80724d0 0x7f85b80a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 >> 172.21.15.17:0/680700787 conn(0x7f85b80aa5c0 msgr2=0x7f85b80c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 shutdown_connections 2024-03-06T07:21:31.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:31.859+0000 7f85bf21c640 1 -- 172.21.15.17:0/680700787 wait complete. 2024-03-06T07:21:31.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:21:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.026+0000 7f8c96688640 1 Processor -- start 2024-03-06T07:21:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.026+0000 7f8c96688640 1 -- start start 2024-03-06T07:21:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c96688640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c900c2790 0x7f8c901566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c96688640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c900c65a0 0x7f8c901539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c96688640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c90156f40 con 0x7f8c90072730 2024-03-06T07:21:32.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c96688640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c90157220 con 0x7f8c900c2790 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c96688640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90157500 con 0x7f8c900c65a0 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c8ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c900c2790 0x7f8c901566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c94c25640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8c90072730 0x7f8c90072ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54246/0 (socket says 172.21.15.17:54246) 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c94c25640 1 -- 172.21.15.17:0/4020817297 learned_addr learned my addr 172.21.15.17:0/4020817297 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.027+0000 7f8c8f7fe640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c900c65a0 0x7f8c901539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8ffff640 1 -- 172.21.15.17:0/4020817297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c900c65a0 msgr2=0x7f8c901539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c900c65a0 0x7f8c901539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8ffff640 1 -- 172.21.15.17:0/4020817297 >> v1:172.21.15.17:6789/0 conn(0x7f8c90072730 legacy=0x7f8c90072ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8ffff640 1 -- 172.21.15.17:0/4020817297 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c901534d0 con 0x7f8c900c2790 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8effd640 1 -- 172.21.15.17:0/4020817297 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155964671 0 0) 0x7f8c90156f40 con 0x7f8c90072730 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c901566f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097a0 tx=0x7f8c8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb3a07fa1116950 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.028+0000 7f8c8effd640 1 -- 172.21.15.17:0/4020817297 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80035020 con 0x7f8c900c2790 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c8effd640 1 -- 172.21.15.17:0/4020817297 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80031830 con 0x7f8c900c2790 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c8effd640 1 -- 172.21.15.17:0/4020817297 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031c00 con 0x7f8c900c2790 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 -- 172.21.15.17:0/4020817297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 msgr2=0x7f8c901566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c901566f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097a0 tx=0x7f8c8002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 -- 172.21.15.17:0/4020817297 shutdown_connections 2024-03-06T07:21:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c900c65a0 0x7f8c901539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 --2- 172.21.15.17:0/4020817297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c901566f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 -- 172.21.15.17:0/4020817297 >> 172.21.15.17:0/4020817297 conn(0x7f8c900a8f70 msgr2=0x7f8c9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 -- 172.21.15.17:0/4020817297 shutdown_connections 2024-03-06T07:21:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.029+0000 7f8c96688640 1 -- 172.21.15.17:0/4020817297 wait complete. 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.030+0000 7f8c96688640 1 Processor -- start 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.030+0000 7f8c96688640 1 -- start start 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 0x7f8c9014b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c900c65a0 0x7f8c901e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900732b0 con 0x7f8c900c2790 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90153180 con 0x7f8c900c65a0 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c96688640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90153f30 con 0x7f8c90072730 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c8f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c8f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42038/0 (socket says 172.21.15.17:42038) 2024-03-06T07:21:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c8f7fe640 1 -- 172.21.15.17:0/1928033050 learned_addr learned my addr 172.21.15.17:0/1928033050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 0x7f8c9014b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.031+0000 7f8c94c25640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c900c65a0 0x7f8c901e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 -- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 msgr2=0x7f8c9014b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 0x7f8c9014b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 -- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c900c65a0 msgr2=0x7f8c901e8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c900c65a0 0x7f8c901e8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 -- 172.21.15.17:0/1928033050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c901ec0b0 con 0x7f8c900c2790 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 0x7f8c9014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8f7fe640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c820 tx=0x7f8c7800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78015070 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78004240 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c901ec390 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800ce10 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.032+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c901ec950 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.033+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9004f330 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.033+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c78007830 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.034+0000 7f8c8cff9640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.034+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8c7808a300 con 0x7f8c900c2790 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.034+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.035+0000 7f8c8ffff640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c80036000 tx=0x7f8c8003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.039+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c78058fd0 con 0x7f8c900c2790 2024-03-06T07:21:32.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.219+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c901e8b40 con 0x7f8c900c2790 2024-03-06T07:21:32.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.858+0000 7f8c8cff9640 1 -- 172.21.15.17:0/1928033050 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c780588d0 con 0x7f8c900c2790 2024-03-06T07:21:32.860 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.861+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.861+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c80036000 tx=0x7f8c8003c040 comp rx=0 tx=0).stop 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.861+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 msgr2=0x7f8c900ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.861+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800c820 tx=0x7f8c7800cce0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 shutdown_connections 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c900c65a0 0x7f8c901e8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c900c2790 0x7f8c900ab250 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 --2- 172.21.15.17:0/1928033050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c90072730 0x7f8c9014b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 >> 172.21.15.17:0/1928033050 conn(0x7f8c900a8f70 msgr2=0x7f8c900c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:32.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 shutdown_connections 2024-03-06T07:21:32.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.862+0000 7f8c96688640 1 -- 172.21.15.17:0/1928033050 wait complete. 2024-03-06T07:21:32.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:21:32.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.988+0000 7f3b4ce2a640 1 Processor -- start 2024-03-06T07:21:32.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.988+0000 7f3b4ce2a640 1 -- start start 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.988+0000 7f3b4ce2a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3b48149a90 0x7f3b48156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.988+0000 7f3b4ce2a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b48152f80 0x7f3b48072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.988+0000 7f3b4ce2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48072a50 0x7f3b480c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4ce2a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b48149e70 con 0x7f3b48149a90 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4ce2a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b48153f00 con 0x7f3b48152f80 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4ce2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480c33b0 con 0x7f3b48072a50 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3b48149a90 0x7f3b48156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3b48149a90 0x7f3b48156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42044/0 (socket says 172.21.15.17:42044) 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 -- 172.21.15.17:0/2266962469 learned_addr learned my addr 172.21.15.17:0/2266962469 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b46d9d640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48072a50 0x7f3b480c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b45d9b640 1 --2- 172.21.15.17:0/2266962469 >> v2:172.21.15.163:3300/0 conn(0x7f3b48152f80 0x7f3b48072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 -- 172.21.15.17:0/2266962469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48072a50 msgr2=0x7f3b480c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48072a50 0x7f3b480c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 -- 172.21.15.17:0/2266962469 >> v2:172.21.15.163:3300/0 conn(0x7f3b48152f80 msgr2=0x7f3b48072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 --2- 172.21.15.17:0/2266962469 >> v2:172.21.15.163:3300/0 conn(0x7f3b48152f80 0x7f3b48072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.989+0000 7f3b4659c640 1 -- 172.21.15.17:0/2266962469 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48156b50 con 0x7f3b48149a90 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4659c640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48149a90 0x7f3b48156570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3b48067880 tx=0x7f3b3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d83126d9a5e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4559a640 1 -- 172.21.15.17:0/2266962469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035020 con 0x7f3b48149a90 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4559a640 1 -- 172.21.15.17:0/2266962469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b38031870 con 0x7f3b48149a90 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4559a640 1 -- 172.21.15.17:0/2266962469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38031c40 con 0x7f3b48149a90 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/2266962469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48149a90 msgr2=0x7f3b48156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.990+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48149a90 0x7f3b48156570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3b48067880 tx=0x7f3b3802b910 comp rx=0 tx=0).stop 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/2266962469 shutdown_connections 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48072a50 0x7f3b480c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/2266962469 >> v2:172.21.15.163:3300/0 conn(0x7f3b48152f80 0x7f3b48072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/2266962469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48149a90 0x7f3b48156570 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/2266962469 >> 172.21.15.17:0/2266962469 conn(0x7f3b480aa630 msgr2=0x7f3b480aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:32.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/2266962469 shutdown_connections 2024-03-06T07:21:32.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.991+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/2266962469 wait complete. 2024-03-06T07:21:32.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.992+0000 7f3b4ce2a640 1 Processor -- start 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.992+0000 7f3b4ce2a640 1 -- start start 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.992+0000 7f3b4ce2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.992+0000 7f3b4ce2a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 0x7f3b48145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.992+0000 7f3b4ce2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48152f80 0x7f3b48146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4ce2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b481573f0 con 0x7f3b48072a50 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4ce2a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b48157110 con 0x7f3b48149a90 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4ce2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ab270 con 0x7f3b48152f80 2024-03-06T07:21:32.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42046/0 (socket says 172.21.15.17:42046) 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 -- 172.21.15.17:0/1285328962 learned_addr learned my addr 172.21.15.17:0/1285328962 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b45d9b640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 0x7f3b48145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b46d9d640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48152f80 0x7f3b48146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 -- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48152f80 msgr2=0x7f3b48146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48152f80 0x7f3b48146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 -- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 msgr2=0x7f3b48145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 0x7f3b48145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 -- 172.21.15.17:0/1285328962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480ae760 con 0x7f3b48072a50 2024-03-06T07:21:32.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b45d9b640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 0x7f3b48145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4659c640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802b950 tx=0x7f3b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38040070 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b38030d00 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4806dec0 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035040 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.993+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b480c2460 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.994+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4804f330 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.995+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b3804b060 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.995+0000 7f3b36ffd640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 0x7f3b2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.995+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f3b380ac000 con 0x7f3b48072a50 2024-03-06T07:21:33.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.995+0000 7f3b45d9b640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 0x7f3b2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.996+0000 7f3b45d9b640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 0x7f3b2806d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3b4806d390 tx=0x7f3b3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:33.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:32.999+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3b3807ae10 con 0x7f3b48072a50 2024-03-06T07:21:33.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.185+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b480c5ad0 con 0x7f3b48072a50 2024-03-06T07:21:33.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.865+0000 7f3b36ffd640 1 -- 172.21.15.17:0/1285328962 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3b3807a710 con 0x7f3b48072a50 2024-03-06T07:21:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:21:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 msgr2=0x7f3b2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 0x7f3b2806d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3b4806d390 tx=0x7f3b3c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 msgr2=0x7f3b480c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802b950 tx=0x7f3b380046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 shutdown_connections 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b2806b500 0x7f3b2806d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b48152f80 0x7f3b48146270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b48149a90 0x7f3b48145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 --2- 172.21.15.17:0/1285328962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b48072a50 0x7f3b480c1f20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 >> 172.21.15.17:0/1285328962 conn(0x7f3b480aa630 msgr2=0x7f3b480a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.869+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 shutdown_connections 2024-03-06T07:21:33.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.870+0000 7f3b4ce2a640 1 -- 172.21.15.17:0/1285328962 wait complete. 2024-03-06T07:21:33.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:21:33.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 Processor -- start 2024-03-06T07:21:34.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 -- start start 2024-03-06T07:21:34.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0be00b0070 0x7f0be0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 0x7f0be00c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be00b0450 con 0x7f0be006ca70 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be006c020 con 0x7f0be00b0070 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.998+0000 7f0be5fbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be006cf20 con 0x7f0be00c5380 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdeffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0be006ca70 0x7f0be00c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54258/0 (socket says 172.21.15.17:54258) 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdeffd640 1 -- 172.21.15.17:0/3627902928 learned_addr learned my addr 172.21.15.17:0/3627902928 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdf7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0be00b0070 0x7f0be0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdffff640 1 --2- 172.21.15.17:0/3627902928 >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 0x7f0be00c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bde7fc640 1 -- 172.21.15.17:0/3627902928 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642363636 0 0) 0x7f0bd4001110 con 0x7f0be006ca70 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bde7fc640 1 -- 172.21.15.17:0/3627902928 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bc8003520 con 0x7f0be006ca70 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3627902928 >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 msgr2=0x7f0be00c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdf7fe640 1 --2- 172.21.15.17:0/3627902928 >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 0x7f0be00c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3627902928 >> v1:172.21.15.17:6789/0 conn(0x7f0be006ca70 legacy=0x7f0be00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3627902928 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be00c77f0 con 0x7f0be00b0070 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:33.999+0000 7f0bdffff640 1 --2- 172.21.15.17:0/3627902928 >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 0x7f0be00c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0bdf7fe640 1 --2- 172.21.15.17:0/3627902928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be00b0070 0x7f0be0155c30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0be00be030 tx=0x7f0bd002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d3a2547b418556 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0bde7fc640 1 -- 172.21.15.17:0/3627902928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00315a0 con 0x7f0be00b0070 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0bde7fc640 1 -- 172.21.15.17:0/3627902928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0037070 con 0x7f0be00b0070 2024-03-06T07:21:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0bde7fc640 1 -- 172.21.15.17:0/3627902928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0031960 con 0x7f0be00b0070 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3627902928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be00b0070 msgr2=0x7f0be0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3627902928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be00b0070 0x7f0be0155c30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0be00be030 tx=0x7f0bd002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3627902928 shutdown_connections 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.000+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3627902928 >> v2:172.21.15.163:3300/0 conn(0x7f0be00c5380 0x7f0be00c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.001+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3627902928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be00b0070 0x7f0be0155c30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.001+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3627902928 >> 172.21.15.17:0/3627902928 conn(0x7f0be00ac590 msgr2=0x7f0be00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.001+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3627902928 shutdown_connections 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.001+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3627902928 wait complete. 2024-03-06T07:21:34.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 Processor -- start 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 -- start start 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be00b0070 0x7f0be01564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0be00c5380 0x7f0be00c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be006c550 con 0x7f0be006ca70 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.002+0000 7f0be5fbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be006c270 con 0x7f0be00b0070 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0be5fbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be006da80 con 0x7f0be00c5380 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42068/0 (socket says 172.21.15.17:42068) 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3156080695 learned_addr learned my addr 172.21.15.17:0/3156080695 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0be00c5380 msgr2=0x7f0be00c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0be00c5380 0x7f0be00c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be00b0070 msgr2=0x7f0be01564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be00b0070 0x7f0be01564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 -- 172.21.15.17:0/3156080695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0156a10 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bdf7fe640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000c00 tx=0x7f0bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032630 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd00327d0 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be0158fd0 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032bc0 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.003+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be01eef10 con 0x7f0be006ca70 2024-03-06T07:21:34.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.004+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be00c2270 con 0x7f0be006ca70 2024-03-06T07:21:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.005+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bd004d3f0 con 0x7f0be006ca70 2024-03-06T07:21:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.005+0000 7f0bc7fff640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.005+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0bd002f030 con 0x7f0be006ca70 2024-03-06T07:21:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.005+0000 7f0bdeffd640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.006+0000 7f0bdeffd640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000f60 tx=0x7f0bd4001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.009+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0bd0040070 con 0x7f0be006ca70 2024-03-06T07:21:34.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.192+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0be004f330 con 0x7f0be006ca70 2024-03-06T07:21:34.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.195+0000 7f0bc7fff640 1 -- 172.21.15.17:0/3156080695 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0bd007a550 con 0x7f0be006ca70 2024-03-06T07:21:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 msgr2=0x7f0bcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4000f60 tx=0x7f0bd4001110 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 msgr2=0x7f0be014b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0000c00 tx=0x7f0bd00046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 shutdown_connections 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bcc06b4d0 0x7f0bcc06d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0be00c5380 0x7f0be00c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be00b0070 0x7f0be01564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 --2- 172.21.15.17:0/3156080695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be006ca70 0x7f0be014b240 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 >> 172.21.15.17:0/3156080695 conn(0x7f0be00ac590 msgr2=0x7f0be01515f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 shutdown_connections 2024-03-06T07:21:34.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.197+0000 7f0be5fbc640 1 -- 172.21.15.17:0/3156080695 wait complete. 2024-03-06T07:21:34.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:21:34.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.311+0000 7fd6a83d0640 1 Processor -- start 2024-03-06T07:21:34.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.311+0000 7fd6a83d0640 1 -- start start 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a83d0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6a0155c30 0x7fd6a006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a83d0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a00b0450 con 0x7fd6a00c5380 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a83d0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a01560e0 con 0x7fd6a00b0070 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a83d0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a006cfb0 con 0x7fd6a0155c30 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a696d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd6a00c5380 0x7fd6a014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54262/0 (socket says 172.21.15.17:54262) 2024-03-06T07:21:34.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a696d640 1 -- 172.21.15.17:0/1268044266 learned_addr learned my addr 172.21.15.17:0/1268044266 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a596b640 1 --2- 172.21.15.17:0/1268044266 >> v2:172.21.15.163:3300/0 conn(0x7fd6a0155c30 0x7fd6a006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045899587 0 0) 0x7fd698001490 con 0x7fd6a00c5380 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd68c003520 con 0x7fd6a00c5380 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678753515 0 0) 0x7fd68c003520 con 0x7fd6a00c5380 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 >> v2:172.21.15.163:3300/0 conn(0x7fd6a0155c30 msgr2=0x7fd6a006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 --2- 172.21.15.17:0/1268044266 >> v2:172.21.15.163:3300/0 conn(0x7fd6a0155c30 0x7fd6a006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 >> v1:172.21.15.163:6789/0 conn(0x7fd6a00b0070 legacy=0x7fd6a00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00c77f0 con 0x7fd6a00c5380 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd698001bc0 con 0x7fd6a00c5380 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 >> v1:172.21.15.17:6789/0 conn(0x7fd6a00c5380 legacy=0x7fd6a014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c0039b0 0x7fd68c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 0x7fd68c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c0051c0 0x7fd68c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a006cfb0 con 0x7fd68c004800 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c00c160 con 0x7fd68c0051c0 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c00c360 con 0x7fd68c0039b0 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.312+0000 7fd6a596b640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 0x7fd68c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a696d640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c0051c0 0x7fd68c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a616c640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c0039b0 0x7fd68c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 -- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c0039b0 msgr2=0x7fd68c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c0039b0 0x7fd68c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 -- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c0051c0 msgr2=0x7fd68c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd68c0051c0 0x7fd68c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a696d640 1 -- 172.21.15.17:0/1268044266 reap_dead start 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 -- 172.21.15.17:0/1268044266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c003520 con 0x7fd68c004800 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a616c640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c0039b0 0x7fd68c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.313+0000 7fd6a596b640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 0x7fd68c004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd690009ac0 tx=0x7fd68c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fcafb6e8f837c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690039070 con 0x7fd68c004800 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69002c6b0 con 0x7fd68c004800 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a516a640 1 -- 172.21.15.17:0/1268044266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690009690 con 0x7fd68c004800 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 -- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 msgr2=0x7fd68c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 0x7fd68c004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd690009ac0 tx=0x7fd68c005830 comp rx=0 tx=0).stop 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 -- 172.21.15.17:0/1268044266 shutdown_connections 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/1268044266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd68c004800 0x7fd68c004c80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 -- 172.21.15.17:0/1268044266 >> 172.21.15.17:0/1268044266 conn(0x7fd6a00ac590 msgr2=0x7fd6a00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 -- 172.21.15.17:0/1268044266 shutdown_connections 2024-03-06T07:21:34.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.314+0000 7fd6a83d0640 1 -- 172.21.15.17:0/1268044266 wait complete. 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.315+0000 7fd6a83d0640 1 Processor -- start 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.315+0000 7fd6a83d0640 1 -- start start 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c004800 0x7fd6a014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 0x7fd6a01ed8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a006c270 con 0x7fd6a00c2910 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a006c020 con 0x7fd6a01ed470 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a83d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a006c550 con 0x7fd68c004800 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42078/0 (socket says 172.21.15.17:42078) 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 -- 172.21.15.17:0/3462691110 learned_addr learned my addr 172.21.15.17:0/3462691110 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a696d640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 0x7fd6a01ed8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 -- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c004800 msgr2=0x7fd6a014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c004800 0x7fd6a014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 -- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 msgr2=0x7fd6a01ed8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 0x7fd6a01ed8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a596b640 1 -- 172.21.15.17:0/3462691110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00c32d0 con 0x7fd6a00c2910 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.316+0000 7fd6a696d640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 0x7fd6a01ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd6a596b640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd690000c00 tx=0x7fd690004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690042020 con 0x7fd6a00c2910 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69003a040 con 0x7fd6a00c2910 2024-03-06T07:21:34.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a01eedb0 con 0x7fd6a00c2910 2024-03-06T07:21:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690039070 con 0x7fd6a00c2910 2024-03-06T07:21:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.317+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a00c4ab0 con 0x7fd6a00c2910 2024-03-06T07:21:34.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.318+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a01554e0 con 0x7fd6a00c2910 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.318+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd690045020 con 0x7fd6a00c2910 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.318+0000 7fd696ffd640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.319+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd6900ab560 con 0x7fd6a00c2910 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.319+0000 7fd6a616c640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.319+0000 7fd6a616c640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 0x7fd68406d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00c3510 tx=0x7fd69c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.323+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6a01554e0 con 0x7fd6a00c2910 2024-03-06T07:21:34.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.510+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7fd6a004f330 con 0x7fd6a00c2910 2024-03-06T07:21:34.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.510+0000 7fd696ffd640 1 -- 172.21.15.17:0/3462691110 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v14) v1 ==== 96+0+0 (secure 0 0 0) 0x7fd69004c070 con 0x7fd6a00c2910 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 msgr2=0x7fd68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 0x7fd68406d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00c3510 tx=0x7fd69c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 msgr2=0x7fd6a00c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd690000c00 tx=0x7fd690004730 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 shutdown_connections 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6a01ed470 0x7fd6a01ed8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6a00c2910 0x7fd6a00c2d90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 --2- 172.21.15.17:0/3462691110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd68c004800 0x7fd6a014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.512+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 >> 172.21.15.17:0/3462691110 conn(0x7fd6a00ac590 msgr2=0x7fd6a00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.513+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 shutdown_connections 2024-03-06T07:21:34.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.513+0000 7fd6a83d0640 1 -- 172.21.15.17:0/3462691110 wait complete. 2024-03-06T07:21:34.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop' 2024-03-06T07:21:34.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 Processor -- start 2024-03-06T07:21:34.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 -- start start 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff81014ba90 0x7ff8101559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8100c8690 0x7ff8100c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c5650 0x7ff810158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81014be70 con 0x7ff81014ba90 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100cad30 con 0x7ff8100c8690 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.625+0000 7ff814a01640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100cb010 con 0x7ff8100c5650 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff81014ba90 0x7ff8101559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff81014ba90 0x7ff8101559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42094/0 (socket says 172.21.15.17:42094) 2024-03-06T07:21:34.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 -- 172.21.15.17:0/1231904581 learned_addr learned my addr 172.21.15.17:0/1231904581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 -- 172.21.15.17:0/1231904581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c5650 msgr2=0x7ff810158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/1231904581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c5650 0x7ff810158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 -- 172.21.15.17:0/1231904581 >> v2:172.21.15.163:3300/0 conn(0x7ff8100c8690 msgr2=0x7ff8100c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/1231904581 >> v2:172.21.15.163:3300/0 conn(0x7ff8100c8690 0x7ff8100c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.626+0000 7ff80ed9d640 1 -- 172.21.15.17:0/1231904581 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100cb2f0 con 0x7ff81014ba90 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/1231904581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81014ba90 0x7ff8101559b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c8250 tx=0x7ff80002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b010ff5a9797e173 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff80dd9b640 1 -- 172.21.15.17:0/1231904581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800035020 con 0x7ff81014ba90 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff80dd9b640 1 -- 172.21.15.17:0/1231904581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800031870 con 0x7ff81014ba90 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff80dd9b640 1 -- 172.21.15.17:0/1231904581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800031c40 con 0x7ff81014ba90 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 -- 172.21.15.17:0/1231904581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81014ba90 msgr2=0x7ff8101559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 --2- 172.21.15.17:0/1231904581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81014ba90 0x7ff8101559b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c8250 tx=0x7ff80002b910 comp rx=0 tx=0).stop 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 -- 172.21.15.17:0/1231904581 shutdown_connections 2024-03-06T07:21:34.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 --2- 172.21.15.17:0/1231904581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c5650 0x7ff810158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 --2- 172.21.15.17:0/1231904581 >> v2:172.21.15.163:3300/0 conn(0x7ff8100c8690 0x7ff8100c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 --2- 172.21.15.17:0/1231904581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff81014ba90 0x7ff8101559b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 -- 172.21.15.17:0/1231904581 >> 172.21.15.17:0/1231904581 conn(0x7ff8100728d0 msgr2=0x7ff810072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 -- 172.21.15.17:0/1231904581 shutdown_connections 2024-03-06T07:21:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.627+0000 7ff814a01640 1 -- 172.21.15.17:0/1231904581 wait complete. 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 Processor -- start 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 -- start start 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 0x7ff810140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff81014ba90 0x7ff81013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100c9240 con 0x7ff8100c5650 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81006cf60 con 0x7ff81014ba90 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff814a01640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff810140530 con 0x7ff8100c8690 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff80ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42102/0 (socket says 172.21.15.17:42102) 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.629+0000 7ff80ed9d640 1 -- 172.21.15.17:0/658011497 learned_addr learned my addr 172.21.15.17:0/658011497 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80e59c640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 0x7ff810140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80f59e640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff81014ba90 0x7ff81013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 -- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 msgr2=0x7ff810140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 0x7ff810140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 -- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff81014ba90 msgr2=0x7ff81013f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff81014ba90 0x7ff81013f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 -- 172.21.15.17:0/658011497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100c01a0 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80e59c640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 0x7ff810140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff80ed9d640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff80002b950 tx=0x7ff8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80003ad60 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8000443f0 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8100c0480 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8000447e0 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.630+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff810141320 con 0x7ff8100c5650 2024-03-06T07:21:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.631+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81004f330 con 0x7ff8100c5650 2024-03-06T07:21:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.632+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff800038020 con 0x7ff8100c5650 2024-03-06T07:21:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.632+0000 7ff7f77fe640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 0x7ff7fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.632+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff8000ac2e0 con 0x7ff8100c5650 2024-03-06T07:21:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.632+0000 7ff80e59c640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 0x7ff7fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.633+0000 7ff80e59c640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 0x7ff7fc06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff810145e90 tx=0x7ff804008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:34.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.636+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff81004f330 con 0x7ff8100c5650 2024-03-06T07:21:34.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:34.886+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]} v 0) v1 -- 0x7ff8100bf610 con 0x7ff8100c5650 2024-03-06T07:21:35.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.870+0000 7ff7f77fe640 1 -- 172.21.15.17:0/658011497 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_drop"]}]=0 v102) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff80003f070 con 0x7ff8100c5650 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 msgr2=0x7ff7fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 0x7ff7fc06d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff810145e90 tx=0x7ff804008040 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 msgr2=0x7ff8100bf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff80002b950 tx=0x7ff8000046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 shutdown_connections 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7fc06b4d0 0x7ff7fc06d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff81014ba90 0x7ff81013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8100c8690 0x7ff810140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 --2- 172.21.15.17:0/658011497 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8100c5650 0x7ff8100bf9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:35.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 >> 172.21.15.17:0/658011497 conn(0x7ff8100728d0 msgr2=0x7ff810072cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:35.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 shutdown_connections 2024-03-06T07:21:35.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.873+0000 7ff814a01640 1 -- 172.21.15.17:0/658011497 wait complete. 2024-03-06T07:21:35.888 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will not cause failover if no high-affinity standby exists. ... ok 2024-03-06T07:21:35.888 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_last_resort (tasks.cephfs.test_failover.TestClusterAffinity) That a standby with mds_join_fs set to another fs is still used if necessary. 2024-03-06T07:21:35.888 INFO:tasks.cephfs_test_runner:test_join_fs_last_resort (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:21:35.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:35.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.994+0000 7fcad0356640 1 Processor -- start 2024-03-06T07:21:35.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.994+0000 7fcad0356640 1 -- start start 2024-03-06T07:21:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcad0356640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac80c1aa0 0x7fcac81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcad0356640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcac8155a40 0x7fcac8155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcad0356640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac8156410 con 0x7fcac814f790 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcad0356640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac80c2680 con 0x7fcac8155a40 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcad0356640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c28d0 con 0x7fcac80c1aa0 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcace8f3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcac8155a40 0x7fcac8155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcace0f2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcac814f790 0x7fcac80c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54274/0 (socket says 172.21.15.17:54274) 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcace0f2640 1 -- 172.21.15.17:0/371303012 learned_addr learned my addr 172.21.15.17:0/371303012 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.995+0000 7fcacd8f1640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac80c1aa0 0x7fcac81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcace8f3640 1 -- 172.21.15.17:0/371303012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac80c1aa0 msgr2=0x7fcac81529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcace8f3640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac80c1aa0 0x7fcac81529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcace8f3640 1 -- 172.21.15.17:0/371303012 >> v1:172.21.15.17:6789/0 conn(0x7fcac814f790 legacy=0x7fcac80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcacd0f0640 1 -- 172.21.15.17:0/371303012 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072876407 0 0) 0x7fcac8156410 con 0x7fcac814f790 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcace8f3640 1 -- 172.21.15.17:0/371303012 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80c2bb0 con 0x7fcac8155a40 2024-03-06T07:21:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcace8f3640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8155ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009d90 tx=0x7fcab802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d520f26377add6c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcacd0f0640 1 -- 172.21.15.17:0/371303012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8039070 con 0x7fcac8155a40 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcacd0f0640 1 -- 172.21.15.17:0/371303012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab802cc50 con 0x7fcac8155a40 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.996+0000 7fcacd0f0640 1 -- 172.21.15.17:0/371303012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8034440 con 0x7fcac8155a40 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 -- 172.21.15.17:0/371303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 msgr2=0x7fcac8155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8155ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009d90 tx=0x7fcab802bea0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 -- 172.21.15.17:0/371303012 shutdown_connections 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8155ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 --2- 172.21.15.17:0/371303012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac80c1aa0 0x7fcac81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 -- 172.21.15.17:0/371303012 >> 172.21.15.17:0/371303012 conn(0x7fcac80c6710 msgr2=0x7fcac80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 -- 172.21.15.17:0/371303012 shutdown_connections 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.997+0000 7fcad0356640 1 -- 172.21.15.17:0/371303012 wait complete. 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.998+0000 7fcad0356640 1 Processor -- start 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.998+0000 7fcad0356640 1 -- start start 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac80c1aa0 0x7fcac8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac814f790 0x7fcac814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c0d60 con 0x7fcac8155a40 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac8155520 con 0x7fcac80c1aa0 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcad0356640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac80c5e30 con 0x7fcac814f790 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42116/0 (socket says 172.21.15.17:42116) 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 -- 172.21.15.17:0/2606612205 learned_addr learned my addr 172.21.15.17:0/2606612205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcacd8f1640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac814f790 0x7fcac814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace0f2640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac80c1aa0 0x7fcac8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 -- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac814f790 msgr2=0x7fcac814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac814f790 0x7fcac814e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 -- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac80c1aa0 msgr2=0x7fcac8153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac80c1aa0 0x7fcac8153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 -- 172.21.15.17:0/2606612205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac81e9930 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:35.999+0000 7fcace8f3640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcab8037040 tx=0x7fcab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.000+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8044020 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.000+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab803a040 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.000+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac8153450 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.000+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8039050 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.000+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac8153a10 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.001+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac8153c10 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.001+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcab8003990 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.002+0000 7fcac6ffd640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.002+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcab80ac520 con 0x7fcac8155a40 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.002+0000 7fcace0f2640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.002+0000 7fcace0f2640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 0x7fcab406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000f60 tx=0x7fcac0007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.006+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcac8153c10 con 0x7fcac8155a40 2024-03-06T07:21:36.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.188+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcac80aa6d0 con 0x7fcac8155a40 2024-03-06T07:21:36.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.189+0000 7fcac6ffd640 1 -- 172.21.15.17:0/2606612205 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13425 (secure 0 0 0) 0x7fcac80aa6d0 con 0x7fcac8155a40 2024-03-06T07:21:36.190 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:36.191 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":43,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:33.859120+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/3720795082":"2024-03-07T07:21:30.847606+0000","172.21.15.17:6832/3720795082":"2024-03-07T07:21:30.847606+0000","172.21.15.163:6837/2927346551":"2024-03-07T07:21:15.105361+0000","172.21.15.163:6835/2927346551":"2024-03-07T07:21:15.105361+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 msgr2=0x7fcab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 0x7fcab406d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcac0000f60 tx=0x7fcac0007b50 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 msgr2=0x7fcac8070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcab8037040 tx=0x7fcab8004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 shutdown_connections 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcab406b4d0 0x7fcab406d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcac8155a40 0x7fcac8070b70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcac814f790 0x7fcac814e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 --2- 172.21.15.17:0/2606612205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcac80c1aa0 0x7fcac8153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 >> 172.21.15.17:0/2606612205 conn(0x7fcac80c6710 msgr2=0x7fcac806dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 shutdown_connections 2024-03-06T07:21:36.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.191+0000 7fcad0356640 1 -- 172.21.15.17:0/2606612205 wait complete. 2024-03-06T07:21:36.203 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:36.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:36.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.306+0000 7ff4bf42a640 1 Processor -- start 2024-03-06T07:21:36.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.306+0000 7ff4bf42a640 1 -- start start 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bf42a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4b806c900 0x7ff4b80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bf42a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b80ab170 0x7ff4b80a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bf42a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80c3ef0 con 0x7ff4b80c5990 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bf42a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b81556d0 con 0x7ff4b806c900 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bf42a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81559b0 con 0x7ff4b80ab170 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bd1c6640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff4b80c5990 0x7ff4b806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54290/0 (socket says 172.21.15.17:54290) 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bd1c6640 1 -- 172.21.15.17:0/576879184 learned_addr learned my addr 172.21.15.17:0/576879184 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bc9c5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4b806c900 0x7ff4b80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4affff640 1 -- 172.21.15.17:0/576879184 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815429889 0 0) 0x7ff4b80c3ef0 con 0x7ff4b80c5990 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bd9c7640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b80ab170 0x7ff4b80a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4affff640 1 -- 172.21.15.17:0/576879184 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a4003520 con 0x7ff4b80c5990 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bc9c5640 1 -- 172.21.15.17:0/576879184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b80ab170 msgr2=0x7ff4b80a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bc9c5640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b80ab170 0x7ff4b80a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bc9c5640 1 -- 172.21.15.17:0/576879184 >> v1:172.21.15.17:6789/0 conn(0x7ff4b80c5990 legacy=0x7ff4b806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.307+0000 7ff4bc9c5640 1 -- 172.21.15.17:0/576879184 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8155c90 con 0x7ff4b806c900 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4bc9c5640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b806c900 0x7ff4b80c0ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097a0 tx=0x7ff4a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f46e369d0de62f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4affff640 1 -- 172.21.15.17:0/576879184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8035020 con 0x7ff4b806c900 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4affff640 1 -- 172.21.15.17:0/576879184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8031830 con 0x7ff4b806c900 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4affff640 1 -- 172.21.15.17:0/576879184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8031c00 con 0x7ff4b806c900 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4bf42a640 1 -- 172.21.15.17:0/576879184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b806c900 msgr2=0x7ff4b80c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.308+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b806c900 0x7ff4b80c0ab0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097a0 tx=0x7ff4a802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 -- 172.21.15.17:0/576879184 shutdown_connections 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b80ab170 0x7ff4b80a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/576879184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b806c900 0x7ff4b80c0ab0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 -- 172.21.15.17:0/576879184 >> 172.21.15.17:0/576879184 conn(0x7ff4b80c6710 msgr2=0x7ff4b80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 -- 172.21.15.17:0/576879184 shutdown_connections 2024-03-06T07:21:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.309+0000 7ff4bf42a640 1 -- 172.21.15.17:0/576879184 wait complete. 2024-03-06T07:21:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 Processor -- start 2024-03-06T07:21:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 -- start start 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b806c900 0x7ff4b81dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b80ab170 0x7ff4b80bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8156610 con 0x7ff4b80c5990 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8156490 con 0x7ff4b80ab170 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.310+0000 7ff4bf42a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80c3af0 con 0x7ff4b806c900 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42132/0 (socket says 172.21.15.17:42132) 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 -- 172.21.15.17:0/869066455 learned_addr learned my addr 172.21.15.17:0/869066455 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 -- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b806c900 msgr2=0x7ff4b81dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b806c900 0x7ff4b81dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 -- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b80ab170 msgr2=0x7ff4b80bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b80ab170 0x7ff4b80bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 -- 172.21.15.17:0/869066455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b81eb840 con 0x7ff4b80c5990 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bd9c7640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007e70 tx=0x7ff4b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0014620 con 0x7ff4b80c5990 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0019070 con 0x7ff4b80c5990 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b80a6330 con 0x7ff4b80c5990 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0014a20 con 0x7ff4b80c5990 2024-03-06T07:21:36.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.311+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b8153650 con 0x7ff4b80c5990 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.312+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b804f330 con 0x7ff4b80c5990 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.313+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4b0014bc0 con 0x7ff4b80c5990 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.313+0000 7ff4adffb640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 0x7ff49c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.313+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff4b008a970 con 0x7ff4b80c5990 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.313+0000 7ff4bd1c6640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 0x7ff49c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.314+0000 7ff4bd1c6640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 0x7ff49c06d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000f60 tx=0x7ff4b40078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.317+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4b00599a0 con 0x7ff4b80c5990 2024-03-06T07:21:36.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.496+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4b80a6500 con 0x7ff4b80c5990 2024-03-06T07:21:36.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.497+0000 7ff4adffb640 1 -- 172.21.15.17:0/869066455 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4b00592a0 con 0x7ff4b80c5990 2024-03-06T07:21:36.499 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:21:36.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 msgr2=0x7ff49c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 0x7ff49c06d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4000f60 tx=0x7ff4b40078f0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 msgr2=0x7ff4b81ede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0007e70 tx=0x7ff4b000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 shutdown_connections 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49c06b4d0 0x7ff49c06d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b80c5990 0x7ff4b81ede20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b80ab170 0x7ff4b80bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 --2- 172.21.15.17:0/869066455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b806c900 0x7ff4b81dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 >> 172.21.15.17:0/869066455 conn(0x7ff4b80c6710 msgr2=0x7ff4b80c1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 shutdown_connections 2024-03-06T07:21:36.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.499+0000 7ff4bf42a640 1 -- 172.21.15.17:0/869066455 wait complete. 2024-03-06T07:21:36.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort' 2024-03-06T07:21:36.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.611+0000 7f8ceac2a640 1 Processor -- start 2024-03-06T07:21:36.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.611+0000 7f8ceac2a640 1 -- start start 2024-03-06T07:21:36.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ceac2a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ce4158db0 0x7f8ce40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ceac2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce40c88b0 0x7f8ce415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ceac2a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce4155110 con 0x7f8ce40c5330 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ceac2a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce41559e0 con 0x7f8ce4158db0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ceac2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce40c8f40 con 0x7f8ce40c88b0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ce4158db0 0x7f8ce40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ce4158db0 0x7f8ce40c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42144/0 (socket says 172.21.15.17:42144) 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 -- 172.21.15.17:0/4217574229 learned_addr learned my addr 172.21.15.17:0/4217574229 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce91c7640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce40c88b0 0x7f8ce415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 -- 172.21.15.17:0/4217574229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce40c88b0 msgr2=0x7f8ce415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce40c88b0 0x7f8ce415fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 -- 172.21.15.17:0/4217574229 >> v1:172.21.15.163:6789/0 conn(0x7f8ce40c5330 legacy=0x7f8ce41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.612+0000 7f8ce3fff640 1 -- 172.21.15.17:0/4217574229 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce4155390 con 0x7f8ce4158db0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ce3fff640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce4158db0 0x7f8ce40c5710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009870 tx=0x7f8cd002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2667ec5b47fd8dff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ce37fe640 1 -- 172.21.15.17:0/4217574229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00317c0 con 0x7f8ce4158db0 2024-03-06T07:21:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ce37fe640 1 -- 172.21.15.17:0/4217574229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd0037070 con 0x7f8ce4158db0 2024-03-06T07:21:36.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ce37fe640 1 -- 172.21.15.17:0/4217574229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0031b80 con 0x7f8ce4158db0 2024-03-06T07:21:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ceac2a640 1 -- 172.21.15.17:0/4217574229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce4158db0 msgr2=0x7f8ce40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.613+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce4158db0 0x7f8ce40c5710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009870 tx=0x7f8cd002b860 comp rx=0 tx=0).stop 2024-03-06T07:21:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 -- 172.21.15.17:0/4217574229 shutdown_connections 2024-03-06T07:21:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce40c88b0 0x7f8ce415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/4217574229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce4158db0 0x7f8ce40c5710 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 -- 172.21.15.17:0/4217574229 >> 172.21.15.17:0/4217574229 conn(0x7f8ce40ac560 msgr2=0x7f8ce40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 -- 172.21.15.17:0/4217574229 shutdown_connections 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.614+0000 7f8ceac2a640 1 -- 172.21.15.17:0/4217574229 wait complete. 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 Processor -- start 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 -- start start 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ce40c88b0 0x7f8ce40c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce414fa10 0x7f8ce41ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce41529d0 con 0x7f8ce40c5330 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce406d090 con 0x7f8ce40c88b0 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.615+0000 7f8ceac2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4152b50 con 0x7f8ce414fa10 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42158/0 (socket says 172.21.15.17:42158) 2024-03-06T07:21:36.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 -- 172.21.15.17:0/1971814951 learned_addr learned my addr 172.21.15.17:0/1971814951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 -- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce414fa10 msgr2=0x7f8ce41ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce414fa10 0x7f8ce41ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 -- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ce40c88b0 msgr2=0x7f8ce40c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ce40c88b0 0x7f8ce40c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 -- 172.21.15.17:0/1971814951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce414d680 con 0x7f8ce40c5330 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce89c6640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c430 tx=0x7f8cd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8011790 con 0x7f8ce40c5330 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8017070 con 0x7f8ce40c5330 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce414f730 con 0x7f8ce40c5330 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8011bb0 con 0x7f8ce40c5330 2024-03-06T07:21:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.616+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce414dc00 con 0x7f8ce40c5330 2024-03-06T07:21:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.617+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce404f330 con 0x7f8ce40c5330 2024-03-06T07:21:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.618+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cd801c460 con 0x7f8ce40c5330 2024-03-06T07:21:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.618+0000 7f8ce17fa640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 0x7f8cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.618+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8cd8089f30 con 0x7f8ce40c5330 2024-03-06T07:21:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.618+0000 7f8ce3fff640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 0x7f8cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.619+0000 7f8ce3fff640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 0x7f8cc806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0036000 tx=0x7f8cd0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:36.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.622+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8cd8058f60 con 0x7f8ce40c5330 2024-03-06T07:21:36.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:36.874+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]} v 0) v1 -- 0x7f8ce4068f00 con 0x7f8ce40c5330 2024-03-06T07:21:37.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.875+0000 7f8ce17fa640 1 -- 172.21.15.17:0/1971814951 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]}]=0 v104) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8cd8058860 con 0x7f8ce40c5330 2024-03-06T07:21:37.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 msgr2=0x7f8cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 0x7f8cc806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0036000 tx=0x7f8cd0030000 comp rx=0 tx=0).stop 2024-03-06T07:21:37.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 msgr2=0x7f8ce414e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c430 tx=0x7f8cd800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 shutdown_connections 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc806b4d0 0x7f8cc806d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ce414fa10 0x7f8ce41ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ce40c88b0 0x7f8ce40c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 --2- 172.21.15.17:0/1971814951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40c5330 0x7f8ce414e400 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 >> 172.21.15.17:0/1971814951 conn(0x7f8ce40ac560 msgr2=0x7f8ce40ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 shutdown_connections 2024-03-06T07:21:37.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:37.878+0000 7f8ceac2a640 1 -- 172.21.15.17:0/1971814951 wait complete. 2024-03-06T07:21:37.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:21:38.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.003+0000 7f5fba5bc640 1 Processor -- start 2024-03-06T07:21:38.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.003+0000 7f5fba5bc640 1 -- start start 2024-03-06T07:21:38.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fba5bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5fb40c67f0 0x7f5fb4155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fba5bc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fb40c4ef0 0x7f5fb4155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fba5bc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb4152cb0 con 0x7f5fb41528d0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fba5bc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb4155ca0 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fba5bc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb40c38b0 con 0x7f5fb40c67f0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fb40c4ef0 0x7f5fb4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fb40c4ef0 0x7f5fb4155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42166/0 (socket says 172.21.15.17:42166) 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 -- 172.21.15.17:0/3261595124 learned_addr learned my addr 172.21.15.17:0/3261595124 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb37fe640 1 --2- 172.21.15.17:0/3261595124 >> v2:172.21.15.163:3300/0 conn(0x7f5fb40c67f0 0x7f5fb4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 -- 172.21.15.17:0/3261595124 >> v2:172.21.15.163:3300/0 conn(0x7f5fb40c67f0 msgr2=0x7f5fb4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 --2- 172.21.15.17:0/3261595124 >> v2:172.21.15.163:3300/0 conn(0x7f5fb40c67f0 0x7f5fb4155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 -- 172.21.15.17:0/3261595124 >> v1:172.21.15.163:6789/0 conn(0x7f5fb41528d0 legacy=0x7f5fb40724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.004+0000 7f5fb8b59640 1 -- 172.21.15.17:0/3261595124 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb41594f0 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.005+0000 7f5fb8b59640 1 --2- 172.21.15.17:0/3261595124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb4155760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4074ff0 tx=0x7f5fac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2403e130655b501d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.005+0000 7f5fb2ffd640 1 -- 172.21.15.17:0/3261595124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac02cb20 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.005+0000 7f5fb2ffd640 1 -- 172.21.15.17:0/3261595124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac02ccc0 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.005+0000 7f5fb2ffd640 1 -- 172.21.15.17:0/3261595124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac034610 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 -- 172.21.15.17:0/3261595124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb40c4ef0 msgr2=0x7f5fb4155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/3261595124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb4155760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4074ff0 tx=0x7f5fac02bf80 comp rx=0 tx=0).stop 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 -- 172.21.15.17:0/3261595124 shutdown_connections 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/3261595124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb4155760 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/3261595124 >> v2:172.21.15.163:3300/0 conn(0x7f5fb40c67f0 0x7f5fb4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 -- 172.21.15.17:0/3261595124 >> 172.21.15.17:0/3261595124 conn(0x7f5fb40aa6a0 msgr2=0x7f5fb40aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 -- 172.21.15.17:0/3261595124 shutdown_connections 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.006+0000 7f5fba5bc640 1 -- 172.21.15.17:0/3261595124 wait complete. 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.007+0000 7f5fba5bc640 1 Processor -- start 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.007+0000 7f5fba5bc640 1 -- start start 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb40701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fb40c67f0 0x7f5fb41511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40a8250 con 0x7f5fb41528d0 2024-03-06T07:21:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40c3390 con 0x7f5fb40c4ef0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb41591a0 con 0x7f5fb40c67f0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42180/0 (socket says 172.21.15.17:42180) 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 -- 172.21.15.17:0/1062847162 learned_addr learned my addr 172.21.15.17:0/1062847162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb3fff640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb40701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb37fe640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fb40c67f0 0x7f5fb41511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 -- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fb40c67f0 msgr2=0x7f5fb41511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fb40c67f0 0x7f5fb41511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 -- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fb40c4ef0 msgr2=0x7f5fb40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb40701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 -- 172.21.15.17:0/1062847162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4154500 con 0x7f5fb41528d0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb8b59640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009f40 tx=0x7f5fac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac043020 con 0x7f5fb41528d0 2024-03-06T07:21:38.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac03e450 con 0x7f5fb41528d0 2024-03-06T07:21:38.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb41547e0 con 0x7f5fb41528d0 2024-03-06T07:21:38.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac009510 con 0x7f5fb41528d0 2024-03-06T07:21:38.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.008+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb4154da0 con 0x7f5fb41528d0 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.010+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb404f330 con 0x7f5fb41528d0 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.010+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fac046020 con 0x7f5fb41528d0 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.010+0000 7f5fb0ff9640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.010+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5fac0ac930 con 0x7f5fb41528d0 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.010+0000 7f5fb3fff640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.011+0000 7f5fb3fff640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 0x7f5f9406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004ea0 tx=0x7f5f9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.015+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5fac04d030 con 0x7f5fb41528d0 2024-03-06T07:21:38.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.197+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5fb4155500 con 0x7f5fb41528d0 2024-03-06T07:21:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.201+0000 7f5fb0ff9640 1 -- 172.21.15.17:0/1062847162 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5fac07b9f0 con 0x7f5fb41528d0 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 msgr2=0x7f5f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 0x7f5f9406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c004ea0 tx=0x7f5f9c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 msgr2=0x7f5fb413e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009f40 tx=0x7f5fac004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 shutdown_connections 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f9406b4d0 0x7f5f9406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fb41528d0 0x7f5fb413e970 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fb40c67f0 0x7f5fb41511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 --2- 172.21.15.17:0/1062847162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fb40c4ef0 0x7f5fb40701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.202+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 >> 172.21.15.17:0/1062847162 conn(0x7f5fb40aa6a0 msgr2=0x7f5fb40c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.203+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 shutdown_connections 2024-03-06T07:21:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.203+0000 7f5fba5bc640 1 -- 172.21.15.17:0/1062847162 wait complete. 2024-03-06T07:21:38.215 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:21:38.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:38.256 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:21:38.308 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:21:38.309 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:21:38.309 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:38.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:21:38.323 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:21:38.324 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:21:38.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:21:38.341 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:21:38.342 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:38.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:38.420 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:21:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 Processor -- start 2024-03-06T07:21:38.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 -- start start 2024-03-06T07:21:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7f80c88b0 0x7fb7f80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f8155110 con 0x7fb7f80c5330 2024-03-06T07:21:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80c8f40 con 0x7fb7f8158db0 2024-03-06T07:21:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7ff047640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f81559e0 con 0x7fb7f80c88b0 2024-03-06T07:21:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.464+0000 7fb7fcde3640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb7f80c5330 0x7fb7f81586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54300/0 (socket says 172.21.15.17:54300) 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7fcde3640 1 -- 172.21.15.17:0/741815326 learned_addr learned my addr 172.21.15.17:0/741815326 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7fd5e4640 1 --2- 172.21.15.17:0/741815326 >> v2:172.21.15.17:3300/0 conn(0x7fb7f80c88b0 0x7fb7f80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7f77fe640 1 -- 172.21.15.17:0/741815326 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551133402 0 0) 0x7fb7ec001110 con 0x7fb7f80c5330 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7f77fe640 1 -- 172.21.15.17:0/741815326 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e8003520 con 0x7fb7f80c5330 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7fd5e4640 1 -- 172.21.15.17:0/741815326 >> v1:172.21.15.163:6789/0 conn(0x7fb7f8158db0 legacy=0x7fb7f815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7fd5e4640 1 -- 172.21.15.17:0/741815326 >> v1:172.21.15.17:6789/0 conn(0x7fb7f80c5330 legacy=0x7fb7f81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.465+0000 7fb7fd5e4640 1 -- 172.21.15.17:0/741815326 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f8155390 con 0x7fb7f80c88b0 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7fd5e4640 1 --2- 172.21.15.17:0/741815326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f80c5710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009d90 tx=0x7fb7f002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5575a39fe8652e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7f77fe640 1 -- 172.21.15.17:0/741815326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0039070 con 0x7fb7f80c88b0 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7f77fe640 1 -- 172.21.15.17:0/741815326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f002cbd0 con 0x7fb7f80c88b0 2024-03-06T07:21:38.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7f77fe640 1 -- 172.21.15.17:0/741815326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0034440 con 0x7fb7f80c88b0 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 -- 172.21.15.17:0/741815326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 msgr2=0x7fb7f80c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 --2- 172.21.15.17:0/741815326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f80c5710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009d90 tx=0x7fb7f002be20 comp rx=0 tx=0).stop 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 -- 172.21.15.17:0/741815326 shutdown_connections 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 --2- 172.21.15.17:0/741815326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f80c5710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 -- 172.21.15.17:0/741815326 >> 172.21.15.17:0/741815326 conn(0x7fb7f80ac560 msgr2=0x7fb7f80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 -- 172.21.15.17:0/741815326 shutdown_connections 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.466+0000 7fb7ff047640 1 -- 172.21.15.17:0/741815326 wait complete. 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.467+0000 7fb7ff047640 1 Processor -- start 2024-03-06T07:21:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.467+0000 7fb7ff047640 1 -- start start 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 0x7fb7f81503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7f81eb1e0 0x7fb7f81e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81529d0 con 0x7fb7f80c88b0 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f806d090 con 0x7fb7f81eb1e0 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7ff047640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8152b50 con 0x7fb7f80c5330 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42204/0 (socket says 172.21.15.17:42204) 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.468+0000 7fb7f7fff640 1 -- 172.21.15.17:0/1933235581 learned_addr learned my addr 172.21.15.17:0/1933235581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7fcde3640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 0x7fb7f81503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7fd5e4640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7f81eb1e0 0x7fb7f81e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 -- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 msgr2=0x7fb7f81503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 0x7fb7f81503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 -- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7f81eb1e0 msgr2=0x7fb7f81e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7f81eb1e0 0x7fb7f81e0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 -- 172.21.15.17:0/1933235581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f8157230 con 0x7fb7f80c88b0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7fcde3640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 0x7fb7f81503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f7fff640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b3f0 tx=0x7fb7e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4015020 con 0x7fb7f80c88b0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4011830 con 0x7fb7f80c88b0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4011c20 con 0x7fb7f80c88b0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f8156f50 con 0x7fb7f80c88b0 2024-03-06T07:21:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.469+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f8151670 con 0x7fb7f80c88b0 2024-03-06T07:21:38.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.470+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f804f330 con 0x7fb7f80c88b0 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.471+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7e4018020 con 0x7fb7f80c88b0 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.471+0000 7fb7f57fa640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 0x7fb7dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.471+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb7e4089c50 con 0x7fb7f80c88b0 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.471+0000 7fb7fcde3640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 0x7fb7dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.472+0000 7fb7fcde3640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 0x7fb7dc06d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0012b0 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.476+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb7e408f020 con 0x7fb7f80c88b0 2024-03-06T07:21:38.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.695+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7f806cdc0 con 0x7fb7f80c88b0 2024-03-06T07:21:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.696+0000 7fb7f57fa640 1 -- 172.21.15.17:0/1933235581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fb7e401f030 con 0x7fb7f80c88b0 2024-03-06T07:21:38.698 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:38.699 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5239,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1442407901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1442407901},{"type":"v1","addr":"172.21.15.163:6836","nonce":1442407901}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5286,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1497992819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1497992819},{"type":"v1","addr":"172.21.15.17:6835","nonce":1497992819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5431,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/871521754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":871521754},{"type":"v1","addr":"172.21.15.163:6837","nonce":871521754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3644208790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3644208790},{"type":"v1","addr":"172.21.15.17:6834","nonce":3644208790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[]} 2024-03-06T07:21:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 25 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.697+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 msgr2=0x7fb7dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.697+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 0x7fb7dc06d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0012b0 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 msgr2=0x7fb7f81eaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b3f0 tx=0x7fb7e400b8b0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 shutdown_connections 2024-03-06T07:21:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb7dc06b500 0x7fb7dc06d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7f81eb1e0 0x7fb7f81e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7f80c88b0 0x7fb7f81eaca0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 --2- 172.21.15.17:0/1933235581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7f80c5330 0x7fb7f81503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 >> 172.21.15.17:0/1933235581 conn(0x7fb7f80ac560 msgr2=0x7fb7f80ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 shutdown_connections 2024-03-06T07:21:38.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.698+0000 7fb7ff047640 1 -- 172.21.15.17:0/1933235581 wait complete. 2024-03-06T07:21:38.710 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:21:38.710 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:21:38.710 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:21:38.711 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.711 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:21:38.711 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:21:38.711 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:21:38.711 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.711 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:21:38.711 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:21:38.711 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:21:38.712 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.712 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:21:38.712 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:21:38.712 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:21:38.712 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:21:38.727 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:21:38.727 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:21:38.728 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:21:38.728 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:21:38.760 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:21:38.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:21:38.761 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:21:38.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:21:38.770 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:21:38.770 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.771 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:21:38.771 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.802 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:21:38.803 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.803 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:21:38.803 DEBUG:teuthology.parallel:result is None 2024-03-06T07:21:38.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:21:38.853 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:21:38.854 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:21:38.854 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:21:38.853+0000 7f777de2f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:21:38.883 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:21:38.884 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:21:38.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.907+0000 7fa868393640 1 Processor -- start 2024-03-06T07:21:38.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.907+0000 7fa868393640 1 -- start start 2024-03-06T07:21:38.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.908+0000 7fa868393640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8600c6dd0 0x7fa860160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.908+0000 7fa868393640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa860155130 con 0x7fa8600c8950 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.908+0000 7fa868393640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600c7200 con 0x7fa8600c69f0 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.908+0000 7fa868393640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa860155440 con 0x7fa8600c6dd0 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa866930640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8600c6dd0 0x7fa860160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa86592e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8600c8950 0x7fa8601558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54306/0 (socket says 172.21.15.17:54306) 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa86592e640 1 -- 172.21.15.17:0/1936385066 learned_addr learned my addr 172.21.15.17:0/1936385066 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa866930640 1 -- 172.21.15.17:0/1936385066 >> v1:172.21.15.163:6789/0 conn(0x7fa8600c69f0 legacy=0x7fa8600acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa866930640 1 -- 172.21.15.17:0/1936385066 >> v1:172.21.15.17:6789/0 conn(0x7fa8600c8950 legacy=0x7fa8601558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa866930640 1 -- 172.21.15.17:0/1936385066 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600ad480 con 0x7fa8600c6dd0 2024-03-06T07:21:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.909+0000 7fa86512d640 1 -- 172.21.15.17:0/1936385066 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961414982 0 0) 0x7fa850001110 con 0x7fa8600c8950 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa866930640 1 --2- 172.21.15.17:0/1936385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c6dd0 0x7fa860160ac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8601555c0 tx=0x7fa85802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849c9e827a64eccb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa86512d640 1 -- 172.21.15.17:0/1936385066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858039070 con 0x7fa8600c6dd0 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa86512d640 1 -- 172.21.15.17:0/1936385066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85802cc50 con 0x7fa8600c6dd0 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa86512d640 1 -- 172.21.15.17:0/1936385066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858034440 con 0x7fa8600c6dd0 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa868393640 1 -- 172.21.15.17:0/1936385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c6dd0 msgr2=0x7fa860160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa868393640 1 --2- 172.21.15.17:0/1936385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c6dd0 0x7fa860160ac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa8601555c0 tx=0x7fa85802bea0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa868393640 1 -- 172.21.15.17:0/1936385066 shutdown_connections 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa868393640 1 --2- 172.21.15.17:0/1936385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c6dd0 0x7fa860160ac0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.910+0000 7fa868393640 1 -- 172.21.15.17:0/1936385066 >> 172.21.15.17:0/1936385066 conn(0x7fa8600cae70 msgr2=0x7fa8600cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.911+0000 7fa868393640 1 -- 172.21.15.17:0/1936385066 shutdown_connections 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.911+0000 7fa868393640 1 -- 172.21.15.17:0/1936385066 wait complete. 2024-03-06T07:21:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 Processor -- start 2024-03-06T07:21:38.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 -- start start 2024-03-06T07:21:38.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8600c69f0 0x7fa8600c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8600c6dd0 0x7fa8601574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c6310 con 0x7fa8600c8950 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c6030 con 0x7fa8600c6dd0 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa868393640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa860155fe0 con 0x7fa8600c69f0 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa866930640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa866930640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42246/0 (socket says 172.21.15.17:42246) 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.912+0000 7fa866930640 1 -- 172.21.15.17:0/2956917783 learned_addr learned my addr 172.21.15.17:0/2956917783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa86612f640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8600c69f0 0x7fa8600c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa86592e640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8600c6dd0 0x7fa8601574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 -- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8600c69f0 msgr2=0x7fa8600c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8600c69f0 0x7fa8600c2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 -- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8600c6dd0 msgr2=0x7fa8601574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:38.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8600c6dd0 0x7fa8601574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 -- 172.21.15.17:0/2956917783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860157a00 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa866930640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa858037040 tx=0x7fa858004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858039050 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858045070 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8600c30d0 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.913+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601eeb70 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.914+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85803e610 con 0x7fa8600c8950 2024-03-06T07:21:38.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.914+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86004f330 con 0x7fa8600c8950 2024-03-06T07:21:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.914+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa85803e7b0 con 0x7fa8600c8950 2024-03-06T07:21:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.915+0000 7fa856ffd640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 0x7fa84406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.915+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa8580acbd0 con 0x7fa8600c8950 2024-03-06T07:21:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.915+0000 7fa86612f640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 0x7fa84406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:38.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.916+0000 7fa86612f640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 0x7fa84406d9e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa85c000f60 tx=0x7fa85c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:38.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:38.919+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa858032e20 con 0x7fa8600c8950 2024-03-06T07:21:39.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.114+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa860150340 con 0x7fa8600c8950 2024-03-06T07:21:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.883+0000 7fa856ffd640 1 -- 172.21.15.17:0/2956917783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa85807bce0 con 0x7fa8600c8950 2024-03-06T07:21:39.885 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 msgr2=0x7fa84406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 0x7fa84406d9e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa85c000f60 tx=0x7fa85c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 msgr2=0x7fa860156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa858037040 tx=0x7fa858004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 shutdown_connections 2024-03-06T07:21:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84406b520 0x7fa84406d9e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8600c8950 0x7fa860156520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8600c6dd0 0x7fa8601574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 --2- 172.21.15.17:0/2956917783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8600c69f0 0x7fa8600c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 >> 172.21.15.17:0/2956917783 conn(0x7fa8600cae70 msgr2=0x7fa8600c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 shutdown_connections 2024-03-06T07:21:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:39.887+0000 7fa868393640 1 -- 172.21.15.17:0/2956917783 wait complete. 2024-03-06T07:21:39.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:21:40.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.008+0000 7f02019f7640 1 Processor -- start 2024-03-06T07:21:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.008+0000 7f02019f7640 1 -- start start 2024-03-06T07:21:40.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f02019f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01fc0c8a00 0x7f01fc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f02019f7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0c7760 con 0x7f01fc159460 2024-03-06T07:21:40.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f02019f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0c7a40 con 0x7f01fc152920 2024-03-06T07:21:40.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f02019f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc1587b0 con 0x7f01fc0c8a00 2024-03-06T07:21:40.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01fc159460 0x7f01fc0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54318/0 (socket says 172.21.15.17:54318) 2024-03-06T07:21:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1049313581 learned_addr learned my addr 172.21.15.17:0/1049313581 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> v2:172.21.15.163:3300/0 conn(0x7f01fc0c8a00 0x7f01fc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853904268 0 0) 0x7f01fc0c7760 con 0x7f01fc159460 2024-03-06T07:21:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e8003520 con 0x7f01fc159460 2024-03-06T07:21:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815132011 0 0) 0x7f01e8003520 con 0x7f01fc159460 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 >> v2:172.21.15.163:3300/0 conn(0x7f01fc0c8a00 msgr2=0x7f01fc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 --2- 172.21.15.17:0/1049313581 >> v2:172.21.15.163:3300/0 conn(0x7f01fc0c8a00 0x7f01fc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 >> v1:172.21.15.163:6789/0 conn(0x7f01fc152920 legacy=0x7f01fc155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0c7d20 con 0x7f01fc159460 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f01f40019f0 con 0x7f01fc159460 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> v2:172.21.15.163:3300/0 conn(0x7f01fc0c8a00 0x7f01fc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 >> v1:172.21.15.17:6789/0 conn(0x7f01fc159460 legacy=0x7f01fc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e80039b0 0x7f01e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 0x7f01e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01e80051c0 0x7f01e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc1587b0 con 0x7f01e8004800 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e800c160 con 0x7f01e80051c0 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e800c360 con 0x7f01e80039b0 2024-03-06T07:21:40.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.009+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 0x7f01e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01fb7fe640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01e80051c0 0x7f01e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01faffd640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e80039b0 0x7f01e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 -- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e80039b0 msgr2=0x7f01e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e80039b0 0x7f01e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 -- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01e80051c0 msgr2=0x7f01e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01e80051c0 0x7f01e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 -- 172.21.15.17:0/1049313581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8003520 con 0x7f01e8004800 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1049313581 reap_dead start 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.010+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 0x7f01e8004c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c90 tx=0x7f01e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca37f0731439ba28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4039070 con 0x7f01e8004800 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e402c8f0 con 0x7f01e8004800 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f01fa7fc640 1 -- 172.21.15.17:0/1049313581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4009860 con 0x7f01e8004800 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 -- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 msgr2=0x7f01e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 0x7f01e8004c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009c90 tx=0x7f01e8005830 comp rx=0 tx=0).stop 2024-03-06T07:21:40.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 -- 172.21.15.17:0/1049313581 shutdown_connections 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 --2- 172.21.15.17:0/1049313581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01e8004800 0x7f01e8004c80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 -- 172.21.15.17:0/1049313581 >> 172.21.15.17:0/1049313581 conn(0x7f01fc0ac560 msgr2=0x7f01fc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 -- 172.21.15.17:0/1049313581 shutdown_connections 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.011+0000 7f02019f7640 1 -- 172.21.15.17:0/1049313581 wait complete. 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.012+0000 7f02019f7640 1 Processor -- start 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.012+0000 7f02019f7640 1 -- start start 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e8004800 0x7f01fc14f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0c35c0 0x7f01fc14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c7390 con 0x7f01fc147af0 2024-03-06T07:21:40.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc06d030 con 0x7f01fc0c35c0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f02019f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c7510 con 0x7f01e8004800 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42260/0 (socket says 172.21.15.17:42260) 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1882601397 learned_addr learned my addr 172.21.15.17:0/1882601397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01f3fff640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0c35c0 0x7f01fc14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01faffd640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e8004800 0x7f01fc14f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e8004800 msgr2=0x7f01fc14f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e8004800 0x7f01fc14f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0c35c0 msgr2=0x7f01fc14cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0c35c0 0x7f01fc14cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.013+0000 7f01fb7fe640 1 -- 172.21.15.17:0/1882601397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0c2530 con 0x7f01fc147af0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f01fb7fe640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01fc06b070 tx=0x7f01f4007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400e9c0 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f400eb80 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc0c27d0 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f400ed70 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.014+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0c3e50 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.015+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc04f330 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.015+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01f401a020 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.016+0000 7f01f8ff9640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 0x7f01dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.016+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f01f4089a00 con 0x7f01fc147af0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.016+0000 7f01faffd640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 0x7f01dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.016+0000 7f01faffd640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 0x7f01dc06d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.020+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f01f4021020 con 0x7f01fc147af0 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.270+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f01fc0c29a0 con 0x7f01fc147af0 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.270+0000 7f01f8ff9640 1 -- 172.21.15.17:0/1882601397 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f01f4058a40 con 0x7f01fc147af0 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:21:40.272 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:40.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:21:40.278 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:21:40.280 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:21:40.283 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.285 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:40.286 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:21:40.287 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:40.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 msgr2=0x7f01dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 0x7f01dc06d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec009040 comp rx=0 tx=0).stop 2024-03-06T07:21:40.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 msgr2=0x7f01fc0c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f01fc06b070 tx=0x7f01f4007af0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 shutdown_connections 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01dc06b4d0 0x7f01dc06d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01fc147af0 0x7f01fc0c4370 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0c35c0 0x7f01fc14cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 --2- 172.21.15.17:0/1882601397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01e8004800 0x7f01fc14f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 >> 172.21.15.17:0/1882601397 conn(0x7f01fc0ac560 msgr2=0x7f01fc14b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 shutdown_connections 2024-03-06T07:21:40.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.272+0000 7f02019f7640 1 -- 172.21.15.17:0/1882601397 wait complete. 2024-03-06T07:21:40.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:40.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.391+0000 7f9dba7f7640 1 Processor -- start 2024-03-06T07:21:40.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.391+0000 7f9dba7f7640 1 -- start start 2024-03-06T07:21:40.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.391+0000 7f9dba7f7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9db4152520 0x7f9db406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.391+0000 7f9dba7f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db40c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9dba7f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40c1ae0 con 0x7f9db40720a0 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9dba7f7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db40c1dc0 con 0x7f9db4152520 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9dba7f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db406e450 con 0x7f9db4155c80 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9db4152520 0x7f9db406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9db4152520 0x7f9db406ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42262/0 (socket says 172.21.15.17:42262) 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 -- 172.21.15.17:0/3804667049 learned_addr learned my addr 172.21.15.17:0/3804667049 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 -- 172.21.15.17:0/3804667049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 msgr2=0x7f9db40c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db37fe640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db40c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db40c12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 -- 172.21.15.17:0/3804667049 >> v1:172.21.15.163:6789/0 conn(0x7f9db40720a0 legacy=0x7f9db406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db3fff640 1 -- 172.21.15.17:0/3804667049 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db406e730 con 0x7f9db4152520 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.392+0000 7f9db37fe640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db40c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:40.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9db3fff640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db406ef30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009870 tx=0x7f9da402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574833d82d682636 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9db2ffd640 1 -- 172.21.15.17:0/3804667049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4035020 con 0x7f9db4152520 2024-03-06T07:21:40.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9db2ffd640 1 -- 172.21.15.17:0/3804667049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da4031640 con 0x7f9db4152520 2024-03-06T07:21:40.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9db2ffd640 1 -- 172.21.15.17:0/3804667049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da402d040 con 0x7f9db4152520 2024-03-06T07:21:40.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 -- 172.21.15.17:0/3804667049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 msgr2=0x7f9db406ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db406ef30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009870 tx=0x7f9da402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 -- 172.21.15.17:0/3804667049 shutdown_connections 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db40c12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/3804667049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db406ef30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.393+0000 7f9dba7f7640 1 -- 172.21.15.17:0/3804667049 >> 172.21.15.17:0/3804667049 conn(0x7f9db40c76b0 msgr2=0x7f9db40c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.394+0000 7f9dba7f7640 1 -- 172.21.15.17:0/3804667049 shutdown_connections 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.394+0000 7f9dba7f7640 1 -- 172.21.15.17:0/3804667049 wait complete. 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 Processor -- start 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 -- start start 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9db40720a0 0x7f9db414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db406b6d0 con 0x7f9db4152520 2024-03-06T07:21:40.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40c17f0 con 0x7f9db40720a0 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.395+0000 7f9dba7f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db406b850 con 0x7f9db4155c80 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42268/0 (socket says 172.21.15.17:42268) 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 -- 172.21.15.17:0/2823481686 learned_addr learned my addr 172.21.15.17:0/2823481686 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db3fff640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9db40720a0 0x7f9db414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db8d94640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 -- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 msgr2=0x7f9db4156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db4156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 -- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9db40720a0 msgr2=0x7f9db414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9db40720a0 0x7f9db414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 -- 172.21.15.17:0/2823481686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db4158210 con 0x7f9db4152520 2024-03-06T07:21:40.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db37fe640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9da800c7f0 tx=0x7f9da800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8015070 con 0x7f9db4152520 2024-03-06T07:21:40.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da8007a30 con 0x7f9db4152520 2024-03-06T07:21:40.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db41eeea0 con 0x7f9db4152520 2024-03-06T07:21:40.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8007e00 con 0x7f9db4152520 2024-03-06T07:21:40.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.396+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db41eeb30 con 0x7f9db4152520 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.397+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db404f330 con 0x7f9db4152520 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.398+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9da8018070 con 0x7f9db4152520 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.398+0000 7f9db0ff9640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.398+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9da8089ff0 con 0x7f9db4152520 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.398+0000 7f9db3fff640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.399+0000 7f9db3fff640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9da4036000 tx=0x7f9da403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.402+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9da8059030 con 0x7f9db4152520 2024-03-06T07:21:40.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.584+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db40ab390 con 0x7f9db4152520 2024-03-06T07:21:40.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.585+0000 7f9db0ff9640 1 -- 172.21.15.17:0/2823481686 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13164 (secure 0 0 0) 0x7f9da8058930 con 0x7f9db4152520 2024-03-06T07:21:40.586 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:40.586 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":47,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:39.882167+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:40.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 msgr2=0x7f9da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9da4036000 tx=0x7f9da403c040 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 msgr2=0x7f9db41565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9da800c7f0 tx=0x7f9da800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 shutdown_connections 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9db4155c80 0x7f9db4156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9db4152520 0x7f9db41565d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 --2- 172.21.15.17:0/2823481686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9db40720a0 0x7f9db414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.586+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 >> 172.21.15.17:0/2823481686 conn(0x7f9db40c76b0 msgr2=0x7f9db40c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.587+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 shutdown_connections 2024-03-06T07:21:40.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.587+0000 7f9dba7f7640 1 -- 172.21.15.17:0/2823481686 wait complete. 2024-03-06T07:21:40.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:40.600 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:40.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 Processor -- start 2024-03-06T07:21:40.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 -- start start 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5d40a8290 0x7fa5d41539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 0x7fa5d4072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40c62a0 con 0x7fa5d40c4740 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4153f00 con 0x7fa5d40a8290 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d8c78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40c4c00 con 0x7fa5d40c69a0 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5d259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5d40a8290 0x7fa5d41539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5cbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa5d40c4740 0x7fa5d40701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39558/0 (socket says 172.21.15.17:39558) 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.703+0000 7fa5cbfff640 1 -- 172.21.15.17:0/1563446079 learned_addr learned my addr 172.21.15.17:0/1563446079 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d2d9d640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 0x7fa5d4072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d259c640 1 -- 172.21.15.17:0/1563446079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 msgr2=0x7fa5d4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d259c640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 0x7fa5d4072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d259c640 1 -- 172.21.15.17:0/1563446079 >> v1:172.21.15.17:6789/0 conn(0x7fa5d40c4740 legacy=0x7fa5d40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d259c640 1 -- 172.21.15.17:0/1563446079 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40c4ee0 con 0x7fa5d40a8290 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d1d9b640 1 -- 172.21.15.17:0/1563446079 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456976106 0 0) 0x7fa5d40c62a0 con 0x7fa5d40c4740 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d2d9d640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 0x7fa5d4072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.704+0000 7fa5d259c640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40a8290 0x7fa5d41539c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40097a0 tx=0x7fa5c402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c46abc07651d7dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d1d9b640 1 -- 172.21.15.17:0/1563446079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4038070 con 0x7fa5d40a8290 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d1d9b640 1 -- 172.21.15.17:0/1563446079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c4031700 con 0x7fa5d40a8290 2024-03-06T07:21:40.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d1d9b640 1 -- 172.21.15.17:0/1563446079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4031ad0 con 0x7fa5d40a8290 2024-03-06T07:21:40.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1563446079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40a8290 msgr2=0x7fa5d41539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40a8290 0x7fa5d41539c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40097a0 tx=0x7fa5c402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1563446079 shutdown_connections 2024-03-06T07:21:40.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40c69a0 0x7fa5d4072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1563446079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40a8290 0x7fa5d41539c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1563446079 >> 172.21.15.17:0/1563446079 conn(0x7fa5d40c5560 msgr2=0x7fa5d40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1563446079 shutdown_connections 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.705+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1563446079 wait complete. 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.706+0000 7fa5d8c78640 1 Processor -- start 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 -- start start 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40a8290 0x7fa5d41ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5d40c4740 0x7fa5d41ec460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40c6e30 con 0x7fa5d40c69a0 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406fc80 con 0x7fa5d40c4740 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d8c78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4072bc0 con 0x7fa5d40a8290 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37346/0 (socket says 172.21.15.17:37346) 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d2d9d640 1 -- 172.21.15.17:0/1982721280 learned_addr learned my addr 172.21.15.17:0/1982721280 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5cbfff640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5d40c4740 0x7fa5d41ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.707+0000 7fa5d259c640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40a8290 0x7fa5d41ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 -- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40a8290 msgr2=0x7fa5d41ebf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40a8290 0x7fa5d41ebf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 -- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5d40c4740 msgr2=0x7fa5d41ec460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5d40c4740 0x7fa5d41ec460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 -- 172.21.15.17:0/1982721280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d41e9bb0 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d2d9d640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00cd50 tx=0x7fa5cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00f680 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc00f840 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d41ecb50 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc00fc30 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.708+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41dce30 con 0x7fa5d40c69a0 2024-03-06T07:21:40.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.709+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d41e9d80 con 0x7fa5d40c69a0 2024-03-06T07:21:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.709+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5cc0213f0 con 0x7fa5d40c69a0 2024-03-06T07:21:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.710+0000 7fa5caffd640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 0x7fa5b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.710+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa5cc08ad80 con 0x7fa5d40c69a0 2024-03-06T07:21:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.710+0000 7fa5d259c640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 0x7fa5b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.710+0000 7fa5d259c640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 0x7fa5b406d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4035000 tx=0x7fa5c4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:40.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.714+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa5cc014030 con 0x7fa5d40c69a0 2024-03-06T07:21:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.894+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5d404f330 con 0x7fa5d40c69a0 2024-03-06T07:21:40.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.895+0000 7fa5caffd640 1 -- 172.21.15.17:0/1982721280 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5cc059e50 con 0x7fa5d40c69a0 2024-03-06T07:21:40.899 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 msgr2=0x7fa5b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 0x7fa5b406d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4035000 tx=0x7fa5c4008030 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 msgr2=0x7fa5d41ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00cd50 tx=0x7fa5cc00a650 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 shutdown_connections 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5b406b4d0 0x7fa5b406d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5d40c69a0 0x7fa5d41ea090 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5d40c4740 0x7fa5d41ec460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 --2- 172.21.15.17:0/1982721280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5d40a8290 0x7fa5d41ebf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:40.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 >> 172.21.15.17:0/1982721280 conn(0x7fa5d40c5560 msgr2=0x7fa5d41562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:40.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 shutdown_connections 2024-03-06T07:21:40.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:40.897+0000 7fa5d8c78640 1 -- 172.21.15.17:0/1982721280 wait complete. 2024-03-06T07:21:40.909 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:21:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.009+0000 7f8e89288640 1 Processor -- start 2024-03-06T07:21:41.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.009+0000 7f8e89288640 1 -- start start 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e89288640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 0x7f8e84155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e89288640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e89288640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e840c5cc0 con 0x7f8e840c5860 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e89288640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8406df40 con 0x7f8e840c5480 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e89288640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8406fdb0 con 0x7f8e841586f0 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e82ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 0x7f8e84155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e827fc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f8e840c5860 0x7f8e8406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:50418/0 (socket says 172.21.15.17:50418) 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e837fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:56484/0 (socket says 172.21.15.17:56484) 2024-03-06T07:21:41.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.010+0000 7f8e827fc640 1 -- 172.21.15.17:0/499899356 learned_addr learned my addr 172.21.15.17:0/499899356 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e81ffb640 1 -- 172.21.15.17:0/499899356 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216649667 0 0) 0x7f8e840c5cc0 con 0x7f8e840c5860 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e81ffb640 1 -- 172.21.15.17:0/499899356 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e70003520 con 0x7f8e840c5860 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e837fe640 1 -- 172.21.15.17:0/499899356 >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 msgr2=0x7f8e84155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e837fe640 1 --2- 172.21.15.17:0/499899356 >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 0x7f8e84155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e837fe640 1 -- 172.21.15.17:0/499899356 >> v1:172.21.15.163:6789/0 conn(0x7f8e840c5860 legacy=0x7f8e8406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e837fe640 1 -- 172.21.15.17:0/499899356 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e841591c0 con 0x7f8e841586f0 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.011+0000 7f8e82ffd640 1 --2- 172.21.15.17:0/499899356 >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 0x7f8e84155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:41.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e837fe640 1 --2- 172.21.15.17:0/499899356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097a0 tx=0x7f8e7c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ce55ac0d7e1d8a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e81ffb640 1 -- 172.21.15.17:0/499899356 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0315a0 con 0x7f8e841586f0 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e81ffb640 1 -- 172.21.15.17:0/499899356 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c037070 con 0x7f8e841586f0 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e81ffb640 1 -- 172.21.15.17:0/499899356 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c031960 con 0x7f8e841586f0 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 -- 172.21.15.17:0/499899356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 msgr2=0x7f8e84158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 --2- 172.21.15.17:0/499899356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097a0 tx=0x7f8e7c02b840 comp rx=0 tx=0).stop 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 -- 172.21.15.17:0/499899356 shutdown_connections 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 --2- 172.21.15.17:0/499899356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e84158b80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 --2- 172.21.15.17:0/499899356 >> v2:172.21.15.163:3300/0 conn(0x7f8e840c5480 0x7f8e84155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 -- 172.21.15.17:0/499899356 >> 172.21.15.17:0/499899356 conn(0x7f8e8406b1a0 msgr2=0x7f8e8406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 -- 172.21.15.17:0/499899356 shutdown_connections 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.012+0000 7f8e89288640 1 -- 172.21.15.17:0/499899356 wait complete. 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 Processor -- start 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 -- start start 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e840c5860 0x7f8e840c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e8414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84155680 con 0x7f8e840c5480 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841553a0 con 0x7f8e840c5860 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e89288640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84152a20 con 0x7f8e841586f0 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e82ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e82ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37360/0 (socket says 172.21.15.17:37360) 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.014+0000 7f8e82ffd640 1 -- 172.21.15.17:0/3101522531 learned_addr learned my addr 172.21.15.17:0/3101522531 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e837fe640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e8414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e827fc640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e840c5860 0x7f8e840c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 -- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 msgr2=0x7f8e8414a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e8414a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 -- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e840c5860 msgr2=0x7f8e840c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e840c5860 0x7f8e840c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 -- 172.21.15.17:0/3101522531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e841ee4e0 con 0x7f8e840c5480 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e82ffd640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400efc0 tx=0x7f8e7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74018020 con 0x7f8e840c5480 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e74014680 con 0x7f8e840c5480 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e841ee7c0 con 0x7f8e840c5480 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74014a50 con 0x7f8e840c5480 2024-03-06T07:21:41.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.015+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e84070dd0 con 0x7f8e840c5480 2024-03-06T07:21:41.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.016+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8406fa10 con 0x7f8e840c5480 2024-03-06T07:21:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.016+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e74005d10 con 0x7f8e840c5480 2024-03-06T07:21:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.017+0000 7f8e7b7fe640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 0x7f8e6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.017+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8e7408aea0 con 0x7f8e840c5480 2024-03-06T07:21:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.017+0000 7f8e827fc640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 0x7f8e6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.018+0000 7f8e827fc640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 0x7f8e6406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e841ef2c0 tx=0x7f8e6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.021+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e8406fa10 con 0x7f8e840c5480 2024-03-06T07:21:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.203+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8e8404f330 con 0x7f8e840c5480 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.203+0000 7f8e7b7fe640 1 -- 172.21.15.17:0/3101522531 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8e74022030 con 0x7f8e840c5480 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:41.206 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:21:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 msgr2=0x7f8e6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 0x7f8e6406d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e841ef2c0 tx=0x7f8e6c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 msgr2=0x7f8e840ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400efc0 tx=0x7f8e7400c490 comp rx=0 tx=0).stop 2024-03-06T07:21:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 shutdown_connections 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e6406b4d0 0x7f8e6406d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e841586f0 0x7f8e8414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e840c5860 0x7f8e840c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 --2- 172.21.15.17:0/3101522531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e840c5480 0x7f8e840ac160 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 >> 172.21.15.17:0/3101522531 conn(0x7f8e8406b1a0 msgr2=0x7f8e8406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 shutdown_connections 2024-03-06T07:21:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.205+0000 7f8e89288640 1 -- 172.21.15.17:0/3101522531 wait complete. 2024-03-06T07:21:41.218 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:21:41.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:21:41.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.321+0000 7ff3b13b6640 1 Processor -- start 2024-03-06T07:21:41.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.321+0000 7ff3b13b6640 1 -- start start 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3b13b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3ac152870 0x7ff3ac155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3b13b6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff3ac0c88e0 0x7ff3ac0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3b13b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ac156100 con 0x7ff3ac0c7930 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3b13b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac06d320 con 0x7ff3ac152870 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3b13b6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ac06b8d0 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3aaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3ac152870 0x7ff3ac155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3ab7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff3ac0c7930 0x7ff3ac06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39574/0 (socket says 172.21.15.17:39574) 2024-03-06T07:21:41.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.322+0000 7ff3ab7fe640 1 -- 172.21.15.17:0/1650189588 learned_addr learned my addr 172.21.15.17:0/1650189588 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3aaffd640 1 -- 172.21.15.17:0/1650189588 >> v2:172.21.15.163:3300/0 conn(0x7ff3ac0c88e0 msgr2=0x7ff3ac0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/1650189588 >> v2:172.21.15.163:3300/0 conn(0x7ff3ac0c88e0 0x7ff3ac0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3aaffd640 1 -- 172.21.15.17:0/1650189588 >> v1:172.21.15.17:6789/0 conn(0x7ff3ac0c7930 legacy=0x7ff3ac06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3aaffd640 1 -- 172.21.15.17:0/1650189588 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0c5400 con 0x7ff3ac152870 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3a9ffb640 1 -- 172.21.15.17:0/1650189588 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263025854 0 0) 0x7ff3ac156100 con 0x7ff3ac0c7930 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/1650189588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac152870 0x7ff3ac155bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0cba40 tx=0x7ff39c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e175c057ed7a1931 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.323+0000 7ff3a9ffb640 1 -- 172.21.15.17:0/1650189588 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c039070 con 0x7ff3ac152870 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3a9ffb640 1 -- 172.21.15.17:0/1650189588 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c02cbf0 con 0x7ff3ac152870 2024-03-06T07:21:41.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3a9ffb640 1 -- 172.21.15.17:0/1650189588 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c034440 con 0x7ff3ac152870 2024-03-06T07:21:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 -- 172.21.15.17:0/1650189588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac152870 msgr2=0x7ff3ac155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/1650189588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac152870 0x7ff3ac155bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0cba40 tx=0x7ff39c02be40 comp rx=0 tx=0).stop 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 -- 172.21.15.17:0/1650189588 shutdown_connections 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/1650189588 >> v2:172.21.15.163:3300/0 conn(0x7ff3ac0c88e0 0x7ff3ac0c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/1650189588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac152870 0x7ff3ac155bc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 -- 172.21.15.17:0/1650189588 >> 172.21.15.17:0/1650189588 conn(0x7ff3ac0ac5f0 msgr2=0x7ff3ac0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 -- 172.21.15.17:0/1650189588 shutdown_connections 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.324+0000 7ff3b13b6640 1 -- 172.21.15.17:0/1650189588 wait complete. 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.325+0000 7ff3b13b6640 1 Processor -- start 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.325+0000 7ff3b13b6640 1 -- start start 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 0x7ff3ac14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 0x7ff3ac14e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c5ca0 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c59c0 con 0x7ff3ac0c7930 2024-03-06T07:21:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3b13b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac0c9470 con 0x7ff3ac152870 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37374/0 (socket says 172.21.15.17:37374) 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 -- 172.21.15.17:0/3271006813 learned_addr learned my addr 172.21.15.17:0/3271006813 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 0x7ff3ac14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3ab7fe640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 0x7ff3ac14e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 -- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 msgr2=0x7ff3ac14e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 0x7ff3ac14e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 -- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 msgr2=0x7ff3ac14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 0x7ff3ac14f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aa7fc640 1 -- 172.21.15.17:0/3271006813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac1ee980 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 0x7ff3ac14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.326+0000 7ff3ab7fe640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 0x7ff3ac14e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3aa7fc640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000b450 tx=0x7ff3a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0015020 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0011890 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac1ea960 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011c80 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.327+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac1eac20 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.328+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac04f330 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.328+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3a0018020 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.329+0000 7ff3937fe640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 0x7ff39806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.329+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff3a008a100 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.329+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 0x7ff39806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:41.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.329+0000 7ff3aaffd640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 0x7ff39806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff39c036000 tx=0x7ff39c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.333+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3a001f020 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.514+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff3ac06cf70 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.888+0000 7ff3937fe640 1 -- 172.21.15.17:0/3271006813 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff3a000d040 con 0x7ff3ac0c88e0 2024-03-06T07:21:41.890 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:21:41.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 msgr2=0x7ff39806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 0x7ff39806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff39c036000 tx=0x7ff39c030000 comp rx=0 tx=0).stop 2024-03-06T07:21:41.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 msgr2=0x7ff3ac1eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:41.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000b450 tx=0x7ff3a000b910 comp rx=0 tx=0).stop 2024-03-06T07:21:41.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 shutdown_connections 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff39806b4d0 0x7ff39806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3ac152870 0x7ff3ac14e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3ac0c88e0 0x7ff3ac1eb420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 --2- 172.21.15.17:0/3271006813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3ac0c7930 0x7ff3ac14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.890+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 >> 172.21.15.17:0/3271006813 conn(0x7ff3ac0ac5f0 msgr2=0x7ff3ac0ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.891+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 shutdown_connections 2024-03-06T07:21:41.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:41.891+0000 7ff3b13b6640 1 -- 172.21.15.17:0/3271006813 wait complete. 2024-03-06T07:21:41.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:21:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.021+0000 7fba5613e640 1 Processor -- start 2024-03-06T07:21:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.021+0000 7fba5613e640 1 -- start start 2024-03-06T07:21:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba5613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 0x7fba50155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba5613e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba5006ca70 0x7fba500c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba5613e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba500b0450 con 0x7fba500c5380 2024-03-06T07:21:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba5613e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5006c020 con 0x7fba5006ca70 2024-03-06T07:21:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba5613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba500c77f0 con 0x7fba500b0070 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba4f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 0x7fba50155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba4ffff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fba500c5380 0x7fba5014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:50420/0 (socket says 172.21.15.17:50420) 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba4ffff640 1 -- 172.21.15.17:0/3245775210 learned_addr learned my addr 172.21.15.17:0/3245775210 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.022+0000 7fba4effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba5006ca70 0x7fba500c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.023+0000 7fba4f7fe640 1 -- 172.21.15.17:0/3245775210 >> v2:172.21.15.163:3300/0 conn(0x7fba5006ca70 msgr2=0x7fba500c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.023+0000 7fba4f7fe640 1 --2- 172.21.15.17:0/3245775210 >> v2:172.21.15.163:3300/0 conn(0x7fba5006ca70 0x7fba500c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.023+0000 7fba4f7fe640 1 -- 172.21.15.17:0/3245775210 >> v1:172.21.15.163:6789/0 conn(0x7fba500c5380 legacy=0x7fba5014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.023+0000 7fba4f7fe640 1 -- 172.21.15.17:0/3245775210 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500c7ad0 con 0x7fba500b0070 2024-03-06T07:21:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba4f7fe640 1 --2- 172.21.15.17:0/3245775210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 0x7fba50155c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba400097a0 tx=0x7fba4002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b3b08f514d3b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba4e7fc640 1 -- 172.21.15.17:0/3245775210 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400315a0 con 0x7fba500b0070 2024-03-06T07:21:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba4e7fc640 1 -- 172.21.15.17:0/3245775210 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40037070 con 0x7fba500b0070 2024-03-06T07:21:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba4e7fc640 1 -- 172.21.15.17:0/3245775210 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba40031960 con 0x7fba500b0070 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 -- 172.21.15.17:0/3245775210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 msgr2=0x7fba50155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 --2- 172.21.15.17:0/3245775210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 0x7fba50155c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba400097a0 tx=0x7fba4002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 -- 172.21.15.17:0/3245775210 shutdown_connections 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 --2- 172.21.15.17:0/3245775210 >> v2:172.21.15.163:3300/0 conn(0x7fba5006ca70 0x7fba500c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 --2- 172.21.15.17:0/3245775210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500b0070 0x7fba50155c30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 -- 172.21.15.17:0/3245775210 >> 172.21.15.17:0/3245775210 conn(0x7fba500ac590 msgr2=0x7fba500ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 -- 172.21.15.17:0/3245775210 shutdown_connections 2024-03-06T07:21:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.024+0000 7fba5613e640 1 -- 172.21.15.17:0/3245775210 wait complete. 2024-03-06T07:21:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.025+0000 7fba5613e640 1 Processor -- start 2024-03-06T07:21:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 -- start start 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba5006ca70 0x7fba501eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500c5380 0x7fba501542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5006cef0 con 0x7fba500b0070 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5006c550 con 0x7fba5006ca70 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba5613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5006da80 con 0x7fba500c5380 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.026+0000 7fba4effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37388/0 (socket says 172.21.15.17:37388) 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 -- 172.21.15.17:0/124308526 learned_addr learned my addr 172.21.15.17:0/124308526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4f7fe640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba5006ca70 0x7fba501eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4ffff640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500c5380 0x7fba501542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 -- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500c5380 msgr2=0x7fba501542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500c5380 0x7fba501542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 -- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba5006ca70 msgr2=0x7fba501eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba5006ca70 0x7fba501eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 -- 172.21.15.17:0/124308526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba501eb090 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba4effd640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba4400e9b0 tx=0x7fba4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44017070 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4400c9a0 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba500c47a0 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4400cd90 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.027+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba501566d0 con 0x7fba500b0070 2024-03-06T07:21:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.028+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5004f330 con 0x7fba500b0070 2024-03-06T07:21:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.029+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba4401a070 con 0x7fba500b0070 2024-03-06T07:21:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.029+0000 7fba37fff640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.029+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fba4408a0b0 con 0x7fba500b0070 2024-03-06T07:21:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.029+0000 7fba4f7fe640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.030+0000 7fba4f7fe640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 0x7fba3c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba40036000 tx=0x7fba4003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:42.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.034+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba440589c0 con 0x7fba500b0070 2024-03-06T07:21:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.215+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fba500c5b10 con 0x7fba500b0070 2024-03-06T07:21:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.892+0000 7fba37fff640 1 -- 172.21.15.17:0/124308526 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7fba4405ca60 con 0x7fba500b0070 2024-03-06T07:21:42.895 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:21:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.897+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 msgr2=0x7fba3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.897+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 0x7fba3c06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba40036000 tx=0x7fba4003d040 comp rx=0 tx=0).stop 2024-03-06T07:21:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.897+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 msgr2=0x7fba501eab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:42.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.897+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fba4400e9b0 tx=0x7fba4400ee70 comp rx=0 tx=0).stop 2024-03-06T07:21:42.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 shutdown_connections 2024-03-06T07:21:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba3c06b4d0 0x7fba3c06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba500c5380 0x7fba501542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba500b0070 0x7fba501eab50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 --2- 172.21.15.17:0/124308526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba5006ca70 0x7fba501eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:42.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.898+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 >> 172.21.15.17:0/124308526 conn(0x7fba500ac590 msgr2=0x7fba500ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.907+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 shutdown_connections 2024-03-06T07:21:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:42.910+0000 7fba5613e640 1 -- 172.21.15.17:0/124308526 wait complete. 2024-03-06T07:21:42.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:21:43.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.037+0000 7fbd0c18c640 1 Processor -- start 2024-03-06T07:21:43.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.037+0000 7fbd0c18c640 1 -- start start 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd0c18c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbd040c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd0c18c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd04072630 con 0x7fbd040c6780 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd0c18c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd04072910 con 0x7fbd0406da40 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd0c18c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0406de20 con 0x7fbd040c7380 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd09727640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbd040c6780 0x7fbd04158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39584/0 (socket says 172.21.15.17:39584) 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd09727640 1 -- 172.21.15.17:0/1149629942 learned_addr learned my addr 172.21.15.17:0/1149629942 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:43.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd0a729640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbd040c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344620105 0 0) 0x7fbd04072630 con 0x7fbd040c6780 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcf0003520 con 0x7fbd040c6780 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248331906 0 0) 0x7fbcf0003520 con 0x7fbd040c6780 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 msgr2=0x7fbd040c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbd040c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 >> v1:172.21.15.163:6789/0 conn(0x7fbd0406da40 legacy=0x7fbd04155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04072bf0 con 0x7fbd040c6780 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbcf4001670 con 0x7fbd040c6780 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 >> v1:172.21.15.17:6789/0 conn(0x7fbd040c6780 legacy=0x7fbd04158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbcf00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf0003f60 0x7fbcf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.038+0000 7fbd08f26640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0004d40 0x7fbcf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0406de20 con 0x7fbcf0003f60 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf000bb50 con 0x7fbcf0004d40 2024-03-06T07:21:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf000bd50 con 0x7fbd040c7380 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf0003f60 0x7fbcf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd0a729640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0004d40 0x7fbcf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09f28640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbcf00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 -- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 msgr2=0x7fbcf00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd040c7380 0x7fbcf00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 -- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0004d40 msgr2=0x7fbcf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0004d40 0x7fbcf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.039+0000 7fbd09727640 1 -- 172.21.15.17:0/1149629942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0003520 con 0x7fbcf0003f60 2024-03-06T07:21:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd09727640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf0003f60 0x7fbcf0004800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c210 tx=0x7fbcf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addb022ed36f6e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4007d30 con 0x7fbcf0003f60 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4030070 con 0x7fbcf0003f60 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd08f26640 1 -- 172.21.15.17:0/1149629942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf402cd60 con 0x7fbcf0003f60 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd0c18c640 1 -- 172.21.15.17:0/1149629942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf0003f60 msgr2=0x7fbcf0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/1149629942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf0003f60 0x7fbcf0004800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400c210 tx=0x7fbcf0005830 comp rx=0 tx=0).stop 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd0a729640 1 -- 172.21.15.17:0/1149629942 reap_dead start 2024-03-06T07:21:43.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd0c18c640 1 -- 172.21.15.17:0/1149629942 shutdown_connections 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.040+0000 7fbd0c18c640 1 -- 172.21.15.17:0/1149629942 >> 172.21.15.17:0/1149629942 conn(0x7fbd040aa630 msgr2=0x7fbd040aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.041+0000 7fbd0c18c640 1 -- 172.21.15.17:0/1149629942 shutdown_connections 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.041+0000 7fbd0c18c640 1 -- 172.21.15.17:0/1149629942 wait complete. 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 Processor -- start 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 -- start start 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0003f60 0x7fbd041eb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd04157c00 0x7fbd040abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c3280 con 0x7fbd040ae640 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd041594a0 con 0x7fbcf0003f60 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd0c18c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd040c3400 con 0x7fbd04157c00 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd09727640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd09727640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37410/0 (socket says 172.21.15.17:37410) 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.042+0000 7fbd09727640 1 -- 172.21.15.17:0/994610056 learned_addr learned my addr 172.21.15.17:0/994610056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09f28640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0003f60 0x7fbd041eb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd0a729640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd04157c00 0x7fbd040abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 -- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd04157c00 msgr2=0x7fbd040abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd04157c00 0x7fbd040abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 -- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0003f60 msgr2=0x7fbd041eb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0003f60 0x7fbd041eb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 -- 172.21.15.17:0/994610056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04150290 con 0x7fbd040ae640 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd09727640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbcf402c570 tx=0x7fbcf4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4045070 con 0x7fbd040ae640 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4000ed0 con 0x7fbd040ae640 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd04150570 con 0x7fbd040ae640 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004e30 con 0x7fbd040ae640 2024-03-06T07:21:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.043+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0414f7c0 con 0x7fbd040ae640 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.044+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd040c4500 con 0x7fbd040ae640 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.045+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcf4007070 con 0x7fbd040ae640 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.045+0000 7fbcfa7fc640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.045+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fbcf40ab530 con 0x7fbd040ae640 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.045+0000 7fbd09f28640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.046+0000 7fbd09f28640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd000035f0 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:43.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.050+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd040c4500 con 0x7fbd040ae640 2024-03-06T07:21:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.268+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbd0404f330 con 0x7fbd040ae640 2024-03-06T07:21:43.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.902+0000 7fbcfa7fc640 1 -- 172.21.15.17:0/994610056 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbcf407a2e0 con 0x7fbd040ae640 2024-03-06T07:21:43.905 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 7 and data pool 8 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 msgr2=0x7fbce806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd000035f0 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 msgr2=0x7fbd04151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbcf402c570 tx=0x7fbcf4001a40 comp rx=0 tx=0).stop 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 shutdown_connections 2024-03-06T07:21:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbd000035f0 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd04157c00 0x7fbd040abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd040ae640 0x7fbd04151cc0 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbcf402c570 tx=0x7fbcf4001a40 comp rx=0 tx=0).stop 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 --2- 172.21.15.17:0/994610056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcf0003f60 0x7fbd041eb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.915+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 >> 172.21.15.17:0/994610056 conn(0x7fbd040aa630 msgr2=0x7fbd0406c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.917+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 shutdown_connections 2024-03-06T07:21:43.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:43.917+0000 7fbd0c18c640 1 -- 172.21.15.17:0/994610056 wait complete. 2024-03-06T07:21:43.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:44.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.044+0000 7f1e2225c640 1 Processor -- start 2024-03-06T07:21:44.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.044+0000 7f1e2225c640 1 -- start start 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e2225c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e1c152920 0x7f1e1c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e2225c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e1c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e2225c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1c156180 con 0x7f1e1c159460 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e2225c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1c1587b0 con 0x7f1e1c152920 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e2225c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0c7760 con 0x7f1e1c0c8a00 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1e1c159460 0x7f1e1c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39596/0 (socket says 172.21.15.17:39596) 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1bfff640 1 -- 172.21.15.17:0/491896134 learned_addr learned my addr 172.21.15.17:0/491896134 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:44.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1affd640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e1c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1b7fe640 1 --2- 172.21.15.17:0/491896134 >> v2:172.21.15.163:3300/0 conn(0x7f1e1c152920 0x7f1e1c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135869264 0 0) 0x7f1e14001490 con 0x7f1e1c159460 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e04003520 con 0x7f1e1c159460 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196601556 0 0) 0x7f1e04003520 con 0x7f1e1c159460 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 msgr2=0x7f1e1c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e1c158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 >> v2:172.21.15.163:3300/0 conn(0x7f1e1c152920 msgr2=0x7f1e1c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 --2- 172.21.15.17:0/491896134 >> v2:172.21.15.163:3300/0 conn(0x7f1e1c152920 0x7f1e1c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0c7a40 con 0x7f1e1c159460 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1e14001bc0 con 0x7f1e1c159460 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 >> v1:172.21.15.17:6789/0 conn(0x7f1e1c159460 legacy=0x7f1e1c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e040039b0 0x7f1e04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1587b0 con 0x7f1e04004d40 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0c7760 con 0x7f1e040039b0 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.045+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0400b9c0 con 0x7f1e1c0c8a00 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1affd640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1b7fe640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e040039b0 0x7f1e04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 -- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 msgr2=0x7f1e04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c0c8a00 0x7f1e04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 -- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e040039b0 msgr2=0x7f1e04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e040039b0 0x7f1e04003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.046+0000 7f1e1bfff640 1 -- 172.21.15.17:0/491896134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04003520 con 0x7f1e04004d40 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e040051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0cba80 tx=0x7f1e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe2276e918da5db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400a690 con 0x7f1e04004d40 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e140072f0 con 0x7f1e04004d40 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e1a7fc640 1 -- 172.21.15.17:0/491896134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e140076e0 con 0x7f1e04004d40 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 -- 172.21.15.17:0/491896134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 msgr2=0x7f1e040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 --2- 172.21.15.17:0/491896134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e040051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0cba80 tx=0x7f1e04005830 comp rx=0 tx=0).stop 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e1bfff640 1 -- 172.21.15.17:0/491896134 reap_dead start 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 -- 172.21.15.17:0/491896134 shutdown_connections 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 -- 172.21.15.17:0/491896134 >> 172.21.15.17:0/491896134 conn(0x7f1e1c0ac560 msgr2=0x7f1e1c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 -- 172.21.15.17:0/491896134 shutdown_connections 2024-03-06T07:21:44.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.047+0000 7f1e2225c640 1 -- 172.21.15.17:0/491896134 wait complete. 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.048+0000 7f1e2225c640 1 Processor -- start 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 -- start start 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 0x7f1e1c1deb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 0x7f1e1c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c06d030 con 0x7f1e04004d40 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c158a00 con 0x7f1e1c1eead0 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e2225c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c0c5a40 con 0x7f1e1c1df040 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e1b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37426/0 (socket says 172.21.15.17:37426) 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.049+0000 7f1e1b7fe640 1 -- 172.21.15.17:0/2709067838 learned_addr learned my addr 172.21.15.17:0/2709067838 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 0x7f1e1c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1affd640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 0x7f1e1c1deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 -- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 msgr2=0x7f1e1c145800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 0x7f1e1c145800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 -- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 msgr2=0x7f1e1c1deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 0x7f1e1c1deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 -- 172.21.15.17:0/2709067838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c1e13f0 con 0x7f1e04004d40 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1bfff640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 0x7f1e1c145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1affd640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 0x7f1e1c1deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e1b7fe640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c011960 tx=0x7f1e0c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00faf0 con 0x7f1e04004d40 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00fc90 con 0x7f1e04004d40 2024-03-06T07:21:44.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c0c9ff0 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c014440 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.050+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c1e0880 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.051+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1c04f330 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.052+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e0c012040 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.052+0000 7f1e03fff640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.052+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e0c08aa30 con 0x7f1e04004d40 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.052+0000 7f1e1affd640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.053+0000 7f1e1affd640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 0x7f1e0806d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c151a20 tx=0x7f1e10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.057+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e0c019030 con 0x7f1e04004d40 2024-03-06T07:21:44.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.240+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e1c06cda0 con 0x7f1e04004d40 2024-03-06T07:21:44.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.241+0000 7f1e03fff640 1 -- 172.21.15.17:0/2709067838 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f1e0c059160 con 0x7f1e04004d40 2024-03-06T07:21:44.243 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:44.244 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":50,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:43.270528+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 msgr2=0x7f1e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 0x7f1e0806d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c151a20 tx=0x7f1e10008040 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 msgr2=0x7f1e1c1ee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c011960 tx=0x7f1e0c011e20 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 shutdown_connections 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0806b4d0 0x7f1e0806d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e1c1df040 0x7f1e1c145800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e1c1eead0 0x7f1e1c1deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 --2- 172.21.15.17:0/2709067838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e04004d40 0x7f1e1c1ee590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 >> 172.21.15.17:0/2709067838 conn(0x7f1e1c0ac560 msgr2=0x7f1e1c0bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 shutdown_connections 2024-03-06T07:21:44.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.243+0000 7f1e2225c640 1 -- 172.21.15.17:0/2709067838 wait complete. 2024-03-06T07:21:44.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:44.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.354+0000 7f0c979c7640 1 Processor -- start 2024-03-06T07:21:44.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.355+0000 7f0c979c7640 1 -- start start 2024-03-06T07:21:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.355+0000 7f0c979c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c900c7380 0x7f0c900c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.355+0000 7f0c979c7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c90072630 con 0x7f0c900c6780 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.355+0000 7f0c979c7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c90072910 con 0x7f0c9006da40 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.355+0000 7f0c979c7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9006de20 con 0x7f0c900c7380 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c95f64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c900c7380 0x7f0c900c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c94f62640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0c900c6780 0x7f0c90158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39602/0 (socket says 172.21.15.17:39602) 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c94f62640 1 -- 172.21.15.17:0/2662102704 learned_addr learned my addr 172.21.15.17:0/2662102704 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c95f64640 1 -- 172.21.15.17:0/2662102704 >> v1:172.21.15.163:6789/0 conn(0x7f0c9006da40 legacy=0x7f0c90155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c95f64640 1 -- 172.21.15.17:0/2662102704 >> v1:172.21.15.17:6789/0 conn(0x7f0c900c6780 legacy=0x7f0c90158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c87fff640 1 -- 172.21.15.17:0/2662102704 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217646831 0 0) 0x7f0c90072630 con 0x7f0c900c6780 2024-03-06T07:21:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.356+0000 7f0c95f64640 1 -- 172.21.15.17:0/2662102704 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90072bf0 con 0x7f0c900c7380 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c95f64640 1 --2- 172.21.15.17:0/2662102704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900c3280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009d90 tx=0x7f0c8802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a669ab962e065a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c87fff640 1 -- 172.21.15.17:0/2662102704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88039070 con 0x7f0c900c7380 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c87fff640 1 -- 172.21.15.17:0/2662102704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8802cc60 con 0x7f0c900c7380 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c87fff640 1 -- 172.21.15.17:0/2662102704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88034660 con 0x7f0c900c7380 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 -- 172.21.15.17:0/2662102704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 msgr2=0x7f0c900c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 --2- 172.21.15.17:0/2662102704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900c3280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009d90 tx=0x7f0c8802beb0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 -- 172.21.15.17:0/2662102704 shutdown_connections 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 --2- 172.21.15.17:0/2662102704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900c3280 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 -- 172.21.15.17:0/2662102704 >> 172.21.15.17:0/2662102704 conn(0x7f0c900aa630 msgr2=0x7f0c900aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 -- 172.21.15.17:0/2662102704 shutdown_connections 2024-03-06T07:21:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.357+0000 7f0c979c7640 1 -- 172.21.15.17:0/2662102704 wait complete. 2024-03-06T07:21:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.358+0000 7f0c979c7640 1 Processor -- start 2024-03-06T07:21:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.358+0000 7f0c979c7640 1 -- start start 2024-03-06T07:21:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 0x7f0c901eb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 0x7f0c901573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900c37c0 con 0x7f0c900c7380 2024-03-06T07:21:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c901594a0 con 0x7f0c900c6780 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c979c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c900c3940 con 0x7f0c9006da40 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c95f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c95f64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37444/0 (socket says 172.21.15.17:37444) 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c95f64640 1 -- 172.21.15.17:0/1844268238 learned_addr learned my addr 172.21.15.17:0/1844268238 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.359+0000 7f0c94f62640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 0x7f0c901573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 -- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 msgr2=0x7f0c901eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95763640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 0x7f0c901eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 0x7f0c901eb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 -- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 msgr2=0x7f0c901573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 0x7f0c901573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 -- 172.21.15.17:0/1844268238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c901eabd0 con 0x7f0c900c7380 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c94f62640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 0x7f0c901573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95763640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 0x7f0c901eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c95f64640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0c88037040 tx=0x7f0c88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88044020 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8803a040 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c901eebf0 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88039050 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.360+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c901eae90 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.361+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9004f330 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.361+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c88003bb0 con 0x7f0c900c7380 2024-03-06T07:21:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.362+0000 7f0c85ffb640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.362+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c880ac820 con 0x7f0c900c7380 2024-03-06T07:21:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.362+0000 7f0c95763640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.363+0000 7f0c95763640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 0x7f0c7406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0c90158520 tx=0x7f0c8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.366+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c8807b5a0 con 0x7f0c900c7380 2024-03-06T07:21:44.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.547+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c90073240 con 0x7f0c900c7380 2024-03-06T07:21:44.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.548+0000 7f0c85ffb640 1 -- 172.21.15.17:0/1844268238 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f0c8807aea0 con 0x7f0c900c7380 2024-03-06T07:21:44.551 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:44.551 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":50,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:43.270528+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 msgr2=0x7f0c7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 0x7f0c7406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0c90158520 tx=0x7f0c8c009040 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 msgr2=0x7f0c900a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0c88037040 tx=0x7f0c88004a40 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 shutdown_connections 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c7406b4d0 0x7f0c7406d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c900c7380 0x7f0c900a9760 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c900c6780 0x7f0c901573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 --2- 172.21.15.17:0/1844268238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c9006da40 0x7f0c901eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 >> 172.21.15.17:0/1844268238 conn(0x7f0c900aa630 msgr2=0x7f0c900aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 shutdown_connections 2024-03-06T07:21:44.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.550+0000 7f0c979c7640 1 -- 172.21.15.17:0/1844268238 wait complete. 2024-03-06T07:21:44.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:21:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.665+0000 7f431580e640 1 Processor -- start 2024-03-06T07:21:44.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.665+0000 7f431580e640 1 -- start start 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f431580e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43100c5690 0x7f43100a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f431580e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431014f5e0 con 0x7f431006db70 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f431580e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43100c6100 con 0x7f431014f200 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f431580e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43101551b0 con 0x7f43100c5690 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f430e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43100c5690 0x7f43100a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f430f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f431006db70 0x7f431006dfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39604/0 (socket says 172.21.15.17:39604) 2024-03-06T07:21:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.666+0000 7f430f7fe640 1 -- 172.21.15.17:0/262176169 learned_addr learned my addr 172.21.15.17:0/262176169 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430e7fc640 1 -- 172.21.15.17:0/262176169 >> v1:172.21.15.163:6789/0 conn(0x7f431014f200 legacy=0x7f43100c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430e7fc640 1 -- 172.21.15.17:0/262176169 >> v1:172.21.15.17:6789/0 conn(0x7f431006db70 legacy=0x7f431006dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430dffb640 1 -- 172.21.15.17:0/262176169 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778200212 0 0) 0x7f431014f5e0 con 0x7f431006db70 2024-03-06T07:21:44.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430e7fc640 1 -- 172.21.15.17:0/262176169 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310155470 con 0x7f43100c5690 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430e7fc640 1 --2- 172.21.15.17:0/262176169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43100c5690 0x7f43100a6cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f43100aa3d0 tx=0x7f430402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8148629c6964a53f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430dffb640 1 -- 172.21.15.17:0/262176169 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304035020 con 0x7f43100c5690 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430dffb640 1 -- 172.21.15.17:0/262176169 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4304031800 con 0x7f43100c5690 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.667+0000 7f430dffb640 1 -- 172.21.15.17:0/262176169 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304031bd0 con 0x7f43100c5690 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 -- 172.21.15.17:0/262176169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43100c5690 msgr2=0x7f43100a6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 --2- 172.21.15.17:0/262176169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43100c5690 0x7f43100a6cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f43100aa3d0 tx=0x7f430402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 -- 172.21.15.17:0/262176169 shutdown_connections 2024-03-06T07:21:44.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 --2- 172.21.15.17:0/262176169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43100c5690 0x7f43100a6cc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 -- 172.21.15.17:0/262176169 >> 172.21.15.17:0/262176169 conn(0x7f43100a9540 msgr2=0x7f43100a9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 -- 172.21.15.17:0/262176169 shutdown_connections 2024-03-06T07:21:44.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.668+0000 7f431580e640 1 -- 172.21.15.17:0/262176169 wait complete. 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.669+0000 7f431580e640 1 Processor -- start 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.669+0000 7f431580e640 1 -- start start 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 0x7f4310073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 0x7f431006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310155d10 con 0x7f431014f200 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310155a30 con 0x7f43100c5690 2024-03-06T07:21:44.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f431580e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100c12d0 con 0x7f431006db70 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37468/0 (socket says 172.21.15.17:37468) 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 -- 172.21.15.17:0/1870625495 learned_addr learned my addr 172.21.15.17:0/1870625495 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430e7fc640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 0x7f431006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 -- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 msgr2=0x7f4310073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430effd640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 0x7f4310073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 0x7f4310073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 -- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 msgr2=0x7f431006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 0x7f431006c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:44.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.670+0000 7f430f7fe640 1 -- 172.21.15.17:0/1870625495 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310073590 con 0x7f431014f200 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f430e7fc640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 0x7f431006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f430effd640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 0x7f4310073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f430f7fe640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f430000c7a0 tx=0x7f430000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300018070 con 0x7f431014f200 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430000a640 con 0x7f431014f200 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431014cd40 con 0x7f431014f200 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000aa70 con 0x7f431014f200 2024-03-06T07:21:44.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.671+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4310072150 con 0x7f431014f200 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.672+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431006d8e0 con 0x7f431014f200 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.672+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f430001b070 con 0x7f431014f200 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.673+0000 7f42f7fff640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.673+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4300089a10 con 0x7f431014f200 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.673+0000 7f430effd640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:44.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.673+0000 7f430effd640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f430800a830 tx=0x7f4308008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:44.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.677+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f431006d8e0 con 0x7f431014f200 2024-03-06T07:21:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:44.901+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f431004f330 con 0x7f431014f200 2024-03-06T07:21:45.886 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:21:45.884+0000 7f9285948640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:21:45.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.903+0000 7f42f7fff640 1 -- 172.21.15.17:0/1870625495 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4300058700 con 0x7f431014f200 2024-03-06T07:21:45.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.905+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 msgr2=0x7f42fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 0x7f42fc06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f430800a830 tx=0x7f4308008040 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 msgr2=0x7f43101eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f430000c7a0 tx=0x7f430000cc60 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 shutdown_connections 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42fc06b4d0 0x7f42fc06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f431014f200 0x7f43101eb040 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43100c5690 0x7f431006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 --2- 172.21.15.17:0/1870625495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f431006db70 0x7f4310073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 >> 172.21.15.17:0/1870625495 conn(0x7f43100a9540 msgr2=0x7f4310149610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 shutdown_connections 2024-03-06T07:21:45.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:45.906+0000 7f431580e640 1 -- 172.21.15.17:0/1870625495 wait complete. 2024-03-06T07:21:45.921 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:21:45.921 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:21:46.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.025+0000 7f994c7d0640 1 Processor -- start 2024-03-06T07:21:46.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.025+0000 7f994c7d0640 1 -- start start 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994c7d0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 0x7f994406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994c7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994c7d0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440b0450 con 0x7f99440b0070 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994c7d0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f994406cfb0 con 0x7f9944155c30 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994c7d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994406c020 con 0x7f99440c5380 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f9949d6b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 0x7f994406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:56588/0 (socket says 172.21.15.17:56588) 2024-03-06T07:21:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 -- 172.21.15.17:0/2888363273 learned_addr learned my addr 172.21.15.17:0/2888363273 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994a56c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f99440b0070 0x7f99440c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:50464/0 (socket says 172.21.15.17:50464) 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 -- 172.21.15.17:0/2888363273 >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 msgr2=0x7f994406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 --2- 172.21.15.17:0/2888363273 >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 0x7f994406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 -- 172.21.15.17:0/2888363273 >> v1:172.21.15.163:6789/0 conn(0x7f99440b0070 legacy=0x7f99440c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f994ad6d640 1 -- 172.21.15.17:0/2888363273 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99441560e0 con 0x7f99440c5380 2024-03-06T07:21:46.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.026+0000 7f9949d6b640 1 --2- 172.21.15.17:0/2888363273 >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 0x7f994406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:46.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.027+0000 7f994ad6d640 1 --2- 172.21.15.17:0/2888363273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f993c009e50 tx=0x7f993c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88cd9d48c2eff59 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.027+0000 7f994956a640 1 -- 172.21.15.17:0/2888363273 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c02ca70 con 0x7f99440c5380 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994956a640 1 -- 172.21.15.17:0/2888363273 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c02cc10 con 0x7f99440c5380 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994956a640 1 -- 172.21.15.17:0/2888363273 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c0343f0 con 0x7f99440c5380 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 -- 172.21.15.17:0/2888363273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 msgr2=0x7f99440c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 --2- 172.21.15.17:0/2888363273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f993c009e50 tx=0x7f993c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 -- 172.21.15.17:0/2888363273 shutdown_connections 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 --2- 172.21.15.17:0/2888363273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99440c5380 0x7f99440c5790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 --2- 172.21.15.17:0/2888363273 >> v2:172.21.15.163:3300/0 conn(0x7f9944155c30 0x7f994406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 -- 172.21.15.17:0/2888363273 >> 172.21.15.17:0/2888363273 conn(0x7f99440ac590 msgr2=0x7f99440ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 -- 172.21.15.17:0/2888363273 shutdown_connections 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.028+0000 7f994c7d0640 1 -- 172.21.15.17:0/2888363273 wait complete. 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.029+0000 7f994c7d0640 1 Processor -- start 2024-03-06T07:21:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.029+0000 7f994c7d0640 1 -- start start 2024-03-06T07:21:46.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99440c5380 0x7f994414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9944155c30 0x7f9944150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994406c270 con 0x7f99440b0070 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440c7ad0 con 0x7f99440c5380 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994c7d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994406c550 con 0x7f9944155c30 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37482/0 (socket says 172.21.15.17:37482) 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 -- 172.21.15.17:0/1867474295 learned_addr learned my addr 172.21.15.17:0/1867474295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f9949d6b640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99440c5380 0x7f994414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994ad6d640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9944155c30 0x7f9944150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 -- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9944155c30 msgr2=0x7f9944150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9944155c30 0x7f9944150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 -- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99440c5380 msgr2=0x7f994414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99440c5380 0x7f994414f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.030+0000 7f994a56c640 1 -- 172.21.15.17:0/1867474295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944159090 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f994a56c640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f994000c3f0 tx=0x7f994000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99400074e0 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9940007680 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994414fd50 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9940004a00 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.031+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99441534e0 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.032+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994404f330 con 0x7f99440b0070 2024-03-06T07:21:46.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.032+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f994001a020 con 0x7f99440b0070 2024-03-06T07:21:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.033+0000 7f993affd640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 0x7f992806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.033+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9940089ac0 con 0x7f99440b0070 2024-03-06T07:21:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.033+0000 7f9949d6b640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 0x7f992806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:46.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.033+0000 7f9949d6b640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 0x7f992806d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9934009560 tx=0x7f9934008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:46.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.041+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99440b0450 con 0x7f99440b0070 2024-03-06T07:21:46.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.261+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9944154550 con 0x7f99440b0070 2024-03-06T07:21:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.915+0000 7f993affd640 1 -- 172.21.15.17:0/1867474295 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99400587c0 con 0x7f99440b0070 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 msgr2=0x7f992806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 0x7f992806d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9934009560 tx=0x7f9934008040 comp rx=0 tx=0).stop 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 msgr2=0x7f99441eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f994000c3f0 tx=0x7f994000c8b0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 shutdown_connections 2024-03-06T07:21:46.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f992806b500 0x7f992806d9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9944155c30 0x7f9944150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99440c5380 0x7f994414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 --2- 172.21.15.17:0/1867474295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99440b0070 0x7f99441eada0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 >> 172.21.15.17:0/1867474295 conn(0x7f99440ac590 msgr2=0x7f99440ad680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 shutdown_connections 2024-03-06T07:21:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:46.917+0000 7f994c7d0640 1 -- 172.21.15.17:0/1867474295 wait complete. 2024-03-06T07:21:46.931 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:21:46.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:47.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.059+0000 7f16a8f19640 1 Processor -- start 2024-03-06T07:21:47.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.059+0000 7f16a8f19640 1 -- start start 2024-03-06T07:21:47.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a8f19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f16a406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a8f19640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a40c1ae0 con 0x7f16a40720a0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a8f19640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a406e450 con 0x7f16a4152520 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a8f19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c1dc0 con 0x7f16a4155c80 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a2d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f16a40720a0 0x7f16a406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39618/0 (socket says 172.21.15.17:39618) 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a2d9d640 1 -- 172.21.15.17:0/924510898 learned_addr learned my addr 172.21.15.17:0/924510898 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f16a406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.060+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705116945 0 0) 0x7f16a40c1ae0 con 0x7f16a40720a0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c003520 con 0x7f16a40720a0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157574208 0 0) 0x7f168c003520 con 0x7f16a40720a0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 msgr2=0x7f16a406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f16a406ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 >> v1:172.21.15.163:6789/0 conn(0x7f16a4152520 legacy=0x7f16a40c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a406e730 con 0x7f16a40720a0 2024-03-06T07:21:47.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f169c001670 con 0x7f16a40720a0 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 >> v1:172.21.15.17:6789/0 conn(0x7f16a40720a0 legacy=0x7f16a406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f168c0039e0 0x7f168c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c004390 0x7f168c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f168c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c1dc0 con 0x7f168c0039e0 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00b9c0 con 0x7f168c004390 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00bbc0 con 0x7f16a4155c80 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f168c0039e0 0x7f168c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a2d9d640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f168c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c004390 0x7f168c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 -- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 msgr2=0x7f168c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f168c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 -- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c004390 msgr2=0x7f168c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c004390 0x7f168c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a259c640 1 -- 172.21.15.17:0/924510898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c003520 con 0x7f168c0039e0 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a2d9d640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4155c80 0x7f168c0050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.061+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c004390 0x7f168c004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.062+0000 7f16a259c640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f168c0039e0 0x7f168c003dc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1698001f60 tx=0x7f168c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb13da5f1cb287d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.062+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698039070 con 0x7f168c0039e0 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.062+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169802c730 con 0x7f168c0039e0 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.062+0000 7f16a159a640 1 -- 172.21.15.17:0/924510898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698001b30 con 0x7f168c0039e0 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 -- 172.21.15.17:0/924510898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f168c0039e0 msgr2=0x7f168c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 --2- 172.21.15.17:0/924510898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f168c0039e0 0x7f168c003dc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1698001f60 tx=0x7f168c005830 comp rx=0 tx=0).stop 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a2d9d640 1 -- 172.21.15.17:0/924510898 reap_dead start 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 -- 172.21.15.17:0/924510898 shutdown_connections 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 -- 172.21.15.17:0/924510898 >> 172.21.15.17:0/924510898 conn(0x7f16a40c76b0 msgr2=0x7f16a40c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 -- 172.21.15.17:0/924510898 shutdown_connections 2024-03-06T07:21:47.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.063+0000 7f16a8f19640 1 -- 172.21.15.17:0/924510898 wait complete. 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.064+0000 7f16a8f19640 1 Processor -- start 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.064+0000 7f16a8f19640 1 -- start start 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c0039e0 0x7f16a41eb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4157120 0x7f16a41575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a406b850 con 0x7f16a414d2a0 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a406b6d0 con 0x7f168c0039e0 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a8f19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4152900 con 0x7f16a4157120 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37498/0 (socket says 172.21.15.17:37498) 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 -- 172.21.15.17:0/3006597965 learned_addr learned my addr 172.21.15.17:0/3006597965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a259c640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c0039e0 0x7f16a41eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a2d9d640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4157120 0x7f16a41575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 -- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4157120 msgr2=0x7f16a41575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4157120 0x7f16a41575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 -- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c0039e0 msgr2=0x7f16a41eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c0039e0 0x7f16a41eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 -- 172.21.15.17:0/3006597965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4156360 con 0x7f16a414d2a0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.065+0000 7f16a1d9b640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f16940116d0 tx=0x7f1694011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.066+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694015870 con 0x7f16a414d2a0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.066+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694015a10 con 0x7f16a414d2a0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.066+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4156640 con 0x7f16a414d2a0 2024-03-06T07:21:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.066+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694015e00 con 0x7f16a414d2a0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.066+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a4158e20 con 0x7f16a414d2a0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.067+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a404f330 con 0x7f16a414d2a0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.068+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169401d400 con 0x7f16a414d2a0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.068+0000 7f1692ffd640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.068+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f169408aad0 con 0x7f16a414d2a0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.068+0000 7f16a259c640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.069+0000 7f16a259c640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 0x7f168406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1698001f60 tx=0x7f1698009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.073+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1694019030 con 0x7f16a414d2a0 2024-03-06T07:21:47.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.291+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16a40c6c10 con 0x7f16a414d2a0 2024-03-06T07:21:47.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.291+0000 7f1692ffd640 1 -- 172.21.15.17:0/3006597965 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f16940597d0 con 0x7f16a414d2a0 2024-03-06T07:21:47.294 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:47.295 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5769,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:21:46.263408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5772":{"gid":5772,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1002585815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1002585815},{"type":"v1","addr":"172.21.15.17:6835","nonce":1002585815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5772,"qdb_cluster":[5772]},"id":3}]} 2024-03-06T07:21:47.295 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 31 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 msgr2=0x7f168406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 0x7f168406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1698001f60 tx=0x7f1698009030 comp rx=0 tx=0).stop 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 msgr2=0x7f16a41588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f16940116d0 tx=0x7f1694011ad0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 shutdown_connections 2024-03-06T07:21:47.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a4157120 0x7f16a41575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414d2a0 0x7f16a41588e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 --2- 172.21.15.17:0/3006597965 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f168c0039e0 0x7f16a41eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 >> 172.21.15.17:0/3006597965 conn(0x7f16a40c76b0 msgr2=0x7f16a4155050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 shutdown_connections 2024-03-06T07:21:47.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.293+0000 7f16a8f19640 1 -- 172.21.15.17:0/3006597965 wait complete. 2024-03-06T07:21:47.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:47.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.407+0000 7f81c3b35640 1 Processor -- start 2024-03-06T07:21:47.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.407+0000 7f81c3b35640 1 -- start start 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c3b35640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f81bc14f360 0x7f81bc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c3b35640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81bc0c88c0 0x7f81bc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c3b35640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc0c73d0 con 0x7f81bc0c5880 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c3b35640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0c8dd0 con 0x7f81bc0c88c0 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c3b35640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0be2e0 con 0x7f81bc14f360 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c10d0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81bc0c88c0 0x7f81bc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c20d2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f81bc0c5880 0x7f81bc158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39620/0 (socket says 172.21.15.17:39620) 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c20d2640 1 -- 172.21.15.17:0/2330740993 learned_addr learned my addr 172.21.15.17:0/2330740993 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c18d1640 1 --2- 172.21.15.17:0/2330740993 >> v2:172.21.15.163:3300/0 conn(0x7f81bc14f360 0x7f81bc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c10d0640 1 -- 172.21.15.17:0/2330740993 >> v2:172.21.15.163:3300/0 conn(0x7f81bc14f360 msgr2=0x7f81bc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c10d0640 1 --2- 172.21.15.17:0/2330740993 >> v2:172.21.15.163:3300/0 conn(0x7f81bc14f360 0x7f81bc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c10d0640 1 -- 172.21.15.17:0/2330740993 >> v1:172.21.15.17:6789/0 conn(0x7f81bc0c5880 legacy=0x7f81bc158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c08cf640 1 -- 172.21.15.17:0/2330740993 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783577847 0 0) 0x7f81bc0c73d0 con 0x7f81bc0c5880 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.408+0000 7f81c10d0640 1 -- 172.21.15.17:0/2330740993 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0c7670 con 0x7f81bc0c88c0 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.409+0000 7f81c10d0640 1 --2- 172.21.15.17:0/2330740993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c88c0 0x7f81bc0c5340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f81bc08c840 tx=0x7f81ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37236e3891a581b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.409+0000 7f81c08cf640 1 -- 172.21.15.17:0/2330740993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0315a0 con 0x7f81bc0c88c0 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.409+0000 7f81c08cf640 1 -- 172.21.15.17:0/2330740993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac037070 con 0x7f81bc0c88c0 2024-03-06T07:21:47.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.409+0000 7f81c08cf640 1 -- 172.21.15.17:0/2330740993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac031960 con 0x7f81bc0c88c0 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 -- 172.21.15.17:0/2330740993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c88c0 msgr2=0x7f81bc0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 --2- 172.21.15.17:0/2330740993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c88c0 0x7f81bc0c5340 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f81bc08c840 tx=0x7f81ac02b840 comp rx=0 tx=0).stop 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 -- 172.21.15.17:0/2330740993 shutdown_connections 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 --2- 172.21.15.17:0/2330740993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c88c0 0x7f81bc0c5340 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 --2- 172.21.15.17:0/2330740993 >> v2:172.21.15.163:3300/0 conn(0x7f81bc14f360 0x7f81bc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 -- 172.21.15.17:0/2330740993 >> 172.21.15.17:0/2330740993 conn(0x7f81bc0ac570 msgr2=0x7f81bc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 -- 172.21.15.17:0/2330740993 shutdown_connections 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.410+0000 7f81c3b35640 1 -- 172.21.15.17:0/2330740993 wait complete. 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.411+0000 7f81c3b35640 1 Processor -- start 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.411+0000 7f81c3b35640 1 -- start start 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81bc0c88c0 0x7f81bc0af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 0x7f81bc0af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc159270 con 0x7f81bc0c5880 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c7c30 con 0x7f81bc14f360 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c3b35640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc1593f0 con 0x7f81bc0c88c0 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37524/0 (socket says 172.21.15.17:37524) 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 -- 172.21.15.17:0/3082293490 learned_addr learned my addr 172.21.15.17:0/3082293490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c10d0640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81bc0c88c0 0x7f81bc0af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c20d2640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 0x7f81bc0af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 -- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81bc0c88c0 msgr2=0x7f81bc0af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81bc0c88c0 0x7f81bc0af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 -- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 msgr2=0x7f81bc0af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 0x7f81bc0af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 -- 172.21.15.17:0/3082293490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0adcf0 con 0x7f81bc0c5880 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c20d2640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 0x7f81bc0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.412+0000 7f81c18d1640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f81b800d620 tx=0x7f81b800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.413+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b800b620 con 0x7f81bc0c5880 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.413+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b8019070 con 0x7f81bc0c5880 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.413+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc0ada10 con 0x7f81bc0c5880 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.413+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b800ba00 con 0x7f81bc0c5880 2024-03-06T07:21:47.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.413+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1eb580 con 0x7f81bc0c5880 2024-03-06T07:21:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.414+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc069460 con 0x7f81bc0c5880 2024-03-06T07:21:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.414+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81b800bba0 con 0x7f81bc0c5880 2024-03-06T07:21:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.414+0000 7f81b27fc640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 0x7f81a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.415+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81b808a650 con 0x7f81bc0c5880 2024-03-06T07:21:47.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.415+0000 7f81c10d0640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 0x7f81a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.415+0000 7f81c10d0640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 0x7f81a006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81ac036000 tx=0x7f81ac03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.419+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81bc069460 con 0x7f81bc0c5880 2024-03-06T07:21:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.598+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81bc06cda0 con 0x7f81bc0c5880 2024-03-06T07:21:47.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.599+0000 7f81b27fc640 1 -- 172.21.15.17:0/3082293490 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f81b80593d0 con 0x7f81bc0c5880 2024-03-06T07:21:47.602 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:47.602 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":51,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:44.903429+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:47.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 msgr2=0x7f81a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 0x7f81a006d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f81ac036000 tx=0x7f81ac03d040 comp rx=0 tx=0).stop 2024-03-06T07:21:47.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 msgr2=0x7f81bc0c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f81b800d620 tx=0x7f81b800dae0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 shutdown_connections 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f81a006b500 0x7f81a006d9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81bc14f360 0x7f81bc0af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81bc0c88c0 0x7f81bc0af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 --2- 172.21.15.17:0/3082293490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81bc0c5880 0x7f81bc0c0e30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 >> 172.21.15.17:0/3082293490 conn(0x7f81bc0ac570 msgr2=0x7f81bc0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 shutdown_connections 2024-03-06T07:21:47.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.601+0000 7f81c3b35640 1 -- 172.21.15.17:0/3082293490 wait complete. 2024-03-06T07:21:47.615 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:21:47.615 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:47.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.742+0000 7f221538c640 1 Processor -- start 2024-03-06T07:21:47.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.742+0000 7f221538c640 1 -- start start 2024-03-06T07:21:47.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f221538c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 0x7f22100c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f221538c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f221538c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2210155110 con 0x7f22100c88b0 2024-03-06T07:21:47.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f221538c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22101559e0 con 0x7f22100c5330 2024-03-06T07:21:47.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f221538c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100c8f40 con 0x7f2210158db0 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f220effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 0x7f22100c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f220f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:56640/0 (socket says 172.21.15.17:56640) 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f220f7fe640 1 -- 172.21.15.17:0/1838006350 learned_addr learned my addr 172.21.15.17:0/1838006350 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f2207fff640 1 --1- 172.21.15.17:0/1838006350 >> v1:172.21.15.163:6789/0 conn(0x7f22100c88b0 0x7f22101586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:50486/0 (socket says 172.21.15.17:50486) 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.743+0000 7f220f7fe640 1 -- 172.21.15.17:0/1838006350 >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 msgr2=0x7f22100c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220f7fe640 1 --2- 172.21.15.17:0/1838006350 >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 0x7f22100c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220f7fe640 1 -- 172.21.15.17:0/1838006350 >> v1:172.21.15.163:6789/0 conn(0x7f22100c88b0 legacy=0x7f22101586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220f7fe640 1 -- 172.21.15.17:0/1838006350 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2210155390 con 0x7f2210158db0 2024-03-06T07:21:47.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220effd640 1 --2- 172.21.15.17:0/1838006350 >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 0x7f22100c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220f7fe640 1 --2- 172.21.15.17:0/1838006350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2208009e50 tx=0x7f220802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1397e546e43bf146 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.744+0000 7f220e7fc640 1 -- 172.21.15.17:0/1838006350 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220802ca70 con 0x7f2210158db0 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f220e7fc640 1 -- 172.21.15.17:0/1838006350 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220802cc10 con 0x7f2210158db0 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f220e7fc640 1 -- 172.21.15.17:0/1838006350 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22080343f0 con 0x7f2210158db0 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 -- 172.21.15.17:0/1838006350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 msgr2=0x7f221015fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 --2- 172.21.15.17:0/1838006350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2208009e50 tx=0x7f220802bed0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 -- 172.21.15.17:0/1838006350 shutdown_connections 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 --2- 172.21.15.17:0/1838006350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2210158db0 0x7f221015fa30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 --2- 172.21.15.17:0/1838006350 >> v2:172.21.15.163:3300/0 conn(0x7f22100c5330 0x7f22100c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 -- 172.21.15.17:0/1838006350 >> 172.21.15.17:0/1838006350 conn(0x7f22100ac560 msgr2=0x7f22100ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 -- 172.21.15.17:0/1838006350 shutdown_connections 2024-03-06T07:21:47.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.745+0000 7f221538c640 1 -- 172.21.15.17:0/1838006350 wait complete. 2024-03-06T07:21:47.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.746+0000 7f221538c640 1 Processor -- start 2024-03-06T07:21:47.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.746+0000 7f221538c640 1 -- start start 2024-03-06T07:21:47.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 0x7f22101514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22100bd9d0 0x7f22101eb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22101529d0 con 0x7f22100c88b0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221006d090 con 0x7f22100c5330 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f221538c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2210152b50 con 0x7f22100bd9d0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37528/0 (socket says 172.21.15.17:37528) 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 -- 172.21.15.17:0/513079315 learned_addr learned my addr 172.21.15.17:0/513079315 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f220effd640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 0x7f22101514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f220f7fe640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22100bd9d0 0x7f22101eb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 -- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22100bd9d0 msgr2=0x7f22101eb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22100bd9d0 0x7f22101eb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 -- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 msgr2=0x7f22101514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 0x7f22101514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f2207fff640 1 -- 172.21.15.17:0/513079315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22101ee050 con 0x7f22100c88b0 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.747+0000 7f220effd640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 0x7f22101514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:47.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f2207fff640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c440 tx=0x7f21f8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f80074e0 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8007680 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8004a20 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22101ee330 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.748+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22101ee8f0 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.749+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2210069460 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.749+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21f801a020 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.750+0000 7f22077fe640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 0x7f21f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.750+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21f8089b10 con 0x7f22100c88b0 2024-03-06T07:21:47.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.750+0000 7f220effd640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 0x7f21f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:47.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.751+0000 7f220effd640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 0x7f21f006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2210151df0 tx=0x7f2200008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:47.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.754+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2210069460 con 0x7f22100c88b0 2024-03-06T07:21:47.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.932+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f221004f330 con 0x7f21f006b4d0 2024-03-06T07:21:47.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.938+0000 7f22077fe640 1 -- 172.21.15.17:0/513079315 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240122 (secure 0 0 0) 0x7f221004f330 con 0x7f21f006b4d0 2024-03-06T07:21:47.944 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:47.946 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 msgr2=0x7f21f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 0x7f21f006d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2210151df0 tx=0x7f2200008040 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 msgr2=0x7f22101eaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c440 tx=0x7f21f8001e70 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 shutdown_connections 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f21f006b4d0 0x7f21f006d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22100bd9d0 0x7f22101eb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22100c88b0 0x7f22101eaae0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 --2- 172.21.15.17:0/513079315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22100c5330 0x7f22101514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.941+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 >> 172.21.15.17:0/513079315 conn(0x7f22100ac560 msgr2=0x7f22101549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:47.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.942+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 shutdown_connections 2024-03-06T07:21:47.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:47.942+0000 7f221538c640 1 -- 172.21.15.17:0/513079315 wait complete. 2024-03-06T07:21:47.955 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-03-06T07:21:47.363706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237368,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737352,"statfs":{"total":773094113280,"available":772851048448,"internally_reserved":0,"allocated":4648960,"data_stored":3177117,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.473970"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.893191+0000","last_change":"2024-03-06T07:21:41.893191+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.893191+0000","last_undegraded":"2024-03-06T07:21:41.893191+0000","last_fullsized":"2024-03-06T07:21:41.893191+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894826+0000","last_change":"2024-03-06T07:21:41.894826+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894826+0000","last_undegraded":"2024-03-06T07:21:41.894826+0000","last_fullsized":"2024-03-06T07:21:41.894826+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896118+0000","last_change":"2024-03-06T07:21:41.896118+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896118+0000","last_undegraded":"2024-03-06T07:21:41.896118+0000","last_fullsized":"2024-03-06T07:21:41.896118+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892875+0000","last_change":"2024-03-06T07:21:41.892875+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892875+0000","last_undegraded":"2024-03-06T07:21:41.892875+0000","last_fullsized":"2024-03-06T07:21:41.892875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891696+0000","last_change":"2024-03-06T07:21:41.891696+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891696+0000","last_undegraded":"2024-03-06T07:21:41.891696+0000","last_fullsized":"2024-03-06T07:21:41.891696+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896366+0000","last_change":"2024-03-06T07:21:41.896366+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896366+0000","last_undegraded":"2024-03-06T07:21:41.896366+0000","last_fullsized":"2024-03-06T07:21:41.896366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894357+0000","last_change":"2024-03-06T07:21:41.894357+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894357+0000","last_undegraded":"2024-03-06T07:21:41.894357+0000","last_fullsized":"2024-03-06T07:21:41.894357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892217+0000","last_change":"2024-03-06T07:21:41.892217+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892217+0000","last_undegraded":"2024-03-06T07:21:41.892217+0000","last_fullsized":"2024-03-06T07:21:41.892217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895169+0000","last_change":"2024-03-06T07:21:41.895169+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895169+0000","last_undegraded":"2024-03-06T07:21:41.895169+0000","last_fullsized":"2024-03-06T07:21:41.895169+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894947+0000","last_change":"2024-03-06T07:21:41.894947+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894947+0000","last_undegraded":"2024-03-06T07:21:41.894947+0000","last_fullsized":"2024-03-06T07:21:41.894947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895280+0000","last_change":"2024-03-06T07:21:41.895280+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895280+0000","last_undegraded":"2024-03-06T07:21:41.895280+0000","last_fullsized":"2024-03-06T07:21:41.895280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891652+0000","last_change":"2024-03-06T07:21:41.891652+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891652+0000","last_undegraded":"2024-03-06T07:21:41.891652+0000","last_fullsized":"2024-03-06T07:21:41.891652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894337+0000","last_change":"2024-03-06T07:21:41.894337+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894337+0000","last_undegraded":"2024-03-06T07:21:41.894337+0000","last_fullsized":"2024-03-06T07:21:41.894337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894446+0000","last_change":"2024-03-06T07:21:41.894446+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894446+0000","last_undegraded":"2024-03-06T07:21:41.894446+0000","last_fullsized":"2024-03-06T07:21:41.894446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894750+0000","last_change":"2024-03-06T07:21:41.894750+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894750+0000","last_undegraded":"2024-03-06T07:21:41.894750+0000","last_fullsized":"2024-03-06T07:21:41.894750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.897741+0000","last_change":"2024-03-06T07:21:41.897741+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.897741+0000","last_undegraded":"2024-03-06T07:21:41.897741+0000","last_fullsized":"2024-03-06T07:21:41.897741+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891655+0000","last_change":"2024-03-06T07:21:41.891655+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891655+0000","last_undegraded":"2024-03-06T07:21:41.891655+0000","last_fullsized":"2024-03-06T07:21:41.891655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892574+0000","last_change":"2024-03-06T07:21:41.892574+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892574+0000","last_undegraded":"2024-03-06T07:21:41.892574+0000","last_fullsized":"2024-03-06T07:21:41.892574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892345+0000","last_change":"2024-03-06T07:21:41.892345+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892345+0000","last_undegraded":"2024-03-06T07:21:41.892345+0000","last_fullsized":"2024-03-06T07:21:41.892345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895440+0000","last_change":"2024-03-06T07:21:41.895440+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895440+0000","last_undegraded":"2024-03-06T07:21:41.895440+0000","last_fullsized":"2024-03-06T07:21:41.895440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894645+0000","last_change":"2024-03-06T07:21:41.894645+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894645+0000","last_undegraded":"2024-03-06T07:21:41.894645+0000","last_fullsized":"2024-03-06T07:21:41.894645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.898090+0000","last_change":"2024-03-06T07:21:41.898090+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.898090+0000","last_undegraded":"2024-03-06T07:21:41.898090+0000","last_fullsized":"2024-03-06T07:21:41.898090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891854+0000","last_change":"2024-03-06T07:21:41.891854+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891854+0000","last_undegraded":"2024-03-06T07:21:41.891854+0000","last_fullsized":"2024-03-06T07:21:41.891854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894386+0000","last_change":"2024-03-06T07:21:41.894386+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894386+0000","last_undegraded":"2024-03-06T07:21:41.894386+0000","last_fullsized":"2024-03-06T07:21:41.894386+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894656+0000","last_change":"2024-03-06T07:21:41.894656+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894656+0000","last_undegraded":"2024-03-06T07:21:41.894656+0000","last_fullsized":"2024-03-06T07:21:41.894656+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894657+0000","last_change":"2024-03-06T07:21:41.894657+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894657+0000","last_undegraded":"2024-03-06T07:21:41.894657+0000","last_fullsized":"2024-03-06T07:21:41.894657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894621+0000","last_change":"2024-03-06T07:21:41.894621+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894621+0000","last_undegraded":"2024-03-06T07:21:41.894621+0000","last_fullsized":"2024-03-06T07:21:41.894621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891640+0000","last_change":"2024-03-06T07:21:41.891640+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891640+0000","last_undegraded":"2024-03-06T07:21:41.891640+0000","last_fullsized":"2024-03-06T07:21:41.891640+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894803+0000","last_change":"2024-03-06T07:21:41.894803+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894803+0000","last_undegraded":"2024-03-06T07:21:41.894803+0000","last_fullsized":"2024-03-06T07:21:41.894803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895131+0000","last_change":"2024-03-06T07:21:41.895131+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895131+0000","last_undegraded":"2024-03-06T07:21:41.895131+0000","last_fullsized":"2024-03-06T07:21:41.895131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-06T07:21:41.895488+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:41.895488+0000","last_peered":"2024-03-06T07:21:41.895488+0000","last_clean":"2024-03-06T07:21:41.895488+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:41.895488+0000","last_undegraded":"2024-03-06T07:21:41.895488+0000","last_fullsized":"2024-03-06T07:21:41.895488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:47.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:48.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.096+0000 7ff4b8565640 1 Processor -- start 2024-03-06T07:21:48.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.096+0000 7ff4b8565640 1 -- start start 2024-03-06T07:21:48.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b8565640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4b0153990 0x7ff4b0156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b8565640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4b00acc00 0x7ff4b00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b8565640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b0070aa0 con 0x7ff4b01506b0 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b8565640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b0156450 con 0x7ff4b0153990 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b8565640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b0156a10 con 0x7ff4b00acc00 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4b0153990 0x7ff4b0156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4b0153990 0x7ff4b0156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37534/0 (socket says 172.21.15.17:37534) 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 -- 172.21.15.17:0/1383761249 learned_addr learned my addr 172.21.15.17:0/1383761249 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 -- 172.21.15.17:0/1383761249 >> v2:172.21.15.163:3300/0 conn(0x7ff4b00acc00 msgr2=0x7ff4b00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 --2- 172.21.15.17:0/1383761249 >> v2:172.21.15.163:3300/0 conn(0x7ff4b00acc00 0x7ff4b00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 -- 172.21.15.17:0/1383761249 >> v1:172.21.15.163:6789/0 conn(0x7ff4b01506b0 legacy=0x7ff4b00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.097+0000 7ff4b5b00640 1 -- 172.21.15.17:0/1383761249 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0070d80 con 0x7ff4b0153990 2024-03-06T07:21:48.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b5b00640 1 --2- 172.21.15.17:0/1383761249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b0153990 0x7ff4b0156ea0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00735e0 tx=0x7ff4a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9917dc3b769b07a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b52ff640 1 -- 172.21.15.17:0/1383761249 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0035020 con 0x7ff4b0153990 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b52ff640 1 -- 172.21.15.17:0/1383761249 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0031870 con 0x7ff4b0153990 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b52ff640 1 -- 172.21.15.17:0/1383761249 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0031c40 con 0x7ff4b0153990 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 -- 172.21.15.17:0/1383761249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b0153990 msgr2=0x7ff4b0156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 --2- 172.21.15.17:0/1383761249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b0153990 0x7ff4b0156ea0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00735e0 tx=0x7ff4a002b910 comp rx=0 tx=0).stop 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 -- 172.21.15.17:0/1383761249 shutdown_connections 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 --2- 172.21.15.17:0/1383761249 >> v2:172.21.15.163:3300/0 conn(0x7ff4b00acc00 0x7ff4b00c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 --2- 172.21.15.17:0/1383761249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b0153990 0x7ff4b0156ea0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 -- 172.21.15.17:0/1383761249 >> 172.21.15.17:0/1383761249 conn(0x7ff4b006b1a0 msgr2=0x7ff4b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.098+0000 7ff4b8565640 1 -- 172.21.15.17:0/1383761249 shutdown_connections 2024-03-06T07:21:48.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.099+0000 7ff4b8565640 1 -- 172.21.15.17:0/1383761249 wait complete. 2024-03-06T07:21:48.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 Processor -- start 2024-03-06T07:21:48.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 -- start start 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b00acc00 0x7ff4b01ec160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 0x7ff4b01466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c5b60 con 0x7ff4b01506b0 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c59e0 con 0x7ff4b0153990 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b8565640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b00c5ce0 con 0x7ff4b00acc00 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b5b00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b5b00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37536/0 (socket says 172.21.15.17:37536) 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b5b00640 1 -- 172.21.15.17:0/2444635571 learned_addr learned my addr 172.21.15.17:0/2444635571 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b6b02640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 0x7ff4b01466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.100+0000 7ff4b6301640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b00acc00 0x7ff4b01ec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 -- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b00acc00 msgr2=0x7ff4b01ec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b00acc00 0x7ff4b01ec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 -- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 msgr2=0x7ff4b01466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 0x7ff4b01466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 -- 172.21.15.17:0/2444635571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0152520 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b6b02640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 0x7ff4b01466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b5b00640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002b950 tx=0x7ff4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a003ad60 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a00443f0 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b0152800 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00447e0 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.101+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b01521b0 con 0x7ff4b01506b0 2024-03-06T07:21:48.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.102+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b004f330 con 0x7ff4b01506b0 2024-03-06T07:21:48.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.103+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4a004b060 con 0x7ff4b01506b0 2024-03-06T07:21:48.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.103+0000 7ff4a6ffd640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.103+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4a00ac470 con 0x7ff4b01506b0 2024-03-06T07:21:48.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.103+0000 7ff4b6301640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.104+0000 7ff4b6301640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 0x7ff49406d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00aa600 tx=0x7ff4ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.107+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4a003f070 con 0x7ff4b01506b0 2024-03-06T07:21:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.286+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4b014abf0 con 0x7ff49406b4d0 2024-03-06T07:21:48.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.292+0000 7ff4a6ffd640 1 -- 172.21.15.17:0/2444635571 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240122 (secure 0 0 0) 0x7ff4b014abf0 con 0x7ff49406b4d0 2024-03-06T07:21:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:48.298 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:48.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 msgr2=0x7ff49406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 0x7ff49406d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff4b00aa600 tx=0x7ff4ac009040 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 msgr2=0x7ff4b00c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002b950 tx=0x7ff4a00046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 shutdown_connections 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4b0153990 0x7ff4b01466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4b01506b0 0x7ff4b00c4770 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 --2- 172.21.15.17:0/2444635571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4b00acc00 0x7ff4b01ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 >> 172.21.15.17:0/2444635571 conn(0x7ff4b006b1a0 msgr2=0x7ff4b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 shutdown_connections 2024-03-06T07:21:48.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.295+0000 7ff4b8565640 1 -- 172.21.15.17:0/2444635571 wait complete. 2024-03-06T07:21:48.308 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-03-06T07:21:47.363706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237368,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737352,"statfs":{"total":773094113280,"available":772851048448,"internally_reserved":0,"allocated":4648960,"data_stored":3177117,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.473970"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.893191+0000","last_change":"2024-03-06T07:21:41.893191+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.893191+0000","last_undegraded":"2024-03-06T07:21:41.893191+0000","last_fullsized":"2024-03-06T07:21:41.893191+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894826+0000","last_change":"2024-03-06T07:21:41.894826+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894826+0000","last_undegraded":"2024-03-06T07:21:41.894826+0000","last_fullsized":"2024-03-06T07:21:41.894826+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896118+0000","last_change":"2024-03-06T07:21:41.896118+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896118+0000","last_undegraded":"2024-03-06T07:21:41.896118+0000","last_fullsized":"2024-03-06T07:21:41.896118+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892875+0000","last_change":"2024-03-06T07:21:41.892875+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892875+0000","last_undegraded":"2024-03-06T07:21:41.892875+0000","last_fullsized":"2024-03-06T07:21:41.892875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891696+0000","last_change":"2024-03-06T07:21:41.891696+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891696+0000","last_undegraded":"2024-03-06T07:21:41.891696+0000","last_fullsized":"2024-03-06T07:21:41.891696+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896366+0000","last_change":"2024-03-06T07:21:41.896366+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896366+0000","last_undegraded":"2024-03-06T07:21:41.896366+0000","last_fullsized":"2024-03-06T07:21:41.896366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894357+0000","last_change":"2024-03-06T07:21:41.894357+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894357+0000","last_undegraded":"2024-03-06T07:21:41.894357+0000","last_fullsized":"2024-03-06T07:21:41.894357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892217+0000","last_change":"2024-03-06T07:21:41.892217+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892217+0000","last_undegraded":"2024-03-06T07:21:41.892217+0000","last_fullsized":"2024-03-06T07:21:41.892217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895169+0000","last_change":"2024-03-06T07:21:41.895169+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895169+0000","last_undegraded":"2024-03-06T07:21:41.895169+0000","last_fullsized":"2024-03-06T07:21:41.895169+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894947+0000","last_change":"2024-03-06T07:21:41.894947+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894947+0000","last_undegraded":"2024-03-06T07:21:41.894947+0000","last_fullsized":"2024-03-06T07:21:41.894947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895280+0000","last_change":"2024-03-06T07:21:41.895280+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895280+0000","last_undegraded":"2024-03-06T07:21:41.895280+0000","last_fullsized":"2024-03-06T07:21:41.895280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891652+0000","last_change":"2024-03-06T07:21:41.891652+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891652+0000","last_undegraded":"2024-03-06T07:21:41.891652+0000","last_fullsized":"2024-03-06T07:21:41.891652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894337+0000","last_change":"2024-03-06T07:21:41.894337+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894337+0000","last_undegraded":"2024-03-06T07:21:41.894337+0000","last_fullsized":"2024-03-06T07:21:41.894337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894446+0000","last_change":"2024-03-06T07:21:41.894446+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894446+0000","last_undegraded":"2024-03-06T07:21:41.894446+0000","last_fullsized":"2024-03-06T07:21:41.894446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894750+0000","last_change":"2024-03-06T07:21:41.894750+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894750+0000","last_undegraded":"2024-03-06T07:21:41.894750+0000","last_fullsized":"2024-03-06T07:21:41.894750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.897741+0000","last_change":"2024-03-06T07:21:41.897741+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.897741+0000","last_undegraded":"2024-03-06T07:21:41.897741+0000","last_fullsized":"2024-03-06T07:21:41.897741+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891655+0000","last_change":"2024-03-06T07:21:41.891655+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891655+0000","last_undegraded":"2024-03-06T07:21:41.891655+0000","last_fullsized":"2024-03-06T07:21:41.891655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892574+0000","last_change":"2024-03-06T07:21:41.892574+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892574+0000","last_undegraded":"2024-03-06T07:21:41.892574+0000","last_fullsized":"2024-03-06T07:21:41.892574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892345+0000","last_change":"2024-03-06T07:21:41.892345+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892345+0000","last_undegraded":"2024-03-06T07:21:41.892345+0000","last_fullsized":"2024-03-06T07:21:41.892345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895440+0000","last_change":"2024-03-06T07:21:41.895440+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895440+0000","last_undegraded":"2024-03-06T07:21:41.895440+0000","last_fullsized":"2024-03-06T07:21:41.895440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894645+0000","last_change":"2024-03-06T07:21:41.894645+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894645+0000","last_undegraded":"2024-03-06T07:21:41.894645+0000","last_fullsized":"2024-03-06T07:21:41.894645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.898090+0000","last_change":"2024-03-06T07:21:41.898090+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.898090+0000","last_undegraded":"2024-03-06T07:21:41.898090+0000","last_fullsized":"2024-03-06T07:21:41.898090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891854+0000","last_change":"2024-03-06T07:21:41.891854+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891854+0000","last_undegraded":"2024-03-06T07:21:41.891854+0000","last_fullsized":"2024-03-06T07:21:41.891854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894386+0000","last_change":"2024-03-06T07:21:41.894386+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894386+0000","last_undegraded":"2024-03-06T07:21:41.894386+0000","last_fullsized":"2024-03-06T07:21:41.894386+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894656+0000","last_change":"2024-03-06T07:21:41.894656+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894656+0000","last_undegraded":"2024-03-06T07:21:41.894656+0000","last_fullsized":"2024-03-06T07:21:41.894656+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894657+0000","last_change":"2024-03-06T07:21:41.894657+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894657+0000","last_undegraded":"2024-03-06T07:21:41.894657+0000","last_fullsized":"2024-03-06T07:21:41.894657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894621+0000","last_change":"2024-03-06T07:21:41.894621+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894621+0000","last_undegraded":"2024-03-06T07:21:41.894621+0000","last_fullsized":"2024-03-06T07:21:41.894621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891640+0000","last_change":"2024-03-06T07:21:41.891640+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891640+0000","last_undegraded":"2024-03-06T07:21:41.891640+0000","last_fullsized":"2024-03-06T07:21:41.891640+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894803+0000","last_change":"2024-03-06T07:21:41.894803+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894803+0000","last_undegraded":"2024-03-06T07:21:41.894803+0000","last_fullsized":"2024-03-06T07:21:41.894803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895131+0000","last_change":"2024-03-06T07:21:41.895131+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895131+0000","last_undegraded":"2024-03-06T07:21:41.895131+0000","last_fullsized":"2024-03-06T07:21:41.895131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-06T07:21:41.895488+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:41.895488+0000","last_peered":"2024-03-06T07:21:41.895488+0000","last_clean":"2024-03-06T07:21:41.895488+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:41.895488+0000","last_undegraded":"2024-03-06T07:21:41.895488+0000","last_fullsized":"2024-03-06T07:21:41.895488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:48.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:48.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.487+0000 7f42f0da0640 1 Processor -- start 2024-03-06T07:21:48.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.487+0000 7f42f0da0640 1 -- start start 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42f0da0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42ec14ba90 0x7f42ec1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42f0da0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42ec0c8690 0x7f42ec0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42f0da0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec14be70 con 0x7f42ec0c5650 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42f0da0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0cad30 con 0x7f42ec14ba90 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42f0da0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0cb010 con 0x7f42ec0c8690 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.488+0000 7f42eb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42ec14ba90 0x7f42ec1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42ebfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f42ec0c5650 0x7f42ec158a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39630/0 (socket says 172.21.15.17:39630) 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42ebfff640 1 -- 172.21.15.17:0/95062357 learned_addr learned my addr 172.21.15.17:0/95062357 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42ec14ba90 0x7f42ec1559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37538/0 (socket says 172.21.15.17:37538) 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eaffd640 1 --2- 172.21.15.17:0/95062357 >> v2:172.21.15.163:3300/0 conn(0x7f42ec0c8690 0x7f42ec0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eb7fe640 1 -- 172.21.15.17:0/95062357 >> v2:172.21.15.163:3300/0 conn(0x7f42ec0c8690 msgr2=0x7f42ec0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eb7fe640 1 --2- 172.21.15.17:0/95062357 >> v2:172.21.15.163:3300/0 conn(0x7f42ec0c8690 0x7f42ec0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eb7fe640 1 -- 172.21.15.17:0/95062357 >> v1:172.21.15.17:6789/0 conn(0x7f42ec0c5650 legacy=0x7f42ec158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42eb7fe640 1 -- 172.21.15.17:0/95062357 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0cb2f0 con 0x7f42ec14ba90 2024-03-06T07:21:48.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.489+0000 7f42ea7fc640 1 -- 172.21.15.17:0/95062357 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508493000 0 0) 0x7f42e4001490 con 0x7f42ec0c5650 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42eb7fe640 1 --2- 172.21.15.17:0/95062357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec14ba90 0x7f42ec1559b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0c8250 tx=0x7f42d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f25a84f06052026 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42ea7fc640 1 -- 172.21.15.17:0/95062357 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4035020 con 0x7f42ec14ba90 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42ea7fc640 1 -- 172.21.15.17:0/95062357 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4031830 con 0x7f42ec14ba90 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42ea7fc640 1 -- 172.21.15.17:0/95062357 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4031c00 con 0x7f42ec14ba90 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 -- 172.21.15.17:0/95062357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec14ba90 msgr2=0x7f42ec1559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 --2- 172.21.15.17:0/95062357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec14ba90 0x7f42ec1559b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0c8250 tx=0x7f42d402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 -- 172.21.15.17:0/95062357 shutdown_connections 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 --2- 172.21.15.17:0/95062357 >> v2:172.21.15.163:3300/0 conn(0x7f42ec0c8690 0x7f42ec0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 --2- 172.21.15.17:0/95062357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec14ba90 0x7f42ec1559b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 -- 172.21.15.17:0/95062357 >> 172.21.15.17:0/95062357 conn(0x7f42ec0728d0 msgr2=0x7f42ec072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 -- 172.21.15.17:0/95062357 shutdown_connections 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.490+0000 7f42f0da0640 1 -- 172.21.15.17:0/95062357 wait complete. 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.491+0000 7f42f0da0640 1 Processor -- start 2024-03-06T07:21:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 -- start start 2024-03-06T07:21:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42ec0c8690 0x7f42ec149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42ec14ba90 0x7f42ec13b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0c9240 con 0x7f42ec0c5650 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec06cf60 con 0x7f42ec0c8690 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42f0da0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec152360 con 0x7f42ec14ba90 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37552/0 (socket says 172.21.15.17:37552) 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.492+0000 7f42eb7fe640 1 -- 172.21.15.17:0/2911928432 learned_addr learned my addr 172.21.15.17:0/2911928432 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 -- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42ec14ba90 msgr2=0x7f42ec13b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42ec14ba90 0x7f42ec13b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 -- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42ec0c8690 msgr2=0x7f42ec149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42ec0c8690 0x7f42ec149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 -- 172.21.15.17:0/2911928432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec1eb5f0 con 0x7f42ec0c5650 2024-03-06T07:21:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42eb7fe640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4035040 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4004250 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec1eb8d0 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d403a700 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.493+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec13f990 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.494+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec069430 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.494+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42d4038020 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.495+0000 7f42e3fff640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.495+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42d40ac120 con 0x7f42ec0c5650 2024-03-06T07:21:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.495+0000 7f42eaffd640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.496+0000 7f42eaffd640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42ec141620 tx=0x7f42dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.499+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f42d407aea0 con 0x7f42ec0c5650 2024-03-06T07:21:48.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.675+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42ec06cd40 con 0x7f42cc06b4d0 2024-03-06T07:21:48.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.681+0000 7f42e3fff640 1 -- 172.21.15.17:0/2911928432 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240122 (secure 0 0 0) 0x7f42ec06cd40 con 0x7f42cc06b4d0 2024-03-06T07:21:48.687 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:48.688 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 msgr2=0x7f42cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42ec141620 tx=0x7f42dc008040 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 msgr2=0x7f42ec1410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f42d4000c00 tx=0x7f42d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 shutdown_connections 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42ec14ba90 0x7f42ec13b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42ec0c8690 0x7f42ec149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 --2- 172.21.15.17:0/2911928432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ec0c5650 0x7f42ec1410e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 >> 172.21.15.17:0/2911928432 conn(0x7f42ec0728d0 msgr2=0x7f42ec145ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 shutdown_connections 2024-03-06T07:21:48.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.684+0000 7f42f0da0640 1 -- 172.21.15.17:0/2911928432 wait complete. 2024-03-06T07:21:48.698 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-03-06T07:21:47.363706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237368,"kb_used_data":4540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737352,"statfs":{"total":773094113280,"available":772851048448,"internally_reserved":0,"allocated":4648960,"data_stored":3177117,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.473970"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.893191+0000","last_change":"2024-03-06T07:21:41.893191+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.893191+0000","last_undegraded":"2024-03-06T07:21:41.893191+0000","last_fullsized":"2024-03-06T07:21:41.893191+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892305+0000","last_change":"2024-03-06T07:21:41.892305+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892305+0000","last_undegraded":"2024-03-06T07:21:41.892305+0000","last_fullsized":"2024-03-06T07:21:41.892305+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894826+0000","last_change":"2024-03-06T07:21:41.894826+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894826+0000","last_undegraded":"2024-03-06T07:21:41.894826+0000","last_fullsized":"2024-03-06T07:21:41.894826+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896118+0000","last_change":"2024-03-06T07:21:41.896118+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896118+0000","last_undegraded":"2024-03-06T07:21:41.896118+0000","last_fullsized":"2024-03-06T07:21:41.896118+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892875+0000","last_change":"2024-03-06T07:21:41.892875+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892875+0000","last_undegraded":"2024-03-06T07:21:41.892875+0000","last_fullsized":"2024-03-06T07:21:41.892875+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891696+0000","last_change":"2024-03-06T07:21:41.891696+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891696+0000","last_undegraded":"2024-03-06T07:21:41.891696+0000","last_fullsized":"2024-03-06T07:21:41.891696+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.896366+0000","last_change":"2024-03-06T07:21:41.896366+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.896366+0000","last_undegraded":"2024-03-06T07:21:41.896366+0000","last_fullsized":"2024-03-06T07:21:41.896366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894357+0000","last_change":"2024-03-06T07:21:41.894357+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894357+0000","last_undegraded":"2024-03-06T07:21:41.894357+0000","last_fullsized":"2024-03-06T07:21:41.894357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892217+0000","last_change":"2024-03-06T07:21:41.892217+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892217+0000","last_undegraded":"2024-03-06T07:21:41.892217+0000","last_fullsized":"2024-03-06T07:21:41.892217+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895169+0000","last_change":"2024-03-06T07:21:41.895169+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895169+0000","last_undegraded":"2024-03-06T07:21:41.895169+0000","last_fullsized":"2024-03-06T07:21:41.895169+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894947+0000","last_change":"2024-03-06T07:21:41.894947+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894947+0000","last_undegraded":"2024-03-06T07:21:41.894947+0000","last_fullsized":"2024-03-06T07:21:41.894947+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895280+0000","last_change":"2024-03-06T07:21:41.895280+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895280+0000","last_undegraded":"2024-03-06T07:21:41.895280+0000","last_fullsized":"2024-03-06T07:21:41.895280+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891652+0000","last_change":"2024-03-06T07:21:41.891652+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891652+0000","last_undegraded":"2024-03-06T07:21:41.891652+0000","last_fullsized":"2024-03-06T07:21:41.891652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894337+0000","last_change":"2024-03-06T07:21:41.894337+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894337+0000","last_undegraded":"2024-03-06T07:21:41.894337+0000","last_fullsized":"2024-03-06T07:21:41.894337+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894446+0000","last_change":"2024-03-06T07:21:41.894446+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894446+0000","last_undegraded":"2024-03-06T07:21:41.894446+0000","last_fullsized":"2024-03-06T07:21:41.894446+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894750+0000","last_change":"2024-03-06T07:21:41.894750+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894750+0000","last_undegraded":"2024-03-06T07:21:41.894750+0000","last_fullsized":"2024-03-06T07:21:41.894750+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.897741+0000","last_change":"2024-03-06T07:21:41.897741+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.897741+0000","last_undegraded":"2024-03-06T07:21:41.897741+0000","last_fullsized":"2024-03-06T07:21:41.897741+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891655+0000","last_change":"2024-03-06T07:21:41.891655+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891655+0000","last_undegraded":"2024-03-06T07:21:41.891655+0000","last_fullsized":"2024-03-06T07:21:41.891655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892574+0000","last_change":"2024-03-06T07:21:41.892574+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892574+0000","last_undegraded":"2024-03-06T07:21:41.892574+0000","last_fullsized":"2024-03-06T07:21:41.892574+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.892345+0000","last_change":"2024-03-06T07:21:41.892345+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.892345+0000","last_undegraded":"2024-03-06T07:21:41.892345+0000","last_fullsized":"2024-03-06T07:21:41.892345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895440+0000","last_change":"2024-03-06T07:21:41.895440+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895440+0000","last_undegraded":"2024-03-06T07:21:41.895440+0000","last_fullsized":"2024-03-06T07:21:41.895440+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894645+0000","last_change":"2024-03-06T07:21:41.894645+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894645+0000","last_undegraded":"2024-03-06T07:21:41.894645+0000","last_fullsized":"2024-03-06T07:21:41.894645+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.898090+0000","last_change":"2024-03-06T07:21:41.898090+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.898090+0000","last_undegraded":"2024-03-06T07:21:41.898090+0000","last_fullsized":"2024-03-06T07:21:41.898090+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891854+0000","last_change":"2024-03-06T07:21:41.891854+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891854+0000","last_undegraded":"2024-03-06T07:21:41.891854+0000","last_fullsized":"2024-03-06T07:21:41.891854+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894386+0000","last_change":"2024-03-06T07:21:41.894386+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894386+0000","last_undegraded":"2024-03-06T07:21:41.894386+0000","last_fullsized":"2024-03-06T07:21:41.894386+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894656+0000","last_change":"2024-03-06T07:21:41.894656+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894656+0000","last_undegraded":"2024-03-06T07:21:41.894656+0000","last_fullsized":"2024-03-06T07:21:41.894656+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894657+0000","last_change":"2024-03-06T07:21:41.894657+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894657+0000","last_undegraded":"2024-03-06T07:21:41.894657+0000","last_fullsized":"2024-03-06T07:21:41.894657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894621+0000","last_change":"2024-03-06T07:21:41.894621+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894621+0000","last_undegraded":"2024-03-06T07:21:41.894621+0000","last_fullsized":"2024-03-06T07:21:41.894621+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.891640+0000","last_change":"2024-03-06T07:21:41.891640+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.891640+0000","last_undegraded":"2024-03-06T07:21:41.891640+0000","last_fullsized":"2024-03-06T07:21:41.891640+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.894803+0000","last_change":"2024-03-06T07:21:41.894803+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.894803+0000","last_undegraded":"2024-03-06T07:21:41.894803+0000","last_fullsized":"2024-03-06T07:21:41.894803+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:21:42.889330+0000","last_change":"2024-03-06T07:21:42.889330+0000","last_active":"2024-03-06T07:21:42.889330+0000","last_peered":"2024-03-06T07:21:42.889330+0000","last_clean":"2024-03-06T07:21:42.889330+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:42.889330+0000","last_undegraded":"2024-03-06T07:21:42.889330+0000","last_fullsized":"2024-03-06T07:21:42.889330+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":48,"state":"creating+peering","last_fresh":"2024-03-06T07:21:41.895131+0000","last_change":"2024-03-06T07:21:41.895131+0000","last_active":"2024-03-06T07:21:41.884298+0000","last_peered":"2024-03-06T07:21:41.884298+0000","last_clean":"2024-03-06T07:21:41.884298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:21:41.895131+0000","last_undegraded":"2024-03-06T07:21:41.895131+0000","last_fullsized":"2024-03-06T07:21:41.895131+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-06T07:21:41.895488+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:41.895488+0000","last_peered":"2024-03-06T07:21:41.895488+0000","last_clean":"2024-03-06T07:21:41.895488+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:41.895488+0000","last_undegraded":"2024-03-06T07:21:41.895488+0000","last_fullsized":"2024-03-06T07:21:41.895488+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":251876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":557056,"data_stored":245667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":842244,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.701 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-03-06T07:21:48.702 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-03-06T07:21:48.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-03-06T07:21:48.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.893191+0000', 'last_change': '2024-03-06T07:21:41.893191+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.893191+0000', 'last_undegraded': '2024-03-06T07:21:41.893191+0000', 'last_fullsized': '2024-03-06T07:21:41.893191+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892305+0000', 'last_change': '2024-03-06T07:21:41.892305+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892305+0000', 'last_undegraded': '2024-03-06T07:21:41.892305+0000', 'last_fullsized': '2024-03-06T07:21:41.892305+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-03-06T07:21:48.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892305+0000', 'last_change': '2024-03-06T07:21:41.892305+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892305+0000', 'last_undegraded': '2024-03-06T07:21:41.892305+0000', 'last_fullsized': '2024-03-06T07:21:41.892305+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894826+0000', 'last_change': '2024-03-06T07:21:41.894826+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894826+0000', 'last_undegraded': '2024-03-06T07:21:41.894826+0000', 'last_fullsized': '2024-03-06T07:21:41.894826+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.896118+0000', 'last_change': '2024-03-06T07:21:41.896118+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.896118+0000', 'last_undegraded': '2024-03-06T07:21:41.896118+0000', 'last_fullsized': '2024-03-06T07:21:41.896118+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-03-06T07:21:48.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892875+0000', 'last_change': '2024-03-06T07:21:41.892875+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892875+0000', 'last_undegraded': '2024-03-06T07:21:41.892875+0000', 'last_fullsized': '2024-03-06T07:21:41.892875+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.891696+0000', 'last_change': '2024-03-06T07:21:41.891696+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.891696+0000', 'last_undegraded': '2024-03-06T07:21:41.891696+0000', 'last_fullsized': '2024-03-06T07:21:41.891696+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.896366+0000', 'last_change': '2024-03-06T07:21:41.896366+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.896366+0000', 'last_undegraded': '2024-03-06T07:21:41.896366+0000', 'last_fullsized': '2024-03-06T07:21:41.896366+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-03-06T07:21:48.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894357+0000', 'last_change': '2024-03-06T07:21:41.894357+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894357+0000', 'last_undegraded': '2024-03-06T07:21:41.894357+0000', 'last_fullsized': '2024-03-06T07:21:41.894357+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892217+0000', 'last_change': '2024-03-06T07:21:41.892217+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892217+0000', 'last_undegraded': '2024-03-06T07:21:41.892217+0000', 'last_fullsized': '2024-03-06T07:21:41.892217+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.895169+0000', 'last_change': '2024-03-06T07:21:41.895169+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.895169+0000', 'last_undegraded': '2024-03-06T07:21:41.895169+0000', 'last_fullsized': '2024-03-06T07:21:41.895169+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894947+0000', 'last_change': '2024-03-06T07:21:41.894947+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894947+0000', 'last_undegraded': '2024-03-06T07:21:41.894947+0000', 'last_fullsized': '2024-03-06T07:21:41.894947+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-03-06T07:21:48.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.895280+0000', 'last_change': '2024-03-06T07:21:41.895280+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.895280+0000', 'last_undegraded': '2024-03-06T07:21:41.895280+0000', 'last_fullsized': '2024-03-06T07:21:41.895280+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.891652+0000', 'last_change': '2024-03-06T07:21:41.891652+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.891652+0000', 'last_undegraded': '2024-03-06T07:21:41.891652+0000', 'last_fullsized': '2024-03-06T07:21:41.891652+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-03-06T07:21:48.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894337+0000', 'last_change': '2024-03-06T07:21:41.894337+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894337+0000', 'last_undegraded': '2024-03-06T07:21:41.894337+0000', 'last_fullsized': '2024-03-06T07:21:41.894337+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894446+0000', 'last_change': '2024-03-06T07:21:41.894446+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894446+0000', 'last_undegraded': '2024-03-06T07:21:41.894446+0000', 'last_fullsized': '2024-03-06T07:21:41.894446+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894750+0000', 'last_change': '2024-03-06T07:21:41.894750+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894750+0000', 'last_undegraded': '2024-03-06T07:21:41.894750+0000', 'last_fullsized': '2024-03-06T07:21:41.894750+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.897741+0000', 'last_change': '2024-03-06T07:21:41.897741+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.897741+0000', 'last_undegraded': '2024-03-06T07:21:41.897741+0000', 'last_fullsized': '2024-03-06T07:21:41.897741+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.891655+0000', 'last_change': '2024-03-06T07:21:41.891655+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.891655+0000', 'last_undegraded': '2024-03-06T07:21:41.891655+0000', 'last_fullsized': '2024-03-06T07:21:41.891655+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892574+0000', 'last_change': '2024-03-06T07:21:41.892574+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892574+0000', 'last_undegraded': '2024-03-06T07:21:41.892574+0000', 'last_fullsized': '2024-03-06T07:21:41.892574+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-03-06T07:21:48.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.892345+0000', 'last_change': '2024-03-06T07:21:41.892345+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.892345+0000', 'last_undegraded': '2024-03-06T07:21:41.892345+0000', 'last_fullsized': '2024-03-06T07:21:41.892345+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.895440+0000', 'last_change': '2024-03-06T07:21:41.895440+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.895440+0000', 'last_undegraded': '2024-03-06T07:21:41.895440+0000', 'last_fullsized': '2024-03-06T07:21:41.895440+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894645+0000', 'last_change': '2024-03-06T07:21:41.894645+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894645+0000', 'last_undegraded': '2024-03-06T07:21:41.894645+0000', 'last_fullsized': '2024-03-06T07:21:41.894645+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-03-06T07:21:48.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.898090+0000', 'last_change': '2024-03-06T07:21:41.898090+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.898090+0000', 'last_undegraded': '2024-03-06T07:21:41.898090+0000', 'last_fullsized': '2024-03-06T07:21:41.898090+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.891854+0000', 'last_change': '2024-03-06T07:21:41.891854+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.891854+0000', 'last_undegraded': '2024-03-06T07:21:41.891854+0000', 'last_fullsized': '2024-03-06T07:21:41.891854+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894386+0000', 'last_change': '2024-03-06T07:21:41.894386+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894386+0000', 'last_undegraded': '2024-03-06T07:21:41.894386+0000', 'last_fullsized': '2024-03-06T07:21:41.894386+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:21:48.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894656+0000', 'last_change': '2024-03-06T07:21:41.894656+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894656+0000', 'last_undegraded': '2024-03-06T07:21:41.894656+0000', 'last_fullsized': '2024-03-06T07:21:41.894656+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894657+0000', 'last_change': '2024-03-06T07:21:41.894657+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894657+0000', 'last_undegraded': '2024-03-06T07:21:41.894657+0000', 'last_fullsized': '2024-03-06T07:21:41.894657+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-03-06T07:21:48.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894621+0000', 'last_change': '2024-03-06T07:21:41.894621+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894621+0000', 'last_undegraded': '2024-03-06T07:21:41.894621+0000', 'last_fullsized': '2024-03-06T07:21:41.894621+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.891640+0000', 'last_change': '2024-03-06T07:21:41.891640+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.891640+0000', 'last_undegraded': '2024-03-06T07:21:41.891640+0000', 'last_fullsized': '2024-03-06T07:21:41.891640+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.894803+0000', 'last_change': '2024-03-06T07:21:41.894803+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.894803+0000', 'last_undegraded': '2024-03-06T07:21:41.894803+0000', 'last_fullsized': '2024-03-06T07:21:41.894803+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:21:42.889330+0000', 'last_change': '2024-03-06T07:21:42.889330+0000', 'last_active': '2024-03-06T07:21:42.889330+0000', 'last_peered': '2024-03-06T07:21:42.889330+0000', 'last_clean': '2024-03-06T07:21:42.889330+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:42.889330+0000', 'last_undegraded': '2024-03-06T07:21:42.889330+0000', 'last_fullsized': '2024-03-06T07:21:42.889330+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:42.889330+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-03-06T07:21:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 48, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:21:41.895131+0000', 'last_change': '2024-03-06T07:21:41.895131+0000', 'last_active': '2024-03-06T07:21:41.884298+0000', 'last_peered': '2024-03-06T07:21:41.884298+0000', 'last_clean': '2024-03-06T07:21:41.884298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:21:41.895131+0000', 'last_undegraded': '2024-03-06T07:21:41.895131+0000', 'last_fullsized': '2024-03-06T07:21:41.895131+0000', 'mapping_epoch': 48, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 48, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'last_clean_scrub_stamp': '2024-03-06T07:21:41.884298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:21:48.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:21:48.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.840+0000 7f27b4fa0640 1 Processor -- start 2024-03-06T07:21:48.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.840+0000 7f27b4fa0640 1 -- start start 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27b4fa0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f27b00c3420 0x7f27b00c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27b4fa0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b006ea10 con 0x7f27b00c5510 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27b4fa0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b006ecf0 con 0x7f27b00c6a70 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27b4fa0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b006fab0 con 0x7f27b00c3420 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27af7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f27b00c5510 0x7f27b0156ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39634/0 (socket says 172.21.15.17:39634) 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27af7fe640 1 -- 172.21.15.17:0/340324222 learned_addr learned my addr 172.21.15.17:0/340324222 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007258068 0 0) 0x7f27b006ea10 con 0x7f27b00c5510 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2798003520 con 0x7f27b00c5510 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.841+0000 7f27aeffd640 1 --2- 172.21.15.17:0/340324222 >> v2:172.21.15.163:3300/0 conn(0x7f27b00c3420 0x7f27b00c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119919004 0 0) 0x7f2798003520 con 0x7f27b00c5510 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 >> v2:172.21.15.163:3300/0 conn(0x7f27b00c3420 msgr2=0x7f27b00c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 --2- 172.21.15.17:0/340324222 >> v2:172.21.15.163:3300/0 conn(0x7f27b00c3420 0x7f27b00c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 >> v1:172.21.15.163:6789/0 conn(0x7f27b00c6a70 legacy=0x7f27b00c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b006efd0 con 0x7f27b00c5510 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f27a0001670 con 0x7f27b00c5510 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 >> v1:172.21.15.17:6789/0 conn(0x7f27b00c5510 legacy=0x7f27b0156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 0x7f2798003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798004800 0x7f2798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27980052e0 0x7f279800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b006fab0 con 0x7f2798003a70 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279800c1b0 con 0x7f27980052e0 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279800c3b0 con 0x7f2798004800 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 0x7f2798003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27aeffd640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798004800 0x7f2798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27affff640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27980052e0 0x7f279800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 -- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798004800 msgr2=0x7f2798004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798004800 0x7f2798004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 -- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27980052e0 msgr2=0x7f279800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27980052e0 0x7f279800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27affff640 1 -- 172.21.15.17:0/340324222 reap_dead start 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.842+0000 7f27af7fe640 1 -- 172.21.15.17:0/340324222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798003520 con 0x7f2798003a70 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.843+0000 7f27af7fe640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 0x7f2798003e90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c240 tx=0x7f2798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109802b615543d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.843+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0007cf0 con 0x7f2798003a70 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.843+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0030070 con 0x7f2798003a70 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.843+0000 7f27ae7fc640 1 -- 172.21.15.17:0/340324222 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0005d70 con 0x7f2798003a70 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 -- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 msgr2=0x7f2798003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 0x7f2798003e90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c240 tx=0x7f2798005830 comp rx=0 tx=0).stop 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 -- 172.21.15.17:0/340324222 shutdown_connections 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/340324222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2798003a70 0x7f2798003e90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 -- 172.21.15.17:0/340324222 >> 172.21.15.17:0/340324222 conn(0x7f27b006b6d0 msgr2=0x7f27b00c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:48.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 -- 172.21.15.17:0/340324222 shutdown_connections 2024-03-06T07:21:48.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.844+0000 7f27b4fa0640 1 -- 172.21.15.17:0/340324222 wait complete. 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 Processor -- start 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 -- start start 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 0x7f27b0148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 0x7f27b014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00c84d0 con 0x7f27b01ec260 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b006f590 con 0x7f27b01ebde0 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.845+0000 7f27b4fa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00c8650 con 0x7f2798003a70 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37562/0 (socket says 172.21.15.17:37562) 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 -- 172.21.15.17:0/1319921017 learned_addr learned my addr 172.21.15.17:0/1319921017 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27af7fe640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 0x7f27b0148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27aeffd640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 0x7f27b014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 -- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 msgr2=0x7f27b0148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 0x7f27b0148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 -- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 msgr2=0x7f27b014e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 0x7f27b014e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 -- 172.21.15.17:0/1319921017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b014a320 con 0x7f27b01ec260 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27aeffd640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 0x7f27b014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27af7fe640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 0x7f27b0148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27affff640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f27b00c4940 tx=0x7f27a800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8019070 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8006b50 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00ab340 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8003970 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.846+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0149d60 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.847+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b0069430 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.848+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27a801c070 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.848+0000 7f2797fff640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 0x7f279c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.848+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27a8089940 con 0x7f27b01ec260 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.848+0000 7f27af7fe640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 0x7f279c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:48.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.849+0000 7f27af7fe640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 0x7f279c06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a0039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:48.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:48.852+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f27a80586c0 con 0x7f27b01ec260 2024-03-06T07:21:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.254+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f27b00adda0 con 0x7f27b01ec260 2024-03-06T07:21:49.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.254+0000 7f2797fff640 1 -- 172.21.15.17:0/1319921017 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1528 (secure 0 0 0) 0x7f27a8057fc0 con 0x7f27b01ec260 2024-03-06T07:21:49.257 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:49.257 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":122,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709597,"num_in_osds":8,"osd_in_since":1709709589,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":243064832,"bytes_avail":772851048448,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":31,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5772}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-06T07:21:31.360491+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:21:49.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.256+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 msgr2=0x7f279c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.256+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 0x7f279c06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a0039040 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.256+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 msgr2=0x7f27b014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.256+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f27b00c4940 tx=0x7f27a800eb90 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 shutdown_connections 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f279c06b500 0x7f279c06d9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27b01ec260 0x7f27b014eb80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f27b01ebde0 0x7f27b014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 --2- 172.21.15.17:0/1319921017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2798003a70 0x7f27b0148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 >> 172.21.15.17:0/1319921017 conn(0x7f27b006b6d0 msgr2=0x7f27b01529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 shutdown_connections 2024-03-06T07:21:49.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.257+0000 7f27b4fa0640 1 -- 172.21.15.17:0/1319921017 wait complete. 2024-03-06T07:21:49.271 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:21:49.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:49.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.370+0000 7f13a3ff7640 1 Processor -- start 2024-03-06T07:21:49.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.371+0000 7f13a3ff7640 1 -- start start 2024-03-06T07:21:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a3ff7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f139c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a3ff7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c0c73d0 con 0x7f139c14f360 2024-03-06T07:21:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a3ff7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139c0be2e0 con 0x7f139c158b60 2024-03-06T07:21:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a3ff7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c159700 con 0x7f139c0c88c0 2024-03-06T07:21:49.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a1d93640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f139c14f360 0x7f139c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39644/0 (socket says 172.21.15.17:39644) 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a1d93640 1 -- 172.21.15.17:0/3330707617 learned_addr learned my addr 172.21.15.17:0/3330707617 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214172012 0 0) 0x7f139c0c73d0 con 0x7f139c14f360 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a1592640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f139c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1388003520 con 0x7f139c14f360 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625183511 0 0) 0x7f1388003520 con 0x7f139c14f360 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 msgr2=0x7f139c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f139c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 >> v1:172.21.15.163:6789/0 conn(0x7f139c158b60 legacy=0x7f139c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c0c7670 con 0x7f139c14f360 2024-03-06T07:21:49.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1394001670 con 0x7f139c14f360 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 >> v1:172.21.15.17:6789/0 conn(0x7f139c14f360 legacy=0x7f139c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f13880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1388003f60 0x7f1388004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388004d40 0x7f13880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c159700 con 0x7f1388003f60 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.372+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138800bb50 con 0x7f1388004d40 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138800bd50 con 0x7f139c0c88c0 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1388003f60 0x7f1388004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1d93640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f13880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a2594640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388004d40 0x7f13880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 -- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 msgr2=0x7f13880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c0c88c0 0x7f13880039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 -- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388004d40 msgr2=0x7f13880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388004d40 0x7f13880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a1592640 1 -- 172.21.15.17:0/3330707617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388003520 con 0x7f1388003f60 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.373+0000 7f13a2594640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388004d40 0x7f13880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a1592640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1388003f60 0x7f1388004800 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f138c009a90 tx=0x7f1388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d81acbbc2f9102 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c039070 con 0x7f1388003f60 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c02c680 con 0x7f1388003f60 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a0d91640 1 -- 172.21.15.17:0/3330707617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c009690 con 0x7f1388003f60 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 -- 172.21.15.17:0/3330707617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1388003f60 msgr2=0x7f1388004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/3330707617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1388003f60 0x7f1388004800 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f138c009a90 tx=0x7f1388005830 comp rx=0 tx=0).stop 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a2594640 1 -- 172.21.15.17:0/3330707617 reap_dead start 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 -- 172.21.15.17:0/3330707617 shutdown_connections 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 -- 172.21.15.17:0/3330707617 >> 172.21.15.17:0/3330707617 conn(0x7f139c0ac570 msgr2=0x7f139c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 -- 172.21.15.17:0/3330707617 shutdown_connections 2024-03-06T07:21:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.374+0000 7f13a3ff7640 1 -- 172.21.15.17:0/3330707617 wait complete. 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.375+0000 7f13a3ff7640 1 Processor -- start 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 -- start start 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 0x7f139c1ed680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c14bf90 0x7f139c0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0c7c30 con 0x7f139c0b0350 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0c5a50 con 0x7f1388003f60 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a3ff7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0c5bd0 con 0x7f139c14bf90 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a2594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a2594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37572/0 (socket says 172.21.15.17:37572) 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a2594640 1 -- 172.21.15.17:0/28931900 learned_addr learned my addr 172.21.15.17:0/28931900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a1592640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c14bf90 0x7f139c0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.376+0000 7f13a1d93640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 0x7f139c1ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 -- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c14bf90 msgr2=0x7f139c0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c14bf90 0x7f139c0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 -- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 msgr2=0x7f139c1ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 0x7f139c1ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 -- 172.21.15.17:0/28931900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c0ae3a0 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a1d93640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 0x7f139c1ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a2594640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f139c158f40 tx=0x7f139800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398006b50 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398006d10 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c0ae680 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398003970 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.377+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c0aec40 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.378+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c04f330 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.379+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f139801a020 con 0x7f139c0b0350 2024-03-06T07:21:49.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.379+0000 7f13927fc640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:49.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.379+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1398089830 con 0x7f139c0b0350 2024-03-06T07:21:49.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.379+0000 7f13a1d93640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:49.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.380+0000 7f13a1d93640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 0x7f138006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1394009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:49.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.383+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1398058520 con 0x7f139c0b0350 2024-03-06T07:21:49.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.642+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f139c0c8d50 con 0x7f138006b500 2024-03-06T07:21:49.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.649+0000 7f13927fc640 1 -- 172.21.15.17:0/28931900 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252030 (secure 0 0 0) 0x7f139c0c8d50 con 0x7f138006b500 2024-03-06T07:21:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 msgr2=0x7f138006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 0x7f138006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1394001eb0 tx=0x7f1394009040 comp rx=0 tx=0).stop 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 msgr2=0x7f139c1ec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f139c158f40 tx=0x7f139800eb90 comp rx=0 tx=0).stop 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 shutdown_connections 2024-03-06T07:21:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f138006b500 0x7f138006d9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f139c0b0350 0x7f139c1ec280 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f139c14bf90 0x7f139c0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 --2- 172.21.15.17:0/28931900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1388003f60 0x7f139c1ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 >> 172.21.15.17:0/28931900 conn(0x7f139c0ac570 msgr2=0x7f139c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 shutdown_connections 2024-03-06T07:21:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:49.652+0000 7f13a3ff7640 1 -- 172.21.15.17:0/28931900 wait complete. 2024-03-06T07:21:49.665 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-03-06T07:21:49.364116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237760,"kb_used_data":4928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736960,"statfs":{"total":773094113280,"available":772850647040,"internally_reserved":0,"allocated":5046272,"data_stored":3364772,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.467177"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182072+0000","last_change":"2024-03-06T07:21:43.913948+0000","last_active":"2024-03-06T07:21:45.182072+0000","last_peered":"2024-03-06T07:21:45.182072+0000","last_clean":"2024-03-06T07:21:45.182072+0000","last_became_active":"2024-03-06T07:21:43.911415+0000","last_became_peered":"2024-03-06T07:21:43.911415+0000","last_unstale":"2024-03-06T07:21:45.182072+0000","last_undegraded":"2024-03-06T07:21:45.182072+0000","last_fullsized":"2024-03-06T07:21:45.182072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:51.582422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922365+0000","last_change":"2024-03-06T07:21:43.914337+0000","last_active":"2024-03-06T07:21:44.922365+0000","last_peered":"2024-03-06T07:21:44.922365+0000","last_clean":"2024-03-06T07:21:44.922365+0000","last_became_active":"2024-03-06T07:21:43.914043+0000","last_became_peered":"2024-03-06T07:21:43.914043+0000","last_unstale":"2024-03-06T07:21:44.922365+0000","last_undegraded":"2024-03-06T07:21:44.922365+0000","last_fullsized":"2024-03-06T07:21:44.922365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.432948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919873+0000","last_change":"2024-03-06T07:21:43.935409+0000","last_active":"2024-03-06T07:21:44.919873+0000","last_peered":"2024-03-06T07:21:44.919873+0000","last_clean":"2024-03-06T07:21:44.919873+0000","last_became_active":"2024-03-06T07:21:43.934832+0000","last_became_peered":"2024-03-06T07:21:43.934832+0000","last_unstale":"2024-03-06T07:21:44.919873+0000","last_undegraded":"2024-03-06T07:21:44.919873+0000","last_fullsized":"2024-03-06T07:21:44.919873+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:57.737436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922160+0000","last_change":"2024-03-06T07:21:43.913875+0000","last_active":"2024-03-06T07:21:44.922160+0000","last_peered":"2024-03-06T07:21:44.922160+0000","last_clean":"2024-03-06T07:21:44.922160+0000","last_became_active":"2024-03-06T07:21:43.912314+0000","last_became_peered":"2024-03-06T07:21:43.912314+0000","last_unstale":"2024-03-06T07:21:44.922160+0000","last_undegraded":"2024-03-06T07:21:44.922160+0000","last_fullsized":"2024-03-06T07:21:44.922160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:01.595661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913814+0000","last_change":"2024-03-06T07:21:43.913216+0000","last_active":"2024-03-06T07:21:44.913814+0000","last_peered":"2024-03-06T07:21:44.913814+0000","last_clean":"2024-03-06T07:21:44.913814+0000","last_became_active":"2024-03-06T07:21:43.912935+0000","last_became_peered":"2024-03-06T07:21:43.912935+0000","last_unstale":"2024-03-06T07:21:44.913814+0000","last_undegraded":"2024-03-06T07:21:44.913814+0000","last_fullsized":"2024-03-06T07:21:44.913814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181857+0000","last_change":"2024-03-06T07:21:43.917357+0000","last_active":"2024-03-06T07:21:45.181857+0000","last_peered":"2024-03-06T07:21:45.181857+0000","last_clean":"2024-03-06T07:21:45.181857+0000","last_became_active":"2024-03-06T07:21:43.916973+0000","last_became_peered":"2024-03-06T07:21:43.916973+0000","last_unstale":"2024-03-06T07:21:45.181857+0000","last_undegraded":"2024-03-06T07:21:45.181857+0000","last_fullsized":"2024-03-06T07:21:45.181857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919492+0000","last_change":"2024-03-06T07:21:43.923785+0000","last_active":"2024-03-06T07:21:44.919492+0000","last_peered":"2024-03-06T07:21:44.919492+0000","last_clean":"2024-03-06T07:21:44.919492+0000","last_became_active":"2024-03-06T07:21:43.923460+0000","last_became_peered":"2024-03-06T07:21:43.923460+0000","last_unstale":"2024-03-06T07:21:44.919492+0000","last_undegraded":"2024-03-06T07:21:44.919492+0000","last_fullsized":"2024-03-06T07:21:44.919492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117372+0000","last_change":"2024-03-06T07:21:43.928597+0000","last_active":"2024-03-06T07:21:45.117372+0000","last_peered":"2024-03-06T07:21:45.117372+0000","last_clean":"2024-03-06T07:21:45.117372+0000","last_became_active":"2024-03-06T07:21:43.928330+0000","last_became_peered":"2024-03-06T07:21:43.928330+0000","last_unstale":"2024-03-06T07:21:45.117372+0000","last_undegraded":"2024-03-06T07:21:45.117372+0000","last_fullsized":"2024-03-06T07:21:45.117372+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117615+0000","last_change":"2024-03-06T07:21:43.938313+0000","last_active":"2024-03-06T07:21:45.117615+0000","last_peered":"2024-03-06T07:21:45.117615+0000","last_clean":"2024-03-06T07:21:45.117615+0000","last_became_active":"2024-03-06T07:21:43.938037+0000","last_became_peered":"2024-03-06T07:21:43.938037+0000","last_unstale":"2024-03-06T07:21:45.117615+0000","last_undegraded":"2024-03-06T07:21:45.117615+0000","last_fullsized":"2024-03-06T07:21:45.117615+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:12.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182055+0000","last_change":"2024-03-06T07:21:43.910022+0000","last_active":"2024-03-06T07:21:45.182055+0000","last_peered":"2024-03-06T07:21:45.182055+0000","last_clean":"2024-03-06T07:21:45.182055+0000","last_became_active":"2024-03-06T07:21:43.909753+0000","last_became_peered":"2024-03-06T07:21:43.909753+0000","last_unstale":"2024-03-06T07:21:45.182055+0000","last_undegraded":"2024-03-06T07:21:45.182055+0000","last_fullsized":"2024-03-06T07:21:45.182055+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:14.243609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.910236+0000","last_change":"2024-03-06T07:21:43.917615+0000","last_active":"2024-03-06T07:21:44.910236+0000","last_peered":"2024-03-06T07:21:44.910236+0000","last_clean":"2024-03-06T07:21:44.910236+0000","last_became_active":"2024-03-06T07:21:43.917363+0000","last_became_peered":"2024-03-06T07:21:43.917363+0000","last_unstale":"2024-03-06T07:21:44.910236+0000","last_undegraded":"2024-03-06T07:21:44.910236+0000","last_fullsized":"2024-03-06T07:21:44.910236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:29.125938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117697+0000","last_change":"2024-03-06T07:21:43.911518+0000","last_active":"2024-03-06T07:21:45.117697+0000","last_peered":"2024-03-06T07:21:45.117697+0000","last_clean":"2024-03-06T07:21:45.117697+0000","last_became_active":"2024-03-06T07:21:43.911238+0000","last_became_peered":"2024-03-06T07:21:43.911238+0000","last_unstale":"2024-03-06T07:21:45.117697+0000","last_undegraded":"2024-03-06T07:21:45.117697+0000","last_fullsized":"2024-03-06T07:21:45.117697+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:14.588826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182046+0000","last_change":"2024-03-06T07:21:43.916170+0000","last_active":"2024-03-06T07:21:45.182046+0000","last_peered":"2024-03-06T07:21:45.182046+0000","last_clean":"2024-03-06T07:21:45.182046+0000","last_became_active":"2024-03-06T07:21:43.915661+0000","last_became_peered":"2024-03-06T07:21:43.915661+0000","last_unstale":"2024-03-06T07:21:45.182046+0000","last_undegraded":"2024-03-06T07:21:45.182046+0000","last_fullsized":"2024-03-06T07:21:45.182046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919810+0000","last_change":"2024-03-06T07:21:43.925030+0000","last_active":"2024-03-06T07:21:44.919810+0000","last_peered":"2024-03-06T07:21:44.919810+0000","last_clean":"2024-03-06T07:21:44.919810+0000","last_became_active":"2024-03-06T07:21:43.924480+0000","last_became_peered":"2024-03-06T07:21:43.924480+0000","last_unstale":"2024-03-06T07:21:44.919810+0000","last_undegraded":"2024-03-06T07:21:44.919810+0000","last_fullsized":"2024-03-06T07:21:44.919810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:36.336480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922091+0000","last_change":"2024-03-06T07:21:43.915656+0000","last_active":"2024-03-06T07:21:44.922091+0000","last_peered":"2024-03-06T07:21:44.922091+0000","last_clean":"2024-03-06T07:21:44.922091+0000","last_became_active":"2024-03-06T07:21:43.915399+0000","last_became_peered":"2024-03-06T07:21:43.915399+0000","last_unstale":"2024-03-06T07:21:44.922091+0000","last_undegraded":"2024-03-06T07:21:44.922091+0000","last_fullsized":"2024-03-06T07:21:44.922091+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181753+0000","last_change":"2024-03-06T07:21:43.909296+0000","last_active":"2024-03-06T07:21:45.181753+0000","last_peered":"2024-03-06T07:21:45.181753+0000","last_clean":"2024-03-06T07:21:45.181753+0000","last_became_active":"2024-03-06T07:21:43.909011+0000","last_became_peered":"2024-03-06T07:21:43.909011+0000","last_unstale":"2024-03-06T07:21:45.181753+0000","last_undegraded":"2024-03-06T07:21:45.181753+0000","last_fullsized":"2024-03-06T07:21:45.181753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920094+0000","last_change":"2024-03-06T07:21:43.930357+0000","last_active":"2024-03-06T07:21:44.920094+0000","last_peered":"2024-03-06T07:21:44.920094+0000","last_clean":"2024-03-06T07:21:44.920094+0000","last_became_active":"2024-03-06T07:21:43.929832+0000","last_became_peered":"2024-03-06T07:21:43.929832+0000","last_unstale":"2024-03-06T07:21:44.920094+0000","last_undegraded":"2024-03-06T07:21:44.920094+0000","last_fullsized":"2024-03-06T07:21:44.920094+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922408+0000","last_change":"2024-03-06T07:21:43.919806+0000","last_active":"2024-03-06T07:21:44.922408+0000","last_peered":"2024-03-06T07:21:44.922408+0000","last_clean":"2024-03-06T07:21:44.922408+0000","last_became_active":"2024-03-06T07:21:43.919524+0000","last_became_peered":"2024-03-06T07:21:43.919524+0000","last_unstale":"2024-03-06T07:21:44.922408+0000","last_undegraded":"2024-03-06T07:21:44.922408+0000","last_fullsized":"2024-03-06T07:21:44.922408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919921+0000","last_change":"2024-03-06T07:21:43.933746+0000","last_active":"2024-03-06T07:21:44.919921+0000","last_peered":"2024-03-06T07:21:44.919921+0000","last_clean":"2024-03-06T07:21:44.919921+0000","last_became_active":"2024-03-06T07:21:43.932918+0000","last_became_peered":"2024-03-06T07:21:43.932918+0000","last_unstale":"2024-03-06T07:21:44.919921+0000","last_undegraded":"2024-03-06T07:21:44.919921+0000","last_fullsized":"2024-03-06T07:21:44.919921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913093+0000","last_change":"2024-03-06T07:21:43.916874+0000","last_active":"2024-03-06T07:21:44.913093+0000","last_peered":"2024-03-06T07:21:44.913093+0000","last_clean":"2024-03-06T07:21:44.913093+0000","last_became_active":"2024-03-06T07:21:43.916593+0000","last_became_peered":"2024-03-06T07:21:43.916593+0000","last_unstale":"2024-03-06T07:21:44.913093+0000","last_undegraded":"2024-03-06T07:21:44.913093+0000","last_fullsized":"2024-03-06T07:21:44.913093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:30.373457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913241+0000","last_change":"2024-03-06T07:21:43.914663+0000","last_active":"2024-03-06T07:21:44.913241+0000","last_peered":"2024-03-06T07:21:44.913241+0000","last_clean":"2024-03-06T07:21:44.913241+0000","last_became_active":"2024-03-06T07:21:43.914383+0000","last_became_peered":"2024-03-06T07:21:43.914383+0000","last_unstale":"2024-03-06T07:21:44.913241+0000","last_undegraded":"2024-03-06T07:21:44.913241+0000","last_fullsized":"2024-03-06T07:21:44.913241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117490+0000","last_change":"2024-03-06T07:21:43.929377+0000","last_active":"2024-03-06T07:21:45.117490+0000","last_peered":"2024-03-06T07:21:45.117490+0000","last_clean":"2024-03-06T07:21:45.117490+0000","last_became_active":"2024-03-06T07:21:43.929093+0000","last_became_peered":"2024-03-06T07:21:43.929093+0000","last_unstale":"2024-03-06T07:21:45.117490+0000","last_undegraded":"2024-03-06T07:21:45.117490+0000","last_fullsized":"2024-03-06T07:21:45.117490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.909992+0000","last_change":"2024-03-06T07:21:43.916941+0000","last_active":"2024-03-06T07:21:44.909992+0000","last_peered":"2024-03-06T07:21:44.909992+0000","last_clean":"2024-03-06T07:21:44.909992+0000","last_became_active":"2024-03-06T07:21:43.916667+0000","last_became_peered":"2024-03-06T07:21:43.916667+0000","last_unstale":"2024-03-06T07:21:44.909992+0000","last_undegraded":"2024-03-06T07:21:44.909992+0000","last_fullsized":"2024-03-06T07:21:44.909992+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:15.683227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919459+0000","last_change":"2024-03-06T07:21:43.924259+0000","last_active":"2024-03-06T07:21:44.919459+0000","last_peered":"2024-03-06T07:21:44.919459+0000","last_clean":"2024-03-06T07:21:44.919459+0000","last_became_active":"2024-03-06T07:21:43.923885+0000","last_became_peered":"2024-03-06T07:21:43.923885+0000","last_unstale":"2024-03-06T07:21:44.919459+0000","last_undegraded":"2024-03-06T07:21:44.919459+0000","last_fullsized":"2024-03-06T07:21:44.919459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920167+0000","last_change":"2024-03-06T07:21:43.931109+0000","last_active":"2024-03-06T07:21:44.920167+0000","last_peered":"2024-03-06T07:21:44.920167+0000","last_clean":"2024-03-06T07:21:44.920167+0000","last_became_active":"2024-03-06T07:21:43.930848+0000","last_became_peered":"2024-03-06T07:21:43.930848+0000","last_unstale":"2024-03-06T07:21:44.920167+0000","last_undegraded":"2024-03-06T07:21:44.920167+0000","last_fullsized":"2024-03-06T07:21:44.920167+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117542+0000","last_change":"2024-03-06T07:21:43.913493+0000","last_active":"2024-03-06T07:21:45.117542+0000","last_peered":"2024-03-06T07:21:45.117542+0000","last_clean":"2024-03-06T07:21:45.117542+0000","last_became_active":"2024-03-06T07:21:43.912992+0000","last_became_peered":"2024-03-06T07:21:43.912992+0000","last_unstale":"2024-03-06T07:21:45.117542+0000","last_undegraded":"2024-03-06T07:21:45.117542+0000","last_fullsized":"2024-03-06T07:21:45.117542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:30.239239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182122+0000","last_change":"2024-03-06T07:21:43.934992+0000","last_active":"2024-03-06T07:21:45.182122+0000","last_peered":"2024-03-06T07:21:45.182122+0000","last_clean":"2024-03-06T07:21:45.182122+0000","last_became_active":"2024-03-06T07:21:43.934664+0000","last_became_peered":"2024-03-06T07:21:43.934664+0000","last_unstale":"2024-03-06T07:21:45.182122+0000","last_undegraded":"2024-03-06T07:21:45.182122+0000","last_fullsized":"2024-03-06T07:21:45.182122+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913248+0000","last_change":"2024-03-06T07:21:43.910958+0000","last_active":"2024-03-06T07:21:44.913248+0000","last_peered":"2024-03-06T07:21:44.913248+0000","last_clean":"2024-03-06T07:21:44.913248+0000","last_became_active":"2024-03-06T07:21:43.910653+0000","last_became_peered":"2024-03-06T07:21:43.910653+0000","last_unstale":"2024-03-06T07:21:44.913248+0000","last_undegraded":"2024-03-06T07:21:44.913248+0000","last_fullsized":"2024-03-06T07:21:44.913248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919622+0000","last_change":"2024-03-06T07:21:43.923484+0000","last_active":"2024-03-06T07:21:44.919622+0000","last_peered":"2024-03-06T07:21:44.919622+0000","last_clean":"2024-03-06T07:21:44.919622+0000","last_became_active":"2024-03-06T07:21:43.921095+0000","last_became_peered":"2024-03-06T07:21:43.921095+0000","last_unstale":"2024-03-06T07:21:44.919622+0000","last_undegraded":"2024-03-06T07:21:44.919622+0000","last_fullsized":"2024-03-06T07:21:44.919622+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922195+0000","last_change":"2024-03-06T07:21:43.917834+0000","last_active":"2024-03-06T07:21:44.922195+0000","last_peered":"2024-03-06T07:21:44.922195+0000","last_clean":"2024-03-06T07:21:44.922195+0000","last_became_active":"2024-03-06T07:21:43.917576+0000","last_became_peered":"2024-03-06T07:21:43.917576+0000","last_unstale":"2024-03-06T07:21:44.922195+0000","last_undegraded":"2024-03-06T07:21:44.922195+0000","last_fullsized":"2024-03-06T07:21:44.922195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919611+0000","last_change":"2024-03-06T07:21:43.921467+0000","last_active":"2024-03-06T07:21:44.919611+0000","last_peered":"2024-03-06T07:21:44.919611+0000","last_clean":"2024-03-06T07:21:44.919611+0000","last_became_active":"2024-03-06T07:21:43.920878+0000","last_became_peered":"2024-03-06T07:21:43.920878+0000","last_unstale":"2024-03-06T07:21:44.919611+0000","last_undegraded":"2024-03-06T07:21:44.919611+0000","last_fullsized":"2024-03-06T07:21:44.919611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182498+0000","last_change":"2024-03-06T07:21:43.911317+0000","last_active":"2024-03-06T07:21:45.182498+0000","last_peered":"2024-03-06T07:21:45.182498+0000","last_clean":"2024-03-06T07:21:45.182498+0000","last_became_active":"2024-03-06T07:21:43.910980+0000","last_became_peered":"2024-03-06T07:21:43.910980+0000","last_unstale":"2024-03-06T07:21:45.182498+0000","last_undegraded":"2024-03-06T07:21:45.182498+0000","last_fullsized":"2024-03-06T07:21:45.182498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.921881+0000","last_change":"2024-03-06T07:21:43.914331+0000","last_active":"2024-03-06T07:21:44.921881+0000","last_peered":"2024-03-06T07:21:44.921881+0000","last_clean":"2024-03-06T07:21:44.921881+0000","last_became_active":"2024-03-06T07:21:43.913681+0000","last_became_peered":"2024-03-06T07:21:43.913681+0000","last_unstale":"2024-03-06T07:21:44.921881+0000","last_undegraded":"2024-03-06T07:21:44.921881+0000","last_fullsized":"2024-03-06T07:21:44.921881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913353+0000","last_change":"2024-03-06T07:21:42.908247+0000","last_active":"2024-03-06T07:21:44.913353+0000","last_peered":"2024-03-06T07:21:44.913353+0000","last_clean":"2024-03-06T07:21:44.913353+0000","last_became_active":"2024-03-06T07:21:42.907863+0000","last_became_peered":"2024-03-06T07:21:42.907863+0000","last_unstale":"2024-03-06T07:21:44.913353+0000","last_undegraded":"2024-03-06T07:21:44.913353+0000","last_fullsized":"2024-03-06T07:21:44.913353+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:44.909708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182455+0000","last_change":"2024-03-06T07:21:43.918214+0000","last_active":"2024-03-06T07:21:45.182455+0000","last_peered":"2024-03-06T07:21:45.182455+0000","last_clean":"2024-03-06T07:21:45.182455+0000","last_became_active":"2024-03-06T07:21:43.917454+0000","last_became_peered":"2024-03-06T07:21:43.917454+0000","last_unstale":"2024-03-06T07:21:45.182455+0000","last_undegraded":"2024-03-06T07:21:45.182455+0000","last_fullsized":"2024-03-06T07:21:45.182455+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181920+0000","last_change":"2024-03-06T07:21:42.915335+0000","last_active":"2024-03-06T07:21:45.181920+0000","last_peered":"2024-03-06T07:21:45.181920+0000","last_clean":"2024-03-06T07:21:45.181920+0000","last_became_active":"2024-03-06T07:21:42.915011+0000","last_became_peered":"2024-03-06T07:21:42.915011+0000","last_unstale":"2024-03-06T07:21:45.181920+0000","last_undegraded":"2024-03-06T07:21:45.181920+0000","last_fullsized":"2024-03-06T07:21:45.181920+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912924+0000","last_change":"2024-03-06T07:21:43.914522+0000","last_active":"2024-03-06T07:21:44.912924+0000","last_peered":"2024-03-06T07:21:44.912924+0000","last_clean":"2024-03-06T07:21:44.912924+0000","last_became_active":"2024-03-06T07:21:43.913602+0000","last_became_peered":"2024-03-06T07:21:43.913602+0000","last_unstale":"2024-03-06T07:21:44.912924+0000","last_undegraded":"2024-03-06T07:21:44.912924+0000","last_fullsized":"2024-03-06T07:21:44.912924+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:02.123078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182059+0000","last_change":"2024-03-06T07:21:42.914470+0000","last_active":"2024-03-06T07:21:45.182059+0000","last_peered":"2024-03-06T07:21:45.182059+0000","last_clean":"2024-03-06T07:21:45.182059+0000","last_became_active":"2024-03-06T07:21:42.913930+0000","last_became_peered":"2024-03-06T07:21:42.913930+0000","last_unstale":"2024-03-06T07:21:45.182059+0000","last_undegraded":"2024-03-06T07:21:45.182059+0000","last_fullsized":"2024-03-06T07:21:45.182059+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:24.367917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117391+0000","last_change":"2024-03-06T07:21:43.912159+0000","last_active":"2024-03-06T07:21:45.117391+0000","last_peered":"2024-03-06T07:21:45.117391+0000","last_clean":"2024-03-06T07:21:45.117391+0000","last_became_active":"2024-03-06T07:21:43.911851+0000","last_became_peered":"2024-03-06T07:21:43.911851+0000","last_unstale":"2024-03-06T07:21:45.117391+0000","last_undegraded":"2024-03-06T07:21:45.117391+0000","last_fullsized":"2024-03-06T07:21:45.117391+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182377+0000","last_change":"2024-03-06T07:21:42.912811+0000","last_active":"2024-03-06T07:21:45.182377+0000","last_peered":"2024-03-06T07:21:45.182377+0000","last_clean":"2024-03-06T07:21:45.182377+0000","last_became_active":"2024-03-06T07:21:42.912544+0000","last_became_peered":"2024-03-06T07:21:42.912544+0000","last_unstale":"2024-03-06T07:21:45.182377+0000","last_undegraded":"2024-03-06T07:21:45.182377+0000","last_fullsized":"2024-03-06T07:21:45.182377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:50.577390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913967+0000","last_change":"2024-03-06T07:21:43.922679+0000","last_active":"2024-03-06T07:21:44.913967+0000","last_peered":"2024-03-06T07:21:44.913967+0000","last_clean":"2024-03-06T07:21:44.913967+0000","last_became_active":"2024-03-06T07:21:43.922322+0000","last_became_peered":"2024-03-06T07:21:43.922322+0000","last_unstale":"2024-03-06T07:21:44.913967+0000","last_undegraded":"2024-03-06T07:21:44.913967+0000","last_fullsized":"2024-03-06T07:21:44.913967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919558+0000","last_change":"2024-03-06T07:21:42.912329+0000","last_active":"2024-03-06T07:21:44.919558+0000","last_peered":"2024-03-06T07:21:44.919558+0000","last_clean":"2024-03-06T07:21:44.919558+0000","last_became_active":"2024-03-06T07:21:42.911690+0000","last_became_peered":"2024-03-06T07:21:42.911690+0000","last_unstale":"2024-03-06T07:21:44.919558+0000","last_undegraded":"2024-03-06T07:21:44.919558+0000","last_fullsized":"2024-03-06T07:21:44.919558+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922131+0000","last_change":"2024-03-06T07:21:43.917686+0000","last_active":"2024-03-06T07:21:44.922131+0000","last_peered":"2024-03-06T07:21:44.922131+0000","last_clean":"2024-03-06T07:21:44.922131+0000","last_became_active":"2024-03-06T07:21:43.917369+0000","last_became_peered":"2024-03-06T07:21:43.917369+0000","last_unstale":"2024-03-06T07:21:44.922131+0000","last_undegraded":"2024-03-06T07:21:44.922131+0000","last_fullsized":"2024-03-06T07:21:44.922131+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.013761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913597+0000","last_change":"2024-03-06T07:21:42.910602+0000","last_active":"2024-03-06T07:21:44.913597+0000","last_peered":"2024-03-06T07:21:44.913597+0000","last_clean":"2024-03-06T07:21:44.913597+0000","last_became_active":"2024-03-06T07:21:42.910279+0000","last_became_peered":"2024-03-06T07:21:42.910279+0000","last_unstale":"2024-03-06T07:21:44.913597+0000","last_undegraded":"2024-03-06T07:21:44.913597+0000","last_fullsized":"2024-03-06T07:21:44.913597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:02.037327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117696+0000","last_change":"2024-03-06T07:21:43.913254+0000","last_active":"2024-03-06T07:21:45.117696+0000","last_peered":"2024-03-06T07:21:45.117696+0000","last_clean":"2024-03-06T07:21:45.117696+0000","last_became_active":"2024-03-06T07:21:43.912620+0000","last_became_peered":"2024-03-06T07:21:43.912620+0000","last_unstale":"2024-03-06T07:21:45.117696+0000","last_undegraded":"2024-03-06T07:21:45.117696+0000","last_fullsized":"2024-03-06T07:21:45.117696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:13.059235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912746+0000","last_change":"2024-03-06T07:21:42.909672+0000","last_active":"2024-03-06T07:21:44.912746+0000","last_peered":"2024-03-06T07:21:44.912746+0000","last_clean":"2024-03-06T07:21:44.912746+0000","last_became_active":"2024-03-06T07:21:42.909402+0000","last_became_peered":"2024-03-06T07:21:42.909402+0000","last_unstale":"2024-03-06T07:21:44.912746+0000","last_undegraded":"2024-03-06T07:21:44.912746+0000","last_fullsized":"2024-03-06T07:21:44.912746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:06.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922430+0000","last_change":"2024-03-06T07:21:43.934375+0000","last_active":"2024-03-06T07:21:44.922430+0000","last_peered":"2024-03-06T07:21:44.922430+0000","last_clean":"2024-03-06T07:21:44.922430+0000","last_became_active":"2024-03-06T07:21:43.934076+0000","last_became_peered":"2024-03-06T07:21:43.934076+0000","last_unstale":"2024-03-06T07:21:44.922430+0000","last_undegraded":"2024-03-06T07:21:44.922430+0000","last_fullsized":"2024-03-06T07:21:44.922430+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919390+0000","last_change":"2024-03-06T07:21:42.910541+0000","last_active":"2024-03-06T07:21:44.919390+0000","last_peered":"2024-03-06T07:21:44.919390+0000","last_clean":"2024-03-06T07:21:44.919390+0000","last_became_active":"2024-03-06T07:21:42.910271+0000","last_became_peered":"2024-03-06T07:21:42.910271+0000","last_unstale":"2024-03-06T07:21:44.919390+0000","last_undegraded":"2024-03-06T07:21:44.919390+0000","last_fullsized":"2024-03-06T07:21:44.919390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913081+0000","last_change":"2024-03-06T07:21:43.905571+0000","last_active":"2024-03-06T07:21:44.913081+0000","last_peered":"2024-03-06T07:21:44.913081+0000","last_clean":"2024-03-06T07:21:44.913081+0000","last_became_active":"2024-03-06T07:21:43.904452+0000","last_became_peered":"2024-03-06T07:21:43.904452+0000","last_unstale":"2024-03-06T07:21:44.913081+0000","last_undegraded":"2024-03-06T07:21:44.913081+0000","last_fullsized":"2024-03-06T07:21:44.913081+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922346+0000","last_change":"2024-03-06T07:21:42.913817+0000","last_active":"2024-03-06T07:21:44.922346+0000","last_peered":"2024-03-06T07:21:44.922346+0000","last_clean":"2024-03-06T07:21:44.922346+0000","last_became_active":"2024-03-06T07:21:42.913464+0000","last_became_peered":"2024-03-06T07:21:42.913464+0000","last_unstale":"2024-03-06T07:21:44.922346+0000","last_undegraded":"2024-03-06T07:21:44.922346+0000","last_fullsized":"2024-03-06T07:21:44.922346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182307+0000","last_change":"2024-03-06T07:21:43.912031+0000","last_active":"2024-03-06T07:21:45.182307+0000","last_peered":"2024-03-06T07:21:45.182307+0000","last_clean":"2024-03-06T07:21:45.182307+0000","last_became_active":"2024-03-06T07:21:43.911759+0000","last_became_peered":"2024-03-06T07:21:43.911759+0000","last_unstale":"2024-03-06T07:21:45.182307+0000","last_undegraded":"2024-03-06T07:21:45.182307+0000","last_fullsized":"2024-03-06T07:21:45.182307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913289+0000","last_change":"2024-03-06T07:21:42.910374+0000","last_active":"2024-03-06T07:21:44.913289+0000","last_peered":"2024-03-06T07:21:44.913289+0000","last_clean":"2024-03-06T07:21:44.913289+0000","last_became_active":"2024-03-06T07:21:42.910082+0000","last_became_peered":"2024-03-06T07:21:42.910082+0000","last_unstale":"2024-03-06T07:21:44.913289+0000","last_undegraded":"2024-03-06T07:21:44.913289+0000","last_fullsized":"2024-03-06T07:21:44.913289+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:55.547305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919688+0000","last_change":"2024-03-06T07:21:43.924815+0000","last_active":"2024-03-06T07:21:44.919688+0000","last_peered":"2024-03-06T07:21:44.919688+0000","last_clean":"2024-03-06T07:21:44.919688+0000","last_became_active":"2024-03-06T07:21:43.924188+0000","last_became_peered":"2024-03-06T07:21:43.924188+0000","last_unstale":"2024-03-06T07:21:44.919688+0000","last_undegraded":"2024-03-06T07:21:44.919688+0000","last_fullsized":"2024-03-06T07:21:44.919688+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:57.007684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922220+0000","last_change":"2024-03-06T07:21:42.912311+0000","last_active":"2024-03-06T07:21:44.922220+0000","last_peered":"2024-03-06T07:21:44.922220+0000","last_clean":"2024-03-06T07:21:44.922220+0000","last_became_active":"2024-03-06T07:21:42.912046+0000","last_became_peered":"2024-03-06T07:21:42.912046+0000","last_unstale":"2024-03-06T07:21:44.922220+0000","last_undegraded":"2024-03-06T07:21:44.922220+0000","last_fullsized":"2024-03-06T07:21:44.922220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913937+0000","last_change":"2024-03-06T07:21:43.911762+0000","last_active":"2024-03-06T07:21:44.913937+0000","last_peered":"2024-03-06T07:21:44.913937+0000","last_clean":"2024-03-06T07:21:44.913937+0000","last_became_active":"2024-03-06T07:21:43.911470+0000","last_became_peered":"2024-03-06T07:21:43.911470+0000","last_unstale":"2024-03-06T07:21:44.913937+0000","last_undegraded":"2024-03-06T07:21:44.913937+0000","last_fullsized":"2024-03-06T07:21:44.913937+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117652+0000","last_change":"2024-03-06T07:21:42.912419+0000","last_active":"2024-03-06T07:21:45.117652+0000","last_peered":"2024-03-06T07:21:45.117652+0000","last_clean":"2024-03-06T07:21:45.117652+0000","last_became_active":"2024-03-06T07:21:42.912141+0000","last_became_peered":"2024-03-06T07:21:42.912141+0000","last_unstale":"2024-03-06T07:21:45.117652+0000","last_undegraded":"2024-03-06T07:21:45.117652+0000","last_fullsized":"2024-03-06T07:21:45.117652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:00.590629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182495+0000","last_change":"2024-03-06T07:21:43.918063+0000","last_active":"2024-03-06T07:21:45.182495+0000","last_peered":"2024-03-06T07:21:45.182495+0000","last_clean":"2024-03-06T07:21:45.182495+0000","last_became_active":"2024-03-06T07:21:43.917325+0000","last_became_peered":"2024-03-06T07:21:43.917325+0000","last_unstale":"2024-03-06T07:21:45.182495+0000","last_undegraded":"2024-03-06T07:21:45.182495+0000","last_fullsized":"2024-03-06T07:21:45.182495+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919672+0000","last_change":"2024-03-06T07:21:42.912594+0000","last_active":"2024-03-06T07:21:44.919672+0000","last_peered":"2024-03-06T07:21:44.919672+0000","last_clean":"2024-03-06T07:21:44.919672+0000","last_became_active":"2024-03-06T07:21:42.912053+0000","last_became_peered":"2024-03-06T07:21:42.912053+0000","last_unstale":"2024-03-06T07:21:44.919672+0000","last_undegraded":"2024-03-06T07:21:44.919672+0000","last_fullsized":"2024-03-06T07:21:44.919672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:42.711847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919982+0000","last_change":"2024-03-06T07:21:43.931801+0000","last_active":"2024-03-06T07:21:44.919982+0000","last_peered":"2024-03-06T07:21:44.919982+0000","last_clean":"2024-03-06T07:21:44.919982+0000","last_became_active":"2024-03-06T07:21:43.931515+0000","last_became_peered":"2024-03-06T07:21:43.931515+0000","last_unstale":"2024-03-06T07:21:44.919982+0000","last_undegraded":"2024-03-06T07:21:44.919982+0000","last_fullsized":"2024-03-06T07:21:44.919982+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912938+0000","last_change":"2024-03-06T07:21:42.910643+0000","last_active":"2024-03-06T07:21:44.912938+0000","last_peered":"2024-03-06T07:21:44.912938+0000","last_clean":"2024-03-06T07:21:44.912938+0000","last_became_active":"2024-03-06T07:21:42.910377+0000","last_became_peered":"2024-03-06T07:21:42.910377+0000","last_unstale":"2024-03-06T07:21:44.912938+0000","last_undegraded":"2024-03-06T07:21:44.912938+0000","last_fullsized":"2024-03-06T07:21:44.912938+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117660+0000","last_change":"2024-03-06T07:21:43.915960+0000","last_active":"2024-03-06T07:21:45.117660+0000","last_peered":"2024-03-06T07:21:45.117660+0000","last_clean":"2024-03-06T07:21:45.117660+0000","last_became_active":"2024-03-06T07:21:43.915665+0000","last_became_peered":"2024-03-06T07:21:43.915665+0000","last_unstale":"2024-03-06T07:21:45.117660+0000","last_undegraded":"2024-03-06T07:21:45.117660+0000","last_fullsized":"2024-03-06T07:21:45.117660+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:43.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922197+0000","last_change":"2024-03-06T07:21:42.911389+0000","last_active":"2024-03-06T07:21:44.922197+0000","last_peered":"2024-03-06T07:21:44.922197+0000","last_clean":"2024-03-06T07:21:44.922197+0000","last_became_active":"2024-03-06T07:21:42.911070+0000","last_became_peered":"2024-03-06T07:21:42.911070+0000","last_unstale":"2024-03-06T07:21:44.922197+0000","last_undegraded":"2024-03-06T07:21:44.922197+0000","last_fullsized":"2024-03-06T07:21:44.922197+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920157+0000","last_change":"2024-03-06T07:21:43.930573+0000","last_active":"2024-03-06T07:21:44.920157+0000","last_peered":"2024-03-06T07:21:44.920157+0000","last_clean":"2024-03-06T07:21:44.920157+0000","last_became_active":"2024-03-06T07:21:43.930105+0000","last_became_peered":"2024-03-06T07:21:43.930105+0000","last_unstale":"2024-03-06T07:21:44.920157+0000","last_undegraded":"2024-03-06T07:21:44.920157+0000","last_fullsized":"2024-03-06T07:21:44.920157+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117431+0000","last_change":"2024-03-06T07:21:42.915873+0000","last_active":"2024-03-06T07:21:45.117431+0000","last_peered":"2024-03-06T07:21:45.117431+0000","last_clean":"2024-03-06T07:21:45.117431+0000","last_became_active":"2024-03-06T07:21:42.915575+0000","last_became_peered":"2024-03-06T07:21:42.915575+0000","last_unstale":"2024-03-06T07:21:45.117431+0000","last_undegraded":"2024-03-06T07:21:45.117431+0000","last_fullsized":"2024-03-06T07:21:45.117431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:30.008729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117436+0000","last_change":"2024-03-06T07:21:43.914468+0000","last_active":"2024-03-06T07:21:45.117436+0000","last_peered":"2024-03-06T07:21:45.117436+0000","last_clean":"2024-03-06T07:21:45.117436+0000","last_became_active":"2024-03-06T07:21:43.914210+0000","last_became_peered":"2024-03-06T07:21:43.914210+0000","last_unstale":"2024-03-06T07:21:45.117436+0000","last_undegraded":"2024-03-06T07:21:45.117436+0000","last_fullsized":"2024-03-06T07:21:45.117436+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920205+0000","last_change":"2024-03-06T07:21:42.914001+0000","last_active":"2024-03-06T07:21:44.920205+0000","last_peered":"2024-03-06T07:21:44.920205+0000","last_clean":"2024-03-06T07:21:44.920205+0000","last_became_active":"2024-03-06T07:21:42.913707+0000","last_became_peered":"2024-03-06T07:21:42.913707+0000","last_unstale":"2024-03-06T07:21:44.920205+0000","last_undegraded":"2024-03-06T07:21:44.920205+0000","last_fullsized":"2024-03-06T07:21:44.920205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:37.811398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181747+0000","last_change":"2024-03-06T07:21:43.928109+0000","last_active":"2024-03-06T07:21:45.181747+0000","last_peered":"2024-03-06T07:21:45.181747+0000","last_clean":"2024-03-06T07:21:45.181747+0000","last_became_active":"2024-03-06T07:21:43.927816+0000","last_became_peered":"2024-03-06T07:21:43.927816+0000","last_unstale":"2024-03-06T07:21:45.181747+0000","last_undegraded":"2024-03-06T07:21:45.181747+0000","last_fullsized":"2024-03-06T07:21:45.181747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.910277+0000","last_change":"2024-03-06T07:21:42.909376+0000","last_active":"2024-03-06T07:21:44.910277+0000","last_peered":"2024-03-06T07:21:44.910277+0000","last_clean":"2024-03-06T07:21:44.910277+0000","last_became_active":"2024-03-06T07:21:42.909101+0000","last_became_peered":"2024-03-06T07:21:42.909101+0000","last_unstale":"2024-03-06T07:21:44.910277+0000","last_undegraded":"2024-03-06T07:21:44.910277+0000","last_fullsized":"2024-03-06T07:21:44.910277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:37.646398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913535+0000","last_change":"2024-03-06T07:21:43.925791+0000","last_active":"2024-03-06T07:21:44.913535+0000","last_peered":"2024-03-06T07:21:44.913535+0000","last_clean":"2024-03-06T07:21:44.913535+0000","last_became_active":"2024-03-06T07:21:43.925489+0000","last_became_peered":"2024-03-06T07:21:43.925489+0000","last_unstale":"2024-03-06T07:21:44.913535+0000","last_undegraded":"2024-03-06T07:21:44.913535+0000","last_fullsized":"2024-03-06T07:21:44.913535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913318+0000","last_change":"2024-03-06T07:21:42.912376+0000","last_active":"2024-03-06T07:21:44.913318+0000","last_peered":"2024-03-06T07:21:44.913318+0000","last_clean":"2024-03-06T07:21:44.913318+0000","last_became_active":"2024-03-06T07:21:42.912102+0000","last_became_peered":"2024-03-06T07:21:42.912102+0000","last_unstale":"2024-03-06T07:21:44.913318+0000","last_undegraded":"2024-03-06T07:21:44.913318+0000","last_fullsized":"2024-03-06T07:21:44.913318+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181871+0000","last_change":"2024-03-06T07:21:43.914898+0000","last_active":"2024-03-06T07:21:45.181871+0000","last_peered":"2024-03-06T07:21:45.181871+0000","last_clean":"2024-03-06T07:21:45.181871+0000","last_became_active":"2024-03-06T07:21:43.914368+0000","last_became_peered":"2024-03-06T07:21:43.914368+0000","last_unstale":"2024-03-06T07:21:45.181871+0000","last_undegraded":"2024-03-06T07:21:45.181871+0000","last_fullsized":"2024-03-06T07:21:45.181871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:32:44.975253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913252+0000","last_change":"2024-03-06T07:21:42.913064+0000","last_active":"2024-03-06T07:21:44.913252+0000","last_peered":"2024-03-06T07:21:44.913252+0000","last_clean":"2024-03-06T07:21:44.913252+0000","last_became_active":"2024-03-06T07:21:42.912792+0000","last_became_peered":"2024-03-06T07:21:42.912792+0000","last_unstale":"2024-03-06T07:21:44.913252+0000","last_undegraded":"2024-03-06T07:21:44.913252+0000","last_fullsized":"2024-03-06T07:21:44.913252+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:13.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182299+0000","last_change":"2024-03-06T07:21:43.912688+0000","last_active":"2024-03-06T07:21:45.182299+0000","last_peered":"2024-03-06T07:21:45.182299+0000","last_clean":"2024-03-06T07:21:45.182299+0000","last_became_active":"2024-03-06T07:21:43.912441+0000","last_became_peered":"2024-03-06T07:21:43.912441+0000","last_unstale":"2024-03-06T07:21:45.182299+0000","last_undegraded":"2024-03-06T07:21:45.182299+0000","last_fullsized":"2024-03-06T07:21:45.182299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181925+0000","last_change":"2024-03-06T07:21:42.911252+0000","last_active":"2024-03-06T07:21:45.181925+0000","last_peered":"2024-03-06T07:21:45.181925+0000","last_clean":"2024-03-06T07:21:45.181925+0000","last_became_active":"2024-03-06T07:21:42.910758+0000","last_became_peered":"2024-03-06T07:21:42.910758+0000","last_unstale":"2024-03-06T07:21:45.181925+0000","last_undegraded":"2024-03-06T07:21:45.181925+0000","last_fullsized":"2024-03-06T07:21:45.181925+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913333+0000","last_change":"2024-03-06T07:21:43.917113+0000","last_active":"2024-03-06T07:21:44.913333+0000","last_peered":"2024-03-06T07:21:44.913333+0000","last_clean":"2024-03-06T07:21:44.913333+0000","last_became_active":"2024-03-06T07:21:43.915472+0000","last_became_peered":"2024-03-06T07:21:43.915472+0000","last_unstale":"2024-03-06T07:21:44.913333+0000","last_undegraded":"2024-03-06T07:21:44.913333+0000","last_fullsized":"2024-03-06T07:21:44.913333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117220+0000","last_change":"2024-03-06T07:21:42.916636+0000","last_active":"2024-03-06T07:21:45.117220+0000","last_peered":"2024-03-06T07:21:45.117220+0000","last_clean":"2024-03-06T07:21:45.117220+0000","last_became_active":"2024-03-06T07:21:42.916345+0000","last_became_peered":"2024-03-06T07:21:42.916345+0000","last_unstale":"2024-03-06T07:21:45.117220+0000","last_undegraded":"2024-03-06T07:21:45.117220+0000","last_fullsized":"2024-03-06T07:21:45.117220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:59.475185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919901+0000","last_change":"2024-03-06T07:21:43.921341+0000","last_active":"2024-03-06T07:21:44.919901+0000","last_peered":"2024-03-06T07:21:44.919901+0000","last_clean":"2024-03-06T07:21:44.919901+0000","last_became_active":"2024-03-06T07:21:43.921062+0000","last_became_peered":"2024-03-06T07:21:43.921062+0000","last_unstale":"2024-03-06T07:21:44.919901+0000","last_undegraded":"2024-03-06T07:21:44.919901+0000","last_fullsized":"2024-03-06T07:21:44.919901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:15.340198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919549+0000","last_change":"2024-03-06T07:21:42.912294+0000","last_active":"2024-03-06T07:21:44.919549+0000","last_peered":"2024-03-06T07:21:44.919549+0000","last_clean":"2024-03-06T07:21:44.919549+0000","last_became_active":"2024-03-06T07:21:42.912023+0000","last_became_peered":"2024-03-06T07:21:42.912023+0000","last_unstale":"2024-03-06T07:21:44.919549+0000","last_undegraded":"2024-03-06T07:21:44.919549+0000","last_fullsized":"2024-03-06T07:21:44.919549+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182129+0000","last_change":"2024-03-06T07:21:43.905564+0000","last_active":"2024-03-06T07:21:45.182129+0000","last_peered":"2024-03-06T07:21:45.182129+0000","last_clean":"2024-03-06T07:21:45.182129+0000","last_became_active":"2024-03-06T07:21:43.905280+0000","last_became_peered":"2024-03-06T07:21:43.905280+0000","last_unstale":"2024-03-06T07:21:45.182129+0000","last_undegraded":"2024-03-06T07:21:45.182129+0000","last_fullsized":"2024-03-06T07:21:45.182129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.909833+0000","last_change":"2024-03-06T07:21:42.908701+0000","last_active":"2024-03-06T07:21:44.909833+0000","last_peered":"2024-03-06T07:21:44.909833+0000","last_clean":"2024-03-06T07:21:44.909833+0000","last_became_active":"2024-03-06T07:21:42.908429+0000","last_became_peered":"2024-03-06T07:21:42.908429+0000","last_unstale":"2024-03-06T07:21:44.909833+0000","last_undegraded":"2024-03-06T07:21:44.909833+0000","last_fullsized":"2024-03-06T07:21:44.909833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:55.863684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182397+0000","last_change":"2024-03-06T07:21:43.909104+0000","last_active":"2024-03-06T07:21:45.182397+0000","last_peered":"2024-03-06T07:21:45.182397+0000","last_clean":"2024-03-06T07:21:45.182397+0000","last_became_active":"2024-03-06T07:21:43.908810+0000","last_became_peered":"2024-03-06T07:21:43.908810+0000","last_unstale":"2024-03-06T07:21:45.182397+0000","last_undegraded":"2024-03-06T07:21:45.182397+0000","last_fullsized":"2024-03-06T07:21:45.182397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:14.273650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913462+0000","last_change":"2024-03-06T07:21:42.914080+0000","last_active":"2024-03-06T07:21:44.913462+0000","last_peered":"2024-03-06T07:21:44.913462+0000","last_clean":"2024-03-06T07:21:44.913462+0000","last_became_active":"2024-03-06T07:21:42.913818+0000","last_became_peered":"2024-03-06T07:21:42.913818+0000","last_unstale":"2024-03-06T07:21:44.913462+0000","last_undegraded":"2024-03-06T07:21:44.913462+0000","last_fullsized":"2024-03-06T07:21:44.913462+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:28.725020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182047+0000","last_change":"2024-03-06T07:21:43.917579+0000","last_active":"2024-03-06T07:21:45.182047+0000","last_peered":"2024-03-06T07:21:45.182047+0000","last_clean":"2024-03-06T07:21:45.182047+0000","last_became_active":"2024-03-06T07:21:43.916663+0000","last_became_peered":"2024-03-06T07:21:43.916663+0000","last_unstale":"2024-03-06T07:21:45.182047+0000","last_undegraded":"2024-03-06T07:21:45.182047+0000","last_fullsized":"2024-03-06T07:21:45.182047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:43.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922291+0000","last_change":"2024-03-06T07:21:42.912139+0000","last_active":"2024-03-06T07:21:44.922291+0000","last_peered":"2024-03-06T07:21:44.922291+0000","last_clean":"2024-03-06T07:21:44.922291+0000","last_became_active":"2024-03-06T07:21:42.911573+0000","last_became_peered":"2024-03-06T07:21:42.911573+0000","last_unstale":"2024-03-06T07:21:44.922291+0000","last_undegraded":"2024-03-06T07:21:44.922291+0000","last_fullsized":"2024-03-06T07:21:44.922291+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117500+0000","last_change":"2024-03-06T07:21:43.912734+0000","last_active":"2024-03-06T07:21:45.117500+0000","last_peered":"2024-03-06T07:21:45.117500+0000","last_clean":"2024-03-06T07:21:45.117500+0000","last_became_active":"2024-03-06T07:21:43.912449+0000","last_became_peered":"2024-03-06T07:21:43.912449+0000","last_unstale":"2024-03-06T07:21:45.117500+0000","last_undegraded":"2024-03-06T07:21:45.117500+0000","last_fullsized":"2024-03-06T07:21:45.117500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919771+0000","last_change":"2024-03-06T07:21:42.907567+0000","last_active":"2024-03-06T07:21:44.919771+0000","last_peered":"2024-03-06T07:21:44.919771+0000","last_clean":"2024-03-06T07:21:44.919771+0000","last_became_active":"2024-03-06T07:21:42.907271+0000","last_became_peered":"2024-03-06T07:21:42.907271+0000","last_unstale":"2024-03-06T07:21:44.919771+0000","last_undegraded":"2024-03-06T07:21:44.919771+0000","last_fullsized":"2024-03-06T07:21:44.919771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919838+0000","last_change":"2024-03-06T07:21:43.921555+0000","last_active":"2024-03-06T07:21:44.919838+0000","last_peered":"2024-03-06T07:21:44.919838+0000","last_clean":"2024-03-06T07:21:44.919838+0000","last_became_active":"2024-03-06T07:21:43.921242+0000","last_became_peered":"2024-03-06T07:21:43.921242+0000","last_unstale":"2024-03-06T07:21:44.919838+0000","last_undegraded":"2024-03-06T07:21:44.919838+0000","last_fullsized":"2024-03-06T07:21:44.919838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:31.942191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117448+0000","last_change":"2024-03-06T07:21:42.915454+0000","last_active":"2024-03-06T07:21:45.117448+0000","last_peered":"2024-03-06T07:21:45.117448+0000","last_clean":"2024-03-06T07:21:45.117448+0000","last_became_active":"2024-03-06T07:21:42.915127+0000","last_became_peered":"2024-03-06T07:21:42.915127+0000","last_unstale":"2024-03-06T07:21:45.117448+0000","last_undegraded":"2024-03-06T07:21:45.117448+0000","last_fullsized":"2024-03-06T07:21:45.117448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:14.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182495+0000","last_change":"2024-03-06T07:21:43.919585+0000","last_active":"2024-03-06T07:21:45.182495+0000","last_peered":"2024-03-06T07:21:45.182495+0000","last_clean":"2024-03-06T07:21:45.182495+0000","last_became_active":"2024-03-06T07:21:43.919255+0000","last_became_peered":"2024-03-06T07:21:43.919255+0000","last_unstale":"2024-03-06T07:21:45.182495+0000","last_undegraded":"2024-03-06T07:21:45.182495+0000","last_fullsized":"2024-03-06T07:21:45.182495+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117317+0000","last_change":"2024-03-06T07:21:42.912512+0000","last_active":"2024-03-06T07:21:45.117317+0000","last_peered":"2024-03-06T07:21:45.117317+0000","last_clean":"2024-03-06T07:21:45.117317+0000","last_became_active":"2024-03-06T07:21:42.912235+0000","last_became_peered":"2024-03-06T07:21:42.912235+0000","last_unstale":"2024-03-06T07:21:45.117317+0000","last_undegraded":"2024-03-06T07:21:45.117317+0000","last_fullsized":"2024-03-06T07:21:45.117317+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.427916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919846+0000","last_change":"2024-03-06T07:21:43.933046+0000","last_active":"2024-03-06T07:21:44.919846+0000","last_peered":"2024-03-06T07:21:44.919846+0000","last_clean":"2024-03-06T07:21:44.919846+0000","last_became_active":"2024-03-06T07:21:43.932764+0000","last_became_peered":"2024-03-06T07:21:43.932764+0000","last_unstale":"2024-03-06T07:21:44.919846+0000","last_undegraded":"2024-03-06T07:21:44.919846+0000","last_fullsized":"2024-03-06T07:21:44.919846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:47.292953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913411+0000","last_change":"2024-03-06T07:21:42.912274+0000","last_active":"2024-03-06T07:21:44.913411+0000","last_peered":"2024-03-06T07:21:44.913411+0000","last_clean":"2024-03-06T07:21:44.913411+0000","last_became_active":"2024-03-06T07:21:42.912000+0000","last_became_peered":"2024-03-06T07:21:42.912000+0000","last_unstale":"2024-03-06T07:21:44.913411+0000","last_undegraded":"2024-03-06T07:21:44.913411+0000","last_fullsized":"2024-03-06T07:21:44.913411+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:00.006841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117661+0000","last_change":"2024-03-06T07:21:43.906537+0000","last_active":"2024-03-06T07:21:45.117661+0000","last_peered":"2024-03-06T07:21:45.117661+0000","last_clean":"2024-03-06T07:21:45.117661+0000","last_became_active":"2024-03-06T07:21:43.906267+0000","last_became_peered":"2024-03-06T07:21:43.906267+0000","last_unstale":"2024-03-06T07:21:45.117661+0000","last_undegraded":"2024-03-06T07:21:45.117661+0000","last_fullsized":"2024-03-06T07:21:45.117661+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:45.221948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182491+0000","last_change":"2024-03-06T07:21:42.912363+0000","last_active":"2024-03-06T07:21:45.182491+0000","last_peered":"2024-03-06T07:21:45.182491+0000","last_clean":"2024-03-06T07:21:45.182491+0000","last_became_active":"2024-03-06T07:21:42.912097+0000","last_became_peered":"2024-03-06T07:21:42.912097+0000","last_unstale":"2024-03-06T07:21:45.182491+0000","last_undegraded":"2024-03-06T07:21:45.182491+0000","last_fullsized":"2024-03-06T07:21:45.182491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:13.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912857+0000","last_change":"2024-03-06T07:21:43.913294+0000","last_active":"2024-03-06T07:21:44.912857+0000","last_peered":"2024-03-06T07:21:44.912857+0000","last_clean":"2024-03-06T07:21:44.912857+0000","last_became_active":"2024-03-06T07:21:43.913017+0000","last_became_peered":"2024-03-06T07:21:43.913017+0000","last_unstale":"2024-03-06T07:21:44.912857+0000","last_undegraded":"2024-03-06T07:21:44.912857+0000","last_fullsized":"2024-03-06T07:21:44.912857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:26.287336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919943+0000","last_change":"2024-03-06T07:21:42.912585+0000","last_active":"2024-03-06T07:21:44.919943+0000","last_peered":"2024-03-06T07:21:44.919943+0000","last_clean":"2024-03-06T07:21:44.919943+0000","last_became_active":"2024-03-06T07:21:42.911650+0000","last_became_peered":"2024-03-06T07:21:42.911650+0000","last_unstale":"2024-03-06T07:21:44.919943+0000","last_undegraded":"2024-03-06T07:21:44.919943+0000","last_fullsized":"2024-03-06T07:21:44.919943+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:42.887622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.921806+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:44.921806+0000","last_peered":"2024-03-06T07:21:44.921806+0000","last_clean":"2024-03-06T07:21:44.921806+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:44.921806+0000","last_undegraded":"2024-03-06T07:21:44.921806+0000","last_fullsized":"2024-03-06T07:21:44.921806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":862894,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":272402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":272984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":274036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":274014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":272562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":679936,"data_stored":862826,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:52.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:21:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.769+0000 7fb92ecf4640 1 Processor -- start 2024-03-06T07:21:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.769+0000 7fb92ecf4640 1 -- start start 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb92ecf4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb92814f6b0 0x7fb928155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb92ecf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb928153be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb92ecf4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280c36b0 con 0x7fb9280c30f0 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb92ecf4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb928071540 con 0x7fb92814f6b0 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb92ecf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb928071790 con 0x7fb928153790 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb927fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb9280c30f0 0x7fb92806bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49882/0 (socket says 172.21.15.17:49882) 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.770+0000 7fb927fff640 1 -- 172.21.15.17:0/4113766314 learned_addr learned my addr 172.21.15.17:0/4113766314 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822592359 0 0) 0x7fb9280c36b0 con 0x7fb9280c30f0 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb918003520 con 0x7fb9280c30f0 2024-03-06T07:21:52.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> v2:172.21.15.163:3300/0 conn(0x7fb92814f6b0 0x7fb928155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92d291640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb928153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1691760414 0 0) 0x7fb918003520 con 0x7fb9280c30f0 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 msgr2=0x7fb928153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb928153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 >> v2:172.21.15.163:3300/0 conn(0x7fb92814f6b0 msgr2=0x7fb928155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 --2- 172.21.15.17:0/4113766314 >> v2:172.21.15.163:3300/0 conn(0x7fb92814f6b0 0x7fb928155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928071a70 con 0x7fb9280c30f0 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb914001690 con 0x7fb9280c30f0 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92d291640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb928153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> v2:172.21.15.163:3300/0 conn(0x7fb92814f6b0 0x7fb928155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 >> v1:172.21.15.17:6789/0 conn(0x7fb9280c30f0 legacy=0x7fb92806bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9180039b0 0x7fb918003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb918004800 0x7fb918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb918005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928071540 con 0x7fb9180039b0 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928071790 con 0x7fb918004800 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91800bbd0 con 0x7fb928153790 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9180039b0 0x7fb918003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb927fff640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb918004800 0x7fb918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92d291640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb918005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 -- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 msgr2=0x7fb918005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb928153790 0x7fb918005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 -- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb918004800 msgr2=0x7fb918004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.771+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb918004800 0x7fb918004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.772+0000 7fb92ca90640 1 -- 172.21.15.17:0/4113766314 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918003520 con 0x7fb9180039b0 2024-03-06T07:21:52.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ca90640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9180039b0 0x7fb918003dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd6504db732bb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00f040 con 0x7fb9180039b0 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91c004230 con 0x7fb9180039b0 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb9277fe640 1 -- 172.21.15.17:0/4113766314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c003610 con 0x7fb9180039b0 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 -- 172.21.15.17:0/4113766314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9180039b0 msgr2=0x7fb918003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/4113766314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9180039b0 0x7fb918003dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb918005830 comp rx=0 tx=0).stop 2024-03-06T07:21:52.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92d291640 1 -- 172.21.15.17:0/4113766314 reap_dead start 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 -- 172.21.15.17:0/4113766314 shutdown_connections 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 -- 172.21.15.17:0/4113766314 >> 172.21.15.17:0/4113766314 conn(0x7fb9280c3ad0 msgr2=0x7fb9280c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 -- 172.21.15.17:0/4113766314 shutdown_connections 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.773+0000 7fb92ecf4640 1 -- 172.21.15.17:0/4113766314 wait complete. 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.774+0000 7fb92ecf4640 1 Processor -- start 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 -- start start 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9180039b0 0x7fb9280c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9280c5520 0x7fb9281ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92806c6e0 con 0x7fb9281eba10 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928071d50 con 0x7fb9180039b0 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92ecf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92806c860 con 0x7fb9280c5520 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92d291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.775+0000 7fb92d291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56476/0 (socket says 172.21.15.17:56476) 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 -- 172.21.15.17:0/75933041 learned_addr learned my addr 172.21.15.17:0/75933041 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb927fff640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9280c5520 0x7fb9281ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92ca90640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9180039b0 0x7fb9280c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 -- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9280c5520 msgr2=0x7fb9281ef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9280c5520 0x7fb9281ef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 -- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9180039b0 msgr2=0x7fb9280c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9180039b0 0x7fb9280c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 -- 172.21.15.17:0/75933041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9281eb230 con 0x7fb9281eba10 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92d291640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb928070ae0 tx=0x7fb920010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920010d20 con 0x7fb9281eba10 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb920003990 con 0x7fb9281eba10 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92814d8a0 con 0x7fb9281eba10 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb920003b80 con 0x7fb9281eba10 2024-03-06T07:21:52.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.776+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9281eb4f0 con 0x7fb9281eba10 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.777+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb928074ac0 con 0x7fb9281eba10 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.778+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb92001e020 con 0x7fb9281eba10 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.778+0000 7fb9257fa640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 0x7fb90c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.778+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb92008a950 con 0x7fb9281eba10 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.779+0000 7fb92ca90640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 0x7fb90c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:52.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.779+0000 7fb92ca90640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 0x7fb90c06d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb91c03a040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:52.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.782+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb928074ac0 con 0x7fb9281eba10 2024-03-06T07:21:52.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.964+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb92806c1c0 con 0x7fb90c06b500 2024-03-06T07:21:52.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.970+0000 7fb9257fa640 1 -- 172.21.15.17:0/75933041 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251918 (secure 0 0 0) 0x7fb91c0040b0 con 0x7fb90c06b500 2024-03-06T07:21:52.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:52.977 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 msgr2=0x7fb90c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 0x7fb90c06d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0098a0 tx=0x7fb91c03a040 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 msgr2=0x7fb9281dea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb928070ae0 tx=0x7fb920010530 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 shutdown_connections 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb90c06b500 0x7fb90c06d9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9281eba10 0x7fb9281dea40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9280c5520 0x7fb9281ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 --2- 172.21.15.17:0/75933041 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9180039b0 0x7fb9280c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 >> 172.21.15.17:0/75933041 conn(0x7fb9280c3ad0 msgr2=0x7fb9281553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 shutdown_connections 2024-03-06T07:21:52.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:52.974+0000 7fb92ecf4640 1 -- 172.21.15.17:0/75933041 wait complete. 2024-03-06T07:21:52.991 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-03-06T07:21:52.931347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237760,"kb_used_data":4928,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736960,"statfs":{"total":773094113280,"available":772850647040,"internally_reserved":0,"allocated":5046272,"data_stored":3364772,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.033987"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182072+0000","last_change":"2024-03-06T07:21:43.913948+0000","last_active":"2024-03-06T07:21:45.182072+0000","last_peered":"2024-03-06T07:21:45.182072+0000","last_clean":"2024-03-06T07:21:45.182072+0000","last_became_active":"2024-03-06T07:21:43.911415+0000","last_became_peered":"2024-03-06T07:21:43.911415+0000","last_unstale":"2024-03-06T07:21:45.182072+0000","last_undegraded":"2024-03-06T07:21:45.182072+0000","last_fullsized":"2024-03-06T07:21:45.182072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:51.582422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922365+0000","last_change":"2024-03-06T07:21:43.914337+0000","last_active":"2024-03-06T07:21:44.922365+0000","last_peered":"2024-03-06T07:21:44.922365+0000","last_clean":"2024-03-06T07:21:44.922365+0000","last_became_active":"2024-03-06T07:21:43.914043+0000","last_became_peered":"2024-03-06T07:21:43.914043+0000","last_unstale":"2024-03-06T07:21:44.922365+0000","last_undegraded":"2024-03-06T07:21:44.922365+0000","last_fullsized":"2024-03-06T07:21:44.922365+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.432948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919873+0000","last_change":"2024-03-06T07:21:43.935409+0000","last_active":"2024-03-06T07:21:44.919873+0000","last_peered":"2024-03-06T07:21:44.919873+0000","last_clean":"2024-03-06T07:21:44.919873+0000","last_became_active":"2024-03-06T07:21:43.934832+0000","last_became_peered":"2024-03-06T07:21:43.934832+0000","last_unstale":"2024-03-06T07:21:44.919873+0000","last_undegraded":"2024-03-06T07:21:44.919873+0000","last_fullsized":"2024-03-06T07:21:44.919873+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:57.737436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922160+0000","last_change":"2024-03-06T07:21:43.913875+0000","last_active":"2024-03-06T07:21:44.922160+0000","last_peered":"2024-03-06T07:21:44.922160+0000","last_clean":"2024-03-06T07:21:44.922160+0000","last_became_active":"2024-03-06T07:21:43.912314+0000","last_became_peered":"2024-03-06T07:21:43.912314+0000","last_unstale":"2024-03-06T07:21:44.922160+0000","last_undegraded":"2024-03-06T07:21:44.922160+0000","last_fullsized":"2024-03-06T07:21:44.922160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:01.595661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913814+0000","last_change":"2024-03-06T07:21:43.913216+0000","last_active":"2024-03-06T07:21:44.913814+0000","last_peered":"2024-03-06T07:21:44.913814+0000","last_clean":"2024-03-06T07:21:44.913814+0000","last_became_active":"2024-03-06T07:21:43.912935+0000","last_became_peered":"2024-03-06T07:21:43.912935+0000","last_unstale":"2024-03-06T07:21:44.913814+0000","last_undegraded":"2024-03-06T07:21:44.913814+0000","last_fullsized":"2024-03-06T07:21:44.913814+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181857+0000","last_change":"2024-03-06T07:21:43.917357+0000","last_active":"2024-03-06T07:21:45.181857+0000","last_peered":"2024-03-06T07:21:45.181857+0000","last_clean":"2024-03-06T07:21:45.181857+0000","last_became_active":"2024-03-06T07:21:43.916973+0000","last_became_peered":"2024-03-06T07:21:43.916973+0000","last_unstale":"2024-03-06T07:21:45.181857+0000","last_undegraded":"2024-03-06T07:21:45.181857+0000","last_fullsized":"2024-03-06T07:21:45.181857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919492+0000","last_change":"2024-03-06T07:21:43.923785+0000","last_active":"2024-03-06T07:21:44.919492+0000","last_peered":"2024-03-06T07:21:44.919492+0000","last_clean":"2024-03-06T07:21:44.919492+0000","last_became_active":"2024-03-06T07:21:43.923460+0000","last_became_peered":"2024-03-06T07:21:43.923460+0000","last_unstale":"2024-03-06T07:21:44.919492+0000","last_undegraded":"2024-03-06T07:21:44.919492+0000","last_fullsized":"2024-03-06T07:21:44.919492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117372+0000","last_change":"2024-03-06T07:21:43.928597+0000","last_active":"2024-03-06T07:21:45.117372+0000","last_peered":"2024-03-06T07:21:45.117372+0000","last_clean":"2024-03-06T07:21:45.117372+0000","last_became_active":"2024-03-06T07:21:43.928330+0000","last_became_peered":"2024-03-06T07:21:43.928330+0000","last_unstale":"2024-03-06T07:21:45.117372+0000","last_undegraded":"2024-03-06T07:21:45.117372+0000","last_fullsized":"2024-03-06T07:21:45.117372+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117615+0000","last_change":"2024-03-06T07:21:43.938313+0000","last_active":"2024-03-06T07:21:45.117615+0000","last_peered":"2024-03-06T07:21:45.117615+0000","last_clean":"2024-03-06T07:21:45.117615+0000","last_became_active":"2024-03-06T07:21:43.938037+0000","last_became_peered":"2024-03-06T07:21:43.938037+0000","last_unstale":"2024-03-06T07:21:45.117615+0000","last_undegraded":"2024-03-06T07:21:45.117615+0000","last_fullsized":"2024-03-06T07:21:45.117615+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:12.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182055+0000","last_change":"2024-03-06T07:21:43.910022+0000","last_active":"2024-03-06T07:21:45.182055+0000","last_peered":"2024-03-06T07:21:45.182055+0000","last_clean":"2024-03-06T07:21:45.182055+0000","last_became_active":"2024-03-06T07:21:43.909753+0000","last_became_peered":"2024-03-06T07:21:43.909753+0000","last_unstale":"2024-03-06T07:21:45.182055+0000","last_undegraded":"2024-03-06T07:21:45.182055+0000","last_fullsized":"2024-03-06T07:21:45.182055+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:14.243609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.910236+0000","last_change":"2024-03-06T07:21:43.917615+0000","last_active":"2024-03-06T07:21:44.910236+0000","last_peered":"2024-03-06T07:21:44.910236+0000","last_clean":"2024-03-06T07:21:44.910236+0000","last_became_active":"2024-03-06T07:21:43.917363+0000","last_became_peered":"2024-03-06T07:21:43.917363+0000","last_unstale":"2024-03-06T07:21:44.910236+0000","last_undegraded":"2024-03-06T07:21:44.910236+0000","last_fullsized":"2024-03-06T07:21:44.910236+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:29.125938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117697+0000","last_change":"2024-03-06T07:21:43.911518+0000","last_active":"2024-03-06T07:21:45.117697+0000","last_peered":"2024-03-06T07:21:45.117697+0000","last_clean":"2024-03-06T07:21:45.117697+0000","last_became_active":"2024-03-06T07:21:43.911238+0000","last_became_peered":"2024-03-06T07:21:43.911238+0000","last_unstale":"2024-03-06T07:21:45.117697+0000","last_undegraded":"2024-03-06T07:21:45.117697+0000","last_fullsized":"2024-03-06T07:21:45.117697+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:14.588826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182046+0000","last_change":"2024-03-06T07:21:43.916170+0000","last_active":"2024-03-06T07:21:45.182046+0000","last_peered":"2024-03-06T07:21:45.182046+0000","last_clean":"2024-03-06T07:21:45.182046+0000","last_became_active":"2024-03-06T07:21:43.915661+0000","last_became_peered":"2024-03-06T07:21:43.915661+0000","last_unstale":"2024-03-06T07:21:45.182046+0000","last_undegraded":"2024-03-06T07:21:45.182046+0000","last_fullsized":"2024-03-06T07:21:45.182046+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919810+0000","last_change":"2024-03-06T07:21:43.925030+0000","last_active":"2024-03-06T07:21:44.919810+0000","last_peered":"2024-03-06T07:21:44.919810+0000","last_clean":"2024-03-06T07:21:44.919810+0000","last_became_active":"2024-03-06T07:21:43.924480+0000","last_became_peered":"2024-03-06T07:21:43.924480+0000","last_unstale":"2024-03-06T07:21:44.919810+0000","last_undegraded":"2024-03-06T07:21:44.919810+0000","last_fullsized":"2024-03-06T07:21:44.919810+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:36.336480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922091+0000","last_change":"2024-03-06T07:21:43.915656+0000","last_active":"2024-03-06T07:21:44.922091+0000","last_peered":"2024-03-06T07:21:44.922091+0000","last_clean":"2024-03-06T07:21:44.922091+0000","last_became_active":"2024-03-06T07:21:43.915399+0000","last_became_peered":"2024-03-06T07:21:43.915399+0000","last_unstale":"2024-03-06T07:21:44.922091+0000","last_undegraded":"2024-03-06T07:21:44.922091+0000","last_fullsized":"2024-03-06T07:21:44.922091+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181753+0000","last_change":"2024-03-06T07:21:43.909296+0000","last_active":"2024-03-06T07:21:45.181753+0000","last_peered":"2024-03-06T07:21:45.181753+0000","last_clean":"2024-03-06T07:21:45.181753+0000","last_became_active":"2024-03-06T07:21:43.909011+0000","last_became_peered":"2024-03-06T07:21:43.909011+0000","last_unstale":"2024-03-06T07:21:45.181753+0000","last_undegraded":"2024-03-06T07:21:45.181753+0000","last_fullsized":"2024-03-06T07:21:45.181753+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920094+0000","last_change":"2024-03-06T07:21:43.930357+0000","last_active":"2024-03-06T07:21:44.920094+0000","last_peered":"2024-03-06T07:21:44.920094+0000","last_clean":"2024-03-06T07:21:44.920094+0000","last_became_active":"2024-03-06T07:21:43.929832+0000","last_became_peered":"2024-03-06T07:21:43.929832+0000","last_unstale":"2024-03-06T07:21:44.920094+0000","last_undegraded":"2024-03-06T07:21:44.920094+0000","last_fullsized":"2024-03-06T07:21:44.920094+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922408+0000","last_change":"2024-03-06T07:21:43.919806+0000","last_active":"2024-03-06T07:21:44.922408+0000","last_peered":"2024-03-06T07:21:44.922408+0000","last_clean":"2024-03-06T07:21:44.922408+0000","last_became_active":"2024-03-06T07:21:43.919524+0000","last_became_peered":"2024-03-06T07:21:43.919524+0000","last_unstale":"2024-03-06T07:21:44.922408+0000","last_undegraded":"2024-03-06T07:21:44.922408+0000","last_fullsized":"2024-03-06T07:21:44.922408+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919921+0000","last_change":"2024-03-06T07:21:43.933746+0000","last_active":"2024-03-06T07:21:44.919921+0000","last_peered":"2024-03-06T07:21:44.919921+0000","last_clean":"2024-03-06T07:21:44.919921+0000","last_became_active":"2024-03-06T07:21:43.932918+0000","last_became_peered":"2024-03-06T07:21:43.932918+0000","last_unstale":"2024-03-06T07:21:44.919921+0000","last_undegraded":"2024-03-06T07:21:44.919921+0000","last_fullsized":"2024-03-06T07:21:44.919921+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913093+0000","last_change":"2024-03-06T07:21:43.916874+0000","last_active":"2024-03-06T07:21:44.913093+0000","last_peered":"2024-03-06T07:21:44.913093+0000","last_clean":"2024-03-06T07:21:44.913093+0000","last_became_active":"2024-03-06T07:21:43.916593+0000","last_became_peered":"2024-03-06T07:21:43.916593+0000","last_unstale":"2024-03-06T07:21:44.913093+0000","last_undegraded":"2024-03-06T07:21:44.913093+0000","last_fullsized":"2024-03-06T07:21:44.913093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:30.373457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913241+0000","last_change":"2024-03-06T07:21:43.914663+0000","last_active":"2024-03-06T07:21:44.913241+0000","last_peered":"2024-03-06T07:21:44.913241+0000","last_clean":"2024-03-06T07:21:44.913241+0000","last_became_active":"2024-03-06T07:21:43.914383+0000","last_became_peered":"2024-03-06T07:21:43.914383+0000","last_unstale":"2024-03-06T07:21:44.913241+0000","last_undegraded":"2024-03-06T07:21:44.913241+0000","last_fullsized":"2024-03-06T07:21:44.913241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117490+0000","last_change":"2024-03-06T07:21:43.929377+0000","last_active":"2024-03-06T07:21:45.117490+0000","last_peered":"2024-03-06T07:21:45.117490+0000","last_clean":"2024-03-06T07:21:45.117490+0000","last_became_active":"2024-03-06T07:21:43.929093+0000","last_became_peered":"2024-03-06T07:21:43.929093+0000","last_unstale":"2024-03-06T07:21:45.117490+0000","last_undegraded":"2024-03-06T07:21:45.117490+0000","last_fullsized":"2024-03-06T07:21:45.117490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.909992+0000","last_change":"2024-03-06T07:21:43.916941+0000","last_active":"2024-03-06T07:21:44.909992+0000","last_peered":"2024-03-06T07:21:44.909992+0000","last_clean":"2024-03-06T07:21:44.909992+0000","last_became_active":"2024-03-06T07:21:43.916667+0000","last_became_peered":"2024-03-06T07:21:43.916667+0000","last_unstale":"2024-03-06T07:21:44.909992+0000","last_undegraded":"2024-03-06T07:21:44.909992+0000","last_fullsized":"2024-03-06T07:21:44.909992+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:15.683227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919459+0000","last_change":"2024-03-06T07:21:43.924259+0000","last_active":"2024-03-06T07:21:44.919459+0000","last_peered":"2024-03-06T07:21:44.919459+0000","last_clean":"2024-03-06T07:21:44.919459+0000","last_became_active":"2024-03-06T07:21:43.923885+0000","last_became_peered":"2024-03-06T07:21:43.923885+0000","last_unstale":"2024-03-06T07:21:44.919459+0000","last_undegraded":"2024-03-06T07:21:44.919459+0000","last_fullsized":"2024-03-06T07:21:44.919459+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920167+0000","last_change":"2024-03-06T07:21:43.931109+0000","last_active":"2024-03-06T07:21:44.920167+0000","last_peered":"2024-03-06T07:21:44.920167+0000","last_clean":"2024-03-06T07:21:44.920167+0000","last_became_active":"2024-03-06T07:21:43.930848+0000","last_became_peered":"2024-03-06T07:21:43.930848+0000","last_unstale":"2024-03-06T07:21:44.920167+0000","last_undegraded":"2024-03-06T07:21:44.920167+0000","last_fullsized":"2024-03-06T07:21:44.920167+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117542+0000","last_change":"2024-03-06T07:21:43.913493+0000","last_active":"2024-03-06T07:21:45.117542+0000","last_peered":"2024-03-06T07:21:45.117542+0000","last_clean":"2024-03-06T07:21:45.117542+0000","last_became_active":"2024-03-06T07:21:43.912992+0000","last_became_peered":"2024-03-06T07:21:43.912992+0000","last_unstale":"2024-03-06T07:21:45.117542+0000","last_undegraded":"2024-03-06T07:21:45.117542+0000","last_fullsized":"2024-03-06T07:21:45.117542+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:30.239239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182122+0000","last_change":"2024-03-06T07:21:43.934992+0000","last_active":"2024-03-06T07:21:45.182122+0000","last_peered":"2024-03-06T07:21:45.182122+0000","last_clean":"2024-03-06T07:21:45.182122+0000","last_became_active":"2024-03-06T07:21:43.934664+0000","last_became_peered":"2024-03-06T07:21:43.934664+0000","last_unstale":"2024-03-06T07:21:45.182122+0000","last_undegraded":"2024-03-06T07:21:45.182122+0000","last_fullsized":"2024-03-06T07:21:45.182122+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913248+0000","last_change":"2024-03-06T07:21:43.910958+0000","last_active":"2024-03-06T07:21:44.913248+0000","last_peered":"2024-03-06T07:21:44.913248+0000","last_clean":"2024-03-06T07:21:44.913248+0000","last_became_active":"2024-03-06T07:21:43.910653+0000","last_became_peered":"2024-03-06T07:21:43.910653+0000","last_unstale":"2024-03-06T07:21:44.913248+0000","last_undegraded":"2024-03-06T07:21:44.913248+0000","last_fullsized":"2024-03-06T07:21:44.913248+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919622+0000","last_change":"2024-03-06T07:21:43.923484+0000","last_active":"2024-03-06T07:21:44.919622+0000","last_peered":"2024-03-06T07:21:44.919622+0000","last_clean":"2024-03-06T07:21:44.919622+0000","last_became_active":"2024-03-06T07:21:43.921095+0000","last_became_peered":"2024-03-06T07:21:43.921095+0000","last_unstale":"2024-03-06T07:21:44.919622+0000","last_undegraded":"2024-03-06T07:21:44.919622+0000","last_fullsized":"2024-03-06T07:21:44.919622+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922195+0000","last_change":"2024-03-06T07:21:43.917834+0000","last_active":"2024-03-06T07:21:44.922195+0000","last_peered":"2024-03-06T07:21:44.922195+0000","last_clean":"2024-03-06T07:21:44.922195+0000","last_became_active":"2024-03-06T07:21:43.917576+0000","last_became_peered":"2024-03-06T07:21:43.917576+0000","last_unstale":"2024-03-06T07:21:44.922195+0000","last_undegraded":"2024-03-06T07:21:44.922195+0000","last_fullsized":"2024-03-06T07:21:44.922195+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919611+0000","last_change":"2024-03-06T07:21:43.921467+0000","last_active":"2024-03-06T07:21:44.919611+0000","last_peered":"2024-03-06T07:21:44.919611+0000","last_clean":"2024-03-06T07:21:44.919611+0000","last_became_active":"2024-03-06T07:21:43.920878+0000","last_became_peered":"2024-03-06T07:21:43.920878+0000","last_unstale":"2024-03-06T07:21:44.919611+0000","last_undegraded":"2024-03-06T07:21:44.919611+0000","last_fullsized":"2024-03-06T07:21:44.919611+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182498+0000","last_change":"2024-03-06T07:21:43.911317+0000","last_active":"2024-03-06T07:21:45.182498+0000","last_peered":"2024-03-06T07:21:45.182498+0000","last_clean":"2024-03-06T07:21:45.182498+0000","last_became_active":"2024-03-06T07:21:43.910980+0000","last_became_peered":"2024-03-06T07:21:43.910980+0000","last_unstale":"2024-03-06T07:21:45.182498+0000","last_undegraded":"2024-03-06T07:21:45.182498+0000","last_fullsized":"2024-03-06T07:21:45.182498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.921881+0000","last_change":"2024-03-06T07:21:43.914331+0000","last_active":"2024-03-06T07:21:44.921881+0000","last_peered":"2024-03-06T07:21:44.921881+0000","last_clean":"2024-03-06T07:21:44.921881+0000","last_became_active":"2024-03-06T07:21:43.913681+0000","last_became_peered":"2024-03-06T07:21:43.913681+0000","last_unstale":"2024-03-06T07:21:44.921881+0000","last_undegraded":"2024-03-06T07:21:44.921881+0000","last_fullsized":"2024-03-06T07:21:44.921881+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913353+0000","last_change":"2024-03-06T07:21:42.908247+0000","last_active":"2024-03-06T07:21:44.913353+0000","last_peered":"2024-03-06T07:21:44.913353+0000","last_clean":"2024-03-06T07:21:44.913353+0000","last_became_active":"2024-03-06T07:21:42.907863+0000","last_became_peered":"2024-03-06T07:21:42.907863+0000","last_unstale":"2024-03-06T07:21:44.913353+0000","last_undegraded":"2024-03-06T07:21:44.913353+0000","last_fullsized":"2024-03-06T07:21:44.913353+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:44.909708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182455+0000","last_change":"2024-03-06T07:21:43.918214+0000","last_active":"2024-03-06T07:21:45.182455+0000","last_peered":"2024-03-06T07:21:45.182455+0000","last_clean":"2024-03-06T07:21:45.182455+0000","last_became_active":"2024-03-06T07:21:43.917454+0000","last_became_peered":"2024-03-06T07:21:43.917454+0000","last_unstale":"2024-03-06T07:21:45.182455+0000","last_undegraded":"2024-03-06T07:21:45.182455+0000","last_fullsized":"2024-03-06T07:21:45.182455+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181920+0000","last_change":"2024-03-06T07:21:42.915335+0000","last_active":"2024-03-06T07:21:45.181920+0000","last_peered":"2024-03-06T07:21:45.181920+0000","last_clean":"2024-03-06T07:21:45.181920+0000","last_became_active":"2024-03-06T07:21:42.915011+0000","last_became_peered":"2024-03-06T07:21:42.915011+0000","last_unstale":"2024-03-06T07:21:45.181920+0000","last_undegraded":"2024-03-06T07:21:45.181920+0000","last_fullsized":"2024-03-06T07:21:45.181920+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912924+0000","last_change":"2024-03-06T07:21:43.914522+0000","last_active":"2024-03-06T07:21:44.912924+0000","last_peered":"2024-03-06T07:21:44.912924+0000","last_clean":"2024-03-06T07:21:44.912924+0000","last_became_active":"2024-03-06T07:21:43.913602+0000","last_became_peered":"2024-03-06T07:21:43.913602+0000","last_unstale":"2024-03-06T07:21:44.912924+0000","last_undegraded":"2024-03-06T07:21:44.912924+0000","last_fullsized":"2024-03-06T07:21:44.912924+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:02.123078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182059+0000","last_change":"2024-03-06T07:21:42.914470+0000","last_active":"2024-03-06T07:21:45.182059+0000","last_peered":"2024-03-06T07:21:45.182059+0000","last_clean":"2024-03-06T07:21:45.182059+0000","last_became_active":"2024-03-06T07:21:42.913930+0000","last_became_peered":"2024-03-06T07:21:42.913930+0000","last_unstale":"2024-03-06T07:21:45.182059+0000","last_undegraded":"2024-03-06T07:21:45.182059+0000","last_fullsized":"2024-03-06T07:21:45.182059+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:24.367917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117391+0000","last_change":"2024-03-06T07:21:43.912159+0000","last_active":"2024-03-06T07:21:45.117391+0000","last_peered":"2024-03-06T07:21:45.117391+0000","last_clean":"2024-03-06T07:21:45.117391+0000","last_became_active":"2024-03-06T07:21:43.911851+0000","last_became_peered":"2024-03-06T07:21:43.911851+0000","last_unstale":"2024-03-06T07:21:45.117391+0000","last_undegraded":"2024-03-06T07:21:45.117391+0000","last_fullsized":"2024-03-06T07:21:45.117391+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182377+0000","last_change":"2024-03-06T07:21:42.912811+0000","last_active":"2024-03-06T07:21:45.182377+0000","last_peered":"2024-03-06T07:21:45.182377+0000","last_clean":"2024-03-06T07:21:45.182377+0000","last_became_active":"2024-03-06T07:21:42.912544+0000","last_became_peered":"2024-03-06T07:21:42.912544+0000","last_unstale":"2024-03-06T07:21:45.182377+0000","last_undegraded":"2024-03-06T07:21:45.182377+0000","last_fullsized":"2024-03-06T07:21:45.182377+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:50.577390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913967+0000","last_change":"2024-03-06T07:21:43.922679+0000","last_active":"2024-03-06T07:21:44.913967+0000","last_peered":"2024-03-06T07:21:44.913967+0000","last_clean":"2024-03-06T07:21:44.913967+0000","last_became_active":"2024-03-06T07:21:43.922322+0000","last_became_peered":"2024-03-06T07:21:43.922322+0000","last_unstale":"2024-03-06T07:21:44.913967+0000","last_undegraded":"2024-03-06T07:21:44.913967+0000","last_fullsized":"2024-03-06T07:21:44.913967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919558+0000","last_change":"2024-03-06T07:21:42.912329+0000","last_active":"2024-03-06T07:21:44.919558+0000","last_peered":"2024-03-06T07:21:44.919558+0000","last_clean":"2024-03-06T07:21:44.919558+0000","last_became_active":"2024-03-06T07:21:42.911690+0000","last_became_peered":"2024-03-06T07:21:42.911690+0000","last_unstale":"2024-03-06T07:21:44.919558+0000","last_undegraded":"2024-03-06T07:21:44.919558+0000","last_fullsized":"2024-03-06T07:21:44.919558+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922131+0000","last_change":"2024-03-06T07:21:43.917686+0000","last_active":"2024-03-06T07:21:44.922131+0000","last_peered":"2024-03-06T07:21:44.922131+0000","last_clean":"2024-03-06T07:21:44.922131+0000","last_became_active":"2024-03-06T07:21:43.917369+0000","last_became_peered":"2024-03-06T07:21:43.917369+0000","last_unstale":"2024-03-06T07:21:44.922131+0000","last_undegraded":"2024-03-06T07:21:44.922131+0000","last_fullsized":"2024-03-06T07:21:44.922131+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.013761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913597+0000","last_change":"2024-03-06T07:21:42.910602+0000","last_active":"2024-03-06T07:21:44.913597+0000","last_peered":"2024-03-06T07:21:44.913597+0000","last_clean":"2024-03-06T07:21:44.913597+0000","last_became_active":"2024-03-06T07:21:42.910279+0000","last_became_peered":"2024-03-06T07:21:42.910279+0000","last_unstale":"2024-03-06T07:21:44.913597+0000","last_undegraded":"2024-03-06T07:21:44.913597+0000","last_fullsized":"2024-03-06T07:21:44.913597+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:02.037327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117696+0000","last_change":"2024-03-06T07:21:43.913254+0000","last_active":"2024-03-06T07:21:45.117696+0000","last_peered":"2024-03-06T07:21:45.117696+0000","last_clean":"2024-03-06T07:21:45.117696+0000","last_became_active":"2024-03-06T07:21:43.912620+0000","last_became_peered":"2024-03-06T07:21:43.912620+0000","last_unstale":"2024-03-06T07:21:45.117696+0000","last_undegraded":"2024-03-06T07:21:45.117696+0000","last_fullsized":"2024-03-06T07:21:45.117696+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:13.059235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912746+0000","last_change":"2024-03-06T07:21:42.909672+0000","last_active":"2024-03-06T07:21:44.912746+0000","last_peered":"2024-03-06T07:21:44.912746+0000","last_clean":"2024-03-06T07:21:44.912746+0000","last_became_active":"2024-03-06T07:21:42.909402+0000","last_became_peered":"2024-03-06T07:21:42.909402+0000","last_unstale":"2024-03-06T07:21:44.912746+0000","last_undegraded":"2024-03-06T07:21:44.912746+0000","last_fullsized":"2024-03-06T07:21:44.912746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:06.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922430+0000","last_change":"2024-03-06T07:21:43.934375+0000","last_active":"2024-03-06T07:21:44.922430+0000","last_peered":"2024-03-06T07:21:44.922430+0000","last_clean":"2024-03-06T07:21:44.922430+0000","last_became_active":"2024-03-06T07:21:43.934076+0000","last_became_peered":"2024-03-06T07:21:43.934076+0000","last_unstale":"2024-03-06T07:21:44.922430+0000","last_undegraded":"2024-03-06T07:21:44.922430+0000","last_fullsized":"2024-03-06T07:21:44.922430+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919390+0000","last_change":"2024-03-06T07:21:42.910541+0000","last_active":"2024-03-06T07:21:44.919390+0000","last_peered":"2024-03-06T07:21:44.919390+0000","last_clean":"2024-03-06T07:21:44.919390+0000","last_became_active":"2024-03-06T07:21:42.910271+0000","last_became_peered":"2024-03-06T07:21:42.910271+0000","last_unstale":"2024-03-06T07:21:44.919390+0000","last_undegraded":"2024-03-06T07:21:44.919390+0000","last_fullsized":"2024-03-06T07:21:44.919390+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913081+0000","last_change":"2024-03-06T07:21:43.905571+0000","last_active":"2024-03-06T07:21:44.913081+0000","last_peered":"2024-03-06T07:21:44.913081+0000","last_clean":"2024-03-06T07:21:44.913081+0000","last_became_active":"2024-03-06T07:21:43.904452+0000","last_became_peered":"2024-03-06T07:21:43.904452+0000","last_unstale":"2024-03-06T07:21:44.913081+0000","last_undegraded":"2024-03-06T07:21:44.913081+0000","last_fullsized":"2024-03-06T07:21:44.913081+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922346+0000","last_change":"2024-03-06T07:21:42.913817+0000","last_active":"2024-03-06T07:21:44.922346+0000","last_peered":"2024-03-06T07:21:44.922346+0000","last_clean":"2024-03-06T07:21:44.922346+0000","last_became_active":"2024-03-06T07:21:42.913464+0000","last_became_peered":"2024-03-06T07:21:42.913464+0000","last_unstale":"2024-03-06T07:21:44.922346+0000","last_undegraded":"2024-03-06T07:21:44.922346+0000","last_fullsized":"2024-03-06T07:21:44.922346+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182307+0000","last_change":"2024-03-06T07:21:43.912031+0000","last_active":"2024-03-06T07:21:45.182307+0000","last_peered":"2024-03-06T07:21:45.182307+0000","last_clean":"2024-03-06T07:21:45.182307+0000","last_became_active":"2024-03-06T07:21:43.911759+0000","last_became_peered":"2024-03-06T07:21:43.911759+0000","last_unstale":"2024-03-06T07:21:45.182307+0000","last_undegraded":"2024-03-06T07:21:45.182307+0000","last_fullsized":"2024-03-06T07:21:45.182307+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913289+0000","last_change":"2024-03-06T07:21:42.910374+0000","last_active":"2024-03-06T07:21:44.913289+0000","last_peered":"2024-03-06T07:21:44.913289+0000","last_clean":"2024-03-06T07:21:44.913289+0000","last_became_active":"2024-03-06T07:21:42.910082+0000","last_became_peered":"2024-03-06T07:21:42.910082+0000","last_unstale":"2024-03-06T07:21:44.913289+0000","last_undegraded":"2024-03-06T07:21:44.913289+0000","last_fullsized":"2024-03-06T07:21:44.913289+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:55.547305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919688+0000","last_change":"2024-03-06T07:21:43.924815+0000","last_active":"2024-03-06T07:21:44.919688+0000","last_peered":"2024-03-06T07:21:44.919688+0000","last_clean":"2024-03-06T07:21:44.919688+0000","last_became_active":"2024-03-06T07:21:43.924188+0000","last_became_peered":"2024-03-06T07:21:43.924188+0000","last_unstale":"2024-03-06T07:21:44.919688+0000","last_undegraded":"2024-03-06T07:21:44.919688+0000","last_fullsized":"2024-03-06T07:21:44.919688+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:36:57.007684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922220+0000","last_change":"2024-03-06T07:21:42.912311+0000","last_active":"2024-03-06T07:21:44.922220+0000","last_peered":"2024-03-06T07:21:44.922220+0000","last_clean":"2024-03-06T07:21:44.922220+0000","last_became_active":"2024-03-06T07:21:42.912046+0000","last_became_peered":"2024-03-06T07:21:42.912046+0000","last_unstale":"2024-03-06T07:21:44.922220+0000","last_undegraded":"2024-03-06T07:21:44.922220+0000","last_fullsized":"2024-03-06T07:21:44.922220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913937+0000","last_change":"2024-03-06T07:21:43.911762+0000","last_active":"2024-03-06T07:21:44.913937+0000","last_peered":"2024-03-06T07:21:44.913937+0000","last_clean":"2024-03-06T07:21:44.913937+0000","last_became_active":"2024-03-06T07:21:43.911470+0000","last_became_peered":"2024-03-06T07:21:43.911470+0000","last_unstale":"2024-03-06T07:21:44.913937+0000","last_undegraded":"2024-03-06T07:21:44.913937+0000","last_fullsized":"2024-03-06T07:21:44.913937+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117652+0000","last_change":"2024-03-06T07:21:42.912419+0000","last_active":"2024-03-06T07:21:45.117652+0000","last_peered":"2024-03-06T07:21:45.117652+0000","last_clean":"2024-03-06T07:21:45.117652+0000","last_became_active":"2024-03-06T07:21:42.912141+0000","last_became_peered":"2024-03-06T07:21:42.912141+0000","last_unstale":"2024-03-06T07:21:45.117652+0000","last_undegraded":"2024-03-06T07:21:45.117652+0000","last_fullsized":"2024-03-06T07:21:45.117652+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:00.590629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182495+0000","last_change":"2024-03-06T07:21:43.918063+0000","last_active":"2024-03-06T07:21:45.182495+0000","last_peered":"2024-03-06T07:21:45.182495+0000","last_clean":"2024-03-06T07:21:45.182495+0000","last_became_active":"2024-03-06T07:21:43.917325+0000","last_became_peered":"2024-03-06T07:21:43.917325+0000","last_unstale":"2024-03-06T07:21:45.182495+0000","last_undegraded":"2024-03-06T07:21:45.182495+0000","last_fullsized":"2024-03-06T07:21:45.182495+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919672+0000","last_change":"2024-03-06T07:21:42.912594+0000","last_active":"2024-03-06T07:21:44.919672+0000","last_peered":"2024-03-06T07:21:44.919672+0000","last_clean":"2024-03-06T07:21:44.919672+0000","last_became_active":"2024-03-06T07:21:42.912053+0000","last_became_peered":"2024-03-06T07:21:42.912053+0000","last_unstale":"2024-03-06T07:21:44.919672+0000","last_undegraded":"2024-03-06T07:21:44.919672+0000","last_fullsized":"2024-03-06T07:21:44.919672+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:42.711847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919982+0000","last_change":"2024-03-06T07:21:43.931801+0000","last_active":"2024-03-06T07:21:44.919982+0000","last_peered":"2024-03-06T07:21:44.919982+0000","last_clean":"2024-03-06T07:21:44.919982+0000","last_became_active":"2024-03-06T07:21:43.931515+0000","last_became_peered":"2024-03-06T07:21:43.931515+0000","last_unstale":"2024-03-06T07:21:44.919982+0000","last_undegraded":"2024-03-06T07:21:44.919982+0000","last_fullsized":"2024-03-06T07:21:44.919982+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912938+0000","last_change":"2024-03-06T07:21:42.910643+0000","last_active":"2024-03-06T07:21:44.912938+0000","last_peered":"2024-03-06T07:21:44.912938+0000","last_clean":"2024-03-06T07:21:44.912938+0000","last_became_active":"2024-03-06T07:21:42.910377+0000","last_became_peered":"2024-03-06T07:21:42.910377+0000","last_unstale":"2024-03-06T07:21:44.912938+0000","last_undegraded":"2024-03-06T07:21:44.912938+0000","last_fullsized":"2024-03-06T07:21:44.912938+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117660+0000","last_change":"2024-03-06T07:21:43.915960+0000","last_active":"2024-03-06T07:21:45.117660+0000","last_peered":"2024-03-06T07:21:45.117660+0000","last_clean":"2024-03-06T07:21:45.117660+0000","last_became_active":"2024-03-06T07:21:43.915665+0000","last_became_peered":"2024-03-06T07:21:43.915665+0000","last_unstale":"2024-03-06T07:21:45.117660+0000","last_undegraded":"2024-03-06T07:21:45.117660+0000","last_fullsized":"2024-03-06T07:21:45.117660+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:43.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922197+0000","last_change":"2024-03-06T07:21:42.911389+0000","last_active":"2024-03-06T07:21:44.922197+0000","last_peered":"2024-03-06T07:21:44.922197+0000","last_clean":"2024-03-06T07:21:44.922197+0000","last_became_active":"2024-03-06T07:21:42.911070+0000","last_became_peered":"2024-03-06T07:21:42.911070+0000","last_unstale":"2024-03-06T07:21:44.922197+0000","last_undegraded":"2024-03-06T07:21:44.922197+0000","last_fullsized":"2024-03-06T07:21:44.922197+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920157+0000","last_change":"2024-03-06T07:21:43.930573+0000","last_active":"2024-03-06T07:21:44.920157+0000","last_peered":"2024-03-06T07:21:44.920157+0000","last_clean":"2024-03-06T07:21:44.920157+0000","last_became_active":"2024-03-06T07:21:43.930105+0000","last_became_peered":"2024-03-06T07:21:43.930105+0000","last_unstale":"2024-03-06T07:21:44.920157+0000","last_undegraded":"2024-03-06T07:21:44.920157+0000","last_fullsized":"2024-03-06T07:21:44.920157+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117431+0000","last_change":"2024-03-06T07:21:42.915873+0000","last_active":"2024-03-06T07:21:45.117431+0000","last_peered":"2024-03-06T07:21:45.117431+0000","last_clean":"2024-03-06T07:21:45.117431+0000","last_became_active":"2024-03-06T07:21:42.915575+0000","last_became_peered":"2024-03-06T07:21:42.915575+0000","last_unstale":"2024-03-06T07:21:45.117431+0000","last_undegraded":"2024-03-06T07:21:45.117431+0000","last_fullsized":"2024-03-06T07:21:45.117431+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:30.008729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117436+0000","last_change":"2024-03-06T07:21:43.914468+0000","last_active":"2024-03-06T07:21:45.117436+0000","last_peered":"2024-03-06T07:21:45.117436+0000","last_clean":"2024-03-06T07:21:45.117436+0000","last_became_active":"2024-03-06T07:21:43.914210+0000","last_became_peered":"2024-03-06T07:21:43.914210+0000","last_unstale":"2024-03-06T07:21:45.117436+0000","last_undegraded":"2024-03-06T07:21:45.117436+0000","last_fullsized":"2024-03-06T07:21:45.117436+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.920205+0000","last_change":"2024-03-06T07:21:42.914001+0000","last_active":"2024-03-06T07:21:44.920205+0000","last_peered":"2024-03-06T07:21:44.920205+0000","last_clean":"2024-03-06T07:21:44.920205+0000","last_became_active":"2024-03-06T07:21:42.913707+0000","last_became_peered":"2024-03-06T07:21:42.913707+0000","last_unstale":"2024-03-06T07:21:44.920205+0000","last_undegraded":"2024-03-06T07:21:44.920205+0000","last_fullsized":"2024-03-06T07:21:44.920205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:37.811398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181747+0000","last_change":"2024-03-06T07:21:43.928109+0000","last_active":"2024-03-06T07:21:45.181747+0000","last_peered":"2024-03-06T07:21:45.181747+0000","last_clean":"2024-03-06T07:21:45.181747+0000","last_became_active":"2024-03-06T07:21:43.927816+0000","last_became_peered":"2024-03-06T07:21:43.927816+0000","last_unstale":"2024-03-06T07:21:45.181747+0000","last_undegraded":"2024-03-06T07:21:45.181747+0000","last_fullsized":"2024-03-06T07:21:45.181747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.910277+0000","last_change":"2024-03-06T07:21:42.909376+0000","last_active":"2024-03-06T07:21:44.910277+0000","last_peered":"2024-03-06T07:21:44.910277+0000","last_clean":"2024-03-06T07:21:44.910277+0000","last_became_active":"2024-03-06T07:21:42.909101+0000","last_became_peered":"2024-03-06T07:21:42.909101+0000","last_unstale":"2024-03-06T07:21:44.910277+0000","last_undegraded":"2024-03-06T07:21:44.910277+0000","last_fullsized":"2024-03-06T07:21:44.910277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:37.646398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913535+0000","last_change":"2024-03-06T07:21:43.925791+0000","last_active":"2024-03-06T07:21:44.913535+0000","last_peered":"2024-03-06T07:21:44.913535+0000","last_clean":"2024-03-06T07:21:44.913535+0000","last_became_active":"2024-03-06T07:21:43.925489+0000","last_became_peered":"2024-03-06T07:21:43.925489+0000","last_unstale":"2024-03-06T07:21:44.913535+0000","last_undegraded":"2024-03-06T07:21:44.913535+0000","last_fullsized":"2024-03-06T07:21:44.913535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913318+0000","last_change":"2024-03-06T07:21:42.912376+0000","last_active":"2024-03-06T07:21:44.913318+0000","last_peered":"2024-03-06T07:21:44.913318+0000","last_clean":"2024-03-06T07:21:44.913318+0000","last_became_active":"2024-03-06T07:21:42.912102+0000","last_became_peered":"2024-03-06T07:21:42.912102+0000","last_unstale":"2024-03-06T07:21:44.913318+0000","last_undegraded":"2024-03-06T07:21:44.913318+0000","last_fullsized":"2024-03-06T07:21:44.913318+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181871+0000","last_change":"2024-03-06T07:21:43.914898+0000","last_active":"2024-03-06T07:21:45.181871+0000","last_peered":"2024-03-06T07:21:45.181871+0000","last_clean":"2024-03-06T07:21:45.181871+0000","last_became_active":"2024-03-06T07:21:43.914368+0000","last_became_peered":"2024-03-06T07:21:43.914368+0000","last_unstale":"2024-03-06T07:21:45.181871+0000","last_undegraded":"2024-03-06T07:21:45.181871+0000","last_fullsized":"2024-03-06T07:21:45.181871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:32:44.975253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913252+0000","last_change":"2024-03-06T07:21:42.913064+0000","last_active":"2024-03-06T07:21:44.913252+0000","last_peered":"2024-03-06T07:21:44.913252+0000","last_clean":"2024-03-06T07:21:44.913252+0000","last_became_active":"2024-03-06T07:21:42.912792+0000","last_became_peered":"2024-03-06T07:21:42.912792+0000","last_unstale":"2024-03-06T07:21:44.913252+0000","last_undegraded":"2024-03-06T07:21:44.913252+0000","last_fullsized":"2024-03-06T07:21:44.913252+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:13.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182299+0000","last_change":"2024-03-06T07:21:43.912688+0000","last_active":"2024-03-06T07:21:45.182299+0000","last_peered":"2024-03-06T07:21:45.182299+0000","last_clean":"2024-03-06T07:21:45.182299+0000","last_became_active":"2024-03-06T07:21:43.912441+0000","last_became_peered":"2024-03-06T07:21:43.912441+0000","last_unstale":"2024-03-06T07:21:45.182299+0000","last_undegraded":"2024-03-06T07:21:45.182299+0000","last_fullsized":"2024-03-06T07:21:45.182299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.181925+0000","last_change":"2024-03-06T07:21:42.911252+0000","last_active":"2024-03-06T07:21:45.181925+0000","last_peered":"2024-03-06T07:21:45.181925+0000","last_clean":"2024-03-06T07:21:45.181925+0000","last_became_active":"2024-03-06T07:21:42.910758+0000","last_became_peered":"2024-03-06T07:21:42.910758+0000","last_unstale":"2024-03-06T07:21:45.181925+0000","last_undegraded":"2024-03-06T07:21:45.181925+0000","last_fullsized":"2024-03-06T07:21:45.181925+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913333+0000","last_change":"2024-03-06T07:21:43.917113+0000","last_active":"2024-03-06T07:21:44.913333+0000","last_peered":"2024-03-06T07:21:44.913333+0000","last_clean":"2024-03-06T07:21:44.913333+0000","last_became_active":"2024-03-06T07:21:43.915472+0000","last_became_peered":"2024-03-06T07:21:43.915472+0000","last_unstale":"2024-03-06T07:21:44.913333+0000","last_undegraded":"2024-03-06T07:21:44.913333+0000","last_fullsized":"2024-03-06T07:21:44.913333+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117220+0000","last_change":"2024-03-06T07:21:42.916636+0000","last_active":"2024-03-06T07:21:45.117220+0000","last_peered":"2024-03-06T07:21:45.117220+0000","last_clean":"2024-03-06T07:21:45.117220+0000","last_became_active":"2024-03-06T07:21:42.916345+0000","last_became_peered":"2024-03-06T07:21:42.916345+0000","last_unstale":"2024-03-06T07:21:45.117220+0000","last_undegraded":"2024-03-06T07:21:45.117220+0000","last_fullsized":"2024-03-06T07:21:45.117220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:59.475185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919901+0000","last_change":"2024-03-06T07:21:43.921341+0000","last_active":"2024-03-06T07:21:44.919901+0000","last_peered":"2024-03-06T07:21:44.919901+0000","last_clean":"2024-03-06T07:21:44.919901+0000","last_became_active":"2024-03-06T07:21:43.921062+0000","last_became_peered":"2024-03-06T07:21:43.921062+0000","last_unstale":"2024-03-06T07:21:44.919901+0000","last_undegraded":"2024-03-06T07:21:44.919901+0000","last_fullsized":"2024-03-06T07:21:44.919901+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:15.340198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919549+0000","last_change":"2024-03-06T07:21:42.912294+0000","last_active":"2024-03-06T07:21:44.919549+0000","last_peered":"2024-03-06T07:21:44.919549+0000","last_clean":"2024-03-06T07:21:44.919549+0000","last_became_active":"2024-03-06T07:21:42.912023+0000","last_became_peered":"2024-03-06T07:21:42.912023+0000","last_unstale":"2024-03-06T07:21:44.919549+0000","last_undegraded":"2024-03-06T07:21:44.919549+0000","last_fullsized":"2024-03-06T07:21:44.919549+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182129+0000","last_change":"2024-03-06T07:21:43.905564+0000","last_active":"2024-03-06T07:21:45.182129+0000","last_peered":"2024-03-06T07:21:45.182129+0000","last_clean":"2024-03-06T07:21:45.182129+0000","last_became_active":"2024-03-06T07:21:43.905280+0000","last_became_peered":"2024-03-06T07:21:43.905280+0000","last_unstale":"2024-03-06T07:21:45.182129+0000","last_undegraded":"2024-03-06T07:21:45.182129+0000","last_fullsized":"2024-03-06T07:21:45.182129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.909833+0000","last_change":"2024-03-06T07:21:42.908701+0000","last_active":"2024-03-06T07:21:44.909833+0000","last_peered":"2024-03-06T07:21:44.909833+0000","last_clean":"2024-03-06T07:21:44.909833+0000","last_became_active":"2024-03-06T07:21:42.908429+0000","last_became_peered":"2024-03-06T07:21:42.908429+0000","last_unstale":"2024-03-06T07:21:44.909833+0000","last_undegraded":"2024-03-06T07:21:44.909833+0000","last_fullsized":"2024-03-06T07:21:44.909833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:55.863684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182397+0000","last_change":"2024-03-06T07:21:43.909104+0000","last_active":"2024-03-06T07:21:45.182397+0000","last_peered":"2024-03-06T07:21:45.182397+0000","last_clean":"2024-03-06T07:21:45.182397+0000","last_became_active":"2024-03-06T07:21:43.908810+0000","last_became_peered":"2024-03-06T07:21:43.908810+0000","last_unstale":"2024-03-06T07:21:45.182397+0000","last_undegraded":"2024-03-06T07:21:45.182397+0000","last_fullsized":"2024-03-06T07:21:45.182397+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:14.273650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913462+0000","last_change":"2024-03-06T07:21:42.914080+0000","last_active":"2024-03-06T07:21:44.913462+0000","last_peered":"2024-03-06T07:21:44.913462+0000","last_clean":"2024-03-06T07:21:44.913462+0000","last_became_active":"2024-03-06T07:21:42.913818+0000","last_became_peered":"2024-03-06T07:21:42.913818+0000","last_unstale":"2024-03-06T07:21:44.913462+0000","last_undegraded":"2024-03-06T07:21:44.913462+0000","last_fullsized":"2024-03-06T07:21:44.913462+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:28.725020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182047+0000","last_change":"2024-03-06T07:21:43.917579+0000","last_active":"2024-03-06T07:21:45.182047+0000","last_peered":"2024-03-06T07:21:45.182047+0000","last_clean":"2024-03-06T07:21:45.182047+0000","last_became_active":"2024-03-06T07:21:43.916663+0000","last_became_peered":"2024-03-06T07:21:43.916663+0000","last_unstale":"2024-03-06T07:21:45.182047+0000","last_undegraded":"2024-03-06T07:21:45.182047+0000","last_fullsized":"2024-03-06T07:21:45.182047+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:43.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.922291+0000","last_change":"2024-03-06T07:21:42.912139+0000","last_active":"2024-03-06T07:21:44.922291+0000","last_peered":"2024-03-06T07:21:44.922291+0000","last_clean":"2024-03-06T07:21:44.922291+0000","last_became_active":"2024-03-06T07:21:42.911573+0000","last_became_peered":"2024-03-06T07:21:42.911573+0000","last_unstale":"2024-03-06T07:21:44.922291+0000","last_undegraded":"2024-03-06T07:21:44.922291+0000","last_fullsized":"2024-03-06T07:21:44.922291+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117500+0000","last_change":"2024-03-06T07:21:43.912734+0000","last_active":"2024-03-06T07:21:45.117500+0000","last_peered":"2024-03-06T07:21:45.117500+0000","last_clean":"2024-03-06T07:21:45.117500+0000","last_became_active":"2024-03-06T07:21:43.912449+0000","last_became_peered":"2024-03-06T07:21:43.912449+0000","last_unstale":"2024-03-06T07:21:45.117500+0000","last_undegraded":"2024-03-06T07:21:45.117500+0000","last_fullsized":"2024-03-06T07:21:45.117500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919771+0000","last_change":"2024-03-06T07:21:42.907567+0000","last_active":"2024-03-06T07:21:44.919771+0000","last_peered":"2024-03-06T07:21:44.919771+0000","last_clean":"2024-03-06T07:21:44.919771+0000","last_became_active":"2024-03-06T07:21:42.907271+0000","last_became_peered":"2024-03-06T07:21:42.907271+0000","last_unstale":"2024-03-06T07:21:44.919771+0000","last_undegraded":"2024-03-06T07:21:44.919771+0000","last_fullsized":"2024-03-06T07:21:44.919771+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919838+0000","last_change":"2024-03-06T07:21:43.921555+0000","last_active":"2024-03-06T07:21:44.919838+0000","last_peered":"2024-03-06T07:21:44.919838+0000","last_clean":"2024-03-06T07:21:44.919838+0000","last_became_active":"2024-03-06T07:21:43.921242+0000","last_became_peered":"2024-03-06T07:21:43.921242+0000","last_unstale":"2024-03-06T07:21:44.919838+0000","last_undegraded":"2024-03-06T07:21:44.919838+0000","last_fullsized":"2024-03-06T07:21:44.919838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:31.942191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117448+0000","last_change":"2024-03-06T07:21:42.915454+0000","last_active":"2024-03-06T07:21:45.117448+0000","last_peered":"2024-03-06T07:21:45.117448+0000","last_clean":"2024-03-06T07:21:45.117448+0000","last_became_active":"2024-03-06T07:21:42.915127+0000","last_became_peered":"2024-03-06T07:21:42.915127+0000","last_unstale":"2024-03-06T07:21:45.117448+0000","last_undegraded":"2024-03-06T07:21:45.117448+0000","last_fullsized":"2024-03-06T07:21:45.117448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:14.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182495+0000","last_change":"2024-03-06T07:21:43.919585+0000","last_active":"2024-03-06T07:21:45.182495+0000","last_peered":"2024-03-06T07:21:45.182495+0000","last_clean":"2024-03-06T07:21:45.182495+0000","last_became_active":"2024-03-06T07:21:43.919255+0000","last_became_peered":"2024-03-06T07:21:43.919255+0000","last_unstale":"2024-03-06T07:21:45.182495+0000","last_undegraded":"2024-03-06T07:21:45.182495+0000","last_fullsized":"2024-03-06T07:21:45.182495+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117317+0000","last_change":"2024-03-06T07:21:42.912512+0000","last_active":"2024-03-06T07:21:45.117317+0000","last_peered":"2024-03-06T07:21:45.117317+0000","last_clean":"2024-03-06T07:21:45.117317+0000","last_became_active":"2024-03-06T07:21:42.912235+0000","last_became_peered":"2024-03-06T07:21:42.912235+0000","last_unstale":"2024-03-06T07:21:45.117317+0000","last_undegraded":"2024-03-06T07:21:45.117317+0000","last_fullsized":"2024-03-06T07:21:45.117317+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.427916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919846+0000","last_change":"2024-03-06T07:21:43.933046+0000","last_active":"2024-03-06T07:21:44.919846+0000","last_peered":"2024-03-06T07:21:44.919846+0000","last_clean":"2024-03-06T07:21:44.919846+0000","last_became_active":"2024-03-06T07:21:43.932764+0000","last_became_peered":"2024-03-06T07:21:43.932764+0000","last_unstale":"2024-03-06T07:21:44.919846+0000","last_undegraded":"2024-03-06T07:21:44.919846+0000","last_fullsized":"2024-03-06T07:21:44.919846+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:47.292953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.913411+0000","last_change":"2024-03-06T07:21:42.912274+0000","last_active":"2024-03-06T07:21:44.913411+0000","last_peered":"2024-03-06T07:21:44.913411+0000","last_clean":"2024-03-06T07:21:44.913411+0000","last_became_active":"2024-03-06T07:21:42.912000+0000","last_became_peered":"2024-03-06T07:21:42.912000+0000","last_unstale":"2024-03-06T07:21:44.913411+0000","last_undegraded":"2024-03-06T07:21:44.913411+0000","last_fullsized":"2024-03-06T07:21:44.913411+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:00.006841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.117661+0000","last_change":"2024-03-06T07:21:43.906537+0000","last_active":"2024-03-06T07:21:45.117661+0000","last_peered":"2024-03-06T07:21:45.117661+0000","last_clean":"2024-03-06T07:21:45.117661+0000","last_became_active":"2024-03-06T07:21:43.906267+0000","last_became_peered":"2024-03-06T07:21:43.906267+0000","last_unstale":"2024-03-06T07:21:45.117661+0000","last_undegraded":"2024-03-06T07:21:45.117661+0000","last_fullsized":"2024-03-06T07:21:45.117661+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:45.221948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:45.182491+0000","last_change":"2024-03-06T07:21:42.912363+0000","last_active":"2024-03-06T07:21:45.182491+0000","last_peered":"2024-03-06T07:21:45.182491+0000","last_clean":"2024-03-06T07:21:45.182491+0000","last_became_active":"2024-03-06T07:21:42.912097+0000","last_became_peered":"2024-03-06T07:21:42.912097+0000","last_unstale":"2024-03-06T07:21:45.182491+0000","last_undegraded":"2024-03-06T07:21:45.182491+0000","last_fullsized":"2024-03-06T07:21:45.182491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:13.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.912857+0000","last_change":"2024-03-06T07:21:43.913294+0000","last_active":"2024-03-06T07:21:44.912857+0000","last_peered":"2024-03-06T07:21:44.912857+0000","last_clean":"2024-03-06T07:21:44.912857+0000","last_became_active":"2024-03-06T07:21:43.913017+0000","last_became_peered":"2024-03-06T07:21:43.913017+0000","last_unstale":"2024-03-06T07:21:44.912857+0000","last_undegraded":"2024-03-06T07:21:44.912857+0000","last_fullsized":"2024-03-06T07:21:44.912857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:26.287336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.919943+0000","last_change":"2024-03-06T07:21:42.912585+0000","last_active":"2024-03-06T07:21:44.919943+0000","last_peered":"2024-03-06T07:21:44.919943+0000","last_clean":"2024-03-06T07:21:44.919943+0000","last_became_active":"2024-03-06T07:21:42.911650+0000","last_became_peered":"2024-03-06T07:21:42.911650+0000","last_unstale":"2024-03-06T07:21:44.919943+0000","last_undegraded":"2024-03-06T07:21:44.919943+0000","last_fullsized":"2024-03-06T07:21:44.919943+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:42.887622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-06T07:21:44.921806+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:21:44.921806+0000","last_peered":"2024-03-06T07:21:44.921806+0000","last_clean":"2024-03-06T07:21:44.921806+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:21:44.921806+0000","last_undegraded":"2024-03-06T07:21:44.921806+0000","last_fullsized":"2024-03-06T07:21:44.921806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":862894,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":614400,"data_stored":273054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":272402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":272984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":274036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":618496,"data_stored":274014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":614400,"data_stored":272562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":679936,"data_stored":862826,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:21:52.995 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:21:52.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:21:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.144+0000 7fb538c15640 1 Processor -- start 2024-03-06T07:21:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.144+0000 7fb538c15640 1 -- start start 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb538c15640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 0x7fb534155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb538c15640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340c8960 0x7fb5340c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb538c15640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53414f630 con 0x7fb5340c5920 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb538c15640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5340c7470 con 0x7fb5340c8960 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb538c15640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340c7f60 con 0x7fb53414f250 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340c8960 0x7fb5340c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb532d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb5340c5920 0x7fb534158aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49896/0 (socket says 172.21.15.17:49896) 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5340c8960 0x7fb5340c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56490/0 (socket says 172.21.15.17:56490) 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb532d9d640 1 -- 172.21.15.17:0/3195651556 learned_addr learned my addr 172.21.15.17:0/3195651556 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb53259c640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 0x7fb534155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 -- 172.21.15.17:0/3195651556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 msgr2=0x7fb534155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 0x7fb534155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 -- 172.21.15.17:0/3195651556 >> v1:172.21.15.17:6789/0 conn(0x7fb5340c5920 legacy=0x7fb534158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb531d9b640 1 -- 172.21.15.17:0/3195651556 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340c76c0 con 0x7fb5340c8960 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb53159a640 1 -- 172.21.15.17:0/3195651556 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125024238 0 0) 0x7fb53414f630 con 0x7fb5340c5920 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.145+0000 7fb53259c640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 0x7fb534155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.146+0000 7fb531d9b640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c8960 0x7fb5340c53e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097a0 tx=0x7fb52802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98330c5b9e16cc9c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.146+0000 7fb53159a640 1 -- 172.21.15.17:0/3195651556 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528038070 con 0x7fb5340c8960 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.146+0000 7fb53159a640 1 -- 172.21.15.17:0/3195651556 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb528031700 con 0x7fb5340c8960 2024-03-06T07:21:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.146+0000 7fb53159a640 1 -- 172.21.15.17:0/3195651556 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528031ad0 con 0x7fb5340c8960 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 -- 172.21.15.17:0/3195651556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c8960 msgr2=0x7fb5340c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c8960 0x7fb5340c53e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097a0 tx=0x7fb52802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 -- 172.21.15.17:0/3195651556 shutdown_connections 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c8960 0x7fb5340c53e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 --2- 172.21.15.17:0/3195651556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb53414f250 0x7fb534155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 -- 172.21.15.17:0/3195651556 >> 172.21.15.17:0/3195651556 conn(0x7fb5340ac610 msgr2=0x7fb5340aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 -- 172.21.15.17:0/3195651556 shutdown_connections 2024-03-06T07:21:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.147+0000 7fb538c15640 1 -- 172.21.15.17:0/3195651556 wait complete. 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.148+0000 7fb538c15640 1 Processor -- start 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.148+0000 7fb538c15640 1 -- start start 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5340c8960 0x7fb5340afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 0x7fb53414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340be310 con 0x7fb5340c5920 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5340c7c80 con 0x7fb53414f250 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb538c15640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5341591b0 con 0x7fb5340c8960 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56506/0 (socket says 172.21.15.17:56506) 2024-03-06T07:21:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 -- 172.21.15.17:0/2754750634 learned_addr learned my addr 172.21.15.17:0/2754750634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb531d9b640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5340c8960 0x7fb5340afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb532d9d640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 0x7fb53414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 -- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5340c8960 msgr2=0x7fb5340afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5340c8960 0x7fb5340afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 -- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 msgr2=0x7fb53414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 0x7fb53414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 -- 172.21.15.17:0/2754750634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340af5c0 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb532d9d640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 0x7fb53414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.149+0000 7fb53259c640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00c7f0 tx=0x7fb52c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.150+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c015070 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.150+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52c007820 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.150+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52c007bd0 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.150+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5341ec010 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.150+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5341ec5d0 con 0x7fb5340c5920 2024-03-06T07:21:53.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.151+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb534159580 con 0x7fb5340c5920 2024-03-06T07:21:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.151+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb52c018070 con 0x7fb5340c5920 2024-03-06T07:21:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.151+0000 7fb522ffd640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.152+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb52c08a3f0 con 0x7fb5340c5920 2024-03-06T07:21:53.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.152+0000 7fb531d9b640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.152+0000 7fb531d9b640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 0x7fb51406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb528035000 tx=0x7fb528008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.159+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb534159580 con 0x7fb5340c5920 2024-03-06T07:21:53.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.410+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb5341ebe00 con 0x7fb5340c5920 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.412+0000 7fb522ffd640 1 -- 172.21.15.17:0/2754750634 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb52c058af0 con 0x7fb5340c5920 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:21:53.415 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:21:53.416 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 msgr2=0x7fb51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 0x7fb51406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb528035000 tx=0x7fb528008030 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 msgr2=0x7fb534154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00c7f0 tx=0x7fb52c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 shutdown_connections 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb51406b4d0 0x7fb51406d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb53414f250 0x7fb53414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5340c8960 0x7fb5340afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 --2- 172.21.15.17:0/2754750634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5340c5920 0x7fb534154900 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 >> 172.21.15.17:0/2754750634 conn(0x7fb5340ac610 msgr2=0x7fb5340c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 shutdown_connections 2024-03-06T07:21:53.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.415+0000 7fb538c15640 1 -- 172.21.15.17:0/2754750634 wait complete. 2024-03-06T07:21:53.432 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.530+0000 7f2c0e0df640 1 Processor -- start 2024-03-06T07:21:53.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.530+0000 7f2c0e0df640 1 -- start start 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c0e0df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c081550c0 0x7f2c081554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c0e0df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2c081559e0 0x7f2c0814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c0e0df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c080aa660 con 0x7f2c080aa180 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c0e0df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c080c4610 con 0x7f2c081550c0 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c0e0df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c080aa940 con 0x7f2c081559e0 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c081550c0 0x7f2c081554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2c081550c0 0x7f2c081554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56522/0 (socket says 172.21.15.17:56522) 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 -- 172.21.15.17:0/1943776271 learned_addr learned my addr 172.21.15.17:0/1943776271 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c06ffd640 1 --2- 172.21.15.17:0/1943776271 >> v2:172.21.15.163:3300/0 conn(0x7f2c081559e0 0x7f2c0814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 -- 172.21.15.17:0/1943776271 >> v2:172.21.15.163:3300/0 conn(0x7f2c081559e0 msgr2=0x7f2c0814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 --2- 172.21.15.17:0/1943776271 >> v2:172.21.15.163:3300/0 conn(0x7f2c081559e0 0x7f2c0814ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 -- 172.21.15.17:0/1943776271 >> v1:172.21.15.163:6789/0 conn(0x7f2c080aa180 legacy=0x7f2c080c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.531+0000 7f2c077fe640 1 -- 172.21.15.17:0/1943776271 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080c4860 con 0x7f2c081550c0 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.532+0000 7f2c077fe640 1 --2- 172.21.15.17:0/1943776271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081550c0 0x7f2c081554a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2c080abe90 tx=0x7f2bf002de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f30a4d4686ad1c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.532+0000 7f2c067fc640 1 -- 172.21.15.17:0/1943776271 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0034070 con 0x7f2c081550c0 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.532+0000 7f2c067fc640 1 -- 172.21.15.17:0/1943776271 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf002ebd0 con 0x7f2c081550c0 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.532+0000 7f2c067fc640 1 -- 172.21.15.17:0/1943776271 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0038400 con 0x7f2c081550c0 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 -- 172.21.15.17:0/1943776271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081550c0 msgr2=0x7f2c081554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/1943776271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081550c0 0x7f2c081554a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2c080abe90 tx=0x7f2bf002de20 comp rx=0 tx=0).stop 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 -- 172.21.15.17:0/1943776271 shutdown_connections 2024-03-06T07:21:53.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/1943776271 >> v2:172.21.15.163:3300/0 conn(0x7f2c081559e0 0x7f2c0814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/1943776271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081550c0 0x7f2c081554a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 -- 172.21.15.17:0/1943776271 >> 172.21.15.17:0/1943776271 conn(0x7f2c080c6710 msgr2=0x7f2c080c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 -- 172.21.15.17:0/1943776271 shutdown_connections 2024-03-06T07:21:53.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.533+0000 7f2c0e0df640 1 -- 172.21.15.17:0/1943776271 wait complete. 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.535+0000 7f2c0e0df640 1 Processor -- start 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.535+0000 7f2c0e0df640 1 -- start start 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c080aa180 0x7f2c081ddf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 0x7f2c081de810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0814f140 con 0x7f2c081559e0 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c080c1560 con 0x7f2c081550c0 2024-03-06T07:21:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c0e0df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0814f2c0 con 0x7f2c080aa180 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56530/0 (socket says 172.21.15.17:56530) 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 -- 172.21.15.17:0/684246963 learned_addr learned my addr 172.21.15.17:0/684246963 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c06ffd640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 0x7f2c081de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c077fe640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c080aa180 0x7f2c081ddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 -- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c080aa180 msgr2=0x7f2c081ddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c080aa180 0x7f2c081ddf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 -- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 msgr2=0x7f2c081de810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 0x7f2c081de810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c07fff640 1 -- 172.21.15.17:0/684246963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0806d090 con 0x7f2c081559e0 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.536+0000 7f2c06ffd640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 0x7f2c081de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2c07fff640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c760 tx=0x7f2c0000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003b70 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00003d10 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0806e1f0 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000aa90 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.537+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c081de440 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.538+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0806d630 con 0x7f2c081559e0 2024-03-06T07:21:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.538+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c0000ac30 con 0x7f2c081559e0 2024-03-06T07:21:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.539+0000 7f2bfffff640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 0x7f2be806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.539+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2c0008a300 con 0x7f2c081559e0 2024-03-06T07:21:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.539+0000 7f2c077fe640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 0x7f2be806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.540+0000 7f2c077fe640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 0x7f2be806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0039000 tx=0x7f2bf0034000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:53.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.547+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c00058f30 con 0x7f2c081559e0 2024-03-06T07:21:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.765+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c0804f330 con 0x7f2c081559e0 2024-03-06T07:21:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.765+0000 7f2bfffff640 1 -- 172.21.15.17:0/684246963 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f2c00058830 con 0x7f2c081559e0 2024-03-06T07:21:53.768 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:53.768 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5769,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:21:46.263408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5772":{"gid":5772,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1002585815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1002585815},{"type":"v1","addr":"172.21.15.17:6835","nonce":1002585815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5772,"qdb_cluster":[5772]},"id":3}]} 2024-03-06T07:21:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 31 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 msgr2=0x7f2be806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 0x7f2be806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0039000 tx=0x7f2bf0034000 comp rx=0 tx=0).stop 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 msgr2=0x7f2c081ded50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c760 tx=0x7f2c0000cc20 comp rx=0 tx=0).stop 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 shutdown_connections 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be806b4d0 0x7f2be806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c081559e0 0x7f2c081ded50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c081550c0 0x7f2c081de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 --2- 172.21.15.17:0/684246963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c080aa180 0x7f2c081ddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 >> 172.21.15.17:0/684246963 conn(0x7f2c080c6710 msgr2=0x7f2c0814bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 shutdown_connections 2024-03-06T07:21:53.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:53.767+0000 7f2c0e0df640 1 -- 172.21.15.17:0/684246963 wait complete. 2024-03-06T07:21:54.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:21:43.270845+0000', 'modified': '2024-03-06T07:21:46.263408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5772}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5772': {'gid': 5772, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/1002585815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1002585815}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1002585815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5772, 'qdb_cluster': [5772]} 2024-03-06T07:21:54.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:21:54.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:21:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.284+0000 7fa06915c640 1 Processor -- start 2024-03-06T07:21:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.284+0000 7fa06915c640 1 -- start start 2024-03-06T07:21:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06915c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0640ab860 0x7fa0640c4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06915c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0640ac2b0 0x7fa0641588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06915c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0641553c0 con 0x7fa06406c240 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06915c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0641556a0 con 0x7fa0640ab860 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06915c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa064155980 con 0x7fa0640ac2b0 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa062d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa06406c240 0x7fa0640ab150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49902/0 (socket says 172.21.15.17:49902) 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa062d9d640 1 -- 172.21.15.17:0/1570451169 learned_addr learned my addr 172.21.15.17:0/1570451169 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06359e640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0640ac2b0 0x7fa0641588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa06259c640 1 --2- 172.21.15.17:0/1570451169 >> v2:172.21.15.163:3300/0 conn(0x7fa0640ab860 0x7fa0640c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080942222 0 0) 0x7fa0641553c0 con 0x7fa06406c240 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa050003520 con 0x7fa06406c240 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818826832 0 0) 0x7fa050003520 con 0x7fa06406c240 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0640ac2b0 msgr2=0x7fa0641588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0640ac2b0 0x7fa0641588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 >> v2:172.21.15.163:3300/0 conn(0x7fa0640ab860 msgr2=0x7fa0640c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 --2- 172.21.15.17:0/1570451169 >> v2:172.21.15.163:3300/0 conn(0x7fa0640ab860 0x7fa0640c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064155c60 con 0x7fa06406c240 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.285+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fa054001670 con 0x7fa06406c240 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 >> v1:172.21.15.17:6789/0 conn(0x7fa06406c240 legacy=0x7fa0640ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa050003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa050004800 0x7fa050004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0500051c0 0x7fa05000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641556a0 con 0x7fa0500039b0 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064155980 con 0x7fa050004800 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05000bf30 con 0x7fa0500051c0 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa06359e640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0640ac2b0 0x7fa0641588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa050003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa06259c640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa050004800 0x7fa050004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa06359e640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0500051c0 0x7fa05000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 -- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0500051c0 msgr2=0x7fa05000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0500051c0 0x7fa05000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 -- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa050004800 msgr2=0x7fa050004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa050004800 0x7fa050004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa062d9d640 1 -- 172.21.15.17:0/1570451169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa050003520 con 0x7fa0500039b0 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.286+0000 7fa06259c640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa050004800 0x7fa050004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa050003dd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c240 tx=0x7fa050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89db4b0a10cc27d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054007d30 con 0x7fa0500039b0 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa054030070 con 0x7fa0500039b0 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa061d9b640 1 -- 172.21.15.17:0/1570451169 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05402cd60 con 0x7fa0500039b0 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 -- 172.21.15.17:0/1570451169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 msgr2=0x7fa050003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 --2- 172.21.15.17:0/1570451169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa050003dd0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c240 tx=0x7fa050005830 comp rx=0 tx=0).stop 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06359e640 1 -- 172.21.15.17:0/1570451169 reap_dead start 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 -- 172.21.15.17:0/1570451169 shutdown_connections 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 -- 172.21.15.17:0/1570451169 >> 172.21.15.17:0/1570451169 conn(0x7fa0640c7670 msgr2=0x7fa0640c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 -- 172.21.15.17:0/1570451169 shutdown_connections 2024-03-06T07:21:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.287+0000 7fa06915c640 1 -- 172.21.15.17:0/1570451169 wait complete. 2024-03-06T07:21:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.289+0000 7fa06915c640 1 Processor -- start 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.289+0000 7fa06915c640 1 -- start start 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.289+0000 7fa06915c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 0x7fa064146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa064146db0 0x7fa0641572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640c4830 con 0x7fa0500039b0 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064156180 con 0x7fa064071f10 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06406c620 con 0x7fa064146db0 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56550/0 (socket says 172.21.15.17:56550) 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 -- 172.21.15.17:0/546694203 learned_addr learned my addr 172.21.15.17:0/546694203 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06359e640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa064146db0 0x7fa0641572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06259c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 0x7fa064146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 -- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa064146db0 msgr2=0x7fa0641572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa064146db0 0x7fa0641572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 -- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 msgr2=0x7fa064146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 0x7fa064146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 -- 172.21.15.17:0/546694203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0640c6230 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06259c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 0x7fa064146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa062d9d640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c240 tx=0x7fa0641552b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa05b7fe640 1 -- 172.21.15.17:0/546694203 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054001c70 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa05b7fe640 1 -- 172.21.15.17:0/546694203 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa054004bf0 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0641567d0 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa05b7fe640 1 -- 172.21.15.17:0/546694203 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054007d30 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.290+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa064156d90 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.292+0000 7fa05b7fe640 1 -- 172.21.15.17:0/546694203 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa054047020 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.292+0000 7fa05b7fe640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 0x7fa04406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.292+0000 7fa05b7fe640 1 -- 172.21.15.17:0/546694203 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa0540ab440 con 0x7fa0500039b0 2024-03-06T07:21:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.293+0000 7fa06259c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 0x7fa04406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.293+0000 7fa06be13740 1 Processor -- start 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.293+0000 7fa06259c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 0x7fa04406d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0640c4080 tx=0x7fa04c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.293+0000 7fa06be13740 1 -- start start 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7285a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72864b0 0x557ff7282680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff71ec6b0 con 0x557ff7288330 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff7195a60 con 0x557ff7287f50 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ff71e8750 con 0x557ff72864b0 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56556/0 (socket says 172.21.15.17:56556) 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 -- 172.21.15.17:0/2536055954 learned_addr learned my addr 172.21.15.17:0/2536055954 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06359e640 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72864b0 0x557ff7282680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa062d9d640 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7285a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 -- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72864b0 msgr2=0x557ff7282680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72864b0 0x557ff7282680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 -- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 msgr2=0x557ff7285a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7285a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.294+0000 7fa06259c640 1 -- 172.21.15.17:0/2536055954 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ff71b3f30 con 0x557ff7288330 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06259c640 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa04c013c50 tx=0x7fa04c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e28114c98707c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa058ff9640 1 -- 172.21.15.17:0/2536055954 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c013e80 con 0x557ff7288330 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa058ff9640 1 -- 172.21.15.17:0/2536055954 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c028070 con 0x557ff7288330 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa058ff9640 1 -- 172.21.15.17:0/2536055954 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c023640 con 0x557ff7288330 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 -- 172.21.15.17:0/2536055954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 msgr2=0x557ff7285f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa04c013c50 tx=0x7fa04c016000 comp rx=0 tx=0).stop 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 -- 172.21.15.17:0/2536055954 shutdown_connections 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72864b0 0x557ff7282680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7285f70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 --2- 172.21.15.17:0/2536055954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7285a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 -- 172.21.15.17:0/2536055954 >> 172.21.15.17:0/2536055954 conn(0x557ff72847d0 msgr2=0x557ff72a93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 -- 172.21.15.17:0/2536055954 shutdown_connections 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.295+0000 7fa06be13740 1 -- 172.21.15.17:0/2536055954 wait complete. 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.296+0000 7fa06be13740 1 -- start start 2024-03-06T07:21:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.296+0000 7fa06be13740 1 Processor -- start 2024-03-06T07:21:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff72d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff72aa230 con 0x557ff7287f50 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff71a8ea0 con 0x557ff7288330 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ff72a9c70 con 0x557ff72ad3b0 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06be13740 10 client.0 _ll_register_callbacks cb 0x557ff71a8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56572/0 (socket says 172.21.15.17:56572) 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 -- 172.21.15.17:0/1536721533 learned_addr learned my addr 172.21.15.17:0/1536721533 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06359e640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06259c640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff72d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 -- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 msgr2=0x557ff72a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72a4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 -- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 msgr2=0x557ff72d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff72d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 -- 172.21.15.17:0/1536721533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ff71a9270 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06359e640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa06259c640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff72d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a97b0 tx=0x7fa054079b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bf5999722d54a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054079cf0 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.297+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa054079e90 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa04c028070 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0540a9bd0 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 10 client.5928 fetch_fsmap learned FSMap version 31 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x557ff6f012d0 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c028070 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa054081d10 con 0x557ff7287f50 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 10 client.5928 fetch_fsmap finished waiting for FSMap version 31 2024-03-06T07:21:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 10 client.5928 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 20 client.5928 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72d7ec0 0x557ff72d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 4 client.5928 mds_command: new command op to 5772 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 --> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ff71f9bf0 con 0x557ff72d7ec0 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06259c640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72d7ec0 0x557ff72d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa06259c640 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72d7ec0 0x557ff72d82a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c31d55626afc41 server_cookie=6743f79b215b95f9 in_seq=0 out_seq=0 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.298+0000 7fa041ffb640 10 client.5928 ms_handle_connect on v2:172.21.15.17:6833/1002585815 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.299+0000 7fa041ffb640 1 -- 172.21.15.17:0/1536721533 <== mds.0 v2:172.21.15.17:6833/1002585815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557ff71f9bf0 con 0x557ff72d7ec0 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.299+0000 7fa041ffb640 10 client.5928 handle_command_reply: tid=0 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.299+0000 7fa06be13740 1 client.5928 shutdown 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.299+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 msgr2=0x557ff72a87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.299+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a97b0 tx=0x7fa054079b50 comp rx=0 tx=0).stop 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 shutdown_connections 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72d7ec0 0x557ff72d82a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff72d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa041ffb640 0 client.5928 ms_handle_reset on v2:172.21.15.17:6833/1002585815 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 --2- 172.21.15.17:0/1536721533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff72a87b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 >> 172.21.15.17:0/1536721533 conn(0x557ff72864b0 msgr2=0x557ff7281510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 shutdown_connections 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 1 -- 172.21.15.17:0/1536721533 wait complete. 2024-03-06T07:21:54.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.300+0000 7fa06be13740 20 client.5928 trim_cache size 0 max 16384 2024-03-06T07:21:54.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.316+0000 7fa06be13740 1 Processor -- start 2024-03-06T07:21:54.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.316+0000 7fa06be13740 1 -- start start 2024-03-06T07:21:54.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7281510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641f75c0 con 0x557ff7288330 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0640c5890 con 0x557ff7287f50 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641f6ff0 con 0x557ff72ad3b0 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56574/0 (socket says 172.21.15.17:56574) 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 -- 172.21.15.17:0/1572882645 learned_addr learned my addr 172.21.15.17:0/1572882645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06359e640 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa062d9d640 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7281510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 -- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 msgr2=0x557ff72c00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72c00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 -- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 msgr2=0x557ff7281510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7281510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 -- 172.21.15.17:0/1572882645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ff6fcb6b0 con 0x557ff7288330 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.317+0000 7fa06259c640 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa04c013c20 tx=0x7fa04c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1144d998b94042f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa043fff640 1 -- 172.21.15.17:0/1572882645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c03e070 con 0x557ff7288330 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa043fff640 1 -- 172.21.15.17:0/1572882645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c028040 con 0x557ff7288330 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa043fff640 1 -- 172.21.15.17:0/1572882645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c013ea0 con 0x557ff7288330 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 -- 172.21.15.17:0/1572882645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 msgr2=0x557ff7286890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa04c013c20 tx=0x7fa04c018000 comp rx=0 tx=0).stop 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 -- 172.21.15.17:0/1572882645 shutdown_connections 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff72c00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7288330 0x557ff7286890 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 --2- 172.21.15.17:0/1572882645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7287f50 0x557ff7281510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 -- 172.21.15.17:0/1572882645 >> 172.21.15.17:0/1572882645 conn(0x557ff72864b0 msgr2=0x557ff72a85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 -- 172.21.15.17:0/1572882645 shutdown_connections 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.318+0000 7fa06be13740 1 -- 172.21.15.17:0/1572882645 wait complete. 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 -- start start 2024-03-06T07:21:54.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 Processor -- start 2024-03-06T07:21:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff73333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff7281910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff72ae4f0 con 0x557ff7287f50 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ff72aa230 con 0x557ff7288330 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ff72d4d20 con 0x557ff72ad3b0 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.319+0000 7fa06be13740 10 client.0 _ll_register_callbacks cb 0x7fa0540793f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56588/0 (socket says 172.21.15.17:56588) 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 -- 172.21.15.17:0/416929338 learned_addr learned my addr 172.21.15.17:0/416929338 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06259c640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff73333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06359e640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff7281910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 -- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 msgr2=0x557ff7281910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff7281910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 -- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 msgr2=0x557ff73333a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff73333a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 -- 172.21.15.17:0/416929338 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ff72c76f0 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06259c640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff73333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06359e640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff7281910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa062d9d640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa054077fc0 tx=0x7fa0540a9710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f817312196cc1d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054079ef0 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa054078ee0 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa04c028040 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0540790e0 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06be13740 10 client.5934 fetch_fsmap learned FSMap version 31 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x557ff72c76f0 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.320+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c028040 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa05407eb00 con 0x557ff7287f50 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 10 client.5934 fetch_fsmap finished waiting for FSMap version 31 2024-03-06T07:21:54.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 10 client.5934 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 20 client.5934 populate_metadata read hostname 'smithi017' 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72acad0 0x557ff7330180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 4 client.5934 mds_command: new command op to 5772 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 --> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557ff71f3f40 con 0x557ff72acad0 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06259c640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72acad0 0x557ff7330180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa041ffb640 10 client.5934 ms_handle_connect on v2:172.21.15.17:6833/1002585815 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06259c640 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72acad0 0x557ff7330180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5efaa9a1b94456da server_cookie=12985ed5d8934096 in_seq=0 out_seq=0 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa041ffb640 1 -- 172.21.15.17:0/416929338 <== mds.0 v2:172.21.15.17:6833/1002585815 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x557ff71f3f40 con 0x557ff72acad0 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa041ffb640 10 client.5934 handle_command_reply: tid=0 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 client.5934 shutdown 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 msgr2=0x557ff7337490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa054077fc0 tx=0x7fa0540a9710 comp rx=0 tx=0).stop 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 shutdown_connections 2024-03-06T07:21:54.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:6833/1002585815,v1:172.21.15.17:6835/1002585815] conn(0x557ff72acad0 0x557ff7330180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557ff72ad3b0 0x557ff7281910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557ff7288330 0x557ff73333a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa041ffb640 0 client.5934 ms_handle_reset on v2:172.21.15.17:6833/1002585815 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 --2- 172.21.15.17:0/416929338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ff7287f50 0x557ff7337490 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.321+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 >> 172.21.15.17:0/416929338 conn(0x557ff72864b0 msgr2=0x557ff72de3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 shutdown_connections 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06be13740 1 -- 172.21.15.17:0/416929338 wait complete. 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06be13740 20 client.5934 trim_cache size 0 max 16384 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:21:54.331 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5772, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 10.418899661999999, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 31, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 53, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 51, 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 15.438071938 2024-03-06T07:21:54.332 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:21:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 msgr2=0x7fa04406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 0x7fa04406d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa0640c4080 tx=0x7fa04c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 msgr2=0x7fa0641ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa05400c240 tx=0x7fa0641552b0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 shutdown_connections 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa04406b500 0x7fa04406d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa064146db0 0x7fa0641572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa064071f10 0x7fa064146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 --2- 172.21.15.17:0/546694203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0500039b0 0x7fa0641ee8d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 >> 172.21.15.17:0/546694203 conn(0x7fa0640c7670 msgr2=0x7fa0640ab080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 shutdown_connections 2024-03-06T07:21:54.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.322+0000 7fa06915c640 1 -- 172.21.15.17:0/546694203 wait complete. 2024-03-06T07:21:54.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:21:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.436+0000 7fb10b33e640 1 Processor -- start 2024-03-06T07:21:54.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.436+0000 7fb10b33e640 1 -- start start 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb10b33e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb10414f360 0x7fb104155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb10b33e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb104158b60 0x7fb104158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb10b33e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040c73d0 con 0x7fb1040c88c0 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb10b33e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040c8dd0 con 0x7fb104158b60 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb10b33e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040be2e0 con 0x7fb10414f360 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb104158b60 0x7fb104158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb104158b60 0x7fb104158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56598/0 (socket says 172.21.15.17:56598) 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 -- 172.21.15.17:0/1287784011 learned_addr learned my addr 172.21.15.17:0/1287784011 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1090da640 1 --2- 172.21.15.17:0/1287784011 >> v2:172.21.15.163:3300/0 conn(0x7fb10414f360 0x7fb104155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 -- 172.21.15.17:0/1287784011 >> v2:172.21.15.163:3300/0 conn(0x7fb10414f360 msgr2=0x7fb104155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 --2- 172.21.15.17:0/1287784011 >> v2:172.21.15.163:3300/0 conn(0x7fb10414f360 0x7fb104155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.437+0000 7fb1098db640 1 -- 172.21.15.17:0/1287784011 >> v1:172.21.15.163:6789/0 conn(0x7fb1040c88c0 legacy=0x7fb1040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.438+0000 7fb1098db640 1 -- 172.21.15.17:0/1287784011 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040c7670 con 0x7fb104158b60 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.438+0000 7fb1098db640 1 --2- 172.21.15.17:0/1287784011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb104158b60 0x7fb104158ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb10408c840 tx=0x7fb0fc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a1266b8b3fe087 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.438+0000 7fb0fbfff640 1 -- 172.21.15.17:0/1287784011 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc02c9f0 con 0x7fb104158b60 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb0fbfff640 1 -- 172.21.15.17:0/1287784011 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc02cb90 con 0x7fb104158b60 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb0fbfff640 1 -- 172.21.15.17:0/1287784011 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0343f0 con 0x7fb104158b60 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 -- 172.21.15.17:0/1287784011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb104158b60 msgr2=0x7fb104158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 --2- 172.21.15.17:0/1287784011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb104158b60 0x7fb104158ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb10408c840 tx=0x7fb0fc02be50 comp rx=0 tx=0).stop 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 -- 172.21.15.17:0/1287784011 shutdown_connections 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 --2- 172.21.15.17:0/1287784011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb104158b60 0x7fb104158ff0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 --2- 172.21.15.17:0/1287784011 >> v2:172.21.15.163:3300/0 conn(0x7fb10414f360 0x7fb104155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 -- 172.21.15.17:0/1287784011 >> 172.21.15.17:0/1287784011 conn(0x7fb1040ac570 msgr2=0x7fb1040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 -- 172.21.15.17:0/1287784011 shutdown_connections 2024-03-06T07:21:54.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.439+0000 7fb10b33e640 1 -- 172.21.15.17:0/1287784011 wait complete. 2024-03-06T07:21:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.440+0000 7fb10b33e640 1 Processor -- start 2024-03-06T07:21:54.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.440+0000 7fb10b33e640 1 -- start start 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040c88c0 0x7fb104151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb104158b60 0x7fb1040b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040c5a50 con 0x7fb10414f360 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040c7c30 con 0x7fb1040c88c0 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb10b33e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040c5bd0 con 0x7fb104158b60 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56600/0 (socket says 172.21.15.17:56600) 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 -- 172.21.15.17:0/4150063652 learned_addr learned my addr 172.21.15.17:0/4150063652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1090da640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040c88c0 0x7fb104151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1098db640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb104158b60 0x7fb1040b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 -- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb104158b60 msgr2=0x7fb1040b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb104158b60 0x7fb1040b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 -- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040c88c0 msgr2=0x7fb104151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040c88c0 0x7fb104151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 -- 172.21.15.17:0/4150063652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040c1480 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.441+0000 7fb1088d9640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400b3c0 tx=0x7fb0f400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.442+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4011830 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.442+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4016070 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.442+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1041eb9a0 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.442+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4011c10 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.442+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1040bef90 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.443+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb104075300 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.443+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0f401b460 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.443+0000 7fb0f9ffb640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 0x7fb0e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.444+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb0f4089d70 con 0x7fb10414f360 2024-03-06T07:21:54.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.444+0000 7fb1090da640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 0x7fb0e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.444+0000 7fb1090da640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 0x7fb0e806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb100006fd0 tx=0x7fb100008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.448+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb104075300 con 0x7fb10414f360 2024-03-06T07:21:54.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.665+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb10404f330 con 0x7fb10414f360 2024-03-06T07:21:54.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.666+0000 7fb0f9ffb640 1 -- 172.21.15.17:0/4150063652 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4980 (secure 0 0 0) 0x7fb0f4020020 con 0x7fb10414f360 2024-03-06T07:21:54.669 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:54.669 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5769,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:21:46.263408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5772":{"gid":5772,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1002585815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1002585815},{"type":"v1","addr":"172.21.15.17:6835","nonce":1002585815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5772,"qdb_cluster":[5772]},"id":3}]} 2024-03-06T07:21:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 31 2024-03-06T07:21:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 msgr2=0x7fb0e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 0x7fb0e806d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb100006fd0 tx=0x7fb100008040 comp rx=0 tx=0).stop 2024-03-06T07:21:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 msgr2=0x7fb1040afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400b3c0 tx=0x7fb0f400b880 comp rx=0 tx=0).stop 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 shutdown_connections 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb0e806b500 0x7fb0e806d9c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb104158b60 0x7fb1040b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb10414f360 0x7fb1040afd70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 --2- 172.21.15.17:0/4150063652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1040c88c0 0x7fb104151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.667+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 >> 172.21.15.17:0/4150063652 conn(0x7fb1040ac570 msgr2=0x7fb1040ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.668+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 shutdown_connections 2024-03-06T07:21:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.668+0000 7fb10b33e640 1 -- 172.21.15.17:0/4150063652 wait complete. 2024-03-06T07:21:54.682 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 31, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:21:43.270845+0000", "damaged": [], "data_pools": [ 8 ], "enabled": true, "epoch": 31, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5772": { "addr": "172.21.15.17:6835/1002585815", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1002585815, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1002585815, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5772, "incarnation": 28, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 7, "modified": "2024-03-06T07:21:46.263408+0000", "qdb_cluster": [ 5772 ], "qdb_leader": 5772, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5772 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/673661445", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 673661445, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 673661445, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 26, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5587, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/3006853169", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3006853169, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3006853169, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 26, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5590, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/3526701035", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3526701035, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 3526701035, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 26, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5769, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:21:54.682 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:21:54.682 INFO:tasks.cephfs.test_failover:current = [{'gid': 5772, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/1002585815', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1002585815}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1002585815}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5590, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/3006853169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3006853169}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3006853169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 26}, {'gid': 5769, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/3526701035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3526701035}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 3526701035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 26}, {'gid': 5587, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/673661445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 673661445}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 673661445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 26}] 2024-03-06T07:21:54.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:21:54.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.784+0000 7f254b89d640 1 Processor -- start 2024-03-06T07:21:54.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.785+0000 7f254b89d640 1 -- start start 2024-03-06T07:21:54.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.785+0000 7f254b89d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25440c8a00 0x7f2544158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.785+0000 7f254b89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2544159460 0x7f25440c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.785+0000 7f254b89d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440c7760 con 0x7f2544152920 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f254b89d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25441587b0 con 0x7f25440c8a00 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f254b89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c7a40 con 0x7f2544159460 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2549639640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2544152920 0x7f2544155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49908/0 (socket says 172.21.15.17:49908) 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2549639640 1 -- 172.21.15.17:0/929500434 learned_addr learned my addr 172.21.15.17:0/929500434 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2549e3a640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2544159460 0x7f25440c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2548e38640 1 --2- 172.21.15.17:0/929500434 >> v2:172.21.15.163:3300/0 conn(0x7f25440c8a00 0x7f2544158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162795861 0 0) 0x7f25440c7760 con 0x7f2544152920 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2530003520 con 0x7f2544152920 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419691184 0 0) 0x7f2530003520 con 0x7f2544152920 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2544159460 msgr2=0x7f25440c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2544159460 0x7f25440c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 >> v2:172.21.15.163:3300/0 conn(0x7f25440c8a00 msgr2=0x7f2544158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 --2- 172.21.15.17:0/929500434 >> v2:172.21.15.163:3300/0 conn(0x7f25440c8a00 0x7f2544158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440c7d20 con 0x7f2544152920 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f2540001670 con 0x7f2544152920 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2549e3a640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2544159460 0x7f25440c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 >> v1:172.21.15.17:6789/0 conn(0x7f2544152920 legacy=0x7f2544155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25300039b0 0x7f2530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2530004800 0x7f2530004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25441587b0 con 0x7f25300039b0 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c7a40 con 0x7f2530004800 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253000bf00 con 0x7f25300051c0 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.786+0000 7f2549639640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25300039b0 0x7f2530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549e3a640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2548e38640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2530004800 0x7f2530004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549639640 1 -- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300051c0 msgr2=0x7f253000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549639640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300051c0 0x7f253000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549639640 1 -- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2530004800 msgr2=0x7f2530004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549639640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2530004800 0x7f2530004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.787+0000 7f2549639640 1 -- 172.21.15.17:0/929500434 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530003520 con 0x7f25300039b0 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f2549639640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25300039b0 0x7f2530003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f254000c210 tx=0x7f2530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb5d292575034f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540007d50 con 0x7f25300039b0 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2540030070 con 0x7f25300039b0 2024-03-06T07:21:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f2537fff640 1 -- 172.21.15.17:0/929500434 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254002cd60 con 0x7f25300039b0 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f254b89d640 1 -- 172.21.15.17:0/929500434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25300039b0 msgr2=0x7f2530003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.788+0000 7f254b89d640 1 --2- 172.21.15.17:0/929500434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25300039b0 0x7f2530003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f254000c210 tx=0x7f2530005830 comp rx=0 tx=0).stop 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.789+0000 7f2549e3a640 1 -- 172.21.15.17:0/929500434 reap_dead start 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.789+0000 7f254b89d640 1 -- 172.21.15.17:0/929500434 shutdown_connections 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.789+0000 7f254b89d640 1 -- 172.21.15.17:0/929500434 >> 172.21.15.17:0/929500434 conn(0x7f25440ac560 msgr2=0x7f25440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.789+0000 7f254b89d640 1 -- 172.21.15.17:0/929500434 shutdown_connections 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.789+0000 7f254b89d640 1 -- 172.21.15.17:0/929500434 wait complete. 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.790+0000 7f254b89d640 1 Processor -- start 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.790+0000 7f254b89d640 1 -- start start 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 0x7f2544151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2544148b30 0x7f2544148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c7390 con 0x7f2544154bb0 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254406d030 con 0x7f2544148b30 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.791+0000 7f254b89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c7510 con 0x7f25300039b0 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56616/0 (socket says 172.21.15.17:56616) 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 -- 172.21.15.17:0/1243207032 learned_addr learned my addr 172.21.15.17:0/1243207032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2549639640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 0x7f2544151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 -- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 msgr2=0x7f2544151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 0x7f2544151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2549e3a640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2544148b30 0x7f2544148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 -- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2544148b30 msgr2=0x7f2544148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2544148b30 0x7f2544148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 -- 172.21.15.17:0/1243207032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25441ee560 con 0x7f2544154bb0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2549639640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 0x7f2544151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2548e38640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2538011990 tx=0x7f2538011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800fa10 con 0x7f2544154bb0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253800fbb0 con 0x7f2544154bb0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25441ee800 con 0x7f2544154bb0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538014440 con 0x7f2544154bb0 2024-03-06T07:21:54.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.792+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25440c4720 con 0x7f2544154bb0 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.794+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254404f330 con 0x7f2544154bb0 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.794+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2538012040 con 0x7f2544154bb0 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.794+0000 7f2535ffb640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 0x7f252806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.795+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f253808a7a0 con 0x7f2544154bb0 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.795+0000 7f2549639640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 0x7f252806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:54.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.795+0000 7f2549639640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 0x7f252806d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f2540007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:54.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.799+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2538019030 con 0x7f2544154bb0 2024-03-06T07:21:54.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.980+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f254406cda0 con 0x7f2544154bb0 2024-03-06T07:21:54.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.983+0000 7f2535ffb640 1 -- 172.21.15.17:0/1243207032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v16)=0 v16) v1 ==== 117+0+0 (secure 0 0 0) 0x7f25380594f0 con 0x7f2544154bb0 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 msgr2=0x7f252806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 0x7f252806d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f2540007000 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 msgr2=0x7f2544157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2538011990 tx=0x7f2538011e50 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 shutdown_connections 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f252806b500 0x7f252806d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2544148b30 0x7f2544148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2544154bb0 0x7f2544157870 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 --2- 172.21.15.17:0/1243207032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25300039b0 0x7f2544151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 >> 172.21.15.17:0/1243207032 conn(0x7f25440ac560 msgr2=0x7f25440bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:54.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 shutdown_connections 2024-03-06T07:21:54.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:54.985+0000 7f254b89d640 1 -- 172.21.15.17:0/1243207032 wait complete. 2024-03-06T07:21:54.998 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs2 2024-03-06T07:21:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.099+0000 7f95b2f56640 1 Processor -- start 2024-03-06T07:21:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.099+0000 7f95b2f56640 1 -- start start 2024-03-06T07:21:55.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b2f56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 0x7f95ac0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b2f56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac155d20 con 0x7f95ac0af720 2024-03-06T07:21:55.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b2f56640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac155fc0 con 0x7f95ac0c5310 2024-03-06T07:21:55.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b2f56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ad900 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b0cf2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f95ac0af720 0x7f95ac14ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49910/0 (socket says 172.21.15.17:49910) 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95b0cf2640 1 -- 172.21.15.17:0/181090716 learned_addr learned my addr 172.21.15.17:0/181090716 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95abfff640 1 --2- 172.21.15.17:0/181090716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 0x7f95ac0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583444625 0 0) 0x7f95ac155d20 con 0x7f95ac0af720 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f959c003520 con 0x7f95ac0af720 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598277304 0 0) 0x7f95ac155fc0 con 0x7f95ac0c5310 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95ac155d20 con 0x7f95ac0c5310 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95abfff640 1 -- 172.21.15.17:0/181090716 >> v1:172.21.15.163:6789/0 conn(0x7f95ac0c5310 legacy=0x7f95ac0c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95abfff640 1 -- 172.21.15.17:0/181090716 >> v1:172.21.15.17:6789/0 conn(0x7f95ac0af720 legacy=0x7f95ac14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.100+0000 7f95abfff640 1 -- 172.21.15.17:0/181090716 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0adb50 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.101+0000 7f95abfff640 1 --2- 172.21.15.17:0/181090716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 0x7f95ac0c8b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95980097d0 tx=0x7f959802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c34c447c7cc6da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598031770 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598037070 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95ab7fe640 1 -- 172.21.15.17:0/181090716 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598031b10 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 -- 172.21.15.17:0/181090716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 msgr2=0x7f95ac0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 --2- 172.21.15.17:0/181090716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 0x7f95ac0c8b30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95980097d0 tx=0x7f959802b810 comp rx=0 tx=0).stop 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 -- 172.21.15.17:0/181090716 shutdown_connections 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 --2- 172.21.15.17:0/181090716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0ca1a0 0x7f95ac0c8b30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 -- 172.21.15.17:0/181090716 >> 172.21.15.17:0/181090716 conn(0x7f95ac0ac7e0 msgr2=0x7f95ac0acbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 -- 172.21.15.17:0/181090716 shutdown_connections 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.102+0000 7f95b2f56640 1 -- 172.21.15.17:0/181090716 wait complete. 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.103+0000 7f95b2f56640 1 Processor -- start 2024-03-06T07:21:55.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.103+0000 7f95b2f56640 1 -- start start 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 0x7f95ac0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 0x7f95ac1eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ade30 con 0x7f95ac0c5310 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0ca6f0 con 0x7f95ac0ca1a0 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b2f56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0adfb0 con 0x7f95ac0af720 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56618/0 (socket says 172.21.15.17:56618) 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95abfff640 1 -- 172.21.15.17:0/2740355205 learned_addr learned my addr 172.21.15.17:0/2740355205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b0cf2640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 0x7f95ac0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.104+0000 7f95b14f3640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 0x7f95ac1eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 -- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 msgr2=0x7f95ac0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 0x7f95ac0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 -- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 msgr2=0x7f95ac1eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 0x7f95ac1eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 -- 172.21.15.17:0/2740355205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0c5c10 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95b0cf2640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 0x7f95ac0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95b14f3640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 0x7f95ac1eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95abfff640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f959802b850 tx=0x7f95980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598032600 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95980327a0 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1eefb0 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598032b70 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.105+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac1ef250 con 0x7f95ac0c5310 2024-03-06T07:21:55.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.106+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac069460 con 0x7f95ac0c5310 2024-03-06T07:21:55.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.107+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f959804c060 con 0x7f95ac0c5310 2024-03-06T07:21:55.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.107+0000 7f95a97fa640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.107+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f95980abcf0 con 0x7f95ac0c5310 2024-03-06T07:21:55.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.107+0000 7f95b0cf2640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.108+0000 7f95b0cf2640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 0x7f959006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007fd0 tx=0x7f95a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.111+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95ac069460 con 0x7f95ac0c5310 2024-03-06T07:21:55.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.291+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f95ac04f330 con 0x7f95ac0c5310 2024-03-06T07:21:55.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.295+0000 7f95a97fa640 1 -- 172.21.15.17:0/2740355205 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v17)=0 v17) v1 ==== 117+0+0 (secure 0 0 0) 0x7f9598040070 con 0x7f95ac0c5310 2024-03-06T07:21:55.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 msgr2=0x7f959006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 0x7f959006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f95a0007fd0 tx=0x7f95a0009040 comp rx=0 tx=0).stop 2024-03-06T07:21:55.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 msgr2=0x7f95ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f959802b850 tx=0x7f95980046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 shutdown_connections 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f959006b4d0 0x7f959006d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95ac0ca1a0 0x7f95ac1eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95ac0c5310 0x7f95ac06d330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 --2- 172.21.15.17:0/2740355205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95ac0af720 0x7f95ac0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 >> 172.21.15.17:0/2740355205 conn(0x7f95ac0ac7e0 msgr2=0x7f95ac14e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.297+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 shutdown_connections 2024-03-06T07:21:55.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.298+0000 7f95b2f56640 1 -- 172.21.15.17:0/2740355205 wait complete. 2024-03-06T07:21:55.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs2 2024-03-06T07:21:55.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.413+0000 7fdefd572640 1 Processor -- start 2024-03-06T07:21:55.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.413+0000 7fdefd572640 1 -- start start 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdefd572640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdef81505f0 0x7fdef8156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdefd572640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 0x7fdef80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdefd572640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef8073580 con 0x7fdef80c7f60 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdefd572640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef8070b20 con 0x7fdef81505f0 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdefd572640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef80701f0 con 0x7fdef80c6710 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdef81505f0 0x7fdef8156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdef80c7f60 0x7fdef8072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49926/0 (socket says 172.21.15.17:49926) 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef67fc640 1 -- 172.21.15.17:0/1525338396 learned_addr learned my addr 172.21.15.17:0/1525338396 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef77fe640 1 --2- 172.21.15.17:0/1525338396 >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 0x7fdef80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef5ffb640 1 -- 172.21.15.17:0/1525338396 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995814514 0 0) 0x7fdef8073580 con 0x7fdef80c7f60 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef6ffd640 1 -- 172.21.15.17:0/1525338396 >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 msgr2=0x7fdef80c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef6ffd640 1 --2- 172.21.15.17:0/1525338396 >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 0x7fdef80c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef6ffd640 1 -- 172.21.15.17:0/1525338396 >> v1:172.21.15.17:6789/0 conn(0x7fdef80c7f60 legacy=0x7fdef8072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef6ffd640 1 -- 172.21.15.17:0/1525338396 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80704d0 con 0x7fdef81505f0 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.414+0000 7fdef77fe640 1 --2- 172.21.15.17:0/1525338396 >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 0x7fdef80c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.415+0000 7fdef6ffd640 1 --2- 172.21.15.17:0/1525338396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef81505f0 0x7fdef8156ef0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdef806ed80 tx=0x7fdee802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdbf47465adad06 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.415+0000 7fdef5ffb640 1 -- 172.21.15.17:0/1525338396 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8035020 con 0x7fdef81505f0 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.415+0000 7fdef5ffb640 1 -- 172.21.15.17:0/1525338396 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8031830 con 0x7fdef81505f0 2024-03-06T07:21:55.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.415+0000 7fdef5ffb640 1 -- 172.21.15.17:0/1525338396 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8031c00 con 0x7fdef81505f0 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 -- 172.21.15.17:0/1525338396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef81505f0 msgr2=0x7fdef8156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 --2- 172.21.15.17:0/1525338396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef81505f0 0x7fdef8156ef0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdef806ed80 tx=0x7fdee802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 -- 172.21.15.17:0/1525338396 shutdown_connections 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 --2- 172.21.15.17:0/1525338396 >> v2:172.21.15.163:3300/0 conn(0x7fdef80c6710 0x7fdef80c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 --2- 172.21.15.17:0/1525338396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef81505f0 0x7fdef8156ef0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 -- 172.21.15.17:0/1525338396 >> 172.21.15.17:0/1525338396 conn(0x7fdef80aa5c0 msgr2=0x7fdef80aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 -- 172.21.15.17:0/1525338396 shutdown_connections 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.416+0000 7fdefd572640 1 -- 172.21.15.17:0/1525338396 wait complete. 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.417+0000 7fdefd572640 1 Processor -- start 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.417+0000 7fdefd572640 1 -- start start 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdef80c6710 0x7fdef8156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 0x7fdef8148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8072c20 con 0x7fdef80c7f60 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef806fec0 con 0x7fdef81505f0 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdefd572640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8072da0 con 0x7fdef80c6710 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56628/0 (socket says 172.21.15.17:56628) 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 -- 172.21.15.17:0/1786610513 learned_addr learned my addr 172.21.15.17:0/1786610513 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef6ffd640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdef80c6710 0x7fdef8156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef77fe640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 0x7fdef8148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 -- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdef80c6710 msgr2=0x7fdef8156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdef80c6710 0x7fdef8156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 -- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 msgr2=0x7fdef8148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 0x7fdef8148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef67fc640 1 -- 172.21.15.17:0/1786610513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef80ad7b0 con 0x7fdef80c7f60 2024-03-06T07:21:55.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.418+0000 7fdef77fe640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 0x7fdef8148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdef67fc640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c500 tx=0x7fdee000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0018070 con 0x7fdef80c7f60 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0007580 con 0x7fdef80c7f60 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef80ada90 con 0x7fdef80c7f60 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004a20 con 0x7fdef80c7f60 2024-03-06T07:21:55.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.419+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef80c3dd0 con 0x7fdef80c7f60 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.420+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8069460 con 0x7fdef80c7f60 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.420+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdee0004bc0 con 0x7fdef80c7f60 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.420+0000 7fdeef7fe640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.421+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdee0089590 con 0x7fdef80c7f60 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.421+0000 7fdef6ffd640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.421+0000 7fdef6ffd640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdee8036000 tx=0x7fdee803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.425+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdef8069460 con 0x7fdef80c7f60 2024-03-06T07:21:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.607+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fdef804f330 con 0x7fdef80c7f60 2024-03-06T07:21:55.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.611+0000 7fdeef7fe640 1 -- 172.21.15.17:0/1786610513 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v18)=0 v18) v1 ==== 117+0+0 (secure 0 0 0) 0x7fdee0057c10 con 0x7fdef80c7f60 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 msgr2=0x7fded806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 0x7fded806d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdee8036000 tx=0x7fdee803d040 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 msgr2=0x7fdef80c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c500 tx=0x7fdee000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 shutdown_connections 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fded806b500 0x7fded806d9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdef81505f0 0x7fdef8148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdef80c7f60 0x7fdef80c3890 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 --2- 172.21.15.17:0/1786610513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdef80c6710 0x7fdef8156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 >> 172.21.15.17:0/1786610513 conn(0x7fdef80aa5c0 msgr2=0x7fdef80a8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 shutdown_connections 2024-03-06T07:21:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.613+0000 7fdefd572640 1 -- 172.21.15.17:0/1786610513 wait complete. 2024-03-06T07:21:55.626 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:21:55.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.725+0000 7f201a6eb640 1 Processor -- start 2024-03-06T07:21:55.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.725+0000 7f201a6eb640 1 -- start start 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f201a6eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20141506b0 0x7f2014153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f201a6eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2014156ea0 0x7f20140702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f201a6eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2014070800 con 0x7f20140acc00 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f201a6eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2014157320 con 0x7f2014156ea0 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f201a6eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2014156450 con 0x7f20141506b0 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f20137fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2014156ea0 0x7f20140702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f20137fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2014156ea0 0x7f20140702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56636/0 (socket says 172.21.15.17:56636) 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f20137fe640 1 -- 172.21.15.17:0/350101633 learned_addr learned my addr 172.21.15.17:0/350101633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.726+0000 7f2013fff640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20141506b0 0x7f2014153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f20137fe640 1 -- 172.21.15.17:0/350101633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20141506b0 msgr2=0x7f2014153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f20137fe640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20141506b0 0x7f2014153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f20137fe640 1 -- 172.21.15.17:0/350101633 >> v1:172.21.15.163:6789/0 conn(0x7f20140acc00 legacy=0x7f20140c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f20137fe640 1 -- 172.21.15.17:0/350101633 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014070aa0 con 0x7f2014156ea0 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f20137fe640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014156ea0 0x7f20140702c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20140c62c0 tx=0x7f200802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e31d6eb3b9bf13 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.727+0000 7f2012ffd640 1 -- 172.21.15.17:0/350101633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008035020 con 0x7f2014156ea0 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f2012ffd640 1 -- 172.21.15.17:0/350101633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008031830 con 0x7f2014156ea0 2024-03-06T07:21:55.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f2012ffd640 1 -- 172.21.15.17:0/350101633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008031c00 con 0x7f2014156ea0 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 -- 172.21.15.17:0/350101633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014156ea0 msgr2=0x7f20140702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014156ea0 0x7f20140702c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20140c62c0 tx=0x7f200802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 -- 172.21.15.17:0/350101633 shutdown_connections 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014156ea0 0x7f20140702c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 --2- 172.21.15.17:0/350101633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20141506b0 0x7f2014153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 -- 172.21.15.17:0/350101633 >> 172.21.15.17:0/350101633 conn(0x7f201406b1a0 msgr2=0x7f201406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 -- 172.21.15.17:0/350101633 shutdown_connections 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.728+0000 7f201a6eb640 1 -- 172.21.15.17:0/350101633 wait complete. 2024-03-06T07:21:55.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.729+0000 7f201a6eb640 1 Processor -- start 2024-03-06T07:21:55.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 -- start start 2024-03-06T07:21:55.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 0x7f201414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2014156ea0 0x7f20141e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20141566a0 con 0x7f20141506b0 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014156a10 con 0x7f20140acc00 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f201a6eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c5bb0 con 0x7f2014156ea0 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f20137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f20137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56650/0 (socket says 172.21.15.17:56650) 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.730+0000 7f20137fe640 1 -- 172.21.15.17:0/914711359 learned_addr learned my addr 172.21.15.17:0/914711359 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2018c88640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2014156ea0 0x7f20141e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2013fff640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 0x7f201414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 -- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2014156ea0 msgr2=0x7f20141e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2014156ea0 0x7f20141e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 -- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 msgr2=0x7f201414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 0x7f201414b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 -- 172.21.15.17:0/914711359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20141ecb10 con 0x7f20141506b0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2013fff640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 0x7f201414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f20137fe640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f200802b910 tx=0x7f20080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008040070 con 0x7f20141506b0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2008039960 con 0x7f20141506b0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008035040 con 0x7f20141506b0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20141ecdb0 con 0x7f20141506b0 2024-03-06T07:21:55.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.731+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20141dc4c0 con 0x7f20141506b0 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.732+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20140c62f0 con 0x7f20141506b0 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.733+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2008038020 con 0x7f20141506b0 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.733+0000 7f2010ff9640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 0x7f200006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.733+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f20080abb30 con 0x7f20141506b0 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.733+0000 7f2013fff640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 0x7f200006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:55.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.734+0000 7f2013fff640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 0x7f200006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2004006fd0 tx=0x7f2004008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:55.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.737+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20140c62f0 con 0x7f20141506b0 2024-03-06T07:21:55.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.971+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f201414e6a0 con 0x7f20141506b0 2024-03-06T07:21:55.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.972+0000 7f2010ff9640 1 -- 172.21.15.17:0/914711359 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+17038 (secure 0 0 0) 0x7f201414e6a0 con 0x7f20141506b0 2024-03-06T07:21:55.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:55.975 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":53,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:53.930534+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 msgr2=0x7f200006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 0x7f200006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2004006fd0 tx=0x7f2004008040 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 msgr2=0x7f20141e8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f200802b910 tx=0x7f20080046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 shutdown_connections 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f200006b4d0 0x7f200006d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2014156ea0 0x7f20141e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20141506b0 0x7f20141e8a90 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 --2- 172.21.15.17:0/914711359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20140acc00 0x7f201414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 >> 172.21.15.17:0/914711359 conn(0x7f201406b1a0 msgr2=0x7f20140ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 shutdown_connections 2024-03-06T07:21:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:55.974+0000 7f201a6eb640 1 -- 172.21.15.17:0/914711359 wait complete. 2024-03-06T07:21:55.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:21:41.533584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:21:42.234700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:21:55.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:21:56.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 Processor -- start 2024-03-06T07:21:56.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 -- start start 2024-03-06T07:21:56.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f981c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c0b0450 con 0x7f981c155c30 2024-03-06T07:21:56.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c1560e0 con 0x7f981c0b0070 2024-03-06T07:21:56.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9820c66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c020 con 0x7f981c06ca70 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9813fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f981c155c30 0x7f981c0c5380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49930/0 (socket says 172.21.15.17:49930) 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.090+0000 7f9813fff640 1 -- 172.21.15.17:0/3416268918 learned_addr learned my addr 172.21.15.17:0/3416268918 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205305869 0 0) 0x7f9804001110 con 0x7f981c155c30 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9808003520 con 0x7f981c155c30 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f981ad9d640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f981c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198443756 0 0) 0x7f9808003520 con 0x7f981c155c30 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 msgr2=0x7f981c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f981c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 >> v1:172.21.15.163:6789/0 conn(0x7f981c0b0070 legacy=0x7f981c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0c77f0 con 0x7f981c155c30 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9804001840 con 0x7f981c155c30 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 >> v1:172.21.15.17:6789/0 conn(0x7f981c155c30 legacy=0x7f981c0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98080039e0 0x7f9808003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f9808004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f9808005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c020 con 0x7f9808004800 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980800bb80 con 0x7f98080039e0 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980800bd80 con 0x7f981c06ca70 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f9813fff640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f9808004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f981a59c640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98080039e0 0x7f9808003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.091+0000 7f981ad9d640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f9808005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 -- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 msgr2=0x7f9808005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c06ca70 0x7f9808005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 -- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98080039e0 msgr2=0x7f9808003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98080039e0 0x7f9808003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 -- 172.21.15.17:0/3416268918 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808003520 con 0x7f9808004800 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f981a59c640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98080039e0 0x7f9808003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9813fff640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f9808004c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f980400c4c0 tx=0x7f9808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb29fe791796703 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400c0e0 con 0x7f9808004800 2024-03-06T07:21:56.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9804030070 con 0x7f9808004800 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.092+0000 7f9819d9b640 1 -- 172.21.15.17:0/3416268918 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040058d0 con 0x7f9808004800 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 -- 172.21.15.17:0/3416268918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 msgr2=0x7f9808004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 --2- 172.21.15.17:0/3416268918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f9808004c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f980400c4c0 tx=0x7f9808005830 comp rx=0 tx=0).stop 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f981ad9d640 1 -- 172.21.15.17:0/3416268918 reap_dead start 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 -- 172.21.15.17:0/3416268918 shutdown_connections 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 -- 172.21.15.17:0/3416268918 >> 172.21.15.17:0/3416268918 conn(0x7f981c0ac590 msgr2=0x7f981c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 -- 172.21.15.17:0/3416268918 shutdown_connections 2024-03-06T07:21:56.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.093+0000 7f9820c66640 1 -- 172.21.15.17:0/3416268918 wait complete. 2024-03-06T07:21:56.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.094+0000 7f9820c66640 1 Processor -- start 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.094+0000 7f9820c66640 1 -- start start 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c14e910 0x7f981c1eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f981c0c4b10 0x7f981c1eb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c550 con 0x7f9808004800 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06c270 con 0x7f981c0c4b10 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9820c66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c06da80 con 0x7f981c14e910 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56672/0 (socket says 172.21.15.17:56672) 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 -- 172.21.15.17:0/479858627 learned_addr learned my addr 172.21.15.17:0/479858627 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981ad9d640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f981c0c4b10 0x7f981c1eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f9813fff640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c14e910 0x7f981c1eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 -- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c14e910 msgr2=0x7f981c1eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c14e910 0x7f981c1eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 -- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f981c0c4b10 msgr2=0x7f981c1eb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f981c0c4b10 0x7f981c1eb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 -- 172.21.15.17:0/479858627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c1ee300 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.095+0000 7f981a59c640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e4d0 tx=0x7f980c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.096+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0052e0 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.096+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c018070 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.096+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c1ee5e0 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.096+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c003810 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.096+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c155540 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.097+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c04f330 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.097+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f980c0039b0 con 0x7f9808004800 2024-03-06T07:21:56.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.097+0000 7f9812ffd640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.097+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f980c089e70 con 0x7f9808004800 2024-03-06T07:21:56.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.098+0000 7f9813fff640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.098+0000 7f9813fff640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 0x7f97fc06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f9804009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.106+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f980c058570 con 0x7f9808004800 2024-03-06T07:21:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.399+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f981c06d860 con 0x7f9808004800 2024-03-06T07:21:56.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.400+0000 7f9812ffd640 1 -- 172.21.15.17:0/479858627 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f980c05c610 con 0x7f9808004800 2024-03-06T07:21:56.403 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 msgr2=0x7f97fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 0x7f97fc06d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9804034000 tx=0x7f9804009030 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 msgr2=0x7f981c155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e4d0 tx=0x7f980c00e990 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 shutdown_connections 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97fc06b4d0 0x7f97fc06d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f981c0c4b10 0x7f981c1eb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f981c14e910 0x7f981c1eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 --2- 172.21.15.17:0/479858627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9808004800 0x7f981c155000 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 >> 172.21.15.17:0/479858627 conn(0x7f981c0ac590 msgr2=0x7f981c151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 shutdown_connections 2024-03-06T07:21:56.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.402+0000 7f9820c66640 1 -- 172.21.15.17:0/479858627 wait complete. 2024-03-06T07:21:56.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:21:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.516+0000 7f58a7977640 1 Processor -- start 2024-03-06T07:21:56.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 -- start start 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58a00b0070 0x7f58a0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58a006ca70 0x7f58a00c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00b0450 con 0x7f58a00c5380 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a006c020 con 0x7f58a00b0070 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a7977640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a006cf20 con 0x7f58a006ca70 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a5713640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58a00b0070 0x7f58a0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a5f14640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58a00c5380 0x7f58a014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49944/0 (socket says 172.21.15.17:49944) 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a5713640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58a00b0070 0x7f58a0155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56678/0 (socket says 172.21.15.17:56678) 2024-03-06T07:21:56.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.517+0000 7f58a5f14640 1 -- 172.21.15.17:0/784138598 learned_addr learned my addr 172.21.15.17:0/784138598 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a4f12640 1 --2- 172.21.15.17:0/784138598 >> v2:172.21.15.163:3300/0 conn(0x7f58a006ca70 0x7f58a00c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a5713640 1 -- 172.21.15.17:0/784138598 >> v2:172.21.15.163:3300/0 conn(0x7f58a006ca70 msgr2=0x7f58a00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a5713640 1 --2- 172.21.15.17:0/784138598 >> v2:172.21.15.163:3300/0 conn(0x7f58a006ca70 0x7f58a00c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a5713640 1 -- 172.21.15.17:0/784138598 >> v1:172.21.15.17:6789/0 conn(0x7f58a00c5380 legacy=0x7f58a014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a5713640 1 -- 172.21.15.17:0/784138598 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00c77f0 con 0x7f58a00b0070 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.518+0000 7f58a5713640 1 --2- 172.21.15.17:0/784138598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a0155c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f58a006d4c0 tx=0x7f589002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6ef9f45ffecd42 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f589ffff640 1 -- 172.21.15.17:0/784138598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58900315a0 con 0x7f58a00b0070 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f589ffff640 1 -- 172.21.15.17:0/784138598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890037070 con 0x7f58a00b0070 2024-03-06T07:21:56.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f589ffff640 1 -- 172.21.15.17:0/784138598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890031960 con 0x7f58a00b0070 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 -- 172.21.15.17:0/784138598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 msgr2=0x7f58a0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 --2- 172.21.15.17:0/784138598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a0155c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f58a006d4c0 tx=0x7f589002b840 comp rx=0 tx=0).stop 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 -- 172.21.15.17:0/784138598 shutdown_connections 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 --2- 172.21.15.17:0/784138598 >> v2:172.21.15.163:3300/0 conn(0x7f58a006ca70 0x7f58a00c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 --2- 172.21.15.17:0/784138598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a0155c30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.519+0000 7f58a7977640 1 -- 172.21.15.17:0/784138598 >> 172.21.15.17:0/784138598 conn(0x7f58a00ac590 msgr2=0x7f58a00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.520+0000 7f58a7977640 1 -- 172.21.15.17:0/784138598 shutdown_connections 2024-03-06T07:21:56.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.520+0000 7f58a7977640 1 -- 172.21.15.17:0/784138598 wait complete. 2024-03-06T07:21:56.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.521+0000 7f58a7977640 1 Processor -- start 2024-03-06T07:21:56.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.521+0000 7f58a7977640 1 -- start start 2024-03-06T07:21:56.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 0x7f58a014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 0x7f58a01ee690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a006c550 con 0x7f58a00b0070 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a006c270 con 0x7f58a006ca70 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a7977640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a006da80 con 0x7f58a00c5380 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a4f12640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a4f12640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56692/0 (socket says 172.21.15.17:56692) 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.522+0000 7f58a4f12640 1 -- 172.21.15.17:0/1547338573 learned_addr learned my addr 172.21.15.17:0/1547338573 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a5713640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 0x7f58a014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 -- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 msgr2=0x7f58a01ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a5f14640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 0x7f58a01ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 0x7f58a01ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 -- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 msgr2=0x7f58a014cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 0x7f58a014cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 -- 172.21.15.17:0/1547338573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00c11d0 con 0x7f58a00b0070 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a5713640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 0x7f58a014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a5f14640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 0x7f58a01ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a4f12640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5894007e40 tx=0x7f589400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894014620 con 0x7f58a00b0070 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5894019070 con 0x7f58a00b0070 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00c1490 con 0x7f58a00b0070 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894014a00 con 0x7f58a00b0070 2024-03-06T07:21:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.523+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a014b860 con 0x7f58a00b0070 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.525+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5894014ba0 con 0x7f58a00b0070 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.525+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a004f330 con 0x7f58a00b0070 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.525+0000 7f589dffb640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 0x7f588c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.525+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f589408abd0 con 0x7f58a00b0070 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.525+0000 7f58a5713640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 0x7f588c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.526+0000 7f58a5713640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 0x7f588c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5890036000 tx=0x7f5890008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.529+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5894059920 con 0x7f58a00b0070 2024-03-06T07:21:56.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.708+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58a006d860 con 0x7f58a00b0070 2024-03-06T07:21:56.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.709+0000 7f589dffb640 1 -- 172.21.15.17:0/1547338573 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5894059220 con 0x7f58a00b0070 2024-03-06T07:21:56.711 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:21:56.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 msgr2=0x7f588c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 0x7f588c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5890036000 tx=0x7f5890008030 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 msgr2=0x7f58a01ee150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5894007e40 tx=0x7f589400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 shutdown_connections 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f588c06b4d0 0x7f588c06d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58a00c5380 0x7f58a01ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58a00b0070 0x7f58a01ee150 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 --2- 172.21.15.17:0/1547338573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58a006ca70 0x7f58a014cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.710+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 >> 172.21.15.17:0/1547338573 conn(0x7f58a00ac590 msgr2=0x7f58a00ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.711+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 shutdown_connections 2024-03-06T07:21:56.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.711+0000 7f58a7977640 1 -- 172.21.15.17:0/1547338573 wait complete. 2024-03-06T07:21:56.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:21:56.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.825+0000 7f875fe8f640 1 Processor -- start 2024-03-06T07:21:56.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.825+0000 7f875fe8f640 1 -- start start 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875fe8f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87580a8290 0x7f87581539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875fe8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c4740 0x7f87580701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875fe8f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87580c62a0 con 0x7f87580c69a0 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875fe8f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8758153f00 con 0x7f87580a8290 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875fe8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580706e0 con 0x7f87580c4740 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875dc2b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87580a8290 0x7f87581539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875dc2b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87580a8290 0x7f87581539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56702/0 (socket says 172.21.15.17:56702) 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875dc2b640 1 -- 172.21.15.17:0/3072298407 learned_addr learned my addr 172.21.15.17:0/3072298407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.826+0000 7f875d42a640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c4740 0x7f87580701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875dc2b640 1 -- 172.21.15.17:0/3072298407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c4740 msgr2=0x7f87580701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875dc2b640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c4740 0x7f87580701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875dc2b640 1 -- 172.21.15.17:0/3072298407 >> v1:172.21.15.163:6789/0 conn(0x7f87580c69a0 legacy=0x7f8758072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875dc2b640 1 -- 172.21.15.17:0/3072298407 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580c4c00 con 0x7f87580a8290 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875dc2b640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87581539c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f2b9852319fcc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875cc29640 1 -- 172.21.15.17:0/3072298407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540317c0 con 0x7f87580a8290 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875cc29640 1 -- 172.21.15.17:0/3072298407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f87580a8290 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.827+0000 7f875cc29640 1 -- 172.21.15.17:0/3072298407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031b80 con 0x7f87580a8290 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 -- 172.21.15.17:0/3072298407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 msgr2=0x7f87581539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87581539c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402b860 comp rx=0 tx=0).stop 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 -- 172.21.15.17:0/3072298407 shutdown_connections 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c4740 0x7f87580701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 --2- 172.21.15.17:0/3072298407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87581539c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 -- 172.21.15.17:0/3072298407 >> 172.21.15.17:0/3072298407 conn(0x7f87580c5560 msgr2=0x7f87580c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 -- 172.21.15.17:0/3072298407 shutdown_connections 2024-03-06T07:21:56.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.828+0000 7f875fe8f640 1 -- 172.21.15.17:0/3072298407 wait complete. 2024-03-06T07:21:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.829+0000 7f875fe8f640 1 Processor -- start 2024-03-06T07:21:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.829+0000 7f875fe8f640 1 -- start start 2024-03-06T07:21:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 0x7f87581ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c69a0 0x7f87581eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875806fc80 con 0x7f87580a8290 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875806f9a0 con 0x7f87580c4740 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875fe8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8758072d90 con 0x7f87580c69a0 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56710/0 (socket says 172.21.15.17:56710) 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 -- 172.21.15.17:0/2024558878 learned_addr learned my addr 172.21.15.17:0/2024558878 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875e42c640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c69a0 0x7f87581eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875d42a640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 0x7f87581ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 -- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c69a0 msgr2=0x7f87581eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c69a0 0x7f87581eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 -- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 msgr2=0x7f87581ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 0x7f87581ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:56.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875dc2b640 1 -- 172.21.15.17:0/2024558878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87581df3f0 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.830+0000 7f875d42a640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 0x7f87581ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f875dc2b640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8754000c00 tx=0x7f87540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754032660 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8754032800 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875806db70 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754032bf0 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.831+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875806c270 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.832+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875804f330 con 0x7f87580a8290 2024-03-06T07:21:56.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.832+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f875404c060 con 0x7f87580a8290 2024-03-06T07:21:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.833+0000 7f874e7fc640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 0x7f873c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.833+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f87540abea0 con 0x7f87580a8290 2024-03-06T07:21:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.833+0000 7f875d42a640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 0x7f873c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:56.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.833+0000 7f875d42a640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 0x7f873c06d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87581ec090 tx=0x7f8748008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:56.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:56.837+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8754040070 con 0x7f87580a8290 2024-03-06T07:21:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.018+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87580735e0 con 0x7f87580a8290 2024-03-06T07:21:57.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.018+0000 7f874e7fc640 1 -- 172.21.15.17:0/2024558878 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f875407abf0 con 0x7f87580a8290 2024-03-06T07:21:57.021 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:21:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 msgr2=0x7f873c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 0x7f873c06d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87581ec090 tx=0x7f8748008040 comp rx=0 tx=0).stop 2024-03-06T07:21:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 msgr2=0x7f87580c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8754000c00 tx=0x7f87540046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 shutdown_connections 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f873c06b4d0 0x7f873c06d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87580c69a0 0x7f87581eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87580c4740 0x7f87581ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 --2- 172.21.15.17:0/2024558878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87580a8290 0x7f87580c2100 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 >> 172.21.15.17:0/2024558878 conn(0x7f87580c5560 msgr2=0x7f87580a9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 shutdown_connections 2024-03-06T07:21:57.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.020+0000 7f875fe8f640 1 -- 172.21.15.17:0/2024558878 wait complete. 2024-03-06T07:21:57.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:21:57.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.133+0000 7f3fcf2af640 1 Processor -- start 2024-03-06T07:21:57.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.133+0000 7f3fcf2af640 1 -- start start 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.133+0000 7f3fcf2af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 0x7f3fc8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcf2af640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc80c73d0 con 0x7f3fc814f360 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcf2af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc80be2e0 con 0x7f3fc80c88c0 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcf2af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8159700 con 0x7f3fc8159270 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd04b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3fc814f360 0x7f3fc80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49952/0 (socket says 172.21.15.17:49952) 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd04b640 1 -- 172.21.15.17:0/4077399228 learned_addr learned my addr 172.21.15.17:0/4077399228 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:21:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd84c640 1 --2- 172.21.15.17:0/4077399228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 0x7f3fc8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586382241 0 0) 0x7f3fc80c73d0 con 0x7f3fc814f360 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fb0003520 con 0x7f3fc814f360 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487806986 0 0) 0x7f3fc80be2e0 con 0x7f3fc80c88c0 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc80c73d0 con 0x7f3fc80c88c0 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 527550950 0 0) 0x7f3fb0003520 con 0x7f3fc814f360 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd84c640 1 -- 172.21.15.17:0/4077399228 >> v1:172.21.15.163:6789/0 conn(0x7f3fc80c88c0 legacy=0x7f3fc8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd84c640 1 -- 172.21.15.17:0/4077399228 >> v1:172.21.15.17:6789/0 conn(0x7f3fc814f360 legacy=0x7f3fc80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.134+0000 7f3fcd84c640 1 -- 172.21.15.17:0/4077399228 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80c7670 con 0x7f3fc8159270 2024-03-06T07:21:57.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.135+0000 7f3fcd84c640 1 --2- 172.21.15.17:0/4077399228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 0x7f3fc8155c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097d0 tx=0x7f3fc002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b370688863fed635 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.135+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0037070 con 0x7f3fc8159270 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.135+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0030510 con 0x7f3fc8159270 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.135+0000 7f3fc7fff640 1 -- 172.21.15.17:0/4077399228 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80019f0 con 0x7f3fc8159270 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 -- 172.21.15.17:0/4077399228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 msgr2=0x7f3fc8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/4077399228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 0x7f3fc8155c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00097d0 tx=0x7f3fc002b770 comp rx=0 tx=0).stop 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 -- 172.21.15.17:0/4077399228 shutdown_connections 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/4077399228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc8159270 0x7f3fc8155c10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 -- 172.21.15.17:0/4077399228 >> 172.21.15.17:0/4077399228 conn(0x7f3fc80ac570 msgr2=0x7f3fc80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 -- 172.21.15.17:0/4077399228 shutdown_connections 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.136+0000 7f3fcf2af640 1 -- 172.21.15.17:0/4077399228 wait complete. 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.137+0000 7f3fcf2af640 1 Processor -- start 2024-03-06T07:21:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.137+0000 7f3fcf2af640 1 -- start start 2024-03-06T07:21:57.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc80c88c0 0x7f3fc81ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc814f360 0x7f3fc81e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80c7c30 con 0x7f3fc8159270 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80c5a50 con 0x7f3fc80c88c0 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcf2af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc80c5bd0 con 0x7f3fc814f360 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56718/0 (socket says 172.21.15.17:56718) 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 -- 172.21.15.17:0/289619630 learned_addr learned my addr 172.21.15.17:0/289619630 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd04b640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc80c88c0 0x7f3fc81ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcc84a640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc814f360 0x7f3fc81e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 -- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc814f360 msgr2=0x7f3fc81e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc814f360 0x7f3fc81e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 -- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc80c88c0 msgr2=0x7f3fc81ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc80c88c0 0x7f3fc81ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 -- 172.21.15.17:0/289619630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8154310 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.138+0000 7f3fcd84c640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3fc002b7b0 tx=0x7f3fc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.139+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0042070 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.139+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc003d850 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.139+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc81dec80 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.139+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0037040 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.139+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc81545b0 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.140+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc804f330 con 0x7f3fc8159270 2024-03-06T07:21:57.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.140+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fc0045070 con 0x7f3fc8159270 2024-03-06T07:21:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.141+0000 7f3fc5ffb640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.141+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3fc00abca0 con 0x7f3fc8159270 2024-03-06T07:21:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.141+0000 7f3fcd04b640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.141+0000 7f3fcd04b640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8002230 tx=0x7f3fb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.145+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fc007a3a0 con 0x7f3fc8159270 2024-03-06T07:21:57.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.330+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3fc80c9ed0 con 0x7f3fc8159270 2024-03-06T07:21:57.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.331+0000 7f3fc5ffb640 1 -- 172.21.15.17:0/289619630 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+207 (secure 0 0 0) 0x7f3fc007e440 con 0x7f3fc8159270 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 7, 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:21:57.334 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 8, 2024-03-06T07:21:57.335 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_data" 2024-03-06T07:21:57.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:21:57.335 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 msgr2=0x7f3fb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 0x7f3fb406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8002230 tx=0x7f3fb8009040 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 msgr2=0x7f3fc80abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3fc002b7b0 tx=0x7f3fc00046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 shutdown_connections 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fb406b4d0 0x7f3fb406d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fc8159270 0x7f3fc80abbb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fc814f360 0x7f3fc81e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 --2- 172.21.15.17:0/289619630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fc80c88c0 0x7f3fc81ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 >> 172.21.15.17:0/289619630 conn(0x7f3fc80ac570 msgr2=0x7f3fc80ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 shutdown_connections 2024-03-06T07:21:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.333+0000 7f3fcf2af640 1 -- 172.21.15.17:0/289619630 wait complete. 2024-03-06T07:21:57.346 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:21:57.347 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:21:57.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.449+0000 7f709f28a640 1 Processor -- start 2024-03-06T07:21:57.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.449+0000 7f709f28a640 1 -- start start 2024-03-06T07:21:57.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709f28a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70980c2330 0x7f70980c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709f28a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70981550c0 0x7f7098155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709f28a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7098159260 con 0x7f709814bb20 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709f28a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980c3070 con 0x7f70980c2330 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709f28a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980a80e0 con 0x7f70981550c0 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70980c2330 0x7f70980c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70980c2330 0x7f70980c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56734/0 (socket says 172.21.15.17:56734) 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 -- 172.21.15.17:0/2732888179 learned_addr learned my addr 172.21.15.17:0/2732888179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709d827640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70981550c0 0x7f7098155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 -- 172.21.15.17:0/2732888179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70981550c0 msgr2=0x7f7098155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70981550c0 0x7f7098155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 -- 172.21.15.17:0/2732888179 >> v1:172.21.15.163:6789/0 conn(0x7f709814bb20 legacy=0x7f70980c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:57.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.450+0000 7f709c825640 1 -- 172.21.15.17:0/2732888179 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980a7d70 con 0x7f70980c2330 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.451+0000 7f709c825640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70980c2330 0x7f70980c5a20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7098073730 tx=0x7f708802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecc8c8bf30d9be5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.451+0000 7f708ffff640 1 -- 172.21.15.17:0/2732888179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880315a0 con 0x7f70980c2330 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.451+0000 7f708ffff640 1 -- 172.21.15.17:0/2732888179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7088037070 con 0x7f70980c2330 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.451+0000 7f708ffff640 1 -- 172.21.15.17:0/2732888179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088031960 con 0x7f70980c2330 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 -- 172.21.15.17:0/2732888179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70980c2330 msgr2=0x7f70980c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70980c2330 0x7f70980c5a20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7098073730 tx=0x7f708802b840 comp rx=0 tx=0).stop 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 -- 172.21.15.17:0/2732888179 shutdown_connections 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70981550c0 0x7f7098155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 --2- 172.21.15.17:0/2732888179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70980c2330 0x7f70980c5a20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 -- 172.21.15.17:0/2732888179 >> 172.21.15.17:0/2732888179 conn(0x7f70980c88f0 msgr2=0x7f70980c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 -- 172.21.15.17:0/2732888179 shutdown_connections 2024-03-06T07:21:57.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.452+0000 7f709f28a640 1 -- 172.21.15.17:0/2732888179 wait complete. 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 Processor -- start 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 -- start start 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70980c2330 0x7f7098156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 0x7f709814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709814bf00 con 0x7f709814bb20 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c5f60 con 0x7f70981550c0 2024-03-06T07:21:57.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.453+0000 7f709f28a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c67f0 con 0x7f70980c2330 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56750/0 (socket says 172.21.15.17:56750) 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 -- 172.21.15.17:0/1609488900 learned_addr learned my addr 172.21.15.17:0/1609488900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709d827640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 0x7f709814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709d026640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70980c2330 0x7f7098156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 -- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70980c2330 msgr2=0x7f7098156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70980c2330 0x7f7098156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 -- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 msgr2=0x7f709814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 0x7f709814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 -- 172.21.15.17:0/1609488900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709814f1a0 con 0x7f709814bb20 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709d827640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 0x7f709814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709c825640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c00 tx=0x7f70880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088032840 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70880329e0 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70981eed50 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.454+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088032dd0 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.455+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7098152780 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.455+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709804f330 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.456+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f708804c020 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.456+0000 7f708dffb640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 0x7f707c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.456+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f708802f020 con 0x7f709814bb20 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.457+0000 7f709d026640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 0x7f707c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.457+0000 7f709d026640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 0x7f707c06d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7094003d90 tx=0x7f7094009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:57.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.461+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7088040070 con 0x7f709814bb20 2024-03-06T07:21:57.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.643+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f70980c2110 con 0x7f709814bb20 2024-03-06T07:21:57.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.988+0000 7f708dffb640 1 -- 172.21.15.17:0/1609488900 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v54) v1 ==== 140+0+0 (secure 0 0 0) 0x7f708807adb0 con 0x7f709814bb20 2024-03-06T07:21:57.991 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 msgr2=0x7f707c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 0x7f707c06d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7094003d90 tx=0x7f7094009040 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 msgr2=0x7f70981ee2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7088000c00 tx=0x7f70880046c0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 shutdown_connections 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f707c06b4d0 0x7f707c06d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70981550c0 0x7f709814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f709814bb20 0x7f70981ee2d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 --2- 172.21.15.17:0/1609488900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70980c2330 0x7f7098156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:57.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 >> 172.21.15.17:0/1609488900 conn(0x7f70980c88f0 msgr2=0x7f7098152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:57.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 shutdown_connections 2024-03-06T07:21:57.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:57.991+0000 7f709f28a640 1 -- 172.21.15.17:0/1609488900 wait complete. 2024-03-06T07:21:58.008 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:21:58.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.119+0000 7f0d67cb5640 1 Processor -- start 2024-03-06T07:21:58.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.119+0000 7f0d67cb5640 1 -- start start 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 0x7f0d60155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 0x7f0d600c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d600c5380 0x7f0d600c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6006c020 con 0x7f0d600c5380 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d600b0450 con 0x7f0d600b0070 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d67cb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6006cf20 con 0x7f0d6006ca70 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d66252640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d600c5380 0x7f0d600c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d66252640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0d600c5380 0x7f0d600c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56766/0 (socket says 172.21.15.17:56766) 2024-03-06T07:21:58.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d66252640 1 -- 172.21.15.17:0/1222719089 learned_addr learned my addr 172.21.15.17:0/1222719089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d65250640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 0x7f0d600c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.120+0000 7f0d65a51640 1 --2- 172.21.15.17:0/1222719089 >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 0x7f0d60155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d66252640 1 -- 172.21.15.17:0/1222719089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 msgr2=0x7f0d600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d66252640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 0x7f0d600c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d66252640 1 -- 172.21.15.17:0/1222719089 >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 msgr2=0x7f0d60155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d66252640 1 --2- 172.21.15.17:0/1222719089 >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 0x7f0d60155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d66252640 1 -- 172.21.15.17:0/1222719089 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600c77f0 con 0x7f0d600c5380 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d65250640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 0x7f0d600c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:58.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.121+0000 7f0d65a51640 1 --2- 172.21.15.17:0/1222719089 >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 0x7f0d60155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:21:58.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d66252640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d600c5380 0x7f0d600c57f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0d600aa3d0 tx=0x7f0d5802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58390b5db14fc444 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d64a4f640 1 -- 172.21.15.17:0/1222719089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5802c9e0 con 0x7f0d600c5380 2024-03-06T07:21:58.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d64a4f640 1 -- 172.21.15.17:0/1222719089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5802cb80 con 0x7f0d600c5380 2024-03-06T07:21:58.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d64a4f640 1 -- 172.21.15.17:0/1222719089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580343f0 con 0x7f0d600c5380 2024-03-06T07:21:58.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d67cb5640 1 -- 172.21.15.17:0/1222719089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d600c5380 msgr2=0x7f0d600c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.122+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d600c5380 0x7f0d600c57f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0d600aa3d0 tx=0x7f0d5802be40 comp rx=0 tx=0).stop 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 -- 172.21.15.17:0/1222719089 shutdown_connections 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d600c5380 0x7f0d600c57f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/1222719089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d6006ca70 0x7f0d600c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/1222719089 >> v2:172.21.15.163:3300/0 conn(0x7f0d600b0070 0x7f0d60155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 -- 172.21.15.17:0/1222719089 >> 172.21.15.17:0/1222719089 conn(0x7f0d600ac590 msgr2=0x7f0d600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 -- 172.21.15.17:0/1222719089 shutdown_connections 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.123+0000 7f0d67cb5640 1 -- 172.21.15.17:0/1222719089 wait complete. 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 Processor -- start 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 -- start start 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d600b0070 0x7f0d601eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d600c5380 0x7f0d60154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6006c550 con 0x7f0d6006ca70 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6006c270 con 0x7f0d600b0070 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.124+0000 7f0d67cb5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6006da80 con 0x7f0d600c5380 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56778/0 (socket says 172.21.15.17:56778) 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 -- 172.21.15.17:0/3573907955 learned_addr learned my addr 172.21.15.17:0/3573907955 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65250640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d600b0070 0x7f0d601eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 -- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d600c5380 msgr2=0x7f0d60154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d66252640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d600c5380 0x7f0d60154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d600c5380 0x7f0d60154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 -- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d600b0070 msgr2=0x7f0d601eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d600b0070 0x7f0d601eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 -- 172.21.15.17:0/3573907955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6014b4a0 con 0x7f0d6006ca70 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d65a51640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d800 tx=0x7f0d5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54004030 con 0x7f0d6006ca70 2024-03-06T07:21:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d540041d0 con 0x7f0d6006ca70 2024-03-06T07:21:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d601582d0 con 0x7f0d6006ca70 2024-03-06T07:21:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d540096a0 con 0x7f0d6006ca70 2024-03-06T07:21:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.125+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d60158890 con 0x7f0d6006ca70 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.126+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6004f330 con 0x7f0d6006ca70 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.127+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d54009840 con 0x7f0d6006ca70 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.127+0000 7f0d527fc640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 0x7f0d4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.128+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6530+0+0 (secure 0 0 0) 0x7f0d5408a790 con 0x7f0d6006ca70 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.128+0000 7f0d65250640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 0x7f0d4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:58.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.128+0000 7f0d65250640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 0x7f0d4406d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c006fd0 tx=0x7f0d5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.132+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d540592d0 con 0x7f0d6006ca70 2024-03-06T07:21:58.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.316+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0d6006d860 con 0x7f0d6006ca70 2024-03-06T07:21:58.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.994+0000 7f0d527fc640 1 -- 172.21.15.17:0/3573907955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v55) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0d54058bd0 con 0x7f0d6006ca70 2024-03-06T07:21:58.997 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' created 2024-03-06T07:21:59.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.998+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 msgr2=0x7f0d4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.998+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 0x7f0d4406d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c006fd0 tx=0x7f0d5c008040 comp rx=0 tx=0).stop 2024-03-06T07:21:59.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.998+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 msgr2=0x7f0d600c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.998+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d800 tx=0x7f0d5400dcc0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 shutdown_connections 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0d4406b500 0x7f0d4406d9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d600c5380 0x7f0d60154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d600b0070 0x7f0d601eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 --2- 172.21.15.17:0/3573907955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d6006ca70 0x7f0d600c4950 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 >> 172.21.15.17:0/3573907955 conn(0x7f0d600ac590 msgr2=0x7f0d600ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:58.999+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 shutdown_connections 2024-03-06T07:21:59.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.000+0000 7f0d67cb5640 1 -- 172.21.15.17:0/3573907955 wait complete. 2024-03-06T07:21:59.024 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:21:59.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.126+0000 7f34f2b3a640 1 Processor -- start 2024-03-06T07:21:59.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.126+0000 7f34f2b3a640 1 -- start start 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f34ec0c5330 0x7f34ec0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34ec0c88b0 0x7f34ec1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c7dd0 0x7f34ec158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec1559e0 con 0x7f34ec0c88b0 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ec155110 con 0x7f34ec0c5330 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f2b3a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0c8f40 con 0x7f34ec0c7dd0 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34ec0c88b0 0x7f34ec1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34ec0c88b0 0x7f34ec1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56792/0 (socket says 172.21.15.17:56792) 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 -- 172.21.15.17:0/3707934209 learned_addr learned my addr 172.21.15.17:0/3707934209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 -- 172.21.15.17:0/3707934209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c7dd0 msgr2=0x7f34ec158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f10d7640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c7dd0 0x7f34ec158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34f08d6640 1 --2- 172.21.15.17:0/3707934209 >> v2:172.21.15.163:3300/0 conn(0x7f34ec0c5330 0x7f34ec0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c7dd0 0x7f34ec158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 -- 172.21.15.17:0/3707934209 >> v2:172.21.15.163:3300/0 conn(0x7f34ec0c5330 msgr2=0x7f34ec0c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 --2- 172.21.15.17:0/3707934209 >> v2:172.21.15.163:3300/0 conn(0x7f34ec0c5330 0x7f34ec0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.127+0000 7f34ebfff640 1 -- 172.21.15.17:0/3707934209 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec155390 con 0x7f34ec0c88b0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.128+0000 7f34ebfff640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c88b0 0x7f34ec1586a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34ec08c6e0 tx=0x7f34e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae0584a52477a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.128+0000 7f34eb7fe640 1 -- 172.21.15.17:0/3707934209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e002e070 con 0x7f34ec0c88b0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.128+0000 7f34eb7fe640 1 -- 172.21.15.17:0/3707934209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0037070 con 0x7f34ec0c88b0 2024-03-06T07:21:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.128+0000 7f34eb7fe640 1 -- 172.21.15.17:0/3707934209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0031a00 con 0x7f34ec0c88b0 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 -- 172.21.15.17:0/3707934209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c88b0 msgr2=0x7f34ec1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c88b0 0x7f34ec1586a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f34ec08c6e0 tx=0x7f34e002b880 comp rx=0 tx=0).stop 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 -- 172.21.15.17:0/3707934209 shutdown_connections 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c7dd0 0x7f34ec158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/3707934209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c88b0 0x7f34ec1586a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/3707934209 >> v2:172.21.15.163:3300/0 conn(0x7f34ec0c5330 0x7f34ec0c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 -- 172.21.15.17:0/3707934209 >> 172.21.15.17:0/3707934209 conn(0x7f34ec0ac560 msgr2=0x7f34ec0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:21:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 -- 172.21.15.17:0/3707934209 shutdown_connections 2024-03-06T07:21:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.129+0000 7f34f2b3a640 1 -- 172.21.15.17:0/3707934209 wait complete. 2024-03-06T07:21:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.130+0000 7f34f2b3a640 1 Processor -- start 2024-03-06T07:21:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.130+0000 7f34f2b3a640 1 -- start start 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 0x7f34ec14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 0x7f34ec14bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec1529d0 con 0x7f34ec0c5330 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec06d090 con 0x7f34ec0c7dd0 2024-03-06T07:21:59.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f2b3a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec152b50 con 0x7f34ec0c88b0 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56798/0 (socket says 172.21.15.17:56798) 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 -- 172.21.15.17:0/2487258581 learned_addr learned my addr 172.21.15.17:0/2487258581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34ebfff640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 0x7f34ec14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f10d7640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 0x7f34ec14bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 -- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 msgr2=0x7f34ec14bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 0x7f34ec14bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 -- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 msgr2=0x7f34ec14b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.131+0000 7f34f08d6640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 0x7f34ec14b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34f08d6640 1 -- 172.21.15.17:0/2487258581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec14f460 con 0x7f34ec0c5330 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34ebfff640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 0x7f34ec14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34f10d7640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 0x7f34ec14bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34f08d6640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34e400cbb0 tx=0x7f34e400a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4015070 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4003b90 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec14c470 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e400d040 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.132+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec1579d0 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.133+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec067910 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.133+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34e400a920 con 0x7f34ec0c5330 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.134+0000 7f34e97fa640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 0x7f34d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:21:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.134+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6913+0+0 (secure 0 0 0) 0x7f34e408a8d0 con 0x7f34ec0c5330 2024-03-06T07:21:59.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.134+0000 7f34ebfff640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 0x7f34d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:21:59.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.135+0000 7f34ebfff640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 0x7f34d806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34e0036000 tx=0x7f34e0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:21:59.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.138+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34ec067910 con 0x7f34ec0c5330 2024-03-06T07:21:59.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:21:59.393+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f34ec04f330 con 0x7f34ec0c5330 2024-03-06T07:22:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.008+0000 7f34e97fa640 1 -- 172.21.15.17:0/2487258581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) v1 ==== 176+0+0 (secure 0 0 0) 0x7f34ec14c470 con 0x7f34ec0c5330 2024-03-06T07:22:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 9 and data pool 10 2024-03-06T07:22:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 msgr2=0x7f34d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 0x7f34d806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34e0036000 tx=0x7f34e0008030 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 msgr2=0x7f34ec1517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34e400cbb0 tx=0x7f34e400a0f0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 shutdown_connections 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34d806b4d0 0x7f34d806d990 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f34e0036000 tx=0x7f34e0008030 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34ec0c88b0 0x7f34ec14bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34ec0c7dd0 0x7f34ec14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 --2- 172.21.15.17:0/2487258581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34ec0c5330 0x7f34ec1517e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 >> 172.21.15.17:0/2487258581 conn(0x7f34ec0ac560 msgr2=0x7f34ec0c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.015+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 shutdown_connections 2024-03-06T07:22:00.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.018+0000 7f34f2b3a640 1 -- 172.21.15.17:0/2487258581 wait complete. 2024-03-06T07:22:00.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:00.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.138+0000 7f40d9647640 1 Processor -- start 2024-03-06T07:22:00.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.138+0000 7f40d9647640 1 -- start start 2024-03-06T07:22:00.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d9647640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 0x7f40d4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d9647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 0x7f40d4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d9647640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40c73d0 con 0x7f40d40c88c0 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d9647640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40c8dd0 con 0x7f40d414f360 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d9647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40be2e0 con 0x7f40d4158b60 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d27fc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f40d40c88c0 0x7f40d40c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:48208/0 (socket says 172.21.15.17:48208) 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d37fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 0x7f40d4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d27fc640 1 -- 172.21.15.17:0/424265352 learned_addr learned my addr 172.21.15.17:0/424265352 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.139+0000 7f40d2ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 0x7f40d4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d37fe640 1 -- 172.21.15.17:0/424265352 >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 msgr2=0x7f40d4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d37fe640 1 --2- 172.21.15.17:0/424265352 >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 0x7f40d4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d37fe640 1 -- 172.21.15.17:0/424265352 >> v1:172.21.15.163:6789/0 conn(0x7f40d40c88c0 legacy=0x7f40d40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d37fe640 1 -- 172.21.15.17:0/424265352 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40c7670 con 0x7f40d4158b60 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d1ffb640 1 -- 172.21.15.17:0/424265352 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 412440250 0 0) 0x7f40d40c73d0 con 0x7f40d40c88c0 2024-03-06T07:22:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.140+0000 7f40d2ffd640 1 --2- 172.21.15.17:0/424265352 >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 0x7f40d4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d37fe640 1 --2- 172.21.15.17:0/424265352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 0x7f40d4158ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009d90 tx=0x7f40cc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2449b83e388171 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d1ffb640 1 -- 172.21.15.17:0/424265352 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039070 con 0x7f40d4158b60 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d1ffb640 1 -- 172.21.15.17:0/424265352 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc02cbc0 con 0x7f40d4158b60 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d1ffb640 1 -- 172.21.15.17:0/424265352 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc034440 con 0x7f40d4158b60 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d9647640 1 -- 172.21.15.17:0/424265352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 msgr2=0x7f40d4158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d9647640 1 --2- 172.21.15.17:0/424265352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 0x7f40d4158ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009d90 tx=0x7f40cc02be10 comp rx=0 tx=0).stop 2024-03-06T07:22:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d9647640 1 -- 172.21.15.17:0/424265352 shutdown_connections 2024-03-06T07:22:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d9647640 1 --2- 172.21.15.17:0/424265352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d4158b60 0x7f40d4158ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.141+0000 7f40d9647640 1 --2- 172.21.15.17:0/424265352 >> v2:172.21.15.163:3300/0 conn(0x7f40d414f360 0x7f40d4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.142+0000 7f40d9647640 1 -- 172.21.15.17:0/424265352 >> 172.21.15.17:0/424265352 conn(0x7f40d40ac570 msgr2=0x7f40d40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.142+0000 7f40d9647640 1 -- 172.21.15.17:0/424265352 shutdown_connections 2024-03-06T07:22:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.142+0000 7f40d9647640 1 -- 172.21.15.17:0/424265352 wait complete. 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 Processor -- start 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 -- start start 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d40c88c0 0x7f40d40be700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d414f360 0x7f40d41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40c5a50 con 0x7f40d4158b60 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40c7c30 con 0x7f40d414f360 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.143+0000 7f40d9647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d40c5bd0 con 0x7f40d40c88c0 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56800/0 (socket says 172.21.15.17:56800) 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 -- 172.21.15.17:0/3271117924 learned_addr learned my addr 172.21.15.17:0/3271117924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d2ffd640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d40c88c0 0x7f40d40be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d27fc640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d414f360 0x7f40d41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 -- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d40c88c0 msgr2=0x7f40d40be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d40c88c0 0x7f40d40be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 -- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d414f360 msgr2=0x7f40d41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d414f360 0x7f40d41547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 -- 172.21.15.17:0/3271117924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d41ea9c0 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d37fe640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40cc037040 tx=0x7f40cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc039050 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc03a040 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d41eac80 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc009450 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.144+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d41eb240 con 0x7f40d4158b60 2024-03-06T07:22:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.145+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d414d040 con 0x7f40d4158b60 2024-03-06T07:22:00.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.146+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40cc047030 con 0x7f40d4158b60 2024-03-06T07:22:00.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.146+0000 7f40cb7fe640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 0x7f40b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.146+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f40cc0ac400 con 0x7f40d4158b60 2024-03-06T07:22:00.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.146+0000 7f40d2ffd640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 0x7f40b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.147+0000 7f40d2ffd640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 0x7f40b406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009560 tx=0x7f40bc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.151+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40d414d040 con 0x7f40d4158b60 2024-03-06T07:22:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.331+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d404f330 con 0x7f40d4158b60 2024-03-06T07:22:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.333+0000 7f40cb7fe640 1 -- 172.21.15.17:0/3271117924 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+20775 (secure 0 0 0) 0x7f40cc07a7a0 con 0x7f40d4158b60 2024-03-06T07:22:00.335 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:00.336 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":56,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:59.395482+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 msgr2=0x7f40b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 0x7f40b406d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009560 tx=0x7f40bc009190 comp rx=0 tx=0).stop 2024-03-06T07:22:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 msgr2=0x7f40d41e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f40cc037040 tx=0x7f40cc004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 shutdown_connections 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40b406b4d0 0x7f40b406d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40d4158b60 0x7f40d41e7dd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40d414f360 0x7f40d41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 --2- 172.21.15.17:0/3271117924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40d40c88c0 0x7f40d40be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.334+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 >> 172.21.15.17:0/3271117924 conn(0x7f40d40ac570 msgr2=0x7f40d406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.335+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 shutdown_connections 2024-03-06T07:22:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.335+0000 7f40d9647640 1 -- 172.21.15.17:0/3271117924 wait complete. 2024-03-06T07:22:00.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:00.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.447+0000 7f023cf8a640 1 Processor -- start 2024-03-06T07:22:00.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.447+0000 7f023cf8a640 1 -- start start 2024-03-06T07:22:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f023cf8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 0x7f0238159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f023cf8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02380c88b0 0x7f02380c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f023cf8a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02380c7c60 con 0x7f02381527b0 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f023cf8a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0238159640 con 0x7f02380c88b0 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f023cf8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02380c8060 con 0x7f02380c7180 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02380c88b0 0x7f02380c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02380c88b0 0x7f02380c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56814/0 (socket says 172.21.15.17:56814) 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 -- 172.21.15.17:0/745649348 learned_addr learned my addr 172.21.15.17:0/745649348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0235d9b640 1 --2- 172.21.15.17:0/745649348 >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 0x7f0238159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 -- 172.21.15.17:0/745649348 >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 msgr2=0x7f0238159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 --2- 172.21.15.17:0/745649348 >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 0x7f0238159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 -- 172.21.15.17:0/745649348 >> v1:172.21.15.163:6789/0 conn(0x7f02381527b0 legacy=0x7f0238155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0236d9d640 1 -- 172.21.15.17:0/745649348 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02380c76a0 con 0x7f02380c88b0 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.448+0000 7f0235d9b640 1 --2- 172.21.15.17:0/745649348 >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 0x7f0238159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.449+0000 7f0236d9d640 1 --2- 172.21.15.17:0/745649348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02380c88b0 0x7f02380c5330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f023808cac0 tx=0x7f023002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6586b4f3c78a9945 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.449+0000 7f023559a640 1 -- 172.21.15.17:0/745649348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023002c9f0 con 0x7f02380c88b0 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.449+0000 7f023559a640 1 -- 172.21.15.17:0/745649348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023002cb90 con 0x7f02380c88b0 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.449+0000 7f023559a640 1 -- 172.21.15.17:0/745649348 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300343f0 con 0x7f02380c88b0 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 -- 172.21.15.17:0/745649348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02380c88b0 msgr2=0x7f02380c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 --2- 172.21.15.17:0/745649348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02380c88b0 0x7f02380c5330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f023808cac0 tx=0x7f023002be50 comp rx=0 tx=0).stop 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 -- 172.21.15.17:0/745649348 shutdown_connections 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 --2- 172.21.15.17:0/745649348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02380c88b0 0x7f02380c5330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 --2- 172.21.15.17:0/745649348 >> v2:172.21.15.163:3300/0 conn(0x7f02380c7180 0x7f0238159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 -- 172.21.15.17:0/745649348 >> 172.21.15.17:0/745649348 conn(0x7f02380ac560 msgr2=0x7f02380ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 -- 172.21.15.17:0/745649348 shutdown_connections 2024-03-06T07:22:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.450+0000 7f023cf8a640 1 -- 172.21.15.17:0/745649348 wait complete. 2024-03-06T07:22:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.451+0000 7f023cf8a640 1 Processor -- start 2024-03-06T07:22:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 -- start start 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 0x7f023814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 0x7f0238148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238155600 con 0x7f02381527b0 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238155320 con 0x7f02380c7180 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f023cf8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0238152b90 con 0x7f02380c88b0 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f0236d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f0236d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56824/0 (socket says 172.21.15.17:56824) 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.452+0000 7f0236d9d640 1 -- 172.21.15.17:0/2280691724 learned_addr learned my addr 172.21.15.17:0/2280691724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0235d9b640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 0x7f0238148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 -- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 msgr2=0x7f0238148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f023659c640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 0x7f023814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 0x7f0238148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 -- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 msgr2=0x7f023814b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 0x7f023814b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 -- 172.21.15.17:0/2280691724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02380c2090 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0235d9b640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 0x7f0238148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f023659c640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 0x7f023814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0236d9d640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f40 tx=0x7f0230004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230043020 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023003e450 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0238153800 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230009510 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.453+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0238153240 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.454+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0238153440 con 0x7f02381527b0 2024-03-06T07:22:00.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.455+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0230046020 con 0x7f02381527b0 2024-03-06T07:22:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.455+0000 7f0226ffd640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 0x7f021806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.455+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f02300acf80 con 0x7f02381527b0 2024-03-06T07:22:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.455+0000 7f023659c640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 0x7f021806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.456+0000 7f023659c640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 0x7f021806d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f02380c1780 tx=0x7f0228008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.459+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0238153440 con 0x7f02381527b0 2024-03-06T07:22:00.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.750+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f023804f330 con 0x7f02381527b0 2024-03-06T07:22:00.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.751+0000 7f0226ffd640 1 -- 172.21.15.17:0/2280691724 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+20775 (secure 0 0 0) 0x7f023004d030 con 0x7f02381527b0 2024-03-06T07:22:00.756 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:00.756 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":56,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:21:59.395482+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 msgr2=0x7f021806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 0x7f021806d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f02380c1780 tx=0x7f0228008040 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 msgr2=0x7f0238149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f40 tx=0x7f0230004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 shutdown_connections 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f021806b4d0 0x7f021806d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02381527b0 0x7f0238149170 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02380c88b0 0x7f0238148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 --2- 172.21.15.17:0/2280691724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02380c7180 0x7f023814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.753+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 >> 172.21.15.17:0/2280691724 conn(0x7f02380ac560 msgr2=0x7f023806c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.754+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 shutdown_connections 2024-03-06T07:22:00.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.754+0000 7f023cf8a640 1 -- 172.21.15.17:0/2280691724 wait complete. 2024-03-06T07:22:00.772 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:22:00.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.873+0000 7fd51599a640 1 Processor -- start 2024-03-06T07:22:00.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.873+0000 7fd51599a640 1 -- start start 2024-03-06T07:22:00.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd51599a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5100b0070 0x7fd510155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd51599a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5100c5380 0x7fd5100c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd51599a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100b0450 con 0x7fd51006ca70 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd51599a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51006c020 con 0x7fd5100b0070 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd51599a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006cf20 con 0x7fd5100c5380 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd50effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5100b0070 0x7fd510155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd50effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5100b0070 0x7fd510155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35692/0 (socket says 172.21.15.17:35692) 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd50effd640 1 -- 172.21.15.17:0/327636363 learned_addr learned my addr 172.21.15.17:0/327636363 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.874+0000 7fd50f7fe640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5100c5380 0x7fd5100c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.875+0000 7fd50effd640 1 -- 172.21.15.17:0/327636363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5100c5380 msgr2=0x7fd5100c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.875+0000 7fd50effd640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5100c5380 0x7fd5100c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.875+0000 7fd50effd640 1 -- 172.21.15.17:0/327636363 >> v1:172.21.15.163:6789/0 conn(0x7fd51006ca70 legacy=0x7fd5100c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:00.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.875+0000 7fd50effd640 1 -- 172.21.15.17:0/327636363 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100c77f0 con 0x7fd5100b0070 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd50effd640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd510155c30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd500009870 tx=0x7fd50002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d33d376566d980 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd50e7fc640 1 -- 172.21.15.17:0/327636363 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500035020 con 0x7fd5100b0070 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd50e7fc640 1 -- 172.21.15.17:0/327636363 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd500031640 con 0x7fd5100b0070 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd50e7fc640 1 -- 172.21.15.17:0/327636363 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50002d040 con 0x7fd5100b0070 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd51599a640 1 -- 172.21.15.17:0/327636363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 msgr2=0x7fd510155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.876+0000 7fd51599a640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd510155c30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd500009870 tx=0x7fd50002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 -- 172.21.15.17:0/327636363 shutdown_connections 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5100c5380 0x7fd5100c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 --2- 172.21.15.17:0/327636363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd510155c30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 -- 172.21.15.17:0/327636363 >> 172.21.15.17:0/327636363 conn(0x7fd5100ac590 msgr2=0x7fd5100ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 -- 172.21.15.17:0/327636363 shutdown_connections 2024-03-06T07:22:00.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.877+0000 7fd51599a640 1 -- 172.21.15.17:0/327636363 wait complete. 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 Processor -- start 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 -- start start 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd51006ca70 0x7fd5101ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5100c5380 0x7fd51014bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006c550 con 0x7fd5100b0070 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006c270 con 0x7fd5100c5380 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd51599a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51006da80 con 0x7fd51006ca70 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd507fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd507fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35704/0 (socket says 172.21.15.17:35704) 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.878+0000 7fd507fff640 1 -- 172.21.15.17:0/153267294 learned_addr learned my addr 172.21.15.17:0/153267294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd50effd640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd51006ca70 0x7fd5101ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd50f7fe640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5100c5380 0x7fd51014bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 -- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd51006ca70 msgr2=0x7fd5101ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd51006ca70 0x7fd5101ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 -- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5100c5380 msgr2=0x7fd51014bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5100c5380 0x7fd51014bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 -- 172.21.15.17:0/153267294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100addc0 con 0x7fd5100b0070 2024-03-06T07:22:00.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd507fff640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c3c0 tx=0x7fd4f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80052d0 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8005490 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5101546c0 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8007260 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.879+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd510153900 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.880+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100678a0 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.881+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4f8007400 con 0x7fd5100b0070 2024-03-06T07:22:00.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.881+0000 7fd50cff9640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 0x7fd4f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.881+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fd4f8089a80 con 0x7fd5100b0070 2024-03-06T07:22:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.881+0000 7fd50effd640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 0x7fd4f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:00.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.882+0000 7fd50effd640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 0x7fd4f006d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd500036000 tx=0x7fd50003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:00.885+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5100678a0 con 0x7fd5100b0070 2024-03-06T07:22:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:01.110+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd5100ad6f0 con 0x7fd5100b0070 2024-03-06T07:22:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.019+0000 7fd50cff9640 1 -- 172.21.15.17:0/153267294 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 119+0+0 (secure 0 0 0) 0x7fd5100ad6f0 con 0x7fd5100b0070 2024-03-06T07:22:02.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 msgr2=0x7fd4f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:02.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 0x7fd4f006d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd500036000 tx=0x7fd50003c040 comp rx=0 tx=0).stop 2024-03-06T07:22:02.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 msgr2=0x7fd51014ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800c3c0 tx=0x7fd4f800c880 comp rx=0 tx=0).stop 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 shutdown_connections 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4f006b4d0 0x7fd4f006d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5100c5380 0x7fd51014bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5100b0070 0x7fd51014ba00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 --2- 172.21.15.17:0/153267294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd51006ca70 0x7fd5101ec3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 >> 172.21.15.17:0/153267294 conn(0x7fd5100ac590 msgr2=0x7fd5100c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 shutdown_connections 2024-03-06T07:22:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.022+0000 7fd51599a640 1 -- 172.21.15.17:0/153267294 wait complete. 2024-03-06T07:22:02.038 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:22:02.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:22:02.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 Processor -- start 2024-03-06T07:22:02.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 -- start start 2024-03-06T07:22:02.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4930155c30 0x7f493006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49300c5380 0x7f49300c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300b0450 con 0x7f49300b0070 2024-03-06T07:22:02.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006cfb0 con 0x7f4930155c30 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4934e4e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f493006c020 con 0x7f49300c5380 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.144+0000 7f4927fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4930155c30 0x7f493006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4930155c30 0x7f493006ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35716/0 (socket says 172.21.15.17:35716) 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 -- 172.21.15.17:0/3381320962 learned_addr learned my addr 172.21.15.17:0/3381320962 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f492ed9d640 1 --2- 172.21.15.17:0/3381320962 >> v2:172.21.15.163:3300/0 conn(0x7f49300c5380 0x7f49300c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 -- 172.21.15.17:0/3381320962 >> v2:172.21.15.163:3300/0 conn(0x7f49300c5380 msgr2=0x7f49300c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 --2- 172.21.15.17:0/3381320962 >> v2:172.21.15.163:3300/0 conn(0x7f49300c5380 0x7f49300c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 -- 172.21.15.17:0/3381320962 >> v1:172.21.15.163:6789/0 conn(0x7f49300b0070 legacy=0x7f49300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.145+0000 7f4927fff640 1 -- 172.21.15.17:0/3381320962 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49301560e0 con 0x7f4930155c30 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4927fff640 1 --2- 172.21.15.17:0/3381320962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f493006ca70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49300be030 tx=0x7f491802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc66f20463c3eb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f492dd9b640 1 -- 172.21.15.17:0/3381320962 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491802e070 con 0x7f4930155c30 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f492dd9b640 1 -- 172.21.15.17:0/3381320962 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4918037070 con 0x7f4930155c30 2024-03-06T07:22:02.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f492dd9b640 1 -- 172.21.15.17:0/3381320962 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918031a00 con 0x7f4930155c30 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 -- 172.21.15.17:0/3381320962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 msgr2=0x7f493006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3381320962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f493006ca70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49300be030 tx=0x7f491802b880 comp rx=0 tx=0).stop 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 -- 172.21.15.17:0/3381320962 shutdown_connections 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3381320962 >> v2:172.21.15.163:3300/0 conn(0x7f49300c5380 0x7f49300c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3381320962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f493006ca70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 -- 172.21.15.17:0/3381320962 >> 172.21.15.17:0/3381320962 conn(0x7f49300ac590 msgr2=0x7f49300ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:02.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 -- 172.21.15.17:0/3381320962 shutdown_connections 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.146+0000 7f4934e4e640 1 -- 172.21.15.17:0/3381320962 wait complete. 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.147+0000 7f4934e4e640 1 Processor -- start 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 -- start start 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49300b0070 0x7f49301e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49300c5380 0x7f4930158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493006c270 con 0x7f4930155c30 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300c7ad0 con 0x7f49300c5380 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4934e4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493006c550 con 0x7f49300b0070 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f492ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f492ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35720/0 (socket says 172.21.15.17:35720) 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f492ed9d640 1 -- 172.21.15.17:0/3441879143 learned_addr learned my addr 172.21.15.17:0/3441879143 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f4927fff640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49300c5380 0x7f4930158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.148+0000 7f492e59c640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49300b0070 0x7f49301e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 -- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49300b0070 msgr2=0x7f49301e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49300b0070 0x7f49301e7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 -- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49300c5380 msgr2=0x7f4930158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49300c5380 0x7f4930158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 -- 172.21.15.17:0/3441879143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49301e8410 con 0x7f4930155c30 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f492ed9d640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f492800cd70 tx=0x7f492800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928003b90 con 0x7f4930155c30 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928003d30 con 0x7f4930155c30 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49300c1ad0 con 0x7f4930155c30 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928004240 con 0x7f4930155c30 2024-03-06T07:22:02.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.149+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49300c2090 con 0x7f4930155c30 2024-03-06T07:22:02.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.150+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49300adcb0 con 0x7f4930155c30 2024-03-06T07:22:02.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.151+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49280043e0 con 0x7f4930155c30 2024-03-06T07:22:02.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.151+0000 7f4926ffd640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:02.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.151+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f492808b080 con 0x7f4930155c30 2024-03-06T07:22:02.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.151+0000 7f492e59c640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:02.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.152+0000 7f492e59c640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 0x7f491006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4920006fd0 tx=0x7f4920009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:02.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.156+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49300adcb0 con 0x7f4930155c30 2024-03-06T07:22:02.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:02.386+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f493006d860 con 0x7f4930155c30 2024-03-06T07:22:03.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.020+0000 7f4926ffd640 1 -- 172.21.15.17:0/3441879143 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4928059420 con 0x7f4930155c30 2024-03-06T07:22:03.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.022+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 msgr2=0x7f491006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.022+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 0x7f491006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4920006fd0 tx=0x7f4920009060 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.022+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 msgr2=0x7f49300c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.022+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f492800cd70 tx=0x7f492800a660 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 shutdown_connections 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f491006b4d0 0x7f491006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930155c30 0x7f49300c2b70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49300c5380 0x7f4930158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 --2- 172.21.15.17:0/3441879143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49300b0070 0x7f49301e7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 >> 172.21.15.17:0/3441879143 conn(0x7f49300ac590 msgr2=0x7f49300ab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 shutdown_connections 2024-03-06T07:22:03.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.023+0000 7f4934e4e640 1 -- 172.21.15.17:0/3441879143 wait complete. 2024-03-06T07:22:03.036 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:22:03.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.136+0000 7f0e26f5e640 1 Processor -- start 2024-03-06T07:22:03.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.136+0000 7f0e26f5e640 1 -- start start 2024-03-06T07:22:03.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e26f5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e2014f360 0x7f0e20155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e26f5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e200c88c0 0x7f0e200c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e26f5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e200c73d0 con 0x7f0e200c5880 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e26f5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200c8dd0 con 0x7f0e2014f360 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e26f5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200be2e0 con 0x7f0e200c88c0 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e2014f360 0x7f0e20155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e2014f360 0x7f0e20155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35722/0 (socket says 172.21.15.17:35722) 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 -- 172.21.15.17:0/1106252197 learned_addr learned my addr 172.21.15.17:0/1106252197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e1ffff640 1 --2- 172.21.15.17:0/1106252197 >> v2:172.21.15.163:3300/0 conn(0x7f0e200c88c0 0x7f0e200c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 -- 172.21.15.17:0/1106252197 >> v2:172.21.15.163:3300/0 conn(0x7f0e200c88c0 msgr2=0x7f0e200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 --2- 172.21.15.17:0/1106252197 >> v2:172.21.15.163:3300/0 conn(0x7f0e200c88c0 0x7f0e200c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 -- 172.21.15.17:0/1106252197 >> v1:172.21.15.163:6789/0 conn(0x7f0e200c5880 legacy=0x7f0e20158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:03.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.137+0000 7f0e24cfa640 1 -- 172.21.15.17:0/1106252197 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200c7670 con 0x7f0e2014f360 2024-03-06T07:22:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e24cfa640 1 --2- 172.21.15.17:0/1106252197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e2014f360 0x7f0e20155c10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0e200c84c0 tx=0x7f0e1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f36bdafe8e8509 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e1f7fe640 1 -- 172.21.15.17:0/1106252197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402e070 con 0x7f0e2014f360 2024-03-06T07:22:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e1f7fe640 1 -- 172.21.15.17:0/1106252197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14037070 con 0x7f0e2014f360 2024-03-06T07:22:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e1f7fe640 1 -- 172.21.15.17:0/1106252197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14031a00 con 0x7f0e2014f360 2024-03-06T07:22:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 -- 172.21.15.17:0/1106252197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e2014f360 msgr2=0x7f0e20155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/1106252197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e2014f360 0x7f0e20155c10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0e200c84c0 tx=0x7f0e1402b880 comp rx=0 tx=0).stop 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 -- 172.21.15.17:0/1106252197 shutdown_connections 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/1106252197 >> v2:172.21.15.163:3300/0 conn(0x7f0e200c88c0 0x7f0e200c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/1106252197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e2014f360 0x7f0e20155c10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 -- 172.21.15.17:0/1106252197 >> 172.21.15.17:0/1106252197 conn(0x7f0e200ac570 msgr2=0x7f0e200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 -- 172.21.15.17:0/1106252197 shutdown_connections 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.139+0000 7f0e26f5e640 1 -- 172.21.15.17:0/1106252197 wait complete. 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.140+0000 7f0e26f5e640 1 Processor -- start 2024-03-06T07:22:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.140+0000 7f0e26f5e640 1 -- start start 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c5880 0x7f0e200bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e2014f360 0x7f0e201eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20159270 con 0x7f0e200c88c0 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200c7c30 con 0x7f0e200c5880 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e26f5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201593f0 con 0x7f0e2014f360 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35730/0 (socket says 172.21.15.17:35730) 2024-03-06T07:22:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e1ffff640 1 -- 172.21.15.17:0/926617358 learned_addr learned my addr 172.21.15.17:0/926617358 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.141+0000 7f0e24cfa640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c5880 0x7f0e200bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e254fb640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e2014f360 0x7f0e201eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 -- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e2014f360 msgr2=0x7f0e201eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e2014f360 0x7f0e201eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 -- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c5880 msgr2=0x7f0e200bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c5880 0x7f0e200bf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 -- 172.21.15.17:0/926617358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200c06e0 con 0x7f0e200c88c0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1ffff640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00d800 tx=0x7f0e0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00b6c0 con 0x7f0e200c88c0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c019070 con 0x7f0e200c88c0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00ba80 con 0x7f0e200c88c0 2024-03-06T07:22:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e200c09c0 con 0x7f0e200c88c0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.142+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e200bf0a0 con 0x7f0e200c88c0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.143+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e20152fc0 con 0x7f0e200c88c0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.144+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e0c00bc20 con 0x7f0e200c88c0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.144+0000 7f0e1d7fa640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 0x7f0e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.144+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f0e0c08a9a0 con 0x7f0e200c88c0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.145+0000 7f0e24cfa640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 0x7f0e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.145+0000 7f0e24cfa640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 0x7f0e0406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0e14036000 tx=0x7f0e1403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.149+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e20152fc0 con 0x7f0e200c88c0 2024-03-06T07:22:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.416+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e2004f330 con 0x7f0e200c88c0 2024-03-06T07:22:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.416+0000 7f0e1d7fa640 1 -- 172.21.15.17:0/926617358 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+6259 (secure 0 0 0) 0x7f0e0c058d40 con 0x7f0e200c88c0 2024-03-06T07:22:03.420 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:03.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5587,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:21:46.263408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5772":{"gid":5772,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1002585815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1002585815},{"type":"v1","addr":"172.21.15.17:6835","nonce":1002585815}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5772,"qdb_cluster":[5772]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 36 2024-03-06T07:22:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.418+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 msgr2=0x7f0e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.418+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 0x7f0e0406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0e14036000 tx=0x7f0e1403d040 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.418+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 msgr2=0x7f0e201eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.418+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00d800 tx=0x7f0e0c00dcc0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 shutdown_connections 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e0406b4d0 0x7f0e0406d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e2014f360 0x7f0e201eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e200c88c0 0x7f0e201eab60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 --2- 172.21.15.17:0/926617358 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e200c5880 0x7f0e200bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 >> 172.21.15.17:0/926617358 conn(0x7f0e200ac570 msgr2=0x7f0e200ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 shutdown_connections 2024-03-06T07:22:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.419+0000 7f0e26f5e640 1 -- 172.21.15.17:0/926617358 wait complete. 2024-03-06T07:22:03.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.532+0000 7f8ca4c84640 1 Processor -- start 2024-03-06T07:22:03.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.532+0000 7f8ca4c84640 1 -- start start 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8ca4c84640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ca00c0df0 0x7f8ca006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8ca4c84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca0155f80 con 0x7f8ca00c05e0 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8ca4c84640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00acea0 con 0x7f8ca0152a60 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8ca4c84640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca0155a50 con 0x7f8ca00c0df0 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9e59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8ca00c05e0 0x7f8ca0152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55646/0 (socket says 172.21.15.17:55646) 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9e59c640 1 -- 172.21.15.17:0/1077341176 learned_addr learned my addr 172.21.15.17:0/1077341176 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9ed9d640 1 --2- 172.21.15.17:0/1077341176 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00c0df0 0x7f8ca006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714314826 0 0) 0x7f8ca0155f80 con 0x7f8ca00c05e0 2024-03-06T07:22:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c88003520 con 0x7f8ca00c05e0 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663485261 0 0) 0x7f8ca00acea0 con 0x7f8ca0152a60 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca0155f80 con 0x7f8ca0152a60 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9ed9d640 1 -- 172.21.15.17:0/1077341176 >> v1:172.21.15.163:6789/0 conn(0x7f8ca0152a60 legacy=0x7f8ca006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9ed9d640 1 -- 172.21.15.17:0/1077341176 >> v1:172.21.15.17:6789/0 conn(0x7f8ca00c05e0 legacy=0x7f8ca0152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.534+0000 7f8c9ed9d640 1 -- 172.21.15.17:0/1077341176 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0155ca0 con 0x7f8ca00c0df0 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.535+0000 7f8c9ed9d640 1 --2- 172.21.15.17:0/1077341176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c0df0 0x7f8ca006cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097a0 tx=0x7f8c9802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54b6caea06adc322 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.535+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980315a0 con 0x7f8ca00c0df0 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.535+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98037070 con 0x7f8ca00c0df0 2024-03-06T07:22:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.535+0000 7f8c9d59a640 1 -- 172.21.15.17:0/1077341176 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98031960 con 0x7f8ca00c0df0 2024-03-06T07:22:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 -- 172.21.15.17:0/1077341176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c0df0 msgr2=0x7f8ca006cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/1077341176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c0df0 0x7f8ca006cae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097a0 tx=0x7f8c9802b840 comp rx=0 tx=0).stop 2024-03-06T07:22:03.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 -- 172.21.15.17:0/1077341176 shutdown_connections 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/1077341176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c0df0 0x7f8ca006cae0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 -- 172.21.15.17:0/1077341176 >> 172.21.15.17:0/1077341176 conn(0x7f8ca006bf00 msgr2=0x7f8ca00c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 -- 172.21.15.17:0/1077341176 shutdown_connections 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.536+0000 7f8ca4c84640 1 -- 172.21.15.17:0/1077341176 wait complete. 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.537+0000 7f8ca4c84640 1 Processor -- start 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 -- start start 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 0x7f8ca00c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca0152a60 0x7f8ca0154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00c36b0 con 0x7f8ca00c0df0 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00acb50 con 0x7f8ca00c05e0 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8ca4c84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca0073110 con 0x7f8ca0152a60 2024-03-06T07:22:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8c9dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8c9dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35732/0 (socket says 172.21.15.17:35732) 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.538+0000 7f8c9dd9b640 1 -- 172.21.15.17:0/2300762810 learned_addr learned my addr 172.21.15.17:0/2300762810 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9e59c640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 0x7f8ca00c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 -- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca0152a60 msgr2=0x7f8ca0154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca0152a60 0x7f8ca0154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 -- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 msgr2=0x7f8ca00c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 0x7f8ca00c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 -- 172.21.15.17:0/2300762810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00c5790 con 0x7f8ca00c0df0 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9e59c640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 0x7f8ca00c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c9dd9b640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c500 tx=0x7f8c9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c940052d0 con 0x7f8ca00c0df0 2024-03-06T07:22:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94005490 con 0x7f8ca00c0df0 2024-03-06T07:22:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca00c5a70 con 0x7f8ca00c0df0 2024-03-06T07:22:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94012870 con 0x7f8ca00c0df0 2024-03-06T07:22:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.539+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca0157b60 con 0x7f8ca00c0df0 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.540+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca004f330 con 0x7f8ca00c0df0 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.541+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c94012a10 con 0x7f8ca00c0df0 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.541+0000 7f8c8effd640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 0x7f8c8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.541+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f8c9408a3e0 con 0x7f8ca00c0df0 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.542+0000 7f8c9e59c640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 0x7f8c8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.542+0000 7f8c9e59c640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 0x7f8c8006d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c90001560 tx=0x7f8c90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.546+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c94058d50 con 0x7f8ca00c0df0 2024-03-06T07:22:03.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.743+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ca00c3830 con 0x7f8ca00c0df0 2024-03-06T07:22:03.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.745+0000 7f8c8effd640 1 -- 172.21.15.17:0/2300762810 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+20762 (secure 0 0 0) 0x7f8c94075e20 con 0x7f8ca00c0df0 2024-03-06T07:22:03.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:03.748 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":57,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:00.997286+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 msgr2=0x7f8c8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 0x7f8c8006d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c90001560 tx=0x7f8c90009040 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 msgr2=0x7f8ca00709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c500 tx=0x7f8c9400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 shutdown_connections 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8006b500 0x7f8c8006d9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca0152a60 0x7f8ca0154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ca00c0df0 0x7f8ca00709f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 --2- 172.21.15.17:0/2300762810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00c05e0 0x7f8ca00c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 >> 172.21.15.17:0/2300762810 conn(0x7f8ca006bf00 msgr2=0x7f8ca006edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 shutdown_connections 2024-03-06T07:22:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.748+0000 7f8ca4c84640 1 -- 172.21.15.17:0/2300762810 wait complete. 2024-03-06T07:22:03.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:22:03.762 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.862+0000 7fa6b1f72640 1 Processor -- start 2024-03-06T07:22:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.862+0000 7fa6b1f72640 1 -- start start 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6b1f72640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa6ac06afc0 0x7fa6ac06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6b1f72640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa6ac06b3a0 0x7fa6ac152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6b1f72640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac153940 con 0x7fa6ac1534f0 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6b1f72640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac06b830 con 0x7fa6ac06afc0 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6b1f72640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac150a20 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa6ac06afc0 0x7fa6ac06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa6ac06afc0 0x7fa6ac06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35746/0 (socket says 172.21.15.17:35746) 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6ab7fe640 1 -- 172.21.15.17:0/1735056114 learned_addr learned my addr 172.21.15.17:0/1735056114 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6aaffd640 1 --2- 172.21.15.17:0/1735056114 >> v2:172.21.15.163:3300/0 conn(0x7fa6ac06b3a0 0x7fa6ac152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6ab7fe640 1 -- 172.21.15.17:0/1735056114 >> v2:172.21.15.163:3300/0 conn(0x7fa6ac06b3a0 msgr2=0x7fa6ac152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.863+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/1735056114 >> v2:172.21.15.163:3300/0 conn(0x7fa6ac06b3a0 0x7fa6ac152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.864+0000 7fa6ab7fe640 1 -- 172.21.15.17:0/1735056114 >> v1:172.21.15.163:6789/0 conn(0x7fa6ac1534f0 legacy=0x7fa6ac0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.864+0000 7fa6ab7fe640 1 -- 172.21.15.17:0/1735056114 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0a70f0 con 0x7fa6ac06afc0 2024-03-06T07:22:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/1735056114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac06f100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009870 tx=0x7fa69c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9112af00fef679a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6aa7fc640 1 -- 172.21.15.17:0/1735056114 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c038070 con 0x7fa6ac06afc0 2024-03-06T07:22:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6aa7fc640 1 -- 172.21.15.17:0/1735056114 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c02f050 con 0x7fa6ac06afc0 2024-03-06T07:22:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6aa7fc640 1 -- 172.21.15.17:0/1735056114 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c031970 con 0x7fa6ac06afc0 2024-03-06T07:22:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 -- 172.21.15.17:0/1735056114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06afc0 msgr2=0x7fa6ac06f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/1735056114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac06f100 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009870 tx=0x7fa69c02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 -- 172.21.15.17:0/1735056114 shutdown_connections 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/1735056114 >> v2:172.21.15.163:3300/0 conn(0x7fa6ac06b3a0 0x7fa6ac152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/1735056114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac06f100 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 -- 172.21.15.17:0/1735056114 >> 172.21.15.17:0/1735056114 conn(0x7fa6ac0c4290 msgr2=0x7fa6ac0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 -- 172.21.15.17:0/1735056114 shutdown_connections 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.865+0000 7fa6b1f72640 1 -- 172.21.15.17:0/1735056114 wait complete. 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.866+0000 7fa6b1f72640 1 Processor -- start 2024-03-06T07:22:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 -- start start 2024-03-06T07:22:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac0c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 0x7fa6ac14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac156ad0 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac1565a0 con 0x7fa6ac06afc0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6b1f72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac156db0 con 0x7fa6ac1534f0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.867+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35748/0 (socket says 172.21.15.17:35748) 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 -- 172.21.15.17:0/3771400154 learned_addr learned my addr 172.21.15.17:0/3771400154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6abfff640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 0x7fa6ac14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac0c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 -- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 msgr2=0x7fa6ac14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 0x7fa6ac14d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 -- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 msgr2=0x7fa6ac0c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac0c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 -- 172.21.15.17:0/3771400154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac1532c0 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac0c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6abfff640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 0x7fa6ac14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6aaffd640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0015070 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a0004220 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac1df330 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0007620 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.868+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac073040 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.870+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac04f330 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.870+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6a00079d0 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.870+0000 7fa693fff640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 0x7fa69806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.871+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fa6a008a770 con 0x7fa6ac06b3a0 2024-03-06T07:22:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.871+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 0x7fa69806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.871+0000 7fa6ab7fe640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 0x7fa69806d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa69c035000 tx=0x7fa69c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:03.876+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa6a0059030 con 0x7fa6ac06b3a0 2024-03-06T07:22:04.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.134+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6ac1df610 con 0x7fa69806b4d0 2024-03-06T07:22:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.146+0000 7fa693fff640 1 -- 172.21.15.17:0/3771400154 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490559 (secure 0 0 0) 0x7fa6ac1df610 con 0x7fa69806b4d0 2024-03-06T07:22:04.154 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:04.158 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.150+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 msgr2=0x7fa69806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.150+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 0x7fa69806d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa69c035000 tx=0x7fa69c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.150+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 msgr2=0x7fa6ac14d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.150+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000c7f0 tx=0x7fa6a000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 shutdown_connections 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa69806b4d0 0x7fa69806d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6ac1534f0 0x7fa6ac14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6ac06b3a0 0x7fa6ac14d010 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 --2- 172.21.15.17:0/3771400154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6ac06afc0 0x7fa6ac0c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 >> 172.21.15.17:0/3771400154 conn(0x7fa6ac0c4290 msgr2=0x7fa6ac0a74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 shutdown_connections 2024-03-06T07:22:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.151+0000 7fa6b1f72640 1 -- 172.21.15.17:0/3771400154 wait complete. 2024-03-06T07:22:04.165 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-06T07:22:03.367492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":343,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238404,"kb_used_data":5556,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736316,"statfs":{"total":773094113280,"available":772849987584,"internally_reserved":0,"allocated":5689344,"data_stored":3707385,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.375849"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150427+0000","last_change":"2024-03-06T07:22:00.021272+0000","last_active":"2024-03-06T07:22:01.150427+0000","last_peered":"2024-03-06T07:22:01.150427+0000","last_clean":"2024-03-06T07:22:01.150427+0000","last_became_active":"2024-03-06T07:22:00.021015+0000","last_became_peered":"2024-03-06T07:22:00.021015+0000","last_unstale":"2024-03-06T07:22:01.150427+0000","last_undegraded":"2024-03-06T07:22:01.150427+0000","last_fullsized":"2024-03-06T07:22:01.150427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001472+0000","last_change":"2024-03-06T07:21:43.913948+0000","last_active":"2024-03-06T07:22:01.001472+0000","last_peered":"2024-03-06T07:22:01.001472+0000","last_clean":"2024-03-06T07:22:01.001472+0000","last_became_active":"2024-03-06T07:21:43.911415+0000","last_became_peered":"2024-03-06T07:21:43.911415+0000","last_unstale":"2024-03-06T07:22:01.001472+0000","last_undegraded":"2024-03-06T07:22:01.001472+0000","last_fullsized":"2024-03-06T07:22:01.001472+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:51.582422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003320+0000","last_change":"2024-03-06T07:22:00.011088+0000","last_active":"2024-03-06T07:22:01.003320+0000","last_peered":"2024-03-06T07:22:01.003320+0000","last_clean":"2024-03-06T07:22:01.003320+0000","last_became_active":"2024-03-06T07:22:00.010827+0000","last_became_peered":"2024-03-06T07:22:00.010827+0000","last_unstale":"2024-03-06T07:22:01.003320+0000","last_undegraded":"2024-03-06T07:22:01.003320+0000","last_fullsized":"2024-03-06T07:22:01.003320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:23.455544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150526+0000","last_change":"2024-03-06T07:21:43.914337+0000","last_active":"2024-03-06T07:22:01.150526+0000","last_peered":"2024-03-06T07:22:01.150526+0000","last_clean":"2024-03-06T07:22:01.150526+0000","last_became_active":"2024-03-06T07:21:43.914043+0000","last_became_peered":"2024-03-06T07:21:43.914043+0000","last_unstale":"2024-03-06T07:22:01.150526+0000","last_undegraded":"2024-03-06T07:22:01.150526+0000","last_fullsized":"2024-03-06T07:22:01.150526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.432948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001336+0000","last_change":"2024-03-06T07:22:00.017320+0000","last_active":"2024-03-06T07:22:01.001336+0000","last_peered":"2024-03-06T07:22:01.001336+0000","last_clean":"2024-03-06T07:22:01.001336+0000","last_became_active":"2024-03-06T07:22:00.016927+0000","last_became_peered":"2024-03-06T07:22:00.016927+0000","last_unstale":"2024-03-06T07:22:01.001336+0000","last_undegraded":"2024-03-06T07:22:01.001336+0000","last_fullsized":"2024-03-06T07:22:01.001336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:10.016374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642785+0000","last_change":"2024-03-06T07:21:43.935409+0000","last_active":"2024-03-06T07:22:01.642785+0000","last_peered":"2024-03-06T07:22:01.642785+0000","last_clean":"2024-03-06T07:22:01.642785+0000","last_became_active":"2024-03-06T07:21:43.934832+0000","last_became_peered":"2024-03-06T07:21:43.934832+0000","last_unstale":"2024-03-06T07:22:01.642785+0000","last_undegraded":"2024-03-06T07:22:01.642785+0000","last_fullsized":"2024-03-06T07:22:01.642785+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:57.737436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001262+0000","last_change":"2024-03-06T07:22:00.012621+0000","last_active":"2024-03-06T07:22:01.001262+0000","last_peered":"2024-03-06T07:22:01.001262+0000","last_clean":"2024-03-06T07:22:01.001262+0000","last_became_active":"2024-03-06T07:22:00.012323+0000","last_became_peered":"2024-03-06T07:22:00.012323+0000","last_unstale":"2024-03-06T07:22:01.001262+0000","last_undegraded":"2024-03-06T07:22:01.001262+0000","last_fullsized":"2024-03-06T07:22:01.001262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:30.690160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150223+0000","last_change":"2024-03-06T07:21:43.913875+0000","last_active":"2024-03-06T07:22:01.150223+0000","last_peered":"2024-03-06T07:22:01.150223+0000","last_clean":"2024-03-06T07:22:01.150223+0000","last_became_active":"2024-03-06T07:21:43.912314+0000","last_became_peered":"2024-03-06T07:21:43.912314+0000","last_unstale":"2024-03-06T07:22:01.150223+0000","last_undegraded":"2024-03-06T07:22:01.150223+0000","last_fullsized":"2024-03-06T07:22:01.150223+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:01.595661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027227+0000","last_change":"2024-03-06T07:22:00.019347+0000","last_active":"2024-03-06T07:22:01.027227+0000","last_peered":"2024-03-06T07:22:01.027227+0000","last_clean":"2024-03-06T07:22:01.027227+0000","last_became_active":"2024-03-06T07:22:00.019096+0000","last_became_peered":"2024-03-06T07:22:00.019096+0000","last_unstale":"2024-03-06T07:22:01.027227+0000","last_undegraded":"2024-03-06T07:22:01.027227+0000","last_fullsized":"2024-03-06T07:22:01.027227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:19.143693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002353+0000","last_change":"2024-03-06T07:21:43.913216+0000","last_active":"2024-03-06T07:22:01.002353+0000","last_peered":"2024-03-06T07:22:01.002353+0000","last_clean":"2024-03-06T07:22:01.002353+0000","last_became_active":"2024-03-06T07:21:43.912935+0000","last_became_peered":"2024-03-06T07:21:43.912935+0000","last_unstale":"2024-03-06T07:22:01.002353+0000","last_undegraded":"2024-03-06T07:22:01.002353+0000","last_fullsized":"2024-03-06T07:22:01.002353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024256+0000","last_change":"2024-03-06T07:22:00.020457+0000","last_active":"2024-03-06T07:22:01.024256+0000","last_peered":"2024-03-06T07:22:01.024256+0000","last_clean":"2024-03-06T07:22:01.024256+0000","last_became_active":"2024-03-06T07:22:00.020191+0000","last_became_peered":"2024-03-06T07:22:00.020191+0000","last_unstale":"2024-03-06T07:22:01.024256+0000","last_undegraded":"2024-03-06T07:22:01.024256+0000","last_fullsized":"2024-03-06T07:22:01.024256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.380776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000860+0000","last_change":"2024-03-06T07:21:43.917357+0000","last_active":"2024-03-06T07:22:01.000860+0000","last_peered":"2024-03-06T07:22:01.000860+0000","last_clean":"2024-03-06T07:22:01.000860+0000","last_became_active":"2024-03-06T07:21:43.916973+0000","last_became_peered":"2024-03-06T07:21:43.916973+0000","last_unstale":"2024-03-06T07:22:01.000860+0000","last_undegraded":"2024-03-06T07:22:01.000860+0000","last_fullsized":"2024-03-06T07:22:01.000860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002044+0000","last_change":"2024-03-06T07:22:00.013447+0000","last_active":"2024-03-06T07:22:01.002044+0000","last_peered":"2024-03-06T07:22:01.002044+0000","last_clean":"2024-03-06T07:22:01.002044+0000","last_became_active":"2024-03-06T07:22:00.013137+0000","last_became_peered":"2024-03-06T07:22:00.013137+0000","last_unstale":"2024-03-06T07:22:01.002044+0000","last_undegraded":"2024-03-06T07:22:01.002044+0000","last_fullsized":"2024-03-06T07:22:01.002044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:52.437815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001803+0000","last_change":"2024-03-06T07:21:54.169067+0000","last_active":"2024-03-06T07:22:01.001803+0000","last_peered":"2024-03-06T07:22:01.001803+0000","last_clean":"2024-03-06T07:22:01.001803+0000","last_became_active":"2024-03-06T07:21:54.168778+0000","last_became_peered":"2024-03-06T07:21:54.168778+0000","last_unstale":"2024-03-06T07:22:01.001803+0000","last_undegraded":"2024-03-06T07:22:01.001803+0000","last_fullsized":"2024-03-06T07:22:01.001803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643264+0000","last_change":"2024-03-06T07:22:00.018830+0000","last_active":"2024-03-06T07:22:01.643264+0000","last_peered":"2024-03-06T07:22:01.643264+0000","last_clean":"2024-03-06T07:22:01.643264+0000","last_became_active":"2024-03-06T07:22:00.018563+0000","last_became_peered":"2024-03-06T07:22:00.018563+0000","last_unstale":"2024-03-06T07:22:01.643264+0000","last_undegraded":"2024-03-06T07:22:01.643264+0000","last_fullsized":"2024-03-06T07:22:01.643264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:47:32.925919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642260+0000","last_change":"2024-03-06T07:21:43.928597+0000","last_active":"2024-03-06T07:22:01.642260+0000","last_peered":"2024-03-06T07:22:01.642260+0000","last_clean":"2024-03-06T07:22:01.642260+0000","last_became_active":"2024-03-06T07:21:43.928330+0000","last_became_peered":"2024-03-06T07:21:43.928330+0000","last_unstale":"2024-03-06T07:22:01.642260+0000","last_undegraded":"2024-03-06T07:22:01.642260+0000","last_fullsized":"2024-03-06T07:22:01.642260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002363+0000","last_change":"2024-03-06T07:22:00.013119+0000","last_active":"2024-03-06T07:22:01.002363+0000","last_peered":"2024-03-06T07:22:01.002363+0000","last_clean":"2024-03-06T07:22:01.002363+0000","last_became_active":"2024-03-06T07:22:00.012851+0000","last_became_peered":"2024-03-06T07:22:00.012851+0000","last_unstale":"2024-03-06T07:22:01.002363+0000","last_undegraded":"2024-03-06T07:22:01.002363+0000","last_fullsized":"2024-03-06T07:22:01.002363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:31.441532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643975+0000","last_change":"2024-03-06T07:21:43.938313+0000","last_active":"2024-03-06T07:22:01.643975+0000","last_peered":"2024-03-06T07:22:01.643975+0000","last_clean":"2024-03-06T07:22:01.643975+0000","last_became_active":"2024-03-06T07:21:43.938037+0000","last_became_peered":"2024-03-06T07:21:43.938037+0000","last_unstale":"2024-03-06T07:22:01.643975+0000","last_undegraded":"2024-03-06T07:22:01.643975+0000","last_fullsized":"2024-03-06T07:22:01.643975+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:12.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024615+0000","last_change":"2024-03-06T07:22:00.016705+0000","last_active":"2024-03-06T07:22:01.024615+0000","last_peered":"2024-03-06T07:22:01.024615+0000","last_clean":"2024-03-06T07:22:01.024615+0000","last_became_active":"2024-03-06T07:22:00.016416+0000","last_became_peered":"2024-03-06T07:22:00.016416+0000","last_unstale":"2024-03-06T07:22:01.024615+0000","last_undegraded":"2024-03-06T07:22:01.024615+0000","last_fullsized":"2024-03-06T07:22:01.024615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:13.376587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001317+0000","last_change":"2024-03-06T07:21:43.910022+0000","last_active":"2024-03-06T07:22:01.001317+0000","last_peered":"2024-03-06T07:22:01.001317+0000","last_clean":"2024-03-06T07:22:01.001317+0000","last_became_active":"2024-03-06T07:21:43.909753+0000","last_became_peered":"2024-03-06T07:21:43.909753+0000","last_unstale":"2024-03-06T07:22:01.001317+0000","last_undegraded":"2024-03-06T07:22:01.001317+0000","last_fullsized":"2024-03-06T07:22:01.001317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:14.243609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150533+0000","last_change":"2024-03-06T07:22:00.031071+0000","last_active":"2024-03-06T07:22:01.150533+0000","last_peered":"2024-03-06T07:22:01.150533+0000","last_clean":"2024-03-06T07:22:01.150533+0000","last_became_active":"2024-03-06T07:22:00.030369+0000","last_became_peered":"2024-03-06T07:22:00.030369+0000","last_unstale":"2024-03-06T07:22:01.150533+0000","last_undegraded":"2024-03-06T07:22:01.150533+0000","last_fullsized":"2024-03-06T07:22:01.150533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:27.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027933+0000","last_change":"2024-03-06T07:21:43.917615+0000","last_active":"2024-03-06T07:22:01.027933+0000","last_peered":"2024-03-06T07:22:01.027933+0000","last_clean":"2024-03-06T07:22:01.027933+0000","last_became_active":"2024-03-06T07:21:43.917363+0000","last_became_peered":"2024-03-06T07:21:43.917363+0000","last_unstale":"2024-03-06T07:22:01.027933+0000","last_undegraded":"2024-03-06T07:22:01.027933+0000","last_fullsized":"2024-03-06T07:22:01.027933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:29.125938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150411+0000","last_change":"2024-03-06T07:22:00.023374+0000","last_active":"2024-03-06T07:22:01.150411+0000","last_peered":"2024-03-06T07:22:01.150411+0000","last_clean":"2024-03-06T07:22:01.150411+0000","last_became_active":"2024-03-06T07:22:00.022882+0000","last_became_peered":"2024-03-06T07:22:00.022882+0000","last_unstale":"2024-03-06T07:22:01.150411+0000","last_undegraded":"2024-03-06T07:22:01.150411+0000","last_fullsized":"2024-03-06T07:22:01.150411+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:13.376587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643701+0000","last_change":"2024-03-06T07:21:43.911518+0000","last_active":"2024-03-06T07:22:01.643701+0000","last_peered":"2024-03-06T07:22:01.643701+0000","last_clean":"2024-03-06T07:22:01.643701+0000","last_became_active":"2024-03-06T07:21:43.911238+0000","last_became_peered":"2024-03-06T07:21:43.911238+0000","last_unstale":"2024-03-06T07:22:01.643701+0000","last_undegraded":"2024-03-06T07:22:01.643701+0000","last_fullsized":"2024-03-06T07:22:01.643701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:14.588826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149924+0000","last_change":"2024-03-06T07:22:00.018284+0000","last_active":"2024-03-06T07:22:01.149924+0000","last_peered":"2024-03-06T07:22:01.149924+0000","last_clean":"2024-03-06T07:22:01.149924+0000","last_became_active":"2024-03-06T07:22:00.017996+0000","last_became_peered":"2024-03-06T07:22:00.017996+0000","last_unstale":"2024-03-06T07:22:01.149924+0000","last_undegraded":"2024-03-06T07:22:01.149924+0000","last_fullsized":"2024-03-06T07:22:01.149924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.380776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000595+0000","last_change":"2024-03-06T07:21:43.916170+0000","last_active":"2024-03-06T07:22:01.000595+0000","last_peered":"2024-03-06T07:22:01.000595+0000","last_clean":"2024-03-06T07:22:01.000595+0000","last_became_active":"2024-03-06T07:21:43.915661+0000","last_became_peered":"2024-03-06T07:21:43.915661+0000","last_unstale":"2024-03-06T07:22:01.000595+0000","last_undegraded":"2024-03-06T07:22:01.000595+0000","last_fullsized":"2024-03-06T07:22:01.000595+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027091+0000","last_change":"2024-03-06T07:22:00.016462+0000","last_active":"2024-03-06T07:22:01.027091+0000","last_peered":"2024-03-06T07:22:01.027091+0000","last_clean":"2024-03-06T07:22:01.027091+0000","last_became_active":"2024-03-06T07:22:00.016021+0000","last_became_peered":"2024-03-06T07:22:00.016021+0000","last_unstale":"2024-03-06T07:22:01.027091+0000","last_undegraded":"2024-03-06T07:22:01.027091+0000","last_fullsized":"2024-03-06T07:22:01.027091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:17.113207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643337+0000","last_change":"2024-03-06T07:21:43.925030+0000","last_active":"2024-03-06T07:22:01.643337+0000","last_peered":"2024-03-06T07:22:01.643337+0000","last_clean":"2024-03-06T07:22:01.643337+0000","last_became_active":"2024-03-06T07:21:43.924480+0000","last_became_peered":"2024-03-06T07:21:43.924480+0000","last_unstale":"2024-03-06T07:22:01.643337+0000","last_undegraded":"2024-03-06T07:22:01.643337+0000","last_fullsized":"2024-03-06T07:22:01.643337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:36.336480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024729+0000","last_change":"2024-03-06T07:22:00.013001+0000","last_active":"2024-03-06T07:22:01.024729+0000","last_peered":"2024-03-06T07:22:01.024729+0000","last_clean":"2024-03-06T07:22:01.024729+0000","last_became_active":"2024-03-06T07:22:00.012722+0000","last_became_peered":"2024-03-06T07:22:00.012722+0000","last_unstale":"2024-03-06T07:22:01.024729+0000","last_undegraded":"2024-03-06T07:22:01.024729+0000","last_fullsized":"2024-03-06T07:22:01.024729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:11.422190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150189+0000","last_change":"2024-03-06T07:21:43.915656+0000","last_active":"2024-03-06T07:22:01.150189+0000","last_peered":"2024-03-06T07:22:01.150189+0000","last_clean":"2024-03-06T07:22:01.150189+0000","last_became_active":"2024-03-06T07:21:43.915399+0000","last_became_peered":"2024-03-06T07:21:43.915399+0000","last_unstale":"2024-03-06T07:22:01.150189+0000","last_undegraded":"2024-03-06T07:22:01.150189+0000","last_fullsized":"2024-03-06T07:22:01.150189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149997+0000","last_change":"2024-03-06T07:22:00.020432+0000","last_active":"2024-03-06T07:22:01.149997+0000","last_peered":"2024-03-06T07:22:01.149997+0000","last_clean":"2024-03-06T07:22:01.149997+0000","last_became_active":"2024-03-06T07:22:00.020162+0000","last_became_peered":"2024-03-06T07:22:00.020162+0000","last_unstale":"2024-03-06T07:22:01.149997+0000","last_undegraded":"2024-03-06T07:22:01.149997+0000","last_fullsized":"2024-03-06T07:22:01.149997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:14.068723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000448+0000","last_change":"2024-03-06T07:21:43.909296+0000","last_active":"2024-03-06T07:22:01.000448+0000","last_peered":"2024-03-06T07:22:01.000448+0000","last_clean":"2024-03-06T07:22:01.000448+0000","last_became_active":"2024-03-06T07:21:43.909011+0000","last_became_peered":"2024-03-06T07:21:43.909011+0000","last_unstale":"2024-03-06T07:22:01.000448+0000","last_undegraded":"2024-03-06T07:22:01.000448+0000","last_fullsized":"2024-03-06T07:22:01.000448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003579+0000","last_change":"2024-03-06T07:22:00.009690+0000","last_active":"2024-03-06T07:22:01.003579+0000","last_peered":"2024-03-06T07:22:01.003579+0000","last_clean":"2024-03-06T07:22:01.003579+0000","last_became_active":"2024-03-06T07:22:00.009425+0000","last_became_peered":"2024-03-06T07:22:00.009425+0000","last_unstale":"2024-03-06T07:22:01.003579+0000","last_undegraded":"2024-03-06T07:22:01.003579+0000","last_fullsized":"2024-03-06T07:22:01.003579+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:23.455544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642755+0000","last_change":"2024-03-06T07:21:43.930357+0000","last_active":"2024-03-06T07:22:01.642755+0000","last_peered":"2024-03-06T07:22:01.642755+0000","last_clean":"2024-03-06T07:22:01.642755+0000","last_became_active":"2024-03-06T07:21:43.929832+0000","last_became_peered":"2024-03-06T07:21:43.929832+0000","last_unstale":"2024-03-06T07:22:01.642755+0000","last_undegraded":"2024-03-06T07:22:01.642755+0000","last_fullsized":"2024-03-06T07:22:01.642755+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642575+0000","last_change":"2024-03-06T07:22:00.018784+0000","last_active":"2024-03-06T07:22:01.642575+0000","last_peered":"2024-03-06T07:22:01.642575+0000","last_clean":"2024-03-06T07:22:01.642575+0000","last_became_active":"2024-03-06T07:22:00.018452+0000","last_became_peered":"2024-03-06T07:22:00.018452+0000","last_unstale":"2024-03-06T07:22:01.642575+0000","last_undegraded":"2024-03-06T07:22:01.642575+0000","last_fullsized":"2024-03-06T07:22:01.642575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:35:49.437077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150596+0000","last_change":"2024-03-06T07:21:43.919806+0000","last_active":"2024-03-06T07:22:01.150596+0000","last_peered":"2024-03-06T07:22:01.150596+0000","last_clean":"2024-03-06T07:22:01.150596+0000","last_became_active":"2024-03-06T07:21:43.919524+0000","last_became_peered":"2024-03-06T07:21:43.919524+0000","last_unstale":"2024-03-06T07:22:01.150596+0000","last_undegraded":"2024-03-06T07:22:01.150596+0000","last_fullsized":"2024-03-06T07:22:01.150596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001416+0000","last_change":"2024-03-06T07:22:00.026720+0000","last_active":"2024-03-06T07:22:01.001416+0000","last_peered":"2024-03-06T07:22:01.001416+0000","last_clean":"2024-03-06T07:22:01.001416+0000","last_became_active":"2024-03-06T07:22:00.026524+0000","last_became_peered":"2024-03-06T07:22:00.026524+0000","last_unstale":"2024-03-06T07:22:01.001416+0000","last_undegraded":"2024-03-06T07:22:01.001416+0000","last_fullsized":"2024-03-06T07:22:01.001416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:02.688029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642951+0000","last_change":"2024-03-06T07:21:43.933746+0000","last_active":"2024-03-06T07:22:01.642951+0000","last_peered":"2024-03-06T07:22:01.642951+0000","last_clean":"2024-03-06T07:22:01.642951+0000","last_became_active":"2024-03-06T07:21:43.932918+0000","last_became_peered":"2024-03-06T07:21:43.932918+0000","last_unstale":"2024-03-06T07:22:01.642951+0000","last_undegraded":"2024-03-06T07:22:01.642951+0000","last_fullsized":"2024-03-06T07:22:01.642951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002885+0000","last_change":"2024-03-06T07:22:00.011775+0000","last_active":"2024-03-06T07:22:01.002885+0000","last_peered":"2024-03-06T07:22:01.002885+0000","last_clean":"2024-03-06T07:22:01.002885+0000","last_became_active":"2024-03-06T07:22:00.011502+0000","last_became_peered":"2024-03-06T07:22:00.011502+0000","last_unstale":"2024-03-06T07:22:01.002885+0000","last_undegraded":"2024-03-06T07:22:01.002885+0000","last_fullsized":"2024-03-06T07:22:01.002885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:01.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003147+0000","last_change":"2024-03-06T07:21:43.916874+0000","last_active":"2024-03-06T07:22:01.003147+0000","last_peered":"2024-03-06T07:22:01.003147+0000","last_clean":"2024-03-06T07:22:01.003147+0000","last_became_active":"2024-03-06T07:21:43.916593+0000","last_became_peered":"2024-03-06T07:21:43.916593+0000","last_unstale":"2024-03-06T07:22:01.003147+0000","last_undegraded":"2024-03-06T07:22:01.003147+0000","last_fullsized":"2024-03-06T07:22:01.003147+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:30.373457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642350+0000","last_change":"2024-03-06T07:22:00.020263+0000","last_active":"2024-03-06T07:22:01.642350+0000","last_peered":"2024-03-06T07:22:01.642350+0000","last_clean":"2024-03-06T07:22:01.642350+0000","last_became_active":"2024-03-06T07:22:00.019933+0000","last_became_peered":"2024-03-06T07:22:00.019933+0000","last_unstale":"2024-03-06T07:22:01.642350+0000","last_undegraded":"2024-03-06T07:22:01.642350+0000","last_fullsized":"2024-03-06T07:22:01.642350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:06.622414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002337+0000","last_change":"2024-03-06T07:21:43.914663+0000","last_active":"2024-03-06T07:22:01.002337+0000","last_peered":"2024-03-06T07:22:01.002337+0000","last_clean":"2024-03-06T07:22:01.002337+0000","last_became_active":"2024-03-06T07:21:43.914383+0000","last_became_peered":"2024-03-06T07:21:43.914383+0000","last_unstale":"2024-03-06T07:22:01.002337+0000","last_undegraded":"2024-03-06T07:22:01.002337+0000","last_fullsized":"2024-03-06T07:22:01.002337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149643+0000","last_change":"2024-03-06T07:22:00.021035+0000","last_active":"2024-03-06T07:22:01.149643+0000","last_peered":"2024-03-06T07:22:01.149643+0000","last_clean":"2024-03-06T07:22:01.149643+0000","last_became_active":"2024-03-06T07:22:00.015228+0000","last_became_peered":"2024-03-06T07:22:00.015228+0000","last_unstale":"2024-03-06T07:22:01.149643+0000","last_undegraded":"2024-03-06T07:22:01.149643+0000","last_fullsized":"2024-03-06T07:22:01.149643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:42.684403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642677+0000","last_change":"2024-03-06T07:21:43.929377+0000","last_active":"2024-03-06T07:22:01.642677+0000","last_peered":"2024-03-06T07:22:01.642677+0000","last_clean":"2024-03-06T07:22:01.642677+0000","last_became_active":"2024-03-06T07:21:43.929093+0000","last_became_peered":"2024-03-06T07:21:43.929093+0000","last_unstale":"2024-03-06T07:22:01.642677+0000","last_undegraded":"2024-03-06T07:22:01.642677+0000","last_fullsized":"2024-03-06T07:22:01.642677+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024612+0000","last_change":"2024-03-06T07:22:00.015714+0000","last_active":"2024-03-06T07:22:01.024612+0000","last_peered":"2024-03-06T07:22:01.024612+0000","last_clean":"2024-03-06T07:22:01.024612+0000","last_became_active":"2024-03-06T07:22:00.015455+0000","last_became_peered":"2024-03-06T07:22:00.015455+0000","last_unstale":"2024-03-06T07:22:01.024612+0000","last_undegraded":"2024-03-06T07:22:01.024612+0000","last_fullsized":"2024-03-06T07:22:01.024612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027278+0000","last_change":"2024-03-06T07:21:43.916941+0000","last_active":"2024-03-06T07:22:01.027278+0000","last_peered":"2024-03-06T07:22:01.027278+0000","last_clean":"2024-03-06T07:22:01.027278+0000","last_became_active":"2024-03-06T07:21:43.916667+0000","last_became_peered":"2024-03-06T07:21:43.916667+0000","last_unstale":"2024-03-06T07:22:01.027278+0000","last_undegraded":"2024-03-06T07:22:01.027278+0000","last_fullsized":"2024-03-06T07:22:01.027278+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:15.683227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642347+0000","last_change":"2024-03-06T07:22:00.019750+0000","last_active":"2024-03-06T07:22:01.642347+0000","last_peered":"2024-03-06T07:22:01.642347+0000","last_clean":"2024-03-06T07:22:01.642347+0000","last_became_active":"2024-03-06T07:22:00.019454+0000","last_became_peered":"2024-03-06T07:22:00.019454+0000","last_unstale":"2024-03-06T07:22:01.642347+0000","last_undegraded":"2024-03-06T07:22:01.642347+0000","last_fullsized":"2024-03-06T07:22:01.642347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:56.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642688+0000","last_change":"2024-03-06T07:21:43.924259+0000","last_active":"2024-03-06T07:22:01.642688+0000","last_peered":"2024-03-06T07:22:01.642688+0000","last_clean":"2024-03-06T07:22:01.642688+0000","last_became_active":"2024-03-06T07:21:43.923885+0000","last_became_peered":"2024-03-06T07:21:43.923885+0000","last_unstale":"2024-03-06T07:22:01.642688+0000","last_undegraded":"2024-03-06T07:22:01.642688+0000","last_fullsized":"2024-03-06T07:22:01.642688+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001866+0000","last_change":"2024-03-06T07:22:00.010001+0000","last_active":"2024-03-06T07:22:01.001866+0000","last_peered":"2024-03-06T07:22:01.001866+0000","last_clean":"2024-03-06T07:22:01.001866+0000","last_became_active":"2024-03-06T07:22:00.009679+0000","last_became_peered":"2024-03-06T07:22:00.009679+0000","last_unstale":"2024-03-06T07:22:01.001866+0000","last_undegraded":"2024-03-06T07:22:01.001866+0000","last_fullsized":"2024-03-06T07:22:01.001866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:35:03.394288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642859+0000","last_change":"2024-03-06T07:21:43.931109+0000","last_active":"2024-03-06T07:22:01.642859+0000","last_peered":"2024-03-06T07:22:01.642859+0000","last_clean":"2024-03-06T07:22:01.642859+0000","last_became_active":"2024-03-06T07:21:43.930848+0000","last_became_peered":"2024-03-06T07:21:43.930848+0000","last_unstale":"2024-03-06T07:22:01.642859+0000","last_undegraded":"2024-03-06T07:22:01.642859+0000","last_fullsized":"2024-03-06T07:22:01.642859+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000984+0000","last_change":"2024-03-06T07:22:00.019071+0000","last_active":"2024-03-06T07:22:01.000984+0000","last_peered":"2024-03-06T07:22:01.000984+0000","last_clean":"2024-03-06T07:22:01.000984+0000","last_became_active":"2024-03-06T07:22:00.018790+0000","last_became_peered":"2024-03-06T07:22:00.018790+0000","last_unstale":"2024-03-06T07:22:01.000984+0000","last_undegraded":"2024-03-06T07:22:01.000984+0000","last_fullsized":"2024-03-06T07:22:01.000984+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:01.323282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643757+0000","last_change":"2024-03-06T07:21:43.913493+0000","last_active":"2024-03-06T07:22:01.643757+0000","last_peered":"2024-03-06T07:22:01.643757+0000","last_clean":"2024-03-06T07:22:01.643757+0000","last_became_active":"2024-03-06T07:21:43.912992+0000","last_became_peered":"2024-03-06T07:21:43.912992+0000","last_unstale":"2024-03-06T07:22:01.643757+0000","last_undegraded":"2024-03-06T07:22:01.643757+0000","last_fullsized":"2024-03-06T07:22:01.643757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:30.239239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.025043+0000","last_change":"2024-03-06T07:22:00.013830+0000","last_active":"2024-03-06T07:22:01.025043+0000","last_peered":"2024-03-06T07:22:01.025043+0000","last_clean":"2024-03-06T07:22:01.025043+0000","last_became_active":"2024-03-06T07:22:00.013588+0000","last_became_peered":"2024-03-06T07:22:00.013588+0000","last_unstale":"2024-03-06T07:22:01.025043+0000","last_undegraded":"2024-03-06T07:22:01.025043+0000","last_fullsized":"2024-03-06T07:22:01.025043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:31.898544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001410+0000","last_change":"2024-03-06T07:21:43.934992+0000","last_active":"2024-03-06T07:22:01.001410+0000","last_peered":"2024-03-06T07:22:01.001410+0000","last_clean":"2024-03-06T07:22:01.001410+0000","last_became_active":"2024-03-06T07:21:43.934664+0000","last_became_peered":"2024-03-06T07:21:43.934664+0000","last_unstale":"2024-03-06T07:22:01.001410+0000","last_undegraded":"2024-03-06T07:22:01.001410+0000","last_fullsized":"2024-03-06T07:22:01.001410+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024893+0000","last_change":"2024-03-06T07:22:00.019639+0000","last_active":"2024-03-06T07:22:01.024893+0000","last_peered":"2024-03-06T07:22:01.024893+0000","last_clean":"2024-03-06T07:22:01.024893+0000","last_became_active":"2024-03-06T07:22:00.018760+0000","last_became_peered":"2024-03-06T07:22:00.018760+0000","last_unstale":"2024-03-06T07:22:01.024893+0000","last_undegraded":"2024-03-06T07:22:01.024893+0000","last_fullsized":"2024-03-06T07:22:01.024893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:00.475072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003272+0000","last_change":"2024-03-06T07:21:43.910958+0000","last_active":"2024-03-06T07:22:01.003272+0000","last_peered":"2024-03-06T07:22:01.003272+0000","last_clean":"2024-03-06T07:22:01.003272+0000","last_became_active":"2024-03-06T07:21:43.910653+0000","last_became_peered":"2024-03-06T07:21:43.910653+0000","last_unstale":"2024-03-06T07:22:01.003272+0000","last_undegraded":"2024-03-06T07:22:01.003272+0000","last_fullsized":"2024-03-06T07:22:01.003272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001730+0000","last_change":"2024-03-06T07:22:00.010036+0000","last_active":"2024-03-06T07:22:01.001730+0000","last_peered":"2024-03-06T07:22:01.001730+0000","last_clean":"2024-03-06T07:22:01.001730+0000","last_became_active":"2024-03-06T07:22:00.009261+0000","last_became_peered":"2024-03-06T07:22:00.009261+0000","last_unstale":"2024-03-06T07:22:01.001730+0000","last_undegraded":"2024-03-06T07:22:01.001730+0000","last_fullsized":"2024-03-06T07:22:01.001730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:48.043526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642451+0000","last_change":"2024-03-06T07:21:43.923484+0000","last_active":"2024-03-06T07:22:01.642451+0000","last_peered":"2024-03-06T07:22:01.642451+0000","last_clean":"2024-03-06T07:22:01.642451+0000","last_became_active":"2024-03-06T07:21:43.921095+0000","last_became_peered":"2024-03-06T07:21:43.921095+0000","last_unstale":"2024-03-06T07:22:01.642451+0000","last_undegraded":"2024-03-06T07:22:01.642451+0000","last_fullsized":"2024-03-06T07:22:01.642451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642421+0000","last_change":"2024-03-06T07:22:00.010970+0000","last_active":"2024-03-06T07:22:01.642421+0000","last_peered":"2024-03-06T07:22:01.642421+0000","last_clean":"2024-03-06T07:22:01.642421+0000","last_became_active":"2024-03-06T07:22:00.009925+0000","last_became_peered":"2024-03-06T07:22:00.009925+0000","last_unstale":"2024-03-06T07:22:01.642421+0000","last_undegraded":"2024-03-06T07:22:01.642421+0000","last_fullsized":"2024-03-06T07:22:01.642421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:00.475072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150054+0000","last_change":"2024-03-06T07:21:43.917834+0000","last_active":"2024-03-06T07:22:01.150054+0000","last_peered":"2024-03-06T07:22:01.150054+0000","last_clean":"2024-03-06T07:22:01.150054+0000","last_became_active":"2024-03-06T07:21:43.917576+0000","last_became_peered":"2024-03-06T07:21:43.917576+0000","last_unstale":"2024-03-06T07:22:01.150054+0000","last_undegraded":"2024-03-06T07:22:01.150054+0000","last_fullsized":"2024-03-06T07:22:01.150054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149828+0000","last_change":"2024-03-06T07:22:00.030272+0000","last_active":"2024-03-06T07:22:01.149828+0000","last_peered":"2024-03-06T07:22:01.149828+0000","last_clean":"2024-03-06T07:22:01.149828+0000","last_became_active":"2024-03-06T07:22:00.029982+0000","last_became_peered":"2024-03-06T07:22:00.029982+0000","last_unstale":"2024-03-06T07:22:01.149828+0000","last_undegraded":"2024-03-06T07:22:01.149828+0000","last_fullsized":"2024-03-06T07:22:01.149828+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:43.126081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643032+0000","last_change":"2024-03-06T07:21:43.921467+0000","last_active":"2024-03-06T07:22:01.643032+0000","last_peered":"2024-03-06T07:22:01.643032+0000","last_clean":"2024-03-06T07:22:01.643032+0000","last_became_active":"2024-03-06T07:21:43.920878+0000","last_became_peered":"2024-03-06T07:21:43.920878+0000","last_unstale":"2024-03-06T07:22:01.643032+0000","last_undegraded":"2024-03-06T07:22:01.643032+0000","last_fullsized":"2024-03-06T07:22:01.643032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024869+0000","last_change":"2024-03-06T07:22:00.021194+0000","last_active":"2024-03-06T07:22:01.024869+0000","last_peered":"2024-03-06T07:22:01.024869+0000","last_clean":"2024-03-06T07:22:01.024869+0000","last_became_active":"2024-03-06T07:22:00.020935+0000","last_became_peered":"2024-03-06T07:22:00.020935+0000","last_unstale":"2024-03-06T07:22:01.024869+0000","last_undegraded":"2024-03-06T07:22:01.024869+0000","last_fullsized":"2024-03-06T07:22:01.024869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:40.809996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024321+0000","last_change":"2024-03-06T07:21:43.911317+0000","last_active":"2024-03-06T07:22:01.024321+0000","last_peered":"2024-03-06T07:22:01.024321+0000","last_clean":"2024-03-06T07:22:01.024321+0000","last_became_active":"2024-03-06T07:21:43.910980+0000","last_became_peered":"2024-03-06T07:21:43.910980+0000","last_unstale":"2024-03-06T07:22:01.024321+0000","last_undegraded":"2024-03-06T07:22:01.024321+0000","last_fullsized":"2024-03-06T07:22:01.024321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002376+0000","last_change":"2024-03-06T07:22:00.010859+0000","last_active":"2024-03-06T07:22:01.002376+0000","last_peered":"2024-03-06T07:22:01.002376+0000","last_clean":"2024-03-06T07:22:01.002376+0000","last_became_active":"2024-03-06T07:22:00.010163+0000","last_became_peered":"2024-03-06T07:22:00.010163+0000","last_unstale":"2024-03-06T07:22:01.002376+0000","last_undegraded":"2024-03-06T07:22:01.002376+0000","last_fullsized":"2024-03-06T07:22:01.002376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:35:03.394288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149656+0000","last_change":"2024-03-06T07:21:59.013995+0000","last_active":"2024-03-06T07:22:01.149656+0000","last_peered":"2024-03-06T07:22:01.149656+0000","last_clean":"2024-03-06T07:22:01.149656+0000","last_became_active":"2024-03-06T07:21:59.013711+0000","last_became_peered":"2024-03-06T07:21:59.013711+0000","last_unstale":"2024-03-06T07:22:01.149656+0000","last_undegraded":"2024-03-06T07:22:01.149656+0000","last_fullsized":"2024-03-06T07:22:01.149656+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:21.623615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149671+0000","last_change":"2024-03-06T07:21:43.914331+0000","last_active":"2024-03-06T07:22:01.149671+0000","last_peered":"2024-03-06T07:22:01.149671+0000","last_clean":"2024-03-06T07:22:01.149671+0000","last_became_active":"2024-03-06T07:21:43.913681+0000","last_became_peered":"2024-03-06T07:21:43.913681+0000","last_unstale":"2024-03-06T07:22:01.149671+0000","last_undegraded":"2024-03-06T07:22:01.149671+0000","last_fullsized":"2024-03-06T07:22:01.149671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":30,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003830+0000","last_change":"2024-03-06T07:21:42.908247+0000","last_active":"2024-03-06T07:22:01.003830+0000","last_peered":"2024-03-06T07:22:01.003830+0000","last_clean":"2024-03-06T07:22:01.003830+0000","last_became_active":"2024-03-06T07:21:42.907863+0000","last_became_peered":"2024-03-06T07:21:42.907863+0000","last_unstale":"2024-03-06T07:22:01.003830+0000","last_undegraded":"2024-03-06T07:22:01.003830+0000","last_fullsized":"2024-03-06T07:22:01.003830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:44.909708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000867+0000","last_change":"2024-03-06T07:22:00.025078+0000","last_active":"2024-03-06T07:22:01.000867+0000","last_peered":"2024-03-06T07:22:01.000867+0000","last_clean":"2024-03-06T07:22:01.000867+0000","last_became_active":"2024-03-06T07:22:00.024793+0000","last_became_peered":"2024-03-06T07:22:00.024793+0000","last_unstale":"2024-03-06T07:22:01.000867+0000","last_undegraded":"2024-03-06T07:22:01.000867+0000","last_fullsized":"2024-03-06T07:22:01.000867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:59.335901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002810+0000","last_change":"2024-03-06T07:21:59.017569+0000","last_active":"2024-03-06T07:22:01.002810+0000","last_peered":"2024-03-06T07:22:01.002810+0000","last_clean":"2024-03-06T07:22:01.002810+0000","last_became_active":"2024-03-06T07:21:59.017285+0000","last_became_peered":"2024-03-06T07:21:59.017285+0000","last_unstale":"2024-03-06T07:22:01.002810+0000","last_undegraded":"2024-03-06T07:22:01.002810+0000","last_fullsized":"2024-03-06T07:22:01.002810+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:46.110694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024500+0000","last_change":"2024-03-06T07:21:43.918214+0000","last_active":"2024-03-06T07:22:01.024500+0000","last_peered":"2024-03-06T07:22:01.024500+0000","last_clean":"2024-03-06T07:22:01.024500+0000","last_became_active":"2024-03-06T07:21:43.917454+0000","last_became_peered":"2024-03-06T07:21:43.917454+0000","last_unstale":"2024-03-06T07:22:01.024500+0000","last_undegraded":"2024-03-06T07:22:01.024500+0000","last_fullsized":"2024-03-06T07:22:01.024500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001412+0000","last_change":"2024-03-06T07:21:42.915335+0000","last_active":"2024-03-06T07:22:01.001412+0000","last_peered":"2024-03-06T07:22:01.001412+0000","last_clean":"2024-03-06T07:22:01.001412+0000","last_became_active":"2024-03-06T07:21:42.915011+0000","last_became_peered":"2024-03-06T07:21:42.915011+0000","last_unstale":"2024-03-06T07:22:01.001412+0000","last_undegraded":"2024-03-06T07:22:01.001412+0000","last_fullsized":"2024-03-06T07:22:01.001412+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001702+0000","last_change":"2024-03-06T07:22:00.017468+0000","last_active":"2024-03-06T07:22:01.001702+0000","last_peered":"2024-03-06T07:22:01.001702+0000","last_clean":"2024-03-06T07:22:01.001702+0000","last_became_active":"2024-03-06T07:22:00.017203+0000","last_became_peered":"2024-03-06T07:22:00.017203+0000","last_unstale":"2024-03-06T07:22:01.001702+0000","last_undegraded":"2024-03-06T07:22:01.001702+0000","last_fullsized":"2024-03-06T07:22:01.001702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:13.109019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149834+0000","last_change":"2024-03-06T07:21:59.011323+0000","last_active":"2024-03-06T07:22:01.149834+0000","last_peered":"2024-03-06T07:22:01.149834+0000","last_clean":"2024-03-06T07:22:01.149834+0000","last_became_active":"2024-03-06T07:21:59.011040+0000","last_became_peered":"2024-03-06T07:21:59.011040+0000","last_unstale":"2024-03-06T07:22:01.149834+0000","last_undegraded":"2024-03-06T07:22:01.149834+0000","last_fullsized":"2024-03-06T07:22:01.149834+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:44.332849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002560+0000","last_change":"2024-03-06T07:21:43.914522+0000","last_active":"2024-03-06T07:22:01.002560+0000","last_peered":"2024-03-06T07:22:01.002560+0000","last_clean":"2024-03-06T07:22:01.002560+0000","last_became_active":"2024-03-06T07:21:43.913602+0000","last_became_peered":"2024-03-06T07:21:43.913602+0000","last_unstale":"2024-03-06T07:22:01.002560+0000","last_undegraded":"2024-03-06T07:22:01.002560+0000","last_fullsized":"2024-03-06T07:22:01.002560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:02.123078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001505+0000","last_change":"2024-03-06T07:21:42.914470+0000","last_active":"2024-03-06T07:22:01.001505+0000","last_peered":"2024-03-06T07:22:01.001505+0000","last_clean":"2024-03-06T07:22:01.001505+0000","last_became_active":"2024-03-06T07:21:42.913930+0000","last_became_peered":"2024-03-06T07:21:42.913930+0000","last_unstale":"2024-03-06T07:22:01.001505+0000","last_undegraded":"2024-03-06T07:22:01.001505+0000","last_fullsized":"2024-03-06T07:22:01.001505+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:24.367917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024379+0000","last_change":"2024-03-06T07:22:00.012632+0000","last_active":"2024-03-06T07:22:01.024379+0000","last_peered":"2024-03-06T07:22:01.024379+0000","last_clean":"2024-03-06T07:22:01.024379+0000","last_became_active":"2024-03-06T07:22:00.012216+0000","last_became_peered":"2024-03-06T07:22:00.012216+0000","last_unstale":"2024-03-06T07:22:01.024379+0000","last_undegraded":"2024-03-06T07:22:01.024379+0000","last_fullsized":"2024-03-06T07:22:01.024379+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:36.032336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024375+0000","last_change":"2024-03-06T07:21:59.011827+0000","last_active":"2024-03-06T07:22:01.024375+0000","last_peered":"2024-03-06T07:22:01.024375+0000","last_clean":"2024-03-06T07:22:01.024375+0000","last_became_active":"2024-03-06T07:21:59.011087+0000","last_became_peered":"2024-03-06T07:21:59.011087+0000","last_unstale":"2024-03-06T07:22:01.024375+0000","last_undegraded":"2024-03-06T07:22:01.024375+0000","last_fullsized":"2024-03-06T07:22:01.024375+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:27.422606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643107+0000","last_change":"2024-03-06T07:21:43.912159+0000","last_active":"2024-03-06T07:22:01.643107+0000","last_peered":"2024-03-06T07:22:01.643107+0000","last_clean":"2024-03-06T07:22:01.643107+0000","last_became_active":"2024-03-06T07:21:43.911851+0000","last_became_peered":"2024-03-06T07:21:43.911851+0000","last_unstale":"2024-03-06T07:22:01.643107+0000","last_undegraded":"2024-03-06T07:22:01.643107+0000","last_fullsized":"2024-03-06T07:22:01.643107+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024735+0000","last_change":"2024-03-06T07:21:42.912811+0000","last_active":"2024-03-06T07:22:01.024735+0000","last_peered":"2024-03-06T07:22:01.024735+0000","last_clean":"2024-03-06T07:22:01.024735+0000","last_became_active":"2024-03-06T07:21:42.912544+0000","last_became_peered":"2024-03-06T07:21:42.912544+0000","last_unstale":"2024-03-06T07:22:01.024735+0000","last_undegraded":"2024-03-06T07:22:01.024735+0000","last_fullsized":"2024-03-06T07:22:01.024735+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:50.577390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003375+0000","last_change":"2024-03-06T07:22:00.010765+0000","last_active":"2024-03-06T07:22:01.003375+0000","last_peered":"2024-03-06T07:22:01.003375+0000","last_clean":"2024-03-06T07:22:01.003375+0000","last_became_active":"2024-03-06T07:22:00.010488+0000","last_became_peered":"2024-03-06T07:22:00.010488+0000","last_unstale":"2024-03-06T07:22:01.003375+0000","last_undegraded":"2024-03-06T07:22:01.003375+0000","last_fullsized":"2024-03-06T07:22:01.003375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:45.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150655+0000","last_change":"2024-03-06T07:21:59.015543+0000","last_active":"2024-03-06T07:22:01.150655+0000","last_peered":"2024-03-06T07:22:01.150655+0000","last_clean":"2024-03-06T07:22:01.150655+0000","last_became_active":"2024-03-06T07:21:59.015052+0000","last_became_peered":"2024-03-06T07:21:59.015052+0000","last_unstale":"2024-03-06T07:22:01.150655+0000","last_undegraded":"2024-03-06T07:22:01.150655+0000","last_fullsized":"2024-03-06T07:22:01.150655+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:56.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003273+0000","last_change":"2024-03-06T07:21:43.922679+0000","last_active":"2024-03-06T07:22:01.003273+0000","last_peered":"2024-03-06T07:22:01.003273+0000","last_clean":"2024-03-06T07:22:01.003273+0000","last_became_active":"2024-03-06T07:21:43.922322+0000","last_became_peered":"2024-03-06T07:21:43.922322+0000","last_unstale":"2024-03-06T07:22:01.003273+0000","last_undegraded":"2024-03-06T07:22:01.003273+0000","last_fullsized":"2024-03-06T07:22:01.003273+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642677+0000","last_change":"2024-03-06T07:21:42.912329+0000","last_active":"2024-03-06T07:22:01.642677+0000","last_peered":"2024-03-06T07:22:01.642677+0000","last_clean":"2024-03-06T07:22:01.642677+0000","last_became_active":"2024-03-06T07:21:42.911690+0000","last_became_peered":"2024-03-06T07:21:42.911690+0000","last_unstale":"2024-03-06T07:22:01.642677+0000","last_undegraded":"2024-03-06T07:22:01.642677+0000","last_fullsized":"2024-03-06T07:22:01.642677+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002846+0000","last_change":"2024-03-06T07:22:00.009765+0000","last_active":"2024-03-06T07:22:01.002846+0000","last_peered":"2024-03-06T07:22:01.002846+0000","last_clean":"2024-03-06T07:22:01.002846+0000","last_became_active":"2024-03-06T07:22:00.009440+0000","last_became_peered":"2024-03-06T07:22:00.009440+0000","last_unstale":"2024-03-06T07:22:01.002846+0000","last_undegraded":"2024-03-06T07:22:01.002846+0000","last_fullsized":"2024-03-06T07:22:01.002846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:22.628016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001125+0000","last_change":"2024-03-06T07:21:59.021111+0000","last_active":"2024-03-06T07:22:01.001125+0000","last_peered":"2024-03-06T07:22:01.001125+0000","last_clean":"2024-03-06T07:22:01.001125+0000","last_became_active":"2024-03-06T07:21:59.020849+0000","last_became_peered":"2024-03-06T07:21:59.020849+0000","last_unstale":"2024-03-06T07:22:01.001125+0000","last_undegraded":"2024-03-06T07:22:01.001125+0000","last_fullsized":"2024-03-06T07:22:01.001125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:51.433414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150387+0000","last_change":"2024-03-06T07:21:43.917686+0000","last_active":"2024-03-06T07:22:01.150387+0000","last_peered":"2024-03-06T07:22:01.150387+0000","last_clean":"2024-03-06T07:22:01.150387+0000","last_became_active":"2024-03-06T07:21:43.917369+0000","last_became_peered":"2024-03-06T07:21:43.917369+0000","last_unstale":"2024-03-06T07:22:01.150387+0000","last_undegraded":"2024-03-06T07:22:01.150387+0000","last_fullsized":"2024-03-06T07:22:01.150387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.013761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001800+0000","last_change":"2024-03-06T07:21:42.910602+0000","last_active":"2024-03-06T07:22:01.001800+0000","last_peered":"2024-03-06T07:22:01.001800+0000","last_clean":"2024-03-06T07:22:01.001800+0000","last_became_active":"2024-03-06T07:21:42.910279+0000","last_became_peered":"2024-03-06T07:21:42.910279+0000","last_unstale":"2024-03-06T07:22:01.001800+0000","last_undegraded":"2024-03-06T07:22:01.001800+0000","last_fullsized":"2024-03-06T07:22:01.001800+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:02.037327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001125+0000","last_change":"2024-03-06T07:22:00.012972+0000","last_active":"2024-03-06T07:22:01.001125+0000","last_peered":"2024-03-06T07:22:01.001125+0000","last_clean":"2024-03-06T07:22:01.001125+0000","last_became_active":"2024-03-06T07:22:00.012692+0000","last_became_peered":"2024-03-06T07:22:00.012692+0000","last_unstale":"2024-03-06T07:22:01.001125+0000","last_undegraded":"2024-03-06T07:22:01.001125+0000","last_fullsized":"2024-03-06T07:22:01.001125+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:29.160570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150655+0000","last_change":"2024-03-06T07:21:59.015229+0000","last_active":"2024-03-06T07:22:01.150655+0000","last_peered":"2024-03-06T07:22:01.150655+0000","last_clean":"2024-03-06T07:22:01.150655+0000","last_became_active":"2024-03-06T07:21:59.014945+0000","last_became_peered":"2024-03-06T07:21:59.014945+0000","last_unstale":"2024-03-06T07:22:01.150655+0000","last_undegraded":"2024-03-06T07:22:01.150655+0000","last_fullsized":"2024-03-06T07:22:01.150655+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:29.685760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643856+0000","last_change":"2024-03-06T07:21:43.913254+0000","last_active":"2024-03-06T07:22:01.643856+0000","last_peered":"2024-03-06T07:22:01.643856+0000","last_clean":"2024-03-06T07:22:01.643856+0000","last_became_active":"2024-03-06T07:21:43.912620+0000","last_became_peered":"2024-03-06T07:21:43.912620+0000","last_unstale":"2024-03-06T07:22:01.643856+0000","last_undegraded":"2024-03-06T07:22:01.643856+0000","last_fullsized":"2024-03-06T07:22:01.643856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:13.059235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002160+0000","last_change":"2024-03-06T07:21:42.909672+0000","last_active":"2024-03-06T07:22:01.002160+0000","last_peered":"2024-03-06T07:22:01.002160+0000","last_clean":"2024-03-06T07:22:01.002160+0000","last_became_active":"2024-03-06T07:21:42.909402+0000","last_became_peered":"2024-03-06T07:21:42.909402+0000","last_unstale":"2024-03-06T07:22:01.002160+0000","last_undegraded":"2024-03-06T07:22:01.002160+0000","last_fullsized":"2024-03-06T07:22:01.002160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:06.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643716+0000","last_change":"2024-03-06T07:22:00.022225+0000","last_active":"2024-03-06T07:22:01.643716+0000","last_peered":"2024-03-06T07:22:01.643716+0000","last_clean":"2024-03-06T07:22:01.643716+0000","last_became_active":"2024-03-06T07:22:00.021970+0000","last_became_peered":"2024-03-06T07:22:00.021970+0000","last_unstale":"2024-03-06T07:22:01.643716+0000","last_undegraded":"2024-03-06T07:22:01.643716+0000","last_fullsized":"2024-03-06T07:22:01.643716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:08.955032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002718+0000","last_change":"2024-03-06T07:21:59.020696+0000","last_active":"2024-03-06T07:22:01.002718+0000","last_peered":"2024-03-06T07:22:01.002718+0000","last_clean":"2024-03-06T07:22:01.002718+0000","last_became_active":"2024-03-06T07:21:59.020474+0000","last_became_peered":"2024-03-06T07:21:59.020474+0000","last_unstale":"2024-03-06T07:22:01.002718+0000","last_undegraded":"2024-03-06T07:22:01.002718+0000","last_fullsized":"2024-03-06T07:22:01.002718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:15.577151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150609+0000","last_change":"2024-03-06T07:21:43.934375+0000","last_active":"2024-03-06T07:22:01.150609+0000","last_peered":"2024-03-06T07:22:01.150609+0000","last_clean":"2024-03-06T07:22:01.150609+0000","last_became_active":"2024-03-06T07:21:43.934076+0000","last_became_peered":"2024-03-06T07:21:43.934076+0000","last_unstale":"2024-03-06T07:22:01.150609+0000","last_undegraded":"2024-03-06T07:22:01.150609+0000","last_fullsized":"2024-03-06T07:22:01.150609+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642730+0000","last_change":"2024-03-06T07:21:42.910541+0000","last_active":"2024-03-06T07:22:01.642730+0000","last_peered":"2024-03-06T07:22:01.642730+0000","last_clean":"2024-03-06T07:22:01.642730+0000","last_became_active":"2024-03-06T07:21:42.910271+0000","last_became_peered":"2024-03-06T07:21:42.910271+0000","last_unstale":"2024-03-06T07:22:01.642730+0000","last_undegraded":"2024-03-06T07:22:01.642730+0000","last_fullsized":"2024-03-06T07:22:01.642730+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002754+0000","last_change":"2024-03-06T07:22:00.011049+0000","last_active":"2024-03-06T07:22:01.002754+0000","last_peered":"2024-03-06T07:22:01.002754+0000","last_clean":"2024-03-06T07:22:01.002754+0000","last_became_active":"2024-03-06T07:22:00.010569+0000","last_became_peered":"2024-03-06T07:22:00.010569+0000","last_unstale":"2024-03-06T07:22:01.002754+0000","last_undegraded":"2024-03-06T07:22:01.002754+0000","last_fullsized":"2024-03-06T07:22:01.002754+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:31.441532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642290+0000","last_change":"2024-03-06T07:21:59.010326+0000","last_active":"2024-03-06T07:22:01.642290+0000","last_peered":"2024-03-06T07:22:01.642290+0000","last_clean":"2024-03-06T07:22:01.642290+0000","last_became_active":"2024-03-06T07:21:59.009373+0000","last_became_peered":"2024-03-06T07:21:59.009373+0000","last_unstale":"2024-03-06T07:22:01.642290+0000","last_undegraded":"2024-03-06T07:22:01.642290+0000","last_fullsized":"2024-03-06T07:22:01.642290+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:59.470671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002962+0000","last_change":"2024-03-06T07:21:43.905571+0000","last_active":"2024-03-06T07:22:01.002962+0000","last_peered":"2024-03-06T07:22:01.002962+0000","last_clean":"2024-03-06T07:22:01.002962+0000","last_became_active":"2024-03-06T07:21:43.904452+0000","last_became_peered":"2024-03-06T07:21:43.904452+0000","last_unstale":"2024-03-06T07:22:01.002962+0000","last_undegraded":"2024-03-06T07:22:01.002962+0000","last_fullsized":"2024-03-06T07:22:01.002962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150719+0000","last_change":"2024-03-06T07:21:42.913817+0000","last_active":"2024-03-06T07:22:01.150719+0000","last_peered":"2024-03-06T07:22:01.150719+0000","last_clean":"2024-03-06T07:22:01.150719+0000","last_became_active":"2024-03-06T07:21:42.913464+0000","last_became_peered":"2024-03-06T07:21:42.913464+0000","last_unstale":"2024-03-06T07:22:01.150719+0000","last_undegraded":"2024-03-06T07:22:01.150719+0000","last_fullsized":"2024-03-06T07:22:01.150719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643376+0000","last_change":"2024-03-06T07:22:00.011586+0000","last_active":"2024-03-06T07:22:01.643376+0000","last_peered":"2024-03-06T07:22:01.643376+0000","last_clean":"2024-03-06T07:22:01.643376+0000","last_became_active":"2024-03-06T07:22:00.011320+0000","last_became_peered":"2024-03-06T07:22:00.011320+0000","last_unstale":"2024-03-06T07:22:01.643376+0000","last_undegraded":"2024-03-06T07:22:01.643376+0000","last_fullsized":"2024-03-06T07:22:01.643376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:56.594846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001643+0000","last_change":"2024-03-06T07:21:59.011672+0000","last_active":"2024-03-06T07:22:01.001643+0000","last_peered":"2024-03-06T07:22:01.001643+0000","last_clean":"2024-03-06T07:22:01.001643+0000","last_became_active":"2024-03-06T07:21:59.011374+0000","last_became_peered":"2024-03-06T07:21:59.011374+0000","last_unstale":"2024-03-06T07:22:01.001643+0000","last_undegraded":"2024-03-06T07:22:01.001643+0000","last_fullsized":"2024-03-06T07:22:01.001643+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:06.679356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024522+0000","last_change":"2024-03-06T07:21:43.912031+0000","last_active":"2024-03-06T07:22:01.024522+0000","last_peered":"2024-03-06T07:22:01.024522+0000","last_clean":"2024-03-06T07:22:01.024522+0000","last_became_active":"2024-03-06T07:21:43.911759+0000","last_became_peered":"2024-03-06T07:21:43.911759+0000","last_unstale":"2024-03-06T07:22:01.024522+0000","last_undegraded":"2024-03-06T07:22:01.024522+0000","last_fullsized":"2024-03-06T07:22:01.024522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003050+0000","last_change":"2024-03-06T07:21:42.910374+0000","last_active":"2024-03-06T07:22:01.003050+0000","last_peered":"2024-03-06T07:22:01.003050+0000","last_clean":"2024-03-06T07:22:01.003050+0000","last_became_active":"2024-03-06T07:21:42.910082+0000","last_became_peered":"2024-03-06T07:21:42.910082+0000","last_unstale":"2024-03-06T07:22:01.003050+0000","last_undegraded":"2024-03-06T07:22:01.003050+0000","last_fullsized":"2024-03-06T07:22:01.003050+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:55.547305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027061+0000","last_change":"2024-03-06T07:22:00.016621+0000","last_active":"2024-03-06T07:22:01.027061+0000","last_peered":"2024-03-06T07:22:01.027061+0000","last_clean":"2024-03-06T07:22:01.027061+0000","last_became_active":"2024-03-06T07:22:00.016279+0000","last_became_peered":"2024-03-06T07:22:00.016279+0000","last_unstale":"2024-03-06T07:22:01.027061+0000","last_undegraded":"2024-03-06T07:22:01.027061+0000","last_fullsized":"2024-03-06T07:22:01.027061+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:41.474284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643060+0000","last_change":"2024-03-06T07:21:59.009295+0000","last_active":"2024-03-06T07:22:01.643060+0000","last_peered":"2024-03-06T07:22:01.643060+0000","last_clean":"2024-03-06T07:22:01.643060+0000","last_became_active":"2024-03-06T07:21:59.009028+0000","last_became_peered":"2024-03-06T07:21:59.009028+0000","last_unstale":"2024-03-06T07:22:01.643060+0000","last_undegraded":"2024-03-06T07:22:01.643060+0000","last_fullsized":"2024-03-06T07:22:01.643060+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:10.417789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000751+0000","last_change":"2024-03-06T07:21:53.936851+0000","last_active":"2024-03-06T07:22:01.000751+0000","last_peered":"2024-03-06T07:22:01.000751+0000","last_clean":"2024-03-06T07:22:01.000751+0000","last_became_active":"2024-03-06T07:21:53.936533+0000","last_became_peered":"2024-03-06T07:21:53.936533+0000","last_unstale":"2024-03-06T07:22:01.000751+0000","last_undegraded":"2024-03-06T07:22:01.000751+0000","last_fullsized":"2024-03-06T07:22:01.000751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":29,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150549+0000","last_change":"2024-03-06T07:21:42.912311+0000","last_active":"2024-03-06T07:22:01.150549+0000","last_peered":"2024-03-06T07:22:01.150549+0000","last_clean":"2024-03-06T07:22:01.150549+0000","last_became_active":"2024-03-06T07:21:42.912046+0000","last_became_peered":"2024-03-06T07:21:42.912046+0000","last_unstale":"2024-03-06T07:22:01.150549+0000","last_undegraded":"2024-03-06T07:22:01.150549+0000","last_fullsized":"2024-03-06T07:22:01.150549+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000444+0000","last_change":"2024-03-06T07:22:00.015427+0000","last_active":"2024-03-06T07:22:01.000444+0000","last_peered":"2024-03-06T07:22:01.000444+0000","last_clean":"2024-03-06T07:22:01.000444+0000","last_became_active":"2024-03-06T07:22:00.014435+0000","last_became_peered":"2024-03-06T07:22:00.014435+0000","last_unstale":"2024-03-06T07:22:01.000444+0000","last_undegraded":"2024-03-06T07:22:01.000444+0000","last_fullsized":"2024-03-06T07:22:01.000444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:46.340573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642538+0000","last_change":"2024-03-06T07:21:59.007501+0000","last_active":"2024-03-06T07:22:01.642538+0000","last_peered":"2024-03-06T07:22:01.642538+0000","last_clean":"2024-03-06T07:22:01.642538+0000","last_became_active":"2024-03-06T07:21:59.007223+0000","last_became_peered":"2024-03-06T07:21:59.007223+0000","last_unstale":"2024-03-06T07:22:01.642538+0000","last_undegraded":"2024-03-06T07:22:01.642538+0000","last_fullsized":"2024-03-06T07:22:01.642538+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.490540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002595+0000","last_change":"2024-03-06T07:21:43.911762+0000","last_active":"2024-03-06T07:22:01.002595+0000","last_peered":"2024-03-06T07:22:01.002595+0000","last_clean":"2024-03-06T07:22:01.002595+0000","last_became_active":"2024-03-06T07:21:43.911470+0000","last_became_peered":"2024-03-06T07:21:43.911470+0000","last_unstale":"2024-03-06T07:22:01.002595+0000","last_undegraded":"2024-03-06T07:22:01.002595+0000","last_fullsized":"2024-03-06T07:22:01.002595+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":32,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643755+0000","last_change":"2024-03-06T07:21:42.912419+0000","last_active":"2024-03-06T07:22:01.643755+0000","last_peered":"2024-03-06T07:22:01.643755+0000","last_clean":"2024-03-06T07:22:01.643755+0000","last_became_active":"2024-03-06T07:21:42.912141+0000","last_became_peered":"2024-03-06T07:21:42.912141+0000","last_unstale":"2024-03-06T07:22:01.643755+0000","last_undegraded":"2024-03-06T07:22:01.643755+0000","last_fullsized":"2024-03-06T07:22:01.643755+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:00.590629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642789+0000","last_change":"2024-03-06T07:22:00.010409+0000","last_active":"2024-03-06T07:22:01.642789+0000","last_peered":"2024-03-06T07:22:01.642789+0000","last_clean":"2024-03-06T07:22:01.642789+0000","last_became_active":"2024-03-06T07:22:00.009751+0000","last_became_peered":"2024-03-06T07:22:00.009751+0000","last_unstale":"2024-03-06T07:22:01.642789+0000","last_undegraded":"2024-03-06T07:22:01.642789+0000","last_fullsized":"2024-03-06T07:22:01.642789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:55.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643176+0000","last_change":"2024-03-06T07:21:59.013433+0000","last_active":"2024-03-06T07:22:01.643176+0000","last_peered":"2024-03-06T07:22:01.643176+0000","last_clean":"2024-03-06T07:22:01.643176+0000","last_became_active":"2024-03-06T07:21:59.013170+0000","last_became_peered":"2024-03-06T07:21:59.013170+0000","last_unstale":"2024-03-06T07:22:01.643176+0000","last_undegraded":"2024-03-06T07:22:01.643176+0000","last_fullsized":"2024-03-06T07:22:01.643176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:41.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024424+0000","last_change":"2024-03-06T07:21:43.918063+0000","last_active":"2024-03-06T07:22:01.024424+0000","last_peered":"2024-03-06T07:22:01.024424+0000","last_clean":"2024-03-06T07:22:01.024424+0000","last_became_active":"2024-03-06T07:21:43.917325+0000","last_became_peered":"2024-03-06T07:21:43.917325+0000","last_unstale":"2024-03-06T07:22:01.024424+0000","last_undegraded":"2024-03-06T07:22:01.024424+0000","last_fullsized":"2024-03-06T07:22:01.024424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642833+0000","last_change":"2024-03-06T07:21:42.912594+0000","last_active":"2024-03-06T07:22:01.642833+0000","last_peered":"2024-03-06T07:22:01.642833+0000","last_clean":"2024-03-06T07:22:01.642833+0000","last_became_active":"2024-03-06T07:21:42.912053+0000","last_became_peered":"2024-03-06T07:21:42.912053+0000","last_unstale":"2024-03-06T07:22:01.642833+0000","last_undegraded":"2024-03-06T07:22:01.642833+0000","last_fullsized":"2024-03-06T07:22:01.642833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:42.711847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150532+0000","last_change":"2024-03-06T07:22:00.029826+0000","last_active":"2024-03-06T07:22:01.150532+0000","last_peered":"2024-03-06T07:22:01.150532+0000","last_clean":"2024-03-06T07:22:01.150532+0000","last_became_active":"2024-03-06T07:22:00.029347+0000","last_became_peered":"2024-03-06T07:22:00.029347+0000","last_unstale":"2024-03-06T07:22:01.150532+0000","last_undegraded":"2024-03-06T07:22:01.150532+0000","last_fullsized":"2024-03-06T07:22:01.150532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:11.422190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643619+0000","last_change":"2024-03-06T07:21:59.010872+0000","last_active":"2024-03-06T07:22:01.643619+0000","last_peered":"2024-03-06T07:22:01.643619+0000","last_clean":"2024-03-06T07:22:01.643619+0000","last_became_active":"2024-03-06T07:21:59.010589+0000","last_became_peered":"2024-03-06T07:21:59.010589+0000","last_unstale":"2024-03-06T07:22:01.643619+0000","last_undegraded":"2024-03-06T07:22:01.643619+0000","last_fullsized":"2024-03-06T07:22:01.643619+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:19.513411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643041+0000","last_change":"2024-03-06T07:21:43.931801+0000","last_active":"2024-03-06T07:22:01.643041+0000","last_peered":"2024-03-06T07:22:01.643041+0000","last_clean":"2024-03-06T07:22:01.643041+0000","last_became_active":"2024-03-06T07:21:43.931515+0000","last_became_peered":"2024-03-06T07:21:43.931515+0000","last_unstale":"2024-03-06T07:22:01.643041+0000","last_undegraded":"2024-03-06T07:22:01.643041+0000","last_fullsized":"2024-03-06T07:22:01.643041+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002748+0000","last_change":"2024-03-06T07:21:42.910643+0000","last_active":"2024-03-06T07:22:01.002748+0000","last_peered":"2024-03-06T07:22:01.002748+0000","last_clean":"2024-03-06T07:22:01.002748+0000","last_became_active":"2024-03-06T07:21:42.910377+0000","last_became_peered":"2024-03-06T07:21:42.910377+0000","last_unstale":"2024-03-06T07:22:01.002748+0000","last_undegraded":"2024-03-06T07:22:01.002748+0000","last_fullsized":"2024-03-06T07:22:01.002748+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003023+0000","last_change":"2024-03-06T07:22:00.016675+0000","last_active":"2024-03-06T07:22:01.003023+0000","last_peered":"2024-03-06T07:22:01.003023+0000","last_clean":"2024-03-06T07:22:01.003023+0000","last_became_active":"2024-03-06T07:22:00.016447+0000","last_became_peered":"2024-03-06T07:22:00.016447+0000","last_unstale":"2024-03-06T07:22:01.003023+0000","last_undegraded":"2024-03-06T07:22:01.003023+0000","last_fullsized":"2024-03-06T07:22:01.003023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:22.628016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001440+0000","last_change":"2024-03-06T07:21:59.011671+0000","last_active":"2024-03-06T07:22:01.001440+0000","last_peered":"2024-03-06T07:22:01.001440+0000","last_clean":"2024-03-06T07:22:01.001440+0000","last_became_active":"2024-03-06T07:21:59.011406+0000","last_became_peered":"2024-03-06T07:21:59.011406+0000","last_unstale":"2024-03-06T07:22:01.001440+0000","last_undegraded":"2024-03-06T07:22:01.001440+0000","last_fullsized":"2024-03-06T07:22:01.001440+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:12.104618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643805+0000","last_change":"2024-03-06T07:21:43.915960+0000","last_active":"2024-03-06T07:22:01.643805+0000","last_peered":"2024-03-06T07:22:01.643805+0000","last_clean":"2024-03-06T07:22:01.643805+0000","last_became_active":"2024-03-06T07:21:43.915665+0000","last_became_peered":"2024-03-06T07:21:43.915665+0000","last_unstale":"2024-03-06T07:22:01.643805+0000","last_undegraded":"2024-03-06T07:22:01.643805+0000","last_fullsized":"2024-03-06T07:22:01.643805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:43.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150413+0000","last_change":"2024-03-06T07:21:42.911389+0000","last_active":"2024-03-06T07:22:01.150413+0000","last_peered":"2024-03-06T07:22:01.150413+0000","last_clean":"2024-03-06T07:22:01.150413+0000","last_became_active":"2024-03-06T07:21:42.911070+0000","last_became_peered":"2024-03-06T07:21:42.911070+0000","last_unstale":"2024-03-06T07:22:01.150413+0000","last_undegraded":"2024-03-06T07:22:01.150413+0000","last_fullsized":"2024-03-06T07:22:01.150413+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002062+0000","last_change":"2024-03-06T07:22:00.012556+0000","last_active":"2024-03-06T07:22:01.002062+0000","last_peered":"2024-03-06T07:22:01.002062+0000","last_clean":"2024-03-06T07:22:01.002062+0000","last_became_active":"2024-03-06T07:22:00.012296+0000","last_became_peered":"2024-03-06T07:22:00.012296+0000","last_unstale":"2024-03-06T07:22:01.002062+0000","last_undegraded":"2024-03-06T07:22:01.002062+0000","last_fullsized":"2024-03-06T07:22:01.002062+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:13.838771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642974+0000","last_change":"2024-03-06T07:21:59.013968+0000","last_active":"2024-03-06T07:22:01.642974+0000","last_peered":"2024-03-06T07:22:01.642974+0000","last_clean":"2024-03-06T07:22:01.642974+0000","last_became_active":"2024-03-06T07:21:59.013106+0000","last_became_peered":"2024-03-06T07:21:59.013106+0000","last_unstale":"2024-03-06T07:22:01.642974+0000","last_undegraded":"2024-03-06T07:22:01.642974+0000","last_fullsized":"2024-03-06T07:22:01.642974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:42.121680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642804+0000","last_change":"2024-03-06T07:21:43.930573+0000","last_active":"2024-03-06T07:22:01.642804+0000","last_peered":"2024-03-06T07:22:01.642804+0000","last_clean":"2024-03-06T07:22:01.642804+0000","last_became_active":"2024-03-06T07:21:43.930105+0000","last_became_peered":"2024-03-06T07:21:43.930105+0000","last_unstale":"2024-03-06T07:22:01.642804+0000","last_undegraded":"2024-03-06T07:22:01.642804+0000","last_fullsized":"2024-03-06T07:22:01.642804+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":34,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642359+0000","last_change":"2024-03-06T07:21:42.915873+0000","last_active":"2024-03-06T07:22:01.642359+0000","last_peered":"2024-03-06T07:22:01.642359+0000","last_clean":"2024-03-06T07:22:01.642359+0000","last_became_active":"2024-03-06T07:21:42.915575+0000","last_became_peered":"2024-03-06T07:21:42.915575+0000","last_unstale":"2024-03-06T07:22:01.642359+0000","last_undegraded":"2024-03-06T07:22:01.642359+0000","last_fullsized":"2024-03-06T07:22:01.642359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:30.008729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000535+0000","last_change":"2024-03-06T07:22:00.013681+0000","last_active":"2024-03-06T07:22:01.000535+0000","last_peered":"2024-03-06T07:22:01.000535+0000","last_clean":"2024-03-06T07:22:01.000535+0000","last_became_active":"2024-03-06T07:22:00.013437+0000","last_became_peered":"2024-03-06T07:22:00.013437+0000","last_unstale":"2024-03-06T07:22:01.000535+0000","last_undegraded":"2024-03-06T07:22:01.000535+0000","last_fullsized":"2024-03-06T07:22:01.000535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:28.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642434+0000","last_change":"2024-03-06T07:21:59.010664+0000","last_active":"2024-03-06T07:22:01.642434+0000","last_peered":"2024-03-06T07:22:01.642434+0000","last_clean":"2024-03-06T07:22:01.642434+0000","last_became_active":"2024-03-06T07:21:59.009227+0000","last_became_peered":"2024-03-06T07:21:59.009227+0000","last_unstale":"2024-03-06T07:22:01.642434+0000","last_undegraded":"2024-03-06T07:22:01.642434+0000","last_fullsized":"2024-03-06T07:22:01.642434+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:30.894143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642292+0000","last_change":"2024-03-06T07:21:43.914468+0000","last_active":"2024-03-06T07:22:01.642292+0000","last_peered":"2024-03-06T07:22:01.642292+0000","last_clean":"2024-03-06T07:22:01.642292+0000","last_became_active":"2024-03-06T07:21:43.914210+0000","last_became_peered":"2024-03-06T07:21:43.914210+0000","last_unstale":"2024-03-06T07:22:01.642292+0000","last_undegraded":"2024-03-06T07:22:01.642292+0000","last_fullsized":"2024-03-06T07:22:01.642292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642981+0000","last_change":"2024-03-06T07:21:42.914001+0000","last_active":"2024-03-06T07:22:01.642981+0000","last_peered":"2024-03-06T07:22:01.642981+0000","last_clean":"2024-03-06T07:22:01.642981+0000","last_became_active":"2024-03-06T07:21:42.913707+0000","last_became_peered":"2024-03-06T07:21:42.913707+0000","last_unstale":"2024-03-06T07:22:01.642981+0000","last_undegraded":"2024-03-06T07:22:01.642981+0000","last_fullsized":"2024-03-06T07:22:01.642981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:37.811398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027656+0000","last_change":"2024-03-06T07:22:00.014164+0000","last_active":"2024-03-06T07:22:01.027656+0000","last_peered":"2024-03-06T07:22:01.027656+0000","last_clean":"2024-03-06T07:22:01.027656+0000","last_became_active":"2024-03-06T07:22:00.013808+0000","last_became_peered":"2024-03-06T07:22:00.013808+0000","last_unstale":"2024-03-06T07:22:01.027656+0000","last_undegraded":"2024-03-06T07:22:01.027656+0000","last_fullsized":"2024-03-06T07:22:01.027656+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:12.653672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000398+0000","last_change":"2024-03-06T07:21:59.010253+0000","last_active":"2024-03-06T07:22:01.000398+0000","last_peered":"2024-03-06T07:22:01.000398+0000","last_clean":"2024-03-06T07:22:01.000398+0000","last_became_active":"2024-03-06T07:21:59.009984+0000","last_became_peered":"2024-03-06T07:21:59.009984+0000","last_unstale":"2024-03-06T07:22:01.000398+0000","last_undegraded":"2024-03-06T07:22:01.000398+0000","last_fullsized":"2024-03-06T07:22:01.000398+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:33.376118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000520+0000","last_change":"2024-03-06T07:21:43.928109+0000","last_active":"2024-03-06T07:22:01.000520+0000","last_peered":"2024-03-06T07:22:01.000520+0000","last_clean":"2024-03-06T07:22:01.000520+0000","last_became_active":"2024-03-06T07:21:43.927816+0000","last_became_peered":"2024-03-06T07:21:43.927816+0000","last_unstale":"2024-03-06T07:22:01.000520+0000","last_undegraded":"2024-03-06T07:22:01.000520+0000","last_fullsized":"2024-03-06T07:22:01.000520+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.028118+0000","last_change":"2024-03-06T07:21:42.909376+0000","last_active":"2024-03-06T07:22:01.028118+0000","last_peered":"2024-03-06T07:22:01.028118+0000","last_clean":"2024-03-06T07:22:01.028118+0000","last_became_active":"2024-03-06T07:21:42.909101+0000","last_became_peered":"2024-03-06T07:21:42.909101+0000","last_unstale":"2024-03-06T07:22:01.028118+0000","last_undegraded":"2024-03-06T07:22:01.028118+0000","last_fullsized":"2024-03-06T07:22:01.028118+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:37.646398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001531+0000","last_change":"2024-03-06T07:22:00.016101+0000","last_active":"2024-03-06T07:22:01.001531+0000","last_peered":"2024-03-06T07:22:01.001531+0000","last_clean":"2024-03-06T07:22:01.001531+0000","last_became_active":"2024-03-06T07:22:00.015844+0000","last_became_peered":"2024-03-06T07:22:00.015844+0000","last_unstale":"2024-03-06T07:22:01.001531+0000","last_undegraded":"2024-03-06T07:22:01.001531+0000","last_fullsized":"2024-03-06T07:22:01.001531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:13.109019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150058+0000","last_change":"2024-03-06T07:21:59.015865+0000","last_active":"2024-03-06T07:22:01.150058+0000","last_peered":"2024-03-06T07:22:01.150058+0000","last_clean":"2024-03-06T07:22:01.150058+0000","last_became_active":"2024-03-06T07:21:59.015574+0000","last_became_peered":"2024-03-06T07:21:59.015574+0000","last_unstale":"2024-03-06T07:22:01.150058+0000","last_undegraded":"2024-03-06T07:22:01.150058+0000","last_fullsized":"2024-03-06T07:22:01.150058+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:28.156169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001680+0000","last_change":"2024-03-06T07:21:43.925791+0000","last_active":"2024-03-06T07:22:01.001680+0000","last_peered":"2024-03-06T07:22:01.001680+0000","last_clean":"2024-03-06T07:22:01.001680+0000","last_became_active":"2024-03-06T07:21:43.925489+0000","last_became_peered":"2024-03-06T07:21:43.925489+0000","last_unstale":"2024-03-06T07:22:01.001680+0000","last_undegraded":"2024-03-06T07:22:01.001680+0000","last_fullsized":"2024-03-06T07:22:01.001680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003685+0000","last_change":"2024-03-06T07:21:42.912376+0000","last_active":"2024-03-06T07:22:01.003685+0000","last_peered":"2024-03-06T07:22:01.003685+0000","last_clean":"2024-03-06T07:22:01.003685+0000","last_became_active":"2024-03-06T07:21:42.912102+0000","last_became_peered":"2024-03-06T07:21:42.912102+0000","last_unstale":"2024-03-06T07:22:01.003685+0000","last_undegraded":"2024-03-06T07:22:01.003685+0000","last_fullsized":"2024-03-06T07:22:01.003685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001552+0000","last_change":"2024-03-06T07:22:00.019349+0000","last_active":"2024-03-06T07:22:01.001552+0000","last_peered":"2024-03-06T07:22:01.001552+0000","last_clean":"2024-03-06T07:22:01.001552+0000","last_became_active":"2024-03-06T07:22:00.019092+0000","last_became_peered":"2024-03-06T07:22:00.019092+0000","last_unstale":"2024-03-06T07:22:01.001552+0000","last_undegraded":"2024-03-06T07:22:01.001552+0000","last_fullsized":"2024-03-06T07:22:01.001552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:52.437815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642631+0000","last_change":"2024-03-06T07:21:59.010025+0000","last_active":"2024-03-06T07:22:01.642631+0000","last_peered":"2024-03-06T07:22:01.642631+0000","last_clean":"2024-03-06T07:22:01.642631+0000","last_became_active":"2024-03-06T07:21:59.009738+0000","last_became_peered":"2024-03-06T07:21:59.009738+0000","last_unstale":"2024-03-06T07:22:01.642631+0000","last_undegraded":"2024-03-06T07:22:01.642631+0000","last_fullsized":"2024-03-06T07:22:01.642631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:39.805595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000663+0000","last_change":"2024-03-06T07:21:43.914898+0000","last_active":"2024-03-06T07:22:01.000663+0000","last_peered":"2024-03-06T07:22:01.000663+0000","last_clean":"2024-03-06T07:22:01.000663+0000","last_became_active":"2024-03-06T07:21:43.914368+0000","last_became_peered":"2024-03-06T07:21:43.914368+0000","last_unstale":"2024-03-06T07:22:01.000663+0000","last_undegraded":"2024-03-06T07:22:01.000663+0000","last_fullsized":"2024-03-06T07:22:01.000663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:32:44.975253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003372+0000","last_change":"2024-03-06T07:21:42.913064+0000","last_active":"2024-03-06T07:22:01.003372+0000","last_peered":"2024-03-06T07:22:01.003372+0000","last_clean":"2024-03-06T07:22:01.003372+0000","last_became_active":"2024-03-06T07:21:42.912792+0000","last_became_peered":"2024-03-06T07:21:42.912792+0000","last_unstale":"2024-03-06T07:22:01.003372+0000","last_undegraded":"2024-03-06T07:22:01.003372+0000","last_fullsized":"2024-03-06T07:22:01.003372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:13.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024826+0000","last_change":"2024-03-06T07:22:00.013094+0000","last_active":"2024-03-06T07:22:01.024826+0000","last_peered":"2024-03-06T07:22:01.024826+0000","last_clean":"2024-03-06T07:22:01.024826+0000","last_became_active":"2024-03-06T07:22:00.012286+0000","last_became_peered":"2024-03-06T07:22:00.012286+0000","last_unstale":"2024-03-06T07:22:01.024826+0000","last_undegraded":"2024-03-06T07:22:01.024826+0000","last_fullsized":"2024-03-06T07:22:01.024826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:27.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642678+0000","last_change":"2024-03-06T07:21:59.013484+0000","last_active":"2024-03-06T07:22:01.642678+0000","last_peered":"2024-03-06T07:22:01.642678+0000","last_clean":"2024-03-06T07:22:01.642678+0000","last_became_active":"2024-03-06T07:21:59.013205+0000","last_became_peered":"2024-03-06T07:21:59.013205+0000","last_unstale":"2024-03-06T07:22:01.642678+0000","last_undegraded":"2024-03-06T07:22:01.642678+0000","last_fullsized":"2024-03-06T07:22:01.642678+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:16.376375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024558+0000","last_change":"2024-03-06T07:21:43.912688+0000","last_active":"2024-03-06T07:22:01.024558+0000","last_peered":"2024-03-06T07:22:01.024558+0000","last_clean":"2024-03-06T07:22:01.024558+0000","last_became_active":"2024-03-06T07:21:43.912441+0000","last_became_peered":"2024-03-06T07:21:43.912441+0000","last_unstale":"2024-03-06T07:22:01.024558+0000","last_undegraded":"2024-03-06T07:22:01.024558+0000","last_fullsized":"2024-03-06T07:22:01.024558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000746+0000","last_change":"2024-03-06T07:21:42.911252+0000","last_active":"2024-03-06T07:22:01.000746+0000","last_peered":"2024-03-06T07:22:01.000746+0000","last_clean":"2024-03-06T07:22:01.000746+0000","last_became_active":"2024-03-06T07:21:42.910758+0000","last_became_peered":"2024-03-06T07:21:42.910758+0000","last_unstale":"2024-03-06T07:22:01.000746+0000","last_undegraded":"2024-03-06T07:22:01.000746+0000","last_fullsized":"2024-03-06T07:22:01.000746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.004017+0000","last_change":"2024-03-06T07:22:00.012703+0000","last_active":"2024-03-06T07:22:01.004017+0000","last_peered":"2024-03-06T07:22:01.004017+0000","last_clean":"2024-03-06T07:22:01.004017+0000","last_became_active":"2024-03-06T07:22:00.012444+0000","last_became_peered":"2024-03-06T07:22:00.012444+0000","last_unstale":"2024-03-06T07:22:01.004017+0000","last_undegraded":"2024-03-06T07:22:01.004017+0000","last_fullsized":"2024-03-06T07:22:01.004017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:46.340573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001217+0000","last_change":"2024-03-06T07:21:59.021045+0000","last_active":"2024-03-06T07:22:01.001217+0000","last_peered":"2024-03-06T07:22:01.001217+0000","last_clean":"2024-03-06T07:22:01.001217+0000","last_became_active":"2024-03-06T07:21:59.020784+0000","last_became_peered":"2024-03-06T07:21:59.020784+0000","last_unstale":"2024-03-06T07:22:01.001217+0000","last_undegraded":"2024-03-06T07:22:01.001217+0000","last_fullsized":"2024-03-06T07:22:01.001217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:47.039125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003581+0000","last_change":"2024-03-06T07:21:43.917113+0000","last_active":"2024-03-06T07:22:01.003581+0000","last_peered":"2024-03-06T07:22:01.003581+0000","last_clean":"2024-03-06T07:22:01.003581+0000","last_became_active":"2024-03-06T07:21:43.915472+0000","last_became_peered":"2024-03-06T07:21:43.915472+0000","last_unstale":"2024-03-06T07:22:01.003581+0000","last_undegraded":"2024-03-06T07:22:01.003581+0000","last_fullsized":"2024-03-06T07:22:01.003581+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642768+0000","last_change":"2024-03-06T07:21:42.916636+0000","last_active":"2024-03-06T07:22:01.642768+0000","last_peered":"2024-03-06T07:22:01.642768+0000","last_clean":"2024-03-06T07:22:01.642768+0000","last_became_active":"2024-03-06T07:21:42.916345+0000","last_became_peered":"2024-03-06T07:21:42.916345+0000","last_unstale":"2024-03-06T07:22:01.642768+0000","last_undegraded":"2024-03-06T07:22:01.642768+0000","last_fullsized":"2024-03-06T07:22:01.642768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:59.475185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643522+0000","last_change":"2024-03-06T07:22:00.009310+0000","last_active":"2024-03-06T07:22:01.643522+0000","last_peered":"2024-03-06T07:22:01.643522+0000","last_clean":"2024-03-06T07:22:01.643522+0000","last_became_active":"2024-03-06T07:22:00.009035+0000","last_became_peered":"2024-03-06T07:22:00.009035+0000","last_unstale":"2024-03-06T07:22:01.643522+0000","last_undegraded":"2024-03-06T07:22:01.643522+0000","last_fullsized":"2024-03-06T07:22:01.643522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:55.535875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.025099+0000","last_change":"2024-03-06T07:21:59.012793+0000","last_active":"2024-03-06T07:22:01.025099+0000","last_peered":"2024-03-06T07:22:01.025099+0000","last_clean":"2024-03-06T07:22:01.025099+0000","last_became_active":"2024-03-06T07:21:59.012504+0000","last_became_peered":"2024-03-06T07:21:59.012504+0000","last_unstale":"2024-03-06T07:22:01.025099+0000","last_undegraded":"2024-03-06T07:22:01.025099+0000","last_fullsized":"2024-03-06T07:22:01.025099+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:56.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643187+0000","last_change":"2024-03-06T07:21:43.921341+0000","last_active":"2024-03-06T07:22:01.643187+0000","last_peered":"2024-03-06T07:22:01.643187+0000","last_clean":"2024-03-06T07:22:01.643187+0000","last_became_active":"2024-03-06T07:21:43.921062+0000","last_became_peered":"2024-03-06T07:21:43.921062+0000","last_unstale":"2024-03-06T07:22:01.643187+0000","last_undegraded":"2024-03-06T07:22:01.643187+0000","last_fullsized":"2024-03-06T07:22:01.643187+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:15.340198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643198+0000","last_change":"2024-03-06T07:21:42.912294+0000","last_active":"2024-03-06T07:22:01.643198+0000","last_peered":"2024-03-06T07:22:01.643198+0000","last_clean":"2024-03-06T07:22:01.643198+0000","last_became_active":"2024-03-06T07:21:42.912023+0000","last_became_peered":"2024-03-06T07:21:42.912023+0000","last_unstale":"2024-03-06T07:22:01.643198+0000","last_undegraded":"2024-03-06T07:22:01.643198+0000","last_fullsized":"2024-03-06T07:22:01.643198+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027972+0000","last_change":"2024-03-06T07:22:00.009261+0000","last_active":"2024-03-06T07:22:01.027972+0000","last_peered":"2024-03-06T07:22:01.027972+0000","last_clean":"2024-03-06T07:22:01.027972+0000","last_became_active":"2024-03-06T07:22:00.008928+0000","last_became_peered":"2024-03-06T07:22:00.008928+0000","last_unstale":"2024-03-06T07:22:01.027972+0000","last_undegraded":"2024-03-06T07:22:01.027972+0000","last_fullsized":"2024-03-06T07:22:01.027972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:19:49.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.026543+0000","last_change":"2024-03-06T07:21:59.008051+0000","last_active":"2024-03-06T07:22:01.026543+0000","last_peered":"2024-03-06T07:22:01.026543+0000","last_clean":"2024-03-06T07:22:01.026543+0000","last_became_active":"2024-03-06T07:21:59.007748+0000","last_became_peered":"2024-03-06T07:21:59.007748+0000","last_unstale":"2024-03-06T07:22:01.026543+0000","last_undegraded":"2024-03-06T07:22:01.026543+0000","last_fullsized":"2024-03-06T07:22:01.026543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:45.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001490+0000","last_change":"2024-03-06T07:21:43.905564+0000","last_active":"2024-03-06T07:22:01.001490+0000","last_peered":"2024-03-06T07:22:01.001490+0000","last_clean":"2024-03-06T07:22:01.001490+0000","last_became_active":"2024-03-06T07:21:43.905280+0000","last_became_peered":"2024-03-06T07:21:43.905280+0000","last_unstale":"2024-03-06T07:22:01.001490+0000","last_undegraded":"2024-03-06T07:22:01.001490+0000","last_fullsized":"2024-03-06T07:22:01.001490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027211+0000","last_change":"2024-03-06T07:21:42.908701+0000","last_active":"2024-03-06T07:22:01.027211+0000","last_peered":"2024-03-06T07:22:01.027211+0000","last_clean":"2024-03-06T07:22:01.027211+0000","last_became_active":"2024-03-06T07:21:42.908429+0000","last_became_peered":"2024-03-06T07:21:42.908429+0000","last_unstale":"2024-03-06T07:22:01.027211+0000","last_undegraded":"2024-03-06T07:22:01.027211+0000","last_fullsized":"2024-03-06T07:22:01.027211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:55.863684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002586+0000","last_change":"2024-03-06T07:22:00.012499+0000","last_active":"2024-03-06T07:22:01.002586+0000","last_peered":"2024-03-06T07:22:01.002586+0000","last_clean":"2024-03-06T07:22:01.002586+0000","last_became_active":"2024-03-06T07:22:00.012216+0000","last_became_peered":"2024-03-06T07:22:00.012216+0000","last_unstale":"2024-03-06T07:22:01.002586+0000","last_undegraded":"2024-03-06T07:22:01.002586+0000","last_fullsized":"2024-03-06T07:22:01.002586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:13.838771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643283+0000","last_change":"2024-03-06T07:21:59.013665+0000","last_active":"2024-03-06T07:22:01.643283+0000","last_peered":"2024-03-06T07:22:01.643283+0000","last_clean":"2024-03-06T07:22:01.643283+0000","last_became_active":"2024-03-06T07:21:59.013401+0000","last_became_peered":"2024-03-06T07:21:59.013401+0000","last_unstale":"2024-03-06T07:22:01.643283+0000","last_undegraded":"2024-03-06T07:22:01.643283+0000","last_fullsized":"2024-03-06T07:22:01.643283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:20.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024334+0000","last_change":"2024-03-06T07:21:43.909104+0000","last_active":"2024-03-06T07:22:01.024334+0000","last_peered":"2024-03-06T07:22:01.024334+0000","last_clean":"2024-03-06T07:22:01.024334+0000","last_became_active":"2024-03-06T07:21:43.908810+0000","last_became_peered":"2024-03-06T07:21:43.908810+0000","last_unstale":"2024-03-06T07:22:01.024334+0000","last_undegraded":"2024-03-06T07:22:01.024334+0000","last_fullsized":"2024-03-06T07:22:01.024334+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:14.273650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003969+0000","last_change":"2024-03-06T07:21:42.914080+0000","last_active":"2024-03-06T07:22:01.003969+0000","last_peered":"2024-03-06T07:22:01.003969+0000","last_clean":"2024-03-06T07:22:01.003969+0000","last_became_active":"2024-03-06T07:21:42.913818+0000","last_became_peered":"2024-03-06T07:21:42.913818+0000","last_unstale":"2024-03-06T07:22:01.003969+0000","last_undegraded":"2024-03-06T07:22:01.003969+0000","last_fullsized":"2024-03-06T07:22:01.003969+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:28.725020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642574+0000","last_change":"2024-03-06T07:22:00.010751+0000","last_active":"2024-03-06T07:22:01.642574+0000","last_peered":"2024-03-06T07:22:01.642574+0000","last_clean":"2024-03-06T07:22:01.642574+0000","last_became_active":"2024-03-06T07:22:00.009885+0000","last_became_peered":"2024-03-06T07:22:00.009885+0000","last_unstale":"2024-03-06T07:22:01.642574+0000","last_undegraded":"2024-03-06T07:22:01.642574+0000","last_fullsized":"2024-03-06T07:22:01.642574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:31.898544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642622+0000","last_change":"2024-03-06T07:21:59.012755+0000","last_active":"2024-03-06T07:22:01.642622+0000","last_peered":"2024-03-06T07:22:01.642622+0000","last_clean":"2024-03-06T07:22:01.642622+0000","last_became_active":"2024-03-06T07:21:59.012388+0000","last_became_peered":"2024-03-06T07:21:59.012388+0000","last_unstale":"2024-03-06T07:22:01.642622+0000","last_undegraded":"2024-03-06T07:22:01.642622+0000","last_fullsized":"2024-03-06T07:22:01.642622+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:13.064323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000959+0000","last_change":"2024-03-06T07:21:43.917579+0000","last_active":"2024-03-06T07:22:01.000959+0000","last_peered":"2024-03-06T07:22:01.000959+0000","last_clean":"2024-03-06T07:22:01.000959+0000","last_became_active":"2024-03-06T07:21:43.916663+0000","last_became_peered":"2024-03-06T07:21:43.916663+0000","last_unstale":"2024-03-06T07:22:01.000959+0000","last_undegraded":"2024-03-06T07:22:01.000959+0000","last_fullsized":"2024-03-06T07:22:01.000959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:43.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150702+0000","last_change":"2024-03-06T07:21:42.912139+0000","last_active":"2024-03-06T07:22:01.150702+0000","last_peered":"2024-03-06T07:22:01.150702+0000","last_clean":"2024-03-06T07:22:01.150702+0000","last_became_active":"2024-03-06T07:21:42.911573+0000","last_became_peered":"2024-03-06T07:21:42.911573+0000","last_unstale":"2024-03-06T07:22:01.150702+0000","last_undegraded":"2024-03-06T07:22:01.150702+0000","last_fullsized":"2024-03-06T07:22:01.150702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024289+0000","last_change":"2024-03-06T07:22:00.012775+0000","last_active":"2024-03-06T07:22:01.024289+0000","last_peered":"2024-03-06T07:22:01.024289+0000","last_clean":"2024-03-06T07:22:01.024289+0000","last_became_active":"2024-03-06T07:22:00.012122+0000","last_became_peered":"2024-03-06T07:22:00.012122+0000","last_unstale":"2024-03-06T07:22:01.024289+0000","last_undegraded":"2024-03-06T07:22:01.024289+0000","last_fullsized":"2024-03-06T07:22:01.024289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:14.068723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024560+0000","last_change":"2024-03-06T07:21:59.010619+0000","last_active":"2024-03-06T07:22:01.024560+0000","last_peered":"2024-03-06T07:22:01.024560+0000","last_clean":"2024-03-06T07:22:01.024560+0000","last_became_active":"2024-03-06T07:21:59.010286+0000","last_became_peered":"2024-03-06T07:21:59.010286+0000","last_unstale":"2024-03-06T07:22:01.024560+0000","last_undegraded":"2024-03-06T07:22:01.024560+0000","last_fullsized":"2024-03-06T07:22:01.024560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:58.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643191+0000","last_change":"2024-03-06T07:21:43.912734+0000","last_active":"2024-03-06T07:22:01.643191+0000","last_peered":"2024-03-06T07:22:01.643191+0000","last_clean":"2024-03-06T07:22:01.643191+0000","last_became_active":"2024-03-06T07:21:43.912449+0000","last_became_peered":"2024-03-06T07:21:43.912449+0000","last_unstale":"2024-03-06T07:22:01.643191+0000","last_undegraded":"2024-03-06T07:22:01.643191+0000","last_fullsized":"2024-03-06T07:22:01.643191+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643724+0000","last_change":"2024-03-06T07:21:42.907567+0000","last_active":"2024-03-06T07:22:01.643724+0000","last_peered":"2024-03-06T07:22:01.643724+0000","last_clean":"2024-03-06T07:22:01.643724+0000","last_became_active":"2024-03-06T07:21:42.907271+0000","last_became_peered":"2024-03-06T07:21:42.907271+0000","last_unstale":"2024-03-06T07:22:01.643724+0000","last_undegraded":"2024-03-06T07:22:01.643724+0000","last_fullsized":"2024-03-06T07:22:01.643724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003128+0000","last_change":"2024-03-06T07:22:00.013666+0000","last_active":"2024-03-06T07:22:01.003128+0000","last_peered":"2024-03-06T07:22:01.003128+0000","last_clean":"2024-03-06T07:22:01.003128+0000","last_became_active":"2024-03-06T07:22:00.013416+0000","last_became_peered":"2024-03-06T07:22:00.013416+0000","last_unstale":"2024-03-06T07:22:01.003128+0000","last_undegraded":"2024-03-06T07:22:01.003128+0000","last_fullsized":"2024-03-06T07:22:01.003128+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:45.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150277+0000","last_change":"2024-03-06T07:21:59.015347+0000","last_active":"2024-03-06T07:22:01.150277+0000","last_peered":"2024-03-06T07:22:01.150277+0000","last_clean":"2024-03-06T07:22:01.150277+0000","last_became_active":"2024-03-06T07:21:59.014691+0000","last_became_peered":"2024-03-06T07:21:59.014691+0000","last_unstale":"2024-03-06T07:22:01.150277+0000","last_undegraded":"2024-03-06T07:22:01.150277+0000","last_fullsized":"2024-03-06T07:22:01.150277+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:00.318477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643772+0000","last_change":"2024-03-06T07:21:43.921555+0000","last_active":"2024-03-06T07:22:01.643772+0000","last_peered":"2024-03-06T07:22:01.643772+0000","last_clean":"2024-03-06T07:22:01.643772+0000","last_became_active":"2024-03-06T07:21:43.921242+0000","last_became_peered":"2024-03-06T07:21:43.921242+0000","last_unstale":"2024-03-06T07:22:01.643772+0000","last_undegraded":"2024-03-06T07:22:01.643772+0000","last_fullsized":"2024-03-06T07:22:01.643772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:31.942191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643425+0000","last_change":"2024-03-06T07:21:42.915454+0000","last_active":"2024-03-06T07:22:01.643425+0000","last_peered":"2024-03-06T07:22:01.643425+0000","last_clean":"2024-03-06T07:22:01.643425+0000","last_became_active":"2024-03-06T07:21:42.915127+0000","last_became_peered":"2024-03-06T07:21:42.915127+0000","last_unstale":"2024-03-06T07:22:01.643425+0000","last_undegraded":"2024-03-06T07:22:01.643425+0000","last_fullsized":"2024-03-06T07:22:01.643425+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:14.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003832+0000","last_change":"2024-03-06T07:22:00.017481+0000","last_active":"2024-03-06T07:22:01.003832+0000","last_peered":"2024-03-06T07:22:01.003832+0000","last_clean":"2024-03-06T07:22:01.003832+0000","last_became_active":"2024-03-06T07:22:00.017193+0000","last_became_peered":"2024-03-06T07:22:00.017193+0000","last_unstale":"2024-03-06T07:22:01.003832+0000","last_undegraded":"2024-03-06T07:22:01.003832+0000","last_fullsized":"2024-03-06T07:22:01.003832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:57.249978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643591+0000","last_change":"2024-03-06T07:21:59.021225+0000","last_active":"2024-03-06T07:22:01.643591+0000","last_peered":"2024-03-06T07:22:01.643591+0000","last_clean":"2024-03-06T07:22:01.643591+0000","last_became_active":"2024-03-06T07:21:59.020920+0000","last_became_peered":"2024-03-06T07:21:59.020920+0000","last_unstale":"2024-03-06T07:22:01.643591+0000","last_undegraded":"2024-03-06T07:22:01.643591+0000","last_fullsized":"2024-03-06T07:22:01.643591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:46.358444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024914+0000","last_change":"2024-03-06T07:21:43.919585+0000","last_active":"2024-03-06T07:22:01.024914+0000","last_peered":"2024-03-06T07:22:01.024914+0000","last_clean":"2024-03-06T07:22:01.024914+0000","last_became_active":"2024-03-06T07:21:43.919255+0000","last_became_peered":"2024-03-06T07:21:43.919255+0000","last_unstale":"2024-03-06T07:22:01.024914+0000","last_undegraded":"2024-03-06T07:22:01.024914+0000","last_fullsized":"2024-03-06T07:22:01.024914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642860+0000","last_change":"2024-03-06T07:21:42.912512+0000","last_active":"2024-03-06T07:22:01.642860+0000","last_peered":"2024-03-06T07:22:01.642860+0000","last_clean":"2024-03-06T07:22:01.642860+0000","last_became_active":"2024-03-06T07:21:42.912235+0000","last_became_peered":"2024-03-06T07:21:42.912235+0000","last_unstale":"2024-03-06T07:22:01.642860+0000","last_undegraded":"2024-03-06T07:22:01.642860+0000","last_fullsized":"2024-03-06T07:22:01.642860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.427916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150157+0000","last_change":"2024-03-06T07:22:00.030378+0000","last_active":"2024-03-06T07:22:01.150157+0000","last_peered":"2024-03-06T07:22:01.150157+0000","last_clean":"2024-03-06T07:22:01.150157+0000","last_became_active":"2024-03-06T07:22:00.030094+0000","last_became_peered":"2024-03-06T07:22:00.030094+0000","last_unstale":"2024-03-06T07:22:01.150157+0000","last_undegraded":"2024-03-06T07:22:01.150157+0000","last_fullsized":"2024-03-06T07:22:01.150157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:36.032336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024923+0000","last_change":"2024-03-06T07:21:59.011385+0000","last_active":"2024-03-06T07:22:01.024923+0000","last_peered":"2024-03-06T07:22:01.024923+0000","last_clean":"2024-03-06T07:22:01.024923+0000","last_became_active":"2024-03-06T07:21:59.011065+0000","last_became_peered":"2024-03-06T07:21:59.011065+0000","last_unstale":"2024-03-06T07:22:01.024923+0000","last_undegraded":"2024-03-06T07:22:01.024923+0000","last_fullsized":"2024-03-06T07:22:01.024923+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:22.451144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643092+0000","last_change":"2024-03-06T07:21:43.933046+0000","last_active":"2024-03-06T07:22:01.643092+0000","last_peered":"2024-03-06T07:22:01.643092+0000","last_clean":"2024-03-06T07:22:01.643092+0000","last_became_active":"2024-03-06T07:21:43.932764+0000","last_became_peered":"2024-03-06T07:21:43.932764+0000","last_unstale":"2024-03-06T07:22:01.643092+0000","last_undegraded":"2024-03-06T07:22:01.643092+0000","last_fullsized":"2024-03-06T07:22:01.643092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:47.292953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002507+0000","last_change":"2024-03-06T07:21:42.912274+0000","last_active":"2024-03-06T07:22:01.002507+0000","last_peered":"2024-03-06T07:22:01.002507+0000","last_clean":"2024-03-06T07:22:01.002507+0000","last_became_active":"2024-03-06T07:21:42.912000+0000","last_became_peered":"2024-03-06T07:21:42.912000+0000","last_unstale":"2024-03-06T07:22:01.002507+0000","last_undegraded":"2024-03-06T07:22:01.002507+0000","last_fullsized":"2024-03-06T07:22:01.002507+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:00.006841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003092+0000","last_change":"2024-03-06T07:22:00.014240+0000","last_active":"2024-03-06T07:22:01.003092+0000","last_peered":"2024-03-06T07:22:01.003092+0000","last_clean":"2024-03-06T07:22:01.003092+0000","last_became_active":"2024-03-06T07:22:00.013819+0000","last_became_peered":"2024-03-06T07:22:00.013819+0000","last_unstale":"2024-03-06T07:22:01.003092+0000","last_undegraded":"2024-03-06T07:22:01.003092+0000","last_fullsized":"2024-03-06T07:22:01.003092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:01.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150387+0000","last_change":"2024-03-06T07:21:59.021336+0000","last_active":"2024-03-06T07:22:01.150387+0000","last_peered":"2024-03-06T07:22:01.150387+0000","last_clean":"2024-03-06T07:22:01.150387+0000","last_became_active":"2024-03-06T07:21:59.021048+0000","last_became_peered":"2024-03-06T07:21:59.021048+0000","last_unstale":"2024-03-06T07:22:01.150387+0000","last_undegraded":"2024-03-06T07:22:01.150387+0000","last_fullsized":"2024-03-06T07:22:01.150387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:22.451144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643974+0000","last_change":"2024-03-06T07:21:43.906537+0000","last_active":"2024-03-06T07:22:01.643974+0000","last_peered":"2024-03-06T07:22:01.643974+0000","last_clean":"2024-03-06T07:22:01.643974+0000","last_became_active":"2024-03-06T07:21:43.906267+0000","last_became_peered":"2024-03-06T07:21:43.906267+0000","last_unstale":"2024-03-06T07:22:01.643974+0000","last_undegraded":"2024-03-06T07:22:01.643974+0000","last_fullsized":"2024-03-06T07:22:01.643974+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:45.221948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024662+0000","last_change":"2024-03-06T07:21:42.912363+0000","last_active":"2024-03-06T07:22:01.024662+0000","last_peered":"2024-03-06T07:22:01.024662+0000","last_clean":"2024-03-06T07:22:01.024662+0000","last_became_active":"2024-03-06T07:21:42.912097+0000","last_became_peered":"2024-03-06T07:21:42.912097+0000","last_unstale":"2024-03-06T07:22:01.024662+0000","last_undegraded":"2024-03-06T07:22:01.024662+0000","last_fullsized":"2024-03-06T07:22:01.024662+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:13.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002208+0000","last_change":"2024-03-06T07:22:00.015444+0000","last_active":"2024-03-06T07:22:01.002208+0000","last_peered":"2024-03-06T07:22:01.002208+0000","last_clean":"2024-03-06T07:22:01.002208+0000","last_became_active":"2024-03-06T07:22:00.015191+0000","last_became_peered":"2024-03-06T07:22:00.015191+0000","last_unstale":"2024-03-06T07:22:01.002208+0000","last_undegraded":"2024-03-06T07:22:01.002208+0000","last_fullsized":"2024-03-06T07:22:01.002208+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:48.043526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642183+0000","last_change":"2024-03-06T07:21:59.008775+0000","last_active":"2024-03-06T07:22:01.642183+0000","last_peered":"2024-03-06T07:22:01.642183+0000","last_clean":"2024-03-06T07:22:01.642183+0000","last_became_active":"2024-03-06T07:21:59.008479+0000","last_became_peered":"2024-03-06T07:21:59.008479+0000","last_unstale":"2024-03-06T07:22:01.642183+0000","last_undegraded":"2024-03-06T07:22:01.642183+0000","last_fullsized":"2024-03-06T07:22:01.642183+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:12.372186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002052+0000","last_change":"2024-03-06T07:21:43.913294+0000","last_active":"2024-03-06T07:22:01.002052+0000","last_peered":"2024-03-06T07:22:01.002052+0000","last_clean":"2024-03-06T07:22:01.002052+0000","last_became_active":"2024-03-06T07:21:43.913017+0000","last_became_peered":"2024-03-06T07:21:43.913017+0000","last_unstale":"2024-03-06T07:22:01.002052+0000","last_undegraded":"2024-03-06T07:22:01.002052+0000","last_fullsized":"2024-03-06T07:22:01.002052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:26.287336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643281+0000","last_change":"2024-03-06T07:21:42.912585+0000","last_active":"2024-03-06T07:22:01.643281+0000","last_peered":"2024-03-06T07:22:01.643281+0000","last_clean":"2024-03-06T07:22:01.643281+0000","last_became_active":"2024-03-06T07:21:42.911650+0000","last_became_peered":"2024-03-06T07:21:42.911650+0000","last_unstale":"2024-03-06T07:22:01.643281+0000","last_undegraded":"2024-03-06T07:22:01.643281+0000","last_fullsized":"2024-03-06T07:22:01.643281+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:42.887622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149855+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:01.149855+0000","last_peered":"2024-03-06T07:22:01.149855+0000","last_clean":"2024-03-06T07:22:01.149855+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:01.149855+0000","last_undegraded":"2024-03-06T07:22:01.149855+0000","last_fullsized":"2024-03-06T07:22:01.149855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":770048,"data_stored":906845,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":316263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":315599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":317771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":317211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":317189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":318113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":741376,"data_stored":898394,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:04.173 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:04.208 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:22:04.205+0000 7fe7e845e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.337+0000 7f635464e640 1 Processor -- start 2024-03-06T07:22:04.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.337+0000 7f635464e640 1 -- start start 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f635464e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f634c1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f635464e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c1591c0 con 0x7f634c155ba0 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f635464e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634c1594a0 con 0x7f634c0c5480 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f635464e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c06df40 con 0x7f634c0c5860 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f6351be9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f634c155ba0 0x7f634c1586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55656/0 (socket says 172.21.15.17:55656) 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f6351be9640 1 -- 172.21.15.17:0/3453358036 learned_addr learned my addr 172.21.15.17:0/3453358036 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f6352beb640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f634c1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764538753 0 0) 0x7f634c1591c0 con 0x7f634c155ba0 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6338003520 con 0x7f634c155ba0 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3888727572 0 0) 0x7f6338003520 con 0x7f634c155ba0 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 msgr2=0x7f634c1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f634c1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 >> v1:172.21.15.163:6789/0 conn(0x7f634c0c5480 legacy=0x7f634c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c1553a0 con 0x7f634c155ba0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f633c001670 con 0x7f634c155ba0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 >> v1:172.21.15.17:6789/0 conn(0x7f634c155ba0 legacy=0x7f634c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63380039e0 0x7f6338003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f6338004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6338004d40 0x7f63380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c06df40 con 0x7f63380039e0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633800baa0 con 0x7f6338004d40 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633800bca0 con 0x7f634c0c5860 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.338+0000 7f63523ea640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63380039e0 0x7f6338003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f6351be9640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f6338004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f6352beb640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6338004d40 0x7f63380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f63523ea640 1 -- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 msgr2=0x7f6338004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f63523ea640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f634c0c5860 0x7f6338004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f63523ea640 1 -- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6338004d40 msgr2=0x7f63380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f63523ea640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6338004d40 0x7f63380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f63523ea640 1 -- 172.21.15.17:0/3453358036 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338003520 con 0x7f63380039e0 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.339+0000 7f6352beb640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6338004d40 0x7f63380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.340+0000 7f63523ea640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63380039e0 0x7f6338003dc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6348001910 tx=0x7f6338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717af8ddbae746d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.340+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63480014e0 con 0x7f63380039e0 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.340+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6348037070 con 0x7f63380039e0 2024-03-06T07:22:04.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.340+0000 7f63513e8640 1 -- 172.21.15.17:0/3453358036 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348032870 con 0x7f63380039e0 2024-03-06T07:22:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 -- 172.21.15.17:0/3453358036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63380039e0 msgr2=0x7f6338003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 --2- 172.21.15.17:0/3453358036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63380039e0 0x7f6338003dc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6348001910 tx=0x7f6338005830 comp rx=0 tx=0).stop 2024-03-06T07:22:04.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f6352beb640 1 -- 172.21.15.17:0/3453358036 reap_dead start 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 -- 172.21.15.17:0/3453358036 shutdown_connections 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 -- 172.21.15.17:0/3453358036 >> 172.21.15.17:0/3453358036 conn(0x7f634c06b1a0 msgr2=0x7f634c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 -- 172.21.15.17:0/3453358036 shutdown_connections 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.341+0000 7f635464e640 1 -- 172.21.15.17:0/3453358036 wait complete. 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.342+0000 7f635464e640 1 Processor -- start 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.342+0000 7f635464e640 1 -- start start 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63380039e0 0x7f634c1deea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 0x7f634c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c0c5cf0 con 0x7f634c151f10 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634c06fdb0 con 0x7f634c151ab0 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f635464e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634c156000 con 0x7f63380039e0 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35772/0 (socket says 172.21.15.17:35772) 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 -- 172.21.15.17:0/3058372914 learned_addr learned my addr 172.21.15.17:0/3058372914 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f63523ea640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63380039e0 0x7f634c1deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6352beb640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 0x7f634c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 -- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63380039e0 msgr2=0x7f634c1deea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63380039e0 0x7f634c1deea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 -- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 msgr2=0x7f634c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 0x7f634c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 -- 172.21.15.17:0/3058372914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c1ef0f0 con 0x7f634c151f10 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6352beb640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 0x7f634c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6351be9640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f633c004ed0 tx=0x7f633c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.343+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00f750 con 0x7f634c151f10 2024-03-06T07:22:04.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.344+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c018070 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.344+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634c0a87d0 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.344+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c00fb50 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.344+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634c0a8ab0 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.345+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634c069460 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.345+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f633c00fcf0 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.346+0000 7f6342ffd640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 0x7f633006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.346+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f633c08a590 con 0x7f634c151f10 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.346+0000 7f63523ea640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 0x7f633006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.347+0000 7f63523ea640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 0x7f633006d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6348036000 tx=0x7f6348009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.350+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f634c069460 con 0x7f634c151f10 2024-03-06T07:22:04.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.525+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f634c04f330 con 0x7f633006b500 2024-03-06T07:22:04.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.538+0000 7f6342ffd640 1 -- 172.21.15.17:0/3058372914 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490559 (secure 0 0 0) 0x7f634c04f330 con 0x7f633006b500 2024-03-06T07:22:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:04.548 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 msgr2=0x7f633006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 0x7f633006d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6348036000 tx=0x7f6348009030 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 msgr2=0x7f634c151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f633c004ed0 tx=0x7f633c007430 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 shutdown_connections 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f633006b500 0x7f633006d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f634c151ab0 0x7f634c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f634c151f10 0x7f634c151570 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 --2- 172.21.15.17:0/3058372914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63380039e0 0x7f634c1deea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 >> 172.21.15.17:0/3058372914 conn(0x7f634c06b1a0 msgr2=0x7f634c0c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 shutdown_connections 2024-03-06T07:22:04.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.541+0000 7f635464e640 1 -- 172.21.15.17:0/3058372914 wait complete. 2024-03-06T07:22:04.558 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-06T07:22:03.367492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":343,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238404,"kb_used_data":5556,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736316,"statfs":{"total":773094113280,"available":772849987584,"internally_reserved":0,"allocated":5689344,"data_stored":3707385,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.375849"},"pg_stats":[{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150427+0000","last_change":"2024-03-06T07:22:00.021272+0000","last_active":"2024-03-06T07:22:01.150427+0000","last_peered":"2024-03-06T07:22:01.150427+0000","last_clean":"2024-03-06T07:22:01.150427+0000","last_became_active":"2024-03-06T07:22:00.021015+0000","last_became_peered":"2024-03-06T07:22:00.021015+0000","last_unstale":"2024-03-06T07:22:01.150427+0000","last_undegraded":"2024-03-06T07:22:01.150427+0000","last_fullsized":"2024-03-06T07:22:01.150427+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001472+0000","last_change":"2024-03-06T07:21:43.913948+0000","last_active":"2024-03-06T07:22:01.001472+0000","last_peered":"2024-03-06T07:22:01.001472+0000","last_clean":"2024-03-06T07:22:01.001472+0000","last_became_active":"2024-03-06T07:21:43.911415+0000","last_became_peered":"2024-03-06T07:21:43.911415+0000","last_unstale":"2024-03-06T07:22:01.001472+0000","last_undegraded":"2024-03-06T07:22:01.001472+0000","last_fullsized":"2024-03-06T07:22:01.001472+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:51.582422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003320+0000","last_change":"2024-03-06T07:22:00.011088+0000","last_active":"2024-03-06T07:22:01.003320+0000","last_peered":"2024-03-06T07:22:01.003320+0000","last_clean":"2024-03-06T07:22:01.003320+0000","last_became_active":"2024-03-06T07:22:00.010827+0000","last_became_peered":"2024-03-06T07:22:00.010827+0000","last_unstale":"2024-03-06T07:22:01.003320+0000","last_undegraded":"2024-03-06T07:22:01.003320+0000","last_fullsized":"2024-03-06T07:22:01.003320+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:23.455544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150526+0000","last_change":"2024-03-06T07:21:43.914337+0000","last_active":"2024-03-06T07:22:01.150526+0000","last_peered":"2024-03-06T07:22:01.150526+0000","last_clean":"2024-03-06T07:22:01.150526+0000","last_became_active":"2024-03-06T07:21:43.914043+0000","last_became_peered":"2024-03-06T07:21:43.914043+0000","last_unstale":"2024-03-06T07:22:01.150526+0000","last_undegraded":"2024-03-06T07:22:01.150526+0000","last_fullsized":"2024-03-06T07:22:01.150526+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.432948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001336+0000","last_change":"2024-03-06T07:22:00.017320+0000","last_active":"2024-03-06T07:22:01.001336+0000","last_peered":"2024-03-06T07:22:01.001336+0000","last_clean":"2024-03-06T07:22:01.001336+0000","last_became_active":"2024-03-06T07:22:00.016927+0000","last_became_peered":"2024-03-06T07:22:00.016927+0000","last_unstale":"2024-03-06T07:22:01.001336+0000","last_undegraded":"2024-03-06T07:22:01.001336+0000","last_fullsized":"2024-03-06T07:22:01.001336+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:10.016374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642785+0000","last_change":"2024-03-06T07:21:43.935409+0000","last_active":"2024-03-06T07:22:01.642785+0000","last_peered":"2024-03-06T07:22:01.642785+0000","last_clean":"2024-03-06T07:22:01.642785+0000","last_became_active":"2024-03-06T07:21:43.934832+0000","last_became_peered":"2024-03-06T07:21:43.934832+0000","last_unstale":"2024-03-06T07:22:01.642785+0000","last_undegraded":"2024-03-06T07:22:01.642785+0000","last_fullsized":"2024-03-06T07:22:01.642785+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:57.737436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001262+0000","last_change":"2024-03-06T07:22:00.012621+0000","last_active":"2024-03-06T07:22:01.001262+0000","last_peered":"2024-03-06T07:22:01.001262+0000","last_clean":"2024-03-06T07:22:01.001262+0000","last_became_active":"2024-03-06T07:22:00.012323+0000","last_became_peered":"2024-03-06T07:22:00.012323+0000","last_unstale":"2024-03-06T07:22:01.001262+0000","last_undegraded":"2024-03-06T07:22:01.001262+0000","last_fullsized":"2024-03-06T07:22:01.001262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:30.690160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150223+0000","last_change":"2024-03-06T07:21:43.913875+0000","last_active":"2024-03-06T07:22:01.150223+0000","last_peered":"2024-03-06T07:22:01.150223+0000","last_clean":"2024-03-06T07:22:01.150223+0000","last_became_active":"2024-03-06T07:21:43.912314+0000","last_became_peered":"2024-03-06T07:21:43.912314+0000","last_unstale":"2024-03-06T07:22:01.150223+0000","last_undegraded":"2024-03-06T07:22:01.150223+0000","last_fullsized":"2024-03-06T07:22:01.150223+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:01.595661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027227+0000","last_change":"2024-03-06T07:22:00.019347+0000","last_active":"2024-03-06T07:22:01.027227+0000","last_peered":"2024-03-06T07:22:01.027227+0000","last_clean":"2024-03-06T07:22:01.027227+0000","last_became_active":"2024-03-06T07:22:00.019096+0000","last_became_peered":"2024-03-06T07:22:00.019096+0000","last_unstale":"2024-03-06T07:22:01.027227+0000","last_undegraded":"2024-03-06T07:22:01.027227+0000","last_fullsized":"2024-03-06T07:22:01.027227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:19.143693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002353+0000","last_change":"2024-03-06T07:21:43.913216+0000","last_active":"2024-03-06T07:22:01.002353+0000","last_peered":"2024-03-06T07:22:01.002353+0000","last_clean":"2024-03-06T07:22:01.002353+0000","last_became_active":"2024-03-06T07:21:43.912935+0000","last_became_peered":"2024-03-06T07:21:43.912935+0000","last_unstale":"2024-03-06T07:22:01.002353+0000","last_undegraded":"2024-03-06T07:22:01.002353+0000","last_fullsized":"2024-03-06T07:22:01.002353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024256+0000","last_change":"2024-03-06T07:22:00.020457+0000","last_active":"2024-03-06T07:22:01.024256+0000","last_peered":"2024-03-06T07:22:01.024256+0000","last_clean":"2024-03-06T07:22:01.024256+0000","last_became_active":"2024-03-06T07:22:00.020191+0000","last_became_peered":"2024-03-06T07:22:00.020191+0000","last_unstale":"2024-03-06T07:22:01.024256+0000","last_undegraded":"2024-03-06T07:22:01.024256+0000","last_fullsized":"2024-03-06T07:22:01.024256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.380776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000860+0000","last_change":"2024-03-06T07:21:43.917357+0000","last_active":"2024-03-06T07:22:01.000860+0000","last_peered":"2024-03-06T07:22:01.000860+0000","last_clean":"2024-03-06T07:22:01.000860+0000","last_became_active":"2024-03-06T07:21:43.916973+0000","last_became_peered":"2024-03-06T07:21:43.916973+0000","last_unstale":"2024-03-06T07:22:01.000860+0000","last_undegraded":"2024-03-06T07:22:01.000860+0000","last_fullsized":"2024-03-06T07:22:01.000860+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002044+0000","last_change":"2024-03-06T07:22:00.013447+0000","last_active":"2024-03-06T07:22:01.002044+0000","last_peered":"2024-03-06T07:22:01.002044+0000","last_clean":"2024-03-06T07:22:01.002044+0000","last_became_active":"2024-03-06T07:22:00.013137+0000","last_became_peered":"2024-03-06T07:22:00.013137+0000","last_unstale":"2024-03-06T07:22:01.002044+0000","last_undegraded":"2024-03-06T07:22:01.002044+0000","last_fullsized":"2024-03-06T07:22:01.002044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:52.437815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001803+0000","last_change":"2024-03-06T07:21:54.169067+0000","last_active":"2024-03-06T07:22:01.001803+0000","last_peered":"2024-03-06T07:22:01.001803+0000","last_clean":"2024-03-06T07:22:01.001803+0000","last_became_active":"2024-03-06T07:21:54.168778+0000","last_became_peered":"2024-03-06T07:21:54.168778+0000","last_unstale":"2024-03-06T07:22:01.001803+0000","last_undegraded":"2024-03-06T07:22:01.001803+0000","last_fullsized":"2024-03-06T07:22:01.001803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643264+0000","last_change":"2024-03-06T07:22:00.018830+0000","last_active":"2024-03-06T07:22:01.643264+0000","last_peered":"2024-03-06T07:22:01.643264+0000","last_clean":"2024-03-06T07:22:01.643264+0000","last_became_active":"2024-03-06T07:22:00.018563+0000","last_became_peered":"2024-03-06T07:22:00.018563+0000","last_unstale":"2024-03-06T07:22:01.643264+0000","last_undegraded":"2024-03-06T07:22:01.643264+0000","last_fullsized":"2024-03-06T07:22:01.643264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:47:32.925919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642260+0000","last_change":"2024-03-06T07:21:43.928597+0000","last_active":"2024-03-06T07:22:01.642260+0000","last_peered":"2024-03-06T07:22:01.642260+0000","last_clean":"2024-03-06T07:22:01.642260+0000","last_became_active":"2024-03-06T07:21:43.928330+0000","last_became_peered":"2024-03-06T07:21:43.928330+0000","last_unstale":"2024-03-06T07:22:01.642260+0000","last_undegraded":"2024-03-06T07:22:01.642260+0000","last_fullsized":"2024-03-06T07:22:01.642260+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002363+0000","last_change":"2024-03-06T07:22:00.013119+0000","last_active":"2024-03-06T07:22:01.002363+0000","last_peered":"2024-03-06T07:22:01.002363+0000","last_clean":"2024-03-06T07:22:01.002363+0000","last_became_active":"2024-03-06T07:22:00.012851+0000","last_became_peered":"2024-03-06T07:22:00.012851+0000","last_unstale":"2024-03-06T07:22:01.002363+0000","last_undegraded":"2024-03-06T07:22:01.002363+0000","last_fullsized":"2024-03-06T07:22:01.002363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:31.441532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643975+0000","last_change":"2024-03-06T07:21:43.938313+0000","last_active":"2024-03-06T07:22:01.643975+0000","last_peered":"2024-03-06T07:22:01.643975+0000","last_clean":"2024-03-06T07:22:01.643975+0000","last_became_active":"2024-03-06T07:21:43.938037+0000","last_became_peered":"2024-03-06T07:21:43.938037+0000","last_unstale":"2024-03-06T07:22:01.643975+0000","last_undegraded":"2024-03-06T07:22:01.643975+0000","last_fullsized":"2024-03-06T07:22:01.643975+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:12.325672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024615+0000","last_change":"2024-03-06T07:22:00.016705+0000","last_active":"2024-03-06T07:22:01.024615+0000","last_peered":"2024-03-06T07:22:01.024615+0000","last_clean":"2024-03-06T07:22:01.024615+0000","last_became_active":"2024-03-06T07:22:00.016416+0000","last_became_peered":"2024-03-06T07:22:00.016416+0000","last_unstale":"2024-03-06T07:22:01.024615+0000","last_undegraded":"2024-03-06T07:22:01.024615+0000","last_fullsized":"2024-03-06T07:22:01.024615+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:13.376587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001317+0000","last_change":"2024-03-06T07:21:43.910022+0000","last_active":"2024-03-06T07:22:01.001317+0000","last_peered":"2024-03-06T07:22:01.001317+0000","last_clean":"2024-03-06T07:22:01.001317+0000","last_became_active":"2024-03-06T07:21:43.909753+0000","last_became_peered":"2024-03-06T07:21:43.909753+0000","last_unstale":"2024-03-06T07:22:01.001317+0000","last_undegraded":"2024-03-06T07:22:01.001317+0000","last_fullsized":"2024-03-06T07:22:01.001317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:14.243609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150533+0000","last_change":"2024-03-06T07:22:00.031071+0000","last_active":"2024-03-06T07:22:01.150533+0000","last_peered":"2024-03-06T07:22:01.150533+0000","last_clean":"2024-03-06T07:22:01.150533+0000","last_became_active":"2024-03-06T07:22:00.030369+0000","last_became_peered":"2024-03-06T07:22:00.030369+0000","last_unstale":"2024-03-06T07:22:01.150533+0000","last_undegraded":"2024-03-06T07:22:01.150533+0000","last_fullsized":"2024-03-06T07:22:01.150533+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:27.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027933+0000","last_change":"2024-03-06T07:21:43.917615+0000","last_active":"2024-03-06T07:22:01.027933+0000","last_peered":"2024-03-06T07:22:01.027933+0000","last_clean":"2024-03-06T07:22:01.027933+0000","last_became_active":"2024-03-06T07:21:43.917363+0000","last_became_peered":"2024-03-06T07:21:43.917363+0000","last_unstale":"2024-03-06T07:22:01.027933+0000","last_undegraded":"2024-03-06T07:22:01.027933+0000","last_fullsized":"2024-03-06T07:22:01.027933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:29.125938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150411+0000","last_change":"2024-03-06T07:22:00.023374+0000","last_active":"2024-03-06T07:22:01.150411+0000","last_peered":"2024-03-06T07:22:01.150411+0000","last_clean":"2024-03-06T07:22:01.150411+0000","last_became_active":"2024-03-06T07:22:00.022882+0000","last_became_peered":"2024-03-06T07:22:00.022882+0000","last_unstale":"2024-03-06T07:22:01.150411+0000","last_undegraded":"2024-03-06T07:22:01.150411+0000","last_fullsized":"2024-03-06T07:22:01.150411+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:13.376587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643701+0000","last_change":"2024-03-06T07:21:43.911518+0000","last_active":"2024-03-06T07:22:01.643701+0000","last_peered":"2024-03-06T07:22:01.643701+0000","last_clean":"2024-03-06T07:22:01.643701+0000","last_became_active":"2024-03-06T07:21:43.911238+0000","last_became_peered":"2024-03-06T07:21:43.911238+0000","last_unstale":"2024-03-06T07:22:01.643701+0000","last_undegraded":"2024-03-06T07:22:01.643701+0000","last_fullsized":"2024-03-06T07:22:01.643701+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:14.588826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149924+0000","last_change":"2024-03-06T07:22:00.018284+0000","last_active":"2024-03-06T07:22:01.149924+0000","last_peered":"2024-03-06T07:22:01.149924+0000","last_clean":"2024-03-06T07:22:01.149924+0000","last_became_active":"2024-03-06T07:22:00.017996+0000","last_became_peered":"2024-03-06T07:22:00.017996+0000","last_unstale":"2024-03-06T07:22:01.149924+0000","last_undegraded":"2024-03-06T07:22:01.149924+0000","last_fullsized":"2024-03-06T07:22:01.149924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:17.380776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000595+0000","last_change":"2024-03-06T07:21:43.916170+0000","last_active":"2024-03-06T07:22:01.000595+0000","last_peered":"2024-03-06T07:22:01.000595+0000","last_clean":"2024-03-06T07:22:01.000595+0000","last_became_active":"2024-03-06T07:21:43.915661+0000","last_became_peered":"2024-03-06T07:21:43.915661+0000","last_unstale":"2024-03-06T07:22:01.000595+0000","last_undegraded":"2024-03-06T07:22:01.000595+0000","last_fullsized":"2024-03-06T07:22:01.000595+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:57.872039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027091+0000","last_change":"2024-03-06T07:22:00.016462+0000","last_active":"2024-03-06T07:22:01.027091+0000","last_peered":"2024-03-06T07:22:01.027091+0000","last_clean":"2024-03-06T07:22:01.027091+0000","last_became_active":"2024-03-06T07:22:00.016021+0000","last_became_peered":"2024-03-06T07:22:00.016021+0000","last_unstale":"2024-03-06T07:22:01.027091+0000","last_undegraded":"2024-03-06T07:22:01.027091+0000","last_fullsized":"2024-03-06T07:22:01.027091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:17.113207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643337+0000","last_change":"2024-03-06T07:21:43.925030+0000","last_active":"2024-03-06T07:22:01.643337+0000","last_peered":"2024-03-06T07:22:01.643337+0000","last_clean":"2024-03-06T07:22:01.643337+0000","last_became_active":"2024-03-06T07:21:43.924480+0000","last_became_peered":"2024-03-06T07:21:43.924480+0000","last_unstale":"2024-03-06T07:22:01.643337+0000","last_undegraded":"2024-03-06T07:22:01.643337+0000","last_fullsized":"2024-03-06T07:22:01.643337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:36.336480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024729+0000","last_change":"2024-03-06T07:22:00.013001+0000","last_active":"2024-03-06T07:22:01.024729+0000","last_peered":"2024-03-06T07:22:01.024729+0000","last_clean":"2024-03-06T07:22:01.024729+0000","last_became_active":"2024-03-06T07:22:00.012722+0000","last_became_peered":"2024-03-06T07:22:00.012722+0000","last_unstale":"2024-03-06T07:22:01.024729+0000","last_undegraded":"2024-03-06T07:22:01.024729+0000","last_fullsized":"2024-03-06T07:22:01.024729+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:11.422190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150189+0000","last_change":"2024-03-06T07:21:43.915656+0000","last_active":"2024-03-06T07:22:01.150189+0000","last_peered":"2024-03-06T07:22:01.150189+0000","last_clean":"2024-03-06T07:22:01.150189+0000","last_became_active":"2024-03-06T07:21:43.915399+0000","last_became_peered":"2024-03-06T07:21:43.915399+0000","last_unstale":"2024-03-06T07:22:01.150189+0000","last_undegraded":"2024-03-06T07:22:01.150189+0000","last_fullsized":"2024-03-06T07:22:01.150189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149997+0000","last_change":"2024-03-06T07:22:00.020432+0000","last_active":"2024-03-06T07:22:01.149997+0000","last_peered":"2024-03-06T07:22:01.149997+0000","last_clean":"2024-03-06T07:22:01.149997+0000","last_became_active":"2024-03-06T07:22:00.020162+0000","last_became_peered":"2024-03-06T07:22:00.020162+0000","last_unstale":"2024-03-06T07:22:01.149997+0000","last_undegraded":"2024-03-06T07:22:01.149997+0000","last_fullsized":"2024-03-06T07:22:01.149997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:14.068723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000448+0000","last_change":"2024-03-06T07:21:43.909296+0000","last_active":"2024-03-06T07:22:01.000448+0000","last_peered":"2024-03-06T07:22:01.000448+0000","last_clean":"2024-03-06T07:22:01.000448+0000","last_became_active":"2024-03-06T07:21:43.909011+0000","last_became_peered":"2024-03-06T07:21:43.909011+0000","last_unstale":"2024-03-06T07:22:01.000448+0000","last_undegraded":"2024-03-06T07:22:01.000448+0000","last_fullsized":"2024-03-06T07:22:01.000448+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003579+0000","last_change":"2024-03-06T07:22:00.009690+0000","last_active":"2024-03-06T07:22:01.003579+0000","last_peered":"2024-03-06T07:22:01.003579+0000","last_clean":"2024-03-06T07:22:01.003579+0000","last_became_active":"2024-03-06T07:22:00.009425+0000","last_became_peered":"2024-03-06T07:22:00.009425+0000","last_unstale":"2024-03-06T07:22:01.003579+0000","last_undegraded":"2024-03-06T07:22:01.003579+0000","last_fullsized":"2024-03-06T07:22:01.003579+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:23.455544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642755+0000","last_change":"2024-03-06T07:21:43.930357+0000","last_active":"2024-03-06T07:22:01.642755+0000","last_peered":"2024-03-06T07:22:01.642755+0000","last_clean":"2024-03-06T07:22:01.642755+0000","last_became_active":"2024-03-06T07:21:43.929832+0000","last_became_peered":"2024-03-06T07:21:43.929832+0000","last_unstale":"2024-03-06T07:22:01.642755+0000","last_undegraded":"2024-03-06T07:22:01.642755+0000","last_fullsized":"2024-03-06T07:22:01.642755+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642575+0000","last_change":"2024-03-06T07:22:00.018784+0000","last_active":"2024-03-06T07:22:01.642575+0000","last_peered":"2024-03-06T07:22:01.642575+0000","last_clean":"2024-03-06T07:22:01.642575+0000","last_became_active":"2024-03-06T07:22:00.018452+0000","last_became_peered":"2024-03-06T07:22:00.018452+0000","last_unstale":"2024-03-06T07:22:01.642575+0000","last_undegraded":"2024-03-06T07:22:01.642575+0000","last_fullsized":"2024-03-06T07:22:01.642575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:35:49.437077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150596+0000","last_change":"2024-03-06T07:21:43.919806+0000","last_active":"2024-03-06T07:22:01.150596+0000","last_peered":"2024-03-06T07:22:01.150596+0000","last_clean":"2024-03-06T07:22:01.150596+0000","last_became_active":"2024-03-06T07:21:43.919524+0000","last_became_peered":"2024-03-06T07:21:43.919524+0000","last_unstale":"2024-03-06T07:22:01.150596+0000","last_undegraded":"2024-03-06T07:22:01.150596+0000","last_fullsized":"2024-03-06T07:22:01.150596+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001416+0000","last_change":"2024-03-06T07:22:00.026720+0000","last_active":"2024-03-06T07:22:01.001416+0000","last_peered":"2024-03-06T07:22:01.001416+0000","last_clean":"2024-03-06T07:22:01.001416+0000","last_became_active":"2024-03-06T07:22:00.026524+0000","last_became_peered":"2024-03-06T07:22:00.026524+0000","last_unstale":"2024-03-06T07:22:01.001416+0000","last_undegraded":"2024-03-06T07:22:01.001416+0000","last_fullsized":"2024-03-06T07:22:01.001416+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:02.688029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642951+0000","last_change":"2024-03-06T07:21:43.933746+0000","last_active":"2024-03-06T07:22:01.642951+0000","last_peered":"2024-03-06T07:22:01.642951+0000","last_clean":"2024-03-06T07:22:01.642951+0000","last_became_active":"2024-03-06T07:21:43.932918+0000","last_became_peered":"2024-03-06T07:21:43.932918+0000","last_unstale":"2024-03-06T07:22:01.642951+0000","last_undegraded":"2024-03-06T07:22:01.642951+0000","last_fullsized":"2024-03-06T07:22:01.642951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:06.526681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002885+0000","last_change":"2024-03-06T07:22:00.011775+0000","last_active":"2024-03-06T07:22:01.002885+0000","last_peered":"2024-03-06T07:22:01.002885+0000","last_clean":"2024-03-06T07:22:01.002885+0000","last_became_active":"2024-03-06T07:22:00.011502+0000","last_became_peered":"2024-03-06T07:22:00.011502+0000","last_unstale":"2024-03-06T07:22:01.002885+0000","last_undegraded":"2024-03-06T07:22:01.002885+0000","last_fullsized":"2024-03-06T07:22:01.002885+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:01.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003147+0000","last_change":"2024-03-06T07:21:43.916874+0000","last_active":"2024-03-06T07:22:01.003147+0000","last_peered":"2024-03-06T07:22:01.003147+0000","last_clean":"2024-03-06T07:22:01.003147+0000","last_became_active":"2024-03-06T07:21:43.916593+0000","last_became_peered":"2024-03-06T07:21:43.916593+0000","last_unstale":"2024-03-06T07:22:01.003147+0000","last_undegraded":"2024-03-06T07:22:01.003147+0000","last_fullsized":"2024-03-06T07:22:01.003147+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:30.373457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642350+0000","last_change":"2024-03-06T07:22:00.020263+0000","last_active":"2024-03-06T07:22:01.642350+0000","last_peered":"2024-03-06T07:22:01.642350+0000","last_clean":"2024-03-06T07:22:01.642350+0000","last_became_active":"2024-03-06T07:22:00.019933+0000","last_became_peered":"2024-03-06T07:22:00.019933+0000","last_unstale":"2024-03-06T07:22:01.642350+0000","last_undegraded":"2024-03-06T07:22:01.642350+0000","last_fullsized":"2024-03-06T07:22:01.642350+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:06.622414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002337+0000","last_change":"2024-03-06T07:21:43.914663+0000","last_active":"2024-03-06T07:22:01.002337+0000","last_peered":"2024-03-06T07:22:01.002337+0000","last_clean":"2024-03-06T07:22:01.002337+0000","last_became_active":"2024-03-06T07:21:43.914383+0000","last_became_peered":"2024-03-06T07:21:43.914383+0000","last_unstale":"2024-03-06T07:22:01.002337+0000","last_undegraded":"2024-03-06T07:22:01.002337+0000","last_fullsized":"2024-03-06T07:22:01.002337+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149643+0000","last_change":"2024-03-06T07:22:00.021035+0000","last_active":"2024-03-06T07:22:01.149643+0000","last_peered":"2024-03-06T07:22:01.149643+0000","last_clean":"2024-03-06T07:22:01.149643+0000","last_became_active":"2024-03-06T07:22:00.015228+0000","last_became_peered":"2024-03-06T07:22:00.015228+0000","last_unstale":"2024-03-06T07:22:01.149643+0000","last_undegraded":"2024-03-06T07:22:01.149643+0000","last_fullsized":"2024-03-06T07:22:01.149643+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:42.684403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642677+0000","last_change":"2024-03-06T07:21:43.929377+0000","last_active":"2024-03-06T07:22:01.642677+0000","last_peered":"2024-03-06T07:22:01.642677+0000","last_clean":"2024-03-06T07:22:01.642677+0000","last_became_active":"2024-03-06T07:21:43.929093+0000","last_became_peered":"2024-03-06T07:21:43.929093+0000","last_unstale":"2024-03-06T07:22:01.642677+0000","last_undegraded":"2024-03-06T07:22:01.642677+0000","last_fullsized":"2024-03-06T07:22:01.642677+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024612+0000","last_change":"2024-03-06T07:22:00.015714+0000","last_active":"2024-03-06T07:22:01.024612+0000","last_peered":"2024-03-06T07:22:01.024612+0000","last_clean":"2024-03-06T07:22:01.024612+0000","last_became_active":"2024-03-06T07:22:00.015455+0000","last_became_peered":"2024-03-06T07:22:00.015455+0000","last_unstale":"2024-03-06T07:22:01.024612+0000","last_undegraded":"2024-03-06T07:22:01.024612+0000","last_fullsized":"2024-03-06T07:22:01.024612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:47.362845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027278+0000","last_change":"2024-03-06T07:21:43.916941+0000","last_active":"2024-03-06T07:22:01.027278+0000","last_peered":"2024-03-06T07:22:01.027278+0000","last_clean":"2024-03-06T07:22:01.027278+0000","last_became_active":"2024-03-06T07:21:43.916667+0000","last_became_peered":"2024-03-06T07:21:43.916667+0000","last_unstale":"2024-03-06T07:22:01.027278+0000","last_undegraded":"2024-03-06T07:22:01.027278+0000","last_fullsized":"2024-03-06T07:22:01.027278+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:15.683227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642347+0000","last_change":"2024-03-06T07:22:00.019750+0000","last_active":"2024-03-06T07:22:01.642347+0000","last_peered":"2024-03-06T07:22:01.642347+0000","last_clean":"2024-03-06T07:22:01.642347+0000","last_became_active":"2024-03-06T07:22:00.019454+0000","last_became_peered":"2024-03-06T07:22:00.019454+0000","last_unstale":"2024-03-06T07:22:01.642347+0000","last_undegraded":"2024-03-06T07:22:01.642347+0000","last_fullsized":"2024-03-06T07:22:01.642347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:56.249609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642688+0000","last_change":"2024-03-06T07:21:43.924259+0000","last_active":"2024-03-06T07:22:01.642688+0000","last_peered":"2024-03-06T07:22:01.642688+0000","last_clean":"2024-03-06T07:22:01.642688+0000","last_became_active":"2024-03-06T07:21:43.923885+0000","last_became_peered":"2024-03-06T07:21:43.923885+0000","last_unstale":"2024-03-06T07:22:01.642688+0000","last_undegraded":"2024-03-06T07:22:01.642688+0000","last_fullsized":"2024-03-06T07:22:01.642688+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001866+0000","last_change":"2024-03-06T07:22:00.010001+0000","last_active":"2024-03-06T07:22:01.001866+0000","last_peered":"2024-03-06T07:22:01.001866+0000","last_clean":"2024-03-06T07:22:01.001866+0000","last_became_active":"2024-03-06T07:22:00.009679+0000","last_became_peered":"2024-03-06T07:22:00.009679+0000","last_unstale":"2024-03-06T07:22:01.001866+0000","last_undegraded":"2024-03-06T07:22:01.001866+0000","last_fullsized":"2024-03-06T07:22:01.001866+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:35:03.394288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642859+0000","last_change":"2024-03-06T07:21:43.931109+0000","last_active":"2024-03-06T07:22:01.642859+0000","last_peered":"2024-03-06T07:22:01.642859+0000","last_clean":"2024-03-06T07:22:01.642859+0000","last_became_active":"2024-03-06T07:21:43.930848+0000","last_became_peered":"2024-03-06T07:21:43.930848+0000","last_unstale":"2024-03-06T07:22:01.642859+0000","last_undegraded":"2024-03-06T07:22:01.642859+0000","last_fullsized":"2024-03-06T07:22:01.642859+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000984+0000","last_change":"2024-03-06T07:22:00.019071+0000","last_active":"2024-03-06T07:22:01.000984+0000","last_peered":"2024-03-06T07:22:01.000984+0000","last_clean":"2024-03-06T07:22:01.000984+0000","last_became_active":"2024-03-06T07:22:00.018790+0000","last_became_peered":"2024-03-06T07:22:00.018790+0000","last_unstale":"2024-03-06T07:22:01.000984+0000","last_undegraded":"2024-03-06T07:22:01.000984+0000","last_fullsized":"2024-03-06T07:22:01.000984+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:01.323282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643757+0000","last_change":"2024-03-06T07:21:43.913493+0000","last_active":"2024-03-06T07:22:01.643757+0000","last_peered":"2024-03-06T07:22:01.643757+0000","last_clean":"2024-03-06T07:22:01.643757+0000","last_became_active":"2024-03-06T07:21:43.912992+0000","last_became_peered":"2024-03-06T07:21:43.912992+0000","last_unstale":"2024-03-06T07:22:01.643757+0000","last_undegraded":"2024-03-06T07:22:01.643757+0000","last_fullsized":"2024-03-06T07:22:01.643757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:30.239239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.025043+0000","last_change":"2024-03-06T07:22:00.013830+0000","last_active":"2024-03-06T07:22:01.025043+0000","last_peered":"2024-03-06T07:22:01.025043+0000","last_clean":"2024-03-06T07:22:01.025043+0000","last_became_active":"2024-03-06T07:22:00.013588+0000","last_became_peered":"2024-03-06T07:22:00.013588+0000","last_unstale":"2024-03-06T07:22:01.025043+0000","last_undegraded":"2024-03-06T07:22:01.025043+0000","last_fullsized":"2024-03-06T07:22:01.025043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:31.898544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001410+0000","last_change":"2024-03-06T07:21:43.934992+0000","last_active":"2024-03-06T07:22:01.001410+0000","last_peered":"2024-03-06T07:22:01.001410+0000","last_clean":"2024-03-06T07:22:01.001410+0000","last_became_active":"2024-03-06T07:21:43.934664+0000","last_became_peered":"2024-03-06T07:21:43.934664+0000","last_unstale":"2024-03-06T07:22:01.001410+0000","last_undegraded":"2024-03-06T07:22:01.001410+0000","last_fullsized":"2024-03-06T07:22:01.001410+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024893+0000","last_change":"2024-03-06T07:22:00.019639+0000","last_active":"2024-03-06T07:22:01.024893+0000","last_peered":"2024-03-06T07:22:01.024893+0000","last_clean":"2024-03-06T07:22:01.024893+0000","last_became_active":"2024-03-06T07:22:00.018760+0000","last_became_peered":"2024-03-06T07:22:00.018760+0000","last_unstale":"2024-03-06T07:22:01.024893+0000","last_undegraded":"2024-03-06T07:22:01.024893+0000","last_fullsized":"2024-03-06T07:22:01.024893+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:00.475072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003272+0000","last_change":"2024-03-06T07:21:43.910958+0000","last_active":"2024-03-06T07:22:01.003272+0000","last_peered":"2024-03-06T07:22:01.003272+0000","last_clean":"2024-03-06T07:22:01.003272+0000","last_became_active":"2024-03-06T07:21:43.910653+0000","last_became_peered":"2024-03-06T07:21:43.910653+0000","last_unstale":"2024-03-06T07:22:01.003272+0000","last_undegraded":"2024-03-06T07:22:01.003272+0000","last_fullsized":"2024-03-06T07:22:01.003272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001730+0000","last_change":"2024-03-06T07:22:00.010036+0000","last_active":"2024-03-06T07:22:01.001730+0000","last_peered":"2024-03-06T07:22:01.001730+0000","last_clean":"2024-03-06T07:22:01.001730+0000","last_became_active":"2024-03-06T07:22:00.009261+0000","last_became_peered":"2024-03-06T07:22:00.009261+0000","last_unstale":"2024-03-06T07:22:01.001730+0000","last_undegraded":"2024-03-06T07:22:01.001730+0000","last_fullsized":"2024-03-06T07:22:01.001730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:48.043526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642451+0000","last_change":"2024-03-06T07:21:43.923484+0000","last_active":"2024-03-06T07:22:01.642451+0000","last_peered":"2024-03-06T07:22:01.642451+0000","last_clean":"2024-03-06T07:22:01.642451+0000","last_became_active":"2024-03-06T07:21:43.921095+0000","last_became_peered":"2024-03-06T07:21:43.921095+0000","last_unstale":"2024-03-06T07:22:01.642451+0000","last_undegraded":"2024-03-06T07:22:01.642451+0000","last_fullsized":"2024-03-06T07:22:01.642451+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642421+0000","last_change":"2024-03-06T07:22:00.010970+0000","last_active":"2024-03-06T07:22:01.642421+0000","last_peered":"2024-03-06T07:22:01.642421+0000","last_clean":"2024-03-06T07:22:01.642421+0000","last_became_active":"2024-03-06T07:22:00.009925+0000","last_became_peered":"2024-03-06T07:22:00.009925+0000","last_unstale":"2024-03-06T07:22:01.642421+0000","last_undegraded":"2024-03-06T07:22:01.642421+0000","last_fullsized":"2024-03-06T07:22:01.642421+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:00.475072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150054+0000","last_change":"2024-03-06T07:21:43.917834+0000","last_active":"2024-03-06T07:22:01.150054+0000","last_peered":"2024-03-06T07:22:01.150054+0000","last_clean":"2024-03-06T07:22:01.150054+0000","last_became_active":"2024-03-06T07:21:43.917576+0000","last_became_peered":"2024-03-06T07:21:43.917576+0000","last_unstale":"2024-03-06T07:22:01.150054+0000","last_undegraded":"2024-03-06T07:22:01.150054+0000","last_fullsized":"2024-03-06T07:22:01.150054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149828+0000","last_change":"2024-03-06T07:22:00.030272+0000","last_active":"2024-03-06T07:22:01.149828+0000","last_peered":"2024-03-06T07:22:01.149828+0000","last_clean":"2024-03-06T07:22:01.149828+0000","last_became_active":"2024-03-06T07:22:00.029982+0000","last_became_peered":"2024-03-06T07:22:00.029982+0000","last_unstale":"2024-03-06T07:22:01.149828+0000","last_undegraded":"2024-03-06T07:22:01.149828+0000","last_fullsized":"2024-03-06T07:22:01.149828+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:43.126081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643032+0000","last_change":"2024-03-06T07:21:43.921467+0000","last_active":"2024-03-06T07:22:01.643032+0000","last_peered":"2024-03-06T07:22:01.643032+0000","last_clean":"2024-03-06T07:22:01.643032+0000","last_became_active":"2024-03-06T07:21:43.920878+0000","last_became_peered":"2024-03-06T07:21:43.920878+0000","last_unstale":"2024-03-06T07:22:01.643032+0000","last_undegraded":"2024-03-06T07:22:01.643032+0000","last_fullsized":"2024-03-06T07:22:01.643032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024869+0000","last_change":"2024-03-06T07:22:00.021194+0000","last_active":"2024-03-06T07:22:01.024869+0000","last_peered":"2024-03-06T07:22:01.024869+0000","last_clean":"2024-03-06T07:22:01.024869+0000","last_became_active":"2024-03-06T07:22:00.020935+0000","last_became_peered":"2024-03-06T07:22:00.020935+0000","last_unstale":"2024-03-06T07:22:01.024869+0000","last_undegraded":"2024-03-06T07:22:01.024869+0000","last_fullsized":"2024-03-06T07:22:01.024869+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:40.809996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024321+0000","last_change":"2024-03-06T07:21:43.911317+0000","last_active":"2024-03-06T07:22:01.024321+0000","last_peered":"2024-03-06T07:22:01.024321+0000","last_clean":"2024-03-06T07:22:01.024321+0000","last_became_active":"2024-03-06T07:21:43.910980+0000","last_became_peered":"2024-03-06T07:21:43.910980+0000","last_unstale":"2024-03-06T07:22:01.024321+0000","last_undegraded":"2024-03-06T07:22:01.024321+0000","last_fullsized":"2024-03-06T07:22:01.024321+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:18.279184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002376+0000","last_change":"2024-03-06T07:22:00.010859+0000","last_active":"2024-03-06T07:22:01.002376+0000","last_peered":"2024-03-06T07:22:01.002376+0000","last_clean":"2024-03-06T07:22:01.002376+0000","last_became_active":"2024-03-06T07:22:00.010163+0000","last_became_peered":"2024-03-06T07:22:00.010163+0000","last_unstale":"2024-03-06T07:22:01.002376+0000","last_undegraded":"2024-03-06T07:22:01.002376+0000","last_fullsized":"2024-03-06T07:22:01.002376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:35:03.394288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149656+0000","last_change":"2024-03-06T07:21:59.013995+0000","last_active":"2024-03-06T07:22:01.149656+0000","last_peered":"2024-03-06T07:22:01.149656+0000","last_clean":"2024-03-06T07:22:01.149656+0000","last_became_active":"2024-03-06T07:21:59.013711+0000","last_became_peered":"2024-03-06T07:21:59.013711+0000","last_unstale":"2024-03-06T07:22:01.149656+0000","last_undegraded":"2024-03-06T07:22:01.149656+0000","last_fullsized":"2024-03-06T07:22:01.149656+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:21.623615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149671+0000","last_change":"2024-03-06T07:21:43.914331+0000","last_active":"2024-03-06T07:22:01.149671+0000","last_peered":"2024-03-06T07:22:01.149671+0000","last_clean":"2024-03-06T07:22:01.149671+0000","last_became_active":"2024-03-06T07:21:43.913681+0000","last_became_peered":"2024-03-06T07:21:43.913681+0000","last_unstale":"2024-03-06T07:22:01.149671+0000","last_undegraded":"2024-03-06T07:22:01.149671+0000","last_fullsized":"2024-03-06T07:22:01.149671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:16.416790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":30,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003830+0000","last_change":"2024-03-06T07:21:42.908247+0000","last_active":"2024-03-06T07:22:01.003830+0000","last_peered":"2024-03-06T07:22:01.003830+0000","last_clean":"2024-03-06T07:22:01.003830+0000","last_became_active":"2024-03-06T07:21:42.907863+0000","last_became_peered":"2024-03-06T07:21:42.907863+0000","last_unstale":"2024-03-06T07:22:01.003830+0000","last_undegraded":"2024-03-06T07:22:01.003830+0000","last_fullsized":"2024-03-06T07:22:01.003830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:44.909708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000867+0000","last_change":"2024-03-06T07:22:00.025078+0000","last_active":"2024-03-06T07:22:01.000867+0000","last_peered":"2024-03-06T07:22:01.000867+0000","last_clean":"2024-03-06T07:22:01.000867+0000","last_became_active":"2024-03-06T07:22:00.024793+0000","last_became_peered":"2024-03-06T07:22:00.024793+0000","last_unstale":"2024-03-06T07:22:01.000867+0000","last_undegraded":"2024-03-06T07:22:01.000867+0000","last_fullsized":"2024-03-06T07:22:01.000867+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:59.335901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002810+0000","last_change":"2024-03-06T07:21:59.017569+0000","last_active":"2024-03-06T07:22:01.002810+0000","last_peered":"2024-03-06T07:22:01.002810+0000","last_clean":"2024-03-06T07:22:01.002810+0000","last_became_active":"2024-03-06T07:21:59.017285+0000","last_became_peered":"2024-03-06T07:21:59.017285+0000","last_unstale":"2024-03-06T07:22:01.002810+0000","last_undegraded":"2024-03-06T07:22:01.002810+0000","last_fullsized":"2024-03-06T07:22:01.002810+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:46.110694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024500+0000","last_change":"2024-03-06T07:21:43.918214+0000","last_active":"2024-03-06T07:22:01.024500+0000","last_peered":"2024-03-06T07:22:01.024500+0000","last_clean":"2024-03-06T07:22:01.024500+0000","last_became_active":"2024-03-06T07:21:43.917454+0000","last_became_peered":"2024-03-06T07:21:43.917454+0000","last_unstale":"2024-03-06T07:22:01.024500+0000","last_undegraded":"2024-03-06T07:22:01.024500+0000","last_fullsized":"2024-03-06T07:22:01.024500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:35.795585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001412+0000","last_change":"2024-03-06T07:21:42.915335+0000","last_active":"2024-03-06T07:22:01.001412+0000","last_peered":"2024-03-06T07:22:01.001412+0000","last_clean":"2024-03-06T07:22:01.001412+0000","last_became_active":"2024-03-06T07:21:42.915011+0000","last_became_peered":"2024-03-06T07:21:42.915011+0000","last_unstale":"2024-03-06T07:22:01.001412+0000","last_undegraded":"2024-03-06T07:22:01.001412+0000","last_fullsized":"2024-03-06T07:22:01.001412+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001702+0000","last_change":"2024-03-06T07:22:00.017468+0000","last_active":"2024-03-06T07:22:01.001702+0000","last_peered":"2024-03-06T07:22:01.001702+0000","last_clean":"2024-03-06T07:22:01.001702+0000","last_became_active":"2024-03-06T07:22:00.017203+0000","last_became_peered":"2024-03-06T07:22:00.017203+0000","last_unstale":"2024-03-06T07:22:01.001702+0000","last_undegraded":"2024-03-06T07:22:01.001702+0000","last_fullsized":"2024-03-06T07:22:01.001702+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:13.109019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149834+0000","last_change":"2024-03-06T07:21:59.011323+0000","last_active":"2024-03-06T07:22:01.149834+0000","last_peered":"2024-03-06T07:22:01.149834+0000","last_clean":"2024-03-06T07:22:01.149834+0000","last_became_active":"2024-03-06T07:21:59.011040+0000","last_became_peered":"2024-03-06T07:21:59.011040+0000","last_unstale":"2024-03-06T07:22:01.149834+0000","last_undegraded":"2024-03-06T07:22:01.149834+0000","last_fullsized":"2024-03-06T07:22:01.149834+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:44.332849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002560+0000","last_change":"2024-03-06T07:21:43.914522+0000","last_active":"2024-03-06T07:22:01.002560+0000","last_peered":"2024-03-06T07:22:01.002560+0000","last_clean":"2024-03-06T07:22:01.002560+0000","last_became_active":"2024-03-06T07:21:43.913602+0000","last_became_peered":"2024-03-06T07:21:43.913602+0000","last_unstale":"2024-03-06T07:22:01.002560+0000","last_undegraded":"2024-03-06T07:22:01.002560+0000","last_fullsized":"2024-03-06T07:22:01.002560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:02.123078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001505+0000","last_change":"2024-03-06T07:21:42.914470+0000","last_active":"2024-03-06T07:22:01.001505+0000","last_peered":"2024-03-06T07:22:01.001505+0000","last_clean":"2024-03-06T07:22:01.001505+0000","last_became_active":"2024-03-06T07:21:42.913930+0000","last_became_peered":"2024-03-06T07:21:42.913930+0000","last_unstale":"2024-03-06T07:22:01.001505+0000","last_undegraded":"2024-03-06T07:22:01.001505+0000","last_fullsized":"2024-03-06T07:22:01.001505+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:24.367917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024379+0000","last_change":"2024-03-06T07:22:00.012632+0000","last_active":"2024-03-06T07:22:01.024379+0000","last_peered":"2024-03-06T07:22:01.024379+0000","last_clean":"2024-03-06T07:22:01.024379+0000","last_became_active":"2024-03-06T07:22:00.012216+0000","last_became_peered":"2024-03-06T07:22:00.012216+0000","last_unstale":"2024-03-06T07:22:01.024379+0000","last_undegraded":"2024-03-06T07:22:01.024379+0000","last_fullsized":"2024-03-06T07:22:01.024379+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:36.032336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024375+0000","last_change":"2024-03-06T07:21:59.011827+0000","last_active":"2024-03-06T07:22:01.024375+0000","last_peered":"2024-03-06T07:22:01.024375+0000","last_clean":"2024-03-06T07:22:01.024375+0000","last_became_active":"2024-03-06T07:21:59.011087+0000","last_became_peered":"2024-03-06T07:21:59.011087+0000","last_unstale":"2024-03-06T07:22:01.024375+0000","last_undegraded":"2024-03-06T07:22:01.024375+0000","last_fullsized":"2024-03-06T07:22:01.024375+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:27.422606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643107+0000","last_change":"2024-03-06T07:21:43.912159+0000","last_active":"2024-03-06T07:22:01.643107+0000","last_peered":"2024-03-06T07:22:01.643107+0000","last_clean":"2024-03-06T07:22:01.643107+0000","last_became_active":"2024-03-06T07:21:43.911851+0000","last_became_peered":"2024-03-06T07:21:43.911851+0000","last_unstale":"2024-03-06T07:22:01.643107+0000","last_undegraded":"2024-03-06T07:22:01.643107+0000","last_fullsized":"2024-03-06T07:22:01.643107+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024735+0000","last_change":"2024-03-06T07:21:42.912811+0000","last_active":"2024-03-06T07:22:01.024735+0000","last_peered":"2024-03-06T07:22:01.024735+0000","last_clean":"2024-03-06T07:22:01.024735+0000","last_became_active":"2024-03-06T07:21:42.912544+0000","last_became_peered":"2024-03-06T07:21:42.912544+0000","last_unstale":"2024-03-06T07:22:01.024735+0000","last_undegraded":"2024-03-06T07:22:01.024735+0000","last_fullsized":"2024-03-06T07:22:01.024735+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:50.577390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003375+0000","last_change":"2024-03-06T07:22:00.010765+0000","last_active":"2024-03-06T07:22:01.003375+0000","last_peered":"2024-03-06T07:22:01.003375+0000","last_clean":"2024-03-06T07:22:01.003375+0000","last_became_active":"2024-03-06T07:22:00.010488+0000","last_became_peered":"2024-03-06T07:22:00.010488+0000","last_unstale":"2024-03-06T07:22:01.003375+0000","last_undegraded":"2024-03-06T07:22:01.003375+0000","last_fullsized":"2024-03-06T07:22:01.003375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:45.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150655+0000","last_change":"2024-03-06T07:21:59.015543+0000","last_active":"2024-03-06T07:22:01.150655+0000","last_peered":"2024-03-06T07:22:01.150655+0000","last_clean":"2024-03-06T07:22:01.150655+0000","last_became_active":"2024-03-06T07:21:59.015052+0000","last_became_peered":"2024-03-06T07:21:59.015052+0000","last_unstale":"2024-03-06T07:22:01.150655+0000","last_undegraded":"2024-03-06T07:22:01.150655+0000","last_fullsized":"2024-03-06T07:22:01.150655+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:56.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003273+0000","last_change":"2024-03-06T07:21:43.922679+0000","last_active":"2024-03-06T07:22:01.003273+0000","last_peered":"2024-03-06T07:22:01.003273+0000","last_clean":"2024-03-06T07:22:01.003273+0000","last_became_active":"2024-03-06T07:21:43.922322+0000","last_became_peered":"2024-03-06T07:21:43.922322+0000","last_unstale":"2024-03-06T07:22:01.003273+0000","last_undegraded":"2024-03-06T07:22:01.003273+0000","last_fullsized":"2024-03-06T07:22:01.003273+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:47.860793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642677+0000","last_change":"2024-03-06T07:21:42.912329+0000","last_active":"2024-03-06T07:22:01.642677+0000","last_peered":"2024-03-06T07:22:01.642677+0000","last_clean":"2024-03-06T07:22:01.642677+0000","last_became_active":"2024-03-06T07:21:42.911690+0000","last_became_peered":"2024-03-06T07:21:42.911690+0000","last_unstale":"2024-03-06T07:22:01.642677+0000","last_undegraded":"2024-03-06T07:22:01.642677+0000","last_fullsized":"2024-03-06T07:22:01.642677+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002846+0000","last_change":"2024-03-06T07:22:00.009765+0000","last_active":"2024-03-06T07:22:01.002846+0000","last_peered":"2024-03-06T07:22:01.002846+0000","last_clean":"2024-03-06T07:22:01.002846+0000","last_became_active":"2024-03-06T07:22:00.009440+0000","last_became_peered":"2024-03-06T07:22:00.009440+0000","last_unstale":"2024-03-06T07:22:01.002846+0000","last_undegraded":"2024-03-06T07:22:01.002846+0000","last_fullsized":"2024-03-06T07:22:01.002846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:22.628016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001125+0000","last_change":"2024-03-06T07:21:59.021111+0000","last_active":"2024-03-06T07:22:01.001125+0000","last_peered":"2024-03-06T07:22:01.001125+0000","last_clean":"2024-03-06T07:22:01.001125+0000","last_became_active":"2024-03-06T07:21:59.020849+0000","last_became_peered":"2024-03-06T07:21:59.020849+0000","last_unstale":"2024-03-06T07:22:01.001125+0000","last_undegraded":"2024-03-06T07:22:01.001125+0000","last_fullsized":"2024-03-06T07:22:01.001125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:51.433414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150387+0000","last_change":"2024-03-06T07:21:43.917686+0000","last_active":"2024-03-06T07:22:01.150387+0000","last_peered":"2024-03-06T07:22:01.150387+0000","last_clean":"2024-03-06T07:22:01.150387+0000","last_became_active":"2024-03-06T07:21:43.917369+0000","last_became_peered":"2024-03-06T07:21:43.917369+0000","last_unstale":"2024-03-06T07:22:01.150387+0000","last_undegraded":"2024-03-06T07:22:01.150387+0000","last_fullsized":"2024-03-06T07:22:01.150387+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:31.013761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001800+0000","last_change":"2024-03-06T07:21:42.910602+0000","last_active":"2024-03-06T07:22:01.001800+0000","last_peered":"2024-03-06T07:22:01.001800+0000","last_clean":"2024-03-06T07:22:01.001800+0000","last_became_active":"2024-03-06T07:21:42.910279+0000","last_became_peered":"2024-03-06T07:21:42.910279+0000","last_unstale":"2024-03-06T07:22:01.001800+0000","last_undegraded":"2024-03-06T07:22:01.001800+0000","last_fullsized":"2024-03-06T07:22:01.001800+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:02.037327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001125+0000","last_change":"2024-03-06T07:22:00.012972+0000","last_active":"2024-03-06T07:22:01.001125+0000","last_peered":"2024-03-06T07:22:01.001125+0000","last_clean":"2024-03-06T07:22:01.001125+0000","last_became_active":"2024-03-06T07:22:00.012692+0000","last_became_peered":"2024-03-06T07:22:00.012692+0000","last_unstale":"2024-03-06T07:22:01.001125+0000","last_undegraded":"2024-03-06T07:22:01.001125+0000","last_fullsized":"2024-03-06T07:22:01.001125+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:29.160570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150655+0000","last_change":"2024-03-06T07:21:59.015229+0000","last_active":"2024-03-06T07:22:01.150655+0000","last_peered":"2024-03-06T07:22:01.150655+0000","last_clean":"2024-03-06T07:22:01.150655+0000","last_became_active":"2024-03-06T07:21:59.014945+0000","last_became_peered":"2024-03-06T07:21:59.014945+0000","last_unstale":"2024-03-06T07:22:01.150655+0000","last_undegraded":"2024-03-06T07:22:01.150655+0000","last_fullsized":"2024-03-06T07:22:01.150655+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:29.685760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643856+0000","last_change":"2024-03-06T07:21:43.913254+0000","last_active":"2024-03-06T07:22:01.643856+0000","last_peered":"2024-03-06T07:22:01.643856+0000","last_clean":"2024-03-06T07:22:01.643856+0000","last_became_active":"2024-03-06T07:21:43.912620+0000","last_became_peered":"2024-03-06T07:21:43.912620+0000","last_unstale":"2024-03-06T07:22:01.643856+0000","last_undegraded":"2024-03-06T07:22:01.643856+0000","last_fullsized":"2024-03-06T07:22:01.643856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:13.059235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002160+0000","last_change":"2024-03-06T07:21:42.909672+0000","last_active":"2024-03-06T07:22:01.002160+0000","last_peered":"2024-03-06T07:22:01.002160+0000","last_clean":"2024-03-06T07:22:01.002160+0000","last_became_active":"2024-03-06T07:21:42.909402+0000","last_became_peered":"2024-03-06T07:21:42.909402+0000","last_unstale":"2024-03-06T07:22:01.002160+0000","last_undegraded":"2024-03-06T07:22:01.002160+0000","last_fullsized":"2024-03-06T07:22:01.002160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:06.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643716+0000","last_change":"2024-03-06T07:22:00.022225+0000","last_active":"2024-03-06T07:22:01.643716+0000","last_peered":"2024-03-06T07:22:01.643716+0000","last_clean":"2024-03-06T07:22:01.643716+0000","last_became_active":"2024-03-06T07:22:00.021970+0000","last_became_peered":"2024-03-06T07:22:00.021970+0000","last_unstale":"2024-03-06T07:22:01.643716+0000","last_undegraded":"2024-03-06T07:22:01.643716+0000","last_fullsized":"2024-03-06T07:22:01.643716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:08.955032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002718+0000","last_change":"2024-03-06T07:21:59.020696+0000","last_active":"2024-03-06T07:22:01.002718+0000","last_peered":"2024-03-06T07:22:01.002718+0000","last_clean":"2024-03-06T07:22:01.002718+0000","last_became_active":"2024-03-06T07:21:59.020474+0000","last_became_peered":"2024-03-06T07:21:59.020474+0000","last_unstale":"2024-03-06T07:22:01.002718+0000","last_undegraded":"2024-03-06T07:22:01.002718+0000","last_fullsized":"2024-03-06T07:22:01.002718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:15.577151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150609+0000","last_change":"2024-03-06T07:21:43.934375+0000","last_active":"2024-03-06T07:22:01.150609+0000","last_peered":"2024-03-06T07:22:01.150609+0000","last_clean":"2024-03-06T07:22:01.150609+0000","last_became_active":"2024-03-06T07:21:43.934076+0000","last_became_peered":"2024-03-06T07:21:43.934076+0000","last_unstale":"2024-03-06T07:22:01.150609+0000","last_undegraded":"2024-03-06T07:22:01.150609+0000","last_fullsized":"2024-03-06T07:22:01.150609+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642730+0000","last_change":"2024-03-06T07:21:42.910541+0000","last_active":"2024-03-06T07:22:01.642730+0000","last_peered":"2024-03-06T07:22:01.642730+0000","last_clean":"2024-03-06T07:22:01.642730+0000","last_became_active":"2024-03-06T07:21:42.910271+0000","last_became_peered":"2024-03-06T07:21:42.910271+0000","last_unstale":"2024-03-06T07:22:01.642730+0000","last_undegraded":"2024-03-06T07:22:01.642730+0000","last_fullsized":"2024-03-06T07:22:01.642730+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002754+0000","last_change":"2024-03-06T07:22:00.011049+0000","last_active":"2024-03-06T07:22:01.002754+0000","last_peered":"2024-03-06T07:22:01.002754+0000","last_clean":"2024-03-06T07:22:01.002754+0000","last_became_active":"2024-03-06T07:22:00.010569+0000","last_became_peered":"2024-03-06T07:22:00.010569+0000","last_unstale":"2024-03-06T07:22:01.002754+0000","last_undegraded":"2024-03-06T07:22:01.002754+0000","last_fullsized":"2024-03-06T07:22:01.002754+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:31.441532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642290+0000","last_change":"2024-03-06T07:21:59.010326+0000","last_active":"2024-03-06T07:22:01.642290+0000","last_peered":"2024-03-06T07:22:01.642290+0000","last_clean":"2024-03-06T07:22:01.642290+0000","last_became_active":"2024-03-06T07:21:59.009373+0000","last_became_peered":"2024-03-06T07:21:59.009373+0000","last_unstale":"2024-03-06T07:22:01.642290+0000","last_undegraded":"2024-03-06T07:22:01.642290+0000","last_fullsized":"2024-03-06T07:22:01.642290+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:59.470671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002962+0000","last_change":"2024-03-06T07:21:43.905571+0000","last_active":"2024-03-06T07:22:01.002962+0000","last_peered":"2024-03-06T07:22:01.002962+0000","last_clean":"2024-03-06T07:22:01.002962+0000","last_became_active":"2024-03-06T07:21:43.904452+0000","last_became_peered":"2024-03-06T07:21:43.904452+0000","last_unstale":"2024-03-06T07:22:01.002962+0000","last_undegraded":"2024-03-06T07:22:01.002962+0000","last_fullsized":"2024-03-06T07:22:01.002962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:56.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150719+0000","last_change":"2024-03-06T07:21:42.913817+0000","last_active":"2024-03-06T07:22:01.150719+0000","last_peered":"2024-03-06T07:22:01.150719+0000","last_clean":"2024-03-06T07:22:01.150719+0000","last_became_active":"2024-03-06T07:21:42.913464+0000","last_became_peered":"2024-03-06T07:21:42.913464+0000","last_unstale":"2024-03-06T07:22:01.150719+0000","last_undegraded":"2024-03-06T07:22:01.150719+0000","last_fullsized":"2024-03-06T07:22:01.150719+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:43:55.780036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643376+0000","last_change":"2024-03-06T07:22:00.011586+0000","last_active":"2024-03-06T07:22:01.643376+0000","last_peered":"2024-03-06T07:22:01.643376+0000","last_clean":"2024-03-06T07:22:01.643376+0000","last_became_active":"2024-03-06T07:22:00.011320+0000","last_became_peered":"2024-03-06T07:22:00.011320+0000","last_unstale":"2024-03-06T07:22:01.643376+0000","last_undegraded":"2024-03-06T07:22:01.643376+0000","last_fullsized":"2024-03-06T07:22:01.643376+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:56.594846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001643+0000","last_change":"2024-03-06T07:21:59.011672+0000","last_active":"2024-03-06T07:22:01.001643+0000","last_peered":"2024-03-06T07:22:01.001643+0000","last_clean":"2024-03-06T07:22:01.001643+0000","last_became_active":"2024-03-06T07:21:59.011374+0000","last_became_peered":"2024-03-06T07:21:59.011374+0000","last_unstale":"2024-03-06T07:22:01.001643+0000","last_undegraded":"2024-03-06T07:22:01.001643+0000","last_fullsized":"2024-03-06T07:22:01.001643+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:06.679356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024522+0000","last_change":"2024-03-06T07:21:43.912031+0000","last_active":"2024-03-06T07:22:01.024522+0000","last_peered":"2024-03-06T07:22:01.024522+0000","last_clean":"2024-03-06T07:22:01.024522+0000","last_became_active":"2024-03-06T07:21:43.911759+0000","last_became_peered":"2024-03-06T07:21:43.911759+0000","last_unstale":"2024-03-06T07:22:01.024522+0000","last_undegraded":"2024-03-06T07:22:01.024522+0000","last_fullsized":"2024-03-06T07:22:01.024522+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:04.621071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003050+0000","last_change":"2024-03-06T07:21:42.910374+0000","last_active":"2024-03-06T07:22:01.003050+0000","last_peered":"2024-03-06T07:22:01.003050+0000","last_clean":"2024-03-06T07:22:01.003050+0000","last_became_active":"2024-03-06T07:21:42.910082+0000","last_became_peered":"2024-03-06T07:21:42.910082+0000","last_unstale":"2024-03-06T07:22:01.003050+0000","last_undegraded":"2024-03-06T07:22:01.003050+0000","last_fullsized":"2024-03-06T07:22:01.003050+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:55.547305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027061+0000","last_change":"2024-03-06T07:22:00.016621+0000","last_active":"2024-03-06T07:22:01.027061+0000","last_peered":"2024-03-06T07:22:01.027061+0000","last_clean":"2024-03-06T07:22:01.027061+0000","last_became_active":"2024-03-06T07:22:00.016279+0000","last_became_peered":"2024-03-06T07:22:00.016279+0000","last_unstale":"2024-03-06T07:22:01.027061+0000","last_undegraded":"2024-03-06T07:22:01.027061+0000","last_fullsized":"2024-03-06T07:22:01.027061+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:41.474284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643060+0000","last_change":"2024-03-06T07:21:59.009295+0000","last_active":"2024-03-06T07:22:01.643060+0000","last_peered":"2024-03-06T07:22:01.643060+0000","last_clean":"2024-03-06T07:22:01.643060+0000","last_became_active":"2024-03-06T07:21:59.009028+0000","last_became_peered":"2024-03-06T07:21:59.009028+0000","last_unstale":"2024-03-06T07:22:01.643060+0000","last_undegraded":"2024-03-06T07:22:01.643060+0000","last_fullsized":"2024-03-06T07:22:01.643060+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:10.417789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000751+0000","last_change":"2024-03-06T07:21:53.936851+0000","last_active":"2024-03-06T07:22:01.000751+0000","last_peered":"2024-03-06T07:22:01.000751+0000","last_clean":"2024-03-06T07:22:01.000751+0000","last_became_active":"2024-03-06T07:21:53.936533+0000","last_became_peered":"2024-03-06T07:21:53.936533+0000","last_unstale":"2024-03-06T07:22:01.000751+0000","last_undegraded":"2024-03-06T07:22:01.000751+0000","last_fullsized":"2024-03-06T07:22:01.000751+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:15.126094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":29,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150549+0000","last_change":"2024-03-06T07:21:42.912311+0000","last_active":"2024-03-06T07:22:01.150549+0000","last_peered":"2024-03-06T07:22:01.150549+0000","last_clean":"2024-03-06T07:22:01.150549+0000","last_became_active":"2024-03-06T07:21:42.912046+0000","last_became_peered":"2024-03-06T07:21:42.912046+0000","last_unstale":"2024-03-06T07:22:01.150549+0000","last_undegraded":"2024-03-06T07:22:01.150549+0000","last_fullsized":"2024-03-06T07:22:01.150549+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:01.118046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000444+0000","last_change":"2024-03-06T07:22:00.015427+0000","last_active":"2024-03-06T07:22:01.000444+0000","last_peered":"2024-03-06T07:22:01.000444+0000","last_clean":"2024-03-06T07:22:01.000444+0000","last_became_active":"2024-03-06T07:22:00.014435+0000","last_became_peered":"2024-03-06T07:22:00.014435+0000","last_unstale":"2024-03-06T07:22:01.000444+0000","last_undegraded":"2024-03-06T07:22:01.000444+0000","last_fullsized":"2024-03-06T07:22:01.000444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:46.340573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642538+0000","last_change":"2024-03-06T07:21:59.007501+0000","last_active":"2024-03-06T07:22:01.642538+0000","last_peered":"2024-03-06T07:22:01.642538+0000","last_clean":"2024-03-06T07:22:01.642538+0000","last_became_active":"2024-03-06T07:21:59.007223+0000","last_became_peered":"2024-03-06T07:21:59.007223+0000","last_unstale":"2024-03-06T07:22:01.642538+0000","last_undegraded":"2024-03-06T07:22:01.642538+0000","last_fullsized":"2024-03-06T07:22:01.642538+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:26.490540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002595+0000","last_change":"2024-03-06T07:21:43.911762+0000","last_active":"2024-03-06T07:22:01.002595+0000","last_peered":"2024-03-06T07:22:01.002595+0000","last_clean":"2024-03-06T07:22:01.002595+0000","last_became_active":"2024-03-06T07:21:43.911470+0000","last_became_peered":"2024-03-06T07:21:43.911470+0000","last_unstale":"2024-03-06T07:22:01.002595+0000","last_undegraded":"2024-03-06T07:22:01.002595+0000","last_fullsized":"2024-03-06T07:22:01.002595+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:43.892653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":32,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643755+0000","last_change":"2024-03-06T07:21:42.912419+0000","last_active":"2024-03-06T07:22:01.643755+0000","last_peered":"2024-03-06T07:22:01.643755+0000","last_clean":"2024-03-06T07:22:01.643755+0000","last_became_active":"2024-03-06T07:21:42.912141+0000","last_became_peered":"2024-03-06T07:21:42.912141+0000","last_unstale":"2024-03-06T07:22:01.643755+0000","last_undegraded":"2024-03-06T07:22:01.643755+0000","last_fullsized":"2024-03-06T07:22:01.643755+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:00.590629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642789+0000","last_change":"2024-03-06T07:22:00.010409+0000","last_active":"2024-03-06T07:22:01.642789+0000","last_peered":"2024-03-06T07:22:01.642789+0000","last_clean":"2024-03-06T07:22:01.642789+0000","last_became_active":"2024-03-06T07:22:00.009751+0000","last_became_peered":"2024-03-06T07:22:00.009751+0000","last_unstale":"2024-03-06T07:22:01.642789+0000","last_undegraded":"2024-03-06T07:22:01.642789+0000","last_fullsized":"2024-03-06T07:22:01.642789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:55.658082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643176+0000","last_change":"2024-03-06T07:21:59.013433+0000","last_active":"2024-03-06T07:22:01.643176+0000","last_peered":"2024-03-06T07:22:01.643176+0000","last_clean":"2024-03-06T07:22:01.643176+0000","last_became_active":"2024-03-06T07:21:59.013170+0000","last_became_peered":"2024-03-06T07:21:59.013170+0000","last_unstale":"2024-03-06T07:22:01.643176+0000","last_undegraded":"2024-03-06T07:22:01.643176+0000","last_fullsized":"2024-03-06T07:22:01.643176+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:41.680002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024424+0000","last_change":"2024-03-06T07:21:43.918063+0000","last_active":"2024-03-06T07:22:01.024424+0000","last_peered":"2024-03-06T07:22:01.024424+0000","last_clean":"2024-03-06T07:22:01.024424+0000","last_became_active":"2024-03-06T07:21:43.917325+0000","last_became_peered":"2024-03-06T07:21:43.917325+0000","last_unstale":"2024-03-06T07:22:01.024424+0000","last_undegraded":"2024-03-06T07:22:01.024424+0000","last_fullsized":"2024-03-06T07:22:01.024424+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.885161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642833+0000","last_change":"2024-03-06T07:21:42.912594+0000","last_active":"2024-03-06T07:22:01.642833+0000","last_peered":"2024-03-06T07:22:01.642833+0000","last_clean":"2024-03-06T07:22:01.642833+0000","last_became_active":"2024-03-06T07:21:42.912053+0000","last_became_peered":"2024-03-06T07:21:42.912053+0000","last_unstale":"2024-03-06T07:22:01.642833+0000","last_undegraded":"2024-03-06T07:22:01.642833+0000","last_fullsized":"2024-03-06T07:22:01.642833+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:42.711847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150532+0000","last_change":"2024-03-06T07:22:00.029826+0000","last_active":"2024-03-06T07:22:01.150532+0000","last_peered":"2024-03-06T07:22:01.150532+0000","last_clean":"2024-03-06T07:22:01.150532+0000","last_became_active":"2024-03-06T07:22:00.029347+0000","last_became_peered":"2024-03-06T07:22:00.029347+0000","last_unstale":"2024-03-06T07:22:01.150532+0000","last_undegraded":"2024-03-06T07:22:01.150532+0000","last_fullsized":"2024-03-06T07:22:01.150532+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:11.422190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643619+0000","last_change":"2024-03-06T07:21:59.010872+0000","last_active":"2024-03-06T07:22:01.643619+0000","last_peered":"2024-03-06T07:22:01.643619+0000","last_clean":"2024-03-06T07:22:01.643619+0000","last_became_active":"2024-03-06T07:21:59.010589+0000","last_became_peered":"2024-03-06T07:21:59.010589+0000","last_unstale":"2024-03-06T07:22:01.643619+0000","last_undegraded":"2024-03-06T07:22:01.643619+0000","last_fullsized":"2024-03-06T07:22:01.643619+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:19.513411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643041+0000","last_change":"2024-03-06T07:21:43.931801+0000","last_active":"2024-03-06T07:22:01.643041+0000","last_peered":"2024-03-06T07:22:01.643041+0000","last_clean":"2024-03-06T07:22:01.643041+0000","last_became_active":"2024-03-06T07:21:43.931515+0000","last_became_peered":"2024-03-06T07:21:43.931515+0000","last_unstale":"2024-03-06T07:22:01.643041+0000","last_undegraded":"2024-03-06T07:22:01.643041+0000","last_fullsized":"2024-03-06T07:22:01.643041+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:51:45.221543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002748+0000","last_change":"2024-03-06T07:21:42.910643+0000","last_active":"2024-03-06T07:22:01.002748+0000","last_peered":"2024-03-06T07:22:01.002748+0000","last_clean":"2024-03-06T07:22:01.002748+0000","last_became_active":"2024-03-06T07:21:42.910377+0000","last_became_peered":"2024-03-06T07:21:42.910377+0000","last_unstale":"2024-03-06T07:22:01.002748+0000","last_undegraded":"2024-03-06T07:22:01.002748+0000","last_fullsized":"2024-03-06T07:22:01.002748+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003023+0000","last_change":"2024-03-06T07:22:00.016675+0000","last_active":"2024-03-06T07:22:01.003023+0000","last_peered":"2024-03-06T07:22:01.003023+0000","last_clean":"2024-03-06T07:22:01.003023+0000","last_became_active":"2024-03-06T07:22:00.016447+0000","last_became_peered":"2024-03-06T07:22:00.016447+0000","last_unstale":"2024-03-06T07:22:01.003023+0000","last_undegraded":"2024-03-06T07:22:01.003023+0000","last_fullsized":"2024-03-06T07:22:01.003023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:22.628016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001440+0000","last_change":"2024-03-06T07:21:59.011671+0000","last_active":"2024-03-06T07:22:01.001440+0000","last_peered":"2024-03-06T07:22:01.001440+0000","last_clean":"2024-03-06T07:22:01.001440+0000","last_became_active":"2024-03-06T07:21:59.011406+0000","last_became_peered":"2024-03-06T07:21:59.011406+0000","last_unstale":"2024-03-06T07:22:01.001440+0000","last_undegraded":"2024-03-06T07:22:01.001440+0000","last_fullsized":"2024-03-06T07:22:01.001440+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:12.104618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643805+0000","last_change":"2024-03-06T07:21:43.915960+0000","last_active":"2024-03-06T07:22:01.643805+0000","last_peered":"2024-03-06T07:22:01.643805+0000","last_clean":"2024-03-06T07:22:01.643805+0000","last_became_active":"2024-03-06T07:21:43.915665+0000","last_became_peered":"2024-03-06T07:21:43.915665+0000","last_unstale":"2024-03-06T07:22:01.643805+0000","last_undegraded":"2024-03-06T07:22:01.643805+0000","last_fullsized":"2024-03-06T07:22:01.643805+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:43.234567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150413+0000","last_change":"2024-03-06T07:21:42.911389+0000","last_active":"2024-03-06T07:22:01.150413+0000","last_peered":"2024-03-06T07:22:01.150413+0000","last_clean":"2024-03-06T07:22:01.150413+0000","last_became_active":"2024-03-06T07:21:42.911070+0000","last_became_peered":"2024-03-06T07:21:42.911070+0000","last_unstale":"2024-03-06T07:22:01.150413+0000","last_undegraded":"2024-03-06T07:22:01.150413+0000","last_fullsized":"2024-03-06T07:22:01.150413+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002062+0000","last_change":"2024-03-06T07:22:00.012556+0000","last_active":"2024-03-06T07:22:01.002062+0000","last_peered":"2024-03-06T07:22:01.002062+0000","last_clean":"2024-03-06T07:22:01.002062+0000","last_became_active":"2024-03-06T07:22:00.012296+0000","last_became_peered":"2024-03-06T07:22:00.012296+0000","last_unstale":"2024-03-06T07:22:01.002062+0000","last_undegraded":"2024-03-06T07:22:01.002062+0000","last_fullsized":"2024-03-06T07:22:01.002062+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:13.838771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642974+0000","last_change":"2024-03-06T07:21:59.013968+0000","last_active":"2024-03-06T07:22:01.642974+0000","last_peered":"2024-03-06T07:22:01.642974+0000","last_clean":"2024-03-06T07:22:01.642974+0000","last_became_active":"2024-03-06T07:21:59.013106+0000","last_became_peered":"2024-03-06T07:21:59.013106+0000","last_unstale":"2024-03-06T07:22:01.642974+0000","last_undegraded":"2024-03-06T07:22:01.642974+0000","last_fullsized":"2024-03-06T07:22:01.642974+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:42.121680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642804+0000","last_change":"2024-03-06T07:21:43.930573+0000","last_active":"2024-03-06T07:22:01.642804+0000","last_peered":"2024-03-06T07:22:01.642804+0000","last_clean":"2024-03-06T07:22:01.642804+0000","last_became_active":"2024-03-06T07:21:43.930105+0000","last_became_peered":"2024-03-06T07:21:43.930105+0000","last_unstale":"2024-03-06T07:22:01.642804+0000","last_undegraded":"2024-03-06T07:22:01.642804+0000","last_fullsized":"2024-03-06T07:22:01.642804+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:07.354210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":34,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642359+0000","last_change":"2024-03-06T07:21:42.915873+0000","last_active":"2024-03-06T07:22:01.642359+0000","last_peered":"2024-03-06T07:22:01.642359+0000","last_clean":"2024-03-06T07:22:01.642359+0000","last_became_active":"2024-03-06T07:21:42.915575+0000","last_became_peered":"2024-03-06T07:21:42.915575+0000","last_unstale":"2024-03-06T07:22:01.642359+0000","last_undegraded":"2024-03-06T07:22:01.642359+0000","last_fullsized":"2024-03-06T07:22:01.642359+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:12:30.008729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000535+0000","last_change":"2024-03-06T07:22:00.013681+0000","last_active":"2024-03-06T07:22:01.000535+0000","last_peered":"2024-03-06T07:22:01.000535+0000","last_clean":"2024-03-06T07:22:01.000535+0000","last_became_active":"2024-03-06T07:22:00.013437+0000","last_became_peered":"2024-03-06T07:22:00.013437+0000","last_unstale":"2024-03-06T07:22:01.000535+0000","last_undegraded":"2024-03-06T07:22:01.000535+0000","last_fullsized":"2024-03-06T07:22:01.000535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:28.427007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642434+0000","last_change":"2024-03-06T07:21:59.010664+0000","last_active":"2024-03-06T07:22:01.642434+0000","last_peered":"2024-03-06T07:22:01.642434+0000","last_clean":"2024-03-06T07:22:01.642434+0000","last_became_active":"2024-03-06T07:21:59.009227+0000","last_became_peered":"2024-03-06T07:21:59.009227+0000","last_unstale":"2024-03-06T07:22:01.642434+0000","last_undegraded":"2024-03-06T07:22:01.642434+0000","last_fullsized":"2024-03-06T07:22:01.642434+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:30.894143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642292+0000","last_change":"2024-03-06T07:21:43.914468+0000","last_active":"2024-03-06T07:22:01.642292+0000","last_peered":"2024-03-06T07:22:01.642292+0000","last_clean":"2024-03-06T07:22:01.642292+0000","last_became_active":"2024-03-06T07:21:43.914210+0000","last_became_peered":"2024-03-06T07:21:43.914210+0000","last_unstale":"2024-03-06T07:22:01.642292+0000","last_undegraded":"2024-03-06T07:22:01.642292+0000","last_fullsized":"2024-03-06T07:22:01.642292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.235915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642981+0000","last_change":"2024-03-06T07:21:42.914001+0000","last_active":"2024-03-06T07:22:01.642981+0000","last_peered":"2024-03-06T07:22:01.642981+0000","last_clean":"2024-03-06T07:22:01.642981+0000","last_became_active":"2024-03-06T07:21:42.913707+0000","last_became_peered":"2024-03-06T07:21:42.913707+0000","last_unstale":"2024-03-06T07:22:01.642981+0000","last_undegraded":"2024-03-06T07:22:01.642981+0000","last_fullsized":"2024-03-06T07:22:01.642981+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:37.811398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027656+0000","last_change":"2024-03-06T07:22:00.014164+0000","last_active":"2024-03-06T07:22:01.027656+0000","last_peered":"2024-03-06T07:22:01.027656+0000","last_clean":"2024-03-06T07:22:01.027656+0000","last_became_active":"2024-03-06T07:22:00.013808+0000","last_became_peered":"2024-03-06T07:22:00.013808+0000","last_unstale":"2024-03-06T07:22:01.027656+0000","last_undegraded":"2024-03-06T07:22:01.027656+0000","last_fullsized":"2024-03-06T07:22:01.027656+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:12.653672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000398+0000","last_change":"2024-03-06T07:21:59.010253+0000","last_active":"2024-03-06T07:22:01.000398+0000","last_peered":"2024-03-06T07:22:01.000398+0000","last_clean":"2024-03-06T07:22:01.000398+0000","last_became_active":"2024-03-06T07:21:59.009984+0000","last_became_peered":"2024-03-06T07:21:59.009984+0000","last_unstale":"2024-03-06T07:22:01.000398+0000","last_undegraded":"2024-03-06T07:22:01.000398+0000","last_fullsized":"2024-03-06T07:22:01.000398+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:33.376118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000520+0000","last_change":"2024-03-06T07:21:43.928109+0000","last_active":"2024-03-06T07:22:01.000520+0000","last_peered":"2024-03-06T07:22:01.000520+0000","last_clean":"2024-03-06T07:22:01.000520+0000","last_became_active":"2024-03-06T07:21:43.927816+0000","last_became_peered":"2024-03-06T07:21:43.927816+0000","last_unstale":"2024-03-06T07:22:01.000520+0000","last_undegraded":"2024-03-06T07:22:01.000520+0000","last_fullsized":"2024-03-06T07:22:01.000520+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.028118+0000","last_change":"2024-03-06T07:21:42.909376+0000","last_active":"2024-03-06T07:22:01.028118+0000","last_peered":"2024-03-06T07:22:01.028118+0000","last_clean":"2024-03-06T07:22:01.028118+0000","last_became_active":"2024-03-06T07:21:42.909101+0000","last_became_peered":"2024-03-06T07:21:42.909101+0000","last_unstale":"2024-03-06T07:22:01.028118+0000","last_undegraded":"2024-03-06T07:22:01.028118+0000","last_fullsized":"2024-03-06T07:22:01.028118+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:46:37.646398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001531+0000","last_change":"2024-03-06T07:22:00.016101+0000","last_active":"2024-03-06T07:22:01.001531+0000","last_peered":"2024-03-06T07:22:01.001531+0000","last_clean":"2024-03-06T07:22:01.001531+0000","last_became_active":"2024-03-06T07:22:00.015844+0000","last_became_peered":"2024-03-06T07:22:00.015844+0000","last_unstale":"2024-03-06T07:22:01.001531+0000","last_undegraded":"2024-03-06T07:22:01.001531+0000","last_fullsized":"2024-03-06T07:22:01.001531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:13.109019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150058+0000","last_change":"2024-03-06T07:21:59.015865+0000","last_active":"2024-03-06T07:22:01.150058+0000","last_peered":"2024-03-06T07:22:01.150058+0000","last_clean":"2024-03-06T07:22:01.150058+0000","last_became_active":"2024-03-06T07:21:59.015574+0000","last_became_peered":"2024-03-06T07:21:59.015574+0000","last_unstale":"2024-03-06T07:22:01.150058+0000","last_undegraded":"2024-03-06T07:22:01.150058+0000","last_fullsized":"2024-03-06T07:22:01.150058+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:28.156169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001680+0000","last_change":"2024-03-06T07:21:43.925791+0000","last_active":"2024-03-06T07:22:01.001680+0000","last_peered":"2024-03-06T07:22:01.001680+0000","last_clean":"2024-03-06T07:22:01.001680+0000","last_became_active":"2024-03-06T07:21:43.925489+0000","last_became_peered":"2024-03-06T07:21:43.925489+0000","last_unstale":"2024-03-06T07:22:01.001680+0000","last_undegraded":"2024-03-06T07:22:01.001680+0000","last_fullsized":"2024-03-06T07:22:01.001680+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:43.716879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003685+0000","last_change":"2024-03-06T07:21:42.912376+0000","last_active":"2024-03-06T07:22:01.003685+0000","last_peered":"2024-03-06T07:22:01.003685+0000","last_clean":"2024-03-06T07:22:01.003685+0000","last_became_active":"2024-03-06T07:21:42.912102+0000","last_became_peered":"2024-03-06T07:21:42.912102+0000","last_unstale":"2024-03-06T07:22:01.003685+0000","last_undegraded":"2024-03-06T07:22:01.003685+0000","last_fullsized":"2024-03-06T07:22:01.003685+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:41.017165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001552+0000","last_change":"2024-03-06T07:22:00.019349+0000","last_active":"2024-03-06T07:22:01.001552+0000","last_peered":"2024-03-06T07:22:01.001552+0000","last_clean":"2024-03-06T07:22:01.001552+0000","last_became_active":"2024-03-06T07:22:00.019092+0000","last_became_peered":"2024-03-06T07:22:00.019092+0000","last_unstale":"2024-03-06T07:22:01.001552+0000","last_undegraded":"2024-03-06T07:22:01.001552+0000","last_fullsized":"2024-03-06T07:22:01.001552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:52.437815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642631+0000","last_change":"2024-03-06T07:21:59.010025+0000","last_active":"2024-03-06T07:22:01.642631+0000","last_peered":"2024-03-06T07:22:01.642631+0000","last_clean":"2024-03-06T07:22:01.642631+0000","last_became_active":"2024-03-06T07:21:59.009738+0000","last_became_peered":"2024-03-06T07:21:59.009738+0000","last_unstale":"2024-03-06T07:22:01.642631+0000","last_undegraded":"2024-03-06T07:22:01.642631+0000","last_fullsized":"2024-03-06T07:22:01.642631+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:39.805595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000663+0000","last_change":"2024-03-06T07:21:43.914898+0000","last_active":"2024-03-06T07:22:01.000663+0000","last_peered":"2024-03-06T07:22:01.000663+0000","last_clean":"2024-03-06T07:22:01.000663+0000","last_became_active":"2024-03-06T07:21:43.914368+0000","last_became_peered":"2024-03-06T07:21:43.914368+0000","last_unstale":"2024-03-06T07:22:01.000663+0000","last_undegraded":"2024-03-06T07:22:01.000663+0000","last_fullsized":"2024-03-06T07:22:01.000663+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:32:44.975253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003372+0000","last_change":"2024-03-06T07:21:42.913064+0000","last_active":"2024-03-06T07:22:01.003372+0000","last_peered":"2024-03-06T07:22:01.003372+0000","last_clean":"2024-03-06T07:22:01.003372+0000","last_became_active":"2024-03-06T07:21:42.912792+0000","last_became_peered":"2024-03-06T07:21:42.912792+0000","last_unstale":"2024-03-06T07:22:01.003372+0000","last_undegraded":"2024-03-06T07:22:01.003372+0000","last_fullsized":"2024-03-06T07:22:01.003372+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:13.058832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024826+0000","last_change":"2024-03-06T07:22:00.013094+0000","last_active":"2024-03-06T07:22:01.024826+0000","last_peered":"2024-03-06T07:22:01.024826+0000","last_clean":"2024-03-06T07:22:01.024826+0000","last_became_active":"2024-03-06T07:22:00.012286+0000","last_became_peered":"2024-03-06T07:22:00.012286+0000","last_unstale":"2024-03-06T07:22:01.024826+0000","last_undegraded":"2024-03-06T07:22:01.024826+0000","last_fullsized":"2024-03-06T07:22:01.024826+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:28:27.494941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642678+0000","last_change":"2024-03-06T07:21:59.013484+0000","last_active":"2024-03-06T07:22:01.642678+0000","last_peered":"2024-03-06T07:22:01.642678+0000","last_clean":"2024-03-06T07:22:01.642678+0000","last_became_active":"2024-03-06T07:21:59.013205+0000","last_became_peered":"2024-03-06T07:21:59.013205+0000","last_unstale":"2024-03-06T07:22:01.642678+0000","last_undegraded":"2024-03-06T07:22:01.642678+0000","last_fullsized":"2024-03-06T07:22:01.642678+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:16.376375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024558+0000","last_change":"2024-03-06T07:21:43.912688+0000","last_active":"2024-03-06T07:22:01.024558+0000","last_peered":"2024-03-06T07:22:01.024558+0000","last_clean":"2024-03-06T07:22:01.024558+0000","last_became_active":"2024-03-06T07:21:43.912441+0000","last_became_peered":"2024-03-06T07:21:43.912441+0000","last_unstale":"2024-03-06T07:22:01.024558+0000","last_undegraded":"2024-03-06T07:22:01.024558+0000","last_fullsized":"2024-03-06T07:22:01.024558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:39.752294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000746+0000","last_change":"2024-03-06T07:21:42.911252+0000","last_active":"2024-03-06T07:22:01.000746+0000","last_peered":"2024-03-06T07:22:01.000746+0000","last_clean":"2024-03-06T07:22:01.000746+0000","last_became_active":"2024-03-06T07:21:42.910758+0000","last_became_peered":"2024-03-06T07:21:42.910758+0000","last_unstale":"2024-03-06T07:22:01.000746+0000","last_undegraded":"2024-03-06T07:22:01.000746+0000","last_fullsized":"2024-03-06T07:22:01.000746+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:43.794100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.004017+0000","last_change":"2024-03-06T07:22:00.012703+0000","last_active":"2024-03-06T07:22:01.004017+0000","last_peered":"2024-03-06T07:22:01.004017+0000","last_clean":"2024-03-06T07:22:01.004017+0000","last_became_active":"2024-03-06T07:22:00.012444+0000","last_became_peered":"2024-03-06T07:22:00.012444+0000","last_unstale":"2024-03-06T07:22:01.004017+0000","last_undegraded":"2024-03-06T07:22:01.004017+0000","last_fullsized":"2024-03-06T07:22:01.004017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:46.340573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001217+0000","last_change":"2024-03-06T07:21:59.021045+0000","last_active":"2024-03-06T07:22:01.001217+0000","last_peered":"2024-03-06T07:22:01.001217+0000","last_clean":"2024-03-06T07:22:01.001217+0000","last_became_active":"2024-03-06T07:21:59.020784+0000","last_became_peered":"2024-03-06T07:21:59.020784+0000","last_unstale":"2024-03-06T07:22:01.001217+0000","last_undegraded":"2024-03-06T07:22:01.001217+0000","last_fullsized":"2024-03-06T07:22:01.001217+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:47.039125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003581+0000","last_change":"2024-03-06T07:21:43.917113+0000","last_active":"2024-03-06T07:22:01.003581+0000","last_peered":"2024-03-06T07:22:01.003581+0000","last_clean":"2024-03-06T07:22:01.003581+0000","last_became_active":"2024-03-06T07:21:43.915472+0000","last_became_peered":"2024-03-06T07:21:43.915472+0000","last_unstale":"2024-03-06T07:22:01.003581+0000","last_undegraded":"2024-03-06T07:22:01.003581+0000","last_fullsized":"2024-03-06T07:22:01.003581+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:33:38.816430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642768+0000","last_change":"2024-03-06T07:21:42.916636+0000","last_active":"2024-03-06T07:22:01.642768+0000","last_peered":"2024-03-06T07:22:01.642768+0000","last_clean":"2024-03-06T07:22:01.642768+0000","last_became_active":"2024-03-06T07:21:42.916345+0000","last_became_peered":"2024-03-06T07:21:42.916345+0000","last_unstale":"2024-03-06T07:22:01.642768+0000","last_undegraded":"2024-03-06T07:22:01.642768+0000","last_fullsized":"2024-03-06T07:22:01.642768+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:59.475185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643522+0000","last_change":"2024-03-06T07:22:00.009310+0000","last_active":"2024-03-06T07:22:01.643522+0000","last_peered":"2024-03-06T07:22:01.643522+0000","last_clean":"2024-03-06T07:22:01.643522+0000","last_became_active":"2024-03-06T07:22:00.009035+0000","last_became_peered":"2024-03-06T07:22:00.009035+0000","last_unstale":"2024-03-06T07:22:01.643522+0000","last_undegraded":"2024-03-06T07:22:01.643522+0000","last_fullsized":"2024-03-06T07:22:01.643522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:55.535875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.025099+0000","last_change":"2024-03-06T07:21:59.012793+0000","last_active":"2024-03-06T07:22:01.025099+0000","last_peered":"2024-03-06T07:22:01.025099+0000","last_clean":"2024-03-06T07:22:01.025099+0000","last_became_active":"2024-03-06T07:21:59.012504+0000","last_became_peered":"2024-03-06T07:21:59.012504+0000","last_unstale":"2024-03-06T07:22:01.025099+0000","last_undegraded":"2024-03-06T07:22:01.025099+0000","last_fullsized":"2024-03-06T07:22:01.025099+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:56.245577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643187+0000","last_change":"2024-03-06T07:21:43.921341+0000","last_active":"2024-03-06T07:22:01.643187+0000","last_peered":"2024-03-06T07:22:01.643187+0000","last_clean":"2024-03-06T07:22:01.643187+0000","last_became_active":"2024-03-06T07:21:43.921062+0000","last_became_peered":"2024-03-06T07:21:43.921062+0000","last_unstale":"2024-03-06T07:22:01.643187+0000","last_undegraded":"2024-03-06T07:22:01.643187+0000","last_fullsized":"2024-03-06T07:22:01.643187+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:15.340198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643198+0000","last_change":"2024-03-06T07:21:42.912294+0000","last_active":"2024-03-06T07:22:01.643198+0000","last_peered":"2024-03-06T07:22:01.643198+0000","last_clean":"2024-03-06T07:22:01.643198+0000","last_became_active":"2024-03-06T07:21:42.912023+0000","last_became_peered":"2024-03-06T07:21:42.912023+0000","last_unstale":"2024-03-06T07:22:01.643198+0000","last_undegraded":"2024-03-06T07:22:01.643198+0000","last_fullsized":"2024-03-06T07:22:01.643198+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027972+0000","last_change":"2024-03-06T07:22:00.009261+0000","last_active":"2024-03-06T07:22:01.027972+0000","last_peered":"2024-03-06T07:22:01.027972+0000","last_clean":"2024-03-06T07:22:01.027972+0000","last_became_active":"2024-03-06T07:22:00.008928+0000","last_became_peered":"2024-03-06T07:22:00.008928+0000","last_unstale":"2024-03-06T07:22:01.027972+0000","last_undegraded":"2024-03-06T07:22:01.027972+0000","last_fullsized":"2024-03-06T07:22:01.027972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:19:49.275933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.026543+0000","last_change":"2024-03-06T07:21:59.008051+0000","last_active":"2024-03-06T07:22:01.026543+0000","last_peered":"2024-03-06T07:22:01.026543+0000","last_clean":"2024-03-06T07:22:01.026543+0000","last_became_active":"2024-03-06T07:21:59.007748+0000","last_became_peered":"2024-03-06T07:21:59.007748+0000","last_unstale":"2024-03-06T07:22:01.026543+0000","last_undegraded":"2024-03-06T07:22:01.026543+0000","last_fullsized":"2024-03-06T07:22:01.026543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:45.336173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.001490+0000","last_change":"2024-03-06T07:21:43.905564+0000","last_active":"2024-03-06T07:22:01.001490+0000","last_peered":"2024-03-06T07:22:01.001490+0000","last_clean":"2024-03-06T07:22:01.001490+0000","last_became_active":"2024-03-06T07:21:43.905280+0000","last_became_peered":"2024-03-06T07:21:43.905280+0000","last_unstale":"2024-03-06T07:22:01.001490+0000","last_undegraded":"2024-03-06T07:22:01.001490+0000","last_fullsized":"2024-03-06T07:22:01.001490+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:00.480217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.027211+0000","last_change":"2024-03-06T07:21:42.908701+0000","last_active":"2024-03-06T07:22:01.027211+0000","last_peered":"2024-03-06T07:22:01.027211+0000","last_clean":"2024-03-06T07:22:01.027211+0000","last_became_active":"2024-03-06T07:21:42.908429+0000","last_became_peered":"2024-03-06T07:21:42.908429+0000","last_unstale":"2024-03-06T07:22:01.027211+0000","last_undegraded":"2024-03-06T07:22:01.027211+0000","last_fullsized":"2024-03-06T07:22:01.027211+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:36:55.863684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002586+0000","last_change":"2024-03-06T07:22:00.012499+0000","last_active":"2024-03-06T07:22:01.002586+0000","last_peered":"2024-03-06T07:22:01.002586+0000","last_clean":"2024-03-06T07:22:01.002586+0000","last_became_active":"2024-03-06T07:22:00.012216+0000","last_became_peered":"2024-03-06T07:22:00.012216+0000","last_unstale":"2024-03-06T07:22:01.002586+0000","last_undegraded":"2024-03-06T07:22:01.002586+0000","last_fullsized":"2024-03-06T07:22:01.002586+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:13.838771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643283+0000","last_change":"2024-03-06T07:21:59.013665+0000","last_active":"2024-03-06T07:22:01.643283+0000","last_peered":"2024-03-06T07:22:01.643283+0000","last_clean":"2024-03-06T07:22:01.643283+0000","last_became_active":"2024-03-06T07:21:59.013401+0000","last_became_peered":"2024-03-06T07:21:59.013401+0000","last_unstale":"2024-03-06T07:22:01.643283+0000","last_undegraded":"2024-03-06T07:22:01.643283+0000","last_fullsized":"2024-03-06T07:22:01.643283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:20.389960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024334+0000","last_change":"2024-03-06T07:21:43.909104+0000","last_active":"2024-03-06T07:22:01.024334+0000","last_peered":"2024-03-06T07:22:01.024334+0000","last_clean":"2024-03-06T07:22:01.024334+0000","last_became_active":"2024-03-06T07:21:43.908810+0000","last_became_peered":"2024-03-06T07:21:43.908810+0000","last_unstale":"2024-03-06T07:22:01.024334+0000","last_undegraded":"2024-03-06T07:22:01.024334+0000","last_fullsized":"2024-03-06T07:22:01.024334+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:14.273650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":28,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003969+0000","last_change":"2024-03-06T07:21:42.914080+0000","last_active":"2024-03-06T07:22:01.003969+0000","last_peered":"2024-03-06T07:22:01.003969+0000","last_clean":"2024-03-06T07:22:01.003969+0000","last_became_active":"2024-03-06T07:21:42.913818+0000","last_became_peered":"2024-03-06T07:21:42.913818+0000","last_unstale":"2024-03-06T07:22:01.003969+0000","last_undegraded":"2024-03-06T07:22:01.003969+0000","last_fullsized":"2024-03-06T07:22:01.003969+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:46:28.725020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642574+0000","last_change":"2024-03-06T07:22:00.010751+0000","last_active":"2024-03-06T07:22:01.642574+0000","last_peered":"2024-03-06T07:22:01.642574+0000","last_clean":"2024-03-06T07:22:01.642574+0000","last_became_active":"2024-03-06T07:22:00.009885+0000","last_became_peered":"2024-03-06T07:22:00.009885+0000","last_unstale":"2024-03-06T07:22:01.642574+0000","last_undegraded":"2024-03-06T07:22:01.642574+0000","last_fullsized":"2024-03-06T07:22:01.642574+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:31.898544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642622+0000","last_change":"2024-03-06T07:21:59.012755+0000","last_active":"2024-03-06T07:22:01.642622+0000","last_peered":"2024-03-06T07:22:01.642622+0000","last_clean":"2024-03-06T07:22:01.642622+0000","last_became_active":"2024-03-06T07:21:59.012388+0000","last_became_peered":"2024-03-06T07:21:59.012388+0000","last_unstale":"2024-03-06T07:22:01.642622+0000","last_undegraded":"2024-03-06T07:22:01.642622+0000","last_fullsized":"2024-03-06T07:22:01.642622+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:13.064323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.000959+0000","last_change":"2024-03-06T07:21:43.917579+0000","last_active":"2024-03-06T07:22:01.000959+0000","last_peered":"2024-03-06T07:22:01.000959+0000","last_clean":"2024-03-06T07:22:01.000959+0000","last_became_active":"2024-03-06T07:21:43.916663+0000","last_became_peered":"2024-03-06T07:21:43.916663+0000","last_unstale":"2024-03-06T07:22:01.000959+0000","last_undegraded":"2024-03-06T07:22:01.000959+0000","last_fullsized":"2024-03-06T07:22:01.000959+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:43.622892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150702+0000","last_change":"2024-03-06T07:21:42.912139+0000","last_active":"2024-03-06T07:22:01.150702+0000","last_peered":"2024-03-06T07:22:01.150702+0000","last_clean":"2024-03-06T07:22:01.150702+0000","last_became_active":"2024-03-06T07:21:42.911573+0000","last_became_peered":"2024-03-06T07:21:42.911573+0000","last_unstale":"2024-03-06T07:22:01.150702+0000","last_undegraded":"2024-03-06T07:22:01.150702+0000","last_fullsized":"2024-03-06T07:22:01.150702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:29.368425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024289+0000","last_change":"2024-03-06T07:22:00.012775+0000","last_active":"2024-03-06T07:22:01.024289+0000","last_peered":"2024-03-06T07:22:01.024289+0000","last_clean":"2024-03-06T07:22:01.024289+0000","last_became_active":"2024-03-06T07:22:00.012122+0000","last_became_peered":"2024-03-06T07:22:00.012122+0000","last_unstale":"2024-03-06T07:22:01.024289+0000","last_undegraded":"2024-03-06T07:22:01.024289+0000","last_fullsized":"2024-03-06T07:22:01.024289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:14.068723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024560+0000","last_change":"2024-03-06T07:21:59.010619+0000","last_active":"2024-03-06T07:22:01.024560+0000","last_peered":"2024-03-06T07:22:01.024560+0000","last_clean":"2024-03-06T07:22:01.024560+0000","last_became_active":"2024-03-06T07:21:59.010286+0000","last_became_peered":"2024-03-06T07:21:59.010286+0000","last_unstale":"2024-03-06T07:22:01.024560+0000","last_undegraded":"2024-03-06T07:22:01.024560+0000","last_fullsized":"2024-03-06T07:22:01.024560+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:58.331501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643191+0000","last_change":"2024-03-06T07:21:43.912734+0000","last_active":"2024-03-06T07:22:01.643191+0000","last_peered":"2024-03-06T07:22:01.643191+0000","last_clean":"2024-03-06T07:22:01.643191+0000","last_became_active":"2024-03-06T07:21:43.912449+0000","last_became_peered":"2024-03-06T07:21:43.912449+0000","last_unstale":"2024-03-06T07:22:01.643191+0000","last_undegraded":"2024-03-06T07:22:01.643191+0000","last_fullsized":"2024-03-06T07:22:01.643191+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:41.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643724+0000","last_change":"2024-03-06T07:21:42.907567+0000","last_active":"2024-03-06T07:22:01.643724+0000","last_peered":"2024-03-06T07:22:01.643724+0000","last_clean":"2024-03-06T07:22:01.643724+0000","last_became_active":"2024-03-06T07:21:42.907271+0000","last_became_peered":"2024-03-06T07:21:42.907271+0000","last_unstale":"2024-03-06T07:22:01.643724+0000","last_undegraded":"2024-03-06T07:22:01.643724+0000","last_fullsized":"2024-03-06T07:22:01.643724+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:25.282305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003128+0000","last_change":"2024-03-06T07:22:00.013666+0000","last_active":"2024-03-06T07:22:01.003128+0000","last_peered":"2024-03-06T07:22:01.003128+0000","last_clean":"2024-03-06T07:22:01.003128+0000","last_became_active":"2024-03-06T07:22:00.013416+0000","last_became_peered":"2024-03-06T07:22:00.013416+0000","last_unstale":"2024-03-06T07:22:01.003128+0000","last_undegraded":"2024-03-06T07:22:01.003128+0000","last_fullsized":"2024-03-06T07:22:01.003128+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:45.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150277+0000","last_change":"2024-03-06T07:21:59.015347+0000","last_active":"2024-03-06T07:22:01.150277+0000","last_peered":"2024-03-06T07:22:01.150277+0000","last_clean":"2024-03-06T07:22:01.150277+0000","last_became_active":"2024-03-06T07:21:59.014691+0000","last_became_peered":"2024-03-06T07:21:59.014691+0000","last_unstale":"2024-03-06T07:22:01.150277+0000","last_undegraded":"2024-03-06T07:22:01.150277+0000","last_fullsized":"2024-03-06T07:22:01.150277+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:00.318477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643772+0000","last_change":"2024-03-06T07:21:43.921555+0000","last_active":"2024-03-06T07:22:01.643772+0000","last_peered":"2024-03-06T07:22:01.643772+0000","last_clean":"2024-03-06T07:22:01.643772+0000","last_became_active":"2024-03-06T07:21:43.921242+0000","last_became_peered":"2024-03-06T07:21:43.921242+0000","last_unstale":"2024-03-06T07:22:01.643772+0000","last_undegraded":"2024-03-06T07:22:01.643772+0000","last_fullsized":"2024-03-06T07:22:01.643772+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:31.942191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643425+0000","last_change":"2024-03-06T07:21:42.915454+0000","last_active":"2024-03-06T07:22:01.643425+0000","last_peered":"2024-03-06T07:22:01.643425+0000","last_clean":"2024-03-06T07:22:01.643425+0000","last_became_active":"2024-03-06T07:21:42.915127+0000","last_became_peered":"2024-03-06T07:21:42.915127+0000","last_unstale":"2024-03-06T07:22:01.643425+0000","last_undegraded":"2024-03-06T07:22:01.643425+0000","last_fullsized":"2024-03-06T07:22:01.643425+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:14.121063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003832+0000","last_change":"2024-03-06T07:22:00.017481+0000","last_active":"2024-03-06T07:22:01.003832+0000","last_peered":"2024-03-06T07:22:01.003832+0000","last_clean":"2024-03-06T07:22:01.003832+0000","last_became_active":"2024-03-06T07:22:00.017193+0000","last_became_peered":"2024-03-06T07:22:00.017193+0000","last_unstale":"2024-03-06T07:22:01.003832+0000","last_undegraded":"2024-03-06T07:22:01.003832+0000","last_fullsized":"2024-03-06T07:22:01.003832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:57.249978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643591+0000","last_change":"2024-03-06T07:21:59.021225+0000","last_active":"2024-03-06T07:22:01.643591+0000","last_peered":"2024-03-06T07:22:01.643591+0000","last_clean":"2024-03-06T07:22:01.643591+0000","last_became_active":"2024-03-06T07:21:59.020920+0000","last_became_peered":"2024-03-06T07:21:59.020920+0000","last_unstale":"2024-03-06T07:22:01.643591+0000","last_undegraded":"2024-03-06T07:22:01.643591+0000","last_fullsized":"2024-03-06T07:22:01.643591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:46.358444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024914+0000","last_change":"2024-03-06T07:21:43.919585+0000","last_active":"2024-03-06T07:22:01.024914+0000","last_peered":"2024-03-06T07:22:01.024914+0000","last_clean":"2024-03-06T07:22:01.024914+0000","last_became_active":"2024-03-06T07:21:43.919255+0000","last_became_peered":"2024-03-06T07:21:43.919255+0000","last_unstale":"2024-03-06T07:22:01.024914+0000","last_undegraded":"2024-03-06T07:22:01.024914+0000","last_fullsized":"2024-03-06T07:22:01.024914+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:37:16.183452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642860+0000","last_change":"2024-03-06T07:21:42.912512+0000","last_active":"2024-03-06T07:22:01.642860+0000","last_peered":"2024-03-06T07:22:01.642860+0000","last_clean":"2024-03-06T07:22:01.642860+0000","last_became_active":"2024-03-06T07:21:42.912235+0000","last_became_peered":"2024-03-06T07:21:42.912235+0000","last_unstale":"2024-03-06T07:22:01.642860+0000","last_undegraded":"2024-03-06T07:22:01.642860+0000","last_fullsized":"2024-03-06T07:22:01.642860+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.427916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150157+0000","last_change":"2024-03-06T07:22:00.030378+0000","last_active":"2024-03-06T07:22:01.150157+0000","last_peered":"2024-03-06T07:22:01.150157+0000","last_clean":"2024-03-06T07:22:01.150157+0000","last_became_active":"2024-03-06T07:22:00.030094+0000","last_became_peered":"2024-03-06T07:22:00.030094+0000","last_unstale":"2024-03-06T07:22:01.150157+0000","last_undegraded":"2024-03-06T07:22:01.150157+0000","last_fullsized":"2024-03-06T07:22:01.150157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:36.032336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024923+0000","last_change":"2024-03-06T07:21:59.011385+0000","last_active":"2024-03-06T07:22:01.024923+0000","last_peered":"2024-03-06T07:22:01.024923+0000","last_clean":"2024-03-06T07:22:01.024923+0000","last_became_active":"2024-03-06T07:21:59.011065+0000","last_became_peered":"2024-03-06T07:21:59.011065+0000","last_unstale":"2024-03-06T07:22:01.024923+0000","last_undegraded":"2024-03-06T07:22:01.024923+0000","last_fullsized":"2024-03-06T07:22:01.024923+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:22.451144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643092+0000","last_change":"2024-03-06T07:21:43.933046+0000","last_active":"2024-03-06T07:22:01.643092+0000","last_peered":"2024-03-06T07:22:01.643092+0000","last_clean":"2024-03-06T07:22:01.643092+0000","last_became_active":"2024-03-06T07:21:43.932764+0000","last_became_peered":"2024-03-06T07:21:43.932764+0000","last_unstale":"2024-03-06T07:22:01.643092+0000","last_undegraded":"2024-03-06T07:22:01.643092+0000","last_fullsized":"2024-03-06T07:22:01.643092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:47.292953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002507+0000","last_change":"2024-03-06T07:21:42.912274+0000","last_active":"2024-03-06T07:22:01.002507+0000","last_peered":"2024-03-06T07:22:01.002507+0000","last_clean":"2024-03-06T07:22:01.002507+0000","last_became_active":"2024-03-06T07:21:42.912000+0000","last_became_peered":"2024-03-06T07:21:42.912000+0000","last_unstale":"2024-03-06T07:22:01.002507+0000","last_undegraded":"2024-03-06T07:22:01.002507+0000","last_fullsized":"2024-03-06T07:22:01.002507+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:00.006841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.003092+0000","last_change":"2024-03-06T07:22:00.014240+0000","last_active":"2024-03-06T07:22:01.003092+0000","last_peered":"2024-03-06T07:22:01.003092+0000","last_clean":"2024-03-06T07:22:01.003092+0000","last_became_active":"2024-03-06T07:22:00.013819+0000","last_became_peered":"2024-03-06T07:22:00.013819+0000","last_unstale":"2024-03-06T07:22:01.003092+0000","last_undegraded":"2024-03-06T07:22:01.003092+0000","last_fullsized":"2024-03-06T07:22:01.003092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:01.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.150387+0000","last_change":"2024-03-06T07:21:59.021336+0000","last_active":"2024-03-06T07:22:01.150387+0000","last_peered":"2024-03-06T07:22:01.150387+0000","last_clean":"2024-03-06T07:22:01.150387+0000","last_became_active":"2024-03-06T07:21:59.021048+0000","last_became_peered":"2024-03-06T07:21:59.021048+0000","last_unstale":"2024-03-06T07:22:01.150387+0000","last_undegraded":"2024-03-06T07:22:01.150387+0000","last_fullsized":"2024-03-06T07:22:01.150387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:22.451144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643974+0000","last_change":"2024-03-06T07:21:43.906537+0000","last_active":"2024-03-06T07:22:01.643974+0000","last_peered":"2024-03-06T07:22:01.643974+0000","last_clean":"2024-03-06T07:22:01.643974+0000","last_became_active":"2024-03-06T07:21:43.906267+0000","last_became_peered":"2024-03-06T07:21:43.906267+0000","last_unstale":"2024-03-06T07:22:01.643974+0000","last_undegraded":"2024-03-06T07:22:01.643974+0000","last_fullsized":"2024-03-06T07:22:01.643974+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:45.221948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":26,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.024662+0000","last_change":"2024-03-06T07:21:42.912363+0000","last_active":"2024-03-06T07:22:01.024662+0000","last_peered":"2024-03-06T07:22:01.024662+0000","last_clean":"2024-03-06T07:22:01.024662+0000","last_became_active":"2024-03-06T07:21:42.912097+0000","last_became_peered":"2024-03-06T07:21:42.912097+0000","last_unstale":"2024-03-06T07:22:01.024662+0000","last_undegraded":"2024-03-06T07:22:01.024662+0000","last_fullsized":"2024-03-06T07:22:01.024662+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:13.238577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002208+0000","last_change":"2024-03-06T07:22:00.015444+0000","last_active":"2024-03-06T07:22:01.002208+0000","last_peered":"2024-03-06T07:22:01.002208+0000","last_clean":"2024-03-06T07:22:01.002208+0000","last_became_active":"2024-03-06T07:22:00.015191+0000","last_became_peered":"2024-03-06T07:22:00.015191+0000","last_unstale":"2024-03-06T07:22:01.002208+0000","last_undegraded":"2024-03-06T07:22:01.002208+0000","last_fullsized":"2024-03-06T07:22:01.002208+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:58.990664+0000","last_clean_scrub_stamp":"2024-03-06T07:21:58.990664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:26:48.043526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.642183+0000","last_change":"2024-03-06T07:21:59.008775+0000","last_active":"2024-03-06T07:22:01.642183+0000","last_peered":"2024-03-06T07:22:01.642183+0000","last_clean":"2024-03-06T07:22:01.642183+0000","last_became_active":"2024-03-06T07:21:59.008479+0000","last_became_peered":"2024-03-06T07:21:59.008479+0000","last_unstale":"2024-03-06T07:22:01.642183+0000","last_undegraded":"2024-03-06T07:22:01.642183+0000","last_fullsized":"2024-03-06T07:22:01.642183+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:57.986264+0000","last_clean_scrub_stamp":"2024-03-06T07:21:57.986264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:12.372186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.002052+0000","last_change":"2024-03-06T07:21:43.913294+0000","last_active":"2024-03-06T07:22:01.002052+0000","last_peered":"2024-03-06T07:22:01.002052+0000","last_clean":"2024-03-06T07:22:01.002052+0000","last_became_active":"2024-03-06T07:21:43.913017+0000","last_became_peered":"2024-03-06T07:21:43.913017+0000","last_unstale":"2024-03-06T07:22:01.002052+0000","last_undegraded":"2024-03-06T07:22:01.002052+0000","last_fullsized":"2024-03-06T07:22:01.002052+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:42.889330+0000","last_clean_scrub_stamp":"2024-03-06T07:21:42.889330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:26.287336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":27,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.643281+0000","last_change":"2024-03-06T07:21:42.912585+0000","last_active":"2024-03-06T07:22:01.643281+0000","last_peered":"2024-03-06T07:22:01.643281+0000","last_clean":"2024-03-06T07:22:01.643281+0000","last_became_active":"2024-03-06T07:21:42.911650+0000","last_became_peered":"2024-03-06T07:21:42.911650+0000","last_unstale":"2024-03-06T07:22:01.643281+0000","last_undegraded":"2024-03-06T07:22:01.643281+0000","last_fullsized":"2024-03-06T07:22:01.643281+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:21:41.884298+0000","last_clean_scrub_stamp":"2024-03-06T07:21:41.884298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:42.887622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-06T07:22:01.149855+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:01.149855+0000","last_peered":"2024-03-06T07:22:01.149855+0000","last_clean":"2024-03-06T07:22:01.149855+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:01.149855+0000","last_undegraded":"2024-03-06T07:22:01.149855+0000","last_fullsized":"2024-03-06T07:22:01.149855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":770048,"data_stored":906845,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":316263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":315599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":317771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":696320,"data_stored":317211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":317189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":318113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":741376,"data_stored":898394,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:04.566 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:04.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 3:0 2024-03-06T07:22:04.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.702+0000 7fe8f77e2640 1 Processor -- start 2024-03-06T07:22:04.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 -- start start 2024-03-06T07:22:04.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8f0072cd0 0x7fe8f006fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f00733f0 con 0x7fe8f006f550 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f0155ca0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f77e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01561d0 con 0x7fe8f0152720 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f4d7d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8f006f550 0x7fe8f00c77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55670/0 (socket says 172.21.15.17:55670) 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f5d7f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8f0072cd0 0x7fe8f006fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f4d7d640 1 -- 172.21.15.17:0/665166116 learned_addr learned my addr 172.21.15.17:0/665166116 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f557e640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f5d7f640 1 -- 172.21.15.17:0/665166116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 msgr2=0x7fe8f01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f5d7f640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f01593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f5d7f640 1 -- 172.21.15.17:0/665166116 >> v1:172.21.15.17:6789/0 conn(0x7fe8f006f550 legacy=0x7fe8f00c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f5d7f640 1 -- 172.21.15.17:0/665166116 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0155ef0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8effff640 1 -- 172.21.15.17:0/665166116 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294790205 0 0) 0x7fe8f00733f0 con 0x7fe8f006f550 2024-03-06T07:22:04.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.703+0000 7fe8f557e640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f5d7f640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f006fd90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009d90 tx=0x7fe8e002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470ed1511f6643e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8effff640 1 -- 172.21.15.17:0/665166116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0039070 con 0x7fe8f0072cd0 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8effff640 1 -- 172.21.15.17:0/665166116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e002cbc0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8effff640 1 -- 172.21.15.17:0/665166116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0034440 con 0x7fe8f0072cd0 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 -- 172.21.15.17:0/665166116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 msgr2=0x7fe8f006fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f006fd90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009d90 tx=0x7fe8e002be10 comp rx=0 tx=0).stop 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 -- 172.21.15.17:0/665166116 shutdown_connections 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f006fd90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/665166116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 -- 172.21.15.17:0/665166116 >> 172.21.15.17:0/665166116 conn(0x7fe8f006ea40 msgr2=0x7fe8f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 -- 172.21.15.17:0/665166116 shutdown_connections 2024-03-06T07:22:04.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.705+0000 7fe8f77e2640 1 -- 172.21.15.17:0/665166116 wait complete. 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 Processor -- start 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 -- start start 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f006f550 0x7fe8f006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f014af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0158ea0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0158bc0 con 0x7fe8f006f550 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.706+0000 7fe8f77e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00c7ef0 con 0x7fe8f0152720 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35786/0 (socket says 172.21.15.17:35786) 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 -- 172.21.15.17:0/1750036690 learned_addr learned my addr 172.21.15.17:0/1750036690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 -- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 msgr2=0x7fe8f014af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f014af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 -- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f006f550 msgr2=0x7fe8f006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f006f550 0x7fe8f006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 -- 172.21.15.17:0/1750036690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f014b4b0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f4d7d640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c4b0 tx=0x7fe8e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4018070 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e4007500 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f01ee7f0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4004a20 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.708+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f0154b50 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.709+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f004f330 con 0x7fe8f0072cd0 2024-03-06T07:22:04.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.709+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e401b070 con 0x7fe8f0072cd0 2024-03-06T07:22:04.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.709+0000 7fe8edffb640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 0x7fe8dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:04.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.709+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 7015+0+0 (secure 0 0 0) 0x7fe8e4089930 con 0x7fe8f0072cd0 2024-03-06T07:22:04.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.710+0000 7fe8f557e640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 0x7fe8dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:04.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.710+0000 7fe8f557e640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 0x7fe8dc06d9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:04.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.715+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8e40583e0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.970+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "3:0"} v 0) v1 -- 0x7fe8f014f4f0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.972 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:22:04.970+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:04.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.972+0000 7fe8edffb640 1 -- 172.21.15.17:0/1750036690 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "3:0"}]=0 failed mds gid 5772 v37) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8e4057ce0 con 0x7fe8f0072cd0 2024-03-06T07:22:04.973 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 5772 2024-03-06T07:22:04.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.974+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 msgr2=0x7fe8dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 0x7fe8dc06d9c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8008040 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 msgr2=0x7fe8f014aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c4b0 tx=0x7fe8e400c970 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 shutdown_connections 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8dc06b500 0x7fe8dc06d9c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f0152720 0x7fe8f014af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f0072cd0 0x7fe8f014aa30 secure :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c4b0 tx=0x7fe8e400c970 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 --2- 172.21.15.17:0/1750036690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f006f550 0x7fe8f006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 >> 172.21.15.17:0/1750036690 conn(0x7fe8f006ea40 msgr2=0x7fe8f014be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 shutdown_connections 2024-03-06T07:22:04.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:04.975+0000 7fe8f77e2640 1 -- 172.21.15.17:0/1750036690 wait complete. 2024-03-06T07:22:04.978 INFO:tasks.ceph.mds.a.smithi017.stderr: -730> 2024-03-06T07:21:45.884+0000 7f9285948640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:04.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:05.006 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:22:05.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.094+0000 7f4948e78640 1 Processor -- start 2024-03-06T07:22:05.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.094+0000 7f4948e78640 1 -- start start 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4948e78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f4944153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4948e78640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4944070aa0 con 0x7f49440acc00 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4948e78640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4944070d80 con 0x7f4944156ea0 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4948e78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944157320 con 0x7f49441506b0 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4942d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49440acc00 0x7f49440c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55698/0 (socket says 172.21.15.17:55698) 2024-03-06T07:22:05.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4942d9d640 1 -- 172.21.15.17:0/2009037201 learned_addr learned my addr 172.21.15.17:0/2009037201 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100035269 0 0) 0x7f4944070aa0 con 0x7f49440acc00 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f494259c640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f4944153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.095+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4930003520 con 0x7f49440acc00 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243256403 0 0) 0x7f4930003520 con 0x7f49440acc00 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 msgr2=0x7f4944153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f4944153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 >> v1:172.21.15.163:6789/0 conn(0x7f4944156ea0 legacy=0x7f49440702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49441566a0 con 0x7f49440acc00 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f493c0019f0 con 0x7f49440acc00 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 >> v1:172.21.15.17:6789/0 conn(0x7f49440acc00 legacy=0x7f49440c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f49300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4930004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930004d40 0x7f49300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944157320 con 0x7f4930003f60 2024-03-06T07:22:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493000baa0 con 0x7f4930004d40 2024-03-06T07:22:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493000bca0 con 0x7f49441506b0 2024-03-06T07:22:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4930004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f494259c640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f49300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f4942d9d640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930004d40 0x7f49300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 -- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 msgr2=0x7f49300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f49300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 -- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930004d40 msgr2=0x7f49300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4930004d40 0x7f49300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f493bfff640 1 -- 172.21.15.17:0/2009037201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930003520 con 0x7f4930003f60 2024-03-06T07:22:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.096+0000 7f494259c640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441506b0 0x7f49300039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f493bfff640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4930004800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f492c001720 tx=0x7f4930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd4294f30790653 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c001370 con 0x7f4930003f60 2024-03-06T07:22:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c0052d0 con 0x7f4930003f60 2024-03-06T07:22:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4941d9b640 1 -- 172.21.15.17:0/2009037201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c003830 con 0x7f4930003f60 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 -- 172.21.15.17:0/2009037201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 msgr2=0x7f4930004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 --2- 172.21.15.17:0/2009037201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4930004800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f492c001720 tx=0x7f4930005830 comp rx=0 tx=0).stop 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4942d9d640 1 -- 172.21.15.17:0/2009037201 reap_dead start 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 -- 172.21.15.17:0/2009037201 shutdown_connections 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 -- 172.21.15.17:0/2009037201 >> 172.21.15.17:0/2009037201 conn(0x7f494406b1a0 msgr2=0x7f494406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 -- 172.21.15.17:0/2009037201 shutdown_connections 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.098+0000 7f4948e78640 1 -- 172.21.15.17:0/2009037201 wait complete. 2024-03-06T07:22:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.099+0000 7f4948e78640 1 Processor -- start 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 -- start start 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49440a87d0 0x7f4944148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441492c0 0x7f49441e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944156a10 con 0x7f4930003f60 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944156450 con 0x7f49440a87d0 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f4948e78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440ad090 con 0x7f49441492c0 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f494259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f494259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35808/0 (socket says 172.21.15.17:35808) 2024-03-06T07:22:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.100+0000 7f494259c640 1 -- 172.21.15.17:0/15804426 learned_addr learned my addr 172.21.15.17:0/15804426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f493bfff640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49440a87d0 0x7f4944148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 -- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441492c0 msgr2=0x7f49441e8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f4942d9d640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441492c0 0x7f49441e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441492c0 0x7f49441e8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 -- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49440a87d0 msgr2=0x7f4944148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49440a87d0 0x7f4944148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 -- 172.21.15.17:0/15804426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440aa090 con 0x7f4930003f60 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f494259c640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4934011ca0 tx=0x7f493400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493400fd20 con 0x7f4930003f60 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934014440 con 0x7f4930003f60 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440ab0d0 con 0x7f4930003f60 2024-03-06T07:22:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934014830 con 0x7f4930003f60 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.101+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4944155fc0 con 0x7f4930003f60 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.102+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49440ab2a0 con 0x7f4930003f60 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.103+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4934012040 con 0x7f4930003f60 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.103+0000 7f493affd640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.103+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7101+0+0 (secure 0 0 0) 0x7f493408afe0 con 0x7f4930003f60 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.103+0000 7f493bfff640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.104+0000 7f493bfff640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 0x7f492406d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f492c033000 tx=0x7f492c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.108+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49440ab2a0 con 0x7f4930003f60 2024-03-06T07:22:05.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.329+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49441496a0 con 0x7f4930003f60 2024-03-06T07:22:05.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.330+0000 7f493affd640 1 -- 172.21.15.17:0/15804426 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f4934019030 con 0x7f4930003f60 2024-03-06T07:22:05.332 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:05.332 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:04.975870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"d","rank":0,"incarnation":38,"state":"up:replay","state_seq":7,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:05.332 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 38 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 msgr2=0x7f492406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 0x7f492406d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f492c033000 tx=0x7f492c009030 comp rx=0 tx=0).stop 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 msgr2=0x7f4944149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4934011ca0 tx=0x7f493400f610 comp rx=0 tx=0).stop 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 shutdown_connections 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f492406b500 0x7f492406d9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49441492c0 0x7f49441e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49440a87d0 0x7f4944148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 --2- 172.21.15.17:0/15804426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4930003f60 0x7f4944149c70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 >> 172.21.15.17:0/15804426 conn(0x7f494406b1a0 msgr2=0x7f49440ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 shutdown_connections 2024-03-06T07:22:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.332+0000 7f4948e78640 1 -- 172.21.15.17:0/15804426 wait complete. 2024-03-06T07:22:05.345 INFO:tasks.cephfs.test_failover:Waiting for former active to reclaim its spot 2024-03-06T07:22:05.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-03-06T07:22:05.494 INFO:teuthology.orchestra.run.smithi017.stdout:{"mds_beacon_grace":"15.000000"} 2024-03-06T07:22:05.494 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-03-06T07:22:05.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.594+0000 7f262bc0e640 1 Processor -- start 2024-03-06T07:22:05.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.594+0000 7f262bc0e640 1 -- start start 2024-03-06T07:22:05.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262bc0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f26241527b0 0x7f2624159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262bc0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 0x7f26240c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262bc0e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26240c7c60 con 0x7f26240c7180 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262bc0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2624159640 con 0x7f26241527b0 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262bc0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c8060 con 0x7f26240c88b0 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f26241527b0 0x7f2624159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f26241527b0 0x7f2624159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35816/0 (socket says 172.21.15.17:35816) 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 -- 172.21.15.17:0/3456676681 learned_addr learned my addr 172.21.15.17:0/3456676681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262a1ab640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 0x7f26240c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 -- 172.21.15.17:0/3456676681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 msgr2=0x7f26240c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 0x7f26240c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 -- 172.21.15.17:0/3456676681 >> v1:172.21.15.163:6789/0 conn(0x7f26240c7180 legacy=0x7f2624155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f26299aa640 1 -- 172.21.15.17:0/3456676681 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240c76a0 con 0x7f26241527b0 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.595+0000 7f262a1ab640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 0x7f26240c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:05.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.596+0000 7f26299aa640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f2624159100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2620009870 tx=0x7f262002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d1002b0180c7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.596+0000 7f26289a8640 1 -- 172.21.15.17:0/3456676681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200317c0 con 0x7f26241527b0 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.596+0000 7f26289a8640 1 -- 172.21.15.17:0/3456676681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620037070 con 0x7f26241527b0 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.596+0000 7f26289a8640 1 -- 172.21.15.17:0/3456676681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620031b80 con 0x7f26241527b0 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 -- 172.21.15.17:0/3456676681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 msgr2=0x7f2624159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f2624159100 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2620009870 tx=0x7f262002b860 comp rx=0 tx=0).stop 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 -- 172.21.15.17:0/3456676681 shutdown_connections 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c88b0 0x7f26240c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 --2- 172.21.15.17:0/3456676681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f2624159100 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 -- 172.21.15.17:0/3456676681 >> 172.21.15.17:0/3456676681 conn(0x7f26240ac560 msgr2=0x7f26240ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 -- 172.21.15.17:0/3456676681 shutdown_connections 2024-03-06T07:22:05.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.597+0000 7f262bc0e640 1 -- 172.21.15.17:0/3456676681 wait complete. 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.598+0000 7f262bc0e640 1 Processor -- start 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.598+0000 7f262bc0e640 1 -- start start 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.599+0000 7f262bc0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c7180 0x7f26240c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26240c88b0 0x7f2624154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624155600 con 0x7f26241527b0 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624155320 con 0x7f26240c88b0 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c8d60 con 0x7f26240c7180 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35824/0 (socket says 172.21.15.17:35824) 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 -- 172.21.15.17:0/2060305577 learned_addr learned my addr 172.21.15.17:0/2060305577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f26299aa640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c7180 0x7f26240c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f26291a9640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26240c88b0 0x7f2624154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 -- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c7180 msgr2=0x7f26240c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c7180 0x7f26240c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 -- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26240c88b0 msgr2=0x7f2624154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26240c88b0 0x7f2624154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 -- 172.21.15.17:0/2060305577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2624157f20 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262a1ab640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f261c00cef0 tx=0x7f261c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c01a070 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c00d860 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2624158200 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c00dc50 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.600+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624157190 con 0x7f26241527b0 2024-03-06T07:22:05.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.601+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262404f330 con 0x7f26241527b0 2024-03-06T07:22:05.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.602+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f261c01d070 con 0x7f26241527b0 2024-03-06T07:22:05.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.602+0000 7f261a7fc640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:05.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.602+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7101+0+0 (secure 0 0 0) 0x7f261c08af60 con 0x7f26241527b0 2024-03-06T07:22:05.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.603+0000 7f26299aa640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:05.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.603+0000 7f26299aa640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2620036000 tx=0x7f262003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:05.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.606+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f261c0599c0 con 0x7f26241527b0 2024-03-06T07:22:05.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.921+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26240b0380 con 0x7f26241527b0 2024-03-06T07:22:05.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.921+0000 7f261a7fc640 1 -- 172.21.15.17:0/2060305577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f261c0592c0 con 0x7f26241527b0 2024-03-06T07:22:05.923 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:05.923 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:04.975870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5587},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"d","rank":0,"incarnation":38,"state":"up:replay","state_seq":7,"addr":"172.21.15.163:6836/673661445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":673661445},{"type":"v1","addr":"172.21.15.163:6836","nonce":673661445}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:05.923 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 38 2024-03-06T07:22:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 msgr2=0x7f260806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2620036000 tx=0x7f262003d040 comp rx=0 tx=0).stop 2024-03-06T07:22:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 msgr2=0x7f262414b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f261c00cef0 tx=0x7f261c00a710 comp rx=0 tx=0).stop 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 shutdown_connections 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26241527b0 0x7f262414b9b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26240c88b0 0x7f2624154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 --2- 172.21.15.17:0/2060305577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26240c7180 0x7f26240c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.923+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 >> 172.21.15.17:0/2060305577 conn(0x7f26240ac560 msgr2=0x7f2624158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.924+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 shutdown_connections 2024-03-06T07:22:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:05.924+0000 7f262bc0e640 1 -- 172.21.15.17:0/2060305577 wait complete. 2024-03-06T07:22:05.936 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-03-06T07:22:08.023 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:22:08.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:08.568 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:10.937 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:11.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.041+0000 7f608931e640 1 Processor -- start 2024-03-06T07:22:11.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.041+0000 7f608931e640 1 -- start start 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f608931e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6084153970 0x7f60840abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f608931e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60840a9450 0x7f60840a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f608931e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60840ac480 con 0x7f60841503c0 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f608931e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6084153e70 con 0x7f6084153970 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f608931e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60840c33b0 con 0x7f60840a9450 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.042+0000 7f60827fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6084153970 0x7f60840abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6084153970 0x7f60840abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34916/0 (socket says 172.21.15.17:34916) 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 -- 172.21.15.17:0/3989392207 learned_addr learned my addr 172.21.15.17:0/3989392207 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60837fe640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60840a9450 0x7f60840a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 -- 172.21.15.17:0/3989392207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60840a9450 msgr2=0x7f60840a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:11.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60840a9450 0x7f60840a98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 -- 172.21.15.17:0/3989392207 >> v1:172.21.15.163:6789/0 conn(0x7f60841503c0 legacy=0x7f60840ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.043+0000 7f60827fc640 1 -- 172.21.15.17:0/3989392207 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60840c3690 con 0x7f6084153970 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f60827fc640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6084153970 0x7f60840abf40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c1ac8ef149b650 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f6081ffb640 1 -- 172.21.15.17:0/3989392207 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607402ca70 con 0x7f6084153970 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f6081ffb640 1 -- 172.21.15.17:0/3989392207 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607402cc10 con 0x7f6084153970 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f6081ffb640 1 -- 172.21.15.17:0/3989392207 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60740343f0 con 0x7f6084153970 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 -- 172.21.15.17:0/3989392207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6084153970 msgr2=0x7f60840abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6084153970 0x7f60840abf40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 -- 172.21.15.17:0/3989392207 shutdown_connections 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60840a9450 0x7f60840a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 --2- 172.21.15.17:0/3989392207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6084153970 0x7f60840abf40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 -- 172.21.15.17:0/3989392207 >> 172.21.15.17:0/3989392207 conn(0x7f608406bf00 msgr2=0x7f60840c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 -- 172.21.15.17:0/3989392207 shutdown_connections 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.044+0000 7f608931e640 1 -- 172.21.15.17:0/3989392207 wait complete. 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 Processor -- start 2024-03-06T07:22:11.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 -- start start 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60841503c0 0x7f60841426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 0x7f6084142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084150920 con 0x7f60840a9450 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60841507a0 con 0x7f6084153970 2024-03-06T07:22:11.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f608931e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608406eda0 con 0x7f60841503c0 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f6082ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f6082ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34924/0 (socket says 172.21.15.17:34924) 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.046+0000 7f6082ffd640 1 -- 172.21.15.17:0/1333439493 learned_addr learned my addr 172.21.15.17:0/1333439493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f60837fe640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 0x7f6084142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f60827fc640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60841503c0 0x7f60841426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 -- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60841503c0 msgr2=0x7f60841426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60841503c0 0x7f60841426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 -- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 msgr2=0x7f6084142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 0x7f6084142be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 -- 172.21.15.17:0/1333439493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084143120 con 0x7f60840a9450 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f60837fe640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 0x7f6084142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f6082ffd640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c430 tx=0x7f606c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0052d0 con 0x7f60840a9450 2024-03-06T07:22:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c005490 con 0x7f60840a9450 2024-03-06T07:22:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60841ee140 con 0x7f60840a9450 2024-03-06T07:22:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c007260 con 0x7f60840a9450 2024-03-06T07:22:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.047+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60841ee700 con 0x7f60840a9450 2024-03-06T07:22:11.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.048+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608404f330 con 0x7f60840a9450 2024-03-06T07:22:11.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.049+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f606c007400 con 0x7f60840a9450 2024-03-06T07:22:11.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.050+0000 7f607b7fe640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.050+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f606c089c60 con 0x7f60840a9450 2024-03-06T07:22:11.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.050+0000 7f60827fc640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.050+0000 7f60827fc640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 0x7f606406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6074035000 tx=0x7f6074030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.054+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f60840ac480 con 0x7f60840a9450 2024-03-06T07:22:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.397+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6084068460 con 0x7f60840a9450 2024-03-06T07:22:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.398+0000 7f607b7fe640 1 -- 172.21.15.17:0/1333439493 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f606c0585a0 con 0x7f60840a9450 2024-03-06T07:22:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:11.400 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:10.991159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/2394960697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2394960697},{"type":"v1","addr":"172.21.15.17:6835","nonce":2394960697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:11.400 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 45 2024-03-06T07:22:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 msgr2=0x7f606406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 0x7f606406d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6074035000 tx=0x7f6074030000 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 msgr2=0x7f6084071770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c430 tx=0x7f606c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 shutdown_connections 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6084153970 0x7f6084142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60841503c0 0x7f60841426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.400+0000 7f608931e640 1 --2- 172.21.15.17:0/1333439493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60840a9450 0x7f6084071770 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.401+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 >> 172.21.15.17:0/1333439493 conn(0x7f608406bf00 msgr2=0x7f608414c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.401+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 shutdown_connections 2024-03-06T07:22:11.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.401+0000 7f608931e640 1 -- 172.21.15.17:0/1333439493 wait complete. 2024-03-06T07:22:11.413 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:22:11.414 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:22:11.414 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:11.462 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:11.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:22:11.507 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:22:11.507 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:11.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:22:11.522 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:22:11.522 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:11.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:22:11.564 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:11.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 Processor -- start 2024-03-06T07:22:11.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 -- start start 2024-03-06T07:22:11.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4a44156ea0 0x7f4a440702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a44070800 con 0x7f4a440acc00 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a44157320 con 0x7f4a44156ea0 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.664+0000 7f4a4a102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44156450 con 0x7f4a441506b0 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4a44156ea0 0x7f4a440702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a43fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4a440acc00 0x7f4a440c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:39298/0 (socket says 172.21.15.17:39298) 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:45106/0 (socket says 172.21.15.17:45106) 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4a44156ea0 0x7f4a440702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50308/0 (socket says 172.21.15.17:50308) 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a43fff640 1 -- 172.21.15.17:0/4184600257 learned_addr learned my addr 172.21.15.17:0/4184600257 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 -- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 msgr2=0x7f4a44153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 -- 172.21.15.17:0/4184600257 >> v1:172.21.15.163:6789/0 conn(0x7f4a440acc00 legacy=0x7f4a440c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a427fc640 1 -- 172.21.15.17:0/4184600257 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199351529 0 0) 0x7f4a44070800 con 0x7f4a440acc00 2024-03-06T07:22:11.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a42ffd640 1 -- 172.21.15.17:0/4184600257 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44070aa0 con 0x7f4a44156ea0 2024-03-06T07:22:11.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.665+0000 7f4a437fe640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:11.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.666+0000 7f4a42ffd640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a44156ea0 0x7f4a440702c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3322b9be0d16a2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.666+0000 7f4a427fc640 1 -- 172.21.15.17:0/4184600257 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38035020 con 0x7f4a44156ea0 2024-03-06T07:22:11.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.666+0000 7f4a427fc640 1 -- 172.21.15.17:0/4184600257 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38031830 con 0x7f4a44156ea0 2024-03-06T07:22:11.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.666+0000 7f4a427fc640 1 -- 172.21.15.17:0/4184600257 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38031c00 con 0x7f4a44156ea0 2024-03-06T07:22:11.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 -- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a44156ea0 msgr2=0x7f4a440702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a44156ea0 0x7f4a440702c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 -- 172.21.15.17:0/4184600257 shutdown_connections 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a44156ea0 0x7f4a440702c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 --2- 172.21.15.17:0/4184600257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a44153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 -- 172.21.15.17:0/4184600257 >> 172.21.15.17:0/4184600257 conn(0x7f4a4406b1a0 msgr2=0x7f4a4406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 -- 172.21.15.17:0/4184600257 shutdown_connections 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.667+0000 7f4a4a102640 1 -- 172.21.15.17:0/4184600257 wait complete. 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.668+0000 7f4a4a102640 1 Processor -- start 2024-03-06T07:22:11.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.668+0000 7f4a4a102640 1 -- start start 2024-03-06T07:22:11.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 0x7f4a440c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a440aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a441566a0 con 0x7f4a44156ea0 2024-03-06T07:22:11.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44156a10 con 0x7f4a440acc00 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a4a102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c5bb0 con 0x7f4a441506b0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34934/0 (socket says 172.21.15.17:34934) 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 -- 172.21.15.17:0/2916752439 learned_addr learned my addr 172.21.15.17:0/2916752439 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a437fe640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 0x7f4a440c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a42ffd640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a440aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 -- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 msgr2=0x7f4a440aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a440aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 -- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 msgr2=0x7f4a440c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 0x7f4a440c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 -- 172.21.15.17:0/2916752439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c4560 con 0x7f4a44156ea0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a437fe640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 0x7f4a440c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.669+0000 7f4a43fff640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00ca70 tx=0x7f4a3c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.670+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00abd0 con 0x7f4a44156ea0 2024-03-06T07:22:11.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.670+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00ad70 con 0x7f4a44156ea0 2024-03-06T07:22:11.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.670+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440c4840 con 0x7f4a44156ea0 2024-03-06T07:22:11.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.670+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c012640 con 0x7f4a44156ea0 2024-03-06T07:22:11.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.670+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440c22a0 con 0x7f4a44156ea0 2024-03-06T07:22:11.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.671+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4404f330 con 0x7f4a44156ea0 2024-03-06T07:22:11.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.672+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a3c0127e0 con 0x7f4a44156ea0 2024-03-06T07:22:11.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.672+0000 7f4a2bfff640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 0x7f4a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:11.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.672+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f4a3c08ab70 con 0x7f4a44156ea0 2024-03-06T07:22:11.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.672+0000 7f4a437fe640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 0x7f4a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:11.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.673+0000 7f4a437fe640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 0x7f4a3006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009560 tx=0x7f4a34009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:11.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.677+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a3c058e60 con 0x7f4a44156ea0 2024-03-06T07:22:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.904+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a440700a0 con 0x7f4a44156ea0 2024-03-06T07:22:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.904+0000 7f4a2bfff640 1 -- 172.21.15.17:0/2916752439 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f4a3c05de20 con 0x7f4a44156ea0 2024-03-06T07:22:11.906 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:11.906 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:10.991159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/2394960697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2394960697},{"type":"v1","addr":"172.21.15.17:6835","nonce":2394960697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 45 2024-03-06T07:22:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 msgr2=0x7f4a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 0x7f4a3006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009560 tx=0x7f4a34009190 comp rx=0 tx=0).stop 2024-03-06T07:22:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 msgr2=0x7f4a44146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00ca70 tx=0x7f4a3c00cf30 comp rx=0 tx=0).stop 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 shutdown_connections 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a3006b4d0 0x7f4a3006d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a44156ea0 0x7f4a44146a10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a441506b0 0x7f4a440aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 --2- 172.21.15.17:0/2916752439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a440acc00 0x7f4a440c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 >> 172.21.15.17:0/2916752439 conn(0x7f4a4406b1a0 msgr2=0x7f4a4414e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 shutdown_connections 2024-03-06T07:22:11.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:11.906+0000 7f4a4a102640 1 -- 172.21.15.17:0/2916752439 wait complete. 2024-03-06T07:22:11.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:12.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.020+0000 7fe114edf640 1 Processor -- start 2024-03-06T07:22:12.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.020+0000 7fe114edf640 1 -- start start 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1101506b0 0x7fe110153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 0x7fe1100702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe1100c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe110156450 con 0x7fe1101506b0 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe110156a10 con 0x7fe110156ea0 2024-03-06T07:22:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe114edf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe110070800 con 0x7fe1100acc00 2024-03-06T07:22:12.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1101506b0 0x7fe110153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1101506b0 0x7fe110153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34940/0 (socket says 172.21.15.17:34940) 2024-03-06T07:22:12.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 -- 172.21.15.17:0/3466763878 learned_addr learned my addr 172.21.15.17:0/3466763878 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10dd9b640 1 --2- 172.21.15.17:0/3466763878 >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 0x7fe1100702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10ed9d640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe1100c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 -- 172.21.15.17:0/3466763878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 msgr2=0x7fe1100c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe1100c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 -- 172.21.15.17:0/3466763878 >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 msgr2=0x7fe1100702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 --2- 172.21.15.17:0/3466763878 >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 0x7fe1100702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10e59c640 1 -- 172.21.15.17:0/3466763878 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110070aa0 con 0x7fe1101506b0 2024-03-06T07:22:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.022+0000 7fe10dd9b640 1 --2- 172.21.15.17:0/3466763878 >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 0x7fe1100702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.023+0000 7fe10e59c640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1101506b0 0x7fe110153990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bf40 tx=0x7fe10002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b22df65b0683029 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.023+0000 7fe10d59a640 1 -- 172.21.15.17:0/3466763878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100035020 con 0x7fe1101506b0 2024-03-06T07:22:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.023+0000 7fe10d59a640 1 -- 172.21.15.17:0/3466763878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100031870 con 0x7fe1101506b0 2024-03-06T07:22:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.023+0000 7fe10d59a640 1 -- 172.21.15.17:0/3466763878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100031c40 con 0x7fe1101506b0 2024-03-06T07:22:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 -- 172.21.15.17:0/3466763878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1101506b0 msgr2=0x7fe110153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1101506b0 0x7fe110153990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bf40 tx=0x7fe10002b910 comp rx=0 tx=0).stop 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 -- 172.21.15.17:0/3466763878 shutdown_connections 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe1100c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 --2- 172.21.15.17:0/3466763878 >> v2:172.21.15.163:3300/0 conn(0x7fe110156ea0 0x7fe1100702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 --2- 172.21.15.17:0/3466763878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1101506b0 0x7fe110153990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 -- 172.21.15.17:0/3466763878 >> 172.21.15.17:0/3466763878 conn(0x7fe11006b1a0 msgr2=0x7fe11006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 -- 172.21.15.17:0/3466763878 shutdown_connections 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.024+0000 7fe114edf640 1 -- 172.21.15.17:0/3466763878 wait complete. 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.025+0000 7fe114edf640 1 Processor -- start 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.025+0000 7fe114edf640 1 -- start start 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe110154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1101506b0 0x7fe1101ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c59e0 con 0x7fe110156ea0 2024-03-06T07:22:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1101566a0 con 0x7fe1101506b0 2024-03-06T07:22:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe114edf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c5b60 con 0x7fe1100acc00 2024-03-06T07:22:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34952/0 (socket says 172.21.15.17:34952) 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 -- 172.21.15.17:0/4124473653 learned_addr learned my addr 172.21.15.17:0/4124473653 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10dd9b640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1101506b0 0x7fe1101ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10e59c640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe110154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 -- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 msgr2=0x7fe110154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe110154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 -- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1101506b0 msgr2=0x7fe1101ee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1101506b0 0x7fe1101ee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 -- 172.21.15.17:0/4124473653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110154830 con 0x7fe110156ea0 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10e59c640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe110154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.026+0000 7fe10ed9d640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe108007d10 tx=0x7fe10800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.027+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108014420 con 0x7fe110156ea0 2024-03-06T07:22:12.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.027+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108019070 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.027+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11006dbd0 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.027+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108014820 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.027+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe110154bf0 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.028+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe110154df0 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.028+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1080149c0 con 0x7fe110156ea0 2024-03-06T07:22:12.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.029+0000 7fe0feffd640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.029+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fe10808af90 con 0x7fe110156ea0 2024-03-06T07:22:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.029+0000 7fe10e59c640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.029+0000 7fe10e59c640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 0x7fe0f006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe100036000 tx=0x7fe100008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.037+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe110154df0 con 0x7fe110156ea0 2024-03-06T07:22:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.225+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe11004f330 con 0x7fe110156ea0 2024-03-06T07:22:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.226+0000 7fe0feffd640 1 -- 172.21.15.17:0/4124473653 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+21021 (secure 0 0 0) 0x7fe1080598d0 con 0x7fe110156ea0 2024-03-06T07:22:12.228 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:12.228 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":59,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:07.986486+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 msgr2=0x7fe0f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 0x7fe0f006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe100036000 tx=0x7fe100008030 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 msgr2=0x7fe1100c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe108007d10 tx=0x7fe10800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 shutdown_connections 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe110156ea0 0x7fe1100c1d60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1101506b0 0x7fe1101ee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 --2- 172.21.15.17:0/4124473653 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1100acc00 0x7fe110154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 >> 172.21.15.17:0/4124473653 conn(0x7fe11006b1a0 msgr2=0x7fe1100c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 shutdown_connections 2024-03-06T07:22:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.229+0000 7fe114edf640 1 -- 172.21.15.17:0/4124473653 wait complete. 2024-03-06T07:22:12.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:21:41.533584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:21:42.234700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:21:57.664157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:21:58.336311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:12.244 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.347+0000 7faa7ad13640 1 Processor -- start 2024-03-06T07:22:12.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.347+0000 7faa7ad13640 1 -- start start 2024-03-06T07:22:12.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa740a8290 0x7faa741539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa740701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa740c69a0 0x7faa74072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa74153f00 con 0x7faa740a8290 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa740c62a0 con 0x7faa740c69a0 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa7ad13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740706e0 con 0x7faa740c4740 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa78aaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa740a8290 0x7faa741539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa78aaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faa740a8290 0x7faa741539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34958/0 (socket says 172.21.15.17:34958) 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.348+0000 7faa78aaf640 1 -- 172.21.15.17:0/225982620 learned_addr learned my addr 172.21.15.17:0/225982620 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa73fff640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa740701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa792b0640 1 --2- 172.21.15.17:0/225982620 >> v2:172.21.15.163:3300/0 conn(0x7faa740c69a0 0x7faa74072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa78aaf640 1 -- 172.21.15.17:0/225982620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 msgr2=0x7faa740701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa78aaf640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa740701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa78aaf640 1 -- 172.21.15.17:0/225982620 >> v2:172.21.15.163:3300/0 conn(0x7faa740c69a0 msgr2=0x7faa74072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa78aaf640 1 --2- 172.21.15.17:0/225982620 >> v2:172.21.15.163:3300/0 conn(0x7faa740c69a0 0x7faa74072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.349+0000 7faa78aaf640 1 -- 172.21.15.17:0/225982620 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa740c4c00 con 0x7faa740a8290 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa78aaf640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa741539c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa74157570 tx=0x7faa5c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f57d8a7834400f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa737fe640 1 -- 172.21.15.17:0/225982620 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c038070 con 0x7faa740a8290 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa737fe640 1 -- 172.21.15.17:0/225982620 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c02f050 con 0x7faa740a8290 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa737fe640 1 -- 172.21.15.17:0/225982620 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c031970 con 0x7faa740a8290 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 -- 172.21.15.17:0/225982620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 msgr2=0x7faa741539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa741539c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7faa74157570 tx=0x7faa5c02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 -- 172.21.15.17:0/225982620 shutdown_connections 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 --2- 172.21.15.17:0/225982620 >> v2:172.21.15.163:3300/0 conn(0x7faa740c69a0 0x7faa74072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa740701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 --2- 172.21.15.17:0/225982620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa741539c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 -- 172.21.15.17:0/225982620 >> 172.21.15.17:0/225982620 conn(0x7faa740c5560 msgr2=0x7faa740c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 -- 172.21.15.17:0/225982620 shutdown_connections 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.350+0000 7faa7ad13640 1 -- 172.21.15.17:0/225982620 wait complete. 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.352+0000 7faa7ad13640 1 Processor -- start 2024-03-06T07:22:12.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.352+0000 7faa7ad13640 1 -- start start 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa741dd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 0x7faa741dd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7406fc80 con 0x7faa740a8290 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7406f9a0 con 0x7faa740c69a0 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa7ad13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c6e10 con 0x7faa740c4740 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34968/0 (socket says 172.21.15.17:34968) 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 -- 172.21.15.17:0/3998498217 learned_addr learned my addr 172.21.15.17:0/3998498217 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa792b0640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 0x7faa741dd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa73fff640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa741dd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 -- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 msgr2=0x7faa741dd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa741dd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 -- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 msgr2=0x7faa741dd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.353+0000 7faa78aaf640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 0x7faa741dd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa78aaf640 1 -- 172.21.15.17:0/3998498217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa741dc580 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa792b0640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 0x7faa741dd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa73fff640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa741dd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa78aaf640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faa5c02b830 tx=0x7faa5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c042070 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c03d680 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa741e95b0 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c038040 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.354+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741dcb00 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.355+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7404f330 con 0x7faa740a8290 2024-03-06T07:22:12.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.355+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa5c045070 con 0x7faa740a8290 2024-03-06T07:22:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.356+0000 7faa717fa640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.356+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7faa5c0ac300 con 0x7faa740a8290 2024-03-06T07:22:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.356+0000 7faa73fff640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.357+0000 7faa73fff640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 0x7faa5406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa740be9b0 tx=0x7faa6400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.360+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa5c07ad60 con 0x7faa740a8290 2024-03-06T07:22:12.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.541+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa7406e6c0 con 0x7faa740a8290 2024-03-06T07:22:12.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.541+0000 7faa717fa640 1 -- 172.21.15.17:0/3998498217 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa5c07a660 con 0x7faa740a8290 2024-03-06T07:22:12.543 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:22:12.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 msgr2=0x7faa5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 0x7faa5406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa740be9b0 tx=0x7faa6400b040 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 msgr2=0x7faa740adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7faa5c02b830 tx=0x7faa5c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 shutdown_connections 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faa5406b4d0 0x7faa5406d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa740c69a0 0x7faa741dd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa740c4740 0x7faa741dd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 --2- 172.21.15.17:0/3998498217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa740a8290 0x7faa740adce0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 >> 172.21.15.17:0/3998498217 conn(0x7faa740c5560 msgr2=0x7faa740c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 shutdown_connections 2024-03-06T07:22:12.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.544+0000 7faa7ad13640 1 -- 172.21.15.17:0/3998498217 wait complete. 2024-03-06T07:22:12.556 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:22:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.663+0000 7f644193e640 1 Processor -- start 2024-03-06T07:22:12.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.663+0000 7f644193e640 1 -- start start 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f644193e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f644193e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f643c0c69a0 0x7f643c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f644193e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c0c62a0 con 0x7f643c0c4740 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f644193e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c153f00 con 0x7f643c0c69a0 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f644193e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c4c00 con 0x7f643c0a8290 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f643c0c69a0 0x7f643c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f643c0c69a0 0x7f643c072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34974/0 (socket says 172.21.15.17:34974) 2024-03-06T07:22:12.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 -- 172.21.15.17:0/333577657 learned_addr learned my addr 172.21.15.17:0/333577657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643affd640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 -- 172.21.15.17:0/333577657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 msgr2=0x7f643c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 -- 172.21.15.17:0/333577657 >> v1:172.21.15.163:6789/0 conn(0x7f643c0c4740 legacy=0x7f643c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.664+0000 7f643b7fe640 1 -- 172.21.15.17:0/333577657 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0c4ee0 con 0x7f643c0c69a0 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.665+0000 7f643b7fe640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c072680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7535473e25132e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.665+0000 7f6439ffb640 1 -- 172.21.15.17:0/333577657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643402c9e0 con 0x7f643c0c69a0 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.665+0000 7f6439ffb640 1 -- 172.21.15.17:0/333577657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643402cb80 con 0x7f643c0c69a0 2024-03-06T07:22:12.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.665+0000 7f6439ffb640 1 -- 172.21.15.17:0/333577657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340343f0 con 0x7f643c0c69a0 2024-03-06T07:22:12.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 -- 172.21.15.17:0/333577657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 msgr2=0x7f643c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c072680 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6434009e50 tx=0x7f643402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:12.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 -- 172.21.15.17:0/333577657 shutdown_connections 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c072680 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 --2- 172.21.15.17:0/333577657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 -- 172.21.15.17:0/333577657 >> 172.21.15.17:0/333577657 conn(0x7f643c0c5560 msgr2=0x7f643c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 -- 172.21.15.17:0/333577657 shutdown_connections 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.666+0000 7f644193e640 1 -- 172.21.15.17:0/333577657 wait complete. 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.668+0000 7f644193e640 1 Processor -- start 2024-03-06T07:22:12.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.668+0000 7f644193e640 1 -- start start 2024-03-06T07:22:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1dd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c0c4740 0x7f643c141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0c6e50 con 0x7f643c0c69a0 2024-03-06T07:22:12.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c06fc80 con 0x7f643c0c4740 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f644193e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c072bc0 con 0x7f643c0a8290 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f643b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f643b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34976/0 (socket says 172.21.15.17:34976) 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f643b7fe640 1 -- 172.21.15.17:0/893513767 learned_addr learned my addr 172.21.15.17:0/893513767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.669+0000 7f643a7fc640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c0c4740 0x7f643c141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643affd640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 -- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 msgr2=0x7f643c1dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1dd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 -- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c0c4740 msgr2=0x7f643c141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c0c4740 0x7f643c141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 -- 172.21.15.17:0/893513767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c1ec620 con 0x7f643c0c69a0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f643b7fe640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6434036040 tx=0x7f6434004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434043020 con 0x7f643c0c69a0 2024-03-06T07:22:12.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643403e450 con 0x7f643c0c69a0 2024-03-06T07:22:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c1ec900 con 0x7f643c0c69a0 2024-03-06T07:22:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434009510 con 0x7f643c0c69a0 2024-03-06T07:22:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.670+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c1ecec0 con 0x7f643c0c69a0 2024-03-06T07:22:12.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.671+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c04f330 con 0x7f643c0c69a0 2024-03-06T07:22:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.672+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6434046020 con 0x7f643c0c69a0 2024-03-06T07:22:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.673+0000 7f6423fff640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 0x7f642806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.673+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f64340ace30 con 0x7f643c0c69a0 2024-03-06T07:22:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.673+0000 7f643affd640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 0x7f642806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.674+0000 7f643affd640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 0x7f642806d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f642c006fd0 tx=0x7f642c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.679+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f643404d030 con 0x7f643c0c69a0 2024-03-06T07:22:12.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.863+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f643c0735e0 con 0x7f643c0c69a0 2024-03-06T07:22:12.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.863+0000 7f6423fff640 1 -- 172.21.15.17:0/893513767 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64340ac020 con 0x7f643c0c69a0 2024-03-06T07:22:12.865 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 msgr2=0x7f642806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 0x7f642806d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f642c006fd0 tx=0x7f642c008040 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 msgr2=0x7f643c142180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6434036040 tx=0x7f6434004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 shutdown_connections 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f642806b4d0 0x7f642806d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643c0c69a0 0x7f643c142180 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c0c4740 0x7f643c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 --2- 172.21.15.17:0/893513767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f643c0a8290 0x7f643c1dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 >> 172.21.15.17:0/893513767 conn(0x7f643c0c5560 msgr2=0x7f643c0aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 shutdown_connections 2024-03-06T07:22:12.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.866+0000 7f644193e640 1 -- 172.21.15.17:0/893513767 wait complete. 2024-03-06T07:22:12.881 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:22:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.984+0000 7f72a99ac640 1 Processor -- start 2024-03-06T07:22:12.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 -- start start 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72a4155c30 0x7f72a406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f72a40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40b0450 con 0x7f72a40b0070 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a406cfb0 con 0x7f72a4155c30 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a99ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a406c020 con 0x7f72a40c5380 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f72a40b0070 0x7f72a40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41796/0 (socket says 172.21.15.17:41796) 2024-03-06T07:22:12.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.985+0000 7f72a2ffd640 1 -- 172.21.15.17:0/3690841038 learned_addr learned my addr 172.21.15.17:0/3690841038 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> v2:172.21.15.163:3300/0 conn(0x7f72a4155c30 0x7f72a406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a37fe640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f72a40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45075525 0 0) 0x7f7294001110 con 0x7f72a40b0070 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728c003520 con 0x7f72a40b0070 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794424572 0 0) 0x7f728c003520 con 0x7f72a40b0070 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 msgr2=0x7f72a40c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f72a40c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 >> v2:172.21.15.163:3300/0 conn(0x7f72a4155c30 msgr2=0x7f72a406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 --2- 172.21.15.17:0/3690841038 >> v2:172.21.15.163:3300/0 conn(0x7f72a4155c30 0x7f72a406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a41560e0 con 0x7f72a40b0070 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7294001840 con 0x7f72a40b0070 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> v2:172.21.15.163:3300/0 conn(0x7f72a4155c30 0x7f72a406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 >> v1:172.21.15.17:6789/0 conn(0x7f72a40b0070 legacy=0x7f72a40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f728c0039b0 0x7f728c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f728c004800 0x7f728c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f728c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a406cfb0 con 0x7f728c004800 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a406c020 con 0x7f728c0039b0 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c00b9c0 con 0x7f72a40c5380 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f728c004800 0x7f728c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a2ffd640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f728c0039b0 0x7f728c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a37fe640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f728c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 -- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 msgr2=0x7f728c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a40c5380 0x7f728c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 -- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f728c0039b0 msgr2=0x7f728c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.986+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f728c0039b0 0x7f728c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.987+0000 7f72a27fc640 1 -- 172.21.15.17:0/3690841038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c003520 con 0x7f728c004800 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.987+0000 7f72a27fc640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f728c004800 0x7f728c004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7298009cc0 tx=0x7f728c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1629c62dc87d2baf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.987+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298039070 con 0x7f728c004800 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.987+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729802c960 con 0x7f728c004800 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.987+0000 7f72a1ffb640 1 -- 172.21.15.17:0/3690841038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009860 con 0x7f728c004800 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 -- 172.21.15.17:0/3690841038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f728c004800 msgr2=0x7f728c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 --2- 172.21.15.17:0/3690841038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f728c004800 0x7f728c004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7298009cc0 tx=0x7f728c005830 comp rx=0 tx=0).stop 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a37fe640 1 -- 172.21.15.17:0/3690841038 reap_dead start 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 -- 172.21.15.17:0/3690841038 shutdown_connections 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 -- 172.21.15.17:0/3690841038 >> 172.21.15.17:0/3690841038 conn(0x7f72a40ac590 msgr2=0x7f72a40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 -- 172.21.15.17:0/3690841038 shutdown_connections 2024-03-06T07:22:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.988+0000 7f72a99ac640 1 -- 172.21.15.17:0/3690841038 wait complete. 2024-03-06T07:22:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.989+0000 7f72a99ac640 1 Processor -- start 2024-03-06T07:22:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.989+0000 7f72a99ac640 1 -- start start 2024-03-06T07:22:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f728c004800 0x7f72a41ee440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a41ee980 0x7f72a414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a406c270 con 0x7f72a40c16d0 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40c7ad0 con 0x7f72a41ee980 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a99ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a406c550 con 0x7f728c004800 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34986/0 (socket says 172.21.15.17:34986) 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.990+0000 7f72a27fc640 1 -- 172.21.15.17:0/321083671 learned_addr learned my addr 172.21.15.17:0/321083671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a37fe640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a41ee980 0x7f72a414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a2ffd640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f728c004800 0x7f72a41ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 -- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f728c004800 msgr2=0x7f72a41ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f728c004800 0x7f72a41ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 -- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a41ee980 msgr2=0x7f72a414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a41ee980 0x7f72a414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:12.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 -- 172.21.15.17:0/321083671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a41eeef0 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a27fc640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7298009cc0 tx=0x7f7298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298039050 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729803a040 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009860 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a40c28e0 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.991+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a40c3c60 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.992+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a40af7d0 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.993+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7298047020 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.993+0000 7f728bfff640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 0x7f729006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.993+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f72980abfa0 con 0x7f72a40c16d0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.993+0000 7f72a2ffd640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 0x7f729006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.994+0000 7f72a2ffd640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 0x7f729006d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7294001eb0 tx=0x7f7294007ac0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:12.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:12.997+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72a40af7d0 con 0x7f72a40c16d0 2024-03-06T07:22:13.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.180+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72a40cb330 con 0x7f72a40c16d0 2024-03-06T07:22:13.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.180+0000 7f728bfff640 1 -- 172.21.15.17:0/321083671 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72a40cb330 con 0x7f72a40c16d0 2024-03-06T07:22:13.182 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:22:13.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.182+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 msgr2=0x7f729006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.182+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 0x7f729006d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7294001eb0 tx=0x7f7294007ac0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.182+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 msgr2=0x7f72a414f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.182+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7298009cc0 tx=0x7f7298004730 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 shutdown_connections 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f729006b4d0 0x7f729006d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a41ee980 0x7f72a414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a40c16d0 0x7f72a414f450 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 --2- 172.21.15.17:0/321083671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f728c004800 0x7f72a41ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 >> 172.21.15.17:0/321083671 conn(0x7f72a40ac590 msgr2=0x7f72a40adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 shutdown_connections 2024-03-06T07:22:13.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.183+0000 7f72a99ac640 1 -- 172.21.15.17:0/321083671 wait complete. 2024-03-06T07:22:13.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:22:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.297+0000 7fd07e516640 1 Processor -- start 2024-03-06T07:22:13.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.297+0000 7fd07e516640 1 -- start start 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd07e516640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd0780b0070 0x7fd078155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd07e516640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0780b0450 con 0x7fd07806ca70 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd07e516640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07806cf20 con 0x7fd0780c5380 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd07e516640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07806c020 con 0x7fd0780b0070 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd0777fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd07806ca70 0x7fd0780c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41800/0 (socket says 172.21.15.17:41800) 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.298+0000 7fd0777fe640 1 -- 172.21.15.17:0/3233376429 learned_addr learned my addr 172.21.15.17:0/3233376429 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd077fff640 1 --2- 172.21.15.17:0/3233376429 >> v2:172.21.15.163:3300/0 conn(0x7fd0780b0070 0x7fd078155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd076ffd640 1 -- 172.21.15.17:0/3233376429 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021567911 0 0) 0x7fd068001110 con 0x7fd07806ca70 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd076ffd640 1 -- 172.21.15.17:0/3233376429 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd060003520 con 0x7fd07806ca70 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd077fff640 1 -- 172.21.15.17:0/3233376429 >> v1:172.21.15.163:6789/0 conn(0x7fd0780c5380 legacy=0x7fd07814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd077fff640 1 -- 172.21.15.17:0/3233376429 >> v1:172.21.15.17:6789/0 conn(0x7fd07806ca70 legacy=0x7fd0780c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.299+0000 7fd077fff640 1 -- 172.21.15.17:0/3233376429 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780c77f0 con 0x7fd0780b0070 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.300+0000 7fd077fff640 1 --2- 172.21.15.17:0/3233376429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0780b0070 0x7fd078155c30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097a0 tx=0x7fd06c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8819088c26cff1d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.300+0000 7fd076ffd640 1 -- 172.21.15.17:0/3233376429 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c035020 con 0x7fd0780b0070 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.300+0000 7fd076ffd640 1 -- 172.21.15.17:0/3233376429 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c031830 con 0x7fd0780b0070 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.300+0000 7fd076ffd640 1 -- 172.21.15.17:0/3233376429 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c031be0 con 0x7fd0780b0070 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 -- 172.21.15.17:0/3233376429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0780b0070 msgr2=0x7fd078155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 --2- 172.21.15.17:0/3233376429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0780b0070 0x7fd078155c30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097a0 tx=0x7fd06c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 -- 172.21.15.17:0/3233376429 shutdown_connections 2024-03-06T07:22:13.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 --2- 172.21.15.17:0/3233376429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0780b0070 0x7fd078155c30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 -- 172.21.15.17:0/3233376429 >> 172.21.15.17:0/3233376429 conn(0x7fd0780ac590 msgr2=0x7fd0780ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 -- 172.21.15.17:0/3233376429 shutdown_connections 2024-03-06T07:22:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.301+0000 7fd07e516640 1 -- 172.21.15.17:0/3233376429 wait complete. 2024-03-06T07:22:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.302+0000 7fd07e516640 1 Processor -- start 2024-03-06T07:22:13.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.302+0000 7fd07e516640 1 -- start start 2024-03-06T07:22:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd07806ca70 0x7fd0780c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 0x7fd07814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07806c550 con 0x7fd0780b0070 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07806c270 con 0x7fd07806ca70 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.303+0000 7fd07e516640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07806da80 con 0x7fd0780c5380 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34994/0 (socket says 172.21.15.17:34994) 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 -- 172.21.15.17:0/789261242 learned_addr learned my addr 172.21.15.17:0/789261242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd077fff640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd07806ca70 0x7fd0780c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd07cab3640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 0x7fd07814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 -- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 msgr2=0x7fd07814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 0x7fd07814b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 -- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd07806ca70 msgr2=0x7fd0780c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd07806ca70 0x7fd0780c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 -- 172.21.15.17:0/789261242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780c2150 con 0x7fd0780b0070 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd07cab3640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 0x7fd07814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd0777fe640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd06800c430 tx=0x7fd06800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068017070 con 0x7fd0780b0070 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068005ce0 con 0x7fd0780b0070 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0780c4750 con 0x7fd0780b0070 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680072d0 con 0x7fd0780b0070 2024-03-06T07:22:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.304+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0780c4d10 con 0x7fd0780b0070 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.306+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07804f330 con 0x7fd0780b0070 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.306+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd06801a070 con 0x7fd0780b0070 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.306+0000 7fd074ff9640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 0x7fd06406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.306+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7fd068089a00 con 0x7fd0780b0070 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.306+0000 7fd077fff640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 0x7fd06406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.307+0000 7fd077fff640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 0x7fd06406d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036000 tx=0x7fd06c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.311+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0780b0450 con 0x7fd0780b0070 2024-03-06T07:22:13.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.589+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd07806d860 con 0x7fd0780b0070 2024-03-06T07:22:13.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.589+0000 7fd074ff9640 1 -- 172.21.15.17:0/789261242 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v59) v1 ==== 105+0+11 (secure 0 0 0) 0x7fd068057cf0 con 0x7fd0780b0070 2024-03-06T07:22:13.591 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:22:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 msgr2=0x7fd06406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 0x7fd06406d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd06c036000 tx=0x7fd06c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 msgr2=0x7fd07814ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd06800c430 tx=0x7fd06800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 shutdown_connections 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd06406b500 0x7fd06406d9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0780c5380 0x7fd07814b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0780b0070 0x7fd07814ae40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 --2- 172.21.15.17:0/789261242 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd07806ca70 0x7fd0780c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 >> 172.21.15.17:0/789261242 conn(0x7fd0780ac590 msgr2=0x7fd078151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.591+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 shutdown_connections 2024-03-06T07:22:13.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.592+0000 7fd07e516640 1 -- 172.21.15.17:0/789261242 wait complete. 2024-03-06T07:22:13.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:22:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.708+0000 7f01f76d4640 1 Processor -- start 2024-03-06T07:22:13.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.708+0000 7f01f76d4640 1 -- start start 2024-03-06T07:22:13.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f76d4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01f00c1990 0x7f01f00c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f76d4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f0155ed0 con 0x7f01f00c12a0 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f76d4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f00c6c30 con 0x7f01f0152770 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f76d4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f00c6a00 con 0x7f01f00c1990 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f4c6f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01f00c12a0 0x7f01f00c7710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41814/0 (socket says 172.21.15.17:41814) 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f4c6f640 1 -- 172.21.15.17:0/2396109224 learned_addr learned my addr 172.21.15.17:0/2396109224 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.709+0000 7f01f5c71640 1 --2- 172.21.15.17:0/2396109224 >> v2:172.21.15.163:3300/0 conn(0x7f01f00c1990 0x7f01f00c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722289074 0 0) 0x7f01f0155ed0 con 0x7f01f00c12a0 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01dc003520 con 0x7f01f00c12a0 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331934862 0 0) 0x7f01f00c6c30 con 0x7f01f0152770 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f0155ed0 con 0x7f01f0152770 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01f5c71640 1 -- 172.21.15.17:0/2396109224 >> v1:172.21.15.163:6789/0 conn(0x7f01f0152770 legacy=0x7f01f006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01f5c71640 1 -- 172.21.15.17:0/2396109224 >> v1:172.21.15.17:6789/0 conn(0x7f01f00c12a0 legacy=0x7f01f00c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.710+0000 7f01f5c71640 1 -- 172.21.15.17:0/2396109224 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00c6f10 con 0x7f01f00c1990 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.711+0000 7f01f5c71640 1 --2- 172.21.15.17:0/2396109224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f00c7e20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009d90 tx=0x7f01e802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157ac49708167055 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.711+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8039070 con 0x7f01f00c1990 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.711+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e802cd00 con 0x7f01f00c1990 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.711+0000 7f01e3fff640 1 -- 172.21.15.17:0/2396109224 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8034660 con 0x7f01f00c1990 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 -- 172.21.15.17:0/2396109224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 msgr2=0x7f01f00c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 --2- 172.21.15.17:0/2396109224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f00c7e20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009d90 tx=0x7f01e802bf50 comp rx=0 tx=0).stop 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 -- 172.21.15.17:0/2396109224 shutdown_connections 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 --2- 172.21.15.17:0/2396109224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f00c7e20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 -- 172.21.15.17:0/2396109224 >> 172.21.15.17:0/2396109224 conn(0x7f01f00a94d0 msgr2=0x7f01f00a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 -- 172.21.15.17:0/2396109224 shutdown_connections 2024-03-06T07:22:13.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.712+0000 7f01f76d4640 1 -- 172.21.15.17:0/2396109224 wait complete. 2024-03-06T07:22:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.714+0000 7f01f76d4640 1 Processor -- start 2024-03-06T07:22:13.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.714+0000 7f01f76d4640 1 -- start start 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f00c12a0 0x7f01f014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f0151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0152b50 con 0x7f01f0152770 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00c1dc0 con 0x7f01f00c1990 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f76d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00c8360 con 0x7f01f00c12a0 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35010/0 (socket says 172.21.15.17:35010) 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 -- 172.21.15.17:0/1415101146 learned_addr learned my addr 172.21.15.17:0/1415101146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:13.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f4c6f640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f0151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5470640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f00c12a0 0x7f01f014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 -- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f00c12a0 msgr2=0x7f01f014cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f00c12a0 0x7f01f014cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 -- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 msgr2=0x7f01f0151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f0151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 -- 172.21.15.17:0/1415101146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0152e00 con 0x7f01f0152770 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.715+0000 7f01f5c71640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009d90 tx=0x7f01e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.716+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8039050 con 0x7f01f0152770 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.716+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e8045070 con 0x7f01f0152770 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.716+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8009450 con 0x7f01f0152770 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.716+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f01ee6f0 con 0x7f01f0152770 2024-03-06T07:22:13.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.716+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f01518c0 con 0x7f01f0152770 2024-03-06T07:22:13.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.717+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f00c6040 con 0x7f01f0152770 2024-03-06T07:22:13.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.717+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01e803e850 con 0x7f01f0152770 2024-03-06T07:22:13.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.718+0000 7f01e1ffb640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 0x7f01d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:13.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.718+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f01e80ad590 con 0x7f01f0152770 2024-03-06T07:22:13.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.718+0000 7f01f5470640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 0x7f01d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:13.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.719+0000 7f01f5470640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 0x7f01d406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:13.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.722+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f01f00c6040 con 0x7f01f0152770 2024-03-06T07:22:13.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.905+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f01f004f330 con 0x7f01f0152770 2024-03-06T07:22:13.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.906+0000 7f01e1ffb640 1 -- 172.21.15.17:0/1415101146 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v59) v1 ==== 101+0+11 (secure 0 0 0) 0x7f01e807b910 con 0x7f01f0152770 2024-03-06T07:22:13.908 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:22:13.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 msgr2=0x7f01d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 0x7f01d406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec009040 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 msgr2=0x7f01f014e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009d90 tx=0x7f01e8004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 shutdown_connections 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f01d406b4d0 0x7f01d406d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01f0152770 0x7f01f014e710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f00c1990 0x7f01f0151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 --2- 172.21.15.17:0/1415101146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f00c12a0 0x7f01f014cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 >> 172.21.15.17:0/1415101146 conn(0x7f01f00a94d0 msgr2=0x7f01f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 shutdown_connections 2024-03-06T07:22:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:13.908+0000 7f01f76d4640 1 -- 172.21.15.17:0/1415101146 wait complete. 2024-03-06T07:22:13.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:14.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.039+0000 7f33edb19640 1 Processor -- start 2024-03-06T07:22:14.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.039+0000 7f33edb19640 1 -- start start 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33edb19640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33e814f360 0x7f33e8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33edb19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c88c0 0x7f33e80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33edb19640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e80c73d0 con 0x7f33e80c5880 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33edb19640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e80c8dd0 con 0x7f33e814f360 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33edb19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80be2e0 con 0x7f33e80c88c0 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33e77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33e814f360 0x7f33e8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33e7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f33e80c5880 0x7f33e8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41816/0 (socket says 172.21.15.17:41816) 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33e77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33e814f360 0x7f33e8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35026/0 (socket says 172.21.15.17:35026) 2024-03-06T07:22:14.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.040+0000 7f33e7fff640 1 -- 172.21.15.17:0/2567990457 learned_addr learned my addr 172.21.15.17:0/2567990457 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e77fe640 1 -- 172.21.15.17:0/2567990457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c88c0 msgr2=0x7f33e80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e6ffd640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c88c0 0x7f33e80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e77fe640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c88c0 0x7f33e80c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e77fe640 1 -- 172.21.15.17:0/2567990457 >> v1:172.21.15.17:6789/0 conn(0x7f33e80c5880 legacy=0x7f33e8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e77fe640 1 -- 172.21.15.17:0/2567990457 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80c7670 con 0x7f33e814f360 2024-03-06T07:22:14.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.041+0000 7f33e67fc640 1 -- 172.21.15.17:0/2567990457 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755585252 0 0) 0x7f33e80c73d0 con 0x7f33e80c5880 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33e77fe640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e8155c10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097a0 tx=0x7f33d002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82787d795f0d4cf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33e67fc640 1 -- 172.21.15.17:0/2567990457 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0031780 con 0x7f33e814f360 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33e67fc640 1 -- 172.21.15.17:0/2567990457 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0037070 con 0x7f33e814f360 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33e67fc640 1 -- 172.21.15.17:0/2567990457 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0031b40 con 0x7f33e814f360 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33edb19640 1 -- 172.21.15.17:0/2567990457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 msgr2=0x7f33e8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33edb19640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e8155c10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097a0 tx=0x7f33d002b820 comp rx=0 tx=0).stop 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33edb19640 1 -- 172.21.15.17:0/2567990457 shutdown_connections 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33edb19640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c88c0 0x7f33e80c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.042+0000 7f33edb19640 1 --2- 172.21.15.17:0/2567990457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e8155c10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.043+0000 7f33edb19640 1 -- 172.21.15.17:0/2567990457 >> 172.21.15.17:0/2567990457 conn(0x7f33e80ac570 msgr2=0x7f33e80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.043+0000 7f33edb19640 1 -- 172.21.15.17:0/2567990457 shutdown_connections 2024-03-06T07:22:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.043+0000 7f33edb19640 1 -- 172.21.15.17:0/2567990457 wait complete. 2024-03-06T07:22:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.044+0000 7f33edb19640 1 Processor -- start 2024-03-06T07:22:14.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.044+0000 7f33edb19640 1 -- start start 2024-03-06T07:22:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.045+0000 7f33edb19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 0x7f33e81ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.045+0000 7f33edb19640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33e80c88c0 0x7f33e814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33edb19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33edb19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8159270 con 0x7f33e814f360 2024-03-06T07:22:14.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33edb19640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80c7c30 con 0x7f33e80c88c0 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33edb19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81593f0 con 0x7f33e80c5880 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35040/0 (socket says 172.21.15.17:35040) 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 -- 172.21.15.17:0/786018459 learned_addr learned my addr 172.21.15.17:0/786018459 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e77fe640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 0x7f33e81ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e6ffd640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33e80c88c0 0x7f33e814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 -- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 msgr2=0x7f33e81ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 0x7f33e81ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 -- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33e80c88c0 msgr2=0x7f33e814bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33e80c88c0 0x7f33e814bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 -- 172.21.15.17:0/786018459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80c12c0 con 0x7f33e814f360 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e77fe640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 0x7f33e81ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33e7fff640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c7a0 tx=0x7f33e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0018070 con 0x7f33e814f360 2024-03-06T07:22:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e000a640 con 0x7f33e814f360 2024-03-06T07:22:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e80befa0 con 0x7f33e814f360 2024-03-06T07:22:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.046+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e000aa70 con 0x7f33e814f360 2024-03-06T07:22:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.047+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e80c0020 con 0x7f33e814f360 2024-03-06T07:22:14.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.047+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e80ca1d0 con 0x7f33e814f360 2024-03-06T07:22:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.048+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33e001b070 con 0x7f33e814f360 2024-03-06T07:22:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.048+0000 7f33dffff640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 0x7f33c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.049+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f33e0089cd0 con 0x7f33e814f360 2024-03-06T07:22:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.049+0000 7f33e77fe640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 0x7f33c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.049+0000 7f33e77fe640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 0x7f33c806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33d0036000 tx=0x7f33d003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.053+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33e80ca1d0 con 0x7f33e814f360 2024-03-06T07:22:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.271+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33e804f330 con 0x7f33e814f360 2024-03-06T07:22:14.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.272+0000 7f33dffff640 1 -- 172.21.15.17:0/786018459 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f33e0057fc0 con 0x7f33e814f360 2024-03-06T07:22:14.274 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:14.274 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:10.991159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/2394960697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2394960697},{"type":"v1","addr":"172.21.15.17:6835","nonce":2394960697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:14.274 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 45 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 msgr2=0x7f33c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 0x7f33c806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33d0036000 tx=0x7f33d003d040 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 msgr2=0x7f33e814c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f33e000c7a0 tx=0x7f33e000cc60 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 shutdown_connections 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33c806b4d0 0x7f33c806d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33e814f360 0x7f33e814c1b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33e80c88c0 0x7f33e814bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 --2- 172.21.15.17:0/786018459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33e80c5880 0x7f33e81ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 >> 172.21.15.17:0/786018459 conn(0x7f33e80ac570 msgr2=0x7f33e80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 shutdown_connections 2024-03-06T07:22:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.274+0000 7f33edb19640 1 -- 172.21.15.17:0/786018459 wait complete. 2024-03-06T07:22:14.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:14.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.387+0000 7ff29e9a2640 1 Processor -- start 2024-03-06T07:22:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.387+0000 7ff29e9a2640 1 -- start start 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e9a2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff29814ba90 0x7ff2981559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e9a2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff298158a40 0x7ff298158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e9a2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29814be70 con 0x7ff2980c8690 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e9a2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2981593f0 con 0x7ff29814ba90 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e9a2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980cad30 con 0x7ff298158a40 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d9a0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff29814ba90 0x7ff2981559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d19f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2980c8690 0x7ff2980c5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41820/0 (socket says 172.21.15.17:41820) 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d19f640 1 -- 172.21.15.17:0/2083555915 learned_addr learned my addr 172.21.15.17:0/2083555915 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29e1a1640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff298158a40 0x7ff298158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d9a0640 1 -- 172.21.15.17:0/2083555915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff298158a40 msgr2=0x7ff298158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d9a0640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff298158a40 0x7ff298158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d9a0640 1 -- 172.21.15.17:0/2083555915 >> v1:172.21.15.17:6789/0 conn(0x7ff2980c8690 legacy=0x7ff2980c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29d9a0640 1 -- 172.21.15.17:0/2083555915 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980cb010 con 0x7ff29814ba90 2024-03-06T07:22:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.388+0000 7ff29c99e640 1 -- 172.21.15.17:0/2083555915 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727326263 0 0) 0x7ff288001110 con 0x7ff2980c8690 2024-03-06T07:22:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.389+0000 7ff29d9a0640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff29814ba90 0x7ff2981559b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097a0 tx=0x7ff29402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e24c865a92e9078 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.389+0000 7ff29c99e640 1 -- 172.21.15.17:0/2083555915 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294035020 con 0x7ff29814ba90 2024-03-06T07:22:14.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.389+0000 7ff29c99e640 1 -- 172.21.15.17:0/2083555915 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff294031830 con 0x7ff29814ba90 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.389+0000 7ff29c99e640 1 -- 172.21.15.17:0/2083555915 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294031c00 con 0x7ff29814ba90 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 -- 172.21.15.17:0/2083555915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff29814ba90 msgr2=0x7ff2981559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff29814ba90 0x7ff2981559b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097a0 tx=0x7ff29402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 -- 172.21.15.17:0/2083555915 shutdown_connections 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff298158a40 0x7ff298158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/2083555915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff29814ba90 0x7ff2981559b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 -- 172.21.15.17:0/2083555915 >> 172.21.15.17:0/2083555915 conn(0x7ff2980728d0 msgr2=0x7ff298072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 -- 172.21.15.17:0/2083555915 shutdown_connections 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.390+0000 7ff29e9a2640 1 -- 172.21.15.17:0/2083555915 wait complete. 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.391+0000 7ff29e9a2640 1 Processor -- start 2024-03-06T07:22:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.391+0000 7ff29e9a2640 1 -- start start 2024-03-06T07:22:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2980c8690 0x7ff298157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 0x7ff298144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29806cf60 con 0x7ff298158a40 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298158520 con 0x7ff2980c8690 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e9a2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2980c5820 con 0x7ff29814ba90 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35052/0 (socket says 172.21.15.17:35052) 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 -- 172.21.15.17:0/1999973717 learned_addr learned my addr 172.21.15.17:0/1999973717 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29d9a0640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2980c8690 0x7ff298157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29d19f640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 0x7ff298144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 -- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 msgr2=0x7ff298144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 0x7ff298144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 -- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2980c8690 msgr2=0x7ff298157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2980c8690 0x7ff298157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 -- 172.21.15.17:0/1999973717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298149e40 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29d19f640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 0x7ff298144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.392+0000 7ff29e1a1640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff290007d20 tx=0x7ff29000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.393+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290014420 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.393+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff290019070 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.393+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29814ae60 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.393+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290014820 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.393+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2980c1440 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.394+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29814b240 con 0x7ff298158a40 2024-03-06T07:22:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.394+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2900149c0 con 0x7ff298158a40 2024-03-06T07:22:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.395+0000 7ff28e7fc640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 0x7ff27c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.395+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7ff29008af60 con 0x7ff298158a40 2024-03-06T07:22:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.395+0000 7ff29d9a0640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 0x7ff27c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.395+0000 7ff29d9a0640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 0x7ff27c06d9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff294008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.399+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff29814b240 con 0x7ff298158a40 2024-03-06T07:22:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.641+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff298069430 con 0x7ff298158a40 2024-03-06T07:22:14.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.643+0000 7ff28e7fc640 1 -- 172.21.15.17:0/1999973717 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+21021 (secure 0 0 0) 0x7ff290059810 con 0x7ff298158a40 2024-03-06T07:22:14.644 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:14.645 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":59,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:07.986486+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:14.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 msgr2=0x7ff27c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 0x7ff27c06d9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff294008030 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 msgr2=0x7ff298145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff290007d20 tx=0x7ff29000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 shutdown_connections 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27c06b500 0x7ff27c06d9c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff298158a40 0x7ff298145410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.644+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff29814ba90 0x7ff298144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.645+0000 7ff29e9a2640 1 --2- 172.21.15.17:0/1999973717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2980c8690 0x7ff298157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.645+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 >> 172.21.15.17:0/1999973717 conn(0x7ff2980728d0 msgr2=0x7ff298146540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.645+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 shutdown_connections 2024-03-06T07:22:14.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.645+0000 7ff29e9a2640 1 -- 172.21.15.17:0/1999973717 wait complete. 2024-03-06T07:22:14.659 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:22:14.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:14.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.759+0000 7f8e2f8a4640 1 Processor -- start 2024-03-06T07:22:14.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.759+0000 7f8e2f8a4640 1 -- start start 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2f8a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e28159270 0x7f8e28155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2f8a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e280c73d0 con 0x7f8e280c88c0 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2f8a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e280be2e0 con 0x7f8e2814f360 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2f8a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e28159700 con 0x7f8e28159270 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2ce3f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8e280c88c0 0x7f8e28158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41824/0 (socket says 172.21.15.17:41824) 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.760+0000 7f8e2ce3f640 1 -- 172.21.15.17:0/82136559 learned_addr learned my addr 172.21.15.17:0/82136559 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:14.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962266360 0 0) 0x7f8e280c73d0 con 0x7f8e280c88c0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e14003520 con 0x7f8e280c88c0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2de41640 1 --2- 172.21.15.17:0/82136559 >> v2:172.21.15.163:3300/0 conn(0x7f8e28159270 0x7f8e28155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1484556798 0 0) 0x7f8e14003520 con 0x7f8e280c88c0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 >> v2:172.21.15.163:3300/0 conn(0x7f8e28159270 msgr2=0x7f8e28155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 --2- 172.21.15.17:0/82136559 >> v2:172.21.15.163:3300/0 conn(0x7f8e28159270 0x7f8e28155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 >> v1:172.21.15.163:6789/0 conn(0x7f8e2814f360 legacy=0x7f8e280c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280c7670 con 0x7f8e280c88c0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f8e18001670 con 0x7f8e280c88c0 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 >> v1:172.21.15.17:6789/0 conn(0x7f8e280c88c0 legacy=0x7f8e28158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140039b0 0x7f8e14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e14004800 0x7f8e14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 0x7f8e1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28159700 con 0x7f8e140052e0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1400c1b0 con 0x7f8e14004800 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1400c3b0 con 0x7f8e140039b0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2de41640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 0x7f8e1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2ce3f640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e14004800 0x7f8e14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2d640640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140039b0 0x7f8e14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2de41640 1 -- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140039b0 msgr2=0x7f8e14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.761+0000 7f8e2de41640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140039b0 0x7f8e14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.762+0000 7f8e2de41640 1 -- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e14004800 msgr2=0x7f8e14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.762+0000 7f8e2de41640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e14004800 0x7f8e14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.762+0000 7f8e2de41640 1 -- 172.21.15.17:0/82136559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14003520 con 0x7f8e140052e0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2de41640 1 -- 172.21.15.17:0/82136559 reap_dead start 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2de41640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 0x7f8e1400b9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8e20007330 tx=0x7f8e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c72e22482f07812 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20009be0 con 0x7f8e140052e0 2024-03-06T07:22:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2002ccd0 con 0x7f8e140052e0 2024-03-06T07:22:14.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e1ffff640 1 -- 172.21.15.17:0/82136559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20003bd0 con 0x7f8e140052e0 2024-03-06T07:22:14.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 msgr2=0x7f8e1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 0x7f8e1400b9c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8e20007330 tx=0x7f8e14005830 comp rx=0 tx=0).stop 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/82136559 shutdown_connections 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/82136559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e140052e0 0x7f8e1400b9c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/82136559 >> 172.21.15.17:0/82136559 conn(0x7f8e280ac570 msgr2=0x7f8e280ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/82136559 shutdown_connections 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.763+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/82136559 wait complete. 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.765+0000 7f8e2f8a4640 1 Processor -- start 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.765+0000 7f8e2f8a4640 1 -- start start 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 0x7f8e280bef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e280c0a50 0x7f8e280c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280c7c30 con 0x7f8e281edac0 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28155c10 con 0x7f8e280c0a50 2024-03-06T07:22:14.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2f8a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28155d90 con 0x7f8e140052e0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2de41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2de41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35076/0 (socket says 172.21.15.17:35076) 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.766+0000 7f8e2de41640 1 -- 172.21.15.17:0/365566844 learned_addr learned my addr 172.21.15.17:0/365566844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2ce3f640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e280c0a50 0x7f8e280c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2d640640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 0x7f8e280bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 -- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 msgr2=0x7f8e280bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 0x7f8e280bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 -- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e280c0a50 msgr2=0x7f8e280c0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e280c0a50 0x7f8e280c0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 -- 172.21.15.17:0/365566844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e281ea550 con 0x7f8e281edac0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2d640640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 0x7f8e280bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2de41640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8e200044c0 tx=0x7f8e20004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20043070 con 0x7f8e281edac0 2024-03-06T07:22:14.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20003e10 con 0x7f8e281edac0 2024-03-06T07:22:14.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e280c14c0 con 0x7f8e281edac0 2024-03-06T07:22:14.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20004730 con 0x7f8e281edac0 2024-03-06T07:22:14.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.767+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281eaaf0 con 0x7f8e281edac0 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.769+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2804f330 con 0x7f8e281edac0 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.769+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e20046070 con 0x7f8e281edac0 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.770+0000 7f8e1dffb640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 0x7f8e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.770+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f8e200abd90 con 0x7f8e281edac0 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.770+0000 7f8e2d640640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 0x7f8e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:14.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.770+0000 7f8e2d640640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 0x7f8e0c06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005580 tx=0x7f8e24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:14.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:14.774+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e2007a640 con 0x7f8e281edac0 2024-03-06T07:22:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.037+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e280c8d50 con 0x7f8e281edac0 2024-03-06T07:22:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.037+0000 7f8e1dffb640 1 -- 172.21.15.17:0/365566844 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f8e20079f40 con 0x7f8e281edac0 2024-03-06T07:22:15.040 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:15.040 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:10.991159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/2394960697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2394960697},{"type":"v1","addr":"172.21.15.17:6835","nonce":2394960697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 45 2024-03-06T07:22:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 msgr2=0x7f8e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 0x7f8e0c06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005580 tx=0x7f8e24009040 comp rx=0 tx=0).stop 2024-03-06T07:22:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 msgr2=0x7f8e281edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8e200044c0 tx=0x7f8e20004ab0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 shutdown_connections 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e0c06b4d0 0x7f8e0c06d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e281edac0 0x7f8e281edf40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e280c0a50 0x7f8e280c0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 --2- 172.21.15.17:0/365566844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e140052e0 0x7f8e280bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 >> 172.21.15.17:0/365566844 conn(0x7f8e280ac570 msgr2=0x7f8e2814cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 shutdown_connections 2024-03-06T07:22:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.040+0000 7f8e2f8a4640 1 -- 172.21.15.17:0/365566844 wait complete. 2024-03-06T07:22:15.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.159+0000 7f0ea0a76640 1 Processor -- start 2024-03-06T07:22:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.159+0000 7f0ea0a76640 1 -- start start 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0ea0a76640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e9c0c5480 0x7f0e9c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0ea0a76640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 0x7f0e9c06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0ea0a76640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9c0c5cc0 con 0x7f0e9c1586f0 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0ea0a76640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c06df40 con 0x7f0e9c0c5480 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0ea0a76640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c1591c0 con 0x7f0e9c0c5860 2024-03-06T07:22:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e9c0c5480 0x7f0e9c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0e9c0c5480 0x7f0e9c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35088/0 (socket says 172.21.15.17:35088) 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 -- 172.21.15.17:0/3617923399 learned_addr learned my addr 172.21.15.17:0/3617923399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e99d9b640 1 --2- 172.21.15.17:0/3617923399 >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 0x7f0e9c06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 -- 172.21.15.17:0/3617923399 >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 msgr2=0x7f0e9c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 --2- 172.21.15.17:0/3617923399 >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 0x7f0e9c06f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 -- 172.21.15.17:0/3617923399 >> v1:172.21.15.163:6789/0 conn(0x7f0e9c1586f0 legacy=0x7f0e9c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e9a59c640 1 -- 172.21.15.17:0/3617923399 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c1594a0 con 0x7f0e9c0c5480 2024-03-06T07:22:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.160+0000 7f0e99d9b640 1 --2- 172.21.15.17:0/3617923399 >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 0x7f0e9c06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.161+0000 7f0e9a59c640 1 --2- 172.21.15.17:0/3617923399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c155ba0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0c9770 tx=0x7f0e9002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2d6b124eeccdf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.161+0000 7f0e9959a640 1 -- 172.21.15.17:0/3617923399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9002e070 con 0x7f0e9c0c5480 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.161+0000 7f0e9959a640 1 -- 172.21.15.17:0/3617923399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e90037070 con 0x7f0e9c0c5480 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.161+0000 7f0e9959a640 1 -- 172.21.15.17:0/3617923399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e90031a00 con 0x7f0e9c0c5480 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 -- 172.21.15.17:0/3617923399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5480 msgr2=0x7f0e9c155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/3617923399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c155ba0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0c9770 tx=0x7f0e9002b880 comp rx=0 tx=0).stop 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 -- 172.21.15.17:0/3617923399 shutdown_connections 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/3617923399 >> v2:172.21.15.163:3300/0 conn(0x7f0e9c0c5860 0x7f0e9c06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/3617923399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c155ba0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 -- 172.21.15.17:0/3617923399 >> 172.21.15.17:0/3617923399 conn(0x7f0e9c06b1a0 msgr2=0x7f0e9c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 -- 172.21.15.17:0/3617923399 shutdown_connections 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.162+0000 7f0ea0a76640 1 -- 172.21.15.17:0/3617923399 wait complete. 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.163+0000 7f0ea0a76640 1 Processor -- start 2024-03-06T07:22:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.163+0000 7f0ea0a76640 1 -- start start 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c1e2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9c1586f0 0x7f0e9c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c152a20 con 0x7f0e9c0c5860 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c155680 con 0x7f0e9c0c5480 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0ea0a76640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c155050 con 0x7f0e9c1586f0 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35092/0 (socket says 172.21.15.17:35092) 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 -- 172.21.15.17:0/669002978 learned_addr learned my addr 172.21.15.17:0/669002978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e9a59c640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c1e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e9ad9d640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9c1586f0 0x7f0e9c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 -- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9c1586f0 msgr2=0x7f0e9c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9c1586f0 0x7f0e9c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 -- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e9c0c5480 msgr2=0x7f0e9c1e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c1e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.164+0000 7f0e99d9b640 1 -- 172.21.15.17:0/669002978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0ab070 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0e99d9b640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c7f0 tx=0x7f0e8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c015070 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c007a30 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c153f60 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c007e00 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.165+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c154520 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.166+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c04f330 con 0x7f0e9c0c5860 2024-03-06T07:22:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.166+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e8c018070 con 0x7f0e9c0c5860 2024-03-06T07:22:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.167+0000 7f0e8affd640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 0x7f0e7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.167+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f0e8c08a640 con 0x7f0e9c0c5860 2024-03-06T07:22:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.167+0000 7f0e9a59c640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 0x7f0e7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.168+0000 7f0e9a59c640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 0x7f0e7c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e90036000 tx=0x7f0e9003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.174+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e8c058930 con 0x7f0e9c0c5860 2024-03-06T07:22:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.391+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e9c0ad550 con 0x7f0e9c0c5860 2024-03-06T07:22:15.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.391+0000 7f0e8affd640 1 -- 172.21.15.17:0/669002978 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f0e8c05c9d0 con 0x7f0e9c0c5860 2024-03-06T07:22:15.393 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:15.393 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":45,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:43.270845+0000","modified":"2024-03-06T07:22:10.991159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6072},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6072":{"gid":6072,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/2394960697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2394960697},{"type":"v1","addr":"172.21.15.17:6835","nonce":2394960697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6072,"qdb_cluster":[6072]},"id":3},{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:02.388296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5769":{"gid":5769,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6834/3526701035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3526701035},{"type":"v1","addr":"172.21.15.17:6834","nonce":3526701035}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5769,"qdb_cluster":[5769]},"id":4}]} 2024-03-06T07:22:15.393 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 45 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 msgr2=0x7f0e7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 0x7f0e7c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0e90036000 tx=0x7f0e9003d040 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 msgr2=0x7f0e9c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c7f0 tx=0x7f0e8c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 shutdown_connections 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e7c06b4d0 0x7f0e7c06d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e9c1586f0 0x7f0e9c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e9c0c5860 0x7f0e9c157780 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 --2- 172.21.15.17:0/669002978 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e9c0c5480 0x7f0e9c1e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 >> 172.21.15.17:0/669002978 conn(0x7f0e9c06b1a0 msgr2=0x7f0e9c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 shutdown_connections 2024-03-06T07:22:15.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.393+0000 7f0ea0a76640 1 -- 172.21.15.17:0/669002978 wait complete. 2024-03-06T07:22:15.406 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:15.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.506+0000 7f20566af640 1 Processor -- start 2024-03-06T07:22:15.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.506+0000 7f20566af640 1 -- start start 2024-03-06T07:22:15.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f20566af640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2050152920 0x7f2050155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f20566af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20500c8a00 0x7f2050158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f20566af640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2050156180 con 0x7f2050159460 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f20566af640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20501587b0 con 0x7f2050152920 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f20566af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20500c7760 con 0x7f20500c8a00 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2050152920 0x7f2050155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f2054c4c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2050159460 0x7f20500c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41836/0 (socket says 172.21.15.17:41836) 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f2054c4c640 1 -- 172.21.15.17:0/3845521474 learned_addr learned my addr 172.21.15.17:0/3845521474 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204f7fe640 1 --2- 172.21.15.17:0/3845521474 >> v2:172.21.15.163:3300/0 conn(0x7f20500c8a00 0x7f2050158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204ffff640 1 -- 172.21.15.17:0/3845521474 >> v2:172.21.15.163:3300/0 conn(0x7f20500c8a00 msgr2=0x7f2050158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204ffff640 1 --2- 172.21.15.17:0/3845521474 >> v2:172.21.15.163:3300/0 conn(0x7f20500c8a00 0x7f2050158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204ffff640 1 -- 172.21.15.17:0/3845521474 >> v1:172.21.15.17:6789/0 conn(0x7f2050159460 legacy=0x7f20500c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.507+0000 7f204ffff640 1 -- 172.21.15.17:0/3845521474 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500c7a40 con 0x7f2050152920 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.508+0000 7f204ffff640 1 --2- 172.21.15.17:0/3845521474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050152920 0x7f2050155c40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2050067c30 tx=0x7f204002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc0317d86e22aa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.508+0000 7f204effd640 1 -- 172.21.15.17:0/3845521474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400315a0 con 0x7f2050152920 2024-03-06T07:22:15.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.508+0000 7f204effd640 1 -- 172.21.15.17:0/3845521474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2040037070 con 0x7f2050152920 2024-03-06T07:22:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.508+0000 7f204effd640 1 -- 172.21.15.17:0/3845521474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040031960 con 0x7f2050152920 2024-03-06T07:22:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 -- 172.21.15.17:0/3845521474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050152920 msgr2=0x7f2050155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 --2- 172.21.15.17:0/3845521474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050152920 0x7f2050155c40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2050067c30 tx=0x7f204002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 -- 172.21.15.17:0/3845521474 shutdown_connections 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 --2- 172.21.15.17:0/3845521474 >> v2:172.21.15.163:3300/0 conn(0x7f20500c8a00 0x7f2050158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 --2- 172.21.15.17:0/3845521474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050152920 0x7f2050155c40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 -- 172.21.15.17:0/3845521474 >> 172.21.15.17:0/3845521474 conn(0x7f20500ac560 msgr2=0x7f20500ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 -- 172.21.15.17:0/3845521474 shutdown_connections 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.509+0000 7f20566af640 1 -- 172.21.15.17:0/3845521474 wait complete. 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.511+0000 7f20566af640 1 Processor -- start 2024-03-06T07:22:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.511+0000 7f20566af640 1 -- start start 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20500c8a00 0x7f205014d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2050152920 0x7f20500abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205006d030 con 0x7f2050149860 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050158a00 con 0x7f20500c8a00 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f20566af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500c5a40 con 0x7f2050152920 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35104/0 (socket says 172.21.15.17:35104) 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 -- 172.21.15.17:0/2272103313 learned_addr learned my addr 172.21.15.17:0/2272103313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f204ffff640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20500c8a00 0x7f205014d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f204f7fe640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2050152920 0x7f20500abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 -- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2050152920 msgr2=0x7f20500abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2050152920 0x7f20500abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 -- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20500c8a00 msgr2=0x7f205014d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20500c8a00 0x7f205014d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.512+0000 7f2054c4c640 1 -- 172.21.15.17:0/2272103313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205014a820 con 0x7f2050149860 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f2054c4c640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f204800cad0 tx=0x7f204800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048017070 con 0x7f2050149860 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800abd0 con 0x7f2050149860 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048012400 con 0x7f2050149860 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205014ab00 con 0x7f2050149860 2024-03-06T07:22:15.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.513+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2050149060 con 0x7f2050149860 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.514+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205004f330 con 0x7f2050149860 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.514+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f204801a070 con 0x7f2050149860 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.515+0000 7f204cff9640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.515+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f204808a280 con 0x7f2050149860 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.515+0000 7f204ffff640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.516+0000 7f204ffff640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 0x7f203c06d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2040036000 tx=0x7f204003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.519+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2050156180 con 0x7f2050149860 2024-03-06T07:22:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.700+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20500be1f0 con 0x7f2050149860 2024-03-06T07:22:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.702+0000 7f204cff9640 1 -- 172.21.15.17:0/2272103313 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+21021 (secure 0 0 0) 0x7f2048058570 con 0x7f2050149860 2024-03-06T07:22:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:15.704 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":59,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:07.986486+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:21:41.533584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-03-06T07:21:42.234700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.39","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 msgr2=0x7f203c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 0x7f203c06d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2040036000 tx=0x7f204003d040 comp rx=0 tx=0).stop 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 msgr2=0x7f20500c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f204800cad0 tx=0x7f204800cf90 comp rx=0 tx=0).stop 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 shutdown_connections 2024-03-06T07:22:15.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f203c06b500 0x7f203c06d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2050149860 0x7f20500c8ec0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2050152920 0x7f20500abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 --2- 172.21.15.17:0/2272103313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20500c8a00 0x7f205014d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 >> 172.21.15.17:0/2272103313 conn(0x7f20500ac560 msgr2=0x7f20500ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 shutdown_connections 2024-03-06T07:22:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.704+0000 7f20566af640 1 -- 172.21.15.17:0/2272103313 wait complete. 2024-03-06T07:22:15.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:22:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.824+0000 7f7d82579640 1 Processor -- start 2024-03-06T07:22:15.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.824+0000 7f7d82579640 1 -- start start 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d82579640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d82579640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d7c1558d0 0x7f7d7c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d82579640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c155130 con 0x7f7d7c0c69f0 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d82579640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c155440 con 0x7f7d7c1558d0 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d82579640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ad480 con 0x7f7d7c0c8950 2024-03-06T07:22:15.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d80b16640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7d7c1558d0 0x7f7d7c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d7bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7d7c0c69f0 0x7f7d7c0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41842/0 (socket says 172.21.15.17:41842) 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d7bfff640 1 -- 172.21.15.17:0/2849383167 learned_addr learned my addr 172.21.15.17:0/2849383167 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d7b7fe640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d80b16640 1 -- 172.21.15.17:0/2849383167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 msgr2=0x7f7d7c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d80b16640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d80b16640 1 -- 172.21.15.17:0/2849383167 >> v1:172.21.15.17:6789/0 conn(0x7f7d7c0c69f0 legacy=0x7f7d7c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d80b16640 1 -- 172.21.15.17:0/2849383167 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0c5d50 con 0x7f7d7c1558d0 2024-03-06T07:22:15.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.825+0000 7f7d7b7fe640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.826+0000 7f7d80b16640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c155d60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009d90 tx=0x7f7d7402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddedad334658df85 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.826+0000 7f7d7affd640 1 -- 172.21.15.17:0/2849383167 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74039070 con 0x7f7d7c1558d0 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.826+0000 7f7d7affd640 1 -- 172.21.15.17:0/2849383167 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7402cbc0 con 0x7f7d7c1558d0 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.826+0000 7f7d7affd640 1 -- 172.21.15.17:0/2849383167 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74034440 con 0x7f7d7c1558d0 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 -- 172.21.15.17:0/2849383167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 msgr2=0x7f7d7c155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c155d60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009d90 tx=0x7f7d7402be10 comp rx=0 tx=0).stop 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 -- 172.21.15.17:0/2849383167 shutdown_connections 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c155d60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 --2- 172.21.15.17:0/2849383167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 -- 172.21.15.17:0/2849383167 >> 172.21.15.17:0/2849383167 conn(0x7f7d7c0cae70 msgr2=0x7f7d7c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 -- 172.21.15.17:0/2849383167 shutdown_connections 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.827+0000 7f7d82579640 1 -- 172.21.15.17:0/2849383167 wait complete. 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.828+0000 7f7d82579640 1 Processor -- start 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.828+0000 7f7d82579640 1 -- start start 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 0x7f7d7c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c152a20 con 0x7f7d7c1558d0 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c6310 con 0x7f7d7c0c69f0 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d82579640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c152ba0 con 0x7f7d7c0c8950 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35126/0 (socket says 172.21.15.17:35126) 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 -- 172.21.15.17:0/3583939575 learned_addr learned my addr 172.21.15.17:0/3583939575 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d7bfff640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 0x7f7d7c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d7b7fe640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 -- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 msgr2=0x7f7d7c0abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 -- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 msgr2=0x7f7d7c06c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 0x7f7d7c06c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d80b16640 1 -- 172.21.15.17:0/3583939575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0af200 con 0x7f7d7c1558d0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d7bfff640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 0x7f7d7c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.829+0000 7f7d7b7fe640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d80b16640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009d90 tx=0x7f7d74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74039050 con 0x7f7d7c1558d0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74045070 con 0x7f7d7c1558d0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0af4e0 con 0x7f7d7c1558d0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7403e830 con 0x7f7d7c1558d0 2024-03-06T07:22:15.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.830+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c0afaa0 con 0x7f7d7c1558d0 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.831+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c04f330 con 0x7f7d7c1558d0 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.831+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d7403e9d0 con 0x7f7d7c1558d0 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.832+0000 7f7d78ff9640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 0x7f7d6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.832+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7187+0+0 (secure 0 0 0) 0x7f7d740ad570 con 0x7f7d7c1558d0 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.832+0000 7f7d7bfff640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 0x7f7d6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:15.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.833+0000 7f7d7bfff640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 0x7f7d6806d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000f60 tx=0x7f7d6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:15.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:15.836+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d7407bf40 con 0x7f7d7c1558d0 2024-03-06T07:22:15.989 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:22:15.987+0000 7f4df659b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:16.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.187+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7d7c0ca650 con 0x7f7d7c1558d0 2024-03-06T07:22:16.189 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:22:16.187+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:16.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.189+0000 7f7d78ff9640 1 -- 172.21.15.17:0/3583939575 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7d7407b840 con 0x7f7d7c1558d0 2024-03-06T07:22:16.190 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 msgr2=0x7f7d6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 0x7f7d6806d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000f60 tx=0x7f7d6c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 msgr2=0x7f7d7c1ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009d90 tx=0x7f7d74004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 shutdown_connections 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d6806b500 0x7f7d6806d9c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d7c1558d0 0x7f7d7c1ee070 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d7c0c8950 0x7f7d7c0abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 --2- 172.21.15.17:0/3583939575 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d7c0c69f0 0x7f7d7c06c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 >> 172.21.15.17:0/3583939575 conn(0x7f7d7c0cae70 msgr2=0x7f7d7c0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 shutdown_connections 2024-03-06T07:22:16.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.192+0000 7f7d82579640 1 -- 172.21.15.17:0/3583939575 wait complete. 2024-03-06T07:22:16.195 INFO:tasks.ceph.mds.a.smithi017.stderr: -40> 2024-03-06T07:22:15.987+0000 7f4df659b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:16.205 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:22:16.222 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:22:16.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.310+0000 7f79854c0640 1 Processor -- start 2024-03-06T07:22:16.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.310+0000 7f79854c0640 1 -- start start 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79854c0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79800c4e60 0x7f798006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79854c0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79854c0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7980156650 con 0x7f7980072bc0 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79854c0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798006ec50 con 0x7f79800c4e60 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79854c0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79801568f0 con 0x7f79800c6080 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f797effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79800c4e60 0x7f798006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79767fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f797effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f79800c4e60 0x7f798006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50498/0 (socket says 172.21.15.17:50498) 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f79767fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:45298/0 (socket says 172.21.15.17:45298) 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f797effd640 1 -- 172.21.15.17:0/1120012334 learned_addr learned my addr 172.21.15.17:0/1120012334 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.311+0000 7f797f7fe640 1 --1- 172.21.15.17:0/1120012334 >> v1:172.21.15.163:6789/0 conn(0x7f7980072bc0 0x7f7980153030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:39366/0 (socket says 172.21.15.17:39366) 2024-03-06T07:22:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.312+0000 7f797effd640 1 -- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 msgr2=0x7f7980072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.312+0000 7f797effd640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.312+0000 7f797effd640 1 -- 172.21.15.17:0/1120012334 >> v1:172.21.15.163:6789/0 conn(0x7f7980072bc0 legacy=0x7f7980153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.312+0000 7f797effd640 1 -- 172.21.15.17:0/1120012334 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980156bd0 con 0x7f79800c4e60 2024-03-06T07:22:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.312+0000 7f79767fc640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.313+0000 7f797effd640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c4e60 0x7f798006f7d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7970009870 tx=0x7f797002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2327e42e934c684d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.313+0000 7f797e7fc640 1 -- 172.21.15.17:0/1120012334 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970034070 con 0x7f79800c4e60 2024-03-06T07:22:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.313+0000 7f797e7fc640 1 -- 172.21.15.17:0/1120012334 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970030840 con 0x7f79800c4e60 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.313+0000 7f797e7fc640 1 -- 172.21.15.17:0/1120012334 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970030c10 con 0x7f79800c4e60 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 -- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c4e60 msgr2=0x7f798006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c4e60 0x7f798006f7d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7970009870 tx=0x7f797002b980 comp rx=0 tx=0).stop 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 -- 172.21.15.17:0/1120012334 shutdown_connections 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79800c6080 0x7f7980072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 --2- 172.21.15.17:0/1120012334 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c4e60 0x7f798006f7d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 -- 172.21.15.17:0/1120012334 >> 172.21.15.17:0/1120012334 conn(0x7f79800c5560 msgr2=0x7f79800c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 -- 172.21.15.17:0/1120012334 shutdown_connections 2024-03-06T07:22:16.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.314+0000 7f79854c0640 1 -- 172.21.15.17:0/1120012334 wait complete. 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 Processor -- start 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 -- start start 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7980072bc0 0x7f798014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c6080 0x7f79801dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800c87d0 con 0x7f79800c4e60 2024-03-06T07:22:16.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980157190 con 0x7f79800c6080 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.316+0000 7f79854c0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980153740 con 0x7f7980072bc0 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35142/0 (socket says 172.21.15.17:35142) 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 -- 172.21.15.17:0/643340622 learned_addr learned my addr 172.21.15.17:0/643340622 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f797f7fe640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c6080 0x7f79801dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f797effd640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7980072bc0 0x7f798014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 -- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7980072bc0 msgr2=0x7f798014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7980072bc0 0x7f798014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 -- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c6080 msgr2=0x7f79801dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c6080 0x7f79801dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 -- 172.21.15.17:0/643340622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800ae890 con 0x7f79800c4e60 2024-03-06T07:22:16.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79767fc640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f796800e900 tx=0x7f796800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968017070 con 0x7f79800c4e60 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800ca00 con 0x7f79800c4e60 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79801eccf0 con 0x7f79800c4e60 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800cdf0 con 0x7f79800c4e60 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.317+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800aba30 con 0x7f79800c4e60 2024-03-06T07:22:16.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.318+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79800c2a70 con 0x7f79800c4e60 2024-03-06T07:22:16.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.319+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f796801a070 con 0x7f79800c4e60 2024-03-06T07:22:16.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.319+0000 7f7977fff640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 0x7f796006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.319+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7273+0+0 (secure 0 0 0) 0x7f796808a2c0 con 0x7f79800c4e60 2024-03-06T07:22:16.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.319+0000 7f797effd640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 0x7f796006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.320+0000 7f797effd640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 0x7f796006d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7970036000 tx=0x7f797003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.328+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7968058bb0 con 0x7f79800c4e60 2024-03-06T07:22:16.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.725+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f79800c8330 con 0x7f79800c4e60 2024-03-06T07:22:16.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.818+0000 7f7977fff640 1 -- 172.21.15.17:0/643340622 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f79680584b0 con 0x7f79800c4e60 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 msgr2=0x7f796006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 0x7f796006d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7970036000 tx=0x7f797003b040 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 msgr2=0x7f79800ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f796800e900 tx=0x7f796800edc0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 shutdown_connections 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f796006b4d0 0x7f796006d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79800c6080 0x7f79801dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79800c4e60 0x7f79800ac470 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 --2- 172.21.15.17:0/643340622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7980072bc0 0x7f798014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 >> 172.21.15.17:0/643340622 conn(0x7f79800c5560 msgr2=0x7f79800c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 shutdown_connections 2024-03-06T07:22:16.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.821+0000 7f79854c0640 1 -- 172.21.15.17:0/643340622 wait complete. 2024-03-06T07:22:16.824 INFO:tasks.ceph.mds.c.smithi017.stderr: -390> 2024-03-06T07:22:04.205+0000 7fe7e845e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:16.837 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:22:16.850 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:22:16.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.954+0000 7fb79ef63640 1 Processor -- start 2024-03-06T07:22:16.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.954+0000 7fb79ef63640 1 -- start start 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb79ef63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb7980c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb79ef63640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980c7a10 con 0x7fb7980c7470 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb79ef63640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980c7cf0 con 0x7fb798152860 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb79ef63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb798158da0 con 0x7fb798159230 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb797fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb7980c7470 0x7fb798155d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41886/0 (socket says 172.21.15.17:41886) 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb797fff640 1 -- 172.21.15.17:0/1387164331 learned_addr learned my addr 172.21.15.17:0/1387164331 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355579027 0 0) 0x7fb7980c7a10 con 0x7fb7980c7470 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb784003520 con 0x7fb7980c7470 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb79d500640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb7980c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283049917 0 0) 0x7fb784003520 con 0x7fb7980c7470 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 msgr2=0x7fb7980c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb7980c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 >> v1:172.21.15.163:6789/0 conn(0x7fb798152860 legacy=0x7fb7980c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798158a30 con 0x7fb7980c7470 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb7880019f0 con 0x7fb7980c7470 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.955+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 >> v1:172.21.15.17:6789/0 conn(0x7fb7980c7470 legacy=0x7fb798155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb784003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb784003fd0 0x7fb784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798158da0 con 0x7fb784004d40 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78400bbb0 con 0x7fb784003fd0 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78400bdb0 con 0x7fb798159230 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb797fff640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb784003fd0 0x7fb784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79ccff640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb784003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 -- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 msgr2=0x7fb784003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb798159230 0x7fb784003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 -- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb784003fd0 msgr2=0x7fb784004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb784003fd0 0x7fb784004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb79d500640 1 -- 172.21.15.17:0/1387164331 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784003520 con 0x7fb784004d40 2024-03-06T07:22:16.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.956+0000 7fb797fff640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb784003fd0 0x7fb784004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79d500640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7840051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ac0 tx=0x7fb784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d74ee5739c5c6ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780039070 con 0x7fb784004d40 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78002c6b0 con 0x7fb784004d40 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb7977fe640 1 -- 172.21.15.17:0/1387164331 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780009690 con 0x7fb784004d40 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 -- 172.21.15.17:0/1387164331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 msgr2=0x7fb7840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 --2- 172.21.15.17:0/1387164331 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7840051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ac0 tx=0x7fb784005830 comp rx=0 tx=0).stop 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79d500640 1 -- 172.21.15.17:0/1387164331 reap_dead start 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 -- 172.21.15.17:0/1387164331 shutdown_connections 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 -- 172.21.15.17:0/1387164331 >> 172.21.15.17:0/1387164331 conn(0x7fb7980ac630 msgr2=0x7fb7980aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 -- 172.21.15.17:0/1387164331 shutdown_connections 2024-03-06T07:22:16.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.957+0000 7fb79ef63640 1 -- 172.21.15.17:0/1387164331 wait complete. 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.958+0000 7fb79ef63640 1 Processor -- start 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.958+0000 7fb79ef63640 1 -- start start 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7981e83b0 0x7fb7980c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7980c2f10 0x7fb7981ee250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798159840 con 0x7fb784004d40 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7981596c0 con 0x7fb7981e83b0 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ef63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c53e0 con 0x7fb7980c2f10 2024-03-06T07:22:16.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ccff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ccff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35166/0 (socket says 172.21.15.17:35166) 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79ccff640 1 -- 172.21.15.17:0/2042373297 learned_addr learned my addr 172.21.15.17:0/2042373297 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb79d500640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7980c2f10 0x7fb7981ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.959+0000 7fb797fff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7981e83b0 0x7fb7980c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 -- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7980c2f10 msgr2=0x7fb7981ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7980c2f10 0x7fb7981ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 -- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7981e83b0 msgr2=0x7fb7980c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7981e83b0 0x7fb7980c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 -- 172.21.15.17:0/2042373297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980c2510 con 0x7fb784004d40 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ccff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e5b0 tx=0x7fb79000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900052e0 con 0x7fb784004d40 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790018070 con 0x7fb784004d40 2024-03-06T07:22:16.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790003810 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980c36b0 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.960+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7980b0470 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.961+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79804f330 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.961+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7900039b0 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.962+0000 7fb7957fa640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 0x7fb77806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.962+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7359+0+0 (secure 0 0 0) 0x7fb79008a320 con 0x7fb784004d40 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.962+0000 7fb797fff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 0x7fb77806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.963+0000 7fb797fff640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 0x7fb77806d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb798155000 tx=0x7fb7880021d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:16.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:16.966+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb7980c2510 con 0x7fb784004d40 2024-03-06T07:22:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:17.148+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7980c8110 con 0x7fb784004d40 2024-03-06T07:22:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.017+0000 7fb7957fa640 1 -- 172.21.15.17:0/2042373297 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb7900584e0 con 0x7fb784004d40 2024-03-06T07:22:18.019 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 msgr2=0x7fb77806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 0x7fb77806d9c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb798155000 tx=0x7fb7880021d0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 msgr2=0x7fb7981e7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb79000e5b0 tx=0x7fb79000ea70 comp rx=0 tx=0).stop 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 shutdown_connections 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb77806b500 0x7fb77806d9c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7980c2f10 0x7fb7981ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7981e83b0 0x7fb7980c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 --2- 172.21.15.17:0/2042373297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb784004d40 0x7fb7981e7e70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.021+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 >> 172.21.15.17:0/2042373297 conn(0x7fb7980ac630 msgr2=0x7fb7980c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.022+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 shutdown_connections 2024-03-06T07:22:18.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.022+0000 7fb79ef63640 1 -- 172.21.15.17:0/2042373297 wait complete. 2024-03-06T07:22:18.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:22:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.147+0000 7f37d9c34640 1 Processor -- start 2024-03-06T07:22:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.147+0000 7f37d9c34640 1 -- start start 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d9c34640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37d40c88e0 0x7f37d40c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d9c34640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37d40c7930 0x7f37d406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d9c34640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d40c5400 con 0x7f37d4152870 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d9c34640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d406d320 con 0x7f37d40c7930 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d9c34640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d406b8d0 con 0x7f37d40c88e0 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37d40c7930 0x7f37d406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37d40c7930 0x7f37d406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35178/0 (socket says 172.21.15.17:35178) 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 -- 172.21.15.17:0/2832516002 learned_addr learned my addr 172.21.15.17:0/2832516002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d2ffd640 1 --2- 172.21.15.17:0/2832516002 >> v2:172.21.15.163:3300/0 conn(0x7f37d40c88e0 0x7f37d40c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 -- 172.21.15.17:0/2832516002 >> v2:172.21.15.163:3300/0 conn(0x7f37d40c88e0 msgr2=0x7f37d40c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 --2- 172.21.15.17:0/2832516002 >> v2:172.21.15.163:3300/0 conn(0x7f37d40c88e0 0x7f37d40c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 -- 172.21.15.17:0/2832516002 >> v1:172.21.15.163:6789/0 conn(0x7f37d4152870 legacy=0x7f37d4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:18.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.148+0000 7f37d3fff640 1 -- 172.21.15.17:0/2832516002 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40c56e0 con 0x7f37d40c7930 2024-03-06T07:22:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.149+0000 7f37d3fff640 1 --2- 172.21.15.17:0/2832516002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c7930 0x7f37d406bd60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ad5b0 tx=0x7f37cc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a2b79490fdca0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.149+0000 7f37d27fc640 1 -- 172.21.15.17:0/2832516002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc02c9f0 con 0x7f37d40c7930 2024-03-06T07:22:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.149+0000 7f37d27fc640 1 -- 172.21.15.17:0/2832516002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc02cb90 con 0x7f37d40c7930 2024-03-06T07:22:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.149+0000 7f37d27fc640 1 -- 172.21.15.17:0/2832516002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0343f0 con 0x7f37d40c7930 2024-03-06T07:22:18.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 -- 172.21.15.17:0/2832516002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c7930 msgr2=0x7f37d406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 --2- 172.21.15.17:0/2832516002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c7930 0x7f37d406bd60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ad5b0 tx=0x7f37cc02be50 comp rx=0 tx=0).stop 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 -- 172.21.15.17:0/2832516002 shutdown_connections 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 --2- 172.21.15.17:0/2832516002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c7930 0x7f37d406bd60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 --2- 172.21.15.17:0/2832516002 >> v2:172.21.15.163:3300/0 conn(0x7f37d40c88e0 0x7f37d40c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 -- 172.21.15.17:0/2832516002 >> 172.21.15.17:0/2832516002 conn(0x7f37d40ac5f0 msgr2=0x7f37d40aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 -- 172.21.15.17:0/2832516002 shutdown_connections 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.150+0000 7f37d9c34640 1 -- 172.21.15.17:0/2832516002 wait complete. 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.151+0000 7f37d9c34640 1 Processor -- start 2024-03-06T07:22:18.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.151+0000 7f37d9c34640 1 -- start start 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 0x7f37d41eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37d4152870 0x7f37d4151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40c9470 con 0x7f37d40c88e0 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40c5ca0 con 0x7f37d40c7930 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40c95f0 con 0x7f37d4152870 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35188/0 (socket says 172.21.15.17:35188) 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 -- 172.21.15.17:0/1531157300 learned_addr learned my addr 172.21.15.17:0/1531157300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d37fe640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 0x7f37d41eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d3fff640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37d4152870 0x7f37d4151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 -- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37d4152870 msgr2=0x7f37d4151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37d4152870 0x7f37d4151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 -- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 msgr2=0x7f37d41eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 0x7f37d41eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 -- 172.21.15.17:0/1531157300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d41eb810 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d37fe640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 0x7f37d41eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d2ffd640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f37c800d5f0 tx=0x7f37c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c80125f0 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8019070 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d40c1d00 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c80129d0 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.152+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d41edfe0 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.153+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d40c2220 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.154+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37c8012b70 con 0x7f37d40c88e0 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.154+0000 7f37bbfff640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 0x7f37c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.154+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f37c808aa00 con 0x7f37d40c88e0 2024-03-06T07:22:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.154+0000 7f37d37fe640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 0x7f37c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:18.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.155+0000 7f37d37fe640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 0x7f37c006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37c4003eb0 tx=0x7f37c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:18.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.158+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f37d41eb810 con 0x7f37d40c88e0 2024-03-06T07:22:18.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:18.339+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37d404f330 con 0x7f37d40c88e0 2024-03-06T07:22:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.018+0000 7f37bbfff640 1 -- 172.21.15.17:0/1531157300 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7f37c8059450 con 0x7f37d40c88e0 2024-03-06T07:22:19.021 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 msgr2=0x7f37c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 0x7f37c006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37c4003eb0 tx=0x7f37c4009040 comp rx=0 tx=0).stop 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 msgr2=0x7f37d414e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f37c800d5f0 tx=0x7f37c800dab0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 shutdown_connections 2024-03-06T07:22:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f37c006b4d0 0x7f37c006d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37d4152870 0x7f37d4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d40c88e0 0x7f37d414e1c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 --2- 172.21.15.17:0/1531157300 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37d40c7930 0x7f37d41eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 >> 172.21.15.17:0/1531157300 conn(0x7f37d40ac5f0 msgr2=0x7f37d406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.022+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 shutdown_connections 2024-03-06T07:22:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.023+0000 7f37d9c34640 1 -- 172.21.15.17:0/1531157300 wait complete. 2024-03-06T07:22:19.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.143+0000 7f2faf4ed640 1 Processor -- start 2024-03-06T07:22:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.143+0000 7f2faf4ed640 1 -- start start 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2faf4ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c8840 0x7f2fa806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2faf4ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fa80c7350 0x7f2fa80c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2faf4ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa806c150 con 0x7f2fa814f250 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2faf4ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa806c480 con 0x7f2fa80c7350 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2faf4ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c8cc0 con 0x7f2fa80c8840 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2fad289640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2fa814f250 0x7f2fa8155c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41898/0 (socket says 172.21.15.17:41898) 2024-03-06T07:22:19.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2fad289640 1 -- 172.21.15.17:0/1267501640 learned_addr learned my addr 172.21.15.17:0/1267501640 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.144+0000 7f2fada8a640 1 --2- 172.21.15.17:0/1267501640 >> v2:172.21.15.17:3300/0 conn(0x7f2fa80c7350 0x7f2fa80c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2f9ffff640 1 -- 172.21.15.17:0/1267501640 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181089182 0 0) 0x7f2fa806c150 con 0x7f2fa814f250 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2f9ffff640 1 -- 172.21.15.17:0/1267501640 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f94003520 con 0x7f2fa814f250 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2faca88640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c8840 0x7f2fa806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2fada8a640 1 -- 172.21.15.17:0/1267501640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c8840 msgr2=0x7f2fa806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2fada8a640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c8840 0x7f2fa806c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2fada8a640 1 -- 172.21.15.17:0/1267501640 >> v1:172.21.15.17:6789/0 conn(0x7f2fa814f250 legacy=0x7f2fa8155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2fada8a640 1 -- 172.21.15.17:0/1267501640 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa814f630 con 0x7f2fa80c7350 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.145+0000 7f2fada8a640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c7350 0x7f2fa80c77e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009d90 tx=0x7f2fa402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ae37edf4fdbd5f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2f9ffff640 1 -- 172.21.15.17:0/1267501640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4039070 con 0x7f2fa80c7350 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2f9ffff640 1 -- 172.21.15.17:0/1267501640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa402cb30 con 0x7f2fa80c7350 2024-03-06T07:22:19.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2f9ffff640 1 -- 172.21.15.17:0/1267501640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4034440 con 0x7f2fa80c7350 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 -- 172.21.15.17:0/1267501640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c7350 msgr2=0x7f2fa80c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c7350 0x7f2fa80c77e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009d90 tx=0x7f2fa402bd80 comp rx=0 tx=0).stop 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 -- 172.21.15.17:0/1267501640 shutdown_connections 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c7350 0x7f2fa80c77e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/1267501640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c8840 0x7f2fa806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 -- 172.21.15.17:0/1267501640 >> 172.21.15.17:0/1267501640 conn(0x7f2fa80ac4d0 msgr2=0x7f2fa80ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 -- 172.21.15.17:0/1267501640 shutdown_connections 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.146+0000 7f2faf4ed640 1 -- 172.21.15.17:0/1267501640 wait complete. 2024-03-06T07:22:19.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.147+0000 7f2faf4ed640 1 Processor -- start 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.147+0000 7f2faf4ed640 1 -- start start 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c7350 0x7f2fa80bee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fa814f250 0x7f2fa80ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa806daf0 con 0x7f2fa80c8840 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c5940 con 0x7f2fa814f250 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faf4ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c5c20 con 0x7f2fa80c7350 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35212/0 (socket says 172.21.15.17:35212) 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 -- 172.21.15.17:0/3213620178 learned_addr learned my addr 172.21.15.17:0/3213620178 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2fad289640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c7350 0x7f2fa80bee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2fada8a640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fa814f250 0x7f2fa80ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 -- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c7350 msgr2=0x7f2fa80bee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c7350 0x7f2fa80bee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 -- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fa814f250 msgr2=0x7f2fa80ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fa814f250 0x7f2fa80ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.148+0000 7f2faca88640 1 -- 172.21.15.17:0/3213620178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa81edb90 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2faca88640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c7f0 tx=0x7f2f9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98015070 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98004240 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa81ede00 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98007620 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.149+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa80bf390 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.150+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa804f330 con 0x7f2fa80c8840 2024-03-06T07:22:19.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.150+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f980079d0 con 0x7f2fa80c8840 2024-03-06T07:22:19.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.151+0000 7f2f9dffb640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 0x7f2f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.151+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f2f9808a5e0 con 0x7f2fa80c8840 2024-03-06T07:22:19.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.151+0000 7f2fad289640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 0x7f2f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.152+0000 7f2fad289640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 0x7f2f8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005880 tx=0x7f2fa0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.155+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f980590e0 con 0x7f2fa80c8840 2024-03-06T07:22:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.336+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fa80c77c0 con 0x7f2fa80c8840 2024-03-06T07:22:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.337+0000 7f2f9dffb640 1 -- 172.21.15.17:0/3213620178 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f2f980589e0 con 0x7f2fa80c8840 2024-03-06T07:22:19.339 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:19.339 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":63,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:19.018373+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.17:6833/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6834/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6832/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.339+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 msgr2=0x7f2f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.339+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 0x7f2f8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005880 tx=0x7f2fa0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.339+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 msgr2=0x7f2fa80c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.339+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c7f0 tx=0x7f2f9800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 shutdown_connections 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f8c06b4d0 0x7f2f8c06d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fa814f250 0x7f2fa80ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fa80c8840 0x7f2fa80c3190 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 --2- 172.21.15.17:0/3213620178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fa80c7350 0x7f2fa80bee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 >> 172.21.15.17:0/3213620178 conn(0x7f2fa80ac4d0 msgr2=0x7f2fa80aa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 shutdown_connections 2024-03-06T07:22:19.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.340+0000 7f2faf4ed640 1 -- 172.21.15.17:0/3213620178 wait complete. 2024-03-06T07:22:19.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:21:57.664157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:21:58.336311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:19.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:19.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.456+0000 7fcf073e2640 1 Processor -- start 2024-03-06T07:22:19.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.456+0000 7fcf073e2640 1 -- start start 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.456+0000 7fcf073e2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf001529f0 0x7fcf000c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.456+0000 7fcf073e2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00152dd0 con 0x7fcf000c5370 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.456+0000 7fcf073e2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00158b00 con 0x7fcf000c57d0 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf073e2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf00159710 con 0x7fcf001529f0 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0497d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcf000c5370 0x7fcf00155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41908/0 (socket says 172.21.15.17:41908) 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0497d640 1 -- 172.21.15.17:0/3218320832 learned_addr learned my addr 172.21.15.17:0/3218320832 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0517e640 1 --2- 172.21.15.17:0/3218320832 >> v2:172.21.15.163:3300/0 conn(0x7fcf001529f0 0x7fcf000c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238864572 0 0) 0x7fcf00152dd0 con 0x7fcf000c5370 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceec003520 con 0x7fcf000c5370 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677014909 0 0) 0x7fceec003520 con 0x7fcf000c5370 2024-03-06T07:22:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0517e640 1 -- 172.21.15.17:0/3218320832 >> v1:172.21.15.163:6789/0 conn(0x7fcf000c57d0 legacy=0x7fcf0015fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0517e640 1 -- 172.21.15.17:0/3218320832 >> v1:172.21.15.17:6789/0 conn(0x7fcf000c5370 legacy=0x7fcf00155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.457+0000 7fcf0517e640 1 -- 172.21.15.17:0/3218320832 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf00158de0 con 0x7fcf001529f0 2024-03-06T07:22:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.458+0000 7fcf0517e640 1 --2- 172.21.15.17:0/3218320832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf001529f0 0x7fcf000c88f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0097a0 tx=0x7fcefc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae00ebc2c528b76 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.458+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc037070 con 0x7fcf001529f0 2024-03-06T07:22:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.458+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc030510 con 0x7fcf001529f0 2024-03-06T07:22:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.458+0000 7fcef7fff640 1 -- 172.21.15.17:0/3218320832 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0001670 con 0x7fcf001529f0 2024-03-06T07:22:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 -- 172.21.15.17:0/3218320832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf001529f0 msgr2=0x7fcf000c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 --2- 172.21.15.17:0/3218320832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf001529f0 0x7fcf000c88f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcefc0097a0 tx=0x7fcefc02b790 comp rx=0 tx=0).stop 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 -- 172.21.15.17:0/3218320832 shutdown_connections 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 --2- 172.21.15.17:0/3218320832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf001529f0 0x7fcf000c88f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 -- 172.21.15.17:0/3218320832 >> 172.21.15.17:0/3218320832 conn(0x7fcf000ac580 msgr2=0x7fcf000ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 -- 172.21.15.17:0/3218320832 shutdown_connections 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.459+0000 7fcf073e2640 1 -- 172.21.15.17:0/3218320832 wait complete. 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 Processor -- start 2024-03-06T07:22:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 -- start start 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 0x7fcf00157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf001529f0 0x7fcf001ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001593a0 con 0x7fcf000c5370 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c68b0 con 0x7fcf000c57d0 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.460+0000 7fcf073e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf00158740 con 0x7fcf001529f0 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35226/0 (socket says 172.21.15.17:35226) 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 -- 172.21.15.17:0/4082602963 learned_addr learned my addr 172.21.15.17:0/4082602963 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0497d640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 0x7fcf00157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0597f640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf001529f0 0x7fcf001ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 -- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf001529f0 msgr2=0x7fcf001ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf001529f0 0x7fcf001ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 -- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 msgr2=0x7fcf00157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 0x7fcf00157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 -- 172.21.15.17:0/4082602963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000af920 con 0x7fcf000c5370 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0497d640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 0x7fcf00157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf0517e640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02b7d0 tx=0x7fcefc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc041020 con 0x7fcf000c5370 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc03c870 con 0x7fcf000c5370 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000b0730 con 0x7fcf000c5370 2024-03-06T07:22:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.461+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc037040 con 0x7fcf000c5370 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.462+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00158120 con 0x7fcf000c5370 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.462+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0004f330 con 0x7fcf000c5370 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.463+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcefc044020 con 0x7fcf000c5370 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.463+0000 7fcef5ffb640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.463+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fcefc0abf10 con 0x7fcf000c5370 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.463+0000 7fcf0497d640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.464+0000 7fcf0497d640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 0x7fcee406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf001eebb0 tx=0x7fcef0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.468+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcefc04b030 con 0x7fcf000c5370 2024-03-06T07:22:19.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.650+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf000c7bf0 con 0x7fcf000c5370 2024-03-06T07:22:19.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.650+0000 7fcef5ffb640 1 -- 172.21.15.17:0/4082602963 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcefc07ab30 con 0x7fcf000c5370 2024-03-06T07:22:19.652 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:22:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 msgr2=0x7fcee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 0x7fcee406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcf001eebb0 tx=0x7fcef0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 msgr2=0x7fcf001eaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02b7d0 tx=0x7fcefc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 shutdown_connections 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcee406b4d0 0x7fcee406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf001529f0 0x7fcf001ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf000c57d0 0x7fcf00157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 --2- 172.21.15.17:0/4082602963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf000c5370 0x7fcf001eaf40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 >> 172.21.15.17:0/4082602963 conn(0x7fcf000ac580 msgr2=0x7fcf000ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 shutdown_connections 2024-03-06T07:22:19.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.652+0000 7fcf073e2640 1 -- 172.21.15.17:0/4082602963 wait complete. 2024-03-06T07:22:19.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:22:19.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.767+0000 7fed41de9640 1 Processor -- start 2024-03-06T07:22:19.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.767+0000 7fed41de9640 1 -- start start 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed41de9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed3c0c67f0 0x7fed3c155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed41de9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed41de9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c152cb0 con 0x7fed3c1528d0 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed41de9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c155ca0 con 0x7fed3c0c67f0 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed41de9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c38b0 con 0x7fed3c0c4ef0 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fed3c1528d0 0x7fed3c0724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41924/0 (socket says 172.21.15.17:41924) 2024-03-06T07:22:19.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fed3c0c67f0 0x7fed3c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3b7fe640 1 -- 172.21.15.17:0/960927110 learned_addr learned my addr 172.21.15.17:0/960927110 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3bfff640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3affd640 1 -- 172.21.15.17:0/960927110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 msgr2=0x7fed3c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3affd640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3affd640 1 -- 172.21.15.17:0/960927110 >> v1:172.21.15.17:6789/0 conn(0x7fed3c1528d0 legacy=0x7fed3c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3affd640 1 -- 172.21.15.17:0/960927110 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c1594f0 con 0x7fed3c0c67f0 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3a7fc640 1 -- 172.21.15.17:0/960927110 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839226120 0 0) 0x7fed3c152cb0 con 0x7fed3c1528d0 2024-03-06T07:22:19.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.768+0000 7fed3bfff640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:19.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.769+0000 7fed3affd640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c155220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17073f5b5d847c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.769+0000 7fed3a7fc640 1 -- 172.21.15.17:0/960927110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30034070 con 0x7fed3c0c67f0 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.769+0000 7fed3a7fc640 1 -- 172.21.15.17:0/960927110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30030610 con 0x7fed3c0c67f0 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.769+0000 7fed3a7fc640 1 -- 172.21.15.17:0/960927110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300309e0 con 0x7fed3c0c67f0 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 -- 172.21.15.17:0/960927110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 msgr2=0x7fed3c155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c155220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002b960 comp rx=0 tx=0).stop 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 -- 172.21.15.17:0/960927110 shutdown_connections 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 --2- 172.21.15.17:0/960927110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c155220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 -- 172.21.15.17:0/960927110 >> 172.21.15.17:0/960927110 conn(0x7fed3c0aa6a0 msgr2=0x7fed3c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 -- 172.21.15.17:0/960927110 shutdown_connections 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.770+0000 7fed41de9640 1 -- 172.21.15.17:0/960927110 wait complete. 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.771+0000 7fed41de9640 1 Processor -- start 2024-03-06T07:22:19.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.771+0000 7fed41de9640 1 -- start start 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed3c1528d0 0x7fed3c1491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0a8250 con 0x7fed3c0c67f0 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c3390 con 0x7fed3c1528d0 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c1591a0 con 0x7fed3c0c4ef0 2024-03-06T07:22:19.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35254/0 (socket says 172.21.15.17:35254) 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 -- 172.21.15.17:0/1625740118 learned_addr learned my addr 172.21.15.17:0/1625740118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3bfff640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed3c1528d0 0x7fed3c1491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3b7fe640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 -- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 msgr2=0x7fed3c14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 -- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed3c1528d0 msgr2=0x7fed3c1491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed3c1528d0 0x7fed3c1491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 -- 172.21.15.17:0/1625740118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c1496e0 con 0x7fed3c0c67f0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3b7fe640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed3affd640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fed3002bf70 tx=0x7fed300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30034040 con 0x7fed3c0c67f0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30039a10 con 0x7fed3c0c67f0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c154fd0 con 0x7fed3c0c67f0 2024-03-06T07:22:19.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.772+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30039e20 con 0x7fed3c0c67f0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.773+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c155f20 con 0x7fed3c0c67f0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.773+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c156250 con 0x7fed3c0c67f0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.774+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed30037070 con 0x7fed3c0c67f0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.774+0000 7fed23fff640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 0x7fed2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.774+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fed300ac430 con 0x7fed3c0c67f0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.775+0000 7fed3b7fe640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 0x7fed2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.775+0000 7fed3b7fe640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 0x7fed2806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed2c000f60 tx=0x7fed2c007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:19.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.779+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed3007b050 con 0x7fed3c0c67f0 2024-03-06T07:22:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.968+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed3c0c8170 con 0x7fed3c0c67f0 2024-03-06T07:22:19.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.968+0000 7fed23fff640 1 -- 172.21.15.17:0/1625740118 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v63) v1 ==== 105+0+11 (secure 0 0 0) 0x7fed3c0c8170 con 0x7fed3c0c67f0 2024-03-06T07:22:19.970 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 msgr2=0x7fed2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 0x7fed2806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed2c000f60 tx=0x7fed2c007ca0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 msgr2=0x7fed3c148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fed3002bf70 tx=0x7fed300046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 shutdown_connections 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2806b4d0 0x7fed2806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed3c1528d0 0x7fed3c1491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed3c0c67f0 0x7fed3c148c60 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 --2- 172.21.15.17:0/1625740118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed3c0c4ef0 0x7fed3c14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 >> 172.21.15.17:0/1625740118 conn(0x7fed3c0aa6a0 msgr2=0x7fed3c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 shutdown_connections 2024-03-06T07:22:19.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:19.970+0000 7fed41de9640 1 -- 172.21.15.17:0/1625740118 wait complete. 2024-03-06T07:22:19.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:22:20.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 Processor -- start 2024-03-06T07:22:20.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 -- start start 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14bc0c5370 0x7f14bc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 0x7f14bc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc152dd0 con 0x7f14bc1529f0 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc159710 con 0x7f14bc0c5370 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.086+0000 7f14c2d20640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc0c68b0 con 0x7f14bc155530 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bbfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14bc0c5370 0x7f14bc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14c12bd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 0x7f14bc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14c0abc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f14bc1529f0 0x7f14bc0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:39416/0 (socket says 172.21.15.17:39416) 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14c0abc640 1 -- 172.21.15.17:0/4220521910 learned_addr learned my addr 172.21.15.17:0/4220521910 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bbfff640 1 -- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 msgr2=0x7f14bc1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:20.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bbfff640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 0x7f14bc1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bbfff640 1 -- 172.21.15.17:0/4220521910 >> v1:172.21.15.163:6789/0 conn(0x7f14bc1529f0 legacy=0x7f14bc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bb7fe640 1 -- 172.21.15.17:0/4220521910 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126569713 0 0) 0x7f14bc152dd0 con 0x7f14bc1529f0 2024-03-06T07:22:20.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14bbfff640 1 -- 172.21.15.17:0/4220521910 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc158b00 con 0x7f14bc0c5370 2024-03-06T07:22:20.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.087+0000 7f14c12bd640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 0x7f14bc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:20.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.088+0000 7f14bbfff640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0c5370 0x7f14bc0c57d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f14a80097a0 tx=0x7f14a802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=158e788722dc43c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.088+0000 7f14bb7fe640 1 -- 172.21.15.17:0/4220521910 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a80315a0 con 0x7f14bc0c5370 2024-03-06T07:22:20.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.088+0000 7f14bb7fe640 1 -- 172.21.15.17:0/4220521910 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8037070 con 0x7f14bc0c5370 2024-03-06T07:22:20.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.088+0000 7f14bb7fe640 1 -- 172.21.15.17:0/4220521910 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8031960 con 0x7f14bc0c5370 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 -- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0c5370 msgr2=0x7f14bc0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0c5370 0x7f14bc0c57d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f14a80097a0 tx=0x7f14a802b840 comp rx=0 tx=0).stop 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 -- 172.21.15.17:0/4220521910 shutdown_connections 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc155530 0x7f14bc1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 --2- 172.21.15.17:0/4220521910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0c5370 0x7f14bc0c57d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 -- 172.21.15.17:0/4220521910 >> 172.21.15.17:0/4220521910 conn(0x7f14bc0ac580 msgr2=0x7f14bc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 -- 172.21.15.17:0/4220521910 shutdown_connections 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.089+0000 7f14c2d20640 1 -- 172.21.15.17:0/4220521910 wait complete. 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.090+0000 7f14c2d20640 1 Processor -- start 2024-03-06T07:22:20.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.090+0000 7f14c2d20640 1 -- start start 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1529f0 0x7f14bc14dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 0x7f14bc1eab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1590c0 con 0x7f14bc0c5370 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc0c9000 con 0x7f14bc155530 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc1593a0 con 0x7f14bc1529f0 2024-03-06T07:22:20.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35256/0 (socket says 172.21.15.17:35256) 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 -- 172.21.15.17:0/3370439428 learned_addr learned my addr 172.21.15.17:0/3370439428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14bbfff640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1529f0 0x7f14bc14dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c12bd640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 0x7f14bc1eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 -- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1529f0 msgr2=0x7f14bc14dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1529f0 0x7f14bc14dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 -- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 msgr2=0x7f14bc1eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 0x7f14bc1eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 -- 172.21.15.17:0/3370439428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0be740 con 0x7f14bc0c5370 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c12bd640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 0x7f14bc1eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c0abc640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c4d0 tx=0x7f14b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b00074e0 con 0x7f14bc0c5370 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b00047e0 con 0x7f14bc0c5370 2024-03-06T07:22:20.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc0c1290 con 0x7f14bc0c5370 2024-03-06T07:22:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0004bd0 con 0x7f14bc0c5370 2024-03-06T07:22:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.091+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc0ae330 con 0x7f14bc0c5370 2024-03-06T07:22:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.092+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14bc0ae570 con 0x7f14bc0c5370 2024-03-06T07:22:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.093+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14b0011070 con 0x7f14bc0c5370 2024-03-06T07:22:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.093+0000 7f14b97fa640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 0x7f14a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.093+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f14b008a010 con 0x7f14bc0c5370 2024-03-06T07:22:20.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.094+0000 7f14bbfff640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 0x7f14a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.094+0000 7f14bbfff640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 0x7f14a006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f14a8036000 tx=0x7f14a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.098+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14bc0ae570 con 0x7f14bc0c5370 2024-03-06T07:22:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.280+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f14bc04f330 con 0x7f14bc0c5370 2024-03-06T07:22:20.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.280+0000 7f14b97fa640 1 -- 172.21.15.17:0/3370439428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v63) v1 ==== 101+0+11 (secure 0 0 0) 0x7f14b0058c30 con 0x7f14bc0c5370 2024-03-06T07:22:20.282 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 msgr2=0x7f14a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 0x7f14a006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f14a8036000 tx=0x7f14a8008030 comp rx=0 tx=0).stop 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 msgr2=0x7f14bc0bfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f14b000c4d0 tx=0x7f14b000c8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 shutdown_connections 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14a006b4d0 0x7f14a006d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc155530 0x7f14bc1eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1529f0 0x7f14bc14dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 --2- 172.21.15.17:0/3370439428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14bc0c5370 0x7f14bc0bfcb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 >> 172.21.15.17:0/3370439428 conn(0x7f14bc0ac580 msgr2=0x7f14bc151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 shutdown_connections 2024-03-06T07:22:20.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.282+0000 7f14c2d20640 1 -- 172.21.15.17:0/3370439428 wait complete. 2024-03-06T07:22:20.295 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 Processor -- start 2024-03-06T07:22:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 -- start start 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f311c0c88c0 0x7f311c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f311c158b60 0x7f311c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c0c73d0 con 0x7f311c14f360 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0c8dd0 con 0x7f311c0c88c0 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3120de2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0be2e0 con 0x7f311c158b60 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3113fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f311c0c88c0 0x7f311c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3113fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f311c0c88c0 0x7f311c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35258/0 (socket says 172.21.15.17:35258) 2024-03-06T07:22:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f3113fff640 1 -- 172.21.15.17:0/3528845850 learned_addr learned my addr 172.21.15.17:0/3528845850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.395+0000 7f311ad9d640 1 --2- 172.21.15.17:0/3528845850 >> v2:172.21.15.163:3300/0 conn(0x7f311c158b60 0x7f311c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.396+0000 7f3113fff640 1 -- 172.21.15.17:0/3528845850 >> v2:172.21.15.163:3300/0 conn(0x7f311c158b60 msgr2=0x7f311c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.396+0000 7f3113fff640 1 --2- 172.21.15.17:0/3528845850 >> v2:172.21.15.163:3300/0 conn(0x7f311c158b60 0x7f311c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.396+0000 7f3113fff640 1 -- 172.21.15.17:0/3528845850 >> v1:172.21.15.163:6789/0 conn(0x7f311c14f360 legacy=0x7f311c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.396+0000 7f3113fff640 1 -- 172.21.15.17:0/3528845850 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0c7670 con 0x7f311c0c88c0 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.396+0000 7f3113fff640 1 --2- 172.21.15.17:0/3528845850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0c88c0 0x7f311c155c10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f311c152d60 tx=0x7f310402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505d01f352e383eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3119d9b640 1 -- 172.21.15.17:0/3528845850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104035020 con 0x7f311c0c88c0 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3119d9b640 1 -- 172.21.15.17:0/3528845850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104031870 con 0x7f311c0c88c0 2024-03-06T07:22:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3119d9b640 1 -- 172.21.15.17:0/3528845850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104031c40 con 0x7f311c0c88c0 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 -- 172.21.15.17:0/3528845850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0c88c0 msgr2=0x7f311c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 --2- 172.21.15.17:0/3528845850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0c88c0 0x7f311c155c10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f311c152d60 tx=0x7f310402b910 comp rx=0 tx=0).stop 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 -- 172.21.15.17:0/3528845850 shutdown_connections 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 --2- 172.21.15.17:0/3528845850 >> v2:172.21.15.163:3300/0 conn(0x7f311c158b60 0x7f311c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 --2- 172.21.15.17:0/3528845850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c0c88c0 0x7f311c155c10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 -- 172.21.15.17:0/3528845850 >> 172.21.15.17:0/3528845850 conn(0x7f311c0ac570 msgr2=0x7f311c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 -- 172.21.15.17:0/3528845850 shutdown_connections 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.397+0000 7f3120de2640 1 -- 172.21.15.17:0/3528845850 wait complete. 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.398+0000 7f3120de2640 1 Processor -- start 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.398+0000 7f3120de2640 1 -- start start 2024-03-06T07:22:20.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f311c0c88c0 0x7f311c1eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f311c14f360 0x7f311c1eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0c5a50 con 0x7f311c158b60 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0c7c30 con 0x7f311c14f360 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3120de2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0c5bd0 con 0x7f311c0c88c0 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f311ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f311ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35274/0 (socket says 172.21.15.17:35274) 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f311ad9d640 1 -- 172.21.15.17:0/3099752214 learned_addr learned my addr 172.21.15.17:0/3099752214 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f3113fff640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f311c14f360 0x7f311c1eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.399+0000 7f311a59c640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f311c0c88c0 0x7f311c1eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 -- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f311c0c88c0 msgr2=0x7f311c1eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f311c0c88c0 0x7f311c1eb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 -- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f311c14f360 msgr2=0x7f311c1eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f311c14f360 0x7f311c1eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 -- 172.21.15.17:0/3099752214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c1eaeb0 con 0x7f311c158b60 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f311ad9d640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3114009dd0 tx=0x7f311400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114018020 con 0x7f311c158b60 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114014480 con 0x7f311c158b60 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c1ef320 con 0x7f311c158b60 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114014870 con 0x7f311c158b60 2024-03-06T07:22:20.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.400+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c0c0f60 con 0x7f311c158b60 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.401+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c04f330 con 0x7f311c158b60 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.401+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3114005d10 con 0x7f311c158b60 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.402+0000 7f3112ffd640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.402+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f311408aec0 con 0x7f311c158b60 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.402+0000 7f311a59c640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.402+0000 7f311a59c640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f311c152970 tx=0x7f310c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.406+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3114022070 con 0x7f311c158b60 2024-03-06T07:22:20.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.627+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f311c156030 con 0x7f311c158b60 2024-03-06T07:22:20.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.628+0000 7f3112ffd640 1 -- 172.21.15.17:0/3099752214 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4978 (secure 0 0 0) 0x7f311c1eaeb0 con 0x7f311c158b60 2024-03-06T07:22:20.629 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:20.630 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/36892111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":36892111},{"type":"v1","addr":"172.21.15.17:6835","nonce":36892111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6198,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2785438993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2785438993},{"type":"v1","addr":"172.21.15.17:6834","nonce":2785438993}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:19.822405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5375":{"gid":5375,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5375,"qdb_cluster":[5375]},"id":4}]} 2024-03-06T07:22:20.630 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 50 2024-03-06T07:22:20.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 msgr2=0x7f30fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f311c152970 tx=0x7f310c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:20.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 msgr2=0x7f311c1ec320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3114009dd0 tx=0x7f311400c490 comp rx=0 tx=0).stop 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 shutdown_connections 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f311c158b60 0x7f311c1ec320 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f311c14f360 0x7f311c1eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.629+0000 7f3120de2640 1 --2- 172.21.15.17:0/3099752214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f311c0c88c0 0x7f311c1eb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.630+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 >> 172.21.15.17:0/3099752214 conn(0x7f311c0ac570 msgr2=0x7f311c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.630+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 shutdown_connections 2024-03-06T07:22:20.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.630+0000 7f3120de2640 1 -- 172.21.15.17:0/3099752214 wait complete. 2024-03-06T07:22:20.642 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:20.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.741+0000 7f3fe4dc7640 1 Processor -- start 2024-03-06T07:22:20.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.741+0000 7f3fe4dc7640 1 -- start start 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fe4dc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0152520 0x7f3fe006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fe4dc7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fe0155c80 0x7f3fe00c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fe4dc7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00c1ae0 con 0x7f3fe00720a0 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fe4dc7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00c1dc0 con 0x7f3fe0155c80 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fe4dc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006e450 con 0x7f3fe0152520 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fddd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fe0155c80 0x7f3fe00c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fded9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3fe00720a0 0x7f3fe006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48392/0 (socket says 172.21.15.17:48392) 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fded9d640 1 -- 172.21.15.17:0/1482402026 learned_addr learned my addr 172.21.15.17:0/1482402026 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fde59c640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0152520 0x7f3fe006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fddd9b640 1 -- 172.21.15.17:0/1482402026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0152520 msgr2=0x7f3fe006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fddd9b640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0152520 0x7f3fe006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fddd9b640 1 -- 172.21.15.17:0/1482402026 >> v1:172.21.15.17:6789/0 conn(0x7f3fe00720a0 legacy=0x7f3fe006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fddd9b640 1 -- 172.21.15.17:0/1482402026 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe006e730 con 0x7f3fe0155c80 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.742+0000 7f3fdd59a640 1 -- 172.21.15.17:0/1482402026 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171288313 0 0) 0x7f3fe00c1ae0 con 0x7f3fe00720a0 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.743+0000 7f3fddd9b640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0155c80 0x7f3fe00c12b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097a0 tx=0x7f3fc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29d3a10562ca2cc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.743+0000 7f3fdd59a640 1 -- 172.21.15.17:0/1482402026 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8035020 con 0x7f3fe0155c80 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.743+0000 7f3fdd59a640 1 -- 172.21.15.17:0/1482402026 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc8031830 con 0x7f3fe0155c80 2024-03-06T07:22:20.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.743+0000 7f3fdd59a640 1 -- 172.21.15.17:0/1482402026 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8031c00 con 0x7f3fe0155c80 2024-03-06T07:22:20.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/1482402026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0155c80 msgr2=0x7f3fe00c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0155c80 0x7f3fe00c12b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097a0 tx=0x7f3fc802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/1482402026 shutdown_connections 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0155c80 0x7f3fe00c12b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/1482402026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0152520 0x7f3fe006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/1482402026 >> 172.21.15.17:0/1482402026 conn(0x7f3fe00c76b0 msgr2=0x7f3fe00c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/1482402026 shutdown_connections 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.744+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/1482402026 wait complete. 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.745+0000 7f3fe4dc7640 1 Processor -- start 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.745+0000 7f3fe4dc7640 1 -- start start 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00720a0 0x7f3fe01de640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0155c80 0x7f3fe01df780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006b6d0 con 0x7f3fe0152520 2024-03-06T07:22:20.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c17f0 con 0x7f3fe00720a0 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fe4dc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006b850 con 0x7f3fe0155c80 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41724/0 (socket says 172.21.15.17:41724) 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 -- 172.21.15.17:0/2436779955 learned_addr learned my addr 172.21.15.17:0/2436779955 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fded9d640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0155c80 0x7f3fe01df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fde59c640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00720a0 0x7f3fe01de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 -- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0155c80 msgr2=0x7f3fe01df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0155c80 0x7f3fe01df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 -- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00720a0 msgr2=0x7f3fe01de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00720a0 0x7f3fe01de640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 -- 172.21.15.17:0/2436779955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe01594e0 con 0x7f3fe0152520 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.746+0000 7f3fddd9b640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097a0 tx=0x7f3fc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.747+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8035070 con 0x7f3fe0152520 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.747+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc8004230 con 0x7f3fe0152520 2024-03-06T07:22:20.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.747+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe0159200 con 0x7f3fe0152520 2024-03-06T07:22:20.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.747+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc803a630 con 0x7f3fe0152520 2024-03-06T07:22:20.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.747+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01eba20 con 0x7f3fe0152520 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.748+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe004f330 con 0x7f3fe0152520 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.748+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fc8038020 con 0x7f3fe0152520 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.748+0000 7f3fd6ffd640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.748+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f3fc80ac010 con 0x7f3fe0152520 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.749+0000 7f3fde59c640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:20.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.749+0000 7f3fde59c640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011340 tx=0x7f3fd000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:20.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.753+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fc8040070 con 0x7f3fe0152520 2024-03-06T07:22:20.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.939+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fe00c0b70 con 0x7f3fe0152520 2024-03-06T07:22:20.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.940+0000 7f3fd6ffd640 1 -- 172.21.15.17:0/2436779955 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f3fc807a550 con 0x7f3fe0152520 2024-03-06T07:22:20.942 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:20.942 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":63,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:19.018373+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.17:6833/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6834/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6832/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:20.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 msgr2=0x7f3fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011340 tx=0x7f3fd000b040 comp rx=0 tx=0).stop 2024-03-06T07:22:20.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 msgr2=0x7f3fe01e0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:20.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097a0 tx=0x7f3fc80046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 shutdown_connections 2024-03-06T07:22:20.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe0155c80 0x7f3fe01df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe0152520 0x7f3fe01e0ae0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 --2- 172.21.15.17:0/2436779955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00720a0 0x7f3fe01de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:20.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 >> 172.21.15.17:0/2436779955 conn(0x7f3fe00c76b0 msgr2=0x7f3fe00c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:20.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 shutdown_connections 2024-03-06T07:22:20.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:20.942+0000 7f3fe4dc7640 1 -- 172.21.15.17:0/2436779955 wait complete. 2024-03-06T07:22:20.954 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:22:20.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.054+0000 7f535a0d4640 1 Processor -- start 2024-03-06T07:22:21.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.054+0000 7f535a0d4640 1 -- start start 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f535a0d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 0x7f5354155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f535a0d4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 0x7f5354158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f535a0d4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53540c5cc0 con 0x7f53540c5860 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f535a0d4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535406df40 con 0x7f53541586f0 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f535a0d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535406fdb0 con 0x7f53540c5480 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f5353fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 0x7f5354158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f53537fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 0x7f5354155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f5352ffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f53540c5860 0x7f535406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40628/0 (socket says 172.21.15.17:40628) 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.055+0000 7f5352ffd640 1 -- 172.21.15.17:0/2843246837 learned_addr learned my addr 172.21.15.17:0/2843246837 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53537fe640 1 -- 172.21.15.17:0/2843246837 >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 msgr2=0x7f5354158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53537fe640 1 --2- 172.21.15.17:0/2843246837 >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 0x7f5354158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53537fe640 1 -- 172.21.15.17:0/2843246837 >> v1:172.21.15.163:6789/0 conn(0x7f53540c5860 legacy=0x7f535406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53527fc640 1 -- 172.21.15.17:0/2843246837 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192589932 0 0) 0x7f53540c5cc0 con 0x7f53540c5860 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53537fe640 1 -- 172.21.15.17:0/2843246837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53541591c0 con 0x7f53540c5480 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f5353fff640 1 --2- 172.21.15.17:0/2843246837 >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 0x7f5354158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.056+0000 7f53537fe640 1 --2- 172.21.15.17:0/2843246837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 0x7f5354155ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53440097a0 tx=0x7f534402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecd36acaa502c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f53527fc640 1 -- 172.21.15.17:0/2843246837 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440315a0 con 0x7f53540c5480 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f53527fc640 1 -- 172.21.15.17:0/2843246837 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344037070 con 0x7f53540c5480 2024-03-06T07:22:21.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f53527fc640 1 -- 172.21.15.17:0/2843246837 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344031960 con 0x7f53540c5480 2024-03-06T07:22:21.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 -- 172.21.15.17:0/2843246837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 msgr2=0x7f5354155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 --2- 172.21.15.17:0/2843246837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 0x7f5354155ba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f53440097a0 tx=0x7f534402b840 comp rx=0 tx=0).stop 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 -- 172.21.15.17:0/2843246837 shutdown_connections 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 --2- 172.21.15.17:0/2843246837 >> v2:172.21.15.163:3300/0 conn(0x7f53541586f0 0x7f5354158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 --2- 172.21.15.17:0/2843246837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53540c5480 0x7f5354155ba0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 -- 172.21.15.17:0/2843246837 >> 172.21.15.17:0/2843246837 conn(0x7f535406b1a0 msgr2=0x7f535406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 -- 172.21.15.17:0/2843246837 shutdown_connections 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.057+0000 7f535a0d4640 1 -- 172.21.15.17:0/2843246837 wait complete. 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.058+0000 7f535a0d4640 1 Processor -- start 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 -- start start 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53540c5860 0x7f535414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53541586f0 0x7f535414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354155680 con 0x7f53540c5480 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53541553a0 con 0x7f53540c5860 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f535a0d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5354152a20 con 0x7f53541586f0 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f53537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f53537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41736/0 (socket says 172.21.15.17:41736) 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f53537fe640 1 -- 172.21.15.17:0/3951763936 learned_addr learned my addr 172.21.15.17:0/3951763936 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f5352ffd640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53540c5860 0x7f535414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.059+0000 7f5353fff640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53541586f0 0x7f535414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 -- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53541586f0 msgr2=0x7f535414cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53541586f0 0x7f535414cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 -- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53540c5860 msgr2=0x7f535414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53540c5860 0x7f535414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 -- 172.21.15.17:0/3951763936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53541506a0 con 0x7f53540c5480 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f53537fe640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5344034040 tx=0x7f53440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344004230 con 0x7f53540c5480 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53440043f0 con 0x7f53540c5480 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344003850 con 0x7f53540c5480 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53541eb580 con 0x7f53540c5480 2024-03-06T07:22:21.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.060+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5354157f70 con 0x7f53540c5480 2024-03-06T07:22:21.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.061+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535404f330 con 0x7f53540c5480 2024-03-06T07:22:21.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.062+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53440039f0 con 0x7f53540c5480 2024-03-06T07:22:21.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.062+0000 7f533bfff640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.062+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f534402f030 con 0x7f53540c5480 2024-03-06T07:22:21.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.062+0000 7f5352ffd640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.063+0000 7f5352ffd640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f535406da40 tx=0x7f5348007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.066+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5344041070 con 0x7f53540c5480 2024-03-06T07:22:21.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.283+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f535406e0c0 con 0x7f53540c5480 2024-03-06T07:22:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.283+0000 7f533bfff640 1 -- 172.21.15.17:0/3951763936 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4978 (secure 0 0 0) 0x7f534407aad0 con 0x7f53540c5480 2024-03-06T07:22:21.285 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:21.285 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/36892111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":36892111},{"type":"v1","addr":"172.21.15.17:6835","nonce":36892111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6198,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2785438993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2785438993},{"type":"v1","addr":"172.21.15.17:6834","nonce":2785438993}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:19.822405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5375":{"gid":5375,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5375,"qdb_cluster":[5375]},"id":4}]} 2024-03-06T07:22:21.285 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 50 2024-03-06T07:22:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 msgr2=0x7f534006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 0x7f534006d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f535406da40 tx=0x7f5348007d50 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 msgr2=0x7f535414fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5344034040 tx=0x7f53440046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 shutdown_connections 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534006b4d0 0x7f534006d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53541586f0 0x7f535414cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53540c5860 0x7f535414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 --2- 172.21.15.17:0/3951763936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53540c5480 0x7f535414fcf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 >> 172.21.15.17:0/3951763936 conn(0x7f535406b1a0 msgr2=0x7f535406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 shutdown_connections 2024-03-06T07:22:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.285+0000 7f535a0d4640 1 -- 172.21.15.17:0/3951763936 wait complete. 2024-03-06T07:22:21.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:21.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.397+0000 7f05a4dc7640 1 Processor -- start 2024-03-06T07:22:21.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.397+0000 7f05a4dc7640 1 -- start start 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f05a4dc7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05a0150920 0x7f05a00c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f05a4dc7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f05a00c2e70 0x7f05a015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f05a4dc7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a0067340 con 0x7f05a006c720 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f05a4dc7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a0150d00 con 0x7f05a0150920 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f05a4dc7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a00c2010 con 0x7f05a00c2e70 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05a0150920 0x7f05a00c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f0597fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f05a006c720 0x7f05a0071980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48400/0 (socket says 172.21.15.17:48400) 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f0597fff640 1 -- 172.21.15.17:0/3623792800 learned_addr learned my addr 172.21.15.17:0/3623792800 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:21.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059ed9d640 1 --2- 172.21.15.17:0/3623792800 >> v2:172.21.15.163:3300/0 conn(0x7f05a00c2e70 0x7f05a015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059e59c640 1 -- 172.21.15.17:0/3623792800 >> v2:172.21.15.163:3300/0 conn(0x7f05a00c2e70 msgr2=0x7f05a015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059e59c640 1 --2- 172.21.15.17:0/3623792800 >> v2:172.21.15.163:3300/0 conn(0x7f05a00c2e70 0x7f05a015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059e59c640 1 -- 172.21.15.17:0/3623792800 >> v1:172.21.15.17:6789/0 conn(0x7f05a006c720 legacy=0x7f05a0071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059dd9b640 1 -- 172.21.15.17:0/3623792800 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811900829 0 0) 0x7f05a0067340 con 0x7f05a006c720 2024-03-06T07:22:21.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.398+0000 7f059e59c640 1 -- 172.21.15.17:0/3623792800 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0067060 con 0x7f05a0150920 2024-03-06T07:22:21.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.399+0000 7f059e59c640 1 --2- 172.21.15.17:0/3623792800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a00c2930 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f05a01543f0 tx=0x7f059002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4302b9ae43cf01a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.399+0000 7f059dd9b640 1 -- 172.21.15.17:0/3623792800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900315a0 con 0x7f05a0150920 2024-03-06T07:22:21.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.399+0000 7f059dd9b640 1 -- 172.21.15.17:0/3623792800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590037070 con 0x7f05a0150920 2024-03-06T07:22:21.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.399+0000 7f059dd9b640 1 -- 172.21.15.17:0/3623792800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590031960 con 0x7f05a0150920 2024-03-06T07:22:21.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 -- 172.21.15.17:0/3623792800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 msgr2=0x7f05a00c2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/3623792800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a00c2930 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f05a01543f0 tx=0x7f059002b840 comp rx=0 tx=0).stop 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 -- 172.21.15.17:0/3623792800 shutdown_connections 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/3623792800 >> v2:172.21.15.163:3300/0 conn(0x7f05a00c2e70 0x7f05a015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/3623792800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a00c2930 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 -- 172.21.15.17:0/3623792800 >> 172.21.15.17:0/3623792800 conn(0x7f05a00c70b0 msgr2=0x7f05a00c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 -- 172.21.15.17:0/3623792800 shutdown_connections 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.400+0000 7f05a4dc7640 1 -- 172.21.15.17:0/3623792800 wait complete. 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.401+0000 7f05a4dc7640 1 Processor -- start 2024-03-06T07:22:21.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.401+0000 7f05a4dc7640 1 -- start start 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.401+0000 7f05a4dc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 0x7f05a0151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05a00c2e70 0x7f05a01e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0073210 con 0x7f05a0150920 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0073090 con 0x7f05a00c2e70 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00c1b40 con 0x7f05a006c720 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41754/0 (socket says 172.21.15.17:41754) 2024-03-06T07:22:21.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 -- 172.21.15.17:0/2410242127 learned_addr learned my addr 172.21.15.17:0/2410242127 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f0597fff640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05a00c2e70 0x7f05a01e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059e59c640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 0x7f05a0151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 -- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 msgr2=0x7f05a0151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 0x7f05a0151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 -- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05a00c2e70 msgr2=0x7f05a01e90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05a00c2e70 0x7f05a01e90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 -- 172.21.15.17:0/2410242127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0149490 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059e59c640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 0x7f05a0151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f059ed9d640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0598009d30 tx=0x7f059800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598018020 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0598014480 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598014870 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a0149770 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.402+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a01ebc80 con 0x7f05a0150920 2024-03-06T07:22:21.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.403+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a004f330 con 0x7f05a0150920 2024-03-06T07:22:21.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.404+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0598005d10 con 0x7f05a0150920 2024-03-06T07:22:21.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.404+0000 7f0596ffd640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 0x7f058006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.405+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f059808ae40 con 0x7f05a0150920 2024-03-06T07:22:21.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.405+0000 7f059e59c640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 0x7f058006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.405+0000 7f059e59c640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 0x7f058006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0590036000 tx=0x7f0590008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.409+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0598022070 con 0x7f05a0150920 2024-03-06T07:22:21.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.632+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05a006eca0 con 0x7f05a0150920 2024-03-06T07:22:21.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.632+0000 7f0596ffd640 1 -- 172.21.15.17:0/2410242127 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4978 (secure 0 0 0) 0x7f0598010040 con 0x7f05a0150920 2024-03-06T07:22:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:21.634 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/36892111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":36892111},{"type":"v1","addr":"172.21.15.17:6835","nonce":36892111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6198,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2785438993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2785438993},{"type":"v1","addr":"172.21.15.17:6834","nonce":2785438993}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:21:59.395822+0000","modified":"2024-03-06T07:22:19.822405+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5375":{"gid":5375,"name":"d","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.163:6836/3777861734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3777861734},{"type":"v1","addr":"172.21.15.163:6836","nonce":3777861734}]},"join_fscid":4,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5375,"qdb_cluster":[5375]},"id":4}]} 2024-03-06T07:22:21.634 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 50 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 msgr2=0x7f058006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 0x7f058006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0590036000 tx=0x7f0590008030 comp rx=0 tx=0).stop 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 msgr2=0x7f05a006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0598009d30 tx=0x7f059800c490 comp rx=0 tx=0).stop 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 shutdown_connections 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f058006b4d0 0x7f058006d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05a0150920 0x7f05a006b230 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05a00c2e70 0x7f05a01e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 --2- 172.21.15.17:0/2410242127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05a006c720 0x7f05a0151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 >> 172.21.15.17:0/2410242127 conn(0x7f05a00c70b0 msgr2=0x7f05a014a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 shutdown_connections 2024-03-06T07:22:21.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.634+0000 7f05a4dc7640 1 -- 172.21.15.17:0/2410242127 wait complete. 2024-03-06T07:22:21.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.746+0000 7f5a9b483640 1 Processor -- start 2024-03-06T07:22:21.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.746+0000 7f5a9b483640 1 -- start start 2024-03-06T07:22:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9b483640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9b483640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a940c3280 0x7f5a940c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9b483640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a94158800 con 0x7f5a941551e0 2024-03-06T07:22:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9b483640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a94158ae0 con 0x7f5a940c3280 2024-03-06T07:22:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9b483640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94158dc0 con 0x7f5a9406d9f0 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a940c3280 0x7f5a940c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a940c3280 0x7f5a940c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41768/0 (socket says 172.21.15.17:41768) 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3173292067 learned_addr learned my addr 172.21.15.17:0/3173292067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a9921f640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3173292067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 msgr2=0x7f5a940a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3173292067 >> v1:172.21.15.163:6789/0 conn(0x7f5a941551e0 legacy=0x7f5a94155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3173292067 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a941590a0 con 0x7f5a940c3280 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.747+0000 7f5a8bfff640 1 -- 172.21.15.17:0/3173292067 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685315164 0 0) 0x7f5a94158800 con 0x7f5a941551e0 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.748+0000 7f5a98a1e640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a940c36e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009870 tx=0x7f5a8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996331d3992e0a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.748+0000 7f5a8bfff640 1 -- 172.21.15.17:0/3173292067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84035020 con 0x7f5a940c3280 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.748+0000 7f5a8bfff640 1 -- 172.21.15.17:0/3173292067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84031640 con 0x7f5a940c3280 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.748+0000 7f5a8bfff640 1 -- 172.21.15.17:0/3173292067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8402d040 con 0x7f5a940c3280 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 -- 172.21.15.17:0/3173292067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 msgr2=0x7f5a940c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a940c36e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009870 tx=0x7f5a8402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 -- 172.21.15.17:0/3173292067 shutdown_connections 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a940c36e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3173292067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 -- 172.21.15.17:0/3173292067 >> 172.21.15.17:0/3173292067 conn(0x7f5a940aa630 msgr2=0x7f5a940aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 -- 172.21.15.17:0/3173292067 shutdown_connections 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.749+0000 7f5a9b483640 1 -- 172.21.15.17:0/3173292067 wait complete. 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 Processor -- start 2024-03-06T07:22:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 -- start start 2024-03-06T07:22:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a941551e0 0x7f5a941eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9406df50 con 0x7f5a940c3280 2024-03-06T07:22:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9406ddd0 con 0x7f5a941551e0 2024-03-06T07:22:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.750+0000 7f5a9b483640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a940a8210 con 0x7f5a9406d9f0 2024-03-06T07:22:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41772/0 (socket says 172.21.15.17:41772) 2024-03-06T07:22:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3039206298 learned_addr learned my addr 172.21.15.17:0/3039206298 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a9921f640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a99a20640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a941551e0 0x7f5a941eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 msgr2=0x7f5a940a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a941551e0 msgr2=0x7f5a941eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a941551e0 0x7f5a941eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 -- 172.21.15.17:0/3039206298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940a9170 con 0x7f5a940c3280 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a98a1e640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a8402b930 tx=0x7f5a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84040070 con 0x7f5a940c3280 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8403e070 con 0x7f5a940c3280 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a940a8e90 con 0x7f5a940c3280 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84035040 con 0x7f5a940c3280 2024-03-06T07:22:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.751+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a940a8800 con 0x7f5a940c3280 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.752+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9404f330 con 0x7f5a940c3280 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.753+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a84038020 con 0x7f5a940c3280 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.753+0000 7f5a89ffb640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 0x7f5a7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.753+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f5a840abc00 con 0x7f5a940c3280 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.753+0000 7f5a9921f640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 0x7f5a7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.754+0000 7f5a9921f640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 0x7f5a7806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009aa0 tx=0x7f5a90009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:21.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.758+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a840b2080 con 0x7f5a940c3280 2024-03-06T07:22:21.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.940+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a94073240 con 0x7f5a940c3280 2024-03-06T07:22:21.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.941+0000 7f5a89ffb640 1 -- 172.21.15.17:0/3039206298 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f5a8407a1d0 con 0x7f5a940c3280 2024-03-06T07:22:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:21.944 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":63,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:19.018373+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:21:57.664157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:21:58.336311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.17:6833/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6834/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.17:6832/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:21.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 msgr2=0x7f5a7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 0x7f5a7806d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009aa0 tx=0x7f5a90009190 comp rx=0 tx=0).stop 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 msgr2=0x7f5a941eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a8402b930 tx=0x7f5a840046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 shutdown_connections 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b500 0x7f5a7806d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a941551e0 0x7f5a941eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a940c3280 0x7f5a941eac80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 --2- 172.21.15.17:0/3039206298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a9406d9f0 0x7f5a940a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:21.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 >> 172.21.15.17:0/3039206298 conn(0x7f5a940aa630 msgr2=0x7f5a940c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:21.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 shutdown_connections 2024-03-06T07:22:21.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:21.943+0000 7f5a9b483640 1 -- 172.21.15.17:0/3039206298 wait complete. 2024-03-06T07:22:21.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:22:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.057+0000 7f6df0f5c640 1 Processor -- start 2024-03-06T07:22:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.057+0000 7f6df0f5c640 1 -- start start 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6df0f5c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dec06d680 0x7f6dec152060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6df0f5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6dec0c28f0 0x7f6dec14e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6df0f5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dec0c1140 con 0x7f6dec072f40 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6df0f5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec0c1d50 con 0x7f6dec0c28f0 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6df0f5c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec1525a0 con 0x7f6dec06d680 2024-03-06T07:22:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6dead9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6dec0c28f0 0x7f6dec14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.058+0000 7f6de9d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6dec072f40 0x7f6dec0c21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48402/0 (socket says 172.21.15.17:48402) 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6de9d9b640 1 -- 172.21.15.17:0/1126860492 learned_addr learned my addr 172.21.15.17:0/1126860492 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6dea59c640 1 --2- 172.21.15.17:0/1126860492 >> v2:172.21.15.163:3300/0 conn(0x7f6dec06d680 0x7f6dec152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6de959a640 1 -- 172.21.15.17:0/1126860492 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517115952 0 0) 0x7f6dec0c1140 con 0x7f6dec072f40 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6de959a640 1 -- 172.21.15.17:0/1126860492 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd4003520 con 0x7f6dec072f40 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6dead9d640 1 -- 172.21.15.17:0/1126860492 >> v2:172.21.15.163:3300/0 conn(0x7f6dec06d680 msgr2=0x7f6dec152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6dead9d640 1 --2- 172.21.15.17:0/1126860492 >> v2:172.21.15.163:3300/0 conn(0x7f6dec06d680 0x7f6dec152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6dead9d640 1 -- 172.21.15.17:0/1126860492 >> v1:172.21.15.17:6789/0 conn(0x7f6dec072f40 legacy=0x7f6dec0c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.059+0000 7f6dead9d640 1 -- 172.21.15.17:0/1126860492 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0c1420 con 0x7f6dec0c28f0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6dead9d640 1 --2- 172.21.15.17:0/1126860492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec0c28f0 0x7f6dec14e440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009dc0 tx=0x7f6de402be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d9bc13c431b7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6de959a640 1 -- 172.21.15.17:0/1126860492 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4039070 con 0x7f6dec0c28f0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6de959a640 1 -- 172.21.15.17:0/1126860492 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de402cbb0 con 0x7f6dec0c28f0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6de959a640 1 -- 172.21.15.17:0/1126860492 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4034440 con 0x7f6dec0c28f0 2024-03-06T07:22:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1126860492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec0c28f0 msgr2=0x7f6dec14e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1126860492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec0c28f0 0x7f6dec14e440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009dc0 tx=0x7f6de402be00 comp rx=0 tx=0).stop 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1126860492 shutdown_connections 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1126860492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec0c28f0 0x7f6dec14e440 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1126860492 >> v2:172.21.15.163:3300/0 conn(0x7f6dec06d680 0x7f6dec152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1126860492 >> 172.21.15.17:0/1126860492 conn(0x7f6dec06b1a0 msgr2=0x7f6dec06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1126860492 shutdown_connections 2024-03-06T07:22:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.060+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1126860492 wait complete. 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 Processor -- start 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 -- start start 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dec06d680 0x7f6dec0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dec0c28f0 0x7f6dec1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c2d80 con 0x7f6dec072f40 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0c19e0 con 0x7f6dec0c28f0 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6df0f5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec06da60 con 0x7f6dec06d680 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6de9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6de9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41784/0 (socket says 172.21.15.17:41784) 2024-03-06T07:22:22.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6de9d9b640 1 -- 172.21.15.17:0/1586970194 learned_addr learned my addr 172.21.15.17:0/1586970194 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.062+0000 7f6dea59c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dec06d680 0x7f6dec0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6dead9d640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dec0c28f0 0x7f6dec1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 -- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dec06d680 msgr2=0x7f6dec0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dec06d680 0x7f6dec0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 -- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dec0c28f0 msgr2=0x7f6dec1430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dec0c28f0 0x7f6dec1430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 -- 172.21.15.17:0/1586970194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec148d00 con 0x7f6dec072f40 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6de9d9b640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6de000c4d0 tx=0x7f6de000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0017070 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0005ce0 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec14fbe0 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0007270 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.063+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec148a20 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.064+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec04f330 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.064+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6de001a070 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.065+0000 7f6ddaffd640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 0x7f6dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.065+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f6de00896c0 con 0x7f6dec072f40 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.065+0000 7f6dea59c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 0x7f6dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.066+0000 7f6dea59c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 0x7f6dcc06d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009b40 tx=0x7f6ddc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.069+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6de0057c90 con 0x7f6dec072f40 2024-03-06T07:22:22.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.288+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f6dec06b7d0 con 0x7f6dec072f40 2024-03-06T07:22:22.290 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:22:22.289+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:22.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.290+0000 7f6ddaffd640 1 -- 172.21.15.17:0/1586970194 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 163+0+0 (secure 0 0 0) 0x7f6de005bd30 con 0x7f6dec072f40 2024-03-06T07:22:22.292 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:22:22.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 msgr2=0x7f6dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 0x7f6dcc06d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009b40 tx=0x7f6ddc009190 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 msgr2=0x7f6dec142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6de000c4d0 tx=0x7f6de000c990 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 shutdown_connections 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6dcc06b4d0 0x7f6dcc06d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dec0c28f0 0x7f6dec1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6dec072f40 0x7f6dec142b90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 --2- 172.21.15.17:0/1586970194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dec06d680 0x7f6dec0c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 >> 172.21.15.17:0/1586970194 conn(0x7f6dec06b1a0 msgr2=0x7f6dec149780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 shutdown_connections 2024-03-06T07:22:22.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.293+0000 7f6df0f5c640 1 -- 172.21.15.17:0/1586970194 wait complete. 2024-03-06T07:22:22.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:22:22.319 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:22:22.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.407+0000 7f428ee9a640 1 Processor -- start 2024-03-06T07:22:22.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.407+0000 7f428ee9a640 1 -- start start 2024-03-06T07:22:22.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428ee9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f42880c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428ee9a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42880c6710 0x7f42881554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428ee9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4288152b10 con 0x7f42880c3750 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428ee9a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4288156110 con 0x7f42880c6710 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428ee9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4288158ad0 con 0x7f4288152730 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42880c6710 0x7f42881554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42880c6710 0x7f42881554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41796/0 (socket says 172.21.15.17:41796) 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 -- 172.21.15.17:0/2872796448 learned_addr learned my addr 172.21.15.17:0/2872796448 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f428cc36640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f42880c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 -- 172.21.15.17:0/2872796448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 msgr2=0x7f42880c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f42880c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 -- 172.21.15.17:0/2872796448 >> v1:172.21.15.163:6789/0 conn(0x7f42880c3750 legacy=0x7f4288155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.408+0000 7f4287fff640 1 -- 172.21.15.17:0/2872796448 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288158db0 con 0x7f42880c6710 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.409+0000 7f4287fff640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881554c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4278009e50 tx=0x7f427802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91d6ff8b2d1e61d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.409+0000 7f42877fe640 1 -- 172.21.15.17:0/2872796448 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427802ca70 con 0x7f42880c6710 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.409+0000 7f42877fe640 1 -- 172.21.15.17:0/2872796448 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427802cc10 con 0x7f42880c6710 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.409+0000 7f42877fe640 1 -- 172.21.15.17:0/2872796448 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42780343f0 con 0x7f42880c6710 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 -- 172.21.15.17:0/2872796448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 msgr2=0x7f42881554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881554c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4278009e50 tx=0x7f427802bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 -- 172.21.15.17:0/2872796448 shutdown_connections 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881554c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 --2- 172.21.15.17:0/2872796448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f42880c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 -- 172.21.15.17:0/2872796448 >> 172.21.15.17:0/2872796448 conn(0x7f42880aa5c0 msgr2=0x7f42880aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 -- 172.21.15.17:0/2872796448 shutdown_connections 2024-03-06T07:22:22.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.410+0000 7f428ee9a640 1 -- 172.21.15.17:0/2872796448 wait complete. 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.411+0000 7f428ee9a640 1 Processor -- start 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.411+0000 7f428ee9a640 1 -- start start 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42880c3750 0x7f42881ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f4288147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881586f0 con 0x7f42880c6710 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288159370 con 0x7f42880c3750 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428ee9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4288158870 con 0x7f4288152730 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41806/0 (socket says 172.21.15.17:41806) 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 -- 172.21.15.17:0/1437451964 learned_addr learned my addr 172.21.15.17:0/1437451964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:22.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428cc36640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42880c3750 0x7f42881ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f428d437640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f4288147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 -- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 msgr2=0x7f4288147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f4288147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 -- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42880c3750 msgr2=0x7f42881ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42880c3750 0x7f42881ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 -- 172.21.15.17:0/1437451964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288157f30 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.412+0000 7f4287fff640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4278036040 tx=0x7f4278004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278043020 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427803e450 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428806ff50 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278009510 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42881eecc0 con 0x7f42880c6710 2024-03-06T07:22:22.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.413+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428804f330 con 0x7f42880c6710 2024-03-06T07:22:22.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.414+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4278046020 con 0x7f42880c6710 2024-03-06T07:22:22.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.414+0000 7f42857fa640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 0x7f426806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:22.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.414+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f42780acb20 con 0x7f42880c6710 2024-03-06T07:22:22.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.415+0000 7f428cc36640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 0x7f426806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:22.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.415+0000 7f428cc36640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 0x7f426806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4280009b40 tx=0x7f4280009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:22.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.419+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f427804d030 con 0x7f42880c6710 2024-03-06T07:22:22.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:22.636+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4288073330 con 0x7f42880c6710 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.292+0000 7f42857fa640 1 -- 172.21.15.17:0/1437451964 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 105+0+0 (secure 0 0 0) 0x7f427807b020 con 0x7f42880c6710 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 msgr2=0x7f426806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 0x7f426806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4280009b40 tx=0x7f4280009190 comp rx=0 tx=0).stop 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 msgr2=0x7f42881477d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4278036040 tx=0x7f4278004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 shutdown_connections 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f426806b500 0x7f426806d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4288152730 0x7f4288147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42880c6710 0x7f42881477d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 --2- 172.21.15.17:0/1437451964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42880c3750 0x7f42881ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 >> 172.21.15.17:0/1437451964 conn(0x7f42880aa5c0 msgr2=0x7f428814ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:23.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 shutdown_connections 2024-03-06T07:22:23.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.295+0000 7f428ee9a640 1 -- 172.21.15.17:0/1437451964 wait complete. 2024-03-06T07:22:23.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:22:23.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.419+0000 7fa74d75c640 1 Processor -- start 2024-03-06T07:22:23.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.419+0000 7fa74d75c640 1 -- start start 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa74d75c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7480c7470 0x7fa748155d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa74d75c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa748156260 con 0x7fa748159230 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa74d75c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7480c7a10 con 0x7fa748152860 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa74d75c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa748158da0 con 0x7fa7480c7470 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa7477fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa748159230 0x7fa7480c53e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48418/0 (socket says 172.21.15.17:48418) 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa7477fe640 1 -- 172.21.15.17:0/1633835662 learned_addr learned my addr 172.21.15.17:0/1633835662 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa7467fc640 1 --2- 172.21.15.17:0/1633835662 >> v2:172.21.15.163:3300/0 conn(0x7fa7480c7470 0x7fa748155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004130499 0 0) 0x7fa738001490 con 0x7fa748159230 2024-03-06T07:22:23.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa730003520 con 0x7fa748159230 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 321724767 0 0) 0x7fa730003520 con 0x7fa748159230 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 >> v2:172.21.15.163:3300/0 conn(0x7fa7480c7470 msgr2=0x7fa748155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 --2- 172.21.15.17:0/1633835662 >> v2:172.21.15.163:3300/0 conn(0x7fa7480c7470 0x7fa748155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 >> v1:172.21.15.163:6789/0 conn(0x7fa748152860 legacy=0x7fa7480c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480c7cf0 con 0x7fa748159230 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fa738001bc0 con 0x7fa748159230 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 >> v1:172.21.15.17:6789/0 conn(0x7fa748159230 legacy=0x7fa7480c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300039b0 0x7fa730003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 0x7fa7300048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa730005140 0x7fa73000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748158da0 con 0x7fa7300043d0 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73000c090 con 0x7fa7300039b0 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.420+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73000c290 con 0x7fa730005140 2024-03-06T07:22:23.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 0x7fa7300048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa746ffd640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300039b0 0x7fa730003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7477fe640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa730005140 0x7fa73000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 -- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa730005140 msgr2=0x7fa73000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa730005140 0x7fa73000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 -- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300039b0 msgr2=0x7fa730003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300039b0 0x7fa730003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7467fc640 1 -- 172.21.15.17:0/1633835662 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730003520 con 0x7fa7300043d0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.421+0000 7fa7477fe640 1 -- 172.21.15.17:0/1633835662 reap_dead start 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa7467fc640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 0x7fa7300048a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009ac0 tx=0x7fa730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9de8e23fe0d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c039070 con 0x7fa7300043d0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c02c6b0 con 0x7fa7300043d0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa745ffb640 1 -- 172.21.15.17:0/1633835662 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c009690 con 0x7fa7300043d0 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 -- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 msgr2=0x7fa7300048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 0x7fa7300048a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009ac0 tx=0x7fa730005830 comp rx=0 tx=0).stop 2024-03-06T07:22:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 -- 172.21.15.17:0/1633835662 shutdown_connections 2024-03-06T07:22:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 --2- 172.21.15.17:0/1633835662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7300043d0 0x7fa7300048a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 -- 172.21.15.17:0/1633835662 >> 172.21.15.17:0/1633835662 conn(0x7fa7480ac630 msgr2=0x7fa7480aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 -- 172.21.15.17:0/1633835662 shutdown_connections 2024-03-06T07:22:23.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.422+0000 7fa74d75c640 1 -- 172.21.15.17:0/1633835662 wait complete. 2024-03-06T07:22:23.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 Processor -- start 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 -- start start 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300043d0 0x7fa74814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7480b0390 0x7fa7481ea670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748159680 con 0x7fa7480ab970 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748152c40 con 0x7fa7300043d0 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa74d75c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa748159800 con 0x7fa7480b0390 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa7467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa7467fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41818/0 (socket says 172.21.15.17:41818) 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.424+0000 7fa7467fc640 1 -- 172.21.15.17:0/998935467 learned_addr learned my addr 172.21.15.17:0/998935467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 -- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7480b0390 msgr2=0x7fa7481ea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7480b0390 0x7fa7481ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 -- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300043d0 msgr2=0x7fa74814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300043d0 0x7fa74814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 -- 172.21.15.17:0/998935467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7481583c0 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa7467fc640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa73c037040 tx=0x7fa73c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c043020 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c02cd90 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7481586a0 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c039040 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.425+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa748157530 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.426+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74804f330 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.427+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa73c046020 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.427+0000 7fa72f7fe640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 0x7fa73406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.427+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6535+0+0 (secure 0 0 0) 0x7fa73c0ab790 con 0x7fa7480ab970 2024-03-06T07:22:23.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.427+0000 7fa746ffd640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 0x7fa73406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:23.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.428+0000 7fa746ffd640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 0x7fa73406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa7400018b0 tx=0x7fa740009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:23.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.431+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa73c07a480 con 0x7fa7480ab970 2024-03-06T07:22:23.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:23.612+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7480c8110 con 0x7fa7480ab970 2024-03-06T07:22:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.301+0000 7fa72f7fe640 1 -- 172.21.15.17:0/998935467 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v65) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa73c079d80 con 0x7fa7480ab970 2024-03-06T07:22:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:22:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 msgr2=0x7fa73406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 0x7fa73406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa7400018b0 tx=0x7fa740009040 comp rx=0 tx=0).stop 2024-03-06T07:22:24.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 msgr2=0x7fa7480afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa73c037040 tx=0x7fa73c004730 comp rx=0 tx=0).stop 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 shutdown_connections 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa73406b4d0 0x7fa73406d990 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa7400018b0 tx=0x7fa740009040 comp rx=0 tx=0).stop 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7480b0390 0x7fa7481ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7480ab970 0x7fa7480afe50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 --2- 172.21.15.17:0/998935467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7300043d0 0x7fa74814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 >> 172.21.15.17:0/998935467 conn(0x7fa7480ac630 msgr2=0x7fa748151b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.304+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 shutdown_connections 2024-03-06T07:22:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.305+0000 7fa74d75c640 1 -- 172.21.15.17:0/998935467 wait complete. 2024-03-06T07:22:24.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:22:24.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.430+0000 7fb72b4e1640 1 Processor -- start 2024-03-06T07:22:24.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.430+0000 7fb72b4e1640 1 -- start start 2024-03-06T07:22:24.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb72b4e1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7240c2900 0x7fb7240c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb72b4e1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb72415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb72b4e1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb724159740 con 0x7fb724152920 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb72b4e1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7241551d0 con 0x7fb7240c2900 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb72b4e1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240c8ab0 con 0x7fb7240c80d0 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7240c2900 0x7fb7240c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7240c2900 0x7fb7240c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41834/0 (socket says 172.21.15.17:41834) 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 -- 172.21.15.17:0/3874603013 learned_addr learned my addr 172.21.15.17:0/3874603013 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb729a7e640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb72415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 -- 172.21.15.17:0/3874603013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 msgr2=0x7fb72415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb72415fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.431+0000 7fb728a7c640 1 -- 172.21.15.17:0/3874603013 >> v1:172.21.15.163:6789/0 conn(0x7fb724152920 legacy=0x7fb724155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.432+0000 7fb728a7c640 1 -- 172.21.15.17:0/3874603013 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7240c8d90 con 0x7fb7240c2900 2024-03-06T07:22:24.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.432+0000 7fb728a7c640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7240c2900 0x7fb7240c7b90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb714009870 tx=0x7fb71402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599b5178619798d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.432+0000 7fb71bfff640 1 -- 172.21.15.17:0/3874603013 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7140317c0 con 0x7fb7240c2900 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.432+0000 7fb71bfff640 1 -- 172.21.15.17:0/3874603013 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714037070 con 0x7fb7240c2900 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.432+0000 7fb71bfff640 1 -- 172.21.15.17:0/3874603013 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714031b80 con 0x7fb7240c2900 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 -- 172.21.15.17:0/3874603013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7240c2900 msgr2=0x7fb7240c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7240c2900 0x7fb7240c7b90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb714009870 tx=0x7fb71402b860 comp rx=0 tx=0).stop 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 -- 172.21.15.17:0/3874603013 shutdown_connections 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb72415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/3874603013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7240c2900 0x7fb7240c7b90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 -- 172.21.15.17:0/3874603013 >> 172.21.15.17:0/3874603013 conn(0x7fb7240aa5c0 msgr2=0x7fb7240aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:24.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 -- 172.21.15.17:0/3874603013 shutdown_connections 2024-03-06T07:22:24.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.433+0000 7fb72b4e1640 1 -- 172.21.15.17:0/3874603013 wait complete. 2024-03-06T07:22:24.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.434+0000 7fb72b4e1640 1 Processor -- start 2024-03-06T07:22:24.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.434+0000 7fb72b4e1640 1 -- start start 2024-03-06T07:22:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7240c2900 0x7fb7240a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb7241eac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724156080 con 0x7fb724152920 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb724155450 con 0x7fb7240c2900 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72b4e1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7240c86c0 con 0x7fb7240c80d0 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41838/0 (socket says 172.21.15.17:41838) 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 -- 172.21.15.17:0/1988626480 learned_addr learned my addr 172.21.15.17:0/1988626480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb728a7c640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb7241eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb72927d640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7240c2900 0x7fb7240a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 -- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 msgr2=0x7fb7241eac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb7241eac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 -- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7240c2900 msgr2=0x7fb7240a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7240c2900 0x7fb7240a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.435+0000 7fb729a7e640 1 -- 172.21.15.17:0/1988626480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724157b30 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb729a7e640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb72000cd70 tx=0x7fb72000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720003b90 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb720003d30 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7241eebd0 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720004240 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.436+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7240a9db0 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.437+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb724157e40 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.437+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb7200043e0 con 0x7fb724152920 2024-03-06T07:22:24.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.438+0000 7fb719ffb640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 0x7fb70806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.438+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6095+0+0 (secure 0 0 0) 0x7fb72008ace0 con 0x7fb724152920 2024-03-06T07:22:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.438+0000 7fb72927d640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 0x7fb70806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:24.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.438+0000 7fb72927d640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 0x7fb70806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006fd0 tx=0x7fb71c009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:24.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.442+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb720059420 con 0x7fb724152920 2024-03-06T07:22:24.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:24.621+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb72404f330 con 0x7fb724152920 2024-03-06T07:22:25.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.303+0000 7fb719ffb640 1 -- 172.21.15.17:0/1988626480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v66) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb72005d4c0 con 0x7fb724152920 2024-03-06T07:22:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' removed 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.306+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 msgr2=0x7fb70806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.306+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 0x7fb70806d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006fd0 tx=0x7fb71c009060 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 msgr2=0x7fb7241eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb72000cd70 tx=0x7fb72000a660 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 shutdown_connections 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70806b4d0 0x7fb70806d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb724152920 0x7fb7241eb190 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7240c80d0 0x7fb7241eac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 --2- 172.21.15.17:0/1988626480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7240c2900 0x7fb7240a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 >> 172.21.15.17:0/1988626480 conn(0x7fb7240aa5c0 msgr2=0x7fb7240c5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 shutdown_connections 2024-03-06T07:22:25.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.307+0000 7fb72b4e1640 1 -- 172.21.15.17:0/1988626480 wait complete. 2024-03-06T07:22:25.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:22:25.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.431+0000 7f7ee5e9a640 1 Processor -- start 2024-03-06T07:22:25.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.431+0000 7f7ee5e9a640 1 -- start start 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ee5e9a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7ee0155b10 0x7f7ee0160a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ee5e9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee0155f80 con 0x7f7ee00b01e0 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ee5e9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00c7e00 con 0x7f7ee01565b0 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ee5e9a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee00c7840 con 0x7f7ee0155b10 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7edf7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ee00b01e0 0x7f7ee00c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48426/0 (socket says 172.21.15.17:48426) 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7edf7fe640 1 -- 172.21.15.17:0/883351472 learned_addr learned my addr 172.21.15.17:0/883351472 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7edffff640 1 --2- 172.21.15.17:0/883351472 >> v2:172.21.15.163:3300/0 conn(0x7f7ee0155b10 0x7f7ee0160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009839444 0 0) 0x7f7ee0155f80 con 0x7f7ee00b01e0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec8003520 con 0x7f7ee00b01e0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4265560522 0 0) 0x7f7ec8003520 con 0x7f7ee00b01e0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 >> v2:172.21.15.163:3300/0 conn(0x7f7ee0155b10 msgr2=0x7f7ee0160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 --2- 172.21.15.17:0/883351472 >> v2:172.21.15.163:3300/0 conn(0x7f7ee0155b10 0x7f7ee0160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 >> v1:172.21.15.163:6789/0 conn(0x7f7ee01565b0 legacy=0x7f7ee006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00c7b20 con 0x7f7ee00b01e0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7ed4001670 con 0x7f7ee00b01e0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 >> v1:172.21.15.17:6789/0 conn(0x7f7ee00b01e0 legacy=0x7f7ee00c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ec80039b0 0x7f7ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ec8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec80051c0 0x7f7ec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00c7840 con 0x7f7ec8004800 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec800c060 con 0x7f7ec80051c0 2024-03-06T07:22:25.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec800c260 con 0x7f7ec80039b0 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.432+0000 7f7edeffd640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ec8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edf7fe640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ec80039b0 0x7f7ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edffff640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec80051c0 0x7f7ec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 -- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ec80039b0 msgr2=0x7f7ec8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ec80039b0 0x7f7ec8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 -- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec80051c0 msgr2=0x7f7ec800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec80051c0 0x7f7ec800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 -- 172.21.15.17:0/883351472 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8003520 con 0x7f7ec8004800 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edffff640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ec80051c0 0x7f7ec800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edffff640 1 -- 172.21.15.17:0/883351472 reap_dead start 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.433+0000 7f7edeffd640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ec8004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0001720 tx=0x7f7ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921890e8cd437fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0001370 con 0x7f7ec8004800 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed00052d0 con 0x7f7ec8004800 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ede7fc640 1 -- 172.21.15.17:0/883351472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0003830 con 0x7f7ec8004800 2024-03-06T07:22:25.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 msgr2=0x7f7ec8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ec8004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0001720 tx=0x7f7ec8005830 comp rx=0 tx=0).stop 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/883351472 shutdown_connections 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/883351472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ec8004c80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/883351472 >> 172.21.15.17:0/883351472 conn(0x7f7ee00ac600 msgr2=0x7f7ee00aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/883351472 shutdown_connections 2024-03-06T07:22:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.434+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/883351472 wait complete. 2024-03-06T07:22:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.435+0000 7f7ee5e9a640 1 Processor -- start 2024-03-06T07:22:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.435+0000 7f7ee5e9a640 1 -- start start 2024-03-06T07:22:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee00c8850 0x7f7ee00c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee01ee710 0x7f7ee00c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00be250 con 0x7f7ec8004800 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee006bc50 con 0x7f7ee01ee710 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7ee5e9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00be3d0 con 0x7f7ee00c8850 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41852/0 (socket says 172.21.15.17:41852) 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 -- 172.21.15.17:0/3309215493 learned_addr learned my addr 172.21.15.17:0/3309215493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 -- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee00c8850 msgr2=0x7f7ee00c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee00c8850 0x7f7ee00c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 -- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee01ee710 msgr2=0x7f7ee00c0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee01ee710 0x7f7ee00c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.436+0000 7f7edf7fe640 1 -- 172.21.15.17:0/3309215493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee01eec50 con 0x7f7ec8004800 2024-03-06T07:22:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7edf7fe640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e800 tx=0x7f7ed400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400c7f0 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4018070 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee00bfb10 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400cbb0 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.437+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee00c00d0 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.438+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee004f330 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.438+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ed401d460 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.439+0000 7f7ec7fff640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 0x7f7ecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.439+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f7ed408a0b0 con 0x7f7ec8004800 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.439+0000 7f7edeffd640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 0x7f7ecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.440+0000 7f7edeffd640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 0x7f7ecc06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0033000 tx=0x7f7ed002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.443+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ed4058f50 con 0x7f7ec8004800 2024-03-06T07:22:25.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.634+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7f7ee006cef0 con 0x7f7ec8004800 2024-03-06T07:22:25.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.637+0000 7f7ec7fff640 1 -- 172.21.15.17:0/3309215493 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v19) v1 ==== 96+0+0 (secure 0 0 0) 0x7f7ed4058850 con 0x7f7ec8004800 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 msgr2=0x7f7ecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 0x7f7ecc06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0033000 tx=0x7f7ed002f000 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 msgr2=0x7f7ee01ee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e800 tx=0x7f7ed400ecc0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 shutdown_connections 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ecc06b4d0 0x7f7ecc06d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ee01ee710 0x7f7ee00c0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ee00c8850 0x7f7ee00c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 --2- 172.21.15.17:0/3309215493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ec8004800 0x7f7ee01ee1d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 >> 172.21.15.17:0/3309215493 conn(0x7f7ee00ac600 msgr2=0x7f7ee0154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 shutdown_connections 2024-03-06T07:22:25.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.639+0000 7f7ee5e9a640 1 -- 172.21.15.17:0/3309215493 wait complete. 2024-03-06T07:22:25.651 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:22:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.753+0000 7fa175735640 1 Processor -- start 2024-03-06T07:22:25.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.753+0000 7fa175735640 1 -- start start 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1700c4e60 0x7fa17006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa170072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa170072bc0 0x7fa170153030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17006ec50 con 0x7fa170072bc0 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa170156650 con 0x7fa1700c4e60 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa175735640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1701568f0 con 0x7fa1700c6080 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa16f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa170072bc0 0x7fa170153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa16f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa170072bc0 0x7fa170153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41860/0 (socket says 172.21.15.17:41860) 2024-03-06T07:22:25.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa16f7fe640 1 -- 172.21.15.17:0/1452688903 learned_addr learned my addr 172.21.15.17:0/1452688903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa16effd640 1 --2- 172.21.15.17:0/1452688903 >> v2:172.21.15.163:3300/0 conn(0x7fa1700c4e60 0x7fa17006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.754+0000 7fa16e7fc640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa170072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 -- 172.21.15.17:0/1452688903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 msgr2=0x7fa170072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa170072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 -- 172.21.15.17:0/1452688903 >> v2:172.21.15.163:3300/0 conn(0x7fa1700c4e60 msgr2=0x7fa17006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 --2- 172.21.15.17:0/1452688903 >> v2:172.21.15.163:3300/0 conn(0x7fa1700c4e60 0x7fa17006f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 -- 172.21.15.17:0/1452688903 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170156bd0 con 0x7fa170072bc0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16f7fe640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa170153030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1700722f0 tx=0x7fa16002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c2c91bd738829a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.755+0000 7fa16dffb640 1 -- 172.21.15.17:0/1452688903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16002ca70 con 0x7fa170072bc0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa16dffb640 1 -- 172.21.15.17:0/1452688903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16002cc10 con 0x7fa170072bc0 2024-03-06T07:22:25.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa16dffb640 1 -- 172.21.15.17:0/1452688903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600343f0 con 0x7fa170072bc0 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 -- 172.21.15.17:0/1452688903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 msgr2=0x7fa170153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa170153030 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1700722f0 tx=0x7fa16002bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 -- 172.21.15.17:0/1452688903 shutdown_connections 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa170153030 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 --2- 172.21.15.17:0/1452688903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa170072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 --2- 172.21.15.17:0/1452688903 >> v2:172.21.15.163:3300/0 conn(0x7fa1700c4e60 0x7fa17006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 -- 172.21.15.17:0/1452688903 >> 172.21.15.17:0/1452688903 conn(0x7fa1700c5560 msgr2=0x7fa1700c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 -- 172.21.15.17:0/1452688903 shutdown_connections 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.756+0000 7fa175735640 1 -- 172.21.15.17:0/1452688903 wait complete. 2024-03-06T07:22:25.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.757+0000 7fa175735640 1 Processor -- start 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.757+0000 7fa175735640 1 -- start start 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1700c4e60 0x7fa170155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa1701ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1700c87d0 con 0x7fa170072bc0 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170157190 con 0x7fa1700c4e60 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa175735640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa170153570 con 0x7fa1700c6080 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41874/0 (socket says 172.21.15.17:41874) 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 -- 172.21.15.17:0/2767689439 learned_addr learned my addr 172.21.15.17:0/2767689439 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16e7fc640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1700c4e60 0x7fa170155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16f7fe640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa1701ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 -- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 msgr2=0x7fa1701ebe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa1701ebe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 -- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1700c4e60 msgr2=0x7fa170155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1700c4e60 0x7fa170155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.758+0000 7fa16effd640 1 -- 172.21.15.17:0/2767689439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa170153d30 con 0x7fa170072bc0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa16effd640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa16800d830 tx=0x7fa16800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800b6c0 con 0x7fa170072bc0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa168019070 con 0x7fa170072bc0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1701dc890 con 0x7fa170072bc0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800ba80 con 0x7fa170072bc0 2024-03-06T07:22:25.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.759+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1701dce50 con 0x7fa170072bc0 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.760+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1700c85f0 con 0x7fa170072bc0 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.760+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa16801e460 con 0x7fa170072bc0 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.761+0000 7fa1677fe640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.761+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fa16808a670 con 0x7fa170072bc0 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.761+0000 7fa16e7fc640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:25.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.761+0000 7fa16e7fc640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 0x7fa15006d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1700a9fa0 tx=0x7fa158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:25.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.765+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1700c85f0 con 0x7fa170072bc0 2024-03-06T07:22:25.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.947+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7fa1700c8330 con 0x7fa170072bc0 2024-03-06T07:22:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.950+0000 7fa1677fe640 1 -- 172.21.15.17:0/2767689439 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v20) v1 ==== 96+0+0 (secure 0 0 0) 0x7fa168058f50 con 0x7fa170072bc0 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 msgr2=0x7fa15006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 0x7fa15006d9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1700a9fa0 tx=0x7fa158008040 comp rx=0 tx=0).stop 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 msgr2=0x7fa1701df860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa16800d830 tx=0x7fa16800dcf0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 shutdown_connections 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa15006b500 0x7fa15006d9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1700c6080 0x7fa1701ebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1700c4e60 0x7fa170155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 --2- 172.21.15.17:0/2767689439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa170072bc0 0x7fa1701df860 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:25.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 >> 172.21.15.17:0/2767689439 conn(0x7fa1700c5560 msgr2=0x7fa170152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:25.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 shutdown_connections 2024-03-06T07:22:25.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:25.952+0000 7fa175735640 1 -- 172.21.15.17:0/2767689439 wait complete. 2024-03-06T07:22:25.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:22:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.066+0000 7ff2246df640 1 Processor -- start 2024-03-06T07:22:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.066+0000 7ff2246df640 1 -- start start 2024-03-06T07:22:26.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff2246df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff21c06c900 0x7ff21c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff2246df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff21c0a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff2246df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21c0c3ef0 con 0x7ff21c0c5990 2024-03-06T07:22:26.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff2246df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c1556d0 con 0x7ff21c06c900 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff2246df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1559b0 con 0x7ff21c0ab170 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff22247b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff21c0c5990 0x7ff21c06c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48434/0 (socket says 172.21.15.17:48434) 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.067+0000 7ff22247b640 1 -- 172.21.15.17:0/1355732195 learned_addr learned my addr 172.21.15.17:0/1355732195 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff21c0a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506295021 0 0) 0x7ff21c0c3ef0 con 0x7ff21c0c5990 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff208003520 con 0x7ff21c0c5990 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221c7a640 1 --2- 172.21.15.17:0/1355732195 >> v2:172.21.15.163:3300/0 conn(0x7ff21c06c900 0x7ff21c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3968164499 0 0) 0x7ff208003520 con 0x7ff21c0c5990 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 msgr2=0x7ff21c0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff21c0a9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 >> v2:172.21.15.163:3300/0 conn(0x7ff21c06c900 msgr2=0x7ff21c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 --2- 172.21.15.17:0/1355732195 >> v2:172.21.15.163:3300/0 conn(0x7ff21c06c900 0x7ff21c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c155c90 con 0x7ff21c0c5990 2024-03-06T07:22:26.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff218001670 con 0x7ff21c0c5990 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 >> v1:172.21.15.17:6789/0 conn(0x7ff21c0c5990 legacy=0x7ff21c06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2080039b0 0x7ff208003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff208004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff208004d40 0x7ff2080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1556d0 con 0x7ff208004d40 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1559b0 con 0x7ff2080039b0 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20800b9c0 con 0x7ff21c0ab170 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff208004d40 0x7ff2080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff22247b640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2080039b0 0x7ff208003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221c7a640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff208004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 -- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 msgr2=0x7ff208004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff208004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 -- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2080039b0 msgr2=0x7ff208003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2080039b0 0x7ff208003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff222c7c640 1 -- 172.21.15.17:0/1355732195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208003520 con 0x7ff208004d40 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff22247b640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2080039b0 0x7ff208003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.068+0000 7ff221c7a640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff21c0ab170 0x7ff208004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.069+0000 7ff222c7c640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff208004d40 0x7ff2080051c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff214009fc0 tx=0x7ff208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a8e1eb3c6f309 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.069+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214039070 con 0x7ff208004d40 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.069+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21402cc20 con 0x7ff208004d40 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.069+0000 7ff221479640 1 -- 172.21.15.17:0/1355732195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214009c10 con 0x7ff208004d40 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 -- 172.21.15.17:0/1355732195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff208004d40 msgr2=0x7ff2080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 --2- 172.21.15.17:0/1355732195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff208004d40 0x7ff2080051c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff214009fc0 tx=0x7ff208005830 comp rx=0 tx=0).stop 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff222c7c640 1 -- 172.21.15.17:0/1355732195 reap_dead start 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 -- 172.21.15.17:0/1355732195 shutdown_connections 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 -- 172.21.15.17:0/1355732195 >> 172.21.15.17:0/1355732195 conn(0x7ff21c0c6710 msgr2=0x7ff21c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 -- 172.21.15.17:0/1355732195 shutdown_connections 2024-03-06T07:22:26.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.070+0000 7ff2246df640 1 -- 172.21.15.17:0/1355732195 wait complete. 2024-03-06T07:22:26.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.071+0000 7ff2246df640 1 Processor -- start 2024-03-06T07:22:26.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.071+0000 7ff2246df640 1 -- start start 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 0x7ff21c14e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 0x7ff21c14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c156610 con 0x7ff21c0a7720 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c156490 con 0x7ff21c14cbc0 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0c3af0 con 0x7ff208004d40 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41878/0 (socket says 172.21.15.17:41878) 2024-03-06T07:22:26.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 -- 172.21.15.17:0/3750233705 learned_addr learned my addr 172.21.15.17:0/3750233705 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff221c7a640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 0x7ff21c14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 -- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 msgr2=0x7ff21c14e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff22247b640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 0x7ff21c14e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 0x7ff21c14e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 -- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 msgr2=0x7ff21c14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 0x7ff21c14ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 -- 172.21.15.17:0/3750233705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0c50b0 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff22247b640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 0x7ff21c14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff221c7a640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 0x7ff21c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff222c7c640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e230 tx=0x7ff214009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214042020 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21403a040 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214039070 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c1e72f0 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.072+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c154c00 con 0x7ff21c0a7720 2024-03-06T07:22:26.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.073+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c04f330 con 0x7ff21c0a7720 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.074+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff214005d10 con 0x7ff21c0a7720 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.074+0000 7ff212ffd640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 0x7ff20006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.074+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7ff2140ac4b0 con 0x7ff21c0a7720 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.074+0000 7ff22247b640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 0x7ff20006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.075+0000 7ff22247b640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 0x7ff20006d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff218005a70 tx=0x7ff2180014e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.078+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff21407b3e0 con 0x7ff21c0a7720 2024-03-06T07:22:26.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.261+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff21c0c28c0 con 0x7ff21c0a7720 2024-03-06T07:22:26.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.264+0000 7ff212ffd640 1 -- 172.21.15.17:0/3750233705 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff2140aae20 con 0x7ff21c0a7720 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 msgr2=0x7ff20006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 0x7ff20006d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff218005a70 tx=0x7ff2180014e0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 msgr2=0x7ff21c0a7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff21400e230 tx=0x7ff214009750 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 shutdown_connections 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff20006b4d0 0x7ff20006d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff21c0a7720 0x7ff21c0a7ba0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff21c14cbc0 0x7ff21c14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 --2- 172.21.15.17:0/3750233705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff208004d40 0x7ff21c14e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 >> 172.21.15.17:0/3750233705 conn(0x7ff21c0c6710 msgr2=0x7ff21c149610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 shutdown_connections 2024-03-06T07:22:26.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.266+0000 7ff2246df640 1 -- 172.21.15.17:0/3750233705 wait complete. 2024-03-06T07:22:26.278 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:22:26.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 Processor -- start 2024-03-06T07:22:26.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 -- start start 2024-03-06T07:22:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152730 0x7efc840c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc840c6710 0x7efc841554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc840c3750 0x7efc84155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84152b10 con 0x7efc840c3750 2024-03-06T07:22:26.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84158ad0 con 0x7efc840c6710 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.381+0000 7efc8bbbf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84158db0 con 0x7efc84152730 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc840c3750 0x7efc84155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc840c3750 0x7efc84155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41880/0 (socket says 172.21.15.17:41880) 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 -- 172.21.15.17:0/153991561 learned_addr learned my addr 172.21.15.17:0/153991561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 -- 172.21.15.17:0/153991561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152730 msgr2=0x7efc840c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 --2- 172.21.15.17:0/153991561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152730 0x7efc840c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 -- 172.21.15.17:0/153991561 >> v2:172.21.15.163:3300/0 conn(0x7efc840c6710 msgr2=0x7efc841554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 --2- 172.21.15.17:0/153991561 >> v2:172.21.15.163:3300/0 conn(0x7efc840c6710 0x7efc841554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.382+0000 7efc8a15c640 1 -- 172.21.15.17:0/153991561 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84159090 con 0x7efc840c3750 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8a15c640 1 --2- 172.21.15.17:0/153991561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c3750 0x7efc84155a00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efc84072220 tx=0x7efc7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235f499bfc36c2ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc88959640 1 -- 172.21.15.17:0/153991561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c02c9e0 con 0x7efc840c3750 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc88959640 1 -- 172.21.15.17:0/153991561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c02cb80 con 0x7efc840c3750 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc88959640 1 -- 172.21.15.17:0/153991561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0343f0 con 0x7efc840c3750 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 -- 172.21.15.17:0/153991561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c3750 msgr2=0x7efc84155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/153991561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c3750 0x7efc84155a00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efc84072220 tx=0x7efc7c02be40 comp rx=0 tx=0).stop 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 -- 172.21.15.17:0/153991561 shutdown_connections 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/153991561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c3750 0x7efc84155a00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/153991561 >> v2:172.21.15.163:3300/0 conn(0x7efc840c6710 0x7efc841554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/153991561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152730 0x7efc840c3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 -- 172.21.15.17:0/153991561 >> 172.21.15.17:0/153991561 conn(0x7efc840aa5c0 msgr2=0x7efc840aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 -- 172.21.15.17:0/153991561 shutdown_connections 2024-03-06T07:22:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.383+0000 7efc8bbbf640 1 -- 172.21.15.17:0/153991561 wait complete. 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 Processor -- start 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 -- start start 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c3750 0x7efc8414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c6710 0x7efc8414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84158870 con 0x7efc84152730 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841586f0 con 0x7efc840c3750 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8bbbf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc84155f40 con 0x7efc840c6710 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8a15c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8a15c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41896/0 (socket says 172.21.15.17:41896) 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8a15c640 1 -- 172.21.15.17:0/2811304763 learned_addr learned my addr 172.21.15.17:0/2811304763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:26.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8915a640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c6710 0x7efc8414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.385+0000 7efc8995b640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c3750 0x7efc8414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 -- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c6710 msgr2=0x7efc8414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c6710 0x7efc8414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 -- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c3750 msgr2=0x7efc8414c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c3750 0x7efc8414c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 -- 172.21.15.17:0/2811304763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8414b720 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8a15c640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009f40 tx=0x7efc7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c043020 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c03e450 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8414bb80 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c009510 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.386+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841ef0c0 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.387+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc841ef2c0 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.387+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc7c046020 con 0x7efc84152730 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.388+0000 7efc7a7fc640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 0x7efc6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.388+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7efc7c0ac6b0 con 0x7efc84152730 2024-03-06T07:22:26.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.388+0000 7efc8995b640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 0x7efc6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.389+0000 7efc8995b640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 0x7efc6806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc841519a0 tx=0x7efc80009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.392+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc7c04d070 con 0x7efc84152730 2024-03-06T07:22:26.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.571+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7efc8414e6d0 con 0x7efc84152730 2024-03-06T07:22:26.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.574+0000 7efc7a7fc640 1 -- 172.21.15.17:0/2811304763 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v22) v1 ==== 96+0+0 (secure 0 0 0) 0x7efc7c07b540 con 0x7efc84152730 2024-03-06T07:22:26.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 msgr2=0x7efc6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 0x7efc6806d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efc841519a0 tx=0x7efc80009060 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 msgr2=0x7efc840a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009f40 tx=0x7efc7c004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 shutdown_connections 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b500 0x7efc6806d9c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84152730 0x7efc840a9e30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c6710 0x7efc8414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 --2- 172.21.15.17:0/2811304763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c3750 0x7efc8414c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 >> 172.21.15.17:0/2811304763 conn(0x7efc840aa5c0 msgr2=0x7efc8414f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 shutdown_connections 2024-03-06T07:22:26.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.576+0000 7efc8bbbf640 1 -- 172.21.15.17:0/2811304763 wait complete. 2024-03-06T07:22:26.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort' 2024-03-06T07:22:26.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.689+0000 7fc1a382a640 1 Processor -- start 2024-03-06T07:22:26.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.689+0000 7fc1a382a640 1 -- start start 2024-03-06T07:22:26.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a382a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a382a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 0x7fc19c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a382a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0c73d0 con 0x7fc19c0c88c0 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a382a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c0c8dd0 con 0x7fc19c158b60 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a382a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0be2e0 con 0x7fc19c14f360 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a15c6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a1dc7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 0x7fc19c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a0dc5640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc19c0c88c0 0x7fc19c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40666/0 (socket says 172.21.15.17:40666) 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a15c6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:50592/0 (socket says 172.21.15.17:50592) 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a0dc5640 1 -- 172.21.15.17:0/112048228 learned_addr learned my addr 172.21.15.17:0/112048228 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc19bfff640 1 -- 172.21.15.17:0/112048228 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197177548 0 0) 0x7fc19c0c73d0 con 0x7fc19c0c88c0 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc19bfff640 1 -- 172.21.15.17:0/112048228 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc184003520 con 0x7fc19c0c88c0 2024-03-06T07:22:26.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.690+0000 7fc1a15c6640 1 -- 172.21.15.17:0/112048228 >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 msgr2=0x7fc19c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:26.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.691+0000 7fc1a15c6640 1 --2- 172.21.15.17:0/112048228 >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 0x7fc19c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.691+0000 7fc1a15c6640 1 -- 172.21.15.17:0/112048228 >> v1:172.21.15.163:6789/0 conn(0x7fc19c0c88c0 legacy=0x7fc19c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.691+0000 7fc1a15c6640 1 -- 172.21.15.17:0/112048228 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0c7670 con 0x7fc19c14f360 2024-03-06T07:22:26.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.691+0000 7fc1a1dc7640 1 --2- 172.21.15.17:0/112048228 >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 0x7fc19c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:26.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.691+0000 7fc1a15c6640 1 --2- 172.21.15.17:0/112048228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097a0 tx=0x7fc19402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88327724b190754c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc19bfff640 1 -- 172.21.15.17:0/112048228 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940315a0 con 0x7fc19c14f360 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc19bfff640 1 -- 172.21.15.17:0/112048228 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194037070 con 0x7fc19c14f360 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc19bfff640 1 -- 172.21.15.17:0/112048228 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194031940 con 0x7fc19c14f360 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 -- 172.21.15.17:0/112048228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 msgr2=0x7fc19c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 --2- 172.21.15.17:0/112048228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097a0 tx=0x7fc19402b840 comp rx=0 tx=0).stop 2024-03-06T07:22:26.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 -- 172.21.15.17:0/112048228 shutdown_connections 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 --2- 172.21.15.17:0/112048228 >> v2:172.21.15.163:3300/0 conn(0x7fc19c158b60 0x7fc19c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 --2- 172.21.15.17:0/112048228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c14f360 0x7fc19c155c10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 -- 172.21.15.17:0/112048228 >> 172.21.15.17:0/112048228 conn(0x7fc19c0ac570 msgr2=0x7fc19c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 -- 172.21.15.17:0/112048228 shutdown_connections 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.692+0000 7fc1a382a640 1 -- 172.21.15.17:0/112048228 wait complete. 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.693+0000 7fc1a382a640 1 Processor -- start 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.693+0000 7fc1a382a640 1 -- start start 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c0c88c0 0x7fc19c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc19c14f360 0x7fc19c0bff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0c5a50 con 0x7fc19c158b60 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0c7c30 con 0x7fc19c14f360 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a382a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0c5bd0 con 0x7fc19c0c88c0 2024-03-06T07:22:26.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41898/0 (socket says 172.21.15.17:41898) 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 -- 172.21.15.17:0/4097639115 learned_addr learned my addr 172.21.15.17:0/4097639115 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a0dc5640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc19c14f360 0x7fc19c0bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a15c6640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c0c88c0 0x7fc19c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 -- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c0c88c0 msgr2=0x7fc19c154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c0c88c0 0x7fc19c154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 -- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc19c14f360 msgr2=0x7fc19c0bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc19c14f360 0x7fc19c0bff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.694+0000 7fc1a1dc7640 1 -- 172.21.15.17:0/4097639115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c157780 con 0x7fc19c158b60 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc1a1dc7640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00efc0 tx=0x7fc18c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c018030 con 0x7fc19c158b60 2024-03-06T07:22:26.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c00cde0 con 0x7fc19c158b60 2024-03-06T07:22:26.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c0bf150 con 0x7fc19c158b60 2024-03-06T07:22:26.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c003bb0 con 0x7fc19c158b60 2024-03-06T07:22:26.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.695+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0bf710 con 0x7fc19c158b60 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.696+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c04f330 con 0x7fc19c158b60 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.696+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc18c01b070 con 0x7fc19c158b60 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.697+0000 7fc199ffb640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 0x7fc18806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.697+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fc18c08ab70 con 0x7fc19c158b60 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.697+0000 7fc1a15c6640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 0x7fc18806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:26.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.697+0000 7fc1a15c6640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 0x7fc18806d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc194036000 tx=0x7fc194008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:26.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.701+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc18c010040 con 0x7fc19c158b60 2024-03-06T07:22:26.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:26.957+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]} v 0) v1 -- 0x7fc19c06cda0 con 0x7fc19c158b60 2024-03-06T07:22:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.307+0000 7fc199ffb640 1 -- 172.21.15.17:0/4097639115 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort"]}]=0 v151) v1 ==== 150+0+0 (secure 0 0 0) 0x7fc18c012340 con 0x7fc19c158b60 2024-03-06T07:22:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.310+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 msgr2=0x7fc18806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.310+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 0x7fc18806d9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc194036000 tx=0x7fc194008030 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 msgr2=0x7fc19c0c0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00efc0 tx=0x7fc18c009f90 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 shutdown_connections 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18806b500 0x7fc18806d9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc19c158b60 0x7fc19c0c0450 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc19c14f360 0x7fc19c0bff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 --2- 172.21.15.17:0/4097639115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc19c0c88c0 0x7fc19c154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 >> 172.21.15.17:0/4097639115 conn(0x7fc19c0ac570 msgr2=0x7fc19c06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 shutdown_connections 2024-03-06T07:22:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.311+0000 7fc1a382a640 1 -- 172.21.15.17:0/4097639115 wait complete. 2024-03-06T07:22:27.324 INFO:tasks.cephfs_test_runner:That a standby with mds_join_fs set to another fs is still used if necessary. ... ok 2024-03-06T07:22:27.325 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_last_resort_refused (tasks.cephfs.test_failover.TestClusterAffinity) That a standby with mds_join_fs set to another fs is not used if refuse_standby_for_another_fs is set. 2024-03-06T07:22:27.325 INFO:tasks.cephfs_test_runner:test_join_fs_last_resort_refused (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:22:27.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:27.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.427+0000 7f964ab5e640 1 Processor -- start 2024-03-06T07:22:27.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.427+0000 7f964ab5e640 1 -- start start 2024-03-06T07:22:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f964ab5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f964414f360 0x7f9644155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f964ab5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96440c88c0 0x7f96440c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f964ab5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440c73d0 con 0x7f96440c5880 2024-03-06T07:22:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f964ab5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96440c8dd0 con 0x7f964414f360 2024-03-06T07:22:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f964ab5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96440be2e0 con 0x7f96440c88c0 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f964414f360 0x7f9644155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f964414f360 0x7f9644155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41906/0 (socket says 172.21.15.17:41906) 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 -- 172.21.15.17:0/3677626333 learned_addr learned my addr 172.21.15.17:0/3677626333 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f9643fff640 1 --2- 172.21.15.17:0/3677626333 >> v2:172.21.15.163:3300/0 conn(0x7f96440c88c0 0x7f96440c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 -- 172.21.15.17:0/3677626333 >> v2:172.21.15.163:3300/0 conn(0x7f96440c88c0 msgr2=0x7f96440c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 --2- 172.21.15.17:0/3677626333 >> v2:172.21.15.163:3300/0 conn(0x7f96440c88c0 0x7f96440c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 -- 172.21.15.17:0/3677626333 >> v1:172.21.15.163:6789/0 conn(0x7f96440c5880 legacy=0x7f9644158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.428+0000 7f96488fa640 1 -- 172.21.15.17:0/3677626333 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440c7670 con 0x7f964414f360 2024-03-06T07:22:27.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.429+0000 7f96488fa640 1 --2- 172.21.15.17:0/3677626333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964414f360 0x7f9644155c10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f96440c84c0 tx=0x7f963402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c55b2d88b4be164 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.429+0000 7f96437fe640 1 -- 172.21.15.17:0/3677626333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963402e070 con 0x7f964414f360 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.429+0000 7f96437fe640 1 -- 172.21.15.17:0/3677626333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634037070 con 0x7f964414f360 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.429+0000 7f96437fe640 1 -- 172.21.15.17:0/3677626333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634031a00 con 0x7f964414f360 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 -- 172.21.15.17:0/3677626333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964414f360 msgr2=0x7f9644155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3677626333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964414f360 0x7f9644155c10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f96440c84c0 tx=0x7f963402b880 comp rx=0 tx=0).stop 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 -- 172.21.15.17:0/3677626333 shutdown_connections 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3677626333 >> v2:172.21.15.163:3300/0 conn(0x7f96440c88c0 0x7f96440c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3677626333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f964414f360 0x7f9644155c10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 -- 172.21.15.17:0/3677626333 >> 172.21.15.17:0/3677626333 conn(0x7f96440ac570 msgr2=0x7f96440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 -- 172.21.15.17:0/3677626333 shutdown_connections 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.430+0000 7f964ab5e640 1 -- 172.21.15.17:0/3677626333 wait complete. 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.431+0000 7f964ab5e640 1 Processor -- start 2024-03-06T07:22:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.431+0000 7f964ab5e640 1 -- start start 2024-03-06T07:22:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5880 0x7f96440bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964414f360 0x7f96441eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9644159270 con 0x7f96440c88c0 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440c7c30 con 0x7f96440c5880 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f964ab5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96441593f0 con 0x7f964414f360 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41908/0 (socket says 172.21.15.17:41908) 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 -- 172.21.15.17:0/3609767645 learned_addr learned my addr 172.21.15.17:0/3609767645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f96490fb640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964414f360 0x7f96441eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f96488fa640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5880 0x7f96440bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 -- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964414f360 msgr2=0x7f96441eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964414f360 0x7f96441eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 -- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5880 msgr2=0x7f96440bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5880 0x7f96440bf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.432+0000 7f9643fff640 1 -- 172.21.15.17:0/3609767645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440c06e0 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f9643fff640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f963800d800 tx=0x7f963800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800b6c0 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9638019070 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96440c09c0 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963800ba80 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.433+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440bf0a0 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.434+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9644152fc0 con 0x7f96440c88c0 2024-03-06T07:22:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.434+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f963800bc20 con 0x7f96440c88c0 2024-03-06T07:22:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.434+0000 7f96417fa640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 0x7f963006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.435+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f963808a5b0 con 0x7f96440c88c0 2024-03-06T07:22:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.435+0000 7f96488fa640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 0x7f963006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.435+0000 7f96488fa640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 0x7f963006d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9634036000 tx=0x7f9634008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.439+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9644152fc0 con 0x7f96440c88c0 2024-03-06T07:22:27.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.620+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f964404f330 con 0x7f96440c88c0 2024-03-06T07:22:27.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.621+0000 7f96417fa640 1 -- 172.21.15.17:0/3609767645 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+13816 (secure 0 0 0) 0x7f96380594e0 con 0x7f96440c88c0 2024-03-06T07:22:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:27.622 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:25.302033+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.17:6833/2394960697":"2024-03-07T07:22:16.188196+0000","172.21.15.163:6836/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.163:6834/673661445":"2024-03-07T07:22:07.986464+0000","172.21.15.17:6834/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6835/1002585815":"2024-03-07T07:22:04.971202+0000","172.21.15.163:6836/3777861734":"2024-03-07T07:22:22.290033+0000","172.21.15.163:6834/3777861734":"2024-03-07T07:22:22.290033+0000","172.21.15.17:6832/3526701035":"2024-03-07T07:22:16.817958+0000","172.21.15.17:6833/1002585815":"2024-03-07T07:22:04.971202+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:27.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 msgr2=0x7f963006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 0x7f963006d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9634036000 tx=0x7f9634008030 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 msgr2=0x7f96441eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f963800d800 tx=0x7f963800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 shutdown_connections 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f963006b4d0 0x7f963006d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f964414f360 0x7f96441eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96440c88c0 0x7f96441eab60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 --2- 172.21.15.17:0/3609767645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5880 0x7f96440bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 >> 172.21.15.17:0/3609767645 conn(0x7f96440ac570 msgr2=0x7f96440ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 shutdown_connections 2024-03-06T07:22:27.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.623+0000 7f964ab5e640 1 -- 172.21.15.17:0/3609767645 wait complete. 2024-03-06T07:22:27.635 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:27.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:27.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.738+0000 7fc932caf640 1 Processor -- start 2024-03-06T07:22:27.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.738+0000 7fc932caf640 1 -- start start 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc932caf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c1528d0 0x7fc92c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc932caf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc92c0c67f0 0x7fc92c155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc932caf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c152cb0 con 0x7fc92c072a10 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc932caf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0c38b0 con 0x7fc92c0c67f0 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc932caf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1594f0 con 0x7fc92c1528d0 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc92c0c67f0 0x7fc92c155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc92c0c67f0 0x7fc92c155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41922/0 (socket says 172.21.15.17:41922) 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 -- 172.21.15.17:0/3102202461 learned_addr learned my addr 172.21.15.17:0/3102202461 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc930a4b640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c1528d0 0x7fc92c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 -- 172.21.15.17:0/3102202461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c1528d0 msgr2=0x7fc92c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c1528d0 0x7fc92c0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 -- 172.21.15.17:0/3102202461 >> v1:172.21.15.163:6789/0 conn(0x7fc92c072a10 legacy=0x7fc92c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.739+0000 7fc93124c640 1 -- 172.21.15.17:0/3102202461 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0a7f70 con 0x7fc92c0c67f0 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.740+0000 7fc93124c640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c0c67f0 0x7fc92c155930 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc924009e50 tx=0x7fc92402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e982c5bb6ef1b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.740+0000 7fc92b7fe640 1 -- 172.21.15.17:0/3102202461 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92402cb00 con 0x7fc92c0c67f0 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.740+0000 7fc92b7fe640 1 -- 172.21.15.17:0/3102202461 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92402cca0 con 0x7fc92c0c67f0 2024-03-06T07:22:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.740+0000 7fc92b7fe640 1 -- 172.21.15.17:0/3102202461 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924034610 con 0x7fc92c0c67f0 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 -- 172.21.15.17:0/3102202461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c0c67f0 msgr2=0x7fc92c155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c0c67f0 0x7fc92c155930 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc924009e50 tx=0x7fc92402bf60 comp rx=0 tx=0).stop 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 -- 172.21.15.17:0/3102202461 shutdown_connections 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c0c67f0 0x7fc92c155930 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 --2- 172.21.15.17:0/3102202461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c1528d0 0x7fc92c0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 -- 172.21.15.17:0/3102202461 >> 172.21.15.17:0/3102202461 conn(0x7fc92c0aa6a0 msgr2=0x7fc92c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 -- 172.21.15.17:0/3102202461 shutdown_connections 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.741+0000 7fc932caf640 1 -- 172.21.15.17:0/3102202461 wait complete. 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.742+0000 7fc932caf640 1 Processor -- start 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.742+0000 7fc932caf640 1 -- start start 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 0x7fc92c149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c0c67f0 0x7fc92c06f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1591a0 con 0x7fc92c1528d0 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0a8250 con 0x7fc92c072a10 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc932caf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0c4ef0 con 0x7fc92c0c67f0 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41924/0 (socket says 172.21.15.17:41924) 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 -- 172.21.15.17:0/4275352476 learned_addr learned my addr 172.21.15.17:0/4275352476 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc92bfff640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c0c67f0 0x7fc92c06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc930a4b640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 0x7fc92c149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 -- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c0c67f0 msgr2=0x7fc92c06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c0c67f0 0x7fc92c06f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 -- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 msgr2=0x7fc92c149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 0x7fc92c149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 -- 172.21.15.17:0/4275352476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c070df0 con 0x7fc92c1528d0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc930a4b640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 0x7fc92c149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.743+0000 7fc93124c640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc924036040 tx=0x7fc924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.744+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924043020 con 0x7fc92c1528d0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.744+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92402ce50 con 0x7fc92c1528d0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.744+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924009510 con 0x7fc92c1528d0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.744+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c1eaae0 con 0x7fc92c1528d0 2024-03-06T07:22:27.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.744+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c1eada0 con 0x7fc92c1528d0 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.745+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c06ca10 con 0x7fc92c1528d0 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.745+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc924046020 con 0x7fc92c1528d0 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.745+0000 7fc9297fa640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 0x7fc91806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.746+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fc9240ac770 con 0x7fc92c1528d0 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.746+0000 7fc930a4b640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 0x7fc91806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:27.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.746+0000 7fc930a4b640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 0x7fc91806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc92c1425a0 tx=0x7fc91c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:27.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.750+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc92c06ca10 con 0x7fc92c1528d0 2024-03-06T07:22:27.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.933+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc92c04f330 con 0x7fc92c1528d0 2024-03-06T07:22:27.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.933+0000 7fc9297fa640 1 -- 172.21.15.17:0/4275352476 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc92404d030 con 0x7fc92c1528d0 2024-03-06T07:22:27.935 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:22:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.935+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 msgr2=0x7fc91806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.935+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 0x7fc91806d990 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc92c1425a0 tx=0x7fc91c008040 comp rx=0 tx=0).stop 2024-03-06T07:22:27.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 msgr2=0x7fc92c06fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc924036040 tx=0x7fc924004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 shutdown_connections 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91806b4d0 0x7fc91806d990 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92c1528d0 0x7fc92c06fed0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc92c0c67f0 0x7fc92c06f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 --2- 172.21.15.17:0/4275352476 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92c072a10 0x7fc92c149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 >> 172.21.15.17:0/4275352476 conn(0x7fc92c0aa6a0 msgr2=0x7fc92c0c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 shutdown_connections 2024-03-06T07:22:27.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:27.936+0000 7fc932caf640 1 -- 172.21.15.17:0/4275352476 wait complete. 2024-03-06T07:22:27.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused' 2024-03-06T07:22:28.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.048+0000 7f9e2a7c9640 1 Processor -- start 2024-03-06T07:22:28.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.048+0000 7f9e2a7c9640 1 -- start start 2024-03-06T07:22:28.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e2a7c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e240c8a00 0x7f9e24158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e2a7c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 0x7f9e240c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e2a7c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e240c7760 con 0x7f9e24152920 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e2a7c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e241587b0 con 0x7f9e240c8a00 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e2a7c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240c7a40 con 0x7f9e24159460 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e237fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e240c8a00 0x7f9e24158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e23fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9e24152920 0x7f9e24155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48446/0 (socket says 172.21.15.17:48446) 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e23fff640 1 -- 172.21.15.17:0/1620955944 learned_addr learned my addr 172.21.15.17:0/1620955944 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e28d66640 1 --2- 172.21.15.17:0/1620955944 >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 0x7f9e240c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e237fe640 1 -- 172.21.15.17:0/1620955944 >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 msgr2=0x7f9e240c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e237fe640 1 --2- 172.21.15.17:0/1620955944 >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 0x7f9e240c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e237fe640 1 -- 172.21.15.17:0/1620955944 >> v1:172.21.15.17:6789/0 conn(0x7f9e24152920 legacy=0x7f9e24155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e22ffd640 1 -- 172.21.15.17:0/1620955944 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343225143 0 0) 0x7f9e240c7760 con 0x7f9e24152920 2024-03-06T07:22:28.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e237fe640 1 -- 172.21.15.17:0/1620955944 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240c7d20 con 0x7f9e240c8a00 2024-03-06T07:22:28.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.049+0000 7f9e28d66640 1 --2- 172.21.15.17:0/1620955944 >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 0x7f9e240c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:28.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.050+0000 7f9e237fe640 1 --2- 172.21.15.17:0/1620955944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e240c8a00 0x7f9e24158f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cba60 tx=0x7f9e0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eada0b6176ddf1b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:28.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.050+0000 7f9e22ffd640 1 -- 172.21.15.17:0/1620955944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0315a0 con 0x7f9e240c8a00 2024-03-06T07:22:28.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.050+0000 7f9e22ffd640 1 -- 172.21.15.17:0/1620955944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c037070 con 0x7f9e240c8a00 2024-03-06T07:22:28.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.050+0000 7f9e22ffd640 1 -- 172.21.15.17:0/1620955944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c031960 con 0x7f9e240c8a00 2024-03-06T07:22:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/1620955944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e240c8a00 msgr2=0x7f9e24158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/1620955944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e240c8a00 0x7f9e24158f20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9e240cba60 tx=0x7f9e0c02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:28.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/1620955944 shutdown_connections 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/1620955944 >> v2:172.21.15.163:3300/0 conn(0x7f9e24159460 0x7f9e240c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/1620955944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e240c8a00 0x7f9e24158f20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/1620955944 >> 172.21.15.17:0/1620955944 conn(0x7f9e240ac560 msgr2=0x7f9e240ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/1620955944 shutdown_connections 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.051+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/1620955944 wait complete. 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.052+0000 7f9e2a7c9640 1 Processor -- start 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.052+0000 7f9e2a7c9640 1 -- start start 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e240c8a00 0x7f9e240c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e240aaef0 0x7f9e24147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240c7390 con 0x7f9e24152920 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2406d030 con 0x7f9e240c8a00 2024-03-06T07:22:28.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240c7510 con 0x7f9e240aaef0 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41936/0 (socket says 172.21.15.17:41936) 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 -- 172.21.15.17:0/4286365346 learned_addr learned my addr 172.21.15.17:0/4286365346 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e23fff640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e240c8a00 0x7f9e240c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e28d66640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e240aaef0 0x7f9e24147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 -- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e240aaef0 msgr2=0x7f9e24147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e240aaef0 0x7f9e24147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 -- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e240c8a00 msgr2=0x7f9e240c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e240c8a00 0x7f9e240c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 -- 172.21.15.17:0/4286365346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240abf00 con 0x7f9e24152920 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e237fe640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009890 tx=0x7f9e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c032630 con 0x7f9e24152920 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0327d0 con 0x7f9e24152920 2024-03-06T07:22:28.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e24147ea0 con 0x7f9e24152920 2024-03-06T07:22:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c032ba0 con 0x7f9e24152920 2024-03-06T07:22:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.053+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24147650 con 0x7f9e24152920 2024-03-06T07:22:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.054+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e24147c20 con 0x7f9e24152920 2024-03-06T07:22:28.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.055+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e0c04d3f0 con 0x7f9e24152920 2024-03-06T07:22:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.055+0000 7f9e20ff9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 0x7f9e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.055+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f9e0c02f030 con 0x7f9e24152920 2024-03-06T07:22:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.056+0000 7f9e23fff640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 0x7f9e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:28.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.056+0000 7f9e23fff640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 0x7f9e0406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000f60 tx=0x7f9e14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:28.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.060+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e24147c20 con 0x7f9e24152920 2024-03-06T07:22:28.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:28.309+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]} v 0) v1 -- 0x7f9e240ac0d0 con 0x7f9e24152920 2024-03-06T07:22:29.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.313+0000 7f9e20ff9640 1 -- 172.21.15.17:0/4286365346 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]}]=0 v153) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9e0c040070 con 0x7f9e24152920 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 msgr2=0x7f9e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 0x7f9e0406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9e14000f60 tx=0x7f9e14009040 comp rx=0 tx=0).stop 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 msgr2=0x7f9e24146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009890 tx=0x7f9e0c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 shutdown_connections 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e0406b4d0 0x7f9e0406d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e240aaef0 0x7f9e24147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e24152920 0x7f9e24146bd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 --2- 172.21.15.17:0/4286365346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e240c8a00 0x7f9e240c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 >> 172.21.15.17:0/4286365346 conn(0x7f9e240ac560 msgr2=0x7f9e240ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 shutdown_connections 2024-03-06T07:22:29.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.316+0000 7f9e2a7c9640 1 -- 172.21.15.17:0/4286365346 wait complete. 2024-03-06T07:22:29.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:22:29.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.439+0000 7f64dd9ac640 1 Processor -- start 2024-03-06T07:22:29.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 -- start start 2024-03-06T07:22:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64d81529d0 0x7f64d80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64d80c57e0 0x7f64d8155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d8152db0 con 0x7f64d8072cd0 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80c6dd0 con 0x7f64d80c57e0 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64dd9ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80c6940 con 0x7f64d81529d0 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64d67fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64d80c57e0 0x7f64d8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64d67fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64d80c57e0 0x7f64d8155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41946/0 (socket says 172.21.15.17:41946) 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.440+0000 7f64d67fc640 1 -- 172.21.15.17:0/1476903759 learned_addr learned my addr 172.21.15.17:0/1476903759 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d67fc640 1 -- 172.21.15.17:0/1476903759 >> v2:172.21.15.163:3300/0 conn(0x7f64d81529d0 msgr2=0x7f64d80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d67fc640 1 --2- 172.21.15.17:0/1476903759 >> v2:172.21.15.163:3300/0 conn(0x7f64d81529d0 0x7f64d80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d67fc640 1 -- 172.21.15.17:0/1476903759 >> v1:172.21.15.163:6789/0 conn(0x7f64d8072cd0 legacy=0x7f64d8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d67fc640 1 -- 172.21.15.17:0/1476903759 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d81590f0 con 0x7f64d80c57e0 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d67fc640 1 --2- 172.21.15.17:0/1476903759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d80c57e0 0x7f64d8155120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f64d80c98d0 tx=0x7f64c802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f8836185144aa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.441+0000 7f64d5ffb640 1 -- 172.21.15.17:0/1476903759 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8038070 con 0x7f64d80c57e0 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64d5ffb640 1 -- 172.21.15.17:0/1476903759 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c802f050 con 0x7f64d80c57e0 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64d5ffb640 1 -- 172.21.15.17:0/1476903759 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8031970 con 0x7f64d80c57e0 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 -- 172.21.15.17:0/1476903759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d80c57e0 msgr2=0x7f64d8155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/1476903759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d80c57e0 0x7f64d8155120 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f64d80c98d0 tx=0x7f64c802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 -- 172.21.15.17:0/1476903759 shutdown_connections 2024-03-06T07:22:29.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/1476903759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d80c57e0 0x7f64d8155120 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/1476903759 >> v2:172.21.15.163:3300/0 conn(0x7f64d81529d0 0x7f64d80c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 -- 172.21.15.17:0/1476903759 >> 172.21.15.17:0/1476903759 conn(0x7f64d80aa630 msgr2=0x7f64d80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 -- 172.21.15.17:0/1476903759 shutdown_connections 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.442+0000 7f64dd9ac640 1 -- 172.21.15.17:0/1476903759 wait complete. 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.443+0000 7f64dd9ac640 1 Processor -- start 2024-03-06T07:22:29.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.443+0000 7f64dd9ac640 1 -- start start 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 0x7f64d806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d80c57e0 0x7f64d8145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8072720 con 0x7f64d81529d0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8155d70 con 0x7f64d8072cd0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64dd9ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8159370 con 0x7f64d80c57e0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41950/0 (socket says 172.21.15.17:41950) 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 -- 172.21.15.17:0/197087889 learned_addr learned my addr 172.21.15.17:0/197087889 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d6ffd640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 0x7f64d806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d67fc640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d80c57e0 0x7f64d8145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 -- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d80c57e0 msgr2=0x7f64d8145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d80c57e0 0x7f64d8145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 -- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 msgr2=0x7f64d806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.444+0000 7f64d77fe640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 0x7f64d806cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64d77fe640 1 -- 172.21.15.17:0/197087889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d81547f0 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64d6ffd640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 0x7f64d806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64d77fe640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ba10 tx=0x7f64d000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0019070 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000cb20 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8154ad0 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d0014440 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.445+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d8155090 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.446+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d80aebf0 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.446+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f64d001c070 con 0x7f64d81529d0 2024-03-06T07:22:29.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.447+0000 7f64bffff640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 0x7f64c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.447+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f64d0089e40 con 0x7f64d81529d0 2024-03-06T07:22:29.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.447+0000 7f64d6ffd640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 0x7f64c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.447+0000 7f64d6ffd640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 0x7f64c406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00a870 tx=0x7f64cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.451+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64d80aebf0 con 0x7f64d81529d0 2024-03-06T07:22:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.631+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f64d80a8cc0 con 0x7f64d81529d0 2024-03-06T07:22:29.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.634+0000 7f64bffff640 1 -- 172.21.15.17:0/197087889 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f64d80a8cc0 con 0x7f64d81529d0 2024-03-06T07:22:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 msgr2=0x7f64c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 0x7f64c406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00a870 tx=0x7f64cc008040 comp rx=0 tx=0).stop 2024-03-06T07:22:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 msgr2=0x7f64d81462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ba10 tx=0x7f64d000bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 shutdown_connections 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f64c406b4d0 0x7f64c406d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64d81529d0 0x7f64d81462b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64d80c57e0 0x7f64d8145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 --2- 172.21.15.17:0/197087889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64d8072cd0 0x7f64d806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 >> 172.21.15.17:0/197087889 conn(0x7f64d80aa630 msgr2=0x7f64d80aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 shutdown_connections 2024-03-06T07:22:29.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.636+0000 7f64dd9ac640 1 -- 172.21.15.17:0/197087889 wait complete. 2024-03-06T07:22:29.649 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:22:29.649 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:29.697 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:22:29.739 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:22:29.739 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:22:29.740 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:29.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:22:29.754 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:22:29.754 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:22:29.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:22:29.798 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:22:29.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.898+0000 7fd967037640 1 Processor -- start 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.898+0000 7fd967037640 1 -- start start 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.898+0000 7fd967037640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd960155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.898+0000 7fd967037640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9600c8a00 0x7fd960158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.898+0000 7fd967037640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd960156180 con 0x7fd960159460 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd967037640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9601587b0 con 0x7fd9600c8a00 2024-03-06T07:22:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd967037640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c7760 con 0x7fd960152920 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd9655d4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd960159460 0x7fd9600c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48452/0 (socket says 172.21.15.17:48452) 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd9655d4640 1 -- 172.21.15.17:0/4104103993 learned_addr learned my addr 172.21.15.17:0/4104103993 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450467065 0 0) 0x7fd958001490 con 0x7fd960159460 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd964dd3640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd960155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95ffff640 1 --2- 172.21.15.17:0/4104103993 >> v2:172.21.15.163:3300/0 conn(0x7fd9600c8a00 0x7fd960158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd950003520 con 0x7fd960159460 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000414957 0 0) 0x7fd950003520 con 0x7fd960159460 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 msgr2=0x7fd960155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd960155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 >> v2:172.21.15.163:3300/0 conn(0x7fd9600c8a00 msgr2=0x7fd960158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 --2- 172.21.15.17:0/4104103993 >> v2:172.21.15.163:3300/0 conn(0x7fd9600c8a00 0x7fd960158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600c7a40 con 0x7fd960159460 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd958001bc0 con 0x7fd960159460 2024-03-06T07:22:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 >> v1:172.21.15.17:6789/0 conn(0x7fd960159460 legacy=0x7fd9600c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd9500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd950003f30 0x7fd950004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd9500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601587b0 con 0x7fd950004d40 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c7760 con 0x7fd950003f30 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95000ba10 con 0x7fd960152920 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd9655d4640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd9500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.899+0000 7fd964dd3640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd9500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd95ffff640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd950003f30 0x7fd950004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 -- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 msgr2=0x7fd9500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd960152920 0x7fd9500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 -- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd950003f30 msgr2=0x7fd950004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd950003f30 0x7fd950004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 -- 172.21.15.17:0/4104103993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950003520 con 0x7fd950004d40 2024-03-06T07:22:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.900+0000 7fd9655d4640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd9500051c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd9600cba80 tx=0x7fd950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a0194d553b5b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800a6b0 con 0x7fd950004d40 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd958030070 con 0x7fd950004d40 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd95f7fe640 1 -- 172.21.15.17:0/4104103993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd958007510 con 0x7fd950004d40 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 -- 172.21.15.17:0/4104103993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 msgr2=0x7fd9500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 --2- 172.21.15.17:0/4104103993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd9500051c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd9600cba80 tx=0x7fd950005830 comp rx=0 tx=0).stop 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd9655d4640 1 -- 172.21.15.17:0/4104103993 reap_dead start 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 -- 172.21.15.17:0/4104103993 shutdown_connections 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 -- 172.21.15.17:0/4104103993 >> 172.21.15.17:0/4104103993 conn(0x7fd9600ac560 msgr2=0x7fd9600ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 -- 172.21.15.17:0/4104103993 shutdown_connections 2024-03-06T07:22:29.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.901+0000 7fd967037640 1 -- 172.21.15.17:0/4104103993 wait complete. 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.902+0000 7fd967037640 1 Processor -- start 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.902+0000 7fd967037640 1 -- start start 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 0x7fd9600c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 0x7fd960148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96006d030 con 0x7fd950004d40 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd960158a00 con 0x7fd960148b10 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd967037640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600c5a40 con 0x7fd9600c4210 2024-03-06T07:22:29.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41972/0 (socket says 172.21.15.17:41972) 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 -- 172.21.15.17:0/302204185 learned_addr learned my addr 172.21.15.17:0/302204185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd95ffff640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 0x7fd9600c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd9655d4640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 0x7fd960148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 -- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 msgr2=0x7fd9600c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 0x7fd9600c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 -- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 msgr2=0x7fd960148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 0x7fd960148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd964dd3640 1 -- 172.21.15.17:0/302204185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9601eb140 con 0x7fd950004d40 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd95ffff640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 0x7fd9600c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.903+0000 7fd9655d4640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 0x7fd960148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd964dd3640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd9540119d0 tx=0x7fd954011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400fb50 con 0x7fd950004d40 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95400fcf0 con 0x7fd950004d40 2024-03-06T07:22:29.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9601eb420 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400fea0 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.904+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9601eb9e0 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.905+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96004f330 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.905+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd954012040 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.905+0000 7fd95d7fa640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 0x7fd94406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.906+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7fd95408a820 con 0x7fd950004d40 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.906+0000 7fd95ffff640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 0x7fd94406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.906+0000 7fd95ffff640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 0x7fd94406d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009a00 tx=0x7fd94c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:29.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:29.910+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd954019030 con 0x7fd950004d40 2024-03-06T07:22:30.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.126+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd96006cda0 con 0x7fd950004d40 2024-03-06T07:22:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.127+0000 7fd95d7fa640 1 -- 172.21.15.17:0/302204185 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3696 (secure 0 0 0) 0x7fd9540596c0 con 0x7fd950004d40 2024-03-06T07:22:30.129 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:30.129 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5590,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/3006853169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3006853169},{"type":"v1","addr":"172.21.15.163:6837","nonce":3006853169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6022,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/461807454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":461807454},{"type":"v1","addr":"172.21.15.163:6836","nonce":461807454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6186,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/36892111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":36892111},{"type":"v1","addr":"172.21.15.17:6835","nonce":36892111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":6198,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2785438993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2785438993},{"type":"v1","addr":"172.21.15.17:6834","nonce":2785438993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[]} 2024-03-06T07:22:30.129 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 52 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 msgr2=0x7fd94406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 0x7fd94406d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009a00 tx=0x7fd94c008040 comp rx=0 tx=0).stop 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 msgr2=0x7fd960157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd9540119d0 tx=0x7fd954011e90 comp rx=0 tx=0).stop 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 shutdown_connections 2024-03-06T07:22:30.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd94406b500 0x7fd94406d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd960148b10 0x7fd960148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9600c4210 0x7fd9600c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 --2- 172.21.15.17:0/302204185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd950004d40 0x7fd960157e10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 >> 172.21.15.17:0/302204185 conn(0x7fd9600ac560 msgr2=0x7fd9600aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 shutdown_connections 2024-03-06T07:22:30.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.129+0000 7fd967037640 1 -- 172.21.15.17:0/302204185 wait complete. 2024-03-06T07:22:30.141 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:22:30.142 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:22:30.142 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:22:30.142 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:30.142 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:22:30.142 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:22:30.142 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:22:30.143 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:30.143 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:22:30.143 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:22:30.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:22:30.143 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:30.143 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:22:30.143 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:22:30.144 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:22:30.144 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:22:30.151 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:22:30.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:22:30.152 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:22:30.152 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:22:30.154 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:22:30.154 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:22:30.155 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:22:30.155 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:22:30.194 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:22:30.194 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:30.195 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:22:30.195 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:30.197 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:22:30.197 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:30.198 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:22:30.198 DEBUG:teuthology.parallel:result is None 2024-03-06T07:22:30.198 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:22:30.270 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:22:30.270 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:22:30.280 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:22:30.280 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:22:30.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 Processor -- start 2024-03-06T07:22:30.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 -- start start 2024-03-06T07:22:30.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea4155130 con 0x7f9ea40c8950 2024-03-06T07:22:30.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea4155440 con 0x7f9ea40c69f0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9eaa837640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40ad480 con 0x7f9ea41558d0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.302+0000 7f9ea3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42012/0 (socket says 172.21.15.17:42012) 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 -- 172.21.15.17:0/2858814150 learned_addr learned my addr 172.21.15.17:0/2858814150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea8dd4640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 -- 172.21.15.17:0/2858814150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 msgr2=0x7f9ea4155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea4155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 -- 172.21.15.17:0/2858814150 >> v1:172.21.15.163:6789/0 conn(0x7f9ea40c8950 legacy=0x7f9ea40acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.303+0000 7f9ea3fff640 1 -- 172.21.15.17:0/2858814150 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40c5d50 con 0x7f9ea40c69f0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009870 tx=0x7f9e9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5282e21853893fd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2858814150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94035020 con 0x7f9ea40c69f0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2858814150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94031640 con 0x7f9ea40c69f0 2024-03-06T07:22:30.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9ea2ffd640 1 -- 172.21.15.17:0/2858814150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94035020 con 0x7f9ea40c69f0 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 -- 172.21.15.17:0/2858814150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c69f0 msgr2=0x7f9ea40c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009870 tx=0x7f9e9402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 -- 172.21.15.17:0/2858814150 shutdown_connections 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea4155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 --2- 172.21.15.17:0/2858814150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c69f0 0x7f9ea40c6dd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 -- 172.21.15.17:0/2858814150 >> 172.21.15.17:0/2858814150 conn(0x7f9ea40cae70 msgr2=0x7f9ea40cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 -- 172.21.15.17:0/2858814150 shutdown_connections 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.304+0000 7f9eaa837640 1 -- 172.21.15.17:0/2858814150 wait complete. 2024-03-06T07:22:30.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.305+0000 7f9eaa837640 1 Processor -- start 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 -- start start 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 0x7f9ea4149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea406c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4152a20 con 0x7f9ea40c8950 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40c6310 con 0x7f9ea40c69f0 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9eaa837640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4152ba0 con 0x7f9ea41558d0 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9ea37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9ea37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42020/0 (socket says 172.21.15.17:42020) 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.306+0000 7f9ea37fe640 1 -- 172.21.15.17:0/1850330920 learned_addr learned my addr 172.21.15.17:0/1850330920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 0x7f9ea4149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea8dd4640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea406c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 -- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 msgr2=0x7f9ea406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea406c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 -- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 msgr2=0x7f9ea4149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 0x7f9ea4149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 -- 172.21.15.17:0/1850330920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea406b8a0 con 0x7f9ea40c8950 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea8dd4640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 0x7f9ea4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea37fe640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c480 tx=0x7f9e9800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980052d0 con 0x7f9ea40c8950 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98005470 con 0x7f9ea40c8950 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea41497d0 con 0x7f9ea40c8950 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98012870 con 0x7f9ea40c8950 2024-03-06T07:22:30.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.307+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea41e13c0 con 0x7f9ea40c8950 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.308+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea404f330 con 0x7f9ea40c8950 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.309+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e98012a10 con 0x7f9ea40c8950 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.309+0000 7f9ea0ff9640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 0x7f9e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.309+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f9e98089de0 con 0x7f9ea40c8950 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.309+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 0x7f9e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:30.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.310+0000 7f9ea3fff640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 0x7f9e9006d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e9403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:30.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.313+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ea4155130 con 0x7f9ea40c8950 2024-03-06T07:22:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:30.510+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9ea40c4d80 con 0x7f9ea40c8950 2024-03-06T07:22:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.327+0000 7f9ea0ff9640 1 -- 172.21.15.17:0/1850330920 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e98058d10 con 0x7f9ea40c8950 2024-03-06T07:22:31.329 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 msgr2=0x7f9e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 0x7f9e9006d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e9403c040 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 msgr2=0x7f9ea406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c480 tx=0x7f9e9800c940 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 shutdown_connections 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9006b4d0 0x7f9e9006d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea41558d0 0x7f9ea406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea40c8950 0x7f9ea406afc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 --2- 172.21.15.17:0/1850330920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea40c69f0 0x7f9ea4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 >> 172.21.15.17:0/1850330920 conn(0x7f9ea40cae70 msgr2=0x7f9ea41501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 shutdown_connections 2024-03-06T07:22:31.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.331+0000 7f9eaa837640 1 -- 172.21.15.17:0/1850330920 wait complete. 2024-03-06T07:22:31.346 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:22:31.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.452+0000 7f4598cf4640 1 Processor -- start 2024-03-06T07:22:31.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.452+0000 7f4598cf4640 1 -- start start 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4598cf4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45940c88c0 0x7f4594155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4598cf4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45940c73d0 con 0x7f459414f360 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4598cf4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45940be2e0 con 0x7f4594158b60 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4598cf4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4594159700 con 0x7f45940c88c0 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f459259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f459414f360 0x7f45940c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58974/0 (socket says 172.21.15.17:58974) 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f459259c640 1 -- 172.21.15.17:0/517069660 learned_addr learned my addr 172.21.15.17:0/517069660 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f458bfff640 1 --2- 172.21.15.17:0/517069660 >> v2:172.21.15.163:3300/0 conn(0x7f45940c88c0 0x7f4594155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127022083 0 0) 0x7f45940c73d0 con 0x7f459414f360 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4580003520 con 0x7f459414f360 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005774317 0 0) 0x7f4580003520 con 0x7f459414f360 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 >> v2:172.21.15.163:3300/0 conn(0x7f45940c88c0 msgr2=0x7f4594155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 --2- 172.21.15.17:0/517069660 >> v2:172.21.15.163:3300/0 conn(0x7f45940c88c0 0x7f4594155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 >> v1:172.21.15.163:6789/0 conn(0x7f4594158b60 legacy=0x7f4594158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940c7670 con 0x7f459414f360 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4584001670 con 0x7f459414f360 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 >> v1:172.21.15.17:6789/0 conn(0x7f459414f360 legacy=0x7f45940c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45800039b0 0x7f4580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 0x7f4580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800051c0 0x7f458000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4594159700 con 0x7f4580004800 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458000c160 con 0x7f45800051c0 2024-03-06T07:22:31.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.453+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458000c360 con 0x7f45800039b0 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 0x7f4580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f4592d9d640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800051c0 0x7f458000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f459259c640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45800039b0 0x7f4580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 -- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45800039b0 msgr2=0x7f4580003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45800039b0 0x7f4580003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 -- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800051c0 msgr2=0x7f458000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45800051c0 0x7f458000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f4592d9d640 1 -- 172.21.15.17:0/517069660 reap_dead start 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.454+0000 7f458bfff640 1 -- 172.21.15.17:0/517069660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580003520 con 0x7f4580004800 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f458bfff640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 0x7f4580004c80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f457c009a90 tx=0x7f4580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4a6111692cc2a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c039070 con 0x7f4580004800 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c02c680 con 0x7f4580004800 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4591d9b640 1 -- 172.21.15.17:0/517069660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c009690 con 0x7f4580004800 2024-03-06T07:22:31.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 -- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 msgr2=0x7f4580004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 0x7f4580004c80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f457c009a90 tx=0x7f4580005830 comp rx=0 tx=0).stop 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 -- 172.21.15.17:0/517069660 shutdown_connections 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 --2- 172.21.15.17:0/517069660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4580004800 0x7f4580004c80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 -- 172.21.15.17:0/517069660 >> 172.21.15.17:0/517069660 conn(0x7f45940ac570 msgr2=0x7f45940ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 -- 172.21.15.17:0/517069660 shutdown_connections 2024-03-06T07:22:31.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.455+0000 7f4598cf4640 1 -- 172.21.15.17:0/517069660 wait complete. 2024-03-06T07:22:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 Processor -- start 2024-03-06T07:22:31.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 -- start start 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580004800 0x7f45941eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45940bea90 0x7f45940c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45940c7c30 con 0x7f45940bf530 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4594155c10 con 0x7f45940bea90 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.457+0000 7f4598cf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4594155d90 con 0x7f4580004800 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35352/0 (socket says 172.21.15.17:35352) 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 -- 172.21.15.17:0/3238285842 learned_addr learned my addr 172.21.15.17:0/3238285842 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f458bfff640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45940bea90 0x7f45940c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f459259c640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580004800 0x7f45941eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 -- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580004800 msgr2=0x7f45941eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580004800 0x7f45941eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 -- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45940bea90 msgr2=0x7f45940c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45940bea90 0x7f45940c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 -- 172.21.15.17:0/3238285842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940be550 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4592d9d640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4594067c30 tx=0x7f458c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c01a070 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c00fe90 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45940be830 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c003970 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.458+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941ee4b0 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.459+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f459404f330 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.460+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f458c01d070 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.460+0000 7f458affd640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 0x7f457406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.460+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f458c0893a0 con 0x7f45940bf530 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.460+0000 7f459259c640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 0x7f457406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.461+0000 7f459259c640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 0x7f457406d9c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4584001cd0 tx=0x7f4584001c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.465+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f458c057e20 con 0x7f45940bf530 2024-03-06T07:22:31.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.713+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f459406cda0 con 0x7f45940bf530 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.713+0000 7f458affd640 1 -- 172.21.15.17:0/3238285842 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f458c05bec0 con 0x7f45940bf530 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:22:31.715 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:22:31.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.718 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:22:31.719 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.721 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.723 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:22:31.725 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:22:31.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:22:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:22:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:31.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:22:31.727 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 msgr2=0x7f457406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 0x7f457406d9c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4584001cd0 tx=0x7f4584001c60 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 msgr2=0x7f45941eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4594067c30 tx=0x7f458c00fbe0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 shutdown_connections 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457406b500 0x7f457406d9c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45940bf530 0x7f45941eabc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45940bea90 0x7f45940c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 --2- 172.21.15.17:0/3238285842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580004800 0x7f45941eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 >> 172.21.15.17:0/3238285842 conn(0x7f45940ac570 msgr2=0x7f45940c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 shutdown_connections 2024-03-06T07:22:31.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.715+0000 7f4598cf4640 1 -- 172.21.15.17:0/3238285842 wait complete. 2024-03-06T07:22:31.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.829+0000 7f491b556640 1 Processor -- start 2024-03-06T07:22:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.829+0000 7f491b556640 1 -- start start 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f491b556640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49140c5330 0x7f49140c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f491b556640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4914155110 con 0x7f49140c88b0 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f491b556640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49140c8f40 con 0x7f4914158db0 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f491b556640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49141559e0 con 0x7f49140c5330 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f4918af1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49140c88b0 0x7f49141586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58980/0 (socket says 172.21.15.17:58980) 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f4918af1640 1 -- 172.21.15.17:0/1896156260 learned_addr learned my addr 172.21.15.17:0/1896156260 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:31.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257940595 0 0) 0x7f4908001110 con 0x7f49140c88b0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48fc003520 con 0x7f49140c88b0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.830+0000 7f49192f2640 1 --2- 172.21.15.17:0/1896156260 >> v2:172.21.15.163:3300/0 conn(0x7f49140c5330 0x7f49140c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1875511844 0 0) 0x7f48fc003520 con 0x7f49140c88b0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 >> v2:172.21.15.163:3300/0 conn(0x7f49140c5330 msgr2=0x7f49140c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 --2- 172.21.15.17:0/1896156260 >> v2:172.21.15.163:3300/0 conn(0x7f49140c5330 0x7f49140c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 >> v1:172.21.15.163:6789/0 conn(0x7f4914158db0 legacy=0x7f491415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4914155390 con 0x7f49140c88b0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4908001840 con 0x7f49140c88b0 2024-03-06T07:22:31.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 >> v1:172.21.15.17:6789/0 conn(0x7f49140c88b0 legacy=0x7f49141586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48fc0039b0 0x7f48fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f48fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48fc0052e0 0x7f48fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141559e0 con 0x7f48fc004800 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc00c1b0 con 0x7f48fc0039b0 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc00c3b0 con 0x7f48fc0052e0 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f48fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f49192f2640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48fc0039b0 0x7f48fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4919af3640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48fc0052e0 0x7f48fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 -- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48fc0052e0 msgr2=0x7f48fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48fc0052e0 0x7f48fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 -- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48fc0039b0 msgr2=0x7f48fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:31.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48fc0039b0 0x7f48fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4918af1640 1 -- 172.21.15.17:0/1896156260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc003520 con 0x7f48fc004800 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4919af3640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48fc0052e0 0x7f48fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.831+0000 7f4919af3640 1 -- 172.21.15.17:0/1896156260 reap_dead start 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f4918af1640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f48fc004c80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f49140c8d10 tx=0x7f48fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ceb5263472f1a94 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490800c0e0 con 0x7f48fc004800 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908030070 con 0x7f48fc004800 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f4913fff640 1 -- 172.21.15.17:0/1896156260 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080058d0 con 0x7f48fc004800 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f491b556640 1 -- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 msgr2=0x7f48fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.832+0000 7f491b556640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f48fc004c80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f49140c8d10 tx=0x7f48fc005830 comp rx=0 tx=0).stop 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.833+0000 7f491b556640 1 -- 172.21.15.17:0/1896156260 shutdown_connections 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.833+0000 7f491b556640 1 --2- 172.21.15.17:0/1896156260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f48fc004c80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.833+0000 7f491b556640 1 -- 172.21.15.17:0/1896156260 >> 172.21.15.17:0/1896156260 conn(0x7f49140ac560 msgr2=0x7f49140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.833+0000 7f491b556640 1 -- 172.21.15.17:0/1896156260 shutdown_connections 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.833+0000 7f491b556640 1 -- 172.21.15.17:0/1896156260 wait complete. 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 Processor -- start 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 -- start start 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f491414ed40 0x7f49141df120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49141df660 0x7f49141edee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141529d0 con 0x7f48fc004800 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491406d090 con 0x7f49141df660 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f491b556640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4914152b50 con 0x7f491414ed40 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.834+0000 7f49192f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35366/0 (socket says 172.21.15.17:35366) 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 -- 172.21.15.17:0/983054989 learned_addr learned my addr 172.21.15.17:0/983054989 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f4918af1640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f491414ed40 0x7f49141df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f4919af3640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49141df660 0x7f49141edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 -- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f491414ed40 msgr2=0x7f49141df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f491414ed40 0x7f49141df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 -- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49141df660 msgr2=0x7f49141edee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49141df660 0x7f49141edee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 -- 172.21.15.17:0/983054989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140c43b0 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f49192f2640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b820 tx=0x7f490c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c018070 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c010830 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491414e290 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c010c00 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.835+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49141eebb0 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.836+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49141eedf0 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.837+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f490c01b070 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.837+0000 7f4911ffb640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 0x7f490006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.837+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f490c08a250 con 0x7f48fc004800 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.837+0000 7f4918af1640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 0x7f490006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.838+0000 7f4918af1640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 0x7f490006d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f4908007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:31.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:31.842+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49141eedf0 con 0x7f48fc004800 2024-03-06T07:22:32.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.022+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f491404f330 con 0x7f48fc004800 2024-03-06T07:22:32.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.023+0000 7f4911ffb640 1 -- 172.21.15.17:0/983054989 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+13165 (secure 0 0 0) 0x7f49140c43b0 con 0x7f48fc004800 2024-03-06T07:22:32.024 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:32.025 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":71,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:31.327136+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 msgr2=0x7f490006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 0x7f490006d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f4908007000 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 msgr2=0x7f491414c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b820 tx=0x7f490c00bc20 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 shutdown_connections 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f490006b500 0x7f490006d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49141df660 0x7f49141edee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f491414ed40 0x7f49141df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 --2- 172.21.15.17:0/983054989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48fc004800 0x7f491414c290 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 >> 172.21.15.17:0/983054989 conn(0x7f49140ac560 msgr2=0x7f4914153420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 shutdown_connections 2024-03-06T07:22:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.025+0000 7f491b556640 1 -- 172.21.15.17:0/983054989 wait complete. 2024-03-06T07:22:32.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:32.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:32.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.140+0000 7f5a67e34640 1 Processor -- start 2024-03-06T07:22:32.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 -- start start 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a6006afc0 0x7f5a6006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a600c2580 0x7f5a60159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a600c5840 con 0x7f5a600c2120 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a60158790 con 0x7f5a6006afc0 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a67e34640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a600c5b20 con 0x7f5a600c2580 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a65bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a6006afc0 0x7f5a6006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a653cf640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5a600c2120 0x7f5a60158c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58984/0 (socket says 172.21.15.17:58984) 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.141+0000 7f5a653cf640 1 -- 172.21.15.17:0/2530858694 learned_addr learned my addr 172.21.15.17:0/2530858694 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a663d1640 1 --2- 172.21.15.17:0/2530858694 >> v2:172.21.15.163:3300/0 conn(0x7f5a600c2580 0x7f5a60159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a64bce640 1 -- 172.21.15.17:0/2530858694 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256636181 0 0) 0x7f5a600c5840 con 0x7f5a600c2120 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a64bce640 1 -- 172.21.15.17:0/2530858694 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a4c003520 con 0x7f5a600c2120 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a65bd0640 1 -- 172.21.15.17:0/2530858694 >> v2:172.21.15.163:3300/0 conn(0x7f5a600c2580 msgr2=0x7f5a60159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a65bd0640 1 --2- 172.21.15.17:0/2530858694 >> v2:172.21.15.163:3300/0 conn(0x7f5a600c2580 0x7f5a60159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a65bd0640 1 -- 172.21.15.17:0/2530858694 >> v1:172.21.15.17:6789/0 conn(0x7f5a600c2120 legacy=0x7f5a60158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a65bd0640 1 -- 172.21.15.17:0/2530858694 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600c5e00 con 0x7f5a6006afc0 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.142+0000 7f5a65bd0640 1 --2- 172.21.15.17:0/2530858694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a6006afc0 0x7f5a6006b3a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006d380 tx=0x7f5a5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1269a73f64a4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a64bce640 1 -- 172.21.15.17:0/2530858694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0315a0 con 0x7f5a6006afc0 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a64bce640 1 -- 172.21.15.17:0/2530858694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c037070 con 0x7f5a6006afc0 2024-03-06T07:22:32.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a64bce640 1 -- 172.21.15.17:0/2530858694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c031940 con 0x7f5a6006afc0 2024-03-06T07:22:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 -- 172.21.15.17:0/2530858694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a6006afc0 msgr2=0x7f5a6006b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 --2- 172.21.15.17:0/2530858694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a6006afc0 0x7f5a6006b3a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006d380 tx=0x7f5a5c02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 -- 172.21.15.17:0/2530858694 shutdown_connections 2024-03-06T07:22:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 --2- 172.21.15.17:0/2530858694 >> v2:172.21.15.163:3300/0 conn(0x7f5a600c2580 0x7f5a60159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 --2- 172.21.15.17:0/2530858694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a6006afc0 0x7f5a6006b3a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 -- 172.21.15.17:0/2530858694 >> 172.21.15.17:0/2530858694 conn(0x7f5a600c4220 msgr2=0x7f5a600c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 -- 172.21.15.17:0/2530858694 shutdown_connections 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.143+0000 7f5a67e34640 1 -- 172.21.15.17:0/2530858694 wait complete. 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.144+0000 7f5a67e34640 1 Processor -- start 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.144+0000 7f5a67e34640 1 -- start start 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a6006afc0 0x7f5a60072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 0x7f5a601e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600c2b30 con 0x7f5a600c2120 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60155220 con 0x7f5a600c2580 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a67e34640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a600c2cb0 con 0x7f5a6006afc0 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a653cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a653cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35390/0 (socket says 172.21.15.17:35390) 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a653cf640 1 -- 172.21.15.17:0/1333015594 learned_addr learned my addr 172.21.15.17:0/1333015594 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.145+0000 7f5a663d1640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 0x7f5a601e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 -- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a6006afc0 msgr2=0x7f5a60072730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a6006afc0 0x7f5a60072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 -- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 msgr2=0x7f5a601e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 0x7f5a601e1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 -- 172.21.15.17:0/1333015594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a601e20b0 con 0x7f5a600c2120 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a663d1640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 0x7f5a601e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a653cf640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c430 tx=0x7f5a5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50017070 con 0x7f5a600c2120 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a50005ce0 con 0x7f5a600c2120 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6014c990 con 0x7f5a600c2120 2024-03-06T07:22:32.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a500072f0 con 0x7f5a600c2120 2024-03-06T07:22:32.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.146+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a601ee470 con 0x7f5a600c2120 2024-03-06T07:22:32.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.147+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6004f330 con 0x7f5a600c2120 2024-03-06T07:22:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.147+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a5001a070 con 0x7f5a600c2120 2024-03-06T07:22:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.148+0000 7f5a567fc640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 0x7f5a4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.148+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5a50089490 con 0x7f5a600c2120 2024-03-06T07:22:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.148+0000 7f5a65bd0640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 0x7f5a4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.149+0000 7f5a65bd0640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 0x7f5a4406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c036000 tx=0x7f5a5c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.152+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a50058560 con 0x7f5a600c2120 2024-03-06T07:22:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.333+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a600c5620 con 0x7f5a600c2120 2024-03-06T07:22:32.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.333+0000 7f5a567fc640 1 -- 172.21.15.17:0/1333015594 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a50057e60 con 0x7f5a600c2120 2024-03-06T07:22:32.335 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:22:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 msgr2=0x7f5a4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 0x7f5a4406d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c036000 tx=0x7f5a5c030000 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 msgr2=0x7f5a601e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c430 tx=0x7f5a5000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 shutdown_connections 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a4406b4d0 0x7f5a4406d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a600c2580 0x7f5a601e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600c2120 0x7f5a601e1630 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.335+0000 7f5a67e34640 1 --2- 172.21.15.17:0/1333015594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a6006afc0 0x7f5a60072730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.336+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 >> 172.21.15.17:0/1333015594 conn(0x7f5a600c4220 msgr2=0x7f5a6006cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.336+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 shutdown_connections 2024-03-06T07:22:32.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.336+0000 7f5a67e34640 1 -- 172.21.15.17:0/1333015594 wait complete. 2024-03-06T07:22:32.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:22:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.447+0000 7f59cb7d6640 1 Processor -- start 2024-03-06T07:22:32.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 -- start start 2024-03-06T07:22:32.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59c41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59c40c1aa0 0x7f59c40c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c40c28d0 con 0x7f59c40c5c60 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c40c2680 con 0x7f59c40c1aa0 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.448+0000 7f59cb7d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40c2bb0 con 0x7f59c414f790 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c9d73640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f59c40c5c60 0x7f59c4155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58994/0 (socket says 172.21.15.17:58994) 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c9d73640 1 -- 172.21.15.17:0/782974329 learned_addr learned my addr 172.21.15.17:0/782974329 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c9572640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59c41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c8d71640 1 --2- 172.21.15.17:0/782974329 >> v2:172.21.15.163:3300/0 conn(0x7f59c40c1aa0 0x7f59c40c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546958924 0 0) 0x7f59c40c28d0 con 0x7f59c40c5c60 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b0003520 con 0x7f59c40c5c60 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2228256427 0 0) 0x7f59b0003520 con 0x7f59c40c5c60 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 msgr2=0x7f59c41529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59c41529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 >> v2:172.21.15.163:3300/0 conn(0x7f59c40c1aa0 msgr2=0x7f59c40c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 --2- 172.21.15.17:0/782974329 >> v2:172.21.15.163:3300/0 conn(0x7f59c40c1aa0 0x7f59c40c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c40c2e90 con 0x7f59c40c5c60 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f59bc0019f0 con 0x7f59c40c5c60 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 >> v1:172.21.15.17:6789/0 conn(0x7f59c40c5c60 legacy=0x7f59c4155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40c2680 con 0x7f59b0004800 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40c2bb0 con 0x7f59b00039b0 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b000b9c0 con 0x7f59c414f790 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c8d71640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c9572640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.449+0000 7f59c9d73640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 -- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 msgr2=0x7f59b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c414f790 0x7f59b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 -- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00039b0 msgr2=0x7f59b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 -- 172.21.15.17:0/782974329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0003520 con 0x7f59b0004800 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c9572640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59c8d71640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ac0 tx=0x7f59b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ac53191e60473 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4039070 con 0x7f59b0004800 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b402c710 con 0x7f59b0004800 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.450+0000 7f59bbfff640 1 -- 172.21.15.17:0/782974329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4009690 con 0x7f59b0004800 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 -- 172.21.15.17:0/782974329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59b0004800 msgr2=0x7f59b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/782974329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ac0 tx=0x7f59b0005830 comp rx=0 tx=0).stop 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59c9d73640 1 -- 172.21.15.17:0/782974329 reap_dead start 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 -- 172.21.15.17:0/782974329 shutdown_connections 2024-03-06T07:22:32.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 -- 172.21.15.17:0/782974329 >> 172.21.15.17:0/782974329 conn(0x7f59c40c6710 msgr2=0x7f59c40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 -- 172.21.15.17:0/782974329 shutdown_connections 2024-03-06T07:22:32.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.451+0000 7f59cb7d6640 1 -- 172.21.15.17:0/782974329 wait complete. 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.452+0000 7f59cb7d6640 1 Processor -- start 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.452+0000 7f59cb7d6640 1 -- start start 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b0004800 0x7f59c41518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c4145f00 0x7f59c414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c406cc00 con 0x7f59c41452d0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40c0d60 con 0x7f59b0004800 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59cb7d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c406cd80 con 0x7f59c4145f00 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35406/0 (socket says 172.21.15.17:35406) 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 -- 172.21.15.17:0/3341506980 learned_addr learned my addr 172.21.15.17:0/3341506980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9572640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b0004800 0x7f59c41518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c8d71640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c4145f00 0x7f59c414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 -- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c4145f00 msgr2=0x7f59c414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c4145f00 0x7f59c414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 -- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b0004800 msgr2=0x7f59c41518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b0004800 0x7f59c41518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 -- 172.21.15.17:0/3341506980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4145cc0 con 0x7f59c41452d0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.453+0000 7f59c9d73640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f59c40c3170 tx=0x7f59bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.454+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00eb30 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.454+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc018070 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.454+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c41533c0 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.454+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc013420 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.454+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c4154d60 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.455+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c404f330 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.455+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59bc0135c0 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.456+0000 7f59b9ffb640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 0x7f59a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.456+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f59bc08a140 con 0x7f59c41452d0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.456+0000 7f59c9572640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 0x7f59a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.456+0000 7f59c9572640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 0x7f59a806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59c000bd00 tx=0x7f59c0013040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.460+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f59bc058bc0 con 0x7f59c41452d0 2024-03-06T07:22:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.641+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59c41e9e30 con 0x7f59c41452d0 2024-03-06T07:22:32.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.641+0000 7f59b9ffb640 1 -- 172.21.15.17:0/3341506980 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+65 (secure 0 0 0) 0x7f59bc05cc60 con 0x7f59c41452d0 2024-03-06T07:22:32.644 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:32.644 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:22:32.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:32.644 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:22:32.644 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:22:32.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:32.645 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 msgr2=0x7f59a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 0x7f59a806d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59c000bd00 tx=0x7f59c0013040 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 msgr2=0x7f59c4145780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f59c40c3170 tx=0x7f59bc00e530 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 shutdown_connections 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f59a806b500 0x7f59a806d9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59c41452d0 0x7f59c4145780 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59c4145f00 0x7f59c414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 --2- 172.21.15.17:0/3341506980 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59b0004800 0x7f59c41518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 >> 172.21.15.17:0/3341506980 conn(0x7f59c40c6710 msgr2=0x7f59c40c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 shutdown_connections 2024-03-06T07:22:32.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.644+0000 7f59cb7d6640 1 -- 172.21.15.17:0/3341506980 wait complete. 2024-03-06T07:22:32.656 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:22:32.656 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:22:32.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.759+0000 7f5a91081640 1 Processor -- start 2024-03-06T07:22:32.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.759+0000 7f5a91081640 1 -- start start 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a91081640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a8c0c3c20 0x7f5a8c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a91081640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c152e90 con 0x7f5a8c06f8d0 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a91081640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a8c06fd40 con 0x7f5a8c153910 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a91081640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8c153480 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a8c0c3c20 0x7f5a8c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5a8c06f8d0 0x7f5a8c156480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59002/0 (socket says 172.21.15.17:59002) 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8a59c640 1 -- 172.21.15.17:0/1942591506 learned_addr learned my addr 172.21.15.17:0/1942591506 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8ad9d640 1 -- 172.21.15.17:0/1942591506 >> v1:172.21.15.163:6789/0 conn(0x7f5a8c153910 legacy=0x7f5a8c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8ad9d640 1 -- 172.21.15.17:0/1942591506 >> v1:172.21.15.17:6789/0 conn(0x7f5a8c06f8d0 legacy=0x7f5a8c156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a8ad9d640 1 -- 172.21.15.17:0/1942591506 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0c6f10 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.760+0000 7f5a89d9b640 1 -- 172.21.15.17:0/1942591506 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832397126 0 0) 0x7f5a80001110 con 0x7f5a8c06f8d0 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.761+0000 7f5a8ad9d640 1 --2- 172.21.15.17:0/1942591506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c0c3c20 0x7f5a8c0c6930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c153600 tx=0x7f5a8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b68a56a98e6001c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.761+0000 7f5a89d9b640 1 -- 172.21.15.17:0/1942591506 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84035020 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.761+0000 7f5a89d9b640 1 -- 172.21.15.17:0/1942591506 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a84031830 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.761+0000 7f5a89d9b640 1 -- 172.21.15.17:0/1942591506 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a84031c00 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 -- 172.21.15.17:0/1942591506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c0c3c20 msgr2=0x7f5a8c0c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 --2- 172.21.15.17:0/1942591506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c0c3c20 0x7f5a8c0c6930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c153600 tx=0x7f5a8402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 -- 172.21.15.17:0/1942591506 shutdown_connections 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 --2- 172.21.15.17:0/1942591506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c0c3c20 0x7f5a8c0c6930 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 -- 172.21.15.17:0/1942591506 >> 172.21.15.17:0/1942591506 conn(0x7f5a8c0c4300 msgr2=0x7f5a8c0c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 -- 172.21.15.17:0/1942591506 shutdown_connections 2024-03-06T07:22:32.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.762+0000 7f5a91081640 1 -- 172.21.15.17:0/1942591506 wait complete. 2024-03-06T07:22:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.763+0000 7f5a91081640 1 Processor -- start 2024-03-06T07:22:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.763+0000 7f5a91081640 1 -- start start 2024-03-06T07:22:32.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8c06f8d0 0x7f5a8c14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a8c0c3c20 0x7f5a8c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c153d60 con 0x7f5a8c153910 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c153110 con 0x7f5a8c06f8d0 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a91081640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c06b6d0 con 0x7f5a8c0c3c20 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35420/0 (socket says 172.21.15.17:35420) 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 -- 172.21.15.17:0/3259126111 learned_addr learned my addr 172.21.15.17:0/3259126111 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8ad9d640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8c06f8d0 0x7f5a8c14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8a59c640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a8c0c3c20 0x7f5a8c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 -- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a8c0c3c20 msgr2=0x7f5a8c0acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a8c0c3c20 0x7f5a8c0acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 -- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8c06f8d0 msgr2=0x7f5a8c14f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8c06f8d0 0x7f5a8c14f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:32.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 -- 172.21.15.17:0/3259126111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c157030 con 0x7f5a8c153910 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.764+0000 7f5a8b59e640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b970 tx=0x7f5a7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.765+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c019070 con 0x7f5a8c153910 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.765+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00cb40 con 0x7f5a8c153910 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.765+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c1567b0 con 0x7f5a8c153910 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.765+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c014440 con 0x7f5a8c153910 2024-03-06T07:22:32.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.765+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c14fd30 con 0x7f5a8c153910 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.766+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c04f330 con 0x7f5a8c153910 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.766+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a7c01c070 con 0x7f5a8c153910 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.766+0000 7f5a737fe640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 0x7f5a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.766+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5a7c089b10 con 0x7f5a8c153910 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.767+0000 7f5a8ad9d640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 0x7f5a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:32.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.767+0000 7f5a8ad9d640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 0x7f5a7806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a84036000 tx=0x7f5a8403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:32.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.771+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a7c058590 con 0x7f5a8c153910 2024-03-06T07:22:32.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:32.953+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5a8c069020 con 0x7f5a8c153910 2024-03-06T07:22:33.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.337+0000 7f5a737fe640 1 -- 172.21.15.17:0/3259126111 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5a7c05c630 con 0x7f5a8c153910 2024-03-06T07:22:33.339 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 msgr2=0x7f5a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 0x7f5a7806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a84036000 tx=0x7f5a8403d040 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 msgr2=0x7f5a8c1df8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b970 tx=0x7f5a7c00be30 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 shutdown_connections 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a7806b4d0 0x7f5a7806d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a8c153910 0x7f5a8c1df8a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a8c0c3c20 0x7f5a8c0acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 --2- 172.21.15.17:0/3259126111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8c06f8d0 0x7f5a8c14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 >> 172.21.15.17:0/3259126111 conn(0x7f5a8c0c4300 msgr2=0x7f5a8c0c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 shutdown_connections 2024-03-06T07:22:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.340+0000 7f5a91081640 1 -- 172.21.15.17:0/3259126111 wait complete. 2024-03-06T07:22:33.356 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:22:33.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.463+0000 7f940429d640 1 Processor -- start 2024-03-06T07:22:33.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.463+0000 7f940429d640 1 -- start start 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940429d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 0x7f93fc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940429d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f93fc06ca70 0x7f93fc0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940429d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc0b0450 con 0x7f93fc0c5380 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940429d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc06c020 con 0x7f93fc06ca70 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940429d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0c77f0 con 0x7f93fc0b0070 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f9401838640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f93fc06ca70 0x7f93fc0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f9401838640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f93fc06ca70 0x7f93fc0c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35434/0 (socket says 172.21.15.17:35434) 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f9401838640 1 -- 172.21.15.17:0/2184604477 learned_addr learned my addr 172.21.15.17:0/2184604477 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f940283a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f93fc0c5380 0x7f93fc14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59010/0 (socket says 172.21.15.17:59010) 2024-03-06T07:22:33.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.464+0000 7f9401838640 1 -- 172.21.15.17:0/2184604477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 msgr2=0x7f93fc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9402039640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 0x7f93fc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401838640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 0x7f93fc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401037640 1 -- 172.21.15.17:0/2184604477 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897018232 0 0) 0x7f93ec001490 con 0x7f93fc0c5380 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401838640 1 -- 172.21.15.17:0/2184604477 >> v1:172.21.15.17:6789/0 conn(0x7f93fc0c5380 legacy=0x7f93fc14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401838640 1 -- 172.21.15.17:0/2184604477 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0c7ad0 con 0x7f93fc06ca70 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9402039640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 0x7f93fc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401838640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc06ca70 0x7f93fc0c8900 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009870 tx=0x7f93f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537e1988a9859732 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401037640 1 -- 172.21.15.17:0/2184604477 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0035020 con 0x7f93fc06ca70 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401037640 1 -- 172.21.15.17:0/2184604477 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f0031640 con 0x7f93fc06ca70 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.465+0000 7f9401037640 1 -- 172.21.15.17:0/2184604477 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f002d040 con 0x7f93fc06ca70 2024-03-06T07:22:33.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 -- 172.21.15.17:0/2184604477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc06ca70 msgr2=0x7f93fc0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc06ca70 0x7f93fc0c8900 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009870 tx=0x7f93f002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 -- 172.21.15.17:0/2184604477 shutdown_connections 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc06ca70 0x7f93fc0c8900 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 --2- 172.21.15.17:0/2184604477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc0b0070 0x7f93fc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 -- 172.21.15.17:0/2184604477 >> 172.21.15.17:0/2184604477 conn(0x7f93fc0ac590 msgr2=0x7f93fc0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 -- 172.21.15.17:0/2184604477 shutdown_connections 2024-03-06T07:22:33.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.466+0000 7f940429d640 1 -- 172.21.15.17:0/2184604477 wait complete. 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.467+0000 7f940429d640 1 Processor -- start 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.467+0000 7f940429d640 1 -- start start 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc06ca70 0x7f93fc14ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 0x7f93fc0c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06cef0 con 0x7f93fc0c5380 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06c550 con 0x7f93fc0b0070 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940429d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc06da80 con 0x7f93fc06ca70 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35444/0 (socket says 172.21.15.17:35444) 2024-03-06T07:22:33.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 -- 172.21.15.17:0/1481496937 learned_addr learned my addr 172.21.15.17:0/1481496937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f9402039640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc06ca70 0x7f93fc14ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f9401838640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 0x7f93fc0c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 -- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc06ca70 msgr2=0x7f93fc14ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc06ca70 0x7f93fc14ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 -- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 msgr2=0x7f93fc0c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 0x7f93fc0c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 -- 172.21.15.17:0/1481496937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0c3cb0 con 0x7f93fc0c5380 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f9401838640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 0x7f93fc0c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.468+0000 7f940283a640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00c700 tx=0x7f93ec00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.469+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec018070 con 0x7f93fc0c5380 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.469+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ec00ce10 con 0x7f93fc0c5380 2024-03-06T07:22:33.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.469+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc14e780 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.469+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ec00a860 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.469+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc0c3790 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.470+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93fc04f330 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.470+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93ec01b070 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.471+0000 7f93faffd640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 0x7f93e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.471+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f93ec089660 con 0x7f93fc0c5380 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.471+0000 7f9402039640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 0x7f93e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:33.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.471+0000 7f9402039640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 0x7f93e806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f93fc1550b0 tx=0x7f93f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:33.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.475+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f93fc0b0450 con 0x7f93fc0c5380 2024-03-06T07:22:33.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:33.654+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f93fc06d860 con 0x7f93fc0c5380 2024-03-06T07:22:34.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.342+0000 7f93faffd640 1 -- 172.21.15.17:0/1481496937 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) v1 ==== 170+0+0 (secure 0 0 0) 0x7f93ec057ee0 con 0x7f93fc0c5380 2024-03-06T07:22:34.345 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:22:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.345+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 msgr2=0x7f93e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.345+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 0x7f93e806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f93fc1550b0 tx=0x7f93f4008040 comp rx=0 tx=0).stop 2024-03-06T07:22:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.345+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 msgr2=0x7f93fc0c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.345+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f93ec00c700 tx=0x7f93ec00cbc0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 shutdown_connections 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f93e806b4d0 0x7f93e806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93fc0c5380 0x7f93fc0c4660 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93fc0b0070 0x7f93fc0c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 --2- 172.21.15.17:0/1481496937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93fc06ca70 0x7f93fc14ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.347+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 >> 172.21.15.17:0/1481496937 conn(0x7f93fc0ac590 msgr2=0x7f93fc0ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:34.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.348+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 shutdown_connections 2024-03-06T07:22:34.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.348+0000 7f940429d640 1 -- 172.21.15.17:0/1481496937 wait complete. 2024-03-06T07:22:34.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:22:34.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.483+0000 7f68dbfac640 1 Processor -- start 2024-03-06T07:22:34.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.483+0000 7f68dbfac640 1 -- start start 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d41509c0 0x7f68d406f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68d40ac7b0 0x7f68d406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d406cab0 0x7f68d4153040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40c7450 con 0x7f68d406cab0 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d406f090 con 0x7f68d40ac7b0 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68dbfac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d406f7b0 con 0x7f68d41509c0 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d406cab0 0x7f68d4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d406cab0 0x7f68d4153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35456/0 (socket says 172.21.15.17:35456) 2024-03-06T07:22:34.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 -- 172.21.15.17:0/140614892 learned_addr learned my addr 172.21.15.17:0/140614892 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 -- 172.21.15.17:0/140614892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d41509c0 msgr2=0x7f68d406f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68d9547640 1 --2- 172.21.15.17:0/140614892 >> v2:172.21.15.163:3300/0 conn(0x7f68d40ac7b0 0x7f68d406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68d9d48640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d41509c0 0x7f68d406f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d41509c0 0x7f68d406f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 -- 172.21.15.17:0/140614892 >> v2:172.21.15.163:3300/0 conn(0x7f68d40ac7b0 msgr2=0x7f68d406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 --2- 172.21.15.17:0/140614892 >> v2:172.21.15.163:3300/0 conn(0x7f68d40ac7b0 0x7f68d406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.484+0000 7f68da549640 1 -- 172.21.15.17:0/140614892 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4156630 con 0x7f68d406cab0 2024-03-06T07:22:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68da549640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d406cab0 0x7f68d4153040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009e50 tx=0x7f68cc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1480a911c3f0a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68d8d46640 1 -- 172.21.15.17:0/140614892 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc02c9e0 con 0x7f68d406cab0 2024-03-06T07:22:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68d8d46640 1 -- 172.21.15.17:0/140614892 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc02cb80 con 0x7f68d406cab0 2024-03-06T07:22:34.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68d8d46640 1 -- 172.21.15.17:0/140614892 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc0343f0 con 0x7f68d406cab0 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 -- 172.21.15.17:0/140614892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d406cab0 msgr2=0x7f68d4153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d406cab0 0x7f68d4153040 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009e50 tx=0x7f68cc02be40 comp rx=0 tx=0).stop 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 -- 172.21.15.17:0/140614892 shutdown_connections 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d406cab0 0x7f68d4153040 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 --2- 172.21.15.17:0/140614892 >> v2:172.21.15.163:3300/0 conn(0x7f68d40ac7b0 0x7f68d406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 --2- 172.21.15.17:0/140614892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d41509c0 0x7f68d406f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.485+0000 7f68dbfac640 1 -- 172.21.15.17:0/140614892 >> 172.21.15.17:0/140614892 conn(0x7f68d40c45d0 msgr2=0x7f68d40c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.486+0000 7f68dbfac640 1 -- 172.21.15.17:0/140614892 shutdown_connections 2024-03-06T07:22:34.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.486+0000 7f68dbfac640 1 -- 172.21.15.17:0/140614892 wait complete. 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 Processor -- start 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 -- start start 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d406cab0 0x7f68d41dee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41509c0 0x7f68d4151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4156bb0 con 0x7f68d40ac7b0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41568d0 con 0x7f68d41509c0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68dbfac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41573b0 con 0x7f68d406cab0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68d9547640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68d9547640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35458/0 (socket says 172.21.15.17:35458) 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.487+0000 7f68d9547640 1 -- 172.21.15.17:0/2665989007 learned_addr learned my addr 172.21.15.17:0/2665989007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9d48640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d406cab0 0x7f68d41dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68da549640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41509c0 0x7f68d4151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 -- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d406cab0 msgr2=0x7f68d41dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d406cab0 0x7f68d41dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 -- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41509c0 msgr2=0x7f68d4151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41509c0 0x7f68d4151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 -- 172.21.15.17:0/2665989007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d414a550 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68d9547640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c820 tx=0x7f68c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4015070 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c4004020 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4004410 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d4149c50 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.488+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d40a7620 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.489+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d404f330 con 0x7f68d40ac7b0 2024-03-06T07:22:34.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.490+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68c4007830 con 0x7f68d40ac7b0 2024-03-06T07:22:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.490+0000 7f68ca7fc640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.490+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f68c408a020 con 0x7f68d40ac7b0 2024-03-06T07:22:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.490+0000 7f68d9d48640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:34.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.491+0000 7f68d9d48640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 0x7f68b806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68d4150660 tx=0x7f68d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:34.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.495+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68d414a550 con 0x7f68d40ac7b0 2024-03-06T07:22:34.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:34.713+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f68d40a77f0 con 0x7f68d40ac7b0 2024-03-06T07:22:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.344+0000 7f68ca7fc640 1 -- 172.21.15.17:0/2665989007 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f68c4058720 con 0x7f68d40ac7b0 2024-03-06T07:22:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 11 and data pool 12 2024-03-06T07:22:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.350+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 msgr2=0x7f68b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.350+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 0x7f68b806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f68d4150660 tx=0x7f68d0008040 comp rx=0 tx=0).stop 2024-03-06T07:22:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.350+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 msgr2=0x7f68d41e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.350+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f68c400c820 tx=0x7f68c400cce0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 shutdown_connections 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b806b4d0 0x7f68b806d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41509c0 0x7f68d4151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40ac7b0 0x7f68d41e9490 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 --2- 172.21.15.17:0/2665989007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d406cab0 0x7f68d41dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.363+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 >> 172.21.15.17:0/2665989007 conn(0x7f68d40c45d0 msgr2=0x7f68d40c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.373+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 shutdown_connections 2024-03-06T07:22:35.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.374+0000 7f68dbfac640 1 -- 172.21.15.17:0/2665989007 wait complete. 2024-03-06T07:22:35.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.493+0000 7fd7fc0fb640 1 Processor -- start 2024-03-06T07:22:35.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 -- start start 2024-03-06T07:22:35.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f40c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f4158800 con 0x7fd7f41551e0 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f4158ae0 con 0x7fd7f406d9f0 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.494+0000 7fd7fc0fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f4158dc0 con 0x7fd7f40c3280 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35466/0 (socket says 172.21.15.17:35466) 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7fa698640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd7f41551e0 0x7fd7f4155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59016/0 (socket says 172.21.15.17:59016) 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 -- 172.21.15.17:0/3636439910 learned_addr learned my addr 172.21.15.17:0/3636439910 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 -- 172.21.15.17:0/3636439910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 msgr2=0x7fd7f40c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/3636439910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f40c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 -- 172.21.15.17:0/3636439910 >> v1:172.21.15.17:6789/0 conn(0x7fd7f41551e0 legacy=0x7fd7f4155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f8e95640 1 -- 172.21.15.17:0/3636439910 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290189207 0 0) 0x7fd7f4158800 con 0x7fd7f41551e0 2024-03-06T07:22:35.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.495+0000 7fd7f9e97640 1 -- 172.21.15.17:0/3636439910 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f41590a0 con 0x7fd7f406d9f0 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/3636439910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009870 tx=0x7fd7f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6038706431d553d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7f8e95640 1 -- 172.21.15.17:0/3636439910 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0035020 con 0x7fd7f406d9f0 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7f8e95640 1 -- 172.21.15.17:0/3636439910 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f0031640 con 0x7fd7f406d9f0 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7f8e95640 1 -- 172.21.15.17:0/3636439910 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f002d040 con 0x7fd7f406d9f0 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/3636439910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f406d9f0 msgr2=0x7fd7f40a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/3636439910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009870 tx=0x7fd7f002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/3636439910 shutdown_connections 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/3636439910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f40c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/3636439910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f406d9f0 0x7fd7f40a7cd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/3636439910 >> 172.21.15.17:0/3636439910 conn(0x7fd7f40aa630 msgr2=0x7fd7f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/3636439910 shutdown_connections 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.496+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/3636439910 wait complete. 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.497+0000 7fd7fc0fb640 1 Processor -- start 2024-03-06T07:22:35.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 -- start start 2024-03-06T07:22:35.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 0x7fd7f41eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406df50 con 0x7fd7f41551e0 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f406ddd0 con 0x7fd7f406d9f0 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fc0fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f40a8210 con 0x7fd7f40c3280 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fa698640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fa698640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35474/0 (socket says 172.21.15.17:35474) 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7fa698640 1 -- 172.21.15.17:0/1131228006 learned_addr learned my addr 172.21.15.17:0/1131228006 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7f9696640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.498+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 0x7fd7f41eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 -- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 msgr2=0x7fd7f414d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f414d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 -- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 msgr2=0x7fd7f41eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 0x7fd7f41eacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 -- 172.21.15.17:0/1131228006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40ae650 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7f9696640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 0x7fd7f41eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fa698640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c800 tx=0x7fd7ec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec018070 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec004d80 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00a880 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f41583e0 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.499+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f41eb670 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.500+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f404f330 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.500+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7ec01b070 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.501+0000 7fd7ea7fc640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 0x7fd7d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.501+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7ec0898d0 con 0x7fd7f41551e0 2024-03-06T07:22:35.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.501+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 0x7fd7d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.502+0000 7fd7f9e97640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 0x7fd7d806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0036000 tx=0x7fd7f003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.505+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7ec058670 con 0x7fd7f41551e0 2024-03-06T07:22:35.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.687+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7f4073240 con 0x7fd7f41551e0 2024-03-06T07:22:35.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.688+0000 7fd7ea7fc640 1 -- 172.21.15.17:0/1131228006 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fd7ec057f70 con 0x7fd7f41551e0 2024-03-06T07:22:35.690 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:35.690 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":74,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:34.714838+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:35.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 msgr2=0x7fd7d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 0x7fd7d806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0036000 tx=0x7fd7f003c040 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 msgr2=0x7fd7f40a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c800 tx=0x7fd7ec00ccc0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 shutdown_connections 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7d806b4d0 0x7fd7d806d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f41551e0 0x7fd7f40a9a70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f40c3280 0x7fd7f414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 --2- 172.21.15.17:0/1131228006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f406d9f0 0x7fd7f41eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.690+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 >> 172.21.15.17:0/1131228006 conn(0x7fd7f40aa630 msgr2=0x7fd7f406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.691+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 shutdown_connections 2024-03-06T07:22:35.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.691+0000 7fd7fc0fb640 1 -- 172.21.15.17:0/1131228006 wait complete. 2024-03-06T07:22:35.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:35.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.803+0000 7fe1eff72640 1 Processor -- start 2024-03-06T07:22:35.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.803+0000 7fe1eff72640 1 -- start start 2024-03-06T07:22:35.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1eff72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1e80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1eff72640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8072630 con 0x7fe1e80c6780 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1eff72640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e8072910 con 0x7fe1e806da40 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1eff72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e806de20 con 0x7fe1e80c7380 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ed50d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe1e80c6780 0x7fe1e8158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59024/0 (socket says 172.21.15.17:59024) 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ed50d640 1 -- 172.21.15.17:0/2908619724 learned_addr learned my addr 172.21.15.17:0/2908619724 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ee50f640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1e80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929291866 0 0) 0x7fe1e8072630 con 0x7fe1e80c6780 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d0003520 con 0x7fe1e80c6780 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794746477 0 0) 0x7fe1e8072910 con 0x7fe1e806da40 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8072630 con 0x7fe1e806da40 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 650721734 0 0) 0x7fe1d0003520 con 0x7fe1e80c6780 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 msgr2=0x7fe1e80c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1e80c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.804+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 >> v1:172.21.15.163:6789/0 conn(0x7fe1e806da40 legacy=0x7fe1e8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8072bf0 con 0x7fe1e80c6780 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe1dc001670 con 0x7fe1e80c6780 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 >> v1:172.21.15.17:6789/0 conn(0x7fe1e80c6780 legacy=0x7fe1e8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1d0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d0004d40 0x7fe1d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e806de20 con 0x7fe1d0003b70 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d000baf0 con 0x7fe1d0004d40 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d000bcf0 con 0x7fe1e80c7380 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1d0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ee50f640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d0004d40 0x7fe1d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1ed50d640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 msgr2=0x7fe1d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80c7380 0x7fe1d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d0004d40 msgr2=0x7fe1d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1d0004d40 0x7fe1d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.805+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2908619724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0003520 con 0x7fe1d0003b70 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1d0003f70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00018d0 tx=0x7fe1d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158b9defac2dcf21 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0037070 con 0x7fe1d0003b70 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e00052d0 con 0x7fe1d0003b70 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1ecd0c640 1 -- 172.21.15.17:0/2908619724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00014f0 con 0x7fe1d0003b70 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1eff72640 1 -- 172.21.15.17:0/2908619724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 msgr2=0x7fe1d0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2908619724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1d0003f70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00018d0 tx=0x7fe1d0005830 comp rx=0 tx=0).stop 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1ee50f640 1 -- 172.21.15.17:0/2908619724 reap_dead start 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1eff72640 1 -- 172.21.15.17:0/2908619724 shutdown_connections 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.806+0000 7fe1eff72640 1 -- 172.21.15.17:0/2908619724 >> 172.21.15.17:0/2908619724 conn(0x7fe1e80aa630 msgr2=0x7fe1e80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:35.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.807+0000 7fe1eff72640 1 -- 172.21.15.17:0/2908619724 shutdown_connections 2024-03-06T07:22:35.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.807+0000 7fe1eff72640 1 -- 172.21.15.17:0/2908619724 wait complete. 2024-03-06T07:22:35.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 Processor -- start 2024-03-06T07:22:35.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 -- start start 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 0x7fe1e8149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 0x7fe1e81483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c3280 con 0x7fe1d0003b70 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81594a0 con 0x7fe1e8147f70 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1eff72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c3400 con 0x7fe1e8148bc0 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1edd0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1edd0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35484/0 (socket says 172.21.15.17:35484) 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.808+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2495841343 learned_addr learned my addr 172.21.15.17:0/2495841343 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1ed50d640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 0x7fe1e8149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1ee50f640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 0x7fe1e81483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 msgr2=0x7fe1e8149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 0x7fe1e8149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 msgr2=0x7fe1e81483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 0x7fe1e81483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 -- 172.21.15.17:0/2495841343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e81eee40 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1ed50d640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 0x7fe1e8149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1ee50f640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 0x7fe1e81483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1edd0e640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0035040 tx=0x7fe1e00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0037070 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0038040 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e81de790 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00014f0 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.809+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e8148930 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.810+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e804f330 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.811+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1e00052d0 con 0x7fe1d0003b70 2024-03-06T07:22:35.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.811+0000 7fe1e67fc640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 0x7fe1d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:35.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.811+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1e00aba60 con 0x7fe1d0003b70 2024-03-06T07:22:35.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.811+0000 7fe1ed50d640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 0x7fe1d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:35.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.812+0000 7fe1ed50d640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 0x7fe1d406d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806d590 tx=0x7fe1dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:35.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:35.815+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1e007a2b0 con 0x7fe1d0003b70 2024-03-06T07:22:36.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.000+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1e8070640 con 0x7fe1d0003b70 2024-03-06T07:22:36.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.001+0000 7fe1e67fc640 1 -- 172.21.15.17:0/2495841343 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fe1e007e350 con 0x7fe1d0003b70 2024-03-06T07:22:36.003 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:36.004 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":74,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:34.714838+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 msgr2=0x7fe1d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 0x7fe1d406d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806d590 tx=0x7fe1dc009040 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 msgr2=0x7fe1e81e1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0035040 tx=0x7fe1e00012d0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 shutdown_connections 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1d406b500 0x7fe1d406d9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e8147f70 0x7fe1e81483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8148bc0 0x7fe1e8149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 --2- 172.21.15.17:0/2495841343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1d0003b70 0x7fe1e81e1f80 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.003+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 >> 172.21.15.17:0/2495841343 conn(0x7fe1e80aa630 msgr2=0x7fe1e80c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.004+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 shutdown_connections 2024-03-06T07:22:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.004+0000 7fe1eff72640 1 -- 172.21.15.17:0/2495841343 wait complete. 2024-03-06T07:22:36.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:22:36.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.120+0000 7f4d08ab5640 1 Processor -- start 2024-03-06T07:22:36.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.120+0000 7f4d08ab5640 1 -- start start 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d08ab5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d04159330 0x7f4d04160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d08ab5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0406b800 con 0x7f4d0406b3a0 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d08ab5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040c5840 con 0x7f4d0406afc0 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d08ab5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d04158790 con 0x7f4d04159330 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4cfbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4d0406b3a0 0x7f4d04158c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59032/0 (socket says 172.21.15.17:59032) 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4cfbfff640 1 -- 172.21.15.17:0/772765467 learned_addr learned my addr 172.21.15.17:0/772765467 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:36.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953683486 0 0) 0x7f4d0406b800 con 0x7f4d0406b3a0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf0003520 con 0x7f4d0406b3a0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> v2:172.21.15.163:3300/0 conn(0x7f4d04159330 0x7f4d04160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981046217 0 0) 0x7f4cf0003520 con 0x7f4d0406b3a0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 >> v2:172.21.15.163:3300/0 conn(0x7f4d04159330 msgr2=0x7f4d04160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 --2- 172.21.15.17:0/772765467 >> v2:172.21.15.163:3300/0 conn(0x7f4d04159330 0x7f4d04160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 >> v1:172.21.15.163:6789/0 conn(0x7f4d0406afc0 legacy=0x7f4d040c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040c5b20 con 0x7f4d0406b3a0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4cec001670 con 0x7f4d0406b3a0 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> v2:172.21.15.163:3300/0 conn(0x7f4d04159330 0x7f4d04160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 >> v1:172.21.15.17:6789/0 conn(0x7f4d0406b3a0 legacy=0x7f4d04158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00039b0 0x7f4cf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf0004800 0x7f4cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 0x7f4cf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04158790 con 0x7f4cf00052e0 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf000c1b0 con 0x7f4cf00039b0 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.121+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf000c3b0 con 0x7f4cf0004800 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 0x7f4cf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d0259c640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00039b0 0x7f4cf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4cfbfff640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf0004800 0x7f4cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 -- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf0004800 msgr2=0x7f4cf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cf0004800 0x7f4cf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 -- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00039b0 msgr2=0x7f4cf0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:36.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00039b0 0x7f4cf0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 -- 172.21.15.17:0/772765467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf0003520 con 0x7f4cf00052e0 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.122+0000 7f4d02d9d640 1 -- 172.21.15.17:0/772765467 reap_dead start 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 0x7f4cf000b9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0073b0 tx=0x7f4cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960d9cc4d72a8568 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc039070 con 0x7f4cf00052e0 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc02cd90 con 0x7f4cf00052e0 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d01d9b640 1 -- 172.21.15.17:0/772765467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc009c80 con 0x7f4cf00052e0 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 -- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 msgr2=0x7f4cf000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 0x7f4cf000b9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0073b0 tx=0x7f4cf0005830 comp rx=0 tx=0).stop 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 -- 172.21.15.17:0/772765467 shutdown_connections 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/772765467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cf00052e0 0x7f4cf000b9c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 -- 172.21.15.17:0/772765467 >> 172.21.15.17:0/772765467 conn(0x7f4d040c4220 msgr2=0x7f4d040c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 -- 172.21.15.17:0/772765467 shutdown_connections 2024-03-06T07:22:36.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.123+0000 7f4d08ab5640 1 -- 172.21.15.17:0/772765467 wait complete. 2024-03-06T07:22:36.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.124+0000 7f4d08ab5640 1 Processor -- start 2024-03-06T07:22:36.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 -- start start 2024-03-06T07:22:36.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00052e0 0x7f4d040a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 0x7f4d041eb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04155220 con 0x7f4d041ee4b0 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c60e0 con 0x7f4cf00052e0 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d08ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040c2b30 con 0x7f4d04071880 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4cfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35496/0 (socket says 172.21.15.17:35496) 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4cfbfff640 1 -- 172.21.15.17:0/1827210370 learned_addr learned my addr 172.21.15.17:0/1827210370 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d0259c640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00052e0 0x7f4d040a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 0x7f4d041eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4cfbfff640 1 -- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 msgr2=0x7f4d041eb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.125+0000 7f4cfbfff640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 0x7f4d041eb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfbfff640 1 -- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00052e0 msgr2=0x7f4d040a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfbfff640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00052e0 0x7f4d040a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfbfff640 1 -- 172.21.15.17:0/1827210370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d041ebec0 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4d02d9d640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 0x7f4d041eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfbfff640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0072c0 tx=0x7f4cec007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec01a070 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec007e10 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00fb60 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d041eaf40 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.126+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d041eec20 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.127+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d041eb110 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.127+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cec01d070 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.128+0000 7f4cfaffd640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 0x7f4ce406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.128+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cec089cb0 con 0x7f4d041ee4b0 2024-03-06T07:22:36.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.128+0000 7f4d0259c640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 0x7f4ce406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.128+0000 7f4d0259c640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 0x7f4ce406d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.132+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d041ebec0 con 0x7f4d041ee4b0 2024-03-06T07:22:36.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.351+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4d0404f330 con 0x7f4d041ee4b0 2024-03-06T07:22:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.362+0000 7f4cfaffd640 1 -- 172.21.15.17:0/1827210370 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4cec058a30 con 0x7f4d041ee4b0 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 msgr2=0x7f4ce406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 0x7f4ce406d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 msgr2=0x7f4d041eb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0072c0 tx=0x7f4cec007780 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 shutdown_connections 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ce406b4d0 0x7f4ce406d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d04071880 0x7f4d041eb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d041ee4b0 0x7f4d041eb440 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 --2- 172.21.15.17:0/1827210370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cf00052e0 0x7f4d040a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 >> 172.21.15.17:0/1827210370 conn(0x7f4d040c4220 msgr2=0x7f4d041553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 shutdown_connections 2024-03-06T07:22:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.365+0000 7f4d08ab5640 1 -- 172.21.15.17:0/1827210370 wait complete. 2024-03-06T07:22:36.377 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:22:36.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:22:36.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.480+0000 7fc52bca4640 1 Processor -- start 2024-03-06T07:22:36.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.480+0000 7fc52bca4640 1 -- start start 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52bca4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 0x7fc5240c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52bca4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 0x7fc5240c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52bca4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc524072600 con 0x7fc5240c7720 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52bca4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240723b0 con 0x7fc5240c5c60 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52bca4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240728e0 con 0x7fc5240c5880 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52a241640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc5240c7720 0x7fc52406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59048/0 (socket says 172.21.15.17:59048) 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52a241640 1 -- 172.21.15.17:0/3699890392 learned_addr learned my addr 172.21.15.17:0/3699890392 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc529a40640 1 --2- 172.21.15.17:0/3699890392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 0x7fc5240c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52923f640 1 --2- 172.21.15.17:0/3699890392 >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 0x7fc5240c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc528a3e640 1 -- 172.21.15.17:0/3699890392 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093332018 0 0) 0x7fc524072600 con 0x7fc5240c7720 2024-03-06T07:22:36.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc528a3e640 1 -- 172.21.15.17:0/3699890392 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc510003520 con 0x7fc5240c7720 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc529a40640 1 -- 172.21.15.17:0/3699890392 >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 msgr2=0x7fc5240c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc529a40640 1 --2- 172.21.15.17:0/3699890392 >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 0x7fc5240c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc529a40640 1 -- 172.21.15.17:0/3699890392 >> v1:172.21.15.17:6789/0 conn(0x7fc5240c7720 legacy=0x7fc52406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc529a40640 1 -- 172.21.15.17:0/3699890392 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524153af0 con 0x7fc5240c5880 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.481+0000 7fc52923f640 1 --2- 172.21.15.17:0/3699890392 >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 0x7fc5240c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.482+0000 7fc529a40640 1 --2- 172.21.15.17:0/3699890392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 0x7fc5240c6ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5200097d0 tx=0x7fc52002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216568ba0be81952 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.482+0000 7fc528a3e640 1 -- 172.21.15.17:0/3699890392 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520039070 con 0x7fc5240c5880 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.482+0000 7fc528a3e640 1 -- 172.21.15.17:0/3699890392 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5200316f0 con 0x7fc5240c5880 2024-03-06T07:22:36.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.482+0000 7fc528a3e640 1 -- 172.21.15.17:0/3699890392 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc520031ac0 con 0x7fc5240c5880 2024-03-06T07:22:36.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 -- 172.21.15.17:0/3699890392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 msgr2=0x7fc5240c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 --2- 172.21.15.17:0/3699890392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 0x7fc5240c6ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5200097d0 tx=0x7fc52002b790 comp rx=0 tx=0).stop 2024-03-06T07:22:36.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 -- 172.21.15.17:0/3699890392 shutdown_connections 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 --2- 172.21.15.17:0/3699890392 >> v2:172.21.15.163:3300/0 conn(0x7fc5240c5c60 0x7fc5240c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 --2- 172.21.15.17:0/3699890392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5880 0x7fc5240c6ca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 -- 172.21.15.17:0/3699890392 >> 172.21.15.17:0/3699890392 conn(0x7fc5240aa630 msgr2=0x7fc5240aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 -- 172.21.15.17:0/3699890392 shutdown_connections 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.483+0000 7fc52bca4640 1 -- 172.21.15.17:0/3699890392 wait complete. 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.484+0000 7fc52bca4640 1 Processor -- start 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.484+0000 7fc52bca4640 1 -- start start 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 0x7fc5240bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5c60 0x7fc5240c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc524073190 con 0x7fc5240c7720 2024-03-06T07:22:36.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240c6780 con 0x7fc5240c5880 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc524073310 con 0x7fc5240c5c60 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc529a40640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 0x7fc5240bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35498/0 (socket says 172.21.15.17:35498) 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 -- 172.21.15.17:0/905828458 learned_addr learned my addr 172.21.15.17:0/905828458 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52923f640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5c60 0x7fc5240c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 -- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5c60 msgr2=0x7fc5240c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5c60 0x7fc5240c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 -- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 msgr2=0x7fc5240bd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 0x7fc5240bd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 -- 172.21.15.17:0/905828458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240bef70 con 0x7fc5240c7720 2024-03-06T07:22:36.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc529a40640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 0x7fc5240bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52a241640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ca00 tx=0x7fc51c00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c017070 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00ab70 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5241e8a20 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c012400 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.485+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5241e8f80 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.486+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52404f330 con 0x7fc5240c7720 2024-03-06T07:22:36.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.487+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc51c01a070 con 0x7fc5240c7720 2024-03-06T07:22:36.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.487+0000 7fc51a7fc640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 0x7fc50806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:36.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.487+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc51c089ed0 con 0x7fc5240c7720 2024-03-06T07:22:36.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.487+0000 7fc529a40640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 0x7fc50806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:36.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.488+0000 7fc529a40640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 0x7fc50806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc520036000 tx=0x7fc520008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:36.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.492+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5241e8a20 con 0x7fc5240c7720 2024-03-06T07:22:36.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:36.708+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc52414ffb0 con 0x7fc5240c7720 2024-03-06T07:22:37.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.367+0000 7fc51a7fc640 1 -- 172.21.15.17:0/905828458 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc51c058580 con 0x7fc5240c7720 2024-03-06T07:22:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 msgr2=0x7fc50806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 0x7fc50806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc520036000 tx=0x7fc520008030 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 msgr2=0x7fc524155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00ca00 tx=0x7fc51c00cec0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 shutdown_connections 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc50806b4d0 0x7fc50806d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5240c7720 0x7fc524155fc0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5240c5c60 0x7fc5240c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 --2- 172.21.15.17:0/905828458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5240c5880 0x7fc5240bd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 >> 172.21.15.17:0/905828458 conn(0x7fc5240aa630 msgr2=0x7fc5240c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 shutdown_connections 2024-03-06T07:22:37.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.370+0000 7fc52bca4640 1 -- 172.21.15.17:0/905828458 wait complete. 2024-03-06T07:22:37.385 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:22:37.385 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.491+0000 7fe37abf7640 1 Processor -- start 2024-03-06T07:22:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.491+0000 7fe37abf7640 1 -- start start 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe37abf7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe37abf7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe374155a40 0x7fe374155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe37abf7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe374156410 con 0x7fe3740c1aa0 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe37abf7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740c2680 con 0x7fe374155a40 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe37abf7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c28d0 con 0x7fe37414f790 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe374155a40 0x7fe374155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe373fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe3740c1aa0 0x7fe3740c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59058/0 (socket says 172.21.15.17:59058) 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe374155a40 0x7fe374155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35512/0 (socket says 172.21.15.17:35512) 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe373fff640 1 -- 172.21.15.17:0/149684262 learned_addr learned my addr 172.21.15.17:0/149684262 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe378993640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 -- 172.21.15.17:0/149684262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 msgr2=0x7fe3741529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 -- 172.21.15.17:0/149684262 >> v1:172.21.15.17:6789/0 conn(0x7fe3740c1aa0 legacy=0x7fe3740c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe3737fe640 1 -- 172.21.15.17:0/149684262 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364317841 0 0) 0x7fe374156410 con 0x7fe3740c1aa0 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.492+0000 7fe379194640 1 -- 172.21.15.17:0/149684262 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740c2bb0 con 0x7fe374155a40 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.493+0000 7fe379194640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe374155ed0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009d90 tx=0x7fe36c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9264913f4767dc90 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.493+0000 7fe3737fe640 1 -- 172.21.15.17:0/149684262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c039070 con 0x7fe374155a40 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.493+0000 7fe3737fe640 1 -- 172.21.15.17:0/149684262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c02cc50 con 0x7fe374155a40 2024-03-06T07:22:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.493+0000 7fe3737fe640 1 -- 172.21.15.17:0/149684262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c034440 con 0x7fe374155a40 2024-03-06T07:22:37.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 -- 172.21.15.17:0/149684262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 msgr2=0x7fe374155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe374155ed0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009d90 tx=0x7fe36c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 -- 172.21.15.17:0/149684262 shutdown_connections 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe374155ed0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 --2- 172.21.15.17:0/149684262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 -- 172.21.15.17:0/149684262 >> 172.21.15.17:0/149684262 conn(0x7fe3740c6710 msgr2=0x7fe3740c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 -- 172.21.15.17:0/149684262 shutdown_connections 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.494+0000 7fe37abf7640 1 -- 172.21.15.17:0/149684262 wait complete. 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.495+0000 7fe37abf7640 1 Processor -- start 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.495+0000 7fe37abf7640 1 -- start start 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3740c1aa0 0x7fe374151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c0d60 con 0x7fe374155a40 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe374155520 con 0x7fe3740c1aa0 2024-03-06T07:22:37.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe37abf7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740c5e30 con 0x7fe37414f790 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35520/0 (socket says 172.21.15.17:35520) 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 -- 172.21.15.17:0/511120775 learned_addr learned my addr 172.21.15.17:0/511120775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe378993640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3740c1aa0 0x7fe374151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe373fff640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 -- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 msgr2=0x7fe3741dd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741dd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 -- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3740c1aa0 msgr2=0x7fe374151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3740c1aa0 0x7fe374151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.496+0000 7fe379194640 1 -- 172.21.15.17:0/511120775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3741515a0 con 0x7fe374155a40 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe379194640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009d90 tx=0x7fe36c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c044020 con 0x7fe374155a40 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c03a040 con 0x7fe374155a40 2024-03-06T07:22:37.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c039050 con 0x7fe374155a40 2024-03-06T07:22:37.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3741db730 con 0x7fe374155a40 2024-03-06T07:22:37.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.497+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe374150580 con 0x7fe374155a40 2024-03-06T07:22:37.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.498+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374073510 con 0x7fe374155a40 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.498+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe36c003990 con 0x7fe374155a40 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.499+0000 7fe3717fa640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.499+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe36c0ac5e0 con 0x7fe374155a40 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.499+0000 7fe378993640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.499+0000 7fe378993640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 0x7fe36006d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe374070b70 tx=0x7fe364009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.503+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe36c04e060 con 0x7fe374155a40 2024-03-06T07:22:37.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.723+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe37404f330 con 0x7fe374155a40 2024-03-06T07:22:37.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.723+0000 7fe3717fa640 1 -- 172.21.15.17:0/511120775 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fe36c07ad10 con 0x7fe374155a40 2024-03-06T07:22:37.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:37.725 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:36.709974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"b","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/1213270757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1213270757},{"type":"v1","addr":"172.21.15.163:6836","nonce":1213270757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-03-06T07:22:37.725 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 56 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 msgr2=0x7fe36006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 0x7fe36006d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe374070b70 tx=0x7fe364009190 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 msgr2=0x7fe3741eb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009d90 tx=0x7fe36c004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 shutdown_connections 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe36006b4d0 0x7fe36006d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe374155a40 0x7fe3741eb3c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe37414f790 0x7fe3741dd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 --2- 172.21.15.17:0/511120775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3740c1aa0 0x7fe374151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 >> 172.21.15.17:0/511120775 conn(0x7fe3740c6710 msgr2=0x7fe37406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 shutdown_connections 2024-03-06T07:22:37.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.725+0000 7fe37abf7640 1 -- 172.21.15.17:0/511120775 wait complete. 2024-03-06T07:22:37.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:37.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.838+0000 7fb6c3839640 1 Processor -- start 2024-03-06T07:22:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.838+0000 7fb6c3839640 1 -- start start 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c3839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 0x7fb6bc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c3839640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc156180 con 0x7fb6bc0c5330 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c3839640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc159630 con 0x7fb6bc0c8a00 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c3839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc1587b0 con 0x7fb6bc152920 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c1dd6640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb6bc0c5330 0x7fb6bc15fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59066/0 (socket says 172.21.15.17:59066) 2024-03-06T07:22:37.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c1dd6640 1 -- 172.21.15.17:0/3317820533 learned_addr learned my addr 172.21.15.17:0/3317820533 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c15d5640 1 --2- 172.21.15.17:0/3317820533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 0x7fb6bc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827896851 0 0) 0x7fb6ac001490 con 0x7fb6bc0c5330 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a4003520 con 0x7fb6bc0c5330 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340033129 0 0) 0x7fb6bc159630 con 0x7fb6bc0c8a00 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6ac001490 con 0x7fb6bc0c8a00 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3317820533 >> v1:172.21.15.163:6789/0 conn(0x7fb6bc0c8a00 legacy=0x7fb6bc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3317820533 >> v1:172.21.15.17:6789/0 conn(0x7fb6bc0c5330 legacy=0x7fb6bc15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.839+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3317820533 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0c7760 con 0x7fb6bc152920 2024-03-06T07:22:37.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.840+0000 7fb6c15d5640 1 --2- 172.21.15.17:0/3317820533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 0x7fb6bc155c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40097d0 tx=0x7fb6b402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5d76f672e4e373 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.840+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4035020 con 0x7fb6bc152920 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.840+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4031800 con 0x7fb6bc152920 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.840+0000 7fb6bbfff640 1 -- 172.21.15.17:0/3317820533 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4031bd0 con 0x7fb6bc152920 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 -- 172.21.15.17:0/3317820533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 msgr2=0x7fb6bc155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3317820533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 0x7fb6bc155c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40097d0 tx=0x7fb6b402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 -- 172.21.15.17:0/3317820533 shutdown_connections 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3317820533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc152920 0x7fb6bc155c40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 -- 172.21.15.17:0/3317820533 >> 172.21.15.17:0/3317820533 conn(0x7fb6bc0ac560 msgr2=0x7fb6bc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 -- 172.21.15.17:0/3317820533 shutdown_connections 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.841+0000 7fb6c3839640 1 -- 172.21.15.17:0/3317820533 wait complete. 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.842+0000 7fb6c3839640 1 Processor -- start 2024-03-06T07:22:37.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.842+0000 7fb6c3839640 1 -- start start 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc0c8a00 0x7fb6bc1ed250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6bc152920 0x7fb6bc1ed790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc158a00 con 0x7fb6bc0c5330 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c7d20 con 0x7fb6bc152920 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc06d030 con 0x7fb6bc0c8a00 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35532/0 (socket says 172.21.15.17:35532) 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3769903274 learned_addr learned my addr 172.21.15.17:0/3769903274 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c1dd6640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6bc152920 0x7fb6bc1ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c0dd4640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc0c8a00 0x7fb6bc1ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc0c8a00 msgr2=0x7fb6bc1ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc0c8a00 0x7fb6bc1ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6bc152920 msgr2=0x7fb6bc1ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6bc152920 0x7fb6bc1ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 -- 172.21.15.17:0/3769903274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc155240 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c15d5640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40098f0 tx=0x7fb6b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4035040 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4039cf0 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc1eea50 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.843+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4039ea0 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.844+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1eed30 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.845+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc04f330 con 0x7fb6bc0c5330 2024-03-06T07:22:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.845+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6b4038020 con 0x7fb6bc0c5330 2024-03-06T07:22:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.846+0000 7fb6b9ffb640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 0x7fb6a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.846+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6b40ac280 con 0x7fb6bc0c5330 2024-03-06T07:22:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.846+0000 7fb6c0dd4640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 0x7fb6a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.846+0000 7fb6c0dd4640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 0x7fb6a806d9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0007fd0 tx=0x7fb6b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:37.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:37.850+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6b407a9b0 con 0x7fb6bc0c5330 2024-03-06T07:22:38.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.032+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6bc06cda0 con 0x7fb6bc0c5330 2024-03-06T07:22:38.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.033+0000 7fb6b9ffb640 1 -- 172.21.15.17:0/3769903274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16945 (secure 0 0 0) 0x7fb6b407ea50 con 0x7fb6bc0c5330 2024-03-06T07:22:38.035 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:38.035 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":75,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:36.359355+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:38.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 msgr2=0x7fb6a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 0x7fb6a806d9c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0007fd0 tx=0x7fb6b0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:38.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 msgr2=0x7fb6bc157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40098f0 tx=0x7fb6b40046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 shutdown_connections 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb6a806b500 0x7fb6a806d9c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb6bc152920 0x7fb6bc1ed790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb6bc0c8a00 0x7fb6bc1ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 --2- 172.21.15.17:0/3769903274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6bc0c5330 0x7fb6bc157de0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 >> 172.21.15.17:0/3769903274 conn(0x7fb6bc0ac560 msgr2=0x7fb6bc0ca660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 shutdown_connections 2024-03-06T07:22:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.035+0000 7fb6c3839640 1 -- 172.21.15.17:0/3769903274 wait complete. 2024-03-06T07:22:38.048 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:22:38.048 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:38.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 Processor -- start 2024-03-06T07:22:38.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 -- start start 2024-03-06T07:22:38.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd2c0c20b0 0x7fbd2c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2c0a9530 con 0x7fbd2c06afc0 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2c0a9810 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.148+0000 7fbd322f4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c0a9af0 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbd2c06afc0 0x7fbd2c153180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59080/0 (socket says 172.21.15.17:59080) 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd2c0c20b0 0x7fbd2c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2bfff640 1 -- 172.21.15.17:0/2523057339 learned_addr learned my addr 172.21.15.17:0/2523057339 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2b7fe640 1 -- 172.21.15.17:0/2523057339 >> v1:172.21.15.163:6789/0 conn(0x7fbd2c0c25c0 legacy=0x7fbd2c153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2b7fe640 1 -- 172.21.15.17:0/2523057339 >> v1:172.21.15.17:6789/0 conn(0x7fbd2c06afc0 legacy=0x7fbd2c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2affd640 1 -- 172.21.15.17:0/2523057339 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638889386 0 0) 0x7fbd2c0a9810 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.149+0000 7fbd2b7fe640 1 -- 172.21.15.17:0/2523057339 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c0a9dd0 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd2b7fe640 1 --2- 172.21.15.17:0/2523057339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c06b3a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4308d5cfe5f847d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd2affd640 1 -- 172.21.15.17:0/2523057339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14037070 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd2affd640 1 -- 172.21.15.17:0/2523057339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14030710 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd2affd640 1 -- 172.21.15.17:0/2523057339 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14030ae0 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 -- 172.21.15.17:0/2523057339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c20b0 msgr2=0x7fbd2c06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2523057339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c06b3a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fbd140097a0 tx=0x7fbd1402b680 comp rx=0 tx=0).stop 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 -- 172.21.15.17:0/2523057339 shutdown_connections 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2523057339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c06b3a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 -- 172.21.15.17:0/2523057339 >> 172.21.15.17:0/2523057339 conn(0x7fbd2c0c76b0 msgr2=0x7fbd2c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 -- 172.21.15.17:0/2523057339 shutdown_connections 2024-03-06T07:22:38.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.150+0000 7fbd322f4640 1 -- 172.21.15.17:0/2523057339 wait complete. 2024-03-06T07:22:38.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.151+0000 7fbd322f4640 1 Processor -- start 2024-03-06T07:22:38.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 -- start start 2024-03-06T07:22:38.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2c06afc0 0x7fbd2c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c150b00 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c150980 con 0x7fbd2c0c20b0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd322f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c0a7d50 con 0x7fbd2c06afc0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd30891640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd30891640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35552/0 (socket says 172.21.15.17:35552) 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd30891640 1 -- 172.21.15.17:0/2089853065 learned_addr learned my addr 172.21.15.17:0/2089853065 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd2bfff640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2c06afc0 0x7fbd2c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.152+0000 7fbd2b7fe640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 -- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2c06afc0 msgr2=0x7fbd2c14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2c06afc0 0x7fbd2c14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 -- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2c0c20b0 msgr2=0x7fbd2c145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 -- 172.21.15.17:0/2089853065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c1501d0 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd30891640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c780 tx=0x7fbd2400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24003b70 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd24003d10 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c14cc50 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24005270 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.153+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c1ecf60 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.154+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c04f330 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.154+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd24005410 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.155+0000 7fbd28ff9640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.155+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd2408a0a0 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.155+0000 7fbd2bfff640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.155+0000 7fbd2bfff640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000f60 tx=0x7fbd1c007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.159+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd24058740 con 0x7fbd2c0c25c0 2024-03-06T07:22:38.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.336+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd2c0c7490 con 0x7fbd0c06b500 2024-03-06T07:22:38.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.343+0000 7fbd28ff9640 1 -- 172.21.15.17:0/2089853065 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247856 (secure 0 0 0) 0x7fbd2c0c7490 con 0x7fbd0c06b500 2024-03-06T07:22:38.347 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:38.348 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:38.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.345+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 msgr2=0x7fbd0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.345+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c000f60 tx=0x7fbd1c007e20 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 msgr2=0x7fbd2c1ec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c780 tx=0x7fbd2400cb80 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 shutdown_connections 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd2c0c25c0 0x7fbd2c1ec100 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2c0c20b0 0x7fbd2c145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 --2- 172.21.15.17:0/2089853065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2c06afc0 0x7fbd2c14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 >> 172.21.15.17:0/2089853065 conn(0x7fbd2c0c76b0 msgr2=0x7fbd2c153180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 shutdown_connections 2024-03-06T07:22:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.346+0000 7fbd322f4640 1 -- 172.21.15.17:0/2089853065 wait complete. 2024-03-06T07:22:38.358 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-06T07:22:37.375626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240008,"kb_used_data":7188,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734712,"statfs":{"total":773094113280,"available":772848345088,"internally_reserved":0,"allocated":7360512,"data_stored":4583538,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035211"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369812+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:35.369812+0000","last_peered":"2024-03-06T07:22:35.369812+0000","last_clean":"2024-03-06T07:22:35.369812+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:35.369812+0000","last_undegraded":"2024-03-06T07:22:35.369812+0000","last_fullsized":"2024-03-06T07:22:35.369812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373335+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:35.373335+0000","last_peered":"2024-03-06T07:22:35.373335+0000","last_clean":"2024-03-06T07:22:35.373335+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:35.373335+0000","last_undegraded":"2024-03-06T07:22:35.373335+0000","last_fullsized":"2024-03-06T07:22:35.373335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:21.055978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294701+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:37.294701+0000","last_peered":"2024-03-06T07:22:37.294701+0000","last_clean":"2024-03-06T07:22:37.294701+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:37.294701+0000","last_undegraded":"2024-03-06T07:22:37.294701+0000","last_fullsized":"2024-03-06T07:22:37.294701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365453+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:36.365453+0000","last_peered":"2024-03-06T07:22:36.365453+0000","last_clean":"2024-03-06T07:22:36.365453+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:36.365453+0000","last_undegraded":"2024-03-06T07:22:36.365453+0000","last_fullsized":"2024-03-06T07:22:36.365453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373881+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:35.373881+0000","last_peered":"2024-03-06T07:22:35.373881+0000","last_clean":"2024-03-06T07:22:35.373881+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:35.373881+0000","last_undegraded":"2024-03-06T07:22:35.373881+0000","last_fullsized":"2024-03-06T07:22:35.373881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294583+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:37.294583+0000","last_peered":"2024-03-06T07:22:37.294583+0000","last_clean":"2024-03-06T07:22:37.294583+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:37.294583+0000","last_undegraded":"2024-03-06T07:22:37.294583+0000","last_fullsized":"2024-03-06T07:22:37.294583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366870+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:36.366870+0000","last_peered":"2024-03-06T07:22:36.366870+0000","last_clean":"2024-03-06T07:22:36.366870+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:36.366870+0000","last_undegraded":"2024-03-06T07:22:36.366870+0000","last_fullsized":"2024-03-06T07:22:36.366870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.374279+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:35.374279+0000","last_peered":"2024-03-06T07:22:35.374279+0000","last_clean":"2024-03-06T07:22:35.374279+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:35.374279+0000","last_undegraded":"2024-03-06T07:22:35.374279+0000","last_fullsized":"2024-03-06T07:22:35.374279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:38.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373392+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:35.373392+0000","last_peered":"2024-03-06T07:22:35.373392+0000","last_clean":"2024-03-06T07:22:35.373392+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:35.373392+0000","last_undegraded":"2024-03-06T07:22:35.373392+0000","last_fullsized":"2024-03-06T07:22:35.373392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:14.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365552+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:36.365552+0000","last_peered":"2024-03-06T07:22:36.365552+0000","last_clean":"2024-03-06T07:22:36.365552+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:36.365552+0000","last_undegraded":"2024-03-06T07:22:36.365552+0000","last_fullsized":"2024-03-06T07:22:36.365552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295121+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:37.295121+0000","last_peered":"2024-03-06T07:22:37.295121+0000","last_clean":"2024-03-06T07:22:37.295121+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:37.295121+0000","last_undegraded":"2024-03-06T07:22:37.295121+0000","last_fullsized":"2024-03-06T07:22:37.295121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366750+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:35.366750+0000","last_peered":"2024-03-06T07:22:35.366750+0000","last_clean":"2024-03-06T07:22:35.366750+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:35.366750+0000","last_undegraded":"2024-03-06T07:22:35.366750+0000","last_fullsized":"2024-03-06T07:22:35.366750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365181+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:36.365181+0000","last_peered":"2024-03-06T07:22:36.365181+0000","last_clean":"2024-03-06T07:22:36.365181+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:36.365181+0000","last_undegraded":"2024-03-06T07:22:36.365181+0000","last_fullsized":"2024-03-06T07:22:36.365181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366477+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:36.366477+0000","last_peered":"2024-03-06T07:22:36.366477+0000","last_clean":"2024-03-06T07:22:36.366477+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:36.366477+0000","last_undegraded":"2024-03-06T07:22:36.366477+0000","last_fullsized":"2024-03-06T07:22:36.366477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294614+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:37.294614+0000","last_peered":"2024-03-06T07:22:37.294614+0000","last_clean":"2024-03-06T07:22:37.294614+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:37.294614+0000","last_undegraded":"2024-03-06T07:22:37.294614+0000","last_fullsized":"2024-03-06T07:22:37.294614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366253+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:36.366253+0000","last_peered":"2024-03-06T07:22:36.366253+0000","last_clean":"2024-03-06T07:22:36.366253+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:36.366253+0000","last_undegraded":"2024-03-06T07:22:36.366253+0000","last_fullsized":"2024-03-06T07:22:36.366253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376691+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:35.376691+0000","last_peered":"2024-03-06T07:22:35.376691+0000","last_clean":"2024-03-06T07:22:35.376691+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:35.376691+0000","last_undegraded":"2024-03-06T07:22:35.376691+0000","last_fullsized":"2024-03-06T07:22:35.376691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:05.191375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295009+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:37.295009+0000","last_peered":"2024-03-06T07:22:37.295009+0000","last_clean":"2024-03-06T07:22:37.295009+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:37.295009+0000","last_undegraded":"2024-03-06T07:22:37.295009+0000","last_fullsized":"2024-03-06T07:22:37.295009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369741+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:35.369741+0000","last_peered":"2024-03-06T07:22:35.369741+0000","last_clean":"2024-03-06T07:22:35.369741+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:35.369741+0000","last_undegraded":"2024-03-06T07:22:35.369741+0000","last_fullsized":"2024-03-06T07:22:35.369741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.360317+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:35.360317+0000","last_peered":"2024-03-06T07:22:35.360317+0000","last_clean":"2024-03-06T07:22:35.360317+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:35.360317+0000","last_undegraded":"2024-03-06T07:22:35.360317+0000","last_fullsized":"2024-03-06T07:22:35.360317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295198+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:37.295198+0000","last_peered":"2024-03-06T07:22:37.295198+0000","last_clean":"2024-03-06T07:22:37.295198+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:37.295198+0000","last_undegraded":"2024-03-06T07:22:37.295198+0000","last_fullsized":"2024-03-06T07:22:37.295198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366415+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:36.366415+0000","last_peered":"2024-03-06T07:22:36.366415+0000","last_clean":"2024-03-06T07:22:36.366415+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:36.366415+0000","last_undegraded":"2024-03-06T07:22:36.366415+0000","last_fullsized":"2024-03-06T07:22:36.366415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367050+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:36.367050+0000","last_peered":"2024-03-06T07:22:36.367050+0000","last_clean":"2024-03-06T07:22:36.367050+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:36.367050+0000","last_undegraded":"2024-03-06T07:22:36.367050+0000","last_fullsized":"2024-03-06T07:22:36.367050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373794+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:35.373794+0000","last_peered":"2024-03-06T07:22:35.373794+0000","last_clean":"2024-03-06T07:22:35.373794+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:35.373794+0000","last_undegraded":"2024-03-06T07:22:35.373794+0000","last_fullsized":"2024-03-06T07:22:35.373794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367160+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:36.367160+0000","last_peered":"2024-03-06T07:22:36.367160+0000","last_clean":"2024-03-06T07:22:36.367160+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:36.367160+0000","last_undegraded":"2024-03-06T07:22:36.367160+0000","last_fullsized":"2024-03-06T07:22:36.367160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372297+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:35.372297+0000","last_peered":"2024-03-06T07:22:35.372297+0000","last_clean":"2024-03-06T07:22:35.372297+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:35.372297+0000","last_undegraded":"2024-03-06T07:22:35.372297+0000","last_fullsized":"2024-03-06T07:22:35.372297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367496+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:36.367496+0000","last_peered":"2024-03-06T07:22:36.367496+0000","last_clean":"2024-03-06T07:22:36.367496+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:36.367496+0000","last_undegraded":"2024-03-06T07:22:36.367496+0000","last_fullsized":"2024-03-06T07:22:36.367496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376711+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:35.376711+0000","last_peered":"2024-03-06T07:22:35.376711+0000","last_clean":"2024-03-06T07:22:35.376711+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:35.376711+0000","last_undegraded":"2024-03-06T07:22:35.376711+0000","last_fullsized":"2024-03-06T07:22:35.376711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294626+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:37.294626+0000","last_peered":"2024-03-06T07:22:37.294626+0000","last_clean":"2024-03-06T07:22:37.294626+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:37.294626+0000","last_undegraded":"2024-03-06T07:22:37.294626+0000","last_fullsized":"2024-03-06T07:22:37.294626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372836+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:35.372836+0000","last_peered":"2024-03-06T07:22:35.372836+0000","last_clean":"2024-03-06T07:22:35.372836+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:35.372836+0000","last_undegraded":"2024-03-06T07:22:35.372836+0000","last_fullsized":"2024-03-06T07:22:35.372836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:52.932700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366830+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:35.366830+0000","last_peered":"2024-03-06T07:22:35.366830+0000","last_clean":"2024-03-06T07:22:35.366830+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:35.366830+0000","last_undegraded":"2024-03-06T07:22:35.366830+0000","last_fullsized":"2024-03-06T07:22:35.366830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365569+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:36.365569+0000","last_peered":"2024-03-06T07:22:36.365569+0000","last_clean":"2024-03-06T07:22:36.365569+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:36.365569+0000","last_undegraded":"2024-03-06T07:22:36.365569+0000","last_fullsized":"2024-03-06T07:22:36.365569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295075+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:37.295075+0000","last_peered":"2024-03-06T07:22:37.295075+0000","last_clean":"2024-03-06T07:22:37.295075+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:37.295075+0000","last_undegraded":"2024-03-06T07:22:37.295075+0000","last_fullsized":"2024-03-06T07:22:37.295075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370865+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:35.370865+0000","last_peered":"2024-03-06T07:22:35.370865+0000","last_clean":"2024-03-06T07:22:35.370865+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:35.370865+0000","last_undegraded":"2024-03-06T07:22:35.370865+0000","last_fullsized":"2024-03-06T07:22:35.370865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.379447+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:35.379447+0000","last_peered":"2024-03-06T07:22:35.379447+0000","last_clean":"2024-03-06T07:22:35.379447+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:35.379447+0000","last_undegraded":"2024-03-06T07:22:35.379447+0000","last_fullsized":"2024-03-06T07:22:35.379447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372879+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:35.372879+0000","last_peered":"2024-03-06T07:22:35.372879+0000","last_clean":"2024-03-06T07:22:35.372879+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:35.372879+0000","last_undegraded":"2024-03-06T07:22:35.372879+0000","last_fullsized":"2024-03-06T07:22:35.372879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:54.478837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376739+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:35.376739+0000","last_peered":"2024-03-06T07:22:35.376739+0000","last_clean":"2024-03-06T07:22:35.376739+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:35.376739+0000","last_undegraded":"2024-03-06T07:22:35.376739+0000","last_fullsized":"2024-03-06T07:22:35.376739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372010+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:35.372010+0000","last_peered":"2024-03-06T07:22:35.372010+0000","last_clean":"2024-03-06T07:22:35.372010+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:35.372010+0000","last_undegraded":"2024-03-06T07:22:35.372010+0000","last_fullsized":"2024-03-06T07:22:35.372010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.364306+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:35.364306+0000","last_peered":"2024-03-06T07:22:35.364306+0000","last_clean":"2024-03-06T07:22:35.364306+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:35.364306+0000","last_undegraded":"2024-03-06T07:22:35.364306+0000","last_fullsized":"2024-03-06T07:22:35.364306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:46.770166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294650+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:37.294650+0000","last_peered":"2024-03-06T07:22:37.294650+0000","last_clean":"2024-03-06T07:22:37.294650+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:37.294650+0000","last_undegraded":"2024-03-06T07:22:37.294650+0000","last_fullsized":"2024-03-06T07:22:37.294650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372602+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:35.372602+0000","last_peered":"2024-03-06T07:22:35.372602+0000","last_clean":"2024-03-06T07:22:35.372602+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:35.372602+0000","last_undegraded":"2024-03-06T07:22:35.372602+0000","last_fullsized":"2024-03-06T07:22:35.372602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.364939+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:36.364939+0000","last_peered":"2024-03-06T07:22:36.364939+0000","last_clean":"2024-03-06T07:22:36.364939+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:36.364939+0000","last_undegraded":"2024-03-06T07:22:36.364939+0000","last_fullsized":"2024-03-06T07:22:36.364939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294660+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:37.294660+0000","last_peered":"2024-03-06T07:22:37.294660+0000","last_clean":"2024-03-06T07:22:37.294660+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:37.294660+0000","last_undegraded":"2024-03-06T07:22:37.294660+0000","last_fullsized":"2024-03-06T07:22:37.294660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294702+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:37.294702+0000","last_peered":"2024-03-06T07:22:37.294702+0000","last_clean":"2024-03-06T07:22:37.294702+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:37.294702+0000","last_undegraded":"2024-03-06T07:22:37.294702+0000","last_fullsized":"2024-03-06T07:22:37.294702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295156+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:37.295156+0000","last_peered":"2024-03-06T07:22:37.295156+0000","last_clean":"2024-03-06T07:22:37.295156+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:37.295156+0000","last_undegraded":"2024-03-06T07:22:37.295156+0000","last_fullsized":"2024-03-06T07:22:37.295156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365086+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:36.365086+0000","last_peered":"2024-03-06T07:22:36.365086+0000","last_clean":"2024-03-06T07:22:36.365086+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:36.365086+0000","last_undegraded":"2024-03-06T07:22:36.365086+0000","last_fullsized":"2024-03-06T07:22:36.365086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351435+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:35.351435+0000","last_peered":"2024-03-06T07:22:35.351435+0000","last_clean":"2024-03-06T07:22:35.351435+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:35.351435+0000","last_undegraded":"2024-03-06T07:22:35.351435+0000","last_fullsized":"2024-03-06T07:22:35.351435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365194+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:36.365194+0000","last_peered":"2024-03-06T07:22:36.365194+0000","last_clean":"2024-03-06T07:22:36.365194+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:36.365194+0000","last_undegraded":"2024-03-06T07:22:36.365194+0000","last_fullsized":"2024-03-06T07:22:36.365194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373686+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:35.373686+0000","last_peered":"2024-03-06T07:22:35.373686+0000","last_clean":"2024-03-06T07:22:35.373686+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:35.373686+0000","last_undegraded":"2024-03-06T07:22:35.373686+0000","last_fullsized":"2024-03-06T07:22:35.373686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366256+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:36.366256+0000","last_peered":"2024-03-06T07:22:36.366256+0000","last_clean":"2024-03-06T07:22:36.366256+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:36.366256+0000","last_undegraded":"2024-03-06T07:22:36.366256+0000","last_fullsized":"2024-03-06T07:22:36.366256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371730+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:35.371730+0000","last_peered":"2024-03-06T07:22:35.371730+0000","last_clean":"2024-03-06T07:22:35.371730+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:35.371730+0000","last_undegraded":"2024-03-06T07:22:35.371730+0000","last_fullsized":"2024-03-06T07:22:35.371730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:35.366022+0000","last_peered":"2024-03-06T07:22:35.366022+0000","last_clean":"2024-03-06T07:22:35.366022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:35.366022+0000","last_undegraded":"2024-03-06T07:22:35.366022+0000","last_fullsized":"2024-03-06T07:22:35.366022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:02.842916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366562+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:36.366562+0000","last_peered":"2024-03-06T07:22:36.366562+0000","last_clean":"2024-03-06T07:22:36.366562+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:36.366562+0000","last_undegraded":"2024-03-06T07:22:36.366562+0000","last_fullsized":"2024-03-06T07:22:36.366562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370130+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:35.370130+0000","last_peered":"2024-03-06T07:22:35.370130+0000","last_clean":"2024-03-06T07:22:35.370130+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:35.370130+0000","last_undegraded":"2024-03-06T07:22:35.370130+0000","last_fullsized":"2024-03-06T07:22:35.370130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:16.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.352766+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:35.352766+0000","last_peered":"2024-03-06T07:22:35.352766+0000","last_clean":"2024-03-06T07:22:35.352766+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:35.352766+0000","last_undegraded":"2024-03-06T07:22:35.352766+0000","last_fullsized":"2024-03-06T07:22:35.352766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373867+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:35.373867+0000","last_peered":"2024-03-06T07:22:35.373867+0000","last_clean":"2024-03-06T07:22:35.373867+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:35.373867+0000","last_undegraded":"2024-03-06T07:22:35.373867+0000","last_fullsized":"2024-03-06T07:22:35.373867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294742+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:37.294742+0000","last_peered":"2024-03-06T07:22:37.294742+0000","last_clean":"2024-03-06T07:22:37.294742+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:37.294742+0000","last_undegraded":"2024-03-06T07:22:37.294742+0000","last_fullsized":"2024-03-06T07:22:37.294742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369774+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:35.369774+0000","last_peered":"2024-03-06T07:22:35.369774+0000","last_clean":"2024-03-06T07:22:35.369774+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:35.369774+0000","last_undegraded":"2024-03-06T07:22:35.369774+0000","last_fullsized":"2024-03-06T07:22:35.369774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.353050+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:35.353050+0000","last_peered":"2024-03-06T07:22:35.353050+0000","last_clean":"2024-03-06T07:22:35.353050+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:35.353050+0000","last_undegraded":"2024-03-06T07:22:35.353050+0000","last_fullsized":"2024-03-06T07:22:35.353050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371951+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:35.371951+0000","last_peered":"2024-03-06T07:22:35.371951+0000","last_clean":"2024-03-06T07:22:35.371951+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:35.371951+0000","last_undegraded":"2024-03-06T07:22:35.371951+0000","last_fullsized":"2024-03-06T07:22:35.371951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351080+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:35.351080+0000","last_peered":"2024-03-06T07:22:35.351080+0000","last_clean":"2024-03-06T07:22:35.351080+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:35.351080+0000","last_undegraded":"2024-03-06T07:22:35.351080+0000","last_fullsized":"2024-03-06T07:22:35.351080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365701+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:36.365701+0000","last_peered":"2024-03-06T07:22:36.365701+0000","last_clean":"2024-03-06T07:22:36.365701+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:36.365701+0000","last_undegraded":"2024-03-06T07:22:36.365701+0000","last_fullsized":"2024-03-06T07:22:36.365701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":262,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-06T07:22:30.808301+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:30.808301+0000","last_peered":"2024-03-06T07:22:30.808301+0000","last_clean":"2024-03-06T07:22:30.808301+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:30.808301+0000","last_undegraded":"2024-03-06T07:22:30.808301+0000","last_fullsized":"2024-03-06T07:22:30.808301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":44,"acting":44,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":431150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":429572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":937984,"data_stored":437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":436812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":430280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":429650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:38.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:38.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.499+0000 7f0799925640 1 Processor -- start 2024-03-06T07:22:38.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.499+0000 7f0799925640 1 -- start start 2024-03-06T07:22:38.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0799925640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f079406da40 0x7f0794155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0799925640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07940c7380 0x7f07940c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0799925640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0794156050 con 0x7f07940c6780 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0799925640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079406de20 con 0x7f07940c7380 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0799925640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0794072630 con 0x7f079406da40 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07937fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07940c7380 0x7f07940c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f078bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07940c6780 0x7f0794158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59094/0 (socket says 172.21.15.17:59094) 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f078bfff640 1 -- 172.21.15.17:0/3783688611 learned_addr learned my addr 172.21.15.17:0/3783688611 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f0792ffd640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f079406da40 0x7f0794155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07937fe640 1 -- 172.21.15.17:0/3783688611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f079406da40 msgr2=0x7f0794155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07937fe640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f079406da40 0x7f0794155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07937fe640 1 -- 172.21.15.17:0/3783688611 >> v1:172.21.15.17:6789/0 conn(0x7f07940c6780 legacy=0x7f0794158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07937fe640 1 -- 172.21.15.17:0/3783688611 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0794072910 con 0x7f07940c7380 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.500+0000 7f07927fc640 1 -- 172.21.15.17:0/3783688611 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688212176 0 0) 0x7f077c001110 con 0x7f07940c6780 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.501+0000 7f07937fe640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07940c7380 0x7f07940c3280 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f078c009d90 tx=0x7f078c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9077d6a5177024 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.501+0000 7f07927fc640 1 -- 172.21.15.17:0/3783688611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c039070 con 0x7f07940c7380 2024-03-06T07:22:38.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.501+0000 7f07927fc640 1 -- 172.21.15.17:0/3783688611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c02cbc0 con 0x7f07940c7380 2024-03-06T07:22:38.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.501+0000 7f07927fc640 1 -- 172.21.15.17:0/3783688611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c034440 con 0x7f07940c7380 2024-03-06T07:22:38.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 -- 172.21.15.17:0/3783688611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07940c7380 msgr2=0x7f07940c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07940c7380 0x7f07940c3280 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f078c009d90 tx=0x7f078c02be10 comp rx=0 tx=0).stop 2024-03-06T07:22:38.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 -- 172.21.15.17:0/3783688611 shutdown_connections 2024-03-06T07:22:38.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07940c7380 0x7f07940c3280 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 --2- 172.21.15.17:0/3783688611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f079406da40 0x7f0794155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 -- 172.21.15.17:0/3783688611 >> 172.21.15.17:0/3783688611 conn(0x7f07940aa630 msgr2=0x7f07940aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 -- 172.21.15.17:0/3783688611 shutdown_connections 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.502+0000 7f0799925640 1 -- 172.21.15.17:0/3783688611 wait complete. 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 Processor -- start 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 -- start start 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07940c6780 0x7f0794157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 0x7f0794157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07941594a0 con 0x7f079406da40 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0794158890 con 0x7f07940c6780 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.503+0000 7f0799925640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07940c37c0 con 0x7f07940c7380 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35568/0 (socket says 172.21.15.17:35568) 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 -- 172.21.15.17:0/3626400232 learned_addr learned my addr 172.21.15.17:0/3626400232 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f078bfff640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07940c6780 0x7f0794157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f07937fe640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 0x7f0794157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 -- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 msgr2=0x7f0794157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 0x7f0794157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 -- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07940c6780 msgr2=0x7f0794157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07940c6780 0x7f0794157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 -- 172.21.15.17:0/3626400232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07941eba10 con 0x7f079406da40 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f07937fe640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 0x7f0794157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0792ffd640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9e0 tx=0x7f078400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784017070 con 0x7f079406da40 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400c9d0 con 0x7f079406da40 2024-03-06T07:22:38.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0794152be0 con 0x7f079406da40 2024-03-06T07:22:38.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400cdc0 con 0x7f079406da40 2024-03-06T07:22:38.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.504+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07940ab640 con 0x7f079406da40 2024-03-06T07:22:38.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.505+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079404f330 con 0x7f079406da40 2024-03-06T07:22:38.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.506+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f078401a070 con 0x7f079406da40 2024-03-06T07:22:38.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.506+0000 7f0790ff9640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.506+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f078408a690 con 0x7f079406da40 2024-03-06T07:22:38.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.506+0000 7f078bfff640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.507+0000 7f078bfff640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 0x7f077406d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f07940c64c0 tx=0x7f077c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.510+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0784058dc0 con 0x7f079406da40 2024-03-06T07:22:38.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.687+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07941eb7c0 con 0x7f077406b4d0 2024-03-06T07:22:38.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.693+0000 7f0790ff9640 1 -- 172.21.15.17:0/3626400232 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247856 (secure 0 0 0) 0x7f07941eb7c0 con 0x7f077406b4d0 2024-03-06T07:22:38.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:38.698 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 msgr2=0x7f077406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 0x7f077406d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f07940c64c0 tx=0x7f077c007b50 comp rx=0 tx=0).stop 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 msgr2=0x7f079414b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f078400e9e0 tx=0x7f078400eea0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 shutdown_connections 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07940c7380 0x7f0794157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07940c6780 0x7f0794157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 --2- 172.21.15.17:0/3626400232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f079406da40 0x7f079414b0a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 >> 172.21.15.17:0/3626400232 conn(0x7f07940aa630 msgr2=0x7f07940aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 shutdown_connections 2024-03-06T07:22:38.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.696+0000 7f0799925640 1 -- 172.21.15.17:0/3626400232 wait complete. 2024-03-06T07:22:38.709 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-06T07:22:37.375626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240008,"kb_used_data":7188,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734712,"statfs":{"total":773094113280,"available":772848345088,"internally_reserved":0,"allocated":7360512,"data_stored":4583538,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035211"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369812+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:35.369812+0000","last_peered":"2024-03-06T07:22:35.369812+0000","last_clean":"2024-03-06T07:22:35.369812+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:35.369812+0000","last_undegraded":"2024-03-06T07:22:35.369812+0000","last_fullsized":"2024-03-06T07:22:35.369812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373335+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:35.373335+0000","last_peered":"2024-03-06T07:22:35.373335+0000","last_clean":"2024-03-06T07:22:35.373335+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:35.373335+0000","last_undegraded":"2024-03-06T07:22:35.373335+0000","last_fullsized":"2024-03-06T07:22:35.373335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:21.055978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294701+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:37.294701+0000","last_peered":"2024-03-06T07:22:37.294701+0000","last_clean":"2024-03-06T07:22:37.294701+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:37.294701+0000","last_undegraded":"2024-03-06T07:22:37.294701+0000","last_fullsized":"2024-03-06T07:22:37.294701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365453+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:36.365453+0000","last_peered":"2024-03-06T07:22:36.365453+0000","last_clean":"2024-03-06T07:22:36.365453+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:36.365453+0000","last_undegraded":"2024-03-06T07:22:36.365453+0000","last_fullsized":"2024-03-06T07:22:36.365453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373881+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:35.373881+0000","last_peered":"2024-03-06T07:22:35.373881+0000","last_clean":"2024-03-06T07:22:35.373881+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:35.373881+0000","last_undegraded":"2024-03-06T07:22:35.373881+0000","last_fullsized":"2024-03-06T07:22:35.373881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294583+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:37.294583+0000","last_peered":"2024-03-06T07:22:37.294583+0000","last_clean":"2024-03-06T07:22:37.294583+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:37.294583+0000","last_undegraded":"2024-03-06T07:22:37.294583+0000","last_fullsized":"2024-03-06T07:22:37.294583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366870+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:36.366870+0000","last_peered":"2024-03-06T07:22:36.366870+0000","last_clean":"2024-03-06T07:22:36.366870+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:36.366870+0000","last_undegraded":"2024-03-06T07:22:36.366870+0000","last_fullsized":"2024-03-06T07:22:36.366870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.374279+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:35.374279+0000","last_peered":"2024-03-06T07:22:35.374279+0000","last_clean":"2024-03-06T07:22:35.374279+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:35.374279+0000","last_undegraded":"2024-03-06T07:22:35.374279+0000","last_fullsized":"2024-03-06T07:22:35.374279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:38.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373392+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:35.373392+0000","last_peered":"2024-03-06T07:22:35.373392+0000","last_clean":"2024-03-06T07:22:35.373392+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:35.373392+0000","last_undegraded":"2024-03-06T07:22:35.373392+0000","last_fullsized":"2024-03-06T07:22:35.373392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:14.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365552+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:36.365552+0000","last_peered":"2024-03-06T07:22:36.365552+0000","last_clean":"2024-03-06T07:22:36.365552+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:36.365552+0000","last_undegraded":"2024-03-06T07:22:36.365552+0000","last_fullsized":"2024-03-06T07:22:36.365552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295121+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:37.295121+0000","last_peered":"2024-03-06T07:22:37.295121+0000","last_clean":"2024-03-06T07:22:37.295121+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:37.295121+0000","last_undegraded":"2024-03-06T07:22:37.295121+0000","last_fullsized":"2024-03-06T07:22:37.295121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366750+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:35.366750+0000","last_peered":"2024-03-06T07:22:35.366750+0000","last_clean":"2024-03-06T07:22:35.366750+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:35.366750+0000","last_undegraded":"2024-03-06T07:22:35.366750+0000","last_fullsized":"2024-03-06T07:22:35.366750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365181+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:36.365181+0000","last_peered":"2024-03-06T07:22:36.365181+0000","last_clean":"2024-03-06T07:22:36.365181+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:36.365181+0000","last_undegraded":"2024-03-06T07:22:36.365181+0000","last_fullsized":"2024-03-06T07:22:36.365181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366477+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:36.366477+0000","last_peered":"2024-03-06T07:22:36.366477+0000","last_clean":"2024-03-06T07:22:36.366477+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:36.366477+0000","last_undegraded":"2024-03-06T07:22:36.366477+0000","last_fullsized":"2024-03-06T07:22:36.366477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294614+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:37.294614+0000","last_peered":"2024-03-06T07:22:37.294614+0000","last_clean":"2024-03-06T07:22:37.294614+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:37.294614+0000","last_undegraded":"2024-03-06T07:22:37.294614+0000","last_fullsized":"2024-03-06T07:22:37.294614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366253+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:36.366253+0000","last_peered":"2024-03-06T07:22:36.366253+0000","last_clean":"2024-03-06T07:22:36.366253+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:36.366253+0000","last_undegraded":"2024-03-06T07:22:36.366253+0000","last_fullsized":"2024-03-06T07:22:36.366253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376691+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:35.376691+0000","last_peered":"2024-03-06T07:22:35.376691+0000","last_clean":"2024-03-06T07:22:35.376691+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:35.376691+0000","last_undegraded":"2024-03-06T07:22:35.376691+0000","last_fullsized":"2024-03-06T07:22:35.376691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:05.191375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295009+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:37.295009+0000","last_peered":"2024-03-06T07:22:37.295009+0000","last_clean":"2024-03-06T07:22:37.295009+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:37.295009+0000","last_undegraded":"2024-03-06T07:22:37.295009+0000","last_fullsized":"2024-03-06T07:22:37.295009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369741+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:35.369741+0000","last_peered":"2024-03-06T07:22:35.369741+0000","last_clean":"2024-03-06T07:22:35.369741+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:35.369741+0000","last_undegraded":"2024-03-06T07:22:35.369741+0000","last_fullsized":"2024-03-06T07:22:35.369741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.360317+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:35.360317+0000","last_peered":"2024-03-06T07:22:35.360317+0000","last_clean":"2024-03-06T07:22:35.360317+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:35.360317+0000","last_undegraded":"2024-03-06T07:22:35.360317+0000","last_fullsized":"2024-03-06T07:22:35.360317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295198+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:37.295198+0000","last_peered":"2024-03-06T07:22:37.295198+0000","last_clean":"2024-03-06T07:22:37.295198+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:37.295198+0000","last_undegraded":"2024-03-06T07:22:37.295198+0000","last_fullsized":"2024-03-06T07:22:37.295198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366415+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:36.366415+0000","last_peered":"2024-03-06T07:22:36.366415+0000","last_clean":"2024-03-06T07:22:36.366415+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:36.366415+0000","last_undegraded":"2024-03-06T07:22:36.366415+0000","last_fullsized":"2024-03-06T07:22:36.366415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367050+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:36.367050+0000","last_peered":"2024-03-06T07:22:36.367050+0000","last_clean":"2024-03-06T07:22:36.367050+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:36.367050+0000","last_undegraded":"2024-03-06T07:22:36.367050+0000","last_fullsized":"2024-03-06T07:22:36.367050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373794+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:35.373794+0000","last_peered":"2024-03-06T07:22:35.373794+0000","last_clean":"2024-03-06T07:22:35.373794+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:35.373794+0000","last_undegraded":"2024-03-06T07:22:35.373794+0000","last_fullsized":"2024-03-06T07:22:35.373794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367160+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:36.367160+0000","last_peered":"2024-03-06T07:22:36.367160+0000","last_clean":"2024-03-06T07:22:36.367160+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:36.367160+0000","last_undegraded":"2024-03-06T07:22:36.367160+0000","last_fullsized":"2024-03-06T07:22:36.367160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372297+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:35.372297+0000","last_peered":"2024-03-06T07:22:35.372297+0000","last_clean":"2024-03-06T07:22:35.372297+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:35.372297+0000","last_undegraded":"2024-03-06T07:22:35.372297+0000","last_fullsized":"2024-03-06T07:22:35.372297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367496+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:36.367496+0000","last_peered":"2024-03-06T07:22:36.367496+0000","last_clean":"2024-03-06T07:22:36.367496+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:36.367496+0000","last_undegraded":"2024-03-06T07:22:36.367496+0000","last_fullsized":"2024-03-06T07:22:36.367496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376711+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:35.376711+0000","last_peered":"2024-03-06T07:22:35.376711+0000","last_clean":"2024-03-06T07:22:35.376711+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:35.376711+0000","last_undegraded":"2024-03-06T07:22:35.376711+0000","last_fullsized":"2024-03-06T07:22:35.376711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294626+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:37.294626+0000","last_peered":"2024-03-06T07:22:37.294626+0000","last_clean":"2024-03-06T07:22:37.294626+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:37.294626+0000","last_undegraded":"2024-03-06T07:22:37.294626+0000","last_fullsized":"2024-03-06T07:22:37.294626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372836+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:35.372836+0000","last_peered":"2024-03-06T07:22:35.372836+0000","last_clean":"2024-03-06T07:22:35.372836+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:35.372836+0000","last_undegraded":"2024-03-06T07:22:35.372836+0000","last_fullsized":"2024-03-06T07:22:35.372836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:52.932700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366830+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:35.366830+0000","last_peered":"2024-03-06T07:22:35.366830+0000","last_clean":"2024-03-06T07:22:35.366830+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:35.366830+0000","last_undegraded":"2024-03-06T07:22:35.366830+0000","last_fullsized":"2024-03-06T07:22:35.366830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365569+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:36.365569+0000","last_peered":"2024-03-06T07:22:36.365569+0000","last_clean":"2024-03-06T07:22:36.365569+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:36.365569+0000","last_undegraded":"2024-03-06T07:22:36.365569+0000","last_fullsized":"2024-03-06T07:22:36.365569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295075+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:37.295075+0000","last_peered":"2024-03-06T07:22:37.295075+0000","last_clean":"2024-03-06T07:22:37.295075+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:37.295075+0000","last_undegraded":"2024-03-06T07:22:37.295075+0000","last_fullsized":"2024-03-06T07:22:37.295075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370865+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:35.370865+0000","last_peered":"2024-03-06T07:22:35.370865+0000","last_clean":"2024-03-06T07:22:35.370865+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:35.370865+0000","last_undegraded":"2024-03-06T07:22:35.370865+0000","last_fullsized":"2024-03-06T07:22:35.370865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.379447+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:35.379447+0000","last_peered":"2024-03-06T07:22:35.379447+0000","last_clean":"2024-03-06T07:22:35.379447+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:35.379447+0000","last_undegraded":"2024-03-06T07:22:35.379447+0000","last_fullsized":"2024-03-06T07:22:35.379447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372879+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:35.372879+0000","last_peered":"2024-03-06T07:22:35.372879+0000","last_clean":"2024-03-06T07:22:35.372879+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:35.372879+0000","last_undegraded":"2024-03-06T07:22:35.372879+0000","last_fullsized":"2024-03-06T07:22:35.372879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:54.478837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376739+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:35.376739+0000","last_peered":"2024-03-06T07:22:35.376739+0000","last_clean":"2024-03-06T07:22:35.376739+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:35.376739+0000","last_undegraded":"2024-03-06T07:22:35.376739+0000","last_fullsized":"2024-03-06T07:22:35.376739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372010+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:35.372010+0000","last_peered":"2024-03-06T07:22:35.372010+0000","last_clean":"2024-03-06T07:22:35.372010+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:35.372010+0000","last_undegraded":"2024-03-06T07:22:35.372010+0000","last_fullsized":"2024-03-06T07:22:35.372010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.364306+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:35.364306+0000","last_peered":"2024-03-06T07:22:35.364306+0000","last_clean":"2024-03-06T07:22:35.364306+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:35.364306+0000","last_undegraded":"2024-03-06T07:22:35.364306+0000","last_fullsized":"2024-03-06T07:22:35.364306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:46.770166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294650+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:37.294650+0000","last_peered":"2024-03-06T07:22:37.294650+0000","last_clean":"2024-03-06T07:22:37.294650+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:37.294650+0000","last_undegraded":"2024-03-06T07:22:37.294650+0000","last_fullsized":"2024-03-06T07:22:37.294650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372602+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:35.372602+0000","last_peered":"2024-03-06T07:22:35.372602+0000","last_clean":"2024-03-06T07:22:35.372602+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:35.372602+0000","last_undegraded":"2024-03-06T07:22:35.372602+0000","last_fullsized":"2024-03-06T07:22:35.372602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.364939+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:36.364939+0000","last_peered":"2024-03-06T07:22:36.364939+0000","last_clean":"2024-03-06T07:22:36.364939+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:36.364939+0000","last_undegraded":"2024-03-06T07:22:36.364939+0000","last_fullsized":"2024-03-06T07:22:36.364939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294660+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:37.294660+0000","last_peered":"2024-03-06T07:22:37.294660+0000","last_clean":"2024-03-06T07:22:37.294660+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:37.294660+0000","last_undegraded":"2024-03-06T07:22:37.294660+0000","last_fullsized":"2024-03-06T07:22:37.294660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294702+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:37.294702+0000","last_peered":"2024-03-06T07:22:37.294702+0000","last_clean":"2024-03-06T07:22:37.294702+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:37.294702+0000","last_undegraded":"2024-03-06T07:22:37.294702+0000","last_fullsized":"2024-03-06T07:22:37.294702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295156+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:37.295156+0000","last_peered":"2024-03-06T07:22:37.295156+0000","last_clean":"2024-03-06T07:22:37.295156+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:37.295156+0000","last_undegraded":"2024-03-06T07:22:37.295156+0000","last_fullsized":"2024-03-06T07:22:37.295156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365086+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:36.365086+0000","last_peered":"2024-03-06T07:22:36.365086+0000","last_clean":"2024-03-06T07:22:36.365086+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:36.365086+0000","last_undegraded":"2024-03-06T07:22:36.365086+0000","last_fullsized":"2024-03-06T07:22:36.365086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351435+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:35.351435+0000","last_peered":"2024-03-06T07:22:35.351435+0000","last_clean":"2024-03-06T07:22:35.351435+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:35.351435+0000","last_undegraded":"2024-03-06T07:22:35.351435+0000","last_fullsized":"2024-03-06T07:22:35.351435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365194+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:36.365194+0000","last_peered":"2024-03-06T07:22:36.365194+0000","last_clean":"2024-03-06T07:22:36.365194+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:36.365194+0000","last_undegraded":"2024-03-06T07:22:36.365194+0000","last_fullsized":"2024-03-06T07:22:36.365194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373686+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:35.373686+0000","last_peered":"2024-03-06T07:22:35.373686+0000","last_clean":"2024-03-06T07:22:35.373686+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:35.373686+0000","last_undegraded":"2024-03-06T07:22:35.373686+0000","last_fullsized":"2024-03-06T07:22:35.373686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366256+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:36.366256+0000","last_peered":"2024-03-06T07:22:36.366256+0000","last_clean":"2024-03-06T07:22:36.366256+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:36.366256+0000","last_undegraded":"2024-03-06T07:22:36.366256+0000","last_fullsized":"2024-03-06T07:22:36.366256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371730+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:35.371730+0000","last_peered":"2024-03-06T07:22:35.371730+0000","last_clean":"2024-03-06T07:22:35.371730+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:35.371730+0000","last_undegraded":"2024-03-06T07:22:35.371730+0000","last_fullsized":"2024-03-06T07:22:35.371730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:35.366022+0000","last_peered":"2024-03-06T07:22:35.366022+0000","last_clean":"2024-03-06T07:22:35.366022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:35.366022+0000","last_undegraded":"2024-03-06T07:22:35.366022+0000","last_fullsized":"2024-03-06T07:22:35.366022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:02.842916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366562+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:36.366562+0000","last_peered":"2024-03-06T07:22:36.366562+0000","last_clean":"2024-03-06T07:22:36.366562+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:36.366562+0000","last_undegraded":"2024-03-06T07:22:36.366562+0000","last_fullsized":"2024-03-06T07:22:36.366562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370130+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:35.370130+0000","last_peered":"2024-03-06T07:22:35.370130+0000","last_clean":"2024-03-06T07:22:35.370130+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:35.370130+0000","last_undegraded":"2024-03-06T07:22:35.370130+0000","last_fullsized":"2024-03-06T07:22:35.370130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:16.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.352766+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:35.352766+0000","last_peered":"2024-03-06T07:22:35.352766+0000","last_clean":"2024-03-06T07:22:35.352766+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:35.352766+0000","last_undegraded":"2024-03-06T07:22:35.352766+0000","last_fullsized":"2024-03-06T07:22:35.352766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373867+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:35.373867+0000","last_peered":"2024-03-06T07:22:35.373867+0000","last_clean":"2024-03-06T07:22:35.373867+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:35.373867+0000","last_undegraded":"2024-03-06T07:22:35.373867+0000","last_fullsized":"2024-03-06T07:22:35.373867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294742+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:37.294742+0000","last_peered":"2024-03-06T07:22:37.294742+0000","last_clean":"2024-03-06T07:22:37.294742+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:37.294742+0000","last_undegraded":"2024-03-06T07:22:37.294742+0000","last_fullsized":"2024-03-06T07:22:37.294742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369774+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:35.369774+0000","last_peered":"2024-03-06T07:22:35.369774+0000","last_clean":"2024-03-06T07:22:35.369774+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:35.369774+0000","last_undegraded":"2024-03-06T07:22:35.369774+0000","last_fullsized":"2024-03-06T07:22:35.369774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.353050+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:35.353050+0000","last_peered":"2024-03-06T07:22:35.353050+0000","last_clean":"2024-03-06T07:22:35.353050+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:35.353050+0000","last_undegraded":"2024-03-06T07:22:35.353050+0000","last_fullsized":"2024-03-06T07:22:35.353050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371951+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:35.371951+0000","last_peered":"2024-03-06T07:22:35.371951+0000","last_clean":"2024-03-06T07:22:35.371951+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:35.371951+0000","last_undegraded":"2024-03-06T07:22:35.371951+0000","last_fullsized":"2024-03-06T07:22:35.371951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351080+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:35.351080+0000","last_peered":"2024-03-06T07:22:35.351080+0000","last_clean":"2024-03-06T07:22:35.351080+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:35.351080+0000","last_undegraded":"2024-03-06T07:22:35.351080+0000","last_fullsized":"2024-03-06T07:22:35.351080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365701+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:36.365701+0000","last_peered":"2024-03-06T07:22:36.365701+0000","last_clean":"2024-03-06T07:22:36.365701+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:36.365701+0000","last_undegraded":"2024-03-06T07:22:36.365701+0000","last_fullsized":"2024-03-06T07:22:36.365701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":262,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-06T07:22:30.808301+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:30.808301+0000","last_peered":"2024-03-06T07:22:30.808301+0000","last_clean":"2024-03-06T07:22:30.808301+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:30.808301+0000","last_undegraded":"2024-03-06T07:22:30.808301+0000","last_fullsized":"2024-03-06T07:22:30.808301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":44,"acting":44,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":431150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":429572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":937984,"data_stored":437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":436812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":430280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":429650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:38.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.849+0000 7efdbbaaf640 1 Processor -- start 2024-03-06T07:22:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.849+0000 7efdbbaaf640 1 -- start start 2024-03-06T07:22:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdbbaaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdb4072e70 0x7efdb40bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdbbaaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb4073250 0x7efdb40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdbbaaf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb406c8e0 con 0x7efdb40c3950 2024-03-06T07:22:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdbbaaf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb406ce80 con 0x7efdb4072e70 2024-03-06T07:22:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdbbaaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40c1b00 con 0x7efdb4073250 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdb984b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdb4072e70 0x7efdb40bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdba04c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efdb40c3950 0x7efdb41589b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59102/0 (socket says 172.21.15.17:59102) 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdb984b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efdb4072e70 0x7efdb40bd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35580/0 (socket says 172.21.15.17:35580) 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdba04c640 1 -- 172.21.15.17:0/4070150908 learned_addr learned my addr 172.21.15.17:0/4070150908 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdb904a640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb4073250 0x7efdb40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdb984b640 1 -- 172.21.15.17:0/4070150908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb4073250 msgr2=0x7efdb40c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.850+0000 7efdb984b640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb4073250 0x7efdb40c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb984b640 1 -- 172.21.15.17:0/4070150908 >> v1:172.21.15.17:6789/0 conn(0x7efdb40c3950 legacy=0x7efdb41589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb8849640 1 -- 172.21.15.17:0/4070150908 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347751976 0 0) 0x7efdb406c8e0 con 0x7efdb40c3950 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb984b640 1 -- 172.21.15.17:0/4070150908 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb406cba0 con 0x7efdb4072e70 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb984b640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4072e70 0x7efdb40bd510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097a0 tx=0x7efdac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c5ab9719f5902a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb8849640 1 -- 172.21.15.17:0/4070150908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac0315a0 con 0x7efdb4072e70 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb8849640 1 -- 172.21.15.17:0/4070150908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac037070 con 0x7efdb4072e70 2024-03-06T07:22:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.851+0000 7efdb8849640 1 -- 172.21.15.17:0/4070150908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac031960 con 0x7efdb4072e70 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 -- 172.21.15.17:0/4070150908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4072e70 msgr2=0x7efdb40bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4072e70 0x7efdb40bd510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097a0 tx=0x7efdac02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 -- 172.21.15.17:0/4070150908 shutdown_connections 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb4073250 0x7efdb40c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/4070150908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4072e70 0x7efdb40bd510 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 -- 172.21.15.17:0/4070150908 >> 172.21.15.17:0/4070150908 conn(0x7efdb406b1a0 msgr2=0x7efdb406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 -- 172.21.15.17:0/4070150908 shutdown_connections 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.852+0000 7efdbbaaf640 1 -- 172.21.15.17:0/4070150908 wait complete. 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.853+0000 7efdbbaaf640 1 Processor -- start 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.853+0000 7efdbbaaf640 1 -- start start 2024-03-06T07:22:38.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdb4072e70 0x7efdb41ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb40c3950 0x7efdb41ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40c4b80 con 0x7efdb4073250 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40c4a00 con 0x7efdb4072e70 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40bda50 con 0x7efdb40c3950 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35584/0 (socket says 172.21.15.17:35584) 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 -- 172.21.15.17:0/2527383107 learned_addr learned my addr 172.21.15.17:0/2527383107 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb984b640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdb4072e70 0x7efdb41ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdba04c640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb40c3950 0x7efdb41ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 -- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb40c3950 msgr2=0x7efdb41ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb40c3950 0x7efdb41ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 -- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdb4072e70 msgr2=0x7efdb41ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:38.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdb4072e70 0x7efdb41ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 -- 172.21.15.17:0/2527383107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb41eb930 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdb904a640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efda400e8a0 tx=0x7efda400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4017070 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda400c940 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb41ec390 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.854+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda400cd30 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.855+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40a83d0 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.855+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb404f330 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.856+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efda401a070 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.856+0000 7efdaa7fc640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 0x7efd9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.856+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efda408a150 con 0x7efdb4073250 2024-03-06T07:22:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.856+0000 7efdb984b640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 0x7efd9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:38.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.857+0000 7efdb984b640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 0x7efd9806d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:38.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:38.860+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efda40587f0 con 0x7efdb4073250 2024-03-06T07:22:38.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:38.881 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:22:39.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.038+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdb40a7ea0 con 0x7efd9806b500 2024-03-06T07:22:39.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.045+0000 7efdaa7fc640 1 -- 172.21.15.17:0/2527383107 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247856 (secure 0 0 0) 0x7efdb40a7ea0 con 0x7efd9806b500 2024-03-06T07:22:39.049 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:39.050 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 msgr2=0x7efd9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 0x7efd9806d9c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac008030 comp rx=0 tx=0).stop 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 msgr2=0x7efdb41eb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efda400e8a0 tx=0x7efda400ed60 comp rx=0 tx=0).stop 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 shutdown_connections 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd9806b500 0x7efd9806d9c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdb40c3950 0x7efdb41ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdb4073250 0x7efdb41eb3f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 --2- 172.21.15.17:0/2527383107 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdb4072e70 0x7efdb41ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 >> 172.21.15.17:0/2527383107 conn(0x7efdb406b1a0 msgr2=0x7efdb4150ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 shutdown_connections 2024-03-06T07:22:39.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.048+0000 7efdbbaaf640 1 -- 172.21.15.17:0/2527383107 wait complete. 2024-03-06T07:22:39.060 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-03-06T07:22:37.375626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":187,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":136,"acting":136,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240008,"kb_used_data":7188,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734712,"statfs":{"total":773094113280,"available":772848345088,"internally_reserved":0,"allocated":7360512,"data_stored":4583538,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035211"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369812+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:35.369812+0000","last_peered":"2024-03-06T07:22:35.369812+0000","last_clean":"2024-03-06T07:22:35.369812+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:35.369812+0000","last_undegraded":"2024-03-06T07:22:35.369812+0000","last_fullsized":"2024-03-06T07:22:35.369812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373335+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:35.373335+0000","last_peered":"2024-03-06T07:22:35.373335+0000","last_clean":"2024-03-06T07:22:35.373335+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:35.373335+0000","last_undegraded":"2024-03-06T07:22:35.373335+0000","last_fullsized":"2024-03-06T07:22:35.373335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:21.055978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294701+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:37.294701+0000","last_peered":"2024-03-06T07:22:37.294701+0000","last_clean":"2024-03-06T07:22:37.294701+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:37.294701+0000","last_undegraded":"2024-03-06T07:22:37.294701+0000","last_fullsized":"2024-03-06T07:22:37.294701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365453+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:36.365453+0000","last_peered":"2024-03-06T07:22:36.365453+0000","last_clean":"2024-03-06T07:22:36.365453+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:36.365453+0000","last_undegraded":"2024-03-06T07:22:36.365453+0000","last_fullsized":"2024-03-06T07:22:36.365453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373881+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:35.373881+0000","last_peered":"2024-03-06T07:22:35.373881+0000","last_clean":"2024-03-06T07:22:35.373881+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:35.373881+0000","last_undegraded":"2024-03-06T07:22:35.373881+0000","last_fullsized":"2024-03-06T07:22:35.373881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294583+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:37.294583+0000","last_peered":"2024-03-06T07:22:37.294583+0000","last_clean":"2024-03-06T07:22:37.294583+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:37.294583+0000","last_undegraded":"2024-03-06T07:22:37.294583+0000","last_fullsized":"2024-03-06T07:22:37.294583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366870+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:36.366870+0000","last_peered":"2024-03-06T07:22:36.366870+0000","last_clean":"2024-03-06T07:22:36.366870+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:36.366870+0000","last_undegraded":"2024-03-06T07:22:36.366870+0000","last_fullsized":"2024-03-06T07:22:36.366870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.374279+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:35.374279+0000","last_peered":"2024-03-06T07:22:35.374279+0000","last_clean":"2024-03-06T07:22:35.374279+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:35.374279+0000","last_undegraded":"2024-03-06T07:22:35.374279+0000","last_fullsized":"2024-03-06T07:22:35.374279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:38.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373392+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:35.373392+0000","last_peered":"2024-03-06T07:22:35.373392+0000","last_clean":"2024-03-06T07:22:35.373392+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:35.373392+0000","last_undegraded":"2024-03-06T07:22:35.373392+0000","last_fullsized":"2024-03-06T07:22:35.373392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:14.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365552+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:36.365552+0000","last_peered":"2024-03-06T07:22:36.365552+0000","last_clean":"2024-03-06T07:22:36.365552+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:36.365552+0000","last_undegraded":"2024-03-06T07:22:36.365552+0000","last_fullsized":"2024-03-06T07:22:36.365552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295121+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:37.295121+0000","last_peered":"2024-03-06T07:22:37.295121+0000","last_clean":"2024-03-06T07:22:37.295121+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:37.295121+0000","last_undegraded":"2024-03-06T07:22:37.295121+0000","last_fullsized":"2024-03-06T07:22:37.295121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366750+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:35.366750+0000","last_peered":"2024-03-06T07:22:35.366750+0000","last_clean":"2024-03-06T07:22:35.366750+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:35.366750+0000","last_undegraded":"2024-03-06T07:22:35.366750+0000","last_fullsized":"2024-03-06T07:22:35.366750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365181+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:36.365181+0000","last_peered":"2024-03-06T07:22:36.365181+0000","last_clean":"2024-03-06T07:22:36.365181+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:36.365181+0000","last_undegraded":"2024-03-06T07:22:36.365181+0000","last_fullsized":"2024-03-06T07:22:36.365181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366477+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:36.366477+0000","last_peered":"2024-03-06T07:22:36.366477+0000","last_clean":"2024-03-06T07:22:36.366477+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:36.366477+0000","last_undegraded":"2024-03-06T07:22:36.366477+0000","last_fullsized":"2024-03-06T07:22:36.366477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294614+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:37.294614+0000","last_peered":"2024-03-06T07:22:37.294614+0000","last_clean":"2024-03-06T07:22:37.294614+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:37.294614+0000","last_undegraded":"2024-03-06T07:22:37.294614+0000","last_fullsized":"2024-03-06T07:22:37.294614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366253+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:36.366253+0000","last_peered":"2024-03-06T07:22:36.366253+0000","last_clean":"2024-03-06T07:22:36.366253+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:36.366253+0000","last_undegraded":"2024-03-06T07:22:36.366253+0000","last_fullsized":"2024-03-06T07:22:36.366253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376691+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:35.376691+0000","last_peered":"2024-03-06T07:22:35.376691+0000","last_clean":"2024-03-06T07:22:35.376691+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:35.376691+0000","last_undegraded":"2024-03-06T07:22:35.376691+0000","last_fullsized":"2024-03-06T07:22:35.376691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:05.191375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295009+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:37.295009+0000","last_peered":"2024-03-06T07:22:37.295009+0000","last_clean":"2024-03-06T07:22:37.295009+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:37.295009+0000","last_undegraded":"2024-03-06T07:22:37.295009+0000","last_fullsized":"2024-03-06T07:22:37.295009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369741+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:35.369741+0000","last_peered":"2024-03-06T07:22:35.369741+0000","last_clean":"2024-03-06T07:22:35.369741+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:35.369741+0000","last_undegraded":"2024-03-06T07:22:35.369741+0000","last_fullsized":"2024-03-06T07:22:35.369741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.360317+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:35.360317+0000","last_peered":"2024-03-06T07:22:35.360317+0000","last_clean":"2024-03-06T07:22:35.360317+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:35.360317+0000","last_undegraded":"2024-03-06T07:22:35.360317+0000","last_fullsized":"2024-03-06T07:22:35.360317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295198+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:37.295198+0000","last_peered":"2024-03-06T07:22:37.295198+0000","last_clean":"2024-03-06T07:22:37.295198+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:37.295198+0000","last_undegraded":"2024-03-06T07:22:37.295198+0000","last_fullsized":"2024-03-06T07:22:37.295198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366415+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:36.366415+0000","last_peered":"2024-03-06T07:22:36.366415+0000","last_clean":"2024-03-06T07:22:36.366415+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:36.366415+0000","last_undegraded":"2024-03-06T07:22:36.366415+0000","last_fullsized":"2024-03-06T07:22:36.366415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367050+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:36.367050+0000","last_peered":"2024-03-06T07:22:36.367050+0000","last_clean":"2024-03-06T07:22:36.367050+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:36.367050+0000","last_undegraded":"2024-03-06T07:22:36.367050+0000","last_fullsized":"2024-03-06T07:22:36.367050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373794+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:35.373794+0000","last_peered":"2024-03-06T07:22:35.373794+0000","last_clean":"2024-03-06T07:22:35.373794+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:35.373794+0000","last_undegraded":"2024-03-06T07:22:35.373794+0000","last_fullsized":"2024-03-06T07:22:35.373794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367160+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:36.367160+0000","last_peered":"2024-03-06T07:22:36.367160+0000","last_clean":"2024-03-06T07:22:36.367160+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:36.367160+0000","last_undegraded":"2024-03-06T07:22:36.367160+0000","last_fullsized":"2024-03-06T07:22:36.367160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372297+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:35.372297+0000","last_peered":"2024-03-06T07:22:35.372297+0000","last_clean":"2024-03-06T07:22:35.372297+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:35.372297+0000","last_undegraded":"2024-03-06T07:22:35.372297+0000","last_fullsized":"2024-03-06T07:22:35.372297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367496+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:36.367496+0000","last_peered":"2024-03-06T07:22:36.367496+0000","last_clean":"2024-03-06T07:22:36.367496+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:36.367496+0000","last_undegraded":"2024-03-06T07:22:36.367496+0000","last_fullsized":"2024-03-06T07:22:36.367496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376711+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:35.376711+0000","last_peered":"2024-03-06T07:22:35.376711+0000","last_clean":"2024-03-06T07:22:35.376711+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:35.376711+0000","last_undegraded":"2024-03-06T07:22:35.376711+0000","last_fullsized":"2024-03-06T07:22:35.376711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294626+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:37.294626+0000","last_peered":"2024-03-06T07:22:37.294626+0000","last_clean":"2024-03-06T07:22:37.294626+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:37.294626+0000","last_undegraded":"2024-03-06T07:22:37.294626+0000","last_fullsized":"2024-03-06T07:22:37.294626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372836+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:35.372836+0000","last_peered":"2024-03-06T07:22:35.372836+0000","last_clean":"2024-03-06T07:22:35.372836+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:35.372836+0000","last_undegraded":"2024-03-06T07:22:35.372836+0000","last_fullsized":"2024-03-06T07:22:35.372836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:52.932700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366830+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:35.366830+0000","last_peered":"2024-03-06T07:22:35.366830+0000","last_clean":"2024-03-06T07:22:35.366830+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:35.366830+0000","last_undegraded":"2024-03-06T07:22:35.366830+0000","last_fullsized":"2024-03-06T07:22:35.366830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365569+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:36.365569+0000","last_peered":"2024-03-06T07:22:36.365569+0000","last_clean":"2024-03-06T07:22:36.365569+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:36.365569+0000","last_undegraded":"2024-03-06T07:22:36.365569+0000","last_fullsized":"2024-03-06T07:22:36.365569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295075+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:37.295075+0000","last_peered":"2024-03-06T07:22:37.295075+0000","last_clean":"2024-03-06T07:22:37.295075+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:37.295075+0000","last_undegraded":"2024-03-06T07:22:37.295075+0000","last_fullsized":"2024-03-06T07:22:37.295075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370865+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:35.370865+0000","last_peered":"2024-03-06T07:22:35.370865+0000","last_clean":"2024-03-06T07:22:35.370865+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:35.370865+0000","last_undegraded":"2024-03-06T07:22:35.370865+0000","last_fullsized":"2024-03-06T07:22:35.370865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.379447+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:35.379447+0000","last_peered":"2024-03-06T07:22:35.379447+0000","last_clean":"2024-03-06T07:22:35.379447+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:35.379447+0000","last_undegraded":"2024-03-06T07:22:35.379447+0000","last_fullsized":"2024-03-06T07:22:35.379447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372879+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:35.372879+0000","last_peered":"2024-03-06T07:22:35.372879+0000","last_clean":"2024-03-06T07:22:35.372879+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:35.372879+0000","last_undegraded":"2024-03-06T07:22:35.372879+0000","last_fullsized":"2024-03-06T07:22:35.372879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:54.478837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376739+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:35.376739+0000","last_peered":"2024-03-06T07:22:35.376739+0000","last_clean":"2024-03-06T07:22:35.376739+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:35.376739+0000","last_undegraded":"2024-03-06T07:22:35.376739+0000","last_fullsized":"2024-03-06T07:22:35.376739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372010+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:35.372010+0000","last_peered":"2024-03-06T07:22:35.372010+0000","last_clean":"2024-03-06T07:22:35.372010+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:35.372010+0000","last_undegraded":"2024-03-06T07:22:35.372010+0000","last_fullsized":"2024-03-06T07:22:35.372010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.364306+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:35.364306+0000","last_peered":"2024-03-06T07:22:35.364306+0000","last_clean":"2024-03-06T07:22:35.364306+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:35.364306+0000","last_undegraded":"2024-03-06T07:22:35.364306+0000","last_fullsized":"2024-03-06T07:22:35.364306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:46.770166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294650+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:37.294650+0000","last_peered":"2024-03-06T07:22:37.294650+0000","last_clean":"2024-03-06T07:22:37.294650+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:37.294650+0000","last_undegraded":"2024-03-06T07:22:37.294650+0000","last_fullsized":"2024-03-06T07:22:37.294650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372602+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:35.372602+0000","last_peered":"2024-03-06T07:22:35.372602+0000","last_clean":"2024-03-06T07:22:35.372602+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:35.372602+0000","last_undegraded":"2024-03-06T07:22:35.372602+0000","last_fullsized":"2024-03-06T07:22:35.372602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.364939+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:36.364939+0000","last_peered":"2024-03-06T07:22:36.364939+0000","last_clean":"2024-03-06T07:22:36.364939+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:36.364939+0000","last_undegraded":"2024-03-06T07:22:36.364939+0000","last_fullsized":"2024-03-06T07:22:36.364939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294660+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:37.294660+0000","last_peered":"2024-03-06T07:22:37.294660+0000","last_clean":"2024-03-06T07:22:37.294660+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:37.294660+0000","last_undegraded":"2024-03-06T07:22:37.294660+0000","last_fullsized":"2024-03-06T07:22:37.294660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294702+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:37.294702+0000","last_peered":"2024-03-06T07:22:37.294702+0000","last_clean":"2024-03-06T07:22:37.294702+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:37.294702+0000","last_undegraded":"2024-03-06T07:22:37.294702+0000","last_fullsized":"2024-03-06T07:22:37.294702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295156+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:37.295156+0000","last_peered":"2024-03-06T07:22:37.295156+0000","last_clean":"2024-03-06T07:22:37.295156+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:37.295156+0000","last_undegraded":"2024-03-06T07:22:37.295156+0000","last_fullsized":"2024-03-06T07:22:37.295156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365086+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:36.365086+0000","last_peered":"2024-03-06T07:22:36.365086+0000","last_clean":"2024-03-06T07:22:36.365086+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:36.365086+0000","last_undegraded":"2024-03-06T07:22:36.365086+0000","last_fullsized":"2024-03-06T07:22:36.365086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351435+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:35.351435+0000","last_peered":"2024-03-06T07:22:35.351435+0000","last_clean":"2024-03-06T07:22:35.351435+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:35.351435+0000","last_undegraded":"2024-03-06T07:22:35.351435+0000","last_fullsized":"2024-03-06T07:22:35.351435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365194+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:36.365194+0000","last_peered":"2024-03-06T07:22:36.365194+0000","last_clean":"2024-03-06T07:22:36.365194+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:36.365194+0000","last_undegraded":"2024-03-06T07:22:36.365194+0000","last_fullsized":"2024-03-06T07:22:36.365194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373686+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:35.373686+0000","last_peered":"2024-03-06T07:22:35.373686+0000","last_clean":"2024-03-06T07:22:35.373686+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:35.373686+0000","last_undegraded":"2024-03-06T07:22:35.373686+0000","last_fullsized":"2024-03-06T07:22:35.373686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366256+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:36.366256+0000","last_peered":"2024-03-06T07:22:36.366256+0000","last_clean":"2024-03-06T07:22:36.366256+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:36.366256+0000","last_undegraded":"2024-03-06T07:22:36.366256+0000","last_fullsized":"2024-03-06T07:22:36.366256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371730+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:35.371730+0000","last_peered":"2024-03-06T07:22:35.371730+0000","last_clean":"2024-03-06T07:22:35.371730+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:35.371730+0000","last_undegraded":"2024-03-06T07:22:35.371730+0000","last_fullsized":"2024-03-06T07:22:35.371730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:35.366022+0000","last_peered":"2024-03-06T07:22:35.366022+0000","last_clean":"2024-03-06T07:22:35.366022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:35.366022+0000","last_undegraded":"2024-03-06T07:22:35.366022+0000","last_fullsized":"2024-03-06T07:22:35.366022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:02.842916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366562+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:36.366562+0000","last_peered":"2024-03-06T07:22:36.366562+0000","last_clean":"2024-03-06T07:22:36.366562+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:36.366562+0000","last_undegraded":"2024-03-06T07:22:36.366562+0000","last_fullsized":"2024-03-06T07:22:36.366562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:33.333775+0000","last_change":"2024-03-06T07:22:33.333775+0000","last_active":"2024-03-06T07:22:33.333775+0000","last_peered":"2024-03-06T07:22:33.333775+0000","last_clean":"2024-03-06T07:22:33.333775+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:33.333775+0000","last_undegraded":"2024-03-06T07:22:33.333775+0000","last_fullsized":"2024-03-06T07:22:33.333775+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370130+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:35.370130+0000","last_peered":"2024-03-06T07:22:35.370130+0000","last_clean":"2024-03-06T07:22:35.370130+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:35.370130+0000","last_undegraded":"2024-03-06T07:22:35.370130+0000","last_fullsized":"2024-03-06T07:22:35.370130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:16.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.352766+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:35.352766+0000","last_peered":"2024-03-06T07:22:35.352766+0000","last_clean":"2024-03-06T07:22:35.352766+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:35.352766+0000","last_undegraded":"2024-03-06T07:22:35.352766+0000","last_fullsized":"2024-03-06T07:22:35.352766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373867+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:35.373867+0000","last_peered":"2024-03-06T07:22:35.373867+0000","last_clean":"2024-03-06T07:22:35.373867+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:35.373867+0000","last_undegraded":"2024-03-06T07:22:35.373867+0000","last_fullsized":"2024-03-06T07:22:35.373867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294742+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:37.294742+0000","last_peered":"2024-03-06T07:22:37.294742+0000","last_clean":"2024-03-06T07:22:37.294742+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:37.294742+0000","last_undegraded":"2024-03-06T07:22:37.294742+0000","last_fullsized":"2024-03-06T07:22:37.294742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369774+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:35.369774+0000","last_peered":"2024-03-06T07:22:35.369774+0000","last_clean":"2024-03-06T07:22:35.369774+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:35.369774+0000","last_undegraded":"2024-03-06T07:22:35.369774+0000","last_fullsized":"2024-03-06T07:22:35.369774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.353050+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:35.353050+0000","last_peered":"2024-03-06T07:22:35.353050+0000","last_clean":"2024-03-06T07:22:35.353050+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:35.353050+0000","last_undegraded":"2024-03-06T07:22:35.353050+0000","last_fullsized":"2024-03-06T07:22:35.353050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371951+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:35.371951+0000","last_peered":"2024-03-06T07:22:35.371951+0000","last_clean":"2024-03-06T07:22:35.371951+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:35.371951+0000","last_undegraded":"2024-03-06T07:22:35.371951+0000","last_fullsized":"2024-03-06T07:22:35.371951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351080+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:35.351080+0000","last_peered":"2024-03-06T07:22:35.351080+0000","last_clean":"2024-03-06T07:22:35.351080+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:35.351080+0000","last_undegraded":"2024-03-06T07:22:35.351080+0000","last_fullsized":"2024-03-06T07:22:35.351080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-06T07:22:34.338640+0000","last_change":"2024-03-06T07:22:34.338640+0000","last_active":"2024-03-06T07:22:34.338640+0000","last_peered":"2024-03-06T07:22:34.338640+0000","last_clean":"2024-03-06T07:22:34.338640+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:22:34.338640+0000","last_undegraded":"2024-03-06T07:22:34.338640+0000","last_fullsized":"2024-03-06T07:22:34.338640+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365701+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:36.365701+0000","last_peered":"2024-03-06T07:22:36.365701+0000","last_clean":"2024-03-06T07:22:36.365701+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:36.365701+0000","last_undegraded":"2024-03-06T07:22:36.365701+0000","last_fullsized":"2024-03-06T07:22:36.365701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":262,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-03-06T07:22:30.808301+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:30.808301+0000","last_peered":"2024-03-06T07:22:30.808301+0000","last_clean":"2024-03-06T07:22:30.808301+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:30.808301+0000","last_undegraded":"2024-03-06T07:22:30.808301+0000","last_fullsized":"2024-03-06T07:22:30.808301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":44,"acting":44,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":431150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":429572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":937984,"data_stored":437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":436812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":430280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":429650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":994064,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2024-03-06T07:22:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2024-03-06T07:22:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-03-06T07:22:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-03-06T07:22:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-03-06T07:22:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:33.333775+0000', 'last_change': '2024-03-06T07:22:33.333775+0000', 'last_active': '2024-03-06T07:22:33.333775+0000', 'last_peered': '2024-03-06T07:22:33.333775+0000', 'last_clean': '2024-03-06T07:22:33.333775+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:33.333775+0000', 'last_undegraded': '2024-03-06T07:22:33.333775+0000', 'last_fullsized': '2024-03-06T07:22:33.333775+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:33.333775+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-03-06T07:22:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-06T07:22:34.338640+0000', 'last_change': '2024-03-06T07:22:34.338640+0000', 'last_active': '2024-03-06T07:22:34.338640+0000', 'last_peered': '2024-03-06T07:22:34.338640+0000', 'last_clean': '2024-03-06T07:22:34.338640+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:22:34.338640+0000', 'last_undegraded': '2024-03-06T07:22:34.338640+0000', 'last_fullsized': '2024-03-06T07:22:34.338640+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'last_clean_scrub_stamp': '2024-03-06T07:22:34.338640+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-06T07:22:39.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:22:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 Processor -- start 2024-03-06T07:22:39.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 -- start start 2024-03-06T07:22:39.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7840072f70 0x7f7840154570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78400bbf80 0x7f7840154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78400bdc30 con 0x7f78400c6560 2024-03-06T07:22:39.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400bd690 con 0x7f78400bbf80 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.200+0000 7f7846ebb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400bd950 con 0x7f7840072f70 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78400bbf80 0x7f7840154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78400bbf80 0x7f7840154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35590/0 (socket says 172.21.15.17:35590) 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 -- 172.21.15.17:0/4132922816 learned_addr learned my addr 172.21.15.17:0/4132922816 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7844c57640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7840072f70 0x7f7840154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 -- 172.21.15.17:0/4132922816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7840072f70 msgr2=0x7f7840154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7840072f70 0x7f7840154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 -- 172.21.15.17:0/4132922816 >> v1:172.21.15.163:6789/0 conn(0x7f78400c6560 legacy=0x7f7840156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.201+0000 7f7845458640 1 -- 172.21.15.17:0/4132922816 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400c5540 con 0x7f78400bbf80 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7845458640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f7840154ab0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7828009870 tx=0x7f782802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be7d84b54b4df41 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f783f7fe640 1 -- 172.21.15.17:0/4132922816 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828035020 con 0x7f78400bbf80 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f783f7fe640 1 -- 172.21.15.17:0/4132922816 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7828031640 con 0x7f78400bbf80 2024-03-06T07:22:39.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f783f7fe640 1 -- 172.21.15.17:0/4132922816 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782802d040 con 0x7f78400bbf80 2024-03-06T07:22:39.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 -- 172.21.15.17:0/4132922816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 msgr2=0x7f7840154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f7840154ab0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7828009870 tx=0x7f782802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 -- 172.21.15.17:0/4132922816 shutdown_connections 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f7840154ab0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 --2- 172.21.15.17:0/4132922816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7840072f70 0x7f7840154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 -- 172.21.15.17:0/4132922816 >> 172.21.15.17:0/4132922816 conn(0x7f784006b1a0 msgr2=0x7f784006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.202+0000 7f7846ebb640 1 -- 172.21.15.17:0/4132922816 shutdown_connections 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.203+0000 7f7846ebb640 1 -- 172.21.15.17:0/4132922816 wait complete. 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 Processor -- start 2024-03-06T07:22:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 -- start start 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7840072f70 0x7f7840154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78400c6560 0x7f78401525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840073350 con 0x7f78400bbf80 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400bba60 con 0x7f7840072f70 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.204+0000 7f7846ebb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400734d0 con 0x7f78400c6560 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35602/0 (socket says 172.21.15.17:35602) 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 -- 172.21.15.17:0/1410476276 learned_addr learned my addr 172.21.15.17:0/1410476276 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f7844c57640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7840072f70 0x7f7840154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f7845458640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78400c6560 0x7f78401525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 -- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78400c6560 msgr2=0x7f78401525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78400c6560 0x7f78401525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 -- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7840072f70 msgr2=0x7f7840154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7840072f70 0x7f7840154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 -- 172.21.15.17:0/1410476276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840156b60 con 0x7f78400bbf80 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783ffff640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f783000c400 tx=0x7f783000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300052d0 con 0x7f78400bbf80 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830005490 con 0x7f78400bbf80 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830007260 con 0x7f78400bbf80 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78400a56a0 con 0x7f78400bbf80 2024-03-06T07:22:39.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.205+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78400a5c60 con 0x7f78400bbf80 2024-03-06T07:22:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.206+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784004f330 con 0x7f78400bbf80 2024-03-06T07:22:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.207+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7830007400 con 0x7f78400bbf80 2024-03-06T07:22:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.207+0000 7f783d7fa640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 0x7f782006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.207+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7830089840 con 0x7f78400bbf80 2024-03-06T07:22:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.207+0000 7f7844c57640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 0x7f782006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.208+0000 7f7844c57640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 0x7f782006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7838011340 tx=0x7f783800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.212+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7830057ed0 con 0x7f78400bbf80 2024-03-06T07:22:39.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.466+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f78400c6260 con 0x7f78400bbf80 2024-03-06T07:22:39.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.466+0000 7f783d7fa640 1 -- 172.21.15.17:0/1410476276 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1508 (secure 0 0 0) 0x7f783005bf70 con 0x7f78400bbf80 2024-03-06T07:22:39.468 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:39.468 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":172,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":75,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709597,"num_in_osds":8,"osd_in_since":1709709589,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":68},{"state_name":"unknown","count":29}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592756,"bytes_used":245768192,"bytes_avail":772848345088,"bytes_total":773094113280,"unknown_pgs_ratio":0.29896906018257141,"write_bytes_sec":3298,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":56,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6387}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-03-06T07:22:17.371743+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:22:39.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 msgr2=0x7f782006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 0x7f782006d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7838011340 tx=0x7f783800b040 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 msgr2=0x7f78401e8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f783000c400 tx=0x7f783000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 shutdown_connections 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f782006b4d0 0x7f782006d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78400c6560 0x7f78401525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78400bbf80 0x7f78401e8c20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 --2- 172.21.15.17:0/1410476276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7840072f70 0x7f7840154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 >> 172.21.15.17:0/1410476276 conn(0x7f784006b1a0 msgr2=0x7f78400720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 shutdown_connections 2024-03-06T07:22:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.469+0000 7f7846ebb640 1 -- 172.21.15.17:0/1410476276 wait complete. 2024-03-06T07:22:39.481 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:22:39.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.581+0000 7fa5491f7640 1 Processor -- start 2024-03-06T07:22:39.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.581+0000 7fa5491f7640 1 -- start start 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa5491f7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa544158db0 0x7fa5440c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa5491f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5440c88b0 0x7fa54415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa5491f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa544155110 con 0x7fa5440c5330 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa5491f7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5441559e0 con 0x7fa544158db0 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa5491f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5440c8f40 con 0x7fa5440c88b0 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa544158db0 0x7fa5440c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa544158db0 0x7fa5440c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35604/0 (socket says 172.21.15.17:35604) 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 -- 172.21.15.17:0/3447705349 learned_addr learned my addr 172.21.15.17:0/3447705349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54359e640 1 --2- 172.21.15.17:0/3447705349 >> v2:172.21.15.163:3300/0 conn(0x7fa5440c88b0 0x7fa54415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 -- 172.21.15.17:0/3447705349 >> v2:172.21.15.163:3300/0 conn(0x7fa5440c88b0 msgr2=0x7fa54415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 --2- 172.21.15.17:0/3447705349 >> v2:172.21.15.163:3300/0 conn(0x7fa5440c88b0 0x7fa54415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 -- 172.21.15.17:0/3447705349 >> v1:172.21.15.163:6789/0 conn(0x7fa5440c5330 legacy=0x7fa5441586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.582+0000 7fa54259c640 1 -- 172.21.15.17:0/3447705349 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544155390 con 0x7fa544158db0 2024-03-06T07:22:39.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.583+0000 7fa54259c640 1 --2- 172.21.15.17:0/3447705349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544158db0 0x7fa5440c5710 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa5440ca870 tx=0x7fa53802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d21918fd9328c39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.583+0000 7fa541d9b640 1 -- 172.21.15.17:0/3447705349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538035020 con 0x7fa544158db0 2024-03-06T07:22:39.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.583+0000 7fa541d9b640 1 -- 172.21.15.17:0/3447705349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa538031870 con 0x7fa544158db0 2024-03-06T07:22:39.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.583+0000 7fa541d9b640 1 -- 172.21.15.17:0/3447705349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538031c40 con 0x7fa544158db0 2024-03-06T07:22:39.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 -- 172.21.15.17:0/3447705349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544158db0 msgr2=0x7fa5440c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 --2- 172.21.15.17:0/3447705349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544158db0 0x7fa5440c5710 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa5440ca870 tx=0x7fa53802b910 comp rx=0 tx=0).stop 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 -- 172.21.15.17:0/3447705349 shutdown_connections 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 --2- 172.21.15.17:0/3447705349 >> v2:172.21.15.163:3300/0 conn(0x7fa5440c88b0 0x7fa54415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 --2- 172.21.15.17:0/3447705349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa544158db0 0x7fa5440c5710 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 -- 172.21.15.17:0/3447705349 >> 172.21.15.17:0/3447705349 conn(0x7fa5440ac560 msgr2=0x7fa5440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 -- 172.21.15.17:0/3447705349 shutdown_connections 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.584+0000 7fa5491f7640 1 -- 172.21.15.17:0/3447705349 wait complete. 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.585+0000 7fa5491f7640 1 Processor -- start 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.585+0000 7fa5491f7640 1 -- start start 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5440c5330 0x7fa54414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5440c88b0 0x7fa5440c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5441529d0 con 0x7fa5440c1fc0 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54406d090 con 0x7fa5440c88b0 2024-03-06T07:22:39.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa5491f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa544152b50 con 0x7fa5440c5330 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35620/0 (socket says 172.21.15.17:35620) 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 -- 172.21.15.17:0/42610168 learned_addr learned my addr 172.21.15.17:0/42610168 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54259c640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5440c88b0 0x7fa5440c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa542d9d640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5440c5330 0x7fa54414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 -- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5440c5330 msgr2=0x7fa54414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5440c5330 0x7fa54414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 -- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5440c88b0 msgr2=0x7fa5440c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5440c88b0 0x7fa5440c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 -- 172.21.15.17:0/42610168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544158140 con 0x7fa5440c1fc0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.586+0000 7fa54359e640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa534009e30 tx=0x7fa53400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.587+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534018020 con 0x7fa5440c1fc0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.587+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa534014480 con 0x7fa5440c1fc0 2024-03-06T07:22:39.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.587+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa534014870 con 0x7fa5440c1fc0 2024-03-06T07:22:39.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.587+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa544158420 con 0x7fa5440c1fc0 2024-03-06T07:22:39.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.587+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa544156550 con 0x7fa5440c1fc0 2024-03-06T07:22:39.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.588+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa544156880 con 0x7fa5440c1fc0 2024-03-06T07:22:39.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.588+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa534005d10 con 0x7fa5440c1fc0 2024-03-06T07:22:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.589+0000 7fa52b7fe640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.589+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa53408b070 con 0x7fa5440c1fc0 2024-03-06T07:22:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.589+0000 7fa542d9d640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.589+0000 7fa542d9d640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 0x7fa53006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa5440c4bf0 tx=0x7fa53c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:39.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.593+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa534022030 con 0x7fa5440c1fc0 2024-03-06T07:22:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.768+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa54404f330 con 0x7fa53006b4d0 2024-03-06T07:22:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.780+0000 7fa52b7fe640 1 -- 172.21.15.17:0/42610168 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252121 (secure 0 0 0) 0x7fa54404f330 con 0x7fa53006b4d0 2024-03-06T07:22:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:39.785 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 msgr2=0x7fa53006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 0x7fa53006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa5440c4bf0 tx=0x7fa53c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 msgr2=0x7fa5441ede00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa534009e30 tx=0x7fa53400c490 comp rx=0 tx=0).stop 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 shutdown_connections 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa53006b4d0 0x7fa53006d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5440c1fc0 0x7fa5441ede00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5440c88b0 0x7fa5440c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 --2- 172.21.15.17:0/42610168 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5440c5330 0x7fa54414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 >> 172.21.15.17:0/42610168 conn(0x7fa5440ac560 msgr2=0x7fa54414f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 shutdown_connections 2024-03-06T07:22:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:39.783+0000 7fa5491f7640 1 -- 172.21.15.17:0/42610168 wait complete. 2024-03-06T07:22:39.795 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-03-06T07:22:39.375937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240128,"kb_used_data":7300,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734592,"statfs":{"total":773094113280,"available":772848222208,"internally_reserved":0,"allocated":7475200,"data_stored":4641936,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000979"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367497+0000","last_change":"2024-03-06T07:22:35.363195+0000","last_active":"2024-03-06T07:22:36.367497+0000","last_peered":"2024-03-06T07:22:36.367497+0000","last_clean":"2024-03-06T07:22:36.367497+0000","last_became_active":"2024-03-06T07:22:35.362811+0000","last_became_peered":"2024-03-06T07:22:35.362811+0000","last_unstale":"2024-03-06T07:22:36.367497+0000","last_undegraded":"2024-03-06T07:22:36.367497+0000","last_fullsized":"2024-03-06T07:22:36.367497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353188+0000","last_change":"2024-03-06T07:22:35.363283+0000","last_active":"2024-03-06T07:22:37.353188+0000","last_peered":"2024-03-06T07:22:37.353188+0000","last_clean":"2024-03-06T07:22:37.353188+0000","last_became_active":"2024-03-06T07:22:35.363028+0000","last_became_peered":"2024-03-06T07:22:35.363028+0000","last_unstale":"2024-03-06T07:22:37.353188+0000","last_undegraded":"2024-03-06T07:22:37.353188+0000","last_fullsized":"2024-03-06T07:22:37.353188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369812+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:35.369812+0000","last_peered":"2024-03-06T07:22:35.369812+0000","last_clean":"2024-03-06T07:22:35.369812+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:35.369812+0000","last_undegraded":"2024-03-06T07:22:35.369812+0000","last_fullsized":"2024-03-06T07:22:35.369812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373335+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:35.373335+0000","last_peered":"2024-03-06T07:22:35.373335+0000","last_clean":"2024-03-06T07:22:35.373335+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:35.373335+0000","last_undegraded":"2024-03-06T07:22:35.373335+0000","last_fullsized":"2024-03-06T07:22:35.373335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:21.055978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294701+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:37.294701+0000","last_peered":"2024-03-06T07:22:37.294701+0000","last_clean":"2024-03-06T07:22:37.294701+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:37.294701+0000","last_undegraded":"2024-03-06T07:22:37.294701+0000","last_fullsized":"2024-03-06T07:22:37.294701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365453+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:36.365453+0000","last_peered":"2024-03-06T07:22:36.365453+0000","last_clean":"2024-03-06T07:22:36.365453+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:36.365453+0000","last_undegraded":"2024-03-06T07:22:36.365453+0000","last_fullsized":"2024-03-06T07:22:36.365453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373881+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:35.373881+0000","last_peered":"2024-03-06T07:22:35.373881+0000","last_clean":"2024-03-06T07:22:35.373881+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:35.373881+0000","last_undegraded":"2024-03-06T07:22:35.373881+0000","last_fullsized":"2024-03-06T07:22:35.373881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294583+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:37.294583+0000","last_peered":"2024-03-06T07:22:37.294583+0000","last_clean":"2024-03-06T07:22:37.294583+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:37.294583+0000","last_undegraded":"2024-03-06T07:22:37.294583+0000","last_fullsized":"2024-03-06T07:22:37.294583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366870+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:36.366870+0000","last_peered":"2024-03-06T07:22:36.366870+0000","last_clean":"2024-03-06T07:22:36.366870+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:36.366870+0000","last_undegraded":"2024-03-06T07:22:36.366870+0000","last_fullsized":"2024-03-06T07:22:36.366870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.374279+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:35.374279+0000","last_peered":"2024-03-06T07:22:35.374279+0000","last_clean":"2024-03-06T07:22:35.374279+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:35.374279+0000","last_undegraded":"2024-03-06T07:22:35.374279+0000","last_fullsized":"2024-03-06T07:22:35.374279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:38.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353236+0000","last_change":"2024-03-06T07:22:35.365945+0000","last_active":"2024-03-06T07:22:37.353236+0000","last_peered":"2024-03-06T07:22:37.353236+0000","last_clean":"2024-03-06T07:22:37.353236+0000","last_became_active":"2024-03-06T07:22:35.365677+0000","last_became_peered":"2024-03-06T07:22:35.365677+0000","last_unstale":"2024-03-06T07:22:37.353236+0000","last_undegraded":"2024-03-06T07:22:37.353236+0000","last_fullsized":"2024-03-06T07:22:37.353236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373392+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:35.373392+0000","last_peered":"2024-03-06T07:22:35.373392+0000","last_clean":"2024-03-06T07:22:35.373392+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:35.373392+0000","last_undegraded":"2024-03-06T07:22:35.373392+0000","last_fullsized":"2024-03-06T07:22:35.373392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:14.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365552+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:36.365552+0000","last_peered":"2024-03-06T07:22:36.365552+0000","last_clean":"2024-03-06T07:22:36.365552+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:36.365552+0000","last_undegraded":"2024-03-06T07:22:36.365552+0000","last_fullsized":"2024-03-06T07:22:36.365552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352693+0000","last_change":"2024-03-06T07:22:35.366910+0000","last_active":"2024-03-06T07:22:37.352693+0000","last_peered":"2024-03-06T07:22:37.352693+0000","last_clean":"2024-03-06T07:22:37.352693+0000","last_became_active":"2024-03-06T07:22:35.366623+0000","last_became_peered":"2024-03-06T07:22:35.366623+0000","last_unstale":"2024-03-06T07:22:37.352693+0000","last_undegraded":"2024-03-06T07:22:37.352693+0000","last_fullsized":"2024-03-06T07:22:37.352693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366692+0000","last_change":"2024-03-06T07:22:35.360892+0000","last_active":"2024-03-06T07:22:36.366692+0000","last_peered":"2024-03-06T07:22:36.366692+0000","last_clean":"2024-03-06T07:22:36.366692+0000","last_became_active":"2024-03-06T07:22:35.360615+0000","last_became_peered":"2024-03-06T07:22:35.360615+0000","last_unstale":"2024-03-06T07:22:36.366692+0000","last_undegraded":"2024-03-06T07:22:36.366692+0000","last_fullsized":"2024-03-06T07:22:36.366692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:15.740118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295121+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:37.295121+0000","last_peered":"2024-03-06T07:22:37.295121+0000","last_clean":"2024-03-06T07:22:37.295121+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:37.295121+0000","last_undegraded":"2024-03-06T07:22:37.295121+0000","last_fullsized":"2024-03-06T07:22:37.295121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366750+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:35.366750+0000","last_peered":"2024-03-06T07:22:35.366750+0000","last_clean":"2024-03-06T07:22:35.366750+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:35.366750+0000","last_undegraded":"2024-03-06T07:22:35.366750+0000","last_fullsized":"2024-03-06T07:22:35.366750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365181+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:36.365181+0000","last_peered":"2024-03-06T07:22:36.365181+0000","last_clean":"2024-03-06T07:22:36.365181+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:36.365181+0000","last_undegraded":"2024-03-06T07:22:36.365181+0000","last_fullsized":"2024-03-06T07:22:36.365181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366477+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:36.366477+0000","last_peered":"2024-03-06T07:22:36.366477+0000","last_clean":"2024-03-06T07:22:36.366477+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:36.366477+0000","last_undegraded":"2024-03-06T07:22:36.366477+0000","last_fullsized":"2024-03-06T07:22:36.366477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294614+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:37.294614+0000","last_peered":"2024-03-06T07:22:37.294614+0000","last_clean":"2024-03-06T07:22:37.294614+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:37.294614+0000","last_undegraded":"2024-03-06T07:22:37.294614+0000","last_fullsized":"2024-03-06T07:22:37.294614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366253+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:36.366253+0000","last_peered":"2024-03-06T07:22:36.366253+0000","last_clean":"2024-03-06T07:22:36.366253+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:36.366253+0000","last_undegraded":"2024-03-06T07:22:36.366253+0000","last_fullsized":"2024-03-06T07:22:36.366253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352681+0000","last_change":"2024-03-06T07:22:35.367093+0000","last_active":"2024-03-06T07:22:37.352681+0000","last_peered":"2024-03-06T07:22:37.352681+0000","last_clean":"2024-03-06T07:22:37.352681+0000","last_became_active":"2024-03-06T07:22:35.366829+0000","last_became_peered":"2024-03-06T07:22:35.366829+0000","last_unstale":"2024-03-06T07:22:37.352681+0000","last_undegraded":"2024-03-06T07:22:37.352681+0000","last_fullsized":"2024-03-06T07:22:37.352681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353121+0000","last_change":"2024-03-06T07:22:35.367432+0000","last_active":"2024-03-06T07:22:37.353121+0000","last_peered":"2024-03-06T07:22:37.353121+0000","last_clean":"2024-03-06T07:22:37.353121+0000","last_became_active":"2024-03-06T07:22:35.367164+0000","last_became_peered":"2024-03-06T07:22:35.367164+0000","last_unstale":"2024-03-06T07:22:37.353121+0000","last_undegraded":"2024-03-06T07:22:37.353121+0000","last_fullsized":"2024-03-06T07:22:37.353121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376691+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:35.376691+0000","last_peered":"2024-03-06T07:22:35.376691+0000","last_clean":"2024-03-06T07:22:35.376691+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:35.376691+0000","last_undegraded":"2024-03-06T07:22:35.376691+0000","last_fullsized":"2024-03-06T07:22:35.376691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:05.191375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367571+0000","last_change":"2024-03-06T07:22:35.373963+0000","last_active":"2024-03-06T07:22:36.367571+0000","last_peered":"2024-03-06T07:22:36.367571+0000","last_clean":"2024-03-06T07:22:36.367571+0000","last_became_active":"2024-03-06T07:22:35.373704+0000","last_became_peered":"2024-03-06T07:22:35.373704+0000","last_unstale":"2024-03-06T07:22:36.367571+0000","last_undegraded":"2024-03-06T07:22:36.367571+0000","last_fullsized":"2024-03-06T07:22:36.367571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295009+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:37.295009+0000","last_peered":"2024-03-06T07:22:37.295009+0000","last_clean":"2024-03-06T07:22:37.295009+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:37.295009+0000","last_undegraded":"2024-03-06T07:22:37.295009+0000","last_fullsized":"2024-03-06T07:22:37.295009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352994+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:37.352994+0000","last_peered":"2024-03-06T07:22:37.352994+0000","last_clean":"2024-03-06T07:22:37.352994+0000","last_became_active":"2024-03-06T07:22:35.361468+0000","last_became_peered":"2024-03-06T07:22:35.361468+0000","last_unstale":"2024-03-06T07:22:37.352994+0000","last_undegraded":"2024-03-06T07:22:37.352994+0000","last_fullsized":"2024-03-06T07:22:37.352994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:50.366055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366836+0000","last_change":"2024-03-06T07:22:35.372500+0000","last_active":"2024-03-06T07:22:36.366836+0000","last_peered":"2024-03-06T07:22:36.366836+0000","last_clean":"2024-03-06T07:22:36.366836+0000","last_became_active":"2024-03-06T07:22:35.372221+0000","last_became_peered":"2024-03-06T07:22:35.372221+0000","last_unstale":"2024-03-06T07:22:36.366836+0000","last_undegraded":"2024-03-06T07:22:36.366836+0000","last_fullsized":"2024-03-06T07:22:36.366836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:23.442086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369741+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:35.369741+0000","last_peered":"2024-03-06T07:22:35.369741+0000","last_clean":"2024-03-06T07:22:35.369741+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:35.369741+0000","last_undegraded":"2024-03-06T07:22:35.369741+0000","last_fullsized":"2024-03-06T07:22:35.369741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366862+0000","last_change":"2024-03-06T07:22:35.363020+0000","last_active":"2024-03-06T07:22:36.366862+0000","last_peered":"2024-03-06T07:22:36.366862+0000","last_clean":"2024-03-06T07:22:36.366862+0000","last_became_active":"2024-03-06T07:22:35.362767+0000","last_became_peered":"2024-03-06T07:22:35.362767+0000","last_unstale":"2024-03-06T07:22:36.366862+0000","last_undegraded":"2024-03-06T07:22:36.366862+0000","last_fullsized":"2024-03-06T07:22:36.366862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:56.148980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.360317+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:35.360317+0000","last_peered":"2024-03-06T07:22:35.360317+0000","last_clean":"2024-03-06T07:22:35.360317+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:35.360317+0000","last_undegraded":"2024-03-06T07:22:35.360317+0000","last_fullsized":"2024-03-06T07:22:35.360317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295198+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:37.295198+0000","last_peered":"2024-03-06T07:22:37.295198+0000","last_clean":"2024-03-06T07:22:37.295198+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:37.295198+0000","last_undegraded":"2024-03-06T07:22:37.295198+0000","last_fullsized":"2024-03-06T07:22:37.295198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352699+0000","last_change":"2024-03-06T07:22:35.368816+0000","last_active":"2024-03-06T07:22:37.352699+0000","last_peered":"2024-03-06T07:22:37.352699+0000","last_clean":"2024-03-06T07:22:37.352699+0000","last_became_active":"2024-03-06T07:22:35.368555+0000","last_became_peered":"2024-03-06T07:22:35.368555+0000","last_unstale":"2024-03-06T07:22:37.352699+0000","last_undegraded":"2024-03-06T07:22:37.352699+0000","last_fullsized":"2024-03-06T07:22:37.352699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367192+0000","last_change":"2024-03-06T07:22:34.366335+0000","last_active":"2024-03-06T07:22:36.367192+0000","last_peered":"2024-03-06T07:22:36.367192+0000","last_clean":"2024-03-06T07:22:36.367192+0000","last_became_active":"2024-03-06T07:22:34.364387+0000","last_became_peered":"2024-03-06T07:22:34.364387+0000","last_unstale":"2024-03-06T07:22:36.367192+0000","last_undegraded":"2024-03-06T07:22:36.367192+0000","last_fullsized":"2024-03-06T07:22:36.367192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366415+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:36.366415+0000","last_peered":"2024-03-06T07:22:36.366415+0000","last_clean":"2024-03-06T07:22:36.366415+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:36.366415+0000","last_undegraded":"2024-03-06T07:22:36.366415+0000","last_fullsized":"2024-03-06T07:22:36.366415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367686+0000","last_change":"2024-03-06T07:22:34.364533+0000","last_active":"2024-03-06T07:22:36.367686+0000","last_peered":"2024-03-06T07:22:36.367686+0000","last_clean":"2024-03-06T07:22:36.367686+0000","last_became_active":"2024-03-06T07:22:34.364224+0000","last_became_peered":"2024-03-06T07:22:34.364224+0000","last_unstale":"2024-03-06T07:22:36.367686+0000","last_undegraded":"2024-03-06T07:22:36.367686+0000","last_fullsized":"2024-03-06T07:22:36.367686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:13.454983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367050+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:36.367050+0000","last_peered":"2024-03-06T07:22:36.367050+0000","last_clean":"2024-03-06T07:22:36.367050+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:36.367050+0000","last_undegraded":"2024-03-06T07:22:36.367050+0000","last_fullsized":"2024-03-06T07:22:36.367050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373794+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:35.373794+0000","last_peered":"2024-03-06T07:22:35.373794+0000","last_clean":"2024-03-06T07:22:35.373794+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:35.373794+0000","last_undegraded":"2024-03-06T07:22:35.373794+0000","last_fullsized":"2024-03-06T07:22:35.373794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353178+0000","last_change":"2024-03-06T07:22:35.365350+0000","last_active":"2024-03-06T07:22:37.353178+0000","last_peered":"2024-03-06T07:22:37.353178+0000","last_clean":"2024-03-06T07:22:37.353178+0000","last_became_active":"2024-03-06T07:22:35.365049+0000","last_became_peered":"2024-03-06T07:22:35.365049+0000","last_unstale":"2024-03-06T07:22:37.353178+0000","last_undegraded":"2024-03-06T07:22:37.353178+0000","last_fullsized":"2024-03-06T07:22:37.353178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367160+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:36.367160+0000","last_peered":"2024-03-06T07:22:36.367160+0000","last_clean":"2024-03-06T07:22:36.367160+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:36.367160+0000","last_undegraded":"2024-03-06T07:22:36.367160+0000","last_fullsized":"2024-03-06T07:22:36.367160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353342+0000","last_change":"2024-03-06T07:22:35.363944+0000","last_active":"2024-03-06T07:22:37.353342+0000","last_peered":"2024-03-06T07:22:37.353342+0000","last_clean":"2024-03-06T07:22:37.353342+0000","last_became_active":"2024-03-06T07:22:35.363662+0000","last_became_peered":"2024-03-06T07:22:35.363662+0000","last_unstale":"2024-03-06T07:22:37.353342+0000","last_undegraded":"2024-03-06T07:22:37.353342+0000","last_fullsized":"2024-03-06T07:22:37.353342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:17.265969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372297+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:35.372297+0000","last_peered":"2024-03-06T07:22:35.372297+0000","last_clean":"2024-03-06T07:22:35.372297+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:35.372297+0000","last_undegraded":"2024-03-06T07:22:35.372297+0000","last_fullsized":"2024-03-06T07:22:35.372297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367496+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:36.367496+0000","last_peered":"2024-03-06T07:22:36.367496+0000","last_clean":"2024-03-06T07:22:36.367496+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:36.367496+0000","last_undegraded":"2024-03-06T07:22:36.367496+0000","last_fullsized":"2024-03-06T07:22:36.367496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352852+0000","last_change":"2024-03-06T07:22:34.366433+0000","last_active":"2024-03-06T07:22:37.352852+0000","last_peered":"2024-03-06T07:22:37.352852+0000","last_clean":"2024-03-06T07:22:37.352852+0000","last_became_active":"2024-03-06T07:22:34.365956+0000","last_became_peered":"2024-03-06T07:22:34.365956+0000","last_unstale":"2024-03-06T07:22:37.352852+0000","last_undegraded":"2024-03-06T07:22:37.352852+0000","last_fullsized":"2024-03-06T07:22:37.352852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376711+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:35.376711+0000","last_peered":"2024-03-06T07:22:35.376711+0000","last_clean":"2024-03-06T07:22:35.376711+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:35.376711+0000","last_undegraded":"2024-03-06T07:22:35.376711+0000","last_fullsized":"2024-03-06T07:22:35.376711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294626+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:37.294626+0000","last_peered":"2024-03-06T07:22:37.294626+0000","last_clean":"2024-03-06T07:22:37.294626+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:37.294626+0000","last_undegraded":"2024-03-06T07:22:37.294626+0000","last_fullsized":"2024-03-06T07:22:37.294626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352824+0000","last_change":"2024-03-06T07:22:35.365470+0000","last_active":"2024-03-06T07:22:37.352824+0000","last_peered":"2024-03-06T07:22:37.352824+0000","last_clean":"2024-03-06T07:22:37.352824+0000","last_became_active":"2024-03-06T07:22:35.365156+0000","last_became_peered":"2024-03-06T07:22:35.365156+0000","last_unstale":"2024-03-06T07:22:37.352824+0000","last_undegraded":"2024-03-06T07:22:37.352824+0000","last_fullsized":"2024-03-06T07:22:37.352824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:45.193188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.366420+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.365577+0000","last_became_peered":"2024-03-06T07:22:34.365577+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:20.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372836+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:35.372836+0000","last_peered":"2024-03-06T07:22:35.372836+0000","last_clean":"2024-03-06T07:22:35.372836+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:35.372836+0000","last_undegraded":"2024-03-06T07:22:35.372836+0000","last_fullsized":"2024-03-06T07:22:35.372836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:52.932700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366830+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:35.366830+0000","last_peered":"2024-03-06T07:22:35.366830+0000","last_clean":"2024-03-06T07:22:35.366830+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:35.366830+0000","last_undegraded":"2024-03-06T07:22:35.366830+0000","last_fullsized":"2024-03-06T07:22:35.366830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365569+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:36.365569+0000","last_peered":"2024-03-06T07:22:36.365569+0000","last_clean":"2024-03-06T07:22:36.365569+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:36.365569+0000","last_undegraded":"2024-03-06T07:22:36.365569+0000","last_fullsized":"2024-03-06T07:22:36.365569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295075+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:37.295075+0000","last_peered":"2024-03-06T07:22:37.295075+0000","last_clean":"2024-03-06T07:22:37.295075+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:37.295075+0000","last_undegraded":"2024-03-06T07:22:37.295075+0000","last_fullsized":"2024-03-06T07:22:37.295075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370865+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:35.370865+0000","last_peered":"2024-03-06T07:22:35.370865+0000","last_clean":"2024-03-06T07:22:35.370865+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:35.370865+0000","last_undegraded":"2024-03-06T07:22:35.370865+0000","last_fullsized":"2024-03-06T07:22:35.370865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.379447+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:35.379447+0000","last_peered":"2024-03-06T07:22:35.379447+0000","last_clean":"2024-03-06T07:22:35.379447+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:35.379447+0000","last_undegraded":"2024-03-06T07:22:35.379447+0000","last_fullsized":"2024-03-06T07:22:35.379447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372879+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:35.372879+0000","last_peered":"2024-03-06T07:22:35.372879+0000","last_clean":"2024-03-06T07:22:35.372879+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:35.372879+0000","last_undegraded":"2024-03-06T07:22:35.372879+0000","last_fullsized":"2024-03-06T07:22:35.372879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:54.478837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376739+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:35.376739+0000","last_peered":"2024-03-06T07:22:35.376739+0000","last_clean":"2024-03-06T07:22:35.376739+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:35.376739+0000","last_undegraded":"2024-03-06T07:22:35.376739+0000","last_fullsized":"2024-03-06T07:22:35.376739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372010+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:35.372010+0000","last_peered":"2024-03-06T07:22:35.372010+0000","last_clean":"2024-03-06T07:22:35.372010+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:35.372010+0000","last_undegraded":"2024-03-06T07:22:35.372010+0000","last_fullsized":"2024-03-06T07:22:35.372010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366957+0000","last_change":"2024-03-06T07:22:34.364874+0000","last_active":"2024-03-06T07:22:36.366957+0000","last_peered":"2024-03-06T07:22:36.366957+0000","last_clean":"2024-03-06T07:22:36.366957+0000","last_became_active":"2024-03-06T07:22:34.364472+0000","last_became_peered":"2024-03-06T07:22:34.364472+0000","last_unstale":"2024-03-06T07:22:36.366957+0000","last_undegraded":"2024-03-06T07:22:36.366957+0000","last_fullsized":"2024-03-06T07:22:36.366957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.364306+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:35.364306+0000","last_peered":"2024-03-06T07:22:35.364306+0000","last_clean":"2024-03-06T07:22:35.364306+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:35.364306+0000","last_undegraded":"2024-03-06T07:22:35.364306+0000","last_fullsized":"2024-03-06T07:22:35.364306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:46.770166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294650+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:37.294650+0000","last_peered":"2024-03-06T07:22:37.294650+0000","last_clean":"2024-03-06T07:22:37.294650+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:37.294650+0000","last_undegraded":"2024-03-06T07:22:37.294650+0000","last_fullsized":"2024-03-06T07:22:37.294650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372602+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:35.372602+0000","last_peered":"2024-03-06T07:22:35.372602+0000","last_clean":"2024-03-06T07:22:35.372602+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:35.372602+0000","last_undegraded":"2024-03-06T07:22:35.372602+0000","last_fullsized":"2024-03-06T07:22:35.372602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352812+0000","last_change":"2024-03-06T07:22:34.361474+0000","last_active":"2024-03-06T07:22:37.352812+0000","last_peered":"2024-03-06T07:22:37.352812+0000","last_clean":"2024-03-06T07:22:37.352812+0000","last_became_active":"2024-03-06T07:22:34.360526+0000","last_became_peered":"2024-03-06T07:22:34.360526+0000","last_unstale":"2024-03-06T07:22:37.352812+0000","last_undegraded":"2024-03-06T07:22:37.352812+0000","last_fullsized":"2024-03-06T07:22:37.352812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.364939+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:36.364939+0000","last_peered":"2024-03-06T07:22:36.364939+0000","last_clean":"2024-03-06T07:22:36.364939+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:36.364939+0000","last_undegraded":"2024-03-06T07:22:36.364939+0000","last_fullsized":"2024-03-06T07:22:36.364939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367712+0000","last_change":"2024-03-06T07:22:34.365404+0000","last_active":"2024-03-06T07:22:36.367712+0000","last_peered":"2024-03-06T07:22:36.367712+0000","last_clean":"2024-03-06T07:22:36.367712+0000","last_became_active":"2024-03-06T07:22:34.365029+0000","last_became_peered":"2024-03-06T07:22:34.365029+0000","last_unstale":"2024-03-06T07:22:36.367712+0000","last_undegraded":"2024-03-06T07:22:36.367712+0000","last_fullsized":"2024-03-06T07:22:36.367712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.473973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294660+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:37.294660+0000","last_peered":"2024-03-06T07:22:37.294660+0000","last_clean":"2024-03-06T07:22:37.294660+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:37.294660+0000","last_undegraded":"2024-03-06T07:22:37.294660+0000","last_fullsized":"2024-03-06T07:22:37.294660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294702+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:37.294702+0000","last_peered":"2024-03-06T07:22:37.294702+0000","last_clean":"2024-03-06T07:22:37.294702+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:37.294702+0000","last_undegraded":"2024-03-06T07:22:37.294702+0000","last_fullsized":"2024-03-06T07:22:37.294702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295156+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:37.295156+0000","last_peered":"2024-03-06T07:22:37.295156+0000","last_clean":"2024-03-06T07:22:37.295156+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:37.295156+0000","last_undegraded":"2024-03-06T07:22:37.295156+0000","last_fullsized":"2024-03-06T07:22:37.295156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365086+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:36.365086+0000","last_peered":"2024-03-06T07:22:36.365086+0000","last_clean":"2024-03-06T07:22:36.365086+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:36.365086+0000","last_undegraded":"2024-03-06T07:22:36.365086+0000","last_fullsized":"2024-03-06T07:22:36.365086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351435+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:35.351435+0000","last_peered":"2024-03-06T07:22:35.351435+0000","last_clean":"2024-03-06T07:22:35.351435+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:35.351435+0000","last_undegraded":"2024-03-06T07:22:35.351435+0000","last_fullsized":"2024-03-06T07:22:35.351435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353248+0000","last_change":"2024-03-06T07:22:35.370112+0000","last_active":"2024-03-06T07:22:37.353248+0000","last_peered":"2024-03-06T07:22:37.353248+0000","last_clean":"2024-03-06T07:22:37.353248+0000","last_became_active":"2024-03-06T07:22:35.369845+0000","last_became_peered":"2024-03-06T07:22:35.369845+0000","last_unstale":"2024-03-06T07:22:37.353248+0000","last_undegraded":"2024-03-06T07:22:37.353248+0000","last_fullsized":"2024-03-06T07:22:37.353248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365194+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:36.365194+0000","last_peered":"2024-03-06T07:22:36.365194+0000","last_clean":"2024-03-06T07:22:36.365194+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:36.365194+0000","last_undegraded":"2024-03-06T07:22:36.365194+0000","last_fullsized":"2024-03-06T07:22:36.365194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353270+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:37.353270+0000","last_peered":"2024-03-06T07:22:37.353270+0000","last_clean":"2024-03-06T07:22:37.353270+0000","last_became_active":"2024-03-06T07:22:35.361332+0000","last_became_peered":"2024-03-06T07:22:35.361332+0000","last_unstale":"2024-03-06T07:22:37.353270+0000","last_undegraded":"2024-03-06T07:22:37.353270+0000","last_fullsized":"2024-03-06T07:22:37.353270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373686+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:35.373686+0000","last_peered":"2024-03-06T07:22:35.373686+0000","last_clean":"2024-03-06T07:22:35.373686+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:35.373686+0000","last_undegraded":"2024-03-06T07:22:35.373686+0000","last_fullsized":"2024-03-06T07:22:35.373686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366256+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:36.366256+0000","last_peered":"2024-03-06T07:22:36.366256+0000","last_clean":"2024-03-06T07:22:36.366256+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:36.366256+0000","last_undegraded":"2024-03-06T07:22:36.366256+0000","last_fullsized":"2024-03-06T07:22:36.366256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371730+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:35.371730+0000","last_peered":"2024-03-06T07:22:35.371730+0000","last_clean":"2024-03-06T07:22:35.371730+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:35.371730+0000","last_undegraded":"2024-03-06T07:22:35.371730+0000","last_fullsized":"2024-03-06T07:22:35.371730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:35.366022+0000","last_peered":"2024-03-06T07:22:35.366022+0000","last_clean":"2024-03-06T07:22:35.366022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:35.366022+0000","last_undegraded":"2024-03-06T07:22:35.366022+0000","last_fullsized":"2024-03-06T07:22:35.366022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:02.842916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.354591+0000","last_change":"2024-03-06T07:22:34.362718+0000","last_active":"2024-03-06T07:22:37.354591+0000","last_peered":"2024-03-06T07:22:37.354591+0000","last_clean":"2024-03-06T07:22:37.354591+0000","last_became_active":"2024-03-06T07:22:34.362448+0000","last_became_peered":"2024-03-06T07:22:34.362448+0000","last_unstale":"2024-03-06T07:22:37.354591+0000","last_undegraded":"2024-03-06T07:22:37.354591+0000","last_fullsized":"2024-03-06T07:22:37.354591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366562+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:36.366562+0000","last_peered":"2024-03-06T07:22:36.366562+0000","last_clean":"2024-03-06T07:22:36.366562+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:36.366562+0000","last_undegraded":"2024-03-06T07:22:36.366562+0000","last_fullsized":"2024-03-06T07:22:36.366562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367833+0000","last_change":"2024-03-06T07:22:34.365631+0000","last_active":"2024-03-06T07:22:36.367833+0000","last_peered":"2024-03-06T07:22:36.367833+0000","last_clean":"2024-03-06T07:22:36.367833+0000","last_became_active":"2024-03-06T07:22:34.364558+0000","last_became_peered":"2024-03-06T07:22:34.364558+0000","last_unstale":"2024-03-06T07:22:36.367833+0000","last_undegraded":"2024-03-06T07:22:36.367833+0000","last_fullsized":"2024-03-06T07:22:36.367833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:37.167691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352939+0000","last_change":"2024-03-06T07:22:35.362565+0000","last_active":"2024-03-06T07:22:37.352939+0000","last_peered":"2024-03-06T07:22:37.352939+0000","last_clean":"2024-03-06T07:22:37.352939+0000","last_became_active":"2024-03-06T07:22:35.362296+0000","last_became_peered":"2024-03-06T07:22:35.362296+0000","last_unstale":"2024-03-06T07:22:37.352939+0000","last_undegraded":"2024-03-06T07:22:37.352939+0000","last_fullsized":"2024-03-06T07:22:37.352939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:19.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367067+0000","last_change":"2024-03-06T07:22:34.360939+0000","last_active":"2024-03-06T07:22:36.367067+0000","last_peered":"2024-03-06T07:22:36.367067+0000","last_clean":"2024-03-06T07:22:36.367067+0000","last_became_active":"2024-03-06T07:22:34.360665+0000","last_became_peered":"2024-03-06T07:22:34.360665+0000","last_unstale":"2024-03-06T07:22:36.367067+0000","last_undegraded":"2024-03-06T07:22:36.367067+0000","last_fullsized":"2024-03-06T07:22:36.367067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370130+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:35.370130+0000","last_peered":"2024-03-06T07:22:35.370130+0000","last_clean":"2024-03-06T07:22:35.370130+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:35.370130+0000","last_undegraded":"2024-03-06T07:22:35.370130+0000","last_fullsized":"2024-03-06T07:22:35.370130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:16.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.352766+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:35.352766+0000","last_peered":"2024-03-06T07:22:35.352766+0000","last_clean":"2024-03-06T07:22:35.352766+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:35.352766+0000","last_undegraded":"2024-03-06T07:22:35.352766+0000","last_fullsized":"2024-03-06T07:22:35.352766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373867+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:35.373867+0000","last_peered":"2024-03-06T07:22:35.373867+0000","last_clean":"2024-03-06T07:22:35.373867+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:35.373867+0000","last_undegraded":"2024-03-06T07:22:35.373867+0000","last_fullsized":"2024-03-06T07:22:35.373867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294742+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:37.294742+0000","last_peered":"2024-03-06T07:22:37.294742+0000","last_clean":"2024-03-06T07:22:37.294742+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:37.294742+0000","last_undegraded":"2024-03-06T07:22:37.294742+0000","last_fullsized":"2024-03-06T07:22:37.294742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369774+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:35.369774+0000","last_peered":"2024-03-06T07:22:35.369774+0000","last_clean":"2024-03-06T07:22:35.369774+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:35.369774+0000","last_undegraded":"2024-03-06T07:22:35.369774+0000","last_fullsized":"2024-03-06T07:22:35.369774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.353050+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:35.353050+0000","last_peered":"2024-03-06T07:22:35.353050+0000","last_clean":"2024-03-06T07:22:35.353050+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:35.353050+0000","last_undegraded":"2024-03-06T07:22:35.353050+0000","last_fullsized":"2024-03-06T07:22:35.353050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371951+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:35.371951+0000","last_peered":"2024-03-06T07:22:35.371951+0000","last_clean":"2024-03-06T07:22:35.371951+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:35.371951+0000","last_undegraded":"2024-03-06T07:22:35.371951+0000","last_fullsized":"2024-03-06T07:22:35.371951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351080+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:35.351080+0000","last_peered":"2024-03-06T07:22:35.351080+0000","last_clean":"2024-03-06T07:22:35.351080+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:35.351080+0000","last_undegraded":"2024-03-06T07:22:35.351080+0000","last_fullsized":"2024-03-06T07:22:35.351080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367852+0000","last_change":"2024-03-06T07:22:35.373140+0000","last_active":"2024-03-06T07:22:36.367852+0000","last_peered":"2024-03-06T07:22:36.367852+0000","last_clean":"2024-03-06T07:22:36.367852+0000","last_became_active":"2024-03-06T07:22:35.372523+0000","last_became_peered":"2024-03-06T07:22:35.372523+0000","last_unstale":"2024-03-06T07:22:36.367852+0000","last_undegraded":"2024-03-06T07:22:36.367852+0000","last_fullsized":"2024-03-06T07:22:36.367852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365701+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:36.365701+0000","last_peered":"2024-03-06T07:22:36.365701+0000","last_clean":"2024-03-06T07:22:36.365701+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:36.365701+0000","last_undegraded":"2024-03-06T07:22:36.365701+0000","last_fullsized":"2024-03-06T07:22:36.365701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366754+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:36.366754+0000","last_peered":"2024-03-06T07:22:36.366754+0000","last_clean":"2024-03-06T07:22:36.366754+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:36.366754+0000","last_undegraded":"2024-03-06T07:22:36.366754+0000","last_fullsized":"2024-03-06T07:22:36.366754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":1026620,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":431150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":429572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":937984,"data_stored":437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":436812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":430280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":429650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":1019906,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:40.642 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:22:40.639+0000 7fd5ec8fd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:42.800 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.911+0000 7f3ce5363640 1 Processor -- start 2024-03-06T07:22:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.911+0000 7f3ce5363640 1 -- start start 2024-03-06T07:22:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3ce5363640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 0x7f3ce00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3ce5363640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 0x7f3ce00c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3ce5363640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce0158800 con 0x7f3ce01551e0 2024-03-06T07:22:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3ce5363640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce0158ae0 con 0x7f3ce006d9f0 2024-03-06T07:22:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3ce5363640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0158dc0 con 0x7f3ce00c3280 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3cde7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 0x7f3ce00c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3cdeffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 0x7f3ce00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3cdf7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3ce01551e0 0x7f3ce0155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46950/0 (socket says 172.21.15.17:46950) 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.912+0000 7f3cdf7fe640 1 -- 172.21.15.17:0/4021137385 learned_addr learned my addr 172.21.15.17:0/4021137385 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cddffb640 1 -- 172.21.15.17:0/4021137385 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488695912 0 0) 0x7f3ce0158800 con 0x7f3ce01551e0 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cddffb640 1 -- 172.21.15.17:0/4021137385 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc8003520 con 0x7f3ce01551e0 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cde7fc640 1 -- 172.21.15.17:0/4021137385 >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 msgr2=0x7f3ce00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cde7fc640 1 --2- 172.21.15.17:0/4021137385 >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 0x7f3ce00a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cde7fc640 1 -- 172.21.15.17:0/4021137385 >> v1:172.21.15.163:6789/0 conn(0x7f3ce01551e0 legacy=0x7f3ce0155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cde7fc640 1 -- 172.21.15.17:0/4021137385 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce01590a0 con 0x7f3ce00c3280 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.913+0000 7f3cdeffd640 1 --2- 172.21.15.17:0/4021137385 >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 0x7f3ce00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3cde7fc640 1 --2- 172.21.15.17:0/4021137385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 0x7f3ce00c36e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82276f2e4d4cfe58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3cddffb640 1 -- 172.21.15.17:0/4021137385 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40315a0 con 0x7f3ce00c3280 2024-03-06T07:22:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3cddffb640 1 -- 172.21.15.17:0/4021137385 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4037070 con 0x7f3ce00c3280 2024-03-06T07:22:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3cddffb640 1 -- 172.21.15.17:0/4021137385 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4031940 con 0x7f3ce00c3280 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 -- 172.21.15.17:0/4021137385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 msgr2=0x7f3ce00c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 --2- 172.21.15.17:0/4021137385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 0x7f3ce00c36e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402b840 comp rx=0 tx=0).stop 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 -- 172.21.15.17:0/4021137385 shutdown_connections 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 --2- 172.21.15.17:0/4021137385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce00c3280 0x7f3ce00c36e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 --2- 172.21.15.17:0/4021137385 >> v2:172.21.15.163:3300/0 conn(0x7f3ce006d9f0 0x7f3ce00a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 -- 172.21.15.17:0/4021137385 >> 172.21.15.17:0/4021137385 conn(0x7f3ce00aa630 msgr2=0x7f3ce00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 -- 172.21.15.17:0/4021137385 shutdown_connections 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.914+0000 7f3ce5363640 1 -- 172.21.15.17:0/4021137385 wait complete. 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 Processor -- start 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 -- start start 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce006d9f0 0x7f3ce00a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ce00c3280 0x7f3ce0156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce006df50 con 0x7f3ce01551e0 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce006ddd0 con 0x7f3ce00c3280 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.916+0000 7f3ce5363640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce00a8210 con 0x7f3ce006d9f0 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40700/0 (socket says 172.21.15.17:40700) 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 -- 172.21.15.17:0/538285211 learned_addr learned my addr 172.21.15.17:0/538285211 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:42.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdeffd640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce006d9f0 0x7f3ce00a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cde7fc640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ce00c3280 0x7f3ce0156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 -- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce006d9f0 msgr2=0x7f3ce00a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce006d9f0 0x7f3ce00a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 -- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ce00c3280 msgr2=0x7f3ce0156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ce00c3280 0x7f3ce0156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 -- 172.21.15.17:0/538285211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce0158590 con 0x7f3ce01551e0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cdf7fe640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c3c0 tx=0x7f3cd800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8017070 con 0x7f3ce01551e0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8005ce0 con 0x7f3ce01551e0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8007310 con 0x7f3ce01551e0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce0157fd0 con 0x7f3ce01551e0 2024-03-06T07:22:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.917+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce00aca70 con 0x7f3ce01551e0 2024-03-06T07:22:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.918+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce004f330 con 0x7f3ce01551e0 2024-03-06T07:22:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.919+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cd801a070 con 0x7f3ce01551e0 2024-03-06T07:22:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.919+0000 7f3cc77fe640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:42.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.919+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cd80895d0 con 0x7f3ce01551e0 2024-03-06T07:22:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.919+0000 7f3cdeffd640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:42.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.920+0000 7f3cdeffd640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009bd0 tx=0x7f3cd000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:42.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:42.924+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3cd8057d00 con 0x7f3ce01551e0 2024-03-06T07:22:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.100+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ce0073240 con 0x7f3ccc06b4d0 2024-03-06T07:22:43.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.107+0000 7f3cc77fe640 1 -- 172.21.15.17:0/538285211 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252121 (secure 0 0 0) 0x7f3ce0073240 con 0x7f3ccc06b4d0 2024-03-06T07:22:43.112 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:43.113 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:43.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 msgr2=0x7f3ccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009bd0 tx=0x7f3cd000b040 comp rx=0 tx=0).stop 2024-03-06T07:22:43.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 msgr2=0x7f3ce014daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800c3c0 tx=0x7f3cd800c880 comp rx=0 tx=0).stop 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 shutdown_connections 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ccc06b4d0 0x7f3ccc06d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ce01551e0 0x7f3ce014daf0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ce00c3280 0x7f3ce0156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 --2- 172.21.15.17:0/538285211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ce006d9f0 0x7f3ce00a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 >> 172.21.15.17:0/538285211 conn(0x7f3ce00aa630 msgr2=0x7f3ce00c7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.109+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 shutdown_connections 2024-03-06T07:22:43.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.110+0000 7f3ce5363640 1 -- 172.21.15.17:0/538285211 wait complete. 2024-03-06T07:22:43.122 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-03-06T07:22:41.376250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240128,"kb_used_data":7300,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734592,"statfs":{"total":773094113280,"available":772848222208,"internally_reserved":0,"allocated":7475200,"data_stored":4641936,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":238276148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.031403"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367497+0000","last_change":"2024-03-06T07:22:35.363195+0000","last_active":"2024-03-06T07:22:36.367497+0000","last_peered":"2024-03-06T07:22:36.367497+0000","last_clean":"2024-03-06T07:22:36.367497+0000","last_became_active":"2024-03-06T07:22:35.362811+0000","last_became_peered":"2024-03-06T07:22:35.362811+0000","last_unstale":"2024-03-06T07:22:36.367497+0000","last_undegraded":"2024-03-06T07:22:36.367497+0000","last_fullsized":"2024-03-06T07:22:36.367497+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353188+0000","last_change":"2024-03-06T07:22:35.363283+0000","last_active":"2024-03-06T07:22:37.353188+0000","last_peered":"2024-03-06T07:22:37.353188+0000","last_clean":"2024-03-06T07:22:37.353188+0000","last_became_active":"2024-03-06T07:22:35.363028+0000","last_became_peered":"2024-03-06T07:22:35.363028+0000","last_unstale":"2024-03-06T07:22:37.353188+0000","last_undegraded":"2024-03-06T07:22:37.353188+0000","last_fullsized":"2024-03-06T07:22:37.353188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369812+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:35.369812+0000","last_peered":"2024-03-06T07:22:35.369812+0000","last_clean":"2024-03-06T07:22:35.369812+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:35.369812+0000","last_undegraded":"2024-03-06T07:22:35.369812+0000","last_fullsized":"2024-03-06T07:22:35.369812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373335+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:35.373335+0000","last_peered":"2024-03-06T07:22:35.373335+0000","last_clean":"2024-03-06T07:22:35.373335+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:35.373335+0000","last_undegraded":"2024-03-06T07:22:35.373335+0000","last_fullsized":"2024-03-06T07:22:35.373335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:21.055978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294701+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:37.294701+0000","last_peered":"2024-03-06T07:22:37.294701+0000","last_clean":"2024-03-06T07:22:37.294701+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:37.294701+0000","last_undegraded":"2024-03-06T07:22:37.294701+0000","last_fullsized":"2024-03-06T07:22:37.294701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365453+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:36.365453+0000","last_peered":"2024-03-06T07:22:36.365453+0000","last_clean":"2024-03-06T07:22:36.365453+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:36.365453+0000","last_undegraded":"2024-03-06T07:22:36.365453+0000","last_fullsized":"2024-03-06T07:22:36.365453+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373881+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:35.373881+0000","last_peered":"2024-03-06T07:22:35.373881+0000","last_clean":"2024-03-06T07:22:35.373881+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:35.373881+0000","last_undegraded":"2024-03-06T07:22:35.373881+0000","last_fullsized":"2024-03-06T07:22:35.373881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294583+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:37.294583+0000","last_peered":"2024-03-06T07:22:37.294583+0000","last_clean":"2024-03-06T07:22:37.294583+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:37.294583+0000","last_undegraded":"2024-03-06T07:22:37.294583+0000","last_fullsized":"2024-03-06T07:22:37.294583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366870+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:36.366870+0000","last_peered":"2024-03-06T07:22:36.366870+0000","last_clean":"2024-03-06T07:22:36.366870+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:36.366870+0000","last_undegraded":"2024-03-06T07:22:36.366870+0000","last_fullsized":"2024-03-06T07:22:36.366870+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.374279+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:35.374279+0000","last_peered":"2024-03-06T07:22:35.374279+0000","last_clean":"2024-03-06T07:22:35.374279+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:35.374279+0000","last_undegraded":"2024-03-06T07:22:35.374279+0000","last_fullsized":"2024-03-06T07:22:35.374279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:38.172555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353236+0000","last_change":"2024-03-06T07:22:35.365945+0000","last_active":"2024-03-06T07:22:37.353236+0000","last_peered":"2024-03-06T07:22:37.353236+0000","last_clean":"2024-03-06T07:22:37.353236+0000","last_became_active":"2024-03-06T07:22:35.365677+0000","last_became_peered":"2024-03-06T07:22:35.365677+0000","last_unstale":"2024-03-06T07:22:37.353236+0000","last_undegraded":"2024-03-06T07:22:37.353236+0000","last_fullsized":"2024-03-06T07:22:37.353236+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373392+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:35.373392+0000","last_peered":"2024-03-06T07:22:35.373392+0000","last_clean":"2024-03-06T07:22:35.373392+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:35.373392+0000","last_undegraded":"2024-03-06T07:22:35.373392+0000","last_fullsized":"2024-03-06T07:22:35.373392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:14.459848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365552+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:36.365552+0000","last_peered":"2024-03-06T07:22:36.365552+0000","last_clean":"2024-03-06T07:22:36.365552+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:36.365552+0000","last_undegraded":"2024-03-06T07:22:36.365552+0000","last_fullsized":"2024-03-06T07:22:36.365552+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352693+0000","last_change":"2024-03-06T07:22:35.366910+0000","last_active":"2024-03-06T07:22:37.352693+0000","last_peered":"2024-03-06T07:22:37.352693+0000","last_clean":"2024-03-06T07:22:37.352693+0000","last_became_active":"2024-03-06T07:22:35.366623+0000","last_became_peered":"2024-03-06T07:22:35.366623+0000","last_unstale":"2024-03-06T07:22:37.352693+0000","last_undegraded":"2024-03-06T07:22:37.352693+0000","last_fullsized":"2024-03-06T07:22:37.352693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366692+0000","last_change":"2024-03-06T07:22:35.360892+0000","last_active":"2024-03-06T07:22:36.366692+0000","last_peered":"2024-03-06T07:22:36.366692+0000","last_clean":"2024-03-06T07:22:36.366692+0000","last_became_active":"2024-03-06T07:22:35.360615+0000","last_became_peered":"2024-03-06T07:22:35.360615+0000","last_unstale":"2024-03-06T07:22:36.366692+0000","last_undegraded":"2024-03-06T07:22:36.366692+0000","last_fullsized":"2024-03-06T07:22:36.366692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:15.740118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295121+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:37.295121+0000","last_peered":"2024-03-06T07:22:37.295121+0000","last_clean":"2024-03-06T07:22:37.295121+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:37.295121+0000","last_undegraded":"2024-03-06T07:22:37.295121+0000","last_fullsized":"2024-03-06T07:22:37.295121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366750+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:35.366750+0000","last_peered":"2024-03-06T07:22:35.366750+0000","last_clean":"2024-03-06T07:22:35.366750+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:35.366750+0000","last_undegraded":"2024-03-06T07:22:35.366750+0000","last_fullsized":"2024-03-06T07:22:35.366750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365181+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:36.365181+0000","last_peered":"2024-03-06T07:22:36.365181+0000","last_clean":"2024-03-06T07:22:36.365181+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:36.365181+0000","last_undegraded":"2024-03-06T07:22:36.365181+0000","last_fullsized":"2024-03-06T07:22:36.365181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294894+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:37.294894+0000","last_peered":"2024-03-06T07:22:37.294894+0000","last_clean":"2024-03-06T07:22:37.294894+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:37.294894+0000","last_undegraded":"2024-03-06T07:22:37.294894+0000","last_fullsized":"2024-03-06T07:22:37.294894+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366477+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:36.366477+0000","last_peered":"2024-03-06T07:22:36.366477+0000","last_clean":"2024-03-06T07:22:36.366477+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:36.366477+0000","last_undegraded":"2024-03-06T07:22:36.366477+0000","last_fullsized":"2024-03-06T07:22:36.366477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294614+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:37.294614+0000","last_peered":"2024-03-06T07:22:37.294614+0000","last_clean":"2024-03-06T07:22:37.294614+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:37.294614+0000","last_undegraded":"2024-03-06T07:22:37.294614+0000","last_fullsized":"2024-03-06T07:22:37.294614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366253+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:36.366253+0000","last_peered":"2024-03-06T07:22:36.366253+0000","last_clean":"2024-03-06T07:22:36.366253+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:36.366253+0000","last_undegraded":"2024-03-06T07:22:36.366253+0000","last_fullsized":"2024-03-06T07:22:36.366253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352681+0000","last_change":"2024-03-06T07:22:35.367093+0000","last_active":"2024-03-06T07:22:37.352681+0000","last_peered":"2024-03-06T07:22:37.352681+0000","last_clean":"2024-03-06T07:22:37.352681+0000","last_became_active":"2024-03-06T07:22:35.366829+0000","last_became_peered":"2024-03-06T07:22:35.366829+0000","last_unstale":"2024-03-06T07:22:37.352681+0000","last_undegraded":"2024-03-06T07:22:37.352681+0000","last_fullsized":"2024-03-06T07:22:37.352681+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353121+0000","last_change":"2024-03-06T07:22:35.367432+0000","last_active":"2024-03-06T07:22:37.353121+0000","last_peered":"2024-03-06T07:22:37.353121+0000","last_clean":"2024-03-06T07:22:37.353121+0000","last_became_active":"2024-03-06T07:22:35.367164+0000","last_became_peered":"2024-03-06T07:22:35.367164+0000","last_unstale":"2024-03-06T07:22:37.353121+0000","last_undegraded":"2024-03-06T07:22:37.353121+0000","last_fullsized":"2024-03-06T07:22:37.353121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376691+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:35.376691+0000","last_peered":"2024-03-06T07:22:35.376691+0000","last_clean":"2024-03-06T07:22:35.376691+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:35.376691+0000","last_undegraded":"2024-03-06T07:22:35.376691+0000","last_fullsized":"2024-03-06T07:22:35.376691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:05.191375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367571+0000","last_change":"2024-03-06T07:22:35.373963+0000","last_active":"2024-03-06T07:22:36.367571+0000","last_peered":"2024-03-06T07:22:36.367571+0000","last_clean":"2024-03-06T07:22:36.367571+0000","last_became_active":"2024-03-06T07:22:35.373704+0000","last_became_peered":"2024-03-06T07:22:35.373704+0000","last_unstale":"2024-03-06T07:22:36.367571+0000","last_undegraded":"2024-03-06T07:22:36.367571+0000","last_fullsized":"2024-03-06T07:22:36.367571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295009+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:37.295009+0000","last_peered":"2024-03-06T07:22:37.295009+0000","last_clean":"2024-03-06T07:22:37.295009+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:37.295009+0000","last_undegraded":"2024-03-06T07:22:37.295009+0000","last_fullsized":"2024-03-06T07:22:37.295009+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352994+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:37.352994+0000","last_peered":"2024-03-06T07:22:37.352994+0000","last_clean":"2024-03-06T07:22:37.352994+0000","last_became_active":"2024-03-06T07:22:35.361468+0000","last_became_peered":"2024-03-06T07:22:35.361468+0000","last_unstale":"2024-03-06T07:22:37.352994+0000","last_undegraded":"2024-03-06T07:22:37.352994+0000","last_fullsized":"2024-03-06T07:22:37.352994+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:50.366055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366836+0000","last_change":"2024-03-06T07:22:35.372500+0000","last_active":"2024-03-06T07:22:36.366836+0000","last_peered":"2024-03-06T07:22:36.366836+0000","last_clean":"2024-03-06T07:22:36.366836+0000","last_became_active":"2024-03-06T07:22:35.372221+0000","last_became_peered":"2024-03-06T07:22:35.372221+0000","last_unstale":"2024-03-06T07:22:36.366836+0000","last_undegraded":"2024-03-06T07:22:36.366836+0000","last_fullsized":"2024-03-06T07:22:36.366836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:23.442086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369741+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:35.369741+0000","last_peered":"2024-03-06T07:22:35.369741+0000","last_clean":"2024-03-06T07:22:35.369741+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:35.369741+0000","last_undegraded":"2024-03-06T07:22:35.369741+0000","last_fullsized":"2024-03-06T07:22:35.369741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366862+0000","last_change":"2024-03-06T07:22:35.363020+0000","last_active":"2024-03-06T07:22:36.366862+0000","last_peered":"2024-03-06T07:22:36.366862+0000","last_clean":"2024-03-06T07:22:36.366862+0000","last_became_active":"2024-03-06T07:22:35.362767+0000","last_became_peered":"2024-03-06T07:22:35.362767+0000","last_unstale":"2024-03-06T07:22:36.366862+0000","last_undegraded":"2024-03-06T07:22:36.366862+0000","last_fullsized":"2024-03-06T07:22:36.366862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:56.148980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.360317+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:35.360317+0000","last_peered":"2024-03-06T07:22:35.360317+0000","last_clean":"2024-03-06T07:22:35.360317+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:35.360317+0000","last_undegraded":"2024-03-06T07:22:35.360317+0000","last_fullsized":"2024-03-06T07:22:35.360317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295198+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:37.295198+0000","last_peered":"2024-03-06T07:22:37.295198+0000","last_clean":"2024-03-06T07:22:37.295198+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:37.295198+0000","last_undegraded":"2024-03-06T07:22:37.295198+0000","last_fullsized":"2024-03-06T07:22:37.295198+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352699+0000","last_change":"2024-03-06T07:22:35.368816+0000","last_active":"2024-03-06T07:22:37.352699+0000","last_peered":"2024-03-06T07:22:37.352699+0000","last_clean":"2024-03-06T07:22:37.352699+0000","last_became_active":"2024-03-06T07:22:35.368555+0000","last_became_peered":"2024-03-06T07:22:35.368555+0000","last_unstale":"2024-03-06T07:22:37.352699+0000","last_undegraded":"2024-03-06T07:22:37.352699+0000","last_fullsized":"2024-03-06T07:22:37.352699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367192+0000","last_change":"2024-03-06T07:22:34.366335+0000","last_active":"2024-03-06T07:22:36.367192+0000","last_peered":"2024-03-06T07:22:36.367192+0000","last_clean":"2024-03-06T07:22:36.367192+0000","last_became_active":"2024-03-06T07:22:34.364387+0000","last_became_peered":"2024-03-06T07:22:34.364387+0000","last_unstale":"2024-03-06T07:22:36.367192+0000","last_undegraded":"2024-03-06T07:22:36.367192+0000","last_fullsized":"2024-03-06T07:22:36.367192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366415+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:36.366415+0000","last_peered":"2024-03-06T07:22:36.366415+0000","last_clean":"2024-03-06T07:22:36.366415+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:36.366415+0000","last_undegraded":"2024-03-06T07:22:36.366415+0000","last_fullsized":"2024-03-06T07:22:36.366415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367686+0000","last_change":"2024-03-06T07:22:34.364533+0000","last_active":"2024-03-06T07:22:36.367686+0000","last_peered":"2024-03-06T07:22:36.367686+0000","last_clean":"2024-03-06T07:22:36.367686+0000","last_became_active":"2024-03-06T07:22:34.364224+0000","last_became_peered":"2024-03-06T07:22:34.364224+0000","last_unstale":"2024-03-06T07:22:36.367686+0000","last_undegraded":"2024-03-06T07:22:36.367686+0000","last_fullsized":"2024-03-06T07:22:36.367686+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:13.454983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367050+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:36.367050+0000","last_peered":"2024-03-06T07:22:36.367050+0000","last_clean":"2024-03-06T07:22:36.367050+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:36.367050+0000","last_undegraded":"2024-03-06T07:22:36.367050+0000","last_fullsized":"2024-03-06T07:22:36.367050+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373794+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:35.373794+0000","last_peered":"2024-03-06T07:22:35.373794+0000","last_clean":"2024-03-06T07:22:35.373794+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:35.373794+0000","last_undegraded":"2024-03-06T07:22:35.373794+0000","last_fullsized":"2024-03-06T07:22:35.373794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353178+0000","last_change":"2024-03-06T07:22:35.365350+0000","last_active":"2024-03-06T07:22:37.353178+0000","last_peered":"2024-03-06T07:22:37.353178+0000","last_clean":"2024-03-06T07:22:37.353178+0000","last_became_active":"2024-03-06T07:22:35.365049+0000","last_became_peered":"2024-03-06T07:22:35.365049+0000","last_unstale":"2024-03-06T07:22:37.353178+0000","last_undegraded":"2024-03-06T07:22:37.353178+0000","last_fullsized":"2024-03-06T07:22:37.353178+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367160+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:36.367160+0000","last_peered":"2024-03-06T07:22:36.367160+0000","last_clean":"2024-03-06T07:22:36.367160+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:36.367160+0000","last_undegraded":"2024-03-06T07:22:36.367160+0000","last_fullsized":"2024-03-06T07:22:36.367160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353342+0000","last_change":"2024-03-06T07:22:35.363944+0000","last_active":"2024-03-06T07:22:37.353342+0000","last_peered":"2024-03-06T07:22:37.353342+0000","last_clean":"2024-03-06T07:22:37.353342+0000","last_became_active":"2024-03-06T07:22:35.363662+0000","last_became_peered":"2024-03-06T07:22:35.363662+0000","last_unstale":"2024-03-06T07:22:37.353342+0000","last_undegraded":"2024-03-06T07:22:37.353342+0000","last_fullsized":"2024-03-06T07:22:37.353342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:17.265969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372297+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:35.372297+0000","last_peered":"2024-03-06T07:22:35.372297+0000","last_clean":"2024-03-06T07:22:35.372297+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:35.372297+0000","last_undegraded":"2024-03-06T07:22:35.372297+0000","last_fullsized":"2024-03-06T07:22:35.372297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367496+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:36.367496+0000","last_peered":"2024-03-06T07:22:36.367496+0000","last_clean":"2024-03-06T07:22:36.367496+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:36.367496+0000","last_undegraded":"2024-03-06T07:22:36.367496+0000","last_fullsized":"2024-03-06T07:22:36.367496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352852+0000","last_change":"2024-03-06T07:22:34.366433+0000","last_active":"2024-03-06T07:22:37.352852+0000","last_peered":"2024-03-06T07:22:37.352852+0000","last_clean":"2024-03-06T07:22:37.352852+0000","last_became_active":"2024-03-06T07:22:34.365956+0000","last_became_peered":"2024-03-06T07:22:34.365956+0000","last_unstale":"2024-03-06T07:22:37.352852+0000","last_undegraded":"2024-03-06T07:22:37.352852+0000","last_fullsized":"2024-03-06T07:22:37.352852+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376711+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:35.376711+0000","last_peered":"2024-03-06T07:22:35.376711+0000","last_clean":"2024-03-06T07:22:35.376711+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:35.376711+0000","last_undegraded":"2024-03-06T07:22:35.376711+0000","last_fullsized":"2024-03-06T07:22:35.376711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294626+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:37.294626+0000","last_peered":"2024-03-06T07:22:37.294626+0000","last_clean":"2024-03-06T07:22:37.294626+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:37.294626+0000","last_undegraded":"2024-03-06T07:22:37.294626+0000","last_fullsized":"2024-03-06T07:22:37.294626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352824+0000","last_change":"2024-03-06T07:22:35.365470+0000","last_active":"2024-03-06T07:22:37.352824+0000","last_peered":"2024-03-06T07:22:37.352824+0000","last_clean":"2024-03-06T07:22:37.352824+0000","last_became_active":"2024-03-06T07:22:35.365156+0000","last_became_peered":"2024-03-06T07:22:35.365156+0000","last_unstale":"2024-03-06T07:22:37.352824+0000","last_undegraded":"2024-03-06T07:22:37.352824+0000","last_fullsized":"2024-03-06T07:22:37.352824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:45.193188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.366420+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.365577+0000","last_became_peered":"2024-03-06T07:22:34.365577+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:20.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372836+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:35.372836+0000","last_peered":"2024-03-06T07:22:35.372836+0000","last_clean":"2024-03-06T07:22:35.372836+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:35.372836+0000","last_undegraded":"2024-03-06T07:22:35.372836+0000","last_fullsized":"2024-03-06T07:22:35.372836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:52.932700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366830+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:35.366830+0000","last_peered":"2024-03-06T07:22:35.366830+0000","last_clean":"2024-03-06T07:22:35.366830+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:35.366830+0000","last_undegraded":"2024-03-06T07:22:35.366830+0000","last_fullsized":"2024-03-06T07:22:35.366830+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365569+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:36.365569+0000","last_peered":"2024-03-06T07:22:36.365569+0000","last_clean":"2024-03-06T07:22:36.365569+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:36.365569+0000","last_undegraded":"2024-03-06T07:22:36.365569+0000","last_fullsized":"2024-03-06T07:22:36.365569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295075+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:37.295075+0000","last_peered":"2024-03-06T07:22:37.295075+0000","last_clean":"2024-03-06T07:22:37.295075+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:37.295075+0000","last_undegraded":"2024-03-06T07:22:37.295075+0000","last_fullsized":"2024-03-06T07:22:37.295075+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370865+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:35.370865+0000","last_peered":"2024-03-06T07:22:35.370865+0000","last_clean":"2024-03-06T07:22:35.370865+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:35.370865+0000","last_undegraded":"2024-03-06T07:22:35.370865+0000","last_fullsized":"2024-03-06T07:22:35.370865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.379447+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:35.379447+0000","last_peered":"2024-03-06T07:22:35.379447+0000","last_clean":"2024-03-06T07:22:35.379447+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:35.379447+0000","last_undegraded":"2024-03-06T07:22:35.379447+0000","last_fullsized":"2024-03-06T07:22:35.379447+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372879+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:35.372879+0000","last_peered":"2024-03-06T07:22:35.372879+0000","last_clean":"2024-03-06T07:22:35.372879+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:35.372879+0000","last_undegraded":"2024-03-06T07:22:35.372879+0000","last_fullsized":"2024-03-06T07:22:35.372879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:54.478837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.376739+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:35.376739+0000","last_peered":"2024-03-06T07:22:35.376739+0000","last_clean":"2024-03-06T07:22:35.376739+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:35.376739+0000","last_undegraded":"2024-03-06T07:22:35.376739+0000","last_fullsized":"2024-03-06T07:22:35.376739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372010+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:35.372010+0000","last_peered":"2024-03-06T07:22:35.372010+0000","last_clean":"2024-03-06T07:22:35.372010+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:35.372010+0000","last_undegraded":"2024-03-06T07:22:35.372010+0000","last_fullsized":"2024-03-06T07:22:35.372010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:35.823047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366957+0000","last_change":"2024-03-06T07:22:34.364874+0000","last_active":"2024-03-06T07:22:36.366957+0000","last_peered":"2024-03-06T07:22:36.366957+0000","last_clean":"2024-03-06T07:22:36.366957+0000","last_became_active":"2024-03-06T07:22:34.364472+0000","last_became_peered":"2024-03-06T07:22:34.364472+0000","last_unstale":"2024-03-06T07:22:36.366957+0000","last_undegraded":"2024-03-06T07:22:36.366957+0000","last_fullsized":"2024-03-06T07:22:36.366957+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.364306+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:35.364306+0000","last_peered":"2024-03-06T07:22:35.364306+0000","last_clean":"2024-03-06T07:22:35.364306+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:35.364306+0000","last_undegraded":"2024-03-06T07:22:35.364306+0000","last_fullsized":"2024-03-06T07:22:35.364306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:46.770166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294650+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:37.294650+0000","last_peered":"2024-03-06T07:22:37.294650+0000","last_clean":"2024-03-06T07:22:37.294650+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:37.294650+0000","last_undegraded":"2024-03-06T07:22:37.294650+0000","last_fullsized":"2024-03-06T07:22:37.294650+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.372602+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:35.372602+0000","last_peered":"2024-03-06T07:22:35.372602+0000","last_clean":"2024-03-06T07:22:35.372602+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:35.372602+0000","last_undegraded":"2024-03-06T07:22:35.372602+0000","last_fullsized":"2024-03-06T07:22:35.372602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352812+0000","last_change":"2024-03-06T07:22:34.361474+0000","last_active":"2024-03-06T07:22:37.352812+0000","last_peered":"2024-03-06T07:22:37.352812+0000","last_clean":"2024-03-06T07:22:37.352812+0000","last_became_active":"2024-03-06T07:22:34.360526+0000","last_became_peered":"2024-03-06T07:22:34.360526+0000","last_unstale":"2024-03-06T07:22:37.352812+0000","last_undegraded":"2024-03-06T07:22:37.352812+0000","last_fullsized":"2024-03-06T07:22:37.352812+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.364939+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:36.364939+0000","last_peered":"2024-03-06T07:22:36.364939+0000","last_clean":"2024-03-06T07:22:36.364939+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:36.364939+0000","last_undegraded":"2024-03-06T07:22:36.364939+0000","last_fullsized":"2024-03-06T07:22:36.364939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367712+0000","last_change":"2024-03-06T07:22:34.365404+0000","last_active":"2024-03-06T07:22:36.367712+0000","last_peered":"2024-03-06T07:22:36.367712+0000","last_clean":"2024-03-06T07:22:36.367712+0000","last_became_active":"2024-03-06T07:22:34.365029+0000","last_became_peered":"2024-03-06T07:22:34.365029+0000","last_unstale":"2024-03-06T07:22:36.367712+0000","last_undegraded":"2024-03-06T07:22:36.367712+0000","last_fullsized":"2024-03-06T07:22:36.367712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.473973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294660+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:37.294660+0000","last_peered":"2024-03-06T07:22:37.294660+0000","last_clean":"2024-03-06T07:22:37.294660+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:37.294660+0000","last_undegraded":"2024-03-06T07:22:37.294660+0000","last_fullsized":"2024-03-06T07:22:37.294660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367602+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:36.367602+0000","last_peered":"2024-03-06T07:22:36.367602+0000","last_clean":"2024-03-06T07:22:36.367602+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:36.367602+0000","last_undegraded":"2024-03-06T07:22:36.367602+0000","last_fullsized":"2024-03-06T07:22:36.367602+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294702+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:37.294702+0000","last_peered":"2024-03-06T07:22:37.294702+0000","last_clean":"2024-03-06T07:22:37.294702+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:37.294702+0000","last_undegraded":"2024-03-06T07:22:37.294702+0000","last_fullsized":"2024-03-06T07:22:37.294702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.295156+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:37.295156+0000","last_peered":"2024-03-06T07:22:37.295156+0000","last_clean":"2024-03-06T07:22:37.295156+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:37.295156+0000","last_undegraded":"2024-03-06T07:22:37.295156+0000","last_fullsized":"2024-03-06T07:22:37.295156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365086+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:36.365086+0000","last_peered":"2024-03-06T07:22:36.365086+0000","last_clean":"2024-03-06T07:22:36.365086+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:36.365086+0000","last_undegraded":"2024-03-06T07:22:36.365086+0000","last_fullsized":"2024-03-06T07:22:36.365086+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351435+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:35.351435+0000","last_peered":"2024-03-06T07:22:35.351435+0000","last_clean":"2024-03-06T07:22:35.351435+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:35.351435+0000","last_undegraded":"2024-03-06T07:22:35.351435+0000","last_fullsized":"2024-03-06T07:22:35.351435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353248+0000","last_change":"2024-03-06T07:22:35.370112+0000","last_active":"2024-03-06T07:22:37.353248+0000","last_peered":"2024-03-06T07:22:37.353248+0000","last_clean":"2024-03-06T07:22:37.353248+0000","last_became_active":"2024-03-06T07:22:35.369845+0000","last_became_peered":"2024-03-06T07:22:35.369845+0000","last_unstale":"2024-03-06T07:22:37.353248+0000","last_undegraded":"2024-03-06T07:22:37.353248+0000","last_fullsized":"2024-03-06T07:22:37.353248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365194+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:36.365194+0000","last_peered":"2024-03-06T07:22:36.365194+0000","last_clean":"2024-03-06T07:22:36.365194+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:36.365194+0000","last_undegraded":"2024-03-06T07:22:36.365194+0000","last_fullsized":"2024-03-06T07:22:36.365194+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.353270+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:37.353270+0000","last_peered":"2024-03-06T07:22:37.353270+0000","last_clean":"2024-03-06T07:22:37.353270+0000","last_became_active":"2024-03-06T07:22:35.361332+0000","last_became_peered":"2024-03-06T07:22:35.361332+0000","last_unstale":"2024-03-06T07:22:37.353270+0000","last_undegraded":"2024-03-06T07:22:37.353270+0000","last_fullsized":"2024-03-06T07:22:37.353270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373686+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:35.373686+0000","last_peered":"2024-03-06T07:22:35.373686+0000","last_clean":"2024-03-06T07:22:35.373686+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:35.373686+0000","last_undegraded":"2024-03-06T07:22:35.373686+0000","last_fullsized":"2024-03-06T07:22:35.373686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365013+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:36.365013+0000","last_peered":"2024-03-06T07:22:36.365013+0000","last_clean":"2024-03-06T07:22:36.365013+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:36.365013+0000","last_undegraded":"2024-03-06T07:22:36.365013+0000","last_fullsized":"2024-03-06T07:22:36.365013+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366256+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:36.366256+0000","last_peered":"2024-03-06T07:22:36.366256+0000","last_clean":"2024-03-06T07:22:36.366256+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:36.366256+0000","last_undegraded":"2024-03-06T07:22:36.366256+0000","last_fullsized":"2024-03-06T07:22:36.366256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371730+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:35.371730+0000","last_peered":"2024-03-06T07:22:35.371730+0000","last_clean":"2024-03-06T07:22:35.371730+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:35.371730+0000","last_undegraded":"2024-03-06T07:22:35.371730+0000","last_fullsized":"2024-03-06T07:22:35.371730+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.366022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:35.366022+0000","last_peered":"2024-03-06T07:22:35.366022+0000","last_clean":"2024-03-06T07:22:35.366022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:35.366022+0000","last_undegraded":"2024-03-06T07:22:35.366022+0000","last_fullsized":"2024-03-06T07:22:35.366022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:02.842916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.354591+0000","last_change":"2024-03-06T07:22:34.362718+0000","last_active":"2024-03-06T07:22:37.354591+0000","last_peered":"2024-03-06T07:22:37.354591+0000","last_clean":"2024-03-06T07:22:37.354591+0000","last_became_active":"2024-03-06T07:22:34.362448+0000","last_became_peered":"2024-03-06T07:22:34.362448+0000","last_unstale":"2024-03-06T07:22:37.354591+0000","last_undegraded":"2024-03-06T07:22:37.354591+0000","last_fullsized":"2024-03-06T07:22:37.354591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366562+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:36.366562+0000","last_peered":"2024-03-06T07:22:36.366562+0000","last_clean":"2024-03-06T07:22:36.366562+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:36.366562+0000","last_undegraded":"2024-03-06T07:22:36.366562+0000","last_fullsized":"2024-03-06T07:22:36.366562+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367833+0000","last_change":"2024-03-06T07:22:34.365631+0000","last_active":"2024-03-06T07:22:36.367833+0000","last_peered":"2024-03-06T07:22:36.367833+0000","last_clean":"2024-03-06T07:22:36.367833+0000","last_became_active":"2024-03-06T07:22:34.364558+0000","last_became_peered":"2024-03-06T07:22:34.364558+0000","last_unstale":"2024-03-06T07:22:36.367833+0000","last_undegraded":"2024-03-06T07:22:36.367833+0000","last_fullsized":"2024-03-06T07:22:36.367833+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:37.167691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.352939+0000","last_change":"2024-03-06T07:22:35.362565+0000","last_active":"2024-03-06T07:22:37.352939+0000","last_peered":"2024-03-06T07:22:37.352939+0000","last_clean":"2024-03-06T07:22:37.352939+0000","last_became_active":"2024-03-06T07:22:35.362296+0000","last_became_peered":"2024-03-06T07:22:35.362296+0000","last_unstale":"2024-03-06T07:22:37.352939+0000","last_undegraded":"2024-03-06T07:22:37.352939+0000","last_fullsized":"2024-03-06T07:22:37.352939+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:19.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367067+0000","last_change":"2024-03-06T07:22:34.360939+0000","last_active":"2024-03-06T07:22:36.367067+0000","last_peered":"2024-03-06T07:22:36.367067+0000","last_clean":"2024-03-06T07:22:36.367067+0000","last_became_active":"2024-03-06T07:22:34.360665+0000","last_became_peered":"2024-03-06T07:22:34.360665+0000","last_unstale":"2024-03-06T07:22:36.367067+0000","last_undegraded":"2024-03-06T07:22:36.367067+0000","last_fullsized":"2024-03-06T07:22:36.367067+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.370130+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:35.370130+0000","last_peered":"2024-03-06T07:22:35.370130+0000","last_clean":"2024-03-06T07:22:35.370130+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:35.370130+0000","last_undegraded":"2024-03-06T07:22:35.370130+0000","last_fullsized":"2024-03-06T07:22:35.370130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:16.157971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.352766+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:35.352766+0000","last_peered":"2024-03-06T07:22:35.352766+0000","last_clean":"2024-03-06T07:22:35.352766+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:35.352766+0000","last_undegraded":"2024-03-06T07:22:35.352766+0000","last_fullsized":"2024-03-06T07:22:35.352766+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.373867+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:35.373867+0000","last_peered":"2024-03-06T07:22:35.373867+0000","last_clean":"2024-03-06T07:22:35.373867+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:35.373867+0000","last_undegraded":"2024-03-06T07:22:35.373867+0000","last_fullsized":"2024-03-06T07:22:35.373867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:37.294742+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:37.294742+0000","last_peered":"2024-03-06T07:22:37.294742+0000","last_clean":"2024-03-06T07:22:37.294742+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:37.294742+0000","last_undegraded":"2024-03-06T07:22:37.294742+0000","last_fullsized":"2024-03-06T07:22:37.294742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.369774+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:35.369774+0000","last_peered":"2024-03-06T07:22:35.369774+0000","last_clean":"2024-03-06T07:22:35.369774+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:35.369774+0000","last_undegraded":"2024-03-06T07:22:35.369774+0000","last_fullsized":"2024-03-06T07:22:35.369774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.353050+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:35.353050+0000","last_peered":"2024-03-06T07:22:35.353050+0000","last_clean":"2024-03-06T07:22:35.353050+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:35.353050+0000","last_undegraded":"2024-03-06T07:22:35.353050+0000","last_fullsized":"2024-03-06T07:22:35.353050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.371951+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:35.371951+0000","last_peered":"2024-03-06T07:22:35.371951+0000","last_clean":"2024-03-06T07:22:35.371951+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:35.371951+0000","last_undegraded":"2024-03-06T07:22:35.371951+0000","last_fullsized":"2024-03-06T07:22:35.371951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-03-06T07:22:35.351080+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:35.351080+0000","last_peered":"2024-03-06T07:22:35.351080+0000","last_clean":"2024-03-06T07:22:35.351080+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:35.351080+0000","last_undegraded":"2024-03-06T07:22:35.351080+0000","last_fullsized":"2024-03-06T07:22:35.351080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.367852+0000","last_change":"2024-03-06T07:22:35.373140+0000","last_active":"2024-03-06T07:22:36.367852+0000","last_peered":"2024-03-06T07:22:36.367852+0000","last_clean":"2024-03-06T07:22:36.367852+0000","last_became_active":"2024-03-06T07:22:35.372523+0000","last_became_peered":"2024-03-06T07:22:35.372523+0000","last_unstale":"2024-03-06T07:22:36.367852+0000","last_undegraded":"2024-03-06T07:22:36.367852+0000","last_fullsized":"2024-03-06T07:22:36.367852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.365701+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:36.365701+0000","last_peered":"2024-03-06T07:22:36.365701+0000","last_clean":"2024-03-06T07:22:36.365701+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:36.365701+0000","last_undegraded":"2024-03-06T07:22:36.365701+0000","last_fullsized":"2024-03-06T07:22:36.365701+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-06T07:22:36.366754+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:36.366754+0000","last_peered":"2024-03-06T07:22:36.366754+0000","last_clean":"2024-03-06T07:22:36.366754+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:36.366754+0000","last_undegraded":"2024-03-06T07:22:36.366754+0000","last_fullsized":"2024-03-06T07:22:36.366754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":987136,"data_stored":1026620,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":431150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":429572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":937984,"data_stored":437946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":436812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":430280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":913408,"data_stored":429650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":1019906,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:43.126 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:43.126 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:22:43.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.269+0000 7ff8e359a640 1 Processor -- start 2024-03-06T07:22:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.269+0000 7ff8e359a640 1 -- start start 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e359a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff8dc072cd0 0x7ff8dc155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e359a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc152db0 con 0x7ff8dc1529d0 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e359a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc1590f0 con 0x7ff8dc073290 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e359a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c6dd0 con 0x7ff8dc072cd0 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e1336640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff8dc1529d0 0x7ff8dc0c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39606/0 (socket says 172.21.15.17:39606) 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e1336640 1 -- 172.21.15.17:0/1919461036 learned_addr learned my addr 172.21.15.17:0/1919461036 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e0b35640 1 --2- 172.21.15.17:0/1919461036 >> v2:172.21.15.163:3300/0 conn(0x7ff8dc072cd0 0x7ff8dc155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 394085018 0 0) 0x7ff8d8001110 con 0x7ff8dc1529d0 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c8003520 con 0x7ff8dc1529d0 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006640387 0 0) 0x7ff8d4001490 con 0x7ff8dc073290 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d8001110 con 0x7ff8dc073290 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e0b35640 1 -- 172.21.15.17:0/1919461036 >> v1:172.21.15.163:6789/0 conn(0x7ff8dc073290 legacy=0x7ff8dc155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e0b35640 1 -- 172.21.15.17:0/1919461036 >> v1:172.21.15.17:6789/0 conn(0x7ff8dc1529d0 legacy=0x7ff8dc0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.270+0000 7ff8e0b35640 1 -- 172.21.15.17:0/1919461036 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc159650 con 0x7ff8dc072cd0 2024-03-06T07:22:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.271+0000 7ff8e0b35640 1 --2- 172.21.15.17:0/1919461036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc155120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c0c9cd8eff3b6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.271+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0315a0 con 0x7ff8dc072cd0 2024-03-06T07:22:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.271+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc037070 con 0x7ff8dc072cd0 2024-03-06T07:22:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.271+0000 7ff8d3fff640 1 -- 172.21.15.17:0/1919461036 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc031960 con 0x7ff8dc072cd0 2024-03-06T07:22:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 -- 172.21.15.17:0/1919461036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 msgr2=0x7ff8dc155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 --2- 172.21.15.17:0/1919461036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc155120 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0097a0 tx=0x7ff8cc02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 -- 172.21.15.17:0/1919461036 shutdown_connections 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 --2- 172.21.15.17:0/1919461036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc155120 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 -- 172.21.15.17:0/1919461036 >> 172.21.15.17:0/1919461036 conn(0x7ff8dc0aa630 msgr2=0x7ff8dc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 -- 172.21.15.17:0/1919461036 shutdown_connections 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.272+0000 7ff8e359a640 1 -- 172.21.15.17:0/1919461036 wait complete. 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.273+0000 7ff8e359a640 1 Processor -- start 2024-03-06T07:22:43.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.273+0000 7ff8e359a640 1 -- start start 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc14b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8dc1529d0 0x7ff8dc142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc155d70 con 0x7ff8dc073290 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c6940 con 0x7ff8dc072cd0 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc159370 con 0x7ff8dc1529d0 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e1336640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc14b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40706/0 (socket says 172.21.15.17:40706) 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 -- 172.21.15.17:0/2605998134 learned_addr learned my addr 172.21.15.17:0/2605998134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e1b37640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8dc1529d0 0x7ff8dc142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 -- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8dc1529d0 msgr2=0x7ff8dc142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8dc1529d0 0x7ff8dc142390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 -- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 msgr2=0x7ff8dc14b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc14b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 -- 172.21.15.17:0/2605998134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc1566b0 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e0b35640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc034040 tx=0x7ff8cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc032630 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0327d0 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc06dc60 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.274+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc032bc0 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.275+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc14b9d0 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.276+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc14ca90 con 0x7ff8dc073290 2024-03-06T07:22:43.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.276+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8cc04e430 con 0x7ff8dc073290 2024-03-06T07:22:43.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.276+0000 7ff8d1ffb640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 0x7ff8c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.276+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8cc02f030 con 0x7ff8dc073290 2024-03-06T07:22:43.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.277+0000 7ff8e1336640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 0x7ff8c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.277+0000 7ff8e1336640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 0x7ff8c006d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000f60 tx=0x7ff8d8007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.281+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8dc14ca90 con 0x7ff8dc073290 2024-03-06T07:22:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.533+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff8dc0694c0 con 0x7ff8dc073290 2024-03-06T07:22:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.535+0000 7ff8d1ffb640 1 -- 172.21.15.17:0/2605998134 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff8dc0694c0 con 0x7ff8dc073290 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:22:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 msgr2=0x7ff8c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 0x7ff8c006d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000f60 tx=0x7ff8d8007d70 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 msgr2=0x7ff8dc141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc034040 tx=0x7ff8cc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 shutdown_connections 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff8c006b500 0x7ff8c006d9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff8dc1529d0 0x7ff8dc142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8dc073290 0x7ff8dc141e50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 --2- 172.21.15.17:0/2605998134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff8dc072cd0 0x7ff8dc14b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 >> 172.21.15.17:0/2605998134 conn(0x7ff8dc0aa630 msgr2=0x7ff8dc06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 shutdown_connections 2024-03-06T07:22:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.537+0000 7ff8e359a640 1 -- 172.21.15.17:0/2605998134 wait complete. 2024-03-06T07:22:43.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:43.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.650+0000 7fce0e866640 1 Processor -- start 2024-03-06T07:22:43.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.650+0000 7fce0e866640 1 -- start start 2024-03-06T07:22:43.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce0e866640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce0e866640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce080c3280 0x7fce080c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce0e866640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce08158800 con 0x7fce081551e0 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce0e866640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce08158ae0 con 0x7fce080c3280 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce0e866640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce08158dc0 con 0x7fce0806d9f0 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce080c3280 0x7fce080c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fce080c3280 0x7fce080c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40708/0 (socket says 172.21.15.17:40708) 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 -- 172.21.15.17:0/4269767426 learned_addr learned my addr 172.21.15.17:0/4269767426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce07fff640 1 --2- 172.21.15.17:0/4269767426 >> v2:172.21.15.163:3300/0 conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 -- 172.21.15.17:0/4269767426 >> v2:172.21.15.163:3300/0 conn(0x7fce0806d9f0 msgr2=0x7fce080a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 --2- 172.21.15.17:0/4269767426 >> v2:172.21.15.163:3300/0 conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 -- 172.21.15.17:0/4269767426 >> v1:172.21.15.163:6789/0 conn(0x7fce081551e0 legacy=0x7fce081555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.651+0000 7fce077fe640 1 -- 172.21.15.17:0/4269767426 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce081590a0 con 0x7fce080c3280 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.652+0000 7fce077fe640 1 --2- 172.21.15.17:0/4269767426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080c3280 0x7fce080c3710 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fce080c9820 tx=0x7fcdf802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b62a94b4727a49a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.652+0000 7fce06ffd640 1 -- 172.21.15.17:0/4269767426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8035020 con 0x7fce080c3280 2024-03-06T07:22:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.652+0000 7fce06ffd640 1 -- 172.21.15.17:0/4269767426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf8031870 con 0x7fce080c3280 2024-03-06T07:22:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.652+0000 7fce06ffd640 1 -- 172.21.15.17:0/4269767426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8031c40 con 0x7fce080c3280 2024-03-06T07:22:43.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 -- 172.21.15.17:0/4269767426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080c3280 msgr2=0x7fce080c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 --2- 172.21.15.17:0/4269767426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080c3280 0x7fce080c3710 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fce080c9820 tx=0x7fcdf802b910 comp rx=0 tx=0).stop 2024-03-06T07:22:43.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 -- 172.21.15.17:0/4269767426 shutdown_connections 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 --2- 172.21.15.17:0/4269767426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080c3280 0x7fce080c3710 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 --2- 172.21.15.17:0/4269767426 >> v2:172.21.15.163:3300/0 conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 -- 172.21.15.17:0/4269767426 >> 172.21.15.17:0/4269767426 conn(0x7fce080aa630 msgr2=0x7fce080aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 -- 172.21.15.17:0/4269767426 shutdown_connections 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.653+0000 7fce0e866640 1 -- 172.21.15.17:0/4269767426 wait complete. 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.654+0000 7fce0e866640 1 Processor -- start 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.654+0000 7fce0e866640 1 -- start start 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080c3280 0x7fce08154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0806df50 con 0x7fce081551e0 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0806ddd0 con 0x7fce0806d9f0 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce080a8210 con 0x7fce080c3280 2024-03-06T07:22:43.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40710/0 (socket says 172.21.15.17:40710) 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 -- 172.21.15.17:0/663420295 learned_addr learned my addr 172.21.15.17:0/663420295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce07fff640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce077fe640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080c3280 0x7fce08154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 -- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080c3280 msgr2=0x7fce08154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080c3280 0x7fce08154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 -- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce0806d9f0 msgr2=0x7fce080a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 -- 172.21.15.17:0/663420295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce080a9660 con 0x7fce081551e0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0ce03640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c730 tx=0x7fce0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00003b70 con 0x7fce081551e0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce00003d30 con 0x7fce081551e0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce081eedf0 con 0x7fce081551e0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00005310 con 0x7fce081551e0 2024-03-06T07:22:43.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.655+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce080a9920 con 0x7fce081551e0 2024-03-06T07:22:43.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.657+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0804f330 con 0x7fce081551e0 2024-03-06T07:22:43.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.657+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce000054b0 con 0x7fce081551e0 2024-03-06T07:22:43.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.657+0000 7fce04ff9640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:43.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.657+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce0008a070 con 0x7fce081551e0 2024-03-06T07:22:43.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.657+0000 7fce07fff640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:43.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.658+0000 7fce07fff640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 0x7fcdf406d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009b40 tx=0x7fcdfc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:43.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.662+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce000587a0 con 0x7fce081551e0 2024-03-06T07:22:43.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.879+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce08073240 con 0x7fce081551e0 2024-03-06T07:22:43.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.880+0000 7fce04ff9640 1 -- 172.21.15.17:0/663420295 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4982 (secure 0 0 0) 0x7fce0005c840 con 0x7fce081551e0 2024-03-06T07:22:43.882 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:43.882 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:36.709974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"b","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/1213270757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1213270757},{"type":"v1","addr":"172.21.15.163:6836","nonce":1213270757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-03-06T07:22:43.882 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 56 2024-03-06T07:22:43.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 msgr2=0x7fcdf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 0x7fcdf406d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009b40 tx=0x7fcdfc009190 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 msgr2=0x7fce080aca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c730 tx=0x7fce0000cbf0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 shutdown_connections 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcdf406b4d0 0x7fcdf406d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce081551e0 0x7fce080aca20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080c3280 0x7fce08154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 --2- 172.21.15.17:0/663420295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce0806d9f0 0x7fce080a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.882+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 >> 172.21.15.17:0/663420295 conn(0x7fce080aa630 msgr2=0x7fce080c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.883+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 shutdown_connections 2024-03-06T07:22:43.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:43.883+0000 7fce0e866640 1 -- 172.21.15.17:0/663420295 wait complete. 2024-03-06T07:22:44.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:22:34.715155+0000', 'modified': '2024-03-06T07:22:36.709974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6387': {'gid': 6387, 'name': 'b', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6836/1213270757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1213270757}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1213270757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6387, 'qdb_cluster': [6387]} 2024-03-06T07:22:44.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:22:44.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:22:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.403+0000 7f1f495a0640 1 Processor -- start 2024-03-06T07:22:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.403+0000 7f1f495a0640 1 -- start start 2024-03-06T07:22:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f495a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 0x7f1f440c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f495a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 0x7f1f4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f495a0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f440ac580 con 0x7f1f440c87e0 2024-03-06T07:22:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f495a0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f44156040 con 0x7f1f440bdf70 2024-03-06T07:22:44.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f495a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4406d080 con 0x7f1f441525e0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f48d9f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1f440c87e0 0x7f1f4406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39616/0 (socket says 172.21.15.17:39616) 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f48d9f640 1 -- 172.21.15.17:0/318700022 learned_addr learned my addr 172.21.15.17:0/318700022 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f437fe640 1 --2- 172.21.15.17:0/318700022 >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 0x7f1f4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f43fff640 1 --2- 172.21.15.17:0/318700022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 0x7f1f440c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f42ffd640 1 -- 172.21.15.17:0/318700022 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303513062 0 0) 0x7f1f440ac580 con 0x7f1f440c87e0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f42ffd640 1 -- 172.21.15.17:0/318700022 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f30003520 con 0x7f1f440c87e0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f43fff640 1 -- 172.21.15.17:0/318700022 >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 msgr2=0x7f1f4406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f43fff640 1 --2- 172.21.15.17:0/318700022 >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 0x7f1f4406b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f43fff640 1 -- 172.21.15.17:0/318700022 >> v1:172.21.15.17:6789/0 conn(0x7f1f440c87e0 legacy=0x7f1f4406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f43fff640 1 -- 172.21.15.17:0/318700022 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f44155a80 con 0x7f1f441525e0 2024-03-06T07:22:44.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.404+0000 7f1f437fe640 1 --2- 172.21.15.17:0/318700022 >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 0x7f1f4406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.405+0000 7f1f43fff640 1 --2- 172.21.15.17:0/318700022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 0x7f1f440c5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0097a0 tx=0x7f1f2c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5085f1f3d1bb68e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.405+0000 7f1f42ffd640 1 -- 172.21.15.17:0/318700022 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0315a0 con 0x7f1f441525e0 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.405+0000 7f1f42ffd640 1 -- 172.21.15.17:0/318700022 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c037070 con 0x7f1f441525e0 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.405+0000 7f1f42ffd640 1 -- 172.21.15.17:0/318700022 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c031940 con 0x7f1f441525e0 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 -- 172.21.15.17:0/318700022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 msgr2=0x7f1f440c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 --2- 172.21.15.17:0/318700022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 0x7f1f440c5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0097a0 tx=0x7f1f2c02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 -- 172.21.15.17:0/318700022 shutdown_connections 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 --2- 172.21.15.17:0/318700022 >> v2:172.21.15.163:3300/0 conn(0x7f1f440bdf70 0x7f1f4406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 --2- 172.21.15.17:0/318700022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f441525e0 0x7f1f440c5260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 -- 172.21.15.17:0/318700022 >> 172.21.15.17:0/318700022 conn(0x7f1f44072ee0 msgr2=0x7f1f440ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 -- 172.21.15.17:0/318700022 shutdown_connections 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.406+0000 7f1f495a0640 1 -- 172.21.15.17:0/318700022 wait complete. 2024-03-06T07:22:44.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.407+0000 7f1f495a0640 1 Processor -- start 2024-03-06T07:22:44.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.407+0000 7f1f495a0640 1 -- start start 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f440bdf70 0x7f1f441ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f440c87e0 0x7f1f4414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440acb00 con 0x7f1f441525e0 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440ac820 con 0x7f1f440c87e0 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f495a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4406c3a0 con 0x7f1f440bdf70 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40720/0 (socket says 172.21.15.17:40720) 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 -- 172.21.15.17:0/490484552 learned_addr learned my addr 172.21.15.17:0/490484552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f43fff640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f440bdf70 0x7f1f441ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f437fe640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f440c87e0 0x7f1f4414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 -- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f440bdf70 msgr2=0x7f1f441ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f440bdf70 0x7f1f441ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 -- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f440c87e0 msgr2=0x7f1f4414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.408+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f440c87e0 0x7f1f4414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f48d9f640 1 -- 172.21.15.17:0/490484552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f441594a0 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00caa0 tx=0x7f1f3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f40ff9640 1 -- 172.21.15.17:0/490484552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f40ff9640 1 -- 172.21.15.17:0/490484552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00ac00 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f441591c0 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f40ff9640 1 -- 172.21.15.17:0/490484552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c012400 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.409+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f441ee080 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.410+0000 7f1f40ff9640 1 -- 172.21.15.17:0/490484552 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f3c01a070 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.411+0000 7f1f40ff9640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.411+0000 7f1f40ff9640 1 -- 172.21.15.17:0/490484552 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f3c08a0c0 con 0x7f1f441525e0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.411+0000 7f1f43fff640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.411+0000 7f1f43fff640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 0x7f1f2406d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036000 tx=0x7f1f2c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 Processor -- start 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 -- start start 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 0x55f2fe5e3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe54d6b0 con 0x55f2fe5e7660 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe4f6a60 con 0x55f2fe5e94e0 2024-03-06T07:22:44.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.412+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe549750 con 0x55f2fe5e9100 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f48d9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 0x55f2fe5e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:41560/0 (socket says 172.21.15.17:41560) 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 -- 172.21.15.17:0/4200939461 learned_addr learned my addr 172.21.15.17:0/4200939461 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:44954/0 (socket says 172.21.15.17:44954) 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 0x55f2fe5e3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40734/0 (socket says 172.21.15.17:40734) 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 -- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 msgr2=0x55f2fe5e7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 -- 172.21.15.17:0/4200939461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 msgr2=0x55f2fe5e3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 0x55f2fe5e3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 -- 172.21.15.17:0/4200939461 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2fe514f30 con 0x55f2fe5e9100 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f437fe640 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f43fff640 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c02be50 tx=0x7f1f2c030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121dda20033fc17c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f39ffb640 1 -- 172.21.15.17:0/4200939461 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c054070 con 0x55f2fe5e9100 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f39ffb640 1 -- 172.21.15.17:0/4200939461 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c037040 con 0x55f2fe5e9100 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.413+0000 7f1f39ffb640 1 -- 172.21.15.17:0/4200939461 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c032980 con 0x55f2fe5e9100 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 msgr2=0x55f2fe5e6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c02be50 tx=0x7f1f2c030000 comp rx=0 tx=0).stop 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/4200939461 shutdown_connections 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e7660 0x55f2fe5e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe5e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/4200939461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe5e6be0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/4200939461 >> 172.21.15.17:0/4200939461 conn(0x55f2fe5e5980 msgr2=0x55f2fe60a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/4200939461 shutdown_connections 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/4200939461 wait complete. 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.414+0000 7f1f4d3e1740 1 -- start start 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 Processor -- start 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe628ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 0x55f2fe604e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe60b3e0 con 0x55f2fe5e94e0 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe509ea0 con 0x55f2fe60a160 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe60ae20 con 0x55f2fe5e9100 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f4d3e1740 10 client.0 _ll_register_callbacks cb 0x55f2fe509870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.415+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40742/0 (socket says 172.21.15.17:40742) 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 -- 172.21.15.17:0/1993536851 learned_addr learned my addr 172.21.15.17:0/1993536851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f43fff640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe628ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 0x55f2fe604e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 -- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 msgr2=0x55f2fe628ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe628ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 -- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 msgr2=0x55f2fe604e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 0x55f2fe604e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 -- 172.21.15.17:0/1993536851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2fe50a270 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f43fff640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe628ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 0x55f2fe604e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f437fe640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1f340146f0 tx=0x7f1f34014af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11771a507ee52056 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004300 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f34003610 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f2c037040 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f34026050 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f4d3e1740 10 client.6555 fetch_fsmap learned FSMap version 56 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.416+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55f2fe2622d0 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c037040 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 56) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f1f34029020 con 0x55f2fe5e94e0 2024-03-06T07:22:44.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 10 client.6555 fetch_fsmap finished waiting for FSMap version 56 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 10 client.6555 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 20 client.6555 populate_metadata read hostname 'smithi017' 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe639a10 0x55f2fe644500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 4 client.6555 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 --> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2fe55abf0 con 0x55f2fe639a10 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f43fff640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe639a10 0x55f2fe644500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f43fff640 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe639a10 0x55f2fe644500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca88a1a8765e765 server_cookie=e1b999bdb95622f3 in_seq=0 out_seq=0 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.417+0000 7f1f22ffd640 10 client.6555 ms_handle_connect on v2:172.21.15.163:6834/1213270757 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f22ffd640 1 -- 172.21.15.17:0/1993536851 <== mds.0 v2:172.21.15.163:6834/1213270757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2fe55abf0 con 0x55f2fe639a10 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f22ffd640 10 client.6555 handle_command_reply: tid=0 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 client.6555 shutdown 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 msgr2=0x55f2fe609c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1f340146f0 tx=0x7f1f34014af0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 shutdown_connections 2024-03-06T07:22:44.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe60a160 0x55f2fe604e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe609c20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe639a10 0x55f2fe644500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/1993536851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe5e9100 0x55f2fe628ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f22ffd640 0 client.6555 ms_handle_reset on v2:172.21.15.163:6834/1213270757 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 >> 172.21.15.17:0/1993536851 conn(0x55f2fe5e7660 msgr2=0x55f2fe625d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 shutdown_connections 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/1993536851 wait complete. 2024-03-06T07:22:44.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.418+0000 7f1f4d3e1740 20 client.6555 trim_cache size 0 max 16384 2024-03-06T07:22:44.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.435+0000 7f1f4d3e1740 1 Processor -- start 2024-03-06T07:22:44.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.435+0000 7f1f4d3e1740 1 -- start start 2024-03-06T07:22:44.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e9100 0x55f2fe649530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe60e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441f6b20 con 0x55f2fe5e94e0 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440ac000 con 0x55f2fe5e9100 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441f6520 con 0x55f2fe60a160 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40746/0 (socket says 172.21.15.17:40746) 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe60e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f437fe640 1 -- 172.21.15.17:0/2831005559 learned_addr learned my addr 172.21.15.17:0/2831005559 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f43fff640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e9100 0x55f2fe649530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 -- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e9100 msgr2=0x55f2fe649530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e9100 0x55f2fe649530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 -- 172.21.15.17:0/2831005559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 msgr2=0x55f2fe60dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 -- 172.21.15.17:0/2831005559 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2fe32c6b0 con 0x55f2fe60a160 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f437fe640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe60e290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0884b0 tx=0x7f1f3c0014d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9796d87f0265f8bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.436+0000 7f1f397fa640 1 -- 172.21.15.17:0/2831005559 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c070070 con 0x55f2fe60a160 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f397fa640 1 -- 172.21.15.17:0/2831005559 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c070210 con 0x55f2fe60a160 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f397fa640 1 -- 172.21.15.17:0/2831005559 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c060b50 con 0x55f2fe60a160 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 msgr2=0x55f2fe60e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe60e290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0884b0 tx=0x7f1f3c0014d0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2831005559 shutdown_connections 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe60e290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e94e0 0x55f2fe60dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2831005559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e9100 0x55f2fe649530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2831005559 >> 172.21.15.17:0/2831005559 conn(0x55f2fe5e7660 msgr2=0x55f2fe5def60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2831005559 shutdown_connections 2024-03-06T07:22:44.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.437+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2831005559 wait complete. 2024-03-06T07:22:44.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.438+0000 7f1f4d3e1740 1 -- start start 2024-03-06T07:22:44.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.438+0000 7f1f4d3e1740 1 Processor -- start 2024-03-06T07:22:44.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe694a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe5e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe627050 con 0x55f2fe5e9100 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe622ca0 con 0x55f2fe5e94e0 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2fe638e60 con 0x55f2fe60a160 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 10 client.0 _ll_register_callbacks cb 0x7f1f34009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40758/0 (socket says 172.21.15.17:40758) 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 -- 172.21.15.17:0/2743000590 learned_addr learned my addr 172.21.15.17:0/2743000590 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f48d9f640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe5e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f437fe640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe694a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 -- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 msgr2=0x55f2fe5e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe5e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 -- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 msgr2=0x55f2fe694a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe694a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 -- 172.21.15.17:0/2743000590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2fe609520 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f43fff640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0098c0 tx=0x7f1f2c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5c866e578cc781 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c054040 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f2c05f180 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.439+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f3c070210 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2c068050 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 10 client.6561 fetch_fsmap learned FSMap version 56 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55f2fe5e7200 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c070210 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 56) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f1f2c06b020 con 0x55f2fe5e9100 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 10 client.6561 fetch_fsmap finished waiting for FSMap version 56 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 10 client.6561 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:22:44.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 20 client.6561 populate_metadata read hostname 'smithi017' 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe5e81c0 0x55f2fe691720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 4 client.6561 mds_command: new command op to 6387 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 --> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f2fe554f40 con 0x55f2fe5e81c0 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f437fe640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe5e81c0 0x55f2fe691720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f22ffd640 10 client.6561 ms_handle_connect on v2:172.21.15.163:6834/1213270757 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.440+0000 7f1f437fe640 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe5e81c0 0x55f2fe691720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ec6b23872c8710 server_cookie=be9675c295c3a40e in_seq=0 out_seq=0 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f22ffd640 1 -- 172.21.15.17:0/2743000590 <== mds.0 v2:172.21.15.163:6834/1213270757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55f2fe554f40 con 0x55f2fe5e81c0 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f22ffd640 10 client.6561 handle_command_reply: tid=0 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 client.6561 shutdown 2024-03-06T07:22:44.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 msgr2=0x55f2fe60cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0098c0 tx=0x7f1f2c02e000 comp rx=0 tx=0).stop 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 shutdown_connections 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:6834/1213270757,v1:172.21.15.163:6836/1213270757] conn(0x55f2fe5e81c0 0x55f2fe691720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55f2fe60a160 0x55f2fe5e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55f2fe5e94e0 0x55f2fe694a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f22ffd640 0 client.6561 ms_handle_reset on v2:172.21.15.163:6834/1213270757 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 --2- 172.21.15.17:0/2743000590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f2fe5e9100 0x55f2fe60cfb0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 >> 172.21.15.17:0/2743000590 conn(0x55f2fe5e7660 msgr2=0x55f2fe60b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 shutdown_connections 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 1 -- 172.21.15.17:0/2743000590 wait complete. 2024-03-06T07:22:44.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f4d3e1740 20 client.6561 trim_cache size 0 max 16384 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 6387, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 9.0963344300000006, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 56, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 75, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 75, 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14.160527639 2024-03-06T07:22:44.450 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:22:44.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 msgr2=0x7f1f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 0x7f1f2406d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036000 tx=0x7f1f2c03d040 comp rx=0 tx=0).stop 2024-03-06T07:22:44.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 msgr2=0x7f1f441ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00caa0 tx=0x7f1f3c00cf60 comp rx=0 tx=0).stop 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 shutdown_connections 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2406b4d0 0x7f1f2406d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f441525e0 0x7f1f441ea790 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f440c87e0 0x7f1f4414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 --2- 172.21.15.17:0/490484552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f440bdf70 0x7f1f441ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.441+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 >> 172.21.15.17:0/490484552 conn(0x7f1f44072ee0 msgr2=0x7f1f4414c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.442+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 shutdown_connections 2024-03-06T07:22:44.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.442+0000 7f1f495a0640 1 -- 172.21.15.17:0/490484552 wait complete. 2024-03-06T07:22:44.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:44.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.553+0000 7f144ba83640 1 Processor -- start 2024-03-06T07:22:44.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.553+0000 7f144ba83640 1 -- start start 2024-03-06T07:22:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144ba83640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1444072720 0x7f1444156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144ba83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f144406f7b0 0x7f14441501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144ba83640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1444157340 con 0x7f14441506f0 2024-03-06T07:22:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144ba83640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14440c3740 con 0x7f1444072720 2024-03-06T07:22:44.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144ba83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444156d80 con 0x7f144406f7b0 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144981f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1444072720 0x7f1444156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144a020640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f14441506f0 0x7f14440c3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39622/0 (socket says 172.21.15.17:39622) 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144a020640 1 -- 172.21.15.17:0/3172423303 learned_addr learned my addr 172.21.15.17:0/3172423303 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144901e640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f144406f7b0 0x7f14441501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144981f640 1 -- 172.21.15.17:0/3172423303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f144406f7b0 msgr2=0x7f14441501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144981f640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f144406f7b0 0x7f14441501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.554+0000 7f144981f640 1 -- 172.21.15.17:0/3172423303 >> v1:172.21.15.17:6789/0 conn(0x7f14441506f0 legacy=0x7f14440c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144981f640 1 -- 172.21.15.17:0/3172423303 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444157060 con 0x7f1444072720 2024-03-06T07:22:44.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144881d640 1 -- 172.21.15.17:0/3172423303 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265957245 0 0) 0x7f1444157340 con 0x7f14441506f0 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144981f640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444156430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f143c0097a0 tx=0x7f143c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13f1ae327034554 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144881d640 1 -- 172.21.15.17:0/3172423303 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c0315a0 con 0x7f1444072720 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144881d640 1 -- 172.21.15.17:0/3172423303 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c037070 con 0x7f1444072720 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.555+0000 7f144881d640 1 -- 172.21.15.17:0/3172423303 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c031960 con 0x7f1444072720 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 -- 172.21.15.17:0/3172423303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 msgr2=0x7f1444156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444156430 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f143c0097a0 tx=0x7f143c02b840 comp rx=0 tx=0).stop 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 -- 172.21.15.17:0/3172423303 shutdown_connections 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f144406f7b0 0x7f14441501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 --2- 172.21.15.17:0/3172423303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444156430 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 -- 172.21.15.17:0/3172423303 >> 172.21.15.17:0/3172423303 conn(0x7f14440aa630 msgr2=0x7f14440aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 -- 172.21.15.17:0/3172423303 shutdown_connections 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.556+0000 7f144ba83640 1 -- 172.21.15.17:0/3172423303 wait complete. 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.557+0000 7f144ba83640 1 Processor -- start 2024-03-06T07:22:44.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.557+0000 7f144ba83640 1 -- start start 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406f7b0 0x7f14441ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14441506f0 0x7f14440ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444156af0 con 0x7f1444072720 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444156970 con 0x7f144406f7b0 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144ba83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444072b00 con 0x7f14441506f0 2024-03-06T07:22:44.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40776/0 (socket says 172.21.15.17:40776) 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 -- 172.21.15.17:0/1251033199 learned_addr learned my addr 172.21.15.17:0/1251033199 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144981f640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406f7b0 0x7f14441ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144a020640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14441506f0 0x7f14440ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 -- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14441506f0 msgr2=0x7f14440ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14441506f0 0x7f14440ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 -- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406f7b0 msgr2=0x7f14441ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406f7b0 0x7f14441ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 -- 172.21.15.17:0/1251033199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444147ce0 con 0x7f1444072720 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f144901e640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9b0 tx=0x7f143800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.558+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1438017070 con 0x7f1444072720 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.559+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143800c9a0 con 0x7f1444072720 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.559+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441ed080 con 0x7f1444072720 2024-03-06T07:22:44.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.559+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143800cd90 con 0x7f1444072720 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.559+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1444148260 con 0x7f1444072720 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.559+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144404f330 con 0x7f1444072720 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.560+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143801a070 con 0x7f1444072720 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.560+0000 7f14427fc640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 0x7f143006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.560+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f143808a290 con 0x7f1444072720 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.560+0000 7f144981f640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 0x7f143006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.561+0000 7f144981f640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 0x7f143006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f143c036000 tx=0x7f143c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.565+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14380589c0 con 0x7f1444072720 2024-03-06T07:22:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.781+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14440c6f80 con 0x7f1444072720 2024-03-06T07:22:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.782+0000 7f14427fc640 1 -- 172.21.15.17:0/1251033199 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4982 (secure 0 0 0) 0x7f143805ca60 con 0x7f1444072720 2024-03-06T07:22:44.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:44.784 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6375,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:36.709974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"b","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/1213270757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1213270757},{"type":"v1","addr":"172.21.15.163:6836","nonce":1213270757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5}]} 2024-03-06T07:22:44.784 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 56 2024-03-06T07:22:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 msgr2=0x7f143006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 0x7f143006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f143c036000 tx=0x7f143c008030 comp rx=0 tx=0).stop 2024-03-06T07:22:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 msgr2=0x7f1444155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f143800e9b0 tx=0x7f143800ee70 comp rx=0 tx=0).stop 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 shutdown_connections 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f143006b4d0 0x7f143006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14441506f0 0x7f14440ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1444072720 0x7f1444155ed0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 --2- 172.21.15.17:0/1251033199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f144406f7b0 0x7f14441ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 >> 172.21.15.17:0/1251033199 conn(0x7f14440aa630 msgr2=0x7f144414b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 shutdown_connections 2024-03-06T07:22:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.784+0000 7f144ba83640 1 -- 172.21.15.17:0/1251033199 wait complete. 2024-03-06T07:22:44.796 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 56, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:22:34.715155+0000", "damaged": [], "data_pools": [ 12 ], "enabled": true, "epoch": 56, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6387": { "addr": "172.21.15.163:6836/1213270757", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1213270757, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1213270757, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6387, "incarnation": 54, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 11, "modified": "2024-03-06T07:22:36.709974+0000", "qdb_cluster": [ 6387 ], "qdb_leader": 6387, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6387 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/1858186621", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 1858186621, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 1858186621, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 53, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6106, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/4026996022", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 4026996022, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 4026996022, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 53, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/285046827", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 285046827, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 285046827, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 53, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6375, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:22:44.797 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:22:44.797 INFO:tasks.cephfs.test_failover:current = [{'gid': 6372, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/4026996022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 4026996022}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 4026996022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 53}, {'gid': 6387, 'name': 'b', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6836/1213270757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1213270757}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1213270757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6375, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/285046827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 285046827}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 285046827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 53}, {'gid': 6106, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/1858186621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 1858186621}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1858186621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 53}] 2024-03-06T07:22:44.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:22:44.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.900+0000 7f2de969a640 1 Processor -- start 2024-03-06T07:22:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.900+0000 7f2de969a640 1 -- start start 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de969a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2de41508c0 0x7f2de40c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de969a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4157340 con 0x7f2de406f7b0 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de969a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4156d80 con 0x7f2de4072720 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de969a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de4156b40 con 0x7f2de41508c0 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2ddbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2de406f7b0 0x7f2de41501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39632/0 (socket says 172.21.15.17:39632) 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de2ffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2de4072720 0x7f2de4156430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46984/0 (socket says 172.21.15.17:46984) 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2ddbfff640 1 -- 172.21.15.17:0/713006658 learned_addr learned my addr 172.21.15.17:0/713006658 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de37fe640 1 --2- 172.21.15.17:0/713006658 >> v2:172.21.15.163:3300/0 conn(0x7f2de41508c0 0x7f2de40c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449971688 0 0) 0x7f2de4157340 con 0x7f2de406f7b0 2024-03-06T07:22:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd0003520 con 0x7f2de406f7b0 2024-03-06T07:22:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168261944 0 0) 0x7f2de4156d80 con 0x7f2de4072720 2024-03-06T07:22:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de37fe640 1 -- 172.21.15.17:0/713006658 >> v1:172.21.15.163:6789/0 conn(0x7f2de4072720 legacy=0x7f2de4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de37fe640 1 -- 172.21.15.17:0/713006658 >> v1:172.21.15.17:6789/0 conn(0x7f2de406f7b0 legacy=0x7f2de41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.901+0000 7f2de37fe640 1 -- 172.21.15.17:0/713006658 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4157060 con 0x7f2de41508c0 2024-03-06T07:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.902+0000 7f2de37fe640 1 --2- 172.21.15.17:0/713006658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de41508c0 0x7f2de40c3bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009d90 tx=0x7f2ddc02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42262abfa6e9a44e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.902+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc039070 con 0x7f2de41508c0 2024-03-06T07:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.902+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc02cc60 con 0x7f2de41508c0 2024-03-06T07:22:44.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.902+0000 7f2de27fc640 1 -- 172.21.15.17:0/713006658 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc034640 con 0x7f2de41508c0 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 -- 172.21.15.17:0/713006658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de41508c0 msgr2=0x7f2de40c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 --2- 172.21.15.17:0/713006658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de41508c0 0x7f2de40c3bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009d90 tx=0x7f2ddc02beb0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 -- 172.21.15.17:0/713006658 shutdown_connections 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 --2- 172.21.15.17:0/713006658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de41508c0 0x7f2de40c3bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 -- 172.21.15.17:0/713006658 >> 172.21.15.17:0/713006658 conn(0x7f2de40aa630 msgr2=0x7f2de40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 -- 172.21.15.17:0/713006658 shutdown_connections 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.903+0000 7f2de969a640 1 -- 172.21.15.17:0/713006658 wait complete. 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 Processor -- start 2024-03-06T07:22:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 -- start start 2024-03-06T07:22:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de4072720 0x7f2de41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2de41508c0 0x7f2de414bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4072b00 con 0x7f2de406f7b0 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40c3740 con 0x7f2de4072720 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.904+0000 7f2de969a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4072c80 con 0x7f2de41508c0 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40786/0 (socket says 172.21.15.17:40786) 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 -- 172.21.15.17:0/4271861768 learned_addr learned my addr 172.21.15.17:0/4271861768 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2ddbfff640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de4072720 0x7f2de41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de37fe640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2de41508c0 0x7f2de414bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 -- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2de41508c0 msgr2=0x7f2de414bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2de41508c0 0x7f2de414bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 -- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de4072720 msgr2=0x7f2de41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de4072720 0x7f2de41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 -- 172.21.15.17:0/4271861768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40ad340 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de2ffd640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c4d0 tx=0x7f2dd400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4018070 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd4007500 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de4151030 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4004a20 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.905+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de41515f0 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.906+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de404f330 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.907+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dd4004bc0 con 0x7f2de406f7b0 2024-03-06T07:22:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.907+0000 7f2ddb7fe640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 0x7f2dc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:44.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.907+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2dd4089540 con 0x7f2de406f7b0 2024-03-06T07:22:44.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.907+0000 7f2ddbfff640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 0x7f2dc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:44.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.908+0000 7f2ddbfff640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 0x7f2dc406d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2de41504c0 tx=0x7f2dcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:44.911+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2dd4057bf0 con 0x7f2de406f7b0 2024-03-06T07:22:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.092+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f2de40aae20 con 0x7f2de406f7b0 2024-03-06T07:22:45.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.095+0000 7f2ddb7fe640 1 -- 172.21.15.17:0/4271861768 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v24)=0 v24) v1 ==== 117+0+0 (secure 0 0 0) 0x7f2dd405bc90 con 0x7f2de406f7b0 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 msgr2=0x7f2dc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 0x7f2dc406d9c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2de41504c0 tx=0x7f2dcc009040 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 msgr2=0x7f2de40adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c4d0 tx=0x7f2dd400c990 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 shutdown_connections 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2dc406b500 0x7f2dc406d9c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2de41508c0 0x7f2de414bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2de4072720 0x7f2de41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 --2- 172.21.15.17:0/4271861768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de406f7b0 0x7f2de40adde0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 >> 172.21.15.17:0/4271861768 conn(0x7f2de40aa630 msgr2=0x7f2de4156050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 shutdown_connections 2024-03-06T07:22:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.097+0000 7f2de969a640 1 -- 172.21.15.17:0/4271861768 wait complete. 2024-03-06T07:22:45.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.a mds_join_fs cephfs2 2024-03-06T07:22:45.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.211+0000 7f967ba9a640 1 Processor -- start 2024-03-06T07:22:45.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f967ba9a640 1 -- start start 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f967ba9a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96740c7800 0x7f967406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f967ba9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9674155260 con 0x7f96741527f0 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f967ba9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9674155540 con 0x7f96740c7370 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f967ba9a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96740c7be0 con 0x7f96740c7800 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f9679836640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96741527f0 0x7f96740c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39646/0 (socket says 172.21.15.17:39646) 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.212+0000 7f9679836640 1 -- 172.21.15.17:0/2857903140 learned_addr learned my addr 172.21.15.17:0/2857903140 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f967a037640 1 --2- 172.21.15.17:0/2857903140 >> v2:172.21.15.163:3300/0 conn(0x7f96740c7800 0x7f967406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667036549 0 0) 0x7f9674155260 con 0x7f96741527f0 2024-03-06T07:22:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9660003520 con 0x7f96741527f0 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211226995 0 0) 0x7f9674155540 con 0x7f96740c7370 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9674155260 con 0x7f96740c7370 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f967a037640 1 -- 172.21.15.17:0/2857903140 >> v1:172.21.15.163:6789/0 conn(0x7f96740c7370 legacy=0x7f967406c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f967a037640 1 -- 172.21.15.17:0/2857903140 >> v1:172.21.15.17:6789/0 conn(0x7f96741527f0 legacy=0x7f96740c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105218894 0 0) 0x7f9660003520 con 0x7f96741527f0 2024-03-06T07:22:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.213+0000 7f967a037640 1 -- 172.21.15.17:0/2857903140 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674155820 con 0x7f96740c7800 2024-03-06T07:22:45.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f967a037640 1 --2- 172.21.15.17:0/2857903140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7800 0x7f967406c7c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f966c009d90 tx=0x7f966c02bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535d6f3168e9d4eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c039070 con 0x7f96740c7800 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966c02c820 con 0x7f96740c7800 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f9678834640 1 -- 172.21.15.17:0/2857903140 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670001670 con 0x7f96740c7800 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f967ba9a640 1 -- 172.21.15.17:0/2857903140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7800 msgr2=0x7f967406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.214+0000 7f967ba9a640 1 --2- 172.21.15.17:0/2857903140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7800 0x7f967406c7c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f966c009d90 tx=0x7f966c02bcf0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.215+0000 7f967ba9a640 1 -- 172.21.15.17:0/2857903140 shutdown_connections 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.215+0000 7f967ba9a640 1 --2- 172.21.15.17:0/2857903140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7800 0x7f967406c7c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.215+0000 7f967ba9a640 1 -- 172.21.15.17:0/2857903140 >> 172.21.15.17:0/2857903140 conn(0x7f96740ac4f0 msgr2=0x7f96740ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.215+0000 7f967ba9a640 1 -- 172.21.15.17:0/2857903140 shutdown_connections 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.215+0000 7f967ba9a640 1 -- 172.21.15.17:0/2857903140 wait complete. 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 Processor -- start 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 -- start start 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7370 0x7f96740afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96741527f0 0x7f967414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967406cd00 con 0x7f96740c7800 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740c8f70 con 0x7f96740c7370 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f967ba9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96740be1a0 con 0x7f96741527f0 2024-03-06T07:22:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f9679035640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.216+0000 7f9679035640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40788/0 (socket says 172.21.15.17:40788) 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 -- 172.21.15.17:0/790843550 learned_addr learned my addr 172.21.15.17:0/790843550 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f967a037640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96741527f0 0x7f967414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679836640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7370 0x7f96740afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 -- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96741527f0 msgr2=0x7f967414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96741527f0 0x7f967414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 -- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7370 msgr2=0x7f96740afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7370 0x7f96740afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 -- 172.21.15.17:0/790843550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96740ae240 con 0x7f96740c7800 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f9679035640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f966400c460 tx=0x7f966400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96640052d0 con 0x7f96740c7800 2024-03-06T07:22:45.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9664005490 con 0x7f96740c7800 2024-03-06T07:22:45.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96740ae520 con 0x7f96740c7800 2024-03-06T07:22:45.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9664012870 con 0x7f96740c7800 2024-03-06T07:22:45.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.217+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96740aeae0 con 0x7f96740c7800 2024-03-06T07:22:45.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.218+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967404f330 con 0x7f96740c7800 2024-03-06T07:22:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.219+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9664012a10 con 0x7f96740c7800 2024-03-06T07:22:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.219+0000 7f966a7fc640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 0x7f965806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.219+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9664089e80 con 0x7f96740c7800 2024-03-06T07:22:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.219+0000 7f9679836640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 0x7f965806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.220+0000 7f9679836640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 0x7f965806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f96700011e0 tx=0x7f9670001690 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.223+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9664058530 con 0x7f96740c7800 2024-03-06T07:22:45.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.405+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f967406d860 con 0x7f96740c7800 2024-03-06T07:22:45.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.408+0000 7f966a7fc640 1 -- 172.21.15.17:0/790843550 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v25)=0 v25) v1 ==== 117+0+0 (secure 0 0 0) 0x7f96640770d0 con 0x7f96740c7800 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 msgr2=0x7f965806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 0x7f965806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f96700011e0 tx=0x7f9670001690 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 msgr2=0x7f967414b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f966400c460 tx=0x7f966400c920 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 shutdown_connections 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965806b4d0 0x7f965806d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96741527f0 0x7f967414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96740c7800 0x7f967414b840 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 --2- 172.21.15.17:0/790843550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96740c7370 0x7f96740afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 >> 172.21.15.17:0/790843550 conn(0x7f96740ac4f0 msgr2=0x7f9674150010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 shutdown_connections 2024-03-06T07:22:45.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.410+0000 7f967ba9a640 1 -- 172.21.15.17:0/790843550 wait complete. 2024-03-06T07:22:45.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs2 2024-03-06T07:22:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.525+0000 7f57fd78c640 1 Processor -- start 2024-03-06T07:22:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.525+0000 7f57fd78c640 1 -- start start 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57fd78c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57f806ee40 0x7f57f8153a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57fd78c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57f806c4f0 0x7f57f806c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57fd78c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f8153000 con 0x7f57f8149950 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57fd78c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f80abf70 con 0x7f57f806c4f0 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57fd78c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f8153280 con 0x7f57f806ee40 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57f8149950 0x7f57f81565f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39660/0 (socket says 172.21.15.17:39660) 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f6ffd640 1 -- 172.21.15.17:0/2764585098 learned_addr learned my addr 172.21.15.17:0/2764585098 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2764585098 >> v2:172.21.15.17:3300/0 conn(0x7f57f806c4f0 0x7f57f806c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f67fc640 1 --2- 172.21.15.17:0/2764585098 >> v2:172.21.15.163:3300/0 conn(0x7f57f806ee40 0x7f57f8153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f5ffb640 1 -- 172.21.15.17:0/2764585098 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729111541 0 0) 0x7f57f0001110 con 0x7f57f8149950 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f5ffb640 1 -- 172.21.15.17:0/2764585098 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e4003520 con 0x7f57f8149950 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f77fe640 1 -- 172.21.15.17:0/2764585098 >> v2:172.21.15.163:3300/0 conn(0x7f57f806ee40 msgr2=0x7f57f8153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2764585098 >> v2:172.21.15.163:3300/0 conn(0x7f57f806ee40 0x7f57f8153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f77fe640 1 -- 172.21.15.17:0/2764585098 >> v1:172.21.15.17:6789/0 conn(0x7f57f8149950 legacy=0x7f57f81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.526+0000 7f57f77fe640 1 -- 172.21.15.17:0/2764585098 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f806f810 con 0x7f57f806c4f0 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.527+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2764585098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f806c4f0 0x7f57f806c960 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f57f8157680 tx=0x7f57e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bddc4bae9c7e379 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.527+0000 7f57f5ffb640 1 -- 172.21.15.17:0/2764585098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8039070 con 0x7f57f806c4f0 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.527+0000 7f57f5ffb640 1 -- 172.21.15.17:0/2764585098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e802cc80 con 0x7f57f806c4f0 2024-03-06T07:22:45.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.527+0000 7f57f5ffb640 1 -- 172.21.15.17:0/2764585098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8034640 con 0x7f57f806c4f0 2024-03-06T07:22:45.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 -- 172.21.15.17:0/2764585098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f806c4f0 msgr2=0x7f57f806c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2764585098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f806c4f0 0x7f57f806c960 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f57f8157680 tx=0x7f57e802bed0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 -- 172.21.15.17:0/2764585098 shutdown_connections 2024-03-06T07:22:45.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2764585098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f806c4f0 0x7f57f806c960 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2764585098 >> v2:172.21.15.163:3300/0 conn(0x7f57f806ee40 0x7f57f8153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 -- 172.21.15.17:0/2764585098 >> 172.21.15.17:0/2764585098 conn(0x7f57f80c45d0 msgr2=0x7f57f80c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 -- 172.21.15.17:0/2764585098 shutdown_connections 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.528+0000 7f57fd78c640 1 -- 172.21.15.17:0/2764585098 wait complete. 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.529+0000 7f57fd78c640 1 Processor -- start 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.529+0000 7f57fd78c640 1 -- start start 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f806c4f0 0x7f57f814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57f806ee40 0x7f57f814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f806e4e0 con 0x7f57f8149950 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f806e360 con 0x7f57f806c4f0 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57fd78c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c69b0 con 0x7f57f806ee40 2024-03-06T07:22:45.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40798/0 (socket says 172.21.15.17:40798) 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 -- 172.21.15.17:0/2477818795 learned_addr learned my addr 172.21.15.17:0/2477818795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f6ffd640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f806c4f0 0x7f57f814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f67fc640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57f806ee40 0x7f57f814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 -- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57f806ee40 msgr2=0x7f57f814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57f806ee40 0x7f57f814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 -- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f806c4f0 msgr2=0x7f57f814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f806c4f0 0x7f57f814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 -- 172.21.15.17:0/2477818795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f814d6d0 con 0x7f57f8149950 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.530+0000 7f57f77fe640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009e80 tx=0x7f57e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.531+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8039050 con 0x7f57f8149950 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.531+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8045070 con 0x7f57f8149950 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.531+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f81ecb60 con 0x7f57f8149950 2024-03-06T07:22:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.531+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8009450 con 0x7f57f8149950 2024-03-06T07:22:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.531+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f814eb70 con 0x7f57f8149950 2024-03-06T07:22:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.532+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f804f330 con 0x7f57f8149950 2024-03-06T07:22:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.532+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57e803e830 con 0x7f57f8149950 2024-03-06T07:22:45.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.532+0000 7f57ef7fe640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 0x7f57d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.533+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57e80ad2f0 con 0x7f57f8149950 2024-03-06T07:22:45.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.533+0000 7f57f6ffd640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 0x7f57d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.533+0000 7f57f6ffd640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 0x7f57d806d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005880 tx=0x7f57f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.537+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57e807bc00 con 0x7f57f8149950 2024-03-06T07:22:45.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.716+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f57f806f2e0 con 0x7f57f8149950 2024-03-06T07:22:45.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.719+0000 7f57ef7fe640 1 -- 172.21.15.17:0/2477818795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v26)=0 v26) v1 ==== 117+0+0 (secure 0 0 0) 0x7f57e807fca0 con 0x7f57f8149950 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 msgr2=0x7f57d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 0x7f57d806d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005880 tx=0x7f57f0009040 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 msgr2=0x7f57f81ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009e80 tx=0x7f57e8004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 shutdown_connections 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57d806b500 0x7f57d806d9c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57f8149950 0x7f57f81ec0b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57f806ee40 0x7f57f814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 --2- 172.21.15.17:0/2477818795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57f806c4f0 0x7f57f814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 >> 172.21.15.17:0/2477818795 conn(0x7f57f80c45d0 msgr2=0x7f57f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 shutdown_connections 2024-03-06T07:22:45.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.721+0000 7f57fd78c640 1 -- 172.21.15.17:0/2477818795 wait complete. 2024-03-06T07:22:45.733 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.832+0000 7fdb30f3e640 1 Processor -- start 2024-03-06T07:22:45.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.832+0000 7fdb30f3e640 1 -- start start 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb30f3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdb2c14f790 0x7fdb2c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb30f3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c0c28d0 con 0x7fdb2c155a40 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb30f3e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2c0c2bb0 con 0x7fdb2c0c1aa0 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb30f3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c2680 con 0x7fdb2c14f790 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2ad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdb2c155a40 0x7fdb2c155e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39676/0 (socket says 172.21.15.17:39676) 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2ad9d640 1 -- 172.21.15.17:0/1718193327 learned_addr learned my addr 172.21.15.17:0/1718193327 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2a59c640 1 --2- 172.21.15.17:0/1718193327 >> v2:172.21.15.163:3300/0 conn(0x7fdb2c14f790 0x7fdb2c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133928747 0 0) 0x7fdb2c0c28d0 con 0x7fdb2c155a40 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18003520 con 0x7fdb2c155a40 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158117233 0 0) 0x7fdb2c0c2bb0 con 0x7fdb2c0c1aa0 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c0c28d0 con 0x7fdb2c0c1aa0 2024-03-06T07:22:45.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2a59c640 1 -- 172.21.15.17:0/1718193327 >> v1:172.21.15.163:6789/0 conn(0x7fdb2c0c1aa0 legacy=0x7fdb2c0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2a59c640 1 -- 172.21.15.17:0/1718193327 >> v1:172.21.15.17:6789/0 conn(0x7fdb2c155a40 legacy=0x7fdb2c155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.833+0000 7fdb2a59c640 1 -- 172.21.15.17:0/1718193327 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0c2e90 con 0x7fdb2c14f790 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.834+0000 7fdb2a59c640 1 --2- 172.21.15.17:0/1718193327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c1529d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c2800 tx=0x7fdb2402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8106f4655b14a416 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.834+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24034070 con 0x7fdb2c14f790 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.834+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24030610 con 0x7fdb2c14f790 2024-03-06T07:22:45.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.834+0000 7fdb2959a640 1 -- 172.21.15.17:0/1718193327 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240309e0 con 0x7fdb2c14f790 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 -- 172.21.15.17:0/1718193327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 msgr2=0x7fdb2c1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/1718193327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c1529d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c2800 tx=0x7fdb2402b960 comp rx=0 tx=0).stop 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 -- 172.21.15.17:0/1718193327 shutdown_connections 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/1718193327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c1529d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 -- 172.21.15.17:0/1718193327 >> 172.21.15.17:0/1718193327 conn(0x7fdb2c0c6710 msgr2=0x7fdb2c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 -- 172.21.15.17:0/1718193327 shutdown_connections 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.835+0000 7fdb30f3e640 1 -- 172.21.15.17:0/1718193327 wait complete. 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.836+0000 7fdb30f3e640 1 Processor -- start 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.836+0000 7fdb30f3e640 1 -- start start 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb2c0c1aa0 0x7fdb2c1dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c5e30 con 0x7fdb2c155a40 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c0d60 con 0x7fdb2c14f790 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb30f3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2c0c5fb0 con 0x7fdb2c0c1aa0 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40808/0 (socket says 172.21.15.17:40808) 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 -- 172.21.15.17:0/3366206808 learned_addr learned my addr 172.21.15.17:0/3366206808 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2a59c640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb2c0c1aa0 0x7fdb2c1dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb29d9b640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 -- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb2c0c1aa0 msgr2=0x7fdb2c1dfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb2c0c1aa0 0x7fdb2c1dfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 -- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 msgr2=0x7fdb2c14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:45.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c14ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.837+0000 7fdb2ad9d640 1 -- 172.21.15.17:0/3366206808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c1ede70 con 0x7fdb2c155a40 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb2ad9d640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00caa0 tx=0x7fdb1c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c017070 con 0x7fdb2c155a40 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb2c070d00 con 0x7fdb2c155a40 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00ac00 con 0x7fdb2c155a40 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c012400 con 0x7fdb2c155a40 2024-03-06T07:22:45.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.838+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb2c071000 con 0x7fdb2c155a40 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.839+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2c04f330 con 0x7fdb2c155a40 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.839+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb1c01a070 con 0x7fdb2c155a40 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.840+0000 7fdb22ffd640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 0x7fdb0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.840+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb1c089f40 con 0x7fdb2c155a40 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.840+0000 7fdb2a59c640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 0x7fdb0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:45.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.840+0000 7fdb2a59c640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 0x7fdb0c06d9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb24036000 tx=0x7fdb2403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:45.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:45.844+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb1c0585f0 con 0x7fdb2c155a40 2024-03-06T07:22:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.027+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb2c0c1f40 con 0x7fdb2c155a40 2024-03-06T07:22:46.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.028+0000 7fdb22ffd640 1 -- 172.21.15.17:0/3366206808 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16945 (secure 0 0 0) 0x7fdb1c05c690 con 0x7fdb2c155a40 2024-03-06T07:22:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:46.030 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":75,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:36.359355+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 msgr2=0x7fdb0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 0x7fdb0c06d9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb24036000 tx=0x7fdb2403b040 comp rx=0 tx=0).stop 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 msgr2=0x7fdb2c153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00caa0 tx=0x7fdb1c00cf60 comp rx=0 tx=0).stop 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 shutdown_connections 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb0c06b500 0x7fdb0c06d9c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c155a40 0x7fdb2c153320 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb2c14f790 0x7fdb2c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 --2- 172.21.15.17:0/3366206808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb2c0c1aa0 0x7fdb2c1dfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 >> 172.21.15.17:0/3366206808 conn(0x7fdb2c0c6710 msgr2=0x7fdb2c072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 shutdown_connections 2024-03-06T07:22:46.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.030+0000 7fdb30f3e640 1 -- 172.21.15.17:0/3366206808 wait complete. 2024-03-06T07:22:46.043 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:22:32.972999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:22:33.673658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:22:46.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:22:46.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.146+0000 7fd4669d7640 1 Processor -- start 2024-03-06T07:22:46.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.146+0000 7fd4669d7640 1 -- start start 2024-03-06T07:22:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd4669d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 0x7fd46014e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd4669d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4600b0450 con 0x7fd4600b0070 2024-03-06T07:22:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd4669d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4601560e0 con 0x7fd460155c30 2024-03-06T07:22:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd4669d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46006c020 con 0x7fd46006ca70 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd4600b0070 0x7fd4600c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39690/0 (socket says 172.21.15.17:39690) 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45ffff640 1 -- 172.21.15.17:0/2095087269 learned_addr learned my addr 172.21.15.17:0/2095087269 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd464f74640 1 --2- 172.21.15.17:0/2095087269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 0x7fd46014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171412483 0 0) 0x7fd450001110 con 0x7fd4600b0070 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd448003520 con 0x7fd4600b0070 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539489640 0 0) 0x7fd4601560e0 con 0x7fd460155c30 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.147+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd450001110 con 0x7fd460155c30 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.148+0000 7fd464f74640 1 -- 172.21.15.17:0/2095087269 >> v1:172.21.15.163:6789/0 conn(0x7fd460155c30 legacy=0x7fd4600c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.148+0000 7fd464f74640 1 -- 172.21.15.17:0/2095087269 >> v1:172.21.15.17:6789/0 conn(0x7fd4600b0070 legacy=0x7fd4600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.148+0000 7fd464f74640 1 -- 172.21.15.17:0/2095087269 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600c77f0 con 0x7fd46006ca70 2024-03-06T07:22:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.148+0000 7fd464f74640 1 --2- 172.21.15.17:0/2095087269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 0x7fd46014e800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4601562b0 tx=0x7fd45802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25445b15c05efe3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458039070 con 0x7fd46006ca70 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45802cc40 con 0x7fd46006ca70 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd45effd640 1 -- 172.21.15.17:0/2095087269 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd458034440 con 0x7fd46006ca70 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 -- 172.21.15.17:0/2095087269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 msgr2=0x7fd46014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 --2- 172.21.15.17:0/2095087269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 0x7fd46014e800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd4601562b0 tx=0x7fd45802be90 comp rx=0 tx=0).stop 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 -- 172.21.15.17:0/2095087269 shutdown_connections 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 --2- 172.21.15.17:0/2095087269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd46006ca70 0x7fd46014e800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 -- 172.21.15.17:0/2095087269 >> 172.21.15.17:0/2095087269 conn(0x7fd4600ac590 msgr2=0x7fd4600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 -- 172.21.15.17:0/2095087269 shutdown_connections 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.149+0000 7fd4669d7640 1 -- 172.21.15.17:0/2095087269 wait complete. 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.150+0000 7fd4669d7640 1 Processor -- start 2024-03-06T07:22:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.150+0000 7fd4669d7640 1 -- start start 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4600b0070 0x7fd460154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460155c30 0x7fd460158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46006c550 con 0x7fd46006ca70 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46006c270 con 0x7fd4600b0070 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd4669d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46006da80 con 0x7fd460155c30 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40818/0 (socket says 172.21.15.17:40818) 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 -- 172.21.15.17:0/1257088436 learned_addr learned my addr 172.21.15.17:0/1257088436 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd464f74640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460155c30 0x7fd460158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45f7fe640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4600b0070 0x7fd460154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 -- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460155c30 msgr2=0x7fd460158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460155c30 0x7fd460158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 -- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4600b0070 msgr2=0x7fd460154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4600b0070 0x7fd460154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.151+0000 7fd45ffff640 1 -- 172.21.15.17:0/1257088436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600c46f0 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd45ffff640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd45000c6b0 tx=0x7fd45000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450017070 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd450005ab0 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600c49d0 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450007070 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.152+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd460156460 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.153+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46004f330 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.153+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd45001a070 con 0x7fd46006ca70 2024-03-06T07:22:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.153+0000 7fd45cff9640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 0x7fd44c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.154+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4500898b0 con 0x7fd46006ca70 2024-03-06T07:22:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.154+0000 7fd45f7fe640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 0x7fd44c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.154+0000 7fd45f7fe640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 0x7fd44c06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd454007fd0 tx=0x7fd454009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.158+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd4600b0450 con 0x7fd46006ca70 2024-03-06T07:22:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.336+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd46006d860 con 0x7fd46006ca70 2024-03-06T07:22:46.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.336+0000 7fd45cff9640 1 -- 172.21.15.17:0/1257088436 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd450057f50 con 0x7fd46006ca70 2024-03-06T07:22:46.338 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 msgr2=0x7fd44c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 0x7fd44c06d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd454007fd0 tx=0x7fd454009040 comp rx=0 tx=0).stop 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 msgr2=0x7fd4600c3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd45000c6b0 tx=0x7fd45000cb70 comp rx=0 tx=0).stop 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 shutdown_connections 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd44c06b4d0 0x7fd44c06d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd460155c30 0x7fd460158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4600b0070 0x7fd460154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 --2- 172.21.15.17:0/1257088436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd46006ca70 0x7fd4600c3ab0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 >> 172.21.15.17:0/1257088436 conn(0x7fd4600ac590 msgr2=0x7fd4600ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 shutdown_connections 2024-03-06T07:22:46.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.338+0000 7fd4669d7640 1 -- 172.21.15.17:0/1257088436 wait complete. 2024-03-06T07:22:46.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:22:46.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.453+0000 7f176d02a640 1 Processor -- start 2024-03-06T07:22:46.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176d02a640 1 -- start start 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176d02a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 0x7f1768159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176d02a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1768155e70 con 0x7f176814bcc0 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176d02a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17680c75e0 con 0x7f17680be150 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176d02a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17680c7f10 con 0x7f1768155a00 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f1766d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f176814bcc0 0x7f17680c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39696/0 (socket says 172.21.15.17:39696) 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f1766d9d640 1 -- 172.21.15.17:0/1309746895 learned_addr learned my addr 172.21.15.17:0/1309746895 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.454+0000 7f176759e640 1 --2- 172.21.15.17:0/1309746895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 0x7f1768159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976562542 0 0) 0x7f1768155e70 con 0x7f176814bcc0 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1750003520 con 0x7f176814bcc0 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249116201 0 0) 0x7f17680c75e0 con 0x7f17680be150 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1768155e70 con 0x7f17680be150 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f176759e640 1 -- 172.21.15.17:0/1309746895 >> v1:172.21.15.163:6789/0 conn(0x7f17680be150 legacy=0x7f1768158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f176759e640 1 -- 172.21.15.17:0/1309746895 >> v1:172.21.15.17:6789/0 conn(0x7f176814bcc0 legacy=0x7f17680c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.455+0000 7f176759e640 1 -- 172.21.15.17:0/1309746895 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17680c78c0 con 0x7f1768155a00 2024-03-06T07:22:46.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176759e640 1 --2- 172.21.15.17:0/1309746895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 0x7f1768159120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1760009dc0 tx=0x7f176002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9ce18d40c61376 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760039070 con 0x7f1768155a00 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176002cd50 con 0x7f1768155a00 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f1765d9b640 1 -- 172.21.15.17:0/1309746895 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760034660 con 0x7f1768155a00 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 -- 172.21.15.17:0/1309746895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 msgr2=0x7f1768159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 --2- 172.21.15.17:0/1309746895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 0x7f1768159120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1760009dc0 tx=0x7f176002bfa0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 -- 172.21.15.17:0/1309746895 shutdown_connections 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 --2- 172.21.15.17:0/1309746895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1768155a00 0x7f1768159120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 -- 172.21.15.17:0/1309746895 >> 172.21.15.17:0/1309746895 conn(0x7f17680ac600 msgr2=0x7f17680aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 -- 172.21.15.17:0/1309746895 shutdown_connections 2024-03-06T07:22:46.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.456+0000 7f176d02a640 1 -- 172.21.15.17:0/1309746895 wait complete. 2024-03-06T07:22:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 Processor -- start 2024-03-06T07:22:46.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 -- start start 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17680be150 0x7f176814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f176814bcc0 0x7f17680c2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176806d1d0 con 0x7f1768155a00 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17680c5ae0 con 0x7f17680be150 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176d02a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17680c5c60 con 0x7f176814bcc0 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40820/0 (socket says 172.21.15.17:40820) 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176759e640 1 -- 172.21.15.17:0/694932996 learned_addr learned my addr 172.21.15.17:0/694932996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f1766d9d640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17680be150 0x7f176814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.458+0000 7f176659c640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f176814bcc0 0x7f17680c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 -- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f176814bcc0 msgr2=0x7f17680c2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f176814bcc0 0x7f17680c2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 -- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17680be150 msgr2=0x7f176814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17680be150 0x7f176814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 -- 172.21.15.17:0/694932996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17681ebc00 con 0x7f1768155a00 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176759e640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1760037040 tx=0x7f1760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760039050 con 0x7f1768155a00 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760045070 con 0x7f1768155a00 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760009450 con 0x7f1768155a00 2024-03-06T07:22:46.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17681ebee0 con 0x7f1768155a00 2024-03-06T07:22:46.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.459+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17681513b0 con 0x7f1768155a00 2024-03-06T07:22:46.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.460+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176804f330 con 0x7f1768155a00 2024-03-06T07:22:46.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.461+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f176003e850 con 0x7f1768155a00 2024-03-06T07:22:46.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.461+0000 7f174f7fe640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 0x7f175406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.461+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17600ad0e0 con 0x7f1768155a00 2024-03-06T07:22:46.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.461+0000 7f1766d9d640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 0x7f175406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.462+0000 7f1766d9d640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 0x7f175406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1758000f60 tx=0x7f1758007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.465+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f176007b8a0 con 0x7f1768155a00 2024-03-06T07:22:46.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.646+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17680c83e0 con 0x7f1768155a00 2024-03-06T07:22:46.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.646+0000 7f174f7fe640 1 -- 172.21.15.17:0/694932996 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f176007f940 con 0x7f1768155a00 2024-03-06T07:22:46.648 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 msgr2=0x7f175406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 0x7f175406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1758000f60 tx=0x7f1758007d70 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 msgr2=0x7f1768151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1760037040 tx=0x7f1760004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 shutdown_connections 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f175406b4d0 0x7f175406d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1768155a00 0x7f1768151bb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f176814bcc0 0x7f17680c2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 --2- 172.21.15.17:0/694932996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17680be150 0x7f176814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 >> 172.21.15.17:0/694932996 conn(0x7f17680ac600 msgr2=0x7f176806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 shutdown_connections 2024-03-06T07:22:46.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.648+0000 7f176d02a640 1 -- 172.21.15.17:0/694932996 wait complete. 2024-03-06T07:22:46.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:22:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.763+0000 7f28fdc8a640 1 Processor -- start 2024-03-06T07:22:46.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.763+0000 7f28fdc8a640 1 -- start start 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28f80c5990 0x7f28f80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28f80ab170 0x7f28f806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f81523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f806cba0 con 0x7f28f80c5990 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80c3ef0 con 0x7f28f80ab170 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28fdc8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81556d0 con 0x7f28f806c730 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28f80c5990 0x7f28f80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28f80c5990 0x7f28f80c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40822/0 (socket says 172.21.15.17:40822) 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 -- 172.21.15.17:0/2311250930 learned_addr learned my addr 172.21.15.17:0/2311250930 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f7fff640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f81523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f6ffd640 1 --2- 172.21.15.17:0/2311250930 >> v2:172.21.15.163:3300/0 conn(0x7f28f80ab170 0x7f28f806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 -- 172.21.15.17:0/2311250930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 msgr2=0x7f28f81523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:46.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f81523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 -- 172.21.15.17:0/2311250930 >> v2:172.21.15.163:3300/0 conn(0x7f28f80ab170 msgr2=0x7f28f806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2311250930 >> v2:172.21.15.163:3300/0 conn(0x7f28f80ab170 0x7f28f806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.764+0000 7f28f77fe640 1 -- 172.21.15.17:0/2311250930 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f81559b0 con 0x7f28f80c5990 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.765+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f80c0ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f28f806e600 tx=0x7f28e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f514d02cbbe61 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.765+0000 7f28f67fc640 1 -- 172.21.15.17:0/2311250930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8035020 con 0x7f28f80c5990 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.765+0000 7f28f67fc640 1 -- 172.21.15.17:0/2311250930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8031870 con 0x7f28f80c5990 2024-03-06T07:22:46.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.765+0000 7f28f67fc640 1 -- 172.21.15.17:0/2311250930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8031c40 con 0x7f28f80c5990 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2311250930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 msgr2=0x7f28f80c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f80c0ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f28f806e600 tx=0x7f28e802b910 comp rx=0 tx=0).stop 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2311250930 shutdown_connections 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f81523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2311250930 >> v2:172.21.15.163:3300/0 conn(0x7f28f80ab170 0x7f28f806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2311250930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f80c0ab0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2311250930 >> 172.21.15.17:0/2311250930 conn(0x7f28f80c6710 msgr2=0x7f28f80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2311250930 shutdown_connections 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.766+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2311250930 wait complete. 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.767+0000 7f28fdc8a640 1 Processor -- start 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.767+0000 7f28fdc8a640 1 -- start start 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f814acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f80ab170 0x7f28f8145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8156490 con 0x7f28f80c5990 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8155f70 con 0x7f28f80ab170 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28fdc8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8156610 con 0x7f28f806c730 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40834/0 (socket says 172.21.15.17:40834) 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f6ffd640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f80ab170 0x7f28f8145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f7fff640 1 -- 172.21.15.17:0/2319640901 learned_addr learned my addr 172.21.15.17:0/2319640901 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f814acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f7fff640 1 -- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 msgr2=0x7f28f814acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.768+0000 7f28f7fff640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f814acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28f7fff640 1 -- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f80ab170 msgr2=0x7f28f8145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28f7fff640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f80ab170 0x7f28f8145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28f7fff640 1 -- 172.21.15.17:0/2319640901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f814cf00 con 0x7f28f80c5990 2024-03-06T07:22:46.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28f7fff640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009d30 tx=0x7f28f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0018020 con 0x7f28f80c5990 2024-03-06T07:22:46.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0014480 con 0x7f28f80c5990 2024-03-06T07:22:46.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0014870 con 0x7f28f80c5990 2024-03-06T07:22:46.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f814cc60 con 0x7f28f80c5990 2024-03-06T07:22:46.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.769+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f814de10 con 0x7f28f80c5990 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.770+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f804f330 con 0x7f28f80c5990 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.770+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28f0005d10 con 0x7f28f80c5990 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.771+0000 7f28dffff640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 0x7f28e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.771+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28f008abc0 con 0x7f28f80c5990 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.771+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 0x7f28e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:46.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.771+0000 7f28f77fe640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 0x7f28e406d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036000 tx=0x7f28e803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:46.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.775+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28f0022070 con 0x7f28f80c5990 2024-03-06T07:22:46.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.954+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28f80c1740 con 0x7f28f80c5990 2024-03-06T07:22:46.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.955+0000 7f28dffff640 1 -- 172.21.15.17:0/2319640901 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28f0010040 con 0x7f28f80c5990 2024-03-06T07:22:46.957 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 msgr2=0x7f28e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 0x7f28e406d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28e8036000 tx=0x7f28e803c040 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 msgr2=0x7f28f81464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009d30 tx=0x7f28f000c490 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 shutdown_connections 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28e406b500 0x7f28e406d9c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f80c5990 0x7f28f81464e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f80ab170 0x7f28f8145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 --2- 172.21.15.17:0/2319640901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f806c730 0x7f28f814acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 >> 172.21.15.17:0/2319640901 conn(0x7f28f80c6710 msgr2=0x7f28f8148da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 shutdown_connections 2024-03-06T07:22:46.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:46.957+0000 7f28fdc8a640 1 -- 172.21.15.17:0/2319640901 wait complete. 2024-03-06T07:22:46.969 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:22:47.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.069+0000 7f13f3ce1640 1 Processor -- start 2024-03-06T07:22:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.069+0000 7f13f3ce1640 1 -- start start 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f3ce1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 0x7f13ec0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f3ce1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f13ec0c3950 0x7f13ec15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f3ce1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0bdc20 con 0x7f13ec072e70 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f3ce1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec06c8e0 con 0x7f13ec0c3950 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f3ce1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec06ce80 con 0x7f13ec073250 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f1a7d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f13ec072e70 0x7f13ec0bd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39710/0 (socket says 172.21.15.17:39710) 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f1a7d640 1 -- 172.21.15.17:0/2475857906 learned_addr learned my addr 172.21.15.17:0/2475857906 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f227e640 1 --2- 172.21.15.17:0/2475857906 >> v2:172.21.15.163:3300/0 conn(0x7f13ec0c3950 0x7f13ec15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f127c640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 0x7f13ec0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f0a7b640 1 -- 172.21.15.17:0/2475857906 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831725850 0 0) 0x7f13ec0bdc20 con 0x7f13ec072e70 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f0a7b640 1 -- 172.21.15.17:0/2475857906 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d8003520 con 0x7f13ec072e70 2024-03-06T07:22:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f227e640 1 -- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 msgr2=0x7f13ec0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f227e640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 0x7f13ec0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f227e640 1 -- 172.21.15.17:0/2475857906 >> v1:172.21.15.17:6789/0 conn(0x7f13ec072e70 legacy=0x7f13ec0bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f227e640 1 -- 172.21.15.17:0/2475857906 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0c1b00 con 0x7f13ec0c3950 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.070+0000 7f13f127c640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 0x7f13ec0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.071+0000 7f13f227e640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec0c3950 0x7f13ec15fce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009d90 tx=0x7f13e802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921876773fb3c672 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.071+0000 7f13f0a7b640 1 -- 172.21.15.17:0/2475857906 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8039070 con 0x7f13ec0c3950 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.071+0000 7f13f0a7b640 1 -- 172.21.15.17:0/2475857906 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e802cbc0 con 0x7f13ec0c3950 2024-03-06T07:22:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.071+0000 7f13f0a7b640 1 -- 172.21.15.17:0/2475857906 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8034440 con 0x7f13ec0c3950 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 -- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec0c3950 msgr2=0x7f13ec15fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec0c3950 0x7f13ec15fce0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009d90 tx=0x7f13e802be10 comp rx=0 tx=0).stop 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 -- 172.21.15.17:0/2475857906 shutdown_connections 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec0c3950 0x7f13ec15fce0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/2475857906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec073250 0x7f13ec0c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 -- 172.21.15.17:0/2475857906 >> 172.21.15.17:0/2475857906 conn(0x7f13ec06b1a0 msgr2=0x7f13ec06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 -- 172.21.15.17:0/2475857906 shutdown_connections 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.072+0000 7f13f3ce1640 1 -- 172.21.15.17:0/2475857906 wait complete. 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.073+0000 7f13f3ce1640 1 Processor -- start 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.073+0000 7f13f3ce1640 1 -- start start 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec072e70 0x7f13ec0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec073250 0x7f13ec1ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c6760 con 0x7f13ec0c3950 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c4a00 con 0x7f13ec073250 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1ef470 con 0x7f13ec072e70 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40850/0 (socket says 172.21.15.17:40850) 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f1a7d640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec072e70 0x7f13ec0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 -- 172.21.15.17:0/3184746246 learned_addr learned my addr 172.21.15.17:0/3184746246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f127c640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec073250 0x7f13ec1ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 -- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec072e70 msgr2=0x7f13ec0718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec072e70 0x7f13ec0718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 -- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec073250 msgr2=0x7f13ec1ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec073250 0x7f13ec1ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 -- 172.21.15.17:0/3184746246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec1eeef0 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f227e640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009eb0 tx=0x7f13e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8039050 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8045070 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.074+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec1ef190 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.075+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8009450 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.075+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec1eb440 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.075+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec04f330 con 0x7f13ec0c3950 2024-03-06T07:22:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.076+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13e803e830 con 0x7f13ec0c3950 2024-03-06T07:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.076+0000 7f13e27fc640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 0x7f13d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.077+0000 7f13f1a7d640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 0x7f13d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.077+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13e80ad110 con 0x7f13ec0c3950 2024-03-06T07:22:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.077+0000 7f13f1a7d640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 0x7f13d006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005880 tx=0x7f13e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.081+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f13e807b8f0 con 0x7f13ec0c3950 2024-03-06T07:22:47.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.263+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f13ec0c6340 con 0x7f13ec0c3950 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.263+0000 7f13e27fc640 1 -- 172.21.15.17:0/3184746246 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+209 (secure 0 0 0) 0x7f13e807f990 con 0x7f13ec0c3950 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 11, 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 12, 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_data" 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:22:47.266 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 msgr2=0x7f13d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 0x7f13d006d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005880 tx=0x7f13e4009040 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 msgr2=0x7f13ec14d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009eb0 tx=0x7f13e8004a40 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 shutdown_connections 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f13d006b4d0 0x7f13d006d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0c3950 0x7f13ec14d930 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13ec073250 0x7f13ec1ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 --2- 172.21.15.17:0/3184746246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13ec072e70 0x7f13ec0718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 >> 172.21.15.17:0/3184746246 conn(0x7f13ec06b1a0 msgr2=0x7f13ec1521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 shutdown_connections 2024-03-06T07:22:47.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.265+0000 7f13f3ce1640 1 -- 172.21.15.17:0/3184746246 wait complete. 2024-03-06T07:22:47.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:22:47.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:22:47.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.381+0000 7f452ae76640 1 Processor -- start 2024-03-06T07:22:47.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.381+0000 7f452ae76640 1 -- start start 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.381+0000 7f452ae76640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.381+0000 7f452ae76640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4524153390 0x7f4524153820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f452ae76640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452406e850 con 0x7f45240c22b0 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f452ae76640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452406d2b0 con 0x7f4524153390 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f452ae76640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452406d590 con 0x7f45240a9830 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4529413640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4524153390 0x7f4524153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:41726/0 (socket says 172.21.15.17:41726) 2024-03-06T07:22:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4528c12640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f45240c22b0 0x7f452414ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:47024/0 (socket says 172.21.15.17:47024) 2024-03-06T07:22:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 -- 172.21.15.17:0/885746496 learned_addr learned my addr 172.21.15.17:0/885746496 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 -- 172.21.15.17:0/885746496 >> v2:172.21.15.163:3300/0 conn(0x7f4524153390 msgr2=0x7f4524153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 --2- 172.21.15.17:0/885746496 >> v2:172.21.15.163:3300/0 conn(0x7f4524153390 0x7f4524153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 -- 172.21.15.17:0/885746496 >> v1:172.21.15.163:6789/0 conn(0x7f45240c22b0 legacy=0x7f452414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.382+0000 7f4523fff640 1 -- 172.21.15.17:0/885746496 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452406d870 con 0x7f45240a9830 2024-03-06T07:22:47.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f4523fff640 1 --2- 172.21.15.17:0/885746496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4518009870 tx=0x7f451802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dde1dfdeb8f5e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f45237fe640 1 -- 172.21.15.17:0/885746496 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518034070 con 0x7f45240a9830 2024-03-06T07:22:47.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f45237fe640 1 -- 172.21.15.17:0/885746496 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4518030840 con 0x7f45240a9830 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f45237fe640 1 -- 172.21.15.17:0/885746496 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518030c10 con 0x7f45240a9830 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f452ae76640 1 -- 172.21.15.17:0/885746496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 msgr2=0x7f4524151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.383+0000 7f452ae76640 1 --2- 172.21.15.17:0/885746496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4518009870 tx=0x7f451802b980 comp rx=0 tx=0).stop 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 -- 172.21.15.17:0/885746496 shutdown_connections 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 --2- 172.21.15.17:0/885746496 >> v2:172.21.15.163:3300/0 conn(0x7f4524153390 0x7f4524153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 --2- 172.21.15.17:0/885746496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524151e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 -- 172.21.15.17:0/885746496 >> 172.21.15.17:0/885746496 conn(0x7f45240c65b0 msgr2=0x7f45240c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 -- 172.21.15.17:0/885746496 shutdown_connections 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.384+0000 7f452ae76640 1 -- 172.21.15.17:0/885746496 wait complete. 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 Processor -- start 2024-03-06T07:22:47.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 -- start start 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45240c22b0 0x7f4524144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452414f7e0 con 0x7f4524153390 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452414f660 con 0x7f45240c22b0 2024-03-06T07:22:47.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.385+0000 7f452ae76640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45240c2690 con 0x7f45240a9830 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40854/0 (socket says 172.21.15.17:40854) 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 -- 172.21.15.17:0/2993069565 learned_addr learned my addr 172.21.15.17:0/2993069565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4523fff640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45240c22b0 0x7f4524144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4528c12640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 -- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 msgr2=0x7f4524146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 -- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45240c22b0 msgr2=0x7f4524144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45240c22b0 0x7f4524144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 -- 172.21.15.17:0/2993069565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524145260 con 0x7f4524153390 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f4529413640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f451c00efc0 tx=0x7f451c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c018020 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c014480 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4524071f30 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c014870 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.386+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45240724f0 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.387+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452404f330 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.388+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f451c005d10 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.388+0000 7f45217fa640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 0x7f451006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.388+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f451c08b0b0 con 0x7f4524153390 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.388+0000 7f4528c12640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 0x7f451006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:47.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.389+0000 7f4528c12640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 0x7f451006d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4514007fd0 tx=0x7f4514007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:47.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.393+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f451c022030 con 0x7f4524153390 2024-03-06T07:22:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:47.573+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f45240733f0 con 0x7f4524153390 2024-03-06T07:22:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.416+0000 7f45217fa640 1 -- 172.21.15.17:0/2993069565 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v76) v1 ==== 140+0+0 (secure 0 0 0) 0x7f4524145260 con 0x7f4524153390 2024-03-06T07:22:48.418 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 msgr2=0x7f451006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 0x7f451006d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4514007fd0 tx=0x7f4514007e80 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 msgr2=0x7f45241df630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f451c00efc0 tx=0x7f451c00c490 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 shutdown_connections 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451006b4d0 0x7f451006d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4524153390 0x7f45241df630 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45240c22b0 0x7f4524144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 --2- 172.21.15.17:0/2993069565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45240a9830 0x7f4524146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 >> 172.21.15.17:0/2993069565 conn(0x7f45240c65b0 msgr2=0x7f4524153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 shutdown_connections 2024-03-06T07:22:48.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.420+0000 7f452ae76640 1 -- 172.21.15.17:0/2993069565 wait complete. 2024-03-06T07:22:48.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:22:48.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.551+0000 7fa873288640 1 Processor -- start 2024-03-06T07:22:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.551+0000 7fa873288640 1 -- start start 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa873288640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa86c073290 0x7fa86c155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa873288640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c152db0 con 0x7fa86c1529d0 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa873288640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c1590f0 con 0x7fa86c072cd0 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa873288640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0c6dd0 con 0x7fa86c073290 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871825640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa86c073290 0x7fa86c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871024640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa86c1529d0 0x7fa86c0c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39726/0 (socket says 172.21.15.17:39726) 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871024640 1 -- 172.21.15.17:0/1329850011 learned_addr learned my addr 172.21.15.17:0/1329850011 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871825640 1 --2- 172.21.15.17:0/1329850011 >> v2:172.21.15.17:3300/0 conn(0x7fa86c073290 0x7fa86c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40860/0 (socket says 172.21.15.17:40860) 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871825640 1 -- 172.21.15.17:0/1329850011 >> v1:172.21.15.163:6789/0 conn(0x7fa86c072cd0 legacy=0x7fa86c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871825640 1 -- 172.21.15.17:0/1329850011 >> v1:172.21.15.17:6789/0 conn(0x7fa86c1529d0 legacy=0x7fa86c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa863fff640 1 -- 172.21.15.17:0/1329850011 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599327382 0 0) 0x7fa864001110 con 0x7fa86c1529d0 2024-03-06T07:22:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.552+0000 7fa871825640 1 -- 172.21.15.17:0/1329850011 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c159650 con 0x7fa86c073290 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.553+0000 7fa871825640 1 --2- 172.21.15.17:0/1329850011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c155830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa868009d90 tx=0x7fa86802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302c4d159840fd6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.553+0000 7fa863fff640 1 -- 172.21.15.17:0/1329850011 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868039070 con 0x7fa86c073290 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.553+0000 7fa863fff640 1 -- 172.21.15.17:0/1329850011 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86802cbd0 con 0x7fa86c073290 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.553+0000 7fa863fff640 1 -- 172.21.15.17:0/1329850011 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868034440 con 0x7fa86c073290 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 -- 172.21.15.17:0/1329850011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 msgr2=0x7fa86c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 --2- 172.21.15.17:0/1329850011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c155830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa868009d90 tx=0x7fa86802be20 comp rx=0 tx=0).stop 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 -- 172.21.15.17:0/1329850011 shutdown_connections 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 --2- 172.21.15.17:0/1329850011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c155830 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 -- 172.21.15.17:0/1329850011 >> 172.21.15.17:0/1329850011 conn(0x7fa86c0aa630 msgr2=0x7fa86c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 -- 172.21.15.17:0/1329850011 shutdown_connections 2024-03-06T07:22:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.554+0000 7fa873288640 1 -- 172.21.15.17:0/1329850011 wait complete. 2024-03-06T07:22:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.555+0000 7fa873288640 1 Processor -- start 2024-03-06T07:22:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.555+0000 7fa873288640 1 -- start start 2024-03-06T07:22:48.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86c072cd0 0x7fa86c1eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa86c1529d0 0x7fa86c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c155d70 con 0x7fa86c073290 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0c6940 con 0x7fa86c1529d0 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c159370 con 0x7fa86c072cd0 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40864/0 (socket says 172.21.15.17:40864) 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 -- 172.21.15.17:0/1071790952 learned_addr learned my addr 172.21.15.17:0/1071790952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa871825640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa86c1529d0 0x7fa86c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa871024640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86c072cd0 0x7fa86c1eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 -- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86c072cd0 msgr2=0x7fa86c1eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86c072cd0 0x7fa86c1eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 -- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa86c1529d0 msgr2=0x7fa86c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa86c1529d0 0x7fa86c14bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 -- 172.21.15.17:0/1071790952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c154f30 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa870823640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c400 tx=0x7fa85c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c017070 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c0052d0 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0ab930 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.556+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c003830 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.557+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0abef0 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.558+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0694c0 con 0x7fa86c073290 2024-03-06T07:22:48.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.558+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa85c01a070 con 0x7fa86c073290 2024-03-06T07:22:48.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.558+0000 7fa861ffb640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 0x7fa85006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.558+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6472+0+0 (secure 0 0 0) 0x7fa85c089740 con 0x7fa86c073290 2024-03-06T07:22:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.558+0000 7fa871024640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 0x7fa85006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:48.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.559+0000 7fa871024640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 0x7fa85006d9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f60 tx=0x7fa864007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:48.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.563+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa86c0694c0 con 0x7fa86c073290 2024-03-06T07:22:48.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:48.747+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa86c04f330 con 0x7fa86c073290 2024-03-06T07:22:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.421+0000 7fa861ffb640 1 -- 172.21.15.17:0/1071790952 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v77) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa86c1590f0 con 0x7fa86c073290 2024-03-06T07:22:49.423 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' created 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 msgr2=0x7fa85006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 0x7fa85006d9c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa864000f60 tx=0x7fa864007e40 comp rx=0 tx=0).stop 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 msgr2=0x7fa86c14ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c400 tx=0x7fa85c00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 shutdown_connections 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa85006b500 0x7fa85006d9c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa86c1529d0 0x7fa86c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa86c073290 0x7fa86c14ba80 secure :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c400 tx=0x7fa85c00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 --2- 172.21.15.17:0/1071790952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86c072cd0 0x7fa86c1eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.426+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 >> 172.21.15.17:0/1071790952 conn(0x7fa86c0aa630 msgr2=0x7fa86c06e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.433+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 shutdown_connections 2024-03-06T07:22:49.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.433+0000 7fa873288640 1 -- 172.21.15.17:0/1071790952 wait complete. 2024-03-06T07:22:49.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:22:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.565+0000 7fcdc922a640 1 Processor -- start 2024-03-06T07:22:49.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.565+0000 7fcdc922a640 1 -- start start 2024-03-06T07:22:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc922a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 0x7fcdc40c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc922a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc922a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc4155260 con 0x7fcdc41527f0 2024-03-06T07:22:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc922a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40c8f70 con 0x7fcdc40c7370 2024-03-06T07:22:49.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc922a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4155540 con 0x7fcdc406c0b0 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:41746/0 (socket says 172.21.15.17:41746) 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 0x7fcdc40c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc359e640 1 -- 172.21.15.17:0/3169942932 learned_addr learned my addr 172.21.15.17:0/3169942932 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.566+0000 7fcdc2d9d640 1 --1- 172.21.15.17:0/3169942932 >> v1:172.21.15.163:6789/0 conn(0x7fcdc41527f0 0x7fcdc40c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:47042/0 (socket says 172.21.15.17:47042) 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc359e640 1 -- 172.21.15.17:0/3169942932 >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 msgr2=0x7fcdc40c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc359e640 1 --2- 172.21.15.17:0/3169942932 >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 0x7fcdc40c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc359e640 1 -- 172.21.15.17:0/3169942932 >> v1:172.21.15.163:6789/0 conn(0x7fcdc41527f0 legacy=0x7fcdc40c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc359e640 1 -- 172.21.15.17:0/3169942932 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4155820 con 0x7fcdc406c0b0 2024-03-06T07:22:49.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc259c640 1 --2- 172.21.15.17:0/3169942932 >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 0x7fcdc40c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.567+0000 7fcdc359e640 1 --2- 172.21.15.17:0/3169942932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009e50 tx=0x7fcdb402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388764f29fcc4487 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc1d9b640 1 -- 172.21.15.17:0/3169942932 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402c9e0 con 0x7fcdc406c0b0 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc1d9b640 1 -- 172.21.15.17:0/3169942932 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb402cb80 con 0x7fcdc406c0b0 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc1d9b640 1 -- 172.21.15.17:0/3169942932 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40343f0 con 0x7fcdc406c0b0 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 -- 172.21.15.17:0/3169942932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 msgr2=0x7fcdc406c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 --2- 172.21.15.17:0/3169942932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009e50 tx=0x7fcdb402be40 comp rx=0 tx=0).stop 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 -- 172.21.15.17:0/3169942932 shutdown_connections 2024-03-06T07:22:49.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 --2- 172.21.15.17:0/3169942932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc406c0b0 0x7fcdc406c4e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 --2- 172.21.15.17:0/3169942932 >> v2:172.21.15.163:3300/0 conn(0x7fcdc40c7370 0x7fcdc40c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 -- 172.21.15.17:0/3169942932 >> 172.21.15.17:0/3169942932 conn(0x7fcdc40ac4f0 msgr2=0x7fcdc40ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 -- 172.21.15.17:0/3169942932 shutdown_connections 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.568+0000 7fcdc922a640 1 -- 172.21.15.17:0/3169942932 wait complete. 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 Processor -- start 2024-03-06T07:22:49.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 -- start start 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdc40c7370 0x7fcdc4148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc41527f0 0x7fcdc4149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc406ca20 con 0x7fcdc406c0b0 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40be1a0 con 0x7fcdc40c7370 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc922a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc406cba0 con 0x7fcdc41527f0 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40870/0 (socket says 172.21.15.17:40870) 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc2d9d640 1 -- 172.21.15.17:0/2066990937 learned_addr learned my addr 172.21.15.17:0/2066990937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc259c640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdc40c7370 0x7fcdc4148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.570+0000 7fcdc359e640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc41527f0 0x7fcdc4149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 -- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc41527f0 msgr2=0x7fcdc4149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc41527f0 0x7fcdc4149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 -- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdc40c7370 msgr2=0x7fcdc4148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdc40c7370 0x7fcdc4148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 -- 172.21.15.17:0/2066990937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc41539f0 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc2d9d640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00c3f0 tx=0x7fcdbc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0074e0 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc007680 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc4153150 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc004a00 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.571+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc4153710 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.572+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc404f330 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.573+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdbc01a020 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.573+0000 7fcdbb7fe640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 0x7fcda406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.573+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6855+0+0 (secure 0 0 0) 0x7fcdbc089be0 con 0x7fcdc406c0b0 2024-03-06T07:22:49.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.573+0000 7fcdc259c640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 0x7fcda406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:49.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.574+0000 7fcdc259c640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 0x7fcda406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009560 tx=0x7fcdac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:49.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.578+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcdbc058020 con 0x7fcdc406c0b0 2024-03-06T07:22:49.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:49.801+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7fcdc406d860 con 0x7fcdc406c0b0 2024-03-06T07:22:50.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.423+0000 7fcdbb7fe640 1 -- 172.21.15.17:0/2066990937 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 13 and data pool 14 v57) v1 ==== 177+0+0 (secure 0 0 0) 0x7fcdbc05c0c0 con 0x7fcdc406c0b0 2024-03-06T07:22:50.425 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 13 and data pool 14 2024-03-06T07:22:50.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.428+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 msgr2=0x7fcda406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.428+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 0x7fcda406d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009560 tx=0x7fcdac009190 comp rx=0 tx=0).stop 2024-03-06T07:22:50.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.428+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 msgr2=0x7fcdc414d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.428+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00c3f0 tx=0x7fcdbc00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 shutdown_connections 2024-03-06T07:22:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcda406b4d0 0x7fcda406d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcdc41527f0 0x7fcdc4149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcdc40c7370 0x7fcdc4148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 --2- 172.21.15.17:0/2066990937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc406c0b0 0x7fcdc414d0a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.430+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 >> 172.21.15.17:0/2066990937 conn(0x7fcdc40ac4f0 msgr2=0x7fcdc414e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:50.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.435+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 shutdown_connections 2024-03-06T07:22:50.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.438+0000 7fcdc922a640 1 -- 172.21.15.17:0/2066990937 wait complete. 2024-03-06T07:22:50.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:50.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.570+0000 7fa283e0e640 1 Processor -- start 2024-03-06T07:22:50.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.570+0000 7fa283e0e640 1 -- start start 2024-03-06T07:22:50.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa283e0e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa27c155ba0 0x7fa27c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa283e0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c1591c0 con 0x7fa27c0c5480 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa283e0e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c1594a0 con 0x7fa27c1586f0 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa283e0e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c06df40 con 0x7fa27c155ba0 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa281baa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa27c0c5480 0x7fa27c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50368/0 (socket says 172.21.15.17:50368) 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa281baa640 1 -- 172.21.15.17:0/2845176459 learned_addr learned my addr 172.21.15.17:0/2845176459 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa2813a9640 1 --2- 172.21.15.17:0/2845176459 >> v2:172.21.15.163:3300/0 conn(0x7fa27c155ba0 0x7fa27c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.571+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981964320 0 0) 0x7fa27c1591c0 con 0x7fa27c0c5480 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa264003520 con 0x7fa27c0c5480 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850534742 0 0) 0x7fa27c1594a0 con 0x7fa27c1586f0 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa27c1591c0 con 0x7fa27c1586f0 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 819666721 0 0) 0x7fa264003520 con 0x7fa27c0c5480 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 >> v2:172.21.15.163:3300/0 conn(0x7fa27c155ba0 msgr2=0x7fa27c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 --2- 172.21.15.17:0/2845176459 >> v2:172.21.15.163:3300/0 conn(0x7fa27c155ba0 0x7fa27c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 >> v1:172.21.15.163:6789/0 conn(0x7fa27c1586f0 legacy=0x7fa27c15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c1553a0 con 0x7fa27c0c5480 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fa2740019f0 con 0x7fa27c0c5480 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa2813a9640 1 --2- 172.21.15.17:0/2845176459 >> v2:172.21.15.163:3300/0 conn(0x7fa27c155ba0 0x7fa27c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 >> v1:172.21.15.17:6789/0 conn(0x7fa27c0c5480 legacy=0x7fa27c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 0x7fa264003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c06df40 con 0x7fa264003b40 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26400c0f0 con 0x7fa264004800 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26400c2f0 con 0x7fa2640051c0 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 0x7fa264003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa2813a9640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa2823ab640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 -- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2640051c0 msgr2=0x7fa26400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:50.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2640051c0 0x7fa26400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 -- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa264004800 msgr2=0x7fa264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa2823ab640 1 -- 172.21.15.17:0/2845176459 reap_dead start 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa281baa640 1 -- 172.21.15.17:0/2845176459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa264003520 con 0x7fa264003b40 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.572+0000 7fa2813a9640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa264004800 0x7fa264004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.573+0000 7fa281baa640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 0x7fa264003f40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa27400a810 tx=0x7fa264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7bd5ded03e53e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.573+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400a430 con 0x7fa264003b40 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.573+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa274034650 con 0x7fa264003b40 2024-03-06T07:22:50.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.573+0000 7fa280ba8640 1 -- 172.21.15.17:0/2845176459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274034a80 con 0x7fa264003b40 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 -- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 msgr2=0x7fa264003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 0x7fa264003f40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa27400a810 tx=0x7fa264005830 comp rx=0 tx=0).stop 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 -- 172.21.15.17:0/2845176459 shutdown_connections 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 --2- 172.21.15.17:0/2845176459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa264003b40 0x7fa264003f40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 -- 172.21.15.17:0/2845176459 >> 172.21.15.17:0/2845176459 conn(0x7fa27c06b1a0 msgr2=0x7fa27c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 -- 172.21.15.17:0/2845176459 shutdown_connections 2024-03-06T07:22:50.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.574+0000 7fa283e0e640 1 -- 172.21.15.17:0/2845176459 wait complete. 2024-03-06T07:22:50.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.575+0000 7fa283e0e640 1 Processor -- start 2024-03-06T07:22:50.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.575+0000 7fa283e0e640 1 -- start start 2024-03-06T07:22:50.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.575+0000 7fa283e0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa264003b40 0x7fa27c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.575+0000 7fa283e0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.575+0000 7fa283e0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa27c0a8520 0x7fa27c150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa283e0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c156000 con 0x7fa27c0a80a0 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa283e0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c06fdb0 con 0x7fa27c0a8520 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa283e0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c152a20 con 0x7fa264003b40 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38106/0 (socket says 172.21.15.17:38106) 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 -- 172.21.15.17:0/3587025509 learned_addr learned my addr 172.21.15.17:0/3587025509 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2823ab640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa27c0a8520 0x7fa27c150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa281baa640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa264003b40 0x7fa27c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 -- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa264003b40 msgr2=0x7fa27c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa264003b40 0x7fa27c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 -- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa27c0a8520 msgr2=0x7fa27c150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa27c0a8520 0x7fa27c150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 -- 172.21.15.17:0/3587025509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c0c43e0 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa2813a9640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa270011c00 tx=0x7fa27000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270019070 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa270004030 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c151730 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270009860 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.576+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c0706e0 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.577+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c04f330 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.578+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa27001c070 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.578+0000 7fa27a7fc640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 0x7fa26806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.578+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fa27008ab40 con 0x7fa27c0a80a0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.578+0000 7fa281baa640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 0x7fa26806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.579+0000 7fa281baa640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 0x7fa26806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa274034000 tx=0x7fa274009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.582+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa270058f20 con 0x7fa27c0a80a0 2024-03-06T07:22:50.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.765+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa27c0c2670 con 0x7fa27c0a80a0 2024-03-06T07:22:50.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.766+0000 7fa27a7fc640 1 -- 172.21.15.17:0/3587025509 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+20682 (secure 0 0 0) 0x7fa27005cfc0 con 0x7fa27c0a80a0 2024-03-06T07:22:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:50.768 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:49.803052+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 msgr2=0x7fa26806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 0x7fa26806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa274034000 tx=0x7fa274009030 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 msgr2=0x7fa27c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa270011c00 tx=0x7fa27000f610 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 shutdown_connections 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa26806b4d0 0x7fa26806d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa27c0a8520 0x7fa27c150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa27c0a80a0 0x7fa27c150820 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 --2- 172.21.15.17:0/3587025509 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa264003b40 0x7fa27c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 >> 172.21.15.17:0/3587025509 conn(0x7fa27c06b1a0 msgr2=0x7fa27c1537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 shutdown_connections 2024-03-06T07:22:50.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.769+0000 7fa283e0e640 1 -- 172.21.15.17:0/3587025509 wait complete. 2024-03-06T07:22:50.781 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:50.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 Processor -- start 2024-03-06T07:22:50.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 -- start start 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980c73d0 con 0x7f3a98158b60 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980be2e0 con 0x7f3a9814f360 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9efac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98159700 con 0x7f3a980c88c0 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9d549640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3a98158b60 0x7f3a98158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50372/0 (socket says 172.21.15.17:50372) 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.881+0000 7f3a9d549640 1 -- 172.21.15.17:0/1005313266 learned_addr learned my addr 172.21.15.17:0/1005313266 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:50.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a97fff640 1 --2- 172.21.15.17:0/1005313266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546704241 0 0) 0x7f3a980c73d0 con 0x7f3a98158b60 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a88003520 con 0x7f3a98158b60 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318774614 0 0) 0x7f3a980be2e0 con 0x7f3a9814f360 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a980c73d0 con 0x7f3a9814f360 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a97fff640 1 -- 172.21.15.17:0/1005313266 >> v1:172.21.15.163:6789/0 conn(0x7f3a9814f360 legacy=0x7f3a980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a97fff640 1 -- 172.21.15.17:0/1005313266 >> v1:172.21.15.17:6789/0 conn(0x7f3a98158b60 legacy=0x7f3a98158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.882+0000 7f3a97fff640 1 -- 172.21.15.17:0/1005313266 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980c7670 con 0x7f3a980c88c0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a97fff640 1 --2- 172.21.15.17:0/1005313266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98155c10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097d0 tx=0x7f3a8402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f324388bd170241 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84035020 con 0x7f3a980c88c0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84031800 con 0x7f3a980c88c0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a977fe640 1 -- 172.21.15.17:0/1005313266 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84031bb0 con 0x7f3a980c88c0 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 -- 172.21.15.17:0/1005313266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 msgr2=0x7f3a98155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 --2- 172.21.15.17:0/1005313266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98155c10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097d0 tx=0x7f3a8402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 -- 172.21.15.17:0/1005313266 shutdown_connections 2024-03-06T07:22:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 --2- 172.21.15.17:0/1005313266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98155c10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 -- 172.21.15.17:0/1005313266 >> 172.21.15.17:0/1005313266 conn(0x7f3a980ac570 msgr2=0x7f3a980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 -- 172.21.15.17:0/1005313266 shutdown_connections 2024-03-06T07:22:50.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.883+0000 7f3a9efac640 1 -- 172.21.15.17:0/1005313266 wait complete. 2024-03-06T07:22:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.884+0000 7f3a9efac640 1 Processor -- start 2024-03-06T07:22:50.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 -- start start 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a9814f360 0x7f3a98144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980c7c30 con 0x7f3a98158b60 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980c5a50 con 0x7f3a9814f360 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9efac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980c5bd0 con 0x7f3a980c88c0 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9d549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9d549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38118/0 (socket says 172.21.15.17:38118) 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9d549640 1 -- 172.21.15.17:0/2090862854 learned_addr learned my addr 172.21.15.17:0/2090862854 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a97fff640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a9814f360 0x7f3a98144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.885+0000 7f3a9cd48640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 -- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 msgr2=0x7f3a98154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 -- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a9814f360 msgr2=0x7f3a98144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a9814f360 0x7f3a98144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 -- 172.21.15.17:0/2090862854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98152730 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9d549640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000cba0 tx=0x7f3a9000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9000ace0 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a98152a10 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90012620 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.886+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a980ac0a0 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.887+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a980cb2d0 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.888+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a9001a070 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.888+0000 7f3a957fa640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 0x7f3a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.888+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f3a9008a470 con 0x7f3a98158b60 2024-03-06T07:22:50.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.888+0000 7f3a9cd48640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 0x7f3a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:50.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.889+0000 7f3a9cd48640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 0x7f3a7c06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007fd0 tx=0x7f3a8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:50.892+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a980cb2d0 con 0x7f3a98158b60 2024-03-06T07:22:51.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.076+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a9804f330 con 0x7f3a98158b60 2024-03-06T07:22:51.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.077+0000 7f3a957fa640 1 -- 172.21.15.17:0/2090862854 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+20682 (secure 0 0 0) 0x7f3a90081e20 con 0x7f3a98158b60 2024-03-06T07:22:51.079 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:51.079 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:49.803052+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:51.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 msgr2=0x7f3a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 0x7f3a7c06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007fd0 tx=0x7f3a8c009040 comp rx=0 tx=0).stop 2024-03-06T07:22:51.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 msgr2=0x7f3a98144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000cba0 tx=0x7f3a9000a430 comp rx=0 tx=0).stop 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 shutdown_connections 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a7c06b4d0 0x7f3a7c06d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a98158b60 0x7f3a98144ed0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a9814f360 0x7f3a98144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 --2- 172.21.15.17:0/2090862854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a980c88c0 0x7f3a98154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 >> 172.21.15.17:0/2090862854 conn(0x7f3a980ac570 msgr2=0x7f3a9806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.079+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 shutdown_connections 2024-03-06T07:22:51.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.080+0000 7f3a9efac640 1 -- 172.21.15.17:0/2090862854 wait complete. 2024-03-06T07:22:51.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:22:51.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.195+0000 7f89395a1640 1 Processor -- start 2024-03-06T07:22:51.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.195+0000 7f89395a1640 1 -- start start 2024-03-06T07:22:51.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89395a1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89340c8ff0 0x7f89340c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89395a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340b04b0 con 0x7f893406bff0 2024-03-06T07:22:51.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89395a1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893406d210 con 0x7f89340b00d0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89395a1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8934156160 con 0x7f89340c8ff0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89327fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f893406bff0 0x7f89340c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50376/0 (socket says 172.21.15.17:50376) 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89327fc640 1 -- 172.21.15.17:0/1874015926 learned_addr learned my addr 172.21.15.17:0/1874015926 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89337fe640 1 --2- 172.21.15.17:0/1874015926 >> v2:172.21.15.163:3300/0 conn(0x7f89340c8ff0 0x7f89340c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384760223 0 0) 0x7f89340b04b0 con 0x7f893406bff0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003520 con 0x7f893406bff0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505231693 0 0) 0x7f893406d210 con 0x7f89340b00d0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89340b04b0 con 0x7f89340b00d0 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89337fe640 1 -- 172.21.15.17:0/1874015926 >> v1:172.21.15.163:6789/0 conn(0x7f89340b00d0 legacy=0x7f893406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.196+0000 7f89337fe640 1 -- 172.21.15.17:0/1874015926 >> v1:172.21.15.17:6789/0 conn(0x7f893406bff0 legacy=0x7f89340c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.197+0000 7f89337fe640 1 -- 172.21.15.17:0/1874015926 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934155e80 con 0x7f89340c8ff0 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.197+0000 7f89337fe640 1 --2- 172.21.15.17:0/1874015926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c7920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f89341562e0 tx=0x7f892402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b72bf4a670e8b4b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.197+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924039070 con 0x7f89340c8ff0 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.197+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892402cce0 con 0x7f89340c8ff0 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.197+0000 7f8931ffb640 1 -- 172.21.15.17:0/1874015926 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924034640 con 0x7f89340c8ff0 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 -- 172.21.15.17:0/1874015926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 msgr2=0x7f89340c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 --2- 172.21.15.17:0/1874015926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c7920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f89341562e0 tx=0x7f892402bf30 comp rx=0 tx=0).stop 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 -- 172.21.15.17:0/1874015926 shutdown_connections 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 --2- 172.21.15.17:0/1874015926 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c7920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 -- 172.21.15.17:0/1874015926 >> 172.21.15.17:0/1874015926 conn(0x7f89340ac810 msgr2=0x7f89340acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 -- 172.21.15.17:0/1874015926 shutdown_connections 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.198+0000 7f89395a1640 1 -- 172.21.15.17:0/1874015926 wait complete. 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.199+0000 7f89395a1640 1 Processor -- start 2024-03-06T07:22:51.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.199+0000 7f89395a1640 1 -- start start 2024-03-06T07:22:51.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b00d0 0x7f89340c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8934152c50 con 0x7f893406bff0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340c5710 con 0x7f89340c8ff0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89395a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340c5ba0 con 0x7f89340b00d0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38130/0 (socket says 172.21.15.17:38130) 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 -- 172.21.15.17:0/2164234729 learned_addr learned my addr 172.21.15.17:0/2164234729 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89327fc640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b00d0 0x7f89340c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f89337fe640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 -- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b00d0 msgr2=0x7f89340c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b00d0 0x7f89340c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 -- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 msgr2=0x7f89340c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.200+0000 7f8932ffd640 1 -- 172.21.15.17:0/2164234729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340c4f20 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f8932ffd640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c530 tx=0x7f892c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c0052d0 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c005490 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893414ef90 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c012870 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.201+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340c2c30 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.202+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893404f330 con 0x7f893406bff0 2024-03-06T07:22:51.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.202+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892c012a10 con 0x7f893406bff0 2024-03-06T07:22:51.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.203+0000 7f891b7fe640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 0x7f892006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.203+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f892c08a180 con 0x7f893406bff0 2024-03-06T07:22:51.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.203+0000 7f89327fc640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 0x7f892006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.203+0000 7f89327fc640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 0x7f892006d9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89340c42c0 tx=0x7f8928007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.207+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f892c0584e0 con 0x7f893406bff0 2024-03-06T07:22:51.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.431+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f893406ce50 con 0x7f893406bff0 2024-03-06T07:22:51.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.444+0000 7f891b7fe640 1 -- 172.21.15.17:0/2164234729 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v58) v1 ==== 119+0+0 (secure 0 0 0) 0x7f892c05c580 con 0x7f893406bff0 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 msgr2=0x7f892006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 0x7f892006d9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f89340c42c0 tx=0x7f8928007d30 comp rx=0 tx=0).stop 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 msgr2=0x7f893414f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f892c00c530 tx=0x7f892c00c9f0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 shutdown_connections 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b500 0x7f892006d9c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c8ff0 0x7f89340c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b00d0 0x7f89340c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 --2- 172.21.15.17:0/2164234729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f893406bff0 0x7f893414f270 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 >> 172.21.15.17:0/2164234729 conn(0x7f89340ac810 msgr2=0x7f893406b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 shutdown_connections 2024-03-06T07:22:51.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.446+0000 7f89395a1640 1 -- 172.21.15.17:0/2164234729 wait complete. 2024-03-06T07:22:51.459 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:22:51.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:22:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.561+0000 7fac9e4df640 1 Processor -- start 2024-03-06T07:22:51.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.562+0000 7fac9e4df640 1 -- start start 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9e4df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9e4df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac980c7710 0x7fac980c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9e4df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac98155ed0 con 0x7fac98152770 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9e4df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac980c6a00 con 0x7fac980c7710 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9e4df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac980c6c30 con 0x7fac980c12a0 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:33010/0 (socket says 172.21.15.17:33010) 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac9ca7c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac980c7710 0x7fac980c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac97fff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fac98152770 0x7fac9806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:60976/0 (socket says 172.21.15.17:60976) 2024-03-06T07:22:51.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 -- 172.21.15.17:0/2196185271 learned_addr learned my addr 172.21.15.17:0/2196185271 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac96ffd640 1 -- 172.21.15.17:0/2196185271 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753114010 0 0) 0x7fac98155ed0 con 0x7fac98152770 2024-03-06T07:22:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 -- 172.21.15.17:0/2196185271 >> v2:172.21.15.163:3300/0 conn(0x7fac980c7710 msgr2=0x7fac980c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 --2- 172.21.15.17:0/2196185271 >> v2:172.21.15.163:3300/0 conn(0x7fac980c7710 0x7fac980c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 -- 172.21.15.17:0/2196185271 >> v1:172.21.15.163:6789/0 conn(0x7fac98152770 legacy=0x7fac9806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.563+0000 7fac977fe640 1 -- 172.21.15.17:0/2196185271 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980c6f10 con 0x7fac980c12a0 2024-03-06T07:22:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.564+0000 7fac977fe640 1 --2- 172.21.15.17:0/2196185271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009870 tx=0x7fac8c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69711704d979a933 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.564+0000 7fac96ffd640 1 -- 172.21.15.17:0/2196185271 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c034070 con 0x7fac980c12a0 2024-03-06T07:22:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.564+0000 7fac96ffd640 1 -- 172.21.15.17:0/2196185271 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c030840 con 0x7fac980c12a0 2024-03-06T07:22:51.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.564+0000 7fac96ffd640 1 -- 172.21.15.17:0/2196185271 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c030c10 con 0x7fac980c12a0 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 -- 172.21.15.17:0/2196185271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 msgr2=0x7fac980c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 --2- 172.21.15.17:0/2196185271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009870 tx=0x7fac8c02b980 comp rx=0 tx=0).stop 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 -- 172.21.15.17:0/2196185271 shutdown_connections 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 --2- 172.21.15.17:0/2196185271 >> v2:172.21.15.163:3300/0 conn(0x7fac980c7710 0x7fac980c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 --2- 172.21.15.17:0/2196185271 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c12a0 0x7fac980c1990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 -- 172.21.15.17:0/2196185271 >> 172.21.15.17:0/2196185271 conn(0x7fac980a94d0 msgr2=0x7fac980a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 -- 172.21.15.17:0/2196185271 shutdown_connections 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.565+0000 7fac9e4df640 1 -- 172.21.15.17:0/2196185271 wait complete. 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.566+0000 7fac9e4df640 1 Processor -- start 2024-03-06T07:22:51.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.566+0000 7fac9e4df640 1 -- start start 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac980c12a0 0x7fac980aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 0x7fac9814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98155ba0 con 0x7fac98152770 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98152b50 con 0x7fac980c12a0 2024-03-06T07:22:51.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9e4df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9806b6d0 con 0x7fac980c7710 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac977fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 0x7fac9814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38134/0 (socket says 172.21.15.17:38134) 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 -- 172.21.15.17:0/3732086377 learned_addr learned my addr 172.21.15.17:0/3732086377 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac97fff640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac980c12a0 0x7fac980aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 -- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 msgr2=0x7fac9814ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 0x7fac9814ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 -- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac980c12a0 msgr2=0x7fac980aafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac980c12a0 0x7fac980aafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac9ca7c640 1 -- 172.21.15.17:0/3732086377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980ab9f0 con 0x7fac98152770 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.567+0000 7fac977fe640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 0x7fac9814ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac9ca7c640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fac9000efc0 tx=0x7fac9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90018020 con 0x7fac98152770 2024-03-06T07:22:51.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac90014480 con 0x7fac98152770 2024-03-06T07:22:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac980ac4b0 con 0x7fac98152770 2024-03-06T07:22:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac90014870 con 0x7fac98152770 2024-03-06T07:22:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.568+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac980aaaf0 con 0x7fac98152770 2024-03-06T07:22:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.569+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9804f330 con 0x7fac98152770 2024-03-06T07:22:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.570+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac90005d10 con 0x7fac98152770 2024-03-06T07:22:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.570+0000 7fac94ff9640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.570+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fac9008b7e0 con 0x7fac98152770 2024-03-06T07:22:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.571+0000 7fac97fff640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.571+0000 7fac97fff640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 0x7fac8406d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:51.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.574+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fac90022030 con 0x7fac98152770 2024-03-06T07:22:51.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:51.792+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fac980c1010 con 0x7fac98152770 2024-03-06T07:22:52.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.445+0000 7fac94ff9640 1 -- 172.21.15.17:0/3732086377 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v59) v1 ==== 123+0+0 (secure 0 0 0) 0x7fac90010040 con 0x7fac98152770 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 msgr2=0x7fac8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 0x7fac8406d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac88007fd0 tx=0x7fac88007de0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 msgr2=0x7fac9814bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fac9000efc0 tx=0x7fac9000c490 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 shutdown_connections 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac8406b4d0 0x7fac8406d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac98152770 0x7fac9814bf60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac980c7710 0x7fac9814ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 --2- 172.21.15.17:0/3732086377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac980c12a0 0x7fac980aafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 >> 172.21.15.17:0/3732086377 conn(0x7fac980a94d0 msgr2=0x7fac9814ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 shutdown_connections 2024-03-06T07:22:52.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.447+0000 7fac9e4df640 1 -- 172.21.15.17:0/3732086377 wait complete. 2024-03-06T07:22:52.460 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:22:52.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.561+0000 7f5f787a1640 1 Processor -- start 2024-03-06T07:22:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.561+0000 7f5f787a1640 1 -- start start 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f787a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f70159100 0x7f5f700c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f787a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700c7c60 con 0x7f5f701527b0 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f787a1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700c8060 con 0x7f5f700c7180 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f787a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f700c8df0 con 0x7f5f70159100 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f76d3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f70159100 0x7f5f700c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f7653d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5f701527b0 0x7f5f70155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50380/0 (socket says 172.21.15.17:50380) 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f76d3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f70159100 0x7f5f700c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38142/0 (socket says 172.21.15.17:38142) 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.562+0000 7f5f7653d640 1 -- 172.21.15.17:0/3683566168 learned_addr learned my addr 172.21.15.17:0/3683566168 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.563+0000 7f5f76d3e640 1 -- 172.21.15.17:0/3683566168 >> v1:172.21.15.163:6789/0 conn(0x7f5f700c7180 legacy=0x7f5f700c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.563+0000 7f5f76d3e640 1 -- 172.21.15.17:0/3683566168 >> v1:172.21.15.17:6789/0 conn(0x7f5f701527b0 legacy=0x7f5f70155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.563+0000 7f5f76d3e640 1 -- 172.21.15.17:0/3683566168 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700c76a0 con 0x7f5f70159100 2024-03-06T07:22:52.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.563+0000 7f5f7553b640 1 -- 172.21.15.17:0/3683566168 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427985192 0 0) 0x7f5f6c001110 con 0x7f5f701527b0 2024-03-06T07:22:52.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f76d3e640 1 --2- 172.21.15.17:0/3683566168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f70159100 0x7f5f700c88b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009d90 tx=0x7f5f6802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96408429f24a17ce server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f7553b640 1 -- 172.21.15.17:0/3683566168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68039070 con 0x7f5f70159100 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f7553b640 1 -- 172.21.15.17:0/3683566168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6802cc60 con 0x7f5f70159100 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f7553b640 1 -- 172.21.15.17:0/3683566168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68034660 con 0x7f5f70159100 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f787a1640 1 -- 172.21.15.17:0/3683566168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f70159100 msgr2=0x7f5f700c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.564+0000 7f5f787a1640 1 --2- 172.21.15.17:0/3683566168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f70159100 0x7f5f700c88b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009d90 tx=0x7f5f6802beb0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.565+0000 7f5f787a1640 1 -- 172.21.15.17:0/3683566168 shutdown_connections 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.565+0000 7f5f787a1640 1 --2- 172.21.15.17:0/3683566168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f70159100 0x7f5f700c88b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.565+0000 7f5f787a1640 1 -- 172.21.15.17:0/3683566168 >> 172.21.15.17:0/3683566168 conn(0x7f5f700ac560 msgr2=0x7f5f700ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.565+0000 7f5f787a1640 1 -- 172.21.15.17:0/3683566168 shutdown_connections 2024-03-06T07:22:52.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.565+0000 7f5f787a1640 1 -- 172.21.15.17:0/3683566168 wait complete. 2024-03-06T07:22:52.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 Processor -- start 2024-03-06T07:22:52.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 -- start start 2024-03-06T07:22:52.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f700c7180 0x7f5f7014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f70159100 0x7f5f701ede80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70155600 con 0x7f5f701527b0 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70155320 con 0x7f5f700c7180 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f787a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70152b90 con 0x7f5f70159100 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f75d3c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f75d3c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38150/0 (socket says 172.21.15.17:38150) 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f75d3c640 1 -- 172.21.15.17:0/281535169 learned_addr learned my addr 172.21.15.17:0/281535169 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.566+0000 7f5f7653d640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f700c7180 0x7f5f7014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f76d3e640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f70159100 0x7f5f701ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 -- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f70159100 msgr2=0x7f5f701ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f70159100 0x7f5f701ede80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 -- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f700c7180 msgr2=0x7f5f7014bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f700c7180 0x7f5f7014bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 -- 172.21.15.17:0/281535169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700bf790 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f75d3c640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c450 tx=0x7f5f6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600052d0 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60005470 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f701ee9a0 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60012870 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.567+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7014ecd0 con 0x7f5f701527b0 2024-03-06T07:22:52.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.568+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7004f330 con 0x7f5f701527b0 2024-03-06T07:22:52.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.569+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f60012a10 con 0x7f5f701527b0 2024-03-06T07:22:52.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.569+0000 7f5f66ffd640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.569+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f5f6008a130 con 0x7f5f701527b0 2024-03-06T07:22:52.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.569+0000 7f5f7653d640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.570+0000 7f5f7653d640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 0x7f5f5406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c000f60 tx=0x7f5f6c007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.574+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5f60058510 con 0x7f5f701527b0 2024-03-06T07:22:52.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.796+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f7014fa60 con 0x7f5f701527b0 2024-03-06T07:22:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.796+0000 7f5f66ffd640 1 -- 172.21.15.17:0/281535169 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+6264 (secure 0 0 0) 0x7f5f6005c5b0 con 0x7f5f701527b0 2024-03-06T07:22:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:52.798 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":59,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:36.709974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6387":{"gid":6387,"name":"b","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/1213270757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1213270757},{"type":"v1","addr":"172.21.15.163:6836","nonce":1213270757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6387,"qdb_cluster":[6387]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:22:52.798 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 59 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.798+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 msgr2=0x7f5f5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.798+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 0x7f5f5406d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c000f60 tx=0x7f5f6c007b50 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.798+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 msgr2=0x7f5f7014de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.798+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c450 tx=0x7f5f6000c910 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 shutdown_connections 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f70159100 0x7f5f701ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f701527b0 0x7f5f7014de20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 --2- 172.21.15.17:0/281535169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f700c7180 0x7f5f7014bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 >> 172.21.15.17:0/281535169 conn(0x7f5f700ac560 msgr2=0x7f5f700c9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 shutdown_connections 2024-03-06T07:22:52.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.799+0000 7f5f787a1640 1 -- 172.21.15.17:0/281535169 wait complete. 2024-03-06T07:22:52.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:22:52.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.910+0000 7fefaf5f7640 1 Processor -- start 2024-03-06T07:22:52.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefaf5f7640 1 -- start start 2024-03-06T07:22:52.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefaf5f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefa81506b0 0x7fefa8153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefaf5f7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa8070aa0 con 0x7fefa80acc00 2024-03-06T07:22:52.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefaf5f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa8070d80 con 0x7fefa8156ea0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefaf5f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa8156450 con 0x7fefa81506b0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefadb94640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fefa80acc00 0x7fefa80c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50384/0 (socket says 172.21.15.17:50384) 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefadb94640 1 -- 172.21.15.17:0/1162156230 learned_addr learned my addr 172.21.15.17:0/1162156230 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.911+0000 7fefad393640 1 --2- 172.21.15.17:0/1162156230 >> v2:172.21.15.163:3300/0 conn(0x7fefa81506b0 0x7fefa8153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626438586 0 0) 0x7fefa8070aa0 con 0x7fefa80acc00 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef94003520 con 0x7fefa80acc00 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fefad393640 1 -- 172.21.15.17:0/1162156230 >> v1:172.21.15.163:6789/0 conn(0x7fefa8156ea0 legacy=0x7fefa80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674620749 0 0) 0x7fefa8070d80 con 0x7fefa8156ea0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fefad393640 1 -- 172.21.15.17:0/1162156230 >> v1:172.21.15.17:6789/0 conn(0x7fefa80acc00 legacy=0x7fefa80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.912+0000 7fefad393640 1 -- 172.21.15.17:0/1162156230 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa81566a0 con 0x7fefa81506b0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefad393640 1 --2- 172.21.15.17:0/1162156230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa8153990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefa40097a0 tx=0x7fefa402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146ccb3793707b25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034070 con 0x7fefa81506b0 2024-03-06T07:22:52.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4030610 con 0x7fefa81506b0 2024-03-06T07:22:52.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fef9ffff640 1 -- 172.21.15.17:0/1162156230 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40309e0 con 0x7fefa81506b0 2024-03-06T07:22:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefaf5f7640 1 -- 172.21.15.17:0/1162156230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 msgr2=0x7fefa8153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/1162156230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa8153990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fefa40097a0 tx=0x7fefa402b960 comp rx=0 tx=0).stop 2024-03-06T07:22:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefaf5f7640 1 -- 172.21.15.17:0/1162156230 shutdown_connections 2024-03-06T07:22:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/1162156230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa8153990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.913+0000 7fefaf5f7640 1 -- 172.21.15.17:0/1162156230 >> 172.21.15.17:0/1162156230 conn(0x7fefa806b1a0 msgr2=0x7fefa806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.914+0000 7fefaf5f7640 1 -- 172.21.15.17:0/1162156230 shutdown_connections 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.914+0000 7fefaf5f7640 1 -- 172.21.15.17:0/1162156230 wait complete. 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 Processor -- start 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 -- start start 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa81ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa8156ea0 0x7fefa80c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80ad050 con 0x7fefa80acc00 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa8156a10 con 0x7fefa81506b0 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefaf5f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80ad1d0 con 0x7fefa8156ea0 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefad393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefad393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38158/0 (socket says 172.21.15.17:38158) 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefad393640 1 -- 172.21.15.17:0/2658588894 learned_addr learned my addr 172.21.15.17:0/2658588894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefacb92640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa81ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.915+0000 7fefadb94640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa8156ea0 0x7fefa80c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 -- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa8156ea0 msgr2=0x7fefa80c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa8156ea0 0x7fefa80c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 -- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 msgr2=0x7fefa81ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:52.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa81ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 -- 172.21.15.17:0/2658588894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa814ec70 con 0x7fefa80acc00 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefad393640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fefa40098c0 tx=0x7fefa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4049070 con 0x7fefa80acc00 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4030410 con 0x7fefa80acc00 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034040 con 0x7fefa80acc00 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa81ebd00 con 0x7fefa80acc00 2024-03-06T07:22:52.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.916+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa80c7e50 con 0x7fefa80acc00 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.917+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa804f330 con 0x7fefa80acc00 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.918+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefa404c070 con 0x7fefa80acc00 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.918+0000 7fef9dffb640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 0x7fef8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.918+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fefa40ac110 con 0x7fefa80acc00 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.918+0000 7fefacb92640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 0x7fef8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:52.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.919+0000 7fefacb92640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 0x7fef8c06d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefa814e5c0 tx=0x7fef98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:52.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:52.923+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fefa40305e0 con 0x7fefa80acc00 2024-03-06T07:22:53.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.104+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa80700a0 con 0x7fefa80acc00 2024-03-06T07:22:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.105+0000 7fef9dffb640 1 -- 172.21.15.17:0/2658588894 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+20669 (secure 0 0 0) 0x7fefa407a4f0 con 0x7fefa80acc00 2024-03-06T07:22:53.107 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:53.107 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":79,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:51.423942+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:22:53.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 msgr2=0x7fef8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 0x7fef8c06d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fefa814e5c0 tx=0x7fef98009040 comp rx=0 tx=0).stop 2024-03-06T07:22:53.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 msgr2=0x7fefa814e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fefa40098c0 tx=0x7fefa40046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 shutdown_connections 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef8c06b4d0 0x7fef8c06d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa8156ea0 0x7fefa80c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa81506b0 0x7fefa81ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 --2- 172.21.15.17:0/2658588894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa80acc00 0x7fefa814e690 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 >> 172.21.15.17:0/2658588894 conn(0x7fefa806b1a0 msgr2=0x7fefa81489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 shutdown_connections 2024-03-06T07:22:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.107+0000 7fefaf5f7640 1 -- 172.21.15.17:0/2658588894 wait complete. 2024-03-06T07:22:53.120 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:22:53.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:53.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.220+0000 7fe2071b5640 1 Processor -- start 2024-03-06T07:22:53.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.220+0000 7fe2071b5640 1 -- start start 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe2071b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe20014f360 0x7fe200155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe2071b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe2000c88c0 0x7fe2000c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe2071b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000c73d0 con 0x7fe2000c5880 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe2071b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000c8dd0 con 0x7fe20014f360 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe2071b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000be2e0 con 0x7fe2000c88c0 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe204f51640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe20014f360 0x7fe200155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe204f51640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe20014f360 0x7fe200155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38164/0 (socket says 172.21.15.17:38164) 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe204f51640 1 -- 172.21.15.17:0/2955537195 learned_addr learned my addr 172.21.15.17:0/2955537195 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.221+0000 7fe1fffff640 1 --2- 172.21.15.17:0/2955537195 >> v2:172.21.15.163:3300/0 conn(0x7fe2000c88c0 0x7fe2000c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe204f51640 1 -- 172.21.15.17:0/2955537195 >> v2:172.21.15.163:3300/0 conn(0x7fe2000c88c0 msgr2=0x7fe2000c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe204f51640 1 --2- 172.21.15.17:0/2955537195 >> v2:172.21.15.163:3300/0 conn(0x7fe2000c88c0 0x7fe2000c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe204f51640 1 -- 172.21.15.17:0/2955537195 >> v1:172.21.15.163:6789/0 conn(0x7fe2000c5880 legacy=0x7fe200158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe204f51640 1 -- 172.21.15.17:0/2955537195 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000c7670 con 0x7fe20014f360 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe204f51640 1 --2- 172.21.15.17:0/2955537195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe20014f360 0x7fe200155c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe2000c84c0 tx=0x7fe1f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7708500fa99b4d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.222+0000 7fe1ff7fe640 1 -- 172.21.15.17:0/2955537195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4035020 con 0x7fe20014f360 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe1ff7fe640 1 -- 172.21.15.17:0/2955537195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4031870 con 0x7fe20014f360 2024-03-06T07:22:53.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe1ff7fe640 1 -- 172.21.15.17:0/2955537195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4031c40 con 0x7fe20014f360 2024-03-06T07:22:53.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 -- 172.21.15.17:0/2955537195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe20014f360 msgr2=0x7fe200155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 --2- 172.21.15.17:0/2955537195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe20014f360 0x7fe200155c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe2000c84c0 tx=0x7fe1f402b910 comp rx=0 tx=0).stop 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 -- 172.21.15.17:0/2955537195 shutdown_connections 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 --2- 172.21.15.17:0/2955537195 >> v2:172.21.15.163:3300/0 conn(0x7fe2000c88c0 0x7fe2000c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 --2- 172.21.15.17:0/2955537195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe20014f360 0x7fe200155c10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 -- 172.21.15.17:0/2955537195 >> 172.21.15.17:0/2955537195 conn(0x7fe2000ac570 msgr2=0x7fe2000ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 -- 172.21.15.17:0/2955537195 shutdown_connections 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.223+0000 7fe2071b5640 1 -- 172.21.15.17:0/2955537195 wait complete. 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.225+0000 7fe2071b5640 1 Processor -- start 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.225+0000 7fe2071b5640 1 -- start start 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000c5880 0x7fe2001ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe20014f360 0x7fe2000ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200159270 con 0x7fe2000c88c0 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c7c30 con 0x7fe20014f360 2024-03-06T07:22:53.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe2071b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001593f0 con 0x7fe2000c5880 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38180/0 (socket says 172.21.15.17:38180) 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 -- 172.21.15.17:0/4195641493 learned_addr learned my addr 172.21.15.17:0/4195641493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe204f51640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000c5880 0x7fe2001ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe205752640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe20014f360 0x7fe2000ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 -- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000c5880 msgr2=0x7fe2001ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000c5880 0x7fe2001ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 -- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe20014f360 msgr2=0x7fe2000ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe20014f360 0x7fe2000ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.226+0000 7fe1fffff640 1 -- 172.21.15.17:0/4195641493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2001eb580 con 0x7fe2000c88c0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe1fffff640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9b0 tx=0x7fe1ec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec017070 con 0x7fe2000c88c0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec00c9a0 con 0x7fe2000c88c0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec00cd70 con 0x7fe2000c88c0 2024-03-06T07:22:53.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000ad8d0 con 0x7fe2000c88c0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.227+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000c0910 con 0x7fe2000c88c0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.228+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20004f330 con 0x7fe2000c88c0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.228+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1ec01a070 con 0x7fe2000c88c0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.229+0000 7fe1fd7fa640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 0x7fe1e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.229+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fe1ec08a670 con 0x7fe2000c88c0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.229+0000 7fe204f51640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 0x7fe1e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.229+0000 7fe204f51640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 0x7fe1e406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4036000 tx=0x7fe1f403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.233+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1ec058930 con 0x7fe2000c88c0 2024-03-06T07:22:53.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.409+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2000c03e0 con 0x7fe1e406b500 2024-03-06T07:22:53.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.421+0000 7fe1fd7fa640 1 -- 172.21.15.17:0/4195641493 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490718 (secure 0 0 0) 0x7fe1f40bfcf0 con 0x7fe1e406b500 2024-03-06T07:22:53.428 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:53.431 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:53.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 msgr2=0x7fe1e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 0x7fe1e406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4036000 tx=0x7fe1f403c040 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 msgr2=0x7fe2001eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00e9b0 tx=0x7fe1ec00ee70 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 shutdown_connections 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1e406b500 0x7fe1e406d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe20014f360 0x7fe2000ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2000c88c0 0x7fe2001eb040 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 --2- 172.21.15.17:0/4195641493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000c5880 0x7fe2001ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 >> 172.21.15.17:0/4195641493 conn(0x7fe2000ac570 msgr2=0x7fe200152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 shutdown_connections 2024-03-06T07:22:53.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.424+0000 7fe2071b5640 1 -- 172.21.15.17:0/4195641493 wait complete. 2024-03-06T07:22:53.437 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-03-06T07:22:53.378277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":248,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":371,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244780,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729940,"statfs":{"total":773094113280,"available":772843458560,"internally_reserved":0,"allocated":7970816,"data_stored":4906390,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":242535988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.960606"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810944+0000","last_change":"2024-03-06T07:22:50.450570+0000","last_active":"2024-03-06T07:22:51.810944+0000","last_peered":"2024-03-06T07:22:51.810944+0000","last_clean":"2024-03-06T07:22:51.810944+0000","last_became_active":"2024-03-06T07:22:50.450305+0000","last_became_peered":"2024-03-06T07:22:50.450305+0000","last_unstale":"2024-03-06T07:22:51.810944+0000","last_undegraded":"2024-03-06T07:22:51.810944+0000","last_fullsized":"2024-03-06T07:22:51.810944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:23.305044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809908+0000","last_change":"2024-03-06T07:22:35.363195+0000","last_active":"2024-03-06T07:22:51.809908+0000","last_peered":"2024-03-06T07:22:51.809908+0000","last_clean":"2024-03-06T07:22:51.809908+0000","last_became_active":"2024-03-06T07:22:35.362811+0000","last_became_peered":"2024-03-06T07:22:35.362811+0000","last_unstale":"2024-03-06T07:22:51.809908+0000","last_undegraded":"2024-03-06T07:22:51.809908+0000","last_fullsized":"2024-03-06T07:22:51.809908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427452+0000","last_change":"2024-03-06T07:22:50.445149+0000","last_active":"2024-03-06T07:22:51.427452+0000","last_peered":"2024-03-06T07:22:51.427452+0000","last_clean":"2024-03-06T07:22:51.427452+0000","last_became_active":"2024-03-06T07:22:50.444878+0000","last_became_peered":"2024-03-06T07:22:50.444878+0000","last_unstale":"2024-03-06T07:22:51.427452+0000","last_undegraded":"2024-03-06T07:22:51.427452+0000","last_fullsized":"2024-03-06T07:22:51.427452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:23.041319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811647+0000","last_change":"2024-03-06T07:22:35.363283+0000","last_active":"2024-03-06T07:22:51.811647+0000","last_peered":"2024-03-06T07:22:51.811647+0000","last_clean":"2024-03-06T07:22:51.811647+0000","last_became_active":"2024-03-06T07:22:35.363028+0000","last_became_peered":"2024-03-06T07:22:35.363028+0000","last_unstale":"2024-03-06T07:22:51.811647+0000","last_undegraded":"2024-03-06T07:22:51.811647+0000","last_fullsized":"2024-03-06T07:22:51.811647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810613+0000","last_change":"2024-03-06T07:22:50.460440+0000","last_active":"2024-03-06T07:22:51.810613+0000","last_peered":"2024-03-06T07:22:51.810613+0000","last_clean":"2024-03-06T07:22:51.810613+0000","last_became_active":"2024-03-06T07:22:50.460172+0000","last_became_peered":"2024-03-06T07:22:50.460172+0000","last_unstale":"2024-03-06T07:22:51.810613+0000","last_undegraded":"2024-03-06T07:22:51.810613+0000","last_fullsized":"2024-03-06T07:22:51.810613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:49.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427534+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:51.427534+0000","last_peered":"2024-03-06T07:22:51.427534+0000","last_clean":"2024-03-06T07:22:51.427534+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:51.427534+0000","last_undegraded":"2024-03-06T07:22:51.427534+0000","last_fullsized":"2024-03-06T07:22:51.427534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:44.303007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810103+0000","last_change":"2024-03-06T07:22:50.454528+0000","last_active":"2024-03-06T07:22:51.810103+0000","last_peered":"2024-03-06T07:22:51.810103+0000","last_clean":"2024-03-06T07:22:51.810103+0000","last_became_active":"2024-03-06T07:22:50.454270+0000","last_became_peered":"2024-03-06T07:22:50.454270+0000","last_unstale":"2024-03-06T07:22:51.810103+0000","last_undegraded":"2024-03-06T07:22:51.810103+0000","last_fullsized":"2024-03-06T07:22:51.810103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:04.171573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810188+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:51.810188+0000","last_peered":"2024-03-06T07:22:51.810188+0000","last_clean":"2024-03-06T07:22:51.810188+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:51.810188+0000","last_undegraded":"2024-03-06T07:22:51.810188+0000","last_fullsized":"2024-03-06T07:22:51.810188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:53.250614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809709+0000","last_change":"2024-03-06T07:22:50.445632+0000","last_active":"2024-03-06T07:22:51.809709+0000","last_peered":"2024-03-06T07:22:51.809709+0000","last_clean":"2024-03-06T07:22:51.809709+0000","last_became_active":"2024-03-06T07:22:50.445397+0000","last_became_peered":"2024-03-06T07:22:50.445397+0000","last_unstale":"2024-03-06T07:22:51.809709+0000","last_undegraded":"2024-03-06T07:22:51.809709+0000","last_fullsized":"2024-03-06T07:22:51.809709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:05:22.123259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810185+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:51.810185+0000","last_peered":"2024-03-06T07:22:51.810185+0000","last_clean":"2024-03-06T07:22:51.810185+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:51.810185+0000","last_undegraded":"2024-03-06T07:22:51.810185+0000","last_fullsized":"2024-03-06T07:22:51.810185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426627+0000","last_change":"2024-03-06T07:22:50.452816+0000","last_active":"2024-03-06T07:22:51.426627+0000","last_peered":"2024-03-06T07:22:51.426627+0000","last_clean":"2024-03-06T07:22:51.426627+0000","last_became_active":"2024-03-06T07:22:50.452150+0000","last_became_peered":"2024-03-06T07:22:50.452150+0000","last_unstale":"2024-03-06T07:22:51.426627+0000","last_undegraded":"2024-03-06T07:22:51.426627+0000","last_fullsized":"2024-03-06T07:22:51.426627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810914+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:51.810914+0000","last_peered":"2024-03-06T07:22:51.810914+0000","last_clean":"2024-03-06T07:22:51.810914+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:51.810914+0000","last_undegraded":"2024-03-06T07:22:51.810914+0000","last_fullsized":"2024-03-06T07:22:51.810914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429183+0000","last_change":"2024-03-06T07:22:50.434924+0000","last_active":"2024-03-06T07:22:51.429183+0000","last_peered":"2024-03-06T07:22:51.429183+0000","last_clean":"2024-03-06T07:22:51.429183+0000","last_became_active":"2024-03-06T07:22:50.434584+0000","last_became_peered":"2024-03-06T07:22:50.434584+0000","last_unstale":"2024-03-06T07:22:51.429183+0000","last_undegraded":"2024-03-06T07:22:51.429183+0000","last_fullsized":"2024-03-06T07:22:51.429183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429929+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:51.429929+0000","last_peered":"2024-03-06T07:22:51.429929+0000","last_clean":"2024-03-06T07:22:51.429929+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:51.429929+0000","last_undegraded":"2024-03-06T07:22:51.429929+0000","last_fullsized":"2024-03-06T07:22:51.429929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811294+0000","last_change":"2024-03-06T07:22:50.449942+0000","last_active":"2024-03-06T07:22:51.811294+0000","last_peered":"2024-03-06T07:22:51.811294+0000","last_clean":"2024-03-06T07:22:51.811294+0000","last_became_active":"2024-03-06T07:22:50.449445+0000","last_became_peered":"2024-03-06T07:22:50.449445+0000","last_unstale":"2024-03-06T07:22:51.811294+0000","last_undegraded":"2024-03-06T07:22:51.811294+0000","last_fullsized":"2024-03-06T07:22:51.811294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:35.875822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809884+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:51.809884+0000","last_peered":"2024-03-06T07:22:51.809884+0000","last_clean":"2024-03-06T07:22:51.809884+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:51.809884+0000","last_undegraded":"2024-03-06T07:22:51.809884+0000","last_fullsized":"2024-03-06T07:22:51.809884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429999+0000","last_change":"2024-03-06T07:22:50.453986+0000","last_active":"2024-03-06T07:22:51.429999+0000","last_peered":"2024-03-06T07:22:51.429999+0000","last_clean":"2024-03-06T07:22:51.429999+0000","last_became_active":"2024-03-06T07:22:50.453604+0000","last_became_peered":"2024-03-06T07:22:50.453604+0000","last_unstale":"2024-03-06T07:22:51.429999+0000","last_undegraded":"2024-03-06T07:22:51.429999+0000","last_fullsized":"2024-03-06T07:22:51.429999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:32.345117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430561+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:51.430561+0000","last_peered":"2024-03-06T07:22:51.430561+0000","last_clean":"2024-03-06T07:22:51.430561+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:51.430561+0000","last_undegraded":"2024-03-06T07:22:51.430561+0000","last_fullsized":"2024-03-06T07:22:51.430561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810235+0000","last_change":"2024-03-06T07:22:50.445186+0000","last_active":"2024-03-06T07:22:51.810235+0000","last_peered":"2024-03-06T07:22:51.810235+0000","last_clean":"2024-03-06T07:22:51.810235+0000","last_became_active":"2024-03-06T07:22:50.444914+0000","last_became_peered":"2024-03-06T07:22:50.444914+0000","last_unstale":"2024-03-06T07:22:51.810235+0000","last_undegraded":"2024-03-06T07:22:51.810235+0000","last_fullsized":"2024-03-06T07:22:51.810235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:50.902194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810228+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:51.810228+0000","last_peered":"2024-03-06T07:22:51.810228+0000","last_clean":"2024-03-06T07:22:51.810228+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:51.810228+0000","last_undegraded":"2024-03-06T07:22:51.810228+0000","last_fullsized":"2024-03-06T07:22:51.810228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:42.233047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430570+0000","last_change":"2024-03-06T07:22:50.438872+0000","last_active":"2024-03-06T07:22:51.430570+0000","last_peered":"2024-03-06T07:22:51.430570+0000","last_clean":"2024-03-06T07:22:51.430570+0000","last_became_active":"2024-03-06T07:22:50.438316+0000","last_became_peered":"2024-03-06T07:22:50.438316+0000","last_unstale":"2024-03-06T07:22:51.430570+0000","last_undegraded":"2024-03-06T07:22:51.430570+0000","last_fullsized":"2024-03-06T07:22:51.430570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812251+0000","last_change":"2024-03-06T07:22:35.365945+0000","last_active":"2024-03-06T07:22:51.812251+0000","last_peered":"2024-03-06T07:22:51.812251+0000","last_clean":"2024-03-06T07:22:51.812251+0000","last_became_active":"2024-03-06T07:22:35.365677+0000","last_became_peered":"2024-03-06T07:22:35.365677+0000","last_unstale":"2024-03-06T07:22:51.812251+0000","last_undegraded":"2024-03-06T07:22:51.812251+0000","last_fullsized":"2024-03-06T07:22:51.812251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810382+0000","last_change":"2024-03-06T07:22:50.432526+0000","last_active":"2024-03-06T07:22:51.810382+0000","last_peered":"2024-03-06T07:22:51.810382+0000","last_clean":"2024-03-06T07:22:51.810382+0000","last_became_active":"2024-03-06T07:22:50.432234+0000","last_became_peered":"2024-03-06T07:22:50.432234+0000","last_unstale":"2024-03-06T07:22:51.810382+0000","last_undegraded":"2024-03-06T07:22:51.810382+0000","last_fullsized":"2024-03-06T07:22:51.810382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:22.325667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810328+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:51.810328+0000","last_peered":"2024-03-06T07:22:51.810328+0000","last_clean":"2024-03-06T07:22:51.810328+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:51.810328+0000","last_undegraded":"2024-03-06T07:22:51.810328+0000","last_fullsized":"2024-03-06T07:22:51.810328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:47.235841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426749+0000","last_change":"2024-03-06T07:22:50.446058+0000","last_active":"2024-03-06T07:22:51.426749+0000","last_peered":"2024-03-06T07:22:51.426749+0000","last_clean":"2024-03-06T07:22:51.426749+0000","last_became_active":"2024-03-06T07:22:50.445777+0000","last_became_peered":"2024-03-06T07:22:50.445777+0000","last_unstale":"2024-03-06T07:22:51.426749+0000","last_undegraded":"2024-03-06T07:22:51.426749+0000","last_fullsized":"2024-03-06T07:22:51.426749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:13.076972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810400+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:51.810400+0000","last_peered":"2024-03-06T07:22:51.810400+0000","last_clean":"2024-03-06T07:22:51.810400+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:51.810400+0000","last_undegraded":"2024-03-06T07:22:51.810400+0000","last_fullsized":"2024-03-06T07:22:51.810400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811648+0000","last_change":"2024-03-06T07:22:50.459245+0000","last_active":"2024-03-06T07:22:51.811648+0000","last_peered":"2024-03-06T07:22:51.811648+0000","last_clean":"2024-03-06T07:22:51.811648+0000","last_became_active":"2024-03-06T07:22:50.458595+0000","last_became_peered":"2024-03-06T07:22:50.458595+0000","last_unstale":"2024-03-06T07:22:51.811648+0000","last_undegraded":"2024-03-06T07:22:51.811648+0000","last_fullsized":"2024-03-06T07:22:51.811648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:24.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811000+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:51.811000+0000","last_peered":"2024-03-06T07:22:51.811000+0000","last_clean":"2024-03-06T07:22:51.811000+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:51.811000+0000","last_undegraded":"2024-03-06T07:22:51.811000+0000","last_fullsized":"2024-03-06T07:22:51.811000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426982+0000","last_change":"2024-03-06T07:22:50.451467+0000","last_active":"2024-03-06T07:22:51.426982+0000","last_peered":"2024-03-06T07:22:51.426982+0000","last_clean":"2024-03-06T07:22:51.426982+0000","last_became_active":"2024-03-06T07:22:50.451203+0000","last_became_peered":"2024-03-06T07:22:50.451203+0000","last_unstale":"2024-03-06T07:22:51.426982+0000","last_undegraded":"2024-03-06T07:22:51.426982+0000","last_fullsized":"2024-03-06T07:22:51.426982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:11.728819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811233+0000","last_change":"2024-03-06T07:22:35.366910+0000","last_active":"2024-03-06T07:22:51.811233+0000","last_peered":"2024-03-06T07:22:51.811233+0000","last_clean":"2024-03-06T07:22:51.811233+0000","last_became_active":"2024-03-06T07:22:35.366623+0000","last_became_peered":"2024-03-06T07:22:35.366623+0000","last_unstale":"2024-03-06T07:22:51.811233+0000","last_undegraded":"2024-03-06T07:22:51.811233+0000","last_fullsized":"2024-03-06T07:22:51.811233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429511+0000","last_change":"2024-03-06T07:22:50.439065+0000","last_active":"2024-03-06T07:22:51.429511+0000","last_peered":"2024-03-06T07:22:51.429511+0000","last_clean":"2024-03-06T07:22:51.429511+0000","last_became_active":"2024-03-06T07:22:50.438051+0000","last_became_peered":"2024-03-06T07:22:50.438051+0000","last_unstale":"2024-03-06T07:22:51.429511+0000","last_undegraded":"2024-03-06T07:22:51.429511+0000","last_fullsized":"2024-03-06T07:22:51.429511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:23.041319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809597+0000","last_change":"2024-03-06T07:22:35.360892+0000","last_active":"2024-03-06T07:22:51.809597+0000","last_peered":"2024-03-06T07:22:51.809597+0000","last_clean":"2024-03-06T07:22:51.809597+0000","last_became_active":"2024-03-06T07:22:35.360615+0000","last_became_peered":"2024-03-06T07:22:35.360615+0000","last_unstale":"2024-03-06T07:22:51.809597+0000","last_undegraded":"2024-03-06T07:22:51.809597+0000","last_fullsized":"2024-03-06T07:22:51.809597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:15.740118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430126+0000","last_change":"2024-03-06T07:22:50.440433+0000","last_active":"2024-03-06T07:22:51.430126+0000","last_peered":"2024-03-06T07:22:51.430126+0000","last_clean":"2024-03-06T07:22:51.430126+0000","last_became_active":"2024-03-06T07:22:50.440166+0000","last_became_peered":"2024-03-06T07:22:50.440166+0000","last_unstale":"2024-03-06T07:22:51.430126+0000","last_undegraded":"2024-03-06T07:22:51.430126+0000","last_fullsized":"2024-03-06T07:22:51.430126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:58.355065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812530+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:51.812530+0000","last_peered":"2024-03-06T07:22:51.812530+0000","last_clean":"2024-03-06T07:22:51.812530+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:51.812530+0000","last_undegraded":"2024-03-06T07:22:51.812530+0000","last_fullsized":"2024-03-06T07:22:51.812530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427626+0000","last_change":"2024-03-06T07:22:50.455367+0000","last_active":"2024-03-06T07:22:51.427626+0000","last_peered":"2024-03-06T07:22:51.427626+0000","last_clean":"2024-03-06T07:22:51.427626+0000","last_became_active":"2024-03-06T07:22:50.455105+0000","last_became_peered":"2024-03-06T07:22:50.455105+0000","last_unstale":"2024-03-06T07:22:51.427626+0000","last_undegraded":"2024-03-06T07:22:51.427626+0000","last_fullsized":"2024-03-06T07:22:51.427626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:32.345117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427784+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:51.427784+0000","last_peered":"2024-03-06T07:22:51.427784+0000","last_clean":"2024-03-06T07:22:51.427784+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:51.427784+0000","last_undegraded":"2024-03-06T07:22:51.427784+0000","last_fullsized":"2024-03-06T07:22:51.427784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812440+0000","last_change":"2024-03-06T07:22:50.459346+0000","last_active":"2024-03-06T07:22:51.812440+0000","last_peered":"2024-03-06T07:22:51.812440+0000","last_clean":"2024-03-06T07:22:51.812440+0000","last_became_active":"2024-03-06T07:22:50.459087+0000","last_became_peered":"2024-03-06T07:22:50.459087+0000","last_unstale":"2024-03-06T07:22:51.812440+0000","last_undegraded":"2024-03-06T07:22:51.812440+0000","last_fullsized":"2024-03-06T07:22:51.812440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:51.978018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810993+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:51.810993+0000","last_peered":"2024-03-06T07:22:51.810993+0000","last_clean":"2024-03-06T07:22:51.810993+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:51.810993+0000","last_undegraded":"2024-03-06T07:22:51.810993+0000","last_fullsized":"2024-03-06T07:22:51.810993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811724+0000","last_change":"2024-03-06T07:22:50.442684+0000","last_active":"2024-03-06T07:22:51.811724+0000","last_peered":"2024-03-06T07:22:51.811724+0000","last_clean":"2024-03-06T07:22:51.811724+0000","last_became_active":"2024-03-06T07:22:50.442400+0000","last_became_peered":"2024-03-06T07:22:50.442400+0000","last_unstale":"2024-03-06T07:22:51.811724+0000","last_undegraded":"2024-03-06T07:22:51.811724+0000","last_fullsized":"2024-03-06T07:22:51.811724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:04.189588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810096+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:51.810096+0000","last_peered":"2024-03-06T07:22:51.810096+0000","last_clean":"2024-03-06T07:22:51.810096+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:51.810096+0000","last_undegraded":"2024-03-06T07:22:51.810096+0000","last_fullsized":"2024-03-06T07:22:51.810096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430030+0000","last_change":"2024-03-06T07:22:50.433096+0000","last_active":"2024-03-06T07:22:51.430030+0000","last_peered":"2024-03-06T07:22:51.430030+0000","last_clean":"2024-03-06T07:22:51.430030+0000","last_became_active":"2024-03-06T07:22:50.432822+0000","last_became_peered":"2024-03-06T07:22:50.432822+0000","last_unstale":"2024-03-06T07:22:51.430030+0000","last_undegraded":"2024-03-06T07:22:51.430030+0000","last_fullsized":"2024-03-06T07:22:51.430030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:00.272335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429577+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:51.429577+0000","last_peered":"2024-03-06T07:22:51.429577+0000","last_clean":"2024-03-06T07:22:51.429577+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:51.429577+0000","last_undegraded":"2024-03-06T07:22:51.429577+0000","last_fullsized":"2024-03-06T07:22:51.429577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809766+0000","last_change":"2024-03-06T07:22:50.450091+0000","last_active":"2024-03-06T07:22:51.809766+0000","last_peered":"2024-03-06T07:22:51.809766+0000","last_clean":"2024-03-06T07:22:51.809766+0000","last_became_active":"2024-03-06T07:22:50.449781+0000","last_became_peered":"2024-03-06T07:22:50.449781+0000","last_unstale":"2024-03-06T07:22:51.809766+0000","last_undegraded":"2024-03-06T07:22:51.809766+0000","last_fullsized":"2024-03-06T07:22:51.809766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:42.449192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810467+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:51.810467+0000","last_peered":"2024-03-06T07:22:51.810467+0000","last_clean":"2024-03-06T07:22:51.810467+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:51.810467+0000","last_undegraded":"2024-03-06T07:22:51.810467+0000","last_fullsized":"2024-03-06T07:22:51.810467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429452+0000","last_change":"2024-03-06T07:22:50.442317+0000","last_active":"2024-03-06T07:22:51.429452+0000","last_peered":"2024-03-06T07:22:51.429452+0000","last_clean":"2024-03-06T07:22:51.429452+0000","last_became_active":"2024-03-06T07:22:50.442066+0000","last_became_peered":"2024-03-06T07:22:50.442066+0000","last_unstale":"2024-03-06T07:22:51.429452+0000","last_undegraded":"2024-03-06T07:22:51.429452+0000","last_fullsized":"2024-03-06T07:22:51.429452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:11.728819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429051+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:51.429051+0000","last_peered":"2024-03-06T07:22:51.429051+0000","last_clean":"2024-03-06T07:22:51.429051+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:51.429051+0000","last_undegraded":"2024-03-06T07:22:51.429051+0000","last_fullsized":"2024-03-06T07:22:51.429051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429444+0000","last_change":"2024-03-06T07:22:50.439392+0000","last_active":"2024-03-06T07:22:51.429444+0000","last_peered":"2024-03-06T07:22:51.429444+0000","last_clean":"2024-03-06T07:22:51.429444+0000","last_became_active":"2024-03-06T07:22:50.439117+0000","last_became_peered":"2024-03-06T07:22:50.439117+0000","last_unstale":"2024-03-06T07:22:51.429444+0000","last_undegraded":"2024-03-06T07:22:51.429444+0000","last_fullsized":"2024-03-06T07:22:51.429444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:13.076972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811697+0000","last_change":"2024-03-06T07:22:35.367093+0000","last_active":"2024-03-06T07:22:51.811697+0000","last_peered":"2024-03-06T07:22:51.811697+0000","last_clean":"2024-03-06T07:22:51.811697+0000","last_became_active":"2024-03-06T07:22:35.366829+0000","last_became_peered":"2024-03-06T07:22:35.366829+0000","last_unstale":"2024-03-06T07:22:51.811697+0000","last_undegraded":"2024-03-06T07:22:51.811697+0000","last_fullsized":"2024-03-06T07:22:51.811697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810614+0000","last_change":"2024-03-06T07:22:50.430917+0000","last_active":"2024-03-06T07:22:51.810614+0000","last_peered":"2024-03-06T07:22:51.810614+0000","last_clean":"2024-03-06T07:22:51.810614+0000","last_became_active":"2024-03-06T07:22:50.430574+0000","last_became_peered":"2024-03-06T07:22:50.430574+0000","last_unstale":"2024-03-06T07:22:51.810614+0000","last_undegraded":"2024-03-06T07:22:51.810614+0000","last_fullsized":"2024-03-06T07:22:51.810614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:46.085205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812491+0000","last_change":"2024-03-06T07:22:35.367432+0000","last_active":"2024-03-06T07:22:51.812491+0000","last_peered":"2024-03-06T07:22:51.812491+0000","last_clean":"2024-03-06T07:22:51.812491+0000","last_became_active":"2024-03-06T07:22:35.367164+0000","last_became_peered":"2024-03-06T07:22:35.367164+0000","last_unstale":"2024-03-06T07:22:51.812491+0000","last_undegraded":"2024-03-06T07:22:51.812491+0000","last_fullsized":"2024-03-06T07:22:51.812491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427630+0000","last_change":"2024-03-06T07:22:50.452478+0000","last_active":"2024-03-06T07:22:51.427630+0000","last_peered":"2024-03-06T07:22:51.427630+0000","last_clean":"2024-03-06T07:22:51.427630+0000","last_became_active":"2024-03-06T07:22:50.452194+0000","last_became_peered":"2024-03-06T07:22:50.452194+0000","last_unstale":"2024-03-06T07:22:51.427630+0000","last_undegraded":"2024-03-06T07:22:51.427630+0000","last_fullsized":"2024-03-06T07:22:51.427630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:58.355065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810383+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:51.810383+0000","last_peered":"2024-03-06T07:22:51.810383+0000","last_clean":"2024-03-06T07:22:51.810383+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:51.810383+0000","last_undegraded":"2024-03-06T07:22:51.810383+0000","last_fullsized":"2024-03-06T07:22:51.810383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:54.664573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810015+0000","last_change":"2024-03-06T07:22:50.441478+0000","last_active":"2024-03-06T07:22:51.810015+0000","last_peered":"2024-03-06T07:22:51.810015+0000","last_clean":"2024-03-06T07:22:51.810015+0000","last_became_active":"2024-03-06T07:22:50.441237+0000","last_became_peered":"2024-03-06T07:22:50.441237+0000","last_unstale":"2024-03-06T07:22:51.810015+0000","last_undegraded":"2024-03-06T07:22:51.810015+0000","last_fullsized":"2024-03-06T07:22:51.810015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:56.658779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810223+0000","last_change":"2024-03-06T07:22:35.373963+0000","last_active":"2024-03-06T07:22:51.810223+0000","last_peered":"2024-03-06T07:22:51.810223+0000","last_clean":"2024-03-06T07:22:51.810223+0000","last_became_active":"2024-03-06T07:22:35.373704+0000","last_became_peered":"2024-03-06T07:22:35.373704+0000","last_unstale":"2024-03-06T07:22:51.810223+0000","last_undegraded":"2024-03-06T07:22:51.810223+0000","last_fullsized":"2024-03-06T07:22:51.810223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429396+0000","last_change":"2024-03-06T07:22:50.437806+0000","last_active":"2024-03-06T07:22:51.429396+0000","last_peered":"2024-03-06T07:22:51.429396+0000","last_clean":"2024-03-06T07:22:51.429396+0000","last_became_active":"2024-03-06T07:22:50.437511+0000","last_became_peered":"2024-03-06T07:22:50.437511+0000","last_unstale":"2024-03-06T07:22:51.429396+0000","last_undegraded":"2024-03-06T07:22:51.429396+0000","last_fullsized":"2024-03-06T07:22:51.429396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:31.447226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810190+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:51.810190+0000","last_peered":"2024-03-06T07:22:51.810190+0000","last_clean":"2024-03-06T07:22:51.810190+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:51.810190+0000","last_undegraded":"2024-03-06T07:22:51.810190+0000","last_fullsized":"2024-03-06T07:22:51.810190+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810537+0000","last_change":"2024-03-06T07:22:50.452226+0000","last_active":"2024-03-06T07:22:51.810537+0000","last_peered":"2024-03-06T07:22:51.810537+0000","last_clean":"2024-03-06T07:22:51.810537+0000","last_became_active":"2024-03-06T07:22:50.452010+0000","last_became_peered":"2024-03-06T07:22:50.452010+0000","last_unstale":"2024-03-06T07:22:51.810537+0000","last_undegraded":"2024-03-06T07:22:51.810537+0000","last_fullsized":"2024-03-06T07:22:51.810537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:57.070313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812607+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:51.812607+0000","last_peered":"2024-03-06T07:22:51.812607+0000","last_clean":"2024-03-06T07:22:51.812607+0000","last_became_active":"2024-03-06T07:22:35.361468+0000","last_became_peered":"2024-03-06T07:22:35.361468+0000","last_unstale":"2024-03-06T07:22:51.812607+0000","last_undegraded":"2024-03-06T07:22:51.812607+0000","last_fullsized":"2024-03-06T07:22:51.812607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:50.366055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809842+0000","last_change":"2024-03-06T07:22:50.459727+0000","last_active":"2024-03-06T07:22:51.809842+0000","last_peered":"2024-03-06T07:22:51.809842+0000","last_clean":"2024-03-06T07:22:51.809842+0000","last_became_active":"2024-03-06T07:22:50.458917+0000","last_became_peered":"2024-03-06T07:22:50.458917+0000","last_unstale":"2024-03-06T07:22:51.809842+0000","last_undegraded":"2024-03-06T07:22:51.809842+0000","last_fullsized":"2024-03-06T07:22:51.809842+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:57:46.908735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810341+0000","last_change":"2024-03-06T07:22:35.372500+0000","last_active":"2024-03-06T07:22:51.810341+0000","last_peered":"2024-03-06T07:22:51.810341+0000","last_clean":"2024-03-06T07:22:51.810341+0000","last_became_active":"2024-03-06T07:22:35.372221+0000","last_became_peered":"2024-03-06T07:22:35.372221+0000","last_unstale":"2024-03-06T07:22:51.810341+0000","last_undegraded":"2024-03-06T07:22:51.810341+0000","last_fullsized":"2024-03-06T07:22:51.810341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:23.442086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810216+0000","last_change":"2024-03-06T07:22:50.441637+0000","last_active":"2024-03-06T07:22:51.810216+0000","last_peered":"2024-03-06T07:22:51.810216+0000","last_clean":"2024-03-06T07:22:51.810216+0000","last_became_active":"2024-03-06T07:22:50.441392+0000","last_became_peered":"2024-03-06T07:22:50.441392+0000","last_unstale":"2024-03-06T07:22:51.810216+0000","last_undegraded":"2024-03-06T07:22:51.810216+0000","last_fullsized":"2024-03-06T07:22:51.810216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:42.449192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426857+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:51.426857+0000","last_peered":"2024-03-06T07:22:51.426857+0000","last_clean":"2024-03-06T07:22:51.426857+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:51.426857+0000","last_undegraded":"2024-03-06T07:22:51.426857+0000","last_fullsized":"2024-03-06T07:22:51.426857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:41.970389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429211+0000","last_change":"2024-03-06T07:22:50.439323+0000","last_active":"2024-03-06T07:22:51.429211+0000","last_peered":"2024-03-06T07:22:51.429211+0000","last_clean":"2024-03-06T07:22:51.429211+0000","last_became_active":"2024-03-06T07:22:50.438372+0000","last_became_peered":"2024-03-06T07:22:50.438372+0000","last_unstale":"2024-03-06T07:22:51.429211+0000","last_undegraded":"2024-03-06T07:22:51.429211+0000","last_fullsized":"2024-03-06T07:22:51.429211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809692+0000","last_change":"2024-03-06T07:22:35.363020+0000","last_active":"2024-03-06T07:22:51.809692+0000","last_peered":"2024-03-06T07:22:51.809692+0000","last_clean":"2024-03-06T07:22:51.809692+0000","last_became_active":"2024-03-06T07:22:35.362767+0000","last_became_peered":"2024-03-06T07:22:35.362767+0000","last_unstale":"2024-03-06T07:22:51.809692+0000","last_undegraded":"2024-03-06T07:22:51.809692+0000","last_fullsized":"2024-03-06T07:22:51.809692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:56.148980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811625+0000","last_change":"2024-03-06T07:22:50.449703+0000","last_active":"2024-03-06T07:22:51.811625+0000","last_peered":"2024-03-06T07:22:51.811625+0000","last_clean":"2024-03-06T07:22:51.811625+0000","last_became_active":"2024-03-06T07:22:50.449106+0000","last_became_peered":"2024-03-06T07:22:50.449106+0000","last_unstale":"2024-03-06T07:22:51.811625+0000","last_undegraded":"2024-03-06T07:22:51.811625+0000","last_fullsized":"2024-03-06T07:22:51.811625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:52.147096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812090+0000","last_change":"2024-03-06T07:22:49.443093+0000","last_active":"2024-03-06T07:22:51.812090+0000","last_peered":"2024-03-06T07:22:51.812090+0000","last_clean":"2024-03-06T07:22:51.812090+0000","last_became_active":"2024-03-06T07:22:49.442628+0000","last_became_peered":"2024-03-06T07:22:49.442628+0000","last_unstale":"2024-03-06T07:22:51.812090+0000","last_undegraded":"2024-03-06T07:22:51.812090+0000","last_fullsized":"2024-03-06T07:22:51.812090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:56.065095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426906+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:51.426906+0000","last_peered":"2024-03-06T07:22:51.426906+0000","last_clean":"2024-03-06T07:22:51.426906+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:51.426906+0000","last_undegraded":"2024-03-06T07:22:51.426906+0000","last_fullsized":"2024-03-06T07:22:51.426906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812431+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:51.812431+0000","last_peered":"2024-03-06T07:22:51.812431+0000","last_clean":"2024-03-06T07:22:51.812431+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:51.812431+0000","last_undegraded":"2024-03-06T07:22:51.812431+0000","last_fullsized":"2024-03-06T07:22:51.812431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427171+0000","last_change":"2024-03-06T07:22:50.440550+0000","last_active":"2024-03-06T07:22:51.427171+0000","last_peered":"2024-03-06T07:22:51.427171+0000","last_clean":"2024-03-06T07:22:51.427171+0000","last_became_active":"2024-03-06T07:22:50.440230+0000","last_became_peered":"2024-03-06T07:22:50.440230+0000","last_unstale":"2024-03-06T07:22:51.427171+0000","last_undegraded":"2024-03-06T07:22:51.427171+0000","last_fullsized":"2024-03-06T07:22:51.427171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:29.051388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809740+0000","last_change":"2024-03-06T07:22:49.442614+0000","last_active":"2024-03-06T07:22:51.809740+0000","last_peered":"2024-03-06T07:22:51.809740+0000","last_clean":"2024-03-06T07:22:51.809740+0000","last_became_active":"2024-03-06T07:22:49.442328+0000","last_became_peered":"2024-03-06T07:22:49.442328+0000","last_unstale":"2024-03-06T07:22:51.809740+0000","last_undegraded":"2024-03-06T07:22:51.809740+0000","last_fullsized":"2024-03-06T07:22:51.809740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.026177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812003+0000","last_change":"2024-03-06T07:22:35.368816+0000","last_active":"2024-03-06T07:22:51.812003+0000","last_peered":"2024-03-06T07:22:51.812003+0000","last_clean":"2024-03-06T07:22:51.812003+0000","last_became_active":"2024-03-06T07:22:35.368555+0000","last_became_peered":"2024-03-06T07:22:35.368555+0000","last_unstale":"2024-03-06T07:22:51.812003+0000","last_undegraded":"2024-03-06T07:22:51.812003+0000","last_fullsized":"2024-03-06T07:22:51.812003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810001+0000","last_change":"2024-03-06T07:22:34.366335+0000","last_active":"2024-03-06T07:22:51.810001+0000","last_peered":"2024-03-06T07:22:51.810001+0000","last_clean":"2024-03-06T07:22:51.810001+0000","last_became_active":"2024-03-06T07:22:34.364387+0000","last_became_peered":"2024-03-06T07:22:34.364387+0000","last_unstale":"2024-03-06T07:22:51.810001+0000","last_undegraded":"2024-03-06T07:22:51.810001+0000","last_fullsized":"2024-03-06T07:22:51.810001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426615+0000","last_change":"2024-03-06T07:22:50.444770+0000","last_active":"2024-03-06T07:22:51.426615+0000","last_peered":"2024-03-06T07:22:51.426615+0000","last_clean":"2024-03-06T07:22:51.426615+0000","last_became_active":"2024-03-06T07:22:50.444438+0000","last_became_peered":"2024-03-06T07:22:50.444438+0000","last_unstale":"2024-03-06T07:22:51.426615+0000","last_undegraded":"2024-03-06T07:22:51.426615+0000","last_fullsized":"2024-03-06T07:22:51.426615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429288+0000","last_change":"2024-03-06T07:22:49.437241+0000","last_active":"2024-03-06T07:22:51.429288+0000","last_peered":"2024-03-06T07:22:51.429288+0000","last_clean":"2024-03-06T07:22:51.429288+0000","last_became_active":"2024-03-06T07:22:49.436832+0000","last_became_peered":"2024-03-06T07:22:49.436832+0000","last_unstale":"2024-03-06T07:22:51.429288+0000","last_undegraded":"2024-03-06T07:22:51.429288+0000","last_fullsized":"2024-03-06T07:22:51.429288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:35.129907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429558+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:51.429558+0000","last_peered":"2024-03-06T07:22:51.429558+0000","last_clean":"2024-03-06T07:22:51.429558+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:51.429558+0000","last_undegraded":"2024-03-06T07:22:51.429558+0000","last_fullsized":"2024-03-06T07:22:51.429558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810318+0000","last_change":"2024-03-06T07:22:34.364533+0000","last_active":"2024-03-06T07:22:51.810318+0000","last_peered":"2024-03-06T07:22:51.810318+0000","last_clean":"2024-03-06T07:22:51.810318+0000","last_became_active":"2024-03-06T07:22:34.364224+0000","last_became_peered":"2024-03-06T07:22:34.364224+0000","last_unstale":"2024-03-06T07:22:51.810318+0000","last_undegraded":"2024-03-06T07:22:51.810318+0000","last_fullsized":"2024-03-06T07:22:51.810318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:13.454983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430167+0000","last_change":"2024-03-06T07:22:50.453853+0000","last_active":"2024-03-06T07:22:51.430167+0000","last_peered":"2024-03-06T07:22:51.430167+0000","last_clean":"2024-03-06T07:22:51.430167+0000","last_became_active":"2024-03-06T07:22:50.453578+0000","last_became_peered":"2024-03-06T07:22:50.453578+0000","last_unstale":"2024-03-06T07:22:51.430167+0000","last_undegraded":"2024-03-06T07:22:51.430167+0000","last_fullsized":"2024-03-06T07:22:51.430167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:34.188282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427107+0000","last_change":"2024-03-06T07:22:49.444227+0000","last_active":"2024-03-06T07:22:51.427107+0000","last_peered":"2024-03-06T07:22:51.427107+0000","last_clean":"2024-03-06T07:22:51.427107+0000","last_became_active":"2024-03-06T07:22:49.443963+0000","last_became_peered":"2024-03-06T07:22:49.443963+0000","last_unstale":"2024-03-06T07:22:51.427107+0000","last_undegraded":"2024-03-06T07:22:51.427107+0000","last_fullsized":"2024-03-06T07:22:51.427107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:56.306976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429664+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:51.429664+0000","last_peered":"2024-03-06T07:22:51.429664+0000","last_clean":"2024-03-06T07:22:51.429664+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:51.429664+0000","last_undegraded":"2024-03-06T07:22:51.429664+0000","last_fullsized":"2024-03-06T07:22:51.429664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431057+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:51.431057+0000","last_peered":"2024-03-06T07:22:51.431057+0000","last_clean":"2024-03-06T07:22:51.431057+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:51.431057+0000","last_undegraded":"2024-03-06T07:22:51.431057+0000","last_fullsized":"2024-03-06T07:22:51.431057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809934+0000","last_change":"2024-03-06T07:22:50.444294+0000","last_active":"2024-03-06T07:22:51.809934+0000","last_peered":"2024-03-06T07:22:51.809934+0000","last_clean":"2024-03-06T07:22:51.809934+0000","last_became_active":"2024-03-06T07:22:50.444066+0000","last_became_peered":"2024-03-06T07:22:50.444066+0000","last_unstale":"2024-03-06T07:22:51.809934+0000","last_undegraded":"2024-03-06T07:22:51.809934+0000","last_fullsized":"2024-03-06T07:22:51.809934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:34.201578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809674+0000","last_change":"2024-03-06T07:22:49.446138+0000","last_active":"2024-03-06T07:22:51.809674+0000","last_peered":"2024-03-06T07:22:51.809674+0000","last_clean":"2024-03-06T07:22:51.809674+0000","last_became_active":"2024-03-06T07:22:49.445911+0000","last_became_peered":"2024-03-06T07:22:49.445911+0000","last_unstale":"2024-03-06T07:22:51.809674+0000","last_undegraded":"2024-03-06T07:22:51.809674+0000","last_fullsized":"2024-03-06T07:22:51.809674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:15.943887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811636+0000","last_change":"2024-03-06T07:22:35.365350+0000","last_active":"2024-03-06T07:22:51.811636+0000","last_peered":"2024-03-06T07:22:51.811636+0000","last_clean":"2024-03-06T07:22:51.811636+0000","last_became_active":"2024-03-06T07:22:35.365049+0000","last_became_peered":"2024-03-06T07:22:35.365049+0000","last_unstale":"2024-03-06T07:22:51.811636+0000","last_undegraded":"2024-03-06T07:22:51.811636+0000","last_fullsized":"2024-03-06T07:22:51.811636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429736+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:51.429736+0000","last_peered":"2024-03-06T07:22:51.429736+0000","last_clean":"2024-03-06T07:22:51.429736+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:51.429736+0000","last_undegraded":"2024-03-06T07:22:51.429736+0000","last_fullsized":"2024-03-06T07:22:51.429736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812558+0000","last_change":"2024-03-06T07:22:50.451370+0000","last_active":"2024-03-06T07:22:51.812558+0000","last_peered":"2024-03-06T07:22:51.812558+0000","last_clean":"2024-03-06T07:22:51.812558+0000","last_became_active":"2024-03-06T07:22:50.451068+0000","last_became_peered":"2024-03-06T07:22:50.451068+0000","last_unstale":"2024-03-06T07:22:51.812558+0000","last_undegraded":"2024-03-06T07:22:51.812558+0000","last_fullsized":"2024-03-06T07:22:51.812558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:53.346438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430820+0000","last_change":"2024-03-06T07:22:49.430583+0000","last_active":"2024-03-06T07:22:51.430820+0000","last_peered":"2024-03-06T07:22:51.430820+0000","last_clean":"2024-03-06T07:22:51.430820+0000","last_became_active":"2024-03-06T07:22:49.430315+0000","last_became_peered":"2024-03-06T07:22:49.430315+0000","last_unstale":"2024-03-06T07:22:51.430820+0000","last_undegraded":"2024-03-06T07:22:51.430820+0000","last_fullsized":"2024-03-06T07:22:51.430820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812287+0000","last_change":"2024-03-06T07:22:35.363944+0000","last_active":"2024-03-06T07:22:51.812287+0000","last_peered":"2024-03-06T07:22:51.812287+0000","last_clean":"2024-03-06T07:22:51.812287+0000","last_became_active":"2024-03-06T07:22:35.363662+0000","last_became_peered":"2024-03-06T07:22:35.363662+0000","last_unstale":"2024-03-06T07:22:51.812287+0000","last_undegraded":"2024-03-06T07:22:51.812287+0000","last_fullsized":"2024-03-06T07:22:51.812287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:17.265969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809835+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:51.809835+0000","last_peered":"2024-03-06T07:22:51.809835+0000","last_clean":"2024-03-06T07:22:51.809835+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:51.809835+0000","last_undegraded":"2024-03-06T07:22:51.809835+0000","last_fullsized":"2024-03-06T07:22:51.809835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811398+0000","last_change":"2024-03-06T07:22:50.453874+0000","last_active":"2024-03-06T07:22:51.811398+0000","last_peered":"2024-03-06T07:22:51.811398+0000","last_clean":"2024-03-06T07:22:51.811398+0000","last_became_active":"2024-03-06T07:22:50.453403+0000","last_became_peered":"2024-03-06T07:22:50.453403+0000","last_unstale":"2024-03-06T07:22:51.811398+0000","last_undegraded":"2024-03-06T07:22:51.811398+0000","last_fullsized":"2024-03-06T07:22:51.811398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:49:39.691686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430540+0000","last_change":"2024-03-06T07:22:49.443801+0000","last_active":"2024-03-06T07:22:51.430540+0000","last_peered":"2024-03-06T07:22:51.430540+0000","last_clean":"2024-03-06T07:22:51.430540+0000","last_became_active":"2024-03-06T07:22:49.443540+0000","last_became_peered":"2024-03-06T07:22:49.443540+0000","last_unstale":"2024-03-06T07:22:51.430540+0000","last_undegraded":"2024-03-06T07:22:51.430540+0000","last_fullsized":"2024-03-06T07:22:51.430540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:01.309770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430841+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:51.430841+0000","last_peered":"2024-03-06T07:22:51.430841+0000","last_clean":"2024-03-06T07:22:51.430841+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:51.430841+0000","last_undegraded":"2024-03-06T07:22:51.430841+0000","last_fullsized":"2024-03-06T07:22:51.430841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812003+0000","last_change":"2024-03-06T07:22:34.366433+0000","last_active":"2024-03-06T07:22:51.812003+0000","last_peered":"2024-03-06T07:22:51.812003+0000","last_clean":"2024-03-06T07:22:51.812003+0000","last_became_active":"2024-03-06T07:22:34.365956+0000","last_became_peered":"2024-03-06T07:22:34.365956+0000","last_unstale":"2024-03-06T07:22:51.812003+0000","last_undegraded":"2024-03-06T07:22:51.812003+0000","last_fullsized":"2024-03-06T07:22:51.812003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812114+0000","last_change":"2024-03-06T07:22:50.453212+0000","last_active":"2024-03-06T07:22:51.812114+0000","last_peered":"2024-03-06T07:22:51.812114+0000","last_clean":"2024-03-06T07:22:51.812114+0000","last_became_active":"2024-03-06T07:22:50.452795+0000","last_became_peered":"2024-03-06T07:22:50.452795+0000","last_unstale":"2024-03-06T07:22:51.812114+0000","last_undegraded":"2024-03-06T07:22:51.812114+0000","last_fullsized":"2024-03-06T07:22:51.812114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:07.928847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430576+0000","last_change":"2024-03-06T07:22:49.439528+0000","last_active":"2024-03-06T07:22:51.430576+0000","last_peered":"2024-03-06T07:22:51.430576+0000","last_clean":"2024-03-06T07:22:51.430576+0000","last_became_active":"2024-03-06T07:22:49.439249+0000","last_became_peered":"2024-03-06T07:22:49.439249+0000","last_unstale":"2024-03-06T07:22:51.430576+0000","last_undegraded":"2024-03-06T07:22:51.430576+0000","last_fullsized":"2024-03-06T07:22:51.430576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:08.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430797+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:51.430797+0000","last_peered":"2024-03-06T07:22:51.430797+0000","last_clean":"2024-03-06T07:22:51.430797+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:51.430797+0000","last_undegraded":"2024-03-06T07:22:51.430797+0000","last_fullsized":"2024-03-06T07:22:51.430797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:24.785053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809979+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:51.809979+0000","last_peered":"2024-03-06T07:22:51.809979+0000","last_clean":"2024-03-06T07:22:51.809979+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:51.809979+0000","last_undegraded":"2024-03-06T07:22:51.809979+0000","last_fullsized":"2024-03-06T07:22:51.809979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810113+0000","last_change":"2024-03-06T07:22:50.437572+0000","last_active":"2024-03-06T07:22:51.810113+0000","last_peered":"2024-03-06T07:22:51.810113+0000","last_clean":"2024-03-06T07:22:51.810113+0000","last_became_active":"2024-03-06T07:22:50.437290+0000","last_became_peered":"2024-03-06T07:22:50.437290+0000","last_unstale":"2024-03-06T07:22:51.810113+0000","last_undegraded":"2024-03-06T07:22:51.810113+0000","last_fullsized":"2024-03-06T07:22:51.810113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:17.922064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426849+0000","last_change":"2024-03-06T07:22:49.431955+0000","last_active":"2024-03-06T07:22:51.426849+0000","last_peered":"2024-03-06T07:22:51.426849+0000","last_clean":"2024-03-06T07:22:51.426849+0000","last_became_active":"2024-03-06T07:22:49.431192+0000","last_became_peered":"2024-03-06T07:22:49.431192+0000","last_unstale":"2024-03-06T07:22:51.426849+0000","last_undegraded":"2024-03-06T07:22:51.426849+0000","last_fullsized":"2024-03-06T07:22:51.426849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:46.923868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811787+0000","last_change":"2024-03-06T07:22:35.365470+0000","last_active":"2024-03-06T07:22:51.811787+0000","last_peered":"2024-03-06T07:22:51.811787+0000","last_clean":"2024-03-06T07:22:51.811787+0000","last_became_active":"2024-03-06T07:22:35.365156+0000","last_became_peered":"2024-03-06T07:22:35.365156+0000","last_unstale":"2024-03-06T07:22:51.811787+0000","last_undegraded":"2024-03-06T07:22:51.811787+0000","last_fullsized":"2024-03-06T07:22:51.811787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:45.193188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809999+0000","last_change":"2024-03-06T07:22:34.366420+0000","last_active":"2024-03-06T07:22:51.809999+0000","last_peered":"2024-03-06T07:22:51.809999+0000","last_clean":"2024-03-06T07:22:51.809999+0000","last_became_active":"2024-03-06T07:22:34.365577+0000","last_became_peered":"2024-03-06T07:22:34.365577+0000","last_unstale":"2024-03-06T07:22:51.809999+0000","last_undegraded":"2024-03-06T07:22:51.809999+0000","last_fullsized":"2024-03-06T07:22:51.809999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:20.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810140+0000","last_change":"2024-03-06T07:22:50.438430+0000","last_active":"2024-03-06T07:22:51.810140+0000","last_peered":"2024-03-06T07:22:51.810140+0000","last_clean":"2024-03-06T07:22:51.810140+0000","last_became_active":"2024-03-06T07:22:50.438163+0000","last_became_peered":"2024-03-06T07:22:50.438163+0000","last_unstale":"2024-03-06T07:22:51.810140+0000","last_undegraded":"2024-03-06T07:22:51.810140+0000","last_fullsized":"2024-03-06T07:22:51.810140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:31.237118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426732+0000","last_change":"2024-03-06T07:22:49.441677+0000","last_active":"2024-03-06T07:22:51.426732+0000","last_peered":"2024-03-06T07:22:51.426732+0000","last_clean":"2024-03-06T07:22:51.426732+0000","last_became_active":"2024-03-06T07:22:49.441346+0000","last_became_peered":"2024-03-06T07:22:49.441346+0000","last_unstale":"2024-03-06T07:22:51.426732+0000","last_undegraded":"2024-03-06T07:22:51.426732+0000","last_fullsized":"2024-03-06T07:22:51.426732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810584+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:51.810584+0000","last_peered":"2024-03-06T07:22:51.810584+0000","last_clean":"2024-03-06T07:22:51.810584+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:51.810584+0000","last_undegraded":"2024-03-06T07:22:51.810584+0000","last_fullsized":"2024-03-06T07:22:51.810584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:32.849939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810474+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:51.810474+0000","last_peered":"2024-03-06T07:22:51.810474+0000","last_clean":"2024-03-06T07:22:51.810474+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:51.810474+0000","last_undegraded":"2024-03-06T07:22:51.810474+0000","last_fullsized":"2024-03-06T07:22:51.810474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426742+0000","last_change":"2024-03-06T07:22:50.453384+0000","last_active":"2024-03-06T07:22:51.426742+0000","last_peered":"2024-03-06T07:22:51.426742+0000","last_clean":"2024-03-06T07:22:51.426742+0000","last_became_active":"2024-03-06T07:22:50.452549+0000","last_became_peered":"2024-03-06T07:22:50.452549+0000","last_unstale":"2024-03-06T07:22:51.426742+0000","last_undegraded":"2024-03-06T07:22:51.426742+0000","last_fullsized":"2024-03-06T07:22:51.426742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:31.447226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810308+0000","last_change":"2024-03-06T07:22:49.443011+0000","last_active":"2024-03-06T07:22:51.810308+0000","last_peered":"2024-03-06T07:22:51.810308+0000","last_clean":"2024-03-06T07:22:51.810308+0000","last_became_active":"2024-03-06T07:22:49.442745+0000","last_became_peered":"2024-03-06T07:22:49.442745+0000","last_unstale":"2024-03-06T07:22:51.810308+0000","last_undegraded":"2024-03-06T07:22:51.810308+0000","last_fullsized":"2024-03-06T07:22:51.810308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:31.339898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811153+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:51.811153+0000","last_peered":"2024-03-06T07:22:51.811153+0000","last_clean":"2024-03-06T07:22:51.811153+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:51.811153+0000","last_undegraded":"2024-03-06T07:22:51.811153+0000","last_fullsized":"2024-03-06T07:22:51.811153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810717+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:51.810717+0000","last_peered":"2024-03-06T07:22:51.810717+0000","last_clean":"2024-03-06T07:22:51.810717+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:51.810717+0000","last_undegraded":"2024-03-06T07:22:51.810717+0000","last_fullsized":"2024-03-06T07:22:51.810717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429075+0000","last_change":"2024-03-06T07:22:50.430626+0000","last_active":"2024-03-06T07:22:51.429075+0000","last_peered":"2024-03-06T07:22:51.429075+0000","last_clean":"2024-03-06T07:22:51.429075+0000","last_became_active":"2024-03-06T07:22:50.430335+0000","last_became_peered":"2024-03-06T07:22:50.430335+0000","last_unstale":"2024-03-06T07:22:51.429075+0000","last_undegraded":"2024-03-06T07:22:51.429075+0000","last_fullsized":"2024-03-06T07:22:51.429075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:34.188282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811394+0000","last_change":"2024-03-06T07:22:49.441435+0000","last_active":"2024-03-06T07:22:51.811394+0000","last_peered":"2024-03-06T07:22:51.811394+0000","last_clean":"2024-03-06T07:22:51.811394+0000","last_became_active":"2024-03-06T07:22:49.441150+0000","last_became_peered":"2024-03-06T07:22:49.441150+0000","last_unstale":"2024-03-06T07:22:51.811394+0000","last_undegraded":"2024-03-06T07:22:51.811394+0000","last_fullsized":"2024-03-06T07:22:51.811394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:56.065095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427220+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:51.427220+0000","last_peered":"2024-03-06T07:22:51.427220+0000","last_clean":"2024-03-06T07:22:51.427220+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:51.427220+0000","last_undegraded":"2024-03-06T07:22:51.427220+0000","last_fullsized":"2024-03-06T07:22:51.427220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:24.785053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431109+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:51.431109+0000","last_peered":"2024-03-06T07:22:51.431109+0000","last_clean":"2024-03-06T07:22:51.431109+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:51.431109+0000","last_undegraded":"2024-03-06T07:22:51.431109+0000","last_fullsized":"2024-03-06T07:22:51.431109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811864+0000","last_change":"2024-03-06T07:22:50.458465+0000","last_active":"2024-03-06T07:22:51.811864+0000","last_peered":"2024-03-06T07:22:51.811864+0000","last_clean":"2024-03-06T07:22:51.811864+0000","last_became_active":"2024-03-06T07:22:50.458179+0000","last_became_peered":"2024-03-06T07:22:50.458179+0000","last_unstale":"2024-03-06T07:22:51.811864+0000","last_undegraded":"2024-03-06T07:22:51.811864+0000","last_fullsized":"2024-03-06T07:22:51.811864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:09.075888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430286+0000","last_change":"2024-03-06T07:22:49.443958+0000","last_active":"2024-03-06T07:22:51.430286+0000","last_peered":"2024-03-06T07:22:51.430286+0000","last_clean":"2024-03-06T07:22:51.430286+0000","last_became_active":"2024-03-06T07:22:49.443712+0000","last_became_peered":"2024-03-06T07:22:49.443712+0000","last_unstale":"2024-03-06T07:22:51.430286+0000","last_undegraded":"2024-03-06T07:22:51.430286+0000","last_fullsized":"2024-03-06T07:22:51.430286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:19.265304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810422+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:51.810422+0000","last_peered":"2024-03-06T07:22:51.810422+0000","last_clean":"2024-03-06T07:22:51.810422+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:51.810422+0000","last_undegraded":"2024-03-06T07:22:51.810422+0000","last_fullsized":"2024-03-06T07:22:51.810422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:41.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810705+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:51.810705+0000","last_peered":"2024-03-06T07:22:51.810705+0000","last_clean":"2024-03-06T07:22:51.810705+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:51.810705+0000","last_undegraded":"2024-03-06T07:22:51.810705+0000","last_fullsized":"2024-03-06T07:22:51.810705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427835+0000","last_change":"2024-03-06T07:22:50.442907+0000","last_active":"2024-03-06T07:22:51.427835+0000","last_peered":"2024-03-06T07:22:51.427835+0000","last_clean":"2024-03-06T07:22:51.427835+0000","last_became_active":"2024-03-06T07:22:50.441488+0000","last_became_peered":"2024-03-06T07:22:50.441488+0000","last_unstale":"2024-03-06T07:22:51.427835+0000","last_undegraded":"2024-03-06T07:22:51.427835+0000","last_fullsized":"2024-03-06T07:22:51.427835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811509+0000","last_change":"2024-03-06T07:22:49.444169+0000","last_active":"2024-03-06T07:22:51.811509+0000","last_peered":"2024-03-06T07:22:51.811509+0000","last_clean":"2024-03-06T07:22:51.811509+0000","last_became_active":"2024-03-06T07:22:49.443899+0000","last_became_peered":"2024-03-06T07:22:49.443899+0000","last_unstale":"2024-03-06T07:22:51.811509+0000","last_undegraded":"2024-03-06T07:22:51.811509+0000","last_fullsized":"2024-03-06T07:22:51.811509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:13.223654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430361+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:51.430361+0000","last_peered":"2024-03-06T07:22:51.430361+0000","last_clean":"2024-03-06T07:22:51.430361+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:51.430361+0000","last_undegraded":"2024-03-06T07:22:51.430361+0000","last_fullsized":"2024-03-06T07:22:51.430361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810433+0000","last_change":"2024-03-06T07:22:34.364874+0000","last_active":"2024-03-06T07:22:51.810433+0000","last_peered":"2024-03-06T07:22:51.810433+0000","last_clean":"2024-03-06T07:22:51.810433+0000","last_became_active":"2024-03-06T07:22:34.364472+0000","last_became_peered":"2024-03-06T07:22:34.364472+0000","last_unstale":"2024-03-06T07:22:51.810433+0000","last_undegraded":"2024-03-06T07:22:51.810433+0000","last_fullsized":"2024-03-06T07:22:51.810433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429504+0000","last_change":"2024-03-06T07:22:50.443326+0000","last_active":"2024-03-06T07:22:51.429504+0000","last_peered":"2024-03-06T07:22:51.429504+0000","last_clean":"2024-03-06T07:22:51.429504+0000","last_became_active":"2024-03-06T07:22:50.442437+0000","last_became_peered":"2024-03-06T07:22:50.442437+0000","last_unstale":"2024-03-06T07:22:51.429504+0000","last_undegraded":"2024-03-06T07:22:51.429504+0000","last_fullsized":"2024-03-06T07:22:51.429504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:29.051388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810109+0000","last_change":"2024-03-06T07:22:49.435101+0000","last_active":"2024-03-06T07:22:51.810109+0000","last_peered":"2024-03-06T07:22:51.810109+0000","last_clean":"2024-03-06T07:22:51.810109+0000","last_became_active":"2024-03-06T07:22:49.434840+0000","last_became_peered":"2024-03-06T07:22:49.434840+0000","last_unstale":"2024-03-06T07:22:51.810109+0000","last_undegraded":"2024-03-06T07:22:51.810109+0000","last_fullsized":"2024-03-06T07:22:51.810109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:30.442008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430287+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:51.430287+0000","last_peered":"2024-03-06T07:22:51.430287+0000","last_clean":"2024-03-06T07:22:51.430287+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:51.430287+0000","last_undegraded":"2024-03-06T07:22:51.430287+0000","last_fullsized":"2024-03-06T07:22:51.430287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:41.970389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809888+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:51.809888+0000","last_peered":"2024-03-06T07:22:51.809888+0000","last_clean":"2024-03-06T07:22:51.809888+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:51.809888+0000","last_undegraded":"2024-03-06T07:22:51.809888+0000","last_fullsized":"2024-03-06T07:22:51.809888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810508+0000","last_change":"2024-03-06T07:22:50.448230+0000","last_active":"2024-03-06T07:22:51.810508+0000","last_peered":"2024-03-06T07:22:51.810508+0000","last_clean":"2024-03-06T07:22:51.810508+0000","last_became_active":"2024-03-06T07:22:50.447938+0000","last_became_peered":"2024-03-06T07:22:50.447938+0000","last_unstale":"2024-03-06T07:22:51.810508+0000","last_undegraded":"2024-03-06T07:22:51.810508+0000","last_fullsized":"2024-03-06T07:22:51.810508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:49.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811952+0000","last_change":"2024-03-06T07:22:49.444457+0000","last_active":"2024-03-06T07:22:51.811952+0000","last_peered":"2024-03-06T07:22:51.811952+0000","last_clean":"2024-03-06T07:22:51.811952+0000","last_became_active":"2024-03-06T07:22:49.444169+0000","last_became_peered":"2024-03-06T07:22:49.444169+0000","last_unstale":"2024-03-06T07:22:51.811952+0000","last_undegraded":"2024-03-06T07:22:51.811952+0000","last_fullsized":"2024-03-06T07:22:51.811952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:42.370899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430818+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:51.430818+0000","last_peered":"2024-03-06T07:22:51.430818+0000","last_clean":"2024-03-06T07:22:51.430818+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:51.430818+0000","last_undegraded":"2024-03-06T07:22:51.430818+0000","last_fullsized":"2024-03-06T07:22:51.430818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:54.954958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811885+0000","last_change":"2024-03-06T07:22:34.361474+0000","last_active":"2024-03-06T07:22:51.811885+0000","last_peered":"2024-03-06T07:22:51.811885+0000","last_clean":"2024-03-06T07:22:51.811885+0000","last_became_active":"2024-03-06T07:22:34.360526+0000","last_became_peered":"2024-03-06T07:22:34.360526+0000","last_unstale":"2024-03-06T07:22:51.811885+0000","last_undegraded":"2024-03-06T07:22:51.811885+0000","last_fullsized":"2024-03-06T07:22:51.811885+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809784+0000","last_change":"2024-03-06T07:22:50.443740+0000","last_active":"2024-03-06T07:22:51.809784+0000","last_peered":"2024-03-06T07:22:51.809784+0000","last_clean":"2024-03-06T07:22:51.809784+0000","last_became_active":"2024-03-06T07:22:50.443428+0000","last_became_peered":"2024-03-06T07:22:50.443428+0000","last_unstale":"2024-03-06T07:22:51.809784+0000","last_undegraded":"2024-03-06T07:22:51.809784+0000","last_fullsized":"2024-03-06T07:22:51.809784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810137+0000","last_change":"2024-03-06T07:22:49.442302+0000","last_active":"2024-03-06T07:22:51.810137+0000","last_peered":"2024-03-06T07:22:51.810137+0000","last_clean":"2024-03-06T07:22:51.810137+0000","last_became_active":"2024-03-06T07:22:49.442038+0000","last_became_peered":"2024-03-06T07:22:49.442038+0000","last_unstale":"2024-03-06T07:22:51.810137+0000","last_undegraded":"2024-03-06T07:22:51.810137+0000","last_fullsized":"2024-03-06T07:22:51.810137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:04.439984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809946+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:51.809946+0000","last_peered":"2024-03-06T07:22:51.809946+0000","last_clean":"2024-03-06T07:22:51.809946+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:51.809946+0000","last_undegraded":"2024-03-06T07:22:51.809946+0000","last_fullsized":"2024-03-06T07:22:51.809946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810097+0000","last_change":"2024-03-06T07:22:34.365404+0000","last_active":"2024-03-06T07:22:51.810097+0000","last_peered":"2024-03-06T07:22:51.810097+0000","last_clean":"2024-03-06T07:22:51.810097+0000","last_became_active":"2024-03-06T07:22:34.365029+0000","last_became_peered":"2024-03-06T07:22:34.365029+0000","last_unstale":"2024-03-06T07:22:51.810097+0000","last_undegraded":"2024-03-06T07:22:51.810097+0000","last_fullsized":"2024-03-06T07:22:51.810097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.473973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812499+0000","last_change":"2024-03-06T07:22:50.454823+0000","last_active":"2024-03-06T07:22:51.812499+0000","last_peered":"2024-03-06T07:22:51.812499+0000","last_clean":"2024-03-06T07:22:51.812499+0000","last_became_active":"2024-03-06T07:22:50.454554+0000","last_became_peered":"2024-03-06T07:22:50.454554+0000","last_unstale":"2024-03-06T07:22:51.812499+0000","last_undegraded":"2024-03-06T07:22:51.812499+0000","last_fullsized":"2024-03-06T07:22:51.812499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:22.682632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810092+0000","last_change":"2024-03-06T07:22:49.439149+0000","last_active":"2024-03-06T07:22:51.810092+0000","last_peered":"2024-03-06T07:22:51.810092+0000","last_clean":"2024-03-06T07:22:51.810092+0000","last_became_active":"2024-03-06T07:22:49.438863+0000","last_became_peered":"2024-03-06T07:22:49.438863+0000","last_unstale":"2024-03-06T07:22:51.810092+0000","last_undegraded":"2024-03-06T07:22:51.810092+0000","last_fullsized":"2024-03-06T07:22:51.810092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:10.816265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809765+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:51.809765+0000","last_peered":"2024-03-06T07:22:51.809765+0000","last_clean":"2024-03-06T07:22:51.809765+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:51.809765+0000","last_undegraded":"2024-03-06T07:22:51.809765+0000","last_fullsized":"2024-03-06T07:22:51.809765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430933+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:51.430933+0000","last_peered":"2024-03-06T07:22:51.430933+0000","last_clean":"2024-03-06T07:22:51.430933+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:51.430933+0000","last_undegraded":"2024-03-06T07:22:51.430933+0000","last_fullsized":"2024-03-06T07:22:51.430933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811433+0000","last_change":"2024-03-06T07:22:50.459509+0000","last_active":"2024-03-06T07:22:51.811433+0000","last_peered":"2024-03-06T07:22:51.811433+0000","last_clean":"2024-03-06T07:22:51.811433+0000","last_became_active":"2024-03-06T07:22:50.458980+0000","last_became_peered":"2024-03-06T07:22:50.458980+0000","last_unstale":"2024-03-06T07:22:51.811433+0000","last_undegraded":"2024-03-06T07:22:51.811433+0000","last_fullsized":"2024-03-06T07:22:51.811433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:09.219746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810745+0000","last_change":"2024-03-06T07:22:49.440502+0000","last_active":"2024-03-06T07:22:51.810745+0000","last_peered":"2024-03-06T07:22:51.810745+0000","last_clean":"2024-03-06T07:22:51.810745+0000","last_became_active":"2024-03-06T07:22:49.440225+0000","last_became_peered":"2024-03-06T07:22:49.440225+0000","last_unstale":"2024-03-06T07:22:51.810745+0000","last_undegraded":"2024-03-06T07:22:51.810745+0000","last_fullsized":"2024-03-06T07:22:51.810745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:33.183064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810488+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:51.810488+0000","last_peered":"2024-03-06T07:22:51.810488+0000","last_clean":"2024-03-06T07:22:51.810488+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:51.810488+0000","last_undegraded":"2024-03-06T07:22:51.810488+0000","last_fullsized":"2024-03-06T07:22:51.810488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810893+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:51.810893+0000","last_peered":"2024-03-06T07:22:51.810893+0000","last_clean":"2024-03-06T07:22:51.810893+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:51.810893+0000","last_undegraded":"2024-03-06T07:22:51.810893+0000","last_fullsized":"2024-03-06T07:22:51.810893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810044+0000","last_change":"2024-03-06T07:22:50.446528+0000","last_active":"2024-03-06T07:22:51.810044+0000","last_peered":"2024-03-06T07:22:51.810044+0000","last_clean":"2024-03-06T07:22:51.810044+0000","last_became_active":"2024-03-06T07:22:50.446004+0000","last_became_peered":"2024-03-06T07:22:50.446004+0000","last_unstale":"2024-03-06T07:22:51.810044+0000","last_undegraded":"2024-03-06T07:22:51.810044+0000","last_fullsized":"2024-03-06T07:22:51.810044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:16.949105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810375+0000","last_change":"2024-03-06T07:22:49.446813+0000","last_active":"2024-03-06T07:22:51.810375+0000","last_peered":"2024-03-06T07:22:51.810375+0000","last_clean":"2024-03-06T07:22:51.810375+0000","last_became_active":"2024-03-06T07:22:49.446544+0000","last_became_peered":"2024-03-06T07:22:49.446544+0000","last_unstale":"2024-03-06T07:22:51.810375+0000","last_undegraded":"2024-03-06T07:22:51.810375+0000","last_fullsized":"2024-03-06T07:22:51.810375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:09.939716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810301+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:51.810301+0000","last_peered":"2024-03-06T07:22:51.810301+0000","last_clean":"2024-03-06T07:22:51.810301+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:51.810301+0000","last_undegraded":"2024-03-06T07:22:51.810301+0000","last_fullsized":"2024-03-06T07:22:51.810301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427806+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:51.427806+0000","last_peered":"2024-03-06T07:22:51.427806+0000","last_clean":"2024-03-06T07:22:51.427806+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:51.427806+0000","last_undegraded":"2024-03-06T07:22:51.427806+0000","last_fullsized":"2024-03-06T07:22:51.427806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431066+0000","last_change":"2024-03-06T07:22:50.433988+0000","last_active":"2024-03-06T07:22:51.431066+0000","last_peered":"2024-03-06T07:22:51.431066+0000","last_clean":"2024-03-06T07:22:51.431066+0000","last_became_active":"2024-03-06T07:22:50.433697+0000","last_became_peered":"2024-03-06T07:22:50.433697+0000","last_unstale":"2024-03-06T07:22:51.431066+0000","last_undegraded":"2024-03-06T07:22:51.431066+0000","last_fullsized":"2024-03-06T07:22:51.431066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:56.658779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430381+0000","last_change":"2024-03-06T07:22:49.445836+0000","last_active":"2024-03-06T07:22:51.430381+0000","last_peered":"2024-03-06T07:22:51.430381+0000","last_clean":"2024-03-06T07:22:51.430381+0000","last_became_active":"2024-03-06T07:22:49.445539+0000","last_became_peered":"2024-03-06T07:22:49.445539+0000","last_unstale":"2024-03-06T07:22:51.430381+0000","last_undegraded":"2024-03-06T07:22:51.430381+0000","last_fullsized":"2024-03-06T07:22:51.430381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:46.923868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811731+0000","last_change":"2024-03-06T07:22:35.370112+0000","last_active":"2024-03-06T07:22:51.811731+0000","last_peered":"2024-03-06T07:22:51.811731+0000","last_clean":"2024-03-06T07:22:51.811731+0000","last_became_active":"2024-03-06T07:22:35.369845+0000","last_became_peered":"2024-03-06T07:22:35.369845+0000","last_unstale":"2024-03-06T07:22:51.811731+0000","last_undegraded":"2024-03-06T07:22:51.811731+0000","last_fullsized":"2024-03-06T07:22:51.811731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810494+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:51.810494+0000","last_peered":"2024-03-06T07:22:51.810494+0000","last_clean":"2024-03-06T07:22:51.810494+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:51.810494+0000","last_undegraded":"2024-03-06T07:22:51.810494+0000","last_fullsized":"2024-03-06T07:22:51.810494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811486+0000","last_change":"2024-03-06T07:22:50.448243+0000","last_active":"2024-03-06T07:22:51.811486+0000","last_peered":"2024-03-06T07:22:51.811486+0000","last_clean":"2024-03-06T07:22:51.811486+0000","last_became_active":"2024-03-06T07:22:50.447419+0000","last_became_peered":"2024-03-06T07:22:50.447419+0000","last_unstale":"2024-03-06T07:22:51.811486+0000","last_undegraded":"2024-03-06T07:22:51.811486+0000","last_fullsized":"2024-03-06T07:22:51.811486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:15.509422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430711+0000","last_change":"2024-03-06T07:22:49.437483+0000","last_active":"2024-03-06T07:22:51.430711+0000","last_peered":"2024-03-06T07:22:51.430711+0000","last_clean":"2024-03-06T07:22:51.430711+0000","last_became_active":"2024-03-06T07:22:49.437192+0000","last_became_peered":"2024-03-06T07:22:49.437192+0000","last_unstale":"2024-03-06T07:22:51.430711+0000","last_undegraded":"2024-03-06T07:22:51.430711+0000","last_fullsized":"2024-03-06T07:22:51.430711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:08.738502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811711+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:51.811711+0000","last_peered":"2024-03-06T07:22:51.811711+0000","last_clean":"2024-03-06T07:22:51.811711+0000","last_became_active":"2024-03-06T07:22:35.361332+0000","last_became_peered":"2024-03-06T07:22:35.361332+0000","last_unstale":"2024-03-06T07:22:51.811711+0000","last_undegraded":"2024-03-06T07:22:51.811711+0000","last_fullsized":"2024-03-06T07:22:51.811711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811474+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:51.811474+0000","last_peered":"2024-03-06T07:22:51.811474+0000","last_clean":"2024-03-06T07:22:51.811474+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:51.811474+0000","last_undegraded":"2024-03-06T07:22:51.811474+0000","last_fullsized":"2024-03-06T07:22:51.811474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430317+0000","last_change":"2024-03-06T07:22:50.438243+0000","last_active":"2024-03-06T07:22:51.430317+0000","last_peered":"2024-03-06T07:22:51.430317+0000","last_clean":"2024-03-06T07:22:51.430317+0000","last_became_active":"2024-03-06T07:22:50.437984+0000","last_became_peered":"2024-03-06T07:22:50.437984+0000","last_unstale":"2024-03-06T07:22:51.430317+0000","last_undegraded":"2024-03-06T07:22:51.430317+0000","last_fullsized":"2024-03-06T07:22:51.430317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809873+0000","last_change":"2024-03-06T07:22:49.442138+0000","last_active":"2024-03-06T07:22:51.809873+0000","last_peered":"2024-03-06T07:22:51.809873+0000","last_clean":"2024-03-06T07:22:51.809873+0000","last_became_active":"2024-03-06T07:22:49.441875+0000","last_became_peered":"2024-03-06T07:22:49.441875+0000","last_unstale":"2024-03-06T07:22:51.809873+0000","last_undegraded":"2024-03-06T07:22:51.809873+0000","last_fullsized":"2024-03-06T07:22:51.809873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:55.653561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430878+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:51.430878+0000","last_peered":"2024-03-06T07:22:51.430878+0000","last_clean":"2024-03-06T07:22:51.430878+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:51.430878+0000","last_undegraded":"2024-03-06T07:22:51.430878+0000","last_fullsized":"2024-03-06T07:22:51.430878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:03.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810092+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:51.810092+0000","last_peered":"2024-03-06T07:22:51.810092+0000","last_clean":"2024-03-06T07:22:51.810092+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:51.810092+0000","last_undegraded":"2024-03-06T07:22:51.810092+0000","last_fullsized":"2024-03-06T07:22:51.810092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430028+0000","last_change":"2024-03-06T07:22:50.437663+0000","last_active":"2024-03-06T07:22:51.430028+0000","last_peered":"2024-03-06T07:22:51.430028+0000","last_clean":"2024-03-06T07:22:51.430028+0000","last_became_active":"2024-03-06T07:22:50.437356+0000","last_became_peered":"2024-03-06T07:22:50.437356+0000","last_unstale":"2024-03-06T07:22:51.430028+0000","last_undegraded":"2024-03-06T07:22:51.430028+0000","last_fullsized":"2024-03-06T07:22:51.430028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:45.163362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429708+0000","last_change":"2024-03-06T07:22:49.434747+0000","last_active":"2024-03-06T07:22:51.429708+0000","last_peered":"2024-03-06T07:22:51.429708+0000","last_clean":"2024-03-06T07:22:51.429708+0000","last_became_active":"2024-03-06T07:22:49.434490+0000","last_became_peered":"2024-03-06T07:22:49.434490+0000","last_unstale":"2024-03-06T07:22:51.429708+0000","last_undegraded":"2024-03-06T07:22:51.429708+0000","last_fullsized":"2024-03-06T07:22:51.429708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:55.696704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429189+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:51.429189+0000","last_peered":"2024-03-06T07:22:51.429189+0000","last_clean":"2024-03-06T07:22:51.429189+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:51.429189+0000","last_undegraded":"2024-03-06T07:22:51.429189+0000","last_fullsized":"2024-03-06T07:22:51.429189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810288+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:51.810288+0000","last_peered":"2024-03-06T07:22:51.810288+0000","last_clean":"2024-03-06T07:22:51.810288+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:51.810288+0000","last_undegraded":"2024-03-06T07:22:51.810288+0000","last_fullsized":"2024-03-06T07:22:51.810288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812604+0000","last_change":"2024-03-06T07:22:50.458608+0000","last_active":"2024-03-06T07:22:51.812604+0000","last_peered":"2024-03-06T07:22:51.812604+0000","last_clean":"2024-03-06T07:22:51.812604+0000","last_became_active":"2024-03-06T07:22:50.458335+0000","last_became_peered":"2024-03-06T07:22:50.458335+0000","last_unstale":"2024-03-06T07:22:51.812604+0000","last_undegraded":"2024-03-06T07:22:51.812604+0000","last_fullsized":"2024-03-06T07:22:51.812604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:59.630524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809454+0000","last_change":"2024-03-06T07:22:49.448683+0000","last_active":"2024-03-06T07:22:51.809454+0000","last_peered":"2024-03-06T07:22:51.809454+0000","last_clean":"2024-03-06T07:22:51.809454+0000","last_became_active":"2024-03-06T07:22:49.448519+0000","last_became_peered":"2024-03-06T07:22:49.448519+0000","last_unstale":"2024-03-06T07:22:51.809454+0000","last_undegraded":"2024-03-06T07:22:51.809454+0000","last_fullsized":"2024-03-06T07:22:51.809454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:57.349847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:51.430022+0000","last_peered":"2024-03-06T07:22:51.430022+0000","last_clean":"2024-03-06T07:22:51.430022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:51.430022+0000","last_undegraded":"2024-03-06T07:22:51.430022+0000","last_fullsized":"2024-03-06T07:22:51.430022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812020+0000","last_change":"2024-03-06T07:22:34.362718+0000","last_active":"2024-03-06T07:22:51.812020+0000","last_peered":"2024-03-06T07:22:51.812020+0000","last_clean":"2024-03-06T07:22:51.812020+0000","last_became_active":"2024-03-06T07:22:34.362448+0000","last_became_peered":"2024-03-06T07:22:34.362448+0000","last_unstale":"2024-03-06T07:22:51.812020+0000","last_undegraded":"2024-03-06T07:22:51.812020+0000","last_fullsized":"2024-03-06T07:22:51.812020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809763+0000","last_change":"2024-03-06T07:22:50.445363+0000","last_active":"2024-03-06T07:22:51.809763+0000","last_peered":"2024-03-06T07:22:51.809763+0000","last_clean":"2024-03-06T07:22:51.809763+0000","last_became_active":"2024-03-06T07:22:50.445076+0000","last_became_peered":"2024-03-06T07:22:50.445076+0000","last_unstale":"2024-03-06T07:22:51.809763+0000","last_undegraded":"2024-03-06T07:22:51.809763+0000","last_fullsized":"2024-03-06T07:22:51.809763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:01.849313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812351+0000","last_change":"2024-03-06T07:22:49.443106+0000","last_active":"2024-03-06T07:22:51.812351+0000","last_peered":"2024-03-06T07:22:51.812351+0000","last_clean":"2024-03-06T07:22:51.812351+0000","last_became_active":"2024-03-06T07:22:49.441939+0000","last_became_peered":"2024-03-06T07:22:49.441939+0000","last_unstale":"2024-03-06T07:22:51.812351+0000","last_undegraded":"2024-03-06T07:22:51.812351+0000","last_fullsized":"2024-03-06T07:22:51.812351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:07.000270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429608+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:51.429608+0000","last_peered":"2024-03-06T07:22:51.429608+0000","last_clean":"2024-03-06T07:22:51.429608+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:51.429608+0000","last_undegraded":"2024-03-06T07:22:51.429608+0000","last_fullsized":"2024-03-06T07:22:51.429608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810579+0000","last_change":"2024-03-06T07:22:34.365631+0000","last_active":"2024-03-06T07:22:51.810579+0000","last_peered":"2024-03-06T07:22:51.810579+0000","last_clean":"2024-03-06T07:22:51.810579+0000","last_became_active":"2024-03-06T07:22:34.364558+0000","last_became_peered":"2024-03-06T07:22:34.364558+0000","last_unstale":"2024-03-06T07:22:51.810579+0000","last_undegraded":"2024-03-06T07:22:51.810579+0000","last_fullsized":"2024-03-06T07:22:51.810579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:37.167691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811425+0000","last_change":"2024-03-06T07:22:50.449436+0000","last_active":"2024-03-06T07:22:51.811425+0000","last_peered":"2024-03-06T07:22:51.811425+0000","last_clean":"2024-03-06T07:22:51.811425+0000","last_became_active":"2024-03-06T07:22:50.446209+0000","last_became_peered":"2024-03-06T07:22:50.446209+0000","last_unstale":"2024-03-06T07:22:51.811425+0000","last_undegraded":"2024-03-06T07:22:51.811425+0000","last_fullsized":"2024-03-06T07:22:51.811425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:04.259512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809638+0000","last_change":"2024-03-06T07:22:49.440025+0000","last_active":"2024-03-06T07:22:51.809638+0000","last_peered":"2024-03-06T07:22:51.809638+0000","last_clean":"2024-03-06T07:22:51.809638+0000","last_became_active":"2024-03-06T07:22:49.439779+0000","last_became_peered":"2024-03-06T07:22:49.439779+0000","last_unstale":"2024-03-06T07:22:51.809638+0000","last_undegraded":"2024-03-06T07:22:51.809638+0000","last_fullsized":"2024-03-06T07:22:51.809638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:20.277188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812653+0000","last_change":"2024-03-06T07:22:35.362565+0000","last_active":"2024-03-06T07:22:51.812653+0000","last_peered":"2024-03-06T07:22:51.812653+0000","last_clean":"2024-03-06T07:22:51.812653+0000","last_became_active":"2024-03-06T07:22:35.362296+0000","last_became_peered":"2024-03-06T07:22:35.362296+0000","last_unstale":"2024-03-06T07:22:51.812653+0000","last_undegraded":"2024-03-06T07:22:51.812653+0000","last_fullsized":"2024-03-06T07:22:51.812653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:19.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810605+0000","last_change":"2024-03-06T07:22:34.360939+0000","last_active":"2024-03-06T07:22:51.810605+0000","last_peered":"2024-03-06T07:22:51.810605+0000","last_clean":"2024-03-06T07:22:51.810605+0000","last_became_active":"2024-03-06T07:22:34.360665+0000","last_became_peered":"2024-03-06T07:22:34.360665+0000","last_unstale":"2024-03-06T07:22:51.810605+0000","last_undegraded":"2024-03-06T07:22:51.810605+0000","last_fullsized":"2024-03-06T07:22:51.810605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811596+0000","last_change":"2024-03-06T07:22:50.447742+0000","last_active":"2024-03-06T07:22:51.811596+0000","last_peered":"2024-03-06T07:22:51.811596+0000","last_clean":"2024-03-06T07:22:51.811596+0000","last_became_active":"2024-03-06T07:22:50.447473+0000","last_became_peered":"2024-03-06T07:22:50.447473+0000","last_unstale":"2024-03-06T07:22:51.811596+0000","last_undegraded":"2024-03-06T07:22:51.811596+0000","last_fullsized":"2024-03-06T07:22:51.811596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:13.315238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430466+0000","last_change":"2024-03-06T07:22:49.444034+0000","last_active":"2024-03-06T07:22:51.430466+0000","last_peered":"2024-03-06T07:22:51.430466+0000","last_clean":"2024-03-06T07:22:51.430466+0000","last_became_active":"2024-03-06T07:22:49.443793+0000","last_became_peered":"2024-03-06T07:22:49.443793+0000","last_unstale":"2024-03-06T07:22:51.430466+0000","last_undegraded":"2024-03-06T07:22:51.430466+0000","last_fullsized":"2024-03-06T07:22:51.430466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430985+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:51.430985+0000","last_peered":"2024-03-06T07:22:51.430985+0000","last_clean":"2024-03-06T07:22:51.430985+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:51.430985+0000","last_undegraded":"2024-03-06T07:22:51.430985+0000","last_fullsized":"2024-03-06T07:22:51.430985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:44.303007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430538+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:51.430538+0000","last_peered":"2024-03-06T07:22:51.430538+0000","last_clean":"2024-03-06T07:22:51.430538+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:51.430538+0000","last_undegraded":"2024-03-06T07:22:51.430538+0000","last_fullsized":"2024-03-06T07:22:51.430538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810456+0000","last_change":"2024-03-06T07:22:50.445006+0000","last_active":"2024-03-06T07:22:51.810456+0000","last_peered":"2024-03-06T07:22:51.810456+0000","last_clean":"2024-03-06T07:22:51.810456+0000","last_became_active":"2024-03-06T07:22:50.444747+0000","last_became_peered":"2024-03-06T07:22:50.444747+0000","last_unstale":"2024-03-06T07:22:51.810456+0000","last_undegraded":"2024-03-06T07:22:51.810456+0000","last_fullsized":"2024-03-06T07:22:51.810456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:43.376118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430914+0000","last_change":"2024-03-06T07:22:49.436674+0000","last_active":"2024-03-06T07:22:51.430914+0000","last_peered":"2024-03-06T07:22:51.430914+0000","last_clean":"2024-03-06T07:22:51.430914+0000","last_became_active":"2024-03-06T07:22:49.436417+0000","last_became_peered":"2024-03-06T07:22:49.436417+0000","last_unstale":"2024-03-06T07:22:51.430914+0000","last_undegraded":"2024-03-06T07:22:51.430914+0000","last_fullsized":"2024-03-06T07:22:51.430914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:56.306976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430437+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:51.430437+0000","last_peered":"2024-03-06T07:22:51.430437+0000","last_clean":"2024-03-06T07:22:51.430437+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:51.430437+0000","last_undegraded":"2024-03-06T07:22:51.430437+0000","last_fullsized":"2024-03-06T07:22:51.430437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:31.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810552+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:51.810552+0000","last_peered":"2024-03-06T07:22:51.810552+0000","last_clean":"2024-03-06T07:22:51.810552+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:51.810552+0000","last_undegraded":"2024-03-06T07:22:51.810552+0000","last_fullsized":"2024-03-06T07:22:51.810552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431106+0000","last_change":"2024-03-06T07:22:50.441776+0000","last_active":"2024-03-06T07:22:51.431106+0000","last_peered":"2024-03-06T07:22:51.431106+0000","last_clean":"2024-03-06T07:22:51.431106+0000","last_became_active":"2024-03-06T07:22:50.441530+0000","last_became_peered":"2024-03-06T07:22:50.441530+0000","last_unstale":"2024-03-06T07:22:51.431106+0000","last_undegraded":"2024-03-06T07:22:51.431106+0000","last_fullsized":"2024-03-06T07:22:51.431106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809856+0000","last_change":"2024-03-06T07:22:49.442594+0000","last_active":"2024-03-06T07:22:51.809856+0000","last_peered":"2024-03-06T07:22:51.809856+0000","last_clean":"2024-03-06T07:22:51.809856+0000","last_became_active":"2024-03-06T07:22:49.442310+0000","last_became_peered":"2024-03-06T07:22:49.442310+0000","last_unstale":"2024-03-06T07:22:51.809856+0000","last_undegraded":"2024-03-06T07:22:51.809856+0000","last_fullsized":"2024-03-06T07:22:51.809856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:03.166354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427812+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:51.427812+0000","last_peered":"2024-03-06T07:22:51.427812+0000","last_clean":"2024-03-06T07:22:51.427812+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:51.427812+0000","last_undegraded":"2024-03-06T07:22:51.427812+0000","last_fullsized":"2024-03-06T07:22:51.427812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:31.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430131+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:51.430131+0000","last_peered":"2024-03-06T07:22:51.430131+0000","last_clean":"2024-03-06T07:22:51.430131+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:51.430131+0000","last_undegraded":"2024-03-06T07:22:51.430131+0000","last_fullsized":"2024-03-06T07:22:51.430131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812206+0000","last_change":"2024-03-06T07:22:50.451967+0000","last_active":"2024-03-06T07:22:51.812206+0000","last_peered":"2024-03-06T07:22:51.812206+0000","last_clean":"2024-03-06T07:22:51.812206+0000","last_became_active":"2024-03-06T07:22:50.451701+0000","last_became_peered":"2024-03-06T07:22:50.451701+0000","last_unstale":"2024-03-06T07:22:51.812206+0000","last_undegraded":"2024-03-06T07:22:51.812206+0000","last_fullsized":"2024-03-06T07:22:51.812206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:59.388088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430679+0000","last_change":"2024-03-06T07:22:49.439060+0000","last_active":"2024-03-06T07:22:51.430679+0000","last_peered":"2024-03-06T07:22:51.430679+0000","last_clean":"2024-03-06T07:22:51.430679+0000","last_became_active":"2024-03-06T07:22:49.438806+0000","last_became_peered":"2024-03-06T07:22:49.438806+0000","last_unstale":"2024-03-06T07:22:51.430679+0000","last_undegraded":"2024-03-06T07:22:51.430679+0000","last_fullsized":"2024-03-06T07:22:51.430679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:28.533777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427851+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:51.427851+0000","last_peered":"2024-03-06T07:22:51.427851+0000","last_clean":"2024-03-06T07:22:51.427851+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:51.427851+0000","last_undegraded":"2024-03-06T07:22:51.427851+0000","last_fullsized":"2024-03-06T07:22:51.427851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:54.954958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426813+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:51.426813+0000","last_peered":"2024-03-06T07:22:51.426813+0000","last_clean":"2024-03-06T07:22:51.426813+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:51.426813+0000","last_undegraded":"2024-03-06T07:22:51.426813+0000","last_fullsized":"2024-03-06T07:22:51.426813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809784+0000","last_change":"2024-03-06T07:22:50.455677+0000","last_active":"2024-03-06T07:22:51.809784+0000","last_peered":"2024-03-06T07:22:51.809784+0000","last_clean":"2024-03-06T07:22:51.809784+0000","last_became_active":"2024-03-06T07:22:50.455436+0000","last_became_peered":"2024-03-06T07:22:50.455436+0000","last_unstale":"2024-03-06T07:22:51.809784+0000","last_undegraded":"2024-03-06T07:22:51.809784+0000","last_fullsized":"2024-03-06T07:22:51.809784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:08.478793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427749+0000","last_change":"2024-03-06T07:22:49.441951+0000","last_active":"2024-03-06T07:22:51.427749+0000","last_peered":"2024-03-06T07:22:51.427749+0000","last_clean":"2024-03-06T07:22:51.427749+0000","last_became_active":"2024-03-06T07:22:49.441543+0000","last_became_peered":"2024-03-06T07:22:49.441543+0000","last_unstale":"2024-03-06T07:22:51.427749+0000","last_undegraded":"2024-03-06T07:22:51.427749+0000","last_fullsized":"2024-03-06T07:22:51.427749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:55.696704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810556+0000","last_change":"2024-03-06T07:22:35.373140+0000","last_active":"2024-03-06T07:22:51.810556+0000","last_peered":"2024-03-06T07:22:51.810556+0000","last_clean":"2024-03-06T07:22:51.810556+0000","last_became_active":"2024-03-06T07:22:35.372523+0000","last_became_peered":"2024-03-06T07:22:35.372523+0000","last_unstale":"2024-03-06T07:22:51.810556+0000","last_undegraded":"2024-03-06T07:22:51.810556+0000","last_fullsized":"2024-03-06T07:22:51.810556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811398+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:51.811398+0000","last_peered":"2024-03-06T07:22:51.811398+0000","last_clean":"2024-03-06T07:22:51.811398+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:51.811398+0000","last_undegraded":"2024-03-06T07:22:51.811398+0000","last_fullsized":"2024-03-06T07:22:51.811398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":280,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809782+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:51.809782+0000","last_peered":"2024-03-06T07:22:51.809782+0000","last_clean":"2024-03-06T07:22:51.809782+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:51.809782+0000","last_undegraded":"2024-03-06T07:22:51.809782+0000","last_fullsized":"2024-03-06T07:22:51.809782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":1056835,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":470217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":458874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":468265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":466971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":467131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":466571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":1051526,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:53.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:22:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.593+0000 7f22fc93e640 1 Processor -- start 2024-03-06T07:22:53.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.593+0000 7f22fc93e640 1 -- start start 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22fc93e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22f8152920 0x7f22f8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22fc93e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f22f80c5330 0x7f22f80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22fc93e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f8156180 con 0x7f22f80c8a00 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22fc93e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f81587b0 con 0x7f22f8152920 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22fc93e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f8159630 con 0x7f22f80c5330 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22f8152920 0x7f22f8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22f8152920 0x7f22f8155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38192/0 (socket says 172.21.15.17:38192) 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 -- 172.21.15.17:0/1038150168 learned_addr learned my addr 172.21.15.17:0/1038150168 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f6d9d640 1 --2- 172.21.15.17:0/1038150168 >> v2:172.21.15.163:3300/0 conn(0x7f22f80c5330 0x7f22f80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 -- 172.21.15.17:0/1038150168 >> v2:172.21.15.163:3300/0 conn(0x7f22f80c5330 msgr2=0x7f22f80c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:53.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 --2- 172.21.15.17:0/1038150168 >> v2:172.21.15.163:3300/0 conn(0x7f22f80c5330 0x7f22f80c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 -- 172.21.15.17:0/1038150168 >> v1:172.21.15.163:6789/0 conn(0x7f22f80c8a00 legacy=0x7f22f8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.594+0000 7f22f659c640 1 -- 172.21.15.17:0/1038150168 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f80c7760 con 0x7f22f8152920 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.595+0000 7f22f659c640 1 --2- 172.21.15.17:0/1038150168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8152920 0x7f22f8155c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f22f80cba60 tx=0x7f22e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412ff4b7e217ed01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.595+0000 7f22f559a640 1 -- 172.21.15.17:0/1038150168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8035020 con 0x7f22f8152920 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.595+0000 7f22f559a640 1 -- 172.21.15.17:0/1038150168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e8031870 con 0x7f22f8152920 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.595+0000 7f22f559a640 1 -- 172.21.15.17:0/1038150168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8031c40 con 0x7f22f8152920 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 -- 172.21.15.17:0/1038150168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8152920 msgr2=0x7f22f8155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 --2- 172.21.15.17:0/1038150168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8152920 0x7f22f8155c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f22f80cba60 tx=0x7f22e802b910 comp rx=0 tx=0).stop 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 -- 172.21.15.17:0/1038150168 shutdown_connections 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 --2- 172.21.15.17:0/1038150168 >> v2:172.21.15.163:3300/0 conn(0x7f22f80c5330 0x7f22f80c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 --2- 172.21.15.17:0/1038150168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f8152920 0x7f22f8155c40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 -- 172.21.15.17:0/1038150168 >> 172.21.15.17:0/1038150168 conn(0x7f22f80ac560 msgr2=0x7f22f80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 -- 172.21.15.17:0/1038150168 shutdown_connections 2024-03-06T07:22:53.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.596+0000 7f22fc93e640 1 -- 172.21.15.17:0/1038150168 wait complete. 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.597+0000 7f22fc93e640 1 Processor -- start 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.597+0000 7f22fc93e640 1 -- start start 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 0x7f22f81499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22f8152920 0x7f22f81eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8158a00 con 0x7f22f80c8a00 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f80c7d20 con 0x7f22f80c5330 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22fc93e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f806d030 con 0x7f22f8152920 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 0x7f22f81499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38208/0 (socket says 172.21.15.17:38208) 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 -- 172.21.15.17:0/908090417 learned_addr learned my addr 172.21.15.17:0/908090417 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f6d9d640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22f8152920 0x7f22f81eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 -- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22f8152920 msgr2=0x7f22f81eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22f8152920 0x7f22f81eeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 -- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 msgr2=0x7f22f81499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 0x7f22f81499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 -- 172.21.15.17:0/908090417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8144e20 con 0x7f22f80c8a00 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f659c640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 0x7f22f81499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.598+0000 7f22f5d9b640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00c3f0 tx=0x7f22ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.599+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0052d0 con 0x7f22f80c8a00 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.599+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec005490 con 0x7f22f80c8a00 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.599+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f814e6d0 con 0x7f22f80c8a00 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.599+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec007260 con 0x7f22f80c8a00 2024-03-06T07:22:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.599+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f8149ee0 con 0x7f22f80c8a00 2024-03-06T07:22:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.600+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f804f330 con 0x7f22f80c8a00 2024-03-06T07:22:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.600+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22ec007400 con 0x7f22f80c8a00 2024-03-06T07:22:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.601+0000 7f22e6ffd640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.601+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f22ec089ae0 con 0x7f22f80c8a00 2024-03-06T07:22:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.601+0000 7f22f659c640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.601+0000 7f22f659c640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 0x7f22d806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f22e8036000 tx=0x7f22e803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.609+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f22f8156180 con 0x7f22f80c8a00 2024-03-06T07:22:53.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.784+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22f8144bd0 con 0x7f22d806b4d0 2024-03-06T07:22:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.796+0000 7f22e6ffd640 1 -- 172.21.15.17:0/908090417 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+490718 (secure 0 0 0) 0x7f22e80bfcf0 con 0x7f22d806b4d0 2024-03-06T07:22:53.803 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:53.807 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 msgr2=0x7f22d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 0x7f22d806d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f22e8036000 tx=0x7f22e803c040 comp rx=0 tx=0).stop 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 msgr2=0x7f22f81ee5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00c3f0 tx=0x7f22ec00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 shutdown_connections 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f22d806b4d0 0x7f22d806d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22f8152920 0x7f22f81eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22f80c8a00 0x7f22f81ee5c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 --2- 172.21.15.17:0/908090417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22f80c5330 0x7f22f81499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 >> 172.21.15.17:0/908090417 conn(0x7f22f80ac560 msgr2=0x7f22f8150de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 shutdown_connections 2024-03-06T07:22:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.801+0000 7f22fc93e640 1 -- 172.21.15.17:0/908090417 wait complete. 2024-03-06T07:22:53.813 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-03-06T07:22:53.378277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":248,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":194,"ondisk_log_size":194,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":371,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244780,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729940,"statfs":{"total":773094113280,"available":772843458560,"internally_reserved":0,"allocated":7970816,"data_stored":4906390,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":242535988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.960606"},"pg_stats":[{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810944+0000","last_change":"2024-03-06T07:22:50.450570+0000","last_active":"2024-03-06T07:22:51.810944+0000","last_peered":"2024-03-06T07:22:51.810944+0000","last_clean":"2024-03-06T07:22:51.810944+0000","last_became_active":"2024-03-06T07:22:50.450305+0000","last_became_peered":"2024-03-06T07:22:50.450305+0000","last_unstale":"2024-03-06T07:22:51.810944+0000","last_undegraded":"2024-03-06T07:22:51.810944+0000","last_fullsized":"2024-03-06T07:22:51.810944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:23.305044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809908+0000","last_change":"2024-03-06T07:22:35.363195+0000","last_active":"2024-03-06T07:22:51.809908+0000","last_peered":"2024-03-06T07:22:51.809908+0000","last_clean":"2024-03-06T07:22:51.809908+0000","last_became_active":"2024-03-06T07:22:35.362811+0000","last_became_peered":"2024-03-06T07:22:35.362811+0000","last_unstale":"2024-03-06T07:22:51.809908+0000","last_undegraded":"2024-03-06T07:22:51.809908+0000","last_fullsized":"2024-03-06T07:22:51.809908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427452+0000","last_change":"2024-03-06T07:22:50.445149+0000","last_active":"2024-03-06T07:22:51.427452+0000","last_peered":"2024-03-06T07:22:51.427452+0000","last_clean":"2024-03-06T07:22:51.427452+0000","last_became_active":"2024-03-06T07:22:50.444878+0000","last_became_peered":"2024-03-06T07:22:50.444878+0000","last_unstale":"2024-03-06T07:22:51.427452+0000","last_undegraded":"2024-03-06T07:22:51.427452+0000","last_fullsized":"2024-03-06T07:22:51.427452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:23.041319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811647+0000","last_change":"2024-03-06T07:22:35.363283+0000","last_active":"2024-03-06T07:22:51.811647+0000","last_peered":"2024-03-06T07:22:51.811647+0000","last_clean":"2024-03-06T07:22:51.811647+0000","last_became_active":"2024-03-06T07:22:35.363028+0000","last_became_peered":"2024-03-06T07:22:35.363028+0000","last_unstale":"2024-03-06T07:22:51.811647+0000","last_undegraded":"2024-03-06T07:22:51.811647+0000","last_fullsized":"2024-03-06T07:22:51.811647+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810613+0000","last_change":"2024-03-06T07:22:50.460440+0000","last_active":"2024-03-06T07:22:51.810613+0000","last_peered":"2024-03-06T07:22:51.810613+0000","last_clean":"2024-03-06T07:22:51.810613+0000","last_became_active":"2024-03-06T07:22:50.460172+0000","last_became_peered":"2024-03-06T07:22:50.460172+0000","last_unstale":"2024-03-06T07:22:51.810613+0000","last_undegraded":"2024-03-06T07:22:51.810613+0000","last_fullsized":"2024-03-06T07:22:51.810613+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:49.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427534+0000","last_change":"2024-03-06T07:22:35.366490+0000","last_active":"2024-03-06T07:22:51.427534+0000","last_peered":"2024-03-06T07:22:51.427534+0000","last_clean":"2024-03-06T07:22:51.427534+0000","last_became_active":"2024-03-06T07:22:35.366162+0000","last_became_peered":"2024-03-06T07:22:35.366162+0000","last_unstale":"2024-03-06T07:22:51.427534+0000","last_undegraded":"2024-03-06T07:22:51.427534+0000","last_fullsized":"2024-03-06T07:22:51.427534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:44.303007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810103+0000","last_change":"2024-03-06T07:22:50.454528+0000","last_active":"2024-03-06T07:22:51.810103+0000","last_peered":"2024-03-06T07:22:51.810103+0000","last_clean":"2024-03-06T07:22:51.810103+0000","last_became_active":"2024-03-06T07:22:50.454270+0000","last_became_peered":"2024-03-06T07:22:50.454270+0000","last_unstale":"2024-03-06T07:22:51.810103+0000","last_undegraded":"2024-03-06T07:22:51.810103+0000","last_fullsized":"2024-03-06T07:22:51.810103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:04.171573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810188+0000","last_change":"2024-03-06T07:22:35.371068+0000","last_active":"2024-03-06T07:22:51.810188+0000","last_peered":"2024-03-06T07:22:51.810188+0000","last_clean":"2024-03-06T07:22:51.810188+0000","last_became_active":"2024-03-06T07:22:35.370800+0000","last_became_peered":"2024-03-06T07:22:35.370800+0000","last_unstale":"2024-03-06T07:22:51.810188+0000","last_undegraded":"2024-03-06T07:22:51.810188+0000","last_fullsized":"2024-03-06T07:22:51.810188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:03:53.250614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809709+0000","last_change":"2024-03-06T07:22:50.445632+0000","last_active":"2024-03-06T07:22:51.809709+0000","last_peered":"2024-03-06T07:22:51.809709+0000","last_clean":"2024-03-06T07:22:51.809709+0000","last_became_active":"2024-03-06T07:22:50.445397+0000","last_became_peered":"2024-03-06T07:22:50.445397+0000","last_unstale":"2024-03-06T07:22:51.809709+0000","last_undegraded":"2024-03-06T07:22:51.809709+0000","last_fullsized":"2024-03-06T07:22:51.809709+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:05:22.123259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810185+0000","last_change":"2024-03-06T07:22:35.364936+0000","last_active":"2024-03-06T07:22:51.810185+0000","last_peered":"2024-03-06T07:22:51.810185+0000","last_clean":"2024-03-06T07:22:51.810185+0000","last_became_active":"2024-03-06T07:22:35.364603+0000","last_became_peered":"2024-03-06T07:22:35.364603+0000","last_unstale":"2024-03-06T07:22:51.810185+0000","last_undegraded":"2024-03-06T07:22:51.810185+0000","last_fullsized":"2024-03-06T07:22:51.810185+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:22:20.685225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426627+0000","last_change":"2024-03-06T07:22:50.452816+0000","last_active":"2024-03-06T07:22:51.426627+0000","last_peered":"2024-03-06T07:22:51.426627+0000","last_clean":"2024-03-06T07:22:51.426627+0000","last_became_active":"2024-03-06T07:22:50.452150+0000","last_became_peered":"2024-03-06T07:22:50.452150+0000","last_unstale":"2024-03-06T07:22:51.426627+0000","last_undegraded":"2024-03-06T07:22:51.426627+0000","last_fullsized":"2024-03-06T07:22:51.426627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810914+0000","last_change":"2024-03-06T07:22:35.362295+0000","last_active":"2024-03-06T07:22:51.810914+0000","last_peered":"2024-03-06T07:22:51.810914+0000","last_clean":"2024-03-06T07:22:51.810914+0000","last_became_active":"2024-03-06T07:22:35.361694+0000","last_became_peered":"2024-03-06T07:22:35.361694+0000","last_unstale":"2024-03-06T07:22:51.810914+0000","last_undegraded":"2024-03-06T07:22:51.810914+0000","last_fullsized":"2024-03-06T07:22:51.810914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429183+0000","last_change":"2024-03-06T07:22:50.434924+0000","last_active":"2024-03-06T07:22:51.429183+0000","last_peered":"2024-03-06T07:22:51.429183+0000","last_clean":"2024-03-06T07:22:51.429183+0000","last_became_active":"2024-03-06T07:22:50.434584+0000","last_became_peered":"2024-03-06T07:22:50.434584+0000","last_unstale":"2024-03-06T07:22:51.429183+0000","last_undegraded":"2024-03-06T07:22:51.429183+0000","last_fullsized":"2024-03-06T07:22:51.429183+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429929+0000","last_change":"2024-03-06T07:22:35.370645+0000","last_active":"2024-03-06T07:22:51.429929+0000","last_peered":"2024-03-06T07:22:51.429929+0000","last_clean":"2024-03-06T07:22:51.429929+0000","last_became_active":"2024-03-06T07:22:35.370398+0000","last_became_peered":"2024-03-06T07:22:35.370398+0000","last_unstale":"2024-03-06T07:22:51.429929+0000","last_undegraded":"2024-03-06T07:22:51.429929+0000","last_fullsized":"2024-03-06T07:22:51.429929+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811294+0000","last_change":"2024-03-06T07:22:50.449942+0000","last_active":"2024-03-06T07:22:51.811294+0000","last_peered":"2024-03-06T07:22:51.811294+0000","last_clean":"2024-03-06T07:22:51.811294+0000","last_became_active":"2024-03-06T07:22:50.449445+0000","last_became_peered":"2024-03-06T07:22:50.449445+0000","last_unstale":"2024-03-06T07:22:51.811294+0000","last_undegraded":"2024-03-06T07:22:51.811294+0000","last_fullsized":"2024-03-06T07:22:51.811294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:32:35.875822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809884+0000","last_change":"2024-03-06T07:22:35.361224+0000","last_active":"2024-03-06T07:22:51.809884+0000","last_peered":"2024-03-06T07:22:51.809884+0000","last_clean":"2024-03-06T07:22:51.809884+0000","last_became_active":"2024-03-06T07:22:35.360941+0000","last_became_peered":"2024-03-06T07:22:35.360941+0000","last_unstale":"2024-03-06T07:22:51.809884+0000","last_undegraded":"2024-03-06T07:22:51.809884+0000","last_fullsized":"2024-03-06T07:22:51.809884+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:00:06.789508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429999+0000","last_change":"2024-03-06T07:22:50.453986+0000","last_active":"2024-03-06T07:22:51.429999+0000","last_peered":"2024-03-06T07:22:51.429999+0000","last_clean":"2024-03-06T07:22:51.429999+0000","last_became_active":"2024-03-06T07:22:50.453604+0000","last_became_peered":"2024-03-06T07:22:50.453604+0000","last_unstale":"2024-03-06T07:22:51.429999+0000","last_undegraded":"2024-03-06T07:22:51.429999+0000","last_fullsized":"2024-03-06T07:22:51.429999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:32.345117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430561+0000","last_change":"2024-03-06T07:22:35.365472+0000","last_active":"2024-03-06T07:22:51.430561+0000","last_peered":"2024-03-06T07:22:51.430561+0000","last_clean":"2024-03-06T07:22:51.430561+0000","last_became_active":"2024-03-06T07:22:35.365219+0000","last_became_peered":"2024-03-06T07:22:35.365219+0000","last_unstale":"2024-03-06T07:22:51.430561+0000","last_undegraded":"2024-03-06T07:22:51.430561+0000","last_fullsized":"2024-03-06T07:22:51.430561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810235+0000","last_change":"2024-03-06T07:22:50.445186+0000","last_active":"2024-03-06T07:22:51.810235+0000","last_peered":"2024-03-06T07:22:51.810235+0000","last_clean":"2024-03-06T07:22:51.810235+0000","last_became_active":"2024-03-06T07:22:50.444914+0000","last_became_peered":"2024-03-06T07:22:50.444914+0000","last_unstale":"2024-03-06T07:22:51.810235+0000","last_undegraded":"2024-03-06T07:22:51.810235+0000","last_fullsized":"2024-03-06T07:22:51.810235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:10:50.902194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810228+0000","last_change":"2024-03-06T07:22:35.372536+0000","last_active":"2024-03-06T07:22:51.810228+0000","last_peered":"2024-03-06T07:22:51.810228+0000","last_clean":"2024-03-06T07:22:51.810228+0000","last_became_active":"2024-03-06T07:22:35.371963+0000","last_became_peered":"2024-03-06T07:22:35.371963+0000","last_unstale":"2024-03-06T07:22:51.810228+0000","last_undegraded":"2024-03-06T07:22:51.810228+0000","last_fullsized":"2024-03-06T07:22:51.810228+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:42.233047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430570+0000","last_change":"2024-03-06T07:22:50.438872+0000","last_active":"2024-03-06T07:22:51.430570+0000","last_peered":"2024-03-06T07:22:51.430570+0000","last_clean":"2024-03-06T07:22:51.430570+0000","last_became_active":"2024-03-06T07:22:50.438316+0000","last_became_peered":"2024-03-06T07:22:50.438316+0000","last_unstale":"2024-03-06T07:22:51.430570+0000","last_undegraded":"2024-03-06T07:22:51.430570+0000","last_fullsized":"2024-03-06T07:22:51.430570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812251+0000","last_change":"2024-03-06T07:22:35.365945+0000","last_active":"2024-03-06T07:22:51.812251+0000","last_peered":"2024-03-06T07:22:51.812251+0000","last_clean":"2024-03-06T07:22:51.812251+0000","last_became_active":"2024-03-06T07:22:35.365677+0000","last_became_peered":"2024-03-06T07:22:35.365677+0000","last_unstale":"2024-03-06T07:22:51.812251+0000","last_undegraded":"2024-03-06T07:22:51.812251+0000","last_fullsized":"2024-03-06T07:22:51.812251+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:16:56.649671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810382+0000","last_change":"2024-03-06T07:22:50.432526+0000","last_active":"2024-03-06T07:22:51.810382+0000","last_peered":"2024-03-06T07:22:51.810382+0000","last_clean":"2024-03-06T07:22:51.810382+0000","last_became_active":"2024-03-06T07:22:50.432234+0000","last_became_peered":"2024-03-06T07:22:50.432234+0000","last_unstale":"2024-03-06T07:22:51.810382+0000","last_undegraded":"2024-03-06T07:22:51.810382+0000","last_fullsized":"2024-03-06T07:22:51.810382+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:22.325667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810328+0000","last_change":"2024-03-06T07:22:35.371284+0000","last_active":"2024-03-06T07:22:51.810328+0000","last_peered":"2024-03-06T07:22:51.810328+0000","last_clean":"2024-03-06T07:22:51.810328+0000","last_became_active":"2024-03-06T07:22:35.369717+0000","last_became_peered":"2024-03-06T07:22:35.369717+0000","last_unstale":"2024-03-06T07:22:51.810328+0000","last_undegraded":"2024-03-06T07:22:51.810328+0000","last_fullsized":"2024-03-06T07:22:51.810328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:47.235841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426749+0000","last_change":"2024-03-06T07:22:50.446058+0000","last_active":"2024-03-06T07:22:51.426749+0000","last_peered":"2024-03-06T07:22:51.426749+0000","last_clean":"2024-03-06T07:22:51.426749+0000","last_became_active":"2024-03-06T07:22:50.445777+0000","last_became_peered":"2024-03-06T07:22:50.445777+0000","last_unstale":"2024-03-06T07:22:51.426749+0000","last_undegraded":"2024-03-06T07:22:51.426749+0000","last_fullsized":"2024-03-06T07:22:51.426749+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:13.076972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810400+0000","last_change":"2024-03-06T07:22:35.368750+0000","last_active":"2024-03-06T07:22:51.810400+0000","last_peered":"2024-03-06T07:22:51.810400+0000","last_clean":"2024-03-06T07:22:51.810400+0000","last_became_active":"2024-03-06T07:22:35.368490+0000","last_became_peered":"2024-03-06T07:22:35.368490+0000","last_unstale":"2024-03-06T07:22:51.810400+0000","last_undegraded":"2024-03-06T07:22:51.810400+0000","last_fullsized":"2024-03-06T07:22:51.810400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:21.688549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811648+0000","last_change":"2024-03-06T07:22:50.459245+0000","last_active":"2024-03-06T07:22:51.811648+0000","last_peered":"2024-03-06T07:22:51.811648+0000","last_clean":"2024-03-06T07:22:51.811648+0000","last_became_active":"2024-03-06T07:22:50.458595+0000","last_became_peered":"2024-03-06T07:22:50.458595+0000","last_unstale":"2024-03-06T07:22:51.811648+0000","last_undegraded":"2024-03-06T07:22:51.811648+0000","last_fullsized":"2024-03-06T07:22:51.811648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:24.446700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811000+0000","last_change":"2024-03-06T07:22:35.360858+0000","last_active":"2024-03-06T07:22:51.811000+0000","last_peered":"2024-03-06T07:22:51.811000+0000","last_clean":"2024-03-06T07:22:51.811000+0000","last_became_active":"2024-03-06T07:22:35.357471+0000","last_became_peered":"2024-03-06T07:22:35.357471+0000","last_unstale":"2024-03-06T07:22:51.811000+0000","last_undegraded":"2024-03-06T07:22:51.811000+0000","last_fullsized":"2024-03-06T07:22:51.811000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426982+0000","last_change":"2024-03-06T07:22:50.451467+0000","last_active":"2024-03-06T07:22:51.426982+0000","last_peered":"2024-03-06T07:22:51.426982+0000","last_clean":"2024-03-06T07:22:51.426982+0000","last_became_active":"2024-03-06T07:22:50.451203+0000","last_became_peered":"2024-03-06T07:22:50.451203+0000","last_unstale":"2024-03-06T07:22:51.426982+0000","last_undegraded":"2024-03-06T07:22:51.426982+0000","last_fullsized":"2024-03-06T07:22:51.426982+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:11.728819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811233+0000","last_change":"2024-03-06T07:22:35.366910+0000","last_active":"2024-03-06T07:22:51.811233+0000","last_peered":"2024-03-06T07:22:51.811233+0000","last_clean":"2024-03-06T07:22:51.811233+0000","last_became_active":"2024-03-06T07:22:35.366623+0000","last_became_peered":"2024-03-06T07:22:35.366623+0000","last_unstale":"2024-03-06T07:22:51.811233+0000","last_undegraded":"2024-03-06T07:22:51.811233+0000","last_fullsized":"2024-03-06T07:22:51.811233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429511+0000","last_change":"2024-03-06T07:22:50.439065+0000","last_active":"2024-03-06T07:22:51.429511+0000","last_peered":"2024-03-06T07:22:51.429511+0000","last_clean":"2024-03-06T07:22:51.429511+0000","last_became_active":"2024-03-06T07:22:50.438051+0000","last_became_peered":"2024-03-06T07:22:50.438051+0000","last_unstale":"2024-03-06T07:22:51.429511+0000","last_undegraded":"2024-03-06T07:22:51.429511+0000","last_fullsized":"2024-03-06T07:22:51.429511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:23.041319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809597+0000","last_change":"2024-03-06T07:22:35.360892+0000","last_active":"2024-03-06T07:22:51.809597+0000","last_peered":"2024-03-06T07:22:51.809597+0000","last_clean":"2024-03-06T07:22:51.809597+0000","last_became_active":"2024-03-06T07:22:35.360615+0000","last_became_peered":"2024-03-06T07:22:35.360615+0000","last_unstale":"2024-03-06T07:22:51.809597+0000","last_undegraded":"2024-03-06T07:22:51.809597+0000","last_fullsized":"2024-03-06T07:22:51.809597+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:15.740118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430126+0000","last_change":"2024-03-06T07:22:50.440433+0000","last_active":"2024-03-06T07:22:51.430126+0000","last_peered":"2024-03-06T07:22:51.430126+0000","last_clean":"2024-03-06T07:22:51.430126+0000","last_became_active":"2024-03-06T07:22:50.440166+0000","last_became_peered":"2024-03-06T07:22:50.440166+0000","last_unstale":"2024-03-06T07:22:51.430126+0000","last_undegraded":"2024-03-06T07:22:51.430126+0000","last_fullsized":"2024-03-06T07:22:51.430126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:58.355065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812530+0000","last_change":"2024-03-06T07:22:35.376779+0000","last_active":"2024-03-06T07:22:51.812530+0000","last_peered":"2024-03-06T07:22:51.812530+0000","last_clean":"2024-03-06T07:22:51.812530+0000","last_became_active":"2024-03-06T07:22:35.375254+0000","last_became_peered":"2024-03-06T07:22:35.375254+0000","last_unstale":"2024-03-06T07:22:51.812530+0000","last_undegraded":"2024-03-06T07:22:51.812530+0000","last_fullsized":"2024-03-06T07:22:51.812530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:15:34.683877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427626+0000","last_change":"2024-03-06T07:22:50.455367+0000","last_active":"2024-03-06T07:22:51.427626+0000","last_peered":"2024-03-06T07:22:51.427626+0000","last_clean":"2024-03-06T07:22:51.427626+0000","last_became_active":"2024-03-06T07:22:50.455105+0000","last_became_peered":"2024-03-06T07:22:50.455105+0000","last_unstale":"2024-03-06T07:22:51.427626+0000","last_undegraded":"2024-03-06T07:22:51.427626+0000","last_fullsized":"2024-03-06T07:22:51.427626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:32.345117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427784+0000","last_change":"2024-03-06T07:22:35.363388+0000","last_active":"2024-03-06T07:22:51.427784+0000","last_peered":"2024-03-06T07:22:51.427784+0000","last_clean":"2024-03-06T07:22:51.427784+0000","last_became_active":"2024-03-06T07:22:35.363097+0000","last_became_peered":"2024-03-06T07:22:35.363097+0000","last_unstale":"2024-03-06T07:22:51.427784+0000","last_undegraded":"2024-03-06T07:22:51.427784+0000","last_fullsized":"2024-03-06T07:22:51.427784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812440+0000","last_change":"2024-03-06T07:22:50.459346+0000","last_active":"2024-03-06T07:22:51.812440+0000","last_peered":"2024-03-06T07:22:51.812440+0000","last_clean":"2024-03-06T07:22:51.812440+0000","last_became_active":"2024-03-06T07:22:50.459087+0000","last_became_peered":"2024-03-06T07:22:50.459087+0000","last_unstale":"2024-03-06T07:22:51.812440+0000","last_undegraded":"2024-03-06T07:22:51.812440+0000","last_fullsized":"2024-03-06T07:22:51.812440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:51.978018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810993+0000","last_change":"2024-03-06T07:22:35.364819+0000","last_active":"2024-03-06T07:22:51.810993+0000","last_peered":"2024-03-06T07:22:51.810993+0000","last_clean":"2024-03-06T07:22:51.810993+0000","last_became_active":"2024-03-06T07:22:35.364529+0000","last_became_peered":"2024-03-06T07:22:35.364529+0000","last_unstale":"2024-03-06T07:22:51.810993+0000","last_undegraded":"2024-03-06T07:22:51.810993+0000","last_fullsized":"2024-03-06T07:22:51.810993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:57.997825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811724+0000","last_change":"2024-03-06T07:22:50.442684+0000","last_active":"2024-03-06T07:22:51.811724+0000","last_peered":"2024-03-06T07:22:51.811724+0000","last_clean":"2024-03-06T07:22:51.811724+0000","last_became_active":"2024-03-06T07:22:50.442400+0000","last_became_peered":"2024-03-06T07:22:50.442400+0000","last_unstale":"2024-03-06T07:22:51.811724+0000","last_undegraded":"2024-03-06T07:22:51.811724+0000","last_fullsized":"2024-03-06T07:22:51.811724+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:04.189588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810096+0000","last_change":"2024-03-06T07:22:35.363080+0000","last_active":"2024-03-06T07:22:51.810096+0000","last_peered":"2024-03-06T07:22:51.810096+0000","last_clean":"2024-03-06T07:22:51.810096+0000","last_became_active":"2024-03-06T07:22:35.362738+0000","last_became_peered":"2024-03-06T07:22:35.362738+0000","last_unstale":"2024-03-06T07:22:51.810096+0000","last_undegraded":"2024-03-06T07:22:51.810096+0000","last_fullsized":"2024-03-06T07:22:51.810096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:32.597953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430030+0000","last_change":"2024-03-06T07:22:50.433096+0000","last_active":"2024-03-06T07:22:51.430030+0000","last_peered":"2024-03-06T07:22:51.430030+0000","last_clean":"2024-03-06T07:22:51.430030+0000","last_became_active":"2024-03-06T07:22:50.432822+0000","last_became_peered":"2024-03-06T07:22:50.432822+0000","last_unstale":"2024-03-06T07:22:51.430030+0000","last_undegraded":"2024-03-06T07:22:51.430030+0000","last_fullsized":"2024-03-06T07:22:51.430030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:39:00.272335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429577+0000","last_change":"2024-03-06T07:22:35.368134+0000","last_active":"2024-03-06T07:22:51.429577+0000","last_peered":"2024-03-06T07:22:51.429577+0000","last_clean":"2024-03-06T07:22:51.429577+0000","last_became_active":"2024-03-06T07:22:35.367824+0000","last_became_peered":"2024-03-06T07:22:35.367824+0000","last_unstale":"2024-03-06T07:22:51.429577+0000","last_undegraded":"2024-03-06T07:22:51.429577+0000","last_fullsized":"2024-03-06T07:22:51.429577+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:30.883851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809766+0000","last_change":"2024-03-06T07:22:50.450091+0000","last_active":"2024-03-06T07:22:51.809766+0000","last_peered":"2024-03-06T07:22:51.809766+0000","last_clean":"2024-03-06T07:22:51.809766+0000","last_became_active":"2024-03-06T07:22:50.449781+0000","last_became_peered":"2024-03-06T07:22:50.449781+0000","last_unstale":"2024-03-06T07:22:51.809766+0000","last_undegraded":"2024-03-06T07:22:51.809766+0000","last_fullsized":"2024-03-06T07:22:51.809766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:42.449192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810467+0000","last_change":"2024-03-06T07:22:35.369823+0000","last_active":"2024-03-06T07:22:51.810467+0000","last_peered":"2024-03-06T07:22:51.810467+0000","last_clean":"2024-03-06T07:22:51.810467+0000","last_became_active":"2024-03-06T07:22:35.369564+0000","last_became_peered":"2024-03-06T07:22:35.369564+0000","last_unstale":"2024-03-06T07:22:51.810467+0000","last_undegraded":"2024-03-06T07:22:51.810467+0000","last_fullsized":"2024-03-06T07:22:51.810467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:20:57.975991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429452+0000","last_change":"2024-03-06T07:22:50.442317+0000","last_active":"2024-03-06T07:22:51.429452+0000","last_peered":"2024-03-06T07:22:51.429452+0000","last_clean":"2024-03-06T07:22:51.429452+0000","last_became_active":"2024-03-06T07:22:50.442066+0000","last_became_peered":"2024-03-06T07:22:50.442066+0000","last_unstale":"2024-03-06T07:22:51.429452+0000","last_undegraded":"2024-03-06T07:22:51.429452+0000","last_fullsized":"2024-03-06T07:22:51.429452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:11.728819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429051+0000","last_change":"2024-03-06T07:22:35.362997+0000","last_active":"2024-03-06T07:22:51.429051+0000","last_peered":"2024-03-06T07:22:51.429051+0000","last_clean":"2024-03-06T07:22:51.429051+0000","last_became_active":"2024-03-06T07:22:35.362735+0000","last_became_peered":"2024-03-06T07:22:35.362735+0000","last_unstale":"2024-03-06T07:22:51.429051+0000","last_undegraded":"2024-03-06T07:22:51.429051+0000","last_fullsized":"2024-03-06T07:22:51.429051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:42:07.246519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429444+0000","last_change":"2024-03-06T07:22:50.439392+0000","last_active":"2024-03-06T07:22:51.429444+0000","last_peered":"2024-03-06T07:22:51.429444+0000","last_clean":"2024-03-06T07:22:51.429444+0000","last_became_active":"2024-03-06T07:22:50.439117+0000","last_became_peered":"2024-03-06T07:22:50.439117+0000","last_unstale":"2024-03-06T07:22:51.429444+0000","last_undegraded":"2024-03-06T07:22:51.429444+0000","last_fullsized":"2024-03-06T07:22:51.429444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:13.076972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811697+0000","last_change":"2024-03-06T07:22:35.367093+0000","last_active":"2024-03-06T07:22:51.811697+0000","last_peered":"2024-03-06T07:22:51.811697+0000","last_clean":"2024-03-06T07:22:51.811697+0000","last_became_active":"2024-03-06T07:22:35.366829+0000","last_became_peered":"2024-03-06T07:22:35.366829+0000","last_unstale":"2024-03-06T07:22:51.811697+0000","last_undegraded":"2024-03-06T07:22:51.811697+0000","last_fullsized":"2024-03-06T07:22:51.811697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:48.647977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810614+0000","last_change":"2024-03-06T07:22:50.430917+0000","last_active":"2024-03-06T07:22:51.810614+0000","last_peered":"2024-03-06T07:22:51.810614+0000","last_clean":"2024-03-06T07:22:51.810614+0000","last_became_active":"2024-03-06T07:22:50.430574+0000","last_became_peered":"2024-03-06T07:22:50.430574+0000","last_unstale":"2024-03-06T07:22:51.810614+0000","last_undegraded":"2024-03-06T07:22:51.810614+0000","last_fullsized":"2024-03-06T07:22:51.810614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:46.085205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812491+0000","last_change":"2024-03-06T07:22:35.367432+0000","last_active":"2024-03-06T07:22:51.812491+0000","last_peered":"2024-03-06T07:22:51.812491+0000","last_clean":"2024-03-06T07:22:51.812491+0000","last_became_active":"2024-03-06T07:22:35.367164+0000","last_became_peered":"2024-03-06T07:22:35.367164+0000","last_unstale":"2024-03-06T07:22:51.812491+0000","last_undegraded":"2024-03-06T07:22:51.812491+0000","last_fullsized":"2024-03-06T07:22:51.812491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427630+0000","last_change":"2024-03-06T07:22:50.452478+0000","last_active":"2024-03-06T07:22:51.427630+0000","last_peered":"2024-03-06T07:22:51.427630+0000","last_clean":"2024-03-06T07:22:51.427630+0000","last_became_active":"2024-03-06T07:22:50.452194+0000","last_became_peered":"2024-03-06T07:22:50.452194+0000","last_unstale":"2024-03-06T07:22:51.427630+0000","last_undegraded":"2024-03-06T07:22:51.427630+0000","last_fullsized":"2024-03-06T07:22:51.427630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:58.355065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810383+0000","last_change":"2024-03-06T07:22:35.373131+0000","last_active":"2024-03-06T07:22:51.810383+0000","last_peered":"2024-03-06T07:22:51.810383+0000","last_clean":"2024-03-06T07:22:51.810383+0000","last_became_active":"2024-03-06T07:22:35.372836+0000","last_became_peered":"2024-03-06T07:22:35.372836+0000","last_unstale":"2024-03-06T07:22:51.810383+0000","last_undegraded":"2024-03-06T07:22:51.810383+0000","last_fullsized":"2024-03-06T07:22:51.810383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:54.664573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810015+0000","last_change":"2024-03-06T07:22:50.441478+0000","last_active":"2024-03-06T07:22:51.810015+0000","last_peered":"2024-03-06T07:22:51.810015+0000","last_clean":"2024-03-06T07:22:51.810015+0000","last_became_active":"2024-03-06T07:22:50.441237+0000","last_became_peered":"2024-03-06T07:22:50.441237+0000","last_unstale":"2024-03-06T07:22:51.810015+0000","last_undegraded":"2024-03-06T07:22:51.810015+0000","last_fullsized":"2024-03-06T07:22:51.810015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:56.658779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810223+0000","last_change":"2024-03-06T07:22:35.373963+0000","last_active":"2024-03-06T07:22:51.810223+0000","last_peered":"2024-03-06T07:22:51.810223+0000","last_clean":"2024-03-06T07:22:51.810223+0000","last_became_active":"2024-03-06T07:22:35.373704+0000","last_became_peered":"2024-03-06T07:22:35.373704+0000","last_unstale":"2024-03-06T07:22:51.810223+0000","last_undegraded":"2024-03-06T07:22:51.810223+0000","last_fullsized":"2024-03-06T07:22:51.810223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429396+0000","last_change":"2024-03-06T07:22:50.437806+0000","last_active":"2024-03-06T07:22:51.429396+0000","last_peered":"2024-03-06T07:22:51.429396+0000","last_clean":"2024-03-06T07:22:51.429396+0000","last_became_active":"2024-03-06T07:22:50.437511+0000","last_became_peered":"2024-03-06T07:22:50.437511+0000","last_unstale":"2024-03-06T07:22:51.429396+0000","last_undegraded":"2024-03-06T07:22:51.429396+0000","last_fullsized":"2024-03-06T07:22:51.429396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:31.447226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810190+0000","last_change":"2024-03-06T07:22:35.377110+0000","last_active":"2024-03-06T07:22:51.810190+0000","last_peered":"2024-03-06T07:22:51.810190+0000","last_clean":"2024-03-06T07:22:51.810190+0000","last_became_active":"2024-03-06T07:22:35.376775+0000","last_became_peered":"2024-03-06T07:22:35.376775+0000","last_unstale":"2024-03-06T07:22:51.810190+0000","last_undegraded":"2024-03-06T07:22:51.810190+0000","last_fullsized":"2024-03-06T07:22:51.810190+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:03.774982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810537+0000","last_change":"2024-03-06T07:22:50.452226+0000","last_active":"2024-03-06T07:22:51.810537+0000","last_peered":"2024-03-06T07:22:51.810537+0000","last_clean":"2024-03-06T07:22:51.810537+0000","last_became_active":"2024-03-06T07:22:50.452010+0000","last_became_peered":"2024-03-06T07:22:50.452010+0000","last_unstale":"2024-03-06T07:22:51.810537+0000","last_undegraded":"2024-03-06T07:22:51.810537+0000","last_fullsized":"2024-03-06T07:22:51.810537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:57.070313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812607+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:51.812607+0000","last_peered":"2024-03-06T07:22:51.812607+0000","last_clean":"2024-03-06T07:22:51.812607+0000","last_became_active":"2024-03-06T07:22:35.361468+0000","last_became_peered":"2024-03-06T07:22:35.361468+0000","last_unstale":"2024-03-06T07:22:51.812607+0000","last_undegraded":"2024-03-06T07:22:51.812607+0000","last_fullsized":"2024-03-06T07:22:51.812607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:50.366055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809842+0000","last_change":"2024-03-06T07:22:50.459727+0000","last_active":"2024-03-06T07:22:51.809842+0000","last_peered":"2024-03-06T07:22:51.809842+0000","last_clean":"2024-03-06T07:22:51.809842+0000","last_became_active":"2024-03-06T07:22:50.458917+0000","last_became_peered":"2024-03-06T07:22:50.458917+0000","last_unstale":"2024-03-06T07:22:51.809842+0000","last_undegraded":"2024-03-06T07:22:51.809842+0000","last_fullsized":"2024-03-06T07:22:51.809842+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:57:46.908735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810341+0000","last_change":"2024-03-06T07:22:35.372500+0000","last_active":"2024-03-06T07:22:51.810341+0000","last_peered":"2024-03-06T07:22:51.810341+0000","last_clean":"2024-03-06T07:22:51.810341+0000","last_became_active":"2024-03-06T07:22:35.372221+0000","last_became_peered":"2024-03-06T07:22:35.372221+0000","last_unstale":"2024-03-06T07:22:51.810341+0000","last_undegraded":"2024-03-06T07:22:51.810341+0000","last_fullsized":"2024-03-06T07:22:51.810341+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:23.442086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810216+0000","last_change":"2024-03-06T07:22:50.441637+0000","last_active":"2024-03-06T07:22:51.810216+0000","last_peered":"2024-03-06T07:22:51.810216+0000","last_clean":"2024-03-06T07:22:51.810216+0000","last_became_active":"2024-03-06T07:22:50.441392+0000","last_became_peered":"2024-03-06T07:22:50.441392+0000","last_unstale":"2024-03-06T07:22:51.810216+0000","last_undegraded":"2024-03-06T07:22:51.810216+0000","last_fullsized":"2024-03-06T07:22:51.810216+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:47:42.449192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426857+0000","last_change":"2024-03-06T07:22:35.365634+0000","last_active":"2024-03-06T07:22:51.426857+0000","last_peered":"2024-03-06T07:22:51.426857+0000","last_clean":"2024-03-06T07:22:51.426857+0000","last_became_active":"2024-03-06T07:22:35.365345+0000","last_became_peered":"2024-03-06T07:22:35.365345+0000","last_unstale":"2024-03-06T07:22:51.426857+0000","last_undegraded":"2024-03-06T07:22:51.426857+0000","last_fullsized":"2024-03-06T07:22:51.426857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:41.970389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429211+0000","last_change":"2024-03-06T07:22:50.439323+0000","last_active":"2024-03-06T07:22:51.429211+0000","last_peered":"2024-03-06T07:22:51.429211+0000","last_clean":"2024-03-06T07:22:51.429211+0000","last_became_active":"2024-03-06T07:22:50.438372+0000","last_became_peered":"2024-03-06T07:22:50.438372+0000","last_unstale":"2024-03-06T07:22:51.429211+0000","last_undegraded":"2024-03-06T07:22:51.429211+0000","last_fullsized":"2024-03-06T07:22:51.429211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809692+0000","last_change":"2024-03-06T07:22:35.363020+0000","last_active":"2024-03-06T07:22:51.809692+0000","last_peered":"2024-03-06T07:22:51.809692+0000","last_clean":"2024-03-06T07:22:51.809692+0000","last_became_active":"2024-03-06T07:22:35.362767+0000","last_became_peered":"2024-03-06T07:22:35.362767+0000","last_unstale":"2024-03-06T07:22:51.809692+0000","last_undegraded":"2024-03-06T07:22:51.809692+0000","last_fullsized":"2024-03-06T07:22:51.809692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:56.148980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811625+0000","last_change":"2024-03-06T07:22:50.449703+0000","last_active":"2024-03-06T07:22:51.811625+0000","last_peered":"2024-03-06T07:22:51.811625+0000","last_clean":"2024-03-06T07:22:51.811625+0000","last_became_active":"2024-03-06T07:22:50.449106+0000","last_became_peered":"2024-03-06T07:22:50.449106+0000","last_unstale":"2024-03-06T07:22:51.811625+0000","last_undegraded":"2024-03-06T07:22:51.811625+0000","last_fullsized":"2024-03-06T07:22:51.811625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:57:52.147096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812090+0000","last_change":"2024-03-06T07:22:49.443093+0000","last_active":"2024-03-06T07:22:51.812090+0000","last_peered":"2024-03-06T07:22:51.812090+0000","last_clean":"2024-03-06T07:22:51.812090+0000","last_became_active":"2024-03-06T07:22:49.442628+0000","last_became_peered":"2024-03-06T07:22:49.442628+0000","last_unstale":"2024-03-06T07:22:51.812090+0000","last_undegraded":"2024-03-06T07:22:51.812090+0000","last_fullsized":"2024-03-06T07:22:51.812090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:56.065095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426906+0000","last_change":"2024-03-06T07:22:35.357934+0000","last_active":"2024-03-06T07:22:51.426906+0000","last_peered":"2024-03-06T07:22:51.426906+0000","last_clean":"2024-03-06T07:22:51.426906+0000","last_became_active":"2024-03-06T07:22:35.357639+0000","last_became_peered":"2024-03-06T07:22:35.357639+0000","last_unstale":"2024-03-06T07:22:51.426906+0000","last_undegraded":"2024-03-06T07:22:51.426906+0000","last_fullsized":"2024-03-06T07:22:51.426906+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812431+0000","last_change":"2024-03-06T07:22:34.362186+0000","last_active":"2024-03-06T07:22:51.812431+0000","last_peered":"2024-03-06T07:22:51.812431+0000","last_clean":"2024-03-06T07:22:51.812431+0000","last_became_active":"2024-03-06T07:22:34.361912+0000","last_became_peered":"2024-03-06T07:22:34.361912+0000","last_unstale":"2024-03-06T07:22:51.812431+0000","last_undegraded":"2024-03-06T07:22:51.812431+0000","last_fullsized":"2024-03-06T07:22:51.812431+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:40:05.570540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427171+0000","last_change":"2024-03-06T07:22:50.440550+0000","last_active":"2024-03-06T07:22:51.427171+0000","last_peered":"2024-03-06T07:22:51.427171+0000","last_clean":"2024-03-06T07:22:51.427171+0000","last_became_active":"2024-03-06T07:22:50.440230+0000","last_became_peered":"2024-03-06T07:22:50.440230+0000","last_unstale":"2024-03-06T07:22:51.427171+0000","last_undegraded":"2024-03-06T07:22:51.427171+0000","last_fullsized":"2024-03-06T07:22:51.427171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:29.051388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809740+0000","last_change":"2024-03-06T07:22:49.442614+0000","last_active":"2024-03-06T07:22:51.809740+0000","last_peered":"2024-03-06T07:22:51.809740+0000","last_clean":"2024-03-06T07:22:51.809740+0000","last_became_active":"2024-03-06T07:22:49.442328+0000","last_became_peered":"2024-03-06T07:22:49.442328+0000","last_unstale":"2024-03-06T07:22:51.809740+0000","last_undegraded":"2024-03-06T07:22:51.809740+0000","last_fullsized":"2024-03-06T07:22:51.809740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:43.026177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812003+0000","last_change":"2024-03-06T07:22:35.368816+0000","last_active":"2024-03-06T07:22:51.812003+0000","last_peered":"2024-03-06T07:22:51.812003+0000","last_clean":"2024-03-06T07:22:51.812003+0000","last_became_active":"2024-03-06T07:22:35.368555+0000","last_became_peered":"2024-03-06T07:22:35.368555+0000","last_unstale":"2024-03-06T07:22:51.812003+0000","last_undegraded":"2024-03-06T07:22:51.812003+0000","last_fullsized":"2024-03-06T07:22:51.812003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:17:20.109504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810001+0000","last_change":"2024-03-06T07:22:34.366335+0000","last_active":"2024-03-06T07:22:51.810001+0000","last_peered":"2024-03-06T07:22:51.810001+0000","last_clean":"2024-03-06T07:22:51.810001+0000","last_became_active":"2024-03-06T07:22:34.364387+0000","last_became_peered":"2024-03-06T07:22:34.364387+0000","last_unstale":"2024-03-06T07:22:51.810001+0000","last_undegraded":"2024-03-06T07:22:51.810001+0000","last_fullsized":"2024-03-06T07:22:51.810001+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426615+0000","last_change":"2024-03-06T07:22:50.444770+0000","last_active":"2024-03-06T07:22:51.426615+0000","last_peered":"2024-03-06T07:22:51.426615+0000","last_clean":"2024-03-06T07:22:51.426615+0000","last_became_active":"2024-03-06T07:22:50.444438+0000","last_became_peered":"2024-03-06T07:22:50.444438+0000","last_unstale":"2024-03-06T07:22:51.426615+0000","last_undegraded":"2024-03-06T07:22:51.426615+0000","last_fullsized":"2024-03-06T07:22:51.426615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429288+0000","last_change":"2024-03-06T07:22:49.437241+0000","last_active":"2024-03-06T07:22:51.429288+0000","last_peered":"2024-03-06T07:22:51.429288+0000","last_clean":"2024-03-06T07:22:51.429288+0000","last_became_active":"2024-03-06T07:22:49.436832+0000","last_became_peered":"2024-03-06T07:22:49.436832+0000","last_unstale":"2024-03-06T07:22:51.429288+0000","last_undegraded":"2024-03-06T07:22:51.429288+0000","last_fullsized":"2024-03-06T07:22:51.429288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:35.129907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429558+0000","last_change":"2024-03-06T07:22:35.370528+0000","last_active":"2024-03-06T07:22:51.429558+0000","last_peered":"2024-03-06T07:22:51.429558+0000","last_clean":"2024-03-06T07:22:51.429558+0000","last_became_active":"2024-03-06T07:22:35.369754+0000","last_became_peered":"2024-03-06T07:22:35.369754+0000","last_unstale":"2024-03-06T07:22:51.429558+0000","last_undegraded":"2024-03-06T07:22:51.429558+0000","last_fullsized":"2024-03-06T07:22:51.429558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:08.273895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810318+0000","last_change":"2024-03-06T07:22:34.364533+0000","last_active":"2024-03-06T07:22:51.810318+0000","last_peered":"2024-03-06T07:22:51.810318+0000","last_clean":"2024-03-06T07:22:51.810318+0000","last_became_active":"2024-03-06T07:22:34.364224+0000","last_became_peered":"2024-03-06T07:22:34.364224+0000","last_unstale":"2024-03-06T07:22:51.810318+0000","last_undegraded":"2024-03-06T07:22:51.810318+0000","last_fullsized":"2024-03-06T07:22:51.810318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:13.454983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430167+0000","last_change":"2024-03-06T07:22:50.453853+0000","last_active":"2024-03-06T07:22:51.430167+0000","last_peered":"2024-03-06T07:22:51.430167+0000","last_clean":"2024-03-06T07:22:51.430167+0000","last_became_active":"2024-03-06T07:22:50.453578+0000","last_became_peered":"2024-03-06T07:22:50.453578+0000","last_unstale":"2024-03-06T07:22:51.430167+0000","last_undegraded":"2024-03-06T07:22:51.430167+0000","last_fullsized":"2024-03-06T07:22:51.430167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:34.188282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427107+0000","last_change":"2024-03-06T07:22:49.444227+0000","last_active":"2024-03-06T07:22:51.427107+0000","last_peered":"2024-03-06T07:22:51.427107+0000","last_clean":"2024-03-06T07:22:51.427107+0000","last_became_active":"2024-03-06T07:22:49.443963+0000","last_became_peered":"2024-03-06T07:22:49.443963+0000","last_unstale":"2024-03-06T07:22:51.427107+0000","last_undegraded":"2024-03-06T07:22:51.427107+0000","last_fullsized":"2024-03-06T07:22:51.427107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:56.306976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429664+0000","last_change":"2024-03-06T07:22:35.361422+0000","last_active":"2024-03-06T07:22:51.429664+0000","last_peered":"2024-03-06T07:22:51.429664+0000","last_clean":"2024-03-06T07:22:51.429664+0000","last_became_active":"2024-03-06T07:22:35.361001+0000","last_became_peered":"2024-03-06T07:22:35.361001+0000","last_unstale":"2024-03-06T07:22:51.429664+0000","last_undegraded":"2024-03-06T07:22:51.429664+0000","last_fullsized":"2024-03-06T07:22:51.429664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:55.348730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431057+0000","last_change":"2024-03-06T07:22:34.366051+0000","last_active":"2024-03-06T07:22:51.431057+0000","last_peered":"2024-03-06T07:22:51.431057+0000","last_clean":"2024-03-06T07:22:51.431057+0000","last_became_active":"2024-03-06T07:22:34.365596+0000","last_became_peered":"2024-03-06T07:22:34.365596+0000","last_unstale":"2024-03-06T07:22:51.431057+0000","last_undegraded":"2024-03-06T07:22:51.431057+0000","last_fullsized":"2024-03-06T07:22:51.431057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:48.411834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809934+0000","last_change":"2024-03-06T07:22:50.444294+0000","last_active":"2024-03-06T07:22:51.809934+0000","last_peered":"2024-03-06T07:22:51.809934+0000","last_clean":"2024-03-06T07:22:51.809934+0000","last_became_active":"2024-03-06T07:22:50.444066+0000","last_became_peered":"2024-03-06T07:22:50.444066+0000","last_unstale":"2024-03-06T07:22:51.809934+0000","last_undegraded":"2024-03-06T07:22:51.809934+0000","last_fullsized":"2024-03-06T07:22:51.809934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:34.201578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809674+0000","last_change":"2024-03-06T07:22:49.446138+0000","last_active":"2024-03-06T07:22:51.809674+0000","last_peered":"2024-03-06T07:22:51.809674+0000","last_clean":"2024-03-06T07:22:51.809674+0000","last_became_active":"2024-03-06T07:22:49.445911+0000","last_became_peered":"2024-03-06T07:22:49.445911+0000","last_unstale":"2024-03-06T07:22:51.809674+0000","last_undegraded":"2024-03-06T07:22:51.809674+0000","last_fullsized":"2024-03-06T07:22:51.809674+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:15.943887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811636+0000","last_change":"2024-03-06T07:22:35.365350+0000","last_active":"2024-03-06T07:22:51.811636+0000","last_peered":"2024-03-06T07:22:51.811636+0000","last_clean":"2024-03-06T07:22:51.811636+0000","last_became_active":"2024-03-06T07:22:35.365049+0000","last_became_peered":"2024-03-06T07:22:35.365049+0000","last_unstale":"2024-03-06T07:22:51.811636+0000","last_undegraded":"2024-03-06T07:22:51.811636+0000","last_fullsized":"2024-03-06T07:22:51.811636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429736+0000","last_change":"2024-03-06T07:22:34.359679+0000","last_active":"2024-03-06T07:22:51.429736+0000","last_peered":"2024-03-06T07:22:51.429736+0000","last_clean":"2024-03-06T07:22:51.429736+0000","last_became_active":"2024-03-06T07:22:34.359409+0000","last_became_peered":"2024-03-06T07:22:34.359409+0000","last_unstale":"2024-03-06T07:22:51.429736+0000","last_undegraded":"2024-03-06T07:22:51.429736+0000","last_fullsized":"2024-03-06T07:22:51.429736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:12:17.027514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812558+0000","last_change":"2024-03-06T07:22:50.451370+0000","last_active":"2024-03-06T07:22:51.812558+0000","last_peered":"2024-03-06T07:22:51.812558+0000","last_clean":"2024-03-06T07:22:51.812558+0000","last_became_active":"2024-03-06T07:22:50.451068+0000","last_became_peered":"2024-03-06T07:22:50.451068+0000","last_unstale":"2024-03-06T07:22:51.812558+0000","last_undegraded":"2024-03-06T07:22:51.812558+0000","last_fullsized":"2024-03-06T07:22:51.812558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:53.346438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430820+0000","last_change":"2024-03-06T07:22:49.430583+0000","last_active":"2024-03-06T07:22:51.430820+0000","last_peered":"2024-03-06T07:22:51.430820+0000","last_clean":"2024-03-06T07:22:51.430820+0000","last_became_active":"2024-03-06T07:22:49.430315+0000","last_became_peered":"2024-03-06T07:22:49.430315+0000","last_unstale":"2024-03-06T07:22:51.430820+0000","last_undegraded":"2024-03-06T07:22:51.430820+0000","last_fullsized":"2024-03-06T07:22:51.430820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812287+0000","last_change":"2024-03-06T07:22:35.363944+0000","last_active":"2024-03-06T07:22:51.812287+0000","last_peered":"2024-03-06T07:22:51.812287+0000","last_clean":"2024-03-06T07:22:51.812287+0000","last_became_active":"2024-03-06T07:22:35.363662+0000","last_became_peered":"2024-03-06T07:22:35.363662+0000","last_unstale":"2024-03-06T07:22:51.812287+0000","last_undegraded":"2024-03-06T07:22:51.812287+0000","last_fullsized":"2024-03-06T07:22:51.812287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:17.265969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809835+0000","last_change":"2024-03-06T07:22:34.360546+0000","last_active":"2024-03-06T07:22:51.809835+0000","last_peered":"2024-03-06T07:22:51.809835+0000","last_clean":"2024-03-06T07:22:51.809835+0000","last_became_active":"2024-03-06T07:22:34.360231+0000","last_became_peered":"2024-03-06T07:22:34.360231+0000","last_unstale":"2024-03-06T07:22:51.809835+0000","last_undegraded":"2024-03-06T07:22:51.809835+0000","last_fullsized":"2024-03-06T07:22:51.809835+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:02.637453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811398+0000","last_change":"2024-03-06T07:22:50.453874+0000","last_active":"2024-03-06T07:22:51.811398+0000","last_peered":"2024-03-06T07:22:51.811398+0000","last_clean":"2024-03-06T07:22:51.811398+0000","last_became_active":"2024-03-06T07:22:50.453403+0000","last_became_peered":"2024-03-06T07:22:50.453403+0000","last_unstale":"2024-03-06T07:22:51.811398+0000","last_undegraded":"2024-03-06T07:22:51.811398+0000","last_fullsized":"2024-03-06T07:22:51.811398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:49:39.691686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430540+0000","last_change":"2024-03-06T07:22:49.443801+0000","last_active":"2024-03-06T07:22:51.430540+0000","last_peered":"2024-03-06T07:22:51.430540+0000","last_clean":"2024-03-06T07:22:51.430540+0000","last_became_active":"2024-03-06T07:22:49.443540+0000","last_became_peered":"2024-03-06T07:22:49.443540+0000","last_unstale":"2024-03-06T07:22:51.430540+0000","last_undegraded":"2024-03-06T07:22:51.430540+0000","last_fullsized":"2024-03-06T07:22:51.430540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:01.309770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430841+0000","last_change":"2024-03-06T07:22:35.365427+0000","last_active":"2024-03-06T07:22:51.430841+0000","last_peered":"2024-03-06T07:22:51.430841+0000","last_clean":"2024-03-06T07:22:51.430841+0000","last_became_active":"2024-03-06T07:22:35.365162+0000","last_became_peered":"2024-03-06T07:22:35.365162+0000","last_unstale":"2024-03-06T07:22:51.430841+0000","last_undegraded":"2024-03-06T07:22:51.430841+0000","last_fullsized":"2024-03-06T07:22:51.430841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812003+0000","last_change":"2024-03-06T07:22:34.366433+0000","last_active":"2024-03-06T07:22:51.812003+0000","last_peered":"2024-03-06T07:22:51.812003+0000","last_clean":"2024-03-06T07:22:51.812003+0000","last_became_active":"2024-03-06T07:22:34.365956+0000","last_became_peered":"2024-03-06T07:22:34.365956+0000","last_unstale":"2024-03-06T07:22:51.812003+0000","last_undegraded":"2024-03-06T07:22:51.812003+0000","last_fullsized":"2024-03-06T07:22:51.812003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812114+0000","last_change":"2024-03-06T07:22:50.453212+0000","last_active":"2024-03-06T07:22:51.812114+0000","last_peered":"2024-03-06T07:22:51.812114+0000","last_clean":"2024-03-06T07:22:51.812114+0000","last_became_active":"2024-03-06T07:22:50.452795+0000","last_became_peered":"2024-03-06T07:22:50.452795+0000","last_unstale":"2024-03-06T07:22:51.812114+0000","last_undegraded":"2024-03-06T07:22:51.812114+0000","last_fullsized":"2024-03-06T07:22:51.812114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:07.928847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430576+0000","last_change":"2024-03-06T07:22:49.439528+0000","last_active":"2024-03-06T07:22:51.430576+0000","last_peered":"2024-03-06T07:22:51.430576+0000","last_clean":"2024-03-06T07:22:51.430576+0000","last_became_active":"2024-03-06T07:22:49.439249+0000","last_became_peered":"2024-03-06T07:22:49.439249+0000","last_unstale":"2024-03-06T07:22:51.430576+0000","last_undegraded":"2024-03-06T07:22:51.430576+0000","last_fullsized":"2024-03-06T07:22:51.430576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:08.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430797+0000","last_change":"2024-03-06T07:22:35.372472+0000","last_active":"2024-03-06T07:22:51.430797+0000","last_peered":"2024-03-06T07:22:51.430797+0000","last_clean":"2024-03-06T07:22:51.430797+0000","last_became_active":"2024-03-06T07:22:35.372200+0000","last_became_peered":"2024-03-06T07:22:35.372200+0000","last_unstale":"2024-03-06T07:22:51.430797+0000","last_undegraded":"2024-03-06T07:22:51.430797+0000","last_fullsized":"2024-03-06T07:22:51.430797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:24.785053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809979+0000","last_change":"2024-03-06T07:22:34.359290+0000","last_active":"2024-03-06T07:22:51.809979+0000","last_peered":"2024-03-06T07:22:51.809979+0000","last_clean":"2024-03-06T07:22:51.809979+0000","last_became_active":"2024-03-06T07:22:34.358968+0000","last_became_peered":"2024-03-06T07:22:34.358968+0000","last_unstale":"2024-03-06T07:22:51.809979+0000","last_undegraded":"2024-03-06T07:22:51.809979+0000","last_fullsized":"2024-03-06T07:22:51.809979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:13:21.458206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810113+0000","last_change":"2024-03-06T07:22:50.437572+0000","last_active":"2024-03-06T07:22:51.810113+0000","last_peered":"2024-03-06T07:22:51.810113+0000","last_clean":"2024-03-06T07:22:51.810113+0000","last_became_active":"2024-03-06T07:22:50.437290+0000","last_became_peered":"2024-03-06T07:22:50.437290+0000","last_unstale":"2024-03-06T07:22:51.810113+0000","last_undegraded":"2024-03-06T07:22:51.810113+0000","last_fullsized":"2024-03-06T07:22:51.810113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:17.922064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426849+0000","last_change":"2024-03-06T07:22:49.431955+0000","last_active":"2024-03-06T07:22:51.426849+0000","last_peered":"2024-03-06T07:22:51.426849+0000","last_clean":"2024-03-06T07:22:51.426849+0000","last_became_active":"2024-03-06T07:22:49.431192+0000","last_became_peered":"2024-03-06T07:22:49.431192+0000","last_unstale":"2024-03-06T07:22:51.426849+0000","last_undegraded":"2024-03-06T07:22:51.426849+0000","last_fullsized":"2024-03-06T07:22:51.426849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:46.923868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811787+0000","last_change":"2024-03-06T07:22:35.365470+0000","last_active":"2024-03-06T07:22:51.811787+0000","last_peered":"2024-03-06T07:22:51.811787+0000","last_clean":"2024-03-06T07:22:51.811787+0000","last_became_active":"2024-03-06T07:22:35.365156+0000","last_became_peered":"2024-03-06T07:22:35.365156+0000","last_unstale":"2024-03-06T07:22:51.811787+0000","last_undegraded":"2024-03-06T07:22:51.811787+0000","last_fullsized":"2024-03-06T07:22:51.811787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:45.193188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809999+0000","last_change":"2024-03-06T07:22:34.366420+0000","last_active":"2024-03-06T07:22:51.809999+0000","last_peered":"2024-03-06T07:22:51.809999+0000","last_clean":"2024-03-06T07:22:51.809999+0000","last_became_active":"2024-03-06T07:22:34.365577+0000","last_became_peered":"2024-03-06T07:22:34.365577+0000","last_unstale":"2024-03-06T07:22:51.809999+0000","last_undegraded":"2024-03-06T07:22:51.809999+0000","last_fullsized":"2024-03-06T07:22:51.809999+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:20.051114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810140+0000","last_change":"2024-03-06T07:22:50.438430+0000","last_active":"2024-03-06T07:22:51.810140+0000","last_peered":"2024-03-06T07:22:51.810140+0000","last_clean":"2024-03-06T07:22:51.810140+0000","last_became_active":"2024-03-06T07:22:50.438163+0000","last_became_peered":"2024-03-06T07:22:50.438163+0000","last_unstale":"2024-03-06T07:22:51.810140+0000","last_undegraded":"2024-03-06T07:22:51.810140+0000","last_fullsized":"2024-03-06T07:22:51.810140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:31.237118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426732+0000","last_change":"2024-03-06T07:22:49.441677+0000","last_active":"2024-03-06T07:22:51.426732+0000","last_peered":"2024-03-06T07:22:51.426732+0000","last_clean":"2024-03-06T07:22:51.426732+0000","last_became_active":"2024-03-06T07:22:49.441346+0000","last_became_peered":"2024-03-06T07:22:49.441346+0000","last_unstale":"2024-03-06T07:22:51.426732+0000","last_undegraded":"2024-03-06T07:22:51.426732+0000","last_fullsized":"2024-03-06T07:22:51.426732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810584+0000","last_change":"2024-03-06T07:22:35.367329+0000","last_active":"2024-03-06T07:22:51.810584+0000","last_peered":"2024-03-06T07:22:51.810584+0000","last_clean":"2024-03-06T07:22:51.810584+0000","last_became_active":"2024-03-06T07:22:35.366708+0000","last_became_peered":"2024-03-06T07:22:35.366708+0000","last_unstale":"2024-03-06T07:22:51.810584+0000","last_undegraded":"2024-03-06T07:22:51.810584+0000","last_fullsized":"2024-03-06T07:22:51.810584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:32.849939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810474+0000","last_change":"2024-03-06T07:22:34.367334+0000","last_active":"2024-03-06T07:22:51.810474+0000","last_peered":"2024-03-06T07:22:51.810474+0000","last_clean":"2024-03-06T07:22:51.810474+0000","last_became_active":"2024-03-06T07:22:34.367039+0000","last_became_peered":"2024-03-06T07:22:34.367039+0000","last_unstale":"2024-03-06T07:22:51.810474+0000","last_undegraded":"2024-03-06T07:22:51.810474+0000","last_fullsized":"2024-03-06T07:22:51.810474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426742+0000","last_change":"2024-03-06T07:22:50.453384+0000","last_active":"2024-03-06T07:22:51.426742+0000","last_peered":"2024-03-06T07:22:51.426742+0000","last_clean":"2024-03-06T07:22:51.426742+0000","last_became_active":"2024-03-06T07:22:50.452549+0000","last_became_peered":"2024-03-06T07:22:50.452549+0000","last_unstale":"2024-03-06T07:22:51.426742+0000","last_undegraded":"2024-03-06T07:22:51.426742+0000","last_fullsized":"2024-03-06T07:22:51.426742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:31.447226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810308+0000","last_change":"2024-03-06T07:22:49.443011+0000","last_active":"2024-03-06T07:22:51.810308+0000","last_peered":"2024-03-06T07:22:51.810308+0000","last_clean":"2024-03-06T07:22:51.810308+0000","last_became_active":"2024-03-06T07:22:49.442745+0000","last_became_peered":"2024-03-06T07:22:49.442745+0000","last_unstale":"2024-03-06T07:22:51.810308+0000","last_undegraded":"2024-03-06T07:22:51.810308+0000","last_fullsized":"2024-03-06T07:22:51.810308+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:31.339898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811153+0000","last_change":"2024-03-06T07:22:35.362082+0000","last_active":"2024-03-06T07:22:51.811153+0000","last_peered":"2024-03-06T07:22:51.811153+0000","last_clean":"2024-03-06T07:22:51.811153+0000","last_became_active":"2024-03-06T07:22:35.361383+0000","last_became_peered":"2024-03-06T07:22:35.361383+0000","last_unstale":"2024-03-06T07:22:51.811153+0000","last_undegraded":"2024-03-06T07:22:51.811153+0000","last_fullsized":"2024-03-06T07:22:51.811153+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":25,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810717+0000","last_change":"2024-03-06T07:22:34.359003+0000","last_active":"2024-03-06T07:22:51.810717+0000","last_peered":"2024-03-06T07:22:51.810717+0000","last_clean":"2024-03-06T07:22:51.810717+0000","last_became_active":"2024-03-06T07:22:34.358700+0000","last_became_peered":"2024-03-06T07:22:34.358700+0000","last_unstale":"2024-03-06T07:22:51.810717+0000","last_undegraded":"2024-03-06T07:22:51.810717+0000","last_fullsized":"2024-03-06T07:22:51.810717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:18:55.065516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429075+0000","last_change":"2024-03-06T07:22:50.430626+0000","last_active":"2024-03-06T07:22:51.429075+0000","last_peered":"2024-03-06T07:22:51.429075+0000","last_clean":"2024-03-06T07:22:51.429075+0000","last_became_active":"2024-03-06T07:22:50.430335+0000","last_became_peered":"2024-03-06T07:22:50.430335+0000","last_unstale":"2024-03-06T07:22:51.429075+0000","last_undegraded":"2024-03-06T07:22:51.429075+0000","last_fullsized":"2024-03-06T07:22:51.429075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:34.188282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811394+0000","last_change":"2024-03-06T07:22:49.441435+0000","last_active":"2024-03-06T07:22:51.811394+0000","last_peered":"2024-03-06T07:22:51.811394+0000","last_clean":"2024-03-06T07:22:51.811394+0000","last_became_active":"2024-03-06T07:22:49.441150+0000","last_became_peered":"2024-03-06T07:22:49.441150+0000","last_unstale":"2024-03-06T07:22:51.811394+0000","last_undegraded":"2024-03-06T07:22:51.811394+0000","last_fullsized":"2024-03-06T07:22:51.811394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:56.065095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427220+0000","last_change":"2024-03-06T07:22:35.367264+0000","last_active":"2024-03-06T07:22:51.427220+0000","last_peered":"2024-03-06T07:22:51.427220+0000","last_clean":"2024-03-06T07:22:51.427220+0000","last_became_active":"2024-03-06T07:22:35.366978+0000","last_became_peered":"2024-03-06T07:22:35.366978+0000","last_unstale":"2024-03-06T07:22:51.427220+0000","last_undegraded":"2024-03-06T07:22:51.427220+0000","last_fullsized":"2024-03-06T07:22:51.427220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:24.785053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431109+0000","last_change":"2024-03-06T07:22:34.354553+0000","last_active":"2024-03-06T07:22:51.431109+0000","last_peered":"2024-03-06T07:22:51.431109+0000","last_clean":"2024-03-06T07:22:51.431109+0000","last_became_active":"2024-03-06T07:22:34.354272+0000","last_became_peered":"2024-03-06T07:22:34.354272+0000","last_unstale":"2024-03-06T07:22:51.431109+0000","last_undegraded":"2024-03-06T07:22:51.431109+0000","last_fullsized":"2024-03-06T07:22:51.431109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:41:10.375447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811864+0000","last_change":"2024-03-06T07:22:50.458465+0000","last_active":"2024-03-06T07:22:51.811864+0000","last_peered":"2024-03-06T07:22:51.811864+0000","last_clean":"2024-03-06T07:22:51.811864+0000","last_became_active":"2024-03-06T07:22:50.458179+0000","last_became_peered":"2024-03-06T07:22:50.458179+0000","last_unstale":"2024-03-06T07:22:51.811864+0000","last_undegraded":"2024-03-06T07:22:51.811864+0000","last_fullsized":"2024-03-06T07:22:51.811864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:09.075888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430286+0000","last_change":"2024-03-06T07:22:49.443958+0000","last_active":"2024-03-06T07:22:51.430286+0000","last_peered":"2024-03-06T07:22:51.430286+0000","last_clean":"2024-03-06T07:22:51.430286+0000","last_became_active":"2024-03-06T07:22:49.443712+0000","last_became_peered":"2024-03-06T07:22:49.443712+0000","last_unstale":"2024-03-06T07:22:51.430286+0000","last_undegraded":"2024-03-06T07:22:51.430286+0000","last_fullsized":"2024-03-06T07:22:51.430286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:19.265304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810422+0000","last_change":"2024-03-06T07:22:35.368702+0000","last_active":"2024-03-06T07:22:51.810422+0000","last_peered":"2024-03-06T07:22:51.810422+0000","last_clean":"2024-03-06T07:22:51.810422+0000","last_became_active":"2024-03-06T07:22:35.368437+0000","last_became_peered":"2024-03-06T07:22:35.368437+0000","last_unstale":"2024-03-06T07:22:51.810422+0000","last_undegraded":"2024-03-06T07:22:51.810422+0000","last_fullsized":"2024-03-06T07:22:51.810422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:41.622775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810705+0000","last_change":"2024-03-06T07:22:34.358732+0000","last_active":"2024-03-06T07:22:51.810705+0000","last_peered":"2024-03-06T07:22:51.810705+0000","last_clean":"2024-03-06T07:22:51.810705+0000","last_became_active":"2024-03-06T07:22:34.358424+0000","last_became_peered":"2024-03-06T07:22:34.358424+0000","last_unstale":"2024-03-06T07:22:51.810705+0000","last_undegraded":"2024-03-06T07:22:51.810705+0000","last_fullsized":"2024-03-06T07:22:51.810705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:53.950093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427835+0000","last_change":"2024-03-06T07:22:50.442907+0000","last_active":"2024-03-06T07:22:51.427835+0000","last_peered":"2024-03-06T07:22:51.427835+0000","last_clean":"2024-03-06T07:22:51.427835+0000","last_became_active":"2024-03-06T07:22:50.441488+0000","last_became_peered":"2024-03-06T07:22:50.441488+0000","last_unstale":"2024-03-06T07:22:51.427835+0000","last_undegraded":"2024-03-06T07:22:51.427835+0000","last_fullsized":"2024-03-06T07:22:51.427835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811509+0000","last_change":"2024-03-06T07:22:49.444169+0000","last_active":"2024-03-06T07:22:51.811509+0000","last_peered":"2024-03-06T07:22:51.811509+0000","last_clean":"2024-03-06T07:22:51.811509+0000","last_became_active":"2024-03-06T07:22:49.443899+0000","last_became_peered":"2024-03-06T07:22:49.443899+0000","last_unstale":"2024-03-06T07:22:51.811509+0000","last_undegraded":"2024-03-06T07:22:51.811509+0000","last_fullsized":"2024-03-06T07:22:51.811509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:13.223654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430361+0000","last_change":"2024-03-06T07:22:35.367268+0000","last_active":"2024-03-06T07:22:51.430361+0000","last_peered":"2024-03-06T07:22:51.430361+0000","last_clean":"2024-03-06T07:22:51.430361+0000","last_became_active":"2024-03-06T07:22:35.365175+0000","last_became_peered":"2024-03-06T07:22:35.365175+0000","last_unstale":"2024-03-06T07:22:51.430361+0000","last_undegraded":"2024-03-06T07:22:51.430361+0000","last_fullsized":"2024-03-06T07:22:51.430361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:31.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810433+0000","last_change":"2024-03-06T07:22:34.364874+0000","last_active":"2024-03-06T07:22:51.810433+0000","last_peered":"2024-03-06T07:22:51.810433+0000","last_clean":"2024-03-06T07:22:51.810433+0000","last_became_active":"2024-03-06T07:22:34.364472+0000","last_became_peered":"2024-03-06T07:22:34.364472+0000","last_unstale":"2024-03-06T07:22:51.810433+0000","last_undegraded":"2024-03-06T07:22:51.810433+0000","last_fullsized":"2024-03-06T07:22:51.810433+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429504+0000","last_change":"2024-03-06T07:22:50.443326+0000","last_active":"2024-03-06T07:22:51.429504+0000","last_peered":"2024-03-06T07:22:51.429504+0000","last_clean":"2024-03-06T07:22:51.429504+0000","last_became_active":"2024-03-06T07:22:50.442437+0000","last_became_peered":"2024-03-06T07:22:50.442437+0000","last_unstale":"2024-03-06T07:22:51.429504+0000","last_undegraded":"2024-03-06T07:22:51.429504+0000","last_fullsized":"2024-03-06T07:22:51.429504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:29.051388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810109+0000","last_change":"2024-03-06T07:22:49.435101+0000","last_active":"2024-03-06T07:22:51.810109+0000","last_peered":"2024-03-06T07:22:51.810109+0000","last_clean":"2024-03-06T07:22:51.810109+0000","last_became_active":"2024-03-06T07:22:49.434840+0000","last_became_peered":"2024-03-06T07:22:49.434840+0000","last_unstale":"2024-03-06T07:22:51.810109+0000","last_undegraded":"2024-03-06T07:22:51.810109+0000","last_fullsized":"2024-03-06T07:22:51.810109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:30.442008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430287+0000","last_change":"2024-03-06T07:22:35.357377+0000","last_active":"2024-03-06T07:22:51.430287+0000","last_peered":"2024-03-06T07:22:51.430287+0000","last_clean":"2024-03-06T07:22:51.430287+0000","last_became_active":"2024-03-06T07:22:35.357071+0000","last_became_peered":"2024-03-06T07:22:35.357071+0000","last_unstale":"2024-03-06T07:22:51.430287+0000","last_undegraded":"2024-03-06T07:22:51.430287+0000","last_fullsized":"2024-03-06T07:22:51.430287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:41.970389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809888+0000","last_change":"2024-03-06T07:22:34.362221+0000","last_active":"2024-03-06T07:22:51.809888+0000","last_peered":"2024-03-06T07:22:51.809888+0000","last_clean":"2024-03-06T07:22:51.809888+0000","last_became_active":"2024-03-06T07:22:34.361405+0000","last_became_peered":"2024-03-06T07:22:34.361405+0000","last_unstale":"2024-03-06T07:22:51.809888+0000","last_undegraded":"2024-03-06T07:22:51.809888+0000","last_fullsized":"2024-03-06T07:22:51.809888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:52.040107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810508+0000","last_change":"2024-03-06T07:22:50.448230+0000","last_active":"2024-03-06T07:22:51.810508+0000","last_peered":"2024-03-06T07:22:51.810508+0000","last_clean":"2024-03-06T07:22:51.810508+0000","last_became_active":"2024-03-06T07:22:50.447938+0000","last_became_peered":"2024-03-06T07:22:50.447938+0000","last_unstale":"2024-03-06T07:22:51.810508+0000","last_undegraded":"2024-03-06T07:22:51.810508+0000","last_fullsized":"2024-03-06T07:22:51.810508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:02:49.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811952+0000","last_change":"2024-03-06T07:22:49.444457+0000","last_active":"2024-03-06T07:22:51.811952+0000","last_peered":"2024-03-06T07:22:51.811952+0000","last_clean":"2024-03-06T07:22:51.811952+0000","last_became_active":"2024-03-06T07:22:49.444169+0000","last_became_peered":"2024-03-06T07:22:49.444169+0000","last_unstale":"2024-03-06T07:22:51.811952+0000","last_undegraded":"2024-03-06T07:22:51.811952+0000","last_fullsized":"2024-03-06T07:22:51.811952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:42.370899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430818+0000","last_change":"2024-03-06T07:22:35.367997+0000","last_active":"2024-03-06T07:22:51.430818+0000","last_peered":"2024-03-06T07:22:51.430818+0000","last_clean":"2024-03-06T07:22:51.430818+0000","last_became_active":"2024-03-06T07:22:35.364741+0000","last_became_peered":"2024-03-06T07:22:35.364741+0000","last_unstale":"2024-03-06T07:22:51.430818+0000","last_undegraded":"2024-03-06T07:22:51.430818+0000","last_fullsized":"2024-03-06T07:22:51.430818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:54.954958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811885+0000","last_change":"2024-03-06T07:22:34.361474+0000","last_active":"2024-03-06T07:22:51.811885+0000","last_peered":"2024-03-06T07:22:51.811885+0000","last_clean":"2024-03-06T07:22:51.811885+0000","last_became_active":"2024-03-06T07:22:34.360526+0000","last_became_peered":"2024-03-06T07:22:34.360526+0000","last_unstale":"2024-03-06T07:22:51.811885+0000","last_undegraded":"2024-03-06T07:22:51.811885+0000","last_fullsized":"2024-03-06T07:22:51.811885+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809784+0000","last_change":"2024-03-06T07:22:50.443740+0000","last_active":"2024-03-06T07:22:51.809784+0000","last_peered":"2024-03-06T07:22:51.809784+0000","last_clean":"2024-03-06T07:22:51.809784+0000","last_became_active":"2024-03-06T07:22:50.443428+0000","last_became_peered":"2024-03-06T07:22:50.443428+0000","last_unstale":"2024-03-06T07:22:51.809784+0000","last_undegraded":"2024-03-06T07:22:51.809784+0000","last_fullsized":"2024-03-06T07:22:51.809784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:44.031396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810137+0000","last_change":"2024-03-06T07:22:49.442302+0000","last_active":"2024-03-06T07:22:51.810137+0000","last_peered":"2024-03-06T07:22:51.810137+0000","last_clean":"2024-03-06T07:22:51.810137+0000","last_became_active":"2024-03-06T07:22:49.442038+0000","last_became_peered":"2024-03-06T07:22:49.442038+0000","last_unstale":"2024-03-06T07:22:51.810137+0000","last_undegraded":"2024-03-06T07:22:51.810137+0000","last_fullsized":"2024-03-06T07:22:51.810137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:04.439984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809946+0000","last_change":"2024-03-06T07:22:35.371218+0000","last_active":"2024-03-06T07:22:51.809946+0000","last_peered":"2024-03-06T07:22:51.809946+0000","last_clean":"2024-03-06T07:22:51.809946+0000","last_became_active":"2024-03-06T07:22:35.370932+0000","last_became_peered":"2024-03-06T07:22:35.370932+0000","last_unstale":"2024-03-06T07:22:51.809946+0000","last_undegraded":"2024-03-06T07:22:51.809946+0000","last_fullsized":"2024-03-06T07:22:51.809946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:06.203258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810097+0000","last_change":"2024-03-06T07:22:34.365404+0000","last_active":"2024-03-06T07:22:51.810097+0000","last_peered":"2024-03-06T07:22:51.810097+0000","last_clean":"2024-03-06T07:22:51.810097+0000","last_became_active":"2024-03-06T07:22:34.365029+0000","last_became_peered":"2024-03-06T07:22:34.365029+0000","last_unstale":"2024-03-06T07:22:51.810097+0000","last_undegraded":"2024-03-06T07:22:51.810097+0000","last_fullsized":"2024-03-06T07:22:51.810097+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:43:53.473973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812499+0000","last_change":"2024-03-06T07:22:50.454823+0000","last_active":"2024-03-06T07:22:51.812499+0000","last_peered":"2024-03-06T07:22:51.812499+0000","last_clean":"2024-03-06T07:22:51.812499+0000","last_became_active":"2024-03-06T07:22:50.454554+0000","last_became_peered":"2024-03-06T07:22:50.454554+0000","last_unstale":"2024-03-06T07:22:51.812499+0000","last_undegraded":"2024-03-06T07:22:51.812499+0000","last_fullsized":"2024-03-06T07:22:51.812499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:22.682632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810092+0000","last_change":"2024-03-06T07:22:49.439149+0000","last_active":"2024-03-06T07:22:51.810092+0000","last_peered":"2024-03-06T07:22:51.810092+0000","last_clean":"2024-03-06T07:22:51.810092+0000","last_became_active":"2024-03-06T07:22:49.438863+0000","last_became_peered":"2024-03-06T07:22:49.438863+0000","last_unstale":"2024-03-06T07:22:51.810092+0000","last_undegraded":"2024-03-06T07:22:51.810092+0000","last_fullsized":"2024-03-06T07:22:51.810092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:10.816265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809765+0000","last_change":"2024-03-06T07:22:35.368823+0000","last_active":"2024-03-06T07:22:51.809765+0000","last_peered":"2024-03-06T07:22:51.809765+0000","last_clean":"2024-03-06T07:22:51.809765+0000","last_became_active":"2024-03-06T07:22:35.368257+0000","last_became_peered":"2024-03-06T07:22:35.368257+0000","last_unstale":"2024-03-06T07:22:51.809765+0000","last_undegraded":"2024-03-06T07:22:51.809765+0000","last_fullsized":"2024-03-06T07:22:51.809765+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:36.670853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430933+0000","last_change":"2024-03-06T07:22:34.358630+0000","last_active":"2024-03-06T07:22:51.430933+0000","last_peered":"2024-03-06T07:22:51.430933+0000","last_clean":"2024-03-06T07:22:51.430933+0000","last_became_active":"2024-03-06T07:22:34.358355+0000","last_became_peered":"2024-03-06T07:22:34.358355+0000","last_unstale":"2024-03-06T07:22:51.430933+0000","last_undegraded":"2024-03-06T07:22:51.430933+0000","last_fullsized":"2024-03-06T07:22:51.430933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:17.469192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811433+0000","last_change":"2024-03-06T07:22:50.459509+0000","last_active":"2024-03-06T07:22:51.811433+0000","last_peered":"2024-03-06T07:22:51.811433+0000","last_clean":"2024-03-06T07:22:51.811433+0000","last_became_active":"2024-03-06T07:22:50.458980+0000","last_became_peered":"2024-03-06T07:22:50.458980+0000","last_unstale":"2024-03-06T07:22:51.811433+0000","last_undegraded":"2024-03-06T07:22:51.811433+0000","last_fullsized":"2024-03-06T07:22:51.811433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:09.219746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810745+0000","last_change":"2024-03-06T07:22:49.440502+0000","last_active":"2024-03-06T07:22:51.810745+0000","last_peered":"2024-03-06T07:22:51.810745+0000","last_clean":"2024-03-06T07:22:51.810745+0000","last_became_active":"2024-03-06T07:22:49.440225+0000","last_became_peered":"2024-03-06T07:22:49.440225+0000","last_unstale":"2024-03-06T07:22:51.810745+0000","last_undegraded":"2024-03-06T07:22:51.810745+0000","last_fullsized":"2024-03-06T07:22:51.810745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:33.183064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810488+0000","last_change":"2024-03-06T07:22:35.370822+0000","last_active":"2024-03-06T07:22:51.810488+0000","last_peered":"2024-03-06T07:22:51.810488+0000","last_clean":"2024-03-06T07:22:51.810488+0000","last_became_active":"2024-03-06T07:22:35.370527+0000","last_became_peered":"2024-03-06T07:22:35.370527+0000","last_unstale":"2024-03-06T07:22:51.810488+0000","last_undegraded":"2024-03-06T07:22:51.810488+0000","last_fullsized":"2024-03-06T07:22:51.810488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:58.803520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810893+0000","last_change":"2024-03-06T07:22:34.360261+0000","last_active":"2024-03-06T07:22:51.810893+0000","last_peered":"2024-03-06T07:22:51.810893+0000","last_clean":"2024-03-06T07:22:51.810893+0000","last_became_active":"2024-03-06T07:22:34.360028+0000","last_became_peered":"2024-03-06T07:22:34.360028+0000","last_unstale":"2024-03-06T07:22:51.810893+0000","last_undegraded":"2024-03-06T07:22:51.810893+0000","last_fullsized":"2024-03-06T07:22:51.810893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:30.196739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810044+0000","last_change":"2024-03-06T07:22:50.446528+0000","last_active":"2024-03-06T07:22:51.810044+0000","last_peered":"2024-03-06T07:22:51.810044+0000","last_clean":"2024-03-06T07:22:51.810044+0000","last_became_active":"2024-03-06T07:22:50.446004+0000","last_became_peered":"2024-03-06T07:22:50.446004+0000","last_unstale":"2024-03-06T07:22:51.810044+0000","last_undegraded":"2024-03-06T07:22:51.810044+0000","last_fullsized":"2024-03-06T07:22:51.810044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:16.949105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810375+0000","last_change":"2024-03-06T07:22:49.446813+0000","last_active":"2024-03-06T07:22:51.810375+0000","last_peered":"2024-03-06T07:22:51.810375+0000","last_clean":"2024-03-06T07:22:51.810375+0000","last_became_active":"2024-03-06T07:22:49.446544+0000","last_became_peered":"2024-03-06T07:22:49.446544+0000","last_unstale":"2024-03-06T07:22:51.810375+0000","last_undegraded":"2024-03-06T07:22:51.810375+0000","last_fullsized":"2024-03-06T07:22:51.810375+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:35:09.939716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810301+0000","last_change":"2024-03-06T07:22:35.368678+0000","last_active":"2024-03-06T07:22:51.810301+0000","last_peered":"2024-03-06T07:22:51.810301+0000","last_clean":"2024-03-06T07:22:51.810301+0000","last_became_active":"2024-03-06T07:22:35.368420+0000","last_became_peered":"2024-03-06T07:22:35.368420+0000","last_unstale":"2024-03-06T07:22:51.810301+0000","last_undegraded":"2024-03-06T07:22:51.810301+0000","last_fullsized":"2024-03-06T07:22:51.810301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:31:16.368079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427806+0000","last_change":"2024-03-06T07:22:34.357772+0000","last_active":"2024-03-06T07:22:51.427806+0000","last_peered":"2024-03-06T07:22:51.427806+0000","last_clean":"2024-03-06T07:22:51.427806+0000","last_became_active":"2024-03-06T07:22:34.357506+0000","last_became_peered":"2024-03-06T07:22:34.357506+0000","last_unstale":"2024-03-06T07:22:51.427806+0000","last_undegraded":"2024-03-06T07:22:51.427806+0000","last_fullsized":"2024-03-06T07:22:51.427806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:34:15.153107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431066+0000","last_change":"2024-03-06T07:22:50.433988+0000","last_active":"2024-03-06T07:22:51.431066+0000","last_peered":"2024-03-06T07:22:51.431066+0000","last_clean":"2024-03-06T07:22:51.431066+0000","last_became_active":"2024-03-06T07:22:50.433697+0000","last_became_peered":"2024-03-06T07:22:50.433697+0000","last_unstale":"2024-03-06T07:22:51.431066+0000","last_undegraded":"2024-03-06T07:22:51.431066+0000","last_fullsized":"2024-03-06T07:22:51.431066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:56.658779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430381+0000","last_change":"2024-03-06T07:22:49.445836+0000","last_active":"2024-03-06T07:22:51.430381+0000","last_peered":"2024-03-06T07:22:51.430381+0000","last_clean":"2024-03-06T07:22:51.430381+0000","last_became_active":"2024-03-06T07:22:49.445539+0000","last_became_peered":"2024-03-06T07:22:49.445539+0000","last_unstale":"2024-03-06T07:22:51.430381+0000","last_undegraded":"2024-03-06T07:22:51.430381+0000","last_fullsized":"2024-03-06T07:22:51.430381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:46.923868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811731+0000","last_change":"2024-03-06T07:22:35.370112+0000","last_active":"2024-03-06T07:22:51.811731+0000","last_peered":"2024-03-06T07:22:51.811731+0000","last_clean":"2024-03-06T07:22:51.811731+0000","last_became_active":"2024-03-06T07:22:35.369845+0000","last_became_peered":"2024-03-06T07:22:35.369845+0000","last_unstale":"2024-03-06T07:22:51.811731+0000","last_undegraded":"2024-03-06T07:22:51.811731+0000","last_fullsized":"2024-03-06T07:22:51.811731+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:13.972240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810494+0000","last_change":"2024-03-06T07:22:34.355822+0000","last_active":"2024-03-06T07:22:51.810494+0000","last_peered":"2024-03-06T07:22:51.810494+0000","last_clean":"2024-03-06T07:22:51.810494+0000","last_became_active":"2024-03-06T07:22:34.355493+0000","last_became_peered":"2024-03-06T07:22:34.355493+0000","last_unstale":"2024-03-06T07:22:51.810494+0000","last_undegraded":"2024-03-06T07:22:51.810494+0000","last_fullsized":"2024-03-06T07:22:51.810494+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:40.617911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811486+0000","last_change":"2024-03-06T07:22:50.448243+0000","last_active":"2024-03-06T07:22:51.811486+0000","last_peered":"2024-03-06T07:22:51.811486+0000","last_clean":"2024-03-06T07:22:51.811486+0000","last_became_active":"2024-03-06T07:22:50.447419+0000","last_became_peered":"2024-03-06T07:22:50.447419+0000","last_unstale":"2024-03-06T07:22:51.811486+0000","last_undegraded":"2024-03-06T07:22:51.811486+0000","last_fullsized":"2024-03-06T07:22:51.811486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:15.509422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430711+0000","last_change":"2024-03-06T07:22:49.437483+0000","last_active":"2024-03-06T07:22:51.430711+0000","last_peered":"2024-03-06T07:22:51.430711+0000","last_clean":"2024-03-06T07:22:51.430711+0000","last_became_active":"2024-03-06T07:22:49.437192+0000","last_became_peered":"2024-03-06T07:22:49.437192+0000","last_unstale":"2024-03-06T07:22:51.430711+0000","last_undegraded":"2024-03-06T07:22:51.430711+0000","last_fullsized":"2024-03-06T07:22:51.430711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:08.738502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811711+0000","last_change":"2024-03-06T07:22:35.361865+0000","last_active":"2024-03-06T07:22:51.811711+0000","last_peered":"2024-03-06T07:22:51.811711+0000","last_clean":"2024-03-06T07:22:51.811711+0000","last_became_active":"2024-03-06T07:22:35.361332+0000","last_became_peered":"2024-03-06T07:22:35.361332+0000","last_unstale":"2024-03-06T07:22:51.811711+0000","last_undegraded":"2024-03-06T07:22:51.811711+0000","last_fullsized":"2024-03-06T07:22:51.811711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:07.962172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811474+0000","last_change":"2024-03-06T07:22:34.366814+0000","last_active":"2024-03-06T07:22:51.811474+0000","last_peered":"2024-03-06T07:22:51.811474+0000","last_clean":"2024-03-06T07:22:51.811474+0000","last_became_active":"2024-03-06T07:22:34.366449+0000","last_became_peered":"2024-03-06T07:22:34.366449+0000","last_unstale":"2024-03-06T07:22:51.811474+0000","last_undegraded":"2024-03-06T07:22:51.811474+0000","last_fullsized":"2024-03-06T07:22:51.811474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:01:22.437222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430317+0000","last_change":"2024-03-06T07:22:50.438243+0000","last_active":"2024-03-06T07:22:51.430317+0000","last_peered":"2024-03-06T07:22:51.430317+0000","last_clean":"2024-03-06T07:22:51.430317+0000","last_became_active":"2024-03-06T07:22:50.437984+0000","last_became_peered":"2024-03-06T07:22:50.437984+0000","last_unstale":"2024-03-06T07:22:51.430317+0000","last_undegraded":"2024-03-06T07:22:51.430317+0000","last_fullsized":"2024-03-06T07:22:51.430317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:05.445202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809873+0000","last_change":"2024-03-06T07:22:49.442138+0000","last_active":"2024-03-06T07:22:51.809873+0000","last_peered":"2024-03-06T07:22:51.809873+0000","last_clean":"2024-03-06T07:22:51.809873+0000","last_became_active":"2024-03-06T07:22:49.441875+0000","last_became_peered":"2024-03-06T07:22:49.441875+0000","last_unstale":"2024-03-06T07:22:51.809873+0000","last_undegraded":"2024-03-06T07:22:51.809873+0000","last_fullsized":"2024-03-06T07:22:51.809873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:55.653561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430878+0000","last_change":"2024-03-06T07:22:35.369964+0000","last_active":"2024-03-06T07:22:51.430878+0000","last_peered":"2024-03-06T07:22:51.430878+0000","last_clean":"2024-03-06T07:22:51.430878+0000","last_became_active":"2024-03-06T07:22:35.369704+0000","last_became_peered":"2024-03-06T07:22:35.369704+0000","last_unstale":"2024-03-06T07:22:51.430878+0000","last_undegraded":"2024-03-06T07:22:51.430878+0000","last_fullsized":"2024-03-06T07:22:51.430878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:03.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810092+0000","last_change":"2024-03-06T07:22:34.361453+0000","last_active":"2024-03-06T07:22:51.810092+0000","last_peered":"2024-03-06T07:22:51.810092+0000","last_clean":"2024-03-06T07:22:51.810092+0000","last_became_active":"2024-03-06T07:22:34.361176+0000","last_became_peered":"2024-03-06T07:22:34.361176+0000","last_unstale":"2024-03-06T07:22:51.810092+0000","last_undegraded":"2024-03-06T07:22:51.810092+0000","last_fullsized":"2024-03-06T07:22:51.810092+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:14.735253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430028+0000","last_change":"2024-03-06T07:22:50.437663+0000","last_active":"2024-03-06T07:22:51.430028+0000","last_peered":"2024-03-06T07:22:51.430028+0000","last_clean":"2024-03-06T07:22:51.430028+0000","last_became_active":"2024-03-06T07:22:50.437356+0000","last_became_peered":"2024-03-06T07:22:50.437356+0000","last_unstale":"2024-03-06T07:22:51.430028+0000","last_undegraded":"2024-03-06T07:22:51.430028+0000","last_fullsized":"2024-03-06T07:22:51.430028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:45.163362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429708+0000","last_change":"2024-03-06T07:22:49.434747+0000","last_active":"2024-03-06T07:22:51.429708+0000","last_peered":"2024-03-06T07:22:51.429708+0000","last_clean":"2024-03-06T07:22:51.429708+0000","last_became_active":"2024-03-06T07:22:49.434490+0000","last_became_peered":"2024-03-06T07:22:49.434490+0000","last_unstale":"2024-03-06T07:22:51.429708+0000","last_undegraded":"2024-03-06T07:22:51.429708+0000","last_fullsized":"2024-03-06T07:22:51.429708+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:55.696704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429189+0000","last_change":"2024-03-06T07:22:35.363333+0000","last_active":"2024-03-06T07:22:51.429189+0000","last_peered":"2024-03-06T07:22:51.429189+0000","last_clean":"2024-03-06T07:22:51.429189+0000","last_became_active":"2024-03-06T07:22:35.363029+0000","last_became_peered":"2024-03-06T07:22:35.363029+0000","last_unstale":"2024-03-06T07:22:51.429189+0000","last_undegraded":"2024-03-06T07:22:51.429189+0000","last_fullsized":"2024-03-06T07:22:51.429189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:31.006057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810288+0000","last_change":"2024-03-06T07:22:34.358001+0000","last_active":"2024-03-06T07:22:51.810288+0000","last_peered":"2024-03-06T07:22:51.810288+0000","last_clean":"2024-03-06T07:22:51.810288+0000","last_became_active":"2024-03-06T07:22:34.357527+0000","last_became_peered":"2024-03-06T07:22:34.357527+0000","last_unstale":"2024-03-06T07:22:51.810288+0000","last_undegraded":"2024-03-06T07:22:51.810288+0000","last_fullsized":"2024-03-06T07:22:51.810288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:09:54.323832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812604+0000","last_change":"2024-03-06T07:22:50.458608+0000","last_active":"2024-03-06T07:22:51.812604+0000","last_peered":"2024-03-06T07:22:51.812604+0000","last_clean":"2024-03-06T07:22:51.812604+0000","last_became_active":"2024-03-06T07:22:50.458335+0000","last_became_peered":"2024-03-06T07:22:50.458335+0000","last_unstale":"2024-03-06T07:22:51.812604+0000","last_undegraded":"2024-03-06T07:22:51.812604+0000","last_fullsized":"2024-03-06T07:22:51.812604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:59.630524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809454+0000","last_change":"2024-03-06T07:22:49.448683+0000","last_active":"2024-03-06T07:22:51.809454+0000","last_peered":"2024-03-06T07:22:51.809454+0000","last_clean":"2024-03-06T07:22:51.809454+0000","last_became_active":"2024-03-06T07:22:49.448519+0000","last_became_peered":"2024-03-06T07:22:49.448519+0000","last_unstale":"2024-03-06T07:22:51.809454+0000","last_undegraded":"2024-03-06T07:22:51.809454+0000","last_fullsized":"2024-03-06T07:22:51.809454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:57:57.349847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430022+0000","last_change":"2024-03-06T07:22:35.360607+0000","last_active":"2024-03-06T07:22:51.430022+0000","last_peered":"2024-03-06T07:22:51.430022+0000","last_clean":"2024-03-06T07:22:51.430022+0000","last_became_active":"2024-03-06T07:22:35.360170+0000","last_became_peered":"2024-03-06T07:22:35.360170+0000","last_unstale":"2024-03-06T07:22:51.430022+0000","last_undegraded":"2024-03-06T07:22:51.430022+0000","last_fullsized":"2024-03-06T07:22:51.430022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:47:54.620500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812020+0000","last_change":"2024-03-06T07:22:34.362718+0000","last_active":"2024-03-06T07:22:51.812020+0000","last_peered":"2024-03-06T07:22:51.812020+0000","last_clean":"2024-03-06T07:22:51.812020+0000","last_became_active":"2024-03-06T07:22:34.362448+0000","last_became_peered":"2024-03-06T07:22:34.362448+0000","last_unstale":"2024-03-06T07:22:51.812020+0000","last_undegraded":"2024-03-06T07:22:51.812020+0000","last_fullsized":"2024-03-06T07:22:51.812020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:51.927836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809763+0000","last_change":"2024-03-06T07:22:50.445363+0000","last_active":"2024-03-06T07:22:51.809763+0000","last_peered":"2024-03-06T07:22:51.809763+0000","last_clean":"2024-03-06T07:22:51.809763+0000","last_became_active":"2024-03-06T07:22:50.445076+0000","last_became_peered":"2024-03-06T07:22:50.445076+0000","last_unstale":"2024-03-06T07:22:51.809763+0000","last_undegraded":"2024-03-06T07:22:51.809763+0000","last_fullsized":"2024-03-06T07:22:51.809763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:01.849313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812351+0000","last_change":"2024-03-06T07:22:49.443106+0000","last_active":"2024-03-06T07:22:51.812351+0000","last_peered":"2024-03-06T07:22:51.812351+0000","last_clean":"2024-03-06T07:22:51.812351+0000","last_became_active":"2024-03-06T07:22:49.441939+0000","last_became_peered":"2024-03-06T07:22:49.441939+0000","last_unstale":"2024-03-06T07:22:51.812351+0000","last_undegraded":"2024-03-06T07:22:51.812351+0000","last_fullsized":"2024-03-06T07:22:51.812351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:01:07.000270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.429608+0000","last_change":"2024-03-06T07:22:35.370746+0000","last_active":"2024-03-06T07:22:51.429608+0000","last_peered":"2024-03-06T07:22:51.429608+0000","last_clean":"2024-03-06T07:22:51.429608+0000","last_became_active":"2024-03-06T07:22:35.370096+0000","last_became_peered":"2024-03-06T07:22:35.370096+0000","last_unstale":"2024-03-06T07:22:51.429608+0000","last_undegraded":"2024-03-06T07:22:51.429608+0000","last_fullsized":"2024-03-06T07:22:51.429608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:08:33.338271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810579+0000","last_change":"2024-03-06T07:22:34.365631+0000","last_active":"2024-03-06T07:22:51.810579+0000","last_peered":"2024-03-06T07:22:51.810579+0000","last_clean":"2024-03-06T07:22:51.810579+0000","last_became_active":"2024-03-06T07:22:34.364558+0000","last_became_peered":"2024-03-06T07:22:34.364558+0000","last_unstale":"2024-03-06T07:22:51.810579+0000","last_undegraded":"2024-03-06T07:22:51.810579+0000","last_fullsized":"2024-03-06T07:22:51.810579+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:37.167691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811425+0000","last_change":"2024-03-06T07:22:50.449436+0000","last_active":"2024-03-06T07:22:51.811425+0000","last_peered":"2024-03-06T07:22:51.811425+0000","last_clean":"2024-03-06T07:22:51.811425+0000","last_became_active":"2024-03-06T07:22:50.446209+0000","last_became_peered":"2024-03-06T07:22:50.446209+0000","last_unstale":"2024-03-06T07:22:51.811425+0000","last_undegraded":"2024-03-06T07:22:51.811425+0000","last_fullsized":"2024-03-06T07:22:51.811425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:04.259512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809638+0000","last_change":"2024-03-06T07:22:49.440025+0000","last_active":"2024-03-06T07:22:51.809638+0000","last_peered":"2024-03-06T07:22:51.809638+0000","last_clean":"2024-03-06T07:22:51.809638+0000","last_became_active":"2024-03-06T07:22:49.439779+0000","last_became_peered":"2024-03-06T07:22:49.439779+0000","last_unstale":"2024-03-06T07:22:51.809638+0000","last_undegraded":"2024-03-06T07:22:51.809638+0000","last_fullsized":"2024-03-06T07:22:51.809638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:20.277188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812653+0000","last_change":"2024-03-06T07:22:35.362565+0000","last_active":"2024-03-06T07:22:51.812653+0000","last_peered":"2024-03-06T07:22:51.812653+0000","last_clean":"2024-03-06T07:22:51.812653+0000","last_became_active":"2024-03-06T07:22:35.362296+0000","last_became_peered":"2024-03-06T07:22:35.362296+0000","last_unstale":"2024-03-06T07:22:51.812653+0000","last_undegraded":"2024-03-06T07:22:51.812653+0000","last_fullsized":"2024-03-06T07:22:51.812653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:03:19.109135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810605+0000","last_change":"2024-03-06T07:22:34.360939+0000","last_active":"2024-03-06T07:22:51.810605+0000","last_peered":"2024-03-06T07:22:51.810605+0000","last_clean":"2024-03-06T07:22:51.810605+0000","last_became_active":"2024-03-06T07:22:34.360665+0000","last_became_peered":"2024-03-06T07:22:34.360665+0000","last_unstale":"2024-03-06T07:22:51.810605+0000","last_undegraded":"2024-03-06T07:22:51.810605+0000","last_fullsized":"2024-03-06T07:22:51.810605+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:57.647438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811596+0000","last_change":"2024-03-06T07:22:50.447742+0000","last_active":"2024-03-06T07:22:51.811596+0000","last_peered":"2024-03-06T07:22:51.811596+0000","last_clean":"2024-03-06T07:22:51.811596+0000","last_became_active":"2024-03-06T07:22:50.447473+0000","last_became_peered":"2024-03-06T07:22:50.447473+0000","last_unstale":"2024-03-06T07:22:51.811596+0000","last_undegraded":"2024-03-06T07:22:51.811596+0000","last_fullsized":"2024-03-06T07:22:51.811596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:13.315238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430466+0000","last_change":"2024-03-06T07:22:49.444034+0000","last_active":"2024-03-06T07:22:51.430466+0000","last_peered":"2024-03-06T07:22:51.430466+0000","last_clean":"2024-03-06T07:22:51.430466+0000","last_became_active":"2024-03-06T07:22:49.443793+0000","last_became_peered":"2024-03-06T07:22:49.443793+0000","last_unstale":"2024-03-06T07:22:51.430466+0000","last_undegraded":"2024-03-06T07:22:51.430466+0000","last_fullsized":"2024-03-06T07:22:51.430466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:07.324544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430985+0000","last_change":"2024-03-06T07:22:35.364703+0000","last_active":"2024-03-06T07:22:51.430985+0000","last_peered":"2024-03-06T07:22:51.430985+0000","last_clean":"2024-03-06T07:22:51.430985+0000","last_became_active":"2024-03-06T07:22:35.364407+0000","last_became_peered":"2024-03-06T07:22:35.364407+0000","last_unstale":"2024-03-06T07:22:51.430985+0000","last_undegraded":"2024-03-06T07:22:51.430985+0000","last_fullsized":"2024-03-06T07:22:51.430985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:44.303007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430538+0000","last_change":"2024-03-06T07:22:34.366593+0000","last_active":"2024-03-06T07:22:51.430538+0000","last_peered":"2024-03-06T07:22:51.430538+0000","last_clean":"2024-03-06T07:22:51.430538+0000","last_became_active":"2024-03-06T07:22:34.366323+0000","last_became_peered":"2024-03-06T07:22:34.366323+0000","last_unstale":"2024-03-06T07:22:51.430538+0000","last_undegraded":"2024-03-06T07:22:51.430538+0000","last_fullsized":"2024-03-06T07:22:51.430538+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:47.719698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810456+0000","last_change":"2024-03-06T07:22:50.445006+0000","last_active":"2024-03-06T07:22:51.810456+0000","last_peered":"2024-03-06T07:22:51.810456+0000","last_clean":"2024-03-06T07:22:51.810456+0000","last_became_active":"2024-03-06T07:22:50.444747+0000","last_became_peered":"2024-03-06T07:22:50.444747+0000","last_unstale":"2024-03-06T07:22:51.810456+0000","last_undegraded":"2024-03-06T07:22:51.810456+0000","last_fullsized":"2024-03-06T07:22:51.810456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:43.376118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430914+0000","last_change":"2024-03-06T07:22:49.436674+0000","last_active":"2024-03-06T07:22:51.430914+0000","last_peered":"2024-03-06T07:22:51.430914+0000","last_clean":"2024-03-06T07:22:51.430914+0000","last_became_active":"2024-03-06T07:22:49.436417+0000","last_became_peered":"2024-03-06T07:22:49.436417+0000","last_unstale":"2024-03-06T07:22:51.430914+0000","last_undegraded":"2024-03-06T07:22:51.430914+0000","last_fullsized":"2024-03-06T07:22:51.430914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:21:56.306976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430437+0000","last_change":"2024-03-06T07:22:35.371497+0000","last_active":"2024-03-06T07:22:51.430437+0000","last_peered":"2024-03-06T07:22:51.430437+0000","last_clean":"2024-03-06T07:22:51.430437+0000","last_became_active":"2024-03-06T07:22:35.371207+0000","last_became_peered":"2024-03-06T07:22:35.371207+0000","last_unstale":"2024-03-06T07:22:51.430437+0000","last_undegraded":"2024-03-06T07:22:51.430437+0000","last_fullsized":"2024-03-06T07:22:51.430437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:31.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810552+0000","last_change":"2024-03-06T07:22:34.362092+0000","last_active":"2024-03-06T07:22:51.810552+0000","last_peered":"2024-03-06T07:22:51.810552+0000","last_clean":"2024-03-06T07:22:51.810552+0000","last_became_active":"2024-03-06T07:22:34.361783+0000","last_became_peered":"2024-03-06T07:22:34.361783+0000","last_unstale":"2024-03-06T07:22:51.810552+0000","last_undegraded":"2024-03-06T07:22:51.810552+0000","last_fullsized":"2024-03-06T07:22:51.810552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:17.877394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.431106+0000","last_change":"2024-03-06T07:22:50.441776+0000","last_active":"2024-03-06T07:22:51.431106+0000","last_peered":"2024-03-06T07:22:51.431106+0000","last_clean":"2024-03-06T07:22:51.431106+0000","last_became_active":"2024-03-06T07:22:50.441530+0000","last_became_peered":"2024-03-06T07:22:50.441530+0000","last_unstale":"2024-03-06T07:22:51.431106+0000","last_undegraded":"2024-03-06T07:22:51.431106+0000","last_fullsized":"2024-03-06T07:22:51.431106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:21.282406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809856+0000","last_change":"2024-03-06T07:22:49.442594+0000","last_active":"2024-03-06T07:22:51.809856+0000","last_peered":"2024-03-06T07:22:51.809856+0000","last_clean":"2024-03-06T07:22:51.809856+0000","last_became_active":"2024-03-06T07:22:49.442310+0000","last_became_peered":"2024-03-06T07:22:49.442310+0000","last_unstale":"2024-03-06T07:22:51.809856+0000","last_undegraded":"2024-03-06T07:22:51.809856+0000","last_fullsized":"2024-03-06T07:22:51.809856+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:50:03.166354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427812+0000","last_change":"2024-03-06T07:22:35.366353+0000","last_active":"2024-03-06T07:22:51.427812+0000","last_peered":"2024-03-06T07:22:51.427812+0000","last_clean":"2024-03-06T07:22:51.427812+0000","last_became_active":"2024-03-06T07:22:35.366069+0000","last_became_peered":"2024-03-06T07:22:35.366069+0000","last_unstale":"2024-03-06T07:22:51.427812+0000","last_undegraded":"2024-03-06T07:22:51.427812+0000","last_fullsized":"2024-03-06T07:22:51.427812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:43:31.942821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430131+0000","last_change":"2024-03-06T07:22:34.357971+0000","last_active":"2024-03-06T07:22:51.430131+0000","last_peered":"2024-03-06T07:22:51.430131+0000","last_clean":"2024-03-06T07:22:51.430131+0000","last_became_active":"2024-03-06T07:22:34.357690+0000","last_became_peered":"2024-03-06T07:22:34.357690+0000","last_unstale":"2024-03-06T07:22:51.430131+0000","last_undegraded":"2024-03-06T07:22:51.430131+0000","last_fullsized":"2024-03-06T07:22:51.430131+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:08:21.705956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.812206+0000","last_change":"2024-03-06T07:22:50.451967+0000","last_active":"2024-03-06T07:22:51.812206+0000","last_peered":"2024-03-06T07:22:51.812206+0000","last_clean":"2024-03-06T07:22:51.812206+0000","last_became_active":"2024-03-06T07:22:50.451701+0000","last_became_peered":"2024-03-06T07:22:50.451701+0000","last_unstale":"2024-03-06T07:22:51.812206+0000","last_undegraded":"2024-03-06T07:22:51.812206+0000","last_fullsized":"2024-03-06T07:22:51.812206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:59.388088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.430679+0000","last_change":"2024-03-06T07:22:49.439060+0000","last_active":"2024-03-06T07:22:51.430679+0000","last_peered":"2024-03-06T07:22:51.430679+0000","last_clean":"2024-03-06T07:22:51.430679+0000","last_became_active":"2024-03-06T07:22:49.438806+0000","last_became_peered":"2024-03-06T07:22:49.438806+0000","last_unstale":"2024-03-06T07:22:51.430679+0000","last_undegraded":"2024-03-06T07:22:51.430679+0000","last_fullsized":"2024-03-06T07:22:51.430679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:28.533777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427851+0000","last_change":"2024-03-06T07:22:35.369547+0000","last_active":"2024-03-06T07:22:51.427851+0000","last_peered":"2024-03-06T07:22:51.427851+0000","last_clean":"2024-03-06T07:22:51.427851+0000","last_became_active":"2024-03-06T07:22:35.369275+0000","last_became_peered":"2024-03-06T07:22:35.369275+0000","last_unstale":"2024-03-06T07:22:51.427851+0000","last_undegraded":"2024-03-06T07:22:51.427851+0000","last_fullsized":"2024-03-06T07:22:51.427851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:33:54.954958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.426813+0000","last_change":"2024-03-06T07:22:34.355793+0000","last_active":"2024-03-06T07:22:51.426813+0000","last_peered":"2024-03-06T07:22:51.426813+0000","last_clean":"2024-03-06T07:22:51.426813+0000","last_became_active":"2024-03-06T07:22:34.355534+0000","last_became_peered":"2024-03-06T07:22:34.355534+0000","last_unstale":"2024-03-06T07:22:51.426813+0000","last_undegraded":"2024-03-06T07:22:51.426813+0000","last_fullsized":"2024-03-06T07:22:51.426813+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:29:01.838052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809784+0000","last_change":"2024-03-06T07:22:50.455677+0000","last_active":"2024-03-06T07:22:51.809784+0000","last_peered":"2024-03-06T07:22:51.809784+0000","last_clean":"2024-03-06T07:22:51.809784+0000","last_became_active":"2024-03-06T07:22:50.455436+0000","last_became_peered":"2024-03-06T07:22:50.455436+0000","last_unstale":"2024-03-06T07:22:51.809784+0000","last_undegraded":"2024-03-06T07:22:51.809784+0000","last_fullsized":"2024-03-06T07:22:51.809784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:49.417787+0000","last_clean_scrub_stamp":"2024-03-06T07:22:49.417787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:08.478793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.427749+0000","last_change":"2024-03-06T07:22:49.441951+0000","last_active":"2024-03-06T07:22:51.427749+0000","last_peered":"2024-03-06T07:22:51.427749+0000","last_clean":"2024-03-06T07:22:51.427749+0000","last_became_active":"2024-03-06T07:22:49.441543+0000","last_became_peered":"2024-03-06T07:22:49.441543+0000","last_unstale":"2024-03-06T07:22:51.427749+0000","last_undegraded":"2024-03-06T07:22:51.427749+0000","last_fullsized":"2024-03-06T07:22:51.427749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:48.412569+0000","last_clean_scrub_stamp":"2024-03-06T07:22:48.412569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:49:55.696704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.810556+0000","last_change":"2024-03-06T07:22:35.373140+0000","last_active":"2024-03-06T07:22:51.810556+0000","last_peered":"2024-03-06T07:22:51.810556+0000","last_clean":"2024-03-06T07:22:51.810556+0000","last_became_active":"2024-03-06T07:22:35.372523+0000","last_became_peered":"2024-03-06T07:22:35.372523+0000","last_unstale":"2024-03-06T07:22:51.810556+0000","last_undegraded":"2024-03-06T07:22:51.810556+0000","last_fullsized":"2024-03-06T07:22:51.810556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:34.338640+0000","last_clean_scrub_stamp":"2024-03-06T07:22:34.338640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:30.084215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.811398+0000","last_change":"2024-03-06T07:22:34.360532+0000","last_active":"2024-03-06T07:22:51.811398+0000","last_peered":"2024-03-06T07:22:51.811398+0000","last_clean":"2024-03-06T07:22:51.811398+0000","last_became_active":"2024-03-06T07:22:34.360249+0000","last_became_peered":"2024-03-06T07:22:34.360249+0000","last_unstale":"2024-03-06T07:22:51.811398+0000","last_undegraded":"2024-03-06T07:22:51.811398+0000","last_fullsized":"2024-03-06T07:22:51.811398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:22:33.333775+0000","last_clean_scrub_stamp":"2024-03-06T07:22:33.333775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:04:53.659708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":280,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-03-06T07:22:51.809782+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:22:51.809782+0000","last_peered":"2024-03-06T07:22:51.809782+0000","last_clean":"2024-03-06T07:22:51.809782+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:22:51.809782+0000","last_undegraded":"2024-03-06T07:22:51.809782+0000","last_fullsized":"2024-03-06T07:22:51.809782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":1056835,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":999424,"data_stored":470217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":458874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":468265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":466971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":467131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":466571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1044480,"data_stored":1051526,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:22:53.821 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:22:53.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs refuse_standby_for_another_fs true 2024-03-06T07:22:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.956+0000 7f3596d7a640 1 Processor -- start 2024-03-06T07:22:53.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.956+0000 7f3596d7a640 1 -- start start 2024-03-06T07:22:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3596d7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3590155c30 0x7f359006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3596d7a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35900c5380 0x7f35900c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3596d7a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35900b0450 con 0x7f35900b0070 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3596d7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359006cfb0 con 0x7f3590155c30 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3596d7a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359006c020 con 0x7f35900c5380 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f358ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3590155c30 0x7f359006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f358ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3590155c30 0x7f359006ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38222/0 (socket says 172.21.15.17:38222) 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f358ffff640 1 -- 172.21.15.17:0/1514552252 learned_addr learned my addr 172.21.15.17:0/1514552252 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.957+0000 7f3595317640 1 --2- 172.21.15.17:0/1514552252 >> v2:172.21.15.163:3300/0 conn(0x7f35900c5380 0x7f35900c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358ffff640 1 -- 172.21.15.17:0/1514552252 >> v2:172.21.15.163:3300/0 conn(0x7f35900c5380 msgr2=0x7f35900c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358ffff640 1 --2- 172.21.15.17:0/1514552252 >> v2:172.21.15.163:3300/0 conn(0x7f35900c5380 0x7f35900c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358ffff640 1 -- 172.21.15.17:0/1514552252 >> v1:172.21.15.163:6789/0 conn(0x7f35900b0070 legacy=0x7f35900c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358ffff640 1 -- 172.21.15.17:0/1514552252 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35901560e0 con 0x7f3590155c30 2024-03-06T07:22:53.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358ffff640 1 --2- 172.21.15.17:0/1514552252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3590155c30 0x7f359006ca70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f35900be030 tx=0x7f357c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbacb76e2207d70 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.958+0000 7f358f7fe640 1 -- 172.21.15.17:0/1514552252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c035020 con 0x7f3590155c30 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f358f7fe640 1 -- 172.21.15.17:0/1514552252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c031870 con 0x7f3590155c30 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f358f7fe640 1 -- 172.21.15.17:0/1514552252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c031c40 con 0x7f3590155c30 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 -- 172.21.15.17:0/1514552252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3590155c30 msgr2=0x7f359006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 --2- 172.21.15.17:0/1514552252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3590155c30 0x7f359006ca70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f35900be030 tx=0x7f357c02b910 comp rx=0 tx=0).stop 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 -- 172.21.15.17:0/1514552252 shutdown_connections 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 --2- 172.21.15.17:0/1514552252 >> v2:172.21.15.163:3300/0 conn(0x7f35900c5380 0x7f35900c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 --2- 172.21.15.17:0/1514552252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3590155c30 0x7f359006ca70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 -- 172.21.15.17:0/1514552252 >> 172.21.15.17:0/1514552252 conn(0x7f35900ac590 msgr2=0x7f35900ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 -- 172.21.15.17:0/1514552252 shutdown_connections 2024-03-06T07:22:53.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.959+0000 7f3596d7a640 1 -- 172.21.15.17:0/1514552252 wait complete. 2024-03-06T07:22:53.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.961+0000 7f3596d7a640 1 Processor -- start 2024-03-06T07:22:53.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.961+0000 7f3596d7a640 1 -- start start 2024-03-06T07:22:53.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35900c5380 0x7f35900c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3590155c30 0x7f35900c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359006c270 con 0x7f35900b0070 2024-03-06T07:22:53.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35900c7ad0 con 0x7f3590155c30 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3596d7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359006c550 con 0x7f35900c5380 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3594b16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3594b16640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38228/0 (socket says 172.21.15.17:38228) 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3594b16640 1 -- 172.21.15.17:0/3431824551 learned_addr learned my addr 172.21.15.17:0/3431824551 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f3595317640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3590155c30 0x7f35900c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.962+0000 7f358ffff640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35900c5380 0x7f35900c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 -- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35900c5380 msgr2=0x7f35900c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35900c5380 0x7f35900c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 -- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3590155c30 msgr2=0x7f35900c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3590155c30 0x7f35900c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 -- 172.21.15.17:0/3431824551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900c1860 con 0x7f35900b0070 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3594b16640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f358400c3f0 tx=0x7f358400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840052d0 con 0x7f35900b0070 2024-03-06T07:22:53.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584005490 con 0x7f35900b0070 2024-03-06T07:22:53.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35901eb8a0 con 0x7f35900b0070 2024-03-06T07:22:53.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584007260 con 0x7f35900b0070 2024-03-06T07:22:53.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.963+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3590158300 con 0x7f35900b0070 2024-03-06T07:22:53.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.964+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359004f330 con 0x7f35900b0070 2024-03-06T07:22:53.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.964+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3584007400 con 0x7f35900b0070 2024-03-06T07:22:53.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.965+0000 7f358d7fa640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 0x7f357406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:53.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.965+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f3584089a40 con 0x7f35900b0070 2024-03-06T07:22:53.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.965+0000 7f358ffff640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 0x7f357406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:53.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.966+0000 7f358ffff640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 0x7f357406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f357c036000 tx=0x7f357c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:53.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:53.969+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35900b0450 con 0x7f35900b0070 2024-03-06T07:22:54.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.213+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "refuse_standby_for_another_fs", "val": "true"} v 0) v1 -- 0x7f359006d860 con 0x7f35900b0070 2024-03-06T07:22:54.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.452+0000 7f358d7fa640 1 -- 172.21.15.17:0/3431824551 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "refuse_standby_for_another_fs", "val": "true"}]=0 set to refuse standby for another fs v60) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3584057e20 con 0x7f35900b0070 2024-03-06T07:22:54.454 INFO:teuthology.orchestra.run.smithi017.stderr:set to refuse standby for another fs 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.454+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 msgr2=0x7f357406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.454+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 0x7f357406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f357c036000 tx=0x7f357c03c040 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 msgr2=0x7f359014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f358400c3f0 tx=0x7f358400c8b0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 shutdown_connections 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f357406b4d0 0x7f357406d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3590155c30 0x7f35900c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35900c5380 0x7f35900c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 --2- 172.21.15.17:0/3431824551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35900b0070 0x7f359014e800 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 >> 172.21.15.17:0/3431824551 conn(0x7f35900ac590 msgr2=0x7f35900c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 shutdown_connections 2024-03-06T07:22:54.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.455+0000 7f3596d7a640 1 -- 172.21.15.17:0/3431824551 wait complete. 2024-03-06T07:22:54.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 5:0 2024-03-06T07:22:54.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.568+0000 7f765470c640 1 Processor -- start 2024-03-06T07:22:54.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.568+0000 7f765470c640 1 -- start start 2024-03-06T07:22:54.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f765470c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f764c0c8f20 0x7f764c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f765470c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0a8390 con 0x7f764c0c7170 2024-03-06T07:22:54.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f765470c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0730f0 con 0x7f764c06ff10 2024-03-06T07:22:54.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f765470c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0c7620 con 0x7f764c0c8f20 2024-03-06T07:22:54.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f7651ca7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f764c0c7170 0x7f764c0c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50390/0 (socket says 172.21.15.17:50390) 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f7651ca7640 1 -- 172.21.15.17:0/1905031460 learned_addr learned my addr 172.21.15.17:0/1905031460 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f7652ca9640 1 --2- 172.21.15.17:0/1905031460 >> v2:172.21.15.163:3300/0 conn(0x7f764c0c8f20 0x7f764c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.569+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305976172 0 0) 0x7f764c0a8390 con 0x7f764c0c7170 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7638003520 con 0x7f764c0c7170 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99205353 0 0) 0x7f764c0730f0 con 0x7f764c06ff10 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c0a8390 con 0x7f764c06ff10 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f7652ca9640 1 -- 172.21.15.17:0/1905031460 >> v1:172.21.15.163:6789/0 conn(0x7f764c06ff10 legacy=0x7f764c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f7652ca9640 1 -- 172.21.15.17:0/1905031460 >> v1:172.21.15.17:6789/0 conn(0x7f764c0c7170 legacy=0x7f764c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f7652ca9640 1 -- 172.21.15.17:0/1905031460 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0733d0 con 0x7f764c0c8f20 2024-03-06T07:22:54.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.570+0000 7f7652ca9640 1 --2- 172.21.15.17:0/1905031460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c15ebb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f764c0c77a0 tx=0x7f764402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a942720bee433e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644039070 con 0x7f764c0c8f20 2024-03-06T07:22:54.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764402cbc0 con 0x7f764c0c8f20 2024-03-06T07:22:54.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f76514a6640 1 -- 172.21.15.17:0/1905031460 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644034440 con 0x7f764c0c8f20 2024-03-06T07:22:54.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 -- 172.21.15.17:0/1905031460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 msgr2=0x7f764c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 --2- 172.21.15.17:0/1905031460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c15ebb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f764c0c77a0 tx=0x7f764402be10 comp rx=0 tx=0).stop 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 -- 172.21.15.17:0/1905031460 shutdown_connections 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 --2- 172.21.15.17:0/1905031460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c15ebb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 -- 172.21.15.17:0/1905031460 >> 172.21.15.17:0/1905031460 conn(0x7f764c06b970 msgr2=0x7f764c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 -- 172.21.15.17:0/1905031460 shutdown_connections 2024-03-06T07:22:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.571+0000 7f765470c640 1 -- 172.21.15.17:0/1905031460 wait complete. 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.572+0000 7f765470c640 1 Processor -- start 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.572+0000 7f765470c640 1 -- start start 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c0c7170 0x7f764c14c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1544b0 con 0x7f764c06ff10 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c154330 con 0x7f764c0c8f20 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f765470c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0702f0 con 0x7f764c0c7170 2024-03-06T07:22:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f76524a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f76524a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38230/0 (socket says 172.21.15.17:38230) 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f76524a8640 1 -- 172.21.15.17:0/1269898617 learned_addr learned my addr 172.21.15.17:0/1269898617 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f7652ca9640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f7651ca7640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c0c7170 0x7f764c14c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f76524a8640 1 -- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c0c7170 msgr2=0x7f764c14c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.573+0000 7f76524a8640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c0c7170 0x7f764c14c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f76524a8640 1 -- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 msgr2=0x7f764c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f76524a8640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f76524a8640 1 -- 172.21.15.17:0/1269898617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c1e86f0 con 0x7f764c06ff10 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f76524a8640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f764000c500 tx=0x7f764000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400052d0 con 0x7f764c06ff10 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7640005490 con 0x7f764c06ff10 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7640012870 con 0x7f764c06ff10 2024-03-06T07:22:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c0bde80 con 0x7f764c06ff10 2024-03-06T07:22:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.574+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1e8c70 con 0x7f764c06ff10 2024-03-06T07:22:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.575+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c04f330 con 0x7f764c06ff10 2024-03-06T07:22:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.575+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7640012a10 con 0x7f764c06ff10 2024-03-06T07:22:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.576+0000 7f763effd640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.576+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f764008a280 con 0x7f764c06ff10 2024-03-06T07:22:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.576+0000 7f7651ca7640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.576+0000 7f7651ca7640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f764c0a7e40 tx=0x7f7648007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:54.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.580+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76400585e0 con 0x7f764c06ff10 2024-03-06T07:22:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.957+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "5:0"} v 0) v1 -- 0x7f764c06fa80 con 0x7f764c06ff10 2024-03-06T07:22:54.959 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:22:54.958+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:22:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.959+0000 7f763effd640 1 -- 172.21.15.17:0/1269898617 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "5:0"}]=0 failed mds gid 6387 v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f764005c680 con 0x7f764c06ff10 2024-03-06T07:22:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 6387 2024-03-06T07:22:54.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.961+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 msgr2=0x7f763006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.961+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 0x7f763006d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f764c0a7e40 tx=0x7f7648007d30 comp rx=0 tx=0).stop 2024-03-06T07:22:54.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.961+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 msgr2=0x7f764c0bd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:54.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.961+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f764000c500 tx=0x7f764000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 shutdown_connections 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f763006b4d0 0x7f763006d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f764c0c8f20 0x7f764c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f764c0c7170 0x7f764c14c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 --2- 172.21.15.17:0/1269898617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f764c06ff10 0x7f764c0bd8a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 >> 172.21.15.17:0/1269898617 conn(0x7f764c06b970 msgr2=0x7f764c0c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 shutdown_connections 2024-03-06T07:22:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:54.962+0000 7f765470c640 1 -- 172.21.15.17:0/1269898617 wait complete. 2024-03-06T07:22:54.965 INFO:tasks.ceph.mds.b.smithi163.stderr: -503> 2024-03-06T07:22:40.639+0000 7fd5ec8fd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:54.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:54.988 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:22:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.081+0000 7efd6a9f6640 1 Processor -- start 2024-03-06T07:22:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 -- start start 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efd640c5480 0x7efd64155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5860 0x7efd6406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd641586f0 0x7efd64158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd6406df40 con 0x7efd641586f0 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640c5cc0 con 0x7efd640c5480 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd6a9f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd641591c0 con 0x7efd640c5860 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd68f93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd641586f0 0x7efd64158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd68f93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd641586f0 0x7efd64158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38240/0 (socket says 172.21.15.17:38240) 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd68f93640 1 -- 172.21.15.17:0/2302105548 learned_addr learned my addr 172.21.15.17:0/2302105548 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:55.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd637fe640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5860 0x7efd6406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.082+0000 7efd63fff640 1 --2- 172.21.15.17:0/2302105548 >> v2:172.21.15.163:3300/0 conn(0x7efd640c5480 0x7efd64155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.083+0000 7efd68f93640 1 -- 172.21.15.17:0/2302105548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5860 msgr2=0x7efd6406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.083+0000 7efd68f93640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5860 0x7efd6406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.083+0000 7efd68f93640 1 -- 172.21.15.17:0/2302105548 >> v2:172.21.15.163:3300/0 conn(0x7efd640c5480 msgr2=0x7efd64155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.083+0000 7efd68f93640 1 --2- 172.21.15.17:0/2302105548 >> v2:172.21.15.163:3300/0 conn(0x7efd640c5480 0x7efd64155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.083+0000 7efd68f93640 1 -- 172.21.15.17:0/2302105548 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd641594a0 con 0x7efd641586f0 2024-03-06T07:22:55.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd68f93640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd641586f0 0x7efd64158b80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efd640c1cf0 tx=0x7efd5c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479ff4fa0e9a0e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd62ffd640 1 -- 172.21.15.17:0/2302105548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c02c9e0 con 0x7efd641586f0 2024-03-06T07:22:55.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd62ffd640 1 -- 172.21.15.17:0/2302105548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c02cb80 con 0x7efd641586f0 2024-03-06T07:22:55.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd62ffd640 1 -- 172.21.15.17:0/2302105548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0343f0 con 0x7efd641586f0 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd6a9f6640 1 -- 172.21.15.17:0/2302105548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd641586f0 msgr2=0x7efd64158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.084+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd641586f0 0x7efd64158b80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efd640c1cf0 tx=0x7efd5c02be40 comp rx=0 tx=0).stop 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 -- 172.21.15.17:0/2302105548 shutdown_connections 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd641586f0 0x7efd64158b80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/2302105548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5860 0x7efd6406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/2302105548 >> v2:172.21.15.163:3300/0 conn(0x7efd640c5480 0x7efd64155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 -- 172.21.15.17:0/2302105548 >> 172.21.15.17:0/2302105548 conn(0x7efd6406b1a0 msgr2=0x7efd6406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 -- 172.21.15.17:0/2302105548 shutdown_connections 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.085+0000 7efd6a9f6640 1 -- 172.21.15.17:0/2302105548 wait complete. 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.086+0000 7efd6a9f6640 1 Processor -- start 2024-03-06T07:22:55.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.086+0000 7efd6a9f6640 1 -- start start 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5480 0x7efd6414f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 0x7efd641538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64152a20 con 0x7efd640c5860 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64155680 con 0x7efd641586f0 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd64155050 con 0x7efd640c5480 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38246/0 (socket says 172.21.15.17:38246) 2024-03-06T07:22:55.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 -- 172.21.15.17:0/3747442770 learned_addr learned my addr 172.21.15.17:0/3747442770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd68f93640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 0x7efd641538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd63fff640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5480 0x7efd6414f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 -- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5480 msgr2=0x7efd6414f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5480 0x7efd6414f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 -- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 msgr2=0x7efd641538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 0x7efd641538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 -- 172.21.15.17:0/3747442770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640a8270 con 0x7efd640c5860 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd68f93640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 0x7efd641538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd637fe640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9b0 tx=0x7efd5800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58017070 con 0x7efd640c5860 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5800c9a0 con 0x7efd640c5860 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800cd90 con 0x7efd640c5860 2024-03-06T07:22:55.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.087+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd640a7cb0 con 0x7efd640c5860 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.088+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd64153e30 con 0x7efd640c5860 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.089+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd641542f0 con 0x7efd640c5860 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.089+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd5801a070 con 0x7efd640c5860 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.089+0000 7efd60ff9640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.089+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7efd5808a660 con 0x7efd640c5860 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.090+0000 7efd63fff640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.090+0000 7efd63fff640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 0x7efd5006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efd54009a10 tx=0x7efd54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.094+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd641542f0 con 0x7efd640c5860 2024-03-06T07:22:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.309+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd6404f330 con 0x7efd640c5860 2024-03-06T07:22:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.310+0000 7efd60ff9640 1 -- 172.21.15.17:0/3747442770 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5453 (secure 0 0 0) 0x7efd58059030 con 0x7efd640c5860 2024-03-06T07:22:55.312 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:55.312 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":61,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":61,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:54.959114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:22:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 61 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 msgr2=0x7efd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 0x7efd5006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efd54009a10 tx=0x7efd54008040 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 msgr2=0x7efd640acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9b0 tx=0x7efd5800ee70 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 shutdown_connections 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd5006b4d0 0x7efd5006d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd641586f0 0x7efd641538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd640c5860 0x7efd640acf60 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 --2- 172.21.15.17:0/3747442770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd640c5480 0x7efd6414f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 >> 172.21.15.17:0/3747442770 conn(0x7efd6406b1a0 msgr2=0x7efd6406ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 shutdown_connections 2024-03-06T07:22:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.312+0000 7efd6a9f6640 1 -- 172.21.15.17:0/3747442770 wait complete. 2024-03-06T07:22:55.328 INFO:tasks.cephfs.test_failover:Waiting for former active to reclaim its spot 2024-03-06T07:22:55.328 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-03-06T07:22:55.435 INFO:teuthology.orchestra.run.smithi017.stdout:{"mds_beacon_grace":"15.000000"} 2024-03-06T07:22:55.436 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-03-06T07:22:55.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:22:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.537+0000 7f23614b5640 1 Processor -- start 2024-03-06T07:22:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.537+0000 7f23614b5640 1 -- start start 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f23614b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f235c158db0 0x7f235c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f23614b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f235c0c88b0 0x7f235c15fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f23614b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c155110 con 0x7f235c0c5330 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f23614b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c1559e0 con 0x7f235c158db0 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f23614b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c0c8f40 con 0x7f235c0c88b0 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f235c158db0 0x7f235c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f235c158db0 0x7f235c0c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38252/0 (socket says 172.21.15.17:38252) 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 -- 172.21.15.17:0/245022234 learned_addr learned my addr 172.21.15.17:0/245022234 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f235b7fe640 1 --2- 172.21.15.17:0/245022234 >> v2:172.21.15.163:3300/0 conn(0x7f235c0c88b0 0x7f235c15fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 -- 172.21.15.17:0/245022234 >> v2:172.21.15.163:3300/0 conn(0x7f235c0c88b0 msgr2=0x7f235c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 --2- 172.21.15.17:0/245022234 >> v2:172.21.15.163:3300/0 conn(0x7f235c0c88b0 0x7f235c15fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 -- 172.21.15.17:0/245022234 >> v1:172.21.15.163:6789/0 conn(0x7f235c0c5330 legacy=0x7f235c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.538+0000 7f2353fff640 1 -- 172.21.15.17:0/245022234 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c155390 con 0x7f235c158db0 2024-03-06T07:22:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.539+0000 7f2353fff640 1 --2- 172.21.15.17:0/245022234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c158db0 0x7f235c0c5710 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f235c0ca870 tx=0x7f234402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0c4ce89ce9d3c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f235a7fc640 1 -- 172.21.15.17:0/245022234 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344035020 con 0x7f235c158db0 2024-03-06T07:22:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f235a7fc640 1 -- 172.21.15.17:0/245022234 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2344031870 con 0x7f235c158db0 2024-03-06T07:22:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f235a7fc640 1 -- 172.21.15.17:0/245022234 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344031c40 con 0x7f235c158db0 2024-03-06T07:22:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 -- 172.21.15.17:0/245022234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c158db0 msgr2=0x7f235c0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 --2- 172.21.15.17:0/245022234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c158db0 0x7f235c0c5710 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f235c0ca870 tx=0x7f234402b910 comp rx=0 tx=0).stop 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 -- 172.21.15.17:0/245022234 shutdown_connections 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 --2- 172.21.15.17:0/245022234 >> v2:172.21.15.163:3300/0 conn(0x7f235c0c88b0 0x7f235c15fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 --2- 172.21.15.17:0/245022234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c158db0 0x7f235c0c5710 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 -- 172.21.15.17:0/245022234 >> 172.21.15.17:0/245022234 conn(0x7f235c0ac560 msgr2=0x7f235c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 -- 172.21.15.17:0/245022234 shutdown_connections 2024-03-06T07:22:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.540+0000 7f23614b5640 1 -- 172.21.15.17:0/245022234 wait complete. 2024-03-06T07:22:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.541+0000 7f23614b5640 1 Processor -- start 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.541+0000 7f23614b5640 1 -- start start 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f235c0c5330 0x7f235c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 0x7f235c1ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c1529d0 con 0x7f235c0c88b0 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c06d090 con 0x7f235c0c1fc0 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f23614b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c152b50 con 0x7f235c0c5330 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38262/0 (socket says 172.21.15.17:38262) 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 -- 172.21.15.17:0/2767011671 learned_addr learned my addr 172.21.15.17:0/2767011671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f235b7fe640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 0x7f235c1ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 -- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f235c0c5330 msgr2=0x7f235c14e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f235c0c5330 0x7f235c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 -- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 msgr2=0x7f235c1ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 0x7f235c1ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f2353fff640 1 -- 172.21.15.17:0/2767011671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c158140 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.542+0000 7f235b7fe640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 0x7f235c1ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f2353fff640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f234402b950 tx=0x7f23440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234403ad60 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23440443f0 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23440447e0 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c158420 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.543+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c156550 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.544+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c156880 con 0x7f235c0c88b0 2024-03-06T07:22:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.544+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2344038020 con 0x7f235c0c88b0 2024-03-06T07:22:55.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.545+0000 7f23537fe640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:22:55.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.545+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f23440ac7b0 con 0x7f235c0c88b0 2024-03-06T07:22:55.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.545+0000 7f235affd640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:22:55.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.545+0000 7f235affd640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f234c000f60 tx=0x7f234c0079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:22:55.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.553+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f234403f070 con 0x7f235c0c88b0 2024-03-06T07:22:55.641 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:22:55.640+0000 7fad48ca0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:22:55.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.775+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f235c04f330 con 0x7f235c0c88b0 2024-03-06T07:22:55.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.775+0000 7f23537fe640 1 -- 172.21.15.17:0/2767011671 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5453 (secure 0 0 0) 0x7f234407b180 con 0x7f235c0c88b0 2024-03-06T07:22:55.777 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:22:55.777 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":61,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":61,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:54.959114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:22:55.777 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 61 2024-03-06T07:22:55.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 msgr2=0x7f233c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 0x7f233c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f234c000f60 tx=0x7f234c0079d0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 msgr2=0x7f235c0c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:22:55.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f234402b950 tx=0x7f23440046c0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 shutdown_connections 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233c06b4d0 0x7f233c06d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f235c0c1fc0 0x7f235c1ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f235c0c88b0 0x7f235c0c1a80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 --2- 172.21.15.17:0/2767011671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f235c0c5330 0x7f235c14e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 >> 172.21.15.17:0/2767011671 conn(0x7f235c0ac560 msgr2=0x7f235c14f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 shutdown_connections 2024-03-06T07:22:55.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:22:55.777+0000 7f23614b5640 1 -- 172.21.15.17:0/2767011671 wait complete. 2024-03-06T07:22:55.790 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-03-06T07:23:00.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.891+0000 7fc76189d640 1 Processor -- start 2024-03-06T07:23:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.891+0000 7fc76189d640 1 -- start start 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc76189d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc75c156150 0x7fc75c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc76189d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c0c28d0 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc76189d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c0c2bb0 con 0x7fc75c14f790 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc76189d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c0c2680 con 0x7fc75c156150 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc75c156150 0x7fc75c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75a7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc75c0c1aa0 0x7fc75c155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58214/0 (socket says 172.21.15.17:58214) 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75a7fc640 1 -- 172.21.15.17:0/1896111961 learned_addr learned my addr 172.21.15.17:0/1896111961 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75b7fe640 1 -- 172.21.15.17:0/1896111961 >> v1:172.21.15.163:6789/0 conn(0x7fc75c14f790 legacy=0x7fc75c0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75b7fe640 1 -- 172.21.15.17:0/1896111961 >> v1:172.21.15.17:6789/0 conn(0x7fc75c0c1aa0 legacy=0x7fc75c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.892+0000 7fc75b7fe640 1 -- 172.21.15.17:0/1896111961 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0c2e90 con 0x7fc75c156150 2024-03-06T07:23:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc75b7fe640 1 --2- 172.21.15.17:0/1896111961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c156150 0x7fc75c1529d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0c2800 tx=0x7fc75402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809a7a70b951457a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc759ffb640 1 -- 172.21.15.17:0/1896111961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754039070 con 0x7fc75c156150 2024-03-06T07:23:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc759ffb640 1 -- 172.21.15.17:0/1896111961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75402cce0 con 0x7fc75c156150 2024-03-06T07:23:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc759ffb640 1 -- 172.21.15.17:0/1896111961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc754034660 con 0x7fc75c156150 2024-03-06T07:23:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc76189d640 1 -- 172.21.15.17:0/1896111961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c156150 msgr2=0x7fc75c1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.893+0000 7fc76189d640 1 --2- 172.21.15.17:0/1896111961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c156150 0x7fc75c1529d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0c2800 tx=0x7fc75402bf30 comp rx=0 tx=0).stop 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.894+0000 7fc76189d640 1 -- 172.21.15.17:0/1896111961 shutdown_connections 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.894+0000 7fc76189d640 1 --2- 172.21.15.17:0/1896111961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c156150 0x7fc75c1529d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.894+0000 7fc76189d640 1 -- 172.21.15.17:0/1896111961 >> 172.21.15.17:0/1896111961 conn(0x7fc75c0c6710 msgr2=0x7fc75c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.894+0000 7fc76189d640 1 -- 172.21.15.17:0/1896111961 shutdown_connections 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.894+0000 7fc76189d640 1 -- 172.21.15.17:0/1896111961 wait complete. 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.895+0000 7fc76189d640 1 Processor -- start 2024-03-06T07:23:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.895+0000 7fc76189d640 1 -- start start 2024-03-06T07:23:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc75c14f790 0x7fc75c1e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc75c156150 0x7fc75c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0c5e30 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0c0d60 con 0x7fc75c14f790 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0c5fb0 con 0x7fc75c156150 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35490/0 (socket says 172.21.15.17:35490) 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 -- 172.21.15.17:0/1450902973 learned_addr learned my addr 172.21.15.17:0/1450902973 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75a7fc640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc75c14f790 0x7fc75c1e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75b7fe640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc75c156150 0x7fc75c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 -- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc75c156150 msgr2=0x7fc75c14b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc75c156150 0x7fc75c14b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 -- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc75c14f790 msgr2=0x7fc75c1e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc75c14f790 0x7fc75c1e9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 -- 172.21.15.17:0/1450902973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c1ecba0 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc75affd640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c460 tx=0x7fc74400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744016020 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7440052f0 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c1e8c00 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744007260 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.896+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c153630 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.897+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c152720 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.898+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc744007400 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.898+0000 7fc753fff640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 0x7fc73c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.898+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fc744089ef0 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.899+0000 7fc75a7fc640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 0x7fc73c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.899+0000 7fc75a7fc640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 0x7fc73c06d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc75c1dc0e0 tx=0x7fc74c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:00.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:00.903+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc75c152720 con 0x7fc75c0c1aa0 2024-03-06T07:23:00.973 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:23:00.971+0000 7faebf760640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:01.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.249+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc75c04f330 con 0x7fc75c0c1aa0 2024-03-06T07:23:01.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.249+0000 7fc753fff640 1 -- 172.21.15.17:0/1450902973 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6263 (secure 0 0 0) 0x7fc744020020 con 0x7fc75c0c1aa0 2024-03-06T07:23:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:01.251 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:58.973213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6427":{"gid":6427,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6836/1549441863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1549441863},{"type":"v1","addr":"172.21.15.163:6836","nonce":1549441863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6427,"qdb_cluster":[6427]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:23:01.251 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 66 2024-03-06T07:23:01.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.251+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 msgr2=0x7fc73c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.251+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 0x7fc73c06d9c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc75c1dc0e0 tx=0x7fc74c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:01.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.251+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 msgr2=0x7fc75c154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.251+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc74400c460 tx=0x7fc74400c920 comp rx=0 tx=0).stop 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 shutdown_connections 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73c06b500 0x7fc73c06d9c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc75c156150 0x7fc75c14b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc75c14f790 0x7fc75c1e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 --2- 172.21.15.17:0/1450902973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc75c0c1aa0 0x7fc75c154230 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 >> 172.21.15.17:0/1450902973 conn(0x7fc75c0c6710 msgr2=0x7fc75c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 shutdown_connections 2024-03-06T07:23:01.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.252+0000 7fc76189d640 1 -- 172.21.15.17:0/1450902973 wait complete. 2024-03-06T07:23:01.264 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:23:01.264 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:01.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:01.309 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:01.355 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:01.355 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:23:01.355 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:01.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:01.372 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:23:01.372 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:01.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:01.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:01.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.550+0000 7f5a89bd0640 1 Processor -- start 2024-03-06T07:23:01.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.550+0000 7f5a89bd0640 1 -- start start 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a89bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a840c1aa0 0x7f5a841529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a89bd0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840c28d0 con 0x7f5a84155a40 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a89bd0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a840c2bb0 con 0x7f5a8414f790 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a89bd0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a840c2680 con 0x7f5a840c1aa0 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a82ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5a840c1aa0 0x7f5a841529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a83fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5a84155a40 0x7f5a84155e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58222/0 (socket says 172.21.15.17:58222) 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a83fff640 1 -- 172.21.15.17:0/1693301170 learned_addr learned my addr 172.21.15.17:0/1693301170 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a82ffd640 1 -- 172.21.15.17:0/1693301170 >> v1:172.21.15.163:6789/0 conn(0x7f5a8414f790 legacy=0x7f5a840c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a82ffd640 1 -- 172.21.15.17:0/1693301170 >> v1:172.21.15.17:6789/0 conn(0x7f5a84155a40 legacy=0x7f5a84155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.551+0000 7f5a82ffd640 1 -- 172.21.15.17:0/1693301170 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840c2e90 con 0x7f5a840c1aa0 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.552+0000 7f5a82ffd640 1 --2- 172.21.15.17:0/1693301170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a840c1aa0 0x7f5a841529d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5a840c2800 tx=0x7f5a7402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3009ae785714cce8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.552+0000 7f5a827fc640 1 -- 172.21.15.17:0/1693301170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74034070 con 0x7f5a840c1aa0 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.552+0000 7f5a827fc640 1 -- 172.21.15.17:0/1693301170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74030610 con 0x7f5a840c1aa0 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.552+0000 7f5a827fc640 1 -- 172.21.15.17:0/1693301170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a740309e0 con 0x7f5a840c1aa0 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 -- 172.21.15.17:0/1693301170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a840c1aa0 msgr2=0x7f5a841529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/1693301170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a840c1aa0 0x7f5a841529d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5a840c2800 tx=0x7f5a7402b960 comp rx=0 tx=0).stop 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 -- 172.21.15.17:0/1693301170 shutdown_connections 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/1693301170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a840c1aa0 0x7f5a841529d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 -- 172.21.15.17:0/1693301170 >> 172.21.15.17:0/1693301170 conn(0x7f5a840c6710 msgr2=0x7f5a840c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 -- 172.21.15.17:0/1693301170 shutdown_connections 2024-03-06T07:23:01.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.553+0000 7f5a89bd0640 1 -- 172.21.15.17:0/1693301170 wait complete. 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.554+0000 7f5a89bd0640 1 Processor -- start 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.554+0000 7f5a89bd0640 1 -- start start 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a840c1aa0 0x7f5a840a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8414f790 0x7f5a84153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840c5e30 con 0x7f5a84155a40 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840c0d60 con 0x7f5a8414f790 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a840c5fb0 con 0x7f5a840c1aa0 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35502/0 (socket says 172.21.15.17:35502) 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 -- 172.21.15.17:0/3478635944 learned_addr learned my addr 172.21.15.17:0/3478635944 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a82ffd640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8414f790 0x7f5a84153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a837fe640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a840c1aa0 0x7f5a840a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 -- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a840c1aa0 msgr2=0x7f5a840a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a840c1aa0 0x7f5a840a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 -- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8414f790 msgr2=0x7f5a84153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8414f790 0x7f5a84153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 -- 172.21.15.17:0/3478635944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a841eefa0 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a83fff640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00ca80 tx=0x7f5a7c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c017070 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7c00ac30 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a841eecc0 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c012600 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.555+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8414d5b0 con 0x7f5a84155a40 2024-03-06T07:23:01.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.556+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a840a7c90 con 0x7f5a84155a40 2024-03-06T07:23:01.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.557+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a7c01a070 con 0x7f5a84155a40 2024-03-06T07:23:01.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.557+0000 7f5a7bfff640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.557+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f5a7c08a310 con 0x7f5a84155a40 2024-03-06T07:23:01.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.557+0000 7f5a837fe640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.558+0000 7f5a837fe640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 0x7f5a6406d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001780 tx=0x7f5a6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.562+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a7c00ae20 con 0x7f5a84155a40 2024-03-06T07:23:01.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.782+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a8404f330 con 0x7f5a84155a40 2024-03-06T07:23:01.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.783+0000 7f5a7bfff640 1 -- 172.21.15.17:0/3478635944 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f5a7c058610 con 0x7f5a84155a40 2024-03-06T07:23:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:01.787 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:58.973213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6427":{"gid":6427,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6836/1549441863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1549441863},{"type":"v1","addr":"172.21.15.163:6836","nonce":1549441863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6427,"qdb_cluster":[6427]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 66 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 msgr2=0x7f5a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 0x7f5a6406d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001780 tx=0x7f5a6c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 msgr2=0x7f5a84153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00ca80 tx=0x7f5a7c00cf40 comp rx=0 tx=0).stop 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 shutdown_connections 2024-03-06T07:23:01.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a84155a40 0x7f5a84153f90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a8414f790 0x7f5a84153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 --2- 172.21.15.17:0/3478635944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a840c1aa0 0x7f5a840a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 >> 172.21.15.17:0/3478635944 conn(0x7f5a840c6710 msgr2=0x7f5a840c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 shutdown_connections 2024-03-06T07:23:01.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.785+0000 7f5a89bd0640 1 -- 172.21.15.17:0/3478635944 wait complete. 2024-03-06T07:23:01.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:01.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.897+0000 7f43cfbe9640 1 Processor -- start 2024-03-06T07:23:01.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.898+0000 7f43cfbe9640 1 -- start start 2024-03-06T07:23:01.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cfbe9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43c814f360 0x7f43c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cfbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 0x7f43c8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cfbe9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43c80c73d0 con 0x7f43c80c88c0 2024-03-06T07:23:01.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cfbe9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43c80c8dd0 con 0x7f43c814f360 2024-03-06T07:23:01.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cfbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80be2e0 con 0x7f43c8158b60 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cd184640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f43c80c88c0 0x7f43c80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58226/0 (socket says 172.21.15.17:58226) 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cd184640 1 -- 172.21.15.17:0/1118452160 learned_addr learned my addr 172.21.15.17:0/1118452160 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43ce186640 1 --2- 172.21.15.17:0/1118452160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 0x7f43c8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cd985640 1 --2- 172.21.15.17:0/1118452160 >> v2:172.21.15.163:3300/0 conn(0x7f43c814f360 0x7f43c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cc983640 1 -- 172.21.15.17:0/1118452160 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069081120 0 0) 0x7f43c80c73d0 con 0x7f43c80c88c0 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43cc983640 1 -- 172.21.15.17:0/1118452160 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b4003520 con 0x7f43c80c88c0 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43ce186640 1 -- 172.21.15.17:0/1118452160 >> v2:172.21.15.163:3300/0 conn(0x7f43c814f360 msgr2=0x7f43c8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43ce186640 1 --2- 172.21.15.17:0/1118452160 >> v2:172.21.15.163:3300/0 conn(0x7f43c814f360 0x7f43c8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43ce186640 1 -- 172.21.15.17:0/1118452160 >> v1:172.21.15.17:6789/0 conn(0x7f43c80c88c0 legacy=0x7f43c80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.899+0000 7f43ce186640 1 -- 172.21.15.17:0/1118452160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80c7670 con 0x7f43c8158b60 2024-03-06T07:23:01.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.900+0000 7f43ce186640 1 --2- 172.21.15.17:0/1118452160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 0x7f43c8158ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097a0 tx=0x7f43c002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0e2329c418184d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.900+0000 7f43cc983640 1 -- 172.21.15.17:0/1118452160 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0031780 con 0x7f43c8158b60 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.900+0000 7f43cc983640 1 -- 172.21.15.17:0/1118452160 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c0037070 con 0x7f43c8158b60 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.900+0000 7f43cc983640 1 -- 172.21.15.17:0/1118452160 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0031b20 con 0x7f43c8158b60 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 -- 172.21.15.17:0/1118452160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 msgr2=0x7f43c8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/1118452160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 0x7f43c8158ff0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097a0 tx=0x7f43c002b820 comp rx=0 tx=0).stop 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 -- 172.21.15.17:0/1118452160 shutdown_connections 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/1118452160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c8158b60 0x7f43c8158ff0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/1118452160 >> v2:172.21.15.163:3300/0 conn(0x7f43c814f360 0x7f43c8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 -- 172.21.15.17:0/1118452160 >> 172.21.15.17:0/1118452160 conn(0x7f43c80ac570 msgr2=0x7f43c80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 -- 172.21.15.17:0/1118452160 shutdown_connections 2024-03-06T07:23:01.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.901+0000 7f43cfbe9640 1 -- 172.21.15.17:0/1118452160 wait complete. 2024-03-06T07:23:01.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.903+0000 7f43cfbe9640 1 Processor -- start 2024-03-06T07:23:01.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.903+0000 7f43cfbe9640 1 -- start start 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c814f360 0x7f43c814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c8158b60 0x7f43c814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c5a50 con 0x7f43c80c88c0 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c7c30 con 0x7f43c8158b60 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cfbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c80c5bd0 con 0x7f43c814f360 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35514/0 (socket says 172.21.15.17:35514) 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 -- 172.21.15.17:0/344623104 learned_addr learned my addr 172.21.15.17:0/344623104 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd184640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c814f360 0x7f43c814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43ce186640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c8158b60 0x7f43c814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 -- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c814f360 msgr2=0x7f43c814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c814f360 0x7f43c814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 -- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c8158b60 msgr2=0x7f43c814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c8158b60 0x7f43c814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.904+0000 7f43cd985640 1 -- 172.21.15.17:0/344623104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8152280 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43cd985640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4017070 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400c9a0 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400cd90 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c80c0650 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.905+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c81524f0 con 0x7f43c80c88c0 2024-03-06T07:23:01.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.906+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c804f330 con 0x7f43c80c88c0 2024-03-06T07:23:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.907+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43c401a070 con 0x7f43c80c88c0 2024-03-06T07:23:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.907+0000 7f43ba7fc640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 0x7f43ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.907+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f43c408a6e0 con 0x7f43c80c88c0 2024-03-06T07:23:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.907+0000 7f43cd184640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 0x7f43ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:01.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.908+0000 7f43cd184640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 0x7f43ac06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f43c80c15e0 tx=0x7f43bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:01.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:01.912+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f43c40589c0 con 0x7f43c80c88c0 2024-03-06T07:23:02.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.094+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43c80ca550 con 0x7f43c80c88c0 2024-03-06T07:23:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.096+0000 7f43ba7fc640 1 -- 172.21.15.17:0/344623104 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20800 (secure 0 0 0) 0x7f43c405ca60 con 0x7f43c80c88c0 2024-03-06T07:23:02.098 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:02.098 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:54.958958+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 msgr2=0x7f43ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 0x7f43ac06d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f43c80c15e0 tx=0x7f43bc009040 comp rx=0 tx=0).stop 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 msgr2=0x7f43c80bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f43c400e9b0 tx=0x7f43c400ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 shutdown_connections 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43ac06b4d0 0x7f43ac06d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43c8158b60 0x7f43c814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43c814f360 0x7f43c814ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 --2- 172.21.15.17:0/344623104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c80c88c0 0x7f43c80bfca0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 >> 172.21.15.17:0/344623104 conn(0x7f43c80ac570 msgr2=0x7f43c80c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 shutdown_connections 2024-03-06T07:23:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.098+0000 7f43cfbe9640 1 -- 172.21.15.17:0/344623104 wait complete. 2024-03-06T07:23:02.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:22:32.972999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:22:33.673658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:22:47.593104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:22:48.766993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:02.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:02.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.218+0000 7f560e6c2640 1 Processor -- start 2024-03-06T07:23:02.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.218+0000 7f560e6c2640 1 -- start start 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f5608155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f560806ca70 0x7f56080c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56080c5380 0x7f56080c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560806c020 con 0x7f560806ca70 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080b0450 con 0x7f56080c5380 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560e6c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c77f0 con 0x7f56080b0070 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f560806ca70 0x7f56080c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f560806ca70 0x7f56080c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35530/0 (socket says 172.21.15.17:35530) 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 -- 172.21.15.17:0/138745313 learned_addr learned my addr 172.21.15.17:0/138745313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:02.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f5607fff640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f5608155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f560cc5f640 1 --2- 172.21.15.17:0/138745313 >> v2:172.21.15.163:3300/0 conn(0x7f56080c5380 0x7f56080c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 -- 172.21.15.17:0/138745313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 msgr2=0x7f5608155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f5608155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 -- 172.21.15.17:0/138745313 >> v2:172.21.15.163:3300/0 conn(0x7f56080c5380 msgr2=0x7f56080c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 --2- 172.21.15.17:0/138745313 >> v2:172.21.15.163:3300/0 conn(0x7f56080c5380 0x7f56080c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.219+0000 7f56077fe640 1 -- 172.21.15.17:0/138745313 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080c7ad0 con 0x7f560806ca70 2024-03-06T07:23:02.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f56077fe640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f560806ca70 0x7f56080c8900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f56080ad5e0 tx=0x7f55f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a509ea7947dc6874 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f5606ffd640 1 -- 172.21.15.17:0/138745313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f002e070 con 0x7f560806ca70 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f5606ffd640 1 -- 172.21.15.17:0/138745313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0037070 con 0x7f560806ca70 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f5606ffd640 1 -- 172.21.15.17:0/138745313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0031a00 con 0x7f560806ca70 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f560e6c2640 1 -- 172.21.15.17:0/138745313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f560806ca70 msgr2=0x7f56080c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.220+0000 7f560e6c2640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f560806ca70 0x7f56080c8900 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f56080ad5e0 tx=0x7f55f002b880 comp rx=0 tx=0).stop 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 -- 172.21.15.17:0/138745313 shutdown_connections 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 --2- 172.21.15.17:0/138745313 >> v2:172.21.15.163:3300/0 conn(0x7f56080c5380 0x7f56080c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f560806ca70 0x7f56080c8900 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 --2- 172.21.15.17:0/138745313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f5608155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 -- 172.21.15.17:0/138745313 >> 172.21.15.17:0/138745313 conn(0x7f56080ac590 msgr2=0x7f56080ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 -- 172.21.15.17:0/138745313 shutdown_connections 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.221+0000 7f560e6c2640 1 -- 172.21.15.17:0/138745313 wait complete. 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.222+0000 7f560e6c2640 1 Processor -- start 2024-03-06T07:23:02.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.222+0000 7f560e6c2640 1 -- start start 2024-03-06T07:23:02.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560806ca70 0x7f56080c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f56081544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560806cef0 con 0x7f56080c5380 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560806c550 con 0x7f560806ca70 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560806da80 con 0x7f56080b0070 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35534/0 (socket says 172.21.15.17:35534) 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 -- 172.21.15.17:0/97414712 learned_addr learned my addr 172.21.15.17:0/97414712 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f56077fe640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f56081544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f5607fff640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560806ca70 0x7f56080c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 -- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 msgr2=0x7f56081544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f56081544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 -- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560806ca70 msgr2=0x7f56080c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560806ca70 0x7f56080c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 -- 172.21.15.17:0/97414712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608159100 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560cc5f640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5600009dd0 tx=0x7f560000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600018020 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5600014480 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600014870 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56081ee9d0 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.223+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5608154a00 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.225+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560804f330 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.225+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5600005d10 con 0x7f56080c5380 2024-03-06T07:23:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.225+0000 7f5604ff9640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 0x7f55e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.226+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f560008b110 con 0x7f56080c5380 2024-03-06T07:23:02.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.226+0000 7f5607fff640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 0x7f55e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.226+0000 7f5607fff640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 0x7f55e806d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009ac0 tx=0x7f55f8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.230+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5600022070 con 0x7f56080c5380 2024-03-06T07:23:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.409+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f560806d860 con 0x7f56080c5380 2024-03-06T07:23:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.410+0000 7f5604ff9640 1 -- 172.21.15.17:0/97414712 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5600010040 con 0x7f56080c5380 2024-03-06T07:23:02.411 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 msgr2=0x7f55e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 0x7f55e806d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009ac0 tx=0x7f55f8009190 comp rx=0 tx=0).stop 2024-03-06T07:23:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 msgr2=0x7f56081ea8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5600009dd0 tx=0x7f560000c490 comp rx=0 tx=0).stop 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 shutdown_connections 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55e806b4d0 0x7f55e806d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56080c5380 0x7f56081ea8d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56080b0070 0x7f56081544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 --2- 172.21.15.17:0/97414712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f560806ca70 0x7f56080c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.411+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 >> 172.21.15.17:0/97414712 conn(0x7f56080ac590 msgr2=0x7f56080ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.412+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 shutdown_connections 2024-03-06T07:23:02.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.412+0000 7f560e6c2640 1 -- 172.21.15.17:0/97414712 wait complete. 2024-03-06T07:23:02.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:23:02.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.526+0000 7fd39fd25640 1 Processor -- start 2024-03-06T07:23:02.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.526+0000 7fd39fd25640 1 -- start start 2024-03-06T07:23:02.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39fd25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3980c3240 0x7fd3980c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39fd25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3981567e0 con 0x7fd3980c2c90 2024-03-06T07:23:02.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39fd25640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd398156a80 con 0x7fd398155fb0 2024-03-06T07:23:02.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39fd25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd398156d60 con 0x7fd3980c3240 2024-03-06T07:23:02.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39d2c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3980c3240 0x7fd3980c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39e2c2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd3980c2c90 0x7fd3980c56d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58232/0 (socket says 172.21.15.17:58232) 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.527+0000 7fd39e2c2640 1 -- 172.21.15.17:0/2408415369 learned_addr learned my addr 172.21.15.17:0/2408415369 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.528+0000 7fd39d2c0640 1 -- 172.21.15.17:0/2408415369 >> v1:172.21.15.163:6789/0 conn(0x7fd398155fb0 legacy=0x7fd3980c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.528+0000 7fd39d2c0640 1 -- 172.21.15.17:0/2408415369 >> v1:172.21.15.17:6789/0 conn(0x7fd3980c2c90 legacy=0x7fd3980c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.528+0000 7fd39cabf640 1 -- 172.21.15.17:0/2408415369 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144844255 0 0) 0x7fd3981567e0 con 0x7fd3980c2c90 2024-03-06T07:23:02.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.528+0000 7fd39d2c0640 1 -- 172.21.15.17:0/2408415369 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398157040 con 0x7fd3980c3240 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.528+0000 7fd39d2c0640 1 --2- 172.21.15.17:0/2408415369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3980c3240 0x7fd3980c2750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097a0 tx=0x7fd39402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3017b62a0946a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39cabf640 1 -- 172.21.15.17:0/2408415369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394035020 con 0x7fd3980c3240 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39cabf640 1 -- 172.21.15.17:0/2408415369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394031830 con 0x7fd3980c3240 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39cabf640 1 -- 172.21.15.17:0/2408415369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394031c00 con 0x7fd3980c3240 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 -- 172.21.15.17:0/2408415369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3980c3240 msgr2=0x7fd3980c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 --2- 172.21.15.17:0/2408415369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3980c3240 0x7fd3980c2750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd3940097a0 tx=0x7fd39402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 -- 172.21.15.17:0/2408415369 shutdown_connections 2024-03-06T07:23:02.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 --2- 172.21.15.17:0/2408415369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3980c3240 0x7fd3980c2750 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 -- 172.21.15.17:0/2408415369 >> 172.21.15.17:0/2408415369 conn(0x7fd3980a94d0 msgr2=0x7fd3980a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 -- 172.21.15.17:0/2408415369 shutdown_connections 2024-03-06T07:23:02.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.529+0000 7fd39fd25640 1 -- 172.21.15.17:0/2408415369 wait complete. 2024-03-06T07:23:02.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.530+0000 7fd39fd25640 1 Processor -- start 2024-03-06T07:23:02.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.530+0000 7fd39fd25640 1 -- start start 2024-03-06T07:23:02.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3980c2c90 0x7fd3981509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3980c3240 0x7fd3981eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c5f60 con 0x7fd398155fb0 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c5de0 con 0x7fd3980c2c90 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39fd25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c60e0 con 0x7fd3980c3240 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39e2c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39e2c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35554/0 (socket says 172.21.15.17:35554) 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39e2c2640 1 -- 172.21.15.17:0/3304193527 learned_addr learned my addr 172.21.15.17:0/3304193527 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39d2c0640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3980c3240 0x7fd3981eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.531+0000 7fd39dac1640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3980c2c90 0x7fd3981509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 -- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3980c3240 msgr2=0x7fd3981eb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3980c3240 0x7fd3981eb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 -- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3980c2c90 msgr2=0x7fd3981509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3980c2c90 0x7fd3981509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 -- 172.21.15.17:0/3304193527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398150370 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39e2c2640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c7d0 tx=0x7fd39000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390018070 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39000a640 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3980abe20 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000aa70 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.532+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3980aa4f0 con 0x7fd398155fb0 2024-03-06T07:23:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.533+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39804f330 con 0x7fd398155fb0 2024-03-06T07:23:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.533+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd39001b070 con 0x7fd398155fb0 2024-03-06T07:23:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.534+0000 7fd38a7fc640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 0x7fd37c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.534+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fd390089c70 con 0x7fd398155fb0 2024-03-06T07:23:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.534+0000 7fd39dac1640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 0x7fd37c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.534+0000 7fd39dac1640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 0x7fd37c06d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd3980c1320 tx=0x7fd38c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.538+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd390057f60 con 0x7fd398155fb0 2024-03-06T07:23:02.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.800+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3980c2530 con 0x7fd398155fb0 2024-03-06T07:23:02.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.800+0000 7fd38a7fc640 1 -- 172.21.15.17:0/3304193527 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd39005c000 con 0x7fd398155fb0 2024-03-06T07:23:02.802 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 msgr2=0x7fd37c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 0x7fd37c06d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd3980c1320 tx=0x7fd38c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 msgr2=0x7fd398154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd39000c7d0 tx=0x7fd39000cc90 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 shutdown_connections 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37c06b4d0 0x7fd37c06d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd398155fb0 0x7fd398154a90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3980c3240 0x7fd3981eb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 --2- 172.21.15.17:0/3304193527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3980c2c90 0x7fd3981509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 >> 172.21.15.17:0/3304193527 conn(0x7fd3980a94d0 msgr2=0x7fd3980721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.802+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 shutdown_connections 2024-03-06T07:23:02.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.803+0000 7fd39fd25640 1 -- 172.21.15.17:0/3304193527 wait complete. 2024-03-06T07:23:02.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.917+0000 7fde36335640 1 Processor -- start 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.917+0000 7fde36335640 1 -- start start 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde36335640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 0x7fde300c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde36335640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 0x7fde30155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde36335640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde30152dd0 con 0x7fde300c5370 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde36335640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde30159710 con 0x7fde301529f0 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde36335640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde30156180 con 0x7fde300c57d0 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde2ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 0x7fde300c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde348d2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 0x7fde30155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde2ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 0x7fde300c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35562/0 (socket says 172.21.15.17:35562) 2024-03-06T07:23:02.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde2f7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fde300c5370 0x7fde30155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:45354/0 (socket says 172.21.15.17:45354) 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde2ffff640 1 -- 172.21.15.17:0/425604599 learned_addr learned my addr 172.21.15.17:0/425604599 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde348d2640 1 -- 172.21.15.17:0/425604599 >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 msgr2=0x7fde300c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde348d2640 1 --2- 172.21.15.17:0/425604599 >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 0x7fde300c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde348d2640 1 -- 172.21.15.17:0/425604599 >> v1:172.21.15.163:6789/0 conn(0x7fde300c5370 legacy=0x7fde30155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde348d2640 1 -- 172.21.15.17:0/425604599 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30158b00 con 0x7fde300c57d0 2024-03-06T07:23:02.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.918+0000 7fde2effd640 1 -- 172.21.15.17:0/425604599 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140649446 0 0) 0x7fde30152dd0 con 0x7fde300c5370 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.919+0000 7fde348d2640 1 --2- 172.21.15.17:0/425604599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 0x7fde30155c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde28009d90 tx=0x7fde2802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f28bf2a1b9bcc77 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.919+0000 7fde2effd640 1 -- 172.21.15.17:0/425604599 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28039070 con 0x7fde300c57d0 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.919+0000 7fde2effd640 1 -- 172.21.15.17:0/425604599 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2802cbc0 con 0x7fde300c57d0 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde2effd640 1 -- 172.21.15.17:0/425604599 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28034440 con 0x7fde300c57d0 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 -- 172.21.15.17:0/425604599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 msgr2=0x7fde30155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 --2- 172.21.15.17:0/425604599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 0x7fde30155c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde28009d90 tx=0x7fde2802be10 comp rx=0 tx=0).stop 2024-03-06T07:23:02.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 -- 172.21.15.17:0/425604599 shutdown_connections 2024-03-06T07:23:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 --2- 172.21.15.17:0/425604599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde300c57d0 0x7fde30155c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 --2- 172.21.15.17:0/425604599 >> v2:172.21.15.17:3300/0 conn(0x7fde301529f0 0x7fde300c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 -- 172.21.15.17:0/425604599 >> 172.21.15.17:0/425604599 conn(0x7fde300ac580 msgr2=0x7fde300ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 -- 172.21.15.17:0/425604599 shutdown_connections 2024-03-06T07:23:02.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.920+0000 7fde36335640 1 -- 172.21.15.17:0/425604599 wait complete. 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.921+0000 7fde36335640 1 Processor -- start 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.921+0000 7fde36335640 1 -- start start 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde300c5370 0x7fde30157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde301529f0 0x7fde3014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301590c0 con 0x7fde300c57d0 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c68b0 con 0x7fde300c5370 2024-03-06T07:23:02.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde36335640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde301593a0 con 0x7fde301529f0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35574/0 (socket says 172.21.15.17:35574) 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 -- 172.21.15.17:0/1582282214 learned_addr learned my addr 172.21.15.17:0/1582282214 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2ffff640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde300c5370 0x7fde30157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde348d2640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde301529f0 0x7fde3014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 -- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde301529f0 msgr2=0x7fde3014d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde301529f0 0x7fde3014d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 -- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde300c5370 msgr2=0x7fde30157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde300c5370 0x7fde30157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.922+0000 7fde2f7fe640 1 -- 172.21.15.17:0/1582282214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30149710 con 0x7fde300c57d0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde2f7fe640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c500 tx=0x7fde2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20018070 con 0x7fde300c57d0 2024-03-06T07:23:02.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20007580 con 0x7fde300c57d0 2024-03-06T07:23:02.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde301eaa80 con 0x7fde300c57d0 2024-03-06T07:23:02.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20004a20 con 0x7fde300c57d0 2024-03-06T07:23:02.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.923+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde301eb040 con 0x7fde300c57d0 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.924+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3004f330 con 0x7fde300c57d0 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.924+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde20004bc0 con 0x7fde300c57d0 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.925+0000 7fde2cff9640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.925+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fde20089990 con 0x7fde300c57d0 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.925+0000 7fde2ffff640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.925+0000 7fde2ffff640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 0x7fde1c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde24009b30 tx=0x7fde24009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:02.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:02.929+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde20057c90 con 0x7fde300c57d0 2024-03-06T07:23:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.113+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fde300cab70 con 0x7fde300c57d0 2024-03-06T07:23:03.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.114+0000 7fde2cff9640 1 -- 172.21.15.17:0/1582282214 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fde2005bd30 con 0x7fde300c57d0 2024-03-06T07:23:03.115 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:23:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.115+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 msgr2=0x7fde1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.115+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 0x7fde1c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fde24009b30 tx=0x7fde24009240 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.115+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 msgr2=0x7fde3014cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.115+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c500 tx=0x7fde2000c9c0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 shutdown_connections 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde301529f0 0x7fde3014d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde300c57d0 0x7fde3014cd50 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 --2- 172.21.15.17:0/1582282214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde300c5370 0x7fde30157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 >> 172.21.15.17:0/1582282214 conn(0x7fde300ac580 msgr2=0x7fde3006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 shutdown_connections 2024-03-06T07:23:03.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.116+0000 7fde36335640 1 -- 172.21.15.17:0/1582282214 wait complete. 2024-03-06T07:23:03.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:23:03.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.230+0000 7f4d5b8c9640 1 Processor -- start 2024-03-06T07:23:03.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 -- start start 2024-03-06T07:23:03.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d540c3a70 0x7f4d540732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406db30 0x7f4d540c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d54156060 con 0x7f4d5406d750 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d54155b30 con 0x7f4d540c3a70 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d5b8c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540a8430 con 0x7f4d5406db30 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d58e64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d540c3a70 0x7f4d540732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d58e64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d540c3a70 0x7f4d540732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35586/0 (socket says 172.21.15.17:35586) 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d58e64640 1 -- 172.21.15.17:0/2437412619 learned_addr learned my addr 172.21.15.17:0/2437412619 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.231+0000 7f4d59e66640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406db30 0x7f4d540c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.232+0000 7f4d58e64640 1 -- 172.21.15.17:0/2437412619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406db30 msgr2=0x7f4d540c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.232+0000 7f4d58e64640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406db30 0x7f4d540c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.232+0000 7f4d58e64640 1 -- 172.21.15.17:0/2437412619 >> v1:172.21.15.163:6789/0 conn(0x7f4d5406d750 legacy=0x7f4d54072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.232+0000 7f4d58e64640 1 -- 172.21.15.17:0/2437412619 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54155d80 con 0x7f4d540c3a70 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.232+0000 7f4d58e64640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d540c3a70 0x7f4d540732a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009870 tx=0x7f4d4402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3523c4b3a9ff7031 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d4bfff640 1 -- 172.21.15.17:0/2437412619 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d440317c0 con 0x7f4d540c3a70 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d4bfff640 1 -- 172.21.15.17:0/2437412619 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44037070 con 0x7f4d540c3a70 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d4bfff640 1 -- 172.21.15.17:0/2437412619 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44031b80 con 0x7f4d540c3a70 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/2437412619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d540c3a70 msgr2=0x7f4d540732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d540c3a70 0x7f4d540732a0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009870 tx=0x7f4d4402b860 comp rx=0 tx=0).stop 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/2437412619 shutdown_connections 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406db30 0x7f4d540c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/2437412619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d540c3a70 0x7f4d540732a0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/2437412619 >> 172.21.15.17:0/2437412619 conn(0x7f4d540c5560 msgr2=0x7f4d540c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/2437412619 shutdown_connections 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.233+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/2437412619 wait complete. 2024-03-06T07:23:03.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.234+0000 7f4d5b8c9640 1 Processor -- start 2024-03-06T07:23:03.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.234+0000 7f4d5b8c9640 1 -- start start 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406d750 0x7f4d541ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d540c3a70 0x7f4d541540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540a7e50 con 0x7f4d5406db30 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540a7cd0 con 0x7f4d540c3a70 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d5b8c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d540a7fd0 con 0x7f4d5406d750 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d58e64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d58e64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35588/0 (socket says 172.21.15.17:35588) 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d58e64640 1 -- 172.21.15.17:0/843221674 learned_addr learned my addr 172.21.15.17:0/843221674 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:03.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d59665640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406d750 0x7f4d541ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.235+0000 7f4d59e66640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d540c3a70 0x7f4d541540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 -- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406d750 msgr2=0x7f4d541ee170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406d750 0x7f4d541ee170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 -- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d540c3a70 msgr2=0x7f4d541540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d540c3a70 0x7f4d541540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 -- 172.21.15.17:0/843221674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d541577e0 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d58e64640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4d44000c00 tx=0x7f4d440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44032650 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d440327f0 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d540abec0 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44032be0 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.236+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d54156720 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.237+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540ac340 con 0x7f4d5406db30 2024-03-06T07:23:03.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.238+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d4404d3f0 con 0x7f4d5406db30 2024-03-06T07:23:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.238+0000 7f4d49ffb640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 0x7f4d3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.238+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f4d440ac090 con 0x7f4d5406db30 2024-03-06T07:23:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.238+0000 7f4d59665640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 0x7f4d3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.239+0000 7f4d59665640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 0x7f4d3806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4d54151600 tx=0x7f4d50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.242+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d44040070 con 0x7f4d5406db30 2024-03-06T07:23:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.421+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d540c6e60 con 0x7f4d5406db30 2024-03-06T07:23:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.421+0000 7f4d49ffb640 1 -- 172.21.15.17:0/843221674 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v80) v1 ==== 105+0+11 (secure 0 0 0) 0x7f4d4407a490 con 0x7f4d5406db30 2024-03-06T07:23:03.423 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 msgr2=0x7f4d3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 0x7f4d3806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4d54151600 tx=0x7f4d50009040 comp rx=0 tx=0).stop 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 msgr2=0x7f4d54153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4d44000c00 tx=0x7f4d440046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 shutdown_connections 2024-03-06T07:23:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d3806b4d0 0x7f4d3806d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d540c3a70 0x7f4d541540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d5406db30 0x7f4d54153b60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 --2- 172.21.15.17:0/843221674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d5406d750 0x7f4d541ee170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 >> 172.21.15.17:0/843221674 conn(0x7f4d540c5560 msgr2=0x7f4d54072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 shutdown_connections 2024-03-06T07:23:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.423+0000 7f4d5b8c9640 1 -- 172.21.15.17:0/843221674 wait complete. 2024-03-06T07:23:03.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:23:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.542+0000 7f772c1b5640 1 Processor -- start 2024-03-06T07:23:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.542+0000 7f772c1b5640 1 -- start start 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f772c1b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f772406afc0 0x7f772406b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f772c1b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77240c5840 con 0x7f77240c2120 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f772c1b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77240c5b20 con 0x7f77240c2580 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f772c1b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7724158790 con 0x7f772406afc0 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729f51640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f772406afc0 0x7f772406b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729750640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f77240c2120 0x7f7724158c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58244/0 (socket says 172.21.15.17:58244) 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729750640 1 -- 172.21.15.17:0/389559121 learned_addr learned my addr 172.21.15.17:0/389559121 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729f51640 1 -- 172.21.15.17:0/389559121 >> v1:172.21.15.163:6789/0 conn(0x7f77240c2580 legacy=0x7f772415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729f51640 1 -- 172.21.15.17:0/389559121 >> v1:172.21.15.17:6789/0 conn(0x7f77240c2120 legacy=0x7f7724158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7728f4f640 1 -- 172.21.15.17:0/389559121 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928594969 0 0) 0x7f77240c5b20 con 0x7f77240c2580 2024-03-06T07:23:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.543+0000 7f7729f51640 1 -- 172.21.15.17:0/389559121 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77240c5e00 con 0x7f772406afc0 2024-03-06T07:23:03.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.544+0000 7f7729f51640 1 --2- 172.21.15.17:0/389559121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772406b3a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f77200097a0 tx=0x7f772002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a710da0bea75e407 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.544+0000 7f7728f4f640 1 -- 172.21.15.17:0/389559121 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720037070 con 0x7f772406afc0 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.544+0000 7f7728f4f640 1 -- 172.21.15.17:0/389559121 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720030710 con 0x7f772406afc0 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.544+0000 7f7728f4f640 1 -- 172.21.15.17:0/389559121 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720030ae0 con 0x7f772406afc0 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 -- 172.21.15.17:0/389559121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 msgr2=0x7f772406b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 --2- 172.21.15.17:0/389559121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772406b3a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f77200097a0 tx=0x7f772002b710 comp rx=0 tx=0).stop 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 -- 172.21.15.17:0/389559121 shutdown_connections 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 --2- 172.21.15.17:0/389559121 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772406b3a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 -- 172.21.15.17:0/389559121 >> 172.21.15.17:0/389559121 conn(0x7f77240c4220 msgr2=0x7f77240c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 -- 172.21.15.17:0/389559121 shutdown_connections 2024-03-06T07:23:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.545+0000 7f772c1b5640 1 -- 172.21.15.17:0/389559121 wait complete. 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.546+0000 7f772c1b5640 1 Processor -- start 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.546+0000 7f772c1b5640 1 -- start start 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77240c2120 0x7f7724072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77240c2580 0x7f77241ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77240c2b30 con 0x7f772406afc0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7724155220 con 0x7f77240c2120 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772c1b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77240c2cb0 con 0x7f77240c2580 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35606/0 (socket says 172.21.15.17:35606) 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 -- 172.21.15.17:0/1463881493 learned_addr learned my addr 172.21.15.17:0/1463881493 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729750640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77240c2120 0x7f7724072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f772a752640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77240c2580 0x7f77241ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 -- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77240c2580 msgr2=0x7f77241ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77240c2580 0x7f77241ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 -- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77240c2120 msgr2=0x7f7724072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77240c2120 0x7f7724072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 -- 172.21.15.17:0/1463881493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724071e30 con 0x7f772406afc0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.547+0000 7f7729f51640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f77200098c0 tx=0x7f77200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.548+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720042070 con 0x7f772406afc0 2024-03-06T07:23:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.548+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720032a40 con 0x7f772406afc0 2024-03-06T07:23:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.548+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7724158320 con 0x7f772406afc0 2024-03-06T07:23:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.548+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720037040 con 0x7f772406afc0 2024-03-06T07:23:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.548+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7724071510 con 0x7f772406afc0 2024-03-06T07:23:03.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.549+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77240c5010 con 0x7f772406afc0 2024-03-06T07:23:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.549+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7720045070 con 0x7f772406afc0 2024-03-06T07:23:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.550+0000 7f77167fc640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.550+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f77200ac770 con 0x7f772406afc0 2024-03-06T07:23:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.550+0000 7f7729750640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.550+0000 7f7729750640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 0x7f770806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f77240744a0 tx=0x7f7718007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.554+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f772007ab80 con 0x7f772406afc0 2024-03-06T07:23:03.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.733+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f772404f330 con 0x7f772406afc0 2024-03-06T07:23:03.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.734+0000 7f77167fc640 1 -- 172.21.15.17:0/1463881493 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v80) v1 ==== 101+0+11 (secure 0 0 0) 0x7f772007ec20 con 0x7f772406afc0 2024-03-06T07:23:03.735 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:23:03.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 msgr2=0x7f770806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 0x7f770806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f77240744a0 tx=0x7f7718007e20 comp rx=0 tx=0).stop 2024-03-06T07:23:03.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 msgr2=0x7f772414a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f77200098c0 tx=0x7f77200046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 shutdown_connections 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77240c2580 0x7f77241ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77240c2120 0x7f7724072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 --2- 172.21.15.17:0/1463881493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772406afc0 0x7f772414a040 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.735+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 >> 172.21.15.17:0/1463881493 conn(0x7f77240c4220 msgr2=0x7f7724158c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.736+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 shutdown_connections 2024-03-06T07:23:03.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.736+0000 7f772c1b5640 1 -- 172.21.15.17:0/1463881493 wait complete. 2024-03-06T07:23:03.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:03.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.848+0000 7f4d1be84640 1 Processor -- start 2024-03-06T07:23:03.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.849+0000 7f4d1be84640 1 -- start start 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.849+0000 7f4d1be84640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d14156150 0x7f4d141529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.849+0000 7f4d1be84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d140c28d0 con 0x7f4d140c1aa0 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.849+0000 7f4d1be84640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d140c2bb0 con 0x7f4d1414f790 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.849+0000 7f4d1be84640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d140c2680 con 0x7f4d14156150 2024-03-06T07:23:03.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1a421640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d14156150 0x7f4d141529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1941f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4d140c1aa0 0x7f4d14155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58248/0 (socket says 172.21.15.17:58248) 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1941f640 1 -- 172.21.15.17:0/2938607489 learned_addr learned my addr 172.21.15.17:0/2938607489 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1a421640 1 -- 172.21.15.17:0/2938607489 >> v1:172.21.15.163:6789/0 conn(0x7f4d1414f790 legacy=0x7f4d140c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1a421640 1 -- 172.21.15.17:0/2938607489 >> v1:172.21.15.17:6789/0 conn(0x7f4d140c1aa0 legacy=0x7f4d14155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d18c1e640 1 -- 172.21.15.17:0/2938607489 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364399412 0 0) 0x7f4d140c28d0 con 0x7f4d140c1aa0 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.850+0000 7f4d1a421640 1 -- 172.21.15.17:0/2938607489 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d140c2e90 con 0x7f4d14156150 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d1a421640 1 --2- 172.21.15.17:0/2938607489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141529d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c2800 tx=0x7f4d0c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73a61ad16df3044 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d18c1e640 1 -- 172.21.15.17:0/2938607489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c039070 con 0x7f4d14156150 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d18c1e640 1 -- 172.21.15.17:0/2938607489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c02cce0 con 0x7f4d14156150 2024-03-06T07:23:03.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d18c1e640 1 -- 172.21.15.17:0/2938607489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c034660 con 0x7f4d14156150 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d1be84640 1 -- 172.21.15.17:0/2938607489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 msgr2=0x7f4d141529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.851+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2938607489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141529d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c2800 tx=0x7f4d0c02bf30 comp rx=0 tx=0).stop 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.852+0000 7f4d1be84640 1 -- 172.21.15.17:0/2938607489 shutdown_connections 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.852+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2938607489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141529d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.852+0000 7f4d1be84640 1 -- 172.21.15.17:0/2938607489 >> 172.21.15.17:0/2938607489 conn(0x7f4d140c6710 msgr2=0x7f4d140c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.852+0000 7f4d1be84640 1 -- 172.21.15.17:0/2938607489 shutdown_connections 2024-03-06T07:23:03.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.852+0000 7f4d1be84640 1 -- 172.21.15.17:0/2938607489 wait complete. 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.853+0000 7f4d1be84640 1 Processor -- start 2024-03-06T07:23:03.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.853+0000 7f4d1be84640 1 -- start start 2024-03-06T07:23:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.854+0000 7f4d1be84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d140c1aa0 0x7f4d14154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.854+0000 7f4d1be84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d1414f790 0x7f4d1414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c5e30 con 0x7f4d14156150 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c0d60 con 0x7f4d1414f790 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d140c5fb0 con 0x7f4d140c1aa0 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35616/0 (socket says 172.21.15.17:35616) 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 -- 172.21.15.17:0/2044636480 learned_addr learned my addr 172.21.15.17:0/2044636480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d19c20640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d140c1aa0 0x7f4d14154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1941f640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d1414f790 0x7f4d1414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 -- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d140c1aa0 msgr2=0x7f4d14154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d140c1aa0 0x7f4d14154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 -- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d1414f790 msgr2=0x7f4d1414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d1414f790 0x7f4d1414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 -- 172.21.15.17:0/2044636480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14073140 con 0x7f4d14156150 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1a421640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c037040 tx=0x7f4d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c044020 con 0x7f4d14156150 2024-03-06T07:23:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c03a040 con 0x7f4d14156150 2024-03-06T07:23:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d140733e0 con 0x7f4d14156150 2024-03-06T07:23:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c039050 con 0x7f4d14156150 2024-03-06T07:23:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.855+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1414d2f0 con 0x7f4d14156150 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.856+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1404f330 con 0x7f4d14156150 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.857+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d0c02cb60 con 0x7f4d14156150 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.858+0000 7f4d0a7fc640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.858+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f4d0c0acec0 con 0x7f4d14156150 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.858+0000 7f4d19c20640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.858+0000 7f4d19c20640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 0x7f4cf806d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c5d30 tx=0x7f4d04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:03.862+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d0c04e020 con 0x7f4d14156150 2024-03-06T07:23:04.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.082+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d140c1f40 con 0x7f4d14156150 2024-03-06T07:23:04.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.083+0000 7f4d0a7fc640 1 -- 172.21.15.17:0/2044636480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f4d0c07b240 con 0x7f4d14156150 2024-03-06T07:23:04.085 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:04.085 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:58.973213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6427":{"gid":6427,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6836/1549441863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1549441863},{"type":"v1","addr":"172.21.15.163:6836","nonce":1549441863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6427,"qdb_cluster":[6427]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:23:04.085 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 66 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 msgr2=0x7f4cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 0x7f4cf806d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4d140c5d30 tx=0x7f4d04009040 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 msgr2=0x7f4d141e85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c037040 tx=0x7f4d0c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 shutdown_connections 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4cf806b4d0 0x7f4cf806d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d14156150 0x7f4d141e85f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d1414f790 0x7f4d1414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 --2- 172.21.15.17:0/2044636480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d140c1aa0 0x7f4d14154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 >> 172.21.15.17:0/2044636480 conn(0x7f4d140c6710 msgr2=0x7f4d1414d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 shutdown_connections 2024-03-06T07:23:04.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.085+0000 7f4d1be84640 1 -- 172.21.15.17:0/2044636480 wait complete. 2024-03-06T07:23:04.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.198+0000 7f00b0cc9640 1 Processor -- start 2024-03-06T07:23:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.198+0000 7f00b0cc9640 1 -- start start 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00b0cc9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00ac1506b0 0x7f00ac153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00b0cc9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00ac0acc00 0x7f00ac0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00b0cc9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac070aa0 con 0x7f00ac156ea0 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00b0cc9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac156450 con 0x7f00ac0acc00 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00b0cc9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac156a10 con 0x7f00ac1506b0 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00aad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00ac0acc00 0x7f00ac0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00a9d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00ac156ea0 0x7f00ac0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58256/0 (socket says 172.21.15.17:58256) 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00a9d9b640 1 -- 172.21.15.17:0/2278775623 learned_addr learned my addr 172.21.15.17:0/2278775623 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00aa59c640 1 --2- 172.21.15.17:0/2278775623 >> v2:172.21.15.163:3300/0 conn(0x7f00ac1506b0 0x7f00ac153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00aad9d640 1 -- 172.21.15.17:0/2278775623 >> v2:172.21.15.163:3300/0 conn(0x7f00ac1506b0 msgr2=0x7f00ac153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00aad9d640 1 --2- 172.21.15.17:0/2278775623 >> v2:172.21.15.163:3300/0 conn(0x7f00ac1506b0 0x7f00ac153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00aad9d640 1 -- 172.21.15.17:0/2278775623 >> v1:172.21.15.17:6789/0 conn(0x7f00ac156ea0 legacy=0x7f00ac0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.199+0000 7f00a959a640 1 -- 172.21.15.17:0/2278775623 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305856229 0 0) 0x7f00ac070aa0 con 0x7f00ac156ea0 2024-03-06T07:23:04.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.200+0000 7f00aad9d640 1 -- 172.21.15.17:0/2278775623 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac070d80 con 0x7f00ac0acc00 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00aad9d640 1 --2- 172.21.15.17:0/2278775623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac0c54a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0c97e0 tx=0x7f00a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd924ee74c7d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00a959a640 1 -- 172.21.15.17:0/2278775623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4039070 con 0x7f00ac0acc00 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00a959a640 1 -- 172.21.15.17:0/2278775623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a402cc80 con 0x7f00ac0acc00 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00a959a640 1 -- 172.21.15.17:0/2278775623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4034660 con 0x7f00ac0acc00 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 -- 172.21.15.17:0/2278775623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 msgr2=0x7f00ac0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/2278775623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac0c54a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0c97e0 tx=0x7f00a402bed0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 -- 172.21.15.17:0/2278775623 shutdown_connections 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/2278775623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac0c54a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/2278775623 >> v2:172.21.15.163:3300/0 conn(0x7f00ac1506b0 0x7f00ac153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 -- 172.21.15.17:0/2278775623 >> 172.21.15.17:0/2278775623 conn(0x7f00ac06b1a0 msgr2=0x7f00ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 -- 172.21.15.17:0/2278775623 shutdown_connections 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.201+0000 7f00b0cc9640 1 -- 172.21.15.17:0/2278775623 wait complete. 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 Processor -- start 2024-03-06T07:23:04.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 -- start start 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00ac1506b0 0x7f00ac14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ac156ea0 0x7f00ac145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0ad1d0 con 0x7f00ac0acc00 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0ad050 con 0x7f00ac156ea0 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00b0cc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0ad350 con 0x7f00ac1506b0 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00aa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00aa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35632/0 (socket says 172.21.15.17:35632) 2024-03-06T07:23:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.203+0000 7f00aa59c640 1 -- 172.21.15.17:0/1106907626 learned_addr learned my addr 172.21.15.17:0/1106907626 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00a9d9b640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00ac1506b0 0x7f00ac14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aad9d640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ac156ea0 0x7f00ac145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 -- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00ac1506b0 msgr2=0x7f00ac14df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00ac1506b0 0x7f00ac14df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 -- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ac156ea0 msgr2=0x7f00ac145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ac156ea0 0x7f00ac145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 -- 172.21.15.17:0/1106907626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac14c690 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00aa59c640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f009400e9b0 tx=0x7f009400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094017070 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400c9a0 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac14c970 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400cd90 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.204+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac14e490 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.205+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac04f330 con 0x7f00ac0acc00 2024-03-06T07:23:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.206+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f009401a070 con 0x7f00ac0acc00 2024-03-06T07:23:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.206+0000 7f00a2ffd640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 0x7f008c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.206+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f009408a660 con 0x7f00ac0acc00 2024-03-06T07:23:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.206+0000 7f00a9d9b640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 0x7f008c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.207+0000 7f00a9d9b640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 0x7f008c06d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a7cd0 tx=0x7f009c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.210+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0094058950 con 0x7f00ac0acc00 2024-03-06T07:23:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.391+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00ac0700a0 con 0x7f00ac0acc00 2024-03-06T07:23:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.393+0000 7f00a2ffd640 1 -- 172.21.15.17:0/1106907626 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20800 (secure 0 0 0) 0x7f009405c9f0 con 0x7f00ac0acc00 2024-03-06T07:23:04.394 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:04.395 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:54.958958+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.394+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 msgr2=0x7f008c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.394+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 0x7f008c06d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a7cd0 tx=0x7f009c007de0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.394+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 msgr2=0x7f00ac14fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.394+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f009400e9b0 tx=0x7f009400ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 shutdown_connections 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f008c06b4d0 0x7f008c06d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00ac156ea0 0x7f00ac145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00ac1506b0 0x7f00ac14df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 --2- 172.21.15.17:0/1106907626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00ac0acc00 0x7f00ac14fae0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 >> 172.21.15.17:0/1106907626 conn(0x7f00ac06b1a0 msgr2=0x7f00ac152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 shutdown_connections 2024-03-06T07:23:04.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.395+0000 7f00b0cc9640 1 -- 172.21.15.17:0/1106907626 wait complete. 2024-03-06T07:23:04.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:23:04.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.508+0000 7f8ca1d7a640 1 Processor -- start 2024-03-06T07:23:04.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.508+0000 7f8ca1d7a640 1 -- start start 2024-03-06T07:23:04.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c9c1550c0 0x7f8c9c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c9c1559e0 0x7f8c9c14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c14f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c4610 con 0x7f8c9c1550c0 2024-03-06T07:23:04.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9c0aa660 con 0x7f8c9c1559e0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8ca1d7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0aa940 con 0x7f8c9c0aa180 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c9c1550c0 0x7f8c9c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c9c1550c0 0x7f8c9c1554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35646/0 (socket says 172.21.15.17:35646) 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 -- 172.21.15.17:0/2155941718 learned_addr learned my addr 172.21.15.17:0/2155941718 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9affd640 1 --2- 172.21.15.17:0/2155941718 >> v2:172.21.15.163:3300/0 conn(0x7f8c9c1559e0 0x7f8c9c14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9bfff640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c14f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 -- 172.21.15.17:0/2155941718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 msgr2=0x7f8c9c14f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c14f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 -- 172.21.15.17:0/2155941718 >> v2:172.21.15.163:3300/0 conn(0x7f8c9c1559e0 msgr2=0x7f8c9c14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/2155941718 >> v2:172.21.15.163:3300/0 conn(0x7f8c9c1559e0 0x7f8c9c14ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.509+0000 7f8c9b7fe640 1 -- 172.21.15.17:0/2155941718 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0c4860 con 0x7f8c9c1550c0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c1554a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0abe90 tx=0x7f8c9002dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd988a125cc95eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8c9a7fc640 1 -- 172.21.15.17:0/2155941718 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9003b070 con 0x7f8c9c1550c0 2024-03-06T07:23:04.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8c9a7fc640 1 -- 172.21.15.17:0/2155941718 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9002ed70 con 0x7f8c9c1550c0 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8c9a7fc640 1 -- 172.21.15.17:0/2155941718 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90036660 con 0x7f8c9c1550c0 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/2155941718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 msgr2=0x7f8c9c1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c1554a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0abe90 tx=0x7f8c9002dfc0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/2155941718 shutdown_connections 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c14f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/2155941718 >> v2:172.21.15.163:3300/0 conn(0x7f8c9c1559e0 0x7f8c9c14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.510+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/2155941718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c1554a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.511+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/2155941718 >> 172.21.15.17:0/2155941718 conn(0x7f8c9c0c6710 msgr2=0x7f8c9c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.511+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/2155941718 shutdown_connections 2024-03-06T07:23:04.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.511+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/2155941718 wait complete. 2024-03-06T07:23:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.512+0000 7f8ca1d7a640 1 Processor -- start 2024-03-06T07:23:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.512+0000 7f8ca1d7a640 1 -- start start 2024-03-06T07:23:04.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.513+0000 7f8ca1d7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c9c1559e0 0x7f8c9c14be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c3210 con 0x7f8c9c1550c0 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c1520 con 0x7f8c9c1559e0 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0c3390 con 0x7f8c9c0aa180 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35662/0 (socket says 172.21.15.17:35662) 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 -- 172.21.15.17:0/421524210 learned_addr learned my addr 172.21.15.17:0/421524210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9bfff640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c9c1559e0 0x7f8c9c14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 -- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 msgr2=0x7f8c9c06deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c06deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 -- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c9c1559e0 msgr2=0x7f8c9c14be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c9c1559e0 0x7f8c9c14be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 -- 172.21.15.17:0/421524210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c0a9110 con 0x7f8c9c1550c0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c9affd640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e8a0 tx=0x7f8c8c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c017070 con 0x7f8c9c1550c0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00c940 con 0x7f8c9c1550c0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c0a8b50 con 0x7f8c9c1550c0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c00cd30 con 0x7f8c9c1550c0 2024-03-06T07:23:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.514+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c06e3f0 con 0x7f8c9c1550c0 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.515+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c04f330 con 0x7f8c9c1550c0 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.516+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c8c01a070 con 0x7f8c9c1550c0 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.517+0000 7f8c83fff640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 0x7f8c8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.517+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f8c8c08a450 con 0x7f8c9c1550c0 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.517+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 0x7f8c8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.517+0000 7f8c9b7fe640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 0x7f8c8806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c90038000 tx=0x7f8c90032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.521+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c8c0587d0 con 0x7f8c9c1550c0 2024-03-06T07:23:04.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.739+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c9c068450 con 0x7f8c9c1550c0 2024-03-06T07:23:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.739+0000 7f8c83fff640 1 -- 172.21.15.17:0/421524210 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f8c8c05c870 con 0x7f8c9c1550c0 2024-03-06T07:23:04.741 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:04.741 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:58.973213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6427":{"gid":6427,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6836/1549441863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1549441863},{"type":"v1","addr":"172.21.15.163:6836","nonce":1549441863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6427,"qdb_cluster":[6427]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:23:04.741 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 66 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 msgr2=0x7f8c8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 0x7f8c8806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c90038000 tx=0x7f8c90032000 comp rx=0 tx=0).stop 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 msgr2=0x7f8c9c14b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00e8a0 tx=0x7f8c8c00ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 shutdown_connections 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8806b4d0 0x7f8c8806d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c9c1559e0 0x7f8c9c14be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c9c1550c0 0x7f8c9c14b930 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 --2- 172.21.15.17:0/421524210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c9c0aa180 0x7f8c9c06deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 >> 172.21.15.17:0/421524210 conn(0x7f8c9c0c6710 msgr2=0x7f8c9c154600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 shutdown_connections 2024-03-06T07:23:04.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.741+0000 7f8ca1d7a640 1 -- 172.21.15.17:0/421524210 wait complete. 2024-03-06T07:23:04.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.854+0000 7f99717ac640 1 Processor -- start 2024-03-06T07:23:04.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.854+0000 7f99717ac640 1 -- start start 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f99717ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f996c1527b0 0x7f996c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f99717ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 0x7f996c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f99717ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c0c7c60 con 0x7f996c155b20 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f99717ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c159640 con 0x7f996c1527b0 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f99717ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c8df0 con 0x7f996c0c7180 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f996affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f996c1527b0 0x7f996c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f996affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f996c1527b0 0x7f996c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35678/0 (socket says 172.21.15.17:35678) 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f996a7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 0x7f996c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f996affd640 1 -- 172.21.15.17:0/1482437629 learned_addr learned my addr 172.21.15.17:0/1482437629 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.855+0000 7f996b7fe640 1 --1- 172.21.15.17:0/1482437629 >> v1:172.21.15.163:6789/0 conn(0x7f996c155b20 0x7f996c0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:45388/0 (socket says 172.21.15.17:45388) 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996affd640 1 -- 172.21.15.17:0/1482437629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 msgr2=0x7f996c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996affd640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 0x7f996c0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996affd640 1 -- 172.21.15.17:0/1482437629 >> v1:172.21.15.163:6789/0 conn(0x7f996c155b20 legacy=0x7f996c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f9969ffb640 1 -- 172.21.15.17:0/1482437629 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510600883 0 0) 0x7f9964001110 con 0x7f996c155b20 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996affd640 1 -- 172.21.15.17:0/1482437629 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0c8060 con 0x7f996c1527b0 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996a7fc640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 0x7f996c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f996affd640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c1527b0 0x7f996c159100 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f995c009e50 tx=0x7f995c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70525a2a1bf34999 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f9969ffb640 1 -- 172.21.15.17:0/1482437629 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c02ca70 con 0x7f996c1527b0 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f9969ffb640 1 -- 172.21.15.17:0/1482437629 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c02cc10 con 0x7f996c1527b0 2024-03-06T07:23:04.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.856+0000 7f9969ffb640 1 -- 172.21.15.17:0/1482437629 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0343f0 con 0x7f996c1527b0 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 -- 172.21.15.17:0/1482437629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c1527b0 msgr2=0x7f996c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c1527b0 0x7f996c159100 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f995c009e50 tx=0x7f995c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 -- 172.21.15.17:0/1482437629 shutdown_connections 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c0c7180 0x7f996c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 --2- 172.21.15.17:0/1482437629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c1527b0 0x7f996c159100 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 -- 172.21.15.17:0/1482437629 >> 172.21.15.17:0/1482437629 conn(0x7f996c0ac560 msgr2=0x7f996c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:04.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 -- 172.21.15.17:0/1482437629 shutdown_connections 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.857+0000 7f99717ac640 1 -- 172.21.15.17:0/1482437629 wait complete. 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.858+0000 7f99717ac640 1 Processor -- start 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.858+0000 7f99717ac640 1 -- start start 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f996c1527b0 0x7f996c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c155b20 0x7f996c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c155320 con 0x7f996c0c7180 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c7980 con 0x7f996c1527b0 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f99717ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c155600 con 0x7f996c155b20 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35680/0 (socket says 172.21.15.17:35680) 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 -- 172.21.15.17:0/3685321657 learned_addr learned my addr 172.21.15.17:0/3685321657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996a7fc640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f996c1527b0 0x7f996c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996b7fe640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c155b20 0x7f996c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 -- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c155b20 msgr2=0x7f996c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c155b20 0x7f996c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 -- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f996c1527b0 msgr2=0x7f996c0b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f996c1527b0 0x7f996c0b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 -- 172.21.15.17:0/3685321657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c1eac10 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.859+0000 7f996affd640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f995c009e50 tx=0x7f995c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.860+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c043020 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.860+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c03e450 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.860+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c1eaef0 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.860+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c009510 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.860+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c1eb4b0 con 0x7f996c0c7180 2024-03-06T07:23:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.861+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c04f330 con 0x7f996c0c7180 2024-03-06T07:23:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.861+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f995c005d10 con 0x7f996c0c7180 2024-03-06T07:23:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.861+0000 7f99537fe640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.861+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f995c0ad0c0 con 0x7f996c0c7180 2024-03-06T07:23:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.862+0000 7f996a7fc640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.862+0000 7f996a7fc640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 0x7f995806d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9960009560 tx=0x7f9960009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:04.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:04.866+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f995c07b3c0 con 0x7f996c0c7180 2024-03-06T07:23:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.084+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f996c06ce00 con 0x7f996c0c7180 2024-03-06T07:23:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.085+0000 7f99537fe640 1 -- 172.21.15.17:0/3685321657 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6263 (secure 0 0 0) 0x7f995c07f460 con 0x7f996c0c7180 2024-03-06T07:23:05.087 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:05.087 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":66,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":146,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":true,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:34.715155+0000","modified":"2024-03-06T07:22:58.973213+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6427":{"gid":6427,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6836/1549441863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1549441863},{"type":"v1","addr":"172.21.15.163:6836","nonce":1549441863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6427,"qdb_cluster":[6427]},"id":5},{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:22:51.793965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6375},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6375":{"gid":6375,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":7,"addr":"172.21.15.17:6835/285046827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":285046827},{"type":"v1","addr":"172.21.15.17:6835","nonce":285046827}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6375,"qdb_cluster":[6375]},"id":6}]} 2024-03-06T07:23:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 66 2024-03-06T07:23:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.087+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 msgr2=0x7f995806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.087+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 0x7f995806d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9960009560 tx=0x7f9960009190 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.087+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 msgr2=0x7f996c1ea630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.087+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f995c009e50 tx=0x7f995c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 shutdown_connections 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f996c155b20 0x7f996c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f996c1527b0 0x7f996c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 --2- 172.21.15.17:0/3685321657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f996c0c7180 0x7f996c1ea630 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 >> 172.21.15.17:0/3685321657 conn(0x7f996c0ac560 msgr2=0x7f996c0ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 shutdown_connections 2024-03-06T07:23:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.088+0000 7f99717ac640 1 -- 172.21.15.17:0/3685321657 wait complete. 2024-03-06T07:23:05.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.200+0000 7fa0ef1e9640 1 Processor -- start 2024-03-06T07:23:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ef1e9640 1 -- start start 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ef1e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 0x7fa0e80c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ef1e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e80c7c60 con 0x7fa0e80c7180 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ef1e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e80c8060 con 0x7fa0e81527b0 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ef1e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c8df0 con 0x7fa0e8159100 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0e7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa0e80c7180 0x7fa0e80c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58272/0 (socket says 172.21.15.17:58272) 2024-03-06T07:23:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0e7fff640 1 -- 172.21.15.17:0/2728016355 learned_addr learned my addr 172.21.15.17:0/2728016355 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.201+0000 7fa0ed786640 1 --2- 172.21.15.17:0/2728016355 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 0x7fa0e80c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397260860 0 0) 0x7fa0d4001110 con 0x7fa0e80c7180 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d8003520 con 0x7fa0e80c7180 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376609605 0 0) 0x7fa0e80c8060 con 0x7fa0e81527b0 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d4001110 con 0x7fa0e81527b0 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0ed786640 1 -- 172.21.15.17:0/2728016355 >> v1:172.21.15.163:6789/0 conn(0x7fa0e81527b0 legacy=0x7fa0e8155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0ed786640 1 -- 172.21.15.17:0/2728016355 >> v1:172.21.15.17:6789/0 conn(0x7fa0e80c7180 legacy=0x7fa0e80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.202+0000 7fa0ed786640 1 -- 172.21.15.17:0/2728016355 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e81595b0 con 0x7fa0e8159100 2024-03-06T07:23:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.203+0000 7fa0ed786640 1 --2- 172.21.15.17:0/2728016355 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 0x7fa0e80c88b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009dc0 tx=0x7fa0e002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17387cca1e2fc9dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.203+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0039070 con 0x7fa0e8159100 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.203+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e002cc40 con 0x7fa0e8159100 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.203+0000 7fa0e77fe640 1 -- 172.21.15.17:0/2728016355 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0034440 con 0x7fa0e8159100 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/2728016355 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 msgr2=0x7fa0e80c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/2728016355 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 0x7fa0e80c88b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009dc0 tx=0x7fa0e002be90 comp rx=0 tx=0).stop 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/2728016355 shutdown_connections 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/2728016355 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e8159100 0x7fa0e80c88b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/2728016355 >> 172.21.15.17:0/2728016355 conn(0x7fa0e80ac560 msgr2=0x7fa0e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/2728016355 shutdown_connections 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.204+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/2728016355 wait complete. 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.205+0000 7fa0ef1e9640 1 Processor -- start 2024-03-06T07:23:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.205+0000 7fa0ef1e9640 1 -- start start 2024-03-06T07:23:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e80c7180 0x7fa0e8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e8159100 0x7fa0e81eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8155320 con 0x7fa0e81527b0 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c7980 con 0x7fa0e8159100 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ef1e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8155600 con 0x7fa0e80c7180 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ecf85640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e80c7180 0x7fa0e8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35690/0 (socket says 172.21.15.17:35690) 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0ed786640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e8159100 0x7fa0e81eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 -- 172.21.15.17:0/3449458275 learned_addr learned my addr 172.21.15.17:0/3449458275 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 -- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e80c7180 msgr2=0x7fa0e8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e80c7180 0x7fa0e8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 -- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e8159100 msgr2=0x7fa0e81eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e8159100 0x7fa0e81eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 -- 172.21.15.17:0/3449458275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80c4840 con 0x7fa0e81527b0 2024-03-06T07:23:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e7fff640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c730 tx=0x7fa0d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d4005af0 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.206+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4007070 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.207+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e81eec60 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.207+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e81583b0 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.208+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e804f330 con 0x7fa0e81527b0 2024-03-06T07:23:05.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.208+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0d401a070 con 0x7fa0e81527b0 2024-03-06T07:23:05.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.208+0000 7fa0e57fa640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 0x7fa0cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.209+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa0d4089a50 con 0x7fa0e81527b0 2024-03-06T07:23:05.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.209+0000 7fa0ecf85640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 0x7fa0cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.209+0000 7fa0ecf85640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 0x7fa0cc06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc007fd0 tx=0x7fa0dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.213+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa0e80c7c60 con 0x7fa0e81527b0 2024-03-06T07:23:05.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.395+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0e806ce00 con 0x7fa0e81527b0 2024-03-06T07:23:05.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.397+0000 7fa0e57fa640 1 -- 172.21.15.17:0/3449458275 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+20800 (secure 0 0 0) 0x7fa0d4057e90 con 0x7fa0e81527b0 2024-03-06T07:23:05.398 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:05.399 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":80,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:22:54.958958+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:22:32.972999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-03-06T07:22:33.673658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 msgr2=0x7fa0cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 0x7fa0cc06d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc007fd0 tx=0x7fa0dc009040 comp rx=0 tx=0).stop 2024-03-06T07:23:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 msgr2=0x7fa0e814f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400c730 tx=0x7fa0d400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 shutdown_connections 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0cc06b4d0 0x7fa0cc06d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0e8159100 0x7fa0e81eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0e81527b0 0x7fa0e814f280 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 --2- 172.21.15.17:0/3449458275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0e80c7180 0x7fa0e8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.398+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 >> 172.21.15.17:0/3449458275 conn(0x7fa0e80ac560 msgr2=0x7fa0e80ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.399+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 shutdown_connections 2024-03-06T07:23:05.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.399+0000 7fa0ef1e9640 1 -- 172.21.15.17:0/3449458275 wait complete. 2024-03-06T07:23:05.411 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:23:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.514+0000 7fece940e640 1 Processor -- start 2024-03-06T07:23:05.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.514+0000 7fece940e640 1 -- start start 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.514+0000 7fece940e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fece40c27b0 0x7fece40c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.514+0000 7fece940e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fece4158720 0x7fece4158b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece940e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece4155150 con 0x7fece4152740 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece940e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece4155d00 con 0x7fece4158720 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece940e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece41553d0 con 0x7fece40c27b0 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fece4158720 0x7fece4158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fece4152740 0x7fece406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58278/0 (socket says 172.21.15.17:58278) 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece2ffd640 1 -- 172.21.15.17:0/2022319222 learned_addr learned my addr 172.21.15.17:0/2022319222 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fecdbfff640 1 --2- 172.21.15.17:0/2022319222 >> v2:172.21.15.163:3300/0 conn(0x7fece40c27b0 0x7fece40c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece37fe640 1 -- 172.21.15.17:0/2022319222 >> v2:172.21.15.163:3300/0 conn(0x7fece40c27b0 msgr2=0x7fece40c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece37fe640 1 --2- 172.21.15.17:0/2022319222 >> v2:172.21.15.163:3300/0 conn(0x7fece40c27b0 0x7fece40c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece37fe640 1 -- 172.21.15.17:0/2022319222 >> v1:172.21.15.17:6789/0 conn(0x7fece4152740 legacy=0x7fece406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece37fe640 1 -- 172.21.15.17:0/2022319222 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece41556b0 con 0x7fece4158720 2024-03-06T07:23:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.515+0000 7fece27fc640 1 -- 172.21.15.17:0/2022319222 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282805223 0 0) 0x7fecd4001110 con 0x7fece4152740 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece37fe640 1 --2- 172.21.15.17:0/2022319222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece4158720 0x7fece4158b00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fece406e8d0 tx=0x7fecdc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27adfbc580218397 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece27fc640 1 -- 172.21.15.17:0/2022319222 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc039070 con 0x7fece4158720 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece27fc640 1 -- 172.21.15.17:0/2022319222 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc02cbf0 con 0x7fece4158720 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece27fc640 1 -- 172.21.15.17:0/2022319222 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc034440 con 0x7fece4158720 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece940e640 1 -- 172.21.15.17:0/2022319222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece4158720 msgr2=0x7fece4158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.516+0000 7fece940e640 1 --2- 172.21.15.17:0/2022319222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece4158720 0x7fece4158b00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fece406e8d0 tx=0x7fecdc02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 -- 172.21.15.17:0/2022319222 shutdown_connections 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 --2- 172.21.15.17:0/2022319222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece4158720 0x7fece4158b00 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 --2- 172.21.15.17:0/2022319222 >> v2:172.21.15.163:3300/0 conn(0x7fece40c27b0 0x7fece40c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 -- 172.21.15.17:0/2022319222 >> 172.21.15.17:0/2022319222 conn(0x7fece40c41b0 msgr2=0x7fece40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 -- 172.21.15.17:0/2022319222 shutdown_connections 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.517+0000 7fece940e640 1 -- 172.21.15.17:0/2022319222 wait complete. 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 Processor -- start 2024-03-06T07:23:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 -- start start 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fece4152740 0x7fece41eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 0x7fece40c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4152b20 con 0x7fece40c27b0 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece406be80 con 0x7fece4152740 2024-03-06T07:23:05.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece940e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece406b6d0 con 0x7fece4158720 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35708/0 (socket says 172.21.15.17:35708) 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fece2ffd640 1 -- 172.21.15.17:0/1737976312 learned_addr learned my addr 172.21.15.17:0/1737976312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.518+0000 7fecdbfff640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fece4152740 0x7fece41eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece37fe640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 0x7fece40c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 -- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 msgr2=0x7fece40c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 0x7fece40c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 -- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fece4152740 msgr2=0x7fece41eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fece4152740 0x7fece41eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 -- 172.21.15.17:0/1737976312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4156a60 con 0x7fece40c27b0 2024-03-06T07:23:05.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece37fe640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 0x7fece40c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece2ffd640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c780 tx=0x7fecd400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4018070 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd40072d0 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40047e0 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece41ebc10 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.519+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece4158190 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.520+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece404f330 con 0x7fece40c27b0 2024-03-06T07:23:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.521+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecd4004980 con 0x7fece40c27b0 2024-03-06T07:23:05.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.521+0000 7fecdb7fe640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 0x7fecc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.521+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fecd4089840 con 0x7fece40c27b0 2024-03-06T07:23:05.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.521+0000 7fecdbfff640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 0x7fecc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.522+0000 7fecdbfff640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 0x7fecc406d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fece40c7080 tx=0x7feccc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.525+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fece4155150 con 0x7fece40c27b0 2024-03-06T07:23:05.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.745+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fece4158360 con 0x7fece40c27b0 2024-03-06T07:23:05.747 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:23:05.745+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.746+0000 7fecdb7fe640 1 -- 172.21.15.17:0/1737976312 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7fecd4057bc0 con 0x7fece40c27b0 2024-03-06T07:23:05.748 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 msgr2=0x7fecc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 0x7fecc406d9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fece40c7080 tx=0x7feccc009190 comp rx=0 tx=0).stop 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 msgr2=0x7fece4156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fecd400c780 tx=0x7fecd400cc40 comp rx=0 tx=0).stop 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 shutdown_connections 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fecc406b500 0x7fecc406d9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fece4158720 0x7fece40c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fece4152740 0x7fece41eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 --2- 172.21.15.17:0/1737976312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fece40c27b0 0x7fece4156520 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 >> 172.21.15.17:0/1737976312 conn(0x7fece40c41b0 msgr2=0x7fece4072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 shutdown_connections 2024-03-06T07:23:05.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.749+0000 7fece940e640 1 -- 172.21.15.17:0/1737976312 wait complete. 2024-03-06T07:23:05.752 INFO:tasks.ceph.mds.b.smithi163.stderr: -155> 2024-03-06T07:23:00.971+0000 7faebf760640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:05.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:23:05.776 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:23:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.864+0000 7f118b59e640 1 Processor -- start 2024-03-06T07:23:05.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.864+0000 7f118b59e640 1 -- start start 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118b59e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c0c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118b59e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f118c0c2d90 0x7f118c0ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118b59e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118c14f780 con 0x7f118c14f3a0 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118b59e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118c0aa7d0 con 0x7f118c0c2d90 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118b59e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0c1820 con 0x7f118c0c2230 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f118c0c2d90 0x7f118c0ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118a59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f118c14f3a0 0x7f118c0c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:45420/0 (socket says 172.21.15.17:45420) 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118a59c640 1 -- 172.21.15.17:0/3898306423 learned_addr learned my addr 172.21.15.17:0/3898306423 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f1189d9b640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c0c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118ad9d640 1 -- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 msgr2=0x7f118c0c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118ad9d640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c0c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118ad9d640 1 -- 172.21.15.17:0/3898306423 >> v1:172.21.15.163:6789/0 conn(0x7f118c14f3a0 legacy=0x7f118c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118ad9d640 1 -- 172.21.15.17:0/3898306423 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c0c1b00 con 0x7f118c0c2d90 2024-03-06T07:23:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.865+0000 7f118959a640 1 -- 172.21.15.17:0/3898306423 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897732798 0 0) 0x7f118c14f780 con 0x7f118c14f3a0 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.866+0000 7f118ad9d640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c0ab110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f11840097a0 tx=0x7f118402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336e5c1d570a7115 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.866+0000 7f118959a640 1 -- 172.21.15.17:0/3898306423 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11840315a0 con 0x7f118c0c2d90 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.866+0000 7f118959a640 1 -- 172.21.15.17:0/3898306423 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1184037070 con 0x7f118c0c2d90 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.866+0000 7f118959a640 1 -- 172.21.15.17:0/3898306423 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184031960 con 0x7f118c0c2d90 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 -- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 msgr2=0x7f118c0ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c0ab110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f11840097a0 tx=0x7f118402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:05.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 -- 172.21.15.17:0/3898306423 shutdown_connections 2024-03-06T07:23:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c0ab110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 --2- 172.21.15.17:0/3898306423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c0c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 -- 172.21.15.17:0/3898306423 >> 172.21.15.17:0/3898306423 conn(0x7f118c0a9ba0 msgr2=0x7f118c0c4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 -- 172.21.15.17:0/3898306423 shutdown_connections 2024-03-06T07:23:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.867+0000 7f118b59e640 1 -- 172.21.15.17:0/3898306423 wait complete. 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.868+0000 7f118b59e640 1 Processor -- start 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.868+0000 7f118b59e640 1 -- start start 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c1df910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 0x7f118c072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0ab650 con 0x7f118c14f3a0 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c152a70 con 0x7f118c0c2d90 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118b59e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0ab7d0 con 0x7f118c0c2230 2024-03-06T07:23:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 0x7f118c072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c1df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 0x7f118c072a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35724/0 (socket says 172.21.15.17:35724) 2024-03-06T07:23:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:44470/0 (socket says 172.21.15.17:44470) 2024-03-06T07:23:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118ad9d640 1 -- 172.21.15.17:0/450604924 learned_addr learned my addr 172.21.15.17:0/450604924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 -- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 msgr2=0x7f118c140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 -- 172.21.15.17:0/450604924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 msgr2=0x7f118c072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 0x7f118c072a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f118a59c640 1 -- 172.21.15.17:0/450604924 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c151b60 con 0x7f118c0c2230 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.869+0000 7f1189d9b640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f118a59c640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c1df910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c7c0 tx=0x7f117c00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c018070 con 0x7f118c0c2230 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c00a640 con 0x7f118c0c2230 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c1518c0 con 0x7f118c0c2230 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00aa30 con 0x7f118c0c2230 2024-03-06T07:23:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.870+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1eea70 con 0x7f118c0c2230 2024-03-06T07:23:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.871+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0693c0 con 0x7f118c0c2230 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.871+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f117c01b070 con 0x7f118c0c2230 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.872+0000 7f117affd640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 0x7f116c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.872+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f117c089c60 con 0x7f118c0c2230 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.872+0000 7f1189d9b640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 0x7f116c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.872+0000 7f1189d9b640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 0x7f116c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f118c0c6c90 tx=0x7f1180009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:05.877+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f117c057f90 con 0x7f118c0c2230 2024-03-06T07:23:06.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.095+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f118c1411e0 con 0x7f118c0c2230 2024-03-06T07:23:06.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.750+0000 7f117affd640 1 -- 172.21.15.17:0/450604924 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f117c05c030 con 0x7f118c0c2230 2024-03-06T07:23:06.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 msgr2=0x7f116c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 0x7f116c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f118c0c6c90 tx=0x7f1180009190 comp rx=0 tx=0).stop 2024-03-06T07:23:06.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 msgr2=0x7f118c1df910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c1df910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c7c0 tx=0x7f117c00cc80 comp rx=0 tx=0).stop 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 shutdown_connections 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f116c06b4d0 0x7f116c06d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c14f3a0 0x7f118c072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f118c0c2d90 0x7f118c140740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 --2- 172.21.15.17:0/450604924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f118c0c2230 0x7f118c1df910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.753+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 >> 172.21.15.17:0/450604924 conn(0x7f118c0a9ba0 msgr2=0x7f118c14dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:06.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.754+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 shutdown_connections 2024-03-06T07:23:06.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.754+0000 7f118b59e640 1 -- 172.21.15.17:0/450604924 wait complete. 2024-03-06T07:23:06.756 INFO:tasks.ceph.mds.c.smithi017.stderr: -368> 2024-03-06T07:22:55.640+0000 7fad48ca0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:06.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:06.783 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:23:06.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.876+0000 7f190ba84640 1 Processor -- start 2024-03-06T07:23:06.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.876+0000 7f190ba84640 1 -- start start 2024-03-06T07:23:06.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190ba84640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f19041587c0 0x7f1904160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190ba84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1904159740 con 0x7f1904152920 2024-03-06T07:23:06.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190ba84640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19041551d0 con 0x7f19040c2900 2024-03-06T07:23:06.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190ba84640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19040c8ab0 con 0x7f19041587c0 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f1909820640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1904152920 0x7f1904155970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58292/0 (socket says 172.21.15.17:58292) 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f1909820640 1 -- 172.21.15.17:0/1341196504 learned_addr learned my addr 172.21.15.17:0/1341196504 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190a021640 1 --2- 172.21.15.17:0/1341196504 >> v2:172.21.15.163:3300/0 conn(0x7f19041587c0 0x7f1904160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039612364 0 0) 0x7f1904159740 con 0x7f1904152920 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ec003520 con 0x7f1904152920 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.877+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3442225004 0 0) 0x7f18ec003520 con 0x7f1904152920 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 >> v2:172.21.15.163:3300/0 conn(0x7f19041587c0 msgr2=0x7f1904160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 --2- 172.21.15.17:0/1341196504 >> v2:172.21.15.163:3300/0 conn(0x7f19041587c0 0x7f1904160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 >> v1:172.21.15.163:6789/0 conn(0x7f19040c2900 legacy=0x7f19040c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040c8d90 con 0x7f1904152920 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f18fc001690 con 0x7f1904152920 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 >> v1:172.21.15.17:6789/0 conn(0x7f1904152920 legacy=0x7f1904155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f18ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18ec0043b0 0x7f18ec004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18ec005120 0x7f18ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040c8ab0 con 0x7f18ec0039b0 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec00c060 con 0x7f18ec0043b0 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec00c260 con 0x7f18ec005120 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f18ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190901f640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18ec0043b0 0x7f18ec004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190a021640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18ec005120 0x7f18ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 -- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18ec005120 msgr2=0x7f18ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18ec005120 0x7f18ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 -- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18ec0043b0 msgr2=0x7f18ec004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18ec0043b0 0x7f18ec004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f1909820640 1 -- 172.21.15.17:0/1341196504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec003520 con 0x7f18ec0039b0 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190a021640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18ec005120 0x7f18ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.878+0000 7f190a021640 1 -- 172.21.15.17:0/1341196504 reap_dead start 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f1909820640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f18ec003d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007fd0 tx=0x7f18ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6377ff98a87c99db server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc007d50 con 0x7f18ec0039b0 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc030070 con 0x7f18ec0039b0 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190881e640 1 -- 172.21.15.17:0/1341196504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc005d30 con 0x7f18ec0039b0 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190ba84640 1 -- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 msgr2=0x7f18ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190ba84640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f18ec003d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007fd0 tx=0x7f18ec005830 comp rx=0 tx=0).stop 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190ba84640 1 -- 172.21.15.17:0/1341196504 shutdown_connections 2024-03-06T07:23:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190ba84640 1 --2- 172.21.15.17:0/1341196504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f18ec003d90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.879+0000 7f190ba84640 1 -- 172.21.15.17:0/1341196504 >> 172.21.15.17:0/1341196504 conn(0x7f19040aa5c0 msgr2=0x7f19040aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.880+0000 7f190ba84640 1 -- 172.21.15.17:0/1341196504 shutdown_connections 2024-03-06T07:23:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.880+0000 7f190ba84640 1 -- 172.21.15.17:0/1341196504 wait complete. 2024-03-06T07:23:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 Processor -- start 2024-03-06T07:23:06.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 -- start start 2024-03-06T07:23:06.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19040a9850 0x7f190414da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 0x7f19041ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040c86c0 con 0x7f18ec0039b0 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904155450 con 0x7f19040a9850 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f190ba84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19040c8840 con 0x7f190414df80 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f1909820640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f1909820640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35744/0 (socket says 172.21.15.17:35744) 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.881+0000 7f1909820640 1 -- 172.21.15.17:0/2620174070 learned_addr learned my addr 172.21.15.17:0/2620174070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f190901f640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19040a9850 0x7f190414da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f190a021640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 0x7f19041ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 -- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 msgr2=0x7f19041ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 0x7f19041ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 -- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19040a9850 msgr2=0x7f190414da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19040a9850 0x7f190414da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 -- 172.21.15.17:0/2620174070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19041eab60 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f190a021640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 0x7f19041ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f1909820640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc046070 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc004c30 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc007d50 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19041eae40 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.882+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19041eb400 con 0x7f18ec0039b0 2024-03-06T07:23:06.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.883+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190404f330 con 0x7f18ec0039b0 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.883+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18fc007070 con 0x7f18ec0039b0 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.884+0000 7f19027fc640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 0x7f18f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.884+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f18fc0ab800 con 0x7f18ec0039b0 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.884+0000 7f190901f640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 0x7f18f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.885+0000 7f190901f640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 0x7f18f006d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19041eef20 tx=0x7f18f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:06.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:06.888+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f18fc079ae0 con 0x7f18ec0039b0 2024-03-06T07:23:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:07.073+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f190404f330 con 0x7f18ec0039b0 2024-03-06T07:23:08.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.005+0000 7f19027fc640 1 -- 172.21.15.17:0/2620174070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7f18fc07db80 con 0x7f18ec0039b0 2024-03-06T07:23:08.008 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 msgr2=0x7f18f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 0x7f18f006d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19041eef20 tx=0x7f18f8009040 comp rx=0 tx=0).stop 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 msgr2=0x7f19040ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc02e070 comp rx=0 tx=0).stop 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 shutdown_connections 2024-03-06T07:23:08.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f18f006b500 0x7f18f006d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f190414df80 0x7f19041ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19040a9850 0x7f190414da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 --2- 172.21.15.17:0/2620174070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18ec0039b0 0x7f19040ad6b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.009+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 >> 172.21.15.17:0/2620174070 conn(0x7f19040aa5c0 msgr2=0x7f1904153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.010+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 shutdown_connections 2024-03-06T07:23:08.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.010+0000 7f190ba84640 1 -- 172.21.15.17:0/2620174070 wait complete. 2024-03-06T07:23:08.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:23:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.176+0000 7f9e4005c640 1 Processor -- start 2024-03-06T07:23:08.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 -- start start 2024-03-06T07:23:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9e380c7f10 0x7f9e380c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e3806f2d0 0x7f9e381556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e38072620 con 0x7f9e380c8720 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e38072900 con 0x7f9e3806f2d0 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.177+0000 7f9e4005c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e38072be0 con 0x7f9e380c7f10 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e3806f2d0 0x7f9e381556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e3806f2d0 0x7f9e381556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35756/0 (socket says 172.21.15.17:35756) 2024-03-06T07:23:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 -- 172.21.15.17:0/2134430917 learned_addr learned my addr 172.21.15.17:0/2134430917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3ddf8640 1 --2- 172.21.15.17:0/2134430917 >> v2:172.21.15.163:3300/0 conn(0x7f9e380c7f10 0x7f9e380c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 -- 172.21.15.17:0/2134430917 >> v2:172.21.15.163:3300/0 conn(0x7f9e380c7f10 msgr2=0x7f9e380c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 --2- 172.21.15.17:0/2134430917 >> v2:172.21.15.163:3300/0 conn(0x7f9e380c7f10 0x7f9e380c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 -- 172.21.15.17:0/2134430917 >> v1:172.21.15.163:6789/0 conn(0x7f9e380c8720 legacy=0x7f9e3806ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.178+0000 7f9e3e5f9640 1 -- 172.21.15.17:0/2134430917 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e38072ec0 con 0x7f9e3806f2d0 2024-03-06T07:23:08.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e3e5f9640 1 --2- 172.21.15.17:0/2134430917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3806f2d0 0x7f9e381556e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9e380ab3a0 tx=0x7f9e2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886ec9f0a15ec882 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e3cdf6640 1 -- 172.21.15.17:0/2134430917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2802e070 con 0x7f9e3806f2d0 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e3cdf6640 1 -- 172.21.15.17:0/2134430917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28037070 con 0x7f9e3806f2d0 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e3cdf6640 1 -- 172.21.15.17:0/2134430917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28031a00 con 0x7f9e3806f2d0 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 -- 172.21.15.17:0/2134430917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3806f2d0 msgr2=0x7f9e381556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 --2- 172.21.15.17:0/2134430917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3806f2d0 0x7f9e381556e0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9e380ab3a0 tx=0x7f9e2802b880 comp rx=0 tx=0).stop 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 -- 172.21.15.17:0/2134430917 shutdown_connections 2024-03-06T07:23:08.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 --2- 172.21.15.17:0/2134430917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e3806f2d0 0x7f9e381556e0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 --2- 172.21.15.17:0/2134430917 >> v2:172.21.15.163:3300/0 conn(0x7f9e380c7f10 0x7f9e380c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.179+0000 7f9e4005c640 1 -- 172.21.15.17:0/2134430917 >> 172.21.15.17:0/2134430917 conn(0x7f9e380aa5c0 msgr2=0x7f9e380aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:08.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.180+0000 7f9e4005c640 1 -- 172.21.15.17:0/2134430917 shutdown_connections 2024-03-06T07:23:08.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.180+0000 7f9e4005c640 1 -- 172.21.15.17:0/2134430917 wait complete. 2024-03-06T07:23:08.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 Processor -- start 2024-03-06T07:23:08.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 -- start start 2024-03-06T07:23:08.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e3806f2d0 0x7f9e380ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e380c8720 0x7f9e381ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3806f8a0 con 0x7f9e380c7f10 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3806f720 con 0x7f9e380c8720 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e4005c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e380c82f0 con 0x7f9e3806f2d0 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e3d5f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e3d5f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35758/0 (socket says 172.21.15.17:35758) 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.181+0000 7f9e3d5f7640 1 -- 172.21.15.17:0/877774056 learned_addr learned my addr 172.21.15.17:0/877774056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3e5f9640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e380c8720 0x7f9e381ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3ddf8640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e3806f2d0 0x7f9e380ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 -- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e3806f2d0 msgr2=0x7f9e380ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e3806f2d0 0x7f9e380ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 -- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e380c8720 msgr2=0x7f9e381ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e380c8720 0x7f9e381ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 -- 172.21.15.17:0/877774056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e381edaf0 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e3d5f7640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00c3f0 tx=0x7f9e2c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0052d0 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c005490 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e381edd90 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c007260 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.182+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e381ee350 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.183+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3804f330 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.184+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e2c007400 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.184+0000 7f9e367fc640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 0x7f9e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.184+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f9e2c089a60 con 0x7f9e380c7f10 2024-03-06T07:23:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.184+0000 7f9e3ddf8640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 0x7f9e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:08.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.185+0000 7f9e3ddf8640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 0x7f9e2406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9e30003b80 tx=0x7f9e300098d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:08.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.188+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e2c057e60 con 0x7f9e380c7f10 2024-03-06T07:23:08.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:08.367+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9e380c8df0 con 0x7f9e380c7f10 2024-03-06T07:23:08.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:08.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:09.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.007+0000 7f9e367fc640 1 -- 172.21.15.17:0/877774056 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9e2c05ce20 con 0x7f9e380c7f10 2024-03-06T07:23:09.009 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:23:09.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 msgr2=0x7f9e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 0x7f9e2406d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9e30003b80 tx=0x7f9e300098d0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 msgr2=0x7f9e380c2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00c3f0 tx=0x7f9e2c00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 shutdown_connections 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e2406b4d0 0x7f9e2406d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e380c8720 0x7f9e381ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e380c7f10 0x7f9e380c2160 secure :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00c3f0 tx=0x7f9e2c00c8b0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 --2- 172.21.15.17:0/877774056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e3806f2d0 0x7f9e380ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 >> 172.21.15.17:0/877774056 conn(0x7f9e380aa5c0 msgr2=0x7f9e380aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.011+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 shutdown_connections 2024-03-06T07:23:09.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.012+0000 7f9e4005c640 1 -- 172.21.15.17:0/877774056 wait complete. 2024-03-06T07:23:09.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.172+0000 7f57af015640 1 Processor -- start 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.172+0000 7f57af015640 1 -- start start 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57af015640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57a814f790 0x7f57a81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57af015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57a8155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57af015640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a81563d0 con 0x7f57a80c1aa0 2024-03-06T07:23:09.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57af015640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80c2680 con 0x7f57a814f790 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57af015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c28d0 con 0x7f57a8155a40 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57a80c1aa0 0x7f57a80c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58298/0 (socket says 172.21.15.17:58298) 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a7fff640 1 -- 172.21.15.17:0/1796281109 learned_addr learned my addr 172.21.15.17:0/1796281109 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536251472 0 0) 0x7f57a81563d0 con 0x7f57a80c1aa0 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5798003520 con 0x7f57a80c1aa0 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57ad5b2640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57a8155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57acdb1640 1 --2- 172.21.15.17:0/1796281109 >> v2:172.21.15.163:3300/0 conn(0x7f57a814f790 0x7f57a81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1303269050 0 0) 0x7f5798003520 con 0x7f57a80c1aa0 2024-03-06T07:23:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 msgr2=0x7f57a8155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57a8155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 >> v2:172.21.15.163:3300/0 conn(0x7f57a814f790 msgr2=0x7f57a81529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 --2- 172.21.15.17:0/1796281109 >> v2:172.21.15.163:3300/0 conn(0x7f57a814f790 0x7f57a81529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80c2bb0 con 0x7f57a80c1aa0 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f5794001670 con 0x7f57a80c1aa0 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57acdb1640 1 --2- 172.21.15.17:0/1796281109 >> v2:172.21.15.163:3300/0 conn(0x7f57a814f790 0x7f57a81529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 >> v1:172.21.15.17:6789/0 conn(0x7f57a80c1aa0 legacy=0x7f57a80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57980039b0 0x7f5798003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.173+0000 7f57a77fe640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5798004800 0x7f5798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a77fe640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c2680 con 0x7f5798004800 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c28d0 con 0x7f57980039b0 2024-03-06T07:23:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579800dc80 con 0x7f57a8155a40 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5798004800 0x7f5798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57acdb1640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57980039b0 0x7f5798003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57ad5b2640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 -- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 msgr2=0x7f57980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8155a40 0x7f57980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 -- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57980039b0 msgr2=0x7f5798003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57980039b0 0x7f5798003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57a7fff640 1 -- 172.21.15.17:0/1796281109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5798003520 con 0x7f5798004800 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.174+0000 7f57acdb1640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57980039b0 0x7f5798003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57a7fff640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5798004800 0x7f5798004c80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5794001eb0 tx=0x7f5798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a26ad6559ea72e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794007d70 con 0x7f5798004800 2024-03-06T07:23:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794030070 con 0x7f5798004800 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57a77fe640 1 -- 172.21.15.17:0/1796281109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794005d30 con 0x7f5798004800 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 -- 172.21.15.17:0/1796281109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5798004800 msgr2=0x7f5798004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 --2- 172.21.15.17:0/1796281109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5798004800 0x7f5798004c80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5794001eb0 tx=0x7f5798005830 comp rx=0 tx=0).stop 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57ad5b2640 1 -- 172.21.15.17:0/1796281109 reap_dead start 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 -- 172.21.15.17:0/1796281109 shutdown_connections 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 -- 172.21.15.17:0/1796281109 >> 172.21.15.17:0/1796281109 conn(0x7f57a80c6710 msgr2=0x7f57a80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 -- 172.21.15.17:0/1796281109 shutdown_connections 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.175+0000 7f57af015640 1 -- 172.21.15.17:0/1796281109 wait complete. 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.176+0000 7f57af015640 1 Processor -- start 2024-03-06T07:23:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.176+0000 7f57af015640 1 -- start start 2024-03-06T07:23:09.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 0x7f57a81db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 0x7f57a81dcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c0d60 con 0x7f57a8142670 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8155520 con 0x7f5798004800 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57af015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80c5e30 con 0x7f57a8142ba0 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35776/0 (socket says 172.21.15.17:35776) 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.177+0000 7f57a7fff640 1 -- 172.21.15.17:0/2852345678 learned_addr learned my addr 172.21.15.17:0/2852345678 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:09.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57ad5b2640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 0x7f57a81dcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57acdb1640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 0x7f57a81db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 -- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 msgr2=0x7f57a81dcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 0x7f57a81dcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 -- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 msgr2=0x7f57a81db7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 0x7f57a81db7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 -- 172.21.15.17:0/2852345678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a81dbcf0 con 0x7f57a8142670 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57ad5b2640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 0x7f57a81dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57acdb1640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 0x7f57a81db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a7fff640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5794001050 tx=0x7f5794000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794045070 con 0x7f57a8142670 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794004bf0 con 0x7f57a8142670 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a81e7780 con 0x7f57a8142670 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794001cc0 con 0x7f57a8142670 2024-03-06T07:23:09.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.178+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81e6a40 con 0x7f57a8142670 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.179+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8067ad0 con 0x7f57a8142670 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.179+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5794007070 con 0x7f57a8142670 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.180+0000 7f57a57fa640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.180+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f57940ab420 con 0x7f57a8142670 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.180+0000 7f57acdb1640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.181+0000 7f57acdb1640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f579c006fd0 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.184+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57a8067ad0 con 0x7f57a8142670 2024-03-06T07:23:09.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.377+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a81e6850 con 0x7f57a8142670 2024-03-06T07:23:09.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.378+0000 7f57a57fa640 1 -- 172.21.15.17:0/2852345678 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17280 (secure 0 0 0) 0x7f5794079b30 con 0x7f57a8142670 2024-03-06T07:23:09.380 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:09.380 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":84,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:09.006305+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.163:6834/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.17:6835/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.17:6833/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:09.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 msgr2=0x7f578c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 0x7f578c06d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f579c006fd0 tx=0x7f579c008040 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 msgr2=0x7f57a81e7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5794001050 tx=0x7f5794000fe0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 shutdown_connections 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f578c06b4d0 0x7f578c06d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57a8142ba0 0x7f57a81dcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57a8142670 0x7f57a81e7240 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 --2- 172.21.15.17:0/2852345678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798004800 0x7f57a81db7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 >> 172.21.15.17:0/2852345678 conn(0x7f57a80c6710 msgr2=0x7f57a80c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 shutdown_connections 2024-03-06T07:23:09.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.380+0000 7f57af015640 1 -- 172.21.15.17:0/2852345678 wait complete. 2024-03-06T07:23:09.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:22:47.593104+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:22:48.766993+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:09.393 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.496+0000 7fdcbea42640 1 Processor -- start 2024-03-06T07:23:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.496+0000 7fdcbea42640 1 -- start start 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbea42640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb80c4c60 0x7fdcb80c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbea42640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcb8152e40 0x7fdcb80724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbea42640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb8070eb0 con 0x7fdcb8070a50 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbea42640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb8156940 con 0x7fdcb8152e40 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbea42640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c80b0 con 0x7fdcb80c4c60 2024-03-06T07:23:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbcfdf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcb8152e40 0x7fdcb80724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcb77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdcb8070a50 0x7fdcb8156dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58308/0 (socket says 172.21.15.17:58308) 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcb77fe640 1 -- 172.21.15.17:0/1801104421 learned_addr learned my addr 172.21.15.17:0/1801104421 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcb7fff640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb80c4c60 0x7fdcb80c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbcfdf640 1 -- 172.21.15.17:0/1801104421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb80c4c60 msgr2=0x7fdcb80c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbcfdf640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb80c4c60 0x7fdcb80c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbcfdf640 1 -- 172.21.15.17:0/1801104421 >> v1:172.21.15.17:6789/0 conn(0x7fdcb8070a50 legacy=0x7fdcb8156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcbcfdf640 1 -- 172.21.15.17:0/1801104421 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80c8390 con 0x7fdcb8152e40 2024-03-06T07:23:09.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.497+0000 7fdcb6ffd640 1 -- 172.21.15.17:0/1801104421 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623720766 0 0) 0x7fdcb8070eb0 con 0x7fdcb8070a50 2024-03-06T07:23:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.498+0000 7fdcbcfdf640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb8152e40 0x7fdcb80724d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ba276c8bfe3c80 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.498+0000 7fdcb6ffd640 1 -- 172.21.15.17:0/1801104421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0039070 con 0x7fdcb8152e40 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.498+0000 7fdcb6ffd640 1 -- 172.21.15.17:0/1801104421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb002cbc0 con 0x7fdcb8152e40 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.498+0000 7fdcb6ffd640 1 -- 172.21.15.17:0/1801104421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0034440 con 0x7fdcb8152e40 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 -- 172.21.15.17:0/1801104421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb8152e40 msgr2=0x7fdcb80724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb8152e40 0x7fdcb80724d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009d90 tx=0x7fdcb002be10 comp rx=0 tx=0).stop 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 -- 172.21.15.17:0/1801104421 shutdown_connections 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb8152e40 0x7fdcb80724d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1801104421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb80c4c60 0x7fdcb80c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 -- 172.21.15.17:0/1801104421 >> 172.21.15.17:0/1801104421 conn(0x7fdcb80aa6a0 msgr2=0x7fdcb80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 -- 172.21.15.17:0/1801104421 shutdown_connections 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.499+0000 7fdcbea42640 1 -- 172.21.15.17:0/1801104421 wait complete. 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.500+0000 7fdcbea42640 1 Processor -- start 2024-03-06T07:23:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.500+0000 7fdcbea42640 1 -- start start 2024-03-06T07:23:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb8070a50 0x7fdcb81ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8152e40 0x7fdcb8149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8072a10 con 0x7fdcb80c4c60 2024-03-06T07:23:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb81532f0 con 0x7fdcb8152e40 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbea42640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8072b90 con 0x7fdcb8070a50 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35788/0 (socket says 172.21.15.17:35788) 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 -- 172.21.15.17:0/1628070197 learned_addr learned my addr 172.21.15.17:0/1628070197 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb7fff640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb8070a50 0x7fdcb81ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcbcfdf640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8152e40 0x7fdcb8149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 -- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb8070a50 msgr2=0x7fdcb81ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb8070a50 0x7fdcb81ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 -- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8152e40 msgr2=0x7fdcb8149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8152e40 0x7fdcb8149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 -- 172.21.15.17:0/1628070197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80c2890 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.501+0000 7fdcb77fe640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c4d0 tx=0x7fdca800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.502+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8018070 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.502+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca8007500 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.502+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb806dbe0 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.502+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8004a20 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.502+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb81eb380 con 0x7fdcb80c4c60 2024-03-06T07:23:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.503+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb804f330 con 0x7fdcb80c4c60 2024-03-06T07:23:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.503+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdca801b070 con 0x7fdcb80c4c60 2024-03-06T07:23:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.504+0000 7fdcb4ff9640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 0x7fdca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.504+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fdca8089700 con 0x7fdcb80c4c60 2024-03-06T07:23:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.504+0000 7fdcb7fff640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 0x7fdca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.504+0000 7fdcb7fff640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 0x7fdca406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80a8ab0 tx=0x7fdcac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.508+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdcb80c2890 con 0x7fdcb80c4c60 2024-03-06T07:23:09.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.689+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcb8073520 con 0x7fdcb80c4c60 2024-03-06T07:23:09.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.689+0000 7fdcb4ff9640 1 -- 172.21.15.17:0/1628070197 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdca8058380 con 0x7fdcb80c4c60 2024-03-06T07:23:09.691 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 msgr2=0x7fdca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 0x7fdca406d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80a8ab0 tx=0x7fdcac008040 comp rx=0 tx=0).stop 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 msgr2=0x7fdcb80c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c4d0 tx=0x7fdca800c990 comp rx=0 tx=0).stop 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 shutdown_connections 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdca406b4d0 0x7fdca406d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcb8152e40 0x7fdcb8149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcb80c4c60 0x7fdcb80c1f90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 --2- 172.21.15.17:0/1628070197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcb8070a50 0x7fdcb81ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 >> 172.21.15.17:0/1628070197 conn(0x7fdcb80aa6a0 msgr2=0x7fdcb8070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 shutdown_connections 2024-03-06T07:23:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.691+0000 7fdcbea42640 1 -- 172.21.15.17:0/1628070197 wait complete. 2024-03-06T07:23:09.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:23:09.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.806+0000 7f8d21b81640 1 Processor -- start 2024-03-06T07:23:09.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 -- start start 2024-03-06T07:23:09.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8d1c0a8290 0x7f8d1c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d1c0c69a0 0x7f8d1c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c153f00 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0c62a0 con 0x7f8d1c0a8290 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d21b81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0706e0 con 0x7f8d1c0c4740 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d1bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d1c0c69a0 0x7f8d1c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d1bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8d1c0c69a0 0x7f8d1c072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35794/0 (socket says 172.21.15.17:35794) 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3824894392 learned_addr learned my addr 172.21.15.17:0/3824894392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d1affd640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.807+0000 7f8d1b7fe640 1 --2- 172.21.15.17:0/3824894392 >> v2:172.21.15.163:3300/0 conn(0x7f8d1c0a8290 0x7f8d1c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.808+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3824894392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 msgr2=0x7f8d1c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.808+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.808+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3824894392 >> v2:172.21.15.163:3300/0 conn(0x7f8d1c0a8290 msgr2=0x7f8d1c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:09.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.808+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3824894392 >> v2:172.21.15.163:3300/0 conn(0x7f8d1c0a8290 0x7f8d1c1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.808+0000 7f8d1bfff640 1 -- 172.21.15.17:0/3824894392 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0c4c00 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c072680 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0735b0 tx=0x7f8d1402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ac688ffb599d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d1a7fc640 1 -- 172.21.15.17:0/3824894392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1402c950 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d1a7fc640 1 -- 172.21.15.17:0/3824894392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1402caf0 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d1a7fc640 1 -- 172.21.15.17:0/3824894392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140343f0 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 -- 172.21.15.17:0/3824894392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 msgr2=0x7f8d1c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c072680 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0735b0 tx=0x7f8d1402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 -- 172.21.15.17:0/3824894392 shutdown_connections 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c072680 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 --2- 172.21.15.17:0/3824894392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 --2- 172.21.15.17:0/3824894392 >> v2:172.21.15.163:3300/0 conn(0x7f8d1c0a8290 0x7f8d1c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 -- 172.21.15.17:0/3824894392 >> 172.21.15.17:0/3824894392 conn(0x7f8d1c0c5560 msgr2=0x7f8d1c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 -- 172.21.15.17:0/3824894392 shutdown_connections 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.809+0000 7f8d21b81640 1 -- 172.21.15.17:0/3824894392 wait complete. 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.810+0000 7f8d21b81640 1 Processor -- start 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 -- start start 2024-03-06T07:23:09.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d1c0a8290 0x7f8d1c0a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06fc80 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c06f9a0 con 0x7f8d1c0a8290 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d21b81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0c6e10 con 0x7f8d1c0c4740 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d1bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35800/0 (socket says 172.21.15.17:35800) 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.811+0000 7f8d1bfff640 1 -- 172.21.15.17:0/2496062676 learned_addr learned my addr 172.21.15.17:0/2496062676 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1affd640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1b7fe640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d1c0a8290 0x7f8d1c0a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 -- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 msgr2=0x7f8d1c0ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 -- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d1c0a8290 msgr2=0x7f8d1c0a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d1c0a8290 0x7f8d1c0a88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 -- 172.21.15.17:0/2496062676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0ac450 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d1bfff640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8d1402c3c0 tx=0x7f8d14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14043020 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1403e450 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14009510 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c0ac730 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.812+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c0a8e20 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.813+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c04f330 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.814+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d14046020 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.814+0000 7f8d03fff640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.814+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f8d140ac9b0 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.814+0000 7f8d1b7fe640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.815+0000 7f8d1b7fe640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 0x7f8d0806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0a90f0 tx=0x7f8d0c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.818+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d1404d030 con 0x7f8d1c0c69a0 2024-03-06T07:23:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.998+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d1c0ac090 con 0x7f8d1c0c69a0 2024-03-06T07:23:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:09.998+0000 7f8d03fff640 1 -- 172.21.15.17:0/2496062676 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v84) v1 ==== 105+0+11 (secure 0 0 0) 0x7f8d1407af50 con 0x7f8d1c0c69a0 2024-03-06T07:23:10.000 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 msgr2=0x7f8d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 0x7f8d0806d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0a90f0 tx=0x7f8d0c009190 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 msgr2=0x7f8d1c1ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8d1402c3c0 tx=0x7f8d14004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 shutdown_connections 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d1c0c69a0 0x7f8d1c1ec290 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d1c0c4740 0x7f8d1c0ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 --2- 172.21.15.17:0/2496062676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d1c0a8290 0x7f8d1c0a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 >> 172.21.15.17:0/2496062676 conn(0x7f8d1c0c5560 msgr2=0x7f8d1c0a9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 shutdown_connections 2024-03-06T07:23:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.000+0000 7f8d21b81640 1 -- 172.21.15.17:0/2496062676 wait complete. 2024-03-06T07:23:10.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:23:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.115+0000 7efc6c993640 1 Processor -- start 2024-03-06T07:23:10.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.115+0000 7efc6c993640 1 -- start start 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc6c993640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc680c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc6c993640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc6806cad0 0x7efc68154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc6c993640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc680c2790 con 0x7efc6814ef70 2024-03-06T07:23:10.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc6c993640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680c3090 con 0x7efc6806cad0 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc6c993640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6814f350 con 0x7efc6806c1f0 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc6806cad0 0x7efc68154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc6806cad0 0x7efc68154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35806/0 (socket says 172.21.15.17:35806) 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 -- 172.21.15.17:0/3811696469 learned_addr learned my addr 172.21.15.17:0/3811696469 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc65d9b640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc680c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 -- 172.21.15.17:0/3811696469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 msgr2=0x7efc680c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc680c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 -- 172.21.15.17:0/3811696469 >> v1:172.21.15.163:6789/0 conn(0x7efc6814ef70 legacy=0x7efc680c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.116+0000 7efc66d9d640 1 -- 172.21.15.17:0/3811696469 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680c1540 con 0x7efc6806cad0 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.117+0000 7efc66d9d640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6806cad0 0x7efc68154c30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d150 tx=0x7efc5802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e605d5d57b3c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.117+0000 7efc6559a640 1 -- 172.21.15.17:0/3811696469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5802ea80 con 0x7efc6806cad0 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.117+0000 7efc6559a640 1 -- 172.21.15.17:0/3811696469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5802ec20 con 0x7efc6806cad0 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.117+0000 7efc6559a640 1 -- 172.21.15.17:0/3811696469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58009600 con 0x7efc6806cad0 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 -- 172.21.15.17:0/3811696469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6806cad0 msgr2=0x7efc68154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6806cad0 0x7efc68154c30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d150 tx=0x7efc5802e0e0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 -- 172.21.15.17:0/3811696469 shutdown_connections 2024-03-06T07:23:10.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6806cad0 0x7efc68154c30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 --2- 172.21.15.17:0/3811696469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc680c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 -- 172.21.15.17:0/3811696469 >> 172.21.15.17:0/3811696469 conn(0x7efc680c6710 msgr2=0x7efc680c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 -- 172.21.15.17:0/3811696469 shutdown_connections 2024-03-06T07:23:10.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.118+0000 7efc6c993640 1 -- 172.21.15.17:0/3811696469 wait complete. 2024-03-06T07:23:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.119+0000 7efc6c993640 1 Processor -- start 2024-03-06T07:23:10.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.119+0000 7efc6c993640 1 -- start start 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc68150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc6806cad0 0x7efc68150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc68155750 con 0x7efc6814ef70 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6806cf60 con 0x7efc6806cad0 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6c993640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc68155170 con 0x7efc6806c1f0 2024-03-06T07:23:10.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35822/0 (socket says 172.21.15.17:35822) 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 -- 172.21.15.17:0/3223698553 learned_addr learned my addr 172.21.15.17:0/3223698553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc6659c640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc68150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc65d9b640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc6806cad0 0x7efc68150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 -- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 msgr2=0x7efc68150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc68150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 -- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc6806cad0 msgr2=0x7efc68150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc6806cad0 0x7efc68150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 -- 172.21.15.17:0/3223698553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68152a30 con 0x7efc6814ef70 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.120+0000 7efc66d9d640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efc58038040 tx=0x7efc58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.121+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58045020 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.121+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58009e30 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.121+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc680c02c0 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.121+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5802ea80 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.121+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc680c05a0 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.122+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6804f330 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.122+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc58048020 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.122+0000 7efc5effd640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 0x7efc4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.122+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7efc580aebc0 con 0x7efc6814ef70 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.122+0000 7efc6659c640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 0x7efc4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.123+0000 7efc6659c640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 0x7efc4806d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc600016e0 tx=0x7efc60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.127+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc5804f030 con 0x7efc6814ef70 2024-03-06T07:23:10.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.307+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7efc680c1110 con 0x7efc6814ef70 2024-03-06T07:23:10.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.307+0000 7efc5effd640 1 -- 172.21.15.17:0/3223698553 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v84) v1 ==== 101+0+11 (secure 0 0 0) 0x7efc5807d170 con 0x7efc6814ef70 2024-03-06T07:23:10.309 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 msgr2=0x7efc4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 0x7efc4806d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7efc600016e0 tx=0x7efc60009040 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 msgr2=0x7efc68149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efc58038040 tx=0x7efc58004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 shutdown_connections 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc4806b500 0x7efc4806d9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc6814ef70 0x7efc68149b00 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc6806cad0 0x7efc68150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 --2- 172.21.15.17:0/3223698553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc6806c1f0 0x7efc68150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 >> 172.21.15.17:0/3223698553 conn(0x7efc680c6710 msgr2=0x7efc680c3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 shutdown_connections 2024-03-06T07:23:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.309+0000 7efc6c993640 1 -- 172.21.15.17:0/3223698553 wait complete. 2024-03-06T07:23:10.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:10.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.420+0000 7f28828a4640 1 Processor -- start 2024-03-06T07:23:10.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.420+0000 7f28828a4640 1 -- start start 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f28828a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f287c159270 0x7f287c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f28828a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c0c73d0 con 0x7f287c14f360 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f28828a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c0be2e0 con 0x7f287c0c88c0 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f28828a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c159700 con 0x7f287c159270 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f287bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f287c14f360 0x7f287c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58312/0 (socket says 172.21.15.17:58312) 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f287bfff640 1 -- 172.21.15.17:0/2838342330 learned_addr learned my addr 172.21.15.17:0/2838342330 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f2880e41640 1 --2- 172.21.15.17:0/2838342330 >> v2:172.21.15.163:3300/0 conn(0x7f287c159270 0x7f287c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.421+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594237182 0 0) 0x7f287c0c73d0 con 0x7f287c14f360 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2864003520 con 0x7f287c14f360 2024-03-06T07:23:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624824525 0 0) 0x7f287c0be2e0 con 0x7f287c0c88c0 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f2880e41640 1 -- 172.21.15.17:0/2838342330 >> v1:172.21.15.163:6789/0 conn(0x7f287c0c88c0 legacy=0x7f287c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f2880e41640 1 -- 172.21.15.17:0/2838342330 >> v1:172.21.15.17:6789/0 conn(0x7f287c14f360 legacy=0x7f287c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f2880e41640 1 -- 172.21.15.17:0/2838342330 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0c7670 con 0x7f287c159270 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.422+0000 7f2880e41640 1 --2- 172.21.15.17:0/2838342330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c159270 0x7f287c155c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614b1bb68ee6124c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874039070 con 0x7f287c159270 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287402cc60 con 0x7f287c159270 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f287affd640 1 -- 172.21.15.17:0/2838342330 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874034660 con 0x7f287c159270 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 -- 172.21.15.17:0/2838342330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c159270 msgr2=0x7f287c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 --2- 172.21.15.17:0/2838342330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c159270 0x7f287c155c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2874009d90 tx=0x7f287402beb0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 -- 172.21.15.17:0/2838342330 shutdown_connections 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 --2- 172.21.15.17:0/2838342330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c159270 0x7f287c155c10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 -- 172.21.15.17:0/2838342330 >> 172.21.15.17:0/2838342330 conn(0x7f287c0ac570 msgr2=0x7f287c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 -- 172.21.15.17:0/2838342330 shutdown_connections 2024-03-06T07:23:10.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.423+0000 7f28828a4640 1 -- 172.21.15.17:0/2838342330 wait complete. 2024-03-06T07:23:10.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.424+0000 7f28828a4640 1 Processor -- start 2024-03-06T07:23:10.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.424+0000 7f28828a4640 1 -- start start 2024-03-06T07:23:10.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c0c88c0 0x7f287c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f287c159270 0x7f287c0c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c7c30 con 0x7f287c14f360 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c5a50 con 0x7f287c0c88c0 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f28828a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0c5bd0 con 0x7f287c159270 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f287b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f287b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35838/0 (socket says 172.21.15.17:35838) 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f287b7fe640 1 -- 172.21.15.17:0/1534610605 learned_addr learned my addr 172.21.15.17:0/1534610605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f2880e41640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f287c159270 0x7f287c0c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.425+0000 7f287bfff640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c0c88c0 0x7f287c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 -- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f287c159270 msgr2=0x7f287c0c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f287c159270 0x7f287c0c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 -- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c0c88c0 msgr2=0x7f287c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c0c88c0 0x7f287c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 -- 172.21.15.17:0/1534610605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0be670 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f287b7fe640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c550 tx=0x7f286c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c017070 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286c0035f0 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c0be950 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0039e0 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.426+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c0bef80 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.427+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c04f330 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.427+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f286c01a070 con 0x7f287c14f360 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.428+0000 7f2878ff9640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 0x7f286806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.428+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f286c089790 con 0x7f287c14f360 2024-03-06T07:23:10.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.428+0000 7f287bfff640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 0x7f286806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.428+0000 7f287bfff640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 0x7f286806d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2870005880 tx=0x7f2870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.432+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f286c057dc0 con 0x7f287c14f360 2024-03-06T07:23:10.642 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-03-06T07:23:10.640+0000 7f4fd6ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:10.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.653+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f287c0c8d50 con 0x7f287c14f360 2024-03-06T07:23:10.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.654+0000 7f2878ff9640 1 -- 172.21.15.17:0/1534610605 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4986 (secure 0 0 0) 0x7f286c05be60 con 0x7f287c14f360 2024-03-06T07:23:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:10.655 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3589667974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3589667974},{"type":"v1","addr":"172.21.15.163:6836","nonce":3589667974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4215297482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":4215297482},{"type":"v1","addr":"172.21.15.17:6835","nonce":4215297482}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:23:09.755245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6106":{"gid":6106,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":13,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":6}]} 2024-03-06T07:23:10.656 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 72 2024-03-06T07:23:10.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 msgr2=0x7f286806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 0x7f286806d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2870005880 tx=0x7f2870009040 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 msgr2=0x7f287c0abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c550 tx=0x7f286c00ca10 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 shutdown_connections 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f286806b4d0 0x7f286806d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f287c159270 0x7f287c0c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f287c14f360 0x7f287c0abce0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 --2- 172.21.15.17:0/1534610605 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f287c0c88c0 0x7f287c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 >> 172.21.15.17:0/1534610605 conn(0x7f287c0ac570 msgr2=0x7f287c0c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 shutdown_connections 2024-03-06T07:23:10.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.656+0000 7f28828a4640 1 -- 172.21.15.17:0/1534610605 wait complete. 2024-03-06T07:23:10.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.768+0000 7fa9376a4640 1 Processor -- start 2024-03-06T07:23:10.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.768+0000 7fa9376a4640 1 -- start start 2024-03-06T07:23:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa9376a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 0x7fa9300bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa9376a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93006c8e0 con 0x7fa930073250 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa9376a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93006ce80 con 0x7fa93006e490 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa9376a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c1b00 con 0x7fa930072e70 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa934c3f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa930073250 0x7fa9300c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59616/0 (socket says 172.21.15.17:59616) 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa934c3f640 1 -- 172.21.15.17:0/2637770140 learned_addr learned my addr 172.21.15.17:0/2637770140 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.769+0000 7fa935440640 1 --2- 172.21.15.17:0/2637770140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 0x7fa9300bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199231164 0 0) 0x7fa93006c8e0 con 0x7fa930073250 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa91c003520 con 0x7fa930073250 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603061999 0 0) 0x7fa93006ce80 con 0x7fa93006e490 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa93006c8e0 con 0x7fa93006e490 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa935440640 1 -- 172.21.15.17:0/2637770140 >> v1:172.21.15.163:6789/0 conn(0x7fa93006e490 legacy=0x7fa93015fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa935440640 1 -- 172.21.15.17:0/2637770140 >> v1:172.21.15.17:6789/0 conn(0x7fa930073250 legacy=0x7fa9300c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.770+0000 7fa935440640 1 -- 172.21.15.17:0/2637770140 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93006cba0 con 0x7fa930072e70 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa935440640 1 --2- 172.21.15.17:0/2637770140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 0x7fa9300bd510 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0097d0 tx=0x7fa92c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b185783c9ead3e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c035020 con 0x7fa930072e70 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c031800 con 0x7fa930072e70 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa927fff640 1 -- 172.21.15.17:0/2637770140 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c031bb0 con 0x7fa930072e70 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 -- 172.21.15.17:0/2637770140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 msgr2=0x7fa9300bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 --2- 172.21.15.17:0/2637770140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 0x7fa9300bd510 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa92c0097d0 tx=0x7fa92c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 -- 172.21.15.17:0/2637770140 shutdown_connections 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 --2- 172.21.15.17:0/2637770140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa930072e70 0x7fa9300bd510 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 -- 172.21.15.17:0/2637770140 >> 172.21.15.17:0/2637770140 conn(0x7fa93006b1a0 msgr2=0x7fa93006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 -- 172.21.15.17:0/2637770140 shutdown_connections 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.771+0000 7fa9376a4640 1 -- 172.21.15.17:0/2637770140 wait complete. 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.772+0000 7fa9376a4640 1 Processor -- start 2024-03-06T07:23:10.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.772+0000 7fa9376a4640 1 -- start start 2024-03-06T07:23:10.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa93006e490 0x7fa9300bda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 0x7fa9300a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c6760 con 0x7fa930072e70 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300c4a00 con 0x7fa930073250 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa9376a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300bdf90 con 0x7fa93006e490 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa934c3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa934c3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51910/0 (socket says 172.21.15.17:51910) 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa934c3f640 1 -- 172.21.15.17:0/953212240 learned_addr learned my addr 172.21.15.17:0/953212240 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa935440640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa93006e490 0x7fa9300bda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.773+0000 7fa935c41640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 0x7fa9300a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 -- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa93006e490 msgr2=0x7fa9300bda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa93006e490 0x7fa9300bda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 -- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 msgr2=0x7fa9300a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 0x7fa9300a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 -- 172.21.15.17:0/953212240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9301eb140 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa935c41640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 0x7fa9300a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa934c3f640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa92000c440 tx=0x7fa92000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920017070 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920005ce0 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9301eb420 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920007270 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.774+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa930156820 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.775+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93004f330 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.775+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa92001a070 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.776+0000 7fa925ffb640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.776+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fa920089620 con 0x7fa930072e70 2024-03-06T07:23:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.776+0000 7fa935440640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.776+0000 7fa935440640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 0x7fa91406d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa92c036000 tx=0x7fa92c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:10.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.780+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa920057c50 con 0x7fa930072e70 2024-03-06T07:23:10.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.960+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9300c6340 con 0x7fa930072e70 2024-03-06T07:23:10.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.961+0000 7fa925ffb640 1 -- 172.21.15.17:0/953212240 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17280 (secure 0 0 0) 0x7fa92005bcf0 con 0x7fa930072e70 2024-03-06T07:23:10.963 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":84,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:09.006305+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.163:6834/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.17:6835/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.17:6833/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:10.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 msgr2=0x7fa91406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 0x7fa91406d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa92c036000 tx=0x7fa92c008030 comp rx=0 tx=0).stop 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 msgr2=0x7fa930158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa92000c440 tx=0x7fa92000c900 comp rx=0 tx=0).stop 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 shutdown_connections 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa930073250 0x7fa9300a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa930072e70 0x7fa930158cb0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 --2- 172.21.15.17:0/953212240 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa93006e490 0x7fa9300bda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 >> 172.21.15.17:0/953212240 conn(0x7fa93006b1a0 msgr2=0x7fa9300c3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 shutdown_connections 2024-03-06T07:23:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:10.963+0000 7fa9376a4640 1 -- 172.21.15.17:0/953212240 wait complete. 2024-03-06T07:23:10.975 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:23:10.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.076+0000 7efd5b89d640 1 Processor -- start 2024-03-06T07:23:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.076+0000 7efd5b89d640 1 -- start start 2024-03-06T07:23:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 0x7efd54158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efd54159460 0x7efd540c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd541587b0 con 0x7efd54152920 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd54156180 con 0x7efd54159460 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd5b89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c7760 con 0x7efd540c8a00 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51914/0 (socket says 172.21.15.17:51914) 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 -- 172.21.15.17:0/1713471118 learned_addr learned my addr 172.21.15.17:0/1713471118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59e3a640 1 --2- 172.21.15.17:0/1713471118 >> v2:172.21.15.163:3300/0 conn(0x7efd54159460 0x7efd540c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd58e38640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 0x7efd54158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 -- 172.21.15.17:0/1713471118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 msgr2=0x7efd54158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 0x7efd54158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 -- 172.21.15.17:0/1713471118 >> v2:172.21.15.163:3300/0 conn(0x7efd54159460 msgr2=0x7efd540c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 --2- 172.21.15.17:0/1713471118 >> v2:172.21.15.163:3300/0 conn(0x7efd54159460 0x7efd540c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd59639640 1 -- 172.21.15.17:0/1713471118 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540c7a40 con 0x7efd54152920 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.077+0000 7efd58e38640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 0x7efd54158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.078+0000 7efd59639640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd54152920 0x7efd54155c40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efd5406cab0 tx=0x7efd4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0910dae03f164b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.078+0000 7efd53fff640 1 -- 172.21.15.17:0/1713471118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02e070 con 0x7efd54152920 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.078+0000 7efd53fff640 1 -- 172.21.15.17:0/1713471118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c037070 con 0x7efd54152920 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.078+0000 7efd53fff640 1 -- 172.21.15.17:0/1713471118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c031a00 con 0x7efd54152920 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 -- 172.21.15.17:0/1713471118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd54152920 msgr2=0x7efd54155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd54152920 0x7efd54155c40 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efd5406cab0 tx=0x7efd4c02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 -- 172.21.15.17:0/1713471118 shutdown_connections 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 --2- 172.21.15.17:0/1713471118 >> v2:172.21.15.163:3300/0 conn(0x7efd54159460 0x7efd540c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd540c8a00 0x7efd54158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 --2- 172.21.15.17:0/1713471118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 -- 172.21.15.17:0/1713471118 >> 172.21.15.17:0/1713471118 conn(0x7efd540ac560 msgr2=0x7efd540ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 -- 172.21.15.17:0/1713471118 shutdown_connections 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.079+0000 7efd5b89d640 1 -- 172.21.15.17:0/1713471118 wait complete. 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.080+0000 7efd5b89d640 1 Processor -- start 2024-03-06T07:23:11.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.080+0000 7efd5b89d640 1 -- start start 2024-03-06T07:23:11.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c8a00 0x7efd540ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5406d030 con 0x7efd5414a990 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd54158a00 con 0x7efd540c8a00 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c7390 con 0x7efd54152920 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51922/0 (socket says 172.21.15.17:51922) 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 -- 172.21.15.17:0/3505167001 learned_addr learned my addr 172.21.15.17:0/3505167001 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59639640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c8a00 0x7efd540ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd58e38640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 -- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 msgr2=0x7efd54145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 -- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c8a00 msgr2=0x7efd540ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c8a00 0x7efd540ab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 -- 172.21.15.17:0/3505167001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5414d860 con 0x7efd5414a990 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd59e3a640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efd4400ef90 tx=0x7efd4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44018020 con 0x7efd5414a990 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd44014480 con 0x7efd5414a990 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541eef90 con 0x7efd5414a990 2024-03-06T07:23:11.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.081+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44014870 con 0x7efd5414a990 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.082+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5414d4f0 con 0x7efd5414a990 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.083+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5404f330 con 0x7efd5414a990 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.083+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd44005d10 con 0x7efd5414a990 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.083+0000 7efd51ffb640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 0x7efd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.084+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7efd4408b120 con 0x7efd5414a990 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.084+0000 7efd59639640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 0x7efd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.084+0000 7efd59639640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 0x7efd4006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd4c036000 tx=0x7efd4c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.088+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd44022030 con 0x7efd5414a990 2024-03-06T07:23:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.307+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd541eaee0 con 0x7efd5414a990 2024-03-06T07:23:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.308+0000 7efd51ffb640 1 -- 172.21.15.17:0/3505167001 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4986 (secure 0 0 0) 0x7efd44010040 con 0x7efd5414a990 2024-03-06T07:23:11.310 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:11.310 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3589667974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3589667974},{"type":"v1","addr":"172.21.15.163:6836","nonce":3589667974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4215297482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":4215297482},{"type":"v1","addr":"172.21.15.17:6835","nonce":4215297482}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:23:09.755245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6106":{"gid":6106,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":13,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":6}]} 2024-03-06T07:23:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 72 2024-03-06T07:23:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 msgr2=0x7efd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 0x7efd4006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efd4c036000 tx=0x7efd4c03d040 comp rx=0 tx=0).stop 2024-03-06T07:23:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 msgr2=0x7efd541eb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efd4400ef90 tx=0x7efd4400c490 comp rx=0 tx=0).stop 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 shutdown_connections 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd4006b4d0 0x7efd4006d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414a990 0x7efd541eb230 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 --2- 172.21.15.17:0/3505167001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c8a00 0x7efd540ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 >> 172.21.15.17:0/3505167001 conn(0x7efd540ac560 msgr2=0x7efd540ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 shutdown_connections 2024-03-06T07:23:11.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.310+0000 7efd5b89d640 1 -- 172.21.15.17:0/3505167001 wait complete. 2024-03-06T07:23:11.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.422+0000 7fd24be0e640 1 Processor -- start 2024-03-06T07:23:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.422+0000 7fd24be0e640 1 -- start start 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd24be0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2440c5860 0x7fd2441609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd24be0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2441591c0 con 0x7fd2440c5480 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd24be0e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2441594a0 con 0x7fd244155ba0 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd24be0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24406df40 con 0x7fd2440c5860 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd249baa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd2440c5480 0x7fd24406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59624/0 (socket says 172.21.15.17:59624) 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd249baa640 1 -- 172.21.15.17:0/1783848643 learned_addr learned my addr 172.21.15.17:0/1783848643 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.423+0000 7fd24a3ab640 1 --2- 172.21.15.17:0/1783848643 >> v2:172.21.15.17:3300/0 conn(0x7fd2440c5860 0x7fd2441609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221227343 0 0) 0x7fd2441591c0 con 0x7fd2440c5480 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003520 con 0x7fd2440c5480 2024-03-06T07:23:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867302 0 0) 0x7fd2441594a0 con 0x7fd244155ba0 2024-03-06T07:23:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd24a3ab640 1 -- 172.21.15.17:0/1783848643 >> v1:172.21.15.163:6789/0 conn(0x7fd244155ba0 legacy=0x7fd2441586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd24a3ab640 1 -- 172.21.15.17:0/1783848643 >> v1:172.21.15.17:6789/0 conn(0x7fd2440c5480 legacy=0x7fd24406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd24a3ab640 1 -- 172.21.15.17:0/1783848643 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2441553a0 con 0x7fd2440c5860 2024-03-06T07:23:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.424+0000 7fd24a3ab640 1 --2- 172.21.15.17:0/1783848643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd2441609e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd234009d90 tx=0x7fd23402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b04b07475a8659b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234039070 con 0x7fd2440c5860 2024-03-06T07:23:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23402cc60 con 0x7fd2440c5860 2024-03-06T07:23:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd248ba8640 1 -- 172.21.15.17:0/1783848643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234034660 con 0x7fd2440c5860 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 -- 172.21.15.17:0/1783848643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 msgr2=0x7fd2441609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 --2- 172.21.15.17:0/1783848643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd2441609e0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd234009d90 tx=0x7fd23402beb0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 -- 172.21.15.17:0/1783848643 shutdown_connections 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 --2- 172.21.15.17:0/1783848643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd2441609e0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 -- 172.21.15.17:0/1783848643 >> 172.21.15.17:0/1783848643 conn(0x7fd24406b1a0 msgr2=0x7fd24406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 -- 172.21.15.17:0/1783848643 shutdown_connections 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.425+0000 7fd24be0e640 1 -- 172.21.15.17:0/1783848643 wait complete. 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.426+0000 7fd24be0e640 1 Processor -- start 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.426+0000 7fd24be0e640 1 -- start start 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 0x7fd244150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd244155ba0 0x7fd24414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c5cf0 con 0x7fd2440c5860 2024-03-06T07:23:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24406fdb0 con 0x7fd2440c5480 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24be0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244156000 con 0x7fd244155ba0 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd2493a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd2493a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51938/0 (socket says 172.21.15.17:51938) 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd2493a9640 1 -- 172.21.15.17:0/2330308780 learned_addr learned my addr 172.21.15.17:0/2330308780 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd249baa640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 0x7fd244150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.427+0000 7fd24a3ab640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd244155ba0 0x7fd24414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 -- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd244155ba0 msgr2=0x7fd24414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd244155ba0 0x7fd24414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 -- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 msgr2=0x7fd244150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 0x7fd244150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 -- 172.21.15.17:0/2330308780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244153110 con 0x7fd2440c5860 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd249baa640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 0x7fd244150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2493a9640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c410 tx=0x7fd23800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238017070 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2380035f0 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2441533f0 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2380039e0 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.428+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441539b0 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.429+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24404f330 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.429+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd23801a070 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.430+0000 7fd2427fc640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 0x7fd23006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.430+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fd2380897a0 con 0x7fd2440c5860 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.430+0000 7fd249baa640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 0x7fd23006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.430+0000 7fd249baa640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 0x7fd23006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005880 tx=0x7fd23c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.434+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd238057dd0 con 0x7fd2440c5860 2024-03-06T07:23:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.652+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2440c2670 con 0x7fd2440c5860 2024-03-06T07:23:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.653+0000 7fd2427fc640 1 -- 172.21.15.17:0/2330308780 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4986 (secure 0 0 0) 0x7fd23805be70 con 0x7fd2440c5860 2024-03-06T07:23:11.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:11.654 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3589667974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3589667974},{"type":"v1","addr":"172.21.15.163:6836","nonce":3589667974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4215297482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":4215297482},{"type":"v1","addr":"172.21.15.17:6835","nonce":4215297482}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:22:49.803362+0000","modified":"2024-03-06T07:23:09.755245+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6106":{"gid":6106,"name":"d","rank":0,"incarnation":69,"state":"up:active","state_seq":13,"addr":"172.21.15.163:6837/1858186621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1858186621},{"type":"v1","addr":"172.21.15.163:6837","nonce":1858186621}]},"join_fscid":6,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6106,"qdb_cluster":[6106]},"id":6}]} 2024-03-06T07:23:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 72 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 msgr2=0x7fd23006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 0x7fd23006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005880 tx=0x7fd23c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 msgr2=0x7fd244154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd23800c410 tx=0x7fd23800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 shutdown_connections 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd23006b4d0 0x7fd23006d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd244155ba0 0x7fd24414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2440c5860 0x7fd244154710 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 --2- 172.21.15.17:0/2330308780 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2440c5480 0x7fd244150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 >> 172.21.15.17:0/2330308780 conn(0x7fd24406b1a0 msgr2=0x7fd24406ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 shutdown_connections 2024-03-06T07:23:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.655+0000 7fd24be0e640 1 -- 172.21.15.17:0/2330308780 wait complete. 2024-03-06T07:23:11.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 Processor -- start 2024-03-06T07:23:11.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 -- start start 2024-03-06T07:23:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3dd406da40 0x7f3dd4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3dd40c6780 0x7f3dd4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd4156050 con 0x7f3dd41592d0 2024-03-06T07:23:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd406de20 con 0x7f3dd406da40 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.767+0000 7f3dda73e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd4072630 con 0x7f3dd40c6780 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3dd406da40 0x7f3dd4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd8cdb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3dd41592d0 0x7f3dd40c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59632/0 (socket says 172.21.15.17:59632) 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd8cdb640 1 -- 172.21.15.17:0/2765312756 learned_addr learned my addr 172.21.15.17:0/2765312756 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd37fe640 1 --2- 172.21.15.17:0/2765312756 >> v2:172.21.15.163:3300/0 conn(0x7f3dd40c6780 0x7f3dd4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd3fff640 1 -- 172.21.15.17:0/2765312756 >> v2:172.21.15.163:3300/0 conn(0x7f3dd40c6780 msgr2=0x7f3dd4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd3fff640 1 --2- 172.21.15.17:0/2765312756 >> v2:172.21.15.163:3300/0 conn(0x7f3dd40c6780 0x7f3dd4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd3fff640 1 -- 172.21.15.17:0/2765312756 >> v1:172.21.15.17:6789/0 conn(0x7f3dd41592d0 legacy=0x7f3dd40c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd3fff640 1 -- 172.21.15.17:0/2765312756 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4072910 con 0x7f3dd406da40 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.768+0000 7f3dd2ffd640 1 -- 172.21.15.17:0/2765312756 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749278914 0 0) 0x7f3dcc001490 con 0x7f3dd41592d0 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dd3fff640 1 --2- 172.21.15.17:0/2765312756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd406da40 0x7f3dd4155b10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40c7120 tx=0x7f3dc802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580cae70f0efa094 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dd2ffd640 1 -- 172.21.15.17:0/2765312756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80315a0 con 0x7f3dd406da40 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dd2ffd640 1 -- 172.21.15.17:0/2765312756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8037070 con 0x7f3dd406da40 2024-03-06T07:23:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dd2ffd640 1 -- 172.21.15.17:0/2765312756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8031960 con 0x7f3dd406da40 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 -- 172.21.15.17:0/2765312756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd406da40 msgr2=0x7f3dd4155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 --2- 172.21.15.17:0/2765312756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd406da40 0x7f3dd4155b10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40c7120 tx=0x7f3dc802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 -- 172.21.15.17:0/2765312756 shutdown_connections 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 --2- 172.21.15.17:0/2765312756 >> v2:172.21.15.163:3300/0 conn(0x7f3dd40c6780 0x7f3dd4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 --2- 172.21.15.17:0/2765312756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd406da40 0x7f3dd4155b10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 -- 172.21.15.17:0/2765312756 >> 172.21.15.17:0/2765312756 conn(0x7f3dd40aa630 msgr2=0x7f3dd40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 -- 172.21.15.17:0/2765312756 shutdown_connections 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.769+0000 7f3dda73e640 1 -- 172.21.15.17:0/2765312756 wait complete. 2024-03-06T07:23:11.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.770+0000 7f3dda73e640 1 Processor -- start 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 -- start start 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd406da40 0x7f3dd4154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dd41592d0 0x7f3dd414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40c3990 con 0x7f3dd40c6780 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4158890 con 0x7f3dd406da40 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dda73e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40c3b10 con 0x7f3dd41592d0 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51952/0 (socket says 172.21.15.17:51952) 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dd37fe640 1 -- 172.21.15.17:0/3555710573 learned_addr learned my addr 172.21.15.17:0/3555710573 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dd8cdb640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dd41592d0 0x7f3dd414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.771+0000 7f3dd3fff640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd406da40 0x7f3dd4154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 -- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dd41592d0 msgr2=0x7f3dd414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dd41592d0 0x7f3dd414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 -- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd406da40 msgr2=0x7f3dd4154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd406da40 0x7f3dd4154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 -- 172.21.15.17:0/3555710573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4155260 con 0x7f3dd40c6780 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd37fe640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9b0 tx=0x7f3dc400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4017070 con 0x7f3dd40c6780 2024-03-06T07:23:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400c9a0 con 0x7f3dd40c6780 2024-03-06T07:23:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400cd90 con 0x7f3dd40c6780 2024-03-06T07:23:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd41eba40 con 0x7f3dd40c6780 2024-03-06T07:23:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.772+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd41deb90 con 0x7f3dd40c6780 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.773+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd404f330 con 0x7f3dd40c6780 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.773+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dc401a070 con 0x7f3dd40c6780 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.774+0000 7f3dd0ff9640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.774+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f3dc408a3e0 con 0x7f3dd40c6780 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.774+0000 7f3dd3fff640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.774+0000 7f3dd3fff640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8036000 tx=0x7f3dc803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.778+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3dc4058970 con 0x7f3dd40c6780 2024-03-06T07:23:11.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.960+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd4073240 con 0x7f3dd40c6780 2024-03-06T07:23:11.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.961+0000 7f3dd0ff9640 1 -- 172.21.15.17:0/3555710573 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+17280 (secure 0 0 0) 0x7f3dc405ca10 con 0x7f3dd40c6780 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:11.963 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":84,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:09.006305+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:22:47.593104+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:22:48.766993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.163:6834/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.17:6835/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.17:6833/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 msgr2=0x7f3dc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8036000 tx=0x7f3dc803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 msgr2=0x7f3dd4154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400e9b0 tx=0x7f3dc400ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 shutdown_connections 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dd41592d0 0x7f3dd414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd40c6780 0x7f3dd4154d20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 --2- 172.21.15.17:0/3555710573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd406da40 0x7f3dd4154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 >> 172.21.15.17:0/3555710573 conn(0x7f3dd40aa630 msgr2=0x7f3dd40c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 shutdown_connections 2024-03-06T07:23:11.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:11.963+0000 7f3dda73e640 1 -- 172.21.15.17:0/3555710573 wait complete. 2024-03-06T07:23:11.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:23:12.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.077+0000 7f794d002640 1 Processor -- start 2024-03-06T07:23:12.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.077+0000 7f794d002640 1 -- start start 2024-03-06T07:23:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794d002640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79481526d0 0x7f79480c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794d002640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 0x7f794806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794d002640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7948155010 con 0x7f79480c8c80 2024-03-06T07:23:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794d002640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7948155ee0 con 0x7f79481526d0 2024-03-06T07:23:12.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794d002640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7948152ab0 con 0x7f794806b890 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79481526d0 0x7f79480c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79481526d0 0x7f79480c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51954/0 (socket says 172.21.15.17:51954) 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 -- 172.21.15.17:0/4011868034 learned_addr learned my addr 172.21.15.17:0/4011868034 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794759e640 1 --2- 172.21.15.17:0/4011868034 >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 0x7f794806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 -- 172.21.15.17:0/4011868034 >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 msgr2=0x7f794806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 --2- 172.21.15.17:0/4011868034 >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 0x7f794806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 -- 172.21.15.17:0/4011868034 >> v1:172.21.15.163:6789/0 conn(0x7f79480c8c80 legacy=0x7f79480c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f7946d9d640 1 -- 172.21.15.17:0/4011868034 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79481552d0 con 0x7f79481526d0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.078+0000 7f794759e640 1 --2- 172.21.15.17:0/4011868034 >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 0x7f794806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f7946d9d640 1 --2- 172.21.15.17:0/4011868034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79481526d0 0x7f79480c8740 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f794808c8a0 tx=0x7f793802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ca7beb27f36ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f794659c640 1 -- 172.21.15.17:0/4011868034 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793802e070 con 0x7f79481526d0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f794659c640 1 -- 172.21.15.17:0/4011868034 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938037070 con 0x7f79481526d0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f794659c640 1 -- 172.21.15.17:0/4011868034 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938031a00 con 0x7f79481526d0 2024-03-06T07:23:12.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f794d002640 1 -- 172.21.15.17:0/4011868034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79481526d0 msgr2=0x7f79480c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.079+0000 7f794d002640 1 --2- 172.21.15.17:0/4011868034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79481526d0 0x7f79480c8740 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f794808c8a0 tx=0x7f793802b880 comp rx=0 tx=0).stop 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 -- 172.21.15.17:0/4011868034 shutdown_connections 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 --2- 172.21.15.17:0/4011868034 >> v2:172.21.15.163:3300/0 conn(0x7f794806b890 0x7f794806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 --2- 172.21.15.17:0/4011868034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79481526d0 0x7f79480c8740 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 -- 172.21.15.17:0/4011868034 >> 172.21.15.17:0/4011868034 conn(0x7f79480ac430 msgr2=0x7f79480ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 -- 172.21.15.17:0/4011868034 shutdown_connections 2024-03-06T07:23:12.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.080+0000 7f794d002640 1 -- 172.21.15.17:0/4011868034 wait complete. 2024-03-06T07:23:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.081+0000 7f794d002640 1 Processor -- start 2024-03-06T07:23:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.081+0000 7f794d002640 1 -- start start 2024-03-06T07:23:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.081+0000 7f794d002640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f794806b890 0x7f79480ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79481526d0 0x7f794814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7948155b70 con 0x7f79480c8c80 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7948155890 con 0x7f794806b890 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79480c58d0 con 0x7f79481526d0 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51956/0 (socket says 172.21.15.17:51956) 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 -- 172.21.15.17:0/3032468599 learned_addr learned my addr 172.21.15.17:0/3032468599 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f7946d9d640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f794806b890 0x7f79480ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794759e640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79481526d0 0x7f794814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 -- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79481526d0 msgr2=0x7f794814b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79481526d0 0x7f794814b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 -- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f794806b890 msgr2=0x7f79480ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f794806b890 0x7f79480ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 -- 172.21.15.17:0/3032468599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480aacc0 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793ffff640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f793000c3f0 tx=0x7f793000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79300052d0 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7930005490 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930007260 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79480cb220 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.082+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79481e0960 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.084+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79480693c0 con 0x7f79480c8c80 2024-03-06T07:23:12.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.084+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7930007400 con 0x7f79480c8c80 2024-03-06T07:23:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.084+0000 7f793f7fe640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 0x7f792806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.084+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f7930089830 con 0x7f79480c8c80 2024-03-06T07:23:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.085+0000 7f7946d9d640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 0x7f792806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.085+0000 7f7946d9d640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 0x7f792806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7938036000 tx=0x7f793803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.089+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7930057e60 con 0x7f79480c8c80 2024-03-06T07:23:12.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.309+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f794806cc50 con 0x7f79480c8c80 2024-03-06T07:23:12.311 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:23:12.310+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.311+0000 7f793f7fe640 1 -- 172.21.15.17:0/3032468599 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) v1 ==== 163+0+0 (secure 0 0 0) 0x7f793005ce20 con 0x7f79480c8c80 2024-03-06T07:23:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 msgr2=0x7f792806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 0x7f792806d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7938036000 tx=0x7f793803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 msgr2=0x7f79480ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f793000c3f0 tx=0x7f793000c8b0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 shutdown_connections 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f792806b4d0 0x7f792806d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79481526d0 0x7f794814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79480c8c80 0x7f79480ad500 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 --2- 172.21.15.17:0/3032468599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f794806b890 0x7f79480ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 >> 172.21.15.17:0/3032468599 conn(0x7f79480ac430 msgr2=0x7f79480c7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 shutdown_connections 2024-03-06T07:23:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.314+0000 7f794d002640 1 -- 172.21.15.17:0/3032468599 wait complete. 2024-03-06T07:23:12.317 INFO:tasks.ceph.mds.d.smithi163.stderr: -67> 2024-03-06T07:23:10.640+0000 7f4fd6ff2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:12.326 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:23:12.339 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:23:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.428+0000 7fc09df3e640 1 Processor -- start 2024-03-06T07:23:12.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.428+0000 7fc09df3e640 1 -- start start 2024-03-06T07:23:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc09df3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc09806afc0 0x7fc09806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc09df3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 0x7fc098159330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc09df3e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980c5840 con 0x7fc0980c2120 2024-03-06T07:23:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc09df3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc098158790 con 0x7fc09806afc0 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc09df3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980c5b20 con 0x7fc0980c2580 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc0977fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc09806afc0 0x7fc09806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc097fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 0x7fc098159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc096ffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc0980c2120 0x7fc098158c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:38164/0 (socket says 172.21.15.17:38164) 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.429+0000 7fc096ffd640 1 -- 172.21.15.17:0/1689057630 learned_addr learned my addr 172.21.15.17:0/1689057630 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0977fe640 1 -- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 msgr2=0x7fc098159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0977fe640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 0x7fc098159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0977fe640 1 -- 172.21.15.17:0/1689057630 >> v1:172.21.15.163:6789/0 conn(0x7fc0980c2120 legacy=0x7fc098158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0977fe640 1 -- 172.21.15.17:0/1689057630 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980c5e00 con 0x7fc09806afc0 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0967fc640 1 -- 172.21.15.17:0/1689057630 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423192769 0 0) 0x7fc0980c5840 con 0x7fc0980c2120 2024-03-06T07:23:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc097fff640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 0x7fc098159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0977fe640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc09806afc0 0x7fc09806b3a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66c5204094abac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0967fc640 1 -- 172.21.15.17:0/1689057630 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880315a0 con 0x7fc09806afc0 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0967fc640 1 -- 172.21.15.17:0/1689057630 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc088037070 con 0x7fc09806afc0 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.430+0000 7fc0967fc640 1 -- 172.21.15.17:0/1689057630 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088031960 con 0x7fc09806afc0 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 -- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc09806afc0 msgr2=0x7fc09806b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc09806afc0 0x7fc09806b3a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 -- 172.21.15.17:0/1689057630 shutdown_connections 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0980c2580 0x7fc098159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 --2- 172.21.15.17:0/1689057630 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc09806afc0 0x7fc09806b3a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 -- 172.21.15.17:0/1689057630 >> 172.21.15.17:0/1689057630 conn(0x7fc0980c4220 msgr2=0x7fc0980c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:12.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 -- 172.21.15.17:0/1689057630 shutdown_connections 2024-03-06T07:23:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.431+0000 7fc09df3e640 1 -- 172.21.15.17:0/1689057630 wait complete. 2024-03-06T07:23:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.432+0000 7fc09df3e640 1 Processor -- start 2024-03-06T07:23:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.432+0000 7fc09df3e640 1 -- start start 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc09806afc0 0x7fc098073010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980c2580 0x7fc0981ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980c2b30 con 0x7fc0980c2120 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098155220 con 0x7fc0980c2580 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc09df3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980c2cb0 con 0x7fc09806afc0 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51964/0 (socket says 172.21.15.17:51964) 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 -- 172.21.15.17:0/833329724 learned_addr learned my addr 172.21.15.17:0/833329724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc097fff640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980c2580 0x7fc0981ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc0977fe640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc09806afc0 0x7fc098073010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 -- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc09806afc0 msgr2=0x7fc098073010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc09806afc0 0x7fc098073010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 -- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980c2580 msgr2=0x7fc0981ee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980c2580 0x7fc0981ee4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.433+0000 7fc096ffd640 1 -- 172.21.15.17:0/833329724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09806fde0 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc096ffd640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c4d0 tx=0x7fc08c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c018070 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c0074e0 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c004a20 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09806df50 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.434+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09806dc70 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.435+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09804f330 con 0x7fc0980c2120 2024-03-06T07:23:12.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.435+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08c01b070 con 0x7fc0980c2120 2024-03-06T07:23:12.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.436+0000 7fc07ffff640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:12.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.436+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fc08c0897e0 con 0x7fc0980c2120 2024-03-06T07:23:12.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.436+0000 7fc0977fe640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:12.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.436+0000 7fc0977fe640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc088036000 tx=0x7fc08803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:12.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.440+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc09806fde0 con 0x7fc0980c2120 2024-03-06T07:23:12.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:12.664+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0980c6730 con 0x7fc0980c2120 2024-03-06T07:23:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.312+0000 7fc07ffff640 1 -- 172.21.15.17:0/833329724 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v74) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc08c057c80 con 0x7fc0980c2120 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 msgr2=0x7fc08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc088036000 tx=0x7fc08803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 msgr2=0x7fc09806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c4d0 tx=0x7fc08c00c990 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 shutdown_connections 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980c2580 0x7fc0981ee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0980c2120 0x7fc09806f8a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 --2- 172.21.15.17:0/833329724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc09806afc0 0x7fc098073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 >> 172.21.15.17:0/833329724 conn(0x7fc0980c4220 msgr2=0x7fc098151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 shutdown_connections 2024-03-06T07:23:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.315+0000 7fc09df3e640 1 -- 172.21.15.17:0/833329724 wait complete. 2024-03-06T07:23:13.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:13.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.441+0000 7f596dfbf640 1 Processor -- start 2024-03-06T07:23:13.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.441+0000 7f596dfbf640 1 -- start start 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.441+0000 7f596dfbf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59681587c0 0x7f5968160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.441+0000 7f596dfbf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5968159740 con 0x7f5968152920 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f596dfbf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59681551d0 con 0x7f59680c2900 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f596dfbf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59680c8ab0 con 0x7f59681587c0 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59677fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5968152920 0x7f5968155970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59634/0 (socket says 172.21.15.17:59634) 2024-03-06T07:23:13.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59677fe640 1 -- 172.21.15.17:0/2024509292 learned_addr learned my addr 172.21.15.17:0/2024509292 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f5967fff640 1 --2- 172.21.15.17:0/2024509292 >> v2:172.21.15.163:3300/0 conn(0x7f59681587c0 0x7f5968160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311317855 0 0) 0x7f5968159740 con 0x7f5968152920 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950003520 con 0x7f5968152920 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981067039 0 0) 0x7f595c001110 con 0x7f59680c2900 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5968159740 con 0x7f59680c2900 2024-03-06T07:23:13.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f5967fff640 1 -- 172.21.15.17:0/2024509292 >> v1:172.21.15.163:6789/0 conn(0x7f59680c2900 legacy=0x7f59680c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 894499039 0 0) 0x7f5950003520 con 0x7f5968152920 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f5967fff640 1 -- 172.21.15.17:0/2024509292 >> v1:172.21.15.17:6789/0 conn(0x7f5968152920 legacy=0x7f5968155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.442+0000 7f5967fff640 1 -- 172.21.15.17:0/2024509292 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680c8d90 con 0x7f59681587c0 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.443+0000 7f5967fff640 1 --2- 172.21.15.17:0/2024509292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59681587c0 0x7f5968160b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5960009d90 tx=0x7f596002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769347dd245a9ec1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.443+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960039070 con 0x7f59681587c0 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.443+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596002c8b0 con 0x7f59681587c0 2024-03-06T07:23:13.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.443+0000 7f59667fc640 1 -- 172.21.15.17:0/2024509292 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958001690 con 0x7f59681587c0 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 -- 172.21.15.17:0/2024509292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59681587c0 msgr2=0x7f5968160b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 --2- 172.21.15.17:0/2024509292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59681587c0 0x7f5968160b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5960009d90 tx=0x7f596002bd80 comp rx=0 tx=0).stop 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 -- 172.21.15.17:0/2024509292 shutdown_connections 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 --2- 172.21.15.17:0/2024509292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59681587c0 0x7f5968160b60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 -- 172.21.15.17:0/2024509292 >> 172.21.15.17:0/2024509292 conn(0x7f59680aa5c0 msgr2=0x7f59680aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:13.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 -- 172.21.15.17:0/2024509292 shutdown_connections 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.444+0000 7f596dfbf640 1 -- 172.21.15.17:0/2024509292 wait complete. 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.445+0000 7f596dfbf640 1 Processor -- start 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.445+0000 7f596dfbf640 1 -- start start 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59680c2900 0x7f59681e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59681587c0 0x7f59681eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968156080 con 0x7f5968152920 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968155450 con 0x7f59680c2900 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f596dfbf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59680c82a0 con 0x7f59681587c0 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f5966ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51976/0 (socket says 172.21.15.17:51976) 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f5966ffd640 1 -- 172.21.15.17:0/1195804927 learned_addr learned my addr 172.21.15.17:0/1195804927 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:13.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f5967fff640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59681587c0 0x7f59681eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.446+0000 7f59677fe640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59680c2900 0x7f59681e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 -- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59681587c0 msgr2=0x7f59681eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59681587c0 0x7f59681eb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 -- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59680c2900 msgr2=0x7f59681e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:13.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59680c2900 0x7f59681e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 -- 172.21.15.17:0/1195804927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59681e05b0 con 0x7f5968152920 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f5966ffd640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c730 tx=0x7f595c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0052d0 con 0x7f5968152920 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c005470 con 0x7f5968152920 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c00a860 con 0x7f5968152920 2024-03-06T07:23:13.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596806ca10 con 0x7f5968152920 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.447+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59681e0850 con 0x7f5968152920 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.448+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596804f330 con 0x7f5968152920 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.448+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f595c00aa00 con 0x7f5968152920 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.449+0000 7f594ffff640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 0x7f595406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.449+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f595c08a090 con 0x7f5968152920 2024-03-06T07:23:13.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.449+0000 7f59677fe640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 0x7f595406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:13.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.450+0000 7f59677fe640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 0x7f595406d9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5958001200 tx=0x7f5958001550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:13.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.453+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f59681551d0 con 0x7f5968152920 2024-03-06T07:23:13.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:13.632+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59680704d0 con 0x7f5968152920 2024-03-06T07:23:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.317+0000 7f594ffff640 1 -- 172.21.15.17:0/1195804927 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v86) v1 ==== 184+0+0 (secure 0 0 0) 0x7f595c058cb0 con 0x7f5968152920 2024-03-06T07:23:14.319 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:23:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 msgr2=0x7f595406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 0x7f595406d9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5958001200 tx=0x7f5958001550 comp rx=0 tx=0).stop 2024-03-06T07:23:14.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 msgr2=0x7f59681eae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c730 tx=0x7f595c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 shutdown_connections 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.320+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f595406b500 0x7f595406d9c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59681587c0 0x7f59681eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5968152920 0x7f59681eae20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 --2- 172.21.15.17:0/1195804927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59680c2900 0x7f59681e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 >> 172.21.15.17:0/1195804927 conn(0x7f59680aa5c0 msgr2=0x7f59680aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 shutdown_connections 2024-03-06T07:23:14.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.321+0000 7f596dfbf640 1 -- 172.21.15.17:0/1195804927 wait complete. 2024-03-06T07:23:14.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.446+0000 7f3f99a84640 1 Processor -- start 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.446+0000 7f3f99a84640 1 -- start start 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f99a84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f941566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f99a84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940c8f60 con 0x7f3f9414f180 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f99a84640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940c9240 con 0x7f3f940c8980 2024-03-06T07:23:14.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f99a84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94156c00 con 0x7f3f941551d0 2024-03-06T07:23:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f937fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3f9414f180 0x7f3f940ca000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59644/0 (socket says 172.21.15.17:59644) 2024-03-06T07:23:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f937fe640 1 -- 172.21.15.17:0/2410696620 learned_addr learned my addr 172.21.15.17:0/2410696620 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f941566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755441447 0 0) 0x7f3f940c8f60 con 0x7f3f9414f180 2024-03-06T07:23:14.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.447+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f80003520 con 0x7f3f9414f180 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935676633 0 0) 0x7f3f940c9240 con 0x7f3f940c8980 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f940c8f60 con 0x7f3f940c8980 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187211200 0 0) 0x7f3f80003520 con 0x7f3f9414f180 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 msgr2=0x7f3f941566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f941566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 >> v1:172.21.15.163:6789/0 conn(0x7f3f940c8980 legacy=0x7f3f9406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940c9520 con 0x7f3f9414f180 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3f84001670 con 0x7f3f9414f180 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 >> v1:172.21.15.17:6789/0 conn(0x7f3f9414f180 legacy=0x7f3f940ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f80003b70 0x7f3f80003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f80004d40 0x7f3f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94156c00 con 0x7f3f80004d40 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8000ba70 con 0x7f3f80003b70 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8000bc70 con 0x7f3f941551d0 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f80004d40 0x7f3f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f937fe640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f80003b70 0x7f3f80003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f8bfff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 -- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 msgr2=0x7f3f80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:14.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f941551d0 0x7f3f80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 -- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f80003b70 msgr2=0x7f3f80003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f80003b70 0x7f3f80003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.448+0000 7f3f93fff640 1 -- 172.21.15.17:0/2410696620 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80003520 con 0x7f3f80004d40 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.449+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f80004d40 0x7f3f800051c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fc0 tx=0x7f3f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecfca9e7cc0b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.449+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c039070 con 0x7f3f80004d40 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.449+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c02cc20 con 0x7f3f80004d40 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.449+0000 7f3f92ffd640 1 -- 172.21.15.17:0/2410696620 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c009c10 con 0x7f3f80004d40 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 -- 172.21.15.17:0/2410696620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f80004d40 msgr2=0x7f3f800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2410696620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f80004d40 0x7f3f800051c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009fc0 tx=0x7f3f80005830 comp rx=0 tx=0).stop 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f93fff640 1 -- 172.21.15.17:0/2410696620 reap_dead start 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 -- 172.21.15.17:0/2410696620 shutdown_connections 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 -- 172.21.15.17:0/2410696620 >> 172.21.15.17:0/2410696620 conn(0x7f3f94072d20 msgr2=0x7f3f940ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 -- 172.21.15.17:0/2410696620 shutdown_connections 2024-03-06T07:23:14.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.450+0000 7f3f99a84640 1 -- 172.21.15.17:0/2410696620 wait complete. 2024-03-06T07:23:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.451+0000 7f3f99a84640 1 Processor -- start 2024-03-06T07:23:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.451+0000 7f3f99a84640 1 -- start start 2024-03-06T07:23:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f9414b690 0x7f3f9414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f9414c270 0x7f3f94158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9414f560 con 0x7f3f94159090 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940c9ae0 con 0x7f3f9414c270 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f99a84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9414f6e0 con 0x7f3f9414b690 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51998/0 (socket says 172.21.15.17:51998) 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f93fff640 1 -- 172.21.15.17:0/2008966119 learned_addr learned my addr 172.21.15.17:0/2008966119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f937fe640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f9414b690 0x7f3f9414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.452+0000 7f3f8bfff640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f9414c270 0x7f3f94158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 -- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f9414b690 msgr2=0x7f3f9414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f9414b690 0x7f3f9414bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 -- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f9414c270 msgr2=0x7f3f94158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f9414c270 0x7f3f94158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 -- 172.21.15.17:0/2008966119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f941e20d0 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f93fff640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c002230 tx=0x7f3f8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c039030 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c043070 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f941e23b0 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c009c10 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.453+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f940abc20 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.454+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f9404f330 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.454+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f8c005d10 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.455+0000 7f3f90ff9640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 0x7f3f7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.455+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6009+0+0 (secure 0 0 0) 0x7f3f8c0ac3a0 con 0x7f3f94159090 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.455+0000 7f3f937fe640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 0x7f3f7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:14.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.456+0000 7f3f937fe640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 0x7f3f7406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f84001050 tx=0x7f3f84000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:14.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.460+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f8c07ab30 con 0x7f3f94159090 2024-03-06T07:23:14.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:14.642+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f940afe70 con 0x7f3f94159090 2024-03-06T07:23:15.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.319+0000 7f3f90ff9640 1 -- 172.21.15.17:0/2008966119 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v87) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3f8c07ebd0 con 0x7f3f94159090 2024-03-06T07:23:15.321 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' removed 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 msgr2=0x7f3f7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 0x7f3f7406d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f84001050 tx=0x7f3f84000f80 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 msgr2=0x7f3f941ee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c002230 tx=0x7f3f8c009750 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 shutdown_connections 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f7406b500 0x7f3f7406d9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f94159090 0x7f3f941ee330 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f9414c270 0x7f3f94158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 --2- 172.21.15.17:0/2008966119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f9414b690 0x7f3f9414bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.322+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 >> 172.21.15.17:0/2008966119 conn(0x7f3f94072d20 msgr2=0x7f3f940ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.323+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 shutdown_connections 2024-03-06T07:23:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.323+0000 7f3f99a84640 1 -- 172.21.15.17:0/2008966119 wait complete. 2024-03-06T07:23:15.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:23:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.448+0000 7faad0f3e640 1 Processor -- start 2024-03-06T07:23:15.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.448+0000 7faad0f3e640 1 -- start start 2024-03-06T07:23:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faad0f3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faacc0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faad0f3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faacc152b10 con 0x7faacc0c3750 2024-03-06T07:23:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faad0f3e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faacc158ad0 con 0x7faacc0c6710 2024-03-06T07:23:15.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faad0f3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc158db0 con 0x7faacc152730 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faacad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faacc0c3750 0x7faacc15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59660/0 (socket says 172.21.15.17:59660) 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faacad9d640 1 -- 172.21.15.17:0/3646196379 learned_addr learned my addr 172.21.15.17:0/3646196379 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1071644499 0 0) 0x7faacc152b10 con 0x7faacc0c3750 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faaca59c640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faacc0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.449+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faab8003520 con 0x7faacc0c3750 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4041355241 0 0) 0x7faab8003520 con 0x7faacc0c3750 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 msgr2=0x7faacc0c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faacc0c3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 >> v1:172.21.15.163:6789/0 conn(0x7faacc0c6710 legacy=0x7faacc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc159090 con 0x7faacc0c3750 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7faac4001670 con 0x7faacc0c3750 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 >> v1:172.21.15.17:6789/0 conn(0x7faacc0c3750 legacy=0x7faacc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab80039e0 0x7faab8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faab8004800 0x7faab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faab8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc158db0 con 0x7faab8004800 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab800baf0 con 0x7faab80039e0 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab800bcf0 con 0x7faacc152730 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faab8004800 0x7faab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faaca59c640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab80039e0 0x7faab8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faacad9d640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faab8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 -- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 msgr2=0x7faab8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc152730 0x7faab8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 -- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab80039e0 msgr2=0x7faab8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab80039e0 0x7faab8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faac9d9b640 1 -- 172.21.15.17:0/3646196379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab8003520 con 0x7faab8004800 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.450+0000 7faaca59c640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab80039e0 0x7faab8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.451+0000 7faac9d9b640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faab8004800 0x7faab8004c80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7faab4001720 tx=0x7faab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f406b10bf68c1dea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.451+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4001370 con 0x7faab8004800 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.451+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab40052d0 con 0x7faab8004800 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.451+0000 7faac959a640 1 -- 172.21.15.17:0/3646196379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4003830 con 0x7faab8004800 2024-03-06T07:23:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 -- 172.21.15.17:0/3646196379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faab8004800 msgr2=0x7faab8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 --2- 172.21.15.17:0/3646196379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faab8004800 0x7faab8004c80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7faab4001720 tx=0x7faab8005830 comp rx=0 tx=0).stop 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faacad9d640 1 -- 172.21.15.17:0/3646196379 reap_dead start 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 -- 172.21.15.17:0/3646196379 shutdown_connections 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 -- 172.21.15.17:0/3646196379 >> 172.21.15.17:0/3646196379 conn(0x7faacc0aa5c0 msgr2=0x7faacc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 -- 172.21.15.17:0/3646196379 shutdown_connections 2024-03-06T07:23:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.452+0000 7faad0f3e640 1 -- 172.21.15.17:0/3646196379 wait complete. 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.453+0000 7faad0f3e640 1 Processor -- start 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.453+0000 7faad0f3e640 1 -- start start 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab8004800 0x7faacc1eb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc0a9480 0x7faacc0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc158870 con 0x7faacc06c500 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc1586f0 con 0x7faab8004800 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faad0f3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc0a7cd0 con 0x7faacc0a9480 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52020/0 (socket says 172.21.15.17:52020) 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 -- 172.21.15.17:0/1494253296 learned_addr learned my addr 172.21.15.17:0/1494253296 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faaca59c640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab8004800 0x7faacc1eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faacad9d640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc0a9480 0x7faacc0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 -- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc0a9480 msgr2=0x7faacc0a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc0a9480 0x7faacc0a9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 -- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab8004800 msgr2=0x7faacc1eb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab8004800 0x7faacc1eb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 -- 172.21.15.17:0/1494253296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc06feb0 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.454+0000 7faac9d9b640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7faab4005040 tx=0x7faab4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.455+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4041070 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.455+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4005480 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.455+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc154800 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.455+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4007a30 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.455+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc154dc0 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.456+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc04f330 con 0x7faacc06c500 2024-03-06T07:23:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.456+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faab4044070 con 0x7faacc06c500 2024-03-06T07:23:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.456+0000 7faac2ffd640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 0x7faaac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.456+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7faab40abca0 con 0x7faacc06c500 2024-03-06T07:23:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.457+0000 7faaca59c640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 0x7faaac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.457+0000 7faaca59c640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 0x7faaac06d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faacc0710c0 tx=0x7faabc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.461+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faab407a6f0 con 0x7faacc06c500 2024-03-06T07:23:15.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.642+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7faacc152f70 con 0x7faacc06c500 2024-03-06T07:23:15.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.646+0000 7faac2ffd640 1 -- 172.21.15.17:0/1494253296 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v27) v1 ==== 96+0+0 (secure 0 0 0) 0x7faab407e790 con 0x7faacc06c500 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.647+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 msgr2=0x7faaac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.647+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 0x7faaac06d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7faacc0710c0 tx=0x7faabc00b040 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 msgr2=0x7faacc14be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7faab4005040 tx=0x7faab4001120 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 shutdown_connections 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faaac06b500 0x7faaac06d9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faacc0a9480 0x7faacc0a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faacc06c500 0x7faacc14be60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 --2- 172.21.15.17:0/1494253296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faab8004800 0x7faacc1eb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 >> 172.21.15.17:0/1494253296 conn(0x7faacc0aa5c0 msgr2=0x7faacc14e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 shutdown_connections 2024-03-06T07:23:15.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.648+0000 7faad0f3e640 1 -- 172.21.15.17:0/1494253296 wait complete. 2024-03-06T07:23:15.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:23:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 Processor -- start 2024-03-06T07:23:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 -- start start 2024-03-06T07:23:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7040b01e0 0x7fc704155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc7040c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7040c7e00 con 0x7fc7040c5910 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040c7840 con 0x7fc7040b01e0 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.762+0000 7fc70a78c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c7b20 con 0x7fc7041565b0 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7040b01e0 0x7fc704155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc7040b01e0 0x7fc704155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52034/0 (socket says 172.21.15.17:52034) 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 -- 172.21.15.17:0/2246440831 learned_addr learned my addr 172.21.15.17:0/2246440831 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc7037fe640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc7040c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 -- 172.21.15.17:0/2246440831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 msgr2=0x7fc7040c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc7040c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 -- 172.21.15.17:0/2246440831 >> v1:172.21.15.163:6789/0 conn(0x7fc7040c5910 legacy=0x7fc70406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.763+0000 7fc703fff640 1 -- 172.21.15.17:0/2246440831 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70406b970 con 0x7fc7040b01e0 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc703fff640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040b01e0 0x7fc704155b10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009870 tx=0x7fc6f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42adc54232605fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc702ffd640 1 -- 172.21.15.17:0/2246440831 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4035020 con 0x7fc7040b01e0 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc702ffd640 1 -- 172.21.15.17:0/2246440831 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f4031640 con 0x7fc7040b01e0 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc702ffd640 1 -- 172.21.15.17:0/2246440831 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f402d040 con 0x7fc7040b01e0 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 -- 172.21.15.17:0/2246440831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040b01e0 msgr2=0x7fc704155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040b01e0 0x7fc704155b10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009870 tx=0x7fc6f402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 -- 172.21.15.17:0/2246440831 shutdown_connections 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc7040c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 --2- 172.21.15.17:0/2246440831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040b01e0 0x7fc704155b10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 -- 172.21.15.17:0/2246440831 >> 172.21.15.17:0/2246440831 conn(0x7fc7040ac600 msgr2=0x7fc7040aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 -- 172.21.15.17:0/2246440831 shutdown_connections 2024-03-06T07:23:15.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.764+0000 7fc70a78c640 1 -- 172.21.15.17:0/2246440831 wait complete. 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.765+0000 7fc70a78c640 1 Processor -- start 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 -- start start 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 0x7fc704156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc70414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040be3d0 con 0x7fc7040c5910 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040be250 con 0x7fc7040b01e0 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc70a78c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c7460 con 0x7fc7041565b0 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52042/0 (socket says 172.21.15.17:52042) 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc7037fe640 1 -- 172.21.15.17:0/1359394139 learned_addr learned my addr 172.21.15.17:0/1359394139 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc708d29640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc70414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.766+0000 7fc703fff640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 0x7fc704156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 -- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 msgr2=0x7fc70414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc70414bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 -- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 msgr2=0x7fc704156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 0x7fc704156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 -- 172.21.15.17:0/1359394139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7041ef060 con 0x7fc7040c5910 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc708d29640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc70414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc703fff640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 0x7fc704156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc7037fe640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d5c0 tx=0x7fc6f800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80125f0 con 0x7fc7040c5910 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8019070 con 0x7fc7040c5910 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7041ee2a0 con 0x7fc7040c5910 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80129d0 con 0x7fc7040c5910 2024-03-06T07:23:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.767+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7041ee860 con 0x7fc7040c5910 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.768+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7040abd70 con 0x7fc7040c5910 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.768+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6f8012b70 con 0x7fc7040c5910 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.769+0000 7fc700ff9640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 0x7fc6f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.769+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fc6f808a4c0 con 0x7fc7040c5910 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.769+0000 7fc703fff640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 0x7fc6f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.769+0000 7fc703fff640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 0x7fc6f006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4036000 tx=0x7fc6f4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.773+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7040abd70 con 0x7fc7040c5910 2024-03-06T07:23:15.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.962+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7fc70404f330 con 0x7fc7040c5910 2024-03-06T07:23:15.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.965+0000 7fc700ff9640 1 -- 172.21.15.17:0/1359394139 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v28) v1 ==== 96+0+0 (secure 0 0 0) 0x7fc6f8059440 con 0x7fc7040c5910 2024-03-06T07:23:15.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 msgr2=0x7fc6f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 0x7fc6f006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4036000 tx=0x7fc6f4008030 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 msgr2=0x7fc7040c0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800d5c0 tx=0x7fc6f800da80 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 shutdown_connections 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc6f006b4d0 0x7fc6f006d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7041565b0 0x7fc70414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7040c5910 0x7fc7040c0280 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 --2- 172.21.15.17:0/1359394139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7040b01e0 0x7fc704156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 >> 172.21.15.17:0/1359394139 conn(0x7fc7040ac600 msgr2=0x7fc7040ab660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 shutdown_connections 2024-03-06T07:23:15.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:15.967+0000 7fc70a78c640 1 -- 172.21.15.17:0/1359394139 wait complete. 2024-03-06T07:23:15.979 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:23:16.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.081+0000 7f1d6e888640 1 Processor -- start 2024-03-06T07:23:16.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d6e888640 1 -- start start 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d6e888640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d6e888640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680c3ef0 con 0x7f1d680c5990 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d6e888640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d681556d0 con 0x7f1d680ab170 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d6e888640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681559b0 con 0x7f1d6806c900 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d67fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1d680c5990 0x7f1d6806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59672/0 (socket says 172.21.15.17:59672) 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.082+0000 7f1d67fff640 1 -- 172.21.15.17:0/3020135654 learned_addr learned my addr 172.21.15.17:0/3020135654 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4099606233 0 0) 0x7f1d680c3ef0 con 0x7f1d680c5990 2024-03-06T07:23:16.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d677fe640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d50003520 con 0x7f1d680c5990 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253672053 0 0) 0x7f1d50003520 con 0x7f1d680c5990 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 msgr2=0x7f1d680c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 >> v1:172.21.15.163:6789/0 conn(0x7f1d680ab170 legacy=0x7f1d680a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68155c90 con 0x7f1d680c5990 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1d58001670 con 0x7f1d680c5990 2024-03-06T07:23:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 >> v1:172.21.15.17:6789/0 conn(0x7f1d680c5990 legacy=0x7f1d6806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d500039e0 0x7f1d50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d50004d40 0x7f1d500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681559b0 con 0x7f1d50004d40 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5000b9c0 con 0x7f1d500039e0 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5000bbc0 con 0x7f1d6806c900 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d50004d40 0x7f1d500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d67fff640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d500039e0 0x7f1d50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d677fe640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 -- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 msgr2=0x7f1d50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d6806c900 0x7f1d50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 -- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d500039e0 msgr2=0x7f1d50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d500039e0 0x7f1d50003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.083+0000 7f1d6ce25640 1 -- 172.21.15.17:0/3020135654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d50003520 con 0x7f1d50004d40 2024-03-06T07:23:16.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.084+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d50004d40 0x7f1d500051c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1d60001f30 tx=0x7f1d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7512ea91858202e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.084+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60001b30 con 0x7f1d50004d40 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.084+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6002d6e0 con 0x7f1d50004d40 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.084+0000 7f1d66ffd640 1 -- 172.21.15.17:0/3020135654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6002dad0 con 0x7f1d50004d40 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 -- 172.21.15.17:0/3020135654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d50004d40 msgr2=0x7f1d500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 --2- 172.21.15.17:0/3020135654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d50004d40 0x7f1d500051c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1d60001f30 tx=0x7f1d50005830 comp rx=0 tx=0).stop 2024-03-06T07:23:16.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6ce25640 1 -- 172.21.15.17:0/3020135654 reap_dead start 2024-03-06T07:23:16.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 -- 172.21.15.17:0/3020135654 shutdown_connections 2024-03-06T07:23:16.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 -- 172.21.15.17:0/3020135654 >> 172.21.15.17:0/3020135654 conn(0x7f1d680c6710 msgr2=0x7f1d680c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 -- 172.21.15.17:0/3020135654 shutdown_connections 2024-03-06T07:23:16.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.085+0000 7f1d6e888640 1 -- 172.21.15.17:0/3020135654 wait complete. 2024-03-06T07:23:16.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.086+0000 7f1d6e888640 1 Processor -- start 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.086+0000 7f1d6e888640 1 -- start start 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d680a7720 0x7f1d68072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 0x7f1d681db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68156610 con 0x7f1d681dbf30 2024-03-06T07:23:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68156490 con 0x7f1d68153990 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6e888640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680c3af0 con 0x7f1d680a7720 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52054/0 (socket says 172.21.15.17:52054) 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 -- 172.21.15.17:0/4012647963 learned_addr learned my addr 172.21.15.17:0/4012647963 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d67fff640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d680a7720 0x7f1d68072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d677fe640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 0x7f1d681db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 -- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d680a7720 msgr2=0x7f1d68072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d680a7720 0x7f1d68072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 -- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 msgr2=0x7f1d681db9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 0x7f1d681db9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 -- 172.21.15.17:0/4012647963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d681ece00 con 0x7f1d681dbf30 2024-03-06T07:23:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d677fe640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 0x7f1d681db9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.087+0000 7f1d6ce25640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002c120 tx=0x7f1d600015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.088+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60001b30 con 0x7f1d681dbf30 2024-03-06T07:23:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.088+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6002ddb0 con 0x7f1d681dbf30 2024-03-06T07:23:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.088+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68154180 con 0x7f1d681dbf30 2024-03-06T07:23:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.088+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60003bb0 con 0x7f1d681dbf30 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.088+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d681eca90 con 0x7f1d681dbf30 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.089+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d681546a0 con 0x7f1d681dbf30 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.089+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d60047020 con 0x7f1d681dbf30 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.089+0000 7f1d64ff9640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 0x7f1d5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.089+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f1d600ab860 con 0x7f1d681dbf30 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.090+0000 7f1d67fff640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 0x7f1d5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.090+0000 7f1d67fff640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 0x7f1d5406d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000c00 tx=0x7f1d580078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.094+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d6007a8a0 con 0x7f1d681dbf30 2024-03-06T07:23:16.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.275+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1d6804f330 con 0x7f1d681dbf30 2024-03-06T07:23:16.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.278+0000 7f1d64ff9640 1 -- 172.21.15.17:0/4012647963 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1d6007a1a0 con 0x7f1d681dbf30 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 msgr2=0x7f1d5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 0x7f1d5406d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d58000c00 tx=0x7f1d580078f0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 msgr2=0x7f1d681458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002c120 tx=0x7f1d600015b0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 shutdown_connections 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d5406b500 0x7f1d5406d9c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d681dbf30 0x7f1d681458b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d68153990 0x7f1d681db9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 --2- 172.21.15.17:0/4012647963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d680a7720 0x7f1d68072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 >> 172.21.15.17:0/4012647963 conn(0x7f1d680c6710 msgr2=0x7f1d6806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 shutdown_connections 2024-03-06T07:23:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.280+0000 7f1d6e888640 1 -- 172.21.15.17:0/4012647963 wait complete. 2024-03-06T07:23:16.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:23:16.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.397+0000 7fb49af25640 1 Processor -- start 2024-03-06T07:23:16.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 -- start start 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb49406b3a0 0x7fb494158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb494159160 0x7fb49415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49406b800 con 0x7fb49406afc0 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb494158790 con 0x7fb49406b3a0 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb49af25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c5840 con 0x7fb494159160 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb493fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb49406b3a0 0x7fb494158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb493fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb49406b3a0 0x7fb494158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52064/0 (socket says 172.21.15.17:52064) 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.398+0000 7fb493fff640 1 -- 172.21.15.17:0/2328740584 learned_addr learned my addr 172.21.15.17:0/2328740584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb4994c2640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb494159160 0x7fb49415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb493fff640 1 -- 172.21.15.17:0/2328740584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb494159160 msgr2=0x7fb49415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb493fff640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb494159160 0x7fb49415fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb493fff640 1 -- 172.21.15.17:0/2328740584 >> v1:172.21.15.163:6789/0 conn(0x7fb49406afc0 legacy=0x7fb4940c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb493fff640 1 -- 172.21.15.17:0/2328740584 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940c5b20 con 0x7fb49406b3a0 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.399+0000 7fb493fff640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494158c20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb480009870 tx=0x7fb48002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba582519f879d3d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb4937fe640 1 -- 172.21.15.17:0/2328740584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4800317c0 con 0x7fb49406b3a0 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb4937fe640 1 -- 172.21.15.17:0/2328740584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480037070 con 0x7fb49406b3a0 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb4937fe640 1 -- 172.21.15.17:0/2328740584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480031b80 con 0x7fb49406b3a0 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 -- 172.21.15.17:0/2328740584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 msgr2=0x7fb494158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494158c20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb480009870 tx=0x7fb48002b860 comp rx=0 tx=0).stop 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 -- 172.21.15.17:0/2328740584 shutdown_connections 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb494159160 0x7fb49415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 --2- 172.21.15.17:0/2328740584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494158c20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 -- 172.21.15.17:0/2328740584 >> 172.21.15.17:0/2328740584 conn(0x7fb4940c4220 msgr2=0x7fb4940c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 -- 172.21.15.17:0/2328740584 shutdown_connections 2024-03-06T07:23:16.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.400+0000 7fb49af25640 1 -- 172.21.15.17:0/2328740584 wait complete. 2024-03-06T07:23:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.401+0000 7fb49af25640 1 Processor -- start 2024-03-06T07:23:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.401+0000 7fb49af25640 1 -- start start 2024-03-06T07:23:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49406afc0 0x7fb494157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 0x7fb494146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4941595f0 con 0x7fb49406b3a0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c60e0 con 0x7fb494159160 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb49af25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb494155220 con 0x7fb49406afc0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb493fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb493fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52074/0 (socket says 172.21.15.17:52074) 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb493fff640 1 -- 172.21.15.17:0/1279412871 learned_addr learned my addr 172.21.15.17:0/1279412871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb498cc1640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49406afc0 0x7fb494157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.402+0000 7fb4994c2640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 0x7fb494146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 -- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49406afc0 msgr2=0x7fb494157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49406afc0 0x7fb494157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 -- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 msgr2=0x7fb494146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 0x7fb494146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 -- 172.21.15.17:0/1279412871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49406e710 con 0x7fb49406b3a0 2024-03-06T07:23:16.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb4994c2640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 0x7fb494146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb493fff640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097c0 tx=0x7fb4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480032860 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480032a20 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49406dc70 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480032e10 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.403+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940a7fc0 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.404+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb494154f50 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.404+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb48004c060 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.405+0000 7fb4917fa640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 0x7fb47806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.405+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fb4800abb80 con 0x7fb49406b3a0 2024-03-06T07:23:16.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.405+0000 7fb498cc1640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 0x7fb47806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.405+0000 7fb498cc1640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 0x7fb47806d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb494154ef0 tx=0x7fb488007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.409+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb494154f50 con 0x7fb49406b3a0 2024-03-06T07:23:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.592+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7fb4940c6730 con 0x7fb49406b3a0 2024-03-06T07:23:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.595+0000 7fb4917fa640 1 -- 172.21.15.17:0/1279412871 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v30) v1 ==== 96+0+0 (secure 0 0 0) 0x7fb480040070 con 0x7fb49406b3a0 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 msgr2=0x7fb47806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 0x7fb47806d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb494154ef0 tx=0x7fb488007e30 comp rx=0 tx=0).stop 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 msgr2=0x7fb494146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097c0 tx=0x7fb4800046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 shutdown_connections 2024-03-06T07:23:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb47806b500 0x7fb47806d9c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb494159160 0x7fb494146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb49406b3a0 0x7fb494146500 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 --2- 172.21.15.17:0/1279412871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb49406afc0 0x7fb494157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 >> 172.21.15.17:0/1279412871 conn(0x7fb4940c4220 msgr2=0x7fb49406c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 shutdown_connections 2024-03-06T07:23:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.597+0000 7fb49af25640 1 -- 172.21.15.17:0/1279412871 wait complete. 2024-03-06T07:23:16.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused' 2024-03-06T07:23:16.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.709+0000 7fab24833640 1 Processor -- start 2024-03-06T07:23:16.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.709+0000 7fab24833640 1 -- start start 2024-03-06T07:23:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab24833640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab200c88c0 0x7fab20155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab24833640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab20158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab24833640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab200c73d0 con 0x7fab2014f360 2024-03-06T07:23:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab24833640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab200c8dd0 con 0x7fab200c88c0 2024-03-06T07:23:16.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab24833640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200be2e0 con 0x7fab20158b60 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab1dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab200c88c0 0x7fab20155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab1dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab200c88c0 0x7fab20155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52078/0 (socket says 172.21.15.17:52078) 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab1dd9b640 1 -- 172.21.15.17:0/1058811059 learned_addr learned my addr 172.21.15.17:0/1058811059 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.710+0000 7fab1ed9d640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab20158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1dd9b640 1 -- 172.21.15.17:0/1058811059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 msgr2=0x7fab20158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab20158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1dd9b640 1 -- 172.21.15.17:0/1058811059 >> v1:172.21.15.163:6789/0 conn(0x7fab2014f360 legacy=0x7fab200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:16.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1dd9b640 1 -- 172.21.15.17:0/1058811059 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200c7670 con 0x7fab200c88c0 2024-03-06T07:23:16.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab20155c10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fab08009870 tx=0x7fab0802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1edc56b5c2203 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1d59a640 1 -- 172.21.15.17:0/1058811059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab080317c0 con 0x7fab200c88c0 2024-03-06T07:23:16.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.711+0000 7fab1d59a640 1 -- 172.21.15.17:0/1058811059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08037070 con 0x7fab200c88c0 2024-03-06T07:23:16.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab1d59a640 1 -- 172.21.15.17:0/1058811059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08031b80 con 0x7fab200c88c0 2024-03-06T07:23:16.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 -- 172.21.15.17:0/1058811059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 msgr2=0x7fab20155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab20155c10 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fab08009870 tx=0x7fab0802b860 comp rx=0 tx=0).stop 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 -- 172.21.15.17:0/1058811059 shutdown_connections 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab20158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 --2- 172.21.15.17:0/1058811059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab20155c10 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 -- 172.21.15.17:0/1058811059 >> 172.21.15.17:0/1058811059 conn(0x7fab200ac570 msgr2=0x7fab200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 -- 172.21.15.17:0/1058811059 shutdown_connections 2024-03-06T07:23:16.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.712+0000 7fab24833640 1 -- 172.21.15.17:0/1058811059 wait complete. 2024-03-06T07:23:16.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.713+0000 7fab24833640 1 Processor -- start 2024-03-06T07:23:16.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.713+0000 7fab24833640 1 -- start start 2024-03-06T07:23:16.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 0x7fab2014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab200abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c5a50 con 0x7fab200c88c0 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c7c30 con 0x7fab2014f360 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab24833640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200c5bd0 con 0x7fab20158b60 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52086/0 (socket says 172.21.15.17:52086) 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 -- 172.21.15.17:0/670264180 learned_addr learned my addr 172.21.15.17:0/670264180 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1ed9d640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab200abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 0x7fab2014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 -- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 msgr2=0x7fab200abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab200abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 -- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 msgr2=0x7fab2014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 0x7fab2014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1e59c640 1 -- 172.21.15.17:0/670264180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2014c3a0 con 0x7fab200c88c0 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.714+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 0x7fab2014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab1e59c640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b3f0 tx=0x7fab1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18015020 con 0x7fab200c88c0 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab18011830 con 0x7fab200c88c0 2024-03-06T07:23:16.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18011c20 con 0x7fab200c88c0 2024-03-06T07:23:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab200c3ed0 con 0x7fab200c88c0 2024-03-06T07:23:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.715+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab201eb000 con 0x7fab200c88c0 2024-03-06T07:23:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.716+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab200c81d0 con 0x7fab200c88c0 2024-03-06T07:23:16.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.716+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab18018020 con 0x7fab200c88c0 2024-03-06T07:23:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.716+0000 7fab16ffd640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 0x7fab0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.717+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fab18089d00 con 0x7fab200c88c0 2024-03-06T07:23:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.717+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 0x7fab0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:16.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.717+0000 7fab1dd9b640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 0x7fab0006d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab08036000 tx=0x7fab08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:16.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.721+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab200c81d0 con 0x7fab200c88c0 2024-03-06T07:23:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:16.975+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]} v 0) v1 -- 0x7fab2004f330 con 0x7fab200c88c0 2024-03-06T07:23:17.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.324+0000 7fab16ffd640 1 -- 172.21.15.17:0/670264180 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_last_resort_refused"]}]=0 v199) v1 ==== 158+0+0 (secure 0 0 0) 0x7fab1801f030 con 0x7fab200c88c0 2024-03-06T07:23:17.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 msgr2=0x7fab0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 0x7fab0006d9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab08036000 tx=0x7fab08008030 comp rx=0 tx=0).stop 2024-03-06T07:23:17.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 msgr2=0x7fab201eee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fab1800b3f0 tx=0x7fab1800b8b0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 shutdown_connections 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab0006b500 0x7fab0006d9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab20158b60 0x7fab200abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab2014f360 0x7fab2014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 --2- 172.21.15.17:0/670264180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab200c88c0 0x7fab201eee20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 >> 172.21.15.17:0/670264180 conn(0x7fab200ac570 msgr2=0x7fab200ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 shutdown_connections 2024-03-06T07:23:17.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.326+0000 7fab24833640 1 -- 172.21.15.17:0/670264180 wait complete. 2024-03-06T07:23:17.340 INFO:tasks.cephfs_test_runner:That a standby with mds_join_fs set to another fs is not used if refuse_standby_for_another_fs is set. ... ok 2024-03-06T07:23:17.341 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_runtime (tasks.cephfs.test_failover.TestClusterAffinity) That setting mds_join_fs at runtime affects the cluster layout. 2024-03-06T07:23:17.341 INFO:tasks.cephfs_test_runner:test_join_fs_runtime (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:23:17.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.444+0000 7f8945fc9640 1 Processor -- start 2024-03-06T07:23:17.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 -- start start 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89400c5480 0x7f8940155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400c5860 0x7f894006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400c5cc0 con 0x7f89401586f0 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894006df40 con 0x7f89400c5860 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f8945fc9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89401591c0 con 0x7f89400c5480 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f893effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f89400c5860 0x7f894006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f893ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f89401586f0 0x7f89400724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59686/0 (socket says 172.21.15.17:59686) 2024-03-06T07:23:17.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.445+0000 7f893ffff640 1 -- 172.21.15.17:0/2044029628 learned_addr learned my addr 172.21.15.17:0/2044029628 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893effd640 1 -- 172.21.15.17:0/2044029628 >> v2:172.21.15.163:3300/0 conn(0x7f89400c5480 msgr2=0x7f8940155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893effd640 1 --2- 172.21.15.17:0/2044029628 >> v2:172.21.15.163:3300/0 conn(0x7f89400c5480 0x7f8940155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893effd640 1 -- 172.21.15.17:0/2044029628 >> v1:172.21.15.17:6789/0 conn(0x7f89401586f0 legacy=0x7f89400724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893effd640 1 -- 172.21.15.17:0/2044029628 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89401594a0 con 0x7f89400c5860 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893e7fc640 1 -- 172.21.15.17:0/2044029628 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321379909 0 0) 0x7f89400c5cc0 con 0x7f89401586f0 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.446+0000 7f893effd640 1 --2- 172.21.15.17:0/2044029628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f894006f6a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f89400c1cf0 tx=0x7f893402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afedcfde8c1fb377 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f893e7fc640 1 -- 172.21.15.17:0/2044029628 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89340315a0 con 0x7f89400c5860 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f893e7fc640 1 -- 172.21.15.17:0/2044029628 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934037070 con 0x7f89400c5860 2024-03-06T07:23:17.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f893e7fc640 1 -- 172.21.15.17:0/2044029628 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934031960 con 0x7f89400c5860 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 -- 172.21.15.17:0/2044029628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 msgr2=0x7f894006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 --2- 172.21.15.17:0/2044029628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f894006f6a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f89400c1cf0 tx=0x7f893402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 -- 172.21.15.17:0/2044029628 shutdown_connections 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 --2- 172.21.15.17:0/2044029628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f894006f6a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 --2- 172.21.15.17:0/2044029628 >> v2:172.21.15.163:3300/0 conn(0x7f89400c5480 0x7f8940155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 -- 172.21.15.17:0/2044029628 >> 172.21.15.17:0/2044029628 conn(0x7f894006b1a0 msgr2=0x7f894006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 -- 172.21.15.17:0/2044029628 shutdown_connections 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.447+0000 7f8945fc9640 1 -- 172.21.15.17:0/2044029628 wait complete. 2024-03-06T07:23:17.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.448+0000 7f8945fc9640 1 Processor -- start 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.448+0000 7f8945fc9640 1 -- start start 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 0x7f8940153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 0x7f8940151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940152a20 con 0x7f89400c5860 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940155680 con 0x7f89401586f0 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f8945fc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8940155050 con 0x7f89400c5480 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52102/0 (socket says 172.21.15.17:52102) 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 -- 172.21.15.17:0/230858740 learned_addr learned my addr 172.21.15.17:0/230858740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893ffff640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 0x7f8940151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 -- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 msgr2=0x7f8940153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893f7fe640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 0x7f8940153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 0x7f8940153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 -- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 msgr2=0x7f8940151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 0x7f8940151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893effd640 1 -- 172.21.15.17:0/230858740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400ab5a0 con 0x7f89400c5860 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893ffff640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 0x7f8940151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.449+0000 7f893f7fe640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 0x7f8940153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f893effd640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8934000c00 tx=0x7f89340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934032840 con 0x7f89400c5860 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934032a00 con 0x7f89400c5860 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89401ee170 con 0x7f89400c5860 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934032df0 con 0x7f89400c5860 2024-03-06T07:23:17.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.450+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89401e1d70 con 0x7f89400c5860 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.451+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89400c7a50 con 0x7f89400c5860 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.451+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f893404c020 con 0x7f89400c5860 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.452+0000 7f8927fff640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 0x7f892c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.452+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f893402f020 con 0x7f89400c5860 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.452+0000 7f893f7fe640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 0x7f892c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.453+0000 7f893f7fe640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 0x7f892c06d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f893000a870 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.456+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89400c7a50 con 0x7f89400c5860 2024-03-06T07:23:17.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.637+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f894004f330 con 0x7f89400c5860 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.638+0000 7f8927fff640 1 -- 172.21.15.17:0/230858740 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+13688 (secure 0 0 0) 0x7f8934040070 con 0x7f89400c5860 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:17.639 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":87,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:15.318157+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/1858186621":"2024-03-07T07:23:12.310865+0000","172.21.15.163:6835/1858186621":"2024-03-07T07:23:12.310865+0000","172.21.15.163:6836/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.163:6834/1549441863":"2024-03-07T07:23:05.746191+0000","172.21.15.17:6835/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.17:6833/285046827":"2024-03-07T07:23:06.750526+0000","172.21.15.163:6836/1213270757":"2024-03-07T07:22:54.958935+0000","172.21.15.163:6834/1213270757":"2024-03-07T07:22:54.958935+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 msgr2=0x7f892c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 0x7f892c06d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f893000a870 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 msgr2=0x7f8940150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8934000c00 tx=0x7f89340046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 shutdown_connections 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892c06b4d0 0x7f892c06d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89401586f0 0x7f8940151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89400c5860 0x7f8940150c30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 --2- 172.21.15.17:0/230858740 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89400c5480 0x7f8940153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 >> 172.21.15.17:0/230858740 conn(0x7f894006b1a0 msgr2=0x7f89401de6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 shutdown_connections 2024-03-06T07:23:17.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.640+0000 7f8945fc9640 1 -- 172.21.15.17:0/230858740 wait complete. 2024-03-06T07:23:17.653 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:17.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:17.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.755+0000 7ff96d9bf640 1 Processor -- start 2024-03-06T07:23:17.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.755+0000 7ff96d9bf640 1 -- start start 2024-03-06T07:23:17.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff96d9bf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9680c5370 0x7ff9680c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff96d9bf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff968155530 0x7ff9681559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff96d9bf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff968152dd0 con 0x7ff9681529f0 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff96d9bf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff968159710 con 0x7ff9680c5370 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff96d9bf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9680c68b0 con 0x7ff968155530 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9667fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9680c5370 0x7ff9680c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff966ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff9681529f0 0x7ff9680c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59698/0 (socket says 172.21.15.17:59698) 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff966ffd640 1 -- 172.21.15.17:0/1968297271 learned_addr learned my addr 172.21.15.17:0/1968297271 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9677fe640 1 --2- 172.21.15.17:0/1968297271 >> v2:172.21.15.163:3300/0 conn(0x7ff968155530 0x7ff9681559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9667fc640 1 -- 172.21.15.17:0/1968297271 >> v2:172.21.15.163:3300/0 conn(0x7ff968155530 msgr2=0x7ff9681559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9667fc640 1 --2- 172.21.15.17:0/1968297271 >> v2:172.21.15.163:3300/0 conn(0x7ff968155530 0x7ff9681559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9667fc640 1 -- 172.21.15.17:0/1968297271 >> v1:172.21.15.17:6789/0 conn(0x7ff9681529f0 legacy=0x7ff9680c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.756+0000 7ff9667fc640 1 -- 172.21.15.17:0/1968297271 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968158b00 con 0x7ff9680c5370 2024-03-06T07:23:17.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.757+0000 7ff9667fc640 1 --2- 172.21.15.17:0/1968297271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9680c5370 0x7ff9680c57d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff9680735c0 tx=0x7ff95002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84e16827c1ab796 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.757+0000 7ff965ffb640 1 -- 172.21.15.17:0/1968297271 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9500315a0 con 0x7ff9680c5370 2024-03-06T07:23:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.757+0000 7ff965ffb640 1 -- 172.21.15.17:0/1968297271 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950037070 con 0x7ff9680c5370 2024-03-06T07:23:17.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.757+0000 7ff965ffb640 1 -- 172.21.15.17:0/1968297271 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950031960 con 0x7ff9680c5370 2024-03-06T07:23:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 -- 172.21.15.17:0/1968297271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9680c5370 msgr2=0x7ff9680c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/1968297271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9680c5370 0x7ff9680c57d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff9680735c0 tx=0x7ff95002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:17.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 -- 172.21.15.17:0/1968297271 shutdown_connections 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/1968297271 >> v2:172.21.15.163:3300/0 conn(0x7ff968155530 0x7ff9681559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/1968297271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9680c5370 0x7ff9680c57d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 -- 172.21.15.17:0/1968297271 >> 172.21.15.17:0/1968297271 conn(0x7ff9680ac580 msgr2=0x7ff9680ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 -- 172.21.15.17:0/1968297271 shutdown_connections 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.758+0000 7ff96d9bf640 1 -- 172.21.15.17:0/1968297271 wait complete. 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.759+0000 7ff96d9bf640 1 Processor -- start 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.759+0000 7ff96d9bf640 1 -- start start 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 0x7ff96814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff968155530 0x7ff9681431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681590c0 con 0x7ff9681529f0 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c9000 con 0x7ff968155530 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681593a0 con 0x7ff9680c5370 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52112/0 (socket says 172.21.15.17:52112) 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 -- 172.21.15.17:0/2782021697 learned_addr learned my addr 172.21.15.17:0/2782021697 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9677fe640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff968155530 0x7ff9681431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff966ffd640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 0x7ff96814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 -- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 msgr2=0x7ff96814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 0x7ff96814cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 -- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff968155530 msgr2=0x7ff9681431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff968155530 0x7ff9681431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 -- 172.21.15.17:0/2782021697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968151ae0 con 0x7ff9681529f0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff966ffd640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 0x7ff96814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff9667fc640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff9500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950032840 con 0x7ff9681529f0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950032a00 con 0x7ff9681529f0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96814dba0 con 0x7ff9681529f0 2024-03-06T07:23:17.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.760+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950032df0 con 0x7ff9681529f0 2024-03-06T07:23:17.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.761+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96814d440 con 0x7ff9681529f0 2024-03-06T07:23:17.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.762+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9680ca180 con 0x7ff9681529f0 2024-03-06T07:23:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.762+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff95004d3f0 con 0x7ff9681529f0 2024-03-06T07:23:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.762+0000 7ff95ffff640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 0x7ff94806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.762+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7ff95002f030 con 0x7ff9681529f0 2024-03-06T07:23:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.763+0000 7ff966ffd640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 0x7ff94806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:17.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.763+0000 7ff966ffd640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 0x7ff94806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff960000f60 tx=0x7ff960009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:17.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.767+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9680ca180 con 0x7ff9681529f0 2024-03-06T07:23:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.947+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff96804f330 con 0x7ff9681529f0 2024-03-06T07:23:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.947+0000 7ff95ffff640 1 -- 172.21.15.17:0/2782021697 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff950040070 con 0x7ff9681529f0 2024-03-06T07:23:17.949 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 msgr2=0x7ff94806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 0x7ff94806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff960000f60 tx=0x7ff960009040 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 msgr2=0x7ff968142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff9500046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 shutdown_connections 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff94806b4d0 0x7ff94806d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff968155530 0x7ff9681431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9681529f0 0x7ff968142c80 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 --2- 172.21.15.17:0/2782021697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9680c5370 0x7ff96814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 >> 172.21.15.17:0/2782021697 conn(0x7ff9680ac580 msgr2=0x7ff9680c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 shutdown_connections 2024-03-06T07:23:17.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:17.950+0000 7ff96d9bf640 1 -- 172.21.15.17:0/2782021697 wait complete. 2024-03-06T07:23:17.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime' 2024-03-06T07:23:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.062+0000 7fd4c5381640 1 Processor -- start 2024-03-06T07:23:18.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.062+0000 7fd4c5381640 1 -- start start 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4c5381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4c01566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4c5381640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4c00c6000 0x7fd4c0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4c5381640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c0153a60 con 0x7fd4c0150580 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4c5381640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c0153d40 con 0x7fd4c00c6000 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4c5381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c006c2c0 con 0x7fd4c00c1320 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4beffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd4c0150580 0x7fd4c006dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59702/0 (socket says 172.21.15.17:59702) 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4beffd640 1 -- 172.21.15.17:0/3404244656 learned_addr learned my addr 172.21.15.17:0/3404244656 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:18.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4be7fc640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4c01566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/3404244656 >> v2:172.21.15.163:3300/0 conn(0x7fd4c00c6000 0x7fd4c0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641038231 0 0) 0x7fd4c0153a60 con 0x7fd4c0150580 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4ac003520 con 0x7fd4c0150580 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260028925 0 0) 0x7fd4ac003520 con 0x7fd4c0150580 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 msgr2=0x7fd4c01566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4c01566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 >> v2:172.21.15.163:3300/0 conn(0x7fd4c00c6000 msgr2=0x7fd4c0156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 --2- 172.21.15.17:0/3404244656 >> v2:172.21.15.163:3300/0 conn(0x7fd4c00c6000 0x7fd4c0156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c006c5a0 con 0x7fd4c0150580 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd4b8001670 con 0x7fd4c0150580 2024-03-06T07:23:18.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 >> v1:172.21.15.17:6789/0 conn(0x7fd4c0150580 legacy=0x7fd4c006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:18.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.063+0000 7fd4bdffb640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac003f30 0x7fd4ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bdffb640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac004d40 0x7fd4ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c0153d40 con 0x7fd4ac004d40 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c006c2c0 con 0x7fd4ac003f30 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac00ba10 con 0x7fd4c00c1320 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac004d40 0x7fd4ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4beffd640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4be7fc640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac003f30 0x7fd4ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 msgr2=0x7fd4ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:18.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac003f30 msgr2=0x7fd4ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac003f30 0x7fd4ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/3404244656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac003520 con 0x7fd4ac004d40 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4beffd640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c00c1320 0x7fd4ac0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.064+0000 7fd4be7fc640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac003f30 0x7fd4ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac004d40 0x7fd4ac0051c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000e170 tx=0x7fd4ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0a68facebbdc81 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0039070 con 0x7fd4ac004d40 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b002cc20 con 0x7fd4ac004d40 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4bdffb640 1 -- 172.21.15.17:0/3404244656 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0009c10 con 0x7fd4ac004d40 2024-03-06T07:23:18.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 -- 172.21.15.17:0/3404244656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac004d40 msgr2=0x7fd4ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 --2- 172.21.15.17:0/3404244656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac004d40 0x7fd4ac0051c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000e170 tx=0x7fd4ac005830 comp rx=0 tx=0).stop 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/3404244656 reap_dead start 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 -- 172.21.15.17:0/3404244656 shutdown_connections 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 -- 172.21.15.17:0/3404244656 >> 172.21.15.17:0/3404244656 conn(0x7fd4c00c6710 msgr2=0x7fd4c00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 -- 172.21.15.17:0/3404244656 shutdown_connections 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.065+0000 7fd4c5381640 1 -- 172.21.15.17:0/3404244656 wait complete. 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 Processor -- start 2024-03-06T07:23:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 -- start start 2024-03-06T07:23:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac004d40 0x7fd4c00a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c01ec990 0x7fd4c01e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c006e5a0 con 0x7fd4c0154f40 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c006e420 con 0x7fd4ac004d40 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4c5381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c00c0c40 con 0x7fd4c01ec990 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52122/0 (socket says 172.21.15.17:52122) 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/263903581 learned_addr learned my addr 172.21.15.17:0/263903581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4beffd640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac004d40 0x7fd4c00a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.067+0000 7fd4be7fc640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c01ec990 0x7fd4c01e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c01ec990 msgr2=0x7fd4c01e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c01ec990 0x7fd4c01e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac004d40 msgr2=0x7fd4c00a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac004d40 0x7fd4c00a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 -- 172.21.15.17:0/263903581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c0072ed0 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4bf7fe640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0037040 tx=0x7fd4b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0039070 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b003a040 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c00731b0 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0003990 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.068+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c00a7c60 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.069+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c0073380 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.069+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4b0046070 con 0x7fd4c0154f40 2024-03-06T07:23:18.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.070+0000 7fd4b77fe640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 0x7fd4a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:18.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.070+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fd4b00abcc0 con 0x7fd4c0154f40 2024-03-06T07:23:18.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.070+0000 7fd4beffd640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 0x7fd4a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:18.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.071+0000 7fd4beffd640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 0x7fd4a006d9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000fb0 tx=0x7fd4b8007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:18.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.074+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd4c0073380 con 0x7fd4c0154f40 2024-03-06T07:23:18.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:18.326+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]} v 0) v1 -- 0x7fd4c0155dc0 con 0x7fd4c0154f40 2024-03-06T07:23:19.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.330+0000 7fd4b77fe640 1 -- 172.21.15.17:0/263903581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]}]=0 v201) v1 ==== 149+0+0 (secure 0 0 0) 0x7fd4b007a560 con 0x7fd4c0154f40 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 msgr2=0x7fd4a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 0x7fd4a006d9c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8000fb0 tx=0x7fd4b8007040 comp rx=0 tx=0).stop 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 msgr2=0x7fd4c01e8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0037040 tx=0x7fd4b0009750 comp rx=0 tx=0).stop 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 shutdown_connections 2024-03-06T07:23:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4a006b500 0x7fd4a006d9c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4c0154f40 0x7fd4c01e8f50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4c01ec990 0x7fd4c01e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 --2- 172.21.15.17:0/263903581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac004d40 0x7fd4c00a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 >> 172.21.15.17:0/263903581 conn(0x7fd4c00c6710 msgr2=0x7fd4c014dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 shutdown_connections 2024-03-06T07:23:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.333+0000 7fd4c5381640 1 -- 172.21.15.17:0/263903581 wait complete. 2024-03-06T07:23:19.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:23:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.457+0000 7fa5fafac640 1 Processor -- start 2024-03-06T07:23:19.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.457+0000 7fa5fafac640 1 -- start start 2024-03-06T07:23:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5f40b0070 0x7fa5f4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 0x7fa5f40c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 0x7fa5f40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f406c020 con 0x7fa5f40b0070 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40b0450 con 0x7fa5f406ca70 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5fafac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f406cf20 con 0x7fa5f40c5380 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5f8d48640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5f40b0070 0x7fa5f4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5f8d48640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa5f40b0070 0x7fa5f4155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52136/0 (socket says 172.21.15.17:52136) 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.458+0000 7fa5f8d48640 1 -- 172.21.15.17:0/3831388869 learned_addr learned my addr 172.21.15.17:0/3831388869 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 -- 172.21.15.17:0/3831388869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 msgr2=0x7fa5f40c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f3fff640 1 --2- 172.21.15.17:0/3831388869 >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 0x7fa5f40c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f9549640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 0x7fa5f40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 0x7fa5f40c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 -- 172.21.15.17:0/3831388869 >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 msgr2=0x7fa5f40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/3831388869 >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 0x7fa5f40c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 -- 172.21.15.17:0/3831388869 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f40c77f0 con 0x7fa5f40b0070 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f3fff640 1 --2- 172.21.15.17:0/3831388869 >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 0x7fa5f40c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f9549640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 0x7fa5f40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.459+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40b0070 0x7fa5f4155c30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa5f406d4c0 tx=0x7fa5ec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1f48e58375899 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5f37fe640 1 -- 172.21.15.17:0/3831388869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec02c9e0 con 0x7fa5f40b0070 2024-03-06T07:23:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5f37fe640 1 -- 172.21.15.17:0/3831388869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec02cb80 con 0x7fa5f40b0070 2024-03-06T07:23:19.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5f37fe640 1 -- 172.21.15.17:0/3831388869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0343f0 con 0x7fa5f40b0070 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 -- 172.21.15.17:0/3831388869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40b0070 msgr2=0x7fa5f4155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40b0070 0x7fa5f4155c30 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa5f406d4c0 tx=0x7fa5ec02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 -- 172.21.15.17:0/3831388869 shutdown_connections 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f40c5380 0x7fa5f40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 --2- 172.21.15.17:0/3831388869 >> v2:172.21.15.163:3300/0 conn(0x7fa5f406ca70 0x7fa5f40c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 --2- 172.21.15.17:0/3831388869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40b0070 0x7fa5f4155c30 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 -- 172.21.15.17:0/3831388869 >> 172.21.15.17:0/3831388869 conn(0x7fa5f40ac590 msgr2=0x7fa5f40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 -- 172.21.15.17:0/3831388869 shutdown_connections 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.460+0000 7fa5fafac640 1 -- 172.21.15.17:0/3831388869 wait complete. 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.461+0000 7fa5fafac640 1 Processor -- start 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.461+0000 7fa5fafac640 1 -- start start 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f406ca70 0x7fa5f4159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5f40b0070 0x7fa5f40c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f406c550 con 0x7fa5f40c5380 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f406c270 con 0x7fa5f40b0070 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5fafac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f406da80 con 0x7fa5f406ca70 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5f9549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5f9549640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52150/0 (socket says 172.21.15.17:52150) 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.462+0000 7fa5f9549640 1 -- 172.21.15.17:0/1177664983 learned_addr learned my addr 172.21.15.17:0/1177664983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f3fff640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5f40b0070 0x7fa5f40c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f406ca70 0x7fa5f4159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 -- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f406ca70 msgr2=0x7fa5f4159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f406ca70 0x7fa5f4159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 -- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5f40b0070 msgr2=0x7fa5f40c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5f40b0070 0x7fa5f40c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 -- 172.21.15.17:0/1177664983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f41579d0 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f9549640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c820 tx=0x7fa5e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4015070 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e4007a30 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f40c45a0 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4007e20 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.463+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f40c4040 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.464+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f404f330 con 0x7fa5f40c5380 2024-03-06T07:23:19.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.464+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5e4018070 con 0x7fa5f40c5380 2024-03-06T07:23:19.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.465+0000 7fa5f17fa640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 0x7fa5e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.465+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa5e408a080 con 0x7fa5f40c5380 2024-03-06T07:23:19.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.465+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 0x7fa5e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.466+0000 7fa5f8d48640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 0x7fa5e006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec035000 tx=0x7fa5ec030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.469+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa5e4059000 con 0x7fa5f40c5380 2024-03-06T07:23:19.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.650+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa5f40af390 con 0x7fa5f40c5380 2024-03-06T07:23:19.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.653+0000 7fa5f17fa640 1 -- 172.21.15.17:0/1177664983 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa5e4058900 con 0x7fa5f40c5380 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 msgr2=0x7fa5e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 0x7fa5e006d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec035000 tx=0x7fa5ec030000 comp rx=0 tx=0).stop 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 msgr2=0x7fa5f414ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400c820 tx=0x7fa5e400cce0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 shutdown_connections 2024-03-06T07:23:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa5e006b4d0 0x7fa5e006d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5f40c5380 0x7fa5f414ac10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5f40b0070 0x7fa5f40c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 --2- 172.21.15.17:0/1177664983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5f406ca70 0x7fa5f4159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.655+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 >> 172.21.15.17:0/1177664983 conn(0x7fa5f40ac590 msgr2=0x7fa5f40adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.656+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 shutdown_connections 2024-03-06T07:23:19.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.656+0000 7fa5fafac640 1 -- 172.21.15.17:0/1177664983 wait complete. 2024-03-06T07:23:19.668 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:19.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:19.717 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:19.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:19.768 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:23:19.768 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:19.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:19.783 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:23:19.783 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:19.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:19.800 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:23:19.800 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:19.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 Processor -- start 2024-03-06T07:23:19.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 -- start start 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4440a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4440c36e0 0x7fd4441558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd444158800 con 0x7fd4440c3280 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4440c3b10 con 0x7fd4440c36e0 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.923+0000 7fd44b078640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd444158ae0 con 0x7fd44406d9f0 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4440c36e0 0x7fd4441558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4440c36e0 0x7fd4441558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52160/0 (socket says 172.21.15.17:52160) 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 -- 172.21.15.17:0/31120724 learned_addr learned my addr 172.21.15.17:0/31120724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd448e14640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4440a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 -- 172.21.15.17:0/31120724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 msgr2=0x7fd4440a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4440a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 -- 172.21.15.17:0/31120724 >> v1:172.21.15.163:6789/0 conn(0x7fd4440c3280 legacy=0x7fd4441551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd449615640 1 -- 172.21.15.17:0/31120724 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444158dc0 con 0x7fd4440c36e0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.924+0000 7fd448e14640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4440a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd449615640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441558f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009e50 tx=0x7fd43c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51f5f4f08da5486 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd4437fe640 1 -- 172.21.15.17:0/31120724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c02c9e0 con 0x7fd4440c36e0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd4437fe640 1 -- 172.21.15.17:0/31120724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c02cb80 con 0x7fd4440c36e0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd4437fe640 1 -- 172.21.15.17:0/31120724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0343f0 con 0x7fd4440c36e0 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 -- 172.21.15.17:0/31120724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 msgr2=0x7fd4441558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441558f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009e50 tx=0x7fd43c02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:19.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 -- 172.21.15.17:0/31120724 shutdown_connections 2024-03-06T07:23:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441558f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 --2- 172.21.15.17:0/31120724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4440a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 -- 172.21.15.17:0/31120724 >> 172.21.15.17:0/31120724 conn(0x7fd4440aa630 msgr2=0x7fd4440aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 -- 172.21.15.17:0/31120724 shutdown_connections 2024-03-06T07:23:19.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.925+0000 7fd44b078640 1 -- 172.21.15.17:0/31120724 wait complete. 2024-03-06T07:23:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.926+0000 7fd44b078640 1 Processor -- start 2024-03-06T07:23:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 -- start start 2024-03-06T07:23:19.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4441eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4440c3280 0x7fd44406f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd444155e30 con 0x7fd4440c36e0 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd444159380 con 0x7fd4440c3280 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd44b078640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44406ddd0 con 0x7fd44406d9f0 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd449615640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd449615640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52162/0 (socket says 172.21.15.17:52162) 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.927+0000 7fd449615640 1 -- 172.21.15.17:0/373129710 learned_addr learned my addr 172.21.15.17:0/373129710 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd443fff640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4440c3280 0x7fd44406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd448e14640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4441eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 -- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 msgr2=0x7fd4441eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4441eeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 -- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4440c3280 msgr2=0x7fd44406f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4440c3280 0x7fd44406f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 -- 172.21.15.17:0/373129710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4440a99e0 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd449615640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd43c036040 tx=0x7fd43c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c043020 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c03e450 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c009510 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4440a9cc0 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.928+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4441ee980 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.929+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44404f330 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.930+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd43c046020 con 0x7fd4440c36e0 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.930+0000 7fd4417fa640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 0x7fd42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:19.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.930+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fd43c0ac800 con 0x7fd4440c36e0 2024-03-06T07:23:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.930+0000 7fd448e14640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 0x7fd42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:19.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.931+0000 7fd448e14640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 0x7fd42806d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd43800aac0 tx=0x7fd438008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:19.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:19.934+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd43c04d030 con 0x7fd4440c36e0 2024-03-06T07:23:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.158+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd44414f7e0 con 0x7fd4440c36e0 2024-03-06T07:23:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.158+0000 7fd4417fa640 1 -- 172.21.15.17:0/373129710 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fd43c07b130 con 0x7fd4440c36e0 2024-03-06T07:23:20.160 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:20.160 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":74,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/925849724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":925849724},{"type":"v1","addr":"172.21.15.163:6837","nonce":925849724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":7,"addr":"172.21.15.17:6834/4026996022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4026996022},{"type":"v1","addr":"172.21.15.17:6834","nonce":4026996022}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":6532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3589667974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3589667974},{"type":"v1","addr":"172.21.15.163:6836","nonce":3589667974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6831,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4215297482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":4215297482},{"type":"v1","addr":"172.21.15.17:6835","nonce":4215297482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[]} 2024-03-06T07:23:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 74 2024-03-06T07:23:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.160+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 msgr2=0x7fd42806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.160+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 0x7fd42806d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd43800aac0 tx=0x7fd438008040 comp rx=0 tx=0).stop 2024-03-06T07:23:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.160+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 msgr2=0x7fd4441eaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.160+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd43c036040 tx=0x7fd43c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 shutdown_connections 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd42806b4d0 0x7fd42806d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4440c36e0 0x7fd4441eaee0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4440c3280 0x7fd44406f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 --2- 172.21.15.17:0/373129710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd44406d9f0 0x7fd4441eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 >> 172.21.15.17:0/373129710 conn(0x7fd4440aa630 msgr2=0x7fd444155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 shutdown_connections 2024-03-06T07:23:20.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.161+0000 7fd44b078640 1 -- 172.21.15.17:0/373129710 wait complete. 2024-03-06T07:23:20.173 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:23:20.173 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:23:20.174 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:23:20.174 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.174 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:23:20.174 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:23:20.175 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:23:20.175 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.175 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:23:20.175 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:23:20.175 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:23:20.175 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.176 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:23:20.176 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:23:20.176 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:23:20.176 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:20.182 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:23:20.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:23:20.184 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:23:20.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:23:20.186 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:23:20.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:23:20.187 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:23:20.188 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:23:20.229 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:23:20.230 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.230 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:23:20.231 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:23:20.231 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.232 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.232 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:23:20.233 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:20.233 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:23:20.306 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:23:20.306 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:23:20.312 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:23:20.312 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:23:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 Processor -- start 2024-03-06T07:23:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 -- start start 2024-03-06T07:23:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 0x7fa1540c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 0x7fa1540c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1541589b0 con 0x7fa154152890 2024-03-06T07:23:20.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1540c1800 con 0x7fa154155db0 2024-03-06T07:23:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.337+0000 7fa15a842640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154158c50 con 0x7fa154159450 2024-03-06T07:23:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa153fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa154152890 0x7fa154072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59716/0 (socket says 172.21.15.17:59716) 2024-03-06T07:23:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa153fff640 1 -- 172.21.15.17:0/201857913 learned_addr learned my addr 172.21.15.17:0/201857913 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa1537fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 0x7fa1540c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa1537fe640 1 -- 172.21.15.17:0/201857913 >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 msgr2=0x7fa1540c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa158ddf640 1 --2- 172.21.15.17:0/201857913 >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 0x7fa1540c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa1537fe640 1 --2- 172.21.15.17:0/201857913 >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 0x7fa1540c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa1537fe640 1 -- 172.21.15.17:0/201857913 >> v1:172.21.15.17:6789/0 conn(0x7fa154152890 legacy=0x7fa154072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.338+0000 7fa1537fe640 1 -- 172.21.15.17:0/201857913 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa154156240 con 0x7fa154159450 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.339+0000 7fa158ddf640 1 --2- 172.21.15.17:0/201857913 >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 0x7fa1540c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.339+0000 7fa1537fe640 1 --2- 172.21.15.17:0/201857913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 0x7fa1540c1170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1480097a0 tx=0x7fa14802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cfa821f1104771 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.345+0000 7fa152ffd640 1 -- 172.21.15.17:0/201857913 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148031780 con 0x7fa154159450 2024-03-06T07:23:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.345+0000 7fa152ffd640 1 -- 172.21.15.17:0/201857913 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148037070 con 0x7fa154159450 2024-03-06T07:23:20.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.345+0000 7fa152ffd640 1 -- 172.21.15.17:0/201857913 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148031b40 con 0x7fa154159450 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 -- 172.21.15.17:0/201857913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 msgr2=0x7fa1540c1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 --2- 172.21.15.17:0/201857913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 0x7fa1540c1170 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1480097a0 tx=0x7fa14802b820 comp rx=0 tx=0).stop 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 -- 172.21.15.17:0/201857913 shutdown_connections 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 --2- 172.21.15.17:0/201857913 >> v2:172.21.15.17:3300/0 conn(0x7fa154155db0 0x7fa1540c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 --2- 172.21.15.17:0/201857913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154159450 0x7fa1540c1170 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 -- 172.21.15.17:0/201857913 >> 172.21.15.17:0/201857913 conn(0x7fa1540c7de0 msgr2=0x7fa1540c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 -- 172.21.15.17:0/201857913 shutdown_connections 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.346+0000 7fa15a842640 1 -- 172.21.15.17:0/201857913 wait complete. 2024-03-06T07:23:20.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.347+0000 7fa15a842640 1 Processor -- start 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.347+0000 7fa15a842640 1 -- start start 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154152890 0x7fa1541ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154155db0 0x7fa1540ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa154152c70 con 0x7fa154159450 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa154072830 con 0x7fa154152890 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa15a842640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa154152df0 con 0x7fa154155db0 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa158ddf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa153fff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154152890 0x7fa1541ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa1537fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154155db0 0x7fa1540ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa158ddf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52202/0 (socket says 172.21.15.17:52202) 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.348+0000 7fa158ddf640 1 -- 172.21.15.17:0/1266014812 learned_addr learned my addr 172.21.15.17:0/1266014812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 -- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154155db0 msgr2=0x7fa1540ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154155db0 0x7fa1540ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 -- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154152890 msgr2=0x7fa1541ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154152890 0x7fa1541ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 -- 172.21.15.17:0/1266014812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1540a9310 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa158ddf640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00cef0 tx=0x7fa14c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c01a070 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00d860 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00dc50 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1540a95f0 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.349+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1541eebe0 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.350+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15404f330 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.351+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa14c01d070 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.351+0000 7fa150ff9640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.351+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fa14c08a8e0 con 0x7fa154159450 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.351+0000 7fa153fff640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.352+0000 7fa153fff640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 0x7fa14006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa144007fd0 tx=0x7fa144007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:20.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.357+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa14c059210 con 0x7fa154159450 2024-03-06T07:23:20.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:20.550+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa15406f270 con 0x7fa154159450 2024-03-06T07:23:21.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.353+0000 7fa150ff9640 1 -- 172.21.15.17:0/1266014812 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa14c05d2b0 con 0x7fa154159450 2024-03-06T07:23:21.355 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:23:21.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.355+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 msgr2=0x7fa14006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.355+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 0x7fa14006d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa144007fd0 tx=0x7fa144007e80 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.355+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 msgr2=0x7fa1540a8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.355+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00cef0 tx=0x7fa14c00a710 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 shutdown_connections 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa154159450 0x7fa1540a8360 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa154155db0 0x7fa1540ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 --2- 172.21.15.17:0/1266014812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa154152890 0x7fa1541ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 >> 172.21.15.17:0/1266014812 conn(0x7fa1540c7de0 msgr2=0x7fa154157160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 shutdown_connections 2024-03-06T07:23:21.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.356+0000 7fa15a842640 1 -- 172.21.15.17:0/1266014812 wait complete. 2024-03-06T07:23:21.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:23:21.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 Processor -- start 2024-03-06T07:23:21.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 -- start start 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe57406da40 0x7fe574155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe5740c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe574156050 con 0x7fe5740c6780 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57406de20 con 0x7fe57406da40 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe579135640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574072630 con 0x7fe5740c7380 2024-03-06T07:23:21.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.477+0000 7fe572d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe57406da40 0x7fe574155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe56bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5740c6780 0x7fe574158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46454/0 (socket says 172.21.15.17:46454) 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe56bfff640 1 -- 172.21.15.17:0/3045499920 learned_addr learned my addr 172.21.15.17:0/3045499920 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe57359e640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe5740c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe57259c640 1 -- 172.21.15.17:0/3045499920 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944638696 0 0) 0x7fe55c001110 con 0x7fe5740c6780 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe57259c640 1 -- 172.21.15.17:0/3045499920 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe560003520 con 0x7fe5740c6780 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe572d9d640 1 -- 172.21.15.17:0/3045499920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 msgr2=0x7fe5740c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe572d9d640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe5740c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe572d9d640 1 -- 172.21.15.17:0/3045499920 >> v1:172.21.15.17:6789/0 conn(0x7fe5740c6780 legacy=0x7fe574158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.478+0000 7fe572d9d640 1 -- 172.21.15.17:0/3045499920 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574072910 con 0x7fe57406da40 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe572d9d640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe574155b10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097a0 tx=0x7fe56402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3dd3b0f9359064 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe57259c640 1 -- 172.21.15.17:0/3045499920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5640315a0 con 0x7fe57406da40 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe57259c640 1 -- 172.21.15.17:0/3045499920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe564037070 con 0x7fe57406da40 2024-03-06T07:23:21.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe57259c640 1 -- 172.21.15.17:0/3045499920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564031960 con 0x7fe57406da40 2024-03-06T07:23:21.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 -- 172.21.15.17:0/3045499920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 msgr2=0x7fe574155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe574155b10 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe5640097a0 tx=0x7fe56402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 -- 172.21.15.17:0/3045499920 shutdown_connections 2024-03-06T07:23:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe5740c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 --2- 172.21.15.17:0/3045499920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe574155b10 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 -- 172.21.15.17:0/3045499920 >> 172.21.15.17:0/3045499920 conn(0x7fe5740aa630 msgr2=0x7fe5740aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 -- 172.21.15.17:0/3045499920 shutdown_connections 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.479+0000 7fe579135640 1 -- 172.21.15.17:0/3045499920 wait complete. 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 Processor -- start 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 -- start start 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5740c6780 0x7fe574150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe574157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5741594a0 con 0x7fe57406da40 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574158890 con 0x7fe5740c6780 2024-03-06T07:23:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe579135640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c37c0 con 0x7fe5740c7380 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe572d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe572d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59920/0 (socket says 172.21.15.17:59920) 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.481+0000 7fe572d9d640 1 -- 172.21.15.17:0/2115033831 learned_addr learned my addr 172.21.15.17:0/2115033831 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe56bfff640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5740c6780 0x7fe574150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe57359e640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe574157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 -- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 msgr2=0x7fe574157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe574157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 -- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5740c6780 msgr2=0x7fe574150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5740c6780 0x7fe574150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 -- 172.21.15.17:0/2115033831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57414f4e0 con 0x7fe57406da40 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe572d9d640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe564034040 tx=0x7fe5640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564032630 con 0x7fe57406da40 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640327d0 con 0x7fe57406da40 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564032bc0 con 0x7fe57406da40 2024-03-06T07:23:21.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574151950 con 0x7fe57406da40 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.482+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe574151f10 con 0x7fe57406da40 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.483+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57404f330 con 0x7fe57406da40 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.483+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe56404e430 con 0x7fe57406da40 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.484+0000 7fe56b7fe640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 0x7fe55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.484+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe56402f030 con 0x7fe57406da40 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.484+0000 7fe56bfff640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 0x7fe55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.485+0000 7fe56bfff640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 0x7fe55406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005880 tx=0x7fe55c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.488+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe57414f4e0 con 0x7fe57406da40 2024-03-06T07:23:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.735+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe574152780 con 0x7fe57406da40 2024-03-06T07:23:21.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.736+0000 7fe56b7fe640 1 -- 172.21.15.17:0/2115033831 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+4724 (secure 0 0 0) 0x7fe564040070 con 0x7fe57406da40 2024-03-06T07:23:21.737 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:21.737 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.738 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:21.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.741 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:23:21.742 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.745 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:23:21.746 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:23:21.747 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:21.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:23:21.749 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.737+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 msgr2=0x7fe55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.737+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 0x7fe55406d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005880 tx=0x7fe55c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.737+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 msgr2=0x7fe57414efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.737+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe564034040 tx=0x7fe5640046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 shutdown_connections 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe55406b4d0 0x7fe55406d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5740c7380 0x7fe574157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5740c6780 0x7fe574150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 --2- 172.21.15.17:0/2115033831 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe57406da40 0x7fe57414efa0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 >> 172.21.15.17:0/2115033831 conn(0x7fe5740aa630 msgr2=0x7fe5740c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 shutdown_connections 2024-03-06T07:23:21.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.738+0000 7fe579135640 1 -- 172.21.15.17:0/2115033831 wait complete. 2024-03-06T07:23:21.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:21.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.851+0000 7fcbf75c9640 1 Processor -- start 2024-03-06T07:23:21.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 -- start start 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbf00c88c0 0x7fcbf00c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00c73d0 con 0x7fcbf00c5880 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf00c8dd0 con 0x7fcbf00c88c0 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf75c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00be2e0 con 0x7fcbf014f360 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf4b64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbf00c88c0 0x7fcbf00c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf5b66640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcbf00c5880 0x7fcbf0158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46470/0 (socket says 172.21.15.17:46470) 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.852+0000 7fcbf5b66640 1 -- 172.21.15.17:0/4001671777 learned_addr learned my addr 172.21.15.17:0/4001671777 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:21.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbf4b64640 1 -- 172.21.15.17:0/4001671777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 msgr2=0x7fcbf0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/4001671777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbf4b64640 1 -- 172.21.15.17:0/4001671777 >> v1:172.21.15.17:6789/0 conn(0x7fcbf00c5880 legacy=0x7fcbf0158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbf4b64640 1 -- 172.21.15.17:0/4001671777 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00c7670 con 0x7fcbf00c88c0 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbeffff640 1 -- 172.21.15.17:0/4001671777 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723052389 0 0) 0x7fcbf00c73d0 con 0x7fcbf00c5880 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.853+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/4001671777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c88c0 0x7fcbf00c5340 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097a0 tx=0x7fcbe402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570e4b917aceac74 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbeffff640 1 -- 172.21.15.17:0/4001671777 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40315a0 con 0x7fcbf00c88c0 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbeffff640 1 -- 172.21.15.17:0/4001671777 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe4037070 con 0x7fcbf00c88c0 2024-03-06T07:23:21.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbeffff640 1 -- 172.21.15.17:0/4001671777 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4031960 con 0x7fcbf00c88c0 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 -- 172.21.15.17:0/4001671777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c88c0 msgr2=0x7fcbf00c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/4001671777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c88c0 0x7fcbf00c5340 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40097a0 tx=0x7fcbe402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 -- 172.21.15.17:0/4001671777 shutdown_connections 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/4001671777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c88c0 0x7fcbf00c5340 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/4001671777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 -- 172.21.15.17:0/4001671777 >> 172.21.15.17:0/4001671777 conn(0x7fcbf00ac570 msgr2=0x7fcbf00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 -- 172.21.15.17:0/4001671777 shutdown_connections 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.854+0000 7fcbf75c9640 1 -- 172.21.15.17:0/4001671777 wait complete. 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.855+0000 7fcbf75c9640 1 Processor -- start 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.855+0000 7fcbf75c9640 1 -- start start 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 0x7fcbf01eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf01eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0159270 con 0x7fcbf00c5880 2024-03-06T07:23:21.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00c7c30 con 0x7fcbf00c88c0 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf75c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01593f0 con 0x7fcbf014f360 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59924/0 (socket says 172.21.15.17:59924) 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 -- 172.21.15.17:0/96156428 learned_addr learned my addr 172.21.15.17:0/96156428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5b66640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf01eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 0x7fcbf01eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 -- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 msgr2=0x7fcbf01eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf01eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 -- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 msgr2=0x7fcbf01eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 0x7fcbf01eab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf5365640 1 -- 172.21.15.17:0/96156428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00af9d0 con 0x7fcbf00c5880 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.856+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 0x7fcbf01eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbf5365640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000ba50 tx=0x7fcbe000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0019070 con 0x7fcbf00c5880 2024-03-06T07:23:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe000cc00 con 0x7fcbf00c5880 2024-03-06T07:23:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf01ee380 con 0x7fcbf00c5880 2024-03-06T07:23:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0014440 con 0x7fcbf00c5880 2024-03-06T07:23:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.857+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf00ae920 con 0x7fcbf00c5880 2024-03-06T07:23:21.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.858+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf01ee170 con 0x7fcbf00c5880 2024-03-06T07:23:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.858+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbe001c070 con 0x7fcbf00c5880 2024-03-06T07:23:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.859+0000 7fcbedffb640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.859+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcbe0089d90 con 0x7fcbf00c5880 2024-03-06T07:23:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.859+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:21.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.859+0000 7fcbf4b64640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4036000 tx=0x7fcbe4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:21.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:21.863+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbe0058e60 con 0x7fcbf00c5880 2024-03-06T07:23:22.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.044+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbf004f330 con 0x7fcbf00c5880 2024-03-06T07:23:22.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.045+0000 7fcbedffb640 1 -- 172.21.15.17:0/96156428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13165 (secure 0 0 0) 0x7fcbe0058760 con 0x7fcbf00c5880 2024-03-06T07:23:22.047 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:22.047 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":92,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:21.352143+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 msgr2=0x7fcbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4036000 tx=0x7fcbe4008030 comp rx=0 tx=0).stop 2024-03-06T07:23:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 msgr2=0x7fcbf00abca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000ba50 tx=0x7fcbe000bf10 comp rx=0 tx=0).stop 2024-03-06T07:23:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 shutdown_connections 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbf014f360 0x7fcbf01eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbf00c88c0 0x7fcbf01eab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 --2- 172.21.15.17:0/96156428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbf00c5880 0x7fcbf00abca0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 >> 172.21.15.17:0/96156428 conn(0x7fcbf00ac570 msgr2=0x7fcbf00c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 shutdown_connections 2024-03-06T07:23:22.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.047+0000 7fcbf75c9640 1 -- 172.21.15.17:0/96156428 wait complete. 2024-03-06T07:23:22.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:22.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:22.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.163+0000 7f4e28f65640 1 Processor -- start 2024-03-06T07:23:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.163+0000 7f4e28f65640 1 -- start start 2024-03-06T07:23:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e241529f0 0x7f4e240c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e240c5370 0x7f4e240c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e24159710 con 0x7f4e241529f0 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e240c68b0 con 0x7f4e240c5370 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e28f65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24152dd0 con 0x7f4e24155530 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e241529f0 0x7f4e240c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e241529f0 0x7f4e240c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59936/0 (socket says 172.21.15.17:59936) 2024-03-06T07:23:22.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 -- 172.21.15.17:0/4142632401 learned_addr learned my addr 172.21.15.17:0/4142632401 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e21d9b640 1 --2- 172.21.15.17:0/4142632401 >> v2:172.21.15.163:3300/0 conn(0x7f4e240c5370 0x7f4e240c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e22d9d640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 -- 172.21.15.17:0/4142632401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 msgr2=0x7f4e241559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 -- 172.21.15.17:0/4142632401 >> v2:172.21.15.163:3300/0 conn(0x7f4e240c5370 msgr2=0x7f4e240c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 --2- 172.21.15.17:0/4142632401 >> v2:172.21.15.163:3300/0 conn(0x7f4e240c5370 0x7f4e240c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e2259c640 1 -- 172.21.15.17:0/4142632401 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e24158b00 con 0x7f4e241529f0 2024-03-06T07:23:22.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.164+0000 7f4e22d9d640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.165+0000 7f4e2259c640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e241529f0 0x7f4e240c88f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4e2408c9b0 tx=0x7f4e1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae6f48015e3f016 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.165+0000 7f4e2159a640 1 -- 172.21.15.17:0/4142632401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c02e070 con 0x7f4e241529f0 2024-03-06T07:23:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.165+0000 7f4e2159a640 1 -- 172.21.15.17:0/4142632401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c037070 con 0x7f4e241529f0 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.165+0000 7f4e2159a640 1 -- 172.21.15.17:0/4142632401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c031a00 con 0x7f4e241529f0 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 -- 172.21.15.17:0/4142632401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e241529f0 msgr2=0x7f4e240c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e241529f0 0x7f4e240c88f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4e2408c9b0 tx=0x7f4e1c02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 -- 172.21.15.17:0/4142632401 shutdown_connections 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 --2- 172.21.15.17:0/4142632401 >> v2:172.21.15.163:3300/0 conn(0x7f4e240c5370 0x7f4e240c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 --2- 172.21.15.17:0/4142632401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e241529f0 0x7f4e240c88f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 -- 172.21.15.17:0/4142632401 >> 172.21.15.17:0/4142632401 conn(0x7f4e240ac580 msgr2=0x7f4e240ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 -- 172.21.15.17:0/4142632401 shutdown_connections 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.166+0000 7f4e28f65640 1 -- 172.21.15.17:0/4142632401 wait complete. 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.167+0000 7f4e28f65640 1 Processor -- start 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.167+0000 7f4e28f65640 1 -- start start 2024-03-06T07:23:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 0x7f4e240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e241593a0 con 0x7f4e240c5370 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e241590c0 con 0x7f4e241529f0 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e28f65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24158740 con 0x7f4e24155530 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59942/0 (socket says 172.21.15.17:59942) 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 -- 172.21.15.17:0/2502980117 learned_addr learned my addr 172.21.15.17:0/2502980117 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e21d9b640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 0x7f4e240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e22d9d640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 -- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 msgr2=0x7f4e241ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241ee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 -- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 msgr2=0x7f4e240b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 0x7f4e240b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 -- 172.21.15.17:0/2502980117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240ae1f0 con 0x7f4e240c5370 2024-03-06T07:23:22.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e21d9b640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 0x7f4e240b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e22d9d640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e2259c640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c02b8c0 tx=0x7f4e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.168+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c032880 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.169+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c032a60 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.169+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e24153e80 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.169+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c032e50 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.169+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e241eedf0 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.170+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2404f330 con 0x7f4e240c5370 2024-03-06T07:23:22.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.170+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e1c0453f0 con 0x7f4e240c5370 2024-03-06T07:23:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.170+0000 7f4e1affd640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 0x7f4e0406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.170+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4e1c0abdb0 con 0x7f4e240c5370 2024-03-06T07:23:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.171+0000 7f4e21d9b640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 0x7f4e0406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.171+0000 7f4e21d9b640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 0x7f4e0406db30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e241552d0 tx=0x7f4e0c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.175+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e1c040070 con 0x7f4e240c5370 2024-03-06T07:23:22.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.356+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e2414fe50 con 0x7f4e240c5370 2024-03-06T07:23:22.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.356+0000 7f4e1affd640 1 -- 172.21.15.17:0/2502980117 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e1c07ae80 con 0x7f4e240c5370 2024-03-06T07:23:22.358 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 msgr2=0x7f4e0406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 0x7f4e0406db30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e241552d0 tx=0x7f4e0c008040 comp rx=0 tx=0).stop 2024-03-06T07:23:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 msgr2=0x7f4e24153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c02b8c0 tx=0x7f4e1c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 shutdown_connections 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e0406b670 0x7f4e0406db30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e24155530 0x7f4e241ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e241529f0 0x7f4e240b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 --2- 172.21.15.17:0/2502980117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e240c5370 0x7f4e24153940 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 >> 172.21.15.17:0/2502980117 conn(0x7f4e240ac580 msgr2=0x7f4e24151810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 shutdown_connections 2024-03-06T07:23:22.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.358+0000 7f4e28f65640 1 -- 172.21.15.17:0/2502980117 wait complete. 2024-03-06T07:23:22.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:23:22.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.472+0000 7f356c1d6640 1 Processor -- start 2024-03-06T07:23:22.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.472+0000 7f356c1d6640 1 -- start start 2024-03-06T07:23:22.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f356c1d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f35640c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f356c1d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3564157340 con 0x7f3564072720 2024-03-06T07:23:22.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f356c1d6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3564156d80 con 0x7f356406f7b0 2024-03-06T07:23:22.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f356c1d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3564156b40 con 0x7f35641508c0 2024-03-06T07:23:22.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f3569f72640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3564072720 0x7f3564156430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46484/0 (socket says 172.21.15.17:46484) 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f3569f72640 1 -- 172.21.15.17:0/2346864362 learned_addr learned my addr 172.21.15.17:0/2346864362 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f356a773640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f35640c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 737585822 0 0) 0x7f3564157340 con 0x7f3564072720 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.473+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f354c003520 con 0x7f3564072720 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179372469 0 0) 0x7f3564156d80 con 0x7f356406f7b0 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3564157340 con 0x7f356406f7b0 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977055019 0 0) 0x7f354c003520 con 0x7f3564072720 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 msgr2=0x7f35640c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f35640c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 >> v1:172.21.15.163:6789/0 conn(0x7f356406f7b0 legacy=0x7f35641501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3564157060 con 0x7f3564072720 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f355c001670 con 0x7f3564072720 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 >> v1:172.21.15.17:6789/0 conn(0x7f3564072720 legacy=0x7f3564156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c003b70 0x7f354c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f354c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f354c004d40 0x7f354c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3564156b40 con 0x7f354c004d40 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c00bb50 con 0x7f354c003b70 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354c00bd20 con 0x7f35641508c0 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f354c004d40 0x7f354c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3569f72640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c003b70 0x7f354c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3569771640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f354c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 -- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 msgr2=0x7f354c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f354c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 -- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c003b70 msgr2=0x7f354c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c003b70 0x7f354c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f356a773640 1 -- 172.21.15.17:0/2346864362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c003520 con 0x7f354c004d40 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3569f72640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f354c003b70 0x7f354c003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.474+0000 7f3569771640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35641508c0 0x7f354c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356a773640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f354c004d40 0x7f354c0051c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3554009f90 tx=0x7f354c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cf6244b53d2a14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554039070 con 0x7f354c004d40 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355402cb90 con 0x7f354c004d40 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f3568f70640 1 -- 172.21.15.17:0/2346864362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554009c10 con 0x7f354c004d40 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356c1d6640 1 -- 172.21.15.17:0/2346864362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f354c004d40 msgr2=0x7f354c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356c1d6640 1 --2- 172.21.15.17:0/2346864362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f354c004d40 0x7f354c0051c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3554009f90 tx=0x7f354c005830 comp rx=0 tx=0).stop 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356a773640 1 -- 172.21.15.17:0/2346864362 reap_dead start 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356c1d6640 1 -- 172.21.15.17:0/2346864362 shutdown_connections 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.475+0000 7f356c1d6640 1 -- 172.21.15.17:0/2346864362 >> 172.21.15.17:0/2346864362 conn(0x7f35640aa630 msgr2=0x7f35640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.476+0000 7f356c1d6640 1 -- 172.21.15.17:0/2346864362 shutdown_connections 2024-03-06T07:23:22.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.476+0000 7f356c1d6640 1 -- 172.21.15.17:0/2346864362 wait complete. 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 Processor -- start 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 -- start start 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f354c004d40 0x7f35640ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3564154750 0x7f35641455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3564072b00 con 0x7f35640adcd0 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35640c3740 con 0x7f3564154750 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f356c1d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3564072c80 con 0x7f354c004d40 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f3569771640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f3569771640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59954/0 (socket says 172.21.15.17:59954) 2024-03-06T07:23:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.477+0000 7f3569771640 1 -- 172.21.15.17:0/1413642475 learned_addr learned my addr 172.21.15.17:0/1413642475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f356a773640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3564154750 0x7f35641455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569f72640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f354c004d40 0x7f35640ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 -- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f354c004d40 msgr2=0x7f35640ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f354c004d40 0x7f35640ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 -- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3564154750 msgr2=0x7f35641455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3564154750 0x7f35641455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 -- 172.21.15.17:0/1413642475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35641ecd90 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f3569771640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f355800e890 tx=0x7f355800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800ee80 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580052e0 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35640c2d50 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007020 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.478+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35640ae150 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.479+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35640a9d50 con 0x7f35640adcd0 2024-03-06T07:23:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.479+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f355801a020 con 0x7f35640adcd0 2024-03-06T07:23:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.480+0000 7f35627fc640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 0x7f355006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.480+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3558089760 con 0x7f35640adcd0 2024-03-06T07:23:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.480+0000 7f3569f72640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 0x7f355006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.481+0000 7f3569f72640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 0x7f355006d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f355c001200 tx=0x7f355c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.484+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35640a9d50 con 0x7f35640adcd0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.665+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f35641ec8a0 con 0x7f35640adcd0 2024-03-06T07:23:22.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.666+0000 7f35627fc640 1 -- 172.21.15.17:0/1413642475 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3558058830 con 0x7f35640adcd0 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:22.668 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 msgr2=0x7f355006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 0x7f355006d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f355c001200 tx=0x7f355c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 msgr2=0x7f3564154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f355800e890 tx=0x7f355800ed50 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 shutdown_connections 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355006b4d0 0x7f355006d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3564154750 0x7f35641455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35640adcd0 0x7f3564154210 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 --2- 172.21.15.17:0/1413642475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f354c004d40 0x7f35640ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 >> 172.21.15.17:0/1413642475 conn(0x7f35640aa630 msgr2=0x7f356414e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 shutdown_connections 2024-03-06T07:23:22.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.668+0000 7f356c1d6640 1 -- 172.21.15.17:0/1413642475 wait complete. 2024-03-06T07:23:22.680 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:23:22.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:23:22.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.783+0000 7f7f8e8c2640 1 Processor -- start 2024-03-06T07:23:22.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.783+0000 7f7f8e8c2640 1 -- start start 2024-03-06T07:23:22.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.784+0000 7f7f8e8c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f8814bb20 0x7f7f880c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.784+0000 7f7f8e8c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f880c2330 0x7f7f880c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.784+0000 7f7f8e8c2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f88159260 con 0x7f7f881550c0 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.784+0000 7f7f8e8c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f880c3070 con 0x7f7f880c2330 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.784+0000 7f7f8e8c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880a80e0 con 0x7f7f8814bb20 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f880c2330 0x7f7f880c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f87fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f8814bb20 0x7f7f880c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f880c2330 0x7f7f880c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59490/0 (socket says 172.21.15.17:59490) 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 -- 172.21.15.17:0/472940357 learned_addr learned my addr 172.21.15.17:0/472940357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f8ce5f640 1 --1- 172.21.15.17:0/472940357 >> v1:172.21.15.163:6789/0 conn(0x7f7f881550c0 0x7f7f88155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:60860/0 (socket says 172.21.15.17:60860) 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 -- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f8814bb20 msgr2=0x7f7f880c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 --2- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f8814bb20 0x7f7f880c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 -- 172.21.15.17:0/472940357 >> v1:172.21.15.163:6789/0 conn(0x7f7f881550c0 legacy=0x7f7f88155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.785+0000 7f7f877fe640 1 -- 172.21.15.17:0/472940357 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880a7d70 con 0x7f7f880c2330 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f877fe640 1 --2- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f880c2330 0x7f7f880c5850 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009870 tx=0x7f7f7c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2668bc6ab579c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f86ffd640 1 -- 172.21.15.17:0/472940357 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c035020 con 0x7f7f880c2330 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f86ffd640 1 -- 172.21.15.17:0/472940357 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c031640 con 0x7f7f880c2330 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f86ffd640 1 -- 172.21.15.17:0/472940357 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c02d040 con 0x7f7f880c2330 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f880c2330 msgr2=0x7f7f880c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f880c2330 0x7f7f880c5850 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009870 tx=0x7f7f7c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/472940357 shutdown_connections 2024-03-06T07:23:22.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f880c2330 0x7f7f880c5850 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/472940357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f8814bb20 0x7f7f880c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.786+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/472940357 >> 172.21.15.17:0/472940357 conn(0x7f7f880c88f0 msgr2=0x7f7f880c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.787+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/472940357 shutdown_connections 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.787+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/472940357 wait complete. 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 Processor -- start 2024-03-06T07:23:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 -- start start 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f880c2330 0x7f7f881573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 0x7f7f8814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880c67f0 con 0x7f7f8814bb20 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8814bf00 con 0x7f7f881550c0 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.788+0000 7f7f8e8c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880c6970 con 0x7f7f880c2330 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59958/0 (socket says 172.21.15.17:59958) 2024-03-06T07:23:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 -- 172.21.15.17:0/2259136537 learned_addr learned my addr 172.21.15.17:0/2259136537 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f87fff640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f880c2330 0x7f7f881573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f8ce5f640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 0x7f7f8814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 -- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f880c2330 msgr2=0x7f7f881573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f880c2330 0x7f7f881573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 -- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 msgr2=0x7f7f8814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 0x7f7f8814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 -- 172.21.15.17:0/2259136537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f881eaca0 con 0x7f7f8814bb20 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f8ce5f640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 0x7f7f8814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f877fe640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02b930 tx=0x7f7f7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c040070 con 0x7f7f8814bb20 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c03e070 con 0x7f7f8814bb20 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c035040 con 0x7f7f8814bb20 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f881eaf80 con 0x7f7f8814bb20 2024-03-06T07:23:22.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.789+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f881eb540 con 0x7f7f8814bb20 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.790+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f881eb780 con 0x7f7f8814bb20 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.791+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f7c038020 con 0x7f7f8814bb20 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.791+0000 7f7f84ff9640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.791+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7f7c0ab8b0 con 0x7f7f8814bb20 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.791+0000 7f7f87fff640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:22.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.792+0000 7f7f87fff640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 0x7f7f7406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009ad0 tx=0x7f7f78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:22.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.795+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7f881eb780 con 0x7f7f8814bb20 2024-03-06T07:23:22.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:22.980+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7f8804f330 con 0x7f7f8814bb20 2024-03-06T07:23:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.362+0000 7f7f84ff9640 1 -- 172.21.15.17:0/2259136537 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f7c07a980 con 0x7f7f8814bb20 2024-03-06T07:23:23.364 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 msgr2=0x7f7f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 0x7f7f7406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009ad0 tx=0x7f7f78008040 comp rx=0 tx=0).stop 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 msgr2=0x7f7f88152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c02b930 tx=0x7f7f7c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 shutdown_connections 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f7406b4d0 0x7f7f7406d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f881550c0 0x7f7f8814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f8814bb20 0x7f7f88152360 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 --2- 172.21.15.17:0/2259136537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f880c2330 0x7f7f881573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 >> 172.21.15.17:0/2259136537 conn(0x7f7f880c88f0 msgr2=0x7f7f88154670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 shutdown_connections 2024-03-06T07:23:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.365+0000 7f7f8e8c2640 1 -- 172.21.15.17:0/2259136537 wait complete. 2024-03-06T07:23:23.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:23:23.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.489+0000 7f4ee144e640 1 Processor -- start 2024-03-06T07:23:23.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.489+0000 7f4ee144e640 1 -- start start 2024-03-06T07:23:23.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ee144e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4edc155c30 0x7f4edc06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ee144e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 0x7f4edc0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ee144e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0b0450 con 0x7f4edc0b0070 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ee144e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc06cfb0 con 0x7f4edc155c30 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ee144e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06c020 con 0x7f4edc0c5380 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4edaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4edc0b0070 0x7f4edc0c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46492/0 (socket says 172.21.15.17:46492) 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4edaffd640 1 -- 172.21.15.17:0/2711515311 learned_addr learned my addr 172.21.15.17:0/2711515311 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/2711515311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 0x7f4edc0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4eda7fc640 1 --2- 172.21.15.17:0/2711515311 >> v2:172.21.15.163:3300/0 conn(0x7f4edc155c30 0x7f4edc06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053473228 0 0) 0x7f4ed4001110 con 0x7f4edc0b0070 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.490+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec8003520 con 0x7f4edc0b0070 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.491+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325043763 0 0) 0x7f4ec8003520 con 0x7f4edc0b0070 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.491+0000 7f4edb7fe640 1 -- 172.21.15.17:0/2711515311 >> v2:172.21.15.163:3300/0 conn(0x7f4edc155c30 msgr2=0x7f4edc06ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.491+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/2711515311 >> v2:172.21.15.163:3300/0 conn(0x7f4edc155c30 0x7f4edc06ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.491+0000 7f4edb7fe640 1 -- 172.21.15.17:0/2711515311 >> v1:172.21.15.17:6789/0 conn(0x7f4edc0b0070 legacy=0x7f4edc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.491+0000 7f4edb7fe640 1 -- 172.21.15.17:0/2711515311 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc1560e0 con 0x7f4edc0c5380 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/2711515311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 0x7f4edc0c5790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc02bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5909394d758436 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc039070 con 0x7f4edc0c5380 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc02c810 con 0x7f4edc0c5380 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ed9ffb640 1 -- 172.21.15.17:0/2711515311 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4001840 con 0x7f4edc0c5380 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 -- 172.21.15.17:0/2711515311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 msgr2=0x7f4edc0c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 --2- 172.21.15.17:0/2711515311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 0x7f4edc0c5790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc02bce0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 -- 172.21.15.17:0/2711515311 shutdown_connections 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 --2- 172.21.15.17:0/2711515311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0c5380 0x7f4edc0c5790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 --2- 172.21.15.17:0/2711515311 >> v2:172.21.15.163:3300/0 conn(0x7f4edc155c30 0x7f4edc06ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 -- 172.21.15.17:0/2711515311 >> 172.21.15.17:0/2711515311 conn(0x7f4edc0ac590 msgr2=0x7f4edc0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:23.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 -- 172.21.15.17:0/2711515311 shutdown_connections 2024-03-06T07:23:23.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.492+0000 7f4ee144e640 1 -- 172.21.15.17:0/2711515311 wait complete. 2024-03-06T07:23:23.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.493+0000 7f4ee144e640 1 Processor -- start 2024-03-06T07:23:23.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 -- start start 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 0x7f4edc158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 0x7f4edc1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06c270 con 0x7f4edc155c30 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0c7ad0 con 0x7f4edc0c5380 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4ee144e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06c550 con 0x7f4edc0b0070 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59974/0 (socket says 172.21.15.17:59974) 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4edb7fe640 1 -- 172.21.15.17:0/1441581701 learned_addr learned my addr 172.21.15.17:0/1441581701 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:23.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4edaffd640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 0x7f4edc158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.494+0000 7f4eda7fc640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 0x7f4edc1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 -- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 msgr2=0x7f4edc158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 0x7f4edc158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 -- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 msgr2=0x7f4edc1513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 0x7f4edc1513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 -- 172.21.15.17:0/1441581701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0c4a90 con 0x7f4edc155c30 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edaffd640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 0x7f4edc158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4eda7fc640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 0x7f4edc1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4edb7fe640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc039050 con 0x7f4edc155c30 2024-03-06T07:23:23.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc03a040 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc0c4d50 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc004490 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.495+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1eb670 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.496+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc0af6a0 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.497+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ecc047020 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.497+0000 7f4ed37fe640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 0x7f4ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.497+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f4ecc0ac490 con 0x7f4edc155c30 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.497+0000 7f4edaffd640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 0x7f4ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:23.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.498+0000 7f4edaffd640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 0x7f4ebc06d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001eb0 tx=0x7f4ed4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:23.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.501+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4edc0af6a0 con 0x7f4edc155c30 2024-03-06T07:23:23.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:23.681+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4edc06ccf0 con 0x7f4edc155c30 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.368+0000 7f4ed37fe640 1 -- 172.21.15.17:0/1441581701 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4ecc04e020 con 0x7f4edc155c30 2024-03-06T07:23:24.370 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 msgr2=0x7f4ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 0x7f4ebc06d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001eb0 tx=0x7f4ed4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 msgr2=0x7f4edc151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009d90 tx=0x7f4ecc004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 shutdown_connections 2024-03-06T07:23:24.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4ebc06b4d0 0x7f4ebc06d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4edc155c30 0x7f4edc151930 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4edc0c5380 0x7f4edc1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 --2- 172.21.15.17:0/1441581701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4edc0b0070 0x7f4edc158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.372+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 >> 172.21.15.17:0/1441581701 conn(0x7f4edc0ac590 msgr2=0x7f4edc0c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:24.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.373+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 shutdown_connections 2024-03-06T07:23:24.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.374+0000 7f4ee144e640 1 -- 172.21.15.17:0/1441581701 wait complete. 2024-03-06T07:23:24.403 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:23:24.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.511+0000 7fdac968f640 1 Processor -- start 2024-03-06T07:23:24.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.511+0000 7fdac968f640 1 -- start start 2024-03-06T07:23:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac968f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdac40c8a00 0x7fdac4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac968f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4159460 0x7fdac40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac968f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac40c7760 con 0x7fdac4152920 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac968f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac41587b0 con 0x7fdac40c8a00 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac968f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40c7a40 con 0x7fdac4159460 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdac40c8a00 0x7fdac4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdac4152920 0x7fdac4155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46496/0 (socket says 172.21.15.17:46496) 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac2ffd640 1 -- 172.21.15.17:0/1989338499 learned_addr learned my addr 172.21.15.17:0/1989338499 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac37fe640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4159460 0x7fdac40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac1ffb640 1 -- 172.21.15.17:0/1989338499 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677491030 0 0) 0x7fdac40c7760 con 0x7fdac4152920 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac1ffb640 1 -- 172.21.15.17:0/1989338499 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaac003520 con 0x7fdac4152920 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac27fc640 1 -- 172.21.15.17:0/1989338499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4159460 msgr2=0x7fdac40c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac27fc640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4159460 0x7fdac40c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac27fc640 1 -- 172.21.15.17:0/1989338499 >> v1:172.21.15.17:6789/0 conn(0x7fdac4152920 legacy=0x7fdac4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.512+0000 7fdac27fc640 1 -- 172.21.15.17:0/1989338499 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40c7d20 con 0x7fdac40c8a00 2024-03-06T07:23:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.513+0000 7fdac27fc640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40c8a00 0x7fdac4158f20 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdab80097a0 tx=0x7fdab802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bb68c22cc83b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.513+0000 7fdac1ffb640 1 -- 172.21.15.17:0/1989338499 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80315a0 con 0x7fdac40c8a00 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.513+0000 7fdac1ffb640 1 -- 172.21.15.17:0/1989338499 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8037070 con 0x7fdac40c8a00 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.513+0000 7fdac1ffb640 1 -- 172.21.15.17:0/1989338499 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8031960 con 0x7fdac40c8a00 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 -- 172.21.15.17:0/1989338499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40c8a00 msgr2=0x7fdac4158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40c8a00 0x7fdac4158f20 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdab80097a0 tx=0x7fdab802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 -- 172.21.15.17:0/1989338499 shutdown_connections 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4159460 0x7fdac40c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 --2- 172.21.15.17:0/1989338499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac40c8a00 0x7fdac4158f20 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 -- 172.21.15.17:0/1989338499 >> 172.21.15.17:0/1989338499 conn(0x7fdac40ac560 msgr2=0x7fdac40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 -- 172.21.15.17:0/1989338499 shutdown_connections 2024-03-06T07:23:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.514+0000 7fdac968f640 1 -- 172.21.15.17:0/1989338499 wait complete. 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.515+0000 7fdac968f640 1 Processor -- start 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.515+0000 7fdac968f640 1 -- start start 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 0x7fdac40ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4150c30 0x7fdac4147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40c7390 con 0x7fdac4152920 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac406d030 con 0x7fdac40c8a00 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac968f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40c7510 con 0x7fdac4150c30 2024-03-06T07:23:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59978/0 (socket says 172.21.15.17:59978) 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 -- 172.21.15.17:0/1699947308 learned_addr learned my addr 172.21.15.17:0/1699947308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac2ffd640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 0x7fdac40ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac37fe640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4150c30 0x7fdac4147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 -- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4150c30 msgr2=0x7fdac4147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4150c30 0x7fdac4147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 -- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 msgr2=0x7fdac40ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 0x7fdac40ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 -- 172.21.15.17:0/1699947308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac41ee720 con 0x7fdac4152920 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac2ffd640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 0x7fdac40ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdac27fc640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098c0 tx=0x7fdab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8032630 con 0x7fdac4152920 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.516+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80327d0 con 0x7fdac4152920 2024-03-06T07:23:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.517+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac40c3f90 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.517+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8032bc0 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.517+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac40abde0 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.517+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac404f330 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.518+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdab804e430 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.518+0000 7fdaab7fe640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 0x7fdab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.519+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fdab802f030 con 0x7fdac4152920 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.519+0000 7fdac2ffd640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 0x7fdab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:24.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.519+0000 7fdac2ffd640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 0x7fdab006d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdabc005880 tx=0x7fdabc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.523+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdab8040070 con 0x7fdac4152920 2024-03-06T07:23:24.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:24.742+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdac41ee8f0 con 0x7fdac4152920 2024-03-06T07:23:25.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.370+0000 7fdaab7fe640 1 -- 172.21.15.17:0/1699947308 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v76) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdab807ae00 con 0x7fdac4152920 2024-03-06T07:23:25.372 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 15 and data pool 16 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 msgr2=0x7fdab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 0x7fdab006d9c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdabc005880 tx=0x7fdabc009040 comp rx=0 tx=0).stop 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 msgr2=0x7fdac41eec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098c0 tx=0x7fdab80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 shutdown_connections 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdab006b500 0x7fdab006d9c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdac4150c30 0x7fdac4147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdac4152920 0x7fdac41eec20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 --2- 172.21.15.17:0/1699947308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdac40c8a00 0x7fdac40ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.374+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 >> 172.21.15.17:0/1699947308 conn(0x7fdac40ac560 msgr2=0x7fdac40c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.376+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 shutdown_connections 2024-03-06T07:23:25.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.376+0000 7fdac968f640 1 -- 172.21.15.17:0/1699947308 wait complete. 2024-03-06T07:23:25.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.516+0000 7f0c06df7640 1 Processor -- start 2024-03-06T07:23:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.516+0000 7f0c06df7640 1 -- start start 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c06df7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0c00155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c06df7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c000c7380 0x7f0c000c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c06df7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c00156050 con 0x7f0c000c6780 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c06df7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0006de20 con 0x7f0c000c7380 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c06df7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00072630 con 0x7f0c0006da40 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0bfffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0c000c6780 0x7f0c00158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46500/0 (socket says 172.21.15.17:46500) 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0bfffff640 1 -- 172.21.15.17:0/3437668102 learned_addr learned my addr 172.21.15.17:0/3437668102 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c05394640 1 --2- 172.21.15.17:0/3437668102 >> v2:172.21.15.163:3300/0 conn(0x7f0c000c7380 0x7f0c000c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0c00155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674244969 0 0) 0x7f0bf4001110 con 0x7f0c000c6780 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.517+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be8003520 con 0x7f0c000c6780 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209714044 0 0) 0x7f0be8003520 con 0x7f0c000c6780 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 msgr2=0x7f0c00155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0c00155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 >> v2:172.21.15.163:3300/0 conn(0x7f0c000c7380 msgr2=0x7f0c000c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 --2- 172.21.15.17:0/3437668102 >> v2:172.21.15.163:3300/0 conn(0x7f0c000c7380 0x7f0c000c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00072910 con 0x7f0c000c6780 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0bf4001840 con 0x7f0c000c6780 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0c05394640 1 --2- 172.21.15.17:0/3437668102 >> v2:172.21.15.163:3300/0 conn(0x7f0c000c7380 0x7f0c000c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 >> v1:172.21.15.17:6789/0 conn(0x7f0c000c6780 legacy=0x7f0c00158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be80039b0 0x7f0be8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be8004800 0x7f0be8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0006de20 con 0x7f0be8004800 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00072630 con 0x7f0be80039b0 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be800b9c0 con 0x7f0c0006da40 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be8004800 0x7f0be8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be80039b0 0x7f0be8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0c05394640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 -- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 msgr2=0x7f0be80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0006da40 0x7f0be80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 -- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be80039b0 msgr2=0x7f0be8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be80039b0 0x7f0be8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.518+0000 7f0bfffff640 1 -- 172.21.15.17:0/3437668102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8003520 con 0x7f0be8004800 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be8004800 0x7f0be8004c80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c3e0 tx=0x7f0be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9963dd0c71968b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400c0e0 con 0x7f0be8004800 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf4030070 con 0x7f0be8004800 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0bff7fe640 1 -- 172.21.15.17:0/3437668102 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40058d0 con 0x7f0be8004800 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0c06df7640 1 -- 172.21.15.17:0/3437668102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be8004800 msgr2=0x7f0be8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3437668102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0be8004800 0x7f0be8004c80 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c3e0 tx=0x7f0be8005830 comp rx=0 tx=0).stop 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.519+0000 7f0c05394640 1 -- 172.21.15.17:0/3437668102 reap_dead start 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.520+0000 7f0c06df7640 1 -- 172.21.15.17:0/3437668102 shutdown_connections 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.520+0000 7f0c06df7640 1 -- 172.21.15.17:0/3437668102 >> 172.21.15.17:0/3437668102 conn(0x7f0c000aa630 msgr2=0x7f0c000aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.520+0000 7f0c06df7640 1 -- 172.21.15.17:0/3437668102 shutdown_connections 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.520+0000 7f0c06df7640 1 -- 172.21.15.17:0/3437668102 wait complete. 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 Processor -- start 2024-03-06T07:23:25.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 -- start start 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 0x7f0c000ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c00157280 0x7f0c00157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c001594a0 con 0x7f0c0014f750 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c00158890 con 0x7f0be8004800 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.521+0000 7f0c06df7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c000c37c0 con 0x7f0c00157280 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59996/0 (socket says 172.21.15.17:59996) 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 -- 172.21.15.17:0/3316750664 learned_addr learned my addr 172.21.15.17:0/3316750664 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 0x7f0c000ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0c05394640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c00157280 0x7f0c00157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 -- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c00157280 msgr2=0x7f0c00157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c00157280 0x7f0c00157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 -- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 msgr2=0x7f0c000ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 0x7f0c000ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 -- 172.21.15.17:0/3316750664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c000ab750 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 0x7f0c000ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfffff640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402c720 tx=0x7f0bf4001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004bf0 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf4004d90 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0014f190 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40072b0 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.522+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c000abcd0 con 0x7f0c0014f750 2024-03-06T07:23:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.523+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0004f330 con 0x7f0c0014f750 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.524+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bf4007450 con 0x7f0c0014f750 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.524+0000 7f0bfd7fa640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.524+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bf40abec0 con 0x7f0c0014f750 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.524+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.525+0000 7f0c04b93640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 0x7f0bec06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8013300 tx=0x7f0bf800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.529+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0c00156050 con 0x7f0c0014f750 2024-03-06T07:23:25.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.709+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c000ae490 con 0x7f0c0014f750 2024-03-06T07:23:25.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.710+0000 7f0bfd7fa640 1 -- 172.21.15.17:0/3316750664 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f0bf40343d0 con 0x7f0c0014f750 2024-03-06T07:23:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:25.712 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":95,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:24.744085+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 msgr2=0x7f0bec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 0x7f0bec06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8013300 tx=0x7f0bf800b040 comp rx=0 tx=0).stop 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 msgr2=0x7f0c0014ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0bf402c720 tx=0x7f0bf4001c10 comp rx=0 tx=0).stop 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 shutdown_connections 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0bec06b4d0 0x7f0bec06d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c00157280 0x7f0c00157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0014f750 0x7f0c0014ec50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 --2- 172.21.15.17:0/3316750664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0be8004800 0x7f0c000ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 >> 172.21.15.17:0/3316750664 conn(0x7f0c000aa630 msgr2=0x7f0c00153490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 shutdown_connections 2024-03-06T07:23:25.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.712+0000 7f0c06df7640 1 -- 172.21.15.17:0/3316750664 wait complete. 2024-03-06T07:23:25.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:25.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.824+0000 7f802ac66640 1 Processor -- start 2024-03-06T07:23:25.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 -- start start 2024-03-06T07:23:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 0x7f802406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80240ac260 0x7f80241559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8024152d40 con 0x7f80241592d0 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8024066e30 con 0x7f80240ac260 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.825+0000 7f802ac66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80240c1dc0 con 0x7f8024152960 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80240ac260 0x7f80241559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80240ac260 0x7f80241559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60012/0 (socket says 172.21.15.17:60012) 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 -- 172.21.15.17:0/2580578902 learned_addr learned my addr 172.21.15.17:0/2580578902 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8028a02640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 0x7f802406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 -- 172.21.15.17:0/2580578902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 msgr2=0x7f802406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 0x7f802406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 -- 172.21.15.17:0/2580578902 >> v1:172.21.15.163:6789/0 conn(0x7f80241592d0 legacy=0x7f80241552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8029203640 1 -- 172.21.15.17:0/2580578902 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80240c1b50 con 0x7f80240ac260 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.826+0000 7f8028a02640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 0x7f802406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f8029203640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80240ac260 0x7f80241559e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8014009870 tx=0x7f801402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba25a5b392af8c84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f80237fe640 1 -- 172.21.15.17:0/2580578902 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014034070 con 0x7f80240ac260 2024-03-06T07:23:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f80237fe640 1 -- 172.21.15.17:0/2580578902 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014030840 con 0x7f80240ac260 2024-03-06T07:23:25.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f80237fe640 1 -- 172.21.15.17:0/2580578902 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014030c10 con 0x7f80240ac260 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 -- 172.21.15.17:0/2580578902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80240ac260 msgr2=0x7f80241559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80240ac260 0x7f80241559e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8014009870 tx=0x7f801402b980 comp rx=0 tx=0).stop 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 -- 172.21.15.17:0/2580578902 shutdown_connections 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80240ac260 0x7f80241559e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 --2- 172.21.15.17:0/2580578902 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8024152960 0x7f802406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 -- 172.21.15.17:0/2580578902 >> 172.21.15.17:0/2580578902 conn(0x7f80240c70b0 msgr2=0x7f80240c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 -- 172.21.15.17:0/2580578902 shutdown_connections 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.827+0000 7f802ac66640 1 -- 172.21.15.17:0/2580578902 wait complete. 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 Processor -- start 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 -- start start 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240ac260 0x7f80241eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024152960 0x7f8024151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80240c20a0 con 0x7f802414dce0 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024159760 con 0x7f8024152960 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.829+0000 7f802ac66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80240c2220 con 0x7f80240ac260 2024-03-06T07:23:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60020/0 (socket says 172.21.15.17:60020) 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 -- 172.21.15.17:0/2923445186 learned_addr learned my addr 172.21.15.17:0/2923445186 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8028a02640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240ac260 0x7f80241eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 -- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240ac260 msgr2=0x7f80241eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8023fff640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024152960 0x7f8024151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240ac260 0x7f80241eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 -- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024152960 msgr2=0x7f8024151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024152960 0x7f8024151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 -- 172.21.15.17:0/2923445186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024156f00 con 0x7f802414dce0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f8029203640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f801402b9c0 tx=0x7f80140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014034040 con 0x7f802414dce0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014039d90 con 0x7f802414dce0 2024-03-06T07:23:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80241571e0 con 0x7f802414dce0 2024-03-06T07:23:25.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014043610 con 0x7f802414dce0 2024-03-06T07:23:25.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.830+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80241ee4c0 con 0x7f802414dce0 2024-03-06T07:23:25.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.831+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802404f330 con 0x7f802414dce0 2024-03-06T07:23:25.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.832+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8014037070 con 0x7f802414dce0 2024-03-06T07:23:25.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.832+0000 7f80217fa640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:25.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.832+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80140ac450 con 0x7f802414dce0 2024-03-06T07:23:25.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.832+0000 7f8028a02640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:25.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.833+0000 7f8028a02640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f801c009f70 tx=0x7f801c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:25.836+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f801403e070 con 0x7f802414dce0 2024-03-06T07:23:26.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.017+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802406ebc0 con 0x7f802414dce0 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.018+0000 7f80217fa640 1 -- 172.21.15.17:0/2923445186 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f801407b140 con 0x7f802414dce0 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:26.020 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":95,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:24.744085+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 msgr2=0x7f801006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f801c009f70 tx=0x7f801c00b040 comp rx=0 tx=0).stop 2024-03-06T07:23:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 msgr2=0x7f802414e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f801402b9c0 tx=0x7f80140046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 shutdown_connections 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802414dce0 0x7f802414e580 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8024152960 0x7f8024151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 --2- 172.21.15.17:0/2923445186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240ac260 0x7f80241eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 >> 172.21.15.17:0/2923445186 conn(0x7f80240c70b0 msgr2=0x7f80240c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 shutdown_connections 2024-03-06T07:23:26.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.020+0000 7f802ac66640 1 -- 172.21.15.17:0/2923445186 wait complete. 2024-03-06T07:23:26.033 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:23:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.136+0000 7fe0ed97a640 1 Processor -- start 2024-03-06T07:23:26.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.136+0000 7fe0ed97a640 1 -- start start 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.136+0000 7fe0ed97a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0e80c26c0 0x7fe0e8155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.136+0000 7fe0ed97a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0ed97a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0e80670c0 con 0x7fe0e80c2aa0 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0ed97a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0e806b740 con 0x7fe0e80c26c0 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0ed97a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e806b0f0 con 0x7fe0e80c0470 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e77fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0e80c26c0 0x7fe0e8155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e77fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:37134/0 (socket says 172.21.15.17:37134) 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0e80c26c0 0x7fe0e8155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:59600/0 (socket says 172.21.15.17:59600) 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e77fe640 1 -- 172.21.15.17:0/393383034 learned_addr learned my addr 172.21.15.17:0/393383034 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0de7fc640 1 --1- 172.21.15.17:0/393383034 >> v1:172.21.15.163:6789/0 conn(0x7fe0e80c2aa0 0x7fe0e806bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:60882/0 (socket says 172.21.15.17:60882) 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e67fc640 1 -- 172.21.15.17:0/393383034 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001628864 0 0) 0x7fe0e80670c0 con 0x7fe0e80c2aa0 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e67fc640 1 -- 172.21.15.17:0/393383034 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d4003520 con 0x7fe0e80c2aa0 2024-03-06T07:23:26.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 msgr2=0x7fe0e8152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/393383034 >> v1:172.21.15.163:6789/0 conn(0x7fe0e80c2aa0 legacy=0x7fe0e806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/393383034 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e806b3d0 con 0x7fe0e80c26c0 2024-03-06T07:23:26.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.137+0000 7fe0e77fe640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.138+0000 7fe0e6ffd640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e8155bd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097a0 tx=0x7fe0d802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17289f7ffb5cf468 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.138+0000 7fe0e67fc640 1 -- 172.21.15.17:0/393383034 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8038070 con 0x7fe0e80c26c0 2024-03-06T07:23:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.138+0000 7fe0e67fc640 1 -- 172.21.15.17:0/393383034 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d8031700 con 0x7fe0e80c26c0 2024-03-06T07:23:26.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.138+0000 7fe0e67fc640 1 -- 172.21.15.17:0/393383034 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8031ad0 con 0x7fe0e80c26c0 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 -- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 msgr2=0x7fe0e8155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e8155bd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097a0 tx=0x7fe0d802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 -- 172.21.15.17:0/393383034 shutdown_connections 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c0470 0x7fe0e8152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/393383034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e8155bd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 -- 172.21.15.17:0/393383034 >> 172.21.15.17:0/393383034 conn(0x7fe0e80c3b40 msgr2=0x7fe0e80c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 -- 172.21.15.17:0/393383034 shutdown_connections 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.139+0000 7fe0ed97a640 1 -- 172.21.15.17:0/393383034 wait complete. 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.140+0000 7fe0ed97a640 1 Processor -- start 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.140+0000 7fe0ed97a640 1 -- start start 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c2aa0 0x7fe0e814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80c20b0 con 0x7fe0e80c0470 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80c1f30 con 0x7fe0e80c26c0 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0ed97a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e8066d00 con 0x7fe0e80c2aa0 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60030/0 (socket says 172.21.15.17:60030) 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/2584417033 learned_addr learned my addr 172.21.15.17:0/2584417033 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0de7fc640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e77fe640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c2aa0 0x7fe0e814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c2aa0 msgr2=0x7fe0e814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c2aa0 0x7fe0e814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 msgr2=0x7fe0e814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 -- 172.21.15.17:0/2584417033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e8153f90 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.141+0000 7fe0e6ffd640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8036040 tx=0x7fe0d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.142+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8042070 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.142+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d803d890 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.142+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e81eb570 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.142+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d8038040 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.142+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e81eaa90 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.143+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e804f330 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.143+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0d803b070 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.143+0000 7fe0e4ff9640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 0x7fe0c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.144+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0d80ab970 con 0x7fe0e80c0470 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.144+0000 7fe0de7fc640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 0x7fe0c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.144+0000 7fe0de7fc640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 0x7fe0c806d9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80acff0 tx=0x7fe0d0007cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.148+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0d8030070 con 0x7fe0e80c0470 2024-03-06T07:23:26.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.365+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0e80c3100 con 0x7fe0e80c0470 2024-03-06T07:23:26.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.382+0000 7fe0e4ff9640 1 -- 172.21.15.17:0/2584417033 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0d807a020 con 0x7fe0e80c0470 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 msgr2=0x7fe0c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 0x7fe0c806d9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80acff0 tx=0x7fe0d0007cf0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 msgr2=0x7fe0e814a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8036040 tx=0x7fe0d80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 shutdown_connections 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe0c806b500 0x7fe0c806d9c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0e80c2aa0 0x7fe0e814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0e80c26c0 0x7fe0e814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 --2- 172.21.15.17:0/2584417033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e80c0470 0x7fe0e814a350 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 >> 172.21.15.17:0/2584417033 conn(0x7fe0e80c3b40 msgr2=0x7fe0e806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 shutdown_connections 2024-03-06T07:23:26.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.384+0000 7fe0ed97a640 1 -- 172.21.15.17:0/2584417033 wait complete. 2024-03-06T07:23:26.398 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:23:26.399 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:23:26.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.502+0000 7fd0d11bf640 1 Processor -- start 2024-03-06T07:23:26.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.502+0000 7fd0d11bf640 1 -- start start 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.502+0000 7fd0d11bf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0cc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0d11bf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0c62a0 con 0x7fd0cc0c4740 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0d11bf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0c4c00 con 0x7fd0cc0a8290 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0d11bf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c4ee0 con 0x7fd0cc1539c0 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0c259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd0cc0c4740 0x7fd0cc0c69a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46510/0 (socket says 172.21.15.17:46510) 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0c259c640 1 -- 172.21.15.17:0/3504536979 learned_addr learned my addr 172.21.15.17:0/3504536979 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221522884 0 0) 0x7fd0cc0c62a0 con 0x7fd0cc0c4740 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b8003520 con 0x7fd0cc0c4740 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0cb59e640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0cc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404519554 0 0) 0x7fd0b8003520 con 0x7fd0cc0c4740 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 msgr2=0x7fd0cc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0cc072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 >> v1:172.21.15.163:6789/0 conn(0x7fd0cc0a8290 legacy=0x7fd0cc0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc06f9a0 con 0x7fd0cc0c4740 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd0b4001670 con 0x7fd0cc0c4740 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 >> v1:172.21.15.17:6789/0 conn(0x7fd0cc0c4740 legacy=0x7fd0cc0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b8004d40 0x7fd0b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c4ee0 con 0x7fd0b8003f60 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b800bc00 con 0x7fd0b8004d40 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b800be00 con 0x7fd0cc1539c0 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.503+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0cb59e640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b8004d40 0x7fd0b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 -- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 msgr2=0x7fd0b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:26.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1539c0 0x7fd0b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 -- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b8004d40 msgr2=0x7fd0b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b8004d40 0x7fd0b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 -- 172.21.15.17:0/3504536979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8003520 con 0x7fd0b8003f60 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.504+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0b8004800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c240 tx=0x7fd0b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66a3a5256907fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4007cf0 con 0x7fd0b8003f60 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4030070 con 0x7fd0b8003f60 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3504536979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4005d70 con 0x7fd0b8003f60 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3504536979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 msgr2=0x7fd0b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3504536979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0b8004800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c240 tx=0x7fd0b8005830 comp rx=0 tx=0).stop 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0cb59e640 1 -- 172.21.15.17:0/3504536979 reap_dead start 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3504536979 shutdown_connections 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3504536979 >> 172.21.15.17:0/3504536979 conn(0x7fd0cc0c5560 msgr2=0x7fd0cc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3504536979 shutdown_connections 2024-03-06T07:23:26.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.505+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3504536979 wait complete. 2024-03-06T07:23:26.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.506+0000 7fd0d11bf640 1 Processor -- start 2024-03-06T07:23:26.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 -- start start 2024-03-06T07:23:26.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0ad8d0 0x7fd0cc14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1eee90 0x7fd0cc0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc072680 con 0x7fd0b8003f60 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc153e50 con 0x7fd0cc0ad8d0 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0d11bf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc072800 con 0x7fd0cc1eee90 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0cad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0cad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60046/0 (socket says 172.21.15.17:60046) 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.507+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3448107500 learned_addr learned my addr 172.21.15.17:0/3448107500 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cb59e640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1eee90 0x7fd0cc0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0ad8d0 0x7fd0cc14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1eee90 msgr2=0x7fd0cc0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1eee90 0x7fd0cc0ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0ad8d0 msgr2=0x7fd0cc14d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0ad8d0 0x7fd0cc14d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3448107500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc14de40 con 0x7fd0b8003f60 2024-03-06T07:23:26.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00f850 tx=0x7fd0bc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00fe40 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc018070 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00d730 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc06cc60 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.508+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1ee690 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.509+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc04f330 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.510+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0bc00d8d0 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.510+0000 7fd0c3fff640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.510+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0bc08a1c0 con 0x7fd0b8003f60 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.510+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:26.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.511+0000 7fd0c259c640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4034000 tx=0x7fd0b4007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:26.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.514+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0bc058f40 con 0x7fd0b8003f60 2024-03-06T07:23:26.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:26.736+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd0cc0735e0 con 0x7fd0b8003f60 2024-03-06T07:23:27.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.392+0000 7fd0c3fff640 1 -- 172.21.15.17:0/3448107500 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0bc058840 con 0x7fd0b8003f60 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 msgr2=0x7fd0ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4034000 tx=0x7fd0b4007000 comp rx=0 tx=0).stop 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 msgr2=0x7fd0cc06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00f850 tx=0x7fd0bc00fd10 comp rx=0 tx=0).stop 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 shutdown_connections 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc1eee90 0x7fd0cc0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0ad8d0 0x7fd0cc14d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 --2- 172.21.15.17:0/3448107500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b8003f60 0x7fd0cc06c680 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 >> 172.21.15.17:0/3448107500 conn(0x7fd0cc0c5560 msgr2=0x7fd0cc0a92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 shutdown_connections 2024-03-06T07:23:27.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.395+0000 7fd0d11bf640 1 -- 172.21.15.17:0/3448107500 wait complete. 2024-03-06T07:23:27.408 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:23:27.408 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:27.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 Processor -- start 2024-03-06T07:23:27.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 -- start start 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7c40c5330 0x7fe7c40c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c4159630 con 0x7fe7c4152920 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c40c7760 con 0x7fe7c40c8a00 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.522+0000 7fe7ca255640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c41587b0 con 0x7fe7c40c5330 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7c4152920 0x7fe7c4155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46516/0 (socket says 172.21.15.17:46516) 2024-03-06T07:23:27.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c37fe640 1 -- 172.21.15.17:0/3250933548 learned_addr learned my addr 172.21.15.17:0/3250933548 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349933604 0 0) 0x7fe7c4159630 con 0x7fe7c4152920 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/3250933548 >> v2:172.21.15.163:3300/0 conn(0x7fe7c40c5330 0x7fe7c40c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ac003520 con 0x7fe7c4152920 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915372506 0 0) 0x7fe7ac003520 con 0x7fe7c4152920 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 >> v2:172.21.15.163:3300/0 conn(0x7fe7c40c5330 msgr2=0x7fe7c40c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 --2- 172.21.15.17:0/3250933548 >> v2:172.21.15.163:3300/0 conn(0x7fe7c40c5330 0x7fe7c40c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 >> v1:172.21.15.163:6789/0 conn(0x7fe7c40c8a00 legacy=0x7fe7c4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40c7a40 con 0x7fe7c4152920 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe7b4001670 con 0x7fe7c4152920 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 >> v1:172.21.15.17:6789/0 conn(0x7fe7c4152920 legacy=0x7fe7c4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7ac0039b0 0x7fe7ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac004800 0x7fe7ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 0x7fe7ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41587b0 con 0x7fe7ac0051c0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00c160 con 0x7fe7ac004800 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac00c360 con 0x7fe7ac0039b0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 0x7fe7ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7ac0039b0 0x7fe7ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.523+0000 7fe7c2ffd640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac004800 0x7fe7ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 -- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7ac0039b0 msgr2=0x7fe7ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7ac0039b0 0x7fe7ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 -- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac004800 msgr2=0x7fe7ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac004800 0x7fe7ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 -- 172.21.15.17:0/3250933548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac003520 con 0x7fe7ac0051c0 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c2ffd640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac004800 0x7fe7ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7ac0039b0 0x7fe7ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 -- 172.21.15.17:0/3250933548 reap_dead start 2024-03-06T07:23:27.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 0x7fe7ac00b9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009fc0 tx=0x7fe7ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffc76e46fd3bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.524+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc009c10 con 0x7fe7ac0051c0 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc02cca0 con 0x7fe7ac0051c0 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7c27fc640 1 -- 172.21.15.17:0/3250933548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc003bd0 con 0x7fe7ac0051c0 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 -- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 msgr2=0x7fe7ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 0x7fe7ac00b9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009fc0 tx=0x7fe7ac005830 comp rx=0 tx=0).stop 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 -- 172.21.15.17:0/3250933548 shutdown_connections 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 --2- 172.21.15.17:0/3250933548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7ac0051c0 0x7fe7ac00b9c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 -- 172.21.15.17:0/3250933548 >> 172.21.15.17:0/3250933548 conn(0x7fe7c40ac560 msgr2=0x7fe7c40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 -- 172.21.15.17:0/3250933548 shutdown_connections 2024-03-06T07:23:27.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.525+0000 7fe7ca255640 1 -- 172.21.15.17:0/3250933548 wait complete. 2024-03-06T07:23:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.526+0000 7fe7ca255640 1 Processor -- start 2024-03-06T07:23:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.526+0000 7fe7ca255640 1 -- start start 2024-03-06T07:23:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 0x7fe7c40aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7c41ec430 0x7fe7c4148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c406d030 con 0x7fe7c41eec00 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4158a00 con 0x7fe7ac0051c0 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7ca255640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c40c7390 con 0x7fe7c41ec430 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60064/0 (socket says 172.21.15.17:60064) 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 -- 172.21.15.17:0/1512615650 learned_addr learned my addr 172.21.15.17:0/1512615650 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 0x7fe7c40aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c3fff640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7c41ec430 0x7fe7c4148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 -- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7c41ec430 msgr2=0x7fe7c4148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7c41ec430 0x7fe7c4148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 -- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 msgr2=0x7fe7c40aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 0x7fe7c40aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c2ffd640 1 -- 172.21.15.17:0/1512615650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4147cd0 con 0x7fe7c41eec00 2024-03-06T07:23:27.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.527+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 0x7fe7c40aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7c2ffd640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e460 tx=0x7fe7b800e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8019070 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b80052e0 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c41ee120 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8003830 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.528+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c40ab390 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.529+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c404f330 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.529+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7b801c070 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.530+0000 7fe7abfff640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 0x7fe7b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.530+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7b80897e0 con 0x7fe7c41eec00 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.530+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 0x7fe7b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.530+0000 7fe7c37fe640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 0x7fe7b006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4001200 tx=0x7fe7b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.538+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe7b8057f10 con 0x7fe7c41eec00 2024-03-06T07:23:27.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.758+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7c4156390 con 0x7fe7c41eec00 2024-03-06T07:23:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.758+0000 7fe7abfff640 1 -- 172.21.15.17:0/1512615650 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fe7b805bfb0 con 0x7fe7c41eec00 2024-03-06T07:23:27.760 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:27.760 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:26.737853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3941058610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3941058610},{"type":"v1","addr":"172.21.15.17:6835","nonce":3941058610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":7}]} 2024-03-06T07:23:27.760 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 78 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 msgr2=0x7fe7b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 0x7fe7b006d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4001200 tx=0x7fe7b4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 msgr2=0x7fe7c41ebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e460 tx=0x7fe7b800e920 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 shutdown_connections 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7b006b500 0x7fe7b006d9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7c41ec430 0x7fe7c4148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c41eec00 0x7fe7c41ebef0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 --2- 172.21.15.17:0/1512615650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7ac0051c0 0x7fe7c40aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 >> 172.21.15.17:0/1512615650 conn(0x7fe7c40ac560 msgr2=0x7fe7c414fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 shutdown_connections 2024-03-06T07:23:27.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.760+0000 7fe7ca255640 1 -- 172.21.15.17:0/1512615650 wait complete. 2024-03-06T07:23:27.773 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:27.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.873+0000 7f6c06e9a640 1 Processor -- start 2024-03-06T07:23:27.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.873+0000 7f6c06e9a640 1 -- start start 2024-03-06T07:23:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6c06e9a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6c000c5860 0x7f6c001609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6c06e9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c001591c0 con 0x7f6c00155ba0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6c06e9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c001594a0 con 0x7f6c000c5480 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6c06e9a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0006df40 con 0x7f6c000c5860 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6bfffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c00155ba0 0x7f6c001586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46518/0 (socket says 172.21.15.17:46518) 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6bfffff640 1 -- 172.21.15.17:0/377415218 learned_addr learned my addr 172.21.15.17:0/377415218 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6c05437640 1 --2- 172.21.15.17:0/377415218 >> v2:172.21.15.163:3300/0 conn(0x7f6c000c5860 0x7f6c001609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.874+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566966200 0 0) 0x7f6c001591c0 con 0x7f6c00155ba0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf0003520 con 0x7f6c00155ba0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185605110 0 0) 0x7f6bf0003520 con 0x7f6c00155ba0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 >> v2:172.21.15.163:3300/0 conn(0x7f6c000c5860 msgr2=0x7f6c001609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 --2- 172.21.15.17:0/377415218 >> v2:172.21.15.163:3300/0 conn(0x7f6c000c5860 0x7f6c001609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 >> v1:172.21.15.163:6789/0 conn(0x7f6c000c5480 legacy=0x7f6c0006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c001553a0 con 0x7f6c00155ba0 2024-03-06T07:23:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6bec001670 con 0x7f6c00155ba0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c05437640 1 --2- 172.21.15.17:0/377415218 >> v2:172.21.15.163:3300/0 conn(0x7f6c000c5860 0x7f6c001609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 >> v1:172.21.15.17:6789/0 conn(0x7f6c00155ba0 legacy=0x7f6c001586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 0x7f6bf0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf0004800 0x7f6bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bf00051c0 0x7f6bf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0006df40 con 0x7f6bf00039b0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf000c160 con 0x7f6bf00051c0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf000c360 con 0x7f6bf0004800 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 0x7f6bf0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c05437640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bf00051c0 0x7f6bf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6bfffff640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf0004800 0x7f6bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 -- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf0004800 msgr2=0x7f6bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf0004800 0x7f6bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 -- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bf00051c0 msgr2=0x7f6bf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bf00051c0 0x7f6bf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c04c36640 1 -- 172.21.15.17:0/377415218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf0003520 con 0x7f6bf00039b0 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.875+0000 7f6c05437640 1 -- 172.21.15.17:0/377415218 reap_dead start 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.876+0000 7f6c04c36640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 0x7f6bf0003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4001750 tx=0x7f6bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca08d412eb166638 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.876+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4001370 con 0x7f6bf00039b0 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.876+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf40052d0 con 0x7f6bf00039b0 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.876+0000 7f6bff7fe640 1 -- 172.21.15.17:0/377415218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4003830 con 0x7f6bf00039b0 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 -- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 msgr2=0x7f6bf0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 0x7f6bf0003dd0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4001750 tx=0x7f6bf0005830 comp rx=0 tx=0).stop 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 -- 172.21.15.17:0/377415218 shutdown_connections 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/377415218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bf00039b0 0x7f6bf0003dd0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 -- 172.21.15.17:0/377415218 >> 172.21.15.17:0/377415218 conn(0x7f6c0006b1a0 msgr2=0x7f6c0006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 -- 172.21.15.17:0/377415218 shutdown_connections 2024-03-06T07:23:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.877+0000 7f6c06e9a640 1 -- 172.21.15.17:0/377415218 wait complete. 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.878+0000 7f6c06e9a640 1 Processor -- start 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.878+0000 7f6c06e9a640 1 -- start start 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf00039b0 0x7f6c00154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c000a9090 0x7f6c00152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c000c5cf0 con 0x7f6c000a8c60 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0006fdb0 con 0x7f6c000a9090 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c06e9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00156000 con 0x7f6bf00039b0 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60080/0 (socket says 172.21.15.17:60080) 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 -- 172.21.15.17:0/1331809438 learned_addr learned my addr 172.21.15.17:0/1331809438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:27.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c04c36640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf00039b0 0x7f6c00154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6c05437640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c000a9090 0x7f6c00152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 -- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf00039b0 msgr2=0x7f6c00154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf00039b0 0x7f6c00154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 -- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c000a9090 msgr2=0x7f6c00152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c000a9090 0x7f6c00152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 -- 172.21.15.17:0/1331809438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c001eea90 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfffff640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00e760 tx=0x7f6bec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.880+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00ee70 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.881+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec018070 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.881+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c001eed30 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.881+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00cab0 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.881+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0009af20 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.881+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0004f330 con 0x7f6c000a8c60 2024-03-06T07:23:27.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.882+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bec00cc50 con 0x7f6c000a8c60 2024-03-06T07:23:27.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.883+0000 7f6bfd7fa640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 0x7f6be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:27.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.883+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6bec08a060 con 0x7f6c000a8c60 2024-03-06T07:23:27.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.883+0000 7f6c04c36640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 0x7f6be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:27.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.884+0000 7f6c04c36640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 0x7f6be406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4033000 tx=0x7f6bf4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:27.887+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6bec0586f0 con 0x7f6c000a8c60 2024-03-06T07:23:28.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.070+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c0006e0c0 con 0x7f6c000a8c60 2024-03-06T07:23:28.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.071+0000 7f6bfd7fa640 1 -- 172.21.15.17:0/1331809438 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f6bec05c790 con 0x7f6c000a8c60 2024-03-06T07:23:28.072 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:28.073 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":96,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:26.382264+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 msgr2=0x7f6be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 0x7f6be406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4033000 tx=0x7f6bf4009030 comp rx=0 tx=0).stop 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 msgr2=0x7f6c00151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00e760 tx=0x7f6bec00ec20 comp rx=0 tx=0).stop 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 shutdown_connections 2024-03-06T07:23:28.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6be406b4d0 0x7f6be406d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c000a9090 0x7f6c00152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c000a8c60 0x7f6c00151ac0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 --2- 172.21.15.17:0/1331809438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bf00039b0 0x7f6c00154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 >> 172.21.15.17:0/1331809438 conn(0x7f6c0006b1a0 msgr2=0x7f6c0006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 shutdown_connections 2024-03-06T07:23:28.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.073+0000 7f6c06e9a640 1 -- 172.21.15.17:0/1331809438 wait complete. 2024-03-06T07:23:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:23:28.086 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.189+0000 7fd3cc09a640 1 Processor -- start 2024-03-06T07:23:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 -- start start 2024-03-06T07:23:28.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3c406d9f0 0x7fd3c40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3c40c3280 0x7fd3c40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c4158800 con 0x7fd3c41551e0 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c4158ae0 con 0x7fd3c40c3280 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.190+0000 7fd3cc09a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c4158dc0 con 0x7fd3c406d9f0 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3c40c3280 0x7fd3c40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3ca637640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd3c41551e0 0x7fd3c41555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46524/0 (socket says 172.21.15.17:46524) 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3ca637640 1 -- 172.21.15.17:0/714814991 learned_addr learned my addr 172.21.15.17:0/714814991 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3c40c3280 0x7fd3c40c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60088/0 (socket says 172.21.15.17:60088) 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9e36640 1 --2- 172.21.15.17:0/714814991 >> v2:172.21.15.163:3300/0 conn(0x7fd3c406d9f0 0x7fd3c40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 -- 172.21.15.17:0/714814991 >> v2:172.21.15.163:3300/0 conn(0x7fd3c406d9f0 msgr2=0x7fd3c40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 --2- 172.21.15.17:0/714814991 >> v2:172.21.15.163:3300/0 conn(0x7fd3c406d9f0 0x7fd3c40a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 -- 172.21.15.17:0/714814991 >> v1:172.21.15.17:6789/0 conn(0x7fd3c41551e0 legacy=0x7fd3c41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c8e34640 1 -- 172.21.15.17:0/714814991 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769412506 0 0) 0x7fd3c4158800 con 0x7fd3c41551e0 2024-03-06T07:23:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.191+0000 7fd3c9635640 1 -- 172.21.15.17:0/714814991 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c41590a0 con 0x7fd3c40c3280 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.192+0000 7fd3c9635640 1 --2- 172.21.15.17:0/714814991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c40c3280 0x7fd3c40c3710 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40c9820 tx=0x7fd3b402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d9c15e1517682 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.192+0000 7fd3c8e34640 1 -- 172.21.15.17:0/714814991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40315a0 con 0x7fd3c40c3280 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.192+0000 7fd3c8e34640 1 -- 172.21.15.17:0/714814991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b4037070 con 0x7fd3c40c3280 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.192+0000 7fd3c8e34640 1 -- 172.21.15.17:0/714814991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4031960 con 0x7fd3c40c3280 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 -- 172.21.15.17:0/714814991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c40c3280 msgr2=0x7fd3c40c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/714814991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c40c3280 0x7fd3c40c3710 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd3c40c9820 tx=0x7fd3b402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 -- 172.21.15.17:0/714814991 shutdown_connections 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/714814991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c40c3280 0x7fd3c40c3710 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/714814991 >> v2:172.21.15.163:3300/0 conn(0x7fd3c406d9f0 0x7fd3c40a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 -- 172.21.15.17:0/714814991 >> 172.21.15.17:0/714814991 conn(0x7fd3c40aa630 msgr2=0x7fd3c40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 -- 172.21.15.17:0/714814991 shutdown_connections 2024-03-06T07:23:28.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.193+0000 7fd3cc09a640 1 -- 172.21.15.17:0/714814991 wait complete. 2024-03-06T07:23:28.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 Processor -- start 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 -- start start 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3c40c3280 0x7fd3c40a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3c41551e0 0x7fd3c414da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c406df50 con 0x7fd3c406d9f0 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c406ddd0 con 0x7fd3c41551e0 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.194+0000 7fd3cc09a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40a8210 con 0x7fd3c40c3280 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60102/0 (socket says 172.21.15.17:60102) 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 -- 172.21.15.17:0/3184948834 learned_addr learned my addr 172.21.15.17:0/3184948834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9635640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3c40c3280 0x7fd3c40a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3ca637640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3c41551e0 0x7fd3c414da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 -- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3c40c3280 msgr2=0x7fd3c40a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3c40c3280 0x7fd3c40a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 -- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3c41551e0 msgr2=0x7fd3c414da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3c41551e0 0x7fd3c414da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 -- 172.21.15.17:0/3184948834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c406e620 con 0x7fd3c406d9f0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3c9e36640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000d800 tx=0x7fd3c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000b660 con 0x7fd3c406d9f0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c0019070 con 0x7fd3c406d9f0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c4156e00 con 0x7fd3c406d9f0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000ba60 con 0x7fd3c406d9f0 2024-03-06T07:23:28.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.195+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c41573c0 con 0x7fd3c406d9f0 2024-03-06T07:23:28.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.197+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c404f330 con 0x7fd3c406d9f0 2024-03-06T07:23:28.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.197+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3c000bc00 con 0x7fd3c406d9f0 2024-03-06T07:23:28.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.197+0000 7fd3ba7fc640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.197+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3c008a790 con 0x7fd3c406d9f0 2024-03-06T07:23:28.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.198+0000 7fd3c9635640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.198+0000 7fd3c9635640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 0x7fd3a806d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4036000 tx=0x7fd3b4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.202+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd3c0059480 con 0x7fd3c406d9f0 2024-03-06T07:23:28.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.379+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3c41521e0 con 0x7fd3a806b500 2024-03-06T07:23:28.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.385+0000 7fd3ba7fc640 1 -- 172.21.15.17:0/3184948834 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252158 (secure 0 0 0) 0x7fd3c41521e0 con 0x7fd3a806b500 2024-03-06T07:23:28.390 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:28.391 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 msgr2=0x7fd3a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 0x7fd3a806d9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4036000 tx=0x7fd3b4008030 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 msgr2=0x7fd3c4157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000d800 tx=0x7fd3c000dcc0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 shutdown_connections 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3c41551e0 0x7fd3c414da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3c40c3280 0x7fd3c40a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 --2- 172.21.15.17:0/3184948834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3c406d9f0 0x7fd3c4157f80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 >> 172.21.15.17:0/3184948834 conn(0x7fd3c40aa630 msgr2=0x7fd3c41507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 shutdown_connections 2024-03-06T07:23:28.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.388+0000 7fd3cc09a640 1 -- 172.21.15.17:0/3184948834 wait complete. 2024-03-06T07:23:28.404 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-06T07:23:27.386176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254756,"kb_used_data":9456,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719964,"statfs":{"total":773094113280,"available":772833243136,"internally_reserved":0,"allocated":9682944,"data_stored":5785084,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022285"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390365+0000","last_change":"2024-03-06T07:23:25.392757+0000","last_active":"2024-03-06T07:23:26.390365+0000","last_peered":"2024-03-06T07:23:26.390365+0000","last_clean":"2024-03-06T07:23:26.390365+0000","last_became_active":"2024-03-06T07:23:25.392461+0000","last_became_peered":"2024-03-06T07:23:25.392461+0000","last_unstale":"2024-03-06T07:23:26.390365+0000","last_undegraded":"2024-03-06T07:23:26.390365+0000","last_fullsized":"2024-03-06T07:23:26.390365+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:53.201629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391785+0000","last_change":"2024-03-06T07:23:25.391927+0000","last_active":"2024-03-06T07:23:26.391785+0000","last_peered":"2024-03-06T07:23:26.391785+0000","last_clean":"2024-03-06T07:23:26.391785+0000","last_became_active":"2024-03-06T07:23:25.391243+0000","last_became_peered":"2024-03-06T07:23:25.391243+0000","last_unstale":"2024-03-06T07:23:26.391785+0000","last_undegraded":"2024-03-06T07:23:26.391785+0000","last_fullsized":"2024-03-06T07:23:26.391785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389845+0000","last_change":"2024-03-06T07:23:25.399576+0000","last_active":"2024-03-06T07:23:26.389845+0000","last_peered":"2024-03-06T07:23:26.389845+0000","last_clean":"2024-03-06T07:23:26.389845+0000","last_became_active":"2024-03-06T07:23:25.399320+0000","last_became_peered":"2024-03-06T07:23:25.399320+0000","last_unstale":"2024-03-06T07:23:26.389845+0000","last_undegraded":"2024-03-06T07:23:26.389845+0000","last_fullsized":"2024-03-06T07:23:26.389845+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:26.612372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972471+0000","last_change":"2024-03-06T07:23:25.388327+0000","last_active":"2024-03-06T07:23:26.972471+0000","last_peered":"2024-03-06T07:23:26.972471+0000","last_clean":"2024-03-06T07:23:26.972471+0000","last_became_active":"2024-03-06T07:23:25.387934+0000","last_became_peered":"2024-03-06T07:23:25.387934+0000","last_unstale":"2024-03-06T07:23:26.972471+0000","last_undegraded":"2024-03-06T07:23:26.972471+0000","last_fullsized":"2024-03-06T07:23:26.972471+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:18.840228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972354+0000","last_change":"2024-03-06T07:23:25.402105+0000","last_active":"2024-03-06T07:23:26.972354+0000","last_peered":"2024-03-06T07:23:26.972354+0000","last_clean":"2024-03-06T07:23:26.972354+0000","last_became_active":"2024-03-06T07:23:25.401273+0000","last_became_peered":"2024-03-06T07:23:25.401273+0000","last_unstale":"2024-03-06T07:23:26.972354+0000","last_undegraded":"2024-03-06T07:23:26.972354+0000","last_fullsized":"2024-03-06T07:23:26.972354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391795+0000","last_change":"2024-03-06T07:23:25.393894+0000","last_active":"2024-03-06T07:23:26.391795+0000","last_peered":"2024-03-06T07:23:26.391795+0000","last_clean":"2024-03-06T07:23:26.391795+0000","last_became_active":"2024-03-06T07:23:25.393624+0000","last_became_peered":"2024-03-06T07:23:25.393624+0000","last_unstale":"2024-03-06T07:23:26.391795+0000","last_undegraded":"2024-03-06T07:23:26.391795+0000","last_fullsized":"2024-03-06T07:23:26.391795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391855+0000","last_change":"2024-03-06T07:23:25.402036+0000","last_active":"2024-03-06T07:23:26.391855+0000","last_peered":"2024-03-06T07:23:26.391855+0000","last_clean":"2024-03-06T07:23:26.391855+0000","last_became_active":"2024-03-06T07:23:25.401786+0000","last_became_peered":"2024-03-06T07:23:25.401786+0000","last_unstale":"2024-03-06T07:23:26.391855+0000","last_undegraded":"2024-03-06T07:23:26.391855+0000","last_fullsized":"2024-03-06T07:23:26.391855+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972114+0000","last_change":"2024-03-06T07:23:25.406026+0000","last_active":"2024-03-06T07:23:26.972114+0000","last_peered":"2024-03-06T07:23:26.972114+0000","last_clean":"2024-03-06T07:23:26.972114+0000","last_became_active":"2024-03-06T07:23:25.404996+0000","last_became_peered":"2024-03-06T07:23:25.404996+0000","last_unstale":"2024-03-06T07:23:26.972114+0000","last_undegraded":"2024-03-06T07:23:26.972114+0000","last_fullsized":"2024-03-06T07:23:26.972114+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:46.674666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392397+0000","last_change":"2024-03-06T07:23:25.389370+0000","last_active":"2024-03-06T07:23:26.392397+0000","last_peered":"2024-03-06T07:23:26.392397+0000","last_clean":"2024-03-06T07:23:26.392397+0000","last_became_active":"2024-03-06T07:23:25.388802+0000","last_became_peered":"2024-03-06T07:23:25.388802+0000","last_unstale":"2024-03-06T07:23:26.392397+0000","last_undegraded":"2024-03-06T07:23:26.392397+0000","last_fullsized":"2024-03-06T07:23:26.392397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:28.292286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972730+0000","last_change":"2024-03-06T07:23:25.400328+0000","last_active":"2024-03-06T07:23:26.972730+0000","last_peered":"2024-03-06T07:23:26.972730+0000","last_clean":"2024-03-06T07:23:26.972730+0000","last_became_active":"2024-03-06T07:23:25.400063+0000","last_became_peered":"2024-03-06T07:23:25.400063+0000","last_unstale":"2024-03-06T07:23:26.972730+0000","last_undegraded":"2024-03-06T07:23:26.972730+0000","last_fullsized":"2024-03-06T07:23:26.972730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471039+0000","last_change":"2024-03-06T07:23:25.397761+0000","last_active":"2024-03-06T07:23:26.471039+0000","last_peered":"2024-03-06T07:23:26.471039+0000","last_clean":"2024-03-06T07:23:26.471039+0000","last_became_active":"2024-03-06T07:23:25.397341+0000","last_became_peered":"2024-03-06T07:23:25.397341+0000","last_unstale":"2024-03-06T07:23:26.471039+0000","last_undegraded":"2024-03-06T07:23:26.471039+0000","last_fullsized":"2024-03-06T07:23:26.471039+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392766+0000","last_change":"2024-03-06T07:23:25.387742+0000","last_active":"2024-03-06T07:23:26.392766+0000","last_peered":"2024-03-06T07:23:26.392766+0000","last_clean":"2024-03-06T07:23:26.392766+0000","last_became_active":"2024-03-06T07:23:25.387236+0000","last_became_peered":"2024-03-06T07:23:25.387236+0000","last_unstale":"2024-03-06T07:23:26.392766+0000","last_undegraded":"2024-03-06T07:23:26.392766+0000","last_fullsized":"2024-03-06T07:23:26.392766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971762+0000","last_change":"2024-03-06T07:23:25.398485+0000","last_active":"2024-03-06T07:23:26.971762+0000","last_peered":"2024-03-06T07:23:26.971762+0000","last_clean":"2024-03-06T07:23:26.971762+0000","last_became_active":"2024-03-06T07:23:25.398197+0000","last_became_peered":"2024-03-06T07:23:25.398197+0000","last_unstale":"2024-03-06T07:23:26.971762+0000","last_undegraded":"2024-03-06T07:23:26.971762+0000","last_fullsized":"2024-03-06T07:23:26.971762+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:25.808803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971851+0000","last_change":"2024-03-06T07:23:25.392696+0000","last_active":"2024-03-06T07:23:26.971851+0000","last_peered":"2024-03-06T07:23:26.971851+0000","last_clean":"2024-03-06T07:23:26.971851+0000","last_became_active":"2024-03-06T07:23:25.392399+0000","last_became_peered":"2024-03-06T07:23:25.392399+0000","last_unstale":"2024-03-06T07:23:26.971851+0000","last_undegraded":"2024-03-06T07:23:26.971851+0000","last_fullsized":"2024-03-06T07:23:26.971851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:16.216161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971692+0000","last_change":"2024-03-06T07:23:25.398526+0000","last_active":"2024-03-06T07:23:26.971692+0000","last_peered":"2024-03-06T07:23:26.971692+0000","last_clean":"2024-03-06T07:23:26.971692+0000","last_became_active":"2024-03-06T07:23:25.398228+0000","last_became_peered":"2024-03-06T07:23:25.398228+0000","last_unstale":"2024-03-06T07:23:26.971692+0000","last_undegraded":"2024-03-06T07:23:26.971692+0000","last_fullsized":"2024-03-06T07:23:26.971692+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:28.292286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971711+0000","last_change":"2024-03-06T07:23:25.390767+0000","last_active":"2024-03-06T07:23:26.971711+0000","last_peered":"2024-03-06T07:23:26.971711+0000","last_clean":"2024-03-06T07:23:26.971711+0000","last_became_active":"2024-03-06T07:23:25.390510+0000","last_became_peered":"2024-03-06T07:23:25.390510+0000","last_unstale":"2024-03-06T07:23:26.971711+0000","last_undegraded":"2024-03-06T07:23:26.971711+0000","last_fullsized":"2024-03-06T07:23:26.971711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:34.576371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390382+0000","last_change":"2024-03-06T07:23:25.398842+0000","last_active":"2024-03-06T07:23:26.390382+0000","last_peered":"2024-03-06T07:23:26.390382+0000","last_clean":"2024-03-06T07:23:26.390382+0000","last_became_active":"2024-03-06T07:23:25.398511+0000","last_became_peered":"2024-03-06T07:23:25.398511+0000","last_unstale":"2024-03-06T07:23:26.390382+0000","last_undegraded":"2024-03-06T07:23:26.390382+0000","last_fullsized":"2024-03-06T07:23:26.390382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:35:38.035908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392004+0000","last_change":"2024-03-06T07:23:25.391708+0000","last_active":"2024-03-06T07:23:26.392004+0000","last_peered":"2024-03-06T07:23:26.392004+0000","last_clean":"2024-03-06T07:23:26.392004+0000","last_became_active":"2024-03-06T07:23:25.391427+0000","last_became_peered":"2024-03-06T07:23:25.391427+0000","last_unstale":"2024-03-06T07:23:26.392004+0000","last_undegraded":"2024-03-06T07:23:26.392004+0000","last_fullsized":"2024-03-06T07:23:26.392004+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390212+0000","last_change":"2024-03-06T07:23:25.400482+0000","last_active":"2024-03-06T07:23:26.390212+0000","last_peered":"2024-03-06T07:23:26.390212+0000","last_clean":"2024-03-06T07:23:26.390212+0000","last_became_active":"2024-03-06T07:23:25.400204+0000","last_became_peered":"2024-03-06T07:23:25.400204+0000","last_unstale":"2024-03-06T07:23:26.390212+0000","last_undegraded":"2024-03-06T07:23:26.390212+0000","last_fullsized":"2024-03-06T07:23:26.390212+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:08.683727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392789+0000","last_change":"2024-03-06T07:23:25.394192+0000","last_active":"2024-03-06T07:23:26.392789+0000","last_peered":"2024-03-06T07:23:26.392789+0000","last_clean":"2024-03-06T07:23:26.392789+0000","last_became_active":"2024-03-06T07:23:25.393910+0000","last_became_peered":"2024-03-06T07:23:25.393910+0000","last_unstale":"2024-03-06T07:23:26.392789+0000","last_undegraded":"2024-03-06T07:23:26.392789+0000","last_fullsized":"2024-03-06T07:23:26.392789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:26.923866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972707+0000","last_change":"2024-03-06T07:23:25.400576+0000","last_active":"2024-03-06T07:23:26.972707+0000","last_peered":"2024-03-06T07:23:26.972707+0000","last_clean":"2024-03-06T07:23:26.972707+0000","last_became_active":"2024-03-06T07:23:25.400306+0000","last_became_peered":"2024-03-06T07:23:25.400306+0000","last_unstale":"2024-03-06T07:23:26.972707+0000","last_undegraded":"2024-03-06T07:23:26.972707+0000","last_fullsized":"2024-03-06T07:23:26.972707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:18.977243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971772+0000","last_change":"2024-03-06T07:23:25.403369+0000","last_active":"2024-03-06T07:23:26.971772+0000","last_peered":"2024-03-06T07:23:26.971772+0000","last_clean":"2024-03-06T07:23:26.971772+0000","last_became_active":"2024-03-06T07:23:25.402913+0000","last_became_peered":"2024-03-06T07:23:25.402913+0000","last_unstale":"2024-03-06T07:23:26.971772+0000","last_undegraded":"2024-03-06T07:23:26.971772+0000","last_fullsized":"2024-03-06T07:23:26.971772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:14.637534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389828+0000","last_change":"2024-03-06T07:23:25.402260+0000","last_active":"2024-03-06T07:23:26.389828+0000","last_peered":"2024-03-06T07:23:26.389828+0000","last_clean":"2024-03-06T07:23:26.389828+0000","last_became_active":"2024-03-06T07:23:25.402004+0000","last_became_peered":"2024-03-06T07:23:25.402004+0000","last_unstale":"2024-03-06T07:23:26.389828+0000","last_undegraded":"2024-03-06T07:23:26.389828+0000","last_fullsized":"2024-03-06T07:23:26.389828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:10.654230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971595+0000","last_change":"2024-03-06T07:23:25.403146+0000","last_active":"2024-03-06T07:23:26.971595+0000","last_peered":"2024-03-06T07:23:26.971595+0000","last_clean":"2024-03-06T07:23:26.971595+0000","last_became_active":"2024-03-06T07:23:25.402880+0000","last_became_peered":"2024-03-06T07:23:25.402880+0000","last_unstale":"2024-03-06T07:23:26.971595+0000","last_undegraded":"2024-03-06T07:23:26.971595+0000","last_fullsized":"2024-03-06T07:23:26.971595+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:10.821669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392734+0000","last_change":"2024-03-06T07:23:25.401581+0000","last_active":"2024-03-06T07:23:26.392734+0000","last_peered":"2024-03-06T07:23:26.392734+0000","last_clean":"2024-03-06T07:23:26.392734+0000","last_became_active":"2024-03-06T07:23:25.401250+0000","last_became_peered":"2024-03-06T07:23:25.401250+0000","last_unstale":"2024-03-06T07:23:26.392734+0000","last_undegraded":"2024-03-06T07:23:26.392734+0000","last_fullsized":"2024-03-06T07:23:26.392734+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.393063+0000","last_change":"2024-03-06T07:23:25.380731+0000","last_active":"2024-03-06T07:23:26.393063+0000","last_peered":"2024-03-06T07:23:26.393063+0000","last_clean":"2024-03-06T07:23:26.393063+0000","last_became_active":"2024-03-06T07:23:25.380442+0000","last_became_peered":"2024-03-06T07:23:25.380442+0000","last_unstale":"2024-03-06T07:23:26.393063+0000","last_undegraded":"2024-03-06T07:23:26.393063+0000","last_fullsized":"2024-03-06T07:23:26.393063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:57.628480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471090+0000","last_change":"2024-03-06T07:23:25.400133+0000","last_active":"2024-03-06T07:23:26.471090+0000","last_peered":"2024-03-06T07:23:26.471090+0000","last_clean":"2024-03-06T07:23:26.471090+0000","last_became_active":"2024-03-06T07:23:25.399237+0000","last_became_peered":"2024-03-06T07:23:25.399237+0000","last_unstale":"2024-03-06T07:23:26.471090+0000","last_undegraded":"2024-03-06T07:23:26.471090+0000","last_fullsized":"2024-03-06T07:23:26.471090+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391565+0000","last_change":"2024-03-06T07:23:25.398808+0000","last_active":"2024-03-06T07:23:26.391565+0000","last_peered":"2024-03-06T07:23:26.391565+0000","last_clean":"2024-03-06T07:23:26.391565+0000","last_became_active":"2024-03-06T07:23:25.398485+0000","last_became_peered":"2024-03-06T07:23:25.398485+0000","last_unstale":"2024-03-06T07:23:26.391565+0000","last_undegraded":"2024-03-06T07:23:26.391565+0000","last_fullsized":"2024-03-06T07:23:26.391565+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972772+0000","last_change":"2024-03-06T07:23:25.392067+0000","last_active":"2024-03-06T07:23:26.972772+0000","last_peered":"2024-03-06T07:23:26.972772+0000","last_clean":"2024-03-06T07:23:26.972772+0000","last_became_active":"2024-03-06T07:23:25.391753+0000","last_became_peered":"2024-03-06T07:23:25.391753+0000","last_unstale":"2024-03-06T07:23:26.972772+0000","last_undegraded":"2024-03-06T07:23:26.972772+0000","last_fullsized":"2024-03-06T07:23:26.972772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:31.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392171+0000","last_change":"2024-03-06T07:23:25.399088+0000","last_active":"2024-03-06T07:23:26.392171+0000","last_peered":"2024-03-06T07:23:26.392171+0000","last_clean":"2024-03-06T07:23:26.392171+0000","last_became_active":"2024-03-06T07:23:25.398486+0000","last_became_peered":"2024-03-06T07:23:25.398486+0000","last_unstale":"2024-03-06T07:23:26.392171+0000","last_undegraded":"2024-03-06T07:23:26.392171+0000","last_fullsized":"2024-03-06T07:23:26.392171+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392286+0000","last_change":"2024-03-06T07:23:25.385062+0000","last_active":"2024-03-06T07:23:26.392286+0000","last_peered":"2024-03-06T07:23:26.392286+0000","last_clean":"2024-03-06T07:23:26.392286+0000","last_became_active":"2024-03-06T07:23:25.384785+0000","last_became_peered":"2024-03-06T07:23:25.384785+0000","last_unstale":"2024-03-06T07:23:26.392286+0000","last_undegraded":"2024-03-06T07:23:26.392286+0000","last_fullsized":"2024-03-06T07:23:26.392286+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:57.628480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972252+0000","last_change":"2024-03-06T07:23:25.382968+0000","last_active":"2024-03-06T07:23:26.972252+0000","last_peered":"2024-03-06T07:23:26.972252+0000","last_clean":"2024-03-06T07:23:26.972252+0000","last_became_active":"2024-03-06T07:23:25.382667+0000","last_became_peered":"2024-03-06T07:23:25.382667+0000","last_unstale":"2024-03-06T07:23:26.972252+0000","last_undegraded":"2024-03-06T07:23:26.972252+0000","last_fullsized":"2024-03-06T07:23:26.972252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:57.987167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389880+0000","last_change":"2024-03-06T07:23:25.394981+0000","last_active":"2024-03-06T07:23:26.389880+0000","last_peered":"2024-03-06T07:23:26.389880+0000","last_clean":"2024-03-06T07:23:26.389880+0000","last_became_active":"2024-03-06T07:23:25.394349+0000","last_became_peered":"2024-03-06T07:23:25.394349+0000","last_unstale":"2024-03-06T07:23:26.389880+0000","last_undegraded":"2024-03-06T07:23:26.389880+0000","last_fullsized":"2024-03-06T07:23:26.389880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:16.831201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972155+0000","last_change":"2024-03-06T07:23:24.388032+0000","last_active":"2024-03-06T07:23:26.972155+0000","last_peered":"2024-03-06T07:23:26.972155+0000","last_clean":"2024-03-06T07:23:26.972155+0000","last_became_active":"2024-03-06T07:23:24.387749+0000","last_became_peered":"2024-03-06T07:23:24.387749+0000","last_unstale":"2024-03-06T07:23:26.972155+0000","last_undegraded":"2024-03-06T07:23:26.972155+0000","last_fullsized":"2024-03-06T07:23:26.972155+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:25.918590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972466+0000","last_change":"2024-03-06T07:23:25.405419+0000","last_active":"2024-03-06T07:23:26.972466+0000","last_peered":"2024-03-06T07:23:26.972466+0000","last_clean":"2024-03-06T07:23:26.972466+0000","last_became_active":"2024-03-06T07:23:25.405156+0000","last_became_peered":"2024-03-06T07:23:25.405156+0000","last_unstale":"2024-03-06T07:23:26.972466+0000","last_undegraded":"2024-03-06T07:23:26.972466+0000","last_fullsized":"2024-03-06T07:23:26.972466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:58:33.300913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972461+0000","last_change":"2024-03-06T07:23:24.385559+0000","last_active":"2024-03-06T07:23:26.972461+0000","last_peered":"2024-03-06T07:23:26.972461+0000","last_clean":"2024-03-06T07:23:26.972461+0000","last_became_active":"2024-03-06T07:23:24.385267+0000","last_became_peered":"2024-03-06T07:23:24.385267+0000","last_unstale":"2024-03-06T07:23:26.972461+0000","last_undegraded":"2024-03-06T07:23:26.972461+0000","last_fullsized":"2024-03-06T07:23:26.972461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:42.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392324+0000","last_change":"2024-03-06T07:23:25.401018+0000","last_active":"2024-03-06T07:23:26.392324+0000","last_peered":"2024-03-06T07:23:26.392324+0000","last_clean":"2024-03-06T07:23:26.392324+0000","last_became_active":"2024-03-06T07:23:25.400711+0000","last_became_peered":"2024-03-06T07:23:25.400711+0000","last_unstale":"2024-03-06T07:23:26.392324+0000","last_undegraded":"2024-03-06T07:23:26.392324+0000","last_fullsized":"2024-03-06T07:23:26.392324+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392672+0000","last_change":"2024-03-06T07:23:24.384292+0000","last_active":"2024-03-06T07:23:26.392672+0000","last_peered":"2024-03-06T07:23:26.392672+0000","last_clean":"2024-03-06T07:23:26.392672+0000","last_became_active":"2024-03-06T07:23:24.383426+0000","last_became_peered":"2024-03-06T07:23:24.383426+0000","last_unstale":"2024-03-06T07:23:26.392672+0000","last_undegraded":"2024-03-06T07:23:26.392672+0000","last_fullsized":"2024-03-06T07:23:26.392672+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:43.016460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471047+0000","last_change":"2024-03-06T07:23:25.397913+0000","last_active":"2024-03-06T07:23:26.471047+0000","last_peered":"2024-03-06T07:23:26.471047+0000","last_clean":"2024-03-06T07:23:26.471047+0000","last_became_active":"2024-03-06T07:23:25.397513+0000","last_became_peered":"2024-03-06T07:23:25.397513+0000","last_unstale":"2024-03-06T07:23:26.471047+0000","last_undegraded":"2024-03-06T07:23:26.471047+0000","last_fullsized":"2024-03-06T07:23:26.471047+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972114+0000","last_change":"2024-03-06T07:23:24.387368+0000","last_active":"2024-03-06T07:23:26.972114+0000","last_peered":"2024-03-06T07:23:26.972114+0000","last_clean":"2024-03-06T07:23:26.972114+0000","last_became_active":"2024-03-06T07:23:24.387094+0000","last_became_peered":"2024-03-06T07:23:24.387094+0000","last_unstale":"2024-03-06T07:23:26.972114+0000","last_undegraded":"2024-03-06T07:23:26.972114+0000","last_fullsized":"2024-03-06T07:23:26.972114+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972605+0000","last_change":"2024-03-06T07:23:25.396196+0000","last_active":"2024-03-06T07:23:26.972605+0000","last_peered":"2024-03-06T07:23:26.972605+0000","last_clean":"2024-03-06T07:23:26.972605+0000","last_became_active":"2024-03-06T07:23:25.395873+0000","last_became_peered":"2024-03-06T07:23:25.395873+0000","last_unstale":"2024-03-06T07:23:26.972605+0000","last_undegraded":"2024-03-06T07:23:26.972605+0000","last_fullsized":"2024-03-06T07:23:26.972605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:00.263605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971801+0000","last_change":"2024-03-06T07:23:24.388038+0000","last_active":"2024-03-06T07:23:26.971801+0000","last_peered":"2024-03-06T07:23:26.971801+0000","last_clean":"2024-03-06T07:23:26.971801+0000","last_became_active":"2024-03-06T07:23:24.387713+0000","last_became_peered":"2024-03-06T07:23:24.387713+0000","last_unstale":"2024-03-06T07:23:26.971801+0000","last_undegraded":"2024-03-06T07:23:26.971801+0000","last_fullsized":"2024-03-06T07:23:26.971801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:47.255810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390305+0000","last_change":"2024-03-06T07:23:25.398157+0000","last_active":"2024-03-06T07:23:26.390305+0000","last_peered":"2024-03-06T07:23:26.390305+0000","last_clean":"2024-03-06T07:23:26.390305+0000","last_became_active":"2024-03-06T07:23:25.397871+0000","last_became_peered":"2024-03-06T07:23:25.397871+0000","last_unstale":"2024-03-06T07:23:26.390305+0000","last_undegraded":"2024-03-06T07:23:26.390305+0000","last_fullsized":"2024-03-06T07:23:26.390305+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:43.322557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470754+0000","last_change":"2024-03-06T07:23:24.385919+0000","last_active":"2024-03-06T07:23:26.470754+0000","last_peered":"2024-03-06T07:23:26.470754+0000","last_clean":"2024-03-06T07:23:26.470754+0000","last_became_active":"2024-03-06T07:23:24.385648+0000","last_became_peered":"2024-03-06T07:23:24.385648+0000","last_unstale":"2024-03-06T07:23:26.470754+0000","last_undegraded":"2024-03-06T07:23:26.470754+0000","last_fullsized":"2024-03-06T07:23:26.470754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392134+0000","last_change":"2024-03-06T07:23:25.401011+0000","last_active":"2024-03-06T07:23:26.392134+0000","last_peered":"2024-03-06T07:23:26.392134+0000","last_clean":"2024-03-06T07:23:26.392134+0000","last_became_active":"2024-03-06T07:23:25.400579+0000","last_became_peered":"2024-03-06T07:23:25.400579+0000","last_unstale":"2024-03-06T07:23:26.392134+0000","last_undegraded":"2024-03-06T07:23:26.392134+0000","last_fullsized":"2024-03-06T07:23:26.392134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:26.923866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391556+0000","last_change":"2024-03-06T07:23:24.390583+0000","last_active":"2024-03-06T07:23:26.391556+0000","last_peered":"2024-03-06T07:23:26.391556+0000","last_clean":"2024-03-06T07:23:26.391556+0000","last_became_active":"2024-03-06T07:23:24.390175+0000","last_became_peered":"2024-03-06T07:23:24.390175+0000","last_unstale":"2024-03-06T07:23:26.391556+0000","last_undegraded":"2024-03-06T07:23:26.391556+0000","last_fullsized":"2024-03-06T07:23:26.391556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972059+0000","last_change":"2024-03-06T07:23:25.394438+0000","last_active":"2024-03-06T07:23:26.972059+0000","last_peered":"2024-03-06T07:23:26.972059+0000","last_clean":"2024-03-06T07:23:26.972059+0000","last_became_active":"2024-03-06T07:23:25.393996+0000","last_became_peered":"2024-03-06T07:23:25.393996+0000","last_unstale":"2024-03-06T07:23:26.972059+0000","last_undegraded":"2024-03-06T07:23:26.972059+0000","last_fullsized":"2024-03-06T07:23:26.972059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:59.801141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971550+0000","last_change":"2024-03-06T07:23:24.384340+0000","last_active":"2024-03-06T07:23:26.971550+0000","last_peered":"2024-03-06T07:23:26.971550+0000","last_clean":"2024-03-06T07:23:26.971550+0000","last_became_active":"2024-03-06T07:23:24.381368+0000","last_became_peered":"2024-03-06T07:23:24.381368+0000","last_unstale":"2024-03-06T07:23:26.971550+0000","last_undegraded":"2024-03-06T07:23:26.971550+0000","last_fullsized":"2024-03-06T07:23:26.971550+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:41.869419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389757+0000","last_change":"2024-03-06T07:23:25.395201+0000","last_active":"2024-03-06T07:23:26.389757+0000","last_peered":"2024-03-06T07:23:26.389757+0000","last_clean":"2024-03-06T07:23:26.389757+0000","last_became_active":"2024-03-06T07:23:25.394691+0000","last_became_peered":"2024-03-06T07:23:25.394691+0000","last_unstale":"2024-03-06T07:23:26.389757+0000","last_undegraded":"2024-03-06T07:23:26.389757+0000","last_fullsized":"2024-03-06T07:23:26.389757+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:43.566376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392764+0000","last_change":"2024-03-06T07:23:24.387279+0000","last_active":"2024-03-06T07:23:26.392764+0000","last_peered":"2024-03-06T07:23:26.392764+0000","last_clean":"2024-03-06T07:23:26.392764+0000","last_became_active":"2024-03-06T07:23:24.387002+0000","last_became_peered":"2024-03-06T07:23:24.387002+0000","last_unstale":"2024-03-06T07:23:26.392764+0000","last_undegraded":"2024-03-06T07:23:26.392764+0000","last_fullsized":"2024-03-06T07:23:26.392764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:48.984688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971872+0000","last_change":"2024-03-06T07:23:25.397113+0000","last_active":"2024-03-06T07:23:26.971872+0000","last_peered":"2024-03-06T07:23:26.971872+0000","last_clean":"2024-03-06T07:23:26.971872+0000","last_became_active":"2024-03-06T07:23:25.396845+0000","last_became_peered":"2024-03-06T07:23:25.396845+0000","last_unstale":"2024-03-06T07:23:26.971872+0000","last_undegraded":"2024-03-06T07:23:26.971872+0000","last_fullsized":"2024-03-06T07:23:26.971872+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390187+0000","last_change":"2024-03-06T07:23:24.385553+0000","last_active":"2024-03-06T07:23:26.390187+0000","last_peered":"2024-03-06T07:23:26.390187+0000","last_clean":"2024-03-06T07:23:26.390187+0000","last_became_active":"2024-03-06T07:23:24.385269+0000","last_became_peered":"2024-03-06T07:23:24.385269+0000","last_unstale":"2024-03-06T07:23:26.390187+0000","last_undegraded":"2024-03-06T07:23:26.390187+0000","last_fullsized":"2024-03-06T07:23:26.390187+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:32.676677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972144+0000","last_change":"2024-03-06T07:23:25.393288+0000","last_active":"2024-03-06T07:23:26.972144+0000","last_peered":"2024-03-06T07:23:26.972144+0000","last_clean":"2024-03-06T07:23:26.972144+0000","last_became_active":"2024-03-06T07:23:25.393025+0000","last_became_peered":"2024-03-06T07:23:25.393025+0000","last_unstale":"2024-03-06T07:23:26.972144+0000","last_undegraded":"2024-03-06T07:23:26.972144+0000","last_fullsized":"2024-03-06T07:23:26.972144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:03.997235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971797+0000","last_change":"2024-03-06T07:23:24.389118+0000","last_active":"2024-03-06T07:23:26.971797+0000","last_peered":"2024-03-06T07:23:26.971797+0000","last_clean":"2024-03-06T07:23:26.971797+0000","last_became_active":"2024-03-06T07:23:24.388765+0000","last_became_peered":"2024-03-06T07:23:24.388765+0000","last_unstale":"2024-03-06T07:23:26.971797+0000","last_undegraded":"2024-03-06T07:23:26.971797+0000","last_fullsized":"2024-03-06T07:23:26.971797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:26.087668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972625+0000","last_change":"2024-03-06T07:23:25.402506+0000","last_active":"2024-03-06T07:23:26.972625+0000","last_peered":"2024-03-06T07:23:26.972625+0000","last_clean":"2024-03-06T07:23:26.972625+0000","last_became_active":"2024-03-06T07:23:25.401547+0000","last_became_peered":"2024-03-06T07:23:25.401547+0000","last_unstale":"2024-03-06T07:23:26.972625+0000","last_undegraded":"2024-03-06T07:23:26.972625+0000","last_fullsized":"2024-03-06T07:23:26.972625+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391738+0000","last_change":"2024-03-06T07:23:24.380046+0000","last_active":"2024-03-06T07:23:26.391738+0000","last_peered":"2024-03-06T07:23:26.391738+0000","last_clean":"2024-03-06T07:23:26.391738+0000","last_became_active":"2024-03-06T07:23:24.379719+0000","last_became_peered":"2024-03-06T07:23:24.379719+0000","last_unstale":"2024-03-06T07:23:26.391738+0000","last_undegraded":"2024-03-06T07:23:26.391738+0000","last_fullsized":"2024-03-06T07:23:26.391738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:42.419365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470920+0000","last_change":"2024-03-06T07:23:25.397907+0000","last_active":"2024-03-06T07:23:26.470920+0000","last_peered":"2024-03-06T07:23:26.470920+0000","last_clean":"2024-03-06T07:23:26.470920+0000","last_became_active":"2024-03-06T07:23:25.397568+0000","last_became_peered":"2024-03-06T07:23:25.397568+0000","last_unstale":"2024-03-06T07:23:26.470920+0000","last_undegraded":"2024-03-06T07:23:26.470920+0000","last_fullsized":"2024-03-06T07:23:26.470920+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972722+0000","last_change":"2024-03-06T07:23:24.386772+0000","last_active":"2024-03-06T07:23:26.972722+0000","last_peered":"2024-03-06T07:23:26.972722+0000","last_clean":"2024-03-06T07:23:26.972722+0000","last_became_active":"2024-03-06T07:23:24.386494+0000","last_became_peered":"2024-03-06T07:23:24.386494+0000","last_unstale":"2024-03-06T07:23:26.972722+0000","last_undegraded":"2024-03-06T07:23:26.972722+0000","last_fullsized":"2024-03-06T07:23:26.972722+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:48:16.389764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391936+0000","last_change":"2024-03-06T07:23:25.388620+0000","last_active":"2024-03-06T07:23:26.391936+0000","last_peered":"2024-03-06T07:23:26.391936+0000","last_clean":"2024-03-06T07:23:26.391936+0000","last_became_active":"2024-03-06T07:23:25.388349+0000","last_became_peered":"2024-03-06T07:23:25.388349+0000","last_unstale":"2024-03-06T07:23:26.391936+0000","last_undegraded":"2024-03-06T07:23:26.391936+0000","last_fullsized":"2024-03-06T07:23:26.391936+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972180+0000","last_change":"2024-03-06T07:23:24.388932+0000","last_active":"2024-03-06T07:23:26.972180+0000","last_peered":"2024-03-06T07:23:26.972180+0000","last_clean":"2024-03-06T07:23:26.972180+0000","last_became_active":"2024-03-06T07:23:24.388656+0000","last_became_peered":"2024-03-06T07:23:24.388656+0000","last_unstale":"2024-03-06T07:23:26.972180+0000","last_undegraded":"2024-03-06T07:23:26.972180+0000","last_fullsized":"2024-03-06T07:23:26.972180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:33.328660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972458+0000","last_change":"2024-03-06T07:23:25.382947+0000","last_active":"2024-03-06T07:23:26.972458+0000","last_peered":"2024-03-06T07:23:26.972458+0000","last_clean":"2024-03-06T07:23:26.972458+0000","last_became_active":"2024-03-06T07:23:25.382627+0000","last_became_peered":"2024-03-06T07:23:25.382627+0000","last_unstale":"2024-03-06T07:23:26.972458+0000","last_undegraded":"2024-03-06T07:23:26.972458+0000","last_fullsized":"2024-03-06T07:23:26.972458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:07.290964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972545+0000","last_change":"2024-03-06T07:23:24.386991+0000","last_active":"2024-03-06T07:23:26.972545+0000","last_peered":"2024-03-06T07:23:26.972545+0000","last_clean":"2024-03-06T07:23:26.972545+0000","last_became_active":"2024-03-06T07:23:24.386673+0000","last_became_peered":"2024-03-06T07:23:24.386673+0000","last_unstale":"2024-03-06T07:23:26.972545+0000","last_undegraded":"2024-03-06T07:23:26.972545+0000","last_fullsized":"2024-03-06T07:23:26.972545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:10.075698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390493+0000","last_change":"2024-03-06T07:23:25.403115+0000","last_active":"2024-03-06T07:23:26.390493+0000","last_peered":"2024-03-06T07:23:26.390493+0000","last_clean":"2024-03-06T07:23:26.390493+0000","last_became_active":"2024-03-06T07:23:25.402845+0000","last_became_peered":"2024-03-06T07:23:25.402845+0000","last_unstale":"2024-03-06T07:23:26.390493+0000","last_undegraded":"2024-03-06T07:23:26.390493+0000","last_fullsized":"2024-03-06T07:23:26.390493+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:44.767746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391244+0000","last_change":"2024-03-06T07:23:24.386565+0000","last_active":"2024-03-06T07:23:26.391244+0000","last_peered":"2024-03-06T07:23:26.391244+0000","last_clean":"2024-03-06T07:23:26.391244+0000","last_became_active":"2024-03-06T07:23:24.386294+0000","last_became_peered":"2024-03-06T07:23:24.386294+0000","last_unstale":"2024-03-06T07:23:26.391244+0000","last_undegraded":"2024-03-06T07:23:26.391244+0000","last_fullsized":"2024-03-06T07:23:26.391244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972772+0000","last_change":"2024-03-06T07:23:25.383321+0000","last_active":"2024-03-06T07:23:26.972772+0000","last_peered":"2024-03-06T07:23:26.972772+0000","last_clean":"2024-03-06T07:23:26.972772+0000","last_became_active":"2024-03-06T07:23:25.383041+0000","last_became_peered":"2024-03-06T07:23:25.383041+0000","last_unstale":"2024-03-06T07:23:26.972772+0000","last_undegraded":"2024-03-06T07:23:26.972772+0000","last_fullsized":"2024-03-06T07:23:26.972772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:51.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391999+0000","last_change":"2024-03-06T07:23:24.384473+0000","last_active":"2024-03-06T07:23:26.391999+0000","last_peered":"2024-03-06T07:23:26.391999+0000","last_clean":"2024-03-06T07:23:26.391999+0000","last_became_active":"2024-03-06T07:23:24.384208+0000","last_became_peered":"2024-03-06T07:23:24.384208+0000","last_unstale":"2024-03-06T07:23:26.391999+0000","last_undegraded":"2024-03-06T07:23:26.391999+0000","last_fullsized":"2024-03-06T07:23:26.391999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:08.142150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392008+0000","last_change":"2024-03-06T07:23:25.388193+0000","last_active":"2024-03-06T07:23:26.392008+0000","last_peered":"2024-03-06T07:23:26.392008+0000","last_clean":"2024-03-06T07:23:26.392008+0000","last_became_active":"2024-03-06T07:23:25.387918+0000","last_became_peered":"2024-03-06T07:23:25.387918+0000","last_unstale":"2024-03-06T07:23:26.392008+0000","last_undegraded":"2024-03-06T07:23:26.392008+0000","last_fullsized":"2024-03-06T07:23:26.392008+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971907+0000","last_change":"2024-03-06T07:23:24.386138+0000","last_active":"2024-03-06T07:23:26.971907+0000","last_peered":"2024-03-06T07:23:26.971907+0000","last_clean":"2024-03-06T07:23:26.971907+0000","last_became_active":"2024-03-06T07:23:24.385638+0000","last_became_peered":"2024-03-06T07:23:24.385638+0000","last_unstale":"2024-03-06T07:23:26.971907+0000","last_undegraded":"2024-03-06T07:23:26.971907+0000","last_fullsized":"2024-03-06T07:23:26.971907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:58.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972489+0000","last_change":"2024-03-06T07:23:25.401883+0000","last_active":"2024-03-06T07:23:26.972489+0000","last_peered":"2024-03-06T07:23:26.972489+0000","last_clean":"2024-03-06T07:23:26.972489+0000","last_became_active":"2024-03-06T07:23:25.400994+0000","last_became_peered":"2024-03-06T07:23:25.400994+0000","last_unstale":"2024-03-06T07:23:26.972489+0000","last_undegraded":"2024-03-06T07:23:26.972489+0000","last_fullsized":"2024-03-06T07:23:26.972489+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470891+0000","last_change":"2024-03-06T07:23:24.387871+0000","last_active":"2024-03-06T07:23:26.470891+0000","last_peered":"2024-03-06T07:23:26.470891+0000","last_clean":"2024-03-06T07:23:26.470891+0000","last_became_active":"2024-03-06T07:23:24.387553+0000","last_became_peered":"2024-03-06T07:23:24.387553+0000","last_unstale":"2024-03-06T07:23:26.470891+0000","last_undegraded":"2024-03-06T07:23:26.470891+0000","last_fullsized":"2024-03-06T07:23:26.470891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389949+0000","last_change":"2024-03-06T07:23:25.386911+0000","last_active":"2024-03-06T07:23:26.389949+0000","last_peered":"2024-03-06T07:23:26.389949+0000","last_clean":"2024-03-06T07:23:26.389949+0000","last_became_active":"2024-03-06T07:23:25.386624+0000","last_became_peered":"2024-03-06T07:23:25.386624+0000","last_unstale":"2024-03-06T07:23:26.389949+0000","last_undegraded":"2024-03-06T07:23:26.389949+0000","last_fullsized":"2024-03-06T07:23:26.389949+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:14.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972815+0000","last_change":"2024-03-06T07:23:24.389648+0000","last_active":"2024-03-06T07:23:26.972815+0000","last_peered":"2024-03-06T07:23:26.972815+0000","last_clean":"2024-03-06T07:23:26.972815+0000","last_became_active":"2024-03-06T07:23:24.389363+0000","last_became_peered":"2024-03-06T07:23:24.389363+0000","last_unstale":"2024-03-06T07:23:26.972815+0000","last_undegraded":"2024-03-06T07:23:26.972815+0000","last_fullsized":"2024-03-06T07:23:26.972815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:21.869658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390199+0000","last_change":"2024-03-06T07:23:25.401602+0000","last_active":"2024-03-06T07:23:26.390199+0000","last_peered":"2024-03-06T07:23:26.390199+0000","last_clean":"2024-03-06T07:23:26.390199+0000","last_became_active":"2024-03-06T07:23:25.401325+0000","last_became_peered":"2024-03-06T07:23:25.401325+0000","last_unstale":"2024-03-06T07:23:26.390199+0000","last_undegraded":"2024-03-06T07:23:26.390199+0000","last_fullsized":"2024-03-06T07:23:26.390199+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:36.825108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972010+0000","last_change":"2024-03-06T07:23:24.391076+0000","last_active":"2024-03-06T07:23:26.972010+0000","last_peered":"2024-03-06T07:23:26.972010+0000","last_clean":"2024-03-06T07:23:26.972010+0000","last_became_active":"2024-03-06T07:23:24.390829+0000","last_became_peered":"2024-03-06T07:23:24.390829+0000","last_unstale":"2024-03-06T07:23:26.972010+0000","last_undegraded":"2024-03-06T07:23:26.972010+0000","last_fullsized":"2024-03-06T07:23:26.972010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:43.498556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971726+0000","last_change":"2024-03-06T07:23:25.387365+0000","last_active":"2024-03-06T07:23:26.971726+0000","last_peered":"2024-03-06T07:23:26.971726+0000","last_clean":"2024-03-06T07:23:26.971726+0000","last_became_active":"2024-03-06T07:23:25.387098+0000","last_became_peered":"2024-03-06T07:23:25.387098+0000","last_unstale":"2024-03-06T07:23:26.971726+0000","last_undegraded":"2024-03-06T07:23:26.971726+0000","last_fullsized":"2024-03-06T07:23:26.971726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:53.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392090+0000","last_change":"2024-03-06T07:23:24.384789+0000","last_active":"2024-03-06T07:23:26.392090+0000","last_peered":"2024-03-06T07:23:26.392090+0000","last_clean":"2024-03-06T07:23:26.392090+0000","last_became_active":"2024-03-06T07:23:24.383845+0000","last_became_peered":"2024-03-06T07:23:24.383845+0000","last_unstale":"2024-03-06T07:23:26.392090+0000","last_undegraded":"2024-03-06T07:23:26.392090+0000","last_fullsized":"2024-03-06T07:23:26.392090+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:48:16.389764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971751+0000","last_change":"2024-03-06T07:23:25.403813+0000","last_active":"2024-03-06T07:23:26.971751+0000","last_peered":"2024-03-06T07:23:26.971751+0000","last_clean":"2024-03-06T07:23:26.971751+0000","last_became_active":"2024-03-06T07:23:25.402357+0000","last_became_peered":"2024-03-06T07:23:25.402357+0000","last_unstale":"2024-03-06T07:23:26.971751+0000","last_undegraded":"2024-03-06T07:23:26.971751+0000","last_fullsized":"2024-03-06T07:23:26.971751+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:06.393074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972352+0000","last_change":"2024-03-06T07:23:24.390357+0000","last_active":"2024-03-06T07:23:26.972352+0000","last_peered":"2024-03-06T07:23:26.972352+0000","last_clean":"2024-03-06T07:23:26.972352+0000","last_became_active":"2024-03-06T07:23:24.390088+0000","last_became_peered":"2024-03-06T07:23:24.390088+0000","last_unstale":"2024-03-06T07:23:26.972352+0000","last_undegraded":"2024-03-06T07:23:26.972352+0000","last_fullsized":"2024-03-06T07:23:26.972352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:03.479567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389738+0000","last_change":"2024-03-06T07:23:25.393297+0000","last_active":"2024-03-06T07:23:26.389738+0000","last_peered":"2024-03-06T07:23:26.389738+0000","last_clean":"2024-03-06T07:23:26.389738+0000","last_became_active":"2024-03-06T07:23:25.392985+0000","last_became_peered":"2024-03-06T07:23:25.392985+0000","last_unstale":"2024-03-06T07:23:26.389738+0000","last_undegraded":"2024-03-06T07:23:26.389738+0000","last_fullsized":"2024-03-06T07:23:26.389738+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:24.052161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971773+0000","last_change":"2024-03-06T07:23:24.384103+0000","last_active":"2024-03-06T07:23:26.971773+0000","last_peered":"2024-03-06T07:23:26.971773+0000","last_clean":"2024-03-06T07:23:26.971773+0000","last_became_active":"2024-03-06T07:23:24.383821+0000","last_became_peered":"2024-03-06T07:23:24.383821+0000","last_unstale":"2024-03-06T07:23:26.971773+0000","last_undegraded":"2024-03-06T07:23:26.971773+0000","last_fullsized":"2024-03-06T07:23:26.971773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:23.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391454+0000","last_change":"2024-03-06T07:23:25.384454+0000","last_active":"2024-03-06T07:23:26.391454+0000","last_peered":"2024-03-06T07:23:26.391454+0000","last_clean":"2024-03-06T07:23:26.391454+0000","last_became_active":"2024-03-06T07:23:25.384193+0000","last_became_peered":"2024-03-06T07:23:25.384193+0000","last_unstale":"2024-03-06T07:23:26.391454+0000","last_undegraded":"2024-03-06T07:23:26.391454+0000","last_fullsized":"2024-03-06T07:23:26.391454+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:27.378633+0000","last_change":"2024-03-06T07:23:24.387350+0000","last_active":"2024-03-06T07:23:27.378633+0000","last_peered":"2024-03-06T07:23:27.378633+0000","last_clean":"2024-03-06T07:23:27.378633+0000","last_became_active":"2024-03-06T07:23:24.386747+0000","last_became_peered":"2024-03-06T07:23:24.386747+0000","last_unstale":"2024-03-06T07:23:27.378633+0000","last_undegraded":"2024-03-06T07:23:27.378633+0000","last_fullsized":"2024-03-06T07:23:27.378633+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:14.030110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972088+0000","last_change":"2024-03-06T07:23:25.397212+0000","last_active":"2024-03-06T07:23:26.972088+0000","last_peered":"2024-03-06T07:23:26.972088+0000","last_clean":"2024-03-06T07:23:26.972088+0000","last_became_active":"2024-03-06T07:23:25.396914+0000","last_became_peered":"2024-03-06T07:23:25.396914+0000","last_unstale":"2024-03-06T07:23:26.972088+0000","last_undegraded":"2024-03-06T07:23:26.972088+0000","last_fullsized":"2024-03-06T07:23:26.972088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:38.499627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392164+0000","last_change":"2024-03-06T07:23:24.387677+0000","last_active":"2024-03-06T07:23:26.392164+0000","last_peered":"2024-03-06T07:23:26.392164+0000","last_clean":"2024-03-06T07:23:26.392164+0000","last_became_active":"2024-03-06T07:23:24.387404+0000","last_became_peered":"2024-03-06T07:23:24.387404+0000","last_unstale":"2024-03-06T07:23:26.392164+0000","last_undegraded":"2024-03-06T07:23:26.392164+0000","last_fullsized":"2024-03-06T07:23:26.392164+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:05:56.063831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470889+0000","last_change":"2024-03-06T07:23:25.404949+0000","last_active":"2024-03-06T07:23:26.470889+0000","last_peered":"2024-03-06T07:23:26.470889+0000","last_clean":"2024-03-06T07:23:26.470889+0000","last_became_active":"2024-03-06T07:23:25.404554+0000","last_became_peered":"2024-03-06T07:23:25.404554+0000","last_unstale":"2024-03-06T07:23:26.470889+0000","last_undegraded":"2024-03-06T07:23:26.470889+0000","last_fullsized":"2024-03-06T07:23:26.470889+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972212+0000","last_change":"2024-03-06T07:23:24.388984+0000","last_active":"2024-03-06T07:23:26.972212+0000","last_peered":"2024-03-06T07:23:26.972212+0000","last_clean":"2024-03-06T07:23:26.972212+0000","last_became_active":"2024-03-06T07:23:24.388670+0000","last_became_peered":"2024-03-06T07:23:24.388670+0000","last_unstale":"2024-03-06T07:23:26.972212+0000","last_undegraded":"2024-03-06T07:23:26.972212+0000","last_fullsized":"2024-03-06T07:23:26.972212+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971636+0000","last_change":"2024-03-06T07:23:25.402955+0000","last_active":"2024-03-06T07:23:26.971636+0000","last_peered":"2024-03-06T07:23:26.971636+0000","last_clean":"2024-03-06T07:23:26.971636+0000","last_became_active":"2024-03-06T07:23:25.402344+0000","last_became_peered":"2024-03-06T07:23:25.402344+0000","last_unstale":"2024-03-06T07:23:26.971636+0000","last_undegraded":"2024-03-06T07:23:26.971636+0000","last_fullsized":"2024-03-06T07:23:26.971636+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:56.228253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392560+0000","last_change":"2024-03-06T07:23:24.389422+0000","last_active":"2024-03-06T07:23:26.392560+0000","last_peered":"2024-03-06T07:23:26.392560+0000","last_clean":"2024-03-06T07:23:26.392560+0000","last_became_active":"2024-03-06T07:23:24.389062+0000","last_became_peered":"2024-03-06T07:23:24.389062+0000","last_unstale":"2024-03-06T07:23:26.392560+0000","last_undegraded":"2024-03-06T07:23:26.392560+0000","last_fullsized":"2024-03-06T07:23:26.392560+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:33.945679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971910+0000","last_change":"2024-03-06T07:23:25.404732+0000","last_active":"2024-03-06T07:23:26.971910+0000","last_peered":"2024-03-06T07:23:26.971910+0000","last_clean":"2024-03-06T07:23:26.971910+0000","last_became_active":"2024-03-06T07:23:25.402434+0000","last_became_peered":"2024-03-06T07:23:25.402434+0000","last_unstale":"2024-03-06T07:23:26.971910+0000","last_undegraded":"2024-03-06T07:23:26.971910+0000","last_fullsized":"2024-03-06T07:23:26.971910+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:48.022820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972232+0000","last_change":"2024-03-06T07:23:24.388946+0000","last_active":"2024-03-06T07:23:26.972232+0000","last_peered":"2024-03-06T07:23:26.972232+0000","last_clean":"2024-03-06T07:23:26.972232+0000","last_became_active":"2024-03-06T07:23:24.388661+0000","last_became_peered":"2024-03-06T07:23:24.388661+0000","last_unstale":"2024-03-06T07:23:26.972232+0000","last_undegraded":"2024-03-06T07:23:26.972232+0000","last_fullsized":"2024-03-06T07:23:26.972232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:22:31.252766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971964+0000","last_change":"2024-03-06T07:23:25.402288+0000","last_active":"2024-03-06T07:23:26.971964+0000","last_peered":"2024-03-06T07:23:26.971964+0000","last_clean":"2024-03-06T07:23:26.971964+0000","last_became_active":"2024-03-06T07:23:25.402025+0000","last_became_peered":"2024-03-06T07:23:25.402025+0000","last_unstale":"2024-03-06T07:23:26.971964+0000","last_undegraded":"2024-03-06T07:23:26.971964+0000","last_fullsized":"2024-03-06T07:23:26.971964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:53.103351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392195+0000","last_change":"2024-03-06T07:23:24.389412+0000","last_active":"2024-03-06T07:23:26.392195+0000","last_peered":"2024-03-06T07:23:26.392195+0000","last_clean":"2024-03-06T07:23:26.392195+0000","last_became_active":"2024-03-06T07:23:24.389133+0000","last_became_peered":"2024-03-06T07:23:24.389133+0000","last_unstale":"2024-03-06T07:23:26.392195+0000","last_undegraded":"2024-03-06T07:23:26.392195+0000","last_fullsized":"2024-03-06T07:23:26.392195+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:23.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471238+0000","last_change":"2024-03-06T07:23:25.399585+0000","last_active":"2024-03-06T07:23:26.471238+0000","last_peered":"2024-03-06T07:23:26.471238+0000","last_clean":"2024-03-06T07:23:26.471238+0000","last_became_active":"2024-03-06T07:23:25.399175+0000","last_became_peered":"2024-03-06T07:23:25.399175+0000","last_unstale":"2024-03-06T07:23:26.471238+0000","last_undegraded":"2024-03-06T07:23:26.471238+0000","last_fullsized":"2024-03-06T07:23:26.471238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972443+0000","last_change":"2024-03-06T07:23:24.387674+0000","last_active":"2024-03-06T07:23:26.972443+0000","last_peered":"2024-03-06T07:23:26.972443+0000","last_clean":"2024-03-06T07:23:26.972443+0000","last_became_active":"2024-03-06T07:23:24.387409+0000","last_became_peered":"2024-03-06T07:23:24.387409+0000","last_unstale":"2024-03-06T07:23:26.972443+0000","last_undegraded":"2024-03-06T07:23:26.972443+0000","last_fullsized":"2024-03-06T07:23:26.972443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:27.192275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470818+0000","last_change":"2024-03-06T07:23:25.397682+0000","last_active":"2024-03-06T07:23:26.470818+0000","last_peered":"2024-03-06T07:23:26.470818+0000","last_clean":"2024-03-06T07:23:26.470818+0000","last_became_active":"2024-03-06T07:23:25.397325+0000","last_became_peered":"2024-03-06T07:23:25.397325+0000","last_unstale":"2024-03-06T07:23:26.470818+0000","last_undegraded":"2024-03-06T07:23:26.470818+0000","last_fullsized":"2024-03-06T07:23:26.470818+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972360+0000","last_change":"2024-03-06T07:23:24.389928+0000","last_active":"2024-03-06T07:23:26.972360+0000","last_peered":"2024-03-06T07:23:26.972360+0000","last_clean":"2024-03-06T07:23:26.972360+0000","last_became_active":"2024-03-06T07:23:24.389634+0000","last_became_peered":"2024-03-06T07:23:24.389634+0000","last_unstale":"2024-03-06T07:23:26.972360+0000","last_undegraded":"2024-03-06T07:23:26.972360+0000","last_fullsized":"2024-03-06T07:23:26.972360+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:54.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971565+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:23:26.971565+0000","last_peered":"2024-03-06T07:23:26.971565+0000","last_clean":"2024-03-06T07:23:26.971565+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:23:26.971565+0000","last_undegraded":"2024-03-06T07:23:26.971565+0000","last_fullsized":"2024-03-06T07:23:26.971565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":1162123,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":580503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":571595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":578379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":572177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":578321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":571629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":1170357,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:28.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.545+0000 7f33a9bc7640 1 Processor -- start 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.545+0000 7f33a9bc7640 1 -- start start 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a9bc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 0x7f33a406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a9bc7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a406b800 con 0x7f33a406b3a0 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a9bc7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a40a70f0 con 0x7f33a41536c0 2024-03-06T07:23:28.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a9bc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4150a20 con 0x7f33a406afc0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f339affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f33a406b3a0 0x7f33a4152fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46532/0 (socket says 172.21.15.17:46532) 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f339affd640 1 -- 172.21.15.17:0/1328250439 learned_addr learned my addr 172.21.15.17:0/1328250439 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a37fe640 1 --2- 172.21.15.17:0/1328250439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 0x7f33a406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906643523 0 0) 0x7f33a406b800 con 0x7f33a406b3a0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3390003520 con 0x7f33a406b3a0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798769164 0 0) 0x7f33a40a70f0 con 0x7f33a41536c0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a406b800 con 0x7f33a41536c0 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a37fe640 1 -- 172.21.15.17:0/1328250439 >> v1:172.21.15.163:6789/0 conn(0x7f33a41536c0 legacy=0x7f33a415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a37fe640 1 -- 172.21.15.17:0/1328250439 >> v1:172.21.15.17:6789/0 conn(0x7f33a406b3a0 legacy=0x7f33a4152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.546+0000 7f33a37fe640 1 -- 172.21.15.17:0/1328250439 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a41567f0 con 0x7f33a406afc0 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.547+0000 7f33a37fe640 1 --2- 172.21.15.17:0/1328250439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 0x7f33a406f100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33940097d0 tx=0x7f339402b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac550fddd7b7ee22 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394031770 con 0x7f33a406afc0 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394037070 con 0x7f33a406afc0 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a2ffd640 1 -- 172.21.15.17:0/1328250439 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394031b10 con 0x7f33a406afc0 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 -- 172.21.15.17:0/1328250439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 msgr2=0x7f33a406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/1328250439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 0x7f33a406f100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33940097d0 tx=0x7f339402b810 comp rx=0 tx=0).stop 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 -- 172.21.15.17:0/1328250439 shutdown_connections 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/1328250439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a406afc0 0x7f33a406f100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 -- 172.21.15.17:0/1328250439 >> 172.21.15.17:0/1328250439 conn(0x7f33a40c4290 msgr2=0x7f33a40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 -- 172.21.15.17:0/1328250439 shutdown_connections 2024-03-06T07:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.548+0000 7f33a9bc7640 1 -- 172.21.15.17:0/1328250439 wait complete. 2024-03-06T07:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.549+0000 7f33a9bc7640 1 Processor -- start 2024-03-06T07:23:28.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 -- start start 2024-03-06T07:23:28.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 0x7f33a41eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a41536c0 0x7f33a41eb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a4156db0 con 0x7f33a406afc0 2024-03-06T07:23:28.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a41565a0 con 0x7f33a406b3a0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a9bc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a40c7360 con 0x7f33a41536c0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60114/0 (socket says 172.21.15.17:60114) 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.550+0000 7f33a37fe640 1 -- 172.21.15.17:0/3494067733 learned_addr learned my addr 172.21.15.17:0/3494067733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f339affd640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 0x7f33a41eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a3fff640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a41536c0 0x7f33a41eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 -- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a41536c0 msgr2=0x7f33a41eb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a41536c0 0x7f33a41eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 -- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 msgr2=0x7f33a41eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 0x7f33a41eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 -- 172.21.15.17:0/3494067733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a41ebe50 con 0x7f33a406afc0 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f339affd640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 0x7f33a41eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:28.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a37fe640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f339402b850 tx=0x7f33940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394032600 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33940327a0 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a41eaed0 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394032b70 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.551+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a4073190 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.552+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4067ad0 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.552+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3394031070 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.553+0000 7f33a0ff9640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 0x7f338406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.553+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33940abb50 con 0x7f33a406afc0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.553+0000 7f339affd640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 0x7f338406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.554+0000 7f339affd640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 0x7f338406d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f338c007fd0 tx=0x7f338c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:28.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.557+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33a4067ad0 con 0x7f33a406afc0 2024-03-06T07:23:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.823+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33a4075300 con 0x7f338406b500 2024-03-06T07:23:28.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.829+0000 7f33a0ff9640 1 -- 172.21.15.17:0/3494067733 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252158 (secure 0 0 0) 0x7f33a4075300 con 0x7f338406b500 2024-03-06T07:23:28.836 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 msgr2=0x7f338406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 0x7f338406d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f338c007fd0 tx=0x7f338c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 msgr2=0x7f33a40c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f339402b850 tx=0x7f33940046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 shutdown_connections 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f338406b500 0x7f338406d9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33a41536c0 0x7f33a41eb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33a406b3a0 0x7f33a41eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 --2- 172.21.15.17:0/3494067733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33a406afc0 0x7f33a40c55f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 >> 172.21.15.17:0/3494067733 conn(0x7f33a40c4290 msgr2=0x7f33a40a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 shutdown_connections 2024-03-06T07:23:28.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.833+0000 7f33a9bc7640 1 -- 172.21.15.17:0/3494067733 wait complete. 2024-03-06T07:23:28.848 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-03-06T07:23:27.386176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254756,"kb_used_data":9456,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719964,"statfs":{"total":773094113280,"available":772833243136,"internally_reserved":0,"allocated":9682944,"data_stored":5785084,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022285"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390365+0000","last_change":"2024-03-06T07:23:25.392757+0000","last_active":"2024-03-06T07:23:26.390365+0000","last_peered":"2024-03-06T07:23:26.390365+0000","last_clean":"2024-03-06T07:23:26.390365+0000","last_became_active":"2024-03-06T07:23:25.392461+0000","last_became_peered":"2024-03-06T07:23:25.392461+0000","last_unstale":"2024-03-06T07:23:26.390365+0000","last_undegraded":"2024-03-06T07:23:26.390365+0000","last_fullsized":"2024-03-06T07:23:26.390365+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:53.201629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391785+0000","last_change":"2024-03-06T07:23:25.391927+0000","last_active":"2024-03-06T07:23:26.391785+0000","last_peered":"2024-03-06T07:23:26.391785+0000","last_clean":"2024-03-06T07:23:26.391785+0000","last_became_active":"2024-03-06T07:23:25.391243+0000","last_became_peered":"2024-03-06T07:23:25.391243+0000","last_unstale":"2024-03-06T07:23:26.391785+0000","last_undegraded":"2024-03-06T07:23:26.391785+0000","last_fullsized":"2024-03-06T07:23:26.391785+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389845+0000","last_change":"2024-03-06T07:23:25.399576+0000","last_active":"2024-03-06T07:23:26.389845+0000","last_peered":"2024-03-06T07:23:26.389845+0000","last_clean":"2024-03-06T07:23:26.389845+0000","last_became_active":"2024-03-06T07:23:25.399320+0000","last_became_peered":"2024-03-06T07:23:25.399320+0000","last_unstale":"2024-03-06T07:23:26.389845+0000","last_undegraded":"2024-03-06T07:23:26.389845+0000","last_fullsized":"2024-03-06T07:23:26.389845+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:26.612372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972471+0000","last_change":"2024-03-06T07:23:25.388327+0000","last_active":"2024-03-06T07:23:26.972471+0000","last_peered":"2024-03-06T07:23:26.972471+0000","last_clean":"2024-03-06T07:23:26.972471+0000","last_became_active":"2024-03-06T07:23:25.387934+0000","last_became_peered":"2024-03-06T07:23:25.387934+0000","last_unstale":"2024-03-06T07:23:26.972471+0000","last_undegraded":"2024-03-06T07:23:26.972471+0000","last_fullsized":"2024-03-06T07:23:26.972471+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:18.840228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972354+0000","last_change":"2024-03-06T07:23:25.402105+0000","last_active":"2024-03-06T07:23:26.972354+0000","last_peered":"2024-03-06T07:23:26.972354+0000","last_clean":"2024-03-06T07:23:26.972354+0000","last_became_active":"2024-03-06T07:23:25.401273+0000","last_became_peered":"2024-03-06T07:23:25.401273+0000","last_unstale":"2024-03-06T07:23:26.972354+0000","last_undegraded":"2024-03-06T07:23:26.972354+0000","last_fullsized":"2024-03-06T07:23:26.972354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391795+0000","last_change":"2024-03-06T07:23:25.393894+0000","last_active":"2024-03-06T07:23:26.391795+0000","last_peered":"2024-03-06T07:23:26.391795+0000","last_clean":"2024-03-06T07:23:26.391795+0000","last_became_active":"2024-03-06T07:23:25.393624+0000","last_became_peered":"2024-03-06T07:23:25.393624+0000","last_unstale":"2024-03-06T07:23:26.391795+0000","last_undegraded":"2024-03-06T07:23:26.391795+0000","last_fullsized":"2024-03-06T07:23:26.391795+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391855+0000","last_change":"2024-03-06T07:23:25.402036+0000","last_active":"2024-03-06T07:23:26.391855+0000","last_peered":"2024-03-06T07:23:26.391855+0000","last_clean":"2024-03-06T07:23:26.391855+0000","last_became_active":"2024-03-06T07:23:25.401786+0000","last_became_peered":"2024-03-06T07:23:25.401786+0000","last_unstale":"2024-03-06T07:23:26.391855+0000","last_undegraded":"2024-03-06T07:23:26.391855+0000","last_fullsized":"2024-03-06T07:23:26.391855+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972114+0000","last_change":"2024-03-06T07:23:25.406026+0000","last_active":"2024-03-06T07:23:26.972114+0000","last_peered":"2024-03-06T07:23:26.972114+0000","last_clean":"2024-03-06T07:23:26.972114+0000","last_became_active":"2024-03-06T07:23:25.404996+0000","last_became_peered":"2024-03-06T07:23:25.404996+0000","last_unstale":"2024-03-06T07:23:26.972114+0000","last_undegraded":"2024-03-06T07:23:26.972114+0000","last_fullsized":"2024-03-06T07:23:26.972114+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:46.674666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392397+0000","last_change":"2024-03-06T07:23:25.389370+0000","last_active":"2024-03-06T07:23:26.392397+0000","last_peered":"2024-03-06T07:23:26.392397+0000","last_clean":"2024-03-06T07:23:26.392397+0000","last_became_active":"2024-03-06T07:23:25.388802+0000","last_became_peered":"2024-03-06T07:23:25.388802+0000","last_unstale":"2024-03-06T07:23:26.392397+0000","last_undegraded":"2024-03-06T07:23:26.392397+0000","last_fullsized":"2024-03-06T07:23:26.392397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:28.292286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972730+0000","last_change":"2024-03-06T07:23:25.400328+0000","last_active":"2024-03-06T07:23:26.972730+0000","last_peered":"2024-03-06T07:23:26.972730+0000","last_clean":"2024-03-06T07:23:26.972730+0000","last_became_active":"2024-03-06T07:23:25.400063+0000","last_became_peered":"2024-03-06T07:23:25.400063+0000","last_unstale":"2024-03-06T07:23:26.972730+0000","last_undegraded":"2024-03-06T07:23:26.972730+0000","last_fullsized":"2024-03-06T07:23:26.972730+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471039+0000","last_change":"2024-03-06T07:23:25.397761+0000","last_active":"2024-03-06T07:23:26.471039+0000","last_peered":"2024-03-06T07:23:26.471039+0000","last_clean":"2024-03-06T07:23:26.471039+0000","last_became_active":"2024-03-06T07:23:25.397341+0000","last_became_peered":"2024-03-06T07:23:25.397341+0000","last_unstale":"2024-03-06T07:23:26.471039+0000","last_undegraded":"2024-03-06T07:23:26.471039+0000","last_fullsized":"2024-03-06T07:23:26.471039+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392766+0000","last_change":"2024-03-06T07:23:25.387742+0000","last_active":"2024-03-06T07:23:26.392766+0000","last_peered":"2024-03-06T07:23:26.392766+0000","last_clean":"2024-03-06T07:23:26.392766+0000","last_became_active":"2024-03-06T07:23:25.387236+0000","last_became_peered":"2024-03-06T07:23:25.387236+0000","last_unstale":"2024-03-06T07:23:26.392766+0000","last_undegraded":"2024-03-06T07:23:26.392766+0000","last_fullsized":"2024-03-06T07:23:26.392766+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971762+0000","last_change":"2024-03-06T07:23:25.398485+0000","last_active":"2024-03-06T07:23:26.971762+0000","last_peered":"2024-03-06T07:23:26.971762+0000","last_clean":"2024-03-06T07:23:26.971762+0000","last_became_active":"2024-03-06T07:23:25.398197+0000","last_became_peered":"2024-03-06T07:23:25.398197+0000","last_unstale":"2024-03-06T07:23:26.971762+0000","last_undegraded":"2024-03-06T07:23:26.971762+0000","last_fullsized":"2024-03-06T07:23:26.971762+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:25.808803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971851+0000","last_change":"2024-03-06T07:23:25.392696+0000","last_active":"2024-03-06T07:23:26.971851+0000","last_peered":"2024-03-06T07:23:26.971851+0000","last_clean":"2024-03-06T07:23:26.971851+0000","last_became_active":"2024-03-06T07:23:25.392399+0000","last_became_peered":"2024-03-06T07:23:25.392399+0000","last_unstale":"2024-03-06T07:23:26.971851+0000","last_undegraded":"2024-03-06T07:23:26.971851+0000","last_fullsized":"2024-03-06T07:23:26.971851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:20:16.216161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971692+0000","last_change":"2024-03-06T07:23:25.398526+0000","last_active":"2024-03-06T07:23:26.971692+0000","last_peered":"2024-03-06T07:23:26.971692+0000","last_clean":"2024-03-06T07:23:26.971692+0000","last_became_active":"2024-03-06T07:23:25.398228+0000","last_became_peered":"2024-03-06T07:23:25.398228+0000","last_unstale":"2024-03-06T07:23:26.971692+0000","last_undegraded":"2024-03-06T07:23:26.971692+0000","last_fullsized":"2024-03-06T07:23:26.971692+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:10:28.292286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971711+0000","last_change":"2024-03-06T07:23:25.390767+0000","last_active":"2024-03-06T07:23:26.971711+0000","last_peered":"2024-03-06T07:23:26.971711+0000","last_clean":"2024-03-06T07:23:26.971711+0000","last_became_active":"2024-03-06T07:23:25.390510+0000","last_became_peered":"2024-03-06T07:23:25.390510+0000","last_unstale":"2024-03-06T07:23:26.971711+0000","last_undegraded":"2024-03-06T07:23:26.971711+0000","last_fullsized":"2024-03-06T07:23:26.971711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:01:34.576371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390382+0000","last_change":"2024-03-06T07:23:25.398842+0000","last_active":"2024-03-06T07:23:26.390382+0000","last_peered":"2024-03-06T07:23:26.390382+0000","last_clean":"2024-03-06T07:23:26.390382+0000","last_became_active":"2024-03-06T07:23:25.398511+0000","last_became_peered":"2024-03-06T07:23:25.398511+0000","last_unstale":"2024-03-06T07:23:26.390382+0000","last_undegraded":"2024-03-06T07:23:26.390382+0000","last_fullsized":"2024-03-06T07:23:26.390382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:35:38.035908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392004+0000","last_change":"2024-03-06T07:23:25.391708+0000","last_active":"2024-03-06T07:23:26.392004+0000","last_peered":"2024-03-06T07:23:26.392004+0000","last_clean":"2024-03-06T07:23:26.392004+0000","last_became_active":"2024-03-06T07:23:25.391427+0000","last_became_peered":"2024-03-06T07:23:25.391427+0000","last_unstale":"2024-03-06T07:23:26.392004+0000","last_undegraded":"2024-03-06T07:23:26.392004+0000","last_fullsized":"2024-03-06T07:23:26.392004+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390212+0000","last_change":"2024-03-06T07:23:25.400482+0000","last_active":"2024-03-06T07:23:26.390212+0000","last_peered":"2024-03-06T07:23:26.390212+0000","last_clean":"2024-03-06T07:23:26.390212+0000","last_became_active":"2024-03-06T07:23:25.400204+0000","last_became_peered":"2024-03-06T07:23:25.400204+0000","last_unstale":"2024-03-06T07:23:26.390212+0000","last_undegraded":"2024-03-06T07:23:26.390212+0000","last_fullsized":"2024-03-06T07:23:26.390212+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:08.683727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392789+0000","last_change":"2024-03-06T07:23:25.394192+0000","last_active":"2024-03-06T07:23:26.392789+0000","last_peered":"2024-03-06T07:23:26.392789+0000","last_clean":"2024-03-06T07:23:26.392789+0000","last_became_active":"2024-03-06T07:23:25.393910+0000","last_became_peered":"2024-03-06T07:23:25.393910+0000","last_unstale":"2024-03-06T07:23:26.392789+0000","last_undegraded":"2024-03-06T07:23:26.392789+0000","last_fullsized":"2024-03-06T07:23:26.392789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:26.923866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972707+0000","last_change":"2024-03-06T07:23:25.400576+0000","last_active":"2024-03-06T07:23:26.972707+0000","last_peered":"2024-03-06T07:23:26.972707+0000","last_clean":"2024-03-06T07:23:26.972707+0000","last_became_active":"2024-03-06T07:23:25.400306+0000","last_became_peered":"2024-03-06T07:23:25.400306+0000","last_unstale":"2024-03-06T07:23:26.972707+0000","last_undegraded":"2024-03-06T07:23:26.972707+0000","last_fullsized":"2024-03-06T07:23:26.972707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:18.977243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971772+0000","last_change":"2024-03-06T07:23:25.403369+0000","last_active":"2024-03-06T07:23:26.971772+0000","last_peered":"2024-03-06T07:23:26.971772+0000","last_clean":"2024-03-06T07:23:26.971772+0000","last_became_active":"2024-03-06T07:23:25.402913+0000","last_became_peered":"2024-03-06T07:23:25.402913+0000","last_unstale":"2024-03-06T07:23:26.971772+0000","last_undegraded":"2024-03-06T07:23:26.971772+0000","last_fullsized":"2024-03-06T07:23:26.971772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:50:14.637534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389828+0000","last_change":"2024-03-06T07:23:25.402260+0000","last_active":"2024-03-06T07:23:26.389828+0000","last_peered":"2024-03-06T07:23:26.389828+0000","last_clean":"2024-03-06T07:23:26.389828+0000","last_became_active":"2024-03-06T07:23:25.402004+0000","last_became_peered":"2024-03-06T07:23:25.402004+0000","last_unstale":"2024-03-06T07:23:26.389828+0000","last_undegraded":"2024-03-06T07:23:26.389828+0000","last_fullsized":"2024-03-06T07:23:26.389828+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:10.654230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971595+0000","last_change":"2024-03-06T07:23:25.403146+0000","last_active":"2024-03-06T07:23:26.971595+0000","last_peered":"2024-03-06T07:23:26.971595+0000","last_clean":"2024-03-06T07:23:26.971595+0000","last_became_active":"2024-03-06T07:23:25.402880+0000","last_became_peered":"2024-03-06T07:23:25.402880+0000","last_unstale":"2024-03-06T07:23:26.971595+0000","last_undegraded":"2024-03-06T07:23:26.971595+0000","last_fullsized":"2024-03-06T07:23:26.971595+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:33:10.821669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392734+0000","last_change":"2024-03-06T07:23:25.401581+0000","last_active":"2024-03-06T07:23:26.392734+0000","last_peered":"2024-03-06T07:23:26.392734+0000","last_clean":"2024-03-06T07:23:26.392734+0000","last_became_active":"2024-03-06T07:23:25.401250+0000","last_became_peered":"2024-03-06T07:23:25.401250+0000","last_unstale":"2024-03-06T07:23:26.392734+0000","last_undegraded":"2024-03-06T07:23:26.392734+0000","last_fullsized":"2024-03-06T07:23:26.392734+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.393063+0000","last_change":"2024-03-06T07:23:25.380731+0000","last_active":"2024-03-06T07:23:26.393063+0000","last_peered":"2024-03-06T07:23:26.393063+0000","last_clean":"2024-03-06T07:23:26.393063+0000","last_became_active":"2024-03-06T07:23:25.380442+0000","last_became_peered":"2024-03-06T07:23:25.380442+0000","last_unstale":"2024-03-06T07:23:26.393063+0000","last_undegraded":"2024-03-06T07:23:26.393063+0000","last_fullsized":"2024-03-06T07:23:26.393063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:57.628480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471090+0000","last_change":"2024-03-06T07:23:25.400133+0000","last_active":"2024-03-06T07:23:26.471090+0000","last_peered":"2024-03-06T07:23:26.471090+0000","last_clean":"2024-03-06T07:23:26.471090+0000","last_became_active":"2024-03-06T07:23:25.399237+0000","last_became_peered":"2024-03-06T07:23:25.399237+0000","last_unstale":"2024-03-06T07:23:26.471090+0000","last_undegraded":"2024-03-06T07:23:26.471090+0000","last_fullsized":"2024-03-06T07:23:26.471090+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:42.874694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391565+0000","last_change":"2024-03-06T07:23:25.398808+0000","last_active":"2024-03-06T07:23:26.391565+0000","last_peered":"2024-03-06T07:23:26.391565+0000","last_clean":"2024-03-06T07:23:26.391565+0000","last_became_active":"2024-03-06T07:23:25.398485+0000","last_became_peered":"2024-03-06T07:23:25.398485+0000","last_unstale":"2024-03-06T07:23:26.391565+0000","last_undegraded":"2024-03-06T07:23:26.391565+0000","last_fullsized":"2024-03-06T07:23:26.391565+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972772+0000","last_change":"2024-03-06T07:23:25.392067+0000","last_active":"2024-03-06T07:23:26.972772+0000","last_peered":"2024-03-06T07:23:26.972772+0000","last_clean":"2024-03-06T07:23:26.972772+0000","last_became_active":"2024-03-06T07:23:25.391753+0000","last_became_peered":"2024-03-06T07:23:25.391753+0000","last_unstale":"2024-03-06T07:23:26.972772+0000","last_undegraded":"2024-03-06T07:23:26.972772+0000","last_fullsized":"2024-03-06T07:23:26.972772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:32:31.604627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392171+0000","last_change":"2024-03-06T07:23:25.399088+0000","last_active":"2024-03-06T07:23:26.392171+0000","last_peered":"2024-03-06T07:23:26.392171+0000","last_clean":"2024-03-06T07:23:26.392171+0000","last_became_active":"2024-03-06T07:23:25.398486+0000","last_became_peered":"2024-03-06T07:23:25.398486+0000","last_unstale":"2024-03-06T07:23:26.392171+0000","last_undegraded":"2024-03-06T07:23:26.392171+0000","last_fullsized":"2024-03-06T07:23:26.392171+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392286+0000","last_change":"2024-03-06T07:23:25.385062+0000","last_active":"2024-03-06T07:23:26.392286+0000","last_peered":"2024-03-06T07:23:26.392286+0000","last_clean":"2024-03-06T07:23:26.392286+0000","last_became_active":"2024-03-06T07:23:25.384785+0000","last_became_peered":"2024-03-06T07:23:25.384785+0000","last_unstale":"2024-03-06T07:23:26.392286+0000","last_undegraded":"2024-03-06T07:23:26.392286+0000","last_fullsized":"2024-03-06T07:23:26.392286+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:57.628480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972252+0000","last_change":"2024-03-06T07:23:25.382968+0000","last_active":"2024-03-06T07:23:26.972252+0000","last_peered":"2024-03-06T07:23:26.972252+0000","last_clean":"2024-03-06T07:23:26.972252+0000","last_became_active":"2024-03-06T07:23:25.382667+0000","last_became_peered":"2024-03-06T07:23:25.382667+0000","last_unstale":"2024-03-06T07:23:26.972252+0000","last_undegraded":"2024-03-06T07:23:26.972252+0000","last_fullsized":"2024-03-06T07:23:26.972252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:41:57.987167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389880+0000","last_change":"2024-03-06T07:23:25.394981+0000","last_active":"2024-03-06T07:23:26.389880+0000","last_peered":"2024-03-06T07:23:26.389880+0000","last_clean":"2024-03-06T07:23:26.389880+0000","last_became_active":"2024-03-06T07:23:25.394349+0000","last_became_peered":"2024-03-06T07:23:25.394349+0000","last_unstale":"2024-03-06T07:23:26.389880+0000","last_undegraded":"2024-03-06T07:23:26.389880+0000","last_fullsized":"2024-03-06T07:23:26.389880+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:16.831201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"95'4","reported_seq":18,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972155+0000","last_change":"2024-03-06T07:23:24.388032+0000","last_active":"2024-03-06T07:23:26.972155+0000","last_peered":"2024-03-06T07:23:26.972155+0000","last_clean":"2024-03-06T07:23:26.972155+0000","last_became_active":"2024-03-06T07:23:24.387749+0000","last_became_peered":"2024-03-06T07:23:24.387749+0000","last_unstale":"2024-03-06T07:23:26.972155+0000","last_undegraded":"2024-03-06T07:23:26.972155+0000","last_fullsized":"2024-03-06T07:23:26.972155+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:25.918590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972466+0000","last_change":"2024-03-06T07:23:25.405419+0000","last_active":"2024-03-06T07:23:26.972466+0000","last_peered":"2024-03-06T07:23:26.972466+0000","last_clean":"2024-03-06T07:23:26.972466+0000","last_became_active":"2024-03-06T07:23:25.405156+0000","last_became_peered":"2024-03-06T07:23:25.405156+0000","last_unstale":"2024-03-06T07:23:26.972466+0000","last_undegraded":"2024-03-06T07:23:26.972466+0000","last_fullsized":"2024-03-06T07:23:26.972466+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:58:33.300913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972461+0000","last_change":"2024-03-06T07:23:24.385559+0000","last_active":"2024-03-06T07:23:26.972461+0000","last_peered":"2024-03-06T07:23:26.972461+0000","last_clean":"2024-03-06T07:23:26.972461+0000","last_became_active":"2024-03-06T07:23:24.385267+0000","last_became_peered":"2024-03-06T07:23:24.385267+0000","last_unstale":"2024-03-06T07:23:26.972461+0000","last_undegraded":"2024-03-06T07:23:26.972461+0000","last_fullsized":"2024-03-06T07:23:26.972461+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:04:42.270334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392324+0000","last_change":"2024-03-06T07:23:25.401018+0000","last_active":"2024-03-06T07:23:26.392324+0000","last_peered":"2024-03-06T07:23:26.392324+0000","last_clean":"2024-03-06T07:23:26.392324+0000","last_became_active":"2024-03-06T07:23:25.400711+0000","last_became_peered":"2024-03-06T07:23:25.400711+0000","last_unstale":"2024-03-06T07:23:26.392324+0000","last_undegraded":"2024-03-06T07:23:26.392324+0000","last_fullsized":"2024-03-06T07:23:26.392324+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392672+0000","last_change":"2024-03-06T07:23:24.384292+0000","last_active":"2024-03-06T07:23:26.392672+0000","last_peered":"2024-03-06T07:23:26.392672+0000","last_clean":"2024-03-06T07:23:26.392672+0000","last_became_active":"2024-03-06T07:23:24.383426+0000","last_became_peered":"2024-03-06T07:23:24.383426+0000","last_unstale":"2024-03-06T07:23:26.392672+0000","last_undegraded":"2024-03-06T07:23:26.392672+0000","last_fullsized":"2024-03-06T07:23:26.392672+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:43.016460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471047+0000","last_change":"2024-03-06T07:23:25.397913+0000","last_active":"2024-03-06T07:23:26.471047+0000","last_peered":"2024-03-06T07:23:26.471047+0000","last_clean":"2024-03-06T07:23:26.471047+0000","last_became_active":"2024-03-06T07:23:25.397513+0000","last_became_peered":"2024-03-06T07:23:25.397513+0000","last_unstale":"2024-03-06T07:23:26.471047+0000","last_undegraded":"2024-03-06T07:23:26.471047+0000","last_fullsized":"2024-03-06T07:23:26.471047+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972114+0000","last_change":"2024-03-06T07:23:24.387368+0000","last_active":"2024-03-06T07:23:26.972114+0000","last_peered":"2024-03-06T07:23:26.972114+0000","last_clean":"2024-03-06T07:23:26.972114+0000","last_became_active":"2024-03-06T07:23:24.387094+0000","last_became_peered":"2024-03-06T07:23:24.387094+0000","last_unstale":"2024-03-06T07:23:26.972114+0000","last_undegraded":"2024-03-06T07:23:26.972114+0000","last_fullsized":"2024-03-06T07:23:26.972114+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972605+0000","last_change":"2024-03-06T07:23:25.396196+0000","last_active":"2024-03-06T07:23:26.972605+0000","last_peered":"2024-03-06T07:23:26.972605+0000","last_clean":"2024-03-06T07:23:26.972605+0000","last_became_active":"2024-03-06T07:23:25.395873+0000","last_became_peered":"2024-03-06T07:23:25.395873+0000","last_unstale":"2024-03-06T07:23:26.972605+0000","last_undegraded":"2024-03-06T07:23:26.972605+0000","last_fullsized":"2024-03-06T07:23:26.972605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:00.263605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971801+0000","last_change":"2024-03-06T07:23:24.388038+0000","last_active":"2024-03-06T07:23:26.971801+0000","last_peered":"2024-03-06T07:23:26.971801+0000","last_clean":"2024-03-06T07:23:26.971801+0000","last_became_active":"2024-03-06T07:23:24.387713+0000","last_became_peered":"2024-03-06T07:23:24.387713+0000","last_unstale":"2024-03-06T07:23:26.971801+0000","last_undegraded":"2024-03-06T07:23:26.971801+0000","last_fullsized":"2024-03-06T07:23:26.971801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:47.255810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390305+0000","last_change":"2024-03-06T07:23:25.398157+0000","last_active":"2024-03-06T07:23:26.390305+0000","last_peered":"2024-03-06T07:23:26.390305+0000","last_clean":"2024-03-06T07:23:26.390305+0000","last_became_active":"2024-03-06T07:23:25.397871+0000","last_became_peered":"2024-03-06T07:23:25.397871+0000","last_unstale":"2024-03-06T07:23:26.390305+0000","last_undegraded":"2024-03-06T07:23:26.390305+0000","last_fullsized":"2024-03-06T07:23:26.390305+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:43.322557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470754+0000","last_change":"2024-03-06T07:23:24.385919+0000","last_active":"2024-03-06T07:23:26.470754+0000","last_peered":"2024-03-06T07:23:26.470754+0000","last_clean":"2024-03-06T07:23:26.470754+0000","last_became_active":"2024-03-06T07:23:24.385648+0000","last_became_peered":"2024-03-06T07:23:24.385648+0000","last_unstale":"2024-03-06T07:23:26.470754+0000","last_undegraded":"2024-03-06T07:23:26.470754+0000","last_fullsized":"2024-03-06T07:23:26.470754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392134+0000","last_change":"2024-03-06T07:23:25.401011+0000","last_active":"2024-03-06T07:23:26.392134+0000","last_peered":"2024-03-06T07:23:26.392134+0000","last_clean":"2024-03-06T07:23:26.392134+0000","last_became_active":"2024-03-06T07:23:25.400579+0000","last_became_peered":"2024-03-06T07:23:25.400579+0000","last_unstale":"2024-03-06T07:23:26.392134+0000","last_undegraded":"2024-03-06T07:23:26.392134+0000","last_fullsized":"2024-03-06T07:23:26.392134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:26.923866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391556+0000","last_change":"2024-03-06T07:23:24.390583+0000","last_active":"2024-03-06T07:23:26.391556+0000","last_peered":"2024-03-06T07:23:26.391556+0000","last_clean":"2024-03-06T07:23:26.391556+0000","last_became_active":"2024-03-06T07:23:24.390175+0000","last_became_peered":"2024-03-06T07:23:24.390175+0000","last_unstale":"2024-03-06T07:23:26.391556+0000","last_undegraded":"2024-03-06T07:23:26.391556+0000","last_fullsized":"2024-03-06T07:23:26.391556+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972059+0000","last_change":"2024-03-06T07:23:25.394438+0000","last_active":"2024-03-06T07:23:26.972059+0000","last_peered":"2024-03-06T07:23:26.972059+0000","last_clean":"2024-03-06T07:23:26.972059+0000","last_became_active":"2024-03-06T07:23:25.393996+0000","last_became_peered":"2024-03-06T07:23:25.393996+0000","last_unstale":"2024-03-06T07:23:26.972059+0000","last_undegraded":"2024-03-06T07:23:26.972059+0000","last_fullsized":"2024-03-06T07:23:26.972059+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:59.801141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971550+0000","last_change":"2024-03-06T07:23:24.384340+0000","last_active":"2024-03-06T07:23:26.971550+0000","last_peered":"2024-03-06T07:23:26.971550+0000","last_clean":"2024-03-06T07:23:26.971550+0000","last_became_active":"2024-03-06T07:23:24.381368+0000","last_became_peered":"2024-03-06T07:23:24.381368+0000","last_unstale":"2024-03-06T07:23:26.971550+0000","last_undegraded":"2024-03-06T07:23:26.971550+0000","last_fullsized":"2024-03-06T07:23:26.971550+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:14:41.869419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389757+0000","last_change":"2024-03-06T07:23:25.395201+0000","last_active":"2024-03-06T07:23:26.389757+0000","last_peered":"2024-03-06T07:23:26.389757+0000","last_clean":"2024-03-06T07:23:26.389757+0000","last_became_active":"2024-03-06T07:23:25.394691+0000","last_became_peered":"2024-03-06T07:23:25.394691+0000","last_unstale":"2024-03-06T07:23:26.389757+0000","last_undegraded":"2024-03-06T07:23:26.389757+0000","last_fullsized":"2024-03-06T07:23:26.389757+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:43.566376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392764+0000","last_change":"2024-03-06T07:23:24.387279+0000","last_active":"2024-03-06T07:23:26.392764+0000","last_peered":"2024-03-06T07:23:26.392764+0000","last_clean":"2024-03-06T07:23:26.392764+0000","last_became_active":"2024-03-06T07:23:24.387002+0000","last_became_peered":"2024-03-06T07:23:24.387002+0000","last_unstale":"2024-03-06T07:23:26.392764+0000","last_undegraded":"2024-03-06T07:23:26.392764+0000","last_fullsized":"2024-03-06T07:23:26.392764+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:17:48.984688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971872+0000","last_change":"2024-03-06T07:23:25.397113+0000","last_active":"2024-03-06T07:23:26.971872+0000","last_peered":"2024-03-06T07:23:26.971872+0000","last_clean":"2024-03-06T07:23:26.971872+0000","last_became_active":"2024-03-06T07:23:25.396845+0000","last_became_peered":"2024-03-06T07:23:25.396845+0000","last_unstale":"2024-03-06T07:23:26.971872+0000","last_undegraded":"2024-03-06T07:23:26.971872+0000","last_fullsized":"2024-03-06T07:23:26.971872+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390187+0000","last_change":"2024-03-06T07:23:24.385553+0000","last_active":"2024-03-06T07:23:26.390187+0000","last_peered":"2024-03-06T07:23:26.390187+0000","last_clean":"2024-03-06T07:23:26.390187+0000","last_became_active":"2024-03-06T07:23:24.385269+0000","last_became_peered":"2024-03-06T07:23:24.385269+0000","last_unstale":"2024-03-06T07:23:26.390187+0000","last_undegraded":"2024-03-06T07:23:26.390187+0000","last_fullsized":"2024-03-06T07:23:26.390187+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:56:32.676677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972144+0000","last_change":"2024-03-06T07:23:25.393288+0000","last_active":"2024-03-06T07:23:26.972144+0000","last_peered":"2024-03-06T07:23:26.972144+0000","last_clean":"2024-03-06T07:23:26.972144+0000","last_became_active":"2024-03-06T07:23:25.393025+0000","last_became_peered":"2024-03-06T07:23:25.393025+0000","last_unstale":"2024-03-06T07:23:26.972144+0000","last_undegraded":"2024-03-06T07:23:26.972144+0000","last_fullsized":"2024-03-06T07:23:26.972144+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:03.997235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"95'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971797+0000","last_change":"2024-03-06T07:23:24.389118+0000","last_active":"2024-03-06T07:23:26.971797+0000","last_peered":"2024-03-06T07:23:26.971797+0000","last_clean":"2024-03-06T07:23:26.971797+0000","last_became_active":"2024-03-06T07:23:24.388765+0000","last_became_peered":"2024-03-06T07:23:24.388765+0000","last_unstale":"2024-03-06T07:23:26.971797+0000","last_undegraded":"2024-03-06T07:23:26.971797+0000","last_fullsized":"2024-03-06T07:23:26.971797+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:26.087668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972625+0000","last_change":"2024-03-06T07:23:25.402506+0000","last_active":"2024-03-06T07:23:26.972625+0000","last_peered":"2024-03-06T07:23:26.972625+0000","last_clean":"2024-03-06T07:23:26.972625+0000","last_became_active":"2024-03-06T07:23:25.401547+0000","last_became_peered":"2024-03-06T07:23:25.401547+0000","last_unstale":"2024-03-06T07:23:26.972625+0000","last_undegraded":"2024-03-06T07:23:26.972625+0000","last_fullsized":"2024-03-06T07:23:26.972625+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:58:27.092943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"95'6","reported_seq":20,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391738+0000","last_change":"2024-03-06T07:23:24.380046+0000","last_active":"2024-03-06T07:23:26.391738+0000","last_peered":"2024-03-06T07:23:26.391738+0000","last_clean":"2024-03-06T07:23:26.391738+0000","last_became_active":"2024-03-06T07:23:24.379719+0000","last_became_peered":"2024-03-06T07:23:24.379719+0000","last_unstale":"2024-03-06T07:23:26.391738+0000","last_undegraded":"2024-03-06T07:23:26.391738+0000","last_fullsized":"2024-03-06T07:23:26.391738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:42.419365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470920+0000","last_change":"2024-03-06T07:23:25.397907+0000","last_active":"2024-03-06T07:23:26.470920+0000","last_peered":"2024-03-06T07:23:26.470920+0000","last_clean":"2024-03-06T07:23:26.470920+0000","last_became_active":"2024-03-06T07:23:25.397568+0000","last_became_peered":"2024-03-06T07:23:25.397568+0000","last_unstale":"2024-03-06T07:23:26.470920+0000","last_undegraded":"2024-03-06T07:23:26.470920+0000","last_fullsized":"2024-03-06T07:23:26.470920+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972722+0000","last_change":"2024-03-06T07:23:24.386772+0000","last_active":"2024-03-06T07:23:26.972722+0000","last_peered":"2024-03-06T07:23:26.972722+0000","last_clean":"2024-03-06T07:23:26.972722+0000","last_became_active":"2024-03-06T07:23:24.386494+0000","last_became_peered":"2024-03-06T07:23:24.386494+0000","last_unstale":"2024-03-06T07:23:26.972722+0000","last_undegraded":"2024-03-06T07:23:26.972722+0000","last_fullsized":"2024-03-06T07:23:26.972722+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:48:16.389764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391936+0000","last_change":"2024-03-06T07:23:25.388620+0000","last_active":"2024-03-06T07:23:26.391936+0000","last_peered":"2024-03-06T07:23:26.391936+0000","last_clean":"2024-03-06T07:23:26.391936+0000","last_became_active":"2024-03-06T07:23:25.388349+0000","last_became_peered":"2024-03-06T07:23:25.388349+0000","last_unstale":"2024-03-06T07:23:26.391936+0000","last_undegraded":"2024-03-06T07:23:26.391936+0000","last_fullsized":"2024-03-06T07:23:26.391936+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:59.392548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972180+0000","last_change":"2024-03-06T07:23:24.388932+0000","last_active":"2024-03-06T07:23:26.972180+0000","last_peered":"2024-03-06T07:23:26.972180+0000","last_clean":"2024-03-06T07:23:26.972180+0000","last_became_active":"2024-03-06T07:23:24.388656+0000","last_became_peered":"2024-03-06T07:23:24.388656+0000","last_unstale":"2024-03-06T07:23:26.972180+0000","last_undegraded":"2024-03-06T07:23:26.972180+0000","last_fullsized":"2024-03-06T07:23:26.972180+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:33.328660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972458+0000","last_change":"2024-03-06T07:23:25.382947+0000","last_active":"2024-03-06T07:23:26.972458+0000","last_peered":"2024-03-06T07:23:26.972458+0000","last_clean":"2024-03-06T07:23:26.972458+0000","last_became_active":"2024-03-06T07:23:25.382627+0000","last_became_peered":"2024-03-06T07:23:25.382627+0000","last_unstale":"2024-03-06T07:23:26.972458+0000","last_undegraded":"2024-03-06T07:23:26.972458+0000","last_fullsized":"2024-03-06T07:23:26.972458+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:07.290964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972545+0000","last_change":"2024-03-06T07:23:24.386991+0000","last_active":"2024-03-06T07:23:26.972545+0000","last_peered":"2024-03-06T07:23:26.972545+0000","last_clean":"2024-03-06T07:23:26.972545+0000","last_became_active":"2024-03-06T07:23:24.386673+0000","last_became_peered":"2024-03-06T07:23:24.386673+0000","last_unstale":"2024-03-06T07:23:26.972545+0000","last_undegraded":"2024-03-06T07:23:26.972545+0000","last_fullsized":"2024-03-06T07:23:26.972545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:10.075698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390493+0000","last_change":"2024-03-06T07:23:25.403115+0000","last_active":"2024-03-06T07:23:26.390493+0000","last_peered":"2024-03-06T07:23:26.390493+0000","last_clean":"2024-03-06T07:23:26.390493+0000","last_became_active":"2024-03-06T07:23:25.402845+0000","last_became_peered":"2024-03-06T07:23:25.402845+0000","last_unstale":"2024-03-06T07:23:26.390493+0000","last_undegraded":"2024-03-06T07:23:26.390493+0000","last_fullsized":"2024-03-06T07:23:26.390493+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:44.767746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"95'8","reported_seq":22,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391244+0000","last_change":"2024-03-06T07:23:24.386565+0000","last_active":"2024-03-06T07:23:26.391244+0000","last_peered":"2024-03-06T07:23:26.391244+0000","last_clean":"2024-03-06T07:23:26.391244+0000","last_became_active":"2024-03-06T07:23:24.386294+0000","last_became_peered":"2024-03-06T07:23:24.386294+0000","last_unstale":"2024-03-06T07:23:26.391244+0000","last_undegraded":"2024-03-06T07:23:26.391244+0000","last_fullsized":"2024-03-06T07:23:26.391244+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972772+0000","last_change":"2024-03-06T07:23:25.383321+0000","last_active":"2024-03-06T07:23:26.972772+0000","last_peered":"2024-03-06T07:23:26.972772+0000","last_clean":"2024-03-06T07:23:26.972772+0000","last_became_active":"2024-03-06T07:23:25.383041+0000","last_became_peered":"2024-03-06T07:23:25.383041+0000","last_unstale":"2024-03-06T07:23:26.972772+0000","last_undegraded":"2024-03-06T07:23:26.972772+0000","last_fullsized":"2024-03-06T07:23:26.972772+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:58:51.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391999+0000","last_change":"2024-03-06T07:23:24.384473+0000","last_active":"2024-03-06T07:23:26.391999+0000","last_peered":"2024-03-06T07:23:26.391999+0000","last_clean":"2024-03-06T07:23:26.391999+0000","last_became_active":"2024-03-06T07:23:24.384208+0000","last_became_peered":"2024-03-06T07:23:24.384208+0000","last_unstale":"2024-03-06T07:23:26.391999+0000","last_undegraded":"2024-03-06T07:23:26.391999+0000","last_fullsized":"2024-03-06T07:23:26.391999+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:08.142150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392008+0000","last_change":"2024-03-06T07:23:25.388193+0000","last_active":"2024-03-06T07:23:26.392008+0000","last_peered":"2024-03-06T07:23:26.392008+0000","last_clean":"2024-03-06T07:23:26.392008+0000","last_became_active":"2024-03-06T07:23:25.387918+0000","last_became_peered":"2024-03-06T07:23:25.387918+0000","last_unstale":"2024-03-06T07:23:26.392008+0000","last_undegraded":"2024-03-06T07:23:26.392008+0000","last_fullsized":"2024-03-06T07:23:26.392008+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"95'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971907+0000","last_change":"2024-03-06T07:23:24.386138+0000","last_active":"2024-03-06T07:23:26.971907+0000","last_peered":"2024-03-06T07:23:26.971907+0000","last_clean":"2024-03-06T07:23:26.971907+0000","last_became_active":"2024-03-06T07:23:24.385638+0000","last_became_peered":"2024-03-06T07:23:24.385638+0000","last_unstale":"2024-03-06T07:23:26.971907+0000","last_undegraded":"2024-03-06T07:23:26.971907+0000","last_fullsized":"2024-03-06T07:23:26.971907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:58.387272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972489+0000","last_change":"2024-03-06T07:23:25.401883+0000","last_active":"2024-03-06T07:23:26.972489+0000","last_peered":"2024-03-06T07:23:26.972489+0000","last_clean":"2024-03-06T07:23:26.972489+0000","last_became_active":"2024-03-06T07:23:25.400994+0000","last_became_peered":"2024-03-06T07:23:25.400994+0000","last_unstale":"2024-03-06T07:23:26.972489+0000","last_undegraded":"2024-03-06T07:23:26.972489+0000","last_fullsized":"2024-03-06T07:23:26.972489+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470891+0000","last_change":"2024-03-06T07:23:24.387871+0000","last_active":"2024-03-06T07:23:26.470891+0000","last_peered":"2024-03-06T07:23:26.470891+0000","last_clean":"2024-03-06T07:23:26.470891+0000","last_became_active":"2024-03-06T07:23:24.387553+0000","last_became_peered":"2024-03-06T07:23:24.387553+0000","last_unstale":"2024-03-06T07:23:26.470891+0000","last_undegraded":"2024-03-06T07:23:26.470891+0000","last_fullsized":"2024-03-06T07:23:26.470891+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:38.200084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389949+0000","last_change":"2024-03-06T07:23:25.386911+0000","last_active":"2024-03-06T07:23:26.389949+0000","last_peered":"2024-03-06T07:23:26.389949+0000","last_clean":"2024-03-06T07:23:26.389949+0000","last_became_active":"2024-03-06T07:23:25.386624+0000","last_became_peered":"2024-03-06T07:23:25.386624+0000","last_unstale":"2024-03-06T07:23:26.389949+0000","last_undegraded":"2024-03-06T07:23:26.389949+0000","last_fullsized":"2024-03-06T07:23:26.389949+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:14.582861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972815+0000","last_change":"2024-03-06T07:23:24.389648+0000","last_active":"2024-03-06T07:23:26.972815+0000","last_peered":"2024-03-06T07:23:26.972815+0000","last_clean":"2024-03-06T07:23:26.972815+0000","last_became_active":"2024-03-06T07:23:24.389363+0000","last_became_peered":"2024-03-06T07:23:24.389363+0000","last_unstale":"2024-03-06T07:23:26.972815+0000","last_undegraded":"2024-03-06T07:23:26.972815+0000","last_fullsized":"2024-03-06T07:23:26.972815+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:21.869658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.390199+0000","last_change":"2024-03-06T07:23:25.401602+0000","last_active":"2024-03-06T07:23:26.390199+0000","last_peered":"2024-03-06T07:23:26.390199+0000","last_clean":"2024-03-06T07:23:26.390199+0000","last_became_active":"2024-03-06T07:23:25.401325+0000","last_became_peered":"2024-03-06T07:23:25.401325+0000","last_unstale":"2024-03-06T07:23:26.390199+0000","last_undegraded":"2024-03-06T07:23:26.390199+0000","last_fullsized":"2024-03-06T07:23:26.390199+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:36.825108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972010+0000","last_change":"2024-03-06T07:23:24.391076+0000","last_active":"2024-03-06T07:23:26.972010+0000","last_peered":"2024-03-06T07:23:26.972010+0000","last_clean":"2024-03-06T07:23:26.972010+0000","last_became_active":"2024-03-06T07:23:24.390829+0000","last_became_peered":"2024-03-06T07:23:24.390829+0000","last_unstale":"2024-03-06T07:23:26.972010+0000","last_undegraded":"2024-03-06T07:23:26.972010+0000","last_fullsized":"2024-03-06T07:23:26.972010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:43.498556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971726+0000","last_change":"2024-03-06T07:23:25.387365+0000","last_active":"2024-03-06T07:23:26.971726+0000","last_peered":"2024-03-06T07:23:26.971726+0000","last_clean":"2024-03-06T07:23:26.971726+0000","last_became_active":"2024-03-06T07:23:25.387098+0000","last_became_peered":"2024-03-06T07:23:25.387098+0000","last_unstale":"2024-03-06T07:23:26.971726+0000","last_undegraded":"2024-03-06T07:23:26.971726+0000","last_fullsized":"2024-03-06T07:23:26.971726+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:53.715816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392090+0000","last_change":"2024-03-06T07:23:24.384789+0000","last_active":"2024-03-06T07:23:26.392090+0000","last_peered":"2024-03-06T07:23:26.392090+0000","last_clean":"2024-03-06T07:23:26.392090+0000","last_became_active":"2024-03-06T07:23:24.383845+0000","last_became_peered":"2024-03-06T07:23:24.383845+0000","last_unstale":"2024-03-06T07:23:26.392090+0000","last_undegraded":"2024-03-06T07:23:26.392090+0000","last_fullsized":"2024-03-06T07:23:26.392090+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:48:16.389764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971751+0000","last_change":"2024-03-06T07:23:25.403813+0000","last_active":"2024-03-06T07:23:26.971751+0000","last_peered":"2024-03-06T07:23:26.971751+0000","last_clean":"2024-03-06T07:23:26.971751+0000","last_became_active":"2024-03-06T07:23:25.402357+0000","last_became_peered":"2024-03-06T07:23:25.402357+0000","last_unstale":"2024-03-06T07:23:26.971751+0000","last_undegraded":"2024-03-06T07:23:26.971751+0000","last_fullsized":"2024-03-06T07:23:26.971751+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:06.393074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972352+0000","last_change":"2024-03-06T07:23:24.390357+0000","last_active":"2024-03-06T07:23:26.972352+0000","last_peered":"2024-03-06T07:23:26.972352+0000","last_clean":"2024-03-06T07:23:26.972352+0000","last_became_active":"2024-03-06T07:23:24.390088+0000","last_became_peered":"2024-03-06T07:23:24.390088+0000","last_unstale":"2024-03-06T07:23:26.972352+0000","last_undegraded":"2024-03-06T07:23:26.972352+0000","last_fullsized":"2024-03-06T07:23:26.972352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:03.479567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.389738+0000","last_change":"2024-03-06T07:23:25.393297+0000","last_active":"2024-03-06T07:23:26.389738+0000","last_peered":"2024-03-06T07:23:26.389738+0000","last_clean":"2024-03-06T07:23:26.389738+0000","last_became_active":"2024-03-06T07:23:25.392985+0000","last_became_peered":"2024-03-06T07:23:25.392985+0000","last_unstale":"2024-03-06T07:23:26.389738+0000","last_undegraded":"2024-03-06T07:23:26.389738+0000","last_fullsized":"2024-03-06T07:23:26.389738+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:24.052161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971773+0000","last_change":"2024-03-06T07:23:24.384103+0000","last_active":"2024-03-06T07:23:26.971773+0000","last_peered":"2024-03-06T07:23:26.971773+0000","last_clean":"2024-03-06T07:23:26.971773+0000","last_became_active":"2024-03-06T07:23:24.383821+0000","last_became_peered":"2024-03-06T07:23:24.383821+0000","last_unstale":"2024-03-06T07:23:26.971773+0000","last_undegraded":"2024-03-06T07:23:26.971773+0000","last_fullsized":"2024-03-06T07:23:26.971773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:23.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.391454+0000","last_change":"2024-03-06T07:23:25.384454+0000","last_active":"2024-03-06T07:23:26.391454+0000","last_peered":"2024-03-06T07:23:26.391454+0000","last_clean":"2024-03-06T07:23:26.391454+0000","last_became_active":"2024-03-06T07:23:25.384193+0000","last_became_peered":"2024-03-06T07:23:25.384193+0000","last_unstale":"2024-03-06T07:23:26.391454+0000","last_undegraded":"2024-03-06T07:23:26.391454+0000","last_fullsized":"2024-03-06T07:23:26.391454+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:08:48.261086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"96'1","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:27.378633+0000","last_change":"2024-03-06T07:23:24.387350+0000","last_active":"2024-03-06T07:23:27.378633+0000","last_peered":"2024-03-06T07:23:27.378633+0000","last_clean":"2024-03-06T07:23:27.378633+0000","last_became_active":"2024-03-06T07:23:24.386747+0000","last_became_peered":"2024-03-06T07:23:24.386747+0000","last_unstale":"2024-03-06T07:23:27.378633+0000","last_undegraded":"2024-03-06T07:23:27.378633+0000","last_fullsized":"2024-03-06T07:23:27.378633+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:14.030110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972088+0000","last_change":"2024-03-06T07:23:25.397212+0000","last_active":"2024-03-06T07:23:26.972088+0000","last_peered":"2024-03-06T07:23:26.972088+0000","last_clean":"2024-03-06T07:23:26.972088+0000","last_became_active":"2024-03-06T07:23:25.396914+0000","last_became_peered":"2024-03-06T07:23:25.396914+0000","last_unstale":"2024-03-06T07:23:26.972088+0000","last_undegraded":"2024-03-06T07:23:26.972088+0000","last_fullsized":"2024-03-06T07:23:26.972088+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:38.499627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392164+0000","last_change":"2024-03-06T07:23:24.387677+0000","last_active":"2024-03-06T07:23:26.392164+0000","last_peered":"2024-03-06T07:23:26.392164+0000","last_clean":"2024-03-06T07:23:26.392164+0000","last_became_active":"2024-03-06T07:23:24.387404+0000","last_became_peered":"2024-03-06T07:23:24.387404+0000","last_unstale":"2024-03-06T07:23:26.392164+0000","last_undegraded":"2024-03-06T07:23:26.392164+0000","last_fullsized":"2024-03-06T07:23:26.392164+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:05:56.063831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470889+0000","last_change":"2024-03-06T07:23:25.404949+0000","last_active":"2024-03-06T07:23:26.470889+0000","last_peered":"2024-03-06T07:23:26.470889+0000","last_clean":"2024-03-06T07:23:26.470889+0000","last_became_active":"2024-03-06T07:23:25.404554+0000","last_became_peered":"2024-03-06T07:23:25.404554+0000","last_unstale":"2024-03-06T07:23:26.470889+0000","last_undegraded":"2024-03-06T07:23:26.470889+0000","last_fullsized":"2024-03-06T07:23:26.470889+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:02:50.455270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972212+0000","last_change":"2024-03-06T07:23:24.388984+0000","last_active":"2024-03-06T07:23:26.972212+0000","last_peered":"2024-03-06T07:23:26.972212+0000","last_clean":"2024-03-06T07:23:26.972212+0000","last_became_active":"2024-03-06T07:23:24.388670+0000","last_became_peered":"2024-03-06T07:23:24.388670+0000","last_unstale":"2024-03-06T07:23:26.972212+0000","last_undegraded":"2024-03-06T07:23:26.972212+0000","last_fullsized":"2024-03-06T07:23:26.972212+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:01:38.130160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971636+0000","last_change":"2024-03-06T07:23:25.402955+0000","last_active":"2024-03-06T07:23:26.971636+0000","last_peered":"2024-03-06T07:23:26.971636+0000","last_clean":"2024-03-06T07:23:26.971636+0000","last_became_active":"2024-03-06T07:23:25.402344+0000","last_became_peered":"2024-03-06T07:23:25.402344+0000","last_unstale":"2024-03-06T07:23:26.971636+0000","last_undegraded":"2024-03-06T07:23:26.971636+0000","last_fullsized":"2024-03-06T07:23:26.971636+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:56.228253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392560+0000","last_change":"2024-03-06T07:23:24.389422+0000","last_active":"2024-03-06T07:23:26.392560+0000","last_peered":"2024-03-06T07:23:26.392560+0000","last_clean":"2024-03-06T07:23:26.392560+0000","last_became_active":"2024-03-06T07:23:24.389062+0000","last_became_peered":"2024-03-06T07:23:24.389062+0000","last_unstale":"2024-03-06T07:23:26.392560+0000","last_undegraded":"2024-03-06T07:23:26.392560+0000","last_fullsized":"2024-03-06T07:23:26.392560+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:33.945679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971910+0000","last_change":"2024-03-06T07:23:25.404732+0000","last_active":"2024-03-06T07:23:26.971910+0000","last_peered":"2024-03-06T07:23:26.971910+0000","last_clean":"2024-03-06T07:23:26.971910+0000","last_became_active":"2024-03-06T07:23:25.402434+0000","last_became_peered":"2024-03-06T07:23:25.402434+0000","last_unstale":"2024-03-06T07:23:26.971910+0000","last_undegraded":"2024-03-06T07:23:26.971910+0000","last_fullsized":"2024-03-06T07:23:26.971910+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:48.022820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972232+0000","last_change":"2024-03-06T07:23:24.388946+0000","last_active":"2024-03-06T07:23:26.972232+0000","last_peered":"2024-03-06T07:23:26.972232+0000","last_clean":"2024-03-06T07:23:26.972232+0000","last_became_active":"2024-03-06T07:23:24.388661+0000","last_became_peered":"2024-03-06T07:23:24.388661+0000","last_unstale":"2024-03-06T07:23:26.972232+0000","last_undegraded":"2024-03-06T07:23:26.972232+0000","last_fullsized":"2024-03-06T07:23:26.972232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:22:31.252766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971964+0000","last_change":"2024-03-06T07:23:25.402288+0000","last_active":"2024-03-06T07:23:26.971964+0000","last_peered":"2024-03-06T07:23:26.971964+0000","last_clean":"2024-03-06T07:23:26.971964+0000","last_became_active":"2024-03-06T07:23:25.402025+0000","last_became_peered":"2024-03-06T07:23:25.402025+0000","last_unstale":"2024-03-06T07:23:26.971964+0000","last_undegraded":"2024-03-06T07:23:26.971964+0000","last_fullsized":"2024-03-06T07:23:26.971964+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:53:53.103351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.392195+0000","last_change":"2024-03-06T07:23:24.389412+0000","last_active":"2024-03-06T07:23:26.392195+0000","last_peered":"2024-03-06T07:23:26.392195+0000","last_clean":"2024-03-06T07:23:26.392195+0000","last_became_active":"2024-03-06T07:23:24.389133+0000","last_became_peered":"2024-03-06T07:23:24.389133+0000","last_unstale":"2024-03-06T07:23:26.392195+0000","last_undegraded":"2024-03-06T07:23:26.392195+0000","last_fullsized":"2024-03-06T07:23:26.392195+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:03:23.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.471238+0000","last_change":"2024-03-06T07:23:25.399585+0000","last_active":"2024-03-06T07:23:26.471238+0000","last_peered":"2024-03-06T07:23:26.471238+0000","last_clean":"2024-03-06T07:23:26.471238+0000","last_became_active":"2024-03-06T07:23:25.399175+0000","last_became_peered":"2024-03-06T07:23:25.399175+0000","last_unstale":"2024-03-06T07:23:26.471238+0000","last_undegraded":"2024-03-06T07:23:26.471238+0000","last_fullsized":"2024-03-06T07:23:26.471238+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:34.333935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972443+0000","last_change":"2024-03-06T07:23:24.387674+0000","last_active":"2024-03-06T07:23:26.972443+0000","last_peered":"2024-03-06T07:23:26.972443+0000","last_clean":"2024-03-06T07:23:26.972443+0000","last_became_active":"2024-03-06T07:23:24.387409+0000","last_became_peered":"2024-03-06T07:23:24.387409+0000","last_unstale":"2024-03-06T07:23:26.972443+0000","last_undegraded":"2024-03-06T07:23:26.972443+0000","last_fullsized":"2024-03-06T07:23:26.972443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:27.192275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.470818+0000","last_change":"2024-03-06T07:23:25.397682+0000","last_active":"2024-03-06T07:23:26.470818+0000","last_peered":"2024-03-06T07:23:26.470818+0000","last_clean":"2024-03-06T07:23:26.470818+0000","last_became_active":"2024-03-06T07:23:25.397325+0000","last_became_peered":"2024-03-06T07:23:25.397325+0000","last_unstale":"2024-03-06T07:23:26.470818+0000","last_undegraded":"2024-03-06T07:23:26.470818+0000","last_fullsized":"2024-03-06T07:23:26.470818+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:24.363635+0000","last_clean_scrub_stamp":"2024-03-06T07:23:24.363635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:44.165594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"95'2","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.972360+0000","last_change":"2024-03-06T07:23:24.389928+0000","last_active":"2024-03-06T07:23:26.972360+0000","last_peered":"2024-03-06T07:23:26.972360+0000","last_clean":"2024-03-06T07:23:26.972360+0000","last_became_active":"2024-03-06T07:23:24.389634+0000","last_became_peered":"2024-03-06T07:23:24.389634+0000","last_unstale":"2024-03-06T07:23:26.972360+0000","last_undegraded":"2024-03-06T07:23:26.972360+0000","last_fullsized":"2024-03-06T07:23:26.972360+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:23.358359+0000","last_clean_scrub_stamp":"2024-03-06T07:23:23.358359+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:54.211094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":312,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-03-06T07:23:26.971565+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:23:26.971565+0000","last_peered":"2024-03-06T07:23:26.971565+0000","last_clean":"2024-03-06T07:23:26.971565+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:23:26.971565+0000","last_undegraded":"2024-03-06T07:23:26.971565+0000","last_fullsized":"2024-03-06T07:23:26.971565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":1162123,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":580503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":571595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":578379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":572177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1196032,"data_stored":578321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1183744,"data_stored":571629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":1170357,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:28.855 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:23:28.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:23:28.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.995+0000 7f5076be2640 1 Processor -- start 2024-03-06T07:23:28.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.995+0000 7f5076be2640 1 -- start start 2024-03-06T07:23:28.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.996+0000 7f5076be2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50700a8290 0x7f50701539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.996+0000 7f5076be2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c69a0 0x7f5070072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.996+0000 7f5076be2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700c62a0 con 0x7f50700c4740 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.996+0000 7f5076be2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5070153f00 con 0x7f50700a8290 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.996+0000 7f5076be2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700c4c00 con 0x7f50700c69a0 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f506ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50700c4740 0x7f50700701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46538/0 (socket says 172.21.15.17:46538) 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507497e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50700a8290 0x7f50701539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f506ffff640 1 -- 172.21.15.17:0/1072443165 learned_addr learned my addr 172.21.15.17:0/1072443165 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507517f640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c69a0 0x7f5070072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507497e640 1 -- 172.21.15.17:0/1072443165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c69a0 msgr2=0x7f5070072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507497e640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c69a0 0x7f5070072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507497e640 1 -- 172.21.15.17:0/1072443165 >> v1:172.21.15.17:6789/0 conn(0x7f50700c4740 legacy=0x7f50700701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f507497e640 1 -- 172.21.15.17:0/1072443165 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700c4ee0 con 0x7f50700a8290 2024-03-06T07:23:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.997+0000 7f506f7fe640 1 -- 172.21.15.17:0/1072443165 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289257699 0 0) 0x7f50700c62a0 con 0x7f50700c4740 2024-03-06T07:23:29.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f507497e640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700a8290 0x7f50701539c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5068009d90 tx=0x7f506802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bddb372a0ea26e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f506f7fe640 1 -- 172.21.15.17:0/1072443165 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068039070 con 0x7f50700a8290 2024-03-06T07:23:29.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f506f7fe640 1 -- 172.21.15.17:0/1072443165 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506802cb30 con 0x7f50700a8290 2024-03-06T07:23:29.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f506f7fe640 1 -- 172.21.15.17:0/1072443165 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068034440 con 0x7f50700a8290 2024-03-06T07:23:29.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 -- 172.21.15.17:0/1072443165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700a8290 msgr2=0x7f50701539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700a8290 0x7f50701539c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f5068009d90 tx=0x7f506802bd80 comp rx=0 tx=0).stop 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 -- 172.21.15.17:0/1072443165 shutdown_connections 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c69a0 0x7f5070072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 --2- 172.21.15.17:0/1072443165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700a8290 0x7f50701539c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 -- 172.21.15.17:0/1072443165 >> 172.21.15.17:0/1072443165 conn(0x7f50700c5560 msgr2=0x7f50700c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 -- 172.21.15.17:0/1072443165 shutdown_connections 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.998+0000 7f5076be2640 1 -- 172.21.15.17:0/1072443165 wait complete. 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.999+0000 7f5076be2640 1 Processor -- start 2024-03-06T07:23:29.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:28.999+0000 7f5076be2640 1 -- start start 2024-03-06T07:23:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700a8290 0x7f507006d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700c4740 0x7f50700c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700c6e30 con 0x7f50700c69a0 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507006fc80 con 0x7f50700c4740 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f5076be2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5070072bc0 con 0x7f50700a8290 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f507517f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f507517f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60122/0 (socket says 172.21.15.17:60122) 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.000+0000 7f507517f640 1 -- 172.21.15.17:0/1914710056 learned_addr learned my addr 172.21.15.17:0/1914710056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 -- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700a8290 msgr2=0x7f507006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507497e640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700a8290 0x7f507006d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f506ffff640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700c4740 0x7f50700c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700a8290 0x7f507006d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 -- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700c4740 msgr2=0x7f50700c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700c4740 0x7f50700c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 -- 172.21.15.17:0/1914710056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507006c980 con 0x7f50700c69a0 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f507517f640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f505800c580 tx=0x7f505800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058015070 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5058004220 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507006cc60 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5058003610 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.001+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5070156ef0 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.002+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507006e880 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.003+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50580037b0 con 0x7f50700c69a0 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.003+0000 7f506d7fa640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 0x7f505006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.003+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f505808a0b0 con 0x7f50700c69a0 2024-03-06T07:23:29.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.003+0000 7f507497e640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 0x7f505006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.004+0000 7f507497e640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 0x7f505006d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5068036000 tx=0x7f5068030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.007+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5058058db0 con 0x7f50700c69a0 2024-03-06T07:23:29.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.258+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f507004f330 con 0x7f50700c69a0 2024-03-06T07:23:29.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.260+0000 7f506d7fa640 1 -- 172.21.15.17:0/1914710056 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f50580586b0 con 0x7f50700c69a0 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:29.262 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 msgr2=0x7f505006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 0x7f505006d9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5068036000 tx=0x7f5068030000 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 msgr2=0x7f507014bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f505800c580 tx=0x7f505800ca40 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 shutdown_connections 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505006b500 0x7f505006d9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c69a0 0x7f507014bcc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700c4740 0x7f50700c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 --2- 172.21.15.17:0/1914710056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700a8290 0x7f507006d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 >> 172.21.15.17:0/1914710056 conn(0x7f50700c5560 msgr2=0x7f50700c3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 shutdown_connections 2024-03-06T07:23:29.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.263+0000 7f5076be2640 1 -- 172.21.15.17:0/1914710056 wait complete. 2024-03-06T07:23:29.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:29.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.375+0000 7fce25fe2640 1 Processor -- start 2024-03-06T07:23:29.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.375+0000 7fce25fe2640 1 -- start start 2024-03-06T07:23:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce25fe2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce201506b0 0x7fce20153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce25fe2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce200c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce25fe2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce20070aa0 con 0x7fce20156ea0 2024-03-06T07:23:29.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce25fe2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce20156450 con 0x7fce201506b0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce25fe2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce20156a10 con 0x7fce200acc00 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fce20156ea0 0x7fce200702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46544/0 (socket says 172.21.15.17:46544) 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1effd640 1 -- 172.21.15.17:0/2383431120 learned_addr learned my addr 172.21.15.17:0/2383431120 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/2383431120 >> v2:172.21.15.163:3300/0 conn(0x7fce201506b0 0x7fce20153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1ffff640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce200c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464120696 0 0) 0x7fce20070aa0 con 0x7fce20156ea0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce08003520 con 0x7fce20156ea0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169671635 0 0) 0x7fce08003520 con 0x7fce20156ea0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 msgr2=0x7fce200c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce200c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 >> v2:172.21.15.163:3300/0 conn(0x7fce201506b0 msgr2=0x7fce20153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.376+0000 7fce1e7fc640 1 --2- 172.21.15.17:0/2383431120 >> v2:172.21.15.163:3300/0 conn(0x7fce201506b0 0x7fce20153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20070d80 con 0x7fce20156ea0 2024-03-06T07:23:29.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fce14001670 con 0x7fce20156ea0 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 >> v1:172.21.15.17:6789/0 conn(0x7fce20156ea0 legacy=0x7fce200702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080039b0 0x7fce08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce08004800 0x7fce08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20156450 con 0x7fce080039b0 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20156a10 con 0x7fce08004800 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0800b9c0 con 0x7fce200acc00 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080039b0 0x7fce08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1effd640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce08004800 0x7fce08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1ffff640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 -- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 msgr2=0x7fce080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 -- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce08004800 msgr2=0x7fce08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce08004800 0x7fce08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1f7fe640 1 -- 172.21.15.17:0/2383431120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08003520 con 0x7fce080039b0 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1ffff640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce200acc00 0x7fce080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.377+0000 7fce1effd640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce08004800 0x7fce08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080039b0 0x7fce08003dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fce10009ac0 tx=0x7fce08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5a186a788c5aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10039070 con 0x7fce080039b0 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1002c710 con 0x7fce080039b0 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce1e7fc640 1 -- 172.21.15.17:0/2383431120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10009690 con 0x7fce080039b0 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce25fe2640 1 -- 172.21.15.17:0/2383431120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080039b0 msgr2=0x7fce08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce25fe2640 1 --2- 172.21.15.17:0/2383431120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce080039b0 0x7fce08003dd0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fce10009ac0 tx=0x7fce08005830 comp rx=0 tx=0).stop 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce1ffff640 1 -- 172.21.15.17:0/2383431120 reap_dead start 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce25fe2640 1 -- 172.21.15.17:0/2383431120 shutdown_connections 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.378+0000 7fce25fe2640 1 -- 172.21.15.17:0/2383431120 >> 172.21.15.17:0/2383431120 conn(0x7fce2006b1a0 msgr2=0x7fce2006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.379+0000 7fce25fe2640 1 -- 172.21.15.17:0/2383431120 shutdown_connections 2024-03-06T07:23:29.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.379+0000 7fce25fe2640 1 -- 172.21.15.17:0/2383431120 wait complete. 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 Processor -- start 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 -- start start 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080039b0 0x7fce2006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce201eee40 0x7fce201eb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce200ad1d0 con 0x7fce200c1d60 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce200ad050 con 0x7fce201eee40 2024-03-06T07:23:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.380+0000 7fce25fe2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce200ad350 con 0x7fce080039b0 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60150/0 (socket says 172.21.15.17:60150) 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 -- 172.21.15.17:0/3868081228 learned_addr learned my addr 172.21.15.17:0/3868081228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 -- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080039b0 msgr2=0x7fce2006d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080039b0 0x7fce2006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 -- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce201eee40 msgr2=0x7fce201eb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce201eee40 0x7fce201eb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 -- 172.21.15.17:0/3868081228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce201ebd80 con 0x7fce200c1d60 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce1ffff640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fce2006be90 tx=0x7fce18010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1801a030 con 0x7fce200c1d60 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce18010e30 con 0x7fce200c1d60 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2006da20 con 0x7fce200c1d60 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18003bb0 con 0x7fce200c1d60 2024-03-06T07:23:29.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.381+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce201ee360 con 0x7fce200c1d60 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.382+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce201ef260 con 0x7fce200c1d60 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.383+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce1801d070 con 0x7fce200c1d60 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.383+0000 7fce07fff640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.383+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce1808a9b0 con 0x7fce200c1d60 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.383+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.384+0000 7fce1f7fe640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 0x7fce0c06d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fce10036000 tx=0x7fce1002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:29.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.387+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce180596a0 con 0x7fce200c1d60 2024-03-06T07:23:29.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.612+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2004f330 con 0x7fce200c1d60 2024-03-06T07:23:29.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.613+0000 7fce07fff640 1 -- 172.21.15.17:0/3868081228 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fce18058fa0 con 0x7fce200c1d60 2024-03-06T07:23:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:29.615 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:26.737853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3941058610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3941058610},{"type":"v1","addr":"172.21.15.17:6835","nonce":3941058610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":7}]} 2024-03-06T07:23:29.615 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 78 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 msgr2=0x7fce0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 0x7fce0c06d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fce10036000 tx=0x7fce1002e000 comp rx=0 tx=0).stop 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 msgr2=0x7fce2006c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fce2006be90 tx=0x7fce18010590 comp rx=0 tx=0).stop 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 shutdown_connections 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce200c1d60 0x7fce2006c0d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce201eee40 0x7fce201eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 --2- 172.21.15.17:0/3868081228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce080039b0 0x7fce2006d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:29.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 >> 172.21.15.17:0/3868081228 conn(0x7fce2006b1a0 msgr2=0x7fce200702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:29.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 shutdown_connections 2024-03-06T07:23:29.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:29.615+0000 7fce25fe2640 1 -- 172.21.15.17:0/3868081228 wait complete. 2024-03-06T07:23:30.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.744389+0000', 'modified': '2024-03-06T07:23:26.737853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3941058610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3941058610}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3941058610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7011, 'qdb_cluster': [7011]} 2024-03-06T07:23:30.031 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:30.032 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:23:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 Processor -- start 2024-03-06T07:23:30.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 -- start start 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf41551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40a7ff0 0x7efcf4155730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf40733f0 con 0x7efcf4152b40 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf4072d70 con 0x7efcf40a7ff0 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.133+0000 7efcf96cf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40c82c0 con 0x7efcf40724d0 2024-03-06T07:23:30.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40a7ff0 0x7efcf4155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf40a7ff0 0x7efcf4155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60154/0 (socket says 172.21.15.17:60154) 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 -- 172.21.15.17:0/3259268195 learned_addr learned my addr 172.21.15.17:0/3259268195 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf41551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 -- 172.21.15.17:0/3259268195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 msgr2=0x7efcf41551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf41551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 -- 172.21.15.17:0/3259268195 >> v1:172.21.15.163:6789/0 conn(0x7efcf4152b40 legacy=0x7efcf40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf37fe640 1 -- 172.21.15.17:0/3259268195 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf406fd30 con 0x7efcf40a7ff0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.134+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4155730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efcec009e50 tx=0x7efcec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adea07b707a55e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf1ffb640 1 -- 172.21.15.17:0/3259268195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec02c9e0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf1ffb640 1 -- 172.21.15.17:0/3259268195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec02cb80 con 0x7efcf40a7ff0 2024-03-06T07:23:30.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf1ffb640 1 -- 172.21.15.17:0/3259268195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0343f0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 -- 172.21.15.17:0/3259268195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 msgr2=0x7efcf4155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4155730 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efcec009e50 tx=0x7efcec02be40 comp rx=0 tx=0).stop 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 -- 172.21.15.17:0/3259268195 shutdown_connections 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4155730 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3259268195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf41551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 -- 172.21.15.17:0/3259268195 >> 172.21.15.17:0/3259268195 conn(0x7efcf406b970 msgr2=0x7efcf40aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.135+0000 7efcf96cf640 1 -- 172.21.15.17:0/3259268195 shutdown_connections 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.136+0000 7efcf96cf640 1 -- 172.21.15.17:0/3259268195 wait complete. 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.137+0000 7efcf96cf640 1 Processor -- start 2024-03-06T07:23:30.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.137+0000 7efcf96cf640 1 -- start start 2024-03-06T07:23:30.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf40a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 0x7efcf4156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4155df0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4155c70 con 0x7efcf4152b40 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.138+0000 7efcf96cf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4070280 con 0x7efcf40724d0 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60162/0 (socket says 172.21.15.17:60162) 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 -- 172.21.15.17:0/3674820084 learned_addr learned my addr 172.21.15.17:0/3674820084 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 0x7efcf4156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf40a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 -- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 msgr2=0x7efcf40a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf40a9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 -- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 msgr2=0x7efcf4156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 0x7efcf4156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 -- 172.21.15.17:0/3674820084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4158aa0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 0x7efcf4156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efce800c7f0 tx=0x7efce800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3674820084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8015070 con 0x7efcf40a7ff0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3674820084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce8007a30 con 0x7efcf40a7ff0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40aa2e0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3674820084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8007e00 con 0x7efcf40a7ff0 2024-03-06T07:23:30.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.139+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf41ea570 con 0x7efcf40a7ff0 2024-03-06T07:23:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.141+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3674820084 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efce8018070 con 0x7efcf40a7ff0 2024-03-06T07:23:30.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.141+0000 7efcdb7fe640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 0x7efce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.141+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3674820084 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efce808a2f0 con 0x7efcf40a7ff0 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.141+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 0x7efce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.142+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 0x7efce006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.142+0000 7efcfc38b740 1 Processor -- start 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.142+0000 7efcfc38b740 1 -- start start 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 0x55bba054ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 0x55bba0549680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba04b36b0 con 0x55bba054f330 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba045ca60 con 0x55bba054ef50 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bba04af750 con 0x55bba054d4b0 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60170/0 (socket says 172.21.15.17:60170) 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 -- 172.21.15.17:0/423917233 learned_addr learned my addr 172.21.15.17:0/423917233 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 0x55bba054ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf37fe640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 0x55bba0549680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 -- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 msgr2=0x55bba0549680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 0x55bba0549680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 -- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 msgr2=0x55bba054ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 0x55bba054ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf27fc640 1 -- 172.21.15.17:0/423917233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bba047af30 con 0x55bba054f330 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.143+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 0x55bba054ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcf37fe640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 0x55bba0549680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcf27fc640 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efce8054d30 tx=0x7efce8058990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ec470386a50668 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcd8ff9640 1 -- 172.21.15.17:0/423917233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce801f040 con 0x55bba054f330 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcd8ff9640 1 -- 172.21.15.17:0/423917233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce808a0c0 con 0x55bba054f330 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcd8ff9640 1 -- 172.21.15.17:0/423917233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8054f40 con 0x55bba054f330 2024-03-06T07:23:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 -- 172.21.15.17:0/423917233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 msgr2=0x55bba054cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7efce8054d30 tx=0x7efce8058990 comp rx=0 tx=0).stop 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 -- 172.21.15.17:0/423917233 shutdown_connections 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054d4b0 0x55bba0549680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba054cf70 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 --2- 172.21.15.17:0/423917233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054ef50 0x55bba054ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.144+0000 7efcfc38b740 1 -- 172.21.15.17:0/423917233 >> 172.21.15.17:0/423917233 conn(0x55bba054b7d0 msgr2=0x55bba05703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.145+0000 7efcfc38b740 1 -- 172.21.15.17:0/423917233 shutdown_connections 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.145+0000 7efcfc38b740 1 -- 172.21.15.17:0/423917233 wait complete. 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.145+0000 7efcfc38b740 1 -- start start 2024-03-06T07:23:30.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.145+0000 7efcfc38b740 1 Processor -- start 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba056c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba056f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba0571230 con 0x55bba054f330 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba046fea0 con 0x55bba056f3b0 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bba0570c70 con 0x55bba054ef50 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcfc38b740 10 client.0 _ll_register_callbacks cb 0x55bba046f870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf2ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba056c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf2ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:37306/0 (socket says 172.21.15.17:37306) 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf2ffd640 1 -- 172.21.15.17:0/1866031767 learned_addr learned my addr 172.21.15.17:0/1866031767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf37fe640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba056f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 -- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 msgr2=0x55bba059f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 -- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 msgr2=0x55bba056f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba056f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.146+0000 7efcf27fc640 1 -- 172.21.15.17:0/1866031767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bba0470270 con 0x55bba054f330 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba056c4f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efce800c7c0 tx=0x7efce8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e67f455fc160a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8085030 con 0x55bba054f330 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efce8088320 con 0x55bba054f330 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efce808a0c0 con 0x55bba054f330 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7efce801f040 con 0x55bba054f330 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 10 client.7149 fetch_fsmap learned FSMap version 78 2024-03-06T07:23:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55bba01c82d0 con 0x55bba054f330 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce808a0c0 con 0x55bba054f330 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 78) v1 ==== 3314+0+0 (secure 0 0 0) 0x7efce8077250 con 0x55bba054f330 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 10 client.7149 fetch_fsmap finished waiting for FSMap version 78 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 10 client.7149 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 20 client.7149 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba059bb50 0x55bba059bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 4 client.7149 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 --> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bba04c0bf0 con 0x55bba059bb50 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.147+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba059bb50 0x55bba059bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba059bb50 0x55bba059bf30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a92a8ca9acdb04 server_cookie=1a16c02c2eeaaec7 in_seq=0 out_seq=0 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcd1ffb640 10 client.7149 ms_handle_connect on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcd1ffb640 1 -- 172.21.15.17:0/1866031767 <== mds.0 v2:172.21.15.17:6832/3941058610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bba04c0bf0 con 0x55bba059bb50 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcd1ffb640 10 client.7149 handle_command_reply: tid=0 2024-03-06T07:23:30.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 client.7149 shutdown 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 msgr2=0x55bba056c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba056c4f0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efce800c7c0 tx=0x7efce8000bc0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 shutdown_connections 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba059bb50 0x55bba059bf30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba056f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcd1ffb640 0 client.7149 ms_handle_reset on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba056c4f0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1866031767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.148+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 >> 172.21.15.17:0/1866031767 conn(0x55bba054d4b0 msgr2=0x55bba054d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.149+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 shutdown_connections 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.149+0000 7efcfc38b740 1 -- 172.21.15.17:0/1866031767 wait complete. 2024-03-06T07:23:30.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.149+0000 7efcfc38b740 20 client.7149 trim_cache size 0 max 16384 2024-03-06T07:23:30.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.165+0000 7efcfc38b740 1 Processor -- start 2024-03-06T07:23:30.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.165+0000 7efcfc38b740 1 -- start start 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba0598a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41f6d00 con 0x55bba054f330 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41f6100 con 0x55bba056f3b0 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41f6700 con 0x55bba054ef50 2024-03-06T07:23:30.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60200/0 (socket says 172.21.15.17:60200) 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 -- 172.21.15.17:0/1218694893 learned_addr learned my addr 172.21.15.17:0/1218694893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 -- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 msgr2=0x55bba059bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 -- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 msgr2=0x55bba0598a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba0598a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 -- 172.21.15.17:0/1218694893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bba02926b0 con 0x55bba054f330 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.166+0000 7efcf27fc640 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efce8060ab0 tx=0x7efce806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe30872dcf37742b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcd3fff640 1 -- 172.21.15.17:0/1218694893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce807e020 con 0x55bba054f330 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcd3fff640 1 -- 172.21.15.17:0/1218694893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce800ce40 con 0x55bba054f330 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcd3fff640 1 -- 172.21.15.17:0/1218694893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce8085020 con 0x55bba054f330 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 -- 172.21.15.17:0/1218694893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 msgr2=0x55bba059c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efce8060ab0 tx=0x7efce806d000 comp rx=0 tx=0).stop 2024-03-06T07:23:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 -- 172.21.15.17:0/1218694893 shutdown_connections 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba056f3b0 0x55bba0598a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba054f330 0x55bba059c470 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 --2- 172.21.15.17:0/1218694893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba059bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 -- 172.21.15.17:0/1218694893 >> 172.21.15.17:0/1218694893 conn(0x55bba054d4b0 msgr2=0x55bba0544e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 -- 172.21.15.17:0/1218694893 shutdown_connections 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.167+0000 7efcfc38b740 1 -- 172.21.15.17:0/1218694893 wait complete. 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 -- start start 2024-03-06T07:23:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 Processor -- start 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba05fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 0x55bba054c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba0571230 con 0x55bba056f3b0 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bba046fea0 con 0x55bba054f330 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bba0575bb0 con 0x55bba054ef50 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.168+0000 7efcfc38b740 10 client.0 _ll_register_callbacks cb 0x7efce80610a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60202/0 (socket says 172.21.15.17:60202) 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 -- 172.21.15.17:0/3659810437 learned_addr learned my addr 172.21.15.17:0/3659810437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba05fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 0x55bba054c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 -- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 msgr2=0x55bba05fdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba05fdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 -- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 msgr2=0x55bba054c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 0x55bba054c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 -- 172.21.15.17:0/3659810437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bba0588ac0 con 0x55bba056f3b0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf27fc640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 0x55bba054c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcf37fe640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c090 tx=0x7efcec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e5c849c7ea1e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec03eb40 con 0x55bba056f3b0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efcec03ece0 con 0x55bba056f3b0 2024-03-06T07:23:30.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bba04baf40 con 0x55bba056f3b0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.169+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcec0343f0 con 0x55bba056f3b0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 10 client.7155 fetch_fsmap learned FSMap version 78 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55bba05fe830 con 0x55bba056f3b0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bba04baf40 con 0x55bba056f3b0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 78) v1 ==== 3314+0+0 (secure 0 0 0) 0x7efcec034590 con 0x55bba056f3b0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 10 client.7155 fetch_fsmap finished waiting for FSMap version 78 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 10 client.7155 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 20 client.7155 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba0572e10 0x55bba05731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 4 client.7155 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 --> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bba04399a0 con 0x55bba0572e10 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba0572e10 0x55bba05731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcd1ffb640 10 client.7155 ms_handle_connect on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcf2ffd640 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba0572e10 0x55bba05731f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3bdb56d08affe11 server_cookie=fa48a20103e92a27 in_seq=0 out_seq=0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcd1ffb640 1 -- 172.21.15.17:0/3659810437 <== mds.0 v2:172.21.15.17:6832/3941058610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bba04399a0 con 0x55bba0572e10 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcd1ffb640 10 client.7155 handle_command_reply: tid=0 2024-03-06T07:23:30.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 client.7155 shutdown 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 msgr2=0x55bba054cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.170+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efcec00c090 tx=0x7efcec004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 shutdown_connections 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x55bba0572e10 0x55bba05731f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bba056f3b0 0x55bba054cec0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55bba054f330 0x55bba054c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcd1ffb640 0 client.7155 ms_handle_reset on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 --2- 172.21.15.17:0/3659810437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55bba054ef50 0x55bba05fdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 >> 172.21.15.17:0/3659810437 conn(0x55bba054d4b0 msgr2=0x55bba058d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 shutdown_connections 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 1 -- 172.21.15.17:0/3659810437 wait complete. 2024-03-06T07:23:30.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcfc38b740 20 client.7155 trim_cache size 0 max 16384 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7011, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.7998574390000002, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 78, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 96, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 96, 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9.8647061130000004 2024-03-06T07:23:30.179 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:30.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 msgr2=0x7efce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 0x7efce006d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).stop 2024-03-06T07:23:30.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 msgr2=0x7efcf4158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efce800c7f0 tx=0x7efce800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 shutdown_connections 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efce006b4d0 0x7efce006d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcf4152b40 0x7efcf4156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf40a7ff0 0x7efcf4158560 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 --2- 172.21.15.17:0/3674820084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcf40724d0 0x7efcf40a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 >> 172.21.15.17:0/3674820084 conn(0x7efcf406b970 msgr2=0x7efcf4153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 shutdown_connections 2024-03-06T07:23:30.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.171+0000 7efcf96cf640 1 -- 172.21.15.17:0/3674820084 wait complete. 2024-03-06T07:23:30.184 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.284+0000 7fd19ca9a640 1 Processor -- start 2024-03-06T07:23:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.284+0000 7fd19ca9a640 1 -- start start 2024-03-06T07:23:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19ca9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd198155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19ca9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1980c73d0 con 0x7fd1980c88c0 2024-03-06T07:23:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19ca9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1980be2e0 con 0x7fd19814f360 2024-03-06T07:23:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19ca9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd198159700 con 0x7fd198159270 2024-03-06T07:23:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd195d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1980c88c0 0x7fd198158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46560/0 (socket says 172.21.15.17:46560) 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd195d9b640 1 -- 172.21.15.17:0/3285565878 learned_addr learned my addr 172.21.15.17:0/3285565878 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd196d9d640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd198155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205977927 0 0) 0x7fd1980c73d0 con 0x7fd1980c88c0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd184003520 con 0x7fd1980c88c0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813804760 0 0) 0x7fd184003520 con 0x7fd1980c88c0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 msgr2=0x7fd198155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd198155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 >> v1:172.21.15.163:6789/0 conn(0x7fd19814f360 legacy=0x7fd1980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980c7670 con 0x7fd1980c88c0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd188001670 con 0x7fd1980c88c0 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 >> v1:172.21.15.17:6789/0 conn(0x7fd1980c88c0 legacy=0x7fd198158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd184003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd184004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd184004d40 0x7fd1840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd198159700 con 0x7fd1840039e0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18400baa0 con 0x7fd184004d40 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.286+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18400bca0 con 0x7fd198159270 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd184003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd195d9b640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd184004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd196d9d640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd184004d40 0x7fd1840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 -- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 msgr2=0x7fd184004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd198159270 0x7fd184004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 -- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd184004d40 msgr2=0x7fd1840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd184004d40 0x7fd1840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.287+0000 7fd19659c640 1 -- 172.21.15.17:0/3285565878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184003520 con 0x7fd1840039e0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19659c640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd184003dc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd180001860 tx=0x7fd184005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5891f5104c4526 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18002f050 con 0x7fd1840039e0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180033650 con 0x7fd1840039e0 2024-03-06T07:23:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19559a640 1 -- 172.21.15.17:0/3285565878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180001430 con 0x7fd1840039e0 2024-03-06T07:23:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3285565878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 msgr2=0x7fd184003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3285565878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd184003dc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd180001860 tx=0x7fd184005830 comp rx=0 tx=0).stop 2024-03-06T07:23:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd196d9d640 1 -- 172.21.15.17:0/3285565878 reap_dead start 2024-03-06T07:23:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3285565878 shutdown_connections 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3285565878 >> 172.21.15.17:0/3285565878 conn(0x7fd1980ac570 msgr2=0x7fd1980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3285565878 shutdown_connections 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.288+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3285565878 wait complete. 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 Processor -- start 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 -- start start 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1981df060 0x7fd1981eac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1981eb170 0x7fd19814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980c7c30 con 0x7fd1840039e0 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980c5a50 con 0x7fd1981df060 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19ca9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980c5bd0 con 0x7fd1981eb170 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60216/0 (socket says 172.21.15.17:60216) 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.290+0000 7fd19659c640 1 -- 172.21.15.17:0/3328513715 learned_addr learned my addr 172.21.15.17:0/3328513715 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd195d9b640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1981df060 0x7fd1981eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd196d9d640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1981eb170 0x7fd19814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 -- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1981eb170 msgr2=0x7fd19814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1981eb170 0x7fd19814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 -- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1981df060 msgr2=0x7fd1981eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1981df060 0x7fd1981eac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 -- 172.21.15.17:0/3328513715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1981ded60 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19659c640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd180001950 tx=0x7fd180001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800035f0 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180003790 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1980c3860 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180003b40 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.291+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1980c3e20 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.292+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd198075300 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.293+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd180003ce0 con 0x7fd1840039e0 2024-03-06T07:23:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.293+0000 7fd18effd640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.293+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1800ac470 con 0x7fd1840039e0 2024-03-06T07:23:30.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.293+0000 7fd195d9b640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.294+0000 7fd195d9b640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 0x7fd17806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1981ebd60 tx=0x7fd188009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.297+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd198075300 con 0x7fd1840039e0 2024-03-06T07:23:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.518+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd19804f330 con 0x7fd1840039e0 2024-03-06T07:23:30.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.518+0000 7fd18effd640 1 -- 172.21.15.17:0/3328513715 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fd18004a060 con 0x7fd1840039e0 2024-03-06T07:23:30.520 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:30.520 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:26.737853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3941058610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3941058610},{"type":"v1","addr":"172.21.15.17:6835","nonce":3941058610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":7}]} 2024-03-06T07:23:30.521 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 78 2024-03-06T07:23:30.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 msgr2=0x7fd17806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 0x7fd17806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd1981ebd60 tx=0x7fd188009040 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 msgr2=0x7fd1980c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd180001950 tx=0x7fd180001230 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 shutdown_connections 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1981eb170 0x7fd19814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1981df060 0x7fd1981eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 --2- 172.21.15.17:0/3328513715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1840039e0 0x7fd1980c0930 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 >> 172.21.15.17:0/3328513715 conn(0x7fd1980ac570 msgr2=0x7fd1980aad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 shutdown_connections 2024-03-06T07:23:30.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.521+0000 7fd19ca9a640 1 -- 172.21.15.17:0/3328513715 wait complete. 2024-03-06T07:23:30.534 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 78, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.744389+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 78, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7011": { "addr": "172.21.15.17:6835/3941058610", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3941058610, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 3941058610, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7011, "incarnation": 76, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:26.737853+0000", "qdb_cluster": [ 7011 ], "qdb_leader": 7011, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7011 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/2259557388", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2259557388, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2259557388, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5963, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6836/3720096092", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3720096092, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3720096092, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6706, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2875090396", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 2875090396, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2875090396, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7008, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:30.534 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:23:30.534 INFO:tasks.cephfs.test_failover:current = [{'gid': 7008, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2875090396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2875090396}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2875090396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 6706, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/3720096092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3720096092}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3720096092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3941058610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3941058610}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3941058610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5963, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2259557388', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2259557388}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2259557388}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}] 2024-03-06T07:23:30.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:23:30.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.658+0000 7fb611913640 1 Processor -- start 2024-03-06T07:23:30.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.658+0000 7fb611913640 1 -- start start 2024-03-06T07:23:30.660 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:23:30.659+0000 7f8ff037a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:30.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb611913640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb60c0c7370 0x7fb60c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb611913640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb60c06c0b0 0x7fb60c06c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb611913640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c155260 con 0x7fb60c1527f0 2024-03-06T07:23:30.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb611913640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c0c8f70 con 0x7fb60c06c0b0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb611913640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c155540 con 0x7fb60c0c7370 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb60c06c0b0 0x7fb60c06c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb60c1527f0 0x7fb60c0c8860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42778/0 (socket says 172.21.15.17:42778) 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60affd640 1 -- 172.21.15.17:0/2144834032 learned_addr learned my addr 172.21.15.17:0/2144834032 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60a7fc640 1 --2- 172.21.15.17:0/2144834032 >> v2:172.21.15.163:3300/0 conn(0x7fb60c0c7370 0x7fb60c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60b7fe640 1 -- 172.21.15.17:0/2144834032 >> v2:172.21.15.163:3300/0 conn(0x7fb60c0c7370 msgr2=0x7fb60c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60b7fe640 1 --2- 172.21.15.17:0/2144834032 >> v2:172.21.15.163:3300/0 conn(0x7fb60c0c7370 0x7fb60c0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60b7fe640 1 -- 172.21.15.17:0/2144834032 >> v1:172.21.15.17:6789/0 conn(0x7fb60c1527f0 legacy=0x7fb60c0c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb60b7fe640 1 -- 172.21.15.17:0/2144834032 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c155820 con 0x7fb60c06c0b0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.659+0000 7fb609ffb640 1 -- 172.21.15.17:0/2144834032 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627461393 0 0) 0x7fb60c155260 con 0x7fb60c1527f0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.660+0000 7fb60b7fe640 1 --2- 172.21.15.17:0/2144834032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c06c0b0 0x7fb60c06c4c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0aa410 tx=0x7fb60402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b651e1ca9c6047 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.660+0000 7fb609ffb640 1 -- 172.21.15.17:0/2144834032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604039070 con 0x7fb60c06c0b0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.660+0000 7fb609ffb640 1 -- 172.21.15.17:0/2144834032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60402cb60 con 0x7fb60c06c0b0 2024-03-06T07:23:30.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.660+0000 7fb609ffb640 1 -- 172.21.15.17:0/2144834032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604034440 con 0x7fb60c06c0b0 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 -- 172.21.15.17:0/2144834032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c06c0b0 msgr2=0x7fb60c06c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 --2- 172.21.15.17:0/2144834032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c06c0b0 0x7fb60c06c4c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0aa410 tx=0x7fb60402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 -- 172.21.15.17:0/2144834032 shutdown_connections 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 --2- 172.21.15.17:0/2144834032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c06c0b0 0x7fb60c06c4c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 --2- 172.21.15.17:0/2144834032 >> v2:172.21.15.163:3300/0 conn(0x7fb60c0c7370 0x7fb60c0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 -- 172.21.15.17:0/2144834032 >> 172.21.15.17:0/2144834032 conn(0x7fb60c0ac4f0 msgr2=0x7fb60c0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 -- 172.21.15.17:0/2144834032 shutdown_connections 2024-03-06T07:23:30.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.661+0000 7fb611913640 1 -- 172.21.15.17:0/2144834032 wait complete. 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.662+0000 7fb611913640 1 Processor -- start 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.662+0000 7fb611913640 1 -- start start 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 0x7fb60c1474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c1527f0 0x7fb60c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c06ca00 con 0x7fb60c0c7370 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0be1a0 con 0x7fb60c1527f0 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c06cb80 con 0x7fb60c06c0b0 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47328/0 (socket says 172.21.15.17:47328) 2024-03-06T07:23:30.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 -- 172.21.15.17:0/1154092589 learned_addr learned my addr 172.21.15.17:0/1154092589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60b7fe640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c1527f0 0x7fb60c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60affd640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 0x7fb60c1474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 -- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 msgr2=0x7fb60c1474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 0x7fb60c1474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 -- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c1527f0 msgr2=0x7fb60c0c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c1527f0 0x7fb60c0c1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 -- 172.21.15.17:0/1154092589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c1479f0 con 0x7fb60c0c7370 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60affd640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 0x7fb60c1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb60a7fc640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb60000d830 tx=0x7fb60000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60000b6c0 con 0x7fb60c0c7370 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600019070 con 0x7fb60c0c7370 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60000ba80 con 0x7fb60c0c7370 2024-03-06T07:23:30.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c14eb40 con 0x7fb60c0c7370 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.663+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c0c4630 con 0x7fb60c0c7370 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.664+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0cb7d0 con 0x7fb60c0c7370 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.665+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb60001e460 con 0x7fb60c0c7370 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.665+0000 7fb5f3fff640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.666+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb60008a8f0 con 0x7fb60c0c7370 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.666+0000 7fb60affd640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.666+0000 7fb60affd640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 0x7fb5f806d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.670+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb60c0cb7d0 con 0x7fb60c0c7370 2024-03-06T07:23:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.850+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fb60c04f330 con 0x7fb60c0c7370 2024-03-06T07:23:30.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.853+0000 7fb5f3fff640 1 -- 172.21.15.17:0/1154092589 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v32)=0 v32) v1 ==== 116+0+0 (secure 0 0 0) 0x7fb600059670 con 0x7fb60c0c7370 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 msgr2=0x7fb5f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 0x7fb5f806d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 msgr2=0x7fb60c153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb60000d830 tx=0x7fb60000dcf0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 shutdown_connections 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5f806b4d0 0x7fb5f806d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb60c1527f0 0x7fb60c0c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb60c0c7370 0x7fb60c153f80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 --2- 172.21.15.17:0/1154092589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb60c06c0b0 0x7fb60c1474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 >> 172.21.15.17:0/1154092589 conn(0x7fb60c0ac4f0 msgr2=0x7fb60c151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 shutdown_connections 2024-03-06T07:23:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.855+0000 7fb611913640 1 -- 172.21.15.17:0/1154092589 wait complete. 2024-03-06T07:23:30.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:30.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.968+0000 7ff71a93a640 1 Processor -- start 2024-03-06T07:23:30.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.968+0000 7ff71a93a640 1 -- start start 2024-03-06T07:23:30.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff71a93a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 0x7ff7140c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff71a93a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7141586f0 0x7ff714158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff71a93a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7141591c0 con 0x7ff7140c5480 2024-03-06T07:23:30.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff71a93a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71406df40 con 0x7ff7141586f0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff71a93a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71406fdb0 con 0x7ff714155ba0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff718ed7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7141586f0 0x7ff714158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff713fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff7140c5480 0x7ff71406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42794/0 (socket says 172.21.15.17:42794) 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.969+0000 7ff713fff640 1 -- 172.21.15.17:0/2951733551 learned_addr learned my addr 172.21.15.17:0/2951733551 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff7137fe640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 0x7ff7140c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff718ed7640 1 -- 172.21.15.17:0/2951733551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 msgr2=0x7ff7140c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff718ed7640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 0x7ff7140c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff718ed7640 1 -- 172.21.15.17:0/2951733551 >> v1:172.21.15.17:6789/0 conn(0x7ff7140c5480 legacy=0x7ff71406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff718ed7640 1 -- 172.21.15.17:0/2951733551 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7141594a0 con 0x7ff7141586f0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff712ffd640 1 -- 172.21.15.17:0/2951733551 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666765438 0 0) 0x7ff7141591c0 con 0x7ff7140c5480 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff7137fe640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 0x7ff7140c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.970+0000 7ff718ed7640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff714158b80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009d90 tx=0x7ff70c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af8c642f96075d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff712ffd640 1 -- 172.21.15.17:0/2951733551 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c039070 con 0x7ff7141586f0 2024-03-06T07:23:30.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff712ffd640 1 -- 172.21.15.17:0/2951733551 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c02cbd0 con 0x7ff7141586f0 2024-03-06T07:23:30.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff712ffd640 1 -- 172.21.15.17:0/2951733551 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c034440 con 0x7ff7141586f0 2024-03-06T07:23:30.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 -- 172.21.15.17:0/2951733551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 msgr2=0x7ff714158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff714158b80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009d90 tx=0x7ff70c02be20 comp rx=0 tx=0).stop 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 -- 172.21.15.17:0/2951733551 shutdown_connections 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff714158b80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 --2- 172.21.15.17:0/2951733551 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff714155ba0 0x7ff7140c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 -- 172.21.15.17:0/2951733551 >> 172.21.15.17:0/2951733551 conn(0x7ff71406b1a0 msgr2=0x7ff71406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 -- 172.21.15.17:0/2951733551 shutdown_connections 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.971+0000 7ff71a93a640 1 -- 172.21.15.17:0/2951733551 wait complete. 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.972+0000 7ff71a93a640 1 Processor -- start 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 -- start start 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7140c5480 0x7ff714156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 0x7ff7140c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714155680 con 0x7ff7141586f0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714156000 con 0x7ff714155ba0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff71a93a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff714152a20 con 0x7ff7140c5480 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff718ed7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff718ed7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47360/0 (socket says 172.21.15.17:47360) 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.973+0000 7ff718ed7640 1 -- 172.21.15.17:0/3228021640 learned_addr learned my addr 172.21.15.17:0/3228021640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff713fff640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7140c5480 0x7ff714156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff7137fe640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 0x7ff7140c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 -- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7140c5480 msgr2=0x7ff714156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7140c5480 0x7ff714156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 -- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 msgr2=0x7ff7140c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 0x7ff7140c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 -- 172.21.15.17:0/3228021640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140ab560 con 0x7ff7141586f0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff7137fe640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 0x7ff7140c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff718ed7640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff70c037040 tx=0x7ff70c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c044020 con 0x7ff7141586f0 2024-03-06T07:23:30.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c03a040 con 0x7ff7141586f0 2024-03-06T07:23:30.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7140abdc0 con 0x7ff7141586f0 2024-03-06T07:23:30.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c039050 con 0x7ff7141586f0 2024-03-06T07:23:30.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.974+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7140ab800 con 0x7ff7141586f0 2024-03-06T07:23:30.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.975+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71404f330 con 0x7ff7141586f0 2024-03-06T07:23:30.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.976+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff70c003bb0 con 0x7ff7141586f0 2024-03-06T07:23:30.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.976+0000 7ff710ff9640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:30.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.976+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff70c0ac9c0 con 0x7ff7141586f0 2024-03-06T07:23:30.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.976+0000 7ff713fff640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:30.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.977+0000 7ff713fff640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 0x7ff70006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff704000f60 tx=0x7ff704007b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:30.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:30.980+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff70c0aee20 con 0x7ff7141586f0 2024-03-06T07:23:31.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.198+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff71406e0c0 con 0x7ff7141586f0 2024-03-06T07:23:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.198+0000 7ff710ff9640 1 -- 172.21.15.17:0/3228021640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4985 (secure 0 0 0) 0x7ff70c04e060 con 0x7ff7141586f0 2024-03-06T07:23:31.200 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:31.200 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:26.737853+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7011},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7011":{"gid":7011,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/3941058610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3941058610},{"type":"v1","addr":"172.21.15.17:6835","nonce":3941058610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7011,"qdb_cluster":[7011]},"id":7}]} 2024-03-06T07:23:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 78 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 msgr2=0x7ff70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 0x7ff70006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff704000f60 tx=0x7ff704007b70 comp rx=0 tx=0).stop 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 msgr2=0x7ff71414d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff70c037040 tx=0x7ff70c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 shutdown_connections 2024-03-06T07:23:31.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff70006b4d0 0x7ff70006d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7141586f0 0x7ff71414d860 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff714155ba0 0x7ff7140c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 --2- 172.21.15.17:0/3228021640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7140c5480 0x7ff714156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 >> 172.21.15.17:0/3228021640 conn(0x7ff71406b1a0 msgr2=0x7ff71406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 shutdown_connections 2024-03-06T07:23:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.200+0000 7ff71a93a640 1 -- 172.21.15.17:0/3228021640 wait complete. 2024-03-06T07:23:31.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.744389+0000', 'modified': '2024-03-06T07:23:26.737853+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7011}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7011': {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3941058610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3941058610}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3941058610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7011, 'qdb_cluster': [7011]} 2024-03-06T07:23:31.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:31.617 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:23:31.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.718+0000 7fc12d6a4640 1 Processor -- start 2024-03-06T07:23:31.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.718+0000 7fc12d6a4640 1 -- start start 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc12d6a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1281504c0 0x7fc1280c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc12d6a4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1280c2130 0x7fc1280a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc12d6a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc128153010 con 0x7fc1280c4e80 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc12d6a4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc128153ee0 con 0x7fc1280c2130 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc12d6a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12806f390 con 0x7fc1281504c0 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc11ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1280c2130 0x7fc1280a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc11ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1280c2130 0x7fc1280a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47368/0 (socket says 172.21.15.17:47368) 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc1277fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1280c4e80 0x7fc1280a99c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42796/0 (socket says 172.21.15.17:42796) 2024-03-06T07:23:31.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.719+0000 7fc11ffff640 1 -- 172.21.15.17:0/280273587 learned_addr learned my addr 172.21.15.17:0/280273587 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc126ffd640 1 --2- 172.21.15.17:0/280273587 >> v2:172.21.15.163:3300/0 conn(0x7fc1281504c0 0x7fc1280c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc11ffff640 1 -- 172.21.15.17:0/280273587 >> v2:172.21.15.163:3300/0 conn(0x7fc1281504c0 msgr2=0x7fc1280c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc11ffff640 1 --2- 172.21.15.17:0/280273587 >> v2:172.21.15.163:3300/0 conn(0x7fc1281504c0 0x7fc1280c4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc11ffff640 1 -- 172.21.15.17:0/280273587 >> v1:172.21.15.17:6789/0 conn(0x7fc1280c4e80 legacy=0x7fc1280a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc1267fc640 1 -- 172.21.15.17:0/280273587 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537778493 0 0) 0x7fc128153010 con 0x7fc1280c4e80 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc11ffff640 1 -- 172.21.15.17:0/280273587 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1281532d0 con 0x7fc1280c2130 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc11ffff640 1 --2- 172.21.15.17:0/280273587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1280a9480 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc12806c060 tx=0x7fc11002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84177942320068f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.720+0000 7fc1267fc640 1 -- 172.21.15.17:0/280273587 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110035020 con 0x7fc1280c2130 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc1267fc640 1 -- 172.21.15.17:0/280273587 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc110031870 con 0x7fc1280c2130 2024-03-06T07:23:31.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc1267fc640 1 -- 172.21.15.17:0/280273587 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110031c40 con 0x7fc1280c2130 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 -- 172.21.15.17:0/280273587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 msgr2=0x7fc1280a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/280273587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1280a9480 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc12806c060 tx=0x7fc11002b910 comp rx=0 tx=0).stop 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 -- 172.21.15.17:0/280273587 shutdown_connections 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/280273587 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1280a9480 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/280273587 >> v2:172.21.15.163:3300/0 conn(0x7fc1281504c0 0x7fc1280c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 -- 172.21.15.17:0/280273587 >> 172.21.15.17:0/280273587 conn(0x7fc1280c7de0 msgr2=0x7fc1280c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 -- 172.21.15.17:0/280273587 shutdown_connections 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.721+0000 7fc12d6a4640 1 -- 172.21.15.17:0/280273587 wait complete. 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.722+0000 7fc12d6a4640 1 Processor -- start 2024-03-06T07:23:31.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.722+0000 7fc12d6a4640 1 -- start start 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1280c4e80 0x7fc1280c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1281504c0 0x7fc1281ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128153b70 con 0x7fc1280c2130 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128153890 con 0x7fc1281504c0 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc12d6a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280aa0d0 con 0x7fc1280c4e80 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47372/0 (socket says 172.21.15.17:47372) 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 -- 172.21.15.17:0/2873154658 learned_addr learned my addr 172.21.15.17:0/2873154658 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 -- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1280c4e80 msgr2=0x7fc1280c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1280c4e80 0x7fc1280c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 -- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1281504c0 msgr2=0x7fc1281ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1281504c0 0x7fc1281ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 -- 172.21.15.17:0/2873154658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280ad690 con 0x7fc1280c2130 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.723+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c7f0 tx=0x7fc11800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.724+0000 7fc11f7fe640 1 -- 172.21.15.17:0/2873154658 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118015070 con 0x7fc1280c2130 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.724+0000 7fc11f7fe640 1 -- 172.21.15.17:0/2873154658 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc118004220 con 0x7fc1280c2130 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.724+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12814f4d0 con 0x7fc1280c2130 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.724+0000 7fc11f7fe640 1 -- 172.21.15.17:0/2873154658 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118007620 con 0x7fc1280c2130 2024-03-06T07:23:31.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.724+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12814fa90 con 0x7fc1280c2130 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.725+0000 7fc11f7fe640 1 -- 172.21.15.17:0/2873154658 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1180079d0 con 0x7fc1280c2130 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.726+0000 7fc11f7fe640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 0x7fc10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.726+0000 7fc11f7fe640 1 -- 172.21.15.17:0/2873154658 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc11808a630 con 0x7fc1280c2130 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.726+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 0x7fc10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.726+0000 7fc13036f740 1 Processor -- start 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 0x7fc10806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc110036000 tx=0x7fc110030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 -- start start 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 0x564d4cc71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc6b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4cbd56b0 con 0x564d4cc71550 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4cb7ea60 con 0x564d4ccab420 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4cbd1750 con 0x564d4ccba800 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47384/0 (socket says 172.21.15.17:47384) 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.727+0000 7fc11ffff640 1 -- 172.21.15.17:0/2638770871 learned_addr learned my addr 172.21.15.17:0/2638770871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 0x564d4cc71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc1277fe640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc6b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 -- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 msgr2=0x564d4cc71010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 0x564d4cc71010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 -- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 msgr2=0x564d4cc6b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc6b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 -- 172.21.15.17:0/2638770871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d4cb9cf30 con 0x564d4cc71550 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 0x564d4cc71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc110031b80 tx=0x7fc110031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8564338d555a6220 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11cff9640 1 -- 172.21.15.17:0/2638770871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11004a070 con 0x564d4cc71550 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11cff9640 1 -- 172.21.15.17:0/2638770871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11003aca0 con 0x564d4cc71550 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.728+0000 7fc11cff9640 1 -- 172.21.15.17:0/2638770871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110040070 con 0x564d4cc71550 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- 172.21.15.17:0/2638770871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 msgr2=0x564d4ccbabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc110031b80 tx=0x7fc110031000 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- 172.21.15.17:0/2638770871 shutdown_connections 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc6b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4cc71550 0x564d4ccbabe0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 --2- 172.21.15.17:0/2638770871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccba800 0x564d4cc71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- 172.21.15.17:0/2638770871 >> 172.21.15.17:0/2638770871 conn(0x564d4cc6d7d0 msgr2=0x564d4cc923c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- 172.21.15.17:0/2638770871 shutdown_connections 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- 172.21.15.17:0/2638770871 wait complete. 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 -- start start 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.729+0000 7fc13036f740 1 Processor -- start 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccad470 0x564d4ccc54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4ccaf590 con 0x564d4ccba800 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4cb91ea0 con 0x564d4ccab420 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4cc92db0 con 0x564d4ccad470 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc13036f740 10 client.0 _ll_register_callbacks cb 0x564d4cb91870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47400/0 (socket says 172.21.15.17:47400) 2024-03-06T07:23:31.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 -- 172.21.15.17:0/1672248073 learned_addr learned my addr 172.21.15.17:0/1672248073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc1277fe640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccad470 0x564d4ccc54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 -- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccad470 msgr2=0x564d4ccc54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccad470 0x564d4ccc54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 -- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 msgr2=0x564d4cc90f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc90f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.730+0000 7fc11ffff640 1 -- 172.21.15.17:0/1672248073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d4cb92270 con 0x564d4ccba800 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250bc031b8bf8c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110040050 con 0x564d4ccba800 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc11003aaa0 con 0x564d4ccba800 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc11003aca0 con 0x564d4ccba800 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc110060050 con 0x564d4ccba800 2024-03-06T07:23:31.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 10 client.7194 fetch_fsmap learned FSMap version 78 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x564d4c8ea2d0 con 0x564d4ccba800 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11003aca0 con 0x564d4ccba800 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 78) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fc110063020 con 0x564d4ccba800 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 10 client.7194 fetch_fsmap finished waiting for FSMap version 78 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 10 client.7194 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 20 client.7194 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cc8ee80 0x564d4cc8f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 4 client.7194 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 --> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d4cbe2bf0 con 0x564d4cc8ee80 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.731+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cc8ee80 0x564d4cc8f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc105ffb640 10 client.7194 ms_handle_connect on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cc8ee80 0x564d4cc8f260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b808196457588e83 server_cookie=7430d4d713094a93 in_seq=0 out_seq=0 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc105ffb640 1 -- 172.21.15.17:0/1672248073 <== mds.0 v2:172.21.15.17:6832/3941058610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d4cbe2bf0 con 0x564d4cc8ee80 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc105ffb640 10 client.7194 handle_command_reply: tid=0 2024-03-06T07:23:31.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 client.7194 shutdown 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 msgr2=0x564d4ccacf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc110000c00 tx=0x7fc110004730 comp rx=0 tx=0).stop 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 shutdown_connections 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cc8ee80 0x564d4cc8f260 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4ccad470 0x564d4ccc54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccacf30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc105ffb640 0 client.7194 ms_handle_reset on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 --2- 172.21.15.17:0/1672248073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4cc90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.732+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 >> 172.21.15.17:0/1672248073 conn(0x564d4cc71550 msgr2=0x564d4cc6be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.733+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 shutdown_connections 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.733+0000 7fc13036f740 1 -- 172.21.15.17:0/1672248073 wait complete. 2024-03-06T07:23:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.733+0000 7fc13036f740 20 client.7194 trim_cache size 0 max 16384 2024-03-06T07:23:31.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.749+0000 7fc13036f740 1 Processor -- start 2024-03-06T07:23:31.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.749+0000 7fc13036f740 1 -- start start 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccaefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4ccc1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128073160 con 0x564d4ccba800 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128072560 con 0x564d4ccab420 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc128072b60 con 0x564d4cc8ee80 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47414/0 (socket says 172.21.15.17:47414) 2024-03-06T07:23:31.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 -- 172.21.15.17:0/2876023533 learned_addr learned my addr 172.21.15.17:0/2876023533 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc126ffd640 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccaefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc11ffff640 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4ccc1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 -- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 msgr2=0x564d4ccaefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccaefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 -- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 msgr2=0x564d4ccc1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4ccc1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.750+0000 7fc1277fe640 1 -- 172.21.15.17:0/2876023533 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d4ccc3100 con 0x564d4ccba800 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc1277fe640 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc120000f80 tx=0x7fc120007560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3f3db55ae8045 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc107fff640 1 -- 172.21.15.17:0/2876023533 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1200017d0 con 0x564d4ccba800 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc107fff640 1 -- 172.21.15.17:0/2876023533 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120019070 con 0x564d4ccba800 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc107fff640 1 -- 172.21.15.17:0/2876023533 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120001bd0 con 0x564d4ccba800 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 -- 172.21.15.17:0/2876023533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 msgr2=0x564d4ccb9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc120000f80 tx=0x7fc120007560 comp rx=0 tx=0).stop 2024-03-06T07:23:31.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 -- 172.21.15.17:0/2876023533 shutdown_connections 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccba800 0x564d4ccb9660 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccab420 0x564d4ccc1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 --2- 172.21.15.17:0/2876023533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccaefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 -- 172.21.15.17:0/2876023533 >> 172.21.15.17:0/2876023533 conn(0x564d4cc71550 msgr2=0x564d4cc90d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 -- 172.21.15.17:0/2876023533 shutdown_connections 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.751+0000 7fc13036f740 1 -- 172.21.15.17:0/2876023533 wait complete. 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.752+0000 7fc13036f740 1 -- start start 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.752+0000 7fc13036f740 1 Processor -- start 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccba800 0x564d4cd1e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4ccb84d0 con 0x564d4ccab420 2024-03-06T07:23:31.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d4ccac790 con 0x564d4ccba800 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d4ccc5cf0 con 0x564d4cc8ee80 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 10 client.0 _ll_register_callbacks cb 0x7fc110004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47430/0 (socket says 172.21.15.17:47430) 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 -- 172.21.15.17:0/1582501222 learned_addr learned my addr 172.21.15.17:0/1582501222 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc1277fe640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccba800 0x564d4cd1e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 -- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 msgr2=0x564d4ccae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 -- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccba800 msgr2=0x564d4cd1e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccba800 0x564d4cd1e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 -- 172.21.15.17:0/1582501222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d4c9b46b0 con 0x564d4ccab420 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:31.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc11ffff640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc1100090b0 tx=0x7fc110008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9003fe282c384ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110069060 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc110003ab0 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.753+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc120019070 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc110040070 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 10 client.7200 fetch_fsmap learned FSMap version 78 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x564d4cc91e50 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120019070 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 78) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fc110063040 con 0x564d4ccab420 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 10 client.7200 fetch_fsmap finished waiting for FSMap version 78 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 10 client.7200 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 20 client.7200 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cca63d0 0x564d4cca67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:31.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 4 client.7200 mds_command: new command op to 7011 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 --> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564d4cbdcf40 con 0x564d4cca63d0 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cca63d0 0x564d4cca67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 10 client.7200 ms_handle_connect on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc126ffd640 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cca63d0 0x564d4cca67b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4b166b442d97856 server_cookie=1bac68b7396c8995 in_seq=0 out_seq=0 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 1 -- 172.21.15.17:0/1582501222 <== mds.0 v2:172.21.15.17:6832/3941058610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564d4cbdcf40 con 0x564d4cca63d0 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc105ffb640 10 client.7200 handle_command_reply: tid=0 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.754+0000 7fc13036f740 1 client.7200 shutdown 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 msgr2=0x564d4ccaa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc1100090b0 tx=0x7fc110008030 comp rx=0 tx=0).stop 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 shutdown_connections 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:6832/3941058610,v1:172.21.15.17:6835/3941058610] conn(0x564d4cca63d0 0x564d4cca67b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564d4ccba800 0x564d4cd1e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc105ffb640 0 client.7200 ms_handle_reset on v2:172.21.15.17:6832/3941058610 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564d4ccab420 0x564d4ccaa2e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 --2- 172.21.15.17:0/1582501222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564d4cc8ee80 0x564d4ccae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 >> 172.21.15.17:0/1582501222 conn(0x564d4cc71550 msgr2=0x564d4cc90990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 shutdown_connections 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 1 -- 172.21.15.17:0/1582501222 wait complete. 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc13036f740 20 client.7200 trim_cache size 0 max 16384 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7011, 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:23:31.764 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 6.3838979130000002, 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 78, 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 96, 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 96, 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11.448746584 2024-03-06T07:23:31.765 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 msgr2=0x7fc10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 0x7fc10806d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc110036000 tx=0x7fc110030000 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 msgr2=0x7fc1281e82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c7f0 tx=0x7fc11800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 shutdown_connections 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc10806b4d0 0x7fc10806d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1281504c0 0x7fc1281ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1280c4e80 0x7fc1280c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 --2- 172.21.15.17:0/2873154658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1280c2130 0x7fc1281e82d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:31.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 >> 172.21.15.17:0/2873154658 conn(0x7fc1280c7de0 msgr2=0x7fc128148d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:31.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.755+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 shutdown_connections 2024-03-06T07:23:31.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:31.756+0000 7fc12d6a4640 1 -- 172.21.15.17:0/2873154658 wait complete. 2024-03-06T07:23:31.767 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 78, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.744389+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 78, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7011": { "addr": "172.21.15.17:6835/3941058610", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3941058610, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 3941058610, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7011, "incarnation": 76, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:26.737853+0000", "qdb_cluster": [ 7011 ], "qdb_leader": 7011, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7011 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/2259557388", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2259557388, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2259557388, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5963, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6836/3720096092", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3720096092, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3720096092, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6706, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2875090396", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 2875090396, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2875090396, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7008, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:31.769 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 7, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:23:31.769 INFO:tasks.cephfs.test_failover:current = [{'gid': 7008, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2875090396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2875090396}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2875090396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 6706, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/3720096092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3720096092}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3720096092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 7011, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/3941058610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3941058610}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 3941058610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 5963, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2259557388', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2259557388}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2259557388}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}] 2024-03-06T07:23:31.769 DEBUG:tasks.cephfs.test_failover:7 != -1 2024-03-06T07:23:31.769 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:23:32.862 INFO:tasks.ceph.mds.c.smithi017.stderr: -104> 2024-03-06T07:23:30.659+0000 7f8ff037a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:32.887 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:23:36.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.882+0000 7febe690f640 1 Processor -- start 2024-03-06T07:23:36.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 -- start start 2024-03-06T07:23:36.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febe00c7180 0x7febe0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febe00c88b0 0x7febe00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00c7c60 con 0x7febe01527b0 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe0159640 con 0x7febe00c88b0 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe690f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe00c8060 con 0x7febe00c7180 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febe4eac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febe00c88b0 0x7febe00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febdffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7febe01527b0 0x7febe0155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42818/0 (socket says 172.21.15.17:42818) 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febdffff640 1 -- 172.21.15.17:0/3121817132 learned_addr learned my addr 172.21.15.17:0/3121817132 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.883+0000 7febdf7fe640 1 --2- 172.21.15.17:0/3121817132 >> v2:172.21.15.163:3300/0 conn(0x7febe00c7180 0x7febe0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.884+0000 7febe4eac640 1 -- 172.21.15.17:0/3121817132 >> v2:172.21.15.163:3300/0 conn(0x7febe00c7180 msgr2=0x7febe0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.884+0000 7febe4eac640 1 --2- 172.21.15.17:0/3121817132 >> v2:172.21.15.163:3300/0 conn(0x7febe00c7180 0x7febe0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.884+0000 7febe4eac640 1 -- 172.21.15.17:0/3121817132 >> v1:172.21.15.17:6789/0 conn(0x7febe01527b0 legacy=0x7febe0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.884+0000 7febe4eac640 1 -- 172.21.15.17:0/3121817132 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00c76a0 con 0x7febe00c88b0 2024-03-06T07:23:36.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.884+0000 7febdeffd640 1 -- 172.21.15.17:0/3121817132 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450245425 0 0) 0x7febd4001110 con 0x7febe01527b0 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe4eac640 1 --2- 172.21.15.17:0/3121817132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c88b0 0x7febe00c5330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7febe008cac0 tx=0x7febd802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd50728d63e254c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febdeffd640 1 -- 172.21.15.17:0/3121817132 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8039070 con 0x7febe00c88b0 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febdeffd640 1 -- 172.21.15.17:0/3121817132 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd802cbf0 con 0x7febe00c88b0 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febdeffd640 1 -- 172.21.15.17:0/3121817132 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8034440 con 0x7febe00c88b0 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 -- 172.21.15.17:0/3121817132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c88b0 msgr2=0x7febe00c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 --2- 172.21.15.17:0/3121817132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c88b0 0x7febe00c5330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7febe008cac0 tx=0x7febd802be40 comp rx=0 tx=0).stop 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 -- 172.21.15.17:0/3121817132 shutdown_connections 2024-03-06T07:23:36.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 --2- 172.21.15.17:0/3121817132 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c88b0 0x7febe00c5330 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 --2- 172.21.15.17:0/3121817132 >> v2:172.21.15.163:3300/0 conn(0x7febe00c7180 0x7febe0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 -- 172.21.15.17:0/3121817132 >> 172.21.15.17:0/3121817132 conn(0x7febe00ac560 msgr2=0x7febe00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 -- 172.21.15.17:0/3121817132 shutdown_connections 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.885+0000 7febe690f640 1 -- 172.21.15.17:0/3121817132 wait complete. 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.886+0000 7febe690f640 1 Processor -- start 2024-03-06T07:23:36.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 -- start start 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88b0 0x7febe0154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe01527b0 0x7febe01eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0155600 con 0x7febe00c7180 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0155320 con 0x7febe01527b0 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.887+0000 7febe690f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe0152b90 con 0x7febe00c88b0 2024-03-06T07:23:36.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47458/0 (socket says 172.21.15.17:47458) 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 -- 172.21.15.17:0/1126902235 learned_addr learned my addr 172.21.15.17:0/1126902235 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febe4eac640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe01527b0 0x7febe01eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdf7fe640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88b0 0x7febe0154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 -- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88b0 msgr2=0x7febe0154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88b0 0x7febe0154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 -- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe01527b0 msgr2=0x7febe01eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe01527b0 0x7febe01eb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 -- 172.21.15.17:0/1126902235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0157ee0 con 0x7febe00c7180 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdffff640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7febd400c460 tx=0x7febd400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4018070 con 0x7febe00c7180 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd4007500 con 0x7febe00c7180 2024-03-06T07:23:36.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0157920 con 0x7febe00c7180 2024-03-06T07:23:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4004a20 con 0x7febe00c7180 2024-03-06T07:23:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.888+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00b0250 con 0x7febe00c7180 2024-03-06T07:23:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.890+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe004f330 con 0x7febe00c7180 2024-03-06T07:23:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.890+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febd401b070 con 0x7febe00c7180 2024-03-06T07:23:36.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.890+0000 7febdcff9640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:36.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.890+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7febd4089500 con 0x7febe00c7180 2024-03-06T07:23:36.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.890+0000 7febdf7fe640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:36.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.891+0000 7febdf7fe640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 0x7febcc06d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7febe0158a30 tx=0x7febd0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:36.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:36.895+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febe00c7c60 con 0x7febe00c7180 2024-03-06T07:23:37.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.113+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febe006ce00 con 0x7febe00c7180 2024-03-06T07:23:37.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.114+0000 7febdcff9640 1 -- 172.21.15.17:0/1126902235 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4985 (secure 0 0 0) 0x7febd4057be0 con 0x7febe00c7180 2024-03-06T07:23:37.117 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:37.117 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:35.864598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5963,"qdb_cluster":[5963]},"id":7}]} 2024-03-06T07:23:37.117 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 msgr2=0x7febcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 0x7febcc06d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7febe0158a30 tx=0x7febd0009190 comp rx=0 tx=0).stop 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 msgr2=0x7febe01eecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7febd400c460 tx=0x7febd400c920 comp rx=0 tx=0).stop 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 shutdown_connections 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe01527b0 0x7febe01eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88b0 0x7febe0154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 --2- 172.21.15.17:0/1126902235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe00c7180 0x7febe01eecf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 >> 172.21.15.17:0/1126902235 conn(0x7febe00ac560 msgr2=0x7febe01507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 shutdown_connections 2024-03-06T07:23:37.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.116+0000 7febe690f640 1 -- 172.21.15.17:0/1126902235 wait complete. 2024-03-06T07:23:37.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:24.744389+0000', 'modified': '2024-03-06T07:23:35.864598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5963}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5963': {'gid': 5963, 'name': 'd', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6837/2259557388', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2259557388}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2259557388}]}, 'join_fscid': 7, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5963, 'qdb_cluster': [5963]} 2024-03-06T07:23:37.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:37.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:23:37.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.637+0000 7f6a9e908640 1 Processor -- start 2024-03-06T07:23:37.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.637+0000 7f6a9e908640 1 -- start start 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9e908640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a9806ff10 0x7f6a98153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9e908640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c8f20 0x7f6a98154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9e908640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a980a8390 con 0x7f6a980c7170 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9e908640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980c7620 con 0x7f6a9806ff10 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9e908640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980730f0 con 0x7f6a980c8f20 2024-03-06T07:23:37.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a977fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6a980c7170 0x7f6a980c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:38836/0 (socket says 172.21.15.17:38836) 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a977fe640 1 -- 172.21.15.17:0/3803443227 learned_addr learned my addr 172.21.15.17:0/3803443227 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a97fff640 1 --2- 172.21.15.17:0/3803443227 >> v2:172.21.15.163:3300/0 conn(0x7f6a9806ff10 0x7f6a98153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.638+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c8f20 0x7f6a98154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.639+0000 7f6a97fff640 1 -- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c8f20 msgr2=0x7f6a98154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.639+0000 7f6a97fff640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c8f20 0x7f6a98154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.639+0000 7f6a97fff640 1 -- 172.21.15.17:0/3803443227 >> v1:172.21.15.163:6789/0 conn(0x7f6a980c7170 legacy=0x7f6a980c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.639+0000 7f6a97fff640 1 -- 172.21.15.17:0/3803443227 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980733d0 con 0x7f6a9806ff10 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.639+0000 7f6a97fff640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a9806ff10 0x7f6a98153c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097a0 tx=0x7f6a8802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f341467cb7fd1c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a96ffd640 1 -- 172.21.15.17:0/3803443227 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88038070 con 0x7f6a9806ff10 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a96ffd640 1 -- 172.21.15.17:0/3803443227 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a88031700 con 0x7f6a9806ff10 2024-03-06T07:23:37.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a96ffd640 1 -- 172.21.15.17:0/3803443227 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88031ad0 con 0x7f6a9806ff10 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 -- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a9806ff10 msgr2=0x7f6a98153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a9806ff10 0x7f6a98153c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097a0 tx=0x7f6a8802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 -- 172.21.15.17:0/3803443227 shutdown_connections 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c8f20 0x7f6a98154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 --2- 172.21.15.17:0/3803443227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a9806ff10 0x7f6a98153c20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 -- 172.21.15.17:0/3803443227 >> 172.21.15.17:0/3803443227 conn(0x7f6a9806b970 msgr2=0x7f6a980aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 -- 172.21.15.17:0/3803443227 shutdown_connections 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.640+0000 7f6a9e908640 1 -- 172.21.15.17:0/3803443227 wait complete. 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.641+0000 7f6a9e908640 1 Processor -- start 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 -- start start 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c7170 0x7f6a980bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a980c8f20 0x7f6a98149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98070470 con 0x7f6a9806ff10 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980702f0 con 0x7f6a980c8f20 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9e908640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980c6860 con 0x7f6a980c7170 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47464/0 (socket says 172.21.15.17:47464) 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a97fff640 1 -- 172.21.15.17:0/1403964835 learned_addr learned my addr 172.21.15.17:0/1403964835 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a980c8f20 0x7f6a98149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.642+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c7170 0x7f6a980bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 -- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c7170 msgr2=0x7f6a980bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c7170 0x7f6a980bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 -- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a980c8f20 msgr2=0x7f6a98149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a980c8f20 0x7f6a98149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 -- 172.21.15.17:0/1403964835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980ad110 con 0x7f6a9806ff10 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098c0 tx=0x7f6a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a94ff9640 1 -- 172.21.15.17:0/1403964835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88038040 con 0x7f6a9806ff10 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a94ff9640 1 -- 172.21.15.17:0/1403964835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8803c6d0 con 0x7f6a9806ff10 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a980acb50 con 0x7f6a9806ff10 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a94ff9640 1 -- 172.21.15.17:0/1403964835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8803caa0 con 0x7f6a9806ff10 2024-03-06T07:23:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.643+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a981564d0 con 0x7f6a9806ff10 2024-03-06T07:23:37.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.645+0000 7f6a94ff9640 1 -- 172.21.15.17:0/1403964835 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a88045020 con 0x7f6a9806ff10 2024-03-06T07:23:37.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.645+0000 7f6a94ff9640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 0x7f6a8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.645+0000 7f6a94ff9640 1 -- 172.21.15.17:0/1403964835 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a880ab210 con 0x7f6a9806ff10 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.645+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 0x7f6a8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.646+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 0x7f6a8406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a980c89d0 tx=0x7f6a8c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.646+0000 7f6aa15c5740 1 Processor -- start 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.646+0000 7f6aa15c5740 1 -- start start 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf8fb7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb84b0 0x559cf8fb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf8f1e6b0 con 0x559cf8fba330 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf8ec7a60 con 0x559cf8fb9f50 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cf8f1a750 con 0x559cf8fb84b0 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47480/0 (socket says 172.21.15.17:47480) 2024-03-06T07:23:37.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 -- 172.21.15.17:0/733478456 learned_addr learned my addr 172.21.15.17:0/733478456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb84b0 0x559cf8fb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a97fff640 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf8fb7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 -- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb84b0 msgr2=0x559cf8fb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb84b0 0x559cf8fb4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 -- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 msgr2=0x559cf8fb7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf8fb7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 -- 172.21.15.17:0/733478456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf8ee5f30 con 0x559cf8fba330 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a977fe640 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000f60 tx=0x7f6a8c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd0b3c74a5943f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a7dffb640 1 -- 172.21.15.17:0/733478456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c026070 con 0x559cf8fba330 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a7dffb640 1 -- 172.21.15.17:0/733478456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c013870 con 0x559cf8fba330 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.647+0000 7f6a7dffb640 1 -- 172.21.15.17:0/733478456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c013c20 con 0x559cf8fba330 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 -- 172.21.15.17:0/733478456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 msgr2=0x559cf8fb7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c000f60 tx=0x7f6a8c013000 comp rx=0 tx=0).stop 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 -- 172.21.15.17:0/733478456 shutdown_connections 2024-03-06T07:23:37.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb84b0 0x559cf8fb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fb7f70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/733478456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf8fb7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 -- 172.21.15.17:0/733478456 >> 172.21.15.17:0/733478456 conn(0x559cf8fb67d0 msgr2=0x559cf8fdb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 -- 172.21.15.17:0/733478456 shutdown_connections 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.648+0000 7f6aa15c5740 1 -- 172.21.15.17:0/733478456 wait complete. 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 -- start start 2024-03-06T07:23:37.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 Processor -- start 2024-03-06T07:23:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fd8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 0x559cf900a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf8fdc230 con 0x559cf8fba330 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf8edaea0 con 0x559cf900a0e0 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.649+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cf8fdbc70 con 0x559cf8fb9f50 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6aa15c5740 10 client.0 _ll_register_callbacks cb 0x559cf8eda870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47488/0 (socket says 172.21.15.17:47488) 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 -- 172.21.15.17:0/1197833871 learned_addr learned my addr 172.21.15.17:0/1197833871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 0x559cf900a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fd8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 -- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 msgr2=0x559cf8fd8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fd8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 -- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 msgr2=0x559cf900a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 0x559cf900a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 -- 172.21.15.17:0/1197833871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf8edb270 con 0x559cf8fba330 2024-03-06T07:23:37.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 0x559cf900a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0078c0 tx=0x7f6a8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87df74c59f5b2881 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c030060 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a8c022070 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a8c013870 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a8c036050 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6aa15c5740 10 client.7224 fetch_fsmap learned FSMap version 83 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.650+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x559cf8c332d0 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8c013870 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 83) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6a8c039020 con 0x559cf8fba330 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 10 client.7224 fetch_fsmap finished waiting for FSMap version 83 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 10 client.7224 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 20 client.7224 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8ff9b40 0x559cf8ff9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 4 client.7224 mds_command: new command op to 5963 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 --> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cf8f2bbf0 con 0x559cf8ff9b40 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8ff9b40 0x559cf8ff9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8ff9b40 0x559cf8ff9f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae740e1150c1c5b2 server_cookie=d17a6841b0bf0ecb in_seq=0 out_seq=0 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.651+0000 7f6a76ffd640 10 client.7224 ms_handle_connect on v2:172.21.15.163:6835/2259557388 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.652+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1197833871 <== mds.0 v2:172.21.15.163:6835/2259557388 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559cf8f2bbf0 con 0x559cf8ff9b40 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.652+0000 7f6a76ffd640 10 client.7224 handle_command_reply: tid=0 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.652+0000 7f6aa15c5740 1 client.7224 shutdown 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 msgr2=0x559cf8fdf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c0078c0 tx=0x7f6a8c000bc0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 shutdown_connections 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8ff9b40 0x559cf8ff9f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf900a0e0 0x559cf900a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6a76ffd640 0 client.7224 ms_handle_reset on v2:172.21.15.163:6835/2259557388 2024-03-06T07:23:37.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf8fba330 0x559cf8fdf040 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1197833871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fd8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 >> 172.21.15.17:0/1197833871 conn(0x559cf8fb84b0 msgr2=0x559cf8fb5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 shutdown_connections 2024-03-06T07:23:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1197833871 wait complete. 2024-03-06T07:23:37.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.653+0000 7f6aa15c5740 20 client.7224 trim_cache size 0 max 16384 2024-03-06T07:23:37.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.669+0000 7f6aa15c5740 1 Processor -- start 2024-03-06T07:23:37.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 -- start start 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf9004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fba330 0x559cf8fb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981f5ca0 con 0x559cf900a0e0 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981f4df0 con 0x559cf8fb9f50 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981f4380 con 0x559cf8fba330 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6a9cea5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6a9cea5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47490/0 (socket says 172.21.15.17:47490) 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1024404776 learned_addr learned my addr 172.21.15.17:0/1024404776 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6a977fe640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fba330 0x559cf8fb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.670+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf9004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fba330 msgr2=0x559cf8fb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fba330 0x559cf8fb3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 msgr2=0x559cf9004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf9004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1024404776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf8cfd6b0 con 0x559cf900a0e0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf9004060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6a90014a70 tx=0x7f6a90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f914c94da5979d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a7d7fa640 1 -- 172.21.15.17:0/1024404776 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90003990 con 0x559cf900a0e0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a7d7fa640 1 -- 172.21.15.17:0/1024404776 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90003b30 con 0x559cf900a0e0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6a7d7fa640 1 -- 172.21.15.17:0/1024404776 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90004030 con 0x559cf900a0e0 2024-03-06T07:23:37.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1024404776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 msgr2=0x559cf8fb88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f6a90014a70 tx=0x7f6a90009750 comp rx=0 tx=0).stop 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1024404776 shutdown_connections 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fb88e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fba330 0x559cf8fb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1024404776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fb9f50 0x559cf9004060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1024404776 >> 172.21.15.17:0/1024404776 conn(0x559cf8fb84b0 msgr2=0x559cf8fafe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1024404776 shutdown_connections 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.671+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1024404776 wait complete. 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.672+0000 7f6aa15c5740 1 -- start start 2024-03-06T07:23:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.672+0000 7f6aa15c5740 1 Processor -- start 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fb8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fba330 0x559cf906d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf900ab90 con 0x559cf900a0e0 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981f4820 con 0x559cf8fba330 2024-03-06T07:23:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cf90067b0 con 0x559cf8fb9f50 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6aa15c5740 10 client.0 _ll_register_callbacks cb 0x7f6a90009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a977fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fba330 0x559cf906d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47504/0 (socket says 172.21.15.17:47504) 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1204297837 learned_addr learned my addr 172.21.15.17:0/1204297837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fb8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 msgr2=0x559cf8fb8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fb8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fba330 msgr2=0x559cf906d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fba330 0x559cf906d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 -- 172.21.15.17:0/1204297837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cf8fbb6d0 con 0x559cf900a0e0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a9cea5640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6a90014a40 tx=0x7f6a90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1c5b2d42dd1fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.673+0000 7f6a76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017e60 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a90004440 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a90003b30 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a90038050 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 10 client.7230 fetch_fsmap learned FSMap version 83 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x559cf906d7e0 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90003b30 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 83) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6a9003b020 con 0x559cf900a0e0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 10 client.7230 fetch_fsmap finished waiting for FSMap version 83 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 10 client.7230 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 20 client.7230 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8fdd1b0 0x559cf9066930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 4 client.7230 mds_command: new command op to 5963 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 --> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559cf8f25f40 con 0x559cf8fdd1b0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.674+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8fdd1b0 0x559cf9066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6a76ffd640 10 client.7230 ms_handle_connect on v2:172.21.15.163:6835/2259557388 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6a97fff640 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8fdd1b0 0x559cf9066930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979ad1818bf83afc server_cookie=a77fa7d43dcab848 in_seq=0 out_seq=0 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6a76ffd640 1 -- 172.21.15.17:0/1204297837 <== mds.0 v2:172.21.15.163:6835/2259557388 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559cf8f25f40 con 0x559cf8fdd1b0 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6a76ffd640 10 client.7230 handle_command_reply: tid=0 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 client.7230 shutdown 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 msgr2=0x559cf8fdc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6a90014a40 tx=0x7f6a90004ab0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 shutdown_connections 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:6835/2259557388,v1:172.21.15.163:6837/2259557388] conn(0x559cf8fdd1b0 0x559cf9066930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559cf900a0e0 0x559cf8fdc600 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559cf8fba330 0x559cf906d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6a76ffd640 0 client.7230 ms_handle_reset on v2:172.21.15.163:6835/2259557388 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 --2- 172.21.15.17:0/1204297837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559cf8fb9f50 0x559cf8fb8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 >> 172.21.15.17:0/1204297837 conn(0x559cf8fb84b0 msgr2=0x559cf8fb3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 shutdown_connections 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 1 -- 172.21.15.17:0/1204297837 wait complete. 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.675+0000 7f6aa15c5740 20 client.7230 trim_cache size 0 max 16384 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5963, 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:23:37.685 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.8179793870000003, 2024-03-06T07:23:37.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 83, 2024-03-06T07:23:37.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 97, 2024-03-06T07:23:37.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 97, 2024-03-06T07:23:37.686 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 17.363467652000001 2024-03-06T07:23:37.686 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 msgr2=0x7f6a8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 0x7f6a8406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a980c89d0 tx=0x7f6a8c007e80 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 msgr2=0x7f6a981ec7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6a880098c0 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 shutdown_connections 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a8406b500 0x7f6a8406d9c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a980c8f20 0x7f6a98149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a980c7170 0x7f6a980bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 --2- 172.21.15.17:0/1403964835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a9806ff10 0x7f6a981ec7f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 >> 172.21.15.17:0/1403964835 conn(0x7f6a9806b970 msgr2=0x7f6a981539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 shutdown_connections 2024-03-06T07:23:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.676+0000 7f6a9e908640 1 -- 172.21.15.17:0/1403964835 wait complete. 2024-03-06T07:23:37.689 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 7, "epoch": 83, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 7, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:24.744389+0000", "damaged": [], "data_pools": [ 16 ], "enabled": true, "epoch": 83, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5963": { "addr": "172.21.15.163:6837/2259557388", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2259557388, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2259557388, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5963, "incarnation": 80, "join_fscid": 7, "name": "d", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 97, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 15, "modified": "2024-03-06T07:23:35.864598+0000", "qdb_cluster": [ 5963 ], "qdb_leader": 5963, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5963 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/3720096092", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3720096092, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3720096092, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6706, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2875090396", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 2875090396, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2875090396, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7008, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/2336694958", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2336694958, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 2336694958, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 81, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7206, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:37.690 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 7, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:23:37.690 INFO:tasks.cephfs.test_failover:current = [{'gid': 7008, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2875090396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2875090396}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2875090396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 6706, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/3720096092', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3720096092}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3720096092}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 75}, {'gid': 7206, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/2336694958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2336694958}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2336694958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 81}, {'gid': 5963, 'name': 'd', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6837/2259557388', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2259557388}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2259557388}]}, 'join_fscid': 7, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:23:37.690 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:23:37.690 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:37.691 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:37.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:37.778 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:37.778 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:23:37.778 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:37.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:37.793 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:23:37.793 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:37.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:37.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.933+0000 7f90c3acf640 1 Processor -- start 2024-03-06T07:23:37.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.933+0000 7f90c3acf640 1 -- start start 2024-03-06T07:23:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c3acf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 0x7f90bc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c3acf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c3acf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc0c28d0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c3acf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc0c2680 con 0x7f90bc14f790 2024-03-06T07:23:37.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c3acf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c2bb0 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c206c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f90bc0c5c60 0x7f90bc155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42824/0 (socket says 172.21.15.17:42824) 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c206c640 1 -- 172.21.15.17:0/1246778616 learned_addr learned my addr 172.21.15.17:0/1246778616 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c106a640 1 --2- 172.21.15.17:0/1246778616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c186b640 1 --2- 172.21.15.17:0/1246778616 >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 0x7f90bc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c0869640 1 -- 172.21.15.17:0/1246778616 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107660551 0 0) 0x7f90bc0c28d0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c0869640 1 -- 172.21.15.17:0/1246778616 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a4003520 con 0x7f90bc0c5c60 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c106a640 1 -- 172.21.15.17:0/1246778616 >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 msgr2=0x7f90bc1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c106a640 1 --2- 172.21.15.17:0/1246778616 >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 0x7f90bc1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c106a640 1 -- 172.21.15.17:0/1246778616 >> v1:172.21.15.17:6789/0 conn(0x7f90bc0c5c60 legacy=0x7f90bc155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c106a640 1 -- 172.21.15.17:0/1246778616 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc0c2e90 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.934+0000 7f90c186b640 1 --2- 172.21.15.17:0/1246778616 >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 0x7f90bc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.935+0000 7f90c106a640 1 --2- 172.21.15.17:0/1246778616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0c5720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90b00097a0 tx=0x7f90b002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1718d17be05f5b30 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.935+0000 7f90c0869640 1 -- 172.21.15.17:0/1246778616 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0035020 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.935+0000 7f90c0869640 1 -- 172.21.15.17:0/1246778616 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0031830 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.935+0000 7f90c0869640 1 -- 172.21.15.17:0/1246778616 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0031be0 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 -- 172.21.15.17:0/1246778616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 msgr2=0x7f90bc0c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 --2- 172.21.15.17:0/1246778616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0c5720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90b00097a0 tx=0x7f90b002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 -- 172.21.15.17:0/1246778616 shutdown_connections 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 --2- 172.21.15.17:0/1246778616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0c5720 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 --2- 172.21.15.17:0/1246778616 >> v2:172.21.15.163:3300/0 conn(0x7f90bc14f790 0x7f90bc1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 -- 172.21.15.17:0/1246778616 >> 172.21.15.17:0/1246778616 conn(0x7f90bc0c6710 msgr2=0x7f90bc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 -- 172.21.15.17:0/1246778616 shutdown_connections 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.936+0000 7f90c3acf640 1 -- 172.21.15.17:0/1246778616 wait complete. 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.937+0000 7f90c3acf640 1 Processor -- start 2024-03-06T07:23:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.937+0000 7f90c3acf640 1 -- start start 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90bc14f790 0x7f90bc0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc06cc00 con 0x7f90bc0c5c60 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc0c0d60 con 0x7f90bc14f790 2024-03-06T07:23:37.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c3acf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc06cd80 con 0x7f90bc0c1aa0 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47514/0 (socket says 172.21.15.17:47514) 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 -- 172.21.15.17:0/2823781400 learned_addr learned my addr 172.21.15.17:0/2823781400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c186b640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c206c640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90bc14f790 0x7f90bc0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 -- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 msgr2=0x7f90bc0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 -- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90bc14f790 msgr2=0x7f90bc0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90bc14f790 0x7f90bc0705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 -- 172.21.15.17:0/2823781400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc14fdc0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c186b640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.938+0000 7f90c106a640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f90b0000c00 tx=0x7f90b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0048070 con 0x7f90bc0c5c60 2024-03-06T07:23:37.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b003a6f0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc1500a0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b003aae0 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc150660 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.939+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc069460 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.940+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90b004b070 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.940+0000 7f90ba7fc640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.941+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f90b00aba00 con 0x7f90bc0c5c60 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.941+0000 7f90c186b640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:37.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.941+0000 7f90c186b640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 0x7f90a806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009560 tx=0x7f90b4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:37.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:37.945+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90bc069460 con 0x7f90bc0c5c60 2024-03-06T07:23:38.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.162+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90bc04f330 con 0x7f90bc0c5c60 2024-03-06T07:23:38.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.162+0000 7f90ba7fc640 1 -- 172.21.15.17:0/2823781400 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f90b007a0e0 con 0x7f90bc0c5c60 2024-03-06T07:23:38.165 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:38.165 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:35.864598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5963,"qdb_cluster":[5963]},"id":7}]} 2024-03-06T07:23:38.165 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 msgr2=0x7f90a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 0x7f90a806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009560 tx=0x7f90b4009190 comp rx=0 tx=0).stop 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 msgr2=0x7f90bc0c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f90b0000c00 tx=0x7f90b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 shutdown_connections 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90a806b4d0 0x7f90a806d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90bc14f790 0x7f90bc0705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90bc0c5c60 0x7f90bc0c3830 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 --2- 172.21.15.17:0/2823781400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90bc0c1aa0 0x7f90bc0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 >> 172.21.15.17:0/2823781400 conn(0x7f90bc0c6710 msgr2=0x7f90bc0c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 shutdown_connections 2024-03-06T07:23:38.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.164+0000 7f90c3acf640 1 -- 172.21.15.17:0/2823781400 wait complete. 2024-03-06T07:23:38.178 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.277+0000 7faaf9876640 1 Processor -- start 2024-03-06T07:23:38.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.277+0000 7faaf9876640 1 -- start start 2024-03-06T07:23:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf9876640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faaf4150580 0x7faaf406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf9876640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faaf40c1320 0x7faaf406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf9876640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf4153a60 con 0x7faaf40c6000 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf9876640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf4153d40 con 0x7faaf4150580 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf9876640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf406c2c0 con 0x7faaf40c1320 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faaf4150580 0x7faaf406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faaf4150580 0x7faaf406dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47522/0 (socket says 172.21.15.17:47522) 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 -- 172.21.15.17:0/2287369597 learned_addr learned my addr 172.21.15.17:0/2287369597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaebfff640 1 --2- 172.21.15.17:0/2287369597 >> v2:172.21.15.163:3300/0 conn(0x7faaf40c1320 0x7faaf406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 -- 172.21.15.17:0/2287369597 >> v2:172.21.15.163:3300/0 conn(0x7faaf40c1320 msgr2=0x7faaf406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 --2- 172.21.15.17:0/2287369597 >> v2:172.21.15.163:3300/0 conn(0x7faaf40c1320 0x7faaf406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 -- 172.21.15.17:0/2287369597 >> v1:172.21.15.163:6789/0 conn(0x7faaf40c6000 legacy=0x7faaf41566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.278+0000 7faaf2ffd640 1 -- 172.21.15.17:0/2287369597 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf406c5a0 con 0x7faaf4150580 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf2ffd640 1 --2- 172.21.15.17:0/2287369597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf4150580 0x7faaf406dd10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7faaf40c7730 tx=0x7faae402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8527000272499c59 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf27fc640 1 -- 172.21.15.17:0/2287369597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4035020 con 0x7faaf4150580 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf27fc640 1 -- 172.21.15.17:0/2287369597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae4031870 con 0x7faaf4150580 2024-03-06T07:23:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf27fc640 1 -- 172.21.15.17:0/2287369597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4031c40 con 0x7faaf4150580 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf9876640 1 -- 172.21.15.17:0/2287369597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf4150580 msgr2=0x7faaf406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.279+0000 7faaf9876640 1 --2- 172.21.15.17:0/2287369597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf4150580 0x7faaf406dd10 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7faaf40c7730 tx=0x7faae402b910 comp rx=0 tx=0).stop 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 -- 172.21.15.17:0/2287369597 shutdown_connections 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 --2- 172.21.15.17:0/2287369597 >> v2:172.21.15.163:3300/0 conn(0x7faaf40c1320 0x7faaf406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 --2- 172.21.15.17:0/2287369597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf4150580 0x7faaf406dd10 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 -- 172.21.15.17:0/2287369597 >> 172.21.15.17:0/2287369597 conn(0x7faaf40c6710 msgr2=0x7faaf40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 -- 172.21.15.17:0/2287369597 shutdown_connections 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.280+0000 7faaf9876640 1 -- 172.21.15.17:0/2287369597 wait complete. 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 Processor -- start 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 -- start start 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaf40c6000 0x7faaf41ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaf4150580 0x7faaf4072f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40c0dc0 con 0x7faaf40c1320 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40c0c40 con 0x7faaf4150580 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf9876640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf4156dc0 con 0x7faaf40c6000 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47530/0 (socket says 172.21.15.17:47530) 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.281+0000 7faaf2ffd640 1 -- 172.21.15.17:0/306592534 learned_addr learned my addr 172.21.15.17:0/306592534 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf37fe640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaf4150580 0x7faaf4072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaebfff640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaf40c6000 0x7faaf41ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 -- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaf40c6000 msgr2=0x7faaf41ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaf40c6000 0x7faaf41ec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 -- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaf4150580 msgr2=0x7faaf4072f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaf4150580 0x7faaf4072f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 -- 172.21.15.17:0/306592534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf41ebf20 con 0x7faaf40c1320 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf2ffd640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4035040 con 0x7faaf40c1320 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae4030d00 con 0x7faaf40c1320 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae40456a0 con 0x7faaf40c1320 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf414f000 con 0x7faaf40c1320 2024-03-06T07:23:38.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.282+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf4156000 con 0x7faaf40c1320 2024-03-06T07:23:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.283+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf404f330 con 0x7faaf40c1320 2024-03-06T07:23:38.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.284+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faae4038020 con 0x7faaf40c1320 2024-03-06T07:23:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.284+0000 7faaf0ff9640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 0x7faad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.284+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faae40ac4a0 con 0x7faaf40c1320 2024-03-06T07:23:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.284+0000 7faaebfff640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 0x7faad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.285+0000 7faaebfff640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 0x7faad406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faaf40c2fb0 tx=0x7faadc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.288+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faae407ab80 con 0x7faaf40c1320 2024-03-06T07:23:38.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.471+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaf40c5de0 con 0x7faaf40c1320 2024-03-06T07:23:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.472+0000 7faaf0ff9640 1 -- 172.21.15.17:0/306592534 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17002 (secure 0 0 0) 0x7faae407ec20 con 0x7faaf40c1320 2024-03-06T07:23:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:38.475 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":97,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:32.856523+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/3941058610":"2024-03-07T07:23:32.856494+0000","172.21.15.17:6832/3941058610":"2024-03-07T07:23:32.856494+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 msgr2=0x7faad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 0x7faad406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7faaf40c2fb0 tx=0x7faadc009190 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 msgr2=0x7faaf4145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae40046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 shutdown_connections 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faad406b4d0 0x7faad406d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faaf4150580 0x7faaf4072f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faaf40c6000 0x7faaf41ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 --2- 172.21.15.17:0/306592534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faaf40c1320 0x7faaf4145ab0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 >> 172.21.15.17:0/306592534 conn(0x7faaf40c6710 msgr2=0x7faaf40c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 shutdown_connections 2024-03-06T07:23:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.474+0000 7faaf9876640 1 -- 172.21.15.17:0/306592534 wait complete. 2024-03-06T07:23:38.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:23:23.000014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:23:23.700451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:38.488 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:38.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.589+0000 7ff2e16df640 1 Processor -- start 2024-03-06T07:23:38.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.589+0000 7ff2e16df640 1 -- start start 2024-03-06T07:23:38.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2e16df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2dc0c57d0 0x7ff2dc155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2e16df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc152dd0 con 0x7ff2dc1529f0 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2e16df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc158b00 con 0x7ff2dc0c5370 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2e16df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc159710 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2db7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff2dc0c57d0 0x7ff2dc155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2daffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2dc1529f0 0x7ff2dc0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42826/0 (socket says 172.21.15.17:42826) 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.590+0000 7ff2daffd640 1 -- 172.21.15.17:0/2760736413 learned_addr learned my addr 172.21.15.17:0/2760736413 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2db7fe640 1 -- 172.21.15.17:0/2760736413 >> v1:172.21.15.163:6789/0 conn(0x7ff2dc0c5370 legacy=0x7ff2dc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2db7fe640 1 -- 172.21.15.17:0/2760736413 >> v1:172.21.15.17:6789/0 conn(0x7ff2dc1529f0 legacy=0x7ff2dc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2d9ffb640 1 -- 172.21.15.17:0/2760736413 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433496835 0 0) 0x7ff2dc158b00 con 0x7ff2dc0c5370 2024-03-06T07:23:38.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2db7fe640 1 -- 172.21.15.17:0/2760736413 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc158de0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2db7fe640 1 --2- 172.21.15.17:0/2760736413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc155c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009d90 tx=0x7ff2cc02bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9282cf394a16a98 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.591+0000 7ff2d9ffb640 1 -- 172.21.15.17:0/2760736413 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc039070 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2d9ffb640 1 -- 172.21.15.17:0/2760736413 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc02ca20 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2d9ffb640 1 -- 172.21.15.17:0/2760736413 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc02cdf0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 -- 172.21.15.17:0/2760736413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 msgr2=0x7ff2dc155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2760736413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc155c40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009d90 tx=0x7ff2cc02bcf0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 -- 172.21.15.17:0/2760736413 shutdown_connections 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2760736413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc155c40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 -- 172.21.15.17:0/2760736413 >> 172.21.15.17:0/2760736413 conn(0x7ff2dc0ac580 msgr2=0x7ff2dc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 -- 172.21.15.17:0/2760736413 shutdown_connections 2024-03-06T07:23:38.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.592+0000 7ff2e16df640 1 -- 172.21.15.17:0/2760736413 wait complete. 2024-03-06T07:23:38.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.593+0000 7ff2e16df640 1 Processor -- start 2024-03-06T07:23:38.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.593+0000 7ff2e16df640 1 -- start start 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2dc0c5370 0x7ff2dc1eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2dc1529f0 0x7ff2dc144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c68b0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc0c9000 con 0x7ff2dc1529f0 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2e16df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1593a0 con 0x7ff2dc0c5370 2024-03-06T07:23:38.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47544/0 (socket says 172.21.15.17:47544) 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 -- 172.21.15.17:0/2758168080 learned_addr learned my addr 172.21.15.17:0/2758168080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2daffd640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2dc0c5370 0x7ff2dc1eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2db7fe640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2dc1529f0 0x7ff2dc144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 -- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2dc0c5370 msgr2=0x7ff2dc1eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2dc0c5370 0x7ff2dc1eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 -- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2dc1529f0 msgr2=0x7ff2dc144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2dc1529f0 0x7ff2dc144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.594+0000 7ff2da7fc640 1 -- 172.21.15.17:0/2758168080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0c00a0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2da7fc640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400c550 tx=0x7ff2c400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4005480 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40035f0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40039e0 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0c0380 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.595+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc0ad920 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.596+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2dc0ca010 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.596+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2c4003b80 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.597+0000 7ff2d37fe640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 0x7ff2bc06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.597+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff2c408a010 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.597+0000 7ff2daffd640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 0x7ff2bc06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.597+0000 7ff2daffd640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 0x7ff2bc06db30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4000f60 tx=0x7ff2d4007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.601+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2dc0ca010 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.783+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2dc0adb20 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.783+0000 7ff2d37fe640 1 -- 172.21.15.17:0/2758168080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2dc0adb20 con 0x7ff2dc0c57d0 2024-03-06T07:23:38.786 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 msgr2=0x7ff2bc06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 0x7ff2bc06db30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4000f60 tx=0x7ff2d4007e20 comp rx=0 tx=0).stop 2024-03-06T07:23:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 msgr2=0x7ff2dc157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400c550 tx=0x7ff2c400ca10 comp rx=0 tx=0).stop 2024-03-06T07:23:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 shutdown_connections 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2bc06b670 0x7ff2bc06db30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2dc1529f0 0x7ff2dc144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2dc0c57d0 0x7ff2dc157d80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 --2- 172.21.15.17:0/2758168080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2dc0c5370 0x7ff2dc1eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 >> 172.21.15.17:0/2758168080 conn(0x7ff2dc0ac580 msgr2=0x7ff2dc155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 shutdown_connections 2024-03-06T07:23:38.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.785+0000 7ff2e16df640 1 -- 172.21.15.17:0/2758168080 wait complete. 2024-03-06T07:23:38.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:23:38.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.901+0000 7f6c7fac9640 1 Processor -- start 2024-03-06T07:23:38.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.901+0000 7f6c7fac9640 1 -- start start 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7fac9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c7814bcc0 0x7f6c78155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7fac9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c78158a10 0x7f6c78158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7fac9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c780c75e0 con 0x7f6c780be150 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7fac9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c780c7f10 con 0x7f6c7814bcc0 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7fac9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c78c0 con 0x7f6c78158a10 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d865640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c7814bcc0 0x7f6c78155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d064640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c780be150 0x7f6c780c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42832/0 (socket says 172.21.15.17:42832) 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d064640 1 -- 172.21.15.17:0/1461609992 learned_addr learned my addr 172.21.15.17:0/1461609992 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7e066640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c78158a10 0x7f6c78158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d865640 1 -- 172.21.15.17:0/1461609992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c78158a10 msgr2=0x7f6c78158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:38.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d865640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c78158a10 0x7f6c78158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d865640 1 -- 172.21.15.17:0/1461609992 >> v1:172.21.15.17:6789/0 conn(0x7f6c780be150 legacy=0x7f6c780c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7c863640 1 -- 172.21.15.17:0/1461609992 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960879111 0 0) 0x7f6c780c75e0 con 0x7f6c780be150 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.902+0000 7f6c7d865640 1 -- 172.21.15.17:0/1461609992 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780c7ba0 con 0x7f6c7814bcc0 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.903+0000 7f6c7d865640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7814bcc0 0x7f6c78155a00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097a0 tx=0x7f6c7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea0d4d4e769554c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.903+0000 7f6c7c863640 1 -- 172.21.15.17:0/1461609992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70034070 con 0x7f6c7814bcc0 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.903+0000 7f6c7c863640 1 -- 172.21.15.17:0/1461609992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c70030610 con 0x7f6c7814bcc0 2024-03-06T07:23:38.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.903+0000 7f6c7c863640 1 -- 172.21.15.17:0/1461609992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700309e0 con 0x7f6c7814bcc0 2024-03-06T07:23:38.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 -- 172.21.15.17:0/1461609992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7814bcc0 msgr2=0x7f6c78155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7814bcc0 0x7f6c78155a00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097a0 tx=0x7f6c7002b960 comp rx=0 tx=0).stop 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 -- 172.21.15.17:0/1461609992 shutdown_connections 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c78158a10 0x7f6c78158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/1461609992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c7814bcc0 0x7f6c78155a00 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 -- 172.21.15.17:0/1461609992 >> 172.21.15.17:0/1461609992 conn(0x7f6c780ac600 msgr2=0x7f6c780aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 -- 172.21.15.17:0/1461609992 shutdown_connections 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.904+0000 7f6c7fac9640 1 -- 172.21.15.17:0/1461609992 wait complete. 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.905+0000 7f6c7fac9640 1 Processor -- start 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.905+0000 7f6c7fac9640 1 -- start start 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c780be150 0x7f6c781481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c7814bcc0 0x7f6c780c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c5c60 con 0x7f6c78158a10 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c780c5ae0 con 0x7f6c780be150 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7fac9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c781593c0 con 0x7f6c7814bcc0 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47566/0 (socket says 172.21.15.17:47566) 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 -- 172.21.15.17:0/848273235 learned_addr learned my addr 172.21.15.17:0/848273235 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7d064640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c7814bcc0 0x7f6c780c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7d865640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c780be150 0x7f6c781481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 -- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c7814bcc0 msgr2=0x7f6c780c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c7814bcc0 0x7f6c780c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 -- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c780be150 msgr2=0x7f6c781481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c780be150 0x7f6c781481d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:38.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 -- 172.21.15.17:0/848273235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780be840 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c7e066640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009d30 tx=0x7f6c6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68018020 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c68014480 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.906+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68014870 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.907+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c781eaf30 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.907+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c781eb4f0 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.907+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7804f330 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.908+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c68005d10 con 0x7f6c78158a10 2024-03-06T07:23:38.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.908+0000 7f6c767fc640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 0x7f6c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:38.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.909+0000 7f6c7d865640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 0x7f6c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:38.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.909+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c6808ac30 con 0x7f6c78158a10 2024-03-06T07:23:38.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.909+0000 7f6c7d865640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 0x7f6c6406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6c70036000 tx=0x7f6c7003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:38.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:38.913+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c68022070 con 0x7f6c78158a10 2024-03-06T07:23:39.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:39.021 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:23:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.094+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c780c83e0 con 0x7f6c78158a10 2024-03-06T07:23:39.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.095+0000 7f6c767fc640 1 -- 172.21.15.17:0/848273235 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c68010040 con 0x7f6c78158a10 2024-03-06T07:23:39.097 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 msgr2=0x7f6c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 0x7f6c6406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6c70036000 tx=0x7f6c7003b040 comp rx=0 tx=0).stop 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 msgr2=0x7f6c780bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009d30 tx=0x7f6c6800c490 comp rx=0 tx=0).stop 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 shutdown_connections 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c6406b4d0 0x7f6c6406d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c78158a10 0x7f6c780bf6d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c7814bcc0 0x7f6c780c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 --2- 172.21.15.17:0/848273235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c780be150 0x7f6c781481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 >> 172.21.15.17:0/848273235 conn(0x7f6c780ac600 msgr2=0x7f6c780bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 shutdown_connections 2024-03-06T07:23:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.097+0000 7f6c7fac9640 1 -- 172.21.15.17:0/848273235 wait complete. 2024-03-06T07:23:39.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:23:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.252+0000 7f88fd99a640 1 Processor -- start 2024-03-06T07:23:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.252+0000 7f88fd99a640 1 -- start start 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88fd99a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88f80b0070 0x7f88f8155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88fd99a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88f806ca70 0x7f88f80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88fd99a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80b0450 con 0x7f88f80c5380 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88fd99a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f806c020 con 0x7f88f80b0070 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88fd99a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f806cf20 con 0x7f88f806ca70 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88f80b0070 0x7f88f8155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88f80b0070 0x7f88f8155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47568/0 (socket says 172.21.15.17:47568) 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 -- 172.21.15.17:0/4176936820 learned_addr learned my addr 172.21.15.17:0/4176936820 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f67fc640 1 --2- 172.21.15.17:0/4176936820 >> v2:172.21.15.163:3300/0 conn(0x7f88f806ca70 0x7f88f80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 -- 172.21.15.17:0/4176936820 >> v2:172.21.15.163:3300/0 conn(0x7f88f806ca70 msgr2=0x7f88f80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 --2- 172.21.15.17:0/4176936820 >> v2:172.21.15.163:3300/0 conn(0x7f88f806ca70 0x7f88f80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 -- 172.21.15.17:0/4176936820 >> v1:172.21.15.163:6789/0 conn(0x7f88f80c5380 legacy=0x7f88f814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.253+0000 7f88f6ffd640 1 -- 172.21.15.17:0/4176936820 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f80c77f0 con 0x7f88f80b0070 2024-03-06T07:23:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.254+0000 7f88f6ffd640 1 --2- 172.21.15.17:0/4176936820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f80b0070 0x7f88f8155c30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f88f806d4c0 tx=0x7f88f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ced99f912831b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.254+0000 7f88f5ffb640 1 -- 172.21.15.17:0/4176936820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0035020 con 0x7f88f80b0070 2024-03-06T07:23:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.254+0000 7f88f5ffb640 1 -- 172.21.15.17:0/4176936820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f0031870 con 0x7f88f80b0070 2024-03-06T07:23:39.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.254+0000 7f88f5ffb640 1 -- 172.21.15.17:0/4176936820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0031c40 con 0x7f88f80b0070 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 -- 172.21.15.17:0/4176936820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f80b0070 msgr2=0x7f88f8155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 --2- 172.21.15.17:0/4176936820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f80b0070 0x7f88f8155c30 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f88f806d4c0 tx=0x7f88f002b910 comp rx=0 tx=0).stop 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 -- 172.21.15.17:0/4176936820 shutdown_connections 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 --2- 172.21.15.17:0/4176936820 >> v2:172.21.15.163:3300/0 conn(0x7f88f806ca70 0x7f88f80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 --2- 172.21.15.17:0/4176936820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f80b0070 0x7f88f8155c30 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 -- 172.21.15.17:0/4176936820 >> 172.21.15.17:0/4176936820 conn(0x7f88f80ac590 msgr2=0x7f88f80ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 -- 172.21.15.17:0/4176936820 shutdown_connections 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.255+0000 7f88fd99a640 1 -- 172.21.15.17:0/4176936820 wait complete. 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.256+0000 7f88fd99a640 1 Processor -- start 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.256+0000 7f88fd99a640 1 -- start start 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88f80b0070 0x7f88f81ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88f80c5380 0x7f88f81ee690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806c550 con 0x7f88f806ca70 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806c270 con 0x7f88f80b0070 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88fd99a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f806da80 con 0x7f88f80c5380 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47582/0 (socket says 172.21.15.17:47582) 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 -- 172.21.15.17:0/1330484091 learned_addr learned my addr 172.21.15.17:0/1330484091 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f77fe640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88f80c5380 0x7f88f81ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f67fc640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88f80b0070 0x7f88f81ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 -- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88f80c5380 msgr2=0x7f88f81ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88f80c5380 0x7f88f81ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 -- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88f80b0070 msgr2=0x7f88f81ee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88f80b0070 0x7f88f81ee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 -- 172.21.15.17:0/1330484091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f80c11d0 con 0x7f88f806ca70 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88f6ffd640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f88f002b950 tx=0x7f88f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.257+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0040070 con 0x7f88f806ca70 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.258+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f0030d00 con 0x7f88f806ca70 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.258+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f80c1490 con 0x7f88f806ca70 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.258+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0035040 con 0x7f88f806ca70 2024-03-06T07:23:39.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.258+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f814b860 con 0x7f88f806ca70 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.258+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f804f330 con 0x7f88f806ca70 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.259+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88f004b060 con 0x7f88f806ca70 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.259+0000 7f88effff640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.259+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f88f00ac010 con 0x7f88f806ca70 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.260+0000 7f88f67fc640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.260+0000 7f88f67fc640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 0x7f88d806d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f88f81554e0 tx=0x7f88e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.263+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88f007a670 con 0x7f88f806ca70 2024-03-06T07:23:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.443+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88f806d860 con 0x7f88f806ca70 2024-03-06T07:23:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.444+0000 7f88effff640 1 -- 172.21.15.17:0/1330484091 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88f007e710 con 0x7f88f806ca70 2024-03-06T07:23:39.446 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:23:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 msgr2=0x7f88d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 0x7f88d806d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f88f81554e0 tx=0x7f88e0008040 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 msgr2=0x7f88f814cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f88f002b950 tx=0x7f88f00046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 shutdown_connections 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88d806b4d0 0x7f88d806d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88f80c5380 0x7f88f81ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88f80b0070 0x7f88f81ee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 --2- 172.21.15.17:0/1330484091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f806ca70 0x7f88f814cd10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 >> 172.21.15.17:0/1330484091 conn(0x7f88f80ac590 msgr2=0x7f88f80ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 shutdown_connections 2024-03-06T07:23:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.446+0000 7f88fd99a640 1 -- 172.21.15.17:0/1330484091 wait complete. 2024-03-06T07:23:39.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.557+0000 7fbaabcf4640 1 Processor -- start 2024-03-06T07:23:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 -- start start 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 0x7fbaa41529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa4155a40 0x7fbaa4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa40c28d0 con 0x7fbaa40c1aa0 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40c2680 con 0x7fbaa4155a40 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaabcf4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40c2bb0 con 0x7fbaa414f790 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaaa291640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa4155a40 0x7fbaa4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaa9a90640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 0x7fbaa41529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.558+0000 7fbaaa291640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbaa4155a40 0x7fbaa4155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47590/0 (socket says 172.21.15.17:47590) 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaaa291640 1 -- 172.21.15.17:0/158178740 learned_addr learned my addr 172.21.15.17:0/158178740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaaa291640 1 -- 172.21.15.17:0/158178740 >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 msgr2=0x7fbaa41529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaaa291640 1 --2- 172.21.15.17:0/158178740 >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 0x7fbaa41529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaaa291640 1 -- 172.21.15.17:0/158178740 >> v1:172.21.15.163:6789/0 conn(0x7fbaa40c1aa0 legacy=0x7fbaa40c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaa8a8e640 1 -- 172.21.15.17:0/158178740 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651930927 0 0) 0x7fbaa40c28d0 con 0x7fbaa40c1aa0 2024-03-06T07:23:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaaa291640 1 -- 172.21.15.17:0/158178740 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40c2e90 con 0x7fbaa4155a40 2024-03-06T07:23:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.559+0000 7fbaa9a90640 1 --2- 172.21.15.17:0/158178740 >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 0x7fbaa41529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaaa291640 1 --2- 172.21.15.17:0/158178740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa4155a40 0x7fbaa4155e50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40c3170 tx=0x7fba9c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d898df5ad69b420 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaa8a8e640 1 -- 172.21.15.17:0/158178740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c02cb20 con 0x7fbaa4155a40 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaa8a8e640 1 -- 172.21.15.17:0/158178740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c02ccc0 con 0x7fbaa4155a40 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaa8a8e640 1 -- 172.21.15.17:0/158178740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c034610 con 0x7fbaa4155a40 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 -- 172.21.15.17:0/158178740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa4155a40 msgr2=0x7fbaa4155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/158178740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa4155a40 0x7fbaa4155e50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40c3170 tx=0x7fba9c02bf80 comp rx=0 tx=0).stop 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 -- 172.21.15.17:0/158178740 shutdown_connections 2024-03-06T07:23:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/158178740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa4155a40 0x7fbaa4155e50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/158178740 >> v2:172.21.15.163:3300/0 conn(0x7fbaa414f790 0x7fbaa41529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 -- 172.21.15.17:0/158178740 >> 172.21.15.17:0/158178740 conn(0x7fbaa40c6710 msgr2=0x7fbaa40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 -- 172.21.15.17:0/158178740 shutdown_connections 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.560+0000 7fbaabcf4640 1 -- 172.21.15.17:0/158178740 wait complete. 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.561+0000 7fbaabcf4640 1 Processor -- start 2024-03-06T07:23:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 -- start start 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 0x7fbaa4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbaa4155a40 0x7fbaa41544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa40c0d60 con 0x7fbaa414f790 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa4156390 con 0x7fbaa40c1aa0 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaabcf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa40c5e30 con 0x7fbaa4155a40 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaa928f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaa928f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47600/0 (socket says 172.21.15.17:47600) 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaa928f640 1 -- 172.21.15.17:0/480241522 learned_addr learned my addr 172.21.15.17:0/480241522 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaa9a90640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 0x7fbaa4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.562+0000 7fbaaa291640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbaa4155a40 0x7fbaa41544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 -- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbaa4155a40 msgr2=0x7fbaa41544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbaa4155a40 0x7fbaa41544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 -- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 msgr2=0x7fbaa4151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 0x7fbaa4151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 -- 172.21.15.17:0/480241522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa41e8660 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa9a90640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 0x7fbaa4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaa928f640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c520 tx=0x7fba9400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94017070 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba940035f0 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940039e0 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa41e8940 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.563+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa41e8f00 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.564+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa404f330 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.564+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba9401a070 con 0x7fbaa414f790 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.565+0000 7fba9a7fc640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 0x7fba8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.565+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fba94089730 con 0x7fbaa414f790 2024-03-06T07:23:39.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.565+0000 7fbaa9a90640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 0x7fba8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.565+0000 7fbaa9a90640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 0x7fba8806d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009560 tx=0x7fbaa0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.569+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba94057d80 con 0x7fbaa414f790 2024-03-06T07:23:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.790+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbaa40c1f40 con 0x7fbaa414f790 2024-03-06T07:23:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.790+0000 7fba9a7fc640 1 -- 172.21.15.17:0/480241522 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fba9405e0d0 con 0x7fbaa414f790 2024-03-06T07:23:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:39.793 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:35.864598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5963,"qdb_cluster":[5963]},"id":7}]} 2024-03-06T07:23:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 msgr2=0x7fba8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 0x7fba8806d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009560 tx=0x7fbaa0009190 comp rx=0 tx=0).stop 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 msgr2=0x7fbaa414bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c520 tx=0x7fba9400c9e0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 shutdown_connections 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba8806b4d0 0x7fba8806d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbaa4155a40 0x7fbaa41544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaa414f790 0x7fbaa414bf80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 --2- 172.21.15.17:0/480241522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbaa40c1aa0 0x7fbaa4151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 >> 172.21.15.17:0/480241522 conn(0x7fbaa40c6710 msgr2=0x7fbaa4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 shutdown_connections 2024-03-06T07:23:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.792+0000 7fbaabcf4640 1 -- 172.21.15.17:0/480241522 wait complete. 2024-03-06T07:23:39.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:39.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.905+0000 7fb7239bc640 1 Processor -- start 2024-03-06T07:23:39.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.905+0000 7fb7239bc640 1 -- start start 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb7239bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 0x7fb71c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb7239bc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb71c0c6000 0x7fb71c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb7239bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb71c153a60 con 0x7fb71c0c1320 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb7239bc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71c153d40 con 0x7fb71c0c6000 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb7239bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c06c2c0 con 0x7fb71c150580 2024-03-06T07:23:39.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb720f57640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb71c0c1320 0x7fb71c1566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42840/0 (socket says 172.21.15.17:42840) 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb720f57640 1 -- 172.21.15.17:0/2840571857 learned_addr learned my addr 172.21.15.17:0/2840571857 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721758640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 0x7fb71c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721f59640 1 --2- 172.21.15.17:0/2840571857 >> v2:172.21.15.163:3300/0 conn(0x7fb71c0c6000 0x7fb71c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb713fff640 1 -- 172.21.15.17:0/2840571857 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283613400 0 0) 0x7fb71c153a60 con 0x7fb71c0c1320 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb713fff640 1 -- 172.21.15.17:0/2840571857 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb708003520 con 0x7fb71c0c1320 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721f59640 1 -- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 msgr2=0x7fb71c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721f59640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 0x7fb71c06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721f59640 1 -- 172.21.15.17:0/2840571857 >> v1:172.21.15.17:6789/0 conn(0x7fb71c0c1320 legacy=0x7fb71c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721f59640 1 -- 172.21.15.17:0/2840571857 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c06c5a0 con 0x7fb71c0c6000 2024-03-06T07:23:39.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.906+0000 7fb721758640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 0x7fb71c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:39.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.907+0000 7fb721f59640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c0c6000 0x7fb71c06e250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb7140097a0 tx=0x7fb71402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a14ca0a2d7a239 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.907+0000 7fb713fff640 1 -- 172.21.15.17:0/2840571857 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7140315a0 con 0x7fb71c0c6000 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.907+0000 7fb713fff640 1 -- 172.21.15.17:0/2840571857 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb714037070 con 0x7fb71c0c6000 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.907+0000 7fb713fff640 1 -- 172.21.15.17:0/2840571857 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714031960 con 0x7fb71c0c6000 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 -- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c0c6000 msgr2=0x7fb71c06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c0c6000 0x7fb71c06e250 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb7140097a0 tx=0x7fb71402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 -- 172.21.15.17:0/2840571857 shutdown_connections 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c0c6000 0x7fb71c06e250 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 --2- 172.21.15.17:0/2840571857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c150580 0x7fb71c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 -- 172.21.15.17:0/2840571857 >> 172.21.15.17:0/2840571857 conn(0x7fb71c0c6710 msgr2=0x7fb71c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 -- 172.21.15.17:0/2840571857 shutdown_connections 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.908+0000 7fb7239bc640 1 -- 172.21.15.17:0/2840571857 wait complete. 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.909+0000 7fb7239bc640 1 Processor -- start 2024-03-06T07:23:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.909+0000 7fb7239bc640 1 -- start start 2024-03-06T07:23:39.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c0c6000 0x7fb71c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c150580 0x7fb71c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0c0dc0 con 0x7fb71c0c1320 2024-03-06T07:23:39.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c0c0c40 con 0x7fb71c150580 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb7239bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c156dc0 con 0x7fb71c0c6000 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47608/0 (socket says 172.21.15.17:47608) 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 -- 172.21.15.17:0/3429473463 learned_addr learned my addr 172.21.15.17:0/3429473463 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb720f57640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c0c6000 0x7fb71c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721f59640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c150580 0x7fb71c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 -- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c0c6000 msgr2=0x7fb71c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c0c6000 0x7fb71c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 -- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c150580 msgr2=0x7fb71c148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c150580 0x7fb71c148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 -- 172.21.15.17:0/3429473463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0a98c0 con 0x7fb71c0c1320 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb721758640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb71800efc0 tx=0x7fb71800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.910+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718018020 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.911+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb718014680 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.911+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c1ee900 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.911+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718014a50 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.911+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c0a7c80 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.912+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c04f330 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.912+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb718005d10 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.913+0000 7fb711ffb640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.913+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb71808afe0 con 0x7fb71c0c1320 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.913+0000 7fb720f57640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:39.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.913+0000 7fb720f57640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 0x7fb70006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c5bf0 tx=0x7fb70c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:39.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:39.917+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb718022030 con 0x7fb71c0c1320 2024-03-06T07:23:40.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.098+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb71c0c5de0 con 0x7fb71c0c1320 2024-03-06T07:23:40.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.099+0000 7fb711ffb640 1 -- 172.21.15.17:0/3429473463 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17002 (secure 0 0 0) 0x7fb718010040 con 0x7fb71c0c1320 2024-03-06T07:23:40.102 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:40.102 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":97,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:32.856523+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/3941058610":"2024-03-07T07:23:32.856494+0000","172.21.15.17:6832/3941058610":"2024-03-07T07:23:32.856494+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 msgr2=0x7fb70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 0x7fb70006d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0c5bf0 tx=0x7fb70c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 msgr2=0x7fb71c0a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb71800efc0 tx=0x7fb71800c490 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 shutdown_connections 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb71c150580 0x7fb71c148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb71c0c6000 0x7fb71c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 --2- 172.21.15.17:0/3429473463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb71c0c1320 0x7fb71c0a6500 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 >> 172.21.15.17:0/3429473463 conn(0x7fb71c0c6710 msgr2=0x7fb71c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 shutdown_connections 2024-03-06T07:23:40.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.101+0000 7fb7239bc640 1 -- 172.21.15.17:0/3429473463 wait complete. 2024-03-06T07:23:40.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:23:40.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:40.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 Processor -- start 2024-03-06T07:23:40.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 -- start start 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc67406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc674155c80 0x7fc6740c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6740c1ae0 con 0x7fc6740720a0 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6740c1dc0 con 0x7fc674155c80 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc67a015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67406e450 con 0x7fc674152520 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc672ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc674155c80 0x7fc6740c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc672ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc674155c80 0x7fc6740c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47610/0 (socket says 172.21.15.17:47610) 2024-03-06T07:23:40.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc672ffd640 1 -- 172.21.15.17:0/2750770315 learned_addr learned my addr 172.21.15.17:0/2750770315 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.214+0000 7fc6737fe640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc67406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.215+0000 7fc672ffd640 1 -- 172.21.15.17:0/2750770315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 msgr2=0x7fc67406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.215+0000 7fc672ffd640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc67406ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.215+0000 7fc672ffd640 1 -- 172.21.15.17:0/2750770315 >> v1:172.21.15.163:6789/0 conn(0x7fc6740720a0 legacy=0x7fc67406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.215+0000 7fc672ffd640 1 -- 172.21.15.17:0/2750770315 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67406e730 con 0x7fc674155c80 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.215+0000 7fc672ffd640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc674155c80 0x7fc6740c12b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc668009870 tx=0x7fc66802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25f401bcd5aa6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc6727fc640 1 -- 172.21.15.17:0/2750770315 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668034070 con 0x7fc674155c80 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc6727fc640 1 -- 172.21.15.17:0/2750770315 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668030840 con 0x7fc674155c80 2024-03-06T07:23:40.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc6727fc640 1 -- 172.21.15.17:0/2750770315 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668030c10 con 0x7fc674155c80 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 -- 172.21.15.17:0/2750770315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc674155c80 msgr2=0x7fc6740c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc674155c80 0x7fc6740c12b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc668009870 tx=0x7fc66802b980 comp rx=0 tx=0).stop 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 -- 172.21.15.17:0/2750770315 shutdown_connections 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc674155c80 0x7fc6740c12b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 --2- 172.21.15.17:0/2750770315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc67406ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 -- 172.21.15.17:0/2750770315 >> 172.21.15.17:0/2750770315 conn(0x7fc6740c76b0 msgr2=0x7fc6740c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 -- 172.21.15.17:0/2750770315 shutdown_connections 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.216+0000 7fc67a015640 1 -- 172.21.15.17:0/2750770315 wait complete. 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.217+0000 7fc67a015640 1 Processor -- start 2024-03-06T07:23:40.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.217+0000 7fc67a015640 1 -- start start 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc6741ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc674155c80 0x7fc6741eea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67406b6d0 con 0x7fc6740720a0 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c17f0 con 0x7fc674155c80 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc67a015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67406b850 con 0x7fc674152520 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47626/0 (socket says 172.21.15.17:47626) 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc6737fe640 1 -- 172.21.15.17:0/3423206669 learned_addr learned my addr 172.21.15.17:0/3423206669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc673fff640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc674155c80 0x7fc6741eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.218+0000 7fc672ffd640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc6741ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 -- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 msgr2=0x7fc6741ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc6741ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 -- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc674155c80 msgr2=0x7fc6741eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc674155c80 0x7fc6741eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 -- 172.21.15.17:0/3423206669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740a7ad0 con 0x7fc6740720a0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc6737fe640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc66400e8a0 tx=0x7fc66400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664017070 con 0x7fc6740720a0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66400c940 con 0x7fc6740720a0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc674156810 con 0x7fc6740720a0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66400cd30 con 0x7fc6740720a0 2024-03-06T07:23:40.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.219+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc674156dd0 con 0x7fc6740720a0 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.220+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67404f330 con 0x7fc6740720a0 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.221+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc66401a070 con 0x7fc6740720a0 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.221+0000 7fc65bfff640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.221+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc66408a4d0 con 0x7fc6740720a0 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.221+0000 7fc672ffd640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.222+0000 7fc672ffd640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 0x7fc66006d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc668036000 tx=0x7fc668008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.225+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc664058bb0 con 0x7fc6740720a0 2024-03-06T07:23:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.443+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6740c0b70 con 0x7fc6740720a0 2024-03-06T07:23:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.444+0000 7fc65bfff640 1 -- 172.21.15.17:0/3423206669 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4985 (secure 0 0 0) 0x7fc66405cc50 con 0x7fc6740720a0 2024-03-06T07:23:40.447 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:40.447 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:35.864598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5963,"qdb_cluster":[5963]},"id":7}]} 2024-03-06T07:23:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2024-03-06T07:23:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 msgr2=0x7fc66006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 0x7fc66006d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc668036000 tx=0x7fc668008030 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 msgr2=0x7fc6740ac570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc66400e8a0 tx=0x7fc66400ed60 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 shutdown_connections 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc674155c80 0x7fc6741eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc674152520 0x7fc6741ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 --2- 172.21.15.17:0/3423206669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6740720a0 0x7fc6740ac570 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 >> 172.21.15.17:0/3423206669 conn(0x7fc6740c76b0 msgr2=0x7fc6740c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 shutdown_connections 2024-03-06T07:23:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.446+0000 7fc67a015640 1 -- 172.21.15.17:0/3423206669 wait complete. 2024-03-06T07:23:40.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:40.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.557+0000 7f87a3ab1640 1 Processor -- start 2024-03-06T07:23:40.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 -- start start 2024-03-06T07:23:40.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f879c152920 0x7f879c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f879c0c8a00 0x7f879c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c156180 con 0x7f879c159460 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c1587b0 con 0x7f879c0c8a00 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a3ab1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0c7760 con 0x7f879c152920 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.558+0000 7f87a104c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f879c0c8a00 0x7f879c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f879c0c8a00 0x7f879c158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35862/0 (socket says 172.21.15.17:35862) 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 -- 172.21.15.17:0/2208565904 learned_addr learned my addr 172.21.15.17:0/2208565904 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a184d640 1 --2- 172.21.15.17:0/2208565904 >> v2:172.21.15.163:3300/0 conn(0x7f879c152920 0x7f879c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 -- 172.21.15.17:0/2208565904 >> v2:172.21.15.163:3300/0 conn(0x7f879c152920 msgr2=0x7f879c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 --2- 172.21.15.17:0/2208565904 >> v2:172.21.15.163:3300/0 conn(0x7f879c152920 0x7f879c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 -- 172.21.15.17:0/2208565904 >> v1:172.21.15.163:6789/0 conn(0x7f879c159460 legacy=0x7f879c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.559+0000 7f87a104c640 1 -- 172.21.15.17:0/2208565904 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0c7a40 con 0x7f879c0c8a00 2024-03-06T07:23:40.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a104c640 1 --2- 172.21.15.17:0/2208565904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c0c8a00 0x7f879c158f20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f879c0be490 tx=0x7f878c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42c7d59f904a3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a084b640 1 -- 172.21.15.17:0/2208565904 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c035020 con 0x7f879c0c8a00 2024-03-06T07:23:40.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a084b640 1 -- 172.21.15.17:0/2208565904 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c031870 con 0x7f879c0c8a00 2024-03-06T07:23:40.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a084b640 1 -- 172.21.15.17:0/2208565904 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c031c40 con 0x7f879c0c8a00 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2208565904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c0c8a00 msgr2=0x7f879c158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2208565904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c0c8a00 0x7f879c158f20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f879c0be490 tx=0x7f878c02b910 comp rx=0 tx=0).stop 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2208565904 shutdown_connections 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2208565904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c0c8a00 0x7f879c158f20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2208565904 >> v2:172.21.15.163:3300/0 conn(0x7f879c152920 0x7f879c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2208565904 >> 172.21.15.17:0/2208565904 conn(0x7f879c0ac560 msgr2=0x7f879c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2208565904 shutdown_connections 2024-03-06T07:23:40.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.560+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2208565904 wait complete. 2024-03-06T07:23:40.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.561+0000 7f87a3ab1640 1 Processor -- start 2024-03-06T07:23:40.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.561+0000 7f87a3ab1640 1 -- start start 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f879c0c8a00 0x7f879c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 0x7f879c14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c06d030 con 0x7f879c152920 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c158a00 con 0x7f879c153030 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a3ab1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0c5a40 con 0x7f879c0c8a00 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35870/0 (socket says 172.21.15.17:35870) 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 -- 172.21.15.17:0/2082359445 learned_addr learned my addr 172.21.15.17:0/2082359445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a184d640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f879c0c8a00 0x7f879c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a204e640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 0x7f879c14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 -- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f879c0c8a00 msgr2=0x7f879c14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f879c0c8a00 0x7f879c14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 -- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 msgr2=0x7f879c14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 0x7f879c14b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a104c640 1 -- 172.21.15.17:0/2082359445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c14dbe0 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.562+0000 7f87a204e640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 0x7f879c14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87a104c640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f878c02b950 tx=0x7f878c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c040070 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c030d00 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c1ebcc0 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c035040 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.563+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c14cbc0 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.564+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c04f330 con 0x7f879c152920 2024-03-06T07:23:40.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.564+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f878c04b060 con 0x7f879c152920 2024-03-06T07:23:40.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.565+0000 7f87927fc640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 0x7f878006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.565+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f878c0ac060 con 0x7f879c152920 2024-03-06T07:23:40.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.565+0000 7f87a184d640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 0x7f878006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.565+0000 7f87a184d640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 0x7f878006d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f879c156150 tx=0x7f8798009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.569+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f878c07ad00 con 0x7f879c152920 2024-03-06T07:23:40.661 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-03-06T07:23:40.660+0000 7f558e895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:40.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.789+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f879c14f530 con 0x7f879c152920 2024-03-06T07:23:40.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.789+0000 7f87927fc640 1 -- 172.21.15.17:0/2082359445 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4985 (secure 0 0 0) 0x7f878c07a600 con 0x7f879c152920 2024-03-06T07:23:40.792 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:40.792 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:24.744389+0000","modified":"2024-03-06T07:23:35.864598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5963},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5963":{"gid":5963,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6837/2259557388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2259557388},{"type":"v1","addr":"172.21.15.163:6837","nonce":2259557388}]},"join_fscid":7,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5963,"qdb_cluster":[5963]},"id":7}]} 2024-03-06T07:23:40.793 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 83 2024-03-06T07:23:40.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 msgr2=0x7f878006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 0x7f878006d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f879c156150 tx=0x7f8798009190 comp rx=0 tx=0).stop 2024-03-06T07:23:40.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 msgr2=0x7f879c14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f878c02b950 tx=0x7f878c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 shutdown_connections 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f878006b4d0 0x7f878006d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f879c153030 0x7f879c14b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f879c152920 0x7f879c14d6a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 --2- 172.21.15.17:0/2082359445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f879c0c8a00 0x7f879c14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 >> 172.21.15.17:0/2082359445 conn(0x7f879c0ac560 msgr2=0x7f879c0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 shutdown_connections 2024-03-06T07:23:40.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.791+0000 7f87a3ab1640 1 -- 172.21.15.17:0/2082359445 wait complete. 2024-03-06T07:23:40.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:40.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.905+0000 7efca8593640 1 Processor -- start 2024-03-06T07:23:40.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.905+0000 7efca8593640 1 -- start start 2024-03-06T07:23:40.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca8593640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01550c0 0x7efca01554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca8593640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efca01559e0 0x7efca014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca8593640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca00aa660 con 0x7efca00aa180 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca8593640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca00c4610 con 0x7efca01559e0 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca8593640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca00aa940 con 0x7efca01550c0 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efca01559e0 0x7efca014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca6b30640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efca00aa180 0x7efca00c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59008/0 (socket says 172.21.15.17:59008) 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca6b30640 1 -- 172.21.15.17:0/4066781912 learned_addr learned my addr 172.21.15.17:0/4066781912 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 --2- 172.21.15.17:0/4066781912 >> v2:172.21.15.17:3300/0 conn(0x7efca01559e0 0x7efca014ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35882/0 (socket says 172.21.15.17:35882) 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca632f640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01550c0 0x7efca01554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 -- 172.21.15.17:0/4066781912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01550c0 msgr2=0x7efca01554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01550c0 0x7efca01554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 -- 172.21.15.17:0/4066781912 >> v1:172.21.15.17:6789/0 conn(0x7efca00aa180 legacy=0x7efca00c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca5b2e640 1 -- 172.21.15.17:0/4066781912 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca00c4860 con 0x7efca01559e0 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.906+0000 7efca532d640 1 -- 172.21.15.17:0/4066781912 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155008954 0 0) 0x7efca00aa660 con 0x7efca00aa180 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.907+0000 7efca5b2e640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca01559e0 0x7efca014ec00 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efc900097a0 tx=0x7efc9002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f639542e843e16 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.907+0000 7efca532d640 1 -- 172.21.15.17:0/4066781912 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90038070 con 0x7efca01559e0 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.907+0000 7efca532d640 1 -- 172.21.15.17:0/4066781912 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc90031710 con 0x7efca01559e0 2024-03-06T07:23:40.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.907+0000 7efca532d640 1 -- 172.21.15.17:0/4066781912 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc90031ae0 con 0x7efca01559e0 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 -- 172.21.15.17:0/4066781912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca01559e0 msgr2=0x7efca014ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca01559e0 0x7efca014ec00 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7efc900097a0 tx=0x7efc9002b7b0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 -- 172.21.15.17:0/4066781912 shutdown_connections 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca01559e0 0x7efca014ec00 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 --2- 172.21.15.17:0/4066781912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01550c0 0x7efca01554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 -- 172.21.15.17:0/4066781912 >> 172.21.15.17:0/4066781912 conn(0x7efca00c6710 msgr2=0x7efca00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 -- 172.21.15.17:0/4066781912 shutdown_connections 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.908+0000 7efca8593640 1 -- 172.21.15.17:0/4066781912 wait complete. 2024-03-06T07:23:40.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 Processor -- start 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 -- start start 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 0x7efca01e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01559e0 0x7efca006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca014f140 con 0x7efca00aa180 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca00c1520 con 0x7efca01550c0 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca8593640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca014f2c0 con 0x7efca01559e0 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.909+0000 7efca632f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35892/0 (socket says 172.21.15.17:35892) 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 -- 172.21.15.17:0/3965176949 learned_addr learned my addr 172.21.15.17:0/3965176949 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca6b30640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01559e0 0x7efca006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca5b2e640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 0x7efca01e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 -- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01559e0 msgr2=0x7efca006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01559e0 0x7efca006dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 -- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 msgr2=0x7efca01e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 0x7efca01e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 -- 172.21.15.17:0/3965176949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca01e9450 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca5b2e640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 0x7efca01e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca632f640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9b0 tx=0x7efc9c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c017070 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00c9a0 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00cd90 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca0145930 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.910+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca0145160 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.911+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca0068f30 con 0x7efca00aa180 2024-03-06T07:23:40.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.912+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc9c01a070 con 0x7efca00aa180 2024-03-06T07:23:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.912+0000 7efc96ffd640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.912+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efc9c08a320 con 0x7efca00aa180 2024-03-06T07:23:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.912+0000 7efca5b2e640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:40.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.913+0000 7efca5b2e640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 0x7efc8406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efc90035000 tx=0x7efc90008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:40.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:40.916+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efca0068f30 con 0x7efca00aa180 2024-03-06T07:23:41.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.096+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efca0068450 con 0x7efca00aa180 2024-03-06T07:23:41.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.097+0000 7efc96ffd640 1 -- 172.21.15.17:0/3965176949 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17002 (secure 0 0 0) 0x7efc9c058970 con 0x7efca00aa180 2024-03-06T07:23:41.100 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:41.100 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":97,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:32.856523+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:23.000014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:23.700451+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/3941058610":"2024-03-07T07:23:32.856494+0000","172.21.15.17:6832/3941058610":"2024-03-07T07:23:32.856494+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 msgr2=0x7efc8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 0x7efc8406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efc90035000 tx=0x7efc90008030 comp rx=0 tx=0).stop 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 msgr2=0x7efca01439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00e9b0 tx=0x7efc9c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 shutdown_connections 2024-03-06T07:23:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc8406b4d0 0x7efc8406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efca01559e0 0x7efca006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efca01550c0 0x7efca01e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 --2- 172.21.15.17:0/3965176949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efca00aa180 0x7efca01439d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 >> 172.21.15.17:0/3965176949 conn(0x7efca00c6710 msgr2=0x7efca00c16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 shutdown_connections 2024-03-06T07:23:41.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.099+0000 7efca8593640 1 -- 172.21.15.17:0/3965176949 wait complete. 2024-03-06T07:23:41.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:23:41.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 Processor -- start 2024-03-06T07:23:41.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 -- start start 2024-03-06T07:23:41.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2bf40a9ea0 0x7f2bf415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf406fa80 con 0x7f2bf40c20b0 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf40a75d0 con 0x7f2bf41571e0 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.213+0000 7f2bfa932640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf4153100 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf8ecf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2bf40a9ea0 0x7f2bf415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2bf40c20b0 0x7f2bf4153590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59014/0 (socket says 172.21.15.17:59014) 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf37fe640 1 -- 172.21.15.17:0/3995776311 learned_addr learned my addr 172.21.15.17:0/3995776311 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/3995776311 >> v1:172.21.15.163:6789/0 conn(0x7f2bf41571e0 legacy=0x7f2bf406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/3995776311 >> v1:172.21.15.17:6789/0 conn(0x7f2bf40c20b0 legacy=0x7f2bf4153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.214+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/3995776311 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40a78b0 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bf8ecf640 1 --2- 172.21.15.17:0/3995776311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf415e860 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009d90 tx=0x7f2bec02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f754b571d75998ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bf2ffd640 1 -- 172.21.15.17:0/3995776311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec039070 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bf2ffd640 1 -- 172.21.15.17:0/3995776311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec02cc60 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bf2ffd640 1 -- 172.21.15.17:0/3995776311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec034660 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 -- 172.21.15.17:0/3995776311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf40a9ea0 msgr2=0x7f2bf415e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 --2- 172.21.15.17:0/3995776311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf415e860 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009d90 tx=0x7f2bec02beb0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 -- 172.21.15.17:0/3995776311 shutdown_connections 2024-03-06T07:23:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 --2- 172.21.15.17:0/3995776311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf415e860 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 -- 172.21.15.17:0/3995776311 >> 172.21.15.17:0/3995776311 conn(0x7f2bf40c41b0 msgr2=0x7f2bf40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 -- 172.21.15.17:0/3995776311 shutdown_connections 2024-03-06T07:23:41.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.215+0000 7f2bfa932640 1 -- 172.21.15.17:0/3995776311 wait complete. 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.216+0000 7f2bfa932640 1 Processor -- start 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 -- start start 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf40c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40c20b0 0x7f2bf40c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40a7e70 con 0x7f2bf41571e0 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf40a72c0 con 0x7f2bf40a9ea0 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bfa932640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf406b6d0 con 0x7f2bf40c20b0 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bf8ecf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bf8ecf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35910/0 (socket says 172.21.15.17:35910) 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.217+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/1199061372 learned_addr learned my addr 172.21.15.17:0/1199061372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf37fe640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40c20b0 0x7f2bf40c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf3fff640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf40c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40c20b0 msgr2=0x7f2bf40c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40c20b0 0x7f2bf40c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf40a9ea0 msgr2=0x7f2bf40c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf40c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 -- 172.21.15.17:0/1199061372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf40c5a10 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf8ecf640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009d90 tx=0x7f2bec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec039050 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec045070 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf40c5cf0 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec03e830 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.218+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf41ebf50 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.219+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf404f330 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.220+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bec03e9d0 con 0x7f2bf41571e0 2024-03-06T07:23:41.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.220+0000 7f2bf0ff9640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 0x7f2be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.220+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2bec0ad210 con 0x7f2bf41571e0 2024-03-06T07:23:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.220+0000 7f2bf3fff640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 0x7f2be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.221+0000 7f2bf3fff640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 0x7f2be006d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2be40016e0 tx=0x7f2be4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.224+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2bec07b980 con 0x7f2bf41571e0 2024-03-06T07:23:41.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.450+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2bf406fc50 con 0x7f2bf41571e0 2024-03-06T07:23:41.453 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:23:41.451+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:23:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.452+0000 7f2bf0ff9640 1 -- 172.21.15.17:0/1199061372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2bec07fa20 con 0x7f2bf41571e0 2024-03-06T07:23:41.454 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 msgr2=0x7f2be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 0x7f2be006d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2be40016e0 tx=0x7f2be4009040 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 msgr2=0x7f2bf41ec750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009d90 tx=0x7f2bec004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 shutdown_connections 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be006b4d0 0x7f2be006d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf41571e0 0x7f2bf41ec750 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf40c20b0 0x7f2bf40c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 --2- 172.21.15.17:0/1199061372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf40a9ea0 0x7f2bf40c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 >> 172.21.15.17:0/1199061372 conn(0x7f2bf40c41b0 msgr2=0x7f2bf4153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 shutdown_connections 2024-03-06T07:23:41.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.455+0000 7f2bfa932640 1 -- 172.21.15.17:0/1199061372 wait complete. 2024-03-06T07:23:41.459 INFO:tasks.ceph.mds.d.smithi163.stderr: -40> 2024-03-06T07:23:40.660+0000 7f558e895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:41.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:23:41.482 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:23:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.569+0000 7f257b502640 1 Processor -- start 2024-03-06T07:23:41.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.570+0000 7f257b502640 1 -- start start 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257b502640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f25740c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257b502640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25740c58b0 0x7f2574155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257b502640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25741596f0 con 0x7f25740c88f0 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257b502640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25740c9130 con 0x7f25740c58b0 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257b502640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2574159260 con 0x7f25741527f0 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2578a9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25740c88f0 0x7f2574155210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59030/0 (socket says 172.21.15.17:59030) 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2578a9d640 1 -- 172.21.15.17:0/658738769 learned_addr learned my addr 172.21.15.17:0/658738769 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257929e640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f25740c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054644283 0 0) 0x7f25741596f0 con 0x7f25740c88f0 2024-03-06T07:23:41.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2579a9f640 1 --2- 172.21.15.17:0/658738769 >> v2:172.21.15.163:3300/0 conn(0x7f25740c58b0 0x7f2574155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f255c003520 con 0x7f25740c88f0 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1315666536 0 0) 0x7f255c003520 con 0x7f25740c88f0 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 msgr2=0x7f25740c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f25740c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 >> v2:172.21.15.163:3300/0 conn(0x7f25740c58b0 msgr2=0x7f2574155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 --2- 172.21.15.17:0/658738769 >> v2:172.21.15.163:3300/0 conn(0x7f25740c58b0 0x7f2574155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740c82a0 con 0x7f25740c88f0 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f2568001670 con 0x7f25740c88f0 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 >> v1:172.21.15.17:6789/0 conn(0x7f25740c88f0 legacy=0x7f2574155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f255c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f255c003f30 0x7f255c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f255c004d40 0x7f255c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740c9130 con 0x7f255c004d40 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2574159260 con 0x7f255c003f30 2024-03-06T07:23:41.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c00baa0 con 0x7f25741527f0 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2579a9f640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f255c004d40 0x7f255c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f257929e640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f255c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.571+0000 7f2578a9d640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f255c003f30 0x7f255c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 -- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 msgr2=0x7f255c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25741527f0 0x7f255c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 -- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f255c003f30 msgr2=0x7f255c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f255c003f30 0x7f255c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 -- 172.21.15.17:0/658738769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c003520 con 0x7f255c004d40 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2579a9f640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f255c004d40 0x7f255c0051c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fc0 tx=0x7f255c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fc58753c3980ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564039070 con 0x7f255c004d40 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256402cc60 con 0x7f255c004d40 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.572+0000 7f2573fff640 1 -- 172.21.15.17:0/658738769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564009c10 con 0x7f255c004d40 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 -- 172.21.15.17:0/658738769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f255c004d40 msgr2=0x7f255c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 --2- 172.21.15.17:0/658738769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f255c004d40 0x7f255c0051c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fc0 tx=0x7f255c005830 comp rx=0 tx=0).stop 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f2579a9f640 1 -- 172.21.15.17:0/658738769 reap_dead start 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 -- 172.21.15.17:0/658738769 shutdown_connections 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 -- 172.21.15.17:0/658738769 >> 172.21.15.17:0/658738769 conn(0x7f25740ac580 msgr2=0x7f25740ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 -- 172.21.15.17:0/658738769 shutdown_connections 2024-03-06T07:23:41.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.573+0000 7f257b502640 1 -- 172.21.15.17:0/658738769 wait complete. 2024-03-06T07:23:41.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.574+0000 7f257b502640 1 Processor -- start 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.574+0000 7f257b502640 1 -- start start 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f257414bba0 0x7f25740c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25741578f0 0x7f2574157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2574158c10 con 0x7f257414b7c0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740c6b90 con 0x7f25741578f0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2574158ef0 con 0x7f257414bba0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35926/0 (socket says 172.21.15.17:35926) 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 -- 172.21.15.17:0/2178789569 learned_addr learned my addr 172.21.15.17:0/2178789569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f2578a9d640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f257414bba0 0x7f25740c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f2579a9f640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25741578f0 0x7f2574157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 -- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f257414bba0 msgr2=0x7f25740c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f257414bba0 0x7f25740c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 -- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25741578f0 msgr2=0x7f2574157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25741578f0 0x7f2574157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 -- 172.21.15.17:0/2178789569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25740c1fe0 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257929e640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f256c0119a0 tx=0x7f256c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c00faf0 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c00fc90 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25741ebee0 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.575+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c014440 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.576+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257414ecd0 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.577+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2574159660 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.577+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f256c012040 con 0x7f257414b7c0 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.577+0000 7f2571ffb640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:41.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.577+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f256c08aad0 con 0x7f257414b7c0 2024-03-06T07:23:41.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.578+0000 7f2578a9d640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:41.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.578+0000 7f2578a9d640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 0x7f256006d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f25680010e0 tx=0x7f2568009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:41.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.582+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2574159660 con 0x7f257414b7c0 2024-03-06T07:23:41.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:41.800+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f257404f330 con 0x7f257414b7c0 2024-03-06T07:23:42.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.453+0000 7f2571ffb640 1 -- 172.21.15.17:0/2178789569 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f256c019030 con 0x7f257414b7c0 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 msgr2=0x7f256006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 0x7f256006d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f25680010e0 tx=0x7f2568009040 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 msgr2=0x7f257414c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f256c0119a0 tx=0x7f256c011e60 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 shutdown_connections 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25741578f0 0x7f2574157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f257414bba0 0x7f25740c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 --2- 172.21.15.17:0/2178789569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f257414b7c0 0x7f257414c010 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 >> 172.21.15.17:0/2178789569 conn(0x7f25740ac580 msgr2=0x7f25741556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 shutdown_connections 2024-03-06T07:23:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.456+0000 7f257b502640 1 -- 172.21.15.17:0/2178789569 wait complete. 2024-03-06T07:23:42.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:23:42.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.581+0000 7fb3cadd7640 1 Processor -- start 2024-03-06T07:23:42.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.581+0000 7fb3cadd7640 1 -- start start 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3cadd7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3c4152920 0x7fb3c4155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3cadd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3c415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3cadd7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c4159740 con 0x7fb3c4155eb0 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3cadd7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c41551d0 con 0x7fb3c4152920 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3cadd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40c8ab0 con 0x7fb3c40c2900 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb3c4155eb0 0x7fb3c40c7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59036/0 (socket says 172.21.15.17:59036) 2024-03-06T07:23:42.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c3fff640 1 -- 172.21.15.17:0/1299446320 learned_addr learned my addr 172.21.15.17:0/1299446320 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c8b73640 1 --2- 172.21.15.17:0/1299446320 >> v2:172.21.15.163:3300/0 conn(0x7fb3c4152920 0x7fb3c4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c9374640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3c415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466084743 0 0) 0x7fb3c4159740 con 0x7fb3c4155eb0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.582+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b0003520 con 0x7fb3c4155eb0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518716575 0 0) 0x7fb3b0003520 con 0x7fb3c4155eb0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 msgr2=0x7fb3c415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3c415fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 >> v2:172.21.15.163:3300/0 conn(0x7fb3c4152920 msgr2=0x7fb3c4155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 --2- 172.21.15.17:0/1299446320 >> v2:172.21.15.163:3300/0 conn(0x7fb3c4152920 0x7fb3c4155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c40c8d90 con 0x7fb3c4155eb0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb3ac001670 con 0x7fb3c4155eb0 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 >> v1:172.21.15.17:6789/0 conn(0x7fb3c4155eb0 legacy=0x7fb3c40c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0003f30 0x7fb3b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3b0004d40 0x7fb3b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c41551d0 con 0x7fb3b0004d40 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40c8ab0 con 0x7fb3b0003f30 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b000ba10 con 0x7fb3c40c2900 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3b0004d40 0x7fb3b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c3fff640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0003f30 0x7fb3b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c8b73640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 -- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 msgr2=0x7fb3b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c40c2900 0x7fb3b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 -- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0003f30 msgr2=0x7fb3b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3b0003f30 0x7fb3b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.583+0000 7fb3c9374640 1 -- 172.21.15.17:0/1299446320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0003520 con 0x7fb3b0004d40 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3c9374640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3b0004d40 0x7fb3b00051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009af0 tx=0x7fb3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa66b2facf48a7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4039070 con 0x7fb3b0004d40 2024-03-06T07:23:42.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b402c6b0 con 0x7fb3b0004d40 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3c37fe640 1 -- 172.21.15.17:0/1299446320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4009690 con 0x7fb3b0004d40 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3cadd7640 1 -- 172.21.15.17:0/1299446320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3b0004d40 msgr2=0x7fb3b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/1299446320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3b0004d40 0x7fb3b00051c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009af0 tx=0x7fb3b0005830 comp rx=0 tx=0).stop 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.584+0000 7fb3c9374640 1 -- 172.21.15.17:0/1299446320 reap_dead start 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.585+0000 7fb3cadd7640 1 -- 172.21.15.17:0/1299446320 shutdown_connections 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.585+0000 7fb3cadd7640 1 -- 172.21.15.17:0/1299446320 >> 172.21.15.17:0/1299446320 conn(0x7fb3c40aa5c0 msgr2=0x7fb3c40aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.585+0000 7fb3cadd7640 1 -- 172.21.15.17:0/1299446320 shutdown_connections 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.585+0000 7fb3cadd7640 1 -- 172.21.15.17:0/1299446320 wait complete. 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 Processor -- start 2024-03-06T07:23:42.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 -- start start 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 0x7fb3c41eea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 0x7fb3c41decb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40c82a0 con 0x7fb3c40c6560 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4155450 con 0x7fb3c41ee5a0 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.586+0000 7fb3cadd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c40c8420 con 0x7fb3c41de830 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35950/0 (socket says 172.21.15.17:35950) 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 -- 172.21.15.17:0/20429001 learned_addr learned my addr 172.21.15.17:0/20429001 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c9374640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 0x7fb3c41decb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c3fff640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 0x7fb3c41eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 -- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 msgr2=0x7fb3c41decb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 0x7fb3c41decb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 -- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 msgr2=0x7fb3c41eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 0x7fb3c41eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 -- 172.21.15.17:0/20429001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c406c200 con 0x7fb3c40c6560 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c9374640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 0x7fb3c41decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c3fff640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 0x7fb3c41eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c8b73640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc011e30 tx=0x7fb3bc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc015630 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3bc0157d0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c406c4e0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc015ba0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.587+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c406c7c0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.588+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c40a94d0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.589+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3bc01d400 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.589+0000 7fb3c17fa640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 0x7fb3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.589+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb3bc08b5a0 con 0x7fb3c40c6560 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.589+0000 7fb3c3fff640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 0x7fb3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:42.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.590+0000 7fb3c3fff640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 0x7fb3a406d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000fb0 tx=0x7fb3ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:42.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.594+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3c40a94d0 con 0x7fb3c40c6560 2024-03-06T07:23:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:42.777+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3c404f330 con 0x7fb3c40c6560 2024-03-06T07:23:43.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.455+0000 7fb3c17fa640 1 -- 172.21.15.17:0/20429001 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb3bc019030 con 0x7fb3c40c6560 2024-03-06T07:23:43.457 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:23:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.458+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 msgr2=0x7fb3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.458+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 0x7fb3a406d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000fb0 tx=0x7fb3ac009040 comp rx=0 tx=0).stop 2024-03-06T07:23:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.458+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 msgr2=0x7fb3c4151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.458+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc011e30 tx=0x7fb3bc00f580 comp rx=0 tx=0).stop 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 shutdown_connections 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3a406b500 0x7fb3a406d9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c41de830 0x7fb3c41decb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c41ee5a0 0x7fb3c41eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 --2- 172.21.15.17:0/20429001 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c40c6560 0x7fb3c4151a80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 >> 172.21.15.17:0/20429001 conn(0x7fb3c40aa5c0 msgr2=0x7fb3c40aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 shutdown_connections 2024-03-06T07:23:43.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.459+0000 7fb3cadd7640 1 -- 172.21.15.17:0/20429001 wait complete. 2024-03-06T07:23:43.476 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:23:43.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.585+0000 7f10b3d0d640 1 Processor -- start 2024-03-06T07:23:43.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.585+0000 7f10b3d0d640 1 -- start start 2024-03-06T07:23:43.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b3d0d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10ac13dc90 0x7f10ac159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b3d0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac13e070 con 0x7f10ac0c69f0 2024-03-06T07:23:43.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b3d0d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac13e420 con 0x7f10ac0c53f0 2024-03-06T07:23:43.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b3d0d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac06c190 con 0x7f10ac13dc90 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b1aa9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10ac13dc90 0x7f10ac159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b22aa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10ac0c69f0 0x7f10ac15fc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59046/0 (socket says 172.21.15.17:59046) 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.586+0000 7f10b22aa640 1 -- 172.21.15.17:0/350744305 learned_addr learned my addr 172.21.15.17:0/350744305 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b1aa9640 1 -- 172.21.15.17:0/350744305 >> v1:172.21.15.163:6789/0 conn(0x7f10ac0c53f0 legacy=0x7f10ac06b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b1aa9640 1 -- 172.21.15.17:0/350744305 >> v1:172.21.15.17:6789/0 conn(0x7f10ac0c69f0 legacy=0x7f10ac15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b1aa9640 1 -- 172.21.15.17:0/350744305 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac13e700 con 0x7f10ac13dc90 2024-03-06T07:23:43.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b1aa9640 1 --2- 172.21.15.17:0/350744305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac13dc90 0x7f10ac159230 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f109c009d90 tx=0x7f109c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e96f620b7d1338 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b0aa7640 1 -- 172.21.15.17:0/350744305 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c039070 con 0x7f10ac13dc90 2024-03-06T07:23:43.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b0aa7640 1 -- 172.21.15.17:0/350744305 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c02cbd0 con 0x7f10ac13dc90 2024-03-06T07:23:43.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.587+0000 7f10b0aa7640 1 -- 172.21.15.17:0/350744305 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c034440 con 0x7f10ac13dc90 2024-03-06T07:23:43.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 -- 172.21.15.17:0/350744305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac13dc90 msgr2=0x7f10ac159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/350744305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac13dc90 0x7f10ac159230 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f109c009d90 tx=0x7f109c02be20 comp rx=0 tx=0).stop 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 -- 172.21.15.17:0/350744305 shutdown_connections 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/350744305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac13dc90 0x7f10ac159230 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 -- 172.21.15.17:0/350744305 >> 172.21.15.17:0/350744305 conn(0x7f10ac0caa70 msgr2=0x7f10ac0cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 -- 172.21.15.17:0/350744305 shutdown_connections 2024-03-06T07:23:43.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.588+0000 7f10b3d0d640 1 -- 172.21.15.17:0/350744305 wait complete. 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.589+0000 7f10b3d0d640 1 Processor -- start 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.589+0000 7f10b3d0d640 1 -- start start 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10ac0c69f0 0x7f10ac13a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 0x7f10ac13ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0c61f0 con 0x7f10ac0c53f0 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac06bde0 con 0x7f10ac13dc90 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b3d0d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac159770 con 0x7f10ac0c69f0 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35966/0 (socket says 172.21.15.17:35966) 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 -- 172.21.15.17:0/115891263 learned_addr learned my addr 172.21.15.17:0/115891263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b12a8640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10ac0c69f0 0x7f10ac13a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b22aa640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 0x7f10ac13ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 -- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10ac0c69f0 msgr2=0x7f10ac13a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10ac0c69f0 0x7f10ac13a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 -- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 msgr2=0x7f10ac13ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 0x7f10ac13ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 -- 172.21.15.17:0/115891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac1ee690 con 0x7f10ac0c53f0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b22aa640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 0x7f10ac13ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.590+0000 7f10b1aa9640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f109c037040 tx=0x7f109c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c044020 con 0x7f10ac0c53f0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c03f670 con 0x7f10ac0c53f0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac1ee970 con 0x7f10ac0c53f0 2024-03-06T07:23:43.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c039040 con 0x7f10ac0c53f0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac1eef30 con 0x7f10ac0c53f0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.591+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac04f330 con 0x7f10ac0c53f0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.592+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f109c047020 con 0x7f10ac0c53f0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.592+0000 7f10aa7fc640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 0x7f109806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.592+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f109c0acef0 con 0x7f10ac0c53f0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.593+0000 7f10b12a8640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 0x7f109806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:43.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.593+0000 7f10b12a8640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 0x7f109806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0ca380 tx=0x7f10a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:43.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.597+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f109c04e030 con 0x7f10ac0c53f0 2024-03-06T07:23:43.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:43.780+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10ac0c9370 con 0x7f10ac0c53f0 2024-03-06T07:23:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.460+0000 7f10aa7fc640 1 -- 172.21.15.17:0/115891263 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f109c07bce0 con 0x7f10ac0c53f0 2024-03-06T07:23:44.463 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:23:44.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 msgr2=0x7f109806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 0x7f109806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0ca380 tx=0x7f10a0009040 comp rx=0 tx=0).stop 2024-03-06T07:23:44.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 msgr2=0x7f10ac13b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f109c037040 tx=0x7f109c004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:44.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 shutdown_connections 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f109806b4d0 0x7f109806d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10ac13dc90 0x7f10ac13ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10ac0c69f0 0x7f10ac13a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 --2- 172.21.15.17:0/115891263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10ac0c53f0 0x7f10ac13b690 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 >> 172.21.15.17:0/115891263 conn(0x7f10ac0caa70 msgr2=0x7f10ac0c0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.463+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 shutdown_connections 2024-03-06T07:23:44.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.464+0000 7f10b3d0d640 1 -- 172.21.15.17:0/115891263 wait complete. 2024-03-06T07:23:44.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:23:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.583+0000 7ff42b29a640 1 Processor -- start 2024-03-06T07:23:44.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.583+0000 7ff42b29a640 1 -- start start 2024-03-06T07:23:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff42b29a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4241565b0 0x7ff424155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff42b29a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240c7e00 con 0x7ff4240b01e0 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff42b29a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4240c7840 con 0x7ff42406c170 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff42b29a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240c7b20 con 0x7ff4241565b0 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff428835640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff4241565b0 0x7ff424155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff429036640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff4240b01e0 0x7ff4240c53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59048/0 (socket says 172.21.15.17:59048) 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff429036640 1 -- 172.21.15.17:0/1040900848 learned_addr learned my addr 172.21.15.17:0/1040900848 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff428835640 1 -- 172.21.15.17:0/1040900848 >> v1:172.21.15.163:6789/0 conn(0x7ff42406c170 legacy=0x7ff42415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff428835640 1 -- 172.21.15.17:0/1040900848 >> v1:172.21.15.17:6789/0 conn(0x7ff4240b01e0 legacy=0x7ff4240c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.584+0000 7ff423fff640 1 -- 172.21.15.17:0/1040900848 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366712269 0 0) 0x7ff4240c7e00 con 0x7ff4240b01e0 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.585+0000 7ff428835640 1 -- 172.21.15.17:0/1040900848 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42406b970 con 0x7ff4241565b0 2024-03-06T07:23:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.585+0000 7ff428835640 1 --2- 172.21.15.17:0/1040900848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff424155b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff4180097a0 tx=0x7ff41802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49256c8e4b5f5cac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.585+0000 7ff423fff640 1 -- 172.21.15.17:0/1040900848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418035020 con 0x7ff4241565b0 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.585+0000 7ff423fff640 1 -- 172.21.15.17:0/1040900848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418031830 con 0x7ff4241565b0 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.585+0000 7ff423fff640 1 -- 172.21.15.17:0/1040900848 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418031c00 con 0x7ff4241565b0 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 -- 172.21.15.17:0/1040900848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 msgr2=0x7ff424155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 --2- 172.21.15.17:0/1040900848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff424155b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff4180097a0 tx=0x7ff41802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 -- 172.21.15.17:0/1040900848 shutdown_connections 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 --2- 172.21.15.17:0/1040900848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff424155b10 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 -- 172.21.15.17:0/1040900848 >> 172.21.15.17:0/1040900848 conn(0x7ff4240ac600 msgr2=0x7ff4240aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 -- 172.21.15.17:0/1040900848 shutdown_connections 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.586+0000 7ff42b29a640 1 -- 172.21.15.17:0/1040900848 wait complete. 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.587+0000 7ff42b29a640 1 Processor -- start 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.587+0000 7ff42b29a640 1 -- start start 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42406c170 0x7ff42414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4240b01e0 0x7ff4240cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c5c60 con 0x7ff4241565b0 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c5ae0 con 0x7ff4240b01e0 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff42b29a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240be250 con 0x7ff42406c170 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35978/0 (socket says 172.21.15.17:35978) 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 -- 172.21.15.17:0/846113670 learned_addr learned my addr 172.21.15.17:0/846113670 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429036640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42406c170 0x7ff42414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff428835640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4240b01e0 0x7ff4240cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 -- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42406c170 msgr2=0x7ff42414f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42406c170 0x7ff42414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 -- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4240b01e0 msgr2=0x7ff4240cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4240b01e0 0x7ff4240cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 -- 172.21.15.17:0/846113670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42414bc90 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff429837640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b970 tx=0x7ff41400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414019070 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.588+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41400cb40 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.589+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42414e7f0 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.589+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414014440 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.589+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4240adee0 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.590+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424075300 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.590+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff41401c070 con 0x7ff4241565b0 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.590+0000 7ff421ffb640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 0x7ff41006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.590+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff414089c80 con 0x7ff4241565b0 2024-03-06T07:23:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.591+0000 7ff429036640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 0x7ff41006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.591+0000 7ff429036640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 0x7ff41006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.595+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff424075300 con 0x7ff4241565b0 2024-03-06T07:23:44.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.778+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7ff42404f330 con 0x7ff4241565b0 2024-03-06T07:23:44.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.781+0000 7ff421ffb640 1 -- 172.21.15.17:0/846113670 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v33) v1 ==== 96+0+0 (secure 0 0 0) 0x7ff414058cb0 con 0x7ff4241565b0 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 msgr2=0x7ff41006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 0x7ff41006d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff41c000f60 tx=0x7ff41c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 msgr2=0x7ff42414daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b970 tx=0x7ff41400be30 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 shutdown_connections 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff41006b4d0 0x7ff41006d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4241565b0 0x7ff42414daa0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4240b01e0 0x7ff4240cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 --2- 172.21.15.17:0/846113670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42406c170 0x7ff42414f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 >> 172.21.15.17:0/846113670 conn(0x7ff4240ac600 msgr2=0x7ff4240abef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 shutdown_connections 2024-03-06T07:23:44.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.783+0000 7ff42b29a640 1 -- 172.21.15.17:0/846113670 wait complete. 2024-03-06T07:23:44.796 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:23:44.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.897+0000 7f499f4c3640 1 Processor -- start 2024-03-06T07:23:44.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.897+0000 7f499f4c3640 1 -- start start 2024-03-06T07:23:44.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499f4c3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 0x7f49980c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499f4c3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f49980a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499f4c3640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49980c3ef0 con 0x7f49980c5990 2024-03-06T07:23:44.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499f4c3640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49981556d0 con 0x7f499806c900 2024-03-06T07:23:44.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499f4c3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49981559b0 con 0x7f49980ab170 2024-03-06T07:23:44.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499ca5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 0x7f49980c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499da60640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f49980a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499d25f640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f49980c5990 0x7f499806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:33816/0 (socket says 172.21.15.17:33816) 2024-03-06T07:23:44.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499d25f640 1 -- 172.21.15.17:0/3822028330 learned_addr learned my addr 172.21.15.17:0/3822028330 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499da60640 1 -- 172.21.15.17:0/3822028330 >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 msgr2=0x7f49980c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499da60640 1 --2- 172.21.15.17:0/3822028330 >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 0x7f49980c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499da60640 1 -- 172.21.15.17:0/3822028330 >> v1:172.21.15.163:6789/0 conn(0x7f49980c5990 legacy=0x7f499806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499da60640 1 -- 172.21.15.17:0/3822028330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998155c90 con 0x7f49980ab170 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f498ffff640 1 -- 172.21.15.17:0/3822028330 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675841317 0 0) 0x7f49980c3ef0 con 0x7f49980c5990 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.898+0000 7f499ca5e640 1 --2- 172.21.15.17:0/3822028330 >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 0x7f49980c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.899+0000 7f499da60640 1 --2- 172.21.15.17:0/3822028330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f49980a9fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d3b9611f13f2d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.899+0000 7f498ffff640 1 -- 172.21.15.17:0/3822028330 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990039070 con 0x7f49980ab170 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.899+0000 7f498ffff640 1 -- 172.21.15.17:0/3822028330 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499002cc50 con 0x7f49980ab170 2024-03-06T07:23:44.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.899+0000 7f498ffff640 1 -- 172.21.15.17:0/3822028330 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990034440 con 0x7f49980ab170 2024-03-06T07:23:44.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 -- 172.21.15.17:0/3822028330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 msgr2=0x7f49980a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 --2- 172.21.15.17:0/3822028330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f49980a9fa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499002bea0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 -- 172.21.15.17:0/3822028330 shutdown_connections 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 --2- 172.21.15.17:0/3822028330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f49980a9fa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 --2- 172.21.15.17:0/3822028330 >> v2:172.21.15.163:3300/0 conn(0x7f499806c900 0x7f49980c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 -- 172.21.15.17:0/3822028330 >> 172.21.15.17:0/3822028330 conn(0x7f49980c6710 msgr2=0x7f49980c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 -- 172.21.15.17:0/3822028330 shutdown_connections 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.900+0000 7f499f4c3640 1 -- 172.21.15.17:0/3822028330 wait complete. 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.901+0000 7f499f4c3640 1 Processor -- start 2024-03-06T07:23:44.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.901+0000 7f499f4c3640 1 -- start start 2024-03-06T07:23:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f499814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980c5990 0x7f49981e9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998156610 con 0x7f499806c900 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998156490 con 0x7f49980c5990 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499f4c3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980c3af0 con 0x7f49980ab170 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35982/0 (socket says 172.21.15.17:35982) 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 -- 172.21.15.17:0/846355416 learned_addr learned my addr 172.21.15.17:0/846355416 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499da60640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980c5990 0x7f49981e9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499ca5e640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f499814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 -- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 msgr2=0x7f499814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f499814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 -- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980c5990 msgr2=0x7f49981e9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980c5990 0x7f49981e9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 -- 172.21.15.17:0/846355416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998153f50 con 0x7f499806c900 2024-03-06T07:23:44.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.902+0000 7f499d25f640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f499400c460 tx=0x7f499400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.903+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994018070 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.903+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4994007500 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.903+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994004a20 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.903+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4998154230 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.903+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49981547f0 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.904+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4998067ad0 con 0x7f499806c900 2024-03-06T07:23:44.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.904+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f499401b070 con 0x7f499806c900 2024-03-06T07:23:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.905+0000 7f498dffb640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.905+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4994089420 con 0x7f499806c900 2024-03-06T07:23:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.905+0000 7f499ca5e640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.905+0000 7f499ca5e640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4988009560 tx=0x7f4988008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:44.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:44.909+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4998067ad0 con 0x7f499806c900 2024-03-06T07:23:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.089+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f499804f330 con 0x7f499806c900 2024-03-06T07:23:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.092+0000 7f498dffb640 1 -- 172.21.15.17:0/846355416 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49940583c0 con 0x7f499806c900 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 msgr2=0x7f497c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4988009560 tx=0x7f4988008040 comp rx=0 tx=0).stop 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 msgr2=0x7f499814ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f499400c460 tx=0x7f499400c920 comp rx=0 tx=0).stop 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 shutdown_connections 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49980c5990 0x7f49981e9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49980ab170 0x7f499814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 --2- 172.21.15.17:0/846355416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f499806c900 0x7f499814ece0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 >> 172.21.15.17:0/846355416 conn(0x7f49980c6710 msgr2=0x7f499814bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 shutdown_connections 2024-03-06T07:23:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.094+0000 7f499f4c3640 1 -- 172.21.15.17:0/846355416 wait complete. 2024-03-06T07:23:45.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime' 2024-03-06T07:23:45.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.206+0000 7fe442d5e640 1 Processor -- start 2024-03-06T07:23:45.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.206+0000 7fe442d5e640 1 -- start start 2024-03-06T07:23:45.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe442d5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe43c14f360 0x7fe43c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe442d5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43c0c73d0 con 0x7fe43c158b60 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe442d5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43c0be2e0 con 0x7fe43c0c88c0 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe442d5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43c159700 con 0x7fe43c14f360 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe4412fb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe43c158b60 0x7fe43c158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59058/0 (socket says 172.21.15.17:59058) 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe4412fb640 1 -- 172.21.15.17:0/219994150 learned_addr learned my addr 172.21.15.17:0/219994150 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe440afa640 1 --2- 172.21.15.17:0/219994150 >> v2:172.21.15.163:3300/0 conn(0x7fe43c14f360 0x7fe43c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.207+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360623509 0 0) 0x7fe43c0c73d0 con 0x7fe43c158b60 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe428003520 con 0x7fe43c158b60 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 101566866 0 0) 0x7fe428003520 con 0x7fe43c158b60 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 >> v2:172.21.15.163:3300/0 conn(0x7fe43c14f360 msgr2=0x7fe43c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 --2- 172.21.15.17:0/219994150 >> v2:172.21.15.163:3300/0 conn(0x7fe43c14f360 0x7fe43c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 >> v1:172.21.15.163:6789/0 conn(0x7fe43c0c88c0 legacy=0x7fe43c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c0c7670 con 0x7fe43c158b60 2024-03-06T07:23:45.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe42c001670 con 0x7fe43c158b60 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 >> v1:172.21.15.17:6789/0 conn(0x7fe43c158b60 legacy=0x7fe43c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 0x7fe428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4280051c0 0x7fe42800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c159700 con 0x7fe428004800 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42800c160 con 0x7fe4280039b0 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42800c360 con 0x7fe4280051c0 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 0x7fe428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe440afa640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe4412fb640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4280051c0 0x7fe42800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 -- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4280051c0 msgr2=0x7fe42800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4280051c0 0x7fe42800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 -- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4280039b0 msgr2=0x7fe428003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4280039b0 0x7fe428003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe4412fb640 1 -- 172.21.15.17:0/219994150 reap_dead start 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.208+0000 7fe43bfff640 1 -- 172.21.15.17:0/219994150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe428003520 con 0x7fe428004800 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.209+0000 7fe43bfff640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 0x7fe428004c80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe424001720 tx=0x7fe428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504796a756ca9bdd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.209+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424001370 con 0x7fe428004800 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.209+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240052d0 con 0x7fe428004800 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.209+0000 7fe43b7fe640 1 -- 172.21.15.17:0/219994150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424003830 con 0x7fe428004800 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 -- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 msgr2=0x7fe428004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 0x7fe428004c80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe424001720 tx=0x7fe428005830 comp rx=0 tx=0).stop 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 -- 172.21.15.17:0/219994150 shutdown_connections 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 --2- 172.21.15.17:0/219994150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe428004800 0x7fe428004c80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 -- 172.21.15.17:0/219994150 >> 172.21.15.17:0/219994150 conn(0x7fe43c0ac570 msgr2=0x7fe43c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 -- 172.21.15.17:0/219994150 shutdown_connections 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.210+0000 7fe442d5e640 1 -- 172.21.15.17:0/219994150 wait complete. 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.211+0000 7fe442d5e640 1 Processor -- start 2024-03-06T07:23:45.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.211+0000 7fe442d5e640 1 -- start start 2024-03-06T07:23:45.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe428004800 0x7fe43c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 0x7fe43c0ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0c7c30 con 0x7fe43c152150 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c155c10 con 0x7fe43c14bd50 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c155d90 con 0x7fe428004800 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35994/0 (socket says 172.21.15.17:35994) 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 -- 172.21.15.17:0/1747505925 learned_addr learned my addr 172.21.15.17:0/1747505925 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe440afa640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe428004800 0x7fe43c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe43bfff640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 0x7fe43c0ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 -- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe428004800 msgr2=0x7fe43c155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe428004800 0x7fe43c155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 -- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 msgr2=0x7fe43c0ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 0x7fe43c0ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 -- 172.21.15.17:0/1747505925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c1eee40 con 0x7fe43c152150 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe43bfff640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 0x7fe43c0ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:45.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4412fb640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0c1860 tx=0x7fe42c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c01a070 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c00f620 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c0ae0e0 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c00fa10 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.212+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c1ee360 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.214+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c04f330 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.214+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe42c01d070 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.214+0000 7fe4397fa640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.214+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe42c089590 con 0x7fe43c152150 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.214+0000 7fe440afa640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.215+0000 7fe440afa640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 0x7fe41c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe434007330 tx=0x7fe43400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.218+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe42c0585c0 con 0x7fe43c152150 2024-03-06T07:23:45.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:45.467+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]} v 0) v1 -- 0x7fe43c0c8d50 con 0x7fe43c152150 2024-03-06T07:23:46.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.463+0000 7fe4397fa640 1 -- 172.21.15.17:0/1747505925 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_runtime"]}]=0 v227) v1 ==== 146+0+0 (secure 0 0 0) 0x7fe42c057ec0 con 0x7fe43c152150 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 msgr2=0x7fe41c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 0x7fe41c06d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe434007330 tx=0x7fe43400b040 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 msgr2=0x7fe43c0adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0c1860 tx=0x7fe42c007430 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 shutdown_connections 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe43c152150 0x7fe43c0adba0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe43c14bd50 0x7fe43c0ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 --2- 172.21.15.17:0/1747505925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe428004800 0x7fe43c155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 >> 172.21.15.17:0/1747505925 conn(0x7fe43c0ac570 msgr2=0x7fe43c14d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 shutdown_connections 2024-03-06T07:23:46.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.466+0000 7fe442d5e640 1 -- 172.21.15.17:0/1747505925 wait complete. 2024-03-06T07:23:46.482 INFO:tasks.cephfs_test_runner:That setting mds_join_fs at runtime affects the cluster layout. ... ok 2024-03-06T07:23:46.482 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_standby_replay (tasks.cephfs.test_failover.TestClusterAffinity) That a standby-replay daemon with weak affinity is replaced by a stronger one. 2024-03-06T07:23:46.482 INFO:tasks.cephfs_test_runner:test_join_fs_standby_replay (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:23:46.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:46.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.588+0000 7f95fc33e640 1 Processor -- start 2024-03-06T07:23:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.588+0000 7f95fc33e640 1 -- start start 2024-03-06T07:23:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fc33e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 0x7f95f4150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fc33e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95f40c3d60 0x7f95f40c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fc33e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f4156f90 con 0x7f95f4072720 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fc33e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f40c38d0 con 0x7f95f40c3d60 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fc33e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4157270 con 0x7f95f4150290 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa8db640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f95f40c3d60 0x7f95f40c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa0da640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f95f4072720 0x7f95f41564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59074/0 (socket says 172.21.15.17:59074) 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa0da640 1 -- 172.21.15.17:0/2644078567 learned_addr learned my addr 172.21.15.17:0/2644078567 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95f98d9640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 0x7f95f4150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa8db640 1 -- 172.21.15.17:0/2644078567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 msgr2=0x7f95f4150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa8db640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 0x7f95f4150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa8db640 1 -- 172.21.15.17:0/2644078567 >> v1:172.21.15.17:6789/0 conn(0x7f95f4072720 legacy=0x7f95f41564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95fa8db640 1 -- 172.21.15.17:0/2644078567 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f40c3280 con 0x7f95f40c3d60 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95f90d8640 1 -- 172.21.15.17:0/2644078567 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944466302 0 0) 0x7f95f4156f90 con 0x7f95f4072720 2024-03-06T07:23:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.589+0000 7f95f98d9640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 0x7f95f4150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95fa8db640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f40c3d60 0x7f95f40c41f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009dc0 tx=0x7f95ec02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40ccc5c984a34b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95f90d8640 1 -- 172.21.15.17:0/2644078567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec039070 con 0x7f95f40c3d60 2024-03-06T07:23:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95f90d8640 1 -- 172.21.15.17:0/2644078567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec02cbb0 con 0x7f95f40c3d60 2024-03-06T07:23:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95f90d8640 1 -- 172.21.15.17:0/2644078567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec034440 con 0x7f95f40c3d60 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95fc33e640 1 -- 172.21.15.17:0/2644078567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f40c3d60 msgr2=0x7f95f40c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.590+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f40c3d60 0x7f95f40c41f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009dc0 tx=0x7f95ec02be00 comp rx=0 tx=0).stop 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 -- 172.21.15.17:0/2644078567 shutdown_connections 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f40c3d60 0x7f95f40c41f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2644078567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f4150290 0x7f95f4150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 -- 172.21.15.17:0/2644078567 >> 172.21.15.17:0/2644078567 conn(0x7f95f40aa630 msgr2=0x7f95f40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 -- 172.21.15.17:0/2644078567 shutdown_connections 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.591+0000 7f95fc33e640 1 -- 172.21.15.17:0/2644078567 wait complete. 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 Processor -- start 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 -- start start 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 0x7f95f41e9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f40c3d60 0x7f95f4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4156bd0 con 0x7f95f4150290 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f406f8f0 con 0x7f95f4072720 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fc33e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f4156d50 con 0x7f95f40c3d60 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.592+0000 7f95fa8db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36010/0 (socket says 172.21.15.17:36010) 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 -- 172.21.15.17:0/2036978950 learned_addr learned my addr 172.21.15.17:0/2036978950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95f98d9640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f40c3d60 0x7f95f4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa0da640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 0x7f95f41e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 -- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f40c3d60 msgr2=0x7f95f4155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f40c3d60 0x7f95f4155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 -- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 msgr2=0x7f95f41e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 0x7f95f41e9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 -- 172.21.15.17:0/2036978950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f40ae0e0 con 0x7f95f4150290 2024-03-06T07:23:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa0da640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 0x7f95f41e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fa8db640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f95ec037040 tx=0x7f95ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec039050 con 0x7f95f4150290 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec045070 con 0x7f95f4150290 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f4153ae0 con 0x7f95f4150290 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec03e850 con 0x7f95f4150290 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.593+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f4155e00 con 0x7f95f4150290 2024-03-06T07:23:46.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.594+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f40aac10 con 0x7f95f4150290 2024-03-06T07:23:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.595+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95ec03e9f0 con 0x7f95f4150290 2024-03-06T07:23:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.595+0000 7f95eaffd640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 0x7f95d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.595+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f95ec0acee0 con 0x7f95f4150290 2024-03-06T07:23:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.596+0000 7f95fa0da640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 0x7f95d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.596+0000 7f95fa0da640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 0x7f95d806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000f60 tx=0x7f95f0007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.600+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f95f40aac10 con 0x7f95f4150290 2024-03-06T07:23:46.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.785+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f40acfb0 con 0x7f95f4150290 2024-03-06T07:23:46.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.785+0000 7f95eaffd640 1 -- 172.21.15.17:0/2036978950 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13427 (secure 0 0 0) 0x7f95ec07bf90 con 0x7f95f4150290 2024-03-06T07:23:46.788 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:46.788 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":100,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:44.457168+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/2259557388":"2024-03-07T07:23:41.452652+0000","172.21.15.163:6835/2259557388":"2024-03-07T07:23:41.452652+0000","172.21.15.17:6835/3941058610":"2024-03-07T07:23:32.856494+0000","172.21.15.17:6832/3941058610":"2024-03-07T07:23:32.856494+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:46.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 msgr2=0x7f95d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 0x7f95d806d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f95f0000f60 tx=0x7f95f0007c10 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 msgr2=0x7f95f40ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f95ec037040 tx=0x7f95ec004a40 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 shutdown_connections 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f95d806b4d0 0x7f95d806d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95f4150290 0x7f95f40ab930 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95f40c3d60 0x7f95f4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 --2- 172.21.15.17:0/2036978950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95f4072720 0x7f95f41e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 >> 172.21.15.17:0/2036978950 conn(0x7f95f40aa630 msgr2=0x7f95f40a9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 shutdown_connections 2024-03-06T07:23:46.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.788+0000 7f95fc33e640 1 -- 172.21.15.17:0/2036978950 wait complete. 2024-03-06T07:23:46.802 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:46.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:46.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.908+0000 7f34bccf4640 1 Processor -- start 2024-03-06T07:23:46.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.908+0000 7f34bccf4640 1 -- start start 2024-03-06T07:23:46.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34bccf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34b80724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34bccf4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b8152cb0 con 0x7f34b80c67f0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34bccf4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80c38b0 con 0x7f34b8072a10 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34bccf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81594f0 con 0x7f34b81528d0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34b80c67f0 0x7f34b815fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59084/0 (socket says 172.21.15.17:59084) 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b6d9d640 1 -- 172.21.15.17:0/2100006140 learned_addr learned my addr 172.21.15.17:0/2100006140 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b659c640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34b80724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403539176 0 0) 0x7f34b8152cb0 con 0x7f34b80c67f0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a0003520 con 0x7f34b80c67f0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419814048 0 0) 0x7f34a0003520 con 0x7f34b80c67f0 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 msgr2=0x7f34b80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34b80724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 >> v1:172.21.15.163:6789/0 conn(0x7f34b8072a10 legacy=0x7f34b8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80a7f70 con 0x7f34b80c67f0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f34b00019f0 con 0x7f34b80c67f0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 >> v1:172.21.15.17:6789/0 conn(0x7f34b80c67f0 legacy=0x7f34b815fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.909+0000 7f34b559a640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a00039e0 0x7f34a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b559a640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34a0004390 0x7f34a0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b559a640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34a00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81594f0 con 0x7f34a0004390 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a000b9c0 con 0x7f34a00039e0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a000bbc0 con 0x7f34b81528d0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34a0004390 0x7f34a0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b659c640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a00039e0 0x7f34a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b6d9d640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34a00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 -- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 msgr2=0x7f34a00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b81528d0 0x7f34a00050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 -- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a00039e0 msgr2=0x7f34a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a00039e0 0x7f34a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.910+0000 7f34b5d9b640 1 -- 172.21.15.17:0/2100006140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a0003520 con 0x7f34a0004390 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34b5d9b640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34a0004390 0x7f34a0004860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f34ac001860 tx=0x7f34a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e551a7f512a4eb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac038070 con 0x7f34a0004390 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac033410 con 0x7f34a0004390 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34b559a640 1 -- 172.21.15.17:0/2100006140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac001430 con 0x7f34a0004390 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 -- 172.21.15.17:0/2100006140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34a0004390 msgr2=0x7f34a0004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 --2- 172.21.15.17:0/2100006140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34a0004390 0x7f34a0004860 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f34ac001860 tx=0x7f34a0005830 comp rx=0 tx=0).stop 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34b6d9d640 1 -- 172.21.15.17:0/2100006140 reap_dead start 2024-03-06T07:23:46.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 -- 172.21.15.17:0/2100006140 shutdown_connections 2024-03-06T07:23:46.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 -- 172.21.15.17:0/2100006140 >> 172.21.15.17:0/2100006140 conn(0x7f34b80aa6a0 msgr2=0x7f34b80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:46.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 -- 172.21.15.17:0/2100006140 shutdown_connections 2024-03-06T07:23:46.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.911+0000 7f34bccf4640 1 -- 172.21.15.17:0/2100006140 wait complete. 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 Processor -- start 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 -- start start 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 0x7f34b814a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b8151d70 0x7f34b81ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81591a0 con 0x7f34b81ead00 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80a8250 con 0x7f34a0004390 2024-03-06T07:23:46.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34bccf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80c4ef0 con 0x7f34b8151d70 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34b6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34b6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36036/0 (socket says 172.21.15.17:36036) 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.913+0000 7f34b6d9d640 1 -- 172.21.15.17:0/1009735721 learned_addr learned my addr 172.21.15.17:0/1009735721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b5d9b640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b8151d70 0x7f34b81ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b659c640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 0x7f34b814a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 -- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b8151d70 msgr2=0x7f34b81ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b8151d70 0x7f34b81ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 -- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 msgr2=0x7f34b814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 0x7f34b814a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 -- 172.21.15.17:0/1009735721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b81ee670 con 0x7f34b81ead00 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b659c640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 0x7f34b814a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34b6d9d640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f34b804f970 tx=0x7f34b0007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0017020 con 0x7f34b81ead00 2024-03-06T07:23:46.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000ac40 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b81ee950 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b00133f0 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.914+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b81516e0 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.915+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b8151920 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.915+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34b001a020 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.916+0000 7f34a6ffd640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 0x7f349806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.916+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f34b008a8a0 con 0x7f34b81ead00 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.916+0000 7f34b659c640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 0x7f349806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.917+0000 7f34b659c640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 0x7f349806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ab0 tx=0x7f34a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:46.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:46.920+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34b8151920 con 0x7f34b81ead00 2024-03-06T07:23:47.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.101+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34b804f330 con 0x7f34b81ead00 2024-03-06T07:23:47.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.101+0000 7f34a6ffd640 1 -- 172.21.15.17:0/1009735721 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34b0021020 con 0x7f34b81ead00 2024-03-06T07:23:47.104 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 msgr2=0x7f349806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 0x7f349806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ab0 tx=0x7f34a8008040 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 msgr2=0x7f34b81500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f34b804f970 tx=0x7f34b0007bd0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 shutdown_connections 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f349806b4d0 0x7f349806d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34b81ead00 0x7f34b81500f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34b8151d70 0x7f34b81ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 --2- 172.21.15.17:0/1009735721 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34a0004390 0x7f34b814a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 >> 172.21.15.17:0/1009735721 conn(0x7f34b80aa6a0 msgr2=0x7f34b814c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 shutdown_connections 2024-03-06T07:23:47.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.103+0000 7f34bccf4640 1 -- 172.21.15.17:0/1009735721 wait complete. 2024-03-06T07:23:47.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay' 2024-03-06T07:23:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.215+0000 7f6433e47640 1 Processor -- start 2024-03-06T07:23:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.215+0000 7f6433e47640 1 -- start start 2024-03-06T07:23:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6433e47640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f642c0c5480 0x7f642c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6433e47640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c1586f0 0x7f642c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6433e47640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c0c5cc0 con 0x7f642c0c5860 2024-03-06T07:23:47.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6433e47640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c06df40 con 0x7f642c0c5480 2024-03-06T07:23:47.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6433e47640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c06fdb0 con 0x7f642c1586f0 2024-03-06T07:23:47.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f6431be3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f642c0c5480 0x7f642c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f64323e4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c1586f0 0x7f642c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f64313e2640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f642c0c5860 0x7f642c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:33850/0 (socket says 172.21.15.17:33850) 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.216+0000 7f64313e2640 1 -- 172.21.15.17:0/2038395623 learned_addr learned my addr 172.21.15.17:0/2038395623 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.217+0000 7f6431be3640 1 -- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c1586f0 msgr2=0x7f642c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.217+0000 7f6431be3640 1 --2- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c1586f0 0x7f642c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.217+0000 7f6431be3640 1 -- 172.21.15.17:0/2038395623 >> v1:172.21.15.163:6789/0 conn(0x7f642c0c5860 legacy=0x7f642c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.217+0000 7f6431be3640 1 -- 172.21.15.17:0/2038395623 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c1591c0 con 0x7f642c0c5480 2024-03-06T07:23:47.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.217+0000 7f6430be1640 1 -- 172.21.15.17:0/2038395623 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309778508 0 0) 0x7f642c0c5cc0 con 0x7f642c0c5860 2024-03-06T07:23:47.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6431be3640 1 --2- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c155ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64240097a0 tx=0x7f642402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdcbec93570540b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6430be1640 1 -- 172.21.15.17:0/2038395623 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64240315a0 con 0x7f642c0c5480 2024-03-06T07:23:47.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6430be1640 1 -- 172.21.15.17:0/2038395623 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424037070 con 0x7f642c0c5480 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6430be1640 1 -- 172.21.15.17:0/2038395623 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424031960 con 0x7f642c0c5480 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 -- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 msgr2=0x7f642c155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 --2- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c155ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f64240097a0 tx=0x7f642402b840 comp rx=0 tx=0).stop 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 -- 172.21.15.17:0/2038395623 shutdown_connections 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 --2- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c1586f0 0x7f642c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 --2- 172.21.15.17:0/2038395623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c155ba0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.218+0000 7f6433e47640 1 -- 172.21.15.17:0/2038395623 >> 172.21.15.17:0/2038395623 conn(0x7f642c06b1a0 msgr2=0x7f642c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.219+0000 7f6433e47640 1 -- 172.21.15.17:0/2038395623 shutdown_connections 2024-03-06T07:23:47.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.219+0000 7f6433e47640 1 -- 172.21.15.17:0/2038395623 wait complete. 2024-03-06T07:23:47.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 Processor -- start 2024-03-06T07:23:47.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 -- start start 2024-03-06T07:23:47.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c14de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c0c5860 0x7f642c14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c155680 con 0x7f642c1586f0 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c1553a0 con 0x7f642c0c5480 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f6433e47640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c152a20 con 0x7f642c0c5860 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f64323e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f64323e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36040/0 (socket says 172.21.15.17:36040) 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.220+0000 7f64323e4640 1 -- 172.21.15.17:0/2027715372 learned_addr learned my addr 172.21.15.17:0/2027715372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64313e2640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c0c5860 0x7f642c14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f6431be3640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c14de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 -- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c0c5860 msgr2=0x7f642c14fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c0c5860 0x7f642c14fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 -- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 msgr2=0x7f642c14de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c14de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:47.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 -- 172.21.15.17:0/2027715372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0c3820 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f6431be3640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f64323e4640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f641c00ec10 tx=0x7f641c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c018020 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c00cd20 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c0c3b00 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0146c0 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.221+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c14e360 con 0x7f642c1586f0 2024-03-06T07:23:47.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.222+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c075300 con 0x7f642c1586f0 2024-03-06T07:23:47.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.223+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f641c005d10 con 0x7f642c1586f0 2024-03-06T07:23:47.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.223+0000 7f642a7fc640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 0x7f641806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:47.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.223+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f641c08aee0 con 0x7f642c1586f0 2024-03-06T07:23:47.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.223+0000 7f6431be3640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 0x7f641806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:47.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.224+0000 7f6431be3640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 0x7f641806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6424036000 tx=0x7f642403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:47.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.227+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f642c075300 con 0x7f642c1586f0 2024-03-06T07:23:47.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:47.482+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]} v 0) v1 -- 0x7f642c04f330 con 0x7f642c1586f0 2024-03-06T07:23:48.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.471+0000 7f642a7fc640 1 -- 172.21.15.17:0/2027715372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]}]=0 v229) v1 ==== 156+0+0 (secure 0 0 0) 0x7f641c022030 con 0x7f642c1586f0 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 msgr2=0x7f641806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 0x7f641806d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6424036000 tx=0x7f642403d040 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 msgr2=0x7f642c1ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f641c00ec10 tx=0x7f641c00c490 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 shutdown_connections 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f641806b4d0 0x7f641806d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642c1586f0 0x7f642c1ee2c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642c0c5860 0x7f642c14fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 --2- 172.21.15.17:0/2027715372 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f642c0c5480 0x7f642c14de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 >> 172.21.15.17:0/2027715372 conn(0x7f642c06b1a0 msgr2=0x7f642c151430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 shutdown_connections 2024-03-06T07:23:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.474+0000 7f6433e47640 1 -- 172.21.15.17:0/2027715372 wait complete. 2024-03-06T07:23:48.490 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:23:48.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.598+0000 7ff2d38f4640 1 Processor -- start 2024-03-06T07:23:48.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.598+0000 7ff2d38f4640 1 -- start start 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d38f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d38f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d38f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2cc152dd0 con 0x7ff2cc155530 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d38f4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2cc159710 con 0x7ff2cc0c5370 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d38f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc0c68b0 con 0x7ff2cc1529f0 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d1690640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d0e8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d1e91640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2cc155530 0x7ff2cc1559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:33858/0 (socket says 172.21.15.17:33858) 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d0e8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:51302/0 (socket says 172.21.15.17:51302) 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d1690640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:38340/0 (socket says 172.21.15.17:38340) 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.599+0000 7ff2d1e91640 1 -- 172.21.15.17:0/780743430 learned_addr learned my addr 172.21.15.17:0/780743430 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2d0e8f640 1 -- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 msgr2=0x7ff2cc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2d0e8f640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2d0e8f640 1 -- 172.21.15.17:0/780743430 >> v1:172.21.15.163:6789/0 conn(0x7ff2cc155530 legacy=0x7ff2cc1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2d0e8f640 1 -- 172.21.15.17:0/780743430 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc158b00 con 0x7ff2cc0c5370 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2cbfff640 1 -- 172.21.15.17:0/780743430 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344595610 0 0) 0x7ff2cc152dd0 con 0x7ff2cc155530 2024-03-06T07:23:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.600+0000 7ff2d1690640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d0e8f640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097a0 tx=0x7ff2c002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975114623ae1b96 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2cbfff640 1 -- 172.21.15.17:0/780743430 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00315a0 con 0x7ff2cc0c5370 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2cbfff640 1 -- 172.21.15.17:0/780743430 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0037070 con 0x7ff2cc0c5370 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2cbfff640 1 -- 172.21.15.17:0/780743430 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0031960 con 0x7ff2cc0c5370 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 -- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 msgr2=0x7ff2cc0c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00097a0 tx=0x7ff2c002b840 comp rx=0 tx=0).stop 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 -- 172.21.15.17:0/780743430 shutdown_connections 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc0c57d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/780743430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc0c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 -- 172.21.15.17:0/780743430 >> 172.21.15.17:0/780743430 conn(0x7ff2cc0ac580 msgr2=0x7ff2cc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 -- 172.21.15.17:0/780743430 shutdown_connections 2024-03-06T07:23:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.601+0000 7ff2d38f4640 1 -- 172.21.15.17:0/780743430 wait complete. 2024-03-06T07:23:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.602+0000 7ff2d38f4640 1 Processor -- start 2024-03-06T07:23:48.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.602+0000 7ff2d38f4640 1 -- start start 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc1eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1593a0 con 0x7ff2cc155530 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1590c0 con 0x7ff2cc0c5370 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d38f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc1560d0 con 0x7ff2cc1529f0 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d1e91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d1e91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36048/0 (socket says 172.21.15.17:36048) 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d1e91640 1 -- 172.21.15.17:0/793278162 learned_addr learned my addr 172.21.15.17:0/793278162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d1690640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.603+0000 7ff2d0e8f640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc1eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 -- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 msgr2=0x7ff2cc1eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc1eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 -- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 msgr2=0x7ff2cc157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 -- 172.21.15.17:0/793278162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc1583d0 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d1e91640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c700 tx=0x7ff2bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc018070 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc00ce10 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc1565a0 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00a860 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.604+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc1486f0 con 0x7ff2cc155530 2024-03-06T07:23:48.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.605+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc0c85c0 con 0x7ff2cc155530 2024-03-06T07:23:48.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.606+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2bc00aa00 con 0x7ff2cc155530 2024-03-06T07:23:48.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.606+0000 7ff2c9ffb640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 0x7ff2b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:48.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.606+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff2bc0896d0 con 0x7ff2cc155530 2024-03-06T07:23:48.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.606+0000 7ff2d1690640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 0x7ff2b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:48.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.607+0000 7ff2d1690640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 0x7ff2b806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ca3f0 tx=0x7ff2c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:48.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.610+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2cc0c85c0 con 0x7ff2cc155530 2024-03-06T07:23:48.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.791+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff2cc04f330 con 0x7ff2cc155530 2024-03-06T07:23:48.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.794+0000 7ff2c9ffb640 1 -- 172.21.15.17:0/793278162 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff2bc058680 con 0x7ff2cc155530 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 msgr2=0x7ff2b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 0x7ff2b806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0ca3f0 tx=0x7ff2c4008040 comp rx=0 tx=0).stop 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 msgr2=0x7ff2cc1481b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c700 tx=0x7ff2bc00cbc0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 shutdown_connections 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2b806b4d0 0x7ff2b806d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2cc155530 0x7ff2cc1481b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2cc1529f0 0x7ff2cc1eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 --2- 172.21.15.17:0/793278162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2cc0c5370 0x7ff2cc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:48.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 >> 172.21.15.17:0/793278162 conn(0x7ff2cc0ac580 msgr2=0x7ff2cc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:48.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 shutdown_connections 2024-03-06T07:23:48.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:48.796+0000 7ff2d38f4640 1 -- 172.21.15.17:0/793278162 wait complete. 2024-03-06T07:23:48.809 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:23:48.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:48.862 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:23:48.907 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:23:48.907 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:23:48.907 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:48.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:23:48.923 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:23:48.923 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:23:48.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:23:48.966 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:23:48.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.064+0000 7f1729f8a640 1 Processor -- start 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.064+0000 7f1729f8a640 1 -- start start 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1729f8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f1724071980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1729f8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17240c2410 0x7f1724071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1729f8a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1724067340 con 0x7f1724150920 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1729f8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1724072400 con 0x7f17240c2410 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1729f8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1724150d00 con 0x7f172406c720 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f17237fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1724150920 0x7f17240c2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59086/0 (socket says 172.21.15.17:59086) 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f17237fe640 1 -- 172.21.15.17:0/2052021660 learned_addr learned my addr 172.21.15.17:0/2052021660 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971036245 0 0) 0x7f1724067340 con 0x7f1724150920 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f170c003520 con 0x7f1724150920 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1722ffd640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f1724071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.065+0000 7f1723fff640 1 --2- 172.21.15.17:0/2052021660 >> v2:172.21.15.163:3300/0 conn(0x7f17240c2410 0x7f1724071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613307916 0 0) 0x7f170c003520 con 0x7f1724150920 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 msgr2=0x7f1724071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f1724071980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 >> v2:172.21.15.163:3300/0 conn(0x7f17240c2410 msgr2=0x7f1724071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 --2- 172.21.15.17:0/2052021660 >> v2:172.21.15.163:3300/0 conn(0x7f17240c2410 0x7f1724071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240c2010 con 0x7f1724150920 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1714001670 con 0x7f1724150920 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 >> v1:172.21.15.17:6789/0 conn(0x7f1724150920 legacy=0x7f17240c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f170c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724072400 con 0x7f170c0039b0 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724150d00 con 0x7f170c004d40 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c00b9c0 con 0x7f172406c720 2024-03-06T07:23:49.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f1722ffd640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f170c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f1723fff640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 -- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 msgr2=0x7f170c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f170c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 -- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c004d40 msgr2=0x7f170c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f170c004d40 0x7f170c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f17237fe640 1 -- 172.21.15.17:0/2052021660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c003520 con 0x7f170c0039b0 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.066+0000 7f1722ffd640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f172406c720 0x7f170c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f17237fe640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1714001050 tx=0x7f170c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ae63f5cd91133 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714007d70 con 0x7f170c0039b0 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714030070 con 0x7f170c0039b0 2024-03-06T07:23:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f17227fc640 1 -- 172.21.15.17:0/2052021660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171402cd80 con 0x7f170c0039b0 2024-03-06T07:23:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f1729f8a640 1 -- 172.21.15.17:0/2052021660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 msgr2=0x7f170c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f1729f8a640 1 --2- 172.21.15.17:0/2052021660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f1714001050 tx=0x7f170c005830 comp rx=0 tx=0).stop 2024-03-06T07:23:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f1723fff640 1 -- 172.21.15.17:0/2052021660 reap_dead start 2024-03-06T07:23:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f1729f8a640 1 -- 172.21.15.17:0/2052021660 shutdown_connections 2024-03-06T07:23:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.067+0000 7f1729f8a640 1 -- 172.21.15.17:0/2052021660 >> 172.21.15.17:0/2052021660 conn(0x7f17240c70b0 msgr2=0x7f17240c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.068+0000 7f1729f8a640 1 -- 172.21.15.17:0/2052021660 shutdown_connections 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.068+0000 7f1729f8a640 1 -- 172.21.15.17:0/2052021660 wait complete. 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 Processor -- start 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 -- start start 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724151150 0x7f1724142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17241434b0 0x7f1724142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c3040 con 0x7f170c0039b0 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724072b70 con 0x7f1724151150 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.069+0000 7f1729f8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240c31c0 con 0x7f17241434b0 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36068/0 (socket says 172.21.15.17:36068) 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 -- 172.21.15.17:0/523617470 learned_addr learned my addr 172.21.15.17:0/523617470 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f1722ffd640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724151150 0x7f1724142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f1723fff640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17241434b0 0x7f1724142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 -- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17241434b0 msgr2=0x7f1724142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17241434b0 0x7f1724142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 -- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724151150 msgr2=0x7f1724142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724151150 0x7f1724142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 -- 172.21.15.17:0/523617470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724143020 con 0x7f170c0039b0 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f17237fe640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f171400c300 tx=0x7f1714000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714045070 con 0x7f170c0039b0 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714004bf0 con 0x7f170c0039b0 2024-03-06T07:23:49.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172414b1e0 con 0x7f170c0039b0 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714007d50 con 0x7f170c0039b0 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.070+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17241ecae0 con 0x7f170c0039b0 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.071+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172404f330 con 0x7f170c0039b0 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.072+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1714007070 con 0x7f170c0039b0 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.072+0000 7f170bfff640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 0x7f171006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.072+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f17140ab0b0 con 0x7f170c0039b0 2024-03-06T07:23:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.072+0000 7f1722ffd640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 0x7f171006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.073+0000 7f1722ffd640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 0x7f171006d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17241e6c00 tx=0x7f1718008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.077+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1714079b70 con 0x7f170c0039b0 2024-03-06T07:23:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.295+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172414dcf0 con 0x7f170c0039b0 2024-03-06T07:23:49.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.296+0000 7f170bfff640 1 -- 172.21.15.17:0/523617470 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f171407dc10 con 0x7f170c0039b0 2024-03-06T07:23:49.300 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:49.300 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6706,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3720096092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3720096092},{"type":"v1","addr":"172.21.15.163:6836","nonce":3720096092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":6937,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1366154986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1366154986},{"type":"v1","addr":"172.21.15.163:6837","nonce":1366154986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7008,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2875090396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2875090396},{"type":"v1","addr":"172.21.15.17:6834","nonce":2875090396}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7206,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2336694958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2336694958},{"type":"v1","addr":"172.21.15.17:6835","nonce":2336694958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[]} 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 85 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 msgr2=0x7f171006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 0x7f171006d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17241e6c00 tx=0x7f1718008040 comp rx=0 tx=0).stop 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 msgr2=0x7f17241ec5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f171400c300 tx=0x7f1714000fe0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 shutdown_connections 2024-03-06T07:23:49.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f171006b500 0x7f171006d9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17241434b0 0x7f1724142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724151150 0x7f1724142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 --2- 172.21.15.17:0/523617470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f170c0039b0 0x7f17241ec5a0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 >> 172.21.15.17:0/523617470 conn(0x7f17240c70b0 msgr2=0x7f172406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 shutdown_connections 2024-03-06T07:23:49.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.298+0000 7f1729f8a640 1 -- 172.21.15.17:0/523617470 wait complete. 2024-03-06T07:23:49.311 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:23:49.312 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:23:49.312 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:23:49.312 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:49.312 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:23:49.312 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:23:49.312 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:23:49.312 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:49.313 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:23:49.313 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:23:49.313 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:23:49.313 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:49.313 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:23:49.313 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:23:49.313 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:23:49.314 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:23:49.321 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:23:49.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:23:49.322 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:23:49.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:23:49.327 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:23:49.327 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:23:49.327 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:23:49.328 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:23:49.364 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:23:49.364 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:49.364 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:23:49.365 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:49.369 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:23:49.369 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:49.370 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:23:49.370 DEBUG:teuthology.parallel:result is None 2024-03-06T07:23:49.370 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:23:49.440 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:23:49.440 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:23:49.452 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:23:49.452 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:23:49.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.474+0000 7f3da13ac640 1 Processor -- start 2024-03-06T07:23:49.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 -- start start 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c06ff10 0x7f3d9c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d9c0c7170 0x7f3d9c154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0c7620 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0a8390 con 0x7f3d9c0c7170 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3da13ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0730f0 con 0x7f3d9c06ff10 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3d9b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3d9b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36110/0 (socket says 172.21.15.17:36110) 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.475+0000 7f3d9b7fe640 1 -- 172.21.15.17:0/704751238 learned_addr learned my addr 172.21.15.17:0/704751238 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9affd640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c06ff10 0x7f3d9c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9a7fc640 1 --2- 172.21.15.17:0/704751238 >> v2:172.21.15.163:3300/0 conn(0x7f3d9c0c7170 0x7f3d9c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9b7fe640 1 -- 172.21.15.17:0/704751238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c06ff10 msgr2=0x7f3d9c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9b7fe640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c06ff10 0x7f3d9c153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9b7fe640 1 -- 172.21.15.17:0/704751238 >> v2:172.21.15.163:3300/0 conn(0x7f3d9c0c7170 msgr2=0x7f3d9c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9b7fe640 1 --2- 172.21.15.17:0/704751238 >> v2:172.21.15.163:3300/0 conn(0x7f3d9c0c7170 0x7f3d9c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.476+0000 7f3d9b7fe640 1 -- 172.21.15.17:0/704751238 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0733d0 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.477+0000 7f3d9b7fe640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0707c0 tx=0x7f3d9402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db722b605b365cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.479+0000 7f3d99ffb640 1 -- 172.21.15.17:0/704751238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9402c950 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.479+0000 7f3d99ffb640 1 -- 172.21.15.17:0/704751238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9402caf0 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.479+0000 7f3d99ffb640 1 -- 172.21.15.17:0/704751238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940343f0 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 -- 172.21.15.17:0/704751238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c0c6c40 msgr2=0x7f3d9c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0707c0 tx=0x7f3d9402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 -- 172.21.15.17:0/704751238 shutdown_connections 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c0c6c40 0x7f3d9c0c8810 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 --2- 172.21.15.17:0/704751238 >> v2:172.21.15.163:3300/0 conn(0x7f3d9c0c7170 0x7f3d9c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 --2- 172.21.15.17:0/704751238 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c06ff10 0x7f3d9c153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 -- 172.21.15.17:0/704751238 >> 172.21.15.17:0/704751238 conn(0x7f3d9c06b970 msgr2=0x7f3d9c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 -- 172.21.15.17:0/704751238 shutdown_connections 2024-03-06T07:23:49.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.480+0000 7f3da13ac640 1 -- 172.21.15.17:0/704751238 wait complete. 2024-03-06T07:23:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.481+0000 7f3da13ac640 1 Processor -- start 2024-03-06T07:23:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.481+0000 7f3da13ac640 1 -- start start 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c0c6c40 0x7f3d9c14cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d9c0c7170 0x7f3d9c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c070470 con 0x7f3d9c06ff10 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0702f0 con 0x7f3d9c0c7170 2024-03-06T07:23:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3da13ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0c6860 con 0x7f3d9c0c6c40 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36122/0 (socket says 172.21.15.17:36122) 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9affd640 1 -- 172.21.15.17:0/3603185584 learned_addr learned my addr 172.21.15.17:0/3603185584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9b7fe640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d9c0c7170 0x7f3d9c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9a7fc640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c0c6c40 0x7f3d9c14cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9affd640 1 -- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c0c6c40 msgr2=0x7f3d9c14cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.482+0000 7f3d9affd640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c0c6c40 0x7f3d9c14cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d9affd640 1 -- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d9c0c7170 msgr2=0x7f3d9c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d9affd640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d9c0c7170 0x7f3d9c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d9affd640 1 -- 172.21.15.17:0/3603185584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c06e0a0 con 0x7f3d9c06ff10 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d9affd640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00e890 tx=0x7f3d8c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c017070 con 0x7f3d9c06ff10 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8c00c940 con 0x7f3d9c06ff10 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8c00cd10 con 0x7f3d9c06ff10 2024-03-06T07:23:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c154c00 con 0x7f3d9c06ff10 2024-03-06T07:23:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.483+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c06db80 con 0x7f3d9c06ff10 2024-03-06T07:23:49.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.484+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c04f330 con 0x7f3d9c06ff10 2024-03-06T07:23:49.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.484+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d8c01a070 con 0x7f3d9c06ff10 2024-03-06T07:23:49.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.485+0000 7f3d837fe640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 0x7f3d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:49.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.485+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3d8c08a2c0 con 0x7f3d9c06ff10 2024-03-06T07:23:49.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.485+0000 7f3d9a7fc640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 0x7f3d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:49.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.486+0000 7f3d9a7fc640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 0x7f3d8806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0c2820 tx=0x7f3d90009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.489+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d8c058ca0 con 0x7f3d9c06ff10 2024-03-06T07:23:49.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:49.684+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3d9c06fa80 con 0x7f3d9c06ff10 2024-03-06T07:23:50.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.491+0000 7f3d837fe640 1 -- 172.21.15.17:0/3603185584 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3d8c05cd40 con 0x7f3d9c06ff10 2024-03-06T07:23:50.494 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 msgr2=0x7f3d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 0x7f3d8806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0c2820 tx=0x7f3d90009190 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 msgr2=0x7f3d9c0c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00e890 tx=0x7f3d8c00ed50 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 shutdown_connections 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d8806b4d0 0x7f3d8806d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d9c0c7170 0x7f3d9c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d9c0c6c40 0x7f3d9c14cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 --2- 172.21.15.17:0/3603185584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d9c06ff10 0x7f3d9c0c1f20 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 >> 172.21.15.17:0/3603185584 conn(0x7f3d9c06b970 msgr2=0x7f3d9c152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 shutdown_connections 2024-03-06T07:23:50.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.494+0000 7f3da13ac640 1 -- 172.21.15.17:0/3603185584 wait complete. 2024-03-06T07:23:50.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:23:50.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.614+0000 7f3e2770f640 1 Processor -- start 2024-03-06T07:23:50.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 -- start start 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e20155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e20158b60 0x7f3e20158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e200c73d0 con 0x7f3e200c88c0 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e200c8dd0 con 0x7f3e20158b60 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e2770f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200be2e0 con 0x7f3e2014f360 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e24caa640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3e200c88c0 0x7f3e200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41094/0 (socket says 172.21.15.17:41094) 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.615+0000 7f3e24caa640 1 -- 172.21.15.17:0/2217745938 learned_addr learned my addr 172.21.15.17:0/2217745938 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e20155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e25cac640 1 --2- 172.21.15.17:0/2217745938 >> v2:172.21.15.163:3300/0 conn(0x7f3e20158b60 0x7f3e20158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853286678 0 0) 0x7f3e200c73d0 con 0x7f3e200c88c0 2024-03-06T07:23:50.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e08003520 con 0x7f3e200c88c0 2024-03-06T07:23:50.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701102812 0 0) 0x7f3e08003520 con 0x7f3e200c88c0 2024-03-06T07:23:50.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 msgr2=0x7f3e20155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e20155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 >> v2:172.21.15.163:3300/0 conn(0x7f3e20158b60 msgr2=0x7f3e20158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2217745938 >> v2:172.21.15.163:3300/0 conn(0x7f3e20158b60 0x7f3e20158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200c7670 con 0x7f3e200c88c0 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3e14001670 con 0x7f3e200c88c0 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e20155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e25cac640 1 --2- 172.21.15.17:0/2217745938 >> v2:172.21.15.163:3300/0 conn(0x7f3e20158b60 0x7f3e20158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 >> v1:172.21.15.17:6789/0 conn(0x7f3e200c88c0 legacy=0x7f3e200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e080039b0 0x7f3e08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08004d40 0x7f3e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200be2e0 con 0x7f3e080039b0 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0800b9c0 con 0x7f3e08004d40 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0800bb40 con 0x7f3e2014f360 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e080039b0 0x7f3e08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e24caa640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.616+0000 7f3e25cac640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08004d40 0x7f3e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 -- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 msgr2=0x7f3e08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e2014f360 0x7f3e08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 -- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08004d40 msgr2=0x7f3e080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e08004d40 0x7f3e080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 -- 172.21.15.17:0/2217745938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08003520 con 0x7f3e080039b0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.617+0000 7f3e254ab640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e080039b0 0x7f3e08003dd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009c90 tx=0x7f3e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cc40a36021c06d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18039070 con 0x7f3e080039b0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1802c8f0 con 0x7f3e080039b0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e1ffff640 1 -- 172.21.15.17:0/2217745938 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18009860 con 0x7f3e080039b0 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 -- 172.21.15.17:0/2217745938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e080039b0 msgr2=0x7f3e08003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 --2- 172.21.15.17:0/2217745938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e080039b0 0x7f3e08003dd0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009c90 tx=0x7f3e08005830 comp rx=0 tx=0).stop 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e25cac640 1 -- 172.21.15.17:0/2217745938 reap_dead start 2024-03-06T07:23:50.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 -- 172.21.15.17:0/2217745938 shutdown_connections 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 -- 172.21.15.17:0/2217745938 >> 172.21.15.17:0/2217745938 conn(0x7f3e200ac570 msgr2=0x7f3e200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 -- 172.21.15.17:0/2217745938 shutdown_connections 2024-03-06T07:23:50.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.618+0000 7f3e2770f640 1 -- 172.21.15.17:0/2217745938 wait complete. 2024-03-06T07:23:50.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.619+0000 7f3e2770f640 1 Processor -- start 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.619+0000 7f3e2770f640 1 -- start start 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e080039b0 0x7f3e201ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e200b0220 0x7f3e200ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c5a50 con 0x7f3e20158470 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c7c30 con 0x7f3e200b0220 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e2770f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c5bd0 con 0x7f3e080039b0 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56702/0 (socket says 172.21.15.17:56702) 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 -- 172.21.15.17:0/3163593381 learned_addr learned my addr 172.21.15.17:0/3163593381 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e24caa640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e200b0220 0x7f3e200ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e254ab640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e080039b0 0x7f3e201ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 -- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e080039b0 msgr2=0x7f3e201ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e080039b0 0x7f3e201ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 -- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e200b0220 msgr2=0x7f3e200ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e200b0220 0x7f3e200ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.620+0000 7f3e25cac640 1 -- 172.21.15.17:0/3163593381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e201ead80 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e25cac640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3e200c4fc0 tx=0x7f3e1000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003990 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e10003b30 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e200ad600 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10004020 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.621+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201eeda0 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.622+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2004f330 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.623+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e100041c0 con 0x7f3e20158470 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.623+0000 7f3e1dffb640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 0x7f3e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.623+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3e1008aca0 con 0x7f3e20158470 2024-03-06T07:23:50.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.623+0000 7f3e254ab640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 0x7f3e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.624+0000 7f3e254ab640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 0x7f3e0c06d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3e18036000 tx=0x7f3e18030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:50.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.627+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3e10019030 con 0x7f3e20158470 2024-03-06T07:23:50.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.877+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3e200b0660 con 0x7f3e20158470 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.877+0000 7f3e1dffb640 1 -- 172.21.15.17:0/3163593381 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f3e10010040 con 0x7f3e20158470 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.880 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:23:50.882 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.883 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.884 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.886 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.887 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.888 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:23:50.889 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:23:50.890 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:23:50.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:23:50.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:50.892 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:23:50.892 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 msgr2=0x7f3e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 0x7f3e0c06d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3e18036000 tx=0x7f3e18030000 comp rx=0 tx=0).stop 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 msgr2=0x7f3e201eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3e200c4fc0 tx=0x7f3e1000f580 comp rx=0 tx=0).stop 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 shutdown_connections 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3e0c06b4d0 0x7f3e0c06d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e20158470 0x7f3e201eb560 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e200b0220 0x7f3e200ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 --2- 172.21.15.17:0/3163593381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e080039b0 0x7f3e201ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 >> 172.21.15.17:0/3163593381 conn(0x7f3e200ac570 msgr2=0x7f3e200ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 shutdown_connections 2024-03-06T07:23:50.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.879+0000 7f3e2770f640 1 -- 172.21.15.17:0/3163593381 wait complete. 2024-03-06T07:23:50.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:50.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.993+0000 7f41e1956640 1 Processor -- start 2024-03-06T07:23:50.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 -- start start 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 0x7f41dc06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41dc0c1320 0x7f41dc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 0x7f41dc1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc153a60 con 0x7f41dc0c1320 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41dc153d40 con 0x7f41dc150580 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41e1956640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc06c2c0 con 0x7f41dc0c6000 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41da7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41dc0c1320 0x7f41dc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41da7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f41dc0c1320 0x7f41dc06e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56708/0 (socket says 172.21.15.17:56708) 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.994+0000 7f41da7fc640 1 -- 172.21.15.17:0/381500893 learned_addr learned my addr 172.21.15.17:0/381500893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41daffd640 1 --2- 172.21.15.17:0/381500893 >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 0x7f41dc06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 -- 172.21.15.17:0/381500893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 msgr2=0x7f41dc1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41db7fe640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 0x7f41dc1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 0x7f41dc1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 -- 172.21.15.17:0/381500893 >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 msgr2=0x7f41dc06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 --2- 172.21.15.17:0/381500893 >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 0x7f41dc06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 -- 172.21.15.17:0/381500893 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc06c5a0 con 0x7f41dc0c1320 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41daffd640 1 --2- 172.21.15.17:0/381500893 >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 0x7f41dc06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41db7fe640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 0x7f41dc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.995+0000 7f41da7fc640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc06e250 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a9550 tx=0x7f41d002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cff9efaf051dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41d9ffb640 1 -- 172.21.15.17:0/381500893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00317d0 con 0x7f41dc0c1320 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41d9ffb640 1 -- 172.21.15.17:0/381500893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d0037070 con 0x7f41dc0c1320 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41d9ffb640 1 -- 172.21.15.17:0/381500893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0031b90 con 0x7f41dc0c1320 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 -- 172.21.15.17:0/381500893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 msgr2=0x7f41dc06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc06e250 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0a9550 tx=0x7f41d002b870 comp rx=0 tx=0).stop 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 -- 172.21.15.17:0/381500893 shutdown_connections 2024-03-06T07:23:51.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc0c6000 0x7f41dc1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 --2- 172.21.15.17:0/381500893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc06e250 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 --2- 172.21.15.17:0/381500893 >> v2:172.21.15.163:3300/0 conn(0x7f41dc150580 0x7f41dc06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 -- 172.21.15.17:0/381500893 >> 172.21.15.17:0/381500893 conn(0x7f41dc0c6710 msgr2=0x7f41dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 -- 172.21.15.17:0/381500893 shutdown_connections 2024-03-06T07:23:51.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.996+0000 7f41e1956640 1 -- 172.21.15.17:0/381500893 wait complete. 2024-03-06T07:23:51.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.997+0000 7f41e1956640 1 Processor -- start 2024-03-06T07:23:51.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.997+0000 7f41e1956640 1 -- start start 2024-03-06T07:23:51.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 0x7f41dc0bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 0x7f41dc1ead50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c0dc0 con 0x7f41dc0c1320 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c0c40 con 0x7f41dc0c6000 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41e1956640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc156bf0 con 0x7f41dc150580 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56714/0 (socket says 172.21.15.17:56714) 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41daffd640 1 -- 172.21.15.17:0/3424127010 learned_addr learned my addr 172.21.15.17:0/3424127010 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.998+0000 7f41da7fc640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 0x7f41dc0bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41db7fe640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 0x7f41dc1ead50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 -- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 msgr2=0x7f41dc1ead50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 0x7f41dc1ead50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 -- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 msgr2=0x7f41dc0bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 0x7f41dc0bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 -- 172.21.15.17:0/3424127010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0aa540 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41da7fc640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 0x7f41dc0bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41db7fe640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 0x7f41dc1ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41daffd640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00c7f0 tx=0x7f41cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc015070 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc007820 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0aa820 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc007bd0 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:50.999+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc0c02f0 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.000+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc067ad0 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.000+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41cc018070 con 0x7f41dc0c1320 2024-03-06T07:23:51.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.001+0000 7f41c3fff640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.001+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f41cc08a160 con 0x7f41dc0c1320 2024-03-06T07:23:51.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.001+0000 7f41da7fc640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.002+0000 7f41da7fc640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 0x7f41c806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036000 tx=0x7f41d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.005+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41dc067ad0 con 0x7f41dc0c1320 2024-03-06T07:23:51.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.188+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41dc04f330 con 0x7f41dc0c1320 2024-03-06T07:23:51.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.189+0000 7f41c3fff640 1 -- 172.21.15.17:0/3424127010 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+13166 (secure 0 0 0) 0x7f41cc086380 con 0x7f41dc0c1320 2024-03-06T07:23:51.191 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:51.191 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":105,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:50.491750+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:51.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 msgr2=0x7f41c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 0x7f41c806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f41d0036000 tx=0x7f41d0008030 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 msgr2=0x7f41dc1ee9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00c7f0 tx=0x7f41cc00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 shutdown_connections 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f41c806b4d0 0x7f41c806d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41dc150580 0x7f41dc1ead50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41dc0c6000 0x7f41dc0bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 --2- 172.21.15.17:0/3424127010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f41dc0c1320 0x7f41dc1ee9d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 >> 172.21.15.17:0/3424127010 conn(0x7f41dc0c6710 msgr2=0x7f41dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 shutdown_connections 2024-03-06T07:23:51.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.191+0000 7f41e1956640 1 -- 172.21.15.17:0/3424127010 wait complete. 2024-03-06T07:23:51.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:23:51.205 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:23:51.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.307+0000 7f3f2ed63640 1 Processor -- start 2024-03-06T07:23:51.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.307+0000 7f3f2ed63640 1 -- start start 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2ed63640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f281529f0 0x7f3f280c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2ed63640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 0x7f3f280c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2ed63640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f28152dd0 con 0x7f3f28155530 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2ed63640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f28159710 con 0x7f3f281529f0 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2ed63640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f280c68b0 con 0x7f3f280c5370 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f281529f0 0x7f3f280c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f281529f0 0x7f3f280c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56726/0 (socket says 172.21.15.17:56726) 2024-03-06T07:23:51.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 -- 172.21.15.17:0/3580011597 learned_addr learned my addr 172.21.15.17:0/3580011597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 -- 172.21.15.17:0/3580011597 >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 msgr2=0x7f3f280c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f27fff640 1 --2- 172.21.15.17:0/3580011597 >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 0x7f3f280c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 --2- 172.21.15.17:0/3580011597 >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 0x7f3f280c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 -- 172.21.15.17:0/3580011597 >> v1:172.21.15.163:6789/0 conn(0x7f3f28155530 legacy=0x7f3f281559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f2caff640 1 -- 172.21.15.17:0/3580011597 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f28158b00 con 0x7f3f281529f0 2024-03-06T07:23:51.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.308+0000 7f3f27fff640 1 --2- 172.21.15.17:0/3580011597 >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 0x7f3f280c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.309+0000 7f3f2caff640 1 --2- 172.21.15.17:0/3580011597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f280c88f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3f2808c9b0 tx=0x7f3f1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9832f43091b1cf2e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.309+0000 7f3f277fe640 1 -- 172.21.15.17:0/3580011597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c02e070 con 0x7f3f281529f0 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.309+0000 7f3f277fe640 1 -- 172.21.15.17:0/3580011597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1c037070 con 0x7f3f281529f0 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.309+0000 7f3f277fe640 1 -- 172.21.15.17:0/3580011597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1c031a00 con 0x7f3f281529f0 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 -- 172.21.15.17:0/3580011597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 msgr2=0x7f3f280c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/3580011597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f280c88f0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3f2808c9b0 tx=0x7f3f1c02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 -- 172.21.15.17:0/3580011597 shutdown_connections 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/3580011597 >> v2:172.21.15.163:3300/0 conn(0x7f3f280c5370 0x7f3f280c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/3580011597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f280c88f0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 -- 172.21.15.17:0/3580011597 >> 172.21.15.17:0/3580011597 conn(0x7f3f280ac580 msgr2=0x7f3f280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 -- 172.21.15.17:0/3580011597 shutdown_connections 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.310+0000 7f3f2ed63640 1 -- 172.21.15.17:0/3580011597 wait complete. 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.311+0000 7f3f2ed63640 1 Processor -- start 2024-03-06T07:23:51.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.311+0000 7f3f2ed63640 1 -- start start 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f280c5370 0x7f3f281e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 0x7f3f28144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f281593a0 con 0x7f3f281529f0 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f281590c0 con 0x7f3f28155530 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f281560d0 con 0x7f3f280c5370 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56742/0 (socket says 172.21.15.17:56742) 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 -- 172.21.15.17:0/1507382783 learned_addr learned my addr 172.21.15.17:0/1507382783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2caff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f280c5370 0x7f3f281e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2d300640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 0x7f3f28144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 -- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f280c5370 msgr2=0x7f3f281e1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f280c5370 0x7f3f281e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 -- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 msgr2=0x7f3f28144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 0x7f3f28144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 -- 172.21.15.17:0/1507382783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f281ee180 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2d300640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 0x7f3f28144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f27fff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400c820 tx=0x7f3f1400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14015070 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14007a30 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14007e00 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f281ee460 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.312+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f281eea20 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.313+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2804f330 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.314+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f14018070 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.314+0000 7f3f257fa640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 0x7f3f0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.314+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f14089f40 con 0x7f3f281529f0 2024-03-06T07:23:51.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.314+0000 7f3f2caff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 0x7f3f0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.315+0000 7f3f2caff640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 0x7f3f0c06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c036000 tx=0x7f3f1c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.319+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f14059010 con 0x7f3f281529f0 2024-03-06T07:23:51.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.497+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f28145520 con 0x7f3f281529f0 2024-03-06T07:23:51.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.497+0000 7f3f257fa640 1 -- 172.21.15.17:0/1507382783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f14058910 con 0x7f3f281529f0 2024-03-06T07:23:51.500 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 msgr2=0x7f3f0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 0x7f3f0c06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c036000 tx=0x7f3f1c008030 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 msgr2=0x7f3f28144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400c820 tx=0x7f3f1400cc20 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 shutdown_connections 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f0c06b500 0x7f3f0c06d9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f28155530 0x7f3f28144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f281529f0 0x7f3f28144aa0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 --2- 172.21.15.17:0/1507382783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f280c5370 0x7f3f281e1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 >> 172.21.15.17:0/1507382783 conn(0x7f3f280ac580 msgr2=0x7f3f28150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 shutdown_connections 2024-03-06T07:23:51.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.499+0000 7f3f2ed63640 1 -- 172.21.15.17:0/1507382783 wait complete. 2024-03-06T07:23:51.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:23:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.611+0000 7f7297763640 1 Processor -- start 2024-03-06T07:23:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.611+0000 7f7297763640 1 -- start start 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f7297763640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72900c88c0 0x7f7290155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f7297763640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900c73d0 con 0x7f729014f360 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f7297763640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72900be2e0 con 0x7f7290158b60 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f7297763640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7290159700 con 0x7f72900c88c0 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f72954ff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f729014f360 0x7f72900c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41110/0 (socket says 172.21.15.17:41110) 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.612+0000 7f72954ff640 1 -- 172.21.15.17:0/506873579 learned_addr learned my addr 172.21.15.17:0/506873579 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604138871 0 0) 0x7f72900c73d0 con 0x7f729014f360 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7278003520 con 0x7f729014f360 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7294cfe640 1 --2- 172.21.15.17:0/506873579 >> v2:172.21.15.163:3300/0 conn(0x7f72900c88c0 0x7f7290155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743325585 0 0) 0x7f7278003520 con 0x7f729014f360 2024-03-06T07:23:51.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 >> v2:172.21.15.163:3300/0 conn(0x7f72900c88c0 msgr2=0x7f7290155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 --2- 172.21.15.17:0/506873579 >> v2:172.21.15.163:3300/0 conn(0x7f72900c88c0 0x7f7290155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 >> v1:172.21.15.163:6789/0 conn(0x7f7290158b60 legacy=0x7f7290158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900c7670 con 0x7f729014f360 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7288001670 con 0x7f729014f360 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 >> v1:172.21.15.17:6789/0 conn(0x7f729014f360 legacy=0x7f72900c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72780039b0 0x7f7278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7278004800 0x7f7278004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 0x7f727800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290159700 con 0x7f72780051c0 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727800c160 con 0x7f7278004800 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727800c360 con 0x7f72780039b0 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 0x7f727800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f72954ff640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72780039b0 0x7f7278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7294cfe640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7278004800 0x7f7278004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 -- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72780039b0 msgr2=0x7f7278003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72780039b0 0x7f7278003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 -- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7278004800 msgr2=0x7f7278004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7278004800 0x7f7278004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.613+0000 7f7295d00640 1 -- 172.21.15.17:0/506873579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278003520 con 0x7f72780051c0 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.614+0000 7f7295d00640 1 -- 172.21.15.17:0/506873579 reap_dead start 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.614+0000 7f7295d00640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 0x7f727800b9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7280001e50 tx=0x7f7278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d0de56c629e2e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.614+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280001a70 con 0x7f72780051c0 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.614+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280003b90 con 0x7f72780051c0 2024-03-06T07:23:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.614+0000 7f728ffff640 1 -- 172.21.15.17:0/506873579 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800050d0 con 0x7f72780051c0 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 -- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 msgr2=0x7f727800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 0x7f727800b9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7280001e50 tx=0x7f7278005830 comp rx=0 tx=0).stop 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 -- 172.21.15.17:0/506873579 shutdown_connections 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 --2- 172.21.15.17:0/506873579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72780051c0 0x7f727800b9c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 -- 172.21.15.17:0/506873579 >> 172.21.15.17:0/506873579 conn(0x7f72900ac570 msgr2=0x7f72900ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 -- 172.21.15.17:0/506873579 shutdown_connections 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.615+0000 7f7297763640 1 -- 172.21.15.17:0/506873579 wait complete. 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.616+0000 7f7297763640 1 Processor -- start 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.616+0000 7f7297763640 1 -- start start 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.616+0000 7f7297763640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 0x7f72901eac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.616+0000 7f7297763640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7297763640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 0x7f729014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7297763640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900c7c30 con 0x7f7290154240 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7297763640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7290155c10 con 0x7f72780051c0 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7297763640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7290155d90 con 0x7f729014f8d0 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56752/0 (socket says 172.21.15.17:56752) 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 -- 172.21.15.17:0/3137976086 learned_addr learned my addr 172.21.15.17:0/3137976086 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7295d00640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 0x7f729014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f72954ff640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 0x7f72901eac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 -- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 msgr2=0x7f729014fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 0x7f729014fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 -- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 msgr2=0x7f72901eac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 0x7f72901eac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 -- 172.21.15.17:0/3137976086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290154dd0 con 0x7f7290154240 2024-03-06T07:23:51.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f72954ff640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 0x7f72901eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7295d00640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 0x7f729014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7294cfe640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7284011960 tx=0x7f7284011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284019070 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7284004240 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7290152360 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.617+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400f820 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.618+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900aef50 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.618+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729004f330 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.619+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f728401c070 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.619+0000 7f728dffb640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 0x7f727c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.619+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f728408a4a0 con 0x7f7290154240 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.620+0000 7f72954ff640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 0x7f727c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.620+0000 7f72954ff640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 0x7f727c06d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72901e8a70 tx=0x7f7288009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.623+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72840594d0 con 0x7f7290154240 2024-03-06T07:23:51.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.809+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f72900c8d50 con 0x7f7290154240 2024-03-06T07:23:51.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.810+0000 7f728dffb640 1 -- 172.21.15.17:0/3137976086 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7284058dd0 con 0x7f7290154240 2024-03-06T07:23:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:51.812 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:23:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:23:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:23:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:23:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:23:51.813 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:23:51.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 msgr2=0x7f727c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 0x7f727c06d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f72901e8a70 tx=0x7f7288009040 comp rx=0 tx=0).stop 2024-03-06T07:23:51.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 msgr2=0x7f72900af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7284011960 tx=0x7f7284011e20 comp rx=0 tx=0).stop 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 shutdown_connections 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f727c06b500 0x7f727c06d9c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f729014f8d0 0x7f729014fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7290154240 0x7f72900af990 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 --2- 172.21.15.17:0/3137976086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72780051c0 0x7f72901eac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 >> 172.21.15.17:0/3137976086 conn(0x7f72900ac570 msgr2=0x7f72900c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 shutdown_connections 2024-03-06T07:23:51.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.811+0000 7f7297763640 1 -- 172.21.15.17:0/3137976086 wait complete. 2024-03-06T07:23:51.825 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:23:51.825 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:23:51.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.927+0000 7f9810bf7640 1 Processor -- start 2024-03-06T07:23:51.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.927+0000 7f9810bf7640 1 -- start start 2024-03-06T07:23:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f9810bf7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f980c0c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f9810bf7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c0c5400 con 0x7f980c0c7930 2024-03-06T07:23:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f9810bf7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c0c56e0 con 0x7f980c152870 2024-03-06T07:23:51.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f9810bf7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c06d320 con 0x7f980c0c88e0 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980ad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f980c0c7930 0x7f980c06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41120/0 (socket says 172.21.15.17:41120) 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980ad9d640 1 -- 172.21.15.17:0/1168801627 learned_addr learned my addr 172.21.15.17:0/1168801627 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650929652 0 0) 0x7f980c0c5400 con 0x7f980c0c7930 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f9809d9b640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f980c0c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003520 con 0x7f980c0c7930 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668003599 0 0) 0x7f97f4003520 con 0x7f980c0c7930 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 msgr2=0x7f980c0c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.928+0000 7f980959a640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f980c0c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 >> v1:172.21.15.163:6789/0 conn(0x7f980c152870 legacy=0x7f980c155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c0c59c0 con 0x7f980c0c7930 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f97fc0019f0 con 0x7f980c0c7930 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 >> v1:172.21.15.17:6789/0 conn(0x7f980c0c7930 legacy=0x7f980c06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f40039e0 0x7f97f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97f4004800 0x7f97f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f97f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c06d320 con 0x7f97f4004800 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400baf0 con 0x7f97f40039e0 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400bcf0 con 0x7f980c0c88e0 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97f4004800 0x7f97f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980a59c640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f40039e0 0x7f97f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f980ad9d640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f97f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 -- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 msgr2=0x7f97f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0c88e0 0x7f97f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 -- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f40039e0 msgr2=0x7f97f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:51.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f40039e0 0x7f97f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.929+0000 7f9809d9b640 1 -- 172.21.15.17:0/1168801627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4003520 con 0x7f97f4004800 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f9809d9b640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97f4004800 0x7f97f4004c80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9800009a90 tx=0x7f97f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24263ee2057048c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800039070 con 0x7f97f4004800 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980002c680 con 0x7f97f4004800 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f980959a640 1 -- 172.21.15.17:0/1168801627 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800009690 con 0x7f97f4004800 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f9810bf7640 1 -- 172.21.15.17:0/1168801627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97f4004800 msgr2=0x7f97f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f9810bf7640 1 --2- 172.21.15.17:0/1168801627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97f4004800 0x7f97f4004c80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f9800009a90 tx=0x7f97f4005830 comp rx=0 tx=0).stop 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.930+0000 7f980ad9d640 1 -- 172.21.15.17:0/1168801627 reap_dead start 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.931+0000 7f9810bf7640 1 -- 172.21.15.17:0/1168801627 shutdown_connections 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.931+0000 7f9810bf7640 1 -- 172.21.15.17:0/1168801627 >> 172.21.15.17:0/1168801627 conn(0x7f980c0ac5f0 msgr2=0x7f980c0aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.931+0000 7f9810bf7640 1 -- 172.21.15.17:0/1168801627 shutdown_connections 2024-03-06T07:23:51.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.931+0000 7f9810bf7640 1 -- 172.21.15.17:0/1168801627 wait complete. 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.932+0000 7f9810bf7640 1 Processor -- start 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.932+0000 7f9810bf7640 1 -- start start 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f4004800 0x7f980c1ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0ae810 0x7f980c148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0c9470 con 0x7f980c0b0550 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c06b8d0 con 0x7f97f4004800 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0c95f0 con 0x7f980c0ae810 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56766/0 (socket says 172.21.15.17:56766) 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 -- 172.21.15.17:0/2423531636 learned_addr learned my addr 172.21.15.17:0/2423531636 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f980a59c640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f4004800 0x7f980c1ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f980ad9d640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0ae810 0x7f980c148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 -- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0ae810 msgr2=0x7f980c148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0ae810 0x7f980c148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 -- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f4004800 msgr2=0x7f980c1ef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f4004800 0x7f980c1ef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 -- 172.21.15.17:0/2423531636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c1ebbd0 con 0x7f980c0b0550 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9809d9b640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f9800004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800042020 con 0x7f980c0b0550 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980003a040 con 0x7f980c0b0550 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c1ebeb0 con 0x7f980c0b0550 2024-03-06T07:23:51.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800039070 con 0x7f980c0b0550 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.933+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c14d870 con 0x7f980c0b0550 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.934+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c0ca6a0 con 0x7f980c0b0550 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.935+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9800045020 con 0x7f980c0b0550 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.935+0000 7f97faffd640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 0x7f97ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.935+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f98000ab440 con 0x7f980c0b0550 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.935+0000 7f980a59c640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 0x7f97ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:51.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.936+0000 7f980a59c640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 0x7f97ec06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9804005590 tx=0x7f9804009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:51.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:51.940+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f980c0ca6a0 con 0x7f980c0b0550 2024-03-06T07:23:52.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.120+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f980c0c85b0 con 0x7f980c0b0550 2024-03-06T07:23:52.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.500+0000 7f97faffd640 1 -- 172.21.15.17:0/2423531636 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) v1 ==== 138+0+0 (secure 0 0 0) 0x7f980c0c85b0 con 0x7f980c0b0550 2024-03-06T07:23:52.503 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 msgr2=0x7f97ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 0x7f97ec06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9804005590 tx=0x7f9804009040 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 msgr2=0x7f980c14d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f9800000c00 tx=0x7f9800004730 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 shutdown_connections 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f97ec06b4d0 0x7f97ec06d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f980c0ae810 0x7f980c148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f980c0b0550 0x7f980c14d330 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 --2- 172.21.15.17:0/2423531636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f4004800 0x7f980c1ef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 >> 172.21.15.17:0/2423531636 conn(0x7f980c0ac5f0 msgr2=0x7f980c150810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:52.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 shutdown_connections 2024-03-06T07:23:52.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.503+0000 7f9810bf7640 1 -- 172.21.15.17:0/2423531636 wait complete. 2024-03-06T07:23:52.521 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:23:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.628+0000 7f0f776df640 1 Processor -- start 2024-03-06T07:23:52.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.629+0000 7f0f776df640 1 -- start start 2024-03-06T07:23:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.629+0000 7f0f776df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f700724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.629+0000 7f0f776df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f70152cb0 con 0x7f0f700c67f0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.629+0000 7f0f776df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f700c38b0 con 0x7f0f70072a10 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.629+0000 7f0f776df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701594f0 con 0x7f0f701528d0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f75c7c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0f700c67f0 0x7f0f7015fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41134/0 (socket says 172.21.15.17:41134) 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f75c7c640 1 -- 172.21.15.17:0/4255158749 learned_addr learned my addr 172.21.15.17:0/4255158749 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712049145 0 0) 0x7f0f70152cb0 con 0x7f0f700c67f0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f58003520 con 0x7f0f700c67f0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f7547b640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f700724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708160835 0 0) 0x7f0f58003520 con 0x7f0f700c67f0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 msgr2=0x7f0f700724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f700724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 >> v1:172.21.15.163:6789/0 conn(0x7f0f70072a10 legacy=0x7f0f70155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700a7f70 con 0x7f0f700c67f0 2024-03-06T07:23:52.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0f600019f0 con 0x7f0f700c67f0 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 >> v1:172.21.15.17:6789/0 conn(0x7f0f700c67f0 legacy=0x7f0f7015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f58003f60 0x7f0f58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701594f0 con 0x7f0f58004d40 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800bb30 con 0x7f0f58003f60 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800bd30 con 0x7f0f701528d0 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f75c7c640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f7547b640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.630+0000 7f0f74c7a640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f58003f60 0x7f0f58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 -- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 msgr2=0x7f0f580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f701528d0 0x7f0f580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 -- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f58003f60 msgr2=0x7f0f58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f58003f60 0x7f0f58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 -- 172.21.15.17:0/4255158749 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58003520 con 0x7f0f58004d40 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f74c7a640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f58003f60 0x7f0f58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f75c7c640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000a850 tx=0x7f0f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbdeb2b683ef5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000a4a0 con 0x7f0f58004d40 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f60034650 con 0x7f0f58004d40 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.631+0000 7f0f6ffff640 1 -- 172.21.15.17:0/4255158749 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60034a80 con 0x7f0f58004d40 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 -- 172.21.15.17:0/4255158749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58004d40 msgr2=0x7f0f580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 --2- 172.21.15.17:0/4255158749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000a850 tx=0x7f0f58005830 comp rx=0 tx=0).stop 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f75c7c640 1 -- 172.21.15.17:0/4255158749 reap_dead start 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 -- 172.21.15.17:0/4255158749 shutdown_connections 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 -- 172.21.15.17:0/4255158749 >> 172.21.15.17:0/4255158749 conn(0x7f0f700aa6a0 msgr2=0x7f0f700aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 -- 172.21.15.17:0/4255158749 shutdown_connections 2024-03-06T07:23:52.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.632+0000 7f0f776df640 1 -- 172.21.15.17:0/4255158749 wait complete. 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.633+0000 7f0f776df640 1 Processor -- start 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.633+0000 7f0f776df640 1 -- start start 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 0x7f0f701ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f701eb8a0 0x7f0f70141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701591a0 con 0x7f0f700a96d0 2024-03-06T07:23:52.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700a8250 con 0x7f0f701eb8a0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f776df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700c4ef0 con 0x7f0f70146680 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56786/0 (socket says 172.21.15.17:56786) 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 -- 172.21.15.17:0/3187546365 learned_addr learned my addr 172.21.15.17:0/3187546365 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f7547b640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 0x7f0f701ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f75c7c640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f701eb8a0 0x7f0f70141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 -- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 msgr2=0x7f0f701ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 0x7f0f701ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 -- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f701eb8a0 msgr2=0x7f0f70141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f701eb8a0 0x7f0f70141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 -- 172.21.15.17:0/3187546365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f701ee880 con 0x7f0f700a96d0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f7547b640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 0x7f0f701ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f74c7a640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400e5b0 tx=0x7f0f6400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.634+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640052e0 con 0x7f0f700a96d0 2024-03-06T07:23:52.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.635+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64018070 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.635+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f701eeb60 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.635+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64003830 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.635+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f701ef120 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.636+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7004f330 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.636+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f640039d0 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.636+0000 7f0f6dffb640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 0x7f0f5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.637+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0f64089c30 con 0x7f0f700a96d0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.637+0000 7f0f7547b640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 0x7f0f5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:52.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.637+0000 7f0f7547b640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 0x7f0f5c06d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009bc0 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:52.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.641+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f64058b90 con 0x7f0f700a96d0 2024-03-06T07:23:52.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:52.821+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0f70073120 con 0x7f0f700a96d0 2024-03-06T07:23:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.505+0000 7f0f6dffb640 1 -- 172.21.15.17:0/3187546365 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0f64058490 con 0x7f0f700a96d0 2024-03-06T07:23:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:23:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.508+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 msgr2=0x7f0f5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.508+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 0x7f0f5c06d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009bc0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2024-03-06T07:23:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.508+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 msgr2=0x7f0f701eb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.508+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0f6400e5b0 tx=0x7f0f6400ea70 comp rx=0 tx=0).stop 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 shutdown_connections 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f5c06b4d0 0x7f0f5c06d990 secure :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009bc0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f701eb8a0 0x7f0f70141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700a96d0 0x7f0f701eb360 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 --2- 172.21.15.17:0/3187546365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f70146680 0x7f0f701ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.509+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 >> 172.21.15.17:0/3187546365 conn(0x7f0f700aa6a0 msgr2=0x7f0f700c5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.510+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 shutdown_connections 2024-03-06T07:23:53.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.510+0000 7f0f776df640 1 -- 172.21.15.17:0/3187546365 wait complete. 2024-03-06T07:23:53.541 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:23:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 Processor -- start 2024-03-06T07:23:53.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 -- start start 2024-03-06T07:23:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e080aa940 0x7f5e080aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 0x7f5e08154af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e081524d0 con 0x7f5e0814ece0 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e08151710 con 0x7f5e080aa940 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.647+0000 7f5e0f777640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e081519f0 con 0x7f5e080c31a0 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e080aa940 0x7f5e080aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e080aa940 0x7f5e080aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56792/0 (socket says 172.21.15.17:56792) 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 -- 172.21.15.17:0/927913384 learned_addr learned my addr 172.21.15.17:0/927913384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0dd14640 1 --2- 172.21.15.17:0/927913384 >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 0x7f5e08154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 -- 172.21.15.17:0/927913384 >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 msgr2=0x7f5e08154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 --2- 172.21.15.17:0/927913384 >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 0x7f5e08154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 -- 172.21.15.17:0/927913384 >> v1:172.21.15.163:6789/0 conn(0x7f5e0814ece0 legacy=0x7f5e080c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0cd12640 1 -- 172.21.15.17:0/927913384 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e08151cd0 con 0x7f5e080aa940 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.648+0000 7f5e0dd14640 1 --2- 172.21.15.17:0/927913384 >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 0x7f5e08154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0cd12640 1 --2- 172.21.15.17:0/927913384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080aada0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e080a69f0 tx=0x7f5dfc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b337c31957a4d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e07fff640 1 -- 172.21.15.17:0/927913384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc02e070 con 0x7f5e080aa940 2024-03-06T07:23:53.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e07fff640 1 -- 172.21.15.17:0/927913384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc037070 con 0x7f5e080aa940 2024-03-06T07:23:53.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e07fff640 1 -- 172.21.15.17:0/927913384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc031a00 con 0x7f5e080aa940 2024-03-06T07:23:53.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 -- 172.21.15.17:0/927913384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 msgr2=0x7f5e080aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 --2- 172.21.15.17:0/927913384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080aada0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5e080a69f0 tx=0x7f5dfc02b880 comp rx=0 tx=0).stop 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 -- 172.21.15.17:0/927913384 shutdown_connections 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 --2- 172.21.15.17:0/927913384 >> v2:172.21.15.163:3300/0 conn(0x7f5e080c31a0 0x7f5e08154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 --2- 172.21.15.17:0/927913384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080aada0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.649+0000 7f5e0f777640 1 -- 172.21.15.17:0/927913384 >> 172.21.15.17:0/927913384 conn(0x7f5e0806afc0 msgr2=0x7f5e0806b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.650+0000 7f5e0f777640 1 -- 172.21.15.17:0/927913384 shutdown_connections 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.650+0000 7f5e0f777640 1 -- 172.21.15.17:0/927913384 wait complete. 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 Processor -- start 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 -- start start 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e080c31a0 0x7f5e080c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e0814ece0 0x7f5e08149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080c2b60 con 0x7f5e080aa940 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e080c29e0 con 0x7f5e0814ece0 2024-03-06T07:23:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0f777640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0806dd30 con 0x7f5e080c31a0 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0d513640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0d513640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56794/0 (socket says 172.21.15.17:56794) 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.651+0000 7f5e0d513640 1 -- 172.21.15.17:0/3572412939 learned_addr learned my addr 172.21.15.17:0/3572412939 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 -- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e080c31a0 msgr2=0x7f5e080c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e080c31a0 0x7f5e080c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 -- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e0814ece0 msgr2=0x7f5e08149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e0814ece0 0x7f5e08149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 -- 172.21.15.17:0/3572412939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0814c500 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0d513640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000c730 tx=0x7f5e0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00016020 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00006d50 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0814c7a0 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00003bb0 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.652+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e080c4c80 con 0x7f5e080aa940 2024-03-06T07:23:53.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.653+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0804f330 con 0x7f5e080aa940 2024-03-06T07:23:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.654+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e0000a850 con 0x7f5e080aa940 2024-03-06T07:23:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.654+0000 7f5e05ffb640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 0x7f5df406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.654+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f5e00089f10 con 0x7f5e080aa940 2024-03-06T07:23:53.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.654+0000 7f5e0cd12640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 0x7f5df406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.655+0000 7f5e0cd12640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 0x7f5df406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc036000 tx=0x7f5dfc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:53.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.659+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5e00020020 con 0x7f5e080aa940 2024-03-06T07:23:53.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:53.881+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5e0806e550 con 0x7f5e080aa940 2024-03-06T07:23:54.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.507+0000 7f5e05ffb640 1 -- 172.21.15.17:0/3572412939 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5e00058cf0 con 0x7f5e080aa940 2024-03-06T07:23:54.510 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 17 and data pool 18 2024-03-06T07:23:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.513+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 msgr2=0x7f5df406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.513+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 0x7f5df406d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc036000 tx=0x7f5dfc030000 comp rx=0 tx=0).stop 2024-03-06T07:23:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.513+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 msgr2=0x7f5e080c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.513+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000c730 tx=0x7f5e0000cbf0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.518+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 shutdown_connections 2024-03-06T07:23:54.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.519+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5df406b500 0x7f5df406d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.519+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e0814ece0 0x7f5e08149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.519+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e080c31a0 0x7f5e080c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.519+0000 7f5e0f777640 1 --2- 172.21.15.17:0/3572412939 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e080aa940 0x7f5e080c4510 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.519+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 >> 172.21.15.17:0/3572412939 conn(0x7f5e0806afc0 msgr2=0x7f5e0806b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:54.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.527+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 shutdown_connections 2024-03-06T07:23:54.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.527+0000 7f5e0f777640 1 -- 172.21.15.17:0/3572412939 wait complete. 2024-03-06T07:23:54.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:54.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.654+0000 7f461824e640 1 Processor -- start 2024-03-06T07:23:54.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.654+0000 7f461824e640 1 -- start start 2024-03-06T07:23:54.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f461824e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46100c7180 0x7f4610159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f461824e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100c7c60 con 0x7f46100c88b0 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f461824e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46100c8060 con 0x7f46101527b0 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f461824e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4610159640 con 0x7f46100c7180 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46167eb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46100c88b0 0x7f46100c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41148/0 (socket says 172.21.15.17:41148) 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46167eb640 1 -- 172.21.15.17:0/3998774552 learned_addr learned my addr 172.21.15.17:0/3998774552 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46157e9640 1 --2- 172.21.15.17:0/3998774552 >> v2:172.21.15.163:3300/0 conn(0x7f46100c7180 0x7f4610159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366380736 0 0) 0x7f4600001490 con 0x7f46100c88b0 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f8003520 con 0x7f46100c88b0 2024-03-06T07:23:54.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215402852 0 0) 0x7f46100c8060 con 0x7f46101527b0 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4600001490 con 0x7f46101527b0 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922310733 0 0) 0x7f45f8003520 con 0x7f46100c88b0 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 >> v2:172.21.15.163:3300/0 conn(0x7f46100c7180 msgr2=0x7f4610159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 --2- 172.21.15.17:0/3998774552 >> v2:172.21.15.163:3300/0 conn(0x7f46100c7180 0x7f4610159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 >> v1:172.21.15.163:6789/0 conn(0x7f46101527b0 legacy=0x7f4610155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100c76a0 con 0x7f46100c88b0 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4600001bc0 con 0x7f46100c88b0 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46157e9640 1 --2- 172.21.15.17:0/3998774552 >> v2:172.21.15.163:3300/0 conn(0x7f46100c7180 0x7f4610159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 >> v1:172.21.15.17:6789/0 conn(0x7f46100c88b0 legacy=0x7f46100c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 0x7f45f8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45f8004800 0x7f45f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f80051c0 0x7f45f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610159640 con 0x7f45f8003b40 2024-03-06T07:23:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f800c0f0 con 0x7f45f80051c0 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f800c2f0 con 0x7f45f8004800 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 0x7f45f8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46167eb640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f80051c0 0x7f45f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46157e9640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45f8004800 0x7f45f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 -- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45f8004800 msgr2=0x7f45f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45f8004800 0x7f45f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 -- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f80051c0 msgr2=0x7f45f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f80051c0 0x7f45f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f4615fea640 1 -- 172.21.15.17:0/3998774552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8003520 con 0x7f45f8003b40 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46167eb640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f80051c0 0x7f45f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.656+0000 7f46167eb640 1 -- 172.21.15.17:0/3998774552 reap_dead start 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.657+0000 7f4615fea640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 0x7f45f8003f40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f46080018d0 tx=0x7f45f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2651b40ee2c2224b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.657+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608037070 con 0x7f45f8003b40 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.657+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46080052d0 con 0x7f45f8003b40 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.657+0000 7f4614fe8640 1 -- 172.21.15.17:0/3998774552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080014f0 con 0x7f45f8003b40 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 -- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 msgr2=0x7f45f8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 0x7f45f8003f40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f46080018d0 tx=0x7f45f8005830 comp rx=0 tx=0).stop 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 -- 172.21.15.17:0/3998774552 shutdown_connections 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 --2- 172.21.15.17:0/3998774552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45f8003b40 0x7f45f8003f40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 -- 172.21.15.17:0/3998774552 >> 172.21.15.17:0/3998774552 conn(0x7f46100ac560 msgr2=0x7f46100ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 -- 172.21.15.17:0/3998774552 shutdown_connections 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.658+0000 7f461824e640 1 -- 172.21.15.17:0/3998774552 wait complete. 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.659+0000 7f461824e640 1 Processor -- start 2024-03-06T07:23:54.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.659+0000 7f461824e640 1 -- start start 2024-03-06T07:23:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f8003b40 0x7f46101eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46100ad650 0x7f46101ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610155600 con 0x7f46100af9b0 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610155320 con 0x7f45f8003b40 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4610152b90 con 0x7f46100ad650 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56816/0 (socket says 172.21.15.17:56816) 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 -- 172.21.15.17:0/987890572 learned_addr learned my addr 172.21.15.17:0/987890572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 -- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46100ad650 msgr2=0x7f46101ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46100ad650 0x7f46101ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 -- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f8003b40 msgr2=0x7f46101eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f4615fea640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f8003b40 0x7f46101eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f8003b40 0x7f46101eba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 -- 172.21.15.17:0/987890572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100c45e0 con 0x7f46100af9b0 2024-03-06T07:23:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f46157e9640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4604011880 tx=0x7f4604011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400fb70 con 0x7f46100af9b0 2024-03-06T07:23:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400fd10 con 0x7f46100af9b0 2024-03-06T07:23:54.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46100c48c0 con 0x7f46100af9b0 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604015660 con 0x7f46100af9b0 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.660+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4610150d20 con 0x7f46100af9b0 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.662+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461004f330 con 0x7f46100af9b0 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.662+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4604015800 con 0x7f46100af9b0 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.662+0000 7f460e7fc640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.662+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f460408ae50 con 0x7f46100af9b0 2024-03-06T07:23:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.663+0000 7f4615fea640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.663+0000 7f4615fea640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 0x7f45fc06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4608034000 tx=0x7f4608031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.667+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4604019030 con 0x7f46100af9b0 2024-03-06T07:23:54.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.851+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46100aa4b0 con 0x7f46100af9b0 2024-03-06T07:23:54.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.853+0000 7f460e7fc640 1 -- 172.21.15.17:0/987890572 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16965 (secure 0 0 0) 0x7f4604059bd0 con 0x7f46100af9b0 2024-03-06T07:23:54.854 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:54.854 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":108,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:53.883840+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:54.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 msgr2=0x7f45fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 0x7f45fc06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4608034000 tx=0x7f4608031000 comp rx=0 tx=0).stop 2024-03-06T07:23:54.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 msgr2=0x7f46101507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4604011880 tx=0x7f4604011d40 comp rx=0 tx=0).stop 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 shutdown_connections 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46100ad650 0x7f46101ee450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46100af9b0 0x7f46101507e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 --2- 172.21.15.17:0/987890572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45f8003b40 0x7f46101eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 >> 172.21.15.17:0/987890572 conn(0x7f46100ac560 msgr2=0x7f46100c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 shutdown_connections 2024-03-06T07:23:54.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.855+0000 7f461824e640 1 -- 172.21.15.17:0/987890572 wait complete. 2024-03-06T07:23:54.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:54.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.967+0000 7fa84e866640 1 Processor -- start 2024-03-06T07:23:54.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.967+0000 7fa84e866640 1 -- start start 2024-03-06T07:23:54.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84e866640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84e866640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8480c36e0 0x7fa8481558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84e866640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa848158800 con 0x7fa8480c3280 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84e866640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8480c3b10 con 0x7fa8480c36e0 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84e866640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa848158ae0 con 0x7fa84806d9f0 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa847fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa847fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49150/0 (socket says 172.21.15.17:49150) 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa847fff640 1 -- 172.21.15.17:0/985789225 learned_addr learned my addr 172.21.15.17:0/985789225 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.968+0000 7fa84ce03640 1 --2- 172.21.15.17:0/985789225 >> v2:172.21.15.163:3300/0 conn(0x7fa8480c36e0 0x7fa8481558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.969+0000 7fa847fff640 1 -- 172.21.15.17:0/985789225 >> v2:172.21.15.163:3300/0 conn(0x7fa8480c36e0 msgr2=0x7fa8481558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.969+0000 7fa847fff640 1 --2- 172.21.15.17:0/985789225 >> v2:172.21.15.163:3300/0 conn(0x7fa8480c36e0 0x7fa8481558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.969+0000 7fa847fff640 1 -- 172.21.15.17:0/985789225 >> v1:172.21.15.163:6789/0 conn(0x7fa8480c3280 legacy=0x7fa8481551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.969+0000 7fa847fff640 1 -- 172.21.15.17:0/985789225 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848158dc0 con 0x7fa84806d9f0 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.969+0000 7fa847fff640 1 --2- 172.21.15.17:0/985789225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009870 tx=0x7fa83c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9652160d7541afe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa846ffd640 1 -- 172.21.15.17:0/985789225 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c035020 con 0x7fa84806d9f0 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa846ffd640 1 -- 172.21.15.17:0/985789225 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c031640 con 0x7fa84806d9f0 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa846ffd640 1 -- 172.21.15.17:0/985789225 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c02d040 con 0x7fa84806d9f0 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 -- 172.21.15.17:0/985789225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 msgr2=0x7fa8480a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 --2- 172.21.15.17:0/985789225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009870 tx=0x7fa83c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 -- 172.21.15.17:0/985789225 shutdown_connections 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 --2- 172.21.15.17:0/985789225 >> v2:172.21.15.163:3300/0 conn(0x7fa8480c36e0 0x7fa8481558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 --2- 172.21.15.17:0/985789225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa8480a7cd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 -- 172.21.15.17:0/985789225 >> 172.21.15.17:0/985789225 conn(0x7fa8480aa630 msgr2=0x7fa8480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:54.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 -- 172.21.15.17:0/985789225 shutdown_connections 2024-03-06T07:23:54.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.970+0000 7fa84e866640 1 -- 172.21.15.17:0/985789225 wait complete. 2024-03-06T07:23:54.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.971+0000 7fa84e866640 1 Processor -- start 2024-03-06T07:23:54.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.971+0000 7fa84e866640 1 -- start start 2024-03-06T07:23:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa84814f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8480c3280 0x7fa8480ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848155e30 con 0x7fa8480c36e0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848159380 con 0x7fa8480c3280 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84e866640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84806ddd0 con 0x7fa84806d9f0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84ce03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84ce03640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56828/0 (socket says 172.21.15.17:56828) 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa84ce03640 1 -- 172.21.15.17:0/942498729 learned_addr learned my addr 172.21.15.17:0/942498729 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa847fff640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa84814f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.972+0000 7fa8477fe640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8480c3280 0x7fa8480ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 -- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 msgr2=0x7fa84814f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa84814f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 -- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8480c3280 msgr2=0x7fa8480ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8480c3280 0x7fa8480ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 -- 172.21.15.17:0/942498729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8481eafc0 con 0x7fa8480c36e0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84ce03640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa84000efc0 tx=0x7fa84000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840018020 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840009b10 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8481eb2a0 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400146c0 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.973+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8481eb860 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.974+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8481ebb90 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.974+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa840005d10 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.975+0000 7fa844ff9640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 0x7fa83406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.975+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa84008abf0 con 0x7fa8480c36e0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.975+0000 7fa847fff640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 0x7fa83406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:54.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.975+0000 7fa847fff640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 0x7fa83406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa83c036000 tx=0x7fa83c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:54.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:54.979+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa840022070 con 0x7fa8480c36e0 2024-03-06T07:23:55.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.160+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa848154370 con 0x7fa8480c36e0 2024-03-06T07:23:55.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.162+0000 7fa844ff9640 1 -- 172.21.15.17:0/942498729 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16965 (secure 0 0 0) 0x7fa848154370 con 0x7fa8480c36e0 2024-03-06T07:23:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:55.163 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":108,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:53.883840+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:55.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.163+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 msgr2=0x7fa83406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.163+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 0x7fa83406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa83c036000 tx=0x7fa83c03c040 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.163+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 msgr2=0x7fa8481576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.163+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa84000efc0 tx=0x7fa84000c490 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 shutdown_connections 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa83406b500 0x7fa83406d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8480c36e0 0x7fa8481576c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8480c3280 0x7fa8480ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 --2- 172.21.15.17:0/942498729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84806d9f0 0x7fa84814f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 >> 172.21.15.17:0/942498729 conn(0x7fa8480aa630 msgr2=0x7fa84806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 shutdown_connections 2024-03-06T07:23:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.164+0000 7fa84e866640 1 -- 172.21.15.17:0/942498729 wait complete. 2024-03-06T07:23:55.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:23:55.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 Processor -- start 2024-03-06T07:23:55.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 -- start start 2024-03-06T07:23:55.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c00ab310 0x7f40c00ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40c01538c0 0x7f40c01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c0154330 con 0x7f40c00a97c0 2024-03-06T07:23:55.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c00c46a0 con 0x7f40c01538c0 2024-03-06T07:23:55.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.280+0000 7f40c50ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0153dc0 con 0x7f40c00ab310 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40c01538c0 0x7f40c01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40bf59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f40c00a97c0 0x7f40c00c4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41154/0 (socket says 172.21.15.17:41154) 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40bf59e640 1 -- 172.21.15.17:0/1693595700 learned_addr learned my addr 172.21.15.17:0/1693595700 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40bed9d640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c00ab310 0x7f40c00ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40b7fff640 1 -- 172.21.15.17:0/1693595700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c00ab310 msgr2=0x7f40c00ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40b7fff640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c00ab310 0x7f40c00ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40b7fff640 1 -- 172.21.15.17:0/1693595700 >> v1:172.21.15.17:6789/0 conn(0x7f40c00a97c0 legacy=0x7f40c00c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40be59c640 1 -- 172.21.15.17:0/1693595700 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226084343 0 0) 0x7f40c0154330 con 0x7f40c00a97c0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.281+0000 7f40b7fff640 1 -- 172.21.15.17:0/1693595700 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00c5d30 con 0x7f40c01538c0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40b7fff640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c01538c0 0x7f40c01494a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c4820 tx=0x7f40a802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6b078645cc6d65 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40be59c640 1 -- 172.21.15.17:0/1693595700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8031770 con 0x7f40c01538c0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40be59c640 1 -- 172.21.15.17:0/1693595700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8037070 con 0x7f40c01538c0 2024-03-06T07:23:55.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40be59c640 1 -- 172.21.15.17:0/1693595700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8031b30 con 0x7f40c01538c0 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 -- 172.21.15.17:0/1693595700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c01538c0 msgr2=0x7f40c01494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c01538c0 0x7f40c01494a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c4820 tx=0x7f40a802b810 comp rx=0 tx=0).stop 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 -- 172.21.15.17:0/1693595700 shutdown_connections 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c01538c0 0x7f40c01494a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 --2- 172.21.15.17:0/1693595700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c00ab310 0x7f40c00ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.282+0000 7f40c50ed640 1 -- 172.21.15.17:0/1693595700 >> 172.21.15.17:0/1693595700 conn(0x7f40c00c6710 msgr2=0x7f40c00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.283+0000 7f40c50ed640 1 -- 172.21.15.17:0/1693595700 shutdown_connections 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.283+0000 7f40c50ed640 1 -- 172.21.15.17:0/1693595700 wait complete. 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 Processor -- start 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 -- start start 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 0x7f40c014feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 0x7f40c01dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00abc30 con 0x7f40c00ab310 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c62f0 con 0x7f40c00a97c0 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.284+0000 7f40c50ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c00c5240 con 0x7f40c01538c0 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56848/0 (socket says 172.21.15.17:56848) 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 -- 172.21.15.17:0/499382304 learned_addr learned my addr 172.21.15.17:0/499382304 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40bed9d640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 0x7f40c014feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40bf59e640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 0x7f40c01dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 -- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 msgr2=0x7f40c01dd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 0x7f40c01dd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 -- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 msgr2=0x7f40c014feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 0x7f40c014feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 -- 172.21.15.17:0/499382304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00c0d70 con 0x7f40c00ab310 2024-03-06T07:23:55.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40bed9d640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 0x7f40c014feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40bf59e640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 0x7f40c01dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b7fff640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004230 con 0x7f40c00ab310 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a80043f0 con 0x7f40c00ab310 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003850 con 0x7f40c00ab310 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c014fbd0 con 0x7f40c00ab310 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.285+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c01eb6f0 con 0x7f40c00ab310 2024-03-06T07:23:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.286+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c004f330 con 0x7f40c00ab310 2024-03-06T07:23:55.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.287+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40a8031070 con 0x7f40c00ab310 2024-03-06T07:23:55.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.287+0000 7f40b77fe640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.287+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40a80abd20 con 0x7f40c00ab310 2024-03-06T07:23:55.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.287+0000 7f40bed9d640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.288+0000 7f40bed9d640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006fd0 tx=0x7f40b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.292+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40a8041070 con 0x7f40c00ab310 2024-03-06T07:23:55.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.512+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f40c00c5a70 con 0x7f40c00ab310 2024-03-06T07:23:55.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.526+0000 7f40b77fe640 1 -- 172.21.15.17:0/499382304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) v1 ==== 118+0+0 (secure 0 0 0) 0x7f40a807abc0 con 0x7f40c00ab310 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 msgr2=0x7f40a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f40b0006fd0 tx=0x7f40b0008040 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 msgr2=0x7f40c00bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a80046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 shutdown_connections 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40c01538c0 0x7f40c01dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40c00ab310 0x7f40c00bfdb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 --2- 172.21.15.17:0/499382304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40c00a97c0 0x7f40c014feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 >> 172.21.15.17:0/499382304 conn(0x7f40c00c6710 msgr2=0x7f40c01df5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 shutdown_connections 2024-03-06T07:23:55.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.529+0000 7f40c50ed640 1 -- 172.21.15.17:0/499382304 wait complete. 2024-03-06T07:23:55.542 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:23:55.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:23:55.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.646+0000 7f07b55eb640 1 Processor -- start 2024-03-06T07:23:55.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.646+0000 7f07b55eb640 1 -- start start 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07b55eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07b006bff0 0x7f07b00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07b55eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07b00c7210 0x7f07b00c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07b55eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b00b04b0 con 0x7f07b00b00d0 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07b55eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b0156160 con 0x7f07b006bff0 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07b55eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00c5710 con 0x7f07b00c7210 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07ae7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07b006bff0 0x7f07b00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07aeffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07b00b00d0 0x7f07b006b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41160/0 (socket says 172.21.15.17:41160) 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07aeffd640 1 -- 172.21.15.17:0/3987405106 learned_addr learned my addr 172.21.15.17:0/3987405106 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07ae7fc640 1 -- 172.21.15.17:0/3987405106 >> v2:172.21.15.163:3300/0 conn(0x7f07b00c7210 msgr2=0x7f07b00c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:55.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07ae7fc640 1 --2- 172.21.15.17:0/3987405106 >> v2:172.21.15.163:3300/0 conn(0x7f07b00c7210 0x7f07b00c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07ae7fc640 1 -- 172.21.15.17:0/3987405106 >> v1:172.21.15.17:6789/0 conn(0x7f07b00b00d0 legacy=0x7f07b006b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07adffb640 1 -- 172.21.15.17:0/3987405106 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169551644 0 0) 0x7f07b00b04b0 con 0x7f07b00b00d0 2024-03-06T07:23:55.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.647+0000 7f07ae7fc640 1 -- 172.21.15.17:0/3987405106 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b006d210 con 0x7f07b006bff0 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.648+0000 7f07ae7fc640 1 --2- 172.21.15.17:0/3987405106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00c8ff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f07b00be2e0 tx=0x7f07a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b819bf1a66f8e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.648+0000 7f07adffb640 1 -- 172.21.15.17:0/3987405106 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4035020 con 0x7f07b006bff0 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.648+0000 7f07adffb640 1 -- 172.21.15.17:0/3987405106 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4031830 con 0x7f07b006bff0 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.648+0000 7f07adffb640 1 -- 172.21.15.17:0/3987405106 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a4031c00 con 0x7f07b006bff0 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 -- 172.21.15.17:0/3987405106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 msgr2=0x7f07b00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 --2- 172.21.15.17:0/3987405106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00c8ff0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f07b00be2e0 tx=0x7f07a402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 -- 172.21.15.17:0/3987405106 shutdown_connections 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 --2- 172.21.15.17:0/3987405106 >> v2:172.21.15.163:3300/0 conn(0x7f07b00c7210 0x7f07b00c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 --2- 172.21.15.17:0/3987405106 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00c8ff0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 -- 172.21.15.17:0/3987405106 >> 172.21.15.17:0/3987405106 conn(0x7f07b00ac810 msgr2=0x7f07b00acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 -- 172.21.15.17:0/3987405106 shutdown_connections 2024-03-06T07:23:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.649+0000 7f07b55eb640 1 -- 172.21.15.17:0/3987405106 wait complete. 2024-03-06T07:23:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.651+0000 7f07b55eb640 1 Processor -- start 2024-03-06T07:23:55.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.651+0000 7f07b55eb640 1 -- start start 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07b00b00d0 0x7f07b014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07b00c7210 0x7f07b014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00c5ba0 con 0x7f07b006bff0 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0152c50 con 0x7f07b00c7210 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07b55eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00c5d20 con 0x7f07b00b00d0 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56862/0 (socket says 172.21.15.17:56862) 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 -- 172.21.15.17:0/1185439678 learned_addr learned my addr 172.21.15.17:0/1185439678 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07af7fe640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07b00c7210 0x7f07b014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07ae7fc640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07b00b00d0 0x7f07b014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 -- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07b00b00d0 msgr2=0x7f07b014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07b00b00d0 0x7f07b014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 -- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07b00c7210 msgr2=0x7f07b014e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07b00c7210 0x7f07b014e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 -- 172.21.15.17:0/1185439678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b01587e0 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07aeffd640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c530 tx=0x7f07a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.652+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0018070 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.653+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0007500 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.653+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b01ef000 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.653+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0004a20 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.653+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b01581e0 con 0x7f07b006bff0 2024-03-06T07:23:55.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.653+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b00c1b30 con 0x7f07b006bff0 2024-03-06T07:23:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.654+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07a001b070 con 0x7f07b006bff0 2024-03-06T07:23:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.654+0000 7f07977fe640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 0x7f079c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.655+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07a0089650 con 0x7f07b006bff0 2024-03-06T07:23:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.655+0000 7f07ae7fc640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 0x7f079c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:55.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.655+0000 7f07ae7fc640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 0x7f079c06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07a4036000 tx=0x7f07a4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:55.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.659+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07a00583d0 con 0x7f07b006bff0 2024-03-06T07:23:55.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:55.880+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f07b004f330 con 0x7f07b006bff0 2024-03-06T07:23:56.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.527+0000 7f07977fe640 1 -- 172.21.15.17:0/1185439678 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07a005e0d0 con 0x7f07b006bff0 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 msgr2=0x7f079c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 0x7f079c06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07a4036000 tx=0x7f07a4008030 comp rx=0 tx=0).stop 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 msgr2=0x7f07b00aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c530 tx=0x7f07a000c9f0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 shutdown_connections 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f079c06b4d0 0x7f079c06d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07b00c7210 0x7f07b014e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07b00b00d0 0x7f07b014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 --2- 172.21.15.17:0/1185439678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07b006bff0 0x7f07b00aee40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 >> 172.21.15.17:0/1185439678 conn(0x7f07b00ac810 msgr2=0x7f07b006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:56.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 shutdown_connections 2024-03-06T07:23:56.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.529+0000 7f07b55eb640 1 -- 172.21.15.17:0/1185439678 wait complete. 2024-03-06T07:23:56.542 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:23:56.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:56.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.642+0000 7f58e24c9640 1 Processor -- start 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 -- start start 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc152920 0x7f58dc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58dc0c2190 0x7f58dc0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58dc0c1280 0x7f58dc0c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc156160 con 0x7f58dc0c2190 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc06afc0 con 0x7f58dc0c1280 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58e24c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc06b2a0 con 0x7f58dc152920 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58db7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58dc0c2190 0x7f58dc0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58db7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58dc0c2190 0x7f58dc0c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56864/0 (socket says 172.21.15.17:56864) 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.643+0000 7f58db7fe640 1 -- 172.21.15.17:0/980544361 learned_addr learned my addr 172.21.15.17:0/980544361 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58dbfff640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc152920 0x7f58dc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58e0a66640 1 --2- 172.21.15.17:0/980544361 >> v2:172.21.15.163:3300/0 conn(0x7f58dc0c1280 0x7f58dc0c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 -- 172.21.15.17:0/980544361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc152920 msgr2=0x7f58dc155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:56.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc152920 0x7f58dc155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 -- 172.21.15.17:0/980544361 >> v2:172.21.15.163:3300/0 conn(0x7f58dc0c1280 msgr2=0x7f58dc0c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:56.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 --2- 172.21.15.17:0/980544361 >> v2:172.21.15.163:3300/0 conn(0x7f58dc0c1280 0x7f58dc0c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 -- 172.21.15.17:0/980544361 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc06b580 con 0x7f58dc0c2190 2024-03-06T07:23:56.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.644+0000 7f58db7fe640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0c25f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0c6320 tx=0x7f58d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c12d950e2b171f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:56.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58daffd640 1 -- 172.21.15.17:0/980544361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0035020 con 0x7f58dc0c2190 2024-03-06T07:23:56.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58daffd640 1 -- 172.21.15.17:0/980544361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0031870 con 0x7f58dc0c2190 2024-03-06T07:23:56.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58daffd640 1 -- 172.21.15.17:0/980544361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0031c40 con 0x7f58dc0c2190 2024-03-06T07:23:56.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 -- 172.21.15.17:0/980544361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 msgr2=0x7f58dc0c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0c25f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0c6320 tx=0x7f58d002b910 comp rx=0 tx=0).stop 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 -- 172.21.15.17:0/980544361 shutdown_connections 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 --2- 172.21.15.17:0/980544361 >> v2:172.21.15.163:3300/0 conn(0x7f58dc0c1280 0x7f58dc0c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0c25f0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 --2- 172.21.15.17:0/980544361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc152920 0x7f58dc155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 -- 172.21.15.17:0/980544361 >> 172.21.15.17:0/980544361 conn(0x7f58dc0c4290 msgr2=0x7f58dc0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 -- 172.21.15.17:0/980544361 shutdown_connections 2024-03-06T07:23:56.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.645+0000 7f58e24c9640 1 -- 172.21.15.17:0/980544361 wait complete. 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.646+0000 7f58e24c9640 1 Processor -- start 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 -- start start 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 0x7f58dc1eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc152920 0x7f58dc1ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c1c50 con 0x7f58dc0c2190 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c0d60 con 0x7f58dc152920 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58e24c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c1dd0 con 0x7f58dc0c1280 2024-03-06T07:23:56.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56880/0 (socket says 172.21.15.17:56880) 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.647+0000 7f58db7fe640 1 -- 172.21.15.17:0/2285615503 learned_addr learned my addr 172.21.15.17:0/2285615503 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58e0a66640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc152920 0x7f58dc1ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58dbfff640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 0x7f58dc1eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 -- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 msgr2=0x7f58dc1eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 0x7f58dc1eb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 -- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc152920 msgr2=0x7f58dc1ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc152920 0x7f58dc1ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 -- 172.21.15.17:0/2285615503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc1eaf30 con 0x7f58dc0c2190 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58dbfff640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 0x7f58dc1eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58db7fe640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f58d002b950 tx=0x7f58d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d003ad60 con 0x7f58dc0c2190 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d00443f0 con 0x7f58dc0c2190 2024-03-06T07:23:56.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc1eef50 con 0x7f58dc0c2190 2024-03-06T07:23:56.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00447c0 con 0x7f58dc0c2190 2024-03-06T07:23:56.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.648+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc1eb1f0 con 0x7f58dc0c2190 2024-03-06T07:23:56.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.649+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc04f330 con 0x7f58dc0c2190 2024-03-06T07:23:56.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.650+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58d0038020 con 0x7f58dc0c2190 2024-03-06T07:23:56.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.650+0000 7f58d8ff9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.650+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58d00ac3b0 con 0x7f58dc0c2190 2024-03-06T07:23:56.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.650+0000 7f58dbfff640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.651+0000 7f58dbfff640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 0x7f58c806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009aa0 tx=0x7f58cc003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:56.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.654+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58d003f070 con 0x7f58dc0c2190 2024-03-06T07:23:56.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.872+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58dc068460 con 0x7f58dc0c2190 2024-03-06T07:23:56.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.872+0000 7f58d8ff9640 1 -- 172.21.15.17:0/2285615503 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4976 (secure 0 0 0) 0x7f58d007ac00 con 0x7f58dc0c2190 2024-03-06T07:23:56.874 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:56.874 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:23:55.881609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75092648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":75092648},{"type":"v1","addr":"172.21.15.17:6835","nonce":75092648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389]},"id":8}]} 2024-03-06T07:23:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 89 2024-03-06T07:23:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 msgr2=0x7f58c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 0x7f58c806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009aa0 tx=0x7f58cc003990 comp rx=0 tx=0).stop 2024-03-06T07:23:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 msgr2=0x7f58dc0726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f58d002b950 tx=0x7f58d00046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 shutdown_connections 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58dc152920 0x7f58dc1ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58dc0c2190 0x7f58dc0726b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 --2- 172.21.15.17:0/2285615503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58dc0c1280 0x7f58dc1eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 >> 172.21.15.17:0/2285615503 conn(0x7f58dc0c4290 msgr2=0x7f58dc0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 shutdown_connections 2024-03-06T07:23:56.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.874+0000 7f58e24c9640 1 -- 172.21.15.17:0/2285615503 wait complete. 2024-03-06T07:23:56.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.987+0000 7fe075755640 1 Processor -- start 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.987+0000 7fe075755640 1 -- start start 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe075755640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe070158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe075755640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe070072630 con 0x7fe0701592d0 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe075755640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe070072910 con 0x7fe07006da40 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe075755640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07006de20 con 0x7fe0700c6780 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe0701592d0 0x7fe0700c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41168/0 (socket says 172.21.15.17:41168) 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06f7fe640 1 -- 172.21.15.17:0/3460251777 learned_addr learned my addr 172.21.15.17:0/3460251777 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267377729 0 0) 0x7fe070072630 con 0x7fe0701592d0 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe058003520 con 0x7fe0701592d0 2024-03-06T07:23:56.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe070158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641690660 0 0) 0x7fe058003520 con 0x7fe0701592d0 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 msgr2=0x7fe070158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe070158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 >> v1:172.21.15.163:6789/0 conn(0x7fe07006da40 legacy=0x7fe070155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070072bf0 con 0x7fe0701592d0 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe0680019f0 con 0x7fe0701592d0 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe070158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 >> v1:172.21.15.17:6789/0 conn(0x7fe0701592d0 legacy=0x7fe0700c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0580039e0 0x7fe058003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe05800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07006de20 con 0x7fe0580039e0 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058005510 con 0x7fe058004800 2024-03-06T07:23:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05800bf00 con 0x7fe0700c6780 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.988+0000 7fe06effd640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0580039e0 0x7fe058003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06f7fe640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe05800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 -- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 msgr2=0x7fe05800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c6780 0x7fe05800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 -- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe058004800 msgr2=0x7fe058004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 -- 172.21.15.17:0/3460251777 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe058003520 con 0x7fe0580039e0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.989+0000 7fe06effd640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0580039e0 0x7fe058003e00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe060001750 tx=0x7fe058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00eb2138962c9a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060001370 con 0x7fe0580039e0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0600052d0 con 0x7fe0580039e0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe06dffb640 1 -- 172.21.15.17:0/3460251777 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060003830 con 0x7fe0580039e0 2024-03-06T07:23:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 -- 172.21.15.17:0/3460251777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0580039e0 msgr2=0x7fe058003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 --2- 172.21.15.17:0/3460251777 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0580039e0 0x7fe058003e00 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe060001750 tx=0x7fe058005830 comp rx=0 tx=0).stop 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe06f7fe640 1 -- 172.21.15.17:0/3460251777 reap_dead start 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 -- 172.21.15.17:0/3460251777 shutdown_connections 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 -- 172.21.15.17:0/3460251777 >> 172.21.15.17:0/3460251777 conn(0x7fe0700aa630 msgr2=0x7fe0700aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 -- 172.21.15.17:0/3460251777 shutdown_connections 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.990+0000 7fe075755640 1 -- 172.21.15.17:0/3460251777 wait complete. 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.991+0000 7fe075755640 1 Processor -- start 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.991+0000 7fe075755640 1 -- start start 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 0x7fe0700a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701574e0 0x7fe07006f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700c3b10 con 0x7fe07014e490 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700c3990 con 0x7fe0701574e0 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe075755640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700c7380 con 0x7fe0580039e0 2024-03-06T07:23:57.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe06e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe06e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56896/0 (socket says 172.21.15.17:56896) 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe06e7fc640 1 -- 172.21.15.17:0/1406960538 learned_addr learned my addr 172.21.15.17:0/1406960538 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe06effd640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 0x7fe0700a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.992+0000 7fe06f7fe640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701574e0 0x7fe07006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 -- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 msgr2=0x7fe0700a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 0x7fe0700a9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 -- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701574e0 msgr2=0x7fe07006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701574e0 0x7fe07006f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 -- 172.21.15.17:0/1406960538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070156410 con 0x7fe07014e490 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06effd640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 0x7fe0700a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe06e7fc640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe064010890 tx=0x7fe064010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06401a070 con 0x7fe07014e490 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06400cb40 con 0x7fe07014e490 2024-03-06T07:23:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701ee030 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400f040 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.993+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07014d100 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.994+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07004f330 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.995+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe06401d070 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.995+0000 7fe0577fe640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.995+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe06408a760 con 0x7fe07014e490 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.995+0000 7fe06effd640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.996+0000 7fe06effd640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe060033000 tx=0x7fe060009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:56.999+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe06400ee60 con 0x7fe07014e490 2024-03-06T07:23:57.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.180+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe070073240 con 0x7fe07014e490 2024-03-06T07:23:57.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.182+0000 7fe0577fe640 1 -- 172.21.15.17:0/1406960538 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16952 (secure 0 0 0) 0x7fe064059450 con 0x7fe07014e490 2024-03-06T07:23:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:57.184 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":109,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:23:55.523270+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 msgr2=0x7fe05c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe060033000 tx=0x7fe060009030 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 msgr2=0x7fe070156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe064010890 tx=0x7fe064010d50 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 shutdown_connections 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701574e0 0x7fe07006f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe07014e490 0x7fe070156fa0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 --2- 172.21.15.17:0/1406960538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0580039e0 0x7fe0700a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 >> 172.21.15.17:0/1406960538 conn(0x7fe0700aa630 msgr2=0x7fe070153550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 shutdown_connections 2024-03-06T07:23:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.184+0000 7fe075755640 1 -- 172.21.15.17:0/1406960538 wait complete. 2024-03-06T07:23:57.197 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:23:57.197 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:57.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.297+0000 7fc784c46640 1 Processor -- start 2024-03-06T07:23:57.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.297+0000 7fc784c46640 1 -- start start 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc784c46640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc78014f650 0x7fc780071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc784c46640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc7800c3150 0x7fc7800c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc784c46640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc780072440 con 0x7fc780152c20 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc784c46640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800c2b10 con 0x7fc78014f650 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc784c46640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc780152140 con 0x7fc7800c3150 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77ed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc780152c20 0x7fc7801554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41170/0 (socket says 172.21.15.17:41170) 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc78014f650 0x7fc780071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77ed9d640 1 -- 172.21.15.17:0/1419170088 learned_addr learned my addr 172.21.15.17:0/1419170088 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77dd9b640 1 --2- 172.21.15.17:0/1419170088 >> v2:172.21.15.163:3300/0 conn(0x7fc7800c3150 0x7fc7800c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77e59c640 1 -- 172.21.15.17:0/1419170088 >> v2:172.21.15.163:3300/0 conn(0x7fc7800c3150 msgr2=0x7fc7800c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77e59c640 1 --2- 172.21.15.17:0/1419170088 >> v2:172.21.15.163:3300/0 conn(0x7fc7800c3150 0x7fc7800c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77e59c640 1 -- 172.21.15.17:0/1419170088 >> v1:172.21.15.17:6789/0 conn(0x7fc780152c20 legacy=0x7fc7801554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.298+0000 7fc77e59c640 1 -- 172.21.15.17:0/1419170088 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780152420 con 0x7fc78014f650 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.299+0000 7fc77e59c640 1 --2- 172.21.15.17:0/1419170088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc78014f650 0x7fc780071880 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc7800c5350 tx=0x7fc77802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc4bf60bcd11900 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.299+0000 7fc77d59a640 1 -- 172.21.15.17:0/1419170088 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7780315a0 con 0x7fc78014f650 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.299+0000 7fc77d59a640 1 -- 172.21.15.17:0/1419170088 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778037070 con 0x7fc78014f650 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.299+0000 7fc77d59a640 1 -- 172.21.15.17:0/1419170088 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778031960 con 0x7fc78014f650 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 -- 172.21.15.17:0/1419170088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc78014f650 msgr2=0x7fc780071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 --2- 172.21.15.17:0/1419170088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc78014f650 0x7fc780071880 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc7800c5350 tx=0x7fc77802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:57.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 -- 172.21.15.17:0/1419170088 shutdown_connections 2024-03-06T07:23:57.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 --2- 172.21.15.17:0/1419170088 >> v2:172.21.15.163:3300/0 conn(0x7fc7800c3150 0x7fc7800c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 --2- 172.21.15.17:0/1419170088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc78014f650 0x7fc780071880 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 -- 172.21.15.17:0/1419170088 >> 172.21.15.17:0/1419170088 conn(0x7fc7800c4290 msgr2=0x7fc7800c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 -- 172.21.15.17:0/1419170088 shutdown_connections 2024-03-06T07:23:57.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.300+0000 7fc784c46640 1 -- 172.21.15.17:0/1419170088 wait complete. 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.301+0000 7fc784c46640 1 Processor -- start 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.301+0000 7fc784c46640 1 -- start start 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 0x7fc780155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014f650 0x7fc7801487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c40c0 con 0x7fc780152c20 2024-03-06T07:23:57.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c3f40 con 0x7fc7800c3150 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc784c46640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc780071dc0 con 0x7fc78014f650 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56916/0 (socket says 172.21.15.17:56916) 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 -- 172.21.15.17:0/700031776 learned_addr learned my addr 172.21.15.17:0/700031776 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77e59c640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 0x7fc780155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77dd9b640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014f650 0x7fc7801487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 -- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014f650 msgr2=0x7fc7801487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014f650 0x7fc7801487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 -- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 msgr2=0x7fc780155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 0x7fc780155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 -- 172.21.15.17:0/700031776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7801492e0 con 0x7fc780152c20 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77e59c640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 0x7fc780155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.302+0000 7fc77ed9d640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cad0 tx=0x7fc77000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.303+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770017070 con 0x7fc780152c20 2024-03-06T07:23:57.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.303+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77000ac30 con 0x7fc780152c20 2024-03-06T07:23:57.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.303+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780151460 con 0x7fc780152c20 2024-03-06T07:23:57.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.303+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770012400 con 0x7fc780152c20 2024-03-06T07:23:57.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.303+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780151a20 con 0x7fc780152c20 2024-03-06T07:23:57.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.304+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78004f330 con 0x7fc780152c20 2024-03-06T07:23:57.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.304+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc77001a070 con 0x7fc780152c20 2024-03-06T07:23:57.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.305+0000 7fc76effd640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 0x7fc76006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.305+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc770089ef0 con 0x7fc780152c20 2024-03-06T07:23:57.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.305+0000 7fc77e59c640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 0x7fc76006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.305+0000 7fc77e59c640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 0x7fc76006d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc778036000 tx=0x7fc77803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.309+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc780072440 con 0x7fc780152c20 2024-03-06T07:23:57.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.497+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc780068460 con 0x7fc76006b4d0 2024-03-06T07:23:57.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.503+0000 7fc76effd640 1 -- 172.21.15.17:0/700031776 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7fc780068460 con 0x7fc76006b4d0 2024-03-06T07:23:57.507 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 msgr2=0x7fc76006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 0x7fc76006d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc778036000 tx=0x7fc77803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 msgr2=0x7fc7801ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cad0 tx=0x7fc77000cf90 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 shutdown_connections 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc76006b4d0 0x7fc76006d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc780152c20 0x7fc7801ec6c0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc78014f650 0x7fc7801487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 --2- 172.21.15.17:0/700031776 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7800c3150 0x7fc780155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 >> 172.21.15.17:0/700031776 conn(0x7fc7800c4290 msgr2=0x7fc7800c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 shutdown_connections 2024-03-06T07:23:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.506+0000 7fc784c46640 1 -- 172.21.15.17:0/700031776 wait complete. 2024-03-06T07:23:57.518 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-03-06T07:23:57.392713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256148,"kb_used_data":10816,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718572,"statfs":{"total":773094113280,"available":772831817728,"internally_reserved":0,"allocated":11075584,"data_stored":6454764,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.889623"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533428+0000","last_change":"2024-03-06T07:23:54.521645+0000","last_active":"2024-03-06T07:23:55.533428+0000","last_peered":"2024-03-06T07:23:55.533428+0000","last_clean":"2024-03-06T07:23:55.533428+0000","last_became_active":"2024-03-06T07:23:54.521252+0000","last_became_peered":"2024-03-06T07:23:54.521252+0000","last_unstale":"2024-03-06T07:23:55.533428+0000","last_undegraded":"2024-03-06T07:23:55.533428+0000","last_fullsized":"2024-03-06T07:23:55.533428+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602645+0000","last_change":"2024-03-06T07:23:54.537527+0000","last_active":"2024-03-06T07:23:55.602645+0000","last_peered":"2024-03-06T07:23:55.602645+0000","last_clean":"2024-03-06T07:23:55.602645+0000","last_became_active":"2024-03-06T07:23:54.537036+0000","last_became_peered":"2024-03-06T07:23:54.537036+0000","last_unstale":"2024-03-06T07:23:55.602645+0000","last_undegraded":"2024-03-06T07:23:55.602645+0000","last_fullsized":"2024-03-06T07:23:55.602645+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:17.701870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602761+0000","last_change":"2024-03-06T07:23:54.530569+0000","last_active":"2024-03-06T07:23:55.602761+0000","last_peered":"2024-03-06T07:23:55.602761+0000","last_clean":"2024-03-06T07:23:55.602761+0000","last_became_active":"2024-03-06T07:23:54.530272+0000","last_became_peered":"2024-03-06T07:23:54.530272+0000","last_unstale":"2024-03-06T07:23:55.602761+0000","last_undegraded":"2024-03-06T07:23:55.602761+0000","last_fullsized":"2024-03-06T07:23:55.602761+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:24.350111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603775+0000","last_change":"2024-03-06T07:23:54.529843+0000","last_active":"2024-03-06T07:23:55.603775+0000","last_peered":"2024-03-06T07:23:55.603775+0000","last_clean":"2024-03-06T07:23:55.603775+0000","last_became_active":"2024-03-06T07:23:54.529140+0000","last_became_peered":"2024-03-06T07:23:54.529140+0000","last_unstale":"2024-03-06T07:23:55.603775+0000","last_undegraded":"2024-03-06T07:23:55.603775+0000","last_fullsized":"2024-03-06T07:23:55.603775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:12.461791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533206+0000","last_change":"2024-03-06T07:23:54.520680+0000","last_active":"2024-03-06T07:23:55.533206+0000","last_peered":"2024-03-06T07:23:55.533206+0000","last_clean":"2024-03-06T07:23:55.533206+0000","last_became_active":"2024-03-06T07:23:54.520397+0000","last_became_peered":"2024-03-06T07:23:54.520397+0000","last_unstale":"2024-03-06T07:23:55.533206+0000","last_undegraded":"2024-03-06T07:23:55.533206+0000","last_fullsized":"2024-03-06T07:23:55.533206+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604095+0000","last_change":"2024-03-06T07:23:54.525453+0000","last_active":"2024-03-06T07:23:55.604095+0000","last_peered":"2024-03-06T07:23:55.604095+0000","last_clean":"2024-03-06T07:23:55.604095+0000","last_became_active":"2024-03-06T07:23:54.525176+0000","last_became_peered":"2024-03-06T07:23:54.525176+0000","last_unstale":"2024-03-06T07:23:55.604095+0000","last_undegraded":"2024-03-06T07:23:55.604095+0000","last_fullsized":"2024-03-06T07:23:55.604095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:50.888132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603900+0000","last_change":"2024-03-06T07:23:54.529179+0000","last_active":"2024-03-06T07:23:55.603900+0000","last_peered":"2024-03-06T07:23:55.603900+0000","last_clean":"2024-03-06T07:23:55.603900+0000","last_became_active":"2024-03-06T07:23:54.528887+0000","last_became_peered":"2024-03-06T07:23:54.528887+0000","last_unstale":"2024-03-06T07:23:55.603900+0000","last_undegraded":"2024-03-06T07:23:55.603900+0000","last_fullsized":"2024-03-06T07:23:55.603900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:54.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602383+0000","last_change":"2024-03-06T07:23:54.530509+0000","last_active":"2024-03-06T07:23:55.602383+0000","last_peered":"2024-03-06T07:23:55.602383+0000","last_clean":"2024-03-06T07:23:55.602383+0000","last_became_active":"2024-03-06T07:23:54.530225+0000","last_became_peered":"2024-03-06T07:23:54.530225+0000","last_unstale":"2024-03-06T07:23:55.602383+0000","last_undegraded":"2024-03-06T07:23:55.602383+0000","last_fullsized":"2024-03-06T07:23:55.602383+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:09.334362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603712+0000","last_change":"2024-03-06T07:23:54.541162+0000","last_active":"2024-03-06T07:23:55.603712+0000","last_peered":"2024-03-06T07:23:55.603712+0000","last_clean":"2024-03-06T07:23:55.603712+0000","last_became_active":"2024-03-06T07:23:54.540890+0000","last_became_peered":"2024-03-06T07:23:54.540890+0000","last_unstale":"2024-03-06T07:23:55.603712+0000","last_undegraded":"2024-03-06T07:23:55.603712+0000","last_fullsized":"2024-03-06T07:23:55.603712+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:37.822960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602762+0000","last_change":"2024-03-06T07:23:54.529160+0000","last_active":"2024-03-06T07:23:55.602762+0000","last_peered":"2024-03-06T07:23:55.602762+0000","last_clean":"2024-03-06T07:23:55.602762+0000","last_became_active":"2024-03-06T07:23:54.528885+0000","last_became_peered":"2024-03-06T07:23:54.528885+0000","last_unstale":"2024-03-06T07:23:55.602762+0000","last_undegraded":"2024-03-06T07:23:55.602762+0000","last_fullsized":"2024-03-06T07:23:55.602762+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533420+0000","last_change":"2024-03-06T07:23:54.532560+0000","last_active":"2024-03-06T07:23:55.533420+0000","last_peered":"2024-03-06T07:23:55.533420+0000","last_clean":"2024-03-06T07:23:55.533420+0000","last_became_active":"2024-03-06T07:23:54.532263+0000","last_became_peered":"2024-03-06T07:23:54.532263+0000","last_unstale":"2024-03-06T07:23:55.533420+0000","last_undegraded":"2024-03-06T07:23:55.533420+0000","last_fullsized":"2024-03-06T07:23:55.533420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604268+0000","last_change":"2024-03-06T07:23:54.530333+0000","last_active":"2024-03-06T07:23:55.604268+0000","last_peered":"2024-03-06T07:23:55.604268+0000","last_clean":"2024-03-06T07:23:55.604268+0000","last_became_active":"2024-03-06T07:23:54.530053+0000","last_became_peered":"2024-03-06T07:23:54.530053+0000","last_unstale":"2024-03-06T07:23:55.604268+0000","last_undegraded":"2024-03-06T07:23:55.604268+0000","last_fullsized":"2024-03-06T07:23:55.604268+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:18.429026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603034+0000","last_change":"2024-03-06T07:23:54.538305+0000","last_active":"2024-03-06T07:23:55.603034+0000","last_peered":"2024-03-06T07:23:55.603034+0000","last_clean":"2024-03-06T07:23:55.603034+0000","last_became_active":"2024-03-06T07:23:54.538040+0000","last_became_peered":"2024-03-06T07:23:54.538040+0000","last_unstale":"2024-03-06T07:23:55.603034+0000","last_undegraded":"2024-03-06T07:23:55.603034+0000","last_fullsized":"2024-03-06T07:23:55.603034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603527+0000","last_change":"2024-03-06T07:23:54.531008+0000","last_active":"2024-03-06T07:23:55.603527+0000","last_peered":"2024-03-06T07:23:55.603527+0000","last_clean":"2024-03-06T07:23:55.603527+0000","last_became_active":"2024-03-06T07:23:54.530735+0000","last_became_peered":"2024-03-06T07:23:54.530735+0000","last_unstale":"2024-03-06T07:23:55.603527+0000","last_undegraded":"2024-03-06T07:23:55.603527+0000","last_fullsized":"2024-03-06T07:23:55.603527+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:24.127859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533130+0000","last_change":"2024-03-06T07:23:54.532003+0000","last_active":"2024-03-06T07:23:55.533130+0000","last_peered":"2024-03-06T07:23:55.533130+0000","last_clean":"2024-03-06T07:23:55.533130+0000","last_became_active":"2024-03-06T07:23:54.531720+0000","last_became_peered":"2024-03-06T07:23:54.531720+0000","last_unstale":"2024-03-06T07:23:55.533130+0000","last_undegraded":"2024-03-06T07:23:55.533130+0000","last_fullsized":"2024-03-06T07:23:55.533130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603676+0000","last_change":"2024-03-06T07:23:54.528496+0000","last_active":"2024-03-06T07:23:55.603676+0000","last_peered":"2024-03-06T07:23:55.603676+0000","last_clean":"2024-03-06T07:23:55.603676+0000","last_became_active":"2024-03-06T07:23:54.528189+0000","last_became_peered":"2024-03-06T07:23:54.528189+0000","last_unstale":"2024-03-06T07:23:55.603676+0000","last_undegraded":"2024-03-06T07:23:55.603676+0000","last_fullsized":"2024-03-06T07:23:55.603676+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:35.102000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603106+0000","last_change":"2024-03-06T07:23:54.538887+0000","last_active":"2024-03-06T07:23:55.603106+0000","last_peered":"2024-03-06T07:23:55.603106+0000","last_clean":"2024-03-06T07:23:55.603106+0000","last_became_active":"2024-03-06T07:23:54.538021+0000","last_became_peered":"2024-03-06T07:23:54.538021+0000","last_unstale":"2024-03-06T07:23:55.603106+0000","last_undegraded":"2024-03-06T07:23:55.603106+0000","last_fullsized":"2024-03-06T07:23:55.603106+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602806+0000","last_change":"2024-03-06T07:23:54.537314+0000","last_active":"2024-03-06T07:23:55.602806+0000","last_peered":"2024-03-06T07:23:55.602806+0000","last_clean":"2024-03-06T07:23:55.602806+0000","last_became_active":"2024-03-06T07:23:54.536670+0000","last_became_peered":"2024-03-06T07:23:54.536670+0000","last_unstale":"2024-03-06T07:23:55.602806+0000","last_undegraded":"2024-03-06T07:23:55.602806+0000","last_fullsized":"2024-03-06T07:23:55.602806+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:17.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532466+0000","last_change":"2024-03-06T07:23:54.530130+0000","last_active":"2024-03-06T07:23:55.532466+0000","last_peered":"2024-03-06T07:23:55.532466+0000","last_clean":"2024-03-06T07:23:55.532466+0000","last_became_active":"2024-03-06T07:23:54.529876+0000","last_became_peered":"2024-03-06T07:23:54.529876+0000","last_unstale":"2024-03-06T07:23:55.532466+0000","last_undegraded":"2024-03-06T07:23:55.532466+0000","last_fullsized":"2024-03-06T07:23:55.532466+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.528915+0000","last_change":"2024-03-06T07:23:54.531498+0000","last_active":"2024-03-06T07:23:55.528915+0000","last_peered":"2024-03-06T07:23:55.528915+0000","last_clean":"2024-03-06T07:23:55.528915+0000","last_became_active":"2024-03-06T07:23:54.531221+0000","last_became_peered":"2024-03-06T07:23:54.531221+0000","last_unstale":"2024-03-06T07:23:55.528915+0000","last_undegraded":"2024-03-06T07:23:55.528915+0000","last_fullsized":"2024-03-06T07:23:55.528915+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:02:08.274669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603877+0000","last_change":"2024-03-06T07:23:54.532907+0000","last_active":"2024-03-06T07:23:55.603877+0000","last_peered":"2024-03-06T07:23:55.603877+0000","last_clean":"2024-03-06T07:23:55.603877+0000","last_became_active":"2024-03-06T07:23:54.531373+0000","last_became_peered":"2024-03-06T07:23:54.531373+0000","last_unstale":"2024-03-06T07:23:55.603877+0000","last_undegraded":"2024-03-06T07:23:55.603877+0000","last_fullsized":"2024-03-06T07:23:55.603877+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:07.175141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533382+0000","last_change":"2024-03-06T07:23:54.524285+0000","last_active":"2024-03-06T07:23:55.533382+0000","last_peered":"2024-03-06T07:23:55.533382+0000","last_clean":"2024-03-06T07:23:55.533382+0000","last_became_active":"2024-03-06T07:23:54.524013+0000","last_became_peered":"2024-03-06T07:23:54.524013+0000","last_unstale":"2024-03-06T07:23:55.533382+0000","last_undegraded":"2024-03-06T07:23:55.533382+0000","last_fullsized":"2024-03-06T07:23:55.533382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603535+0000","last_change":"2024-03-06T07:23:54.535573+0000","last_active":"2024-03-06T07:23:55.603535+0000","last_peered":"2024-03-06T07:23:55.603535+0000","last_clean":"2024-03-06T07:23:55.603535+0000","last_became_active":"2024-03-06T07:23:54.533542+0000","last_became_peered":"2024-03-06T07:23:54.533542+0000","last_unstale":"2024-03-06T07:23:55.603535+0000","last_undegraded":"2024-03-06T07:23:55.603535+0000","last_fullsized":"2024-03-06T07:23:55.603535+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:43.722095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602926+0000","last_change":"2024-03-06T07:23:54.534260+0000","last_active":"2024-03-06T07:23:55.602926+0000","last_peered":"2024-03-06T07:23:55.602926+0000","last_clean":"2024-03-06T07:23:55.602926+0000","last_became_active":"2024-03-06T07:23:54.532730+0000","last_became_peered":"2024-03-06T07:23:54.532730+0000","last_unstale":"2024-03-06T07:23:55.602926+0000","last_undegraded":"2024-03-06T07:23:55.602926+0000","last_fullsized":"2024-03-06T07:23:55.602926+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603687+0000","last_change":"2024-03-06T07:23:54.531365+0000","last_active":"2024-03-06T07:23:55.603687+0000","last_peered":"2024-03-06T07:23:55.603687+0000","last_clean":"2024-03-06T07:23:55.603687+0000","last_became_active":"2024-03-06T07:23:54.529649+0000","last_became_peered":"2024-03-06T07:23:54.529649+0000","last_unstale":"2024-03-06T07:23:55.603687+0000","last_undegraded":"2024-03-06T07:23:55.603687+0000","last_fullsized":"2024-03-06T07:23:55.603687+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:55.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529134+0000","last_change":"2024-03-06T07:23:54.521415+0000","last_active":"2024-03-06T07:23:55.529134+0000","last_peered":"2024-03-06T07:23:55.529134+0000","last_clean":"2024-03-06T07:23:55.529134+0000","last_became_active":"2024-03-06T07:23:54.521112+0000","last_became_peered":"2024-03-06T07:23:54.521112+0000","last_unstale":"2024-03-06T07:23:55.529134+0000","last_undegraded":"2024-03-06T07:23:55.529134+0000","last_fullsized":"2024-03-06T07:23:55.529134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:04.090188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532610+0000","last_change":"2024-03-06T07:23:54.529429+0000","last_active":"2024-03-06T07:23:55.532610+0000","last_peered":"2024-03-06T07:23:55.532610+0000","last_clean":"2024-03-06T07:23:55.532610+0000","last_became_active":"2024-03-06T07:23:54.529131+0000","last_became_peered":"2024-03-06T07:23:54.529131+0000","last_unstale":"2024-03-06T07:23:55.532610+0000","last_undegraded":"2024-03-06T07:23:55.532610+0000","last_fullsized":"2024-03-06T07:23:55.532610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604378+0000","last_change":"2024-03-06T07:23:54.531739+0000","last_active":"2024-03-06T07:23:55.604378+0000","last_peered":"2024-03-06T07:23:55.604378+0000","last_clean":"2024-03-06T07:23:55.604378+0000","last_became_active":"2024-03-06T07:23:54.531418+0000","last_became_peered":"2024-03-06T07:23:54.531418+0000","last_unstale":"2024-03-06T07:23:55.604378+0000","last_undegraded":"2024-03-06T07:23:55.604378+0000","last_fullsized":"2024-03-06T07:23:55.604378+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:43.405621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529203+0000","last_change":"2024-03-06T07:23:54.523562+0000","last_active":"2024-03-06T07:23:55.529203+0000","last_peered":"2024-03-06T07:23:55.529203+0000","last_clean":"2024-03-06T07:23:55.529203+0000","last_became_active":"2024-03-06T07:23:54.523301+0000","last_became_peered":"2024-03-06T07:23:54.523301+0000","last_unstale":"2024-03-06T07:23:55.529203+0000","last_undegraded":"2024-03-06T07:23:55.529203+0000","last_fullsized":"2024-03-06T07:23:55.529203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:19.129197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603679+0000","last_change":"2024-03-06T07:23:54.524738+0000","last_active":"2024-03-06T07:23:55.603679+0000","last_peered":"2024-03-06T07:23:55.603679+0000","last_clean":"2024-03-06T07:23:55.603679+0000","last_became_active":"2024-03-06T07:23:54.524108+0000","last_became_peered":"2024-03-06T07:23:54.524108+0000","last_unstale":"2024-03-06T07:23:55.603679+0000","last_undegraded":"2024-03-06T07:23:55.603679+0000","last_fullsized":"2024-03-06T07:23:55.603679+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:54.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603135+0000","last_change":"2024-03-06T07:23:54.530124+0000","last_active":"2024-03-06T07:23:55.603135+0000","last_peered":"2024-03-06T07:23:55.603135+0000","last_clean":"2024-03-06T07:23:55.603135+0000","last_became_active":"2024-03-06T07:23:54.529714+0000","last_became_peered":"2024-03-06T07:23:54.529714+0000","last_unstale":"2024-03-06T07:23:55.603135+0000","last_undegraded":"2024-03-06T07:23:55.603135+0000","last_fullsized":"2024-03-06T07:23:55.603135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602930+0000","last_change":"2024-03-06T07:23:54.531367+0000","last_active":"2024-03-06T07:23:55.602930+0000","last_peered":"2024-03-06T07:23:55.602930+0000","last_clean":"2024-03-06T07:23:55.602930+0000","last_became_active":"2024-03-06T07:23:54.531075+0000","last_became_peered":"2024-03-06T07:23:54.531075+0000","last_unstale":"2024-03-06T07:23:55.602930+0000","last_undegraded":"2024-03-06T07:23:55.602930+0000","last_fullsized":"2024-03-06T07:23:55.602930+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603633+0000","last_change":"2024-03-06T07:23:54.528967+0000","last_active":"2024-03-06T07:23:55.603633+0000","last_peered":"2024-03-06T07:23:55.603633+0000","last_clean":"2024-03-06T07:23:55.603633+0000","last_became_active":"2024-03-06T07:23:54.528683+0000","last_became_peered":"2024-03-06T07:23:54.528683+0000","last_unstale":"2024-03-06T07:23:55.603633+0000","last_undegraded":"2024-03-06T07:23:55.603633+0000","last_fullsized":"2024-03-06T07:23:55.603633+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:39.793464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532323+0000","last_change":"2024-03-06T07:23:53.522047+0000","last_active":"2024-03-06T07:23:55.532323+0000","last_peered":"2024-03-06T07:23:55.532323+0000","last_clean":"2024-03-06T07:23:55.532323+0000","last_became_active":"2024-03-06T07:23:53.521682+0000","last_became_peered":"2024-03-06T07:23:53.521682+0000","last_unstale":"2024-03-06T07:23:55.532323+0000","last_undegraded":"2024-03-06T07:23:55.532323+0000","last_fullsized":"2024-03-06T07:23:55.532323+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:43.169621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602804+0000","last_change":"2024-03-06T07:23:54.530485+0000","last_active":"2024-03-06T07:23:55.602804+0000","last_peered":"2024-03-06T07:23:55.602804+0000","last_clean":"2024-03-06T07:23:55.602804+0000","last_became_active":"2024-03-06T07:23:54.530226+0000","last_became_peered":"2024-03-06T07:23:54.530226+0000","last_unstale":"2024-03-06T07:23:55.602804+0000","last_undegraded":"2024-03-06T07:23:55.602804+0000","last_fullsized":"2024-03-06T07:23:55.602804+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:34:39.865419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602869+0000","last_change":"2024-03-06T07:23:53.524591+0000","last_active":"2024-03-06T07:23:55.602869+0000","last_peered":"2024-03-06T07:23:55.602869+0000","last_clean":"2024-03-06T07:23:55.602869+0000","last_became_active":"2024-03-06T07:23:53.524313+0000","last_became_peered":"2024-03-06T07:23:53.524313+0000","last_unstale":"2024-03-06T07:23:55.602869+0000","last_undegraded":"2024-03-06T07:23:55.602869+0000","last_fullsized":"2024-03-06T07:23:55.602869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:06.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603411+0000","last_change":"2024-03-06T07:23:54.538108+0000","last_active":"2024-03-06T07:23:55.603411+0000","last_peered":"2024-03-06T07:23:55.603411+0000","last_clean":"2024-03-06T07:23:55.603411+0000","last_became_active":"2024-03-06T07:23:54.537771+0000","last_became_peered":"2024-03-06T07:23:54.537771+0000","last_unstale":"2024-03-06T07:23:55.603411+0000","last_undegraded":"2024-03-06T07:23:55.603411+0000","last_fullsized":"2024-03-06T07:23:55.603411+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:39.793464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604187+0000","last_change":"2024-03-06T07:23:53.524512+0000","last_active":"2024-03-06T07:23:55.604187+0000","last_peered":"2024-03-06T07:23:55.604187+0000","last_clean":"2024-03-06T07:23:55.604187+0000","last_became_active":"2024-03-06T07:23:53.524230+0000","last_became_peered":"2024-03-06T07:23:53.524230+0000","last_unstale":"2024-03-06T07:23:55.604187+0000","last_undegraded":"2024-03-06T07:23:55.604187+0000","last_fullsized":"2024-03-06T07:23:55.604187+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:29.292121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603866+0000","last_change":"2024-03-06T07:23:54.542211+0000","last_active":"2024-03-06T07:23:55.603866+0000","last_peered":"2024-03-06T07:23:55.603866+0000","last_clean":"2024-03-06T07:23:55.603866+0000","last_became_active":"2024-03-06T07:23:54.541948+0000","last_became_peered":"2024-03-06T07:23:54.541948+0000","last_unstale":"2024-03-06T07:23:55.603866+0000","last_undegraded":"2024-03-06T07:23:55.603866+0000","last_fullsized":"2024-03-06T07:23:55.603866+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:22.340862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603789+0000","last_change":"2024-03-06T07:23:53.529569+0000","last_active":"2024-03-06T07:23:55.603789+0000","last_peered":"2024-03-06T07:23:55.603789+0000","last_clean":"2024-03-06T07:23:55.603789+0000","last_became_active":"2024-03-06T07:23:53.528175+0000","last_became_peered":"2024-03-06T07:23:53.528175+0000","last_unstale":"2024-03-06T07:23:55.603789+0000","last_undegraded":"2024-03-06T07:23:55.603789+0000","last_fullsized":"2024-03-06T07:23:55.603789+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:46.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602477+0000","last_change":"2024-03-06T07:23:54.537036+0000","last_active":"2024-03-06T07:23:55.602477+0000","last_peered":"2024-03-06T07:23:55.602477+0000","last_clean":"2024-03-06T07:23:55.602477+0000","last_became_active":"2024-03-06T07:23:54.531441+0000","last_became_peered":"2024-03-06T07:23:54.531441+0000","last_unstale":"2024-03-06T07:23:55.602477+0000","last_undegraded":"2024-03-06T07:23:55.602477+0000","last_fullsized":"2024-03-06T07:23:55.602477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533416+0000","last_change":"2024-03-06T07:23:53.524983+0000","last_active":"2024-03-06T07:23:55.533416+0000","last_peered":"2024-03-06T07:23:55.533416+0000","last_clean":"2024-03-06T07:23:55.533416+0000","last_became_active":"2024-03-06T07:23:53.524421+0000","last_became_peered":"2024-03-06T07:23:53.524421+0000","last_unstale":"2024-03-06T07:23:55.533416+0000","last_undegraded":"2024-03-06T07:23:55.533416+0000","last_fullsized":"2024-03-06T07:23:55.533416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:01.816188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533146+0000","last_change":"2024-03-06T07:23:54.525774+0000","last_active":"2024-03-06T07:23:55.533146+0000","last_peered":"2024-03-06T07:23:55.533146+0000","last_clean":"2024-03-06T07:23:55.533146+0000","last_became_active":"2024-03-06T07:23:54.525487+0000","last_became_peered":"2024-03-06T07:23:54.525487+0000","last_unstale":"2024-03-06T07:23:55.533146+0000","last_undegraded":"2024-03-06T07:23:55.533146+0000","last_fullsized":"2024-03-06T07:23:55.533146+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603980+0000","last_change":"2024-03-06T07:23:53.527677+0000","last_active":"2024-03-06T07:23:55.603980+0000","last_peered":"2024-03-06T07:23:55.603980+0000","last_clean":"2024-03-06T07:23:55.603980+0000","last_became_active":"2024-03-06T07:23:53.527399+0000","last_became_peered":"2024-03-06T07:23:53.527399+0000","last_unstale":"2024-03-06T07:23:55.603980+0000","last_undegraded":"2024-03-06T07:23:55.603980+0000","last_fullsized":"2024-03-06T07:23:55.603980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:00:27.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603826+0000","last_change":"2024-03-06T07:23:54.530826+0000","last_active":"2024-03-06T07:23:55.603826+0000","last_peered":"2024-03-06T07:23:55.603826+0000","last_clean":"2024-03-06T07:23:55.603826+0000","last_became_active":"2024-03-06T07:23:54.530298+0000","last_became_peered":"2024-03-06T07:23:54.530298+0000","last_unstale":"2024-03-06T07:23:55.603826+0000","last_undegraded":"2024-03-06T07:23:55.603826+0000","last_fullsized":"2024-03-06T07:23:55.603826+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:27.693614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603231+0000","last_change":"2024-03-06T07:23:53.528171+0000","last_active":"2024-03-06T07:23:55.603231+0000","last_peered":"2024-03-06T07:23:55.603231+0000","last_clean":"2024-03-06T07:23:55.603231+0000","last_became_active":"2024-03-06T07:23:53.527873+0000","last_became_peered":"2024-03-06T07:23:53.527873+0000","last_unstale":"2024-03-06T07:23:55.603231+0000","last_undegraded":"2024-03-06T07:23:55.603231+0000","last_fullsized":"2024-03-06T07:23:55.603231+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:46.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603175+0000","last_change":"2024-03-06T07:23:54.531226+0000","last_active":"2024-03-06T07:23:55.603175+0000","last_peered":"2024-03-06T07:23:55.603175+0000","last_clean":"2024-03-06T07:23:55.603175+0000","last_became_active":"2024-03-06T07:23:54.530960+0000","last_became_peered":"2024-03-06T07:23:54.530960+0000","last_unstale":"2024-03-06T07:23:55.603175+0000","last_undegraded":"2024-03-06T07:23:55.603175+0000","last_fullsized":"2024-03-06T07:23:55.603175+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603981+0000","last_change":"2024-03-06T07:23:53.524011+0000","last_active":"2024-03-06T07:23:55.603981+0000","last_peered":"2024-03-06T07:23:55.603981+0000","last_clean":"2024-03-06T07:23:55.603981+0000","last_became_active":"2024-03-06T07:23:53.523742+0000","last_became_peered":"2024-03-06T07:23:53.523742+0000","last_unstale":"2024-03-06T07:23:55.603981+0000","last_undegraded":"2024-03-06T07:23:55.603981+0000","last_fullsized":"2024-03-06T07:23:55.603981+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:19.811535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532130+0000","last_change":"2024-03-06T07:23:54.525862+0000","last_active":"2024-03-06T07:23:55.532130+0000","last_peered":"2024-03-06T07:23:55.532130+0000","last_clean":"2024-03-06T07:23:55.532130+0000","last_became_active":"2024-03-06T07:23:54.525532+0000","last_became_peered":"2024-03-06T07:23:54.525532+0000","last_unstale":"2024-03-06T07:23:55.532130+0000","last_undegraded":"2024-03-06T07:23:55.532130+0000","last_fullsized":"2024-03-06T07:23:55.532130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:52.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532111+0000","last_change":"2024-03-06T07:23:53.519536+0000","last_active":"2024-03-06T07:23:55.532111+0000","last_peered":"2024-03-06T07:23:55.532111+0000","last_clean":"2024-03-06T07:23:55.532111+0000","last_became_active":"2024-03-06T07:23:53.519209+0000","last_became_peered":"2024-03-06T07:23:53.519209+0000","last_unstale":"2024-03-06T07:23:55.532111+0000","last_undegraded":"2024-03-06T07:23:55.532111+0000","last_fullsized":"2024-03-06T07:23:55.532111+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:00.179219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603338+0000","last_change":"2024-03-06T07:23:54.531752+0000","last_active":"2024-03-06T07:23:55.603338+0000","last_peered":"2024-03-06T07:23:55.603338+0000","last_clean":"2024-03-06T07:23:55.603338+0000","last_became_active":"2024-03-06T07:23:54.530846+0000","last_became_peered":"2024-03-06T07:23:54.530846+0000","last_unstale":"2024-03-06T07:23:55.603338+0000","last_undegraded":"2024-03-06T07:23:55.603338+0000","last_fullsized":"2024-03-06T07:23:55.603338+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603363+0000","last_change":"2024-03-06T07:23:53.527709+0000","last_active":"2024-03-06T07:23:55.603363+0000","last_peered":"2024-03-06T07:23:55.603363+0000","last_clean":"2024-03-06T07:23:55.603363+0000","last_became_active":"2024-03-06T07:23:53.526482+0000","last_became_peered":"2024-03-06T07:23:53.526482+0000","last_unstale":"2024-03-06T07:23:55.603363+0000","last_undegraded":"2024-03-06T07:23:55.603363+0000","last_fullsized":"2024-03-06T07:23:55.603363+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:28.397840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602946+0000","last_change":"2024-03-06T07:23:54.526827+0000","last_active":"2024-03-06T07:23:55.602946+0000","last_peered":"2024-03-06T07:23:55.602946+0000","last_clean":"2024-03-06T07:23:55.602946+0000","last_became_active":"2024-03-06T07:23:54.526530+0000","last_became_peered":"2024-03-06T07:23:54.526530+0000","last_unstale":"2024-03-06T07:23:55.602946+0000","last_undegraded":"2024-03-06T07:23:55.602946+0000","last_fullsized":"2024-03-06T07:23:55.602946+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:37.607508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603353+0000","last_change":"2024-03-06T07:23:53.527293+0000","last_active":"2024-03-06T07:23:55.603353+0000","last_peered":"2024-03-06T07:23:55.603353+0000","last_clean":"2024-03-06T07:23:55.603353+0000","last_became_active":"2024-03-06T07:23:53.527043+0000","last_became_peered":"2024-03-06T07:23:53.527043+0000","last_unstale":"2024-03-06T07:23:55.603353+0000","last_undegraded":"2024-03-06T07:23:55.603353+0000","last_fullsized":"2024-03-06T07:23:55.603353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603837+0000","last_change":"2024-03-06T07:23:54.531117+0000","last_active":"2024-03-06T07:23:55.603837+0000","last_peered":"2024-03-06T07:23:55.603837+0000","last_clean":"2024-03-06T07:23:55.603837+0000","last_became_active":"2024-03-06T07:23:54.529940+0000","last_became_peered":"2024-03-06T07:23:54.529940+0000","last_unstale":"2024-03-06T07:23:55.603837+0000","last_undegraded":"2024-03-06T07:23:55.603837+0000","last_fullsized":"2024-03-06T07:23:55.603837+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:45.970434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603143+0000","last_change":"2024-03-06T07:23:53.526550+0000","last_active":"2024-03-06T07:23:55.603143+0000","last_peered":"2024-03-06T07:23:55.603143+0000","last_clean":"2024-03-06T07:23:55.603143+0000","last_became_active":"2024-03-06T07:23:53.526251+0000","last_became_peered":"2024-03-06T07:23:53.526251+0000","last_unstale":"2024-03-06T07:23:55.603143+0000","last_undegraded":"2024-03-06T07:23:55.603143+0000","last_fullsized":"2024-03-06T07:23:55.603143+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:52.186396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532508+0000","last_change":"2024-03-06T07:23:54.524567+0000","last_active":"2024-03-06T07:23:55.532508+0000","last_peered":"2024-03-06T07:23:55.532508+0000","last_clean":"2024-03-06T07:23:55.532508+0000","last_became_active":"2024-03-06T07:23:54.523152+0000","last_became_peered":"2024-03-06T07:23:54.523152+0000","last_unstale":"2024-03-06T07:23:55.532508+0000","last_undegraded":"2024-03-06T07:23:55.532508+0000","last_fullsized":"2024-03-06T07:23:55.532508+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602862+0000","last_change":"2024-03-06T07:23:53.523939+0000","last_active":"2024-03-06T07:23:55.602862+0000","last_peered":"2024-03-06T07:23:55.602862+0000","last_clean":"2024-03-06T07:23:55.602862+0000","last_became_active":"2024-03-06T07:23:53.523644+0000","last_became_peered":"2024-03-06T07:23:53.523644+0000","last_unstale":"2024-03-06T07:23:55.602862+0000","last_undegraded":"2024-03-06T07:23:55.602862+0000","last_fullsized":"2024-03-06T07:23:55.602862+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:11.700611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603682+0000","last_change":"2024-03-06T07:23:54.534612+0000","last_active":"2024-03-06T07:23:55.603682+0000","last_peered":"2024-03-06T07:23:55.603682+0000","last_clean":"2024-03-06T07:23:55.603682+0000","last_became_active":"2024-03-06T07:23:54.534313+0000","last_became_peered":"2024-03-06T07:23:54.534313+0000","last_unstale":"2024-03-06T07:23:55.603682+0000","last_undegraded":"2024-03-06T07:23:55.603682+0000","last_fullsized":"2024-03-06T07:23:55.603682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:55.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604100+0000","last_change":"2024-03-06T07:23:53.528572+0000","last_active":"2024-03-06T07:23:55.604100+0000","last_peered":"2024-03-06T07:23:55.604100+0000","last_clean":"2024-03-06T07:23:55.604100+0000","last_became_active":"2024-03-06T07:23:53.528303+0000","last_became_peered":"2024-03-06T07:23:53.528303+0000","last_unstale":"2024-03-06T07:23:55.604100+0000","last_undegraded":"2024-03-06T07:23:55.604100+0000","last_fullsized":"2024-03-06T07:23:55.604100+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.965436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.528865+0000","last_change":"2024-03-06T07:23:54.526252+0000","last_active":"2024-03-06T07:23:55.528865+0000","last_peered":"2024-03-06T07:23:55.528865+0000","last_clean":"2024-03-06T07:23:55.528865+0000","last_became_active":"2024-03-06T07:23:54.525950+0000","last_became_peered":"2024-03-06T07:23:54.525950+0000","last_unstale":"2024-03-06T07:23:55.528865+0000","last_undegraded":"2024-03-06T07:23:55.528865+0000","last_fullsized":"2024-03-06T07:23:55.528865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:19:08.344593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603414+0000","last_change":"2024-03-06T07:23:53.525007+0000","last_active":"2024-03-06T07:23:55.603414+0000","last_peered":"2024-03-06T07:23:55.603414+0000","last_clean":"2024-03-06T07:23:55.603414+0000","last_became_active":"2024-03-06T07:23:53.524715+0000","last_became_peered":"2024-03-06T07:23:53.524715+0000","last_unstale":"2024-03-06T07:23:55.603414+0000","last_undegraded":"2024-03-06T07:23:55.603414+0000","last_fullsized":"2024-03-06T07:23:55.603414+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:38.788465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603259+0000","last_change":"2024-03-06T07:23:54.534792+0000","last_active":"2024-03-06T07:23:55.603259+0000","last_peered":"2024-03-06T07:23:55.603259+0000","last_clean":"2024-03-06T07:23:55.603259+0000","last_became_active":"2024-03-06T07:23:54.534509+0000","last_became_peered":"2024-03-06T07:23:54.534509+0000","last_unstale":"2024-03-06T07:23:55.603259+0000","last_undegraded":"2024-03-06T07:23:55.603259+0000","last_fullsized":"2024-03-06T07:23:55.603259+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603202+0000","last_change":"2024-03-06T07:23:53.529919+0000","last_active":"2024-03-06T07:23:55.603202+0000","last_peered":"2024-03-06T07:23:55.603202+0000","last_clean":"2024-03-06T07:23:55.603202+0000","last_became_active":"2024-03-06T07:23:53.529441+0000","last_became_peered":"2024-03-06T07:23:53.529441+0000","last_unstale":"2024-03-06T07:23:55.603202+0000","last_undegraded":"2024-03-06T07:23:55.603202+0000","last_fullsized":"2024-03-06T07:23:55.603202+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:28.397840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533629+0000","last_change":"2024-03-06T07:23:54.523118+0000","last_active":"2024-03-06T07:23:55.533629+0000","last_peered":"2024-03-06T07:23:55.533629+0000","last_clean":"2024-03-06T07:23:55.533629+0000","last_became_active":"2024-03-06T07:23:54.522808+0000","last_became_peered":"2024-03-06T07:23:54.522808+0000","last_unstale":"2024-03-06T07:23:55.533629+0000","last_undegraded":"2024-03-06T07:23:55.533629+0000","last_fullsized":"2024-03-06T07:23:55.533629+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529142+0000","last_change":"2024-03-06T07:23:53.522861+0000","last_active":"2024-03-06T07:23:55.529142+0000","last_peered":"2024-03-06T07:23:55.529142+0000","last_clean":"2024-03-06T07:23:55.529142+0000","last_became_active":"2024-03-06T07:23:53.522582+0000","last_became_peered":"2024-03-06T07:23:53.522582+0000","last_unstale":"2024-03-06T07:23:55.529142+0000","last_undegraded":"2024-03-06T07:23:55.529142+0000","last_fullsized":"2024-03-06T07:23:55.529142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:25.203342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603522+0000","last_change":"2024-03-06T07:23:54.536692+0000","last_active":"2024-03-06T07:23:55.603522+0000","last_peered":"2024-03-06T07:23:55.603522+0000","last_clean":"2024-03-06T07:23:55.603522+0000","last_became_active":"2024-03-06T07:23:54.536377+0000","last_became_peered":"2024-03-06T07:23:54.536377+0000","last_unstale":"2024-03-06T07:23:55.603522+0000","last_undegraded":"2024-03-06T07:23:55.603522+0000","last_fullsized":"2024-03-06T07:23:55.603522+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:39.084730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604209+0000","last_change":"2024-03-06T07:23:53.529661+0000","last_active":"2024-03-06T07:23:55.604209+0000","last_peered":"2024-03-06T07:23:55.604209+0000","last_clean":"2024-03-06T07:23:55.604209+0000","last_became_active":"2024-03-06T07:23:53.529379+0000","last_became_peered":"2024-03-06T07:23:53.529379+0000","last_unstale":"2024-03-06T07:23:55.604209+0000","last_undegraded":"2024-03-06T07:23:55.604209+0000","last_fullsized":"2024-03-06T07:23:55.604209+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:26.088856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603994+0000","last_change":"2024-03-06T07:23:54.533950+0000","last_active":"2024-03-06T07:23:55.603994+0000","last_peered":"2024-03-06T07:23:55.603994+0000","last_clean":"2024-03-06T07:23:55.603994+0000","last_became_active":"2024-03-06T07:23:54.533678+0000","last_became_peered":"2024-03-06T07:23:54.533678+0000","last_unstale":"2024-03-06T07:23:55.603994+0000","last_undegraded":"2024-03-06T07:23:55.603994+0000","last_fullsized":"2024-03-06T07:23:55.603994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:43.722095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603869+0000","last_change":"2024-03-06T07:23:53.529086+0000","last_active":"2024-03-06T07:23:55.603869+0000","last_peered":"2024-03-06T07:23:55.603869+0000","last_clean":"2024-03-06T07:23:55.603869+0000","last_became_active":"2024-03-06T07:23:53.528727+0000","last_became_peered":"2024-03-06T07:23:53.528727+0000","last_unstale":"2024-03-06T07:23:55.603869+0000","last_undegraded":"2024-03-06T07:23:55.603869+0000","last_fullsized":"2024-03-06T07:23:55.603869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:51.065848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603823+0000","last_change":"2024-03-06T07:23:54.531828+0000","last_active":"2024-03-06T07:23:55.603823+0000","last_peered":"2024-03-06T07:23:55.603823+0000","last_clean":"2024-03-06T07:23:55.603823+0000","last_became_active":"2024-03-06T07:23:54.531530+0000","last_became_peered":"2024-03-06T07:23:54.531530+0000","last_unstale":"2024-03-06T07:23:55.603823+0000","last_undegraded":"2024-03-06T07:23:55.603823+0000","last_fullsized":"2024-03-06T07:23:55.603823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:01:20.040910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602647+0000","last_change":"2024-03-06T07:23:53.520721+0000","last_active":"2024-03-06T07:23:55.602647+0000","last_peered":"2024-03-06T07:23:55.602647+0000","last_clean":"2024-03-06T07:23:55.602647+0000","last_became_active":"2024-03-06T07:23:53.520457+0000","last_became_peered":"2024-03-06T07:23:53.520457+0000","last_unstale":"2024-03-06T07:23:55.602647+0000","last_undegraded":"2024-03-06T07:23:55.602647+0000","last_fullsized":"2024-03-06T07:23:55.602647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:00.608814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603177+0000","last_change":"2024-03-06T07:23:54.532922+0000","last_active":"2024-03-06T07:23:55.603177+0000","last_peered":"2024-03-06T07:23:55.603177+0000","last_clean":"2024-03-06T07:23:55.603177+0000","last_became_active":"2024-03-06T07:23:54.531965+0000","last_became_peered":"2024-03-06T07:23:54.531965+0000","last_unstale":"2024-03-06T07:23:55.603177+0000","last_undegraded":"2024-03-06T07:23:55.603177+0000","last_fullsized":"2024-03-06T07:23:55.603177+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:40.648191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603511+0000","last_change":"2024-03-06T07:23:53.522227+0000","last_active":"2024-03-06T07:23:55.603511+0000","last_peered":"2024-03-06T07:23:55.603511+0000","last_clean":"2024-03-06T07:23:55.603511+0000","last_became_active":"2024-03-06T07:23:53.521963+0000","last_became_peered":"2024-03-06T07:23:53.521963+0000","last_unstale":"2024-03-06T07:23:55.603511+0000","last_undegraded":"2024-03-06T07:23:55.603511+0000","last_fullsized":"2024-03-06T07:23:55.603511+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:06.286897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603975+0000","last_change":"2024-03-06T07:23:54.527266+0000","last_active":"2024-03-06T07:23:55.603975+0000","last_peered":"2024-03-06T07:23:55.603975+0000","last_clean":"2024-03-06T07:23:55.603975+0000","last_became_active":"2024-03-06T07:23:54.526952+0000","last_became_peered":"2024-03-06T07:23:54.526952+0000","last_unstale":"2024-03-06T07:23:55.603975+0000","last_undegraded":"2024-03-06T07:23:55.603975+0000","last_fullsized":"2024-03-06T07:23:55.603975+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:00.922090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603504+0000","last_change":"2024-03-06T07:23:53.529962+0000","last_active":"2024-03-06T07:23:55.603504+0000","last_peered":"2024-03-06T07:23:55.603504+0000","last_clean":"2024-03-06T07:23:55.603504+0000","last_became_active":"2024-03-06T07:23:53.529658+0000","last_became_peered":"2024-03-06T07:23:53.529658+0000","last_unstale":"2024-03-06T07:23:55.603504+0000","last_undegraded":"2024-03-06T07:23:55.603504+0000","last_fullsized":"2024-03-06T07:23:55.603504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:54.746607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603198+0000","last_change":"2024-03-06T07:23:54.530750+0000","last_active":"2024-03-06T07:23:55.603198+0000","last_peered":"2024-03-06T07:23:55.603198+0000","last_clean":"2024-03-06T07:23:55.603198+0000","last_became_active":"2024-03-06T07:23:54.530443+0000","last_became_peered":"2024-03-06T07:23:54.530443+0000","last_unstale":"2024-03-06T07:23:55.603198+0000","last_undegraded":"2024-03-06T07:23:55.603198+0000","last_fullsized":"2024-03-06T07:23:55.603198+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602926+0000","last_change":"2024-03-06T07:23:53.525298+0000","last_active":"2024-03-06T07:23:55.602926+0000","last_peered":"2024-03-06T07:23:55.602926+0000","last_clean":"2024-03-06T07:23:55.602926+0000","last_became_active":"2024-03-06T07:23:53.524598+0000","last_became_peered":"2024-03-06T07:23:53.524598+0000","last_unstale":"2024-03-06T07:23:55.602926+0000","last_undegraded":"2024-03-06T07:23:55.602926+0000","last_fullsized":"2024-03-06T07:23:55.602926+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:14:04.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604396+0000","last_change":"2024-03-06T07:23:54.525516+0000","last_active":"2024-03-06T07:23:55.604396+0000","last_peered":"2024-03-06T07:23:55.604396+0000","last_clean":"2024-03-06T07:23:55.604396+0000","last_became_active":"2024-03-06T07:23:54.525239+0000","last_became_peered":"2024-03-06T07:23:54.525239+0000","last_unstale":"2024-03-06T07:23:55.604396+0000","last_undegraded":"2024-03-06T07:23:55.604396+0000","last_fullsized":"2024-03-06T07:23:55.604396+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:38.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603096+0000","last_change":"2024-03-06T07:23:53.516520+0000","last_active":"2024-03-06T07:23:55.603096+0000","last_peered":"2024-03-06T07:23:55.603096+0000","last_clean":"2024-03-06T07:23:55.603096+0000","last_became_active":"2024-03-06T07:23:53.516253+0000","last_became_peered":"2024-03-06T07:23:53.516253+0000","last_unstale":"2024-03-06T07:23:55.603096+0000","last_undegraded":"2024-03-06T07:23:55.603096+0000","last_fullsized":"2024-03-06T07:23:55.603096+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.348229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603364+0000","last_change":"2024-03-06T07:23:54.538195+0000","last_active":"2024-03-06T07:23:55.603364+0000","last_peered":"2024-03-06T07:23:55.603364+0000","last_clean":"2024-03-06T07:23:55.603364+0000","last_became_active":"2024-03-06T07:23:54.537861+0000","last_became_peered":"2024-03-06T07:23:54.537861+0000","last_unstale":"2024-03-06T07:23:55.603364+0000","last_undegraded":"2024-03-06T07:23:55.603364+0000","last_fullsized":"2024-03-06T07:23:55.603364+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:56.520529+0000","last_change":"2024-03-06T07:23:53.520240+0000","last_active":"2024-03-06T07:23:56.520529+0000","last_peered":"2024-03-06T07:23:56.520529+0000","last_clean":"2024-03-06T07:23:56.520529+0000","last_became_active":"2024-03-06T07:23:53.519976+0000","last_became_peered":"2024-03-06T07:23:53.519976+0000","last_unstale":"2024-03-06T07:23:56.520529+0000","last_undegraded":"2024-03-06T07:23:56.520529+0000","last_fullsized":"2024-03-06T07:23:56.520529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:01.816188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603699+0000","last_change":"2024-03-06T07:23:54.530486+0000","last_active":"2024-03-06T07:23:55.603699+0000","last_peered":"2024-03-06T07:23:55.603699+0000","last_clean":"2024-03-06T07:23:55.603699+0000","last_became_active":"2024-03-06T07:23:54.530126+0000","last_became_peered":"2024-03-06T07:23:54.530126+0000","last_unstale":"2024-03-06T07:23:55.603699+0000","last_undegraded":"2024-03-06T07:23:55.603699+0000","last_fullsized":"2024-03-06T07:23:55.603699+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:46:05.964342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532627+0000","last_change":"2024-03-06T07:23:53.515848+0000","last_active":"2024-03-06T07:23:55.532627+0000","last_peered":"2024-03-06T07:23:55.532627+0000","last_clean":"2024-03-06T07:23:55.532627+0000","last_became_active":"2024-03-06T07:23:53.515558+0000","last_became_peered":"2024-03-06T07:23:53.515558+0000","last_unstale":"2024-03-06T07:23:55.532627+0000","last_undegraded":"2024-03-06T07:23:55.532627+0000","last_fullsized":"2024-03-06T07:23:55.532627+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:12.403487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529093+0000","last_change":"2024-03-06T07:23:54.532805+0000","last_active":"2024-03-06T07:23:55.529093+0000","last_peered":"2024-03-06T07:23:55.529093+0000","last_clean":"2024-03-06T07:23:55.529093+0000","last_became_active":"2024-03-06T07:23:54.532354+0000","last_became_peered":"2024-03-06T07:23:54.532354+0000","last_unstale":"2024-03-06T07:23:55.529093+0000","last_undegraded":"2024-03-06T07:23:55.529093+0000","last_fullsized":"2024-03-06T07:23:55.529093+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:13.160969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604478+0000","last_change":"2024-03-06T07:23:53.525174+0000","last_active":"2024-03-06T07:23:55.604478+0000","last_peered":"2024-03-06T07:23:55.604478+0000","last_clean":"2024-03-06T07:23:55.604478+0000","last_became_active":"2024-03-06T07:23:53.524911+0000","last_became_peered":"2024-03-06T07:23:53.524911+0000","last_unstale":"2024-03-06T07:23:55.604478+0000","last_undegraded":"2024-03-06T07:23:55.604478+0000","last_fullsized":"2024-03-06T07:23:55.604478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:03.343482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604016+0000","last_change":"2024-03-06T07:23:54.530786+0000","last_active":"2024-03-06T07:23:55.604016+0000","last_peered":"2024-03-06T07:23:55.604016+0000","last_clean":"2024-03-06T07:23:55.604016+0000","last_became_active":"2024-03-06T07:23:54.529882+0000","last_became_peered":"2024-03-06T07:23:54.529882+0000","last_unstale":"2024-03-06T07:23:55.604016+0000","last_undegraded":"2024-03-06T07:23:55.604016+0000","last_fullsized":"2024-03-06T07:23:55.604016+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:59.301002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533336+0000","last_change":"2024-03-06T07:23:53.524692+0000","last_active":"2024-03-06T07:23:55.533336+0000","last_peered":"2024-03-06T07:23:55.533336+0000","last_clean":"2024-03-06T07:23:55.533336+0000","last_became_active":"2024-03-06T07:23:53.524065+0000","last_became_peered":"2024-03-06T07:23:53.524065+0000","last_unstale":"2024-03-06T07:23:55.533336+0000","last_undegraded":"2024-03-06T07:23:55.533336+0000","last_fullsized":"2024-03-06T07:23:55.533336+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:43.169621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603323+0000","last_change":"2024-03-06T07:23:54.526729+0000","last_active":"2024-03-06T07:23:55.603323+0000","last_peered":"2024-03-06T07:23:55.603323+0000","last_clean":"2024-03-06T07:23:55.603323+0000","last_became_active":"2024-03-06T07:23:54.526451+0000","last_became_peered":"2024-03-06T07:23:54.526451+0000","last_unstale":"2024-03-06T07:23:55.603323+0000","last_undegraded":"2024-03-06T07:23:55.603323+0000","last_fullsized":"2024-03-06T07:23:55.603323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:29.125635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533488+0000","last_change":"2024-03-06T07:23:53.525414+0000","last_active":"2024-03-06T07:23:55.533488+0000","last_peered":"2024-03-06T07:23:55.533488+0000","last_clean":"2024-03-06T07:23:55.533488+0000","last_became_active":"2024-03-06T07:23:53.525136+0000","last_became_peered":"2024-03-06T07:23:53.525136+0000","last_unstale":"2024-03-06T07:23:55.533488+0000","last_undegraded":"2024-03-06T07:23:55.533488+0000","last_fullsized":"2024-03-06T07:23:55.533488+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604039+0000","last_change":"2024-03-06T07:23:54.529519+0000","last_active":"2024-03-06T07:23:55.604039+0000","last_peered":"2024-03-06T07:23:55.604039+0000","last_clean":"2024-03-06T07:23:55.604039+0000","last_became_active":"2024-03-06T07:23:54.529218+0000","last_became_peered":"2024-03-06T07:23:54.529218+0000","last_unstale":"2024-03-06T07:23:55.604039+0000","last_undegraded":"2024-03-06T07:23:55.604039+0000","last_fullsized":"2024-03-06T07:23:55.604039+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:39:13.906979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604084+0000","last_change":"2024-03-06T07:23:53.525343+0000","last_active":"2024-03-06T07:23:55.604084+0000","last_peered":"2024-03-06T07:23:55.604084+0000","last_clean":"2024-03-06T07:23:55.604084+0000","last_became_active":"2024-03-06T07:23:53.525010+0000","last_became_peered":"2024-03-06T07:23:53.525010+0000","last_unstale":"2024-03-06T07:23:55.604084+0000","last_undegraded":"2024-03-06T07:23:55.604084+0000","last_fullsized":"2024-03-06T07:23:55.604084+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:20:51.036735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603318+0000","last_change":"2024-03-06T07:23:54.536217+0000","last_active":"2024-03-06T07:23:55.603318+0000","last_peered":"2024-03-06T07:23:55.603318+0000","last_clean":"2024-03-06T07:23:55.603318+0000","last_became_active":"2024-03-06T07:23:54.535943+0000","last_became_peered":"2024-03-06T07:23:54.535943+0000","last_unstale":"2024-03-06T07:23:55.603318+0000","last_undegraded":"2024-03-06T07:23:55.603318+0000","last_fullsized":"2024-03-06T07:23:55.603318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602936+0000","last_change":"2024-03-06T07:23:53.524633+0000","last_active":"2024-03-06T07:23:55.602936+0000","last_peered":"2024-03-06T07:23:55.602936+0000","last_clean":"2024-03-06T07:23:55.602936+0000","last_became_active":"2024-03-06T07:23:53.524208+0000","last_became_peered":"2024-03-06T07:23:53.524208+0000","last_unstale":"2024-03-06T07:23:55.602936+0000","last_undegraded":"2024-03-06T07:23:55.602936+0000","last_fullsized":"2024-03-06T07:23:55.602936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:39.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604024+0000","last_change":"2024-03-06T07:23:54.535792+0000","last_active":"2024-03-06T07:23:55.604024+0000","last_peered":"2024-03-06T07:23:55.604024+0000","last_clean":"2024-03-06T07:23:55.604024+0000","last_became_active":"2024-03-06T07:23:54.535473+0000","last_became_peered":"2024-03-06T07:23:54.535473+0000","last_unstale":"2024-03-06T07:23:55.604024+0000","last_undegraded":"2024-03-06T07:23:55.604024+0000","last_fullsized":"2024-03-06T07:23:55.604024+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:57.778369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602487+0000","last_change":"2024-03-06T07:23:53.523825+0000","last_active":"2024-03-06T07:23:55.602487+0000","last_peered":"2024-03-06T07:23:55.602487+0000","last_clean":"2024-03-06T07:23:55.602487+0000","last_became_active":"2024-03-06T07:23:53.523542+0000","last_became_peered":"2024-03-06T07:23:53.523542+0000","last_unstale":"2024-03-06T07:23:55.602487+0000","last_undegraded":"2024-03-06T07:23:55.602487+0000","last_fullsized":"2024-03-06T07:23:55.602487+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:28.818165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":335,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603143+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:23:55.603143+0000","last_peered":"2024-03-06T07:23:55.603143+0000","last_clean":"2024-03-06T07:23:55.603143+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:23:55.603143+0000","last_undegraded":"2024-03-06T07:23:55.603143+0000","last_fullsized":"2024-03-06T07:23:55.603143+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":1249795,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":659233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":658811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":660263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":660297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":658651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":658673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":1249041,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:57.557 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:23:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.659+0000 7fa886f7a640 1 Processor -- start 2024-03-06T07:23:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.659+0000 7fa886f7a640 1 -- start start 2024-03-06T07:23:57.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa886f7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 0x7fa8800c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa886f7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8801586f0 0x7fa880158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa886f7a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8801591c0 con 0x7fa8800c5480 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa886f7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88006df40 con 0x7fa8801586f0 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa886f7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88006fdb0 con 0x7fa880155ba0 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8801586f0 0x7fa880158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8801586f0 0x7fa880158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56930/0 (socket says 172.21.15.17:56930) 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 -- 172.21.15.17:0/2550744199 learned_addr learned my addr 172.21.15.17:0/2550744199 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa87ffff640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 0x7fa8800c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 -- 172.21.15.17:0/2550744199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 msgr2=0x7fa8800c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 0x7fa8800c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 -- 172.21.15.17:0/2550744199 >> v1:172.21.15.163:6789/0 conn(0x7fa8800c5480 legacy=0x7fa88006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa885517640 1 -- 172.21.15.17:0/2550744199 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8801594a0 con 0x7fa8801586f0 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.660+0000 7fa87ffff640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 0x7fa8800c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa885517640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8801586f0 0x7fa880158b80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e50 tx=0x7fa87802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11373e1a3ebfd679 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa87f7fe640 1 -- 172.21.15.17:0/2550744199 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87802c9e0 con 0x7fa8801586f0 2024-03-06T07:23:57.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa87f7fe640 1 -- 172.21.15.17:0/2550744199 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87802cb80 con 0x7fa8801586f0 2024-03-06T07:23:57.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa87f7fe640 1 -- 172.21.15.17:0/2550744199 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780343f0 con 0x7fa8801586f0 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa886f7a640 1 -- 172.21.15.17:0/2550744199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8801586f0 msgr2=0x7fa880158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa886f7a640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8801586f0 0x7fa880158b80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e50 tx=0x7fa87802be40 comp rx=0 tx=0).stop 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa886f7a640 1 -- 172.21.15.17:0/2550744199 shutdown_connections 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.661+0000 7fa886f7a640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8801586f0 0x7fa880158b80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.662+0000 7fa886f7a640 1 --2- 172.21.15.17:0/2550744199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa880155ba0 0x7fa8800c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.662+0000 7fa886f7a640 1 -- 172.21.15.17:0/2550744199 >> 172.21.15.17:0/2550744199 conn(0x7fa88006b1a0 msgr2=0x7fa88006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.662+0000 7fa886f7a640 1 -- 172.21.15.17:0/2550744199 shutdown_connections 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.662+0000 7fa886f7a640 1 -- 172.21.15.17:0/2550744199 wait complete. 2024-03-06T07:23:57.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 Processor -- start 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 -- start start 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 0x7fa8801eb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8801586f0 0x7fa8801de920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880155680 con 0x7fa880155ba0 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa880156000 con 0x7fa8800c5480 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.663+0000 7fa886f7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880152a20 con 0x7fa8801586f0 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56936/0 (socket says 172.21.15.17:56936) 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 -- 172.21.15.17:0/23296778 learned_addr learned my addr 172.21.15.17:0/23296778 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:57.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa885517640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8801586f0 0x7fa8801de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa884d16640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 0x7fa8801eb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 -- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8801586f0 msgr2=0x7fa8801de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8801586f0 0x7fa8801de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 -- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 msgr2=0x7fa8801eb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 0x7fa8801eb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 -- 172.21.15.17:0/23296778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800c3ca0 con 0x7fa880155ba0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa884d16640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 0x7fa8801eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87ffff640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c7f0 tx=0x7fa86c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c015070 con 0x7fa880155ba0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c007a30 con 0x7fa880155ba0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800c2ea0 con 0x7fa880155ba0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c007e00 con 0x7fa880155ba0 2024-03-06T07:23:57.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.664+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801e0470 con 0x7fa880155ba0 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.665+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8800c6870 con 0x7fa880155ba0 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.666+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa86c018070 con 0x7fa880155ba0 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.666+0000 7fa87d7fa640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 0x7fa86406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.667+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa86c08a210 con 0x7fa880155ba0 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.667+0000 7fa884d16640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 0x7fa86406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:57.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.667+0000 7fa884d16640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 0x7fa86406d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8800aad80 tx=0x7fa874009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:57.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.671+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8800c6870 con 0x7fa880155ba0 2024-03-06T07:23:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.847+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa88004f330 con 0x7fa86406b500 2024-03-06T07:23:57.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.854+0000 7fa87d7fa640 1 -- 172.21.15.17:0/23296778 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7fa88004f330 con 0x7fa86406b500 2024-03-06T07:23:57.858 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:57.859 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:23:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 msgr2=0x7fa86406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 0x7fa86406d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa8800aad80 tx=0x7fa874009040 comp rx=0 tx=0).stop 2024-03-06T07:23:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 msgr2=0x7fa8800c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00c7f0 tx=0x7fa86c00cbf0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 shutdown_connections 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa86406b500 0x7fa86406d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8801586f0 0x7fa8801de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa880155ba0 0x7fa8800c3760 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 --2- 172.21.15.17:0/23296778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8800c5480 0x7fa8801eb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 >> 172.21.15.17:0/23296778 conn(0x7fa88006b1a0 msgr2=0x7fa8800c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 shutdown_connections 2024-03-06T07:23:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:57.857+0000 7fa886f7a640 1 -- 172.21.15.17:0/23296778 wait complete. 2024-03-06T07:23:57.870 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-03-06T07:23:57.392713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256148,"kb_used_data":10816,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754718572,"statfs":{"total":773094113280,"available":772831817728,"internally_reserved":0,"allocated":11075584,"data_stored":6454764,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":251055668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.889623"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533428+0000","last_change":"2024-03-06T07:23:54.521645+0000","last_active":"2024-03-06T07:23:55.533428+0000","last_peered":"2024-03-06T07:23:55.533428+0000","last_clean":"2024-03-06T07:23:55.533428+0000","last_became_active":"2024-03-06T07:23:54.521252+0000","last_became_peered":"2024-03-06T07:23:54.521252+0000","last_unstale":"2024-03-06T07:23:55.533428+0000","last_undegraded":"2024-03-06T07:23:55.533428+0000","last_fullsized":"2024-03-06T07:23:55.533428+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602645+0000","last_change":"2024-03-06T07:23:54.537527+0000","last_active":"2024-03-06T07:23:55.602645+0000","last_peered":"2024-03-06T07:23:55.602645+0000","last_clean":"2024-03-06T07:23:55.602645+0000","last_became_active":"2024-03-06T07:23:54.537036+0000","last_became_peered":"2024-03-06T07:23:54.537036+0000","last_unstale":"2024-03-06T07:23:55.602645+0000","last_undegraded":"2024-03-06T07:23:55.602645+0000","last_fullsized":"2024-03-06T07:23:55.602645+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:17.701870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602761+0000","last_change":"2024-03-06T07:23:54.530569+0000","last_active":"2024-03-06T07:23:55.602761+0000","last_peered":"2024-03-06T07:23:55.602761+0000","last_clean":"2024-03-06T07:23:55.602761+0000","last_became_active":"2024-03-06T07:23:54.530272+0000","last_became_peered":"2024-03-06T07:23:54.530272+0000","last_unstale":"2024-03-06T07:23:55.602761+0000","last_undegraded":"2024-03-06T07:23:55.602761+0000","last_fullsized":"2024-03-06T07:23:55.602761+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:24.350111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603775+0000","last_change":"2024-03-06T07:23:54.529843+0000","last_active":"2024-03-06T07:23:55.603775+0000","last_peered":"2024-03-06T07:23:55.603775+0000","last_clean":"2024-03-06T07:23:55.603775+0000","last_became_active":"2024-03-06T07:23:54.529140+0000","last_became_peered":"2024-03-06T07:23:54.529140+0000","last_unstale":"2024-03-06T07:23:55.603775+0000","last_undegraded":"2024-03-06T07:23:55.603775+0000","last_fullsized":"2024-03-06T07:23:55.603775+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:26:12.461791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533206+0000","last_change":"2024-03-06T07:23:54.520680+0000","last_active":"2024-03-06T07:23:55.533206+0000","last_peered":"2024-03-06T07:23:55.533206+0000","last_clean":"2024-03-06T07:23:55.533206+0000","last_became_active":"2024-03-06T07:23:54.520397+0000","last_became_peered":"2024-03-06T07:23:54.520397+0000","last_unstale":"2024-03-06T07:23:55.533206+0000","last_undegraded":"2024-03-06T07:23:55.533206+0000","last_fullsized":"2024-03-06T07:23:55.533206+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604095+0000","last_change":"2024-03-06T07:23:54.525453+0000","last_active":"2024-03-06T07:23:55.604095+0000","last_peered":"2024-03-06T07:23:55.604095+0000","last_clean":"2024-03-06T07:23:55.604095+0000","last_became_active":"2024-03-06T07:23:54.525176+0000","last_became_peered":"2024-03-06T07:23:54.525176+0000","last_unstale":"2024-03-06T07:23:55.604095+0000","last_undegraded":"2024-03-06T07:23:55.604095+0000","last_fullsized":"2024-03-06T07:23:55.604095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:50.888132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603900+0000","last_change":"2024-03-06T07:23:54.529179+0000","last_active":"2024-03-06T07:23:55.603900+0000","last_peered":"2024-03-06T07:23:55.603900+0000","last_clean":"2024-03-06T07:23:55.603900+0000","last_became_active":"2024-03-06T07:23:54.528887+0000","last_became_peered":"2024-03-06T07:23:54.528887+0000","last_unstale":"2024-03-06T07:23:55.603900+0000","last_undegraded":"2024-03-06T07:23:55.603900+0000","last_fullsized":"2024-03-06T07:23:55.603900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:30:54.916203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602383+0000","last_change":"2024-03-06T07:23:54.530509+0000","last_active":"2024-03-06T07:23:55.602383+0000","last_peered":"2024-03-06T07:23:55.602383+0000","last_clean":"2024-03-06T07:23:55.602383+0000","last_became_active":"2024-03-06T07:23:54.530225+0000","last_became_peered":"2024-03-06T07:23:54.530225+0000","last_unstale":"2024-03-06T07:23:55.602383+0000","last_undegraded":"2024-03-06T07:23:55.602383+0000","last_fullsized":"2024-03-06T07:23:55.602383+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:09.334362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603712+0000","last_change":"2024-03-06T07:23:54.541162+0000","last_active":"2024-03-06T07:23:55.603712+0000","last_peered":"2024-03-06T07:23:55.603712+0000","last_clean":"2024-03-06T07:23:55.603712+0000","last_became_active":"2024-03-06T07:23:54.540890+0000","last_became_peered":"2024-03-06T07:23:54.540890+0000","last_unstale":"2024-03-06T07:23:55.603712+0000","last_undegraded":"2024-03-06T07:23:55.603712+0000","last_fullsized":"2024-03-06T07:23:55.603712+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:37.822960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602762+0000","last_change":"2024-03-06T07:23:54.529160+0000","last_active":"2024-03-06T07:23:55.602762+0000","last_peered":"2024-03-06T07:23:55.602762+0000","last_clean":"2024-03-06T07:23:55.602762+0000","last_became_active":"2024-03-06T07:23:54.528885+0000","last_became_peered":"2024-03-06T07:23:54.528885+0000","last_unstale":"2024-03-06T07:23:55.602762+0000","last_undegraded":"2024-03-06T07:23:55.602762+0000","last_fullsized":"2024-03-06T07:23:55.602762+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533420+0000","last_change":"2024-03-06T07:23:54.532560+0000","last_active":"2024-03-06T07:23:55.533420+0000","last_peered":"2024-03-06T07:23:55.533420+0000","last_clean":"2024-03-06T07:23:55.533420+0000","last_became_active":"2024-03-06T07:23:54.532263+0000","last_became_peered":"2024-03-06T07:23:54.532263+0000","last_unstale":"2024-03-06T07:23:55.533420+0000","last_undegraded":"2024-03-06T07:23:55.533420+0000","last_fullsized":"2024-03-06T07:23:55.533420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604268+0000","last_change":"2024-03-06T07:23:54.530333+0000","last_active":"2024-03-06T07:23:55.604268+0000","last_peered":"2024-03-06T07:23:55.604268+0000","last_clean":"2024-03-06T07:23:55.604268+0000","last_became_active":"2024-03-06T07:23:54.530053+0000","last_became_peered":"2024-03-06T07:23:54.530053+0000","last_unstale":"2024-03-06T07:23:55.604268+0000","last_undegraded":"2024-03-06T07:23:55.604268+0000","last_fullsized":"2024-03-06T07:23:55.604268+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:18.429026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603034+0000","last_change":"2024-03-06T07:23:54.538305+0000","last_active":"2024-03-06T07:23:55.603034+0000","last_peered":"2024-03-06T07:23:55.603034+0000","last_clean":"2024-03-06T07:23:55.603034+0000","last_became_active":"2024-03-06T07:23:54.538040+0000","last_became_peered":"2024-03-06T07:23:54.538040+0000","last_unstale":"2024-03-06T07:23:55.603034+0000","last_undegraded":"2024-03-06T07:23:55.603034+0000","last_fullsized":"2024-03-06T07:23:55.603034+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603527+0000","last_change":"2024-03-06T07:23:54.531008+0000","last_active":"2024-03-06T07:23:55.603527+0000","last_peered":"2024-03-06T07:23:55.603527+0000","last_clean":"2024-03-06T07:23:55.603527+0000","last_became_active":"2024-03-06T07:23:54.530735+0000","last_became_peered":"2024-03-06T07:23:54.530735+0000","last_unstale":"2024-03-06T07:23:55.603527+0000","last_undegraded":"2024-03-06T07:23:55.603527+0000","last_fullsized":"2024-03-06T07:23:55.603527+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:51:24.127859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533130+0000","last_change":"2024-03-06T07:23:54.532003+0000","last_active":"2024-03-06T07:23:55.533130+0000","last_peered":"2024-03-06T07:23:55.533130+0000","last_clean":"2024-03-06T07:23:55.533130+0000","last_became_active":"2024-03-06T07:23:54.531720+0000","last_became_peered":"2024-03-06T07:23:54.531720+0000","last_unstale":"2024-03-06T07:23:55.533130+0000","last_undegraded":"2024-03-06T07:23:55.533130+0000","last_fullsized":"2024-03-06T07:23:55.533130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603676+0000","last_change":"2024-03-06T07:23:54.528496+0000","last_active":"2024-03-06T07:23:55.603676+0000","last_peered":"2024-03-06T07:23:55.603676+0000","last_clean":"2024-03-06T07:23:55.603676+0000","last_became_active":"2024-03-06T07:23:54.528189+0000","last_became_peered":"2024-03-06T07:23:54.528189+0000","last_unstale":"2024-03-06T07:23:55.603676+0000","last_undegraded":"2024-03-06T07:23:55.603676+0000","last_fullsized":"2024-03-06T07:23:55.603676+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:28:35.102000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603106+0000","last_change":"2024-03-06T07:23:54.538887+0000","last_active":"2024-03-06T07:23:55.603106+0000","last_peered":"2024-03-06T07:23:55.603106+0000","last_clean":"2024-03-06T07:23:55.603106+0000","last_became_active":"2024-03-06T07:23:54.538021+0000","last_became_peered":"2024-03-06T07:23:54.538021+0000","last_unstale":"2024-03-06T07:23:55.603106+0000","last_undegraded":"2024-03-06T07:23:55.603106+0000","last_fullsized":"2024-03-06T07:23:55.603106+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602806+0000","last_change":"2024-03-06T07:23:54.537314+0000","last_active":"2024-03-06T07:23:55.602806+0000","last_peered":"2024-03-06T07:23:55.602806+0000","last_clean":"2024-03-06T07:23:55.602806+0000","last_became_active":"2024-03-06T07:23:54.536670+0000","last_became_peered":"2024-03-06T07:23:54.536670+0000","last_unstale":"2024-03-06T07:23:55.602806+0000","last_undegraded":"2024-03-06T07:23:55.602806+0000","last_fullsized":"2024-03-06T07:23:55.602806+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:17.015711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532466+0000","last_change":"2024-03-06T07:23:54.530130+0000","last_active":"2024-03-06T07:23:55.532466+0000","last_peered":"2024-03-06T07:23:55.532466+0000","last_clean":"2024-03-06T07:23:55.532466+0000","last_became_active":"2024-03-06T07:23:54.529876+0000","last_became_peered":"2024-03-06T07:23:54.529876+0000","last_unstale":"2024-03-06T07:23:55.532466+0000","last_undegraded":"2024-03-06T07:23:55.532466+0000","last_fullsized":"2024-03-06T07:23:55.532466+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.528915+0000","last_change":"2024-03-06T07:23:54.531498+0000","last_active":"2024-03-06T07:23:55.528915+0000","last_peered":"2024-03-06T07:23:55.528915+0000","last_clean":"2024-03-06T07:23:55.528915+0000","last_became_active":"2024-03-06T07:23:54.531221+0000","last_became_peered":"2024-03-06T07:23:54.531221+0000","last_unstale":"2024-03-06T07:23:55.528915+0000","last_undegraded":"2024-03-06T07:23:55.528915+0000","last_fullsized":"2024-03-06T07:23:55.528915+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:02:08.274669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603877+0000","last_change":"2024-03-06T07:23:54.532907+0000","last_active":"2024-03-06T07:23:55.603877+0000","last_peered":"2024-03-06T07:23:55.603877+0000","last_clean":"2024-03-06T07:23:55.603877+0000","last_became_active":"2024-03-06T07:23:54.531373+0000","last_became_peered":"2024-03-06T07:23:54.531373+0000","last_unstale":"2024-03-06T07:23:55.603877+0000","last_undegraded":"2024-03-06T07:23:55.603877+0000","last_fullsized":"2024-03-06T07:23:55.603877+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:07.175141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533382+0000","last_change":"2024-03-06T07:23:54.524285+0000","last_active":"2024-03-06T07:23:55.533382+0000","last_peered":"2024-03-06T07:23:55.533382+0000","last_clean":"2024-03-06T07:23:55.533382+0000","last_became_active":"2024-03-06T07:23:54.524013+0000","last_became_peered":"2024-03-06T07:23:54.524013+0000","last_unstale":"2024-03-06T07:23:55.533382+0000","last_undegraded":"2024-03-06T07:23:55.533382+0000","last_fullsized":"2024-03-06T07:23:55.533382+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603535+0000","last_change":"2024-03-06T07:23:54.535573+0000","last_active":"2024-03-06T07:23:55.603535+0000","last_peered":"2024-03-06T07:23:55.603535+0000","last_clean":"2024-03-06T07:23:55.603535+0000","last_became_active":"2024-03-06T07:23:54.533542+0000","last_became_peered":"2024-03-06T07:23:54.533542+0000","last_unstale":"2024-03-06T07:23:55.603535+0000","last_undegraded":"2024-03-06T07:23:55.603535+0000","last_fullsized":"2024-03-06T07:23:55.603535+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:43.722095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602926+0000","last_change":"2024-03-06T07:23:54.534260+0000","last_active":"2024-03-06T07:23:55.602926+0000","last_peered":"2024-03-06T07:23:55.602926+0000","last_clean":"2024-03-06T07:23:55.602926+0000","last_became_active":"2024-03-06T07:23:54.532730+0000","last_became_peered":"2024-03-06T07:23:54.532730+0000","last_unstale":"2024-03-06T07:23:55.602926+0000","last_undegraded":"2024-03-06T07:23:55.602926+0000","last_fullsized":"2024-03-06T07:23:55.602926+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603687+0000","last_change":"2024-03-06T07:23:54.531365+0000","last_active":"2024-03-06T07:23:55.603687+0000","last_peered":"2024-03-06T07:23:55.603687+0000","last_clean":"2024-03-06T07:23:55.603687+0000","last_became_active":"2024-03-06T07:23:54.529649+0000","last_became_peered":"2024-03-06T07:23:54.529649+0000","last_unstale":"2024-03-06T07:23:55.603687+0000","last_undegraded":"2024-03-06T07:23:55.603687+0000","last_fullsized":"2024-03-06T07:23:55.603687+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:55.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529134+0000","last_change":"2024-03-06T07:23:54.521415+0000","last_active":"2024-03-06T07:23:55.529134+0000","last_peered":"2024-03-06T07:23:55.529134+0000","last_clean":"2024-03-06T07:23:55.529134+0000","last_became_active":"2024-03-06T07:23:54.521112+0000","last_became_peered":"2024-03-06T07:23:54.521112+0000","last_unstale":"2024-03-06T07:23:55.529134+0000","last_undegraded":"2024-03-06T07:23:55.529134+0000","last_fullsized":"2024-03-06T07:23:55.529134+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:04.090188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532610+0000","last_change":"2024-03-06T07:23:54.529429+0000","last_active":"2024-03-06T07:23:55.532610+0000","last_peered":"2024-03-06T07:23:55.532610+0000","last_clean":"2024-03-06T07:23:55.532610+0000","last_became_active":"2024-03-06T07:23:54.529131+0000","last_became_peered":"2024-03-06T07:23:54.529131+0000","last_unstale":"2024-03-06T07:23:55.532610+0000","last_undegraded":"2024-03-06T07:23:55.532610+0000","last_fullsized":"2024-03-06T07:23:55.532610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604378+0000","last_change":"2024-03-06T07:23:54.531739+0000","last_active":"2024-03-06T07:23:55.604378+0000","last_peered":"2024-03-06T07:23:55.604378+0000","last_clean":"2024-03-06T07:23:55.604378+0000","last_became_active":"2024-03-06T07:23:54.531418+0000","last_became_peered":"2024-03-06T07:23:54.531418+0000","last_unstale":"2024-03-06T07:23:55.604378+0000","last_undegraded":"2024-03-06T07:23:55.604378+0000","last_fullsized":"2024-03-06T07:23:55.604378+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:43.405621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529203+0000","last_change":"2024-03-06T07:23:54.523562+0000","last_active":"2024-03-06T07:23:55.529203+0000","last_peered":"2024-03-06T07:23:55.529203+0000","last_clean":"2024-03-06T07:23:55.529203+0000","last_became_active":"2024-03-06T07:23:54.523301+0000","last_became_peered":"2024-03-06T07:23:54.523301+0000","last_unstale":"2024-03-06T07:23:55.529203+0000","last_undegraded":"2024-03-06T07:23:55.529203+0000","last_fullsized":"2024-03-06T07:23:55.529203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:18:19.129197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603679+0000","last_change":"2024-03-06T07:23:54.524738+0000","last_active":"2024-03-06T07:23:55.603679+0000","last_peered":"2024-03-06T07:23:55.603679+0000","last_clean":"2024-03-06T07:23:55.603679+0000","last_became_active":"2024-03-06T07:23:54.524108+0000","last_became_peered":"2024-03-06T07:23:54.524108+0000","last_unstale":"2024-03-06T07:23:55.603679+0000","last_undegraded":"2024-03-06T07:23:55.603679+0000","last_fullsized":"2024-03-06T07:23:55.603679+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:34:54.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603135+0000","last_change":"2024-03-06T07:23:54.530124+0000","last_active":"2024-03-06T07:23:55.603135+0000","last_peered":"2024-03-06T07:23:55.603135+0000","last_clean":"2024-03-06T07:23:55.603135+0000","last_became_active":"2024-03-06T07:23:54.529714+0000","last_became_peered":"2024-03-06T07:23:54.529714+0000","last_unstale":"2024-03-06T07:23:55.603135+0000","last_undegraded":"2024-03-06T07:23:55.603135+0000","last_fullsized":"2024-03-06T07:23:55.603135+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:54.194570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602930+0000","last_change":"2024-03-06T07:23:54.531367+0000","last_active":"2024-03-06T07:23:55.602930+0000","last_peered":"2024-03-06T07:23:55.602930+0000","last_clean":"2024-03-06T07:23:55.602930+0000","last_became_active":"2024-03-06T07:23:54.531075+0000","last_became_peered":"2024-03-06T07:23:54.531075+0000","last_unstale":"2024-03-06T07:23:55.602930+0000","last_undegraded":"2024-03-06T07:23:55.602930+0000","last_fullsized":"2024-03-06T07:23:55.602930+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:03:18.162482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603633+0000","last_change":"2024-03-06T07:23:54.528967+0000","last_active":"2024-03-06T07:23:55.603633+0000","last_peered":"2024-03-06T07:23:55.603633+0000","last_clean":"2024-03-06T07:23:55.603633+0000","last_became_active":"2024-03-06T07:23:54.528683+0000","last_became_peered":"2024-03-06T07:23:54.528683+0000","last_unstale":"2024-03-06T07:23:55.603633+0000","last_undegraded":"2024-03-06T07:23:55.603633+0000","last_fullsized":"2024-03-06T07:23:55.603633+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:39.793464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"108'4","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532323+0000","last_change":"2024-03-06T07:23:53.522047+0000","last_active":"2024-03-06T07:23:55.532323+0000","last_peered":"2024-03-06T07:23:55.532323+0000","last_clean":"2024-03-06T07:23:55.532323+0000","last_became_active":"2024-03-06T07:23:53.521682+0000","last_became_peered":"2024-03-06T07:23:53.521682+0000","last_unstale":"2024-03-06T07:23:55.532323+0000","last_undegraded":"2024-03-06T07:23:55.532323+0000","last_fullsized":"2024-03-06T07:23:55.532323+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:43.169621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602804+0000","last_change":"2024-03-06T07:23:54.530485+0000","last_active":"2024-03-06T07:23:55.602804+0000","last_peered":"2024-03-06T07:23:55.602804+0000","last_clean":"2024-03-06T07:23:55.602804+0000","last_became_active":"2024-03-06T07:23:54.530226+0000","last_became_peered":"2024-03-06T07:23:54.530226+0000","last_unstale":"2024-03-06T07:23:55.602804+0000","last_undegraded":"2024-03-06T07:23:55.602804+0000","last_fullsized":"2024-03-06T07:23:55.602804+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:34:39.865419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602869+0000","last_change":"2024-03-06T07:23:53.524591+0000","last_active":"2024-03-06T07:23:55.602869+0000","last_peered":"2024-03-06T07:23:55.602869+0000","last_clean":"2024-03-06T07:23:55.602869+0000","last_became_active":"2024-03-06T07:23:53.524313+0000","last_became_peered":"2024-03-06T07:23:53.524313+0000","last_unstale":"2024-03-06T07:23:55.602869+0000","last_undegraded":"2024-03-06T07:23:55.602869+0000","last_fullsized":"2024-03-06T07:23:55.602869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:06.633862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603411+0000","last_change":"2024-03-06T07:23:54.538108+0000","last_active":"2024-03-06T07:23:55.603411+0000","last_peered":"2024-03-06T07:23:55.603411+0000","last_clean":"2024-03-06T07:23:55.603411+0000","last_became_active":"2024-03-06T07:23:54.537771+0000","last_became_peered":"2024-03-06T07:23:54.537771+0000","last_unstale":"2024-03-06T07:23:55.603411+0000","last_undegraded":"2024-03-06T07:23:55.603411+0000","last_fullsized":"2024-03-06T07:23:55.603411+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:39.793464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604187+0000","last_change":"2024-03-06T07:23:53.524512+0000","last_active":"2024-03-06T07:23:55.604187+0000","last_peered":"2024-03-06T07:23:55.604187+0000","last_clean":"2024-03-06T07:23:55.604187+0000","last_became_active":"2024-03-06T07:23:53.524230+0000","last_became_peered":"2024-03-06T07:23:53.524230+0000","last_unstale":"2024-03-06T07:23:55.604187+0000","last_undegraded":"2024-03-06T07:23:55.604187+0000","last_fullsized":"2024-03-06T07:23:55.604187+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:29.292121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603866+0000","last_change":"2024-03-06T07:23:54.542211+0000","last_active":"2024-03-06T07:23:55.603866+0000","last_peered":"2024-03-06T07:23:55.603866+0000","last_clean":"2024-03-06T07:23:55.603866+0000","last_became_active":"2024-03-06T07:23:54.541948+0000","last_became_peered":"2024-03-06T07:23:54.541948+0000","last_unstale":"2024-03-06T07:23:55.603866+0000","last_undegraded":"2024-03-06T07:23:55.603866+0000","last_fullsized":"2024-03-06T07:23:55.603866+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:22.340862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603789+0000","last_change":"2024-03-06T07:23:53.529569+0000","last_active":"2024-03-06T07:23:55.603789+0000","last_peered":"2024-03-06T07:23:55.603789+0000","last_clean":"2024-03-06T07:23:55.603789+0000","last_became_active":"2024-03-06T07:23:53.528175+0000","last_became_peered":"2024-03-06T07:23:53.528175+0000","last_unstale":"2024-03-06T07:23:55.603789+0000","last_undegraded":"2024-03-06T07:23:55.603789+0000","last_fullsized":"2024-03-06T07:23:55.603789+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:46.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602477+0000","last_change":"2024-03-06T07:23:54.537036+0000","last_active":"2024-03-06T07:23:55.602477+0000","last_peered":"2024-03-06T07:23:55.602477+0000","last_clean":"2024-03-06T07:23:55.602477+0000","last_became_active":"2024-03-06T07:23:54.531441+0000","last_became_peered":"2024-03-06T07:23:54.531441+0000","last_unstale":"2024-03-06T07:23:55.602477+0000","last_undegraded":"2024-03-06T07:23:55.602477+0000","last_fullsized":"2024-03-06T07:23:55.602477+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:30.297119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533416+0000","last_change":"2024-03-06T07:23:53.524983+0000","last_active":"2024-03-06T07:23:55.533416+0000","last_peered":"2024-03-06T07:23:55.533416+0000","last_clean":"2024-03-06T07:23:55.533416+0000","last_became_active":"2024-03-06T07:23:53.524421+0000","last_became_peered":"2024-03-06T07:23:53.524421+0000","last_unstale":"2024-03-06T07:23:55.533416+0000","last_undegraded":"2024-03-06T07:23:55.533416+0000","last_fullsized":"2024-03-06T07:23:55.533416+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:01.816188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533146+0000","last_change":"2024-03-06T07:23:54.525774+0000","last_active":"2024-03-06T07:23:55.533146+0000","last_peered":"2024-03-06T07:23:55.533146+0000","last_clean":"2024-03-06T07:23:55.533146+0000","last_became_active":"2024-03-06T07:23:54.525487+0000","last_became_peered":"2024-03-06T07:23:54.525487+0000","last_unstale":"2024-03-06T07:23:55.533146+0000","last_undegraded":"2024-03-06T07:23:55.533146+0000","last_fullsized":"2024-03-06T07:23:55.533146+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603980+0000","last_change":"2024-03-06T07:23:53.527677+0000","last_active":"2024-03-06T07:23:55.603980+0000","last_peered":"2024-03-06T07:23:55.603980+0000","last_clean":"2024-03-06T07:23:55.603980+0000","last_became_active":"2024-03-06T07:23:53.527399+0000","last_became_peered":"2024-03-06T07:23:53.527399+0000","last_unstale":"2024-03-06T07:23:55.603980+0000","last_undegraded":"2024-03-06T07:23:55.603980+0000","last_fullsized":"2024-03-06T07:23:55.603980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:00:27.023207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603826+0000","last_change":"2024-03-06T07:23:54.530826+0000","last_active":"2024-03-06T07:23:55.603826+0000","last_peered":"2024-03-06T07:23:55.603826+0000","last_clean":"2024-03-06T07:23:55.603826+0000","last_became_active":"2024-03-06T07:23:54.530298+0000","last_became_peered":"2024-03-06T07:23:54.530298+0000","last_unstale":"2024-03-06T07:23:55.603826+0000","last_undegraded":"2024-03-06T07:23:55.603826+0000","last_fullsized":"2024-03-06T07:23:55.603826+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:47:27.693614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603231+0000","last_change":"2024-03-06T07:23:53.528171+0000","last_active":"2024-03-06T07:23:55.603231+0000","last_peered":"2024-03-06T07:23:55.603231+0000","last_clean":"2024-03-06T07:23:55.603231+0000","last_became_active":"2024-03-06T07:23:53.527873+0000","last_became_peered":"2024-03-06T07:23:53.527873+0000","last_unstale":"2024-03-06T07:23:55.603231+0000","last_undegraded":"2024-03-06T07:23:55.603231+0000","last_fullsized":"2024-03-06T07:23:55.603231+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:01:46.974463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603175+0000","last_change":"2024-03-06T07:23:54.531226+0000","last_active":"2024-03-06T07:23:55.603175+0000","last_peered":"2024-03-06T07:23:55.603175+0000","last_clean":"2024-03-06T07:23:55.603175+0000","last_became_active":"2024-03-06T07:23:54.530960+0000","last_became_peered":"2024-03-06T07:23:54.530960+0000","last_unstale":"2024-03-06T07:23:55.603175+0000","last_undegraded":"2024-03-06T07:23:55.603175+0000","last_fullsized":"2024-03-06T07:23:55.603175+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603981+0000","last_change":"2024-03-06T07:23:53.524011+0000","last_active":"2024-03-06T07:23:55.603981+0000","last_peered":"2024-03-06T07:23:55.603981+0000","last_clean":"2024-03-06T07:23:55.603981+0000","last_became_active":"2024-03-06T07:23:53.523742+0000","last_became_peered":"2024-03-06T07:23:53.523742+0000","last_unstale":"2024-03-06T07:23:55.603981+0000","last_undegraded":"2024-03-06T07:23:55.603981+0000","last_fullsized":"2024-03-06T07:23:55.603981+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:19.811535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532130+0000","last_change":"2024-03-06T07:23:54.525862+0000","last_active":"2024-03-06T07:23:55.532130+0000","last_peered":"2024-03-06T07:23:55.532130+0000","last_clean":"2024-03-06T07:23:55.532130+0000","last_became_active":"2024-03-06T07:23:54.525532+0000","last_became_peered":"2024-03-06T07:23:54.525532+0000","last_unstale":"2024-03-06T07:23:55.532130+0000","last_undegraded":"2024-03-06T07:23:55.532130+0000","last_fullsized":"2024-03-06T07:23:55.532130+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:52.070847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532111+0000","last_change":"2024-03-06T07:23:53.519536+0000","last_active":"2024-03-06T07:23:55.532111+0000","last_peered":"2024-03-06T07:23:55.532111+0000","last_clean":"2024-03-06T07:23:55.532111+0000","last_became_active":"2024-03-06T07:23:53.519209+0000","last_became_peered":"2024-03-06T07:23:53.519209+0000","last_unstale":"2024-03-06T07:23:55.532111+0000","last_undegraded":"2024-03-06T07:23:55.532111+0000","last_fullsized":"2024-03-06T07:23:55.532111+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:00.179219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603338+0000","last_change":"2024-03-06T07:23:54.531752+0000","last_active":"2024-03-06T07:23:55.603338+0000","last_peered":"2024-03-06T07:23:55.603338+0000","last_clean":"2024-03-06T07:23:55.603338+0000","last_became_active":"2024-03-06T07:23:54.530846+0000","last_became_peered":"2024-03-06T07:23:54.530846+0000","last_unstale":"2024-03-06T07:23:55.603338+0000","last_undegraded":"2024-03-06T07:23:55.603338+0000","last_fullsized":"2024-03-06T07:23:55.603338+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603363+0000","last_change":"2024-03-06T07:23:53.527709+0000","last_active":"2024-03-06T07:23:55.603363+0000","last_peered":"2024-03-06T07:23:55.603363+0000","last_clean":"2024-03-06T07:23:55.603363+0000","last_became_active":"2024-03-06T07:23:53.526482+0000","last_became_peered":"2024-03-06T07:23:53.526482+0000","last_unstale":"2024-03-06T07:23:55.603363+0000","last_undegraded":"2024-03-06T07:23:55.603363+0000","last_fullsized":"2024-03-06T07:23:55.603363+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:28.397840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602946+0000","last_change":"2024-03-06T07:23:54.526827+0000","last_active":"2024-03-06T07:23:55.602946+0000","last_peered":"2024-03-06T07:23:55.602946+0000","last_clean":"2024-03-06T07:23:55.602946+0000","last_became_active":"2024-03-06T07:23:54.526530+0000","last_became_peered":"2024-03-06T07:23:54.526530+0000","last_unstale":"2024-03-06T07:23:55.602946+0000","last_undegraded":"2024-03-06T07:23:55.602946+0000","last_fullsized":"2024-03-06T07:23:55.602946+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:37.607508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"108'3","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603353+0000","last_change":"2024-03-06T07:23:53.527293+0000","last_active":"2024-03-06T07:23:55.603353+0000","last_peered":"2024-03-06T07:23:55.603353+0000","last_clean":"2024-03-06T07:23:55.603353+0000","last_became_active":"2024-03-06T07:23:53.527043+0000","last_became_peered":"2024-03-06T07:23:53.527043+0000","last_unstale":"2024-03-06T07:23:55.603353+0000","last_undegraded":"2024-03-06T07:23:55.603353+0000","last_fullsized":"2024-03-06T07:23:55.603353+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603837+0000","last_change":"2024-03-06T07:23:54.531117+0000","last_active":"2024-03-06T07:23:55.603837+0000","last_peered":"2024-03-06T07:23:55.603837+0000","last_clean":"2024-03-06T07:23:55.603837+0000","last_became_active":"2024-03-06T07:23:54.529940+0000","last_became_peered":"2024-03-06T07:23:54.529940+0000","last_unstale":"2024-03-06T07:23:55.603837+0000","last_undegraded":"2024-03-06T07:23:55.603837+0000","last_fullsized":"2024-03-06T07:23:55.603837+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:45.970434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"108'6","reported_seq":20,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603143+0000","last_change":"2024-03-06T07:23:53.526550+0000","last_active":"2024-03-06T07:23:55.603143+0000","last_peered":"2024-03-06T07:23:55.603143+0000","last_clean":"2024-03-06T07:23:55.603143+0000","last_became_active":"2024-03-06T07:23:53.526251+0000","last_became_peered":"2024-03-06T07:23:53.526251+0000","last_unstale":"2024-03-06T07:23:55.603143+0000","last_undegraded":"2024-03-06T07:23:55.603143+0000","last_fullsized":"2024-03-06T07:23:55.603143+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:52.186396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532508+0000","last_change":"2024-03-06T07:23:54.524567+0000","last_active":"2024-03-06T07:23:55.532508+0000","last_peered":"2024-03-06T07:23:55.532508+0000","last_clean":"2024-03-06T07:23:55.532508+0000","last_became_active":"2024-03-06T07:23:54.523152+0000","last_became_peered":"2024-03-06T07:23:54.523152+0000","last_unstale":"2024-03-06T07:23:55.532508+0000","last_undegraded":"2024-03-06T07:23:55.532508+0000","last_fullsized":"2024-03-06T07:23:55.532508+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:27.093855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602862+0000","last_change":"2024-03-06T07:23:53.523939+0000","last_active":"2024-03-06T07:23:55.602862+0000","last_peered":"2024-03-06T07:23:55.602862+0000","last_clean":"2024-03-06T07:23:55.602862+0000","last_became_active":"2024-03-06T07:23:53.523644+0000","last_became_peered":"2024-03-06T07:23:53.523644+0000","last_unstale":"2024-03-06T07:23:55.602862+0000","last_undegraded":"2024-03-06T07:23:55.602862+0000","last_fullsized":"2024-03-06T07:23:55.602862+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:11.700611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603682+0000","last_change":"2024-03-06T07:23:54.534612+0000","last_active":"2024-03-06T07:23:55.603682+0000","last_peered":"2024-03-06T07:23:55.603682+0000","last_clean":"2024-03-06T07:23:55.603682+0000","last_became_active":"2024-03-06T07:23:54.534313+0000","last_became_peered":"2024-03-06T07:23:54.534313+0000","last_unstale":"2024-03-06T07:23:55.603682+0000","last_undegraded":"2024-03-06T07:23:55.603682+0000","last_fullsized":"2024-03-06T07:23:55.603682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:55.751605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604100+0000","last_change":"2024-03-06T07:23:53.528572+0000","last_active":"2024-03-06T07:23:55.604100+0000","last_peered":"2024-03-06T07:23:55.604100+0000","last_clean":"2024-03-06T07:23:55.604100+0000","last_became_active":"2024-03-06T07:23:53.528303+0000","last_became_peered":"2024-03-06T07:23:53.528303+0000","last_unstale":"2024-03-06T07:23:55.604100+0000","last_undegraded":"2024-03-06T07:23:55.604100+0000","last_fullsized":"2024-03-06T07:23:55.604100+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:22:44.965436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.528865+0000","last_change":"2024-03-06T07:23:54.526252+0000","last_active":"2024-03-06T07:23:55.528865+0000","last_peered":"2024-03-06T07:23:55.528865+0000","last_clean":"2024-03-06T07:23:55.528865+0000","last_became_active":"2024-03-06T07:23:54.525950+0000","last_became_peered":"2024-03-06T07:23:54.525950+0000","last_unstale":"2024-03-06T07:23:55.528865+0000","last_undegraded":"2024-03-06T07:23:55.528865+0000","last_fullsized":"2024-03-06T07:23:55.528865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:19:08.344593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603414+0000","last_change":"2024-03-06T07:23:53.525007+0000","last_active":"2024-03-06T07:23:55.603414+0000","last_peered":"2024-03-06T07:23:55.603414+0000","last_clean":"2024-03-06T07:23:55.603414+0000","last_became_active":"2024-03-06T07:23:53.524715+0000","last_became_peered":"2024-03-06T07:23:53.524715+0000","last_unstale":"2024-03-06T07:23:55.603414+0000","last_undegraded":"2024-03-06T07:23:55.603414+0000","last_fullsized":"2024-03-06T07:23:55.603414+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:08:38.788465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603259+0000","last_change":"2024-03-06T07:23:54.534792+0000","last_active":"2024-03-06T07:23:55.603259+0000","last_peered":"2024-03-06T07:23:55.603259+0000","last_clean":"2024-03-06T07:23:55.603259+0000","last_became_active":"2024-03-06T07:23:54.534509+0000","last_became_peered":"2024-03-06T07:23:54.534509+0000","last_unstale":"2024-03-06T07:23:55.603259+0000","last_undegraded":"2024-03-06T07:23:55.603259+0000","last_fullsized":"2024-03-06T07:23:55.603259+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603202+0000","last_change":"2024-03-06T07:23:53.529919+0000","last_active":"2024-03-06T07:23:55.603202+0000","last_peered":"2024-03-06T07:23:55.603202+0000","last_clean":"2024-03-06T07:23:55.603202+0000","last_became_active":"2024-03-06T07:23:53.529441+0000","last_became_peered":"2024-03-06T07:23:53.529441+0000","last_unstale":"2024-03-06T07:23:55.603202+0000","last_undegraded":"2024-03-06T07:23:55.603202+0000","last_fullsized":"2024-03-06T07:23:55.603202+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:28.397840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533629+0000","last_change":"2024-03-06T07:23:54.523118+0000","last_active":"2024-03-06T07:23:55.533629+0000","last_peered":"2024-03-06T07:23:55.533629+0000","last_clean":"2024-03-06T07:23:55.533629+0000","last_became_active":"2024-03-06T07:23:54.522808+0000","last_became_peered":"2024-03-06T07:23:54.522808+0000","last_unstale":"2024-03-06T07:23:55.533629+0000","last_undegraded":"2024-03-06T07:23:55.533629+0000","last_fullsized":"2024-03-06T07:23:55.533629+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529142+0000","last_change":"2024-03-06T07:23:53.522861+0000","last_active":"2024-03-06T07:23:55.529142+0000","last_peered":"2024-03-06T07:23:55.529142+0000","last_clean":"2024-03-06T07:23:55.529142+0000","last_became_active":"2024-03-06T07:23:53.522582+0000","last_became_peered":"2024-03-06T07:23:53.522582+0000","last_unstale":"2024-03-06T07:23:55.529142+0000","last_undegraded":"2024-03-06T07:23:55.529142+0000","last_fullsized":"2024-03-06T07:23:55.529142+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:06:25.203342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603522+0000","last_change":"2024-03-06T07:23:54.536692+0000","last_active":"2024-03-06T07:23:55.603522+0000","last_peered":"2024-03-06T07:23:55.603522+0000","last_clean":"2024-03-06T07:23:55.603522+0000","last_became_active":"2024-03-06T07:23:54.536377+0000","last_became_peered":"2024-03-06T07:23:54.536377+0000","last_unstale":"2024-03-06T07:23:55.603522+0000","last_undegraded":"2024-03-06T07:23:55.603522+0000","last_fullsized":"2024-03-06T07:23:55.603522+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:46:39.084730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"108'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604209+0000","last_change":"2024-03-06T07:23:53.529661+0000","last_active":"2024-03-06T07:23:55.604209+0000","last_peered":"2024-03-06T07:23:55.604209+0000","last_clean":"2024-03-06T07:23:55.604209+0000","last_became_active":"2024-03-06T07:23:53.529379+0000","last_became_peered":"2024-03-06T07:23:53.529379+0000","last_unstale":"2024-03-06T07:23:55.604209+0000","last_undegraded":"2024-03-06T07:23:55.604209+0000","last_fullsized":"2024-03-06T07:23:55.604209+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:26.088856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603994+0000","last_change":"2024-03-06T07:23:54.533950+0000","last_active":"2024-03-06T07:23:55.603994+0000","last_peered":"2024-03-06T07:23:55.603994+0000","last_clean":"2024-03-06T07:23:55.603994+0000","last_became_active":"2024-03-06T07:23:54.533678+0000","last_became_peered":"2024-03-06T07:23:54.533678+0000","last_unstale":"2024-03-06T07:23:55.603994+0000","last_undegraded":"2024-03-06T07:23:55.603994+0000","last_fullsized":"2024-03-06T07:23:55.603994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:43.722095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603869+0000","last_change":"2024-03-06T07:23:53.529086+0000","last_active":"2024-03-06T07:23:55.603869+0000","last_peered":"2024-03-06T07:23:55.603869+0000","last_clean":"2024-03-06T07:23:55.603869+0000","last_became_active":"2024-03-06T07:23:53.528727+0000","last_became_peered":"2024-03-06T07:23:53.528727+0000","last_unstale":"2024-03-06T07:23:55.603869+0000","last_undegraded":"2024-03-06T07:23:55.603869+0000","last_fullsized":"2024-03-06T07:23:55.603869+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:23:51.065848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603823+0000","last_change":"2024-03-06T07:23:54.531828+0000","last_active":"2024-03-06T07:23:55.603823+0000","last_peered":"2024-03-06T07:23:55.603823+0000","last_clean":"2024-03-06T07:23:55.603823+0000","last_became_active":"2024-03-06T07:23:54.531530+0000","last_became_peered":"2024-03-06T07:23:54.531530+0000","last_unstale":"2024-03-06T07:23:55.603823+0000","last_undegraded":"2024-03-06T07:23:55.603823+0000","last_fullsized":"2024-03-06T07:23:55.603823+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:01:20.040910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602647+0000","last_change":"2024-03-06T07:23:53.520721+0000","last_active":"2024-03-06T07:23:55.602647+0000","last_peered":"2024-03-06T07:23:55.602647+0000","last_clean":"2024-03-06T07:23:55.602647+0000","last_became_active":"2024-03-06T07:23:53.520457+0000","last_became_peered":"2024-03-06T07:23:53.520457+0000","last_unstale":"2024-03-06T07:23:55.602647+0000","last_undegraded":"2024-03-06T07:23:55.602647+0000","last_fullsized":"2024-03-06T07:23:55.602647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:00.608814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603177+0000","last_change":"2024-03-06T07:23:54.532922+0000","last_active":"2024-03-06T07:23:55.603177+0000","last_peered":"2024-03-06T07:23:55.603177+0000","last_clean":"2024-03-06T07:23:55.603177+0000","last_became_active":"2024-03-06T07:23:54.531965+0000","last_became_peered":"2024-03-06T07:23:54.531965+0000","last_unstale":"2024-03-06T07:23:55.603177+0000","last_undegraded":"2024-03-06T07:23:55.603177+0000","last_fullsized":"2024-03-06T07:23:55.603177+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:40.648191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603511+0000","last_change":"2024-03-06T07:23:53.522227+0000","last_active":"2024-03-06T07:23:55.603511+0000","last_peered":"2024-03-06T07:23:55.603511+0000","last_clean":"2024-03-06T07:23:55.603511+0000","last_became_active":"2024-03-06T07:23:53.521963+0000","last_became_peered":"2024-03-06T07:23:53.521963+0000","last_unstale":"2024-03-06T07:23:55.603511+0000","last_undegraded":"2024-03-06T07:23:55.603511+0000","last_fullsized":"2024-03-06T07:23:55.603511+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:06.286897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603975+0000","last_change":"2024-03-06T07:23:54.527266+0000","last_active":"2024-03-06T07:23:55.603975+0000","last_peered":"2024-03-06T07:23:55.603975+0000","last_clean":"2024-03-06T07:23:55.603975+0000","last_became_active":"2024-03-06T07:23:54.526952+0000","last_became_peered":"2024-03-06T07:23:54.526952+0000","last_unstale":"2024-03-06T07:23:55.603975+0000","last_undegraded":"2024-03-06T07:23:55.603975+0000","last_fullsized":"2024-03-06T07:23:55.603975+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:00.922090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603504+0000","last_change":"2024-03-06T07:23:53.529962+0000","last_active":"2024-03-06T07:23:55.603504+0000","last_peered":"2024-03-06T07:23:55.603504+0000","last_clean":"2024-03-06T07:23:55.603504+0000","last_became_active":"2024-03-06T07:23:53.529658+0000","last_became_peered":"2024-03-06T07:23:53.529658+0000","last_unstale":"2024-03-06T07:23:55.603504+0000","last_undegraded":"2024-03-06T07:23:55.603504+0000","last_fullsized":"2024-03-06T07:23:55.603504+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:54.746607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603198+0000","last_change":"2024-03-06T07:23:54.530750+0000","last_active":"2024-03-06T07:23:55.603198+0000","last_peered":"2024-03-06T07:23:55.603198+0000","last_clean":"2024-03-06T07:23:55.603198+0000","last_became_active":"2024-03-06T07:23:54.530443+0000","last_became_peered":"2024-03-06T07:23:54.530443+0000","last_unstale":"2024-03-06T07:23:55.603198+0000","last_undegraded":"2024-03-06T07:23:55.603198+0000","last_fullsized":"2024-03-06T07:23:55.603198+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:29.823163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602926+0000","last_change":"2024-03-06T07:23:53.525298+0000","last_active":"2024-03-06T07:23:55.602926+0000","last_peered":"2024-03-06T07:23:55.602926+0000","last_clean":"2024-03-06T07:23:55.602926+0000","last_became_active":"2024-03-06T07:23:53.524598+0000","last_became_peered":"2024-03-06T07:23:53.524598+0000","last_unstale":"2024-03-06T07:23:55.602926+0000","last_undegraded":"2024-03-06T07:23:55.602926+0000","last_fullsized":"2024-03-06T07:23:55.602926+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:14:04.607814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604396+0000","last_change":"2024-03-06T07:23:54.525516+0000","last_active":"2024-03-06T07:23:55.604396+0000","last_peered":"2024-03-06T07:23:55.604396+0000","last_clean":"2024-03-06T07:23:55.604396+0000","last_became_active":"2024-03-06T07:23:54.525239+0000","last_became_peered":"2024-03-06T07:23:54.525239+0000","last_unstale":"2024-03-06T07:23:55.604396+0000","last_undegraded":"2024-03-06T07:23:55.604396+0000","last_fullsized":"2024-03-06T07:23:55.604396+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:09:38.033435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"108'2","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603096+0000","last_change":"2024-03-06T07:23:53.516520+0000","last_active":"2024-03-06T07:23:55.603096+0000","last_peered":"2024-03-06T07:23:55.603096+0000","last_clean":"2024-03-06T07:23:55.603096+0000","last_became_active":"2024-03-06T07:23:53.516253+0000","last_became_peered":"2024-03-06T07:23:53.516253+0000","last_unstale":"2024-03-06T07:23:55.603096+0000","last_undegraded":"2024-03-06T07:23:55.603096+0000","last_fullsized":"2024-03-06T07:23:55.603096+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:26:51.348229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603364+0000","last_change":"2024-03-06T07:23:54.538195+0000","last_active":"2024-03-06T07:23:55.603364+0000","last_peered":"2024-03-06T07:23:55.603364+0000","last_clean":"2024-03-06T07:23:55.603364+0000","last_became_active":"2024-03-06T07:23:54.537861+0000","last_became_peered":"2024-03-06T07:23:54.537861+0000","last_unstale":"2024-03-06T07:23:55.603364+0000","last_undegraded":"2024-03-06T07:23:55.603364+0000","last_fullsized":"2024-03-06T07:23:55.603364+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:12.705609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"109'1","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:56.520529+0000","last_change":"2024-03-06T07:23:53.520240+0000","last_active":"2024-03-06T07:23:56.520529+0000","last_peered":"2024-03-06T07:23:56.520529+0000","last_clean":"2024-03-06T07:23:56.520529+0000","last_became_active":"2024-03-06T07:23:53.519976+0000","last_became_peered":"2024-03-06T07:23:53.519976+0000","last_unstale":"2024-03-06T07:23:56.520529+0000","last_undegraded":"2024-03-06T07:23:56.520529+0000","last_fullsized":"2024-03-06T07:23:56.520529+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:01.816188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603699+0000","last_change":"2024-03-06T07:23:54.530486+0000","last_active":"2024-03-06T07:23:55.603699+0000","last_peered":"2024-03-06T07:23:55.603699+0000","last_clean":"2024-03-06T07:23:55.603699+0000","last_became_active":"2024-03-06T07:23:54.530126+0000","last_became_peered":"2024-03-06T07:23:54.530126+0000","last_unstale":"2024-03-06T07:23:55.603699+0000","last_undegraded":"2024-03-06T07:23:55.603699+0000","last_fullsized":"2024-03-06T07:23:55.603699+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:46:05.964342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.532627+0000","last_change":"2024-03-06T07:23:53.515848+0000","last_active":"2024-03-06T07:23:55.532627+0000","last_peered":"2024-03-06T07:23:55.532627+0000","last_clean":"2024-03-06T07:23:55.532627+0000","last_became_active":"2024-03-06T07:23:53.515558+0000","last_became_peered":"2024-03-06T07:23:53.515558+0000","last_unstale":"2024-03-06T07:23:55.532627+0000","last_undegraded":"2024-03-06T07:23:55.532627+0000","last_fullsized":"2024-03-06T07:23:55.532627+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:12.403487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.529093+0000","last_change":"2024-03-06T07:23:54.532805+0000","last_active":"2024-03-06T07:23:55.529093+0000","last_peered":"2024-03-06T07:23:55.529093+0000","last_clean":"2024-03-06T07:23:55.529093+0000","last_became_active":"2024-03-06T07:23:54.532354+0000","last_became_peered":"2024-03-06T07:23:54.532354+0000","last_unstale":"2024-03-06T07:23:55.529093+0000","last_undegraded":"2024-03-06T07:23:55.529093+0000","last_fullsized":"2024-03-06T07:23:55.529093+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:13.160969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604478+0000","last_change":"2024-03-06T07:23:53.525174+0000","last_active":"2024-03-06T07:23:55.604478+0000","last_peered":"2024-03-06T07:23:55.604478+0000","last_clean":"2024-03-06T07:23:55.604478+0000","last_became_active":"2024-03-06T07:23:53.524911+0000","last_became_peered":"2024-03-06T07:23:53.524911+0000","last_unstale":"2024-03-06T07:23:55.604478+0000","last_undegraded":"2024-03-06T07:23:55.604478+0000","last_fullsized":"2024-03-06T07:23:55.604478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:03.343482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604016+0000","last_change":"2024-03-06T07:23:54.530786+0000","last_active":"2024-03-06T07:23:55.604016+0000","last_peered":"2024-03-06T07:23:55.604016+0000","last_clean":"2024-03-06T07:23:55.604016+0000","last_became_active":"2024-03-06T07:23:54.529882+0000","last_became_peered":"2024-03-06T07:23:54.529882+0000","last_unstale":"2024-03-06T07:23:55.604016+0000","last_undegraded":"2024-03-06T07:23:55.604016+0000","last_fullsized":"2024-03-06T07:23:55.604016+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:54:59.301002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533336+0000","last_change":"2024-03-06T07:23:53.524692+0000","last_active":"2024-03-06T07:23:55.533336+0000","last_peered":"2024-03-06T07:23:55.533336+0000","last_clean":"2024-03-06T07:23:55.533336+0000","last_became_active":"2024-03-06T07:23:53.524065+0000","last_became_peered":"2024-03-06T07:23:53.524065+0000","last_unstale":"2024-03-06T07:23:55.533336+0000","last_undegraded":"2024-03-06T07:23:55.533336+0000","last_fullsized":"2024-03-06T07:23:55.533336+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:50:43.169621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603323+0000","last_change":"2024-03-06T07:23:54.526729+0000","last_active":"2024-03-06T07:23:55.603323+0000","last_peered":"2024-03-06T07:23:55.603323+0000","last_clean":"2024-03-06T07:23:55.603323+0000","last_became_active":"2024-03-06T07:23:54.526451+0000","last_became_peered":"2024-03-06T07:23:54.526451+0000","last_unstale":"2024-03-06T07:23:55.603323+0000","last_undegraded":"2024-03-06T07:23:55.603323+0000","last_fullsized":"2024-03-06T07:23:55.603323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:47:29.125635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.533488+0000","last_change":"2024-03-06T07:23:53.525414+0000","last_active":"2024-03-06T07:23:55.533488+0000","last_peered":"2024-03-06T07:23:55.533488+0000","last_clean":"2024-03-06T07:23:55.533488+0000","last_became_active":"2024-03-06T07:23:53.525136+0000","last_became_peered":"2024-03-06T07:23:53.525136+0000","last_unstale":"2024-03-06T07:23:55.533488+0000","last_undegraded":"2024-03-06T07:23:55.533488+0000","last_fullsized":"2024-03-06T07:23:55.533488+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:52:40.521118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604039+0000","last_change":"2024-03-06T07:23:54.529519+0000","last_active":"2024-03-06T07:23:55.604039+0000","last_peered":"2024-03-06T07:23:55.604039+0000","last_clean":"2024-03-06T07:23:55.604039+0000","last_became_active":"2024-03-06T07:23:54.529218+0000","last_became_peered":"2024-03-06T07:23:54.529218+0000","last_unstale":"2024-03-06T07:23:55.604039+0000","last_undegraded":"2024-03-06T07:23:55.604039+0000","last_fullsized":"2024-03-06T07:23:55.604039+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:39:13.906979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604084+0000","last_change":"2024-03-06T07:23:53.525343+0000","last_active":"2024-03-06T07:23:55.604084+0000","last_peered":"2024-03-06T07:23:55.604084+0000","last_clean":"2024-03-06T07:23:55.604084+0000","last_became_active":"2024-03-06T07:23:53.525010+0000","last_became_peered":"2024-03-06T07:23:53.525010+0000","last_unstale":"2024-03-06T07:23:55.604084+0000","last_undegraded":"2024-03-06T07:23:55.604084+0000","last_fullsized":"2024-03-06T07:23:55.604084+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:20:51.036735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603318+0000","last_change":"2024-03-06T07:23:54.536217+0000","last_active":"2024-03-06T07:23:55.603318+0000","last_peered":"2024-03-06T07:23:55.603318+0000","last_clean":"2024-03-06T07:23:55.603318+0000","last_became_active":"2024-03-06T07:23:54.535943+0000","last_became_peered":"2024-03-06T07:23:54.535943+0000","last_unstale":"2024-03-06T07:23:55.603318+0000","last_undegraded":"2024-03-06T07:23:55.603318+0000","last_fullsized":"2024-03-06T07:23:55.603318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:17:53.191394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602936+0000","last_change":"2024-03-06T07:23:53.524633+0000","last_active":"2024-03-06T07:23:55.602936+0000","last_peered":"2024-03-06T07:23:55.602936+0000","last_clean":"2024-03-06T07:23:55.602936+0000","last_became_active":"2024-03-06T07:23:53.524208+0000","last_became_peered":"2024-03-06T07:23:53.524208+0000","last_unstale":"2024-03-06T07:23:55.602936+0000","last_undegraded":"2024-03-06T07:23:55.602936+0000","last_fullsized":"2024-03-06T07:23:55.602936+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:00:39.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.604024+0000","last_change":"2024-03-06T07:23:54.535792+0000","last_active":"2024-03-06T07:23:55.604024+0000","last_peered":"2024-03-06T07:23:55.604024+0000","last_clean":"2024-03-06T07:23:55.604024+0000","last_became_active":"2024-03-06T07:23:54.535473+0000","last_became_peered":"2024-03-06T07:23:54.535473+0000","last_unstale":"2024-03-06T07:23:55.604024+0000","last_undegraded":"2024-03-06T07:23:55.604024+0000","last_fullsized":"2024-03-06T07:23:55.604024+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:53.502868+0000","last_clean_scrub_stamp":"2024-03-06T07:23:53.502868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:01:57.778369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"108'8","reported_seq":22,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.602487+0000","last_change":"2024-03-06T07:23:53.523825+0000","last_active":"2024-03-06T07:23:55.602487+0000","last_peered":"2024-03-06T07:23:55.602487+0000","last_clean":"2024-03-06T07:23:55.602487+0000","last_became_active":"2024-03-06T07:23:53.523542+0000","last_became_peered":"2024-03-06T07:23:53.523542+0000","last_unstale":"2024-03-06T07:23:55.602487+0000","last_undegraded":"2024-03-06T07:23:55.602487+0000","last_fullsized":"2024-03-06T07:23:55.602487+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:23:52.497870+0000","last_clean_scrub_stamp":"2024-03-06T07:23:52.497870+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:28.818165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":335,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-03-06T07:23:55.603143+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:23:55.603143+0000","last_peered":"2024-03-06T07:23:55.603143+0000","last_clean":"2024-03-06T07:23:55.603143+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:23:55.603143+0000","last_undegraded":"2024-03-06T07:23:55.603143+0000","last_fullsized":"2024-03-06T07:23:55.603143+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":1249795,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":659233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28360,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343480,"statfs":{"total":96636764160,"available":96607723520,"internally_reserved":0,"allocated":1368064,"data_stored":658811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":660263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":660297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1359872,"data_stored":658651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":658673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1429504,"data_stored":1249041,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:23:57.874 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:23:57.875 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:23:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.048+0000 7f472a9a1640 1 Processor -- start 2024-03-06T07:23:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.048+0000 7f472a9a1640 1 -- start start 2024-03-06T07:23:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f472a9a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f472a9a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4724156060 con 0x7f472406d750 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f472a9a1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47240a8430 con 0x7f472406db30 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f472a9a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4724155b30 con 0x7f47240c3a70 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4723fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f472406d750 0x7f4724072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41184/0 (socket says 172.21.15.17:41184) 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4723fff640 1 -- 172.21.15.17:0/213336992 learned_addr learned my addr 172.21.15.17:0/213336992 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f47237fe640 1 --2- 172.21.15.17:0/213336992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218611273 0 0) 0x7f4724156060 con 0x7f472406d750 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470c003520 con 0x7f472406d750 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404356499 0 0) 0x7f47240a8430 con 0x7f472406db30 2024-03-06T07:23:58.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4724156060 con 0x7f472406db30 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1322728144 0 0) 0x7f470c003520 con 0x7f472406d750 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f47237fe640 1 -- 172.21.15.17:0/213336992 >> v1:172.21.15.163:6789/0 conn(0x7f472406db30 legacy=0x7f47240c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f47237fe640 1 -- 172.21.15.17:0/213336992 >> v1:172.21.15.17:6789/0 conn(0x7f472406d750 legacy=0x7f4724072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.049+0000 7f47237fe640 1 -- 172.21.15.17:0/213336992 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724155d80 con 0x7f47240c3a70 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.050+0000 7f47237fe640 1 --2- 172.21.15.17:0/213336992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47240732a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47180097d0 tx=0x7f471802b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=802c6e4fc94ce07f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.050+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718037070 con 0x7f47240c3a70 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.050+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718030510 con 0x7f47240c3a70 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.050+0000 7f4722ffd640 1 -- 172.21.15.17:0/213336992 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714001670 con 0x7f47240c3a70 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 -- 172.21.15.17:0/213336992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 msgr2=0x7f47240732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 --2- 172.21.15.17:0/213336992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47240732a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47180097d0 tx=0x7f471802b6e0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 -- 172.21.15.17:0/213336992 shutdown_connections 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 --2- 172.21.15.17:0/213336992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47240732a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 -- 172.21.15.17:0/213336992 >> 172.21.15.17:0/213336992 conn(0x7f47240c5560 msgr2=0x7f47240c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 -- 172.21.15.17:0/213336992 shutdown_connections 2024-03-06T07:23:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.051+0000 7f472a9a1640 1 -- 172.21.15.17:0/213336992 wait complete. 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.052+0000 7f472a9a1640 1 Processor -- start 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.052+0000 7f472a9a1640 1 -- start start 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472406d750 0x7f4724153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47241edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c2720 con 0x7f472406db30 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c25a0 con 0x7f472406d750 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f472a9a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240a7cd0 con 0x7f47240c3a70 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56948/0 (socket says 172.21.15.17:56948) 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 -- 172.21.15.17:0/2715752250 learned_addr learned my addr 172.21.15.17:0/2715752250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f4723fff640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472406d750 0x7f4724153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f4728f3e640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47241edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 -- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 msgr2=0x7f47241edf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47241edf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 -- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472406d750 msgr2=0x7f4724153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472406d750 0x7f4724153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 -- 172.21.15.17:0/2715752250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724152f20 con 0x7f472406db30 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.053+0000 7f47237fe640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f47180098f0 tx=0x7f47180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.054+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718040020 con 0x7f472406db30 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.054+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4718038040 con 0x7f472406db30 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.054+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718037070 con 0x7f472406db30 2024-03-06T07:23:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.054+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241531e0 con 0x7f472406db30 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.054+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472406fa10 con 0x7f472406db30 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.055+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404f330 con 0x7f472406db30 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.055+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4718043020 con 0x7f472406db30 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.055+0000 7f4720ff9640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 0x7f471006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.056+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47180ab580 con 0x7f472406db30 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.056+0000 7f4723fff640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 0x7f471006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.056+0000 7f4723fff640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 0x7f471006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f47140011e0 tx=0x7f4714001710 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.060+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f471804a030 con 0x7f472406db30 2024-03-06T07:23:58.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.307+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4724155400 con 0x7f472406db30 2024-03-06T07:23:58.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.309+0000 7f4720ff9640 1 -- 172.21.15.17:0/2715752250 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4718079cb0 con 0x7f472406db30 2024-03-06T07:23:58.312 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:23:58.313 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:23:58.313 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:23:58.313 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:23:58.313 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:23:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:23:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 msgr2=0x7f471006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 0x7f471006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f47140011e0 tx=0x7f4714001710 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 msgr2=0x7f4724154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f47180098f0 tx=0x7f47180046c0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 shutdown_connections 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f471006b4d0 0x7f471006d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47240c3a70 0x7f47241edf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f472406db30 0x7f4724154ad0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 --2- 172.21.15.17:0/2715752250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f472406d750 0x7f4724153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 >> 172.21.15.17:0/2715752250 conn(0x7f47240c5560 msgr2=0x7f47240c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 shutdown_connections 2024-03-06T07:23:58.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.311+0000 7f472a9a1640 1 -- 172.21.15.17:0/2715752250 wait complete. 2024-03-06T07:23:58.324 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:58.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.424+0000 7f5777c66640 1 Processor -- start 2024-03-06T07:23:58.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.425+0000 7f5777c66640 1 -- start start 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.425+0000 7f5777c66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5770156ea0 0x7f57700c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.425+0000 7f5777c66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700ad310 con 0x7f57701506b0 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.425+0000 7f5777c66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57701572a0 con 0x7f5770153990 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.425+0000 7f5777c66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5770156450 con 0x7f5770156ea0 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5776203640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5770156ea0 0x7f57700c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5775a02640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57701506b0 0x7f57700702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41186/0 (socket says 172.21.15.17:41186) 2024-03-06T07:23:58.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5775a02640 1 -- 172.21.15.17:0/594565778 learned_addr learned my addr 172.21.15.17:0/594565778 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:58.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5776203640 1 -- 172.21.15.17:0/594565778 >> v1:172.21.15.163:6789/0 conn(0x7f5770153990 legacy=0x7f57700acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5776203640 1 -- 172.21.15.17:0/594565778 >> v1:172.21.15.17:6789/0 conn(0x7f57701506b0 legacy=0x7f57700702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5774a00640 1 -- 172.21.15.17:0/594565778 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109722876 0 0) 0x7f57700ad310 con 0x7f57701506b0 2024-03-06T07:23:58.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.426+0000 7f5776203640 1 -- 172.21.15.17:0/594565778 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770070aa0 con 0x7f5770156ea0 2024-03-06T07:23:58.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.427+0000 7f5776203640 1 --2- 172.21.15.17:0/594565778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770156ea0 0x7f57700c54a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5e0035e34b12b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.427+0000 7f5774a00640 1 -- 172.21.15.17:0/594565778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768039070 con 0x7f5770156ea0 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.427+0000 7f5774a00640 1 -- 172.21.15.17:0/594565778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576802cd00 con 0x7f5770156ea0 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.427+0000 7f5774a00640 1 -- 172.21.15.17:0/594565778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768034660 con 0x7f5770156ea0 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 -- 172.21.15.17:0/594565778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770156ea0 msgr2=0x7f57700c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 --2- 172.21.15.17:0/594565778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770156ea0 0x7f57700c54a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5768009d90 tx=0x7f576802bf50 comp rx=0 tx=0).stop 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 -- 172.21.15.17:0/594565778 shutdown_connections 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 --2- 172.21.15.17:0/594565778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770156ea0 0x7f57700c54a0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 -- 172.21.15.17:0/594565778 >> 172.21.15.17:0/594565778 conn(0x7f577006b1a0 msgr2=0x7f577006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 -- 172.21.15.17:0/594565778 shutdown_connections 2024-03-06T07:23:58.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.428+0000 7f5777c66640 1 -- 172.21.15.17:0/594565778 wait complete. 2024-03-06T07:23:58.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 Processor -- start 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 -- start start 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 0x7f57701eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 0x7f57701eab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701566a0 con 0x7f5770153990 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770156a10 con 0x7f57701506b0 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.429+0000 7f5777c66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700c59e0 con 0x7f5770156ea0 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56952/0 (socket says 172.21.15.17:56952) 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 -- 172.21.15.17:0/3601769732 learned_addr learned my addr 172.21.15.17:0/3601769732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775a02640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 0x7f57701eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5776203640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 0x7f57701eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 -- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 msgr2=0x7f57701eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 0x7f57701eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 -- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 msgr2=0x7f57701eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 0x7f57701eeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 -- 172.21.15.17:0/3601769732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770149ce0 con 0x7f5770153990 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775a02640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 0x7f57701eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5776203640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 0x7f57701eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5775201640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f576400c450 tx=0x7f576400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640052d0 con 0x7f5770153990 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5764005470 con 0x7f5770153990 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764012870 con 0x7f5770153990 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770147860 con 0x7f5770153990 2024-03-06T07:23:58.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.430+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57700c22a0 con 0x7f5770153990 2024-03-06T07:23:58.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.431+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577004f330 con 0x7f5770153990 2024-03-06T07:23:58.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.432+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5764012a10 con 0x7f5770153990 2024-03-06T07:23:58.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.432+0000 7f57627fc640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 0x7f575406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:58.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.432+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f576408a920 con 0x7f5770153990 2024-03-06T07:23:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.433+0000 7f5775a02640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 0x7f575406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.433+0000 7f5775a02640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 0x7f575406d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f576c000f60 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.436+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5770149ce0 con 0x7f5770153990 2024-03-06T07:23:58.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.771+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57700700a0 con 0x7f5770153990 2024-03-06T07:23:58.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.771+0000 7f57627fc640 1 -- 172.21.15.17:0/3601769732 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4976 (secure 0 0 0) 0x7f5764059780 con 0x7f5770153990 2024-03-06T07:23:58.774 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:58.774 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:23:55.881609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75092648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":75092648},{"type":"v1","addr":"172.21.15.17:6835","nonce":75092648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389]},"id":8}]} 2024-03-06T07:23:58.774 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 89 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 msgr2=0x7f575406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 0x7f575406d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f576c000f60 tx=0x7f576c009040 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 msgr2=0x7f57700c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f576400c450 tx=0x7f576400c910 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 shutdown_connections 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f575406b4d0 0x7f575406d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5770156ea0 0x7f57701eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5770153990 0x7f57700c1d60 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 --2- 172.21.15.17:0/3601769732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57701506b0 0x7f57701eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 >> 172.21.15.17:0/3601769732 conn(0x7f577006b1a0 msgr2=0x7f57700acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 shutdown_connections 2024-03-06T07:23:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:58.774+0000 7f5777c66640 1 -- 172.21.15.17:0/3601769732 wait complete. 2024-03-06T07:23:59.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:53.884146+0000', 'modified': '2024-03-06T07:23:55.881609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7389}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7389': {'gid': 7389, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75092648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 75092648}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75092648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7389, 'qdb_cluster': [7389]} 2024-03-06T07:23:59.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:23:59.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:23:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.293+0000 7f3130da0640 1 Processor -- start 2024-03-06T07:23:59.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.293+0000 7f3130da0640 1 -- start start 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3130da0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f312c0c8950 0x7f312c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3130da0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c155130 con 0x7f312c0c69f0 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3130da0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0ad480 con 0x7f312c1558d0 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3130da0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c155440 con 0x7f312c0c8950 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f312a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f312c0c69f0 0x7f312c0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41190/0 (socket says 172.21.15.17:41190) 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f312a59c640 1 -- 172.21.15.17:0/3136791846 learned_addr learned my addr 172.21.15.17:0/3136791846 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3136791846 >> v2:172.21.15.163:3300/0 conn(0x7f312c0c8950 0x7f312c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313110115 0 0) 0x7f311c001110 con 0x7f312c0c69f0 2024-03-06T07:23:59.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.294+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3118003520 con 0x7f312c0c69f0 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2824525781 0 0) 0x7f3118003520 con 0x7f312c0c69f0 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 >> v2:172.21.15.163:3300/0 conn(0x7f312c0c8950 msgr2=0x7f312c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 --2- 172.21.15.17:0/3136791846 >> v2:172.21.15.163:3300/0 conn(0x7f312c0c8950 0x7f312c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 >> v1:172.21.15.163:6789/0 conn(0x7f312c1558d0 legacy=0x7f312c15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0c5d50 con 0x7f312c0c69f0 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f311c001840 con 0x7f312c0c69f0 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 >> v1:172.21.15.17:6789/0 conn(0x7f312c0c69f0 legacy=0x7f312c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31180039b0 0x7f3118003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31180043b0 0x7f3118004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f311800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c155440 con 0x7f3118005120 2024-03-06T07:23:59.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311800c060 con 0x7f31180043b0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311800c260 con 0x7f31180039b0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f311800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31180043b0 0x7f3118004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312a59c640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31180039b0 0x7f3118003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 -- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31180039b0 msgr2=0x7f3118003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31180039b0 0x7f3118003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 -- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31180043b0 msgr2=0x7f3118004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31180043b0 0x7f3118004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.295+0000 7f312ad9d640 1 -- 172.21.15.17:0/3136791846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118003520 con 0x7f3118005120 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.296+0000 7f312ad9d640 1 -- 172.21.15.17:0/3136791846 reap_dead start 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.296+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f311800b9c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3124001f60 tx=0x7f3118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0aa3ece2f658f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.296+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124001b30 con 0x7f3118005120 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.296+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312402d6f0 con 0x7f3118005120 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.296+0000 7f3129d9b640 1 -- 172.21.15.17:0/3136791846 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312402dae0 con 0x7f3118005120 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 -- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 msgr2=0x7f311800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f311800b9c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f3124001f60 tx=0x7f3118005830 comp rx=0 tx=0).stop 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 -- 172.21.15.17:0/3136791846 shutdown_connections 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 --2- 172.21.15.17:0/3136791846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f311800b9c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 -- 172.21.15.17:0/3136791846 >> 172.21.15.17:0/3136791846 conn(0x7f312c0cae70 msgr2=0x7f312c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 -- 172.21.15.17:0/3136791846 shutdown_connections 2024-03-06T07:23:59.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.297+0000 7f3130da0640 1 -- 172.21.15.17:0/3136791846 wait complete. 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.298+0000 7f3130da0640 1 Processor -- start 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.298+0000 7f3130da0640 1 -- start start 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f312c157ad0 0x7f312c157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f312c158490 0x7f312c1480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c152a20 con 0x7f3118005120 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0c6310 con 0x7f312c157ad0 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3130da0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c152ba0 con 0x7f312c158490 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56964/0 (socket says 172.21.15.17:56964) 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 -- 172.21.15.17:0/405374228 learned_addr learned my addr 172.21.15.17:0/405374228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f3121d9b640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f312c157ad0 0x7f312c157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312ad9d640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f312c158490 0x7f312c1480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 -- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f312c158490 msgr2=0x7f312c1480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f312c158490 0x7f312c1480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 -- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f312c157ad0 msgr2=0x7f312c157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f312c157ad0 0x7f312c157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.299+0000 7f312a59c640 1 -- 172.21.15.17:0/405374228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c1ee0d0 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f312a59c640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f311c0072f0 tx=0x7f311c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f31237fe640 1 -- 172.21.15.17:0/405374228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c001810 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f31237fe640 1 -- 172.21.15.17:0/405374228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c018070 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c1ee3b0 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f31237fe640 1 -- 172.21.15.17:0/405374228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00fb90 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.300+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c1ee970 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.301+0000 7f31237fe640 1 -- 172.21.15.17:0/405374228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f311c00fd30 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.302+0000 7f31237fe640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.302+0000 7f31237fe640 1 -- 172.21.15.17:0/405374228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f311c08a410 con 0x7f3118005120 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.302+0000 7f3121d9b640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.302+0000 7f3121d9b640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f312c0bfbc0 tx=0x7f3114008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.303+0000 7f3133982740 1 Processor -- start 2024-03-06T07:23:59.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.303+0000 7f3133982740 1 -- start start 2024-03-06T07:23:59.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddba86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba874b0 0x559ddba83680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddb9ed6b0 con 0x559ddba88f50 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddb996a60 con 0x559ddba89330 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.304+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ddb9e9750 con 0x559ddba874b0 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56976/0 (socket says 172.21.15.17:56976) 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 -- 172.21.15.17:0/3549628028 learned_addr learned my addr 172.21.15.17:0/3549628028 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba874b0 0x559ddba83680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddba86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 -- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba874b0 msgr2=0x559ddba83680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba874b0 0x559ddba83680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 -- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 msgr2=0x559ddba86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddba86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 -- 172.21.15.17:0/3549628028 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ddb9b4f30 con 0x559ddba88f50 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddba86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f312a59c640 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f311c060be0 tx=0x7f311c007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127a775e827c316c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f310bfff640 1 -- 172.21.15.17:0/3549628028 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c084070 con 0x559ddba88f50 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f310bfff640 1 -- 172.21.15.17:0/3549628028 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c061110 con 0x559ddba88f50 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.305+0000 7f310bfff640 1 -- 172.21.15.17:0/3549628028 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c054fe0 con 0x559ddba88f50 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- 172.21.15.17:0/3549628028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 msgr2=0x559ddba86a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f311c060be0 tx=0x7f311c007ac0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- 172.21.15.17:0/3549628028 shutdown_connections 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba874b0 0x559ddba83680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddba86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 --2- 172.21.15.17:0/3549628028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba86a30 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- 172.21.15.17:0/3549628028 >> 172.21.15.17:0/3549628028 conn(0x559ddba857d0 msgr2=0x559ddbaaa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- 172.21.15.17:0/3549628028 shutdown_connections 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- 172.21.15.17:0/3549628028 wait complete. 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.306+0000 7f3133982740 1 -- start start 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 Processor -- start 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba88f50 0x559ddbac4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba89330 0x559ddbaabb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbaab230 con 0x559ddbaacac0 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddb9a9ea0 con 0x559ddba88f50 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ddbaaac70 con 0x559ddba89330 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.307+0000 7f3133982740 10 client.0 _ll_register_callbacks cb 0x559ddb9a9870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56980/0 (socket says 172.21.15.17:56980) 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 -- 172.21.15.17:0/3226904418 learned_addr learned my addr 172.21.15.17:0/3226904418 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba89330 0x559ddbaabb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312a59c640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba88f50 0x559ddbac4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 -- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba89330 msgr2=0x559ddbaabb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba89330 0x559ddbaabb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 -- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba88f50 msgr2=0x559ddbac4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba88f50 0x559ddbac4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 -- 172.21.15.17:0/3226904418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ddb9aa270 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3124002050 tx=0x7f312402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24088584c6ca0e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f31097fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312402de30 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3124006b50 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f311c061110 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f312404d050 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f3133982740 10 client.7533 fetch_fsmap learned FSMap version 89 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.308+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x559ddb7022d0 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c061110 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f3124050020 con 0x559ddbaacac0 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 10 client.7533 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:23:59.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 10 client.7533 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 20 client.7533 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbaa6680 0x559ddbae4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 4 client.7533 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 --> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ddb9fabf0 con 0x559ddbaa6680 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f312a59c640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbaa6680 0x559ddbae4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f31097fa640 10 client.7533 ms_handle_connect on v2:172.21.15.17:6833/75092648 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.309+0000 7f312a59c640 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbaa6680 0x559ddbae4440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d99efa674222bb4 server_cookie=e2975480efffe934 in_seq=0 out_seq=0 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f31097fa640 1 -- 172.21.15.17:0/3226904418 <== mds.0 v2:172.21.15.17:6833/75092648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559ddb9fabf0 con 0x559ddbaa6680 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f31097fa640 10 client.7533 handle_command_reply: tid=0 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 client.7533 shutdown 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 msgr2=0x559ddbaa5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3124002050 tx=0x7f312402e000 comp rx=0 tx=0).stop 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 shutdown_connections 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbaa6680 0x559ddbae4440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddbaacac0 0x559ddbaa5fc0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f31097fa640 0 client.7533 ms_handle_reset on v2:172.21.15.17:6833/75092648 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba89330 0x559ddbaabb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 --2- 172.21.15.17:0/3226904418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba88f50 0x559ddbac4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 >> 172.21.15.17:0/3226904418 conn(0x559ddba874b0 msgr2=0x559ddba82510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 shutdown_connections 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 1 -- 172.21.15.17:0/3226904418 wait complete. 2024-03-06T07:23:59.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.310+0000 7f3133982740 20 client.7533 trim_cache size 0 max 16384 2024-03-06T07:23:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 Processor -- start 2024-03-06T07:23:59.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 -- start start 2024-03-06T07:23:59.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbad2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba86ae0 0x559ddbad2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.327+0000 7f3133982740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c1f74a0 con 0x559ddba88f50 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c1f68a0 con 0x559ddba89330 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c1f6ea0 con 0x559ddba86ae0 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56982/0 (socket says 172.21.15.17:56982) 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 -- 172.21.15.17:0/2408158853 learned_addr learned my addr 172.21.15.17:0/2408158853 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 -- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba86ae0 msgr2=0x559ddbad2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba86ae0 0x559ddbad2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 -- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 msgr2=0x559ddbad2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbad2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 -- 172.21.15.17:0/2408158853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ddb7cc6b0 con 0x559ddba88f50 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f312a59c640 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f311c0078e0 tx=0x7f311c089000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f040fb6a3373b337 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f310b7fe640 1 -- 172.21.15.17:0/2408158853 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c084040 con 0x559ddba88f50 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f310b7fe640 1 -- 172.21.15.17:0/2408158853 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c061110 con 0x559ddba88f50 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.328+0000 7f310b7fe640 1 -- 172.21.15.17:0/2408158853 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c054c10 con 0x559ddba88f50 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- 172.21.15.17:0/2408158853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 msgr2=0x559ddba865a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f311c0078e0 tx=0x7f311c089000 comp rx=0 tx=0).stop 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- 172.21.15.17:0/2408158853 shutdown_connections 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba86ae0 0x559ddbad2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbad2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 --2- 172.21.15.17:0/2408158853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba88f50 0x559ddba865a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- 172.21.15.17:0/2408158853 >> 172.21.15.17:0/2408158853 conn(0x559ddba874b0 msgr2=0x559ddbac4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- 172.21.15.17:0/2408158853 shutdown_connections 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- 172.21.15.17:0/2408158853 wait complete. 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 -- start start 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.329+0000 7f3133982740 1 Processor -- start 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba88f50 0x559ddbac1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbb38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbaa6500 con 0x559ddba86ae0 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ddbaac580 con 0x559ddba89330 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ddbad8da0 con 0x559ddba88f50 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3133982740 10 client.0 _ll_register_callbacks cb 0x7f3124001db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f312a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56994/0 (socket says 172.21.15.17:56994) 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f312a59c640 1 -- 172.21.15.17:0/3091114064 learned_addr learned my addr 172.21.15.17:0/3091114064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba88f50 0x559ddbac1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.330+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbb38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 -- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba88f50 msgr2=0x559ddbac1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba88f50 0x559ddbac1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 -- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 msgr2=0x559ddbb38a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbb38a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 -- 172.21.15.17:0/3091114064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ddbac23c0 con 0x559ddba86ae0 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312ad9d640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbb38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f312a59c640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f311c0072c0 tx=0x7f311c001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41976c7d72817dd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:23:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c084040 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f311c08a0d0 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f311c061110 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f311c0584b0 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 10 client.7539 fetch_fsmap learned FSMap version 89 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x559ddbaa3750 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c061110 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f311c054fe0 con 0x559ddba86ae0 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 10 client.7539 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 10 client.7539 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 20 client.7539 populate_metadata read hostname 'smithi017' 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbb32ab0 0x559ddbae8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 4 client.7539 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.331+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 --> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559ddb9f4f40 con 0x559ddbb32ab0 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbb32ab0 0x559ddbae8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3121d9b640 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbb32ab0 0x559ddbae8460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969b83dc9d705944 server_cookie=3ca31dbb7f5e4c32 in_seq=0 out_seq=0 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f31097fa640 10 client.7539 ms_handle_connect on v2:172.21.15.17:6833/75092648 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f31097fa640 1 -- 172.21.15.17:0/3091114064 <== mds.0 v2:172.21.15.17:6833/75092648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x559ddb9f4f40 con 0x559ddbb32ab0 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f31097fa640 10 client.7539 handle_command_reply: tid=0 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 client.7539 shutdown 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 msgr2=0x559ddbaa2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f311c0072c0 tx=0x7f311c001c10 comp rx=0 tx=0).stop 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 shutdown_connections 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x559ddbb32ab0 0x559ddbae8460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x559ddba89330 0x559ddbb38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x559ddba88f50 0x559ddbac1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f31097fa640 0 client.7539 ms_handle_reset on v2:172.21.15.17:6833/75092648 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 --2- 172.21.15.17:0/3091114064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ddba86ae0 0x559ddbaa2fb0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 >> 172.21.15.17:0/3091114064 conn(0x559ddba874b0 msgr2=0x559ddba82510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 shutdown_connections 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 1 -- 172.21.15.17:0/3091114064 wait complete. 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.332+0000 7f3133982740 20 client.7539 trim_cache size 0 max 16384 2024-03-06T07:23:59.344 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7389, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.8237807080000001, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 89, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 109, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 109, 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9.8921207550000005 2024-03-06T07:23:59.345 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 msgr2=0x7f310c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 0x7f310c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f312c0bfbc0 tx=0x7f3114008040 comp rx=0 tx=0).stop 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 msgr2=0x7f312c0ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f311c0072f0 tx=0x7f311c0077b0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 shutdown_connections 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f310c06b4d0 0x7f310c06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f312c158490 0x7f312c1480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f312c157ad0 0x7f312c157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 --2- 172.21.15.17:0/405374228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3118005120 0x7f312c0ab7d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 >> 172.21.15.17:0/405374228 conn(0x7f312c0cae70 msgr2=0x7f312c0c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 shutdown_connections 2024-03-06T07:23:59.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.333+0000 7f3130da0640 1 -- 172.21.15.17:0/405374228 wait complete. 2024-03-06T07:23:59.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:23:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.451+0000 7f722ecf4640 1 Processor -- start 2024-03-06T07:23:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 -- start start 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f722806d9f0 0x7f72280a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72280c3280 0x7f72280c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7228158800 con 0x7f72281551e0 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7228158ae0 con 0x7f722806d9f0 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ecf4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7228158dc0 con 0x7f72280c3280 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722ca90640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f722806d9f0 0x7f72280a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722d291640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f72281551e0 0x7f72281555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41202/0 (socket says 172.21.15.17:41202) 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.452+0000 7f722d291640 1 -- 172.21.15.17:0/967288027 learned_addr learned my addr 172.21.15.17:0/967288027 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f7227fff640 1 --2- 172.21.15.17:0/967288027 >> v2:172.21.15.163:3300/0 conn(0x7f72280c3280 0x7f72280c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f722ca90640 1 -- 172.21.15.17:0/967288027 >> v2:172.21.15.163:3300/0 conn(0x7f72280c3280 msgr2=0x7f72280c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f722ca90640 1 --2- 172.21.15.17:0/967288027 >> v2:172.21.15.163:3300/0 conn(0x7f72280c3280 0x7f72280c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f722ca90640 1 -- 172.21.15.17:0/967288027 >> v1:172.21.15.17:6789/0 conn(0x7f72281551e0 legacy=0x7f72281555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f72277fe640 1 -- 172.21.15.17:0/967288027 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639559119 0 0) 0x7f7228158800 con 0x7f72281551e0 2024-03-06T07:23:59.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.453+0000 7f722ca90640 1 -- 172.21.15.17:0/967288027 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72281590a0 con 0x7f722806d9f0 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ca90640 1 --2- 172.21.15.17:0/967288027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f722806d9f0 0x7f72280a7cd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f72280721a0 tx=0x7f721802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df54d493775062b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f72277fe640 1 -- 172.21.15.17:0/967288027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180315a0 con 0x7f722806d9f0 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f72277fe640 1 -- 172.21.15.17:0/967288027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218037070 con 0x7f722806d9f0 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f72277fe640 1 -- 172.21.15.17:0/967288027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218031960 con 0x7f722806d9f0 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 -- 172.21.15.17:0/967288027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f722806d9f0 msgr2=0x7f72280a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 --2- 172.21.15.17:0/967288027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f722806d9f0 0x7f72280a7cd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f72280721a0 tx=0x7f721802b840 comp rx=0 tx=0).stop 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 -- 172.21.15.17:0/967288027 shutdown_connections 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 --2- 172.21.15.17:0/967288027 >> v2:172.21.15.163:3300/0 conn(0x7f72280c3280 0x7f72280c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 --2- 172.21.15.17:0/967288027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f722806d9f0 0x7f72280a7cd0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.454+0000 7f722ecf4640 1 -- 172.21.15.17:0/967288027 >> 172.21.15.17:0/967288027 conn(0x7f72280aa630 msgr2=0x7f72280aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.455+0000 7f722ecf4640 1 -- 172.21.15.17:0/967288027 shutdown_connections 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.455+0000 7f722ecf4640 1 -- 172.21.15.17:0/967288027 wait complete. 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 Processor -- start 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 -- start start 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 0x7f72280a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72280c3280 0x7f722806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722806df50 con 0x7f72281551e0 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722806ddd0 con 0x7f722806d9f0 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.456+0000 7f722ecf4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72280a8210 con 0x7f72280c3280 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57018/0 (socket says 172.21.15.17:57018) 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 -- 172.21.15.17:0/64246340 learned_addr learned my addr 172.21.15.17:0/64246340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f7227fff640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72280c3280 0x7f722806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722ca90640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 0x7f72280a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 -- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72280c3280 msgr2=0x7f722806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72280c3280 0x7f722806e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 -- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 msgr2=0x7f72280a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 0x7f72280a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 -- 172.21.15.17:0/64246340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72281468f0 con 0x7f72281551e0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722ca90640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 0x7f72280a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722d291640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f722000c780 tx=0x7f722000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7220018070 con 0x7f72281551e0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722000ce30 con 0x7f72281551e0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7228146bd0 con 0x7f72281551e0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722000a840 con 0x7f72281551e0 2024-03-06T07:23:59.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.457+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72280a9d40 con 0x7f72281551e0 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.459+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722806e0d0 con 0x7f72281551e0 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.459+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f722001b070 con 0x7f72281551e0 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.459+0000 7f72257fa640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 0x7f721406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.459+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72200897e0 con 0x7f72281551e0 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.459+0000 7f722ca90640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 0x7f721406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.460+0000 7f722ca90640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 0x7f721406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7218036000 tx=0x7f721803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.463+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f722806e0d0 con 0x7f72281551e0 2024-03-06T07:23:59.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.798+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f722804f330 con 0x7f72281551e0 2024-03-06T07:23:59.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.798+0000 7f72257fa640 1 -- 172.21.15.17:0/64246340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4976 (secure 0 0 0) 0x7f7220057f10 con 0x7f72281551e0 2024-03-06T07:23:59.800 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:23:59.800 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:23:55.881609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75092648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":75092648},{"type":"v1","addr":"172.21.15.17:6835","nonce":75092648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389]},"id":8}]} 2024-03-06T07:23:59.800 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 89 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 msgr2=0x7f721406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 0x7f721406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7218036000 tx=0x7f721803d040 comp rx=0 tx=0).stop 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 msgr2=0x7f722806eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f722000c780 tx=0x7f722000cc40 comp rx=0 tx=0).stop 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 shutdown_connections 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721406b500 0x7f721406d9c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72281551e0 0x7f722806eaf0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72280c3280 0x7f722806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 --2- 172.21.15.17:0/64246340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f722806d9f0 0x7f72280a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 >> 172.21.15.17:0/64246340 conn(0x7f72280aa630 msgr2=0x7f72280c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 shutdown_connections 2024-03-06T07:23:59.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.800+0000 7f722ecf4640 1 -- 172.21.15.17:0/64246340 wait complete. 2024-03-06T07:23:59.806 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:23:59.805+0000 7fc4c32e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:23:59.813 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:53.884146+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7389": { "addr": "172.21.15.17:6835/75092648", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 75092648, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 75092648, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 87, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:23:55.881609+0000", "qdb_cluster": [ 7389 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1364627789", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1364627789, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1364627789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6203, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/386947795", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 386947795, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 386947795, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7021, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2673518972", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2673518972, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2673518972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7386, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:23:59.813 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:23:59.813 INFO:tasks.cephfs.test_failover:current = [{'gid': 7389, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75092648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 75092648}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75092648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7021, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 7386, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2673518972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2673518972}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2673518972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 6203, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:23:59.814 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs 2024-03-06T07:23:59.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.916+0000 7f84f6e4e640 1 Processor -- start 2024-03-06T07:23:59.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.916+0000 7f84f6e4e640 1 -- start start 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f6e4e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84f00a7cd0 0x7f84f00c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f6e4e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f0155f40 con 0x7f84f0152880 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f6e4e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f0072760 con 0x7f84f00c6710 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f6e4e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f0072510 con 0x7f84f00a7cd0 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f53eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f84f00a7cd0 0x7f84f00c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f4bea640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f84f0152880 0x7f84f00c3210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41206/0 (socket says 172.21.15.17:41206) 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f4bea640 1 -- 172.21.15.17:0/2517329363 learned_addr learned my addr 172.21.15.17:0/2517329363 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f53eb640 1 -- 172.21.15.17:0/2517329363 >> v1:172.21.15.163:6789/0 conn(0x7f84f00c6710 legacy=0x7f84f00c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f53eb640 1 -- 172.21.15.17:0/2517329363 >> v1:172.21.15.17:6789/0 conn(0x7f84f0152880 legacy=0x7f84f00c3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84f53eb640 1 -- 172.21.15.17:0/2517329363 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0072a40 con 0x7f84f00a7cd0 2024-03-06T07:23:59.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.917+0000 7f84ef7fe640 1 -- 172.21.15.17:0/2517329363 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697939295 0 0) 0x7f84f0155f40 con 0x7f84f0152880 2024-03-06T07:23:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.918+0000 7f84f53eb640 1 --2- 172.21.15.17:0/2517329363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00a7cd0 0x7f84f00c2590 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ed824e00595288 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.918+0000 7f84ef7fe640 1 -- 172.21.15.17:0/2517329363 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0039070 con 0x7f84f00a7cd0 2024-03-06T07:23:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.918+0000 7f84ef7fe640 1 -- 172.21.15.17:0/2517329363 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e002cbd0 con 0x7f84f00a7cd0 2024-03-06T07:23:59.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.918+0000 7f84ef7fe640 1 -- 172.21.15.17:0/2517329363 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0034440 con 0x7f84f00a7cd0 2024-03-06T07:23:59.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2517329363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00a7cd0 msgr2=0x7f84f00c2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2517329363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00a7cd0 0x7f84f00c2590 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e002be20 comp rx=0 tx=0).stop 2024-03-06T07:23:59.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2517329363 shutdown_connections 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2517329363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00a7cd0 0x7f84f00c2590 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2517329363 >> 172.21.15.17:0/2517329363 conn(0x7f84f00aa5c0 msgr2=0x7f84f00aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2517329363 shutdown_connections 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.919+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2517329363 wait complete. 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.920+0000 7f84f6e4e640 1 Processor -- start 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.920+0000 7f84f6e4e640 1 -- start start 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f00a7cd0 0x7f84f00ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f0152880 0x7f84f0146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00c3920 con 0x7f84f00c6710 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0072db0 con 0x7f84f0152880 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f6e4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00c3aa0 con 0x7f84f00a7cd0 2024-03-06T07:23:59.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57044/0 (socket says 172.21.15.17:57044) 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 -- 172.21.15.17:0/2386703668 learned_addr learned my addr 172.21.15.17:0/2386703668 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f53eb640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f0152880 0x7f84f0146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84f4bea640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f00a7cd0 0x7f84f00ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 -- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f00a7cd0 msgr2=0x7f84f00ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f00a7cd0 0x7f84f00ab920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 -- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f0152880 msgr2=0x7f84f0146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f0152880 0x7f84f0146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 -- 172.21.15.17:0/2386703668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f01ebe60 con 0x7f84f00c6710 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.921+0000 7f84effff640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f84e400b3f0 tx=0x7f84e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.922+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4015020 con 0x7f84f00c6710 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.922+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e4011830 con 0x7f84f00c6710 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.922+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f01540d0 con 0x7f84f00c6710 2024-03-06T07:23:59.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.922+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011c20 con 0x7f84f00c6710 2024-03-06T07:23:59.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.922+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00abe60 con 0x7f84f00c6710 2024-03-06T07:23:59.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.923+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f00a94e0 con 0x7f84f00c6710 2024-03-06T07:23:59.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.923+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84e4018020 con 0x7f84f00c6710 2024-03-06T07:23:59.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.923+0000 7f84ed7fa640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 0x7f84dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:23:59.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.924+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84e4089f00 con 0x7f84f00c6710 2024-03-06T07:23:59.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.924+0000 7f84f4bea640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 0x7f84dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:23:59.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.924+0000 7f84f4bea640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 0x7f84dc06d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84e8000f60 tx=0x7f84e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:23:59.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:23:59.932+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f84f00a94e0 con 0x7f84f00c6710 2024-03-06T07:24:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.113+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f84f004f330 con 0x7f84f00c6710 2024-03-06T07:24:00.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.116+0000 7f84ed7fa640 1 -- 172.21.15.17:0/2386703668 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v36)=0 v36) v1 ==== 116+0+0 (secure 0 0 0) 0x7f84e401f030 con 0x7f84f00c6710 2024-03-06T07:24:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.118+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 msgr2=0x7f84dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.118+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 0x7f84dc06d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84e8000f60 tx=0x7f84e8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.118+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 msgr2=0x7f84f0146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.118+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f84e400b3f0 tx=0x7f84e400b8b0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 shutdown_connections 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f84dc06b4d0 0x7f84dc06d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f0152880 0x7f84f0146a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f00c6710 0x7f84f0146530 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 --2- 172.21.15.17:0/2386703668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f00a7cd0 0x7f84f00ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 >> 172.21.15.17:0/2386703668 conn(0x7f84f00aa5c0 msgr2=0x7f84f00a9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 shutdown_connections 2024-03-06T07:24:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.119+0000 7f84f6e4e640 1 -- 172.21.15.17:0/2386703668 wait complete. 2024-03-06T07:24:00.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:00.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.234+0000 7fc2f765c640 1 Processor -- start 2024-03-06T07:24:00.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.234+0000 7fc2f765c640 1 -- start start 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.234+0000 7fc2f765c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.234+0000 7fc2f765c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2f00c7180 0x7fc2f00c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.234+0000 7fc2f765c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00c7c60 con 0x7fc2f0155b20 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f765c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f0159640 con 0x7fc2f00c7180 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f765c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c8df0 con 0x7fc2f01527b0 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49354/0 (socket says 172.21.15.17:49354) 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f4bf7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2f00c7180 0x7fc2f00c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 -- 172.21.15.17:0/1333898616 learned_addr learned my addr 172.21.15.17:0/1333898616 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 -- 172.21.15.17:0/1333898616 >> v2:172.21.15.163:3300/0 conn(0x7fc2f00c7180 msgr2=0x7fc2f00c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/1333898616 >> v2:172.21.15.163:3300/0 conn(0x7fc2f00c7180 0x7fc2f00c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 -- 172.21.15.17:0/1333898616 >> v1:172.21.15.163:6789/0 conn(0x7fc2f0155b20 legacy=0x7fc2f00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.235+0000 7fc2f53f8640 1 -- 172.21.15.17:0/1333898616 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00c8060 con 0x7fc2f01527b0 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.236+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/1333898616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009870 tx=0x7fc2ec02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09d6ab51b1f4044 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.236+0000 7fc2e7fff640 1 -- 172.21.15.17:0/1333898616 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec035020 con 0x7fc2f01527b0 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.236+0000 7fc2e7fff640 1 -- 172.21.15.17:0/1333898616 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec031640 con 0x7fc2f01527b0 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.236+0000 7fc2e7fff640 1 -- 172.21.15.17:0/1333898616 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec02d040 con 0x7fc2f01527b0 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 -- 172.21.15.17:0/1333898616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 msgr2=0x7fc2f0159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 --2- 172.21.15.17:0/1333898616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009870 tx=0x7fc2ec02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 -- 172.21.15.17:0/1333898616 shutdown_connections 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 --2- 172.21.15.17:0/1333898616 >> v2:172.21.15.163:3300/0 conn(0x7fc2f00c7180 0x7fc2f00c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 --2- 172.21.15.17:0/1333898616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0159100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 -- 172.21.15.17:0/1333898616 >> 172.21.15.17:0/1333898616 conn(0x7fc2f00ac560 msgr2=0x7fc2f00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 -- 172.21.15.17:0/1333898616 shutdown_connections 2024-03-06T07:24:00.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.237+0000 7fc2f765c640 1 -- 172.21.15.17:0/1333898616 wait complete. 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.238+0000 7fc2f765c640 1 Processor -- start 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.238+0000 7fc2f765c640 1 -- start start 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 0x7fc2f0145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0155320 con 0x7fc2f0155b20 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c7980 con 0x7fc2f00c7180 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f765c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0155600 con 0x7fc2f01527b0 2024-03-06T07:24:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57054/0 (socket says 172.21.15.17:57054) 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 -- 172.21.15.17:0/939808684 learned_addr learned my addr 172.21.15.17:0/939808684 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f4bf7640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 0x7fc2f0145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 -- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 msgr2=0x7fc2f0157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 -- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 msgr2=0x7fc2f0145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 0x7fc2f0145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 -- 172.21.15.17:0/939808684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0157e50 con 0x7fc2f0155b20 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 0x7fc2f0145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.239+0000 7fc2f5bf9640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c6b0 tx=0x7fc2e800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.240+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8003b70 con 0x7fc2f0155b20 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.240+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e8003d30 con 0x7fc2f0155b20 2024-03-06T07:24:00.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.240+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0158320 con 0x7fc2f0155b20 2024-03-06T07:24:00.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.240+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8005310 con 0x7fc2f0155b20 2024-03-06T07:24:00.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.240+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f01585c0 con 0x7fc2f0155b20 2024-03-06T07:24:00.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.241+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f004f330 con 0x7fc2f0155b20 2024-03-06T07:24:00.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.241+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2e80054b0 con 0x7fc2f0155b20 2024-03-06T07:24:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.242+0000 7fc2e5ffb640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 0x7fc2d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.242+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2e808a100 con 0x7fc2f0155b20 2024-03-06T07:24:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.242+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 0x7fc2d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.242+0000 7fc2f53f8640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 0x7fc2d406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec036000 tx=0x7fc2ec03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.246+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc2f00c7c60 con 0x7fc2f0155b20 2024-03-06T07:24:00.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.466+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2f006ce00 con 0x7fc2f0155b20 2024-03-06T07:24:00.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.467+0000 7fc2e5ffb640 1 -- 172.21.15.17:0/939808684 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4976 (secure 0 0 0) 0x7fc2e8058e80 con 0x7fc2f0155b20 2024-03-06T07:24:00.469 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:00.469 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":89,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:23:55.881609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/75092648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":75092648},{"type":"v1","addr":"172.21.15.17:6835","nonce":75092648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389]},"id":8}]} 2024-03-06T07:24:00.469 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 89 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 msgr2=0x7fc2d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 0x7fc2d406d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec036000 tx=0x7fc2ec03c040 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 msgr2=0x7fc2f00c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800c6b0 tx=0x7fc2e800cb70 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 shutdown_connections 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2d406b4d0 0x7fc2d406d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2f0155b20 0x7fc2f00c1a80 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2f01527b0 0x7fc2f0157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 --2- 172.21.15.17:0/939808684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2f00c7180 0x7fc2f0145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 >> 172.21.15.17:0/939808684 conn(0x7fc2f00ac560 msgr2=0x7fc2f00aa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 shutdown_connections 2024-03-06T07:24:00.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.469+0000 7fc2f765c640 1 -- 172.21.15.17:0/939808684 wait complete. 2024-03-06T07:24:00.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:53.884146+0000', 'modified': '2024-03-06T07:23:55.881609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7389}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7389': {'gid': 7389, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75092648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 75092648}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75092648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7389, 'qdb_cluster': [7389]} 2024-03-06T07:24:00.884 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:00.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:24:00.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.986+0000 7f8cb1476640 1 Processor -- start 2024-03-06T07:24:00.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.986+0000 7f8cb1476640 1 -- start start 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cb1476640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cac0c8f20 0x7f8cac15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cb1476640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac0a8390 con 0x7f8cac0c7170 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cb1476640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac0730f0 con 0x7f8cac06ff10 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cb1476640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0c7620 con 0x7f8cac0c8f20 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cac0c8f20 0x7f8cac15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cac0c8f20 0x7f8cac15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33104/0 (socket says 172.21.15.17:33104) 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8ca3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8cac0c7170 0x7f8cac0c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52732/0 (socket says 172.21.15.17:52732) 2024-03-06T07:24:00.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 -- 172.21.15.17:0/1035013967 learned_addr learned my addr 172.21.15.17:0/1035013967 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 -- 172.21.15.17:0/1035013967 >> v1:172.21.15.163:6789/0 conn(0x7f8cac06ff10 legacy=0x7f8cac153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 -- 172.21.15.17:0/1035013967 >> v1:172.21.15.17:6789/0 conn(0x7f8cac0c7170 legacy=0x7f8cac0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.987+0000 7f8cab7fe640 1 -- 172.21.15.17:0/1035013967 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0733d0 con 0x7f8cac0c8f20 2024-03-06T07:24:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.988+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/1035013967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac0c8f20 0x7f8cac15ebb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c77a0 tx=0x7f8ca402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d159181b1cc7fb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.988+0000 7f8caa7fc640 1 -- 172.21.15.17:0/1035013967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca402c9e0 con 0x7f8cac0c8f20 2024-03-06T07:24:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.988+0000 7f8caa7fc640 1 -- 172.21.15.17:0/1035013967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca402cb80 con 0x7f8cac0c8f20 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.988+0000 7f8caa7fc640 1 -- 172.21.15.17:0/1035013967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40343f0 con 0x7f8cac0c8f20 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 -- 172.21.15.17:0/1035013967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac0c8f20 msgr2=0x7f8cac15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 --2- 172.21.15.17:0/1035013967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac0c8f20 0x7f8cac15ebb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c77a0 tx=0x7f8ca402be40 comp rx=0 tx=0).stop 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 -- 172.21.15.17:0/1035013967 shutdown_connections 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 --2- 172.21.15.17:0/1035013967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac0c8f20 0x7f8cac15ebb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 -- 172.21.15.17:0/1035013967 >> 172.21.15.17:0/1035013967 conn(0x7f8cac06b970 msgr2=0x7f8cac0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 -- 172.21.15.17:0/1035013967 shutdown_connections 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.989+0000 7f8cb1476640 1 -- 172.21.15.17:0/1035013967 wait complete. 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 Processor -- start 2024-03-06T07:24:00.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 -- start start 2024-03-06T07:24:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 0x7f8cac0ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 0x7f8cac154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1544b0 con 0x7f8cac06ff10 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac154330 con 0x7f8cac0c8f20 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.990+0000 7f8cb1476640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0702f0 con 0x7f8cac0c7170 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33106/0 (socket says 172.21.15.17:33106) 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 -- 172.21.15.17:0/2696786700 learned_addr learned my addr 172.21.15.17:0/2696786700 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 0x7f8cac154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 0x7f8cac0ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 -- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 msgr2=0x7f8cac0ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 0x7f8cac0ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 -- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 msgr2=0x7f8cac154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 0x7f8cac154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:00.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 -- 172.21.15.17:0/2696786700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac151e80 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 0x7f8cac0ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 0x7f8cac154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8caaffd640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c430 tx=0x7f8c9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8ca37fe640 1 -- 172.21.15.17:0/2696786700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011790 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8ca37fe640 1 -- 172.21.15.17:0/2696786700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c017070 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8ca37fe640 1 -- 172.21.15.17:0/2696786700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c011bb0 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac1e97b0 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.991+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0bdc20 con 0x7f8cac06ff10 2024-03-06T07:24:00.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.993+0000 7f8ca37fe640 1 -- 172.21.15.17:0/2696786700 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c9c01c460 con 0x7f8cac06ff10 2024-03-06T07:24:01.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.993+0000 7f8ca37fe640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 0x7f8c8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.993+0000 7f8ca37fe640 1 -- 172.21.15.17:0/2696786700 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c9c08a190 con 0x7f8cac06ff10 2024-03-06T07:24:01.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.994+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 0x7f8c8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.994+0000 7f8cb405d740 1 Processor -- start 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.994+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 0x7f8c8c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.994+0000 7f8cb405d740 1 -- start start 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2e4b0 0x55830ff2a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830fe946b0 con 0x55830ff30330 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830fe3da60 con 0x55830ff2ff50 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55830fe90750 con 0x55830ff2e4b0 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33118/0 (socket says 172.21.15.17:33118) 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 -- 172.21.15.17:0/900526917 learned_addr learned my addr 172.21.15.17:0/900526917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8caaffd640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2e4b0 0x55830ff2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 -- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2e4b0 msgr2=0x55830ff2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2e4b0 0x55830ff2a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 -- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 msgr2=0x55830ff2da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff2da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8ca3fff640 1 -- 172.21.15.17:0/900526917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55830fe5bf30 con 0x55830ff30330 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.995+0000 7f8caaffd640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff2da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8c94005fd0 tx=0x7f8c94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8fdba7b2ff4739 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8ca0ff9640 1 -- 172.21.15.17:0/900526917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94015110 con 0x55830ff30330 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8ca0ff9640 1 -- 172.21.15.17:0/900526917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94015990 con 0x55830ff30330 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8ca0ff9640 1 -- 172.21.15.17:0/900526917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94015d40 con 0x55830ff30330 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 -- 172.21.15.17:0/900526917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 msgr2=0x55830ff2df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8c94005fd0 tx=0x7f8c94012000 comp rx=0 tx=0).stop 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 -- 172.21.15.17:0/900526917 shutdown_connections 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2e4b0 0x55830ff2a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff2df70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 --2- 172.21.15.17:0/900526917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff2da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.996+0000 7f8cb405d740 1 -- 172.21.15.17:0/900526917 >> 172.21.15.17:0/900526917 conn(0x55830ff2c7d0 msgr2=0x55830ff513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.997+0000 7f8cb405d740 1 -- 172.21.15.17:0/900526917 shutdown_connections 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.997+0000 7f8cb405d740 1 -- 172.21.15.17:0/900526917 wait complete. 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.997+0000 7f8cb405d740 1 -- start start 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.997+0000 7f8cb405d740 1 Processor -- start 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ff684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830ff52230 con 0x55830ff30330 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830fe50ea0 con 0x55830ff2ff50 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55830ff51c70 con 0x55830ff69780 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cb405d740 10 client.0 _ll_register_callbacks cb 0x55830fe50870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33132/0 (socket says 172.21.15.17:33132) 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 -- 172.21.15.17:0/3097134755 learned_addr learned my addr 172.21.15.17:0/3097134755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8caaffd640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ff684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 -- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 msgr2=0x55830ff684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ff684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 -- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 msgr2=0x55830ff507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.998+0000 7f8ca3fff640 1 -- 172.21.15.17:0/3097134755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55830fe51270 con 0x55830ff30330 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8c94015270 tx=0x7f8c94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaea4787ea8f838 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94016de0 con 0x55830ff30330 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c94010070 con 0x55830ff30330 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c94015990 con 0x55830ff30330 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c94034050 con 0x55830ff30330 2024-03-06T07:24:01.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 10 client.7575 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55830fba92d0 con 0x55830ff30330 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c94015990 con 0x55830ff30330 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8c94037020 con 0x55830ff30330 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 10 client.7575 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 10 client.7575 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 20 client.7575 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff52f30 0x55830ff8b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 4 client.7575 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 --> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55830fea1bf0 con 0x55830ff52f30 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:00.999+0000 7f8caaffd640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff52f30 0x55830ff8b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.000+0000 7f8c89ffb640 10 client.7575 ms_handle_connect on v2:172.21.15.17:6833/75092648 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.000+0000 7f8caaffd640 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff52f30 0x55830ff8b4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbc6b312f55252a server_cookie=c8798e5f27012296 in_seq=0 out_seq=0 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.000+0000 7f8c89ffb640 1 -- 172.21.15.17:0/3097134755 <== mds.0 v2:172.21.15.17:6833/75092648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55830fea1bf0 con 0x55830ff52f30 2024-03-06T07:24:01.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.000+0000 7f8c89ffb640 10 client.7575 handle_command_reply: tid=0 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 client.7575 shutdown 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 msgr2=0x55830ff7c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8c94015270 tx=0x7f8c94000bc0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 shutdown_connections 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff52f30 0x55830ff8b4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ff684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff30330 0x55830ff7c810 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8c89ffb640 0 client.7575 ms_handle_reset on v2:172.21.15.17:6833/75092648 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 --2- 172.21.15.17:0/3097134755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff2ff50 0x55830ff507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 >> 172.21.15.17:0/3097134755 conn(0x55830ff2e4b0 msgr2=0x55830ff2f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 shutdown_connections 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 1 -- 172.21.15.17:0/3097134755 wait complete. 2024-03-06T07:24:01.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.001+0000 7f8cb405d740 20 client.7575 trim_cache size 0 max 16384 2024-03-06T07:24:01.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.017+0000 7f8cb405d740 1 Processor -- start 2024-03-06T07:24:01.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 -- start start 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2ff50 0x55830ff7fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ff79a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1f4b80 con 0x55830ff69780 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1eca70 con 0x55830ff30330 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac1f4580 con 0x55830ff2ff50 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33148/0 (socket says 172.21.15.17:33148) 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8cab7fe640 1 -- 172.21.15.17:0/357329113 learned_addr learned my addr 172.21.15.17:0/357329113 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8caaffd640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2ff50 0x55830ff7fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.018+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ff79a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 -- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2ff50 msgr2=0x55830ff7fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2ff50 0x55830ff7fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 -- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 msgr2=0x55830ff79a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ff79a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 -- 172.21.15.17:0/357329113 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55830fc736b0 con 0x55830ff69780 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ff79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009c90 tx=0x7f8ca402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fa617a3844f9e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8c8bfff640 1 -- 172.21.15.17:0/357329113 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca404c070 con 0x55830ff69780 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8c8bfff640 1 -- 172.21.15.17:0/357329113 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca402c9d0 con 0x55830ff69780 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8c8bfff640 1 -- 172.21.15.17:0/357329113 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4047650 con 0x55830ff69780 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cb405d740 1 -- 172.21.15.17:0/357329113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 msgr2=0x55830ff79f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.019+0000 7f8cb405d740 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009c90 tx=0x7f8ca402e000 comp rx=0 tx=0).stop 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 -- 172.21.15.17:0/357329113 shutdown_connections 2024-03-06T07:24:01.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff69780 0x55830ff79f70 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ff79a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 --2- 172.21.15.17:0/357329113 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff2ff50 0x55830ff7fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 -- 172.21.15.17:0/357329113 >> 172.21.15.17:0/357329113 conn(0x55830ff2e4b0 msgr2=0x55830ff31100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 -- 172.21.15.17:0/357329113 shutdown_connections 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 -- 172.21.15.17:0/357329113 wait complete. 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 -- start start 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.020+0000 7f8cb405d740 1 Processor -- start 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ffe42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ffde040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830ff68eb0 con 0x55830ff2ff50 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830ff57c50 con 0x55830ff30330 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55830ff7cda0 con 0x55830ff69780 2024-03-06T07:24:01.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cb405d740 10 client.0 _ll_register_callbacks cb 0x7f8c940075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33152/0 (socket says 172.21.15.17:33152) 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 -- 172.21.15.17:0/529079622 learned_addr learned my addr 172.21.15.17:0/529079622 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8cab7fe640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ffde040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ffe42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 -- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 msgr2=0x55830ffde040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ffde040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 -- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 msgr2=0x55830ffe42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ffe42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 -- 172.21.15.17:0/529079622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55830ff546e0 con 0x55830ff2ff50 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ffe42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8caaffd640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0885c0 tx=0x7f8c9c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e435050e98ed9de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.021+0000 7f8c89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c075060 con 0x55830ff2ff50 2024-03-06T07:24:01.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c9c054e80 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ca402c9d0 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9c0882c0 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 10 client.7581 fetch_fsmap learned FSMap version 89 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55830ffe4b50 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca402c9d0 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 89) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f8c9c070190 con 0x55830ff2ff50 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 10 client.7581 fetch_fsmap finished waiting for FSMap version 89 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 10 client.7581 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 20 client.7581 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff4d9a0 0x55830ff4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 4 client.7581 mds_command: new command op to 7389 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 --> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55830fe9bf40 con 0x55830ff4d9a0 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff4d9a0 0x55830ff4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:01.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8c89ffb640 10 client.7581 ms_handle_connect on v2:172.21.15.17:6833/75092648 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.022+0000 7f8ca3fff640 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff4d9a0 0x55830ff4dd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b544182841b4b2 server_cookie=fbabba091fda7789 in_seq=0 out_seq=0 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8c89ffb640 1 -- 172.21.15.17:0/529079622 <== mds.0 v2:172.21.15.17:6833/75092648 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55830fe9bf40 con 0x55830ff4d9a0 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8c89ffb640 10 client.7581 handle_command_reply: tid=0 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 client.7581 shutdown 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 msgr2=0x55830ff53cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0885c0 tx=0x7f8c9c01e040 comp rx=0 tx=0).stop 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 shutdown_connections 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:6833/75092648,v1:172.21.15.17:6835/75092648] conn(0x55830ff4d9a0 0x55830ff4dd80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55830ff69780 0x55830ffde040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8c89ffb640 0 client.7581 ms_handle_reset on v2:172.21.15.17:6833/75092648 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55830ff30330 0x55830ffe42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 --2- 172.21.15.17:0/529079622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55830ff2ff50 0x55830ff53cb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 >> 172.21.15.17:0/529079622 conn(0x55830ff2e4b0 msgr2=0x55830ff51c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 shutdown_connections 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 1 -- 172.21.15.17:0/529079622 wait complete. 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb405d740 20 client.7581 trim_cache size 0 max 16384 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7389, 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:01.033 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 6.514318684, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 89, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 109, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 109, 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11.582659917999999 2024-03-06T07:24:01.034 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 msgr2=0x7f8c8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 0x7f8c8c06d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94007d60 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 msgr2=0x7f8cac149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c430 tx=0x7f8c9c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 shutdown_connections 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c8c06b500 0x7f8c8c06d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cac0c8f20 0x7f8cac154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cac0c7170 0x7f8cac0ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 --2- 172.21.15.17:0/2696786700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cac06ff10 0x7f8cac149630 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.023+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 >> 172.21.15.17:0/2696786700 conn(0x7f8cac06b970 msgr2=0x7f8cac0aa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.024+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 shutdown_connections 2024-03-06T07:24:01.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:01.024+0000 7f8cb1476640 1 -- 172.21.15.17:0/2696786700 wait complete. 2024-03-06T07:24:01.036 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:53.884146+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7389": { "addr": "172.21.15.17:6835/75092648", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 75092648, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 75092648, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 87, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:23:55.881609+0000", "qdb_cluster": [ 7389 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1364627789", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1364627789, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1364627789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6203, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/386947795", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 386947795, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 386947795, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7021, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2673518972", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2673518972, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2673518972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7386, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:01.037 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:24:01.037 INFO:tasks.cephfs.test_failover:current = [{'gid': 7389, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/75092648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 75092648}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 75092648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7021, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 7386, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2673518972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2673518972}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2673518972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 6203, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}] 2024-03-06T07:24:01.038 DEBUG:tasks.cephfs.test_failover:8 != -1 2024-03-06T07:24:01.038 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:01.559 INFO:tasks.ceph.mds.a.smithi017.stderr: -104> 2024-03-06T07:23:59.805+0000 7fc4c32e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:01.584 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:24:04.807 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-03-06T07:24:04.805+0000 7fb307348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:06.038 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.150+0000 7fb1c3071640 1 Processor -- start 2024-03-06T07:24:06.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.150+0000 7fb1c3071640 1 -- start start 2024-03-06T07:24:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc14f360 0x7fb1bc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb1bc0c88c0 0x7fb1bc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0c5880 0x7fb1bc158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0c8dd0 con 0x7fb1bc0c5880 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0c73d0 con 0x7fb1bc0c88c0 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c3071640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0be2e0 con 0x7fb1bc14f360 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c160e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0c5880 0x7fb1bc158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c160e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb1bc0c5880 0x7fb1bc158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33186/0 (socket says 172.21.15.17:33186) 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c160e640 1 -- 172.21.15.17:0/2829267609 learned_addr learned my addr 172.21.15.17:0/2829267609 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1bbfff640 1 --2- 172.21.15.17:0/2829267609 >> v2:172.21.15.163:3300/0 conn(0x7fb1bc0c88c0 0x7fb1bc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.151+0000 7fb1c0e0d640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc14f360 0x7fb1bc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 -- 172.21.15.17:0/2829267609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc14f360 msgr2=0x7fb1bc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc14f360 0x7fb1bc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 -- 172.21.15.17:0/2829267609 >> v2:172.21.15.163:3300/0 conn(0x7fb1bc0c88c0 msgr2=0x7fb1bc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 --2- 172.21.15.17:0/2829267609 >> v2:172.21.15.163:3300/0 conn(0x7fb1bc0c88c0 0x7fb1bc0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 -- 172.21.15.17:0/2829267609 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0c7670 con 0x7fb1bc0c5880 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.152+0000 7fb1c160e640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc158b60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0cbac0 tx=0x7fb1ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da177b9e8f0f4783 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1bb7fe640 1 -- 172.21.15.17:0/2829267609 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac02e070 con 0x7fb1bc0c5880 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1bb7fe640 1 -- 172.21.15.17:0/2829267609 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac037070 con 0x7fb1bc0c5880 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1bb7fe640 1 -- 172.21.15.17:0/2829267609 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac031a00 con 0x7fb1bc0c5880 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 -- 172.21.15.17:0/2829267609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0c5880 msgr2=0x7fb1bc158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc158b60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0cbac0 tx=0x7fb1ac02b880 comp rx=0 tx=0).stop 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 -- 172.21.15.17:0/2829267609 shutdown_connections 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc158b60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 --2- 172.21.15.17:0/2829267609 >> v2:172.21.15.163:3300/0 conn(0x7fb1bc0c88c0 0x7fb1bc0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 --2- 172.21.15.17:0/2829267609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc14f360 0x7fb1bc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 -- 172.21.15.17:0/2829267609 >> 172.21.15.17:0/2829267609 conn(0x7fb1bc0ac570 msgr2=0x7fb1bc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 -- 172.21.15.17:0/2829267609 shutdown_connections 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.153+0000 7fb1c3071640 1 -- 172.21.15.17:0/2829267609 wait complete. 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.154+0000 7fb1c3071640 1 Processor -- start 2024-03-06T07:24:06.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.154+0000 7fb1c3071640 1 -- start start 2024-03-06T07:24:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc0c88c0 0x7fb1bc0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1590a0 con 0x7fb1bc14f360 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0c7c30 con 0x7fb1bc0c5880 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c3071640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc159220 con 0x7fb1bc0c88c0 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c160e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c160e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33190/0 (socket says 172.21.15.17:33190) 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c160e640 1 -- 172.21.15.17:0/1542672956 learned_addr learned my addr 172.21.15.17:0/1542672956 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1c0e0d640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.155+0000 7fb1bbfff640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc0c88c0 0x7fb1bc0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 -- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc0c88c0 msgr2=0x7fb1bc0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc0c88c0 0x7fb1bc0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 -- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1bc0c5880 msgr2=0x7fb1bc0adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc0adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 -- 172.21.15.17:0/1542672956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0afcd0 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c160e640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0090b0 tx=0x7fb1ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac032670 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac032810 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac032c00 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc1eecb0 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.156+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1512f0 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.157+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc04f330 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.158+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1ac04c060 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.158+0000 7fb1b97fa640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 0x7fb1a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.158+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1ac0abb20 con 0x7fb1bc14f360 2024-03-06T07:24:06.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.158+0000 7fb1c0e0d640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 0x7fb1a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.159+0000 7fb1c0e0d640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 0x7fb1a806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc151280 tx=0x7fb1b400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.162+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb1ac040070 con 0x7fb1bc14f360 2024-03-06T07:24:06.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.381+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1bc06cda0 con 0x7fb1bc14f360 2024-03-06T07:24:06.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.381+0000 7fb1b97fa640 1 -- 172.21.15.17:0/1542672956 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4980 (secure 0 0 0) 0x7fb1ac07a180 con 0x7fb1bc14f360 2024-03-06T07:24:06.383 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:06.383 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":94,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7587,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/617166148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":617166148},{"type":"v1","addr":"172.21.15.17:6835","nonce":617166148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:04.561622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:06.384 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 94 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 msgr2=0x7fb1a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 0x7fb1a806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc151280 tx=0x7fb1b400b040 comp rx=0 tx=0).stop 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 msgr2=0x7fb1bc0b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac0090b0 tx=0x7fb1ac0046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 shutdown_connections 2024-03-06T07:24:06.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb1a806b4d0 0x7fb1a806d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1bc14f360 0x7fb1bc0b00f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1bc0c88c0 0x7fb1bc0ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 --2- 172.21.15.17:0/1542672956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb1bc0c5880 0x7fb1bc0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 >> 172.21.15.17:0/1542672956 conn(0x7fb1bc0ac570 msgr2=0x7fb1bc0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 shutdown_connections 2024-03-06T07:24:06.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.383+0000 7fb1c3071640 1 -- 172.21.15.17:0/1542672956 wait complete. 2024-03-06T07:24:06.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:23:53.884146+0000', 'modified': '2024-03-06T07:24:04.561622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6203': {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6203, 'qdb_cluster': [6203]} 2024-03-06T07:24:06.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:06.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:24:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.902+0000 7f44c7037640 1 Processor -- start 2024-03-06T07:24:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.902+0000 7f44c7037640 1 -- start start 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c7037640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44c0072f40 0x7f44c0152060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c7037640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f44c00c21e0 0x7f44c00c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c7037640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c00c1140 con 0x7f44c006d680 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c7037640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00c6f50 con 0x7f44c0072f40 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c7037640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00c1d50 con 0x7f44c00c21e0 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44c0072f40 0x7f44c0152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44c0072f40 0x7f44c0152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33194/0 (socket says 172.21.15.17:33194) 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3124607685 learned_addr learned my addr 172.21.15.17:0/3124607685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c55d4640 1 --2- 172.21.15.17:0/3124607685 >> v2:172.21.15.163:3300/0 conn(0x7f44c00c21e0 0x7f44c00c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3124607685 >> v2:172.21.15.163:3300/0 conn(0x7f44c00c21e0 msgr2=0x7f44c00c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3124607685 >> v2:172.21.15.163:3300/0 conn(0x7f44c00c21e0 0x7f44c00c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3124607685 >> v1:172.21.15.163:6789/0 conn(0x7f44c006d680 legacy=0x7f44c00c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.903+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3124607685 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00c1420 con 0x7f44c0072f40 2024-03-06T07:24:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.904+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3124607685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0152060 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098a0 tx=0x7f44b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dc144db8033bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.904+0000 7f44bf7fe640 1 -- 172.21.15.17:0/3124607685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8035020 con 0x7f44c0072f40 2024-03-06T07:24:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.904+0000 7f44bf7fe640 1 -- 172.21.15.17:0/3124607685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b8031640 con 0x7f44c0072f40 2024-03-06T07:24:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.904+0000 7f44bf7fe640 1 -- 172.21.15.17:0/3124607685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b802d040 con 0x7f44c0072f40 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 -- 172.21.15.17:0/3124607685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 msgr2=0x7f44c0152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 --2- 172.21.15.17:0/3124607685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0152060 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098a0 tx=0x7f44b802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 -- 172.21.15.17:0/3124607685 shutdown_connections 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 --2- 172.21.15.17:0/3124607685 >> v2:172.21.15.163:3300/0 conn(0x7f44c00c21e0 0x7f44c00c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 --2- 172.21.15.17:0/3124607685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0152060 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 -- 172.21.15.17:0/3124607685 >> 172.21.15.17:0/3124607685 conn(0x7f44c006b1a0 msgr2=0x7f44c006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 -- 172.21.15.17:0/3124607685 shutdown_connections 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.905+0000 7f44c7037640 1 -- 172.21.15.17:0/3124607685 wait complete. 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.906+0000 7f44c7037640 1 Processor -- start 2024-03-06T07:24:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.906+0000 7f44c7037640 1 -- start start 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 0x7f44c01ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44c00c21e0 0x7f44c0071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00c19e0 con 0x7f44c0072f40 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c01525a0 con 0x7f44c006d680 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c7037640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c006db10 con 0x7f44c00c21e0 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 0x7f44c01ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33206/0 (socket says 172.21.15.17:33206) 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 0x7f44c01ec8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50748/0 (socket says 172.21.15.17:50748) 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 -- 172.21.15.17:0/3111002 learned_addr learned my addr 172.21.15.17:0/3111002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44c55d4640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44c00c21e0 0x7f44c0071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 -- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44c00c21e0 msgr2=0x7f44c0071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44c00c21e0 0x7f44c0071e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 -- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 msgr2=0x7f44c01ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 0x7f44c01ec8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 -- 172.21.15.17:0/3111002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c014f390 con 0x7f44c0072f40 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.907+0000 7f44bffff640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c3c0 tx=0x7f44b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.908+0000 7f44bd7fa640 1 -- 172.21.15.17:0/3111002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b40052d0 con 0x7f44c0072f40 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.908+0000 7f44bd7fa640 1 -- 172.21.15.17:0/3111002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4005490 con 0x7f44c0072f40 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.908+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c01ebdc0 con 0x7f44c0072f40 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.908+0000 7f44bd7fa640 1 -- 172.21.15.17:0/3111002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4007260 con 0x7f44c0072f40 2024-03-06T07:24:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.908+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c01ec380 con 0x7f44c0072f40 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.910+0000 7f44bd7fa640 1 -- 172.21.15.17:0/3111002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44b4007400 con 0x7f44c0072f40 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.910+0000 7f44bd7fa640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.910+0000 7f44bd7fa640 1 -- 172.21.15.17:0/3111002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44b4089800 con 0x7f44c0072f40 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.910+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 0x7f44ac06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44b8036000 tx=0x7f44b803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 Processor -- start 2024-03-06T07:24:06.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 -- start start 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b445f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 0x564b1b442680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b3ac6b0 con 0x564b1b447f50 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b355a60 con 0x564b1b4464b0 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.911+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1b3a8750 con 0x564b1b448330 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33214/0 (socket says 172.21.15.17:33214) 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3395681968 learned_addr learned my addr 172.21.15.17:0/3395681968 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44bffff640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b445f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c55d4640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 0x564b1b442680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 msgr2=0x564b1b445f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b445f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 msgr2=0x564b1b442680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 0x564b1b442680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 -- 172.21.15.17:0/3395681968 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1b373f30 con 0x564b1b447f50 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c55d4640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 0x564b1b442680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f44b80090b0 tx=0x7f44b8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2cfd788ab74311 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44a67fc640 1 -- 172.21.15.17:0/3395681968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8035040 con 0x564b1b447f50 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44a67fc640 1 -- 172.21.15.17:0/3395681968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b803a620 con 0x564b1b447f50 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.912+0000 7f44a67fc640 1 -- 172.21.15.17:0/3395681968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b803a9f0 con 0x564b1b447f50 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- 172.21.15.17:0/3395681968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 msgr2=0x564b1b445a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f44b80090b0 tx=0x7f44b8030000 comp rx=0 tx=0).stop 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- 172.21.15.17:0/3395681968 shutdown_connections 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b4464b0 0x564b1b442680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b445f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/3395681968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b445a30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- 172.21.15.17:0/3395681968 >> 172.21.15.17:0/3395681968 conn(0x564b1b4447d0 msgr2=0x564b1b4693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- 172.21.15.17:0/3395681968 shutdown_connections 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- 172.21.15.17:0/3395681968 wait complete. 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.913+0000 7f44c9c1a740 1 -- start start 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 Processor -- start 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b484210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b46dda0 0x564b1b46e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b46a230 con 0x564b1b447f50 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b368ea0 con 0x564b1b46dda0 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1b469c70 con 0x564b1b448330 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c9c1a740 10 client.0 _ll_register_callbacks cb 0x564b1b368870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33216/0 (socket says 172.21.15.17:33216) 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c4dd3640 1 -- 172.21.15.17:0/409838589 learned_addr learned my addr 172.21.15.17:0/409838589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44bffff640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b484210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.914+0000 7f44c55d4640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b46dda0 0x564b1b46e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 -- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 msgr2=0x564b1b484210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b484210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 -- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b46dda0 msgr2=0x564b1b46e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b46dda0 0x564b1b46e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 -- 172.21.15.17:0/409838589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1b369270 con 0x564b1b447f50 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f44b8000c00 tx=0x7f44b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4001dc5774bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8057410 con 0x564b1b447f50 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44b80575b0 con 0x564b1b447f50 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b803a620 con 0x564b1b447f50 2024-03-06T07:24:06.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b8064050 con 0x564b1b447f50 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 10 client.7605 fetch_fsmap learned FSMap version 94 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x564b1b0c12d0 con 0x564b1b447f50 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b803a620 con 0x564b1b447f50 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 94) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f44b8067020 con 0x564b1b447f50 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 10 client.7605 fetch_fsmap finished waiting for FSMap version 94 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 10 client.7605 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 20 client.7605 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b498390 0x564b1b498770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 4 client.7605 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.915+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b1b3b9bf0 con 0x564b1b498390 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.916+0000 7f44bffff640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b498390 0x564b1b498770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.916+0000 7f44bffff640 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b498390 0x564b1b498770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a89f3b6039753f4 server_cookie=33eb6b97fa1c9a7e in_seq=0 out_seq=0 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.916+0000 7f449f7fe640 10 client.7605 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f449f7fe640 1 -- 172.21.15.17:0/409838589 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564b1b3b9bf0 con 0x564b1b498390 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f449f7fe640 10 client.7605 handle_command_reply: tid=0 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 client.7605 shutdown 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 msgr2=0x564b1b494950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f44b8000c00 tx=0x7f44b8004730 comp rx=0 tx=0).stop 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 shutdown_connections 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b498390 0x564b1b498770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b46dda0 0x564b1b46e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f449f7fe640 0 client.7605 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b484210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/409838589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b447f50 0x564b1b494950 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 >> 172.21.15.17:0/409838589 conn(0x564b1b4464b0 msgr2=0x564b1b441510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 shutdown_connections 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 1 -- 172.21.15.17:0/409838589 wait complete. 2024-03-06T07:24:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.917+0000 7f44c9c1a740 20 client.7605 trim_cache size 0 max 16384 2024-03-06T07:24:06.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.934+0000 7f44c9c1a740 1 Processor -- start 2024-03-06T07:24:06.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.934+0000 7f44c9c1a740 1 -- start start 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.934+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b49ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.934+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c01f4d30 con 0x564b1b46dda0 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c01f4230 con 0x564b1b447f50 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c01f4730 con 0x564b1b448330 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33222/0 (socket says 172.21.15.17:33222) 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 -- 172.21.15.17:0/716609892 learned_addr learned my addr 172.21.15.17:0/716609892 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b49ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44bffff640 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 -- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 msgr2=0x564b1b4435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 -- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 msgr2=0x564b1b49ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b49ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 -- 172.21.15.17:0/716609892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b1b18b6b0 con 0x564b1b46dda0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44c55d4640 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ced06f1a73c040 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44a5ffb640 1 -- 172.21.15.17:0/716609892 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0020630 con 0x564b1b46dda0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44a5ffb640 1 -- 172.21.15.17:0/716609892 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00207f0 con 0x564b1b46dda0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.935+0000 7f44a5ffb640 1 -- 172.21.15.17:0/716609892 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0020be0 con 0x564b1b46dda0 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- 172.21.15.17:0/716609892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 msgr2=0x564b1b491a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b0009750 comp rx=0 tx=0).stop 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- 172.21.15.17:0/716609892 shutdown_connections 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b491a30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/716609892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b49ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- 172.21.15.17:0/716609892 >> 172.21.15.17:0/716609892 conn(0x564b1b4464b0 msgr2=0x564b1b4459a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- 172.21.15.17:0/716609892 shutdown_connections 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- 172.21.15.17:0/716609892 wait complete. 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.936+0000 7f44c9c1a740 1 -- start start 2024-03-06T07:24:06.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 Processor -- start 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b4f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b46a230 con 0x564b1b46dda0 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b1b484c20 con 0x564b1b447f50 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b1b46b610 con 0x564b1b448330 2024-03-06T07:24:06.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c9c1a740 10 client.0 _ll_register_callbacks cb 0x7f44b803ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c55d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c4dd3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b4f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c55d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33232/0 (socket says 172.21.15.17:33232) 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44c55d4640 1 -- 172.21.15.17:0/1619918757 learned_addr learned my addr 172.21.15.17:0/1619918757 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.937+0000 7f44bffff640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 -- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 msgr2=0x564b1b4952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 -- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 msgr2=0x564b1b4f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b4f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 -- 172.21.15.17:0/1619918757 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b1b469720 con 0x564b1b46dda0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c55d4640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f44b001d000 tx=0x7f44b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2445da483ae49951 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f449f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0025040 con 0x564b1b46dda0 2024-03-06T07:24:06.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44b00209a0 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b00207f0 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b0020ba0 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c9c1a740 10 client.7611 fetch_fsmap learned FSMap version 94 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x564b1b448ce0 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.938+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00207f0 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 94) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f44b0028070 con 0x564b1b46dda0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 10 client.7611 fetch_fsmap finished waiting for FSMap version 94 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 10 client.7611 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 20 client.7611 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b490ae0 0x564b1b490ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 4 client.7611 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b1b3b3f40 con 0x564b1b490ae0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b490ae0 0x564b1b490ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:06.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c4dd3640 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b490ae0 0x564b1b490ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b3f04b05fa8f03 server_cookie=bf693e642f5c4815 in_seq=0 out_seq=0 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f449f7fe640 10 client.7611 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f449f7fe640 1 -- 172.21.15.17:0/1619918757 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x564b1b3b3f40 con 0x564b1b490ae0 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f449f7fe640 10 client.7611 handle_command_reply: tid=0 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 1 client.7611 shutdown 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 msgr2=0x564b1b443800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.939+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f44b001d000 tx=0x7f44b0004ab0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 shutdown_connections 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564b1b46dda0 0x564b1b443800 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564b1b448330 0x564b1b4952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x564b1b490ae0 0x564b1b490ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 --2- 172.21.15.17:0/1619918757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564b1b447f50 0x564b1b4f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f449f7fe640 0 client.7611 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 >> 172.21.15.17:0/1619918757 conn(0x564b1b4464b0 msgr2=0x564b1b4423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 shutdown_connections 2024-03-06T07:24:06.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 1 -- 172.21.15.17:0/1619918757 wait complete. 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c9c1a740 20 client.7611 trim_cache size 0 max 16384 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 6203, 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 5.384660802, 2024-03-06T07:24:06.948 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 94, 2024-03-06T07:24:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 110, 2024-03-06T07:24:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 110, 2024-03-06T07:24:06.949 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 17.487302790000001 2024-03-06T07:24:06.949 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:06.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 msgr2=0x7f44ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 0x7f44ac06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44b8036000 tx=0x7f44b803c040 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 msgr2=0x7f44c0070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c3c0 tx=0x7f44b400c880 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 shutdown_connections 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f44ac06b4d0 0x7f44ac06d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44c00c21e0 0x7f44c0071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c0072f40 0x7f44c0070c70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 --2- 172.21.15.17:0/3111002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44c006d680 0x7f44c01ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 >> 172.21.15.17:0/3111002 conn(0x7f44c006b1a0 msgr2=0x7f44c006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 shutdown_connections 2024-03-06T07:24:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:06.940+0000 7f44c7037640 1 -- 172.21.15.17:0/3111002 wait complete. 2024-03-06T07:24:06.952 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 94, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:53.884146+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 94, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6203": { "addr": "172.21.15.163:6836/1364627789", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1364627789, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1364627789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6203, "incarnation": 91, "join_fscid": 8, "name": "d", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 110, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:04.561622+0000", "qdb_cluster": [ 6203 ], "qdb_leader": 6203, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6203 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/386947795", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 386947795, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 386947795, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7021, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2673518972", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2673518972, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2673518972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7386, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/617166148", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 617166148, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 617166148, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 92, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7587, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:06.953 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:24:06.953 INFO:tasks.cephfs.test_failover:current = [{'gid': 7587, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/617166148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 617166148}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 617166148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 92}, {'gid': 7021, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 7386, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2673518972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2673518972}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2673518972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:24:06.954 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:06.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-06T07:24:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.056+0000 7fa450f1e640 1 Processor -- start 2024-03-06T07:24:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.056+0000 7fa450f1e640 1 -- start start 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa450f1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 0x7fa44c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa450f1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0c80b0 con 0x7fa44c0c4c60 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa450f1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0c8390 con 0x7fa44c0c7a20 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa450f1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0c7e90 con 0x7fa44c070a50 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa44c0c4c60 0x7fa44c156dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52750/0 (socket says 172.21.15.17:52750) 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44a59c640 1 -- 172.21.15.17:0/2465778137 learned_addr learned my addr 172.21.15.17:0/2465778137 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44ad9d640 1 --2- 172.21.15.17:0/2465778137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 0x7fa44c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123410882 0 0) 0x7fa44c0c80b0 con 0x7fa44c0c4c60 2024-03-06T07:24:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa438003520 con 0x7fa44c0c4c60 2024-03-06T07:24:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44ad9d640 1 -- 172.21.15.17:0/2465778137 >> v1:172.21.15.163:6789/0 conn(0x7fa44c0c7a20 legacy=0x7fa44c152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44ad9d640 1 -- 172.21.15.17:0/2465778137 >> v1:172.21.15.17:6789/0 conn(0x7fa44c0c4c60 legacy=0x7fa44c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56815821 0 0) 0x7fa44c0c8390 con 0x7fa44c0c7a20 2024-03-06T07:24:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.057+0000 7fa44ad9d640 1 -- 172.21.15.17:0/2465778137 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c1565d0 con 0x7fa44c070a50 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.058+0000 7fa44ad9d640 1 --2- 172.21.15.17:0/2465778137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 0x7fa44c0724d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009dc0 tx=0x7fa43c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1f74c657d97d02c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.058+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c039070 con 0x7fa44c070a50 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.058+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c02cc40 con 0x7fa44c070a50 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.058+0000 7fa44959a640 1 -- 172.21.15.17:0/2465778137 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c034440 con 0x7fa44c070a50 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 -- 172.21.15.17:0/2465778137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 msgr2=0x7fa44c0724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 --2- 172.21.15.17:0/2465778137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 0x7fa44c0724d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009dc0 tx=0x7fa43c02be90 comp rx=0 tx=0).stop 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 -- 172.21.15.17:0/2465778137 shutdown_connections 2024-03-06T07:24:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 --2- 172.21.15.17:0/2465778137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c070a50 0x7fa44c0724d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 -- 172.21.15.17:0/2465778137 >> 172.21.15.17:0/2465778137 conn(0x7fa44c0aa6a0 msgr2=0x7fa44c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 -- 172.21.15.17:0/2465778137 shutdown_connections 2024-03-06T07:24:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.059+0000 7fa450f1e640 1 -- 172.21.15.17:0/2465778137 wait complete. 2024-03-06T07:24:07.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.060+0000 7fa450f1e640 1 Processor -- start 2024-03-06T07:24:07.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.060+0000 7fa450f1e640 1 -- start start 2024-03-06T07:24:07.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa44c0c4c60 0x7fa44c1e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 0x7fa44c1e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c070ee0 con 0x7fa44c070a50 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c072ca0 con 0x7fa44c0c4c60 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa450f1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c156940 con 0x7fa44c0c7a20 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa449d9b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa44c0c4c60 0x7fa44c1e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33238/0 (socket says 172.21.15.17:33238) 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 -- 172.21.15.17:0/243311108 learned_addr learned my addr 172.21.15.17:0/243311108 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44ad9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 0x7fa44c1e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 -- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 msgr2=0x7fa44c1e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 0x7fa44c1e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 -- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa44c0c4c60 msgr2=0x7fa44c1e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa44c0c4c60 0x7fa44c1e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44a59c640 1 -- 172.21.15.17:0/243311108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c1ec160 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.061+0000 7fa44ad9d640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 0x7fa44c1e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa44a59c640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c430 tx=0x7fa44400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444017070 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444005ce0 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c1ec440 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4440072f0 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.062+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c1eca00 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.063+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c04f330 con 0x7fa44c070a50 2024-03-06T07:24:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.063+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa44401a070 con 0x7fa44c070a50 2024-03-06T07:24:07.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.063+0000 7fa442ffd640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.064+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa444089820 con 0x7fa44c070a50 2024-03-06T07:24:07.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.064+0000 7fa449d9b640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.064+0000 7fa449d9b640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 0x7fa42c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa44c14ebe0 tx=0x7fa434009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.068+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa444057e60 con 0x7fa44c070a50 2024-03-06T07:24:07.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.286+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa44c0c2fd0 con 0x7fa44c070a50 2024-03-06T07:24:07.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.575+0000 7fa442ffd640 1 -- 172.21.15.17:0/243311108 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v95) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa44405ce20 con 0x7fa44c070a50 2024-03-06T07:24:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 msgr2=0x7fa42c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 0x7fa42c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa44c14ebe0 tx=0x7fa434009040 comp rx=0 tx=0).stop 2024-03-06T07:24:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 msgr2=0x7fa44c1dfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c430 tx=0x7fa44400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 shutdown_connections 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa42c06b4d0 0x7fa42c06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.577+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa44c0c7a20 0x7fa44c1e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.578+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa44c0c4c60 0x7fa44c1e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.578+0000 7fa450f1e640 1 --2- 172.21.15.17:0/243311108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa44c070a50 0x7fa44c1dfe50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.578+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 >> 172.21.15.17:0/243311108 conn(0x7fa44c0aa6a0 msgr2=0x7fa44c0c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.578+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 shutdown_connections 2024-03-06T07:24:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.578+0000 7fa450f1e640 1 -- 172.21.15.17:0/243311108 wait complete. 2024-03-06T07:24:07.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:07.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.693+0000 7fa7a1cc3640 1 Processor -- start 2024-03-06T07:24:07.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.693+0000 7fa7a1cc3640 1 -- start start 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.693+0000 7fa7a1cc3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa7a1cc3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 0x7fa79c155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa7a1cc3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c158800 con 0x7fa79c06d9f0 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa7a1cc3640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79c158ae0 con 0x7fa79c1551e0 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa7a1cc3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c158dc0 con 0x7fa79c0c3280 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa79c06d9f0 0x7fa79c0a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52766/0 (socket says 172.21.15.17:52766) 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79b7fe640 1 -- 172.21.15.17:0/1076662562 learned_addr learned my addr 172.21.15.17:0/1076662562 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79bfff640 1 --2- 172.21.15.17:0/1076662562 >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 0x7fa79c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79affd640 1 --2- 172.21.15.17:0/1076662562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79a7fc640 1 -- 172.21.15.17:0/1076662562 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548886893 0 0) 0x7fa79c158800 con 0x7fa79c06d9f0 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79a7fc640 1 -- 172.21.15.17:0/1076662562 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa784003520 con 0x7fa79c06d9f0 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79affd640 1 -- 172.21.15.17:0/1076662562 >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 msgr2=0x7fa79c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79affd640 1 --2- 172.21.15.17:0/1076662562 >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 0x7fa79c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79affd640 1 -- 172.21.15.17:0/1076662562 >> v1:172.21.15.17:6789/0 conn(0x7fa79c06d9f0 legacy=0x7fa79c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79affd640 1 -- 172.21.15.17:0/1076662562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c1590a0 con 0x7fa79c0c3280 2024-03-06T07:24:07.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.694+0000 7fa79bfff640 1 --2- 172.21.15.17:0/1076662562 >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 0x7fa79c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.695+0000 7fa79affd640 1 --2- 172.21.15.17:0/1076662562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c0c36e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7900097a0 tx=0x7fa79002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8a9e04a2f0588e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.695+0000 7fa79a7fc640 1 -- 172.21.15.17:0/1076662562 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900315a0 con 0x7fa79c0c3280 2024-03-06T07:24:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.695+0000 7fa79a7fc640 1 -- 172.21.15.17:0/1076662562 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa790037070 con 0x7fa79c0c3280 2024-03-06T07:24:07.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.695+0000 7fa79a7fc640 1 -- 172.21.15.17:0/1076662562 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790031940 con 0x7fa79c0c3280 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/1076662562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 msgr2=0x7fa79c0c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/1076662562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c0c36e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7900097a0 tx=0x7fa79002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/1076662562 shutdown_connections 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/1076662562 >> v2:172.21.15.163:3300/0 conn(0x7fa79c1551e0 0x7fa79c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/1076662562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c0c36e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/1076662562 >> 172.21.15.17:0/1076662562 conn(0x7fa79c0aa630 msgr2=0x7fa79c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/1076662562 shutdown_connections 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.696+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/1076662562 wait complete. 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.697+0000 7fa7a1cc3640 1 Processor -- start 2024-03-06T07:24:07.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.697+0000 7fa7a1cc3640 1 -- start start 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.697+0000 7fa7a1cc3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c1ee650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa79c1551e0 0x7fa79c1dff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c06df50 con 0x7fa79c06d9f0 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c06ddd0 con 0x7fa79c1551e0 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0c6910 con 0x7fa79c0c3280 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33244/0 (socket says 172.21.15.17:33244) 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 -- 172.21.15.17:0/3744527285 learned_addr learned my addr 172.21.15.17:0/3744527285 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79bfff640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa79c1551e0 0x7fa79c1dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79affd640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c1ee650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 -- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 msgr2=0x7fa79c1ee650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c1ee650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 -- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa79c1551e0 msgr2=0x7fa79c1dff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa79c1551e0 0x7fa79c1dff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 -- 172.21.15.17:0/3744527285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c1eef20 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa79b7fe640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c390 tx=0x7fa78c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c017070 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c005ce0 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c1ef1c0 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c0072f0 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.698+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c06ea90 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.699+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c04f330 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.700+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa78c01a070 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.700+0000 7fa783fff640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 0x7fa78806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.700+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa78c089730 con 0x7fa79c06d9f0 2024-03-06T07:24:07.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.700+0000 7fa79affd640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 0x7fa78806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:07.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.701+0000 7fa79affd640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 0x7fa78806d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa790036000 tx=0x7fa79003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:07.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.704+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa78c057d80 con 0x7fa79c06d9f0 2024-03-06T07:24:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.924+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa79c06d4a0 con 0x7fa79c06d9f0 2024-03-06T07:24:07.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.925+0000 7fa783fff640 1 -- 172.21.15.17:0/3744527285 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4987 (secure 0 0 0) 0x7fa78c05be20 con 0x7fa79c06d9f0 2024-03-06T07:24:07.926 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:07.927 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":96,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:07.577937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7587":{"gid":7587,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/617166148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":617166148},{"type":"v1","addr":"172.21.15.17:6835","nonce":617166148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:07.927 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 96 2024-03-06T07:24:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.926+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 msgr2=0x7fa78806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.926+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 0x7fa78806d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa790036000 tx=0x7fa79003d040 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.926+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 msgr2=0x7fa79c06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.926+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00c390 tx=0x7fa78c00c850 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 shutdown_connections 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa78806b4d0 0x7fa78806d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa79c1551e0 0x7fa79c1dff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa79c0c3280 0x7fa79c1ee650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 --2- 172.21.15.17:0/3744527285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa79c06d9f0 0x7fa79c06f5c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 >> 172.21.15.17:0/3744527285 conn(0x7fa79c0aa630 msgr2=0x7fa79c154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 shutdown_connections 2024-03-06T07:24:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:07.927+0000 7fa7a1cc3640 1 -- 172.21.15.17:0/3744527285 wait complete. 2024-03-06T07:24:07.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs 2024-03-06T07:24:08.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 Processor -- start 2024-03-06T07:24:08.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 -- start start 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 0x7f53080c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5308155260 con 0x7f53080c7370 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5308155540 con 0x7f53080c77d0 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.042+0000 7f530f113640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080c7c00 con 0x7f53081527f0 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f5307fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f53080c7370 0x7f530806c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52776/0 (socket says 172.21.15.17:52776) 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f5307fff640 1 -- 172.21.15.17:0/1468807873 learned_addr learned my addr 172.21.15.17:0/1468807873 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f530ceaf640 1 --2- 172.21.15.17:0/1468807873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 0x7f53080c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416726019 0 0) 0x7f5308155260 con 0x7f53080c7370 2024-03-06T07:24:08.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f8003520 con 0x7f53080c7370 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500161583 0 0) 0x7f5308155540 con 0x7f53080c77d0 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5308155260 con 0x7f53080c77d0 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f530ceaf640 1 -- 172.21.15.17:0/1468807873 >> v1:172.21.15.163:6789/0 conn(0x7f53080c77d0 legacy=0x7f530806c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f530ceaf640 1 -- 172.21.15.17:0/1468807873 >> v1:172.21.15.17:6789/0 conn(0x7f53080c7370 legacy=0x7f530806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.043+0000 7f530ceaf640 1 -- 172.21.15.17:0/1468807873 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308155820 con 0x7f53081527f0 2024-03-06T07:24:08.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f530ceaf640 1 --2- 172.21.15.17:0/1468807873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 0x7f53080c8860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097d0 tx=0x7f52fc02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978ecaabe0ff00b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc031770 con 0x7f53081527f0 2024-03-06T07:24:08.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc037070 con 0x7f53081527f0 2024-03-06T07:24:08.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f53077fe640 1 -- 172.21.15.17:0/1468807873 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc031b10 con 0x7f53081527f0 2024-03-06T07:24:08.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f530f113640 1 -- 172.21.15.17:0/1468807873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 msgr2=0x7f53080c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.044+0000 7f530f113640 1 --2- 172.21.15.17:0/1468807873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 0x7f53080c8860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0097d0 tx=0x7f52fc02b810 comp rx=0 tx=0).stop 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.045+0000 7f530f113640 1 -- 172.21.15.17:0/1468807873 shutdown_connections 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.045+0000 7f530f113640 1 --2- 172.21.15.17:0/1468807873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53081527f0 0x7f53080c8860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.045+0000 7f530f113640 1 -- 172.21.15.17:0/1468807873 >> 172.21.15.17:0/1468807873 conn(0x7f53080ac4f0 msgr2=0x7f53080ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.045+0000 7f530f113640 1 -- 172.21.15.17:0/1468807873 shutdown_connections 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.045+0000 7f530f113640 1 -- 172.21.15.17:0/1468807873 wait complete. 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 Processor -- start 2024-03-06T07:24:08.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 -- start start 2024-03-06T07:24:08.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53080c7370 0x7f53080b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53080c77d0 0x7f5308157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080c8da0 con 0x7f53081527f0 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080be1a0 con 0x7f53080c77d0 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.046+0000 7f530f113640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080c8f20 con 0x7f53080c7370 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33258/0 (socket says 172.21.15.17:33258) 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 -- 172.21.15.17:0/736028196 learned_addr learned my addr 172.21.15.17:0/736028196 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f5307fff640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53080c77d0 0x7f5308157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530ceaf640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53080c7370 0x7f53080b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 -- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53080c7370 msgr2=0x7f53080b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53080c7370 0x7f53080b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 -- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53080c77d0 msgr2=0x7f5308157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53080c77d0 0x7f5308157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 -- 172.21.15.17:0/736028196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53080b05a0 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530d6b0640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f530000c6e0 tx=0x7f530000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300003b70 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300003d10 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53000052d0 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53081ee900 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.047+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53081574a0 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.048+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530804f330 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.049+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5300005470 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.049+0000 7f53057fa640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.049+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f530008a110 con 0x7f53081527f0 2024-03-06T07:24:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.049+0000 7f530ceaf640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.050+0000 7f530ceaf640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f52fc036000 tx=0x7f52fc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.053+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53000587f0 con 0x7f53081527f0 2024-03-06T07:24:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.233+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f530806d860 con 0x7f53081527f0 2024-03-06T07:24:08.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.236+0000 7f53057fa640 1 -- 172.21.15.17:0/736028196 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v37)=0 v37) v1 ==== 116+0+0 (secure 0 0 0) 0x7f530005c890 con 0x7f53081527f0 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 msgr2=0x7f52ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f52fc036000 tx=0x7f52fc03d040 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 msgr2=0x7f53081ede50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f530000c6e0 tx=0x7f530000cba0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 shutdown_connections 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53081527f0 0x7f53081ede50 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53080c77d0 0x7f5308157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 --2- 172.21.15.17:0/736028196 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53080c7370 0x7f53080b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 >> 172.21.15.17:0/736028196 conn(0x7f53080ac4f0 msgr2=0x7f530814ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 shutdown_connections 2024-03-06T07:24:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.238+0000 7f530f113640 1 -- 172.21.15.17:0/736028196 wait complete. 2024-03-06T07:24:08.250 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.349+0000 7f6a4db0c640 1 Processor -- start 2024-03-06T07:24:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.349+0000 7f6a4db0c640 1 -- start start 2024-03-06T07:24:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a4db0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 0x7f6a481529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a4db0c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480c28d0 con 0x7f6a4814f790 2024-03-06T07:24:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a4db0c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480c2bb0 con 0x7f6a480c1aa0 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a4db0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c2680 con 0x7f6a48156150 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a477fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6a4814f790 0x7f6a480c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52782/0 (socket says 172.21.15.17:52782) 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a477fe640 1 -- 172.21.15.17:0/3699235153 learned_addr learned my addr 172.21.15.17:0/3699235153 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a47fff640 1 --2- 172.21.15.17:0/3699235153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 0x7f6a481529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411976487 0 0) 0x7f6a480c28d0 con 0x7f6a4814f790 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a30003520 con 0x7f6a4814f790 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485789380 0 0) 0x7f6a480c2bb0 con 0x7f6a480c1aa0 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a480c28d0 con 0x7f6a480c1aa0 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a47fff640 1 -- 172.21.15.17:0/3699235153 >> v1:172.21.15.163:6789/0 conn(0x7f6a480c1aa0 legacy=0x7f6a48155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a47fff640 1 -- 172.21.15.17:0/3699235153 >> v1:172.21.15.17:6789/0 conn(0x7f6a4814f790 legacy=0x7f6a480c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.350+0000 7f6a47fff640 1 -- 172.21.15.17:0/3699235153 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480c2e90 con 0x7f6a48156150 2024-03-06T07:24:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.351+0000 7f6a47fff640 1 --2- 172.21.15.17:0/3699235153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 0x7f6a481529d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009dc0 tx=0x7f6a4002bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b473a0f154d7716 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.351+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40039070 con 0x7f6a48156150 2024-03-06T07:24:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.351+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4002ccd0 con 0x7f6a48156150 2024-03-06T07:24:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.351+0000 7f6a467fc640 1 -- 172.21.15.17:0/3699235153 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40034640 con 0x7f6a48156150 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 -- 172.21.15.17:0/3699235153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 msgr2=0x7f6a481529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/3699235153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 0x7f6a481529d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009dc0 tx=0x7f6a4002bf20 comp rx=0 tx=0).stop 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 -- 172.21.15.17:0/3699235153 shutdown_connections 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/3699235153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a48156150 0x7f6a481529d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 -- 172.21.15.17:0/3699235153 >> 172.21.15.17:0/3699235153 conn(0x7f6a480c6710 msgr2=0x7f6a480c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 -- 172.21.15.17:0/3699235153 shutdown_connections 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.352+0000 7f6a4db0c640 1 -- 172.21.15.17:0/3699235153 wait complete. 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.353+0000 7f6a4db0c640 1 Processor -- start 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.353+0000 7f6a4db0c640 1 -- start start 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a480c1aa0 0x7f6a481ec440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a4814f790 0x7f6a481496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c5e30 con 0x7f6a48156150 2024-03-06T07:24:08.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c0d60 con 0x7f6a4814f790 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c5fb0 con 0x7f6a480c1aa0 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a480c1aa0 0x7f6a481ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33264/0 (socket says 172.21.15.17:33264) 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a46ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a4814f790 0x7f6a481496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 -- 172.21.15.17:0/2317743732 learned_addr learned my addr 172.21.15.17:0/2317743732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 -- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a480c1aa0 msgr2=0x7f6a481ec440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a480c1aa0 0x7f6a481ec440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 -- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a4814f790 msgr2=0x7f6a481496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a4814f790 0x7f6a481496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 -- 172.21.15.17:0/2317743732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48070470 con 0x7f6a48156150 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a47fff640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002c530 tx=0x7f6a40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40039050 con 0x7f6a48156150 2024-03-06T07:24:08.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481ecec0 con 0x7f6a48156150 2024-03-06T07:24:08.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40045070 con 0x7f6a48156150 2024-03-06T07:24:08.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.354+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40009450 con 0x7f6a48156150 2024-03-06T07:24:08.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.355+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a48070a10 con 0x7f6a48156150 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.355+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a48069460 con 0x7f6a48156150 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.356+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a4003e850 con 0x7f6a48156150 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.356+0000 7f6a2ffff640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.356+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6a400ad280 con 0x7f6a48156150 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.356+0000 7f6a477fe640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:08.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.357+0000 7f6a477fe640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 0x7f6a3406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000f60 tx=0x7f6a38007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:08.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.361+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6a48069460 con 0x7f6a48156150 2024-03-06T07:24:08.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.578+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a4804f330 con 0x7f6a48156150 2024-03-06T07:24:08.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.578+0000 7f6a2ffff640 1 -- 172.21.15.17:0/2317743732 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4987 (secure 0 0 0) 0x7f6a4007b990 con 0x7f6a48156150 2024-03-06T07:24:08.580 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:08.580 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7021,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":96,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:07.577937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7587":{"gid":7587,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/617166148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":617166148},{"type":"v1","addr":"172.21.15.17:6835","nonce":617166148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:08.581 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 96 2024-03-06T07:24:08.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.580+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 msgr2=0x7f6a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.580+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 0x7f6a3406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a38000f60 tx=0x7f6a38007d70 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.580+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 msgr2=0x7f6a48149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.580+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002c530 tx=0x7f6a40004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.580+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 shutdown_connections 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6a3406b4d0 0x7f6a3406d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6a48156150 0x7f6a48149c30 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a4814f790 0x7f6a481496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 --2- 172.21.15.17:0/2317743732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a480c1aa0 0x7f6a481ec440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 >> 172.21.15.17:0/2317743732 conn(0x7f6a480c6710 msgr2=0x7f6a480c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 shutdown_connections 2024-03-06T07:24:08.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:08.581+0000 7f6a4db0c640 1 -- 172.21.15.17:0/2317743732 wait complete. 2024-03-06T07:24:08.995 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-06T07:23:53.884146+0000', 'modified': '2024-03-06T07:24:07.577937+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6203': {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7587': {'gid': 7587, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6835/617166148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 617166148}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 617166148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6203, 'qdb_cluster': [6203]} 2024-03-06T07:24:08.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:08.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:24:09.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:09.080 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:09.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.099+0000 7fe7e65f7640 1 Processor -- start 2024-03-06T07:24:09.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.099+0000 7fe7e65f7640 1 -- start start 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e65f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7e006e0a0 0x7fe7e0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e65f7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e0070550 con 0x7fe7e0072f00 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e65f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e0070830 con 0x7fe7e00c0fa0 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e65f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e0070b10 con 0x7fe7e006e0a0 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7dffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7e0072f00 0x7fe7e00c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52788/0 (socket says 172.21.15.17:52788) 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7dffff640 1 -- 172.21.15.17:0/3159984833 learned_addr learned my addr 172.21.15.17:0/3159984833 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/3159984833 >> v2:172.21.15.163:3300/0 conn(0x7fe7e006e0a0 0x7fe7e0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917865566 0 0) 0x7fe7e0070550 con 0x7fe7e0072f00 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c8003520 con 0x7fe7e0072f00 2024-03-06T07:24:09.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327321402 0 0) 0x7fe7e0070830 con 0x7fe7e00c0fa0 2024-03-06T07:24:09.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7e0070550 con 0x7fe7e00c0fa0 2024-03-06T07:24:09.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e4b94640 1 -- 172.21.15.17:0/3159984833 >> v1:172.21.15.163:6789/0 conn(0x7fe7e00c0fa0 legacy=0x7fe7e006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e4b94640 1 -- 172.21.15.17:0/3159984833 >> v1:172.21.15.17:6789/0 conn(0x7fe7e0072f00 legacy=0x7fe7e00c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.100+0000 7fe7e4b94640 1 -- 172.21.15.17:0/3159984833 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e00732e0 con 0x7fe7e006e0a0 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.101+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/3159984833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e006e0a0 0x7fe7e0153990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009d90 tx=0x7fe7d802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51aca1bef2acd896 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.101+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8039070 con 0x7fe7e006e0a0 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.101+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d802cd00 con 0x7fe7e006e0a0 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.101+0000 7fe7deffd640 1 -- 172.21.15.17:0/3159984833 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8034660 con 0x7fe7e006e0a0 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 -- 172.21.15.17:0/3159984833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e006e0a0 msgr2=0x7fe7e0153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/3159984833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e006e0a0 0x7fe7e0153990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009d90 tx=0x7fe7d802bf50 comp rx=0 tx=0).stop 2024-03-06T07:24:09.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 -- 172.21.15.17:0/3159984833 shutdown_connections 2024-03-06T07:24:09.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/3159984833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e006e0a0 0x7fe7e0153990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 -- 172.21.15.17:0/3159984833 >> 172.21.15.17:0/3159984833 conn(0x7fe7e006b1a0 msgr2=0x7fe7e006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 -- 172.21.15.17:0/3159984833 shutdown_connections 2024-03-06T07:24:09.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.102+0000 7fe7e65f7640 1 -- 172.21.15.17:0/3159984833 wait complete. 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.103+0000 7fe7e65f7640 1 Processor -- start 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.103+0000 7fe7e65f7640 1 -- start start 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7e006e0a0 0x7fe7e00c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e00c0fa0 0x7fe7e01ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0070f70 con 0x7fe7e0072f00 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0070df0 con 0x7fe7e00c0fa0 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e65f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0072400 con 0x7fe7e006e0a0 2024-03-06T07:24:09.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33272/0 (socket says 172.21.15.17:33272) 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7df7fe640 1 -- 172.21.15.17:0/2774352929 learned_addr learned my addr 172.21.15.17:0/2774352929 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7dffff640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7e006e0a0 0x7fe7e00c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.104+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e00c0fa0 0x7fe7e01ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 -- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7e006e0a0 msgr2=0x7fe7e00c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7e006e0a0 0x7fe7e00c4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 -- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e00c0fa0 msgr2=0x7fe7e01ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e00c0fa0 0x7fe7e01ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 -- 172.21.15.17:0/2774352929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e01567e0 con 0x7fe7e0072f00 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c3c0 tx=0x7fe7d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7dcff9640 1 -- 172.21.15.17:0/2774352929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4017070 con 0x7fe7e0072f00 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7dcff9640 1 -- 172.21.15.17:0/2774352929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d40052f0 con 0x7fe7e0072f00 2024-03-06T07:24:09.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7dcff9640 1 -- 172.21.15.17:0/2774352929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4003830 con 0x7fe7e0072f00 2024-03-06T07:24:09.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01550a0 con 0x7fe7e0072f00 2024-03-06T07:24:09.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.105+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e0156d60 con 0x7fe7e0072f00 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.106+0000 7fe7dcff9640 1 -- 172.21.15.17:0/2774352929 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7d401a070 con 0x7fe7e0072f00 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.107+0000 7fe7dcff9640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 0x7fe7cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.107+0000 7fe7dcff9640 1 -- 172.21.15.17:0/2774352929 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe7d40896e0 con 0x7fe7e0072f00 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.107+0000 7fe7dffff640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 0x7fe7cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.107+0000 7fe7dffff640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 0x7fe7cc06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00011e0 tx=0x7fe7d0007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 Processor -- start 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 -- start start 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 0x560cbf4bfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 0x560cbf4bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c04b0 0x560cbf4bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf4266b0 con 0x560cbf4c04b0 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf3cfa60 con 0x560cbf4c1f50 2024-03-06T07:24:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.108+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cbf422750 con 0x560cbf4c2330 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 0x560cbf4bfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 0x560cbf4bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7e4b94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c04b0 0x560cbf4bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 0x560cbf4bfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50822/0 (socket says 172.21.15.17:50822) 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7dffff640 1 -- 172.21.15.17:0/967756119 learned_addr learned my addr 172.21.15.17:0/967756119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 -- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 msgr2=0x560cbf4bfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 0x560cbf4bfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 -- 172.21.15.17:0/967756119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c04b0 msgr2=0x560cbf4bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c04b0 0x560cbf4bc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 -- 172.21.15.17:0/967756119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cbf3edf30 con 0x560cbf4c2330 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 0x560cbf4bff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001eb0 tx=0x7fe7d4057d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fbc71793aa517b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7c5ffb640 1 -- 172.21.15.17:0/967756119 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4055d30 con 0x560cbf4c2330 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7c5ffb640 1 -- 172.21.15.17:0/967756119 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d4055f90 con 0x560cbf4c2330 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.109+0000 7fe7c5ffb640 1 -- 172.21.15.17:0/967756119 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d406bba0 con 0x560cbf4c2330 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 msgr2=0x560cbf4bff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 0x560cbf4bff70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001eb0 tx=0x7fe7d4057d50 comp rx=0 tx=0).stop 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- 172.21.15.17:0/967756119 shutdown_connections 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c04b0 0x560cbf4bc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c2330 0x560cbf4bff70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/967756119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c1f50 0x560cbf4bfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- 172.21.15.17:0/967756119 >> 172.21.15.17:0/967756119 conn(0x560cbf4be7d0 msgr2=0x560cbf4e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- 172.21.15.17:0/967756119 shutdown_connections 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- 172.21.15.17:0/967756119 wait complete. 2024-03-06T07:24:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.110+0000 7fe7e91e3740 1 -- start start 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 Processor -- start 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf50e740 0x560cbf4fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf4e4230 con 0x560cbf4c2330 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf3e2ea0 con 0x560cbf50e740 2024-03-06T07:24:09.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cbf4e3c70 con 0x560cbf4c1f50 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7e91e3740 10 client.0 _ll_register_callbacks cb 0x560cbf3e2870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.111+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33296/0 (socket says 172.21.15.17:33296) 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 -- 172.21.15.17:0/1126860706 learned_addr learned my addr 172.21.15.17:0/1126860706 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf50e740 0x560cbf4fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 -- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 msgr2=0x560cbf4e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 -- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf50e740 msgr2=0x560cbf4fa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf50e740 0x560cbf4fa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 -- 172.21.15.17:0/1126860706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cbf3e3270 con 0x560cbf4c2330 2024-03-06T07:24:09.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4057bd0 tx=0x7fe7d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9808ec2f0372ba14 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d406bba0 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe7d406bd40 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7d4055f90 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d40570e0 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7e91e3740 10 client.7647 fetch_fsmap learned FSMap version 96 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x560cbf13b2d0 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d4055f90 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.112+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 96) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe7d4082070 con 0x560cbf4c2330 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 10 client.7647 fetch_fsmap finished waiting for FSMap version 96 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 10 client.7647 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 20 client.7647 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:09.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf4e4bb0 0x560cbf4e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 4 client.7647 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cbf433bf0 con 0x560cbf4e4bb0 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7dffff640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf4e4bb0 0x560cbf4e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7beffd640 10 client.7647 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.113+0000 7fe7dffff640 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf4e4bb0 0x560cbf4e4f90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5016db5ed8de5f36 server_cookie=53793652a7d74791 in_seq=0 out_seq=0 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7beffd640 1 -- 172.21.15.17:0/1126860706 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560cbf433bf0 con 0x560cbf4e4bb0 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7beffd640 10 client.7647 handle_command_reply: tid=0 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 client.7647 shutdown 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 msgr2=0x560cbf4e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4057bd0 tx=0x7fe7d4000bc0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 shutdown_connections 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf4e4bb0 0x560cbf4e4f90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf50e740 0x560cbf4fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c2330 0x560cbf4e7be0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7beffd640 0 client.7647 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/1126860706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4e2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 >> 172.21.15.17:0/1126860706 conn(0x560cbf4c04b0 msgr2=0x560cbf4c08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 shutdown_connections 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 1 -- 172.21.15.17:0/1126860706 wait complete. 2024-03-06T07:24:09.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.114+0000 7fe7e91e3740 20 client.7647 trim_cache size 0 max 16384 2024-03-06T07:24:09.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 Processor -- start 2024-03-06T07:24:09.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 -- start start 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4bb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01f7d90 con 0x560cbf50e740 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e00c0400 con 0x560cbf4c2330 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.131+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01f69a0 con 0x560cbf4c1f50 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33306/0 (socket says 172.21.15.17:33306) 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 -- 172.21.15.17:0/2823601638 learned_addr learned my addr 172.21.15.17:0/2823601638 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7dffff640 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4bb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 -- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 msgr2=0x560cbf4bb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4bb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 -- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 msgr2=0x560cbf4bba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 -- 172.21.15.17:0/2823601638 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cbf4e7f50 con 0x560cbf50e740 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009e80 tx=0x7fe7d802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4b6feba6158691 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7c57fa640 1 -- 172.21.15.17:0/2823601638 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8039040 con 0x560cbf50e740 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7c57fa640 1 -- 172.21.15.17:0/2823601638 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d803a040 con 0x560cbf50e740 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.132+0000 7fe7c57fa640 1 -- 172.21.15.17:0/2823601638 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8041440 con 0x560cbf50e740 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- 172.21.15.17:0/2823601638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 msgr2=0x560cbf50ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009e80 tx=0x7fe7d802e000 comp rx=0 tx=0).stop 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- 172.21.15.17:0/2823601638 shutdown_connections 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf50e740 0x560cbf50ae10 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/2823601638 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf4c1f50 0x560cbf4bb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- 172.21.15.17:0/2823601638 >> 172.21.15.17:0/2823601638 conn(0x560cbf4c04b0 msgr2=0x560cbf4e2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- 172.21.15.17:0/2823601638 shutdown_connections 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- 172.21.15.17:0/2823601638 wait complete. 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.133+0000 7fe7e91e3740 1 -- start start 2024-03-06T07:24:09.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 Processor -- start 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf50e740 0x560cbf4bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf4e3c70 con 0x560cbf4c1f50 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cbf3e2ea0 con 0x560cbf4c2330 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cbf4e4230 con 0x560cbf50e740 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e91e3740 10 client.0 _ll_register_callbacks cb 0x7fe7c80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33320/0 (socket says 172.21.15.17:33320) 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7dffff640 1 -- 172.21.15.17:0/3979156349 learned_addr learned my addr 172.21.15.17:0/3979156349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.134+0000 7fe7e4b94640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf50e740 0x560cbf4bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 -- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf50e740 msgr2=0x560cbf4bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf50e740 0x560cbf4bf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 -- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 msgr2=0x560cbf4bb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 -- 172.21.15.17:0/3979156349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cbf4e9fa0 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7dffff640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006fd0 tx=0x7fe7d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bcbdbbf5a7c941 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d002e020 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe7d00062d0 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7d803a040 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d0035050 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 10 client.7653 fetch_fsmap learned FSMap version 96 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x560cbf2056b0 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d803a040 con 0x560cbf4c1f50 2024-03-06T07:24:09.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 96) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe7d0038020 con 0x560cbf4c1f50 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 10 client.7653 fetch_fsmap finished waiting for FSMap version 96 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 10 client.7653 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 20 client.7653 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf567fc0 0x560cbf4bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 4 client.7653 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.135+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560cbf42df40 con 0x560cbf567fc0 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf567fc0 0x560cbf4bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7beffd640 10 client.7653 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7df7fe640 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf567fc0 0x560cbf4bf8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe96e79ed0428b7 server_cookie=37ba0462cc04aa14 in_seq=0 out_seq=0 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7beffd640 1 -- 172.21.15.17:0/3979156349 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560cbf42df40 con 0x560cbf567fc0 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7beffd640 10 client.7653 handle_command_reply: tid=0 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 client.7653 shutdown 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 msgr2=0x560cbf521e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006fd0 tx=0x7fe7d0012000 comp rx=0 tx=0).stop 2024-03-06T07:24:09.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 shutdown_connections 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560cbf50e740 0x560cbf4bf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x560cbf567fc0 0x560cbf4bf8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560cbf4c2330 0x560cbf4bb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 --2- 172.21.15.17:0/3979156349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560cbf4c1f50 0x560cbf521e30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7beffd640 0 client.7653 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 >> 172.21.15.17:0/3979156349 conn(0x560cbf4c04b0 msgr2=0x560cbf4e86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 shutdown_connections 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 1 -- 172.21.15.17:0/3979156349 wait complete. 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.136+0000 7fe7e91e3740 20 client.7653 trim_cache size 0 max 16384 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:09.144 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 6203, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 7.5814344589999996, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 96, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 110, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 110, 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 19.684076383000001 2024-03-06T07:24:09.145 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:09.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 msgr2=0x7fe7cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 0x7fe7cc06d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7d00011e0 tx=0x7fe7d0007d30 comp rx=0 tx=0).stop 2024-03-06T07:24:09.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 msgr2=0x7fe7e01ee330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:09.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400c3c0 tx=0x7fe7d400c880 comp rx=0 tx=0).stop 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 shutdown_connections 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe7cc06b4d0 0x7fe7cc06d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7e00c0fa0 0x7fe7e01ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7e0072f00 0x7fe7e01ee330 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 --2- 172.21.15.17:0/2774352929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7e006e0a0 0x7fe7e00c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 >> 172.21.15.17:0/2774352929 conn(0x7fe7e006b1a0 msgr2=0x7fe7e006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 shutdown_connections 2024-03-06T07:24:09.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:09.137+0000 7fe7e65f7640 1 -- 172.21.15.17:0/2774352929 wait complete. 2024-03-06T07:24:09.151 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 96, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:53.884146+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 96, "ever_allowed_features": 32, "explicitly_allowed_features": 32, "failed": [], "flags": 50, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": true, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6203": { "addr": "172.21.15.163:6836/1364627789", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1364627789, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1364627789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6203, "incarnation": 91, "join_fscid": 8, "name": "d", "rank": 0, "state": "up:active", "state_seq": 7 }, "gid_7587": { "addr": "172.21.15.17:6835/617166148", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 617166148, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 617166148, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7587, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:standby-replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 110, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:07.577937+0000", "qdb_cluster": [ 6203 ], "qdb_leader": 6203, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6203 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/386947795", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 386947795, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 386947795, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7021, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/2673518972", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2673518972, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2673518972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7386, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:09.153 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:standby-replay'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:24:09.153 INFO:tasks.cephfs.test_failover:current = [{'gid': 7587, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6835/617166148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 617166148}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 617166148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7021, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 7386, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2673518972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2673518972}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2673518972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:24:09.153 DEBUG:tasks.cephfs.test_failover:8 != -1 2024-03-06T07:24:09.153 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:09.611 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:24:14.154 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.266+0000 7f96fc913640 1 Processor -- start 2024-03-06T07:24:14.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.266+0000 7f96fc913640 1 -- start start 2024-03-06T07:24:14.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96fc913640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96f80c3710 0x7f96f81558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:14.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96fc913640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8158800 con 0x7f96f80c3280 2024-03-06T07:24:14.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96fc913640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8158ae0 con 0x7f96f806d9f0 2024-03-06T07:24:14.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96fc913640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f8158dc0 con 0x7f96f80c3710 2024-03-06T07:24:14.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96effff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96f80c3280 0x7f96f81551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58882/0 (socket says 172.21.15.17:58882) 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96effff640 1 -- 172.21.15.17:0/2750266407 learned_addr learned my addr 172.21.15.17:0/2750266407 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f6d9d640 1 --2- 172.21.15.17:0/2750266407 >> v2:172.21.15.163:3300/0 conn(0x7f96f80c3710 0x7f96f81558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859076816 0 0) 0x7f96f8158800 con 0x7f96f80c3280 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e4003520 con 0x7f96f80c3280 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73726112 0 0) 0x7f96f8158ae0 con 0x7f96f806d9f0 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f8158800 con 0x7f96f806d9f0 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f6d9d640 1 -- 172.21.15.17:0/2750266407 >> v1:172.21.15.163:6789/0 conn(0x7f96f806d9f0 legacy=0x7f96f80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f6d9d640 1 -- 172.21.15.17:0/2750266407 >> v1:172.21.15.17:6789/0 conn(0x7f96f80c3280 legacy=0x7f96f81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.267+0000 7f96f6d9d640 1 -- 172.21.15.17:0/2750266407 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f81590a0 con 0x7f96f80c3710 2024-03-06T07:24:14.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.268+0000 7f96f6d9d640 1 --2- 172.21.15.17:0/2750266407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f80c3710 0x7f96f81558f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009d90 tx=0x7f96f002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b710b27f762da053 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.268+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0039070 con 0x7f96f80c3710 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.268+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f002cc60 con 0x7f96f80c3710 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.268+0000 7f96f5d9b640 1 -- 172.21.15.17:0/2750266407 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0034660 con 0x7f96f80c3710 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 -- 172.21.15.17:0/2750266407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f80c3710 msgr2=0x7f96f81558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 --2- 172.21.15.17:0/2750266407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f80c3710 0x7f96f81558f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009d90 tx=0x7f96f002beb0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 -- 172.21.15.17:0/2750266407 shutdown_connections 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 --2- 172.21.15.17:0/2750266407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f80c3710 0x7f96f81558f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 -- 172.21.15.17:0/2750266407 >> 172.21.15.17:0/2750266407 conn(0x7f96f80aa630 msgr2=0x7f96f80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 -- 172.21.15.17:0/2750266407 shutdown_connections 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.269+0000 7f96fc913640 1 -- 172.21.15.17:0/2750266407 wait complete. 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.270+0000 7f96fc913640 1 Processor -- start 2024-03-06T07:24:14.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.270+0000 7f96fc913640 1 -- start start 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 0x7f96f81ee130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f80c3280 0x7f96f81ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f806ddd0 con 0x7f96f80c3710 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8155e30 con 0x7f96f806d9f0 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96fc913640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f806df50 con 0x7f96f80c3280 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f659c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 0x7f96f81ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44766/0 (socket says 172.21.15.17:44766) 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 -- 172.21.15.17:0/3304051360 learned_addr learned my addr 172.21.15.17:0/3304051360 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96effff640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f80c3280 0x7f96f81ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 -- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f80c3280 msgr2=0x7f96f81ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f80c3280 0x7f96f81ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 -- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 msgr2=0x7f96f81ee130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 0x7f96f81ee130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f6d9d640 1 -- 172.21.15.17:0/3304051360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8153ab0 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.271+0000 7f96f659c640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 0x7f96f81ee130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96f6d9d640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009d90 tx=0x7f96f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0039050 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0045070 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f81eec90 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0009450 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.272+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f80abb50 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.273+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f804f330 con 0x7f96f80c3710 2024-03-06T07:24:14.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.273+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96f003e850 con 0x7f96f80c3710 2024-03-06T07:24:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.274+0000 7f96eeffd640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.274+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f96f00ad170 con 0x7f96f80c3710 2024-03-06T07:24:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.274+0000 7f96f659c640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.274+0000 7f96f659c640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 0x7f96d806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007fd0 tx=0x7f96e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:14.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.278+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96f007b970 con 0x7f96f80c3710 2024-03-06T07:24:14.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.498+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96f8073240 con 0x7f96f80c3710 2024-03-06T07:24:14.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.499+0000 7f96eeffd640 1 -- 172.21.15.17:0/3304051360 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4987 (secure 0 0 0) 0x7f96f007fa10 con 0x7f96f80c3710 2024-03-06T07:24:14.501 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:14.501 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:09.583188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7021":{"gid":7021,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":6,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:14.501 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 99 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 msgr2=0x7f96d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 0x7f96d806d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f96e8007fd0 tx=0x7f96e8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 msgr2=0x7f96f814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009d90 tx=0x7f96f0004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 shutdown_connections 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d806b4d0 0x7f96d806d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f80c3710 0x7f96f814e640 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f80c3280 0x7f96f81ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 --2- 172.21.15.17:0/3304051360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96f806d9f0 0x7f96f81ee130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:14.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 >> 172.21.15.17:0/3304051360 conn(0x7f96f80aa630 msgr2=0x7f96f8154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:14.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 shutdown_connections 2024-03-06T07:24:14.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:14.501+0000 7f96fc913640 1 -- 172.21.15.17:0/3304051360 wait complete. 2024-03-06T07:24:14.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-06T07:23:53.884146+0000', 'modified': '2024-03-06T07:24:09.583188+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6203': {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7021': {'gid': 7021, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 6, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6203, 'qdb_cluster': [6203]} 2024-03-06T07:24:14.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:14.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:24:15.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.021+0000 7f045e113640 1 Processor -- start 2024-03-06T07:24:15.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.021+0000 7f045e113640 1 -- start start 2024-03-06T07:24:15.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04580c69f0 0x7f04580c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580c8950 0x7f04580acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f04580ad2b0 0x7f04581558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0458155440 con 0x7f04580c69f0 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0458155130 con 0x7f04580ad2b0 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f045e113640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c5d50 con 0x7f04580c8950 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f04577fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04580c69f0 0x7f04580c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f04577fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04580c69f0 0x7f04580c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44772/0 (socket says 172.21.15.17:44772) 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f04577fe640 1 -- 172.21.15.17:0/3948567651 learned_addr learned my addr 172.21.15.17:0/3948567651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f0457fff640 1 --2- 172.21.15.17:0/3948567651 >> v2:172.21.15.163:3300/0 conn(0x7f04580ad2b0 0x7f04581558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.022+0000 7f0456ffd640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580c8950 0x7f04580acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 -- 172.21.15.17:0/3948567651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580c8950 msgr2=0x7f04580acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580c8950 0x7f04580acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 -- 172.21.15.17:0/3948567651 >> v2:172.21.15.163:3300/0 conn(0x7f04580ad2b0 msgr2=0x7f04581558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 --2- 172.21.15.17:0/3948567651 >> v2:172.21.15.163:3300/0 conn(0x7f04580ad2b0 0x7f04581558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 -- 172.21.15.17:0/3948567651 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04580c6030 con 0x7f04580c69f0 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.023+0000 7f04577fe640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f04580c6dd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0458159820 tx=0x7f044802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80758135c3180946 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f04567fc640 1 -- 172.21.15.17:0/3948567651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044802e070 con 0x7f04580c69f0 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f04567fc640 1 -- 172.21.15.17:0/3948567651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448037070 con 0x7f04580c69f0 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f04567fc640 1 -- 172.21.15.17:0/3948567651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448031a00 con 0x7f04580c69f0 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 -- 172.21.15.17:0/3948567651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 msgr2=0x7f04580c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f04580c6dd0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f0458159820 tx=0x7f044802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 -- 172.21.15.17:0/3948567651 shutdown_connections 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 --2- 172.21.15.17:0/3948567651 >> v2:172.21.15.163:3300/0 conn(0x7f04580ad2b0 0x7f04581558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580c8950 0x7f04580acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 --2- 172.21.15.17:0/3948567651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f04580c6dd0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 -- 172.21.15.17:0/3948567651 >> 172.21.15.17:0/3948567651 conn(0x7f04580cae70 msgr2=0x7f04580cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 -- 172.21.15.17:0/3948567651 shutdown_connections 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.024+0000 7f045e113640 1 -- 172.21.15.17:0/3948567651 wait complete. 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.025+0000 7f045e113640 1 Processor -- start 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.025+0000 7f045e113640 1 -- start start 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580ad2b0 0x7f045814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04580c8950 0x7f045806cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458152ba0 con 0x7f04580c69f0 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458152a20 con 0x7f04580c8950 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f045e113640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c58e0 con 0x7f04580ad2b0 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44778/0 (socket says 172.21.15.17:44778) 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 -- 172.21.15.17:0/4028517291 learned_addr learned my addr 172.21.15.17:0/4028517291 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0457fff640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04580c8950 0x7f045806cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f04577fe640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580ad2b0 0x7f045814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 -- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580ad2b0 msgr2=0x7f045814d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580ad2b0 0x7f045814d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 -- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04580c8950 msgr2=0x7f045806cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04580c8950 0x7f045806cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.026+0000 7f0456ffd640 1 -- 172.21.15.17:0/4028517291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04581edd00 con 0x7f04580c69f0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f0456ffd640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e870 tx=0x7f044c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f043ffff640 1 -- 172.21.15.17:0/4028517291 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c017070 con 0x7f04580c69f0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f043ffff640 1 -- 172.21.15.17:0/4028517291 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c00c940 con 0x7f04580c69f0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04581590e0 con 0x7f04580c69f0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f043ffff640 1 -- 172.21.15.17:0/4028517291 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00cd30 con 0x7f04580c69f0 2024-03-06T07:24:15.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.027+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04581ee2a0 con 0x7f04580c69f0 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.028+0000 7f043ffff640 1 -- 172.21.15.17:0/4028517291 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f044c01a070 con 0x7f04580c69f0 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.029+0000 7f043ffff640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 0x7f044406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.029+0000 7f043ffff640 1 -- 172.21.15.17:0/4028517291 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f044c08a200 con 0x7f04580c69f0 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.029+0000 7f04577fe640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 0x7f044406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.029+0000 7f04577fe640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 0x7f044406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0448036000 tx=0x7f044803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.030+0000 7f0460dde740 1 Processor -- start 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.030+0000 7f0460dde740 1 -- start start 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.030+0000 7f0460dde740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.030+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fd85f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.030+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd864b0 0x56123fd82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f0460dde740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fcec6b0 con 0x56123fd87f50 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fc95a60 con 0x56123fd864b0 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56123fce8750 con 0x56123fd88330 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44788/0 (socket says 172.21.15.17:44788) 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 -- 172.21.15.17:0/3424932940 learned_addr learned my addr 172.21.15.17:0/3424932940 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f0457fff640 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd864b0 0x56123fd82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f0456ffd640 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fd85f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 -- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 msgr2=0x56123fd85f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fd85f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 -- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd864b0 msgr2=0x56123fd82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd864b0 0x56123fd82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 -- 172.21.15.17:0/3424932940 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123fcb3f30 con 0x56123fd87f50 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f04577fe640 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0448034040 tx=0x7f0448008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e9abba7e6e674a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f043d7fa640 1 -- 172.21.15.17:0/3424932940 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448055070 con 0x56123fd87f50 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f043d7fa640 1 -- 172.21.15.17:0/3424932940 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448037040 con 0x56123fd87f50 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.031+0000 7f043d7fa640 1 -- 172.21.15.17:0/3424932940 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448032dc0 con 0x56123fd87f50 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 -- 172.21.15.17:0/3424932940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 msgr2=0x56123fd85a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0448034040 tx=0x7f0448008030 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 -- 172.21.15.17:0/3424932940 shutdown_connections 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd864b0 0x56123fd82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fd85f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 --2- 172.21.15.17:0/3424932940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fd85a30 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 -- 172.21.15.17:0/3424932940 >> 172.21.15.17:0/3424932940 conn(0x56123fd847d0 msgr2=0x56123fda93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 -- 172.21.15.17:0/3424932940 shutdown_connections 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.032+0000 7f0460dde740 1 -- 172.21.15.17:0/3424932940 wait complete. 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 -- start start 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 Processor -- start 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fdd5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fdc7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fdaa230 con 0x56123fdd81c0 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fca8ea0 con 0x56123fd88330 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56123fda9c70 con 0x56123fd87f50 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0460dde740 10 client.0 _ll_register_callbacks cb 0x56123fca8870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0457fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fdc7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0457fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44804/0 (socket says 172.21.15.17:44804) 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f04577fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fdd5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.033+0000 7f0457fff640 1 -- 172.21.15.17:0/148239046 learned_addr learned my addr 172.21.15.17:0/148239046 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 -- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 msgr2=0x56123fdd5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fdd5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 -- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 msgr2=0x56123fdc7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fdc7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 -- 172.21.15.17:0/148239046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56123fca9270 con 0x56123fdd81c0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04577fe640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fdd5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0457fff640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0450014cd0 tx=0x7f04500125f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac78d6abb0ca8e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045001c070 con 0x56123fdd81c0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0450003990 con 0x56123fdd81c0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0448037040 con 0x56123fdd81c0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0450027050 con 0x56123fdd81c0 2024-03-06T07:24:15.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0460dde740 10 client.7677 fetch_fsmap learned FSMap version 99 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x56123fa012d0 con 0x56123fdd81c0 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448037040 con 0x56123fdd81c0 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f045002a020 con 0x56123fdd81c0 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.034+0000 7f0460dde740 10 client.7677 fetch_fsmap finished waiting for FSMap version 99 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f0460dde740 10 client.7677 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f0460dde740 20 client.7677 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fda8b20 0x56123fde37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f0460dde740 4 client.7677 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56123fcf9bf0 con 0x56123fda8b20 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f04577fe640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fda8b20 0x56123fde37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f04577fe640 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fda8b20 0x56123fde37d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5c341bb5553b24 server_cookie=4538633fdd177c74 in_seq=0 out_seq=0 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.035+0000 7f04367fc640 10 client.7677 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f04367fc640 1 -- 172.21.15.17:0/148239046 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56123fcf9bf0 con 0x56123fda8b20 2024-03-06T07:24:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f04367fc640 10 client.7677 handle_command_reply: tid=0 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 client.7677 shutdown 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 msgr2=0x56123fdd8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0450014cd0 tx=0x7f04500125f0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 shutdown_connections 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fdd8640 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fda8b20 0x56123fde37d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fdc7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 --2- 172.21.15.17:0/148239046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fdd5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f04367fc640 0 client.7677 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 >> 172.21.15.17:0/148239046 conn(0x56123fd864b0 msgr2=0x56123fda5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 shutdown_connections 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 1 -- 172.21.15.17:0/148239046 wait complete. 2024-03-06T07:24:15.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.036+0000 7f0460dde740 20 client.7677 trim_cache size 0 max 16384 2024-03-06T07:24:15.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 Processor -- start 2024-03-06T07:24:15.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 -- start start 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fda5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fdd81c0 0x56123fdacb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581f7d30 con 0x56123fd87f50 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581f62f0 con 0x56123fdd81c0 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.053+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04581f68f0 con 0x56123fd88330 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44820/0 (socket says 172.21.15.17:44820) 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 -- 172.21.15.17:0/1755421966 learned_addr learned my addr 172.21.15.17:0/1755421966 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f0457fff640 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fdd81c0 0x56123fdacb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f0456ffd640 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fda5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 -- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 msgr2=0x56123fda5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fda5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 -- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fdd81c0 msgr2=0x56123fdacb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fdd81c0 0x56123fdacb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 -- 172.21.15.17:0/1755421966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56123facb6b0 con 0x56123fd87f50 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f04577fe640 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f044802b8c0 tx=0x7f0448030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf57e901ebd043a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f043cff9640 1 -- 172.21.15.17:0/1755421966 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0448055040 con 0x56123fd87f50 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f043cff9640 1 -- 172.21.15.17:0/1755421966 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448037040 con 0x56123fd87f50 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.054+0000 7f043cff9640 1 -- 172.21.15.17:0/1755421966 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044805c950 con 0x56123fd87f50 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- 172.21.15.17:0/1755421966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 msgr2=0x56123fde8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f044802b8c0 tx=0x7f0448030000 comp rx=0 tx=0).stop 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- 172.21.15.17:0/1755421966 shutdown_connections 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fdd81c0 0x56123fdacb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd88330 0x56123fda5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 --2- 172.21.15.17:0/1755421966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fd87f50 0x56123fde8800 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- 172.21.15.17:0/1755421966 >> 172.21.15.17:0/1755421966 conn(0x56123fd864b0 msgr2=0x56123fdd2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- 172.21.15.17:0/1755421966 shutdown_connections 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- 172.21.15.17:0/1755421966 wait complete. 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 -- start start 2024-03-06T07:24:15.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.055+0000 7f0460dde740 1 Processor -- start 2024-03-06T07:24:15.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fe31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fda5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fda89a0 con 0x56123fdd81c0 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56123fdd8cf0 con 0x56123fd88330 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56123fdade40 con 0x56123fd87f50 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0460dde740 10 client.0 _ll_register_callbacks cb 0x7f0450009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fda5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44828/0 (socket says 172.21.15.17:44828) 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 -- 172.21.15.17:0/4119800080 learned_addr learned my addr 172.21.15.17:0/4119800080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f04577fe640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fe31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 -- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 msgr2=0x56123fe31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fe31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 -- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 msgr2=0x56123fda5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fda5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.056+0000 7f0457fff640 1 -- 172.21.15.17:0/4119800080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56123fdac2b0 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0457fff640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0450014cd0 tx=0x7f0450004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c9c5c0ec863af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045002f020 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0450021440 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0448037040 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0450039050 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 10 client.7683 fetch_fsmap learned FSMap version 99 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x56123fdad5c0 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0448037040 con 0x56123fdd81c0 2024-03-06T07:24:15.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f045003c020 con 0x56123fdd81c0 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 10 client.7683 fetch_fsmap finished waiting for FSMap version 99 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 10 client.7683 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 20 client.7683 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fe37b80 0x56123fe37f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 4 client.7683 mds_command: new command op to 6203 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 --> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56123fcf3f40 con 0x56123fe37b80 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.057+0000 7f04577fe640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fe37b80 0x56123fe37f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f04367fc640 10 client.7683 ms_handle_connect on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f04577fe640 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fe37b80 0x56123fe37f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d538964198cd08 server_cookie=79c430da97b93dc in_seq=0 out_seq=0 2024-03-06T07:24:15.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f04367fc640 1 -- 172.21.15.17:0/4119800080 <== mds.0 v2:172.21.15.163:6834/1364627789 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56123fcf3f40 con 0x56123fe37b80 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f04367fc640 10 client.7683 handle_command_reply: tid=0 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 client.7683 shutdown 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 msgr2=0x56123fde7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0450014cd0 tx=0x7f0450004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 shutdown_connections 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56123fdd81c0 0x56123fde7c80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:6834/1364627789,v1:172.21.15.163:6836/1364627789] conn(0x56123fe37b80 0x56123fe37f60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56123fd88330 0x56123fda5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 --2- 172.21.15.17:0/4119800080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56123fd87f50 0x56123fe31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f04367fc640 0 client.7683 ms_handle_reset on v2:172.21.15.163:6834/1364627789 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 >> 172.21.15.17:0/4119800080 conn(0x56123fd864b0 msgr2=0x56123fdc6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 shutdown_connections 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 1 -- 172.21.15.17:0/4119800080 wait complete. 2024-03-06T07:24:15.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.058+0000 7f0460dde740 20 client.7683 trim_cache size 0 max 16384 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 6203, 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 13.503398346000001, 2024-03-06T07:24:15.067 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 98, 2024-03-06T07:24:15.068 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 110, 2024-03-06T07:24:15.068 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 110, 2024-03-06T07:24:15.068 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25.606041624 2024-03-06T07:24:15.068 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 msgr2=0x7f044406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 0x7f044406d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0448036000 tx=0x7f044803d040 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 msgr2=0x7f045806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f044c00e870 tx=0x7f044c00ed30 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 shutdown_connections 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f044406b4d0 0x7f044406d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04580c8950 0x7f045806cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04580c69f0 0x7f045806c5a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 --2- 172.21.15.17:0/4028517291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04580ad2b0 0x7f045814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 >> 172.21.15.17:0/4028517291 conn(0x7f04580cae70 msgr2=0x7f04580c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 shutdown_connections 2024-03-06T07:24:15.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.059+0000 7f045e113640 1 -- 172.21.15.17:0/4028517291 wait complete. 2024-03-06T07:24:15.073 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 8, "epoch": 99, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 8, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:23:53.884146+0000", "damaged": [], "data_pools": [ 18 ], "enabled": true, "epoch": 98, "ever_allowed_features": 32, "explicitly_allowed_features": 32, "failed": [], "flags": 50, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": true, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6203": { "addr": "172.21.15.163:6836/1364627789", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1364627789, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1364627789, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6203, "incarnation": 91, "join_fscid": 8, "name": "d", "rank": 0, "state": "up:active", "state_seq": 7 }, "gid_7021": { "addr": "172.21.15.163:6837/386947795", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 386947795, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 386947795, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7021, "incarnation": 0, "join_fscid": 8, "name": "b", "rank": 0, "state": "up:standby-replay", "state_seq": 6 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 17, "modified": "2024-03-06T07:24:09.583188+0000", "qdb_cluster": [ 6203 ], "qdb_leader": 6203, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6203 } } } ], "standbys": [ { "addr": "172.21.15.17:6834/2673518972", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2673518972, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2673518972, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 86, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7386, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1634180734", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1634180734, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1634180734, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 99, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7659, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:15.074 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 8, 'name': 'b', 'state': 'up:standby-replay'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 8, 'name': 'd', 'state': 'up:active'}] 2024-03-06T07:24:15.075 INFO:tasks.cephfs.test_failover:current = [{'gid': 7659, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1634180734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1634180734}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1634180734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 99}, {'gid': 7021, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 6, 'addr': '172.21.15.163:6837/386947795', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 386947795}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 386947795}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 7386, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/2673518972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2673518972}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2673518972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 86}, {'gid': 6203, 'name': 'd', 'rank': 0, 'incarnation': 91, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.163:6836/1364627789', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1364627789}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1364627789}]}, 'join_fscid': 8, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:24:15.075 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:15.075 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:24:15.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:15.124 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:15.180 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:24:15.180 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:24:15.180 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:15.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:24:15.196 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:24:15.196 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:15.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:24:15.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:15.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 Processor -- start 2024-03-06T07:24:15.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 -- start start 2024-03-06T07:24:15.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b840c5330 0x7f2b840c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b84159630 con 0x7f2b84152920 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b840c7760 con 0x7f2b840c8a00 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b8a4fb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b841587b0 con 0x7f2b840c5330 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b88a98640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b840c5330 0x7f2b840c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b83fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2b84152920 0x7f2b84155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58884/0 (socket says 172.21.15.17:58884) 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.335+0000 7f2b83fff640 1 -- 172.21.15.17:0/3045238245 learned_addr learned my addr 172.21.15.17:0/3045238245 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.336+0000 7f2b88a98640 1 -- 172.21.15.17:0/3045238245 >> v1:172.21.15.163:6789/0 conn(0x7f2b840c8a00 legacy=0x7f2b84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.336+0000 7f2b88a98640 1 -- 172.21.15.17:0/3045238245 >> v1:172.21.15.17:6789/0 conn(0x7f2b84152920 legacy=0x7f2b84155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.336+0000 7f2b88a98640 1 -- 172.21.15.17:0/3045238245 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840c7a40 con 0x7f2b840c5330 2024-03-06T07:24:15.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.336+0000 7f2b82ffd640 1 -- 172.21.15.17:0/3045238245 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592605349 0 0) 0x7f2b84159630 con 0x7f2b84152920 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b88a98640 1 --2- 172.21.15.17:0/3045238245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c5330 0x7f2b840c57a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009d90 tx=0x7f2b7c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778019c7183a0b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b82ffd640 1 -- 172.21.15.17:0/3045238245 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c039070 con 0x7f2b840c5330 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b82ffd640 1 -- 172.21.15.17:0/3045238245 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c02cc60 con 0x7f2b840c5330 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b82ffd640 1 -- 172.21.15.17:0/3045238245 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034660 con 0x7f2b840c5330 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3045238245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c5330 msgr2=0x7f2b840c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3045238245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c5330 0x7f2b840c57a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009d90 tx=0x7f2b7c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3045238245 shutdown_connections 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3045238245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c5330 0x7f2b840c57a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3045238245 >> 172.21.15.17:0/3045238245 conn(0x7f2b840ac560 msgr2=0x7f2b840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3045238245 shutdown_connections 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.337+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3045238245 wait complete. 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.338+0000 7f2b8a4fb640 1 Processor -- start 2024-03-06T07:24:15.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 -- start start 2024-03-06T07:24:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b840c5330 0x7f2b84156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b84152920 0x7f2b840ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8406d030 con 0x7f2b840c8a00 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84158a00 con 0x7f2b840c5330 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b8a4fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840c7390 con 0x7f2b84152920 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44844/0 (socket says 172.21.15.17:44844) 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b837fe640 1 -- 172.21.15.17:0/3785266117 learned_addr learned my addr 172.21.15.17:0/3785266117 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b83fff640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b840c5330 0x7f2b84156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.339+0000 7f2b88a98640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b84152920 0x7f2b840ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 -- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b84152920 msgr2=0x7f2b840ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b84152920 0x7f2b840ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 -- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b840c5330 msgr2=0x7f2b84156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b840c5330 0x7f2b84156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 -- 172.21.15.17:0/3785266117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840aa550 con 0x7f2b840c8a00 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b837fe640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c430 tx=0x7f2b7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78017070 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b780052d0 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78003830 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b840aa830 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.340+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b840ab910 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.341+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8404f330 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.341+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b7801a070 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.342+0000 7f2b80ff9640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 0x7f2b7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.342+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2b78089620 con 0x7f2b840c8a00 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.342+0000 7f2b83fff640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 0x7f2b7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.343+0000 7f2b83fff640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 0x7f2b7006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007f70 tx=0x7f2b74007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.346+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b78057d00 con 0x7f2b840c8a00 2024-03-06T07:24:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.567+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b84156390 con 0x7f2b840c8a00 2024-03-06T07:24:15.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.568+0000 7f2b80ff9640 1 -- 172.21.15.17:0/3785266117 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4987 (secure 0 0 0) 0x7f2b7805bda0 con 0x7f2b840c8a00 2024-03-06T07:24:15.570 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:15.570 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:09.583188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7021":{"gid":7021,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":6,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:15.570 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 99 2024-03-06T07:24:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 msgr2=0x7f2b7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 0x7f2b7006d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007f70 tx=0x7f2b74007f00 comp rx=0 tx=0).stop 2024-03-06T07:24:15.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 msgr2=0x7f2b84157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c430 tx=0x7f2b7800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 shutdown_connections 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b7006b4d0 0x7f2b7006d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b84152920 0x7f2b840ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b840c8a00 0x7f2b84157930 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 --2- 172.21.15.17:0/3785266117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b840c5330 0x7f2b84156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 >> 172.21.15.17:0/3785266117 conn(0x7f2b840ac560 msgr2=0x7f2b840c8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 shutdown_connections 2024-03-06T07:24:15.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.570+0000 7f2b8a4fb640 1 -- 172.21.15.17:0/3785266117 wait complete. 2024-03-06T07:24:15.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:15.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.682+0000 7febe4bd7640 1 Processor -- start 2024-03-06T07:24:15.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.682+0000 7febe4bd7640 1 -- start start 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febe4bd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe014f360 0x7febe0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febe4bd7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febe0158b60 0x7febe0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febe4bd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00c73d0 con 0x7febe00c88c0 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febe4bd7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe00c8dd0 con 0x7febe0158b60 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febe4bd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00be2e0 con 0x7febe014f360 2024-03-06T07:24:15.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febded9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febe0158b60 0x7febe0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febded9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7febe0158b60 0x7febe0158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44854/0 (socket says 172.21.15.17:44854) 2024-03-06T07:24:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febded9d640 1 -- 172.21.15.17:0/2194190170 learned_addr learned my addr 172.21.15.17:0/2194190170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.683+0000 7febde59c640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe014f360 0x7febe0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febded9d640 1 -- 172.21.15.17:0/2194190170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe014f360 msgr2=0x7febe0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febded9d640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe014f360 0x7febe0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febded9d640 1 -- 172.21.15.17:0/2194190170 >> v1:172.21.15.163:6789/0 conn(0x7febe00c88c0 legacy=0x7febe00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febded9d640 1 -- 172.21.15.17:0/2194190170 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00c7670 con 0x7febe0158b60 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febded9d640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe0158b60 0x7febe0158ff0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7febd8009e50 tx=0x7febd802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30b875127965542 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febddd9b640 1 -- 172.21.15.17:0/2194190170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd802ca70 con 0x7febe0158b60 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febddd9b640 1 -- 172.21.15.17:0/2194190170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd802cc10 con 0x7febe0158b60 2024-03-06T07:24:15.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.684+0000 7febddd9b640 1 -- 172.21.15.17:0/2194190170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd80343f0 con 0x7febe0158b60 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 -- 172.21.15.17:0/2194190170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe0158b60 msgr2=0x7febe0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe0158b60 0x7febe0158ff0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7febd8009e50 tx=0x7febd802bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 -- 172.21.15.17:0/2194190170 shutdown_connections 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe0158b60 0x7febe0158ff0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 --2- 172.21.15.17:0/2194190170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe014f360 0x7febe0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 -- 172.21.15.17:0/2194190170 >> 172.21.15.17:0/2194190170 conn(0x7febe00ac570 msgr2=0x7febe00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 -- 172.21.15.17:0/2194190170 shutdown_connections 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.685+0000 7febe4bd7640 1 -- 172.21.15.17:0/2194190170 wait complete. 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.686+0000 7febe4bd7640 1 Processor -- start 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.686+0000 7febe4bd7640 1 -- start start 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88c0 0x7febe00c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe0158b60 0x7febe00c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00c5a50 con 0x7febe014f360 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00c7c30 con 0x7febe0158b60 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febe4bd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00c5bd0 con 0x7febe00c88c0 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44866/0 (socket says 172.21.15.17:44866) 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 -- 172.21.15.17:0/3845334563 learned_addr learned my addr 172.21.15.17:0/3845334563 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febded9d640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe0158b60 0x7febe00c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febde59c640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88c0 0x7febe00c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 -- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88c0 msgr2=0x7febe00c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88c0 0x7febe00c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 -- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe0158b60 msgr2=0x7febe00c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe0158b60 0x7febe00c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.687+0000 7febd7fff640 1 -- 172.21.15.17:0/3845334563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe01587a0 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febd7fff640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7febc800c470 tx=0x7febc800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80052d0 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8005490 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0157380 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8007260 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.688+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe01584c0 con 0x7febe014f360 2024-03-06T07:24:15.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.689+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe004f330 con 0x7febe014f360 2024-03-06T07:24:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.689+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febc8007400 con 0x7febe014f360 2024-03-06T07:24:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.689+0000 7febd6ffd640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 0x7febc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.690+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7febc8089720 con 0x7febe014f360 2024-03-06T07:24:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.690+0000 7febde59c640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 0x7febc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.690+0000 7febde59c640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 0x7febc006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febe0155b50 tx=0x7febd0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:15.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.694+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febc8057e00 con 0x7febe014f360 2024-03-06T07:24:15.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.875+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febe00b0680 con 0x7febe014f360 2024-03-06T07:24:15.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.876+0000 7febd6ffd640 1 -- 172.21.15.17:0/3845334563 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17077 (secure 0 0 0) 0x7febc805bea0 con 0x7febe014f360 2024-03-06T07:24:15.878 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:15.878 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":110,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:01.553807+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/75092648":"2024-03-07T07:24:01.553761+0000","172.21.15.17:6833/75092648":"2024-03-07T07:24:01.553761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:15.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 msgr2=0x7febc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 0x7febc006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7febe0155b50 tx=0x7febd0009190 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 msgr2=0x7febe0157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7febc800c470 tx=0x7febc800c930 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 shutdown_connections 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febc006b4d0 0x7febc006d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febe0158b60 0x7febe00c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febe014f360 0x7febe0157b80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 --2- 172.21.15.17:0/3845334563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febe00c88c0 0x7febe00c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 >> 172.21.15.17:0/3845334563 conn(0x7febe00ac570 msgr2=0x7febe014cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 shutdown_connections 2024-03-06T07:24:15.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.879+0000 7febe4bd7640 1 -- 172.21.15.17:0/3845334563 wait complete. 2024-03-06T07:24:15.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:23:52.140520+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:23:52.842527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:15.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:15.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.994+0000 7fc7542b5640 1 Processor -- start 2024-03-06T07:24:15.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.994+0000 7fc7542b5640 1 -- start start 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc74c1529f0 0x7fc74c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c0c5370 0x7fc74c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc74c155530 0x7fc74c1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c159710 con 0x7fc74c1529f0 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0c68b0 con 0x7fc74c155530 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc7542b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c152dd0 con 0x7fc74c0c5370 2024-03-06T07:24:15.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc74c1529f0 0x7fc74c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc74c1529f0 0x7fc74c0c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44870/0 (socket says 172.21.15.17:44870) 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 -- 172.21.15.17:0/1813551038 learned_addr learned my addr 172.21.15.17:0/1813551038 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc751850640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c0c5370 0x7fc74c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752852640 1 --2- 172.21.15.17:0/1813551038 >> v2:172.21.15.163:3300/0 conn(0x7fc74c155530 0x7fc74c1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 -- 172.21.15.17:0/1813551038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c0c5370 msgr2=0x7fc74c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c0c5370 0x7fc74c0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 -- 172.21.15.17:0/1813551038 >> v2:172.21.15.163:3300/0 conn(0x7fc74c155530 msgr2=0x7fc74c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 --2- 172.21.15.17:0/1813551038 >> v2:172.21.15.163:3300/0 conn(0x7fc74c155530 0x7fc74c1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.995+0000 7fc752051640 1 -- 172.21.15.17:0/1813551038 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c158b00 con 0x7fc74c1529f0 2024-03-06T07:24:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.996+0000 7fc752051640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c1529f0 0x7fc74c0c88f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc74c067ca0 tx=0x7fc74402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314c4cd2cb5ee3dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.996+0000 7fc75104f640 1 -- 172.21.15.17:0/1813551038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74402e070 con 0x7fc74c1529f0 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.996+0000 7fc75104f640 1 -- 172.21.15.17:0/1813551038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc74c1529f0 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.996+0000 7fc75104f640 1 -- 172.21.15.17:0/1813551038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744031a00 con 0x7fc74c1529f0 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 -- 172.21.15.17:0/1813551038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c1529f0 msgr2=0x7fc74c0c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c1529f0 0x7fc74c0c88f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc74c067ca0 tx=0x7fc74402b880 comp rx=0 tx=0).stop 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 -- 172.21.15.17:0/1813551038 shutdown_connections 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 --2- 172.21.15.17:0/1813551038 >> v2:172.21.15.163:3300/0 conn(0x7fc74c155530 0x7fc74c1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c0c5370 0x7fc74c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 --2- 172.21.15.17:0/1813551038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c1529f0 0x7fc74c0c88f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 -- 172.21.15.17:0/1813551038 >> 172.21.15.17:0/1813551038 conn(0x7fc74c0ac580 msgr2=0x7fc74c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 -- 172.21.15.17:0/1813551038 shutdown_connections 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.997+0000 7fc7542b5640 1 -- 172.21.15.17:0/1813551038 wait complete. 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.998+0000 7fc7542b5640 1 Processor -- start 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.998+0000 7fc7542b5640 1 -- start start 2024-03-06T07:24:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c1529f0 0x7fc74c147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc74c155530 0x7fc74c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c1593a0 con 0x7fc74c0c5370 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c1590c0 con 0x7fc74c155530 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c158740 con 0x7fc74c1529f0 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44878/0 (socket says 172.21.15.17:44878) 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 -- 172.21.15.17:0/3354922996 learned_addr learned my addr 172.21.15.17:0/3354922996 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc751850640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c1529f0 0x7fc74c147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752852640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc74c155530 0x7fc74c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 -- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c1529f0 msgr2=0x7fc74c147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c1529f0 0x7fc74c147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 -- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc74c155530 msgr2=0x7fc74c1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc74c155530 0x7fc74c1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 -- 172.21.15.17:0/3354922996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c1eeec0 con 0x7fc74c0c5370 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc752051640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc74402b8c0 tx=0x7fc7440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744032670 con 0x7fc74c0c5370 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744032830 con 0x7fc74c0c5370 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c1eebe0 con 0x7fc74c0c5370 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:15.999+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744032c20 con 0x7fc74c0c5370 2024-03-06T07:24:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.000+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c157f70 con 0x7fc74c0c5370 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.001+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c04f330 con 0x7fc74c0c5370 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.001+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7440453f0 con 0x7fc74c0c5370 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.001+0000 7fc742ffd640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.001+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc7440ac1f0 con 0x7fc74c0c5370 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.001+0000 7fc751850640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.002+0000 7fc751850640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 0x7fc73006d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0b0ac0 tx=0x7fc73c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.006+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc744040070 con 0x7fc74c0c5370 2024-03-06T07:24:16.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.187+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc74c06ccb0 con 0x7fc74c0c5370 2024-03-06T07:24:16.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.188+0000 7fc742ffd640 1 -- 172.21.15.17:0/3354922996 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc74407a9f0 con 0x7fc74c0c5370 2024-03-06T07:24:16.189 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 msgr2=0x7fc73006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 0x7fc73006d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0b0ac0 tx=0x7fc73c009190 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 msgr2=0x7fc74c154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc74402b8c0 tx=0x7fc7440046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 shutdown_connections 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc74c155530 0x7fc74c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc74c1529f0 0x7fc74c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 --2- 172.21.15.17:0/3354922996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc74c0c5370 0x7fc74c154240 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 >> 172.21.15.17:0/3354922996 conn(0x7fc74c0ac580 msgr2=0x7fc74c14e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 shutdown_connections 2024-03-06T07:24:16.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.190+0000 7fc7542b5640 1 -- 172.21.15.17:0/3354922996 wait complete. 2024-03-06T07:24:16.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:24:16.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.305+0000 7fce4b3d0640 1 Processor -- start 2024-03-06T07:24:16.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.305+0000 7fce4b3d0640 1 -- start start 2024-03-06T07:24:16.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4b3d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 0x7fce440c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4b3d0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4b3d0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce44072600 con 0x7fce440c7720 2024-03-06T07:24:16.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4b3d0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce440723b0 con 0x7fce440c5c60 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4b3d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440728e0 con 0x7fce440c5880 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4896b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4916c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 0x7fce440c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4896b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:39896/0 (socket says 172.21.15.17:39896) 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4896b640 1 -- 172.21.15.17:0/2121267772 learned_addr learned my addr 172.21.15.17:0/2121267772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4996d640 1 --1- 172.21.15.17:0/2121267772 >> v1:172.21.15.163:6789/0 conn(0x7fce440c7720 0x7fce4406f4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:34800/0 (socket says 172.21.15.17:34800) 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4916c640 1 -- 172.21.15.17:0/2121267772 >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 msgr2=0x7fce440c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4916c640 1 --2- 172.21.15.17:0/2121267772 >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4916c640 1 -- 172.21.15.17:0/2121267772 >> v1:172.21.15.163:6789/0 conn(0x7fce440c7720 legacy=0x7fce4406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4916c640 1 -- 172.21.15.17:0/2121267772 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44153af0 con 0x7fce440c5880 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce3bfff640 1 -- 172.21.15.17:0/2121267772 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275509338 0 0) 0x7fce44072600 con 0x7fce440c7720 2024-03-06T07:24:16.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.306+0000 7fce4896b640 1 --2- 172.21.15.17:0/2121267772 >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.307+0000 7fce4916c640 1 --2- 172.21.15.17:0/2121267772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 0x7fce440c6ca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0097d0 tx=0x7fce3c02b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38912840218f6178 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.307+0000 7fce3bfff640 1 -- 172.21.15.17:0/2121267772 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c038070 con 0x7fce440c5880 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.307+0000 7fce3bfff640 1 -- 172.21.15.17:0/2121267772 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c031700 con 0x7fce440c5880 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.307+0000 7fce3bfff640 1 -- 172.21.15.17:0/2121267772 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c031ad0 con 0x7fce440c5880 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 -- 172.21.15.17:0/2121267772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 msgr2=0x7fce440c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/2121267772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 0x7fce440c6ca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0097d0 tx=0x7fce3c02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 -- 172.21.15.17:0/2121267772 shutdown_connections 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/2121267772 >> v2:172.21.15.163:3300/0 conn(0x7fce440c5c60 0x7fce440c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/2121267772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5880 0x7fce440c6ca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 -- 172.21.15.17:0/2121267772 >> 172.21.15.17:0/2121267772 conn(0x7fce440aa630 msgr2=0x7fce440aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 -- 172.21.15.17:0/2121267772 shutdown_connections 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.308+0000 7fce4b3d0640 1 -- 172.21.15.17:0/2121267772 wait complete. 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.309+0000 7fce4b3d0640 1 Processor -- start 2024-03-06T07:24:16.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.309+0000 7fce4b3d0640 1 -- start start 2024-03-06T07:24:16.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5c60 0x7fce441e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c7720 0x7fce441e9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce44073190 con 0x7fce440c5880 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c6780 con 0x7fce440c7720 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4b3d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce44073310 con 0x7fce440c5c60 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44886/0 (socket says 172.21.15.17:44886) 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 -- 172.21.15.17:0/3748488055 learned_addr learned my addr 172.21.15.17:0/3748488055 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4996d640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c7720 0x7fce441e9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4896b640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5c60 0x7fce441e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 -- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5c60 msgr2=0x7fce441e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5c60 0x7fce441e9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 -- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c7720 msgr2=0x7fce441e9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c7720 0x7fce441e9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 -- 172.21.15.17:0/3748488055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440ad5f0 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.310+0000 7fce4916c640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fce3c000c00 tx=0x7fce3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.311+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c043070 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.311+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c03f870 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.311+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce440ae200 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.311+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c038040 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.311+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441ec980 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.312+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4404f330 con 0x7fce440c5880 2024-03-06T07:24:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.312+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce3c046070 con 0x7fce440c5880 2024-03-06T07:24:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.313+0000 7fce39ffb640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.313+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce3c0abfb0 con 0x7fce440c5880 2024-03-06T07:24:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.313+0000 7fce4896b640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.313+0000 7fce4896b640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 0x7fce2806d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fce440ad320 tx=0x7fce34009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.317+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fce440ad5f0 con 0x7fce440c5880 2024-03-06T07:24:16.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.500+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce44074700 con 0x7fce440c5880 2024-03-06T07:24:16.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.500+0000 7fce39ffb640 1 -- 172.21.15.17:0/3748488055 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce3c030070 con 0x7fce440c5880 2024-03-06T07:24:16.502 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:24:16.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 msgr2=0x7fce2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 0x7fce2806d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fce440ad320 tx=0x7fce34009240 comp rx=0 tx=0).stop 2024-03-06T07:24:16.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 msgr2=0x7fce441ebd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fce3c000c00 tx=0x7fce3c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 shutdown_connections 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fce2806b4d0 0x7fce2806d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce440c7720 0x7fce441e9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce440c5c60 0x7fce441e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.502+0000 7fce4b3d0640 1 --2- 172.21.15.17:0/3748488055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce440c5880 0x7fce441ebd60 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.503+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 >> 172.21.15.17:0/3748488055 conn(0x7fce440aa630 msgr2=0x7fce440a9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.503+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 shutdown_connections 2024-03-06T07:24:16.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.503+0000 7fce4b3d0640 1 -- 172.21.15.17:0/3748488055 wait complete. 2024-03-06T07:24:16.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:24:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.618+0000 7f2bfdec9640 1 Processor -- start 2024-03-06T07:24:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.618+0000 7f2bfdec9640 1 -- start start 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bfdec9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2bf80b00d0 0x7f2bf80c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bfdec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 0x7f2bf80c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bfdec9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf80b04b0 con 0x7f2bf806b8e0 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bfdec9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf8156160 con 0x7f2bf80b00d0 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bfdec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c5710 con 0x7f2bf80c7210 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2bf80b00d0 0x7f2bf80c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2bf806b8e0 0x7f2bf806bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58894/0 (socket says 172.21.15.17:58894) 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf6ffd640 1 -- 172.21.15.17:0/3902133213 learned_addr learned my addr 172.21.15.17:0/3902133213 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf7fff640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 0x7f2bf80c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf77fe640 1 -- 172.21.15.17:0/3902133213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 msgr2=0x7f2bf80c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf77fe640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 0x7f2bf80c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.619+0000 7f2bf77fe640 1 -- 172.21.15.17:0/3902133213 >> v1:172.21.15.17:6789/0 conn(0x7f2bf806b8e0 legacy=0x7f2bf806bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf77fe640 1 -- 172.21.15.17:0/3902133213 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf806d210 con 0x7f2bf80b00d0 2024-03-06T07:24:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf7fff640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 0x7f2bf80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf77fe640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c8ff0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2be80097a0 tx=0x7f2be802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e25e7657f270b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf67fc640 1 -- 172.21.15.17:0/3902133213 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8035020 con 0x7f2bf80b00d0 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf67fc640 1 -- 172.21.15.17:0/3902133213 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8031830 con 0x7f2bf80b00d0 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.620+0000 7f2bf67fc640 1 -- 172.21.15.17:0/3902133213 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8031c00 con 0x7f2bf80b00d0 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 -- 172.21.15.17:0/3902133213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 msgr2=0x7f2bf80c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c8ff0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2be80097a0 tx=0x7f2be802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 -- 172.21.15.17:0/3902133213 shutdown_connections 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf80c7210 0x7f2bf80c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/3902133213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c8ff0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 -- 172.21.15.17:0/3902133213 >> 172.21.15.17:0/3902133213 conn(0x7f2bf80ac810 msgr2=0x7f2bf80acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 -- 172.21.15.17:0/3902133213 shutdown_connections 2024-03-06T07:24:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.621+0000 7f2bfdec9640 1 -- 172.21.15.17:0/3902133213 wait complete. 2024-03-06T07:24:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.622+0000 7f2bfdec9640 1 Processor -- start 2024-03-06T07:24:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.622+0000 7f2bfdec9640 1 -- start start 2024-03-06T07:24:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf806b8e0 0x7f2bf80c2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf80c7210 0x7f2bf8154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c5ba0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf8152c50 con 0x7f2bf80c7210 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bfdec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c5d20 con 0x7f2bf806b8e0 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44894/0 (socket says 172.21.15.17:44894) 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 -- 172.21.15.17:0/837495557 learned_addr learned my addr 172.21.15.17:0/837495557 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf77fe640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf806b8e0 0x7f2bf80c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf7fff640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf80c7210 0x7f2bf8154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 -- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf806b8e0 msgr2=0x7f2bf80c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf806b8e0 0x7f2bf80c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 -- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf80c7210 msgr2=0x7f2bf8154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf80c7210 0x7f2bf8154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 -- 172.21.15.17:0/837495557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf81500c0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.623+0000 7f2bf6ffd640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c3c0 tx=0x7f2bec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.624+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec017070 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.624+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec005ce0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.624+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf814fde0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.624+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec007310 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.624+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf80c20b0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.625+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf804f330 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.625+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bec01a070 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.625+0000 7f2bdffff640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 0x7f2be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.626+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2bec0896f0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.626+0000 7f2bf77fe640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 0x7f2be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.626+0000 7f2bf77fe640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 0x7f2be406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2be8036000 tx=0x7f2be8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.630+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2bec057cb0 con 0x7f2bf80b00d0 2024-03-06T07:24:16.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.808+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bf80c2280 con 0x7f2bf80b00d0 2024-03-06T07:24:16.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.808+0000 7f2bdffff640 1 -- 172.21.15.17:0/837495557 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bec05bd50 con 0x7f2bf80b00d0 2024-03-06T07:24:16.810 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 msgr2=0x7f2be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 0x7f2be406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2be8036000 tx=0x7f2be8008030 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 msgr2=0x7f2bf80c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00c3c0 tx=0x7f2bec00c880 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 shutdown_connections 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2be406b4d0 0x7f2be406d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bf80c7210 0x7f2bf8154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2bf80b00d0 0x7f2bf80c2cc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 --2- 172.21.15.17:0/837495557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bf806b8e0 0x7f2bf80c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 >> 172.21.15.17:0/837495557 conn(0x7f2bf80ac810 msgr2=0x7f2bf80ab170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 shutdown_connections 2024-03-06T07:24:16.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.810+0000 7f2bfdec9640 1 -- 172.21.15.17:0/837495557 wait complete. 2024-03-06T07:24:16.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:16.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.921+0000 7fa45f765640 1 Processor -- start 2024-03-06T07:24:16.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.921+0000 7fa45f765640 1 -- start start 2024-03-06T07:24:16.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45f765640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa458158db0 0x7fa4580c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45f765640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa458155110 con 0x7fa4580c5330 2024-03-06T07:24:16.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45f765640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4580c8f40 con 0x7fa4580c88b0 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45f765640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4581559e0 con 0x7fa458158db0 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45d501640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa4580c5330 0x7fa4581586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58908/0 (socket says 172.21.15.17:58908) 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45d501640 1 -- 172.21.15.17:0/802147142 learned_addr learned my addr 172.21.15.17:0/802147142 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa45cd00640 1 --2- 172.21.15.17:0/802147142 >> v2:172.21.15.163:3300/0 conn(0x7fa458158db0 0x7fa4580c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.922+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312074911 0 0) 0x7fa450001110 con 0x7fa4580c5330 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa440003520 con 0x7fa4580c5330 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa45cd00640 1 -- 172.21.15.17:0/802147142 >> v1:172.21.15.163:6789/0 conn(0x7fa4580c88b0 legacy=0x7fa45815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa45cd00640 1 -- 172.21.15.17:0/802147142 >> v1:172.21.15.17:6789/0 conn(0x7fa4580c5330 legacy=0x7fa4581586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665087760 0 0) 0x7fa4580c8f40 con 0x7fa4580c88b0 2024-03-06T07:24:16.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa45cd00640 1 -- 172.21.15.17:0/802147142 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458155390 con 0x7fa458158db0 2024-03-06T07:24:16.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.923+0000 7fa45cd00640 1 --2- 172.21.15.17:0/802147142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa458158db0 0x7fa4580c5710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0097a0 tx=0x7fa44c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267f31a7b72dbbb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c035020 con 0x7fa458158db0 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c031830 con 0x7fa458158db0 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa457fff640 1 -- 172.21.15.17:0/802147142 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c031c00 con 0x7fa458158db0 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 -- 172.21.15.17:0/802147142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa458158db0 msgr2=0x7fa4580c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 --2- 172.21.15.17:0/802147142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa458158db0 0x7fa4580c5710 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0097a0 tx=0x7fa44c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 -- 172.21.15.17:0/802147142 shutdown_connections 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 --2- 172.21.15.17:0/802147142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa458158db0 0x7fa4580c5710 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 -- 172.21.15.17:0/802147142 >> 172.21.15.17:0/802147142 conn(0x7fa4580ac560 msgr2=0x7fa4580ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 -- 172.21.15.17:0/802147142 shutdown_connections 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.924+0000 7fa45f765640 1 -- 172.21.15.17:0/802147142 wait complete. 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.925+0000 7fa45f765640 1 Processor -- start 2024-03-06T07:24:16.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.925+0000 7fa45f765640 1 -- start start 2024-03-06T07:24:16.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4580c5330 0x7fa45814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4581ec250 0x7fa45814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4581529d0 con 0x7fa4580c88b0 2024-03-06T07:24:16.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45806d090 con 0x7fa4580c5330 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45f765640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa458152b50 con 0x7fa4581ec250 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44906/0 (socket says 172.21.15.17:44906) 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 -- 172.21.15.17:0/821768326 learned_addr learned my addr 172.21.15.17:0/821768326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45d501640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4580c5330 0x7fa45814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45dd02640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4581ec250 0x7fa45814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 -- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4581ec250 msgr2=0x7fa45814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4581ec250 0x7fa45814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 -- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4580c5330 msgr2=0x7fa45814e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4580c5330 0x7fa45814e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 -- 172.21.15.17:0/821768326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4580bdb60 con 0x7fa4580c88b0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.926+0000 7fa45cd00640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa44c034040 tx=0x7fa44c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.927+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c040070 con 0x7fa4580c88b0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.927+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c03a8f0 con 0x7fa4580c88b0 2024-03-06T07:24:16.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.927+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c035040 con 0x7fa4580c88b0 2024-03-06T07:24:16.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.927+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45814f4c0 con 0x7fa4580c88b0 2024-03-06T07:24:16.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.927+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45814db00 con 0x7fa4580c88b0 2024-03-06T07:24:16.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.928+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4581521a0 con 0x7fa4580c88b0 2024-03-06T07:24:16.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.928+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa44c038020 con 0x7fa4580c88b0 2024-03-06T07:24:16.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.929+0000 7fa455ffb640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 0x7fa44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:16.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.929+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa44c0abb00 con 0x7fa4580c88b0 2024-03-06T07:24:16.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.929+0000 7fa45d501640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 0x7fa44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:16.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.929+0000 7fa45d501640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 0x7fa44406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa450005880 tx=0x7fa450009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:16.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:16.933+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa4581521a0 con 0x7fa4580c88b0 2024-03-06T07:24:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.150+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa45804f330 con 0x7fa4580c88b0 2024-03-06T07:24:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.151+0000 7fa455ffb640 1 -- 172.21.15.17:0/821768326 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4987 (secure 0 0 0) 0x7fa44c07a1e0 con 0x7fa4580c88b0 2024-03-06T07:24:17.153 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:17.153 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:09.583188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7021":{"gid":7021,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":6,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 99 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 msgr2=0x7fa44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 0x7fa44406d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa450005880 tx=0x7fa450009040 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 msgr2=0x7fa4581ebd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa44c034040 tx=0x7fa44c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 shutdown_connections 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa44406b4d0 0x7fa44406d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4581ec250 0x7fa45814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4580c88b0 0x7fa4581ebd10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 --2- 172.21.15.17:0/821768326 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4580c5330 0x7fa45814e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 >> 172.21.15.17:0/821768326 conn(0x7fa4580ac560 msgr2=0x7fa458153fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 shutdown_connections 2024-03-06T07:24:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.153+0000 7fa45f765640 1 -- 172.21.15.17:0/821768326 wait complete. 2024-03-06T07:24:17.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.264+0000 7fabe044e640 1 Processor -- start 2024-03-06T07:24:17.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.264+0000 7fabe044e640 1 -- start start 2024-03-06T07:24:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabe044e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fabd8150580 0x7fabd806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabe044e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8153a60 con 0x7fabd80c6000 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabe044e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd8153d40 con 0x7fabd80c1320 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabe044e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd806c2c0 con 0x7fabd8150580 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabde9eb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fabd80c6000 0x7fabd8156dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58910/0 (socket says 172.21.15.17:58910) 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.265+0000 7fabde9eb640 1 -- 172.21.15.17:0/895592136 learned_addr learned my addr 172.21.15.17:0/895592136 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabde1ea640 1 --2- 172.21.15.17:0/895592136 >> v2:172.21.15.163:3300/0 conn(0x7fabd8150580 0x7fabd806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697241404 0 0) 0x7fabd8153a60 con 0x7fabd80c6000 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc0003520 con 0x7fabd80c6000 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359064178 0 0) 0x7fabd8153d40 con 0x7fabd80c1320 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8153a60 con 0x7fabd80c1320 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabde1ea640 1 -- 172.21.15.17:0/895592136 >> v1:172.21.15.163:6789/0 conn(0x7fabd80c1320 legacy=0x7fabd81566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabde1ea640 1 -- 172.21.15.17:0/895592136 >> v1:172.21.15.17:6789/0 conn(0x7fabd80c6000 legacy=0x7fabd8156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.266+0000 7fabde1ea640 1 -- 172.21.15.17:0/895592136 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd806c5a0 con 0x7fabd8150580 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabde1ea640 1 --2- 172.21.15.17:0/895592136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd8150580 0x7fabd806dd10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fabd00097a0 tx=0x7fabd002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42bf7c594827597 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0035020 con 0x7fabd8150580 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd0031830 con 0x7fabd8150580 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabdd1e8640 1 -- 172.21.15.17:0/895592136 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0031c00 con 0x7fabd8150580 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 -- 172.21.15.17:0/895592136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd8150580 msgr2=0x7fabd806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 --2- 172.21.15.17:0/895592136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd8150580 0x7fabd806dd10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fabd00097a0 tx=0x7fabd002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 -- 172.21.15.17:0/895592136 shutdown_connections 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 --2- 172.21.15.17:0/895592136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd8150580 0x7fabd806dd10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 -- 172.21.15.17:0/895592136 >> 172.21.15.17:0/895592136 conn(0x7fabd80c6710 msgr2=0x7fabd80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 -- 172.21.15.17:0/895592136 shutdown_connections 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.267+0000 7fabe044e640 1 -- 172.21.15.17:0/895592136 wait complete. 2024-03-06T07:24:17.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.268+0000 7fabe044e640 1 Processor -- start 2024-03-06T07:24:17.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 -- start start 2024-03-06T07:24:17.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabd80c1320 0x7fabd80bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd80c6000 0x7fabd80ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c0dc0 con 0x7fabd8150580 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c0c40 con 0x7fabd80c6000 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabe044e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd806e250 con 0x7fabd80c1320 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabde9eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabde9eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44910/0 (socket says 172.21.15.17:44910) 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabde9eb640 1 -- 172.21.15.17:0/2027194683 learned_addr learned my addr 172.21.15.17:0/2027194683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabdd9e9640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd80c6000 0x7fabd80ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.269+0000 7fabde1ea640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabd80c1320 0x7fabd80bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 -- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabd80c1320 msgr2=0x7fabd80bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabd80c1320 0x7fabd80bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 -- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd80c6000 msgr2=0x7fabd80ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd80c6000 0x7fabd80ab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 -- 172.21.15.17:0/2027194683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8151c00 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabde9eb640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fabc800cbd0 tx=0x7fabc800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8017070 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc800ace0 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8012620 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd81eea20 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.270+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81eefe0 con 0x7fabd8150580 2024-03-06T07:24:17.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.271+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd804f330 con 0x7fabd8150580 2024-03-06T07:24:17.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.271+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabc801a070 con 0x7fabd8150580 2024-03-06T07:24:17.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.272+0000 7fabd6ffd640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 0x7fabc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.272+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fabc808a400 con 0x7fabd8150580 2024-03-06T07:24:17.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.272+0000 7fabde1ea640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 0x7fabc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.273+0000 7fabde1ea640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 0x7fabc406d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fabd0036000 tx=0x7fabd003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.276+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabc80589c0 con 0x7fabd8150580 2024-03-06T07:24:17.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.457+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd80c5de0 con 0x7fabd8150580 2024-03-06T07:24:17.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.458+0000 7fabd6ffd640 1 -- 172.21.15.17:0/2027194683 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17077 (secure 0 0 0) 0x7fabc805ca60 con 0x7fabd8150580 2024-03-06T07:24:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:17.461 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":110,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:01.553807+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/75092648":"2024-03-07T07:24:01.553761+0000","172.21.15.17:6833/75092648":"2024-03-07T07:24:01.553761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 msgr2=0x7fabc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 0x7fabc406d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fabd0036000 tx=0x7fabd003c040 comp rx=0 tx=0).stop 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 msgr2=0x7fabd80a9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fabc800cbd0 tx=0x7fabc800a430 comp rx=0 tx=0).stop 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 shutdown_connections 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabc406b4d0 0x7fabc406d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd8150580 0x7fabd80a9ac0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabd80c6000 0x7fabd80ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 --2- 172.21.15.17:0/2027194683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabd80c1320 0x7fabd80bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 >> 172.21.15.17:0/2027194683 conn(0x7fabd80c6710 msgr2=0x7fabd806cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 shutdown_connections 2024-03-06T07:24:17.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.460+0000 7fabe044e640 1 -- 172.21.15.17:0/2027194683 wait complete. 2024-03-06T07:24:17.473 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:24:17.473 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.573+0000 7f04d7d5c640 1 Processor -- start 2024-03-06T07:24:17.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d7d5c640 1 -- start start 2024-03-06T07:24:17.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d7d5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04d0155c80 0x7f04d006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d7d5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00c1ae0 con 0x7f04d00720a0 2024-03-06T07:24:17.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d7d5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d006e450 con 0x7f04d0152520 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d7d5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d00c1dc0 con 0x7f04d0155c80 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.574+0000 7f04d52f7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04d0155c80 0x7f04d006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d62f9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04d00720a0 0x7f04d006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58918/0 (socket says 172.21.15.17:58918) 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d62f9640 1 -- 172.21.15.17:0/3280367559 learned_addr learned my addr 172.21.15.17:0/3280367559 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d52f7640 1 -- 172.21.15.17:0/3280367559 >> v1:172.21.15.163:6789/0 conn(0x7f04d0152520 legacy=0x7f04d00c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d52f7640 1 -- 172.21.15.17:0/3280367559 >> v1:172.21.15.17:6789/0 conn(0x7f04d00720a0 legacy=0x7f04d006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d52f7640 1 -- 172.21.15.17:0/3280367559 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d006e730 con 0x7f04d0155c80 2024-03-06T07:24:17.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.575+0000 7f04d4af6640 1 -- 172.21.15.17:0/3280367559 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669701961 0 0) 0x7f04d00c1ae0 con 0x7f04d00720a0 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d52f7640 1 --2- 172.21.15.17:0/3280367559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155c80 0x7f04d006ef30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097a0 tx=0x7f04c002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8db19cd8dc0fab server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d4af6640 1 -- 172.21.15.17:0/3280367559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0034070 con 0x7f04d0155c80 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d4af6640 1 -- 172.21.15.17:0/3280367559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0030610 con 0x7f04d0155c80 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d4af6640 1 -- 172.21.15.17:0/3280367559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c00309e0 con 0x7f04d0155c80 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 -- 172.21.15.17:0/3280367559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155c80 msgr2=0x7f04d006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/3280367559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155c80 0x7f04d006ef30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097a0 tx=0x7f04c002b960 comp rx=0 tx=0).stop 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 -- 172.21.15.17:0/3280367559 shutdown_connections 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/3280367559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155c80 0x7f04d006ef30 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 -- 172.21.15.17:0/3280367559 >> 172.21.15.17:0/3280367559 conn(0x7f04d00c76b0 msgr2=0x7f04d00c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 -- 172.21.15.17:0/3280367559 shutdown_connections 2024-03-06T07:24:17.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.576+0000 7f04d7d5c640 1 -- 172.21.15.17:0/3280367559 wait complete. 2024-03-06T07:24:17.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.577+0000 7f04d7d5c640 1 Processor -- start 2024-03-06T07:24:17.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.577+0000 7f04d7d5c640 1 -- start start 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00720a0 0x7f04d0156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d0155c80 0x7f04d01496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d006b850 con 0x7f04d0152520 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d006b6d0 con 0x7f04d00720a0 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d7d5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0152900 con 0x7f04d0155c80 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d52f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d52f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44928/0 (socket says 172.21.15.17:44928) 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d52f7640 1 -- 172.21.15.17:0/1950618675 learned_addr learned my addr 172.21.15.17:0/1950618675 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:17.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d5af8640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00720a0 0x7f04d0156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.578+0000 7f04d62f9640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d0155c80 0x7f04d01496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 -- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d0155c80 msgr2=0x7f04d01496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d0155c80 0x7f04d01496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 -- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00720a0 msgr2=0x7f04d0156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00720a0 0x7f04d0156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 -- 172.21.15.17:0/1950618675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0158890 con 0x7f04d0152520 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d52f7640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f04c00098c0 tx=0x7f04c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0034040 con 0x7f04d0152520 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c003acc0 con 0x7f04d0152520 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d01eecb0 con 0x7f04d0152520 2024-03-06T07:24:17.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0044630 con 0x7f04d0152520 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.579+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d014d170 con 0x7f04d0152520 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.580+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d004f330 con 0x7f04d0152520 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.580+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04c0037070 con 0x7f04d0152520 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.581+0000 7f04c67fc640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 0x7f04b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.581+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f04c00ac230 con 0x7f04d0152520 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.581+0000 7f04d5af8640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 0x7f04b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.581+0000 7f04d5af8640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 0x7f04b406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000f60 tx=0x7f04cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.585+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04c003f070 con 0x7f04d0152520 2024-03-06T07:24:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.806+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04d00c0b70 con 0x7f04d0152520 2024-03-06T07:24:17.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.807+0000 7f04c67fc640 1 -- 172.21.15.17:0/1950618675 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4987 (secure 0 0 0) 0x7f04c007a910 con 0x7f04d0152520 2024-03-06T07:24:17.808 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:17.809 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:09.583188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7021":{"gid":7021,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":6,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:17.809 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 99 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 msgr2=0x7f04b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 0x7f04b406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000f60 tx=0x7f04cc009040 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 msgr2=0x7f04d01ee470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f04c00098c0 tx=0x7f04c00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 shutdown_connections 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b406b4d0 0x7f04b406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d0155c80 0x7f04d01496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0152520 0x7f04d01ee470 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 --2- 172.21.15.17:0/1950618675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00720a0 0x7f04d0156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 >> 172.21.15.17:0/1950618675 conn(0x7f04d00c76b0 msgr2=0x7f04d00c2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 shutdown_connections 2024-03-06T07:24:17.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.809+0000 7f04d7d5c640 1 -- 172.21.15.17:0/1950618675 wait complete. 2024-03-06T07:24:17.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:17.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 Processor -- start 2024-03-06T07:24:17.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 -- start start 2024-03-06T07:24:17.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca9c06f7b0 0x7fca9c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c1506f0 0x7fca9c0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c157340 con 0x7fca9c072720 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c156b40 con 0x7fca9c06f7b0 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.921+0000 7fcaa177a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c3740 con 0x7fca9c1506f0 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca9c06f7b0 0x7fca9c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca9c06f7b0 0x7fca9c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44944/0 (socket says 172.21.15.17:44944) 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 -- 172.21.15.17:0/2117380032 learned_addr learned my addr 172.21.15.17:0/2117380032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9b7fe640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c1506f0 0x7fca9c0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 -- 172.21.15.17:0/2117380032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c1506f0 msgr2=0x7fca9c0c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c1506f0 0x7fca9c0c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 -- 172.21.15.17:0/2117380032 >> v1:172.21.15.163:6789/0 conn(0x7fca9c072720 legacy=0x7fca9c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca9a7fc640 1 -- 172.21.15.17:0/2117380032 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c156d80 con 0x7fca9c06f7b0 2024-03-06T07:24:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.922+0000 7fca99ffb640 1 -- 172.21.15.17:0/2117380032 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578185166 0 0) 0x7fca9c157340 con 0x7fca9c072720 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fca9a7fc640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1501b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b90c62a0fda17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fca99ffb640 1 -- 172.21.15.17:0/2117380032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90035020 con 0x7fca9c06f7b0 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fca99ffb640 1 -- 172.21.15.17:0/2117380032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90031640 con 0x7fca9c06f7b0 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fca99ffb640 1 -- 172.21.15.17:0/2117380032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9002d040 con 0x7fca9c06f7b0 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 -- 172.21.15.17:0/2117380032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c06f7b0 msgr2=0x7fca9c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1501b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 -- 172.21.15.17:0/2117380032 shutdown_connections 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c1506f0 0x7fca9c0c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2117380032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1501b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 -- 172.21.15.17:0/2117380032 >> 172.21.15.17:0/2117380032 conn(0x7fca9c0aa630 msgr2=0x7fca9c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 -- 172.21.15.17:0/2117380032 shutdown_connections 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.923+0000 7fcaa177a640 1 -- 172.21.15.17:0/2117380032 wait complete. 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 Processor -- start 2024-03-06T07:24:17.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 -- start start 2024-03-06T07:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c072720 0x7fca9c1e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c072b00 con 0x7fca9c1506f0 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c33d0 con 0x7fca9c06f7b0 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fcaa177a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c072c80 con 0x7fca9c072720 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fca9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fca9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44952/0 (socket says 172.21.15.17:44952) 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fca9b7fe640 1 -- 172.21.15.17:0/2098158582 learned_addr learned my addr 172.21.15.17:0/2098158582 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.925+0000 7fca9affd640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9a7fc640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c072720 0x7fca9c1e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 -- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c072720 msgr2=0x7fca9c1e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c072720 0x7fca9c1e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 -- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c06f7b0 msgr2=0x7fca9c1ec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1ec5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 -- 172.21.15.17:0/2098158582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c150f40 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca9b7fe640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ec40 tx=0x7fca9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94018020 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9400ee50 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c151220 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca940146c0 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.926+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c1ebdd0 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.927+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c04f330 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.928+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca94005d10 con 0x7fca9c1506f0 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.928+0000 7fca837fe640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:17.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.928+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca9408abd0 con 0x7fca9c1506f0 2024-03-06T07:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.928+0000 7fca9affd640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:17.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.929+0000 7fca9affd640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 0x7fca8806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000f60 tx=0x7fca8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:17.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:17.932+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fca94022070 con 0x7fca9c1506f0 2024-03-06T07:24:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.153+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca9c1513f0 con 0x7fca9c1506f0 2024-03-06T07:24:18.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.153+0000 7fca837fe640 1 -- 172.21.15.17:0/2098158582 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4987 (secure 0 0 0) 0x7fca94010040 con 0x7fca9c1506f0 2024-03-06T07:24:18.155 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:18.155 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":98,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-06T07:23:53.884146+0000","modified":"2024-03-06T07:24:09.583188+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6203":{"gid":6203,"name":"d","rank":0,"incarnation":91,"state":"up:active","state_seq":7,"addr":"172.21.15.163:6836/1364627789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1364627789},{"type":"v1","addr":"172.21.15.163:6836","nonce":1364627789}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7021":{"gid":7021,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":6,"addr":"172.21.15.163:6837/386947795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":386947795},{"type":"v1","addr":"172.21.15.163:6837","nonce":386947795}]},"join_fscid":8,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6203,"qdb_cluster":[6203]},"id":8}]} 2024-03-06T07:24:18.155 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 99 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 msgr2=0x7fca8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 0x7fca8806d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fca8c000f60 tx=0x7fca8c009040 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 msgr2=0x7fca9c1e8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ec40 tx=0x7fca9400c490 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 shutdown_connections 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca8806b4d0 0x7fca8806d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca9c1506f0 0x7fca9c1e8f80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca9c072720 0x7fca9c1e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 --2- 172.21.15.17:0/2098158582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca9c06f7b0 0x7fca9c1ec5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 >> 172.21.15.17:0/2098158582 conn(0x7fca9c0aa630 msgr2=0x7fca9c0a8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 shutdown_connections 2024-03-06T07:24:18.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.155+0000 7fcaa177a640 1 -- 172.21.15.17:0/2098158582 wait complete. 2024-03-06T07:24:18.167 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 Processor -- start 2024-03-06T07:24:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 -- start start 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e94152920 0x7f2e94155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e94156180 con 0x7f2e940c8a00 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e94159630 con 0x7f2e940c5330 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.267+0000 7f2e9a8ec640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e941587b0 con 0x7f2e94152920 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e93fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e94152920 0x7f2e94155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e937fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2e940c8a00 0x7f2e94158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58928/0 (socket says 172.21.15.17:58928) 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e937fe640 1 -- 172.21.15.17:0/1721638084 learned_addr learned my addr 172.21.15.17:0/1721638084 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192981711 0 0) 0x7f2e88001110 con 0x7f2e940c8a00 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003520 con 0x7f2e940c8a00 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e93fff640 1 -- 172.21.15.17:0/1721638084 >> v1:172.21.15.163:6789/0 conn(0x7f2e940c5330 legacy=0x7f2e9415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534255383 0 0) 0x7f2e94159630 con 0x7f2e940c5330 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e93fff640 1 -- 172.21.15.17:0/1721638084 >> v1:172.21.15.17:6789/0 conn(0x7f2e940c8a00 legacy=0x7f2e94158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.268+0000 7f2e93fff640 1 -- 172.21.15.17:0/1721638084 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940c7760 con 0x7f2e94152920 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e93fff640 1 --2- 172.21.15.17:0/1721638084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e94152920 0x7f2e94155c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2e840097a0 tx=0x7f2e8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f232bc51fae23e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84035020 con 0x7f2e94152920 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e84031830 con 0x7f2e94152920 2024-03-06T07:24:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e92ffd640 1 -- 172.21.15.17:0/1721638084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84031c00 con 0x7f2e94152920 2024-03-06T07:24:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1721638084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e94152920 msgr2=0x7f2e94155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1721638084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e94152920 0x7f2e94155c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2e840097a0 tx=0x7f2e8402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1721638084 shutdown_connections 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1721638084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e94152920 0x7f2e94155c40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1721638084 >> 172.21.15.17:0/1721638084 conn(0x7f2e940ac560 msgr2=0x7f2e940ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1721638084 shutdown_connections 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.269+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1721638084 wait complete. 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.270+0000 7f2e9a8ec640 1 Processor -- start 2024-03-06T07:24:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 -- start start 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e940c8a00 0x7f2e9414b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 0x7f2e941eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94158a00 con 0x7f2e940c5330 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e940c7d20 con 0x7f2e940c8a00 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e9a8ec640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9406d030 con 0x7f2e94152920 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44960/0 (socket says 172.21.15.17:44960) 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.271+0000 7f2e93fff640 1 -- 172.21.15.17:0/1782368159 learned_addr learned my addr 172.21.15.17:0/1782368159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e937fe640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e940c8a00 0x7f2e9414b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e98e89640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 0x7f2e941eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 -- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 msgr2=0x7f2e941eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 0x7f2e941eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 -- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e940c8a00 msgr2=0x7f2e9414b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e940c8a00 0x7f2e9414b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 -- 172.21.15.17:0/1782368159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e941555c0 con 0x7f2e940c5330 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e98e89640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 0x7f2e941eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e93fff640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2e840098c0 tx=0x7f2e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84035040 con 0x7f2e940c5330 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8403a840 con 0x7f2e940c5330 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8403ac10 con 0x7f2e940c5330 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e941558a0 con 0x7f2e940c5330 2024-03-06T07:24:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.272+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e94155e60 con 0x7f2e940c5330 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.273+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e941595e0 con 0x7f2e940c5330 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.274+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e84038020 con 0x7f2e940c5330 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.274+0000 7f2e90ff9640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 0x7f2e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.274+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2e840ac500 con 0x7f2e940c5330 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.274+0000 7f2e937fe640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 0x7f2e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.274+0000 7f2e937fe640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 0x7f2e8006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2e9414ca00 tx=0x7f2e88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.278+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e941595e0 con 0x7f2e940c5330 2024-03-06T07:24:18.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.462+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e9404f330 con 0x7f2e940c5330 2024-03-06T07:24:18.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.463+0000 7f2e90ff9640 1 -- 172.21.15.17:0/1782368159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17077 (secure 0 0 0) 0x7f2e840a92d0 con 0x7f2e940c5330 2024-03-06T07:24:18.465 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:18.465 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":110,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:01.553807+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:23:52.140520+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-03-06T07:23:52.842527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/75092648":"2024-03-07T07:24:01.553761+0000","172.21.15.17:6833/75092648":"2024-03-07T07:24:01.553761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 msgr2=0x7f2e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 0x7f2e8006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2e9414ca00 tx=0x7f2e88009040 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 msgr2=0x7f2e94158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2e840098c0 tx=0x7f2e840046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 shutdown_connections 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e8006b4d0 0x7f2e8006d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e94152920 0x7f2e941eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e940c8a00 0x7f2e9414b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 --2- 172.21.15.17:0/1782368159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e940c5330 0x7f2e94158250 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 >> 172.21.15.17:0/1782368159 conn(0x7f2e940ac560 msgr2=0x7f2e940ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 shutdown_connections 2024-03-06T07:24:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.465+0000 7f2e9a8ec640 1 -- 172.21.15.17:0/1782368159 wait complete. 2024-03-06T07:24:18.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:24:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.580+0000 7fef834d6640 1 Processor -- start 2024-03-06T07:24:18.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.580+0000 7fef834d6640 1 -- start start 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.580+0000 7fef834d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.580+0000 7fef834d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c155150 con 0x7fef7c152740 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.580+0000 7fef834d6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c1553d0 con 0x7fef7c158720 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef834d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c155d00 con 0x7fef7c0c27b0 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef81272640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fef7c152740 0x7fef7c06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:58936/0 (socket says 172.21.15.17:58936) 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef81272640 1 -- 172.21.15.17:0/3982384767 learned_addr learned my addr 172.21.15.17:0/3982384767 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef80a71640 1 --2- 172.21.15.17:0/3982384767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c0c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894132191 0 0) 0x7fef6c001490 con 0x7fef7c152740 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef64003520 con 0x7fef7c152740 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291605334 0 0) 0x7fef7c1553d0 con 0x7fef7c158720 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef6c001490 con 0x7fef7c158720 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef80a71640 1 -- 172.21.15.17:0/3982384767 >> v1:172.21.15.163:6789/0 conn(0x7fef7c158720 legacy=0x7fef7c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef80a71640 1 -- 172.21.15.17:0/3982384767 >> v1:172.21.15.17:6789/0 conn(0x7fef7c152740 legacy=0x7fef7c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.581+0000 7fef80a71640 1 -- 172.21.15.17:0/3982384767 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c1556b0 con 0x7fef7c0c27b0 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.582+0000 7fef80a71640 1 --2- 172.21.15.17:0/3982384767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c0c20b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef700097d0 tx=0x7fef7002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=121bd3f9f984887e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.582+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70035020 con 0x7fef7c0c27b0 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.582+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70031800 con 0x7fef7c0c27b0 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.582+0000 7fef7bfff640 1 -- 172.21.15.17:0/3982384767 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70031bd0 con 0x7fef7c0c27b0 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 -- 172.21.15.17:0/3982384767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 msgr2=0x7fef7c0c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 --2- 172.21.15.17:0/3982384767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c0c20b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fef700097d0 tx=0x7fef7002b8a0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 -- 172.21.15.17:0/3982384767 shutdown_connections 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 --2- 172.21.15.17:0/3982384767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c0c20b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 -- 172.21.15.17:0/3982384767 >> 172.21.15.17:0/3982384767 conn(0x7fef7c0c41b0 msgr2=0x7fef7c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 -- 172.21.15.17:0/3982384767 shutdown_connections 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.583+0000 7fef834d6640 1 -- 172.21.15.17:0/3982384767 wait complete. 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 Processor -- start 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 -- start start 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef7c158720 0x7fef7c14d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c06be80 con 0x7fef7c152740 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c159280 con 0x7fef7c158720 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef834d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c152b20 con 0x7fef7c0c27b0 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef80a71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef80a71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44968/0 (socket says 172.21.15.17:44968) 2024-03-06T07:24:18.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.584+0000 7fef80a71640 1 -- 172.21.15.17:0/4045735013 learned_addr learned my addr 172.21.15.17:0/4045735013 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef81272640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef81a73640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef7c158720 0x7fef7c14d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 -- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 msgr2=0x7fef7c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 -- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef7c158720 msgr2=0x7fef7c14d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef7c158720 0x7fef7c14d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 -- 172.21.15.17:0/4045735013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c1ebb80 con 0x7fef7c152740 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef80a71640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fef7002beb0 tx=0x7fef700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70035040 con 0x7fef7c152740 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7003ad00 con 0x7fef7c152740 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c156230 con 0x7fef7c152740 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700466c0 con 0x7fef7c152740 2024-03-06T07:24:18.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.585+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1567f0 con 0x7fef7c152740 2024-03-06T07:24:18.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.586+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c14f340 con 0x7fef7c152740 2024-03-06T07:24:18.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.587+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef70038020 con 0x7fef7c152740 2024-03-06T07:24:18.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.587+0000 7fef79ffb640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.587+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fef700ac3c0 con 0x7fef7c152740 2024-03-06T07:24:18.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.587+0000 7fef81272640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.588+0000 7fef81272640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fef6c002230 tx=0x7fef6c001ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.591+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fef7c14f340 con 0x7fef7c152740 2024-03-06T07:24:18.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.812+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fef7c04f330 con 0x7fef7c152740 2024-03-06T07:24:18.814 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:24:18.813+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.814+0000 7fef79ffb640 1 -- 172.21.15.17:0/4045735013 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7fef7007aa00 con 0x7fef7c152740 2024-03-06T07:24:18.815 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 msgr2=0x7fef6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fef6c002230 tx=0x7fef6c001ca0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 msgr2=0x7fef7c14ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fef7002beb0 tx=0x7fef700046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 shutdown_connections 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef7c158720 0x7fef7c14d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef7c152740 0x7fef7c14ce80 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 --2- 172.21.15.17:0/4045735013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef7c0c27b0 0x7fef7c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 >> 172.21.15.17:0/4045735013 conn(0x7fef7c0c41b0 msgr2=0x7fef7c072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 shutdown_connections 2024-03-06T07:24:18.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.816+0000 7fef834d6640 1 -- 172.21.15.17:0/4045735013 wait complete. 2024-03-06T07:24:18.825 INFO:tasks.ceph.mds.d.smithi163.stderr: -592> 2024-03-06T07:24:04.805+0000 7fb307348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:18.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:24:18.844 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:24:18.849 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:24:18.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 Processor -- start 2024-03-06T07:24:18.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 -- start start 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa784159070 0x7fa7841558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa784155de0 0x7fa7840cac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7840c5580 con 0x7fa784159070 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7840c5860 con 0x7fa784155de0 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.931+0000 7fa789404640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840c5b40 con 0x7fa784152610 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa784159070 0x7fa7841558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa784159070 0x7fa7841558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44978/0 (socket says 172.21.15.17:44978) 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 -- 172.21.15.17:0/2763351211 learned_addr learned my addr 172.21.15.17:0/2763351211 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa782ffd640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa7837fe640 1 --2- 172.21.15.17:0/2763351211 >> v2:172.21.15.163:3300/0 conn(0x7fa784155de0 0x7fa7840cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 -- 172.21.15.17:0/2763351211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 msgr2=0x7fa7840c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 -- 172.21.15.17:0/2763351211 >> v2:172.21.15.163:3300/0 conn(0x7fa784155de0 msgr2=0x7fa7840cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:18.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 --2- 172.21.15.17:0/2763351211 >> v2:172.21.15.163:3300/0 conn(0x7fa784155de0 0x7fa7840cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.932+0000 7fa77a7fc640 1 -- 172.21.15.17:0/2763351211 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7841553a0 con 0x7fa784159070 2024-03-06T07:24:18.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa77a7fc640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841558a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa78406c830 tx=0x7fa76c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664285e399d6608 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa7827fc640 1 -- 172.21.15.17:0/2763351211 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c035020 con 0x7fa784159070 2024-03-06T07:24:18.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa7827fc640 1 -- 172.21.15.17:0/2763351211 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c031870 con 0x7fa784159070 2024-03-06T07:24:18.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa7827fc640 1 -- 172.21.15.17:0/2763351211 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c031c40 con 0x7fa784159070 2024-03-06T07:24:18.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 -- 172.21.15.17:0/2763351211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 msgr2=0x7fa7841558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841558a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa78406c830 tx=0x7fa76c02b910 comp rx=0 tx=0).stop 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 -- 172.21.15.17:0/2763351211 shutdown_connections 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 --2- 172.21.15.17:0/2763351211 >> v2:172.21.15.163:3300/0 conn(0x7fa784155de0 0x7fa7840cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841558a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 --2- 172.21.15.17:0/2763351211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 -- 172.21.15.17:0/2763351211 >> 172.21.15.17:0/2763351211 conn(0x7fa7840728d0 msgr2=0x7fa784072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 -- 172.21.15.17:0/2763351211 shutdown_connections 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.933+0000 7fa789404640 1 -- 172.21.15.17:0/2763351211 wait complete. 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.934+0000 7fa789404640 1 Processor -- start 2024-03-06T07:24:18.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 -- start start 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa784155de0 0x7fa784153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840c5210 con 0x7fa784159070 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841529f0 con 0x7fa784155de0 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa789404640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841594d0 con 0x7fa784152610 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa7837fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44994/0 (socket says 172.21.15.17:44994) 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa7837fe640 1 -- 172.21.15.17:0/3475719148 learned_addr learned my addr 172.21.15.17:0/3475719148 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa782ffd640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.935+0000 7fa77a7fc640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa784155de0 0x7fa784153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 -- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 msgr2=0x7fa7840c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 -- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa784155de0 msgr2=0x7fa784153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa784155de0 0x7fa784153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 -- 172.21.15.17:0/3475719148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784156d50 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa7837fe640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00efc0 tx=0x7fa77c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c018020 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c014480 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78414bec0 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c014870 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.936+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7841541e0 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.937+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78414bc40 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.938+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa77c005d10 con 0x7fa784159070 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.938+0000 7fa77bfff640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 0x7fa76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:18.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.938+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa77c08ac20 con 0x7fa784159070 2024-03-06T07:24:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.938+0000 7fa782ffd640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 0x7fa76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:18.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.939+0000 7fa782ffd640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 0x7fa76406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa774009aa0 tx=0x7fa774009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:18.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:18.943+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa78414bc40 con 0x7fa784159070 2024-03-06T07:24:19.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.161+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa78414b7f0 con 0x7fa784159070 2024-03-06T07:24:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.818+0000 7fa77bfff640 1 -- 172.21.15.17:0/3475719148 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa77c022070 con 0x7fa784159070 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 msgr2=0x7fa76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 0x7fa76406d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa774009aa0 tx=0x7fa774009190 comp rx=0 tx=0).stop 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 msgr2=0x7fa7841ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa77c00efc0 tx=0x7fa77c00c490 comp rx=0 tx=0).stop 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 shutdown_connections 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76406b4d0 0x7fa76406d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa784159070 0x7fa7841ee080 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa784155de0 0x7fa784153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 --2- 172.21.15.17:0/3475719148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa784152610 0x7fa7840c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 >> 172.21.15.17:0/3475719148 conn(0x7fa7840728d0 msgr2=0x7fa784151390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 shutdown_connections 2024-03-06T07:24:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.821+0000 7fa789404640 1 -- 172.21.15.17:0/3475719148 wait complete. 2024-03-06T07:24:19.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:24:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.945+0000 7f5869181640 1 Processor -- start 2024-03-06T07:24:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.945+0000 7f5869181640 1 -- start start 2024-03-06T07:24:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5869181640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58640c53f0 0x7f586406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5869181640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5869181640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586413e070 con 0x7f586413dc90 2024-03-06T07:24:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5869181640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586406c190 con 0x7f58640c53f0 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5869181640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586406bde0 con 0x7f586406b700 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f586359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f586259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58640c53f0 0x7f586406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5862d9d640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f586413dc90 0x7f5864159230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:34860/0 (socket says 172.21.15.17:34860) 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f586359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:37838/0 (socket says 172.21.15.17:37838) 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f586259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f58640c53f0 0x7f586406b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:39996/0 (socket says 172.21.15.17:39996) 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.946+0000 7f5862d9d640 1 -- 172.21.15.17:0/1263133876 learned_addr learned my addr 172.21.15.17:0/1263133876 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f586259c640 1 -- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 msgr2=0x7f58640c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f586259c640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f586259c640 1 -- 172.21.15.17:0/1263133876 >> v1:172.21.15.163:6789/0 conn(0x7f586413dc90 legacy=0x7f5864159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f5861d9b640 1 -- 172.21.15.17:0/1263133876 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943410464 0 0) 0x7f586413e070 con 0x7f586413dc90 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f586259c640 1 -- 172.21.15.17:0/1263133876 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586413e420 con 0x7f58640c53f0 2024-03-06T07:24:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.947+0000 7f586359e640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:19.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f586259c640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58640c53f0 0x7f586406b1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8523e03fd84d3e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5861d9b640 1 -- 172.21.15.17:0/1263133876 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0315a0 con 0x7f58640c53f0 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5861d9b640 1 -- 172.21.15.17:0/1263133876 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c037070 con 0x7f58640c53f0 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5861d9b640 1 -- 172.21.15.17:0/1263133876 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c031960 con 0x7f58640c53f0 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 -- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58640c53f0 msgr2=0x7f586406b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58640c53f0 0x7f586406b1c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02b840 comp rx=0 tx=0).stop 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 -- 172.21.15.17:0/1263133876 shutdown_connections 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f586406b700 0x7f58640c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 --2- 172.21.15.17:0/1263133876 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58640c53f0 0x7f586406b1c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 -- 172.21.15.17:0/1263133876 >> 172.21.15.17:0/1263133876 conn(0x7f58640caa70 msgr2=0x7f58640cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 -- 172.21.15.17:0/1263133876 shutdown_connections 2024-03-06T07:24:19.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.948+0000 7f5869181640 1 -- 172.21.15.17:0/1263133876 wait complete. 2024-03-06T07:24:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 Processor -- start 2024-03-06T07:24:19.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 -- start start 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586406b700 0x7f58641e0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58640c53f0 0x7f58640afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58640c61f0 con 0x7f586413dc90 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586413e9e0 con 0x7f586406b700 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.950+0000 7f5869181640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58640c8970 con 0x7f58640c53f0 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44998/0 (socket says 172.21.15.17:44998) 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 -- 172.21.15.17:0/581051406 learned_addr learned my addr 172.21.15.17:0/581051406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f5862d9d640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586406b700 0x7f58641e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586259c640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58640c53f0 0x7f58640afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 -- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58640c53f0 msgr2=0x7f58640afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58640c53f0 0x7f58640afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 -- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586406b700 msgr2=0x7f58641e0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586406b700 0x7f58641e0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 -- 172.21.15.17:0/581051406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640aed80 con 0x7f586413dc90 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f586359e640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e880 tx=0x7f585c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c017070 con 0x7f586413dc90 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c004020 con 0x7f586413dc90 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c0043f0 con 0x7f586413dc90 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58640af060 con 0x7f586413dc90 2024-03-06T07:24:19.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.951+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641ee910 con 0x7f586413dc90 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.952+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58641eee50 con 0x7f586413dc90 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.953+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f585c00c940 con 0x7f586413dc90 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.953+0000 7f585b7fe640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.953+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f585c08a530 con 0x7f586413dc90 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.954+0000 7f5862d9d640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.954+0000 7f5862d9d640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f58540012e0 tx=0x7f5854001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:19.958+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f585c058b20 con 0x7f586413dc90 2024-03-06T07:24:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.137+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f586404f330 con 0x7f586413dc90 2024-03-06T07:24:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.820+0000 7f585b7fe640 1 -- 172.21.15.17:0/581051406 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7f585c05cbc0 con 0x7f586413dc90 2024-03-06T07:24:20.822 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:24:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 msgr2=0x7f584406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 0x7f584406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f58540012e0 tx=0x7f5854001700 comp rx=0 tx=0).stop 2024-03-06T07:24:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 msgr2=0x7f58640ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e880 tx=0x7f585c00ed40 comp rx=0 tx=0).stop 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 shutdown_connections 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f584406b4d0 0x7f584406d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586413dc90 0x7f58640ad9a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58640c53f0 0x7f58640afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 --2- 172.21.15.17:0/581051406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586406b700 0x7f58641e0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 >> 172.21.15.17:0/581051406 conn(0x7f58640caa70 msgr2=0x7f5864158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 shutdown_connections 2024-03-06T07:24:20.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.824+0000 7f5869181640 1 -- 172.21.15.17:0/581051406 wait complete. 2024-03-06T07:24:20.840 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:24:20.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.949+0000 7f8b15037640 1 Processor -- start 2024-03-06T07:24:20.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.949+0000 7f8b15037640 1 -- start start 2024-03-06T07:24:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b15037640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8b100c88e0 0x7f8b100c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b15037640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b100c7930 0x7f8b1006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b15037640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100c5400 con 0x7f8b10152870 2024-03-06T07:24:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b15037640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1006d320 con 0x7f8b100c88e0 2024-03-06T07:24:20.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b15037640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1006b8d0 con 0x7f8b100c7930 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8b100c88e0 0x7f8b100c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8b100c88e0 0x7f8b100c73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40798/0 (socket says 172.21.15.17:40798) 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 -- 172.21.15.17:0/3910128607 learned_addr learned my addr 172.21.15.17:0/3910128607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 -- 172.21.15.17:0/3910128607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b100c7930 msgr2=0x7f8b1006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 --2- 172.21.15.17:0/3910128607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b100c7930 0x7f8b1006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 -- 172.21.15.17:0/3910128607 >> v1:172.21.15.163:6789/0 conn(0x7f8b10152870 legacy=0x7f8b10155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.950+0000 7f8b0e59c640 1 -- 172.21.15.17:0/3910128607 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100c56e0 con 0x7f8b100c88e0 2024-03-06T07:24:20.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.951+0000 7f8b0e59c640 1 --2- 172.21.15.17:0/3910128607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100c73f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009870 tx=0x7f8af802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d17020a6ebc358 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.951+0000 7f8b0dd9b640 1 -- 172.21.15.17:0/3910128607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80317c0 con 0x7f8b100c88e0 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.951+0000 7f8b0dd9b640 1 -- 172.21.15.17:0/3910128607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8037070 con 0x7f8b100c88e0 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.951+0000 7f8b0dd9b640 1 -- 172.21.15.17:0/3910128607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8031b80 con 0x7f8b100c88e0 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 -- 172.21.15.17:0/3910128607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 msgr2=0x7f8b100c73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 --2- 172.21.15.17:0/3910128607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100c73f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009870 tx=0x7f8af802b860 comp rx=0 tx=0).stop 2024-03-06T07:24:20.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 -- 172.21.15.17:0/3910128607 shutdown_connections 2024-03-06T07:24:20.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 --2- 172.21.15.17:0/3910128607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b100c7930 0x7f8b1006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 --2- 172.21.15.17:0/3910128607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100c73f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 -- 172.21.15.17:0/3910128607 >> 172.21.15.17:0/3910128607 conn(0x7f8b100ac5f0 msgr2=0x7f8b100aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:20.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 -- 172.21.15.17:0/3910128607 shutdown_connections 2024-03-06T07:24:20.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.952+0000 7f8b15037640 1 -- 172.21.15.17:0/3910128607 wait complete. 2024-03-06T07:24:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.953+0000 7f8b15037640 1 Processor -- start 2024-03-06T07:24:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.953+0000 7f8b15037640 1 -- start start 2024-03-06T07:24:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b100c7930 0x7f8b100c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b10152870 0x7f8b100c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c9470 con 0x7f8b100c88e0 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c5ca0 con 0x7f8b100c7930 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b15037640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c95f0 con 0x7f8b10152870 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b0e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b0e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40806/0 (socket says 172.21.15.17:40806) 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b0e59c640 1 -- 172.21.15.17:0/1871336329 learned_addr learned my addr 172.21.15.17:0/1871336329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b0f59e640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b10152870 0x7f8b100c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.954+0000 7f8b0ed9d640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b100c7930 0x7f8b100c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 -- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b10152870 msgr2=0x7f8b100c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b10152870 0x7f8b100c1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 -- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b100c7930 msgr2=0x7f8b100c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b100c7930 0x7f8b100c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 -- 172.21.15.17:0/1871336329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100c46f0 con 0x7f8b100c88e0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b0e59c640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8af802b8a0 tx=0x7f8af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8032860 con 0x7f8b100c88e0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8032a20 con 0x7f8b100c88e0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b100ae290 con 0x7f8b100c88e0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8032e10 con 0x7f8b100c88e0 2024-03-06T07:24:20.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.955+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b100ae850 con 0x7f8b100c88e0 2024-03-06T07:24:20.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.957+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b100aeb80 con 0x7f8b100c88e0 2024-03-06T07:24:20.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.957+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8af804c060 con 0x7f8b100c88e0 2024-03-06T07:24:20.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.958+0000 7f8b077fe640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 0x7f8af006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:20.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.958+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f8af80ab9a0 con 0x7f8b100c88e0 2024-03-06T07:24:20.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.958+0000 7f8b0ed9d640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 0x7f8af006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:20.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.959+0000 7f8b0ed9d640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 0x7f8af006d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8b00001ef0 tx=0x7f8b00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:20.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:20.963+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b100c46f0 con 0x7f8b100c88e0 2024-03-06T07:24:21.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.149+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b1004f330 con 0x7f8b100c88e0 2024-03-06T07:24:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.825+0000 7f8b077fe640 1 -- 172.21.15.17:0/1871336329 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8af8040070 con 0x7f8b100c88e0 2024-03-06T07:24:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:24:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.828+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 msgr2=0x7f8af006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.828+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 0x7f8af006d9c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8b00001ef0 tx=0x7f8b00009040 comp rx=0 tx=0).stop 2024-03-06T07:24:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.828+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 msgr2=0x7f8b100af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.828+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8af802b8a0 tx=0x7f8af80046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 shutdown_connections 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8af006b500 0x7f8af006d9c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b10152870 0x7f8b100c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b100c88e0 0x7f8b100af4d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 --2- 172.21.15.17:0/1871336329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b100c7930 0x7f8b100c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 >> 172.21.15.17:0/1871336329 conn(0x7f8b100ac5f0 msgr2=0x7f8b100c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 shutdown_connections 2024-03-06T07:24:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.829+0000 7f8b15037640 1 -- 172.21.15.17:0/1871336329 wait complete. 2024-03-06T07:24:21.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:24:21.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.953+0000 7f0edaaaa640 1 Processor -- start 2024-03-06T07:24:21.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.953+0000 7f0edaaaa640 1 -- start start 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.953+0000 7f0edaaaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.953+0000 7f0edaaaa640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0ed40c8370 0x7f0ed4158750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0edaaaa640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed40c2c00 con 0x7f0ed4158c90 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0edaaaa640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed4155270 con 0x7f0ed40c8370 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0edaaaa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4155550 con 0x7f0ed40724d0 2024-03-06T07:24:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ed8846640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ecbfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0ed40c8370 0x7f0ed4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ed8846640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46022/0 (socket says 172.21.15.17:46022) 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ed8846640 1 -- 172.21.15.17:0/4043804184 learned_addr learned my addr 172.21.15.17:0/4043804184 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ed9047640 1 --1- 172.21.15.17:0/4043804184 >> v1:172.21.15.163:6789/0 conn(0x7f0ed4158c90 0x7f0ed40728b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59118/0 (socket says 172.21.15.17:59118) 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ecbfff640 1 -- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 msgr2=0x7f0ed4155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ecbfff640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ecbfff640 1 -- 172.21.15.17:0/4043804184 >> v1:172.21.15.163:6789/0 conn(0x7f0ed4158c90 legacy=0x7f0ed40728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ecbfff640 1 -- 172.21.15.17:0/4043804184 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4155830 con 0x7f0ed40c8370 2024-03-06T07:24:21.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.954+0000 7f0ed8846640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.955+0000 7f0ecbfff640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed40c8370 0x7f0ed4158750 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40097a0 tx=0x7f0ec402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd16f3eea791e1a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.955+0000 7f0ecb7fe640 1 -- 172.21.15.17:0/4043804184 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40315a0 con 0x7f0ed40c8370 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.955+0000 7f0ecb7fe640 1 -- 172.21.15.17:0/4043804184 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4037070 con 0x7f0ed40c8370 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.955+0000 7f0ecb7fe640 1 -- 172.21.15.17:0/4043804184 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4031960 con 0x7f0ed40c8370 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 -- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed40c8370 msgr2=0x7f0ed4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed40c8370 0x7f0ed4158750 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40097a0 tx=0x7f0ec402b840 comp rx=0 tx=0).stop 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 -- 172.21.15.17:0/4043804184 shutdown_connections 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed40c8370 0x7f0ed4158750 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/4043804184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40724d0 0x7f0ed4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 -- 172.21.15.17:0/4043804184 >> 172.21.15.17:0/4043804184 conn(0x7f0ed40aa5c0 msgr2=0x7f0ed40aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 -- 172.21.15.17:0/4043804184 shutdown_connections 2024-03-06T07:24:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.956+0000 7f0edaaaa640 1 -- 172.21.15.17:0/4043804184 wait complete. 2024-03-06T07:24:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.957+0000 7f0edaaaa640 1 Processor -- start 2024-03-06T07:24:21.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.957+0000 7f0edaaaa640 1 -- start start 2024-03-06T07:24:21.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40c8370 0x7f0ed41eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed4158c90 0x7f0ed4157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c2170 con 0x7f0ed40724d0 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c2880 con 0x7f0ed4158c90 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0edaaaa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c22f0 con 0x7f0ed40c8370 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40820/0 (socket says 172.21.15.17:40820) 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 -- 172.21.15.17:0/1037907468 learned_addr learned my addr 172.21.15.17:0/1037907468 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ecbfff640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40c8370 0x7f0ed41eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed9047640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed4158c90 0x7f0ed4157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 -- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40c8370 msgr2=0x7f0ed41eea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40c8370 0x7f0ed41eea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 -- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed4158c90 msgr2=0x7f0ed4157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed4158c90 0x7f0ed4157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 -- 172.21.15.17:0/1037907468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4156550 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.958+0000 7f0ed8846640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000e9b0 tx=0x7f0ed000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.959+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0017070 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.959+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed000c9a0 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.959+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed41ef050 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.959+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000cd50 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.959+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed40ad9a0 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.960+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed404f330 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.960+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ed001a070 con 0x7f0ed40724d0 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.961+0000 7f0ec97fa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 0x7f0ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:21.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.961+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0ed0089f20 con 0x7f0ed40724d0 2024-03-06T07:24:21.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.961+0000 7f0ecbfff640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 0x7f0ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:21.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.962+0000 7f0ecbfff640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 0x7f0ec006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036000 tx=0x7f0ec403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:21.965+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0ed0058880 con 0x7f0ed40724d0 2024-03-06T07:24:22.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.147+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7f0ed4156a30 con 0x7f0ed40724d0 2024-03-06T07:24:22.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.150+0000 7f0ec97fa640 1 -- 172.21.15.17:0/1037907468 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v38) v1 ==== 96+0+0 (secure 0 0 0) 0x7f0ed005c920 con 0x7f0ed40724d0 2024-03-06T07:24:22.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.151+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 msgr2=0x7f0ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 0x7f0ec006d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4036000 tx=0x7f0ec403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 msgr2=0x7f0ed41ee4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000e9b0 tx=0x7f0ed000ee70 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 shutdown_connections 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ec006b4d0 0x7f0ec006d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ed4158c90 0x7f0ed4157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ed40c8370 0x7f0ed41eea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 --2- 172.21.15.17:0/1037907468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40724d0 0x7f0ed41ee4f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 >> 172.21.15.17:0/1037907468 conn(0x7f0ed40aa5c0 msgr2=0x7f0ed40aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 shutdown_connections 2024-03-06T07:24:22.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.152+0000 7f0edaaaa640 1 -- 172.21.15.17:0/1037907468 wait complete. 2024-03-06T07:24:22.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:24:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.265+0000 7f0b5b5f6640 1 Processor -- start 2024-03-06T07:24:22.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 -- start start 2024-03-06T07:24:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 0x7f0b54155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b540c2aa0 0x7f0b5406bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5406c110 con 0x7f0b540c0470 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5406b740 con 0x7f0b540c2aa0 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b5b5f6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540670c0 con 0x7f0b540c26c0 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.266+0000 7f0b58b91640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b540c2aa0 0x7f0b5406bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b540c2aa0 0x7f0b5406bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40826/0 (socket says 172.21.15.17:40826) 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 -- 172.21.15.17:0/1058746172 learned_addr learned my addr 172.21.15.17:0/1058746172 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b59392640 1 --2- 172.21.15.17:0/1058746172 >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 0x7f0b54155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 -- 172.21.15.17:0/1058746172 >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 msgr2=0x7f0b54155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 --2- 172.21.15.17:0/1058746172 >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 0x7f0b54155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 -- 172.21.15.17:0/1058746172 >> v1:172.21.15.163:6789/0 conn(0x7f0b540c0470 legacy=0x7f0b54152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b58b91640 1 -- 172.21.15.17:0/1058746172 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5406b0f0 con 0x7f0b540c2aa0 2024-03-06T07:24:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.267+0000 7f0b59392640 1 --2- 172.21.15.17:0/1058746172 >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 0x7f0b54155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b58b91640 1 --2- 172.21.15.17:0/1058746172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c2aa0 0x7f0b5406bbd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0b54067560 tx=0x7f0b4402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c8a490ee3784f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b4bfff640 1 -- 172.21.15.17:0/1058746172 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031760 con 0x7f0b540c2aa0 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b4bfff640 1 -- 172.21.15.17:0/1058746172 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44037070 con 0x7f0b540c2aa0 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b4bfff640 1 -- 172.21.15.17:0/1058746172 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031b20 con 0x7f0b540c2aa0 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1058746172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c2aa0 msgr2=0x7f0b5406bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1058746172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c2aa0 0x7f0b5406bbd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0b54067560 tx=0x7f0b4402b800 comp rx=0 tx=0).stop 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1058746172 shutdown_connections 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1058746172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c2aa0 0x7f0b5406bbd0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1058746172 >> v2:172.21.15.163:3300/0 conn(0x7f0b540c26c0 0x7f0b54155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1058746172 >> 172.21.15.17:0/1058746172 conn(0x7f0b540c3b40 msgr2=0x7f0b540c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1058746172 shutdown_connections 2024-03-06T07:24:22.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.268+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1058746172 wait complete. 2024-03-06T07:24:22.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.269+0000 7f0b5b5f6640 1 Processor -- start 2024-03-06T07:24:22.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.269+0000 7f0b5b5f6640 1 -- start start 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b540c26c0 0x7f0b54153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b540c2aa0 0x7f0b5414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c1f30 con 0x7f0b540c0470 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c0900 con 0x7f0b540c26c0 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b5b5f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c20b0 con 0x7f0b540c2aa0 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40840/0 (socket says 172.21.15.17:40840) 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 -- 172.21.15.17:0/1800329079 learned_addr learned my addr 172.21.15.17:0/1800329079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 -- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b540c2aa0 msgr2=0x7f0b5414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b540c2aa0 0x7f0b5414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 -- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b540c26c0 msgr2=0x7f0b54153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b540c26c0 0x7f0b54153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.270+0000 7f0b59392640 1 -- 172.21.15.17:0/1800329079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540ac0d0 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b59392640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c7f0 tx=0x7f0b5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50015070 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b540ac3b0 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b50007a30 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50007e00 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.271+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b540ac970 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.272+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5404f330 con 0x7f0b540c0470 2024-03-06T07:24:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.272+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b50018070 con 0x7f0b540c0470 2024-03-06T07:24:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.273+0000 7f0b49ffb640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.273+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0b5008a080 con 0x7f0b540c0470 2024-03-06T07:24:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.273+0000 7f0b58b91640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.273+0000 7f0b58b91640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b44036000 tx=0x7f0b44030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.277+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0b50059030 con 0x7f0b540c0470 2024-03-06T07:24:22.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.465+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7f0b54067590 con 0x7f0b540c0470 2024-03-06T07:24:22.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.468+0000 7f0b49ffb640 1 -- 172.21.15.17:0/1800329079 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v39) v1 ==== 96+0+0 (secure 0 0 0) 0x7f0b50058930 con 0x7f0b540c0470 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 msgr2=0x7f0b3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 0x7f0b3806d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b44036000 tx=0x7f0b44030000 comp rx=0 tx=0).stop 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 msgr2=0x7f0b5414a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000c7f0 tx=0x7f0b5000cbf0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 shutdown_connections 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b3806b4d0 0x7f0b3806d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b540c2aa0 0x7f0b5414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b540c26c0 0x7f0b54153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 --2- 172.21.15.17:0/1800329079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b540c0470 0x7f0b5414a560 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 >> 172.21.15.17:0/1800329079 conn(0x7f0b540c3b40 msgr2=0x7f0b5414fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 shutdown_connections 2024-03-06T07:24:22.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.470+0000 7f0b5b5f6640 1 -- 172.21.15.17:0/1800329079 wait complete. 2024-03-06T07:24:22.482 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:24:22.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.584+0000 7f85684df640 1 Processor -- start 2024-03-06T07:24:22.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.584+0000 7f85684df640 1 -- start start 2024-03-06T07:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f85684df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f85600c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f85684df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 0x7f85601587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f85684df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8560155300 con 0x7f85600c4d10 2024-03-06T07:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f85684df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85601555e0 con 0x7f85600c7ff0 2024-03-06T07:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f85684df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85601558c0 con 0x7f85600c4930 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f856627b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f85600c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f8566a7c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 0x7f85601587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f8565a7a640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f85600c4d10 0x7f85600c1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59128/0 (socket says 172.21.15.17:59128) 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f8565a7a640 1 -- 172.21.15.17:0/379365914 learned_addr learned my addr 172.21.15.17:0/379365914 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f8565279640 1 -- 172.21.15.17:0/379365914 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16289672 0 0) 0x7f8560155300 con 0x7f85600c4d10 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f8565279640 1 -- 172.21.15.17:0/379365914 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f854c003520 con 0x7f85600c4d10 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f856627b640 1 -- 172.21.15.17:0/379365914 >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 msgr2=0x7f85601587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f856627b640 1 --2- 172.21.15.17:0/379365914 >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 0x7f85601587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f856627b640 1 -- 172.21.15.17:0/379365914 >> v1:172.21.15.163:6789/0 conn(0x7f85600c4d10 legacy=0x7f85600c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.585+0000 7f856627b640 1 -- 172.21.15.17:0/379365914 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8560155ba0 con 0x7f85600c4930 2024-03-06T07:24:22.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.586+0000 7f8566a7c640 1 --2- 172.21.15.17:0/379365914 >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 0x7f85601587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.586+0000 7f856627b640 1 --2- 172.21.15.17:0/379365914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f85600c7ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f855c0097a0 tx=0x7f855c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e54e92eb7eaa13 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f8565279640 1 -- 172.21.15.17:0/379365914 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0315a0 con 0x7f85600c4930 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f8565279640 1 -- 172.21.15.17:0/379365914 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c037070 con 0x7f85600c4930 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f8565279640 1 -- 172.21.15.17:0/379365914 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c031940 con 0x7f85600c4930 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 -- 172.21.15.17:0/379365914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 msgr2=0x7f85600c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 --2- 172.21.15.17:0/379365914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f85600c7ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f855c0097a0 tx=0x7f855c02b840 comp rx=0 tx=0).stop 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 -- 172.21.15.17:0/379365914 shutdown_connections 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 --2- 172.21.15.17:0/379365914 >> v2:172.21.15.163:3300/0 conn(0x7f85600c7ff0 0x7f85601587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 --2- 172.21.15.17:0/379365914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f85600c7ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 -- 172.21.15.17:0/379365914 >> 172.21.15.17:0/379365914 conn(0x7f85600aa5c0 msgr2=0x7f85600aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 -- 172.21.15.17:0/379365914 shutdown_connections 2024-03-06T07:24:22.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.587+0000 7f85684df640 1 -- 172.21.15.17:0/379365914 wait complete. 2024-03-06T07:24:22.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.588+0000 7f85684df640 1 Processor -- start 2024-03-06T07:24:22.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.588+0000 7f85684df640 1 -- start start 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f856006f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85600c4d10 0x7f856006fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85600c2390 con 0x7f85600c7ff0 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85601560c0 con 0x7f85600c4d10 2024-03-06T07:24:22.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f85684df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600c2510 con 0x7f85600c4930 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40852/0 (socket says 172.21.15.17:40852) 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 -- 172.21.15.17:0/261413290 learned_addr learned my addr 172.21.15.17:0/261413290 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8565a7a640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85600c4d10 0x7f856006fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f856627b640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f856006f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 -- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 msgr2=0x7f856006f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f856006f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 -- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85600c4d10 msgr2=0x7f856006fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85600c4d10 0x7f856006fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.589+0000 7f8566a7c640 1 -- 172.21.15.17:0/261413290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856014ffd0 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f8566a7c640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8558009e60 tx=0x7f855800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558018030 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855800cdf0 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85601502b0 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558003bb0 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.590+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85600a96c0 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.591+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856004f330 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.591+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f855801b070 con 0x7f85600c7ff0 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.592+0000 7f8556ffd640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 0x7f854406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.592+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f855808a8c0 con 0x7f85600c7ff0 2024-03-06T07:24:22.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.592+0000 7f856627b640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 0x7f854406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.592+0000 7f856627b640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 0x7f854406d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f855c036000 tx=0x7f855c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.596+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8558010040 con 0x7f85600c7ff0 2024-03-06T07:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.775+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f85600a8120 con 0x7f85600c7ff0 2024-03-06T07:24:22.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.778+0000 7f8556ffd640 1 -- 172.21.15.17:0/261413290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85580592a0 con 0x7f85600c7ff0 2024-03-06T07:24:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.780+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 msgr2=0x7f854406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.780+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 0x7f854406d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f855c036000 tx=0x7f855c03d040 comp rx=0 tx=0).stop 2024-03-06T07:24:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.780+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 msgr2=0x7f856006d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.780+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8558009e60 tx=0x7f855800c490 comp rx=0 tx=0).stop 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 shutdown_connections 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f854406b4d0 0x7f854406d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85600c7ff0 0x7f856006d120 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85600c4d10 0x7f856006fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 --2- 172.21.15.17:0/261413290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85600c4930 0x7f856006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 >> 172.21.15.17:0/261413290 conn(0x7f85600aa5c0 msgr2=0x7f85600c87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 shutdown_connections 2024-03-06T07:24:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.781+0000 7f85684df640 1 -- 172.21.15.17:0/261413290 wait complete. 2024-03-06T07:24:22.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay' 2024-03-06T07:24:22.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.893+0000 7f657cf25640 1 Processor -- start 2024-03-06T07:24:22.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.893+0000 7f657cf25640 1 -- start start 2024-03-06T07:24:22.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657cf25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6578152920 0x7f6578155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657cf25640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 0x7f6578158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657cf25640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6578156180 con 0x7f6578159460 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657cf25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65781587b0 con 0x7f6578152920 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657cf25640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65780c7760 con 0x7f65780c8a00 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6578152920 0x7f6578155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6578152920 0x7f6578155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40860/0 (socket says 172.21.15.17:40860) 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 -- 172.21.15.17:0/1133821264 learned_addr learned my addr 172.21.15.17:0/1133821264 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f656ffff640 1 --2- 172.21.15.17:0/1133821264 >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 0x7f6578158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 -- 172.21.15.17:0/1133821264 >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 msgr2=0x7f6578158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 --2- 172.21.15.17:0/1133821264 >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 0x7f6578158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 -- 172.21.15.17:0/1133821264 >> v1:172.21.15.163:6789/0 conn(0x7f6578159460 legacy=0x7f65780c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f657659c640 1 -- 172.21.15.17:0/1133821264 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780c7a40 con 0x7f6578152920 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.894+0000 7f656ffff640 1 --2- 172.21.15.17:0/1133821264 >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 0x7f6578158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657659c640 1 --2- 172.21.15.17:0/1133821264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6578152920 0x7f6578155c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6578067c30 tx=0x7f656802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9e2048ba3b729b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f6575d9b640 1 -- 172.21.15.17:0/1133821264 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656802e070 con 0x7f6578152920 2024-03-06T07:24:22.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f6575d9b640 1 -- 172.21.15.17:0/1133821264 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568037070 con 0x7f6578152920 2024-03-06T07:24:22.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f6575d9b640 1 -- 172.21.15.17:0/1133821264 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568031a00 con 0x7f6578152920 2024-03-06T07:24:22.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657cf25640 1 -- 172.21.15.17:0/1133821264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6578152920 msgr2=0x7f6578155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657cf25640 1 --2- 172.21.15.17:0/1133821264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6578152920 0x7f6578155c40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6578067c30 tx=0x7f656802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657cf25640 1 -- 172.21.15.17:0/1133821264 shutdown_connections 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657cf25640 1 --2- 172.21.15.17:0/1133821264 >> v2:172.21.15.163:3300/0 conn(0x7f65780c8a00 0x7f6578158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.895+0000 7f657cf25640 1 --2- 172.21.15.17:0/1133821264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6578152920 0x7f6578155c40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.896+0000 7f657cf25640 1 -- 172.21.15.17:0/1133821264 >> 172.21.15.17:0/1133821264 conn(0x7f65780ac560 msgr2=0x7f65780ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.896+0000 7f657cf25640 1 -- 172.21.15.17:0/1133821264 shutdown_connections 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.896+0000 7f657cf25640 1 -- 172.21.15.17:0/1133821264 wait complete. 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 Processor -- start 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 -- start start 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 0x7f6578149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 0x7f65781eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657806d030 con 0x7f657814a4d0 2024-03-06T07:24:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6578158a00 con 0x7f6578152920 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.897+0000 7f657cf25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65780c5a40 con 0x7f65780c8a00 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40864/0 (socket says 172.21.15.17:40864) 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 -- 172.21.15.17:0/3326334562 learned_addr learned my addr 172.21.15.17:0/3326334562 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f657659c640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 0x7f6578149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f656ffff640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 0x7f65781eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 -- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 msgr2=0x7f6578149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 0x7f6578149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 -- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 msgr2=0x7f65781eaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 0x7f65781eaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 -- 172.21.15.17:0/3326334562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65781ebb40 con 0x7f657814a4d0 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f657659c640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 0x7f6578149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f656ffff640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 0x7f65781eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:22.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f6576d9d640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f657000b9e0 tx=0x7f657000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570003b70 con 0x7f657814a4d0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570003d10 con 0x7f657814a4d0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6578148430 con 0x7f657814a4d0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657000caa0 con 0x7f657814a4d0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.898+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657814eb50 con 0x7f657814a4d0 2024-03-06T07:24:22.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.899+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6578150140 con 0x7f657814a4d0 2024-03-06T07:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.900+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f657000cc40 con 0x7f657814a4d0 2024-03-06T07:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.900+0000 7f656effd640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.900+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6570089d70 con 0x7f657814a4d0 2024-03-06T07:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.901+0000 7f657659c640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.901+0000 7f657659c640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 0x7f655806d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6568036000 tx=0x7f656803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:22.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:22.905+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6578150140 con 0x7f657814a4d0 2024-03-06T07:24:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.158+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]} v 0) v1 -- 0x7f657804f330 con 0x7f657814a4d0 2024-03-06T07:24:23.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.831+0000 7f656effd640 1 -- 172.21.15.17:0/3326334562 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_standby_replay"]}]=0 v262) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6570058da0 con 0x7f657814a4d0 2024-03-06T07:24:23.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 msgr2=0x7f655806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:23.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 0x7f655806d9c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6568036000 tx=0x7f656803d040 comp rx=0 tx=0).stop 2024-03-06T07:24:23.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 msgr2=0x7f65781eb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:23.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f657000b9e0 tx=0x7f657000bea0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 shutdown_connections 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f655806b500 0x7f655806d9c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657814a4d0 0x7f65781eb3f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.833+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6578152920 0x7f65781eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.834+0000 7f657cf25640 1 --2- 172.21.15.17:0/3326334562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f65780c8a00 0x7f6578149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.834+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 >> 172.21.15.17:0/3326334562 conn(0x7f65780ac560 msgr2=0x7f6578151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.834+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 shutdown_connections 2024-03-06T07:24:23.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.834+0000 7f657cf25640 1 -- 172.21.15.17:0/3326334562 wait complete. 2024-03-06T07:24:23.848 INFO:tasks.cephfs_test_runner:That a standby-replay daemon with weak affinity is replaced by a stronger one. ... ok 2024-03-06T07:24:23.849 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_steady (tasks.cephfs.test_failover.TestClusterAffinity) That a sole MDS with mds_join_fs set will come back as active eventually even after failover. 2024-03-06T07:24:23.849 INFO:tasks.cephfs_test_runner:test_join_fs_steady (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:24:23.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:23.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.955+0000 7f2f29efa640 1 Processor -- start 2024-03-06T07:24:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.955+0000 7f2f29efa640 1 -- start start 2024-03-06T07:24:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f2406da40 0x7f2f24155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f240c6780 0x7f2f24158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 0x7f2f240c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2406de20 con 0x7f2f240c6780 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f24156050 con 0x7f2f2406da40 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f29efa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24072630 con 0x7f2f241592d0 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f240c6780 0x7f2f24158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f240c6780 0x7f2f24158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40866/0 (socket says 172.21.15.17:40866) 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 -- 172.21.15.17:0/3510996951 learned_addr learned my addr 172.21.15.17:0/3510996951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f23fff640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 0x7f2f240c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f237fe640 1 --2- 172.21.15.17:0/3510996951 >> v2:172.21.15.163:3300/0 conn(0x7f2f2406da40 0x7f2f24155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 -- 172.21.15.17:0/3510996951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 msgr2=0x7f2f240c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 0x7f2f240c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 -- 172.21.15.17:0/3510996951 >> v2:172.21.15.163:3300/0 conn(0x7f2f2406da40 msgr2=0x7f2f24155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 --2- 172.21.15.17:0/3510996951 >> v2:172.21.15.163:3300/0 conn(0x7f2f2406da40 0x7f2f24155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f22ffd640 1 -- 172.21.15.17:0/3510996951 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24072910 con 0x7f2f240c6780 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.956+0000 7f2f23fff640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 0x7f2f240c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f22ffd640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f240c6780 0x7f2f24158d90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2f240c98a0 tx=0x7f2f1802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f11e426e48949 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f227fc640 1 -- 172.21.15.17:0/3510996951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180317d0 con 0x7f2f240c6780 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f227fc640 1 -- 172.21.15.17:0/3510996951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f18037070 con 0x7f2f240c6780 2024-03-06T07:24:23.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f227fc640 1 -- 172.21.15.17:0/3510996951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18031b90 con 0x7f2f240c6780 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f29efa640 1 -- 172.21.15.17:0/3510996951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f240c6780 msgr2=0x7f2f24158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.957+0000 7f2f29efa640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f240c6780 0x7f2f24158d90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2f240c98a0 tx=0x7f2f1802b870 comp rx=0 tx=0).stop 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 -- 172.21.15.17:0/3510996951 shutdown_connections 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f241592d0 0x7f2f240c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 --2- 172.21.15.17:0/3510996951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f240c6780 0x7f2f24158d90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 --2- 172.21.15.17:0/3510996951 >> v2:172.21.15.163:3300/0 conn(0x7f2f2406da40 0x7f2f24155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 -- 172.21.15.17:0/3510996951 >> 172.21.15.17:0/3510996951 conn(0x7f2f240aa630 msgr2=0x7f2f240aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 -- 172.21.15.17:0/3510996951 shutdown_connections 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.958+0000 7f2f29efa640 1 -- 172.21.15.17:0/3510996951 wait complete. 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.959+0000 7f2f29efa640 1 Processor -- start 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.959+0000 7f2f29efa640 1 -- start start 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 0x7f2f2414ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 0x7f2f2406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c37c0 con 0x7f2f241592d0 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24158890 con 0x7f2f2406da40 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240c3940 con 0x7f2f240c6780 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40880/0 (socket says 172.21.15.17:40880) 2024-03-06T07:24:23.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 -- 172.21.15.17:0/732918378 learned_addr learned my addr 172.21.15.17:0/732918378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f22ffd640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 0x7f2f2406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f237fe640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 0x7f2f2414ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 -- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 msgr2=0x7f2f2406fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 0x7f2f2406fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 -- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 msgr2=0x7f2f2414ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 0x7f2f2414ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 -- 172.21.15.17:0/732918378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24152be0 con 0x7f2f241592d0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f22ffd640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 0x7f2f2406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f237fe640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 0x7f2f2414ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f23fff640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00ce40 tx=0x7f2f1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c01a070 con 0x7f2f241592d0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00d840 con 0x7f2f241592d0 2024-03-06T07:24:23.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00dc30 con 0x7f2f241592d0 2024-03-06T07:24:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.960+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241eacb0 con 0x7f2f241592d0 2024-03-06T07:24:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.961+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241eb270 con 0x7f2f241592d0 2024-03-06T07:24:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.961+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f241eaaa0 con 0x7f2f241592d0 2024-03-06T07:24:23.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.962+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f1c01d070 con 0x7f2f241592d0 2024-03-06T07:24:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.962+0000 7f2f0bfff640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 0x7f2f1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.962+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2f1c08a880 con 0x7f2f241592d0 2024-03-06T07:24:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.963+0000 7f2f237fe640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 0x7f2f1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:23.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.963+0000 7f2f237fe640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 0x7f2f1006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ba50 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:23.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:23.967+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f1c0598b0 con 0x7f2f241592d0 2024-03-06T07:24:24.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.149+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f2404f330 con 0x7f2f241592d0 2024-03-06T07:24:24.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.150+0000 7f2f0bfff640 1 -- 172.21.15.17:0/732918378 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13431 (secure 0 0 0) 0x7f2f1c0591b0 con 0x7f2f241592d0 2024-03-06T07:24:24.151 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:24.152 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":113,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:21.821211+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/1364627789":"2024-03-07T07:24:18.813767+0000","172.21.15.163:6834/1364627789":"2024-03-07T07:24:18.813767+0000","172.21.15.17:6835/75092648":"2024-03-07T07:24:01.553761+0000","172.21.15.17:6833/75092648":"2024-03-07T07:24:01.553761+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 msgr2=0x7f2f1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 0x7f2f1006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ba50 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 msgr2=0x7f2f24157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00ce40 tx=0x7f2f1c00a710 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 shutdown_connections 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f1006b4d0 0x7f2f1006d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f241592d0 0x7f2f24157a70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f240c6780 0x7f2f2406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 --2- 172.21.15.17:0/732918378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f2406da40 0x7f2f2414ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 >> 172.21.15.17:0/732918378 conn(0x7f2f240aa630 msgr2=0x7f2f2414e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 shutdown_connections 2024-03-06T07:24:24.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.152+0000 7f2f29efa640 1 -- 172.21.15.17:0/732918378 wait complete. 2024-03-06T07:24:24.164 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:24.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:24.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.267+0000 7f340029a640 1 Processor -- start 2024-03-06T07:24:24.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.267+0000 7f340029a640 1 -- start start 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.267+0000 7f340029a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33f806afc0 0x7f33f806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.267+0000 7f340029a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f8158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f340029a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80c5840 con 0x7f33f80c2580 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f340029a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f8158790 con 0x7f33f806afc0 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f340029a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c5b20 con 0x7f33f80c2120 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33f806afc0 0x7f33f806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33f806afc0 0x7f33f806b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40896/0 (socket says 172.21.15.17:40896) 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 -- 172.21.15.17:0/3053656818 learned_addr learned my addr 172.21.15.17:0/3053656818 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fd835640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f8158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 -- 172.21.15.17:0/3053656818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 msgr2=0x7f33f8158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f8158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 -- 172.21.15.17:0/3053656818 >> v1:172.21.15.163:6789/0 conn(0x7f33f80c2580 legacy=0x7f33f8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:24.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.268+0000 7f33fe036640 1 -- 172.21.15.17:0/3053656818 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80c5e00 con 0x7f33f806afc0 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f33fe036640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f806afc0 0x7f33f806b3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009870 tx=0x7f33f402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2390f5b575a38ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f33fd034640 1 -- 172.21.15.17:0/3053656818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40317c0 con 0x7f33f806afc0 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f33fd034640 1 -- 172.21.15.17:0/3053656818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4037070 con 0x7f33f806afc0 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f33fd034640 1 -- 172.21.15.17:0/3053656818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031b80 con 0x7f33f806afc0 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 -- 172.21.15.17:0/3053656818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f806afc0 msgr2=0x7f33f806b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f806afc0 0x7f33f806b3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009870 tx=0x7f33f402b860 comp rx=0 tx=0).stop 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 -- 172.21.15.17:0/3053656818 shutdown_connections 2024-03-06T07:24:24.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f8158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 --2- 172.21.15.17:0/3053656818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f806afc0 0x7f33f806b3a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.269+0000 7f340029a640 1 -- 172.21.15.17:0/3053656818 >> 172.21.15.17:0/3053656818 conn(0x7f33f80c4220 msgr2=0x7f33f80c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.270+0000 7f340029a640 1 -- 172.21.15.17:0/3053656818 shutdown_connections 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.270+0000 7f340029a640 1 -- 172.21.15.17:0/3053656818 wait complete. 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 Processor -- start 2024-03-06T07:24:24.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 -- start start 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33f806afc0 0x7f33f814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f81496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c2b30 con 0x7f33f80c2580 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8155220 con 0x7f33f806afc0 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f340029a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c2cb0 con 0x7f33f80c2120 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.271+0000 7f33fe837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40902/0 (socket says 172.21.15.17:40902) 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 -- 172.21.15.17:0/3802519399 learned_addr learned my addr 172.21.15.17:0/3802519399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe036640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33f806afc0 0x7f33f814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fd835640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f81496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 -- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 msgr2=0x7f33f81496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f81496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 -- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33f806afc0 msgr2=0x7f33f814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33f806afc0 0x7f33f814a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 -- 172.21.15.17:0/3802519399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f814cd90 con 0x7f33f80c2580 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33fe837640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b970 tx=0x7f33f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0019070 con 0x7f33f80c2580 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f000cb40 con 0x7f33f80c2580 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f814d070 con 0x7f33f80c2580 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0014440 con 0x7f33f80c2580 2024-03-06T07:24:24.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.272+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f814d630 con 0x7f33f80c2580 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.273+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f804f330 con 0x7f33f80c2580 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.274+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33f001c070 con 0x7f33f80c2580 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.274+0000 7f33eeffd640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 0x7f33dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.274+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f33f0089b90 con 0x7f33f80c2580 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.274+0000 7f33fe036640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 0x7f33dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.275+0000 7f33fe036640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 0x7f33dc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.278+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33f0058570 con 0x7f33f80c2580 2024-03-06T07:24:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.460+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33f80c6730 con 0x7f33f80c2580 2024-03-06T07:24:24.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.460+0000 7f33eeffd640 1 -- 172.21.15.17:0/3802519399 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33f005c610 con 0x7f33f80c2580 2024-03-06T07:24:24.462 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:24:24.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 msgr2=0x7f33dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 0x7f33dc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 msgr2=0x7f33f8149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f33f000b970 tx=0x7f33f000be30 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 shutdown_connections 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f33dc06b4d0 0x7f33dc06d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33f80c2580 0x7f33f8149c30 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33f80c2120 0x7f33f81496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 --2- 172.21.15.17:0/3802519399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33f806afc0 0x7f33f814a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 >> 172.21.15.17:0/3802519399 conn(0x7f33f80c4220 msgr2=0x7f33f81507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 shutdown_connections 2024-03-06T07:24:24.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.462+0000 7f340029a640 1 -- 172.21.15.17:0/3802519399 wait complete. 2024-03-06T07:24:24.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady' 2024-03-06T07:24:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.574+0000 7f905788d640 1 Processor -- start 2024-03-06T07:24:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.574+0000 7f905788d640 1 -- start start 2024-03-06T07:24:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f905788d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90501609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f905788d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90501591c0 con 0x7f9050155ba0 2024-03-06T07:24:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f905788d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90501594a0 con 0x7f90500c5480 2024-03-06T07:24:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f905788d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905006df40 con 0x7f90500c5860 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f9054e28640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9050155ba0 0x7f90501586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45882/0 (socket says 172.21.15.17:45882) 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f9054e28640 1 -- 172.21.15.17:0/3806417567 learned_addr learned my addr 172.21.15.17:0/3806417567 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f9055e2a640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90501609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472689622 0 0) 0x7f90501591c0 con 0x7f9050155ba0 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.575+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9038003520 con 0x7f9050155ba0 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863652984 0 0) 0x7f90501594a0 con 0x7f90500c5480 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90501591c0 con 0x7f90500c5480 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2760928655 0 0) 0x7f9038003520 con 0x7f9050155ba0 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 msgr2=0x7f90501609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90501609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 >> v1:172.21.15.163:6789/0 conn(0x7f90500c5480 legacy=0x7f905006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90501553a0 con 0x7f9050155ba0 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f90440015f0 con 0x7f9050155ba0 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9055e2a640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90501609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 >> v1:172.21.15.17:6789/0 conn(0x7f9050155ba0 legacy=0x7f90501586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9038003b70 0x7f9038003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f9038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90380051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905006df40 con 0x7f9038004800 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903800bb00 con 0x7f9038003b70 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903800bd00 con 0x7f90500c5860 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f9038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9055629640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9038003b70 0x7f9038003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9055e2a640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90380051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 -- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 msgr2=0x7f90380051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500c5860 0x7f90380051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 -- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9038003b70 msgr2=0x7f9038003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9038003b70 0x7f9038003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.576+0000 7f9054e28640 1 -- 172.21.15.17:0/3806417567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038003520 con 0x7f9038004800 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f9054e28640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f9038004c80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9044007f80 tx=0x7f9038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86af977ec98453a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044007ba0 con 0x7f9038004800 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9044030070 con 0x7f9038004800 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f904ffff640 1 -- 172.21.15.17:0/3806417567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044005df0 con 0x7f9038004800 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f905788d640 1 -- 172.21.15.17:0/3806417567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 msgr2=0x7f9038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f905788d640 1 --2- 172.21.15.17:0/3806417567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f9038004c80 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9044007f80 tx=0x7f9038005830 comp rx=0 tx=0).stop 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f9055e2a640 1 -- 172.21.15.17:0/3806417567 reap_dead start 2024-03-06T07:24:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f905788d640 1 -- 172.21.15.17:0/3806417567 shutdown_connections 2024-03-06T07:24:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.577+0000 7f905788d640 1 -- 172.21.15.17:0/3806417567 >> 172.21.15.17:0/3806417567 conn(0x7f905006b1a0 msgr2=0x7f905006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.578+0000 7f905788d640 1 -- 172.21.15.17:0/3806417567 shutdown_connections 2024-03-06T07:24:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.578+0000 7f905788d640 1 -- 172.21.15.17:0/3806417567 wait complete. 2024-03-06T07:24:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 Processor -- start 2024-03-06T07:24:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 -- start start 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90500a9c70 0x7f90500aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500ac3f0 0x7f90500ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500c5cf0 con 0x7f9038004800 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905006fdb0 con 0x7f90500a9c70 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f905788d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9050156000 con 0x7f90500ac3f0 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f9055629640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f9055629640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40922/0 (socket says 172.21.15.17:40922) 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.579+0000 7f9055629640 1 -- 172.21.15.17:0/3935547732 learned_addr learned my addr 172.21.15.17:0/3935547732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9054e28640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90500a9c70 0x7f90500aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055e2a640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500ac3f0 0x7f90500ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 -- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500ac3f0 msgr2=0x7f90500ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500ac3f0 0x7f90500ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 -- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90500a9c70 msgr2=0x7f90500aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90500a9c70 0x7f90500aa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 -- 172.21.15.17:0/3935547732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90500a8550 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f9055629640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f904800fc30 tx=0x7f904800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904800fe40 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9048018070 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500a8830 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048003970 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.580+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90500a8df0 con 0x7f9038004800 2024-03-06T07:24:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.581+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905004f330 con 0x7f9038004800 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.581+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9048003b10 con 0x7f9038004800 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.582+0000 7f904dffb640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.582+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f904808a420 con 0x7f9038004800 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.582+0000 7f9054e28640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.583+0000 7f9054e28640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 0x7f903c06d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9044007f50 tx=0x7f9044009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.586+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9048059450 con 0x7f9038004800 2024-03-06T07:24:24.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:24.838+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]} v 0) v1 -- 0x7f90500ac0a0 con 0x7f9038004800 2024-03-06T07:24:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.837+0000 7f904dffb640 1 -- 172.21.15.17:0/3935547732 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]}]=0 v264) v1 ==== 148+0+0 (secure 0 0 0) 0x7f9048058d50 con 0x7f9038004800 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 msgr2=0x7f903c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 0x7f903c06d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9044007f50 tx=0x7f9044009030 comp rx=0 tx=0).stop 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 msgr2=0x7f90501eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f904800fc30 tx=0x7f904800d520 comp rx=0 tx=0).stop 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 shutdown_connections 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f903c06b500 0x7f903c06d9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90500ac3f0 0x7f90500ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90500a9c70 0x7f90500aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 --2- 172.21.15.17:0/3935547732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9038004800 0x7f90501eb8f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 >> 172.21.15.17:0/3935547732 conn(0x7f905006b1a0 msgr2=0x7f905006c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 shutdown_connections 2024-03-06T07:24:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.840+0000 7f905788d640 1 -- 172.21.15.17:0/3935547732 wait complete. 2024-03-06T07:24:25.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:24:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 Processor -- start 2024-03-06T07:24:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 -- start start 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02a00c4ef0 0x7f02a0155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a0152cb0 con 0x7f02a00c67f0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a00c38b0 con 0x7f02a01528d0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.964+0000 7f02a8516640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a01594f0 con 0x7f02a00c4ef0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a5ab1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f02a00c67f0 0x7f02a0155220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45886/0 (socket says 172.21.15.17:45886) 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a5ab1640 1 -- 172.21.15.17:0/3325235201 learned_addr learned my addr 172.21.15.17:0/3325235201 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/3325235201 >> v2:172.21.15.163:3300/0 conn(0x7f02a00c4ef0 0x7f02a0155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156206530 0 0) 0x7f02a0152cb0 con 0x7f02a00c67f0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0288003520 con 0x7f02a00c67f0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012195984 0 0) 0x7f0288003520 con 0x7f02a00c67f0 2024-03-06T07:24:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 >> v2:172.21.15.163:3300/0 conn(0x7f02a00c4ef0 msgr2=0x7f02a0155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 --2- 172.21.15.17:0/3325235201 >> v2:172.21.15.163:3300/0 conn(0x7f02a00c4ef0 0x7f02a0155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 >> v1:172.21.15.163:6789/0 conn(0x7f02a01528d0 legacy=0x7f02a00724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00a7f70 con 0x7f02a00c67f0 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0294001670 con 0x7f02a00c67f0 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 >> v1:172.21.15.17:6789/0 conn(0x7f02a00c67f0 legacy=0x7f02a0155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880039b0 0x7f0288003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0288004800 0x7f0288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 0x7f028800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01594f0 con 0x7f02880051c0 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028800c070 con 0x7f0288004800 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028800c270 con 0x7f02880039b0 2024-03-06T07:24:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a5ab1640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0288004800 0x7f0288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 0x7f028800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.965+0000 7f02a62b2640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880039b0 0x7f0288003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 -- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880039b0 msgr2=0x7f0288003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880039b0 0x7f0288003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 -- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0288004800 msgr2=0x7f0288004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0288004800 0x7f0288004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 -- 172.21.15.17:0/3325235201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288003520 con 0x7f02880051c0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 -- 172.21.15.17:0/3325235201 reap_dead start 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.966+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 0x7f028800b9c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0290009f90 tx=0x7f0288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20776f890099fd34 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290039070 con 0x7f02880051c0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029002cc20 con 0x7f02880051c0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a52b0640 1 -- 172.21.15.17:0/3325235201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290009c10 con 0x7f02880051c0 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 -- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 msgr2=0x7f028800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 0x7f028800b9c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0290009f90 tx=0x7f0288005830 comp rx=0 tx=0).stop 2024-03-06T07:24:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 -- 172.21.15.17:0/3325235201 shutdown_connections 2024-03-06T07:24:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 --2- 172.21.15.17:0/3325235201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02880051c0 0x7f028800b9c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 -- 172.21.15.17:0/3325235201 >> 172.21.15.17:0/3325235201 conn(0x7f02a00aa6a0 msgr2=0x7f02a00aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 -- 172.21.15.17:0/3325235201 shutdown_connections 2024-03-06T07:24:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.967+0000 7f02a8516640 1 -- 172.21.15.17:0/3325235201 wait complete. 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.968+0000 7f02a8516640 1 Processor -- start 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.968+0000 7f02a8516640 1 -- start start 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880051c0 0x7f02a0151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 0x7f02a0148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a01591a0 con 0x7f02a0070890 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00a8250 con 0x7f02a0155ea0 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a8516640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00724d0 con 0x7f02880051c0 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40946/0 (socket says 172.21.15.17:40946) 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 -- 172.21.15.17:0/2965736116 learned_addr learned my addr 172.21.15.17:0/2965736116 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a62b2640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880051c0 0x7f02a0151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a5ab1640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 0x7f02a0148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 -- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880051c0 msgr2=0x7f02a0151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880051c0 0x7f02a0151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 -- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 msgr2=0x7f02a0148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 0x7f02a0148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.969+0000 7f02a6ab3640 1 -- 172.21.15.17:0/2965736116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0148d50 con 0x7f02a0070890 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f02a5ab1640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 0x7f02a0148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f02a6ab3640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f029000e230 tx=0x7f0290009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290039040 con 0x7f02a0070890 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0290045070 con 0x7f02a0070890 2024-03-06T07:24:25.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a014ff80 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029003e6c0 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.970+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a01eecb0 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.971+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a00c6cb0 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.971+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f029003c070 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.972+0000 7f029effd640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.972+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f02900ac080 con 0x7f02a0070890 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.972+0000 7f02a62b2640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.973+0000 7f02a62b2640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 0x7f028c06d9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:25.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:25.976+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f02a00c6cb0 con 0x7f02a0070890 2024-03-06T07:24:26.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.159+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02a0152cb0 con 0x7f02a0070890 2024-03-06T07:24:26.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.162+0000 7f029effd640 1 -- 172.21.15.17:0/2965736116 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02a0152cb0 con 0x7f02a0070890 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 msgr2=0x7f028c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 0x7f028c06d9c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298009040 comp rx=0 tx=0).stop 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 msgr2=0x7f02a014ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f029000e230 tx=0x7f0290009750 comp rx=0 tx=0).stop 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 shutdown_connections 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f028c06b500 0x7f028c06d9c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02a0070890 0x7f02a014ac00 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02a0155ea0 0x7f02a0148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 --2- 172.21.15.17:0/2965736116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02880051c0 0x7f02a0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 >> 172.21.15.17:0/2965736116 conn(0x7f02a00aa6a0 msgr2=0x7f02a006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 shutdown_connections 2024-03-06T07:24:26.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.164+0000 7f02a8516640 1 -- 172.21.15.17:0/2965736116 wait complete. 2024-03-06T07:24:26.176 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:24:26.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:26.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:26.264 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:24:26.265 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:24:26.265 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:26.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:24:26.279 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:24:26.279 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:26.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:24:26.296 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:24:26.297 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:26.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.418+0000 7f154fd6c640 1 Processor -- start 2024-03-06T07:24:26.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.418+0000 7f154fd6c640 1 -- start start 2024-03-06T07:24:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154fd6c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15480c5480 0x7f1548155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154fd6c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 0x7f1548158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154fd6c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480c5cc0 con 0x7f15480c5860 2024-03-06T07:24:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154fd6c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154806df40 con 0x7f15480c5480 2024-03-06T07:24:26.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154fd6c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154806fdb0 con 0x7f15481586f0 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154db08640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15480c5480 0x7f1548155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154db08640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f15480c5480 0x7f1548155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40952/0 (socket says 172.21.15.17:40952) 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.419+0000 7f154db08640 1 -- 172.21.15.17:0/344821312 learned_addr learned my addr 172.21.15.17:0/344821312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154e309640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 0x7f1548158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154db08640 1 -- 172.21.15.17:0/344821312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 msgr2=0x7f1548158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154db08640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 0x7f1548158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154db08640 1 -- 172.21.15.17:0/344821312 >> v1:172.21.15.163:6789/0 conn(0x7f15480c5860 legacy=0x7f154806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154db08640 1 -- 172.21.15.17:0/344821312 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15481591c0 con 0x7f15480c5480 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154e309640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 0x7f1548158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.420+0000 7f154db08640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5480 0x7f1548155ba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1538009e50 tx=0x7f153802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70d7d54910be501 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154cb06640 1 -- 172.21.15.17:0/344821312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153802c9e0 con 0x7f15480c5480 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154cb06640 1 -- 172.21.15.17:0/344821312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153802cb80 con 0x7f15480c5480 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154cb06640 1 -- 172.21.15.17:0/344821312 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15380343f0 con 0x7f15480c5480 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 -- 172.21.15.17:0/344821312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5480 msgr2=0x7f1548155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5480 0x7f1548155ba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1538009e50 tx=0x7f153802be40 comp rx=0 tx=0).stop 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 -- 172.21.15.17:0/344821312 shutdown_connections 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15481586f0 0x7f1548158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 --2- 172.21.15.17:0/344821312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5480 0x7f1548155ba0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 -- 172.21.15.17:0/344821312 >> 172.21.15.17:0/344821312 conn(0x7f154806b1a0 msgr2=0x7f154806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 -- 172.21.15.17:0/344821312 shutdown_connections 2024-03-06T07:24:26.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.421+0000 7f154fd6c640 1 -- 172.21.15.17:0/344821312 wait complete. 2024-03-06T07:24:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.422+0000 7f154fd6c640 1 Processor -- start 2024-03-06T07:24:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.422+0000 7f154fd6c640 1 -- start start 2024-03-06T07:24:26.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15480c5480 0x7f1548151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15481586f0 0x7f15480aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548155680 con 0x7f15480c5860 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481553a0 con 0x7f15481586f0 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154fd6c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1548152a20 con 0x7f15480c5480 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40960/0 (socket says 172.21.15.17:40960) 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 -- 172.21.15.17:0/42610693 learned_addr learned my addr 172.21.15.17:0/42610693 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154db08640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15480c5480 0x7f1548151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154e309640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15481586f0 0x7f15480aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 -- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15480c5480 msgr2=0x7f1548151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15480c5480 0x7f1548151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 -- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15481586f0 msgr2=0x7f15480aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.423+0000 7f154d307640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15481586f0 0x7f15480aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f154d307640 1 -- 172.21.15.17:0/42610693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15481deb40 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f154d307640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c430 tx=0x7f153c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011790 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c017070 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480c48d0 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c011bb0 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.424+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15481f0260 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.425+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154804f330 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.425+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f153c01c460 con 0x7f15480c5860 2024-03-06T07:24:26.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.426+0000 7f15467fc640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.426+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f153c08a040 con 0x7f15480c5860 2024-03-06T07:24:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.426+0000 7f154db08640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.427+0000 7f154db08640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 0x7f153406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1538035000 tx=0x7f1538030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.430+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15481deb40 con 0x7f15480c5860 2024-03-06T07:24:26.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.652+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f154806d4b0 con 0x7f15480c5860 2024-03-06T07:24:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.652+0000 7f15467fc640 1 -- 172.21.15.17:0/42610693 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f153c058fd0 con 0x7f15480c5860 2024-03-06T07:24:26.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:26.654 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2107217647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2107217647},{"type":"v1","addr":"172.21.15.163:6835","nonce":2107217647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3746943765","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":3746943765},{"type":"v1","addr":"172.21.15.163:6837","nonce":3746943765}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2673518972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2673518972},{"type":"v1","addr":"172.21.15.17:6834","nonce":2673518972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":7659,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1634180734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1634180734},{"type":"v1","addr":"172.21.15.17:6835","nonce":1634180734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99}],"filesystems":[]} 2024-03-06T07:24:26.654 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 101 2024-03-06T07:24:26.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 msgr2=0x7f153406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 0x7f153406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1538035000 tx=0x7f1538030000 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 msgr2=0x7f15480aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c430 tx=0x7f153c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 shutdown_connections 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f153406b4d0 0x7f153406d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15481586f0 0x7f15480aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15480c5860 0x7f15480aa530 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 --2- 172.21.15.17:0/42610693 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15480c5480 0x7f1548151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 >> 172.21.15.17:0/42610693 conn(0x7f154806b1a0 msgr2=0x7f1548070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 shutdown_connections 2024-03-06T07:24:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.654+0000 7f154fd6c640 1 -- 172.21.15.17:0/42610693 wait complete. 2024-03-06T07:24:26.667 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:24:26.667 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:24:26.667 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:24:26.667 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:26.667 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:24:26.668 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:24:26.668 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:24:26.668 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:26.668 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:24:26.668 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:24:26.668 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:24:26.668 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:26.669 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:24:26.669 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:24:26.669 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:24:26.669 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:24:26.676 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:24:26.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:24:26.677 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:24:26.677 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:24:26.680 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:24:26.680 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:24:26.681 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:24:26.681 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:24:26.719 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:24:26.719 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:26.719 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:24:26.720 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:26.723 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:24:26.723 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:26.724 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:24:26.724 DEBUG:teuthology.parallel:result is None 2024-03-06T07:24:26.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:24:26.794 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:24:26.795 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:24:26.805 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:24:26.805 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:24:26.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.830+0000 7f1bf45e9640 1 Processor -- start 2024-03-06T07:24:26.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.830+0000 7f1bf45e9640 1 -- start start 2024-03-06T07:24:26.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bec0c6ab0 0x7f1bec1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec1561a0 0x7f1bec0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1bec0aa430 0x7f1bec14f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec155d10 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0a9990 con 0x7f1bec0aa430 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf45e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1556c0 con 0x7f1bec1561a0 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bec0c6ab0 0x7f1bec1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1bec0c6ab0 0x7f1bec1529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40992/0 (socket says 172.21.15.17:40992) 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 -- 172.21.15.17:0/1150245723 learned_addr learned my addr 172.21.15.17:0/1150245723 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf1b84640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec1561a0 0x7f1bec0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2b86640 1 --2- 172.21.15.17:0/1150245723 >> v2:172.21.15.163:3300/0 conn(0x7f1bec0aa430 0x7f1bec14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 -- 172.21.15.17:0/1150245723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec1561a0 msgr2=0x7f1bec0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec1561a0 0x7f1bec0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 -- 172.21.15.17:0/1150245723 >> v2:172.21.15.163:3300/0 conn(0x7f1bec0aa430 msgr2=0x7f1bec14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:26.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 --2- 172.21.15.17:0/1150245723 >> v2:172.21.15.163:3300/0 conn(0x7f1bec0aa430 0x7f1bec14f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.831+0000 7f1bf2385640 1 -- 172.21.15.17:0/1150245723 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec1559a0 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.832+0000 7f1bf2385640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec0c6ab0 0x7f1bec1529e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0a92d0 tx=0x7f1be802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207878128844d174 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.832+0000 7f1bf1383640 1 -- 172.21.15.17:0/1150245723 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be803b070 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.832+0000 7f1bf1383640 1 -- 172.21.15.17:0/1150245723 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be802ee00 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.832+0000 7f1bf1383640 1 -- 172.21.15.17:0/1150245723 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8036660 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 -- 172.21.15.17:0/1150245723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec0c6ab0 msgr2=0x7f1bec1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec0c6ab0 0x7f1bec1529e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0a92d0 tx=0x7f1be802e050 comp rx=0 tx=0).stop 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 -- 172.21.15.17:0/1150245723 shutdown_connections 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/1150245723 >> v2:172.21.15.163:3300/0 conn(0x7f1bec0aa430 0x7f1bec14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec1561a0 0x7f1bec0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/1150245723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec0c6ab0 0x7f1bec1529e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 -- 172.21.15.17:0/1150245723 >> 172.21.15.17:0/1150245723 conn(0x7f1bec06b1a0 msgr2=0x7f1bec06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:26.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 -- 172.21.15.17:0/1150245723 shutdown_connections 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.833+0000 7f1bf45e9640 1 -- 172.21.15.17:0/1150245723 wait complete. 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.834+0000 7f1bf45e9640 1 Processor -- start 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.834+0000 7f1bf45e9640 1 -- start start 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bec0aa430 0x7f1bec1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 0x7f1bec0717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c2c10 con 0x7f1bec1561a0 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c2a90 con 0x7f1bec0aa430 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf45e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0c6e90 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf2b86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf2b86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40994/0 (socket says 172.21.15.17:40994) 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf2b86640 1 -- 172.21.15.17:0/3858585973 learned_addr learned my addr 172.21.15.17:0/3858585973 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 0x7f1bec0717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf2385640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bec0aa430 0x7f1bec1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 -- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bec0aa430 msgr2=0x7f1bec1544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bec0aa430 0x7f1bec1544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 -- 172.21.15.17:0/3858585973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 msgr2=0x7f1bec144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 -- 172.21.15.17:0/3858585973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec155210 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf2b86640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.835+0000 7f1bf1b84640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 0x7f1bec0717c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e880 tx=0x7f1bdc00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc017070 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc00c940 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec1e7540 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00cd30 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec071d00 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.836+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec04f330 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.837+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bdc01a070 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.837+0000 7f1be2ffd640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 0x7f1bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.838+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1bdc08a080 con 0x7f1bec0c6ab0 2024-03-06T07:24:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.838+0000 7f1bf2385640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 0x7f1bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:26.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.838+0000 7f1bf2385640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 0x7f1bd006d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1be8038000 tx=0x7f1be8032000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:26.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:26.842+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1bdc058930 con 0x7f1bec0c6ab0 2024-03-06T07:24:27.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.039+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1bec14c870 con 0x7f1bec0c6ab0 2024-03-06T07:24:27.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.856+0000 7f1be2ffd640 1 -- 172.21.15.17:0/3858585973 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1bdc05c9d0 con 0x7f1bec0c6ab0 2024-03-06T07:24:27.858 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 msgr2=0x7f1bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 0x7f1bd006d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1be8038000 tx=0x7f1be8032000 comp rx=0 tx=0).stop 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 msgr2=0x7f1bec0717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 0x7f1bec0717c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00e880 tx=0x7f1bdc00ed40 comp rx=0 tx=0).stop 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 shutdown_connections 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1bd006b4d0 0x7f1bd006d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1bec1561a0 0x7f1bec144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1bec0c6ab0 0x7f1bec0717c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 --2- 172.21.15.17:0/3858585973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1bec0aa430 0x7f1bec1544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 >> 172.21.15.17:0/3858585973 conn(0x7f1bec06b1a0 msgr2=0x7f1bec0c1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 shutdown_connections 2024-03-06T07:24:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.859+0000 7f1bf45e9640 1 -- 172.21.15.17:0/3858585973 wait complete. 2024-03-06T07:24:27.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:24:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.980+0000 7fde4693e640 1 Processor -- start 2024-03-06T07:24:27.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.980+0000 7fde4693e640 1 -- start start 2024-03-06T07:24:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.981+0000 7fde4693e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.981+0000 7fde4693e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.981+0000 7fde4693e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde400c5cc0 con 0x7fde401586f0 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.981+0000 7fde4693e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4006df40 con 0x7fde400c5480 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.981+0000 7fde4693e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde401591c0 con 0x7fde400c5860 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde44edb640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fde401586f0 0x7fde400724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59194/0 (socket says 172.21.15.17:59194) 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:35910/0 (socket says 172.21.15.17:35910) 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde44edb640 1 -- 172.21.15.17:0/4132091750 learned_addr learned my addr 172.21.15.17:0/4132091750 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46204/0 (socket says 172.21.15.17:46204) 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 -- 172.21.15.17:0/4132091750 >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 msgr2=0x7fde40155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 --2- 172.21.15.17:0/4132091750 >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 -- 172.21.15.17:0/4132091750 >> v1:172.21.15.163:6789/0 conn(0x7fde401586f0 legacy=0x7fde400724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3f7fe640 1 -- 172.21.15.17:0/4132091750 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde401594a0 con 0x7fde400c5860 2024-03-06T07:24:27.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3effd640 1 -- 172.21.15.17:0/4132091750 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688004422 0 0) 0x7fde400c5cc0 con 0x7fde401586f0 2024-03-06T07:24:27.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.982+0000 7fde3ffff640 1 --2- 172.21.15.17:0/4132091750 >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.983+0000 7fde3f7fe640 1 --2- 172.21.15.17:0/4132091750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde340097a0 tx=0x7fde3402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f09d1de3f72e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.983+0000 7fde3effd640 1 -- 172.21.15.17:0/4132091750 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340315a0 con 0x7fde400c5860 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.983+0000 7fde3effd640 1 -- 172.21.15.17:0/4132091750 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34037070 con 0x7fde400c5860 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.983+0000 7fde3effd640 1 -- 172.21.15.17:0/4132091750 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34031960 con 0x7fde400c5860 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 -- 172.21.15.17:0/4132091750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 msgr2=0x7fde4006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 --2- 172.21.15.17:0/4132091750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde340097a0 tx=0x7fde3402b840 comp rx=0 tx=0).stop 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 -- 172.21.15.17:0/4132091750 shutdown_connections 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 --2- 172.21.15.17:0/4132091750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde4006f6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 --2- 172.21.15.17:0/4132091750 >> v2:172.21.15.163:3300/0 conn(0x7fde400c5480 0x7fde40155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 -- 172.21.15.17:0/4132091750 >> 172.21.15.17:0/4132091750 conn(0x7fde4006b1a0 msgr2=0x7fde4006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 -- 172.21.15.17:0/4132091750 shutdown_connections 2024-03-06T07:24:27.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.984+0000 7fde4693e640 1 -- 172.21.15.17:0/4132091750 wait complete. 2024-03-06T07:24:27.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.985+0000 7fde4693e640 1 Processor -- start 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.985+0000 7fde4693e640 1 -- start start 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde400c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde401586f0 0x7fde40156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40152a20 con 0x7fde400c5480 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40155680 con 0x7fde401586f0 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde40155050 con 0x7fde400c5860 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41000/0 (socket says 172.21.15.17:41000) 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 -- 172.21.15.17:0/3788394931 learned_addr learned my addr 172.21.15.17:0/3788394931 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3f7fe640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde400c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde44edb640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde401586f0 0x7fde40156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 -- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 msgr2=0x7fde400c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde400c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 -- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde401586f0 msgr2=0x7fde40156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde401586f0 0x7fde40156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 -- 172.21.15.17:0/3788394931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde401507b0 con 0x7fde400c5480 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3ffff640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ec10 tx=0x7fde3000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30017070 con 0x7fde400c5480 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3000cc30 con 0x7fde400c5480 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.986+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde400c4e60 con 0x7fde400c5480 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.987+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30013470 con 0x7fde400c5480 2024-03-06T07:24:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.987+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde40151870 con 0x7fde400c5480 2024-03-06T07:24:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.988+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40151ab0 con 0x7fde400c5480 2024-03-06T07:24:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.988+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde3001a070 con 0x7fde400c5480 2024-03-06T07:24:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.988+0000 7fde3cff9640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.988+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fde30089e60 con 0x7fde400c5480 2024-03-06T07:24:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.989+0000 7fde3f7fe640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.989+0000 7fde3f7fe640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde34036000 tx=0x7fde3403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:27.993+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde40151ab0 con 0x7fde400c5480 2024-03-06T07:24:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.244+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fde40075300 con 0x7fde400c5480 2024-03-06T07:24:28.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.244+0000 7fde3cff9640 1 -- 172.21.15.17:0/3788394931 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+4724 (secure 0 0 0) 0x7fde40075300 con 0x7fde400c5480 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:24:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:24:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:24:28.252 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:24:28.253 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.258 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:24:28.260 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 msgr2=0x7fde2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 0x7fde2c06d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fde34036000 tx=0x7fde3403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 msgr2=0x7fde4014fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fde3000ec10 tx=0x7fde3000c520 comp rx=0 tx=0).stop 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 shutdown_connections 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde2c06b4d0 0x7fde2c06d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde401586f0 0x7fde40156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde400c5860 0x7fde400c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 --2- 172.21.15.17:0/3788394931 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde400c5480 0x7fde4014fb80 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 >> 172.21.15.17:0/3788394931 conn(0x7fde4006b1a0 msgr2=0x7fde400c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 shutdown_connections 2024-03-06T07:24:28.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.246+0000 7fde4693e640 1 -- 172.21.15.17:0/3788394931 wait complete. 2024-03-06T07:24:28.263 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.362+0000 7f724040e640 1 Processor -- start 2024-03-06T07:24:28.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.362+0000 7f724040e640 1 -- start start 2024-03-06T07:24:28.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.363+0000 7f724040e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.363+0000 7f724040e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72381586f0 0x7f7238158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.363+0000 7f724040e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380c5cc0 con 0x7f72380c5860 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.363+0000 7f724040e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723806df40 con 0x7f72381586f0 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.363+0000 7f724040e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723806fdb0 con 0x7f72380c5480 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:46214/0 (socket says 172.21.15.17:46214) 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 -- 172.21.15.17:0/1520562260 learned_addr learned my addr 172.21.15.17:0/1520562260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e9ab640 1 --2- 172.21.15.17:0/1520562260 >> v2:172.21.15.163:3300/0 conn(0x7f72381586f0 0x7f7238158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 -- 172.21.15.17:0/1520562260 >> v2:172.21.15.163:3300/0 conn(0x7f72381586f0 msgr2=0x7f7238158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 --2- 172.21.15.17:0/1520562260 >> v2:172.21.15.163:3300/0 conn(0x7f72381586f0 0x7f7238158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 -- 172.21.15.17:0/1520562260 >> v1:172.21.15.163:6789/0 conn(0x7f72380c5860 legacy=0x7f723806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.364+0000 7f723e1aa640 1 -- 172.21.15.17:0/1520562260 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72381591c0 con 0x7f72380c5480 2024-03-06T07:24:28.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f723e1aa640 1 --2- 172.21.15.17:0/1520562260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f723402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78aa81fb21cf0938 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f723d1a8640 1 -- 172.21.15.17:0/1520562260 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234035020 con 0x7f72380c5480 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f723d1a8640 1 -- 172.21.15.17:0/1520562260 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234031640 con 0x7f72380c5480 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f723d1a8640 1 -- 172.21.15.17:0/1520562260 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723402d040 con 0x7f72380c5480 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f724040e640 1 -- 172.21.15.17:0/1520562260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 msgr2=0x7f7238155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.365+0000 7f724040e640 1 --2- 172.21.15.17:0/1520562260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f723402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 -- 172.21.15.17:0/1520562260 shutdown_connections 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 --2- 172.21.15.17:0/1520562260 >> v2:172.21.15.163:3300/0 conn(0x7f72381586f0 0x7f7238158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 --2- 172.21.15.17:0/1520562260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5480 0x7f7238155ba0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 -- 172.21.15.17:0/1520562260 >> 172.21.15.17:0/1520562260 conn(0x7f723806b1a0 msgr2=0x7f723806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 -- 172.21.15.17:0/1520562260 shutdown_connections 2024-03-06T07:24:28.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.366+0000 7f724040e640 1 -- 172.21.15.17:0/1520562260 wait complete. 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 Processor -- start 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 -- start start 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5860 0x7f72380c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72381586f0 0x7f72381e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7238155680 con 0x7f72380c5480 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72381553a0 con 0x7f72381586f0 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f724040e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7238152a20 con 0x7f72380c5860 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f723e1aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f723e1aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41008/0 (socket says 172.21.15.17:41008) 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.367+0000 7f723e1aa640 1 -- 172.21.15.17:0/1953283267 learned_addr learned my addr 172.21.15.17:0/1953283267 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e9ab640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72381586f0 0x7f72381e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723d9a9640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5860 0x7f72380c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 -- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5860 msgr2=0x7f72380c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5860 0x7f72380c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 -- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72381586f0 msgr2=0x7f72381e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72381586f0 0x7f72381e1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 -- 172.21.15.17:0/1953283267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72380c3500 con 0x7f72380c5480 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f723e1aa640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f723402b930 tx=0x7f72340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234040070 con 0x7f72380c5480 2024-03-06T07:24:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723403e070 con 0x7f72380c5480 2024-03-06T07:24:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7238157cb0 con 0x7f72380c5480 2024-03-06T07:24:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234035040 con 0x7f72380c5480 2024-03-06T07:24:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.368+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238158270 con 0x7f72380c5480 2024-03-06T07:24:28.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.369+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72380ac150 con 0x7f72380c5480 2024-03-06T07:24:28.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.370+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7234038020 con 0x7f72380c5480 2024-03-06T07:24:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.370+0000 7f722effd640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 0x7f721c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.370+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f72340ab9b0 con 0x7f72380c5480 2024-03-06T07:24:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.370+0000 7f723d9a9640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 0x7f721c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.371+0000 7f723d9a9640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 0x7f721c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f723806fc80 tx=0x7f7228009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.375+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f723407aa80 con 0x7f72380c5480 2024-03-06T07:24:28.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.555+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f723804f330 con 0x7f72380c5480 2024-03-06T07:24:28.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.556+0000 7f722effd640 1 -- 172.21.15.17:0/1953283267 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f7234043e40 con 0x7f72380c5480 2024-03-06T07:24:28.558 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:28.558 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":118,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:27.852481+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 msgr2=0x7f721c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 0x7f721c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f723806fc80 tx=0x7f7228009040 comp rx=0 tx=0).stop 2024-03-06T07:24:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 msgr2=0x7f72381ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f723402b930 tx=0x7f72340046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 shutdown_connections 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f721c06b500 0x7f721c06d9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72381586f0 0x7f72381e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72380c5860 0x7f72380c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 --2- 172.21.15.17:0/1953283267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72380c5480 0x7f72381ead40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 >> 172.21.15.17:0/1953283267 conn(0x7f723806b1a0 msgr2=0x7f72380c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 shutdown_connections 2024-03-06T07:24:28.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.558+0000 7f724040e640 1 -- 172.21.15.17:0/1953283267 wait complete. 2024-03-06T07:24:28.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:28.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.674+0000 7f2d3d215640 1 Processor -- start 2024-03-06T07:24:28.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.674+0000 7f2d3d215640 1 -- start start 2024-03-06T07:24:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3d215640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 0x7f2d38155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3d215640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d380c2aa0 0x7f2d3806bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3d215640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3806c110 con 0x7f2d380c0470 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3d215640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3806b740 con 0x7f2d380c2aa0 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3d215640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380670c0 con 0x7f2d380c26c0 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d380c2aa0 0x7f2d3806bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3759e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2d380c0470 0x7f2d38152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45920/0 (socket says 172.21.15.17:45920) 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3759e640 1 -- 172.21.15.17:0/2540132487 learned_addr learned my addr 172.21.15.17:0/2540132487 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d36d9d640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 0x7f2d38155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3659c640 1 -- 172.21.15.17:0/2540132487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 msgr2=0x7f2d38155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3659c640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 0x7f2d38155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3659c640 1 -- 172.21.15.17:0/2540132487 >> v1:172.21.15.17:6789/0 conn(0x7f2d380c0470 legacy=0x7f2d38152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d3659c640 1 -- 172.21.15.17:0/2540132487 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3806b0f0 con 0x7f2d380c2aa0 2024-03-06T07:24:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d35d9b640 1 -- 172.21.15.17:0/2540132487 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065486131 0 0) 0x7f2d3806c110 con 0x7f2d380c0470 2024-03-06T07:24:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.675+0000 7f2d36d9d640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 0x7f2d38155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.676+0000 7f2d3659c640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d3806bbd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097a0 tx=0x7f2d2002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc61cb91ff735ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.676+0000 7f2d35d9b640 1 -- 172.21.15.17:0/2540132487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20038070 con 0x7f2d380c2aa0 2024-03-06T07:24:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.676+0000 7f2d35d9b640 1 -- 172.21.15.17:0/2540132487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20031700 con 0x7f2d380c2aa0 2024-03-06T07:24:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.676+0000 7f2d35d9b640 1 -- 172.21.15.17:0/2540132487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20031ad0 con 0x7f2d380c2aa0 2024-03-06T07:24:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 -- 172.21.15.17:0/2540132487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 msgr2=0x7f2d3806bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d3806bbd0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097a0 tx=0x7f2d2002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 -- 172.21.15.17:0/2540132487 shutdown_connections 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d3806bbd0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 --2- 172.21.15.17:0/2540132487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c26c0 0x7f2d38155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 -- 172.21.15.17:0/2540132487 >> 172.21.15.17:0/2540132487 conn(0x7f2d380c3b40 msgr2=0x7f2d380c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 -- 172.21.15.17:0/2540132487 shutdown_connections 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.677+0000 7f2d3d215640 1 -- 172.21.15.17:0/2540132487 wait complete. 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.678+0000 7f2d3d215640 1 Processor -- start 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.678+0000 7f2d3d215640 1 -- start start 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c0470 0x7f2d3814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d380c26c0 0x7f2d381eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380c1f30 con 0x7f2d380c2aa0 2024-03-06T07:24:28.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380c0900 con 0x7f2d380c26c0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3d215640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d380c20b0 con 0x7f2d380c0470 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41034/0 (socket says 172.21.15.17:41034) 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 -- 172.21.15.17:0/3750570336 learned_addr learned my addr 172.21.15.17:0/3750570336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d36d9d640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c0470 0x7f2d3814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3659c640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d380c26c0 0x7f2d381eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 -- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c0470 msgr2=0x7f2d3814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c0470 0x7f2d3814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 -- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d380c26c0 msgr2=0x7f2d381eb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d380c26c0 0x7f2d381eb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 -- 172.21.15.17:0/3750570336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3814e930 con 0x7f2d380c2aa0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.679+0000 7f2d3759e640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c450 tx=0x7f2d3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.680+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30018070 con 0x7f2d380c2aa0 2024-03-06T07:24:28.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.680+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d30007500 con 0x7f2d380c2aa0 2024-03-06T07:24:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.680+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3814e370 con 0x7f2d380c2aa0 2024-03-06T07:24:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.680+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004a20 con 0x7f2d380c2aa0 2024-03-06T07:24:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.680+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d380abb30 con 0x7f2d380c2aa0 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.681+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3804f330 con 0x7f2d380c2aa0 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.681+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d30004bc0 con 0x7f2d380c2aa0 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.681+0000 7f2d2f7fe640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.681+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2d300891b0 con 0x7f2d380c2aa0 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.682+0000 7f2d36d9d640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.682+0000 7f2d36d9d640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d28003b80 tx=0x7f2d28009910 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.686+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d30057bb0 con 0x7f2d380c2aa0 2024-03-06T07:24:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.865+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d3814bb20 con 0x7f2d380c2aa0 2024-03-06T07:24:28.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.866+0000 7f2d2f7fe640 1 -- 172.21.15.17:0/3750570336 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d3005bc50 con 0x7f2d380c2aa0 2024-03-06T07:24:28.867 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:24:28.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 msgr2=0x7f2d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 0x7f2d1806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d28003b80 tx=0x7f2d28009910 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 msgr2=0x7f2d38071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000c450 tx=0x7f2d3000c910 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 shutdown_connections 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d1806b4d0 0x7f2d1806d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d380c2aa0 0x7f2d38071fc0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d380c26c0 0x7f2d381eb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 --2- 172.21.15.17:0/3750570336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d380c0470 0x7f2d3814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 >> 172.21.15.17:0/3750570336 conn(0x7f2d380c3b40 msgr2=0x7f2d381513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 shutdown_connections 2024-03-06T07:24:28.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.868+0000 7f2d3d215640 1 -- 172.21.15.17:0/3750570336 wait complete. 2024-03-06T07:24:28.880 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:24:28.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.980+0000 7f441d4c2640 1 Processor -- start 2024-03-06T07:24:28.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.980+0000 7f441d4c2640 1 -- start start 2024-03-06T07:24:28.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f441d4c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44181527b0 0x7f4418159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f441d4c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c88b0 0x7f44180c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f441d4c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180c7c60 con 0x7f44180c7180 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f441d4c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4418159640 con 0x7f44181527b0 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f441d4c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c8060 con 0x7f44180c88b0 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f4416ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44181527b0 0x7f4418159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f44167fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f44180c7180 0x7f4418155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45924/0 (socket says 172.21.15.17:45924) 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f44167fc640 1 -- 172.21.15.17:0/2871263581 learned_addr learned my addr 172.21.15.17:0/2871263581 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.981+0000 7f4416ffd640 1 -- 172.21.15.17:0/2871263581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c88b0 msgr2=0x7f44180c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f44177fe640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c88b0 0x7f44180c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4416ffd640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c88b0 0x7f44180c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4416ffd640 1 -- 172.21.15.17:0/2871263581 >> v1:172.21.15.17:6789/0 conn(0x7f44180c7180 legacy=0x7f4418155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4415ffb640 1 -- 172.21.15.17:0/2871263581 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892537152 0 0) 0x7f4400001110 con 0x7f44180c7180 2024-03-06T07:24:28.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4416ffd640 1 -- 172.21.15.17:0/2871263581 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180c76a0 con 0x7f44181527b0 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4416ffd640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f4418159100 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af48c06be581a435 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4415ffb640 1 -- 172.21.15.17:0/2871263581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100315a0 con 0x7f44181527b0 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.982+0000 7f4415ffb640 1 -- 172.21.15.17:0/2871263581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f44181527b0 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f4415ffb640 1 -- 172.21.15.17:0/2871263581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410031960 con 0x7f44181527b0 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 -- 172.21.15.17:0/2871263581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 msgr2=0x7f4418159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f4418159100 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f44100097a0 tx=0x7f441002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 -- 172.21.15.17:0/2871263581 shutdown_connections 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c88b0 0x7f44180c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 --2- 172.21.15.17:0/2871263581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f4418159100 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 -- 172.21.15.17:0/2871263581 >> 172.21.15.17:0/2871263581 conn(0x7f44180ac560 msgr2=0x7f44180ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 -- 172.21.15.17:0/2871263581 shutdown_connections 2024-03-06T07:24:28.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.983+0000 7f441d4c2640 1 -- 172.21.15.17:0/2871263581 wait complete. 2024-03-06T07:24:28.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.984+0000 7f441d4c2640 1 Processor -- start 2024-03-06T07:24:28.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.984+0000 7f441d4c2640 1 -- start start 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c7180 0x7f44180c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 0x7f44180c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418155600 con 0x7f44181527b0 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418155320 con 0x7f44180c88b0 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f441d4c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c8d60 con 0x7f44180c7180 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41050/0 (socket says 172.21.15.17:41050) 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 -- 172.21.15.17:0/1015332087 learned_addr learned my addr 172.21.15.17:0/1015332087 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44167fc640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 0x7f44180c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f4416ffd640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c7180 0x7f44180c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 -- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c7180 msgr2=0x7f44180c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c7180 0x7f44180c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 -- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 msgr2=0x7f44180c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 0x7f44180c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44177fe640 1 -- 172.21.15.17:0/1015332087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180c31b0 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.985+0000 7f44167fc640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 0x7f44180c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f44177fe640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f440800cda0 tx=0x7f440800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408003b90 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408003d30 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44180c3490 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408004240 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.986+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44180c3a50 con 0x7f44181527b0 2024-03-06T07:24:28.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.987+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4418075300 con 0x7f44181527b0 2024-03-06T07:24:28.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.987+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44080043e0 con 0x7f44181527b0 2024-03-06T07:24:28.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.988+0000 7f440f7fe640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 0x7f43f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:28.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.988+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f440808ab20 con 0x7f44181527b0 2024-03-06T07:24:28.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.988+0000 7f4416ffd640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 0x7f43f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:28.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.988+0000 7f4416ffd640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 0x7f43f806d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4410036000 tx=0x7f4410008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:28.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:28.992+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4418075300 con 0x7f44181527b0 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.175+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f441804f330 con 0x7f44181527b0 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.175+0000 7f440f7fe640 1 -- 172.21.15.17:0/1015332087 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4408059bf0 con 0x7f44181527b0 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:24:29.177 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:24:29.178 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:24:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 msgr2=0x7f43f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 0x7f43f806d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4410036000 tx=0x7f4410008030 comp rx=0 tx=0).stop 2024-03-06T07:24:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 msgr2=0x7f441814b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f440800cda0 tx=0x7f440800a660 comp rx=0 tx=0).stop 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 shutdown_connections 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f43f806b4d0 0x7f43f806d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44181527b0 0x7f441814b180 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44180c88b0 0x7f44180c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 --2- 172.21.15.17:0/1015332087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44180c7180 0x7f44180c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 >> 172.21.15.17:0/1015332087 conn(0x7f44180ac560 msgr2=0x7f44180c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 shutdown_connections 2024-03-06T07:24:29.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.177+0000 7f441d4c2640 1 -- 172.21.15.17:0/1015332087 wait complete. 2024-03-06T07:24:29.190 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:24:29.190 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:24:29.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.293+0000 7f1355a5c640 1 Processor -- start 2024-03-06T07:24:29.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.293+0000 7f1355a5c640 1 -- start start 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f135013dc90 0x7f1350159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13500c53f0 0x7f135006b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13500c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135006c190 con 0x7f13500c53f0 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135006bde0 con 0x7f135013dc90 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f1355a5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135013e070 con 0x7f135006b700 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13500c53f0 0x7f135006b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f13500c53f0 0x7f135006b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41052/0 (socket says 172.21.15.17:41052) 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 -- 172.21.15.17:0/91803007 learned_addr learned my addr 172.21.15.17:0/91803007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:29.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134effd640 1 --2- 172.21.15.17:0/91803007 >> v2:172.21.15.163:3300/0 conn(0x7f135013dc90 0x7f1350159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 -- 172.21.15.17:0/91803007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 msgr2=0x7f13500c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 --2- 172.21.15.17:0/91803007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13500c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 -- 172.21.15.17:0/91803007 >> v2:172.21.15.163:3300/0 conn(0x7f135013dc90 msgr2=0x7f1350159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 --2- 172.21.15.17:0/91803007 >> v2:172.21.15.163:3300/0 conn(0x7f135013dc90 0x7f1350159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.294+0000 7f134e7fc640 1 -- 172.21.15.17:0/91803007 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135013e420 con 0x7f13500c53f0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.295+0000 7f134e7fc640 1 --2- 172.21.15.17:0/91803007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500c53f0 0x7f135006b1c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1350068720 tx=0x7f133802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2e8b3ac0b6e36c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.295+0000 7f134dffb640 1 -- 172.21.15.17:0/91803007 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133802e070 con 0x7f13500c53f0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.295+0000 7f134dffb640 1 -- 172.21.15.17:0/91803007 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338037070 con 0x7f13500c53f0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.295+0000 7f134dffb640 1 -- 172.21.15.17:0/91803007 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338031a00 con 0x7f13500c53f0 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 -- 172.21.15.17:0/91803007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500c53f0 msgr2=0x7f135006b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 --2- 172.21.15.17:0/91803007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500c53f0 0x7f135006b1c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1350068720 tx=0x7f133802b880 comp rx=0 tx=0).stop 2024-03-06T07:24:29.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 -- 172.21.15.17:0/91803007 shutdown_connections 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 --2- 172.21.15.17:0/91803007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13500c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 --2- 172.21.15.17:0/91803007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13500c53f0 0x7f135006b1c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 --2- 172.21.15.17:0/91803007 >> v2:172.21.15.163:3300/0 conn(0x7f135013dc90 0x7f1350159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 -- 172.21.15.17:0/91803007 >> 172.21.15.17:0/91803007 conn(0x7f13500caa70 msgr2=0x7f13500cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 -- 172.21.15.17:0/91803007 shutdown_connections 2024-03-06T07:24:29.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.296+0000 7f1355a5c640 1 -- 172.21.15.17:0/91803007 wait complete. 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.297+0000 7f1355a5c640 1 Processor -- start 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.297+0000 7f1355a5c640 1 -- start start 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13501370e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 0x7f1350134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13500c61f0 con 0x7f135013dc90 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135013e9e0 con 0x7f13500c53f0 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f1355a5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1350159770 con 0x7f135006b700 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41062/0 (socket says 172.21.15.17:41062) 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 -- 172.21.15.17:0/844971765 learned_addr learned my addr 172.21.15.17:0/844971765 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134e7fc640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 0x7f1350134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134effd640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13501370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 -- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 msgr2=0x7f13501370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13501370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 -- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 msgr2=0x7f1350134d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 0x7f1350134d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 -- 172.21.15.17:0/844971765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13501eee90 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134e7fc640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 0x7f1350134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.298+0000 7f134f7fe640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f134000b9a0 tx=0x7f134000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.299+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340019070 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.299+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134000cbc0 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.299+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501de800 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.299+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340014440 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.299+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1350137620 con 0x7f135013dc90 2024-03-06T07:24:29.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.300+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1350075300 con 0x7f135013dc90 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.300+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f134001c070 con 0x7f135013dc90 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.300+0000 7f1347fff640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 0x7f133006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.301+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1340089cc0 con 0x7f135013dc90 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.301+0000 7f134effd640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 0x7f133006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.301+0000 7f134effd640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 0x7f133006d9c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f134800a970 tx=0x7f1348008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.305+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1350075300 con 0x7f135013dc90 2024-03-06T07:24:29.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.487+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f135004f330 con 0x7f135013dc90 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.860+0000 7f1347fff640 1 -- 172.21.15.17:0/844971765 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1340058d90 con 0x7f135013dc90 2024-03-06T07:24:29.862 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:24:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.862+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 msgr2=0x7f133006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.862+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 0x7f133006d9c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f134800a970 tx=0x7f1348008040 comp rx=0 tx=0).stop 2024-03-06T07:24:29.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.862+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 msgr2=0x7f13501352a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.862+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f134000b9a0 tx=0x7f134000be60 comp rx=0 tx=0).stop 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 shutdown_connections 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f133006b500 0x7f133006d9c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f135013dc90 0x7f13501352a0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13500c53f0 0x7f1350134d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 --2- 172.21.15.17:0/844971765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f135006b700 0x7f13501370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 >> 172.21.15.17:0/844971765 conn(0x7f13500caa70 msgr2=0x7f13500c0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 shutdown_connections 2024-03-06T07:24:29.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.863+0000 7f1355a5c640 1 -- 172.21.15.17:0/844971765 wait complete. 2024-03-06T07:24:29.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:24:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.986+0000 7fc48f325640 1 Processor -- start 2024-03-06T07:24:29.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.986+0000 7fc48f325640 1 -- start start 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48f325640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4880c5370 0x7fc4880c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48f325640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc488152dd0 con 0x7fc4881529f0 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48f325640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc488158b00 con 0x7fc488155530 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48f325640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc488159710 con 0x7fc4880c5370 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48c8c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4880c5370 0x7fc4880c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48d0c1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc4881529f0 0x7fc4880c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45932/0 (socket says 172.21.15.17:45932) 2024-03-06T07:24:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48d0c1640 1 -- 172.21.15.17:0/1197506423 learned_addr learned my addr 172.21.15.17:0/1197506423 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48c8c0640 1 -- 172.21.15.17:0/1197506423 >> v1:172.21.15.163:6789/0 conn(0x7fc488155530 legacy=0x7fc4881559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48c8c0640 1 -- 172.21.15.17:0/1197506423 >> v1:172.21.15.17:6789/0 conn(0x7fc4881529f0 legacy=0x7fc4880c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc487fff640 1 -- 172.21.15.17:0/1197506423 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008596515 0 0) 0x7fc488152dd0 con 0x7fc4881529f0 2024-03-06T07:24:29.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.987+0000 7fc48c8c0640 1 -- 172.21.15.17:0/1197506423 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc488158de0 con 0x7fc4880c5370 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.988+0000 7fc48c8c0640 1 --2- 172.21.15.17:0/1197506423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c57d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0097a0 tx=0x7fc47c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7524955294fe21c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.988+0000 7fc487fff640 1 -- 172.21.15.17:0/1197506423 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0315a0 con 0x7fc4880c5370 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.988+0000 7fc487fff640 1 -- 172.21.15.17:0/1197506423 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c037070 con 0x7fc4880c5370 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.988+0000 7fc487fff640 1 -- 172.21.15.17:0/1197506423 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c031960 con 0x7fc4880c5370 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 -- 172.21.15.17:0/1197506423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 msgr2=0x7fc4880c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 --2- 172.21.15.17:0/1197506423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c57d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0097a0 tx=0x7fc47c02b840 comp rx=0 tx=0).stop 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 -- 172.21.15.17:0/1197506423 shutdown_connections 2024-03-06T07:24:29.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 --2- 172.21.15.17:0/1197506423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c57d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 -- 172.21.15.17:0/1197506423 >> 172.21.15.17:0/1197506423 conn(0x7fc4880ac580 msgr2=0x7fc4880ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 -- 172.21.15.17:0/1197506423 shutdown_connections 2024-03-06T07:24:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.989+0000 7fc48f325640 1 -- 172.21.15.17:0/1197506423 wait complete. 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.990+0000 7fc48f325640 1 Processor -- start 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.990+0000 7fc48f325640 1 -- start start 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4881529f0 0x7fc4880c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 0x7fc48814dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880c9000 con 0x7fc4880c5370 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880c68b0 con 0x7fc488155530 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881593a0 con 0x7fc4881529f0 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41086/0 (socket says 172.21.15.17:41086) 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 -- 172.21.15.17:0/2298507812 learned_addr learned my addr 172.21.15.17:0/2298507812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d8c2640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 0x7fc48814dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48c8c0640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4881529f0 0x7fc4880c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 -- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4881529f0 msgr2=0x7fc4880c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4881529f0 0x7fc4880c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 -- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 msgr2=0x7fc48814dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 0x7fc48814dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 -- 172.21.15.17:0/2298507812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4881ef260 con 0x7fc4880c5370 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d8c2640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 0x7fc48814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48d0c1640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c4d0 tx=0x7fc48000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480018070 con 0x7fc4880c5370 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480007500 con 0x7fc4880c5370 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4881eece0 con 0x7fc4880c5370 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.991+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480004a20 con 0x7fc4880c5370 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.992+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4880c2000 con 0x7fc4880c5370 2024-03-06T07:24:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.993+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48804f330 con 0x7fc4880c5370 2024-03-06T07:24:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.993+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48001b070 con 0x7fc4880c5370 2024-03-06T07:24:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.994+0000 7fc485ffb640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.994+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fc4800897f0 con 0x7fc4880c5370 2024-03-06T07:24:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.994+0000 7fc48c8c0640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:29.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.994+0000 7fc48c8c0640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc47c036000 tx=0x7fc47c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:29.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:29.998+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc480058730 con 0x7fc4880c5370 2024-03-06T07:24:30.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.178+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc4881eef80 con 0x7fc4880c5370 2024-03-06T07:24:30.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.862+0000 7fc485ffb640 1 -- 172.21.15.17:0/2298507812 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc480058030 con 0x7fc4880c5370 2024-03-06T07:24:30.864 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:24:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 msgr2=0x7fc47406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc47c036000 tx=0x7fc47c008030 comp rx=0 tx=0).stop 2024-03-06T07:24:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 msgr2=0x7fc4880c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:30.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c4d0 tx=0x7fc48000c990 comp rx=0 tx=0).stop 2024-03-06T07:24:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 shutdown_connections 2024-03-06T07:24:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc488155530 0x7fc48814dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4881529f0 0x7fc4880c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 --2- 172.21.15.17:0/2298507812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4880c5370 0x7fc4880c1ac0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:30.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.866+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 >> 172.21.15.17:0/2298507812 conn(0x7fc4880ac580 msgr2=0x7fc488155980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:30.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.867+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 shutdown_connections 2024-03-06T07:24:30.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:30.867+0000 7fc48f325640 1 -- 172.21.15.17:0/2298507812 wait complete. 2024-03-06T07:24:30.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:24:31.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.004+0000 7f5aacde9640 1 Processor -- start 2024-03-06T07:24:31.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.004+0000 7f5aacde9640 1 -- start start 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aacde9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5aa806da40 0x7f5aa8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aacde9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5aa80c7380 0x7f5aa80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aacde9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa8156050 con 0x7f5aa80c6780 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aacde9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa806de20 con 0x7f5aa80c7380 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aacde9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa8072630 con 0x7f5aa806da40 2024-03-06T07:24:31.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5aa80c7380 0x7f5aa80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5aa80c7380 0x7f5aa80c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57678/0 (socket says 172.21.15.17:57678) 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 -- 172.21.15.17:0/3564511119 learned_addr learned my addr 172.21.15.17:0/3564511119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa659c640 1 --2- 172.21.15.17:0/3564511119 >> v2:172.21.15.163:3300/0 conn(0x7f5aa806da40 0x7f5aa8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 -- 172.21.15.17:0/3564511119 >> v2:172.21.15.163:3300/0 conn(0x7f5aa806da40 msgr2=0x7f5aa8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 --2- 172.21.15.17:0/3564511119 >> v2:172.21.15.163:3300/0 conn(0x7f5aa806da40 0x7f5aa8155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 -- 172.21.15.17:0/3564511119 >> v1:172.21.15.163:6789/0 conn(0x7f5aa80c6780 legacy=0x7f5aa8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.005+0000 7f5aa6d9d640 1 -- 172.21.15.17:0/3564511119 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8072910 con 0x7f5aa80c7380 2024-03-06T07:24:31.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.006+0000 7f5aa6d9d640 1 --2- 172.21.15.17:0/3564511119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa80c7380 0x7f5aa80c3280 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80c98a0 tx=0x7f5aa002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7325953987d262 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.006+0000 7f5aa559a640 1 -- 172.21.15.17:0/3564511119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa002c9f0 con 0x7f5aa80c7380 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.006+0000 7f5aa559a640 1 -- 172.21.15.17:0/3564511119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa002cb90 con 0x7f5aa80c7380 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.006+0000 7f5aa559a640 1 -- 172.21.15.17:0/3564511119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00343f0 con 0x7f5aa80c7380 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 -- 172.21.15.17:0/3564511119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa80c7380 msgr2=0x7f5aa80c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3564511119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa80c7380 0x7f5aa80c3280 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80c98a0 tx=0x7f5aa002be50 comp rx=0 tx=0).stop 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 -- 172.21.15.17:0/3564511119 shutdown_connections 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3564511119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa80c7380 0x7f5aa80c3280 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3564511119 >> v2:172.21.15.163:3300/0 conn(0x7f5aa806da40 0x7f5aa8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 -- 172.21.15.17:0/3564511119 >> 172.21.15.17:0/3564511119 conn(0x7f5aa80aa630 msgr2=0x7f5aa80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 -- 172.21.15.17:0/3564511119 shutdown_connections 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.007+0000 7f5aacde9640 1 -- 172.21.15.17:0/3564511119 wait complete. 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.008+0000 7f5aacde9640 1 Processor -- start 2024-03-06T07:24:31.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.008+0000 7f5aacde9640 1 -- start start 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa80c6780 0x7f5aa8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa80c7380 0x7f5aa8156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa81594a0 con 0x7f5aa806da40 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8158890 con 0x7f5aa80c7380 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa80c37c0 con 0x7f5aa80c6780 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57694/0 (socket says 172.21.15.17:57694) 2024-03-06T07:24:31.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 -- 172.21.15.17:0/3142887471 learned_addr learned my addr 172.21.15.17:0/3142887471 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa6d9d640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa80c7380 0x7f5aa8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa5d9b640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa80c6780 0x7f5aa8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 -- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa80c6780 msgr2=0x7f5aa8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa80c6780 0x7f5aa8157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 -- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa80c7380 msgr2=0x7f5aa8156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa80c7380 0x7f5aa8156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 -- 172.21.15.17:0/3142887471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa81eb970 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aa659c640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d800 tx=0x7f5a9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800b6c0 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a98019070 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.009+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa81eae50 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.010+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800ba80 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.010+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa81eb690 con 0x7f5aa806da40 2024-03-06T07:24:31.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.010+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa806cb20 con 0x7f5aa806da40 2024-03-06T07:24:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.011+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a9800bc20 con 0x7f5aa806da40 2024-03-06T07:24:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.011+0000 7f5a96ffd640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 0x7f5a8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.011+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f5a9808a6b0 con 0x7f5aa806da40 2024-03-06T07:24:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.012+0000 7f5aa5d9b640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 0x7f5a8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:31.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.012+0000 7f5aa5d9b640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 0x7f5a8806d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80aa5a0 tx=0x7f5a9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:31.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.016+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5aa806cb20 con 0x7f5aa806da40 2024-03-06T07:24:31.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.237+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5aa81585d0 con 0x7f5aa806da40 2024-03-06T07:24:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.886+0000 7f5a96ffd640 1 -- 172.21.15.17:0/3142887471 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v103) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5aa81eae50 con 0x7f5aa806da40 2024-03-06T07:24:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 19 and data pool 20 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 msgr2=0x7f5a8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 0x7f5a8806d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80aa5a0 tx=0x7f5a9c009040 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 msgr2=0x7f5aa814ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d800 tx=0x7f5a9800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 shutdown_connections 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a8806b4d0 0x7f5a8806d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5aa80c7380 0x7f5aa8156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5aa80c6780 0x7f5aa8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 --2- 172.21.15.17:0/3142887471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aa806da40 0x7f5aa814ac50 secure :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d800 tx=0x7f5a9800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 >> 172.21.15.17:0/3142887471 conn(0x7f5aa80aa630 msgr2=0x7f5aa80c2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.896+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 shutdown_connections 2024-03-06T07:24:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:31.897+0000 7f5aacde9640 1 -- 172.21.15.17:0/3142887471 wait complete. 2024-03-06T07:24:31.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:32.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.024+0000 7f54a1d5e640 1 Processor -- start 2024-03-06T07:24:32.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.024+0000 7f54a1d5e640 1 -- start start 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f54a1d5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f549c0c5330 0x7f549c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f54a1d5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c155110 con 0x7f549c158db0 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f54a1d5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0c8f40 con 0x7f549c0c88b0 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f54a1d5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c1559e0 con 0x7f549c0c5330 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f549c0c5330 0x7f549c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f549c158db0 0x7f549c15fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44556/0 (socket says 172.21.15.17:44556) 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549bfff640 1 -- 172.21.15.17:0/3138722865 learned_addr learned my addr 172.21.15.17:0/3138722865 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549b7fe640 1 -- 172.21.15.17:0/3138722865 >> v1:172.21.15.163:6789/0 conn(0x7f549c0c88b0 legacy=0x7f549c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549b7fe640 1 -- 172.21.15.17:0/3138722865 >> v1:172.21.15.17:6789/0 conn(0x7f549c158db0 legacy=0x7f549c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549a7fc640 1 -- 172.21.15.17:0/3138722865 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582030446 0 0) 0x7f5494001490 con 0x7f549c158db0 2024-03-06T07:24:32.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.025+0000 7f549b7fe640 1 -- 172.21.15.17:0/3138722865 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c155390 con 0x7f549c0c5330 2024-03-06T07:24:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.026+0000 7f549b7fe640 1 --2- 172.21.15.17:0/3138722865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c5330 0x7f549c0c5710 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f548c0097a0 tx=0x7f548c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2684ae8307a825ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.026+0000 7f549a7fc640 1 -- 172.21.15.17:0/3138722865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c035020 con 0x7f549c0c5330 2024-03-06T07:24:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.026+0000 7f549a7fc640 1 -- 172.21.15.17:0/3138722865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c031830 con 0x7f549c0c5330 2024-03-06T07:24:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.026+0000 7f549a7fc640 1 -- 172.21.15.17:0/3138722865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c031c00 con 0x7f549c0c5330 2024-03-06T07:24:32.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 -- 172.21.15.17:0/3138722865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c5330 msgr2=0x7f549c0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/3138722865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c5330 0x7f549c0c5710 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f548c0097a0 tx=0x7f548c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 -- 172.21.15.17:0/3138722865 shutdown_connections 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/3138722865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c5330 0x7f549c0c5710 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 -- 172.21.15.17:0/3138722865 >> 172.21.15.17:0/3138722865 conn(0x7f549c0ac560 msgr2=0x7f549c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 -- 172.21.15.17:0/3138722865 shutdown_connections 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.027+0000 7f54a1d5e640 1 -- 172.21.15.17:0/3138722865 wait complete. 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.028+0000 7f54a1d5e640 1 Processor -- start 2024-03-06T07:24:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.028+0000 7f54a1d5e640 1 -- start start 2024-03-06T07:24:32.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 0x7f549c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 0x7f549c1ec690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1529d0 con 0x7f549c0c88b0 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c06d090 con 0x7f549c0c5330 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f54a1d5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c152b50 con 0x7f549c156cd0 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57702/0 (socket says 172.21.15.17:57702) 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 -- 172.21.15.17:0/859005324 learned_addr learned my addr 172.21.15.17:0/859005324 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549b7fe640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 0x7f549c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 -- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 msgr2=0x7f549c1ec690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549bfff640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 0x7f549c1ec690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 0x7f549c1ec690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 -- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 msgr2=0x7f549c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 0x7f549c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 -- 172.21.15.17:0/859005324 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c1ee530 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549b7fe640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 0x7f549c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549bfff640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 0x7f549c1ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f549affd640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f549000c430 tx=0x7f549000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490017070 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.029+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54900052d0 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.030+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c1ee250 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.030+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490003830 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.030+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c1561f0 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.031+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c04f330 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.031+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f549001a070 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.031+0000 7f5483fff640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 0x7f548806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.031+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54900896a0 con 0x7f549c0c88b0 2024-03-06T07:24:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.032+0000 7f549b7fe640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 0x7f548806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.032+0000 7f549b7fe640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 0x7f548806d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f548c036000 tx=0x7f548c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.036+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5490058420 con 0x7f549c0c88b0 2024-03-06T07:24:32.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.219+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f549c06cdc0 con 0x7f549c0c88b0 2024-03-06T07:24:32.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.220+0000 7f5483fff640 1 -- 172.21.15.17:0/859005324 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f5490057d20 con 0x7f549c0c88b0 2024-03-06T07:24:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:32.222 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":121,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:31.239044+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:32.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.221+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 msgr2=0x7f548806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.221+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 0x7f548806d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f548c036000 tx=0x7f548c008030 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.221+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 msgr2=0x7f549c1ec150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.221+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f549000c430 tx=0x7f549000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 shutdown_connections 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f548806b4d0 0x7f548806d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f549c156cd0 0x7f549c1ec690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f549c0c88b0 0x7f549c1ec150 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 --2- 172.21.15.17:0/859005324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f549c0c5330 0x7f549c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 >> 172.21.15.17:0/859005324 conn(0x7f549c0ac560 msgr2=0x7f549c14f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 shutdown_connections 2024-03-06T07:24:32.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.222+0000 7f54a1d5e640 1 -- 172.21.15.17:0/859005324 wait complete. 2024-03-06T07:24:32.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.334+0000 7f5caa481640 1 Processor -- start 2024-03-06T07:24:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.334+0000 7f5caa481640 1 -- start start 2024-03-06T07:24:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152920 0x7f5ca4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5ca40c8a00 0x7f5ca4158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ca4159460 0x7f5ca40c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca41587b0 con 0x7f5ca4159460 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca4156180 con 0x7f5ca40c8a00 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5caa481640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40c7760 con 0x7f5ca4152920 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ca4159460 0x7f5ca40c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5ca4159460 0x7f5ca40c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57714/0 (socket says 172.21.15.17:57714) 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 -- 172.21.15.17:0/315966185 learned_addr learned my addr 172.21.15.17:0/315966185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca37fe640 1 --2- 172.21.15.17:0/315966185 >> v2:172.21.15.163:3300/0 conn(0x7f5ca40c8a00 0x7f5ca4158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca3fff640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152920 0x7f5ca4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 -- 172.21.15.17:0/315966185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152920 msgr2=0x7f5ca4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152920 0x7f5ca4155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 -- 172.21.15.17:0/315966185 >> v2:172.21.15.163:3300/0 conn(0x7f5ca40c8a00 msgr2=0x7f5ca4158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 --2- 172.21.15.17:0/315966185 >> v2:172.21.15.163:3300/0 conn(0x7f5ca40c8a00 0x7f5ca4158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.335+0000 7f5ca8a1e640 1 -- 172.21.15.17:0/315966185 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40c7a40 con 0x7f5ca4159460 2024-03-06T07:24:32.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.336+0000 7f5ca8a1e640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca4159460 0x7f5ca40c5330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5ca408c6e0 tx=0x7f5c9c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4343fda43c738900 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.336+0000 7f5ca2ffd640 1 -- 172.21.15.17:0/315966185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c02c9e0 con 0x7f5ca4159460 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.336+0000 7f5ca2ffd640 1 -- 172.21.15.17:0/315966185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c02cb80 con 0x7f5ca4159460 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.336+0000 7f5ca2ffd640 1 -- 172.21.15.17:0/315966185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0343f0 con 0x7f5ca4159460 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 -- 172.21.15.17:0/315966185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca4159460 msgr2=0x7f5ca40c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca4159460 0x7f5ca40c5330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5ca408c6e0 tx=0x7f5c9c02be40 comp rx=0 tx=0).stop 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 -- 172.21.15.17:0/315966185 shutdown_connections 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca4159460 0x7f5ca40c5330 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 --2- 172.21.15.17:0/315966185 >> v2:172.21.15.163:3300/0 conn(0x7f5ca40c8a00 0x7f5ca4158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 --2- 172.21.15.17:0/315966185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4152920 0x7f5ca4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 -- 172.21.15.17:0/315966185 >> 172.21.15.17:0/315966185 conn(0x7f5ca40ac560 msgr2=0x7f5ca40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 -- 172.21.15.17:0/315966185 shutdown_connections 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.337+0000 7f5caa481640 1 -- 172.21.15.17:0/315966185 wait complete. 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.338+0000 7f5caa481640 1 Processor -- start 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.338+0000 7f5caa481640 1 -- start start 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 0x7f5ca4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 0x7f5ca4147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca406d030 con 0x7f5ca40c8a00 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4158a00 con 0x7f5ca4152920 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40c7390 con 0x7f5ca4153560 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57718/0 (socket says 172.21.15.17:57718) 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 -- 172.21.15.17:0/947439649 learned_addr learned my addr 172.21.15.17:0/947439649 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca8a1e640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 0x7f5ca4147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca37fe640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 0x7f5ca4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 -- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 msgr2=0x7f5ca4147cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 0x7f5ca4147cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 -- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 msgr2=0x7f5ca4153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 0x7f5ca4153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 -- 172.21.15.17:0/947439649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca4153310 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca8a1e640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 0x7f5ca4147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca37fe640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 0x7f5ca4153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca3fff640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e870 tx=0x7f5c9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98017070 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9800c8e0 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca41555d0 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9800ccd0 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.339+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca41eea90 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.340+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca404f330 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.341+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c9801a070 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.341+0000 7f5ca0ff9640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 0x7f5c9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.341+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c9808a190 con 0x7f5ca40c8a00 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.341+0000 7f5ca37fe640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 0x7f5c9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.342+0000 7f5ca37fe640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 0x7f5c9006d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40aaab0 tx=0x7f5c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.345+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5c98058e70 con 0x7f5ca40c8a00 2024-03-06T07:24:32.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.526+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ca406c800 con 0x7f5ca40c8a00 2024-03-06T07:24:32.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.527+0000 7f5ca0ff9640 1 -- 172.21.15.17:0/947439649 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16913 (secure 0 0 0) 0x7f5c98058770 con 0x7f5ca40c8a00 2024-03-06T07:24:32.529 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:32.529 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":121,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:31.239044+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 msgr2=0x7f5c9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 0x7f5c9006d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40aaab0 tx=0x7f5c94008040 comp rx=0 tx=0).stop 2024-03-06T07:24:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 msgr2=0x7f5ca4155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800e870 tx=0x7f5c9800ed30 comp rx=0 tx=0).stop 2024-03-06T07:24:32.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 shutdown_connections 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c9006b500 0x7f5c9006d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5ca4153560 0x7f5ca4147cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5ca4152920 0x7f5ca4153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 --2- 172.21.15.17:0/947439649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5ca40c8a00 0x7f5ca4155090 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 >> 172.21.15.17:0/947439649 conn(0x7f5ca40ac560 msgr2=0x7f5ca414ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.529+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 shutdown_connections 2024-03-06T07:24:32.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.530+0000 7f5caa481640 1 -- 172.21.15.17:0/947439649 wait complete. 2024-03-06T07:24:32.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:24:32.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.645+0000 7f2651b93640 1 Processor -- start 2024-03-06T07:24:32.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.645+0000 7f2651b93640 1 -- start start 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f264c0c5880 0x7f264c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f264c0c5c60 0x7f264c0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c06f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c0723b0 con 0x7f264c0c5c60 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264c0728e0 con 0x7f264c0c5880 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f2651b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c072600 con 0x7f264c0c7720 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f264c0c5c60 0x7f264c0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f264c0c5c60 0x7f264c0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57728/0 (socket says 172.21.15.17:57728) 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 -- 172.21.15.17:0/3894588483 learned_addr learned my addr 172.21.15.17:0/3894588483 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264b7fe640 1 --2- 172.21.15.17:0/3894588483 >> v2:172.21.15.163:3300/0 conn(0x7f264c0c5880 0x7f264c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264bfff640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c06f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 -- 172.21.15.17:0/3894588483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 msgr2=0x7f264c06f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c06f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 -- 172.21.15.17:0/3894588483 >> v2:172.21.15.163:3300/0 conn(0x7f264c0c5880 msgr2=0x7f264c0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 --2- 172.21.15.17:0/3894588483 >> v2:172.21.15.163:3300/0 conn(0x7f264c0c5880 0x7f264c0c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264affd640 1 -- 172.21.15.17:0/3894588483 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c153af0 con 0x7f264c0c5c60 2024-03-06T07:24:32.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.646+0000 7f264bfff640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c06f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:32.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.647+0000 7f264affd640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5c60 0x7f264c0c71e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f264c070f80 tx=0x7f264002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb0d2f425107a56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.647+0000 7f264a7fc640 1 -- 172.21.15.17:0/3894588483 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640038070 con 0x7f264c0c5c60 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.647+0000 7f264a7fc640 1 -- 172.21.15.17:0/3894588483 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2640031740 con 0x7f264c0c5c60 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.647+0000 7f264a7fc640 1 -- 172.21.15.17:0/3894588483 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264002f050 con 0x7f264c0c5c60 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 -- 172.21.15.17:0/3894588483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5c60 msgr2=0x7f264c0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5c60 0x7f264c0c71e0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f264c070f80 tx=0x7f264002b7e0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 -- 172.21.15.17:0/3894588483 shutdown_connections 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c06f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 --2- 172.21.15.17:0/3894588483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5c60 0x7f264c0c71e0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 --2- 172.21.15.17:0/3894588483 >> v2:172.21.15.163:3300/0 conn(0x7f264c0c5880 0x7f264c0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 -- 172.21.15.17:0/3894588483 >> 172.21.15.17:0/3894588483 conn(0x7f264c0aa630 msgr2=0x7f264c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 -- 172.21.15.17:0/3894588483 shutdown_connections 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.648+0000 7f2651b93640 1 -- 172.21.15.17:0/3894588483 wait complete. 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.649+0000 7f2651b93640 1 Processor -- start 2024-03-06T07:24:32.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.649+0000 7f2651b93640 1 -- start start 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.649+0000 7f2651b93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f264c0c5c60 0x7f264c0a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c073190 con 0x7f264c0c5880 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c0c6780 con 0x7f264c0c5c60 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c073310 con 0x7f264c0c7720 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57736/0 (socket says 172.21.15.17:57736) 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 -- 172.21.15.17:0/3155594323 learned_addr learned my addr 172.21.15.17:0/3155594323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:32.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264affd640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f264c0c5c60 0x7f264c0a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264bfff640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 -- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 msgr2=0x7f264c14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 -- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f264c0c5c60 msgr2=0x7f264c0a9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f264c0c5c60 0x7f264c0a9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 -- 172.21.15.17:0/3155594323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c14cd10 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264bfff640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f264b7fe640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d5f0 tx=0x7f263c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0125f0 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c019070 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c1ebca0 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0129d0 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.650+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c0aa0d0 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.651+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264c1eba90 con 0x7f264c0c5880 2024-03-06T07:24:32.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.652+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f263c012b70 con 0x7f264c0c5880 2024-03-06T07:24:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.652+0000 7f2633fff640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 0x7f263806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.652+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f263c08a710 con 0x7f264c0c5880 2024-03-06T07:24:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.652+0000 7f264affd640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 0x7f263806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:32.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.653+0000 7f264affd640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 0x7f263806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2640035000 tx=0x7f2640008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:32.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.657+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f263c059490 con 0x7f264c0c5880 2024-03-06T07:24:32.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.879+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f264c04f330 con 0x7f264c0c5880 2024-03-06T07:24:32.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.887+0000 7f2633fff640 1 -- 172.21.15.17:0/3155594323 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7f263c058d90 con 0x7f264c0c5880 2024-03-06T07:24:32.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 msgr2=0x7f263806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 0x7f263806d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2640035000 tx=0x7f2640008030 comp rx=0 tx=0).stop 2024-03-06T07:24:32.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 msgr2=0x7f264c1e8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:32.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d5f0 tx=0x7f263c00dab0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 shutdown_connections 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f263806b4d0 0x7f263806d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f264c0c7720 0x7f264c14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f264c0c5c60 0x7f264c0a9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 --2- 172.21.15.17:0/3155594323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f264c0c5880 0x7f264c1e8f40 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.889+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 >> 172.21.15.17:0/3155594323 conn(0x7f264c0aa630 msgr2=0x7f264c0c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.890+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 shutdown_connections 2024-03-06T07:24:32.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:32.890+0000 7f2651b93640 1 -- 172.21.15.17:0/3155594323 wait complete. 2024-03-06T07:24:32.904 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:24:32.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:24:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.008+0000 7f90e73f7640 1 Processor -- start 2024-03-06T07:24:33.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.008+0000 7f90e73f7640 1 -- start start 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e73f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e73f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90e00c69a0 0x7f90e0072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e73f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e00c62a0 con 0x7f90e00c4740 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e73f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e0153f00 con 0x7f90e00c69a0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e73f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c4c00 con 0x7f90e00a8290 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5994640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90e00c69a0 0x7f90e0072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:57916/0 (socket says 172.21.15.17:57916) 2024-03-06T07:24:33.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 -- 172.21.15.17:0/1337722847 learned_addr learned my addr 172.21.15.17:0/1337722847 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e4992640 1 --1- 172.21.15.17:0/1337722847 >> v1:172.21.15.163:6789/0 conn(0x7f90e00c4740 0x7f90e00701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40564/0 (socket says 172.21.15.17:40564) 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90d7fff640 1 -- 172.21.15.17:0/1337722847 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635800395 0 0) 0x7f90e00c62a0 con 0x7f90e00c4740 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 -- 172.21.15.17:0/1337722847 >> v2:172.21.15.163:3300/0 conn(0x7f90e00c69a0 msgr2=0x7f90e0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 --2- 172.21.15.17:0/1337722847 >> v2:172.21.15.163:3300/0 conn(0x7f90e00c69a0 0x7f90e0072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 -- 172.21.15.17:0/1337722847 >> v1:172.21.15.163:6789/0 conn(0x7f90e00c4740 legacy=0x7f90e00701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.009+0000 7f90e5193640 1 -- 172.21.15.17:0/1337722847 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00c4ee0 con 0x7f90e00a8290 2024-03-06T07:24:33.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.010+0000 7f90e5193640 1 --2- 172.21.15.17:0/1337722847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009870 tx=0x7f90dc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a996bc832811e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.010+0000 7f90d7fff640 1 -- 172.21.15.17:0/1337722847 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0317c0 con 0x7f90e00a8290 2024-03-06T07:24:33.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.010+0000 7f90d7fff640 1 -- 172.21.15.17:0/1337722847 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc037070 con 0x7f90e00a8290 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.010+0000 7f90d7fff640 1 -- 172.21.15.17:0/1337722847 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc031b80 con 0x7f90e00a8290 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 -- 172.21.15.17:0/1337722847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 msgr2=0x7f90e01539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1337722847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009870 tx=0x7f90dc02b860 comp rx=0 tx=0).stop 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 -- 172.21.15.17:0/1337722847 shutdown_connections 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1337722847 >> v2:172.21.15.163:3300/0 conn(0x7f90e00c69a0 0x7f90e0072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1337722847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e01539c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 -- 172.21.15.17:0/1337722847 >> 172.21.15.17:0/1337722847 conn(0x7f90e00c5560 msgr2=0x7f90e00c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 -- 172.21.15.17:0/1337722847 shutdown_connections 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.011+0000 7f90e73f7640 1 -- 172.21.15.17:0/1337722847 wait complete. 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.012+0000 7f90e73f7640 1 Processor -- start 2024-03-06T07:24:33.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.012+0000 7f90e73f7640 1 -- start start 2024-03-06T07:24:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e0156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90e00c4740 0x7f90e01ec080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c6e50 con 0x7f90e00c69a0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e006fc80 con 0x7f90e00c4740 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e73f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0072bc0 con 0x7f90e00a8290 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57752/0 (socket says 172.21.15.17:57752) 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 -- 172.21.15.17:0/1177414118 learned_addr learned my addr 172.21.15.17:0/1177414118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5193640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e0156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e4992640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90e00c4740 0x7f90e01ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 -- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 msgr2=0x7f90e0156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e0156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 -- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90e00c4740 msgr2=0x7f90e01ec080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90e00c4740 0x7f90e01ec080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 -- 172.21.15.17:0/1177414118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e014ba70 con 0x7f90e00c69a0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.013+0000 7f90e5994640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009d30 tx=0x7f90d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.014+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8018020 con 0x7f90e00c69a0 2024-03-06T07:24:33.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.014+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d8014480 con 0x7f90e00c69a0 2024-03-06T07:24:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.014+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e014d9d0 con 0x7f90e00c69a0 2024-03-06T07:24:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.014+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d8014870 con 0x7f90e00c69a0 2024-03-06T07:24:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.014+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e00ae710 con 0x7f90e00c69a0 2024-03-06T07:24:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.015+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0067fd0 con 0x7f90e00c69a0 2024-03-06T07:24:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.015+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90d8005d10 con 0x7f90e00c69a0 2024-03-06T07:24:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.016+0000 7f90d5ffb640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.016+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90d808aba0 con 0x7f90e00c69a0 2024-03-06T07:24:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.016+0000 7f90e5193640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.016+0000 7f90e5193640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 0x7f90c406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90dc036000 tx=0x7f90dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.020+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90e0067fd0 con 0x7f90e00c69a0 2024-03-06T07:24:33.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.239+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f90e014b3b0 con 0x7f90e00c69a0 2024-03-06T07:24:33.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.895+0000 7f90d5ffb640 1 -- 172.21.15.17:0/1177414118 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7f90d8022070 con 0x7f90e00c69a0 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 msgr2=0x7f90c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 0x7f90c406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f90dc036000 tx=0x7f90dc008030 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 msgr2=0x7f90e01ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009d30 tx=0x7f90d800c490 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 shutdown_connections 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c406b4d0 0x7f90c406d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90e00c69a0 0x7f90e01ec5c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90e00c4740 0x7f90e01ec080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 --2- 172.21.15.17:0/1177414118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90e00a8290 0x7f90e0156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 >> 172.21.15.17:0/1177414118 conn(0x7f90e00c5560 msgr2=0x7f90e00c3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 shutdown_connections 2024-03-06T07:24:33.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:33.898+0000 7f90e73f7640 1 -- 172.21.15.17:0/1177414118 wait complete. 2024-03-06T07:24:33.913 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:24:33.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:34.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 Processor -- start 2024-03-06T07:24:34.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 -- start start 2024-03-06T07:24:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83d80c88c0 0x7f83d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d80c73d0 con 0x7f83d8158b60 2024-03-06T07:24:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d80be2e0 con 0x7f83d814f360 2024-03-06T07:24:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83de440640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d8159700 con 0x7f83d80c88c0 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83dc9dd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f83d8158b60 0x7f83d8158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44560/0 (socket says 172.21.15.17:44560) 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.024+0000 7f83dc9dd640 1 -- 172.21.15.17:0/2495695665 learned_addr learned my addr 172.21.15.17:0/2495695665 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d77fe640 1 --2- 172.21.15.17:0/2495695665 >> v2:172.21.15.163:3300/0 conn(0x7f83d80c88c0 0x7f83d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228154490 0 0) 0x7f83d80c73d0 con 0x7f83d8158b60 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003520 con 0x7f83d8158b60 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874323867 0 0) 0x7f83c0003520 con 0x7f83d8158b60 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 >> v2:172.21.15.163:3300/0 conn(0x7f83d80c88c0 msgr2=0x7f83d8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 --2- 172.21.15.17:0/2495695665 >> v2:172.21.15.163:3300/0 conn(0x7f83d80c88c0 0x7f83d8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 >> v1:172.21.15.163:6789/0 conn(0x7f83d814f360 legacy=0x7f83d80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80c7670 con 0x7f83d8158b60 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f83d00019f0 con 0x7f83d8158b60 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 >> v1:172.21.15.17:6789/0 conn(0x7f83d8158b60 legacy=0x7f83d8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8159700 con 0x7f83c00051c0 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c160 con 0x7f83c0004800 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c360 con 0x7f83c00039b0 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83dc9dd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d77fe640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.025+0000 7f83d7fff640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 -- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83c00039b0 msgr2=0x7f83c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 -- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83c0004800 msgr2=0x7f83c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 -- 172.21.15.17:0/2495695665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0003520 con 0x7f83c00051c0 2024-03-06T07:24:34.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 -- 172.21.15.17:0/2495695665 reap_dead start 2024-03-06T07:24:34.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83dc9dd640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83c000b9c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f83d000a850 tx=0x7f83c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25dd31c32554a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.026+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000a4a0 con 0x7f83c00051c0 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d0034650 con 0x7f83c00051c0 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83d6ffd640 1 -- 172.21.15.17:0/2495695665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0034a40 con 0x7f83c00051c0 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 -- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 msgr2=0x7f83c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83c000b9c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f83d000a850 tx=0x7f83c0005830 comp rx=0 tx=0).stop 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 -- 172.21.15.17:0/2495695665 shutdown_connections 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 --2- 172.21.15.17:0/2495695665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 -- 172.21.15.17:0/2495695665 >> 172.21.15.17:0/2495695665 conn(0x7f83d80ac570 msgr2=0x7f83d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 -- 172.21.15.17:0/2495695665 shutdown_connections 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.027+0000 7f83de440640 1 -- 172.21.15.17:0/2495695665 wait complete. 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.028+0000 7f83de440640 1 Processor -- start 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.028+0000 7f83de440640 1 -- start start 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d80ab6a0 0x7f83d80af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d80afb10 0x7f83d814abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d80c7c30 con 0x7f83c00051c0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8155c10 con 0x7f83d80ab6a0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83de440640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d8155d90 con 0x7f83d80afb10 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57770/0 (socket says 172.21.15.17:57770) 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 -- 172.21.15.17:0/1592174798 learned_addr learned my addr 172.21.15.17:0/1592174798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d77fe640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d80ab6a0 0x7f83d80af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 -- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d80afb10 msgr2=0x7f83d814abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d80afb10 0x7f83d814abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 -- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d80ab6a0 msgr2=0x7f83d80af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d80ab6a0 0x7f83d80af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.029+0000 7f83d7fff640 1 -- 172.21.15.17:0/1592174798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d80c1520 con 0x7f83c00051c0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83d7fff640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f83c800f650 tx=0x7f83c800fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80050d0 con 0x7f83c00051c0 2024-03-06T07:24:34.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c8018070 con 0x7f83c00051c0 2024-03-06T07:24:34.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d80abeb0 con 0x7f83c00051c0 2024-03-06T07:24:34.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8005490 con 0x7f83c00051c0 2024-03-06T07:24:34.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.030+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d80c1240 con 0x7f83c00051c0 2024-03-06T07:24:34.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.031+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d804f330 con 0x7f83c00051c0 2024-03-06T07:24:34.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.031+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83c800d730 con 0x7f83c00051c0 2024-03-06T07:24:34.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.032+0000 7f83d4ff9640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.032+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83c808a130 con 0x7f83c00051c0 2024-03-06T07:24:34.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.032+0000 7f83d77fe640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.032+0000 7f83d77fe640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 0x7f83c406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83d814c020 tx=0x7f83cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.036+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83c8058e30 con 0x7f83c00051c0 2024-03-06T07:24:34.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.254+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83d80c8d50 con 0x7f83c00051c0 2024-03-06T07:24:34.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.255+0000 7f83d4ff9640 1 -- 172.21.15.17:0/1592174798 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f83c8058730 con 0x7f83c00051c0 2024-03-06T07:24:34.256 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:34.256 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:33.240915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/410228237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":410228237},{"type":"v1","addr":"172.21.15.17:6835","nonce":410228237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":9}]} 2024-03-06T07:24:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 105 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 msgr2=0x7f83c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 0x7f83c406d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f83d814c020 tx=0x7f83cc008040 comp rx=0 tx=0).stop 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 msgr2=0x7f83d81ee6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f83c800f650 tx=0x7f83c800fb10 comp rx=0 tx=0).stop 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 shutdown_connections 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f83c406b4d0 0x7f83c406d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d80afb10 0x7f83d814abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d80ab6a0 0x7f83d80af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 --2- 172.21.15.17:0/1592174798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83c00051c0 0x7f83d81ee6e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 >> 172.21.15.17:0/1592174798 conn(0x7f83d80ac570 msgr2=0x7f83d80c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 shutdown_connections 2024-03-06T07:24:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.256+0000 7f83de440640 1 -- 172.21.15.17:0/1592174798 wait complete. 2024-03-06T07:24:34.269 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:34.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 Processor -- start 2024-03-06T07:24:34.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 -- start start 2024-03-06T07:24:34.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848406afc0 0x7f848406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f848406b3a0 0x7f8484152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8484153940 con 0x7f84841534f0 2024-03-06T07:24:34.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848406b830 con 0x7f848406afc0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f848978a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8484150a20 con 0x7f848406b3a0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f8482ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848406afc0 0x7f848406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f8482ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f848406afc0 0x7f848406f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57776/0 (socket says 172.21.15.17:57776) 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.369+0000 7f8482ffd640 1 -- 172.21.15.17:0/224677432 learned_addr learned my addr 172.21.15.17:0/224677432 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f847bfff640 1 --2- 172.21.15.17:0/224677432 >> v2:172.21.15.163:3300/0 conn(0x7f848406b3a0 0x7f8484152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f8482ffd640 1 -- 172.21.15.17:0/224677432 >> v2:172.21.15.163:3300/0 conn(0x7f848406b3a0 msgr2=0x7f8484152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f8482ffd640 1 --2- 172.21.15.17:0/224677432 >> v2:172.21.15.163:3300/0 conn(0x7f848406b3a0 0x7f8484152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f8482ffd640 1 -- 172.21.15.17:0/224677432 >> v1:172.21.15.163:6789/0 conn(0x7f84841534f0 legacy=0x7f84840c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f8482ffd640 1 -- 172.21.15.17:0/224677432 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840a70f0 con 0x7f848406afc0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.370+0000 7f8482ffd640 1 --2- 172.21.15.17:0/224677432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406afc0 0x7f848406f100 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8474009870 tx=0x7f847402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c9d01f5dd95f47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f84827fc640 1 -- 172.21.15.17:0/224677432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474038070 con 0x7f848406afc0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f84827fc640 1 -- 172.21.15.17:0/224677432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847402f050 con 0x7f848406afc0 2024-03-06T07:24:34.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f84827fc640 1 -- 172.21.15.17:0/224677432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474031970 con 0x7f848406afc0 2024-03-06T07:24:34.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 -- 172.21.15.17:0/224677432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406afc0 msgr2=0x7f848406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 --2- 172.21.15.17:0/224677432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406afc0 0x7f848406f100 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8474009870 tx=0x7f847402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 -- 172.21.15.17:0/224677432 shutdown_connections 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 --2- 172.21.15.17:0/224677432 >> v2:172.21.15.163:3300/0 conn(0x7f848406b3a0 0x7f8484152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 --2- 172.21.15.17:0/224677432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406afc0 0x7f848406f100 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.371+0000 7f848978a640 1 -- 172.21.15.17:0/224677432 >> 172.21.15.17:0/224677432 conn(0x7f84840c4290 msgr2=0x7f84840c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.372+0000 7f848978a640 1 -- 172.21.15.17:0/224677432 shutdown_connections 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.372+0000 7f848978a640 1 -- 172.21.15.17:0/224677432 wait complete. 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 Processor -- start 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 -- start start 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848406afc0 0x7f8484156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84841534f0 0x7f848414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484156ad0 con 0x7f848406b3a0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84841565a0 con 0x7f848406afc0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f848978a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8484156db0 con 0x7f84841534f0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f847bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f847bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57788/0 (socket says 172.21.15.17:57788) 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.373+0000 7f847bfff640 1 -- 172.21.15.17:0/3378944149 learned_addr learned my addr 172.21.15.17:0/3378944149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f84837fe640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84841534f0 0x7f848414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f8482ffd640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848406afc0 0x7f8484156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 -- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84841534f0 msgr2=0x7f848414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84841534f0 0x7f848414e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 -- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848406afc0 msgr2=0x7f8484156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848406afc0 0x7f8484156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 -- 172.21.15.17:0/3378944149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84841eb850 con 0x7f848406b3a0 2024-03-06T07:24:34.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847bfff640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c7f0 tx=0x7f846c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c015070 con 0x7f848406b3a0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c004220 con 0x7f848406b3a0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84840732b0 con 0x7f848406b3a0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c007620 con 0x7f848406b3a0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.374+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8484071540 con 0x7f848406b3a0 2024-03-06T07:24:34.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.375+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84841ebd30 con 0x7f848406b3a0 2024-03-06T07:24:34.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.376+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f846c0079d0 con 0x7f848406b3a0 2024-03-06T07:24:34.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.376+0000 7f847b7fe640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.376+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f846c08a6d0 con 0x7f848406b3a0 2024-03-06T07:24:34.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.376+0000 7f8482ffd640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.377+0000 7f8482ffd640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8474035000 tx=0x7f8474008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.380+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f846c059530 con 0x7f848406b3a0 2024-03-06T07:24:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.562+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f848404f330 con 0x7f848406b3a0 2024-03-06T07:24:34.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.564+0000 7f847b7fe640 1 -- 172.21.15.17:0/3378944149 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f846c058e30 con 0x7f848406b3a0 2024-03-06T07:24:34.566 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:34.566 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":122,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:32.873958+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 msgr2=0x7f846406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8474035000 tx=0x7f8474008030 comp rx=0 tx=0).stop 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 msgr2=0x7f84840c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c7f0 tx=0x7f846c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 shutdown_connections 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84841534f0 0x7f848414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406b3a0 0x7f84840c0e30 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 --2- 172.21.15.17:0/3378944149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f848406afc0 0x7f8484156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.566+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 >> 172.21.15.17:0/3378944149 conn(0x7f84840c4290 msgr2=0x7f84840a8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.567+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 shutdown_connections 2024-03-06T07:24:34.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.567+0000 7f848978a640 1 -- 172.21.15.17:0/3378944149 wait complete. 2024-03-06T07:24:34.591 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:24:34.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.697+0000 7fc58281c640 1 Processor -- start 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.697+0000 7fc58281c640 1 -- start start 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.697+0000 7fc58281c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc57c153990 0x7fc57c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.697+0000 7fc58281c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57c070aa0 con 0x7fc57c1506b0 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.697+0000 7fc58281c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57c070d80 con 0x7fc57c0acc00 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc58281c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c156450 con 0x7fc57c153990 2024-03-06T07:24:34.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc57c1506b0 0x7fc57c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44568/0 (socket says 172.21.15.17:44568) 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57bfff640 1 -- 172.21.15.17:0/1943596576 learned_addr learned my addr 172.21.15.17:0/1943596576 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57b7fe640 1 --2- 172.21.15.17:0/1943596576 >> v2:172.21.15.163:3300/0 conn(0x7fc57c153990 0x7fc57c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958642374 0 0) 0x7fc57c070aa0 con 0x7fc57c1506b0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc564003520 con 0x7fc57c1506b0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3398012170 0 0) 0x7fc564003520 con 0x7fc57c1506b0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 >> v2:172.21.15.163:3300/0 conn(0x7fc57c153990 msgr2=0x7fc57c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 --2- 172.21.15.17:0/1943596576 >> v2:172.21.15.163:3300/0 conn(0x7fc57c153990 0x7fc57c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 >> v1:172.21.15.163:6789/0 conn(0x7fc57c0acc00 legacy=0x7fc57c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c1566a0 con 0x7fc57c1506b0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc56c001670 con 0x7fc57c1506b0 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 >> v1:172.21.15.17:6789/0 conn(0x7fc57c1506b0 legacy=0x7fc57c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 0x7fc564003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5640043b0 0x7fc564004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564005080 0x7fc56400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c156450 con 0x7fc5640039b0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56400bfe0 con 0x7fc564005080 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56400c1e0 con 0x7fc5640043b0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.698+0000 7fc57bfff640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 0x7fc564003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc580db9640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564005080 0x7fc56400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57b7fe640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5640043b0 0x7fc564004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 -- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5640043b0 msgr2=0x7fc564004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5640043b0 0x7fc564004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 -- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564005080 msgr2=0x7fc56400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc564005080 0x7fc56400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 -- 172.21.15.17:0/1943596576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564003520 con 0x7fc5640039b0 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc580db9640 1 -- 172.21.15.17:0/1943596576 reap_dead start 2024-03-06T07:24:34.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57b7fe640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5640043b0 0x7fc564004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.699+0000 7fc57bfff640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 0x7fc564003d90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c240 tx=0x7fc564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f0b1915673926d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c007d10 con 0x7fc5640039b0 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c030070 con 0x7fc5640039b0 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc57affd640 1 -- 172.21.15.17:0/1943596576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c02cda0 con 0x7fc5640039b0 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 -- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 msgr2=0x7fc564003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 0x7fc564003d90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00c240 tx=0x7fc564005830 comp rx=0 tx=0).stop 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 -- 172.21.15.17:0/1943596576 shutdown_connections 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 --2- 172.21.15.17:0/1943596576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5640039b0 0x7fc564003d90 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 -- 172.21.15.17:0/1943596576 >> 172.21.15.17:0/1943596576 conn(0x7fc57c06b1a0 msgr2=0x7fc57c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 -- 172.21.15.17:0/1943596576 shutdown_connections 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.700+0000 7fc58281c640 1 -- 172.21.15.17:0/1943596576 wait complete. 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.701+0000 7fc58281c640 1 Processor -- start 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 -- start start 2024-03-06T07:24:34.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 0x7fc57c1eb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 0x7fc57c0c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c156a10 con 0x7fc57c06c0d0 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0ad210 con 0x7fc5640039b0 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc58281c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c156ea0 con 0x7fc57c0c8130 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc580db9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc580db9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57792/0 (socket says 172.21.15.17:57792) 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc580db9640 1 -- 172.21.15.17:0/3777053134 learned_addr learned my addr 172.21.15.17:0/3777053134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc57bfff640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 0x7fc57c1eb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.702+0000 7fc57b7fe640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 0x7fc57c0c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 -- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 msgr2=0x7fc57c0c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 0x7fc57c0c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 -- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 msgr2=0x7fc57c1eb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 0x7fc57c1eb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 -- 172.21.15.17:0/3777053134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c1eac80 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc57b7fe640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 0x7fc57c0c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc57bfff640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 0x7fc57c1eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc580db9640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0c6270 tx=0x7fc57400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57401a070 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc574006d60 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c156040 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574003bb0 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.703+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c0c22a0 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.704+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c070d80 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.705+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc57401d070 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.705+0000 7fc578ff9640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 0x7fc56806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.705+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc574089930 con 0x7fc57c06c0d0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.705+0000 7fc57bfff640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 0x7fc56806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:34.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.706+0000 7fc57bfff640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 0x7fc56806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc56c034000 tx=0x7fc56c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:34.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.710+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc57c070d80 con 0x7fc57c06c0d0 2024-03-06T07:24:34.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.919+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc57c04f330 con 0x7fc56806b500 2024-03-06T07:24:34.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.926+0000 7fc578ff9640 1 -- 172.21.15.17:0/3777053134 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7fc56c083a60 con 0x7fc56806b500 2024-03-06T07:24:34.930 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:34.932 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:24:34.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 msgr2=0x7fc56806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 0x7fc56806d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc56c034000 tx=0x7fc56c007000 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 msgr2=0x7fc57c0c6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0c6270 tx=0x7fc57400fd40 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 shutdown_connections 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc56806b500 0x7fc56806d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc57c06c0d0 0x7fc57c0c6520 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc57c0c8130 0x7fc57c0c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 --2- 172.21.15.17:0/3777053134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5640039b0 0x7fc57c1eb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 >> 172.21.15.17:0/3777053134 conn(0x7fc57c06b1a0 msgr2=0x7fc57c0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 shutdown_connections 2024-03-06T07:24:34.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:34.929+0000 7fc58281c640 1 -- 172.21.15.17:0/3777053134 wait complete. 2024-03-06T07:24:34.945 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-03-06T07:24:33.400720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261392,"kb_used_data":11908,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713328,"statfs":{"total":773094113280,"available":772826447872,"internally_reserved":0,"allocated":12193792,"data_stored":6996380,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001111"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898318+0000","last_change":"2024-03-06T07:24:31.893689+0000","last_active":"2024-03-06T07:24:31.898318+0000","last_peered":"2024-03-06T07:24:31.898318+0000","last_clean":"2024-03-06T07:24:31.898318+0000","last_became_active":"2024-03-06T07:24:31.892752+0000","last_became_peered":"2024-03-06T07:24:31.892752+0000","last_unstale":"2024-03-06T07:24:31.898318+0000","last_undegraded":"2024-03-06T07:24:31.898318+0000","last_fullsized":"2024-03-06T07:24:31.898318+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:18.884670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895780+0000","last_change":"2024-03-06T07:24:31.886106+0000","last_active":"2024-03-06T07:24:31.895780+0000","last_peered":"2024-03-06T07:24:31.895780+0000","last_clean":"2024-03-06T07:24:31.895780+0000","last_became_active":"2024-03-06T07:24:31.885804+0000","last_became_peered":"2024-03-06T07:24:31.885804+0000","last_unstale":"2024-03-06T07:24:31.895780+0000","last_undegraded":"2024-03-06T07:24:31.895780+0000","last_fullsized":"2024-03-06T07:24:31.895780+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895589+0000","last_change":"2024-03-06T07:24:31.893097+0000","last_active":"2024-03-06T07:24:31.895589+0000","last_peered":"2024-03-06T07:24:31.895589+0000","last_clean":"2024-03-06T07:24:31.895589+0000","last_became_active":"2024-03-06T07:24:31.892812+0000","last_became_peered":"2024-03-06T07:24:31.892812+0000","last_unstale":"2024-03-06T07:24:31.895589+0000","last_undegraded":"2024-03-06T07:24:31.895589+0000","last_fullsized":"2024-03-06T07:24:31.895589+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:06.484188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897149+0000","last_change":"2024-03-06T07:24:31.890231+0000","last_active":"2024-03-06T07:24:31.897149+0000","last_peered":"2024-03-06T07:24:31.897149+0000","last_clean":"2024-03-06T07:24:31.897149+0000","last_became_active":"2024-03-06T07:24:31.890006+0000","last_became_peered":"2024-03-06T07:24:31.890006+0000","last_unstale":"2024-03-06T07:24:31.897149+0000","last_undegraded":"2024-03-06T07:24:31.897149+0000","last_fullsized":"2024-03-06T07:24:31.897149+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897731+0000","last_change":"2024-03-06T07:24:31.896446+0000","last_active":"2024-03-06T07:24:31.897731+0000","last_peered":"2024-03-06T07:24:31.897731+0000","last_clean":"2024-03-06T07:24:31.897731+0000","last_became_active":"2024-03-06T07:24:31.896174+0000","last_became_peered":"2024-03-06T07:24:31.896174+0000","last_unstale":"2024-03-06T07:24:31.897731+0000","last_undegraded":"2024-03-06T07:24:31.897731+0000","last_fullsized":"2024-03-06T07:24:31.897731+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:55.787579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904724+0000","last_change":"2024-03-06T07:24:31.896903+0000","last_active":"2024-03-06T07:24:31.904724+0000","last_peered":"2024-03-06T07:24:31.904724+0000","last_clean":"2024-03-06T07:24:31.904724+0000","last_became_active":"2024-03-06T07:24:31.896613+0000","last_became_peered":"2024-03-06T07:24:31.896613+0000","last_unstale":"2024-03-06T07:24:31.904724+0000","last_undegraded":"2024-03-06T07:24:31.904724+0000","last_fullsized":"2024-03-06T07:24:31.904724+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:27.997609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899946+0000","last_change":"2024-03-06T07:24:31.897380+0000","last_active":"2024-03-06T07:24:31.899946+0000","last_peered":"2024-03-06T07:24:31.899946+0000","last_clean":"2024-03-06T07:24:31.899946+0000","last_became_active":"2024-03-06T07:24:31.897136+0000","last_became_peered":"2024-03-06T07:24:31.897136+0000","last_unstale":"2024-03-06T07:24:31.899946+0000","last_undegraded":"2024-03-06T07:24:31.899946+0000","last_fullsized":"2024-03-06T07:24:31.899946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:34.962764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887484+0000","last_change":"2024-03-06T07:24:31.884139+0000","last_active":"2024-03-06T07:24:31.887484+0000","last_peered":"2024-03-06T07:24:31.887484+0000","last_clean":"2024-03-06T07:24:31.887484+0000","last_became_active":"2024-03-06T07:24:31.883827+0000","last_became_peered":"2024-03-06T07:24:31.883827+0000","last_unstale":"2024-03-06T07:24:31.887484+0000","last_undegraded":"2024-03-06T07:24:31.887484+0000","last_fullsized":"2024-03-06T07:24:31.887484+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:01.486412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904840+0000","last_change":"2024-03-06T07:24:31.897009+0000","last_active":"2024-03-06T07:24:31.904840+0000","last_peered":"2024-03-06T07:24:31.904840+0000","last_clean":"2024-03-06T07:24:31.904840+0000","last_became_active":"2024-03-06T07:24:31.891217+0000","last_became_peered":"2024-03-06T07:24:31.891217+0000","last_unstale":"2024-03-06T07:24:31.904840+0000","last_undegraded":"2024-03-06T07:24:31.904840+0000","last_fullsized":"2024-03-06T07:24:31.904840+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:31.649044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887851+0000","last_change":"2024-03-06T07:24:31.883659+0000","last_active":"2024-03-06T07:24:31.887851+0000","last_peered":"2024-03-06T07:24:31.887851+0000","last_clean":"2024-03-06T07:24:31.887851+0000","last_became_active":"2024-03-06T07:24:31.883333+0000","last_became_peered":"2024-03-06T07:24:31.883333+0000","last_unstale":"2024-03-06T07:24:31.887851+0000","last_undegraded":"2024-03-06T07:24:31.887851+0000","last_fullsized":"2024-03-06T07:24:31.887851+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:54.014228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897092+0000","last_change":"2024-03-06T07:24:31.893748+0000","last_active":"2024-03-06T07:24:31.897092+0000","last_peered":"2024-03-06T07:24:31.897092+0000","last_clean":"2024-03-06T07:24:31.897092+0000","last_became_active":"2024-03-06T07:24:31.893382+0000","last_became_peered":"2024-03-06T07:24:31.893382+0000","last_unstale":"2024-03-06T07:24:31.897092+0000","last_undegraded":"2024-03-06T07:24:31.897092+0000","last_fullsized":"2024-03-06T07:24:31.897092+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:29.711780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887500+0000","last_change":"2024-03-06T07:24:31.885623+0000","last_active":"2024-03-06T07:24:31.887500+0000","last_peered":"2024-03-06T07:24:31.887500+0000","last_clean":"2024-03-06T07:24:31.887500+0000","last_became_active":"2024-03-06T07:24:31.885357+0000","last_became_peered":"2024-03-06T07:24:31.885357+0000","last_unstale":"2024-03-06T07:24:31.887500+0000","last_undegraded":"2024-03-06T07:24:31.887500+0000","last_fullsized":"2024-03-06T07:24:31.887500+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:06.298928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904888+0000","last_change":"2024-03-06T07:24:31.896445+0000","last_active":"2024-03-06T07:24:31.904888+0000","last_peered":"2024-03-06T07:24:31.904888+0000","last_clean":"2024-03-06T07:24:31.904888+0000","last_became_active":"2024-03-06T07:24:31.896082+0000","last_became_peered":"2024-03-06T07:24:31.896082+0000","last_unstale":"2024-03-06T07:24:31.904888+0000","last_undegraded":"2024-03-06T07:24:31.904888+0000","last_fullsized":"2024-03-06T07:24:31.904888+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.400287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900625+0000","last_change":"2024-03-06T07:24:31.897231+0000","last_active":"2024-03-06T07:24:31.900625+0000","last_peered":"2024-03-06T07:24:31.900625+0000","last_clean":"2024-03-06T07:24:31.900625+0000","last_became_active":"2024-03-06T07:24:31.896897+0000","last_became_peered":"2024-03-06T07:24:31.896897+0000","last_unstale":"2024-03-06T07:24:31.900625+0000","last_undegraded":"2024-03-06T07:24:31.900625+0000","last_fullsized":"2024-03-06T07:24:31.900625+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:50.767038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898953+0000","last_change":"2024-03-06T07:24:31.895044+0000","last_active":"2024-03-06T07:24:31.898953+0000","last_peered":"2024-03-06T07:24:31.898953+0000","last_clean":"2024-03-06T07:24:31.898953+0000","last_became_active":"2024-03-06T07:24:31.894762+0000","last_became_peered":"2024-03-06T07:24:31.894762+0000","last_unstale":"2024-03-06T07:24:31.898953+0000","last_undegraded":"2024-03-06T07:24:31.898953+0000","last_fullsized":"2024-03-06T07:24:31.898953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:40.179739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907226+0000","last_change":"2024-03-06T07:24:31.898603+0000","last_active":"2024-03-06T07:24:31.907226+0000","last_peered":"2024-03-06T07:24:31.907226+0000","last_clean":"2024-03-06T07:24:31.907226+0000","last_became_active":"2024-03-06T07:24:31.898283+0000","last_became_peered":"2024-03-06T07:24:31.898283+0000","last_unstale":"2024-03-06T07:24:31.907226+0000","last_undegraded":"2024-03-06T07:24:31.907226+0000","last_fullsized":"2024-03-06T07:24:31.907226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:50.642673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904815+0000","last_change":"2024-03-06T07:24:31.894775+0000","last_active":"2024-03-06T07:24:31.904815+0000","last_peered":"2024-03-06T07:24:31.904815+0000","last_clean":"2024-03-06T07:24:31.904815+0000","last_became_active":"2024-03-06T07:24:31.894503+0000","last_became_peered":"2024-03-06T07:24:31.894503+0000","last_unstale":"2024-03-06T07:24:31.904815+0000","last_undegraded":"2024-03-06T07:24:31.904815+0000","last_fullsized":"2024-03-06T07:24:31.904815+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:41.707033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900714+0000","last_change":"2024-03-06T07:24:31.898333+0000","last_active":"2024-03-06T07:24:31.900714+0000","last_peered":"2024-03-06T07:24:31.900714+0000","last_clean":"2024-03-06T07:24:31.900714+0000","last_became_active":"2024-03-06T07:24:31.898046+0000","last_became_peered":"2024-03-06T07:24:31.898046+0000","last_unstale":"2024-03-06T07:24:31.900714+0000","last_undegraded":"2024-03-06T07:24:31.900714+0000","last_fullsized":"2024-03-06T07:24:31.900714+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:20.764174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901939+0000","last_change":"2024-03-06T07:24:31.900986+0000","last_active":"2024-03-06T07:24:31.901939+0000","last_peered":"2024-03-06T07:24:31.901939+0000","last_clean":"2024-03-06T07:24:31.901939+0000","last_became_active":"2024-03-06T07:24:31.900604+0000","last_became_peered":"2024-03-06T07:24:31.900604+0000","last_unstale":"2024-03-06T07:24:31.901939+0000","last_undegraded":"2024-03-06T07:24:31.901939+0000","last_fullsized":"2024-03-06T07:24:31.901939+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:57.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.903408+0000","last_change":"2024-03-06T07:24:31.901613+0000","last_active":"2024-03-06T07:24:31.903408+0000","last_peered":"2024-03-06T07:24:31.903408+0000","last_clean":"2024-03-06T07:24:31.903408+0000","last_became_active":"2024-03-06T07:24:31.901344+0000","last_became_peered":"2024-03-06T07:24:31.901344+0000","last_unstale":"2024-03-06T07:24:31.903408+0000","last_undegraded":"2024-03-06T07:24:31.903408+0000","last_fullsized":"2024-03-06T07:24:31.903408+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:51.389165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895899+0000","last_change":"2024-03-06T07:24:31.889184+0000","last_active":"2024-03-06T07:24:31.895899+0000","last_peered":"2024-03-06T07:24:31.895899+0000","last_clean":"2024-03-06T07:24:31.895899+0000","last_became_active":"2024-03-06T07:24:31.888837+0000","last_became_peered":"2024-03-06T07:24:31.888837+0000","last_unstale":"2024-03-06T07:24:31.895899+0000","last_undegraded":"2024-03-06T07:24:31.895899+0000","last_fullsized":"2024-03-06T07:24:31.895899+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887963+0000","last_change":"2024-03-06T07:24:31.886938+0000","last_active":"2024-03-06T07:24:31.887963+0000","last_peered":"2024-03-06T07:24:31.887963+0000","last_clean":"2024-03-06T07:24:31.887963+0000","last_became_active":"2024-03-06T07:24:31.886618+0000","last_became_peered":"2024-03-06T07:24:31.886618+0000","last_unstale":"2024-03-06T07:24:31.887963+0000","last_undegraded":"2024-03-06T07:24:31.887963+0000","last_fullsized":"2024-03-06T07:24:31.887963+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:44.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898166+0000","last_change":"2024-03-06T07:24:31.895136+0000","last_active":"2024-03-06T07:24:31.898166+0000","last_peered":"2024-03-06T07:24:31.898166+0000","last_clean":"2024-03-06T07:24:31.898166+0000","last_became_active":"2024-03-06T07:24:31.893817+0000","last_became_peered":"2024-03-06T07:24:31.893817+0000","last_unstale":"2024-03-06T07:24:31.898166+0000","last_undegraded":"2024-03-06T07:24:31.898166+0000","last_fullsized":"2024-03-06T07:24:31.898166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:41.707033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907204+0000","last_change":"2024-03-06T07:24:31.896742+0000","last_active":"2024-03-06T07:24:31.907204+0000","last_peered":"2024-03-06T07:24:31.907204+0000","last_clean":"2024-03-06T07:24:31.907204+0000","last_became_active":"2024-03-06T07:24:31.896454+0000","last_became_peered":"2024-03-06T07:24:31.896454+0000","last_unstale":"2024-03-06T07:24:31.907204+0000","last_undegraded":"2024-03-06T07:24:31.907204+0000","last_fullsized":"2024-03-06T07:24:31.907204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:58.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897995+0000","last_change":"2024-03-06T07:24:31.897006+0000","last_active":"2024-03-06T07:24:31.897995+0000","last_peered":"2024-03-06T07:24:31.897995+0000","last_clean":"2024-03-06T07:24:31.897995+0000","last_became_active":"2024-03-06T07:24:31.896728+0000","last_became_peered":"2024-03-06T07:24:31.896728+0000","last_unstale":"2024-03-06T07:24:31.897995+0000","last_undegraded":"2024-03-06T07:24:31.897995+0000","last_fullsized":"2024-03-06T07:24:31.897995+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895521+0000","last_change":"2024-03-06T07:24:31.892789+0000","last_active":"2024-03-06T07:24:31.895521+0000","last_peered":"2024-03-06T07:24:31.895521+0000","last_clean":"2024-03-06T07:24:31.895521+0000","last_became_active":"2024-03-06T07:24:31.892477+0000","last_became_peered":"2024-03-06T07:24:31.892477+0000","last_unstale":"2024-03-06T07:24:31.895521+0000","last_undegraded":"2024-03-06T07:24:31.895521+0000","last_fullsized":"2024-03-06T07:24:31.895521+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:18.006744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907656+0000","last_change":"2024-03-06T07:24:31.905233+0000","last_active":"2024-03-06T07:24:31.907656+0000","last_peered":"2024-03-06T07:24:31.907656+0000","last_clean":"2024-03-06T07:24:31.907656+0000","last_became_active":"2024-03-06T07:24:31.904956+0000","last_became_peered":"2024-03-06T07:24:31.904956+0000","last_unstale":"2024-03-06T07:24:31.907656+0000","last_undegraded":"2024-03-06T07:24:31.907656+0000","last_fullsized":"2024-03-06T07:24:31.907656+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:17.223972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.888486+0000","last_change":"2024-03-06T07:24:31.882523+0000","last_active":"2024-03-06T07:24:31.888486+0000","last_peered":"2024-03-06T07:24:31.888486+0000","last_clean":"2024-03-06T07:24:31.888486+0000","last_became_active":"2024-03-06T07:24:31.882240+0000","last_became_peered":"2024-03-06T07:24:31.882240+0000","last_unstale":"2024-03-06T07:24:31.888486+0000","last_undegraded":"2024-03-06T07:24:31.888486+0000","last_fullsized":"2024-03-06T07:24:31.888486+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:03.777899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.893347+0000","last_change":"2024-03-06T07:24:31.889970+0000","last_active":"2024-03-06T07:24:31.893347+0000","last_peered":"2024-03-06T07:24:31.893347+0000","last_clean":"2024-03-06T07:24:31.893347+0000","last_became_active":"2024-03-06T07:24:31.889662+0000","last_became_peered":"2024-03-06T07:24:31.889662+0000","last_unstale":"2024-03-06T07:24:31.893347+0000","last_undegraded":"2024-03-06T07:24:31.893347+0000","last_fullsized":"2024-03-06T07:24:31.893347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:55.060423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897216+0000","last_change":"2024-03-06T07:24:31.896135+0000","last_active":"2024-03-06T07:24:31.897216+0000","last_peered":"2024-03-06T07:24:31.897216+0000","last_clean":"2024-03-06T07:24:31.897216+0000","last_became_active":"2024-03-06T07:24:31.895880+0000","last_became_peered":"2024-03-06T07:24:31.895880+0000","last_unstale":"2024-03-06T07:24:31.897216+0000","last_undegraded":"2024-03-06T07:24:31.897216+0000","last_fullsized":"2024-03-06T07:24:31.897216+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897569+0000","last_change":"2024-03-06T07:24:31.895162+0000","last_active":"2024-03-06T07:24:31.897569+0000","last_peered":"2024-03-06T07:24:31.897569+0000","last_clean":"2024-03-06T07:24:31.897569+0000","last_became_active":"2024-03-06T07:24:31.894813+0000","last_became_peered":"2024-03-06T07:24:31.894813+0000","last_unstale":"2024-03-06T07:24:31.897569+0000","last_undegraded":"2024-03-06T07:24:31.897569+0000","last_fullsized":"2024-03-06T07:24:31.897569+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.892016+0000","last_change":"2024-03-06T07:24:31.887068+0000","last_active":"2024-03-06T07:24:31.892016+0000","last_peered":"2024-03-06T07:24:31.892016+0000","last_clean":"2024-03-06T07:24:31.892016+0000","last_became_active":"2024-03-06T07:24:31.886570+0000","last_became_peered":"2024-03-06T07:24:31.886570+0000","last_unstale":"2024-03-06T07:24:31.892016+0000","last_undegraded":"2024-03-06T07:24:31.892016+0000","last_fullsized":"2024-03-06T07:24:31.892016+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:17.026975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899922+0000","last_change":"2024-03-06T07:24:31.897006+0000","last_active":"2024-03-06T07:24:31.899922+0000","last_peered":"2024-03-06T07:24:31.899922+0000","last_clean":"2024-03-06T07:24:31.899922+0000","last_became_active":"2024-03-06T07:24:31.896753+0000","last_became_peered":"2024-03-06T07:24:31.896753+0000","last_unstale":"2024-03-06T07:24:31.899922+0000","last_undegraded":"2024-03-06T07:24:31.899922+0000","last_fullsized":"2024-03-06T07:24:31.899922+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"121'4","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909708+0000","last_change":"2024-03-06T07:24:30.881388+0000","last_active":"2024-03-06T07:24:31.909708+0000","last_peered":"2024-03-06T07:24:31.909708+0000","last_clean":"2024-03-06T07:24:31.909708+0000","last_became_active":"2024-03-06T07:24:30.881117+0000","last_became_peered":"2024-03-06T07:24:30.881117+0000","last_unstale":"2024-03-06T07:24:31.909708+0000","last_undegraded":"2024-03-06T07:24:31.909708+0000","last_fullsized":"2024-03-06T07:24:31.909708+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:51:10.031258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898375+0000","last_change":"2024-03-06T07:24:31.892778+0000","last_active":"2024-03-06T07:24:31.898375+0000","last_peered":"2024-03-06T07:24:31.898375+0000","last_clean":"2024-03-06T07:24:31.898375+0000","last_became_active":"2024-03-06T07:24:31.892436+0000","last_became_peered":"2024-03-06T07:24:31.892436+0000","last_unstale":"2024-03-06T07:24:31.898375+0000","last_undegraded":"2024-03-06T07:24:31.898375+0000","last_fullsized":"2024-03-06T07:24:31.898375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:02:25.338014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.908140+0000","last_change":"2024-03-06T07:24:30.884206+0000","last_active":"2024-03-06T07:24:31.908140+0000","last_peered":"2024-03-06T07:24:31.908140+0000","last_clean":"2024-03-06T07:24:31.908140+0000","last_became_active":"2024-03-06T07:24:30.883762+0000","last_became_peered":"2024-03-06T07:24:30.883762+0000","last_unstale":"2024-03-06T07:24:31.908140+0000","last_undegraded":"2024-03-06T07:24:31.908140+0000","last_fullsized":"2024-03-06T07:24:31.908140+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:01:56.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898146+0000","last_change":"2024-03-06T07:24:31.893918+0000","last_active":"2024-03-06T07:24:31.898146+0000","last_peered":"2024-03-06T07:24:31.898146+0000","last_clean":"2024-03-06T07:24:31.898146+0000","last_became_active":"2024-03-06T07:24:31.893150+0000","last_became_peered":"2024-03-06T07:24:31.893150+0000","last_unstale":"2024-03-06T07:24:31.898146+0000","last_undegraded":"2024-03-06T07:24:31.898146+0000","last_fullsized":"2024-03-06T07:24:31.898146+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.972365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909206+0000","last_change":"2024-03-06T07:24:30.882894+0000","last_active":"2024-03-06T07:24:31.909206+0000","last_peered":"2024-03-06T07:24:31.909206+0000","last_clean":"2024-03-06T07:24:31.909206+0000","last_became_active":"2024-03-06T07:24:30.882548+0000","last_became_peered":"2024-03-06T07:24:30.882548+0000","last_unstale":"2024-03-06T07:24:31.909206+0000","last_undegraded":"2024-03-06T07:24:31.909206+0000","last_fullsized":"2024-03-06T07:24:31.909206+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:57.169452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899593+0000","last_change":"2024-03-06T07:24:31.896400+0000","last_active":"2024-03-06T07:24:31.899593+0000","last_peered":"2024-03-06T07:24:31.899593+0000","last_clean":"2024-03-06T07:24:31.899593+0000","last_became_active":"2024-03-06T07:24:31.896134+0000","last_became_peered":"2024-03-06T07:24:31.896134+0000","last_unstale":"2024-03-06T07:24:31.899593+0000","last_undegraded":"2024-03-06T07:24:31.899593+0000","last_fullsized":"2024-03-06T07:24:31.899593+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:38.542770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905649+0000","last_change":"2024-03-06T07:24:30.883865+0000","last_active":"2024-03-06T07:24:31.905649+0000","last_peered":"2024-03-06T07:24:31.905649+0000","last_clean":"2024-03-06T07:24:31.905649+0000","last_became_active":"2024-03-06T07:24:30.883527+0000","last_became_peered":"2024-03-06T07:24:30.883527+0000","last_unstale":"2024-03-06T07:24:31.905649+0000","last_undegraded":"2024-03-06T07:24:31.905649+0000","last_fullsized":"2024-03-06T07:24:31.905649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:37.966731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897597+0000","last_change":"2024-03-06T07:24:31.894846+0000","last_active":"2024-03-06T07:24:31.897597+0000","last_peered":"2024-03-06T07:24:31.897597+0000","last_clean":"2024-03-06T07:24:31.897597+0000","last_became_active":"2024-03-06T07:24:31.894495+0000","last_became_peered":"2024-03-06T07:24:31.894495+0000","last_unstale":"2024-03-06T07:24:31.897597+0000","last_undegraded":"2024-03-06T07:24:31.897597+0000","last_fullsized":"2024-03-06T07:24:31.897597+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907689+0000","last_change":"2024-03-06T07:24:30.880595+0000","last_active":"2024-03-06T07:24:31.907689+0000","last_peered":"2024-03-06T07:24:31.907689+0000","last_clean":"2024-03-06T07:24:31.907689+0000","last_became_active":"2024-03-06T07:24:30.880307+0000","last_became_peered":"2024-03-06T07:24:30.880307+0000","last_unstale":"2024-03-06T07:24:31.907689+0000","last_undegraded":"2024-03-06T07:24:31.907689+0000","last_fullsized":"2024-03-06T07:24:31.907689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:13.960426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.896456+0000","last_change":"2024-03-06T07:24:31.885800+0000","last_active":"2024-03-06T07:24:31.896456+0000","last_peered":"2024-03-06T07:24:31.896456+0000","last_clean":"2024-03-06T07:24:31.896456+0000","last_became_active":"2024-03-06T07:24:31.885508+0000","last_became_peered":"2024-03-06T07:24:31.885508+0000","last_unstale":"2024-03-06T07:24:31.896456+0000","last_undegraded":"2024-03-06T07:24:31.896456+0000","last_fullsized":"2024-03-06T07:24:31.896456+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905270+0000","last_change":"2024-03-06T07:24:30.879944+0000","last_active":"2024-03-06T07:24:31.905270+0000","last_peered":"2024-03-06T07:24:31.905270+0000","last_clean":"2024-03-06T07:24:31.905270+0000","last_became_active":"2024-03-06T07:24:30.879411+0000","last_became_peered":"2024-03-06T07:24:30.879411+0000","last_unstale":"2024-03-06T07:24:31.905270+0000","last_undegraded":"2024-03-06T07:24:31.905270+0000","last_fullsized":"2024-03-06T07:24:31.905270+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:16.218338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900878+0000","last_change":"2024-03-06T07:24:31.899661+0000","last_active":"2024-03-06T07:24:31.900878+0000","last_peered":"2024-03-06T07:24:31.900878+0000","last_clean":"2024-03-06T07:24:31.900878+0000","last_became_active":"2024-03-06T07:24:31.899376+0000","last_became_peered":"2024-03-06T07:24:31.899376+0000","last_unstale":"2024-03-06T07:24:31.900878+0000","last_undegraded":"2024-03-06T07:24:31.900878+0000","last_fullsized":"2024-03-06T07:24:31.900878+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:15.391988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880759+0000","last_change":"2024-03-06T07:24:30.875837+0000","last_active":"2024-03-06T07:24:31.880759+0000","last_peered":"2024-03-06T07:24:31.880759+0000","last_clean":"2024-03-06T07:24:31.880759+0000","last_became_active":"2024-03-06T07:24:30.875477+0000","last_became_peered":"2024-03-06T07:24:30.875477+0000","last_unstale":"2024-03-06T07:24:31.880759+0000","last_undegraded":"2024-03-06T07:24:31.880759+0000","last_fullsized":"2024-03-06T07:24:31.880759+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901301+0000","last_change":"2024-03-06T07:24:31.899078+0000","last_active":"2024-03-06T07:24:31.901301+0000","last_peered":"2024-03-06T07:24:31.901301+0000","last_clean":"2024-03-06T07:24:31.901301+0000","last_became_active":"2024-03-06T07:24:31.898559+0000","last_became_peered":"2024-03-06T07:24:31.898559+0000","last_unstale":"2024-03-06T07:24:31.901301+0000","last_undegraded":"2024-03-06T07:24:31.901301+0000","last_fullsized":"2024-03-06T07:24:31.901301+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:48.635017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882935+0000","last_change":"2024-03-06T07:24:30.884381+0000","last_active":"2024-03-06T07:24:31.882935+0000","last_peered":"2024-03-06T07:24:31.882935+0000","last_clean":"2024-03-06T07:24:31.882935+0000","last_became_active":"2024-03-06T07:24:30.884115+0000","last_became_peered":"2024-03-06T07:24:30.884115+0000","last_unstale":"2024-03-06T07:24:31.882935+0000","last_undegraded":"2024-03-06T07:24:31.882935+0000","last_fullsized":"2024-03-06T07:24:31.882935+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:45.687281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898329+0000","last_change":"2024-03-06T07:24:31.893624+0000","last_active":"2024-03-06T07:24:31.898329+0000","last_peered":"2024-03-06T07:24:31.898329+0000","last_clean":"2024-03-06T07:24:31.898329+0000","last_became_active":"2024-03-06T07:24:31.893340+0000","last_became_peered":"2024-03-06T07:24:31.893340+0000","last_unstale":"2024-03-06T07:24:31.898329+0000","last_undegraded":"2024-03-06T07:24:31.898329+0000","last_fullsized":"2024-03-06T07:24:31.898329+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:06.761391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882490+0000","last_change":"2024-03-06T07:24:30.882235+0000","last_active":"2024-03-06T07:24:31.882490+0000","last_peered":"2024-03-06T07:24:31.882490+0000","last_clean":"2024-03-06T07:24:31.882490+0000","last_became_active":"2024-03-06T07:24:30.881955+0000","last_became_peered":"2024-03-06T07:24:30.881955+0000","last_unstale":"2024-03-06T07:24:31.882490+0000","last_undegraded":"2024-03-06T07:24:31.882490+0000","last_fullsized":"2024-03-06T07:24:31.882490+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905533+0000","last_change":"2024-03-06T07:24:31.899401+0000","last_active":"2024-03-06T07:24:31.905533+0000","last_peered":"2024-03-06T07:24:31.905533+0000","last_clean":"2024-03-06T07:24:31.905533+0000","last_became_active":"2024-03-06T07:24:31.899139+0000","last_became_peered":"2024-03-06T07:24:31.899139+0000","last_unstale":"2024-03-06T07:24:31.905533+0000","last_undegraded":"2024-03-06T07:24:31.905533+0000","last_fullsized":"2024-03-06T07:24:31.905533+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:18.216855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880168+0000","last_change":"2024-03-06T07:24:30.882954+0000","last_active":"2024-03-06T07:24:31.880168+0000","last_peered":"2024-03-06T07:24:31.880168+0000","last_clean":"2024-03-06T07:24:31.880168+0000","last_became_active":"2024-03-06T07:24:30.881699+0000","last_became_peered":"2024-03-06T07:24:30.881699+0000","last_unstale":"2024-03-06T07:24:31.880168+0000","last_undegraded":"2024-03-06T07:24:31.880168+0000","last_fullsized":"2024-03-06T07:24:31.880168+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:58.595503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900361+0000","last_change":"2024-03-06T07:24:31.897819+0000","last_active":"2024-03-06T07:24:31.900361+0000","last_peered":"2024-03-06T07:24:31.900361+0000","last_clean":"2024-03-06T07:24:31.900361+0000","last_became_active":"2024-03-06T07:24:31.897541+0000","last_became_peered":"2024-03-06T07:24:31.897541+0000","last_unstale":"2024-03-06T07:24:31.900361+0000","last_undegraded":"2024-03-06T07:24:31.900361+0000","last_fullsized":"2024-03-06T07:24:31.900361+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"121'3","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.906645+0000","last_change":"2024-03-06T07:24:30.888131+0000","last_active":"2024-03-06T07:24:31.906645+0000","last_peered":"2024-03-06T07:24:31.906645+0000","last_clean":"2024-03-06T07:24:31.906645+0000","last_became_active":"2024-03-06T07:24:30.887837+0000","last_became_peered":"2024-03-06T07:24:30.887837+0000","last_unstale":"2024-03-06T07:24:31.906645+0000","last_undegraded":"2024-03-06T07:24:31.906645+0000","last_fullsized":"2024-03-06T07:24:31.906645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:26.991975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895566+0000","last_change":"2024-03-06T07:24:31.893287+0000","last_active":"2024-03-06T07:24:31.895566+0000","last_peered":"2024-03-06T07:24:31.895566+0000","last_clean":"2024-03-06T07:24:31.895566+0000","last_became_active":"2024-03-06T07:24:31.892943+0000","last_became_peered":"2024-03-06T07:24:31.892943+0000","last_unstale":"2024-03-06T07:24:31.895566+0000","last_undegraded":"2024-03-06T07:24:31.895566+0000","last_fullsized":"2024-03-06T07:24:31.895566+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:01.486412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"121'6","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.911727+0000","last_change":"2024-03-06T07:24:30.882029+0000","last_active":"2024-03-06T07:24:31.911727+0000","last_peered":"2024-03-06T07:24:31.911727+0000","last_clean":"2024-03-06T07:24:31.911727+0000","last_became_active":"2024-03-06T07:24:30.881720+0000","last_became_peered":"2024-03-06T07:24:30.881720+0000","last_unstale":"2024-03-06T07:24:31.911727+0000","last_undegraded":"2024-03-06T07:24:31.911727+0000","last_fullsized":"2024-03-06T07:24:31.911727+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:40.701399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889200+0000","last_change":"2024-03-06T07:24:31.885898+0000","last_active":"2024-03-06T07:24:31.889200+0000","last_peered":"2024-03-06T07:24:31.889200+0000","last_clean":"2024-03-06T07:24:31.889200+0000","last_became_active":"2024-03-06T07:24:31.885585+0000","last_became_peered":"2024-03-06T07:24:31.885585+0000","last_unstale":"2024-03-06T07:24:31.889200+0000","last_undegraded":"2024-03-06T07:24:31.889200+0000","last_fullsized":"2024-03-06T07:24:31.889200+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:46.188889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907748+0000","last_change":"2024-03-06T07:24:30.881281+0000","last_active":"2024-03-06T07:24:31.907748+0000","last_peered":"2024-03-06T07:24:31.907748+0000","last_clean":"2024-03-06T07:24:31.907748+0000","last_became_active":"2024-03-06T07:24:30.881025+0000","last_became_peered":"2024-03-06T07:24:30.881025+0000","last_unstale":"2024-03-06T07:24:31.907748+0000","last_undegraded":"2024-03-06T07:24:31.907748+0000","last_fullsized":"2024-03-06T07:24:31.907748+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:31.300955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901001+0000","last_change":"2024-03-06T07:24:31.899044+0000","last_active":"2024-03-06T07:24:31.901001+0000","last_peered":"2024-03-06T07:24:31.901001+0000","last_clean":"2024-03-06T07:24:31.901001+0000","last_became_active":"2024-03-06T07:24:31.898707+0000","last_became_peered":"2024-03-06T07:24:31.898707+0000","last_unstale":"2024-03-06T07:24:31.901001+0000","last_undegraded":"2024-03-06T07:24:31.901001+0000","last_fullsized":"2024-03-06T07:24:31.901001+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:12.948471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882272+0000","last_change":"2024-03-06T07:24:30.882286+0000","last_active":"2024-03-06T07:24:31.882272+0000","last_peered":"2024-03-06T07:24:31.882272+0000","last_clean":"2024-03-06T07:24:31.882272+0000","last_became_active":"2024-03-06T07:24:30.882015+0000","last_became_peered":"2024-03-06T07:24:30.882015+0000","last_unstale":"2024-03-06T07:24:31.882272+0000","last_undegraded":"2024-03-06T07:24:31.882272+0000","last_fullsized":"2024-03-06T07:24:31.882272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:17:44.736491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.890080+0000","last_change":"2024-03-06T07:24:31.886626+0000","last_active":"2024-03-06T07:24:31.890080+0000","last_peered":"2024-03-06T07:24:31.890080+0000","last_clean":"2024-03-06T07:24:31.890080+0000","last_became_active":"2024-03-06T07:24:31.886303+0000","last_became_peered":"2024-03-06T07:24:31.886303+0000","last_unstale":"2024-03-06T07:24:31.890080+0000","last_undegraded":"2024-03-06T07:24:31.890080+0000","last_fullsized":"2024-03-06T07:24:31.890080+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:32.274756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"121'8","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.910001+0000","last_change":"2024-03-06T07:24:30.877333+0000","last_active":"2024-03-06T07:24:31.910001+0000","last_peered":"2024-03-06T07:24:31.910001+0000","last_clean":"2024-03-06T07:24:31.910001+0000","last_became_active":"2024-03-06T07:24:30.877063+0000","last_became_peered":"2024-03-06T07:24:30.877063+0000","last_unstale":"2024-03-06T07:24:31.910001+0000","last_undegraded":"2024-03-06T07:24:31.910001+0000","last_fullsized":"2024-03-06T07:24:31.910001+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:00.480778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895158+0000","last_change":"2024-03-06T07:24:31.892187+0000","last_active":"2024-03-06T07:24:31.895158+0000","last_peered":"2024-03-06T07:24:31.895158+0000","last_clean":"2024-03-06T07:24:31.895158+0000","last_became_active":"2024-03-06T07:24:31.891694+0000","last_became_peered":"2024-03-06T07:24:31.891694+0000","last_unstale":"2024-03-06T07:24:31.895158+0000","last_undegraded":"2024-03-06T07:24:31.895158+0000","last_fullsized":"2024-03-06T07:24:31.895158+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:54.374264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905769+0000","last_change":"2024-03-06T07:24:30.882889+0000","last_active":"2024-03-06T07:24:31.905769+0000","last_peered":"2024-03-06T07:24:31.905769+0000","last_clean":"2024-03-06T07:24:31.905769+0000","last_became_active":"2024-03-06T07:24:30.882639+0000","last_became_peered":"2024-03-06T07:24:30.882639+0000","last_unstale":"2024-03-06T07:24:31.905769+0000","last_undegraded":"2024-03-06T07:24:31.905769+0000","last_fullsized":"2024-03-06T07:24:31.905769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899187+0000","last_change":"2024-03-06T07:24:31.892868+0000","last_active":"2024-03-06T07:24:31.899187+0000","last_peered":"2024-03-06T07:24:31.899187+0000","last_clean":"2024-03-06T07:24:31.899187+0000","last_became_active":"2024-03-06T07:24:31.892598+0000","last_became_peered":"2024-03-06T07:24:31.892598+0000","last_unstale":"2024-03-06T07:24:31.899187+0000","last_undegraded":"2024-03-06T07:24:31.899187+0000","last_fullsized":"2024-03-06T07:24:31.899187+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:18.216855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909689+0000","last_change":"2024-03-06T07:24:30.878054+0000","last_active":"2024-03-06T07:24:31.909689+0000","last_peered":"2024-03-06T07:24:31.909689+0000","last_clean":"2024-03-06T07:24:31.909689+0000","last_became_active":"2024-03-06T07:24:30.877505+0000","last_became_peered":"2024-03-06T07:24:30.877505+0000","last_unstale":"2024-03-06T07:24:31.909689+0000","last_undegraded":"2024-03-06T07:24:31.909689+0000","last_fullsized":"2024-03-06T07:24:31.909689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:57.664032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909360+0000","last_change":"2024-03-06T07:24:31.908268+0000","last_active":"2024-03-06T07:24:31.909360+0000","last_peered":"2024-03-06T07:24:31.909360+0000","last_clean":"2024-03-06T07:24:31.909360+0000","last_became_active":"2024-03-06T07:24:31.907970+0000","last_became_peered":"2024-03-06T07:24:31.907970+0000","last_unstale":"2024-03-06T07:24:31.909360+0000","last_undegraded":"2024-03-06T07:24:31.909360+0000","last_fullsized":"2024-03-06T07:24:31.909360+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:16.443283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909973+0000","last_change":"2024-03-06T07:24:30.876831+0000","last_active":"2024-03-06T07:24:31.909973+0000","last_peered":"2024-03-06T07:24:31.909973+0000","last_clean":"2024-03-06T07:24:31.909973+0000","last_became_active":"2024-03-06T07:24:30.876530+0000","last_became_peered":"2024-03-06T07:24:30.876530+0000","last_unstale":"2024-03-06T07:24:31.909973+0000","last_undegraded":"2024-03-06T07:24:31.909973+0000","last_fullsized":"2024-03-06T07:24:31.909973+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:37.275009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.894187+0000","last_change":"2024-03-06T07:24:31.889002+0000","last_active":"2024-03-06T07:24:31.894187+0000","last_peered":"2024-03-06T07:24:31.894187+0000","last_clean":"2024-03-06T07:24:31.894187+0000","last_became_active":"2024-03-06T07:24:31.888695+0000","last_became_peered":"2024-03-06T07:24:31.888695+0000","last_unstale":"2024-03-06T07:24:31.894187+0000","last_undegraded":"2024-03-06T07:24:31.894187+0000","last_fullsized":"2024-03-06T07:24:31.894187+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:58.115551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882989+0000","last_change":"2024-03-06T07:24:30.881711+0000","last_active":"2024-03-06T07:24:31.882989+0000","last_peered":"2024-03-06T07:24:31.882989+0000","last_clean":"2024-03-06T07:24:31.882989+0000","last_became_active":"2024-03-06T07:24:30.881415+0000","last_became_peered":"2024-03-06T07:24:30.881415+0000","last_unstale":"2024-03-06T07:24:31.882989+0000","last_undegraded":"2024-03-06T07:24:31.882989+0000","last_fullsized":"2024-03-06T07:24:31.882989+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:11.454919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.891375+0000","last_change":"2024-03-06T07:24:31.890911+0000","last_active":"2024-03-06T07:24:31.891375+0000","last_peered":"2024-03-06T07:24:31.891375+0000","last_clean":"2024-03-06T07:24:31.891375+0000","last_became_active":"2024-03-06T07:24:31.890616+0000","last_became_peered":"2024-03-06T07:24:31.890616+0000","last_unstale":"2024-03-06T07:24:31.891375+0000","last_undegraded":"2024-03-06T07:24:31.891375+0000","last_fullsized":"2024-03-06T07:24:31.891375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:36.659555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909771+0000","last_change":"2024-03-06T07:24:30.879830+0000","last_active":"2024-03-06T07:24:31.909771+0000","last_peered":"2024-03-06T07:24:31.909771+0000","last_clean":"2024-03-06T07:24:31.909771+0000","last_became_active":"2024-03-06T07:24:30.879527+0000","last_became_peered":"2024-03-06T07:24:30.879527+0000","last_unstale":"2024-03-06T07:24:31.909771+0000","last_undegraded":"2024-03-06T07:24:31.909771+0000","last_fullsized":"2024-03-06T07:24:31.909771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:53.368630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898903+0000","last_change":"2024-03-06T07:24:31.895720+0000","last_active":"2024-03-06T07:24:31.898903+0000","last_peered":"2024-03-06T07:24:31.898903+0000","last_clean":"2024-03-06T07:24:31.898903+0000","last_became_active":"2024-03-06T07:24:31.895455+0000","last_became_peered":"2024-03-06T07:24:31.895455+0000","last_unstale":"2024-03-06T07:24:31.898903+0000","last_undegraded":"2024-03-06T07:24:31.898903+0000","last_fullsized":"2024-03-06T07:24:31.898903+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:21.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.881273+0000","last_change":"2024-03-06T07:24:30.886999+0000","last_active":"2024-03-06T07:24:31.881273+0000","last_peered":"2024-03-06T07:24:31.881273+0000","last_clean":"2024-03-06T07:24:31.881273+0000","last_became_active":"2024-03-06T07:24:30.886737+0000","last_became_peered":"2024-03-06T07:24:30.886737+0000","last_unstale":"2024-03-06T07:24:31.881273+0000","last_undegraded":"2024-03-06T07:24:31.881273+0000","last_fullsized":"2024-03-06T07:24:31.881273+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897440+0000","last_change":"2024-03-06T07:24:31.895426+0000","last_active":"2024-03-06T07:24:31.897440+0000","last_peered":"2024-03-06T07:24:31.897440+0000","last_clean":"2024-03-06T07:24:31.897440+0000","last_became_active":"2024-03-06T07:24:31.895155+0000","last_became_peered":"2024-03-06T07:24:31.895155+0000","last_unstale":"2024-03-06T07:24:31.897440+0000","last_undegraded":"2024-03-06T07:24:31.897440+0000","last_fullsized":"2024-03-06T07:24:31.897440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:28.246685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887333+0000","last_change":"2024-03-06T07:24:30.887512+0000","last_active":"2024-03-06T07:24:31.887333+0000","last_peered":"2024-03-06T07:24:31.887333+0000","last_clean":"2024-03-06T07:24:31.887333+0000","last_became_active":"2024-03-06T07:24:30.887240+0000","last_became_peered":"2024-03-06T07:24:30.887240+0000","last_unstale":"2024-03-06T07:24:31.887333+0000","last_undegraded":"2024-03-06T07:24:31.887333+0000","last_fullsized":"2024-03-06T07:24:31.887333+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:28.394653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898119+0000","last_change":"2024-03-06T07:24:31.891999+0000","last_active":"2024-03-06T07:24:31.898119+0000","last_peered":"2024-03-06T07:24:31.898119+0000","last_clean":"2024-03-06T07:24:31.898119+0000","last_became_active":"2024-03-06T07:24:31.891704+0000","last_became_peered":"2024-03-06T07:24:31.891704+0000","last_unstale":"2024-03-06T07:24:31.898119+0000","last_undegraded":"2024-03-06T07:24:31.898119+0000","last_fullsized":"2024-03-06T07:24:31.898119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.881693+0000","last_change":"2024-03-06T07:24:30.880756+0000","last_active":"2024-03-06T07:24:31.881693+0000","last_peered":"2024-03-06T07:24:31.881693+0000","last_clean":"2024-03-06T07:24:31.881693+0000","last_became_active":"2024-03-06T07:24:30.880489+0000","last_became_peered":"2024-03-06T07:24:30.880489+0000","last_unstale":"2024-03-06T07:24:31.881693+0000","last_undegraded":"2024-03-06T07:24:31.881693+0000","last_fullsized":"2024-03-06T07:24:31.881693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:30.790137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901765+0000","last_change":"2024-03-06T07:24:31.900531+0000","last_active":"2024-03-06T07:24:31.901765+0000","last_peered":"2024-03-06T07:24:31.901765+0000","last_clean":"2024-03-06T07:24:31.901765+0000","last_became_active":"2024-03-06T07:24:31.900171+0000","last_became_peered":"2024-03-06T07:24:31.900171+0000","last_unstale":"2024-03-06T07:24:31.901765+0000","last_undegraded":"2024-03-06T07:24:31.901765+0000","last_fullsized":"2024-03-06T07:24:31.901765+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:28:31.410346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907914+0000","last_change":"2024-03-06T07:24:30.880990+0000","last_active":"2024-03-06T07:24:31.907914+0000","last_peered":"2024-03-06T07:24:31.907914+0000","last_clean":"2024-03-06T07:24:31.907914+0000","last_became_active":"2024-03-06T07:24:30.880641+0000","last_became_peered":"2024-03-06T07:24:30.880641+0000","last_unstale":"2024-03-06T07:24:31.907914+0000","last_undegraded":"2024-03-06T07:24:31.907914+0000","last_fullsized":"2024-03-06T07:24:31.907914+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:59.207968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895192+0000","last_change":"2024-03-06T07:24:31.891381+0000","last_active":"2024-03-06T07:24:31.895192+0000","last_peered":"2024-03-06T07:24:31.895192+0000","last_clean":"2024-03-06T07:24:31.895192+0000","last_became_active":"2024-03-06T07:24:31.890857+0000","last_became_peered":"2024-03-06T07:24:31.890857+0000","last_unstale":"2024-03-06T07:24:31.895192+0000","last_undegraded":"2024-03-06T07:24:31.895192+0000","last_fullsized":"2024-03-06T07:24:31.895192+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:01.708663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879957+0000","last_change":"2024-03-06T07:24:30.882078+0000","last_active":"2024-03-06T07:24:31.879957+0000","last_peered":"2024-03-06T07:24:31.879957+0000","last_clean":"2024-03-06T07:24:31.879957+0000","last_became_active":"2024-03-06T07:24:30.881801+0000","last_became_peered":"2024-03-06T07:24:30.881801+0000","last_unstale":"2024-03-06T07:24:31.879957+0000","last_undegraded":"2024-03-06T07:24:31.879957+0000","last_fullsized":"2024-03-06T07:24:31.879957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899990+0000","last_change":"2024-03-06T07:24:31.898000+0000","last_active":"2024-03-06T07:24:31.899990+0000","last_peered":"2024-03-06T07:24:31.899990+0000","last_clean":"2024-03-06T07:24:31.899990+0000","last_became_active":"2024-03-06T07:24:31.897747+0000","last_became_peered":"2024-03-06T07:24:31.897747+0000","last_unstale":"2024-03-06T07:24:31.899990+0000","last_undegraded":"2024-03-06T07:24:31.899990+0000","last_fullsized":"2024-03-06T07:24:31.899990+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:37.175758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879966+0000","last_change":"2024-03-06T07:24:30.884132+0000","last_active":"2024-03-06T07:24:31.879966+0000","last_peered":"2024-03-06T07:24:31.879966+0000","last_clean":"2024-03-06T07:24:31.879966+0000","last_became_active":"2024-03-06T07:24:30.883817+0000","last_became_peered":"2024-03-06T07:24:30.883817+0000","last_unstale":"2024-03-06T07:24:31.879966+0000","last_undegraded":"2024-03-06T07:24:31.879966+0000","last_fullsized":"2024-03-06T07:24:31.879966+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.884375+0000","last_change":"2024-03-06T07:24:31.881762+0000","last_active":"2024-03-06T07:24:31.884375+0000","last_peered":"2024-03-06T07:24:31.884375+0000","last_clean":"2024-03-06T07:24:31.884375+0000","last_became_active":"2024-03-06T07:24:31.881468+0000","last_became_peered":"2024-03-06T07:24:31.881468+0000","last_unstale":"2024-03-06T07:24:31.884375+0000","last_undegraded":"2024-03-06T07:24:31.884375+0000","last_fullsized":"2024-03-06T07:24:31.884375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:58.833455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882292+0000","last_change":"2024-03-06T07:24:30.878173+0000","last_active":"2024-03-06T07:24:31.882292+0000","last_peered":"2024-03-06T07:24:31.882292+0000","last_clean":"2024-03-06T07:24:31.882292+0000","last_became_active":"2024-03-06T07:24:30.877908+0000","last_became_peered":"2024-03-06T07:24:30.877908+0000","last_unstale":"2024-03-06T07:24:31.882292+0000","last_undegraded":"2024-03-06T07:24:31.882292+0000","last_fullsized":"2024-03-06T07:24:31.882292+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:04.046533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898595+0000","last_change":"2024-03-06T07:24:31.889510+0000","last_active":"2024-03-06T07:24:31.898595+0000","last_peered":"2024-03-06T07:24:31.898595+0000","last_clean":"2024-03-06T07:24:31.898595+0000","last_became_active":"2024-03-06T07:24:31.888010+0000","last_became_peered":"2024-03-06T07:24:31.888010+0000","last_unstale":"2024-03-06T07:24:31.898595+0000","last_undegraded":"2024-03-06T07:24:31.898595+0000","last_fullsized":"2024-03-06T07:24:31.898595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:29.711780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882546+0000","last_change":"2024-03-06T07:24:30.881222+0000","last_active":"2024-03-06T07:24:31.882546+0000","last_peered":"2024-03-06T07:24:31.882546+0000","last_clean":"2024-03-06T07:24:31.882546+0000","last_became_active":"2024-03-06T07:24:30.880886+0000","last_became_peered":"2024-03-06T07:24:30.880886+0000","last_unstale":"2024-03-06T07:24:31.882546+0000","last_undegraded":"2024-03-06T07:24:31.882546+0000","last_fullsized":"2024-03-06T07:24:31.882546+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:00.480778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889857+0000","last_change":"2024-03-06T07:24:31.888646+0000","last_active":"2024-03-06T07:24:31.889857+0000","last_peered":"2024-03-06T07:24:31.889857+0000","last_clean":"2024-03-06T07:24:31.889857+0000","last_became_active":"2024-03-06T07:24:31.888359+0000","last_became_peered":"2024-03-06T07:24:31.888359+0000","last_unstale":"2024-03-06T07:24:31.889857+0000","last_undegraded":"2024-03-06T07:24:31.889857+0000","last_fullsized":"2024-03-06T07:24:31.889857+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889974+0000","last_change":"2024-03-06T07:24:30.880337+0000","last_active":"2024-03-06T07:24:31.889974+0000","last_peered":"2024-03-06T07:24:31.889974+0000","last_clean":"2024-03-06T07:24:31.889974+0000","last_became_active":"2024-03-06T07:24:30.880068+0000","last_became_peered":"2024-03-06T07:24:30.880068+0000","last_unstale":"2024-03-06T07:24:31.889974+0000","last_undegraded":"2024-03-06T07:24:31.889974+0000","last_fullsized":"2024-03-06T07:24:31.889974+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:49.637038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904843+0000","last_change":"2024-03-06T07:24:31.894727+0000","last_active":"2024-03-06T07:24:31.904843+0000","last_peered":"2024-03-06T07:24:31.904843+0000","last_clean":"2024-03-06T07:24:31.904843+0000","last_became_active":"2024-03-06T07:24:31.894429+0000","last_became_peered":"2024-03-06T07:24:31.894429+0000","last_unstale":"2024-03-06T07:24:31.904843+0000","last_undegraded":"2024-03-06T07:24:31.904843+0000","last_fullsized":"2024-03-06T07:24:31.904843+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.972365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880820+0000","last_change":"2024-03-06T07:24:30.879025+0000","last_active":"2024-03-06T07:24:31.880820+0000","last_peered":"2024-03-06T07:24:31.880820+0000","last_clean":"2024-03-06T07:24:31.880820+0000","last_became_active":"2024-03-06T07:24:30.877933+0000","last_became_peered":"2024-03-06T07:24:30.877933+0000","last_unstale":"2024-03-06T07:24:31.880820+0000","last_undegraded":"2024-03-06T07:24:31.880820+0000","last_fullsized":"2024-03-06T07:24:31.880820+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899949+0000","last_change":"2024-03-06T07:24:31.896772+0000","last_active":"2024-03-06T07:24:31.899949+0000","last_peered":"2024-03-06T07:24:31.899949+0000","last_clean":"2024-03-06T07:24:31.899949+0000","last_became_active":"2024-03-06T07:24:31.896491+0000","last_became_peered":"2024-03-06T07:24:31.896491+0000","last_unstale":"2024-03-06T07:24:31.899949+0000","last_undegraded":"2024-03-06T07:24:31.899949+0000","last_fullsized":"2024-03-06T07:24:31.899949+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:44.110497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879885+0000","last_change":"2024-03-06T07:24:30.881511+0000","last_active":"2024-03-06T07:24:31.879885+0000","last_peered":"2024-03-06T07:24:31.879885+0000","last_clean":"2024-03-06T07:24:31.879885+0000","last_became_active":"2024-03-06T07:24:30.881203+0000","last_became_peered":"2024-03-06T07:24:30.881203+0000","last_unstale":"2024-03-06T07:24:31.879885+0000","last_undegraded":"2024-03-06T07:24:31.879885+0000","last_fullsized":"2024-03-06T07:24:31.879885+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:15.437648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895570+0000","last_change":"2024-03-06T07:24:31.892700+0000","last_active":"2024-03-06T07:24:31.895570+0000","last_peered":"2024-03-06T07:24:31.895570+0000","last_clean":"2024-03-06T07:24:31.895570+0000","last_became_active":"2024-03-06T07:24:31.892440+0000","last_became_peered":"2024-03-06T07:24:31.892440+0000","last_unstale":"2024-03-06T07:24:31.895570+0000","last_undegraded":"2024-03-06T07:24:31.895570+0000","last_fullsized":"2024-03-06T07:24:31.895570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:14.966060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889268+0000","last_change":"2024-03-06T07:24:30.887838+0000","last_active":"2024-03-06T07:24:31.889268+0000","last_peered":"2024-03-06T07:24:31.889268+0000","last_clean":"2024-03-06T07:24:31.889268+0000","last_became_active":"2024-03-06T07:24:30.887554+0000","last_became_peered":"2024-03-06T07:24:30.887554+0000","last_unstale":"2024-03-06T07:24:31.889268+0000","last_undegraded":"2024-03-06T07:24:31.889268+0000","last_fullsized":"2024-03-06T07:24:31.889268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:30.643410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":357,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.886719+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:24:31.886719+0000","last_peered":"2024-03-06T07:24:31.886719+0000","last_clean":"2024-03-06T07:24:31.886719+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:24:31.886719+0000","last_undegraded":"2024-03-06T07:24:31.886719+0000","last_fullsized":"2024-03-06T07:24:31.886719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":1315516,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":732385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":1322739,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:34.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:24:35.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.088+0000 7f527759e640 1 Processor -- start 2024-03-06T07:24:35.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.088+0000 7f527759e640 1 -- start start 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527759e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f527814ba90 0x7f52781559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527759e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5278158a40 0x7f5278158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527759e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527814be70 con 0x7f52780c8690 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527759e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52781593f0 con 0x7f527814ba90 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527759e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52780cad30 con 0x7f5278158a40 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f527814ba90 0x7f52781559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f526dd9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52780c8690 0x7f52780c5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44578/0 (socket says 172.21.15.17:44578) 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f526dd9b640 1 -- 172.21.15.17:0/1162450402 learned_addr learned my addr 172.21.15.17:0/1162450402 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f5276d9d640 1 --2- 172.21.15.17:0/1162450402 >> v2:172.21.15.163:3300/0 conn(0x7f5278158a40 0x7f5278158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527659c640 1 -- 172.21.15.17:0/1162450402 >> v2:172.21.15.163:3300/0 conn(0x7f5278158a40 msgr2=0x7f5278158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527659c640 1 --2- 172.21.15.17:0/1162450402 >> v2:172.21.15.163:3300/0 conn(0x7f5278158a40 0x7f5278158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527659c640 1 -- 172.21.15.17:0/1162450402 >> v1:172.21.15.17:6789/0 conn(0x7f52780c8690 legacy=0x7f52780c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f5275d9b640 1 -- 172.21.15.17:0/1162450402 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857963769 0 0) 0x7f5260001110 con 0x7f52780c8690 2024-03-06T07:24:35.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.089+0000 7f527659c640 1 -- 172.21.15.17:0/1162450402 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780cb010 con 0x7f527814ba90 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.090+0000 7f527659c640 1 --2- 172.21.15.17:0/1162450402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f527814ba90 0x7f52781559b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5278154f30 tx=0x7f526802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1e549d72146dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.090+0000 7f5275d9b640 1 -- 172.21.15.17:0/1162450402 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268035020 con 0x7f527814ba90 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.090+0000 7f5275d9b640 1 -- 172.21.15.17:0/1162450402 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268031830 con 0x7f527814ba90 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.090+0000 7f5275d9b640 1 -- 172.21.15.17:0/1162450402 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268031c00 con 0x7f527814ba90 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 -- 172.21.15.17:0/1162450402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f527814ba90 msgr2=0x7f52781559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 --2- 172.21.15.17:0/1162450402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f527814ba90 0x7f52781559b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f5278154f30 tx=0x7f526802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 -- 172.21.15.17:0/1162450402 shutdown_connections 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 --2- 172.21.15.17:0/1162450402 >> v2:172.21.15.163:3300/0 conn(0x7f5278158a40 0x7f5278158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 --2- 172.21.15.17:0/1162450402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f527814ba90 0x7f52781559b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 -- 172.21.15.17:0/1162450402 >> 172.21.15.17:0/1162450402 conn(0x7f52780728d0 msgr2=0x7f5278072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 -- 172.21.15.17:0/1162450402 shutdown_connections 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.091+0000 7f527759e640 1 -- 172.21.15.17:0/1162450402 wait complete. 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.093+0000 7f527759e640 1 Processor -- start 2024-03-06T07:24:35.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.093+0000 7f527759e640 1 -- start start 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f527814ba90 0x7f5278141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 0x7f5278142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527806cf60 con 0x7f52780c8690 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5278158520 con 0x7f527814ba90 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527759e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52780c5820 con 0x7f5278158a40 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57800/0 (socket says 172.21.15.17:57800) 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f527659c640 1 -- 172.21.15.17:0/3096168779 learned_addr learned my addr 172.21.15.17:0/3096168779 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f526dd9b640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f527814ba90 0x7f5278141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.094+0000 7f5276d9d640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 0x7f5278142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 -- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 msgr2=0x7f5278142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 0x7f5278142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 -- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f527814ba90 msgr2=0x7f5278141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f527814ba90 0x7f5278141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 -- 172.21.15.17:0/3096168779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52781425b0 con 0x7f52780c8690 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f5276d9d640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 0x7f5278142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527659c640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5268009890 tx=0x7f52680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268035070 con 0x7f52780c8690 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268004250 con 0x7f52780c8690 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5278157e80 con 0x7f52780c8690 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526803a6f0 con 0x7f52780c8690 2024-03-06T07:24:35.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.095+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5278142a60 con 0x7f52780c8690 2024-03-06T07:24:35.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.096+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5278069430 con 0x7f52780c8690 2024-03-06T07:24:35.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.097+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5268038020 con 0x7f52780c8690 2024-03-06T07:24:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.097+0000 7f526f7fe640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 0x7f525806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.097+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52680abfb0 con 0x7f52780c8690 2024-03-06T07:24:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.097+0000 7f526dd9b640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 0x7f525806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.098+0000 7f526dd9b640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 0x7f525806d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f52780c1240 tx=0x7f5260007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.104+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5268040070 con 0x7f52780c8690 2024-03-06T07:24:35.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.281+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f527806cd40 con 0x7f525806b4d0 2024-03-06T07:24:35.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.287+0000 7f526f7fe640 1 -- 172.21.15.17:0/3096168779 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7f527806cd40 con 0x7f525806b4d0 2024-03-06T07:24:35.292 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:35.293 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:24:35.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 msgr2=0x7f525806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 0x7f525806d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f52780c1240 tx=0x7f5260007b50 comp rx=0 tx=0).stop 2024-03-06T07:24:35.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 msgr2=0x7f52781575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f5268009890 tx=0x7f52680046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 shutdown_connections 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f525806b4d0 0x7f525806d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5278158a40 0x7f5278142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f527814ba90 0x7f5278141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 --2- 172.21.15.17:0/3096168779 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780c8690 0x7f52781575e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 >> 172.21.15.17:0/3096168779 conn(0x7f52780728d0 msgr2=0x7f52780c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 shutdown_connections 2024-03-06T07:24:35.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.290+0000 7f527759e640 1 -- 172.21.15.17:0/3096168779 wait complete. 2024-03-06T07:24:35.302 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-03-06T07:24:33.400720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261392,"kb_used_data":11908,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713328,"statfs":{"total":773094113280,"available":772826447872,"internally_reserved":0,"allocated":12193792,"data_stored":6996380,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001111"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898318+0000","last_change":"2024-03-06T07:24:31.893689+0000","last_active":"2024-03-06T07:24:31.898318+0000","last_peered":"2024-03-06T07:24:31.898318+0000","last_clean":"2024-03-06T07:24:31.898318+0000","last_became_active":"2024-03-06T07:24:31.892752+0000","last_became_peered":"2024-03-06T07:24:31.892752+0000","last_unstale":"2024-03-06T07:24:31.898318+0000","last_undegraded":"2024-03-06T07:24:31.898318+0000","last_fullsized":"2024-03-06T07:24:31.898318+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:18.884670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895780+0000","last_change":"2024-03-06T07:24:31.886106+0000","last_active":"2024-03-06T07:24:31.895780+0000","last_peered":"2024-03-06T07:24:31.895780+0000","last_clean":"2024-03-06T07:24:31.895780+0000","last_became_active":"2024-03-06T07:24:31.885804+0000","last_became_peered":"2024-03-06T07:24:31.885804+0000","last_unstale":"2024-03-06T07:24:31.895780+0000","last_undegraded":"2024-03-06T07:24:31.895780+0000","last_fullsized":"2024-03-06T07:24:31.895780+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895589+0000","last_change":"2024-03-06T07:24:31.893097+0000","last_active":"2024-03-06T07:24:31.895589+0000","last_peered":"2024-03-06T07:24:31.895589+0000","last_clean":"2024-03-06T07:24:31.895589+0000","last_became_active":"2024-03-06T07:24:31.892812+0000","last_became_peered":"2024-03-06T07:24:31.892812+0000","last_unstale":"2024-03-06T07:24:31.895589+0000","last_undegraded":"2024-03-06T07:24:31.895589+0000","last_fullsized":"2024-03-06T07:24:31.895589+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:06.484188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897149+0000","last_change":"2024-03-06T07:24:31.890231+0000","last_active":"2024-03-06T07:24:31.897149+0000","last_peered":"2024-03-06T07:24:31.897149+0000","last_clean":"2024-03-06T07:24:31.897149+0000","last_became_active":"2024-03-06T07:24:31.890006+0000","last_became_peered":"2024-03-06T07:24:31.890006+0000","last_unstale":"2024-03-06T07:24:31.897149+0000","last_undegraded":"2024-03-06T07:24:31.897149+0000","last_fullsized":"2024-03-06T07:24:31.897149+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897731+0000","last_change":"2024-03-06T07:24:31.896446+0000","last_active":"2024-03-06T07:24:31.897731+0000","last_peered":"2024-03-06T07:24:31.897731+0000","last_clean":"2024-03-06T07:24:31.897731+0000","last_became_active":"2024-03-06T07:24:31.896174+0000","last_became_peered":"2024-03-06T07:24:31.896174+0000","last_unstale":"2024-03-06T07:24:31.897731+0000","last_undegraded":"2024-03-06T07:24:31.897731+0000","last_fullsized":"2024-03-06T07:24:31.897731+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:04:55.787579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904724+0000","last_change":"2024-03-06T07:24:31.896903+0000","last_active":"2024-03-06T07:24:31.904724+0000","last_peered":"2024-03-06T07:24:31.904724+0000","last_clean":"2024-03-06T07:24:31.904724+0000","last_became_active":"2024-03-06T07:24:31.896613+0000","last_became_peered":"2024-03-06T07:24:31.896613+0000","last_unstale":"2024-03-06T07:24:31.904724+0000","last_undegraded":"2024-03-06T07:24:31.904724+0000","last_fullsized":"2024-03-06T07:24:31.904724+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:27.997609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899946+0000","last_change":"2024-03-06T07:24:31.897380+0000","last_active":"2024-03-06T07:24:31.899946+0000","last_peered":"2024-03-06T07:24:31.899946+0000","last_clean":"2024-03-06T07:24:31.899946+0000","last_became_active":"2024-03-06T07:24:31.897136+0000","last_became_peered":"2024-03-06T07:24:31.897136+0000","last_unstale":"2024-03-06T07:24:31.899946+0000","last_undegraded":"2024-03-06T07:24:31.899946+0000","last_fullsized":"2024-03-06T07:24:31.899946+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:19:34.962764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887484+0000","last_change":"2024-03-06T07:24:31.884139+0000","last_active":"2024-03-06T07:24:31.887484+0000","last_peered":"2024-03-06T07:24:31.887484+0000","last_clean":"2024-03-06T07:24:31.887484+0000","last_became_active":"2024-03-06T07:24:31.883827+0000","last_became_peered":"2024-03-06T07:24:31.883827+0000","last_unstale":"2024-03-06T07:24:31.887484+0000","last_undegraded":"2024-03-06T07:24:31.887484+0000","last_fullsized":"2024-03-06T07:24:31.887484+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:01.486412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904840+0000","last_change":"2024-03-06T07:24:31.897009+0000","last_active":"2024-03-06T07:24:31.904840+0000","last_peered":"2024-03-06T07:24:31.904840+0000","last_clean":"2024-03-06T07:24:31.904840+0000","last_became_active":"2024-03-06T07:24:31.891217+0000","last_became_peered":"2024-03-06T07:24:31.891217+0000","last_unstale":"2024-03-06T07:24:31.904840+0000","last_undegraded":"2024-03-06T07:24:31.904840+0000","last_fullsized":"2024-03-06T07:24:31.904840+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:31.649044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887851+0000","last_change":"2024-03-06T07:24:31.883659+0000","last_active":"2024-03-06T07:24:31.887851+0000","last_peered":"2024-03-06T07:24:31.887851+0000","last_clean":"2024-03-06T07:24:31.887851+0000","last_became_active":"2024-03-06T07:24:31.883333+0000","last_became_peered":"2024-03-06T07:24:31.883333+0000","last_unstale":"2024-03-06T07:24:31.887851+0000","last_undegraded":"2024-03-06T07:24:31.887851+0000","last_fullsized":"2024-03-06T07:24:31.887851+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:54.014228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897092+0000","last_change":"2024-03-06T07:24:31.893748+0000","last_active":"2024-03-06T07:24:31.897092+0000","last_peered":"2024-03-06T07:24:31.897092+0000","last_clean":"2024-03-06T07:24:31.897092+0000","last_became_active":"2024-03-06T07:24:31.893382+0000","last_became_peered":"2024-03-06T07:24:31.893382+0000","last_unstale":"2024-03-06T07:24:31.897092+0000","last_undegraded":"2024-03-06T07:24:31.897092+0000","last_fullsized":"2024-03-06T07:24:31.897092+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:29.711780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887500+0000","last_change":"2024-03-06T07:24:31.885623+0000","last_active":"2024-03-06T07:24:31.887500+0000","last_peered":"2024-03-06T07:24:31.887500+0000","last_clean":"2024-03-06T07:24:31.887500+0000","last_became_active":"2024-03-06T07:24:31.885357+0000","last_became_peered":"2024-03-06T07:24:31.885357+0000","last_unstale":"2024-03-06T07:24:31.887500+0000","last_undegraded":"2024-03-06T07:24:31.887500+0000","last_fullsized":"2024-03-06T07:24:31.887500+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:06.298928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904888+0000","last_change":"2024-03-06T07:24:31.896445+0000","last_active":"2024-03-06T07:24:31.904888+0000","last_peered":"2024-03-06T07:24:31.904888+0000","last_clean":"2024-03-06T07:24:31.904888+0000","last_became_active":"2024-03-06T07:24:31.896082+0000","last_became_peered":"2024-03-06T07:24:31.896082+0000","last_unstale":"2024-03-06T07:24:31.904888+0000","last_undegraded":"2024-03-06T07:24:31.904888+0000","last_fullsized":"2024-03-06T07:24:31.904888+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:29.400287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900625+0000","last_change":"2024-03-06T07:24:31.897231+0000","last_active":"2024-03-06T07:24:31.900625+0000","last_peered":"2024-03-06T07:24:31.900625+0000","last_clean":"2024-03-06T07:24:31.900625+0000","last_became_active":"2024-03-06T07:24:31.896897+0000","last_became_peered":"2024-03-06T07:24:31.896897+0000","last_unstale":"2024-03-06T07:24:31.900625+0000","last_undegraded":"2024-03-06T07:24:31.900625+0000","last_fullsized":"2024-03-06T07:24:31.900625+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:50.767038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898953+0000","last_change":"2024-03-06T07:24:31.895044+0000","last_active":"2024-03-06T07:24:31.898953+0000","last_peered":"2024-03-06T07:24:31.898953+0000","last_clean":"2024-03-06T07:24:31.898953+0000","last_became_active":"2024-03-06T07:24:31.894762+0000","last_became_peered":"2024-03-06T07:24:31.894762+0000","last_unstale":"2024-03-06T07:24:31.898953+0000","last_undegraded":"2024-03-06T07:24:31.898953+0000","last_fullsized":"2024-03-06T07:24:31.898953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:40.179739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907226+0000","last_change":"2024-03-06T07:24:31.898603+0000","last_active":"2024-03-06T07:24:31.907226+0000","last_peered":"2024-03-06T07:24:31.907226+0000","last_clean":"2024-03-06T07:24:31.907226+0000","last_became_active":"2024-03-06T07:24:31.898283+0000","last_became_peered":"2024-03-06T07:24:31.898283+0000","last_unstale":"2024-03-06T07:24:31.907226+0000","last_undegraded":"2024-03-06T07:24:31.907226+0000","last_fullsized":"2024-03-06T07:24:31.907226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:50.642673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904815+0000","last_change":"2024-03-06T07:24:31.894775+0000","last_active":"2024-03-06T07:24:31.904815+0000","last_peered":"2024-03-06T07:24:31.904815+0000","last_clean":"2024-03-06T07:24:31.904815+0000","last_became_active":"2024-03-06T07:24:31.894503+0000","last_became_peered":"2024-03-06T07:24:31.894503+0000","last_unstale":"2024-03-06T07:24:31.904815+0000","last_undegraded":"2024-03-06T07:24:31.904815+0000","last_fullsized":"2024-03-06T07:24:31.904815+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:41.707033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900714+0000","last_change":"2024-03-06T07:24:31.898333+0000","last_active":"2024-03-06T07:24:31.900714+0000","last_peered":"2024-03-06T07:24:31.900714+0000","last_clean":"2024-03-06T07:24:31.900714+0000","last_became_active":"2024-03-06T07:24:31.898046+0000","last_became_peered":"2024-03-06T07:24:31.898046+0000","last_unstale":"2024-03-06T07:24:31.900714+0000","last_undegraded":"2024-03-06T07:24:31.900714+0000","last_fullsized":"2024-03-06T07:24:31.900714+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:20.764174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901939+0000","last_change":"2024-03-06T07:24:31.900986+0000","last_active":"2024-03-06T07:24:31.901939+0000","last_peered":"2024-03-06T07:24:31.901939+0000","last_clean":"2024-03-06T07:24:31.901939+0000","last_became_active":"2024-03-06T07:24:31.900604+0000","last_became_peered":"2024-03-06T07:24:31.900604+0000","last_unstale":"2024-03-06T07:24:31.901939+0000","last_undegraded":"2024-03-06T07:24:31.901939+0000","last_fullsized":"2024-03-06T07:24:31.901939+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:57.579869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.903408+0000","last_change":"2024-03-06T07:24:31.901613+0000","last_active":"2024-03-06T07:24:31.903408+0000","last_peered":"2024-03-06T07:24:31.903408+0000","last_clean":"2024-03-06T07:24:31.903408+0000","last_became_active":"2024-03-06T07:24:31.901344+0000","last_became_peered":"2024-03-06T07:24:31.901344+0000","last_unstale":"2024-03-06T07:24:31.903408+0000","last_undegraded":"2024-03-06T07:24:31.903408+0000","last_fullsized":"2024-03-06T07:24:31.903408+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:13:51.389165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895899+0000","last_change":"2024-03-06T07:24:31.889184+0000","last_active":"2024-03-06T07:24:31.895899+0000","last_peered":"2024-03-06T07:24:31.895899+0000","last_clean":"2024-03-06T07:24:31.895899+0000","last_became_active":"2024-03-06T07:24:31.888837+0000","last_became_peered":"2024-03-06T07:24:31.888837+0000","last_unstale":"2024-03-06T07:24:31.895899+0000","last_undegraded":"2024-03-06T07:24:31.895899+0000","last_fullsized":"2024-03-06T07:24:31.895899+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887963+0000","last_change":"2024-03-06T07:24:31.886938+0000","last_active":"2024-03-06T07:24:31.887963+0000","last_peered":"2024-03-06T07:24:31.887963+0000","last_clean":"2024-03-06T07:24:31.887963+0000","last_became_active":"2024-03-06T07:24:31.886618+0000","last_became_peered":"2024-03-06T07:24:31.886618+0000","last_unstale":"2024-03-06T07:24:31.887963+0000","last_undegraded":"2024-03-06T07:24:31.887963+0000","last_fullsized":"2024-03-06T07:24:31.887963+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:44.997413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898166+0000","last_change":"2024-03-06T07:24:31.895136+0000","last_active":"2024-03-06T07:24:31.898166+0000","last_peered":"2024-03-06T07:24:31.898166+0000","last_clean":"2024-03-06T07:24:31.898166+0000","last_became_active":"2024-03-06T07:24:31.893817+0000","last_became_peered":"2024-03-06T07:24:31.893817+0000","last_unstale":"2024-03-06T07:24:31.898166+0000","last_undegraded":"2024-03-06T07:24:31.898166+0000","last_fullsized":"2024-03-06T07:24:31.898166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:41.707033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907204+0000","last_change":"2024-03-06T07:24:31.896742+0000","last_active":"2024-03-06T07:24:31.907204+0000","last_peered":"2024-03-06T07:24:31.907204+0000","last_clean":"2024-03-06T07:24:31.907204+0000","last_became_active":"2024-03-06T07:24:31.896454+0000","last_became_peered":"2024-03-06T07:24:31.896454+0000","last_unstale":"2024-03-06T07:24:31.907204+0000","last_undegraded":"2024-03-06T07:24:31.907204+0000","last_fullsized":"2024-03-06T07:24:31.907204+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:58.175086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897995+0000","last_change":"2024-03-06T07:24:31.897006+0000","last_active":"2024-03-06T07:24:31.897995+0000","last_peered":"2024-03-06T07:24:31.897995+0000","last_clean":"2024-03-06T07:24:31.897995+0000","last_became_active":"2024-03-06T07:24:31.896728+0000","last_became_peered":"2024-03-06T07:24:31.896728+0000","last_unstale":"2024-03-06T07:24:31.897995+0000","last_undegraded":"2024-03-06T07:24:31.897995+0000","last_fullsized":"2024-03-06T07:24:31.897995+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895521+0000","last_change":"2024-03-06T07:24:31.892789+0000","last_active":"2024-03-06T07:24:31.895521+0000","last_peered":"2024-03-06T07:24:31.895521+0000","last_clean":"2024-03-06T07:24:31.895521+0000","last_became_active":"2024-03-06T07:24:31.892477+0000","last_became_peered":"2024-03-06T07:24:31.892477+0000","last_unstale":"2024-03-06T07:24:31.895521+0000","last_undegraded":"2024-03-06T07:24:31.895521+0000","last_fullsized":"2024-03-06T07:24:31.895521+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:18.006744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907656+0000","last_change":"2024-03-06T07:24:31.905233+0000","last_active":"2024-03-06T07:24:31.907656+0000","last_peered":"2024-03-06T07:24:31.907656+0000","last_clean":"2024-03-06T07:24:31.907656+0000","last_became_active":"2024-03-06T07:24:31.904956+0000","last_became_peered":"2024-03-06T07:24:31.904956+0000","last_unstale":"2024-03-06T07:24:31.907656+0000","last_undegraded":"2024-03-06T07:24:31.907656+0000","last_fullsized":"2024-03-06T07:24:31.907656+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:17.223972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.888486+0000","last_change":"2024-03-06T07:24:31.882523+0000","last_active":"2024-03-06T07:24:31.888486+0000","last_peered":"2024-03-06T07:24:31.888486+0000","last_clean":"2024-03-06T07:24:31.888486+0000","last_became_active":"2024-03-06T07:24:31.882240+0000","last_became_peered":"2024-03-06T07:24:31.882240+0000","last_unstale":"2024-03-06T07:24:31.888486+0000","last_undegraded":"2024-03-06T07:24:31.888486+0000","last_fullsized":"2024-03-06T07:24:31.888486+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:03.777899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.893347+0000","last_change":"2024-03-06T07:24:31.889970+0000","last_active":"2024-03-06T07:24:31.893347+0000","last_peered":"2024-03-06T07:24:31.893347+0000","last_clean":"2024-03-06T07:24:31.893347+0000","last_became_active":"2024-03-06T07:24:31.889662+0000","last_became_peered":"2024-03-06T07:24:31.889662+0000","last_unstale":"2024-03-06T07:24:31.893347+0000","last_undegraded":"2024-03-06T07:24:31.893347+0000","last_fullsized":"2024-03-06T07:24:31.893347+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:50:55.060423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897216+0000","last_change":"2024-03-06T07:24:31.896135+0000","last_active":"2024-03-06T07:24:31.897216+0000","last_peered":"2024-03-06T07:24:31.897216+0000","last_clean":"2024-03-06T07:24:31.897216+0000","last_became_active":"2024-03-06T07:24:31.895880+0000","last_became_peered":"2024-03-06T07:24:31.895880+0000","last_unstale":"2024-03-06T07:24:31.897216+0000","last_undegraded":"2024-03-06T07:24:31.897216+0000","last_fullsized":"2024-03-06T07:24:31.897216+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897569+0000","last_change":"2024-03-06T07:24:31.895162+0000","last_active":"2024-03-06T07:24:31.897569+0000","last_peered":"2024-03-06T07:24:31.897569+0000","last_clean":"2024-03-06T07:24:31.897569+0000","last_became_active":"2024-03-06T07:24:31.894813+0000","last_became_peered":"2024-03-06T07:24:31.894813+0000","last_unstale":"2024-03-06T07:24:31.897569+0000","last_undegraded":"2024-03-06T07:24:31.897569+0000","last_fullsized":"2024-03-06T07:24:31.897569+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:05.052167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.892016+0000","last_change":"2024-03-06T07:24:31.887068+0000","last_active":"2024-03-06T07:24:31.892016+0000","last_peered":"2024-03-06T07:24:31.892016+0000","last_clean":"2024-03-06T07:24:31.892016+0000","last_became_active":"2024-03-06T07:24:31.886570+0000","last_became_peered":"2024-03-06T07:24:31.886570+0000","last_unstale":"2024-03-06T07:24:31.892016+0000","last_undegraded":"2024-03-06T07:24:31.892016+0000","last_fullsized":"2024-03-06T07:24:31.892016+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:17.026975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899922+0000","last_change":"2024-03-06T07:24:31.897006+0000","last_active":"2024-03-06T07:24:31.899922+0000","last_peered":"2024-03-06T07:24:31.899922+0000","last_clean":"2024-03-06T07:24:31.899922+0000","last_became_active":"2024-03-06T07:24:31.896753+0000","last_became_peered":"2024-03-06T07:24:31.896753+0000","last_unstale":"2024-03-06T07:24:31.899922+0000","last_undegraded":"2024-03-06T07:24:31.899922+0000","last_fullsized":"2024-03-06T07:24:31.899922+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"121'4","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909708+0000","last_change":"2024-03-06T07:24:30.881388+0000","last_active":"2024-03-06T07:24:31.909708+0000","last_peered":"2024-03-06T07:24:31.909708+0000","last_clean":"2024-03-06T07:24:31.909708+0000","last_became_active":"2024-03-06T07:24:30.881117+0000","last_became_peered":"2024-03-06T07:24:30.881117+0000","last_unstale":"2024-03-06T07:24:31.909708+0000","last_undegraded":"2024-03-06T07:24:31.909708+0000","last_fullsized":"2024-03-06T07:24:31.909708+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:51:10.031258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898375+0000","last_change":"2024-03-06T07:24:31.892778+0000","last_active":"2024-03-06T07:24:31.898375+0000","last_peered":"2024-03-06T07:24:31.898375+0000","last_clean":"2024-03-06T07:24:31.898375+0000","last_became_active":"2024-03-06T07:24:31.892436+0000","last_became_peered":"2024-03-06T07:24:31.892436+0000","last_unstale":"2024-03-06T07:24:31.898375+0000","last_undegraded":"2024-03-06T07:24:31.898375+0000","last_fullsized":"2024-03-06T07:24:31.898375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:02:25.338014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.908140+0000","last_change":"2024-03-06T07:24:30.884206+0000","last_active":"2024-03-06T07:24:31.908140+0000","last_peered":"2024-03-06T07:24:31.908140+0000","last_clean":"2024-03-06T07:24:31.908140+0000","last_became_active":"2024-03-06T07:24:30.883762+0000","last_became_peered":"2024-03-06T07:24:30.883762+0000","last_unstale":"2024-03-06T07:24:31.908140+0000","last_undegraded":"2024-03-06T07:24:31.908140+0000","last_fullsized":"2024-03-06T07:24:31.908140+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:01:56.393829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898146+0000","last_change":"2024-03-06T07:24:31.893918+0000","last_active":"2024-03-06T07:24:31.898146+0000","last_peered":"2024-03-06T07:24:31.898146+0000","last_clean":"2024-03-06T07:24:31.898146+0000","last_became_active":"2024-03-06T07:24:31.893150+0000","last_became_peered":"2024-03-06T07:24:31.893150+0000","last_unstale":"2024-03-06T07:24:31.898146+0000","last_undegraded":"2024-03-06T07:24:31.898146+0000","last_fullsized":"2024-03-06T07:24:31.898146+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.972365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909206+0000","last_change":"2024-03-06T07:24:30.882894+0000","last_active":"2024-03-06T07:24:31.909206+0000","last_peered":"2024-03-06T07:24:31.909206+0000","last_clean":"2024-03-06T07:24:31.909206+0000","last_became_active":"2024-03-06T07:24:30.882548+0000","last_became_peered":"2024-03-06T07:24:30.882548+0000","last_unstale":"2024-03-06T07:24:31.909206+0000","last_undegraded":"2024-03-06T07:24:31.909206+0000","last_fullsized":"2024-03-06T07:24:31.909206+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:24:57.169452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899593+0000","last_change":"2024-03-06T07:24:31.896400+0000","last_active":"2024-03-06T07:24:31.899593+0000","last_peered":"2024-03-06T07:24:31.899593+0000","last_clean":"2024-03-06T07:24:31.899593+0000","last_became_active":"2024-03-06T07:24:31.896134+0000","last_became_peered":"2024-03-06T07:24:31.896134+0000","last_unstale":"2024-03-06T07:24:31.899593+0000","last_undegraded":"2024-03-06T07:24:31.899593+0000","last_fullsized":"2024-03-06T07:24:31.899593+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:38.542770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905649+0000","last_change":"2024-03-06T07:24:30.883865+0000","last_active":"2024-03-06T07:24:31.905649+0000","last_peered":"2024-03-06T07:24:31.905649+0000","last_clean":"2024-03-06T07:24:31.905649+0000","last_became_active":"2024-03-06T07:24:30.883527+0000","last_became_peered":"2024-03-06T07:24:30.883527+0000","last_unstale":"2024-03-06T07:24:31.905649+0000","last_undegraded":"2024-03-06T07:24:31.905649+0000","last_fullsized":"2024-03-06T07:24:31.905649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:37.966731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897597+0000","last_change":"2024-03-06T07:24:31.894846+0000","last_active":"2024-03-06T07:24:31.897597+0000","last_peered":"2024-03-06T07:24:31.897597+0000","last_clean":"2024-03-06T07:24:31.897597+0000","last_became_active":"2024-03-06T07:24:31.894495+0000","last_became_peered":"2024-03-06T07:24:31.894495+0000","last_unstale":"2024-03-06T07:24:31.897597+0000","last_undegraded":"2024-03-06T07:24:31.897597+0000","last_fullsized":"2024-03-06T07:24:31.897597+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907689+0000","last_change":"2024-03-06T07:24:30.880595+0000","last_active":"2024-03-06T07:24:31.907689+0000","last_peered":"2024-03-06T07:24:31.907689+0000","last_clean":"2024-03-06T07:24:31.907689+0000","last_became_active":"2024-03-06T07:24:30.880307+0000","last_became_peered":"2024-03-06T07:24:30.880307+0000","last_unstale":"2024-03-06T07:24:31.907689+0000","last_undegraded":"2024-03-06T07:24:31.907689+0000","last_fullsized":"2024-03-06T07:24:31.907689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:13.960426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.896456+0000","last_change":"2024-03-06T07:24:31.885800+0000","last_active":"2024-03-06T07:24:31.896456+0000","last_peered":"2024-03-06T07:24:31.896456+0000","last_clean":"2024-03-06T07:24:31.896456+0000","last_became_active":"2024-03-06T07:24:31.885508+0000","last_became_peered":"2024-03-06T07:24:31.885508+0000","last_unstale":"2024-03-06T07:24:31.896456+0000","last_undegraded":"2024-03-06T07:24:31.896456+0000","last_fullsized":"2024-03-06T07:24:31.896456+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:31.795771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905270+0000","last_change":"2024-03-06T07:24:30.879944+0000","last_active":"2024-03-06T07:24:31.905270+0000","last_peered":"2024-03-06T07:24:31.905270+0000","last_clean":"2024-03-06T07:24:31.905270+0000","last_became_active":"2024-03-06T07:24:30.879411+0000","last_became_peered":"2024-03-06T07:24:30.879411+0000","last_unstale":"2024-03-06T07:24:31.905270+0000","last_undegraded":"2024-03-06T07:24:31.905270+0000","last_fullsized":"2024-03-06T07:24:31.905270+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:16.218338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900878+0000","last_change":"2024-03-06T07:24:31.899661+0000","last_active":"2024-03-06T07:24:31.900878+0000","last_peered":"2024-03-06T07:24:31.900878+0000","last_clean":"2024-03-06T07:24:31.900878+0000","last_became_active":"2024-03-06T07:24:31.899376+0000","last_became_peered":"2024-03-06T07:24:31.899376+0000","last_unstale":"2024-03-06T07:24:31.900878+0000","last_undegraded":"2024-03-06T07:24:31.900878+0000","last_fullsized":"2024-03-06T07:24:31.900878+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:15.391988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880759+0000","last_change":"2024-03-06T07:24:30.875837+0000","last_active":"2024-03-06T07:24:31.880759+0000","last_peered":"2024-03-06T07:24:31.880759+0000","last_clean":"2024-03-06T07:24:31.880759+0000","last_became_active":"2024-03-06T07:24:30.875477+0000","last_became_peered":"2024-03-06T07:24:30.875477+0000","last_unstale":"2024-03-06T07:24:31.880759+0000","last_undegraded":"2024-03-06T07:24:31.880759+0000","last_fullsized":"2024-03-06T07:24:31.880759+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901301+0000","last_change":"2024-03-06T07:24:31.899078+0000","last_active":"2024-03-06T07:24:31.901301+0000","last_peered":"2024-03-06T07:24:31.901301+0000","last_clean":"2024-03-06T07:24:31.901301+0000","last_became_active":"2024-03-06T07:24:31.898559+0000","last_became_peered":"2024-03-06T07:24:31.898559+0000","last_unstale":"2024-03-06T07:24:31.901301+0000","last_undegraded":"2024-03-06T07:24:31.901301+0000","last_fullsized":"2024-03-06T07:24:31.901301+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:48.635017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882935+0000","last_change":"2024-03-06T07:24:30.884381+0000","last_active":"2024-03-06T07:24:31.882935+0000","last_peered":"2024-03-06T07:24:31.882935+0000","last_clean":"2024-03-06T07:24:31.882935+0000","last_became_active":"2024-03-06T07:24:30.884115+0000","last_became_peered":"2024-03-06T07:24:30.884115+0000","last_unstale":"2024-03-06T07:24:31.882935+0000","last_undegraded":"2024-03-06T07:24:31.882935+0000","last_fullsized":"2024-03-06T07:24:31.882935+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:45.687281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898329+0000","last_change":"2024-03-06T07:24:31.893624+0000","last_active":"2024-03-06T07:24:31.898329+0000","last_peered":"2024-03-06T07:24:31.898329+0000","last_clean":"2024-03-06T07:24:31.898329+0000","last_became_active":"2024-03-06T07:24:31.893340+0000","last_became_peered":"2024-03-06T07:24:31.893340+0000","last_unstale":"2024-03-06T07:24:31.898329+0000","last_undegraded":"2024-03-06T07:24:31.898329+0000","last_fullsized":"2024-03-06T07:24:31.898329+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:06.761391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882490+0000","last_change":"2024-03-06T07:24:30.882235+0000","last_active":"2024-03-06T07:24:31.882490+0000","last_peered":"2024-03-06T07:24:31.882490+0000","last_clean":"2024-03-06T07:24:31.882490+0000","last_became_active":"2024-03-06T07:24:30.881955+0000","last_became_peered":"2024-03-06T07:24:30.881955+0000","last_unstale":"2024-03-06T07:24:31.882490+0000","last_undegraded":"2024-03-06T07:24:31.882490+0000","last_fullsized":"2024-03-06T07:24:31.882490+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905533+0000","last_change":"2024-03-06T07:24:31.899401+0000","last_active":"2024-03-06T07:24:31.905533+0000","last_peered":"2024-03-06T07:24:31.905533+0000","last_clean":"2024-03-06T07:24:31.905533+0000","last_became_active":"2024-03-06T07:24:31.899139+0000","last_became_peered":"2024-03-06T07:24:31.899139+0000","last_unstale":"2024-03-06T07:24:31.905533+0000","last_undegraded":"2024-03-06T07:24:31.905533+0000","last_fullsized":"2024-03-06T07:24:31.905533+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:18.216855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880168+0000","last_change":"2024-03-06T07:24:30.882954+0000","last_active":"2024-03-06T07:24:31.880168+0000","last_peered":"2024-03-06T07:24:31.880168+0000","last_clean":"2024-03-06T07:24:31.880168+0000","last_became_active":"2024-03-06T07:24:30.881699+0000","last_became_peered":"2024-03-06T07:24:30.881699+0000","last_unstale":"2024-03-06T07:24:31.880168+0000","last_undegraded":"2024-03-06T07:24:31.880168+0000","last_fullsized":"2024-03-06T07:24:31.880168+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:58.595503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.900361+0000","last_change":"2024-03-06T07:24:31.897819+0000","last_active":"2024-03-06T07:24:31.900361+0000","last_peered":"2024-03-06T07:24:31.900361+0000","last_clean":"2024-03-06T07:24:31.900361+0000","last_became_active":"2024-03-06T07:24:31.897541+0000","last_became_peered":"2024-03-06T07:24:31.897541+0000","last_unstale":"2024-03-06T07:24:31.900361+0000","last_undegraded":"2024-03-06T07:24:31.900361+0000","last_fullsized":"2024-03-06T07:24:31.900361+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:38.280643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"121'3","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.906645+0000","last_change":"2024-03-06T07:24:30.888131+0000","last_active":"2024-03-06T07:24:31.906645+0000","last_peered":"2024-03-06T07:24:31.906645+0000","last_clean":"2024-03-06T07:24:31.906645+0000","last_became_active":"2024-03-06T07:24:30.887837+0000","last_became_peered":"2024-03-06T07:24:30.887837+0000","last_unstale":"2024-03-06T07:24:31.906645+0000","last_undegraded":"2024-03-06T07:24:31.906645+0000","last_fullsized":"2024-03-06T07:24:31.906645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:26.991975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895566+0000","last_change":"2024-03-06T07:24:31.893287+0000","last_active":"2024-03-06T07:24:31.895566+0000","last_peered":"2024-03-06T07:24:31.895566+0000","last_clean":"2024-03-06T07:24:31.895566+0000","last_became_active":"2024-03-06T07:24:31.892943+0000","last_became_peered":"2024-03-06T07:24:31.892943+0000","last_unstale":"2024-03-06T07:24:31.895566+0000","last_undegraded":"2024-03-06T07:24:31.895566+0000","last_fullsized":"2024-03-06T07:24:31.895566+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:01.486412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"121'6","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.911727+0000","last_change":"2024-03-06T07:24:30.882029+0000","last_active":"2024-03-06T07:24:31.911727+0000","last_peered":"2024-03-06T07:24:31.911727+0000","last_clean":"2024-03-06T07:24:31.911727+0000","last_became_active":"2024-03-06T07:24:30.881720+0000","last_became_peered":"2024-03-06T07:24:30.881720+0000","last_unstale":"2024-03-06T07:24:31.911727+0000","last_undegraded":"2024-03-06T07:24:31.911727+0000","last_fullsized":"2024-03-06T07:24:31.911727+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:01:40.701399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889200+0000","last_change":"2024-03-06T07:24:31.885898+0000","last_active":"2024-03-06T07:24:31.889200+0000","last_peered":"2024-03-06T07:24:31.889200+0000","last_clean":"2024-03-06T07:24:31.889200+0000","last_became_active":"2024-03-06T07:24:31.885585+0000","last_became_peered":"2024-03-06T07:24:31.885585+0000","last_unstale":"2024-03-06T07:24:31.889200+0000","last_undegraded":"2024-03-06T07:24:31.889200+0000","last_fullsized":"2024-03-06T07:24:31.889200+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:50:46.188889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907748+0000","last_change":"2024-03-06T07:24:30.881281+0000","last_active":"2024-03-06T07:24:31.907748+0000","last_peered":"2024-03-06T07:24:31.907748+0000","last_clean":"2024-03-06T07:24:31.907748+0000","last_became_active":"2024-03-06T07:24:30.881025+0000","last_became_peered":"2024-03-06T07:24:30.881025+0000","last_unstale":"2024-03-06T07:24:31.907748+0000","last_undegraded":"2024-03-06T07:24:31.907748+0000","last_fullsized":"2024-03-06T07:24:31.907748+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:31.300955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901001+0000","last_change":"2024-03-06T07:24:31.899044+0000","last_active":"2024-03-06T07:24:31.901001+0000","last_peered":"2024-03-06T07:24:31.901001+0000","last_clean":"2024-03-06T07:24:31.901001+0000","last_became_active":"2024-03-06T07:24:31.898707+0000","last_became_peered":"2024-03-06T07:24:31.898707+0000","last_unstale":"2024-03-06T07:24:31.901001+0000","last_undegraded":"2024-03-06T07:24:31.901001+0000","last_fullsized":"2024-03-06T07:24:31.901001+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:12.948471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882272+0000","last_change":"2024-03-06T07:24:30.882286+0000","last_active":"2024-03-06T07:24:31.882272+0000","last_peered":"2024-03-06T07:24:31.882272+0000","last_clean":"2024-03-06T07:24:31.882272+0000","last_became_active":"2024-03-06T07:24:30.882015+0000","last_became_peered":"2024-03-06T07:24:30.882015+0000","last_unstale":"2024-03-06T07:24:31.882272+0000","last_undegraded":"2024-03-06T07:24:31.882272+0000","last_fullsized":"2024-03-06T07:24:31.882272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:17:44.736491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.890080+0000","last_change":"2024-03-06T07:24:31.886626+0000","last_active":"2024-03-06T07:24:31.890080+0000","last_peered":"2024-03-06T07:24:31.890080+0000","last_clean":"2024-03-06T07:24:31.890080+0000","last_became_active":"2024-03-06T07:24:31.886303+0000","last_became_peered":"2024-03-06T07:24:31.886303+0000","last_unstale":"2024-03-06T07:24:31.890080+0000","last_undegraded":"2024-03-06T07:24:31.890080+0000","last_fullsized":"2024-03-06T07:24:31.890080+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:32.274756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"121'8","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.910001+0000","last_change":"2024-03-06T07:24:30.877333+0000","last_active":"2024-03-06T07:24:31.910001+0000","last_peered":"2024-03-06T07:24:31.910001+0000","last_clean":"2024-03-06T07:24:31.910001+0000","last_became_active":"2024-03-06T07:24:30.877063+0000","last_became_peered":"2024-03-06T07:24:30.877063+0000","last_unstale":"2024-03-06T07:24:31.910001+0000","last_undegraded":"2024-03-06T07:24:31.910001+0000","last_fullsized":"2024-03-06T07:24:31.910001+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:00.480778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895158+0000","last_change":"2024-03-06T07:24:31.892187+0000","last_active":"2024-03-06T07:24:31.895158+0000","last_peered":"2024-03-06T07:24:31.895158+0000","last_clean":"2024-03-06T07:24:31.895158+0000","last_became_active":"2024-03-06T07:24:31.891694+0000","last_became_peered":"2024-03-06T07:24:31.891694+0000","last_unstale":"2024-03-06T07:24:31.895158+0000","last_undegraded":"2024-03-06T07:24:31.895158+0000","last_fullsized":"2024-03-06T07:24:31.895158+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:54.374264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.905769+0000","last_change":"2024-03-06T07:24:30.882889+0000","last_active":"2024-03-06T07:24:31.905769+0000","last_peered":"2024-03-06T07:24:31.905769+0000","last_clean":"2024-03-06T07:24:31.905769+0000","last_became_active":"2024-03-06T07:24:30.882639+0000","last_became_peered":"2024-03-06T07:24:30.882639+0000","last_unstale":"2024-03-06T07:24:31.905769+0000","last_undegraded":"2024-03-06T07:24:31.905769+0000","last_fullsized":"2024-03-06T07:24:31.905769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899187+0000","last_change":"2024-03-06T07:24:31.892868+0000","last_active":"2024-03-06T07:24:31.899187+0000","last_peered":"2024-03-06T07:24:31.899187+0000","last_clean":"2024-03-06T07:24:31.899187+0000","last_became_active":"2024-03-06T07:24:31.892598+0000","last_became_peered":"2024-03-06T07:24:31.892598+0000","last_unstale":"2024-03-06T07:24:31.899187+0000","last_undegraded":"2024-03-06T07:24:31.899187+0000","last_fullsized":"2024-03-06T07:24:31.899187+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:18:18.216855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909689+0000","last_change":"2024-03-06T07:24:30.878054+0000","last_active":"2024-03-06T07:24:31.909689+0000","last_peered":"2024-03-06T07:24:31.909689+0000","last_clean":"2024-03-06T07:24:31.909689+0000","last_became_active":"2024-03-06T07:24:30.877505+0000","last_became_peered":"2024-03-06T07:24:30.877505+0000","last_unstale":"2024-03-06T07:24:31.909689+0000","last_undegraded":"2024-03-06T07:24:31.909689+0000","last_fullsized":"2024-03-06T07:24:31.909689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:39:57.664032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909360+0000","last_change":"2024-03-06T07:24:31.908268+0000","last_active":"2024-03-06T07:24:31.909360+0000","last_peered":"2024-03-06T07:24:31.909360+0000","last_clean":"2024-03-06T07:24:31.909360+0000","last_became_active":"2024-03-06T07:24:31.907970+0000","last_became_peered":"2024-03-06T07:24:31.907970+0000","last_unstale":"2024-03-06T07:24:31.909360+0000","last_undegraded":"2024-03-06T07:24:31.909360+0000","last_fullsized":"2024-03-06T07:24:31.909360+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:16.443283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"121'1","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909973+0000","last_change":"2024-03-06T07:24:30.876831+0000","last_active":"2024-03-06T07:24:31.909973+0000","last_peered":"2024-03-06T07:24:31.909973+0000","last_clean":"2024-03-06T07:24:31.909973+0000","last_became_active":"2024-03-06T07:24:30.876530+0000","last_became_peered":"2024-03-06T07:24:30.876530+0000","last_unstale":"2024-03-06T07:24:31.909973+0000","last_undegraded":"2024-03-06T07:24:31.909973+0000","last_fullsized":"2024-03-06T07:24:31.909973+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:37.275009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.894187+0000","last_change":"2024-03-06T07:24:31.889002+0000","last_active":"2024-03-06T07:24:31.894187+0000","last_peered":"2024-03-06T07:24:31.894187+0000","last_clean":"2024-03-06T07:24:31.894187+0000","last_became_active":"2024-03-06T07:24:31.888695+0000","last_became_peered":"2024-03-06T07:24:31.888695+0000","last_unstale":"2024-03-06T07:24:31.894187+0000","last_undegraded":"2024-03-06T07:24:31.894187+0000","last_fullsized":"2024-03-06T07:24:31.894187+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:58.115551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882989+0000","last_change":"2024-03-06T07:24:30.881711+0000","last_active":"2024-03-06T07:24:31.882989+0000","last_peered":"2024-03-06T07:24:31.882989+0000","last_clean":"2024-03-06T07:24:31.882989+0000","last_became_active":"2024-03-06T07:24:30.881415+0000","last_became_peered":"2024-03-06T07:24:30.881415+0000","last_unstale":"2024-03-06T07:24:31.882989+0000","last_undegraded":"2024-03-06T07:24:31.882989+0000","last_fullsized":"2024-03-06T07:24:31.882989+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:11.454919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.891375+0000","last_change":"2024-03-06T07:24:31.890911+0000","last_active":"2024-03-06T07:24:31.891375+0000","last_peered":"2024-03-06T07:24:31.891375+0000","last_clean":"2024-03-06T07:24:31.891375+0000","last_became_active":"2024-03-06T07:24:31.890616+0000","last_became_peered":"2024-03-06T07:24:31.890616+0000","last_unstale":"2024-03-06T07:24:31.891375+0000","last_undegraded":"2024-03-06T07:24:31.891375+0000","last_fullsized":"2024-03-06T07:24:31.891375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:36.659555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.909771+0000","last_change":"2024-03-06T07:24:30.879830+0000","last_active":"2024-03-06T07:24:31.909771+0000","last_peered":"2024-03-06T07:24:31.909771+0000","last_clean":"2024-03-06T07:24:31.909771+0000","last_became_active":"2024-03-06T07:24:30.879527+0000","last_became_peered":"2024-03-06T07:24:30.879527+0000","last_unstale":"2024-03-06T07:24:31.909771+0000","last_undegraded":"2024-03-06T07:24:31.909771+0000","last_fullsized":"2024-03-06T07:24:31.909771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:53.368630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898903+0000","last_change":"2024-03-06T07:24:31.895720+0000","last_active":"2024-03-06T07:24:31.898903+0000","last_peered":"2024-03-06T07:24:31.898903+0000","last_clean":"2024-03-06T07:24:31.898903+0000","last_became_active":"2024-03-06T07:24:31.895455+0000","last_became_peered":"2024-03-06T07:24:31.895455+0000","last_unstale":"2024-03-06T07:24:31.898903+0000","last_undegraded":"2024-03-06T07:24:31.898903+0000","last_fullsized":"2024-03-06T07:24:31.898903+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:21.533172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.881273+0000","last_change":"2024-03-06T07:24:30.886999+0000","last_active":"2024-03-06T07:24:31.881273+0000","last_peered":"2024-03-06T07:24:31.881273+0000","last_clean":"2024-03-06T07:24:31.881273+0000","last_became_active":"2024-03-06T07:24:30.886737+0000","last_became_peered":"2024-03-06T07:24:30.886737+0000","last_unstale":"2024-03-06T07:24:31.881273+0000","last_undegraded":"2024-03-06T07:24:31.881273+0000","last_fullsized":"2024-03-06T07:24:31.881273+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:46.232308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.897440+0000","last_change":"2024-03-06T07:24:31.895426+0000","last_active":"2024-03-06T07:24:31.897440+0000","last_peered":"2024-03-06T07:24:31.897440+0000","last_clean":"2024-03-06T07:24:31.897440+0000","last_became_active":"2024-03-06T07:24:31.895155+0000","last_became_peered":"2024-03-06T07:24:31.895155+0000","last_unstale":"2024-03-06T07:24:31.897440+0000","last_undegraded":"2024-03-06T07:24:31.897440+0000","last_fullsized":"2024-03-06T07:24:31.897440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:28.246685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.887333+0000","last_change":"2024-03-06T07:24:30.887512+0000","last_active":"2024-03-06T07:24:31.887333+0000","last_peered":"2024-03-06T07:24:31.887333+0000","last_clean":"2024-03-06T07:24:31.887333+0000","last_became_active":"2024-03-06T07:24:30.887240+0000","last_became_peered":"2024-03-06T07:24:30.887240+0000","last_unstale":"2024-03-06T07:24:31.887333+0000","last_undegraded":"2024-03-06T07:24:31.887333+0000","last_fullsized":"2024-03-06T07:24:31.887333+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:28.394653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898119+0000","last_change":"2024-03-06T07:24:31.891999+0000","last_active":"2024-03-06T07:24:31.898119+0000","last_peered":"2024-03-06T07:24:31.898119+0000","last_clean":"2024-03-06T07:24:31.898119+0000","last_became_active":"2024-03-06T07:24:31.891704+0000","last_became_peered":"2024-03-06T07:24:31.891704+0000","last_unstale":"2024-03-06T07:24:31.898119+0000","last_undegraded":"2024-03-06T07:24:31.898119+0000","last_fullsized":"2024-03-06T07:24:31.898119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.881693+0000","last_change":"2024-03-06T07:24:30.880756+0000","last_active":"2024-03-06T07:24:31.881693+0000","last_peered":"2024-03-06T07:24:31.881693+0000","last_clean":"2024-03-06T07:24:31.881693+0000","last_became_active":"2024-03-06T07:24:30.880489+0000","last_became_peered":"2024-03-06T07:24:30.880489+0000","last_unstale":"2024-03-06T07:24:31.881693+0000","last_undegraded":"2024-03-06T07:24:31.881693+0000","last_fullsized":"2024-03-06T07:24:31.881693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:35:30.790137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.901765+0000","last_change":"2024-03-06T07:24:31.900531+0000","last_active":"2024-03-06T07:24:31.901765+0000","last_peered":"2024-03-06T07:24:31.901765+0000","last_clean":"2024-03-06T07:24:31.901765+0000","last_became_active":"2024-03-06T07:24:31.900171+0000","last_became_peered":"2024-03-06T07:24:31.900171+0000","last_unstale":"2024-03-06T07:24:31.901765+0000","last_undegraded":"2024-03-06T07:24:31.901765+0000","last_fullsized":"2024-03-06T07:24:31.901765+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:28:31.410346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"121'2","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.907914+0000","last_change":"2024-03-06T07:24:30.880990+0000","last_active":"2024-03-06T07:24:31.907914+0000","last_peered":"2024-03-06T07:24:31.907914+0000","last_clean":"2024-03-06T07:24:31.907914+0000","last_became_active":"2024-03-06T07:24:30.880641+0000","last_became_peered":"2024-03-06T07:24:30.880641+0000","last_unstale":"2024-03-06T07:24:31.907914+0000","last_undegraded":"2024-03-06T07:24:31.907914+0000","last_fullsized":"2024-03-06T07:24:31.907914+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:40:59.207968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895192+0000","last_change":"2024-03-06T07:24:31.891381+0000","last_active":"2024-03-06T07:24:31.895192+0000","last_peered":"2024-03-06T07:24:31.895192+0000","last_clean":"2024-03-06T07:24:31.895192+0000","last_became_active":"2024-03-06T07:24:31.890857+0000","last_became_peered":"2024-03-06T07:24:31.890857+0000","last_unstale":"2024-03-06T07:24:31.895192+0000","last_undegraded":"2024-03-06T07:24:31.895192+0000","last_fullsized":"2024-03-06T07:24:31.895192+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:18:01.708663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879957+0000","last_change":"2024-03-06T07:24:30.882078+0000","last_active":"2024-03-06T07:24:31.879957+0000","last_peered":"2024-03-06T07:24:31.879957+0000","last_clean":"2024-03-06T07:24:31.879957+0000","last_became_active":"2024-03-06T07:24:30.881801+0000","last_became_peered":"2024-03-06T07:24:30.881801+0000","last_unstale":"2024-03-06T07:24:31.879957+0000","last_undegraded":"2024-03-06T07:24:31.879957+0000","last_fullsized":"2024-03-06T07:24:31.879957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:17.001110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899990+0000","last_change":"2024-03-06T07:24:31.898000+0000","last_active":"2024-03-06T07:24:31.899990+0000","last_peered":"2024-03-06T07:24:31.899990+0000","last_clean":"2024-03-06T07:24:31.899990+0000","last_became_active":"2024-03-06T07:24:31.897747+0000","last_became_peered":"2024-03-06T07:24:31.897747+0000","last_unstale":"2024-03-06T07:24:31.899990+0000","last_undegraded":"2024-03-06T07:24:31.899990+0000","last_fullsized":"2024-03-06T07:24:31.899990+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:20:37.175758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879966+0000","last_change":"2024-03-06T07:24:30.884132+0000","last_active":"2024-03-06T07:24:31.879966+0000","last_peered":"2024-03-06T07:24:31.879966+0000","last_clean":"2024-03-06T07:24:31.879966+0000","last_became_active":"2024-03-06T07:24:30.883817+0000","last_became_peered":"2024-03-06T07:24:30.883817+0000","last_unstale":"2024-03-06T07:24:31.879966+0000","last_undegraded":"2024-03-06T07:24:31.879966+0000","last_fullsized":"2024-03-06T07:24:31.879966+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.884375+0000","last_change":"2024-03-06T07:24:31.881762+0000","last_active":"2024-03-06T07:24:31.884375+0000","last_peered":"2024-03-06T07:24:31.884375+0000","last_clean":"2024-03-06T07:24:31.884375+0000","last_became_active":"2024-03-06T07:24:31.881468+0000","last_became_peered":"2024-03-06T07:24:31.881468+0000","last_unstale":"2024-03-06T07:24:31.884375+0000","last_undegraded":"2024-03-06T07:24:31.884375+0000","last_fullsized":"2024-03-06T07:24:31.884375+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:58.833455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882292+0000","last_change":"2024-03-06T07:24:30.878173+0000","last_active":"2024-03-06T07:24:31.882292+0000","last_peered":"2024-03-06T07:24:31.882292+0000","last_clean":"2024-03-06T07:24:31.882292+0000","last_became_active":"2024-03-06T07:24:30.877908+0000","last_became_peered":"2024-03-06T07:24:30.877908+0000","last_unstale":"2024-03-06T07:24:31.882292+0000","last_undegraded":"2024-03-06T07:24:31.882292+0000","last_fullsized":"2024-03-06T07:24:31.882292+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:04.046533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.898595+0000","last_change":"2024-03-06T07:24:31.889510+0000","last_active":"2024-03-06T07:24:31.898595+0000","last_peered":"2024-03-06T07:24:31.898595+0000","last_clean":"2024-03-06T07:24:31.898595+0000","last_became_active":"2024-03-06T07:24:31.888010+0000","last_became_peered":"2024-03-06T07:24:31.888010+0000","last_unstale":"2024-03-06T07:24:31.898595+0000","last_undegraded":"2024-03-06T07:24:31.898595+0000","last_fullsized":"2024-03-06T07:24:31.898595+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:27:29.711780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.882546+0000","last_change":"2024-03-06T07:24:30.881222+0000","last_active":"2024-03-06T07:24:31.882546+0000","last_peered":"2024-03-06T07:24:31.882546+0000","last_clean":"2024-03-06T07:24:31.882546+0000","last_became_active":"2024-03-06T07:24:30.880886+0000","last_became_peered":"2024-03-06T07:24:30.880886+0000","last_unstale":"2024-03-06T07:24:31.882546+0000","last_undegraded":"2024-03-06T07:24:31.882546+0000","last_fullsized":"2024-03-06T07:24:31.882546+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:52:00.480778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889857+0000","last_change":"2024-03-06T07:24:31.888646+0000","last_active":"2024-03-06T07:24:31.889857+0000","last_peered":"2024-03-06T07:24:31.889857+0000","last_clean":"2024-03-06T07:24:31.889857+0000","last_became_active":"2024-03-06T07:24:31.888359+0000","last_became_peered":"2024-03-06T07:24:31.888359+0000","last_unstale":"2024-03-06T07:24:31.889857+0000","last_undegraded":"2024-03-06T07:24:31.889857+0000","last_fullsized":"2024-03-06T07:24:31.889857+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:05.386758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889974+0000","last_change":"2024-03-06T07:24:30.880337+0000","last_active":"2024-03-06T07:24:31.889974+0000","last_peered":"2024-03-06T07:24:31.889974+0000","last_clean":"2024-03-06T07:24:31.889974+0000","last_became_active":"2024-03-06T07:24:30.880068+0000","last_became_peered":"2024-03-06T07:24:30.880068+0000","last_unstale":"2024-03-06T07:24:31.889974+0000","last_undegraded":"2024-03-06T07:24:31.889974+0000","last_fullsized":"2024-03-06T07:24:31.889974+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:49.637038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.904843+0000","last_change":"2024-03-06T07:24:31.894727+0000","last_active":"2024-03-06T07:24:31.904843+0000","last_peered":"2024-03-06T07:24:31.904843+0000","last_clean":"2024-03-06T07:24:31.904843+0000","last_became_active":"2024-03-06T07:24:31.894429+0000","last_became_peered":"2024-03-06T07:24:31.894429+0000","last_unstale":"2024-03-06T07:24:31.904843+0000","last_undegraded":"2024-03-06T07:24:31.904843+0000","last_fullsized":"2024-03-06T07:24:31.904843+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.972365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.880820+0000","last_change":"2024-03-06T07:24:30.879025+0000","last_active":"2024-03-06T07:24:31.880820+0000","last_peered":"2024-03-06T07:24:31.880820+0000","last_clean":"2024-03-06T07:24:31.880820+0000","last_became_active":"2024-03-06T07:24:30.877933+0000","last_became_peered":"2024-03-06T07:24:30.877933+0000","last_unstale":"2024-03-06T07:24:31.880820+0000","last_undegraded":"2024-03-06T07:24:31.880820+0000","last_fullsized":"2024-03-06T07:24:31.880820+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:48:05.478554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.899949+0000","last_change":"2024-03-06T07:24:31.896772+0000","last_active":"2024-03-06T07:24:31.899949+0000","last_peered":"2024-03-06T07:24:31.899949+0000","last_clean":"2024-03-06T07:24:31.899949+0000","last_became_active":"2024-03-06T07:24:31.896491+0000","last_became_peered":"2024-03-06T07:24:31.896491+0000","last_unstale":"2024-03-06T07:24:31.899949+0000","last_undegraded":"2024-03-06T07:24:31.899949+0000","last_fullsized":"2024-03-06T07:24:31.899949+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:27:44.110497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.879885+0000","last_change":"2024-03-06T07:24:30.881511+0000","last_active":"2024-03-06T07:24:31.879885+0000","last_peered":"2024-03-06T07:24:31.879885+0000","last_clean":"2024-03-06T07:24:31.879885+0000","last_became_active":"2024-03-06T07:24:30.881203+0000","last_became_peered":"2024-03-06T07:24:30.881203+0000","last_unstale":"2024-03-06T07:24:31.879885+0000","last_undegraded":"2024-03-06T07:24:31.879885+0000","last_fullsized":"2024-03-06T07:24:31.879885+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:15.437648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.895570+0000","last_change":"2024-03-06T07:24:31.892700+0000","last_active":"2024-03-06T07:24:31.895570+0000","last_peered":"2024-03-06T07:24:31.895570+0000","last_clean":"2024-03-06T07:24:31.895570+0000","last_became_active":"2024-03-06T07:24:31.892440+0000","last_became_peered":"2024-03-06T07:24:31.892440+0000","last_unstale":"2024-03-06T07:24:31.895570+0000","last_undegraded":"2024-03-06T07:24:31.895570+0000","last_fullsized":"2024-03-06T07:24:31.895570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:30.861421+0000","last_clean_scrub_stamp":"2024-03-06T07:24:30.861421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:14.966060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.889268+0000","last_change":"2024-03-06T07:24:30.887838+0000","last_active":"2024-03-06T07:24:31.889268+0000","last_peered":"2024-03-06T07:24:31.889268+0000","last_clean":"2024-03-06T07:24:31.889268+0000","last_became_active":"2024-03-06T07:24:30.887554+0000","last_became_peered":"2024-03-06T07:24:30.887554+0000","last_unstale":"2024-03-06T07:24:31.889268+0000","last_undegraded":"2024-03-06T07:24:31.889268+0000","last_fullsized":"2024-03-06T07:24:31.889268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:24:29.855787+0000","last_clean_scrub_stamp":"2024-03-06T07:24:29.855787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:05:30.643410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":357,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-06T07:24:31.886719+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:24:31.886719+0000","last_peered":"2024-03-06T07:24:31.886719+0000","last_clean":"2024-03-06T07:24:31.886719+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:24:31.886719+0000","last_undegraded":"2024-03-06T07:24:31.886719+0000","last_fullsized":"2024-03-06T07:24:31.886719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":1315516,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":725148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":732385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1589248,"data_stored":1322739,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:24:35.306 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:24:35.306 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:24:35.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.452+0000 7fa7748b5640 1 Processor -- start 2024-03-06T07:24:35.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.452+0000 7fa7748b5640 1 -- start start 2024-03-06T07:24:35.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.453+0000 7fa7748b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.453+0000 7fa7748b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7700a97c0 0x7fa7701499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.453+0000 7fa7748b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa770154330 con 0x7fa7700ab310 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.453+0000 7fa7748b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700c46a0 con 0x7fa7700a97c0 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.453+0000 7fa7748b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa770153dc0 con 0x7fa7701538c0 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa76ed9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7700a97c0 0x7fa7701499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa76e59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa7700ab310 0x7fa7700ab6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40612/0 (socket says 172.21.15.17:40612) 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:57990/0 (socket says 172.21.15.17:57990) 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa76e59c640 1 -- 172.21.15.17:0/1107536589 learned_addr learned my addr 172.21.15.17:0/1107536589 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 -- 172.21.15.17:0/1107536589 >> v2:172.21.15.163:3300/0 conn(0x7fa7700a97c0 msgr2=0x7fa7701499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 --2- 172.21.15.17:0/1107536589 >> v2:172.21.15.163:3300/0 conn(0x7fa7700a97c0 0x7fa7701499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 -- 172.21.15.17:0/1107536589 >> v1:172.21.15.163:6789/0 conn(0x7fa7700ab310 legacy=0x7fa7700ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1107536589 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594189797 0 0) 0x7fa770154330 con 0x7fa7700ab310 2024-03-06T07:24:35.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.454+0000 7fa765d9b640 1 -- 172.21.15.17:0/1107536589 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700c5d30 con 0x7fa7701538c0 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa765d9b640 1 --2- 172.21.15.17:0/1107536589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097a0 tx=0x7fa75802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b8314e1479d27c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1107536589 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580315a0 con 0x7fa7701538c0 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1107536589 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa758037070 con 0x7fa7701538c0 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1107536589 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758031960 con 0x7fa7701538c0 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 -- 172.21.15.17:0/1107536589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 msgr2=0x7fa7701494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 --2- 172.21.15.17:0/1107536589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097a0 tx=0x7fa75802b840 comp rx=0 tx=0).stop 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 -- 172.21.15.17:0/1107536589 shutdown_connections 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 --2- 172.21.15.17:0/1107536589 >> v2:172.21.15.163:3300/0 conn(0x7fa7700a97c0 0x7fa7701499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 --2- 172.21.15.17:0/1107536589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa7701494a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.455+0000 7fa7748b5640 1 -- 172.21.15.17:0/1107536589 >> 172.21.15.17:0/1107536589 conn(0x7fa7700c6710 msgr2=0x7fa7700c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.456+0000 7fa7748b5640 1 -- 172.21.15.17:0/1107536589 shutdown_connections 2024-03-06T07:24:35.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.456+0000 7fa7748b5640 1 -- 172.21.15.17:0/1107536589 wait complete. 2024-03-06T07:24:35.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.457+0000 7fa7748b5640 1 Processor -- start 2024-03-06T07:24:35.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.457+0000 7fa7748b5640 1 -- start start 2024-03-06T07:24:35.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7700a97c0 0x7fa7701ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa770141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c4b30 con 0x7fa7700ab310 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c62f0 con 0x7fa7700a97c0 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7748b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c4cb0 con 0x7fa7701538c0 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57802/0 (socket says 172.21.15.17:57802) 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 -- 172.21.15.17:0/3522949430 learned_addr learned my addr 172.21.15.17:0/3522949430 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa76e59c640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7700a97c0 0x7fa7701ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa76ed9d640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa770141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 -- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 msgr2=0x7fa770141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa770141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 -- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7700a97c0 msgr2=0x7fa7701ebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7700a97c0 0x7fa7701ebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 -- 172.21.15.17:0/3522949430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770144690 con 0x7fa7700ab310 2024-03-06T07:24:35.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa765d9b640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c00 tx=0x7fa7580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.458+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758004230 con 0x7fa7700ab310 2024-03-06T07:24:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.459+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580043f0 con 0x7fa7700ab310 2024-03-06T07:24:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.459+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758003850 con 0x7fa7700ab310 2024-03-06T07:24:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.459+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa770070e80 con 0x7fa7700ab310 2024-03-06T07:24:35.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.459+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa770142e60 con 0x7fa7700ab310 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.460+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77004f330 con 0x7fa7700ab310 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.460+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7580039f0 con 0x7fa7700ab310 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.460+0000 7fa7677fe640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 0x7fa75006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.460+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa75802f030 con 0x7fa7700ab310 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.461+0000 7fa76e59c640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 0x7fa75006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.461+0000 7fa76e59c640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 0x7fa75006d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa760007f70 tx=0x7fa760007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.465+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa758041070 con 0x7fa7700ab310 2024-03-06T07:24:35.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.712+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa7700c5a70 con 0x7fa7700ab310 2024-03-06T07:24:35.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.714+0000 7fa7677fe640 1 -- 172.21.15.17:0/3522949430 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa75807a7d0 con 0x7fa7700ab310 2024-03-06T07:24:35.715 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:24:35.716 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:24:35.716 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:24:35.716 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:24:35.716 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:24:35.716 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 msgr2=0x7fa75006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 0x7fa75006d9c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa760007f70 tx=0x7fa760007f00 comp rx=0 tx=0).stop 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 msgr2=0x7fa7701ec420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa758000c00 tx=0x7fa7580046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 shutdown_connections 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa75006b500 0x7fa75006d9c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7701538c0 0x7fa770141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7700ab310 0x7fa7701ec420 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 --2- 172.21.15.17:0/3522949430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7700a97c0 0x7fa7701ebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 >> 172.21.15.17:0/3522949430 conn(0x7fa7700c6710 msgr2=0x7fa7701546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 shutdown_connections 2024-03-06T07:24:35.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.716+0000 7fa7748b5640 1 -- 172.21.15.17:0/3522949430 wait complete. 2024-03-06T07:24:35.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.829+0000 7fd6ebf9a640 1 Processor -- start 2024-03-06T07:24:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.829+0000 7fd6ebf9a640 1 -- start start 2024-03-06T07:24:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ebf9a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6e4153990 0x7fd6e4156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ebf9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ebf9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e4070aa0 con 0x7fd6e41506b0 2024-03-06T07:24:35.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ebf9a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e4156450 con 0x7fd6e4153990 2024-03-06T07:24:35.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ebf9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4156a10 con 0x7fd6e40acc00 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ea537640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ea537640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:58010/0 (socket says 172.21.15.17:58010) 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6ea537640 1 -- 172.21.15.17:0/1730184650 learned_addr learned my addr 172.21.15.17:0/1730184650 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.830+0000 7fd6e9535640 1 --2- 172.21.15.17:0/1730184650 >> v2:172.21.15.163:3300/0 conn(0x7fd6e4153990 0x7fd6e4156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.831+0000 7fd6ea537640 1 -- 172.21.15.17:0/1730184650 >> v2:172.21.15.163:3300/0 conn(0x7fd6e4153990 msgr2=0x7fd6e4156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.831+0000 7fd6ea537640 1 --2- 172.21.15.17:0/1730184650 >> v2:172.21.15.163:3300/0 conn(0x7fd6e4153990 0x7fd6e4156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.831+0000 7fd6ea537640 1 -- 172.21.15.17:0/1730184650 >> v1:172.21.15.163:6789/0 conn(0x7fd6e41506b0 legacy=0x7fd6e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.831+0000 7fd6ea537640 1 -- 172.21.15.17:0/1730184650 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4070d80 con 0x7fd6e40acc00 2024-03-06T07:24:35.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ea537640 1 --2- 172.21.15.17:0/1730184650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54436e2cdd1d456 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6e8d34640 1 -- 172.21.15.17:0/1730184650 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d402cb00 con 0x7fd6e40acc00 2024-03-06T07:24:35.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6e8d34640 1 -- 172.21.15.17:0/1730184650 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d402cca0 con 0x7fd6e40acc00 2024-03-06T07:24:35.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6e8d34640 1 -- 172.21.15.17:0/1730184650 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4034610 con 0x7fd6e40acc00 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/1730184650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 msgr2=0x7fd6e40c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/1730184650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d402bf60 comp rx=0 tx=0).stop 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/1730184650 shutdown_connections 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/1730184650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e40acc00 0x7fd6e40c54a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/1730184650 >> v2:172.21.15.163:3300/0 conn(0x7fd6e4153990 0x7fd6e4156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/1730184650 >> 172.21.15.17:0/1730184650 conn(0x7fd6e406b1a0 msgr2=0x7fd6e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/1730184650 shutdown_connections 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.832+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/1730184650 wait complete. 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.833+0000 7fd6ebf9a640 1 Processor -- start 2024-03-06T07:24:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 -- start start 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e41506b0 0x7fd6e40c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e4153990 0x7fd6e414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40ad1d0 con 0x7fd6e40acc00 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40ad050 con 0x7fd6e41506b0 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ebf9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40ad350 con 0x7fd6e4153990 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6e9d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6e9d36640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57806/0 (socket says 172.21.15.17:57806) 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6e9d36640 1 -- 172.21.15.17:0/2140692210 learned_addr learned my addr 172.21.15.17:0/2140692210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6ea537640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e4153990 0x7fd6e414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.834+0000 7fd6e9535640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e41506b0 0x7fd6e40c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 -- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e4153990 msgr2=0x7fd6e414aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e4153990 0x7fd6e414aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 -- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e41506b0 msgr2=0x7fd6e40c1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e41506b0 0x7fd6e40c1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 -- 172.21.15.17:0/2140692210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e40c2770 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e9d36640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c460 tx=0x7fd6dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0052d0 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e41ee300 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc012630 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0054b0 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.835+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e40c22a0 con 0x7fd6e40acc00 2024-03-06T07:24:35.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.836+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e41ee910 con 0x7fd6e40acc00 2024-03-06T07:24:35.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.837+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6dc012a20 con 0x7fd6e40acc00 2024-03-06T07:24:35.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.837+0000 7fd6e27fc640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:35.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.838+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6dc089de0 con 0x7fd6e40acc00 2024-03-06T07:24:35.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.838+0000 7fd6e9535640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:35.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.838+0000 7fd6e9535640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 0x7fd6d006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6e41ebe20 tx=0x7fd6d8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:35.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:35.842+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6dc058490 con 0x7fd6e40acc00 2024-03-06T07:24:36.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.061+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6e404f330 con 0x7fd6e40acc00 2024-03-06T07:24:36.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.061+0000 7fd6e27fc640 1 -- 172.21.15.17:0/2140692210 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fd6dc05c530 con 0x7fd6e40acc00 2024-03-06T07:24:36.063 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:36.063 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:33.240915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/410228237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":410228237},{"type":"v1","addr":"172.21.15.17:6835","nonce":410228237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":9}]} 2024-03-06T07:24:36.063 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 105 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 msgr2=0x7fd6d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 0x7fd6d006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6e41ebe20 tx=0x7fd6d8009190 comp rx=0 tx=0).stop 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 msgr2=0x7fd6e41eede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c460 tx=0x7fd6dc00c920 comp rx=0 tx=0).stop 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 shutdown_connections 2024-03-06T07:24:36.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6e4153990 0x7fd6e414aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6e41506b0 0x7fd6e40c1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 --2- 172.21.15.17:0/2140692210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6e40acc00 0x7fd6e41eede0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 >> 172.21.15.17:0/2140692210 conn(0x7fd6e406b1a0 msgr2=0x7fd6e40c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 shutdown_connections 2024-03-06T07:24:36.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.063+0000 7fd6ebf9a640 1 -- 172.21.15.17:0/2140692210 wait complete. 2024-03-06T07:24:36.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:33.240915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7848': {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7848, 'qdb_cluster': [7848]} 2024-03-06T07:24:36.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:36.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.584+0000 7fd4b2276640 1 Processor -- start 2024-03-06T07:24:36.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.584+0000 7fd4b2276640 1 -- start start 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.584+0000 7fd4b2276640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4ac0c30b0 0x7fd4ac14f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.584+0000 7fd4b2276640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 0x7fd4ac155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4b2276640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac152030 con 0x7fd4ac0c4bb0 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4b2276640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac152920 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4b2276640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac152db0 con 0x7fd4ac14f660 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4ab7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4ac0c30b0 0x7fd4ac14f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4abfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd4ac0c4bb0 0x7fd4ac0c3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44582/0 (socket says 172.21.15.17:44582) 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4abfff640 1 -- 172.21.15.17:0/36942695 learned_addr learned my addr 172.21.15.17:0/36942695 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/36942695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 msgr2=0x7fd4ac155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4b0813640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 0x7fd4ac155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 0x7fd4ac155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4aaffd640 1 -- 172.21.15.17:0/36942695 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179895484 0 0) 0x7fd4ac152030 con 0x7fd4ac0c4bb0 2024-03-06T07:24:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/36942695 >> v1:172.21.15.17:6789/0 conn(0x7fd4ac0c4bb0 legacy=0x7fd4ac0c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/36942695 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac1522d0 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.585+0000 7fd4b0813640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 0x7fd4ac155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac14f120 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00097a0 tx=0x7fd4a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8489273afebadb8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4aaffd640 1 -- 172.21.15.17:0/36942695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a00315a0 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4aaffd640 1 -- 172.21.15.17:0/36942695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0037070 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4aaffd640 1 -- 172.21.15.17:0/36942695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031960 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 -- 172.21.15.17:0/36942695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac0c30b0 msgr2=0x7fd4ac14f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac14f120 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00097a0 tx=0x7fd4a002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 -- 172.21.15.17:0/36942695 shutdown_connections 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac14f660 0x7fd4ac155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 --2- 172.21.15.17:0/36942695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac14f120 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.586+0000 7fd4b2276640 1 -- 172.21.15.17:0/36942695 >> 172.21.15.17:0/36942695 conn(0x7fd4ac0c7de0 msgr2=0x7fd4ac0c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.587+0000 7fd4b2276640 1 -- 172.21.15.17:0/36942695 shutdown_connections 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.587+0000 7fd4b2276640 1 -- 172.21.15.17:0/36942695 wait complete. 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 Processor -- start 2024-03-06T07:24:36.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 -- start start 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac0aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac0c4bb0 0x7fd4ac1ec090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c4f90 con 0x7fd4ac14f660 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c3fd0 con 0x7fd4ac0c30b0 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.588+0000 7fd4b2276640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c5110 con 0x7fd4ac0c4bb0 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57832/0 (socket says 172.21.15.17:57832) 2024-03-06T07:24:36.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 -- 172.21.15.17:0/3059296427 learned_addr learned my addr 172.21.15.17:0/3059296427 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac0c4bb0 0x7fd4ac1ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac0aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 -- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac0c4bb0 msgr2=0x7fd4ac1ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac0c4bb0 0x7fd4ac1ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 -- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac0c30b0 msgr2=0x7fd4ac0aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac0aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 -- 172.21.15.17:0/3059296427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac150fb0 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b0813640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400cd80 tx=0x7fd4a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4a8ff9640 1 -- 172.21.15.17:0/3059296427 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400f680 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4a8ff9640 1 -- 172.21.15.17:0/3059296427 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a400f840 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac154740 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4a8ff9640 1 -- 172.21.15.17:0/3059296427 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400fc30 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.589+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac0ab4f0 con 0x7fd4ac14f660 2024-03-06T07:24:36.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.591+0000 7fd4a8ff9640 1 -- 172.21.15.17:0/3059296427 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4a40213f0 con 0x7fd4ac14f660 2024-03-06T07:24:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.591+0000 7fd4a8ff9640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 0x7fd49806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.591+0000 7fd4a8ff9640 1 -- 172.21.15.17:0/3059296427 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4a408af40 con 0x7fd4ac14f660 2024-03-06T07:24:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.591+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 0x7fd49806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.592+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 0x7fd49806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000f60 tx=0x7fd49c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.592+0000 7fd4b4e5e740 1 Processor -- start 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.592+0000 7fd4b4e5e740 1 -- start start 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c78330 0x560684c75f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c764b0 0x560684c72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684bdc6b0 con 0x560684c77f50 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684b85a60 con 0x560684c78330 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560684bd8750 con 0x560684c764b0 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57844/0 (socket says 172.21.15.17:57844) 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 -- 172.21.15.17:0/482216756 learned_addr learned my addr 172.21.15.17:0/482216756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c78330 0x560684c75f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4b0813640 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c764b0 0x560684c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 -- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c764b0 msgr2=0x560684c72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c764b0 0x560684c72680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 -- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c78330 msgr2=0x560684c75f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c78330 0x560684c75f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 -- 172.21.15.17:0/482216756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560684ba3f30 con 0x560684c77f50 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd4abfff640 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007aa0 tx=0x7fd49c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3651bcec874e5ce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd491ffb640 1 -- 172.21.15.17:0/482216756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c026070 con 0x560684c77f50 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.593+0000 7fd491ffb640 1 -- 172.21.15.17:0/482216756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c013b80 con 0x560684c77f50 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd491ffb640 1 -- 172.21.15.17:0/482216756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c012350 con 0x560684c77f50 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/482216756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 msgr2=0x560684c75a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007aa0 tx=0x7fd49c013000 comp rx=0 tx=0).stop 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/482216756 shutdown_connections 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c764b0 0x560684c72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c78330 0x560684c75f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/482216756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684c75a30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/482216756 >> 172.21.15.17:0/482216756 conn(0x560684c747d0 msgr2=0x560684c993c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/482216756 shutdown_connections 2024-03-06T07:24:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.594+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/482216756 wait complete. 2024-03-06T07:24:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.595+0000 7fd4b4e5e740 1 -- start start 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.595+0000 7fd4b4e5e740 1 Processor -- start 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.596+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cc7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684cb08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684c9a230 con 0x560684c78330 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684b98ea0 con 0x560684c77f50 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560684c99c70 con 0x560684cc5640 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 10 client.0 _ll_register_callbacks cb 0x560684b98870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57860/0 (socket says 172.21.15.17:57860) 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/1137346855 learned_addr learned my addr 172.21.15.17:0/1137346855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4abfff640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cc7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b0813640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684cb08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 msgr2=0x560684cb08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684cb08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 msgr2=0x560684cc7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cc7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/1137346855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560684b99270 con 0x560684c78330 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098c0 tx=0x7fd4a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29a1d45dd8ac129 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd48affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0003cd0 con 0x560684c78330 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd4a00315a0 con 0x560684c78330 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.597+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd49c013b80 con 0x560684c78330 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4a0053050 con 0x560684c78330 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 10 client.7986 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:36.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5606848f12d0 con 0x560684c78330 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c013b80 con 0x560684c78330 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd4a0056020 con 0x560684c78330 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 10 client.7986 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 10 client.7986 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 20 client.7986 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c9afb0 0x560684cd35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 4 client.7986 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560684be9bf0 con 0x560684c9afb0 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4abfff640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c9afb0 0x560684cd35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd4abfff640 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c9afb0 0x560684cd35a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6aaacc0c3f9805 server_cookie=a1eabdde6d062f72 in_seq=0 out_seq=0 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.598+0000 7fd48affd640 10 client.7986 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd48affd640 1 -- 172.21.15.17:0/1137346855 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560684be9bf0 con 0x560684c9afb0 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd48affd640 10 client.7986 handle_command_reply: tid=0 2024-03-06T07:24:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 client.7986 shutdown 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 msgr2=0x560684c98a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00098c0 tx=0x7fd4a0004730 comp rx=0 tx=0).stop 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 shutdown_connections 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c9afb0 0x560684cd35a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684cb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c98a60 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/1137346855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cc7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd48affd640 0 client.7986 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.599+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 >> 172.21.15.17:0/1137346855 conn(0x560684c764b0 msgr2=0x560684cb4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.600+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 shutdown_connections 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.600+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/1137346855 wait complete. 2024-03-06T07:24:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.600+0000 7fd4b4e5e740 20 client.7986 trim_cache size 0 max 16384 2024-03-06T07:24:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.616+0000 7fd4b4e5e740 1 Processor -- start 2024-03-06T07:24:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.616+0000 7fd4b4e5e740 1 -- start start 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cb03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684c972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1f4a30 con 0x560684c78330 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac153c20 con 0x560684c77f50 2024-03-06T07:24:36.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1f45d0 con 0x560684cc5640 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57862/0 (socket says 172.21.15.17:57862) 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/662653526 learned_addr learned my addr 172.21.15.17:0/662653526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 msgr2=0x560684c972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684c972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 msgr2=0x560684cb03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cb03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.617+0000 7fd4ab7fe640 1 -- 172.21.15.17:0/662653526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606849bb6b0 con 0x560684c78330 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001eb0 tx=0x7fd4a0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdcb87ee8381d1c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4917fa640 1 -- 172.21.15.17:0/662653526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0003ad0 con 0x560684c78330 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4917fa640 1 -- 172.21.15.17:0/662653526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a004b070 con 0x560684c78330 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4917fa640 1 -- 172.21.15.17:0/662653526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0031c40 con 0x560684c78330 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/662653526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 msgr2=0x560684c96d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001eb0 tx=0x7fd4a0044000 comp rx=0 tx=0).stop 2024-03-06T07:24:36.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/662653526 shutdown_connections 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684cc5640 0x560684c972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c78330 0x560684c96d60 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/662653526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684c77f50 0x560684cb03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/662653526 >> 172.21.15.17:0/662653526 conn(0x560684c764b0 msgr2=0x560684c9b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/662653526 shutdown_connections 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.618+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/662653526 wait complete. 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.619+0000 7fd4b4e5e740 1 -- start start 2024-03-06T07:24:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.619+0000 7fd4b4e5e740 1 Processor -- start 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 0x560684d28050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684cc5640 0x560684d1e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684cb12f0 con 0x560684c77f50 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560684cc4e40 con 0x560684cc5640 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560684c9bdf0 con 0x560684c78330 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b4e5e740 10 client.0 _ll_register_callbacks cb 0x7fd4a00044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57874/0 (socket says 172.21.15.17:57874) 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 -- 172.21.15.17:0/3233453643 learned_addr learned my addr 172.21.15.17:0/3233453643 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 0x560684d28050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4b0813640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684cc5640 0x560684d1e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 -- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 msgr2=0x560684d28050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 0x560684d28050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 -- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684cc5640 msgr2=0x560684d1e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684cc5640 0x560684d1e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 -- 172.21.15.17:0/3233453643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560684cc1190 con 0x560684c77f50 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 0x560684d28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd4abfff640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007a70 tx=0x7fd49c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af9b4773388c55f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.620+0000 7fd48affd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c037060 con 0x560684c77f50 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd49c031830 con 0x560684c77f50 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4a004b070 con 0x560684c77f50 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd49c03d050 con 0x560684c77f50 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 10 client.7992 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:36.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x560684cd7c70 con 0x560684c77f50 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a004b070 con 0x560684c77f50 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd49c040020 con 0x560684c77f50 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 10 client.7992 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 10 client.7992 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 20 client.7992 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c75370 0x560684c75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 4 client.7992 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560684be3f40 con 0x560684c75370 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c75370 0x560684c75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd4ab7fe640 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c75370 0x560684c75750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2968d65a9e39acf server_cookie=6e75333a671977f5 in_seq=0 out_seq=0 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.621+0000 7fd48affd640 10 client.7992 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd48affd640 1 -- 172.21.15.17:0/3233453643 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x560684be3f40 con 0x560684c75370 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd48affd640 10 client.7992 handle_command_reply: tid=0 2024-03-06T07:24:36.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 client.7992 shutdown 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 msgr2=0x560684cb07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007a70 tx=0x7fd49c001080 comp rx=0 tx=0).stop 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 shutdown_connections 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560684c75370 0x560684c75750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560684cc5640 0x560684d1e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd48affd640 0 client.7992 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560684c78330 0x560684d28050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 --2- 172.21.15.17:0/3233453643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560684c77f50 0x560684cb07f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 >> 172.21.15.17:0/3233453643 conn(0x560684c764b0 msgr2=0x560684c76ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 shutdown_connections 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 1 -- 172.21.15.17:0/3233453643 wait complete. 2024-03-06T07:24:36.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b4e5e740 20 client.7992 trim_cache size 0 max 16384 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7848, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.7346920319999999, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 105, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 122, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 122, 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9.8262000950000008 2024-03-06T07:24:36.630 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 msgr2=0x7fd49806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 0x7fd49806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd49c000f60 tx=0x7fd49c007d50 comp rx=0 tx=0).stop 2024-03-06T07:24:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.622+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 msgr2=0x7fd4ac1ec5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400cd80 tx=0x7fd4a400a650 comp rx=0 tx=0).stop 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 shutdown_connections 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd49806b4d0 0x7fd49806d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4ac14f660 0x7fd4ac1ec5d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4ac0c4bb0 0x7fd4ac1ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 --2- 172.21.15.17:0/3059296427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4ac0c30b0 0x7fd4ac0aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 >> 172.21.15.17:0/3059296427 conn(0x7fd4ac0c7de0 msgr2=0x7fd4ac072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 shutdown_connections 2024-03-06T07:24:36.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.623+0000 7fd4b2276640 1 -- 172.21.15.17:0/3059296427 wait complete. 2024-03-06T07:24:36.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.736+0000 7f6af5b5e640 1 Processor -- start 2024-03-06T07:24:36.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.736+0000 7f6af5b5e640 1 -- start start 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6af5b5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 0x7f6af014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6af5b5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6af014f140 0x7f6af00c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6af5b5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af014f5d0 con 0x7f6af01550c0 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6af5b5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00c4610 con 0x7f6af014f140 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6af5b5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af0155bb0 con 0x7f6af00aa180 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6af014f140 0x7f6af00c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aef7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6af01550c0 0x7f6af01554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44590/0 (socket says 172.21.15.17:44590) 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aef7fe640 1 -- 172.21.15.17:0/3590597093 learned_addr learned my addr 172.21.15.17:0/3590597093 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeeffd640 1 --2- 172.21.15.17:0/3590597093 >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 0x7f6af014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeffff640 1 -- 172.21.15.17:0/3590597093 >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 msgr2=0x7f6af014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeffff640 1 --2- 172.21.15.17:0/3590597093 >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 0x7f6af014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeffff640 1 -- 172.21.15.17:0/3590597093 >> v1:172.21.15.17:6789/0 conn(0x7f6af01550c0 legacy=0x7f6af01554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeffff640 1 -- 172.21.15.17:0/3590597093 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00aa660 con 0x7f6af014f140 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aee7fc640 1 -- 172.21.15.17:0/3590597093 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016566207 0 0) 0x7f6af014f5d0 con 0x7f6af01550c0 2024-03-06T07:24:36.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.737+0000 7f6aeeffd640 1 --2- 172.21.15.17:0/3590597093 >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 0x7f6af014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.738+0000 7f6aeffff640 1 --2- 172.21.15.17:0/3590597093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af014f140 0x7f6af00c5060 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6af00abe90 tx=0x7f6ae802cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2081ecb4e1af6e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6aee7fc640 1 -- 172.21.15.17:0/3590597093 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802da40 con 0x7f6af014f140 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6aee7fc640 1 -- 172.21.15.17:0/3590597093 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae802dbe0 con 0x7f6af014f140 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6aee7fc640 1 -- 172.21.15.17:0/3590597093 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8008440 con 0x7f6af014f140 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 -- 172.21.15.17:0/3590597093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af014f140 msgr2=0x7f6af00c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/3590597093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af014f140 0x7f6af00c5060 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6af00abe90 tx=0x7f6ae802cea0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 -- 172.21.15.17:0/3590597093 shutdown_connections 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/3590597093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af014f140 0x7f6af00c5060 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/3590597093 >> v2:172.21.15.163:3300/0 conn(0x7f6af00aa180 0x7f6af014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 -- 172.21.15.17:0/3590597093 >> 172.21.15.17:0/3590597093 conn(0x7f6af00c6710 msgr2=0x7f6af00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 -- 172.21.15.17:0/3590597093 shutdown_connections 2024-03-06T07:24:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.739+0000 7f6af5b5e640 1 -- 172.21.15.17:0/3590597093 wait complete. 2024-03-06T07:24:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.740+0000 7f6af5b5e640 1 Processor -- start 2024-03-06T07:24:36.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.740+0000 7f6af5b5e640 1 -- start start 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 0x7f6af00bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 0x7f6af0149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c1310 con 0x7f6af01550c0 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c4860 con 0x7f6af014f140 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6af5b5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c1490 con 0x7f6af00aa180 2024-03-06T07:24:36.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57896/0 (socket says 172.21.15.17:57896) 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aeffff640 1 -- 172.21.15.17:0/84991893 learned_addr learned my addr 172.21.15.17:0/84991893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aeeffd640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 0x7f6af0149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aef7fe640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 0x7f6af00bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.741+0000 7f6aeffff640 1 -- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 msgr2=0x7f6af00bfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeffff640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 0x7f6af00bfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeffff640 1 -- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 msgr2=0x7f6af0149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeffff640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 0x7f6af0149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeffff640 1 -- 172.21.15.17:0/84991893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af006c7f0 con 0x7f6af01550c0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aef7fe640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 0x7f6af00bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeeffd640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 0x7f6af0149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6aeffff640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800af90 tx=0x7f6ae8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8044020 con 0x7f6af01550c0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8040670 con 0x7f6af01550c0 2024-03-06T07:24:36.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af006cad0 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800a560 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.742+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af006d090 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.743+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af00c4300 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.743+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae8047020 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.744+0000 7f6ad7fff640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 0x7f6adc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.744+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ae80ad540 con 0x7f6af01550c0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.744+0000 7f6aef7fe640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 0x7f6adc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:36.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.745+0000 7f6aef7fe640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 0x7f6adc06d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0000f60 tx=0x7f6ae0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:36.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.752+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6af00c4300 con 0x7f6af01550c0 2024-03-06T07:24:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.997+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af004f330 con 0x7f6af01550c0 2024-03-06T07:24:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:36.997+0000 7f6ad7fff640 1 -- 172.21.15.17:0/84991893 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f6ae804e020 con 0x7f6af01550c0 2024-03-06T07:24:36.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:36.999 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:33.240915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/410228237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":410228237},{"type":"v1","addr":"172.21.15.17:6835","nonce":410228237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":9}]} 2024-03-06T07:24:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 105 2024-03-06T07:24:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 msgr2=0x7f6adc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 0x7f6adc06d9c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0000f60 tx=0x7f6ae0009040 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 msgr2=0x7f6af0149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800af90 tx=0x7f6ae8004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 shutdown_connections 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6adc06b500 0x7f6adc06d9c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af01550c0 0x7f6af0149ec0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af014f140 0x7f6af0149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 --2- 172.21.15.17:0/84991893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af00aa180 0x7f6af00bfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 >> 172.21.15.17:0/84991893 conn(0x7f6af00c6710 msgr2=0x7f6af014bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 shutdown_connections 2024-03-06T07:24:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.000+0000 7f6af5b5e640 1 -- 172.21.15.17:0/84991893 wait complete. 2024-03-06T07:24:37.013 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:31.239352+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7848": { "addr": "172.21.15.17:6835/410228237", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6834", "nonce": 410228237, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 410228237, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7848, "incarnation": 103, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:33.240915+0000", "qdb_cluster": [ 7848 ], "qdb_leader": 7848, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7848 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1963602121", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1963602121, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1963602121, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6524, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2989703655", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2989703655, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2989703655, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6527, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6833/1367930359", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 1367930359, "type": "v2" }, { "addr": "172.21.15.17:6833", "nonce": 1367930359, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7845, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:37.013 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:37.013 INFO:tasks.cephfs.test_failover:current = [{'gid': 7845, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6833/1367930359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 1367930359}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 1367930359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 6527, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2989703655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2989703655}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2989703655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6524, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1963602121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1963602121}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1963602121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}] 2024-03-06T07:24:37.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs 2024-03-06T07:24:37.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.116+0000 7efc27965640 1 Processor -- start 2024-03-06T07:24:37.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.116+0000 7efc27965640 1 -- start start 2024-03-06T07:24:37.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc27965640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc20156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc27965640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200c72b0 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc27965640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc20073580 con 0x7efc201505f0 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc27965640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc20070b20 con 0x7efc200c7f60 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc25f02640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efc200c6710 0x7efc200c6ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44606/0 (socket says 172.21.15.17:44606) 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc25f02640 1 -- 172.21.15.17:0/2584000116 learned_addr learned my addr 172.21.15.17:0/2584000116 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc24f00640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc20156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.117+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980548884 0 0) 0x7efc200c72b0 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc0c003520 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484343045 0 0) 0x7efc0c003520 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 msgr2=0x7efc20156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc20156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 >> v1:172.21.15.163:6789/0 conn(0x7efc201505f0 legacy=0x7efc20072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200701f0 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7efc180019f0 con 0x7efc200c6710 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 >> v1:172.21.15.17:6789/0 conn(0x7efc200c6710 legacy=0x7efc200c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0c004d40 0x7efc0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20070b20 con 0x7efc0c0039e0 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c00ba70 con 0x7efc0c004d40 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c00bc70 con 0x7efc200c7f60 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc24f00640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25f02640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0c004d40 0x7efc0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 -- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 msgr2=0x7efc0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 -- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0c004d40 msgr2=0x7efc0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0c004d40 0x7efc0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25701640 1 -- 172.21.15.17:0/2584000116 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c003520 con 0x7efc0c0039e0 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc24f00640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c7f60 0x7efc0c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.118+0000 7efc25f02640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc0c004d40 0x7efc0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.119+0000 7efc25701640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc0c003e00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001720 tx=0x7efc0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90d6c435c4aa105 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.119+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c001370 con 0x7efc0c0039e0 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.119+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0052d0 con 0x7efc0c0039e0 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.119+0000 7efc17fff640 1 -- 172.21.15.17:0/2584000116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c003830 con 0x7efc0c0039e0 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 -- 172.21.15.17:0/2584000116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 msgr2=0x7efc0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 --2- 172.21.15.17:0/2584000116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc0c003e00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efc1c001720 tx=0x7efc0c005830 comp rx=0 tx=0).stop 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc25f02640 1 -- 172.21.15.17:0/2584000116 reap_dead start 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 -- 172.21.15.17:0/2584000116 shutdown_connections 2024-03-06T07:24:37.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 -- 172.21.15.17:0/2584000116 >> 172.21.15.17:0/2584000116 conn(0x7efc200aa5c0 msgr2=0x7efc200aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 -- 172.21.15.17:0/2584000116 shutdown_connections 2024-03-06T07:24:37.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.120+0000 7efc27965640 1 -- 172.21.15.17:0/2584000116 wait complete. 2024-03-06T07:24:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.121+0000 7efc27965640 1 Processor -- start 2024-03-06T07:24:37.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.121+0000 7efc27965640 1 -- start start 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201eafd0 0x7efc200c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c2700 0x7efc20152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2006fec0 con 0x7efc0c0039e0 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200707b0 con 0x7efc201eafd0 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc27965640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc20072c20 con 0x7efc200c2700 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57918/0 (socket says 172.21.15.17:57918) 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 -- 172.21.15.17:0/4011721508 learned_addr learned my addr 172.21.15.17:0/4011721508 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25f02640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c2700 0x7efc20152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc24f00640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201eafd0 0x7efc200c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 -- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c2700 msgr2=0x7efc20152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c2700 0x7efc20152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 -- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201eafd0 msgr2=0x7efc200c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201eafd0 0x7efc200c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 -- 172.21.15.17:0/4011721508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc201eb460 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.122+0000 7efc25701640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efc1c008fd0 tx=0x7efc1c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.123+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c041070 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.123+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c005480 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.123+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2014b010 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.123+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c007a30 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.123+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2014b5d0 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.124+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2004f330 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.124+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc1c044070 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.125+0000 7efc15ffb640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 0x7efc0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.125+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc1c0abdf0 con 0x7efc0c0039e0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.125+0000 7efc24f00640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 0x7efc0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.125+0000 7efc24f00640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 0x7efc0406d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc200c3060 tx=0x7efc10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.129+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc1c07a520 con 0x7efc0c0039e0 2024-03-06T07:24:37.169 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:24:37.168+0000 7ff5d2948640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:37.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.309+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7efc200c6f20 con 0x7efc0c0039e0 2024-03-06T07:24:37.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.312+0000 7efc15ffb640 1 -- 172.21.15.17:0/4011721508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v42)=0 v42) v1 ==== 116+0+0 (secure 0 0 0) 0x7efc1c07e5c0 con 0x7efc0c0039e0 2024-03-06T07:24:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 msgr2=0x7efc0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 0x7efc0406d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efc200c3060 tx=0x7efc10008040 comp rx=0 tx=0).stop 2024-03-06T07:24:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 msgr2=0x7efc2014bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efc1c008fd0 tx=0x7efc1c001120 comp rx=0 tx=0).stop 2024-03-06T07:24:37.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 shutdown_connections 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc0406b500 0x7efc0406d9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc200c2700 0x7efc20152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc201eafd0 0x7efc200c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 --2- 172.21.15.17:0/4011721508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc0c0039e0 0x7efc2014bdd0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 >> 172.21.15.17:0/4011721508 conn(0x7efc200aa5c0 msgr2=0x7efc2014e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 shutdown_connections 2024-03-06T07:24:37.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.314+0000 7efc27965640 1 -- 172.21.15.17:0/4011721508 wait complete. 2024-03-06T07:24:37.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:37.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.426+0000 7f33b3be9640 1 Processor -- start 2024-03-06T07:24:37.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 -- start start 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33ac0c3a10 0x7f33ac06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f33ac072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0c6a90 con 0x7f33ac0c3630 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac0c7660 con 0x7f33ac0c3a10 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b3be9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c6d30 con 0x7f33ac06f3b0 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b1985640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f33ac0c3630 0x7f33ac0c4b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44612/0 (socket says 172.21.15.17:44612) 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.427+0000 7f33b1985640 1 -- 172.21.15.17:0/1118160569 learned_addr learned my addr 172.21.15.17:0/1118160569 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f33ac072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b1184640 1 --2- 172.21.15.17:0/1118160569 >> v2:172.21.15.163:3300/0 conn(0x7f33ac0c3a10 0x7f33ac06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273339372 0 0) 0x7f33ac0c6a90 con 0x7f33ac0c3630 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3398003520 con 0x7f33ac0c3630 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520642022 0 0) 0x7f3398003520 con 0x7f33ac0c3630 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 msgr2=0x7f33ac072650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f33ac072650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 >> v2:172.21.15.163:3300/0 conn(0x7f33ac0c3a10 msgr2=0x7f33ac06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 --2- 172.21.15.17:0/1118160569 >> v2:172.21.15.163:3300/0 conn(0x7f33ac0c3a10 0x7f33ac06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0c7010 con 0x7f33ac0c3630 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f33a8001670 con 0x7f33ac0c3630 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 >> v1:172.21.15.17:6789/0 conn(0x7f33ac0c3630 legacy=0x7f33ac0c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33980039b0 0x7f3398003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f3398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3398004d40 0x7f33980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c7660 con 0x7f3398004d40 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c6d30 con 0x7f33980039b0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339800b9c0 con 0x7f33ac06f3b0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3398004d40 0x7f33980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b1184640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f3398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b1985640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33980039b0 0x7f3398003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 -- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 msgr2=0x7f3398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac06f3b0 0x7f3398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 -- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33980039b0 msgr2=0x7f3398003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33980039b0 0x7f3398003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.428+0000 7f33b2186640 1 -- 172.21.15.17:0/1118160569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398003520 con 0x7f3398004d40 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.429+0000 7f33b1985640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33980039b0 0x7f3398003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.429+0000 7f33b2186640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3398004d40 0x7f33980051c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fc0 tx=0x7f3398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2690e9fe4da14847 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.429+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4039070 con 0x7f3398004d40 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.429+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a402cc20 con 0x7f3398004d40 2024-03-06T07:24:37.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.429+0000 7f33b0983640 1 -- 172.21.15.17:0/1118160569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4009c10 con 0x7f3398004d40 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 -- 172.21.15.17:0/1118160569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3398004d40 msgr2=0x7f33980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1118160569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3398004d40 0x7f33980051c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009fc0 tx=0x7f3398005830 comp rx=0 tx=0).stop 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b2186640 1 -- 172.21.15.17:0/1118160569 reap_dead start 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 -- 172.21.15.17:0/1118160569 shutdown_connections 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 -- 172.21.15.17:0/1118160569 >> 172.21.15.17:0/1118160569 conn(0x7f33ac06b5f0 msgr2=0x7f33ac0aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 -- 172.21.15.17:0/1118160569 shutdown_connections 2024-03-06T07:24:37.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.430+0000 7f33b3be9640 1 -- 172.21.15.17:0/1118160569 wait complete. 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.432+0000 7f33b3be9640 1 Processor -- start 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.432+0000 7f33b3be9640 1 -- start start 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac0c4a70 0x7f33ac0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 0x7f33ac1d68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c4020 con 0x7f33ac1d6de0 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0c3ea0 con 0x7f33ac0c2b10 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b3be9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac06f800 con 0x7f33ac0c4a70 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57940/0 (socket says 172.21.15.17:57940) 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 -- 172.21.15.17:0/1012312517 learned_addr learned my addr 172.21.15.17:0/1012312517 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b1184640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 0x7f33ac1d68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b1985640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac0c4a70 0x7f33ac0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 -- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac0c4a70 msgr2=0x7f33ac0c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac0c4a70 0x7f33ac0c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 -- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 msgr2=0x7f33ac1d68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 0x7f33ac1d68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b2186640 1 -- 172.21.15.17:0/1012312517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0a7bc0 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.433+0000 7f33b1184640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 0x7f33ac1d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33b2186640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f33a4002230 tx=0x7f33a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4039030 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4043070 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac1d7260 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4009c10 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.434+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac0be460 con 0x7f33ac1d6de0 2024-03-06T07:24:37.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.435+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac04f330 con 0x7f33ac1d6de0 2024-03-06T07:24:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.435+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33a4005d10 con 0x7f33ac1d6de0 2024-03-06T07:24:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.436+0000 7f33a27fc640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 0x7f339006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.436+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33a40ac4b0 con 0x7f33ac1d6de0 2024-03-06T07:24:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.436+0000 7f33b1985640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 0x7f339006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:37.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.436+0000 7f33b1985640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 0x7f339006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:37.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.444+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33a407b230 con 0x7f33ac1d6de0 2024-03-06T07:24:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.662+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33ac1da570 con 0x7f33ac1d6de0 2024-03-06T07:24:37.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.663+0000 7f33a27fc640 1 -- 172.21.15.17:0/1012312517 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f33a407ab30 con 0x7f33ac1d6de0 2024-03-06T07:24:37.665 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:37.665 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":105,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:33.240915+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/410228237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":410228237},{"type":"v1","addr":"172.21.15.17:6835","nonce":410228237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":9}]} 2024-03-06T07:24:37.665 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 105 2024-03-06T07:24:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 msgr2=0x7f339006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 0x7f339006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 msgr2=0x7f33ac1e8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f33a4002230 tx=0x7f33a4009750 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 shutdown_connections 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f339006b4d0 0x7f339006d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33ac1d6de0 0x7f33ac1e8450 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33ac0c2b10 0x7f33ac1d68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 --2- 172.21.15.17:0/1012312517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33ac0c4a70 0x7f33ac0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 >> 172.21.15.17:0/1012312517 conn(0x7f33ac06b5f0 msgr2=0x7f33ac1daa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 shutdown_connections 2024-03-06T07:24:37.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:37.665+0000 7f33b3be9640 1 -- 172.21.15.17:0/1012312517 wait complete. 2024-03-06T07:24:38.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:33.240915+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7848': {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7848, 'qdb_cluster': [7848]} 2024-03-06T07:24:38.080 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:38.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.181+0000 7fad0617a640 1 Processor -- start 2024-03-06T07:24:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.181+0000 7fad0617a640 1 -- start start 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7fad0617a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7fad001551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7fad0617a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000733f0 con 0x7fad000a7ff0 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7fad0617a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad000c82c0 con 0x7fad00152b40 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7fad0617a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad00072d70 con 0x7fad000724d0 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7facfffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fad000a7ff0 0x7fad00155730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:44622/0 (socket says 172.21.15.17:44622) 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7facfffff640 1 -- 172.21.15.17:0/309364431 learned_addr learned my addr 172.21.15.17:0/309364431 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7facfeffd640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7fad001551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.182+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266003808 0 0) 0x7fad000733f0 con 0x7fad000a7ff0 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facec003520 con 0x7fad000a7ff0 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850827986 0 0) 0x7fad000c82c0 con 0x7fad00152b40 2024-03-06T07:24:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad000733f0 con 0x7fad00152b40 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752641137 0 0) 0x7facec003520 con 0x7fad000a7ff0 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 msgr2=0x7fad001551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7fad001551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 >> v1:172.21.15.163:6789/0 conn(0x7fad00152b40 legacy=0x7fad000c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0006fd30 con 0x7fad000a7ff0 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7facf80019f0 con 0x7fad000a7ff0 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 >> v1:172.21.15.17:6789/0 conn(0x7fad000a7ff0 legacy=0x7fad00155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec003b70 0x7facec003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7facec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004d40 0x7facec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00072d70 con 0x7facec004d40 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec00baf0 con 0x7facec003b70 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec00bcf0 con 0x7fad000724d0 2024-03-06T07:24:38.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004d40 0x7facec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facff7fe640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec003b70 0x7facec003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfeffd640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7facec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 -- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 msgr2=0x7facec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7facec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 -- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec003b70 msgr2=0x7facec003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec003b70 0x7facec003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfffff640 1 -- 172.21.15.17:0/309364431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec003520 con 0x7facec004d40 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.183+0000 7facfeffd640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad000724d0 0x7facec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.184+0000 7facfffff640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004d40 0x7facec0051c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fad000c5350 tx=0x7facec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8ab0be368a0b94 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.184+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf800a410 con 0x7facec004d40 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.184+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf8034650 con 0x7facec004d40 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.184+0000 7facfe7fc640 1 -- 172.21.15.17:0/309364431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8034a40 con 0x7facec004d40 2024-03-06T07:24:38.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 -- 172.21.15.17:0/309364431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004d40 msgr2=0x7facec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 --2- 172.21.15.17:0/309364431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec004d40 0x7facec0051c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fad000c5350 tx=0x7facec005830 comp rx=0 tx=0).stop 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7facfffff640 1 -- 172.21.15.17:0/309364431 reap_dead start 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 -- 172.21.15.17:0/309364431 shutdown_connections 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 -- 172.21.15.17:0/309364431 >> 172.21.15.17:0/309364431 conn(0x7fad0006b970 msgr2=0x7fad000aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 -- 172.21.15.17:0/309364431 shutdown_connections 2024-03-06T07:24:38.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.185+0000 7fad0617a640 1 -- 172.21.15.17:0/309364431 wait complete. 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.186+0000 7fad0617a640 1 Processor -- start 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.186+0000 7fad0617a640 1 -- start start 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec004d40 0x7fad0006cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad001e12e0 0x7fad000ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00070400 con 0x7fad00147e80 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00070280 con 0x7fad001e12e0 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7fad0617a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad001551f0 con 0x7facec004d40 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57954/0 (socket says 172.21.15.17:57954) 2024-03-06T07:24:38.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 -- 172.21.15.17:0/4258381943 learned_addr learned my addr 172.21.15.17:0/4258381943 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facff7fe640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec004d40 0x7fad0006cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfeffd640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad001e12e0 0x7fad000ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 -- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec004d40 msgr2=0x7fad0006cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec004d40 0x7fad0006cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 -- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad001e12e0 msgr2=0x7fad000ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad001e12e0 0x7fad000ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.187+0000 7facfffff640 1 -- 172.21.15.17:0/4258381943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad001eee20 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7facfffff640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7facf802cb70 tx=0x7fad000c76c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7facf7fff640 1 -- 172.21.15.17:0/4258381943 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8045070 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7facf7fff640 1 -- 172.21.15.17:0/4258381943 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf803a040 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000abce0 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7facf7fff640 1 -- 172.21.15.17:0/4258381943 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf80019f0 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.188+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0006d020 con 0x7fad00147e80 2024-03-06T07:24:38.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.189+0000 7facf7fff640 1 -- 172.21.15.17:0/4258381943 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facf8048070 con 0x7fad00147e80 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.190+0000 7facf7fff640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 0x7face006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.190+0000 7facf7fff640 1 -- 172.21.15.17:0/4258381943 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facf80ac110 con 0x7fad00147e80 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.190+0000 7facff7fe640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 0x7face006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.190+0000 7facff7fe640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 0x7face006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7face8005530 tx=0x7face8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 Processor -- start 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 -- start start 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b89f50 0x560302b87a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b884b0 0x560302b84680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302aee6b0 con 0x560302b8a330 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302a97a60 con 0x560302b89f50 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560302aea750 con 0x560302b884b0 2024-03-06T07:24:38.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.191+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57964/0 (socket says 172.21.15.17:57964) 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 -- 172.21.15.17:0/1073137772 learned_addr learned my addr 172.21.15.17:0/1073137772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfffff640 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b884b0 0x560302b84680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facff7fe640 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b89f50 0x560302b87a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 -- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b884b0 msgr2=0x560302b84680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b884b0 0x560302b84680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 -- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b89f50 msgr2=0x560302b87a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b89f50 0x560302b87a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 -- 172.21.15.17:0/1073137772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560302ab5f30 con 0x560302b8a330 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facfeffd640 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7facf0009ad0 tx=0x7facf0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba23e41e2c41fda server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facf57fa640 1 -- 172.21.15.17:0/1073137772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001a030 con 0x560302b8a330 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facf57fa640 1 -- 172.21.15.17:0/1073137772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0013a40 con 0x560302b8a330 2024-03-06T07:24:38.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.192+0000 7facf57fa640 1 -- 172.21.15.17:0/1073137772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0013e10 con 0x560302b8a330 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 -- 172.21.15.17:0/1073137772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 msgr2=0x560302b87f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7facf0009ad0 tx=0x7facf0009190 comp rx=0 tx=0).stop 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 -- 172.21.15.17:0/1073137772 shutdown_connections 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b884b0 0x560302b84680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b87f70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 --2- 172.21.15.17:0/1073137772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b89f50 0x560302b87a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 -- 172.21.15.17:0/1073137772 >> 172.21.15.17:0/1073137772 conn(0x560302b867d0 msgr2=0x560302bab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 -- 172.21.15.17:0/1073137772 shutdown_connections 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.193+0000 7fad08d66740 1 -- 172.21.15.17:0/1073137772 wait complete. 2024-03-06T07:24:38.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.194+0000 7fad08d66740 1 -- start start 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.194+0000 7fad08d66740 1 Processor -- start 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302b88ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302bc2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302bac230 con 0x560302b8a330 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302aaaea0 con 0x560302bb1af0 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560302babc70 con 0x560302b89f50 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7fad08d66740 10 client.0 _ll_register_callbacks cb 0x560302aaa870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57970/0 (socket says 172.21.15.17:57970) 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.195+0000 7facfeffd640 1 -- 172.21.15.17:0/3088270851 learned_addr learned my addr 172.21.15.17:0/3088270851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facff7fe640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302b88ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfffff640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302bc2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 -- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 msgr2=0x560302b88ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302b88ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 -- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 msgr2=0x560302bc2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302bc2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 -- 172.21.15.17:0/3088270851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560302aab270 con 0x560302b8a330 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facff7fe640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302b88ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfffff640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302bc2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facfeffd640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7facf0000c00 tx=0x7facf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfbcd2da4c950b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf002d060 con 0x560302b8a330 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf0013a40 con 0x560302b8a330 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facf0016670 con 0x560302b8a330 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf0005990 con 0x560302b8a330 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7fad08d66740 10 client.8025 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:38.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x5603028032d0 con 0x560302b8a330 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.196+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0013a40 con 0x560302b8a330 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7facf0005b30 con 0x560302b8a330 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 10 client.8025 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 10 client.8025 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 20 client.8025 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302bda7a0 0x560302be5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 4 client.8025 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560302afbbf0 con 0x560302bda7a0 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7facff7fe640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302bda7a0 0x560302be5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7facde7fc640 10 client.8025 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.197+0000 7facff7fe640 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302bda7a0 0x560302be5500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c005f6a0924a8c server_cookie=9caaca672ef3e4aa in_seq=0 out_seq=0 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7facde7fc640 1 -- 172.21.15.17:0/3088270851 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560302afbbf0 con 0x560302bda7a0 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7facde7fc640 10 client.8025 handle_command_reply: tid=0 2024-03-06T07:24:38.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 client.8025 shutdown 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 msgr2=0x560302bafa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7facf0000c00 tx=0x7facf00046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 shutdown_connections 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302bda7a0 0x560302be5500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302bc2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302bafa30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 --2- 172.21.15.17:0/3088270851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302b88ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 >> 172.21.15.17:0/3088270851 conn(0x560302b884b0 msgr2=0x560302b84590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7facde7fc640 0 client.8025 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 shutdown_connections 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 1 -- 172.21.15.17:0/3088270851 wait complete. 2024-03-06T07:24:38.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.198+0000 7fad08d66740 20 client.8025 trim_cache size 0 max 16384 2024-03-06T07:24:38.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.215+0000 7fad08d66740 1 Processor -- start 2024-03-06T07:24:38.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.215+0000 7fad08d66740 1 -- start start 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302bc7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302b8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001f75c0 con 0x560302b8a330 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000a9690 con 0x560302bb1af0 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad001f6ff0 con 0x560302b89f50 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57978/0 (socket says 172.21.15.17:57978) 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 -- 172.21.15.17:0/131650669 learned_addr learned my addr 172.21.15.17:0/131650669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facff7fe640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302bc7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfffff640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302b8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 -- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 msgr2=0x560302bc7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302bc7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 -- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 msgr2=0x560302b8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302b8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 -- 172.21.15.17:0/131650669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560302bafda0 con 0x560302b8a330 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facff7fe640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302bc7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfffff640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302b8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.216+0000 7facfeffd640 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7facf000b010 tx=0x7facf0024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a82628bc2410b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7facf4ff9640 1 -- 172.21.15.17:0/131650669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001a070 con 0x560302b8a330 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7facf4ff9640 1 -- 172.21.15.17:0/131650669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf001b040 con 0x560302b8a330 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7facf4ff9640 1 -- 172.21.15.17:0/131650669 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00042c0 con 0x560302b8a330 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 -- 172.21.15.17:0/131650669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 msgr2=0x560302b8a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7facf000b010 tx=0x7facf0024000 comp rx=0 tx=0).stop 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 -- 172.21.15.17:0/131650669 shutdown_connections 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302bb1af0 0x560302b8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b8a330 0x560302b8a710 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 --2- 172.21.15.17:0/131650669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302b89f50 0x560302bc7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 -- 172.21.15.17:0/131650669 >> 172.21.15.17:0/131650669 conn(0x560302b884b0 msgr2=0x560302b850a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 -- 172.21.15.17:0/131650669 shutdown_connections 2024-03-06T07:24:38.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.217+0000 7fad08d66740 1 -- 172.21.15.17:0/131650669 wait complete. 2024-03-06T07:24:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.218+0000 7fad08d66740 1 -- start start 2024-03-06T07:24:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.218+0000 7fad08d66740 1 Processor -- start 2024-03-06T07:24:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b8a330 0x560302be07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 0x560302c2d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302bb0490 con 0x560302b89f50 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560302bad1e0 con 0x560302b8a330 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560302bd6950 con 0x560302bb1af0 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7fad08d66740 10 client.0 _ll_register_callbacks cb 0x7facf0009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7facff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7facff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57982/0 (socket says 172.21.15.17:57982) 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.219+0000 7facff7fe640 1 -- 172.21.15.17:0/883327352 learned_addr learned my addr 172.21.15.17:0/883327352 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facfeffd640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b8a330 0x560302be07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facfffff640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 0x560302c2d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 -- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 msgr2=0x560302c2d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 0x560302c2d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 -- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b8a330 msgr2=0x560302be07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b8a330 0x560302be07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 -- 172.21.15.17:0/883327352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603028cd6b0 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facfffff640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 0x560302c2d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facff7fe640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7face8005620 tx=0x7face80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2c596c0296e960 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8033410 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7face8034df0 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf001b040 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7face80243f0 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7fad08d66740 10 client.8031 fetch_fsmap learned FSMap version 105 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x560302b85490 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.220+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf001b040 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 105) v1 ==== 3314+0+0 (secure 0 0 0) 0x7face803b050 con 0x560302b89f50 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 10 client.8031 fetch_fsmap finished waiting for FSMap version 105 2024-03-06T07:24:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 10 client.8031 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 20 client.8031 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302c368c0 0x560302b87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 4 client.8031 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560302af5f40 con 0x560302c368c0 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facfeffd640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302c368c0 0x560302b87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facfeffd640 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302c368c0 0x560302b87370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d8a6090f9a64d6 server_cookie=d5a36039db7aea65 in_seq=0 out_seq=0 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facde7fc640 10 client.8031 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facde7fc640 1 -- 172.21.15.17:0/883327352 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560302af5f40 con 0x560302c368c0 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facde7fc640 10 client.8031 handle_command_reply: tid=0 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 client.8031 shutdown 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 msgr2=0x560302bdb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7face8005620 tx=0x7face80012d0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 shutdown_connections 2024-03-06T07:24:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560302c368c0 0x560302b87370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560302bb1af0 0x560302c2d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560302b8a330 0x560302be07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7facde7fc640 0 client.8031 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 --2- 172.21.15.17:0/883327352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560302b89f50 0x560302bdb500 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.221+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 >> 172.21.15.17:0/883327352 conn(0x560302b884b0 msgr2=0x560302babe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 shutdown_connections 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad08d66740 1 -- 172.21.15.17:0/883327352 wait complete. 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad08d66740 20 client.8031 trim_cache size 0 max 16384 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:38.229 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7848, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 6.334310479, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 105, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 122, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 122, 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11.425818123999999 2024-03-06T07:24:38.230 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 msgr2=0x7face006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 0x7face006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7face8005530 tx=0x7face8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 msgr2=0x7fad001edf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7facf802cb70 tx=0x7fad000c76c0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 shutdown_connections 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7face006b4d0 0x7face006d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad00147e80 0x7fad001edf20 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad001e12e0 0x7fad000ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 --2- 172.21.15.17:0/4258381943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec004d40 0x7fad0006cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:38.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 >> 172.21.15.17:0/4258381943 conn(0x7fad0006b970 msgr2=0x7fad00154060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:38.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 shutdown_connections 2024-03-06T07:24:38.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:38.222+0000 7fad0617a640 1 -- 172.21.15.17:0/4258381943 wait complete. 2024-03-06T07:24:38.234 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 105, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:31.239352+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 105, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7848": { "addr": "172.21.15.17:6835/410228237", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6834", "nonce": 410228237, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 410228237, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7848, "incarnation": 103, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:33.240915+0000", "qdb_cluster": [ 7848 ], "qdb_leader": 7848, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7848 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1963602121", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1963602121, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1963602121, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6524, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2989703655", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2989703655, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2989703655, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6527, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6833/1367930359", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 1367930359, "type": "v2" }, { "addr": "172.21.15.17:6833", "nonce": 1367930359, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7845, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:38.235 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:38.235 INFO:tasks.cephfs.test_failover:current = [{'gid': 7845, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6833/1367930359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 1367930359}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 1367930359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 6527, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2989703655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2989703655}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2989703655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6524, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1963602121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1963602121}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1963602121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}] 2024-03-06T07:24:38.236 DEBUG:tasks.cephfs.test_failover:9 != -1 2024-03-06T07:24:38.236 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:24:39.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:39.218 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:24:43.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:43.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.338+0000 7f5a80716640 1 Processor -- start 2024-03-06T07:24:43.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.338+0000 7f5a80716640 1 -- start start 2024-03-06T07:24:43.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a80716640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a80716640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a78158b60 0x7f5a78158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a80716640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780c73d0 con 0x7f5a7814f360 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a80716640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780c8dd0 con 0x7f5a78158b60 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a80716640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780be2e0 con 0x7f5a780c88c0 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a7dcb1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.339+0000 7f5a7ecb3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a78158b60 0x7f5a78158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7e4b2640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5a7814f360 0x7f5a780c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46948/0 (socket says 172.21.15.17:46948) 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7ecb3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a78158b60 0x7f5a78158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:47894/0 (socket says 172.21.15.17:47894) 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7dcb1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52380/0 (socket says 172.21.15.17:52380) 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7e4b2640 1 -- 172.21.15.17:0/3020294891 learned_addr learned my addr 172.21.15.17:0/3020294891 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7ecb3640 1 -- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 msgr2=0x7f5a78155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7ecb3640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7ecb3640 1 -- 172.21.15.17:0/3020294891 >> v1:172.21.15.163:6789/0 conn(0x7f5a7814f360 legacy=0x7f5a780c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7ecb3640 1 -- 172.21.15.17:0/3020294891 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780c7670 con 0x7f5a78158b60 2024-03-06T07:24:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.340+0000 7f5a7dcb1640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.341+0000 7f5a7ecb3640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a78158b60 0x7f5a78158ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009d90 tx=0x7f5a7002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265abdc9055961be server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.341+0000 7f5a7d4b0640 1 -- 172.21.15.17:0/3020294891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70039070 con 0x7f5a78158b60 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.341+0000 7f5a7d4b0640 1 -- 172.21.15.17:0/3020294891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7002cbc0 con 0x7f5a78158b60 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.341+0000 7f5a7d4b0640 1 -- 172.21.15.17:0/3020294891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a70034440 con 0x7f5a78158b60 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 -- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a78158b60 msgr2=0x7f5a78158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a78158b60 0x7f5a78158ff0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009d90 tx=0x7f5a7002be10 comp rx=0 tx=0).stop 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 -- 172.21.15.17:0/3020294891 shutdown_connections 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a78158b60 0x7f5a78158ff0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 --2- 172.21.15.17:0/3020294891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a780c88c0 0x7f5a78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 -- 172.21.15.17:0/3020294891 >> 172.21.15.17:0/3020294891 conn(0x7f5a780ac570 msgr2=0x7f5a780ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 -- 172.21.15.17:0/3020294891 shutdown_connections 2024-03-06T07:24:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.342+0000 7f5a80716640 1 -- 172.21.15.17:0/3020294891 wait complete. 2024-03-06T07:24:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.343+0000 7f5a80716640 1 Processor -- start 2024-03-06T07:24:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.343+0000 7f5a80716640 1 -- start start 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a7814f360 0x7f5a7814d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 0x7f5a780c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780c5a50 con 0x7f5a780c88c0 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780c7c30 con 0x7f5a7814f360 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780c5bd0 con 0x7f5a78158b60 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40560/0 (socket says 172.21.15.17:40560) 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 -- 172.21.15.17:0/110714246 learned_addr learned my addr 172.21.15.17:0/110714246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7dcb1640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a7814f360 0x7f5a7814d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7ecb3640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 0x7f5a780c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 -- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 msgr2=0x7f5a780c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 0x7f5a780c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 -- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a7814f360 msgr2=0x7f5a7814d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a7814f360 0x7f5a7814d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 -- 172.21.15.17:0/110714246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780c0870 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7ecb3640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 0x7f5a780c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a7e4b2640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400c3c0 tx=0x7f5a7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74017070 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a74005ce0 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7814bdc0 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a740072f0 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.345+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a780c11c0 con 0x7f5a780c88c0 2024-03-06T07:24:43.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.346+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a780c0140 con 0x7f5a780c88c0 2024-03-06T07:24:43.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.347+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a7401a070 con 0x7f5a780c88c0 2024-03-06T07:24:43.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.347+0000 7f5a6effd640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:43.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.347+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5a740896b0 con 0x7f5a780c88c0 2024-03-06T07:24:43.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.348+0000 7f5a7dcb1640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:43.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.348+0000 7f5a7dcb1640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5a78152c70 tx=0x7f5a68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:43.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.356+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a74058430 con 0x7f5a780c88c0 2024-03-06T07:24:43.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.574+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a7804f330 con 0x7f5a780c88c0 2024-03-06T07:24:43.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.574+0000 7f5a6effd640 1 -- 172.21.15.17:0/110714246 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4987 (secure 0 0 0) 0x7f5a74057d30 con 0x7f5a780c88c0 2024-03-06T07:24:43.576 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:43.576 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":106,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7845,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:39.914109+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7848},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7848":{"gid":7848,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":5,"addr":"172.21.15.17:6835/410228237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":410228237},{"type":"v1","addr":"172.21.15.17:6835","nonce":410228237}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7848,"qdb_cluster":[7848]},"id":9}]} 2024-03-06T07:24:43.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 106 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 msgr2=0x7f5a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5a78152c70 tx=0x7f5a68008040 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 msgr2=0x7f5a781eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400c3c0 tx=0x7f5a7400c880 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 shutdown_connections 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a78158b60 0x7f5a780c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a7814f360 0x7f5a7814d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 --2- 172.21.15.17:0/110714246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a780c88c0 0x7f5a781eb180 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 >> 172.21.15.17:0/110714246 conn(0x7f5a780ac570 msgr2=0x7f5a780ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:43.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 shutdown_connections 2024-03-06T07:24:43.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:43.576+0000 7f5a80716640 1 -- 172.21.15.17:0/110714246 wait complete. 2024-03-06T07:24:43.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:39.914109+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7848}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7848': {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7848, 'qdb_cluster': [7848]} 2024-03-06T07:24:43.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:43.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.092+0000 7fd3f0fa0640 1 Processor -- start 2024-03-06T07:24:44.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.092+0000 7fd3f0fa0640 1 -- start start 2024-03-06T07:24:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 0x7fd3ec0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd3ec155d70 0x7fd3ec159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec1558e0 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec1596c0 con 0x7fd3ec155d70 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3f0fa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0729c0 con 0x7fd3ec13e490 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3ebfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3ebfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40574/0 (socket says 172.21.15.17:40574) 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.093+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2928845229 learned_addr learned my addr 172.21.15.17:0/2928845229 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/2928845229 >> v2:172.21.15.163:3300/0 conn(0x7fd3ec155d70 0x7fd3ec159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 0x7fd3ec0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2928845229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 msgr2=0x7fd3ec0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 0x7fd3ec0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2928845229 >> v2:172.21.15.163:3300/0 conn(0x7fd3ec155d70 msgr2=0x7fd3ec159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2928845229 >> v2:172.21.15.163:3300/0 conn(0x7fd3ec155d70 0x7fd3ec159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2928845229 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec155290 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 0x7fd3ec0723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.094+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec06eb00 tx=0x7fd3e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7548189bcb9d922 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3ea7fc640 1 -- 172.21.15.17:0/2928845229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e402ca70 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3ea7fc640 1 -- 172.21.15.17:0/2928845229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402cc10 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3ea7fc640 1 -- 172.21.15.17:0/2928845229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40343f0 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/2928845229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 msgr2=0x7fd3ec0c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec06eb00 tx=0x7fd3e402bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/2928845229 shutdown_connections 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec0c50c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/2928845229 >> v2:172.21.15.163:3300/0 conn(0x7fd3ec155d70 0x7fd3ec159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/2928845229 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec13e490 0x7fd3ec0723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/2928845229 >> 172.21.15.17:0/2928845229 conn(0x7fd3ec06afc0 msgr2=0x7fd3ec06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/2928845229 shutdown_connections 2024-03-06T07:24:44.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.095+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/2928845229 wait complete. 2024-03-06T07:24:44.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.096+0000 7fd3f0fa0640 1 Processor -- start 2024-03-06T07:24:44.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.096+0000 7fd3f0fa0640 1 -- start start 2024-03-06T07:24:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 0x7fd3ec0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec155d70 0x7fd3ec1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec072e20 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec072ca0 con 0x7fd3ec13e490 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3f0fa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0c3fb0 con 0x7fd3ec155d70 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40580/0 (socket says 172.21.15.17:40580) 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3497592925 learned_addr learned my addr 172.21.15.17:0/3497592925 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec155d70 0x7fd3ec1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 0x7fd3ec0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec155d70 msgr2=0x7fd3ec1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.097+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec155d70 0x7fd3ec1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 msgr2=0x7fd3ec0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 0x7fd3ec0c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3497592925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0c92c0 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 0x7fd3ec0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c7c0 tx=0x7fd3dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3e3fff640 1 -- 172.21.15.17:0/3497592925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc015070 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3e3fff640 1 -- 172.21.15.17:0/3497592925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc007a30 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3e3fff640 1 -- 172.21.15.17:0/3497592925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc007e00 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec0c7e70 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.098+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec1566e0 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.100+0000 7fd3e3fff640 1 -- 172.21.15.17:0/3497592925 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3dc018070 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.100+0000 7fd3e3fff640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.100+0000 7fd3e3fff640 1 -- 172.21.15.17:0/3497592925 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3dc08a2b0 con 0x7fd3ec0c3b20 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.100+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.101+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4006fd0 tx=0x7fd3d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.101+0000 7fd3f4deb740 1 Processor -- start 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.101+0000 7fd3f4deb740 1 -- start start 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869619be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608695806b0 con 0x56086961a660 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560869529a60 con 0x56086961c4e0 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.102+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56086957c750 con 0x56086961c100 2024-03-06T07:24:44.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40582/0 (socket says 172.21.15.17:40582) 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2292190962 learned_addr learned my addr 172.21.15.17:0/2292190962 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869619be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 msgr2=0x560869619be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869619be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 msgr2=0x56086961a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 -- 172.21.15.17:0/2292190962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560869547f30 con 0x56086961a660 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009f40 tx=0x7fd3e402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d07090bfde3d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3e17fa640 1 -- 172.21.15.17:0/2292190962 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e402ca70 con 0x56086961a660 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3e17fa640 1 -- 172.21.15.17:0/2292190962 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402cc10 con 0x56086961a660 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.103+0000 7fd3e17fa640 1 -- 172.21.15.17:0/2292190962 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e403e630 con 0x56086961a660 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 -- 172.21.15.17:0/2292190962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 msgr2=0x560869616830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009f40 tx=0x7fd3e402e000 comp rx=0 tx=0).stop 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 -- 172.21.15.17:0/2292190962 shutdown_connections 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961a660 0x560869616830 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/2292190962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869619be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 -- 172.21.15.17:0/2292190962 >> 172.21.15.17:0/2292190962 conn(0x560869618980 msgr2=0x56086963d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 -- 172.21.15.17:0/2292190962 shutdown_connections 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.104+0000 7fd3f4deb740 1 -- 172.21.15.17:0/2292190962 wait complete. 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.105+0000 7fd3f4deb740 1 -- start start 2024-03-06T07:24:44.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.105+0000 7fd3f4deb740 1 Processor -- start 2024-03-06T07:24:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086963fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 0x560869640db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086963e3e0 con 0x56086961c100 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56086953cea0 con 0x56086961c4e0 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56086963de20 con 0x560869640280 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40598/0 (socket says 172.21.15.17:40598) 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/135065174 learned_addr learned my addr 172.21.15.17:0/135065174 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3f4deb740 10 client.0 _ll_register_callbacks cb 0x56086953c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086963fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 msgr2=0x560869640db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 0x560869640db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 0x560869640db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 msgr2=0x56086963fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086963fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/135065174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56086953d270 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086963fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 0x560869640db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006b10 tx=0x7fd3dc0590f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331a18491ff917d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.106+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc01f040 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3dc059be0 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e402cc10 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc060fc0 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 10 client.8046 fetch_fsmap learned FSMap version 106 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x5608692952d0 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402cc10 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 106) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd3dc070170 con 0x56086961c100 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 10 client.8046 fetch_fsmap finished waiting for FSMap version 106 2024-03-06T07:24:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 10 client.8046 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 20 client.8046 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560869655130 0x56086963f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 4 client.8046 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56086958dbf0 con 0x560869655130 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.107+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560869655130 0x56086963f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.108+0000 7fd3ca7fc640 10 client.8046 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.108+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560869655130 0x56086963f030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bee8737a24b108 server_cookie=aa9d01a3a93bdff5 in_seq=0 out_seq=0 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.108+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/135065174 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56086958dbf0 con 0x560869655130 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.108+0000 7fd3ca7fc640 10 client.8046 handle_command_reply: tid=0 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.108+0000 7fd3f4deb740 1 client.8046 shutdown 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 msgr2=0x56086963c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006b10 tx=0x7fd3dc0590f0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 shutdown_connections 2024-03-06T07:24:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x560869655130 0x56086963f030 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x560869640280 0x560869640db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086963fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/135065174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x56086963c960 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3ca7fc640 0 client.8046 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 >> 172.21.15.17:0/135065174 conn(0x56086961a660 msgr2=0x560869659c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 shutdown_connections 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 1 -- 172.21.15.17:0/135065174 wait complete. 2024-03-06T07:24:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.109+0000 7fd3f4deb740 20 client.8046 trim_cache size 0 max 16384 2024-03-06T07:24:44.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.126+0000 7fd3f4deb740 1 Processor -- start 2024-03-06T07:24:44.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.126+0000 7fd3f4deb740 1 -- start start 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869642be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec1f6d10 con 0x560869640280 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec1eeb60 con 0x56086961c4e0 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec1f6710 con 0x56086961c100 2024-03-06T07:24:44.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40600/0 (socket says 172.21.15.17:40600) 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 -- 172.21.15.17:0/3306204875 learned_addr learned my addr 172.21.15.17:0/3306204875 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869642be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 -- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 msgr2=0x560869642be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869642be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 -- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 msgr2=0x56086961aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 -- 172.21.15.17:0/3306204875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56086935f6b0 con 0x560869640280 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402bf10 tx=0x7fd3e4009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96dcf35aac900f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3e0ff9640 1 -- 172.21.15.17:0/3306204875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e402ca70 con 0x560869640280 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3e0ff9640 1 -- 172.21.15.17:0/3306204875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402cc10 con 0x560869640280 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.127+0000 7fd3e0ff9640 1 -- 172.21.15.17:0/3306204875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4003b90 con 0x560869640280 2024-03-06T07:24:44.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3306204875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 msgr2=0x560869619750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402bf10 tx=0x7fd3e4009000 comp rx=0 tx=0).stop 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3306204875 shutdown_connections 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560869640280 0x560869619750 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56086961c4e0 0x56086961aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3306204875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c100 0x560869642be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3306204875 >> 172.21.15.17:0/3306204875 conn(0x56086961a660 msgr2=0x56086963cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3306204875 shutdown_connections 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.128+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3306204875 wait complete. 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 -- start start 2024-03-06T07:24:44.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 Processor -- start 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c4e0 0x5608696cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560869640280 0x5608696c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560869654fb0 con 0x56086961c100 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x560869655ec0 con 0x560869640280 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x560869655520 con 0x56086961c4e0 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3f4deb740 10 client.0 _ll_register_callbacks cb 0x7fd3dc058210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40616/0 (socket says 172.21.15.17:40616) 2024-03-06T07:24:44.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3233049549 learned_addr learned my addr 172.21.15.17:0/3233049549 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.129+0000 7fd3ebfff640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560869640280 0x5608696c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c4e0 0x5608696cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c4e0 msgr2=0x5608696cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c4e0 0x5608696cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560869640280 msgr2=0x5608696c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560869640280 0x5608696c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 -- 172.21.15.17:0/3233049549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560869616c90 con 0x56086961c100 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3eb7fe640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006fd0 tx=0x7fd3dc0885b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a1ac680ff66e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc01f070 con 0x56086961c100 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3dc059be0 con 0x56086961c100 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e402cc10 con 0x56086961c100 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc059de0 con 0x56086961c100 2024-03-06T07:24:44.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 10 client.8052 fetch_fsmap learned FSMap version 106 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) v3 -- 0x560869640f40 con 0x56086961c100 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e402cc10 con 0x56086961c100 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 106) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd3dc057d00 con 0x56086961c100 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 10 client.8052 fetch_fsmap finished waiting for FSMap version 106 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 10 client.8052 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 20 client.8052 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x5608696c5a70 0x56086963b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 4 client.8052 mds_command: new command op to 7848 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.130+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 --> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560869587f40 con 0x5608696c5a70 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x5608696c5a70 0x56086963b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3ca7fc640 10 client.8052 ms_handle_connect on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3eaffd640 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x5608696c5a70 0x56086963b060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713a105c7129cca0 server_cookie=9bda05119da0189d in_seq=0 out_seq=0 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3ca7fc640 1 -- 172.21.15.17:0/3233049549 <== mds.0 v2:172.21.15.17:6834/410228237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x560869587f40 con 0x5608696c5a70 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3ca7fc640 10 client.8052 handle_command_reply: tid=0 2024-03-06T07:24:44.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 client.8052 shutdown 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 msgr2=0x560869617a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006fd0 tx=0x7fd3dc0885b0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 shutdown_connections 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:6834/410228237,v1:172.21.15.17:6835/410228237] conn(0x5608696c5a70 0x56086963b060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x560869640280 0x5608696c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56086961c4e0 0x5608696cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3ca7fc640 0 client.8052 ms_handle_reset on v2:172.21.15.17:6834/410228237 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 --2- 172.21.15.17:0/3233049549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56086961c100 0x560869617a80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 >> 172.21.15.17:0/3233049549 conn(0x56086961a660 msgr2=0x56086961aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 shutdown_connections 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 1 -- 172.21.15.17:0/3233049549 wait complete. 2024-03-06T07:24:44.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.131+0000 7fd3f4deb740 20 client.8052 trim_cache size 0 max 16384 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7848, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 12.244064784000001, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 106, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 122, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 122, 2024-03-06T07:24:44.140 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 17.335573133 2024-03-06T07:24:44.141 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:44.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 msgr2=0x7fd3cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4006fd0 tx=0x7fd3d4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 msgr2=0x7fd3ec1ebed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c7c0 tx=0x7fd3dc00cc80 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 shutdown_connections 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd3cc06b4d0 0x7fd3cc06d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3ec155d70 0x7fd3ec1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3ec13e490 0x7fd3ec0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 --2- 172.21.15.17:0/3497592925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3ec0c3b20 0x7fd3ec1ebed0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 >> 172.21.15.17:0/3497592925 conn(0x7fd3ec06afc0 msgr2=0x7fd3ec13d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 shutdown_connections 2024-03-06T07:24:44.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.132+0000 7fd3f0fa0640 1 -- 172.21.15.17:0/3497592925 wait complete. 2024-03-06T07:24:44.144 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 106, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:31.239352+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 106, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7848": { "addr": "172.21.15.17:6835/410228237", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6834", "nonce": 410228237, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 410228237, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7848, "incarnation": 103, "join_fscid": 9, "name": "c", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:39.914109+0000", "qdb_cluster": [ 7848 ], "qdb_leader": 7848, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7848 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1963602121", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1963602121, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1963602121, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6524, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2989703655", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2989703655, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2989703655, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6527, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6833/1367930359", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 1367930359, "type": "v2" }, { "addr": "172.21.15.17:6833", "nonce": 1367930359, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7845, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:44.145 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:44.145 INFO:tasks.cephfs.test_failover:current = [{'gid': 7845, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6833/1367930359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 1367930359}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 1367930359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 6527, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2989703655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2989703655}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2989703655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 7848, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.17:6835/410228237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 410228237}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 410228237}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6524, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1963602121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1963602121}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1963602121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}] 2024-03-06T07:24:44.146 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:24:44.146 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 9:0 2024-03-06T07:24:44.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.246+0000 7fedb175c640 1 Processor -- start 2024-03-06T07:24:44.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.246+0000 7fedb175c640 1 -- start start 2024-03-06T07:24:44.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedb175c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fedac0c7170 0x7fedac0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedb175c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fedac0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedb175c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac0a8390 con 0x7fedac06ff10 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedb175c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac0c7620 con 0x7fedac0c7170 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedb175c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0730f0 con 0x7fedac0c6c40 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedaaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fedac06ff10 0x7fedac153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52332/0 (socket says 172.21.15.17:52332) 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedaaffd640 1 -- 172.21.15.17:0/2634853981 learned_addr learned my addr 172.21.15.17:0/2634853981 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7fedab7fe640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fedac0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.247+0000 7feda3fff640 1 --2- 172.21.15.17:0/2634853981 >> v2:172.21.15.163:3300/0 conn(0x7fedac0c7170 0x7fedac0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072482639 0 0) 0x7fedac0a8390 con 0x7fedac06ff10 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed98003520 con 0x7fedac06ff10 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945243935 0 0) 0x7fed98003520 con 0x7fedac06ff10 2024-03-06T07:24:44.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 msgr2=0x7fedac0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fedac0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 >> v2:172.21.15.163:3300/0 conn(0x7fedac0c7170 msgr2=0x7fedac0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/2634853981 >> v2:172.21.15.163:3300/0 conn(0x7fedac0c7170 0x7fedac0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0733d0 con 0x7fedac06ff10 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fed9c001650 con 0x7fedac06ff10 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 >> v1:172.21.15.17:6789/0 conn(0x7fedac06ff10 legacy=0x7fedac153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fed98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed98004800 0x7fed98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fed980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0c7620 con 0x7fed980039b0 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0730f0 con 0x7fed98004800 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9800ba40 con 0x7fedac0c6c40 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fed98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7feda3fff640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed98004800 0x7fed98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedab7fe640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fed980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 -- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 msgr2=0x7fed980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0c6c40 0x7fed980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 -- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed98004800 msgr2=0x7fed98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed98004800 0x7fed98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.248+0000 7fedaaffd640 1 -- 172.21.15.17:0/2634853981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98003520 con 0x7fed980039b0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.249+0000 7fedaaffd640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fed98003dd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c210 tx=0x7fed98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af250c8c962446f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.249+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007cf0 con 0x7fed980039b0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.249+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c030030 con 0x7fed980039b0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.249+0000 7fedaa7fc640 1 -- 172.21.15.17:0/2634853981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c004d00 con 0x7fed980039b0 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 -- 172.21.15.17:0/2634853981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 msgr2=0x7fed98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 --2- 172.21.15.17:0/2634853981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fed98003dd0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c210 tx=0x7fed98005830 comp rx=0 tx=0).stop 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedab7fe640 1 -- 172.21.15.17:0/2634853981 reap_dead start 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 -- 172.21.15.17:0/2634853981 shutdown_connections 2024-03-06T07:24:44.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 -- 172.21.15.17:0/2634853981 >> 172.21.15.17:0/2634853981 conn(0x7fedac06b970 msgr2=0x7fedac0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 -- 172.21.15.17:0/2634853981 shutdown_connections 2024-03-06T07:24:44.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.250+0000 7fedb175c640 1 -- 172.21.15.17:0/2634853981 wait complete. 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.252+0000 7fedb175c640 1 Processor -- start 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.252+0000 7fedb175c640 1 -- start start 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0acb40 0x7fedac1ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 0x7fedac14bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1544b0 con 0x7fed980039b0 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac154330 con 0x7fedac14b6f0 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedb175c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0702f0 con 0x7fedac0acb40 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40644/0 (socket says 172.21.15.17:40644) 2024-03-06T07:24:44.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 -- 172.21.15.17:0/3958938716 learned_addr learned my addr 172.21.15.17:0/3958938716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7feda3fff640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0acb40 0x7fedac1ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedab7fe640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 0x7fedac14bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 -- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0acb40 msgr2=0x7fedac1ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0acb40 0x7fedac1ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 -- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 msgr2=0x7fedac14bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 0x7fedac14bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 -- 172.21.15.17:0/3958938716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac154870 con 0x7fed980039b0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedab7fe640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 0x7fedac14bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.253+0000 7fedaaffd640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c330 tx=0x7fed9c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.254+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02de60 con 0x7fed980039b0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.254+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c007cd0 con 0x7fed980039b0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.254+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007310 con 0x7fed980039b0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.254+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac0bdf00 con 0x7fed980039b0 2024-03-06T07:24:44.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.254+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac0ac340 con 0x7fed980039b0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.255+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac04f330 con 0x7fed980039b0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.255+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed9c0074b0 con 0x7fed980039b0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.255+0000 7feda37fe640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.256+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed9c0aadf0 con 0x7fed980039b0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.256+0000 7feda3fff640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.257+0000 7feda3fff640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 0x7fed8c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fedac0ad580 tx=0x7fed94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.260+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fedac154870 con 0x7fed980039b0 2024-03-06T07:24:44.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.512+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "9:0"} v 0) v1 -- 0x7fedac06fa80 con 0x7fed980039b0 2024-03-06T07:24:44.514 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:24:44.512+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:24:44.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.514+0000 7feda37fe640 1 -- 172.21.15.17:0/3958938716 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "9:0"}]=0 failed mds gid 7848 v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7fed9c079b70 con 0x7fed980039b0 2024-03-06T07:24:44.515 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 7848 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 msgr2=0x7fed8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 0x7fed8c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fedac0ad580 tx=0x7fed94008040 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 msgr2=0x7fedac1552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c330 tx=0x7fed9c001a20 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 shutdown_connections 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed8c06b4d0 0x7fed8c06d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedac14b6f0 0x7fedac14bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedac0acb40 0x7fedac1ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 --2- 172.21.15.17:0/3958938716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed980039b0 0x7fedac1552c0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 >> 172.21.15.17:0/3958938716 conn(0x7fedac06b970 msgr2=0x7fedac1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 shutdown_connections 2024-03-06T07:24:44.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.517+0000 7fedb175c640 1 -- 172.21.15.17:0/3958938716 wait complete. 2024-03-06T07:24:44.519 INFO:tasks.ceph.mds.c.smithi017.stderr: -317> 2024-03-06T07:24:37.168+0000 7ff5d2948640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:44.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:44.545 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:24:44.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.633+0000 7fac7dc29640 1 Processor -- start 2024-03-06T07:24:44.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.633+0000 7fac7dc29640 1 -- start start 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac7dc29640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 0x7fac78155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac7dc29640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac780c88c0 0x7fac780c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac7dc29640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac780c73d0 con 0x7fac780c5880 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac7dc29640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac780c8dd0 con 0x7fac780c88c0 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac7dc29640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac780be2e0 con 0x7fac7814f360 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac76ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac780c88c0 0x7fac780c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac76ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac780c88c0 0x7fac780c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40664/0 (socket says 172.21.15.17:40664) 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac76ffd640 1 -- 172.21.15.17:0/2525125445 learned_addr learned my addr 172.21.15.17:0/2525125445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.634+0000 7fac777fe640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 0x7fac78155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac76ffd640 1 -- 172.21.15.17:0/2525125445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 msgr2=0x7fac78155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac76ffd640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 0x7fac78155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac76ffd640 1 -- 172.21.15.17:0/2525125445 >> v1:172.21.15.163:6789/0 conn(0x7fac780c5880 legacy=0x7fac78158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac76ffd640 1 -- 172.21.15.17:0/2525125445 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780c7670 con 0x7fac780c88c0 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac777fe640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 0x7fac78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac76ffd640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac780c88c0 0x7fac780c5340 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009870 tx=0x7fac6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6b192302d2e10e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac767fc640 1 -- 172.21.15.17:0/2525125445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c035020 con 0x7fac780c88c0 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac767fc640 1 -- 172.21.15.17:0/2525125445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c031640 con 0x7fac780c88c0 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.635+0000 7fac767fc640 1 -- 172.21.15.17:0/2525125445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c02d040 con 0x7fac780c88c0 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 -- 172.21.15.17:0/2525125445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac780c88c0 msgr2=0x7fac780c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac780c88c0 0x7fac780c5340 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009870 tx=0x7fac6c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 -- 172.21.15.17:0/2525125445 shutdown_connections 2024-03-06T07:24:44.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac780c88c0 0x7fac780c5340 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 --2- 172.21.15.17:0/2525125445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac7814f360 0x7fac78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 -- 172.21.15.17:0/2525125445 >> 172.21.15.17:0/2525125445 conn(0x7fac780ac570 msgr2=0x7fac780ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 -- 172.21.15.17:0/2525125445 shutdown_connections 2024-03-06T07:24:44.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.636+0000 7fac7dc29640 1 -- 172.21.15.17:0/2525125445 wait complete. 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.637+0000 7fac7dc29640 1 Processor -- start 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.637+0000 7fac7dc29640 1 -- start start 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac780c5880 0x7fac780ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 0x7fac781488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78159270 con 0x7fac7814f360 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac780c7c30 con 0x7fac780c88c0 2024-03-06T07:24:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac7dc29640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac781593f0 con 0x7fac780c5880 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40666/0 (socket says 172.21.15.17:40666) 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 -- 172.21.15.17:0/829551421 learned_addr learned my addr 172.21.15.17:0/829551421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac777fe640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac780c5880 0x7fac780ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac76ffd640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 0x7fac781488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 -- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac780c5880 msgr2=0x7fac780ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac780c5880 0x7fac780ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 -- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 msgr2=0x7fac781488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 0x7fac781488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac77fff640 1 -- 172.21.15.17:0/829551421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac781ee9f0 con 0x7fac7814f360 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.638+0000 7fac76ffd640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 0x7fac781488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac77fff640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c7d0 tx=0x7fac7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70003b70 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac70003d30 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac781eec90 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7000aa90 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.639+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac781582a0 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.640+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7804f330 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.640+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac7000ac30 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.640+0000 7fac5ffff640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 0x7fac6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.641+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fac7008a1d0 con 0x7fac7814f360 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.641+0000 7fac777fe640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 0x7fac6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.641+0000 7fac777fe640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 0x7fac6406d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fac68006fd0 tx=0x7fac68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:44.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.645+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fac70058f00 con 0x7fac7814f360 2024-03-06T07:24:44.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.862+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac7806cda0 con 0x7fac7814f360 2024-03-06T07:24:44.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.863+0000 7fac5ffff640 1 -- 172.21.15.17:0/829551421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4194 (secure 0 0 0) 0x7fac70058800 con 0x7fac7814f360 2024-03-06T07:24:44.865 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:44.865 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":108,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:44.517787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7845":{"gid":7845,"name":"a","rank":0,"incarnation":108,"state":"up:replay","state_seq":1,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:24:44.865 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 108 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 msgr2=0x7fac6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 0x7fac6406d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fac68006fd0 tx=0x7fac68008040 comp rx=0 tx=0).stop 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 msgr2=0x7fac78148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c7d0 tx=0x7fac7000cc90 comp rx=0 tx=0).stop 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 shutdown_connections 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac6406b500 0x7fac6406d9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac7814f360 0x7fac78148e20 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac780c88c0 0x7fac781488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 --2- 172.21.15.17:0/829551421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac780c5880 0x7fac780ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:44.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 >> 172.21.15.17:0/829551421 conn(0x7fac780ac570 msgr2=0x7fac780c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:44.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 shutdown_connections 2024-03-06T07:24:44.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:44.865+0000 7fac7dc29640 1 -- 172.21.15.17:0/829551421 wait complete. 2024-03-06T07:24:45.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:44.517787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7845': {'gid': 7845, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.17:6833/1367930359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 1367930359}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 1367930359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:24:45.281 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7845:up:replay 2024-03-06T07:24:46.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:46.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.393+0000 7f02d8377640 1 Processor -- start 2024-03-06T07:24:46.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.393+0000 7f02d8377640 1 -- start start 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d8377640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02d0152920 0x7f02d0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d8377640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02d00c8a00 0x7f02d0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d8377640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d0156180 con 0x7f02d0159460 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d8377640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d01587b0 con 0x7f02d00c8a00 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d8377640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d00c7760 con 0x7f02d0152920 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5912640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f02d00c8a00 0x7f02d0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d6914640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f02d0159460 0x7f02d00c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52354/0 (socket says 172.21.15.17:52354) 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d6914640 1 -- 172.21.15.17:0/4232463389 learned_addr learned my addr 172.21.15.17:0/4232463389 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:46.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d6113640 1 --2- 172.21.15.17:0/4232463389 >> v2:172.21.15.163:3300/0 conn(0x7f02d0152920 0x7f02d0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5912640 1 -- 172.21.15.17:0/4232463389 >> v2:172.21.15.163:3300/0 conn(0x7f02d0152920 msgr2=0x7f02d0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5912640 1 --2- 172.21.15.17:0/4232463389 >> v2:172.21.15.163:3300/0 conn(0x7f02d0152920 0x7f02d0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5912640 1 -- 172.21.15.17:0/4232463389 >> v1:172.21.15.17:6789/0 conn(0x7f02d0159460 legacy=0x7f02d00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5912640 1 -- 172.21.15.17:0/4232463389 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00c7a40 con 0x7f02d00c8a00 2024-03-06T07:24:46.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.394+0000 7f02d5111640 1 -- 172.21.15.17:0/4232463389 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656349260 0 0) 0x7f02cc001490 con 0x7f02d0159460 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.395+0000 7f02d5912640 1 --2- 172.21.15.17:0/4232463389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d00c8a00 0x7f02d0158f20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f02d00be490 tx=0x7f02c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178235306cdb8b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.395+0000 7f02d5111640 1 -- 172.21.15.17:0/4232463389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00315a0 con 0x7f02d00c8a00 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.395+0000 7f02d5111640 1 -- 172.21.15.17:0/4232463389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c0037070 con 0x7f02d00c8a00 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.395+0000 7f02d5111640 1 -- 172.21.15.17:0/4232463389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0031960 con 0x7f02d00c8a00 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 -- 172.21.15.17:0/4232463389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d00c8a00 msgr2=0x7f02d0158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 --2- 172.21.15.17:0/4232463389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d00c8a00 0x7f02d0158f20 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f02d00be490 tx=0x7f02c002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 -- 172.21.15.17:0/4232463389 shutdown_connections 2024-03-06T07:24:46.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 --2- 172.21.15.17:0/4232463389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d00c8a00 0x7f02d0158f20 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 --2- 172.21.15.17:0/4232463389 >> v2:172.21.15.163:3300/0 conn(0x7f02d0152920 0x7f02d0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 -- 172.21.15.17:0/4232463389 >> 172.21.15.17:0/4232463389 conn(0x7f02d00ac560 msgr2=0x7f02d00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 -- 172.21.15.17:0/4232463389 shutdown_connections 2024-03-06T07:24:46.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.396+0000 7f02d8377640 1 -- 172.21.15.17:0/4232463389 wait complete. 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.397+0000 7f02d8377640 1 Processor -- start 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.397+0000 7f02d8377640 1 -- start start 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02d00c8a00 0x7f02d014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d0152920 0x7f02d00c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d006d030 con 0x7f02d014e3f0 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0158a00 con 0x7f02d0152920 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d8377640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00c5a40 con 0x7f02d00c8a00 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40674/0 (socket says 172.21.15.17:40674) 2024-03-06T07:24:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 -- 172.21.15.17:0/2984889356 learned_addr learned my addr 172.21.15.17:0/2984889356 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d5912640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d0152920 0x7f02d00c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6113640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02d00c8a00 0x7f02d014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 -- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02d00c8a00 msgr2=0x7f02d014b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02d00c8a00 0x7f02d014b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 -- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d0152920 msgr2=0x7f02d00c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d0152920 0x7f02d00c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 -- 172.21.15.17:0/2984889356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d014d9b0 con 0x7f02d014e3f0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.398+0000 7f02d6914640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c6d0 tx=0x7f02cc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.399+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc018070 con 0x7f02d014e3f0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.399+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc004d60 con 0x7f02d014e3f0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.399+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d014dc90 con 0x7f02d014e3f0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.399+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00a880 con 0x7f02d014e3f0 2024-03-06T07:24:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.399+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d00c2b20 con 0x7f02d014e3f0 2024-03-06T07:24:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.400+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d0067910 con 0x7f02d014e3f0 2024-03-06T07:24:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.400+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02cc01b070 con 0x7f02d014e3f0 2024-03-06T07:24:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.401+0000 7f02c6ffd640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 0x7f02b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:46.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.401+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f02cc089960 con 0x7f02d014e3f0 2024-03-06T07:24:46.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.401+0000 7f02d6113640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 0x7f02b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:46.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.402+0000 7f02d6113640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 0x7f02b406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.405+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f02d0067910 con 0x7f02d014e3f0 2024-03-06T07:24:46.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.628+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d004f330 con 0x7f02d014e3f0 2024-03-06T07:24:46.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.629+0000 7f02c6ffd640 1 -- 172.21.15.17:0/2984889356 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f02d0156180 con 0x7f02d014e3f0 2024-03-06T07:24:46.631 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:46.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":110,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8064,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:45.526343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7845":{"gid":7845,"name":"a","rank":0,"incarnation":108,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.17:6833/1367930359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":1367930359},{"type":"v1","addr":"172.21.15.17:6833","nonce":1367930359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:24:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 110 2024-03-06T07:24:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 msgr2=0x7f02b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 0x7f02b406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f02c8006fd0 tx=0x7f02c8008040 comp rx=0 tx=0).stop 2024-03-06T07:24:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 msgr2=0x7f02d00c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c6d0 tx=0x7f02cc00cb90 comp rx=0 tx=0).stop 2024-03-06T07:24:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 shutdown_connections 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f02b406b4d0 0x7f02b406d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d014e3f0 0x7f02d00c3580 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02d0152920 0x7f02d00c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 --2- 172.21.15.17:0/2984889356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02d00c8a00 0x7f02d014b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 >> 172.21.15.17:0/2984889356 conn(0x7f02d00ac560 msgr2=0x7f02d0150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 shutdown_connections 2024-03-06T07:24:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:46.631+0000 7f02d8377640 1 -- 172.21.15.17:0/2984889356 wait complete. 2024-03-06T07:24:46.865 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:24:47.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:45.526343+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7845': {'gid': 7845, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.17:6833/1367930359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 1367930359}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 1367930359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:24:47.046 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7845:up:rejoin 2024-03-06T07:24:48.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.159+0000 7f416724e640 1 Processor -- start 2024-03-06T07:24:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.159+0000 7f416724e640 1 -- start start 2024-03-06T07:24:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f416724e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f416724e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f416724e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4160156180 con 0x7f41600c8a00 2024-03-06T07:24:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f416724e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41601587b0 con 0x7f41600c5330 2024-03-06T07:24:48.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f416724e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4160159630 con 0x7f4160152920 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f4164fea640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f41657eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f41657eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:48052/0 (socket says 172.21.15.17:48052) 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f415ffff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f41600c8a00 0x7f4160158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46980/0 (socket says 172.21.15.17:46980) 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f4164fea640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52548/0 (socket says 172.21.15.17:52548) 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.160+0000 7f41657eb640 1 -- 172.21.15.17:0/715143073 learned_addr learned my addr 172.21.15.17:0/715143073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.161+0000 7f4164fea640 1 -- 172.21.15.17:0/715143073 >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 msgr2=0x7f41600c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.161+0000 7f4164fea640 1 --2- 172.21.15.17:0/715143073 >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.161+0000 7f4164fea640 1 -- 172.21.15.17:0/715143073 >> v1:172.21.15.163:6789/0 conn(0x7f41600c8a00 legacy=0x7f4160158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.161+0000 7f4164fea640 1 -- 172.21.15.17:0/715143073 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600c7760 con 0x7f4160152920 2024-03-06T07:24:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.161+0000 7f41657eb640 1 --2- 172.21.15.17:0/715143073 >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.162+0000 7f4164fea640 1 --2- 172.21.15.17:0/715143073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41500097a0 tx=0x7f415002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38db9f82521ca0d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.162+0000 7f415f7fe640 1 -- 172.21.15.17:0/715143073 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41500315a0 con 0x7f4160152920 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.162+0000 7f415f7fe640 1 -- 172.21.15.17:0/715143073 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4150037070 con 0x7f4160152920 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.162+0000 7f415f7fe640 1 -- 172.21.15.17:0/715143073 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150031960 con 0x7f4160152920 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 -- 172.21.15.17:0/715143073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 msgr2=0x7f4160155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 --2- 172.21.15.17:0/715143073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f41500097a0 tx=0x7f415002b840 comp rx=0 tx=0).stop 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 -- 172.21.15.17:0/715143073 shutdown_connections 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 --2- 172.21.15.17:0/715143073 >> v2:172.21.15.163:3300/0 conn(0x7f41600c5330 0x7f41600c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 --2- 172.21.15.17:0/715143073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4160152920 0x7f4160155c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 -- 172.21.15.17:0/715143073 >> 172.21.15.17:0/715143073 conn(0x7f41600ac560 msgr2=0x7f41600ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 -- 172.21.15.17:0/715143073 shutdown_connections 2024-03-06T07:24:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.163+0000 7f416724e640 1 -- 172.21.15.17:0/715143073 wait complete. 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.164+0000 7f416724e640 1 Processor -- start 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.164+0000 7f416724e640 1 -- start start 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41600c5330 0x7f41601ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41600c8a00 0x7f41600ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4160158a00 con 0x7f4160152920 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600c7d20 con 0x7f41600c5330 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f416724e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416006d030 con 0x7f41600c8a00 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40696/0 (socket says 172.21.15.17:40696) 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 -- 172.21.15.17:0/2816821142 learned_addr learned my addr 172.21.15.17:0/2816821142 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f4164fea640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41600c5330 0x7f41601ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f415ffff640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41600c8a00 0x7f41600ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 -- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41600c8a00 msgr2=0x7f41600ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41600c8a00 0x7f41600ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 -- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41600c5330 msgr2=0x7f41601ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41600c5330 0x7f41601ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.165+0000 7f41657eb640 1 -- 172.21.15.17:0/2816821142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416014e6f0 con 0x7f4160152920 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f41657eb640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f415800efc0 tx=0x7f415800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800cbe0 con 0x7f4160152920 2024-03-06T07:24:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800cd80 con 0x7f4160152920 2024-03-06T07:24:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416014e9d0 con 0x7f4160152920 2024-03-06T07:24:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158003bb0 con 0x7f4160152920 2024-03-06T07:24:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.166+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41600c45a0 con 0x7f4160152920 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.167+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416004f330 con 0x7f4160152920 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.167+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4158005d10 con 0x7f4160152920 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.168+0000 7f415d7fa640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 0x7f414c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.168+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f415808a6b0 con 0x7f4160152920 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.168+0000 7f4164fea640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 0x7f414c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.168+0000 7f4164fea640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 0x7f414c06d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4150036000 tx=0x7f415003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:48.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.172+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4158022020 con 0x7f4160152920 2024-03-06T07:24:48.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.394+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f416006cda0 con 0x7f4160152920 2024-03-06T07:24:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.394+0000 7f415d7fa640 1 -- 172.21.15.17:0/2816821142 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f4158010040 con 0x7f4160152920 2024-03-06T07:24:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:48.396 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":112,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:46.847185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":9}]} 2024-03-06T07:24:48.396 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 112 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 msgr2=0x7f414c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 0x7f414c06d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4150036000 tx=0x7f415003d040 comp rx=0 tx=0).stop 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 msgr2=0x7f4160147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f415800efc0 tx=0x7f415800c490 comp rx=0 tx=0).stop 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 shutdown_connections 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f414c06b4d0 0x7f414c06d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4160152920 0x7f4160147d50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41600c8a00 0x7f41600ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 --2- 172.21.15.17:0/2816821142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41600c5330 0x7f41601ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 >> 172.21.15.17:0/2816821142 conn(0x7f41600ac560 msgr2=0x7f4160151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 shutdown_connections 2024-03-06T07:24:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:48.396+0000 7f416724e640 1 -- 172.21.15.17:0/2816821142 wait complete. 2024-03-06T07:24:48.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:46.847185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 124, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8064}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8064': {'gid': 8064, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.17:6835/4288775541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 4288775541}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 4288775541}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:24:48.811 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8064:up:reconnect 2024-03-06T07:24:49.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.926+0000 7fc61670c640 1 Processor -- start 2024-03-06T07:24:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.926+0000 7fc61670c640 1 -- start start 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc61670c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc61014f360 0x7fc610155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc61670c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 0x7fc610158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc61670c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6100c73d0 con 0x7fc6100c88c0 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc61670c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6100c8dd0 con 0x7fc61014f360 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc61670c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100be2e0 con 0x7fc610158b60 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc614ca9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 0x7fc610158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc60ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc61014f360 0x7fc610155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc60ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc61014f360 0x7fc610155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:48074/0 (socket says 172.21.15.17:48074) 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc60ffff640 1 -- 172.21.15.17:0/3489153007 learned_addr learned my addr 172.21.15.17:0/3489153007 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc614ca9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 0x7fc610158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:52560/0 (socket says 172.21.15.17:52560) 2024-03-06T07:24:49.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.927+0000 7fc60f7fe640 1 --1- 172.21.15.17:0/3489153007 >> v1:172.21.15.163:6789/0 conn(0x7fc6100c88c0 0x7fc6100c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46990/0 (socket says 172.21.15.17:46990) 2024-03-06T07:24:49.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.928+0000 7fc60ffff640 1 -- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 msgr2=0x7fc610158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.928+0000 7fc60ffff640 1 --2- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 0x7fc610158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.928+0000 7fc60ffff640 1 -- 172.21.15.17:0/3489153007 >> v1:172.21.15.163:6789/0 conn(0x7fc6100c88c0 legacy=0x7fc6100c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.928+0000 7fc60ffff640 1 -- 172.21.15.17:0/3489153007 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6100c7670 con 0x7fc61014f360 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc60ffff640 1 --2- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61014f360 0x7fc610155c10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009870 tx=0x7fc5f802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fce6d8ab3bbe896 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc60effd640 1 -- 172.21.15.17:0/3489153007 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8035020 con 0x7fc61014f360 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc60effd640 1 -- 172.21.15.17:0/3489153007 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8031640 con 0x7fc61014f360 2024-03-06T07:24:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc60effd640 1 -- 172.21.15.17:0/3489153007 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f802d040 con 0x7fc61014f360 2024-03-06T07:24:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc61670c640 1 -- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61014f360 msgr2=0x7fc610155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.929+0000 7fc61670c640 1 --2- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61014f360 0x7fc610155c10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009870 tx=0x7fc5f802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 -- 172.21.15.17:0/3489153007 shutdown_connections 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 --2- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc610158b60 0x7fc610158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 --2- 172.21.15.17:0/3489153007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc61014f360 0x7fc610155c10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 -- 172.21.15.17:0/3489153007 >> 172.21.15.17:0/3489153007 conn(0x7fc6100ac570 msgr2=0x7fc6100ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 -- 172.21.15.17:0/3489153007 shutdown_connections 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.930+0000 7fc61670c640 1 -- 172.21.15.17:0/3489153007 wait complete. 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.931+0000 7fc61670c640 1 Processor -- start 2024-03-06T07:24:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.932+0000 7fc61670c640 1 -- start start 2024-03-06T07:24:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc61014f360 0x7fc6100afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc610158b60 0x7fc6100b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100c5a50 con 0x7fc6100c88c0 2024-03-06T07:24:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100c7c30 con 0x7fc610158b60 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc61670c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100c5bd0 con 0x7fc61014f360 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40698/0 (socket says 172.21.15.17:40698) 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 -- 172.21.15.17:0/1616554330 learned_addr learned my addr 172.21.15.17:0/1616554330 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc614ca9640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc610158b60 0x7fc6100b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60f7fe640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc61014f360 0x7fc6100afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 -- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc61014f360 msgr2=0x7fc6100afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc61014f360 0x7fc6100afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 -- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc610158b60 msgr2=0x7fc6100b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc610158b60 0x7fc6100b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 -- 172.21.15.17:0/1616554330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6101ebc60 con 0x7fc6100c88c0 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.933+0000 7fc60ffff640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8000c00 tx=0x7fc5f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.934+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8040070 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.934+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f80399f0 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.934+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6100bf6f0 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.934+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8035040 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.934+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101ebf00 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.935+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61004f330 con 0x7fc6100c88c0 2024-03-06T07:24:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.935+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5f804b060 con 0x7fc6100c88c0 2024-03-06T07:24:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.936+0000 7fc60cff9640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.936+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc5f80abb10 con 0x7fc6100c88c0 2024-03-06T07:24:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.936+0000 7fc60f7fe640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.937+0000 7fc60f7fe640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 0x7fc5f006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc610153b70 tx=0x7fc600007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:49.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:49.940+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5f807a100 con 0x7fc6100c88c0 2024-03-06T07:24:50.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.158+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc61006cda0 con 0x7fc6100c88c0 2024-03-06T07:24:50.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.159+0000 7fc60cff9640 1 -- 172.21.15.17:0/1616554330 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4992 (secure 0 0 0) 0x7fc5f807e1a0 con 0x7fc6100c88c0 2024-03-06T07:24:50.161 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:50.161 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:49.847563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8064,"qdb_cluster":[8064]},"id":9}]} 2024-03-06T07:24:50.162 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 114 2024-03-06T07:24:50.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.160+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 msgr2=0x7fc5f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.160+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 0x7fc5f006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc610153b70 tx=0x7fc600007e80 comp rx=0 tx=0).stop 2024-03-06T07:24:50.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.160+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 msgr2=0x7fc6100c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.160+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8000c00 tx=0x7fc5f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 shutdown_connections 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc610158b60 0x7fc6100b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc61014f360 0x7fc6100afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 --2- 172.21.15.17:0/1616554330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6100c88c0 0x7fc6100c0ed0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 >> 172.21.15.17:0/1616554330 conn(0x7fc6100ac570 msgr2=0x7fc610158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 shutdown_connections 2024-03-06T07:24:50.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.161+0000 7fc61670c640 1 -- 172.21.15.17:0/1616554330 wait complete. 2024-03-06T07:24:50.530 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:24:50.529+0000 7f84dc7a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:50.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:24:31.239352+0000', 'modified': '2024-03-06T07:24:49.847563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 124, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8064}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8064': {'gid': 8064, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6835/4288775541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 4288775541}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 4288775541}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8064, 'qdb_cluster': [8064]} 2024-03-06T07:24:50.579 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:24:50.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:24:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.679+0000 7fcf00bd7640 1 Processor -- start 2024-03-06T07:24:50.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.679+0000 7fcf00bd7640 1 -- start start 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcf00bd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 0x7fcefc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcf00bd7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcefc0c8950 0x7fcefc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcf00bd7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc155130 con 0x7fcefc0ad2b0 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcf00bd7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc155440 con 0x7fcefc0c8950 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcf00bd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c5d50 con 0x7fcefc0c69f0 2024-03-06T07:24:50.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcefad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcefc0ad2b0 0x7fcefc1558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49688/0 (socket says 172.21.15.17:49688) 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcefad9d640 1 -- 172.21.15.17:0/1406676755 learned_addr learned my addr 172.21.15.17:0/1406676755 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcefa59c640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 0x7fcefc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1406676755 >> v2:172.21.15.163:3300/0 conn(0x7fcefc0c8950 0x7fcefc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675676519 0 0) 0x7fceec001490 con 0x7fcefc0ad2b0 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.681+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee4003520 con 0x7fcefc0ad2b0 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef9d9b640 1 -- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 msgr2=0x7fcefc0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 0x7fcefc0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef9d9b640 1 -- 172.21.15.17:0/1406676755 >> v1:172.21.15.17:6789/0 conn(0x7fcefc0ad2b0 legacy=0x7fcefc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174398136 0 0) 0x7fcee4003520 con 0x7fcefc0ad2b0 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef9d9b640 1 -- 172.21.15.17:0/1406676755 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0c6030 con 0x7fcefc0c8950 2024-03-06T07:24:50.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcefa59c640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 0x7fcefc0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:50.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.682+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c8950 0x7fcefc0acd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcef00097a0 tx=0x7fcef002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96537b302c2dd343 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0037070 con 0x7fcefc0c8950 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0030510 con 0x7fcefc0c8950 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcef959a640 1 -- 172.21.15.17:0/1406676755 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec001bc0 con 0x7fcefc0c8950 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 -- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c8950 msgr2=0x7fcefc0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c8950 0x7fcefc0acd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcef00097a0 tx=0x7fcef002b710 comp rx=0 tx=0).stop 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 -- 172.21.15.17:0/1406676755 shutdown_connections 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c8950 0x7fcefc0acd70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/1406676755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0c69f0 0x7fcefc0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 -- 172.21.15.17:0/1406676755 >> 172.21.15.17:0/1406676755 conn(0x7fcefc0cae70 msgr2=0x7fcefc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 -- 172.21.15.17:0/1406676755 shutdown_connections 2024-03-06T07:24:50.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.683+0000 7fcf00bd7640 1 -- 172.21.15.17:0/1406676755 wait complete. 2024-03-06T07:24:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.684+0000 7fcf00bd7640 1 Processor -- start 2024-03-06T07:24:50.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.684+0000 7fcf00bd7640 1 -- start start 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 0x7fcefc158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 0x7fcefc14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc152a20 con 0x7fcefc0c8950 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc155fe0 con 0x7fcefc0c69f0 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcf00bd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc152ba0 con 0x7fcefc0ad2b0 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49498/0 (socket says 172.21.15.17:49498) 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 -- 172.21.15.17:0/2522924292 learned_addr learned my addr 172.21.15.17:0/2522924292 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefa59c640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 0x7fcefc158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 0x7fcefc14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 -- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 msgr2=0x7fcefc158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 0x7fcefc158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.685+0000 7fcefad9d640 1 -- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 msgr2=0x7fcefc14cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcefad9d640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 0x7fcefc14cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcefad9d640 1 -- 172.21.15.17:0/2522924292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc14d220 con 0x7fcefc0c8950 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcefa59c640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 0x7fcefc158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 0x7fcefc14cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcefad9d640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fceec00cbe0 tx=0x7fceec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fceeaffd640 1 -- 172.21.15.17:0/2522924292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00ad40 con 0x7fcefc0c8950 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fceeaffd640 1 -- 172.21.15.17:0/2522924292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec012400 con 0x7fcefc0c8950 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc1eab60 con 0x7fcefc0c8950 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fceeaffd640 1 -- 172.21.15.17:0/2522924292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec012830 con 0x7fcefc0c8950 2024-03-06T07:24:50.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.686+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc157cf0 con 0x7fcefc0c8950 2024-03-06T07:24:50.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.687+0000 7fceeaffd640 1 -- 172.21.15.17:0/2522924292 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fceec0129d0 con 0x7fcefc0c8950 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.688+0000 7fceeaffd640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 0x7fcedc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.688+0000 7fceeaffd640 1 -- 172.21.15.17:0/2522924292 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fceec08ab60 con 0x7fcefc0c8950 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.688+0000 7fcefa59c640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 0x7fcedc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.689+0000 7fcf037c1740 1 Processor -- start 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.689+0000 7fcefa59c640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 0x7fcedc06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009ac0 tx=0x7fcef4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.689+0000 7fcf037c1740 1 -- start start 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.689+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f4f50 0x5583344f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.689+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 0x5583344ef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcf037c1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583344596b0 con 0x5583344f5330 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334402a60 con 0x5583344f4f50 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558334455750 con 0x5583344f34b0 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49512/0 (socket says 172.21.15.17:49512) 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 -- 172.21.15.17:0/4048682226 learned_addr learned my addr 172.21.15.17:0/4048682226 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcefa59c640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f4f50 0x5583344f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcefad9d640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 0x5583344ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 -- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 msgr2=0x5583344ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 0x5583344ef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 -- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f4f50 msgr2=0x5583344f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f4f50 0x5583344f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 -- 172.21.15.17:0/4048682226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558334420f30 con 0x5583344f5330 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcefad9d640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 0x5583344ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcef0035040 tx=0x7fcef002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddd0b82becac3bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcedbfff640 1 -- 172.21.15.17:0/4048682226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0037040 con 0x5583344f5330 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcedbfff640 1 -- 172.21.15.17:0/4048682226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0030910 con 0x5583344f5330 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.690+0000 7fcedbfff640 1 -- 172.21.15.17:0/4048682226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0030cc0 con 0x5583344f5330 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 -- 172.21.15.17:0/4048682226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 msgr2=0x5583344f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcef0035040 tx=0x7fcef002f000 comp rx=0 tx=0).stop 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 -- 172.21.15.17:0/4048682226 shutdown_connections 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f34b0 0x5583344ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x5583344f2f70 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 --2- 172.21.15.17:0/4048682226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f4f50 0x5583344f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 -- 172.21.15.17:0/4048682226 >> 172.21.15.17:0/4048682226 conn(0x5583344f17d0 msgr2=0x5583345163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 -- 172.21.15.17:0/4048682226 shutdown_connections 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.691+0000 7fcf037c1740 1 -- 172.21.15.17:0/4048682226 wait complete. 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 -- start start 2024-03-06T07:24:50.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 Processor -- start 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f4f50 0x5583345124f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x558334544b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334517230 con 0x5583344f5330 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334415ea0 con 0x55833451aa30 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.692+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558334516c70 con 0x5583344f4f50 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcf037c1740 10 client.0 _ll_register_callbacks cb 0x558334415870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49520/0 (socket says 172.21.15.17:49520) 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 -- 172.21.15.17:0/952696467 learned_addr learned my addr 172.21.15.17:0/952696467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcefa59c640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f4f50 0x5583345124f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcefad9d640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x558334544b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 -- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f4f50 msgr2=0x5583345124f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f4f50 0x5583345124f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 -- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 msgr2=0x558334544b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x558334544b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 -- 172.21.15.17:0/952696467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558334416270 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcefad9d640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x558334544b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcef00098c0 tx=0x7fcef0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62ba3317d234f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fced97fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0037040 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fcef0004240 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcef0030910 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef004e050 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcf037c1740 10 client.8103 fetch_fsmap learned FSMap version 114 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.693+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x55833416e2d0 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0030910 con 0x5583344f5330 2024-03-06T07:24:50.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 114) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcef0051020 con 0x5583344f5330 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 10 client.8103 fetch_fsmap finished waiting for FSMap version 114 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 10 client.8103 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 20 client.8103 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x558334517f10 0x558334550180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 4 client.8103 mds_command: new command op to 8064 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 --> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558334466bf0 con 0x558334517f10 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcefa59c640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x558334517f10 0x558334550180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fced97fa640 10 client.8103 ms_handle_connect on v2:172.21.15.17:6834/4288775541 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.694+0000 7fcefa59c640 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x558334517f10 0x558334550180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b16177ba269a878 server_cookie=c86168e5c4d920de in_seq=0 out_seq=0 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fced97fa640 1 -- 172.21.15.17:0/952696467 <== mds.0 v2:172.21.15.17:6834/4288775541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558334466bf0 con 0x558334517f10 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fced97fa640 10 client.8103 handle_command_reply: tid=0 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 client.8103 shutdown 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 msgr2=0x558334530630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcef00098c0 tx=0x7fcef0004730 comp rx=0 tx=0).stop 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 shutdown_connections 2024-03-06T07:24:50.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x558334517f10 0x558334550180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x558334544b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f5330 0x558334530630 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 --2- 172.21.15.17:0/952696467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f4f50 0x5583345124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fced97fa640 0 client.8103 ms_handle_reset on v2:172.21.15.17:6834/4288775541 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 >> 172.21.15.17:0/952696467 conn(0x5583344f34b0 msgr2=0x5583344f38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 shutdown_connections 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 1 -- 172.21.15.17:0/952696467 wait complete. 2024-03-06T07:24:50.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.695+0000 7fcf037c1740 20 client.8103 trim_cache size 0 max 16384 2024-03-06T07:24:50.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 Processor -- start 2024-03-06T07:24:50.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 -- start start 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f5330 0x5583344ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55833451aa30 0x5583344f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1f6fe0 con 0x5583344f4f50 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1f6610 con 0x5583344f5330 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1f6a60 con 0x55833451aa30 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcefa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.712+0000 7fcefa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49524/0 (socket says 172.21.15.17:49524) 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 -- 172.21.15.17:0/3462533032 learned_addr learned my addr 172.21.15.17:0/3462533032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f5330 0x5583344ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefad9d640 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55833451aa30 0x5583344f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 -- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55833451aa30 msgr2=0x5583344f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55833451aa30 0x5583344f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 -- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f5330 msgr2=0x5583344ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f5330 0x5583344ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 -- 172.21.15.17:0/3462533032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583342386b0 con 0x5583344f4f50 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcefa59c640 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x558334517960 tx=0x7fcef4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a14a569ba9e820d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcedb7fe640 1 -- 172.21.15.17:0/3462533032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401c040 con 0x5583344f4f50 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcedb7fe640 1 -- 172.21.15.17:0/3462533032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef401d040 con 0x5583344f4f50 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.713+0000 7fcedb7fe640 1 -- 172.21.15.17:0/3462533032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef400f040 con 0x5583344f4f50 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- 172.21.15.17:0/3462533032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 msgr2=0x55833451c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x558334517960 tx=0x7fcef4013000 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- 172.21.15.17:0/3462533032 shutdown_connections 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55833451aa30 0x5583344f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5583344f5330 0x5583344ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 --2- 172.21.15.17:0/3462533032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x55833451c050 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- 172.21.15.17:0/3462533032 >> 172.21.15.17:0/3462533032 conn(0x5583344f34b0 msgr2=0x558334533430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- 172.21.15.17:0/3462533032 shutdown_connections 2024-03-06T07:24:50.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- 172.21.15.17:0/3462533032 wait complete. 2024-03-06T07:24:50.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 -- start start 2024-03-06T07:24:50.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.714+0000 7fcf037c1740 1 Processor -- start 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f5330 0x55833452fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x55833459d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833451cc00 con 0x5583344f4f50 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583345316e0 con 0x55833451aa30 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558334517a30 con 0x5583344f5330 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcf037c1740 10 client.0 _ll_register_callbacks cb 0x7fcef4009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcefa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcefa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49540/0 (socket says 172.21.15.17:49540) 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcefa59c640 1 -- 172.21.15.17:0/1501632404 learned_addr learned my addr 172.21.15.17:0/1501632404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcefad9d640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x55833459d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.715+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f5330 0x55833452fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 -- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f5330 msgr2=0x55833452fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f5330 0x55833452fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 -- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 msgr2=0x55833459d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x55833459d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 -- 172.21.15.17:0/1501632404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583344f5be0 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcefa59c640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcef4008010 tx=0x7fcef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc166d1078880d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4038060 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fcef402b730 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcef401d040 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcef403e050 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 10 client.8109 fetch_fsmap learned FSMap version 114 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x5583344f25c0 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef401d040 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 114) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcef4041020 con 0x5583344f4f50 2024-03-06T07:24:50.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 10 client.8109 fetch_fsmap finished waiting for FSMap version 114 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 10 client.8109 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 20 client.8109 populate_metadata read hostname 'smithi017' 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x5583345182f0 0x5583345186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 4 client.8109 mds_command: new command op to 8064 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.716+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 --> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558334460f40 con 0x5583345182f0 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x5583345182f0 0x5583345186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcef9d9b640 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x5583345182f0 0x5583345186d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48451378bcef9b0f server_cookie=d419954b66ce3460 in_seq=0 out_seq=0 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fced97fa640 10 client.8109 ms_handle_connect on v2:172.21.15.17:6834/4288775541 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fced97fa640 1 -- 172.21.15.17:0/1501632404 <== mds.0 v2:172.21.15.17:6834/4288775541 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x558334460f40 con 0x5583345182f0 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fced97fa640 10 client.8109 handle_command_reply: tid=0 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 client.8109 shutdown 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 msgr2=0x5583344eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcef4008010 tx=0x7fcef40046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 shutdown_connections 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:6834/4288775541,v1:172.21.15.17:6835/4288775541] conn(0x5583345182f0 0x5583345186d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55833451aa30 0x55833459d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5583344f5330 0x55833452fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fced97fa640 0 client.8109 ms_handle_reset on v2:172.21.15.17:6834/4288775541 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 --2- 172.21.15.17:0/1501632404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583344f4f50 0x5583344eec00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 >> 172.21.15.17:0/1501632404 conn(0x5583344f34b0 msgr2=0x55833453e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 shutdown_connections 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 1 -- 172.21.15.17:0/1501632404 wait complete. 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.717+0000 7fcf037c1740 20 client.8109 trim_cache size 0 max 16384 2024-03-06T07:24:50.725 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8064, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 3.8806073969999999, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 114, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 124, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 124, 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 6.1720123149999999 2024-03-06T07:24:50.726 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 msgr2=0x7fcedc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 0x7fcedc06d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009ac0 tx=0x7fcef4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 msgr2=0x7fcefc1541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fceec00cbe0 tx=0x7fceec00a430 comp rx=0 tx=0).stop 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 shutdown_connections 2024-03-06T07:24:50.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcedc06b4d0 0x7fcedc06d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcefc0c8950 0x7fcefc1541d0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcefc0c69f0 0x7fcefc14cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 --2- 172.21.15.17:0/2522924292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcefc0ad2b0 0x7fcefc158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 >> 172.21.15.17:0/2522924292 conn(0x7fcefc0cae70 msgr2=0x7fcefc0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 shutdown_connections 2024-03-06T07:24:50.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.718+0000 7fcf00bd7640 1 -- 172.21.15.17:0/2522924292 wait complete. 2024-03-06T07:24:50.730 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 9, "epoch": 114, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 9, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:24:31.239352+0000", "damaged": [], "data_pools": [ 20 ], "enabled": true, "epoch": 114, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8064": { "addr": "172.21.15.17:6835/4288775541", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6834", "nonce": 4288775541, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 4288775541, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8064, "incarnation": 111, "join_fscid": 9, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 124, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 19, "modified": "2024-03-06T07:24:49.847563+0000", "qdb_cluster": [ 8064 ], "qdb_leader": 8064, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8064 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/1963602121", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 1963602121, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 1963602121, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6524, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2989703655", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2989703655, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2989703655, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 102, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6527, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6833/3939235170", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 3939235170, "type": "v2" }, { "addr": "172.21.15.17:6833", "nonce": 3939235170, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 112, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8085, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:24:50.731 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 9, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:24:50.731 INFO:tasks.cephfs.test_failover:current = [{'gid': 8085, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6833/3939235170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3939235170}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 3939235170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 112}, {'gid': 6527, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2989703655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2989703655}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2989703655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}, {'gid': 8064, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6835/4288775541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 4288775541}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 4288775541}]}, 'join_fscid': 9, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6524, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/1963602121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 1963602121}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 1963602121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 102}] 2024-03-06T07:24:50.732 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-06T07:24:50.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:24:50.732 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:50.773 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:24:50.817 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:24:50.817 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:24:50.817 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:50.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:24:50.833 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:24:50.833 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:24:50.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:24:50.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:50.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.976+0000 7f06c75f6640 1 Processor -- start 2024-03-06T07:24:50.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.976+0000 7f06c75f6640 1 -- start start 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c75f6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f06c00c88c0 0x7f06c0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c75f6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06c0158b60 0x7f06c0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c75f6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00c73d0 con 0x7f06c014f360 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c75f6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00c8dd0 con 0x7f06c0158b60 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c75f6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00be2e0 con 0x7f06c00c88c0 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06c0158b60 0x7f06c0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5392640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f06c014f360 0x7f06c00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49704/0 (socket says 172.21.15.17:49704) 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5392640 1 -- 172.21.15.17:0/629857115 learned_addr learned my addr 172.21.15.17:0/629857115 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c4b91640 1 --2- 172.21.15.17:0/629857115 >> v2:172.21.15.163:3300/0 conn(0x7f06c00c88c0 0x7f06c0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5b93640 1 -- 172.21.15.17:0/629857115 >> v2:172.21.15.163:3300/0 conn(0x7f06c00c88c0 msgr2=0x7f06c0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5b93640 1 --2- 172.21.15.17:0/629857115 >> v2:172.21.15.163:3300/0 conn(0x7f06c00c88c0 0x7f06c0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5b93640 1 -- 172.21.15.17:0/629857115 >> v1:172.21.15.17:6789/0 conn(0x7f06c014f360 legacy=0x7f06c00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06c5b93640 1 -- 172.21.15.17:0/629857115 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00c7670 con 0x7f06c0158b60 2024-03-06T07:24:50.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.977+0000 7f06b7fff640 1 -- 172.21.15.17:0/629857115 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297793862 0 0) 0x7f06c00c73d0 con 0x7f06c014f360 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.978+0000 7f06c5b93640 1 --2- 172.21.15.17:0/629857115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c0158b60 0x7f06c0158ff0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f06c00c84c0 tx=0x7f06b802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f0621683b13a8e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.978+0000 7f06b7fff640 1 -- 172.21.15.17:0/629857115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8039070 con 0x7f06c0158b60 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.978+0000 7f06b7fff640 1 -- 172.21.15.17:0/629857115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b802cbf0 con 0x7f06c0158b60 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06b7fff640 1 -- 172.21.15.17:0/629857115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8034440 con 0x7f06c0158b60 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 -- 172.21.15.17:0/629857115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c0158b60 msgr2=0x7f06c0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 --2- 172.21.15.17:0/629857115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c0158b60 0x7f06c0158ff0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f06c00c84c0 tx=0x7f06b802be40 comp rx=0 tx=0).stop 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 -- 172.21.15.17:0/629857115 shutdown_connections 2024-03-06T07:24:50.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 --2- 172.21.15.17:0/629857115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c0158b60 0x7f06c0158ff0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 --2- 172.21.15.17:0/629857115 >> v2:172.21.15.163:3300/0 conn(0x7f06c00c88c0 0x7f06c0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 -- 172.21.15.17:0/629857115 >> 172.21.15.17:0/629857115 conn(0x7f06c00ac570 msgr2=0x7f06c00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:50.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 -- 172.21.15.17:0/629857115 shutdown_connections 2024-03-06T07:24:50.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.979+0000 7f06c75f6640 1 -- 172.21.15.17:0/629857115 wait complete. 2024-03-06T07:24:50.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.980+0000 7f06c75f6640 1 Processor -- start 2024-03-06T07:24:50.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.980+0000 7f06c75f6640 1 -- start start 2024-03-06T07:24:50.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 0x7f06c01ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 0x7f06c01ec490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c5a50 con 0x7f06c00c88c0 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c7c30 con 0x7f06c014f360 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c75f6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c5bd0 con 0x7f06c0158b60 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49554/0 (socket says 172.21.15.17:49554) 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 -- 172.21.15.17:0/58351085 learned_addr learned my addr 172.21.15.17:0/58351085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c4b91640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 0x7f06c01ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5b93640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 0x7f06c01ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 -- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 msgr2=0x7f06c01ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 0x7f06c01ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 -- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 msgr2=0x7f06c01ebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 0x7f06c01ebf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:50.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5392640 1 -- 172.21.15.17:0/58351085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0158710 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c5b93640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 0x7f06c01ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.981+0000 7f06c4b91640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 0x7f06c01ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06c5392640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c4d0 tx=0x7f06bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc018070 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc007500 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c00befa0 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc004a20 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.982+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c00bf560 con 0x7f06c00c88c0 2024-03-06T07:24:50.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.983+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c004f330 con 0x7f06c00c88c0 2024-03-06T07:24:50.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.983+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06bc01b070 con 0x7f06c00c88c0 2024-03-06T07:24:50.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.984+0000 7f06b5ffb640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 0x7f06a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:50.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.984+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f06bc0896b0 con 0x7f06c00c88c0 2024-03-06T07:24:50.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.984+0000 7f06c4b91640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 0x7f06a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:50.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.985+0000 7f06c4b91640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 0x7f06a406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006fd0 tx=0x7f06b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:50.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:50.988+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f06bc058380 con 0x7f06c00c88c0 2024-03-06T07:24:51.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.207+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06c006cda0 con 0x7f06c00c88c0 2024-03-06T07:24:51.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.208+0000 7f06b5ffb640 1 -- 172.21.15.17:0/58351085 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4992 (secure 0 0 0) 0x7f06bc057c80 con 0x7f06c00c88c0 2024-03-06T07:24:51.209 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:51.210 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:49.847563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8064,"qdb_cluster":[8064]},"id":9}]} 2024-03-06T07:24:51.210 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 114 2024-03-06T07:24:51.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 msgr2=0x7f06a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 0x7f06a406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f06b0006fd0 tx=0x7f06b0008040 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 msgr2=0x7f06c00c0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00c4d0 tx=0x7f06bc00c990 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 shutdown_connections 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f06a406b4d0 0x7f06a406d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06c0158b60 0x7f06c01ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06c014f360 0x7f06c01ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 --2- 172.21.15.17:0/58351085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06c00c88c0 0x7f06c00c0b70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 >> 172.21.15.17:0/58351085 conn(0x7f06c00ac570 msgr2=0x7f06c00c9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 shutdown_connections 2024-03-06T07:24:51.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.210+0000 7f06c75f6640 1 -- 172.21.15.17:0/58351085 wait complete. 2024-03-06T07:24:51.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:51.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 Processor -- start 2024-03-06T07:24:51.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 -- start start 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 0x7f49fc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49fc158b60 0x7f49fc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49fc0c73d0 con 0x7f49fc14f360 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49fc0c8dd0 con 0x7f49fc158b60 2024-03-06T07:24:51.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.325+0000 7f4a01dae640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0be2e0 con 0x7f49fc0c88c0 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49fc158b60 0x7f49fc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49fc158b60 0x7f49fc158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49562/0 (socket says 172.21.15.17:49562) 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 -- 172.21.15.17:0/1301622385 learned_addr learned my addr 172.21.15.17:0/1301622385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49faffd640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 0x7f49fc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 -- 172.21.15.17:0/1301622385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 msgr2=0x7f49fc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 0x7f49fc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 -- 172.21.15.17:0/1301622385 >> v1:172.21.15.163:6789/0 conn(0x7f49fc14f360 legacy=0x7f49fc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49fbfff640 1 -- 172.21.15.17:0/1301622385 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0c7670 con 0x7f49fc158b60 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.326+0000 7f49faffd640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 0x7f49fc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f49fbfff640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc158b60 0x7f49fc158ff0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009e50 tx=0x7f49f402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ab56e0df2755b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f49fa7fc640 1 -- 172.21.15.17:0/1301622385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f402c9e0 con 0x7f49fc158b60 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f49fa7fc640 1 -- 172.21.15.17:0/1301622385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f402cb80 con 0x7f49fc158b60 2024-03-06T07:24:51.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f49fa7fc640 1 -- 172.21.15.17:0/1301622385 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40343f0 con 0x7f49fc158b60 2024-03-06T07:24:51.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 -- 172.21.15.17:0/1301622385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc158b60 msgr2=0x7f49fc158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc158b60 0x7f49fc158ff0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009e50 tx=0x7f49f402be40 comp rx=0 tx=0).stop 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 -- 172.21.15.17:0/1301622385 shutdown_connections 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc158b60 0x7f49fc158ff0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 --2- 172.21.15.17:0/1301622385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc0c88c0 0x7f49fc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 -- 172.21.15.17:0/1301622385 >> 172.21.15.17:0/1301622385 conn(0x7f49fc0ac570 msgr2=0x7f49fc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 -- 172.21.15.17:0/1301622385 shutdown_connections 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.327+0000 7f4a01dae640 1 -- 172.21.15.17:0/1301622385 wait complete. 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.328+0000 7f4a01dae640 1 Processor -- start 2024-03-06T07:24:51.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.329+0000 7f4a01dae640 1 -- start start 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0c88c0 0x7f49fc14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc158b60 0x7f49fc0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0c5a50 con 0x7f49fc14f360 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0c7c30 con 0x7f49fc0c88c0 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f4a01dae640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc0c5bd0 con 0x7f49fc158b60 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49566/0 (socket says 172.21.15.17:49566) 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 -- 172.21.15.17:0/2740707555 learned_addr learned my addr 172.21.15.17:0/2740707555 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49fb7fe640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0c88c0 0x7f49fc14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 -- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc158b60 msgr2=0x7f49fc0c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc158b60 0x7f49fc0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 -- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0c88c0 msgr2=0x7f49fc14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0c88c0 0x7f49fc14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 -- 172.21.15.17:0/2740707555 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc1ee500 con 0x7f49fc14f360 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49faffd640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c7c0 tx=0x7f49f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.330+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0015070 con 0x7f49fc14f360 2024-03-06T07:24:51.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.331+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f00077a0 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.331+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc0c1170 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.331+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0007b70 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.331+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc0bf040 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.331+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc075300 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.332+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49f0018070 con 0x7f49fc14f360 2024-03-06T07:24:51.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.332+0000 7f49e3fff640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 0x7f49e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.333+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f49f008a1c0 con 0x7f49fc14f360 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.333+0000 7f49fb7fe640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 0x7f49e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.333+0000 7f49fb7fe640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 0x7f49e806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00a750 tx=0x7f49ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.337+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49fc075300 con 0x7f49fc14f360 2024-03-06T07:24:51.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.518+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49fc0c8590 con 0x7f49fc14f360 2024-03-06T07:24:51.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.519+0000 7f49e3fff640 1 -- 172.21.15.17:0/2740707555 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17157 (secure 0 0 0) 0x7f49fc0c8590 con 0x7f49fc14f360 2024-03-06T07:24:51.521 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:51.521 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":124,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:46.835679+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6833/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6835/410228237":"2024-03-07T07:24:44.513269+0000","172.21.15.17:6832/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6834/410228237":"2024-03-07T07:24:44.513269+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 msgr2=0x7f49e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 0x7f49e806d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00a750 tx=0x7f49ec008040 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 msgr2=0x7f49fc0ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c7c0 tx=0x7f49f000cc80 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 shutdown_connections 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49e806b4d0 0x7f49e806d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49fc158b60 0x7f49fc0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49fc14f360 0x7f49fc0ab560 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 --2- 172.21.15.17:0/2740707555 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49fc0c88c0 0x7f49fc14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 >> 172.21.15.17:0/2740707555 conn(0x7f49fc0ac570 msgr2=0x7f49fc14b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 shutdown_connections 2024-03-06T07:24:51.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.521+0000 7f4a01dae640 1 -- 172.21.15.17:0/2740707555 wait complete. 2024-03-06T07:24:51.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:24:29.507391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:24:30.198128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:24:51.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:24:51.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.636+0000 7f893ad63640 1 Processor -- start 2024-03-06T07:24:51.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.636+0000 7f893ad63640 1 -- start start 2024-03-06T07:24:51.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.637+0000 7f893ad63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.637+0000 7f893ad63640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89340c5380 0x7f89340c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.637+0000 7f893ad63640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340b0450 con 0x7f893406ca70 2024-03-06T07:24:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.637+0000 7f893ad63640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893406c020 con 0x7f89340c5380 2024-03-06T07:24:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.637+0000 7f893ad63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340c77f0 con 0x7f89340b0070 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8938aff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8938aff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49318/0 (socket says 172.21.15.17:49318) 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8939300640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89340c5380 0x7f89340c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8938aff640 1 -- 172.21.15.17:0/1759904329 learned_addr learned my addr 172.21.15.17:0/1759904329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f892bfff640 1 --1- 172.21.15.17:0/1759904329 >> v1:172.21.15.163:6789/0 conn(0x7f893406ca70 0x7f89340c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59324/0 (socket says 172.21.15.17:59324) 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8939300640 1 -- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 msgr2=0x7f8934155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8939300640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8939300640 1 -- 172.21.15.17:0/1759904329 >> v1:172.21.15.163:6789/0 conn(0x7f893406ca70 legacy=0x7f89340c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8939300640 1 -- 172.21.15.17:0/1759904329 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340c7ad0 con 0x7f89340c5380 2024-03-06T07:24:51.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.638+0000 7f8938aff640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:51.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.639+0000 7f8939300640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c5380 0x7f89340c57d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f892c009d90 tx=0x7f892c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4a5f777659f19a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.639+0000 7f892b7fe640 1 -- 172.21.15.17:0/1759904329 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c039070 con 0x7f89340c5380 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.639+0000 7f892b7fe640 1 -- 172.21.15.17:0/1759904329 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c02cbc0 con 0x7f89340c5380 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.639+0000 7f892b7fe640 1 -- 172.21.15.17:0/1759904329 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c034440 con 0x7f89340c5380 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 -- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c5380 msgr2=0x7f89340c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c5380 0x7f89340c57d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f892c009d90 tx=0x7f892c02be10 comp rx=0 tx=0).stop 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 -- 172.21.15.17:0/1759904329 shutdown_connections 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89340c5380 0x7f89340c57d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 --2- 172.21.15.17:0/1759904329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f8934155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 -- 172.21.15.17:0/1759904329 >> 172.21.15.17:0/1759904329 conn(0x7f89340ac590 msgr2=0x7f89340ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 -- 172.21.15.17:0/1759904329 shutdown_connections 2024-03-06T07:24:51.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.640+0000 7f893ad63640 1 -- 172.21.15.17:0/1759904329 wait complete. 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.641+0000 7f893ad63640 1 Processor -- start 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.641+0000 7f893ad63640 1 -- start start 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.641+0000 7f893ad63640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893406ca70 0x7f89340c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.641+0000 7f893ad63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f89340c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893406cef0 con 0x7f89340c5380 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893406c550 con 0x7f893406ca70 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893406da80 con 0x7f89340b0070 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49570/0 (socket says 172.21.15.17:49570) 2024-03-06T07:24:51.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 -- 172.21.15.17:0/552421928 learned_addr learned my addr 172.21.15.17:0/552421928 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8938aff640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893406ca70 0x7f89340c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f892bfff640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f89340c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 -- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 msgr2=0x7f89340c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f89340c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 -- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893406ca70 msgr2=0x7f89340c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893406ca70 0x7f89340c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 -- 172.21.15.17:0/552421928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340c4d90 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f8939300640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f892c009eb0 tx=0x7f892c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c039050 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c045070 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89340c2010 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c009450 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.642+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340c3830 con 0x7f89340c5380 2024-03-06T07:24:51.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.643+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893404f330 con 0x7f89340c5380 2024-03-06T07:24:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.644+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892c03e850 con 0x7f89340c5380 2024-03-06T07:24:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.645+0000 7f89297fa640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.645+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f892c0ad280 con 0x7f89340c5380 2024-03-06T07:24:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.645+0000 7f8938aff640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.645+0000 7f8938aff640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 0x7f892006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8930009630 tx=0x7f8930009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.649+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f892c07b990 con 0x7f89340c5380 2024-03-06T07:24:51.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.831+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89340c98d0 con 0x7f89340c5380 2024-03-06T07:24:51.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.832+0000 7f89297fa640 1 -- 172.21.15.17:0/552421928 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f892c07fa30 con 0x7f89340c5380 2024-03-06T07:24:51.834 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 msgr2=0x7f892006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 0x7f892006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8930009630 tx=0x7f8930009240 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 msgr2=0x7f8934154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f892c009eb0 tx=0x7f892c004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 shutdown_connections 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f892006b4d0 0x7f892006d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89340c5380 0x7f8934154dd0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89340b0070 0x7f89340c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 --2- 172.21.15.17:0/552421928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f893406ca70 0x7f89340c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 >> 172.21.15.17:0/552421928 conn(0x7f89340ac590 msgr2=0x7f89340ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 shutdown_connections 2024-03-06T07:24:51.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.835+0000 7f893ad63640 1 -- 172.21.15.17:0/552421928 wait complete. 2024-03-06T07:24:51.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:24:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 Processor -- start 2024-03-06T07:24:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 -- start start 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e80a8290 0x7fd8e81539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80c4740 0x7fd8e80701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8e80c69a0 0x7fd8e8072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e8153f00 con 0x7fd8e80a8290 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80c62a0 con 0x7fd8e80c69a0 2024-03-06T07:24:51.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8eeeaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80706e0 con 0x7fd8e80c4740 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.965+0000 7fd8ecc4b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e80a8290 0x7fd8e81539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8e80a8290 0x7fd8e81539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49586/0 (socket says 172.21.15.17:49586) 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 -- 172.21.15.17:0/2154172585 learned_addr learned my addr 172.21.15.17:0/2154172585 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ed44c640 1 --2- 172.21.15.17:0/2154172585 >> v2:172.21.15.163:3300/0 conn(0x7fd8e80c69a0 0x7fd8e8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8dffff640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80c4740 0x7fd8e80701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 -- 172.21.15.17:0/2154172585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80c4740 msgr2=0x7fd8e80701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80c4740 0x7fd8e80701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 -- 172.21.15.17:0/2154172585 >> v2:172.21.15.163:3300/0 conn(0x7fd8e80c69a0 msgr2=0x7fd8e8072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 --2- 172.21.15.17:0/2154172585 >> v2:172.21.15.163:3300/0 conn(0x7fd8e80c69a0 0x7fd8e8072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.966+0000 7fd8ecc4b640 1 -- 172.21.15.17:0/2154172585 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80c4c00 con 0x7fd8e80a8290 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8ecc4b640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80a8290 0x7fd8e81539c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8157570 tx=0x7fd8e402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60232bb4498968e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8df7fe640 1 -- 172.21.15.17:0/2154172585 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4038070 con 0x7fd8e80a8290 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8df7fe640 1 -- 172.21.15.17:0/2154172585 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e402f050 con 0x7fd8e80a8290 2024-03-06T07:24:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8df7fe640 1 -- 172.21.15.17:0/2154172585 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4031970 con 0x7fd8e80a8290 2024-03-06T07:24:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/2154172585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80a8290 msgr2=0x7fd8e81539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80a8290 0x7fd8e81539c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8157570 tx=0x7fd8e402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/2154172585 shutdown_connections 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/2154172585 >> v2:172.21.15.163:3300/0 conn(0x7fd8e80c69a0 0x7fd8e8072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80c4740 0x7fd8e80701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/2154172585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80a8290 0x7fd8e81539c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.967+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/2154172585 >> 172.21.15.17:0/2154172585 conn(0x7fd8e80c5560 msgr2=0x7fd8e80c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.968+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/2154172585 shutdown_connections 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.968+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/2154172585 wait complete. 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.969+0000 7fd8eeeaf640 1 Processor -- start 2024-03-06T07:24:51.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.969+0000 7fd8eeeaf640 1 -- start start 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80a8290 0x7fd8e81ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e80c69a0 0x7fd8e81de050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e806fc80 con 0x7fd8e80c4740 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e806f9a0 con 0x7fd8e80c69a0 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.970+0000 7fd8eeeaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c6e10 con 0x7fd8e80a8290 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49588/0 (socket says 172.21.15.17:49588) 2024-03-06T07:24:51.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 -- 172.21.15.17:0/1329293822 learned_addr learned my addr 172.21.15.17:0/1329293822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 -- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80a8290 msgr2=0x7fd8e81ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80a8290 0x7fd8e81ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 -- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e80c69a0 msgr2=0x7fd8e81de050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e80c69a0 0x7fd8e81de050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 -- 172.21.15.17:0/1329293822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e81ecf90 con 0x7fd8e80c4740 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dffff640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800d830 tx=0x7fd8d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800b6c0 con 0x7fd8e80c4740 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d8019070 con 0x7fd8e80c4740 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e80abf50 con 0x7fd8e80c4740 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800ba80 con 0x7fd8e80c4740 2024-03-06T07:24:51.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.971+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e8153bd0 con 0x7fd8e80c4740 2024-03-06T07:24:51.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.972+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e804f330 con 0x7fd8e80c4740 2024-03-06T07:24:51.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.973+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8d801e460 con 0x7fd8e80c4740 2024-03-06T07:24:51.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.973+0000 7fd8dd7fa640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:51.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.973+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd8d808ab20 con 0x7fd8e80c4740 2024-03-06T07:24:51.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.973+0000 7fd8ecc4b640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:51.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.974+0000 7fd8ecc4b640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 0x7fd8d406d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4035000 tx=0x7fd8e4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:51.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:51.977+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8d8059760 con 0x7fd8e80c4740 2024-03-06T07:24:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.159+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8e80c8480 con 0x7fd8e80c4740 2024-03-06T07:24:52.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.159+0000 7fd8dd7fa640 1 -- 172.21.15.17:0/1329293822 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8d8059060 con 0x7fd8e80c4740 2024-03-06T07:24:52.161 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 msgr2=0x7fd8d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 0x7fd8d406d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4035000 tx=0x7fd8e4030000 comp rx=0 tx=0).stop 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 msgr2=0x7fd8e81ec850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800d830 tx=0x7fd8d800dcf0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 shutdown_connections 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd8d406b4d0 0x7fd8d406d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8e80c69a0 0x7fd8e81de050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e80c4740 0x7fd8e81ec850 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 --2- 172.21.15.17:0/1329293822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8e80a8290 0x7fd8e81ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 >> 172.21.15.17:0/1329293822 conn(0x7fd8e80c5560 msgr2=0x7fd8e80c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 shutdown_connections 2024-03-06T07:24:52.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.161+0000 7fd8eeeaf640 1 -- 172.21.15.17:0/1329293822 wait complete. 2024-03-06T07:24:52.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:24:52.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.277+0000 7f44b8cd4640 1 Processor -- start 2024-03-06T07:24:52.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.277+0000 7f44b8cd4640 1 -- start start 2024-03-06T07:24:52.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b8cd4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f44b4155c30 0x7f44b406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b8cd4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40b0450 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b8cd4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b41560e0 con 0x7f44b40b0070 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b8cd4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b406cfb0 con 0x7f44b4155c30 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b2d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f44b40c5380 0x7f44b414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49720/0 (socket says 172.21.15.17:49720) 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b2d9d640 1 -- 172.21.15.17:0/916528938 learned_addr learned my addr 172.21.15.17:0/916528938 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44abfff640 1 --2- 172.21.15.17:0/916528938 >> v2:172.21.15.163:3300/0 conn(0x7f44b4155c30 0x7f44b406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538632548 0 0) 0x7f44ac001490 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003520 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3662370392 0 0) 0x7f44a0003520 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 >> v2:172.21.15.163:3300/0 conn(0x7f44b4155c30 msgr2=0x7f44b406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 --2- 172.21.15.17:0/916528938 >> v2:172.21.15.163:3300/0 conn(0x7f44b4155c30 0x7f44b406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 >> v1:172.21.15.163:6789/0 conn(0x7f44b40b0070 legacy=0x7f44b40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40c77f0 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f44ac001bc0 con 0x7f44b40c5380 2024-03-06T07:24:52.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.278+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 >> v1:172.21.15.17:6789/0 conn(0x7f44b40c5380 legacy=0x7f44b414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b406cfb0 con 0x7f44a00051c0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c160 con 0x7f44a0004800 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c360 con 0x7f44a00039b0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44abfff640 1 --2- 172.21.15.17:0/916528938 >> v2:172.21.15.163:3300/0 conn(0x7f44b4155c30 0x7f44b406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b259c640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44abfff640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 -- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a00039b0 msgr2=0x7f44a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a00039b0 0x7f44a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 -- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a0004800 msgr2=0x7f44a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a0004800 0x7f44a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.279+0000 7f44b2d9d640 1 -- 172.21.15.17:0/916528938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0003520 con 0x7f44a00051c0 2024-03-06T07:24:52.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.280+0000 7f44b2d9d640 1 -- 172.21.15.17:0/916528938 reap_dead start 2024-03-06T07:24:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.280+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00aa30 tx=0x7f44a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5cfabab32f8e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.280+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac038070 con 0x7f44a00051c0 2024-03-06T07:24:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.280+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac033620 con 0x7f44a00051c0 2024-03-06T07:24:52.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.280+0000 7f44b1d9b640 1 -- 172.21.15.17:0/916528938 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00a650 con 0x7f44a00051c0 2024-03-06T07:24:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 -- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 msgr2=0x7f44a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00aa30 tx=0x7f44a0005830 comp rx=0 tx=0).stop 2024-03-06T07:24:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 -- 172.21.15.17:0/916528938 shutdown_connections 2024-03-06T07:24:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/916528938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44a000b9c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 -- 172.21.15.17:0/916528938 >> 172.21.15.17:0/916528938 conn(0x7f44b40ac590 msgr2=0x7f44b40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 -- 172.21.15.17:0/916528938 shutdown_connections 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.281+0000 7f44b8cd4640 1 -- 172.21.15.17:0/916528938 wait complete. 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.282+0000 7f44b8cd4640 1 Processor -- start 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.282+0000 7f44b8cd4640 1 -- start start 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 0x7f44b4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 0x7f44b41deca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b406c270 con 0x7f44a00051c0 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b406c020 con 0x7f44b414c2e0 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b8cd4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b406c550 con 0x7f44b40aec10 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49606/0 (socket says 172.21.15.17:49606) 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b259c640 1 -- 172.21.15.17:0/2548854987 learned_addr learned my addr 172.21.15.17:0/2548854987 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 0x7f44b41deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.283+0000 7f44abfff640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 0x7f44b4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 -- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 msgr2=0x7f44b41deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 0x7f44b41deca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 -- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 msgr2=0x7f44b4158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 0x7f44b4158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 -- 172.21.15.17:0/2548854987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b41e02f0 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44abfff640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 0x7f44b4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b2d9d640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 0x7f44b41deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b259c640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f44a400f650 tx=0x7f44a400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40050d0 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a4018070 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4005490 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b414c090 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.284+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b41eb9d0 con 0x7f44a00051c0 2024-03-06T07:24:52.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.285+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b40c85d0 con 0x7f44a00051c0 2024-03-06T07:24:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.285+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44a400d6d0 con 0x7f44a00051c0 2024-03-06T07:24:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.286+0000 7f44aaffd640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.286+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f44a408a080 con 0x7f44a00051c0 2024-03-06T07:24:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.286+0000 7f44abfff640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.287+0000 7f44abfff640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 0x7f449406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f44b41dfb80 tx=0x7f449c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.290+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44b40c85d0 con 0x7f44a00051c0 2024-03-06T07:24:52.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.470+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44b40678a0 con 0x7f44a00051c0 2024-03-06T07:24:52.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.470+0000 7f44aaffd640 1 -- 172.21.15.17:0/2548854987 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44b40678a0 con 0x7f44a00051c0 2024-03-06T07:24:52.472 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:24:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 msgr2=0x7f449406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 0x7f449406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f44b41dfb80 tx=0x7f449c008040 comp rx=0 tx=0).stop 2024-03-06T07:24:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 msgr2=0x7f44b40ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f44a400f650 tx=0x7f44a400fb10 comp rx=0 tx=0).stop 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 shutdown_connections 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f449406b4d0 0x7f449406d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40aec10 0x7f44b41deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b414c2e0 0x7f44b4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 --2- 172.21.15.17:0/2548854987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a00051c0 0x7f44b40ae6d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 >> 172.21.15.17:0/2548854987 conn(0x7f44b40ac590 msgr2=0x7f44b41e18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 shutdown_connections 2024-03-06T07:24:52.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.472+0000 7f44b8cd4640 1 -- 172.21.15.17:0/2548854987 wait complete. 2024-03-06T07:24:52.485 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:52.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.586+0000 7f14ebafb640 1 Processor -- start 2024-03-06T07:24:52.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.586+0000 7f14ebafb640 1 -- start start 2024-03-06T07:24:52.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.587+0000 7f14ebafb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14e40c88c0 0x7f14e4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.587+0000 7f14ebafb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 0x7f14e4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.587+0000 7f14ebafb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e40c73d0 con 0x7f14e414f360 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.587+0000 7f14ebafb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40c8dd0 con 0x7f14e40c88c0 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.587+0000 7f14ebafb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40be2e0 con 0x7f14e4158b60 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14ea098640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 0x7f14e4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14e40c88c0 0x7f14e4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14e40c88c0 0x7f14e4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:42122/0 (socket says 172.21.15.17:42122) 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9897640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f14e414f360 0x7f14e40c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59342/0 (socket says 172.21.15.17:59342) 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14ea098640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 0x7f14e4158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49380/0 (socket says 172.21.15.17:49380) 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 -- 172.21.15.17:0/1185177685 learned_addr learned my addr 172.21.15.17:0/1185177685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 -- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 msgr2=0x7f14e4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 --2- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 0x7f14e4158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 -- 172.21.15.17:0/1185177685 >> v1:172.21.15.163:6789/0 conn(0x7f14e414f360 legacy=0x7f14e40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e8895640 1 -- 172.21.15.17:0/1185177685 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457082890 0 0) 0x7f14e40c73d0 con 0x7f14e414f360 2024-03-06T07:24:52.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.588+0000 7f14e9096640 1 -- 172.21.15.17:0/1185177685 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40c7670 con 0x7f14e40c88c0 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.589+0000 7f14e9096640 1 --2- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c88c0 0x7f14e4155c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c1012941640385 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.589+0000 7f14e8895640 1 -- 172.21.15.17:0/1185177685 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4035020 con 0x7f14e40c88c0 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.589+0000 7f14e8895640 1 -- 172.21.15.17:0/1185177685 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4031640 con 0x7f14e40c88c0 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.589+0000 7f14e8895640 1 -- 172.21.15.17:0/1185177685 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d402d040 con 0x7f14e40c88c0 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 -- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c88c0 msgr2=0x7f14e4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 --2- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c88c0 0x7f14e4155c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 -- 172.21.15.17:0/1185177685 shutdown_connections 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 --2- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e4158b60 0x7f14e4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 --2- 172.21.15.17:0/1185177685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e40c88c0 0x7f14e4155c10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 -- 172.21.15.17:0/1185177685 >> 172.21.15.17:0/1185177685 conn(0x7f14e40ac570 msgr2=0x7f14e40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 -- 172.21.15.17:0/1185177685 shutdown_connections 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.590+0000 7f14ebafb640 1 -- 172.21.15.17:0/1185177685 wait complete. 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.592+0000 7f14ebafb640 1 Processor -- start 2024-03-06T07:24:52.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.592+0000 7f14ebafb640 1 -- start start 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e40c88c0 0x7f14e40c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e4158b60 0x7f14e4157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c5a50 con 0x7f14e414f360 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c7c30 con 0x7f14e4158b60 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ebafb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40c5bd0 con 0x7f14e40c88c0 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49612/0 (socket says 172.21.15.17:49612) 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 -- 172.21.15.17:0/2193821953 learned_addr learned my addr 172.21.15.17:0/2193821953 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14ea098640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e4158b60 0x7f14e4157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9897640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e40c88c0 0x7f14e40c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 -- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e40c88c0 msgr2=0x7f14e40c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e40c88c0 0x7f14e40c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 -- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e4158b60 msgr2=0x7f14e4157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e4158b60 0x7f14e4157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.593+0000 7f14e9096640 1 -- 172.21.15.17:0/2193821953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e414c220 con 0x7f14e414f360 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14e9096640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f14d4034040 tx=0x7f14d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4040070 con 0x7f14e414f360 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d40399f0 con 0x7f14e414f360 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e40c3870 con 0x7f14e414f360 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d4035040 con 0x7f14e414f360 2024-03-06T07:24:52.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.594+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e40c1a90 con 0x7f14e414f360 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.595+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e404f330 con 0x7f14e414f360 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.595+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14d4038020 con 0x7f14e414f360 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.596+0000 7f14da7fc640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 0x7f14c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.596+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f14d40abce0 con 0x7f14e414f360 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.596+0000 7f14e9897640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 0x7f14c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.596+0000 7f14e9897640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 0x7f14c806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f14e0000f60 tx=0x7f14e0007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.600+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f14d407a9b0 con 0x7f14e414f360 2024-03-06T07:24:52.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.821+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e406cda0 con 0x7f14e414f360 2024-03-06T07:24:52.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.821+0000 7f14da7fc640 1 -- 172.21.15.17:0/2193821953 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4992 (secure 0 0 0) 0x7f14d407a2b0 con 0x7f14e414f360 2024-03-06T07:24:52.823 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:52.823 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:49.847563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8064,"qdb_cluster":[8064]},"id":9}]} 2024-03-06T07:24:52.823 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 114 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 msgr2=0x7f14c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 0x7f14c806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f14e0000f60 tx=0x7f14e0007b50 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 msgr2=0x7f14e414bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f14d4034040 tx=0x7f14d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 shutdown_connections 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f14c806b4d0 0x7f14c806d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14e4158b60 0x7f14e4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e414f360 0x7f14e414bce0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 --2- 172.21.15.17:0/2193821953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14e40c88c0 0x7f14e40c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 >> 172.21.15.17:0/2193821953 conn(0x7f14e40ac570 msgr2=0x7f14e406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 shutdown_connections 2024-03-06T07:24:52.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.824+0000 7f14ebafb640 1 -- 172.21.15.17:0/2193821953 wait complete. 2024-03-06T07:24:52.837 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.940+0000 7f8bba68d640 1 Processor -- start 2024-03-06T07:24:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.940+0000 7f8bba68d640 1 -- start start 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bba68d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8bb414e940 0x7f8bb406e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bba68d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb406cae0 0x7f8bb4152950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bba68d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb4155740 con 0x7f8bb4154a50 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bba68d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb406d9d0 con 0x7f8bb414e940 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bba68d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb406dcb0 con 0x7f8bb406cae0 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8bb414e940 0x7f8bb406e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb8c2a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8bb4154a50 0x7f8bb4154ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49728/0 (socket says 172.21.15.17:49728) 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb8c2a640 1 -- 172.21.15.17:0/1745094736 learned_addr learned my addr 172.21.15.17:0/1745094736 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb37fe640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb406cae0 0x7f8bb4152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb3fff640 1 -- 172.21.15.17:0/1745094736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb406cae0 msgr2=0x7f8bb4152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb3fff640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb406cae0 0x7f8bb4152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb3fff640 1 -- 172.21.15.17:0/1745094736 >> v1:172.21.15.17:6789/0 conn(0x7f8bb4154a50 legacy=0x7f8bb4154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb3fff640 1 -- 172.21.15.17:0/1745094736 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb406df90 con 0x7f8bb414e940 2024-03-06T07:24:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.941+0000 7f8bb2ffd640 1 -- 172.21.15.17:0/1745094736 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983614808 0 0) 0x7f8bb4155740 con 0x7f8bb4154a50 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.942+0000 7f8bb3fff640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb414e940 0x7f8bb406e790 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400a940 tx=0x7f8ba402ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a65c43594d47178 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.942+0000 7f8bb2ffd640 1 -- 172.21.15.17:0/1745094736 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba403a070 con 0x7f8bb414e940 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.942+0000 7f8bb2ffd640 1 -- 172.21.15.17:0/1745094736 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba402daa0 con 0x7f8bb414e940 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.942+0000 7f8bb2ffd640 1 -- 172.21.15.17:0/1745094736 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400b040 con 0x7f8bb414e940 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 -- 172.21.15.17:0/1745094736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb414e940 msgr2=0x7f8bb406e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb414e940 0x7f8bb406e790 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400a940 tx=0x7f8ba402ccf0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 -- 172.21.15.17:0/1745094736 shutdown_connections 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb406cae0 0x7f8bb4152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 --2- 172.21.15.17:0/1745094736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb414e940 0x7f8bb406e790 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 -- 172.21.15.17:0/1745094736 >> 172.21.15.17:0/1745094736 conn(0x7f8bb406bf00 msgr2=0x7f8bb40c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 -- 172.21.15.17:0/1745094736 shutdown_connections 2024-03-06T07:24:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.943+0000 7f8bba68d640 1 -- 172.21.15.17:0/1745094736 wait complete. 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.944+0000 7f8bba68d640 1 Processor -- start 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.944+0000 7f8bba68d640 1 -- start start 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 0x7f8bb41dd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 0x7f8bb40c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4152e90 con 0x7f8bb406cae0 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c15a0 con 0x7f8bb4154a50 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4153010 con 0x7f8bb414e940 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49626/0 (socket says 172.21.15.17:49626) 2024-03-06T07:24:52.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 -- 172.21.15.17:0/4272033215 learned_addr learned my addr 172.21.15.17:0/4272033215 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb37fe640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 0x7f8bb41dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb8c2a640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 0x7f8bb40c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 -- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 msgr2=0x7f8bb41dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 0x7f8bb41dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 -- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 msgr2=0x7f8bb40c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 0x7f8bb40c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 -- 172.21.15.17:0/4272033215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb41ea850 con 0x7f8bb406cae0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb37fe640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 0x7f8bb41dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb8c2a640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 0x7f8bb40c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb3fff640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4038040 tx=0x7f8ba4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4045070 con 0x7f8bb406cae0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4036b80 con 0x7f8bb406cae0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.945+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb40721f0 con 0x7f8bb406cae0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.946+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba403a040 con 0x7f8bb406cae0 2024-03-06T07:24:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.946+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb41eab10 con 0x7f8bb406cae0 2024-03-06T07:24:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.947+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb404f330 con 0x7f8bb406cae0 2024-03-06T07:24:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.947+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ba4048070 con 0x7f8bb406cae0 2024-03-06T07:24:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.947+0000 7f8bb0ff9640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 0x7f8ba006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.948+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8ba40ab9e0 con 0x7f8bb406cae0 2024-03-06T07:24:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.948+0000 7f8bb37fe640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 0x7f8ba006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.948+0000 7f8bb37fe640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 0x7f8ba006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c4dc0 tx=0x7f8ba8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:52.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:52.956+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8ba407a6b0 con 0x7f8bb406cae0 2024-03-06T07:24:53.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.138+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb4068460 con 0x7f8bb406cae0 2024-03-06T07:24:53.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.139+0000 7f8bb0ff9640 1 -- 172.21.15.17:0/4272033215 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17157 (secure 0 0 0) 0x7f8ba4079fb0 con 0x7f8bb406cae0 2024-03-06T07:24:53.141 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:53.141 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":124,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:46.835679+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6833/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6835/410228237":"2024-03-07T07:24:44.513269+0000","172.21.15.17:6832/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6834/410228237":"2024-03-07T07:24:44.513269+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.141+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 msgr2=0x7f8ba006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.141+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 0x7f8ba006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40c4dc0 tx=0x7f8ba8008040 comp rx=0 tx=0).stop 2024-03-06T07:24:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 msgr2=0x7f8bb41eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4038040 tx=0x7f8ba4004730 comp rx=0 tx=0).stop 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 shutdown_connections 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ba006b4d0 0x7f8ba006d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb4154a50 0x7f8bb40c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb414e940 0x7f8bb41dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 --2- 172.21.15.17:0/4272033215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bb406cae0 0x7f8bb41eb310 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 >> 172.21.15.17:0/4272033215 conn(0x7f8bb406bf00 msgr2=0x7f8bb414d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 shutdown_connections 2024-03-06T07:24:53.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.142+0000 7f8bba68d640 1 -- 172.21.15.17:0/4272033215 wait complete. 2024-03-06T07:24:53.155 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:24:53.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.291+0000 7fd21e64e640 1 Processor -- start 2024-03-06T07:24:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.291+0000 7fd21e64e640 1 -- start start 2024-03-06T07:24:53.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21e64e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd2181506b0 0x7fd218153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21e64e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2180acc00 0x7fd2180c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21e64e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd218070aa0 con 0x7fd218156ea0 2024-03-06T07:24:53.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21e64e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd218156450 con 0x7fd2180acc00 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21e64e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd218156a10 con 0x7fd2181506b0 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd21cbeb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd2180acc00 0x7fd2180c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd2177fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd218156ea0 0x7fd2180702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49738/0 (socket says 172.21.15.17:49738) 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd2177fe640 1 -- 172.21.15.17:0/3692860401 learned_addr learned my addr 172.21.15.17:0/3692860401 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.292+0000 7fd217fff640 1 --2- 172.21.15.17:0/3692860401 >> v2:172.21.15.163:3300/0 conn(0x7fd2181506b0 0x7fd218153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd21cbeb640 1 -- 172.21.15.17:0/3692860401 >> v2:172.21.15.163:3300/0 conn(0x7fd2181506b0 msgr2=0x7fd218153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd21cbeb640 1 --2- 172.21.15.17:0/3692860401 >> v2:172.21.15.163:3300/0 conn(0x7fd2181506b0 0x7fd218153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd21cbeb640 1 -- 172.21.15.17:0/3692860401 >> v1:172.21.15.17:6789/0 conn(0x7fd218156ea0 legacy=0x7fd2180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd216ffd640 1 -- 172.21.15.17:0/3692860401 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408251423 0 0) 0x7fd218070aa0 con 0x7fd218156ea0 2024-03-06T07:24:53.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd21cbeb640 1 -- 172.21.15.17:0/3692860401 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218070d80 con 0x7fd2180acc00 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.293+0000 7fd21cbeb640 1 --2- 172.21.15.17:0/3692860401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2180acc00 0x7fd2180c54a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd2180c97e0 tx=0x7fd21002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b773fb7561ee78 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd216ffd640 1 -- 172.21.15.17:0/3692860401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210039070 con 0x7fd2180acc00 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd216ffd640 1 -- 172.21.15.17:0/3692860401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21002cc80 con 0x7fd2180acc00 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd216ffd640 1 -- 172.21.15.17:0/3692860401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210034660 con 0x7fd2180acc00 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 -- 172.21.15.17:0/3692860401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2180acc00 msgr2=0x7fd2180c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 --2- 172.21.15.17:0/3692860401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2180acc00 0x7fd2180c54a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd2180c97e0 tx=0x7fd21002bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 -- 172.21.15.17:0/3692860401 shutdown_connections 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 --2- 172.21.15.17:0/3692860401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2180acc00 0x7fd2180c54a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 --2- 172.21.15.17:0/3692860401 >> v2:172.21.15.163:3300/0 conn(0x7fd2181506b0 0x7fd218153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 -- 172.21.15.17:0/3692860401 >> 172.21.15.17:0/3692860401 conn(0x7fd21806b1a0 msgr2=0x7fd21806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 -- 172.21.15.17:0/3692860401 shutdown_connections 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.294+0000 7fd21e64e640 1 -- 172.21.15.17:0/3692860401 wait complete. 2024-03-06T07:24:53.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.295+0000 7fd21e64e640 1 Processor -- start 2024-03-06T07:24:53.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.295+0000 7fd21e64e640 1 -- start start 2024-03-06T07:24:53.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 0x7fd21814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd218156ea0 0x7fd218145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180ad1d0 con 0x7fd2181506b0 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180ad050 con 0x7fd218156ea0 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21e64e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180ad350 con 0x7fd2180acc00 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd2177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd2177fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49656/0 (socket says 172.21.15.17:49656) 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd2177fe640 1 -- 172.21.15.17:0/2258338347 learned_addr learned my addr 172.21.15.17:0/2258338347 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd21cbeb640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd218156ea0 0x7fd218145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.296+0000 7fd217fff640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 0x7fd21814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 -- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 msgr2=0x7fd21814fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 0x7fd21814fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 -- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd218156ea0 msgr2=0x7fd218145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd218156ea0 0x7fd218145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 -- 172.21.15.17:0/2258338347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21814c690 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd217fff640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 0x7fd21814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd2177fe640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c410 tx=0x7fd20c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c0074e0 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c0047e0 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21814c970 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c004bd0 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.297+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21814e490 con 0x7fd2181506b0 2024-03-06T07:24:53.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.298+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21804f330 con 0x7fd2181506b0 2024-03-06T07:24:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.299+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd20c011070 con 0x7fd2181506b0 2024-03-06T07:24:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.299+0000 7fd214ff9640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 0x7fd20406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.299+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd20c08a020 con 0x7fd2181506b0 2024-03-06T07:24:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.299+0000 7fd217fff640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 0x7fd20406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.300+0000 7fd217fff640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 0x7fd20406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd21806cac0 tx=0x7fd208008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.307+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd20c00e280 con 0x7fd2181506b0 2024-03-06T07:24:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.600+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2180700a0 con 0x7fd2181506b0 2024-03-06T07:24:53.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.601+0000 7fd214ff9640 1 -- 172.21.15.17:0/2258338347 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4992 (secure 0 0 0) 0x7fd20c058c60 con 0x7fd2181506b0 2024-03-06T07:24:53.602 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:53.602 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:49.847563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8064,"qdb_cluster":[8064]},"id":9}]} 2024-03-06T07:24:53.603 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 114 2024-03-06T07:24:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 msgr2=0x7fd20406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 0x7fd20406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd21806cac0 tx=0x7fd208008040 comp rx=0 tx=0).stop 2024-03-06T07:24:53.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 msgr2=0x7fd21814df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00c410 tx=0x7fd20c00c8d0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 shutdown_connections 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd20406b4d0 0x7fd20406d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd218156ea0 0x7fd218145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2181506b0 0x7fd21814df50 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 --2- 172.21.15.17:0/2258338347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2180acc00 0x7fd21814fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.603+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 >> 172.21.15.17:0/2258338347 conn(0x7fd21806b1a0 msgr2=0x7fd218152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.604+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 shutdown_connections 2024-03-06T07:24:53.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.604+0000 7fd21e64e640 1 -- 172.21.15.17:0/2258338347 wait complete. 2024-03-06T07:24:53.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:24:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.715+0000 7f50c0ea4640 1 Processor -- start 2024-03-06T07:24:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.715+0000 7f50c0ea4640 1 -- start start 2024-03-06T07:24:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50c0ea4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 0x7f50bc155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50c0ea4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50bc1586f0 0x7f50bc158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50c0ea4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0c5cc0 con 0x7f50bc0c5860 2024-03-06T07:24:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50c0ea4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc06df40 con 0x7f50bc1586f0 2024-03-06T07:24:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50c0ea4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc06fdb0 con 0x7f50bc0c5480 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50b1d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50bc0c5860 0x7f50bc06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49752/0 (socket says 172.21.15.17:49752) 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50b1d9b640 1 -- 172.21.15.17:0/2865283153 learned_addr learned my addr 172.21.15.17:0/2865283153 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50ba59c640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 0x7f50bc155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.716+0000 7f50bad9d640 1 --2- 172.21.15.17:0/2865283153 >> v2:172.21.15.163:3300/0 conn(0x7f50bc1586f0 0x7f50bc158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50b9d9b640 1 -- 172.21.15.17:0/2865283153 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302695156 0 0) 0x7f50bc0c5cc0 con 0x7f50bc0c5860 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50b9d9b640 1 -- 172.21.15.17:0/2865283153 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a8003520 con 0x7f50bc0c5860 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50bad9d640 1 -- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 msgr2=0x7f50bc155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50bad9d640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 0x7f50bc155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50bad9d640 1 -- 172.21.15.17:0/2865283153 >> v1:172.21.15.17:6789/0 conn(0x7f50bc0c5860 legacy=0x7f50bc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50bad9d640 1 -- 172.21.15.17:0/2865283153 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc1591c0 con 0x7f50bc1586f0 2024-03-06T07:24:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.717+0000 7f50ba59c640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 0x7f50bc155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:24:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50bad9d640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc1586f0 0x7f50bc158b80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da19b120311cbb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50b9d9b640 1 -- 172.21.15.17:0/2865283153 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4039070 con 0x7f50bc1586f0 2024-03-06T07:24:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50b9d9b640 1 -- 172.21.15.17:0/2865283153 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b402cbc0 con 0x7f50bc1586f0 2024-03-06T07:24:53.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50b9d9b640 1 -- 172.21.15.17:0/2865283153 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4034440 con 0x7f50bc1586f0 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc1586f0 msgr2=0x7f50bc158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc1586f0 0x7f50bc158b80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009d90 tx=0x7f50b402be10 comp rx=0 tx=0).stop 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2865283153 shutdown_connections 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc1586f0 0x7f50bc158b80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2865283153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc0c5480 0x7f50bc155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2865283153 >> 172.21.15.17:0/2865283153 conn(0x7f50bc06b1a0 msgr2=0x7f50bc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2865283153 shutdown_connections 2024-03-06T07:24:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.718+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2865283153 wait complete. 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 Processor -- start 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 -- start start 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc0c5480 0x7f50bc157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 0x7f50bc0c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.720+0000 7f50c0ea4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc155680 con 0x7f50bc0c5860 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50c0ea4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc1553a0 con 0x7f50bc0c5480 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50c0ea4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc152a20 con 0x7f50bc1586f0 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49668/0 (socket says 172.21.15.17:49668) 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 -- 172.21.15.17:0/2104844072 learned_addr learned my addr 172.21.15.17:0/2104844072 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50bad9d640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 0x7f50bc0c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50ba59c640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc0c5480 0x7f50bc157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 -- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 msgr2=0x7f50bc0c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 0x7f50bc0c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 -- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc0c5480 msgr2=0x7f50bc157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc0c5480 0x7f50bc157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 -- 172.21.15.17:0/2104844072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0ad050 con 0x7f50bc0c5860 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50bad9d640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 0x7f50bc0c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b1d9b640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f50a400c430 tx=0x7f50a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4017070 con 0x7f50bc0c5860 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a4005ce0 con 0x7f50bc0c5860 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc0ab100 con 0x7f50bc0c5860 2024-03-06T07:24:53.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a40072f0 con 0x7f50bc0c5860 2024-03-06T07:24:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.721+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc157c40 con 0x7f50bc0c5860 2024-03-06T07:24:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.723+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc0ad220 con 0x7f50bc0c5860 2024-03-06T07:24:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.723+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50a401a070 con 0x7f50bc0c5860 2024-03-06T07:24:53.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.723+0000 7f50b37fe640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 0x7f509c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.723+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f50a4089860 con 0x7f50bc0c5860 2024-03-06T07:24:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.724+0000 7f50ba59c640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 0x7f509c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:53.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.724+0000 7f50ba59c640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 0x7f509c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009560 tx=0x7f50ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:53.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.727+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50a4058530 con 0x7f50bc0c5860 2024-03-06T07:24:53.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.947+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50bc04f330 con 0x7f50bc0c5860 2024-03-06T07:24:53.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.947+0000 7f50b37fe640 1 -- 172.21.15.17:0/2104844072 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4992 (secure 0 0 0) 0x7f50a4057e30 con 0x7f50bc0c5860 2024-03-06T07:24:53.949 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:53.949 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":114,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:24:31.239352+0000","modified":"2024-03-06T07:24:49.847563+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8064},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8064":{"gid":8064,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6835/4288775541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":4288775541},{"type":"v1","addr":"172.21.15.17:6835","nonce":4288775541}]},"join_fscid":9,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8064,"qdb_cluster":[8064]},"id":9}]} 2024-03-06T07:24:53.950 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 114 2024-03-06T07:24:53.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 msgr2=0x7f509c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 0x7f509c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009560 tx=0x7f50ac008040 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 msgr2=0x7f50bc0c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f50a400c430 tx=0x7f50a400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 shutdown_connections 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f509c06b4d0 0x7f509c06d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50bc1586f0 0x7f50bc0c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0c5860 0x7f50bc0c3310 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 --2- 172.21.15.17:0/2104844072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50bc0c5480 0x7f50bc157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 >> 172.21.15.17:0/2104844072 conn(0x7f50bc06b1a0 msgr2=0x7f50bc0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 shutdown_connections 2024-03-06T07:24:53.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:53.950+0000 7f50c0ea4640 1 -- 172.21.15.17:0/2104844072 wait complete. 2024-03-06T07:24:53.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:24:54.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.062+0000 7f23a5793640 1 Processor -- start 2024-03-06T07:24:54.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.062+0000 7f23a5793640 1 -- start start 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23a00c7180 0x7f23a00c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23a0155b20 0x7f23a00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a0159640 con 0x7f23a00c7180 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00c8df0 con 0x7f23a0155b20 2024-03-06T07:24:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f23a5793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c7c60 con 0x7f23a01527b0 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23a00c7180 0x7f23a00c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f23a00c7180 0x7f23a00c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49684/0 (socket says 172.21.15.17:49684) 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 -- 172.21.15.17:0/56970836 learned_addr learned my addr 172.21.15.17:0/56970836 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239effd640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239f7fe640 1 --2- 172.21.15.17:0/56970836 >> v2:172.21.15.163:3300/0 conn(0x7f23a0155b20 0x7f23a00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 -- 172.21.15.17:0/56970836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 msgr2=0x7f23a0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 -- 172.21.15.17:0/56970836 >> v2:172.21.15.163:3300/0 conn(0x7f23a0155b20 msgr2=0x7f23a00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 --2- 172.21.15.17:0/56970836 >> v2:172.21.15.163:3300/0 conn(0x7f23a0155b20 0x7f23a00c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.063+0000 7f239e7fc640 1 -- 172.21.15.17:0/56970836 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00c8060 con 0x7f23a00c7180 2024-03-06T07:24:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f239e7fc640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a00c7180 0x7f23a00c88b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f23a006b170 tx=0x7f239402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a120fca174d21c01 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f239dffb640 1 -- 172.21.15.17:0/56970836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239402e070 con 0x7f23a00c7180 2024-03-06T07:24:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f239dffb640 1 -- 172.21.15.17:0/56970836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394037070 con 0x7f23a00c7180 2024-03-06T07:24:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f239dffb640 1 -- 172.21.15.17:0/56970836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394031a00 con 0x7f23a00c7180 2024-03-06T07:24:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 -- 172.21.15.17:0/56970836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a00c7180 msgr2=0x7f23a00c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a00c7180 0x7f23a00c88b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f23a006b170 tx=0x7f239402b880 comp rx=0 tx=0).stop 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 -- 172.21.15.17:0/56970836 shutdown_connections 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 --2- 172.21.15.17:0/56970836 >> v2:172.21.15.163:3300/0 conn(0x7f23a0155b20 0x7f23a00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a00c7180 0x7f23a00c88b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 --2- 172.21.15.17:0/56970836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 -- 172.21.15.17:0/56970836 >> 172.21.15.17:0/56970836 conn(0x7f23a00ac560 msgr2=0x7f23a00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 -- 172.21.15.17:0/56970836 shutdown_connections 2024-03-06T07:24:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.065+0000 7f23a5793640 1 -- 172.21.15.17:0/56970836 wait complete. 2024-03-06T07:24:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 Processor -- start 2024-03-06T07:24:54.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 -- start start 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a00c7180 0x7f23a014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0155320 con 0x7f23a0155b20 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c7980 con 0x7f23a00c7180 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f23a5793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0155600 con 0x7f23a01527b0 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49692/0 (socket says 172.21.15.17:49692) 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.067+0000 7f239f7fe640 1 -- 172.21.15.17:0/2916000330 learned_addr learned my addr 172.21.15.17:0/2916000330 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239effd640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a00c7180 0x7f23a014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239e7fc640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 -- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 msgr2=0x7f23a0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 -- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a00c7180 msgr2=0x7f23a014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a00c7180 0x7f23a014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 -- 172.21.15.17:0/2916000330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0154780 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f239f7fe640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f239800c580 tx=0x7f239800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398015070 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398004220 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a0154a60 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398003610 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.068+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0156d30 con 0x7f23a0155b20 2024-03-06T07:24:54.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.069+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a004f330 con 0x7f23a0155b20 2024-03-06T07:24:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.070+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23980037b0 con 0x7f23a0155b20 2024-03-06T07:24:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.070+0000 7f23877fe640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.071+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f239808a180 con 0x7f23a0155b20 2024-03-06T07:24:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.071+0000 7f239effd640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.071+0000 7f239effd640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 0x7f238c06d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f23900070b0 tx=0x7f2390007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.075+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2398058dd0 con 0x7f23a0155b20 2024-03-06T07:24:54.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.257+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a014c5e0 con 0x7f23a0155b20 2024-03-06T07:24:54.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.258+0000 7f23877fe640 1 -- 172.21.15.17:0/2916000330 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17157 (secure 0 0 0) 0x7f23980586d0 con 0x7f23a0155b20 2024-03-06T07:24:54.260 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:24:54.260 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":124,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:46.835679+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:24:29.507391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-03-06T07:24:30.198128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6833/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6835/410228237":"2024-03-07T07:24:44.513269+0000","172.21.15.17:6832/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6834/410228237":"2024-03-07T07:24:44.513269+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:24:54.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 msgr2=0x7f238c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 0x7f238c06d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f23900070b0 tx=0x7f2390007040 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 msgr2=0x7f23a0148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f239800c580 tx=0x7f239800ca40 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 shutdown_connections 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f238c06b4d0 0x7f238c06d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23a0155b20 0x7f23a0148f60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23a01527b0 0x7f23a0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 --2- 172.21.15.17:0/2916000330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23a00c7180 0x7f23a014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 >> 172.21.15.17:0/2916000330 conn(0x7f23a00ac560 msgr2=0x7f23a00bdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 shutdown_connections 2024-03-06T07:24:54.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.260+0000 7f23a5793640 1 -- 172.21.15.17:0/2916000330 wait complete. 2024-03-06T07:24:54.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:24:54.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.372+0000 7fc321135640 1 Processor -- start 2024-03-06T07:24:54.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.372+0000 7fc321135640 1 -- start start 2024-03-06T07:24:54.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc321135640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 0x7fc31c153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc321135640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc31c0c7170 0x7fc31c154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc321135640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31c0a8390 con 0x7fc31c0c6c40 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc321135640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31c0c7620 con 0x7fc31c0c7170 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc321135640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0730f0 con 0x7fc31c06ff10 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc31ad9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 0x7fc31c153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc313fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc31c0c7170 0x7fc31c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc31b59e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc31c0c6c40 0x7fc31c0c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:59352/0 (socket says 172.21.15.17:59352) 2024-03-06T07:24:54.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc31ad9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 0x7fc31c153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49480/0 (socket says 172.21.15.17:49480) 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc313fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc31c0c7170 0x7fc31c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:42204/0 (socket says 172.21.15.17:42204) 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.373+0000 7fc31ad9d640 1 -- 172.21.15.17:0/954901429 learned_addr learned my addr 172.21.15.17:0/954901429 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.374+0000 7fc313fff640 1 -- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 msgr2=0x7fc31c153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.374+0000 7fc313fff640 1 --2- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 0x7fc31c153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.374+0000 7fc313fff640 1 -- 172.21.15.17:0/954901429 >> v1:172.21.15.163:6789/0 conn(0x7fc31c0c6c40 legacy=0x7fc31c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.374+0000 7fc313fff640 1 -- 172.21.15.17:0/954901429 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c0733d0 con 0x7fc31c0c7170 2024-03-06T07:24:54.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.374+0000 7fc31a59c640 1 -- 172.21.15.17:0/954901429 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211427663 0 0) 0x7fc31c0a8390 con 0x7fc31c0c6c40 2024-03-06T07:24:54.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc313fff640 1 --2- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c0c7170 0x7fc31c154160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097a0 tx=0x7fc30402b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7ba741d658f640 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc31a59c640 1 -- 172.21.15.17:0/954901429 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304038070 con 0x7fc31c0c7170 2024-03-06T07:24:54.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc31a59c640 1 -- 172.21.15.17:0/954901429 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc304031700 con 0x7fc31c0c7170 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc31a59c640 1 -- 172.21.15.17:0/954901429 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304031ad0 con 0x7fc31c0c7170 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 -- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c0c7170 msgr2=0x7fc31c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 --2- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c0c7170 0x7fc31c154160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097a0 tx=0x7fc30402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 -- 172.21.15.17:0/954901429 shutdown_connections 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 --2- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c0c7170 0x7fc31c154160 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 --2- 172.21.15.17:0/954901429 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c06ff10 0x7fc31c153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.375+0000 7fc321135640 1 -- 172.21.15.17:0/954901429 >> 172.21.15.17:0/954901429 conn(0x7fc31c06b970 msgr2=0x7fc31c0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.376+0000 7fc321135640 1 -- 172.21.15.17:0/954901429 shutdown_connections 2024-03-06T07:24:54.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.376+0000 7fc321135640 1 -- 172.21.15.17:0/954901429 wait complete. 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.377+0000 7fc321135640 1 Processor -- start 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.377+0000 7fc321135640 1 -- start start 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c06ff10 0x7fc31c155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c0c6c40 0x7fc31c1e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c070470 con 0x7fc31c0c7170 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0702f0 con 0x7fc31c06ff10 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.378+0000 7fc321135640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0c6860 con 0x7fc31c0c6c40 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49704/0 (socket says 172.21.15.17:49704) 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 -- 172.21.15.17:0/1006789817 learned_addr learned my addr 172.21.15.17:0/1006789817 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc313fff640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c0c6c40 0x7fc31c1e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31ad9d640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c06ff10 0x7fc31c155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 -- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c0c6c40 msgr2=0x7fc31c1e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c0c6c40 0x7fc31c1e8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 -- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c06ff10 msgr2=0x7fc31c155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c06ff10 0x7fc31c155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 -- 172.21.15.17:0/1006789817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c154c40 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc31b59e640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c7d0 tx=0x7fc31400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314018070 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31400a640 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c154f20 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400aa70 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.379+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c1ebd50 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.381+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c04f330 con 0x7fc31c0c7170 2024-03-06T07:24:54.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.381+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc31401b070 con 0x7fc31c0c7170 2024-03-06T07:24:54.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.381+0000 7fc3137fe640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.381+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc314089920 con 0x7fc31c0c7170 2024-03-06T07:24:54.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.381+0000 7fc31ad9d640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.382+0000 7fc31ad9d640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009bd0 tx=0x7fc30c009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.386+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc314057fa0 con 0x7fc31c0c7170 2024-03-06T07:24:54.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.612+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc31c06fa80 con 0x7fc31c0c7170 2024-03-06T07:24:54.614 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:24:54.613+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:24:54.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.614+0000 7fc3137fe640 1 -- 172.21.15.17:0/1006789817 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc31405c040 con 0x7fc31c0c7170 2024-03-06T07:24:54.616 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 msgr2=0x7fc2fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009bd0 tx=0x7fc30c009240 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 msgr2=0x7fc31c0acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c7d0 tx=0x7fc31400cc90 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 shutdown_connections 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc31c0c7170 0x7fc31c0acb10 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc31c0c6c40 0x7fc31c1e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 --2- 172.21.15.17:0/1006789817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc31c06ff10 0x7fc31c155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 >> 172.21.15.17:0/1006789817 conn(0x7fc31c06b970 msgr2=0x7fc31c152950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 shutdown_connections 2024-03-06T07:24:54.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.617+0000 7fc321135640 1 -- 172.21.15.17:0/1006789817 wait complete. 2024-03-06T07:24:54.620 INFO:tasks.ceph.mds.c.smithi017.stderr: -173> 2024-03-06T07:24:50.529+0000 7f84dc7a7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:24:54.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:24:54.643 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:24:54.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 Processor -- start 2024-03-06T07:24:54.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 -- start start 2024-03-06T07:24:54.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04d0155920 0x7f04d0160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d01596f0 con 0x7f04d01527f0 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00c82a0 con 0x7f04d00c88f0 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.733+0000 7f04d53b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d0155d70 con 0x7f04d0155920 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04cf7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04d0155920 0x7f04d0160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04ceffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04d01527f0 0x7f04d00c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49768/0 (socket says 172.21.15.17:49768) 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04ceffd640 1 -- 172.21.15.17:0/1376793373 learned_addr learned my addr 172.21.15.17:0/1376793373 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04cf7fe640 1 -- 172.21.15.17:0/1376793373 >> v1:172.21.15.163:6789/0 conn(0x7f04d00c88f0 legacy=0x7f04d0155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04cf7fe640 1 -- 172.21.15.17:0/1376793373 >> v1:172.21.15.17:6789/0 conn(0x7f04d01527f0 legacy=0x7f04d00c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04cf7fe640 1 -- 172.21.15.17:0/1376793373 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0158930 con 0x7f04d0155920 2024-03-06T07:24:54.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.734+0000 7f04ce7fc640 1 -- 172.21.15.17:0/1376793373 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747735980 0 0) 0x7f04d01596f0 con 0x7f04d01527f0 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04cf7fe640 1 --2- 172.21.15.17:0/1376793373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0160ba0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903333c8624ef0ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04ce7fc640 1 -- 172.21.15.17:0/1376793373 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8039070 con 0x7f04d0155920 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04ce7fc640 1 -- 172.21.15.17:0/1376793373 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c802cbd0 con 0x7f04d0155920 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04ce7fc640 1 -- 172.21.15.17:0/1376793373 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8034440 con 0x7f04d0155920 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 -- 172.21.15.17:0/1376793373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 msgr2=0x7f04d0160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 --2- 172.21.15.17:0/1376793373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0160ba0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802be20 comp rx=0 tx=0).stop 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 -- 172.21.15.17:0/1376793373 shutdown_connections 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 --2- 172.21.15.17:0/1376793373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0160ba0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 -- 172.21.15.17:0/1376793373 >> 172.21.15.17:0/1376793373 conn(0x7f04d00ac580 msgr2=0x7f04d00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 -- 172.21.15.17:0/1376793373 shutdown_connections 2024-03-06T07:24:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.735+0000 7f04d53b6640 1 -- 172.21.15.17:0/1376793373 wait complete. 2024-03-06T07:24:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.736+0000 7f04d53b6640 1 Processor -- start 2024-03-06T07:24:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.736+0000 7f04d53b6640 1 -- start start 2024-03-06T07:24:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00c88f0 0x7f04d00ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 0x7f04d014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0159260 con 0x7f04d0155920 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00c9130 con 0x7f04d00c88f0 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04d53b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00c6b90 con 0x7f04d01527f0 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04cf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04cf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49736/0 (socket says 172.21.15.17:49736) 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04cf7fe640 1 -- 172.21.15.17:0/3485746179 learned_addr learned my addr 172.21.15.17:0/3485746179 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04c7fff640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 0x7f04d014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.737+0000 7f04ceffd640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00c88f0 0x7f04d00ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 -- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 msgr2=0x7f04d014d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 0x7f04d014d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 -- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00c88f0 msgr2=0x7f04d00ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00c88f0 0x7f04d00ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 -- 172.21.15.17:0/3485746179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d01ea9b0 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04c7fff640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 0x7f04d014d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04cf7fe640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8044020 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c803a040 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d01eac90 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8039050 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.738+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d01eb250 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.739+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d004f330 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.739+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04c8004030 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.740+0000 7f04c77fe640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 0x7f04b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.740+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f04c80acc20 con 0x7f04d0155920 2024-03-06T07:24:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.740+0000 7f04ceffd640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 0x7f04b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.740+0000 7f04ceffd640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 0x7f04b006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f04c0000f60 tx=0x7f04c0007b90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:54.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.744+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04c804e060 con 0x7f04d0155920 2024-03-06T07:24:54.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:54.967+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f04d00c8080 con 0x7f04d0155920 2024-03-06T07:24:55.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.615+0000 7f04c77fe640 1 -- 172.21.15.17:0/3485746179 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) v1 ==== 104+0+0 (secure 0 0 0) 0x7f04c807b8a0 con 0x7f04d0155920 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 msgr2=0x7f04b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 0x7f04b006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f04c0000f60 tx=0x7f04c0007b90 comp rx=0 tx=0).stop 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 msgr2=0x7f04d0154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c8004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 shutdown_connections 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f04b006b4d0 0x7f04b006d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d0155920 0x7f04d0154300 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04d01527f0 0x7f04d014d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 --2- 172.21.15.17:0/3485746179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04d00c88f0 0x7f04d00ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 >> 172.21.15.17:0/3485746179 conn(0x7f04d00ac580 msgr2=0x7f04d0150280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 shutdown_connections 2024-03-06T07:24:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.619+0000 7f04d53b6640 1 -- 172.21.15.17:0/3485746179 wait complete. 2024-03-06T07:24:55.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:24:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.785+0000 7f42d0c5c640 1 Processor -- start 2024-03-06T07:24:55.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.785+0000 7f42d0c5c640 1 -- start start 2024-03-06T07:24:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42d0c5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42cc14bb20 0x7f42cc0c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42d0c5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42d0c5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42cc159260 con 0x7f42cc0c2330 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42d0c5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0c3070 con 0x7f42cc14bb20 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42d0c5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0a80e0 con 0x7f42cc1550c0 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42ca59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42cc14bb20 0x7f42cc0c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42ca59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42cc14bb20 0x7f42cc0c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49738/0 (socket says 172.21.15.17:49738) 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42ca59c640 1 -- 172.21.15.17:0/3438314732 learned_addr learned my addr 172.21.15.17:0/3438314732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.786+0000 7f42cad9d640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42ca59c640 1 -- 172.21.15.17:0/3438314732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 msgr2=0x7f42cc155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42ca59c640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42ca59c640 1 -- 172.21.15.17:0/3438314732 >> v1:172.21.15.163:6789/0 conn(0x7f42cc0c2330 legacy=0x7f42cc0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42ca59c640 1 -- 172.21.15.17:0/3438314732 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0a7d70 con 0x7f42cc14bb20 2024-03-06T07:24:55.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42cad9d640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:55.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42ca59c640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc14bb20 0x7f42cc0c5310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0735a0 tx=0x7f42bc02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2a240e74a39f4c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:55.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.787+0000 7f42c959a640 1 -- 172.21.15.17:0/3438314732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc02c9a0 con 0x7f42cc14bb20 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42c959a640 1 -- 172.21.15.17:0/3438314732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc02cb40 con 0x7f42cc14bb20 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42c959a640 1 -- 172.21.15.17:0/3438314732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0343f0 con 0x7f42cc14bb20 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 -- 172.21.15.17:0/3438314732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc14bb20 msgr2=0x7f42cc0c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc14bb20 0x7f42cc0c5310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0735a0 tx=0x7f42bc02be00 comp rx=0 tx=0).stop 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 -- 172.21.15.17:0/3438314732 shutdown_connections 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/3438314732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc14bb20 0x7f42cc0c5310 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 -- 172.21.15.17:0/3438314732 >> 172.21.15.17:0/3438314732 conn(0x7f42cc0c88f0 msgr2=0x7f42cc0c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 -- 172.21.15.17:0/3438314732 shutdown_connections 2024-03-06T07:24:55.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.788+0000 7f42d0c5c640 1 -- 172.21.15.17:0/3438314732 wait complete. 2024-03-06T07:24:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.789+0000 7f42d0c5c640 1 Processor -- start 2024-03-06T07:24:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.789+0000 7f42d0c5c640 1 -- start start 2024-03-06T07:24:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 0x7f42cc1ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc14bf00 con 0x7f42cc0c2330 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c5f60 con 0x7f42cc14bb20 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42d0c5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c67f0 con 0x7f42cc1550c0 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42ca59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42ca59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49746/0 (socket says 172.21.15.17:49746) 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42ca59c640 1 -- 172.21.15.17:0/1843331330 learned_addr learned my addr 172.21.15.17:0/1843331330 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42c9d9b640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 0x7f42cc1ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.790+0000 7f42cad9d640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 -- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 msgr2=0x7f42cc152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 -- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 msgr2=0x7f42cc1ee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 0x7f42cc1ee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 -- 172.21.15.17:0/1843331330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc156d00 con 0x7f42cc0c2330 2024-03-06T07:24:55.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42c9d9b640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 0x7f42cc1ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42ca59c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f42bc000f80 tx=0x7f42bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc043020 con 0x7f42cc0c2330 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc03e450 con 0x7f42cc0c2330 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc03e840 con 0x7f42cc0c2330 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc157dc0 con 0x7f42cc0c2330 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.791+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1eec30 con 0x7f42cc0c2330 2024-03-06T07:24:55.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.792+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc04f330 con 0x7f42cc0c2330 2024-03-06T07:24:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.792+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42bc046020 con 0x7f42cc0c2330 2024-03-06T07:24:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.793+0000 7f42c2ffd640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 0x7f42ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.793+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f42bc0acd70 con 0x7f42cc0c2330 2024-03-06T07:24:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.793+0000 7f42c9d9b640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 0x7f42ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.794+0000 7f42c9d9b640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 0x7f42ac06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f42cc1512d0 tx=0x7f42b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:55.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.797+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f42bc04d030 con 0x7f42cc0c2330 2024-03-06T07:24:55.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:55.983+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f42cc0c6da0 con 0x7f42cc0c2330 2024-03-06T07:24:56.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.869+0000 7f42c2ffd640 1 -- 172.21.15.17:0/1843331330 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7f42bc07b9f0 con 0x7f42cc0c2330 2024-03-06T07:24:56.871 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 msgr2=0x7f42ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 0x7f42ac06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f42cc1512d0 tx=0x7f42b4008040 comp rx=0 tx=0).stop 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 msgr2=0x7f42cc151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f42bc000f80 tx=0x7f42bc004a40 comp rx=0 tx=0).stop 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 shutdown_connections 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42ac06b4d0 0x7f42ac06d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42cc1550c0 0x7f42cc152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42cc14bb20 0x7f42cc1ee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 --2- 172.21.15.17:0/1843331330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42cc0c2330 0x7f42cc151650 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 >> 172.21.15.17:0/1843331330 conn(0x7f42cc0c88f0 msgr2=0x7f42cc0a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 shutdown_connections 2024-03-06T07:24:56.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.873+0000 7f42d0c5c640 1 -- 172.21.15.17:0/1843331330 wait complete. 2024-03-06T07:24:56.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:24:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.998+0000 7f6aed302640 1 Processor -- start 2024-03-06T07:24:57.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.998+0000 7f6aed302640 1 -- start start 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 0x7f6ae80c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ae80c2330 0x7f6ae80c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 0x7f6ae8155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae80c3070 con 0x7f6ae80c2330 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae80a80e0 con 0x7f6ae814bb20 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6aed302640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8159260 con 0x7f6ae81550c0 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae67fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ae80c2330 0x7f6ae80c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae67fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6ae80c2330 0x7f6ae80c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49750/0 (socket says 172.21.15.17:49750) 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae67fc640 1 -- 172.21.15.17:0/3823193445 learned_addr learned my addr 172.21.15.17:0/3823193445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae6ffd640 1 --2- 172.21.15.17:0/3823193445 >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 0x7f6ae80c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae77fe640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 0x7f6ae8155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:56.999+0000 7f6ae67fc640 1 -- 172.21.15.17:0/3823193445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 msgr2=0x7f6ae8155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 0x7f6ae8155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae67fc640 1 -- 172.21.15.17:0/3823193445 >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 msgr2=0x7f6ae80c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/3823193445 >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 0x7f6ae80c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae67fc640 1 -- 172.21.15.17:0/3823193445 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80a7d70 con 0x7f6ae80c2330 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae6ffd640 1 --2- 172.21.15.17:0/3823193445 >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 0x7f6ae80c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae77fe640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 0x7f6ae8155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae80c5850 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80735a0 tx=0x7f6ad002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d471df4bae41c0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.000+0000 7f6ae5ffb640 1 -- 172.21.15.17:0/3823193445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad002e070 con 0x7f6ae80c2330 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6ae5ffb640 1 -- 172.21.15.17:0/3823193445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad0037070 con 0x7f6ae80c2330 2024-03-06T07:24:57.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6ae5ffb640 1 -- 172.21.15.17:0/3823193445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0031a00 con 0x7f6ae80c2330 2024-03-06T07:24:57.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 -- 172.21.15.17:0/3823193445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 msgr2=0x7f6ae80c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae80c5850 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80735a0 tx=0x7f6ad002b880 comp rx=0 tx=0).stop 2024-03-06T07:24:57.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 -- 172.21.15.17:0/3823193445 shutdown_connections 2024-03-06T07:24:57.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae81550c0 0x7f6ae8155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 --2- 172.21.15.17:0/3823193445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae80c5850 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 --2- 172.21.15.17:0/3823193445 >> v2:172.21.15.163:3300/0 conn(0x7f6ae814bb20 0x7f6ae80c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 -- 172.21.15.17:0/3823193445 >> 172.21.15.17:0/3823193445 conn(0x7f6ae80c88f0 msgr2=0x7f6ae80c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 -- 172.21.15.17:0/3823193445 shutdown_connections 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.001+0000 7f6aed302640 1 -- 172.21.15.17:0/3823193445 wait complete. 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.002+0000 7f6aed302640 1 Processor -- start 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.002+0000 7f6aed302640 1 -- start start 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 0x7f6ae8157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 0x7f6ae81ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80c67f0 con 0x7f6ae80c2330 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae814bf00 con 0x7f6ae81550c0 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6aed302640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae80c6970 con 0x7f6ae814bb20 2024-03-06T07:24:57.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49758/0 (socket says 172.21.15.17:49758) 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae6ffd640 1 -- 172.21.15.17:0/1146599053 learned_addr learned my addr 172.21.15.17:0/1146599053 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 0x7f6ae8157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae6ffd640 1 -- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 msgr2=0x7f6ae8157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.003+0000 7f6ae77fe640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 0x7f6ae81ebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae6ffd640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 0x7f6ae8157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae6ffd640 1 -- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 msgr2=0x7f6ae81ebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae6ffd640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 0x7f6ae81ebec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae6ffd640 1 -- 172.21.15.17:0/1146599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae81ee740 con 0x7f6ae80c2330 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 0x7f6ae8157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae77fe640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 0x7f6ae81ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6ae6ffd640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c820 tx=0x7f6ae000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0015070 con 0x7f6ae80c2330 2024-03-06T07:24:57.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0007a30 con 0x7f6ae80c2330 2024-03-06T07:24:57.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae814f500 con 0x7f6ae80c2330 2024-03-06T07:24:57.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0007e00 con 0x7f6ae80c2330 2024-03-06T07:24:57.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.004+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae80ab6b0 con 0x7f6ae80c2330 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.005+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae81ebb00 con 0x7f6ae80c2330 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.006+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae0018070 con 0x7f6ae80c2330 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.006+0000 7f6adf7fe640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 0x7f6ac806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.006+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f6ae008a190 con 0x7f6ae80c2330 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.006+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 0x7f6ac806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:57.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.007+0000 7f6ae67fc640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 0x7f6ac806d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0036000 tx=0x7f6ad0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:57.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.010+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ae0058fc0 con 0x7f6ae80c2330 2024-03-06T07:24:57.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.193+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6ae804f330 con 0x7f6ae80c2330 2024-03-06T07:24:57.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.874+0000 7f6adf7fe640 1 -- 172.21.15.17:0/1146599053 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6ae00588c0 con 0x7f6ae80c2330 2024-03-06T07:24:57.876 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:24:57.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 msgr2=0x7f6ac806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 0x7f6ac806d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0036000 tx=0x7f6ad0008030 comp rx=0 tx=0).stop 2024-03-06T07:24:57.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 msgr2=0x7f6ae814a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:57.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c820 tx=0x7f6ae000cc20 comp rx=0 tx=0).stop 2024-03-06T07:24:57.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 shutdown_connections 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ac806b500 0x7f6ac806d9c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae81550c0 0x7f6ae81ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6ae814bb20 0x7f6ae8157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 --2- 172.21.15.17:0/1146599053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae80c2330 0x7f6ae814a900 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.878+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 >> 172.21.15.17:0/1146599053 conn(0x7f6ae80c88f0 msgr2=0x7f6ae80723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.879+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 shutdown_connections 2024-03-06T07:24:57.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:57.879+0000 7f6aed302640 1 -- 172.21.15.17:0/1146599053 wait complete. 2024-03-06T07:24:57.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:24:58.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.003+0000 7f24c4181640 1 Processor -- start 2024-03-06T07:24:58.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.003+0000 7f24c4181640 1 -- start start 2024-03-06T07:24:58.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c4181640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24bc152730 0x7f24bc0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c4181640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24bc0c6710 0x7f24bc1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c4181640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24bc152b10 con 0x7f24bc0c3750 2024-03-06T07:24:58.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c4181640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24bc156110 con 0x7f24bc152730 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c4181640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24bc158ad0 con 0x7f24bc0c6710 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24bc152730 0x7f24bc0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24bc152730 0x7f24bc0c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49770/0 (socket says 172.21.15.17:49770) 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 -- 172.21.15.17:0/186144498 learned_addr learned my addr 172.21.15.17:0/186144498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 -- 172.21.15.17:0/186144498 >> v2:172.21.15.163:3300/0 conn(0x7f24bc0c6710 msgr2=0x7f24bc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/186144498 >> v2:172.21.15.163:3300/0 conn(0x7f24bc0c6710 0x7f24bc1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 -- 172.21.15.17:0/186144498 >> v1:172.21.15.163:6789/0 conn(0x7f24bc0c3750 legacy=0x7f24bc155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:58.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.004+0000 7f24c1f1d640 1 -- 172.21.15.17:0/186144498 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc158db0 con 0x7f24bc152730 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.005+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/186144498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc152730 0x7f24bc0c3210 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ab3a0 tx=0x7f24b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b43bcfaeb4a655b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.005+0000 7f24c0f1b640 1 -- 172.21.15.17:0/186144498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b402e070 con 0x7f24bc152730 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.005+0000 7f24c0f1b640 1 -- 172.21.15.17:0/186144498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4037070 con 0x7f24bc152730 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.005+0000 7f24c0f1b640 1 -- 172.21.15.17:0/186144498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4031a00 con 0x7f24bc152730 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 -- 172.21.15.17:0/186144498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc152730 msgr2=0x7f24bc0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 --2- 172.21.15.17:0/186144498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc152730 0x7f24bc0c3210 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ab3a0 tx=0x7f24b402b880 comp rx=0 tx=0).stop 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 -- 172.21.15.17:0/186144498 shutdown_connections 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 --2- 172.21.15.17:0/186144498 >> v2:172.21.15.163:3300/0 conn(0x7f24bc0c6710 0x7f24bc1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 --2- 172.21.15.17:0/186144498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc152730 0x7f24bc0c3210 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 -- 172.21.15.17:0/186144498 >> 172.21.15.17:0/186144498 conn(0x7f24bc0aa5c0 msgr2=0x7f24bc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 -- 172.21.15.17:0/186144498 shutdown_connections 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.006+0000 7f24c4181640 1 -- 172.21.15.17:0/186144498 wait complete. 2024-03-06T07:24:58.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.007+0000 7f24c4181640 1 Processor -- start 2024-03-06T07:24:58.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.007+0000 7f24c4181640 1 -- start start 2024-03-06T07:24:58.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 0x7f24bc151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc152730 0x7f24bc14b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc1586f0 con 0x7f24bc0c6710 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc159370 con 0x7f24bc152730 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c4181640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc158870 con 0x7f24bc0c3750 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49782/0 (socket says 172.21.15.17:49782) 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 -- 172.21.15.17:0/1878859378 learned_addr learned my addr 172.21.15.17:0/1878859378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c271e640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc152730 0x7f24bc14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 0x7f24bc151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 -- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 msgr2=0x7f24bc151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 0x7f24bc151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 -- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc152730 msgr2=0x7f24bc14b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc152730 0x7f24bc14b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 -- 172.21.15.17:0/1878859378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc06c170 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 0x7f24bc151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24c171c640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b450 tx=0x7f24b000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.008+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0015020 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.009+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0011890 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.009+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc06eab0 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.009+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0011c80 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.009+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc151bb0 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.009+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc04f330 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.010+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f24b0018020 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.010+0000 7f24ba7fc640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 0x7f24a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.010+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f24b008a2a0 con 0x7f24bc0c6710 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.011+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 0x7f24a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.011+0000 7f24c1f1d640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 0x7f24a806d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f24b4036000 tx=0x7f24b403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.015+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24b001f020 con 0x7f24bc0c6710 2024-03-06T07:24:58.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.198+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7f24bc073330 con 0x7f24bc0c6710 2024-03-06T07:24:58.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.201+0000 7f24ba7fc640 1 -- 172.21.15.17:0/1878859378 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v43) v1 ==== 96+0+0 (secure 0 0 0) 0x7f24b000d040 con 0x7f24bc0c6710 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 msgr2=0x7f24a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 0x7f24a806d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f24b4036000 tx=0x7f24b403d040 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 msgr2=0x7f24bc14aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f24b000b450 tx=0x7f24b000b910 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 shutdown_connections 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f24a806b4d0 0x7f24a806d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24bc152730 0x7f24bc14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24bc0c6710 0x7f24bc14aee0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 --2- 172.21.15.17:0/1878859378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24bc0c3750 0x7f24bc151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 >> 172.21.15.17:0/1878859378 conn(0x7f24bc0aa5c0 msgr2=0x7f24bc154360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 shutdown_connections 2024-03-06T07:24:58.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.204+0000 7f24c4181640 1 -- 172.21.15.17:0/1878859378 wait complete. 2024-03-06T07:24:58.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:24:58.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.322+0000 7f303b7ac640 1 Processor -- start 2024-03-06T07:24:58.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.322+0000 7f303b7ac640 1 -- start start 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30340b01e0 0x7f3034155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30341565b0 0x7f30340c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30340c5910 0x7f303406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340c7e00 con 0x7f30340c5910 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340c7840 con 0x7f30340b01e0 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f303b7ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c7b20 con 0x7f30341565b0 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30340c5910 0x7f303406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30340c5910 0x7f303406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49790/0 (socket says 172.21.15.17:49790) 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 -- 172.21.15.17:0/2579783528 learned_addr learned my addr 172.21.15.17:0/2579783528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:58.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3038d47640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30341565b0 0x7f30340c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039548640 1 --2- 172.21.15.17:0/2579783528 >> v2:172.21.15.163:3300/0 conn(0x7f30340b01e0 0x7f3034155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 -- 172.21.15.17:0/2579783528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30341565b0 msgr2=0x7f30340c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30341565b0 0x7f30340c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 -- 172.21.15.17:0/2579783528 >> v2:172.21.15.163:3300/0 conn(0x7f30340b01e0 msgr2=0x7f3034155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 --2- 172.21.15.17:0/2579783528 >> v2:172.21.15.163:3300/0 conn(0x7f30340b01e0 0x7f3034155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.323+0000 7f3039d49640 1 -- 172.21.15.17:0/2579783528 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303406b970 con 0x7f30340c5910 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f3039d49640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340c5910 0x7f303406c170 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3034073630 tx=0x7f302c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30d8e5691c41cf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f302bfff640 1 -- 172.21.15.17:0/2579783528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c02ca70 con 0x7f30340c5910 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f302bfff640 1 -- 172.21.15.17:0/2579783528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c02cc10 con 0x7f30340c5910 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f302bfff640 1 -- 172.21.15.17:0/2579783528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0343f0 con 0x7f30340c5910 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 -- 172.21.15.17:0/2579783528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340c5910 msgr2=0x7f303406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340c5910 0x7f303406c170 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3034073630 tx=0x7f302c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 -- 172.21.15.17:0/2579783528 shutdown_connections 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340c5910 0x7f303406c170 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 --2- 172.21.15.17:0/2579783528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30341565b0 0x7f30340c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 --2- 172.21.15.17:0/2579783528 >> v2:172.21.15.163:3300/0 conn(0x7f30340b01e0 0x7f3034155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.324+0000 7f303b7ac640 1 -- 172.21.15.17:0/2579783528 >> 172.21.15.17:0/2579783528 conn(0x7f30340ac600 msgr2=0x7f30340aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.325+0000 7f303b7ac640 1 -- 172.21.15.17:0/2579783528 shutdown_connections 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.325+0000 7f303b7ac640 1 -- 172.21.15.17:0/2579783528 wait complete. 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 Processor -- start 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 -- start start 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30340c5910 0x7f30340ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30341565b0 0x7f30340be860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340be3d0 con 0x7f30340b01e0 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340be250 con 0x7f30341565b0 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f303b7ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340c7460 con 0x7f30340c5910 2024-03-06T07:24:58.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f3039548640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f3039548640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49794/0 (socket says 172.21.15.17:49794) 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f3039548640 1 -- 172.21.15.17:0/1194475470 learned_addr learned my addr 172.21.15.17:0/1194475470 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f3038d47640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30340c5910 0x7f30340ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.326+0000 7f3039d49640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30341565b0 0x7f30340be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 -- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30340c5910 msgr2=0x7f30340ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30340c5910 0x7f30340ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 -- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30341565b0 msgr2=0x7f30340be860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30341565b0 0x7f30340be860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 -- 172.21.15.17:0/1194475470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303414ef10 con 0x7f30340b01e0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3039548640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f303000d800 tx=0x7f303000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030018030 con 0x7f30340b01e0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300096a0 con 0x7f30340b01e0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030009870 con 0x7f30340b01e0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30341ee7b0 con 0x7f30340b01e0 2024-03-06T07:24:58.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.327+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30340ca8b0 con 0x7f30340b01e0 2024-03-06T07:24:58.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.328+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034155050 con 0x7f30340b01e0 2024-03-06T07:24:58.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.329+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f303001b070 con 0x7f30340b01e0 2024-03-06T07:24:58.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.329+0000 7f3029ffb640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.329+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f303008a190 con 0x7f30340b01e0 2024-03-06T07:24:58.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.329+0000 7f3038d47640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.330+0000 7f3038d47640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f30340af530 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.333+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3030058b10 con 0x7f30340b01e0 2024-03-06T07:24:58.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.514+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f303404f330 con 0x7f30340b01e0 2024-03-06T07:24:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.517+0000 7f3029ffb640 1 -- 172.21.15.17:0/1194475470 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f303005cbb0 con 0x7f30340b01e0 2024-03-06T07:24:58.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 msgr2=0x7f301806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f30340af530 tx=0x7f3024008040 comp rx=0 tx=0).stop 2024-03-06T07:24:58.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 msgr2=0x7f3034155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f303000d800 tx=0x7f303000dcc0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 shutdown_connections 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30341565b0 0x7f30340be860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30340c5910 0x7f30340ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 --2- 172.21.15.17:0/1194475470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30340b01e0 0x7f3034155410 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 >> 172.21.15.17:0/1194475470 conn(0x7f30340ac600 msgr2=0x7f30340ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.519+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 shutdown_connections 2024-03-06T07:24:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.520+0000 7f303b7ac640 1 -- 172.21.15.17:0/1194475470 wait complete. 2024-03-06T07:24:58.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady' 2024-03-06T07:24:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.632+0000 7fad0df56640 1 Processor -- start 2024-03-06T07:24:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.632+0000 7fad0df56640 1 -- start start 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad0df56640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad08155ba0 0x7fad080c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad0df56640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad081586f0 0x7fad08158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad0df56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad081591c0 con 0x7fad080c5480 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad0df56640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0806df40 con 0x7fad08155ba0 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad0df56640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0806fdb0 con 0x7fad081586f0 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad06ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fad08155ba0 0x7fad080c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad077fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fad080c5480 0x7fad0806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49774/0 (socket says 172.21.15.17:49774) 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad077fe640 1 -- 172.21.15.17:0/1712050694 learned_addr learned my addr 172.21.15.17:0/1712050694 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad07fff640 1 --2- 172.21.15.17:0/1712050694 >> v2:172.21.15.163:3300/0 conn(0x7fad081586f0 0x7fad08158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad06ffd640 1 -- 172.21.15.17:0/1712050694 >> v2:172.21.15.163:3300/0 conn(0x7fad081586f0 msgr2=0x7fad08158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:24:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad06ffd640 1 --2- 172.21.15.17:0/1712050694 >> v2:172.21.15.163:3300/0 conn(0x7fad081586f0 0x7fad08158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad06ffd640 1 -- 172.21.15.17:0/1712050694 >> v1:172.21.15.17:6789/0 conn(0x7fad080c5480 legacy=0x7fad0806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.633+0000 7fad06ffd640 1 -- 172.21.15.17:0/1712050694 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad081594a0 con 0x7fad08155ba0 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.634+0000 7fad06ffd640 1 --2- 172.21.15.17:0/1712050694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad08155ba0 0x7fad080c5860 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fad080ab500 tx=0x7facfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b39541434c6ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.634+0000 7fad067fc640 1 -- 172.21.15.17:0/1712050694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc0315a0 con 0x7fad08155ba0 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.634+0000 7fad067fc640 1 -- 172.21.15.17:0/1712050694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc037070 con 0x7fad08155ba0 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.634+0000 7fad067fc640 1 -- 172.21.15.17:0/1712050694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc031960 con 0x7fad08155ba0 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 -- 172.21.15.17:0/1712050694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad08155ba0 msgr2=0x7fad080c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 --2- 172.21.15.17:0/1712050694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad08155ba0 0x7fad080c5860 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fad080ab500 tx=0x7facfc02b840 comp rx=0 tx=0).stop 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 -- 172.21.15.17:0/1712050694 shutdown_connections 2024-03-06T07:24:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 --2- 172.21.15.17:0/1712050694 >> v2:172.21.15.163:3300/0 conn(0x7fad081586f0 0x7fad08158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 --2- 172.21.15.17:0/1712050694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad08155ba0 0x7fad080c5860 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 -- 172.21.15.17:0/1712050694 >> 172.21.15.17:0/1712050694 conn(0x7fad0806b1a0 msgr2=0x7fad0806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 -- 172.21.15.17:0/1712050694 shutdown_connections 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.635+0000 7fad0df56640 1 -- 172.21.15.17:0/1712050694 wait complete. 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 Processor -- start 2024-03-06T07:24:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 -- start start 2024-03-06T07:24:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad080c5480 0x7fad080701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 0x7fad0814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08155680 con 0x7fad081586f0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad08156000 con 0x7fad08155ba0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.636+0000 7fad0df56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad08152a20 con 0x7fad080c5480 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49802/0 (socket says 172.21.15.17:49802) 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 -- 172.21.15.17:0/1547193908 learned_addr learned my addr 172.21.15.17:0/1547193908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad077fe640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad080c5480 0x7fad080701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad06ffd640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 0x7fad0814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 -- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad080c5480 msgr2=0x7fad080701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad080c5480 0x7fad080701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 -- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 msgr2=0x7fad0814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 0x7fad0814fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 -- 172.21.15.17:0/1547193908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08150920 con 0x7fad081586f0 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad06ffd640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 0x7fad0814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:24:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad07fff640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fad00007d20 tx=0x7fad0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00014420 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad00019070 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00014820 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad08147320 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.637+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad08145c80 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.638+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad08069460 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.639+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad000149c0 con 0x7fad081586f0 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.639+0000 7faceffff640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 0x7facf406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:24:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.639+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fad0008a940 con 0x7fad081586f0 2024-03-06T07:24:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.639+0000 7fad077fe640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 0x7facf406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:24:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.640+0000 7fad077fe640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 0x7facf406d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7facf8000f60 tx=0x7facf8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:24:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.643+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fad08069460 con 0x7fad081586f0 2024-03-06T07:24:58.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:58.897+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]} v 0) v1 -- 0x7fad0804f330 con 0x7fad081586f0 2024-03-06T07:24:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.880+0000 7faceffff640 1 -- 172.21.15.17:0/1547193908 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_steady"]}]=0 v296) v1 ==== 145+0+0 (secure 0 0 0) 0x7fad08150920 con 0x7fad081586f0 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 msgr2=0x7facf406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 0x7facf406d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7facf8000f60 tx=0x7facf8009040 comp rx=0 tx=0).stop 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 msgr2=0x7fad081503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fad00007d20 tx=0x7fad0000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 shutdown_connections 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facf406b500 0x7facf406d9c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad081586f0 0x7fad081503e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad08155ba0 0x7fad0814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 --2- 172.21.15.17:0/1547193908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad080c5480 0x7fad080701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:24:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 >> 172.21.15.17:0/1547193908 conn(0x7fad0806b1a0 msgr2=0x7fad08150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:24:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 shutdown_connections 2024-03-06T07:24:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:24:59.883+0000 7fad0df56640 1 -- 172.21.15.17:0/1547193908 wait complete. 2024-03-06T07:24:59.898 INFO:tasks.cephfs_test_runner:That a sole MDS with mds_join_fs set will come back as active eventually even after failover. ... ok 2024-03-06T07:24:59.898 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_unset (tasks.cephfs.test_failover.TestClusterAffinity) That unsetting mds_join_fs will cause failover if another high-affinity standby exists. 2024-03-06T07:24:59.898 INFO:tasks.cephfs_test_runner:test_join_fs_unset (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:24:59.899 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:00.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.005+0000 7f3b29b1e640 1 Processor -- start 2024-03-06T07:25:00.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.005+0000 7f3b29b1e640 1 -- start start 2024-03-06T07:25:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b29b1e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b24152520 0x7f3b2406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b29b1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b240720a0 0x7f3b2406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b29b1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b240c1ae0 con 0x7f3b24155c80 2024-03-06T07:25:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b29b1e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240c1dc0 con 0x7f3b24152520 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b29b1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2406e450 con 0x7f3b240720a0 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b22ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3b24155c80 0x7f3b240c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49786/0 (socket says 172.21.15.17:49786) 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b22ffd640 1 -- 172.21.15.17:0/3741711004 learned_addr learned my addr 172.21.15.17:0/3741711004 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b237fe640 1 --2- 172.21.15.17:0/3741711004 >> v2:172.21.15.163:3300/0 conn(0x7f3b24152520 0x7f3b2406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b23fff640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b240720a0 0x7f3b2406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b227fc640 1 -- 172.21.15.17:0/3741711004 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398941228 0 0) 0x7f3b240c1ae0 con 0x7f3b24155c80 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.006+0000 7f3b227fc640 1 -- 172.21.15.17:0/3741711004 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b0c003520 con 0x7f3b24155c80 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.007+0000 7f3b237fe640 1 -- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b240720a0 msgr2=0x7f3b2406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.007+0000 7f3b237fe640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b240720a0 0x7f3b2406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.007+0000 7f3b237fe640 1 -- 172.21.15.17:0/3741711004 >> v1:172.21.15.17:6789/0 conn(0x7f3b24155c80 legacy=0x7f3b240c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.007+0000 7f3b237fe640 1 -- 172.21.15.17:0/3741711004 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2406e730 con 0x7f3b24152520 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.007+0000 7f3b237fe640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b24152520 0x7f3b2406ef30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b140097a0 tx=0x7f3b1402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8447ec6d0b83e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b227fc640 1 -- 172.21.15.17:0/3741711004 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14035020 con 0x7f3b24152520 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b227fc640 1 -- 172.21.15.17:0/3741711004 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14031830 con 0x7f3b24152520 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b227fc640 1 -- 172.21.15.17:0/3741711004 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14031be0 con 0x7f3b24152520 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 -- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b24152520 msgr2=0x7f3b2406ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b24152520 0x7f3b2406ef30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b140097a0 tx=0x7f3b1402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 -- 172.21.15.17:0/3741711004 shutdown_connections 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b240720a0 0x7f3b2406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/3741711004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b24152520 0x7f3b2406ef30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 -- 172.21.15.17:0/3741711004 >> 172.21.15.17:0/3741711004 conn(0x7f3b240c76b0 msgr2=0x7f3b240c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 -- 172.21.15.17:0/3741711004 shutdown_connections 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.008+0000 7f3b29b1e640 1 -- 172.21.15.17:0/3741711004 wait complete. 2024-03-06T07:25:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.009+0000 7f3b29b1e640 1 Processor -- start 2024-03-06T07:25:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.009+0000 7f3b29b1e640 1 -- start start 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b240720a0 0x7f3b241e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b24155c80 0x7f3b241e10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24152a80 con 0x7f3b24152520 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b24152900 con 0x7f3b240720a0 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b29b1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2406b500 con 0x7f3b24155c80 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49804/0 (socket says 172.21.15.17:49804) 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 -- 172.21.15.17:0/963971070 learned_addr learned my addr 172.21.15.17:0/963971070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b237fe640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b240720a0 0x7f3b241e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b23fff640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b24155c80 0x7f3b241e10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 -- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b24155c80 msgr2=0x7f3b241e10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b24155c80 0x7f3b241e10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 -- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b240720a0 msgr2=0x7f3b241e1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b240720a0 0x7f3b241e1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.010+0000 7f3b22ffd640 1 -- 172.21.15.17:0/963971070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b241e15f0 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b22ffd640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800c430 tx=0x7f3b1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18017070 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b18005ce0 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b180072f0 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b24152fe0 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.011+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b241563d0 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.012+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2404f330 con 0x7f3b24152520 2024-03-06T07:25:00.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.012+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b1801a070 con 0x7f3b24152520 2024-03-06T07:25:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.013+0000 7f3b0bfff640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.013+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3b18089600 con 0x7f3b24152520 2024-03-06T07:25:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.013+0000 7f3b237fe640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.013+0000 7f3b237fe640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 0x7f3b1006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3b14036000 tx=0x7f3b14008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.017+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3b18057e60 con 0x7f3b24152520 2024-03-06T07:25:00.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.200+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b240c0b70 con 0x7f3b24152520 2024-03-06T07:25:00.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.201+0000 7f3b0bfff640 1 -- 172.21.15.17:0/963971070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+13561 (secure 0 0 0) 0x7f3b1805ce20 con 0x7f3b24152520 2024-03-06T07:25:00.207 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:00.207 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":127,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:24:57.870684+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/4288775541":"2024-03-07T07:24:54.613820+0000","172.21.15.17:6833/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6835/410228237":"2024-03-07T07:24:44.513269+0000","172.21.15.17:6835/4288775541":"2024-03-07T07:24:54.613820+0000","172.21.15.17:6832/1367930359":"2024-03-07T07:24:46.835623+0000","172.21.15.17:6834/410228237":"2024-03-07T07:24:44.513269+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 msgr2=0x7f3b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 0x7f3b1006d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3b14036000 tx=0x7f3b14008030 comp rx=0 tx=0).stop 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 msgr2=0x7f3b241e0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800c430 tx=0x7f3b1800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 shutdown_connections 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b24155c80 0x7f3b241e10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b24152520 0x7f3b241e0b70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 --2- 172.21.15.17:0/963971070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b240720a0 0x7f3b241e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 >> 172.21.15.17:0/963971070 conn(0x7f3b240c76b0 msgr2=0x7f3b240c6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 shutdown_connections 2024-03-06T07:25:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.203+0000 7f3b29b1e640 1 -- 172.21.15.17:0/963971070 wait complete. 2024-03-06T07:25:00.216 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:00.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.321+0000 7f91715f7640 1 Processor -- start 2024-03-06T07:25:00.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.321+0000 7f91715f7640 1 -- start start 2024-03-06T07:25:00.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f91715f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f916c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f91715f7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f916c072cd0 0x7f916c155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f91715f7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c152db0 con 0x7f916c0c57e0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f91715f7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0c6dd0 con 0x7f916c072cd0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f91715f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c6940 con 0x7f916c1529d0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f916a7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f916c0c57e0 0x7f916c155120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49790/0 (socket says 172.21.15.17:49790) 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f916a7fc640 1 -- 172.21.15.17:0/3181722045 learned_addr learned my addr 172.21.15.17:0/3181722045 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f916affd640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f916c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.322+0000 7f916b7fe640 1 --2- 172.21.15.17:0/3181722045 >> v2:172.21.15.163:3300/0 conn(0x7f916c072cd0 0x7f916c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536002086 0 0) 0x7f9160001110 con 0x7f916c0c57e0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9154003520 con 0x7f916c0c57e0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1694130285 0 0) 0x7f9154003520 con 0x7f916c0c57e0 2024-03-06T07:25:00.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 msgr2=0x7f916c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f916c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 >> v2:172.21.15.163:3300/0 conn(0x7f916c072cd0 msgr2=0x7f916c155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 --2- 172.21.15.17:0/3181722045 >> v2:172.21.15.163:3300/0 conn(0x7f916c072cd0 0x7f916c155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c1590f0 con 0x7f916c0c57e0 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9160001840 con 0x7f916c0c57e0 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 >> v1:172.21.15.17:6789/0 conn(0x7f916c0c57e0 legacy=0x7f916c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f91540039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9154003f30 0x7f9154004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c6dd0 con 0x7f9154004d40 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c6940 con 0x7f9154003f30 2024-03-06T07:25:00.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915400baa0 con 0x7f916c1529d0 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916affd640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f91540039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916a7fc640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9154003f30 0x7f9154004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 -- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 msgr2=0x7f91540039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c1529d0 0x7f91540039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 -- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9154003f30 msgr2=0x7f9154004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9154003f30 0x7f9154004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916b7fe640 1 -- 172.21.15.17:0/3181722045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9154003520 con 0x7f9154004d40 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.323+0000 7f916a7fc640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9154003f30 0x7f9154004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:00.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.324+0000 7f916b7fe640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9154004d40 0x7f91540051c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9164009ac0 tx=0x7f9154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50fad773c4fc67e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.324+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164039070 con 0x7f9154004d40 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.324+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916402c710 con 0x7f9154004d40 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.324+0000 7f9169ffb640 1 -- 172.21.15.17:0/3181722045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164009690 con 0x7f9154004d40 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 -- 172.21.15.17:0/3181722045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9154004d40 msgr2=0x7f91540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 --2- 172.21.15.17:0/3181722045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9154004d40 0x7f91540051c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9164009ac0 tx=0x7f9154005830 comp rx=0 tx=0).stop 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f916b7fe640 1 -- 172.21.15.17:0/3181722045 reap_dead start 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 -- 172.21.15.17:0/3181722045 shutdown_connections 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 -- 172.21.15.17:0/3181722045 >> 172.21.15.17:0/3181722045 conn(0x7f916c0aa630 msgr2=0x7f916c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 -- 172.21.15.17:0/3181722045 shutdown_connections 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.325+0000 7f91715f7640 1 -- 172.21.15.17:0/3181722045 wait complete. 2024-03-06T07:25:00.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.326+0000 7f91715f7640 1 Processor -- start 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.326+0000 7f91715f7640 1 -- start start 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c0ad8a0 0x7f916c0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f916c0adc80 0x7f916c0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c072720 con 0x7f916c0ae100 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c155d70 con 0x7f916c0adc80 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c159370 con 0x7f916c0ad8a0 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49822/0 (socket says 172.21.15.17:49822) 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 -- 172.21.15.17:0/1413103849 learned_addr learned my addr 172.21.15.17:0/1413103849 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:00.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916affd640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c0ad8a0 0x7f916c0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916a7fc640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f916c0adc80 0x7f916c0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 -- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c0ad8a0 msgr2=0x7f916c0c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c0ad8a0 0x7f916c0c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 -- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f916c0adc80 msgr2=0x7f916c0ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f916c0adc80 0x7f916c0ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 -- 172.21.15.17:0/1413103849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c1eb640 con 0x7f916c0ae100 2024-03-06T07:25:00.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f916b7fe640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9164009be0 tx=0x7f9164004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164039070 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164043070 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164009690 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.327+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c1eb8e0 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.328+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c14c3f0 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.328+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c04f330 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.329+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9164003850 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.329+0000 7f91537fe640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.329+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f91640ab6d0 con 0x7f916c0ae100 2024-03-06T07:25:00.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.330+0000 7f916affd640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.330+0000 7f916affd640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 0x7f915806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f915c014510 tx=0x7f915c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.334+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f916407a050 con 0x7f916c0ae100 2024-03-06T07:25:00.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.513+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f916c073250 con 0x7f916c0ae100 2024-03-06T07:25:00.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.513+0000 7f91537fe640 1 -- 172.21.15.17:0/1413103849 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f916407e0f0 con 0x7f916c0ae100 2024-03-06T07:25:00.515 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 msgr2=0x7f915806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 0x7f915806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f915c014510 tx=0x7f915c00b040 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 msgr2=0x7f916c1edf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9164009be0 tx=0x7f9164004730 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 shutdown_connections 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f915806b4d0 0x7f915806d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f916c0ae100 0x7f916c1edf90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f916c0adc80 0x7f916c0ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 --2- 172.21.15.17:0/1413103849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f916c0ad8a0 0x7f916c0c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 >> 172.21.15.17:0/1413103849 conn(0x7f916c0aa630 msgr2=0x7f916c0c2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 shutdown_connections 2024-03-06T07:25:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.515+0000 7f91715f7640 1 -- 172.21.15.17:0/1413103849 wait complete. 2024-03-06T07:25:00.527 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset' 2024-03-06T07:25:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.627+0000 7f2f6df1e640 1 Processor -- start 2024-03-06T07:25:00.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.627+0000 7f2f6df1e640 1 -- start start 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f681506b0 0x7f2f68153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2f68156ea0 0x7f2f680702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f680c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f68156450 con 0x7f2f681506b0 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f68156a10 con 0x7f2f68156ea0 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f6df1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f68070800 con 0x7f2f680acc00 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f681506b0 0x7f2f68153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f681506b0 0x7f2f68153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33512/0 (socket says 172.21.15.17:33512) 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 -- 172.21.15.17:0/199480994 learned_addr learned my addr 172.21.15.17:0/199480994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:00.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f66ffd640 1 --2- 172.21.15.17:0/199480994 >> v2:172.21.15.163:3300/0 conn(0x7f2f68156ea0 0x7f2f680702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f67fff640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f680c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 -- 172.21.15.17:0/199480994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 msgr2=0x7f2f680c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f680c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 -- 172.21.15.17:0/199480994 >> v2:172.21.15.163:3300/0 conn(0x7f2f68156ea0 msgr2=0x7f2f680702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 --2- 172.21.15.17:0/199480994 >> v2:172.21.15.163:3300/0 conn(0x7f2f68156ea0 0x7f2f680702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.628+0000 7f2f677fe640 1 -- 172.21.15.17:0/199480994 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68070aa0 con 0x7f2f681506b0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.629+0000 7f2f677fe640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68153990 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2f6806bf40 tx=0x7f2f5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e228c70641585c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.629+0000 7f2f667fc640 1 -- 172.21.15.17:0/199480994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58035020 con 0x7f2f681506b0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.629+0000 7f2f667fc640 1 -- 172.21.15.17:0/199480994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f58031870 con 0x7f2f681506b0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.629+0000 7f2f667fc640 1 -- 172.21.15.17:0/199480994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58031c40 con 0x7f2f681506b0 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 -- 172.21.15.17:0/199480994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 msgr2=0x7f2f68153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68153990 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2f6806bf40 tx=0x7f2f5802b910 comp rx=0 tx=0).stop 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 -- 172.21.15.17:0/199480994 shutdown_connections 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f680c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/199480994 >> v2:172.21.15.163:3300/0 conn(0x7f2f68156ea0 0x7f2f680702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/199480994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68153990 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 -- 172.21.15.17:0/199480994 >> 172.21.15.17:0/199480994 conn(0x7f2f6806b1a0 msgr2=0x7f2f6806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 -- 172.21.15.17:0/199480994 shutdown_connections 2024-03-06T07:25:00.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.630+0000 7f2f6df1e640 1 -- 172.21.15.17:0/199480994 wait complete. 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.631+0000 7f2f6df1e640 1 Processor -- start 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.631+0000 7f2f6df1e640 1 -- start start 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f681490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f68156ea0 0x7f2f681466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680c59e0 con 0x7f2f681506b0 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f681566a0 con 0x7f2f68156ea0 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f6df1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680c5b60 con 0x7f2f680acc00 2024-03-06T07:25:00.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33522/0 (socket says 172.21.15.17:33522) 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 -- 172.21.15.17:0/401799068 learned_addr learned my addr 172.21.15.17:0/401799068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f67fff640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f68156ea0 0x7f2f681466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f677fe640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f681490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 -- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 msgr2=0x7f2f681490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f681490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 -- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f68156ea0 msgr2=0x7f2f681466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f68156ea0 0x7f2f681466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 -- 172.21.15.17:0/401799068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68155810 con 0x7f2f681506b0 2024-03-06T07:25:00.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f677fe640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f681490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.632+0000 7f2f66ffd640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00d620 tx=0x7f2f5c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.633+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00b620 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.633+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c019070 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.633+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00ba00 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.633+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f68155af0 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.633+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f68146be0 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.634+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6804f330 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.634+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f5c00bba0 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.634+0000 7f2f4ffff640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 0x7f2f5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.635+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2f5c08a490 con 0x7f2f681506b0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.635+0000 7f2f677fe640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 0x7f2f5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:00.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.635+0000 7f2f677fe640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 0x7f2f5406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f58008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:00.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.639+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f5c059460 con 0x7f2f681506b0 2024-03-06T07:25:00.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:00.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]} v 0) v1 -- 0x7f2f680700a0 con 0x7f2f681506b0 2024-03-06T07:25:01.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.886+0000 7f2f4ffff640 1 -- 172.21.15.17:0/401799068 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]}]=0 v298) v1 ==== 147+0+0 (secure 0 0 0) 0x7f2f5c058d60 con 0x7f2f681506b0 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 msgr2=0x7f2f5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 0x7f2f5406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f58008030 comp rx=0 tx=0).stop 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 msgr2=0x7f2f68146160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00d620 tx=0x7f2f5c00dae0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 shutdown_connections 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f5406b500 0x7f2f5406d9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f68156ea0 0x7f2f681466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f681506b0 0x7f2f68146160 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 --2- 172.21.15.17:0/401799068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f680acc00 0x7f2f681490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 >> 172.21.15.17:0/401799068 conn(0x7f2f6806b1a0 msgr2=0x7f2f6814f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.888+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 shutdown_connections 2024-03-06T07:25:01.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:01.889+0000 7f2f6df1e640 1 -- 172.21.15.17:0/401799068 wait complete. 2024-03-06T07:25:01.903 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:25:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.012+0000 7fc155d25640 1 Processor -- start 2024-03-06T07:25:02.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.012+0000 7fc155d25640 1 -- start start 2024-03-06T07:25:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc155d25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc15006ca70 0x7fc15014e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc155d25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1500b0450 con 0x7fc1500b0070 2024-03-06T07:25:02.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc155d25640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1501560e0 con 0x7fc150155c30 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc155d25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15006c020 con 0x7fc15006ca70 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc14ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc15006ca70 0x7fc15014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc14f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1500b0070 0x7fc1500c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41086/0 (socket says 172.21.15.17:41086) 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.013+0000 7fc14f7fe640 1 -- 172.21.15.17:0/3453484416 learned_addr learned my addr 172.21.15.17:0/3453484416 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.014+0000 7fc14ffff640 1 -- 172.21.15.17:0/3453484416 >> v1:172.21.15.163:6789/0 conn(0x7fc150155c30 legacy=0x7fc1500c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.014+0000 7fc14ffff640 1 -- 172.21.15.17:0/3453484416 >> v1:172.21.15.17:6789/0 conn(0x7fc1500b0070 legacy=0x7fc1500c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.014+0000 7fc14ffff640 1 -- 172.21.15.17:0/3453484416 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500c77f0 con 0x7fc15006ca70 2024-03-06T07:25:02.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.014+0000 7fc14e7fc640 1 -- 172.21.15.17:0/3453484416 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051923519 0 0) 0x7fc140001110 con 0x7fc1500b0070 2024-03-06T07:25:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.014+0000 7fc14ffff640 1 --2- 172.21.15.17:0/3453484416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc15014e800 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc148009d90 tx=0x7fc14802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25d33ae40660017 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc14e7fc640 1 -- 172.21.15.17:0/3453484416 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148039070 con 0x7fc15006ca70 2024-03-06T07:25:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc14e7fc640 1 -- 172.21.15.17:0/3453484416 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14802cc60 con 0x7fc15006ca70 2024-03-06T07:25:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc14e7fc640 1 -- 172.21.15.17:0/3453484416 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148034660 con 0x7fc15006ca70 2024-03-06T07:25:02.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 -- 172.21.15.17:0/3453484416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 msgr2=0x7fc15014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 --2- 172.21.15.17:0/3453484416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc15014e800 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc148009d90 tx=0x7fc14802beb0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 -- 172.21.15.17:0/3453484416 shutdown_connections 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 --2- 172.21.15.17:0/3453484416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc15014e800 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 -- 172.21.15.17:0/3453484416 >> 172.21.15.17:0/3453484416 conn(0x7fc1500ac590 msgr2=0x7fc1500ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 -- 172.21.15.17:0/3453484416 shutdown_connections 2024-03-06T07:25:02.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.015+0000 7fc155d25640 1 -- 172.21.15.17:0/3453484416 wait complete. 2024-03-06T07:25:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.016+0000 7fc155d25640 1 Processor -- start 2024-03-06T07:25:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.016+0000 7fc155d25640 1 -- start start 2024-03-06T07:25:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 0x7fc150150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc150155c30 0x7fc15014b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15006c550 con 0x7fc15006ca70 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15006c270 con 0x7fc150155c30 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc155d25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15006da80 con 0x7fc1500b0070 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc14f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33540/0 (socket says 172.21.15.17:33540) 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc14f7fe640 1 -- 172.21.15.17:0/1150079754 learned_addr learned my addr 172.21.15.17:0/1150079754 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc14ffff640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc150155c30 0x7fc15014b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.017+0000 7fc14effd640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 0x7fc150150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 -- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 msgr2=0x7fc150150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 0x7fc150150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 -- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc150155c30 msgr2=0x7fc15014b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc150155c30 0x7fc15014b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 -- 172.21.15.17:0/1150079754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1501eb7a0 con 0x7fc15006ca70 2024-03-06T07:25:02.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14effd640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 0x7fc150150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc14f7fe640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c7b0 tx=0x7fc14000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140018070 con 0x7fc15006ca70 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1400072d0 con 0x7fc15006ca70 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1500c4bf0 con 0x7fc15006ca70 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1400047e0 con 0x7fc15006ca70 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.018+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501ee620 con 0x7fc15006ca70 2024-03-06T07:25:02.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.019+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15004f330 con 0x7fc15006ca70 2024-03-06T07:25:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.019+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc14001b070 con 0x7fc15006ca70 2024-03-06T07:25:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.020+0000 7fc137fff640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 0x7fc13c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.020+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc140089270 con 0x7fc15006ca70 2024-03-06T07:25:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.020+0000 7fc14effd640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 0x7fc13c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.020+0000 7fc14effd640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 0x7fc13c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc144007fd0 tx=0x7fc144009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.024+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc1500b0450 con 0x7fc15006ca70 2024-03-06T07:25:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.203+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc15006d860 con 0x7fc15006ca70 2024-03-06T07:25:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.206+0000 7fc137fff640 1 -- 172.21.15.17:0/1150079754 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc140007d00 con 0x7fc15006ca70 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 msgr2=0x7fc13c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 0x7fc13c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc144007fd0 tx=0x7fc144009040 comp rx=0 tx=0).stop 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 msgr2=0x7fc1501ead70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c7b0 tx=0x7fc14000cc70 comp rx=0 tx=0).stop 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 shutdown_connections 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc13c06b4d0 0x7fc13c06d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc150155c30 0x7fc15014b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1500b0070 0x7fc150150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 --2- 172.21.15.17:0/1150079754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc15006ca70 0x7fc1501ead70 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 >> 172.21.15.17:0/1150079754 conn(0x7fc1500ac590 msgr2=0x7fc150151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 shutdown_connections 2024-03-06T07:25:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.208+0000 7fc155d25640 1 -- 172.21.15.17:0/1150079754 wait complete. 2024-03-06T07:25:02.221 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:02.221 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:02.263 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:02.307 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:02.307 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:25:02.307 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:02.322 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:02.322 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:25:02.322 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:02.366 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:25:02.366 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:25:02.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.465+0000 7fea7cedf640 1 Processor -- start 2024-03-06T07:25:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.465+0000 7fea7cedf640 1 -- start start 2024-03-06T07:25:02.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea7cedf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea7814f360 0x7fea78155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea7cedf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea780c88c0 0x7fea780c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea7cedf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea780c73d0 con 0x7fea780c5880 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea7cedf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780c8dd0 con 0x7fea780c88c0 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea7cedf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea780be2e0 con 0x7fea7814f360 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea75d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea780c88c0 0x7fea780c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea75d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea780c88c0 0x7fea780c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33544/0 (socket says 172.21.15.17:33544) 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.466+0000 7fea75d9b640 1 -- 172.21.15.17:0/3093433527 learned_addr learned my addr 172.21.15.17:0/3093433527 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea75d9b640 1 -- 172.21.15.17:0/3093433527 >> v2:172.21.15.163:3300/0 conn(0x7fea7814f360 msgr2=0x7fea78155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea7659c640 1 --2- 172.21.15.17:0/3093433527 >> v2:172.21.15.163:3300/0 conn(0x7fea7814f360 0x7fea78155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea75d9b640 1 --2- 172.21.15.17:0/3093433527 >> v2:172.21.15.163:3300/0 conn(0x7fea7814f360 0x7fea78155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea75d9b640 1 -- 172.21.15.17:0/3093433527 >> v1:172.21.15.163:6789/0 conn(0x7fea780c5880 legacy=0x7fea78158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea75d9b640 1 -- 172.21.15.17:0/3093433527 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780c7670 con 0x7fea780c88c0 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea75d9b640 1 --2- 172.21.15.17:0/3093433527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea780c88c0 0x7fea780c5340 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fea7808c840 tx=0x7fea6002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ad8148d55233b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.467+0000 7fea7559a640 1 -- 172.21.15.17:0/3093433527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6002e070 con 0x7fea780c88c0 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7559a640 1 -- 172.21.15.17:0/3093433527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea60037070 con 0x7fea780c88c0 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7559a640 1 -- 172.21.15.17:0/3093433527 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60031a00 con 0x7fea780c88c0 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 -- 172.21.15.17:0/3093433527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea780c88c0 msgr2=0x7fea780c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3093433527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea780c88c0 0x7fea780c5340 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fea7808c840 tx=0x7fea6002b880 comp rx=0 tx=0).stop 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 -- 172.21.15.17:0/3093433527 shutdown_connections 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3093433527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea780c88c0 0x7fea780c5340 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3093433527 >> v2:172.21.15.163:3300/0 conn(0x7fea7814f360 0x7fea78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 -- 172.21.15.17:0/3093433527 >> 172.21.15.17:0/3093433527 conn(0x7fea780ac570 msgr2=0x7fea780ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 -- 172.21.15.17:0/3093433527 shutdown_connections 2024-03-06T07:25:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.468+0000 7fea7cedf640 1 -- 172.21.15.17:0/3093433527 wait complete. 2024-03-06T07:25:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.469+0000 7fea7cedf640 1 Processor -- start 2024-03-06T07:25:02.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.469+0000 7fea7cedf640 1 -- start start 2024-03-06T07:25:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 0x7fea781eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c88c0 0x7fea780aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea78159270 con 0x7fea7814f360 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea780c7c30 con 0x7fea780c5880 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7cedf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea781593f0 con 0x7fea780c88c0 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33560/0 (socket says 172.21.15.17:33560) 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 -- 172.21.15.17:0/3368547845 learned_addr learned my addr 172.21.15.17:0/3368547845 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7659c640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 0x7fea781eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea75d9b640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c88c0 0x7fea780aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 -- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c88c0 msgr2=0x7fea780aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c88c0 0x7fea780aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 -- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 msgr2=0x7fea781eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 0x7fea781eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea76d9d640 1 -- 172.21.15.17:0/3368547845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780af380 con 0x7fea7814f360 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.470+0000 7fea7659c640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 0x7fea781eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:02.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea76d9d640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fea6800c780 tx=0x7fea6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68004b60 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68004d00 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea78153330 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6800aa90 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.471+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea780aff80 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.472+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7804f330 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.472+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea6800ac30 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.473+0000 7fea6effd640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.473+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fea6808a670 con 0x7fea7814f360 2024-03-06T07:25:02.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.473+0000 7fea7659c640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.473+0000 7fea7659c640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.477+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea680595b0 con 0x7fea7814f360 2024-03-06T07:25:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.694+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea7806cda0 con 0x7fea7814f360 2024-03-06T07:25:02.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.695+0000 7fea6effd640 1 -- 172.21.15.17:0/3368547845 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+3710 (secure 0 0 0) 0x7fea68058eb0 con 0x7fea7814f360 2024-03-06T07:25:02.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:02.697 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1963602121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1963602121},{"type":"v1","addr":"172.21.15.163:6836","nonce":1963602121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6527,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2989703655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2989703655},{"type":"v1","addr":"172.21.15.163:6837","nonce":2989703655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":8085,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3939235170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3939235170},{"type":"v1","addr":"172.21.15.17:6833","nonce":3939235170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112},{"gid":8181,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/3213632895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3213632895},{"type":"v1","addr":"172.21.15.17:6835","nonce":3213632895}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[]} 2024-03-06T07:25:02.697 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 116 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 msgr2=0x7fea5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 msgr2=0x7fea78153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fea6800c780 tx=0x7fea6800cc40 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 shutdown_connections 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea7814f360 0x7fea78153ef0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea780c88c0 0x7fea780aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 --2- 172.21.15.17:0/3368547845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea780c5880 0x7fea781eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 >> 172.21.15.17:0/3368547845 conn(0x7fea780ac570 msgr2=0x7fea7806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 shutdown_connections 2024-03-06T07:25:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.697+0000 7fea7cedf640 1 -- 172.21.15.17:0/3368547845 wait complete. 2024-03-06T07:25:02.709 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:25:02.710 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:25:02.710 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:25:02.710 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:02.710 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:25:02.710 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:25:02.710 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:25:02.711 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:02.711 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:25:02.711 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:25:02.711 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:25:02.711 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:02.711 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:25:02.711 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:25:02.711 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:25:02.712 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:02.719 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:25:02.719 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:25:02.719 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:25:02.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:25:02.723 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:25:02.723 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:25:02.723 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:25:02.724 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:25:02.762 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:25:02.762 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:02.762 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:25:02.763 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:02.765 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:25:02.765 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:02.766 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:25:02.766 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:02.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:25:02.838 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:25:02.838 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:25:02.848 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:25:02.848 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:25:02.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 Processor -- start 2024-03-06T07:25:02.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 -- start start 2024-03-06T07:25:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2438158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2438155150 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24381553d0 con 0x7f24380c27b0 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.871+0000 7f243ca2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24380c24c0 con 0x7f24380c20b0 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2438152740 0x7f243806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41098/0 (socket says 172.21.15.17:41098) 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243659c640 1 -- 172.21.15.17:0/2037744280 learned_addr learned my addr 172.21.15.17:0/2037744280 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2438158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381453221 0 0) 0x7f2420001110 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2424003520 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 245408714 0 0) 0x7f2424003520 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 msgr2=0x7f2438158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2438158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 >> v1:172.21.15.163:6789/0 conn(0x7f24380c27b0 legacy=0x7f2438158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24381556b0 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f2420001840 con 0x7f2438152740 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2438158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:02.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 >> v1:172.21.15.17:6789/0 conn(0x7f2438152740 legacy=0x7f243806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2424003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2424004390 0x7f2424004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2424005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24380c24c0 con 0x7f2424004390 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242400ba10 con 0x7f24240039e0 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242400bc10 con 0x7f24380c20b0 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f243659c640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2424003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2424005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.872+0000 7f2435d9b640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2424004390 0x7f2424004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 -- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 msgr2=0x7f2424005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24380c20b0 0x7f2424005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 -- 172.21.15.17:0/2037744280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2424004390 msgr2=0x7f2424004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2424004390 0x7f2424004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 -- 172.21.15.17:0/2037744280 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424003520 con 0x7f24240039e0 2024-03-06T07:25:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.873+0000 7f243659c640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2424003dc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f242000c4c0 tx=0x7f2424005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1e2ec45ca838604 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242000c0e0 con 0x7f24240039e0 2024-03-06T07:25:02.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2420030070 con 0x7f24240039e0 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243559a640 1 -- 172.21.15.17:0/2037744280 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24200058d0 con 0x7f24240039e0 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 -- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 msgr2=0x7f2424003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2037744280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2424003dc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f242000c4c0 tx=0x7f2424005830 comp rx=0 tx=0).stop 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f2436d9d640 1 -- 172.21.15.17:0/2037744280 reap_dead start 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 -- 172.21.15.17:0/2037744280 shutdown_connections 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 -- 172.21.15.17:0/2037744280 >> 172.21.15.17:0/2037744280 conn(0x7f24380c41b0 msgr2=0x7f24380c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 -- 172.21.15.17:0/2037744280 shutdown_connections 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.874+0000 7f243ca2a640 1 -- 172.21.15.17:0/2037744280 wait complete. 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 Processor -- start 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 -- start start 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2438149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24381ebe10 0x7f24381ec250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243806be80 con 0x7f24381ec790 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438155d00 con 0x7f24240039e0 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.876+0000 7f243ca2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2438152b20 con 0x7f24381ebe10 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33618/0 (socket says 172.21.15.17:33618) 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 -- 172.21.15.17:0/2042791143 learned_addr learned my addr 172.21.15.17:0/2042791143 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2435d9b640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24381ebe10 0x7f24381ec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f243659c640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2438149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 -- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24381ebe10 msgr2=0x7f24381ec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24381ebe10 0x7f24381ec250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 -- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 msgr2=0x7f2438149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2438149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 -- 172.21.15.17:0/2042791143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24381ee7c0 con 0x7f24381ec790 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f243659c640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2438149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f2436d9d640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f24380aa500 tx=0x7f2430009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243000fcf0 con 0x7f24381ec790 2024-03-06T07:25:02.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430003990 con 0x7f24381ec790 2024-03-06T07:25:02.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24380a6cd0 con 0x7f24381ec790 2024-03-06T07:25:02.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430003d80 con 0x7f24381ec790 2024-03-06T07:25:02.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.877+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2438144fe0 con 0x7f24381ec790 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.879+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243804f330 con 0x7f24381ec790 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.879+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f243001f020 con 0x7f24381ec790 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.879+0000 7f242effd640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 0x7f241806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.879+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f243008a410 con 0x7f24381ec790 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.880+0000 7f243659c640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 0x7f241806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:02.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.880+0000 7f243659c640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 0x7f241806d9c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2420034000 tx=0x7f2420007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:02.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:02.884+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2430010040 con 0x7f24381ec790 2024-03-06T07:25:03.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.082+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24381451b0 con 0x7f24381ec790 2024-03-06T07:25:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.899+0000 7f242effd640 1 -- 172.21.15.17:0/2042791143 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7f24381451b0 con 0x7f24381ec790 2024-03-06T07:25:03.900 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:25:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 msgr2=0x7f241806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 0x7f241806d9c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2420034000 tx=0x7f2420007000 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 msgr2=0x7f2438145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f24380aa500 tx=0x7f2430009f90 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 shutdown_connections 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f241806b500 0x7f241806d9c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24381ec790 0x7f2438145fe0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24381ebe10 0x7f24381ec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 --2- 172.21.15.17:0/2042791143 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240039e0 0x7f2438149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 >> 172.21.15.17:0/2042791143 conn(0x7f24380c41b0 msgr2=0x7f243806f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 shutdown_connections 2024-03-06T07:25:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:03.902+0000 7f243ca2a640 1 -- 172.21.15.17:0/2042791143 wait complete. 2024-03-06T07:25:03.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:25:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.023+0000 7f673f8a4640 1 Processor -- start 2024-03-06T07:25:04.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.023+0000 7f673f8a4640 1 -- start start 2024-03-06T07:25:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673f8a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67380c7380 0x7f67380c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673f8a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6738072630 con 0x7f673806da40 2024-03-06T07:25:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673f8a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6738072910 con 0x7f67380c6780 2024-03-06T07:25:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673f8a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673806de20 con 0x7f67380c7380 2024-03-06T07:25:04.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673d640640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f673806da40 0x7f6738155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41104/0 (socket says 172.21.15.17:41104) 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673d640640 1 -- 172.21.15.17:0/3909618310 learned_addr learned my addr 172.21.15.17:0/3909618310 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165836909 0 0) 0x7f6738072630 con 0x7f673806da40 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6724003520 con 0x7f673806da40 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f673de41640 1 --2- 172.21.15.17:0/3909618310 >> v2:172.21.15.163:3300/0 conn(0x7f67380c7380 0x7f67380c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208107611 0 0) 0x7f6724003520 con 0x7f673806da40 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.024+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 >> v2:172.21.15.163:3300/0 conn(0x7f67380c7380 msgr2=0x7f67380c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 --2- 172.21.15.17:0/3909618310 >> v2:172.21.15.163:3300/0 conn(0x7f67380c7380 0x7f67380c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 >> v1:172.21.15.163:6789/0 conn(0x7f67380c6780 legacy=0x7f6738158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738072bf0 con 0x7f673806da40 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6730001670 con 0x7f673806da40 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 >> v1:172.21.15.17:6789/0 conn(0x7f673806da40 legacy=0x7f6738155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 0x7f6724003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6724004800 0x7f6724004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240052e0 0x7f672400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673806de20 con 0x7f67240039b0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672400c1b0 con 0x7f6724004800 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672400c3b0 con 0x7f67240052e0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673de41640 1 --2- 172.21.15.17:0/3909618310 >> v2:172.21.15.163:3300/0 conn(0x7f67380c7380 0x7f67380c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 0x7f6724003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673ce3f640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6724004800 0x7f6724004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673de41640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240052e0 0x7f672400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 -- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240052e0 msgr2=0x7f672400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240052e0 0x7f672400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 -- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6724004800 msgr2=0x7f6724004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6724004800 0x7f6724004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673d640640 1 -- 172.21.15.17:0/3909618310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724003520 con 0x7f67240039b0 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673de41640 1 -- 172.21.15.17:0/3909618310 reap_dead start 2024-03-06T07:25:04.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.025+0000 7f673ce3f640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6724004800 0x7f6724004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673d640640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 0x7f6724003dd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6730001eb0 tx=0x7f6724005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f1b28efe68c71 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730007d30 con 0x7f67240039b0 2024-03-06T07:25:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6730030070 con 0x7f67240039b0 2024-03-06T07:25:04.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f672ffff640 1 -- 172.21.15.17:0/3909618310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730005d70 con 0x7f67240039b0 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673f8a4640 1 -- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 msgr2=0x7f6724003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 0x7f6724003dd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6730001eb0 tx=0x7f6724005830 comp rx=0 tx=0).stop 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673f8a4640 1 -- 172.21.15.17:0/3909618310 shutdown_connections 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3909618310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67240039b0 0x7f6724003dd0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.026+0000 7f673f8a4640 1 -- 172.21.15.17:0/3909618310 >> 172.21.15.17:0/3909618310 conn(0x7f67380aa630 msgr2=0x7f67380aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.027+0000 7f673f8a4640 1 -- 172.21.15.17:0/3909618310 shutdown_connections 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.027+0000 7f673f8a4640 1 -- 172.21.15.17:0/3909618310 wait complete. 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 Processor -- start 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 -- start start 2024-03-06T07:25:04.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 0x7f67380acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 0x7f67380ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738155c90 con 0x7f67380abb80 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738155b10 con 0x7f673806f5a0 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673f8a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6738155e10 con 0x7f67240039b0 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673de41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.028+0000 7f673de41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33634/0 (socket says 172.21.15.17:33634) 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 -- 172.21.15.17:0/3789728846 learned_addr learned my addr 172.21.15.17:0/3789728846 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673ce3f640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 0x7f67380ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673d640640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 0x7f67380acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 -- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 msgr2=0x7f67380acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 0x7f67380acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 -- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 msgr2=0x7f67380ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 0x7f67380ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 -- 172.21.15.17:0/3789728846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6738070240 con 0x7f67380abb80 2024-03-06T07:25:04.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673ce3f640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 0x7f67380ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673d640640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 0x7f67380acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673de41640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f67380c7830 tx=0x7f6734004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734007d60 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734019070 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67380ad370 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734014650 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.029+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67381eb060 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.030+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673804f330 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.031+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67340147f0 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.031+0000 7f672dffb640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 0x7f671c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.031+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f673408a490 con 0x7f67380abb80 2024-03-06T07:25:04.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.031+0000 7f673d640640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 0x7f671c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.032+0000 7f673d640640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 0x7f671c06d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6730034000 tx=0x7f6730009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.035+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6734059560 con 0x7f67380abb80 2024-03-06T07:25:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.286+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6738073240 con 0x7f67380abb80 2024-03-06T07:25:04.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.287+0000 7f672dffb640 1 -- 172.21.15.17:0/3789728846 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f6734058e60 con 0x7f67380abb80 2024-03-06T07:25:04.288 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.289 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:04.290 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.292 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:04.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:25:04.298 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:25:04.299 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 msgr2=0x7f671c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 0x7f671c06d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6730034000 tx=0x7f6730009030 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 msgr2=0x7f67381edfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f67380c7830 tx=0x7f6734004840 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 shutdown_connections 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f671c06b500 0x7f671c06d9c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67380abb80 0x7f67381edfa0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f673806f5a0 0x7f67380ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 --2- 172.21.15.17:0/3789728846 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67240039b0 0x7f67380acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 >> 172.21.15.17:0/3789728846 conn(0x7f67380aa630 msgr2=0x7f67380c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 shutdown_connections 2024-03-06T07:25:04.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.289+0000 7f673f8a4640 1 -- 172.21.15.17:0/3789728846 wait complete. 2024-03-06T07:25:04.302 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:04.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.401+0000 7efde5dac640 1 Processor -- start 2024-03-06T07:25:04.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.401+0000 7efde5dac640 1 -- start start 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efde5dac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efde5dac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efde00acc00 0x7efde00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efde5dac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde0070aa0 con 0x7efde0156ea0 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efde5dac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde0157320 con 0x7efde00acc00 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efde5dac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde0156450 con 0x7efde01506b0 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efde00acc00 0x7efde00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddeffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efde0156ea0 0x7efde00702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41114/0 (socket says 172.21.15.17:41114) 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddeffd640 1 -- 172.21.15.17:0/751127182 learned_addr learned my addr 172.21.15.17:0/751127182 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddf7fe640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddffff640 1 -- 172.21.15.17:0/751127182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 msgr2=0x7efde0153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddffff640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde0153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddffff640 1 -- 172.21.15.17:0/751127182 >> v1:172.21.15.17:6789/0 conn(0x7efde0156ea0 legacy=0x7efde00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efddffff640 1 -- 172.21.15.17:0/751127182 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0070d80 con 0x7efde00acc00 2024-03-06T07:25:04.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.402+0000 7efdde7fc640 1 -- 172.21.15.17:0/751127182 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991741901 0 0) 0x7efde0070aa0 con 0x7efde0156ea0 2024-03-06T07:25:04.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.403+0000 7efddffff640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde00acc00 0x7efde00c54a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c470af5672519b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.403+0000 7efdde7fc640 1 -- 172.21.15.17:0/751127182 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8039070 con 0x7efde00acc00 2024-03-06T07:25:04.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.403+0000 7efdde7fc640 1 -- 172.21.15.17:0/751127182 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd802cc50 con 0x7efde00acc00 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.403+0000 7efdde7fc640 1 -- 172.21.15.17:0/751127182 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8034440 con 0x7efde00acc00 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 -- 172.21.15.17:0/751127182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde00acc00 msgr2=0x7efde00c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde00acc00 0x7efde00c54a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd802bea0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 -- 172.21.15.17:0/751127182 shutdown_connections 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde00acc00 0x7efde00c54a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 --2- 172.21.15.17:0/751127182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 -- 172.21.15.17:0/751127182 >> 172.21.15.17:0/751127182 conn(0x7efde006b1a0 msgr2=0x7efde006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 -- 172.21.15.17:0/751127182 shutdown_connections 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.404+0000 7efde5dac640 1 -- 172.21.15.17:0/751127182 wait complete. 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.405+0000 7efde5dac640 1 Processor -- start 2024-03-06T07:25:04.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.405+0000 7efde5dac640 1 -- start start 2024-03-06T07:25:04.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 0x7efde01e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde00aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00ad090 con 0x7efde0156ea0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde0156a10 con 0x7efde00acc00 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efde5dac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde00ad210 con 0x7efde01506b0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33656/0 (socket says 172.21.15.17:33656) 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 -- 172.21.15.17:0/1832548128 learned_addr learned my addr 172.21.15.17:0/1832548128 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddf7fe640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 0x7efde01e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddeffd640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde00aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 -- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 msgr2=0x7efde00aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde00aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 -- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 msgr2=0x7efde01e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 0x7efde01e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 -- 172.21.15.17:0/1832548128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00a89f0 con 0x7efde0156ea0 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddeffd640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde00aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddf7fe640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 0x7efde01e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:04.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efddffff640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.406+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8039050 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.407+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd8045070 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.407+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde00a8cd0 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.407+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8009450 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.407+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde00ab280 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.408+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde0069460 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.408+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdd803e610 con 0x7efde0156ea0 2024-03-06T07:25:04.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.408+0000 7efdc7fff640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 0x7efdcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.409+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efdd80aca60 con 0x7efde0156ea0 2024-03-06T07:25:04.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.409+0000 7efddf7fe640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 0x7efdcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.409+0000 7efddf7fe640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 0x7efdcc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.413+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efde0069460 con 0x7efde0156ea0 2024-03-06T07:25:04.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.596+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efde004f330 con 0x7efde0156ea0 2024-03-06T07:25:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.597+0000 7efdc7fff640 1 -- 172.21.15.17:0/1832548128 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+13174 (secure 0 0 0) 0x7efdd807bb30 con 0x7efde0156ea0 2024-03-06T07:25:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:04.599 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":132,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:03.898451+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 msgr2=0x7efdcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 0x7efdcc06d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).stop 2024-03-06T07:25:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 msgr2=0x7efde014b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd8004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 shutdown_connections 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efdcc06b4d0 0x7efdcc06d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde0156ea0 0x7efde014b4d0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efde01506b0 0x7efde00aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 --2- 172.21.15.17:0/1832548128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efde00acc00 0x7efde01e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 >> 172.21.15.17:0/1832548128 conn(0x7efde006b1a0 msgr2=0x7efde014f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 shutdown_connections 2024-03-06T07:25:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.599+0000 7efde5dac640 1 -- 172.21.15.17:0/1832548128 wait complete. 2024-03-06T07:25:04.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:04.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:04.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 Processor -- start 2024-03-06T07:25:04.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 -- start start 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3bf80c67f0 0x7f3bf8155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3bf8155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf8152cb0 con 0x7f3bf81528d0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf8155ca0 con 0x7f3bf80c67f0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfeb83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80c38b0 con 0x7f3bf80c4ef0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfc91f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3bf81528d0 0x7f3bf80724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41120/0 (socket says 172.21.15.17:41120) 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.715+0000 7f3bfc91f640 1 -- 172.21.15.17:0/161327911 learned_addr learned my addr 172.21.15.17:0/161327911 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/161327911 >> v2:172.21.15.163:3300/0 conn(0x7f3bf80c67f0 0x7f3bf8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bfd120640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3bf8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829535461 0 0) 0x7f3bf8152cb0 con 0x7f3bf81528d0 2024-03-06T07:25:04.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be8003520 con 0x7f3bf81528d0 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030154163 0 0) 0x7f3be8003520 con 0x7f3bf81528d0 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 msgr2=0x7f3bf8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3bf8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 >> v2:172.21.15.163:3300/0 conn(0x7f3bf80c67f0 msgr2=0x7f3bf8155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 --2- 172.21.15.17:0/161327911 >> v2:172.21.15.163:3300/0 conn(0x7f3bf80c67f0 0x7f3bf8155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf81594f0 con 0x7f3bf81528d0 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3bec001670 con 0x7f3bf81528d0 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 >> v1:172.21.15.17:6789/0 conn(0x7f3bf81528d0 legacy=0x7f3bf80724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3be80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be8003f30 0x7f3be8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8004d40 0x7f3be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf8155ca0 con 0x7f3be8003f30 2024-03-06T07:25:04.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80c38b0 con 0x7f3be8004d40 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be800b9f0 con 0x7f3bf80c4ef0 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be8003f30 0x7f3be8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bfc91f640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3be80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bfd120640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8004d40 0x7f3be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 -- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 msgr2=0x7f3be80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf80c4ef0 0x7f3be80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 -- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8004d40 msgr2=0x7f3be80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8004d40 0x7f3be80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.716+0000 7f3bf7fff640 1 -- 172.21.15.17:0/161327911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8003520 con 0x7f3be8003f30 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.717+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be8003f30 0x7f3be8004800 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009ac0 tx=0x7f3be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e4c48612b412f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.717+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4039070 con 0x7f3be8003f30 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.717+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be402c710 con 0x7f3be8003f30 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.717+0000 7f3bf77fe640 1 -- 172.21.15.17:0/161327911 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4009690 con 0x7f3be8003f30 2024-03-06T07:25:04.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 -- 172.21.15.17:0/161327911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be8003f30 msgr2=0x7f3be8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/161327911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be8003f30 0x7f3be8004800 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3be4009ac0 tx=0x7f3be8005830 comp rx=0 tx=0).stop 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfd120640 1 -- 172.21.15.17:0/161327911 reap_dead start 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 -- 172.21.15.17:0/161327911 shutdown_connections 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 -- 172.21.15.17:0/161327911 >> 172.21.15.17:0/161327911 conn(0x7f3bf80aa6a0 msgr2=0x7f3bf80aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 -- 172.21.15.17:0/161327911 shutdown_connections 2024-03-06T07:25:04.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.718+0000 7f3bfeb83640 1 -- 172.21.15.17:0/161327911 wait complete. 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.719+0000 7f3bfeb83640 1 Processor -- start 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.719+0000 7f3bfeb83640 1 -- start start 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 0x7f3bf81ea650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8158390 0x7f3bf8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80a8250 con 0x7f3bf81514e0 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf80c3390 con 0x7f3be8003f30 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfeb83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf81591a0 con 0x7f3bf8158390 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33676/0 (socket says 172.21.15.17:33676) 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 -- 172.21.15.17:0/1367033795 learned_addr learned my addr 172.21.15.17:0/1367033795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfc91f640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 0x7f3bf81ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfd120640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8158390 0x7f3bf8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 -- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8158390 msgr2=0x7f3bf8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8158390 0x7f3bf8158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 -- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 msgr2=0x7f3bf81ea650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 0x7f3bf81ea650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bf7fff640 1 -- 172.21.15.17:0/1367033795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf81eac30 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.720+0000 7f3bfc91f640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 0x7f3bf81ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bf7fff640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f3be4000c00 tx=0x7f3be4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4039050 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4045070 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf81eaf10 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40044e0 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.721+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf81eb4d0 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.722+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf804f330 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.722+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3be4003a50 con 0x7f3bf81514e0 2024-03-06T07:25:04.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.723+0000 7f3bf57fa640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 0x7f3bdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:04.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.723+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3be40abb50 con 0x7f3bf81514e0 2024-03-06T07:25:04.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.723+0000 7f3bfc91f640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 0x7f3bdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:04.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.723+0000 7f3bfc91f640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 0x7f3bdc06d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3bec000f60 tx=0x7f3bec000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:04.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.727+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3be404e020 con 0x7f3bf81514e0 2024-03-06T07:25:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.908+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bf8073100 con 0x7f3bf81514e0 2024-03-06T07:25:04.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.908+0000 7f3bf57fa640 1 -- 172.21.15.17:0/1367033795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3be407ad40 con 0x7f3bf81514e0 2024-03-06T07:25:04.910 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:25:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 msgr2=0x7f3bdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 0x7f3bdc06d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3bec000f60 tx=0x7f3bec000ed0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 msgr2=0x7f3bf8152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f3be4000c00 tx=0x7f3be4004730 comp rx=0 tx=0).stop 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 shutdown_connections 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bdc06b500 0x7f3bdc06d9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bf8158390 0x7f3bf8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bf81514e0 0x7f3bf8152d40 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 --2- 172.21.15.17:0/1367033795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be8003f30 0x7f3bf81ea650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 >> 172.21.15.17:0/1367033795 conn(0x7f3bf80aa6a0 msgr2=0x7f3bf8155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 shutdown_connections 2024-03-06T07:25:04.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:04.910+0000 7f3bfeb83640 1 -- 172.21.15.17:0/1367033795 wait complete. 2024-03-06T07:25:04.923 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:25:05.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.022+0000 7f705673e640 1 Processor -- start 2024-03-06T07:25:05.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.022+0000 7f705673e640 1 -- start start 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f705673e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f705006d9f0 0x7f70500a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f705673e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70500c3710 0x7f70501558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f705673e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7050158800 con 0x7f70500c3280 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f705673e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7050158ae0 con 0x7f705006d9f0 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f705673e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7050158dc0 con 0x7f70500c3710 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f705006d9f0 0x7f70500a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f705006d9f0 0x7f70500a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33688/0 (socket says 172.21.15.17:33688) 2024-03-06T07:25:05.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 -- 172.21.15.17:0/4113605994 learned_addr learned my addr 172.21.15.17:0/4113605994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f7054cdb640 1 --2- 172.21.15.17:0/4113605994 >> v2:172.21.15.163:3300/0 conn(0x7f70500c3710 0x7f70501558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 -- 172.21.15.17:0/4113605994 >> v2:172.21.15.163:3300/0 conn(0x7f70500c3710 msgr2=0x7f70501558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 --2- 172.21.15.17:0/4113605994 >> v2:172.21.15.163:3300/0 conn(0x7f70500c3710 0x7f70501558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 -- 172.21.15.17:0/4113605994 >> v1:172.21.15.163:6789/0 conn(0x7f70500c3280 legacy=0x7f70501551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.023+0000 7f704ffff640 1 -- 172.21.15.17:0/4113605994 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70501590a0 con 0x7f705006d9f0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.024+0000 7f704ffff640 1 --2- 172.21.15.17:0/4113605994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705006d9f0 0x7f70500a7cd0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7050159760 tx=0x7f703802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83991ee7e515b7f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.024+0000 7f704effd640 1 -- 172.21.15.17:0/4113605994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703802e070 con 0x7f705006d9f0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.024+0000 7f704effd640 1 -- 172.21.15.17:0/4113605994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038037070 con 0x7f705006d9f0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.024+0000 7f704effd640 1 -- 172.21.15.17:0/4113605994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038031a00 con 0x7f705006d9f0 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 -- 172.21.15.17:0/4113605994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705006d9f0 msgr2=0x7f70500a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 --2- 172.21.15.17:0/4113605994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705006d9f0 0x7f70500a7cd0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f7050159760 tx=0x7f703802b880 comp rx=0 tx=0).stop 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 -- 172.21.15.17:0/4113605994 shutdown_connections 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 --2- 172.21.15.17:0/4113605994 >> v2:172.21.15.163:3300/0 conn(0x7f70500c3710 0x7f70501558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 --2- 172.21.15.17:0/4113605994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705006d9f0 0x7f70500a7cd0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 -- 172.21.15.17:0/4113605994 >> 172.21.15.17:0/4113605994 conn(0x7f70500aa630 msgr2=0x7f70500aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 -- 172.21.15.17:0/4113605994 shutdown_connections 2024-03-06T07:25:05.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.025+0000 7f705673e640 1 -- 172.21.15.17:0/4113605994 wait complete. 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.026+0000 7f705673e640 1 Processor -- start 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.026+0000 7f705673e640 1 -- start start 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 0x7f70501eba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70500c3280 0x7f705006c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705006ddd0 con 0x7f70500c3710 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050155e30 con 0x7f705006d9f0 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f705673e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705006df50 con 0x7f70500c3280 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33692/0 (socket says 172.21.15.17:33692) 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 -- 172.21.15.17:0/2460393688 learned_addr learned my addr 172.21.15.17:0/2460393688 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f704ffff640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 0x7f70501eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f704f7fe640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70500c3280 0x7f705006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 -- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70500c3280 msgr2=0x7f705006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70500c3280 0x7f705006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 -- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 msgr2=0x7f70501eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 0x7f70501eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 -- 172.21.15.17:0/2460393688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70501e21a0 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f704ffff640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 0x7f70501eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.027+0000 7f7054cdb640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7048007cf0 tx=0x7f704800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.028+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048014420 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.028+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048019070 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.028+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70501e2480 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.028+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048014820 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.028+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501eb260 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.029+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705004f330 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.029+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70480149c0 con 0x7f70500c3710 2024-03-06T07:25:05.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.029+0000 7f704cff9640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 0x7f703006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.029+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f704808a800 con 0x7f70500c3710 2024-03-06T07:25:05.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.030+0000 7f704ffff640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 0x7f703006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.030+0000 7f704ffff640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 0x7f703006d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f703803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.034+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70480598d0 con 0x7f70500c3710 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.217+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7050070500 con 0x7f70500c3710 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.218+0000 7f704cff9640 1 -- 172.21.15.17:0/2460393688 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) v1 ==== 84+0+65 (secure 0 0 0) 0x7f70480591d0 con 0x7f70500c3710 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:05.219 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:25:05.220 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:25:05.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:05.220 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 msgr2=0x7f703006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 0x7f703006d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f703803d040 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 msgr2=0x7f70501ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7048007cf0 tx=0x7f704800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 shutdown_connections 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703006b4d0 0x7f703006d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70500c3710 0x7f70501ee3e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70500c3280 0x7f705006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 --2- 172.21.15.17:0/2460393688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f705006d9f0 0x7f70501eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 >> 172.21.15.17:0/2460393688 conn(0x7f70500aa630 msgr2=0x7f7050153380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 shutdown_connections 2024-03-06T07:25:05.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.220+0000 7f705673e640 1 -- 172.21.15.17:0/2460393688 wait complete. 2024-03-06T07:25:05.232 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:25:05.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:25:05.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.337+0000 7fac0b9bc640 1 Processor -- start 2024-03-06T07:25:05.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.337+0000 7fac0b9bc640 1 -- start start 2024-03-06T07:25:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac0b9bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fac040c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac0b9bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0414ef90 con 0x7fac040a9980 2024-03-06T07:25:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac0b9bc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac04155780 con 0x7fac0414ebb0 2024-03-06T07:25:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac0b9bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac04155530 con 0x7fac041521b0 2024-03-06T07:25:05.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac08f57640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac040a9980 0x7fac0406dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41136/0 (socket says 172.21.15.17:41136) 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac08f57640 1 -- 172.21.15.17:0/1554351323 learned_addr learned my addr 172.21.15.17:0/1554351323 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243000199 0 0) 0x7fabf4001110 con 0x7fac040a9980 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf0003520 con 0x7fac040a9980 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac09f59640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fac040c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636471229 0 0) 0x7fabf0003520 con 0x7fac040a9980 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 msgr2=0x7fac040c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fac040c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 >> v1:172.21.15.163:6789/0 conn(0x7fac0414ebb0 legacy=0x7fac040a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac041516d0 con 0x7fac040a9980 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fabf4001840 con 0x7fac040a9980 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 >> v1:172.21.15.17:6789/0 conn(0x7fac040a9980 legacy=0x7fac0406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf00039e0 0x7fabf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fabf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04155530 con 0x7fabf0004d40 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf000bbb0 con 0x7fabf00039e0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf000bdb0 con 0x7fac041521b0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.338+0000 7fac09f59640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac08f57640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fabf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09758640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf00039e0 0x7fabf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 -- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 msgr2=0x7fabf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac041521b0 0x7fabf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 -- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf00039e0 msgr2=0x7fabf0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf00039e0 0x7fabf0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 -- 172.21.15.17:0/1554351323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf0003520 con 0x7fabf0004d40 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.339+0000 7fac09f59640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00a250 tx=0x7fabf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d95d7c2dabae005 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc032070 con 0x7fabf0004d40 2024-03-06T07:25:05.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc02cd80 con 0x7fabf0004d40 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fabfbfff640 1 -- 172.21.15.17:0/1554351323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00adf0 con 0x7fabf0004d40 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 -- 172.21.15.17:0/1554351323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0004d40 msgr2=0x7fabf00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/1554351323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf0004d40 0x7fabf00051c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00a250 tx=0x7fabf0005830 comp rx=0 tx=0).stop 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac09f59640 1 -- 172.21.15.17:0/1554351323 reap_dead start 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 -- 172.21.15.17:0/1554351323 shutdown_connections 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 -- 172.21.15.17:0/1554351323 >> 172.21.15.17:0/1554351323 conn(0x7fac040c65b0 msgr2=0x7fac040c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 -- 172.21.15.17:0/1554351323 shutdown_connections 2024-03-06T07:25:05.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.340+0000 7fac0b9bc640 1 -- 172.21.15.17:0/1554351323 wait complete. 2024-03-06T07:25:05.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.341+0000 7fac0b9bc640 1 Processor -- start 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.341+0000 7fac0b9bc640 1 -- start start 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 0x7fac041ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac040bfbd0 0x7fac04143690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04151c90 con 0x7fac040728b0 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04155a60 con 0x7fabf0004d40 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac0b9bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac040aa6c0 con 0x7fac040bfbd0 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33708/0 (socket says 172.21.15.17:33708) 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 -- 172.21.15.17:0/3668933985 learned_addr learned my addr 172.21.15.17:0/3668933985 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 -- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac040bfbd0 msgr2=0x7fac04143690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac09f59640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac040bfbd0 0x7fac04143690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac09758640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 0x7fac041ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac040bfbd0 0x7fac04143690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 -- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 msgr2=0x7fac041ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 0x7fac041ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac08f57640 1 -- 172.21.15.17:0/3668933985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040c2c40 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.342+0000 7fac09758640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 0x7fac041ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fac08f57640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fabf4007300 tx=0x7fabf40077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf400e6e0 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf4018070 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac040c2f20 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf400eae0 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.343+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040720b0 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.344+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040708c0 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.344+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabf400ec80 con 0x7fac040728b0 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.345+0000 7fabf9ffb640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 0x7fabe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:05.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.345+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fabf4089f60 con 0x7fac040728b0 2024-03-06T07:25:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.345+0000 7fac09758640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 0x7fabe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.345+0000 7fac09758640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 0x7fabe806d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fac040c75e0 tx=0x7fac00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:05.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.349+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fac040708c0 con 0x7fac040728b0 2024-03-06T07:25:05.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.534+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fac0404f330 con 0x7fac040728b0 2024-03-06T07:25:05.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.906+0000 7fabf9ffb640 1 -- 172.21.15.17:0/3668933985 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) v1 ==== 138+0+0 (secure 0 0 0) 0x7fac0414ef90 con 0x7fac040728b0 2024-03-06T07:25:05.908 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 msgr2=0x7fabe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 0x7fabe806d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fac040c75e0 tx=0x7fac00009040 comp rx=0 tx=0).stop 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 msgr2=0x7fac040c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fabf4007300 tx=0x7fabf40077c0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 shutdown_connections 2024-03-06T07:25:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabe806b4d0 0x7fabe806d990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac040bfbd0 0x7fac04143690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac040728b0 0x7fac040c3750 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 --2- 172.21.15.17:0/3668933985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf0004d40 0x7fac041ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.909+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 >> 172.21.15.17:0/3668933985 conn(0x7fac040c65b0 msgr2=0x7fac0414a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.910+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 shutdown_connections 2024-03-06T07:25:05.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:05.910+0000 7fac0b9bc640 1 -- 172.21.15.17:0/3668933985 wait complete. 2024-03-06T07:25:05.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:25:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 Processor -- start 2024-03-06T07:25:06.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 -- start start 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcab40ab170 0x7fcab40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40c3ef0 con 0x7fcab40c5990 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab41556d0 con 0x7fcab406c900 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.037+0000 7fcab986f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab41559b0 con 0x7fcab40ab170 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcab40c5990 0x7fcab406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41140/0 (socket says 172.21.15.17:41140) 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab2ffd640 1 -- 172.21.15.17:0/197851678 learned_addr learned my addr 172.21.15.17:0/197851678 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab37fe640 1 --2- 172.21.15.17:0/197851678 >> v2:172.21.15.163:3300/0 conn(0x7fcab40ab170 0x7fcab40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 708562158 0 0) 0x7fcab40c3ef0 con 0x7fcab40c5990 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa0003520 con 0x7fcab40c5990 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 427779058 0 0) 0x7fcaa0003520 con 0x7fcab40c5990 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 >> v2:172.21.15.163:3300/0 conn(0x7fcab40ab170 msgr2=0x7fcab40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 --2- 172.21.15.17:0/197851678 >> v2:172.21.15.163:3300/0 conn(0x7fcab40ab170 0x7fcab40c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 >> v1:172.21.15.163:6789/0 conn(0x7fcab406c900 legacy=0x7fcab40a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4155c90 con 0x7fcab40c5990 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fcaa4001670 con 0x7fcab40c5990 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 >> v1:172.21.15.17:6789/0 conn(0x7fcab40c5990 legacy=0x7fcab406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00039e0 0x7fcaa0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 0x7fcaa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa00051c0 0x7fcaa000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41559b0 con 0x7fcaa0004800 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa000c090 con 0x7fcaa00039e0 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa000c290 con 0x7fcaa00051c0 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcaabfff640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 0x7fcaa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.038+0000 7fcab2ffd640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00039e0 0x7fcaa0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 -- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa00051c0 msgr2=0x7fcaa000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaa00051c0 0x7fcaa000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 -- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00039e0 msgr2=0x7fcaa0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00039e0 0x7fcaa0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 -- 172.21.15.17:0/197851678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0003520 con 0x7fcaa0004800 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcab37fe640 1 -- 172.21.15.17:0/197851678 reap_dead start 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcab2ffd640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa00039e0 0x7fcaa0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcaabfff640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 0x7fcaa0004c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001750 tx=0x7fcaa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d96baa2cf1514c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c001370 con 0x7fcaa0004800 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c0052d0 con 0x7fcaa0004800 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.039+0000 7fcab27fc640 1 -- 172.21.15.17:0/197851678 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c003830 con 0x7fcaa0004800 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 -- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 msgr2=0x7fcaa0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 0x7fcaa0004c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001750 tx=0x7fcaa0005830 comp rx=0 tx=0).stop 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 -- 172.21.15.17:0/197851678 shutdown_connections 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 --2- 172.21.15.17:0/197851678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaa0004800 0x7fcaa0004c80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 -- 172.21.15.17:0/197851678 >> 172.21.15.17:0/197851678 conn(0x7fcab40c6710 msgr2=0x7fcab40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 -- 172.21.15.17:0/197851678 shutdown_connections 2024-03-06T07:25:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.040+0000 7fcab986f640 1 -- 172.21.15.17:0/197851678 wait complete. 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.041+0000 7fcab986f640 1 Processor -- start 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.041+0000 7fcab986f640 1 -- start start 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa0004800 0x7fcab4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcab4154ad0 0x7fcab41489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4156610 con 0x7fcab40c0bd0 2024-03-06T07:25:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4156490 con 0x7fcaa0004800 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40c3af0 con 0x7fcab4154ad0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33720/0 (socket says 172.21.15.17:33720) 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 -- 172.21.15.17:0/218120065 learned_addr learned my addr 172.21.15.17:0/218120065 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab37fe640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcab4154ad0 0x7fcab41489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab2ffd640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa0004800 0x7fcab4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 -- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcab4154ad0 msgr2=0x7fcab41489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcab4154ad0 0x7fcab41489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 -- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa0004800 msgr2=0x7fcab4154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa0004800 0x7fcab4154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 -- 172.21.15.17:0/218120065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4155480 con 0x7fcab40c0bd0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcaabfff640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fca9c034040 tx=0x7fca9c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c041070 con 0x7fcab40c0bd0 2024-03-06T07:25:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c005480 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab414df20 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.042+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c007a30 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.043+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab414e1c0 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.044+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab404f330 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.044+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca9c044070 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.045+0000 7fcab0ff9640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.045+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fca9c0abd10 con 0x7fcab40c0bd0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.045+0000 7fcab2ffd640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.045+0000 7fcab2ffd640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 0x7fca9406d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4001eb0 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:06.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.049+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fca9c07ab50 con 0x7fcab40c0bd0 2024-03-06T07:25:06.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.231+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcab40c6d20 con 0x7fcab40c0bd0 2024-03-06T07:25:06.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.912+0000 7fcab0ff9640 1 -- 172.21.15.17:0/218120065 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) v1 ==== 170+0+0 (secure 0 0 0) 0x7fca9c07a450 con 0x7fcab40c0bd0 2024-03-06T07:25:06.914 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:25:06.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.916+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 msgr2=0x7fca9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.916+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 0x7fca9406d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4001eb0 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2024-03-06T07:25:06.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.916+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 msgr2=0x7fcab4148490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:06.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.916+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fca9c034040 tx=0x7fca9c001120 comp rx=0 tx=0).stop 2024-03-06T07:25:06.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 shutdown_connections 2024-03-06T07:25:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca9406b4d0 0x7fca9406d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcab4154ad0 0x7fcab41489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcab40c0bd0 0x7fcab4148490 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 --2- 172.21.15.17:0/218120065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaa0004800 0x7fcab4154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:06.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.919+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 >> 172.21.15.17:0/218120065 conn(0x7fcab40c6710 msgr2=0x7fcab406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:06.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.924+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 shutdown_connections 2024-03-06T07:25:06.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:06.925+0000 7fcab986f640 1 -- 172.21.15.17:0/218120065 wait complete. 2024-03-06T07:25:06.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:25:07.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.052+0000 7fab7117a640 1 Processor -- start 2024-03-06T07:25:07.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 -- start start 2024-03-06T07:25:07.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab6c1527f0 0x7fab6c0c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab6c0c7370 0x7fab6c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c155260 con 0x7fab6c06c0b0 2024-03-06T07:25:07.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c155540 con 0x7fab6c1527f0 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab7117a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c155820 con 0x7fab6c0c7370 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab6ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab6c1527f0 0x7fab6c0c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab6ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab6c1527f0 0x7fab6c0c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33724/0 (socket says 172.21.15.17:33724) 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.053+0000 7fab6ad9d640 1 -- 172.21.15.17:0/3265751984 learned_addr learned my addr 172.21.15.17:0/3265751984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6ad9d640 1 -- 172.21.15.17:0/3265751984 >> v2:172.21.15.163:3300/0 conn(0x7fab6c0c7370 msgr2=0x7fab6c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6ad9d640 1 --2- 172.21.15.17:0/3265751984 >> v2:172.21.15.163:3300/0 conn(0x7fab6c0c7370 0x7fab6c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6ad9d640 1 -- 172.21.15.17:0/3265751984 >> v1:172.21.15.163:6789/0 conn(0x7fab6c06c0b0 legacy=0x7fab6c06c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:07.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6ad9d640 1 -- 172.21.15.17:0/3265751984 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c155b00 con 0x7fab6c1527f0 2024-03-06T07:25:07.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6ad9d640 1 --2- 172.21.15.17:0/3265751984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c0c8860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06d590 tx=0x7fab5c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97384e608664700d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.054+0000 7fab6a59c640 1 -- 172.21.15.17:0/3265751984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c038070 con 0x7fab6c1527f0 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab6a59c640 1 -- 172.21.15.17:0/3265751984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c02f050 con 0x7fab6c1527f0 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab6a59c640 1 -- 172.21.15.17:0/3265751984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031970 con 0x7fab6c1527f0 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 -- 172.21.15.17:0/3265751984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 msgr2=0x7fab6c0c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 --2- 172.21.15.17:0/3265751984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c0c8860 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06d590 tx=0x7fab5c02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 -- 172.21.15.17:0/3265751984 shutdown_connections 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 --2- 172.21.15.17:0/3265751984 >> v2:172.21.15.163:3300/0 conn(0x7fab6c0c7370 0x7fab6c0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 --2- 172.21.15.17:0/3265751984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c0c8860 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 -- 172.21.15.17:0/3265751984 >> 172.21.15.17:0/3265751984 conn(0x7fab6c0ac4f0 msgr2=0x7fab6c0ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 -- 172.21.15.17:0/3265751984 shutdown_connections 2024-03-06T07:25:07.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.055+0000 7fab7117a640 1 -- 172.21.15.17:0/3265751984 wait complete. 2024-03-06T07:25:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.056+0000 7fab7117a640 1 Processor -- start 2024-03-06T07:25:07.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.056+0000 7fab7117a640 1 -- start start 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab6c06c0b0 0x7fab6c0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab6c0c7370 0x7fab6c1549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8f20 con 0x7fab6c1527f0 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0c8da0 con 0x7fab6c0c7370 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab7117a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c06cbd0 con 0x7fab6c06c0b0 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab6b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab6b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33734/0 (socket says 172.21.15.17:33734) 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab6b59e640 1 -- 172.21.15.17:0/3748022904 learned_addr learned my addr 172.21.15.17:0/3748022904 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab63fff640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab6c0c7370 0x7fab6c1549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.057+0000 7fab6ad9d640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab6c06c0b0 0x7fab6c0c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 -- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab6c06c0b0 msgr2=0x7fab6c0c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab6c06c0b0 0x7fab6c0c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 -- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab6c0c7370 msgr2=0x7fab6c1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab6c0c7370 0x7fab6c1549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 -- 172.21.15.17:0/3748022904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c0c25f0 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab6b59e640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b990 tx=0x7fab6400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64019070 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400cba0 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1eaf30 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64014440 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.058+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c1eb4f0 con 0x7fab6c1527f0 2024-03-06T07:25:07.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.059+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c04f330 con 0x7fab6c1527f0 2024-03-06T07:25:07.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.060+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab6401c070 con 0x7fab6c1527f0 2024-03-06T07:25:07.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.060+0000 7fab637fe640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:07.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.060+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fab64089fb0 con 0x7fab6c1527f0 2024-03-06T07:25:07.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.060+0000 7fab6ad9d640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:07.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.061+0000 7fab6ad9d640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 0x7fab4c06d9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fab5c035000 tx=0x7fab5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:07.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.065+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab640586c0 con 0x7fab6c1527f0 2024-03-06T07:25:07.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.284+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fab6c06d860 con 0x7fab6c1527f0 2024-03-06T07:25:07.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.923+0000 7fab637fe640 1 -- 172.21.15.17:0/3748022904 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v118) v1 ==== 174+0+0 (secure 0 0 0) 0x7fab6405c760 con 0x7fab6c1527f0 2024-03-06T07:25:07.925 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 21 and data pool 22 2024-03-06T07:25:07.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.928+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 msgr2=0x7fab4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.928+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 0x7fab4c06d9c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fab5c035000 tx=0x7fab5c008030 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.928+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 msgr2=0x7fab6c14b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.928+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b990 tx=0x7fab6400be50 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 shutdown_connections 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab6c1527f0 0x7fab6c14b960 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab6c0c7370 0x7fab6c1549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 --2- 172.21.15.17:0/3748022904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab6c06c0b0 0x7fab6c0c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.929+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 >> 172.21.15.17:0/3748022904 conn(0x7fab6c0ac4f0 msgr2=0x7fab6c0c51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.931+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 shutdown_connections 2024-03-06T07:25:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:07.933+0000 7fab7117a640 1 -- 172.21.15.17:0/3748022904 wait complete. 2024-03-06T07:25:07.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:08.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.061+0000 7f28f6f0d640 1 Processor -- start 2024-03-06T07:25:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.061+0000 7f28f6f0d640 1 -- start start 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f6f0d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28f00c5330 0x7f28f00c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f6f0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f0159630 con 0x7f28f00c8a00 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f6f0d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00c7760 con 0x7f28f0152920 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f6f0d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f01587b0 con 0x7f28f00c5330 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f54aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28f00c5330 0x7f28f00c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28effff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f28f00c8a00 0x7f28f0158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41156/0 (socket says 172.21.15.17:41156) 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28effff640 1 -- 172.21.15.17:0/2342852965 learned_addr learned my addr 172.21.15.17:0/2342852965 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f54aa640 1 -- 172.21.15.17:0/2342852965 >> v1:172.21.15.163:6789/0 conn(0x7f28f0152920 legacy=0x7f28f0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f54aa640 1 -- 172.21.15.17:0/2342852965 >> v1:172.21.15.17:6789/0 conn(0x7f28f00c8a00 legacy=0x7f28f0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28f54aa640 1 -- 172.21.15.17:0/2342852965 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00c7a40 con 0x7f28f00c5330 2024-03-06T07:25:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.062+0000 7f28ef7fe640 1 -- 172.21.15.17:0/2342852965 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867077201 0 0) 0x7f28f0159630 con 0x7f28f00c8a00 2024-03-06T07:25:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.063+0000 7f28f54aa640 1 --2- 172.21.15.17:0/2342852965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c5330 0x7f28f00c57a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009d90 tx=0x7f28e802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2f17d95cd1f57b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.063+0000 7f28ef7fe640 1 -- 172.21.15.17:0/2342852965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8039070 con 0x7f28f00c5330 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.063+0000 7f28ef7fe640 1 -- 172.21.15.17:0/2342852965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802cbd0 con 0x7f28f00c5330 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.063+0000 7f28ef7fe640 1 -- 172.21.15.17:0/2342852965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8034440 con 0x7f28f00c5330 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 -- 172.21.15.17:0/2342852965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c5330 msgr2=0x7f28f00c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/2342852965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c5330 0x7f28f00c57a0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009d90 tx=0x7f28e802be20 comp rx=0 tx=0).stop 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 -- 172.21.15.17:0/2342852965 shutdown_connections 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/2342852965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c5330 0x7f28f00c57a0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 -- 172.21.15.17:0/2342852965 >> 172.21.15.17:0/2342852965 conn(0x7f28f00ac560 msgr2=0x7f28f00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 -- 172.21.15.17:0/2342852965 shutdown_connections 2024-03-06T07:25:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.064+0000 7f28f6f0d640 1 -- 172.21.15.17:0/2342852965 wait complete. 2024-03-06T07:25:08.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 Processor -- start 2024-03-06T07:25:08.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 -- start start 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f00c5330 0x7f28f0147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 0x7f28f00bf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f006d030 con 0x7f28f00c8a00 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0158a00 con 0x7f28f0152920 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.065+0000 7f28f6f0d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c7390 con 0x7f28f00c5330 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33756/0 (socket says 172.21.15.17:33756) 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 -- 172.21.15.17:0/4119648202 learned_addr learned my addr 172.21.15.17:0/4119648202 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28f54aa640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 0x7f28f00bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28f4ca9640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f00c5330 0x7f28f0147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 -- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f00c5330 msgr2=0x7f28f0147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f00c5330 0x7f28f0147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 -- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 msgr2=0x7f28f00bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 0x7f28f00bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 -- 172.21.15.17:0/4119648202 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f014df90 con 0x7f28f00c8a00 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28f54aa640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 0x7f28f00bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28effff640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c460 tx=0x7f28dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc018070 con 0x7f28f00c8a00 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc007500 con 0x7f28f00c8a00 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f01ef2f0 con 0x7f28f00c8a00 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc004a20 con 0x7f28f00c8a00 2024-03-06T07:25:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.066+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01eb2d0 con 0x7f28f00c8a00 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.067+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f004f330 con 0x7f28f00c8a00 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.068+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28dc01b070 con 0x7f28f00c8a00 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.068+0000 7f28ed7fa640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.068+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28dc0895c0 con 0x7f28f00c8a00 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.069+0000 7f28f4ca9640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.069+0000 7f28f4ca9640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 0x7f28d406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f28e4004ea0 tx=0x7f28e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.073+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f28f014df90 con 0x7f28f00c8a00 2024-03-06T07:25:08.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.256+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28f00c6d10 con 0x7f28f00c8a00 2024-03-06T07:25:08.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.257+0000 7f28ed7fa640 1 -- 172.21.15.17:0/4119648202 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16905 (secure 0 0 0) 0x7f28dc058340 con 0x7f28f00c8a00 2024-03-06T07:25:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:08.259 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":135,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:07.285640+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 msgr2=0x7f28d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 0x7f28d406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f28e4004ea0 tx=0x7f28e4009040 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 msgr2=0x7f28f0149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00c460 tx=0x7f28dc00c920 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 shutdown_connections 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f28d406b4d0 0x7f28d406d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28f0152920 0x7f28f00bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f00c8a00 0x7f28f0149610 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 --2- 172.21.15.17:0/4119648202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28f00c5330 0x7f28f0147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 >> 172.21.15.17:0/4119648202 conn(0x7f28f00ac560 msgr2=0x7f28f00ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 shutdown_connections 2024-03-06T07:25:08.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.259+0000 7f28f6f0d640 1 -- 172.21.15.17:0/4119648202 wait complete. 2024-03-06T07:25:08.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.371+0000 7fcbb875c640 1 Processor -- start 2024-03-06T07:25:08.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.371+0000 7fcbb875c640 1 -- start start 2024-03-06T07:25:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb875c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcbb00c5480 0x7fcbb0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb875c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbb00c5860 0x7fcbb006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb875c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb00c5cc0 con 0x7fcbb01586f0 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb875c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb006df40 con 0x7fcbb00c5860 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb875c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb01591c0 con 0x7fcbb00c5480 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb5cf7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbb00c5860 0x7fcbb006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb5cf7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcbb00c5860 0x7fcbb006f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33770/0 (socket says 172.21.15.17:33770) 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.372+0000 7fcbb5cf7640 1 -- 172.21.15.17:0/439098618 learned_addr learned my addr 172.21.15.17:0/439098618 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb5cf7640 1 -- 172.21.15.17:0/439098618 >> v2:172.21.15.163:3300/0 conn(0x7fcbb00c5480 msgr2=0x7fcbb0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb5cf7640 1 --2- 172.21.15.17:0/439098618 >> v2:172.21.15.163:3300/0 conn(0x7fcbb00c5480 0x7fcbb0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb5cf7640 1 -- 172.21.15.17:0/439098618 >> v1:172.21.15.163:6789/0 conn(0x7fcbb01586f0 legacy=0x7fcbb00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb5cf7640 1 -- 172.21.15.17:0/439098618 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb01594a0 con 0x7fcbb00c5860 2024-03-06T07:25:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb5cf7640 1 --2- 172.21.15.17:0/439098618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5860 0x7fcbb006f6a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00c1cf0 tx=0x7fcba402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4145885972e2ced server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.373+0000 7fcbb54f6640 1 -- 172.21.15.17:0/439098618 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba402e070 con 0x7fcbb00c5860 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb54f6640 1 -- 172.21.15.17:0/439098618 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba4037070 con 0x7fcbb00c5860 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb54f6640 1 -- 172.21.15.17:0/439098618 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4031a00 con 0x7fcbb00c5860 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 -- 172.21.15.17:0/439098618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5860 msgr2=0x7fcbb006f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 --2- 172.21.15.17:0/439098618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5860 0x7fcbb006f6a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00c1cf0 tx=0x7fcba402b880 comp rx=0 tx=0).stop 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 -- 172.21.15.17:0/439098618 shutdown_connections 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 --2- 172.21.15.17:0/439098618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5860 0x7fcbb006f6a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 --2- 172.21.15.17:0/439098618 >> v2:172.21.15.163:3300/0 conn(0x7fcbb00c5480 0x7fcbb0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 -- 172.21.15.17:0/439098618 >> 172.21.15.17:0/439098618 conn(0x7fcbb006b1a0 msgr2=0x7fcbb006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 -- 172.21.15.17:0/439098618 shutdown_connections 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.374+0000 7fcbb875c640 1 -- 172.21.15.17:0/439098618 wait complete. 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.375+0000 7fcbb875c640 1 Processor -- start 2024-03-06T07:25:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.375+0000 7fcbb875c640 1 -- start start 2024-03-06T07:25:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb00c5860 0x7fcbb01eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01586f0 0x7fcbb01eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0152a20 con 0x7fcbb00c5480 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0155680 con 0x7fcbb01586f0 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb875c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0155050 con 0x7fcbb00c5860 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33786/0 (socket says 172.21.15.17:33786) 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 -- 172.21.15.17:0/323860519 learned_addr learned my addr 172.21.15.17:0/323860519 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 -- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb00c5860 msgr2=0x7fcbb01eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb00c5860 0x7fcbb01eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb6cf9640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01586f0 0x7fcbb01eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 -- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01586f0 msgr2=0x7fcbb01eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01586f0 0x7fcbb01eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.376+0000 7fcbb64f8640 1 -- 172.21.15.17:0/323860519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb01ee8f0 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbb64f8640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fcba800b4a0 tx=0x7fcba800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8014070 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8010610 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8010a00 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb01eebd0 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.377+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb01ef190 con 0x7fcbb00c5480 2024-03-06T07:25:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.378+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb004f330 con 0x7fcbb00c5480 2024-03-06T07:25:08.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.378+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcba8017070 con 0x7fcbb00c5480 2024-03-06T07:25:08.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.379+0000 7fcbaeffd640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 0x7fcb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.379+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcba8089e20 con 0x7fcbb00c5480 2024-03-06T07:25:08.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.379+0000 7fcbb5cf7640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 0x7fcb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.380+0000 7fcbb5cf7640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 0x7fcb9c06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcba4036000 tx=0x7fcba4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.383+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcba8058b10 con 0x7fcbb00c5480 2024-03-06T07:25:08.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.563+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbb006e0c0 con 0x7fcbb00c5480 2024-03-06T07:25:08.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.565+0000 7fcbaeffd640 1 -- 172.21.15.17:0/323860519 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16905 (secure 0 0 0) 0x7fcba8058410 con 0x7fcbb00c5480 2024-03-06T07:25:08.566 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:08.566 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":135,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:07.285640+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:08.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 msgr2=0x7fcb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 0x7fcb9c06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcba4036000 tx=0x7fcba4030000 comp rx=0 tx=0).stop 2024-03-06T07:25:08.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 msgr2=0x7fcbb00c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fcba800b4a0 tx=0x7fcba800b960 comp rx=0 tx=0).stop 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 shutdown_connections 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb9c06b4d0 0x7fcb9c06d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.566+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbb01586f0 0x7fcbb01eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.567+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbb00c5860 0x7fcbb01eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.567+0000 7fcbb875c640 1 --2- 172.21.15.17:0/323860519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbb00c5480 0x7fcbb00c4780 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.567+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 >> 172.21.15.17:0/323860519 conn(0x7fcbb006b1a0 msgr2=0x7fcbb00c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.567+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 shutdown_connections 2024-03-06T07:25:08.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.567+0000 7fcbb875c640 1 -- 172.21.15.17:0/323860519 wait complete. 2024-03-06T07:25:08.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:25:08.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.681+0000 7fe8b2f5e640 1 Processor -- start 2024-03-06T07:25:08.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.681+0000 7fe8b2f5e640 1 -- start start 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b2f5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b2f5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ac0c4740 0x7fe8ac0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b2f5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac0c62a0 con 0x7fe8ac0c69a0 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b2f5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac153f00 con 0x7fe8ac0c4740 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b2f5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac0706e0 con 0x7fe8ac0a8290 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8abfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe8ac0c4740 0x7fe8ac0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b14fb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe8ac0c69a0 0x7fe8ac072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41164/0 (socket says 172.21.15.17:41164) 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8b14fb640 1 -- 172.21.15.17:0/1249625789 learned_addr learned my addr 172.21.15.17:0/1249625789 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.682+0000 7fe8abfff640 1 -- 172.21.15.17:0/1249625789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 msgr2=0x7fe8ac1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8b0cfa640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8abfff640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8abfff640 1 -- 172.21.15.17:0/1249625789 >> v1:172.21.15.17:6789/0 conn(0x7fe8ac0c69a0 legacy=0x7fe8ac072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8ab7fe640 1 -- 172.21.15.17:0/1249625789 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618909351 0 0) 0x7fe8ac0c62a0 con 0x7fe8ac0c69a0 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8abfff640 1 -- 172.21.15.17:0/1249625789 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0c4c00 con 0x7fe8ac0c4740 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8abfff640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac0701a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097a0 tx=0x7fe8a002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283de6834eda5771 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8ab7fe640 1 -- 172.21.15.17:0/1249625789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0038070 con 0x7fe8ac0c4740 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8ab7fe640 1 -- 172.21.15.17:0/1249625789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0031700 con 0x7fe8ac0c4740 2024-03-06T07:25:08.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.683+0000 7fe8ab7fe640 1 -- 172.21.15.17:0/1249625789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0031ad0 con 0x7fe8ac0c4740 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/1249625789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 msgr2=0x7fe8ac0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac0701a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097a0 tx=0x7fe8a002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/1249625789 shutdown_connections 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac0701a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/1249625789 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/1249625789 >> 172.21.15.17:0/1249625789 conn(0x7fe8ac0c5560 msgr2=0x7fe8ac0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/1249625789 shutdown_connections 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.684+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/1249625789 wait complete. 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.685+0000 7fe8b2f5e640 1 Processor -- start 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.685+0000 7fe8b2f5e640 1 -- start start 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 0x7fe8ac1dca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06fc80 con 0x7fe8ac0c4740 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06f9a0 con 0x7fe8ac0c69a0 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b2f5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac072d90 con 0x7fe8ac0a8290 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33818/0 (socket says 172.21.15.17:33818) 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 -- 172.21.15.17:0/2978421686 learned_addr learned my addr 172.21.15.17:0/2978421686 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b14fb640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 0x7fe8ac1dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b0cfa640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 -- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 msgr2=0x7fe8ac0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac0adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 -- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 msgr2=0x7fe8ac1dca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 0x7fe8ac1dca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8abfff640 1 -- 172.21.15.17:0/2978421686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0ae8a0 con 0x7fe8ac0c4740 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.686+0000 7fe8b14fb640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 0x7fe8ac1dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8abfff640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe8a002bd80 tx=0x7fe8a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0042070 con 0x7fe8ac0c4740 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0033a30 con 0x7fe8ac0c4740 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac0aeb80 con 0x7fe8ac0c4740 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0038040 con 0x7fe8ac0c4740 2024-03-06T07:25:08.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.687+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac0ae140 con 0x7fe8ac0c4740 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.688+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac04f330 con 0x7fe8ac0c4740 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.688+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a0045070 con 0x7fe8ac0c4740 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.689+0000 7fe8a97fa640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.689+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8a00ac0e0 con 0x7fe8ac0c4740 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.689+0000 7fe8b0cfa640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:08.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.689+0000 7fe8b0cfa640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 0x7fe89806d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac14e8c0 tx=0x7fe8a40079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:08.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.693+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8a0030070 con 0x7fe8ac0c4740 2024-03-06T07:25:08.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.915+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe8ac0c8480 con 0x7fe8ac0c4740 2024-03-06T07:25:08.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.931+0000 7fe8a97fa640 1 -- 172.21.15.17:0/2978421686 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v119) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe8a007af80 con 0x7fe8ac0c4740 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 msgr2=0x7fe89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 0x7fe89806d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac14e8c0 tx=0x7fe8a40079a0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 msgr2=0x7fe8ac1dc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe8a002bd80 tx=0x7fe8a00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 shutdown_connections 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe89806b4d0 0x7fe89806d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ac0c69a0 0x7fe8ac1dca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8ac0c4740 0x7fe8ac1dc4e0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 --2- 172.21.15.17:0/2978421686 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ac0a8290 0x7fe8ac0adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:08.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 >> 172.21.15.17:0/2978421686 conn(0x7fe8ac0c5560 msgr2=0x7fe8ac0c6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:08.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 shutdown_connections 2024-03-06T07:25:08.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:08.934+0000 7fe8b2f5e640 1 -- 172.21.15.17:0/2978421686 wait complete. 2024-03-06T07:25:08.947 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:25:08.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:25:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.049+0000 7f90d442a640 1 Processor -- start 2024-03-06T07:25:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.049+0000 7f90d442a640 1 -- start start 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d442a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc14bcc0 0x7f90cc155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d442a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc158a10 0x7f90cc158e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d442a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90cc0c75e0 con 0x7f90cc0be150 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d442a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90cc0c7f10 con 0x7f90cc158a10 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d442a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0c78c0 con 0x7f90cc14bcc0 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc158a10 0x7f90cc158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90cc158a10 0x7f90cc158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33834/0 (socket says 172.21.15.17:33834) 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 -- 172.21.15.17:0/446183468 learned_addr learned my addr 172.21.15.17:0/446183468 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d21c6640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc14bcc0 0x7f90cc155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 -- 172.21.15.17:0/446183468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc14bcc0 msgr2=0x7f90cc155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc14bcc0 0x7f90cc155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 -- 172.21.15.17:0/446183468 >> v1:172.21.15.163:6789/0 conn(0x7f90cc0be150 legacy=0x7f90cc0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.050+0000 7f90d29c7640 1 -- 172.21.15.17:0/446183468 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0c7ba0 con 0x7f90cc158a10 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.051+0000 7f90d29c7640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc158a10 0x7f90cc158e20 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009e50 tx=0x7f90bc02bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5184d93f8b93a512 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.051+0000 7f90d11c4640 1 -- 172.21.15.17:0/446183468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02cb90 con 0x7f90cc158a10 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.051+0000 7f90d11c4640 1 -- 172.21.15.17:0/446183468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc02cd30 con 0x7f90cc158a10 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.051+0000 7f90d11c4640 1 -- 172.21.15.17:0/446183468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc034610 con 0x7f90cc158a10 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 -- 172.21.15.17:0/446183468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc158a10 msgr2=0x7f90cc158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc158a10 0x7f90cc158e20 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009e50 tx=0x7f90bc02bff0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 -- 172.21.15.17:0/446183468 shutdown_connections 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc158a10 0x7f90cc158e20 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 --2- 172.21.15.17:0/446183468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc14bcc0 0x7f90cc155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 -- 172.21.15.17:0/446183468 >> 172.21.15.17:0/446183468 conn(0x7f90cc0ac600 msgr2=0x7f90cc0aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 -- 172.21.15.17:0/446183468 shutdown_connections 2024-03-06T07:25:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.052+0000 7f90d442a640 1 -- 172.21.15.17:0/446183468 wait complete. 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.053+0000 7f90d442a640 1 Processor -- start 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.053+0000 7f90d442a640 1 -- start start 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0be150 0x7f90cc0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc158a10 0x7f90cc0c0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0c5c60 con 0x7f90cc14bcc0 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc0c5ae0 con 0x7f90cc0be150 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc159360 con 0x7f90cc158a10 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33836/0 (socket says 172.21.15.17:33836) 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 -- 172.21.15.17:0/1636720702 learned_addr learned my addr 172.21.15.17:0/1636720702 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d21c6640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0be150 0x7f90cc0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d29c7640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc158a10 0x7f90cc0c0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 -- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc158a10 msgr2=0x7f90cc0c0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc158a10 0x7f90cc0c0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 -- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0be150 msgr2=0x7f90cc0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0be150 0x7f90cc0c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 -- 172.21.15.17:0/1636720702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc154e20 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d19c5640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f90c000c400 tx=0x7f90c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c00052d0 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0005490 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc1eb7d0 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0007260 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.054+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc1eba50 con 0x7f90cc14bcc0 2024-03-06T07:25:09.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.055+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc04f330 con 0x7f90cc14bcc0 2024-03-06T07:25:09.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.056+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c0007400 con 0x7f90cc14bcc0 2024-03-06T07:25:09.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.056+0000 7f90caffd640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:09.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.056+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90c00897d0 con 0x7f90cc14bcc0 2024-03-06T07:25:09.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.056+0000 7f90d21c6640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:09.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.057+0000 7f90d21c6640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 0x7f90b806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009b40 tx=0x7f90c4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:09.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.061+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90cc1eb7d0 con 0x7f90cc14bcc0 2024-03-06T07:25:09.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.278+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f90cc0c1580 con 0x7f90cc14bcc0 2024-03-06T07:25:09.346 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:09.349 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:09.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.935+0000 7f90caffd640 1 -- 172.21.15.17:0/1636720702 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v120) v1 ==== 122+0+0 (secure 0 0 0) 0x7f90cc154e20 con 0x7f90cc14bcc0 2024-03-06T07:25:09.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.937+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 msgr2=0x7f90b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.937+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 0x7f90b806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009b40 tx=0x7f90c4009190 comp rx=0 tx=0).stop 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.937+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 msgr2=0x7f90cc0bf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.937+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f90c000c400 tx=0x7f90c000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 shutdown_connections 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90cc158a10 0x7f90cc0c0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90cc14bcc0 0x7f90cc0bf160 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 --2- 172.21.15.17:0/1636720702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90cc0be150 0x7f90cc0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:09.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 >> 172.21.15.17:0/1636720702 conn(0x7f90cc0ac600 msgr2=0x7f90cc0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:09.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 shutdown_connections 2024-03-06T07:25:09.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:09.938+0000 7f90d442a640 1 -- 172.21.15.17:0/1636720702 wait complete. 2024-03-06T07:25:09.952 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:25:09.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:10.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.058+0000 7fe9060c2640 1 Processor -- start 2024-03-06T07:25:10.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.058+0000 7fe9060c2640 1 -- start start 2024-03-06T07:25:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe900155c80 0x7fe9000c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe9000720a0 0x7fe90006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000c1dc0 con 0x7fe900155c80 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000c1ae0 con 0x7fe9000720a0 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe9060c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90006e450 con 0x7fe900152520 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe900155c80 0x7fe9000c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe900155c80 0x7fe9000c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33844/0 (socket says 172.21.15.17:33844) 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 -- 172.21.15.17:0/1539572468 learned_addr learned my addr 172.21.15.17:0/1539572468 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8ff7fe640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8fffff640 1 --2- 172.21.15.17:0/1539572468 >> v2:172.21.15.163:3300/0 conn(0x7fe9000720a0 0x7fe90006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 -- 172.21.15.17:0/1539572468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 msgr2=0x7fe90006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:10.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 -- 172.21.15.17:0/1539572468 >> v2:172.21.15.163:3300/0 conn(0x7fe9000720a0 msgr2=0x7fe90006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 --2- 172.21.15.17:0/1539572468 >> v2:172.21.15.163:3300/0 conn(0x7fe9000720a0 0x7fe90006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.059+0000 7fe8feffd640 1 -- 172.21.15.17:0/1539572468 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90006e730 con 0x7fe900155c80 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.060+0000 7fe8feffd640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe900155c80 0x7fe9000c12b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe900067c80 tx=0x7fe8f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669112616258a47d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.060+0000 7fe8fe7fc640 1 -- 172.21.15.17:0/1539572468 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0035020 con 0x7fe900155c80 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.060+0000 7fe8fe7fc640 1 -- 172.21.15.17:0/1539572468 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f0031870 con 0x7fe900155c80 2024-03-06T07:25:10.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.060+0000 7fe8fe7fc640 1 -- 172.21.15.17:0/1539572468 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0031c40 con 0x7fe900155c80 2024-03-06T07:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 -- 172.21.15.17:0/1539572468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe900155c80 msgr2=0x7fe9000c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe900155c80 0x7fe9000c12b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe900067c80 tx=0x7fe8f002b910 comp rx=0 tx=0).stop 2024-03-06T07:25:10.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 -- 172.21.15.17:0/1539572468 shutdown_connections 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 --2- 172.21.15.17:0/1539572468 >> v2:172.21.15.163:3300/0 conn(0x7fe9000720a0 0x7fe90006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe900155c80 0x7fe9000c12b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 --2- 172.21.15.17:0/1539572468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 -- 172.21.15.17:0/1539572468 >> 172.21.15.17:0/1539572468 conn(0x7fe9000c76b0 msgr2=0x7fe9000c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 -- 172.21.15.17:0/1539572468 shutdown_connections 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.061+0000 7fe9060c2640 1 -- 172.21.15.17:0/1539572468 wait complete. 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.062+0000 7fe9060c2640 1 Processor -- start 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.062+0000 7fe9060c2640 1 -- start start 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe900155c80 0x7fe90014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900152900 con 0x7fe9000720a0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000c17f0 con 0x7fe900155c80 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe9060c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe900152a80 con 0x7fe900152520 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33846/0 (socket says 172.21.15.17:33846) 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 -- 172.21.15.17:0/841089145 learned_addr learned my addr 172.21.15.17:0/841089145 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8feffd640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8fffff640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe900155c80 0x7fe90014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 -- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 msgr2=0x7fe90014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90014eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 -- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe900155c80 msgr2=0x7fe90014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe900155c80 0x7fe90014d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 -- 172.21.15.17:0/841089145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90014e0e0 con 0x7fe9000720a0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8ff7fe640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e8a0 tx=0x7fe8f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4017070 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.063+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f400c940 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.064+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90014de00 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.064+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400cd30 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.064+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe900154e90 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.064+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90004f330 con 0x7fe9000720a0 2024-03-06T07:25:10.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.065+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f401a070 con 0x7fe9000720a0 2024-03-06T07:25:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.065+0000 7fe8e7fff640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 0x7fe8ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.065+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8f408a030 con 0x7fe9000720a0 2024-03-06T07:25:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.066+0000 7fe8feffd640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 0x7fe8ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.066+0000 7fe8feffd640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 0x7fe8ec06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0036000 tx=0x7fe8f003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.070+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8f4058760 con 0x7fe9000720a0 2024-03-06T07:25:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.290+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9000c3270 con 0x7fe9000720a0 2024-03-06T07:25:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.290+0000 7fe8e7fff640 1 -- 172.21.15.17:0/841089145 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4990 (secure 0 0 0) 0x7fe8f405c800 con 0x7fe9000720a0 2024-03-06T07:25:10.292 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:10.292 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":120,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:09.279368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"d","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/3327948285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3327948285},{"type":"v1","addr":"172.21.15.163:6837","nonce":3327948285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8280,"qdb_cluster":[8280]},"id":10}]} 2024-03-06T07:25:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 120 2024-03-06T07:25:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 msgr2=0x7fe8ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 0x7fe8ec06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0036000 tx=0x7fe8f003c040 comp rx=0 tx=0).stop 2024-03-06T07:25:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 msgr2=0x7fe9001eeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400e8a0 tx=0x7fe8f400ed60 comp rx=0 tx=0).stop 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 shutdown_connections 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8ec06b4d0 0x7fe8ec06d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe900155c80 0x7fe90014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe900152520 0x7fe90014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 --2- 172.21.15.17:0/841089145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9000720a0 0x7fe9001eeab0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.292+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 >> 172.21.15.17:0/841089145 conn(0x7fe9000c76b0 msgr2=0x7fe9000c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.293+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 shutdown_connections 2024-03-06T07:25:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.293+0000 7fe9060c2640 1 -- 172.21.15.17:0/841089145 wait complete. 2024-03-06T07:25:10.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:10.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.404+0000 7fb0afcbb640 1 Processor -- start 2024-03-06T07:25:10.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 -- start start 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0a814f790 0x7fb0a81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb0a80c1aa0 0x7fb0a80c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a80c28d0 con 0x7fb0a80c5c60 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80c2680 con 0x7fb0a814f790 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.405+0000 7fb0afcbb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80c2bb0 con 0x7fb0a80c1aa0 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ada57640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0a814f790 0x7fb0a81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ae258640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb0a80c5c60 0x7fb0a8155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41180/0 (socket says 172.21.15.17:41180) 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ae258640 1 -- 172.21.15.17:0/954088346 learned_addr learned my addr 172.21.15.17:0/954088346 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ad256640 1 --2- 172.21.15.17:0/954088346 >> v2:172.21.15.163:3300/0 conn(0x7fb0a80c1aa0 0x7fb0a80c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0aca55640 1 -- 172.21.15.17:0/954088346 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652855916 0 0) 0x7fb0a80c28d0 con 0x7fb0a80c5c60 2024-03-06T07:25:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0aca55640 1 -- 172.21.15.17:0/954088346 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb094003520 con 0x7fb0a80c5c60 2024-03-06T07:25:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ada57640 1 -- 172.21.15.17:0/954088346 >> v2:172.21.15.163:3300/0 conn(0x7fb0a80c1aa0 msgr2=0x7fb0a80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ada57640 1 --2- 172.21.15.17:0/954088346 >> v2:172.21.15.163:3300/0 conn(0x7fb0a80c1aa0 0x7fb0a80c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ada57640 1 -- 172.21.15.17:0/954088346 >> v1:172.21.15.17:6789/0 conn(0x7fb0a80c5c60 legacy=0x7fb0a8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.406+0000 7fb0ada57640 1 -- 172.21.15.17:0/954088346 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80c2e90 con 0x7fb0a814f790 2024-03-06T07:25:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0ada57640 1 --2- 172.21.15.17:0/954088346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81529d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8070280 tx=0x7fb0a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca631a83373a130 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0aca55640 1 -- 172.21.15.17:0/954088346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4035020 con 0x7fb0a814f790 2024-03-06T07:25:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0aca55640 1 -- 172.21.15.17:0/954088346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4031830 con 0x7fb0a814f790 2024-03-06T07:25:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0aca55640 1 -- 172.21.15.17:0/954088346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4031c00 con 0x7fb0a814f790 2024-03-06T07:25:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 -- 172.21.15.17:0/954088346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 msgr2=0x7fb0a81529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/954088346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81529d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8070280 tx=0x7fb0a402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 -- 172.21.15.17:0/954088346 shutdown_connections 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/954088346 >> v2:172.21.15.163:3300/0 conn(0x7fb0a80c1aa0 0x7fb0a80c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/954088346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81529d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 -- 172.21.15.17:0/954088346 >> 172.21.15.17:0/954088346 conn(0x7fb0a80c6710 msgr2=0x7fb0a80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 -- 172.21.15.17:0/954088346 shutdown_connections 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.407+0000 7fb0afcbb640 1 -- 172.21.15.17:0/954088346 wait complete. 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 Processor -- start 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 -- start start 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0a80c1aa0 0x7fb0a8146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0a80c5c60 0x7fb0a81469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a806cc00 con 0x7fb0a814f790 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80c0d60 con 0x7fb0a80c1aa0 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0afcbb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a806cd80 con 0x7fb0a80c5c60 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0ae258640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0ae258640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33850/0 (socket says 172.21.15.17:33850) 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0ae258640 1 -- 172.21.15.17:0/1177302964 learned_addr learned my addr 172.21.15.17:0/1177302964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0ada57640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0a80c1aa0 0x7fb0a8146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.409+0000 7fb0ad256640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0a80c5c60 0x7fb0a81469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 -- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0a80c5c60 msgr2=0x7fb0a81469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0a80c5c60 0x7fb0a81469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 -- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0a80c1aa0 msgr2=0x7fb0a8146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0a80c1aa0 0x7fb0a8146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 -- 172.21.15.17:0/1177302964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a814fe50 con 0x7fb0a814f790 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0ae258640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000cb40 tx=0x7fb0a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0017070 con 0x7fb0a814f790 2024-03-06T07:25:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a000aca0 con 0x7fb0a814f790 2024-03-06T07:25:10.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a81516d0 con 0x7fb0a814f790 2024-03-06T07:25:10.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0012620 con 0x7fb0a814f790 2024-03-06T07:25:10.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.410+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8151c90 con 0x7fb0a814f790 2024-03-06T07:25:10.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.411+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a80c28d0 con 0x7fb0a814f790 2024-03-06T07:25:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.411+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a001a070 con 0x7fb0a814f790 2024-03-06T07:25:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.412+0000 7fb09e7fc640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 0x7fb08c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.412+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb0a008a0d0 con 0x7fb0a814f790 2024-03-06T07:25:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.412+0000 7fb0ada57640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 0x7fb08c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.412+0000 7fb0ada57640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 0x7fb08c06d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4036000 tx=0x7fb0a403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.416+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb0a80c28d0 con 0x7fb0a814f790 2024-03-06T07:25:10.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.602+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0a81521b0 con 0x7fb0a814f790 2024-03-06T07:25:10.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.604+0000 7fb09e7fc640 1 -- 172.21.15.17:0/1177302964 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16892 (secure 0 0 0) 0x7fb0a0058780 con 0x7fb0a814f790 2024-03-06T07:25:10.605 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:10.605 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":136,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:08.921115+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:10.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 msgr2=0x7fb08c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 0x7fb08c06d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4036000 tx=0x7fb0a403c040 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 msgr2=0x7fb0a81dfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000cb40 tx=0x7fb0a000a060 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 shutdown_connections 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb08c06b4d0 0x7fb08c06d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a814f790 0x7fb0a81dfda0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0a80c5c60 0x7fb0a81469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 --2- 172.21.15.17:0/1177302964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0a80c1aa0 0x7fb0a8146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 >> 172.21.15.17:0/1177302964 conn(0x7fb0a80c6710 msgr2=0x7fb0a80c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 shutdown_connections 2024-03-06T07:25:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.606+0000 7fb0afcbb640 1 -- 172.21.15.17:0/1177302964 wait complete. 2024-03-06T07:25:10.618 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:25:10.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.717+0000 7ff4fe0c0640 1 Processor -- start 2024-03-06T07:25:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4fe0c0640 1 -- start start 2024-03-06T07:25:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4fe0c0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4f80ac260 0x7ff4f81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4fe0c0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f8152d40 con 0x7ff4f8152960 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4fe0c0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80c1b50 con 0x7ff4f81592d0 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4fe0c0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f8066e30 con 0x7ff4f80ac260 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4f77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff4f8152960 0x7ff4f806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54234/0 (socket says 172.21.15.17:54234) 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.718+0000 7ff4f77fe640 1 -- 172.21.15.17:0/1263457241 learned_addr learned my addr 172.21.15.17:0/1263457241 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f7fff640 1 --2- 172.21.15.17:0/1263457241 >> v2:172.21.15.163:3300/0 conn(0x7ff4f80ac260 0x7ff4f81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551623831 0 0) 0x7ff4f8152d40 con 0x7ff4f8152960 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e0003520 con 0x7ff4f8152960 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1263457241 >> v1:172.21.15.163:6789/0 conn(0x7ff4f81592d0 legacy=0x7ff4f81552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941186683 0 0) 0x7ff4f80c1b50 con 0x7ff4f81592d0 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1263457241 >> v1:172.21.15.17:6789/0 conn(0x7ff4f8152960 legacy=0x7ff4f806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.719+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1263457241 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80c1dc0 con 0x7ff4f80ac260 2024-03-06T07:25:10.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4f7fff640 1 --2- 172.21.15.17:0/1263457241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80ac260 0x7ff4f81559e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f002bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14137d07469d50ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0039070 con 0x7ff4f80ac260 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f002cd00 con 0x7ff4f80ac260 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4f67fc640 1 -- 172.21.15.17:0/1263457241 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0034640 con 0x7ff4f80ac260 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1263457241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80ac260 msgr2=0x7ff4f81559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1263457241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80ac260 0x7ff4f81559e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f002bf50 comp rx=0 tx=0).stop 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1263457241 shutdown_connections 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1263457241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f80ac260 0x7ff4f81559e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1263457241 >> 172.21.15.17:0/1263457241 conn(0x7ff4f80c70b0 msgr2=0x7ff4f80c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1263457241 shutdown_connections 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.720+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1263457241 wait complete. 2024-03-06T07:25:10.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.721+0000 7ff4fe0c0640 1 Processor -- start 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 -- start start 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80ac260 0x7ff4f81edf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f8152960 0x7ff4f8140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f806e3a0 con 0x7ff4f8152f20 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f8159760 con 0x7ff4f8152960 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4fe0c0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f806e520 con 0x7ff4f80ac260 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54160/0 (socket says 172.21.15.17:54160) 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1857583766 learned_addr learned my addr 172.21.15.17:0/1857583766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4f77fe640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80ac260 0x7ff4f81edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.722+0000 7ff4f6ffd640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f8152960 0x7ff4f8140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80ac260 msgr2=0x7ff4f81edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80ac260 0x7ff4f81edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f8152960 msgr2=0x7ff4f8140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f8152960 0x7ff4f8140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 -- 172.21.15.17:0/1857583766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f81ee9d0 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4f7fff640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009eb0 tx=0x7ff4f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0039050 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0045070 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f8153860 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0009450 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.723+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f8120230 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.724+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f804f330 con 0x7ff4f8152f20 2024-03-06T07:25:10.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.724+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4f003e830 con 0x7ff4f8152f20 2024-03-06T07:25:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.725+0000 7ff4dffff640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 0x7ff4e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.725+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4f00ace20 con 0x7ff4f8152f20 2024-03-06T07:25:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.725+0000 7ff4f77fe640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 0x7ff4e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.725+0000 7ff4f77fe640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 0x7ff4e406d9c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005880 tx=0x7ff4e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:10.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.729+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4f004f020 con 0x7ff4f8152f20 2024-03-06T07:25:10.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.906+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4f8156270 con 0x7ff4e406b500 2024-03-06T07:25:10.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.915+0000 7ff4dffff640 1 -- 172.21.15.17:0/1857583766 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247169 (secure 0 0 0) 0x7ff4f8156270 con 0x7ff4e406b500 2024-03-06T07:25:10.919 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:10.921 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 msgr2=0x7ff4e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 0x7ff4e406d9c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005880 tx=0x7ff4e8009040 comp rx=0 tx=0).stop 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 msgr2=0x7ff4f81412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009eb0 tx=0x7ff4f0004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 shutdown_connections 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4e406b500 0x7ff4e406d9c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f8152f20 0x7ff4f81412e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4f8152960 0x7ff4f8140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 --2- 172.21.15.17:0/1857583766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4f80ac260 0x7ff4f81edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 >> 172.21.15.17:0/1857583766 conn(0x7ff4f80c70b0 msgr2=0x7ff4f8155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 shutdown_connections 2024-03-06T07:25:10.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:10.918+0000 7ff4fe0c0640 1 -- 172.21.15.17:0/1857583766 wait complete. 2024-03-06T07:25:10.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-03-06T07:25:09.409375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262676,"kb_used_data":13220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712044,"statfs":{"total":773094113280,"available":772825133056,"internally_reserved":0,"allocated":13537280,"data_stored":7649345,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000996"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925142+0000","last_change":"2024-03-06T07:25:06.925142+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925142+0000","last_undegraded":"2024-03-06T07:25:06.925142+0000","last_fullsized":"2024-03-06T07:25:06.925142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.923368+0000","last_change":"2024-03-06T07:25:06.923368+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.923368+0000","last_undegraded":"2024-03-06T07:25:06.923368+0000","last_fullsized":"2024-03-06T07:25:06.923368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924643+0000","last_change":"2024-03-06T07:25:06.924643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924643+0000","last_undegraded":"2024-03-06T07:25:06.924643+0000","last_fullsized":"2024-03-06T07:25:06.924643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924710+0000","last_change":"2024-03-06T07:25:06.924710+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924710+0000","last_undegraded":"2024-03-06T07:25:06.924710+0000","last_fullsized":"2024-03-06T07:25:06.924710+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922611+0000","last_change":"2024-03-06T07:25:06.922611+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922611+0000","last_undegraded":"2024-03-06T07:25:06.922611+0000","last_fullsized":"2024-03-06T07:25:06.922611+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922132+0000","last_change":"2024-03-06T07:25:06.922132+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922132+0000","last_undegraded":"2024-03-06T07:25:06.922132+0000","last_fullsized":"2024-03-06T07:25:06.922132+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927603+0000","last_change":"2024-03-06T07:25:06.927603+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927603+0000","last_undegraded":"2024-03-06T07:25:06.927603+0000","last_fullsized":"2024-03-06T07:25:06.927603+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918232+0000","last_change":"2024-03-06T07:25:06.918232+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918232+0000","last_undegraded":"2024-03-06T07:25:06.918232+0000","last_fullsized":"2024-03-06T07:25:06.918232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921643+0000","last_change":"2024-03-06T07:25:06.921643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921643+0000","last_undegraded":"2024-03-06T07:25:06.921643+0000","last_fullsized":"2024-03-06T07:25:06.921643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921166+0000","last_change":"2024-03-06T07:25:06.921166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921166+0000","last_undegraded":"2024-03-06T07:25:06.921166+0000","last_fullsized":"2024-03-06T07:25:06.921166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920314+0000","last_change":"2024-03-06T07:25:06.920314+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920314+0000","last_undegraded":"2024-03-06T07:25:06.920314+0000","last_fullsized":"2024-03-06T07:25:06.920314+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924221+0000","last_change":"2024-03-06T07:25:06.924221+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924221+0000","last_undegraded":"2024-03-06T07:25:06.924221+0000","last_fullsized":"2024-03-06T07:25:06.924221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921124+0000","last_change":"2024-03-06T07:25:06.921124+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921124+0000","last_undegraded":"2024-03-06T07:25:06.921124+0000","last_fullsized":"2024-03-06T07:25:06.921124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924524+0000","last_change":"2024-03-06T07:25:06.924524+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924524+0000","last_undegraded":"2024-03-06T07:25:06.924524+0000","last_fullsized":"2024-03-06T07:25:06.924524+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921656+0000","last_change":"2024-03-06T07:25:06.921656+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921656+0000","last_undegraded":"2024-03-06T07:25:06.921656+0000","last_fullsized":"2024-03-06T07:25:06.921656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921140+0000","last_change":"2024-03-06T07:25:06.921140+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921140+0000","last_undegraded":"2024-03-06T07:25:06.921140+0000","last_fullsized":"2024-03-06T07:25:06.921140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924794+0000","last_change":"2024-03-06T07:25:06.924794+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924794+0000","last_undegraded":"2024-03-06T07:25:06.924794+0000","last_fullsized":"2024-03-06T07:25:06.924794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918730+0000","last_change":"2024-03-06T07:25:06.918730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918730+0000","last_undegraded":"2024-03-06T07:25:06.918730+0000","last_fullsized":"2024-03-06T07:25:06.918730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918574+0000","last_change":"2024-03-06T07:25:06.918574+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918574+0000","last_undegraded":"2024-03-06T07:25:06.918574+0000","last_fullsized":"2024-03-06T07:25:06.918574+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920290+0000","last_change":"2024-03-06T07:25:06.920290+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920290+0000","last_undegraded":"2024-03-06T07:25:06.920290+0000","last_fullsized":"2024-03-06T07:25:06.920290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920527+0000","last_change":"2024-03-06T07:25:06.920527+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920527+0000","last_undegraded":"2024-03-06T07:25:06.920527+0000","last_fullsized":"2024-03-06T07:25:06.920527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.926454+0000","last_change":"2024-03-06T07:25:06.926454+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.926454+0000","last_undegraded":"2024-03-06T07:25:06.926454+0000","last_fullsized":"2024-03-06T07:25:06.926454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917006+0000","last_change":"2024-03-06T07:25:06.917006+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917006+0000","last_undegraded":"2024-03-06T07:25:06.917006+0000","last_fullsized":"2024-03-06T07:25:06.917006+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917762+0000","last_change":"2024-03-06T07:25:06.917762+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917762+0000","last_undegraded":"2024-03-06T07:25:06.917762+0000","last_fullsized":"2024-03-06T07:25:06.917762+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924716+0000","last_change":"2024-03-06T07:25:06.924716+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924716+0000","last_undegraded":"2024-03-06T07:25:06.924716+0000","last_fullsized":"2024-03-06T07:25:06.924716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925850+0000","last_change":"2024-03-06T07:25:06.925850+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925850+0000","last_undegraded":"2024-03-06T07:25:06.925850+0000","last_fullsized":"2024-03-06T07:25:06.925850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919602+0000","last_change":"2024-03-06T07:25:06.919602+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919602+0000","last_undegraded":"2024-03-06T07:25:06.919602+0000","last_fullsized":"2024-03-06T07:25:06.919602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918067+0000","last_change":"2024-03-06T07:25:06.918067+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918067+0000","last_undegraded":"2024-03-06T07:25:06.918067+0000","last_fullsized":"2024-03-06T07:25:06.918067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918954+0000","last_change":"2024-03-06T07:25:06.918954+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918954+0000","last_undegraded":"2024-03-06T07:25:06.918954+0000","last_fullsized":"2024-03-06T07:25:06.918954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918136+0000","last_change":"2024-03-06T07:25:06.918136+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918136+0000","last_undegraded":"2024-03-06T07:25:06.918136+0000","last_fullsized":"2024-03-06T07:25:06.918136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917781+0000","last_change":"2024-03-06T07:25:06.917781+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917781+0000","last_undegraded":"2024-03-06T07:25:06.917781+0000","last_fullsized":"2024-03-06T07:25:06.917781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922036+0000","last_change":"2024-03-06T07:25:06.922036+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922036+0000","last_undegraded":"2024-03-06T07:25:06.922036+0000","last_fullsized":"2024-03-06T07:25:06.922036+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.928472+0000","last_change":"2024-03-06T07:25:06.926918+0000","last_active":"2024-03-06T07:25:06.928472+0000","last_peered":"2024-03-06T07:25:06.928472+0000","last_clean":"2024-03-06T07:25:06.928472+0000","last_became_active":"2024-03-06T07:25:06.926475+0000","last_became_peered":"2024-03-06T07:25:06.926475+0000","last_unstale":"2024-03-06T07:25:06.928472+0000","last_undegraded":"2024-03-06T07:25:06.928472+0000","last_fullsized":"2024-03-06T07:25:06.928472+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:35.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924223+0000","last_change":"2024-03-06T07:25:06.924223+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924223+0000","last_undegraded":"2024-03-06T07:25:06.924223+0000","last_fullsized":"2024-03-06T07:25:06.924223+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937287+0000","last_change":"2024-03-06T07:25:06.933126+0000","last_active":"2024-03-06T07:25:06.937287+0000","last_peered":"2024-03-06T07:25:06.937287+0000","last_clean":"2024-03-06T07:25:06.937287+0000","last_became_active":"2024-03-06T07:25:06.932843+0000","last_became_peered":"2024-03-06T07:25:06.932843+0000","last_unstale":"2024-03-06T07:25:06.937287+0000","last_undegraded":"2024-03-06T07:25:06.937287+0000","last_fullsized":"2024-03-06T07:25:06.937287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:33.155673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916730+0000","last_change":"2024-03-06T07:25:06.916730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916730+0000","last_undegraded":"2024-03-06T07:25:06.916730+0000","last_fullsized":"2024-03-06T07:25:06.916730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.934052+0000","last_change":"2024-03-06T07:25:06.932116+0000","last_active":"2024-03-06T07:25:06.934052+0000","last_peered":"2024-03-06T07:25:06.934052+0000","last_clean":"2024-03-06T07:25:06.934052+0000","last_became_active":"2024-03-06T07:25:06.931835+0000","last_became_peered":"2024-03-06T07:25:06.931835+0000","last_unstale":"2024-03-06T07:25:06.934052+0000","last_undegraded":"2024-03-06T07:25:06.934052+0000","last_fullsized":"2024-03-06T07:25:06.934052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:52.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918299+0000","last_change":"2024-03-06T07:25:06.918299+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918299+0000","last_undegraded":"2024-03-06T07:25:06.918299+0000","last_fullsized":"2024-03-06T07:25:06.918299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937080+0000","last_change":"2024-03-06T07:25:06.936612+0000","last_active":"2024-03-06T07:25:06.937080+0000","last_peered":"2024-03-06T07:25:06.937080+0000","last_clean":"2024-03-06T07:25:06.937080+0000","last_became_active":"2024-03-06T07:25:06.936165+0000","last_became_peered":"2024-03-06T07:25:06.936165+0000","last_unstale":"2024-03-06T07:25:06.937080+0000","last_undegraded":"2024-03-06T07:25:06.937080+0000","last_fullsized":"2024-03-06T07:25:06.937080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:41.959433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920192+0000","last_change":"2024-03-06T07:25:06.920192+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920192+0000","last_undegraded":"2024-03-06T07:25:06.920192+0000","last_fullsized":"2024-03-06T07:25:06.920192+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937555+0000","last_change":"2024-03-06T07:25:06.932157+0000","last_active":"2024-03-06T07:25:06.937555+0000","last_peered":"2024-03-06T07:25:06.937555+0000","last_clean":"2024-03-06T07:25:06.937555+0000","last_became_active":"2024-03-06T07:25:06.931866+0000","last_became_peered":"2024-03-06T07:25:06.931866+0000","last_unstale":"2024-03-06T07:25:06.937555+0000","last_undegraded":"2024-03-06T07:25:06.937555+0000","last_fullsized":"2024-03-06T07:25:06.937555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.500674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925548+0000","last_change":"2024-03-06T07:25:06.925548+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925548+0000","last_undegraded":"2024-03-06T07:25:06.925548+0000","last_fullsized":"2024-03-06T07:25:06.925548+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933326+0000","last_change":"2024-03-06T07:25:06.932382+0000","last_active":"2024-03-06T07:25:06.933326+0000","last_peered":"2024-03-06T07:25:06.933326+0000","last_clean":"2024-03-06T07:25:06.933326+0000","last_became_active":"2024-03-06T07:25:06.932043+0000","last_became_peered":"2024-03-06T07:25:06.932043+0000","last_unstale":"2024-03-06T07:25:06.933326+0000","last_undegraded":"2024-03-06T07:25:06.933326+0000","last_fullsized":"2024-03-06T07:25:06.933326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:37.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921418+0000","last_change":"2024-03-06T07:25:06.921418+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921418+0000","last_undegraded":"2024-03-06T07:25:06.921418+0000","last_fullsized":"2024-03-06T07:25:06.921418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936978+0000","last_change":"2024-03-06T07:25:06.934761+0000","last_active":"2024-03-06T07:25:06.936978+0000","last_peered":"2024-03-06T07:25:06.936978+0000","last_clean":"2024-03-06T07:25:06.936978+0000","last_became_active":"2024-03-06T07:25:06.934441+0000","last_became_peered":"2024-03-06T07:25:06.934441+0000","last_unstale":"2024-03-06T07:25:06.936978+0000","last_undegraded":"2024-03-06T07:25:06.936978+0000","last_fullsized":"2024-03-06T07:25:06.936978+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:30.827701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927721+0000","last_change":"2024-03-06T07:25:06.927721+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927721+0000","last_undegraded":"2024-03-06T07:25:06.927721+0000","last_fullsized":"2024-03-06T07:25:06.927721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936449+0000","last_change":"2024-03-06T07:25:06.935857+0000","last_active":"2024-03-06T07:25:06.936449+0000","last_peered":"2024-03-06T07:25:06.936449+0000","last_clean":"2024-03-06T07:25:06.936449+0000","last_became_active":"2024-03-06T07:25:06.935520+0000","last_became_peered":"2024-03-06T07:25:06.935520+0000","last_unstale":"2024-03-06T07:25:06.936449+0000","last_undegraded":"2024-03-06T07:25:06.936449+0000","last_fullsized":"2024-03-06T07:25:06.936449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:41.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916666+0000","last_change":"2024-03-06T07:25:06.916666+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916666+0000","last_undegraded":"2024-03-06T07:25:06.916666+0000","last_fullsized":"2024-03-06T07:25:06.916666+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931892+0000","last_change":"2024-03-06T07:25:06.928957+0000","last_active":"2024-03-06T07:25:06.931892+0000","last_peered":"2024-03-06T07:25:06.931892+0000","last_clean":"2024-03-06T07:25:06.931892+0000","last_became_active":"2024-03-06T07:25:06.928262+0000","last_became_peered":"2024-03-06T07:25:06.928262+0000","last_unstale":"2024-03-06T07:25:06.931892+0000","last_undegraded":"2024-03-06T07:25:06.931892+0000","last_fullsized":"2024-03-06T07:25:06.931892+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:21.229011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919913+0000","last_change":"2024-03-06T07:25:06.919913+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919913+0000","last_undegraded":"2024-03-06T07:25:06.919913+0000","last_fullsized":"2024-03-06T07:25:06.919913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935245+0000","last_change":"2024-03-06T07:25:06.931781+0000","last_active":"2024-03-06T07:25:06.935245+0000","last_peered":"2024-03-06T07:25:06.935245+0000","last_clean":"2024-03-06T07:25:06.935245+0000","last_became_active":"2024-03-06T07:25:06.931456+0000","last_became_peered":"2024-03-06T07:25:06.931456+0000","last_unstale":"2024-03-06T07:25:06.935245+0000","last_undegraded":"2024-03-06T07:25:06.935245+0000","last_fullsized":"2024-03-06T07:25:06.935245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918286+0000","last_change":"2024-03-06T07:25:06.918286+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918286+0000","last_undegraded":"2024-03-06T07:25:06.918286+0000","last_fullsized":"2024-03-06T07:25:06.918286+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926217+0000","last_change":"2024-03-06T07:25:06.925504+0000","last_active":"2024-03-06T07:25:06.926217+0000","last_peered":"2024-03-06T07:25:06.926217+0000","last_clean":"2024-03-06T07:25:06.926217+0000","last_became_active":"2024-03-06T07:25:06.925180+0000","last_became_peered":"2024-03-06T07:25:06.925180+0000","last_unstale":"2024-03-06T07:25:06.926217+0000","last_undegraded":"2024-03-06T07:25:06.926217+0000","last_fullsized":"2024-03-06T07:25:06.926217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:19.573816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917364+0000","last_change":"2024-03-06T07:25:06.917364+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917364+0000","last_undegraded":"2024-03-06T07:25:06.917364+0000","last_fullsized":"2024-03-06T07:25:06.917364+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926965+0000","last_change":"2024-03-06T07:25:06.926155+0000","last_active":"2024-03-06T07:25:06.926965+0000","last_peered":"2024-03-06T07:25:06.926965+0000","last_clean":"2024-03-06T07:25:06.926965+0000","last_became_active":"2024-03-06T07:25:06.925662+0000","last_became_peered":"2024-03-06T07:25:06.925662+0000","last_unstale":"2024-03-06T07:25:06.926965+0000","last_undegraded":"2024-03-06T07:25:06.926965+0000","last_fullsized":"2024-03-06T07:25:06.926965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:36.511111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919303+0000","last_change":"2024-03-06T07:25:06.919303+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919303+0000","last_undegraded":"2024-03-06T07:25:06.919303+0000","last_fullsized":"2024-03-06T07:25:06.919303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942235+0000","last_change":"2024-03-06T07:25:06.936715+0000","last_active":"2024-03-06T07:25:06.942235+0000","last_peered":"2024-03-06T07:25:06.942235+0000","last_clean":"2024-03-06T07:25:06.942235+0000","last_became_active":"2024-03-06T07:25:06.936361+0000","last_became_peered":"2024-03-06T07:25:06.936361+0000","last_unstale":"2024-03-06T07:25:06.942235+0000","last_undegraded":"2024-03-06T07:25:06.942235+0000","last_fullsized":"2024-03-06T07:25:06.942235+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919086+0000","last_change":"2024-03-06T07:25:06.919086+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919086+0000","last_undegraded":"2024-03-06T07:25:06.919086+0000","last_fullsized":"2024-03-06T07:25:06.919086+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937421+0000","last_change":"2024-03-06T07:25:06.934580+0000","last_active":"2024-03-06T07:25:06.937421+0000","last_peered":"2024-03-06T07:25:06.937421+0000","last_clean":"2024-03-06T07:25:06.937421+0000","last_became_active":"2024-03-06T07:25:06.934296+0000","last_became_peered":"2024-03-06T07:25:06.934296+0000","last_unstale":"2024-03-06T07:25:06.937421+0000","last_undegraded":"2024-03-06T07:25:06.937421+0000","last_fullsized":"2024-03-06T07:25:06.937421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:32.439585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920493+0000","last_change":"2024-03-06T07:25:06.920493+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920493+0000","last_undegraded":"2024-03-06T07:25:06.920493+0000","last_fullsized":"2024-03-06T07:25:06.920493+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937088+0000","last_change":"2024-03-06T07:25:06.934527+0000","last_active":"2024-03-06T07:25:06.937088+0000","last_peered":"2024-03-06T07:25:06.937088+0000","last_clean":"2024-03-06T07:25:06.937088+0000","last_became_active":"2024-03-06T07:25:06.934249+0000","last_became_peered":"2024-03-06T07:25:06.934249+0000","last_unstale":"2024-03-06T07:25:06.937088+0000","last_undegraded":"2024-03-06T07:25:06.937088+0000","last_fullsized":"2024-03-06T07:25:06.937088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917750+0000","last_change":"2024-03-06T07:25:06.917750+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917750+0000","last_undegraded":"2024-03-06T07:25:06.917750+0000","last_fullsized":"2024-03-06T07:25:06.917750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.925249+0000","last_change":"2024-03-06T07:25:06.924175+0000","last_active":"2024-03-06T07:25:06.925249+0000","last_peered":"2024-03-06T07:25:06.925249+0000","last_clean":"2024-03-06T07:25:06.925249+0000","last_became_active":"2024-03-06T07:25:06.923894+0000","last_became_peered":"2024-03-06T07:25:06.923894+0000","last_unstale":"2024-03-06T07:25:06.925249+0000","last_undegraded":"2024-03-06T07:25:06.925249+0000","last_fullsized":"2024-03-06T07:25:06.925249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:34.739537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918166+0000","last_change":"2024-03-06T07:25:06.918166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918166+0000","last_undegraded":"2024-03-06T07:25:06.918166+0000","last_fullsized":"2024-03-06T07:25:06.918166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931792+0000","last_change":"2024-03-06T07:25:06.926576+0000","last_active":"2024-03-06T07:25:06.931792+0000","last_peered":"2024-03-06T07:25:06.931792+0000","last_clean":"2024-03-06T07:25:06.931792+0000","last_became_active":"2024-03-06T07:25:06.926315+0000","last_became_peered":"2024-03-06T07:25:06.926315+0000","last_unstale":"2024-03-06T07:25:06.931792+0000","last_undegraded":"2024-03-06T07:25:06.931792+0000","last_fullsized":"2024-03-06T07:25:06.931792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:38.818021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925302+0000","last_change":"2024-03-06T07:25:06.925302+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925302+0000","last_undegraded":"2024-03-06T07:25:06.925302+0000","last_fullsized":"2024-03-06T07:25:06.925302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935481+0000","last_change":"2024-03-06T07:25:06.935036+0000","last_active":"2024-03-06T07:25:06.935481+0000","last_peered":"2024-03-06T07:25:06.935481+0000","last_clean":"2024-03-06T07:25:06.935481+0000","last_became_active":"2024-03-06T07:25:06.934723+0000","last_became_peered":"2024-03-06T07:25:06.934723+0000","last_unstale":"2024-03-06T07:25:06.935481+0000","last_undegraded":"2024-03-06T07:25:06.935481+0000","last_fullsized":"2024-03-06T07:25:06.935481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:25.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924247+0000","last_change":"2024-03-06T07:25:06.924247+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924247+0000","last_undegraded":"2024-03-06T07:25:06.924247+0000","last_fullsized":"2024-03-06T07:25:06.924247+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942252+0000","last_change":"2024-03-06T07:25:06.937027+0000","last_active":"2024-03-06T07:25:06.942252+0000","last_peered":"2024-03-06T07:25:06.942252+0000","last_clean":"2024-03-06T07:25:06.942252+0000","last_became_active":"2024-03-06T07:25:06.936721+0000","last_became_peered":"2024-03-06T07:25:06.936721+0000","last_unstale":"2024-03-06T07:25:06.942252+0000","last_undegraded":"2024-03-06T07:25:06.942252+0000","last_fullsized":"2024-03-06T07:25:06.942252+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917215+0000","last_change":"2024-03-06T07:25:06.917215+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917215+0000","last_undegraded":"2024-03-06T07:25:06.917215+0000","last_fullsized":"2024-03-06T07:25:06.917215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932764+0000","last_change":"2024-03-06T07:25:06.930306+0000","last_active":"2024-03-06T07:25:06.932764+0000","last_peered":"2024-03-06T07:25:06.932764+0000","last_clean":"2024-03-06T07:25:06.932764+0000","last_became_active":"2024-03-06T07:25:06.929975+0000","last_became_peered":"2024-03-06T07:25:06.929975+0000","last_unstale":"2024-03-06T07:25:06.932764+0000","last_undegraded":"2024-03-06T07:25:06.932764+0000","last_fullsized":"2024-03-06T07:25:06.932764+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:14.435468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918737+0000","last_change":"2024-03-06T07:25:06.918737+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918737+0000","last_undegraded":"2024-03-06T07:25:06.918737+0000","last_fullsized":"2024-03-06T07:25:06.918737+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935878+0000","last_change":"2024-03-06T07:25:06.930143+0000","last_active":"2024-03-06T07:25:06.935878+0000","last_peered":"2024-03-06T07:25:06.935878+0000","last_clean":"2024-03-06T07:25:06.935878+0000","last_became_active":"2024-03-06T07:25:06.929880+0000","last_became_peered":"2024-03-06T07:25:06.929880+0000","last_unstale":"2024-03-06T07:25:06.935878+0000","last_undegraded":"2024-03-06T07:25:06.935878+0000","last_fullsized":"2024-03-06T07:25:06.935878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:51.483404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917337+0000","last_change":"2024-03-06T07:25:06.917337+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917337+0000","last_undegraded":"2024-03-06T07:25:06.917337+0000","last_fullsized":"2024-03-06T07:25:06.917337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932734+0000","last_change":"2024-03-06T07:25:06.930314+0000","last_active":"2024-03-06T07:25:06.932734+0000","last_peered":"2024-03-06T07:25:06.932734+0000","last_clean":"2024-03-06T07:25:06.932734+0000","last_became_active":"2024-03-06T07:25:06.929990+0000","last_became_peered":"2024-03-06T07:25:06.929990+0000","last_unstale":"2024-03-06T07:25:06.932734+0000","last_undegraded":"2024-03-06T07:25:06.932734+0000","last_fullsized":"2024-03-06T07:25:06.932734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:38.139179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932209+0000","last_change":"2024-03-06T07:25:06.925110+0000","last_active":"2024-03-06T07:25:06.932209+0000","last_peered":"2024-03-06T07:25:06.932209+0000","last_clean":"2024-03-06T07:25:06.932209+0000","last_became_active":"2024-03-06T07:25:06.924842+0000","last_became_peered":"2024-03-06T07:25:06.924842+0000","last_unstale":"2024-03-06T07:25:06.932209+0000","last_undegraded":"2024-03-06T07:25:06.932209+0000","last_fullsized":"2024-03-06T07:25:06.932209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:52.264094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919201+0000","last_change":"2024-03-06T07:25:06.919201+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919201+0000","last_undegraded":"2024-03-06T07:25:06.919201+0000","last_fullsized":"2024-03-06T07:25:06.919201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942224+0000","last_change":"2024-03-06T07:25:06.936235+0000","last_active":"2024-03-06T07:25:06.942224+0000","last_peered":"2024-03-06T07:25:06.942224+0000","last_clean":"2024-03-06T07:25:06.942224+0000","last_became_active":"2024-03-06T07:25:06.935934+0000","last_became_peered":"2024-03-06T07:25:06.935934+0000","last_unstale":"2024-03-06T07:25:06.942224+0000","last_undegraded":"2024-03-06T07:25:06.942224+0000","last_fullsized":"2024-03-06T07:25:06.942224+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925078+0000","last_change":"2024-03-06T07:25:06.925078+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925078+0000","last_undegraded":"2024-03-06T07:25:06.925078+0000","last_fullsized":"2024-03-06T07:25:06.925078+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935264+0000","last_change":"2024-03-06T07:25:06.933630+0000","last_active":"2024-03-06T07:25:06.935264+0000","last_peered":"2024-03-06T07:25:06.935264+0000","last_clean":"2024-03-06T07:25:06.935264+0000","last_became_active":"2024-03-06T07:25:06.933254+0000","last_became_peered":"2024-03-06T07:25:06.933254+0000","last_unstale":"2024-03-06T07:25:06.935264+0000","last_undegraded":"2024-03-06T07:25:06.935264+0000","last_fullsized":"2024-03-06T07:25:06.935264+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924327+0000","last_change":"2024-03-06T07:25:06.924327+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924327+0000","last_undegraded":"2024-03-06T07:25:06.924327+0000","last_fullsized":"2024-03-06T07:25:06.924327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936838+0000","last_change":"2024-03-06T07:25:06.936426+0000","last_active":"2024-03-06T07:25:06.936838+0000","last_peered":"2024-03-06T07:25:06.936838+0000","last_clean":"2024-03-06T07:25:06.936838+0000","last_became_active":"2024-03-06T07:25:06.936160+0000","last_became_peered":"2024-03-06T07:25:06.936160+0000","last_unstale":"2024-03-06T07:25:06.936838+0000","last_undegraded":"2024-03-06T07:25:06.936838+0000","last_fullsized":"2024-03-06T07:25:06.936838+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:39.491573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919378+0000","last_change":"2024-03-06T07:25:06.919378+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919378+0000","last_undegraded":"2024-03-06T07:25:06.919378+0000","last_fullsized":"2024-03-06T07:25:06.919378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932280+0000","last_change":"2024-03-06T07:25:06.930511+0000","last_active":"2024-03-06T07:25:06.932280+0000","last_peered":"2024-03-06T07:25:06.932280+0000","last_clean":"2024-03-06T07:25:06.932280+0000","last_became_active":"2024-03-06T07:25:06.930205+0000","last_became_peered":"2024-03-06T07:25:06.930205+0000","last_unstale":"2024-03-06T07:25:06.932280+0000","last_undegraded":"2024-03-06T07:25:06.932280+0000","last_fullsized":"2024-03-06T07:25:06.932280+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:47.054740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916851+0000","last_change":"2024-03-06T07:25:06.916851+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916851+0000","last_undegraded":"2024-03-06T07:25:06.916851+0000","last_fullsized":"2024-03-06T07:25:06.916851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936972+0000","last_change":"2024-03-06T07:25:06.935597+0000","last_active":"2024-03-06T07:25:06.936972+0000","last_peered":"2024-03-06T07:25:06.936972+0000","last_clean":"2024-03-06T07:25:06.936972+0000","last_became_active":"2024-03-06T07:25:06.935195+0000","last_became_peered":"2024-03-06T07:25:06.935195+0000","last_unstale":"2024-03-06T07:25:06.936972+0000","last_undegraded":"2024-03-06T07:25:06.936972+0000","last_fullsized":"2024-03-06T07:25:06.936972+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:50.432110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924317+0000","last_change":"2024-03-06T07:25:06.924317+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924317+0000","last_undegraded":"2024-03-06T07:25:06.924317+0000","last_fullsized":"2024-03-06T07:25:06.924317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932947+0000","last_change":"2024-03-06T07:25:06.931052+0000","last_active":"2024-03-06T07:25:06.932947+0000","last_peered":"2024-03-06T07:25:06.932947+0000","last_clean":"2024-03-06T07:25:06.932947+0000","last_became_active":"2024-03-06T07:25:06.930765+0000","last_became_peered":"2024-03-06T07:25:06.930765+0000","last_unstale":"2024-03-06T07:25:06.932947+0000","last_undegraded":"2024-03-06T07:25:06.932947+0000","last_fullsized":"2024-03-06T07:25:06.932947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:26.305654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920999+0000","last_change":"2024-03-06T07:25:06.920999+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920999+0000","last_undegraded":"2024-03-06T07:25:06.920999+0000","last_fullsized":"2024-03-06T07:25:06.920999+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937201+0000","last_change":"2024-03-06T07:25:06.935076+0000","last_active":"2024-03-06T07:25:06.937201+0000","last_peered":"2024-03-06T07:25:06.937201+0000","last_clean":"2024-03-06T07:25:06.937201+0000","last_became_active":"2024-03-06T07:25:06.934424+0000","last_became_peered":"2024-03-06T07:25:06.934424+0000","last_unstale":"2024-03-06T07:25:06.937201+0000","last_undegraded":"2024-03-06T07:25:06.937201+0000","last_fullsized":"2024-03-06T07:25:06.937201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:55.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919352+0000","last_change":"2024-03-06T07:25:06.919352+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919352+0000","last_undegraded":"2024-03-06T07:25:06.919352+0000","last_fullsized":"2024-03-06T07:25:06.919352+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931872+0000","last_change":"2024-03-06T07:25:06.927019+0000","last_active":"2024-03-06T07:25:06.931872+0000","last_peered":"2024-03-06T07:25:06.931872+0000","last_clean":"2024-03-06T07:25:06.931872+0000","last_became_active":"2024-03-06T07:25:06.926737+0000","last_became_peered":"2024-03-06T07:25:06.926737+0000","last_unstale":"2024-03-06T07:25:06.931872+0000","last_undegraded":"2024-03-06T07:25:06.931872+0000","last_fullsized":"2024-03-06T07:25:06.931872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:29.054350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916675+0000","last_change":"2024-03-06T07:25:06.916675+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916675+0000","last_undegraded":"2024-03-06T07:25:06.916675+0000","last_fullsized":"2024-03-06T07:25:06.916675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933731+0000","last_change":"2024-03-06T07:25:06.933005+0000","last_active":"2024-03-06T07:25:06.933731+0000","last_peered":"2024-03-06T07:25:06.933731+0000","last_clean":"2024-03-06T07:25:06.933731+0000","last_became_active":"2024-03-06T07:25:06.932668+0000","last_became_peered":"2024-03-06T07:25:06.932668+0000","last_unstale":"2024-03-06T07:25:06.933731+0000","last_undegraded":"2024-03-06T07:25:06.933731+0000","last_fullsized":"2024-03-06T07:25:06.933731+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:20.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.918569+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:06.918569+0000","last_peered":"2024-03-06T07:25:06.918569+0000","last_clean":"2024-03-06T07:25:06.918569+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:06.918569+0000","last_undegraded":"2024-03-06T07:25:06.918569+0000","last_fullsized":"2024-03-06T07:25:06.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:10.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.070+0000 7f4d727bc640 1 Processor -- start 2024-03-06T07:25:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.070+0000 7f4d727bc640 1 -- start start 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d727bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 0x7f4d6c1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d727bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c152d40 con 0x7f4d6c152960 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d727bc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6c0c1b50 con 0x7f4d6c0ac260 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d727bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c066e30 con 0x7f4d6c1592d0 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4d6c152960 0x7f4d6c06dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54242/0 (socket says 172.21.15.17:54242) 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6bfff640 1 -- 172.21.15.17:0/3306441164 learned_addr learned my addr 172.21.15.17:0/3306441164 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6b7fe640 1 --2- 172.21.15.17:0/3306441164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 0x7f4d6c1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484449002 0 0) 0x7f4d6c152d40 con 0x7f4d6c152960 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d54003520 con 0x7f4d6c152960 2024-03-06T07:25:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113271852 0 0) 0x7f4d6c0c1b50 con 0x7f4d6c0ac260 2024-03-06T07:25:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c152d40 con 0x7f4d6c0ac260 2024-03-06T07:25:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6b7fe640 1 -- 172.21.15.17:0/3306441164 >> v1:172.21.15.163:6789/0 conn(0x7f4d6c0ac260 legacy=0x7f4d6c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6b7fe640 1 -- 172.21.15.17:0/3306441164 >> v1:172.21.15.17:6789/0 conn(0x7f4d6c152960 legacy=0x7f4d6c06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.071+0000 7f4d6b7fe640 1 -- 172.21.15.17:0/3306441164 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0c1dc0 con 0x7f4d6c1592d0 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.072+0000 7f4d6b7fe640 1 --2- 172.21.15.17:0/3306441164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 0x7f4d6c1552d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0097d0 tx=0x7f4d5c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e11cfc74e7986aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.072+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c035020 con 0x7f4d6c1592d0 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.072+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c031890 con 0x7f4d6c1592d0 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.072+0000 7f4d6affd640 1 -- 172.21.15.17:0/3306441164 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c031c60 con 0x7f4d6c1592d0 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 -- 172.21.15.17:0/3306441164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 msgr2=0x7f4d6c1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 --2- 172.21.15.17:0/3306441164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 0x7f4d6c1552d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0097d0 tx=0x7f4d5c02b930 comp rx=0 tx=0).stop 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 -- 172.21.15.17:0/3306441164 shutdown_connections 2024-03-06T07:25:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 --2- 172.21.15.17:0/3306441164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c1592d0 0x7f4d6c1552d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 -- 172.21.15.17:0/3306441164 >> 172.21.15.17:0/3306441164 conn(0x7f4d6c0c70b0 msgr2=0x7f4d6c0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 -- 172.21.15.17:0/3306441164 shutdown_connections 2024-03-06T07:25:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.073+0000 7f4d727bc640 1 -- 172.21.15.17:0/3306441164 wait complete. 2024-03-06T07:25:11.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.074+0000 7f4d727bc640 1 Processor -- start 2024-03-06T07:25:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.074+0000 7f4d727bc640 1 -- start start 2024-03-06T07:25:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c0ac260 0x7f4d6c0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c06e3a0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c159760 con 0x7f4d6c152960 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c06e520 con 0x7f4d6c0ac260 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:55392/0 (socket says 172.21.15.17:55392) 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d6bfff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c0ac260 0x7f4d6c0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d6b7fe640 1 -- 172.21.15.17:0/44494599 learned_addr learned my addr 172.21.15.17:0/44494599 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54170/0 (socket says 172.21.15.17:54170) 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 -- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c0ac260 msgr2=0x7f4d6c0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c0ac260 0x7f4d6c0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 -- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 msgr2=0x7f4d6c1ead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 -- 172.21.15.17:0/44494599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c1545e0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d6b7fe640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d70d59640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c690 tx=0x7f4d6400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64003b70 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d64003d30 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d64005310 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.075+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c1ef060 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.076+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c156b50 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.076+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c074ac0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.077+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d640054b0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.077+0000 7f4d68ff9640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 0x7f4d5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.077+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d64089fd0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.078+0000 7f4d6bfff640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 0x7f4d5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.078+0000 7f4d6bfff640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 0x7f4d5806d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4d60000f60 tx=0x7f4d60007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.082+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d6c074ac0 con 0x7f4d6c1dedc0 2024-03-06T07:25:11.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.258+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d6c156270 con 0x7f4d5806b500 2024-03-06T07:25:11.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.264+0000 7f4d68ff9640 1 -- 172.21.15.17:0/44494599 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247169 (secure 0 0 0) 0x7f4d6c156270 con 0x7f4d5806b500 2024-03-06T07:25:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 msgr2=0x7f4d5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 0x7f4d5806d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4d60000f60 tx=0x7f4d60007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 msgr2=0x7f4d6c1eb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4d6400c690 tx=0x7f4d6400cb50 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 shutdown_connections 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d5806b500 0x7f4d5806d9c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d6c1dedc0 0x7f4d6c1eb2b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d6c152960 0x7f4d6c1ead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 --2- 172.21.15.17:0/44494599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d6c0ac260 0x7f4d6c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 >> 172.21.15.17:0/44494599 conn(0x7f4d6c0c70b0 msgr2=0x7f4d6c0c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 shutdown_connections 2024-03-06T07:25:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.267+0000 7f4d727bc640 1 -- 172.21.15.17:0/44494599 wait complete. 2024-03-06T07:25:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-03-06T07:25:09.409375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262676,"kb_used_data":13220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712044,"statfs":{"total":773094113280,"available":772825133056,"internally_reserved":0,"allocated":13537280,"data_stored":7649345,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000996"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925142+0000","last_change":"2024-03-06T07:25:06.925142+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925142+0000","last_undegraded":"2024-03-06T07:25:06.925142+0000","last_fullsized":"2024-03-06T07:25:06.925142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.923368+0000","last_change":"2024-03-06T07:25:06.923368+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.923368+0000","last_undegraded":"2024-03-06T07:25:06.923368+0000","last_fullsized":"2024-03-06T07:25:06.923368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924643+0000","last_change":"2024-03-06T07:25:06.924643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924643+0000","last_undegraded":"2024-03-06T07:25:06.924643+0000","last_fullsized":"2024-03-06T07:25:06.924643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924710+0000","last_change":"2024-03-06T07:25:06.924710+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924710+0000","last_undegraded":"2024-03-06T07:25:06.924710+0000","last_fullsized":"2024-03-06T07:25:06.924710+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922611+0000","last_change":"2024-03-06T07:25:06.922611+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922611+0000","last_undegraded":"2024-03-06T07:25:06.922611+0000","last_fullsized":"2024-03-06T07:25:06.922611+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922132+0000","last_change":"2024-03-06T07:25:06.922132+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922132+0000","last_undegraded":"2024-03-06T07:25:06.922132+0000","last_fullsized":"2024-03-06T07:25:06.922132+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927603+0000","last_change":"2024-03-06T07:25:06.927603+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927603+0000","last_undegraded":"2024-03-06T07:25:06.927603+0000","last_fullsized":"2024-03-06T07:25:06.927603+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918232+0000","last_change":"2024-03-06T07:25:06.918232+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918232+0000","last_undegraded":"2024-03-06T07:25:06.918232+0000","last_fullsized":"2024-03-06T07:25:06.918232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921643+0000","last_change":"2024-03-06T07:25:06.921643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921643+0000","last_undegraded":"2024-03-06T07:25:06.921643+0000","last_fullsized":"2024-03-06T07:25:06.921643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921166+0000","last_change":"2024-03-06T07:25:06.921166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921166+0000","last_undegraded":"2024-03-06T07:25:06.921166+0000","last_fullsized":"2024-03-06T07:25:06.921166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920314+0000","last_change":"2024-03-06T07:25:06.920314+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920314+0000","last_undegraded":"2024-03-06T07:25:06.920314+0000","last_fullsized":"2024-03-06T07:25:06.920314+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924221+0000","last_change":"2024-03-06T07:25:06.924221+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924221+0000","last_undegraded":"2024-03-06T07:25:06.924221+0000","last_fullsized":"2024-03-06T07:25:06.924221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921124+0000","last_change":"2024-03-06T07:25:06.921124+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921124+0000","last_undegraded":"2024-03-06T07:25:06.921124+0000","last_fullsized":"2024-03-06T07:25:06.921124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924524+0000","last_change":"2024-03-06T07:25:06.924524+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924524+0000","last_undegraded":"2024-03-06T07:25:06.924524+0000","last_fullsized":"2024-03-06T07:25:06.924524+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921656+0000","last_change":"2024-03-06T07:25:06.921656+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921656+0000","last_undegraded":"2024-03-06T07:25:06.921656+0000","last_fullsized":"2024-03-06T07:25:06.921656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921140+0000","last_change":"2024-03-06T07:25:06.921140+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921140+0000","last_undegraded":"2024-03-06T07:25:06.921140+0000","last_fullsized":"2024-03-06T07:25:06.921140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924794+0000","last_change":"2024-03-06T07:25:06.924794+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924794+0000","last_undegraded":"2024-03-06T07:25:06.924794+0000","last_fullsized":"2024-03-06T07:25:06.924794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918730+0000","last_change":"2024-03-06T07:25:06.918730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918730+0000","last_undegraded":"2024-03-06T07:25:06.918730+0000","last_fullsized":"2024-03-06T07:25:06.918730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918574+0000","last_change":"2024-03-06T07:25:06.918574+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918574+0000","last_undegraded":"2024-03-06T07:25:06.918574+0000","last_fullsized":"2024-03-06T07:25:06.918574+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920290+0000","last_change":"2024-03-06T07:25:06.920290+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920290+0000","last_undegraded":"2024-03-06T07:25:06.920290+0000","last_fullsized":"2024-03-06T07:25:06.920290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920527+0000","last_change":"2024-03-06T07:25:06.920527+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920527+0000","last_undegraded":"2024-03-06T07:25:06.920527+0000","last_fullsized":"2024-03-06T07:25:06.920527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.926454+0000","last_change":"2024-03-06T07:25:06.926454+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.926454+0000","last_undegraded":"2024-03-06T07:25:06.926454+0000","last_fullsized":"2024-03-06T07:25:06.926454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917006+0000","last_change":"2024-03-06T07:25:06.917006+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917006+0000","last_undegraded":"2024-03-06T07:25:06.917006+0000","last_fullsized":"2024-03-06T07:25:06.917006+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917762+0000","last_change":"2024-03-06T07:25:06.917762+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917762+0000","last_undegraded":"2024-03-06T07:25:06.917762+0000","last_fullsized":"2024-03-06T07:25:06.917762+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924716+0000","last_change":"2024-03-06T07:25:06.924716+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924716+0000","last_undegraded":"2024-03-06T07:25:06.924716+0000","last_fullsized":"2024-03-06T07:25:06.924716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925850+0000","last_change":"2024-03-06T07:25:06.925850+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925850+0000","last_undegraded":"2024-03-06T07:25:06.925850+0000","last_fullsized":"2024-03-06T07:25:06.925850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919602+0000","last_change":"2024-03-06T07:25:06.919602+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919602+0000","last_undegraded":"2024-03-06T07:25:06.919602+0000","last_fullsized":"2024-03-06T07:25:06.919602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918067+0000","last_change":"2024-03-06T07:25:06.918067+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918067+0000","last_undegraded":"2024-03-06T07:25:06.918067+0000","last_fullsized":"2024-03-06T07:25:06.918067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918954+0000","last_change":"2024-03-06T07:25:06.918954+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918954+0000","last_undegraded":"2024-03-06T07:25:06.918954+0000","last_fullsized":"2024-03-06T07:25:06.918954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918136+0000","last_change":"2024-03-06T07:25:06.918136+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918136+0000","last_undegraded":"2024-03-06T07:25:06.918136+0000","last_fullsized":"2024-03-06T07:25:06.918136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917781+0000","last_change":"2024-03-06T07:25:06.917781+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917781+0000","last_undegraded":"2024-03-06T07:25:06.917781+0000","last_fullsized":"2024-03-06T07:25:06.917781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922036+0000","last_change":"2024-03-06T07:25:06.922036+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922036+0000","last_undegraded":"2024-03-06T07:25:06.922036+0000","last_fullsized":"2024-03-06T07:25:06.922036+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.928472+0000","last_change":"2024-03-06T07:25:06.926918+0000","last_active":"2024-03-06T07:25:06.928472+0000","last_peered":"2024-03-06T07:25:06.928472+0000","last_clean":"2024-03-06T07:25:06.928472+0000","last_became_active":"2024-03-06T07:25:06.926475+0000","last_became_peered":"2024-03-06T07:25:06.926475+0000","last_unstale":"2024-03-06T07:25:06.928472+0000","last_undegraded":"2024-03-06T07:25:06.928472+0000","last_fullsized":"2024-03-06T07:25:06.928472+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:35.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924223+0000","last_change":"2024-03-06T07:25:06.924223+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924223+0000","last_undegraded":"2024-03-06T07:25:06.924223+0000","last_fullsized":"2024-03-06T07:25:06.924223+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937287+0000","last_change":"2024-03-06T07:25:06.933126+0000","last_active":"2024-03-06T07:25:06.937287+0000","last_peered":"2024-03-06T07:25:06.937287+0000","last_clean":"2024-03-06T07:25:06.937287+0000","last_became_active":"2024-03-06T07:25:06.932843+0000","last_became_peered":"2024-03-06T07:25:06.932843+0000","last_unstale":"2024-03-06T07:25:06.937287+0000","last_undegraded":"2024-03-06T07:25:06.937287+0000","last_fullsized":"2024-03-06T07:25:06.937287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:33.155673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916730+0000","last_change":"2024-03-06T07:25:06.916730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916730+0000","last_undegraded":"2024-03-06T07:25:06.916730+0000","last_fullsized":"2024-03-06T07:25:06.916730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.934052+0000","last_change":"2024-03-06T07:25:06.932116+0000","last_active":"2024-03-06T07:25:06.934052+0000","last_peered":"2024-03-06T07:25:06.934052+0000","last_clean":"2024-03-06T07:25:06.934052+0000","last_became_active":"2024-03-06T07:25:06.931835+0000","last_became_peered":"2024-03-06T07:25:06.931835+0000","last_unstale":"2024-03-06T07:25:06.934052+0000","last_undegraded":"2024-03-06T07:25:06.934052+0000","last_fullsized":"2024-03-06T07:25:06.934052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:52.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918299+0000","last_change":"2024-03-06T07:25:06.918299+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918299+0000","last_undegraded":"2024-03-06T07:25:06.918299+0000","last_fullsized":"2024-03-06T07:25:06.918299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937080+0000","last_change":"2024-03-06T07:25:06.936612+0000","last_active":"2024-03-06T07:25:06.937080+0000","last_peered":"2024-03-06T07:25:06.937080+0000","last_clean":"2024-03-06T07:25:06.937080+0000","last_became_active":"2024-03-06T07:25:06.936165+0000","last_became_peered":"2024-03-06T07:25:06.936165+0000","last_unstale":"2024-03-06T07:25:06.937080+0000","last_undegraded":"2024-03-06T07:25:06.937080+0000","last_fullsized":"2024-03-06T07:25:06.937080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:41.959433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920192+0000","last_change":"2024-03-06T07:25:06.920192+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920192+0000","last_undegraded":"2024-03-06T07:25:06.920192+0000","last_fullsized":"2024-03-06T07:25:06.920192+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937555+0000","last_change":"2024-03-06T07:25:06.932157+0000","last_active":"2024-03-06T07:25:06.937555+0000","last_peered":"2024-03-06T07:25:06.937555+0000","last_clean":"2024-03-06T07:25:06.937555+0000","last_became_active":"2024-03-06T07:25:06.931866+0000","last_became_peered":"2024-03-06T07:25:06.931866+0000","last_unstale":"2024-03-06T07:25:06.937555+0000","last_undegraded":"2024-03-06T07:25:06.937555+0000","last_fullsized":"2024-03-06T07:25:06.937555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.500674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925548+0000","last_change":"2024-03-06T07:25:06.925548+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925548+0000","last_undegraded":"2024-03-06T07:25:06.925548+0000","last_fullsized":"2024-03-06T07:25:06.925548+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933326+0000","last_change":"2024-03-06T07:25:06.932382+0000","last_active":"2024-03-06T07:25:06.933326+0000","last_peered":"2024-03-06T07:25:06.933326+0000","last_clean":"2024-03-06T07:25:06.933326+0000","last_became_active":"2024-03-06T07:25:06.932043+0000","last_became_peered":"2024-03-06T07:25:06.932043+0000","last_unstale":"2024-03-06T07:25:06.933326+0000","last_undegraded":"2024-03-06T07:25:06.933326+0000","last_fullsized":"2024-03-06T07:25:06.933326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:37.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921418+0000","last_change":"2024-03-06T07:25:06.921418+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921418+0000","last_undegraded":"2024-03-06T07:25:06.921418+0000","last_fullsized":"2024-03-06T07:25:06.921418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936978+0000","last_change":"2024-03-06T07:25:06.934761+0000","last_active":"2024-03-06T07:25:06.936978+0000","last_peered":"2024-03-06T07:25:06.936978+0000","last_clean":"2024-03-06T07:25:06.936978+0000","last_became_active":"2024-03-06T07:25:06.934441+0000","last_became_peered":"2024-03-06T07:25:06.934441+0000","last_unstale":"2024-03-06T07:25:06.936978+0000","last_undegraded":"2024-03-06T07:25:06.936978+0000","last_fullsized":"2024-03-06T07:25:06.936978+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:30.827701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927721+0000","last_change":"2024-03-06T07:25:06.927721+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927721+0000","last_undegraded":"2024-03-06T07:25:06.927721+0000","last_fullsized":"2024-03-06T07:25:06.927721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936449+0000","last_change":"2024-03-06T07:25:06.935857+0000","last_active":"2024-03-06T07:25:06.936449+0000","last_peered":"2024-03-06T07:25:06.936449+0000","last_clean":"2024-03-06T07:25:06.936449+0000","last_became_active":"2024-03-06T07:25:06.935520+0000","last_became_peered":"2024-03-06T07:25:06.935520+0000","last_unstale":"2024-03-06T07:25:06.936449+0000","last_undegraded":"2024-03-06T07:25:06.936449+0000","last_fullsized":"2024-03-06T07:25:06.936449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:41.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916666+0000","last_change":"2024-03-06T07:25:06.916666+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916666+0000","last_undegraded":"2024-03-06T07:25:06.916666+0000","last_fullsized":"2024-03-06T07:25:06.916666+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931892+0000","last_change":"2024-03-06T07:25:06.928957+0000","last_active":"2024-03-06T07:25:06.931892+0000","last_peered":"2024-03-06T07:25:06.931892+0000","last_clean":"2024-03-06T07:25:06.931892+0000","last_became_active":"2024-03-06T07:25:06.928262+0000","last_became_peered":"2024-03-06T07:25:06.928262+0000","last_unstale":"2024-03-06T07:25:06.931892+0000","last_undegraded":"2024-03-06T07:25:06.931892+0000","last_fullsized":"2024-03-06T07:25:06.931892+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:21.229011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919913+0000","last_change":"2024-03-06T07:25:06.919913+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919913+0000","last_undegraded":"2024-03-06T07:25:06.919913+0000","last_fullsized":"2024-03-06T07:25:06.919913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935245+0000","last_change":"2024-03-06T07:25:06.931781+0000","last_active":"2024-03-06T07:25:06.935245+0000","last_peered":"2024-03-06T07:25:06.935245+0000","last_clean":"2024-03-06T07:25:06.935245+0000","last_became_active":"2024-03-06T07:25:06.931456+0000","last_became_peered":"2024-03-06T07:25:06.931456+0000","last_unstale":"2024-03-06T07:25:06.935245+0000","last_undegraded":"2024-03-06T07:25:06.935245+0000","last_fullsized":"2024-03-06T07:25:06.935245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918286+0000","last_change":"2024-03-06T07:25:06.918286+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918286+0000","last_undegraded":"2024-03-06T07:25:06.918286+0000","last_fullsized":"2024-03-06T07:25:06.918286+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926217+0000","last_change":"2024-03-06T07:25:06.925504+0000","last_active":"2024-03-06T07:25:06.926217+0000","last_peered":"2024-03-06T07:25:06.926217+0000","last_clean":"2024-03-06T07:25:06.926217+0000","last_became_active":"2024-03-06T07:25:06.925180+0000","last_became_peered":"2024-03-06T07:25:06.925180+0000","last_unstale":"2024-03-06T07:25:06.926217+0000","last_undegraded":"2024-03-06T07:25:06.926217+0000","last_fullsized":"2024-03-06T07:25:06.926217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:19.573816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917364+0000","last_change":"2024-03-06T07:25:06.917364+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917364+0000","last_undegraded":"2024-03-06T07:25:06.917364+0000","last_fullsized":"2024-03-06T07:25:06.917364+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926965+0000","last_change":"2024-03-06T07:25:06.926155+0000","last_active":"2024-03-06T07:25:06.926965+0000","last_peered":"2024-03-06T07:25:06.926965+0000","last_clean":"2024-03-06T07:25:06.926965+0000","last_became_active":"2024-03-06T07:25:06.925662+0000","last_became_peered":"2024-03-06T07:25:06.925662+0000","last_unstale":"2024-03-06T07:25:06.926965+0000","last_undegraded":"2024-03-06T07:25:06.926965+0000","last_fullsized":"2024-03-06T07:25:06.926965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:36.511111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919303+0000","last_change":"2024-03-06T07:25:06.919303+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919303+0000","last_undegraded":"2024-03-06T07:25:06.919303+0000","last_fullsized":"2024-03-06T07:25:06.919303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942235+0000","last_change":"2024-03-06T07:25:06.936715+0000","last_active":"2024-03-06T07:25:06.942235+0000","last_peered":"2024-03-06T07:25:06.942235+0000","last_clean":"2024-03-06T07:25:06.942235+0000","last_became_active":"2024-03-06T07:25:06.936361+0000","last_became_peered":"2024-03-06T07:25:06.936361+0000","last_unstale":"2024-03-06T07:25:06.942235+0000","last_undegraded":"2024-03-06T07:25:06.942235+0000","last_fullsized":"2024-03-06T07:25:06.942235+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919086+0000","last_change":"2024-03-06T07:25:06.919086+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919086+0000","last_undegraded":"2024-03-06T07:25:06.919086+0000","last_fullsized":"2024-03-06T07:25:06.919086+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937421+0000","last_change":"2024-03-06T07:25:06.934580+0000","last_active":"2024-03-06T07:25:06.937421+0000","last_peered":"2024-03-06T07:25:06.937421+0000","last_clean":"2024-03-06T07:25:06.937421+0000","last_became_active":"2024-03-06T07:25:06.934296+0000","last_became_peered":"2024-03-06T07:25:06.934296+0000","last_unstale":"2024-03-06T07:25:06.937421+0000","last_undegraded":"2024-03-06T07:25:06.937421+0000","last_fullsized":"2024-03-06T07:25:06.937421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:32.439585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920493+0000","last_change":"2024-03-06T07:25:06.920493+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920493+0000","last_undegraded":"2024-03-06T07:25:06.920493+0000","last_fullsized":"2024-03-06T07:25:06.920493+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937088+0000","last_change":"2024-03-06T07:25:06.934527+0000","last_active":"2024-03-06T07:25:06.937088+0000","last_peered":"2024-03-06T07:25:06.937088+0000","last_clean":"2024-03-06T07:25:06.937088+0000","last_became_active":"2024-03-06T07:25:06.934249+0000","last_became_peered":"2024-03-06T07:25:06.934249+0000","last_unstale":"2024-03-06T07:25:06.937088+0000","last_undegraded":"2024-03-06T07:25:06.937088+0000","last_fullsized":"2024-03-06T07:25:06.937088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917750+0000","last_change":"2024-03-06T07:25:06.917750+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917750+0000","last_undegraded":"2024-03-06T07:25:06.917750+0000","last_fullsized":"2024-03-06T07:25:06.917750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.925249+0000","last_change":"2024-03-06T07:25:06.924175+0000","last_active":"2024-03-06T07:25:06.925249+0000","last_peered":"2024-03-06T07:25:06.925249+0000","last_clean":"2024-03-06T07:25:06.925249+0000","last_became_active":"2024-03-06T07:25:06.923894+0000","last_became_peered":"2024-03-06T07:25:06.923894+0000","last_unstale":"2024-03-06T07:25:06.925249+0000","last_undegraded":"2024-03-06T07:25:06.925249+0000","last_fullsized":"2024-03-06T07:25:06.925249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:34.739537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918166+0000","last_change":"2024-03-06T07:25:06.918166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918166+0000","last_undegraded":"2024-03-06T07:25:06.918166+0000","last_fullsized":"2024-03-06T07:25:06.918166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931792+0000","last_change":"2024-03-06T07:25:06.926576+0000","last_active":"2024-03-06T07:25:06.931792+0000","last_peered":"2024-03-06T07:25:06.931792+0000","last_clean":"2024-03-06T07:25:06.931792+0000","last_became_active":"2024-03-06T07:25:06.926315+0000","last_became_peered":"2024-03-06T07:25:06.926315+0000","last_unstale":"2024-03-06T07:25:06.931792+0000","last_undegraded":"2024-03-06T07:25:06.931792+0000","last_fullsized":"2024-03-06T07:25:06.931792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:38.818021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925302+0000","last_change":"2024-03-06T07:25:06.925302+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925302+0000","last_undegraded":"2024-03-06T07:25:06.925302+0000","last_fullsized":"2024-03-06T07:25:06.925302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935481+0000","last_change":"2024-03-06T07:25:06.935036+0000","last_active":"2024-03-06T07:25:06.935481+0000","last_peered":"2024-03-06T07:25:06.935481+0000","last_clean":"2024-03-06T07:25:06.935481+0000","last_became_active":"2024-03-06T07:25:06.934723+0000","last_became_peered":"2024-03-06T07:25:06.934723+0000","last_unstale":"2024-03-06T07:25:06.935481+0000","last_undegraded":"2024-03-06T07:25:06.935481+0000","last_fullsized":"2024-03-06T07:25:06.935481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:25.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924247+0000","last_change":"2024-03-06T07:25:06.924247+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924247+0000","last_undegraded":"2024-03-06T07:25:06.924247+0000","last_fullsized":"2024-03-06T07:25:06.924247+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942252+0000","last_change":"2024-03-06T07:25:06.937027+0000","last_active":"2024-03-06T07:25:06.942252+0000","last_peered":"2024-03-06T07:25:06.942252+0000","last_clean":"2024-03-06T07:25:06.942252+0000","last_became_active":"2024-03-06T07:25:06.936721+0000","last_became_peered":"2024-03-06T07:25:06.936721+0000","last_unstale":"2024-03-06T07:25:06.942252+0000","last_undegraded":"2024-03-06T07:25:06.942252+0000","last_fullsized":"2024-03-06T07:25:06.942252+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917215+0000","last_change":"2024-03-06T07:25:06.917215+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917215+0000","last_undegraded":"2024-03-06T07:25:06.917215+0000","last_fullsized":"2024-03-06T07:25:06.917215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932764+0000","last_change":"2024-03-06T07:25:06.930306+0000","last_active":"2024-03-06T07:25:06.932764+0000","last_peered":"2024-03-06T07:25:06.932764+0000","last_clean":"2024-03-06T07:25:06.932764+0000","last_became_active":"2024-03-06T07:25:06.929975+0000","last_became_peered":"2024-03-06T07:25:06.929975+0000","last_unstale":"2024-03-06T07:25:06.932764+0000","last_undegraded":"2024-03-06T07:25:06.932764+0000","last_fullsized":"2024-03-06T07:25:06.932764+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:14.435468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918737+0000","last_change":"2024-03-06T07:25:06.918737+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918737+0000","last_undegraded":"2024-03-06T07:25:06.918737+0000","last_fullsized":"2024-03-06T07:25:06.918737+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935878+0000","last_change":"2024-03-06T07:25:06.930143+0000","last_active":"2024-03-06T07:25:06.935878+0000","last_peered":"2024-03-06T07:25:06.935878+0000","last_clean":"2024-03-06T07:25:06.935878+0000","last_became_active":"2024-03-06T07:25:06.929880+0000","last_became_peered":"2024-03-06T07:25:06.929880+0000","last_unstale":"2024-03-06T07:25:06.935878+0000","last_undegraded":"2024-03-06T07:25:06.935878+0000","last_fullsized":"2024-03-06T07:25:06.935878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:51.483404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917337+0000","last_change":"2024-03-06T07:25:06.917337+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917337+0000","last_undegraded":"2024-03-06T07:25:06.917337+0000","last_fullsized":"2024-03-06T07:25:06.917337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932734+0000","last_change":"2024-03-06T07:25:06.930314+0000","last_active":"2024-03-06T07:25:06.932734+0000","last_peered":"2024-03-06T07:25:06.932734+0000","last_clean":"2024-03-06T07:25:06.932734+0000","last_became_active":"2024-03-06T07:25:06.929990+0000","last_became_peered":"2024-03-06T07:25:06.929990+0000","last_unstale":"2024-03-06T07:25:06.932734+0000","last_undegraded":"2024-03-06T07:25:06.932734+0000","last_fullsized":"2024-03-06T07:25:06.932734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:38.139179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932209+0000","last_change":"2024-03-06T07:25:06.925110+0000","last_active":"2024-03-06T07:25:06.932209+0000","last_peered":"2024-03-06T07:25:06.932209+0000","last_clean":"2024-03-06T07:25:06.932209+0000","last_became_active":"2024-03-06T07:25:06.924842+0000","last_became_peered":"2024-03-06T07:25:06.924842+0000","last_unstale":"2024-03-06T07:25:06.932209+0000","last_undegraded":"2024-03-06T07:25:06.932209+0000","last_fullsized":"2024-03-06T07:25:06.932209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:52.264094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919201+0000","last_change":"2024-03-06T07:25:06.919201+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919201+0000","last_undegraded":"2024-03-06T07:25:06.919201+0000","last_fullsized":"2024-03-06T07:25:06.919201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942224+0000","last_change":"2024-03-06T07:25:06.936235+0000","last_active":"2024-03-06T07:25:06.942224+0000","last_peered":"2024-03-06T07:25:06.942224+0000","last_clean":"2024-03-06T07:25:06.942224+0000","last_became_active":"2024-03-06T07:25:06.935934+0000","last_became_peered":"2024-03-06T07:25:06.935934+0000","last_unstale":"2024-03-06T07:25:06.942224+0000","last_undegraded":"2024-03-06T07:25:06.942224+0000","last_fullsized":"2024-03-06T07:25:06.942224+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925078+0000","last_change":"2024-03-06T07:25:06.925078+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925078+0000","last_undegraded":"2024-03-06T07:25:06.925078+0000","last_fullsized":"2024-03-06T07:25:06.925078+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935264+0000","last_change":"2024-03-06T07:25:06.933630+0000","last_active":"2024-03-06T07:25:06.935264+0000","last_peered":"2024-03-06T07:25:06.935264+0000","last_clean":"2024-03-06T07:25:06.935264+0000","last_became_active":"2024-03-06T07:25:06.933254+0000","last_became_peered":"2024-03-06T07:25:06.933254+0000","last_unstale":"2024-03-06T07:25:06.935264+0000","last_undegraded":"2024-03-06T07:25:06.935264+0000","last_fullsized":"2024-03-06T07:25:06.935264+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924327+0000","last_change":"2024-03-06T07:25:06.924327+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924327+0000","last_undegraded":"2024-03-06T07:25:06.924327+0000","last_fullsized":"2024-03-06T07:25:06.924327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936838+0000","last_change":"2024-03-06T07:25:06.936426+0000","last_active":"2024-03-06T07:25:06.936838+0000","last_peered":"2024-03-06T07:25:06.936838+0000","last_clean":"2024-03-06T07:25:06.936838+0000","last_became_active":"2024-03-06T07:25:06.936160+0000","last_became_peered":"2024-03-06T07:25:06.936160+0000","last_unstale":"2024-03-06T07:25:06.936838+0000","last_undegraded":"2024-03-06T07:25:06.936838+0000","last_fullsized":"2024-03-06T07:25:06.936838+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:39.491573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919378+0000","last_change":"2024-03-06T07:25:06.919378+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919378+0000","last_undegraded":"2024-03-06T07:25:06.919378+0000","last_fullsized":"2024-03-06T07:25:06.919378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932280+0000","last_change":"2024-03-06T07:25:06.930511+0000","last_active":"2024-03-06T07:25:06.932280+0000","last_peered":"2024-03-06T07:25:06.932280+0000","last_clean":"2024-03-06T07:25:06.932280+0000","last_became_active":"2024-03-06T07:25:06.930205+0000","last_became_peered":"2024-03-06T07:25:06.930205+0000","last_unstale":"2024-03-06T07:25:06.932280+0000","last_undegraded":"2024-03-06T07:25:06.932280+0000","last_fullsized":"2024-03-06T07:25:06.932280+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:47.054740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916851+0000","last_change":"2024-03-06T07:25:06.916851+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916851+0000","last_undegraded":"2024-03-06T07:25:06.916851+0000","last_fullsized":"2024-03-06T07:25:06.916851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936972+0000","last_change":"2024-03-06T07:25:06.935597+0000","last_active":"2024-03-06T07:25:06.936972+0000","last_peered":"2024-03-06T07:25:06.936972+0000","last_clean":"2024-03-06T07:25:06.936972+0000","last_became_active":"2024-03-06T07:25:06.935195+0000","last_became_peered":"2024-03-06T07:25:06.935195+0000","last_unstale":"2024-03-06T07:25:06.936972+0000","last_undegraded":"2024-03-06T07:25:06.936972+0000","last_fullsized":"2024-03-06T07:25:06.936972+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:50.432110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924317+0000","last_change":"2024-03-06T07:25:06.924317+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924317+0000","last_undegraded":"2024-03-06T07:25:06.924317+0000","last_fullsized":"2024-03-06T07:25:06.924317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932947+0000","last_change":"2024-03-06T07:25:06.931052+0000","last_active":"2024-03-06T07:25:06.932947+0000","last_peered":"2024-03-06T07:25:06.932947+0000","last_clean":"2024-03-06T07:25:06.932947+0000","last_became_active":"2024-03-06T07:25:06.930765+0000","last_became_peered":"2024-03-06T07:25:06.930765+0000","last_unstale":"2024-03-06T07:25:06.932947+0000","last_undegraded":"2024-03-06T07:25:06.932947+0000","last_fullsized":"2024-03-06T07:25:06.932947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:26.305654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920999+0000","last_change":"2024-03-06T07:25:06.920999+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920999+0000","last_undegraded":"2024-03-06T07:25:06.920999+0000","last_fullsized":"2024-03-06T07:25:06.920999+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937201+0000","last_change":"2024-03-06T07:25:06.935076+0000","last_active":"2024-03-06T07:25:06.937201+0000","last_peered":"2024-03-06T07:25:06.937201+0000","last_clean":"2024-03-06T07:25:06.937201+0000","last_became_active":"2024-03-06T07:25:06.934424+0000","last_became_peered":"2024-03-06T07:25:06.934424+0000","last_unstale":"2024-03-06T07:25:06.937201+0000","last_undegraded":"2024-03-06T07:25:06.937201+0000","last_fullsized":"2024-03-06T07:25:06.937201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:55.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919352+0000","last_change":"2024-03-06T07:25:06.919352+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919352+0000","last_undegraded":"2024-03-06T07:25:06.919352+0000","last_fullsized":"2024-03-06T07:25:06.919352+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931872+0000","last_change":"2024-03-06T07:25:06.927019+0000","last_active":"2024-03-06T07:25:06.931872+0000","last_peered":"2024-03-06T07:25:06.931872+0000","last_clean":"2024-03-06T07:25:06.931872+0000","last_became_active":"2024-03-06T07:25:06.926737+0000","last_became_peered":"2024-03-06T07:25:06.926737+0000","last_unstale":"2024-03-06T07:25:06.931872+0000","last_undegraded":"2024-03-06T07:25:06.931872+0000","last_fullsized":"2024-03-06T07:25:06.931872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:29.054350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916675+0000","last_change":"2024-03-06T07:25:06.916675+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916675+0000","last_undegraded":"2024-03-06T07:25:06.916675+0000","last_fullsized":"2024-03-06T07:25:06.916675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933731+0000","last_change":"2024-03-06T07:25:06.933005+0000","last_active":"2024-03-06T07:25:06.933731+0000","last_peered":"2024-03-06T07:25:06.933731+0000","last_clean":"2024-03-06T07:25:06.933731+0000","last_became_active":"2024-03-06T07:25:06.932668+0000","last_became_peered":"2024-03-06T07:25:06.932668+0000","last_unstale":"2024-03-06T07:25:06.933731+0000","last_undegraded":"2024-03-06T07:25:06.933731+0000","last_fullsized":"2024-03-06T07:25:06.933731+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:20.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.918569+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:06.918569+0000","last_peered":"2024-03-06T07:25:06.918569+0000","last_clean":"2024-03-06T07:25:06.918569+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:06.918569+0000","last_undegraded":"2024-03-06T07:25:06.918569+0000","last_fullsized":"2024-03-06T07:25:06.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:11.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:11.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 Processor -- start 2024-03-06T07:25:11.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 -- start start 2024-03-06T07:25:11.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f831c150290 0x7f831c150710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c156f90 con 0x7f831c0c3d60 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831c157270 con 0x7f831c072720 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.419+0000 7f83236aa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f831c0c38d0 con 0x7f831c150290 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8320c45640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f831c150290 0x7f831c150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8321c47640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f831c0c3d60 0x7f831c0c41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54248/0 (socket says 172.21.15.17:54248) 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8321c47640 1 -- 172.21.15.17:0/2289960673 learned_addr learned my addr 172.21.15.17:0/2289960673 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8320c45640 1 -- 172.21.15.17:0/2289960673 >> v1:172.21.15.163:6789/0 conn(0x7f831c072720 legacy=0x7f831c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8320c45640 1 -- 172.21.15.17:0/2289960673 >> v1:172.21.15.17:6789/0 conn(0x7f831c0c3d60 legacy=0x7f831c0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8313fff640 1 -- 172.21.15.17:0/2289960673 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088239744 0 0) 0x7f831c156f90 con 0x7f831c0c3d60 2024-03-06T07:25:11.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.420+0000 7f8320c45640 1 -- 172.21.15.17:0/2289960673 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0c3280 con 0x7f831c150290 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f8320c45640 1 --2- 172.21.15.17:0/2289960673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c150290 0x7f831c150710 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f830c0097a0 tx=0x7f830c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bef46acc455b5c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f8313fff640 1 -- 172.21.15.17:0/2289960673 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c035020 con 0x7f831c150290 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f8313fff640 1 -- 172.21.15.17:0/2289960673 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c031830 con 0x7f831c150290 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f8313fff640 1 -- 172.21.15.17:0/2289960673 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c031c00 con 0x7f831c150290 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 -- 172.21.15.17:0/2289960673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c150290 msgr2=0x7f831c150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 --2- 172.21.15.17:0/2289960673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c150290 0x7f831c150710 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f830c0097a0 tx=0x7f830c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 -- 172.21.15.17:0/2289960673 shutdown_connections 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 --2- 172.21.15.17:0/2289960673 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c150290 0x7f831c150710 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 -- 172.21.15.17:0/2289960673 >> 172.21.15.17:0/2289960673 conn(0x7f831c0aa630 msgr2=0x7f831c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 -- 172.21.15.17:0/2289960673 shutdown_connections 2024-03-06T07:25:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.421+0000 7f83236aa640 1 -- 172.21.15.17:0/2289960673 wait complete. 2024-03-06T07:25:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.422+0000 7f83236aa640 1 Processor -- start 2024-03-06T07:25:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.422+0000 7f83236aa640 1 -- start start 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f831c072720 0x7f831c0ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f831c150290 0x7f831c1ec610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c156bd0 con 0x7f831c0c3d60 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831c06f8f0 con 0x7f831c072720 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f83236aa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831c156d50 con 0x7f831c150290 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f8320c45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f8320c45640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54182/0 (socket says 172.21.15.17:54182) 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f8320c45640 1 -- 172.21.15.17:0/1197534854 learned_addr learned my addr 172.21.15.17:0/1197534854 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f8321c47640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f831c150290 0x7f831c1ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.423+0000 7f8321446640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f831c072720 0x7f831c0ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 -- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f831c150290 msgr2=0x7f831c1ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f831c150290 0x7f831c1ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 -- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f831c072720 msgr2=0x7f831c0ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f831c072720 0x7f831c0ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 -- 172.21.15.17:0/1197534854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c0ae9f0 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8320c45640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f830c034040 tx=0x7f830c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c041070 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c03a990 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c0aca90 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c035040 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.424+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831c0ae430 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.425+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831c04f330 con 0x7f831c0c3d60 2024-03-06T07:25:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.426+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f830c04c060 con 0x7f831c0c3d60 2024-03-06T07:25:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.426+0000 7f8311ffb640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 0x7f830006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.426+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f830c0abbe0 con 0x7f831c0c3d60 2024-03-06T07:25:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.426+0000 7f8321446640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 0x7f830006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.427+0000 7f8321446640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 0x7f830006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f831c1e68b0 tx=0x7f8318009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.430+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f830c07a310 con 0x7f831c0c3d60 2024-03-06T07:25:11.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.608+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f831c14c700 con 0x7f830006b4d0 2024-03-06T07:25:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.614+0000 7f8311ffb640 1 -- 172.21.15.17:0/1197534854 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247169 (secure 0 0 0) 0x7f831c14c700 con 0x7f830006b4d0 2024-03-06T07:25:11.618 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:11.619 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 msgr2=0x7f830006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 0x7f830006d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f831c1e68b0 tx=0x7f8318009040 comp rx=0 tx=0).stop 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 msgr2=0x7f831c1554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f830c034040 tx=0x7f830c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 shutdown_connections 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f830006b4d0 0x7f830006d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f831c150290 0x7f831c1ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f831c0c3d60 0x7f831c1554f0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 --2- 172.21.15.17:0/1197534854 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f831c072720 0x7f831c0ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 >> 172.21.15.17:0/1197534854 conn(0x7f831c0aa630 msgr2=0x7f831c0a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 shutdown_connections 2024-03-06T07:25:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.617+0000 7f83236aa640 1 -- 172.21.15.17:0/1197534854 wait complete. 2024-03-06T07:25:11.629 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-03-06T07:25:11.409677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262676,"kb_used_data":13220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712044,"statfs":{"total":773094113280,"available":772825133056,"internally_reserved":0,"allocated":13537280,"data_stored":7649345,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.501197"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925142+0000","last_change":"2024-03-06T07:25:06.925142+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925142+0000","last_undegraded":"2024-03-06T07:25:06.925142+0000","last_fullsized":"2024-03-06T07:25:06.925142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.923368+0000","last_change":"2024-03-06T07:25:06.923368+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.923368+0000","last_undegraded":"2024-03-06T07:25:06.923368+0000","last_fullsized":"2024-03-06T07:25:06.923368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924643+0000","last_change":"2024-03-06T07:25:06.924643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924643+0000","last_undegraded":"2024-03-06T07:25:06.924643+0000","last_fullsized":"2024-03-06T07:25:06.924643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924710+0000","last_change":"2024-03-06T07:25:06.924710+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924710+0000","last_undegraded":"2024-03-06T07:25:06.924710+0000","last_fullsized":"2024-03-06T07:25:06.924710+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922611+0000","last_change":"2024-03-06T07:25:06.922611+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922611+0000","last_undegraded":"2024-03-06T07:25:06.922611+0000","last_fullsized":"2024-03-06T07:25:06.922611+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922132+0000","last_change":"2024-03-06T07:25:06.922132+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922132+0000","last_undegraded":"2024-03-06T07:25:06.922132+0000","last_fullsized":"2024-03-06T07:25:06.922132+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927603+0000","last_change":"2024-03-06T07:25:06.927603+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927603+0000","last_undegraded":"2024-03-06T07:25:06.927603+0000","last_fullsized":"2024-03-06T07:25:06.927603+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918232+0000","last_change":"2024-03-06T07:25:06.918232+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918232+0000","last_undegraded":"2024-03-06T07:25:06.918232+0000","last_fullsized":"2024-03-06T07:25:06.918232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921643+0000","last_change":"2024-03-06T07:25:06.921643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921643+0000","last_undegraded":"2024-03-06T07:25:06.921643+0000","last_fullsized":"2024-03-06T07:25:06.921643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921166+0000","last_change":"2024-03-06T07:25:06.921166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921166+0000","last_undegraded":"2024-03-06T07:25:06.921166+0000","last_fullsized":"2024-03-06T07:25:06.921166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920314+0000","last_change":"2024-03-06T07:25:06.920314+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920314+0000","last_undegraded":"2024-03-06T07:25:06.920314+0000","last_fullsized":"2024-03-06T07:25:06.920314+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924221+0000","last_change":"2024-03-06T07:25:06.924221+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924221+0000","last_undegraded":"2024-03-06T07:25:06.924221+0000","last_fullsized":"2024-03-06T07:25:06.924221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921124+0000","last_change":"2024-03-06T07:25:06.921124+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921124+0000","last_undegraded":"2024-03-06T07:25:06.921124+0000","last_fullsized":"2024-03-06T07:25:06.921124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924524+0000","last_change":"2024-03-06T07:25:06.924524+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924524+0000","last_undegraded":"2024-03-06T07:25:06.924524+0000","last_fullsized":"2024-03-06T07:25:06.924524+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921656+0000","last_change":"2024-03-06T07:25:06.921656+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921656+0000","last_undegraded":"2024-03-06T07:25:06.921656+0000","last_fullsized":"2024-03-06T07:25:06.921656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921140+0000","last_change":"2024-03-06T07:25:06.921140+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921140+0000","last_undegraded":"2024-03-06T07:25:06.921140+0000","last_fullsized":"2024-03-06T07:25:06.921140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924794+0000","last_change":"2024-03-06T07:25:06.924794+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924794+0000","last_undegraded":"2024-03-06T07:25:06.924794+0000","last_fullsized":"2024-03-06T07:25:06.924794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918730+0000","last_change":"2024-03-06T07:25:06.918730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918730+0000","last_undegraded":"2024-03-06T07:25:06.918730+0000","last_fullsized":"2024-03-06T07:25:06.918730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918574+0000","last_change":"2024-03-06T07:25:06.918574+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918574+0000","last_undegraded":"2024-03-06T07:25:06.918574+0000","last_fullsized":"2024-03-06T07:25:06.918574+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920290+0000","last_change":"2024-03-06T07:25:06.920290+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920290+0000","last_undegraded":"2024-03-06T07:25:06.920290+0000","last_fullsized":"2024-03-06T07:25:06.920290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920527+0000","last_change":"2024-03-06T07:25:06.920527+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920527+0000","last_undegraded":"2024-03-06T07:25:06.920527+0000","last_fullsized":"2024-03-06T07:25:06.920527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.926454+0000","last_change":"2024-03-06T07:25:06.926454+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.926454+0000","last_undegraded":"2024-03-06T07:25:06.926454+0000","last_fullsized":"2024-03-06T07:25:06.926454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917006+0000","last_change":"2024-03-06T07:25:06.917006+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917006+0000","last_undegraded":"2024-03-06T07:25:06.917006+0000","last_fullsized":"2024-03-06T07:25:06.917006+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917762+0000","last_change":"2024-03-06T07:25:06.917762+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917762+0000","last_undegraded":"2024-03-06T07:25:06.917762+0000","last_fullsized":"2024-03-06T07:25:06.917762+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924716+0000","last_change":"2024-03-06T07:25:06.924716+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924716+0000","last_undegraded":"2024-03-06T07:25:06.924716+0000","last_fullsized":"2024-03-06T07:25:06.924716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925850+0000","last_change":"2024-03-06T07:25:06.925850+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925850+0000","last_undegraded":"2024-03-06T07:25:06.925850+0000","last_fullsized":"2024-03-06T07:25:06.925850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919602+0000","last_change":"2024-03-06T07:25:06.919602+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919602+0000","last_undegraded":"2024-03-06T07:25:06.919602+0000","last_fullsized":"2024-03-06T07:25:06.919602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918067+0000","last_change":"2024-03-06T07:25:06.918067+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918067+0000","last_undegraded":"2024-03-06T07:25:06.918067+0000","last_fullsized":"2024-03-06T07:25:06.918067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918954+0000","last_change":"2024-03-06T07:25:06.918954+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918954+0000","last_undegraded":"2024-03-06T07:25:06.918954+0000","last_fullsized":"2024-03-06T07:25:06.918954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918136+0000","last_change":"2024-03-06T07:25:06.918136+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918136+0000","last_undegraded":"2024-03-06T07:25:06.918136+0000","last_fullsized":"2024-03-06T07:25:06.918136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917781+0000","last_change":"2024-03-06T07:25:06.917781+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917781+0000","last_undegraded":"2024-03-06T07:25:06.917781+0000","last_fullsized":"2024-03-06T07:25:06.917781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922036+0000","last_change":"2024-03-06T07:25:06.922036+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922036+0000","last_undegraded":"2024-03-06T07:25:06.922036+0000","last_fullsized":"2024-03-06T07:25:06.922036+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.928472+0000","last_change":"2024-03-06T07:25:06.926918+0000","last_active":"2024-03-06T07:25:06.928472+0000","last_peered":"2024-03-06T07:25:06.928472+0000","last_clean":"2024-03-06T07:25:06.928472+0000","last_became_active":"2024-03-06T07:25:06.926475+0000","last_became_peered":"2024-03-06T07:25:06.926475+0000","last_unstale":"2024-03-06T07:25:06.928472+0000","last_undegraded":"2024-03-06T07:25:06.928472+0000","last_fullsized":"2024-03-06T07:25:06.928472+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:35.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924223+0000","last_change":"2024-03-06T07:25:06.924223+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924223+0000","last_undegraded":"2024-03-06T07:25:06.924223+0000","last_fullsized":"2024-03-06T07:25:06.924223+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937287+0000","last_change":"2024-03-06T07:25:06.933126+0000","last_active":"2024-03-06T07:25:06.937287+0000","last_peered":"2024-03-06T07:25:06.937287+0000","last_clean":"2024-03-06T07:25:06.937287+0000","last_became_active":"2024-03-06T07:25:06.932843+0000","last_became_peered":"2024-03-06T07:25:06.932843+0000","last_unstale":"2024-03-06T07:25:06.937287+0000","last_undegraded":"2024-03-06T07:25:06.937287+0000","last_fullsized":"2024-03-06T07:25:06.937287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:33.155673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916730+0000","last_change":"2024-03-06T07:25:06.916730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916730+0000","last_undegraded":"2024-03-06T07:25:06.916730+0000","last_fullsized":"2024-03-06T07:25:06.916730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.934052+0000","last_change":"2024-03-06T07:25:06.932116+0000","last_active":"2024-03-06T07:25:06.934052+0000","last_peered":"2024-03-06T07:25:06.934052+0000","last_clean":"2024-03-06T07:25:06.934052+0000","last_became_active":"2024-03-06T07:25:06.931835+0000","last_became_peered":"2024-03-06T07:25:06.931835+0000","last_unstale":"2024-03-06T07:25:06.934052+0000","last_undegraded":"2024-03-06T07:25:06.934052+0000","last_fullsized":"2024-03-06T07:25:06.934052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:52.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918299+0000","last_change":"2024-03-06T07:25:06.918299+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918299+0000","last_undegraded":"2024-03-06T07:25:06.918299+0000","last_fullsized":"2024-03-06T07:25:06.918299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937080+0000","last_change":"2024-03-06T07:25:06.936612+0000","last_active":"2024-03-06T07:25:06.937080+0000","last_peered":"2024-03-06T07:25:06.937080+0000","last_clean":"2024-03-06T07:25:06.937080+0000","last_became_active":"2024-03-06T07:25:06.936165+0000","last_became_peered":"2024-03-06T07:25:06.936165+0000","last_unstale":"2024-03-06T07:25:06.937080+0000","last_undegraded":"2024-03-06T07:25:06.937080+0000","last_fullsized":"2024-03-06T07:25:06.937080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:41.959433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920192+0000","last_change":"2024-03-06T07:25:06.920192+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920192+0000","last_undegraded":"2024-03-06T07:25:06.920192+0000","last_fullsized":"2024-03-06T07:25:06.920192+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937555+0000","last_change":"2024-03-06T07:25:06.932157+0000","last_active":"2024-03-06T07:25:06.937555+0000","last_peered":"2024-03-06T07:25:06.937555+0000","last_clean":"2024-03-06T07:25:06.937555+0000","last_became_active":"2024-03-06T07:25:06.931866+0000","last_became_peered":"2024-03-06T07:25:06.931866+0000","last_unstale":"2024-03-06T07:25:06.937555+0000","last_undegraded":"2024-03-06T07:25:06.937555+0000","last_fullsized":"2024-03-06T07:25:06.937555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.500674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925548+0000","last_change":"2024-03-06T07:25:06.925548+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925548+0000","last_undegraded":"2024-03-06T07:25:06.925548+0000","last_fullsized":"2024-03-06T07:25:06.925548+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933326+0000","last_change":"2024-03-06T07:25:06.932382+0000","last_active":"2024-03-06T07:25:06.933326+0000","last_peered":"2024-03-06T07:25:06.933326+0000","last_clean":"2024-03-06T07:25:06.933326+0000","last_became_active":"2024-03-06T07:25:06.932043+0000","last_became_peered":"2024-03-06T07:25:06.932043+0000","last_unstale":"2024-03-06T07:25:06.933326+0000","last_undegraded":"2024-03-06T07:25:06.933326+0000","last_fullsized":"2024-03-06T07:25:06.933326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:37.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921418+0000","last_change":"2024-03-06T07:25:06.921418+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921418+0000","last_undegraded":"2024-03-06T07:25:06.921418+0000","last_fullsized":"2024-03-06T07:25:06.921418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936978+0000","last_change":"2024-03-06T07:25:06.934761+0000","last_active":"2024-03-06T07:25:06.936978+0000","last_peered":"2024-03-06T07:25:06.936978+0000","last_clean":"2024-03-06T07:25:06.936978+0000","last_became_active":"2024-03-06T07:25:06.934441+0000","last_became_peered":"2024-03-06T07:25:06.934441+0000","last_unstale":"2024-03-06T07:25:06.936978+0000","last_undegraded":"2024-03-06T07:25:06.936978+0000","last_fullsized":"2024-03-06T07:25:06.936978+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:30.827701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927721+0000","last_change":"2024-03-06T07:25:06.927721+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927721+0000","last_undegraded":"2024-03-06T07:25:06.927721+0000","last_fullsized":"2024-03-06T07:25:06.927721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936449+0000","last_change":"2024-03-06T07:25:06.935857+0000","last_active":"2024-03-06T07:25:06.936449+0000","last_peered":"2024-03-06T07:25:06.936449+0000","last_clean":"2024-03-06T07:25:06.936449+0000","last_became_active":"2024-03-06T07:25:06.935520+0000","last_became_peered":"2024-03-06T07:25:06.935520+0000","last_unstale":"2024-03-06T07:25:06.936449+0000","last_undegraded":"2024-03-06T07:25:06.936449+0000","last_fullsized":"2024-03-06T07:25:06.936449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:41.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916666+0000","last_change":"2024-03-06T07:25:06.916666+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916666+0000","last_undegraded":"2024-03-06T07:25:06.916666+0000","last_fullsized":"2024-03-06T07:25:06.916666+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931892+0000","last_change":"2024-03-06T07:25:06.928957+0000","last_active":"2024-03-06T07:25:06.931892+0000","last_peered":"2024-03-06T07:25:06.931892+0000","last_clean":"2024-03-06T07:25:06.931892+0000","last_became_active":"2024-03-06T07:25:06.928262+0000","last_became_peered":"2024-03-06T07:25:06.928262+0000","last_unstale":"2024-03-06T07:25:06.931892+0000","last_undegraded":"2024-03-06T07:25:06.931892+0000","last_fullsized":"2024-03-06T07:25:06.931892+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:21.229011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919913+0000","last_change":"2024-03-06T07:25:06.919913+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919913+0000","last_undegraded":"2024-03-06T07:25:06.919913+0000","last_fullsized":"2024-03-06T07:25:06.919913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935245+0000","last_change":"2024-03-06T07:25:06.931781+0000","last_active":"2024-03-06T07:25:06.935245+0000","last_peered":"2024-03-06T07:25:06.935245+0000","last_clean":"2024-03-06T07:25:06.935245+0000","last_became_active":"2024-03-06T07:25:06.931456+0000","last_became_peered":"2024-03-06T07:25:06.931456+0000","last_unstale":"2024-03-06T07:25:06.935245+0000","last_undegraded":"2024-03-06T07:25:06.935245+0000","last_fullsized":"2024-03-06T07:25:06.935245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918286+0000","last_change":"2024-03-06T07:25:06.918286+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918286+0000","last_undegraded":"2024-03-06T07:25:06.918286+0000","last_fullsized":"2024-03-06T07:25:06.918286+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926217+0000","last_change":"2024-03-06T07:25:06.925504+0000","last_active":"2024-03-06T07:25:06.926217+0000","last_peered":"2024-03-06T07:25:06.926217+0000","last_clean":"2024-03-06T07:25:06.926217+0000","last_became_active":"2024-03-06T07:25:06.925180+0000","last_became_peered":"2024-03-06T07:25:06.925180+0000","last_unstale":"2024-03-06T07:25:06.926217+0000","last_undegraded":"2024-03-06T07:25:06.926217+0000","last_fullsized":"2024-03-06T07:25:06.926217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:19.573816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917364+0000","last_change":"2024-03-06T07:25:06.917364+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917364+0000","last_undegraded":"2024-03-06T07:25:06.917364+0000","last_fullsized":"2024-03-06T07:25:06.917364+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926965+0000","last_change":"2024-03-06T07:25:06.926155+0000","last_active":"2024-03-06T07:25:06.926965+0000","last_peered":"2024-03-06T07:25:06.926965+0000","last_clean":"2024-03-06T07:25:06.926965+0000","last_became_active":"2024-03-06T07:25:06.925662+0000","last_became_peered":"2024-03-06T07:25:06.925662+0000","last_unstale":"2024-03-06T07:25:06.926965+0000","last_undegraded":"2024-03-06T07:25:06.926965+0000","last_fullsized":"2024-03-06T07:25:06.926965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:36.511111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919303+0000","last_change":"2024-03-06T07:25:06.919303+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919303+0000","last_undegraded":"2024-03-06T07:25:06.919303+0000","last_fullsized":"2024-03-06T07:25:06.919303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942235+0000","last_change":"2024-03-06T07:25:06.936715+0000","last_active":"2024-03-06T07:25:06.942235+0000","last_peered":"2024-03-06T07:25:06.942235+0000","last_clean":"2024-03-06T07:25:06.942235+0000","last_became_active":"2024-03-06T07:25:06.936361+0000","last_became_peered":"2024-03-06T07:25:06.936361+0000","last_unstale":"2024-03-06T07:25:06.942235+0000","last_undegraded":"2024-03-06T07:25:06.942235+0000","last_fullsized":"2024-03-06T07:25:06.942235+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919086+0000","last_change":"2024-03-06T07:25:06.919086+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919086+0000","last_undegraded":"2024-03-06T07:25:06.919086+0000","last_fullsized":"2024-03-06T07:25:06.919086+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937421+0000","last_change":"2024-03-06T07:25:06.934580+0000","last_active":"2024-03-06T07:25:06.937421+0000","last_peered":"2024-03-06T07:25:06.937421+0000","last_clean":"2024-03-06T07:25:06.937421+0000","last_became_active":"2024-03-06T07:25:06.934296+0000","last_became_peered":"2024-03-06T07:25:06.934296+0000","last_unstale":"2024-03-06T07:25:06.937421+0000","last_undegraded":"2024-03-06T07:25:06.937421+0000","last_fullsized":"2024-03-06T07:25:06.937421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:32.439585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920493+0000","last_change":"2024-03-06T07:25:06.920493+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920493+0000","last_undegraded":"2024-03-06T07:25:06.920493+0000","last_fullsized":"2024-03-06T07:25:06.920493+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937088+0000","last_change":"2024-03-06T07:25:06.934527+0000","last_active":"2024-03-06T07:25:06.937088+0000","last_peered":"2024-03-06T07:25:06.937088+0000","last_clean":"2024-03-06T07:25:06.937088+0000","last_became_active":"2024-03-06T07:25:06.934249+0000","last_became_peered":"2024-03-06T07:25:06.934249+0000","last_unstale":"2024-03-06T07:25:06.937088+0000","last_undegraded":"2024-03-06T07:25:06.937088+0000","last_fullsized":"2024-03-06T07:25:06.937088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917750+0000","last_change":"2024-03-06T07:25:06.917750+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917750+0000","last_undegraded":"2024-03-06T07:25:06.917750+0000","last_fullsized":"2024-03-06T07:25:06.917750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.925249+0000","last_change":"2024-03-06T07:25:06.924175+0000","last_active":"2024-03-06T07:25:06.925249+0000","last_peered":"2024-03-06T07:25:06.925249+0000","last_clean":"2024-03-06T07:25:06.925249+0000","last_became_active":"2024-03-06T07:25:06.923894+0000","last_became_peered":"2024-03-06T07:25:06.923894+0000","last_unstale":"2024-03-06T07:25:06.925249+0000","last_undegraded":"2024-03-06T07:25:06.925249+0000","last_fullsized":"2024-03-06T07:25:06.925249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:34.739537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918166+0000","last_change":"2024-03-06T07:25:06.918166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918166+0000","last_undegraded":"2024-03-06T07:25:06.918166+0000","last_fullsized":"2024-03-06T07:25:06.918166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931792+0000","last_change":"2024-03-06T07:25:06.926576+0000","last_active":"2024-03-06T07:25:06.931792+0000","last_peered":"2024-03-06T07:25:06.931792+0000","last_clean":"2024-03-06T07:25:06.931792+0000","last_became_active":"2024-03-06T07:25:06.926315+0000","last_became_peered":"2024-03-06T07:25:06.926315+0000","last_unstale":"2024-03-06T07:25:06.931792+0000","last_undegraded":"2024-03-06T07:25:06.931792+0000","last_fullsized":"2024-03-06T07:25:06.931792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:38.818021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925302+0000","last_change":"2024-03-06T07:25:06.925302+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925302+0000","last_undegraded":"2024-03-06T07:25:06.925302+0000","last_fullsized":"2024-03-06T07:25:06.925302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935481+0000","last_change":"2024-03-06T07:25:06.935036+0000","last_active":"2024-03-06T07:25:06.935481+0000","last_peered":"2024-03-06T07:25:06.935481+0000","last_clean":"2024-03-06T07:25:06.935481+0000","last_became_active":"2024-03-06T07:25:06.934723+0000","last_became_peered":"2024-03-06T07:25:06.934723+0000","last_unstale":"2024-03-06T07:25:06.935481+0000","last_undegraded":"2024-03-06T07:25:06.935481+0000","last_fullsized":"2024-03-06T07:25:06.935481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:25.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924247+0000","last_change":"2024-03-06T07:25:06.924247+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924247+0000","last_undegraded":"2024-03-06T07:25:06.924247+0000","last_fullsized":"2024-03-06T07:25:06.924247+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942252+0000","last_change":"2024-03-06T07:25:06.937027+0000","last_active":"2024-03-06T07:25:06.942252+0000","last_peered":"2024-03-06T07:25:06.942252+0000","last_clean":"2024-03-06T07:25:06.942252+0000","last_became_active":"2024-03-06T07:25:06.936721+0000","last_became_peered":"2024-03-06T07:25:06.936721+0000","last_unstale":"2024-03-06T07:25:06.942252+0000","last_undegraded":"2024-03-06T07:25:06.942252+0000","last_fullsized":"2024-03-06T07:25:06.942252+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917215+0000","last_change":"2024-03-06T07:25:06.917215+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917215+0000","last_undegraded":"2024-03-06T07:25:06.917215+0000","last_fullsized":"2024-03-06T07:25:06.917215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932764+0000","last_change":"2024-03-06T07:25:06.930306+0000","last_active":"2024-03-06T07:25:06.932764+0000","last_peered":"2024-03-06T07:25:06.932764+0000","last_clean":"2024-03-06T07:25:06.932764+0000","last_became_active":"2024-03-06T07:25:06.929975+0000","last_became_peered":"2024-03-06T07:25:06.929975+0000","last_unstale":"2024-03-06T07:25:06.932764+0000","last_undegraded":"2024-03-06T07:25:06.932764+0000","last_fullsized":"2024-03-06T07:25:06.932764+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:14.435468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918737+0000","last_change":"2024-03-06T07:25:06.918737+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918737+0000","last_undegraded":"2024-03-06T07:25:06.918737+0000","last_fullsized":"2024-03-06T07:25:06.918737+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935878+0000","last_change":"2024-03-06T07:25:06.930143+0000","last_active":"2024-03-06T07:25:06.935878+0000","last_peered":"2024-03-06T07:25:06.935878+0000","last_clean":"2024-03-06T07:25:06.935878+0000","last_became_active":"2024-03-06T07:25:06.929880+0000","last_became_peered":"2024-03-06T07:25:06.929880+0000","last_unstale":"2024-03-06T07:25:06.935878+0000","last_undegraded":"2024-03-06T07:25:06.935878+0000","last_fullsized":"2024-03-06T07:25:06.935878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:51.483404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917337+0000","last_change":"2024-03-06T07:25:06.917337+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917337+0000","last_undegraded":"2024-03-06T07:25:06.917337+0000","last_fullsized":"2024-03-06T07:25:06.917337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932734+0000","last_change":"2024-03-06T07:25:06.930314+0000","last_active":"2024-03-06T07:25:06.932734+0000","last_peered":"2024-03-06T07:25:06.932734+0000","last_clean":"2024-03-06T07:25:06.932734+0000","last_became_active":"2024-03-06T07:25:06.929990+0000","last_became_peered":"2024-03-06T07:25:06.929990+0000","last_unstale":"2024-03-06T07:25:06.932734+0000","last_undegraded":"2024-03-06T07:25:06.932734+0000","last_fullsized":"2024-03-06T07:25:06.932734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:38.139179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932209+0000","last_change":"2024-03-06T07:25:06.925110+0000","last_active":"2024-03-06T07:25:06.932209+0000","last_peered":"2024-03-06T07:25:06.932209+0000","last_clean":"2024-03-06T07:25:06.932209+0000","last_became_active":"2024-03-06T07:25:06.924842+0000","last_became_peered":"2024-03-06T07:25:06.924842+0000","last_unstale":"2024-03-06T07:25:06.932209+0000","last_undegraded":"2024-03-06T07:25:06.932209+0000","last_fullsized":"2024-03-06T07:25:06.932209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:52.264094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919201+0000","last_change":"2024-03-06T07:25:06.919201+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919201+0000","last_undegraded":"2024-03-06T07:25:06.919201+0000","last_fullsized":"2024-03-06T07:25:06.919201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942224+0000","last_change":"2024-03-06T07:25:06.936235+0000","last_active":"2024-03-06T07:25:06.942224+0000","last_peered":"2024-03-06T07:25:06.942224+0000","last_clean":"2024-03-06T07:25:06.942224+0000","last_became_active":"2024-03-06T07:25:06.935934+0000","last_became_peered":"2024-03-06T07:25:06.935934+0000","last_unstale":"2024-03-06T07:25:06.942224+0000","last_undegraded":"2024-03-06T07:25:06.942224+0000","last_fullsized":"2024-03-06T07:25:06.942224+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925078+0000","last_change":"2024-03-06T07:25:06.925078+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925078+0000","last_undegraded":"2024-03-06T07:25:06.925078+0000","last_fullsized":"2024-03-06T07:25:06.925078+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935264+0000","last_change":"2024-03-06T07:25:06.933630+0000","last_active":"2024-03-06T07:25:06.935264+0000","last_peered":"2024-03-06T07:25:06.935264+0000","last_clean":"2024-03-06T07:25:06.935264+0000","last_became_active":"2024-03-06T07:25:06.933254+0000","last_became_peered":"2024-03-06T07:25:06.933254+0000","last_unstale":"2024-03-06T07:25:06.935264+0000","last_undegraded":"2024-03-06T07:25:06.935264+0000","last_fullsized":"2024-03-06T07:25:06.935264+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924327+0000","last_change":"2024-03-06T07:25:06.924327+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924327+0000","last_undegraded":"2024-03-06T07:25:06.924327+0000","last_fullsized":"2024-03-06T07:25:06.924327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936838+0000","last_change":"2024-03-06T07:25:06.936426+0000","last_active":"2024-03-06T07:25:06.936838+0000","last_peered":"2024-03-06T07:25:06.936838+0000","last_clean":"2024-03-06T07:25:06.936838+0000","last_became_active":"2024-03-06T07:25:06.936160+0000","last_became_peered":"2024-03-06T07:25:06.936160+0000","last_unstale":"2024-03-06T07:25:06.936838+0000","last_undegraded":"2024-03-06T07:25:06.936838+0000","last_fullsized":"2024-03-06T07:25:06.936838+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:39.491573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919378+0000","last_change":"2024-03-06T07:25:06.919378+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919378+0000","last_undegraded":"2024-03-06T07:25:06.919378+0000","last_fullsized":"2024-03-06T07:25:06.919378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932280+0000","last_change":"2024-03-06T07:25:06.930511+0000","last_active":"2024-03-06T07:25:06.932280+0000","last_peered":"2024-03-06T07:25:06.932280+0000","last_clean":"2024-03-06T07:25:06.932280+0000","last_became_active":"2024-03-06T07:25:06.930205+0000","last_became_peered":"2024-03-06T07:25:06.930205+0000","last_unstale":"2024-03-06T07:25:06.932280+0000","last_undegraded":"2024-03-06T07:25:06.932280+0000","last_fullsized":"2024-03-06T07:25:06.932280+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:47.054740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916851+0000","last_change":"2024-03-06T07:25:06.916851+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916851+0000","last_undegraded":"2024-03-06T07:25:06.916851+0000","last_fullsized":"2024-03-06T07:25:06.916851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936972+0000","last_change":"2024-03-06T07:25:06.935597+0000","last_active":"2024-03-06T07:25:06.936972+0000","last_peered":"2024-03-06T07:25:06.936972+0000","last_clean":"2024-03-06T07:25:06.936972+0000","last_became_active":"2024-03-06T07:25:06.935195+0000","last_became_peered":"2024-03-06T07:25:06.935195+0000","last_unstale":"2024-03-06T07:25:06.936972+0000","last_undegraded":"2024-03-06T07:25:06.936972+0000","last_fullsized":"2024-03-06T07:25:06.936972+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:50.432110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924317+0000","last_change":"2024-03-06T07:25:06.924317+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924317+0000","last_undegraded":"2024-03-06T07:25:06.924317+0000","last_fullsized":"2024-03-06T07:25:06.924317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932947+0000","last_change":"2024-03-06T07:25:06.931052+0000","last_active":"2024-03-06T07:25:06.932947+0000","last_peered":"2024-03-06T07:25:06.932947+0000","last_clean":"2024-03-06T07:25:06.932947+0000","last_became_active":"2024-03-06T07:25:06.930765+0000","last_became_peered":"2024-03-06T07:25:06.930765+0000","last_unstale":"2024-03-06T07:25:06.932947+0000","last_undegraded":"2024-03-06T07:25:06.932947+0000","last_fullsized":"2024-03-06T07:25:06.932947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:26.305654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920999+0000","last_change":"2024-03-06T07:25:06.920999+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920999+0000","last_undegraded":"2024-03-06T07:25:06.920999+0000","last_fullsized":"2024-03-06T07:25:06.920999+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937201+0000","last_change":"2024-03-06T07:25:06.935076+0000","last_active":"2024-03-06T07:25:06.937201+0000","last_peered":"2024-03-06T07:25:06.937201+0000","last_clean":"2024-03-06T07:25:06.937201+0000","last_became_active":"2024-03-06T07:25:06.934424+0000","last_became_peered":"2024-03-06T07:25:06.934424+0000","last_unstale":"2024-03-06T07:25:06.937201+0000","last_undegraded":"2024-03-06T07:25:06.937201+0000","last_fullsized":"2024-03-06T07:25:06.937201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:55.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919352+0000","last_change":"2024-03-06T07:25:06.919352+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919352+0000","last_undegraded":"2024-03-06T07:25:06.919352+0000","last_fullsized":"2024-03-06T07:25:06.919352+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931872+0000","last_change":"2024-03-06T07:25:06.927019+0000","last_active":"2024-03-06T07:25:06.931872+0000","last_peered":"2024-03-06T07:25:06.931872+0000","last_clean":"2024-03-06T07:25:06.931872+0000","last_became_active":"2024-03-06T07:25:06.926737+0000","last_became_peered":"2024-03-06T07:25:06.926737+0000","last_unstale":"2024-03-06T07:25:06.931872+0000","last_undegraded":"2024-03-06T07:25:06.931872+0000","last_fullsized":"2024-03-06T07:25:06.931872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:29.054350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916675+0000","last_change":"2024-03-06T07:25:06.916675+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916675+0000","last_undegraded":"2024-03-06T07:25:06.916675+0000","last_fullsized":"2024-03-06T07:25:06.916675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933731+0000","last_change":"2024-03-06T07:25:06.933005+0000","last_active":"2024-03-06T07:25:06.933731+0000","last_peered":"2024-03-06T07:25:06.933731+0000","last_clean":"2024-03-06T07:25:06.933731+0000","last_became_active":"2024-03-06T07:25:06.932668+0000","last_became_peered":"2024-03-06T07:25:06.932668+0000","last_unstale":"2024-03-06T07:25:06.933731+0000","last_undegraded":"2024-03-06T07:25:06.933731+0000","last_fullsized":"2024-03-06T07:25:06.933731+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:20.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.918569+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:06.918569+0000","last_peered":"2024-03-06T07:25:06.918569+0000","last_clean":"2024-03-06T07:25:06.918569+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:06.918569+0000","last_undegraded":"2024-03-06T07:25:06.918569+0000","last_fullsized":"2024-03-06T07:25:06.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:11.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2024-03-06T07:25:11.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.925142+0000', 'last_change': '2024-03-06T07:25:06.925142+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.925142+0000', 'last_undegraded': '2024-03-06T07:25:06.925142+0000', 'last_fullsized': '2024-03-06T07:25:06.925142+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.923368+0000', 'last_change': '2024-03-06T07:25:06.923368+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.923368+0000', 'last_undegraded': '2024-03-06T07:25:06.923368+0000', 'last_fullsized': '2024-03-06T07:25:06.923368+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924643+0000', 'last_change': '2024-03-06T07:25:06.924643+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924643+0000', 'last_undegraded': '2024-03-06T07:25:06.924643+0000', 'last_fullsized': '2024-03-06T07:25:06.924643+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924710+0000', 'last_change': '2024-03-06T07:25:06.924710+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924710+0000', 'last_undegraded': '2024-03-06T07:25:06.924710+0000', 'last_fullsized': '2024-03-06T07:25:06.924710+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.922611+0000', 'last_change': '2024-03-06T07:25:06.922611+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.922611+0000', 'last_undegraded': '2024-03-06T07:25:06.922611+0000', 'last_fullsized': '2024-03-06T07:25:06.922611+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2024-03-06T07:25:11.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.922132+0000', 'last_change': '2024-03-06T07:25:06.922132+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.922132+0000', 'last_undegraded': '2024-03-06T07:25:06.922132+0000', 'last_fullsized': '2024-03-06T07:25:06.922132+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.927603+0000', 'last_change': '2024-03-06T07:25:06.927603+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.927603+0000', 'last_undegraded': '2024-03-06T07:25:06.927603+0000', 'last_fullsized': '2024-03-06T07:25:06.927603+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918232+0000', 'last_change': '2024-03-06T07:25:06.918232+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918232+0000', 'last_undegraded': '2024-03-06T07:25:06.918232+0000', 'last_fullsized': '2024-03-06T07:25:06.918232+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921643+0000', 'last_change': '2024-03-06T07:25:06.921643+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921643+0000', 'last_undegraded': '2024-03-06T07:25:06.921643+0000', 'last_fullsized': '2024-03-06T07:25:06.921643+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921166+0000', 'last_change': '2024-03-06T07:25:06.921166+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921166+0000', 'last_undegraded': '2024-03-06T07:25:06.921166+0000', 'last_fullsized': '2024-03-06T07:25:06.921166+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920314+0000', 'last_change': '2024-03-06T07:25:06.920314+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920314+0000', 'last_undegraded': '2024-03-06T07:25:06.920314+0000', 'last_fullsized': '2024-03-06T07:25:06.920314+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924221+0000', 'last_change': '2024-03-06T07:25:06.924221+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924221+0000', 'last_undegraded': '2024-03-06T07:25:06.924221+0000', 'last_fullsized': '2024-03-06T07:25:06.924221+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921124+0000', 'last_change': '2024-03-06T07:25:06.921124+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921124+0000', 'last_undegraded': '2024-03-06T07:25:06.921124+0000', 'last_fullsized': '2024-03-06T07:25:06.921124+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924524+0000', 'last_change': '2024-03-06T07:25:06.924524+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924524+0000', 'last_undegraded': '2024-03-06T07:25:06.924524+0000', 'last_fullsized': '2024-03-06T07:25:06.924524+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921656+0000', 'last_change': '2024-03-06T07:25:06.921656+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921656+0000', 'last_undegraded': '2024-03-06T07:25:06.921656+0000', 'last_fullsized': '2024-03-06T07:25:06.921656+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2024-03-06T07:25:11.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921140+0000', 'last_change': '2024-03-06T07:25:06.921140+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921140+0000', 'last_undegraded': '2024-03-06T07:25:06.921140+0000', 'last_fullsized': '2024-03-06T07:25:06.921140+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924794+0000', 'last_change': '2024-03-06T07:25:06.924794+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924794+0000', 'last_undegraded': '2024-03-06T07:25:06.924794+0000', 'last_fullsized': '2024-03-06T07:25:06.924794+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918730+0000', 'last_change': '2024-03-06T07:25:06.918730+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918730+0000', 'last_undegraded': '2024-03-06T07:25:06.918730+0000', 'last_fullsized': '2024-03-06T07:25:06.918730+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918574+0000', 'last_change': '2024-03-06T07:25:06.918574+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918574+0000', 'last_undegraded': '2024-03-06T07:25:06.918574+0000', 'last_fullsized': '2024-03-06T07:25:06.918574+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920290+0000', 'last_change': '2024-03-06T07:25:06.920290+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920290+0000', 'last_undegraded': '2024-03-06T07:25:06.920290+0000', 'last_fullsized': '2024-03-06T07:25:06.920290+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920527+0000', 'last_change': '2024-03-06T07:25:06.920527+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920527+0000', 'last_undegraded': '2024-03-06T07:25:06.920527+0000', 'last_fullsized': '2024-03-06T07:25:06.920527+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2024-03-06T07:25:11.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.926454+0000', 'last_change': '2024-03-06T07:25:06.926454+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.926454+0000', 'last_undegraded': '2024-03-06T07:25:06.926454+0000', 'last_fullsized': '2024-03-06T07:25:06.926454+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917006+0000', 'last_change': '2024-03-06T07:25:06.917006+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917006+0000', 'last_undegraded': '2024-03-06T07:25:06.917006+0000', 'last_fullsized': '2024-03-06T07:25:06.917006+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917762+0000', 'last_change': '2024-03-06T07:25:06.917762+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917762+0000', 'last_undegraded': '2024-03-06T07:25:06.917762+0000', 'last_fullsized': '2024-03-06T07:25:06.917762+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924716+0000', 'last_change': '2024-03-06T07:25:06.924716+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924716+0000', 'last_undegraded': '2024-03-06T07:25:06.924716+0000', 'last_fullsized': '2024-03-06T07:25:06.924716+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2024-03-06T07:25:11.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.925850+0000', 'last_change': '2024-03-06T07:25:06.925850+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.925850+0000', 'last_undegraded': '2024-03-06T07:25:06.925850+0000', 'last_fullsized': '2024-03-06T07:25:06.925850+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919602+0000', 'last_change': '2024-03-06T07:25:06.919602+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919602+0000', 'last_undegraded': '2024-03-06T07:25:06.919602+0000', 'last_fullsized': '2024-03-06T07:25:06.919602+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918067+0000', 'last_change': '2024-03-06T07:25:06.918067+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918067+0000', 'last_undegraded': '2024-03-06T07:25:06.918067+0000', 'last_fullsized': '2024-03-06T07:25:06.918067+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918954+0000', 'last_change': '2024-03-06T07:25:06.918954+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918954+0000', 'last_undegraded': '2024-03-06T07:25:06.918954+0000', 'last_fullsized': '2024-03-06T07:25:06.918954+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918136+0000', 'last_change': '2024-03-06T07:25:06.918136+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918136+0000', 'last_undegraded': '2024-03-06T07:25:06.918136+0000', 'last_fullsized': '2024-03-06T07:25:06.918136+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920031+0000', 'last_change': '2024-03-06T07:25:06.920031+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920031+0000', 'last_undegraded': '2024-03-06T07:25:06.920031+0000', 'last_fullsized': '2024-03-06T07:25:06.920031+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2024-03-06T07:25:11.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917781+0000', 'last_change': '2024-03-06T07:25:06.917781+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917781+0000', 'last_undegraded': '2024-03-06T07:25:06.917781+0000', 'last_fullsized': '2024-03-06T07:25:06.917781+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.922036+0000', 'last_change': '2024-03-06T07:25:06.922036+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.922036+0000', 'last_undegraded': '2024-03-06T07:25:06.922036+0000', 'last_fullsized': '2024-03-06T07:25:06.922036+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924223+0000', 'last_change': '2024-03-06T07:25:06.924223+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924223+0000', 'last_undegraded': '2024-03-06T07:25:06.924223+0000', 'last_fullsized': '2024-03-06T07:25:06.924223+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.916730+0000', 'last_change': '2024-03-06T07:25:06.916730+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.916730+0000', 'last_undegraded': '2024-03-06T07:25:06.916730+0000', 'last_fullsized': '2024-03-06T07:25:06.916730+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2024-03-06T07:25:11.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918299+0000', 'last_change': '2024-03-06T07:25:06.918299+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918299+0000', 'last_undegraded': '2024-03-06T07:25:06.918299+0000', 'last_fullsized': '2024-03-06T07:25:06.918299+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920192+0000', 'last_change': '2024-03-06T07:25:06.920192+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920192+0000', 'last_undegraded': '2024-03-06T07:25:06.920192+0000', 'last_fullsized': '2024-03-06T07:25:06.920192+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.925548+0000', 'last_change': '2024-03-06T07:25:06.925548+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.925548+0000', 'last_undegraded': '2024-03-06T07:25:06.925548+0000', 'last_fullsized': '2024-03-06T07:25:06.925548+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.921418+0000', 'last_change': '2024-03-06T07:25:06.921418+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.921418+0000', 'last_undegraded': '2024-03-06T07:25:06.921418+0000', 'last_fullsized': '2024-03-06T07:25:06.921418+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.927721+0000', 'last_change': '2024-03-06T07:25:06.927721+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.927721+0000', 'last_undegraded': '2024-03-06T07:25:06.927721+0000', 'last_fullsized': '2024-03-06T07:25:06.927721+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.916666+0000', 'last_change': '2024-03-06T07:25:06.916666+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.916666+0000', 'last_undegraded': '2024-03-06T07:25:06.916666+0000', 'last_fullsized': '2024-03-06T07:25:06.916666+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919913+0000', 'last_change': '2024-03-06T07:25:06.919913+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919913+0000', 'last_undegraded': '2024-03-06T07:25:06.919913+0000', 'last_fullsized': '2024-03-06T07:25:06.919913+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918286+0000', 'last_change': '2024-03-06T07:25:06.918286+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918286+0000', 'last_undegraded': '2024-03-06T07:25:06.918286+0000', 'last_fullsized': '2024-03-06T07:25:06.918286+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917364+0000', 'last_change': '2024-03-06T07:25:06.917364+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917364+0000', 'last_undegraded': '2024-03-06T07:25:06.917364+0000', 'last_fullsized': '2024-03-06T07:25:06.917364+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919303+0000', 'last_change': '2024-03-06T07:25:06.919303+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919303+0000', 'last_undegraded': '2024-03-06T07:25:06.919303+0000', 'last_fullsized': '2024-03-06T07:25:06.919303+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919086+0000', 'last_change': '2024-03-06T07:25:06.919086+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919086+0000', 'last_undegraded': '2024-03-06T07:25:06.919086+0000', 'last_fullsized': '2024-03-06T07:25:06.919086+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920493+0000', 'last_change': '2024-03-06T07:25:06.920493+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920493+0000', 'last_undegraded': '2024-03-06T07:25:06.920493+0000', 'last_fullsized': '2024-03-06T07:25:06.920493+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917750+0000', 'last_change': '2024-03-06T07:25:06.917750+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917750+0000', 'last_undegraded': '2024-03-06T07:25:06.917750+0000', 'last_fullsized': '2024-03-06T07:25:06.917750+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918166+0000', 'last_change': '2024-03-06T07:25:06.918166+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918166+0000', 'last_undegraded': '2024-03-06T07:25:06.918166+0000', 'last_fullsized': '2024-03-06T07:25:06.918166+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.925302+0000', 'last_change': '2024-03-06T07:25:06.925302+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.925302+0000', 'last_undegraded': '2024-03-06T07:25:06.925302+0000', 'last_fullsized': '2024-03-06T07:25:06.925302+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924247+0000', 'last_change': '2024-03-06T07:25:06.924247+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924247+0000', 'last_undegraded': '2024-03-06T07:25:06.924247+0000', 'last_fullsized': '2024-03-06T07:25:06.924247+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2024-03-06T07:25:11.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917215+0000', 'last_change': '2024-03-06T07:25:06.917215+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917215+0000', 'last_undegraded': '2024-03-06T07:25:06.917215+0000', 'last_fullsized': '2024-03-06T07:25:06.917215+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.918737+0000', 'last_change': '2024-03-06T07:25:06.918737+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.918737+0000', 'last_undegraded': '2024-03-06T07:25:06.918737+0000', 'last_fullsized': '2024-03-06T07:25:06.918737+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.917337+0000', 'last_change': '2024-03-06T07:25:06.917337+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.917337+0000', 'last_undegraded': '2024-03-06T07:25:06.917337+0000', 'last_fullsized': '2024-03-06T07:25:06.917337+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920031+0000', 'last_change': '2024-03-06T07:25:06.920031+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920031+0000', 'last_undegraded': '2024-03-06T07:25:06.920031+0000', 'last_fullsized': '2024-03-06T07:25:06.920031+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919201+0000', 'last_change': '2024-03-06T07:25:06.919201+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919201+0000', 'last_undegraded': '2024-03-06T07:25:06.919201+0000', 'last_fullsized': '2024-03-06T07:25:06.919201+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2024-03-06T07:25:11.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.925078+0000', 'last_change': '2024-03-06T07:25:06.925078+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.925078+0000', 'last_undegraded': '2024-03-06T07:25:06.925078+0000', 'last_fullsized': '2024-03-06T07:25:06.925078+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924327+0000', 'last_change': '2024-03-06T07:25:06.924327+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924327+0000', 'last_undegraded': '2024-03-06T07:25:06.924327+0000', 'last_fullsized': '2024-03-06T07:25:06.924327+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919378+0000', 'last_change': '2024-03-06T07:25:06.919378+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919378+0000', 'last_undegraded': '2024-03-06T07:25:06.919378+0000', 'last_fullsized': '2024-03-06T07:25:06.919378+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.916851+0000', 'last_change': '2024-03-06T07:25:06.916851+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.916851+0000', 'last_undegraded': '2024-03-06T07:25:06.916851+0000', 'last_fullsized': '2024-03-06T07:25:06.916851+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.924317+0000', 'last_change': '2024-03-06T07:25:06.924317+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.924317+0000', 'last_undegraded': '2024-03-06T07:25:06.924317+0000', 'last_fullsized': '2024-03-06T07:25:06.924317+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2024-03-06T07:25:11.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.920999+0000', 'last_change': '2024-03-06T07:25:06.920999+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.920999+0000', 'last_undegraded': '2024-03-06T07:25:06.920999+0000', 'last_fullsized': '2024-03-06T07:25:06.920999+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:25:11.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2024-03-06T07:25:11.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.919352+0000', 'last_change': '2024-03-06T07:25:06.919352+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.919352+0000', 'last_undegraded': '2024-03-06T07:25:06.919352+0000', 'last_fullsized': '2024-03-06T07:25:06.919352+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2024-03-06T07:25:11.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 134, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:25:06.916675+0000', 'last_change': '2024-03-06T07:25:06.916675+0000', 'last_active': '2024-03-06T07:25:06.907627+0000', 'last_peered': '2024-03-06T07:25:06.907627+0000', 'last_clean': '2024-03-06T07:25:06.907627+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:25:06.916675+0000', 'last_undegraded': '2024-03-06T07:25:06.916675+0000', 'last_fullsized': '2024-03-06T07:25:06.916675+0000', 'mapping_epoch': 134, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 134, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'last_clean_scrub_stamp': '2024-03-06T07:25:06.907627+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:25:11.646 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:25:11.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 Processor -- start 2024-03-06T07:25:11.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 -- start start 2024-03-06T07:25:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07cc06d9f0 0x7f07cc0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc0c3710 0x7f07cc1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc158800 con 0x7f07cc0c3280 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc158ae0 con 0x7f07cc06d9f0 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.768+0000 7f07d20eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc158dc0 con 0x7f07cc0c3710 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07caffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07cc0c3280 0x7f07cc1551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54260/0 (socket says 172.21.15.17:54260) 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07caffd640 1 -- 172.21.15.17:0/4078182985 learned_addr learned my addr 172.21.15.17:0/4078182985 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/4078182985 >> v2:172.21.15.163:3300/0 conn(0x7f07cc06d9f0 0x7f07cc0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cbfff640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc0c3710 0x7f07cc1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07ca7fc640 1 -- 172.21.15.17:0/4078182985 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018859670 0 0) 0x7f07cc158800 con 0x7f07cc0c3280 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07ca7fc640 1 -- 172.21.15.17:0/4078182985 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b4003520 con 0x7f07cc0c3280 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cb7fe640 1 -- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc0c3710 msgr2=0x7f07cc1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc0c3710 0x7f07cc1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cb7fe640 1 -- 172.21.15.17:0/4078182985 >> v1:172.21.15.17:6789/0 conn(0x7f07cc0c3280 legacy=0x7f07cc1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.769+0000 7f07cb7fe640 1 -- 172.21.15.17:0/4078182985 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc1590a0 con 0x7f07cc06d9f0 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc06d9f0 0x7f07cc0a7cd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0097a0 tx=0x7f07bc02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8884b3a560410b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07ca7fc640 1 -- 172.21.15.17:0/4078182985 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0315a0 con 0x7f07cc06d9f0 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07ca7fc640 1 -- 172.21.15.17:0/4078182985 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc037070 con 0x7f07cc06d9f0 2024-03-06T07:25:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07ca7fc640 1 -- 172.21.15.17:0/4078182985 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc031940 con 0x7f07cc06d9f0 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07d20eb640 1 -- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc06d9f0 msgr2=0x7f07cc0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.770+0000 7f07d20eb640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc06d9f0 0x7f07cc0a7cd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0097a0 tx=0x7f07bc02b840 comp rx=0 tx=0).stop 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 -- 172.21.15.17:0/4078182985 shutdown_connections 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc0c3710 0x7f07cc1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 --2- 172.21.15.17:0/4078182985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc06d9f0 0x7f07cc0a7cd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 -- 172.21.15.17:0/4078182985 >> 172.21.15.17:0/4078182985 conn(0x7f07cc0aa630 msgr2=0x7f07cc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 -- 172.21.15.17:0/4078182985 shutdown_connections 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.771+0000 7f07d20eb640 1 -- 172.21.15.17:0/4078182985 wait complete. 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.772+0000 7f07d20eb640 1 Processor -- start 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.772+0000 7f07d20eb640 1 -- start start 2024-03-06T07:25:11.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.772+0000 7f07d20eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc06d9f0 0x7f07cc0ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.772+0000 7f07d20eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc0c3280 0x7f07cc0abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.772+0000 7f07d20eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07d20eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc06ddd0 con 0x7f07cc0c3710 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07d20eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc155e30 con 0x7f07cc0c3280 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07d20eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc06df50 con 0x7f07cc06d9f0 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54198/0 (socket says 172.21.15.17:54198) 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 -- 172.21.15.17:0/3951405812 learned_addr learned my addr 172.21.15.17:0/3951405812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07caffd640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc0c3280 0x7f07cc0abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc06d9f0 0x7f07cc0ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 -- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc06d9f0 msgr2=0x7f07cc0ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc06d9f0 0x7f07cc0ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 -- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc0c3280 msgr2=0x7f07cc0abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc0c3280 0x7f07cc0abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 -- 172.21.15.17:0/3951405812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc1eb910 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07cbfff640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f07c400ec10 tx=0x7f07c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4018020 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c400cd20 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c40146c0 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc14fe60 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.773+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc1eb630 con 0x7f07cc0c3710 2024-03-06T07:25:11.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.774+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc04f330 con 0x7f07cc0c3710 2024-03-06T07:25:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.775+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07c4005d10 con 0x7f07cc0c3710 2024-03-06T07:25:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.775+0000 7f07b3fff640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 0x7f07b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.775+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07c408ad50 con 0x7f07cc0c3710 2024-03-06T07:25:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.775+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 0x7f07b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:11.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.776+0000 7f07cb7fe640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 0x7f07b806d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f07bc036000 tx=0x7f07bc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:11.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:11.780+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07c408e030 con 0x7f07cc0c3710 2024-03-06T07:25:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.031+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f07cc073240 con 0x7f07cc0c3710 2024-03-06T07:25:12.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.031+0000 7f07b3fff640 1 -- 172.21.15.17:0/3951405812 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1458 (secure 0 0 0) 0x7f07c4022070 con 0x7f07cc0c3710 2024-03-06T07:25:12.033 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:12.033 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709597,"num_in_osds":8,"osd_in_since":1709709589,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":268980224,"bytes_avail":772825133056,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":120,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"d","status":"up:active","gid":8280}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-03-06T07:24:55.406211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 msgr2=0x7f07b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 0x7f07b806d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f07bc036000 tx=0x7f07bc03d040 comp rx=0 tx=0).stop 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 msgr2=0x7f07cc1548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f07c400ec10 tx=0x7f07c400c490 comp rx=0 tx=0).stop 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 shutdown_connections 2024-03-06T07:25:12.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07b806b4d0 0x7f07b806d990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07cc0c3710 0x7f07cc1548f0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07cc0c3280 0x7f07cc0abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 --2- 172.21.15.17:0/3951405812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07cc06d9f0 0x7f07cc0ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 >> 172.21.15.17:0/3951405812 conn(0x7f07cc0aa630 msgr2=0x7f07cc151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 shutdown_connections 2024-03-06T07:25:12.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.033+0000 7f07d20eb640 1 -- 172.21.15.17:0/3951405812 wait complete. 2024-03-06T07:25:12.046 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:25:12.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:12.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.145+0000 7f5fd3839640 1 Processor -- start 2024-03-06T07:25:12.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 -- start start 2024-03-06T07:25:12.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fcc0c88c0 0x7f5fcc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5fcc158b60 0x7f5fcc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc0c73d0 con 0x7f5fcc14f360 2024-03-06T07:25:12.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fcc0c8dd0 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd3839640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fcc0be2e0 con 0x7f5fcc158b60 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd0dd4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fcc0c88c0 0x7f5fcc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd0dd4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5fcc0c88c0 0x7f5fcc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54206/0 (socket says 172.21.15.17:54206) 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd0dd4640 1 -- 172.21.15.17:0/4050717080 learned_addr learned my addr 172.21.15.17:0/4050717080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.146+0000 7f5fd1dd6640 1 --2- 172.21.15.17:0/4050717080 >> v2:172.21.15.163:3300/0 conn(0x7f5fcc158b60 0x7f5fcc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.147+0000 7f5fd0dd4640 1 -- 172.21.15.17:0/4050717080 >> v2:172.21.15.163:3300/0 conn(0x7f5fcc158b60 msgr2=0x7f5fcc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.147+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/4050717080 >> v2:172.21.15.163:3300/0 conn(0x7f5fcc158b60 0x7f5fcc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.147+0000 7f5fd0dd4640 1 -- 172.21.15.17:0/4050717080 >> v1:172.21.15.163:6789/0 conn(0x7f5fcc14f360 legacy=0x7f5fcc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.147+0000 7f5fd0dd4640 1 -- 172.21.15.17:0/4050717080 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0c7670 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.147+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/4050717080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc155c10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc152d60 tx=0x7f5fc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278da792ebb3555d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fcbfff640 1 -- 172.21.15.17:0/4050717080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0035020 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fcbfff640 1 -- 172.21.15.17:0/4050717080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0031870 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fcbfff640 1 -- 172.21.15.17:0/4050717080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0031c40 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 -- 172.21.15.17:0/4050717080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 msgr2=0x7f5fcc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 --2- 172.21.15.17:0/4050717080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc155c10 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc152d60 tx=0x7f5fc002b910 comp rx=0 tx=0).stop 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 -- 172.21.15.17:0/4050717080 shutdown_connections 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 --2- 172.21.15.17:0/4050717080 >> v2:172.21.15.163:3300/0 conn(0x7f5fcc158b60 0x7f5fcc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 --2- 172.21.15.17:0/4050717080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc155c10 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 -- 172.21.15.17:0/4050717080 >> 172.21.15.17:0/4050717080 conn(0x7f5fcc0ac570 msgr2=0x7f5fcc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 -- 172.21.15.17:0/4050717080 shutdown_connections 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.148+0000 7f5fd3839640 1 -- 172.21.15.17:0/4050717080 wait complete. 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.149+0000 7f5fd3839640 1 Processor -- start 2024-03-06T07:25:12.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.149+0000 7f5fd3839640 1 -- start start 2024-03-06T07:25:12.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 0x7f5fcc0bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fcc158b60 0x7f5fcc0bf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0c5a50 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0c7c30 con 0x7f5fcc158b60 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd3839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0c5bd0 con 0x7f5fcc14f360 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54208/0 (socket says 172.21.15.17:54208) 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 -- 172.21.15.17:0/664606696 learned_addr learned my addr 172.21.15.17:0/664606696 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd1dd6640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fcc158b60 0x7f5fcc0bf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 0x7f5fcc0bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 -- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 msgr2=0x7f5fcc0bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 0x7f5fcc0bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 -- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fcc158b60 msgr2=0x7f5fcc0bf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fcc158b60 0x7f5fcc0bf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 -- 172.21.15.17:0/664606696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0be620 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 0x7f5fcc0bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.150+0000 7f5fd15d5640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c400 tx=0x7f5fc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.151+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40052d0 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.151+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4005490 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.151+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc0be900 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.151+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4007260 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.151+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc0beec0 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.152+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc04f330 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.152+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fc4007400 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.153+0000 7f5fc9ffb640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 0x7f5fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:12.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.153+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5fc40897a0 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.153+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 0x7f5fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:12.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.153+0000 7f5fd0dd4640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 0x7f5fb806d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0036000 tx=0x7f5fc003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:12.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.157+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5fc4057ed0 con 0x7f5fcc0c88c0 2024-03-06T07:25:12.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.335+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5fcc1ee020 con 0x7f5fb806b4d0 2024-03-06T07:25:12.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.341+0000 7f5fc9ffb640 1 -- 172.21.15.17:0/664606696 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247169 (secure 0 0 0) 0x7f5fcc1ee020 con 0x7f5fb806b4d0 2024-03-06T07:25:12.345 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:12.346 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:12.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 msgr2=0x7f5fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 0x7f5fb806d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0036000 tx=0x7f5fc003c040 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 msgr2=0x7f5fcc0c0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c400 tx=0x7f5fc400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 shutdown_connections 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5fb806b4d0 0x7f5fb806d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fcc158b60 0x7f5fcc0bf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fcc14f360 0x7f5fcc0bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 --2- 172.21.15.17:0/664606696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5fcc0c88c0 0x7f5fcc0c0850 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 >> 172.21.15.17:0/664606696 conn(0x7f5fcc0ac570 msgr2=0x7f5fcc14cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 shutdown_connections 2024-03-06T07:25:12.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:12.344+0000 7f5fd3839640 1 -- 172.21.15.17:0/664606696 wait complete. 2024-03-06T07:25:12.356 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-03-06T07:25:11.409677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262676,"kb_used_data":13220,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712044,"statfs":{"total":773094113280,"available":772825133056,"internally_reserved":0,"allocated":13537280,"data_stored":7649345,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.501197"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925142+0000","last_change":"2024-03-06T07:25:06.925142+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925142+0000","last_undegraded":"2024-03-06T07:25:06.925142+0000","last_fullsized":"2024-03-06T07:25:06.925142+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.923368+0000","last_change":"2024-03-06T07:25:06.923368+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.923368+0000","last_undegraded":"2024-03-06T07:25:06.923368+0000","last_fullsized":"2024-03-06T07:25:06.923368+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924643+0000","last_change":"2024-03-06T07:25:06.924643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924643+0000","last_undegraded":"2024-03-06T07:25:06.924643+0000","last_fullsized":"2024-03-06T07:25:06.924643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924710+0000","last_change":"2024-03-06T07:25:06.924710+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924710+0000","last_undegraded":"2024-03-06T07:25:06.924710+0000","last_fullsized":"2024-03-06T07:25:06.924710+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922611+0000","last_change":"2024-03-06T07:25:06.922611+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922611+0000","last_undegraded":"2024-03-06T07:25:06.922611+0000","last_fullsized":"2024-03-06T07:25:06.922611+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922132+0000","last_change":"2024-03-06T07:25:06.922132+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922132+0000","last_undegraded":"2024-03-06T07:25:06.922132+0000","last_fullsized":"2024-03-06T07:25:06.922132+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927603+0000","last_change":"2024-03-06T07:25:06.927603+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927603+0000","last_undegraded":"2024-03-06T07:25:06.927603+0000","last_fullsized":"2024-03-06T07:25:06.927603+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918232+0000","last_change":"2024-03-06T07:25:06.918232+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918232+0000","last_undegraded":"2024-03-06T07:25:06.918232+0000","last_fullsized":"2024-03-06T07:25:06.918232+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921643+0000","last_change":"2024-03-06T07:25:06.921643+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921643+0000","last_undegraded":"2024-03-06T07:25:06.921643+0000","last_fullsized":"2024-03-06T07:25:06.921643+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921166+0000","last_change":"2024-03-06T07:25:06.921166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921166+0000","last_undegraded":"2024-03-06T07:25:06.921166+0000","last_fullsized":"2024-03-06T07:25:06.921166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920314+0000","last_change":"2024-03-06T07:25:06.920314+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920314+0000","last_undegraded":"2024-03-06T07:25:06.920314+0000","last_fullsized":"2024-03-06T07:25:06.920314+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924221+0000","last_change":"2024-03-06T07:25:06.924221+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924221+0000","last_undegraded":"2024-03-06T07:25:06.924221+0000","last_fullsized":"2024-03-06T07:25:06.924221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921124+0000","last_change":"2024-03-06T07:25:06.921124+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921124+0000","last_undegraded":"2024-03-06T07:25:06.921124+0000","last_fullsized":"2024-03-06T07:25:06.921124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924524+0000","last_change":"2024-03-06T07:25:06.924524+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924524+0000","last_undegraded":"2024-03-06T07:25:06.924524+0000","last_fullsized":"2024-03-06T07:25:06.924524+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921656+0000","last_change":"2024-03-06T07:25:06.921656+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921656+0000","last_undegraded":"2024-03-06T07:25:06.921656+0000","last_fullsized":"2024-03-06T07:25:06.921656+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921140+0000","last_change":"2024-03-06T07:25:06.921140+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921140+0000","last_undegraded":"2024-03-06T07:25:06.921140+0000","last_fullsized":"2024-03-06T07:25:06.921140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924794+0000","last_change":"2024-03-06T07:25:06.924794+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924794+0000","last_undegraded":"2024-03-06T07:25:06.924794+0000","last_fullsized":"2024-03-06T07:25:06.924794+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918730+0000","last_change":"2024-03-06T07:25:06.918730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918730+0000","last_undegraded":"2024-03-06T07:25:06.918730+0000","last_fullsized":"2024-03-06T07:25:06.918730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918574+0000","last_change":"2024-03-06T07:25:06.918574+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918574+0000","last_undegraded":"2024-03-06T07:25:06.918574+0000","last_fullsized":"2024-03-06T07:25:06.918574+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920290+0000","last_change":"2024-03-06T07:25:06.920290+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920290+0000","last_undegraded":"2024-03-06T07:25:06.920290+0000","last_fullsized":"2024-03-06T07:25:06.920290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920527+0000","last_change":"2024-03-06T07:25:06.920527+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920527+0000","last_undegraded":"2024-03-06T07:25:06.920527+0000","last_fullsized":"2024-03-06T07:25:06.920527+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.926454+0000","last_change":"2024-03-06T07:25:06.926454+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.926454+0000","last_undegraded":"2024-03-06T07:25:06.926454+0000","last_fullsized":"2024-03-06T07:25:06.926454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917006+0000","last_change":"2024-03-06T07:25:06.917006+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917006+0000","last_undegraded":"2024-03-06T07:25:06.917006+0000","last_fullsized":"2024-03-06T07:25:06.917006+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917762+0000","last_change":"2024-03-06T07:25:06.917762+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917762+0000","last_undegraded":"2024-03-06T07:25:06.917762+0000","last_fullsized":"2024-03-06T07:25:06.917762+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924716+0000","last_change":"2024-03-06T07:25:06.924716+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924716+0000","last_undegraded":"2024-03-06T07:25:06.924716+0000","last_fullsized":"2024-03-06T07:25:06.924716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925850+0000","last_change":"2024-03-06T07:25:06.925850+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925850+0000","last_undegraded":"2024-03-06T07:25:06.925850+0000","last_fullsized":"2024-03-06T07:25:06.925850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919602+0000","last_change":"2024-03-06T07:25:06.919602+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919602+0000","last_undegraded":"2024-03-06T07:25:06.919602+0000","last_fullsized":"2024-03-06T07:25:06.919602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918067+0000","last_change":"2024-03-06T07:25:06.918067+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918067+0000","last_undegraded":"2024-03-06T07:25:06.918067+0000","last_fullsized":"2024-03-06T07:25:06.918067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918954+0000","last_change":"2024-03-06T07:25:06.918954+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918954+0000","last_undegraded":"2024-03-06T07:25:06.918954+0000","last_fullsized":"2024-03-06T07:25:06.918954+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918136+0000","last_change":"2024-03-06T07:25:06.918136+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918136+0000","last_undegraded":"2024-03-06T07:25:06.918136+0000","last_fullsized":"2024-03-06T07:25:06.918136+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917781+0000","last_change":"2024-03-06T07:25:06.917781+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917781+0000","last_undegraded":"2024-03-06T07:25:06.917781+0000","last_fullsized":"2024-03-06T07:25:06.917781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.922036+0000","last_change":"2024-03-06T07:25:06.922036+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.922036+0000","last_undegraded":"2024-03-06T07:25:06.922036+0000","last_fullsized":"2024-03-06T07:25:06.922036+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.928472+0000","last_change":"2024-03-06T07:25:06.926918+0000","last_active":"2024-03-06T07:25:06.928472+0000","last_peered":"2024-03-06T07:25:06.928472+0000","last_clean":"2024-03-06T07:25:06.928472+0000","last_became_active":"2024-03-06T07:25:06.926475+0000","last_became_peered":"2024-03-06T07:25:06.926475+0000","last_unstale":"2024-03-06T07:25:06.928472+0000","last_undegraded":"2024-03-06T07:25:06.928472+0000","last_fullsized":"2024-03-06T07:25:06.928472+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:35.860907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924223+0000","last_change":"2024-03-06T07:25:06.924223+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924223+0000","last_undegraded":"2024-03-06T07:25:06.924223+0000","last_fullsized":"2024-03-06T07:25:06.924223+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937287+0000","last_change":"2024-03-06T07:25:06.933126+0000","last_active":"2024-03-06T07:25:06.937287+0000","last_peered":"2024-03-06T07:25:06.937287+0000","last_clean":"2024-03-06T07:25:06.937287+0000","last_became_active":"2024-03-06T07:25:06.932843+0000","last_became_peered":"2024-03-06T07:25:06.932843+0000","last_unstale":"2024-03-06T07:25:06.937287+0000","last_undegraded":"2024-03-06T07:25:06.937287+0000","last_fullsized":"2024-03-06T07:25:06.937287+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:33.155673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916730+0000","last_change":"2024-03-06T07:25:06.916730+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916730+0000","last_undegraded":"2024-03-06T07:25:06.916730+0000","last_fullsized":"2024-03-06T07:25:06.916730+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.934052+0000","last_change":"2024-03-06T07:25:06.932116+0000","last_active":"2024-03-06T07:25:06.934052+0000","last_peered":"2024-03-06T07:25:06.934052+0000","last_clean":"2024-03-06T07:25:06.934052+0000","last_became_active":"2024-03-06T07:25:06.931835+0000","last_became_peered":"2024-03-06T07:25:06.931835+0000","last_unstale":"2024-03-06T07:25:06.934052+0000","last_undegraded":"2024-03-06T07:25:06.934052+0000","last_fullsized":"2024-03-06T07:25:06.934052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:52.067097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918299+0000","last_change":"2024-03-06T07:25:06.918299+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918299+0000","last_undegraded":"2024-03-06T07:25:06.918299+0000","last_fullsized":"2024-03-06T07:25:06.918299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937080+0000","last_change":"2024-03-06T07:25:06.936612+0000","last_active":"2024-03-06T07:25:06.937080+0000","last_peered":"2024-03-06T07:25:06.937080+0000","last_clean":"2024-03-06T07:25:06.937080+0000","last_became_active":"2024-03-06T07:25:06.936165+0000","last_became_peered":"2024-03-06T07:25:06.936165+0000","last_unstale":"2024-03-06T07:25:06.937080+0000","last_undegraded":"2024-03-06T07:25:06.937080+0000","last_fullsized":"2024-03-06T07:25:06.937080+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:41.959433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920192+0000","last_change":"2024-03-06T07:25:06.920192+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920192+0000","last_undegraded":"2024-03-06T07:25:06.920192+0000","last_fullsized":"2024-03-06T07:25:06.920192+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937555+0000","last_change":"2024-03-06T07:25:06.932157+0000","last_active":"2024-03-06T07:25:06.937555+0000","last_peered":"2024-03-06T07:25:06.937555+0000","last_clean":"2024-03-06T07:25:06.937555+0000","last_became_active":"2024-03-06T07:25:06.931866+0000","last_became_peered":"2024-03-06T07:25:06.931866+0000","last_unstale":"2024-03-06T07:25:06.937555+0000","last_undegraded":"2024-03-06T07:25:06.937555+0000","last_fullsized":"2024-03-06T07:25:06.937555+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:47.500674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925548+0000","last_change":"2024-03-06T07:25:06.925548+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925548+0000","last_undegraded":"2024-03-06T07:25:06.925548+0000","last_fullsized":"2024-03-06T07:25:06.925548+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933326+0000","last_change":"2024-03-06T07:25:06.932382+0000","last_active":"2024-03-06T07:25:06.933326+0000","last_peered":"2024-03-06T07:25:06.933326+0000","last_clean":"2024-03-06T07:25:06.933326+0000","last_became_active":"2024-03-06T07:25:06.932043+0000","last_became_peered":"2024-03-06T07:25:06.932043+0000","last_unstale":"2024-03-06T07:25:06.933326+0000","last_undegraded":"2024-03-06T07:25:06.933326+0000","last_fullsized":"2024-03-06T07:25:06.933326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:37.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.921418+0000","last_change":"2024-03-06T07:25:06.921418+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.921418+0000","last_undegraded":"2024-03-06T07:25:06.921418+0000","last_fullsized":"2024-03-06T07:25:06.921418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936978+0000","last_change":"2024-03-06T07:25:06.934761+0000","last_active":"2024-03-06T07:25:06.936978+0000","last_peered":"2024-03-06T07:25:06.936978+0000","last_clean":"2024-03-06T07:25:06.936978+0000","last_became_active":"2024-03-06T07:25:06.934441+0000","last_became_peered":"2024-03-06T07:25:06.934441+0000","last_unstale":"2024-03-06T07:25:06.936978+0000","last_undegraded":"2024-03-06T07:25:06.936978+0000","last_fullsized":"2024-03-06T07:25:06.936978+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:05:30.827701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.927721+0000","last_change":"2024-03-06T07:25:06.927721+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.927721+0000","last_undegraded":"2024-03-06T07:25:06.927721+0000","last_fullsized":"2024-03-06T07:25:06.927721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936449+0000","last_change":"2024-03-06T07:25:06.935857+0000","last_active":"2024-03-06T07:25:06.936449+0000","last_peered":"2024-03-06T07:25:06.936449+0000","last_clean":"2024-03-06T07:25:06.936449+0000","last_became_active":"2024-03-06T07:25:06.935520+0000","last_became_peered":"2024-03-06T07:25:06.935520+0000","last_unstale":"2024-03-06T07:25:06.936449+0000","last_undegraded":"2024-03-06T07:25:06.936449+0000","last_fullsized":"2024-03-06T07:25:06.936449+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:41.659020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916666+0000","last_change":"2024-03-06T07:25:06.916666+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916666+0000","last_undegraded":"2024-03-06T07:25:06.916666+0000","last_fullsized":"2024-03-06T07:25:06.916666+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931892+0000","last_change":"2024-03-06T07:25:06.928957+0000","last_active":"2024-03-06T07:25:06.931892+0000","last_peered":"2024-03-06T07:25:06.931892+0000","last_clean":"2024-03-06T07:25:06.931892+0000","last_became_active":"2024-03-06T07:25:06.928262+0000","last_became_peered":"2024-03-06T07:25:06.928262+0000","last_unstale":"2024-03-06T07:25:06.931892+0000","last_undegraded":"2024-03-06T07:25:06.931892+0000","last_fullsized":"2024-03-06T07:25:06.931892+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:21.229011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919913+0000","last_change":"2024-03-06T07:25:06.919913+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919913+0000","last_undegraded":"2024-03-06T07:25:06.919913+0000","last_fullsized":"2024-03-06T07:25:06.919913+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935245+0000","last_change":"2024-03-06T07:25:06.931781+0000","last_active":"2024-03-06T07:25:06.935245+0000","last_peered":"2024-03-06T07:25:06.935245+0000","last_clean":"2024-03-06T07:25:06.935245+0000","last_became_active":"2024-03-06T07:25:06.931456+0000","last_became_peered":"2024-03-06T07:25:06.931456+0000","last_unstale":"2024-03-06T07:25:06.935245+0000","last_undegraded":"2024-03-06T07:25:06.935245+0000","last_fullsized":"2024-03-06T07:25:06.935245+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918286+0000","last_change":"2024-03-06T07:25:06.918286+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918286+0000","last_undegraded":"2024-03-06T07:25:06.918286+0000","last_fullsized":"2024-03-06T07:25:06.918286+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926217+0000","last_change":"2024-03-06T07:25:06.925504+0000","last_active":"2024-03-06T07:25:06.926217+0000","last_peered":"2024-03-06T07:25:06.926217+0000","last_clean":"2024-03-06T07:25:06.926217+0000","last_became_active":"2024-03-06T07:25:06.925180+0000","last_became_peered":"2024-03-06T07:25:06.925180+0000","last_unstale":"2024-03-06T07:25:06.926217+0000","last_undegraded":"2024-03-06T07:25:06.926217+0000","last_fullsized":"2024-03-06T07:25:06.926217+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:37:19.573816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917364+0000","last_change":"2024-03-06T07:25:06.917364+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917364+0000","last_undegraded":"2024-03-06T07:25:06.917364+0000","last_fullsized":"2024-03-06T07:25:06.917364+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.926965+0000","last_change":"2024-03-06T07:25:06.926155+0000","last_active":"2024-03-06T07:25:06.926965+0000","last_peered":"2024-03-06T07:25:06.926965+0000","last_clean":"2024-03-06T07:25:06.926965+0000","last_became_active":"2024-03-06T07:25:06.925662+0000","last_became_peered":"2024-03-06T07:25:06.925662+0000","last_unstale":"2024-03-06T07:25:06.926965+0000","last_undegraded":"2024-03-06T07:25:06.926965+0000","last_fullsized":"2024-03-06T07:25:06.926965+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:36.511111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919303+0000","last_change":"2024-03-06T07:25:06.919303+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919303+0000","last_undegraded":"2024-03-06T07:25:06.919303+0000","last_fullsized":"2024-03-06T07:25:06.919303+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942235+0000","last_change":"2024-03-06T07:25:06.936715+0000","last_active":"2024-03-06T07:25:06.942235+0000","last_peered":"2024-03-06T07:25:06.942235+0000","last_clean":"2024-03-06T07:25:06.942235+0000","last_became_active":"2024-03-06T07:25:06.936361+0000","last_became_peered":"2024-03-06T07:25:06.936361+0000","last_unstale":"2024-03-06T07:25:06.942235+0000","last_undegraded":"2024-03-06T07:25:06.942235+0000","last_fullsized":"2024-03-06T07:25:06.942235+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919086+0000","last_change":"2024-03-06T07:25:06.919086+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919086+0000","last_undegraded":"2024-03-06T07:25:06.919086+0000","last_fullsized":"2024-03-06T07:25:06.919086+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937421+0000","last_change":"2024-03-06T07:25:06.934580+0000","last_active":"2024-03-06T07:25:06.937421+0000","last_peered":"2024-03-06T07:25:06.937421+0000","last_clean":"2024-03-06T07:25:06.937421+0000","last_became_active":"2024-03-06T07:25:06.934296+0000","last_became_peered":"2024-03-06T07:25:06.934296+0000","last_unstale":"2024-03-06T07:25:06.937421+0000","last_undegraded":"2024-03-06T07:25:06.937421+0000","last_fullsized":"2024-03-06T07:25:06.937421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:32.439585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920493+0000","last_change":"2024-03-06T07:25:06.920493+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920493+0000","last_undegraded":"2024-03-06T07:25:06.920493+0000","last_fullsized":"2024-03-06T07:25:06.920493+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937088+0000","last_change":"2024-03-06T07:25:06.934527+0000","last_active":"2024-03-06T07:25:06.937088+0000","last_peered":"2024-03-06T07:25:06.937088+0000","last_clean":"2024-03-06T07:25:06.937088+0000","last_became_active":"2024-03-06T07:25:06.934249+0000","last_became_peered":"2024-03-06T07:25:06.934249+0000","last_unstale":"2024-03-06T07:25:06.937088+0000","last_undegraded":"2024-03-06T07:25:06.937088+0000","last_fullsized":"2024-03-06T07:25:06.937088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917750+0000","last_change":"2024-03-06T07:25:06.917750+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917750+0000","last_undegraded":"2024-03-06T07:25:06.917750+0000","last_fullsized":"2024-03-06T07:25:06.917750+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.925249+0000","last_change":"2024-03-06T07:25:06.924175+0000","last_active":"2024-03-06T07:25:06.925249+0000","last_peered":"2024-03-06T07:25:06.925249+0000","last_clean":"2024-03-06T07:25:06.925249+0000","last_became_active":"2024-03-06T07:25:06.923894+0000","last_became_peered":"2024-03-06T07:25:06.923894+0000","last_unstale":"2024-03-06T07:25:06.925249+0000","last_undegraded":"2024-03-06T07:25:06.925249+0000","last_fullsized":"2024-03-06T07:25:06.925249+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:34.739537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918166+0000","last_change":"2024-03-06T07:25:06.918166+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918166+0000","last_undegraded":"2024-03-06T07:25:06.918166+0000","last_fullsized":"2024-03-06T07:25:06.918166+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931792+0000","last_change":"2024-03-06T07:25:06.926576+0000","last_active":"2024-03-06T07:25:06.931792+0000","last_peered":"2024-03-06T07:25:06.931792+0000","last_clean":"2024-03-06T07:25:06.931792+0000","last_became_active":"2024-03-06T07:25:06.926315+0000","last_became_peered":"2024-03-06T07:25:06.926315+0000","last_unstale":"2024-03-06T07:25:06.931792+0000","last_undegraded":"2024-03-06T07:25:06.931792+0000","last_fullsized":"2024-03-06T07:25:06.931792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:38.818021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925302+0000","last_change":"2024-03-06T07:25:06.925302+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925302+0000","last_undegraded":"2024-03-06T07:25:06.925302+0000","last_fullsized":"2024-03-06T07:25:06.925302+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935481+0000","last_change":"2024-03-06T07:25:06.935036+0000","last_active":"2024-03-06T07:25:06.935481+0000","last_peered":"2024-03-06T07:25:06.935481+0000","last_clean":"2024-03-06T07:25:06.935481+0000","last_became_active":"2024-03-06T07:25:06.934723+0000","last_became_peered":"2024-03-06T07:25:06.934723+0000","last_unstale":"2024-03-06T07:25:06.935481+0000","last_undegraded":"2024-03-06T07:25:06.935481+0000","last_fullsized":"2024-03-06T07:25:06.935481+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:25.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924247+0000","last_change":"2024-03-06T07:25:06.924247+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924247+0000","last_undegraded":"2024-03-06T07:25:06.924247+0000","last_fullsized":"2024-03-06T07:25:06.924247+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942252+0000","last_change":"2024-03-06T07:25:06.937027+0000","last_active":"2024-03-06T07:25:06.942252+0000","last_peered":"2024-03-06T07:25:06.942252+0000","last_clean":"2024-03-06T07:25:06.942252+0000","last_became_active":"2024-03-06T07:25:06.936721+0000","last_became_peered":"2024-03-06T07:25:06.936721+0000","last_unstale":"2024-03-06T07:25:06.942252+0000","last_undegraded":"2024-03-06T07:25:06.942252+0000","last_fullsized":"2024-03-06T07:25:06.942252+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:03.286807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917215+0000","last_change":"2024-03-06T07:25:06.917215+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917215+0000","last_undegraded":"2024-03-06T07:25:06.917215+0000","last_fullsized":"2024-03-06T07:25:06.917215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932764+0000","last_change":"2024-03-06T07:25:06.930306+0000","last_active":"2024-03-06T07:25:06.932764+0000","last_peered":"2024-03-06T07:25:06.932764+0000","last_clean":"2024-03-06T07:25:06.932764+0000","last_became_active":"2024-03-06T07:25:06.929975+0000","last_became_peered":"2024-03-06T07:25:06.929975+0000","last_unstale":"2024-03-06T07:25:06.932764+0000","last_undegraded":"2024-03-06T07:25:06.932764+0000","last_fullsized":"2024-03-06T07:25:06.932764+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:14.435468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.918737+0000","last_change":"2024-03-06T07:25:06.918737+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.918737+0000","last_undegraded":"2024-03-06T07:25:06.918737+0000","last_fullsized":"2024-03-06T07:25:06.918737+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935878+0000","last_change":"2024-03-06T07:25:06.930143+0000","last_active":"2024-03-06T07:25:06.935878+0000","last_peered":"2024-03-06T07:25:06.935878+0000","last_clean":"2024-03-06T07:25:06.935878+0000","last_became_active":"2024-03-06T07:25:06.929880+0000","last_became_peered":"2024-03-06T07:25:06.929880+0000","last_unstale":"2024-03-06T07:25:06.935878+0000","last_undegraded":"2024-03-06T07:25:06.935878+0000","last_fullsized":"2024-03-06T07:25:06.935878+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:51.483404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.917337+0000","last_change":"2024-03-06T07:25:06.917337+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.917337+0000","last_undegraded":"2024-03-06T07:25:06.917337+0000","last_fullsized":"2024-03-06T07:25:06.917337+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932734+0000","last_change":"2024-03-06T07:25:06.930314+0000","last_active":"2024-03-06T07:25:06.932734+0000","last_peered":"2024-03-06T07:25:06.932734+0000","last_clean":"2024-03-06T07:25:06.932734+0000","last_became_active":"2024-03-06T07:25:06.929990+0000","last_became_peered":"2024-03-06T07:25:06.929990+0000","last_unstale":"2024-03-06T07:25:06.932734+0000","last_undegraded":"2024-03-06T07:25:06.932734+0000","last_fullsized":"2024-03-06T07:25:06.932734+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:38.139179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920031+0000","last_change":"2024-03-06T07:25:06.920031+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920031+0000","last_undegraded":"2024-03-06T07:25:06.920031+0000","last_fullsized":"2024-03-06T07:25:06.920031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932209+0000","last_change":"2024-03-06T07:25:06.925110+0000","last_active":"2024-03-06T07:25:06.932209+0000","last_peered":"2024-03-06T07:25:06.932209+0000","last_clean":"2024-03-06T07:25:06.932209+0000","last_became_active":"2024-03-06T07:25:06.924842+0000","last_became_peered":"2024-03-06T07:25:06.924842+0000","last_unstale":"2024-03-06T07:25:06.932209+0000","last_undegraded":"2024-03-06T07:25:06.932209+0000","last_fullsized":"2024-03-06T07:25:06.932209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:35:52.264094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919201+0000","last_change":"2024-03-06T07:25:06.919201+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919201+0000","last_undegraded":"2024-03-06T07:25:06.919201+0000","last_fullsized":"2024-03-06T07:25:06.919201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.942224+0000","last_change":"2024-03-06T07:25:06.936235+0000","last_active":"2024-03-06T07:25:06.942224+0000","last_peered":"2024-03-06T07:25:06.942224+0000","last_clean":"2024-03-06T07:25:06.942224+0000","last_became_active":"2024-03-06T07:25:06.935934+0000","last_became_peered":"2024-03-06T07:25:06.935934+0000","last_unstale":"2024-03-06T07:25:06.942224+0000","last_undegraded":"2024-03-06T07:25:06.942224+0000","last_fullsized":"2024-03-06T07:25:06.942224+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.925078+0000","last_change":"2024-03-06T07:25:06.925078+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.925078+0000","last_undegraded":"2024-03-06T07:25:06.925078+0000","last_fullsized":"2024-03-06T07:25:06.925078+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.935264+0000","last_change":"2024-03-06T07:25:06.933630+0000","last_active":"2024-03-06T07:25:06.935264+0000","last_peered":"2024-03-06T07:25:06.935264+0000","last_clean":"2024-03-06T07:25:06.935264+0000","last_became_active":"2024-03-06T07:25:06.933254+0000","last_became_peered":"2024-03-06T07:25:06.933254+0000","last_unstale":"2024-03-06T07:25:06.935264+0000","last_undegraded":"2024-03-06T07:25:06.935264+0000","last_fullsized":"2024-03-06T07:25:06.935264+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924327+0000","last_change":"2024-03-06T07:25:06.924327+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924327+0000","last_undegraded":"2024-03-06T07:25:06.924327+0000","last_fullsized":"2024-03-06T07:25:06.924327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936838+0000","last_change":"2024-03-06T07:25:06.936426+0000","last_active":"2024-03-06T07:25:06.936838+0000","last_peered":"2024-03-06T07:25:06.936838+0000","last_clean":"2024-03-06T07:25:06.936838+0000","last_became_active":"2024-03-06T07:25:06.936160+0000","last_became_peered":"2024-03-06T07:25:06.936160+0000","last_unstale":"2024-03-06T07:25:06.936838+0000","last_undegraded":"2024-03-06T07:25:06.936838+0000","last_fullsized":"2024-03-06T07:25:06.936838+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:39.491573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919378+0000","last_change":"2024-03-06T07:25:06.919378+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919378+0000","last_undegraded":"2024-03-06T07:25:06.919378+0000","last_fullsized":"2024-03-06T07:25:06.919378+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932280+0000","last_change":"2024-03-06T07:25:06.930511+0000","last_active":"2024-03-06T07:25:06.932280+0000","last_peered":"2024-03-06T07:25:06.932280+0000","last_clean":"2024-03-06T07:25:06.932280+0000","last_became_active":"2024-03-06T07:25:06.930205+0000","last_became_peered":"2024-03-06T07:25:06.930205+0000","last_unstale":"2024-03-06T07:25:06.932280+0000","last_undegraded":"2024-03-06T07:25:06.932280+0000","last_fullsized":"2024-03-06T07:25:06.932280+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:47.054740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916851+0000","last_change":"2024-03-06T07:25:06.916851+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916851+0000","last_undegraded":"2024-03-06T07:25:06.916851+0000","last_fullsized":"2024-03-06T07:25:06.916851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.936972+0000","last_change":"2024-03-06T07:25:06.935597+0000","last_active":"2024-03-06T07:25:06.936972+0000","last_peered":"2024-03-06T07:25:06.936972+0000","last_clean":"2024-03-06T07:25:06.936972+0000","last_became_active":"2024-03-06T07:25:06.935195+0000","last_became_peered":"2024-03-06T07:25:06.935195+0000","last_unstale":"2024-03-06T07:25:06.936972+0000","last_undegraded":"2024-03-06T07:25:06.936972+0000","last_fullsized":"2024-03-06T07:25:06.936972+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:50.432110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.924317+0000","last_change":"2024-03-06T07:25:06.924317+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.924317+0000","last_undegraded":"2024-03-06T07:25:06.924317+0000","last_fullsized":"2024-03-06T07:25:06.924317+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.932947+0000","last_change":"2024-03-06T07:25:06.931052+0000","last_active":"2024-03-06T07:25:06.932947+0000","last_peered":"2024-03-06T07:25:06.932947+0000","last_clean":"2024-03-06T07:25:06.932947+0000","last_became_active":"2024-03-06T07:25:06.930765+0000","last_became_peered":"2024-03-06T07:25:06.930765+0000","last_unstale":"2024-03-06T07:25:06.932947+0000","last_undegraded":"2024-03-06T07:25:06.932947+0000","last_fullsized":"2024-03-06T07:25:06.932947+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:40:26.305654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.920999+0000","last_change":"2024-03-06T07:25:06.920999+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.920999+0000","last_undegraded":"2024-03-06T07:25:06.920999+0000","last_fullsized":"2024-03-06T07:25:06.920999+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.937201+0000","last_change":"2024-03-06T07:25:06.935076+0000","last_active":"2024-03-06T07:25:06.937201+0000","last_peered":"2024-03-06T07:25:06.937201+0000","last_clean":"2024-03-06T07:25:06.937201+0000","last_became_active":"2024-03-06T07:25:06.934424+0000","last_became_peered":"2024-03-06T07:25:06.934424+0000","last_unstale":"2024-03-06T07:25:06.937201+0000","last_undegraded":"2024-03-06T07:25:06.937201+0000","last_fullsized":"2024-03-06T07:25:06.937201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:55.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.919352+0000","last_change":"2024-03-06T07:25:06.919352+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.919352+0000","last_undegraded":"2024-03-06T07:25:06.919352+0000","last_fullsized":"2024-03-06T07:25:06.919352+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.931872+0000","last_change":"2024-03-06T07:25:06.927019+0000","last_active":"2024-03-06T07:25:06.931872+0000","last_peered":"2024-03-06T07:25:06.931872+0000","last_clean":"2024-03-06T07:25:06.931872+0000","last_became_active":"2024-03-06T07:25:06.926737+0000","last_became_peered":"2024-03-06T07:25:06.926737+0000","last_unstale":"2024-03-06T07:25:06.931872+0000","last_undegraded":"2024-03-06T07:25:06.931872+0000","last_fullsized":"2024-03-06T07:25:06.931872+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:29.054350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":134,"state":"creating+peering","last_fresh":"2024-03-06T07:25:06.916675+0000","last_change":"2024-03-06T07:25:06.916675+0000","last_active":"2024-03-06T07:25:06.907627+0000","last_peered":"2024-03-06T07:25:06.907627+0000","last_clean":"2024-03-06T07:25:06.907627+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:25:06.916675+0000","last_undegraded":"2024-03-06T07:25:06.916675+0000","last_fullsized":"2024-03-06T07:25:06.916675+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.933731+0000","last_change":"2024-03-06T07:25:06.933005+0000","last_active":"2024-03-06T07:25:06.933731+0000","last_peered":"2024-03-06T07:25:06.933731+0000","last_clean":"2024-03-06T07:25:06.933731+0000","last_became_active":"2024-03-06T07:25:06.932668+0000","last_became_peered":"2024-03-06T07:25:06.932668+0000","last_unstale":"2024-03-06T07:25:06.933731+0000","last_undegraded":"2024-03-06T07:25:06.933731+0000","last_fullsized":"2024-03-06T07:25:06.933731+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:20.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-03-06T07:25:06.918569+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:06.918569+0000","last_peered":"2024-03-06T07:25:06.918569+0000","last_clean":"2024-03-06T07:25:06.918569+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:06.918569+0000","last_undegraded":"2024-03-06T07:25:06.918569+0000","last_fullsized":"2024-03-06T07:25:06.918569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":806076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":812743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1736704,"data_stored":1396444,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:13.214 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-03-06T07:25:13.212+0000 7f15f2851640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:15.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:15.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.473+0000 7fb14560e640 1 Processor -- start 2024-03-06T07:25:15.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.473+0000 7fb14560e640 1 -- start start 2024-03-06T07:25:15.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb14560e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb14560e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 0x7fb1400c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb14560e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400c7760 con 0x7fb140152920 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb14560e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1401587b0 con 0x7fb140159460 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb14560e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400c7a40 con 0x7fb1400c8a00 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13e7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13f7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 0x7fb1400c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13effd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb140152920 0x7fb140155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:52932/0 (socket says 172.21.15.17:52932) 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13effd640 1 -- 172.21.15.17:0/668805486 learned_addr learned my addr 172.21.15.17:0/668805486 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13dffb640 1 -- 172.21.15.17:0/668805486 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793225131 0 0) 0x7fb1400c7760 con 0x7fb140152920 2024-03-06T07:25:15.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13dffb640 1 -- 172.21.15.17:0/668805486 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb128003520 con 0x7fb140152920 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13e7fc640 1 -- 172.21.15.17:0/668805486 >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 msgr2=0x7fb1400c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13e7fc640 1 --2- 172.21.15.17:0/668805486 >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 0x7fb1400c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13e7fc640 1 -- 172.21.15.17:0/668805486 >> v1:172.21.15.163:6789/0 conn(0x7fb140152920 legacy=0x7fb140155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13e7fc640 1 -- 172.21.15.17:0/668805486 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400c7d20 con 0x7fb1400c8a00 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.474+0000 7fb13f7fe640 1 --2- 172.21.15.17:0/668805486 >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 0x7fb1400c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.475+0000 7fb13e7fc640 1 --2- 172.21.15.17:0/668805486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140158f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62f7cc41695b86f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.475+0000 7fb13dffb640 1 -- 172.21.15.17:0/668805486 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1340315a0 con 0x7fb1400c8a00 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.475+0000 7fb13dffb640 1 -- 172.21.15.17:0/668805486 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134037070 con 0x7fb1400c8a00 2024-03-06T07:25:15.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.475+0000 7fb13dffb640 1 -- 172.21.15.17:0/668805486 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134031960 con 0x7fb1400c8a00 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 -- 172.21.15.17:0/668805486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 msgr2=0x7fb140158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 --2- 172.21.15.17:0/668805486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140158f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 -- 172.21.15.17:0/668805486 shutdown_connections 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 --2- 172.21.15.17:0/668805486 >> v2:172.21.15.163:3300/0 conn(0x7fb140159460 0x7fb1400c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 --2- 172.21.15.17:0/668805486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140158f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 -- 172.21.15.17:0/668805486 >> 172.21.15.17:0/668805486 conn(0x7fb1400ac560 msgr2=0x7fb1400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 -- 172.21.15.17:0/668805486 shutdown_connections 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.476+0000 7fb14560e640 1 -- 172.21.15.17:0/668805486 wait complete. 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.477+0000 7fb14560e640 1 Processor -- start 2024-03-06T07:25:15.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.477+0000 7fb14560e640 1 -- start start 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb140148c10 0x7fb140147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400c7390 con 0x7fb140152920 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14006d030 con 0x7fb140148c10 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb14560e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400c7510 con 0x7fb1400c8a00 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54222/0 (socket says 172.21.15.17:54222) 2024-03-06T07:25:15.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 -- 172.21.15.17:0/680588342 learned_addr learned my addr 172.21.15.17:0/680588342 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13effd640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13f7fe640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb140148c10 0x7fb140147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 -- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 msgr2=0x7fb140157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 -- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb140148c10 msgr2=0x7fb140147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb140148c10 0x7fb140147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13e7fc640 1 -- 172.21.15.17:0/680588342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1401eae20 con 0x7fb140152920 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.478+0000 7fb13effd640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb13e7fc640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fb134034040 tx=0x7fb1340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134032430 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1340325d0 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb140149180 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1340329c0 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.479+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1401eb3c0 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.480+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14004f330 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.480+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb134032b60 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.481+0000 7fb1277fe640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 0x7fb12c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.481+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb13402f030 con 0x7fb140152920 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.481+0000 7fb13effd640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 0x7fb12c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.481+0000 7fb13effd640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 0x7fb12c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb130005880 tx=0x7fb130009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.485+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb134040070 con 0x7fb140152920 2024-03-06T07:25:15.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.664+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb140156390 con 0x7fb12c06b4d0 2024-03-06T07:25:15.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.670+0000 7fb1277fe640 1 -- 172.21.15.17:0/680588342 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7fb140156390 con 0x7fb12c06b4d0 2024-03-06T07:25:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:15.677 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 msgr2=0x7fb12c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 0x7fb12c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb130005880 tx=0x7fb130009040 comp rx=0 tx=0).stop 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 msgr2=0x7fb14014b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fb134034040 tx=0x7fb1340046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 shutdown_connections 2024-03-06T07:25:15.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb12c06b4d0 0x7fb12c06d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb140148c10 0x7fb140147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb140152920 0x7fb14014b320 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 --2- 172.21.15.17:0/680588342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1400c8a00 0x7fb140157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.673+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 >> 172.21.15.17:0/680588342 conn(0x7fb1400ac560 msgr2=0x7fb1400c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.674+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 shutdown_connections 2024-03-06T07:25:15.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.674+0000 7fb14560e640 1 -- 172.21.15.17:0/680588342 wait complete. 2024-03-06T07:25:15.686 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-03-06T07:25:15.410337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262988,"kb_used_data":13536,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711732,"statfs":{"total":773094113280,"available":772824813568,"internally_reserved":0,"allocated":13860864,"data_stored":7798372,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001605"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940262+0000","last_change":"2024-03-06T07:25:07.942118+0000","last_active":"2024-03-06T07:25:08.940262+0000","last_peered":"2024-03-06T07:25:08.940262+0000","last_clean":"2024-03-06T07:25:08.940262+0000","last_became_active":"2024-03-06T07:25:07.941795+0000","last_became_peered":"2024-03-06T07:25:07.941795+0000","last_unstale":"2024-03-06T07:25:08.940262+0000","last_undegraded":"2024-03-06T07:25:08.940262+0000","last_fullsized":"2024-03-06T07:25:08.940262+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:55.267860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924612+0000","last_change":"2024-03-06T07:25:07.924415+0000","last_active":"2024-03-06T07:25:08.924612+0000","last_peered":"2024-03-06T07:25:08.924612+0000","last_clean":"2024-03-06T07:25:08.924612+0000","last_became_active":"2024-03-06T07:25:07.923974+0000","last_became_peered":"2024-03-06T07:25:07.923974+0000","last_unstale":"2024-03-06T07:25:08.924612+0000","last_undegraded":"2024-03-06T07:25:08.924612+0000","last_fullsized":"2024-03-06T07:25:08.924612+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:15.441553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924440+0000","last_change":"2024-03-06T07:25:07.936461+0000","last_active":"2024-03-06T07:25:08.924440+0000","last_peered":"2024-03-06T07:25:08.924440+0000","last_clean":"2024-03-06T07:25:08.924440+0000","last_became_active":"2024-03-06T07:25:07.935545+0000","last_became_peered":"2024-03-06T07:25:07.935545+0000","last_unstale":"2024-03-06T07:25:08.924440+0000","last_undegraded":"2024-03-06T07:25:08.924440+0000","last_fullsized":"2024-03-06T07:25:08.924440+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:45.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.929011+0000","last_change":"2024-03-06T07:25:07.929707+0000","last_active":"2024-03-06T07:25:08.929011+0000","last_peered":"2024-03-06T07:25:08.929011+0000","last_clean":"2024-03-06T07:25:08.929011+0000","last_became_active":"2024-03-06T07:25:07.929056+0000","last_became_peered":"2024-03-06T07:25:07.929056+0000","last_unstale":"2024-03-06T07:25:08.929011+0000","last_undegraded":"2024-03-06T07:25:08.929011+0000","last_fullsized":"2024-03-06T07:25:08.929011+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:22.235096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386614+0000","last_change":"2024-03-06T07:25:07.941615+0000","last_active":"2024-03-06T07:25:09.386614+0000","last_peered":"2024-03-06T07:25:09.386614+0000","last_clean":"2024-03-06T07:25:09.386614+0000","last_became_active":"2024-03-06T07:25:07.941329+0000","last_became_peered":"2024-03-06T07:25:07.941329+0000","last_unstale":"2024-03-06T07:25:09.386614+0000","last_undegraded":"2024-03-06T07:25:09.386614+0000","last_fullsized":"2024-03-06T07:25:09.386614+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:28:29.528476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386161+0000","last_change":"2024-03-06T07:25:07.939607+0000","last_active":"2024-03-06T07:25:09.386161+0000","last_peered":"2024-03-06T07:25:09.386161+0000","last_clean":"2024-03-06T07:25:09.386161+0000","last_became_active":"2024-03-06T07:25:07.939347+0000","last_became_peered":"2024-03-06T07:25:07.939347+0000","last_unstale":"2024-03-06T07:25:09.386161+0000","last_undegraded":"2024-03-06T07:25:09.386161+0000","last_fullsized":"2024-03-06T07:25:09.386161+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:50.878910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.939946+0000","last_change":"2024-03-06T07:25:07.942233+0000","last_active":"2024-03-06T07:25:08.939946+0000","last_peered":"2024-03-06T07:25:08.939946+0000","last_clean":"2024-03-06T07:25:08.939946+0000","last_became_active":"2024-03-06T07:25:07.941956+0000","last_became_peered":"2024-03-06T07:25:07.941956+0000","last_unstale":"2024-03-06T07:25:08.939946+0000","last_undegraded":"2024-03-06T07:25:08.939946+0000","last_fullsized":"2024-03-06T07:25:08.939946+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:34.161758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928258+0000","last_change":"2024-03-06T07:25:07.934614+0000","last_active":"2024-03-06T07:25:08.928258+0000","last_peered":"2024-03-06T07:25:08.928258+0000","last_clean":"2024-03-06T07:25:08.928258+0000","last_became_active":"2024-03-06T07:25:07.934044+0000","last_became_peered":"2024-03-06T07:25:07.934044+0000","last_unstale":"2024-03-06T07:25:08.928258+0000","last_undegraded":"2024-03-06T07:25:08.928258+0000","last_fullsized":"2024-03-06T07:25:08.928258+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:06:10.279203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386491+0000","last_change":"2024-03-06T07:25:07.935722+0000","last_active":"2024-03-06T07:25:09.386491+0000","last_peered":"2024-03-06T07:25:09.386491+0000","last_clean":"2024-03-06T07:25:09.386491+0000","last_became_active":"2024-03-06T07:25:07.935464+0000","last_became_peered":"2024-03-06T07:25:07.935464+0000","last_unstale":"2024-03-06T07:25:09.386491+0000","last_undegraded":"2024-03-06T07:25:09.386491+0000","last_fullsized":"2024-03-06T07:25:09.386491+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:46.813982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386561+0000","last_change":"2024-03-06T07:25:07.931313+0000","last_active":"2024-03-06T07:25:09.386561+0000","last_peered":"2024-03-06T07:25:09.386561+0000","last_clean":"2024-03-06T07:25:09.386561+0000","last_became_active":"2024-03-06T07:25:07.931036+0000","last_became_peered":"2024-03-06T07:25:07.931036+0000","last_unstale":"2024-03-06T07:25:09.386561+0000","last_undegraded":"2024-03-06T07:25:09.386561+0000","last_fullsized":"2024-03-06T07:25:09.386561+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:06.516185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385535+0000","last_change":"2024-03-06T07:25:07.935265+0000","last_active":"2024-03-06T07:25:09.385535+0000","last_peered":"2024-03-06T07:25:09.385535+0000","last_clean":"2024-03-06T07:25:09.385535+0000","last_became_active":"2024-03-06T07:25:07.934985+0000","last_became_peered":"2024-03-06T07:25:07.934985+0000","last_unstale":"2024-03-06T07:25:09.385535+0000","last_undegraded":"2024-03-06T07:25:09.385535+0000","last_fullsized":"2024-03-06T07:25:09.385535+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:48:41.098373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.929026+0000","last_change":"2024-03-06T07:25:07.931782+0000","last_active":"2024-03-06T07:25:08.929026+0000","last_peered":"2024-03-06T07:25:08.929026+0000","last_clean":"2024-03-06T07:25:08.929026+0000","last_became_active":"2024-03-06T07:25:07.931295+0000","last_became_peered":"2024-03-06T07:25:07.931295+0000","last_unstale":"2024-03-06T07:25:08.929026+0000","last_undegraded":"2024-03-06T07:25:08.929026+0000","last_fullsized":"2024-03-06T07:25:08.929026+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:53.073182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386333+0000","last_change":"2024-03-06T07:25:07.930562+0000","last_active":"2024-03-06T07:25:09.386333+0000","last_peered":"2024-03-06T07:25:09.386333+0000","last_clean":"2024-03-06T07:25:09.386333+0000","last_became_active":"2024-03-06T07:25:07.930272+0000","last_became_peered":"2024-03-06T07:25:07.930272+0000","last_unstale":"2024-03-06T07:25:09.386333+0000","last_undegraded":"2024-03-06T07:25:09.386333+0000","last_fullsized":"2024-03-06T07:25:09.386333+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:52:32.198826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928694+0000","last_change":"2024-03-06T07:25:07.930981+0000","last_active":"2024-03-06T07:25:08.928694+0000","last_peered":"2024-03-06T07:25:08.928694+0000","last_clean":"2024-03-06T07:25:08.928694+0000","last_became_active":"2024-03-06T07:25:07.930638+0000","last_became_peered":"2024-03-06T07:25:07.930638+0000","last_unstale":"2024-03-06T07:25:08.928694+0000","last_undegraded":"2024-03-06T07:25:08.928694+0000","last_fullsized":"2024-03-06T07:25:08.928694+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:30.060434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924312+0000","last_change":"2024-03-06T07:25:07.924453+0000","last_active":"2024-03-06T07:25:08.924312+0000","last_peered":"2024-03-06T07:25:08.924312+0000","last_clean":"2024-03-06T07:25:08.924312+0000","last_became_active":"2024-03-06T07:25:07.923977+0000","last_became_peered":"2024-03-06T07:25:07.923977+0000","last_unstale":"2024-03-06T07:25:08.924312+0000","last_undegraded":"2024-03-06T07:25:08.924312+0000","last_fullsized":"2024-03-06T07:25:08.924312+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:48.060825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385454+0000","last_change":"2024-03-06T07:25:07.937139+0000","last_active":"2024-03-06T07:25:09.385454+0000","last_peered":"2024-03-06T07:25:09.385454+0000","last_clean":"2024-03-06T07:25:09.385454+0000","last_became_active":"2024-03-06T07:25:07.936846+0000","last_became_peered":"2024-03-06T07:25:07.936846+0000","last_unstale":"2024-03-06T07:25:09.385454+0000","last_undegraded":"2024-03-06T07:25:09.385454+0000","last_fullsized":"2024-03-06T07:25:09.385454+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:49.567115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928500+0000","last_change":"2024-03-06T07:25:07.930031+0000","last_active":"2024-03-06T07:25:08.928500+0000","last_peered":"2024-03-06T07:25:08.928500+0000","last_clean":"2024-03-06T07:25:08.928500+0000","last_became_active":"2024-03-06T07:25:07.928492+0000","last_became_peered":"2024-03-06T07:25:07.928492+0000","last_unstale":"2024-03-06T07:25:08.928500+0000","last_undegraded":"2024-03-06T07:25:08.928500+0000","last_fullsized":"2024-03-06T07:25:08.928500+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:39.824106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385890+0000","last_change":"2024-03-06T07:25:07.943330+0000","last_active":"2024-03-06T07:25:09.385890+0000","last_peered":"2024-03-06T07:25:09.385890+0000","last_clean":"2024-03-06T07:25:09.385890+0000","last_became_active":"2024-03-06T07:25:07.943062+0000","last_became_peered":"2024-03-06T07:25:07.943062+0000","last_unstale":"2024-03-06T07:25:09.385890+0000","last_undegraded":"2024-03-06T07:25:09.385890+0000","last_fullsized":"2024-03-06T07:25:09.385890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:15.018572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.943148+0000","last_change":"2024-03-06T07:25:07.945642+0000","last_active":"2024-03-06T07:25:08.943148+0000","last_peered":"2024-03-06T07:25:08.943148+0000","last_clean":"2024-03-06T07:25:08.943148+0000","last_became_active":"2024-03-06T07:25:07.945365+0000","last_became_peered":"2024-03-06T07:25:07.945365+0000","last_unstale":"2024-03-06T07:25:08.943148+0000","last_undegraded":"2024-03-06T07:25:08.943148+0000","last_fullsized":"2024-03-06T07:25:08.943148+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:15.441553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386471+0000","last_change":"2024-03-06T07:25:07.931975+0000","last_active":"2024-03-06T07:25:09.386471+0000","last_peered":"2024-03-06T07:25:09.386471+0000","last_clean":"2024-03-06T07:25:09.386471+0000","last_became_active":"2024-03-06T07:25:07.931704+0000","last_became_peered":"2024-03-06T07:25:07.931704+0000","last_unstale":"2024-03-06T07:25:09.386471+0000","last_undegraded":"2024-03-06T07:25:09.386471+0000","last_fullsized":"2024-03-06T07:25:09.386471+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:17.780648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386476+0000","last_change":"2024-03-06T07:25:07.935027+0000","last_active":"2024-03-06T07:25:09.386476+0000","last_peered":"2024-03-06T07:25:09.386476+0000","last_clean":"2024-03-06T07:25:09.386476+0000","last_became_active":"2024-03-06T07:25:07.934739+0000","last_became_peered":"2024-03-06T07:25:07.934739+0000","last_unstale":"2024-03-06T07:25:09.386476+0000","last_undegraded":"2024-03-06T07:25:09.386476+0000","last_fullsized":"2024-03-06T07:25:09.386476+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:26:35.906637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.939991+0000","last_change":"2024-03-06T07:25:07.942941+0000","last_active":"2024-03-06T07:25:08.939991+0000","last_peered":"2024-03-06T07:25:08.939991+0000","last_clean":"2024-03-06T07:25:08.939991+0000","last_became_active":"2024-03-06T07:25:07.942642+0000","last_became_peered":"2024-03-06T07:25:07.942642+0000","last_unstale":"2024-03-06T07:25:08.939991+0000","last_undegraded":"2024-03-06T07:25:08.939991+0000","last_fullsized":"2024-03-06T07:25:08.939991+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:09:35.160252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928143+0000","last_change":"2024-03-06T07:25:07.942261+0000","last_active":"2024-03-06T07:25:08.928143+0000","last_peered":"2024-03-06T07:25:08.928143+0000","last_clean":"2024-03-06T07:25:08.928143+0000","last_became_active":"2024-03-06T07:25:07.941984+0000","last_became_peered":"2024-03-06T07:25:07.941984+0000","last_unstale":"2024-03-06T07:25:08.928143+0000","last_undegraded":"2024-03-06T07:25:08.928143+0000","last_fullsized":"2024-03-06T07:25:08.928143+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:39.036503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385526+0000","last_change":"2024-03-06T07:25:07.932464+0000","last_active":"2024-03-06T07:25:09.385526+0000","last_peered":"2024-03-06T07:25:09.385526+0000","last_clean":"2024-03-06T07:25:09.385526+0000","last_became_active":"2024-03-06T07:25:07.932140+0000","last_became_peered":"2024-03-06T07:25:07.932140+0000","last_unstale":"2024-03-06T07:25:09.385526+0000","last_undegraded":"2024-03-06T07:25:09.385526+0000","last_fullsized":"2024-03-06T07:25:09.385526+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:36:07.841978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940353+0000","last_change":"2024-03-06T07:25:07.940592+0000","last_active":"2024-03-06T07:25:08.940353+0000","last_peered":"2024-03-06T07:25:08.940353+0000","last_clean":"2024-03-06T07:25:08.940353+0000","last_became_active":"2024-03-06T07:25:07.938291+0000","last_became_peered":"2024-03-06T07:25:07.938291+0000","last_unstale":"2024-03-06T07:25:08.940353+0000","last_undegraded":"2024-03-06T07:25:08.940353+0000","last_fullsized":"2024-03-06T07:25:08.940353+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:37.517195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940260+0000","last_change":"2024-03-06T07:25:07.939435+0000","last_active":"2024-03-06T07:25:08.940260+0000","last_peered":"2024-03-06T07:25:08.940260+0000","last_clean":"2024-03-06T07:25:08.940260+0000","last_became_active":"2024-03-06T07:25:07.938758+0000","last_became_peered":"2024-03-06T07:25:07.938758+0000","last_unstale":"2024-03-06T07:25:08.940260+0000","last_undegraded":"2024-03-06T07:25:08.940260+0000","last_fullsized":"2024-03-06T07:25:08.940260+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:51.935334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386732+0000","last_change":"2024-03-06T07:25:07.943323+0000","last_active":"2024-03-06T07:25:09.386732+0000","last_peered":"2024-03-06T07:25:09.386732+0000","last_clean":"2024-03-06T07:25:09.386732+0000","last_became_active":"2024-03-06T07:25:07.943042+0000","last_became_peered":"2024-03-06T07:25:07.943042+0000","last_unstale":"2024-03-06T07:25:09.386732+0000","last_undegraded":"2024-03-06T07:25:09.386732+0000","last_fullsized":"2024-03-06T07:25:09.386732+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:20:27.967179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385696+0000","last_change":"2024-03-06T07:25:07.928447+0000","last_active":"2024-03-06T07:25:09.385696+0000","last_peered":"2024-03-06T07:25:09.385696+0000","last_clean":"2024-03-06T07:25:09.385696+0000","last_became_active":"2024-03-06T07:25:07.928184+0000","last_became_peered":"2024-03-06T07:25:07.928184+0000","last_unstale":"2024-03-06T07:25:09.385696+0000","last_undegraded":"2024-03-06T07:25:09.385696+0000","last_fullsized":"2024-03-06T07:25:09.385696+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:28:05.757987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928178+0000","last_change":"2024-03-06T07:25:07.938600+0000","last_active":"2024-03-06T07:25:08.928178+0000","last_peered":"2024-03-06T07:25:08.928178+0000","last_clean":"2024-03-06T07:25:08.928178+0000","last_became_active":"2024-03-06T07:25:07.938015+0000","last_became_peered":"2024-03-06T07:25:07.938015+0000","last_unstale":"2024-03-06T07:25:08.928178+0000","last_undegraded":"2024-03-06T07:25:08.928178+0000","last_fullsized":"2024-03-06T07:25:08.928178+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:35:37.743454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385465+0000","last_change":"2024-03-06T07:25:07.927749+0000","last_active":"2024-03-06T07:25:09.385465+0000","last_peered":"2024-03-06T07:25:09.385465+0000","last_clean":"2024-03-06T07:25:09.385465+0000","last_became_active":"2024-03-06T07:25:07.927455+0000","last_became_peered":"2024-03-06T07:25:07.927455+0000","last_unstale":"2024-03-06T07:25:09.385465+0000","last_undegraded":"2024-03-06T07:25:09.385465+0000","last_fullsized":"2024-03-06T07:25:09.385465+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:14.326850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386062+0000","last_change":"2024-03-06T07:25:07.937857+0000","last_active":"2024-03-06T07:25:09.386062+0000","last_peered":"2024-03-06T07:25:09.386062+0000","last_clean":"2024-03-06T07:25:09.386062+0000","last_became_active":"2024-03-06T07:25:07.937502+0000","last_became_peered":"2024-03-06T07:25:07.937502+0000","last_unstale":"2024-03-06T07:25:09.386062+0000","last_undegraded":"2024-03-06T07:25:09.386062+0000","last_fullsized":"2024-03-06T07:25:09.386062+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:08:03.946222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928431+0000","last_change":"2024-03-06T07:25:07.934865+0000","last_active":"2024-03-06T07:25:08.928431+0000","last_peered":"2024-03-06T07:25:08.928431+0000","last_clean":"2024-03-06T07:25:08.928431+0000","last_became_active":"2024-03-06T07:25:07.934358+0000","last_became_peered":"2024-03-06T07:25:07.934358+0000","last_unstale":"2024-03-06T07:25:08.928431+0000","last_undegraded":"2024-03-06T07:25:08.928431+0000","last_fullsized":"2024-03-06T07:25:08.928431+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:39.824106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924542+0000","last_change":"2024-03-06T07:25:07.933510+0000","last_active":"2024-03-06T07:25:08.924542+0000","last_peered":"2024-03-06T07:25:08.924542+0000","last_clean":"2024-03-06T07:25:08.924542+0000","last_became_active":"2024-03-06T07:25:07.933146+0000","last_became_peered":"2024-03-06T07:25:07.933146+0000","last_unstale":"2024-03-06T07:25:08.924542+0000","last_undegraded":"2024-03-06T07:25:08.924542+0000","last_fullsized":"2024-03-06T07:25:08.924542+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:14:29.976656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"135'4","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386566+0000","last_change":"2024-03-06T07:25:06.926918+0000","last_active":"2024-03-06T07:25:09.386566+0000","last_peered":"2024-03-06T07:25:09.386566+0000","last_clean":"2024-03-06T07:25:09.386566+0000","last_became_active":"2024-03-06T07:25:06.926475+0000","last_became_peered":"2024-03-06T07:25:06.926475+0000","last_unstale":"2024-03-06T07:25:09.386566+0000","last_undegraded":"2024-03-06T07:25:09.386566+0000","last_fullsized":"2024-03-06T07:25:09.386566+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:44.439984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924530+0000","last_change":"2024-03-06T07:25:07.922854+0000","last_active":"2024-03-06T07:25:08.924530+0000","last_peered":"2024-03-06T07:25:08.924530+0000","last_clean":"2024-03-06T07:25:08.924530+0000","last_became_active":"2024-03-06T07:25:07.922582+0000","last_became_peered":"2024-03-06T07:25:07.922582+0000","last_unstale":"2024-03-06T07:25:08.924530+0000","last_undegraded":"2024-03-06T07:25:08.924530+0000","last_fullsized":"2024-03-06T07:25:08.924530+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:03.856688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942848+0000","last_change":"2024-03-06T07:25:06.933126+0000","last_active":"2024-03-06T07:25:08.942848+0000","last_peered":"2024-03-06T07:25:08.942848+0000","last_clean":"2024-03-06T07:25:08.942848+0000","last_became_active":"2024-03-06T07:25:06.932843+0000","last_became_peered":"2024-03-06T07:25:06.932843+0000","last_unstale":"2024-03-06T07:25:08.942848+0000","last_undegraded":"2024-03-06T07:25:08.942848+0000","last_fullsized":"2024-03-06T07:25:08.942848+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:54.261775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928478+0000","last_change":"2024-03-06T07:25:07.937602+0000","last_active":"2024-03-06T07:25:08.928478+0000","last_peered":"2024-03-06T07:25:08.928478+0000","last_clean":"2024-03-06T07:25:08.928478+0000","last_became_active":"2024-03-06T07:25:07.937346+0000","last_became_peered":"2024-03-06T07:25:07.937346+0000","last_unstale":"2024-03-06T07:25:08.928478+0000","last_undegraded":"2024-03-06T07:25:08.928478+0000","last_fullsized":"2024-03-06T07:25:08.928478+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:05:30.060434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.943258+0000","last_change":"2024-03-06T07:25:06.932116+0000","last_active":"2024-03-06T07:25:08.943258+0000","last_peered":"2024-03-06T07:25:08.943258+0000","last_clean":"2024-03-06T07:25:08.943258+0000","last_became_active":"2024-03-06T07:25:06.931835+0000","last_became_peered":"2024-03-06T07:25:06.931835+0000","last_unstale":"2024-03-06T07:25:08.943258+0000","last_undegraded":"2024-03-06T07:25:08.943258+0000","last_fullsized":"2024-03-06T07:25:08.943258+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:20.516698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.943024+0000","last_change":"2024-03-06T07:25:07.945021+0000","last_active":"2024-03-06T07:25:08.943024+0000","last_peered":"2024-03-06T07:25:08.943024+0000","last_clean":"2024-03-06T07:25:08.943024+0000","last_became_active":"2024-03-06T07:25:07.944614+0000","last_became_peered":"2024-03-06T07:25:07.944614+0000","last_unstale":"2024-03-06T07:25:08.943024+0000","last_undegraded":"2024-03-06T07:25:08.943024+0000","last_fullsized":"2024-03-06T07:25:08.943024+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:53.095601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924610+0000","last_change":"2024-03-06T07:25:06.936612+0000","last_active":"2024-03-06T07:25:08.924610+0000","last_peered":"2024-03-06T07:25:08.924610+0000","last_clean":"2024-03-06T07:25:08.924610+0000","last_became_active":"2024-03-06T07:25:06.936165+0000","last_became_peered":"2024-03-06T07:25:06.936165+0000","last_unstale":"2024-03-06T07:25:08.924610+0000","last_undegraded":"2024-03-06T07:25:08.924610+0000","last_fullsized":"2024-03-06T07:25:08.924610+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:38.030419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385885+0000","last_change":"2024-03-06T07:25:07.930781+0000","last_active":"2024-03-06T07:25:09.385885+0000","last_peered":"2024-03-06T07:25:09.385885+0000","last_clean":"2024-03-06T07:25:09.385885+0000","last_became_active":"2024-03-06T07:25:07.930504+0000","last_became_peered":"2024-03-06T07:25:07.930504+0000","last_unstale":"2024-03-06T07:25:09.385885+0000","last_undegraded":"2024-03-06T07:25:09.385885+0000","last_fullsized":"2024-03-06T07:25:09.385885+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:27:34.239647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.927971+0000","last_change":"2024-03-06T07:25:06.932157+0000","last_active":"2024-03-06T07:25:08.927971+0000","last_peered":"2024-03-06T07:25:08.927971+0000","last_clean":"2024-03-06T07:25:08.927971+0000","last_became_active":"2024-03-06T07:25:06.931866+0000","last_became_peered":"2024-03-06T07:25:06.931866+0000","last_unstale":"2024-03-06T07:25:08.927971+0000","last_undegraded":"2024-03-06T07:25:08.927971+0000","last_fullsized":"2024-03-06T07:25:08.927971+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:11.699676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.927890+0000","last_change":"2024-03-06T07:25:07.928874+0000","last_active":"2024-03-06T07:25:08.927890+0000","last_peered":"2024-03-06T07:25:08.927890+0000","last_clean":"2024-03-06T07:25:08.927890+0000","last_became_active":"2024-03-06T07:25:07.928600+0000","last_became_peered":"2024-03-06T07:25:07.928600+0000","last_unstale":"2024-03-06T07:25:08.927890+0000","last_undegraded":"2024-03-06T07:25:08.927890+0000","last_fullsized":"2024-03-06T07:25:08.927890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:52:39.036503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386327+0000","last_change":"2024-03-06T07:25:06.932382+0000","last_active":"2024-03-06T07:25:09.386327+0000","last_peered":"2024-03-06T07:25:09.386327+0000","last_clean":"2024-03-06T07:25:09.386327+0000","last_became_active":"2024-03-06T07:25:06.932043+0000","last_became_peered":"2024-03-06T07:25:06.932043+0000","last_unstale":"2024-03-06T07:25:09.386327+0000","last_undegraded":"2024-03-06T07:25:09.386327+0000","last_fullsized":"2024-03-06T07:25:09.386327+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:14:28.970571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924287+0000","last_change":"2024-03-06T07:25:07.925973+0000","last_active":"2024-03-06T07:25:08.924287+0000","last_peered":"2024-03-06T07:25:08.924287+0000","last_clean":"2024-03-06T07:25:08.924287+0000","last_became_active":"2024-03-06T07:25:07.925639+0000","last_became_peered":"2024-03-06T07:25:07.925639+0000","last_unstale":"2024-03-06T07:25:08.924287+0000","last_undegraded":"2024-03-06T07:25:08.924287+0000","last_fullsized":"2024-03-06T07:25:08.924287+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:36.866991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.939925+0000","last_change":"2024-03-06T07:25:06.934761+0000","last_active":"2024-03-06T07:25:08.939925+0000","last_peered":"2024-03-06T07:25:08.939925+0000","last_clean":"2024-03-06T07:25:08.939925+0000","last_became_active":"2024-03-06T07:25:06.934441+0000","last_became_peered":"2024-03-06T07:25:06.934441+0000","last_unstale":"2024-03-06T07:25:08.939925+0000","last_undegraded":"2024-03-06T07:25:08.939925+0000","last_fullsized":"2024-03-06T07:25:08.939925+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:58:25.891777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.939989+0000","last_change":"2024-03-06T07:25:07.942609+0000","last_active":"2024-03-06T07:25:08.939989+0000","last_peered":"2024-03-06T07:25:08.939989+0000","last_clean":"2024-03-06T07:25:08.939989+0000","last_became_active":"2024-03-06T07:25:07.942048+0000","last_became_peered":"2024-03-06T07:25:07.942048+0000","last_unstale":"2024-03-06T07:25:08.939989+0000","last_undegraded":"2024-03-06T07:25:08.939989+0000","last_fullsized":"2024-03-06T07:25:08.939989+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:28.549412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385456+0000","last_change":"2024-03-06T07:25:06.935857+0000","last_active":"2024-03-06T07:25:09.385456+0000","last_peered":"2024-03-06T07:25:09.385456+0000","last_clean":"2024-03-06T07:25:09.385456+0000","last_became_active":"2024-03-06T07:25:06.935520+0000","last_became_peered":"2024-03-06T07:25:06.935520+0000","last_unstale":"2024-03-06T07:25:09.385456+0000","last_undegraded":"2024-03-06T07:25:09.385456+0000","last_fullsized":"2024-03-06T07:25:09.385456+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:03.399529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928840+0000","last_change":"2024-03-06T07:25:07.936915+0000","last_active":"2024-03-06T07:25:08.928840+0000","last_peered":"2024-03-06T07:25:08.928840+0000","last_clean":"2024-03-06T07:25:08.928840+0000","last_became_active":"2024-03-06T07:25:07.936609+0000","last_became_peered":"2024-03-06T07:25:07.936609+0000","last_unstale":"2024-03-06T07:25:08.928840+0000","last_undegraded":"2024-03-06T07:25:08.928840+0000","last_fullsized":"2024-03-06T07:25:08.928840+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:00:34.161758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942992+0000","last_change":"2024-03-06T07:25:06.928957+0000","last_active":"2024-03-06T07:25:08.942992+0000","last_peered":"2024-03-06T07:25:08.942992+0000","last_clean":"2024-03-06T07:25:08.942992+0000","last_became_active":"2024-03-06T07:25:06.928262+0000","last_became_peered":"2024-03-06T07:25:06.928262+0000","last_unstale":"2024-03-06T07:25:08.942992+0000","last_undegraded":"2024-03-06T07:25:08.942992+0000","last_fullsized":"2024-03-06T07:25:08.942992+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:09:34.154167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385693+0000","last_change":"2024-03-06T07:25:07.935484+0000","last_active":"2024-03-06T07:25:09.385693+0000","last_peered":"2024-03-06T07:25:09.385693+0000","last_clean":"2024-03-06T07:25:09.385693+0000","last_became_active":"2024-03-06T07:25:07.934943+0000","last_became_peered":"2024-03-06T07:25:07.934943+0000","last_unstale":"2024-03-06T07:25:09.385693+0000","last_undegraded":"2024-03-06T07:25:09.385693+0000","last_fullsized":"2024-03-06T07:25:09.385693+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:01:41.432965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940444+0000","last_change":"2024-03-06T07:25:06.931781+0000","last_active":"2024-03-06T07:25:08.940444+0000","last_peered":"2024-03-06T07:25:08.940444+0000","last_clean":"2024-03-06T07:25:08.940444+0000","last_became_active":"2024-03-06T07:25:06.931456+0000","last_became_peered":"2024-03-06T07:25:06.931456+0000","last_unstale":"2024-03-06T07:25:08.940444+0000","last_undegraded":"2024-03-06T07:25:08.940444+0000","last_fullsized":"2024-03-06T07:25:08.940444+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:40.641214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385968+0000","last_change":"2024-03-06T07:25:07.942616+0000","last_active":"2024-03-06T07:25:09.385968+0000","last_peered":"2024-03-06T07:25:09.385968+0000","last_clean":"2024-03-06T07:25:09.385968+0000","last_became_active":"2024-03-06T07:25:07.942348+0000","last_became_peered":"2024-03-06T07:25:07.942348+0000","last_unstale":"2024-03-06T07:25:09.385968+0000","last_undegraded":"2024-03-06T07:25:09.385968+0000","last_fullsized":"2024-03-06T07:25:09.385968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:02:17.753240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"135'3","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385780+0000","last_change":"2024-03-06T07:25:06.925504+0000","last_active":"2024-03-06T07:25:09.385780+0000","last_peered":"2024-03-06T07:25:09.385780+0000","last_clean":"2024-03-06T07:25:09.385780+0000","last_became_active":"2024-03-06T07:25:06.925180+0000","last_became_peered":"2024-03-06T07:25:06.925180+0000","last_unstale":"2024-03-06T07:25:09.385780+0000","last_undegraded":"2024-03-06T07:25:09.385780+0000","last_fullsized":"2024-03-06T07:25:09.385780+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:51:46.077014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928439+0000","last_change":"2024-03-06T07:25:07.935081+0000","last_active":"2024-03-06T07:25:08.928439+0000","last_peered":"2024-03-06T07:25:08.928439+0000","last_clean":"2024-03-06T07:25:08.928439+0000","last_became_active":"2024-03-06T07:25:07.934729+0000","last_became_peered":"2024-03-06T07:25:07.934729+0000","last_unstale":"2024-03-06T07:25:08.928439+0000","last_undegraded":"2024-03-06T07:25:08.928439+0000","last_fullsized":"2024-03-06T07:25:08.928439+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:34.879662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386691+0000","last_change":"2024-03-06T07:25:06.926155+0000","last_active":"2024-03-06T07:25:09.386691+0000","last_peered":"2024-03-06T07:25:09.386691+0000","last_clean":"2024-03-06T07:25:09.386691+0000","last_became_active":"2024-03-06T07:25:06.925662+0000","last_became_peered":"2024-03-06T07:25:06.925662+0000","last_unstale":"2024-03-06T07:25:09.386691+0000","last_undegraded":"2024-03-06T07:25:09.386691+0000","last_fullsized":"2024-03-06T07:25:09.386691+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:18:02.850604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924339+0000","last_change":"2024-03-06T07:25:07.923517+0000","last_active":"2024-03-06T07:25:08.924339+0000","last_peered":"2024-03-06T07:25:08.924339+0000","last_clean":"2024-03-06T07:25:08.924339+0000","last_became_active":"2024-03-06T07:25:07.923237+0000","last_became_peered":"2024-03-06T07:25:07.923237+0000","last_unstale":"2024-03-06T07:25:08.924339+0000","last_undegraded":"2024-03-06T07:25:08.924339+0000","last_fullsized":"2024-03-06T07:25:08.924339+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:38.930530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928887+0000","last_change":"2024-03-06T07:25:06.936715+0000","last_active":"2024-03-06T07:25:08.928887+0000","last_peered":"2024-03-06T07:25:08.928887+0000","last_clean":"2024-03-06T07:25:08.928887+0000","last_became_active":"2024-03-06T07:25:06.936361+0000","last_became_peered":"2024-03-06T07:25:06.936361+0000","last_unstale":"2024-03-06T07:25:08.928887+0000","last_undegraded":"2024-03-06T07:25:08.928887+0000","last_fullsized":"2024-03-06T07:25:08.928887+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:26.429287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386312+0000","last_change":"2024-03-06T07:25:07.931386+0000","last_active":"2024-03-06T07:25:09.386312+0000","last_peered":"2024-03-06T07:25:09.386312+0000","last_clean":"2024-03-06T07:25:09.386312+0000","last_became_active":"2024-03-06T07:25:07.931136+0000","last_became_peered":"2024-03-06T07:25:07.931136+0000","last_unstale":"2024-03-06T07:25:09.386312+0000","last_undegraded":"2024-03-06T07:25:09.386312+0000","last_fullsized":"2024-03-06T07:25:09.386312+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:44:38.539087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928156+0000","last_change":"2024-03-06T07:25:06.934580+0000","last_active":"2024-03-06T07:25:08.928156+0000","last_peered":"2024-03-06T07:25:08.928156+0000","last_clean":"2024-03-06T07:25:08.928156+0000","last_became_active":"2024-03-06T07:25:06.934296+0000","last_became_peered":"2024-03-06T07:25:06.934296+0000","last_unstale":"2024-03-06T07:25:08.928156+0000","last_undegraded":"2024-03-06T07:25:08.928156+0000","last_fullsized":"2024-03-06T07:25:08.928156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:46:55.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924357+0000","last_change":"2024-03-06T07:25:07.926768+0000","last_active":"2024-03-06T07:25:08.924357+0000","last_peered":"2024-03-06T07:25:08.924357+0000","last_clean":"2024-03-06T07:25:08.924357+0000","last_became_active":"2024-03-06T07:25:07.926457+0000","last_became_peered":"2024-03-06T07:25:07.926457+0000","last_unstale":"2024-03-06T07:25:08.924357+0000","last_undegraded":"2024-03-06T07:25:08.924357+0000","last_fullsized":"2024-03-06T07:25:08.924357+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:42.665105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"135'8","reported_seq":22,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.939753+0000","last_change":"2024-03-06T07:25:06.934527+0000","last_active":"2024-03-06T07:25:08.939753+0000","last_peered":"2024-03-06T07:25:08.939753+0000","last_clean":"2024-03-06T07:25:08.939753+0000","last_became_active":"2024-03-06T07:25:06.934249+0000","last_became_peered":"2024-03-06T07:25:06.934249+0000","last_unstale":"2024-03-06T07:25:08.939753+0000","last_undegraded":"2024-03-06T07:25:08.939753+0000","last_fullsized":"2024-03-06T07:25:08.939753+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:14:26.429287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942817+0000","last_change":"2024-03-06T07:25:07.936356+0000","last_active":"2024-03-06T07:25:08.942817+0000","last_peered":"2024-03-06T07:25:08.942817+0000","last_clean":"2024-03-06T07:25:08.942817+0000","last_became_active":"2024-03-06T07:25:07.936100+0000","last_became_peered":"2024-03-06T07:25:07.936100+0000","last_unstale":"2024-03-06T07:25:08.942817+0000","last_undegraded":"2024-03-06T07:25:08.942817+0000","last_fullsized":"2024-03-06T07:25:08.942817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:14:29.976656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385509+0000","last_change":"2024-03-06T07:25:06.924175+0000","last_active":"2024-03-06T07:25:09.385509+0000","last_peered":"2024-03-06T07:25:09.385509+0000","last_clean":"2024-03-06T07:25:09.385509+0000","last_became_active":"2024-03-06T07:25:06.923894+0000","last_became_peered":"2024-03-06T07:25:06.923894+0000","last_unstale":"2024-03-06T07:25:09.385509+0000","last_undegraded":"2024-03-06T07:25:09.385509+0000","last_fullsized":"2024-03-06T07:25:09.385509+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:40:33.709788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.943193+0000","last_change":"2024-03-06T07:25:07.946324+0000","last_active":"2024-03-06T07:25:08.943193+0000","last_peered":"2024-03-06T07:25:08.943193+0000","last_clean":"2024-03-06T07:25:08.943193+0000","last_became_active":"2024-03-06T07:25:07.946079+0000","last_became_peered":"2024-03-06T07:25:07.946079+0000","last_unstale":"2024-03-06T07:25:08.943193+0000","last_undegraded":"2024-03-06T07:25:08.943193+0000","last_fullsized":"2024-03-06T07:25:08.943193+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:39.145264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942961+0000","last_change":"2024-03-06T07:25:06.926576+0000","last_active":"2024-03-06T07:25:08.942961+0000","last_peered":"2024-03-06T07:25:08.942961+0000","last_clean":"2024-03-06T07:25:08.942961+0000","last_became_active":"2024-03-06T07:25:06.926315+0000","last_became_peered":"2024-03-06T07:25:06.926315+0000","last_unstale":"2024-03-06T07:25:08.942961+0000","last_undegraded":"2024-03-06T07:25:08.942961+0000","last_fullsized":"2024-03-06T07:25:08.942961+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:07:50.929249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940294+0000","last_change":"2024-03-06T07:25:07.939667+0000","last_active":"2024-03-06T07:25:08.940294+0000","last_peered":"2024-03-06T07:25:08.940294+0000","last_clean":"2024-03-06T07:25:08.940294+0000","last_became_active":"2024-03-06T07:25:07.939075+0000","last_became_peered":"2024-03-06T07:25:07.939075+0000","last_unstale":"2024-03-06T07:25:08.940294+0000","last_undegraded":"2024-03-06T07:25:08.940294+0000","last_fullsized":"2024-03-06T07:25:08.940294+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:20:21.522783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924539+0000","last_change":"2024-03-06T07:25:06.935036+0000","last_active":"2024-03-06T07:25:08.924539+0000","last_peered":"2024-03-06T07:25:08.924539+0000","last_clean":"2024-03-06T07:25:08.924539+0000","last_became_active":"2024-03-06T07:25:06.934723+0000","last_became_peered":"2024-03-06T07:25:06.934723+0000","last_unstale":"2024-03-06T07:25:08.924539+0000","last_undegraded":"2024-03-06T07:25:08.924539+0000","last_fullsized":"2024-03-06T07:25:08.924539+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:06:09.273119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940452+0000","last_change":"2024-03-06T07:25:07.935046+0000","last_active":"2024-03-06T07:25:08.940452+0000","last_peered":"2024-03-06T07:25:08.940452+0000","last_clean":"2024-03-06T07:25:08.940452+0000","last_became_active":"2024-03-06T07:25:07.934716+0000","last_became_peered":"2024-03-06T07:25:07.934716+0000","last_unstale":"2024-03-06T07:25:08.940452+0000","last_undegraded":"2024-03-06T07:25:08.940452+0000","last_fullsized":"2024-03-06T07:25:08.940452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:57:48.060825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.929122+0000","last_change":"2024-03-06T07:25:06.937027+0000","last_active":"2024-03-06T07:25:08.929122+0000","last_peered":"2024-03-06T07:25:08.929122+0000","last_clean":"2024-03-06T07:25:08.929122+0000","last_became_active":"2024-03-06T07:25:06.936721+0000","last_became_peered":"2024-03-06T07:25:06.936721+0000","last_unstale":"2024-03-06T07:25:08.929122+0000","last_undegraded":"2024-03-06T07:25:08.929122+0000","last_fullsized":"2024-03-06T07:25:08.929122+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:50.432110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942716+0000","last_change":"2024-03-06T07:25:07.937051+0000","last_active":"2024-03-06T07:25:08.942716+0000","last_peered":"2024-03-06T07:25:08.942716+0000","last_clean":"2024-03-06T07:25:08.942716+0000","last_became_active":"2024-03-06T07:25:07.936782+0000","last_became_peered":"2024-03-06T07:25:07.936782+0000","last_unstale":"2024-03-06T07:25:08.942716+0000","last_undegraded":"2024-03-06T07:25:08.942716+0000","last_fullsized":"2024-03-06T07:25:08.942716+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:32.567857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386315+0000","last_change":"2024-03-06T07:25:06.930306+0000","last_active":"2024-03-06T07:25:09.386315+0000","last_peered":"2024-03-06T07:25:09.386315+0000","last_clean":"2024-03-06T07:25:09.386315+0000","last_became_active":"2024-03-06T07:25:06.929975+0000","last_became_peered":"2024-03-06T07:25:06.929975+0000","last_unstale":"2024-03-06T07:25:09.386315+0000","last_undegraded":"2024-03-06T07:25:09.386315+0000","last_fullsized":"2024-03-06T07:25:09.386315+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:46:56.094353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924548+0000","last_change":"2024-03-06T07:25:07.941382+0000","last_active":"2024-03-06T07:25:08.924548+0000","last_peered":"2024-03-06T07:25:08.924548+0000","last_clean":"2024-03-06T07:25:08.924548+0000","last_became_active":"2024-03-06T07:25:07.941106+0000","last_became_peered":"2024-03-06T07:25:07.941106+0000","last_unstale":"2024-03-06T07:25:08.924548+0000","last_undegraded":"2024-03-06T07:25:08.924548+0000","last_fullsized":"2024-03-06T07:25:08.924548+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:04.405613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.927873+0000","last_change":"2024-03-06T07:25:06.930143+0000","last_active":"2024-03-06T07:25:08.927873+0000","last_peered":"2024-03-06T07:25:08.927873+0000","last_clean":"2024-03-06T07:25:08.927873+0000","last_became_active":"2024-03-06T07:25:06.929880+0000","last_became_peered":"2024-03-06T07:25:06.929880+0000","last_unstale":"2024-03-06T07:25:08.927873+0000","last_undegraded":"2024-03-06T07:25:08.927873+0000","last_fullsized":"2024-03-06T07:25:08.927873+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:07.314878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942748+0000","last_change":"2024-03-06T07:25:07.932337+0000","last_active":"2024-03-06T07:25:08.942748+0000","last_peered":"2024-03-06T07:25:08.942748+0000","last_clean":"2024-03-06T07:25:08.942748+0000","last_became_active":"2024-03-06T07:25:07.932063+0000","last_became_peered":"2024-03-06T07:25:07.932063+0000","last_unstale":"2024-03-06T07:25:08.942748+0000","last_undegraded":"2024-03-06T07:25:08.942748+0000","last_fullsized":"2024-03-06T07:25:08.942748+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:22:04.405613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385902+0000","last_change":"2024-03-06T07:25:06.930314+0000","last_active":"2024-03-06T07:25:09.385902+0000","last_peered":"2024-03-06T07:25:09.385902+0000","last_clean":"2024-03-06T07:25:09.385902+0000","last_became_active":"2024-03-06T07:25:06.929990+0000","last_became_peered":"2024-03-06T07:25:06.929990+0000","last_unstale":"2024-03-06T07:25:09.385902+0000","last_undegraded":"2024-03-06T07:25:09.385902+0000","last_fullsized":"2024-03-06T07:25:09.385902+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:34.425702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924524+0000","last_change":"2024-03-06T07:25:07.928627+0000","last_active":"2024-03-06T07:25:08.924524+0000","last_peered":"2024-03-06T07:25:08.924524+0000","last_clean":"2024-03-06T07:25:08.924524+0000","last_became_active":"2024-03-06T07:25:07.928334+0000","last_became_peered":"2024-03-06T07:25:07.928334+0000","last_unstale":"2024-03-06T07:25:08.924524+0000","last_undegraded":"2024-03-06T07:25:08.924524+0000","last_fullsized":"2024-03-06T07:25:08.924524+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:32.567857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928730+0000","last_change":"2024-03-06T07:25:06.925110+0000","last_active":"2024-03-06T07:25:08.928730+0000","last_peered":"2024-03-06T07:25:08.928730+0000","last_clean":"2024-03-06T07:25:08.928730+0000","last_became_active":"2024-03-06T07:25:06.924842+0000","last_became_peered":"2024-03-06T07:25:06.924842+0000","last_unstale":"2024-03-06T07:25:08.928730+0000","last_undegraded":"2024-03-06T07:25:08.928730+0000","last_fullsized":"2024-03-06T07:25:08.928730+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:32.439585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386229+0000","last_change":"2024-03-06T07:25:07.940191+0000","last_active":"2024-03-06T07:25:09.386229+0000","last_peered":"2024-03-06T07:25:09.386229+0000","last_clean":"2024-03-06T07:25:09.386229+0000","last_became_active":"2024-03-06T07:25:07.939943+0000","last_became_peered":"2024-03-06T07:25:07.939943+0000","last_unstale":"2024-03-06T07:25:09.386229+0000","last_undegraded":"2024-03-06T07:25:09.386229+0000","last_fullsized":"2024-03-06T07:25:09.386229+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:11.208900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928622+0000","last_change":"2024-03-06T07:25:06.936235+0000","last_active":"2024-03-06T07:25:08.928622+0000","last_peered":"2024-03-06T07:25:08.928622+0000","last_clean":"2024-03-06T07:25:08.928622+0000","last_became_active":"2024-03-06T07:25:06.935934+0000","last_became_peered":"2024-03-06T07:25:06.935934+0000","last_unstale":"2024-03-06T07:25:08.928622+0000","last_undegraded":"2024-03-06T07:25:08.928622+0000","last_fullsized":"2024-03-06T07:25:08.928622+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:29:06.450468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928340+0000","last_change":"2024-03-06T07:25:07.930732+0000","last_active":"2024-03-06T07:25:08.928340+0000","last_peered":"2024-03-06T07:25:08.928340+0000","last_clean":"2024-03-06T07:25:08.928340+0000","last_became_active":"2024-03-06T07:25:07.930468+0000","last_became_peered":"2024-03-06T07:25:07.930468+0000","last_unstale":"2024-03-06T07:25:08.928340+0000","last_undegraded":"2024-03-06T07:25:08.928340+0000","last_fullsized":"2024-03-06T07:25:08.928340+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:34.879662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"136'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940551+0000","last_change":"2024-03-06T07:25:06.933630+0000","last_active":"2024-03-06T07:25:08.940551+0000","last_peered":"2024-03-06T07:25:08.940551+0000","last_clean":"2024-03-06T07:25:08.940551+0000","last_became_active":"2024-03-06T07:25:06.933254+0000","last_became_peered":"2024-03-06T07:25:06.933254+0000","last_unstale":"2024-03-06T07:25:08.940551+0000","last_undegraded":"2024-03-06T07:25:08.940551+0000","last_fullsized":"2024-03-06T07:25:08.940551+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:33.848509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940428+0000","last_change":"2024-03-06T07:25:07.943107+0000","last_active":"2024-03-06T07:25:08.940428+0000","last_peered":"2024-03-06T07:25:08.940428+0000","last_clean":"2024-03-06T07:25:08.940428+0000","last_became_active":"2024-03-06T07:25:07.942817+0000","last_became_peered":"2024-03-06T07:25:07.942817+0000","last_unstale":"2024-03-06T07:25:08.940428+0000","last_undegraded":"2024-03-06T07:25:08.940428+0000","last_fullsized":"2024-03-06T07:25:08.940428+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:36.866991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924642+0000","last_change":"2024-03-06T07:25:06.936426+0000","last_active":"2024-03-06T07:25:08.924642+0000","last_peered":"2024-03-06T07:25:08.924642+0000","last_clean":"2024-03-06T07:25:08.924642+0000","last_became_active":"2024-03-06T07:25:06.936160+0000","last_became_peered":"2024-03-06T07:25:06.936160+0000","last_unstale":"2024-03-06T07:25:08.924642+0000","last_undegraded":"2024-03-06T07:25:08.924642+0000","last_fullsized":"2024-03-06T07:25:08.924642+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:57:33.873577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386486+0000","last_change":"2024-03-06T07:25:07.931970+0000","last_active":"2024-03-06T07:25:09.386486+0000","last_peered":"2024-03-06T07:25:09.386486+0000","last_clean":"2024-03-06T07:25:09.386486+0000","last_became_active":"2024-03-06T07:25:07.931705+0000","last_became_peered":"2024-03-06T07:25:07.931705+0000","last_unstale":"2024-03-06T07:25:09.386486+0000","last_undegraded":"2024-03-06T07:25:09.386486+0000","last_fullsized":"2024-03-06T07:25:09.386486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:59.569132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386192+0000","last_change":"2024-03-06T07:25:06.930511+0000","last_active":"2024-03-06T07:25:09.386192+0000","last_peered":"2024-03-06T07:25:09.386192+0000","last_clean":"2024-03-06T07:25:09.386192+0000","last_became_active":"2024-03-06T07:25:06.930205+0000","last_became_peered":"2024-03-06T07:25:06.930205+0000","last_unstale":"2024-03-06T07:25:09.386192+0000","last_undegraded":"2024-03-06T07:25:09.386192+0000","last_fullsized":"2024-03-06T07:25:09.386192+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:31.561773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928605+0000","last_change":"2024-03-06T07:25:07.925933+0000","last_active":"2024-03-06T07:25:08.928605+0000","last_peered":"2024-03-06T07:25:08.928605+0000","last_clean":"2024-03-06T07:25:08.928605+0000","last_became_active":"2024-03-06T07:25:07.925644+0000","last_became_peered":"2024-03-06T07:25:07.925644+0000","last_unstale":"2024-03-06T07:25:08.928605+0000","last_undegraded":"2024-03-06T07:25:08.928605+0000","last_fullsized":"2024-03-06T07:25:08.928605+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:51:22.235096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940000+0000","last_change":"2024-03-06T07:25:06.935597+0000","last_active":"2024-03-06T07:25:08.940000+0000","last_peered":"2024-03-06T07:25:08.940000+0000","last_clean":"2024-03-06T07:25:08.940000+0000","last_became_active":"2024-03-06T07:25:06.935195+0000","last_became_peered":"2024-03-06T07:25:06.935195+0000","last_unstale":"2024-03-06T07:25:08.940000+0000","last_undegraded":"2024-03-06T07:25:08.940000+0000","last_fullsized":"2024-03-06T07:25:08.940000+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:51.366460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928053+0000","last_change":"2024-03-06T07:25:07.930794+0000","last_active":"2024-03-06T07:25:08.928053+0000","last_peered":"2024-03-06T07:25:08.928053+0000","last_clean":"2024-03-06T07:25:08.928053+0000","last_became_active":"2024-03-06T07:25:07.930536+0000","last_became_peered":"2024-03-06T07:25:07.930536+0000","last_unstale":"2024-03-06T07:25:08.928053+0000","last_undegraded":"2024-03-06T07:25:08.928053+0000","last_fullsized":"2024-03-06T07:25:08.928053+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:06:10.279203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.385507+0000","last_change":"2024-03-06T07:25:06.931052+0000","last_active":"2024-03-06T07:25:09.385507+0000","last_peered":"2024-03-06T07:25:09.385507+0000","last_clean":"2024-03-06T07:25:09.385507+0000","last_became_active":"2024-03-06T07:25:06.930765+0000","last_became_peered":"2024-03-06T07:25:06.930765+0000","last_unstale":"2024-03-06T07:25:09.385507+0000","last_undegraded":"2024-03-06T07:25:09.385507+0000","last_fullsized":"2024-03-06T07:25:09.385507+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:18:20.782247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.924340+0000","last_change":"2024-03-06T07:25:07.937529+0000","last_active":"2024-03-06T07:25:08.924340+0000","last_peered":"2024-03-06T07:25:08.924340+0000","last_clean":"2024-03-06T07:25:08.924340+0000","last_became_active":"2024-03-06T07:25:07.937247+0000","last_became_peered":"2024-03-06T07:25:07.937247+0000","last_unstale":"2024-03-06T07:25:08.924340+0000","last_undegraded":"2024-03-06T07:25:08.924340+0000","last_fullsized":"2024-03-06T07:25:08.924340+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:45:53.095601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.940034+0000","last_change":"2024-03-06T07:25:06.935076+0000","last_active":"2024-03-06T07:25:08.940034+0000","last_peered":"2024-03-06T07:25:08.940034+0000","last_clean":"2024-03-06T07:25:08.940034+0000","last_became_active":"2024-03-06T07:25:06.934424+0000","last_became_peered":"2024-03-06T07:25:06.934424+0000","last_unstale":"2024-03-06T07:25:08.940034+0000","last_undegraded":"2024-03-06T07:25:08.940034+0000","last_fullsized":"2024-03-06T07:25:08.940034+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:55:39.491573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.927842+0000","last_change":"2024-03-06T07:25:07.938857+0000","last_active":"2024-03-06T07:25:08.927842+0000","last_peered":"2024-03-06T07:25:08.927842+0000","last_clean":"2024-03-06T07:25:08.927842+0000","last_became_active":"2024-03-06T07:25:07.938308+0000","last_became_peered":"2024-03-06T07:25:07.938308+0000","last_unstale":"2024-03-06T07:25:08.927842+0000","last_undegraded":"2024-03-06T07:25:08.927842+0000","last_fullsized":"2024-03-06T07:25:08.927842+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:42.965518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942637+0000","last_change":"2024-03-06T07:25:06.927019+0000","last_active":"2024-03-06T07:25:08.942637+0000","last_peered":"2024-03-06T07:25:08.942637+0000","last_clean":"2024-03-06T07:25:08.942637+0000","last_became_active":"2024-03-06T07:25:06.926737+0000","last_became_peered":"2024-03-06T07:25:06.926737+0000","last_unstale":"2024-03-06T07:25:08.942637+0000","last_undegraded":"2024-03-06T07:25:08.942637+0000","last_fullsized":"2024-03-06T07:25:08.942637+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:36.511111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.928775+0000","last_change":"2024-03-06T07:25:07.936883+0000","last_active":"2024-03-06T07:25:08.928775+0000","last_peered":"2024-03-06T07:25:08.928775+0000","last_clean":"2024-03-06T07:25:08.928775+0000","last_became_active":"2024-03-06T07:25:07.936552+0000","last_became_peered":"2024-03-06T07:25:07.936552+0000","last_unstale":"2024-03-06T07:25:08.928775+0000","last_undegraded":"2024-03-06T07:25:08.928775+0000","last_fullsized":"2024-03-06T07:25:08.928775+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:06.907627+0000","last_clean_scrub_stamp":"2024-03-06T07:25:06.907627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:36:53.073182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"135'6","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:09.386561+0000","last_change":"2024-03-06T07:25:06.933005+0000","last_active":"2024-03-06T07:25:09.386561+0000","last_peered":"2024-03-06T07:25:09.386561+0000","last_clean":"2024-03-06T07:25:09.386561+0000","last_became_active":"2024-03-06T07:25:06.932668+0000","last_became_peered":"2024-03-06T07:25:06.932668+0000","last_unstale":"2024-03-06T07:25:09.386561+0000","last_undegraded":"2024-03-06T07:25:09.386561+0000","last_fullsized":"2024-03-06T07:25:09.386561+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:05.901543+0000","last_clean_scrub_stamp":"2024-03-06T07:25:05.901543+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:11.729210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":384,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-03-06T07:25:08.942800+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:08.942800+0000","last_peered":"2024-03-06T07:25:08.942800+0000","last_clean":"2024-03-06T07:25:08.942800+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:08.942800+0000","last_undegraded":"2024-03-06T07:25:08.942800+0000","last_fullsized":"2024-03-06T07:25:08.942800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32920,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338920,"statfs":{"total":96636764160,"available":96603054080,"internally_reserved":0,"allocated":1777664,"data_stored":1416992,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":826672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1708032,"data_stored":826602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1720320,"data_stored":828214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":827218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":826714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":826672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32936,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338904,"statfs":{"total":96636764160,"available":96603037696,"internally_reserved":0,"allocated":1798144,"data_stored":1419288,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:15.691 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:25:15.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:25:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.834+0000 7f3f08b13640 1 Processor -- start 2024-03-06T07:25:15.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.834+0000 7f3f08b13640 1 -- start start 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f041529f0 0x7f3f040c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f040c5370 0x7f3f040c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f041559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f04159710 con 0x7f3f040c5370 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f040c68b0 con 0x7f3f041529f0 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f08b13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04152dd0 con 0x7f3f04155530 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f040c5370 0x7f3f040c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3f040c5370 0x7f3f040c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54234/0 (socket says 172.21.15.17:54234) 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 -- 172.21.15.17:0/2527738908 learned_addr learned my addr 172.21.15.17:0/2527738908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:15.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f02d9d640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f041559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f0259c640 1 --2- 172.21.15.17:0/2527738908 >> v2:172.21.15.163:3300/0 conn(0x7f3f041529f0 0x7f3f040c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 -- 172.21.15.17:0/2527738908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 msgr2=0x7f3f041559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f041559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 -- 172.21.15.17:0/2527738908 >> v2:172.21.15.163:3300/0 conn(0x7f3f041529f0 msgr2=0x7f3f040c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/2527738908 >> v2:172.21.15.163:3300/0 conn(0x7f3f041529f0 0x7f3f040c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.835+0000 7f3f01d9b640 1 -- 172.21.15.17:0/2527738908 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04158b00 con 0x7f3f040c5370 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.836+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f040c57d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3f040cba90 tx=0x7f3eec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef46eb89750b39d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.836+0000 7f3f0159a640 1 -- 172.21.15.17:0/2527738908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec02e070 con 0x7f3f040c5370 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.836+0000 7f3f0159a640 1 -- 172.21.15.17:0/2527738908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec037070 con 0x7f3f040c5370 2024-03-06T07:25:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.836+0000 7f3f0159a640 1 -- 172.21.15.17:0/2527738908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec031a00 con 0x7f3f040c5370 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 -- 172.21.15.17:0/2527738908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 msgr2=0x7f3f040c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f040c57d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f3f040cba90 tx=0x7f3eec02b880 comp rx=0 tx=0).stop 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 -- 172.21.15.17:0/2527738908 shutdown_connections 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f041559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 --2- 172.21.15.17:0/2527738908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f040c57d0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 --2- 172.21.15.17:0/2527738908 >> v2:172.21.15.163:3300/0 conn(0x7f3f041529f0 0x7f3f040c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 -- 172.21.15.17:0/2527738908 >> 172.21.15.17:0/2527738908 conn(0x7f3f040ac580 msgr2=0x7f3f040ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 -- 172.21.15.17:0/2527738908 shutdown_connections 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.837+0000 7f3f08b13640 1 -- 172.21.15.17:0/2527738908 wait complete. 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.838+0000 7f3f08b13640 1 Processor -- start 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.838+0000 7f3f08b13640 1 -- start start 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f041529f0 0x7f3f04154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f040be940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f041593a0 con 0x7f3f040c5370 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f041590c0 con 0x7f3f041529f0 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f08b13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f04158740 con 0x7f3f04155530 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54242/0 (socket says 172.21.15.17:54242) 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 -- 172.21.15.17:0/1535502785 learned_addr learned my addr 172.21.15.17:0/1535502785 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f041529f0 0x7f3f04154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f02d9d640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f040be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 -- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 msgr2=0x7f3f040be940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f040be940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 -- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f041529f0 msgr2=0x7f3f04154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f041529f0 0x7f3f04154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 -- 172.21.15.17:0/1535502785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04148b50 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.839+0000 7f3f0259c640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00e8c0 tx=0x7f3efc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.840+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc017070 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.840+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00c940 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.840+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00cd10 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.840+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f041eea90 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.840+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f04154780 con 0x7f3f040c5370 2024-03-06T07:25:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.841+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0404f330 con 0x7f3f040c5370 2024-03-06T07:25:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.841+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3efc01a070 con 0x7f3f040c5370 2024-03-06T07:25:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.841+0000 7f3efaffd640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 0x7f3ee406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.842+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3efc08a650 con 0x7f3f040c5370 2024-03-06T07:25:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.842+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 0x7f3ee406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:15.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.842+0000 7f3f01d9b640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 0x7f3ee406d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3eec036000 tx=0x7f3eec03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:15.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:15.846+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3efc058cf0 con 0x7f3f040c5370 2024-03-06T07:25:16.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.098+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3f04151dd0 con 0x7f3f040c5370 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.099+0000 7f3efaffd640 1 -- 172.21.15.17:0/1535502785 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3efc05cd90 con 0x7f3f040c5370 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:25:16.101 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:16.102 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:25:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 msgr2=0x7f3ee406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 0x7f3ee406d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3eec036000 tx=0x7f3eec03d040 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 msgr2=0x7f3f04148e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00e8c0 tx=0x7f3efc00ed80 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 shutdown_connections 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ee406b500 0x7f3ee406d9c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f04155530 0x7f3f040be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f041529f0 0x7f3f04154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 --2- 172.21.15.17:0/1535502785 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f040c5370 0x7f3f04148e30 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.101+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 >> 172.21.15.17:0/1535502785 conn(0x7f3f040ac580 msgr2=0x7f3f040aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.102+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 shutdown_connections 2024-03-06T07:25:16.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.102+0000 7f3f08b13640 1 -- 172.21.15.17:0/1535502785 wait complete. 2024-03-06T07:25:16.114 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:16.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 Processor -- start 2024-03-06T07:25:16.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 -- start start 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78e80c8a00 0x7f78e8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e8159460 0x7f78e80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80c7760 con 0x7f78e8152920 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e81587b0 con 0x7f78e80c8a00 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.214+0000 7f78ed20f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80c7a40 con 0x7f78e8159460 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78e80c8a00 0x7f78e8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78e80c8a00 0x7f78e8158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54248/0 (socket says 172.21.15.17:54248) 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 -- 172.21.15.17:0/151477215 learned_addr learned my addr 172.21.15.17:0/151477215 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78e759e640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e8159460 0x7f78e80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 -- 172.21.15.17:0/151477215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e8159460 msgr2=0x7f78e80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e8159460 0x7f78e80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 -- 172.21.15.17:0/151477215 >> v1:172.21.15.163:6789/0 conn(0x7f78e8152920 legacy=0x7f78e8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.215+0000 7f78dffff640 1 -- 172.21.15.17:0/151477215 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e80c7d20 con 0x7f78e80c8a00 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78dffff640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e80c8a00 0x7f78e8158f20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686feb9b75ec60d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78e659c640 1 -- 172.21.15.17:0/151477215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0035020 con 0x7f78e80c8a00 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78e659c640 1 -- 172.21.15.17:0/151477215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d0031640 con 0x7f78e80c8a00 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78e659c640 1 -- 172.21.15.17:0/151477215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d002d040 con 0x7f78e80c8a00 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 -- 172.21.15.17:0/151477215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e80c8a00 msgr2=0x7f78e8158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e80c8a00 0x7f78e8158f20 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009870 tx=0x7f78d002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 -- 172.21.15.17:0/151477215 shutdown_connections 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e8159460 0x7f78e80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 --2- 172.21.15.17:0/151477215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e80c8a00 0x7f78e8158f20 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 -- 172.21.15.17:0/151477215 >> 172.21.15.17:0/151477215 conn(0x7f78e80ac560 msgr2=0x7f78e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 -- 172.21.15.17:0/151477215 shutdown_connections 2024-03-06T07:25:16.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.216+0000 7f78ed20f640 1 -- 172.21.15.17:0/151477215 wait complete. 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 Processor -- start 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 -- start start 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78e80c8a00 0x7f78e81480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e80c3420 0x7f78e80c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80c7390 con 0x7f78e8152920 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e806d030 con 0x7f78e80c8a00 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78ed20f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e80c7510 con 0x7f78e80c3420 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.218+0000 7f78dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54250/0 (socket says 172.21.15.17:54250) 2024-03-06T07:25:16.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 -- 172.21.15.17:0/2291912045 learned_addr learned my addr 172.21.15.17:0/2291912045 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78e6d9d640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78e80c8a00 0x7f78e81480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78e759e640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e80c3420 0x7f78e80c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 -- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e80c3420 msgr2=0x7f78e80c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e80c3420 0x7f78e80c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 -- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78e80c8a00 msgr2=0x7f78e81480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78e80c8a00 0x7f78e81480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 -- 172.21.15.17:0/2291912045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e80c1b10 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78dffff640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0040070 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00399f0 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e80c4620 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0035040 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.219+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e80c3c30 con 0x7f78e8152920 2024-03-06T07:25:16.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.220+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e804f330 con 0x7f78e8152920 2024-03-06T07:25:16.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.221+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78d004b060 con 0x7f78e8152920 2024-03-06T07:25:16.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.221+0000 7f78df7fe640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 0x7f78c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.221+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78d00abb00 con 0x7f78e8152920 2024-03-06T07:25:16.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.221+0000 7f78e6d9d640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 0x7f78c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.222+0000 7f78e6d9d640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 0x7f78c806d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f78d8001780 tx=0x7f78d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.225+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78e80c1b10 con 0x7f78e8152920 2024-03-06T07:25:16.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.444+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78e81eb130 con 0x7f78e8152920 2024-03-06T07:25:16.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.444+0000 7f78df7fe640 1 -- 172.21.15.17:0/2291912045 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f78d007a1a0 con 0x7f78e8152920 2024-03-06T07:25:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:16.446 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":120,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:09.279368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"d","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/3327948285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3327948285},{"type":"v1","addr":"172.21.15.163:6837","nonce":3327948285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8280,"qdb_cluster":[8280]},"id":10}]} 2024-03-06T07:25:16.446 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 120 2024-03-06T07:25:16.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 msgr2=0x7f78c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 0x7f78c806d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f78d8001780 tx=0x7f78d8009040 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 msgr2=0x7f78e80c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000c00 tx=0x7f78d00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 shutdown_connections 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c806b4d0 0x7f78c806d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78e80c3420 0x7f78e80c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78e8152920 0x7f78e80c2730 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 --2- 172.21.15.17:0/2291912045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78e80c8a00 0x7f78e81480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 >> 172.21.15.17:0/2291912045 conn(0x7f78e80ac560 msgr2=0x7f78e806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 shutdown_connections 2024-03-06T07:25:16.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.446+0000 7f78ed20f640 1 -- 172.21.15.17:0/2291912045 wait complete. 2024-03-06T07:25:16.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:09.279368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8280': {'gid': 8280, 'name': 'd', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/3327948285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3327948285}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3327948285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8280, 'qdb_cluster': [8280]} 2024-03-06T07:25:16.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:16.861 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:25:16.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.968+0000 7fac23b81640 1 Processor -- start 2024-03-06T07:25:16.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.968+0000 7fac23b81640 1 -- start start 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac23b81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac1c0bfe90 0x7fac1c155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac23b81640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac1c1522d0 0x7fac1c152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac23b81640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c0c7200 con 0x7fac1c0c6d70 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac23b81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c158710 con 0x7fac1c0bfe90 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac23b81640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c159590 con 0x7fac1c1522d0 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2191d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac1c0bfe90 0x7fac1c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2111c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac1c0c6d70 0x7fac1c158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54274/0 (socket says 172.21.15.17:54274) 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2111c640 1 -- 172.21.15.17:0/1388975859 learned_addr learned my addr 172.21.15.17:0/1388975859 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2211e640 1 --2- 172.21.15.17:0/1388975859 >> v2:172.21.15.163:3300/0 conn(0x7fac1c1522d0 0x7fac1c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2191d640 1 -- 172.21.15.17:0/1388975859 >> v2:172.21.15.163:3300/0 conn(0x7fac1c1522d0 msgr2=0x7fac1c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2191d640 1 --2- 172.21.15.17:0/1388975859 >> v2:172.21.15.163:3300/0 conn(0x7fac1c1522d0 0x7fac1c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2191d640 1 -- 172.21.15.17:0/1388975859 >> v1:172.21.15.17:6789/0 conn(0x7fac1c0c6d70 legacy=0x7fac1c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2191d640 1 -- 172.21.15.17:0/1388975859 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0c59d0 con 0x7fac1c0bfe90 2024-03-06T07:25:16.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.969+0000 7fac2091b640 1 -- 172.21.15.17:0/1388975859 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374197544 0 0) 0x7fac0c001110 con 0x7fac1c0c6d70 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac2191d640 1 --2- 172.21.15.17:0/1388975859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c155120 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fac180097a0 tx=0x7fac1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548c7249cf1644b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac2091b640 1 -- 172.21.15.17:0/1388975859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180315a0 con 0x7fac1c0bfe90 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac2091b640 1 -- 172.21.15.17:0/1388975859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18037070 con 0x7fac1c0bfe90 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac2091b640 1 -- 172.21.15.17:0/1388975859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18031960 con 0x7fac1c0bfe90 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac23b81640 1 -- 172.21.15.17:0/1388975859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 msgr2=0x7fac1c155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.970+0000 7fac23b81640 1 --2- 172.21.15.17:0/1388975859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c155120 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fac180097a0 tx=0x7fac1802b840 comp rx=0 tx=0).stop 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 -- 172.21.15.17:0/1388975859 shutdown_connections 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 --2- 172.21.15.17:0/1388975859 >> v2:172.21.15.163:3300/0 conn(0x7fac1c1522d0 0x7fac1c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 --2- 172.21.15.17:0/1388975859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c155120 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 -- 172.21.15.17:0/1388975859 >> 172.21.15.17:0/1388975859 conn(0x7fac1c0acaa0 msgr2=0x7fac1c0aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 -- 172.21.15.17:0/1388975859 shutdown_connections 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.971+0000 7fac23b81640 1 -- 172.21.15.17:0/1388975859 wait complete. 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 Processor -- start 2024-03-06T07:25:16.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 -- start start 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 0x7fac1c0ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c1522d0 0x7fac1c14c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c158960 con 0x7fac1c0bfe90 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0c5f90 con 0x7fac1c0c6d70 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac23b81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0ca700 con 0x7fac1c1522d0 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.972+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54274/0 (socket says 172.21.15.17:54274) 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 -- 172.21.15.17:0/2467801244 learned_addr learned my addr 172.21.15.17:0/2467801244 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:16.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2211e640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c1522d0 0x7fac1c14c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2111c640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 0x7fac1c0ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 -- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c1522d0 msgr2=0x7fac1c14c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c1522d0 0x7fac1c14c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 -- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 msgr2=0x7fac1c0ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 0x7fac1c0ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 -- 172.21.15.17:0/2467801244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c14d220 con 0x7fac1c0bfe90 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2111c640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 0x7fac1c0ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac2191d640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fac18034040 tx=0x7fac180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac127fc640 1 -- 172.21.15.17:0/2467801244 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18032630 con 0x7fac1c0bfe90 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac127fc640 1 -- 172.21.15.17:0/2467801244 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180327f0 con 0x7fac1c0bfe90 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c1eecc0 con 0x7fac1c0bfe90 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac127fc640 1 -- 172.21.15.17:0/2467801244 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18032be0 con 0x7fac1c0bfe90 2024-03-06T07:25:16.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.973+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c154490 con 0x7fac1c0bfe90 2024-03-06T07:25:16.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.975+0000 7fac127fc640 1 -- 172.21.15.17:0/2467801244 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac1804e430 con 0x7fac1c0bfe90 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.975+0000 7fac127fc640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.975+0000 7fac127fc640 1 -- 172.21.15.17:0/2467801244 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac1802f030 con 0x7fac1c0bfe90 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.976+0000 7fac2111c640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.976+0000 7fac2111c640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 0x7fac0006d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0b0bf0 tx=0x7fac0c007a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.976+0000 7fac2676b740 1 Processor -- start 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 -- start start 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4374f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc43754b0 0x55ffc4371680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc42db6b0 con 0x55ffc4376f50 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4284a60 con 0x55ffc4377330 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc42d7750 con 0x55ffc43754b0 2024-03-06T07:25:16.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54290/0 (socket says 172.21.15.17:54290) 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2191d640 1 -- 172.21.15.17:0/2965550045 learned_addr learned my addr 172.21.15.17:0/2965550045 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2111c640 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4374f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.977+0000 7fac2211e640 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc43754b0 0x55ffc4371680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 -- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc43754b0 msgr2=0x55ffc4371680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc43754b0 0x55ffc4371680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 -- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 msgr2=0x55ffc4374f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4374f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 -- 172.21.15.17:0/2965550045 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc42a2f30 con 0x55ffc4376f50 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2191d640 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fac18076ba0 tx=0x7fac18009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6950dc85523981b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fabff7fe640 1 -- 172.21.15.17:0/2965550045 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18092070 con 0x55ffc4376f50 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fabff7fe640 1 -- 172.21.15.17:0/2965550045 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180a3e50 con 0x55ffc4376f50 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fabff7fe640 1 -- 172.21.15.17:0/2965550045 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1807a740 con 0x55ffc4376f50 2024-03-06T07:25:16.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2676b740 1 -- 172.21.15.17:0/2965550045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 msgr2=0x55ffc4374a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.978+0000 7fac2676b740 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fac18076ba0 tx=0x7fac18009190 comp rx=0 tx=0).stop 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 -- 172.21.15.17:0/2965550045 shutdown_connections 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc43754b0 0x55ffc4371680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4374f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 --2- 172.21.15.17:0/2965550045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc4374a30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 -- 172.21.15.17:0/2965550045 >> 172.21.15.17:0/2965550045 conn(0x55ffc43737d0 msgr2=0x55ffc43983c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 -- 172.21.15.17:0/2965550045 shutdown_connections 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 -- 172.21.15.17:0/2965550045 wait complete. 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 -- start start 2024-03-06T07:25:16.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.979+0000 7fac2676b740 1 Processor -- start 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc4394e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43aa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4284a60 con 0x55ffc4377330 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4297ea0 con 0x55ffc43ac470 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc42db6b0 con 0x55ffc4376f50 2024-03-06T07:25:16.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2676b740 10 client.0 _ll_register_callbacks cb 0x55ffc4297870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54296/0 (socket says 172.21.15.17:54296) 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 -- 172.21.15.17:0/1415719532 learned_addr learned my addr 172.21.15.17:0/1415719532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2211e640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2191d640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc4394e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 -- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 msgr2=0x55ffc4394e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc4394e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 -- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 msgr2=0x55ffc43aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.980+0000 7fac2111c640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2111c640 1 -- 172.21.15.17:0/1415719532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc4298270 con 0x55ffc4377330 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2111c640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fac0c001d20 tx=0x7fac0c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3a6125a6974b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:16.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c027060 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac0c010040 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac180a3e50 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac0c02d050 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 10 client.8439 fetch_fsmap learned FSMap version 120 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x55ffc3ff02d0 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180a3e50 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 120) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fac0c030020 con 0x55ffc4377330 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 10 client.8439 fetch_fsmap finished waiting for FSMap version 120 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 10 client.8439 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 20 client.8439 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc43c7800 0x55ffc43d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 4 client.8439 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.981+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 --> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffc42e8bf0 con 0x55ffc43c7800 2024-03-06T07:25:16.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.982+0000 7fac2191d640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc43c7800 0x55ffc43d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.982+0000 7fac2191d640 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc43c7800 0x55ffc43d24c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168292ebd2ed94f6 server_cookie=f112f80fe46fae0b in_seq=0 out_seq=0 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.982+0000 7fabfcff9640 10 client.8439 ms_handle_connect on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fabfcff9640 1 -- 172.21.15.17:0/1415719532 <== mds.0 v2:172.21.15.163:6835/3327948285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ffc42e8bf0 con 0x55ffc43c7800 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fabfcff9640 10 client.8439 handle_command_reply: tid=0 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 client.8439 shutdown 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 msgr2=0x55ffc43abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fac0c001d20 tx=0x7fac0c000ed0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 shutdown_connections 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc43c7800 0x55ffc43d24c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4377330 0x55ffc43abf30 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fabfcff9640 0 client.8439 ms_handle_reset on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 --2- 172.21.15.17:0/1415719532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc4394e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:16.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 >> 172.21.15.17:0/1415719532 conn(0x55ffc43754b0 msgr2=0x55ffc43714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 shutdown_connections 2024-03-06T07:25:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 1 -- 172.21.15.17:0/1415719532 wait complete. 2024-03-06T07:25:16.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:16.983+0000 7fac2676b740 20 client.8439 trim_cache size 0 max 16384 2024-03-06T07:25:17.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.000+0000 7fac2676b740 1 Processor -- start 2024-03-06T07:25:17.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.000+0000 7fac2676b740 1 -- start start 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.000+0000 7fac2676b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4377330 0x55ffc43ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2676b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1f7e50 con 0x55ffc4376f50 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1f7020 con 0x55ffc43ac470 2024-03-06T07:25:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1500f0 con 0x55ffc4377330 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54300/0 (socket says 172.21.15.17:54300) 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 -- 172.21.15.17:0/3871919695 learned_addr learned my addr 172.21.15.17:0/3871919695 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2111c640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4377330 0x55ffc43ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2211e640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 -- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4377330 msgr2=0x55ffc43ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4377330 0x55ffc43ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 -- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 msgr2=0x55ffc43abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 -- 172.21.15.17:0/3871919695 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc40ba6b0 con 0x55ffc4376f50 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2211e640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fac2191d640 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fac180aa540 tx=0x7fac1808f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b98f27f3e4db551 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fabfeffd640 1 -- 172.21.15.17:0/3871919695 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18092040 con 0x55ffc4376f50 2024-03-06T07:25:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fabfeffd640 1 -- 172.21.15.17:0/3871919695 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180a3e50 con 0x55ffc4376f50 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.001+0000 7fabfeffd640 1 -- 172.21.15.17:0/3871919695 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1807a740 con 0x55ffc4376f50 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- 172.21.15.17:0/3871919695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 msgr2=0x55ffc43aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fac180aa540 tx=0x7fac1808f000 comp rx=0 tx=0).stop 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- 172.21.15.17:0/3871919695 shutdown_connections 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc43ac470 0x55ffc43abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4377330 0x55ffc43ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 --2- 172.21.15.17:0/3871919695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc4376f50 0x55ffc43aa370 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- 172.21.15.17:0/3871919695 >> 172.21.15.17:0/3871919695 conn(0x55ffc43754b0 msgr2=0x55ffc43b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- 172.21.15.17:0/3871919695 shutdown_connections 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- 172.21.15.17:0/3871919695 wait complete. 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.002+0000 7fac2676b740 1 -- start start 2024-03-06T07:25:17.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 Processor -- start 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc43ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4371a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc43c3850 con 0x55ffc43ac470 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1f6a20 con 0x55ffc4377330 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffc43ad720 con 0x55ffc4376f50 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2676b740 10 client.0 _ll_register_callbacks cb 0x7fac0c001970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2211e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2211e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54306/0 (socket says 172.21.15.17:54306) 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2211e640 1 -- 172.21.15.17:0/2978460656 learned_addr learned my addr 172.21.15.17:0/2978460656 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.003+0000 7fac2191d640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc43ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2111c640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4371a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 -- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 msgr2=0x55ffc43ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc43ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 -- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 msgr2=0x55ffc4371a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4371a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 -- 172.21.15.17:0/2978460656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc43981a0 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2211e640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fac14017eb0 tx=0x7fac14015760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bd2423b02eafb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14025060 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac14003990 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac180a3e50 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1402b050 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2676b740 10 client.8445 fetch_fsmap learned FSMap version 120 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x55ffc439abe0 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180a3e50 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.004+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 120) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fac1402e020 con 0x55ffc43ac470 2024-03-06T07:25:17.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 10 client.8445 fetch_fsmap finished waiting for FSMap version 120 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 10 client.8445 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 20 client.8445 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc4399690 0x55ffc4399a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 4 client.8445 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 --> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ffc42e2f40 con 0x55ffc4399690 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2191d640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc4399690 0x55ffc4399a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2191d640 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc4399690 0x55ffc4399a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43096360ae9266c1 server_cookie=754964f075dfff92 in_seq=0 out_seq=0 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fabfcff9640 10 client.8445 ms_handle_connect on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fabfcff9640 1 -- 172.21.15.17:0/2978460656 <== mds.0 v2:172.21.15.163:6835/3327948285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55ffc42e2f40 con 0x55ffc4399690 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fabfcff9640 10 client.8445 handle_command_reply: tid=0 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 client.8445 shutdown 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 msgr2=0x55ffc4420570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fac14017eb0 tx=0x7fac14015760 comp rx=0 tx=0).stop 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 shutdown_connections 2024-03-06T07:25:17.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x55ffc4399690 0x55ffc4399a70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ffc43ac470 0x55ffc4420570 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ffc4377330 0x55ffc4371a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fabfcff9640 0 client.8445 ms_handle_reset on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 --2- 172.21.15.17:0/2978460656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ffc4376f50 0x55ffc43ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.005+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 >> 172.21.15.17:0/2978460656 conn(0x55ffc43754b0 msgr2=0x55ffc43aa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 shutdown_connections 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac2676b740 1 -- 172.21.15.17:0/2978460656 wait complete. 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac2676b740 20 client.8445 trim_cache size 0 max 16384 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8280, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 9.0790811500000004, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 120, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 136, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 136, 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14.156863465000001 2024-03-06T07:25:17.015 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 msgr2=0x7fac0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 0x7fac0006d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0b0bf0 tx=0x7fac0c007a10 comp rx=0 tx=0).stop 2024-03-06T07:25:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 msgr2=0x7fac1c14d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fac18034040 tx=0x7fac180046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 shutdown_connections 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fac0006b500 0x7fac0006d9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c1522d0 0x7fac1c14c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0c6d70 0x7fac1c0ab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 --2- 172.21.15.17:0/2467801244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac1c0bfe90 0x7fac1c14d500 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 >> 172.21.15.17:0/2467801244 conn(0x7fac1c0acaa0 msgr2=0x7fac1c0aa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 shutdown_connections 2024-03-06T07:25:17.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.006+0000 7fac23b81640 1 -- 172.21.15.17:0/2467801244 wait complete. 2024-03-06T07:25:17.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.118+0000 7f7c8f89d640 1 Processor -- start 2024-03-06T07:25:17.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.118+0000 7f7c8f89d640 1 -- start start 2024-03-06T07:25:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8f89d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c880c5330 0x7f7c880c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8f89d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7c880c88b0 0x7f7c881586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8f89d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c88155110 con 0x7f7c880c7dd0 2024-03-06T07:25:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8f89d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c881559e0 con 0x7f7c880c5330 2024-03-06T07:25:17.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8f89d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c880c8f40 con 0x7f7c880c88b0 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8d639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c880c5330 0x7f7c880c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8de3a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7c880c7dd0 0x7f7c8806b920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54278/0 (socket says 172.21.15.17:54278) 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8de3a640 1 -- 172.21.15.17:0/1168768669 learned_addr learned my addr 172.21.15.17:0/1168768669 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8ce38640 1 --2- 172.21.15.17:0/1168768669 >> v2:172.21.15.163:3300/0 conn(0x7f7c880c88b0 0x7f7c881586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8d639640 1 -- 172.21.15.17:0/1168768669 >> v2:172.21.15.163:3300/0 conn(0x7f7c880c88b0 msgr2=0x7f7c881586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8d639640 1 --2- 172.21.15.17:0/1168768669 >> v2:172.21.15.163:3300/0 conn(0x7f7c880c88b0 0x7f7c881586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8d639640 1 -- 172.21.15.17:0/1168768669 >> v1:172.21.15.17:6789/0 conn(0x7f7c880c7dd0 legacy=0x7f7c8806b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.119+0000 7f7c8d639640 1 -- 172.21.15.17:0/1168768669 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88155390 con 0x7f7c880c5330 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8d639640 1 --2- 172.21.15.17:0/1168768669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c5330 0x7f7c880c5710 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7c88156240 tx=0x7f7c8402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eca8dcc6d45b99 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c7ffff640 1 -- 172.21.15.17:0/1168768669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840315a0 con 0x7f7c880c5330 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c7ffff640 1 -- 172.21.15.17:0/1168768669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84037070 con 0x7f7c880c5330 2024-03-06T07:25:17.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c7ffff640 1 -- 172.21.15.17:0/1168768669 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84031960 con 0x7f7c880c5330 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 -- 172.21.15.17:0/1168768669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c5330 msgr2=0x7f7c880c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/1168768669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c5330 0x7f7c880c5710 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7c88156240 tx=0x7f7c8402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 -- 172.21.15.17:0/1168768669 shutdown_connections 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/1168768669 >> v2:172.21.15.163:3300/0 conn(0x7f7c880c88b0 0x7f7c881586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/1168768669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c5330 0x7f7c880c5710 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.120+0000 7f7c8f89d640 1 -- 172.21.15.17:0/1168768669 >> 172.21.15.17:0/1168768669 conn(0x7f7c880ac560 msgr2=0x7f7c880ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.121+0000 7f7c8f89d640 1 -- 172.21.15.17:0/1168768669 shutdown_connections 2024-03-06T07:25:17.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.121+0000 7f7c8f89d640 1 -- 172.21.15.17:0/1168768669 wait complete. 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 Processor -- start 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 -- start start 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c880c5330 0x7f7c880c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c880c7dd0 0x7f7c88147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c881529d0 con 0x7f7c880c88b0 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8806d090 con 0x7f7c880c7dd0 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8f89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c88152b50 con 0x7f7c880c5330 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8de3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8de3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54322/0 (socket says 172.21.15.17:54322) 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.122+0000 7f7c8de3a640 1 -- 172.21.15.17:0/4103319938 learned_addr learned my addr 172.21.15.17:0/4103319938 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8d639640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c880c5330 0x7f7c880c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8ce38640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c880c7dd0 0x7f7c88147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 -- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c880c5330 msgr2=0x7f7c880c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c880c5330 0x7f7c880c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 -- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c880c7dd0 msgr2=0x7f7c88147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c880c7dd0 0x7f7c88147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 -- 172.21.15.17:0/4103319938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c881595f0 con 0x7f7c880c88b0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8de3a640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000cbf0 tx=0x7f7c8000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80017070 con 0x7f7c880c88b0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8000ad40 con 0x7f7c880c88b0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c88159030 con 0x7f7c880c88b0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80012600 con 0x7f7c880c88b0 2024-03-06T07:25:17.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.123+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c880c3070 con 0x7f7c880c88b0 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.124+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8804f330 con 0x7f7c880c88b0 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.125+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c8001a070 con 0x7f7c880c88b0 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.125+0000 7f7c7dffb640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.125+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c80089f50 con 0x7f7c880c88b0 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.125+0000 7f7c8d639640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.126+0000 7f7c8d639640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7c84036000 tx=0x7f7c8403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.129+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c88155110 con 0x7f7c880c88b0 2024-03-06T07:25:17.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.348+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c8806cdc0 con 0x7f7c880c88b0 2024-03-06T07:25:17.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.348+0000 7f7c7dffb640 1 -- 172.21.15.17:0/4103319938 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f7c80058680 con 0x7f7c880c88b0 2024-03-06T07:25:17.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:17.350 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":120,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:09.279368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"d","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/3327948285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3327948285},{"type":"v1","addr":"172.21.15.163:6837","nonce":3327948285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8280,"qdb_cluster":[8280]},"id":10}]} 2024-03-06T07:25:17.351 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 120 2024-03-06T07:25:17.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 msgr2=0x7f7c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7c84036000 tx=0x7f7c8403d040 comp rx=0 tx=0).stop 2024-03-06T07:25:17.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 msgr2=0x7f7c881480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000cbf0 tx=0x7f7c8000a430 comp rx=0 tx=0).stop 2024-03-06T07:25:17.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 shutdown_connections 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c6c06b4d0 0x7f7c6c06d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c880c88b0 0x7f7c881480d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c880c7dd0 0x7f7c88147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 --2- 172.21.15.17:0/4103319938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c880c5330 0x7f7c880c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 >> 172.21.15.17:0/4103319938 conn(0x7f7c880ac560 msgr2=0x7f7c8814c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 shutdown_connections 2024-03-06T07:25:17.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.350+0000 7f7c8f89d640 1 -- 172.21.15.17:0/4103319938 wait complete. 2024-03-06T07:25:17.363 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 120, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:07.286023+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 120, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8280": { "addr": "172.21.15.163:6837/3327948285", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3327948285, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3327948285, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8280, "incarnation": 118, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:09.279368+0000", "qdb_cluster": [ 8280 ], "qdb_leader": 8280, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8280 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/3255496602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3255496602, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3255496602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7783, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/824017697", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 824017697, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 824017697, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8265, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1732861093", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1732861093, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1732861093, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:17.363 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:17.363 INFO:tasks.cephfs.test_failover:current = [{'gid': 8268, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1732861093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1732861093}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1732861093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 7783, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 8265, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 8280, 'name': 'd', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/3327948285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3327948285}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3327948285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:25:17.364 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.b mds_join_fs cephfs 2024-03-06T07:25:17.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.467+0000 7f4aa8c66640 1 Processor -- start 2024-03-06T07:25:17.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.467+0000 7f4aa8c66640 1 -- start start 2024-03-06T07:25:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa8c66640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4aa40c5330 0x7f4aa40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa8c66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa4155110 con 0x7f4aa40c88b0 2024-03-06T07:25:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa8c66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40c8f40 con 0x7f4aa4158db0 2024-03-06T07:25:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa8c66640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa41559e0 con 0x7f4aa40c5330 2024-03-06T07:25:17.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4a9bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4aa40c88b0 0x7f4aa41586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54284/0 (socket says 172.21.15.17:54284) 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4a9bfff640 1 -- 172.21.15.17:0/4050506321 learned_addr learned my addr 172.21.15.17:0/4050506321 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa259c640 1 --2- 172.21.15.17:0/4050506321 >> v2:172.21.15.163:3300/0 conn(0x7f4aa40c5330 0x7f4aa40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196328309 0 0) 0x7f4a8c001110 con 0x7f4aa40c88b0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a90003520 con 0x7f4aa40c88b0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050273871 0 0) 0x7f4aa40c8f40 con 0x7f4aa4158db0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a8c001110 con 0x7f4aa4158db0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3120158101 0 0) 0x7f4a90003520 con 0x7f4aa40c88b0 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa259c640 1 -- 172.21.15.17:0/4050506321 >> v1:172.21.15.163:6789/0 conn(0x7f4aa4158db0 legacy=0x7f4aa415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa259c640 1 -- 172.21.15.17:0/4050506321 >> v1:172.21.15.17:6789/0 conn(0x7f4aa40c88b0 legacy=0x7f4aa41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.468+0000 7f4aa259c640 1 -- 172.21.15.17:0/4050506321 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4155390 con 0x7f4aa40c5330 2024-03-06T07:25:17.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.469+0000 7f4aa259c640 1 --2- 172.21.15.17:0/4050506321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa40c5330 0x7f4aa40c5710 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a940097a0 tx=0x7f4a9402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fbd94b92fe952ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.469+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94037070 con 0x7f4aa40c5330 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.469+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a94030510 con 0x7f4aa40c5330 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.469+0000 7f4aa1d9b640 1 -- 172.21.15.17:0/4050506321 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c001840 con 0x7f4aa40c5330 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 -- 172.21.15.17:0/4050506321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa40c5330 msgr2=0x7f4aa40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/4050506321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa40c5330 0x7f4aa40c5710 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a940097a0 tx=0x7f4a9402b790 comp rx=0 tx=0).stop 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 -- 172.21.15.17:0/4050506321 shutdown_connections 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/4050506321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa40c5330 0x7f4aa40c5710 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 -- 172.21.15.17:0/4050506321 >> 172.21.15.17:0/4050506321 conn(0x7f4aa40ac560 msgr2=0x7f4aa40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 -- 172.21.15.17:0/4050506321 shutdown_connections 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.470+0000 7f4aa8c66640 1 -- 172.21.15.17:0/4050506321 wait complete. 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.471+0000 7f4aa8c66640 1 Processor -- start 2024-03-06T07:25:17.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.471+0000 7f4aa8c66640 1 -- start start 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4aa40c5330 0x7f4aa4150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa4151d80 0x7f4aa4148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41529d0 con 0x7f4aa40c88b0 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa406d090 con 0x7f4aa4151d80 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa8c66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4152b50 con 0x7f4aa40c5330 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54328/0 (socket says 172.21.15.17:54328) 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa259c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4aa40c5330 0x7f4aa4150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 -- 172.21.15.17:0/346598274 learned_addr learned my addr 172.21.15.17:0/346598274 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4aa2d9d640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa4151d80 0x7f4aa4148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 -- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4aa40c5330 msgr2=0x7f4aa4150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4aa40c5330 0x7f4aa4150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 -- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa4151d80 msgr2=0x7f4aa4148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa4151d80 0x7f4aa4148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 -- 172.21.15.17:0/346598274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4150fe0 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9bfff640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00cac0 tx=0x7f4a8c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.472+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c017070 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.473+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a8c005ce0 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.473+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa4157150 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.473+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0072b0 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.473+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa4157710 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.473+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa4152f40 con 0x7f4aa40c88b0 2024-03-06T07:25:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.474+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a8c01a070 con 0x7f4aa40c88b0 2024-03-06T07:25:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.474+0000 7f4a9affd640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 0x7f4a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.474+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a8c089880 con 0x7f4aa40c88b0 2024-03-06T07:25:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.475+0000 7f4aa259c640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 0x7f4a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.475+0000 7f4aa259c640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 0x7f4a8406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a94034000 tx=0x7f4a9402f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.479+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4aa4152f40 con 0x7f4aa40c88b0 2024-03-06T07:25:17.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.659+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f4aa404f330 con 0x7f4aa40c88b0 2024-03-06T07:25:17.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.662+0000 7f4a9affd640 1 -- 172.21.15.17:0/346598274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v46)=0 v46) v1 ==== 116+0+0 (secure 0 0 0) 0x7f4aa4155110 con 0x7f4aa40c88b0 2024-03-06T07:25:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 msgr2=0x7f4a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 0x7f4a8406d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a94034000 tx=0x7f4a9402f000 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 msgr2=0x7f4aa414ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00cac0 tx=0x7f4a8c00cf80 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 shutdown_connections 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a8406b4d0 0x7f4a8406d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4aa4151d80 0x7f4aa4148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa40c88b0 0x7f4aa414ae80 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 --2- 172.21.15.17:0/346598274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4aa40c5330 0x7f4aa4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 >> 172.21.15.17:0/346598274 conn(0x7f4aa40ac560 msgr2=0x7f4aa406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 shutdown_connections 2024-03-06T07:25:17.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.664+0000 7f4aa8c66640 1 -- 172.21.15.17:0/346598274 wait complete. 2024-03-06T07:25:17.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.c mds_join_fs cephfs 2024-03-06T07:25:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.778+0000 7fc3be7f7640 1 Processor -- start 2024-03-06T07:25:17.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.778+0000 7fc3be7f7640 1 -- start start 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3be7f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3be7f7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3b80c7380 0x7fc3b80c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3be7f7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b8156050 con 0x7fc3b80c6780 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3be7f7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b806de20 con 0x7fc3b80c7380 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3be7f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8072630 con 0x7fc3b806da40 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3bcd94640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3b80c7380 0x7fc3b80c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3b77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc3b80c6780 0x7fc3b8158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54300/0 (socket says 172.21.15.17:54300) 2024-03-06T07:25:17.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3b77fe640 1 -- 172.21.15.17:0/1389832630 learned_addr learned my addr 172.21.15.17:0/1389832630 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.779+0000 7fc3b7fff640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3bcd94640 1 -- 172.21.15.17:0/1389832630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 msgr2=0x7fc3b8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3bcd94640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3bcd94640 1 -- 172.21.15.17:0/1389832630 >> v1:172.21.15.17:6789/0 conn(0x7fc3b80c6780 legacy=0x7fc3b8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3bcd94640 1 -- 172.21.15.17:0/1389832630 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8072910 con 0x7fc3b80c7380 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3b6ffd640 1 -- 172.21.15.17:0/1389832630 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786346149 0 0) 0x7fc3ac001110 con 0x7fc3b80c6780 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.780+0000 7fc3bcd94640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80c3280 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009d90 tx=0x7fc3b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7075dad8d1100954 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3b6ffd640 1 -- 172.21.15.17:0/1389832630 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0039070 con 0x7fc3b80c7380 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3b6ffd640 1 -- 172.21.15.17:0/1389832630 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b002cbc0 con 0x7fc3b80c7380 2024-03-06T07:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3b6ffd640 1 -- 172.21.15.17:0/1389832630 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0034440 con 0x7fc3b80c7380 2024-03-06T07:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 -- 172.21.15.17:0/1389832630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 msgr2=0x7fc3b80c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80c3280 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009d90 tx=0x7fc3b002be10 comp rx=0 tx=0).stop 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 -- 172.21.15.17:0/1389832630 shutdown_connections 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80c3280 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/1389832630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 -- 172.21.15.17:0/1389832630 >> 172.21.15.17:0/1389832630 conn(0x7fc3b80aa630 msgr2=0x7fc3b80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 -- 172.21.15.17:0/1389832630 shutdown_connections 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.781+0000 7fc3be7f7640 1 -- 172.21.15.17:0/1389832630 wait complete. 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.782+0000 7fc3be7f7640 1 Processor -- start 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.782+0000 7fc3be7f7640 1 -- start start 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b80c6780 0x7fc3b8153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b81594a0 con 0x7fc3b80c7380 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8158890 con 0x7fc3b80c6780 2024-03-06T07:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3be7f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80c37c0 con 0x7fc3b806da40 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54346/0 (socket says 172.21.15.17:54346) 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 -- 172.21.15.17:0/3505267090 learned_addr learned my addr 172.21.15.17:0/3505267090 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3b77fe640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b80c6780 0x7fc3b8153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3b7fff640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 -- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 msgr2=0x7fc3b8152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 -- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b80c6780 msgr2=0x7fc3b8153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b80c6780 0x7fc3b8153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.783+0000 7fc3bcd94640 1 -- 172.21.15.17:0/3505267090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b81ec1c0 con 0x7fc3b80c7380 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3bcd94640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009d90 tx=0x7fc3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0039050 con 0x7fc3b80c7380 2024-03-06T07:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b0045070 con 0x7fc3b80c7380 2024-03-06T07:25:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b80ab7c0 con 0x7fc3b80c7380 2024-03-06T07:25:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0009450 con 0x7fc3b80c7380 2024-03-06T07:25:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.784+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b81ebeb0 con 0x7fc3b80c7380 2024-03-06T07:25:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.785+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b804f330 con 0x7fc3b80c7380 2024-03-06T07:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.785+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3b003e850 con 0x7fc3b80c7380 2024-03-06T07:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.786+0000 7fc3b4ff9640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 0x7fc3a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.786+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3b00acfb0 con 0x7fc3b80c7380 2024-03-06T07:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.786+0000 7fc3b7fff640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 0x7fc3a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.786+0000 7fc3b7fff640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 0x7fc3a406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80c8360 tx=0x7fc3a8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.790+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3b007b800 con 0x7fc3b80c7380 2024-03-06T07:25:17.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.971+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fc3b8073240 con 0x7fc3b80c7380 2024-03-06T07:25:17.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.973+0000 7fc3b4ff9640 1 -- 172.21.15.17:0/3505267090 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v47)=0 v47) v1 ==== 116+0+0 (secure 0 0 0) 0x7fc3b007f8a0 con 0x7fc3b80c7380 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 msgr2=0x7fc3a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 0x7fc3a406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80c8360 tx=0x7fc3a8009190 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 msgr2=0x7fc3b80ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0009d90 tx=0x7fc3b0004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 shutdown_connections 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3a406b4d0 0x7fc3a406d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b80c7380 0x7fc3b80ade90 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3b80c6780 0x7fc3b8153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 --2- 172.21.15.17:0/3505267090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3b806da40 0x7fc3b8152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 >> 172.21.15.17:0/3505267090 conn(0x7fc3b80aa630 msgr2=0x7fc3b8154770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 shutdown_connections 2024-03-06T07:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:17.975+0000 7fc3be7f7640 1 -- 172.21.15.17:0/3505267090 wait complete. 2024-03-06T07:25:17.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.086+0000 7f8aa7be9640 1 Processor -- start 2024-03-06T07:25:18.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.086+0000 7f8aa7be9640 1 -- start start 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa7be9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8aa0155ba0 0x7f8aa00c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa7be9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa7be9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa01591c0 con 0x7f8aa00c5480 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa7be9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa006df40 con 0x7f8aa0155ba0 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa7be9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa006fdb0 con 0x7f8aa01586f0 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa5184640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8aa0155ba0 0x7f8aa00c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa5184640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8aa0155ba0 0x7f8aa00c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54350/0 (socket says 172.21.15.17:54350) 2024-03-06T07:25:18.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa5184640 1 -- 172.21.15.17:0/1650189505 learned_addr learned my addr 172.21.15.17:0/1650189505 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.087+0000 7f8aa6186640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa5184640 1 -- 172.21.15.17:0/1650189505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 msgr2=0x7f8aa0158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa5184640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa0158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa5184640 1 -- 172.21.15.17:0/1650189505 >> v1:172.21.15.163:6789/0 conn(0x7f8aa00c5480 legacy=0x7f8aa006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa5184640 1 -- 172.21.15.17:0/1650189505 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa01594a0 con 0x7f8aa0155ba0 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa5184640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa0155ba0 0x7f8aa00c5860 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009870 tx=0x7f8a9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a5d8506ba2c31c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa4983640 1 -- 172.21.15.17:0/1650189505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94035020 con 0x7f8aa0155ba0 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa4983640 1 -- 172.21.15.17:0/1650189505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94031640 con 0x7f8aa0155ba0 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.088+0000 7f8aa4983640 1 -- 172.21.15.17:0/1650189505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9402d040 con 0x7f8aa0155ba0 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 -- 172.21.15.17:0/1650189505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa0155ba0 msgr2=0x7f8aa00c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa0155ba0 0x7f8aa00c5860 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009870 tx=0x7f8a9402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 -- 172.21.15.17:0/1650189505 shutdown_connections 2024-03-06T07:25:18.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa0158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/1650189505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa0155ba0 0x7f8aa00c5860 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 -- 172.21.15.17:0/1650189505 >> 172.21.15.17:0/1650189505 conn(0x7f8aa006b1a0 msgr2=0x7f8aa006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 -- 172.21.15.17:0/1650189505 shutdown_connections 2024-03-06T07:25:18.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.089+0000 7f8aa7be9640 1 -- 172.21.15.17:0/1650189505 wait complete. 2024-03-06T07:25:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.090+0000 7f8aa7be9640 1 Processor -- start 2024-03-06T07:25:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.090+0000 7f8aa7be9640 1 -- start start 2024-03-06T07:25:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aa0155ba0 0x7f8aa01519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0155680 con 0x7f8aa00c5480 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0156000 con 0x7f8aa0155ba0 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa7be9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0152a20 con 0x7f8aa01586f0 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54364/0 (socket says 172.21.15.17:54364) 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 -- 172.21.15.17:0/160536134 learned_addr learned my addr 172.21.15.17:0/160536134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5184640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aa0155ba0 0x7f8aa01519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa6186640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 -- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 msgr2=0x7f8aa014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 -- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aa0155ba0 msgr2=0x7f8aa01519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aa0155ba0 0x7f8aa01519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.091+0000 7f8aa5985640 1 -- 172.21.15.17:0/160536134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00ad210 con 0x7f8aa00c5480 2024-03-06T07:25:18.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8aa5985640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c430 tx=0x7f8a9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980052d0 con 0x7f8aa00c5480 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a98005490 con 0x7f8aa00c5480 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98007260 con 0x7f8aa00c5480 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa00ad4f0 con 0x7f8aa00c5480 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.092+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01511d0 con 0x7f8aa00c5480 2024-03-06T07:25:18.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.093+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa004f330 con 0x7f8aa00c5480 2024-03-06T07:25:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.093+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a98007400 con 0x7f8aa00c5480 2024-03-06T07:25:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.094+0000 7f8a9e7fc640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 0x7f8a8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.094+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8a98089740 con 0x7f8aa00c5480 2024-03-06T07:25:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.094+0000 7f8aa5184640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 0x7f8a8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.094+0000 7f8aa5184640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 0x7f8a8c06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8a94036000 tx=0x7f8a9403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.098+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a98057e70 con 0x7f8aa00c5480 2024-03-06T07:25:18.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.316+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aa00c9560 con 0x7f8aa00c5480 2024-03-06T07:25:18.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.317+0000 7f8a9e7fc640 1 -- 172.21.15.17:0/160536134 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f8a9805bf10 con 0x7f8aa00c5480 2024-03-06T07:25:18.319 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:18.319 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":120,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7783,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:09.279368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8280":{"gid":8280,"name":"d","rank":0,"incarnation":118,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/3327948285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3327948285},{"type":"v1","addr":"172.21.15.163:6837","nonce":3327948285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8280,"qdb_cluster":[8280]},"id":10}]} 2024-03-06T07:25:18.319 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 120 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 msgr2=0x7f8a8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 0x7f8a8c06d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8a94036000 tx=0x7f8a9403c040 comp rx=0 tx=0).stop 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 msgr2=0x7f8aa00c3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800c430 tx=0x7f8a9800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 shutdown_connections 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8c06b4d0 0x7f8a8c06d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aa01586f0 0x7f8aa014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aa0155ba0 0x7f8aa01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 --2- 172.21.15.17:0/160536134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8aa00c5480 0x7f8aa00c3230 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 >> 172.21.15.17:0/160536134 conn(0x7f8aa006b1a0 msgr2=0x7f8aa00c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 shutdown_connections 2024-03-06T07:25:18.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.319+0000 7f8aa7be9640 1 -- 172.21.15.17:0/160536134 wait complete. 2024-03-06T07:25:18.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:09.279368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8280': {'gid': 8280, 'name': 'd', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/3327948285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3327948285}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3327948285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8280, 'qdb_cluster': [8280]} 2024-03-06T07:25:18.734 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:18.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:25:18.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.841+0000 7f00d23bc640 1 Processor -- start 2024-03-06T07:25:18.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.841+0000 7f00d23bc640 1 -- start start 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d23bc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00cc0c7170 0x7f00cc0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d23bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d23bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0a8390 con 0x7f00cc06ff10 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d23bc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0c7620 con 0x7f00cc0c7170 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d23bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0730f0 con 0x7f00cc0c6c40 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f00cc0c7170 0x7f00cc0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00cc06ff10 0x7f00cc153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54302/0 (socket says 172.21.15.17:54302) 2024-03-06T07:25:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cbfff640 1 -- 172.21.15.17:0/4079673772 learned_addr learned my addr 172.21.15.17:0/4079673772 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00d0959640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cb7fe640 1 -- 172.21.15.17:0/4079673772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 msgr2=0x7f00cc0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cb7fe640 1 -- 172.21.15.17:0/4079673772 >> v1:172.21.15.17:6789/0 conn(0x7f00cc06ff10 legacy=0x7f00cc153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00caffd640 1 -- 172.21.15.17:0/4079673772 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723269087 0 0) 0x7f00cc0a8390 con 0x7f00cc06ff10 2024-03-06T07:25:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.842+0000 7f00cb7fe640 1 -- 172.21.15.17:0/4079673772 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0733d0 con 0x7f00cc0c7170 2024-03-06T07:25:18.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.843+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc0c7170 0x7f00cc0c8810 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097a0 tx=0x7f00bc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f862687627f2db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.843+0000 7f00caffd640 1 -- 172.21.15.17:0/4079673772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc038070 con 0x7f00cc0c7170 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.843+0000 7f00caffd640 1 -- 172.21.15.17:0/4079673772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc031700 con 0x7f00cc0c7170 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.843+0000 7f00caffd640 1 -- 172.21.15.17:0/4079673772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc031ad0 con 0x7f00cc0c7170 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 -- 172.21.15.17:0/4079673772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc0c7170 msgr2=0x7f00cc0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc0c7170 0x7f00cc0c8810 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097a0 tx=0x7f00bc02b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 -- 172.21.15.17:0/4079673772 shutdown_connections 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 --2- 172.21.15.17:0/4079673772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc0c7170 0x7f00cc0c8810 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 -- 172.21.15.17:0/4079673772 >> 172.21.15.17:0/4079673772 conn(0x7f00cc06b970 msgr2=0x7f00cc0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 -- 172.21.15.17:0/4079673772 shutdown_connections 2024-03-06T07:25:18.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.844+0000 7f00d23bc640 1 -- 172.21.15.17:0/4079673772 wait complete. 2024-03-06T07:25:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.845+0000 7f00d23bc640 1 Processor -- start 2024-03-06T07:25:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.845+0000 7f00d23bc640 1 -- start start 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 0x7f00cc14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1544b0 con 0x7f00cc06ff10 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc154330 con 0x7f00cc0c7170 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0702f0 con 0x7f00cc0c6c40 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54396/0 (socket says 172.21.15.17:54396) 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 -- 172.21.15.17:0/1403271624 learned_addr learned my addr 172.21.15.17:0/1403271624 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d0959640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 0x7f00cc14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 -- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 msgr2=0x7f00cc0c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 -- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 msgr2=0x7f00cc14ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 0x7f00cc14ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 -- 172.21.15.17:0/1403271624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc1eb360 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d0959640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 0x7f00cc14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c460 tx=0x7f00b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00c8ff9640 1 -- 172.21.15.17:0/1403271624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4018070 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00c8ff9640 1 -- 172.21.15.17:0/1403271624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4007500 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc154930 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00c8ff9640 1 -- 172.21.15.17:0/1403271624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004a20 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.846+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc0c2460 con 0x7f00cc06ff10 2024-03-06T07:25:18.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.848+0000 7f00c8ff9640 1 -- 172.21.15.17:0/1403271624 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00b401b070 con 0x7f00cc06ff10 2024-03-06T07:25:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.848+0000 7f00c8ff9640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.848+0000 7f00c8ff9640 1 -- 172.21.15.17:0/1403271624 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00b40895d0 con 0x7f00cc06ff10 2024-03-06T07:25:18.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.848+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.849+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 0x7f00ac06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f00bc035000 tx=0x7f00bc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.849+0000 7f00d4fa2740 1 Processor -- start 2024-03-06T07:25:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.849+0000 7f00d4fa2740 1 -- start start 2024-03-06T07:25:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6c4b0 0x564ba1b68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1ad26b0 con 0x564ba1b6df50 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1a7ba60 con 0x564ba1b6e330 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba1ace750 con 0x564ba1b6c4b0 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54410/0 (socket says 172.21.15.17:54410) 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 -- 172.21.15.17:0/3929065362 learned_addr learned my addr 172.21.15.17:0/3929065362 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00d0959640 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6c4b0 0x564ba1b68680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 -- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6c4b0 msgr2=0x564ba1b68680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6c4b0 0x564ba1b68680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 -- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 msgr2=0x564ba1b6bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b6bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.850+0000 7f00cbfff640 1 -- 172.21.15.17:0/3929065362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba1a99f30 con 0x564ba1b6df50 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00cbfff640 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001120 tx=0x7f00b4058460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3409e3441a61245 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00c1ffb640 1 -- 172.21.15.17:0/3929065362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4060360 con 0x564ba1b6df50 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00c1ffb640 1 -- 172.21.15.17:0/3929065362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4055f50 con 0x564ba1b6df50 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00c1ffb640 1 -- 172.21.15.17:0/3929065362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004bc0 con 0x564ba1b6df50 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 -- 172.21.15.17:0/3929065362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 msgr2=0x564ba1b6ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f00b4001120 tx=0x7f00b4058460 comp rx=0 tx=0).stop 2024-03-06T07:25:18.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 -- 172.21.15.17:0/3929065362 shutdown_connections 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6c4b0 0x564ba1b68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b6bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/3929065362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b6ba30 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 -- 172.21.15.17:0/3929065362 >> 172.21.15.17:0/3929065362 conn(0x564ba1b6a7d0 msgr2=0x564ba1b8f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 -- 172.21.15.17:0/3929065362 shutdown_connections 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.851+0000 7f00d4fa2740 1 -- 172.21.15.17:0/3929065362 wait complete. 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.852+0000 7f00d4fa2740 1 -- start start 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.852+0000 7f00d4fa2740 1 Processor -- start 2024-03-06T07:25:18.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6e330 0x564ba1b92fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b926b0 0x564ba1b90600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1b90230 con 0x564ba1b6df50 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1a8eea0 con 0x564ba1b926b0 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba1b8fc70 con 0x564ba1b6e330 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 10 client.0 _ll_register_callbacks cb 0x564ba1a8e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54414/0 (socket says 172.21.15.17:54414) 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 -- 172.21.15.17:0/1625502474 learned_addr learned my addr 172.21.15.17:0/1625502474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d0959640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b926b0 0x564ba1b90600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6e330 0x564ba1b92fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 -- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6e330 msgr2=0x564ba1b92fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6e330 0x564ba1b92fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 -- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b926b0 msgr2=0x564ba1b90600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b926b0 0x564ba1b90600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 -- 172.21.15.17:0/1625502474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ba1a8f270 con 0x564ba1b6df50 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f00b4087a50 tx=0x7f00b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8253e1f918e40e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4060360 con 0x564ba1b6df50 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00b40570a0 con 0x564ba1b6df50 2024-03-06T07:25:18.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.853+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00b4055f50 con 0x564ba1b6df50 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00b40572c0 con 0x564ba1b6df50 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 10 client.8490 fetch_fsmap learned FSMap version 120 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x564ba17e72d0 con 0x564ba1b6df50 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b4055f50 con 0x564ba1b6df50 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 120) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f00b40539a0 con 0x564ba1b6df50 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 10 client.8490 fetch_fsmap finished waiting for FSMap version 120 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 10 client.8490 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 20 client.8490 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1bbe470 0x564ba1bbe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 4 client.8490 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 --> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ba1adfbf0 con 0x564ba1bbe470 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.854+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1bbe470 0x564ba1bbe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.855+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1bbe470 0x564ba1bbe850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60091153e67e6763 server_cookie=dd5e2efaaf47791d in_seq=0 out_seq=0 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.855+0000 7f00aaffd640 10 client.8490 ms_handle_connect on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:18.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.855+0000 7f00aaffd640 1 -- 172.21.15.17:0/1625502474 <== mds.0 v2:172.21.15.163:6835/3327948285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564ba1adfbf0 con 0x564ba1bbe470 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.855+0000 7f00aaffd640 10 client.8490 handle_command_reply: tid=0 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.855+0000 7f00d4fa2740 1 client.8490 shutdown 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 msgr2=0x564ba1b8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f00b4087a50 tx=0x7f00b4000ed0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 shutdown_connections 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1bbe470 0x564ba1bbe850 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b926b0 0x564ba1b90600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6e330 0x564ba1b92fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00aaffd640 0 client.8490 ms_handle_reset on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1625502474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6df50 0x564ba1b8e680 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 >> 172.21.15.17:0/1625502474 conn(0x564ba1b6c4b0 msgr2=0x564ba1baab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 shutdown_connections 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1625502474 wait complete. 2024-03-06T07:25:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.856+0000 7f00d4fa2740 20 client.8490 trim_cache size 0 max 16384 2024-03-06T07:25:18.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 Processor -- start 2024-03-06T07:25:18.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 -- start start 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 0x564ba1b92db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 0x564ba1b8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f6c50 con 0x564ba1b6e330 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f60a0 con 0x564ba1b6df50 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f6650 con 0x564ba1b926b0 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54420/0 (socket says 172.21.15.17:54420) 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00cb7fe640 1 -- 172.21.15.17:0/1755711800 learned_addr learned my addr 172.21.15.17:0/1755711800 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00d0959640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 0x564ba1b8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.873+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 0x564ba1b92db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 -- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 msgr2=0x564ba1b8e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 0x564ba1b8e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 -- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 msgr2=0x564ba1b92db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 0x564ba1b92db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 -- 172.21.15.17:0/1755711800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ba1b90970 con 0x564ba1b6e330 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d0959640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 0x564ba1b8e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cbfff640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 0x564ba1b92db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f00bc036040 tx=0x7f00bc031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e3e788495e3ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00c17fa640 1 -- 172.21.15.17:0/1755711800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc038040 con 0x564ba1b6e330 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00c17fa640 1 -- 172.21.15.17:0/1755711800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0501e0 con 0x564ba1b6e330 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00c17fa640 1 -- 172.21.15.17:0/1755711800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc050a70 con 0x564ba1b6e330 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1755711800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 msgr2=0x564ba1b8dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f00bc036040 tx=0x7f00bc031000 comp rx=0 tx=0).stop 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1755711800 shutdown_connections 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b926b0 0x564ba1b8e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b6e330 0x564ba1b8dd70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/1755711800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6df50 0x564ba1b92db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.874+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1755711800 >> 172.21.15.17:0/1755711800 conn(0x564ba1b6c4b0 msgr2=0x564ba1b6d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.875+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1755711800 shutdown_connections 2024-03-06T07:25:18.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.875+0000 7f00d4fa2740 1 -- 172.21.15.17:0/1755711800 wait complete. 2024-03-06T07:25:18.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.875+0000 7f00d4fa2740 1 -- start start 2024-03-06T07:25:18.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.875+0000 7f00d4fa2740 1 Processor -- start 2024-03-06T07:25:18.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6df50 0x564ba1b67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1b90230 con 0x564ba1b926b0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ba1a8eea0 con 0x564ba1b6e330 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ba1ba6ec0 con 0x564ba1b6df50 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d4fa2740 10 client.0 _ll_register_callbacks cb 0x7f00b4057860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54434/0 (socket says 172.21.15.17:54434) 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 -- 172.21.15.17:0/2118582611 learned_addr learned my addr 172.21.15.17:0/2118582611 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00cb7fe640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00cbfff640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6df50 0x564ba1b67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 -- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6df50 msgr2=0x564ba1b67a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6df50 0x564ba1b67a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 -- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 msgr2=0x564ba1b683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 -- 172.21.15.17:0/2118582611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ba18b16b0 con 0x564ba1b926b0 2024-03-06T07:25:18.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00d0959640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f00c4019e80 tx=0x7f00c4017780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367aebf6a6485595 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.876+0000 7f00aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4015540 con 0x564ba1b926b0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00c40156e0 con 0x564ba1b926b0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00bc0501e0 con 0x564ba1b926b0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00c402d050 con 0x564ba1b926b0 2024-03-06T07:25:18.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 10 client.8496 fetch_fsmap learned FSMap version 120 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x564ba1c1a9d0 con 0x564ba1b926b0 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0501e0 con 0x564ba1b926b0 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 120) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f00c4030020 con 0x564ba1b926b0 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 10 client.8496 fetch_fsmap finished waiting for FSMap version 120 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 10 client.8496 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 20 client.8496 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1b93a30 0x564ba1b93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 4 client.8496 mds_command: new command op to 8280 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 --> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564ba1ad9f40 con 0x564ba1b93a30 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.877+0000 7f00cbfff640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1b93a30 0x564ba1b93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:18.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00cbfff640 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1b93a30 0x564ba1b93e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548429198f3d4b29 server_cookie=acae21d105a677bb in_seq=0 out_seq=0 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00aaffd640 10 client.8496 ms_handle_connect on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00aaffd640 1 -- 172.21.15.17:0/2118582611 <== mds.0 v2:172.21.15.163:6835/3327948285 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x564ba1ad9f40 con 0x564ba1b93a30 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00aaffd640 10 client.8496 handle_command_reply: tid=0 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 client.8496 shutdown 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 msgr2=0x564ba1c16e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f00c4019e80 tx=0x7f00c4017780 comp rx=0 tx=0).stop 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 shutdown_connections 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:6835/3327948285,v1:172.21.15.163:6837/3327948285] conn(0x564ba1b93a30 0x564ba1b93e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564ba1b926b0 0x564ba1c16e40 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x564ba1b6e330 0x564ba1b683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 --2- 172.21.15.17:0/2118582611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x564ba1b6df50 0x564ba1b67a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00aaffd640 0 client.8496 ms_handle_reset on v2:172.21.15.163:6835/3327948285 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 >> 172.21.15.17:0/2118582611 conn(0x564ba1b6c4b0 msgr2=0x564ba1b94c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 shutdown_connections 2024-03-06T07:25:18.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 1 -- 172.21.15.17:0/2118582611 wait complete. 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.878+0000 7f00d4fa2740 20 client.8496 trim_cache size 0 max 16384 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8280, 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 10.951542409, 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 120, 2024-03-06T07:25:18.887 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 136, 2024-03-06T07:25:18.888 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 136, 2024-03-06T07:25:18.888 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 16.029324448000001 2024-03-06T07:25:18.888 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 msgr2=0x7f00ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 0x7f00ac06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f00bc035000 tx=0x7f00bc008030 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 msgr2=0x7f00cc0a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00b400c460 tx=0x7f00b400c920 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 shutdown_connections 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00cc0c7170 0x7f00cc14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00cc0c6c40 0x7f00cc0c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 --2- 172.21.15.17:0/1403271624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f00cc06ff10 0x7f00cc0a8940 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 >> 172.21.15.17:0/1403271624 conn(0x7f00cc06b970 msgr2=0x7f00cc0c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 shutdown_connections 2024-03-06T07:25:18.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:18.879+0000 7f00d23bc640 1 -- 172.21.15.17:0/1403271624 wait complete. 2024-03-06T07:25:18.891 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 120, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:07.286023+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 120, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8280": { "addr": "172.21.15.163:6837/3327948285", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3327948285, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3327948285, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8280, "incarnation": 118, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:09.279368+0000", "qdb_cluster": [ 8280 ], "qdb_leader": 8280, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8280 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/3255496602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3255496602, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3255496602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7783, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/824017697", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 824017697, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 824017697, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8265, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1732861093", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1732861093, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1732861093, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:18.892 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 10, 'name': 'b'}, {'join_fscid': 10, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:18.892 INFO:tasks.cephfs.test_failover:current = [{'gid': 8268, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1732861093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1732861093}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1732861093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 7783, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 8265, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 8280, 'name': 'd', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/3327948285', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3327948285}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3327948285}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:25:18.892 DEBUG:tasks.cephfs.test_failover:10 != -1 2024-03-06T07:25:18.893 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:25:18.983 INFO:tasks.ceph.mds.d.smithi163.stderr: -264> 2024-03-06T07:25:13.212+0000 7f15f2851640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:19.008 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:25:23.214 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:25:23.213+0000 7f4b43b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:23.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:24.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.007+0000 7fb98e61c640 1 Processor -- start 2024-03-06T07:25:24.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.007+0000 7fb98e61c640 1 -- start start 2024-03-06T07:25:24.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98e61c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9881506b0 0x7fb988153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98e61c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb9880acc00 0x7fb9880c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98e61c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb988070aa0 con 0x7fb988156ea0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98e61c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb988157320 con 0x7fb9880acc00 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98e61c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb988156450 con 0x7fb9881506b0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb9877fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb988156ea0 0x7fb9880702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48920/0 (socket says 172.21.15.17:48920) 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98cbb9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb9880acc00 0x7fb9880c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb9877fe640 1 -- 172.21.15.17:0/962169381 learned_addr learned my addr 172.21.15.17:0/962169381 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb987fff640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9881506b0 0x7fb988153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb986ffd640 1 -- 172.21.15.17:0/962169381 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069397305 0 0) 0x7fb988070aa0 con 0x7fb988156ea0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb986ffd640 1 -- 172.21.15.17:0/962169381 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb970003520 con 0x7fb988156ea0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98cbb9640 1 -- 172.21.15.17:0/962169381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9881506b0 msgr2=0x7fb988153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98cbb9640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9881506b0 0x7fb988153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98cbb9640 1 -- 172.21.15.17:0/962169381 >> v1:172.21.15.17:6789/0 conn(0x7fb988156ea0 legacy=0x7fb9880702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.008+0000 7fb98cbb9640 1 -- 172.21.15.17:0/962169381 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb988070d80 con 0x7fb9880acc00 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.009+0000 7fb98cbb9640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c54a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb980009d90 tx=0x7fb98002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced1e5a7316eb3c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.009+0000 7fb986ffd640 1 -- 172.21.15.17:0/962169381 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980039070 con 0x7fb9880acc00 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.009+0000 7fb986ffd640 1 -- 172.21.15.17:0/962169381 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98002cc50 con 0x7fb9880acc00 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.009+0000 7fb986ffd640 1 -- 172.21.15.17:0/962169381 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980034440 con 0x7fb9880acc00 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.009+0000 7fb98e61c640 1 -- 172.21.15.17:0/962169381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 msgr2=0x7fb9880c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c54a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fb980009d90 tx=0x7fb98002bea0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 -- 172.21.15.17:0/962169381 shutdown_connections 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c54a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 --2- 172.21.15.17:0/962169381 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9881506b0 0x7fb988153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 -- 172.21.15.17:0/962169381 >> 172.21.15.17:0/962169381 conn(0x7fb98806b1a0 msgr2=0x7fb98806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 -- 172.21.15.17:0/962169381 shutdown_connections 2024-03-06T07:25:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.010+0000 7fb98e61c640 1 -- 172.21.15.17:0/962169381 wait complete. 2024-03-06T07:25:24.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.011+0000 7fb98e61c640 1 Processor -- start 2024-03-06T07:25:24.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.011+0000 7fb98e61c640 1 -- start start 2024-03-06T07:25:24.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9881506b0 0x7fb9880c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb988156ea0 0x7fb988146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880ad090 con 0x7fb9880acc00 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988156a10 con 0x7fb9881506b0 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880ad210 con 0x7fb988156ea0 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46566/0 (socket says 172.21.15.17:46566) 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 -- 172.21.15.17:0/1343229640 learned_addr learned my addr 172.21.15.17:0/1343229640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb9877fe640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9881506b0 0x7fb9880c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98cbb9640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb988156ea0 0x7fb988146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 -- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb988156ea0 msgr2=0x7fb988146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb988156ea0 0x7fb988146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 -- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9881506b0 msgr2=0x7fb9880c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9881506b0 0x7fb9880c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 -- 172.21.15.17:0/1343229640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880c3500 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb987fff640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00e9b0 tx=0x7fb97c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c017070 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c00c9a0 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb988151e50 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.012+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00cd90 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.013+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb988152410 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.013+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98804f330 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.014+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb97c01a070 con 0x7fb9880acc00 2024-03-06T07:25:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.014+0000 7fb984ff9640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 0x7fb97406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.014+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb97c08a290 con 0x7fb9880acc00 2024-03-06T07:25:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.014+0000 7fb9877fe640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 0x7fb97406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.015+0000 7fb9877fe640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 0x7fb97406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb98814d860 tx=0x7fb978009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.019+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb97c058970 con 0x7fb9880acc00 2024-03-06T07:25:24.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.242+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9880700a0 con 0x7fb9880acc00 2024-03-06T07:25:24.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.243+0000 7fb984ff9640 1 -- 172.21.15.17:0/1343229640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4992 (secure 0 0 0) 0x7fb97c05ca10 con 0x7fb9880acc00 2024-03-06T07:25:24.245 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:24.246 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":125,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:21.987141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7783":{"gid":7783,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7783,"qdb_cluster":[7783]},"id":10}]} 2024-03-06T07:25:24.246 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 125 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 msgr2=0x7fb97406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 0x7fb97406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb98814d860 tx=0x7fb978009040 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 msgr2=0x7fb9880c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00e9b0 tx=0x7fb97c00ee70 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 shutdown_connections 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb97406b4d0 0x7fb97406d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb988156ea0 0x7fb988146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9881506b0 0x7fb9880c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 --2- 172.21.15.17:0/1343229640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9880acc00 0x7fb9880c2fc0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 >> 172.21.15.17:0/1343229640 conn(0x7fb98806b1a0 msgr2=0x7fb98814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 shutdown_connections 2024-03-06T07:25:24.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.245+0000 7fb98e61c640 1 -- 172.21.15.17:0/1343229640 wait complete. 2024-03-06T07:25:24.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:21.987141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7783}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7783': {'gid': 7783, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7783, 'qdb_cluster': [7783]} 2024-03-06T07:25:24.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:24.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:24.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.768+0000 7f2aa878a640 1 Processor -- start 2024-03-06T07:25:24.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.768+0000 7f2aa878a640 1 -- start start 2024-03-06T07:25:24.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa878a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa00c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa878a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2aa01587e0 0x7f2aa0158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa878a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa00c5140 con 0x7f2aa01526c0 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa878a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa00c3790 con 0x7f2aa01587e0 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa878a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0155bb0 con 0x7f2aa00c2b20 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6d27640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2aa01587e0 0x7f2aa0158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6526640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2aa01526c0 0x7f2aa00c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48924/0 (socket says 172.21.15.17:48924) 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6526640 1 -- 172.21.15.17:0/1795801567 learned_addr learned my addr 172.21.15.17:0/1795801567 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa00c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1795801567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 msgr2=0x7f2aa00c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa00c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1795801567 >> v1:172.21.15.17:6789/0 conn(0x7f2aa01526c0 legacy=0x7f2aa00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.769+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1795801567 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0155e90 con 0x7f2aa01587e0 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.770+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa0158c90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009d90 tx=0x7f2a9802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd78555d587b5ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.770+0000 7f2aa5524640 1 -- 172.21.15.17:0/1795801567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98039070 con 0x7f2aa01587e0 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.770+0000 7f2aa5524640 1 -- 172.21.15.17:0/1795801567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9802cbc0 con 0x7f2aa01587e0 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.770+0000 7f2aa5524640 1 -- 172.21.15.17:0/1795801567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98034440 con 0x7f2aa01587e0 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 -- 172.21.15.17:0/1795801567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 msgr2=0x7f2aa0158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa0158c90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009d90 tx=0x7f2a9802be10 comp rx=0 tx=0).stop 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 -- 172.21.15.17:0/1795801567 shutdown_connections 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa0158c90 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1795801567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa00c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 -- 172.21.15.17:0/1795801567 >> 172.21.15.17:0/1795801567 conn(0x7f2aa00aa810 msgr2=0x7f2aa00aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 -- 172.21.15.17:0/1795801567 shutdown_connections 2024-03-06T07:25:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.771+0000 7f2aa878a640 1 -- 172.21.15.17:0/1795801567 wait complete. 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.772+0000 7f2aa878a640 1 Processor -- start 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.772+0000 7f2aa878a640 1 -- start start 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.772+0000 7f2aa878a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa01519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.772+0000 7f2aa878a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa01526c0 0x7f2aa00a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00c9040 con 0x7f2aa01587e0 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0152aa0 con 0x7f2aa01526c0 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00c91c0 con 0x7f2aa00c2b20 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46584/0 (socket says 172.21.15.17:46584) 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1784414287 learned_addr learned my addr 172.21.15.17:0/1784414287 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa01519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa01526c0 0x7f2aa00a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 msgr2=0x7f2aa01519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa01519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa01526c0 msgr2=0x7f2aa00a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa01526c0 0x7f2aa00a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 -- 172.21.15.17:0/1784414287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0150010 con 0x7f2aa01587e0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009d90 tx=0x7f2a98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2a96ffd640 1 -- 172.21.15.17:0/1784414287 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98039050 con 0x7f2aa01587e0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2a96ffd640 1 -- 172.21.15.17:0/1784414287 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a98045070 con 0x7f2aa01587e0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2a96ffd640 1 -- 172.21.15.17:0/1784414287 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98009450 con 0x7f2aa01587e0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa01502f0 con 0x7f2aa01587e0 2024-03-06T07:25:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.773+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa00aa190 con 0x7f2aa01587e0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.775+0000 7f2a96ffd640 1 -- 172.21.15.17:0/1784414287 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a9803e850 con 0x7f2aa01587e0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.775+0000 7f2a96ffd640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 0x7f2a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.775+0000 7f2a96ffd640 1 -- 172.21.15.17:0/1784414287 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2a980ad090 con 0x7f2aa01587e0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.776+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 0x7f2a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.776+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 0x7f2a8406d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2aa006e210 tx=0x7f2a9c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.776+0000 7f2aab371740 1 Processor -- start 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 -- start start 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a684b0 0x563243a64680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632439ce6b0 con 0x563243a69f50 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563243977a60 con 0x563243a6a330 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632439ca750 con 0x563243a684b0 2024-03-06T07:25:24.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aa6526640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aa6526640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46598/0 (socket says 172.21.15.17:46598) 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aa6526640 1 -- 172.21.15.17:0/1275316637 learned_addr learned my addr 172.21.15.17:0/1275316637 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a684b0 0x563243a64680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.777+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 -- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a684b0 msgr2=0x563243a64680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a684b0 0x563243a64680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 -- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 msgr2=0x563243a67f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a67f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 -- 172.21.15.17:0/1275316637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563243995f30 con 0x563243a69f50 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aa6526640 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0017f0 tx=0x7f2a9c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c168960270cb94 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2a83fff640 1 -- 172.21.15.17:0/1275316637 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c026070 con 0x563243a69f50 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2a83fff640 1 -- 172.21.15.17:0/1275316637 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c013970 con 0x563243a69f50 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2a83fff640 1 -- 172.21.15.17:0/1275316637 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c013d60 con 0x563243a69f50 2024-03-06T07:25:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aab371740 1 -- 172.21.15.17:0/1275316637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 msgr2=0x563243a67a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.778+0000 7f2aab371740 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0017f0 tx=0x7f2a9c013000 comp rx=0 tx=0).stop 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 -- 172.21.15.17:0/1275316637 shutdown_connections 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a684b0 0x563243a64680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a67f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 --2- 172.21.15.17:0/1275316637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243a69f50 0x563243a67a30 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 -- 172.21.15.17:0/1275316637 >> 172.21.15.17:0/1275316637 conn(0x563243a667d0 msgr2=0x563243a8b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 -- 172.21.15.17:0/1275316637 shutdown_connections 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 -- 172.21.15.17:0/1275316637 wait complete. 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 -- start start 2024-03-06T07:25:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.779+0000 7f2aab371740 1 Processor -- start 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 0x563243a90990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a6a330 0x563243a8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563243a8c230 con 0x563243ab9d20 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324398aea0 con 0x563243a69f50 2024-03-06T07:25:24.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x563243a8bc70 con 0x563243a6a330 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aab371740 10 client.0 _ll_register_callbacks cb 0x56324398a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46608/0 (socket says 172.21.15.17:46608) 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aa6d27640 1 -- 172.21.15.17:0/3913996671 learned_addr learned my addr 172.21.15.17:0/3913996671 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aa6526640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 0x563243a90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.780+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a6a330 0x563243a8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 -- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a6a330 msgr2=0x563243a8ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a6a330 0x563243a8ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 -- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 msgr2=0x563243a90990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 0x563243a90990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 -- 172.21.15.17:0/3913996671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324398b270 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6526640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 0x563243a90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2a9807b590 tx=0x7f2a9802cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82939dee487ec247 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98093030 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2a980931d0 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a9c013970 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a98083ef0 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aab371740 10 client.8523 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x5632436e32d0 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c013970 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f2a98083a60 con 0x563243ab9d20 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.781+0000 7f2aab371740 10 client.8523 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aab371740 10 client.8523 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aab371740 20 client.8523 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243a8cbb0 0x563243a8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aab371740 4 client.8523 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632439dbbf0 con 0x563243a8cbb0 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aa6526640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243a8cbb0 0x563243a8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2aa6526640 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243a8cbb0 0x563243a8cf90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9982734220b2c1bb server_cookie=f3852e618b3f189c in_seq=0 out_seq=0 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.782+0000 7f2a817fa640 10 client.8523 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2a817fa640 1 -- 172.21.15.17:0/3913996671 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5632439dbbf0 con 0x563243a8cbb0 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2a817fa640 10 client.8523 handle_command_reply: tid=0 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 client.8523 shutdown 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 msgr2=0x563243a86fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2a9807b590 tx=0x7f2a9802cc80 comp rx=0 tx=0).stop 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 shutdown_connections 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243a8cbb0 0x563243a8cf90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a86fc0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a6a330 0x563243a8ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 --2- 172.21.15.17:0/3913996671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a69f50 0x563243a90990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2a817fa640 0 client.8523 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 >> 172.21.15.17:0/3913996671 conn(0x563243a684b0 msgr2=0x563243aa6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 shutdown_connections 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 1 -- 172.21.15.17:0/3913996671 wait complete. 2024-03-06T07:25:24.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.783+0000 7f2aab371740 20 client.8523 trim_cache size 0 max 16384 2024-03-06T07:25:24.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.800+0000 7f2aab371740 1 Processor -- start 2024-03-06T07:25:24.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.800+0000 7f2aab371740 1 -- start start 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa01f6fa0 con 0x563243ab9d20 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0149740 con 0x563243a6a330 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa01f69a0 con 0x563243a69f50 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46622/0 (socket says 172.21.15.17:46622) 2024-03-06T07:25:24.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2325226475 learned_addr learned my addr 172.21.15.17:0/2325226475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6526640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 msgr2=0x563243ab3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 msgr2=0x563243a65520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2325226475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632437ad6b0 con 0x563243ab9d20 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6526640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2a980ad3b0 tx=0x7f2a9807f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b08783d260c07d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2a837fe640 1 -- 172.21.15.17:0/2325226475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98093040 con 0x563243ab9d20 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2a837fe640 1 -- 172.21.15.17:0/2325226475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9803a040 con 0x563243ab9d20 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.801+0000 7f2a837fe640 1 -- 172.21.15.17:0/2325226475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a980931e0 con 0x563243ab9d20 2024-03-06T07:25:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- 172.21.15.17:0/2325226475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 msgr2=0x563243a675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2a980ad3b0 tx=0x7f2a9807f000 comp rx=0 tx=0).stop 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- 172.21.15.17:0/2325226475 shutdown_connections 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243a675a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 --2- 172.21.15.17:0/2325226475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- 172.21.15.17:0/2325226475 >> 172.21.15.17:0/2325226475 conn(0x563243a684b0 msgr2=0x563243a5fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- 172.21.15.17:0/2325226475 shutdown_connections 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- 172.21.15.17:0/2325226475 wait complete. 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 -- start start 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.802+0000 7f2aab371740 1 Processor -- start 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563243a8ca30 con 0x563243ab9d20 2024-03-06T07:25:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x563243a8df10 con 0x563243a6a330 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x563243a90090 con 0x563243a69f50 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aab371740 10 client.0 _ll_register_callbacks cb 0x7f2a9807b3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aa6d27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46624/0 (socket says 172.21.15.17:46624) 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2958914400 learned_addr learned my addr 172.21.15.17:0/2958914400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aa5d25640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.803+0000 7f2aa6526640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 msgr2=0x563243ab6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 msgr2=0x563243a65920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 -- 172.21.15.17:0/2958914400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563243a67d60 con 0x563243ab9d20 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aa6d27640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f2a9807b700 tx=0x7f2a98009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad4b17b584f4670 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:24.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98093040 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2a980931e0 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a9803a040 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9807ace0 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 10 client.8529 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x563243a67d60 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9803a040 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f2a9807ae80 con 0x563243ab9d20 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 10 client.8529 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 10 client.8529 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 20 client.8529 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243b12a80 0x563243b12e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 4 client.8529 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.804+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563243a8b460 con 0x563243b12a80 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2aa6526640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243b12a80 0x563243b12e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2aa6526640 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243b12a80 0x563243b12e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1546c6095e00c31 server_cookie=2ab3dd5c62450157 in_seq=0 out_seq=0 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2a817fa640 10 client.8529 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2a817fa640 1 -- 172.21.15.17:0/2958914400 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x563243a8b460 con 0x563243b12a80 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2a817fa640 10 client.8529 handle_command_reply: tid=0 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2aab371740 1 client.8529 shutdown 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 msgr2=0x563243b18f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.805+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f2a9807b700 tx=0x7f2a98009750 comp rx=0 tx=0).stop 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 shutdown_connections 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x563243b12a80 0x563243b12e60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563243ab9d20 0x563243b18f50 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x563243a6a330 0x563243a65920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 --2- 172.21.15.17:0/2958914400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x563243a69f50 0x563243ab6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2a817fa640 0 client.8529 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 >> 172.21.15.17:0/2958914400 conn(0x563243a684b0 msgr2=0x563243a89f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 shutdown_connections 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 1 -- 172.21.15.17:0/2958914400 wait complete. 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aab371740 20 client.8529 trim_cache size 0 max 16384 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7783, 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 5.8263811590000003, 2024-03-06T07:25:24.815 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 125, 2024-03-06T07:25:24.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 137, 2024-03-06T07:25:24.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 137, 2024-03-06T07:25:24.816 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 21.957798378 2024-03-06T07:25:24.816 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 msgr2=0x7f2a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 0x7f2a8406d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2aa006e210 tx=0x7f2a9c007e80 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 msgr2=0x7f2aa006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009d90 tx=0x7f2a98004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 shutdown_connections 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2a8406b4d0 0x7f2a8406d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2aa01587e0 0x7f2aa006f5e0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aa01526c0 0x7f2aa00a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 --2- 172.21.15.17:0/1784414287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aa00c2b20 0x7f2aa01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 >> 172.21.15.17:0/1784414287 conn(0x7f2aa00aa810 msgr2=0x7f2aa014f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 shutdown_connections 2024-03-06T07:25:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:24.806+0000 7f2aa878a640 1 -- 172.21.15.17:0/1784414287 wait complete. 2024-03-06T07:25:24.819 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 125, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:07.286023+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 125, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7783": { "addr": "172.21.15.163:6836/3255496602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3255496602, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3255496602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7783, "incarnation": 122, "join_fscid": 10, "name": "b", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 137, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:21.987141+0000", "qdb_cluster": [ 7783 ], "qdb_leader": 7783, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7783 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/3725508931", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3725508931, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3725508931, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 123, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6926, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/824017697", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 824017697, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 824017697, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 121, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8265, "incarnation": 0, "join_fscid": 10, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 5 }, { "addr": "172.21.15.17:6835/1732861093", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1732861093, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1732861093, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:24.820 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': 10, 'name': 'b'}, {'join_fscid': 10, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:24.821 INFO:tasks.cephfs.test_failover:current = [{'gid': 8268, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1732861093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1732861093}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1732861093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 7783, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8265, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 5, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 121}, {'gid': 6926, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/3725508931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3725508931}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3725508931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 123}] 2024-03-06T07:25:24.821 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:25:29.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:29.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.932+0000 7f178bdbc640 1 Processor -- start 2024-03-06T07:25:29.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.933+0000 7f178bdbc640 1 -- start start 2024-03-06T07:25:29.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.933+0000 7f178bdbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 0x7f1784155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.933+0000 7f178bdbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840c73d0 con 0x7f178414f360 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.933+0000 7f178bdbc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17840be2e0 con 0x7f17840c88c0 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.933+0000 7f178bdbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1784159700 con 0x7f1784159270 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f1789b58640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f178414f360 0x7f17840c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48928/0 (socket says 172.21.15.17:48928) 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f1789b58640 1 -- 172.21.15.17:0/276330757 learned_addr learned my addr 172.21.15.17:0/276330757 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f178a359640 1 --2- 172.21.15.17:0/276330757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 0x7f1784155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353553899 0 0) 0x7f17840c73d0 con 0x7f178414f360 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1770003520 con 0x7f178414f360 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f178a359640 1 -- 172.21.15.17:0/276330757 >> v1:172.21.15.163:6789/0 conn(0x7f17840c88c0 legacy=0x7f1784158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718700578 0 0) 0x7f17840be2e0 con 0x7f17840c88c0 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f178a359640 1 -- 172.21.15.17:0/276330757 >> v1:172.21.15.17:6789/0 conn(0x7f178414f360 legacy=0x7f17840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.934+0000 7f178a359640 1 -- 172.21.15.17:0/276330757 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840c7670 con 0x7f1784159270 2024-03-06T07:25:29.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f178a359640 1 --2- 172.21.15.17:0/276330757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 0x7f1784155c10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f177c009dc0 tx=0x7f177c02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15cef58fb57c03a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c039070 con 0x7f1784159270 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c02cc40 con 0x7f1784159270 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f1788b56640 1 -- 172.21.15.17:0/276330757 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c034440 con 0x7f1784159270 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f178bdbc640 1 -- 172.21.15.17:0/276330757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 msgr2=0x7f1784155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f178bdbc640 1 --2- 172.21.15.17:0/276330757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 0x7f1784155c10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f177c009dc0 tx=0x7f177c02be90 comp rx=0 tx=0).stop 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f178bdbc640 1 -- 172.21.15.17:0/276330757 shutdown_connections 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.935+0000 7f178bdbc640 1 --2- 172.21.15.17:0/276330757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1784159270 0x7f1784155c10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.936+0000 7f178bdbc640 1 -- 172.21.15.17:0/276330757 >> 172.21.15.17:0/276330757 conn(0x7f17840ac570 msgr2=0x7f17840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.936+0000 7f178bdbc640 1 -- 172.21.15.17:0/276330757 shutdown_connections 2024-03-06T07:25:29.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.936+0000 7f178bdbc640 1 -- 172.21.15.17:0/276330757 wait complete. 2024-03-06T07:25:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 Processor -- start 2024-03-06T07:25:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 -- start start 2024-03-06T07:25:29.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 0x7f178414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784159270 0x7f17841ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c7c30 con 0x7f178414f360 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c5a50 con 0x7f1784159270 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.937+0000 7f178bdbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c5bd0 con 0x7f17840c88c0 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789b58640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 0x7f178414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46634/0 (socket says 172.21.15.17:46634) 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 -- 172.21.15.17:0/1878907613 learned_addr learned my addr 172.21.15.17:0/1878907613 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f178a359640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784159270 0x7f17841ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 -- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 msgr2=0x7f178414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 0x7f178414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 -- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784159270 msgr2=0x7f17841ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784159270 0x7f17841ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 -- 172.21.15.17:0/1878907613 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17841eeb50 con 0x7f178414f360 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789b58640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 0x7f178414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f1789357640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f177400c410 tx=0x7f177400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774017070 con 0x7f178414f360 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740035f0 con 0x7f178414f360 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774003a00 con 0x7f178414f360 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17840ad660 con 0x7f178414f360 2024-03-06T07:25:29.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.938+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17840afee0 con 0x7f178414f360 2024-03-06T07:25:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.940+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178404f330 con 0x7f178414f360 2024-03-06T07:25:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.940+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f177401a070 con 0x7f178414f360 2024-03-06T07:25:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.940+0000 7f177a7fc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 0x7f176806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.940+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1774089690 con 0x7f178414f360 2024-03-06T07:25:29.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.940+0000 7f1789b58640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 0x7f176806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:29.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.941+0000 7f1789b58640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 0x7f176806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1780005880 tx=0x7f1780009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:29.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:29.945+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1774057d70 con 0x7f178414f360 2024-03-06T07:25:30.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.165+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17840c8d50 con 0x7f178414f360 2024-03-06T07:25:30.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.165+0000 7f177a7fc640 1 -- 172.21.15.17:0/1878907613 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4992 (secure 0 0 0) 0x7f177405e0c0 con 0x7f178414f360 2024-03-06T07:25:30.168 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:30.168 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":125,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:21.987141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7783":{"gid":7783,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7783,"qdb_cluster":[7783]},"id":10}]} 2024-03-06T07:25:30.168 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 125 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 msgr2=0x7f176806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 0x7f176806d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1780005880 tx=0x7f1780009040 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 msgr2=0x7f17841ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f177400c410 tx=0x7f177400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 shutdown_connections 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f176806b4d0 0x7f176806d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1784159270 0x7f17841ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f178414f360 0x7f17841ee0d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 --2- 172.21.15.17:0/1878907613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17840c88c0 0x7f178414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 >> 172.21.15.17:0/1878907613 conn(0x7f17840ac570 msgr2=0x7f1784158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 shutdown_connections 2024-03-06T07:25:30.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.167+0000 7f178bdbc640 1 -- 172.21.15.17:0/1878907613 wait complete. 2024-03-06T07:25:30.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:21.987141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7783}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7783': {'gid': 7783, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7783, 'qdb_cluster': [7783]} 2024-03-06T07:25:30.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:30.583 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:30.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.692+0000 7fcba589a640 1 Processor -- start 2024-03-06T07:25:30.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.692+0000 7fcba589a640 1 -- start start 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcba589a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcba00bfe90 0x7fcba0155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcba589a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba00c7200 con 0x7fcba00c6d70 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcba589a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba0159590 con 0x7fcba01522d0 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcba589a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba0158710 con 0x7fcba00bfe90 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb97fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcba00c6d70 0x7fcba0158e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50802/0 (socket says 172.21.15.17:50802) 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb97fff640 1 -- 172.21.15.17:0/1799283074 learned_addr learned my addr 172.21.15.17:0/1799283074 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9effd640 1 --2- 172.21.15.17:0/1799283074 >> v2:172.21.15.163:3300/0 conn(0x7fcba00bfe90 0x7fcba0155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507954461 0 0) 0x7fcb88001110 con 0x7fcba00c6d70 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb8c003520 con 0x7fcba00c6d70 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996823698 0 0) 0x7fcba0159590 con 0x7fcba01522d0 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb88001110 con 0x7fcba01522d0 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9effd640 1 -- 172.21.15.17:0/1799283074 >> v1:172.21.15.163:6789/0 conn(0x7fcba01522d0 legacy=0x7fcba015faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9effd640 1 -- 172.21.15.17:0/1799283074 >> v1:172.21.15.17:6789/0 conn(0x7fcba00c6d70 legacy=0x7fcba0158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.693+0000 7fcb9effd640 1 -- 172.21.15.17:0/1799283074 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00c59d0 con 0x7fcba00bfe90 2024-03-06T07:25:30.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.694+0000 7fcb9effd640 1 --2- 172.21.15.17:0/1799283074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00bfe90 0x7fcba0155120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcb900097d0 tx=0x7fcb9002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8ab9143cd27023 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.694+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90035020 con 0x7fcba00bfe90 2024-03-06T07:25:30.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.694+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90031830 con 0x7fcba00bfe90 2024-03-06T07:25:30.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.694+0000 7fcb9e7fc640 1 -- 172.21.15.17:0/1799283074 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90031c00 con 0x7fcba00bfe90 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 -- 172.21.15.17:0/1799283074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00bfe90 msgr2=0x7fcba0155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 --2- 172.21.15.17:0/1799283074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00bfe90 0x7fcba0155120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcb900097d0 tx=0x7fcb9002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 -- 172.21.15.17:0/1799283074 shutdown_connections 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 --2- 172.21.15.17:0/1799283074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00bfe90 0x7fcba0155120 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 -- 172.21.15.17:0/1799283074 >> 172.21.15.17:0/1799283074 conn(0x7fcba00acaa0 msgr2=0x7fcba00aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 -- 172.21.15.17:0/1799283074 shutdown_connections 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.695+0000 7fcba589a640 1 -- 172.21.15.17:0/1799283074 wait complete. 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.696+0000 7fcba589a640 1 Processor -- start 2024-03-06T07:25:30.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.696+0000 7fcba589a640 1 -- start start 2024-03-06T07:25:30.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 0x7fcba00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 0x7fcba01ead30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0158960 con 0x7fcba00bfe90 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c5f90 con 0x7fcba00c6d70 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcba589a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00ca700 con 0x7fcba01522d0 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57284/0 (socket says 172.21.15.17:57284) 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 -- 172.21.15.17:0/426735756 learned_addr learned my addr 172.21.15.17:0/426735756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb97fff640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 0x7fcba00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 0x7fcba01ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 -- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 msgr2=0x7fcba01ead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 0x7fcba01ead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 -- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 msgr2=0x7fcba00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 0x7fcba00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 -- 172.21.15.17:0/426735756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0154ac0 con 0x7fcba00bfe90 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb97fff640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 0x7fcba00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 0x7fcba01ead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.697+0000 7fcb9effd640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcb900098f0 tx=0x7fcb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.698+0000 7fcb9cff9640 1 -- 172.21.15.17:0/426735756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90040070 con 0x7fcba00bfe90 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.698+0000 7fcb9cff9640 1 -- 172.21.15.17:0/426735756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9003a830 con 0x7fcba00bfe90 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.698+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba0154da0 con 0x7fcba00bfe90 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.698+0000 7fcb9cff9640 1 -- 172.21.15.17:0/426735756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90035040 con 0x7fcba00bfe90 2024-03-06T07:25:30.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.698+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba006c730 con 0x7fcba00bfe90 2024-03-06T07:25:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.700+0000 7fcb9cff9640 1 -- 172.21.15.17:0/426735756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb90038020 con 0x7fcba00bfe90 2024-03-06T07:25:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.700+0000 7fcb9cff9640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.700+0000 7fcb9cff9640 1 -- 172.21.15.17:0/426735756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb900abb30 con 0x7fcba00bfe90 2024-03-06T07:25:30.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.700+0000 7fcb97fff640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.701+0000 7fcba8122740 1 Processor -- start 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.701+0000 7fcb97fff640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 0x7fcb8006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007fd0 tx=0x7fcb88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.701+0000 7fcba8122740 1 -- start start 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec27a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec27f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 0x56071ec24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071eb8e6b0 con 0x56071ec284b0 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071eb37a60 con 0x56071ec2a330 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56071eb8a750 con 0x56071ec29f50 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb9f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 0x56071ec24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb9f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 0x56071ec24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57300/0 (socket says 172.21.15.17:57300) 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb9f7fe640 1 -- 172.21.15.17:0/3270513903 learned_addr learned my addr 172.21.15.17:0/3270513903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec27f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec27a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 -- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 msgr2=0x56071ec27a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec27a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 -- 172.21.15.17:0/3270513903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 msgr2=0x56071ec24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 0x56071ec24680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 -- 172.21.15.17:0/3270513903 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56071eb55f30 con 0x56071ec2a330 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec27f70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcba00aad20 tx=0x7fcb88011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7dc3100b2ef6ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb957fa640 1 -- 172.21.15.17:0/3270513903 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880070d0 con 0x56071ec2a330 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb957fa640 1 -- 172.21.15.17:0/3270513903 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88007270 con 0x56071ec2a330 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.702+0000 7fcb957fa640 1 -- 172.21.15.17:0/3270513903 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88007620 con 0x56071ec2a330 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 -- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 msgr2=0x56071ec27f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec27f70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcba00aad20 tx=0x7fcb88011000 comp rx=0 tx=0).stop 2024-03-06T07:25:30.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 -- 172.21.15.17:0/3270513903 shutdown_connections 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec284b0 0x56071ec24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec27f70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 --2- 172.21.15.17:0/3270513903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 -- 172.21.15.17:0/3270513903 >> 172.21.15.17:0/3270513903 conn(0x56071ec267d0 msgr2=0x56071ec4b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 -- 172.21.15.17:0/3270513903 shutdown_connections 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.703+0000 7fcba8122740 1 -- 172.21.15.17:0/3270513903 wait complete. 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.704+0000 7fcba8122740 1 -- start start 2024-03-06T07:25:30.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.704+0000 7fcba8122740 1 Processor -- start 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.704+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec4a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 0x56071ec763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071ec4c230 con 0x56071ec4f370 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071eb4aea0 con 0x56071ec2a330 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56071ec4bc70 con 0x56071ec29f50 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 10 client.0 _ll_register_callbacks cb 0x56071eb4a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:30.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 0x56071ec763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43778/0 (socket says 172.21.15.17:43778) 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9effd640 1 -- 172.21.15.17:0/280060632 learned_addr learned my addr 172.21.15.17:0/280060632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb97fff640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec4a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 -- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 msgr2=0x56071ec4c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 -- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 msgr2=0x56071ec4a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec4a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 -- 172.21.15.17:0/280060632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56071eb4b270 con 0x56071ec4f370 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9effd640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 0x56071ec763f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcb980115f0 tx=0x7fcb98011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd2b55325ba073 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98003c40 con 0x56071ec4f370 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb88007270 con 0x56071ec4f370 2024-03-06T07:25:30.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb98003de0 con 0x56071ec4f370 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb98022050 con 0x56071ec4f370 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 10 client.8547 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.705+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x56071e8a32d0 con 0x56071ec4f370 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88007270 con 0x56071ec4f370 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcb98025020 con 0x56071ec4f370 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 10 client.8547 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 10 client.8547 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 20 client.8547 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ec76ab0 0x56071ec76e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 4 client.8547 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56071eb9bbf0 con 0x56071ec76ab0 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcb9effd640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ec76ab0 0x56071ec76e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcb7e7fc640 10 client.8547 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.706+0000 7fcb9effd640 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ec76ab0 0x56071ec76e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732fd5a9462b3de5 server_cookie=b94c231305f7a7c0 in_seq=0 out_seq=0 2024-03-06T07:25:30.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/280060632 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56071eb9bbf0 con 0x56071ec76ab0 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcb7e7fc640 10 client.8547 handle_command_reply: tid=0 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 client.8547 shutdown 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 msgr2=0x56071ec763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 0x56071ec763f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcb980115f0 tx=0x7fcb98011ab0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 shutdown_connections 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ec76ab0 0x56071ec76e90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec4f370 0x56071ec763f0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcb7e7fc640 0 client.8547 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec2a330 0x56071ec4a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 --2- 172.21.15.17:0/280060632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec4c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 >> 172.21.15.17:0/280060632 conn(0x56071ec284b0 msgr2=0x56071ec288e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 shutdown_connections 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 1 -- 172.21.15.17:0/280060632 wait complete. 2024-03-06T07:25:30.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.707+0000 7fcba8122740 20 client.8547 trim_cache size 0 max 16384 2024-03-06T07:25:30.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.724+0000 7fcba8122740 1 Processor -- start 2024-03-06T07:25:30.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.724+0000 7fcba8122740 1 -- start start 2024-03-06T07:25:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec23510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec27cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba01f6e80 con 0x56071ec2a330 2024-03-06T07:25:30.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0153b40 con 0x56071ec4f370 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba01f6880 con 0x56071ec29f50 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57324/0 (socket says 172.21.15.17:57324) 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb97fff640 1 -- 172.21.15.17:0/3313293908 learned_addr learned my addr 172.21.15.17:0/3313293908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec23510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec27cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 -- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 msgr2=0x56071ec27cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec27cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 -- 172.21.15.17:0/3313293908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 msgr2=0x56071ec27790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb9effd640 1 -- 172.21.15.17:0/3313293908 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56071e96d6b0 con 0x56071ec29f50 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.725+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec23510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb9002bee0 tx=0x7fcb900a9d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1074cc2e867c2bd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcb94ff9640 1 -- 172.21.15.17:0/3313293908 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900827d0 con 0x56071ec29f50 2024-03-06T07:25:30.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcb94ff9640 1 -- 172.21.15.17:0/3313293908 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90091e50 con 0x56071ec29f50 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcb94ff9640 1 -- 172.21.15.17:0/3313293908 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900783c0 con 0x56071ec29f50 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 -- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 msgr2=0x56071ec23510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec23510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb9002bee0 tx=0x7fcb900a9d40 comp rx=0 tx=0).stop 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 -- 172.21.15.17:0/3313293908 shutdown_connections 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec27cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec2a330 0x56071ec27790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 --2- 172.21.15.17:0/3313293908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec29f50 0x56071ec23510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 -- 172.21.15.17:0/3313293908 >> 172.21.15.17:0/3313293908 conn(0x56071ec284b0 msgr2=0x56071ec2a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 -- 172.21.15.17:0/3313293908 shutdown_connections 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.726+0000 7fcba8122740 1 -- 172.21.15.17:0/3313293908 wait complete. 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.727+0000 7fcba8122740 1 -- start start 2024-03-06T07:25:30.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.727+0000 7fcba8122740 1 Processor -- start 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec2a330 0x56071ec23910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec4e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071ec76930 con 0x56071ec29f50 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56071ec4fa30 con 0x56071ec4f370 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56071ec4ddf0 con 0x56071ec2a330 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 10 client.0 _ll_register_callbacks cb 0x7fcb98001650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57332/0 (socket says 172.21.15.17:57332) 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 -- 172.21.15.17:0/3847207572 learned_addr learned my addr 172.21.15.17:0/3847207572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9f7fe640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec4e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec2a330 0x56071ec23910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 -- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec2a330 msgr2=0x56071ec23910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec2a330 0x56071ec23910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 -- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 msgr2=0x56071ec4e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec4e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 -- 172.21.15.17:0/3847207572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56071ec4ee70 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb9effd640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcb90079d70 tx=0x7fcb9003c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74903abd7fc1b316 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900783c0 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb900a9e70 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb90091e50 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb90076730 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 10 client.8553 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.728+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x56071ecd97b0 con 0x56071ec29f50 2024-03-06T07:25:30.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90091e50 con 0x56071ec29f50 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fcb90082340 con 0x56071ec29f50 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 10 client.8553 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 10 client.8553 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 20 client.8553 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ecd3790 0x56071ec62180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 4 client.8553 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56071eb95f40 con 0x56071ecd3790 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ecd3790 0x56071ec62180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb7e7fc640 10 client.8553 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb97fff640 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ecd3790 0x56071ec62180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce7f5b48dcb11d0 server_cookie=7a9131e03a80125d in_seq=0 out_seq=0 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb7e7fc640 1 -- 172.21.15.17:0/3847207572 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56071eb95f40 con 0x56071ecd3790 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcb7e7fc640 10 client.8553 handle_command_reply: tid=0 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 1 client.8553 shutdown 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 msgr2=0x56071ec49670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.729+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcb90079d70 tx=0x7fcb9003c040 comp rx=0 tx=0).stop 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 shutdown_connections 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x56071ecd3790 0x56071ec62180 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56071ec4f370 0x56071ec4e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56071ec2a330 0x56071ec23910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcb7e7fc640 0 client.8553 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 --2- 172.21.15.17:0/3847207572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56071ec29f50 0x56071ec49670 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 >> 172.21.15.17:0/3847207572 conn(0x56071ec284b0 msgr2=0x56071ec738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 shutdown_connections 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 1 -- 172.21.15.17:0/3847207572 wait complete. 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba8122740 20 client.8553 trim_cache size 0 max 16384 2024-03-06T07:25:30.738 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7783, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 11.750682810000001, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 125, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 137, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 137, 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 27.882096812 2024-03-06T07:25:30.739 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 msgr2=0x7fcb8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 0x7fcb8006d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007fd0 tx=0x7fcb88009040 comp rx=0 tx=0).stop 2024-03-06T07:25:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 msgr2=0x7fcba006c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcb900098f0 tx=0x7fcb900046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 shutdown_connections 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcb8006b4d0 0x7fcb8006d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcba01522d0 0x7fcba01ead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcba00c6d70 0x7fcba00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 --2- 172.21.15.17:0/426735756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcba00bfe90 0x7fcba006c1f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 >> 172.21.15.17:0/426735756 conn(0x7fcba00acaa0 msgr2=0x7fcba00aa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 shutdown_connections 2024-03-06T07:25:30.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.730+0000 7fcba589a640 1 -- 172.21.15.17:0/426735756 wait complete. 2024-03-06T07:25:30.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:25:30.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.844+0000 7f92185c9640 1 Processor -- start 2024-03-06T07:25:30.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.844+0000 7f92185c9640 1 -- start start 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f92100c88f0 0x7f9210155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92100c58b0 0x7f9210155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100c9130 con 0x7f92100c58b0 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9210159260 con 0x7f92100c88f0 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f92185c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100c6b90 con 0x7f92101527f0 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92100c58b0 0x7f9210155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216365640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f92100c58b0 0x7f9210155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57334/0 (socket says 172.21.15.17:57334) 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216365640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43810/0 (socket says 172.21.15.17:43810) 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 -- 172.21.15.17:0/4246312915 learned_addr learned my addr 172.21.15.17:0/4246312915 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9215b64640 1 --2- 172.21.15.17:0/4246312915 >> v2:172.21.15.163:3300/0 conn(0x7f92100c88f0 0x7f9210155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 -- 172.21.15.17:0/4246312915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 msgr2=0x7f92100c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 -- 172.21.15.17:0/4246312915 >> v2:172.21.15.163:3300/0 conn(0x7f92100c88f0 msgr2=0x7f9210155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 --2- 172.21.15.17:0/4246312915 >> v2:172.21.15.163:3300/0 conn(0x7f92100c88f0 0x7f9210155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216b66640 1 -- 172.21.15.17:0/4246312915 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92101596f0 con 0x7f92100c58b0 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.845+0000 7f9216365640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.846+0000 7f9216b66640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92100c58b0 0x7f9210155750 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f921008c4f0 tx=0x7f920002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e02bb22929619a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.846+0000 7f9215363640 1 -- 172.21.15.17:0/4246312915 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920002c950 con 0x7f92100c58b0 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.846+0000 7f9215363640 1 -- 172.21.15.17:0/4246312915 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920002caf0 con 0x7f92100c58b0 2024-03-06T07:25:30.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.846+0000 7f9215363640 1 -- 172.21.15.17:0/4246312915 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000343f0 con 0x7f92100c58b0 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 -- 172.21.15.17:0/4246312915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92100c58b0 msgr2=0x7f9210155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92100c58b0 0x7f9210155750 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f921008c4f0 tx=0x7f920002bdb0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 -- 172.21.15.17:0/4246312915 shutdown_connections 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92100c58b0 0x7f9210155750 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 --2- 172.21.15.17:0/4246312915 >> v2:172.21.15.163:3300/0 conn(0x7f92100c88f0 0x7f9210155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 --2- 172.21.15.17:0/4246312915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92101527f0 0x7f92100c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 -- 172.21.15.17:0/4246312915 >> 172.21.15.17:0/4246312915 conn(0x7f92100ac580 msgr2=0x7f92100ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 -- 172.21.15.17:0/4246312915 shutdown_connections 2024-03-06T07:25:30.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.847+0000 7f92185c9640 1 -- 172.21.15.17:0/4246312915 wait complete. 2024-03-06T07:25:30.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.848+0000 7f92185c9640 1 Processor -- start 2024-03-06T07:25:30.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.848+0000 7f92185c9640 1 -- start start 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92100c58b0 0x7f92101ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92100c88f0 0x7f92101e0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210158c10 con 0x7f92101527f0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210158930 con 0x7f92100c58b0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210158ef0 con 0x7f92100c88f0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57342/0 (socket says 172.21.15.17:57342) 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 -- 172.21.15.17:0/2699568308 learned_addr learned my addr 172.21.15.17:0/2699568308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216365640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92100c58b0 0x7f92101ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9215b64640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92100c88f0 0x7f92101e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 -- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92100c88f0 msgr2=0x7f92101e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92100c88f0 0x7f92101e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 -- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92100c58b0 msgr2=0x7f92101ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92100c58b0 0x7f92101ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 -- 172.21.15.17:0/2699568308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100bead0 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f9216b66640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f40 tx=0x7f9200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200043020 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920003e450 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92101eea80 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200009510 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.849+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92101e0fc0 con 0x7f92101527f0 2024-03-06T07:25:30.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.850+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921004f330 con 0x7f92101527f0 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.851+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9200046020 con 0x7f92101527f0 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.851+0000 7f920effd640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.851+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f92000ac990 con 0x7f92101527f0 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.851+0000 7f9216365640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.852+0000 7f9216365640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 0x7f91fc06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f92101df890 tx=0x7f9208009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:30.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:30.856+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f920004d030 con 0x7f92101527f0 2024-03-06T07:25:31.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.037+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7f92100c8080 con 0x7f92101527f0 2024-03-06T07:25:31.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.039+0000 7f920effd640 1 -- 172.21.15.17:0/2699568308 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v48) v1 ==== 96+0+0 (secure 0 0 0) 0x7f920007af50 con 0x7f92101527f0 2024-03-06T07:25:31.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 msgr2=0x7f91fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 0x7f91fc06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f92101df890 tx=0x7f9208009190 comp rx=0 tx=0).stop 2024-03-06T07:25:31.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 msgr2=0x7f92101ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f40 tx=0x7f9200004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:31.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 shutdown_connections 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f92101527f0 0x7f92101ee180 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92100c88f0 0x7f92101e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 --2- 172.21.15.17:0/2699568308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92100c58b0 0x7f92101ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 >> 172.21.15.17:0/2699568308 conn(0x7f92100ac580 msgr2=0x7f9210151980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 shutdown_connections 2024-03-06T07:25:31.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.041+0000 7f92185c9640 1 -- 172.21.15.17:0/2699568308 wait complete. 2024-03-06T07:25:31.055 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:31.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 Processor -- start 2024-03-06T07:25:31.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 -- start start 2024-03-06T07:25:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57c80c8a00 0x7f57c8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c8159460 0x7f57c80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c80c7760 con 0x7f57c8152920 2024-03-06T07:25:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c81587b0 con 0x7f57c80c8a00 2024-03-06T07:25:31.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.153+0000 7f57cff25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80c7a40 con 0x7f57c8159460 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cd4c0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f57c80c8a00 0x7f57c8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cdcc1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57c8152920 0x7f57c8155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50806/0 (socket says 172.21.15.17:50806) 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cdcc1640 1 -- 172.21.15.17:0/2996848291 learned_addr learned my addr 172.21.15.17:0/2996848291 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57ce4c2640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c8159460 0x7f57c80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cd4c0640 1 -- 172.21.15.17:0/2996848291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c8159460 msgr2=0x7f57c80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cd4c0640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c8159460 0x7f57c80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cd4c0640 1 -- 172.21.15.17:0/2996848291 >> v1:172.21.15.17:6789/0 conn(0x7f57c8152920 legacy=0x7f57c8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cd4c0640 1 -- 172.21.15.17:0/2996848291 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80c7d20 con 0x7f57c80c8a00 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.154+0000 7f57cccbf640 1 -- 172.21.15.17:0/2996848291 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146929033 0 0) 0x7f57c80c7760 con 0x7f57c8152920 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cd4c0640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c80c8a00 0x7f57c8158f20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097a0 tx=0x7f57b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b803ff973d07eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cccbf640 1 -- 172.21.15.17:0/2996848291 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80315a0 con 0x7f57c80c8a00 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cccbf640 1 -- 172.21.15.17:0/2996848291 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8037070 con 0x7f57c80c8a00 2024-03-06T07:25:31.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cccbf640 1 -- 172.21.15.17:0/2996848291 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8031960 con 0x7f57c80c8a00 2024-03-06T07:25:31.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 -- 172.21.15.17:0/2996848291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c80c8a00 msgr2=0x7f57c8158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c80c8a00 0x7f57c8158f20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f57b80097a0 tx=0x7f57b802b840 comp rx=0 tx=0).stop 2024-03-06T07:25:31.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 -- 172.21.15.17:0/2996848291 shutdown_connections 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c8159460 0x7f57c80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 --2- 172.21.15.17:0/2996848291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c80c8a00 0x7f57c8158f20 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 -- 172.21.15.17:0/2996848291 >> 172.21.15.17:0/2996848291 conn(0x7f57c80ac560 msgr2=0x7f57c80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 -- 172.21.15.17:0/2996848291 shutdown_connections 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.155+0000 7f57cff25640 1 -- 172.21.15.17:0/2996848291 wait complete. 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 Processor -- start 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 -- start start 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57c80c8a00 0x7f57c80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c814e8b0 0x7f57c814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80c7390 con 0x7f57c8152920 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c806d030 con 0x7f57c80c8a00 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cff25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80c7510 con 0x7f57c814e8b0 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cd4c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cd4c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57354/0 (socket says 172.21.15.17:57354) 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cd4c0640 1 -- 172.21.15.17:0/1105857811 learned_addr learned my addr 172.21.15.17:0/1105857811 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57ce4c2640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c814e8b0 0x7f57c814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.157+0000 7f57cdcc1640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57c80c8a00 0x7f57c80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 -- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c814e8b0 msgr2=0x7f57c814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c814e8b0 0x7f57c814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 -- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57c80c8a00 msgr2=0x7f57c80ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57c80c8a00 0x7f57c80ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 -- 172.21.15.17:0/1105857811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80c3860 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cd4c0640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f57b80098c0 tx=0x7f57b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8004230 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b80043d0 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c80c3b40 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8003850 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.158+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c80c4100 con 0x7f57c8152920 2024-03-06T07:25:31.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.159+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c804f330 con 0x7f57c8152920 2024-03-06T07:25:31.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.159+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57b80039f0 con 0x7f57c8152920 2024-03-06T07:25:31.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.160+0000 7f57be7fc640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 0x7f57ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.160+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f57b802f030 con 0x7f57c8152920 2024-03-06T07:25:31.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.160+0000 7f57cdcc1640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 0x7f57ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.160+0000 7f57cdcc1640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 0x7f57ac06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f60 tx=0x7f57c4007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.164+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57b8041070 con 0x7f57c8152920 2024-03-06T07:25:31.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.382+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c8156390 con 0x7f57c8152920 2024-03-06T07:25:31.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.382+0000 7f57be7fc640 1 -- 172.21.15.17:0/1105857811 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4992 (secure 0 0 0) 0x7f57b807aad0 con 0x7f57c8152920 2024-03-06T07:25:31.385 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:31.385 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":125,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":8265,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":5,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":121},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:21.987141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7783":{"gid":7783,"name":"b","rank":0,"incarnation":122,"state":"up:active","state_seq":8,"addr":"172.21.15.163:6836/3255496602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3255496602},{"type":"v1","addr":"172.21.15.163:6836","nonce":3255496602}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7783,"qdb_cluster":[7783]},"id":10}]} 2024-03-06T07:25:31.385 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 125 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 msgr2=0x7f57ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 0x7f57ac06d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57c4000f60 tx=0x7f57c4007d50 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 msgr2=0x7f57c814ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f57b80098c0 tx=0x7f57b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 shutdown_connections 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57ac06b4d0 0x7f57ac06d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57c814e8b0 0x7f57c814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57c8152920 0x7f57c814ccc0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 --2- 172.21.15.17:0/1105857811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57c80c8a00 0x7f57c80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 >> 172.21.15.17:0/1105857811 conn(0x7f57c80ac560 msgr2=0x7f57c814aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 shutdown_connections 2024-03-06T07:25:31.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.384+0000 7f57cff25640 1 -- 172.21.15.17:0/1105857811 wait complete. 2024-03-06T07:25:31.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:21.987141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 137, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7783}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7783': {'gid': 7783, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7783, 'qdb_cluster': [7783]} 2024-03-06T07:25:31.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:31.800 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.905+0000 7f6c5e640640 1 Processor -- start 2024-03-06T07:25:31.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.905+0000 7f6c5e640640 1 -- start start 2024-03-06T07:25:31.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5e640640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c5806e0a0 0x7f6c58153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5e640640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c58070550 con 0x7f6c58072f00 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5e640640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c58070830 con 0x7f6c580c0fa0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5e640640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c58070b10 con 0x7f6c5806e0a0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5cbdd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c5806e0a0 0x7f6c58153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c57fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6c58072f00 0x7f6c580c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50816/0 (socket says 172.21.15.17:50816) 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c57fff640 1 -- 172.21.15.17:0/2327400023 learned_addr learned my addr 172.21.15.17:0/2327400023 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/2327400023 >> v1:172.21.15.163:6789/0 conn(0x7f6c580c0fa0 legacy=0x7f6c5806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/2327400023 >> v1:172.21.15.17:6789/0 conn(0x7f6c58072f00 legacy=0x7f6c580c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c56ffd640 1 -- 172.21.15.17:0/2327400023 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302502527 0 0) 0x7f6c58070550 con 0x7f6c58072f00 2024-03-06T07:25:31.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.906+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/2327400023 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580732e0 con 0x7f6c5806e0a0 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/2327400023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c5806e0a0 0x7f6c58153990 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009d90 tx=0x7f6c5002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cc363ff8cf830 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c56ffd640 1 -- 172.21.15.17:0/2327400023 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50039070 con 0x7f6c5806e0a0 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c56ffd640 1 -- 172.21.15.17:0/2327400023 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5002cae0 con 0x7f6c5806e0a0 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c56ffd640 1 -- 172.21.15.17:0/2327400023 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50034440 con 0x7f6c5806e0a0 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c5e640640 1 -- 172.21.15.17:0/2327400023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c5806e0a0 msgr2=0x7f6c58153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.907+0000 7f6c5e640640 1 --2- 172.21.15.17:0/2327400023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c5806e0a0 0x7f6c58153990 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009d90 tx=0x7f6c5002bdb0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.908+0000 7f6c5e640640 1 -- 172.21.15.17:0/2327400023 shutdown_connections 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.908+0000 7f6c5e640640 1 --2- 172.21.15.17:0/2327400023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c5806e0a0 0x7f6c58153990 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.908+0000 7f6c5e640640 1 -- 172.21.15.17:0/2327400023 >> 172.21.15.17:0/2327400023 conn(0x7f6c5806b1a0 msgr2=0x7f6c5806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.908+0000 7f6c5e640640 1 -- 172.21.15.17:0/2327400023 shutdown_connections 2024-03-06T07:25:31.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.908+0000 7f6c5e640640 1 -- 172.21.15.17:0/2327400023 wait complete. 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 Processor -- start 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 -- start start 2024-03-06T07:25:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c5806e0a0 0x7f6c581dc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c58072f00 0x7f6c581dd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58070f70 con 0x7f6c580c0fa0 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58070df0 con 0x7f6c5806e0a0 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.909+0000 7f6c5e640640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58072400 con 0x7f6c58072f00 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57368/0 (socket says 172.21.15.17:57368) 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1502357071 learned_addr learned my addr 172.21.15.17:0/1502357071 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c5806e0a0 0x7f6c581dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c577fe640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c58072f00 0x7f6c581dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c58072f00 msgr2=0x7f6c581dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c58072f00 0x7f6c581dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c5806e0a0 msgr2=0x7f6c581dc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c5806e0a0 0x7f6c581dc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1502357071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c581dce30 con 0x7f6c580c0fa0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009d90 tx=0x7f6c50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c54ff9640 1 -- 172.21.15.17:0/1502357071 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50044030 con 0x7f6c580c0fa0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c54ff9640 1 -- 172.21.15.17:0/1502357071 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5002c8e0 con 0x7f6c580c0fa0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c581dcb50 con 0x7f6c580c0fa0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c54ff9640 1 -- 172.21.15.17:0/1502357071 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50039040 con 0x7f6c580c0fa0 2024-03-06T07:25:31.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.910+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581df2e0 con 0x7f6c580c0fa0 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.912+0000 7f6c54ff9640 1 -- 172.21.15.17:0/1502357071 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c50047070 con 0x7f6c580c0fa0 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.912+0000 7f6c54ff9640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.912+0000 7f6c54ff9640 1 -- 172.21.15.17:0/1502357071 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c500acb10 con 0x7f6c580c0fa0 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.913+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.913+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 0x7f6c4406d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007f70 tx=0x7f6c48007f00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.913+0000 7f6c61221740 1 Processor -- start 2024-03-06T07:25:31.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.913+0000 7f6c61221740 1 -- start start 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad847800 0x5603ad7fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad838420 0x5603ad7f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad7626b0 con 0x5603ad7fe550 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad70ba60 con 0x5603ad838420 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603ad75e750 con 0x5603ad847800 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57376/0 (socket says 172.21.15.17:57376) 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 -- 172.21.15.17:0/2337736900 learned_addr learned my addr 172.21.15.17:0/2337736900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c57fff640 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad847800 0x5603ad7fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad838420 0x5603ad7f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 -- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad847800 msgr2=0x5603ad7fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad847800 0x5603ad7fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 -- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad838420 msgr2=0x5603ad7f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad838420 0x5603ad7f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.914+0000 7f6c577fe640 1 -- 172.21.15.17:0/2337736900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ad729f30 con 0x5603ad7fe550 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c577fe640 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001970 tx=0x7f6c4c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a62f2bec2fdde server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c3dffb640 1 -- 172.21.15.17:0/2337736900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c0052e0 con 0x5603ad7fe550 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c3dffb640 1 -- 172.21.15.17:0/2337736900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c018070 con 0x5603ad7fe550 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c3dffb640 1 -- 172.21.15.17:0/2337736900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00fb30 con 0x5603ad7fe550 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 -- 172.21.15.17:0/2337736900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 msgr2=0x5603ad847be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c001970 tx=0x7f6c4c007db0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 -- 172.21.15.17:0/2337736900 shutdown_connections 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad838420 0x5603ad7f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad7fe550 0x5603ad847be0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 --2- 172.21.15.17:0/2337736900 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad847800 0x5603ad7fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 -- 172.21.15.17:0/2337736900 >> 172.21.15.17:0/2337736900 conn(0x5603ad7fa7d0 msgr2=0x5603ad81f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 -- 172.21.15.17:0/2337736900 shutdown_connections 2024-03-06T07:25:31.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.915+0000 7f6c61221740 1 -- 172.21.15.17:0/2337736900 wait complete. 2024-03-06T07:25:31.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.916+0000 7f6c61221740 1 -- start start 2024-03-06T07:25:31.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.916+0000 7f6c61221740 1 Processor -- start 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad81c490 0x5603ad81bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad83c590 con 0x5603ad838420 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad71eea0 con 0x5603ad847800 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603ad81fdb0 con 0x5603ad81c490 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c61221740 10 client.0 _ll_register_callbacks cb 0x5603ad71e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57384/0 (socket says 172.21.15.17:57384) 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 -- 172.21.15.17:0/4169619153 learned_addr learned my addr 172.21.15.17:0/4169619153 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad81c490 0x5603ad81bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c577fe640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 -- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad81c490 msgr2=0x5603ad81bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad81c490 0x5603ad81bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 -- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 msgr2=0x5603ad81b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 -- 172.21.15.17:0/4169619153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603ad71f270 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c57fff640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6c480136d0 tx=0x7f6c48007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fc1b0ab0ba42b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.917+0000 7f6c36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48020640 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c4801ba20 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c4c018070 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c4801bc40 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 10 client.8583 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x5603ad4772d0 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c018070 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6c480207e0 con 0x5603ad838420 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 10 client.8583 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:31.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 10 client.8583 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 20 client.8583 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad7fd170 0x5603ad85d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 4 client.8583 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603ad76fbf0 con 0x5603ad7fd170 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.918+0000 7f6c577fe640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad7fd170 0x5603ad85d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c577fe640 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad7fd170 0x5603ad85d6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d68f662569afde server_cookie=62245810f793d4d4 in_seq=0 out_seq=0 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c36ffd640 10 client.8583 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c36ffd640 1 -- 172.21.15.17:0/4169619153 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5603ad76fbf0 con 0x5603ad7fd170 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c36ffd640 10 client.8583 handle_command_reply: tid=0 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 client.8583 shutdown 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 msgr2=0x5603ad83a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f6c480136d0 tx=0x7f6c48007d10 comp rx=0 tx=0).stop 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 shutdown_connections 2024-03-06T07:25:31.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad7fd170 0x5603ad85d6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad81c490 0x5603ad81bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c61221740 1 --2- 172.21.15.17:0/4169619153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad838420 0x5603ad83a3a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.919+0000 7f6c36ffd640 0 client.8583 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.920+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 >> 172.21.15.17:0/4169619153 conn(0x5603ad7fe550 msgr2=0x5603ad7f8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.920+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 shutdown_connections 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.920+0000 7f6c61221740 1 -- 172.21.15.17:0/4169619153 wait complete. 2024-03-06T07:25:31.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.920+0000 7f6c61221740 20 client.8583 trim_cache size 0 max 16384 2024-03-06T07:25:31.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 Processor -- start 2024-03-06T07:25:31.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 -- start start 2024-03-06T07:25:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580ab080 con 0x5603ad81c490 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5814a3d0 con 0x5603ad847800 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.939+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580aaa80 con 0x5603ad838420 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57392/0 (socket says 172.21.15.17:57392) 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 -- 172.21.15.17:0/1000475441 learned_addr learned my addr 172.21.15.17:0/1000475441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c577fe640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 -- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 msgr2=0x5603ad7f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 -- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 msgr2=0x5603ad81b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 -- 172.21.15.17:0/1000475441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603ad5416b0 con 0x5603ad81c490 2024-03-06T07:25:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c577fe640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007ad0 tx=0x7f6c48021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12320d72df5a108f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c3d7fa640 1 -- 172.21.15.17:0/1000475441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48039070 con 0x5603ad81c490 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c3d7fa640 1 -- 172.21.15.17:0/1000475441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48026070 con 0x5603ad81c490 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.940+0000 7f6c3d7fa640 1 -- 172.21.15.17:0/1000475441 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48020640 con 0x5603ad81c490 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- 172.21.15.17:0/1000475441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 msgr2=0x5603ad8628d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007ad0 tx=0x7f6c48021000 comp rx=0 tx=0).stop 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- 172.21.15.17:0/1000475441 shutdown_connections 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad847800 0x5603ad81b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 --2- 172.21.15.17:0/1000475441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad81c490 0x5603ad8628d0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- 172.21.15.17:0/1000475441 >> 172.21.15.17:0/1000475441 conn(0x5603ad7fe550 msgr2=0x5603ad7f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- 172.21.15.17:0/1000475441 shutdown_connections 2024-03-06T07:25:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- 172.21.15.17:0/1000475441 wait complete. 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.941+0000 7f6c61221740 1 -- start start 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 Processor -- start 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad81c490 0x5603ad830470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad7fcff0 con 0x5603ad847800 2024-03-06T07:25:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603ad846a40 con 0x5603ad81c490 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603ad846fc0 con 0x5603ad838420 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c61221740 10 client.0 _ll_register_callbacks cb 0x7f6c48004e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c5cbdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c5cbdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57406/0 (socket says 172.21.15.17:57406) 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1296410714 learned_addr learned my addr 172.21.15.17:0/1296410714 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.942+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad81c490 0x5603ad830470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 msgr2=0x5603ad7f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad81c490 msgr2=0x5603ad830470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad81c490 0x5603ad830470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 -- 172.21.15.17:0/1296410714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603ad81b5a0 con 0x5603ad847800 2024-03-06T07:25:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c5cbdd640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6c500aadf0 tx=0x7f6c5007f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b6ae4555fc4a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50097060 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c500833e0 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c48026070 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c500835e0 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 10 client.8589 fetch_fsmap learned FSMap version 125 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x5603ad8aeb20 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48026070 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 125) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f6c5007a660 con 0x5603ad847800 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 10 client.8589 fetch_fsmap finished waiting for FSMap version 125 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 10 client.8589 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 20 client.8589 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad8b1a50 0x5603ad8b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 4 client.8589 mds_command: new command op to 7783 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.943+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 --> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603ad769f40 con 0x5603ad8b1a50 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad8b1a50 0x5603ad8b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad8b1a50 0x5603ad8b1e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d3fc99cf89cf438 server_cookie=2c145912a776e75c in_seq=0 out_seq=0 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c36ffd640 10 client.8589 ms_handle_connect on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c36ffd640 1 -- 172.21.15.17:0/1296410714 <== mds.0 v2:172.21.15.163:6834/3255496602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5603ad769f40 con 0x5603ad8b1a50 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c36ffd640 10 client.8589 handle_command_reply: tid=0 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 client.8589 shutdown 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 msgr2=0x5603ad8ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f6c500aadf0 tx=0x7f6c5007f370 comp rx=0 tx=0).stop 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 shutdown_connections 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:6834/3255496602,v1:172.21.15.163:6836/3255496602] conn(0x5603ad8b1a50 0x5603ad8b1e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5603ad847800 0x5603ad8ae4e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5603ad838420 0x5603ad7f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 --2- 172.21.15.17:0/1296410714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5603ad81c490 0x5603ad830470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c36ffd640 0 client.8589 ms_handle_reset on v2:172.21.15.163:6834/3255496602 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 >> 172.21.15.17:0/1296410714 conn(0x5603ad7fe550 msgr2=0x5603ad7f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 shutdown_connections 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.944+0000 7f6c61221740 1 -- 172.21.15.17:0/1296410714 wait complete. 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c61221740 20 client.8589 trim_cache size 0 max 16384 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 7783, 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:31.953 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 12.965476015, 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 125, 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 137, 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 137, 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 29.096890048999999 2024-03-06T07:25:31.954 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 msgr2=0x7f6c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 0x7f6c4406d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007f70 tx=0x7f6c48007f00 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 msgr2=0x7f6c581e8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009d90 tx=0x7f6c50004a40 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 shutdown_connections 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4406b4d0 0x7f6c4406d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c580c0fa0 0x7f6c581e8920 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c58072f00 0x7f6c581dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 --2- 172.21.15.17:0/1502357071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c5806e0a0 0x7f6c581dc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 >> 172.21.15.17:0/1502357071 conn(0x7f6c5806b1a0 msgr2=0x7f6c58153850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 shutdown_connections 2024-03-06T07:25:31.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:31.945+0000 7f6c5e640640 1 -- 172.21.15.17:0/1502357071 wait complete. 2024-03-06T07:25:31.958 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 125, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:07.286023+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 125, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7783": { "addr": "172.21.15.163:6836/3255496602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 3255496602, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 3255496602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7783, "incarnation": 122, "join_fscid": 10, "name": "b", "rank": 0, "state": "up:active", "state_seq": 8 } }, "last_failure": 0, "last_failure_osd_epoch": 137, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:21.987141+0000", "qdb_cluster": [ 7783 ], "qdb_leader": 7783, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7783 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/3725508931", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3725508931, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3725508931, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 123, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6926, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6834/824017697", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 824017697, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 824017697, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 121, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8265, "incarnation": 0, "join_fscid": 10, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 5 }, { "addr": "172.21.15.17:6835/1732861093", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1732861093, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1732861093, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:31.959 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 10, 'name': 'c', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:31.959 INFO:tasks.cephfs.test_failover:current = [{'gid': 8268, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1732861093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1732861093}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1732861093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 7783, 'name': 'b', 'rank': 0, 'incarnation': 122, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.163:6836/3255496602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3255496602}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 3255496602}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8265, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 5, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 121}, {'gid': 6926, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/3725508931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3725508931}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3725508931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 123}] 2024-03-06T07:25:31.960 DEBUG:tasks.cephfs.test_failover:-1 != 10 2024-03-06T07:25:31.960 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:25:33.054 INFO:tasks.ceph.mds.b.smithi163.stderr: -447> 2024-03-06T07:25:23.213+0000 7f4b43b40640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:33.078 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:25:36.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:37.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.060+0000 7f037d763640 1 Processor -- start 2024-03-06T07:25:37.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.060+0000 7f037d763640 1 -- start start 2024-03-06T07:25:37.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f037d763640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03780724b0 0x7f0378159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f037d763640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03780a7e50 con 0x7f03780c3940 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f037d763640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03780a8130 con 0x7f03780c67f0 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f037d763640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780a8410 con 0x7f03780724b0 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f0376ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03780724b0 0x7f0378159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f03767fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f03780c3940 0x7f0378155090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50830/0 (socket says 172.21.15.17:50830) 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f0376ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03780724b0 0x7f0378159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57412/0 (socket says 172.21.15.17:57412) 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.061+0000 7f03767fc640 1 -- 172.21.15.17:0/268759603 learned_addr learned my addr 172.21.15.17:0/268759603 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0376ffd640 1 -- 172.21.15.17:0/268759603 >> v1:172.21.15.163:6789/0 conn(0x7f03780c67f0 legacy=0x7f037815fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0376ffd640 1 -- 172.21.15.17:0/268759603 >> v1:172.21.15.17:6789/0 conn(0x7f03780c3940 legacy=0x7f0378155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0376ffd640 1 -- 172.21.15.17:0/268759603 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780c3420 con 0x7f03780724b0 2024-03-06T07:25:37.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0376ffd640 1 --2- 172.21.15.17:0/268759603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780724b0 0x7f0378159090 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f03780a8590 tx=0x7f036802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a265572accee8b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0375ffb640 1 -- 172.21.15.17:0/268759603 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368038070 con 0x7f03780724b0 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0375ffb640 1 -- 172.21.15.17:0/268759603 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0368031700 con 0x7f03780724b0 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.062+0000 7f0375ffb640 1 -- 172.21.15.17:0/268759603 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368031ad0 con 0x7f03780724b0 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 -- 172.21.15.17:0/268759603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780724b0 msgr2=0x7f0378159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 --2- 172.21.15.17:0/268759603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780724b0 0x7f0378159090 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f03780a8590 tx=0x7f036802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 -- 172.21.15.17:0/268759603 shutdown_connections 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 --2- 172.21.15.17:0/268759603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780724b0 0x7f0378159090 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 -- 172.21.15.17:0/268759603 >> 172.21.15.17:0/268759603 conn(0x7f03780aa6a0 msgr2=0x7f03780aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 -- 172.21.15.17:0/268759603 shutdown_connections 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.063+0000 7f037d763640 1 -- 172.21.15.17:0/268759603 wait complete. 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.064+0000 7f037d763640 1 Processor -- start 2024-03-06T07:25:37.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.064+0000 7f037d763640 1 -- start start 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03780724b0 0x7f0378150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03780c3940 0x7f03781eac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0378152ae0 con 0x7f03780c67f0 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0378152960 con 0x7f03780724b0 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f037d763640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03781557a0 con 0x7f03780c3940 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57416/0 (socket says 172.21.15.17:57416) 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 -- 172.21.15.17:0/1946937840 learned_addr learned my addr 172.21.15.17:0/1946937840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f0376ffd640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03780724b0 0x7f0378150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03767fc640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03780c3940 0x7f03781eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 -- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03780c3940 msgr2=0x7f03781eac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03780c3940 0x7f03781eac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 -- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03780724b0 msgr2=0x7f0378150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03780724b0 0x7f0378150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.065+0000 7f03777fe640 1 -- 172.21.15.17:0/1946937840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03781555b0 con 0x7f03780c67f0 2024-03-06T07:25:37.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.066+0000 7f03777fe640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f037000b970 tx=0x7f037000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.068+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370019070 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.068+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037000cb40 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.068+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0378153e40 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.068+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370014440 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.068+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781540e0 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.069+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037804f330 con 0x7f03780c67f0 2024-03-06T07:25:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.069+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f037001c070 con 0x7f03780c67f0 2024-03-06T07:25:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.070+0000 7f035f7fe640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 0x7f036406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.070+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f037008a0e0 con 0x7f03780c67f0 2024-03-06T07:25:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.070+0000 7f0376ffd640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 0x7f036406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.070+0000 7f0376ffd640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 0x7f036406d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0368035000 tx=0x7f0368008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.074+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0370058640 con 0x7f03780c67f0 2024-03-06T07:25:37.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.291+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0378155e50 con 0x7f03780c67f0 2024-03-06T07:25:37.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.292+0000 7f035f7fe640 1 -- 172.21.15.17:0/1946937840 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f037005c6e0 con 0x7f03780c67f0 2024-03-06T07:25:37.294 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:37.294 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":130,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:36.061572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8265":{"gid":8265,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8265,"qdb_cluster":[8265]},"id":10}]} 2024-03-06T07:25:37.295 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 130 2024-03-06T07:25:37.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.293+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 msgr2=0x7f036406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.293+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 0x7f036406d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0368035000 tx=0x7f0368008030 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.293+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 msgr2=0x7f03781eb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.293+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f037000b970 tx=0x7f037000be30 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 shutdown_connections 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f036406b500 0x7f036406d9c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03780c67f0 0x7f03781eb150 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03780c3940 0x7f03781eac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 --2- 172.21.15.17:0/1946937840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03780724b0 0x7f0378150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 >> 172.21.15.17:0/1946937840 conn(0x7f03780aa6a0 msgr2=0x7f03780c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 shutdown_connections 2024-03-06T07:25:37.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.294+0000 7f037d763640 1 -- 172.21.15.17:0/1946937840 wait complete. 2024-03-06T07:25:37.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:07.286023+0000', 'modified': '2024-03-06T07:25:36.061572+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 138, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8265}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8265': {'gid': 8265, 'name': 'c', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8265, 'qdb_cluster': [8265]} 2024-03-06T07:25:37.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:37.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:25:37.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.810+0000 7fea72c8f640 1 Processor -- start 2024-03-06T07:25:37.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.810+0000 7fea72c8f640 1 -- start start 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea72c8f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea6c0c69f0 0x7fea6c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea72c8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea6c0c8950 0x7fea6c0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea72c8f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6c155130 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea72c8f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c155440 con 0x7fea6c0c69f0 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea72c8f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6c0c5d50 con 0x7fea6c0c8950 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea70a2b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea6c0c69f0 0x7fea6c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea7122c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fea6c0ad2b0 0x7fea6c1558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50832/0 (socket says 172.21.15.17:50832) 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea7122c640 1 -- 172.21.15.17:0/886309748 learned_addr learned my addr 172.21.15.17:0/886309748 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea63fff640 1 --2- 172.21.15.17:0/886309748 >> v2:172.21.15.163:3300/0 conn(0x7fea6c0c8950 0x7fea6c0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea70a2b640 1 -- 172.21.15.17:0/886309748 >> v2:172.21.15.163:3300/0 conn(0x7fea6c0c8950 msgr2=0x7fea6c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea70a2b640 1 --2- 172.21.15.17:0/886309748 >> v2:172.21.15.163:3300/0 conn(0x7fea6c0c8950 0x7fea6c0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea70a2b640 1 -- 172.21.15.17:0/886309748 >> v1:172.21.15.17:6789/0 conn(0x7fea6c0ad2b0 legacy=0x7fea6c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea70a2b640 1 -- 172.21.15.17:0/886309748 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c0c6030 con 0x7fea6c0c69f0 2024-03-06T07:25:37.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.811+0000 7fea637fe640 1 -- 172.21.15.17:0/886309748 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354415253 0 0) 0x7fea64001490 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.812+0000 7fea70a2b640 1 --2- 172.21.15.17:0/886309748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0c69f0 0x7fea6c0c6dd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0c7430 tx=0x7fea6802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3763e4d421d250 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.812+0000 7fea637fe640 1 -- 172.21.15.17:0/886309748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea680315a0 con 0x7fea6c0c69f0 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.812+0000 7fea637fe640 1 -- 172.21.15.17:0/886309748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68037070 con 0x7fea6c0c69f0 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.812+0000 7fea637fe640 1 -- 172.21.15.17:0/886309748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68031960 con 0x7fea6c0c69f0 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 -- 172.21.15.17:0/886309748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0c69f0 msgr2=0x7fea6c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 --2- 172.21.15.17:0/886309748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0c69f0 0x7fea6c0c6dd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0c7430 tx=0x7fea6802b840 comp rx=0 tx=0).stop 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 -- 172.21.15.17:0/886309748 shutdown_connections 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 --2- 172.21.15.17:0/886309748 >> v2:172.21.15.163:3300/0 conn(0x7fea6c0c8950 0x7fea6c0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 --2- 172.21.15.17:0/886309748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0c69f0 0x7fea6c0c6dd0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 -- 172.21.15.17:0/886309748 >> 172.21.15.17:0/886309748 conn(0x7fea6c0cae70 msgr2=0x7fea6c0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 -- 172.21.15.17:0/886309748 shutdown_connections 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.813+0000 7fea72c8f640 1 -- 172.21.15.17:0/886309748 wait complete. 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.814+0000 7fea72c8f640 1 Processor -- start 2024-03-06T07:25:37.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.814+0000 7fea72c8f640 1 -- start start 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea6c0c69f0 0x7fea6c1ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea6c0c8950 0x7fea6c1eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c152a20 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c155fe0 con 0x7fea6c0c69f0 2024-03-06T07:25:37.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea72c8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c152ba0 con 0x7fea6c0c8950 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea70a2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea70a2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57432/0 (socket says 172.21.15.17:57432) 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.815+0000 7fea70a2b640 1 -- 172.21.15.17:0/265566851 learned_addr learned my addr 172.21.15.17:0/265566851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 -- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea6c0c8950 msgr2=0x7fea6c1eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea6c0c8950 0x7fea6c1eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 -- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea6c0c69f0 msgr2=0x7fea6c1ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea6c0c69f0 0x7fea6c1ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 -- 172.21.15.17:0/265566851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c14d360 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea70a2b640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fea68000c00 tx=0x7fea680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea617fa640 1 -- 172.21.15.17:0/265566851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68004230 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea617fa640 1 -- 172.21.15.17:0/265566851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea680043d0 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1ebf70 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea617fa640 1 -- 172.21.15.17:0/265566851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68003850 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.816+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c1eb490 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.818+0000 7fea617fa640 1 -- 172.21.15.17:0/265566851 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea680039f0 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.818+0000 7fea617fa640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.818+0000 7fea617fa640 1 -- 172.21.15.17:0/265566851 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fea6802f030 con 0x7fea6c0ad2b0 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.819+0000 7fea63fff640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.819+0000 7fea75872740 1 Processor -- start 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.819+0000 7fea63fff640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fea6c06d180 tx=0x7fea5c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.819+0000 7fea75872740 1 -- start start 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a6ff50 0x557492a6da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a6df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574929d46b0 con 0x557492a6e4b0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749297da60 con 0x557492a70330 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574929d0750 con 0x557492a6ff50 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57434/0 (socket says 172.21.15.17:57434) 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 -- 172.21.15.17:0/3061455589 learned_addr learned my addr 172.21.15.17:0/3061455589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea63fff640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a6df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea70a2b640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a6ff50 0x557492a6da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 -- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a6ff50 msgr2=0x557492a6da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a6ff50 0x557492a6da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 -- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 msgr2=0x557492a6df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a6df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 -- 172.21.15.17:0/3061455589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55749299bf30 con 0x557492a6e4b0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea63fff640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.820+0000 7fea7122c640 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fea64001ca0 tx=0x7fea64002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68de1c95f902931 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea527fc640 1 -- 172.21.15.17:0/3061455589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6400c840 con 0x557492a6e4b0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea527fc640 1 -- 172.21.15.17:0/3061455589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6400ca00 con 0x557492a6e4b0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea527fc640 1 -- 172.21.15.17:0/3061455589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6400ce30 con 0x557492a6e4b0 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 -- 172.21.15.17:0/3061455589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 msgr2=0x557492a6a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fea64001ca0 tx=0x7fea64002160 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 -- 172.21.15.17:0/3061455589 shutdown_connections 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6e4b0 0x557492a6a680 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a6df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 --2- 172.21.15.17:0/3061455589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a6ff50 0x557492a6da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 -- 172.21.15.17:0/3061455589 >> 172.21.15.17:0/3061455589 conn(0x557492a6c7d0 msgr2=0x557492a913c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 -- 172.21.15.17:0/3061455589 shutdown_connections 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.821+0000 7fea75872740 1 -- 172.21.15.17:0/3061455589 wait complete. 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.822+0000 7fea75872740 1 -- start start 2024-03-06T07:25:37.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.822+0000 7fea75872740 1 Processor -- start 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a97960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a92970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557492a92230 con 0x557492a70330 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557492990ea0 con 0x557492a6ff50 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557492a91c70 con 0x557492a93970 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 10 client.0 _ll_register_callbacks cb 0x557492990870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57440/0 (socket says 172.21.15.17:57440) 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 -- 172.21.15.17:0/383322157 learned_addr learned my addr 172.21.15.17:0/383322157 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea7122c640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a92970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea70a2b640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a97960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 -- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 msgr2=0x557492a92970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a92970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 -- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 msgr2=0x557492a97960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a97960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 -- 172.21.15.17:0/383322157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557492991270 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea63fff640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00ddf0 tx=0x7fea5c006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b692cee7e66cf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c020420 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea6400ca00 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.823+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fea5c00f040 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5c005990 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 10 client.8613 fetch_fsmap learned FSMap version 130 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x5574926e92d0 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6400ca00 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 130) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fea5c030080 con 0x557492a70330 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 10 client.8613 fetch_fsmap finished waiting for FSMap version 130 2024-03-06T07:25:37.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 10 client.8613 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 20 client.8613 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492ac04e0 0x557492acb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 4 client.8613 mds_command: new command op to 8265 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 --> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574929e1bf0 con 0x557492ac04e0 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea70a2b640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492ac04e0 0x557492acb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea4b7fe640 10 client.8613 ms_handle_connect on v2:172.21.15.17:6832/824017697 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.824+0000 7fea70a2b640 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492ac04e0 0x557492acb2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf2d2ea40206da3 server_cookie=2b1facc4ac912dd5 in_seq=0 out_seq=0 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.825+0000 7fea4b7fe640 1 -- 172.21.15.17:0/383322157 <== mds.0 v2:172.21.15.17:6832/824017697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5574929e1bf0 con 0x557492ac04e0 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.825+0000 7fea4b7fe640 10 client.8613 handle_command_reply: tid=0 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.825+0000 7fea75872740 1 client.8613 shutdown 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 msgr2=0x557492a96620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fea5c00ddf0 tx=0x7fea5c006e10 comp rx=0 tx=0).stop 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 shutdown_connections 2024-03-06T07:25:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492ac04e0 0x557492acb2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a92970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea4b7fe640 0 client.8613 ms_handle_reset on v2:172.21.15.17:6832/824017697 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a70330 0x557492a96620 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 --2- 172.21.15.17:0/383322157 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a97960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 >> 172.21.15.17:0/383322157 conn(0x557492a6e4b0 msgr2=0x557492a6e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 shutdown_connections 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 1 -- 172.21.15.17:0/383322157 wait complete. 2024-03-06T07:25:37.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.826+0000 7fea75872740 20 client.8613 trim_cache size 0 max 16384 2024-03-06T07:25:37.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 Processor -- start 2024-03-06T07:25:37.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 -- start start 2024-03-06T07:25:37.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a6d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1f8080 con 0x557492a6ff50 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1f71a0 con 0x557492a70330 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c156f40 con 0x557492a93970 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea70a2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea70a2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57446/0 (socket says 172.21.15.17:57446) 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.843+0000 7fea70a2b640 1 -- 172.21.15.17:0/3346272005 learned_addr learned my addr 172.21.15.17:0/3346272005 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 -- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 msgr2=0x557492a6d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea63fff640 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a6d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 -- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 msgr2=0x557492a70c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a70c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 -- 172.21.15.17:0/3346272005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574927b36b0 con 0x557492a6ff50 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea63fff640 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea70a2b640 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fea68009560 tx=0x7fea68009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc2fcc641bc575 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea51ffb640 1 -- 172.21.15.17:0/3346272005 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68097070 con 0x557492a6ff50 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea51ffb640 1 -- 172.21.15.17:0/3346272005 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68037040 con 0x557492a6ff50 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea51ffb640 1 -- 172.21.15.17:0/3346272005 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6807eae0 con 0x557492a6ff50 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 -- 172.21.15.17:0/3346272005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 msgr2=0x557492a90ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fea68009560 tx=0x7fea68009190 comp rx=0 tx=0).stop 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 -- 172.21.15.17:0/3346272005 shutdown_connections 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a93970 0x557492a6d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a70330 0x557492a70c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 --2- 172.21.15.17:0/3346272005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a6ff50 0x557492a90ff0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.844+0000 7fea75872740 1 -- 172.21.15.17:0/3346272005 >> 172.21.15.17:0/3346272005 conn(0x557492a6e4b0 msgr2=0x557492a65e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.845+0000 7fea75872740 1 -- 172.21.15.17:0/3346272005 shutdown_connections 2024-03-06T07:25:37.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.845+0000 7fea75872740 1 -- 172.21.15.17:0/3346272005 wait complete. 2024-03-06T07:25:37.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.845+0000 7fea75872740 1 -- start start 2024-03-06T07:25:37.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.845+0000 7fea75872740 1 Processor -- start 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a70330 0x557492a6d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557492a93470 con 0x557492a93970 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1f6c60 con 0x557492a6ff50 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557492abff90 con 0x557492a70330 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea75872740 10 client.0 _ll_register_callbacks cb 0x7fea5c00d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57452/0 (socket says 172.21.15.17:57452) 2024-03-06T07:25:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 -- 172.21.15.17:0/479809861 learned_addr learned my addr 172.21.15.17:0/479809861 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea63fff640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a70330 0x557492a6d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea70a2b640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 -- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a70330 msgr2=0x557492a6d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a70330 0x557492a6d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 -- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 msgr2=0x557492a913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a913f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 -- 172.21.15.17:0/479809861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557492aaf010 con 0x557492a93970 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea7122c640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c570 tx=0x7fea64001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea97bbcb751fa56 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.846+0000 7fea4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64019040 con 0x557492a93970 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fea64001750 con 0x557492a93970 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea68037040 con 0x557492a93970 2024-03-06T07:25:37.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea640070d0 con 0x557492a93970 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 10 client.8619 fetch_fsmap learned FSMap version 130 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x557492a8bac0 con 0x557492a93970 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68037040 con 0x557492a93970 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 130) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fea6401f040 con 0x557492a93970 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 10 client.8619 fetch_fsmap finished waiting for FSMap version 130 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 10 client.8619 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 20 client.8619 populate_metadata read hostname 'smithi017' 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492acf310 0x557492acf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 4 client.8619 mds_command: new command op to 8265 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 --> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574929dbf40 con 0x557492acf310 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.847+0000 7fea70a2b640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492acf310 0x557492acf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea4b7fe640 10 client.8619 ms_handle_connect on v2:172.21.15.17:6832/824017697 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea70a2b640 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492acf310 0x557492acf6f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c04d8c7b455709e server_cookie=47b2487ff4a3c524 in_seq=0 out_seq=0 2024-03-06T07:25:37.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea4b7fe640 1 -- 172.21.15.17:0/479809861 <== mds.0 v2:172.21.15.17:6832/824017697 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5574929dbf40 con 0x557492acf310 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea4b7fe640 10 client.8619 handle_command_reply: tid=0 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 client.8619 shutdown 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 msgr2=0x557492b18b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fea6400c570 tx=0x7fea64001250 comp rx=0 tx=0).stop 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 shutdown_connections 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:6832/824017697,v1:172.21.15.17:6834/824017697] conn(0x557492acf310 0x557492acf6f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557492a93970 0x557492b18b60 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557492a70330 0x557492a6d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 --2- 172.21.15.17:0/479809861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557492a6ff50 0x557492a913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 >> 172.21.15.17:0/479809861 conn(0x557492a6e4b0 msgr2=0x557492aaf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea4b7fe640 0 client.8619 ms_handle_reset on v2:172.21.15.17:6832/824017697 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 shutdown_connections 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 1 -- 172.21.15.17:0/479809861 wait complete. 2024-03-06T07:25:37.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.848+0000 7fea75872740 20 client.8619 trim_cache size 0 max 16384 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8265, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.8008259659999997, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 130, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 138, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 138, 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 35.011112101000002 2024-03-06T07:25:37.858 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:37.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 msgr2=0x7fea5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fea6c06d180 tx=0x7fea5c008040 comp rx=0 tx=0).stop 2024-03-06T07:25:37.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 msgr2=0x7fea6c06c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fea68000c00 tx=0x7fea680046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 shutdown_connections 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea5806b4d0 0x7fea5806d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea6c0c8950 0x7fea6c1eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea6c0c69f0 0x7fea6c1ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 --2- 172.21.15.17:0/265566851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea6c0ad2b0 0x7fea6c06c7c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 >> 172.21.15.17:0/265566851 conn(0x7fea6c0cae70 msgr2=0x7fea6c0bfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 shutdown_connections 2024-03-06T07:25:37.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:37.849+0000 7fea72c8f640 1 -- 172.21.15.17:0/265566851 wait complete. 2024-03-06T07:25:37.862 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 10, "epoch": 130, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 10, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:07.286023+0000", "damaged": [], "data_pools": [ 22 ], "enabled": true, "epoch": 130, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8265": { "addr": "172.21.15.17:6834/824017697", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 824017697, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 824017697, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8265, "incarnation": 127, "join_fscid": 10, "name": "c", "rank": 0, "state": "up:active", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 138, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 21, "modified": "2024-03-06T07:25:36.061572+0000", "qdb_cluster": [ 8265 ], "qdb_leader": 8265, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8265 } } } ], "standbys": [ { "addr": "172.21.15.163:6837/3725508931", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 3725508931, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 3725508931, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 123, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6926, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6836/862042831", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 862042831, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 862042831, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 128, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7004, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/1732861093", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1732861093, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1732861093, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 117, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8268, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:25:37.863 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': 10, 'name': 'c', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:25:37.863 INFO:tasks.cephfs.test_failover:current = [{'gid': 8268, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/1732861093', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1732861093}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1732861093}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 117}, {'gid': 7004, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/862042831', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 862042831}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 862042831}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 128}, {'gid': 8265, 'name': 'c', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.17:6834/824017697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 824017697}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 824017697}]}, 'join_fscid': 10, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 6926, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/3725508931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 3725508931}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 3725508931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 123}] 2024-03-06T07:25:37.863 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-06T07:25:37.863 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:37.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:37.912 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:37.962 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:37.962 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:25:37.962 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:37.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:37.977 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:25:37.977 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:37.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:25:37.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:38.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.118+0000 7f3dbff79640 1 Processor -- start 2024-03-06T07:25:38.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.118+0000 7f3dbff79640 1 -- start start 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbff79640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3db814f360 0x7f3db8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbff79640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3db8158b60 0x7f3db8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbff79640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80c73d0 con 0x7f3db80c88c0 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbff79640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80c8dd0 con 0x7f3db8158b60 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbff79640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80be2e0 con 0x7f3db814f360 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbe516640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3db8158b60 0x7f3db8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbe516640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3db8158b60 0x7f3db8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57462/0 (socket says 172.21.15.17:57462) 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbe516640 1 -- 172.21.15.17:0/2202574115 learned_addr learned my addr 172.21.15.17:0/2202574115 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbdd15640 1 --2- 172.21.15.17:0/2202574115 >> v2:172.21.15.163:3300/0 conn(0x7f3db814f360 0x7f3db8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbe516640 1 -- 172.21.15.17:0/2202574115 >> v2:172.21.15.163:3300/0 conn(0x7f3db814f360 msgr2=0x7f3db8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:38.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.119+0000 7f3dbe516640 1 --2- 172.21.15.17:0/2202574115 >> v2:172.21.15.163:3300/0 conn(0x7f3db814f360 0x7f3db8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbe516640 1 -- 172.21.15.17:0/2202574115 >> v1:172.21.15.163:6789/0 conn(0x7f3db80c88c0 legacy=0x7f3db80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbe516640 1 -- 172.21.15.17:0/2202574115 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80c7670 con 0x7f3db8158b60 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbe516640 1 --2- 172.21.15.17:0/2202574115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db8158b60 0x7f3db8158ff0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f3db808c840 tx=0x7f3db002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d9b840466507c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbcd13640 1 -- 172.21.15.17:0/2202574115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db002c9f0 con 0x7f3db8158b60 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbcd13640 1 -- 172.21.15.17:0/2202574115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db002cb90 con 0x7f3db8158b60 2024-03-06T07:25:38.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.120+0000 7f3dbcd13640 1 -- 172.21.15.17:0/2202574115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db00343f0 con 0x7f3db8158b60 2024-03-06T07:25:38.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 -- 172.21.15.17:0/2202574115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db8158b60 msgr2=0x7f3db8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 --2- 172.21.15.17:0/2202574115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db8158b60 0x7f3db8158ff0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f3db808c840 tx=0x7f3db002be50 comp rx=0 tx=0).stop 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 -- 172.21.15.17:0/2202574115 shutdown_connections 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 --2- 172.21.15.17:0/2202574115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db8158b60 0x7f3db8158ff0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 --2- 172.21.15.17:0/2202574115 >> v2:172.21.15.163:3300/0 conn(0x7f3db814f360 0x7f3db8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 -- 172.21.15.17:0/2202574115 >> 172.21.15.17:0/2202574115 conn(0x7f3db80ac570 msgr2=0x7f3db80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 -- 172.21.15.17:0/2202574115 shutdown_connections 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.121+0000 7f3dbff79640 1 -- 172.21.15.17:0/2202574115 wait complete. 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.122+0000 7f3dbff79640 1 Processor -- start 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.122+0000 7f3dbff79640 1 -- start start 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3db80c88c0 0x7f3db81e0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3db8158b60 0x7f3db81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c5a50 con 0x7f3db814f360 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c7c30 con 0x7f3db8158b60 2024-03-06T07:25:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbff79640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80c5bd0 con 0x7f3db80c88c0 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57466/0 (socket says 172.21.15.17:57466) 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 -- 172.21.15.17:0/3341912343 learned_addr learned my addr 172.21.15.17:0/3341912343 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbe516640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3db8158b60 0x7f3db81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbdd15640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3db80c88c0 0x7f3db81e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 -- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3db80c88c0 msgr2=0x7f3db81e0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3db80c88c0 0x7f3db81e0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 -- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3db8158b60 msgr2=0x7f3db81577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3db8158b60 0x7f3db81577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 -- 172.21.15.17:0/3341912343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80c14e0 con 0x7f3db814f360 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dbd514640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b3f0 tx=0x7f3da800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.123+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8015020 con 0x7f3db814f360 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.124+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8011830 con 0x7f3db814f360 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.124+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8011c20 con 0x7f3db814f360 2024-03-06T07:25:38.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.124+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80c1200 con 0x7f3db814f360 2024-03-06T07:25:38.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.124+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db81deb70 con 0x7f3db814f360 2024-03-06T07:25:38.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.124+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db804f330 con 0x7f3db814f360 2024-03-06T07:25:38.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.125+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3da8018020 con 0x7f3db814f360 2024-03-06T07:25:38.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.125+0000 7f3dae7fc640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 0x7f3d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.126+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f3da8089e80 con 0x7f3db814f360 2024-03-06T07:25:38.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.126+0000 7f3dbdd15640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 0x7f3d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.126+0000 7f3dbdd15640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 0x7f3d9c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.130+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3da801f030 con 0x7f3db814f360 2024-03-06T07:25:38.214 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:25:38.211+0000 7fc4a720e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.349+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3db8157d10 con 0x7f3db814f360 2024-03-06T07:25:38.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.350+0000 7f3dae7fc640 1 -- 172.21.15.17:0/3341912343 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f3da8058470 con 0x7f3db814f360 2024-03-06T07:25:38.352 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:38.352 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":130,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:36.061572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8265":{"gid":8265,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8265,"qdb_cluster":[8265]},"id":10}]} 2024-03-06T07:25:38.353 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 130 2024-03-06T07:25:38.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 msgr2=0x7f3d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 0x7f3d9c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 msgr2=0x7f3db81df370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3da800b3f0 tx=0x7f3da800b8b0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 shutdown_connections 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3d9c06b4d0 0x7f3d9c06d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3db8158b60 0x7f3db81577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3db814f360 0x7f3db81df370 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 --2- 172.21.15.17:0/3341912343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3db80c88c0 0x7f3db81e0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 >> 172.21.15.17:0/3341912343 conn(0x7f3db80ac570 msgr2=0x7f3db814dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 shutdown_connections 2024-03-06T07:25:38.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.352+0000 7f3dbff79640 1 -- 172.21.15.17:0/3341912343 wait complete. 2024-03-06T07:25:38.365 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.464+0000 7f9a3be34640 1 Processor -- start 2024-03-06T07:25:38.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.464+0000 7f9a3be34640 1 -- start start 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.465+0000 7f9a3be34640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a340c5480 0x7f9a34155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a3be34640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a341586f0 0x7f9a34158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a3be34640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a340c5cc0 con 0x7f9a340c5860 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a3be34640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3406df40 con 0x7f9a340c5480 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a3be34640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3406fdb0 con 0x7f9a341586f0 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a340c5480 0x7f9a34155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a340c5480 0x7f9a34155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57478/0 (socket says 172.21.15.17:57478) 2024-03-06T07:25:38.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 -- 172.21.15.17:0/548141496 learned_addr learned my addr 172.21.15.17:0/548141496 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a3a3d1640 1 --2- 172.21.15.17:0/548141496 >> v2:172.21.15.163:3300/0 conn(0x7f9a341586f0 0x7f9a34158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 -- 172.21.15.17:0/548141496 >> v2:172.21.15.163:3300/0 conn(0x7f9a341586f0 msgr2=0x7f9a34158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 --2- 172.21.15.17:0/548141496 >> v2:172.21.15.163:3300/0 conn(0x7f9a341586f0 0x7f9a34158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 -- 172.21.15.17:0/548141496 >> v1:172.21.15.163:6789/0 conn(0x7f9a340c5860 legacy=0x7f9a3406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.466+0000 7f9a39bd0640 1 -- 172.21.15.17:0/548141496 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a341591c0 con 0x7f9a340c5480 2024-03-06T07:25:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.467+0000 7f9a39bd0640 1 --2- 172.21.15.17:0/548141496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5480 0x7f9a34155ba0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9a340ab500 tx=0x7f9a2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ae5f566b58e7e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.467+0000 7f9a38bce640 1 -- 172.21.15.17:0/548141496 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c035020 con 0x7f9a340c5480 2024-03-06T07:25:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.467+0000 7f9a38bce640 1 -- 172.21.15.17:0/548141496 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c031870 con 0x7f9a340c5480 2024-03-06T07:25:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.467+0000 7f9a38bce640 1 -- 172.21.15.17:0/548141496 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c031c40 con 0x7f9a340c5480 2024-03-06T07:25:38.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 -- 172.21.15.17:0/548141496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5480 msgr2=0x7f9a34155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 --2- 172.21.15.17:0/548141496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5480 0x7f9a34155ba0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9a340ab500 tx=0x7f9a2c02b910 comp rx=0 tx=0).stop 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 -- 172.21.15.17:0/548141496 shutdown_connections 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 --2- 172.21.15.17:0/548141496 >> v2:172.21.15.163:3300/0 conn(0x7f9a341586f0 0x7f9a34158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 --2- 172.21.15.17:0/548141496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5480 0x7f9a34155ba0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 -- 172.21.15.17:0/548141496 >> 172.21.15.17:0/548141496 conn(0x7f9a3406b1a0 msgr2=0x7f9a3406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 -- 172.21.15.17:0/548141496 shutdown_connections 2024-03-06T07:25:38.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.468+0000 7f9a3be34640 1 -- 172.21.15.17:0/548141496 wait complete. 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.469+0000 7f9a3be34640 1 Processor -- start 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.469+0000 7f9a3be34640 1 -- start start 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a340c5480 0x7f9a3414b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a341586f0 0x7f9a34157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34155680 con 0x7f9a340c5860 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a341553a0 con 0x7f9a340c5480 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3be34640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34152a20 con 0x7f9a341586f0 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57490/0 (socket says 172.21.15.17:57490) 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 -- 172.21.15.17:0/2539049102 learned_addr learned my addr 172.21.15.17:0/2539049102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:38.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a39bd0640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a340c5480 0x7f9a3414b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a3a3d1640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a341586f0 0x7f9a34157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 -- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a341586f0 msgr2=0x7f9a34157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a341586f0 0x7f9a34157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 -- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a340c5480 msgr2=0x7f9a3414b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a340c5480 0x7f9a3414b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.470+0000 7f9a393cf640 1 -- 172.21.15.17:0/2539049102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a341eaec0 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a393cf640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c400 tx=0x7f9a2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a240052d0 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a24005490 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3414cd00 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a24007260 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.471+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a341eb460 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.472+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a340c2960 con 0x7f9a340c5860 2024-03-06T07:25:38.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.472+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a24007400 con 0x7f9a340c5860 2024-03-06T07:25:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.473+0000 7f9a2a7fc640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 0x7f9a1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.473+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f9a24089850 con 0x7f9a340c5860 2024-03-06T07:25:38.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.473+0000 7f9a39bd0640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 0x7f9a1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.473+0000 7f9a39bd0640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 0x7f9a1806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c036000 tx=0x7f9a2c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.477+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a340c2960 con 0x7f9a340c5860 2024-03-06T07:25:38.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.660+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a3404f330 con 0x7f9a340c5860 2024-03-06T07:25:38.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.661+0000 7f9a2a7fc640 1 -- 172.21.15.17:0/2539049102 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+17155 (secure 0 0 0) 0x7f9a24057ed0 con 0x7f9a340c5860 2024-03-06T07:25:38.664 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:38.664 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":138,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:33.047446+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6837/3327948285":"2024-03-07T07:25:18.978040+0000","172.21.15.163:6834/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6835/3327948285":"2024-03-07T07:25:18.978040+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:38.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 msgr2=0x7f9a1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 0x7f9a1806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c036000 tx=0x7f9a2c03c040 comp rx=0 tx=0).stop 2024-03-06T07:25:38.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 msgr2=0x7f9a341e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c400 tx=0x7f9a2400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 shutdown_connections 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9a1806b4d0 0x7f9a1806d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a341586f0 0x7f9a34157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a340c5860 0x7f9a341e06e0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 --2- 172.21.15.17:0/2539049102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a340c5480 0x7f9a3414b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 >> 172.21.15.17:0/2539049102 conn(0x7f9a3406b1a0 msgr2=0x7f9a340c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 shutdown_connections 2024-03-06T07:25:38.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.663+0000 7f9a3be34640 1 -- 172.21.15.17:0/2539049102 wait complete. 2024-03-06T07:25:38.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:25:05.553791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:25:06.251353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:38.677 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:38.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.780+0000 7f34e3c4e640 1 Processor -- start 2024-03-06T07:25:38.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.780+0000 7f34e3c4e640 1 -- start start 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e3c4e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f34dc0a8290 0x7f34dc1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e3c4e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34dc0c4740 0x7f34dc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e3c4e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34dc0c62a0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e3c4e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc153f00 con 0x7f34dc0c4740 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e3c4e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34dc0706e0 con 0x7f34dc0a8290 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e11e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34dc0c4740 0x7f34dc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e21eb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34dc0c69a0 0x7f34dc072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50834/0 (socket says 172.21.15.17:50834) 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e21eb640 1 -- 172.21.15.17:0/3580402789 learned_addr learned my addr 172.21.15.17:0/3580402789 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:38.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e19ea640 1 --2- 172.21.15.17:0/3580402789 >> v2:172.21.15.163:3300/0 conn(0x7f34dc0a8290 0x7f34dc1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e11e9640 1 -- 172.21.15.17:0/3580402789 >> v2:172.21.15.163:3300/0 conn(0x7f34dc0a8290 msgr2=0x7f34dc1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e11e9640 1 --2- 172.21.15.17:0/3580402789 >> v2:172.21.15.163:3300/0 conn(0x7f34dc0a8290 0x7f34dc1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e11e9640 1 -- 172.21.15.17:0/3580402789 >> v1:172.21.15.17:6789/0 conn(0x7f34dc0c69a0 legacy=0x7f34dc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e11e9640 1 -- 172.21.15.17:0/3580402789 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0c4c00 con 0x7f34dc0c4740 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.781+0000 7f34e09e8640 1 -- 172.21.15.17:0/3580402789 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934792087 0 0) 0x7f34dc0c62a0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.782+0000 7f34e11e9640 1 --2- 172.21.15.17:0/3580402789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c4740 0x7f34dc0701a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0735b0 tx=0x7f34cc02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d87ad0416c38cc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.782+0000 7f34e09e8640 1 -- 172.21.15.17:0/3580402789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc038070 con 0x7f34dc0c4740 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.782+0000 7f34e09e8640 1 -- 172.21.15.17:0/3580402789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc031710 con 0x7f34dc0c4740 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.782+0000 7f34e09e8640 1 -- 172.21.15.17:0/3580402789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc031ae0 con 0x7f34dc0c4740 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3580402789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c4740 msgr2=0x7f34dc0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3580402789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c4740 0x7f34dc0701a0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0735b0 tx=0x7f34cc02b7b0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3580402789 shutdown_connections 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3580402789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c4740 0x7f34dc0701a0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3580402789 >> v2:172.21.15.163:3300/0 conn(0x7f34dc0a8290 0x7f34dc1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3580402789 >> 172.21.15.17:0/3580402789 conn(0x7f34dc0c5560 msgr2=0x7f34dc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3580402789 shutdown_connections 2024-03-06T07:25:38.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.783+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3580402789 wait complete. 2024-03-06T07:25:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.784+0000 7f34e3c4e640 1 Processor -- start 2024-03-06T07:25:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.784+0000 7f34e3c4e640 1 -- start start 2024-03-06T07:25:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.784+0000 7f34e3c4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34dc0a8290 0x7f34dc0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.784+0000 7f34e3c4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 0x7f34dc0c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc06fc80 con 0x7f34dc0c69a0 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc06f9a0 con 0x7f34dc0a8290 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc072d90 con 0x7f34dc0c4740 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57516/0 (socket says 172.21.15.17:57516) 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 -- 172.21.15.17:0/3513843188 learned_addr learned my addr 172.21.15.17:0/3513843188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e19ea640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34dc0a8290 0x7f34dc0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e11e9640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 0x7f34dc0c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 -- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 msgr2=0x7f34dc0c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 0x7f34dc0c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 -- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34dc0a8290 msgr2=0x7f34dc0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34dc0a8290 0x7f34dc0adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 -- 172.21.15.17:0/3513843188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc14dfe0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e11e9640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 0x7f34dc0c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e21eb640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c700 tx=0x7f34d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4018070 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d400ce10 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d400a800 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc0a9950 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.785+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc0ac350 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.786+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc04f330 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.787+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34d401b070 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.787+0000 7f34d27fc640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 0x7f34c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.787+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f34d40898f0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.788+0000 7f34e19ea640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 0x7f34c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:38.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.788+0000 7f34e19ea640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 0x7f34c006d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009ad0 tx=0x7f34d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:38.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.792+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34d40585c0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.973+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34dc0c8480 con 0x7f34dc0c69a0 2024-03-06T07:25:38.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.973+0000 7f34d27fc640 1 -- 172.21.15.17:0/3513843188 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34d4057ec0 con 0x7f34dc0c69a0 2024-03-06T07:25:38.976 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 msgr2=0x7f34c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 0x7f34c006d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009ad0 tx=0x7f34d8008040 comp rx=0 tx=0).stop 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 msgr2=0x7f34dc0ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f34d400c700 tx=0x7f34d400cbc0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 shutdown_connections 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34c006b500 0x7f34c006d9c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34dc0c69a0 0x7f34dc0ab8a0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34dc0c4740 0x7f34dc0c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 --2- 172.21.15.17:0/3513843188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34dc0a8290 0x7f34dc0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:38.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 >> 172.21.15.17:0/3513843188 conn(0x7f34dc0c5560 msgr2=0x7f34dc0a9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:38.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 shutdown_connections 2024-03-06T07:25:38.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:38.975+0000 7f34e3c4e640 1 -- 172.21.15.17:0/3513843188 wait complete. 2024-03-06T07:25:38.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:25:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 Processor -- start 2024-03-06T07:25:39.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 -- start start 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f967014e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700b0450 con 0x7f9670155c30 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96701560e0 con 0x7f96700b0070 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.091+0000 7f9676dee640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967006c020 con 0x7f967006ca70 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9670155c30 0x7f96700c5380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50840/0 (socket says 172.21.15.17:50840) 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966ffff640 1 -- 172.21.15.17:0/40076589 learned_addr learned my addr 172.21.15.17:0/40076589 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257863204 0 0) 0x7f965c001110 con 0x7f9670155c30 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9660003520 con 0x7f9670155c30 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f967538b640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f967014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612783913 0 0) 0x7f9660003520 con 0x7f9670155c30 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 msgr2=0x7f967014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f967014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 >> v1:172.21.15.163:6789/0 conn(0x7f96700b0070 legacy=0x7f96700c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700c77f0 con 0x7f9670155c30 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f965c001840 con 0x7f9670155c30 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 >> v1:172.21.15.17:6789/0 conn(0x7f9670155c30 legacy=0x7f96700c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96600039e0 0x7f9660003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f9660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9660004d40 0x7f96600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967006c020 con 0x7f9660004d40 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966000bb50 con 0x7f96600039e0 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f966000bd20 con 0x7f967006ca70 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f966ffff640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f9660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f9674b8a640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96600039e0 0x7f9660003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.092+0000 7f967538b640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9660004d40 0x7f96600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 -- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 msgr2=0x7f9660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f9660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 -- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96600039e0 msgr2=0x7f9660003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96600039e0 0x7f9660003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 -- 172.21.15.17:0/40076589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660003520 con 0x7f9660004d40 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f966ffff640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f967006ca70 0x7f9660004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.093+0000 7f967538b640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9660004d40 0x7f96600051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f9668009fc0 tx=0x7f9660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20453206bc9a6dc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668039070 con 0x7f9660004d40 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966802cbd0 con 0x7f9660004d40 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f966f7fe640 1 -- 172.21.15.17:0/40076589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668009c10 con 0x7f9660004d40 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 -- 172.21.15.17:0/40076589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9660004d40 msgr2=0x7f96600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 --2- 172.21.15.17:0/40076589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9660004d40 0x7f96600051c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f9668009fc0 tx=0x7f9660005830 comp rx=0 tx=0).stop 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f967538b640 1 -- 172.21.15.17:0/40076589 reap_dead start 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 -- 172.21.15.17:0/40076589 shutdown_connections 2024-03-06T07:25:39.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 -- 172.21.15.17:0/40076589 >> 172.21.15.17:0/40076589 conn(0x7f96700ac590 msgr2=0x7f96700ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:39.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 -- 172.21.15.17:0/40076589 shutdown_connections 2024-03-06T07:25:39.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.094+0000 7f9676dee640 1 -- 172.21.15.17:0/40076589 wait complete. 2024-03-06T07:25:39.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.095+0000 7f9676dee640 1 Processor -- start 2024-03-06T07:25:39.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.095+0000 7f9676dee640 1 -- start start 2024-03-06T07:25:39.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9670150a40 0x7f96700c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96700c45f0 0x7f9670153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967006c550 con 0x7f96701de810 2024-03-06T07:25:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967006c270 con 0x7f9670150a40 2024-03-06T07:25:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9676dee640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967006da80 con 0x7f96700c45f0 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f966ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f966ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57528/0 (socket says 172.21.15.17:57528) 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f966ffff640 1 -- 172.21.15.17:0/158195878 learned_addr learned my addr 172.21.15.17:0/158195878 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f9674b8a640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9670150a40 0x7f96700c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.096+0000 7f967538b640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96700c45f0 0x7f9670153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 -- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96700c45f0 msgr2=0x7f9670153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96700c45f0 0x7f9670153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 -- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9670150a40 msgr2=0x7f96700c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9670150a40 0x7f96700c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 -- 172.21.15.17:0/158195878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670155070 con 0x7f96701de810 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966ffff640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f965c007290 tx=0x7f965c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00ca70 con 0x7f96701de810 2024-03-06T07:25:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c018070 con 0x7f96701de810 2024-03-06T07:25:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96701eebb0 con 0x7f96701de810 2024-03-06T07:25:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00ce10 con 0x7f96701de810 2024-03-06T07:25:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.097+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700c1f50 con 0x7f96701de810 2024-03-06T07:25:39.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.098+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967004f330 con 0x7f96701de810 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.098+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f965c01d460 con 0x7f96701de810 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.099+0000 7f966d7fa640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.099+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f965c08a720 con 0x7f96701de810 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.099+0000 7f9674b8a640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.100+0000 7f9674b8a640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96640017e0 tx=0x7f9664009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.103+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96700b0450 con 0x7f96701de810 2024-03-06T07:25:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.282+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f967006d860 con 0x7f96701de810 2024-03-06T07:25:39.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.282+0000 7f966d7fa640 1 -- 172.21.15.17:0/158195878 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f965c058d10 con 0x7f96701de810 2024-03-06T07:25:39.285 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:25:39.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96640017e0 tx=0x7f9664009040 comp rx=0 tx=0).stop 2024-03-06T07:25:39.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 msgr2=0x7f96700c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f965c007290 tx=0x7f965c007750 comp rx=0 tx=0).stop 2024-03-06T07:25:39.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 shutdown_connections 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96700c45f0 0x7f9670153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96701de810 0x7f96700c40b0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 --2- 172.21.15.17:0/158195878 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9670150a40 0x7f96700c3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.284+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 >> 172.21.15.17:0/158195878 conn(0x7f96700ac590 msgr2=0x7f96700af040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.285+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 shutdown_connections 2024-03-06T07:25:39.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.285+0000 7f9676dee640 1 -- 172.21.15.17:0/158195878 wait complete. 2024-03-06T07:25:39.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:25:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.400+0000 7f15d7d0c640 1 Processor -- start 2024-03-06T07:25:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.400+0000 7f15d7d0c640 1 -- start start 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d7d0c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 0x7f15d01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d7d0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d7d0c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00c62a0 con 0x7f15d00c4740 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d7d0c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d0153f00 con 0x7f15d00a8290 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d7d0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00c4c00 con 0x7f15d00c69a0 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43998/0 (socket says 172.21.15.17:43998) 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d5aa8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 0x7f15d01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 -- 172.21.15.17:0/1695360286 learned_addr learned my addr 172.21.15.17:0/1695360286 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 -- 172.21.15.17:0/1695360286 >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 msgr2=0x7f15d01539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 --2- 172.21.15.17:0/1695360286 >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 0x7f15d01539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 -- 172.21.15.17:0/1695360286 >> v1:172.21.15.163:6789/0 conn(0x7f15d00c4740 legacy=0x7f15d00701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d62a9640 1 -- 172.21.15.17:0/1695360286 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00c4ee0 con 0x7f15d00c69a0 2024-03-06T07:25:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.401+0000 7f15d5aa8640 1 --2- 172.21.15.17:0/1695360286 >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 0x7f15d01539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.402+0000 7f15d62a9640 1 --2- 172.21.15.17:0/1695360286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009e50 tx=0x7f15c802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2e29a598a9c5a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d4aa6640 1 -- 172.21.15.17:0/1695360286 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c802c9e0 con 0x7f15d00c69a0 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d4aa6640 1 -- 172.21.15.17:0/1695360286 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c802cb80 con 0x7f15d00c69a0 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d4aa6640 1 -- 172.21.15.17:0/1695360286 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80343f0 con 0x7f15d00c69a0 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 -- 172.21.15.17:0/1695360286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 msgr2=0x7f15d0072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/1695360286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009e50 tx=0x7f15c802be40 comp rx=0 tx=0).stop 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 -- 172.21.15.17:0/1695360286 shutdown_connections 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/1695360286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d0072680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/1695360286 >> v2:172.21.15.163:3300/0 conn(0x7f15d00a8290 0x7f15d01539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 -- 172.21.15.17:0/1695360286 >> 172.21.15.17:0/1695360286 conn(0x7f15d00c5560 msgr2=0x7f15d00c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 -- 172.21.15.17:0/1695360286 shutdown_connections 2024-03-06T07:25:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.403+0000 7f15d7d0c640 1 -- 172.21.15.17:0/1695360286 wait complete. 2024-03-06T07:25:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.404+0000 7f15d7d0c640 1 Processor -- start 2024-03-06T07:25:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.404+0000 7f15d7d0c640 1 -- start start 2024-03-06T07:25:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15d00a8290 0x7f15d00abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00c6e30 con 0x7f15d00c4740 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d006fc80 con 0x7f15d00a8290 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d7d0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0072bc0 con 0x7f15d00c69a0 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57534/0 (socket says 172.21.15.17:57534) 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 -- 172.21.15.17:0/3497140700 learned_addr learned my addr 172.21.15.17:0/3497140700 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d5aa8640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15d00a8290 0x7f15d00abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d62a9640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 -- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 msgr2=0x7f15d014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 -- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15d00a8290 msgr2=0x7f15d00abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15d00a8290 0x7f15d00abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.405+0000 7f15d52a7640 1 -- 172.21.15.17:0/3497140700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0155d60 con 0x7f15d00c4740 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15d52a7640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c400 tx=0x7f15c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c00052d0 con 0x7f15d00c4740 2024-03-06T07:25:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c0005490 con 0x7f15d00c4740 2024-03-06T07:25:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d0156e80 con 0x7f15d00c4740 2024-03-06T07:25:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0007260 con 0x7f15d00c4740 2024-03-06T07:25:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.406+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d00ac050 con 0x7f15d00c4740 2024-03-06T07:25:39.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.407+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d004f330 con 0x7f15d00c4740 2024-03-06T07:25:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.407+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15c0007400 con 0x7f15d00c4740 2024-03-06T07:25:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.408+0000 7f15c67fc640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 0x7f15b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.408+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f15c0089880 con 0x7f15d00c4740 2024-03-06T07:25:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.408+0000 7f15d5aa8640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 0x7f15b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.409+0000 7f15d5aa8640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 0x7f15b406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f15d00aefe0 tx=0x7f15cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.412+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15c0057f00 con 0x7f15d00c4740 2024-03-06T07:25:39.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:39.424 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:25:39.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.610+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15d00c8480 con 0x7f15d00c4740 2024-03-06T07:25:39.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.610+0000 7f15c67fc640 1 -- 172.21.15.17:0/3497140700 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15c005bfa0 con 0x7f15d00c4740 2024-03-06T07:25:39.613 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:25:39.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 msgr2=0x7f15b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 0x7f15b406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f15d00aefe0 tx=0x7f15cc009190 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 msgr2=0x7f15d014ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f15c000c400 tx=0x7f15c000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 shutdown_connections 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f15b406b4d0 0x7f15b406d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15d00c69a0 0x7f15d014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15d00c4740 0x7f15d014ceb0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 --2- 172.21.15.17:0/3497140700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15d00a8290 0x7f15d00abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 >> 172.21.15.17:0/3497140700 conn(0x7f15d00c5560 msgr2=0x7f15d00aa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 shutdown_connections 2024-03-06T07:25:39.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.612+0000 7f15d7d0c640 1 -- 172.21.15.17:0/3497140700 wait complete. 2024-03-06T07:25:39.625 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.723+0000 7f8e988eb640 1 Processor -- start 2024-03-06T07:25:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.723+0000 7f8e988eb640 1 -- start start 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e988eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e94159270 0x7f8e94155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e988eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940c73d0 con 0x7f8e9414f360 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e988eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940be2e0 con 0x7f8e940c88c0 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e988eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e94159700 con 0x7f8e94159270 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e9259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8e9414f360 0x7f8e940c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50846/0 (socket says 172.21.15.17:50846) 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.724+0000 7f8e9259c640 1 -- 172.21.15.17:0/2197221947 learned_addr learned my addr 172.21.15.17:0/2197221947 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e92d9d640 1 --2- 172.21.15.17:0/2197221947 >> v2:172.21.15.163:3300/0 conn(0x7f8e94159270 0x7f8e94155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180288795 0 0) 0x7f8e940c73d0 con 0x7f8e9414f360 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e7c003520 con 0x7f8e9414f360 2024-03-06T07:25:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067304938 0 0) 0x7f8e7c003520 con 0x7f8e9414f360 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 >> v2:172.21.15.163:3300/0 conn(0x7f8e94159270 msgr2=0x7f8e94155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 --2- 172.21.15.17:0/2197221947 >> v2:172.21.15.163:3300/0 conn(0x7f8e94159270 0x7f8e94155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 >> v1:172.21.15.163:6789/0 conn(0x7f8e940c88c0 legacy=0x7f8e94158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940c7670 con 0x7f8e9414f360 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f8e840019f0 con 0x7f8e9414f360 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 >> v1:172.21.15.17:6789/0 conn(0x7f8e9414f360 legacy=0x7f8e940c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e7c0039b0 0x7f8e7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c0051c0 0x7f8e7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94159700 con 0x7f8e7c004800 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c00c160 con 0x7f8e7c0039b0 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c00c360 con 0x7f8e7c0051c0 2024-03-06T07:25:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9259c640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e7c0039b0 0x7f8e7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e92d9d640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c0051c0 0x7f8e7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 -- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c0051c0 msgr2=0x7f8e7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c0051c0 0x7f8e7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 -- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e7c0039b0 msgr2=0x7f8e7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e7c0039b0 0x7f8e7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e91d9b640 1 -- 172.21.15.17:0/2197221947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c003520 con 0x7f8e7c004800 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e92d9d640 1 -- 172.21.15.17:0/2197221947 reap_dead start 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.725+0000 7f8e9259c640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e7c0039b0 0x7f8e7c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.726+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f8e88001720 tx=0x7f8e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6c603680907695 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.726+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88001370 con 0x7f8e7c004800 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.726+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e880052d0 con 0x7f8e7c004800 2024-03-06T07:25:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.726+0000 7f8e9159a640 1 -- 172.21.15.17:0/2197221947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88003830 con 0x7f8e7c004800 2024-03-06T07:25:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 -- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 msgr2=0x7f8e7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f8e88001720 tx=0x7f8e7c005830 comp rx=0 tx=0).stop 2024-03-06T07:25:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 -- 172.21.15.17:0/2197221947 shutdown_connections 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 --2- 172.21.15.17:0/2197221947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e7c004800 0x7f8e7c004c80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 -- 172.21.15.17:0/2197221947 >> 172.21.15.17:0/2197221947 conn(0x7f8e940ac570 msgr2=0x7f8e940ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 -- 172.21.15.17:0/2197221947 shutdown_connections 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.727+0000 7f8e988eb640 1 -- 172.21.15.17:0/2197221947 wait complete. 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.728+0000 7f8e988eb640 1 Processor -- start 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.728+0000 7f8e988eb640 1 -- start start 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c004800 0x7f8e9414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e9414adf0 0x7f8e940afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e940c7c30 con 0x7f8e941eaea0 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94155c10 con 0x7f8e9414adf0 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e988eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94155d90 con 0x7f8e7c004800 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57562/0 (socket says 172.21.15.17:57562) 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 -- 172.21.15.17:0/1901112898 learned_addr learned my addr 172.21.15.17:0/1901112898 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e9259c640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c004800 0x7f8e9414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 -- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c004800 msgr2=0x7f8e9414b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c004800 0x7f8e9414b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 -- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e9414adf0 msgr2=0x7f8e940afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e9414adf0 0x7f8e940afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.729+0000 7f8e91d9b640 1 -- 172.21.15.17:0/1901112898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e941e13b0 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e91d9b640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005040 tx=0x7f8e88001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88041070 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e88005480 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e941e1690 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e88007a30 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.730+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e941e0e60 con 0x7f8e941eaea0 2024-03-06T07:25:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.731+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9404f330 con 0x7f8e941eaea0 2024-03-06T07:25:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.731+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e88044070 con 0x7f8e941eaea0 2024-03-06T07:25:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.732+0000 7f8e82ffd640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.732+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8e880abf60 con 0x7f8e941eaea0 2024-03-06T07:25:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.732+0000 7f8e9259c640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.732+0000 7f8e9259c640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 0x7f8e7406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8e84001330 tx=0x7f8e84006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:39.736+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e8807aba0 con 0x7f8e941eaea0 2024-03-06T07:25:40.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.004+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e940c95b0 con 0x7f8e941eaea0 2024-03-06T07:25:40.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.005+0000 7f8e82ffd640 1 -- 172.21.15.17:0/1901112898 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f8e8807a4a0 con 0x7f8e941eaea0 2024-03-06T07:25:40.008 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:40.008 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":130,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:36.061572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8265":{"gid":8265,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8265,"qdb_cluster":[8265]},"id":10}]} 2024-03-06T07:25:40.008 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 130 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 msgr2=0x7f8e7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 0x7f8e7406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8e84001330 tx=0x7f8e84006040 comp rx=0 tx=0).stop 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 msgr2=0x7f8e941e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005040 tx=0x7f8e88001120 comp rx=0 tx=0).stop 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 shutdown_connections 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8e7406b4d0 0x7f8e7406d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e9414adf0 0x7f8e940afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e941eaea0 0x7f8e941e1ea0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 --2- 172.21.15.17:0/1901112898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e7c004800 0x7f8e9414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 >> 172.21.15.17:0/1901112898 conn(0x7f8e940ac570 msgr2=0x7f8e940c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 shutdown_connections 2024-03-06T07:25:40.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.007+0000 7f8e988eb640 1 -- 172.21.15.17:0/1901112898 wait complete. 2024-03-06T07:25:40.021 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:40.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 Processor -- start 2024-03-06T07:25:40.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 -- start start 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f021c14f360 0x7f021c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0c73d0 con 0x7f021c158b60 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0be2e0 con 0x7f021c0c88c0 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.120+0000 7f0224193640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c159700 con 0x7f021c14f360 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0222730640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f021c158b60 0x7f021c158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50852/0 (socket says 172.21.15.17:50852) 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0222730640 1 -- 172.21.15.17:0/521898750 learned_addr learned my addr 172.21.15.17:0/521898750 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:40.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0221f2f640 1 --2- 172.21.15.17:0/521898750 >> v2:172.21.15.163:3300/0 conn(0x7f021c14f360 0x7f021c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366578004 0 0) 0x7f021c0c73d0 con 0x7f021c158b60 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0204003520 con 0x7f021c158b60 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434825215 0 0) 0x7f0204003520 con 0x7f021c158b60 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 >> v2:172.21.15.163:3300/0 conn(0x7f021c14f360 msgr2=0x7f021c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 --2- 172.21.15.17:0/521898750 >> v2:172.21.15.163:3300/0 conn(0x7f021c14f360 0x7f021c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 >> v1:172.21.15.163:6789/0 conn(0x7f021c0c88c0 legacy=0x7f021c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0c7670 con 0x7f021c158b60 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f02140019f0 con 0x7f021c158b60 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 >> v1:172.21.15.17:6789/0 conn(0x7f021c158b60 legacy=0x7f021c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 0x7f0204003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0204004800 0x7f0204004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02040051c0 0x7f020400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c159700 con 0x7f02040039b0 2024-03-06T07:25:40.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020400c160 con 0x7f02040051c0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020400c360 con 0x7f0204004800 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0221f2f640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 0x7f0204003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f0222730640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02040051c0 0x7f020400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.121+0000 7f022172e640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0204004800 0x7f0204004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 -- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0204004800 msgr2=0x7f0204004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0204004800 0x7f0204004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 -- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02040051c0 msgr2=0x7f020400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02040051c0 0x7f020400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 -- 172.21.15.17:0/521898750 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0204003520 con 0x7f02040039b0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0222730640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02040051c0 0x7f020400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0222730640 1 -- 172.21.15.17:0/521898750 reap_dead start 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0221f2f640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 0x7f0204003dd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f020c009ac0 tx=0x7f0204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858ad5ab257a98f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.122+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c039070 con 0x7f02040039b0 2024-03-06T07:25:40.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c02c6b0 con 0x7f02040039b0 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0220f2d640 1 -- 172.21.15.17:0/521898750 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c009690 con 0x7f02040039b0 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 -- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 msgr2=0x7f0204003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 0x7f0204003dd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f020c009ac0 tx=0x7f0204005830 comp rx=0 tx=0).stop 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 -- 172.21.15.17:0/521898750 shutdown_connections 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 --2- 172.21.15.17:0/521898750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02040039b0 0x7f0204003dd0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 -- 172.21.15.17:0/521898750 >> 172.21.15.17:0/521898750 conn(0x7f021c0ac570 msgr2=0x7f021c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 -- 172.21.15.17:0/521898750 shutdown_connections 2024-03-06T07:25:40.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.123+0000 7f0224193640 1 -- 172.21.15.17:0/521898750 wait complete. 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.125+0000 7f0224193640 1 Processor -- start 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.125+0000 7f0224193640 1 -- start start 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02040039b0 0x7f021c154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c0b02b0 0x7f021c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0c7c30 con 0x7f021c0bff10 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c155c10 con 0x7f021c0b02b0 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0224193640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c155d90 con 0x7f02040039b0 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57576/0 (socket says 172.21.15.17:57576) 2024-03-06T07:25:40.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 -- 172.21.15.17:0/1365207881 learned_addr learned my addr 172.21.15.17:0/1365207881 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f022172e640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c0b02b0 0x7f021c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0221f2f640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02040039b0 0x7f021c154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 -- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02040039b0 msgr2=0x7f021c154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02040039b0 0x7f021c154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 -- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c0b02b0 msgr2=0x7f021c0adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c0b02b0 0x7f021c0adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.126+0000 7f0222730640 1 -- 172.21.15.17:0/1365207881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c1ee010 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f0222730640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f021c0c1860 tx=0x7f021400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400edd0 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214014430 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c1ee2f0 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214014840 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.127+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c154ed0 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.128+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c04f330 con 0x7f021c0bff10 2024-03-06T07:25:40.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.128+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02140149e0 con 0x7f021c0bff10 2024-03-06T07:25:40.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.128+0000 7f021a7fc640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.129+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f021408a2b0 con 0x7f021c0bff10 2024-03-06T07:25:40.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.129+0000 7f0221f2f640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.129+0000 7f0221f2f640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f020c036000 tx=0x7f020c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.133+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0214058930 con 0x7f021c0bff10 2024-03-06T07:25:40.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.315+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f021c0c8d50 con 0x7f021c0bff10 2024-03-06T07:25:40.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.317+0000 7f021a7fc640 1 -- 172.21.15.17:0/1365207881 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+17155 (secure 0 0 0) 0x7f021405c9d0 con 0x7f021c0bff10 2024-03-06T07:25:40.320 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:40.320 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":138,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:33.047446+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6837/3327948285":"2024-03-07T07:25:18.978040+0000","172.21.15.163:6834/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6835/3327948285":"2024-03-07T07:25:18.978040+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 msgr2=0x7f020806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f020c036000 tx=0x7f020c030000 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 msgr2=0x7f021c0adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f021c0c1860 tx=0x7f021400e530 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 shutdown_connections 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f021c0bff10 0x7f021c0adff0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c0b02b0 0x7f021c0adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 --2- 172.21.15.17:0/1365207881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02040039b0 0x7f021c154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 >> 172.21.15.17:0/1365207881 conn(0x7f021c0ac570 msgr2=0x7f021c0ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 shutdown_connections 2024-03-06T07:25:40.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.319+0000 7f0224193640 1 -- 172.21.15.17:0/1365207881 wait complete. 2024-03-06T07:25:40.336 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:25:40.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:40.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.435+0000 7fc8a277a640 1 Processor -- start 2024-03-06T07:25:40.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.435+0000 7fc8a277a640 1 -- start start 2024-03-06T07:25:40.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc8a277a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5480 0x7fc89c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc8a277a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc89c0c5860 0x7fc89c06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc8a277a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0c5cc0 con 0x7fc89c1586f0 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc8a277a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c06df40 con 0x7fc89c0c5860 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc8a277a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1591c0 con 0x7fc89c0c5480 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc89b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc89c0c5860 0x7fc89c06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc89b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc89c0c5860 0x7fc89c06f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57586/0 (socket says 172.21.15.17:57586) 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc89b7fe640 1 -- 172.21.15.17:0/1565190204 learned_addr learned my addr 172.21.15.17:0/1565190204 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.436+0000 7fc89bfff640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5480 0x7fc89c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.437+0000 7fc89b7fe640 1 -- 172.21.15.17:0/1565190204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5480 msgr2=0x7fc89c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.437+0000 7fc89b7fe640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5480 0x7fc89c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.437+0000 7fc89b7fe640 1 -- 172.21.15.17:0/1565190204 >> v1:172.21.15.163:6789/0 conn(0x7fc89c1586f0 legacy=0x7fc89c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.437+0000 7fc89b7fe640 1 -- 172.21.15.17:0/1565190204 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c1594a0 con 0x7fc89c0c5860 2024-03-06T07:25:40.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.437+0000 7fc89b7fe640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c0c5860 0x7fc89c06f6a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009870 tx=0x7fc88c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbbebd54e06c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc89affd640 1 -- 172.21.15.17:0/1565190204 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c035020 con 0x7fc89c0c5860 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc89affd640 1 -- 172.21.15.17:0/1565190204 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c031640 con 0x7fc89c0c5860 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc89affd640 1 -- 172.21.15.17:0/1565190204 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c02d040 con 0x7fc89c0c5860 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 -- 172.21.15.17:0/1565190204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c0c5860 msgr2=0x7fc89c06f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c0c5860 0x7fc89c06f6a0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009870 tx=0x7fc88c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 -- 172.21.15.17:0/1565190204 shutdown_connections 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c0c5860 0x7fc89c06f6a0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 --2- 172.21.15.17:0/1565190204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5480 0x7fc89c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 -- 172.21.15.17:0/1565190204 >> 172.21.15.17:0/1565190204 conn(0x7fc89c06b1a0 msgr2=0x7fc89c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 -- 172.21.15.17:0/1565190204 shutdown_connections 2024-03-06T07:25:40.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.438+0000 7fc8a277a640 1 -- 172.21.15.17:0/1565190204 wait complete. 2024-03-06T07:25:40.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.439+0000 7fc8a277a640 1 Processor -- start 2024-03-06T07:25:40.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.439+0000 7fc8a277a640 1 -- start start 2024-03-06T07:25:40.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc89c0c5480 0x7fc89c1eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5860 0x7fc89c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c152a20 con 0x7fc89c1586f0 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c155680 con 0x7fc89c0c5480 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a277a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c155050 con 0x7fc89c0c5860 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a0d17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a0d17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57592/0 (socket says 172.21.15.17:57592) 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc8a0d17640 1 -- 172.21.15.17:0/2363719217 learned_addr learned my addr 172.21.15.17:0/2363719217 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc89b7fe640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5860 0x7fc89c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.440+0000 7fc89bfff640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc89c0c5480 0x7fc89c1eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 -- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5860 msgr2=0x7fc89c14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5860 0x7fc89c14c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 -- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc89c0c5480 msgr2=0x7fc89c1eac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc89c0c5480 0x7fc89c1eac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 -- 172.21.15.17:0/2363719217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c1eb340 con 0x7fc89c1586f0 2024-03-06T07:25:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a0d17640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c730 tx=0x7fc89400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003b70 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894003d30 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1eb620 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894005310 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.441+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c14c560 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.442+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c04f330 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.442+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8940054b0 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.443+0000 7fc898ff9640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 0x7fc87c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.443+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc89408a0d0 con 0x7fc89c1586f0 2024-03-06T07:25:40.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.443+0000 7fc89bfff640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 0x7fc87c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.443+0000 7fc89bfff640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 0x7fc87c06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc884009b40 tx=0x7fc884009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.447+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc894058750 con 0x7fc89c1586f0 2024-03-06T07:25:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.684+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89c06e0c0 con 0x7fc89c1586f0 2024-03-06T07:25:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.685+0000 7fc898ff9640 1 -- 172.21.15.17:0/2363719217 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4990 (secure 0 0 0) 0x7fc89405c7f0 con 0x7fc89c1586f0 2024-03-06T07:25:40.688 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:40.688 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":130,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:36.061572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8265":{"gid":8265,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8265,"qdb_cluster":[8265]},"id":10}]} 2024-03-06T07:25:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 130 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 msgr2=0x7fc87c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 0x7fc87c06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc884009b40 tx=0x7fc884009190 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 msgr2=0x7fc89c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c730 tx=0x7fc89400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 shutdown_connections 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc87c06b4d0 0x7fc87c06d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc89c1586f0 0x7fc89c153a60 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc89c0c5860 0x7fc89c14c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 --2- 172.21.15.17:0/2363719217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc89c0c5480 0x7fc89c1eac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 >> 172.21.15.17:0/2363719217 conn(0x7fc89c06b1a0 msgr2=0x7fc89c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 shutdown_connections 2024-03-06T07:25:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.687+0000 7fc8a277a640 1 -- 172.21.15.17:0/2363719217 wait complete. 2024-03-06T07:25:40.701 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:40.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 Processor -- start 2024-03-06T07:25:40.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 -- start start 2024-03-06T07:25:40.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c840c88c0 0x7f8c84155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c84158b60 0x7f8c84158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c840c73d0 con 0x7f8c8414f360 2024-03-06T07:25:40.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c840c8dd0 con 0x7f8c84158b60 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c8b36b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840be2e0 con 0x7f8c840c88c0 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.800+0000 7f8c89908640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c84158b60 0x7f8c84158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c84158b60 0x7f8c84158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48816/0 (socket says 172.21.15.17:48816) 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 -- 172.21.15.17:0/3527107607 learned_addr learned my addr 172.21.15.17:0/3527107607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c88906640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c840c88c0 0x7f8c84155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 -- 172.21.15.17:0/3527107607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c840c88c0 msgr2=0x7f8c84155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c840c88c0 0x7f8c84155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 -- 172.21.15.17:0/3527107607 >> v1:172.21.15.163:6789/0 conn(0x7f8c8414f360 legacy=0x7f8c840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:40.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.801+0000 7f8c89908640 1 -- 172.21.15.17:0/3527107607 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840c7670 con 0x7f8c84158b60 2024-03-06T07:25:40.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c89908640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c84158b60 0x7f8c84158ff0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763b2b43f4400f2d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c83fff640 1 -- 172.21.15.17:0/3527107607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0317c0 con 0x7f8c84158b60 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c83fff640 1 -- 172.21.15.17:0/3527107607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c037070 con 0x7f8c84158b60 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c83fff640 1 -- 172.21.15.17:0/3527107607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031b80 con 0x7f8c84158b60 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 -- 172.21.15.17:0/3527107607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c84158b60 msgr2=0x7f8c84158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c84158b60 0x7f8c84158ff0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c02b860 comp rx=0 tx=0).stop 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 -- 172.21.15.17:0/3527107607 shutdown_connections 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c84158b60 0x7f8c84158ff0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/3527107607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c840c88c0 0x7f8c84155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 -- 172.21.15.17:0/3527107607 >> 172.21.15.17:0/3527107607 conn(0x7f8c840ac570 msgr2=0x7f8c840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 -- 172.21.15.17:0/3527107607 shutdown_connections 2024-03-06T07:25:40.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.802+0000 7f8c8b36b640 1 -- 172.21.15.17:0/3527107607 wait complete. 2024-03-06T07:25:40.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.803+0000 7f8c8b36b640 1 Processor -- start 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.803+0000 7f8c8b36b640 1 -- start start 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 0x7f8c84150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 0x7f8c84151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840c5a50 con 0x7f8c840c88c0 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840c7c30 con 0x7f8c8414f360 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c8b36b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840c5bd0 con 0x7f8c84158b60 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48832/0 (socket says 172.21.15.17:48832) 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 -- 172.21.15.17:0/1581672760 learned_addr learned my addr 172.21.15.17:0/1581672760 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c88906640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 0x7f8c84150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89908640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 0x7f8c84151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 -- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 msgr2=0x7f8c84151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 0x7f8c84151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 -- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 msgr2=0x7f8c84150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 0x7f8c84150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89107640 1 -- 172.21.15.17:0/1581672760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c841ee810 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c88906640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 0x7f8c84150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.804+0000 7f8c89908640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 0x7f8c84151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c89107640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400ba50 tx=0x7f8c7400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74019070 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400cc20 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c84150450 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74014680 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.805+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c840c0380 con 0x7f8c840c88c0 2024-03-06T07:25:40.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.806+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8406c770 con 0x7f8c840c88c0 2024-03-06T07:25:40.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.806+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c7401c070 con 0x7f8c840c88c0 2024-03-06T07:25:40.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.807+0000 7f8c81ffb640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:40.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.807+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8c7408a080 con 0x7f8c840c88c0 2024-03-06T07:25:40.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.807+0000 7f8c88906640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:40.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.807+0000 7f8c88906640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8c84152d10 tx=0x7f8c78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:40.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:40.811+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c8406c770 con 0x7f8c840c88c0 2024-03-06T07:25:41.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.140+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c8404f330 con 0x7f8c840c88c0 2024-03-06T07:25:41.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.141+0000 7f8c81ffb640 1 -- 172.21.15.17:0/1581672760 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f8c74058700 con 0x7f8c840c88c0 2024-03-06T07:25:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:41.143 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":130,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:07.286023+0000","modified":"2024-03-06T07:25:36.061572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8265":{"gid":8265,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6834/824017697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":824017697},{"type":"v1","addr":"172.21.15.17:6834","nonce":824017697}]},"join_fscid":10,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8265,"qdb_cluster":[8265]},"id":10}]} 2024-03-06T07:25:41.144 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 130 2024-03-06T07:25:41.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8c84152d10 tx=0x7f8c78008040 comp rx=0 tx=0).stop 2024-03-06T07:25:41.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 msgr2=0x7f8c84151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400ba50 tx=0x7f8c7400bf10 comp rx=0 tx=0).stop 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 shutdown_connections 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c84158b60 0x7f8c84151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8414f360 0x7f8c84150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 --2- 172.21.15.17:0/1581672760 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c840c88c0 0x7f8c84151ac0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 >> 172.21.15.17:0/1581672760 conn(0x7f8c840ac570 msgr2=0x7f8c840ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 shutdown_connections 2024-03-06T07:25:41.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.143+0000 7f8c8b36b640 1 -- 172.21.15.17:0/1581672760 wait complete. 2024-03-06T07:25:41.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:41.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.287+0000 7f16a99b6640 1 Processor -- start 2024-03-06T07:25:41.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.287+0000 7f16a99b6640 1 -- start start 2024-03-06T07:25:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.287+0000 7f16a99b6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16a40aa180 0x7f16a40c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.287+0000 7f16a99b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16a414ec00 0x7f16a414f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.287+0000 7f16a99b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a41550c0 0x7f16a4155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a99b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40c4610 con 0x7f16a414ec00 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a99b6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a414f5d0 con 0x7f16a40aa180 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a99b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40aa660 con 0x7f16a41550c0 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16a414ec00 0x7f16a414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16a414ec00 0x7f16a414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48848/0 (socket says 172.21.15.17:48848) 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 -- 172.21.15.17:0/2398686405 learned_addr learned my addr 172.21.15.17:0/2398686405 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a37fe640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a41550c0 0x7f16a4155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a2ffd640 1 --2- 172.21.15.17:0/2398686405 >> v2:172.21.15.163:3300/0 conn(0x7f16a40aa180 0x7f16a40c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 -- 172.21.15.17:0/2398686405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a41550c0 msgr2=0x7f16a4155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a41550c0 0x7f16a4155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 -- 172.21.15.17:0/2398686405 >> v2:172.21.15.163:3300/0 conn(0x7f16a40aa180 msgr2=0x7f16a40c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 --2- 172.21.15.17:0/2398686405 >> v2:172.21.15.163:3300/0 conn(0x7f16a40aa180 0x7f16a40c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.288+0000 7f16a27fc640 1 -- 172.21.15.17:0/2398686405 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40aa940 con 0x7f16a414ec00 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a27fc640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414ec00 0x7f16a414f090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f16a40c3fc0 tx=0x7f169802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3922a5e53f1b244 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a1ffb640 1 -- 172.21.15.17:0/2398686405 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698038070 con 0x7f16a414ec00 2024-03-06T07:25:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a1ffb640 1 -- 172.21.15.17:0/2398686405 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169802f050 con 0x7f16a414ec00 2024-03-06T07:25:41.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a1ffb640 1 -- 172.21.15.17:0/2398686405 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698031970 con 0x7f16a414ec00 2024-03-06T07:25:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 -- 172.21.15.17:0/2398686405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414ec00 msgr2=0x7f16a414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414ec00 0x7f16a414f090 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f16a40c3fc0 tx=0x7f169802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 -- 172.21.15.17:0/2398686405 shutdown_connections 2024-03-06T07:25:41.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a41550c0 0x7f16a4155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 --2- 172.21.15.17:0/2398686405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a414ec00 0x7f16a414f090 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 --2- 172.21.15.17:0/2398686405 >> v2:172.21.15.163:3300/0 conn(0x7f16a40aa180 0x7f16a40c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.289+0000 7f16a99b6640 1 -- 172.21.15.17:0/2398686405 >> 172.21.15.17:0/2398686405 conn(0x7f16a40c6710 msgr2=0x7f16a40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.290+0000 7f16a99b6640 1 -- 172.21.15.17:0/2398686405 shutdown_connections 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.290+0000 7f16a99b6640 1 -- 172.21.15.17:0/2398686405 wait complete. 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 Processor -- start 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 -- start start 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a414ec00 0x7f16a406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16a41550c0 0x7f16a40c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4155c10 con 0x7f16a40aa180 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4155a90 con 0x7f16a41550c0 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.291+0000 7f16a99b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c36f0 con 0x7f16a414ec00 2024-03-06T07:25:41.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48860/0 (socket says 172.21.15.17:48860) 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 -- 172.21.15.17:0/727605559 learned_addr learned my addr 172.21.15.17:0/727605559 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a27fc640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a414ec00 0x7f16a406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a37fe640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16a41550c0 0x7f16a40c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 -- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a414ec00 msgr2=0x7f16a406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a414ec00 0x7f16a406d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 -- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16a41550c0 msgr2=0x7f16a40c1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16a41550c0 0x7f16a40c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 -- 172.21.15.17:0/727605559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a414c990 con 0x7f16a40aa180 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a2ffd640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f169400d800 tx=0x7f169400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694018030 con 0x7f16a40aa180 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16940096a0 con 0x7f16a40aa180 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694009870 con 0x7f16a40aa180 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a406e300 con 0x7f16a40aa180 2024-03-06T07:25:41.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.292+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a406e020 con 0x7f16a40aa180 2024-03-06T07:25:41.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.293+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a404f330 con 0x7f16a40aa180 2024-03-06T07:25:41.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.294+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169401b070 con 0x7f16a40aa180 2024-03-06T07:25:41.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.295+0000 7f168bfff640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.295+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f169408a6a0 con 0x7f16a40aa180 2024-03-06T07:25:41.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.295+0000 7f16a27fc640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.295+0000 7f16a27fc640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1698035000 tx=0x7f1698008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.299+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f16940592e0 con 0x7f16a40aa180 2024-03-06T07:25:41.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.484+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a40c6b90 con 0x7f16a40aa180 2024-03-06T07:25:41.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.485+0000 7f168bfff640 1 -- 172.21.15.17:0/727605559 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+17155 (secure 0 0 0) 0x7f1694058be0 con 0x7f16a40aa180 2024-03-06T07:25:41.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:41.488 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":138,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:33.047446+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:05.553791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:06.251353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6837/3327948285":"2024-03-07T07:25:18.978040+0000","172.21.15.163:6834/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6835/3327948285":"2024-03-07T07:25:18.978040+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 msgr2=0x7f169006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 0x7f169006d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1698035000 tx=0x7f1698008030 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 msgr2=0x7f16a41eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f169400d800 tx=0x7f169400dcc0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 shutdown_connections 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f169006b4d0 0x7f169006d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16a41550c0 0x7f16a40c1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16a414ec00 0x7f16a406d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 --2- 172.21.15.17:0/727605559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16a40aa180 0x7f16a41eb760 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 >> 172.21.15.17:0/727605559 conn(0x7f16a40c6710 msgr2=0x7f16a41546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 shutdown_connections 2024-03-06T07:25:41.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.488+0000 7f16a99b6640 1 -- 172.21.15.17:0/727605559 wait complete. 2024-03-06T07:25:41.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:25:41.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.602+0000 7fbc8be33640 1 Processor -- start 2024-03-06T07:25:41.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.602+0000 7fbc8be33640 1 -- start start 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8be33640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbc8406ce70 0x7fbc84156e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8be33640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc84072950 con 0x7fbc840c5620 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8be33640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc84072c30 con 0x7fbc84150570 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8be33640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840c4000 con 0x7fbc8406ce70 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8a3d0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbc840c5620 0x7fbc840c5a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40892/0 (socket says 172.21.15.17:40892) 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc8a3d0640 1 -- 172.21.15.17:0/3590439526 learned_addr learned my addr 172.21.15.17:0/3590439526 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.603+0000 7fbc893ce640 1 --2- 172.21.15.17:0/3590439526 >> v2:172.21.15.163:3300/0 conn(0x7fbc8406ce70 0x7fbc84156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174598866 0 0) 0x7fbc84072950 con 0x7fbc840c5620 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc70003520 con 0x7fbc840c5620 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 552255110 0 0) 0x7fbc70003520 con 0x7fbc840c5620 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 >> v2:172.21.15.163:3300/0 conn(0x7fbc8406ce70 msgr2=0x7fbc84156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 --2- 172.21.15.17:0/3590439526 >> v2:172.21.15.163:3300/0 conn(0x7fbc8406ce70 0x7fbc84156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 >> v1:172.21.15.163:6789/0 conn(0x7fbc84150570 legacy=0x7fbc8406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84072f10 con 0x7fbc840c5620 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbc7c0019f0 con 0x7fbc840c5620 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 >> v1:172.21.15.17:6789/0 conn(0x7fbc840c5620 legacy=0x7fbc840c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840c4000 con 0x7fbc700039b0 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000c060 con 0x7fbc700051c0 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7000c260 con 0x7fbc70004800 2024-03-06T07:25:41.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc893ce640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc8a3d0640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 -- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc70004800 msgr2=0x7fbc70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc70004800 0x7fbc70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 -- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700051c0 msgr2=0x7fbc7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700051c0 0x7fbc7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc8a3d0640 1 -- 172.21.15.17:0/3590439526 reap_dead start 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.604+0000 7fbc89bcf640 1 -- 172.21.15.17:0/3590439526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc70003520 con 0x7fbc700039b0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.605+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbc80001750 tx=0x7fbc70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb2ea035977ce9a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80001370 con 0x7fbc700039b0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc800052d0 con 0x7fbc700039b0 2024-03-06T07:25:41.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc88bcd640 1 -- 172.21.15.17:0/3590439526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80003830 con 0x7fbc700039b0 2024-03-06T07:25:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 -- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 msgr2=0x7fbc70003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbc80001750 tx=0x7fbc70005830 comp rx=0 tx=0).stop 2024-03-06T07:25:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 -- 172.21.15.17:0/3590439526 shutdown_connections 2024-03-06T07:25:41.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 --2- 172.21.15.17:0/3590439526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc700039b0 0x7fbc70003dd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 -- 172.21.15.17:0/3590439526 >> 172.21.15.17:0/3590439526 conn(0x7fbc840c4850 msgr2=0x7fbc8406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 -- 172.21.15.17:0/3590439526 shutdown_connections 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.606+0000 7fbc8be33640 1 -- 172.21.15.17:0/3590439526 wait complete. 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.607+0000 7fbc8be33640 1 Processor -- start 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.607+0000 7fbc8be33640 1 -- start start 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 0x7fbc84148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 0x7fbc840ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84150950 con 0x7fbc8414b9b0 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84072620 con 0x7fbc700039b0 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8be33640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84150ad0 con 0x7fbc840ac9a0 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc893ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc893ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48886/0 (socket says 172.21.15.17:48886) 2024-03-06T07:25:41.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc893ce640 1 -- 172.21.15.17:0/1140168976 learned_addr learned my addr 172.21.15.17:0/1140168976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc8a3d0640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 0x7fbc840ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.608+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 0x7fbc84148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 -- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 msgr2=0x7fbc840ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 0x7fbc840ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 -- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 msgr2=0x7fbc84148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 0x7fbc84148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 -- 172.21.15.17:0/1140168976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84149310 con 0x7fbc8414b9b0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc8a3d0640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 0x7fbc840ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 0x7fbc84148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc893ce640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011990 tx=0x7fbc74011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400fad0 con 0x7fbc8414b9b0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7400fc70 con 0x7fbc8414b9b0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc84148b30 con 0x7fbc8414b9b0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74014440 con 0x7fbc8414b9b0 2024-03-06T07:25:41.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.609+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc840aaba0 con 0x7fbc8414b9b0 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.610+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8404f330 con 0x7fbc8414b9b0 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.610+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc74012040 con 0x7fbc8414b9b0 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.611+0000 7fbc7a7fc640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.611+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbc7408a940 con 0x7fbc8414b9b0 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.611+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.611+0000 7fbc89bcf640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 0x7fbc6806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbc80033000 tx=0x7fbc80009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.615+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc84149310 con 0x7fbc8414b9b0 2024-03-06T07:25:41.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.831+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc840a75e0 con 0x7fbc8414b9b0 2024-03-06T07:25:41.836 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:25:41.832+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:25:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.834+0000 7fbc7a7fc640 1 -- 172.21.15.17:0/1140168976 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v131) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc74019030 con 0x7fbc8414b9b0 2024-03-06T07:25:41.837 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 msgr2=0x7fbc6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 0x7fbc6806d9c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbc80033000 tx=0x7fbc80009030 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 msgr2=0x7fbc840a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbc74011990 tx=0x7fbc74011e50 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 shutdown_connections 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc6806b500 0x7fbc6806d9c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc840ac9a0 0x7fbc840ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc8414b9b0 0x7fbc840a7db0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 --2- 172.21.15.17:0/1140168976 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc700039b0 0x7fbc84148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.836+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 >> 172.21.15.17:0/1140168976 conn(0x7fbc840c4850 msgr2=0x7fbc8406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.837+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 shutdown_connections 2024-03-06T07:25:41.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.837+0000 7fbc8be33640 1 -- 172.21.15.17:0/1140168976 wait complete. 2024-03-06T07:25:41.840 INFO:tasks.ceph.mds.c.smithi017.stderr: -95> 2024-03-06T07:25:38.211+0000 7fc4a720e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:25:41.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:25:41.865 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:25:41.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.953+0000 7f689a92b640 1 Processor -- start 2024-03-06T07:25:41.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.953+0000 7f689a92b640 1 -- start start 2024-03-06T07:25:41.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f689a92b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68941538c0 0x7f68941494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f689a92b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f689a92b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6894154330 con 0x7f68940ab310 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f689a92b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68940c46a0 con 0x7f68941538c0 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f689a92b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6894153dc0 con 0x7f68940a97c0 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f6898ec8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f68937fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68941538c0 0x7f68941494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f6898ec8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:48734/0 (socket says 172.21.15.17:48734) 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.954+0000 7f6898ec8640 1 -- 172.21.15.17:0/3805101568 learned_addr learned my addr 172.21.15.17:0/3805101568 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f68937fe640 1 -- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 msgr2=0x7f68941499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f68937fe640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f68937fe640 1 -- 172.21.15.17:0/3805101568 >> v1:172.21.15.163:6789/0 conn(0x7f68940ab310 legacy=0x7f68940ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f68937fe640 1 -- 172.21.15.17:0/3805101568 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940c5d30 con 0x7f68941538c0 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f6892ffd640 1 -- 172.21.15.17:0/3805101568 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638435658 0 0) 0x7f6894154330 con 0x7f68940ab310 2024-03-06T07:25:41.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.955+0000 7f6898ec8640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f68937fe640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68941494a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f688402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9ac9c24bfcf8177 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f6892ffd640 1 -- 172.21.15.17:0/3805101568 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840315a0 con 0x7f68941538c0 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f6892ffd640 1 -- 172.21.15.17:0/3805101568 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884037070 con 0x7f68941538c0 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f6892ffd640 1 -- 172.21.15.17:0/3805101568 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884031960 con 0x7f68941538c0 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 -- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 msgr2=0x7f68941494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68941494a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f688402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 -- 172.21.15.17:0/3805101568 shutdown_connections 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 --2- 172.21.15.17:0/3805101568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68941494a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 -- 172.21.15.17:0/3805101568 >> 172.21.15.17:0/3805101568 conn(0x7f68940c6710 msgr2=0x7f68940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 -- 172.21.15.17:0/3805101568 shutdown_connections 2024-03-06T07:25:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.956+0000 7f689a92b640 1 -- 172.21.15.17:0/3805101568 wait complete. 2024-03-06T07:25:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.957+0000 7f689a92b640 1 Processor -- start 2024-03-06T07:25:41.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 -- start start 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68940713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940c4b30 con 0x7f68940ab310 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940c62f0 con 0x7f68941538c0 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f689a92b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940c4cb0 con 0x7f68940a97c0 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f68937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f68937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48916/0 (socket says 172.21.15.17:48916) 2024-03-06T07:25:41.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.958+0000 7f68937fe640 1 -- 172.21.15.17:0/3426597396 learned_addr learned my addr 172.21.15.17:0/3426597396 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f6898ec8640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68940713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f6893fff640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 -- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 msgr2=0x7f68941e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 -- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 msgr2=0x7f68940713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68940713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 -- 172.21.15.17:0/3426597396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68941453a0 con 0x7f68940ab310 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f68937fe640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f68840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884004230 con 0x7f68940ab310 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68840043d0 con 0x7f68940ab310 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894070350 con 0x7f68940ab310 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884003850 con 0x7f68940ab310 2024-03-06T07:25:41.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.959+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6894070910 con 0x7f68940ab310 2024-03-06T07:25:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.960+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689404f330 con 0x7f68940ab310 2024-03-06T07:25:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.961+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68840039f0 con 0x7f68940ab310 2024-03-06T07:25:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.961+0000 7f6890ff9640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.961+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f688402f030 con 0x7f68940ab310 2024-03-06T07:25:41.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.961+0000 7f6893fff640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:41.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.962+0000 7f6893fff640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f687c000f60 tx=0x7f687c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:41.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:41.966+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6884041070 con 0x7f68940ab310 2024-03-06T07:25:42.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.185+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6894073310 con 0x7f68940ab310 2024-03-06T07:25:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.836+0000 7f6890ff9640 1 -- 172.21.15.17:0/3426597396 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 104+0+0 (secure 0 0 0) 0x7f688407b1e0 con 0x7f68940ab310 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 msgr2=0x7f687406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f687c000f60 tx=0x7f687c009040 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 msgr2=0x7f6894072b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f68840046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 shutdown_connections 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68941538c0 0x7f68940713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68940ab310 0x7f6894072b20 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 --2- 172.21.15.17:0/3426597396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68940a97c0 0x7f68941e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 >> 172.21.15.17:0/3426597396 conn(0x7f68940c6710 msgr2=0x7f68940c16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:42.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 shutdown_connections 2024-03-06T07:25:42.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:42.839+0000 7f689a92b640 1 -- 172.21.15.17:0/3426597396 wait complete. 2024-03-06T07:25:42.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:25:43.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.006+0000 7f058e7d7640 1 Processor -- start 2024-03-06T07:25:43.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f058e7d7640 1 -- start start 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f058e7d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0588151e10 0x7f0588153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f058e7d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058806e850 con 0x7f05880c22b0 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f058e7d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058806d2b0 con 0x7f05880a9830 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f058e7d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058806d590 con 0x7f0588151e10 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f0587fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f05880c22b0 0x7f058814ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40920/0 (socket says 172.21.15.17:40920) 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.007+0000 7f0587fff640 1 -- 172.21.15.17:0/1056479563 learned_addr learned my addr 172.21.15.17:0/1056479563 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:43.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131779888 0 0) 0x7f0578001110 con 0x7f05880c22b0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0570005730 con 0x7f05880c22b0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f058cd74640 1 --2- 172.21.15.17:0/1056479563 >> v2:172.21.15.163:3300/0 conn(0x7f0588151e10 0x7f0588153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3106170549 0 0) 0x7f0570005730 con 0x7f05880c22b0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 >> v2:172.21.15.163:3300/0 conn(0x7f0588151e10 msgr2=0x7f0588153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 --2- 172.21.15.17:0/1056479563 >> v2:172.21.15.163:3300/0 conn(0x7f0588151e10 0x7f0588153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 >> v1:172.21.15.163:6789/0 conn(0x7f05880a9830 legacy=0x7f0588153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058806d870 con 0x7f05880c22b0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0578001840 con 0x7f05880c22b0 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 >> v1:172.21.15.17:6789/0 conn(0x7f05880c22b0 legacy=0x7f058814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0570005bc0 0x7f0570005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f0570006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05700073d0 0x7f057000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058806d590 con 0x7f0570006a10 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057000e320 con 0x7f0570005bc0 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057000e520 con 0x7f05700073d0 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f058cd74640 1 --2- 172.21.15.17:0/1056479563 >> v2:172.21.15.163:3300/0 conn(0x7f0588151e10 0x7f0588153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f05877fe640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f0570006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f0587fff640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0570005bc0 0x7f0570005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f058cd74640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05700073d0 0x7f057000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f05877fe640 1 -- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05700073d0 msgr2=0x7f057000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f05877fe640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05700073d0 0x7f057000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f05877fe640 1 -- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0570005bc0 msgr2=0x7f0570005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.008+0000 7f05877fe640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0570005bc0 0x7f0570005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f05877fe640 1 -- 172.21.15.17:0/1056479563 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570005730 con 0x7f0570006a10 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f058cd74640 1 -- 172.21.15.17:0/1056479563 reap_dead start 2024-03-06T07:25:43.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f05877fe640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f0570006e90 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f057c001750 tx=0x7f0570007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f039993aea236c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c001370 con 0x7f0570006a10 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c0052d0 con 0x7f0570006a10 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.009+0000 7f0586ffd640 1 -- 172.21.15.17:0/1056479563 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c003830 con 0x7f0570006a10 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 -- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 msgr2=0x7f0570006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f0570006e90 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f057c001750 tx=0x7f0570007a40 comp rx=0 tx=0).stop 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 -- 172.21.15.17:0/1056479563 shutdown_connections 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 --2- 172.21.15.17:0/1056479563 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f0570006e90 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 -- 172.21.15.17:0/1056479563 >> 172.21.15.17:0/1056479563 conn(0x7f05880c65b0 msgr2=0x7f05880c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 -- 172.21.15.17:0/1056479563 shutdown_connections 2024-03-06T07:25:43.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.010+0000 7f058e7d7640 1 -- 172.21.15.17:0/1056479563 wait complete. 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.011+0000 7f058e7d7640 1 Processor -- start 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.011+0000 7f058e7d7640 1 -- start start 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05880c4a70 0x7f05880721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05880c4ef0 0x7f0588072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05880c2810 con 0x7f0570006a10 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05880c2690 con 0x7f05880c4a70 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058e7d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058814ef50 con 0x7f05880c4ef0 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48948/0 (socket says 172.21.15.17:48948) 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 -- 172.21.15.17:0/2703688512 learned_addr learned my addr 172.21.15.17:0/2703688512 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f058cd74640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05880c4ef0 0x7f0588072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 -- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05880c4ef0 msgr2=0x7f0588072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05880c4ef0 0x7f0588072720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 -- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05880c4a70 msgr2=0x7f05880721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.012+0000 7f0587fff640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05880c4a70 0x7f05880721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f0587fff640 1 -- 172.21.15.17:0/2703688512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05880c1bb0 con 0x7f0570006a10 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f0587fff640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0578007310 tx=0x7f05780077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057801a070 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057800f8e0 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05881e9bf0 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800fcb0 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.013+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05881ea6d0 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.014+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05881e99e0 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.014+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f057801d070 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.015+0000 7f0584ff9640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 0x7f057406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.015+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f0578089c90 con 0x7f0570006a10 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.015+0000 7f05877fe640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 0x7f057406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:43.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.015+0000 7f05877fe640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 0x7f057406d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f057c033000 tx=0x7f057c02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:43.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.019+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f058806e850 con 0x7f0570006a10 2024-03-06T07:25:43.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:43.201+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f058804f330 con 0x7f0570006a10 2024-03-06T07:25:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.089+0000 7f0584ff9640 1 -- 172.21.15.17:0/2703688512 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0578058910 con 0x7f0570006a10 2024-03-06T07:25:44.091 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:25:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.093+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 msgr2=0x7f057406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.093+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 0x7f057406d9c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f057c033000 tx=0x7f057c02f000 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.093+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 msgr2=0x7f05880c1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.093+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f0578007310 tx=0x7f05780077d0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 shutdown_connections 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f057406b500 0x7f057406d9c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05880c4ef0 0x7f0588072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f05880c4a70 0x7f05880721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 --2- 172.21.15.17:0/2703688512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0570006a10 0x7f05880c1440 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 >> 172.21.15.17:0/2703688512 conn(0x7f05880c65b0 msgr2=0x7f0588154760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 shutdown_connections 2024-03-06T07:25:44.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.094+0000 7f058e7d7640 1 -- 172.21.15.17:0/2703688512 wait complete. 2024-03-06T07:25:44.109 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:25:44.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.219+0000 7fb0925b5640 1 Processor -- start 2024-03-06T07:25:44.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.219+0000 7fb0925b5640 1 -- start start 2024-03-06T07:25:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.219+0000 7fb0925b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb08c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb0925b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c156150 con 0x7fb08c158780 2024-03-06T07:25:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb0925b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0a7d70 con 0x7fb08c0c6780 2024-03-06T07:25:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb0925b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0a8050 con 0x7fb08c06fe20 2024-03-06T07:25:44.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb090b52640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb08c158780 0x7fb08c158bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40926/0 (socket says 172.21.15.17:40926) 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb090b52640 1 -- 172.21.15.17:0/4220232543 learned_addr learned my addr 172.21.15.17:0/4220232543 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904327666 0 0) 0x7fb084001490 con 0x7fb08c158780 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08bfff640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb08c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb074003520 con 0x7fb08c158780 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133431926 0 0) 0x7fb074003520 con 0x7fb08c158780 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 msgr2=0x7fb08c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb08c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 >> v1:172.21.15.163:6789/0 conn(0x7fb08c0c6780 legacy=0x7fb08c072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c0a8330 con 0x7fb08c158780 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb084001bc0 con 0x7fb08c158780 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 >> v1:172.21.15.17:6789/0 conn(0x7fb08c158780 legacy=0x7fb08c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb0740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb074003fb0 0x7fb074004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0a8050 con 0x7fb074004d40 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07400bbb0 con 0x7fb074003fb0 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07400bdb0 con 0x7fb08c06fe20 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.220+0000 7fb090b52640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb08b7fe640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb074003fb0 0x7fb074004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb08bfff640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb0740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb090b52640 1 -- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 msgr2=0x7fb0740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb090b52640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c06fe20 0x7fb0740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb090b52640 1 -- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb074003fb0 msgr2=0x7fb074004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb090b52640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb074003fb0 0x7fb074004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.221+0000 7fb090b52640 1 -- 172.21.15.17:0/4220232543 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074003520 con 0x7fb074004d40 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb090b52640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb08400a950 tx=0x7fb074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b95b7ff9695405 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08400a570 con 0x7fb074004d40 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb084007310 con 0x7fb074004d40 2024-03-06T07:25:44.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb08affd640 1 -- 172.21.15.17:0/4220232543 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084007700 con 0x7fb074004d40 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 -- 172.21.15.17:0/4220232543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb074004d40 msgr2=0x7fb0740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 --2- 172.21.15.17:0/4220232543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb074004d40 0x7fb0740051c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb08400a950 tx=0x7fb074005830 comp rx=0 tx=0).stop 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb090b52640 1 -- 172.21.15.17:0/4220232543 reap_dead start 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 -- 172.21.15.17:0/4220232543 shutdown_connections 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 -- 172.21.15.17:0/4220232543 >> 172.21.15.17:0/4220232543 conn(0x7fb08c0aa630 msgr2=0x7fb08c0aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 -- 172.21.15.17:0/4220232543 shutdown_connections 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.222+0000 7fb0925b5640 1 -- 172.21.15.17:0/4220232543 wait complete. 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.223+0000 7fb0925b5640 1 Processor -- start 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 -- start start 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c1ee690 0x7fb08c1eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb08c1e1360 0x7fb08c0ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c070200 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0c3600 con 0x7fb08c1e1360 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb0925b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c072c20 con 0x7fb08c1ee690 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb090b52640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb090b52640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48960/0 (socket says 172.21.15.17:48960) 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.224+0000 7fb090b52640 1 -- 172.21.15.17:0/1951860245 learned_addr learned my addr 172.21.15.17:0/1951860245 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb08b7fe640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb08c1e1360 0x7fb08c0ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb08bfff640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c1ee690 0x7fb08c1eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 -- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c1ee690 msgr2=0x7fb08c1eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c1ee690 0x7fb08c1eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 -- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb08c1e1360 msgr2=0x7fb08c0ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb08c1e1360 0x7fb08c0ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 -- 172.21.15.17:0/1951860245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c0ae5f0 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb090b52640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb08402cd60 tx=0x7fb08c155190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084045070 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08403a040 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c0ae8d0 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08400a570 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.225+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c0aad20 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.226+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c04f330 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.227+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb084048070 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.227+0000 7fb088ff9640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 0x7fb07806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.227+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5921+0+0 (secure 0 0 0) 0x7fb0840abac0 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.227+0000 7fb08bfff640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 0x7fb07806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:44.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.228+0000 7fb08bfff640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 0x7fb07806d9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb08c06dcd0 tx=0x7fb07c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:44.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.231+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb08c156150 con 0x7fb08c0ac8e0 2024-03-06T07:25:44.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:44.413+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb08c0ac100 con 0x7fb08c0ac8e0 2024-03-06T07:25:45.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.091+0000 7fb088ff9640 1 -- 172.21.15.17:0/1951860245 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb08407a360 con 0x7fb08c0ac8e0 2024-03-06T07:25:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 msgr2=0x7fb07806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 0x7fb07806d9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb08c06dcd0 tx=0x7fb07c009190 comp rx=0 tx=0).stop 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 msgr2=0x7fb08c14c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb08402cd60 tx=0x7fb08c155190 comp rx=0 tx=0).stop 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 shutdown_connections 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb07806b500 0x7fb07806d9c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb08c0ac8e0 0x7fb08c14c400 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb08c1e1360 0x7fb08c0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 --2- 172.21.15.17:0/1951860245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb08c1ee690 0x7fb08c1eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 >> 172.21.15.17:0/1951860245 conn(0x7fb08c0aa630 msgr2=0x7fb08c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.095+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 shutdown_connections 2024-03-06T07:25:45.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.096+0000 7fb0925b5640 1 -- 172.21.15.17:0/1951860245 wait complete. 2024-03-06T07:25:45.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.b mds_join_fs 2024-03-06T07:25:45.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.220+0000 7fd9e82a4640 1 Processor -- start 2024-03-06T07:25:45.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.220+0000 7fd9e82a4640 1 -- start start 2024-03-06T07:25:45.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e82a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9e006afc0 0x7fd9e006b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e82a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00c5840 con 0x7fd9e00c2120 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e82a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00c5b20 con 0x7fd9e00c2580 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e82a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e0158790 con 0x7fd9e006afc0 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e583f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd9e00c2120 0x7fd9e0158c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40938/0 (socket says 172.21.15.17:40938) 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e583f640 1 -- 172.21.15.17:0/3063194682 learned_addr learned my addr 172.21.15.17:0/3063194682 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988634928 0 0) 0x7fd9e00c5840 con 0x7fd9e00c2120 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9cc003520 con 0x7fd9e00c2120 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e6040640 1 --2- 172.21.15.17:0/3063194682 >> v2:172.21.15.163:3300/0 conn(0x7fd9e006afc0 0x7fd9e006b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3931870582 0 0) 0x7fd9cc003520 con 0x7fd9e00c2120 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 >> v2:172.21.15.163:3300/0 conn(0x7fd9e006afc0 msgr2=0x7fd9e006b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.221+0000 7fd9e503e640 1 --2- 172.21.15.17:0/3063194682 >> v2:172.21.15.163:3300/0 conn(0x7fd9e006afc0 0x7fd9e006b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 >> v1:172.21.15.163:6789/0 conn(0x7fd9e00c2580 legacy=0x7fd9e015fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00c5e00 con 0x7fd9e00c2120 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd9d0001670 con 0x7fd9e00c2120 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6040640 1 --2- 172.21.15.17:0/3063194682 >> v2:172.21.15.163:3300/0 conn(0x7fd9e006afc0 0x7fd9e006b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 >> v1:172.21.15.17:6789/0 conn(0x7fd9e00c2120 legacy=0x7fd9e0158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9cc0039b0 0x7fd9cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9cc004800 0x7fd9cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e0158790 con 0x7fd9cc0052e0 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00c1b0 con 0x7fd9cc004800 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00c3b0 con 0x7fd9cc0039b0 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6040640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9cc0039b0 0x7fd9cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e583f640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9cc004800 0x7fd9cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 -- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9cc0039b0 msgr2=0x7fd9cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9cc0039b0 0x7fd9cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 -- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9cc004800 msgr2=0x7fd9cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9cc004800 0x7fd9cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.222+0000 7fd9e6841640 1 -- 172.21.15.17:0/3063194682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc003520 con 0x7fd9cc0052e0 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e6841640 1 -- 172.21.15.17:0/3063194682 reap_dead start 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e6841640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9cc00b9c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00c6580 tx=0x7fd9cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81bd3795546913 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc001a70 con 0x7fd9cc0052e0 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc003b70 con 0x7fd9cc0052e0 2024-03-06T07:25:45.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e503e640 1 -- 172.21.15.17:0/3063194682 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0050d0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e82a4640 1 -- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 msgr2=0x7fd9cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.223+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9cc00b9c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00c6580 tx=0x7fd9cc005830 comp rx=0 tx=0).stop 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.224+0000 7fd9e82a4640 1 -- 172.21.15.17:0/3063194682 shutdown_connections 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.224+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/3063194682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9cc00b9c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.224+0000 7fd9e82a4640 1 -- 172.21.15.17:0/3063194682 >> 172.21.15.17:0/3063194682 conn(0x7fd9e00c4220 msgr2=0x7fd9e00c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.224+0000 7fd9e82a4640 1 -- 172.21.15.17:0/3063194682 shutdown_connections 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.224+0000 7fd9e82a4640 1 -- 172.21.15.17:0/3063194682 wait complete. 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 Processor -- start 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 -- start start 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00a8310 0x7fd9e00a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 0x7fd9e01eb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c2b30 con 0x7fd9cc0052e0 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e0155220 con 0x7fd9e01eb090 2024-03-06T07:25:45.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.225+0000 7fd9e82a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c2cb0 con 0x7fd9e00a8310 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48974/0 (socket says 172.21.15.17:48974) 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 -- 172.21.15.17:0/1179048405 learned_addr learned my addr 172.21.15.17:0/1179048405 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 -- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00a8310 msgr2=0x7fd9e00a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e583f640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00a8310 0x7fd9e00a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6841640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 0x7fd9e01eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00a8310 0x7fd9e00a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 -- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 msgr2=0x7fd9e01eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 0x7fd9e01eb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 -- 172.21.15.17:0/1179048405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e006e5b0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6841640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 0x7fd9e01eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e6040640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8011590 tx=0x7fd9d8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8019070 con 0x7fd9cc0052e0 2024-03-06T07:25:45.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8004220 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e0071470 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800f040 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.226+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00719c0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.227+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e006b3a0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.228+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9d8015670 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.228+0000 7fd9d6ffd640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 0x7fd9c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.228+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd9d808a2f0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.228+0000 7fd9e583f640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 0x7fd9c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.229+0000 7fd9e583f640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 0x7fd9c406d9c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd9e01e8ed0 tx=0x7fd9d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.232+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd9e006b3a0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.421+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"} v 0) v1 -- 0x7fd9e004f330 con 0x7fd9cc0052e0 2024-03-06T07:25:45.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.422+0000 7fd9d6ffd640 1 -- 172.21.15.17:0/1179048405 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.b", "name": "mds_join_fs"}]=0 v48) v1 ==== 96+0+0 (secure 0 0 0) 0x7fd9d80592c0 con 0x7fd9cc0052e0 2024-03-06T07:25:45.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.423+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 msgr2=0x7fd9c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.423+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 0x7fd9c406d9c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd9e01e8ed0 tx=0x7fd9d0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 msgr2=0x7fd9e014d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8011590 tx=0x7fd9d8011a50 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 shutdown_connections 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd9c406b500 0x7fd9c406d9c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e01eb090 0x7fd9e01eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00a8310 0x7fd9e00a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 --2- 172.21.15.17:0/1179048405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9cc0052e0 0x7fd9e014d810 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 >> 172.21.15.17:0/1179048405 conn(0x7fd9e00c4220 msgr2=0x7fd9e0153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 shutdown_connections 2024-03-06T07:25:45.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.424+0000 7fd9e82a4640 1 -- 172.21.15.17:0/1179048405 wait complete. 2024-03-06T07:25:45.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.c mds_join_fs 2024-03-06T07:25:45.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.537+0000 7fb07039a640 1 Processor -- start 2024-03-06T07:25:45.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.537+0000 7fb07039a640 1 -- start start 2024-03-06T07:25:45.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb07039a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0681527f0 0x7fb0680c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb07039a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 0x7fb068155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb07039a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0681596f0 con 0x7fb0680c88f0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb07039a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680c9130 con 0x7fb0681527f0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb07039a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb068159260 con 0x7fb0680c58b0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb06e136640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0681527f0 0x7fb0680c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb06e136640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0681527f0 0x7fb0680c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48986/0 (socket says 172.21.15.17:48986) 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.538+0000 7fb06e136640 1 -- 172.21.15.17:0/1772044063 learned_addr learned my addr 172.21.15.17:0/1772044063 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e937640 1 --2- 172.21.15.17:0/1772044063 >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 0x7fb068155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e136640 1 -- 172.21.15.17:0/1772044063 >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 msgr2=0x7fb068155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e136640 1 --2- 172.21.15.17:0/1772044063 >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 0x7fb068155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e136640 1 -- 172.21.15.17:0/1772044063 >> v1:172.21.15.163:6789/0 conn(0x7fb0680c88f0 legacy=0x7fb068155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e136640 1 -- 172.21.15.17:0/1772044063 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680c82a0 con 0x7fb0681527f0 2024-03-06T07:25:45.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e937640 1 --2- 172.21.15.17:0/1772044063 >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 0x7fb068155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.539+0000 7fb06e136640 1 --2- 172.21.15.17:0/1772044063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0681527f0 0x7fb0680c5370 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb0680c4cf0 tx=0x7fb06402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1139d9a31517a570 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb06d134640 1 -- 172.21.15.17:0/1772044063 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06402e070 con 0x7fb0681527f0 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb06d134640 1 -- 172.21.15.17:0/1772044063 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064037070 con 0x7fb0681527f0 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb06d134640 1 -- 172.21.15.17:0/1772044063 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064031a00 con 0x7fb0681527f0 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 -- 172.21.15.17:0/1772044063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0681527f0 msgr2=0x7fb0680c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 --2- 172.21.15.17:0/1772044063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0681527f0 0x7fb0680c5370 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fb0680c4cf0 tx=0x7fb06402b880 comp rx=0 tx=0).stop 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 -- 172.21.15.17:0/1772044063 shutdown_connections 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 --2- 172.21.15.17:0/1772044063 >> v2:172.21.15.163:3300/0 conn(0x7fb0680c58b0 0x7fb068155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 --2- 172.21.15.17:0/1772044063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0681527f0 0x7fb0680c5370 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 -- 172.21.15.17:0/1772044063 >> 172.21.15.17:0/1772044063 conn(0x7fb0680ac580 msgr2=0x7fb0680ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 -- 172.21.15.17:0/1772044063 shutdown_connections 2024-03-06T07:25:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.540+0000 7fb07039a640 1 -- 172.21.15.17:0/1772044063 wait complete. 2024-03-06T07:25:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.541+0000 7fb07039a640 1 Processor -- start 2024-03-06T07:25:45.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.541+0000 7fb07039a640 1 -- start start 2024-03-06T07:25:45.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680c88f0 0x7fb0681ead90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 0x7fb0681eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068158c10 con 0x7fb0680c58b0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c6b90 con 0x7fb0681527f0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb07039a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb068158ef0 con 0x7fb0680c88f0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48992/0 (socket says 172.21.15.17:48992) 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 -- 172.21.15.17:0/3555588929 learned_addr learned my addr 172.21.15.17:0/3555588929 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e937640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 0x7fb0681eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06d935640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680c88f0 0x7fb0681ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 -- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680c88f0 msgr2=0x7fb0681ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680c88f0 0x7fb0681ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 -- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 msgr2=0x7fb0681eb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 0x7fb0681eb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e136640 1 -- 172.21.15.17:0/3555588929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0681ef1f0 con 0x7fb0680c58b0 2024-03-06T07:25:45.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.542+0000 7fb06e937640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 0x7fb0681eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb06e136640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb064000c00 tx=0x7fb0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064032670 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064032850 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0681ee430 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064032c40 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.543+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681ee9f0 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.544+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06804f330 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.544+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb06404c060 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.545+0000 7fb05effd640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 0x7fb04c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.545+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb0640abb00 con 0x7fb0680c58b0 2024-03-06T07:25:45.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.545+0000 7fb06d935640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 0x7fb04c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.545+0000 7fb06d935640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 0x7fb04c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb058007fd0 tx=0x7fb058009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.549+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb064040070 con 0x7fb0680c58b0 2024-03-06T07:25:45.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.731+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"} v 0) v1 -- 0x7fb0680b0930 con 0x7fb0680c58b0 2024-03-06T07:25:45.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.734+0000 7fb05effd640 1 -- 172.21.15.17:0/3555588929 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.c", "name": "mds_join_fs"}]=0 v49) v1 ==== 96+0+0 (secure 0 0 0) 0x7fb06407aad0 con 0x7fb0680c58b0 2024-03-06T07:25:45.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 msgr2=0x7fb04c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 0x7fb04c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb058007fd0 tx=0x7fb058009040 comp rx=0 tx=0).stop 2024-03-06T07:25:45.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 msgr2=0x7fb068153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb064000c00 tx=0x7fb0640046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 shutdown_connections 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb04c06b4d0 0x7fb04c06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0681527f0 0x7fb0681eb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680c88f0 0x7fb0681ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 --2- 172.21.15.17:0/3555588929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0680c58b0 0x7fb068153770 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 >> 172.21.15.17:0/3555588929 conn(0x7fb0680ac580 msgr2=0x7fb0680ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 shutdown_connections 2024-03-06T07:25:45.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.736+0000 7fb07039a640 1 -- 172.21.15.17:0/3555588929 wait complete. 2024-03-06T07:25:45.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:25:45.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.852+0000 7f8ce47c9640 1 Processor -- start 2024-03-06T07:25:45.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.852+0000 7f8ce47c9640 1 -- start start 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce47c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cdc152730 0x7f8cdc0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce47c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc152b10 con 0x7f8cdc0c3750 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce47c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc158ad0 con 0x7f8cdc0c6710 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce47c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc158db0 con 0x7f8cdc152730 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2565640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cdc152730 0x7f8cdc0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2d66640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8cdc0c3750 0x7f8cdc15fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40944/0 (socket says 172.21.15.17:40944) 2024-03-06T07:25:45.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2d66640 1 -- 172.21.15.17:0/1816176051 learned_addr learned my addr 172.21.15.17:0/1816176051 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2565640 1 -- 172.21.15.17:0/1816176051 >> v1:172.21.15.163:6789/0 conn(0x7f8cdc0c6710 legacy=0x7f8cdc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2565640 1 -- 172.21.15.17:0/1816176051 >> v1:172.21.15.17:6789/0 conn(0x7f8cdc0c3750 legacy=0x7f8cdc15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce2565640 1 -- 172.21.15.17:0/1816176051 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc159090 con 0x7f8cdc152730 2024-03-06T07:25:45.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.853+0000 7f8ce1563640 1 -- 172.21.15.17:0/1816176051 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898512446 0 0) 0x7f8cdc152b10 con 0x7f8cdc0c3750 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.854+0000 7f8ce2565640 1 --2- 172.21.15.17:0/1816176051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc152730 0x7f8cdc0c3210 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8c33404ac2ea28 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.854+0000 7f8ce1563640 1 -- 172.21.15.17:0/1816176051 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4039070 con 0x7f8cdc152730 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.854+0000 7f8ce1563640 1 -- 172.21.15.17:0/1816176051 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd402cbd0 con 0x7f8cdc152730 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.854+0000 7f8ce1563640 1 -- 172.21.15.17:0/1816176051 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4034440 con 0x7f8cdc152730 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 -- 172.21.15.17:0/1816176051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc152730 msgr2=0x7f8cdc0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/1816176051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc152730 0x7f8cdc0c3210 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009d90 tx=0x7f8cd402be20 comp rx=0 tx=0).stop 2024-03-06T07:25:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 -- 172.21.15.17:0/1816176051 shutdown_connections 2024-03-06T07:25:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/1816176051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc152730 0x7f8cdc0c3210 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 -- 172.21.15.17:0/1816176051 >> 172.21.15.17:0/1816176051 conn(0x7f8cdc0aa5c0 msgr2=0x7f8cdc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 -- 172.21.15.17:0/1816176051 shutdown_connections 2024-03-06T07:25:45.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.855+0000 7f8ce47c9640 1 -- 172.21.15.17:0/1816176051 wait complete. 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.856+0000 7f8ce47c9640 1 Processor -- start 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.856+0000 7f8ce47c9640 1 -- start start 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cdc0c3750 0x7f8cdc06ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 0x7f8cdc157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc158870 con 0x7f8cdc0c6710 2024-03-06T07:25:45.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1586f0 con 0x7f8cdc152730 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce47c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0a7cd0 con 0x7f8cdc0c3750 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49004/0 (socket says 172.21.15.17:49004) 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 -- 172.21.15.17:0/3436582850 learned_addr learned my addr 172.21.15.17:0/3436582850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 -- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cdc0c3750 msgr2=0x7f8cdc06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cdc0c3750 0x7f8cdc06ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce2d66640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 0x7f8cdc157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 -- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 msgr2=0x7f8cdc157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 0x7f8cdc157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 -- 172.21.15.17:0/3436582850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc06d950 con 0x7f8cdc0c6710 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce2d66640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 0x7f8cdc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8ce1d64640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c430 tx=0x7f8ccc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011790 con 0x7f8cdc0c6710 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.857+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc017070 con 0x7f8cdc0c6710 2024-03-06T07:25:45.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.858+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cdc155bf0 con 0x7f8cdc0c6710 2024-03-06T07:25:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.858+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011b70 con 0x7f8cdc0c6710 2024-03-06T07:25:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.858+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc06c4e0 con 0x7f8cdc0c6710 2024-03-06T07:25:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.858+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc04f330 con 0x7f8cdc0c6710 2024-03-06T07:25:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.859+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ccc011d10 con 0x7f8cdc0c6710 2024-03-06T07:25:45.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.859+0000 7f8cd2ffd640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 0x7f8cc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.859+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8ccc089f20 con 0x7f8cdc0c6710 2024-03-06T07:25:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.860+0000 7f8ce2565640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 0x7f8cc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.860+0000 7f8ce2565640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 0x7f8cc006d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4036000 tx=0x7f8cd402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:45.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:45.863+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8ccc058ef0 con 0x7f8cdc0c6710 2024-03-06T07:25:46.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.045+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8cdc073330 con 0x7f8cdc0c6710 2024-03-06T07:25:46.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.048+0000 7f8cd2ffd640 1 -- 172.21.15.17:0/3436582850 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8ccc0587f0 con 0x7f8cdc0c6710 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 msgr2=0x7f8cc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 0x7f8cc006d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4036000 tx=0x7f8cd402e000 comp rx=0 tx=0).stop 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 msgr2=0x7f8cdc06d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c430 tx=0x7f8ccc00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 shutdown_connections 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8cc006b4d0 0x7f8cc006d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cdc152730 0x7f8cdc157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cdc0c6710 0x7f8cdc06d410 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 --2- 172.21.15.17:0/3436582850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cdc0c3750 0x7f8cdc06ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 >> 172.21.15.17:0/3436582850 conn(0x7f8cdc0aa5c0 msgr2=0x7f8cdc0c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:46.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 shutdown_connections 2024-03-06T07:25:46.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.050+0000 7f8ce47c9640 1 -- 172.21.15.17:0/3436582850 wait complete. 2024-03-06T07:25:46.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset' 2024-03-06T07:25:46.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.163+0000 7f4c9eca4640 1 Processor -- start 2024-03-06T07:25:46.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.163+0000 7f4c9eca4640 1 -- start start 2024-03-06T07:25:46.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.163+0000 7f4c9eca4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9eca4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c980720a0 0x7f4c9806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9eca4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c980c1ae0 con 0x7f4c98155c80 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9eca4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980c1dc0 con 0x7f4c980720a0 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9eca4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9806e450 con 0x7f4c98152520 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9ca40640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9d241640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c980720a0 0x7f4c9806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9ca40640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:48828/0 (socket says 172.21.15.17:48828) 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9ca40640 1 -- 172.21.15.17:0/2879854767 learned_addr learned my addr 172.21.15.17:0/2879854767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c97fff640 1 --1- 172.21.15.17:0/2879854767 >> v1:172.21.15.163:6789/0 conn(0x7f4c98155c80 0x7f4c980c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:34938/0 (socket says 172.21.15.17:34938) 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9d241640 1 -- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 msgr2=0x7f4c9806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9d241640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9d241640 1 -- 172.21.15.17:0/2879854767 >> v1:172.21.15.163:6789/0 conn(0x7f4c98155c80 legacy=0x7f4c980c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9d241640 1 -- 172.21.15.17:0/2879854767 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9806e730 con 0x7f4c980720a0 2024-03-06T07:25:46.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.164+0000 7f4c9ca40640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:46.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.165+0000 7f4c9d241640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c980720a0 0x7f4c9806afc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009d90 tx=0x7f4c9002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e243dfa2233359c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.165+0000 7f4c977fe640 1 -- 172.21.15.17:0/2879854767 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90039070 con 0x7f4c980720a0 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.165+0000 7f4c977fe640 1 -- 172.21.15.17:0/2879854767 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9002cc50 con 0x7f4c980720a0 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.165+0000 7f4c977fe640 1 -- 172.21.15.17:0/2879854767 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90034440 con 0x7f4c980720a0 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 -- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c980720a0 msgr2=0x7f4c9806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c980720a0 0x7f4c9806afc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009d90 tx=0x7f4c9002bea0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 -- 172.21.15.17:0/2879854767 shutdown_connections 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c980720a0 0x7f4c9806afc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/2879854767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c98152520 0x7f4c9806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 -- 172.21.15.17:0/2879854767 >> 172.21.15.17:0/2879854767 conn(0x7f4c980c76b0 msgr2=0x7f4c980c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 -- 172.21.15.17:0/2879854767 shutdown_connections 2024-03-06T07:25:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.166+0000 7f4c9eca4640 1 -- 172.21.15.17:0/2879854767 wait complete. 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.167+0000 7f4c9eca4640 1 Processor -- start 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.167+0000 7f4c9eca4640 1 -- start start 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 0x7f4c98158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c98155c80 0x7f4c9814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98152a80 con 0x7f4c98152520 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98152900 con 0x7f4c98155c80 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9eca4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9806b500 con 0x7f4c980720a0 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49018/0 (socket says 172.21.15.17:49018) 2024-03-06T07:25:46.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 -- 172.21.15.17:0/1631781579 learned_addr learned my addr 172.21.15.17:0/1631781579 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:46.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9ca40640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 0x7f4c98158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9d241640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c98155c80 0x7f4c9814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 -- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 msgr2=0x7f4c98158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 0x7f4c98158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 -- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c98155c80 msgr2=0x7f4c9814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c98155c80 0x7f4c9814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 -- 172.21.15.17:0/1631781579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980ad680 con 0x7f4c98152520 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c9ca40640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 0x7f4c98158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.168+0000 7f4c97fff640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c400 tx=0x7f4c8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.169+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840074e0 con 0x7f4c98152520 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.169+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84007680 con 0x7f4c98152520 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.169+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84004a00 con 0x7f4c98152520 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.169+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c98156d40 con 0x7f4c98152520 2024-03-06T07:25:46.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.169+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c98159410 con 0x7f4c98152520 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.170+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c980ad850 con 0x7f4c98152520 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.170+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c8401a020 con 0x7f4c98152520 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.170+0000 7f4c957fa640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 0x7f4c7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.170+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f4c84089960 con 0x7f4c98152520 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.171+0000 7f4c9ca40640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 0x7f4c7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.171+0000 7f4c9ca40640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 0x7f4c7c06d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009630 tx=0x7f4c8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.175+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4c84010300 con 0x7f4c98152520 2024-03-06T07:25:46.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:46.425+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]} v 0) v1 -- 0x7f4c9804f330 con 0x7f4c98152520 2024-03-06T07:25:47.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.109+0000 7f4c957fa640 1 -- 172.21.15.17:0/1631781579 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_unset"]}]=0 v338) v1 ==== 144+0+0 (secure 0 0 0) 0x7f4c840588a0 con 0x7f4c98152520 2024-03-06T07:25:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 msgr2=0x7f4c7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 0x7f4c7c06d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009630 tx=0x7f4c8c008040 comp rx=0 tx=0).stop 2024-03-06T07:25:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 msgr2=0x7f4c98156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400c400 tx=0x7f4c8400c8c0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 shutdown_connections 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4c7c06b500 0x7f4c7c06d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c98155c80 0x7f4c9814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.112+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4c98152520 0x7f4c98156350 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.113+0000 7f4c9eca4640 1 --2- 172.21.15.17:0/1631781579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c980720a0 0x7f4c98158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.113+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 >> 172.21.15.17:0/1631781579 conn(0x7f4c980c76b0 msgr2=0x7f4c98150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.113+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 shutdown_connections 2024-03-06T07:25:47.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.113+0000 7f4c9eca4640 1 -- 172.21.15.17:0/1631781579 wait complete. 2024-03-06T07:25:47.127 INFO:tasks.cephfs_test_runner:That unsetting mds_join_fs will cause failover if another high-affinity standby exists. ... ok 2024-03-06T07:25:47.128 INFO:tasks.cephfs_test_runner:Starting test: test_join_fs_vanilla (tasks.cephfs.test_failover.TestClusterAffinity) That a vanilla standby is preferred over others with mds_join_fs set to another fs. 2024-03-06T07:25:47.128 INFO:tasks.cephfs_test_runner:test_join_fs_vanilla (tasks.cephfs.test_failover.TestClusterAffinity) 2024-03-06T07:25:47.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:47.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.234+0000 7fbfe995c640 1 Processor -- start 2024-03-06T07:25:47.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.234+0000 7fbfe995c640 1 -- start start 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe995c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfe406d9f0 0x7fbfe40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe995c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbfe40c3280 0x7fbfe40c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe995c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe4158800 con 0x7fbfe41551e0 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe995c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe4158ae0 con 0x7fbfe40c3280 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe995c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe4158dc0 con 0x7fbfe406d9f0 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbfe40c3280 0x7fbfe40c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbfe40c3280 0x7fbfe40c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49034/0 (socket says 172.21.15.17:49034) 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 -- 172.21.15.17:0/3800078016 learned_addr learned my addr 172.21.15.17:0/3800078016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:47.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 -- 172.21.15.17:0/3800078016 >> v2:172.21.15.163:3300/0 conn(0x7fbfe406d9f0 msgr2=0x7fbfe40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/3800078016 >> v2:172.21.15.163:3300/0 conn(0x7fbfe406d9f0 0x7fbfe40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 -- 172.21.15.17:0/3800078016 >> v1:172.21.15.163:6789/0 conn(0x7fbfe41551e0 legacy=0x7fbfe41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.235+0000 7fbfe27fc640 1 -- 172.21.15.17:0/3800078016 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe41590a0 con 0x7fbfe40c3280 2024-03-06T07:25:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.236+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/3800078016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40c3280 0x7fbfe40c3710 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40c9820 tx=0x7fbfcc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5530bf0791d96ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.236+0000 7fbfe1ffb640 1 -- 172.21.15.17:0/3800078016 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc02e070 con 0x7fbfe40c3280 2024-03-06T07:25:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.236+0000 7fbfe1ffb640 1 -- 172.21.15.17:0/3800078016 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc037070 con 0x7fbfe40c3280 2024-03-06T07:25:47.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.236+0000 7fbfe1ffb640 1 -- 172.21.15.17:0/3800078016 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc031a00 con 0x7fbfe40c3280 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 -- 172.21.15.17:0/3800078016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40c3280 msgr2=0x7fbfe40c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 --2- 172.21.15.17:0/3800078016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40c3280 0x7fbfe40c3710 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40c9820 tx=0x7fbfcc02b880 comp rx=0 tx=0).stop 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 -- 172.21.15.17:0/3800078016 shutdown_connections 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 --2- 172.21.15.17:0/3800078016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40c3280 0x7fbfe40c3710 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 --2- 172.21.15.17:0/3800078016 >> v2:172.21.15.163:3300/0 conn(0x7fbfe406d9f0 0x7fbfe40a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 -- 172.21.15.17:0/3800078016 >> 172.21.15.17:0/3800078016 conn(0x7fbfe40aa630 msgr2=0x7fbfe40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 -- 172.21.15.17:0/3800078016 shutdown_connections 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.237+0000 7fbfe995c640 1 -- 172.21.15.17:0/3800078016 wait complete. 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.238+0000 7fbfe995c640 1 Processor -- start 2024-03-06T07:25:47.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.238+0000 7fbfe995c640 1 -- start start 2024-03-06T07:25:47.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 0x7fbfe414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 0x7fbfe414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe406df50 con 0x7fbfe406d9f0 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe406ddd0 con 0x7fbfe41551e0 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40a8210 con 0x7fbfe40c3280 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49046/0 (socket says 172.21.15.17:49046) 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 -- 172.21.15.17:0/717869615 learned_addr learned my addr 172.21.15.17:0/717869615 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:47.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe37fe640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 0x7fbfe414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 0x7fbfe414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 -- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 msgr2=0x7fbfe414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 0x7fbfe414d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 -- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 msgr2=0x7fbfe414dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 0x7fbfe414dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 -- 172.21.15.17:0/717869615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4158230 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe37fe640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 0x7fbfe414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 0x7fbfe414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe2ffd640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b4d0 tx=0x7fbfdc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc014070 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc010870 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe41570c0 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc010c60 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.239+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe414e400 con 0x7fbfe406d9f0 2024-03-06T07:25:47.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.240+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4156ed0 con 0x7fbfe406d9f0 2024-03-06T07:25:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.241+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfdc017070 con 0x7fbfe406d9f0 2024-03-06T07:25:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.241+0000 7fbfdbfff640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 0x7fbfc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.241+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbfdc089e60 con 0x7fbfe406d9f0 2024-03-06T07:25:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.242+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 0x7fbfc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.242+0000 7fbfe27fc640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 0x7fbfc406d9c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036000 tx=0x7fbfcc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.246+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbfdc058e30 con 0x7fbfe406d9f0 2024-03-06T07:25:47.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.427+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfe404f330 con 0x7fbfe406d9f0 2024-03-06T07:25:47.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.428+0000 7fbfdbfff640 1 -- 172.21.15.17:0/717869615 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+13565 (secure 0 0 0) 0x7fbfdc058730 con 0x7fbfe406d9f0 2024-03-06T07:25:47.430 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:47.430 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":141,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:45.090596+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6832/824017697":"2024-03-07T07:25:41.833983+0000","172.21.15.17:6834/824017697":"2024-03-07T07:25:41.833983+0000","172.21.15.163:6836/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6837/3327948285":"2024-03-07T07:25:18.978040+0000","172.21.15.163:6834/3255496602":"2024-03-07T07:25:33.047401+0000","172.21.15.163:6835/3327948285":"2024-03-07T07:25:18.978040+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 msgr2=0x7fbfc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 0x7fbfc406d9c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc036000 tx=0x7fbfcc008030 comp rx=0 tx=0).stop 2024-03-06T07:25:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 msgr2=0x7fbfe40a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b4d0 tx=0x7fbfdc00b990 comp rx=0 tx=0).stop 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 shutdown_connections 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbfc406b500 0x7fbfc406d9c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfe41551e0 0x7fbfe414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfe40c3280 0x7fbfe414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 --2- 172.21.15.17:0/717869615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe406d9f0 0x7fbfe40a97b0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 >> 172.21.15.17:0/717869615 conn(0x7fbfe40aa630 msgr2=0x7fbfe4151150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 shutdown_connections 2024-03-06T07:25:47.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.430+0000 7fbfe995c640 1 -- 172.21.15.17:0/717869615 wait complete. 2024-03-06T07:25:47.443 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:47.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.547+0000 7f96f8439640 1 Processor -- start 2024-03-06T07:25:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.547+0000 7f96f8439640 1 -- start start 2024-03-06T07:25:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.547+0000 7f96f8439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96f0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.547+0000 7f96f8439640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f0152dd0 con 0x7f96f01529f0 2024-03-06T07:25:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f8439640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f0158b00 con 0x7f96f00c5370 2024-03-06T07:25:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f8439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0159710 con 0x7f96f00c57d0 2024-03-06T07:25:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f61d5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f96f01529f0 0x7f96f00c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40960/0 (socket says 172.21.15.17:40960) 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f61d5640 1 -- 172.21.15.17:0/3412958375 learned_addr learned my addr 172.21.15.17:0/3412958375 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f69d6640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96f0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518095615 0 0) 0x7f96f0152dd0 con 0x7f96f01529f0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96dc003520 con 0x7f96f01529f0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238680125 0 0) 0x7f96f0158b00 con 0x7f96f00c5370 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f0152dd0 con 0x7f96f00c5370 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010050866 0 0) 0x7f96dc003520 con 0x7f96f01529f0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 msgr2=0x7f96f0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96f0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 >> v1:172.21.15.163:6789/0 conn(0x7f96f00c5370 legacy=0x7f96f0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0158de0 con 0x7f96f01529f0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f96e80015f0 con 0x7f96f01529f0 2024-03-06T07:25:47.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 >> v1:172.21.15.17:6789/0 conn(0x7f96f01529f0 legacy=0x7f96f00c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc003b70 0x7f96dc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96dc004800 0x7f96dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96dc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0159710 con 0x7f96dc004800 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc00bb50 con 0x7f96dc003b70 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc00bd50 con 0x7f96f00c57d0 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.548+0000 7f96f59d4640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96dc004800 0x7f96dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f61d5640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc003b70 0x7f96dc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f69d6640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96dc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f59d4640 1 -- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 msgr2=0x7f96dc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f59d4640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f00c57d0 0x7f96dc005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f59d4640 1 -- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc003b70 msgr2=0x7f96dc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f59d4640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc003b70 0x7f96dc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.549+0000 7f96f59d4640 1 -- 172.21.15.17:0/3412958375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc003520 con 0x7f96dc004800 2024-03-06T07:25:47.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f59d4640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96dc004800 0x7f96dc004c80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f96e00018d0 tx=0x7f96dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365f203a65004a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0037070 con 0x7f96dc004800 2024-03-06T07:25:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00052d0 con 0x7f96dc004800 2024-03-06T07:25:47.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f51d3640 1 -- 172.21.15.17:0/3412958375 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00014f0 con 0x7f96dc004800 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 -- 172.21.15.17:0/3412958375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96dc004800 msgr2=0x7f96dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 --2- 172.21.15.17:0/3412958375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96dc004800 0x7f96dc004c80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f96e00018d0 tx=0x7f96dc005830 comp rx=0 tx=0).stop 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f69d6640 1 -- 172.21.15.17:0/3412958375 reap_dead start 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 -- 172.21.15.17:0/3412958375 shutdown_connections 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 -- 172.21.15.17:0/3412958375 >> 172.21.15.17:0/3412958375 conn(0x7f96f00ac580 msgr2=0x7f96f00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 -- 172.21.15.17:0/3412958375 shutdown_connections 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.550+0000 7f96f8439640 1 -- 172.21.15.17:0/3412958375 wait complete. 2024-03-06T07:25:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 Processor -- start 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 -- start start 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 0x7f96f01e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f0148ce0 0x7f96f0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c9000 con 0x7f96f01ea770 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c68b0 con 0x7f96dc004800 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f8439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01593a0 con 0x7f96f0148ce0 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f69d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f69d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49064/0 (socket says 172.21.15.17:49064) 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.552+0000 7f96f69d6640 1 -- 172.21.15.17:0/1053183718 learned_addr learned my addr 172.21.15.17:0/1053183718 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f61d5640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 0x7f96f01e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f59d4640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f0148ce0 0x7f96f0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 -- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f0148ce0 msgr2=0x7f96f0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f0148ce0 0x7f96f0157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 -- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 msgr2=0x7f96f01e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 0x7f96f01e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 -- 172.21.15.17:0/1053183718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00c9fe0 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f61d5640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 0x7f96f01e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f69d6640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f96f00b0b60 tx=0x7f96ec00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec015430 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec01d070 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f01e9fb0 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec015830 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.553+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01ea250 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.554+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f004f330 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.555+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96ec0159d0 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.555+0000 7f96e6ffd640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 0x7f96d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.555+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f96ec08a9d0 con 0x7f96f01ea770 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.555+0000 7f96f61d5640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 0x7f96d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.556+0000 7f96f61d5640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 0x7f96d406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f96e80011e0 tx=0x7f96e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.559+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f96f00c9fe0 con 0x7f96f01ea770 2024-03-06T07:25:47.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.740+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96f00b08b0 con 0x7f96f01ea770 2024-03-06T07:25:47.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.740+0000 7f96e6ffd640 1 -- 172.21.15.17:0/1053183718 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96ec010040 con 0x7f96f01ea770 2024-03-06T07:25:47.742 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 msgr2=0x7f96d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 0x7f96d406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f96e80011e0 tx=0x7f96e8009040 comp rx=0 tx=0).stop 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 msgr2=0x7f96f0148090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f96f00b0b60 tx=0x7f96ec00f580 comp rx=0 tx=0).stop 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 shutdown_connections 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96d406b4d0 0x7f96d406d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f01ea770 0x7f96f0148090 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96f0148ce0 0x7f96f0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 --2- 172.21.15.17:0/1053183718 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96dc004800 0x7f96f01e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 >> 172.21.15.17:0/1053183718 conn(0x7f96f00ac580 msgr2=0x7f96f00abdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 shutdown_connections 2024-03-06T07:25:47.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.742+0000 7f96f8439640 1 -- 172.21.15.17:0/1053183718 wait complete. 2024-03-06T07:25:47.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla' 2024-03-06T07:25:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.859+0000 7fcbd3239640 1 Processor -- start 2024-03-06T07:25:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.859+0000 7fcbd3239640 1 -- start start 2024-03-06T07:25:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.859+0000 7fcbd3239640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbcc14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.859+0000 7fcbd3239640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcbcc14f140 0x7fcbcc0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd3239640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc155bb0 con 0x7fcbcc1550c0 2024-03-06T07:25:47.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd3239640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c4610 con 0x7fcbcc14f140 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd3239640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc14f5f0 con 0x7fcbcc0aa180 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd0fd5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcbcc1550c0 0x7fcbcc1554a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40964/0 (socket says 172.21.15.17:40964) 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd0fd5640 1 -- 172.21.15.17:0/3324465215 learned_addr learned my addr 172.21.15.17:0/3324465215 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcbfff640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbcc14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854859022 0 0) 0x7fcbcc155bb0 con 0x7fcbcc1550c0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbbc005730 con 0x7fcbcc1550c0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/3324465215 >> v2:172.21.15.163:3300/0 conn(0x7fcbcc14f140 0x7fcbcc0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1479082108 0 0) 0x7fcbbc005730 con 0x7fcbcc1550c0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 msgr2=0x7fcbcc14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbcc14ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 >> v2:172.21.15.163:3300/0 conn(0x7fcbcc14f140 msgr2=0x7fcbcc0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3324465215 >> v2:172.21.15.163:3300/0 conn(0x7fcbcc14f140 0x7fcbcc0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0aa660 con 0x7fcbcc1550c0 2024-03-06T07:25:47.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fcbc0001670 con 0x7fcbcc1550c0 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 >> v1:172.21.15.17:6789/0 conn(0x7fcbcc1550c0 legacy=0x7fcbcc1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbbc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc006140 0x7fcbbc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbbc006f50 0x7fcbbc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c4610 con 0x7fcbbc006f50 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc14f5f0 con 0x7fcbbc006140 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.860+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc00dcb0 con 0x7fcbcc0aa180 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd0fd5640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbbc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbbc006f50 0x7fcbbc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbcbfff640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc006140 0x7fcbbc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 -- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 msgr2=0x7fcbbc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbcc0aa180 0x7fcbbc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 -- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc006140 msgr2=0x7fcbbc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbbc006140 0x7fcbbc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.861+0000 7fcbd17d6640 1 -- 172.21.15.17:0/3324465215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc005730 con 0x7fcbbc006f50 2024-03-06T07:25:47.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbbc006f50 0x7fcbbc0073d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009fc0 tx=0x7fcbbc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce95ab35cde38401 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4039070 con 0x7fcbbc006f50 2024-03-06T07:25:47.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc402cc60 con 0x7fcbbc006f50 2024-03-06T07:25:47.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbcb7fe640 1 -- 172.21.15.17:0/3324465215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4009c10 con 0x7fcbbc006f50 2024-03-06T07:25:47.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd3239640 1 -- 172.21.15.17:0/3324465215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbbc006f50 msgr2=0x7fcbbc0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd3239640 1 --2- 172.21.15.17:0/3324465215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbbc006f50 0x7fcbbc0073d0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009fc0 tx=0x7fcbbc007a40 comp rx=0 tx=0).stop 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd17d6640 1 -- 172.21.15.17:0/3324465215 reap_dead start 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd3239640 1 -- 172.21.15.17:0/3324465215 shutdown_connections 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.862+0000 7fcbd3239640 1 -- 172.21.15.17:0/3324465215 >> 172.21.15.17:0/3324465215 conn(0x7fcbcc0c6710 msgr2=0x7fcbcc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.863+0000 7fcbd3239640 1 -- 172.21.15.17:0/3324465215 shutdown_connections 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.863+0000 7fcbd3239640 1 -- 172.21.15.17:0/3324465215 wait complete. 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.864+0000 7fcbd3239640 1 Processor -- start 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.864+0000 7fcbd3239640 1 -- start start 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbbc006f50 0x7fcbcc1506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 0x7fcbcc0a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c1310 con 0x7fcbcc152020 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c4860 con 0x7fcbcc0bfdb0 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0c1490 con 0x7fcbbc006f50 2024-03-06T07:25:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49068/0 (socket says 172.21.15.17:49068) 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 -- 172.21.15.17:0/1373185758 learned_addr learned my addr 172.21.15.17:0/1373185758 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 0x7fcbcc0a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd0fd5640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbbc006f50 0x7fcbcc1506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 -- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbbc006f50 msgr2=0x7fcbcc1506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbbc006f50 0x7fcbcc1506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 -- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 msgr2=0x7fcbcc0a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 0x7fcbcc0a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 -- 172.21.15.17:0/1373185758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc1524d0 con 0x7fcbcc152020 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd17d6640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 0x7fcbcc0a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbcbfff640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80119a0 tx=0x7fcbb8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800fa90 con 0x7fcbcc152020 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb800fc30 con 0x7fcbcc152020 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc14fc00 con 0x7fcbcc152020 2024-03-06T07:25:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8014440 con 0x7fcbcc152020 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.865+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc06da50 con 0x7fcbcc152020 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.867+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc04f330 con 0x7fcbcc152020 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.867+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbb8012040 con 0x7fcbcc152020 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.867+0000 7fcbc97fa640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 0x7fcbb006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.868+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcbb808a720 con 0x7fcbcc152020 2024-03-06T07:25:47.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.868+0000 7fcbd0fd5640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 0x7fcbb006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:47.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.868+0000 7fcbd0fd5640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 0x7fcbb006d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00010e0 tx=0x7fcbc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:47.872+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbb8019030 con 0x7fcbcc152020 2024-03-06T07:25:48.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:48.125+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]} v 0) v1 -- 0x7fcbcc068450 con 0x7fcbcc152020 2024-03-06T07:25:49.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.116+0000 7fcbc97fa640 1 -- 172.21.15.17:0/1373185758 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]}]=0 v340) v1 ==== 149+0+0 (secure 0 0 0) 0x7fcbb80596f0 con 0x7fcbcc152020 2024-03-06T07:25:49.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.118+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 msgr2=0x7fcbb006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.118+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 0x7fcbb006d9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00010e0 tx=0x7fcbc0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.118+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 msgr2=0x7fcbcc1eb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.118+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80119a0 tx=0x7fcbb8011e60 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 shutdown_connections 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbb006b500 0x7fcbb006d9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbcc0bfdb0 0x7fcbcc0a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbcc152020 0x7fcbcc1eb300 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 --2- 172.21.15.17:0/1373185758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbbc006f50 0x7fcbcc1506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 >> 172.21.15.17:0/1373185758 conn(0x7fcbcc0c6710 msgr2=0x7fcbcc14c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 shutdown_connections 2024-03-06T07:25:49.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.119+0000 7fcbd3239640 1 -- 172.21.15.17:0/1373185758 wait complete. 2024-03-06T07:25:49.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:25:49.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.243+0000 7fca1a088640 1 Processor -- start 2024-03-06T07:25:49.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.243+0000 7fca1a088640 1 -- start start 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca1a088640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca140c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca1a088640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca140ab170 0x7fca140a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca1a088640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca140c3ef0 con 0x7fca140c5990 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca1a088640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca141556d0 con 0x7fca140ab170 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca1a088640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca141559b0 con 0x7fca1406c900 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca13fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fca140ab170 0x7fca140a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca137fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fca140c5990 0x7fca1406c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40974/0 (socket says 172.21.15.17:40974) 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca137fe640 1 -- 172.21.15.17:0/3899802519 learned_addr learned my addr 172.21.15.17:0/3899802519 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca13fff640 1 -- 172.21.15.17:0/3899802519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 msgr2=0x7fca140c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca13fff640 1 --2- 172.21.15.17:0/3899802519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca140c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca13fff640 1 -- 172.21.15.17:0/3899802519 >> v1:172.21.15.17:6789/0 conn(0x7fca140c5990 legacy=0x7fca1406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca13fff640 1 -- 172.21.15.17:0/3899802519 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca14155c90 con 0x7fca140ab170 2024-03-06T07:25:49.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.244+0000 7fca127fc640 1 -- 172.21.15.17:0/3899802519 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050461699 0 0) 0x7fca140c3ef0 con 0x7fca140c5990 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca13fff640 1 --2- 172.21.15.17:0/3899802519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca140a9fa0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009d90 tx=0x7fca0c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cda3e1d24e08348 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca127fc640 1 -- 172.21.15.17:0/3899802519 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c039070 con 0x7fca140ab170 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca127fc640 1 -- 172.21.15.17:0/3899802519 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c02cc50 con 0x7fca140ab170 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca127fc640 1 -- 172.21.15.17:0/3899802519 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c034440 con 0x7fca140ab170 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca1a088640 1 -- 172.21.15.17:0/3899802519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 msgr2=0x7fca140a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.245+0000 7fca1a088640 1 --2- 172.21.15.17:0/3899802519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca140a9fa0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009d90 tx=0x7fca0c02bea0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 -- 172.21.15.17:0/3899802519 shutdown_connections 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 --2- 172.21.15.17:0/3899802519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca140a9fa0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 --2- 172.21.15.17:0/3899802519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca140c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 -- 172.21.15.17:0/3899802519 >> 172.21.15.17:0/3899802519 conn(0x7fca140c6710 msgr2=0x7fca140c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 -- 172.21.15.17:0/3899802519 shutdown_connections 2024-03-06T07:25:49.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.246+0000 7fca1a088640 1 -- 172.21.15.17:0/3899802519 wait complete. 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.247+0000 7fca1a088640 1 Processor -- start 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.247+0000 7fca1a088640 1 -- start start 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca1414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca140c5990 0x7fca14145010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca14156610 con 0x7fca140ab170 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca14156490 con 0x7fca140c5990 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca1a088640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca140c3af0 con 0x7fca1406c900 2024-03-06T07:25:49.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49086/0 (socket says 172.21.15.17:49086) 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 -- 172.21.15.17:0/371463561 learned_addr learned my addr 172.21.15.17:0/371463561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca137fe640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca1414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca13fff640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca140c5990 0x7fca14145010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 -- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 msgr2=0x7fca1414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca1414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 -- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca140c5990 msgr2=0x7fca14145010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca140c5990 0x7fca14145010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 -- 172.21.15.17:0/371463561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140c2fc0 con 0x7fca140ab170 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.248+0000 7fca12ffd640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b480 tx=0x7fca0800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.249+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08014070 con 0x7fca140ab170 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.249+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08010870 con 0x7fca140ab170 2024-03-06T07:25:49.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.249+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca14153ce0 con 0x7fca140ab170 2024-03-06T07:25:49.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.249+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08010c60 con 0x7fca140ab170 2024-03-06T07:25:49.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.249+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca141542a0 con 0x7fca140ab170 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.250+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca14154890 con 0x7fca140ab170 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.250+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca08017070 con 0x7fca140ab170 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.251+0000 7fc9fbfff640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 0x7fca0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.251+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fca08089de0 con 0x7fca140ab170 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.251+0000 7fca137fe640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 0x7fca0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.251+0000 7fca137fe640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 0x7fca0006d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fca04007fd0 tx=0x7fca04007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.255+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fca08058760 con 0x7fca140ab170 2024-03-06T07:25:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.436+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca140c3300 con 0x7fca140ab170 2024-03-06T07:25:49.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.439+0000 7fc9fbfff640 1 -- 172.21.15.17:0/371463561 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca0805c800 con 0x7fca140ab170 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 msgr2=0x7fca0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 0x7fca0006d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fca04007fd0 tx=0x7fca04007de0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 msgr2=0x7fca14144ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b480 tx=0x7fca0800b940 comp rx=0 tx=0).stop 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 shutdown_connections 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fca0006b500 0x7fca0006d9c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca140c5990 0x7fca14145010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca140ab170 0x7fca14144ad0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 --2- 172.21.15.17:0/371463561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca1406c900 0x7fca1414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 >> 172.21.15.17:0/371463561 conn(0x7fca140c6710 msgr2=0x7fca1406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 shutdown_connections 2024-03-06T07:25:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.441+0000 7fca1a088640 1 -- 172.21.15.17:0/371463561 wait complete. 2024-03-06T07:25:49.454 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:25:49.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:49.493 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:25:49.544 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:25:49.544 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:25:49.544 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:49.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:25:49.559 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:25:49.559 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:25:49.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:25:49.577 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:25:49.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:49.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.698+0000 7f52f5d02640 1 Processor -- start 2024-03-06T07:25:49.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.698+0000 7f52f5d02640 1 -- start start 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52f5d02640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52f006d9f0 0x7f52f00a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52f5d02640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f52f00c3280 0x7f52f00c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52f5d02640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f0158800 con 0x7f52f01551e0 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52f5d02640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f0158ae0 con 0x7f52f006d9f0 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52f5d02640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f0158dc0 con 0x7f52f00c3280 2024-03-06T07:25:49.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ef7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52f006d9f0 0x7f52f00a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52effff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f52f01551e0 0x7f52f01555f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40978/0 (socket says 172.21.15.17:40978) 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52effff640 1 -- 172.21.15.17:0/2400678 learned_addr learned my addr 172.21.15.17:0/2400678 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ef7fe640 1 -- 172.21.15.17:0/2400678 >> v2:172.21.15.163:3300/0 conn(0x7f52f00c3280 msgr2=0x7f52f00c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ef7fe640 1 --2- 172.21.15.17:0/2400678 >> v2:172.21.15.163:3300/0 conn(0x7f52f00c3280 0x7f52f00c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ef7fe640 1 -- 172.21.15.17:0/2400678 >> v1:172.21.15.17:6789/0 conn(0x7f52f01551e0 legacy=0x7f52f01555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ef7fe640 1 -- 172.21.15.17:0/2400678 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f01590a0 con 0x7f52f006d9f0 2024-03-06T07:25:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.699+0000 7f52ee7fc640 1 -- 172.21.15.17:0/2400678 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473677029 0 0) 0x7f52f0158800 con 0x7f52f01551e0 2024-03-06T07:25:49.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.700+0000 7f52ef7fe640 1 --2- 172.21.15.17:0/2400678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f006d9f0 0x7f52f00a7cd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f52f00721a0 tx=0x7f52e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d1f21a6593e74c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.700+0000 7f52ee7fc640 1 -- 172.21.15.17:0/2400678 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00315a0 con 0x7f52f006d9f0 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.700+0000 7f52ee7fc640 1 -- 172.21.15.17:0/2400678 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e0037070 con 0x7f52f006d9f0 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.700+0000 7f52ee7fc640 1 -- 172.21.15.17:0/2400678 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0031960 con 0x7f52f006d9f0 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 -- 172.21.15.17:0/2400678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f006d9f0 msgr2=0x7f52f00a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 --2- 172.21.15.17:0/2400678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f006d9f0 0x7f52f00a7cd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f52f00721a0 tx=0x7f52e002b840 comp rx=0 tx=0).stop 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 -- 172.21.15.17:0/2400678 shutdown_connections 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 --2- 172.21.15.17:0/2400678 >> v2:172.21.15.163:3300/0 conn(0x7f52f00c3280 0x7f52f00c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 --2- 172.21.15.17:0/2400678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f006d9f0 0x7f52f00a7cd0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 -- 172.21.15.17:0/2400678 >> 172.21.15.17:0/2400678 conn(0x7f52f00aa630 msgr2=0x7f52f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 -- 172.21.15.17:0/2400678 shutdown_connections 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.701+0000 7f52f5d02640 1 -- 172.21.15.17:0/2400678 wait complete. 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.702+0000 7f52f5d02640 1 Processor -- start 2024-03-06T07:25:49.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.702+0000 7f52f5d02640 1 -- start start 2024-03-06T07:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52f006d9f0 0x7f52f01edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 0x7f52f0152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f006df50 con 0x7f52f01551e0 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f006ddd0 con 0x7f52f00c3280 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52f5d02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00a8210 con 0x7f52f006d9f0 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49100/0 (socket says 172.21.15.17:49100) 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 -- 172.21.15.17:0/1159736937 learned_addr learned my addr 172.21.15.17:0/1159736937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52eeffd640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 0x7f52f0152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52ef7fe640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52f006d9f0 0x7f52f01edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 -- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52f006d9f0 msgr2=0x7f52f01edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52f006d9f0 0x7f52f01edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 -- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 msgr2=0x7f52f0152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 0x7f52f0152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 -- 172.21.15.17:0/1159736937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f01eea40 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52eeffd640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 0x7f52f0152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.703+0000 7f52effff640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c7b0 tx=0x7f52e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.704+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8018070 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.704+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e800a640 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.704+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f01573c0 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.704+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800ce60 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.704+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f0158460 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.705+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f004f330 con 0x7f52f01551e0 2024-03-06T07:25:49.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.705+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52e801b070 con 0x7f52f01551e0 2024-03-06T07:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.705+0000 7f52d7fff640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 0x7f52dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.706+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f52e8089670 con 0x7f52f01551e0 2024-03-06T07:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.706+0000 7f52ef7fe640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 0x7f52dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.706+0000 7f52ef7fe640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 0x7f52dc06d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f52e0036000 tx=0x7f52e003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.710+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f52e8058640 con 0x7f52f01551e0 2024-03-06T07:25:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.927+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52f0073240 con 0x7f52f01551e0 2024-03-06T07:25:49.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.928+0000 7f52d7fff640 1 -- 172.21.15.17:0/1159736937 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f52e8057f40 con 0x7f52f01551e0 2024-03-06T07:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":132,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3725508931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3725508931},{"type":"v1","addr":"172.21.15.163:6837","nonce":3725508931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123},{"gid":7004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/862042831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":862042831},{"type":"v1","addr":"172.21.15.163:6836","nonce":862042831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":128},{"gid":8268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1732861093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1732861093},{"type":"v1","addr":"172.21.15.17:6835","nonce":1732861093}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":117},{"gid":8691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/3389742493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3389742493},{"type":"v1","addr":"172.21.15.17:6834","nonce":3389742493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[]} 2024-03-06T07:25:49.930 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 132 2024-03-06T07:25:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 msgr2=0x7f52dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 0x7f52dc06d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f52e0036000 tx=0x7f52e003d040 comp rx=0 tx=0).stop 2024-03-06T07:25:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 msgr2=0x7f52f0153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c7b0 tx=0x7f52e800cc70 comp rx=0 tx=0).stop 2024-03-06T07:25:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 shutdown_connections 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52dc06b500 0x7f52dc06d9c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52f01551e0 0x7f52f0153420 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52f00c3280 0x7f52f0152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 --2- 172.21.15.17:0/1159736937 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52f006d9f0 0x7f52f01edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.929+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 >> 172.21.15.17:0/1159736937 conn(0x7f52f00aa630 msgr2=0x7f52f00c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.930+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 shutdown_connections 2024-03-06T07:25:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:49.930+0000 7f52f5d02640 1 -- 172.21.15.17:0/1159736937 wait complete. 2024-03-06T07:25:49.942 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:25:49.942 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:25:49.942 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:25:49.943 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:49.943 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:25:49.943 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:25:49.943 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:25:49.943 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:49.943 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:25:49.943 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:25:49.944 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:25:49.944 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:49.944 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:25:49.944 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:25:49.944 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:25:49.944 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:25:49.951 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:25:49.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:25:49.952 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:25:49.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:25:49.955 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:25:49.955 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:25:49.956 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:25:49.956 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:25:49.994 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:25:49.994 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:49.995 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:25:49.995 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:49.998 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:25:49.998 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:49.999 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:25:49.999 DEBUG:teuthology.parallel:result is None 2024-03-06T07:25:49.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:25:50.071 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:25:50.071 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:25:50.081 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:25:50.081 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:25:50.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.105+0000 7fc61435e640 1 Processor -- start 2024-03-06T07:25:50.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.105+0000 7fc61435e640 1 -- start start 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc61435e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc60c0c6ab0 0x7fc60c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc61435e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 0x7fc60c0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc61435e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0a9990 con 0x7fc60c0aa430 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc61435e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c155d10 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc61435e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c1556c0 con 0x7fc60c1561a0 2024-03-06T07:25:50.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc6120fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc60c0c6ab0 0x7fc60c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc6128fb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc60c0aa430 0x7fc60c14f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41010/0 (socket says 172.21.15.17:41010) 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc6128fb640 1 -- 172.21.15.17:0/938380455 learned_addr learned my addr 172.21.15.17:0/938380455 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.106+0000 7fc6118f9640 1 --2- 172.21.15.17:0/938380455 >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 0x7fc60c0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6110f8640 1 -- 172.21.15.17:0/938380455 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881271520 0 0) 0x7fc60c0a9990 con 0x7fc60c0aa430 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6110f8640 1 -- 172.21.15.17:0/938380455 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f4003520 con 0x7fc60c0aa430 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6120fa640 1 -- 172.21.15.17:0/938380455 >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 msgr2=0x7fc60c0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6120fa640 1 --2- 172.21.15.17:0/938380455 >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 0x7fc60c0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6120fa640 1 -- 172.21.15.17:0/938380455 >> v1:172.21.15.17:6789/0 conn(0x7fc60c0aa430 legacy=0x7fc60c14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6120fa640 1 -- 172.21.15.17:0/938380455 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c1559a0 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6118f9640 1 --2- 172.21.15.17:0/938380455 >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 0x7fc60c0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.107+0000 7fc6120fa640 1 --2- 172.21.15.17:0/938380455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c0c6ab0 0x7fc60c1529e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0c0b80 tx=0x7fc60402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d55c7533fcc6cea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc6110f8640 1 -- 172.21.15.17:0/938380455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6040315a0 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc6110f8640 1 -- 172.21.15.17:0/938380455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc604037070 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc6110f8640 1 -- 172.21.15.17:0/938380455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604031960 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 -- 172.21.15.17:0/938380455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c0c6ab0 msgr2=0x7fc60c1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 --2- 172.21.15.17:0/938380455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c0c6ab0 0x7fc60c1529e0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0c0b80 tx=0x7fc60402b840 comp rx=0 tx=0).stop 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 -- 172.21.15.17:0/938380455 shutdown_connections 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 --2- 172.21.15.17:0/938380455 >> v2:172.21.15.163:3300/0 conn(0x7fc60c1561a0 0x7fc60c0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 --2- 172.21.15.17:0/938380455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c0c6ab0 0x7fc60c1529e0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 -- 172.21.15.17:0/938380455 >> 172.21.15.17:0/938380455 conn(0x7fc60c06b1a0 msgr2=0x7fc60c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 -- 172.21.15.17:0/938380455 shutdown_connections 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.108+0000 7fc61435e640 1 -- 172.21.15.17:0/938380455 wait complete. 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.109+0000 7fc61435e640 1 Processor -- start 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.109+0000 7fc61435e640 1 -- start start 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc60c0aa430 0x7fc60c14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc60c0c6ab0 0x7fc60c13f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c14f820 con 0x7fc60c1561a0 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0c2a90 con 0x7fc60c0aa430 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc61435e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c1543e0 con 0x7fc60c0c6ab0 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49148/0 (socket says 172.21.15.17:49148) 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 -- 172.21.15.17:0/3142460862 learned_addr learned my addr 172.21.15.17:0/3142460862 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:50.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6120fa640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc60c0aa430 0x7fc60c14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6118f9640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc60c0c6ab0 0x7fc60c13f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 -- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc60c0c6ab0 msgr2=0x7fc60c13f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc60c0c6ab0 0x7fc60c13f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 -- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc60c0aa430 msgr2=0x7fc60c14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc60c0aa430 0x7fc60c14df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.110+0000 7fc6128fb640 1 -- 172.21.15.17:0/3142460862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0c6300 con 0x7fc60c1561a0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc6128fb640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00caa0 tx=0x7fc5fc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc017070 con 0x7fc60c1561a0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc00ac00 con 0x7fc60c1561a0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c1473a0 con 0x7fc60c1561a0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc012400 con 0x7fc60c1561a0 2024-03-06T07:25:50.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.111+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c154100 con 0x7fc60c1561a0 2024-03-06T07:25:50.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.112+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c04f330 con 0x7fc60c1561a0 2024-03-06T07:25:50.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.112+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5fc01a070 con 0x7fc60c1561a0 2024-03-06T07:25:50.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.112+0000 7fc60affd640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 0x7fc5f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:50.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.113+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc5fc089e00 con 0x7fc60c1561a0 2024-03-06T07:25:50.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.113+0000 7fc6120fa640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 0x7fc5f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:50.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.113+0000 7fc6120fa640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 0x7fc5f806d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc604036000 tx=0x7fc60403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:50.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.117+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc60c0c6300 con 0x7fc60c1561a0 2024-03-06T07:25:50.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:50.313+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc60c0c21d0 con 0x7fc60c1561a0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.142+0000 7fc60affd640 1 -- 172.21.15.17:0/3142460862 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v146) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc5fc058780 con 0x7fc60c1561a0 2024-03-06T07:25:51.143 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:25:51.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 msgr2=0x7fc5f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 0x7fc5f806d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc604036000 tx=0x7fc60403d040 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 msgr2=0x7fc60c13fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00caa0 tx=0x7fc5fc00cf60 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 shutdown_connections 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5f806b4d0 0x7fc5f806d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc60c1561a0 0x7fc60c13fac0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc60c0c6ab0 0x7fc60c13f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 --2- 172.21.15.17:0/3142460862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc60c0aa430 0x7fc60c14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 >> 172.21.15.17:0/3142460862 conn(0x7fc60c06b1a0 msgr2=0x7fc60c0c0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 shutdown_connections 2024-03-06T07:25:51.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.145+0000 7fc61435e640 1 -- 172.21.15.17:0/3142460862 wait complete. 2024-03-06T07:25:51.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:25:51.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.266+0000 7f0803288640 1 Processor -- start 2024-03-06T07:25:51.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.266+0000 7f0803288640 1 -- start start 2024-03-06T07:25:51.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07fc155c80 0x7f07fc0c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07fc0720a0 0x7f07fc06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc0c1dc0 con 0x7f07fc155c80 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc0c1ae0 con 0x7f07fc0720a0 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0803288640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc06e450 con 0x7f07fc152520 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07fc155c80 0x7f07fc0c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07fc155c80 0x7f07fc0c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56954/0 (socket says 172.21.15.17:56954) 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 -- 172.21.15.17:0/3083183494 learned_addr learned my addr 172.21.15.17:0/3083183494 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0801024640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0801825640 1 --2- 172.21.15.17:0/3083183494 >> v2:172.21.15.163:3300/0 conn(0x7f07fc0720a0 0x7f07fc06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 -- 172.21.15.17:0/3083183494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 msgr2=0x7f07fc06ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc06ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 -- 172.21.15.17:0/3083183494 >> v2:172.21.15.163:3300/0 conn(0x7f07fc0720a0 msgr2=0x7f07fc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 --2- 172.21.15.17:0/3083183494 >> v2:172.21.15.163:3300/0 conn(0x7f07fc0720a0 0x7f07fc06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.267+0000 7f0800823640 1 -- 172.21.15.17:0/3083183494 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc06e730 con 0x7f07fc155c80 2024-03-06T07:25:51.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.268+0000 7f0800823640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc0c12b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f07fc067c80 tx=0x7f07ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eae161e13a471da server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.268+0000 7f07f3fff640 1 -- 172.21.15.17:0/3083183494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec035020 con 0x7f07fc155c80 2024-03-06T07:25:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.268+0000 7f07f3fff640 1 -- 172.21.15.17:0/3083183494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec031870 con 0x7f07fc155c80 2024-03-06T07:25:51.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.268+0000 7f07f3fff640 1 -- 172.21.15.17:0/3083183494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec031c40 con 0x7f07fc155c80 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 -- 172.21.15.17:0/3083183494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 msgr2=0x7f07fc0c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc0c12b0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f07fc067c80 tx=0x7f07ec02b910 comp rx=0 tx=0).stop 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 -- 172.21.15.17:0/3083183494 shutdown_connections 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 --2- 172.21.15.17:0/3083183494 >> v2:172.21.15.163:3300/0 conn(0x7f07fc0720a0 0x7f07fc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc0c12b0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 --2- 172.21.15.17:0/3083183494 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc06ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 -- 172.21.15.17:0/3083183494 >> 172.21.15.17:0/3083183494 conn(0x7f07fc0c76b0 msgr2=0x7f07fc0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 -- 172.21.15.17:0/3083183494 shutdown_connections 2024-03-06T07:25:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.269+0000 7f0803288640 1 -- 172.21.15.17:0/3083183494 wait complete. 2024-03-06T07:25:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.270+0000 7f0803288640 1 Processor -- start 2024-03-06T07:25:51.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.270+0000 7f0803288640 1 -- start start 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 0x7f07fc1eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc152900 con 0x7f07fc155c80 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0c17f0 con 0x7f07fc0720a0 2024-03-06T07:25:51.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc152a80 con 0x7f07fc152520 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56966/0 (socket says 172.21.15.17:56966) 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 -- 172.21.15.17:0/3847869328 learned_addr learned my addr 172.21.15.17:0/3847869328 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0800823640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801024640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 0x7f07fc1eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 -- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 msgr2=0x7f07fc14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc14eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 -- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 msgr2=0x7f07fc1eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 0x7f07fc1eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 -- 172.21.15.17:0/3847869328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc14e0e0 con 0x7f07fc155c80 2024-03-06T07:25:51.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0800823640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801024640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 0x7f07fc1eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0801825640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007e90 tx=0x7f07f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4014420 con 0x7f07fc155c80 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f4019070 con 0x7f07fc155c80 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc14de00 con 0x7f07fc155c80 2024-03-06T07:25:51.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.271+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4014800 con 0x7f07fc155c80 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.272+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc154e90 con 0x7f07fc155c80 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.272+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc04f330 con 0x7f07fc155c80 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.273+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07f40149a0 con 0x7f07fc155c80 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.273+0000 7f07f1ffb640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.273+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f07f408a7e0 con 0x7f07fc155c80 2024-03-06T07:25:51.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.274+0000 7f0801024640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.274+0000 7f0801024640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 0x7f07e006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f07f800b970 tx=0x7f07f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.278+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07f40598b0 con 0x7f07fc155c80 2024-03-06T07:25:51.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.530+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f07fc0c3270 con 0x7f07fc155c80 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.530+0000 7f07f1ffb640 1 -- 172.21.15.17:0/3847869328 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f07f40591b0 con 0x7f07fc155c80 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:25:51.532 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.535 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.537 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:25:51.538 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:25:51.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:25:51.541 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:51.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:25:51.543 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 msgr2=0x7f07e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 0x7f07e006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f07f800b970 tx=0x7f07f8008040 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 msgr2=0x7f07fc14d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f07f4007e90 tx=0x7f07f400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 shutdown_connections 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07e006b4d0 0x7f07e006d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07fc155c80 0x7f07fc14d000 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07fc152520 0x7f07fc14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 --2- 172.21.15.17:0/3847869328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07fc0720a0 0x7f07fc1eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.532+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 >> 172.21.15.17:0/3847869328 conn(0x7f07fc0c76b0 msgr2=0x7f07fc0c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.533+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 shutdown_connections 2024-03-06T07:25:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.533+0000 7f0803288640 1 -- 172.21.15.17:0/3847869328 wait complete. 2024-03-06T07:25:51.546 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.645+0000 7f716640e640 1 Processor -- start 2024-03-06T07:25:51.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.645+0000 7f716640e640 1 -- start start 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f716640e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 0x7f71601529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f716640e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71600c1aa0 0x7f71600c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f716640e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600c28d0 con 0x7f71600c5c60 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f716640e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600c2680 con 0x7f71600c1aa0 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f716640e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600c2bb0 con 0x7f716014f790 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71600c1aa0 0x7f71600c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71600c1aa0 0x7f71600c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56974/0 (socket says 172.21.15.17:56974) 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 -- 172.21.15.17:0/2089621181 learned_addr learned my addr 172.21.15.17:0/2089621181 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715ffff640 1 --2- 172.21.15.17:0/2089621181 >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 0x7f71601529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 -- 172.21.15.17:0/2089621181 >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 msgr2=0x7f71601529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 --2- 172.21.15.17:0/2089621181 >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 0x7f71601529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 -- 172.21.15.17:0/2089621181 >> v1:172.21.15.163:6789/0 conn(0x7f71600c5c60 legacy=0x7f7160155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715f7fe640 1 -- 172.21.15.17:0/2089621181 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600c2e90 con 0x7f71600c1aa0 2024-03-06T07:25:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.646+0000 7f715ffff640 1 --2- 172.21.15.17:0/2089621181 >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 0x7f71601529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.647+0000 7f715f7fe640 1 --2- 172.21.15.17:0/2089621181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f71600c5720 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f71600c3170 tx=0x7f715002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666c609710c6377 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.647+0000 7f715effd640 1 -- 172.21.15.17:0/2089621181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150034070 con 0x7f71600c1aa0 2024-03-06T07:25:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.647+0000 7f715effd640 1 -- 172.21.15.17:0/2089621181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7150030860 con 0x7f71600c1aa0 2024-03-06T07:25:51.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.647+0000 7f715effd640 1 -- 172.21.15.17:0/2089621181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150030c30 con 0x7f71600c1aa0 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 -- 172.21.15.17:0/2089621181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 msgr2=0x7f71600c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 --2- 172.21.15.17:0/2089621181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f71600c5720 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f71600c3170 tx=0x7f715002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 -- 172.21.15.17:0/2089621181 shutdown_connections 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 --2- 172.21.15.17:0/2089621181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f71600c5720 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 --2- 172.21.15.17:0/2089621181 >> v2:172.21.15.163:3300/0 conn(0x7f716014f790 0x7f71601529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 -- 172.21.15.17:0/2089621181 >> 172.21.15.17:0/2089621181 conn(0x7f71600c6710 msgr2=0x7f71600c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 -- 172.21.15.17:0/2089621181 shutdown_connections 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.648+0000 7f716640e640 1 -- 172.21.15.17:0/2089621181 wait complete. 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.649+0000 7f716640e640 1 Processor -- start 2024-03-06T07:25:51.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.649+0000 7f716640e640 1 -- start start 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71600c5c60 0x7f716014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716014f790 0x7f7160154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716006cc00 con 0x7f71600c1aa0 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c0d60 con 0x7f71600c5c60 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f716640e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716006cd80 con 0x7f716014f790 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56990/0 (socket says 172.21.15.17:56990) 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 -- 172.21.15.17:0/3104903992 learned_addr learned my addr 172.21.15.17:0/3104903992 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715f7fe640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71600c5c60 0x7f716014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f71649ab640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716014f790 0x7f7160154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 -- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716014f790 msgr2=0x7f7160154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716014f790 0x7f7160154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 -- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71600c5c60 msgr2=0x7f716014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71600c5c60 0x7f716014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.650+0000 7f715ffff640 1 -- 172.21.15.17:0/3104903992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160144f70 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f715ffff640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f715400d7d0 tx=0x7f715400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154004030 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71540041d0 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7160147150 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154009640 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.651+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71601ec630 con 0x7f71600c1aa0 2024-03-06T07:25:51.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.652+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716014de90 con 0x7f71600c1aa0 2024-03-06T07:25:51.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.652+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71540097e0 con 0x7f71600c1aa0 2024-03-06T07:25:51.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.652+0000 7f715cff9640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 0x7f714c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.652+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f715408a0f0 con 0x7f71600c1aa0 2024-03-06T07:25:51.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.653+0000 7f715f7fe640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 0x7f714c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.653+0000 7f715f7fe640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 0x7f714c06d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7150036000 tx=0x7f7150008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.657+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7154058af0 con 0x7f71600c1aa0 2024-03-06T07:25:51.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.838+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f716004f330 con 0x7f71600c1aa0 2024-03-06T07:25:51.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.839+0000 7f715cff9640 1 -- 172.21.15.17:0/3104903992 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f715405cb90 con 0x7f71600c1aa0 2024-03-06T07:25:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:51.841 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:51.141146+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:51.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 msgr2=0x7f714c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 0x7f714c06d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7150036000 tx=0x7f7150008030 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 msgr2=0x7f7160144a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f715400d7d0 tx=0x7f715400dc90 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 shutdown_connections 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714c06b4d0 0x7f714c06d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f716014f790 0x7f7160154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71600c5c60 0x7f716014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 --2- 172.21.15.17:0/3104903992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71600c1aa0 0x7f7160144a30 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 >> 172.21.15.17:0/3104903992 conn(0x7f71600c6710 msgr2=0x7f7160072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 shutdown_connections 2024-03-06T07:25:51.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.841+0000 7f716640e640 1 -- 172.21.15.17:0/3104903992 wait complete. 2024-03-06T07:25:51.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:25:51.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:25:51.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.956+0000 7f521b633640 1 Processor -- start 2024-03-06T07:25:51.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.956+0000 7f521b633640 1 -- start start 2024-03-06T07:25:51.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f52141528d0 0x7f52140724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5214072a10 0x7f5214155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52140c67f0 0x7f5214155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5214152cb0 con 0x7f52140c67f0 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5214155ca0 con 0x7f52141528d0 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f521b633640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c38b0 con 0x7f5214072a10 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52140c67f0 0x7f5214155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52140c67f0 0x7f5214155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57004/0 (socket says 172.21.15.17:57004) 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 -- 172.21.15.17:0/3189155350 learned_addr learned my addr 172.21.15.17:0/3189155350 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5218bce640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5214072a10 0x7f5214155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 -- 172.21.15.17:0/3189155350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5214072a10 msgr2=0x7f5214155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f52193cf640 1 --2- 172.21.15.17:0/3189155350 >> v2:172.21.15.163:3300/0 conn(0x7f52141528d0 0x7f52140724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5214072a10 0x7f5214155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 -- 172.21.15.17:0/3189155350 >> v2:172.21.15.163:3300/0 conn(0x7f52141528d0 msgr2=0x7f52140724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 --2- 172.21.15.17:0/3189155350 >> v2:172.21.15.163:3300/0 conn(0x7f52141528d0 0x7f52140724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.957+0000 7f5219bd0640 1 -- 172.21.15.17:0/3189155350 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52141594f0 con 0x7f52140c67f0 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.958+0000 7f5219bd0640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52140c67f0 0x7f5214155760 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f521406e3e0 tx=0x7f520c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7876678a69379158 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.958+0000 7f520bfff640 1 -- 172.21.15.17:0/3189155350 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c02cb00 con 0x7f52140c67f0 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.958+0000 7f520bfff640 1 -- 172.21.15.17:0/3189155350 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c02cca0 con 0x7f52140c67f0 2024-03-06T07:25:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.958+0000 7f520bfff640 1 -- 172.21.15.17:0/3189155350 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c034610 con 0x7f52140c67f0 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 -- 172.21.15.17:0/3189155350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52140c67f0 msgr2=0x7f5214155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52140c67f0 0x7f5214155760 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f521406e3e0 tx=0x7f520c02bf60 comp rx=0 tx=0).stop 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 -- 172.21.15.17:0/3189155350 shutdown_connections 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52140c67f0 0x7f5214155760 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 --2- 172.21.15.17:0/3189155350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5214072a10 0x7f5214155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 --2- 172.21.15.17:0/3189155350 >> v2:172.21.15.163:3300/0 conn(0x7f52141528d0 0x7f52140724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 -- 172.21.15.17:0/3189155350 >> 172.21.15.17:0/3189155350 conn(0x7f52140aa6a0 msgr2=0x7f52140aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 -- 172.21.15.17:0/3189155350 shutdown_connections 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.959+0000 7f521b633640 1 -- 172.21.15.17:0/3189155350 wait complete. 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 Processor -- start 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 -- start start 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52140c67f0 0x7f52141eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 0x7f5214155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140a8250 con 0x7f5214072a10 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c3390 con 0x7f52141528d0 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f521b633640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52141591a0 con 0x7f52140c67f0 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.960+0000 7f52193cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57016/0 (socket says 172.21.15.17:57016) 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 -- 172.21.15.17:0/83748227 learned_addr learned my addr 172.21.15.17:0/83748227 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 -- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52140c67f0 msgr2=0x7f52141eb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52140c67f0 0x7f52141eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 -- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 msgr2=0x7f5214155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f5219bd0640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 0x7f5214155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 0x7f5214155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 -- 172.21.15.17:0/83748227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52141eb8f0 con 0x7f5214072a10 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f5219bd0640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 0x7f5214155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f52193cf640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f521000c7f0 tx=0x7f521000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210015070 con 0x7f5214072a10 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5210004240 con 0x7f5214072a10 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5210007620 con 0x7f5214072a10 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52141ea910 con 0x7f5214072a10 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.961+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521414d5f0 con 0x7f5214072a10 2024-03-06T07:25:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.962+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52140ab370 con 0x7f5214072a10 2024-03-06T07:25:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.963+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52100079d0 con 0x7f5214072a10 2024-03-06T07:25:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.963+0000 7f5209ffb640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.963+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f521008a410 con 0x7f5214072a10 2024-03-06T07:25:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.963+0000 7f5218bce640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.964+0000 7f5218bce640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 0x7f51f806d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f521406dfd0 tx=0x7f5204008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:51.967+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f52140ab370 con 0x7f5214072a10 2024-03-06T07:25:52.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.147+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f521406e000 con 0x7f5214072a10 2024-03-06T07:25:52.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.148+0000 7f5209ffb640 1 -- 172.21.15.17:0/83748227 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52141eb8f0 con 0x7f5214072a10 2024-03-06T07:25:52.150 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:25:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 msgr2=0x7f51f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 0x7f51f806d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f521406dfd0 tx=0x7f5204008040 comp rx=0 tx=0).stop 2024-03-06T07:25:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 msgr2=0x7f52141ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f521000c7f0 tx=0x7f521000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 shutdown_connections 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51f806b4d0 0x7f51f806d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52141528d0 0x7f5214155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52140c67f0 0x7f52141eb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 --2- 172.21.15.17:0/83748227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5214072a10 0x7f52141ee910 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 >> 172.21.15.17:0/83748227 conn(0x7f52140aa6a0 msgr2=0x7f521406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 shutdown_connections 2024-03-06T07:25:52.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.150+0000 7f521b633640 1 -- 172.21.15.17:0/83748227 wait complete. 2024-03-06T07:25:52.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:25:52.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.262+0000 7f8a3098a640 1 Processor -- start 2024-03-06T07:25:52.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.262+0000 7f8a3098a640 1 -- start start 2024-03-06T07:25:52.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a3098a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a2c0c8a00 0x7f8a2c158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a3098a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a2c159460 0x7f8a2c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a3098a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c0c7760 con 0x7f8a2c152920 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a3098a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c1587b0 con 0x7f8a2c159460 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a3098a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c0c7a40 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a2c159460 0x7f8a2c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8a2c152920 0x7f8a2c155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52196/0 (socket says 172.21.15.17:52196) 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2a59c640 1 -- 172.21.15.17:0/2669203461 learned_addr learned my addr 172.21.15.17:0/2669203461 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2ad9d640 1 -- 172.21.15.17:0/2669203461 >> v2:172.21.15.163:3300/0 conn(0x7f8a2c0c8a00 msgr2=0x7f8a2c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2ad9d640 1 --2- 172.21.15.17:0/2669203461 >> v2:172.21.15.163:3300/0 conn(0x7f8a2c0c8a00 0x7f8a2c158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2ad9d640 1 -- 172.21.15.17:0/2669203461 >> v1:172.21.15.17:6789/0 conn(0x7f8a2c152920 legacy=0x7f8a2c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a2ad9d640 1 -- 172.21.15.17:0/2669203461 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c0c7d20 con 0x7f8a2c159460 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.263+0000 7f8a29d9b640 1 -- 172.21.15.17:0/2669203461 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206968206 0 0) 0x7f8a2c0c7760 con 0x7f8a2c152920 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.264+0000 7f8a2ad9d640 1 --2- 172.21.15.17:0/2669203461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c159460 0x7f8a2c0c5330 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c067c30 tx=0x7f8a2402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e7cae58fd7e84a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.264+0000 7f8a29d9b640 1 -- 172.21.15.17:0/2669203461 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24039070 con 0x7f8a2c159460 2024-03-06T07:25:52.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.264+0000 7f8a29d9b640 1 -- 172.21.15.17:0/2669203461 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2402cbf0 con 0x7f8a2c159460 2024-03-06T07:25:52.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.264+0000 7f8a29d9b640 1 -- 172.21.15.17:0/2669203461 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a24034440 con 0x7f8a2c159460 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 -- 172.21.15.17:0/2669203461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c159460 msgr2=0x7f8a2c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 --2- 172.21.15.17:0/2669203461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c159460 0x7f8a2c0c5330 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c067c30 tx=0x7f8a2402be40 comp rx=0 tx=0).stop 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 -- 172.21.15.17:0/2669203461 shutdown_connections 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 --2- 172.21.15.17:0/2669203461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c159460 0x7f8a2c0c5330 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 --2- 172.21.15.17:0/2669203461 >> v2:172.21.15.163:3300/0 conn(0x7f8a2c0c8a00 0x7f8a2c158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 -- 172.21.15.17:0/2669203461 >> 172.21.15.17:0/2669203461 conn(0x7f8a2c0ac560 msgr2=0x7f8a2c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 -- 172.21.15.17:0/2669203461 shutdown_connections 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.265+0000 7f8a3098a640 1 -- 172.21.15.17:0/2669203461 wait complete. 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.266+0000 7f8a3098a640 1 Processor -- start 2024-03-06T07:25:52.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.266+0000 7f8a3098a640 1 -- start start 2024-03-06T07:25:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a2c152920 0x7f8a2c14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a2c14a290 0x7f8a2c1edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0c7390 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c06d030 con 0x7f8a2c152920 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c0c7510 con 0x7f8a2c14a290 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57044/0 (socket says 172.21.15.17:57044) 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 -- 172.21.15.17:0/1903071759 learned_addr learned my addr 172.21.15.17:0/1903071759 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a23fff640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a2c152920 0x7f8a2c14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2ad9d640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a2c14a290 0x7f8a2c1edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 -- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a2c14a290 msgr2=0x7f8a2c1edeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a2c14a290 0x7f8a2c1edeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 -- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a2c152920 msgr2=0x7f8a2c14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a2c152920 0x7f8a2c14aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 -- 172.21.15.17:0/1903071759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c150280 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a2a59c640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c500 tx=0x7f8a1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c018070 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c007580 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c1eb330 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.267+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c004a20 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.268+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c151240 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.269+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c04f330 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.269+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a1c01b070 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.269+0000 7f8a22ffd640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 0x7f8a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.269+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8a1c089300 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.270+0000 7f8a23fff640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 0x7f8a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.270+0000 7f8a23fff640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 0x7f8a0c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c14edb0 tx=0x7f8a14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.274+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a1c0583d0 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.458+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a2c0c4900 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.458+0000 7f8a22ffd640 1 -- 172.21.15.17:0/1903071759 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8a1c057cd0 con 0x7f8a2c0c8a00 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:25:52.460 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:25:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 msgr2=0x7f8a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 0x7f8a0c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c14edb0 tx=0x7f8a14008040 comp rx=0 tx=0).stop 2024-03-06T07:25:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 msgr2=0x7f8a2c149c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00c500 tx=0x7f8a1c00c9c0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 shutdown_connections 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.460+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a0c06b4d0 0x7f8a0c06d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a2c14a290 0x7f8a2c1edeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a2c152920 0x7f8a2c14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 --2- 172.21.15.17:0/1903071759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a2c0c8a00 0x7f8a2c149c00 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 >> 172.21.15.17:0/1903071759 conn(0x7f8a2c0ac560 msgr2=0x7f8a2c0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 shutdown_connections 2024-03-06T07:25:52.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.461+0000 7f8a3098a640 1 -- 172.21.15.17:0/1903071759 wait complete. 2024-03-06T07:25:52.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:25:52.473 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:25:52.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.576+0000 7f2b8fa34640 1 Processor -- start 2024-03-06T07:25:52.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 -- start start 2024-03-06T07:25:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b881551d0 0x7f2b881566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b880c8f60 con 0x7f2b880ca000 2024-03-06T07:25:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b88156c00 con 0x7f2b881551d0 2024-03-06T07:25:52.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.577+0000 7f2b8fa34640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880c9240 con 0x7f2b8814f180 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b881551d0 0x7f2b881566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8d7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b881551d0 0x7f2b881566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:43212/0 (socket says 172.21.15.17:43212) 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8dfd1640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2b880ca000 0x7f2b8806c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:36848/0 (socket says 172.21.15.17:36848) 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8d7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:41800/0 (socket says 172.21.15.17:41800) 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 -- 172.21.15.17:0/1696479510 learned_addr learned my addr 172.21.15.17:0/1696479510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:52.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 -- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 msgr2=0x7f2b880c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 -- 172.21.15.17:0/1696479510 >> v1:172.21.15.163:6789/0 conn(0x7f2b880ca000 legacy=0x7f2b8806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b7ffff640 1 -- 172.21.15.17:0/1696479510 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101612233 0 0) 0x7f2b880c8f60 con 0x7f2b880ca000 2024-03-06T07:25:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8cfcf640 1 -- 172.21.15.17:0/1696479510 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880c9520 con 0x7f2b881551d0 2024-03-06T07:25:52.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.578+0000 7f2b8d7d0640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.579+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b881566c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009870 tx=0x7f2b7802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4db8ca7b0708cee0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.579+0000 7f2b7ffff640 1 -- 172.21.15.17:0/1696479510 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78034070 con 0x7f2b881551d0 2024-03-06T07:25:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.579+0000 7f2b7ffff640 1 -- 172.21.15.17:0/1696479510 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78030840 con 0x7f2b881551d0 2024-03-06T07:25:52.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.579+0000 7f2b7ffff640 1 -- 172.21.15.17:0/1696479510 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78030c10 con 0x7f2b881551d0 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 -- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 msgr2=0x7f2b881566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b881566c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009870 tx=0x7f2b7802b980 comp rx=0 tx=0).stop 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 -- 172.21.15.17:0/1696479510 shutdown_connections 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b881566c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/1696479510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b880c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 -- 172.21.15.17:0/1696479510 >> 172.21.15.17:0/1696479510 conn(0x7f2b88072d20 msgr2=0x7f2b880ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 -- 172.21.15.17:0/1696479510 shutdown_connections 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.580+0000 7f2b8fa34640 1 -- 172.21.15.17:0/1696479510 wait complete. 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.581+0000 7f2b8fa34640 1 Processor -- start 2024-03-06T07:25:52.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.581+0000 7f2b8fa34640 1 -- start start 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b881eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b8814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8814f560 con 0x7f2b880ca000 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b880c9ae0 con 0x7f2b881551d0 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8814f6e0 con 0x7f2b8814f180 2024-03-06T07:25:52.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57052/0 (socket says 172.21.15.17:57052) 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 -- 172.21.15.17:0/418262981 learned_addr learned my addr 172.21.15.17:0/418262981 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b881eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8dfd1640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b8814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 -- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 msgr2=0x7f2b881eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b881eb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 -- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 msgr2=0x7f2b8814d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b8814d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 -- 172.21.15.17:0/418262981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b880af520 con 0x7f2b880ca000 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b881eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8dfd1640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b8814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8d7d0640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ebe0 tx=0x7f2b8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84017070 con 0x7f2b880ca000 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b880af800 con 0x7f2b880ca000 2024-03-06T07:25:52.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8400cc30 con 0x7f2b880ca000 2024-03-06T07:25:52.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.582+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84013470 con 0x7f2b880ca000 2024-03-06T07:25:52.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.583+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8814de60 con 0x7f2b880ca000 2024-03-06T07:25:52.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.584+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8804f330 con 0x7f2b880ca000 2024-03-06T07:25:52.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.584+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b8401a070 con 0x7f2b880ca000 2024-03-06T07:25:52.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.584+0000 7f2b7dffb640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 0x7f2b6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:52.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.585+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2b84089f90 con 0x7f2b880ca000 2024-03-06T07:25:52.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.585+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 0x7f2b6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:52.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.585+0000 7f2b8cfcf640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 0x7f2b6c06d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2b78036000 tx=0x7f2b78008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:52.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.589+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b84058fe0 con 0x7f2b880ca000 2024-03-06T07:25:52.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:52.771+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b88159160 con 0x7f2b880ca000 2024-03-06T07:25:53.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.152+0000 7f2b7dffb640 1 -- 172.21.15.17:0/418262981 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v147) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2b840588e0 con 0x7f2b880ca000 2024-03-06T07:25:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:25:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.155+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 msgr2=0x7f2b6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.155+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 0x7f2b6c06d9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2b78036000 tx=0x7f2b78008030 comp rx=0 tx=0).stop 2024-03-06T07:25:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.155+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 msgr2=0x7f2b8814fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.155+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ebe0 tx=0x7f2b8400c520 comp rx=0 tx=0).stop 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 shutdown_connections 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b6c06b500 0x7f2b6c06d9c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b881551d0 0x7f2b8814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b8814f180 0x7f2b881eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 --2- 172.21.15.17:0/418262981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b880ca000 0x7f2b8814fea0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 >> 172.21.15.17:0/418262981 conn(0x7f2b88072d20 msgr2=0x7f2b881546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 shutdown_connections 2024-03-06T07:25:53.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.156+0000 7f2b8fa34640 1 -- 172.21.15.17:0/418262981 wait complete. 2024-03-06T07:25:53.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:25:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 Processor -- start 2024-03-06T07:25:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 -- start start 2024-03-06T07:25:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe340c57d0 0x7fbe34155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe34152dd0 con 0x7fbe341529f0 2024-03-06T07:25:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe34158b00 con 0x7fbe340c5370 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe3a7e2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe34159710 con 0x7fbe340c57d0 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe38d7f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbe340c57d0 0x7fbe34155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe33fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbe341529f0 0x7fbe340c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52206/0 (socket says 172.21.15.17:52206) 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.283+0000 7fbe33fff640 1 -- 172.21.15.17:0/3635351741 learned_addr learned my addr 172.21.15.17:0/3635351741 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.284+0000 7fbe38d7f640 1 -- 172.21.15.17:0/3635351741 >> v1:172.21.15.163:6789/0 conn(0x7fbe340c5370 legacy=0x7fbe34155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.284+0000 7fbe38d7f640 1 -- 172.21.15.17:0/3635351741 >> v1:172.21.15.17:6789/0 conn(0x7fbe341529f0 legacy=0x7fbe340c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.284+0000 7fbe32ffd640 1 -- 172.21.15.17:0/3635351741 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432837086 0 0) 0x7fbe34152dd0 con 0x7fbe341529f0 2024-03-06T07:25:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.284+0000 7fbe38d7f640 1 -- 172.21.15.17:0/3635351741 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34158de0 con 0x7fbe340c57d0 2024-03-06T07:25:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe38d7f640 1 --2- 172.21.15.17:0/3635351741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c57d0 0x7fbe34155c40 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009d90 tx=0x7fbe2c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c64d1ebec91230e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe32ffd640 1 -- 172.21.15.17:0/3635351741 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c039070 con 0x7fbe340c57d0 2024-03-06T07:25:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe32ffd640 1 -- 172.21.15.17:0/3635351741 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c02cbd0 con 0x7fbe340c57d0 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe32ffd640 1 -- 172.21.15.17:0/3635351741 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c034440 con 0x7fbe340c57d0 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/3635351741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c57d0 msgr2=0x7fbe34155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/3635351741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c57d0 0x7fbe34155c40 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009d90 tx=0x7fbe2c02be20 comp rx=0 tx=0).stop 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/3635351741 shutdown_connections 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/3635351741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c57d0 0x7fbe34155c40 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/3635351741 >> 172.21.15.17:0/3635351741 conn(0x7fbe340ac580 msgr2=0x7fbe340ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/3635351741 shutdown_connections 2024-03-06T07:25:53.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.285+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/3635351741 wait complete. 2024-03-06T07:25:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.286+0000 7fbe3a7e2640 1 Processor -- start 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 -- start start 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe340c57d0 0x7fbe34157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe341529f0 0x7fbe340c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340c68b0 con 0x7fbe340c5370 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe340c9000 con 0x7fbe341529f0 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe3a7e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe341593a0 con 0x7fbe340c57d0 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe33fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe33fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57068/0 (socket says 172.21.15.17:57068) 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.287+0000 7fbe33fff640 1 -- 172.21.15.17:0/513079888 learned_addr learned my addr 172.21.15.17:0/513079888 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe38d7f640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe341529f0 0x7fbe340c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe337fe640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe340c57d0 0x7fbe34157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 -- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe340c57d0 msgr2=0x7fbe34157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe340c57d0 0x7fbe34157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 -- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe341529f0 msgr2=0x7fbe340c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe341529f0 0x7fbe340c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 -- 172.21.15.17:0/513079888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340ad980 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe33fff640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c460 tx=0x7fbe2400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24018070 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24007500 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe34148a70 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24004a20 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.288+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe340c3dc0 con 0x7fbe340c5370 2024-03-06T07:25:53.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.289+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe340c3fc0 con 0x7fbe340c5370 2024-03-06T07:25:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.290+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe2401b070 con 0x7fbe340c5370 2024-03-06T07:25:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.290+0000 7fbe30ff9640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 0x7fbe2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.290+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fbe24089420 con 0x7fbe340c5370 2024-03-06T07:25:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.290+0000 7fbe337fe640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 0x7fbe2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:53.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.291+0000 7fbe337fe640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 0x7fbe2006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbe341eedc0 tx=0x7fbe28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:53.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.295+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe340c3fc0 con 0x7fbe340c5370 2024-03-06T07:25:53.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:53.478+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe3404f330 con 0x7fbe340c5370 2024-03-06T07:25:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.157+0000 7fbe30ff9640 1 -- 172.21.15.17:0/513079888 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v148) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe240582f0 con 0x7fbe340c5370 2024-03-06T07:25:54.158 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:25:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.159+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 msgr2=0x7fbe2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.159+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 0x7fbe2006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbe341eedc0 tx=0x7fbe28009040 comp rx=0 tx=0).stop 2024-03-06T07:25:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.159+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 msgr2=0x7fbe341edd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.159+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c460 tx=0x7fbe2400c920 comp rx=0 tx=0).stop 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 shutdown_connections 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbe2006b4d0 0x7fbe2006d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe341529f0 0x7fbe340c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe340c57d0 0x7fbe34157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 --2- 172.21.15.17:0/513079888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe340c5370 0x7fbe341edd00 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 >> 172.21.15.17:0/513079888 conn(0x7fbe340ac580 msgr2=0x7fbe340ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 shutdown_connections 2024-03-06T07:25:54.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.160+0000 7fbe3a7e2640 1 -- 172.21.15.17:0/513079888 wait complete. 2024-03-06T07:25:54.187 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:25:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.294+0000 7f34d170c640 1 Processor -- start 2024-03-06T07:25:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.294+0000 7f34d170c640 1 -- start start 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34d170c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f34cc14ece0 0x7f34cc0c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34d170c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 0x7f34cc154f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34d170c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc1524d0 con 0x7f34cc0c2810 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34d170c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34cc151710 con 0x7f34cc14ece0 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34d170c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc1519f0 con 0x7f34cc154af0 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34ca7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34cc0c2810 0x7f34cc0aa940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52222/0 (socket says 172.21.15.17:52222) 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34ca7fc640 1 -- 172.21.15.17:0/2638265862 learned_addr learned my addr 172.21.15.17:0/2638265862 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34cb7fe640 1 --2- 172.21.15.17:0/2638265862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 0x7f34cc154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34caffd640 1 --2- 172.21.15.17:0/2638265862 >> v2:172.21.15.163:3300/0 conn(0x7f34cc14ece0 0x7f34cc0c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34c9ffb640 1 -- 172.21.15.17:0/2638265862 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576101400 0 0) 0x7f34cc1524d0 con 0x7f34cc0c2810 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34c9ffb640 1 -- 172.21.15.17:0/2638265862 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b8003520 con 0x7f34cc0c2810 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34cb7fe640 1 -- 172.21.15.17:0/2638265862 >> v2:172.21.15.163:3300/0 conn(0x7f34cc14ece0 msgr2=0x7f34cc0c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34cb7fe640 1 --2- 172.21.15.17:0/2638265862 >> v2:172.21.15.163:3300/0 conn(0x7f34cc14ece0 0x7f34cc0c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34cb7fe640 1 -- 172.21.15.17:0/2638265862 >> v1:172.21.15.17:6789/0 conn(0x7f34cc0c2810 legacy=0x7f34cc0aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.295+0000 7f34cb7fe640 1 -- 172.21.15.17:0/2638265862 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc151cd0 con 0x7f34cc154af0 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.296+0000 7f34cb7fe640 1 --2- 172.21.15.17:0/2638265862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 0x7f34cc154f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00bfa0 tx=0x7f34bc02df80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6cbd75d264f38d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.296+0000 7f34c9ffb640 1 -- 172.21.15.17:0/2638265862 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc034070 con 0x7f34cc154af0 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.296+0000 7f34c9ffb640 1 -- 172.21.15.17:0/2638265862 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc02ed30 con 0x7f34cc154af0 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.296+0000 7f34c9ffb640 1 -- 172.21.15.17:0/2638265862 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc038600 con 0x7f34cc154af0 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 -- 172.21.15.17:0/2638265862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 msgr2=0x7f34cc154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 --2- 172.21.15.17:0/2638265862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 0x7f34cc154f80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00bfa0 tx=0x7f34bc02df80 comp rx=0 tx=0).stop 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 -- 172.21.15.17:0/2638265862 shutdown_connections 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 --2- 172.21.15.17:0/2638265862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc154af0 0x7f34cc154f80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 --2- 172.21.15.17:0/2638265862 >> v2:172.21.15.163:3300/0 conn(0x7f34cc14ece0 0x7f34cc0c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 -- 172.21.15.17:0/2638265862 >> 172.21.15.17:0/2638265862 conn(0x7f34cc06afc0 msgr2=0x7f34cc06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 -- 172.21.15.17:0/2638265862 shutdown_connections 2024-03-06T07:25:54.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.297+0000 7f34d170c640 1 -- 172.21.15.17:0/2638265862 wait complete. 2024-03-06T07:25:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.298+0000 7f34d170c640 1 Processor -- start 2024-03-06T07:25:54.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.298+0000 7f34d170c640 1 -- start start 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc0c2810 0x7f34cc1eb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34cc154af0 0x7f34cc1ede20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc06dd30 con 0x7f34cc14ece0 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc1554c0 con 0x7f34cc154af0 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34d170c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc1eaf90 con 0x7f34cc0c2810 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57078/0 (socket says 172.21.15.17:57078) 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 -- 172.21.15.17:0/245226661 learned_addr learned my addr 172.21.15.17:0/245226661 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34caffd640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc0c2810 0x7f34cc1eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34cb7fe640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34cc154af0 0x7f34cc1ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 -- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc0c2810 msgr2=0x7f34cc1eb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc0c2810 0x7f34cc1eb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 -- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34cc154af0 msgr2=0x7f34cc1ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34cc154af0 0x7f34cc1ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.299+0000 7f34ca7fc640 1 -- 172.21.15.17:0/245226661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc145a20 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34ca7fc640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c430 tx=0x7f34b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b4017070 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4005ce0 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc1ea980 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40072f0 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.300+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc1eac20 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.301+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc04f330 con 0x7f34cc14ece0 2024-03-06T07:25:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.301+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34b401a070 con 0x7f34cc14ece0 2024-03-06T07:25:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.302+0000 7f34c37fe640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 0x7f34ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.302+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f34b4089660 con 0x7f34cc14ece0 2024-03-06T07:25:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.302+0000 7f34caffd640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 0x7f34ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.302+0000 7f34caffd640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 0x7f34ac06d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f34cc151690 tx=0x7f34c4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:54.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.306+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34cc145a20 con 0x7f34cc14ece0 2024-03-06T07:25:54.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:54.525+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f34cc06deb0 con 0x7f34cc14ece0 2024-03-06T07:25:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.158+0000 7f34c37fe640 1 -- 172.21.15.17:0/245226661 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v134) v1 ==== 174+0+0 (secure 0 0 0) 0x7f34b4057df0 con 0x7f34cc14ece0 2024-03-06T07:25:55.160 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 23 and data pool 24 2024-03-06T07:25:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.164+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 msgr2=0x7f34ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.164+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 0x7f34ac06d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f34cc151690 tx=0x7f34c4009190 comp rx=0 tx=0).stop 2024-03-06T07:25:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.164+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 msgr2=0x7f34cc14f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.164+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f34b400c430 tx=0x7f34b400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.165+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 shutdown_connections 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.165+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f34ac06b500 0x7f34ac06d9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.166+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34cc154af0 0x7f34cc1ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.166+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34cc14ece0 0x7f34cc14f560 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.166+0000 7f34d170c640 1 --2- 172.21.15.17:0/245226661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34cc0c2810 0x7f34cc1eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.166+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 >> 172.21.15.17:0/245226661 conn(0x7f34cc06afc0 msgr2=0x7f34cc0c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.173+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 shutdown_connections 2024-03-06T07:25:55.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.173+0000 7f34d170c640 1 -- 172.21.15.17:0/245226661 wait complete. 2024-03-06T07:25:55.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.303+0000 7f7eacc88640 1 Processor -- start 2024-03-06T07:25:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.303+0000 7f7eacc88640 1 -- start start 2024-03-06T07:25:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7eacc88640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ea806d9f0 0x7f7ea80a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7eacc88640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea8158800 con 0x7f7ea80c3280 2024-03-06T07:25:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7eacc88640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea8158ae0 con 0x7f7ea80c3710 2024-03-06T07:25:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7eacc88640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea8158dc0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ea806d9f0 0x7f7ea80a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea5d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ea80c3280 0x7f7ea81551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52232/0 (socket says 172.21.15.17:52232) 2024-03-06T07:25:55.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea5d9b640 1 -- 172.21.15.17:0/193046709 learned_addr learned my addr 172.21.15.17:0/193046709 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea659c640 1 -- 172.21.15.17:0/193046709 >> v1:172.21.15.163:6789/0 conn(0x7f7ea80c3710 legacy=0x7f7ea81558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea659c640 1 -- 172.21.15.17:0/193046709 >> v1:172.21.15.17:6789/0 conn(0x7f7ea80c3280 legacy=0x7f7ea81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea659c640 1 -- 172.21.15.17:0/193046709 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea81590a0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.304+0000 7f7ea559a640 1 -- 172.21.15.17:0/193046709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035252116 0 0) 0x7f7ea8158800 con 0x7f7ea80c3280 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.305+0000 7f7ea659c640 1 --2- 172.21.15.17:0/193046709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea80a7cd0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00097a0 tx=0x7f7ea002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e5f1dd2d3fb9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.305+0000 7f7ea559a640 1 -- 172.21.15.17:0/193046709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0038070 con 0x7f7ea806d9f0 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.305+0000 7f7ea559a640 1 -- 172.21.15.17:0/193046709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0031700 con 0x7f7ea806d9f0 2024-03-06T07:25:55.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.305+0000 7f7ea559a640 1 -- 172.21.15.17:0/193046709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0031ad0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 -- 172.21.15.17:0/193046709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 msgr2=0x7f7ea80a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 --2- 172.21.15.17:0/193046709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea80a7cd0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00097a0 tx=0x7f7ea002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 -- 172.21.15.17:0/193046709 shutdown_connections 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 --2- 172.21.15.17:0/193046709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea80a7cd0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 -- 172.21.15.17:0/193046709 >> 172.21.15.17:0/193046709 conn(0x7f7ea80aa630 msgr2=0x7f7ea80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 -- 172.21.15.17:0/193046709 shutdown_connections 2024-03-06T07:25:55.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.306+0000 7f7eacc88640 1 -- 172.21.15.17:0/193046709 wait complete. 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.307+0000 7f7eacc88640 1 Processor -- start 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.307+0000 7f7eacc88640 1 -- start start 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 0x7f7ea80a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea80c3710 0x7f7ea80adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806df50 con 0x7f7ea806d9f0 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806ddd0 con 0x7f7ea80c3280 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7eacc88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80a8210 con 0x7f7ea80c3710 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57102/0 (socket says 172.21.15.17:57102) 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 -- 172.21.15.17:0/4288590341 learned_addr learned my addr 172.21.15.17:0/4288590341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea5d9b640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 0x7f7ea80a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea6d9d640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea80c3710 0x7f7ea80adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 -- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea80c3710 msgr2=0x7f7ea80adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea80c3710 0x7f7ea80adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 -- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 msgr2=0x7f7ea80a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 0x7f7ea80a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 -- 172.21.15.17:0/4288590341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea806e2f0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea5d9b640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 0x7f7ea80a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:55.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.308+0000 7f7ea659c640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.309+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0038040 con 0x7f7ea806d9f0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.309+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0033a30 con 0x7f7ea806d9f0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.309+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea806f9e0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.309+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0006e10 con 0x7f7ea806d9f0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.309+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea806ffa0 con 0x7f7ea806d9f0 2024-03-06T07:25:55.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.310+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea804f330 con 0x7f7ea806d9f0 2024-03-06T07:25:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.310+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ea0004030 con 0x7f7ea806d9f0 2024-03-06T07:25:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.311+0000 7f7e9effd640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 0x7f7e8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.311+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ea00abf80 con 0x7f7ea806d9f0 2024-03-06T07:25:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.311+0000 7f7ea5d9b640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 0x7f7e8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.311+0000 7f7ea5d9b640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 0x7f7e8806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80ae740 tx=0x7f7e90007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.315+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ea0043020 con 0x7f7ea806d9f0 2024-03-06T07:25:55.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.496+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ea80ab360 con 0x7f7ea806d9f0 2024-03-06T07:25:55.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.497+0000 7f7e9effd640 1 -- 172.21.15.17:0/4288590341 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f7ea0030070 con 0x7f7ea806d9f0 2024-03-06T07:25:55.499 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:55.499 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":149,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:54.526447+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 msgr2=0x7f7e8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 0x7f7e8806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80ae740 tx=0x7f7e90007c10 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 msgr2=0x7f7ea81500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea00046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 shutdown_connections 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e8806b4d0 0x7f7e8806d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ea80c3710 0x7f7ea80adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ea80c3280 0x7f7ea80a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 --2- 172.21.15.17:0/4288590341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ea806d9f0 0x7f7ea81500f0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 >> 172.21.15.17:0/4288590341 conn(0x7f7ea80aa630 msgr2=0x7f7ea80c7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 shutdown_connections 2024-03-06T07:25:55.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.499+0000 7f7eacc88640 1 -- 172.21.15.17:0/4288590341 wait complete. 2024-03-06T07:25:55.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.614+0000 7fe207e4e640 1 Processor -- start 2024-03-06T07:25:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.614+0000 7fe207e4e640 1 -- start start 2024-03-06T07:25:55.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe207e4e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe200159100 0x7fe2000c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe207e4e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000c7c60 con 0x7fe2001527b0 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe207e4e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000c8060 con 0x7fe2000c7180 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe207e4e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000c8df0 con 0x7fe200159100 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe205bea640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe2001527b0 0x7fe200155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52242/0 (socket says 172.21.15.17:52242) 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.615+0000 7fe205bea640 1 -- 172.21.15.17:0/2381830008 learned_addr learned my addr 172.21.15.17:0/2381830008 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2063eb640 1 --2- 172.21.15.17:0/2381830008 >> v2:172.21.15.163:3300/0 conn(0x7fe200159100 0x7fe2000c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264912078 0 0) 0x7fe1f8001110 con 0x7fe2001527b0 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8003520 con 0x7fe2001527b0 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 480916044 0 0) 0x7fe1e8003520 con 0x7fe2001527b0 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 >> v2:172.21.15.163:3300/0 conn(0x7fe200159100 msgr2=0x7fe2000c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 --2- 172.21.15.17:0/2381830008 >> v2:172.21.15.163:3300/0 conn(0x7fe200159100 0x7fe2000c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 >> v1:172.21.15.163:6789/0 conn(0x7fe2000c7180 legacy=0x7fe2000c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000c76a0 con 0x7fe2001527b0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe1f8001840 con 0x7fe2001527b0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 >> v1:172.21.15.17:6789/0 conn(0x7fe2001527b0 legacy=0x7fe200155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80039b0 0x7fe1e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe1e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80051c0 0x7fe1e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c8df0 con 0x7fe1e8004800 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e800c160 con 0x7fe1e80039b0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e800c360 con 0x7fe1e80051c0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe1e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe205bea640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80039b0 0x7fe1e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2063eb640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80051c0 0x7fe1e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 -- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80051c0 msgr2=0x7fe1e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e80051c0 0x7fe1e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 -- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80039b0 msgr2=0x7fe1e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80039b0 0x7fe1e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2053e9640 1 -- 172.21.15.17:0/2381830008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8003520 con 0x7fe1e8004800 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.616+0000 7fe2063eb640 1 -- 172.21.15.17:0/2381830008 reap_dead start 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.617+0000 7fe2053e9640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe1e8004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4001750 tx=0x7fe1e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b5070f0af3195d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.617+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4001370 con 0x7fe1e8004800 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.617+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f40052d0 con 0x7fe1e8004800 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.617+0000 7fe204be8640 1 -- 172.21.15.17:0/2381830008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4003830 con 0x7fe1e8004800 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 -- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 msgr2=0x7fe1e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe1e8004c80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4001750 tx=0x7fe1e8005830 comp rx=0 tx=0).stop 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 -- 172.21.15.17:0/2381830008 shutdown_connections 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 --2- 172.21.15.17:0/2381830008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe1e8004c80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 -- 172.21.15.17:0/2381830008 >> 172.21.15.17:0/2381830008 conn(0x7fe2000ac560 msgr2=0x7fe2000ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 -- 172.21.15.17:0/2381830008 shutdown_connections 2024-03-06T07:25:55.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.618+0000 7fe207e4e640 1 -- 172.21.15.17:0/2381830008 wait complete. 2024-03-06T07:25:55.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.619+0000 7fe207e4e640 1 Processor -- start 2024-03-06T07:25:55.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.619+0000 7fe207e4e640 1 -- start start 2024-03-06T07:25:55.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 0x7fe2000afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000adc80 0x7fe2000ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200155600 con 0x7fe1e8004800 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe200155320 con 0x7fe2000af5d0 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe200152b90 con 0x7fe2000adc80 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57114/0 (socket says 172.21.15.17:57114) 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 -- 172.21.15.17:0/637759762 learned_addr learned my addr 172.21.15.17:0/637759762 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe2053e9640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 0x7fe2000afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe2063eb640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000adc80 0x7fe2000ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 -- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000adc80 msgr2=0x7fe2000ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000adc80 0x7fe2000ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 -- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 msgr2=0x7fe2000afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 0x7fe2000afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 -- 172.21.15.17:0/637759762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20014fae0 con 0x7fe1e8004800 2024-03-06T07:25:55.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe2053e9640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 0x7fe2000afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe205bea640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007290 tx=0x7fe1f8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8001230 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8018070 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800fba0 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.620+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000c4a30 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.621+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2001eb830 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.622+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20004f330 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.622+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1f800fd40 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.622+0000 7fe1fe7fc640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.622+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1f808a270 con 0x7fe1e8004800 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.623+0000 7fe2053e9640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.623+0000 7fe2053e9640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.627+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe2000c7c60 con 0x7fe1e8004800 2024-03-06T07:25:55.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.807+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe20006ce00 con 0x7fe1e8004800 2024-03-06T07:25:55.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.809+0000 7fe1fe7fc640 1 -- 172.21.15.17:0/637759762 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fe1f8058ff0 con 0x7fe1e8004800 2024-03-06T07:25:55.810 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:55.810 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":149,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:54.526447+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:55.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 msgr2=0x7fe1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009030 comp rx=0 tx=0).stop 2024-03-06T07:25:55.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 msgr2=0x7fe2000c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007290 tx=0x7fe1f8007750 comp rx=0 tx=0).stop 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 shutdown_connections 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2000adc80 0x7fe2000ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2000af5d0 0x7fe2000afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 --2- 172.21.15.17:0/637759762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e8004800 0x7fe2000c44f0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 >> 172.21.15.17:0/637759762 conn(0x7fe2000ac560 msgr2=0x7fe2000c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 shutdown_connections 2024-03-06T07:25:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.810+0000 7fe207e4e640 1 -- 172.21.15.17:0/637759762 wait complete. 2024-03-06T07:25:55.823 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:25:55.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.925+0000 7f9dcf9e2640 1 Processor -- start 2024-03-06T07:25:55.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.925+0000 7f9dcf9e2640 1 -- start start 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcf9e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc80c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcf9e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dc80a8fd0 0x7f9dc806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcf9e2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80c5d30 con 0x7f9dc806e3f0 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcf9e2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80c0f00 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcf9e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80c6940 con 0x7f9dc81571e0 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dc80a8fd0 0x7f9dc806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9dc80a8fd0 0x7f9dc806deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57118/0 (socket says 172.21.15.17:57118) 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 -- 172.21.15.17:0/295316256 learned_addr learned my addr 172.21.15.17:0/295316256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dcd77e640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc80c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 -- 172.21.15.17:0/295316256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 msgr2=0x7f9dc80c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc80c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 -- 172.21.15.17:0/295316256 >> v1:172.21.15.163:6789/0 conn(0x7f9dc806e3f0 legacy=0x7f9dc8070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:55.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.926+0000 7f9dccf7d640 1 -- 172.21.15.17:0/295316256 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80c6c20 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.927+0000 7f9dccf7d640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc806deb0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009870 tx=0x7f9dbc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b4f3b5ed05edfe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.927+0000 7f9dc7fff640 1 -- 172.21.15.17:0/295316256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc038070 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.927+0000 7f9dc7fff640 1 -- 172.21.15.17:0/295316256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc031740 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.927+0000 7f9dc7fff640 1 -- 172.21.15.17:0/295316256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc02f050 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/295316256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 msgr2=0x7f9dc806deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc806deb0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009870 tx=0x7f9dbc02b7e0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/295316256 shutdown_connections 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc806deb0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/295316256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc80c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/295316256 >> 172.21.15.17:0/295316256 conn(0x7f9dc806cb60 msgr2=0x7f9dc80a86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/295316256 shutdown_connections 2024-03-06T07:25:55.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.928+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/295316256 wait complete. 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.929+0000 7f9dcf9e2640 1 Processor -- start 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.929+0000 7f9dcf9e2640 1 -- start start 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dc806e3f0 0x7f9dc81eca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc81541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80c6010 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80c7420 con 0x7f9dc806e3f0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcf9e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc81575c0 con 0x7f9dc81571e0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57128/0 (socket says 172.21.15.17:57128) 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 -- 172.21.15.17:0/1020212921 learned_addr learned my addr 172.21.15.17:0/1020212921 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcd77e640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dc806e3f0 0x7f9dc81eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dcdf7f640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc81541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 -- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 msgr2=0x7f9dc81541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc81541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 -- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dc806e3f0 msgr2=0x7f9dc81eca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dc806e3f0 0x7f9dc81eca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.930+0000 7f9dccf7d640 1 -- 172.21.15.17:0/1020212921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc81ecf40 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dccf7d640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc042070 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc033a70 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc81ea4a0 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc038040 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.931+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc80c0ad0 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.932+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc804f330 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.932+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dbc045070 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.933+0000 7f9dc5ffb640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.933+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dbc0abfd0 con 0x7f9dc80a8fd0 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.933+0000 7f9dcd77e640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:55.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.934+0000 7f9dcd77e640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 0x7f9db406d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9dc81ebfc0 tx=0x7f9dc0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:55.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:55.937+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9dbc04c030 con 0x7f9dc80a8fd0 2024-03-06T07:25:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.157+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9dc8156930 con 0x7f9dc80a8fd0 2024-03-06T07:25:56.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.176+0000 7f9dc5ffb640 1 -- 172.21.15.17:0/1020212921 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v135) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9dbc07a700 con 0x7f9dc80a8fd0 2024-03-06T07:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 msgr2=0x7f9db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 0x7f9db406d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9dc81ebfc0 tx=0x7f9dc0008040 comp rx=0 tx=0).stop 2024-03-06T07:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 msgr2=0x7f9dc81e9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc000c00 tx=0x7f9dbc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 shutdown_connections 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9db406b4d0 0x7f9db406d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dc81571e0 0x7f9dc81541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9dc80a8fd0 0x7f9dc81e9f60 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 --2- 172.21.15.17:0/1020212921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dc806e3f0 0x7f9dc81eca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.178+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 >> 172.21.15.17:0/1020212921 conn(0x7f9dc806cb60 msgr2=0x7f9dc8154f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.179+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 shutdown_connections 2024-03-06T07:25:56.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.179+0000 7f9dcf9e2640 1 -- 172.21.15.17:0/1020212921 wait complete. 2024-03-06T07:25:56.191 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:25:56.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:25:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.293+0000 7f6020f93640 1 Processor -- start 2024-03-06T07:25:56.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 -- start start 2024-03-06T07:25:56.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f601c0c1180 0x7f601c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c06b500 0x7f601c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c0c19f0 con 0x7f601c0c3a20 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c0c1cd0 con 0x7f601c0c1180 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.294+0000 7f6020f93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c6960 con 0x7f601c06b500 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f6019d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f601c0c1180 0x7f601c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f601a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f601c0c3a20 0x7f601c155b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52244/0 (socket says 172.21.15.17:52244) 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f601a59c640 1 -- 172.21.15.17:0/4244627281 learned_addr learned my addr 172.21.15.17:0/4244627281 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f601ad9d640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c06b500 0x7f601c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f6019d9b640 1 -- 172.21.15.17:0/4244627281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c06b500 msgr2=0x7f601c0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f6019d9b640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c06b500 0x7f601c0c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f6019d9b640 1 -- 172.21.15.17:0/4244627281 >> v1:172.21.15.17:6789/0 conn(0x7f601c0c3a20 legacy=0x7f601c155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f6019d9b640 1 -- 172.21.15.17:0/4244627281 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0c6c40 con 0x7f601c0c1180 2024-03-06T07:25:56.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.295+0000 7f601959a640 1 -- 172.21.15.17:0/4244627281 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374652000 0 0) 0x7f601c0c19f0 con 0x7f601c0c3a20 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6019d9b640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c0c1180 0x7f601c06afc0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f60100097a0 tx=0x7f601002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80df504406a24f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f601959a640 1 -- 172.21.15.17:0/4244627281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100315a0 con 0x7f601c0c1180 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f601959a640 1 -- 172.21.15.17:0/4244627281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010037070 con 0x7f601c0c1180 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f601959a640 1 -- 172.21.15.17:0/4244627281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010031960 con 0x7f601c0c1180 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 -- 172.21.15.17:0/4244627281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c0c1180 msgr2=0x7f601c06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c0c1180 0x7f601c06afc0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f60100097a0 tx=0x7f601002b840 comp rx=0 tx=0).stop 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 -- 172.21.15.17:0/4244627281 shutdown_connections 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c06b500 0x7f601c0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 --2- 172.21.15.17:0/4244627281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c0c1180 0x7f601c06afc0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.296+0000 7f6020f93640 1 -- 172.21.15.17:0/4244627281 >> 172.21.15.17:0/4244627281 conn(0x7f601c0a94d0 msgr2=0x7f601c0a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.297+0000 7f6020f93640 1 -- 172.21.15.17:0/4244627281 shutdown_connections 2024-03-06T07:25:56.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.297+0000 7f6020f93640 1 -- 172.21.15.17:0/4244627281 wait complete. 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 Processor -- start 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 -- start start 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c0c1180 0x7f601c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 0x7f601c06ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c3f80 con 0x7f601c06b500 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c3e00 con 0x7f601c0c3a20 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.298+0000 7f6020f93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c7980 con 0x7f601c0c1180 2024-03-06T07:25:56.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57150/0 (socket says 172.21.15.17:57150) 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 -- 172.21.15.17:0/2080043183 learned_addr learned my addr 172.21.15.17:0/2080043183 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601ad9d640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 0x7f601c06ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f6019d9b640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c0c1180 0x7f601c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 -- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c0c1180 msgr2=0x7f601c0abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c0c1180 0x7f601c0abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 -- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 msgr2=0x7f601c06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 0x7f601c06ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 -- 172.21.15.17:0/2080043183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c14f8f0 con 0x7f601c06b500 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601ad9d640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 0x7f601c06ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f601a59c640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f600c00c460 tx=0x7f600c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c018070 con 0x7f601c06b500 2024-03-06T07:25:56.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c007500 con 0x7f601c06b500 2024-03-06T07:25:56.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c14fb90 con 0x7f601c06b500 2024-03-06T07:25:56.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c004a20 con 0x7f601c06b500 2024-03-06T07:25:56.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.299+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c0731c0 con 0x7f601c06b500 2024-03-06T07:25:56.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.300+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c04f330 con 0x7f601c06b500 2024-03-06T07:25:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.301+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f600c01b070 con 0x7f601c06b500 2024-03-06T07:25:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.301+0000 7f600affd640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.301+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f600c089570 con 0x7f601c06b500 2024-03-06T07:25:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.301+0000 7f6019d9b640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:56.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.302+0000 7f6019d9b640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6010036000 tx=0x7f601003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:56.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.306+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f600c0582f0 con 0x7f601c06b500 2024-03-06T07:25:56.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:56.621+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f601c0c0f60 con 0x7f601c06b500 2024-03-06T07:25:57.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.182+0000 7f600affd640 1 -- 172.21.15.17:0/2080043183 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v136) v1 ==== 122+0+0 (secure 0 0 0) 0x7f600c057bf0 con 0x7f601c06b500 2024-03-06T07:25:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.184+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 msgr2=0x7f5ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.184+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6010036000 tx=0x7f601003d040 comp rx=0 tx=0).stop 2024-03-06T07:25:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.184+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 msgr2=0x7f601c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.184+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f600c00c460 tx=0x7f600c00c920 comp rx=0 tx=0).stop 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 shutdown_connections 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f601c0c3a20 0x7f601c06ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f601c0c1180 0x7f601c0abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 --2- 172.21.15.17:0/2080043183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c06b500 0x7f601c072c80 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 >> 172.21.15.17:0/2080043183 conn(0x7f601c0a94d0 msgr2=0x7f601c0c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 shutdown_connections 2024-03-06T07:25:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.185+0000 7f6020f93640 1 -- 172.21.15.17:0/2080043183 wait complete. 2024-03-06T07:25:57.197 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:25:57.197 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:57.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.311+0000 7fcc03d8a640 1 Processor -- start 2024-03-06T07:25:57.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 -- start start 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbfc06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcbfc0720a0 0x7fcbfc06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc0c1ae0 con 0x7fcbfc152520 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc0c1dc0 con 0x7fcbfc0720a0 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.312+0000 7fcc03d8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06e450 con 0x7fcbfc155c80 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01b26640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcbfc152520 0x7fcbfc0c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52250/0 (socket says 172.21.15.17:52250) 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01b26640 1 -- 172.21.15.17:0/1087325849 learned_addr learned my addr 172.21.15.17:0/1087325849 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> v2:172.21.15.163:3300/0 conn(0x7fcbfc0720a0 0x7fcbfc06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01325640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbfc06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167454535 0 0) 0x7fcbfc0c1ae0 con 0x7fcbfc152520 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe8003520 con 0x7fcbfc152520 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301120114 0 0) 0x7fcbe8003520 con 0x7fcbfc152520 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 msgr2=0x7fcbfc06ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbfc06ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 >> v2:172.21.15.163:3300/0 conn(0x7fcbfc0720a0 msgr2=0x7fcbfc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 --2- 172.21.15.17:0/1087325849 >> v2:172.21.15.163:3300/0 conn(0x7fcbfc0720a0 0x7fcbfc06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc06e730 con 0x7fcbfc152520 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fcbf0001670 con 0x7fcbfc152520 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> v2:172.21.15.163:3300/0 conn(0x7fcbfc0720a0 0x7fcbfc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01325640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbfc06ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 >> v1:172.21.15.17:6789/0 conn(0x7fcbfc152520 legacy=0x7fcbfc0c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe80039b0 0x7fcbe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbe8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbe8004d40 0x7fcbe80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0c1dc0 con 0x7fcbe8004d40 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06e450 con 0x7fcbe80039b0 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe800b9c0 con 0x7fcbfc155c80 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbe8004d40 0x7fcbe80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01b26640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe80039b0 0x7fcbe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.313+0000 7fcc01325640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbe8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 -- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 msgr2=0x7fcbe8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbfc155c80 0x7fcbe8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 -- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe80039b0 msgr2=0x7fcbe8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe80039b0 0x7fcbe8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 -- 172.21.15.17:0/1087325849 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8003520 con 0x7fcbe8004d40 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.314+0000 7fcc02327640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbe8004d40 0x7fcbe80051c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400e350 tx=0x7fcbe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9502103f2115205 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4039070 con 0x7fcbe8004d40 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf402ce00 con 0x7fcbe8004d40 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc00b24640 1 -- 172.21.15.17:0/1087325849 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4009de0 con 0x7fcbe8004d40 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 -- 172.21.15.17:0/1087325849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbe8004d40 msgr2=0x7fcbe80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/1087325849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbe8004d40 0x7fcbe80051c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400e350 tx=0x7fcbe8005830 comp rx=0 tx=0).stop 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc02327640 1 -- 172.21.15.17:0/1087325849 reap_dead start 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 -- 172.21.15.17:0/1087325849 shutdown_connections 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 -- 172.21.15.17:0/1087325849 >> 172.21.15.17:0/1087325849 conn(0x7fcbfc0c76b0 msgr2=0x7fcbfc0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 -- 172.21.15.17:0/1087325849 shutdown_connections 2024-03-06T07:25:57.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.315+0000 7fcc03d8a640 1 -- 172.21.15.17:0/1087325849 wait complete. 2024-03-06T07:25:57.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.316+0000 7fcc03d8a640 1 Processor -- start 2024-03-06T07:25:57.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.316+0000 7fcc03d8a640 1 -- start start 2024-03-06T07:25:57.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe8004d40 0x7fcbfc1eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbfc157770 0x7fcbfc157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc152a80 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc152900 con 0x7fcbfc157770 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc03d8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc06b500 con 0x7fcbe8004d40 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57170/0 (socket says 172.21.15.17:57170) 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 -- 172.21.15.17:0/4050494624 learned_addr learned my addr 172.21.15.17:0/4050494624 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01b26640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe8004d40 0x7fcbfc1eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc02327640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbfc157770 0x7fcbfc157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 -- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe8004d40 msgr2=0x7fcbfc1eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe8004d40 0x7fcbfc1eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 -- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbfc157770 msgr2=0x7fcbfc157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbfc157770 0x7fcbfc157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.317+0000 7fcc01325640 1 -- 172.21.15.17:0/4050494624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc158820 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcc01325640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80119d0 tx=0x7fcbf8011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf800fa90 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf800fc30 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc158b00 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8004030 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.318+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc156100 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.319+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc04f330 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.319+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbf8012040 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.320+0000 7fcbee7fc640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.320+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcbf808a590 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.320+0000 7fcc01b26640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.320+0000 7fcc01b26640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 0x7fcbe006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0adac0 tx=0x7fcbf00078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.324+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbf8019070 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.573+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbfc14d750 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.574+0000 7fcbee7fc640 1 -- 172.21.15.17:0/4050494624 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4981 (secure 0 0 0) 0x7fcbf8058cc0 con 0x7fcbfc14d1e0 2024-03-06T07:25:57.575 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:57.576 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":136,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/81289984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":81289984},{"type":"v1","addr":"172.21.15.17:6834","nonce":81289984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:25:56.622881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8781":{"gid":8781,"name":"c","rank":0,"incarnation":134,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2429596225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2429596225},{"type":"v1","addr":"172.21.15.17:6835","nonce":2429596225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8781,"qdb_cluster":[8781]},"id":11}]} 2024-03-06T07:25:57.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 136 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 msgr2=0x7fcbe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 0x7fcbe006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0adac0 tx=0x7fcbf00078f0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 msgr2=0x7fcbfc1e1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80119d0 tx=0x7fcbf8011dd0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 shutdown_connections 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcbe006b4d0 0x7fcbe006d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbfc157770 0x7fcbfc157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcbfc14d1e0 0x7fcbfc1e1df0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 --2- 172.21.15.17:0/4050494624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe8004d40 0x7fcbfc1eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 >> 172.21.15.17:0/4050494624 conn(0x7fcbfc0c76b0 msgr2=0x7fcbfc152e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 shutdown_connections 2024-03-06T07:25:57.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.576+0000 7fcc03d8a640 1 -- 172.21.15.17:0/4050494624 wait complete. 2024-03-06T07:25:57.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:25:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.689+0000 7fa94c5e9640 1 Processor -- start 2024-03-06T07:25:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.689+0000 7fa94c5e9640 1 -- start start 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa944152920 0x7fa944155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa9440c8a00 0x7fa944158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944159460 0x7fa9440c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9441587b0 con 0x7fa944152920 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa944156180 con 0x7fa9440c8a00 2024-03-06T07:25:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94c5e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440c7760 con 0x7fa944159460 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa944152920 0x7fa944155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa944152920 0x7fa944155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57182/0 (socket says 172.21.15.17:57182) 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 -- 172.21.15.17:0/3907316431 learned_addr learned my addr 172.21.15.17:0/3907316431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa949b84640 1 --2- 172.21.15.17:0/3907316431 >> v2:172.21.15.163:3300/0 conn(0x7fa9440c8a00 0x7fa944158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94ab86640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944159460 0x7fa9440c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 -- 172.21.15.17:0/3907316431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944159460 msgr2=0x7fa9440c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944159460 0x7fa9440c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 -- 172.21.15.17:0/3907316431 >> v2:172.21.15.163:3300/0 conn(0x7fa9440c8a00 msgr2=0x7fa944158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 --2- 172.21.15.17:0/3907316431 >> v2:172.21.15.163:3300/0 conn(0x7fa9440c8a00 0x7fa944158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.690+0000 7fa94a385640 1 -- 172.21.15.17:0/3907316431 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9440c7a40 con 0x7fa944152920 2024-03-06T07:25:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.691+0000 7fa94a385640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa944152920 0x7fa944155c40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa944067c30 tx=0x7fa93802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f13a07966e58c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.691+0000 7fa949383640 1 -- 172.21.15.17:0/3907316431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93802e070 con 0x7fa944152920 2024-03-06T07:25:57.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.691+0000 7fa949383640 1 -- 172.21.15.17:0/3907316431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938037070 con 0x7fa944152920 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.691+0000 7fa949383640 1 -- 172.21.15.17:0/3907316431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938031a00 con 0x7fa944152920 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 -- 172.21.15.17:0/3907316431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa944152920 msgr2=0x7fa944155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa944152920 0x7fa944155c40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa944067c30 tx=0x7fa93802b880 comp rx=0 tx=0).stop 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 -- 172.21.15.17:0/3907316431 shutdown_connections 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944159460 0x7fa9440c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/3907316431 >> v2:172.21.15.163:3300/0 conn(0x7fa9440c8a00 0x7fa944158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/3907316431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa944152920 0x7fa944155c40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 -- 172.21.15.17:0/3907316431 >> 172.21.15.17:0/3907316431 conn(0x7fa9440ac560 msgr2=0x7fa9440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 -- 172.21.15.17:0/3907316431 shutdown_connections 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.692+0000 7fa94c5e9640 1 -- 172.21.15.17:0/3907316431 wait complete. 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.693+0000 7fa94c5e9640 1 Processor -- start 2024-03-06T07:25:57.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.693+0000 7fa94c5e9640 1 -- start start 2024-03-06T07:25:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944152920 0x7fa9441ee440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 0x7fa944155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94406d030 con 0x7fa9440c8a00 2024-03-06T07:25:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa944158a00 con 0x7fa944152920 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9440c7390 con 0x7fa944155660 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57188/0 (socket says 172.21.15.17:57188) 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 -- 172.21.15.17:0/246468680 learned_addr learned my addr 172.21.15.17:0/246468680 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94ab86640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 0x7fa944155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa949b84640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944152920 0x7fa9441ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 -- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 msgr2=0x7fa944155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 0x7fa944155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 -- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944152920 msgr2=0x7fa9441ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944152920 0x7fa9441ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 -- 172.21.15.17:0/246468680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9441eec80 con 0x7fa9440c8a00 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94ab86640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 0x7fa944155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94a385640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa938000c00 tx=0x7fa9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938032670 con 0x7fa9440c8a00 2024-03-06T07:25:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa938032850 con 0x7fa9440c8a00 2024-03-06T07:25:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.694+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9441584f0 con 0x7fa9440c8a00 2024-03-06T07:25:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.695+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938032c40 con 0x7fa9440c8a00 2024-03-06T07:25:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.695+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa944158210 con 0x7fa9440c8a00 2024-03-06T07:25:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.695+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94404f330 con 0x7fa9440c8a00 2024-03-06T07:25:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.696+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9380453f0 con 0x7fa9440c8a00 2024-03-06T07:25:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.696+0000 7fa936ffd640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 0x7fa92806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.696+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9380ac0f0 con 0x7fa9440c8a00 2024-03-06T07:25:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.697+0000 7fa949b84640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 0x7fa92806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.697+0000 7fa949b84640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 0x7fa92806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa940006fd0 tx=0x7fa940008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:57.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:57.701+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa938040070 con 0x7fa9440c8a00 2024-03-06T07:25:58.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.030+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa94406cda0 con 0x7fa9440c8a00 2024-03-06T07:25:58.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.031+0000 7fa936ffd640 1 -- 172.21.15.17:0/246468680 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16884 (secure 0 0 0) 0x7fa93807ae70 con 0x7fa9440c8a00 2024-03-06T07:25:58.033 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:58.033 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":150,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:56.159340+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 msgr2=0x7fa92806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 0x7fa92806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa940006fd0 tx=0x7fa940008040 comp rx=0 tx=0).stop 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 msgr2=0x7fa944145840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa938000c00 tx=0x7fa9380046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 shutdown_connections 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa92806b4d0 0x7fa92806d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa944155660 0x7fa944155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa944152920 0x7fa9441ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 --2- 172.21.15.17:0/246468680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9440c8a00 0x7fa944145840 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 >> 172.21.15.17:0/246468680 conn(0x7fa9440ac560 msgr2=0x7fa944151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 shutdown_connections 2024-03-06T07:25:58.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.033+0000 7fa94c5e9640 1 -- 172.21.15.17:0/246468680 wait complete. 2024-03-06T07:25:58.046 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:25:58.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:58.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.146+0000 7faeb788d640 1 Processor -- start 2024-03-06T07:25:58.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.146+0000 7faeb788d640 1 -- start start 2024-03-06T07:25:58.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb788d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 0x7faeb0071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb788d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faeb0152c20 0x7faeb015c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb788d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb0072440 con 0x7faeb00c3150 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb788d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb00c2b10 con 0x7faeb0152c20 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb788d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb0152140 con 0x7faeb014f650 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb5e2a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faeb0152c20 0x7faeb015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb4e28640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faeb00c3150 0x7faeb00c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52256/0 (socket says 172.21.15.17:52256) 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.147+0000 7faeb4e28640 1 -- 172.21.15.17:0/3353774320 learned_addr learned my addr 172.21.15.17:0/3353774320 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5629640 1 --2- 172.21.15.17:0/3353774320 >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 0x7faeb0071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5e2a640 1 -- 172.21.15.17:0/3353774320 >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 msgr2=0x7faeb0071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5e2a640 1 --2- 172.21.15.17:0/3353774320 >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 0x7faeb0071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5e2a640 1 -- 172.21.15.17:0/3353774320 >> v1:172.21.15.17:6789/0 conn(0x7faeb00c3150 legacy=0x7faeb00c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5e2a640 1 -- 172.21.15.17:0/3353774320 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0152420 con 0x7faeb0152c20 2024-03-06T07:25:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5629640 1 --2- 172.21.15.17:0/3353774320 >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 0x7faeb0071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.148+0000 7faeb5e2a640 1 --2- 172.21.15.17:0/3353774320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb0152c20 0x7faeb015c810 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7faeb0070a90 tx=0x7faeac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250560ccf4178b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faea7fff640 1 -- 172.21.15.17:0/3353774320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac039070 con 0x7faeb0152c20 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faea7fff640 1 -- 172.21.15.17:0/3353774320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac02cbf0 con 0x7faeb0152c20 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faea7fff640 1 -- 172.21.15.17:0/3353774320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac034440 con 0x7faeb0152c20 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 -- 172.21.15.17:0/3353774320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb0152c20 msgr2=0x7faeb015c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 --2- 172.21.15.17:0/3353774320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb0152c20 0x7faeb015c810 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7faeb0070a90 tx=0x7faeac02be40 comp rx=0 tx=0).stop 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 -- 172.21.15.17:0/3353774320 shutdown_connections 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 --2- 172.21.15.17:0/3353774320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb0152c20 0x7faeb015c810 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 --2- 172.21.15.17:0/3353774320 >> v2:172.21.15.163:3300/0 conn(0x7faeb014f650 0x7faeb0071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 -- 172.21.15.17:0/3353774320 >> 172.21.15.17:0/3353774320 conn(0x7faeb00c4290 msgr2=0x7faeb00c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 -- 172.21.15.17:0/3353774320 shutdown_connections 2024-03-06T07:25:58.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.149+0000 7faeb788d640 1 -- 172.21.15.17:0/3353774320 wait complete. 2024-03-06T07:25:58.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.150+0000 7faeb788d640 1 Processor -- start 2024-03-06T07:25:58.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.150+0000 7faeb788d640 1 -- start start 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 0x7faeb006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 0x7faeb01ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c40c0 con 0x7faeb00c3150 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c3f40 con 0x7faeb0152c20 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb788d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb00c5da0 con 0x7faeb014f650 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb5629640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb5629640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57212/0 (socket says 172.21.15.17:57212) 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb5629640 1 -- 172.21.15.17:0/165303085 learned_addr learned my addr 172.21.15.17:0/165303085 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb5e2a640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 0x7faeb01ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb5629640 1 -- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 msgr2=0x7faeb006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.151+0000 7faeb4e28640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 0x7faeb006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5629640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 0x7faeb006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5629640 1 -- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 msgr2=0x7faeb01ebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5629640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 0x7faeb01ebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5629640 1 -- 172.21.15.17:0/165303085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb006f160 con 0x7faeb00c3150 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5e2a640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 0x7faeb01ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb4e28640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 0x7faeb006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb5629640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7faea800c820 tx=0x7faea800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8015070 con 0x7faeb00c3150 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8007a30 con 0x7faeb00c3150 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb006cf10 con 0x7faeb00c3150 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8007e00 con 0x7faeb00c3150 2024-03-06T07:25:58.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.152+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb006f400 con 0x7faeb00c3150 2024-03-06T07:25:58.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.153+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb004f330 con 0x7faeb00c3150 2024-03-06T07:25:58.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.154+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faea8018070 con 0x7faeb00c3150 2024-03-06T07:25:58.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.154+0000 7faea5ffb640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.154+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faea808a240 con 0x7faeb00c3150 2024-03-06T07:25:58.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.154+0000 7faeb4e28640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.155+0000 7faeb4e28640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 0x7fae9406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faeb014b810 tx=0x7faea0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.158+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faea8058fc0 con 0x7faeb00c3150 2024-03-06T07:25:58.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.337+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faeb0068460 con 0x7fae9406b4d0 2024-03-06T07:25:58.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.343+0000 7faea5ffb640 1 -- 172.21.15.17:0/165303085 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252547 (secure 0 0 0) 0x7faeb0068460 con 0x7fae9406b4d0 2024-03-06T07:25:58.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:58.349 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:58.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 msgr2=0x7fae9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 0x7fae9406d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faeb014b810 tx=0x7faea0009040 comp rx=0 tx=0).stop 2024-03-06T07:25:58.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 msgr2=0x7faeb006baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7faea800c820 tx=0x7faea800cc20 comp rx=0 tx=0).stop 2024-03-06T07:25:58.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 shutdown_connections 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae9406b4d0 0x7fae9406d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeb0152c20 0x7faeb01ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeb014f650 0x7faeb006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 --2- 172.21.15.17:0/165303085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faeb00c3150 0x7faeb006baa0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 >> 172.21.15.17:0/165303085 conn(0x7faeb00c4290 msgr2=0x7faeb014d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 shutdown_connections 2024-03-06T07:25:58.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.347+0000 7faeb788d640 1 -- 172.21.15.17:0/165303085 wait complete. 2024-03-06T07:25:58.359 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-03-06T07:25:57.418866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264396,"kb_used_data":14932,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710324,"statfs":{"total":773094113280,"available":772823371776,"internally_reserved":0,"allocated":15290368,"data_stored":8492376,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.265246"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491644+0000","last_change":"2024-03-06T07:25:55.171050+0000","last_active":"2024-03-06T07:25:56.491644+0000","last_peered":"2024-03-06T07:25:56.491644+0000","last_clean":"2024-03-06T07:25:56.491644+0000","last_became_active":"2024-03-06T07:25:55.170737+0000","last_became_peered":"2024-03-06T07:25:55.170737+0000","last_unstale":"2024-03-06T07:25:56.491644+0000","last_undegraded":"2024-03-06T07:25:56.491644+0000","last_fullsized":"2024-03-06T07:25:56.491644+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977294+0000","last_change":"2024-03-06T07:25:55.179421+0000","last_active":"2024-03-06T07:25:56.977294+0000","last_peered":"2024-03-06T07:25:56.977294+0000","last_clean":"2024-03-06T07:25:56.977294+0000","last_became_active":"2024-03-06T07:25:55.179145+0000","last_became_peered":"2024-03-06T07:25:55.179145+0000","last_unstale":"2024-03-06T07:25:56.977294+0000","last_undegraded":"2024-03-06T07:25:56.977294+0000","last_fullsized":"2024-03-06T07:25:56.977294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491503+0000","last_change":"2024-03-06T07:25:55.178609+0000","last_active":"2024-03-06T07:25:56.491503+0000","last_peered":"2024-03-06T07:25:56.491503+0000","last_clean":"2024-03-06T07:25:56.491503+0000","last_became_active":"2024-03-06T07:25:55.178334+0000","last_became_peered":"2024-03-06T07:25:55.178334+0000","last_unstale":"2024-03-06T07:25:56.491503+0000","last_undegraded":"2024-03-06T07:25:56.491503+0000","last_fullsized":"2024-03-06T07:25:56.491503+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:19.444548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491505+0000","last_change":"2024-03-06T07:25:55.181844+0000","last_active":"2024-03-06T07:25:56.491505+0000","last_peered":"2024-03-06T07:25:56.491505+0000","last_clean":"2024-03-06T07:25:56.491505+0000","last_became_active":"2024-03-06T07:25:55.181282+0000","last_became_peered":"2024-03-06T07:25:55.181282+0000","last_unstale":"2024-03-06T07:25:56.491505+0000","last_undegraded":"2024-03-06T07:25:56.491505+0000","last_fullsized":"2024-03-06T07:25:56.491505+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:58.454621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406406+0000","last_change":"2024-03-06T07:25:55.188179+0000","last_active":"2024-03-06T07:25:56.406406+0000","last_peered":"2024-03-06T07:25:56.406406+0000","last_clean":"2024-03-06T07:25:56.406406+0000","last_became_active":"2024-03-06T07:25:55.187905+0000","last_became_peered":"2024-03-06T07:25:55.187905+0000","last_unstale":"2024-03-06T07:25:56.406406+0000","last_undegraded":"2024-03-06T07:25:56.406406+0000","last_fullsized":"2024-03-06T07:25:56.406406+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:31.959459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976857+0000","last_change":"2024-03-06T07:25:55.179259+0000","last_active":"2024-03-06T07:25:56.976857+0000","last_peered":"2024-03-06T07:25:56.976857+0000","last_clean":"2024-03-06T07:25:56.976857+0000","last_became_active":"2024-03-06T07:25:55.178971+0000","last_became_peered":"2024-03-06T07:25:55.178971+0000","last_unstale":"2024-03-06T07:25:56.976857+0000","last_undegraded":"2024-03-06T07:25:56.976857+0000","last_fullsized":"2024-03-06T07:25:56.976857+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:52.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167499+0000","last_change":"2024-03-06T07:25:55.189198+0000","last_active":"2024-03-06T07:25:57.167499+0000","last_peered":"2024-03-06T07:25:57.167499+0000","last_clean":"2024-03-06T07:25:57.167499+0000","last_became_active":"2024-03-06T07:25:55.188937+0000","last_became_peered":"2024-03-06T07:25:55.188937+0000","last_unstale":"2024-03-06T07:25:57.167499+0000","last_undegraded":"2024-03-06T07:25:57.167499+0000","last_fullsized":"2024-03-06T07:25:57.167499+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167456+0000","last_change":"2024-03-06T07:25:55.174913+0000","last_active":"2024-03-06T07:25:57.167456+0000","last_peered":"2024-03-06T07:25:57.167456+0000","last_clean":"2024-03-06T07:25:57.167456+0000","last_became_active":"2024-03-06T07:25:55.174627+0000","last_became_peered":"2024-03-06T07:25:55.174627+0000","last_unstale":"2024-03-06T07:25:57.167456+0000","last_undegraded":"2024-03-06T07:25:57.167456+0000","last_fullsized":"2024-03-06T07:25:57.167456+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167916+0000","last_change":"2024-03-06T07:25:55.175881+0000","last_active":"2024-03-06T07:25:57.167916+0000","last_peered":"2024-03-06T07:25:57.167916+0000","last_clean":"2024-03-06T07:25:57.167916+0000","last_became_active":"2024-03-06T07:25:55.175604+0000","last_became_peered":"2024-03-06T07:25:55.175604+0000","last_unstale":"2024-03-06T07:25:57.167916+0000","last_undegraded":"2024-03-06T07:25:57.167916+0000","last_fullsized":"2024-03-06T07:25:57.167916+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:18.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167953+0000","last_change":"2024-03-06T07:25:55.174883+0000","last_active":"2024-03-06T07:25:57.167953+0000","last_peered":"2024-03-06T07:25:57.167953+0000","last_clean":"2024-03-06T07:25:57.167953+0000","last_became_active":"2024-03-06T07:25:55.174590+0000","last_became_peered":"2024-03-06T07:25:55.174590+0000","last_unstale":"2024-03-06T07:25:57.167953+0000","last_undegraded":"2024-03-06T07:25:57.167953+0000","last_fullsized":"2024-03-06T07:25:57.167953+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164754+0000","last_change":"2024-03-06T07:25:55.182365+0000","last_active":"2024-03-06T07:25:56.164754+0000","last_peered":"2024-03-06T07:25:56.164754+0000","last_clean":"2024-03-06T07:25:56.164754+0000","last_became_active":"2024-03-06T07:25:55.182083+0000","last_became_peered":"2024-03-06T07:25:55.182083+0000","last_unstale":"2024-03-06T07:25:56.164754+0000","last_undegraded":"2024-03-06T07:25:56.164754+0000","last_fullsized":"2024-03-06T07:25:56.164754+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:53.751341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977198+0000","last_change":"2024-03-06T07:25:55.179820+0000","last_active":"2024-03-06T07:25:56.977198+0000","last_peered":"2024-03-06T07:25:56.977198+0000","last_clean":"2024-03-06T07:25:56.977198+0000","last_became_active":"2024-03-06T07:25:55.179573+0000","last_became_peered":"2024-03-06T07:25:55.179573+0000","last_unstale":"2024-03-06T07:25:56.977198+0000","last_undegraded":"2024-03-06T07:25:56.977198+0000","last_fullsized":"2024-03-06T07:25:56.977198+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162398+0000","last_change":"2024-03-06T07:25:55.178116+0000","last_active":"2024-03-06T07:25:56.162398+0000","last_peered":"2024-03-06T07:25:56.162398+0000","last_clean":"2024-03-06T07:25:56.162398+0000","last_became_active":"2024-03-06T07:25:55.177769+0000","last_became_peered":"2024-03-06T07:25:55.177769+0000","last_unstale":"2024-03-06T07:25:56.162398+0000","last_undegraded":"2024-03-06T07:25:56.162398+0000","last_fullsized":"2024-03-06T07:25:56.162398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:19.818533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406254+0000","last_change":"2024-03-06T07:25:55.185952+0000","last_active":"2024-03-06T07:25:56.406254+0000","last_peered":"2024-03-06T07:25:56.406254+0000","last_clean":"2024-03-06T07:25:56.406254+0000","last_became_active":"2024-03-06T07:25:55.185677+0000","last_became_peered":"2024-03-06T07:25:55.185677+0000","last_unstale":"2024-03-06T07:25:56.406254+0000","last_undegraded":"2024-03-06T07:25:56.406254+0000","last_fullsized":"2024-03-06T07:25:56.406254+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:02.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164195+0000","last_change":"2024-03-06T07:25:55.180387+0000","last_active":"2024-03-06T07:25:56.164195+0000","last_peered":"2024-03-06T07:25:56.164195+0000","last_clean":"2024-03-06T07:25:56.164195+0000","last_became_active":"2024-03-06T07:25:55.179778+0000","last_became_peered":"2024-03-06T07:25:55.179778+0000","last_unstale":"2024-03-06T07:25:56.164195+0000","last_undegraded":"2024-03-06T07:25:56.164195+0000","last_fullsized":"2024-03-06T07:25:56.164195+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:16.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491231+0000","last_change":"2024-03-06T07:25:55.180281+0000","last_active":"2024-03-06T07:25:56.491231+0000","last_peered":"2024-03-06T07:25:56.491231+0000","last_clean":"2024-03-06T07:25:56.491231+0000","last_became_active":"2024-03-06T07:25:55.180014+0000","last_became_peered":"2024-03-06T07:25:55.180014+0000","last_unstale":"2024-03-06T07:25:56.491231+0000","last_undegraded":"2024-03-06T07:25:56.491231+0000","last_fullsized":"2024-03-06T07:25:56.491231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:08:51.191943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167944+0000","last_change":"2024-03-06T07:25:55.180083+0000","last_active":"2024-03-06T07:25:57.167944+0000","last_peered":"2024-03-06T07:25:57.167944+0000","last_clean":"2024-03-06T07:25:57.167944+0000","last_became_active":"2024-03-06T07:25:55.179804+0000","last_became_peered":"2024-03-06T07:25:55.179804+0000","last_unstale":"2024-03-06T07:25:57.167944+0000","last_undegraded":"2024-03-06T07:25:57.167944+0000","last_fullsized":"2024-03-06T07:25:57.167944+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:03.229947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164987+0000","last_change":"2024-03-06T07:25:55.176450+0000","last_active":"2024-03-06T07:25:56.164987+0000","last_peered":"2024-03-06T07:25:56.164987+0000","last_clean":"2024-03-06T07:25:56.164987+0000","last_became_active":"2024-03-06T07:25:55.176194+0000","last_became_peered":"2024-03-06T07:25:55.176194+0000","last_unstale":"2024-03-06T07:25:56.164987+0000","last_undegraded":"2024-03-06T07:25:56.164987+0000","last_fullsized":"2024-03-06T07:25:56.164987+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:53.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164959+0000","last_change":"2024-03-06T07:25:55.176884+0000","last_active":"2024-03-06T07:25:56.164959+0000","last_peered":"2024-03-06T07:25:56.164959+0000","last_clean":"2024-03-06T07:25:56.164959+0000","last_became_active":"2024-03-06T07:25:55.176546+0000","last_became_peered":"2024-03-06T07:25:55.176546+0000","last_unstale":"2024-03-06T07:25:56.164959+0000","last_undegraded":"2024-03-06T07:25:56.164959+0000","last_fullsized":"2024-03-06T07:25:56.164959+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:22.733869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.184293+0000","last_change":"2024-03-06T07:25:55.181620+0000","last_active":"2024-03-06T07:25:55.184293+0000","last_peered":"2024-03-06T07:25:55.184293+0000","last_clean":"2024-03-06T07:25:55.184293+0000","last_became_active":"2024-03-06T07:25:55.181336+0000","last_became_peered":"2024-03-06T07:25:55.181336+0000","last_unstale":"2024-03-06T07:25:55.184293+0000","last_undegraded":"2024-03-06T07:25:55.184293+0000","last_fullsized":"2024-03-06T07:25:55.184293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:28.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162472+0000","last_change":"2024-03-06T07:25:55.171658+0000","last_active":"2024-03-06T07:25:56.162472+0000","last_peered":"2024-03-06T07:25:56.162472+0000","last_clean":"2024-03-06T07:25:56.162472+0000","last_became_active":"2024-03-06T07:25:55.171366+0000","last_became_peered":"2024-03-06T07:25:55.171366+0000","last_unstale":"2024-03-06T07:25:56.162472+0000","last_undegraded":"2024-03-06T07:25:56.162472+0000","last_fullsized":"2024-03-06T07:25:56.162472+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:03.613423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976898+0000","last_change":"2024-03-06T07:25:55.185023+0000","last_active":"2024-03-06T07:25:56.976898+0000","last_peered":"2024-03-06T07:25:56.976898+0000","last_clean":"2024-03-06T07:25:56.976898+0000","last_became_active":"2024-03-06T07:25:55.184708+0000","last_became_peered":"2024-03-06T07:25:55.184708+0000","last_unstale":"2024-03-06T07:25:56.976898+0000","last_undegraded":"2024-03-06T07:25:56.976898+0000","last_fullsized":"2024-03-06T07:25:56.976898+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:25.354294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406171+0000","last_change":"2024-03-06T07:25:55.189044+0000","last_active":"2024-03-06T07:25:56.406171+0000","last_peered":"2024-03-06T07:25:56.406171+0000","last_clean":"2024-03-06T07:25:56.406171+0000","last_became_active":"2024-03-06T07:25:55.188772+0000","last_became_peered":"2024-03-06T07:25:55.188772+0000","last_unstale":"2024-03-06T07:25:56.406171+0000","last_undegraded":"2024-03-06T07:25:56.406171+0000","last_fullsized":"2024-03-06T07:25:56.406171+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491346+0000","last_change":"2024-03-06T07:25:55.179621+0000","last_active":"2024-03-06T07:25:56.491346+0000","last_peered":"2024-03-06T07:25:56.491346+0000","last_clean":"2024-03-06T07:25:56.491346+0000","last_became_active":"2024-03-06T07:25:55.179320+0000","last_became_peered":"2024-03-06T07:25:55.179320+0000","last_unstale":"2024-03-06T07:25:56.491346+0000","last_undegraded":"2024-03-06T07:25:56.491346+0000","last_fullsized":"2024-03-06T07:25:56.491346+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:38.124631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162773+0000","last_change":"2024-03-06T07:25:55.183095+0000","last_active":"2024-03-06T07:25:56.162773+0000","last_peered":"2024-03-06T07:25:56.162773+0000","last_clean":"2024-03-06T07:25:56.162773+0000","last_became_active":"2024-03-06T07:25:55.182807+0000","last_became_peered":"2024-03-06T07:25:55.182807+0000","last_unstale":"2024-03-06T07:25:56.162773+0000","last_undegraded":"2024-03-06T07:25:56.162773+0000","last_fullsized":"2024-03-06T07:25:56.162773+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:02.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.183583+0000","last_change":"2024-03-06T07:25:55.180267+0000","last_active":"2024-03-06T07:25:55.183583+0000","last_peered":"2024-03-06T07:25:55.183583+0000","last_clean":"2024-03-06T07:25:55.183583+0000","last_became_active":"2024-03-06T07:25:55.180006+0000","last_became_peered":"2024-03-06T07:25:55.180006+0000","last_unstale":"2024-03-06T07:25:55.183583+0000","last_undegraded":"2024-03-06T07:25:55.183583+0000","last_fullsized":"2024-03-06T07:25:55.183583+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:14.681093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977250+0000","last_change":"2024-03-06T07:25:55.185010+0000","last_active":"2024-03-06T07:25:56.977250+0000","last_peered":"2024-03-06T07:25:56.977250+0000","last_clean":"2024-03-06T07:25:56.977250+0000","last_became_active":"2024-03-06T07:25:55.184723+0000","last_became_peered":"2024-03-06T07:25:55.184723+0000","last_unstale":"2024-03-06T07:25:56.977250+0000","last_undegraded":"2024-03-06T07:25:56.977250+0000","last_fullsized":"2024-03-06T07:25:56.977250+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162381+0000","last_change":"2024-03-06T07:25:55.171489+0000","last_active":"2024-03-06T07:25:56.162381+0000","last_peered":"2024-03-06T07:25:56.162381+0000","last_clean":"2024-03-06T07:25:56.162381+0000","last_became_active":"2024-03-06T07:25:55.171184+0000","last_became_peered":"2024-03-06T07:25:55.171184+0000","last_unstale":"2024-03-06T07:25:56.162381+0000","last_undegraded":"2024-03-06T07:25:56.162381+0000","last_fullsized":"2024-03-06T07:25:56.162381+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:03.888318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977078+0000","last_change":"2024-03-06T07:25:55.175885+0000","last_active":"2024-03-06T07:25:56.977078+0000","last_peered":"2024-03-06T07:25:56.977078+0000","last_clean":"2024-03-06T07:25:56.977078+0000","last_became_active":"2024-03-06T07:25:55.175607+0000","last_became_peered":"2024-03-06T07:25:55.175607+0000","last_unstale":"2024-03-06T07:25:56.977078+0000","last_undegraded":"2024-03-06T07:25:56.977078+0000","last_fullsized":"2024-03-06T07:25:56.977078+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165294+0000","last_change":"2024-03-06T07:25:55.182544+0000","last_active":"2024-03-06T07:25:56.165294+0000","last_peered":"2024-03-06T07:25:56.165294+0000","last_clean":"2024-03-06T07:25:56.165294+0000","last_became_active":"2024-03-06T07:25:55.182215+0000","last_became_peered":"2024-03-06T07:25:55.182215+0000","last_unstale":"2024-03-06T07:25:56.165294+0000","last_undegraded":"2024-03-06T07:25:56.165294+0000","last_fullsized":"2024-03-06T07:25:56.165294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:47.717714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164251+0000","last_change":"2024-03-06T07:25:55.174623+0000","last_active":"2024-03-06T07:25:56.164251+0000","last_peered":"2024-03-06T07:25:56.164251+0000","last_clean":"2024-03-06T07:25:56.164251+0000","last_became_active":"2024-03-06T07:25:55.174336+0000","last_became_peered":"2024-03-06T07:25:55.174336+0000","last_unstale":"2024-03-06T07:25:56.164251+0000","last_undegraded":"2024-03-06T07:25:56.164251+0000","last_fullsized":"2024-03-06T07:25:56.164251+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:15.212900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164787+0000","last_change":"2024-03-06T07:25:55.173469+0000","last_active":"2024-03-06T07:25:56.164787+0000","last_peered":"2024-03-06T07:25:56.164787+0000","last_clean":"2024-03-06T07:25:56.164787+0000","last_became_active":"2024-03-06T07:25:55.173211+0000","last_became_peered":"2024-03-06T07:25:55.173211+0000","last_unstale":"2024-03-06T07:25:56.164787+0000","last_undegraded":"2024-03-06T07:25:56.164787+0000","last_fullsized":"2024-03-06T07:25:56.164787+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:44.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976975+0000","last_change":"2024-03-06T07:25:55.183313+0000","last_active":"2024-03-06T07:25:56.976975+0000","last_peered":"2024-03-06T07:25:56.976975+0000","last_clean":"2024-03-06T07:25:56.976975+0000","last_became_active":"2024-03-06T07:25:55.183007+0000","last_became_peered":"2024-03-06T07:25:55.183007+0000","last_unstale":"2024-03-06T07:25:56.976975+0000","last_undegraded":"2024-03-06T07:25:56.976975+0000","last_fullsized":"2024-03-06T07:25:56.976975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:53.878454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491720+0000","last_change":"2024-03-06T07:25:54.177412+0000","last_active":"2024-03-06T07:25:56.491720+0000","last_peered":"2024-03-06T07:25:56.491720+0000","last_clean":"2024-03-06T07:25:56.491720+0000","last_became_active":"2024-03-06T07:25:54.177071+0000","last_became_peered":"2024-03-06T07:25:54.177071+0000","last_unstale":"2024-03-06T07:25:56.491720+0000","last_undegraded":"2024-03-06T07:25:56.491720+0000","last_fullsized":"2024-03-06T07:25:56.491720+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:58.975791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162259+0000","last_change":"2024-03-06T07:25:55.169574+0000","last_active":"2024-03-06T07:25:56.162259+0000","last_peered":"2024-03-06T07:25:56.162259+0000","last_clean":"2024-03-06T07:25:56.162259+0000","last_became_active":"2024-03-06T07:25:55.168987+0000","last_became_peered":"2024-03-06T07:25:55.168987+0000","last_unstale":"2024-03-06T07:25:56.162259+0000","last_undegraded":"2024-03-06T07:25:56.162259+0000","last_fullsized":"2024-03-06T07:25:56.162259+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:55.457043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406436+0000","last_change":"2024-03-06T07:25:54.178532+0000","last_active":"2024-03-06T07:25:56.406436+0000","last_peered":"2024-03-06T07:25:56.406436+0000","last_clean":"2024-03-06T07:25:56.406436+0000","last_became_active":"2024-03-06T07:25:54.178228+0000","last_became_peered":"2024-03-06T07:25:54.178228+0000","last_unstale":"2024-03-06T07:25:56.406436+0000","last_undegraded":"2024-03-06T07:25:56.406436+0000","last_fullsized":"2024-03-06T07:25:56.406436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:07.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406279+0000","last_change":"2024-03-06T07:25:55.181488+0000","last_active":"2024-03-06T07:25:56.406279+0000","last_peered":"2024-03-06T07:25:56.406279+0000","last_clean":"2024-03-06T07:25:56.406279+0000","last_became_active":"2024-03-06T07:25:55.181212+0000","last_became_peered":"2024-03-06T07:25:55.181212+0000","last_unstale":"2024-03-06T07:25:56.406279+0000","last_undegraded":"2024-03-06T07:25:56.406279+0000","last_fullsized":"2024-03-06T07:25:56.406279+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:07.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165147+0000","last_change":"2024-03-06T07:25:54.176834+0000","last_active":"2024-03-06T07:25:56.165147+0000","last_peered":"2024-03-06T07:25:56.165147+0000","last_clean":"2024-03-06T07:25:56.165147+0000","last_became_active":"2024-03-06T07:25:54.176517+0000","last_became_peered":"2024-03-06T07:25:54.176517+0000","last_unstale":"2024-03-06T07:25:56.165147+0000","last_undegraded":"2024-03-06T07:25:56.165147+0000","last_fullsized":"2024-03-06T07:25:56.165147+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:46.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977126+0000","last_change":"2024-03-06T07:25:55.186954+0000","last_active":"2024-03-06T07:25:56.977126+0000","last_peered":"2024-03-06T07:25:56.977126+0000","last_clean":"2024-03-06T07:25:56.977126+0000","last_became_active":"2024-03-06T07:25:55.186700+0000","last_became_peered":"2024-03-06T07:25:55.186700+0000","last_unstale":"2024-03-06T07:25:56.977126+0000","last_undegraded":"2024-03-06T07:25:56.977126+0000","last_fullsized":"2024-03-06T07:25:56.977126+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"149'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.193360+0000","last_change":"2024-03-06T07:25:54.173923+0000","last_active":"2024-03-06T07:25:55.193360+0000","last_peered":"2024-03-06T07:25:55.193360+0000","last_clean":"2024-03-06T07:25:55.193360+0000","last_became_active":"2024-03-06T07:25:54.173628+0000","last_became_peered":"2024-03-06T07:25:54.173628+0000","last_unstale":"2024-03-06T07:25:55.193360+0000","last_undegraded":"2024-03-06T07:25:55.193360+0000","last_fullsized":"2024-03-06T07:25:55.193360+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:18.074282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167641+0000","last_change":"2024-03-06T07:25:55.179023+0000","last_active":"2024-03-06T07:25:57.167641+0000","last_peered":"2024-03-06T07:25:57.167641+0000","last_clean":"2024-03-06T07:25:57.167641+0000","last_became_active":"2024-03-06T07:25:55.178604+0000","last_became_peered":"2024-03-06T07:25:55.178604+0000","last_unstale":"2024-03-06T07:25:57.167641+0000","last_undegraded":"2024-03-06T07:25:57.167641+0000","last_fullsized":"2024-03-06T07:25:57.167641+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167634+0000","last_change":"2024-03-06T07:25:54.179340+0000","last_active":"2024-03-06T07:25:57.167634+0000","last_peered":"2024-03-06T07:25:57.167634+0000","last_clean":"2024-03-06T07:25:57.167634+0000","last_became_active":"2024-03-06T07:25:54.179061+0000","last_became_peered":"2024-03-06T07:25:54.179061+0000","last_unstale":"2024-03-06T07:25:57.167634+0000","last_undegraded":"2024-03-06T07:25:57.167634+0000","last_fullsized":"2024-03-06T07:25:57.167634+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:24.779584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.193063+0000","last_change":"2024-03-06T07:25:55.186844+0000","last_active":"2024-03-06T07:25:55.193063+0000","last_peered":"2024-03-06T07:25:55.193063+0000","last_clean":"2024-03-06T07:25:55.193063+0000","last_became_active":"2024-03-06T07:25:55.186558+0000","last_became_peered":"2024-03-06T07:25:55.186558+0000","last_unstale":"2024-03-06T07:25:55.193063+0000","last_undegraded":"2024-03-06T07:25:55.193063+0000","last_fullsized":"2024-03-06T07:25:55.193063+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:56:27.743379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406356+0000","last_change":"2024-03-06T07:25:54.182713+0000","last_active":"2024-03-06T07:25:56.406356+0000","last_peered":"2024-03-06T07:25:56.406356+0000","last_clean":"2024-03-06T07:25:56.406356+0000","last_became_active":"2024-03-06T07:25:54.182447+0000","last_became_peered":"2024-03-06T07:25:54.182447+0000","last_unstale":"2024-03-06T07:25:56.406356+0000","last_undegraded":"2024-03-06T07:25:56.406356+0000","last_fullsized":"2024-03-06T07:25:56.406356+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:17.509614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.172898+0000","last_change":"2024-03-06T07:25:55.172223+0000","last_active":"2024-03-06T07:25:55.172898+0000","last_peered":"2024-03-06T07:25:55.172898+0000","last_clean":"2024-03-06T07:25:55.172898+0000","last_became_active":"2024-03-06T07:25:55.171895+0000","last_became_peered":"2024-03-06T07:25:55.171895+0000","last_unstale":"2024-03-06T07:25:55.172898+0000","last_undegraded":"2024-03-06T07:25:55.172898+0000","last_fullsized":"2024-03-06T07:25:55.172898+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:38.683915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.166186+0000","last_change":"2024-03-06T07:25:54.172467+0000","last_active":"2024-03-06T07:25:55.166186+0000","last_peered":"2024-03-06T07:25:55.166186+0000","last_clean":"2024-03-06T07:25:55.166186+0000","last_became_active":"2024-03-06T07:25:54.172185+0000","last_became_peered":"2024-03-06T07:25:54.172185+0000","last_unstale":"2024-03-06T07:25:55.166186+0000","last_undegraded":"2024-03-06T07:25:55.166186+0000","last_fullsized":"2024-03-06T07:25:55.166186+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:58.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491699+0000","last_change":"2024-03-06T07:25:55.175586+0000","last_active":"2024-03-06T07:25:56.491699+0000","last_peered":"2024-03-06T07:25:56.491699+0000","last_clean":"2024-03-06T07:25:56.491699+0000","last_became_active":"2024-03-06T07:25:55.175328+0000","last_became_peered":"2024-03-06T07:25:55.175328+0000","last_unstale":"2024-03-06T07:25:56.491699+0000","last_undegraded":"2024-03-06T07:25:56.491699+0000","last_fullsized":"2024-03-06T07:25:56.491699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:34.059703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167555+0000","last_change":"2024-03-06T07:25:54.175104+0000","last_active":"2024-03-06T07:25:57.167555+0000","last_peered":"2024-03-06T07:25:57.167555+0000","last_clean":"2024-03-06T07:25:57.167555+0000","last_became_active":"2024-03-06T07:25:54.174837+0000","last_became_peered":"2024-03-06T07:25:54.174837+0000","last_unstale":"2024-03-06T07:25:57.167555+0000","last_undegraded":"2024-03-06T07:25:57.167555+0000","last_fullsized":"2024-03-06T07:25:57.167555+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:15.768972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976860+0000","last_change":"2024-03-06T07:25:55.183382+0000","last_active":"2024-03-06T07:25:56.976860+0000","last_peered":"2024-03-06T07:25:56.976860+0000","last_clean":"2024-03-06T07:25:56.976860+0000","last_became_active":"2024-03-06T07:25:55.182975+0000","last_became_peered":"2024-03-06T07:25:55.182975+0000","last_unstale":"2024-03-06T07:25:56.976860+0000","last_undegraded":"2024-03-06T07:25:56.976860+0000","last_fullsized":"2024-03-06T07:25:56.976860+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:04.379898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162704+0000","last_change":"2024-03-06T07:25:54.184577+0000","last_active":"2024-03-06T07:25:56.162704+0000","last_peered":"2024-03-06T07:25:56.162704+0000","last_clean":"2024-03-06T07:25:56.162704+0000","last_became_active":"2024-03-06T07:25:54.184068+0000","last_became_peered":"2024-03-06T07:25:54.184068+0000","last_unstale":"2024-03-06T07:25:56.162704+0000","last_undegraded":"2024-03-06T07:25:56.162704+0000","last_fullsized":"2024-03-06T07:25:56.162704+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:30.954235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162382+0000","last_change":"2024-03-06T07:25:55.175578+0000","last_active":"2024-03-06T07:25:56.162382+0000","last_peered":"2024-03-06T07:25:56.162382+0000","last_clean":"2024-03-06T07:25:56.162382+0000","last_became_active":"2024-03-06T07:25:55.175257+0000","last_became_peered":"2024-03-06T07:25:55.175257+0000","last_unstale":"2024-03-06T07:25:56.162382+0000","last_undegraded":"2024-03-06T07:25:56.162382+0000","last_fullsized":"2024-03-06T07:25:56.162382+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:56.174605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491490+0000","last_change":"2024-03-06T07:25:54.177422+0000","last_active":"2024-03-06T07:25:56.491490+0000","last_peered":"2024-03-06T07:25:56.491490+0000","last_clean":"2024-03-06T07:25:56.491490+0000","last_became_active":"2024-03-06T07:25:54.177048+0000","last_became_peered":"2024-03-06T07:25:54.177048+0000","last_unstale":"2024-03-06T07:25:56.491490+0000","last_undegraded":"2024-03-06T07:25:56.491490+0000","last_fullsized":"2024-03-06T07:25:56.491490+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:21.672283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.184323+0000","last_change":"2024-03-06T07:25:55.182193+0000","last_active":"2024-03-06T07:25:55.184323+0000","last_peered":"2024-03-06T07:25:55.184323+0000","last_clean":"2024-03-06T07:25:55.184323+0000","last_became_active":"2024-03-06T07:25:55.181940+0000","last_became_peered":"2024-03-06T07:25:55.181940+0000","last_unstale":"2024-03-06T07:25:55.184323+0000","last_undegraded":"2024-03-06T07:25:55.184323+0000","last_fullsized":"2024-03-06T07:25:55.184323+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:22.100315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162576+0000","last_change":"2024-03-06T07:25:54.184311+0000","last_active":"2024-03-06T07:25:56.162576+0000","last_peered":"2024-03-06T07:25:56.162576+0000","last_clean":"2024-03-06T07:25:56.162576+0000","last_became_active":"2024-03-06T07:25:54.183969+0000","last_became_peered":"2024-03-06T07:25:54.183969+0000","last_unstale":"2024-03-06T07:25:56.162576+0000","last_undegraded":"2024-03-06T07:25:56.162576+0000","last_fullsized":"2024-03-06T07:25:56.162576+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:08.451250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164497+0000","last_change":"2024-03-06T07:25:55.174144+0000","last_active":"2024-03-06T07:25:56.164497+0000","last_peered":"2024-03-06T07:25:56.164497+0000","last_clean":"2024-03-06T07:25:56.164497+0000","last_became_active":"2024-03-06T07:25:55.173885+0000","last_became_peered":"2024-03-06T07:25:55.173885+0000","last_unstale":"2024-03-06T07:25:56.164497+0000","last_undegraded":"2024-03-06T07:25:56.164497+0000","last_fullsized":"2024-03-06T07:25:56.164497+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:59.981015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406391+0000","last_change":"2024-03-06T07:25:54.175000+0000","last_active":"2024-03-06T07:25:56.406391+0000","last_peered":"2024-03-06T07:25:56.406391+0000","last_clean":"2024-03-06T07:25:56.406391+0000","last_became_active":"2024-03-06T07:25:54.174717+0000","last_became_peered":"2024-03-06T07:25:54.174717+0000","last_unstale":"2024-03-06T07:25:56.406391+0000","last_undegraded":"2024-03-06T07:25:56.406391+0000","last_fullsized":"2024-03-06T07:25:56.406391+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:52.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167823+0000","last_change":"2024-03-06T07:25:55.186488+0000","last_active":"2024-03-06T07:25:57.167823+0000","last_peered":"2024-03-06T07:25:57.167823+0000","last_clean":"2024-03-06T07:25:57.167823+0000","last_became_active":"2024-03-06T07:25:55.185803+0000","last_became_peered":"2024-03-06T07:25:55.185803+0000","last_unstale":"2024-03-06T07:25:57.167823+0000","last_undegraded":"2024-03-06T07:25:57.167823+0000","last_fullsized":"2024-03-06T07:25:57.167823+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162315+0000","last_change":"2024-03-06T07:25:54.178593+0000","last_active":"2024-03-06T07:25:56.162315+0000","last_peered":"2024-03-06T07:25:56.162315+0000","last_clean":"2024-03-06T07:25:56.162315+0000","last_became_active":"2024-03-06T07:25:54.178229+0000","last_became_peered":"2024-03-06T07:25:54.178229+0000","last_unstale":"2024-03-06T07:25:56.162315+0000","last_undegraded":"2024-03-06T07:25:56.162315+0000","last_fullsized":"2024-03-06T07:25:56.162315+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:59.187499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165275+0000","last_change":"2024-03-06T07:25:55.175110+0000","last_active":"2024-03-06T07:25:56.165275+0000","last_peered":"2024-03-06T07:25:56.165275+0000","last_clean":"2024-03-06T07:25:56.165275+0000","last_became_active":"2024-03-06T07:25:55.174818+0000","last_became_peered":"2024-03-06T07:25:55.174818+0000","last_unstale":"2024-03-06T07:25:56.165275+0000","last_undegraded":"2024-03-06T07:25:56.165275+0000","last_fullsized":"2024-03-06T07:25:56.165275+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:18.191924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164306+0000","last_change":"2024-03-06T07:25:54.178997+0000","last_active":"2024-03-06T07:25:56.164306+0000","last_peered":"2024-03-06T07:25:56.164306+0000","last_clean":"2024-03-06T07:25:56.164306+0000","last_became_active":"2024-03-06T07:25:54.178699+0000","last_became_peered":"2024-03-06T07:25:54.178699+0000","last_unstale":"2024-03-06T07:25:56.164306+0000","last_undegraded":"2024-03-06T07:25:56.164306+0000","last_fullsized":"2024-03-06T07:25:56.164306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:04.021145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.185957+0000","last_change":"2024-03-06T07:25:55.181498+0000","last_active":"2024-03-06T07:25:55.185957+0000","last_peered":"2024-03-06T07:25:55.185957+0000","last_clean":"2024-03-06T07:25:55.185957+0000","last_became_active":"2024-03-06T07:25:55.180900+0000","last_became_peered":"2024-03-06T07:25:55.180900+0000","last_unstale":"2024-03-06T07:25:55.185957+0000","last_undegraded":"2024-03-06T07:25:55.185957+0000","last_fullsized":"2024-03-06T07:25:55.185957+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:39.618266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167725+0000","last_change":"2024-03-06T07:25:54.177051+0000","last_active":"2024-03-06T07:25:57.167725+0000","last_peered":"2024-03-06T07:25:57.167725+0000","last_clean":"2024-03-06T07:25:57.167725+0000","last_became_active":"2024-03-06T07:25:54.176774+0000","last_became_peered":"2024-03-06T07:25:54.176774+0000","last_unstale":"2024-03-06T07:25:57.167725+0000","last_undegraded":"2024-03-06T07:25:57.167725+0000","last_fullsized":"2024-03-06T07:25:57.167725+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:14.207675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167454+0000","last_change":"2024-03-06T07:25:55.190742+0000","last_active":"2024-03-06T07:25:57.167454+0000","last_peered":"2024-03-06T07:25:57.167454+0000","last_clean":"2024-03-06T07:25:57.167454+0000","last_became_active":"2024-03-06T07:25:55.190479+0000","last_became_peered":"2024-03-06T07:25:55.190479+0000","last_unstale":"2024-03-06T07:25:57.167454+0000","last_undegraded":"2024-03-06T07:25:57.167454+0000","last_fullsized":"2024-03-06T07:25:57.167454+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162770+0000","last_change":"2024-03-06T07:25:54.178661+0000","last_active":"2024-03-06T07:25:56.162770+0000","last_peered":"2024-03-06T07:25:56.162770+0000","last_clean":"2024-03-06T07:25:56.162770+0000","last_became_active":"2024-03-06T07:25:54.178379+0000","last_became_peered":"2024-03-06T07:25:54.178379+0000","last_unstale":"2024-03-06T07:25:56.162770+0000","last_undegraded":"2024-03-06T07:25:56.162770+0000","last_fullsized":"2024-03-06T07:25:56.162770+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:30.562792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165181+0000","last_change":"2024-03-06T07:25:55.178452+0000","last_active":"2024-03-06T07:25:56.165181+0000","last_peered":"2024-03-06T07:25:56.165181+0000","last_clean":"2024-03-06T07:25:56.165181+0000","last_became_active":"2024-03-06T07:25:55.178193+0000","last_became_peered":"2024-03-06T07:25:55.178193+0000","last_unstale":"2024-03-06T07:25:56.165181+0000","last_undegraded":"2024-03-06T07:25:56.165181+0000","last_fullsized":"2024-03-06T07:25:56.165181+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:32.109891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167947+0000","last_change":"2024-03-06T07:25:54.173705+0000","last_active":"2024-03-06T07:25:57.167947+0000","last_peered":"2024-03-06T07:25:57.167947+0000","last_clean":"2024-03-06T07:25:57.167947+0000","last_became_active":"2024-03-06T07:25:54.173438+0000","last_became_peered":"2024-03-06T07:25:54.173438+0000","last_unstale":"2024-03-06T07:25:57.167947+0000","last_undegraded":"2024-03-06T07:25:57.167947+0000","last_fullsized":"2024-03-06T07:25:57.167947+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:51.526470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164497+0000","last_change":"2024-03-06T07:25:55.172838+0000","last_active":"2024-03-06T07:25:56.164497+0000","last_peered":"2024-03-06T07:25:56.164497+0000","last_clean":"2024-03-06T07:25:56.164497+0000","last_became_active":"2024-03-06T07:25:55.172505+0000","last_became_peered":"2024-03-06T07:25:55.172505+0000","last_unstale":"2024-03-06T07:25:56.164497+0000","last_undegraded":"2024-03-06T07:25:56.164497+0000","last_fullsized":"2024-03-06T07:25:56.164497+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:25.784809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406602+0000","last_change":"2024-03-06T07:25:54.177100+0000","last_active":"2024-03-06T07:25:56.406602+0000","last_peered":"2024-03-06T07:25:56.406602+0000","last_clean":"2024-03-06T07:25:56.406602+0000","last_became_active":"2024-03-06T07:25:54.176822+0000","last_became_peered":"2024-03-06T07:25:54.176822+0000","last_unstale":"2024-03-06T07:25:56.406602+0000","last_undegraded":"2024-03-06T07:25:56.406602+0000","last_fullsized":"2024-03-06T07:25:56.406602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.349070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164733+0000","last_change":"2024-03-06T07:25:55.183106+0000","last_active":"2024-03-06T07:25:56.164733+0000","last_peered":"2024-03-06T07:25:56.164733+0000","last_clean":"2024-03-06T07:25:56.164733+0000","last_became_active":"2024-03-06T07:25:55.182843+0000","last_became_peered":"2024-03-06T07:25:55.182843+0000","last_unstale":"2024-03-06T07:25:56.164733+0000","last_undegraded":"2024-03-06T07:25:56.164733+0000","last_fullsized":"2024-03-06T07:25:56.164733+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:52.531694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167699+0000","last_change":"2024-03-06T07:25:54.172691+0000","last_active":"2024-03-06T07:25:57.167699+0000","last_peered":"2024-03-06T07:25:57.167699+0000","last_clean":"2024-03-06T07:25:57.167699+0000","last_became_active":"2024-03-06T07:25:54.172355+0000","last_became_peered":"2024-03-06T07:25:54.172355+0000","last_unstale":"2024-03-06T07:25:57.167699+0000","last_undegraded":"2024-03-06T07:25:57.167699+0000","last_fullsized":"2024-03-06T07:25:57.167699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.756681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162524+0000","last_change":"2024-03-06T07:25:55.179180+0000","last_active":"2024-03-06T07:25:56.162524+0000","last_peered":"2024-03-06T07:25:56.162524+0000","last_clean":"2024-03-06T07:25:56.162524+0000","last_became_active":"2024-03-06T07:25:55.178890+0000","last_became_peered":"2024-03-06T07:25:55.178890+0000","last_unstale":"2024-03-06T07:25:56.162524+0000","last_undegraded":"2024-03-06T07:25:56.162524+0000","last_fullsized":"2024-03-06T07:25:56.162524+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:00.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406499+0000","last_change":"2024-03-06T07:25:54.181928+0000","last_active":"2024-03-06T07:25:56.406499+0000","last_peered":"2024-03-06T07:25:56.406499+0000","last_clean":"2024-03-06T07:25:56.406499+0000","last_became_active":"2024-03-06T07:25:54.181140+0000","last_became_peered":"2024-03-06T07:25:54.181140+0000","last_unstale":"2024-03-06T07:25:56.406499+0000","last_undegraded":"2024-03-06T07:25:56.406499+0000","last_fullsized":"2024-03-06T07:25:56.406499+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:57.291913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406638+0000","last_change":"2024-03-06T07:25:55.192859+0000","last_active":"2024-03-06T07:25:56.406638+0000","last_peered":"2024-03-06T07:25:56.406638+0000","last_clean":"2024-03-06T07:25:56.406638+0000","last_became_active":"2024-03-06T07:25:55.192570+0000","last_became_peered":"2024-03-06T07:25:55.192570+0000","last_unstale":"2024-03-06T07:25:56.406638+0000","last_undegraded":"2024-03-06T07:25:56.406638+0000","last_fullsized":"2024-03-06T07:25:56.406638+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:00.192724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164517+0000","last_change":"2024-03-06T07:25:54.180637+0000","last_active":"2024-03-06T07:25:56.164517+0000","last_peered":"2024-03-06T07:25:56.164517+0000","last_clean":"2024-03-06T07:25:56.164517+0000","last_became_active":"2024-03-06T07:25:54.180270+0000","last_became_peered":"2024-03-06T07:25:54.180270+0000","last_unstale":"2024-03-06T07:25:56.164517+0000","last_undegraded":"2024-03-06T07:25:56.164517+0000","last_fullsized":"2024-03-06T07:25:56.164517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:20.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406333+0000","last_change":"2024-03-06T07:25:55.194175+0000","last_active":"2024-03-06T07:25:56.406333+0000","last_peered":"2024-03-06T07:25:56.406333+0000","last_clean":"2024-03-06T07:25:56.406333+0000","last_became_active":"2024-03-06T07:25:55.192548+0000","last_became_peered":"2024-03-06T07:25:55.192548+0000","last_unstale":"2024-03-06T07:25:56.406333+0000","last_undegraded":"2024-03-06T07:25:56.406333+0000","last_fullsized":"2024-03-06T07:25:56.406333+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:09.456474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.166384+0000","last_change":"2024-03-06T07:25:54.171781+0000","last_active":"2024-03-06T07:25:55.166384+0000","last_peered":"2024-03-06T07:25:55.166384+0000","last_clean":"2024-03-06T07:25:55.166384+0000","last_became_active":"2024-03-06T07:25:54.171466+0000","last_became_peered":"2024-03-06T07:25:54.171466+0000","last_unstale":"2024-03-06T07:25:55.166384+0000","last_undegraded":"2024-03-06T07:25:55.166384+0000","last_fullsized":"2024-03-06T07:25:55.166384+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:50.533388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977321+0000","last_change":"2024-03-06T07:25:55.173561+0000","last_active":"2024-03-06T07:25:56.977321+0000","last_peered":"2024-03-06T07:25:56.977321+0000","last_clean":"2024-03-06T07:25:56.977321+0000","last_became_active":"2024-03-06T07:25:55.173309+0000","last_became_peered":"2024-03-06T07:25:55.173309+0000","last_unstale":"2024-03-06T07:25:56.977321+0000","last_undegraded":"2024-03-06T07:25:56.977321+0000","last_fullsized":"2024-03-06T07:25:56.977321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977300+0000","last_change":"2024-03-06T07:25:54.178755+0000","last_active":"2024-03-06T07:25:56.977300+0000","last_peered":"2024-03-06T07:25:56.977300+0000","last_clean":"2024-03-06T07:25:56.977300+0000","last_became_active":"2024-03-06T07:25:54.178492+0000","last_became_peered":"2024-03-06T07:25:54.178492+0000","last_unstale":"2024-03-06T07:25:56.977300+0000","last_undegraded":"2024-03-06T07:25:56.977300+0000","last_fullsized":"2024-03-06T07:25:56.977300+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:18.439323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164157+0000","last_change":"2024-03-06T07:25:55.172471+0000","last_active":"2024-03-06T07:25:56.164157+0000","last_peered":"2024-03-06T07:25:56.164157+0000","last_clean":"2024-03-06T07:25:56.164157+0000","last_became_active":"2024-03-06T07:25:55.172189+0000","last_became_peered":"2024-03-06T07:25:55.172189+0000","last_unstale":"2024-03-06T07:25:56.164157+0000","last_undegraded":"2024-03-06T07:25:56.164157+0000","last_fullsized":"2024-03-06T07:25:56.164157+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:53.761906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406503+0000","last_change":"2024-03-06T07:25:54.171718+0000","last_active":"2024-03-06T07:25:56.406503+0000","last_peered":"2024-03-06T07:25:56.406503+0000","last_clean":"2024-03-06T07:25:56.406503+0000","last_became_active":"2024-03-06T07:25:54.171431+0000","last_became_peered":"2024-03-06T07:25:54.171431+0000","last_unstale":"2024-03-06T07:25:56.406503+0000","last_undegraded":"2024-03-06T07:25:56.406503+0000","last_fullsized":"2024-03-06T07:25:56.406503+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:51.464247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167565+0000","last_change":"2024-03-06T07:25:55.184864+0000","last_active":"2024-03-06T07:25:57.167565+0000","last_peered":"2024-03-06T07:25:57.167565+0000","last_clean":"2024-03-06T07:25:57.167565+0000","last_became_active":"2024-03-06T07:25:55.184462+0000","last_became_peered":"2024-03-06T07:25:55.184462+0000","last_unstale":"2024-03-06T07:25:57.167565+0000","last_undegraded":"2024-03-06T07:25:57.167565+0000","last_fullsized":"2024-03-06T07:25:57.167565+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"150'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167010+0000","last_change":"2024-03-06T07:25:54.184297+0000","last_active":"2024-03-06T07:25:57.167010+0000","last_peered":"2024-03-06T07:25:57.167010+0000","last_clean":"2024-03-06T07:25:57.167010+0000","last_became_active":"2024-03-06T07:25:54.184000+0000","last_became_peered":"2024-03-06T07:25:54.184000+0000","last_unstale":"2024-03-06T07:25:57.167010+0000","last_undegraded":"2024-03-06T07:25:57.167010+0000","last_fullsized":"2024-03-06T07:25:57.167010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:06.525995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.178996+0000","last_change":"2024-03-06T07:25:55.173629+0000","last_active":"2024-03-06T07:25:55.178996+0000","last_peered":"2024-03-06T07:25:55.178996+0000","last_clean":"2024-03-06T07:25:55.178996+0000","last_became_active":"2024-03-06T07:25:55.173365+0000","last_became_peered":"2024-03-06T07:25:55.173365+0000","last_unstale":"2024-03-06T07:25:55.178996+0000","last_undegraded":"2024-03-06T07:25:55.178996+0000","last_fullsized":"2024-03-06T07:25:55.178996+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:29:54.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162613+0000","last_change":"2024-03-06T07:25:54.180119+0000","last_active":"2024-03-06T07:25:56.162613+0000","last_peered":"2024-03-06T07:25:56.162613+0000","last_clean":"2024-03-06T07:25:56.162613+0000","last_became_active":"2024-03-06T07:25:54.179853+0000","last_became_peered":"2024-03-06T07:25:54.179853+0000","last_unstale":"2024-03-06T07:25:56.162613+0000","last_undegraded":"2024-03-06T07:25:56.162613+0000","last_fullsized":"2024-03-06T07:25:56.162613+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:43.160051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167659+0000","last_change":"2024-03-06T07:25:55.184654+0000","last_active":"2024-03-06T07:25:57.167659+0000","last_peered":"2024-03-06T07:25:57.167659+0000","last_clean":"2024-03-06T07:25:57.167659+0000","last_became_active":"2024-03-06T07:25:55.184305+0000","last_became_peered":"2024-03-06T07:25:55.184305+0000","last_unstale":"2024-03-06T07:25:57.167659+0000","last_undegraded":"2024-03-06T07:25:57.167659+0000","last_fullsized":"2024-03-06T07:25:57.167659+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.167631+0000","last_change":"2024-03-06T07:25:54.174742+0000","last_active":"2024-03-06T07:25:55.167631+0000","last_peered":"2024-03-06T07:25:55.167631+0000","last_clean":"2024-03-06T07:25:55.167631+0000","last_became_active":"2024-03-06T07:25:54.174462+0000","last_became_peered":"2024-03-06T07:25:54.174462+0000","last_unstale":"2024-03-06T07:25:55.167631+0000","last_undegraded":"2024-03-06T07:25:55.167631+0000","last_fullsized":"2024-03-06T07:25:55.167631+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:47:43.050877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491525+0000","last_change":"2024-03-06T07:25:55.181956+0000","last_active":"2024-03-06T07:25:56.491525+0000","last_peered":"2024-03-06T07:25:56.491525+0000","last_clean":"2024-03-06T07:25:56.491525+0000","last_became_active":"2024-03-06T07:25:55.181658+0000","last_became_peered":"2024-03-06T07:25:55.181658+0000","last_unstale":"2024-03-06T07:25:56.491525+0000","last_undegraded":"2024-03-06T07:25:56.491525+0000","last_fullsized":"2024-03-06T07:25:56.491525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:23.152358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491294+0000","last_change":"2024-03-06T07:25:54.172421+0000","last_active":"2024-03-06T07:25:56.491294+0000","last_peered":"2024-03-06T07:25:56.491294+0000","last_clean":"2024-03-06T07:25:56.491294+0000","last_became_active":"2024-03-06T07:25:54.172157+0000","last_became_peered":"2024-03-06T07:25:54.172157+0000","last_unstale":"2024-03-06T07:25:56.491294+0000","last_undegraded":"2024-03-06T07:25:56.491294+0000","last_fullsized":"2024-03-06T07:25:56.491294+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:07.283064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977285+0000","last_change":"2024-03-06T07:25:55.183461+0000","last_active":"2024-03-06T07:25:56.977285+0000","last_peered":"2024-03-06T07:25:56.977285+0000","last_clean":"2024-03-06T07:25:56.977285+0000","last_became_active":"2024-03-06T07:25:55.183161+0000","last_became_peered":"2024-03-06T07:25:55.183161+0000","last_unstale":"2024-03-06T07:25:56.977285+0000","last_undegraded":"2024-03-06T07:25:56.977285+0000","last_fullsized":"2024-03-06T07:25:56.977285+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162403+0000","last_change":"2024-03-06T07:25:54.182221+0000","last_active":"2024-03-06T07:25:56.162403+0000","last_peered":"2024-03-06T07:25:56.162403+0000","last_clean":"2024-03-06T07:25:56.162403+0000","last_became_active":"2024-03-06T07:25:54.181966+0000","last_became_peered":"2024-03-06T07:25:54.181966+0000","last_unstale":"2024-03-06T07:25:56.162403+0000","last_undegraded":"2024-03-06T07:25:56.162403+0000","last_fullsized":"2024-03-06T07:25:56.162403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:48.357604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162690+0000","last_change":"2024-03-06T07:25:55.178896+0000","last_active":"2024-03-06T07:25:56.162690+0000","last_peered":"2024-03-06T07:25:56.162690+0000","last_clean":"2024-03-06T07:25:56.162690+0000","last_became_active":"2024-03-06T07:25:55.178599+0000","last_became_peered":"2024-03-06T07:25:55.178599+0000","last_unstale":"2024-03-06T07:25:56.162690+0000","last_undegraded":"2024-03-06T07:25:56.162690+0000","last_fullsized":"2024-03-06T07:25:56.162690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:34.372833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164400+0000","last_change":"2024-03-06T07:25:54.176751+0000","last_active":"2024-03-06T07:25:56.164400+0000","last_peered":"2024-03-06T07:25:56.164400+0000","last_clean":"2024-03-06T07:25:56.164400+0000","last_became_active":"2024-03-06T07:25:54.176461+0000","last_became_peered":"2024-03-06T07:25:54.176461+0000","last_unstale":"2024-03-06T07:25:56.164400+0000","last_undegraded":"2024-03-06T07:25:56.164400+0000","last_fullsized":"2024-03-06T07:25:56.164400+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:09.865297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.187012+0000","last_change":"2024-03-06T07:25:55.184289+0000","last_active":"2024-03-06T07:25:55.187012+0000","last_peered":"2024-03-06T07:25:55.187012+0000","last_clean":"2024-03-06T07:25:55.187012+0000","last_became_active":"2024-03-06T07:25:55.184007+0000","last_became_peered":"2024-03-06T07:25:55.184007+0000","last_unstale":"2024-03-06T07:25:55.187012+0000","last_undegraded":"2024-03-06T07:25:55.187012+0000","last_fullsized":"2024-03-06T07:25:55.187012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:59:14.143583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406080+0000","last_change":"2024-03-06T07:25:54.182027+0000","last_active":"2024-03-06T07:25:56.406080+0000","last_peered":"2024-03-06T07:25:56.406080+0000","last_clean":"2024-03-06T07:25:56.406080+0000","last_became_active":"2024-03-06T07:25:54.181755+0000","last_became_peered":"2024-03-06T07:25:54.181755+0000","last_unstale":"2024-03-06T07:25:56.406080+0000","last_undegraded":"2024-03-06T07:25:56.406080+0000","last_fullsized":"2024-03-06T07:25:56.406080+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:03.374674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976862+0000","last_change":"2024-03-06T07:25:55.176555+0000","last_active":"2024-03-06T07:25:56.976862+0000","last_peered":"2024-03-06T07:25:56.976862+0000","last_clean":"2024-03-06T07:25:56.976862+0000","last_became_active":"2024-03-06T07:25:55.176295+0000","last_became_peered":"2024-03-06T07:25:55.176295+0000","last_unstale":"2024-03-06T07:25:56.976862+0000","last_undegraded":"2024-03-06T07:25:56.976862+0000","last_fullsized":"2024-03-06T07:25:56.976862+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:18.514838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162641+0000","last_change":"2024-03-06T07:25:54.184373+0000","last_active":"2024-03-06T07:25:56.162641+0000","last_peered":"2024-03-06T07:25:56.162641+0000","last_clean":"2024-03-06T07:25:56.162641+0000","last_became_active":"2024-03-06T07:25:54.184109+0000","last_became_peered":"2024-03-06T07:25:54.184109+0000","last_unstale":"2024-03-06T07:25:56.162641+0000","last_undegraded":"2024-03-06T07:25:56.162641+0000","last_fullsized":"2024-03-06T07:25:56.162641+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:01.955245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":409,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977257+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:56.977257+0000","last_peered":"2024-03-06T07:25:56.977257+0000","last_clean":"2024-03-06T07:25:56.977257+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:56.977257+0000","last_undegraded":"2024-03-06T07:25:56.977257+0000","last_fullsized":"2024-03-06T07:25:56.977257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":1504557,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":914189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1896448,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":914953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1908736,"data_stored":916405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1875968,"data_stored":907497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1900544,"data_stored":915801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":1504661,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:58.364 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:25:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.508+0000 7f6582b3a640 1 Processor -- start 2024-03-06T07:25:58.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.508+0000 7f6582b3a640 1 -- start start 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f6582b3a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f657c14f360 0x7f657c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f6582b3a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c0c88c0 0x7f657c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f6582b3a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0c73d0 con 0x7f657c0c5880 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f6582b3a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0c8dd0 con 0x7f657c14f360 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f6582b3a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0be2e0 con 0x7f657c0c88c0 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f657c14f360 0x7f657c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f657c14f360 0x7f657c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57224/0 (socket says 172.21.15.17:57224) 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 -- 172.21.15.17:0/288386478 learned_addr learned my addr 172.21.15.17:0/288386478 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f657bfff640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c0c88c0 0x7f657c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 -- 172.21.15.17:0/288386478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c0c88c0 msgr2=0x7f657c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c0c88c0 0x7f657c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 -- 172.21.15.17:0/288386478 >> v1:172.21.15.163:6789/0 conn(0x7f657c0c5880 legacy=0x7f657c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.509+0000 7f65808d6640 1 -- 172.21.15.17:0/288386478 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0c7670 con 0x7f657c14f360 2024-03-06T07:25:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.510+0000 7f65808d6640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c14f360 0x7f657c155c10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6570009870 tx=0x7f657002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53840462b64abc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.510+0000 7f657b7fe640 1 -- 172.21.15.17:0/288386478 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570035020 con 0x7f657c14f360 2024-03-06T07:25:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.510+0000 7f657b7fe640 1 -- 172.21.15.17:0/288386478 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6570031640 con 0x7f657c14f360 2024-03-06T07:25:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.510+0000 7f657b7fe640 1 -- 172.21.15.17:0/288386478 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657002d040 con 0x7f657c14f360 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 -- 172.21.15.17:0/288386478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c14f360 msgr2=0x7f657c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c14f360 0x7f657c155c10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6570009870 tx=0x7f657002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 -- 172.21.15.17:0/288386478 shutdown_connections 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c0c88c0 0x7f657c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 --2- 172.21.15.17:0/288386478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c14f360 0x7f657c155c10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 -- 172.21.15.17:0/288386478 >> 172.21.15.17:0/288386478 conn(0x7f657c0ac570 msgr2=0x7f657c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 -- 172.21.15.17:0/288386478 shutdown_connections 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.511+0000 7f6582b3a640 1 -- 172.21.15.17:0/288386478 wait complete. 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.512+0000 7f6582b3a640 1 Processor -- start 2024-03-06T07:25:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.513+0000 7f6582b3a640 1 -- start start 2024-03-06T07:25:58.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f657c0c88c0 0x7f657c157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c14f360 0x7f657c0bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c159270 con 0x7f657c0c5880 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c7c30 con 0x7f657c0c88c0 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f6582b3a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1593f0 con 0x7f657c14f360 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57226/0 (socket says 172.21.15.17:57226) 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 -- 172.21.15.17:0/3968236800 learned_addr learned my addr 172.21.15.17:0/3968236800 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f657bfff640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f657c0c88c0 0x7f657c157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65810d7640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c14f360 0x7f657c0bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 -- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c14f360 msgr2=0x7f657c0bfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c14f360 0x7f657c0bfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 -- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f657c0c88c0 msgr2=0x7f657c157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f657c0c88c0 0x7f657c157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 -- 172.21.15.17:0/3968236800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0c0820 con 0x7f657c0c5880 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65808d6640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f657002b930 tx=0x7f65700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.514+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570040070 con 0x7f657c0c5880 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.515+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657003e070 con 0x7f657c0c5880 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.515+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6570035040 con 0x7f657c0c5880 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.515+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c0c0ae0 con 0x7f657c0c5880 2024-03-06T07:25:58.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.515+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c0bf6e0 con 0x7f657c0c5880 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.515+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c04f330 con 0x7f657c0c5880 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.516+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6570038020 con 0x7f657c0c5880 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.517+0000 7f65797fa640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 0x7f656006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.517+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65700ac0b0 con 0x7f657c0c5880 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.517+0000 7f657bfff640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 0x7f656006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.517+0000 7f657bfff640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 0x7f656006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f657c1e87e0 tx=0x7f6568008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.520+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f657007ae10 con 0x7f657c0c5880 2024-03-06T07:25:58.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.697+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f657c06cda0 con 0x7f656006b4d0 2024-03-06T07:25:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.703+0000 7f65797fa640 1 -- 172.21.15.17:0/3968236800 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252547 (secure 0 0 0) 0x7f657c06cda0 con 0x7f656006b4d0 2024-03-06T07:25:58.707 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:25:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.706+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 msgr2=0x7f656006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.706+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 0x7f656006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f657c1e87e0 tx=0x7f6568008040 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 msgr2=0x7f657c1eb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f657002b930 tx=0x7f65700046c0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 shutdown_connections 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f656006b4d0 0x7f656006d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f657c14f360 0x7f657c0bfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f657c0c88c0 0x7f657c157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 --2- 172.21.15.17:0/3968236800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f657c0c5880 0x7f657c1eb420 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 >> 172.21.15.17:0/3968236800 conn(0x7f657c0ac570 msgr2=0x7f657c0ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 shutdown_connections 2024-03-06T07:25:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.707+0000 7f6582b3a640 1 -- 172.21.15.17:0/3968236800 wait complete. 2024-03-06T07:25:58.720 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-03-06T07:25:57.418866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264396,"kb_used_data":14932,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710324,"statfs":{"total":773094113280,"available":772823371776,"internally_reserved":0,"allocated":15290368,"data_stored":8492376,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.265246"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491644+0000","last_change":"2024-03-06T07:25:55.171050+0000","last_active":"2024-03-06T07:25:56.491644+0000","last_peered":"2024-03-06T07:25:56.491644+0000","last_clean":"2024-03-06T07:25:56.491644+0000","last_became_active":"2024-03-06T07:25:55.170737+0000","last_became_peered":"2024-03-06T07:25:55.170737+0000","last_unstale":"2024-03-06T07:25:56.491644+0000","last_undegraded":"2024-03-06T07:25:56.491644+0000","last_fullsized":"2024-03-06T07:25:56.491644+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977294+0000","last_change":"2024-03-06T07:25:55.179421+0000","last_active":"2024-03-06T07:25:56.977294+0000","last_peered":"2024-03-06T07:25:56.977294+0000","last_clean":"2024-03-06T07:25:56.977294+0000","last_became_active":"2024-03-06T07:25:55.179145+0000","last_became_peered":"2024-03-06T07:25:55.179145+0000","last_unstale":"2024-03-06T07:25:56.977294+0000","last_undegraded":"2024-03-06T07:25:56.977294+0000","last_fullsized":"2024-03-06T07:25:56.977294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491503+0000","last_change":"2024-03-06T07:25:55.178609+0000","last_active":"2024-03-06T07:25:56.491503+0000","last_peered":"2024-03-06T07:25:56.491503+0000","last_clean":"2024-03-06T07:25:56.491503+0000","last_became_active":"2024-03-06T07:25:55.178334+0000","last_became_peered":"2024-03-06T07:25:55.178334+0000","last_unstale":"2024-03-06T07:25:56.491503+0000","last_undegraded":"2024-03-06T07:25:56.491503+0000","last_fullsized":"2024-03-06T07:25:56.491503+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:19.444548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491505+0000","last_change":"2024-03-06T07:25:55.181844+0000","last_active":"2024-03-06T07:25:56.491505+0000","last_peered":"2024-03-06T07:25:56.491505+0000","last_clean":"2024-03-06T07:25:56.491505+0000","last_became_active":"2024-03-06T07:25:55.181282+0000","last_became_peered":"2024-03-06T07:25:55.181282+0000","last_unstale":"2024-03-06T07:25:56.491505+0000","last_undegraded":"2024-03-06T07:25:56.491505+0000","last_fullsized":"2024-03-06T07:25:56.491505+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:58.454621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406406+0000","last_change":"2024-03-06T07:25:55.188179+0000","last_active":"2024-03-06T07:25:56.406406+0000","last_peered":"2024-03-06T07:25:56.406406+0000","last_clean":"2024-03-06T07:25:56.406406+0000","last_became_active":"2024-03-06T07:25:55.187905+0000","last_became_peered":"2024-03-06T07:25:55.187905+0000","last_unstale":"2024-03-06T07:25:56.406406+0000","last_undegraded":"2024-03-06T07:25:56.406406+0000","last_fullsized":"2024-03-06T07:25:56.406406+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:31.959459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976857+0000","last_change":"2024-03-06T07:25:55.179259+0000","last_active":"2024-03-06T07:25:56.976857+0000","last_peered":"2024-03-06T07:25:56.976857+0000","last_clean":"2024-03-06T07:25:56.976857+0000","last_became_active":"2024-03-06T07:25:55.178971+0000","last_became_peered":"2024-03-06T07:25:55.178971+0000","last_unstale":"2024-03-06T07:25:56.976857+0000","last_undegraded":"2024-03-06T07:25:56.976857+0000","last_fullsized":"2024-03-06T07:25:56.976857+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:52.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167499+0000","last_change":"2024-03-06T07:25:55.189198+0000","last_active":"2024-03-06T07:25:57.167499+0000","last_peered":"2024-03-06T07:25:57.167499+0000","last_clean":"2024-03-06T07:25:57.167499+0000","last_became_active":"2024-03-06T07:25:55.188937+0000","last_became_peered":"2024-03-06T07:25:55.188937+0000","last_unstale":"2024-03-06T07:25:57.167499+0000","last_undegraded":"2024-03-06T07:25:57.167499+0000","last_fullsized":"2024-03-06T07:25:57.167499+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167456+0000","last_change":"2024-03-06T07:25:55.174913+0000","last_active":"2024-03-06T07:25:57.167456+0000","last_peered":"2024-03-06T07:25:57.167456+0000","last_clean":"2024-03-06T07:25:57.167456+0000","last_became_active":"2024-03-06T07:25:55.174627+0000","last_became_peered":"2024-03-06T07:25:55.174627+0000","last_unstale":"2024-03-06T07:25:57.167456+0000","last_undegraded":"2024-03-06T07:25:57.167456+0000","last_fullsized":"2024-03-06T07:25:57.167456+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167916+0000","last_change":"2024-03-06T07:25:55.175881+0000","last_active":"2024-03-06T07:25:57.167916+0000","last_peered":"2024-03-06T07:25:57.167916+0000","last_clean":"2024-03-06T07:25:57.167916+0000","last_became_active":"2024-03-06T07:25:55.175604+0000","last_became_peered":"2024-03-06T07:25:55.175604+0000","last_unstale":"2024-03-06T07:25:57.167916+0000","last_undegraded":"2024-03-06T07:25:57.167916+0000","last_fullsized":"2024-03-06T07:25:57.167916+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:18.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167953+0000","last_change":"2024-03-06T07:25:55.174883+0000","last_active":"2024-03-06T07:25:57.167953+0000","last_peered":"2024-03-06T07:25:57.167953+0000","last_clean":"2024-03-06T07:25:57.167953+0000","last_became_active":"2024-03-06T07:25:55.174590+0000","last_became_peered":"2024-03-06T07:25:55.174590+0000","last_unstale":"2024-03-06T07:25:57.167953+0000","last_undegraded":"2024-03-06T07:25:57.167953+0000","last_fullsized":"2024-03-06T07:25:57.167953+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164754+0000","last_change":"2024-03-06T07:25:55.182365+0000","last_active":"2024-03-06T07:25:56.164754+0000","last_peered":"2024-03-06T07:25:56.164754+0000","last_clean":"2024-03-06T07:25:56.164754+0000","last_became_active":"2024-03-06T07:25:55.182083+0000","last_became_peered":"2024-03-06T07:25:55.182083+0000","last_unstale":"2024-03-06T07:25:56.164754+0000","last_undegraded":"2024-03-06T07:25:56.164754+0000","last_fullsized":"2024-03-06T07:25:56.164754+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:53.751341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977198+0000","last_change":"2024-03-06T07:25:55.179820+0000","last_active":"2024-03-06T07:25:56.977198+0000","last_peered":"2024-03-06T07:25:56.977198+0000","last_clean":"2024-03-06T07:25:56.977198+0000","last_became_active":"2024-03-06T07:25:55.179573+0000","last_became_peered":"2024-03-06T07:25:55.179573+0000","last_unstale":"2024-03-06T07:25:56.977198+0000","last_undegraded":"2024-03-06T07:25:56.977198+0000","last_fullsized":"2024-03-06T07:25:56.977198+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162398+0000","last_change":"2024-03-06T07:25:55.178116+0000","last_active":"2024-03-06T07:25:56.162398+0000","last_peered":"2024-03-06T07:25:56.162398+0000","last_clean":"2024-03-06T07:25:56.162398+0000","last_became_active":"2024-03-06T07:25:55.177769+0000","last_became_peered":"2024-03-06T07:25:55.177769+0000","last_unstale":"2024-03-06T07:25:56.162398+0000","last_undegraded":"2024-03-06T07:25:56.162398+0000","last_fullsized":"2024-03-06T07:25:56.162398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:19.818533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406254+0000","last_change":"2024-03-06T07:25:55.185952+0000","last_active":"2024-03-06T07:25:56.406254+0000","last_peered":"2024-03-06T07:25:56.406254+0000","last_clean":"2024-03-06T07:25:56.406254+0000","last_became_active":"2024-03-06T07:25:55.185677+0000","last_became_peered":"2024-03-06T07:25:55.185677+0000","last_unstale":"2024-03-06T07:25:56.406254+0000","last_undegraded":"2024-03-06T07:25:56.406254+0000","last_fullsized":"2024-03-06T07:25:56.406254+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:02.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164195+0000","last_change":"2024-03-06T07:25:55.180387+0000","last_active":"2024-03-06T07:25:56.164195+0000","last_peered":"2024-03-06T07:25:56.164195+0000","last_clean":"2024-03-06T07:25:56.164195+0000","last_became_active":"2024-03-06T07:25:55.179778+0000","last_became_peered":"2024-03-06T07:25:55.179778+0000","last_unstale":"2024-03-06T07:25:56.164195+0000","last_undegraded":"2024-03-06T07:25:56.164195+0000","last_fullsized":"2024-03-06T07:25:56.164195+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:16.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491231+0000","last_change":"2024-03-06T07:25:55.180281+0000","last_active":"2024-03-06T07:25:56.491231+0000","last_peered":"2024-03-06T07:25:56.491231+0000","last_clean":"2024-03-06T07:25:56.491231+0000","last_became_active":"2024-03-06T07:25:55.180014+0000","last_became_peered":"2024-03-06T07:25:55.180014+0000","last_unstale":"2024-03-06T07:25:56.491231+0000","last_undegraded":"2024-03-06T07:25:56.491231+0000","last_fullsized":"2024-03-06T07:25:56.491231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:08:51.191943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167944+0000","last_change":"2024-03-06T07:25:55.180083+0000","last_active":"2024-03-06T07:25:57.167944+0000","last_peered":"2024-03-06T07:25:57.167944+0000","last_clean":"2024-03-06T07:25:57.167944+0000","last_became_active":"2024-03-06T07:25:55.179804+0000","last_became_peered":"2024-03-06T07:25:55.179804+0000","last_unstale":"2024-03-06T07:25:57.167944+0000","last_undegraded":"2024-03-06T07:25:57.167944+0000","last_fullsized":"2024-03-06T07:25:57.167944+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:03.229947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164987+0000","last_change":"2024-03-06T07:25:55.176450+0000","last_active":"2024-03-06T07:25:56.164987+0000","last_peered":"2024-03-06T07:25:56.164987+0000","last_clean":"2024-03-06T07:25:56.164987+0000","last_became_active":"2024-03-06T07:25:55.176194+0000","last_became_peered":"2024-03-06T07:25:55.176194+0000","last_unstale":"2024-03-06T07:25:56.164987+0000","last_undegraded":"2024-03-06T07:25:56.164987+0000","last_fullsized":"2024-03-06T07:25:56.164987+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:53.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164959+0000","last_change":"2024-03-06T07:25:55.176884+0000","last_active":"2024-03-06T07:25:56.164959+0000","last_peered":"2024-03-06T07:25:56.164959+0000","last_clean":"2024-03-06T07:25:56.164959+0000","last_became_active":"2024-03-06T07:25:55.176546+0000","last_became_peered":"2024-03-06T07:25:55.176546+0000","last_unstale":"2024-03-06T07:25:56.164959+0000","last_undegraded":"2024-03-06T07:25:56.164959+0000","last_fullsized":"2024-03-06T07:25:56.164959+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:22.733869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.184293+0000","last_change":"2024-03-06T07:25:55.181620+0000","last_active":"2024-03-06T07:25:55.184293+0000","last_peered":"2024-03-06T07:25:55.184293+0000","last_clean":"2024-03-06T07:25:55.184293+0000","last_became_active":"2024-03-06T07:25:55.181336+0000","last_became_peered":"2024-03-06T07:25:55.181336+0000","last_unstale":"2024-03-06T07:25:55.184293+0000","last_undegraded":"2024-03-06T07:25:55.184293+0000","last_fullsized":"2024-03-06T07:25:55.184293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:28.893019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162472+0000","last_change":"2024-03-06T07:25:55.171658+0000","last_active":"2024-03-06T07:25:56.162472+0000","last_peered":"2024-03-06T07:25:56.162472+0000","last_clean":"2024-03-06T07:25:56.162472+0000","last_became_active":"2024-03-06T07:25:55.171366+0000","last_became_peered":"2024-03-06T07:25:55.171366+0000","last_unstale":"2024-03-06T07:25:56.162472+0000","last_undegraded":"2024-03-06T07:25:56.162472+0000","last_fullsized":"2024-03-06T07:25:56.162472+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:03.613423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976898+0000","last_change":"2024-03-06T07:25:55.185023+0000","last_active":"2024-03-06T07:25:56.976898+0000","last_peered":"2024-03-06T07:25:56.976898+0000","last_clean":"2024-03-06T07:25:56.976898+0000","last_became_active":"2024-03-06T07:25:55.184708+0000","last_became_peered":"2024-03-06T07:25:55.184708+0000","last_unstale":"2024-03-06T07:25:56.976898+0000","last_undegraded":"2024-03-06T07:25:56.976898+0000","last_fullsized":"2024-03-06T07:25:56.976898+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:25.354294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406171+0000","last_change":"2024-03-06T07:25:55.189044+0000","last_active":"2024-03-06T07:25:56.406171+0000","last_peered":"2024-03-06T07:25:56.406171+0000","last_clean":"2024-03-06T07:25:56.406171+0000","last_became_active":"2024-03-06T07:25:55.188772+0000","last_became_peered":"2024-03-06T07:25:55.188772+0000","last_unstale":"2024-03-06T07:25:56.406171+0000","last_undegraded":"2024-03-06T07:25:56.406171+0000","last_fullsized":"2024-03-06T07:25:56.406171+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491346+0000","last_change":"2024-03-06T07:25:55.179621+0000","last_active":"2024-03-06T07:25:56.491346+0000","last_peered":"2024-03-06T07:25:56.491346+0000","last_clean":"2024-03-06T07:25:56.491346+0000","last_became_active":"2024-03-06T07:25:55.179320+0000","last_became_peered":"2024-03-06T07:25:55.179320+0000","last_unstale":"2024-03-06T07:25:56.491346+0000","last_undegraded":"2024-03-06T07:25:56.491346+0000","last_fullsized":"2024-03-06T07:25:56.491346+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:38.124631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162773+0000","last_change":"2024-03-06T07:25:55.183095+0000","last_active":"2024-03-06T07:25:56.162773+0000","last_peered":"2024-03-06T07:25:56.162773+0000","last_clean":"2024-03-06T07:25:56.162773+0000","last_became_active":"2024-03-06T07:25:55.182807+0000","last_became_peered":"2024-03-06T07:25:55.182807+0000","last_unstale":"2024-03-06T07:25:56.162773+0000","last_undegraded":"2024-03-06T07:25:56.162773+0000","last_fullsized":"2024-03-06T07:25:56.162773+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:02.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.183583+0000","last_change":"2024-03-06T07:25:55.180267+0000","last_active":"2024-03-06T07:25:55.183583+0000","last_peered":"2024-03-06T07:25:55.183583+0000","last_clean":"2024-03-06T07:25:55.183583+0000","last_became_active":"2024-03-06T07:25:55.180006+0000","last_became_peered":"2024-03-06T07:25:55.180006+0000","last_unstale":"2024-03-06T07:25:55.183583+0000","last_undegraded":"2024-03-06T07:25:55.183583+0000","last_fullsized":"2024-03-06T07:25:55.183583+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:15:14.681093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977250+0000","last_change":"2024-03-06T07:25:55.185010+0000","last_active":"2024-03-06T07:25:56.977250+0000","last_peered":"2024-03-06T07:25:56.977250+0000","last_clean":"2024-03-06T07:25:56.977250+0000","last_became_active":"2024-03-06T07:25:55.184723+0000","last_became_peered":"2024-03-06T07:25:55.184723+0000","last_unstale":"2024-03-06T07:25:56.977250+0000","last_undegraded":"2024-03-06T07:25:56.977250+0000","last_fullsized":"2024-03-06T07:25:56.977250+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162381+0000","last_change":"2024-03-06T07:25:55.171489+0000","last_active":"2024-03-06T07:25:56.162381+0000","last_peered":"2024-03-06T07:25:56.162381+0000","last_clean":"2024-03-06T07:25:56.162381+0000","last_became_active":"2024-03-06T07:25:55.171184+0000","last_became_peered":"2024-03-06T07:25:55.171184+0000","last_unstale":"2024-03-06T07:25:56.162381+0000","last_undegraded":"2024-03-06T07:25:56.162381+0000","last_fullsized":"2024-03-06T07:25:56.162381+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:03.888318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977078+0000","last_change":"2024-03-06T07:25:55.175885+0000","last_active":"2024-03-06T07:25:56.977078+0000","last_peered":"2024-03-06T07:25:56.977078+0000","last_clean":"2024-03-06T07:25:56.977078+0000","last_became_active":"2024-03-06T07:25:55.175607+0000","last_became_peered":"2024-03-06T07:25:55.175607+0000","last_unstale":"2024-03-06T07:25:56.977078+0000","last_undegraded":"2024-03-06T07:25:56.977078+0000","last_fullsized":"2024-03-06T07:25:56.977078+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165294+0000","last_change":"2024-03-06T07:25:55.182544+0000","last_active":"2024-03-06T07:25:56.165294+0000","last_peered":"2024-03-06T07:25:56.165294+0000","last_clean":"2024-03-06T07:25:56.165294+0000","last_became_active":"2024-03-06T07:25:55.182215+0000","last_became_peered":"2024-03-06T07:25:55.182215+0000","last_unstale":"2024-03-06T07:25:56.165294+0000","last_undegraded":"2024-03-06T07:25:56.165294+0000","last_fullsized":"2024-03-06T07:25:56.165294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:47.717714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164251+0000","last_change":"2024-03-06T07:25:55.174623+0000","last_active":"2024-03-06T07:25:56.164251+0000","last_peered":"2024-03-06T07:25:56.164251+0000","last_clean":"2024-03-06T07:25:56.164251+0000","last_became_active":"2024-03-06T07:25:55.174336+0000","last_became_peered":"2024-03-06T07:25:55.174336+0000","last_unstale":"2024-03-06T07:25:56.164251+0000","last_undegraded":"2024-03-06T07:25:56.164251+0000","last_fullsized":"2024-03-06T07:25:56.164251+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:15.212900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164787+0000","last_change":"2024-03-06T07:25:55.173469+0000","last_active":"2024-03-06T07:25:56.164787+0000","last_peered":"2024-03-06T07:25:56.164787+0000","last_clean":"2024-03-06T07:25:56.164787+0000","last_became_active":"2024-03-06T07:25:55.173211+0000","last_became_peered":"2024-03-06T07:25:55.173211+0000","last_unstale":"2024-03-06T07:25:56.164787+0000","last_undegraded":"2024-03-06T07:25:56.164787+0000","last_fullsized":"2024-03-06T07:25:56.164787+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:44.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976975+0000","last_change":"2024-03-06T07:25:55.183313+0000","last_active":"2024-03-06T07:25:56.976975+0000","last_peered":"2024-03-06T07:25:56.976975+0000","last_clean":"2024-03-06T07:25:56.976975+0000","last_became_active":"2024-03-06T07:25:55.183007+0000","last_became_peered":"2024-03-06T07:25:55.183007+0000","last_unstale":"2024-03-06T07:25:56.976975+0000","last_undegraded":"2024-03-06T07:25:56.976975+0000","last_fullsized":"2024-03-06T07:25:56.976975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:53.878454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491720+0000","last_change":"2024-03-06T07:25:54.177412+0000","last_active":"2024-03-06T07:25:56.491720+0000","last_peered":"2024-03-06T07:25:56.491720+0000","last_clean":"2024-03-06T07:25:56.491720+0000","last_became_active":"2024-03-06T07:25:54.177071+0000","last_became_peered":"2024-03-06T07:25:54.177071+0000","last_unstale":"2024-03-06T07:25:56.491720+0000","last_undegraded":"2024-03-06T07:25:56.491720+0000","last_fullsized":"2024-03-06T07:25:56.491720+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:58.975791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162259+0000","last_change":"2024-03-06T07:25:55.169574+0000","last_active":"2024-03-06T07:25:56.162259+0000","last_peered":"2024-03-06T07:25:56.162259+0000","last_clean":"2024-03-06T07:25:56.162259+0000","last_became_active":"2024-03-06T07:25:55.168987+0000","last_became_peered":"2024-03-06T07:25:55.168987+0000","last_unstale":"2024-03-06T07:25:56.162259+0000","last_undegraded":"2024-03-06T07:25:56.162259+0000","last_fullsized":"2024-03-06T07:25:56.162259+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:55.457043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406436+0000","last_change":"2024-03-06T07:25:54.178532+0000","last_active":"2024-03-06T07:25:56.406436+0000","last_peered":"2024-03-06T07:25:56.406436+0000","last_clean":"2024-03-06T07:25:56.406436+0000","last_became_active":"2024-03-06T07:25:54.178228+0000","last_became_peered":"2024-03-06T07:25:54.178228+0000","last_unstale":"2024-03-06T07:25:56.406436+0000","last_undegraded":"2024-03-06T07:25:56.406436+0000","last_fullsized":"2024-03-06T07:25:56.406436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:07.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406279+0000","last_change":"2024-03-06T07:25:55.181488+0000","last_active":"2024-03-06T07:25:56.406279+0000","last_peered":"2024-03-06T07:25:56.406279+0000","last_clean":"2024-03-06T07:25:56.406279+0000","last_became_active":"2024-03-06T07:25:55.181212+0000","last_became_peered":"2024-03-06T07:25:55.181212+0000","last_unstale":"2024-03-06T07:25:56.406279+0000","last_undegraded":"2024-03-06T07:25:56.406279+0000","last_fullsized":"2024-03-06T07:25:56.406279+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:07.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165147+0000","last_change":"2024-03-06T07:25:54.176834+0000","last_active":"2024-03-06T07:25:56.165147+0000","last_peered":"2024-03-06T07:25:56.165147+0000","last_clean":"2024-03-06T07:25:56.165147+0000","last_became_active":"2024-03-06T07:25:54.176517+0000","last_became_peered":"2024-03-06T07:25:54.176517+0000","last_unstale":"2024-03-06T07:25:56.165147+0000","last_undegraded":"2024-03-06T07:25:56.165147+0000","last_fullsized":"2024-03-06T07:25:56.165147+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:46.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977126+0000","last_change":"2024-03-06T07:25:55.186954+0000","last_active":"2024-03-06T07:25:56.977126+0000","last_peered":"2024-03-06T07:25:56.977126+0000","last_clean":"2024-03-06T07:25:56.977126+0000","last_became_active":"2024-03-06T07:25:55.186700+0000","last_became_peered":"2024-03-06T07:25:55.186700+0000","last_unstale":"2024-03-06T07:25:56.977126+0000","last_undegraded":"2024-03-06T07:25:56.977126+0000","last_fullsized":"2024-03-06T07:25:56.977126+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"149'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.193360+0000","last_change":"2024-03-06T07:25:54.173923+0000","last_active":"2024-03-06T07:25:55.193360+0000","last_peered":"2024-03-06T07:25:55.193360+0000","last_clean":"2024-03-06T07:25:55.193360+0000","last_became_active":"2024-03-06T07:25:54.173628+0000","last_became_peered":"2024-03-06T07:25:54.173628+0000","last_unstale":"2024-03-06T07:25:55.193360+0000","last_undegraded":"2024-03-06T07:25:55.193360+0000","last_fullsized":"2024-03-06T07:25:55.193360+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:18.074282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167641+0000","last_change":"2024-03-06T07:25:55.179023+0000","last_active":"2024-03-06T07:25:57.167641+0000","last_peered":"2024-03-06T07:25:57.167641+0000","last_clean":"2024-03-06T07:25:57.167641+0000","last_became_active":"2024-03-06T07:25:55.178604+0000","last_became_peered":"2024-03-06T07:25:55.178604+0000","last_unstale":"2024-03-06T07:25:57.167641+0000","last_undegraded":"2024-03-06T07:25:57.167641+0000","last_fullsized":"2024-03-06T07:25:57.167641+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167634+0000","last_change":"2024-03-06T07:25:54.179340+0000","last_active":"2024-03-06T07:25:57.167634+0000","last_peered":"2024-03-06T07:25:57.167634+0000","last_clean":"2024-03-06T07:25:57.167634+0000","last_became_active":"2024-03-06T07:25:54.179061+0000","last_became_peered":"2024-03-06T07:25:54.179061+0000","last_unstale":"2024-03-06T07:25:57.167634+0000","last_undegraded":"2024-03-06T07:25:57.167634+0000","last_fullsized":"2024-03-06T07:25:57.167634+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:24.779584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.193063+0000","last_change":"2024-03-06T07:25:55.186844+0000","last_active":"2024-03-06T07:25:55.193063+0000","last_peered":"2024-03-06T07:25:55.193063+0000","last_clean":"2024-03-06T07:25:55.193063+0000","last_became_active":"2024-03-06T07:25:55.186558+0000","last_became_peered":"2024-03-06T07:25:55.186558+0000","last_unstale":"2024-03-06T07:25:55.193063+0000","last_undegraded":"2024-03-06T07:25:55.193063+0000","last_fullsized":"2024-03-06T07:25:55.193063+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:56:27.743379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406356+0000","last_change":"2024-03-06T07:25:54.182713+0000","last_active":"2024-03-06T07:25:56.406356+0000","last_peered":"2024-03-06T07:25:56.406356+0000","last_clean":"2024-03-06T07:25:56.406356+0000","last_became_active":"2024-03-06T07:25:54.182447+0000","last_became_peered":"2024-03-06T07:25:54.182447+0000","last_unstale":"2024-03-06T07:25:56.406356+0000","last_undegraded":"2024-03-06T07:25:56.406356+0000","last_fullsized":"2024-03-06T07:25:56.406356+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:17.509614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.172898+0000","last_change":"2024-03-06T07:25:55.172223+0000","last_active":"2024-03-06T07:25:55.172898+0000","last_peered":"2024-03-06T07:25:55.172898+0000","last_clean":"2024-03-06T07:25:55.172898+0000","last_became_active":"2024-03-06T07:25:55.171895+0000","last_became_peered":"2024-03-06T07:25:55.171895+0000","last_unstale":"2024-03-06T07:25:55.172898+0000","last_undegraded":"2024-03-06T07:25:55.172898+0000","last_fullsized":"2024-03-06T07:25:55.172898+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:11:38.683915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.166186+0000","last_change":"2024-03-06T07:25:54.172467+0000","last_active":"2024-03-06T07:25:55.166186+0000","last_peered":"2024-03-06T07:25:55.166186+0000","last_clean":"2024-03-06T07:25:55.166186+0000","last_became_active":"2024-03-06T07:25:54.172185+0000","last_became_peered":"2024-03-06T07:25:54.172185+0000","last_unstale":"2024-03-06T07:25:55.166186+0000","last_undegraded":"2024-03-06T07:25:55.166186+0000","last_fullsized":"2024-03-06T07:25:55.166186+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:58.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491699+0000","last_change":"2024-03-06T07:25:55.175586+0000","last_active":"2024-03-06T07:25:56.491699+0000","last_peered":"2024-03-06T07:25:56.491699+0000","last_clean":"2024-03-06T07:25:56.491699+0000","last_became_active":"2024-03-06T07:25:55.175328+0000","last_became_peered":"2024-03-06T07:25:55.175328+0000","last_unstale":"2024-03-06T07:25:56.491699+0000","last_undegraded":"2024-03-06T07:25:56.491699+0000","last_fullsized":"2024-03-06T07:25:56.491699+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:34.059703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167555+0000","last_change":"2024-03-06T07:25:54.175104+0000","last_active":"2024-03-06T07:25:57.167555+0000","last_peered":"2024-03-06T07:25:57.167555+0000","last_clean":"2024-03-06T07:25:57.167555+0000","last_became_active":"2024-03-06T07:25:54.174837+0000","last_became_peered":"2024-03-06T07:25:54.174837+0000","last_unstale":"2024-03-06T07:25:57.167555+0000","last_undegraded":"2024-03-06T07:25:57.167555+0000","last_fullsized":"2024-03-06T07:25:57.167555+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:15.768972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976860+0000","last_change":"2024-03-06T07:25:55.183382+0000","last_active":"2024-03-06T07:25:56.976860+0000","last_peered":"2024-03-06T07:25:56.976860+0000","last_clean":"2024-03-06T07:25:56.976860+0000","last_became_active":"2024-03-06T07:25:55.182975+0000","last_became_peered":"2024-03-06T07:25:55.182975+0000","last_unstale":"2024-03-06T07:25:56.976860+0000","last_undegraded":"2024-03-06T07:25:56.976860+0000","last_fullsized":"2024-03-06T07:25:56.976860+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:04.379898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162704+0000","last_change":"2024-03-06T07:25:54.184577+0000","last_active":"2024-03-06T07:25:56.162704+0000","last_peered":"2024-03-06T07:25:56.162704+0000","last_clean":"2024-03-06T07:25:56.162704+0000","last_became_active":"2024-03-06T07:25:54.184068+0000","last_became_peered":"2024-03-06T07:25:54.184068+0000","last_unstale":"2024-03-06T07:25:56.162704+0000","last_undegraded":"2024-03-06T07:25:56.162704+0000","last_fullsized":"2024-03-06T07:25:56.162704+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:30.954235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162382+0000","last_change":"2024-03-06T07:25:55.175578+0000","last_active":"2024-03-06T07:25:56.162382+0000","last_peered":"2024-03-06T07:25:56.162382+0000","last_clean":"2024-03-06T07:25:56.162382+0000","last_became_active":"2024-03-06T07:25:55.175257+0000","last_became_peered":"2024-03-06T07:25:55.175257+0000","last_unstale":"2024-03-06T07:25:56.162382+0000","last_undegraded":"2024-03-06T07:25:56.162382+0000","last_fullsized":"2024-03-06T07:25:56.162382+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:56.174605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491490+0000","last_change":"2024-03-06T07:25:54.177422+0000","last_active":"2024-03-06T07:25:56.491490+0000","last_peered":"2024-03-06T07:25:56.491490+0000","last_clean":"2024-03-06T07:25:56.491490+0000","last_became_active":"2024-03-06T07:25:54.177048+0000","last_became_peered":"2024-03-06T07:25:54.177048+0000","last_unstale":"2024-03-06T07:25:56.491490+0000","last_undegraded":"2024-03-06T07:25:56.491490+0000","last_fullsized":"2024-03-06T07:25:56.491490+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:21.672283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.184323+0000","last_change":"2024-03-06T07:25:55.182193+0000","last_active":"2024-03-06T07:25:55.184323+0000","last_peered":"2024-03-06T07:25:55.184323+0000","last_clean":"2024-03-06T07:25:55.184323+0000","last_became_active":"2024-03-06T07:25:55.181940+0000","last_became_peered":"2024-03-06T07:25:55.181940+0000","last_unstale":"2024-03-06T07:25:55.184323+0000","last_undegraded":"2024-03-06T07:25:55.184323+0000","last_fullsized":"2024-03-06T07:25:55.184323+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:46:22.100315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162576+0000","last_change":"2024-03-06T07:25:54.184311+0000","last_active":"2024-03-06T07:25:56.162576+0000","last_peered":"2024-03-06T07:25:56.162576+0000","last_clean":"2024-03-06T07:25:56.162576+0000","last_became_active":"2024-03-06T07:25:54.183969+0000","last_became_peered":"2024-03-06T07:25:54.183969+0000","last_unstale":"2024-03-06T07:25:56.162576+0000","last_undegraded":"2024-03-06T07:25:56.162576+0000","last_fullsized":"2024-03-06T07:25:56.162576+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:08.451250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164497+0000","last_change":"2024-03-06T07:25:55.174144+0000","last_active":"2024-03-06T07:25:56.164497+0000","last_peered":"2024-03-06T07:25:56.164497+0000","last_clean":"2024-03-06T07:25:56.164497+0000","last_became_active":"2024-03-06T07:25:55.173885+0000","last_became_peered":"2024-03-06T07:25:55.173885+0000","last_unstale":"2024-03-06T07:25:56.164497+0000","last_undegraded":"2024-03-06T07:25:56.164497+0000","last_fullsized":"2024-03-06T07:25:56.164497+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:59.981015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406391+0000","last_change":"2024-03-06T07:25:54.175000+0000","last_active":"2024-03-06T07:25:56.406391+0000","last_peered":"2024-03-06T07:25:56.406391+0000","last_clean":"2024-03-06T07:25:56.406391+0000","last_became_active":"2024-03-06T07:25:54.174717+0000","last_became_peered":"2024-03-06T07:25:54.174717+0000","last_unstale":"2024-03-06T07:25:56.406391+0000","last_undegraded":"2024-03-06T07:25:56.406391+0000","last_fullsized":"2024-03-06T07:25:56.406391+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:52.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167823+0000","last_change":"2024-03-06T07:25:55.186488+0000","last_active":"2024-03-06T07:25:57.167823+0000","last_peered":"2024-03-06T07:25:57.167823+0000","last_clean":"2024-03-06T07:25:57.167823+0000","last_became_active":"2024-03-06T07:25:55.185803+0000","last_became_peered":"2024-03-06T07:25:55.185803+0000","last_unstale":"2024-03-06T07:25:57.167823+0000","last_undegraded":"2024-03-06T07:25:57.167823+0000","last_fullsized":"2024-03-06T07:25:57.167823+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162315+0000","last_change":"2024-03-06T07:25:54.178593+0000","last_active":"2024-03-06T07:25:56.162315+0000","last_peered":"2024-03-06T07:25:56.162315+0000","last_clean":"2024-03-06T07:25:56.162315+0000","last_became_active":"2024-03-06T07:25:54.178229+0000","last_became_peered":"2024-03-06T07:25:54.178229+0000","last_unstale":"2024-03-06T07:25:56.162315+0000","last_undegraded":"2024-03-06T07:25:56.162315+0000","last_fullsized":"2024-03-06T07:25:56.162315+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:59.187499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165275+0000","last_change":"2024-03-06T07:25:55.175110+0000","last_active":"2024-03-06T07:25:56.165275+0000","last_peered":"2024-03-06T07:25:56.165275+0000","last_clean":"2024-03-06T07:25:56.165275+0000","last_became_active":"2024-03-06T07:25:55.174818+0000","last_became_peered":"2024-03-06T07:25:55.174818+0000","last_unstale":"2024-03-06T07:25:56.165275+0000","last_undegraded":"2024-03-06T07:25:56.165275+0000","last_fullsized":"2024-03-06T07:25:56.165275+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:18.191924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164306+0000","last_change":"2024-03-06T07:25:54.178997+0000","last_active":"2024-03-06T07:25:56.164306+0000","last_peered":"2024-03-06T07:25:56.164306+0000","last_clean":"2024-03-06T07:25:56.164306+0000","last_became_active":"2024-03-06T07:25:54.178699+0000","last_became_peered":"2024-03-06T07:25:54.178699+0000","last_unstale":"2024-03-06T07:25:56.164306+0000","last_undegraded":"2024-03-06T07:25:56.164306+0000","last_fullsized":"2024-03-06T07:25:56.164306+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:04.021145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.185957+0000","last_change":"2024-03-06T07:25:55.181498+0000","last_active":"2024-03-06T07:25:55.185957+0000","last_peered":"2024-03-06T07:25:55.185957+0000","last_clean":"2024-03-06T07:25:55.185957+0000","last_became_active":"2024-03-06T07:25:55.180900+0000","last_became_peered":"2024-03-06T07:25:55.180900+0000","last_unstale":"2024-03-06T07:25:55.185957+0000","last_undegraded":"2024-03-06T07:25:55.185957+0000","last_fullsized":"2024-03-06T07:25:55.185957+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:39.618266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167725+0000","last_change":"2024-03-06T07:25:54.177051+0000","last_active":"2024-03-06T07:25:57.167725+0000","last_peered":"2024-03-06T07:25:57.167725+0000","last_clean":"2024-03-06T07:25:57.167725+0000","last_became_active":"2024-03-06T07:25:54.176774+0000","last_became_peered":"2024-03-06T07:25:54.176774+0000","last_unstale":"2024-03-06T07:25:57.167725+0000","last_undegraded":"2024-03-06T07:25:57.167725+0000","last_fullsized":"2024-03-06T07:25:57.167725+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:14.207675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167454+0000","last_change":"2024-03-06T07:25:55.190742+0000","last_active":"2024-03-06T07:25:57.167454+0000","last_peered":"2024-03-06T07:25:57.167454+0000","last_clean":"2024-03-06T07:25:57.167454+0000","last_became_active":"2024-03-06T07:25:55.190479+0000","last_became_peered":"2024-03-06T07:25:55.190479+0000","last_unstale":"2024-03-06T07:25:57.167454+0000","last_undegraded":"2024-03-06T07:25:57.167454+0000","last_fullsized":"2024-03-06T07:25:57.167454+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162770+0000","last_change":"2024-03-06T07:25:54.178661+0000","last_active":"2024-03-06T07:25:56.162770+0000","last_peered":"2024-03-06T07:25:56.162770+0000","last_clean":"2024-03-06T07:25:56.162770+0000","last_became_active":"2024-03-06T07:25:54.178379+0000","last_became_peered":"2024-03-06T07:25:54.178379+0000","last_unstale":"2024-03-06T07:25:56.162770+0000","last_undegraded":"2024-03-06T07:25:56.162770+0000","last_fullsized":"2024-03-06T07:25:56.162770+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:30.562792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.165181+0000","last_change":"2024-03-06T07:25:55.178452+0000","last_active":"2024-03-06T07:25:56.165181+0000","last_peered":"2024-03-06T07:25:56.165181+0000","last_clean":"2024-03-06T07:25:56.165181+0000","last_became_active":"2024-03-06T07:25:55.178193+0000","last_became_peered":"2024-03-06T07:25:55.178193+0000","last_unstale":"2024-03-06T07:25:56.165181+0000","last_undegraded":"2024-03-06T07:25:56.165181+0000","last_fullsized":"2024-03-06T07:25:56.165181+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:32.109891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167947+0000","last_change":"2024-03-06T07:25:54.173705+0000","last_active":"2024-03-06T07:25:57.167947+0000","last_peered":"2024-03-06T07:25:57.167947+0000","last_clean":"2024-03-06T07:25:57.167947+0000","last_became_active":"2024-03-06T07:25:54.173438+0000","last_became_peered":"2024-03-06T07:25:54.173438+0000","last_unstale":"2024-03-06T07:25:57.167947+0000","last_undegraded":"2024-03-06T07:25:57.167947+0000","last_fullsized":"2024-03-06T07:25:57.167947+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:51.526470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164497+0000","last_change":"2024-03-06T07:25:55.172838+0000","last_active":"2024-03-06T07:25:56.164497+0000","last_peered":"2024-03-06T07:25:56.164497+0000","last_clean":"2024-03-06T07:25:56.164497+0000","last_became_active":"2024-03-06T07:25:55.172505+0000","last_became_peered":"2024-03-06T07:25:55.172505+0000","last_unstale":"2024-03-06T07:25:56.164497+0000","last_undegraded":"2024-03-06T07:25:56.164497+0000","last_fullsized":"2024-03-06T07:25:56.164497+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:25.784809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406602+0000","last_change":"2024-03-06T07:25:54.177100+0000","last_active":"2024-03-06T07:25:56.406602+0000","last_peered":"2024-03-06T07:25:56.406602+0000","last_clean":"2024-03-06T07:25:56.406602+0000","last_became_active":"2024-03-06T07:25:54.176822+0000","last_became_peered":"2024-03-06T07:25:54.176822+0000","last_unstale":"2024-03-06T07:25:56.406602+0000","last_undegraded":"2024-03-06T07:25:56.406602+0000","last_fullsized":"2024-03-06T07:25:56.406602+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.349070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164733+0000","last_change":"2024-03-06T07:25:55.183106+0000","last_active":"2024-03-06T07:25:56.164733+0000","last_peered":"2024-03-06T07:25:56.164733+0000","last_clean":"2024-03-06T07:25:56.164733+0000","last_became_active":"2024-03-06T07:25:55.182843+0000","last_became_peered":"2024-03-06T07:25:55.182843+0000","last_unstale":"2024-03-06T07:25:56.164733+0000","last_undegraded":"2024-03-06T07:25:56.164733+0000","last_fullsized":"2024-03-06T07:25:56.164733+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:52.531694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167699+0000","last_change":"2024-03-06T07:25:54.172691+0000","last_active":"2024-03-06T07:25:57.167699+0000","last_peered":"2024-03-06T07:25:57.167699+0000","last_clean":"2024-03-06T07:25:57.167699+0000","last_became_active":"2024-03-06T07:25:54.172355+0000","last_became_peered":"2024-03-06T07:25:54.172355+0000","last_unstale":"2024-03-06T07:25:57.167699+0000","last_undegraded":"2024-03-06T07:25:57.167699+0000","last_fullsized":"2024-03-06T07:25:57.167699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.756681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162524+0000","last_change":"2024-03-06T07:25:55.179180+0000","last_active":"2024-03-06T07:25:56.162524+0000","last_peered":"2024-03-06T07:25:56.162524+0000","last_clean":"2024-03-06T07:25:56.162524+0000","last_became_active":"2024-03-06T07:25:55.178890+0000","last_became_peered":"2024-03-06T07:25:55.178890+0000","last_unstale":"2024-03-06T07:25:56.162524+0000","last_undegraded":"2024-03-06T07:25:56.162524+0000","last_fullsized":"2024-03-06T07:25:56.162524+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:00.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406499+0000","last_change":"2024-03-06T07:25:54.181928+0000","last_active":"2024-03-06T07:25:56.406499+0000","last_peered":"2024-03-06T07:25:56.406499+0000","last_clean":"2024-03-06T07:25:56.406499+0000","last_became_active":"2024-03-06T07:25:54.181140+0000","last_became_peered":"2024-03-06T07:25:54.181140+0000","last_unstale":"2024-03-06T07:25:56.406499+0000","last_undegraded":"2024-03-06T07:25:56.406499+0000","last_fullsized":"2024-03-06T07:25:56.406499+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:57.291913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406638+0000","last_change":"2024-03-06T07:25:55.192859+0000","last_active":"2024-03-06T07:25:56.406638+0000","last_peered":"2024-03-06T07:25:56.406638+0000","last_clean":"2024-03-06T07:25:56.406638+0000","last_became_active":"2024-03-06T07:25:55.192570+0000","last_became_peered":"2024-03-06T07:25:55.192570+0000","last_unstale":"2024-03-06T07:25:56.406638+0000","last_undegraded":"2024-03-06T07:25:56.406638+0000","last_fullsized":"2024-03-06T07:25:56.406638+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:00.192724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164517+0000","last_change":"2024-03-06T07:25:54.180637+0000","last_active":"2024-03-06T07:25:56.164517+0000","last_peered":"2024-03-06T07:25:56.164517+0000","last_clean":"2024-03-06T07:25:56.164517+0000","last_became_active":"2024-03-06T07:25:54.180270+0000","last_became_peered":"2024-03-06T07:25:54.180270+0000","last_unstale":"2024-03-06T07:25:56.164517+0000","last_undegraded":"2024-03-06T07:25:56.164517+0000","last_fullsized":"2024-03-06T07:25:56.164517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:20.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406333+0000","last_change":"2024-03-06T07:25:55.194175+0000","last_active":"2024-03-06T07:25:56.406333+0000","last_peered":"2024-03-06T07:25:56.406333+0000","last_clean":"2024-03-06T07:25:56.406333+0000","last_became_active":"2024-03-06T07:25:55.192548+0000","last_became_peered":"2024-03-06T07:25:55.192548+0000","last_unstale":"2024-03-06T07:25:56.406333+0000","last_undegraded":"2024-03-06T07:25:56.406333+0000","last_fullsized":"2024-03-06T07:25:56.406333+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:09.456474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.166384+0000","last_change":"2024-03-06T07:25:54.171781+0000","last_active":"2024-03-06T07:25:55.166384+0000","last_peered":"2024-03-06T07:25:55.166384+0000","last_clean":"2024-03-06T07:25:55.166384+0000","last_became_active":"2024-03-06T07:25:54.171466+0000","last_became_peered":"2024-03-06T07:25:54.171466+0000","last_unstale":"2024-03-06T07:25:55.166384+0000","last_undegraded":"2024-03-06T07:25:55.166384+0000","last_fullsized":"2024-03-06T07:25:55.166384+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:50.533388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977321+0000","last_change":"2024-03-06T07:25:55.173561+0000","last_active":"2024-03-06T07:25:56.977321+0000","last_peered":"2024-03-06T07:25:56.977321+0000","last_clean":"2024-03-06T07:25:56.977321+0000","last_became_active":"2024-03-06T07:25:55.173309+0000","last_became_peered":"2024-03-06T07:25:55.173309+0000","last_unstale":"2024-03-06T07:25:56.977321+0000","last_undegraded":"2024-03-06T07:25:56.977321+0000","last_fullsized":"2024-03-06T07:25:56.977321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977300+0000","last_change":"2024-03-06T07:25:54.178755+0000","last_active":"2024-03-06T07:25:56.977300+0000","last_peered":"2024-03-06T07:25:56.977300+0000","last_clean":"2024-03-06T07:25:56.977300+0000","last_became_active":"2024-03-06T07:25:54.178492+0000","last_became_peered":"2024-03-06T07:25:54.178492+0000","last_unstale":"2024-03-06T07:25:56.977300+0000","last_undegraded":"2024-03-06T07:25:56.977300+0000","last_fullsized":"2024-03-06T07:25:56.977300+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:18.439323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164157+0000","last_change":"2024-03-06T07:25:55.172471+0000","last_active":"2024-03-06T07:25:56.164157+0000","last_peered":"2024-03-06T07:25:56.164157+0000","last_clean":"2024-03-06T07:25:56.164157+0000","last_became_active":"2024-03-06T07:25:55.172189+0000","last_became_peered":"2024-03-06T07:25:55.172189+0000","last_unstale":"2024-03-06T07:25:56.164157+0000","last_undegraded":"2024-03-06T07:25:56.164157+0000","last_fullsized":"2024-03-06T07:25:56.164157+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:53.761906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406503+0000","last_change":"2024-03-06T07:25:54.171718+0000","last_active":"2024-03-06T07:25:56.406503+0000","last_peered":"2024-03-06T07:25:56.406503+0000","last_clean":"2024-03-06T07:25:56.406503+0000","last_became_active":"2024-03-06T07:25:54.171431+0000","last_became_peered":"2024-03-06T07:25:54.171431+0000","last_unstale":"2024-03-06T07:25:56.406503+0000","last_undegraded":"2024-03-06T07:25:56.406503+0000","last_fullsized":"2024-03-06T07:25:56.406503+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:51.464247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167565+0000","last_change":"2024-03-06T07:25:55.184864+0000","last_active":"2024-03-06T07:25:57.167565+0000","last_peered":"2024-03-06T07:25:57.167565+0000","last_clean":"2024-03-06T07:25:57.167565+0000","last_became_active":"2024-03-06T07:25:55.184462+0000","last_became_peered":"2024-03-06T07:25:55.184462+0000","last_unstale":"2024-03-06T07:25:57.167565+0000","last_undegraded":"2024-03-06T07:25:57.167565+0000","last_fullsized":"2024-03-06T07:25:57.167565+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"150'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167010+0000","last_change":"2024-03-06T07:25:54.184297+0000","last_active":"2024-03-06T07:25:57.167010+0000","last_peered":"2024-03-06T07:25:57.167010+0000","last_clean":"2024-03-06T07:25:57.167010+0000","last_became_active":"2024-03-06T07:25:54.184000+0000","last_became_peered":"2024-03-06T07:25:54.184000+0000","last_unstale":"2024-03-06T07:25:57.167010+0000","last_undegraded":"2024-03-06T07:25:57.167010+0000","last_fullsized":"2024-03-06T07:25:57.167010+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:06.525995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.178996+0000","last_change":"2024-03-06T07:25:55.173629+0000","last_active":"2024-03-06T07:25:55.178996+0000","last_peered":"2024-03-06T07:25:55.178996+0000","last_clean":"2024-03-06T07:25:55.178996+0000","last_became_active":"2024-03-06T07:25:55.173365+0000","last_became_peered":"2024-03-06T07:25:55.173365+0000","last_unstale":"2024-03-06T07:25:55.178996+0000","last_undegraded":"2024-03-06T07:25:55.178996+0000","last_fullsized":"2024-03-06T07:25:55.178996+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:29:54.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162613+0000","last_change":"2024-03-06T07:25:54.180119+0000","last_active":"2024-03-06T07:25:56.162613+0000","last_peered":"2024-03-06T07:25:56.162613+0000","last_clean":"2024-03-06T07:25:56.162613+0000","last_became_active":"2024-03-06T07:25:54.179853+0000","last_became_peered":"2024-03-06T07:25:54.179853+0000","last_unstale":"2024-03-06T07:25:56.162613+0000","last_undegraded":"2024-03-06T07:25:56.162613+0000","last_fullsized":"2024-03-06T07:25:56.162613+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:43.160051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:57.167659+0000","last_change":"2024-03-06T07:25:55.184654+0000","last_active":"2024-03-06T07:25:57.167659+0000","last_peered":"2024-03-06T07:25:57.167659+0000","last_clean":"2024-03-06T07:25:57.167659+0000","last_became_active":"2024-03-06T07:25:55.184305+0000","last_became_peered":"2024-03-06T07:25:55.184305+0000","last_unstale":"2024-03-06T07:25:57.167659+0000","last_undegraded":"2024-03-06T07:25:57.167659+0000","last_fullsized":"2024-03-06T07:25:57.167659+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.167631+0000","last_change":"2024-03-06T07:25:54.174742+0000","last_active":"2024-03-06T07:25:55.167631+0000","last_peered":"2024-03-06T07:25:55.167631+0000","last_clean":"2024-03-06T07:25:55.167631+0000","last_became_active":"2024-03-06T07:25:54.174462+0000","last_became_peered":"2024-03-06T07:25:54.174462+0000","last_unstale":"2024-03-06T07:25:55.167631+0000","last_undegraded":"2024-03-06T07:25:55.167631+0000","last_fullsized":"2024-03-06T07:25:55.167631+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:47:43.050877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491525+0000","last_change":"2024-03-06T07:25:55.181956+0000","last_active":"2024-03-06T07:25:56.491525+0000","last_peered":"2024-03-06T07:25:56.491525+0000","last_clean":"2024-03-06T07:25:56.491525+0000","last_became_active":"2024-03-06T07:25:55.181658+0000","last_became_peered":"2024-03-06T07:25:55.181658+0000","last_unstale":"2024-03-06T07:25:56.491525+0000","last_undegraded":"2024-03-06T07:25:56.491525+0000","last_fullsized":"2024-03-06T07:25:56.491525+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:23.152358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.491294+0000","last_change":"2024-03-06T07:25:54.172421+0000","last_active":"2024-03-06T07:25:56.491294+0000","last_peered":"2024-03-06T07:25:56.491294+0000","last_clean":"2024-03-06T07:25:56.491294+0000","last_became_active":"2024-03-06T07:25:54.172157+0000","last_became_peered":"2024-03-06T07:25:54.172157+0000","last_unstale":"2024-03-06T07:25:56.491294+0000","last_undegraded":"2024-03-06T07:25:56.491294+0000","last_fullsized":"2024-03-06T07:25:56.491294+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:07.283064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977285+0000","last_change":"2024-03-06T07:25:55.183461+0000","last_active":"2024-03-06T07:25:56.977285+0000","last_peered":"2024-03-06T07:25:56.977285+0000","last_clean":"2024-03-06T07:25:56.977285+0000","last_became_active":"2024-03-06T07:25:55.183161+0000","last_became_peered":"2024-03-06T07:25:55.183161+0000","last_unstale":"2024-03-06T07:25:56.977285+0000","last_undegraded":"2024-03-06T07:25:56.977285+0000","last_fullsized":"2024-03-06T07:25:56.977285+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162403+0000","last_change":"2024-03-06T07:25:54.182221+0000","last_active":"2024-03-06T07:25:56.162403+0000","last_peered":"2024-03-06T07:25:56.162403+0000","last_clean":"2024-03-06T07:25:56.162403+0000","last_became_active":"2024-03-06T07:25:54.181966+0000","last_became_peered":"2024-03-06T07:25:54.181966+0000","last_unstale":"2024-03-06T07:25:56.162403+0000","last_undegraded":"2024-03-06T07:25:56.162403+0000","last_fullsized":"2024-03-06T07:25:56.162403+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:48.357604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162690+0000","last_change":"2024-03-06T07:25:55.178896+0000","last_active":"2024-03-06T07:25:56.162690+0000","last_peered":"2024-03-06T07:25:56.162690+0000","last_clean":"2024-03-06T07:25:56.162690+0000","last_became_active":"2024-03-06T07:25:55.178599+0000","last_became_peered":"2024-03-06T07:25:55.178599+0000","last_unstale":"2024-03-06T07:25:56.162690+0000","last_undegraded":"2024-03-06T07:25:56.162690+0000","last_fullsized":"2024-03-06T07:25:56.162690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:34.372833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.164400+0000","last_change":"2024-03-06T07:25:54.176751+0000","last_active":"2024-03-06T07:25:56.164400+0000","last_peered":"2024-03-06T07:25:56.164400+0000","last_clean":"2024-03-06T07:25:56.164400+0000","last_became_active":"2024-03-06T07:25:54.176461+0000","last_became_peered":"2024-03-06T07:25:54.176461+0000","last_unstale":"2024-03-06T07:25:56.164400+0000","last_undegraded":"2024-03-06T07:25:56.164400+0000","last_fullsized":"2024-03-06T07:25:56.164400+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:09.865297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-03-06T07:25:55.187012+0000","last_change":"2024-03-06T07:25:55.184289+0000","last_active":"2024-03-06T07:25:55.187012+0000","last_peered":"2024-03-06T07:25:55.187012+0000","last_clean":"2024-03-06T07:25:55.187012+0000","last_became_active":"2024-03-06T07:25:55.184007+0000","last_became_peered":"2024-03-06T07:25:55.184007+0000","last_unstale":"2024-03-06T07:25:55.187012+0000","last_undegraded":"2024-03-06T07:25:55.187012+0000","last_fullsized":"2024-03-06T07:25:55.187012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:59:14.143583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.406080+0000","last_change":"2024-03-06T07:25:54.182027+0000","last_active":"2024-03-06T07:25:56.406080+0000","last_peered":"2024-03-06T07:25:56.406080+0000","last_clean":"2024-03-06T07:25:56.406080+0000","last_became_active":"2024-03-06T07:25:54.181755+0000","last_became_peered":"2024-03-06T07:25:54.181755+0000","last_unstale":"2024-03-06T07:25:56.406080+0000","last_undegraded":"2024-03-06T07:25:56.406080+0000","last_fullsized":"2024-03-06T07:25:56.406080+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:03.374674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.976862+0000","last_change":"2024-03-06T07:25:55.176555+0000","last_active":"2024-03-06T07:25:56.976862+0000","last_peered":"2024-03-06T07:25:56.976862+0000","last_clean":"2024-03-06T07:25:56.976862+0000","last_became_active":"2024-03-06T07:25:55.176295+0000","last_became_peered":"2024-03-06T07:25:55.176295+0000","last_unstale":"2024-03-06T07:25:56.976862+0000","last_undegraded":"2024-03-06T07:25:56.976862+0000","last_fullsized":"2024-03-06T07:25:56.976862+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:18.514838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.162641+0000","last_change":"2024-03-06T07:25:54.184373+0000","last_active":"2024-03-06T07:25:56.162641+0000","last_peered":"2024-03-06T07:25:56.162641+0000","last_clean":"2024-03-06T07:25:56.162641+0000","last_became_active":"2024-03-06T07:25:54.184109+0000","last_became_peered":"2024-03-06T07:25:54.184109+0000","last_unstale":"2024-03-06T07:25:56.162641+0000","last_undegraded":"2024-03-06T07:25:56.162641+0000","last_fullsized":"2024-03-06T07:25:56.162641+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:01.955245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":409,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-03-06T07:25:56.977257+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:25:56.977257+0000","last_peered":"2024-03-06T07:25:56.977257+0000","last_clean":"2024-03-06T07:25:56.977257+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:25:56.977257+0000","last_undegraded":"2024-03-06T07:25:56.977257+0000","last_fullsized":"2024-03-06T07:25:56.977257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1953792,"data_stored":1504557,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33032,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338808,"statfs":{"total":96636764160,"available":96602939392,"internally_reserved":0,"allocated":1888256,"data_stored":914189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1896448,"data_stored":914313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1904640,"data_stored":914953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1908736,"data_stored":916405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1875968,"data_stored":907497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1900544,"data_stored":915801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":1504661,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:25:58.724 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:25:58.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:25:58.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.866+0000 7ff65fb02640 1 Processor -- start 2024-03-06T07:25:58.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 -- start start 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff658155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6580c5380 0x7ff6580c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6580b0450 con 0x7ff65806ca70 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65806c020 con 0x7ff6580c5380 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65fb02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580c77f0 con 0x7ff6580b0070 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65d09d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff65806ca70 0x7ff6580c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52264/0 (socket says 172.21.15.17:52264) 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.867+0000 7ff65d09d640 1 -- 172.21.15.17:0/2750894458 learned_addr learned my addr 172.21.15.17:0/2750894458 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65d89e640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff658155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65e09f640 1 --2- 172.21.15.17:0/2750894458 >> v2:172.21.15.163:3300/0 conn(0x7ff6580c5380 0x7ff6580c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015373962 0 0) 0x7ff648001110 con 0x7ff65806ca70 2024-03-06T07:25:58.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff644003520 con 0x7ff65806ca70 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893203758 0 0) 0x7ff644003520 con 0x7ff65806ca70 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 msgr2=0x7ff658155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff658155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 >> v2:172.21.15.163:3300/0 conn(0x7ff6580c5380 msgr2=0x7ff6580c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 --2- 172.21.15.17:0/2750894458 >> v2:172.21.15.163:3300/0 conn(0x7ff6580c5380 0x7ff6580c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580c7ad0 con 0x7ff65806ca70 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff648001840 con 0x7ff65806ca70 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 >> v1:172.21.15.17:6789/0 conn(0x7ff65806ca70 legacy=0x7ff6580c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff644004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff644004d40 0x7ff6440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806c020 con 0x7ff6440039b0 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580c77f0 con 0x7ff644004d40 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64400b9c0 con 0x7ff6580b0070 2024-03-06T07:25:58.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65d89e640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65d09d640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff644004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.868+0000 7ff65e09f640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff644004d40 0x7ff6440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 -- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 msgr2=0x7ff644004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580b0070 0x7ff644004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 -- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff644004d40 msgr2=0x7ff6440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff644004d40 0x7ff6440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 -- 172.21.15.17:0/2750894458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644003520 con 0x7ff6440039b0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65d89e640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ac0 tx=0x7ff644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dff4b2caf6bd19 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650039070 con 0x7ff6440039b0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65002c6b0 con 0x7ff6440039b0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.869+0000 7ff65c89c640 1 -- 172.21.15.17:0/2750894458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650009690 con 0x7ff6440039b0 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 -- 172.21.15.17:0/2750894458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6440039b0 msgr2=0x7ff644003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 --2- 172.21.15.17:0/2750894458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6440039b0 0x7ff644003dd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ac0 tx=0x7ff644005830 comp rx=0 tx=0).stop 2024-03-06T07:25:58.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65e09f640 1 -- 172.21.15.17:0/2750894458 reap_dead start 2024-03-06T07:25:58.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 -- 172.21.15.17:0/2750894458 shutdown_connections 2024-03-06T07:25:58.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 -- 172.21.15.17:0/2750894458 >> 172.21.15.17:0/2750894458 conn(0x7ff6580ac590 msgr2=0x7ff6580ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:58.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 -- 172.21.15.17:0/2750894458 shutdown_connections 2024-03-06T07:25:58.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.870+0000 7ff65fb02640 1 -- 172.21.15.17:0/2750894458 wait complete. 2024-03-06T07:25:58.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.872+0000 7ff65fb02640 1 Processor -- start 2024-03-06T07:25:58.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.872+0000 7ff65fb02640 1 -- start start 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6440039b0 0x7ff6581ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 0x7ff658159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806cef0 con 0x7ff658157f30 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806c550 con 0x7ff6440039b0 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65fb02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806da80 con 0x7ff6580c4850 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57242/0 (socket says 172.21.15.17:57242) 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 -- 172.21.15.17:0/3286346467 learned_addr learned my addr 172.21.15.17:0/3286346467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65d09d640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 0x7ff658159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65d89e640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6440039b0 0x7ff6581ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 -- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 msgr2=0x7ff658159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 0x7ff658159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 -- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6440039b0 msgr2=0x7ff6581ee520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6440039b0 0x7ff6581ee520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65e09f640 1 -- 172.21.15.17:0/3286346467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580c1d20 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.873+0000 7ff65d09d640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 0x7ff658159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff65e09f640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff6580aa4b0 tx=0x7ff65400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400fdb0 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654015430 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6580c2000 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654015820 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.874+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6580c25c0 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.875+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65804f330 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.875+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6540159c0 con 0x7ff658157f30 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.875+0000 7ff64e7fc640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 0x7ff63c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:58.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.876+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff65408b0f0 con 0x7ff658157f30 2024-03-06T07:25:58.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.876+0000 7ff65d89e640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 0x7ff63c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:58.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.876+0000 7ff65d89e640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 0x7ff63c06d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff650036000 tx=0x7ff650030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:58.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:58.884+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff654019030 con 0x7ff658157f30 2024-03-06T07:25:59.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.131+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff65806d860 con 0x7ff658157f30 2024-03-06T07:25:59.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.133+0000 7ff64e7fc640 1 -- 172.21.15.17:0/3286346467 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff654010040 con 0x7ff658157f30 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:25:59.135 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 msgr2=0x7ff63c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 0x7ff63c06d9c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff650036000 tx=0x7ff650030000 comp rx=0 tx=0).stop 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 msgr2=0x7ff6580c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff6580aa4b0 tx=0x7ff65400f580 comp rx=0 tx=0).stop 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 shutdown_connections 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff63c06b500 0x7ff63c06d9c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff658157f30 0x7ff6580c2b00 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6580c4850 0x7ff658159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 --2- 172.21.15.17:0/3286346467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6440039b0 0x7ff6581ee520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 >> 172.21.15.17:0/3286346467 conn(0x7ff6580ac590 msgr2=0x7ff6580ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 shutdown_connections 2024-03-06T07:25:59.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.135+0000 7ff65fb02640 1 -- 172.21.15.17:0/3286346467 wait complete. 2024-03-06T07:25:59.147 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:25:59.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.250+0000 7f77b0dc9640 1 Processor -- start 2024-03-06T07:25:59.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.250+0000 7f77b0dc9640 1 -- start start 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77b0dc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77ac156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77b0dc9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac070aa0 con 0x7f77ac0acc00 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77b0dc9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac070d80 con 0x7f77ac1506b0 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77b0dc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac153e10 con 0x7f77ac153990 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77aad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f77ac0acc00 0x7f77ac0c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52278/0 (socket says 172.21.15.17:52278) 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77aad9d640 1 -- 172.21.15.17:0/2987734877 learned_addr learned my addr 172.21.15.17:0/2987734877 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134477548 0 0) 0x7f77ac070aa0 con 0x7f77ac0acc00 2024-03-06T07:25:59.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7798003520 con 0x7f77ac0acc00 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.251+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77ac156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809256359 0 0) 0x7f7798003520 con 0x7f77ac0acc00 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 msgr2=0x7f77ac156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77ac156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 >> v1:172.21.15.163:6789/0 conn(0x7f77ac1506b0 legacy=0x7f77ac0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac1566a0 con 0x7f77ac0acc00 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f77a40019f0 con 0x7f77ac0acc00 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 >> v1:172.21.15.17:6789/0 conn(0x7f77ac0acc00 legacy=0x7f77ac0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f7798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7798004d40 0x7f77980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac153e10 con 0x7f7798004010 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779800bb80 con 0x7f7798004d40 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779800bd80 con 0x7f77ac153990 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f7798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77aa59c640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77aad9d640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7798004d40 0x7f77980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 -- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 msgr2=0x7f77980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac153990 0x7f77980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 -- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7798004d40 msgr2=0x7f77980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7798004d40 0x7f77980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77a1d9b640 1 -- 172.21.15.17:0/2987734877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798003520 con 0x7f7798004010 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.252+0000 7f77aad9d640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7798004d40 0x7f77980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.253+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f7798004800 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f779400ba30 tx=0x7f7798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c098cde95d8d1665 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.253+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794036020 con 0x7f7798004010 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.253+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794032640 con 0x7f7798004010 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.253+0000 7f77a9d9b640 1 -- 172.21.15.17:0/2987734877 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400b600 con 0x7f7798004010 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2987734877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 msgr2=0x7f7798004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2987734877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f7798004800 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f779400ba30 tx=0x7f7798005830 comp rx=0 tx=0).stop 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77aad9d640 1 -- 172.21.15.17:0/2987734877 reap_dead start 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2987734877 shutdown_connections 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2987734877 >> 172.21.15.17:0/2987734877 conn(0x7f77ac06b1a0 msgr2=0x7f77ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2987734877 shutdown_connections 2024-03-06T07:25:59.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.254+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2987734877 wait complete. 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.255+0000 7f77b0dc9640 1 Processor -- start 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.255+0000 7f77b0dc9640 1 -- start start 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77ac1dfd20 0x7f77ac1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 0x7f77ac1ec450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac156a10 con 0x7f7798004010 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac156450 con 0x7f77ac1dfd20 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77b0dc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0ad070 con 0x7f77ac149ef0 2024-03-06T07:25:59.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57260/0 (socket says 172.21.15.17:57260) 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 -- 172.21.15.17:0/2222308715 learned_addr learned my addr 172.21.15.17:0/2222308715 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77ac1dfd20 0x7f77ac1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aad9d640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 0x7f77ac1ec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 -- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 msgr2=0x7f77ac1ec450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 0x7f77ac1ec450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 -- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77ac1dfd20 msgr2=0x7f77ac1499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77ac1dfd20 0x7f77ac1499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 -- 172.21.15.17:0/2222308715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0c82d0 con 0x7f7798004010 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aad9d640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 0x7f77ac1ec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77aa59c640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f779c00f880 tx=0x7f779c00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.256+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c001d20 con 0x7f7798004010 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.257+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c018070 con 0x7f7798004010 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.257+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00d730 con 0x7f7798004010 2024-03-06T07:25:59.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.257+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ac14a370 con 0x7f7798004010 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.257+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac0aad00 con 0x7f7798004010 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.258+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac04f330 con 0x7f7798004010 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.258+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f779c00d8d0 con 0x7f7798004010 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.259+0000 7f77a37fe640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 0x7f778c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.259+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f779c08a040 con 0x7f7798004010 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.259+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 0x7f778c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:25:59.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.259+0000 7f77a1d9b640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 0x7f778c06d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f779400ba00 tx=0x7f779403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:25:59.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.266+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f779c058dc0 con 0x7f7798004010 2024-03-06T07:25:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.541+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77ac0ab580 con 0x7f7798004010 2024-03-06T07:25:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.541+0000 7f77a37fe640 1 -- 172.21.15.17:0/2222308715 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+4981 (secure 0 0 0) 0x7f779c0586c0 con 0x7f7798004010 2024-03-06T07:25:59.543 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:25:59.543 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":136,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/81289984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":81289984},{"type":"v1","addr":"172.21.15.17:6834","nonce":81289984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:25:56.622881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8781":{"gid":8781,"name":"c","rank":0,"incarnation":134,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2429596225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2429596225},{"type":"v1","addr":"172.21.15.17:6835","nonce":2429596225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8781,"qdb_cluster":[8781]},"id":11}]} 2024-03-06T07:25:59.543 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 136 2024-03-06T07:25:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 msgr2=0x7f778c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 0x7f778c06d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f779400ba00 tx=0x7f779403d040 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 msgr2=0x7f77ac0c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f779c00f880 tx=0x7f779c00fd40 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 shutdown_connections 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f778c06b500 0x7f778c06d9c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77ac149ef0 0x7f77ac1ec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77ac1dfd20 0x7f77ac1499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 --2- 172.21.15.17:0/2222308715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7798004010 0x7f77ac0c7910 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 >> 172.21.15.17:0/2222308715 conn(0x7f77ac06b1a0 msgr2=0x7f77ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 shutdown_connections 2024-03-06T07:25:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:25:59.544+0000 7f77b0dc9640 1 -- 172.21.15.17:0/2222308715 wait complete. 2024-03-06T07:25:59.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 136, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:54.526748+0000', 'modified': '2024-03-06T07:25:56.622881+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8781}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8781': {'gid': 8781, 'name': 'c', 'rank': 0, 'incarnation': 134, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2429596225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2429596225}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2429596225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8781, 'qdb_cluster': [8781]} 2024-03-06T07:25:59.959 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:25:59.959 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:26:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.059+0000 7f85f94ed640 1 Processor -- start 2024-03-06T07:26:00.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.059+0000 7f85f94ed640 1 -- start start 2024-03-06T07:26:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85f41526d0 0x7f85f406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85f40c35b0 0x7f85f40c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f4155110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f406b960 con 0x7f85f41526d0 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f406feb0 con 0x7f85f40c35b0 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f94ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406f0f0 con 0x7f85f40c25f0 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85f41526d0 0x7f85f406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f85f41526d0 0x7f85f406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57262/0 (socket says 172.21.15.17:57262) 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.060+0000 7f85f2ffd640 1 -- 172.21.15.17:0/2370838505 learned_addr learned my addr 172.21.15.17:0/2370838505 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 -- 172.21.15.17:0/2370838505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 msgr2=0x7f85f4155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2370838505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f4155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 -- 172.21.15.17:0/2370838505 >> v2:172.21.15.163:3300/0 conn(0x7f85f40c35b0 msgr2=0x7f85f40c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2370838505 >> v2:172.21.15.163:3300/0 conn(0x7f85f40c35b0 0x7f85f40c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 -- 172.21.15.17:0/2370838505 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f406f3d0 con 0x7f85f41526d0 2024-03-06T07:26:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2370838505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f41526d0 0x7f85f406afc0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85f40c5050 tx=0x7f85e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a2ede74ad748d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.061+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2370838505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e402e070 con 0x7f85f41526d0 2024-03-06T07:26:00.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2370838505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4037070 con 0x7f85f41526d0 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f1ffb640 1 -- 172.21.15.17:0/2370838505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4031a00 con 0x7f85f41526d0 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 -- 172.21.15.17:0/2370838505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f41526d0 msgr2=0x7f85f406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 --2- 172.21.15.17:0/2370838505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f41526d0 0x7f85f406afc0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f85f40c5050 tx=0x7f85e402b880 comp rx=0 tx=0).stop 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 -- 172.21.15.17:0/2370838505 shutdown_connections 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 --2- 172.21.15.17:0/2370838505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f4155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 --2- 172.21.15.17:0/2370838505 >> v2:172.21.15.163:3300/0 conn(0x7f85f40c35b0 0x7f85f40c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 --2- 172.21.15.17:0/2370838505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f41526d0 0x7f85f406afc0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 -- 172.21.15.17:0/2370838505 >> 172.21.15.17:0/2370838505 conn(0x7f85f40c41b0 msgr2=0x7f85f40c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 -- 172.21.15.17:0/2370838505 shutdown_connections 2024-03-06T07:26:00.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.062+0000 7f85f94ed640 1 -- 172.21.15.17:0/2370838505 wait complete. 2024-03-06T07:26:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.064+0000 7f85f94ed640 1 Processor -- start 2024-03-06T07:26:00.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.064+0000 7f85f94ed640 1 -- start start 2024-03-06T07:26:00.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f41ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 0x7f85f414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4152ab0 con 0x7f85f40c35b0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406f990 con 0x7f85f41526d0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f94ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406b500 con 0x7f85f40c25f0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57276/0 (socket says 172.21.15.17:57276) 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.065+0000 7f85f27fc640 1 -- 172.21.15.17:0/233723677 learned_addr learned my addr 172.21.15.17:0/233723677 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f41ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f37fe640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 0x7f85f414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 -- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 msgr2=0x7f85f41ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f41ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 -- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 msgr2=0x7f85f414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 0x7f85f414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 -- 172.21.15.17:0/233723677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4153120 con 0x7f85f40c35b0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f41ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f37fe640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 0x7f85f414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f27fc640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b410 tx=0x7f85e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85db7fe640 1 -- 172.21.15.17:0/233723677 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8015020 con 0x7f85f40c35b0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85db7fe640 1 -- 172.21.15.17:0/233723677 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e8011850 con 0x7f85f40c35b0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85db7fe640 1 -- 172.21.15.17:0/233723677 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8011c40 con 0x7f85f40c35b0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f4153400 con 0x7f85f40c35b0 2024-03-06T07:26:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.066+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f414beb0 con 0x7f85f40c35b0 2024-03-06T07:26:00.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.068+0000 7f85db7fe640 1 -- 172.21.15.17:0/233723677 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85e8018020 con 0x7f85f40c35b0 2024-03-06T07:26:00.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.068+0000 7f85db7fe640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.068+0000 7f85db7fe640 1 -- 172.21.15.17:0/233723677 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85e808a200 con 0x7f85f40c35b0 2024-03-06T07:26:00.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.068+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.069+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 0x7f85e006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036000 tx=0x7f85e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.070+0000 7f85fc1b1740 1 Processor -- start 2024-03-06T07:26:00.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.070+0000 7f85fc1b1740 1 -- start start 2024-03-06T07:26:00.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf588cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588d4b0 0x556bf5889680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf57f36b0 con 0x556bf588ef50 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf579ca60 con 0x556bf588f330 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bf57ef750 con 0x556bf588d4b0 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57278/0 (socket says 172.21.15.17:57278) 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85f2ffd640 1 -- 172.21.15.17:0/942187379 learned_addr learned my addr 172.21.15.17:0/942187379 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85f27fc640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf588cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.071+0000 7f85f37fe640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588d4b0 0x556bf5889680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 -- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588d4b0 msgr2=0x556bf5889680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588d4b0 0x556bf5889680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 -- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 msgr2=0x556bf588cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf588cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 -- 172.21.15.17:0/942187379 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bf57baf30 con 0x556bf588ef50 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f27fc640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf588cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:00.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f85e4034040 tx=0x7f85e4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccfb46b7d608c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85d8ff9640 1 -- 172.21.15.17:0/942187379 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4032c00 con 0x556bf588ef50 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85d8ff9640 1 -- 172.21.15.17:0/942187379 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4037040 con 0x556bf588ef50 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.072+0000 7f85d8ff9640 1 -- 172.21.15.17:0/942187379 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4051480 con 0x556bf588ef50 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- 172.21.15.17:0/942187379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 msgr2=0x556bf588ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f85e4034040 tx=0x7f85e4030000 comp rx=0 tx=0).stop 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- 172.21.15.17:0/942187379 shutdown_connections 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588d4b0 0x556bf5889680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf588cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/942187379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf588ca30 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- 172.21.15.17:0/942187379 >> 172.21.15.17:0/942187379 conn(0x556bf588b7d0 msgr2=0x556bf58b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- 172.21.15.17:0/942187379 shutdown_connections 2024-03-06T07:26:00.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- 172.21.15.17:0/942187379 wait complete. 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 -- start start 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.073+0000 7f85fc1b1740 1 Processor -- start 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf58b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf58b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf58b1230 con 0x556bf588ef50 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf57afea0 con 0x556bf588f330 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bf58b0c70 con 0x556bf58b36b0 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85fc1b1740 10 client.0 _ll_register_callbacks cb 0x556bf57af870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57282/0 (socket says 172.21.15.17:57282) 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 -- 172.21.15.17:0/1692542733 learned_addr learned my addr 172.21.15.17:0/1692542733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f37fe640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf58b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f27fc640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf58b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 -- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 msgr2=0x556bf58b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf58b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 -- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 msgr2=0x556bf58b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf58b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.074+0000 7f85f2ffd640 1 -- 172.21.15.17:0/1692542733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bf57b0270 con 0x556bf588ef50 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85f37fe640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf58b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f85e402be90 tx=0x7f85e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57682c2892b6dd76 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e403bc30 con 0x556bf588ef50 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85e403bdd0 con 0x556bf588ef50 2024-03-06T07:26:00.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85e4037040 con 0x556bf588ef50 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e4063050 con 0x556bf588ef50 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 10 client.8931 fetch_fsmap learned FSMap version 136 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x556bf55082d0 con 0x556bf588ef50 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4037040 con 0x556bf588ef50 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 136) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f85e4066020 con 0x556bf588ef50 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 10 client.8931 fetch_fsmap finished waiting for FSMap version 136 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 10 client.8931 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 20 client.8931 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58df470 0x556bf58df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 4 client.8931 mds_command: new command op to 8781 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 --> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bf5800bf0 con 0x556bf58df470 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.075+0000 7f85f27fc640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58df470 0x556bf58df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.076+0000 7f85d1ffb640 10 client.8931 ms_handle_connect on v2:172.21.15.17:6833/2429596225 2024-03-06T07:26:00.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.076+0000 7f85f27fc640 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58df470 0x556bf58df870 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531c8dd7a2d7a2a4 server_cookie=7d706b8949883b29 in_seq=0 out_seq=0 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.076+0000 7f85d1ffb640 1 -- 172.21.15.17:0/1692542733 <== mds.0 v2:172.21.15.17:6833/2429596225 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556bf5800bf0 con 0x556bf58df470 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.076+0000 7f85d1ffb640 10 client.8931 handle_command_reply: tid=0 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.076+0000 7f85fc1b1740 1 client.8931 shutdown 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 msgr2=0x556bf58af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f85e402be90 tx=0x7f85e4004730 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 shutdown_connections 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58df470 0x556bf58df870 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf58b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85d1ffb640 0 client.8931 ms_handle_reset on v2:172.21.15.17:6833/2429596225 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588f330 0x556bf58b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/1692542733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588ef50 0x556bf58af680 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 >> 172.21.15.17:0/1692542733 conn(0x556bf588d4b0 msgr2=0x556bf58cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 shutdown_connections 2024-03-06T07:26:00.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 1 -- 172.21.15.17:0/1692542733 wait complete. 2024-03-06T07:26:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.077+0000 7f85fc1b1740 20 client.8931 trim_cache size 0 max 16384 2024-03-06T07:26:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.109+0000 7f85fc1b1740 1 Processor -- start 2024-03-06T07:26:00.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.109+0000 7f85fc1b1740 1 -- start start 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.110+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 0x556bf58b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f41f7080 con 0x556bf58b36b0 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f40c72d0 con 0x556bf588ef50 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f414fa10 con 0x556bf588f330 2024-03-06T07:26:00.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57290/0 (socket says 172.21.15.17:57290) 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 -- 172.21.15.17:0/2881325581 learned_addr learned my addr 172.21.15.17:0/2881325581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f27fc640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 0x556bf58b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 -- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 msgr2=0x556bf58b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 0x556bf58b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 -- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 msgr2=0x556bf58e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 -- 172.21.15.17:0/2881325581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bf55d26b0 con 0x556bf58b36b0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f27fc640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 0x556bf58b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85f37fe640 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0103f0 tx=0x7f85ec0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd2e9688268b365 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85d3fff640 1 -- 172.21.15.17:0/2881325581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011900 con 0x556bf58b36b0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85d3fff640 1 -- 172.21.15.17:0/2881325581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec015070 con 0x556bf58b36b0 2024-03-06T07:26:00.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.111+0000 7f85d3fff640 1 -- 172.21.15.17:0/2881325581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011cc0 con 0x556bf58b36b0 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- 172.21.15.17:0/2881325581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 msgr2=0x556bf58b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0103f0 tx=0x7f85ec0108b0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- 172.21.15.17:0/2881325581 shutdown_connections 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf58b36b0 0x556bf58b2c60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf588f330 0x556bf58b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/2881325581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- 172.21.15.17:0/2881325581 >> 172.21.15.17:0/2881325581 conn(0x556bf588d4b0 msgr2=0x556bf58ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- 172.21.15.17:0/2881325581 shutdown_connections 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- 172.21.15.17:0/2881325581 wait complete. 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 -- start start 2024-03-06T07:26:00.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.112+0000 7f85fc1b1740 1 Processor -- start 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf5937630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf58b1230 con 0x556bf588f330 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bf57afea0 con 0x556bf588ef50 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bf58c7ec0 con 0x556bf58b36b0 2024-03-06T07:26:00.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85fc1b1740 10 client.0 _ll_register_callbacks cb 0x7f85e4032de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57294/0 (socket says 172.21.15.17:57294) 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85f27fc640 1 -- 172.21.15.17:0/3058538754 learned_addr learned my addr 172.21.15.17:0/3058538754 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85f37fe640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf5937630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.113+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 -- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 msgr2=0x556bf5937630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf5937630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 -- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 msgr2=0x556bf58b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 -- 172.21.15.17:0/3058538754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bf58b5a60 con 0x556bf588f330 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85f27fc640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f85e8058fa0 tx=0x7f85e808a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644cca2b49a8208e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8056b80 con 0x556bf588f330 2024-03-06T07:26:00.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85e8088220 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85ec015070 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=136) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85e8054ef0 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 10 client.8937 fetch_fsmap learned FSMap version 136 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=136,monmap=2+}) v3 -- 0x556bf58d7bc0 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec015070 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 136) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f85e8059bb0 con 0x556bf588f330 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 10 client.8937 fetch_fsmap finished waiting for FSMap version 136 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 10 client.8937 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 20 client.8937 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58893e0 0x556bf58897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 4 client.8937 mds_command: new command op to 8781 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.114+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 --> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556bf57faf40 con 0x556bf58893e0 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58893e0 0x556bf58897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85d1ffb640 10 client.8937 ms_handle_connect on v2:172.21.15.17:6833/2429596225 2024-03-06T07:26:00.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85f2ffd640 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58893e0 0x556bf58897c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1212a254c276081 server_cookie=1c698a72385afb65 in_seq=0 out_seq=0 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85d1ffb640 1 -- 172.21.15.17:0/3058538754 <== mds.0 v2:172.21.15.17:6833/2429596225 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x556bf57faf40 con 0x556bf58893e0 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85d1ffb640 10 client.8937 handle_command_reply: tid=0 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 client.8937 shutdown 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 msgr2=0x556bf58d80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f85e8058fa0 tx=0x7f85e808a040 comp rx=0 tx=0).stop 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 shutdown_connections 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:6833/2429596225,v1:172.21.15.17:6835/2429596225] conn(0x556bf58893e0 0x556bf58897c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x556bf58b36b0 0x556bf5937630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556bf588f330 0x556bf58d80e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85d1ffb640 0 client.8937 ms_handle_reset on v2:172.21.15.17:6833/2429596225 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 --2- 172.21.15.17:0/3058538754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x556bf588ef50 0x556bf58b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 >> 172.21.15.17:0/3058538754 conn(0x556bf588d4b0 msgr2=0x556bf5888510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 shutdown_connections 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 1 -- 172.21.15.17:0/3058538754 wait complete. 2024-03-06T07:26:00.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.115+0000 7f85fc1b1740 20 client.8937 trim_cache size 0 max 16384 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8781, 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.9561571449999997, 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 136, 2024-03-06T07:26:00.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 150, 2024-03-06T07:26:00.125 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 150, 2024-03-06T07:26:00.125 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10.043152686000001 2024-03-06T07:26:00.125 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:26:00.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 msgr2=0x7f85e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 0x7f85e006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85e4036000 tx=0x7f85e4008030 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 msgr2=0x7f85f41ebcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85e800b410 tx=0x7f85e800b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 shutdown_connections 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85f41526d0 0x7f85f414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85f40c35b0 0x7f85f41ebcc0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 --2- 172.21.15.17:0/233723677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85f40c25f0 0x7f85f41ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 >> 172.21.15.17:0/233723677 conn(0x7f85f40c41b0 msgr2=0x7f85f4073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 shutdown_connections 2024-03-06T07:26:00.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.116+0000 7f85f94ed640 1 -- 172.21.15.17:0/233723677 wait complete. 2024-03-06T07:26:00.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:00.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 Processor -- start 2024-03-06T07:26:00.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 -- start start 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9040158db0 0x7f90400c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c88b0 0x7f904015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9040155110 con 0x7f90400c5330 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90401559e0 con 0x7f9040158db0 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f9046702640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90400c8f40 con 0x7f90400c88b0 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f903f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9040158db0 0x7f90400c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.229+0000 7f903f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9040158db0 0x7f90400c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57304/0 (socket says 172.21.15.17:57304) 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 -- 172.21.15.17:0/916527372 learned_addr learned my addr 172.21.15.17:0/916527372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f9044c9f640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c88b0 0x7f904015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 -- 172.21.15.17:0/916527372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c88b0 msgr2=0x7f904015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c88b0 0x7f904015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 -- 172.21.15.17:0/916527372 >> v1:172.21.15.163:6789/0 conn(0x7f90400c5330 legacy=0x7f90401586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 -- 172.21.15.17:0/916527372 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040155390 con 0x7f9040158db0 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.230+0000 7f903f7fe640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9040158db0 0x7f90400c5710 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9028009870 tx=0x7f902802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9ac79796e45243 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f903effd640 1 -- 172.21.15.17:0/916527372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028035020 con 0x7f9040158db0 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f903effd640 1 -- 172.21.15.17:0/916527372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9028031640 con 0x7f9040158db0 2024-03-06T07:26:00.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f903effd640 1 -- 172.21.15.17:0/916527372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902802d040 con 0x7f9040158db0 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 -- 172.21.15.17:0/916527372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9040158db0 msgr2=0x7f90400c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9040158db0 0x7f90400c5710 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f9028009870 tx=0x7f902802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 -- 172.21.15.17:0/916527372 shutdown_connections 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c88b0 0x7f904015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 --2- 172.21.15.17:0/916527372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9040158db0 0x7f90400c5710 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 -- 172.21.15.17:0/916527372 >> 172.21.15.17:0/916527372 conn(0x7f90400ac560 msgr2=0x7f90400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 -- 172.21.15.17:0/916527372 shutdown_connections 2024-03-06T07:26:00.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.231+0000 7f9046702640 1 -- 172.21.15.17:0/916527372 wait complete. 2024-03-06T07:26:00.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.232+0000 7f9046702640 1 Processor -- start 2024-03-06T07:26:00.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 -- start start 2024-03-06T07:26:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 0x7f9040150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400c88b0 0x7f90400ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401529d0 con 0x7f90400ac0d0 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904006d090 con 0x7f90400c88b0 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9046702640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9040152b50 con 0x7f90400c5330 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9044c9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9044c9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57314/0 (socket says 172.21.15.17:57314) 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f9044c9f640 1 -- 172.21.15.17:0/998626612 learned_addr learned my addr 172.21.15.17:0/998626612 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f903ffff640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 0x7f9040150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.233+0000 7f903f7fe640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400c88b0 0x7f90400ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 -- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 msgr2=0x7f9040150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 0x7f9040150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 -- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400c88b0 msgr2=0x7f90400ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400c88b0 0x7f90400ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 -- 172.21.15.17:0/998626612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400c1070 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f903ffff640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 0x7f9040150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9044c9f640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f903800efc0 tx=0x7f903800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038018020 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038014480 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038014870 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904014d740 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.234+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90400abde0 con 0x7f90400ac0d0 2024-03-06T07:26:00.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.235+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904014d980 con 0x7f90400ac0d0 2024-03-06T07:26:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.236+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9038005d10 con 0x7f90400ac0d0 2024-03-06T07:26:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.236+0000 7f903cff9640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 0x7f902006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.236+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f903808adf0 con 0x7f90400ac0d0 2024-03-06T07:26:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.236+0000 7f903ffff640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 0x7f902006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.237+0000 7f903ffff640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 0x7f902006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9030001700 tx=0x7f9030009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.240+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f904014d980 con 0x7f90400ac0d0 2024-03-06T07:26:00.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.422+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f904004f330 con 0x7f90400ac0d0 2024-03-06T07:26:00.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.423+0000 7f903cff9640 1 -- 172.21.15.17:0/998626612 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f9038022070 con 0x7f90400ac0d0 2024-03-06T07:26:00.424 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:00.425 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":150,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:25:56.159340+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:00.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 msgr2=0x7f902006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 0x7f902006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9030001700 tx=0x7f9030009040 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 msgr2=0x7f9040148bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f903800efc0 tx=0x7f903800c490 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 shutdown_connections 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902006b4d0 0x7f902006d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90400ac0d0 0x7f9040148bf0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90400c88b0 0x7f90400ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 --2- 172.21.15.17:0/998626612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90400c5330 0x7f9040150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 >> 172.21.15.17:0/998626612 conn(0x7f90400ac560 msgr2=0x7f9040153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 shutdown_connections 2024-03-06T07:26:00.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.425+0000 7f9046702640 1 -- 172.21.15.17:0/998626612 wait complete. 2024-03-06T07:26:00.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:25:52.790714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:25:53.497871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:00.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:00.446 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:26:00.444+0000 7f8095cb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:00.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.541+0000 7f453b97a640 1 Processor -- start 2024-03-06T07:26:00.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.541+0000 7f453b97a640 1 -- start start 2024-03-06T07:26:00.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f453b97a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45340b0070 0x7f4534155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f453b97a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 0x7f45340c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f453b97a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340b0450 con 0x7f45340c5380 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f453b97a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453406c020 con 0x7f45340b0070 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f453b97a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453406cf20 con 0x7f453406ca70 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f4539f17640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f45340c5380 0x7f453414e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57226/0 (socket says 172.21.15.17:57226) 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f4539f17640 1 -- 172.21.15.17:0/3196059379 learned_addr learned my addr 172.21.15.17:0/3196059379 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f4538f15640 1 --2- 172.21.15.17:0/3196059379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 0x7f45340c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f4539716640 1 --2- 172.21.15.17:0/3196059379 >> v2:172.21.15.163:3300/0 conn(0x7f45340b0070 0x7f4534155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.542+0000 7f452bfff640 1 -- 172.21.15.17:0/3196059379 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775799721 0 0) 0x7f452c001490 con 0x7f45340c5380 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f452bfff640 1 -- 172.21.15.17:0/3196059379 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4520003520 con 0x7f45340c5380 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f4538f15640 1 -- 172.21.15.17:0/3196059379 >> v2:172.21.15.163:3300/0 conn(0x7f45340b0070 msgr2=0x7f4534155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f4538f15640 1 --2- 172.21.15.17:0/3196059379 >> v2:172.21.15.163:3300/0 conn(0x7f45340b0070 0x7f4534155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f4538f15640 1 -- 172.21.15.17:0/3196059379 >> v1:172.21.15.17:6789/0 conn(0x7f45340c5380 legacy=0x7f453414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f4538f15640 1 -- 172.21.15.17:0/3196059379 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340c77f0 con 0x7f453406ca70 2024-03-06T07:26:00.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.543+0000 7f4538f15640 1 --2- 172.21.15.17:0/3196059379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 0x7f45340c8900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45240097a0 tx=0x7f452402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7f631ab68cea7f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f452bfff640 1 -- 172.21.15.17:0/3196059379 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031780 con 0x7f453406ca70 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f452bfff640 1 -- 172.21.15.17:0/3196059379 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4524037070 con 0x7f453406ca70 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f452bfff640 1 -- 172.21.15.17:0/3196059379 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031b40 con 0x7f453406ca70 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 -- 172.21.15.17:0/3196059379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 msgr2=0x7f45340c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 --2- 172.21.15.17:0/3196059379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 0x7f45340c8900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45240097a0 tx=0x7f452402b820 comp rx=0 tx=0).stop 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 -- 172.21.15.17:0/3196059379 shutdown_connections 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 --2- 172.21.15.17:0/3196059379 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f453406ca70 0x7f45340c8900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 --2- 172.21.15.17:0/3196059379 >> v2:172.21.15.163:3300/0 conn(0x7f45340b0070 0x7f4534155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 -- 172.21.15.17:0/3196059379 >> 172.21.15.17:0/3196059379 conn(0x7f45340ac590 msgr2=0x7f45340ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 -- 172.21.15.17:0/3196059379 shutdown_connections 2024-03-06T07:26:00.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.544+0000 7f453b97a640 1 -- 172.21.15.17:0/3196059379 wait complete. 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 Processor -- start 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 -- start start 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453406ca70 0x7f4534157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c5380 0x7f453414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453406c550 con 0x7f45340b0070 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453406c270 con 0x7f453406ca70 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f453b97a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453406da80 con 0x7f45340c5380 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f4538f15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f4538f15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39494/0 (socket says 172.21.15.17:39494) 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.546+0000 7f4538f15640 1 -- 172.21.15.17:0/574880508 learned_addr learned my addr 172.21.15.17:0/574880508 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4539f17640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c5380 0x7f453414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4539716640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453406ca70 0x7f4534157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 -- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c5380 msgr2=0x7f453414de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c5380 0x7f453414de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 -- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453406ca70 msgr2=0x7f4534157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453406ca70 0x7f4534157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 -- 172.21.15.17:0/574880508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340c20f0 con 0x7f45340b0070 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4538f15640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4524034040 tx=0x7f45240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524032610 con 0x7f45340b0070 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45240327f0 con 0x7f45340b0070 2024-03-06T07:26:00.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453414e360 con 0x7f45340b0070 2024-03-06T07:26:00.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524032be0 con 0x7f45340b0070 2024-03-06T07:26:00.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.547+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45340c1e10 con 0x7f45340b0070 2024-03-06T07:26:00.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.548+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453404f330 con 0x7f45340b0070 2024-03-06T07:26:00.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.549+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f452404c060 con 0x7f45340b0070 2024-03-06T07:26:00.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.550+0000 7f4529ffb640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.550+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45240abbf0 con 0x7f45340b0070 2024-03-06T07:26:00.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.550+0000 7f4539716640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.550+0000 7f4539716640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 0x7f451806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4534156610 tx=0x7f4530008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.554+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f453414e360 con 0x7f45340b0070 2024-03-06T07:26:00.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.734+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45340c22c0 con 0x7f45340b0070 2024-03-06T07:26:00.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.734+0000 7f4529ffb640 1 -- 172.21.15.17:0/574880508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4524040070 con 0x7f45340b0070 2024-03-06T07:26:00.736 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 msgr2=0x7f451806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 0x7f451806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4534156610 tx=0x7f4530008040 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 msgr2=0x7f45340c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f4524034040 tx=0x7f45240046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 shutdown_connections 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f451806b4d0 0x7f451806d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c5380 0x7f453414de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340b0070 0x7f45340c24e0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 --2- 172.21.15.17:0/574880508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453406ca70 0x7f4534157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 >> 172.21.15.17:0/574880508 conn(0x7f45340ac590 msgr2=0x7f45340ad760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 shutdown_connections 2024-03-06T07:26:00.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.736+0000 7f453b97a640 1 -- 172.21.15.17:0/574880508 wait complete. 2024-03-06T07:26:00.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:26:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 Processor -- start 2024-03-06T07:26:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 -- start start 2024-03-06T07:26:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6c600b0070 0x7f6c60155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 0x7f6c600c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c600b0450 con 0x7f6c600c5380 2024-03-06T07:26:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6006c020 con 0x7f6c600b0070 2024-03-06T07:26:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.891+0000 7f6c66d72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006cf20 con 0x7f6c6006ca70 2024-03-06T07:26:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 0x7f6c600c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c64b0e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6c600b0070 0x7f6c60155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c6530f640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6c600c5380 0x7f6c6014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:55982/0 (socket says 172.21.15.17:55982) 2024-03-06T07:26:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c6530f640 1 -- 172.21.15.17:0/463579083 learned_addr learned my addr 172.21.15.17:0/463579083 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c57fff640 1 -- 172.21.15.17:0/463579083 >> v2:172.21.15.163:3300/0 conn(0x7f6c600b0070 msgr2=0x7f6c60155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c57fff640 1 --2- 172.21.15.17:0/463579083 >> v2:172.21.15.163:3300/0 conn(0x7f6c600b0070 0x7f6c60155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c57fff640 1 -- 172.21.15.17:0/463579083 >> v1:172.21.15.163:6789/0 conn(0x7f6c600c5380 legacy=0x7f6c6014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c577fe640 1 -- 172.21.15.17:0/463579083 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038681083 0 0) 0x7f6c58001490 con 0x7f6c600c5380 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.892+0000 7f6c57fff640 1 -- 172.21.15.17:0/463579083 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600c77f0 con 0x7f6c6006ca70 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.893+0000 7f6c57fff640 1 --2- 172.21.15.17:0/463579083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 0x7f6c600c8900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c500097a0 tx=0x7f6c5002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c0fe875a0cb2c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.893+0000 7f6c577fe640 1 -- 172.21.15.17:0/463579083 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500315a0 con 0x7f6c6006ca70 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.893+0000 7f6c577fe640 1 -- 172.21.15.17:0/463579083 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c50037070 con 0x7f6c6006ca70 2024-03-06T07:26:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.893+0000 7f6c577fe640 1 -- 172.21.15.17:0/463579083 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50031960 con 0x7f6c6006ca70 2024-03-06T07:26:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 -- 172.21.15.17:0/463579083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 msgr2=0x7f6c600c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 --2- 172.21.15.17:0/463579083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 0x7f6c600c8900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c500097a0 tx=0x7f6c5002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 -- 172.21.15.17:0/463579083 shutdown_connections 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 --2- 172.21.15.17:0/463579083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c6006ca70 0x7f6c600c8900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 --2- 172.21.15.17:0/463579083 >> v2:172.21.15.163:3300/0 conn(0x7f6c600b0070 0x7f6c60155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 -- 172.21.15.17:0/463579083 >> 172.21.15.17:0/463579083 conn(0x7f6c600ac590 msgr2=0x7f6c600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 -- 172.21.15.17:0/463579083 shutdown_connections 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.894+0000 7f6c66d72640 1 -- 172.21.15.17:0/463579083 wait complete. 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.895+0000 7f6c66d72640 1 Processor -- start 2024-03-06T07:26:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.895+0000 7f6c66d72640 1 -- start start 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6006ca70 0x7f6c601ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c600b0070 0x7f6c601592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006c550 con 0x7f6c600c5380 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006c270 con 0x7f6c6006ca70 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c66d72640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006da80 con 0x7f6c600b0070 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39502/0 (socket says 172.21.15.17:39502) 2024-03-06T07:26:00.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 -- 172.21.15.17:0/1754579685 learned_addr learned my addr 172.21.15.17:0/1754579685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c64b0e640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6006ca70 0x7f6c601ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c57fff640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c600b0070 0x7f6c601592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 -- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c600b0070 msgr2=0x7f6c601592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c600b0070 0x7f6c601592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 -- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6006ca70 msgr2=0x7f6c601ea8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6006ca70 0x7f6c601ea8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.896+0000 7f6c6530f640 1 -- 172.21.15.17:0/1754579685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c60156eb0 con 0x7f6c600c5380 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c6530f640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c7f0 tx=0x7f6c5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58018070 con 0x7f6c600c5380 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5800a5e0 con 0x7f6c600c5380 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c600c3840 con 0x7f6c600c5380 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5800aa10 con 0x7f6c600c5380 2024-03-06T07:26:00.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.897+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c600c3e00 con 0x7f6c600c5380 2024-03-06T07:26:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.898+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6004f330 con 0x7f6c600c5380 2024-03-06T07:26:00.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.898+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c5801b070 con 0x7f6c600c5380 2024-03-06T07:26:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.899+0000 7f6c557fa640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 0x7f6c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.899+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c58089840 con 0x7f6c600c5380 2024-03-06T07:26:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.899+0000 7f6c64b0e640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 0x7f6c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:00.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.900+0000 7f6c64b0e640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 0x7f6c4c06d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009b40 tx=0x7f6c5c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:00.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:00.903+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c600b0450 con 0x7f6c600c5380 2024-03-06T07:26:01.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.082+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c6006d860 con 0x7f6c600c5380 2024-03-06T07:26:01.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.083+0000 7f6c557fa640 1 -- 172.21.15.17:0/1754579685 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c58057f70 con 0x7f6c600c5380 2024-03-06T07:26:01.084 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:26:01.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.084+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 msgr2=0x7f6c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.084+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 0x7f6c4c06d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c009b40 tx=0x7f6c5c009190 comp rx=0 tx=0).stop 2024-03-06T07:26:01.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.084+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 msgr2=0x7f6c60154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.084+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800c7f0 tx=0x7f6c5800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 shutdown_connections 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c4c06b4d0 0x7f6c4c06d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c600c5380 0x7f6c60154930 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c600b0070 0x7f6c601592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 --2- 172.21.15.17:0/1754579685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c6006ca70 0x7f6c601ea8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 >> 172.21.15.17:0/1754579685 conn(0x7f6c600ac590 msgr2=0x7f6c600ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 shutdown_connections 2024-03-06T07:26:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.085+0000 7f6c66d72640 1 -- 172.21.15.17:0/1754579685 wait complete. 2024-03-06T07:26:01.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:26:01.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.201+0000 7f1006234640 1 Processor -- start 2024-03-06T07:26:01.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.201+0000 7f1006234640 1 -- start start 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f1006234640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f100014f1c0 0x7f100006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f1006234640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10000ad150 con 0x7f10000c1010 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f1006234640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1000155ee0 con 0x7f1000072960 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f1006234640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1000071f30 con 0x7f100014f1c0 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0fff7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f100014f1c0 0x7f100006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0ffeffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10000c1010 0x7f100006afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57238/0 (socket says 172.21.15.17:57238) 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0fff7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f100014f1c0 0x7f100006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39506/0 (socket says 172.21.15.17:39506) 2024-03-06T07:26:01.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0ffeffd640 1 -- 172.21.15.17:0/2615649901 learned_addr learned my addr 172.21.15.17:0/2615649901 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:01.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0fff7fe640 1 -- 172.21.15.17:0/2615649901 >> v1:172.21.15.163:6789/0 conn(0x7f1000072960 legacy=0x7f10000c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0fff7fe640 1 -- 172.21.15.17:0/2615649901 >> v1:172.21.15.17:6789/0 conn(0x7f10000c1010 legacy=0x7f100006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0fff7fe640 1 -- 172.21.15.17:0/2615649901 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000072160 con 0x7f100014f1c0 2024-03-06T07:26:01.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.202+0000 7f0ffe7fc640 1 -- 172.21.15.17:0/2615649901 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336547693 0 0) 0x7f10000ad150 con 0x7f10000c1010 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.203+0000 7f0fff7fe640 1 --2- 172.21.15.17:0/2615649901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f100014f1c0 0x7f100006c310 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80097d0 tx=0x7f0fe802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd57bc3520af9df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.203+0000 7f0ffe7fc640 1 -- 172.21.15.17:0/2615649901 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8034070 con 0x7f100014f1c0 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.203+0000 7f0ffe7fc640 1 -- 172.21.15.17:0/2615649901 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8030610 con 0x7f100014f1c0 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.203+0000 7f0ffe7fc640 1 -- 172.21.15.17:0/2615649901 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80309e0 con 0x7f100014f1c0 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 -- 172.21.15.17:0/2615649901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f100014f1c0 msgr2=0x7f100006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 --2- 172.21.15.17:0/2615649901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f100014f1c0 0x7f100006c310 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80097d0 tx=0x7f0fe802b960 comp rx=0 tx=0).stop 2024-03-06T07:26:01.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 -- 172.21.15.17:0/2615649901 shutdown_connections 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 --2- 172.21.15.17:0/2615649901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f100014f1c0 0x7f100006c310 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 -- 172.21.15.17:0/2615649901 >> 172.21.15.17:0/2615649901 conn(0x7f10000c3ab0 msgr2=0x7f10000c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 -- 172.21.15.17:0/2615649901 shutdown_connections 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.204+0000 7f1006234640 1 -- 172.21.15.17:0/2615649901 wait complete. 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.205+0000 7f1006234640 1 Processor -- start 2024-03-06T07:26:01.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.205+0000 7f1006234640 1 -- start start 2024-03-06T07:26:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 0x7f10001ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f100014f1c0 0x7f10001ec040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100014f5a0 con 0x7f1000072960 2024-03-06T07:26:01.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000155bf0 con 0x7f100014f1c0 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f1006234640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000072e10 con 0x7f10000c1010 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f0fff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f0fff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39516/0 (socket says 172.21.15.17:39516) 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.206+0000 7f0fff7fe640 1 -- 172.21.15.17:0/718210012 learned_addr learned my addr 172.21.15.17:0/718210012 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ffeffd640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 0x7f10001ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ffffff640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f100014f1c0 0x7f10001ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 -- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 msgr2=0x7f10001ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 0x7f10001ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 -- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f100014f1c0 msgr2=0x7f10001ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f100014f1c0 0x7f10001ec040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 -- 172.21.15.17:0/718210012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10001eebf0 con 0x7f1000072960 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ffeffd640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 0x7f10001ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0fff7fe640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80098f0 tx=0x7f0fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8034040 con 0x7f1000072960 2024-03-06T07:26:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe803acc0 con 0x7f1000072960 2024-03-06T07:26:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8044630 con 0x7f1000072960 2024-03-06T07:26:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10001e13f0 con 0x7f1000072960 2024-03-06T07:26:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.207+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10001e16d0 con 0x7f1000072960 2024-03-06T07:26:01.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.208+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100004f330 con 0x7f1000072960 2024-03-06T07:26:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.209+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe804b060 con 0x7f1000072960 2024-03-06T07:26:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.209+0000 7f0ff7fff640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 0x7f0fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.209+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fe80ac2d0 con 0x7f1000072960 2024-03-06T07:26:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.209+0000 7f0ffeffd640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 0x7f0fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.210+0000 7f0ffeffd640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 0x7f0fe006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f10000c2ed0 tx=0x7f0ff0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.214+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0fe803f070 con 0x7f1000072960 2024-03-06T07:26:01.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.394+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10000acf30 con 0x7f1000072960 2024-03-06T07:26:01.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.395+0000 7f0ff7fff640 1 -- 172.21.15.17:0/718210012 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fe804b350 con 0x7f1000072960 2024-03-06T07:26:01.397 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 msgr2=0x7f0fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 0x7f0fe006d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f10000c2ed0 tx=0x7f0ff0009040 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 msgr2=0x7f10001e0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80098f0 tx=0x7f0fe80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 shutdown_connections 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0fe006b4d0 0x7f0fe006d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f100014f1c0 0x7f10001ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10000c1010 0x7f10001ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 --2- 172.21.15.17:0/718210012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1000072960 0x7f10001e0ca0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 >> 172.21.15.17:0/718210012 conn(0x7f10000c3ab0 msgr2=0x7f100014d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 shutdown_connections 2024-03-06T07:26:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.397+0000 7f1006234640 1 -- 172.21.15.17:0/718210012 wait complete. 2024-03-06T07:26:01.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:26:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.509+0000 7fc5a2fbc640 1 Processor -- start 2024-03-06T07:26:01.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.509+0000 7fc5a2fbc640 1 -- start start 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a2fbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc59c14f650 0x7fc59c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a2fbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c072440 con 0x7fc59c152c20 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a2fbc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0c2b10 con 0x7fc59c0c3150 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a2fbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c152140 con 0x7fc59c14f650 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a1559640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc59c152c20 0x7fc59c15c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57252/0 (socket says 172.21.15.17:57252) 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a1559640 1 -- 172.21.15.17:0/2619870720 learned_addr learned my addr 172.21.15.17:0/2619870720 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a0d58640 1 --2- 172.21.15.17:0/2619870720 >> v2:172.21.15.17:3300/0 conn(0x7fc59c14f650 0x7fc59c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a0d58640 1 -- 172.21.15.17:0/2619870720 >> v1:172.21.15.163:6789/0 conn(0x7fc59c0c3150 legacy=0x7fc59c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a0d58640 1 -- 172.21.15.17:0/2619870720 >> v1:172.21.15.17:6789/0 conn(0x7fc59c152c20 legacy=0x7fc59c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.510+0000 7fc5a0d58640 1 -- 172.21.15.17:0/2619870720 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c152420 con 0x7fc59c14f650 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.511+0000 7fc5a0d58640 1 --2- 172.21.15.17:0/2619870720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c14f650 0x7fc59c071880 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc5900097a0 tx=0x7fc59002da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3727145670b4e060 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.511+0000 7fc59b7fe640 1 -- 172.21.15.17:0/2619870720 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590031870 con 0x7fc59c14f650 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.511+0000 7fc59b7fe640 1 -- 172.21.15.17:0/2619870720 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc590031a10 con 0x7fc59c14f650 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.511+0000 7fc59b7fe640 1 -- 172.21.15.17:0/2619870720 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590031de0 con 0x7fc59c14f650 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/2619870720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c14f650 msgr2=0x7fc59c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/2619870720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c14f650 0x7fc59c071880 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc5900097a0 tx=0x7fc59002da50 comp rx=0 tx=0).stop 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/2619870720 shutdown_connections 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/2619870720 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c14f650 0x7fc59c071880 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/2619870720 >> 172.21.15.17:0/2619870720 conn(0x7fc59c0c4290 msgr2=0x7fc59c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/2619870720 shutdown_connections 2024-03-06T07:26:01.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.512+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/2619870720 wait complete. 2024-03-06T07:26:01.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.513+0000 7fc5a2fbc640 1 Processor -- start 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.513+0000 7fc5a2fbc640 1 -- start start 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 0x7fc59c146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc59c152c20 0x7fc59c1e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c40c0 con 0x7fc59c0c3150 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c3f40 con 0x7fc59c14f650 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a2fbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c5da0 con 0x7fc59c152c20 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39534/0 (socket says 172.21.15.17:39534) 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 -- 172.21.15.17:0/1730648997 learned_addr learned my addr 172.21.15.17:0/1730648997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc59bfff640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 0x7fc59c146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a1559640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc59c152c20 0x7fc59c1e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 -- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc59c152c20 msgr2=0x7fc59c1e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc59c152c20 0x7fc59c1e67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 -- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 msgr2=0x7fc59c146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 0x7fc59c146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 -- 172.21.15.17:0/1730648997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c06ea50 con 0x7fc59c0c3150 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc59bfff640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 0x7fc59c146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5a0d58640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc5900098c0 tx=0x7fc590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590049070 con 0x7fc59c0c3150 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc590039e10 con 0x7fc59c0c3150 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.514+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59002f640 con 0x7fc59c0c3150 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.515+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c1e9f70 con 0x7fc59c0c3150 2024-03-06T07:26:01.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.515+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c146670 con 0x7fc59c0c3150 2024-03-06T07:26:01.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.516+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c04f330 con 0x7fc59c0c3150 2024-03-06T07:26:01.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.516+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc59004c070 con 0x7fc59c0c3150 2024-03-06T07:26:01.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.517+0000 7fc5997fa640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.517+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5900ae5f0 con 0x7fc59c0c3150 2024-03-06T07:26:01.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.517+0000 7fc59bfff640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.517+0000 7fc59bfff640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 0x7fc58006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc59c06e860 tx=0x7fc588009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.521+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc590035030 con 0x7fc59c0c3150 2024-03-06T07:26:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.703+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc59c068460 con 0x7fc59c0c3150 2024-03-06T07:26:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.704+0000 7fc5997fa640 1 -- 172.21.15.17:0/1730648997 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) v1 ==== 84+0+209 (secure 0 0 0) 0x7fc59007d370 con 0x7fc59c0c3150 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 23, 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_metadata" 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 24, 2024-03-06T07:26:01.706 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": "cephfs_data" 2024-03-06T07:26:01.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:01.707 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 msgr2=0x7fc58006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 0x7fc58006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc59c06e860 tx=0x7fc588009040 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 msgr2=0x7fc59c1ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc5900098c0 tx=0x7fc590004730 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 shutdown_connections 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc59c152c20 0x7fc59c1e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59c14f650 0x7fc59c146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 --2- 172.21.15.17:0/1730648997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc59c0c3150 0x7fc59c1ea770 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 >> 172.21.15.17:0/1730648997 conn(0x7fc59c0c4290 msgr2=0x7fc59c148350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 shutdown_connections 2024-03-06T07:26:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.706+0000 7fc5a2fbc640 1 -- 172.21.15.17:0/1730648997 wait complete. 2024-03-06T07:26:01.718 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs2' 2024-03-06T07:26:01.719 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_metadata --pg_num_min 64 2024-03-06T07:26:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.826+0000 7f87b973e640 1 Processor -- start 2024-03-06T07:26:01.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.826+0000 7f87b973e640 1 -- start start 2024-03-06T07:26:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b973e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87b4152860 0x7f87b40c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b973e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40c8ea0 con 0x7f87b40c7470 2024-03-06T07:26:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b973e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40c7a10 con 0x7f87b4159230 2024-03-06T07:26:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b973e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b4158da0 con 0x7f87b4152860 2024-03-06T07:26:01.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87abfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f87b40c7470 0x7f87b4155d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57268/0 (socket says 172.21.15.17:57268) 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87abfff640 1 -- 172.21.15.17:0/339515670 learned_addr learned my addr 172.21.15.17:0/339515670 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b2ffd640 1 --2- 172.21.15.17:0/339515670 >> v2:172.21.15.163:3300/0 conn(0x7f87b4152860 0x7f87b40c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050616928 0 0) 0x7f87b40c8ea0 con 0x7f87b40c7470 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a0003520 con 0x7f87b40c7470 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029941222 0 0) 0x7f87a0003520 con 0x7f87b40c7470 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 >> v2:172.21.15.163:3300/0 conn(0x7f87b4152860 msgr2=0x7f87b40c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 --2- 172.21.15.17:0/339515670 >> v2:172.21.15.163:3300/0 conn(0x7f87b4152860 0x7f87b40c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 >> v1:172.21.15.163:6789/0 conn(0x7f87b4159230 legacy=0x7f87b40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40c7cf0 con 0x7f87b40c7470 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f879c001670 con 0x7f87b40c7470 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 >> v1:172.21.15.17:6789/0 conn(0x7f87b40c7470 legacy=0x7f87b4155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00039b0 0x7f87a0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.827+0000 7f87b27fc640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0005210 0x7f87a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4158da0 con 0x7f87a0004800 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000c140 con 0x7f87a0005210 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a000c340 con 0x7f87a00039b0 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b2ffd640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00039b0 0x7f87a0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b37fe640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0005210 0x7f87a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 -- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00039b0 msgr2=0x7f87a0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87a00039b0 0x7f87a0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 -- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0005210 msgr2=0x7f87a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0005210 0x7f87a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87abfff640 1 -- 172.21.15.17:0/339515670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0003520 con 0x7f87a0004800 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b37fe640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87a0005210 0x7f87a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.828+0000 7f87b37fe640 1 -- 172.21.15.17:0/339515670 reap_dead start 2024-03-06T07:26:01.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87abfff640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87a0004c80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c240 tx=0x7f87a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647d4d302ab98f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c007cf0 con 0x7f87a0004800 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c030070 con 0x7f87a0004800 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b27fc640 1 -- 172.21.15.17:0/339515670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c005d70 con 0x7f87a0004800 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 -- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 msgr2=0x7f87a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87a0004c80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f879c00c240 tx=0x7f87a0005830 comp rx=0 tx=0).stop 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 -- 172.21.15.17:0/339515670 shutdown_connections 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 --2- 172.21.15.17:0/339515670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87a0004c80 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 -- 172.21.15.17:0/339515670 >> 172.21.15.17:0/339515670 conn(0x7f87b40ac630 msgr2=0x7f87b40aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 -- 172.21.15.17:0/339515670 shutdown_connections 2024-03-06T07:26:01.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.829+0000 7f87b973e640 1 -- 172.21.15.17:0/339515670 wait complete. 2024-03-06T07:26:01.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.830+0000 7f87b973e640 1 Processor -- start 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.830+0000 7f87b973e640 1 -- start start 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b40be950 0x7f87b40c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87b40c4370 0x7f87b4157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4159680 con 0x7f87a0004800 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4152c40 con 0x7f87b40be950 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b973e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4159800 con 0x7f87b40c4370 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39562/0 (socket says 172.21.15.17:39562) 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b2ffd640 1 -- 172.21.15.17:0/3622763361 learned_addr learned my addr 172.21.15.17:0/3622763361 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b2ffd640 1 -- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87b40c4370 msgr2=0x7f87b4157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.831+0000 7f87b2ffd640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87b40c4370 0x7f87b4157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b2ffd640 1 -- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b40be950 msgr2=0x7f87b40c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:01.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b2ffd640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b40be950 0x7f87b40c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b2ffd640 1 -- 172.21.15.17:0/3622763361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b41eee80 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b2ffd640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f87a4011a50 tx=0x7f87a4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a400fb30 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400fcd0 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40bf350 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4014660 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.832+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40c2c90 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.833+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b404f330 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.833+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87a4012040 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.834+0000 7f87ab7fe640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 0x7f879406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.834+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87a408a9f0 con 0x7f87a0004800 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.834+0000 7f87abfff640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 0x7f879406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.835+0000 7f87abfff640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 0x7f879406d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f879c034000 tx=0x7f879c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:01.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:01.838+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87a4019030 con 0x7f87a0004800 2024-03-06T07:26:02.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.019+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87b40c8110 con 0x7f87a0004800 2024-03-06T07:26:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.204+0000 7f87ab7fe640 1 -- 172.21.15.17:0/3622763361 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_metadata", "pg_num_min": 64}]=0 pool 'cephfs2_metadata' created v151) v1 ==== 140+0+0 (secure 0 0 0) 0x7f87a408a070 con 0x7f87a0004800 2024-03-06T07:26:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' created 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 msgr2=0x7f879406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 0x7f879406d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f879c034000 tx=0x7f879c02e000 comp rx=0 tx=0).stop 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 msgr2=0x7f87b40bee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f87a4011a50 tx=0x7f87a4011f10 comp rx=0 tx=0).stop 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 shutdown_connections 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f879406b500 0x7f879406d9c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87b40c4370 0x7f87b4157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87b40be950 0x7f87b40c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 --2- 172.21.15.17:0/3622763361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87a0004800 0x7f87b40bee10 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 >> 172.21.15.17:0/3622763361 conn(0x7f87b40ac630 msgr2=0x7f87b40ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:02.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 shutdown_connections 2024-03-06T07:26:02.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.206+0000 7f87b973e640 1 -- 172.21.15.17:0/3622763361 wait complete. 2024-03-06T07:26:02.224 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs2_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:26:02.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.335+0000 7fb46c00e640 1 Processor -- start 2024-03-06T07:26:02.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.335+0000 7fb46c00e640 1 -- start start 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.335+0000 7fb46c00e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4640c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46c00e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4640c12a0 0x7fb46406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46c00e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb464155ed0 con 0x7fb46406b500 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46c00e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640c6a00 con 0x7fb4640c12a0 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46c00e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640c6c30 con 0x7fb464152770 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46a5ab640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb46406b500 0x7fb4640c7710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57272/0 (socket says 172.21.15.17:57272) 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2916268333 learned_addr learned my addr 172.21.15.17:0/2916268333 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742493761 0 0) 0x7fb464155ed0 con 0x7fb46406b500 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb450003520 con 0x7fb46406b500 2024-03-06T07:26:02.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb469daa640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4640c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> v2:172.21.15.163:3300/0 conn(0x7fb4640c12a0 0x7fb46406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153834249 0 0) 0x7fb450003520 con 0x7fb46406b500 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 msgr2=0x7fb4640c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4640c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 >> v2:172.21.15.163:3300/0 conn(0x7fb4640c12a0 msgr2=0x7fb46406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 --2- 172.21.15.17:0/2916268333 >> v2:172.21.15.163:3300/0 conn(0x7fb4640c12a0 0x7fb46406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640c6f10 con 0x7fb46406b500 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb45c0019f0 con 0x7fb46406b500 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 >> v1:172.21.15.17:6789/0 conn(0x7fb46406b500 legacy=0x7fb4640c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb450003f30 0x7fb450004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450004d40 0x7fb4500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640c6c30 con 0x7fb450003f30 2024-03-06T07:26:02.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45000b9c0 con 0x7fb450004d40 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45000bb90 con 0x7fb464152770 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.336+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> v2:172.21.15.163:3300/0 conn(0x7fb4640c12a0 0x7fb46406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb469daa640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb450003f30 0x7fb450004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb46a5ab640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450004d40 0x7fb4500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 -- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 msgr2=0x7fb4500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464152770 0x7fb4500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 -- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450004d40 msgr2=0x7fb4500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450004d40 0x7fb4500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.337+0000 7fb4695a9640 1 -- 172.21.15.17:0/2916268333 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450003520 con 0x7fb450003f30 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.338+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb450003f30 0x7fb450004800 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb4640c16e0 tx=0x7fb450005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3536696239dc234 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.338+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454034070 con 0x7fb450003f30 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.338+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454030660 con 0x7fb450003f30 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.338+0000 7fb468da8640 1 -- 172.21.15.17:0/2916268333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400f040 con 0x7fb450003f30 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 -- 172.21.15.17:0/2916268333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb450003f30 msgr2=0x7fb450004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2916268333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb450003f30 0x7fb450004800 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb4640c16e0 tx=0x7fb450005830 comp rx=0 tx=0).stop 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2916268333 reap_dead start 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 -- 172.21.15.17:0/2916268333 shutdown_connections 2024-03-06T07:26:02.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 -- 172.21.15.17:0/2916268333 >> 172.21.15.17:0/2916268333 conn(0x7fb4640a94d0 msgr2=0x7fb4640a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 -- 172.21.15.17:0/2916268333 shutdown_connections 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.339+0000 7fb46c00e640 1 -- 172.21.15.17:0/2916268333 wait complete. 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.340+0000 7fb46c00e640 1 Processor -- start 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.340+0000 7fb46c00e640 1 -- start start 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450003f30 0x7fb4641ee510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464071380 0x7fb4641567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464155ba0 con 0x7fb4640ab890 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464152b50 con 0x7fb450003f30 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46c00e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640c1990 con 0x7fb464071380 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39572/0 (socket says 172.21.15.17:39572) 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2307256859 learned_addr learned my addr 172.21.15.17:0/2307256859 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb4695a9640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464071380 0x7fb4641567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb469daa640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450003f30 0x7fb4641ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464071380 msgr2=0x7fb4641567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464071380 0x7fb4641567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450003f30 msgr2=0x7fb4641ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450003f30 0x7fb4641ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 -- 172.21.15.17:0/2307256859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640ac790 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb46a5ab640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb464074d30 tx=0x7fb45c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.341+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c019070 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.342+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464158360 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.342+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00ab60 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.342+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0133f0 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.342+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464158600 con 0x7fb4640ab890 2024-03-06T07:26:02.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.343+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46404f330 con 0x7fb4640ab890 2024-03-06T07:26:02.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.343+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb45c01c070 con 0x7fb4640ab890 2024-03-06T07:26:02.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.344+0000 7fb45a7fc640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:02.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.344+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6472+0+0 (secure 0 0 0) 0x7fb45c08a060 con 0x7fb4640ab890 2024-03-06T07:26:02.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.344+0000 7fb469daa640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:02.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.344+0000 7fb469daa640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 0x7fb44806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb460009a20 tx=0x7fb460008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:02.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.348+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb45c058b60 con 0x7fb4640ab890 2024-03-06T07:26:02.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:02.529+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4640c1010 con 0x7fb4640ab890 2024-03-06T07:26:03.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.210+0000 7fb45a7fc640 1 -- 172.21.15.17:0/2307256859 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs2_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs2_data' created v152) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb45c058460 con 0x7fb4640ab890 2024-03-06T07:26:03.212 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' created 2024-03-06T07:26:03.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.213+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 msgr2=0x7fb44806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.213+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 0x7fb44806d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb460009a20 tx=0x7fb460008040 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.213+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 msgr2=0x7fb4640abd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.213+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fb464074d30 tx=0x7fb45c007a90 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 shutdown_connections 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb44806b4d0 0x7fb44806d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4640ab890 0x7fb4640abd10 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb464071380 0x7fb4641567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 --2- 172.21.15.17:0/2307256859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb450003f30 0x7fb4641ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.214+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 >> 172.21.15.17:0/2307256859 conn(0x7fb4640a94d0 msgr2=0x7fb46406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:03.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.219+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 shutdown_connections 2024-03-06T07:26:03.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.220+0000 7fb46c00e640 1 -- 172.21.15.17:0/2307256859 wait complete. 2024-03-06T07:26:03.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs2 cephfs2_metadata cephfs2_data 2024-03-06T07:26:03.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.345+0000 7f8a98a01640 1 Processor -- start 2024-03-06T07:26:03.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.345+0000 7f8a98a01640 1 -- start start 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a98a01640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a940727e0 0x7f8a940c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a98a01640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a940c7700 con 0x7f8a940c8a30 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a98a01640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a940c79e0 con 0x7f8a9414b980 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a98a01640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a940c7cc0 con 0x7f8a940727e0 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a940727e0 0x7f8a940c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9359e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8a940c8a30 0x7f8a94155280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57274/0 (socket says 172.21.15.17:57274) 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9359e640 1 -- 172.21.15.17:0/310276564 learned_addr learned my addr 172.21.15.17:0/310276564 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9259c640 1 -- 172.21.15.17:0/310276564 >> v1:172.21.15.163:6789/0 conn(0x7f8a9414b980 legacy=0x7f8a94072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9259c640 1 -- 172.21.15.17:0/310276564 >> v1:172.21.15.17:6789/0 conn(0x7f8a940c8a30 legacy=0x7f8a94155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.346+0000 7f8a9259c640 1 -- 172.21.15.17:0/310276564 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a940c7fa0 con 0x7f8a940727e0 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.347+0000 7f8a9259c640 1 --2- 172.21.15.17:0/310276564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a940c84f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53656cb50225275 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.347+0000 7f8a91d9b640 1 -- 172.21.15.17:0/310276564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84034070 con 0x7f8a940727e0 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.347+0000 7f8a91d9b640 1 -- 172.21.15.17:0/310276564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a84030610 con 0x7f8a940727e0 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.347+0000 7f8a91d9b640 1 -- 172.21.15.17:0/310276564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840309e0 con 0x7f8a940727e0 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 -- 172.21.15.17:0/310276564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 msgr2=0x7f8a940c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 --2- 172.21.15.17:0/310276564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a940c84f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402b960 comp rx=0 tx=0).stop 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 -- 172.21.15.17:0/310276564 shutdown_connections 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 --2- 172.21.15.17:0/310276564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a940c84f0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 -- 172.21.15.17:0/310276564 >> 172.21.15.17:0/310276564 conn(0x7f8a940aa2d0 msgr2=0x7f8a940aa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 -- 172.21.15.17:0/310276564 shutdown_connections 2024-03-06T07:26:03.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.348+0000 7f8a98a01640 1 -- 172.21.15.17:0/310276564 wait complete. 2024-03-06T07:26:03.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.349+0000 7f8a98a01640 1 Processor -- start 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.349+0000 7f8a98a01640 1 -- start start 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a940c8a30 0x7f8a941574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9414b980 0x7f8a94149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94155990 con 0x7f8a940727e0 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9414bd60 con 0x7f8a940c8a30 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a98a01640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a94155b10 con 0x7f8a9414b980 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39596/0 (socket says 172.21.15.17:39596) 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 -- 172.21.15.17:0/1191283547 learned_addr learned my addr 172.21.15.17:0/1191283547 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a9359e640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9414b980 0x7f8a94149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a9259c640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a940c8a30 0x7f8a941574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 -- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9414b980 msgr2=0x7f8a94149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9414b980 0x7f8a94149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 -- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a940c8a30 msgr2=0x7f8a941574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a940c8a30 0x7f8a941574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 -- 172.21.15.17:0/1191283547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a941eb720 con 0x7f8a940727e0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.350+0000 7f8a92d9d640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c3c0 tx=0x7f8a8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.351+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880052d0 con 0x7f8a940727e0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.351+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a88005490 con 0x7f8a940727e0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.351+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88007260 con 0x7f8a940727e0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.351+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9406fd90 con 0x7f8a940727e0 2024-03-06T07:26:03.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.351+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a94157a20 con 0x7f8a940727e0 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.352+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a940693c0 con 0x7f8a940727e0 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.352+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a88007400 con 0x7f8a940727e0 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.353+0000 7f8a7b7fe640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.353+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6855+0+0 (secure 0 0 0) 0x7f8a88089ed0 con 0x7f8a940727e0 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.353+0000 7f8a9259c640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:03.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.353+0000 7f8a9259c640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8a84036000 tx=0x7f8a8403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:03.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.357+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8a88058a40 con 0x7f8a940727e0 2024-03-06T07:26:03.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:03.652+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"} v 0) v1 -- 0x7f8a9414c000 con 0x7f8a940727e0 2024-03-06T07:26:04.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.223+0000 7f8a7b7fe640 1 -- 172.21.15.17:0/1191283547 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs2", "metadata": "cephfs2_metadata", "data": "cephfs2_data"}]=0 new fs with metadata pool 25 and data pool 26 v137) v1 ==== 177+0+0 (secure 0 0 0) 0x7f8a88058340 con 0x7f8a940727e0 2024-03-06T07:26:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 25 and data pool 26 2024-03-06T07:26:04.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 msgr2=0x7f8a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f8a84036000 tx=0x7f8a8403b040 comp rx=0 tx=0).stop 2024-03-06T07:26:04.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 msgr2=0x7f8a9414aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c3c0 tx=0x7f8a8800c880 comp rx=0 tx=0).stop 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 shutdown_connections 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9414b980 0x7f8a94149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a940c8a30 0x7f8a941574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 --2- 172.21.15.17:0/1191283547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a940727e0 0x7f8a9414aba0 secure :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c3c0 tx=0x7f8a8800c880 comp rx=0 tx=0).stop 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.232+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 >> 172.21.15.17:0/1191283547 conn(0x7f8a940aa2d0 msgr2=0x7f8a940c1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.233+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 shutdown_connections 2024-03-06T07:26:04.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.233+0000 7f8a98a01640 1 -- 172.21.15.17:0/1191283547 wait complete. 2024-03-06T07:26:04.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:04.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.352+0000 7f561f450640 1 Processor -- start 2024-03-06T07:26:04.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 -- start start 2024-03-06T07:26:04.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 0x7f5618153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f56180acc00 0x7f56180c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5618070aa0 con 0x7f5618156ea0 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5618156450 con 0x7f56180acc00 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561f450640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5618156a10 con 0x7f56181506b0 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561d9ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f56180acc00 0x7f56180c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561c9eb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5618156ea0 0x7f56180702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57282/0 (socket says 172.21.15.17:57282) 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.353+0000 7f561c9eb640 1 -- 172.21.15.17:0/1011232131 learned_addr learned my addr 172.21.15.17:0/1011232131 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d9ed640 1 -- 172.21.15.17:0/1011232131 >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 msgr2=0x7f5618153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1011232131 >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 0x7f5618153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d9ed640 1 --2- 172.21.15.17:0/1011232131 >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 0x7f5618153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d9ed640 1 -- 172.21.15.17:0/1011232131 >> v1:172.21.15.17:6789/0 conn(0x7f5618156ea0 legacy=0x7f56180702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d9ed640 1 -- 172.21.15.17:0/1011232131 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618070d80 con 0x7f56180acc00 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f560ffff640 1 -- 172.21.15.17:0/1011232131 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705348892 0 0) 0x7f5618070aa0 con 0x7f5618156ea0 2024-03-06T07:26:04.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1011232131 >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 0x7f5618153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.354+0000 7f561d9ed640 1 --2- 172.21.15.17:0/1011232131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56180acc00 0x7f56180c54a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f56180c97e0 tx=0x7f561002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42656a7116c9e25e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f560ffff640 1 -- 172.21.15.17:0/1011232131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610039070 con 0x7f56180acc00 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f560ffff640 1 -- 172.21.15.17:0/1011232131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561002cc80 con 0x7f56180acc00 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f560ffff640 1 -- 172.21.15.17:0/1011232131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610034660 con 0x7f56180acc00 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 -- 172.21.15.17:0/1011232131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56180acc00 msgr2=0x7f56180c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 --2- 172.21.15.17:0/1011232131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56180acc00 0x7f56180c54a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f56180c97e0 tx=0x7f561002bed0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 -- 172.21.15.17:0/1011232131 shutdown_connections 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 --2- 172.21.15.17:0/1011232131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56180acc00 0x7f56180c54a0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 --2- 172.21.15.17:0/1011232131 >> v2:172.21.15.163:3300/0 conn(0x7f56181506b0 0x7f5618153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 -- 172.21.15.17:0/1011232131 >> 172.21.15.17:0/1011232131 conn(0x7f561806b1a0 msgr2=0x7f561806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 -- 172.21.15.17:0/1011232131 shutdown_connections 2024-03-06T07:26:04.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.355+0000 7f561f450640 1 -- 172.21.15.17:0/1011232131 wait complete. 2024-03-06T07:26:04.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.356+0000 7f561f450640 1 Processor -- start 2024-03-06T07:26:04.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.356+0000 7f561f450640 1 -- start start 2024-03-06T07:26:04.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 0x7f56180c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 0x7f5618144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180ad1d0 con 0x7f56181506b0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56180ad050 con 0x7f5618156ea0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561f450640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56180ad350 con 0x7f56180acc00 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561c9eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561c9eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39622/0 (socket says 172.21.15.17:39622) 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561c9eb640 1 -- 172.21.15.17:0/1293154159 learned_addr learned my addr 172.21.15.17:0/1293154159 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 0x7f56180c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.357+0000 7f561d9ed640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 0x7f5618144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 -- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 msgr2=0x7f56180c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 0x7f56180c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 -- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 msgr2=0x7f5618144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 0x7f5618144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 -- 172.21.15.17:0/1293154159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56180c8160 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 0x7f56180c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561d9ed640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 0x7f5618144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561c9eb640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f560800c410 tx=0x7f560800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56080074e0 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56080047e0 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56181ea830 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608004bd0 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.358+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181eaad0 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.359+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5608011070 con 0x7f56181506b0 2024-03-06T07:26:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.359+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56180a8160 con 0x7f56181506b0 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.360+0000 7f560dffb640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.360+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f560808a230 con 0x7f56181506b0 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.360+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.361+0000 7f561d1ec640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f561400aa30 tx=0x7f5614008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.364+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f560800e280 con 0x7f56181506b0 2024-03-06T07:26:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.564+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f561804f330 con 0x7f56181506b0 2024-03-06T07:26:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.565+0000 7f560dffb640 1 -- 172.21.15.17:0/1293154159 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+20615 (secure 0 0 0) 0x7f5608058c60 con 0x7f56181506b0 2024-03-06T07:26:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:04.567 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":153,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:03.653966+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 msgr2=0x7f55fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 0x7f55fc06d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f561400aa30 tx=0x7f5614008040 comp rx=0 tx=0).stop 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 msgr2=0x7f56181edc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f560800c410 tx=0x7f560800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 shutdown_connections 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f55fc06b4d0 0x7f55fc06d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5618156ea0 0x7f5618144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56181506b0 0x7f56181edc60 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 --2- 172.21.15.17:0/1293154159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56180acc00 0x7f56180c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 >> 172.21.15.17:0/1293154159 conn(0x7f561806b1a0 msgr2=0x7f5618152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 shutdown_connections 2024-03-06T07:26:04.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.567+0000 7f561f450640 1 -- 172.21.15.17:0/1293154159 wait complete. 2024-03-06T07:26:04.580 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.679+0000 7f3738f2b640 1 Processor -- start 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.679+0000 7f3738f2b640 1 -- start start 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3738f2b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3734152920 0x7f3734155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3738f2b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 0x7f37340c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3738f2b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373406afc0 con 0x7f37340c2190 2024-03-06T07:26:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3738f2b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3734156160 con 0x7f3734152920 2024-03-06T07:26:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3738f2b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373406b2a0 con 0x7f37340c2620 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f373259c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3734152920 0x7f3734155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3729d9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f37340c2190 0x7f37340c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:56018/0 (socket says 172.21.15.17:56018) 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3729d9b640 1 -- 172.21.15.17:0/3098986015 learned_addr learned my addr 172.21.15.17:0/3098986015 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3732d9d640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 0x7f37340c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3731d9b640 1 -- 172.21.15.17:0/3098986015 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622036196 0 0) 0x7f373406afc0 con 0x7f37340c2190 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.680+0000 7f3731d9b640 1 -- 172.21.15.17:0/3098986015 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3720003520 con 0x7f37340c2190 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f373259c640 1 -- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 msgr2=0x7f37340c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f373259c640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 0x7f37340c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f373259c640 1 -- 172.21.15.17:0/3098986015 >> v1:172.21.15.163:6789/0 conn(0x7f37340c2190 legacy=0x7f37340c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f373259c640 1 -- 172.21.15.17:0/3098986015 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373406b580 con 0x7f3734152920 2024-03-06T07:26:04.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f3732d9d640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 0x7f37340c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.681+0000 7f373259c640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3734152920 0x7f3734155c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f37240097a0 tx=0x7f372402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b154ab9d9657742 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3731d9b640 1 -- 172.21.15.17:0/3098986015 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724035020 con 0x7f3734152920 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3731d9b640 1 -- 172.21.15.17:0/3098986015 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3724031830 con 0x7f3734152920 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3731d9b640 1 -- 172.21.15.17:0/3098986015 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724031be0 con 0x7f3734152920 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 -- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3734152920 msgr2=0x7f3734155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3734152920 0x7f3734155c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f37240097a0 tx=0x7f372402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 -- 172.21.15.17:0/3098986015 shutdown_connections 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2620 0x7f37340c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3098986015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3734152920 0x7f3734155c20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 -- 172.21.15.17:0/3098986015 >> 172.21.15.17:0/3098986015 conn(0x7f37340c4290 msgr2=0x7f37340c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 -- 172.21.15.17:0/3098986015 shutdown_connections 2024-03-06T07:26:04.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.682+0000 7f3738f2b640 1 -- 172.21.15.17:0/3098986015 wait complete. 2024-03-06T07:26:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.684+0000 7f3738f2b640 1 Processor -- start 2024-03-06T07:26:04.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.684+0000 7f3738f2b640 1 -- start start 2024-03-06T07:26:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2190 0x7f373414be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 0x7f37341df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37340a7100 con 0x7f3734152920 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37340c0d60 con 0x7f37340c2620 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.685+0000 7f3738f2b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37340a7280 con 0x7f37340c2190 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39624/0 (socket says 172.21.15.17:39624) 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 -- 172.21.15.17:0/3931540972 learned_addr learned my addr 172.21.15.17:0/3931540972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3729d9b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 0x7f37341df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f373259c640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2190 0x7f373414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 -- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2190 msgr2=0x7f373414be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2190 0x7f373414be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 -- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 msgr2=0x7f37341df6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 0x7f37341df6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 -- 172.21.15.17:0/3931540972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373414c3d0 con 0x7f3734152920 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3729d9b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 0x7f37341df6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3732d9d640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f372c009f00 tx=0x7f372c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c018030 con 0x7f3734152920 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c00ce50 con 0x7f3734152920 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c003bb0 con 0x7f3734152920 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37341e0ae0 con 0x7f3734152920 2024-03-06T07:26:04.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.686+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37341e10a0 con 0x7f3734152920 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.688+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373404f330 con 0x7f3734152920 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.688+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f372c005d10 con 0x7f3734152920 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.688+0000 7f372b7fe640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 0x7f371406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.688+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f372c08b0a0 con 0x7f3734152920 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.689+0000 7f373259c640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 0x7f371406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.689+0000 7f373259c640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 0x7f371406d9c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3724036000 tx=0x7f3724008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:04.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.693+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f372c022070 con 0x7f3734152920 2024-03-06T07:26:04.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.874+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3734068460 con 0x7f3734152920 2024-03-06T07:26:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.876+0000 7f372b7fe640 1 -- 172.21.15.17:0/3931540972 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+20615 (secure 0 0 0) 0x7f373414c3d0 con 0x7f3734152920 2024-03-06T07:26:04.878 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:04.878 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":153,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:03.653966+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:04.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 msgr2=0x7f371406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 0x7f371406d9c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3724036000 tx=0x7f3724008030 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 msgr2=0x7f37341ee8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f372c009f00 tx=0x7f372c00c490 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 shutdown_connections 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f371406b500 0x7f371406d9c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3734152920 0x7f37341ee8e0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37340c2620 0x7f37341df6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 --2- 172.21.15.17:0/3931540972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37340c2190 0x7f373414be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 >> 172.21.15.17:0/3931540972 conn(0x7f37340c4290 msgr2=0x7f373414da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 shutdown_connections 2024-03-06T07:26:04.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.879+0000 7f3738f2b640 1 -- 172.21.15.17:0/3931540972 wait complete. 2024-03-06T07:26:04.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 standby_count_wanted 0 2024-03-06T07:26:04.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 Processor -- start 2024-03-06T07:26:04.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 -- start start 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 0x7f965c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0b0450 con 0x7f965c0c5380 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c0c77f0 con 0x7f965c06ca70 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f966282a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c06c020 con 0x7f965c0b0070 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f9660dc7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f965c0c5380 0x7f965c14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57286/0 (socket says 172.21.15.17:57286) 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.995+0000 7f9660dc7640 1 -- 172.21.15.17:0/3399567508 learned_addr learned my addr 172.21.15.17:0/3399567508 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965bfff640 1 --2- 172.21.15.17:0/3399567508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 0x7f965c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797524252 0 0) 0x7f9654001490 con 0x7f965c0c5380 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9644003520 con 0x7f965c0c5380 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965bfff640 1 -- 172.21.15.17:0/3399567508 >> v1:172.21.15.163:6789/0 conn(0x7f965c06ca70 legacy=0x7f965c0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154653655 0 0) 0x7f965c0c77f0 con 0x7f965c06ca70 2024-03-06T07:26:04.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965bfff640 1 -- 172.21.15.17:0/3399567508 >> v1:172.21.15.17:6789/0 conn(0x7f965c0c5380 legacy=0x7f965c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:04.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.996+0000 7f965bfff640 1 -- 172.21.15.17:0/3399567508 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c0c7ad0 con 0x7f965c0b0070 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f965bfff640 1 --2- 172.21.15.17:0/3399567508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 0x7f965c155c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097d0 tx=0x7f964c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d091b818db5768a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c035020 con 0x7f965c0b0070 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c031800 con 0x7f965c0b0070 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f965affd640 1 -- 172.21.15.17:0/3399567508 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c031bb0 con 0x7f965c0b0070 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f966282a640 1 -- 172.21.15.17:0/3399567508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 msgr2=0x7f965c155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.997+0000 7f966282a640 1 --2- 172.21.15.17:0/3399567508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 0x7f965c155c30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097d0 tx=0x7f964c02b8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.998+0000 7f966282a640 1 -- 172.21.15.17:0/3399567508 shutdown_connections 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.998+0000 7f966282a640 1 --2- 172.21.15.17:0/3399567508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0b0070 0x7f965c155c30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.998+0000 7f966282a640 1 -- 172.21.15.17:0/3399567508 >> 172.21.15.17:0/3399567508 conn(0x7f965c0ac590 msgr2=0x7f965c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.998+0000 7f966282a640 1 -- 172.21.15.17:0/3399567508 shutdown_connections 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.998+0000 7f966282a640 1 -- 172.21.15.17:0/3399567508 wait complete. 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 Processor -- start 2024-03-06T07:26:05.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 -- start start 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 0x7f965c1eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 0x7f965c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c06cef0 con 0x7f965c0b0070 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c06c550 con 0x7f965c06ca70 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:04.999+0000 7f966282a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c06da80 con 0x7f965c0c5380 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39630/0 (socket says 172.21.15.17:39630) 2024-03-06T07:26:05.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 -- 172.21.15.17:0/2445645294 learned_addr learned my addr 172.21.15.17:0/2445645294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f9660dc7640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 0x7f965c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965bfff640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 0x7f965c1eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 -- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 msgr2=0x7f965c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 0x7f965c154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 -- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 msgr2=0x7f965c1eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 0x7f965c1eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 -- 172.21.15.17:0/2445645294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c1596d0 con 0x7f965c0b0070 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f9660dc7640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 0x7f965c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965bfff640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 0x7f965c1eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f965b7fe640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f965000c4b0 tx=0x7f965000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650017070 con 0x7f965c0b0070 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650005480 con 0x7f965c0b0070 2024-03-06T07:26:05.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650003830 con 0x7f965c0b0070 2024-03-06T07:26:05.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c1593f0 con 0x7f965c0b0070 2024-03-06T07:26:05.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.000+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c1ead30 con 0x7f965c0b0070 2024-03-06T07:26:05.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.001+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c04f330 con 0x7f965c0b0070 2024-03-06T07:26:05.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.002+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f965001a070 con 0x7f965c0b0070 2024-03-06T07:26:05.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.002+0000 7f9658ff9640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:05.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.002+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f96500899d0 con 0x7f965c0b0070 2024-03-06T07:26:05.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.002+0000 7f965bfff640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:05.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.003+0000 7f965bfff640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 0x7f964806d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f964c036000 tx=0x7f964c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:05.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.006+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9650058400 con 0x7f965c0b0070 2024-03-06T07:26:05.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:05.225+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f965c06d860 con 0x7f965c0b0070 2024-03-06T07:26:05.446 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:26:05.445+0000 7eff03eaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.237+0000 7f9658ff9640 1 -- 172.21.15.17:0/2445645294 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "standby_count_wanted", "val": "0"}]=0 v140) v1 ==== 119+0+0 (secure 0 0 0) 0x7f9650057d00 con 0x7f965c0b0070 2024-03-06T07:26:06.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 msgr2=0x7f964806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 0x7f964806d9c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f964c036000 tx=0x7f964c008030 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 msgr2=0x7f965c158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f965000c4b0 tx=0x7f965000c970 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 shutdown_connections 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f964806b500 0x7f964806d9c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f965c0c5380 0x7f965c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f965c0b0070 0x7f965c158d80 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 --2- 172.21.15.17:0/2445645294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f965c06ca70 0x7f965c1eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 >> 172.21.15.17:0/2445645294 conn(0x7f965c0ac590 msgr2=0x7f965c0c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 shutdown_connections 2024-03-06T07:26:06.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.239+0000 7f966282a640 1 -- 172.21.15.17:0/2445645294 wait complete. 2024-03-06T07:26:06.252 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:26:06.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs2 allow_standby_replay false 2024-03-06T07:26:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.359+0000 7f3fe6bd7640 1 Processor -- start 2024-03-06T07:26:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.360+0000 7f3fe6bd7640 1 -- start start 2024-03-06T07:26:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe6bd7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fe00b0070 0x7f3fe0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe6bd7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00b0450 con 0x7f3fe00c5380 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe6bd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe006cf20 con 0x7f3fe006ca70 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe6bd7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe006c020 con 0x7f3fe00b0070 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe4973640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3fe00b0070 0x7f3fe0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe5174640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3fe00c5380 0x7f3fe014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57288/0 (socket says 172.21.15.17:57288) 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe5174640 1 -- 172.21.15.17:0/147182864 learned_addr learned my addr 172.21.15.17:0/147182864 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe4973640 1 -- 172.21.15.17:0/147182864 >> v1:172.21.15.163:6789/0 conn(0x7f3fe006ca70 legacy=0x7f3fe00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe4973640 1 -- 172.21.15.17:0/147182864 >> v1:172.21.15.17:6789/0 conn(0x7f3fe00c5380 legacy=0x7f3fe014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fdf7fe640 1 -- 172.21.15.17:0/147182864 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752773991 0 0) 0x7f3fd8001490 con 0x7f3fe00c5380 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.361+0000 7f3fe4973640 1 -- 172.21.15.17:0/147182864 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00c77f0 con 0x7f3fe00b0070 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.362+0000 7f3fe4973640 1 --2- 172.21.15.17:0/147182864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe00b0070 0x7f3fe0155c30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370ed15bff6b545e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.362+0000 7f3fdf7fe640 1 -- 172.21.15.17:0/147182864 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40315a0 con 0x7f3fe00b0070 2024-03-06T07:26:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.362+0000 7f3fdf7fe640 1 -- 172.21.15.17:0/147182864 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4037070 con 0x7f3fe00b0070 2024-03-06T07:26:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.362+0000 7f3fdf7fe640 1 -- 172.21.15.17:0/147182864 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4031960 con 0x7f3fe00b0070 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/147182864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe00b0070 msgr2=0x7f3fe0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/147182864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe00b0070 0x7f3fe0155c30 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/147182864 shutdown_connections 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/147182864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe00b0070 0x7f3fe0155c30 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/147182864 >> 172.21.15.17:0/147182864 conn(0x7f3fe00ac590 msgr2=0x7f3fe00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/147182864 shutdown_connections 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.363+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/147182864 wait complete. 2024-03-06T07:26:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.364+0000 7f3fe6bd7640 1 Processor -- start 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.364+0000 7f3fe6bd7640 1 -- start start 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 0x7f3fe0154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00c5380 0x7f3fe00c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006c550 con 0x7f3fe006ca70 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006c270 con 0x7f3fe00c5380 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe6bd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006da80 con 0x7f3fe00b0070 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39646/0 (socket says 172.21.15.17:39646) 2024-03-06T07:26:06.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 -- 172.21.15.17:0/1805337590 learned_addr learned my addr 172.21.15.17:0/1805337590 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe5174640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00c5380 0x7f3fe00c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fdffff640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 0x7f3fe0154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 -- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 msgr2=0x7f3fe0154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 0x7f3fe0154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 -- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00c5380 msgr2=0x7f3fe00c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00c5380 0x7f3fe00c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 -- 172.21.15.17:0/1805337590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe01576c0 con 0x7f3fe006ca70 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fdffff640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 0x7f3fe0154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.365+0000 7f3fe4973640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40098c0 tx=0x7f3fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.366+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4032840 con 0x7f3fe006ca70 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.366+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd40329e0 con 0x7f3fe006ca70 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.366+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe01573e0 con 0x7f3fe006ca70 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.366+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4032dd0 con 0x7f3fe006ca70 2024-03-06T07:26:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.366+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01ee310 con 0x7f3fe006ca70 2024-03-06T07:26:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.367+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe004f330 con 0x7f3fe006ca70 2024-03-06T07:26:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.368+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fd404c020 con 0x7f3fe006ca70 2024-03-06T07:26:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.368+0000 7f3fdd7fa640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.368+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f3fd402f020 con 0x7f3fe006ca70 2024-03-06T07:26:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.368+0000 7f3fdffff640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.369+0000 7f3fdffff640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 0x7f3fc406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3fe01eef50 tx=0x7f3fcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:06.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.372+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fd4040070 con 0x7f3fe006ca70 2024-03-06T07:26:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:06.595+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3fe006d860 con 0x7f3fe006ca70 2024-03-06T07:26:07.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.245+0000 7f3fdd7fa640 1 -- 172.21.15.17:0/1805337590 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs2", "var": "allow_standby_replay", "val": "false"}]=0 v141) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3fd407ade0 con 0x7f3fe006ca70 2024-03-06T07:26:07.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.247+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 msgr2=0x7f3fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.247+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 0x7f3fc406d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3fe01eef50 tx=0x7f3fcc008040 comp rx=0 tx=0).stop 2024-03-06T07:26:07.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.247+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 msgr2=0x7f3fe0156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.247+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40098c0 tx=0x7f3fd40046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 shutdown_connections 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3fc406b4d0 0x7f3fc406d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3fe00c5380 0x7f3fe00c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fe00b0070 0x7f3fe0154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 --2- 172.21.15.17:0/1805337590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fe006ca70 0x7f3fe0156ce0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 >> 172.21.15.17:0/1805337590 conn(0x7f3fe00ac590 msgr2=0x7f3fe01512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 shutdown_connections 2024-03-06T07:26:07.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.248+0000 7f3fe6bd7640 1 -- 172.21.15.17:0/1805337590 wait complete. 2024-03-06T07:26:07.260 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs2' 2024-03-06T07:26:07.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:07.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.360+0000 7f3df1725640 1 Processor -- start 2024-03-06T07:26:07.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.360+0000 7f3df1725640 1 -- start start 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3df1725640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3dec0c3150 0x7f3dec0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3df1725640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dec0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3df1725640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec072440 con 0x7f3dec14f650 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3df1725640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dec0c2b10 con 0x7f3dec0c3150 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3df1725640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec152140 con 0x7f3dec152c20 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3deaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3dec14f650 0x7f3dec071880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57298/0 (socket says 172.21.15.17:57298) 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.361+0000 7f3deaffd640 1 -- 172.21.15.17:0/588850959 learned_addr learned my addr 172.21.15.17:0/588850959 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643211653 0 0) 0x7f3de0001110 con 0x7f3dec14f650 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd4003520 con 0x7f3dec14f650 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/588850959 >> v2:172.21.15.163:3300/0 conn(0x7f3dec0c3150 0x7f3dec0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dec0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 376164608 0 0) 0x7f3dd4003520 con 0x7f3dec14f650 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 msgr2=0x7f3dec0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dec0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 >> v2:172.21.15.163:3300/0 conn(0x7f3dec0c3150 msgr2=0x7f3dec0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 --2- 172.21.15.17:0/588850959 >> v2:172.21.15.163:3300/0 conn(0x7f3dec0c3150 0x7f3dec0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec152420 con 0x7f3dec14f650 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3de0001840 con 0x7f3dec14f650 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 >> v1:172.21.15.17:6789/0 conn(0x7f3dec14f650 legacy=0x7f3dec071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd4004d40 0x7f3dd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0c2b10 con 0x7f3dd4004d40 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec152140 con 0x7f3dd40039b0 2024-03-06T07:26:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd400baa0 con 0x7f3dec152c20 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd4004d40 0x7f3dd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deaffd640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 -- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 msgr2=0x7f3dd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 -- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd40039b0 msgr2=0x7f3dd4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deb7fe640 1 -- 172.21.15.17:0/588850959 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4003520 con 0x7f3dd4004d40 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec152c20 0x7f3dd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.362+0000 7f3deaffd640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd40039b0 0x7f3dd4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.363+0000 7f3deb7fe640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd4004d40 0x7f3dd40051c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3de4007330 tx=0x7f3dd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56a3082a7878416 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.363+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4039070 con 0x7f3dd4004d40 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.363+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de402cc90 con 0x7f3dd4004d40 2024-03-06T07:26:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.363+0000 7f3de9ffb640 1 -- 172.21.15.17:0/588850959 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4009be0 con 0x7f3dd4004d40 2024-03-06T07:26:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 -- 172.21.15.17:0/588850959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd4004d40 msgr2=0x7f3dd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 --2- 172.21.15.17:0/588850959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dd4004d40 0x7f3dd40051c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3de4007330 tx=0x7f3dd4005830 comp rx=0 tx=0).stop 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3deb7fe640 1 -- 172.21.15.17:0/588850959 reap_dead start 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 -- 172.21.15.17:0/588850959 shutdown_connections 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 -- 172.21.15.17:0/588850959 >> 172.21.15.17:0/588850959 conn(0x7f3dec0c4290 msgr2=0x7f3dec0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 -- 172.21.15.17:0/588850959 shutdown_connections 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.364+0000 7f3df1725640 1 -- 172.21.15.17:0/588850959 wait complete. 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.365+0000 7f3df1725640 1 Processor -- start 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.365+0000 7f3df1725640 1 -- start start 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd4004d40 0x7f3dec14b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec1eb210 0x7f3dec1eb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec072110 con 0x7f3dec1488a0 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec071f90 con 0x7f3dd4004d40 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3df1725640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0c3f40 con 0x7f3dec1eb210 2024-03-06T07:26:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39664/0 (socket says 172.21.15.17:39664) 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 -- 172.21.15.17:0/46331515 learned_addr learned my addr 172.21.15.17:0/46331515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 -- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec1eb210 msgr2=0x7f3dec1eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec1eb210 0x7f3dec1eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 -- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd4004d40 msgr2=0x7f3dec14b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd4004d40 0x7f3dec14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 -- 172.21.15.17:0/46331515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec1ebbd0 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.366+0000 7f3dea7fc640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00f4e0 tx=0x7f3ddc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.367+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc018020 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.367+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc014870 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.367+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec14c070 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.367+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc014c40 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.367+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec14c630 con 0x7f3dec1488a0 2024-03-06T07:26:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.368+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec04f330 con 0x7f3dec1488a0 2024-03-06T07:26:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.368+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ddc01b020 con 0x7f3dec1488a0 2024-03-06T07:26:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.369+0000 7f3dd37fe640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 0x7f3dd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.369+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f3ddc08aad0 con 0x7f3dec1488a0 2024-03-06T07:26:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.369+0000 7f3deaffd640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 0x7f3dd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.370+0000 7f3deaffd640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 0x7f3dd806d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3de0000f60 tx=0x7f3de0000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.373+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ddc022020 con 0x7f3dec1488a0 2024-03-06T07:26:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.625+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3dec068460 con 0x7f3dec1488a0 2024-03-06T07:26:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.625+0000 7f3dd37fe640 1 -- 172.21.15.17:0/46331515 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+6266 (secure 0 0 0) 0x7f3dec1ebbd0 con 0x7f3dec1488a0 2024-03-06T07:26:07.627 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:07.627 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":141,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:25:56.622881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8781":{"gid":8781,"name":"c","rank":0,"incarnation":134,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2429596225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2429596225},{"type":"v1","addr":"172.21.15.17:6835","nonce":2429596225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8781,"qdb_cluster":[8781]},"id":11},{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:06.596155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8778":{"gid":8778,"name":"a","rank":0,"incarnation":137,"state":"up:active","state_seq":5,"addr":"172.21.15.17:6834/81289984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":81289984},{"type":"v1","addr":"172.21.15.17:6834","nonce":81289984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8778,"qdb_cluster":[8778]},"id":12}]} 2024-03-06T07:26:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 141 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 msgr2=0x7f3dd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 0x7f3dd806d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3de0000f60 tx=0x7f3de0000ed0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 msgr2=0x7f3dec14a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00f4e0 tx=0x7f3ddc004610 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 shutdown_connections 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3dd806b500 0x7f3dd806d9c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.627+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dec1eb210 0x7f3dec1eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.628+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3dec1488a0 0x7f3dec14a3c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.628+0000 7f3df1725640 1 --2- 172.21.15.17:0/46331515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dd4004d40 0x7f3dec14b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.628+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 >> 172.21.15.17:0/46331515 conn(0x7f3dec0c4290 msgr2=0x7f3dec0c5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.628+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 shutdown_connections 2024-03-06T07:26:07.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.628+0000 7f3df1725640 1 -- 172.21.15.17:0/46331515 wait complete. 2024-03-06T07:26:07.641 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:07.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.744+0000 7f898bceb640 1 Processor -- start 2024-03-06T07:26:07.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.744+0000 7f898bceb640 1 -- start start 2024-03-06T07:26:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898bceb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89840c5860 0x7f89841609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898bceb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89841591c0 con 0x7f89840c5480 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898bceb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89841594a0 con 0x7f8984155ba0 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898bceb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898406df40 con 0x7f89840c5860 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8989a87640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f89840c5480 0x7f898406f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57302/0 (socket says 172.21.15.17:57302) 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8989a87640 1 -- 172.21.15.17:0/3334691761 learned_addr learned my addr 172.21.15.17:0/3334691761 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898a288640 1 --2- 172.21.15.17:0/3334691761 >> v2:172.21.15.163:3300/0 conn(0x7f89840c5860 0x7f89841609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612041905 0 0) 0x7f89841591c0 con 0x7f89840c5480 2024-03-06T07:26:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8970003520 con 0x7f89840c5480 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636653274 0 0) 0x7f89841594a0 con 0x7f8984155ba0 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89841591c0 con 0x7f8984155ba0 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800883849 0 0) 0x7f8970003520 con 0x7f89840c5480 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 >> v2:172.21.15.163:3300/0 conn(0x7f89840c5860 msgr2=0x7f89841609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 --2- 172.21.15.17:0/3334691761 >> v2:172.21.15.163:3300/0 conn(0x7f89840c5860 0x7f89841609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 >> v1:172.21.15.163:6789/0 conn(0x7f8984155ba0 legacy=0x7f89841586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89841553a0 con 0x7f89840c5480 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f8980001670 con 0x7f89840c5480 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 >> v1:172.21.15.17:6789/0 conn(0x7f89840c5480 legacy=0x7f898406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 0x7f8970003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970004800 0x7f8970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89700051c0 0x7f897000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898406df40 con 0x7f8970003b40 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897000c0f0 con 0x7f8970004800 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897000c2f0 con 0x7f89700051c0 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f898a288640 1 --2- 172.21.15.17:0/3334691761 >> v2:172.21.15.163:3300/0 conn(0x7f89840c5860 0x7f89841609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.745+0000 7f8989a87640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 0x7f8970003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989286640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970004800 0x7f8970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 -- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89700051c0 msgr2=0x7f897000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f898a288640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89700051c0 0x7f897000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89700051c0 0x7f897000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 -- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970004800 msgr2=0x7f8970004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970004800 0x7f8970004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 -- 172.21.15.17:0/3334691761 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970003520 con 0x7f8970003b40 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f898a288640 1 -- 172.21.15.17:0/3334691761 reap_dead start 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989286640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970004800 0x7f8970004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.746+0000 7f8989a87640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 0x7f8970003f40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f898000c190 tx=0x7f8970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bf7acb99337196 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980007cc0 con 0x7f8970003b40 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980030070 con 0x7f8970003b40 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f8988a85640 1 -- 172.21.15.17:0/3334691761 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980005d70 con 0x7f8970003b40 2024-03-06T07:26:07.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 -- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 msgr2=0x7f8970003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 0x7f8970003f40 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f898000c190 tx=0x7f8970005830 comp rx=0 tx=0).stop 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 -- 172.21.15.17:0/3334691761 shutdown_connections 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 --2- 172.21.15.17:0/3334691761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8970003b40 0x7f8970003f40 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 -- 172.21.15.17:0/3334691761 >> 172.21.15.17:0/3334691761 conn(0x7f898406b1a0 msgr2=0x7f898406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 -- 172.21.15.17:0/3334691761 shutdown_connections 2024-03-06T07:26:07.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.747+0000 7f898bceb640 1 -- 172.21.15.17:0/3334691761 wait complete. 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.749+0000 7f898bceb640 1 Processor -- start 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.749+0000 7f898bceb640 1 -- start start 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 0x7f89840ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 0x7f89840a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89840c5cf0 con 0x7f89840a8230 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898406fdb0 con 0x7f8970003b40 2024-03-06T07:26:07.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898bceb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8984156000 con 0x7f89840ad890 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898a288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898a288640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39690/0 (socket says 172.21.15.17:39690) 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.750+0000 7f898a288640 1 -- 172.21.15.17:0/4290721114 learned_addr learned my addr 172.21.15.17:0/4290721114 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f8989a87640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 0x7f89840ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f8989286640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 0x7f89840a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 -- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 msgr2=0x7f89840a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 0x7f89840a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 -- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 msgr2=0x7f89840ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 0x7f89840ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 -- 172.21.15.17:0/4290721114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89841eb9d0 con 0x7f89840a8230 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f8989286640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 0x7f89840a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f8989a87640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 0x7f89840ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898a288640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f89840c2360 tx=0x7f897c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c016620 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c01b070 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c016a00 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89841de830 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.751+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89840acd60 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.752+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898404f330 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.753+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f897c016ba0 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.753+0000 7f897a7fc640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.753+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f897c08b530 con 0x7f89840a8230 2024-03-06T07:26:07.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.753+0000 7f8989a87640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.754+0000 7f8989a87640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8980034000 tx=0x7f8980039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.758+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f897c059f60 con 0x7f89840a8230 2024-03-06T07:26:07.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.938+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f898406e0c0 con 0x7f89840a8230 2024-03-06T07:26:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.940+0000 7f897a7fc640 1 -- 172.21.15.17:0/4290721114 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+20602 (secure 0 0 0) 0x7f897c059860 con 0x7f89840a8230 2024-03-06T07:26:07.942 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:07.942 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":154,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:05.224053+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 msgr2=0x7f896806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8980034000 tx=0x7f8980039040 comp rx=0 tx=0).stop 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 msgr2=0x7f89841ee1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f89840c2360 tx=0x7f897c00f520 comp rx=0 tx=0).stop 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 shutdown_connections 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89840a8230 0x7f89841ee1e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89840ad890 0x7f89840a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 --2- 172.21.15.17:0/4290721114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8970003b40 0x7f89840ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:07.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 >> 172.21.15.17:0/4290721114 conn(0x7f898406b1a0 msgr2=0x7f898406ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:07.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 shutdown_connections 2024-03-06T07:26:07.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:07.942+0000 7f898bceb640 1 -- 172.21.15.17:0/4290721114 wait complete. 2024-03-06T07:26:07.955 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:26:07.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.090+0000 7f4304932640 1 Processor -- start 2024-03-06T07:26:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.090+0000 7f4304932640 1 -- start start 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f4304932640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4300156ea0 0x7f43000c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f4304932640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43000ad310 con 0x7f43001506b0 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f4304932640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43001572a0 con 0x7f4300153990 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f4304932640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4300156450 con 0x7f4300156ea0 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fe59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f43001506b0 0x7f43000702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57304/0 (socket says 172.21.15.17:57304) 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fe59c640 1 -- 172.21.15.17:0/393083383 learned_addr learned my addr 172.21.15.17:0/393083383 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fed9d640 1 --2- 172.21.15.17:0/393083383 >> v2:172.21.15.163:3300/0 conn(0x7f4300156ea0 0x7f43000c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280770421 0 0) 0x7f43000ad310 con 0x7f43001506b0 2024-03-06T07:26:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e8003520 con 0x7f43001506b0 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090328929 0 0) 0x7f43001572a0 con 0x7f4300153990 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43000ad310 con 0x7f4300153990 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232101672 0 0) 0x7f42e8003520 con 0x7f43001506b0 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 >> v2:172.21.15.163:3300/0 conn(0x7f4300156ea0 msgr2=0x7f43000c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 --2- 172.21.15.17:0/393083383 >> v2:172.21.15.163:3300/0 conn(0x7f4300156ea0 0x7f43000c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.091+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 >> v1:172.21.15.163:6789/0 conn(0x7f4300153990 legacy=0x7f43000acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4300070aa0 con 0x7f43001506b0 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f42f00015f0 con 0x7f43001506b0 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fed9d640 1 --2- 172.21.15.17:0/393083383 >> v2:172.21.15.163:3300/0 conn(0x7f4300156ea0 0x7f43000c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 >> v1:172.21.15.17:6789/0 conn(0x7f43001506b0 legacy=0x7f43000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8003b90 0x7f42e8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300156450 con 0x7f42e8004800 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e800c090 con 0x7f42e80051c0 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e800c290 con 0x7f42e8003b90 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fe59c640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8003b90 0x7f42e8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fed9d640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 -- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8003b90 msgr2=0x7f42e8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8003b90 0x7f42e8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 -- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e80051c0 msgr2=0x7f42e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e80051c0 0x7f42e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fed9d640 1 -- 172.21.15.17:0/393083383 reap_dead start 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.092+0000 7f42fdd9b640 1 -- 172.21.15.17:0/393083383 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e8003520 con 0x7f42e8004800 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.093+0000 7f42fdd9b640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f42f40018d0 tx=0x7f42e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff6880d64e3a56f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.093+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4037070 con 0x7f42e8004800 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.093+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f40052d0 con 0x7f42e8004800 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.093+0000 7f42fd59a640 1 -- 172.21.15.17:0/393083383 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40014f0 con 0x7f42e8004800 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 -- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 msgr2=0x7f42e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f42f40018d0 tx=0x7f42e8005830 comp rx=0 tx=0).stop 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 -- 172.21.15.17:0/393083383 shutdown_connections 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 --2- 172.21.15.17:0/393083383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e8004800 0x7f42e8004c80 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 -- 172.21.15.17:0/393083383 >> 172.21.15.17:0/393083383 conn(0x7f430006b1a0 msgr2=0x7f430006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 -- 172.21.15.17:0/393083383 shutdown_connections 2024-03-06T07:26:08.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.094+0000 7f4304932640 1 -- 172.21.15.17:0/393083383 wait complete. 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.095+0000 7f4304932640 1 Processor -- start 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.095+0000 7f4304932640 1 -- start start 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8004800 0x7f43001eb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43000a8350 0x7f43001494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43001566a0 con 0x7f43000c2fc0 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300156a10 con 0x7f43000a8350 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f4304932640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000702c0 con 0x7f42e8004800 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39702/0 (socket says 172.21.15.17:39702) 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 -- 172.21.15.17:0/3663322070 learned_addr learned my addr 172.21.15.17:0/3663322070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:08.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fe59c640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8004800 0x7f43001eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fdd9b640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43000a8350 0x7f43001494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 -- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8004800 msgr2=0x7f43001eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8004800 0x7f43001eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 -- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43000a8350 msgr2=0x7f43001494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43000a8350 0x7f43001494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.096+0000 7f42fed9d640 1 -- 172.21.15.17:0/3663322070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430006c690 con 0x7f43000c2fc0 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f42fed9d640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f43000c6270 tx=0x7f42f800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800fd80 con 0x7f43000c2fc0 2024-03-06T07:26:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f8003990 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f8003d80 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43000c49b0 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.097+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001499f0 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.098+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42f8004020 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.098+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430004f330 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.099+0000 7f42eeffd640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 0x7f42e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.099+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f42f808b190 con 0x7f43000c2fc0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.099+0000 7f42fe59c640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 0x7f42e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.099+0000 7f42fe59c640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 0x7f42e006d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0001cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.103+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f42f8019030 con 0x7f43000c2fc0 2024-03-06T07:26:08.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.303+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43000c2c80 con 0x7f42e006b500 2024-03-06T07:26:08.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.315+0000 7f42eeffd640 1 -- 172.21.15.17:0/3663322070 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+491533 (secure 0 0 0) 0x7f42f00070c0 con 0x7f42e006b500 2024-03-06T07:26:08.322 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:08.326 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 msgr2=0x7f42e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 0x7f42e006d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f0001cd0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 msgr2=0x7f43000c3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f43000c6270 tx=0x7f42f800f580 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 shutdown_connections 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42e006b500 0x7f42e006d9c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43000c2fc0 0x7f43000c3450 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43000a8350 0x7f43001494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 --2- 172.21.15.17:0/3663322070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e8004800 0x7f43001eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 >> 172.21.15.17:0/3663322070 conn(0x7f430006b1a0 msgr2=0x7f43001522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 shutdown_connections 2024-03-06T07:26:08.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.319+0000 7f4304932640 1 -- 172.21.15.17:0/3663322070 wait complete. 2024-03-06T07:26:08.334 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-06T07:26:07.421246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":357,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264860,"kb_used_data":15392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709860,"statfs":{"total":773094113280,"available":772822896640,"internally_reserved":0,"allocated":15761408,"data_stored":8745160,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.212984"},"pg_stats":[{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239997+0000","last_change":"2024-03-06T07:26:04.229926+0000","last_active":"2024-03-06T07:26:05.239997+0000","last_peered":"2024-03-06T07:26:05.239997+0000","last_clean":"2024-03-06T07:26:05.239997+0000","last_became_active":"2024-03-06T07:26:04.229626+0000","last_became_peered":"2024-03-06T07:26:04.229626+0000","last_unstale":"2024-03-06T07:26:05.239997+0000","last_undegraded":"2024-03-06T07:26:05.239997+0000","last_fullsized":"2024-03-06T07:26:05.239997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:34.836516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238917+0000","last_change":"2024-03-06T07:25:55.171050+0000","last_active":"2024-03-06T07:26:05.238917+0000","last_peered":"2024-03-06T07:26:05.238917+0000","last_clean":"2024-03-06T07:26:05.238917+0000","last_became_active":"2024-03-06T07:25:55.170737+0000","last_became_peered":"2024-03-06T07:25:55.170737+0000","last_unstale":"2024-03-06T07:26:05.238917+0000","last_undegraded":"2024-03-06T07:26:05.238917+0000","last_fullsized":"2024-03-06T07:26:05.238917+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234080+0000","last_change":"2024-03-06T07:26:04.232419+0000","last_active":"2024-03-06T07:26:05.234080+0000","last_peered":"2024-03-06T07:26:05.234080+0000","last_clean":"2024-03-06T07:26:05.234080+0000","last_became_active":"2024-03-06T07:26:04.230470+0000","last_became_peered":"2024-03-06T07:26:04.230470+0000","last_unstale":"2024-03-06T07:26:05.234080+0000","last_undegraded":"2024-03-06T07:26:05.234080+0000","last_fullsized":"2024-03-06T07:26:05.234080+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:34.455713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736021+0000","last_change":"2024-03-06T07:25:55.179421+0000","last_active":"2024-03-06T07:26:05.736021+0000","last_peered":"2024-03-06T07:26:05.736021+0000","last_clean":"2024-03-06T07:26:05.736021+0000","last_became_active":"2024-03-06T07:25:55.179145+0000","last_became_peered":"2024-03-06T07:25:55.179145+0000","last_unstale":"2024-03-06T07:26:05.736021+0000","last_undegraded":"2024-03-06T07:26:05.736021+0000","last_fullsized":"2024-03-06T07:26:05.736021+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736957+0000","last_change":"2024-03-06T07:26:04.234996+0000","last_active":"2024-03-06T07:26:05.736957+0000","last_peered":"2024-03-06T07:26:05.736957+0000","last_clean":"2024-03-06T07:26:05.736957+0000","last_became_active":"2024-03-06T07:26:04.234723+0000","last_became_peered":"2024-03-06T07:26:04.234723+0000","last_unstale":"2024-03-06T07:26:05.736957+0000","last_undegraded":"2024-03-06T07:26:05.736957+0000","last_fullsized":"2024-03-06T07:26:05.736957+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:16.379024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240371+0000","last_change":"2024-03-06T07:25:55.178609+0000","last_active":"2024-03-06T07:26:05.240371+0000","last_peered":"2024-03-06T07:26:05.240371+0000","last_clean":"2024-03-06T07:26:05.240371+0000","last_became_active":"2024-03-06T07:25:55.178334+0000","last_became_peered":"2024-03-06T07:25:55.178334+0000","last_unstale":"2024-03-06T07:26:05.240371+0000","last_undegraded":"2024-03-06T07:26:05.240371+0000","last_fullsized":"2024-03-06T07:26:05.240371+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:19.444548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737016+0000","last_change":"2024-03-06T07:26:04.237958+0000","last_active":"2024-03-06T07:26:05.737016+0000","last_peered":"2024-03-06T07:26:05.737016+0000","last_clean":"2024-03-06T07:26:05.737016+0000","last_became_active":"2024-03-06T07:26:04.237445+0000","last_became_peered":"2024-03-06T07:26:04.237445+0000","last_unstale":"2024-03-06T07:26:05.737016+0000","last_undegraded":"2024-03-06T07:26:05.737016+0000","last_fullsized":"2024-03-06T07:26:05.737016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:47.243949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239188+0000","last_change":"2024-03-06T07:25:55.181844+0000","last_active":"2024-03-06T07:26:05.239188+0000","last_peered":"2024-03-06T07:26:05.239188+0000","last_clean":"2024-03-06T07:26:05.239188+0000","last_became_active":"2024-03-06T07:25:55.181282+0000","last_became_peered":"2024-03-06T07:25:55.181282+0000","last_unstale":"2024-03-06T07:26:05.239188+0000","last_undegraded":"2024-03-06T07:26:05.239188+0000","last_fullsized":"2024-03-06T07:26:05.239188+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:58.454621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735040+0000","last_change":"2024-03-06T07:26:04.229646+0000","last_active":"2024-03-06T07:26:05.735040+0000","last_peered":"2024-03-06T07:26:05.735040+0000","last_clean":"2024-03-06T07:26:05.735040+0000","last_became_active":"2024-03-06T07:26:04.229225+0000","last_became_peered":"2024-03-06T07:26:04.229225+0000","last_unstale":"2024-03-06T07:26:05.735040+0000","last_undegraded":"2024-03-06T07:26:05.735040+0000","last_fullsized":"2024-03-06T07:26:05.735040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:17.339996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233491+0000","last_change":"2024-03-06T07:25:55.188179+0000","last_active":"2024-03-06T07:26:05.233491+0000","last_peered":"2024-03-06T07:26:05.233491+0000","last_clean":"2024-03-06T07:26:05.233491+0000","last_became_active":"2024-03-06T07:25:55.187905+0000","last_became_peered":"2024-03-06T07:25:55.187905+0000","last_unstale":"2024-03-06T07:26:05.233491+0000","last_undegraded":"2024-03-06T07:26:05.233491+0000","last_fullsized":"2024-03-06T07:26:05.233491+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:31.959459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734882+0000","last_change":"2024-03-06T07:26:04.233412+0000","last_active":"2024-03-06T07:26:05.734882+0000","last_peered":"2024-03-06T07:26:05.734882+0000","last_clean":"2024-03-06T07:26:05.734882+0000","last_became_active":"2024-03-06T07:26:04.232888+0000","last_became_peered":"2024-03-06T07:26:04.232888+0000","last_unstale":"2024-03-06T07:26:05.734882+0000","last_undegraded":"2024-03-06T07:26:05.734882+0000","last_fullsized":"2024-03-06T07:26:05.734882+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:54.282221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734702+0000","last_change":"2024-03-06T07:25:55.179259+0000","last_active":"2024-03-06T07:26:05.734702+0000","last_peered":"2024-03-06T07:26:05.734702+0000","last_clean":"2024-03-06T07:26:05.734702+0000","last_became_active":"2024-03-06T07:25:55.178971+0000","last_became_peered":"2024-03-06T07:25:55.178971+0000","last_unstale":"2024-03-06T07:26:05.734702+0000","last_undegraded":"2024-03-06T07:26:05.734702+0000","last_fullsized":"2024-03-06T07:26:05.734702+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:52.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734671+0000","last_change":"2024-03-06T07:26:04.231613+0000","last_active":"2024-03-06T07:26:05.734671+0000","last_peered":"2024-03-06T07:26:05.734671+0000","last_clean":"2024-03-06T07:26:05.734671+0000","last_became_active":"2024-03-06T07:26:04.231368+0000","last_became_peered":"2024-03-06T07:26:04.231368+0000","last_unstale":"2024-03-06T07:26:05.734671+0000","last_undegraded":"2024-03-06T07:26:05.734671+0000","last_fullsized":"2024-03-06T07:26:05.734671+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:47:32.806030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735512+0000","last_change":"2024-03-06T07:25:55.189198+0000","last_active":"2024-03-06T07:26:05.735512+0000","last_peered":"2024-03-06T07:26:05.735512+0000","last_clean":"2024-03-06T07:26:05.735512+0000","last_became_active":"2024-03-06T07:25:55.188937+0000","last_became_peered":"2024-03-06T07:25:55.188937+0000","last_unstale":"2024-03-06T07:26:05.735512+0000","last_undegraded":"2024-03-06T07:26:05.735512+0000","last_fullsized":"2024-03-06T07:26:05.735512+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735116+0000","last_change":"2024-03-06T07:26:04.229595+0000","last_active":"2024-03-06T07:26:05.735116+0000","last_peered":"2024-03-06T07:26:05.735116+0000","last_clean":"2024-03-06T07:26:05.735116+0000","last_became_active":"2024-03-06T07:26:04.229347+0000","last_became_peered":"2024-03-06T07:26:04.229347+0000","last_unstale":"2024-03-06T07:26:05.735116+0000","last_undegraded":"2024-03-06T07:26:05.735116+0000","last_fullsized":"2024-03-06T07:26:05.735116+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:33:23.378545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734586+0000","last_change":"2024-03-06T07:25:55.174913+0000","last_active":"2024-03-06T07:26:05.734586+0000","last_peered":"2024-03-06T07:26:05.734586+0000","last_clean":"2024-03-06T07:26:05.734586+0000","last_became_active":"2024-03-06T07:25:55.174627+0000","last_became_peered":"2024-03-06T07:25:55.174627+0000","last_unstale":"2024-03-06T07:26:05.734586+0000","last_undegraded":"2024-03-06T07:26:05.734586+0000","last_fullsized":"2024-03-06T07:26:05.734586+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734949+0000","last_change":"2024-03-06T07:26:04.230891+0000","last_active":"2024-03-06T07:26:05.734949+0000","last_peered":"2024-03-06T07:26:05.734949+0000","last_clean":"2024-03-06T07:26:05.734949+0000","last_became_active":"2024-03-06T07:26:04.230580+0000","last_became_peered":"2024-03-06T07:26:04.230580+0000","last_unstale":"2024-03-06T07:26:05.734949+0000","last_undegraded":"2024-03-06T07:26:05.734949+0000","last_fullsized":"2024-03-06T07:26:05.734949+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:12.012177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737488+0000","last_change":"2024-03-06T07:25:55.175881+0000","last_active":"2024-03-06T07:26:05.737488+0000","last_peered":"2024-03-06T07:26:05.737488+0000","last_clean":"2024-03-06T07:26:05.737488+0000","last_became_active":"2024-03-06T07:25:55.175604+0000","last_became_peered":"2024-03-06T07:25:55.175604+0000","last_unstale":"2024-03-06T07:26:05.737488+0000","last_undegraded":"2024-03-06T07:26:05.737488+0000","last_fullsized":"2024-03-06T07:26:05.737488+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:18.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233654+0000","last_change":"2024-03-06T07:26:04.230741+0000","last_active":"2024-03-06T07:26:05.233654+0000","last_peered":"2024-03-06T07:26:05.233654+0000","last_clean":"2024-03-06T07:26:05.233654+0000","last_became_active":"2024-03-06T07:26:04.230058+0000","last_became_peered":"2024-03-06T07:26:04.230058+0000","last_unstale":"2024-03-06T07:26:05.233654+0000","last_undegraded":"2024-03-06T07:26:05.233654+0000","last_fullsized":"2024-03-06T07:26:05.233654+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:47.243949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737255+0000","last_change":"2024-03-06T07:25:55.174883+0000","last_active":"2024-03-06T07:26:05.737255+0000","last_peered":"2024-03-06T07:26:05.737255+0000","last_clean":"2024-03-06T07:26:05.737255+0000","last_became_active":"2024-03-06T07:25:55.174590+0000","last_became_peered":"2024-03-06T07:25:55.174590+0000","last_unstale":"2024-03-06T07:26:05.737255+0000","last_undegraded":"2024-03-06T07:26:05.737255+0000","last_fullsized":"2024-03-06T07:26:05.737255+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737375+0000","last_change":"2024-03-06T07:26:04.238866+0000","last_active":"2024-03-06T07:26:05.737375+0000","last_peered":"2024-03-06T07:26:05.737375+0000","last_clean":"2024-03-06T07:26:05.737375+0000","last_became_active":"2024-03-06T07:26:04.238193+0000","last_became_peered":"2024-03-06T07:26:04.238193+0000","last_unstale":"2024-03-06T07:26:05.737375+0000","last_undegraded":"2024-03-06T07:26:05.737375+0000","last_fullsized":"2024-03-06T07:26:05.737375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:25.186145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735440+0000","last_change":"2024-03-06T07:25:55.182365+0000","last_active":"2024-03-06T07:26:05.735440+0000","last_peered":"2024-03-06T07:26:05.735440+0000","last_clean":"2024-03-06T07:26:05.735440+0000","last_became_active":"2024-03-06T07:25:55.182083+0000","last_became_peered":"2024-03-06T07:25:55.182083+0000","last_unstale":"2024-03-06T07:26:05.735440+0000","last_undegraded":"2024-03-06T07:26:05.735440+0000","last_fullsized":"2024-03-06T07:26:05.735440+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:53.751341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735925+0000","last_change":"2024-03-06T07:26:04.220512+0000","last_active":"2024-03-06T07:26:05.735925+0000","last_peered":"2024-03-06T07:26:05.735925+0000","last_clean":"2024-03-06T07:26:05.735925+0000","last_became_active":"2024-03-06T07:26:04.220222+0000","last_became_peered":"2024-03-06T07:26:04.220222+0000","last_unstale":"2024-03-06T07:26:05.735925+0000","last_undegraded":"2024-03-06T07:26:05.735925+0000","last_fullsized":"2024-03-06T07:26:05.735925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:58.340820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735742+0000","last_change":"2024-03-06T07:25:55.179820+0000","last_active":"2024-03-06T07:26:05.735742+0000","last_peered":"2024-03-06T07:26:05.735742+0000","last_clean":"2024-03-06T07:26:05.735742+0000","last_became_active":"2024-03-06T07:25:55.179573+0000","last_became_peered":"2024-03-06T07:25:55.179573+0000","last_unstale":"2024-03-06T07:26:05.735742+0000","last_undegraded":"2024-03-06T07:26:05.735742+0000","last_fullsized":"2024-03-06T07:26:05.735742+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735695+0000","last_change":"2024-03-06T07:26:04.234781+0000","last_active":"2024-03-06T07:26:05.735695+0000","last_peered":"2024-03-06T07:26:05.735695+0000","last_clean":"2024-03-06T07:26:05.735695+0000","last_became_active":"2024-03-06T07:26:04.234368+0000","last_became_peered":"2024-03-06T07:26:04.234368+0000","last_unstale":"2024-03-06T07:26:05.735695+0000","last_undegraded":"2024-03-06T07:26:05.735695+0000","last_fullsized":"2024-03-06T07:26:05.735695+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:11.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734840+0000","last_change":"2024-03-06T07:25:55.178116+0000","last_active":"2024-03-06T07:26:05.734840+0000","last_peered":"2024-03-06T07:26:05.734840+0000","last_clean":"2024-03-06T07:26:05.734840+0000","last_became_active":"2024-03-06T07:25:55.177769+0000","last_became_peered":"2024-03-06T07:25:55.177769+0000","last_unstale":"2024-03-06T07:26:05.734840+0000","last_undegraded":"2024-03-06T07:26:05.734840+0000","last_fullsized":"2024-03-06T07:26:05.734840+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:19.818533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734678+0000","last_change":"2024-03-06T07:26:04.228971+0000","last_active":"2024-03-06T07:26:05.734678+0000","last_peered":"2024-03-06T07:26:05.734678+0000","last_clean":"2024-03-06T07:26:05.734678+0000","last_became_active":"2024-03-06T07:26:04.228716+0000","last_became_peered":"2024-03-06T07:26:04.228716+0000","last_unstale":"2024-03-06T07:26:05.734678+0000","last_undegraded":"2024-03-06T07:26:05.734678+0000","last_fullsized":"2024-03-06T07:26:05.734678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:20.406031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233509+0000","last_change":"2024-03-06T07:25:55.185952+0000","last_active":"2024-03-06T07:26:05.233509+0000","last_peered":"2024-03-06T07:26:05.233509+0000","last_clean":"2024-03-06T07:26:05.233509+0000","last_became_active":"2024-03-06T07:25:55.185677+0000","last_became_peered":"2024-03-06T07:25:55.185677+0000","last_unstale":"2024-03-06T07:26:05.233509+0000","last_undegraded":"2024-03-06T07:26:05.233509+0000","last_fullsized":"2024-03-06T07:26:05.233509+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:02.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238997+0000","last_change":"2024-03-06T07:26:04.230638+0000","last_active":"2024-03-06T07:26:05.238997+0000","last_peered":"2024-03-06T07:26:05.238997+0000","last_clean":"2024-03-06T07:26:05.238997+0000","last_became_active":"2024-03-06T07:26:04.230366+0000","last_became_peered":"2024-03-06T07:26:04.230366+0000","last_unstale":"2024-03-06T07:26:05.238997+0000","last_undegraded":"2024-03-06T07:26:05.238997+0000","last_fullsized":"2024-03-06T07:26:05.238997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:28.040894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734512+0000","last_change":"2024-03-06T07:25:55.180387+0000","last_active":"2024-03-06T07:26:05.734512+0000","last_peered":"2024-03-06T07:26:05.734512+0000","last_clean":"2024-03-06T07:26:05.734512+0000","last_became_active":"2024-03-06T07:25:55.179778+0000","last_became_peered":"2024-03-06T07:25:55.179778+0000","last_unstale":"2024-03-06T07:26:05.734512+0000","last_undegraded":"2024-03-06T07:26:05.734512+0000","last_fullsized":"2024-03-06T07:26:05.734512+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:16.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735486+0000","last_change":"2024-03-06T07:26:04.232126+0000","last_active":"2024-03-06T07:26:05.735486+0000","last_peered":"2024-03-06T07:26:05.735486+0000","last_clean":"2024-03-06T07:26:05.735486+0000","last_became_active":"2024-03-06T07:26:04.230221+0000","last_became_peered":"2024-03-06T07:26:04.230221+0000","last_unstale":"2024-03-06T07:26:05.735486+0000","last_undegraded":"2024-03-06T07:26:05.735486+0000","last_fullsized":"2024-03-06T07:26:05.735486+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:31.729215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238322+0000","last_change":"2024-03-06T07:25:55.180281+0000","last_active":"2024-03-06T07:26:05.238322+0000","last_peered":"2024-03-06T07:26:05.238322+0000","last_clean":"2024-03-06T07:26:05.238322+0000","last_became_active":"2024-03-06T07:25:55.180014+0000","last_became_peered":"2024-03-06T07:25:55.180014+0000","last_unstale":"2024-03-06T07:26:05.238322+0000","last_undegraded":"2024-03-06T07:26:05.238322+0000","last_fullsized":"2024-03-06T07:26:05.238322+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:08:51.191943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239078+0000","last_change":"2024-03-06T07:26:04.228988+0000","last_active":"2024-03-06T07:26:05.239078+0000","last_peered":"2024-03-06T07:26:05.239078+0000","last_clean":"2024-03-06T07:26:05.239078+0000","last_became_active":"2024-03-06T07:26:04.228619+0000","last_became_peered":"2024-03-06T07:26:04.228619+0000","last_unstale":"2024-03-06T07:26:05.239078+0000","last_undegraded":"2024-03-06T07:26:05.239078+0000","last_fullsized":"2024-03-06T07:26:05.239078+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:12.281654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737511+0000","last_change":"2024-03-06T07:25:55.180083+0000","last_active":"2024-03-06T07:26:05.737511+0000","last_peered":"2024-03-06T07:26:05.737511+0000","last_clean":"2024-03-06T07:26:05.737511+0000","last_became_active":"2024-03-06T07:25:55.179804+0000","last_became_peered":"2024-03-06T07:25:55.179804+0000","last_unstale":"2024-03-06T07:26:05.737511+0000","last_undegraded":"2024-03-06T07:26:05.737511+0000","last_fullsized":"2024-03-06T07:26:05.737511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:03.229947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735046+0000","last_change":"2024-03-06T07:26:04.226151+0000","last_active":"2024-03-06T07:26:05.735046+0000","last_peered":"2024-03-06T07:26:05.735046+0000","last_clean":"2024-03-06T07:26:05.735046+0000","last_became_active":"2024-03-06T07:26:04.225917+0000","last_became_peered":"2024-03-06T07:26:04.225917+0000","last_unstale":"2024-03-06T07:26:05.735046+0000","last_undegraded":"2024-03-06T07:26:05.735046+0000","last_fullsized":"2024-03-06T07:26:05.735046+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:00.625515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240221+0000","last_change":"2024-03-06T07:25:55.176450+0000","last_active":"2024-03-06T07:26:05.240221+0000","last_peered":"2024-03-06T07:26:05.240221+0000","last_clean":"2024-03-06T07:26:05.240221+0000","last_became_active":"2024-03-06T07:25:55.176194+0000","last_became_peered":"2024-03-06T07:25:55.176194+0000","last_unstale":"2024-03-06T07:26:05.240221+0000","last_undegraded":"2024-03-06T07:26:05.240221+0000","last_fullsized":"2024-03-06T07:26:05.240221+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:53.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735881+0000","last_change":"2024-03-06T07:26:04.224711+0000","last_active":"2024-03-06T07:26:05.735881+0000","last_peered":"2024-03-06T07:26:05.735881+0000","last_clean":"2024-03-06T07:26:05.735881+0000","last_became_active":"2024-03-06T07:26:04.224195+0000","last_became_peered":"2024-03-06T07:26:04.224195+0000","last_unstale":"2024-03-06T07:26:05.735881+0000","last_undegraded":"2024-03-06T07:26:05.735881+0000","last_fullsized":"2024-03-06T07:26:05.735881+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:48.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735464+0000","last_change":"2024-03-06T07:25:55.176884+0000","last_active":"2024-03-06T07:26:05.735464+0000","last_peered":"2024-03-06T07:26:05.735464+0000","last_clean":"2024-03-06T07:26:05.735464+0000","last_became_active":"2024-03-06T07:25:55.176546+0000","last_became_peered":"2024-03-06T07:25:55.176546+0000","last_unstale":"2024-03-06T07:26:05.735464+0000","last_undegraded":"2024-03-06T07:26:05.735464+0000","last_fullsized":"2024-03-06T07:26:05.735464+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:22.733869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240474+0000","last_change":"2024-03-06T07:26:04.229350+0000","last_active":"2024-03-06T07:26:05.240474+0000","last_peered":"2024-03-06T07:26:05.240474+0000","last_clean":"2024-03-06T07:26:05.240474+0000","last_became_active":"2024-03-06T07:26:04.229068+0000","last_became_peered":"2024-03-06T07:26:04.229068+0000","last_unstale":"2024-03-06T07:26:05.240474+0000","last_undegraded":"2024-03-06T07:26:05.240474+0000","last_fullsized":"2024-03-06T07:26:05.240474+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:18.508182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735759+0000","last_change":"2024-03-06T07:25:55.181620+0000","last_active":"2024-03-06T07:26:05.735759+0000","last_peered":"2024-03-06T07:26:05.735759+0000","last_clean":"2024-03-06T07:26:05.735759+0000","last_became_active":"2024-03-06T07:25:55.181336+0000","last_became_peered":"2024-03-06T07:25:55.181336+0000","last_unstale":"2024-03-06T07:26:05.735759+0000","last_undegraded":"2024-03-06T07:26:05.735759+0000","last_fullsized":"2024-03-06T07:26:05.735759+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:53:26.282224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734640+0000","last_change":"2024-03-06T07:26:04.234275+0000","last_active":"2024-03-06T07:26:05.734640+0000","last_peered":"2024-03-06T07:26:05.734640+0000","last_clean":"2024-03-06T07:26:05.734640+0000","last_became_active":"2024-03-06T07:26:04.234011+0000","last_became_peered":"2024-03-06T07:26:04.234011+0000","last_unstale":"2024-03-06T07:26:05.734640+0000","last_undegraded":"2024-03-06T07:26:05.734640+0000","last_fullsized":"2024-03-06T07:26:05.734640+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:19.072059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734938+0000","last_change":"2024-03-06T07:25:55.171658+0000","last_active":"2024-03-06T07:26:05.734938+0000","last_peered":"2024-03-06T07:26:05.734938+0000","last_clean":"2024-03-06T07:26:05.734938+0000","last_became_active":"2024-03-06T07:25:55.171366+0000","last_became_peered":"2024-03-06T07:25:55.171366+0000","last_unstale":"2024-03-06T07:26:05.734938+0000","last_undegraded":"2024-03-06T07:26:05.734938+0000","last_fullsized":"2024-03-06T07:26:05.734938+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:03.613423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734764+0000","last_change":"2024-03-06T07:26:04.235220+0000","last_active":"2024-03-06T07:26:05.734764+0000","last_peered":"2024-03-06T07:26:05.734764+0000","last_clean":"2024-03-06T07:26:05.734764+0000","last_became_active":"2024-03-06T07:26:04.234786+0000","last_became_peered":"2024-03-06T07:26:04.234786+0000","last_unstale":"2024-03-06T07:26:05.734764+0000","last_undegraded":"2024-03-06T07:26:05.734764+0000","last_fullsized":"2024-03-06T07:26:05.734764+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:09.370102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734708+0000","last_change":"2024-03-06T07:25:55.185023+0000","last_active":"2024-03-06T07:26:05.734708+0000","last_peered":"2024-03-06T07:26:05.734708+0000","last_clean":"2024-03-06T07:26:05.734708+0000","last_became_active":"2024-03-06T07:25:55.184708+0000","last_became_peered":"2024-03-06T07:25:55.184708+0000","last_unstale":"2024-03-06T07:26:05.734708+0000","last_undegraded":"2024-03-06T07:26:05.734708+0000","last_fullsized":"2024-03-06T07:26:05.734708+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:25.354294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736684+0000","last_change":"2024-03-06T07:26:04.239321+0000","last_active":"2024-03-06T07:26:05.736684+0000","last_peered":"2024-03-06T07:26:05.736684+0000","last_clean":"2024-03-06T07:26:05.736684+0000","last_became_active":"2024-03-06T07:26:04.238746+0000","last_became_peered":"2024-03-06T07:26:04.238746+0000","last_unstale":"2024-03-06T07:26:05.736684+0000","last_undegraded":"2024-03-06T07:26:05.736684+0000","last_fullsized":"2024-03-06T07:26:05.736684+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:46:44.443786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233321+0000","last_change":"2024-03-06T07:25:55.189044+0000","last_active":"2024-03-06T07:26:05.233321+0000","last_peered":"2024-03-06T07:26:05.233321+0000","last_clean":"2024-03-06T07:26:05.233321+0000","last_became_active":"2024-03-06T07:25:55.188772+0000","last_became_peered":"2024-03-06T07:25:55.188772+0000","last_unstale":"2024-03-06T07:26:05.233321+0000","last_undegraded":"2024-03-06T07:26:05.233321+0000","last_fullsized":"2024-03-06T07:26:05.233321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736084+0000","last_change":"2024-03-06T07:26:04.239527+0000","last_active":"2024-03-06T07:26:05.736084+0000","last_peered":"2024-03-06T07:26:05.736084+0000","last_clean":"2024-03-06T07:26:05.736084+0000","last_became_active":"2024-03-06T07:26:04.238895+0000","last_became_peered":"2024-03-06T07:26:04.238895+0000","last_unstale":"2024-03-06T07:26:05.736084+0000","last_undegraded":"2024-03-06T07:26:05.736084+0000","last_fullsized":"2024-03-06T07:26:05.736084+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:23:51.470414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238399+0000","last_change":"2024-03-06T07:25:55.179621+0000","last_active":"2024-03-06T07:26:05.238399+0000","last_peered":"2024-03-06T07:26:05.238399+0000","last_clean":"2024-03-06T07:26:05.238399+0000","last_became_active":"2024-03-06T07:25:55.179320+0000","last_became_peered":"2024-03-06T07:25:55.179320+0000","last_unstale":"2024-03-06T07:26:05.238399+0000","last_undegraded":"2024-03-06T07:26:05.238399+0000","last_fullsized":"2024-03-06T07:26:05.238399+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:38.124631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735672+0000","last_change":"2024-03-06T07:26:04.226226+0000","last_active":"2024-03-06T07:26:05.735672+0000","last_peered":"2024-03-06T07:26:05.735672+0000","last_clean":"2024-03-06T07:26:05.735672+0000","last_became_active":"2024-03-06T07:26:04.225948+0000","last_became_peered":"2024-03-06T07:26:04.225948+0000","last_unstale":"2024-03-06T07:26:05.735672+0000","last_undegraded":"2024-03-06T07:26:05.735672+0000","last_fullsized":"2024-03-06T07:26:05.735672+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:06.910190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736090+0000","last_change":"2024-03-06T07:25:55.183095+0000","last_active":"2024-03-06T07:26:05.736090+0000","last_peered":"2024-03-06T07:26:05.736090+0000","last_clean":"2024-03-06T07:26:05.736090+0000","last_became_active":"2024-03-06T07:25:55.182807+0000","last_became_peered":"2024-03-06T07:25:55.182807+0000","last_unstale":"2024-03-06T07:26:05.736090+0000","last_undegraded":"2024-03-06T07:26:05.736090+0000","last_fullsized":"2024-03-06T07:26:05.736090+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:02.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736473+0000","last_change":"2024-03-06T07:26:04.235785+0000","last_active":"2024-03-06T07:26:05.736473+0000","last_peered":"2024-03-06T07:26:05.736473+0000","last_clean":"2024-03-06T07:26:05.736473+0000","last_became_active":"2024-03-06T07:26:04.232700+0000","last_became_peered":"2024-03-06T07:26:04.232700+0000","last_unstale":"2024-03-06T07:26:05.736473+0000","last_undegraded":"2024-03-06T07:26:05.736473+0000","last_fullsized":"2024-03-06T07:26:05.736473+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:02.137613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736145+0000","last_change":"2024-03-06T07:25:55.180267+0000","last_active":"2024-03-06T07:26:05.736145+0000","last_peered":"2024-03-06T07:26:05.736145+0000","last_clean":"2024-03-06T07:26:05.736145+0000","last_became_active":"2024-03-06T07:25:55.180006+0000","last_became_peered":"2024-03-06T07:25:55.180006+0000","last_unstale":"2024-03-06T07:26:05.736145+0000","last_undegraded":"2024-03-06T07:26:05.736145+0000","last_fullsized":"2024-03-06T07:26:05.736145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:22.125383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735859+0000","last_change":"2024-03-06T07:26:04.226834+0000","last_active":"2024-03-06T07:26:05.735859+0000","last_peered":"2024-03-06T07:26:05.735859+0000","last_clean":"2024-03-06T07:26:05.735859+0000","last_became_active":"2024-03-06T07:26:04.226551+0000","last_became_peered":"2024-03-06T07:26:04.226551+0000","last_unstale":"2024-03-06T07:26:05.735859+0000","last_undegraded":"2024-03-06T07:26:05.735859+0000","last_fullsized":"2024-03-06T07:26:05.735859+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:02.565042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735547+0000","last_change":"2024-03-06T07:25:55.185010+0000","last_active":"2024-03-06T07:26:05.735547+0000","last_peered":"2024-03-06T07:26:05.735547+0000","last_clean":"2024-03-06T07:26:05.735547+0000","last_became_active":"2024-03-06T07:25:55.184723+0000","last_became_peered":"2024-03-06T07:25:55.184723+0000","last_unstale":"2024-03-06T07:26:05.735547+0000","last_undegraded":"2024-03-06T07:26:05.735547+0000","last_fullsized":"2024-03-06T07:26:05.735547+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736042+0000","last_change":"2024-03-06T07:26:04.239795+0000","last_active":"2024-03-06T07:26:05.736042+0000","last_peered":"2024-03-06T07:26:05.736042+0000","last_clean":"2024-03-06T07:26:05.736042+0000","last_became_active":"2024-03-06T07:26:04.239202+0000","last_became_peered":"2024-03-06T07:26:04.239202+0000","last_unstale":"2024-03-06T07:26:05.736042+0000","last_undegraded":"2024-03-06T07:26:05.736042+0000","last_fullsized":"2024-03-06T07:26:05.736042+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:18.508182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734876+0000","last_change":"2024-03-06T07:25:55.171489+0000","last_active":"2024-03-06T07:26:05.734876+0000","last_peered":"2024-03-06T07:26:05.734876+0000","last_clean":"2024-03-06T07:26:05.734876+0000","last_became_active":"2024-03-06T07:25:55.171184+0000","last_became_peered":"2024-03-06T07:25:55.171184+0000","last_unstale":"2024-03-06T07:26:05.734876+0000","last_undegraded":"2024-03-06T07:26:05.734876+0000","last_fullsized":"2024-03-06T07:26:05.734876+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:03.888318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734435+0000","last_change":"2024-03-06T07:26:04.231741+0000","last_active":"2024-03-06T07:26:05.734435+0000","last_peered":"2024-03-06T07:26:05.734435+0000","last_clean":"2024-03-06T07:26:05.734435+0000","last_became_active":"2024-03-06T07:26:04.231482+0000","last_became_peered":"2024-03-06T07:26:04.231482+0000","last_unstale":"2024-03-06T07:26:05.734435+0000","last_undegraded":"2024-03-06T07:26:05.734435+0000","last_fullsized":"2024-03-06T07:26:05.734435+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:35.442692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735237+0000","last_change":"2024-03-06T07:25:55.175885+0000","last_active":"2024-03-06T07:26:05.735237+0000","last_peered":"2024-03-06T07:26:05.735237+0000","last_clean":"2024-03-06T07:26:05.735237+0000","last_became_active":"2024-03-06T07:25:55.175607+0000","last_became_peered":"2024-03-06T07:25:55.175607+0000","last_unstale":"2024-03-06T07:26:05.735237+0000","last_undegraded":"2024-03-06T07:26:05.735237+0000","last_fullsized":"2024-03-06T07:26:05.735237+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734794+0000","last_change":"2024-03-06T07:26:04.239640+0000","last_active":"2024-03-06T07:26:05.734794+0000","last_peered":"2024-03-06T07:26:05.734794+0000","last_clean":"2024-03-06T07:26:05.734794+0000","last_became_active":"2024-03-06T07:26:04.239381+0000","last_became_peered":"2024-03-06T07:26:04.239381+0000","last_unstale":"2024-03-06T07:26:05.734794+0000","last_undegraded":"2024-03-06T07:26:05.734794+0000","last_fullsized":"2024-03-06T07:26:05.734794+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:14.833381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239631+0000","last_change":"2024-03-06T07:25:55.182544+0000","last_active":"2024-03-06T07:26:05.239631+0000","last_peered":"2024-03-06T07:26:05.239631+0000","last_clean":"2024-03-06T07:26:05.239631+0000","last_became_active":"2024-03-06T07:25:55.182215+0000","last_became_peered":"2024-03-06T07:25:55.182215+0000","last_unstale":"2024-03-06T07:26:05.239631+0000","last_undegraded":"2024-03-06T07:26:05.239631+0000","last_fullsized":"2024-03-06T07:26:05.239631+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:47.717714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735040+0000","last_change":"2024-03-06T07:26:04.234363+0000","last_active":"2024-03-06T07:26:05.735040+0000","last_peered":"2024-03-06T07:26:05.735040+0000","last_clean":"2024-03-06T07:26:05.735040+0000","last_became_active":"2024-03-06T07:26:04.234100+0000","last_became_peered":"2024-03-06T07:26:04.234100+0000","last_unstale":"2024-03-06T07:26:05.735040+0000","last_undegraded":"2024-03-06T07:26:05.735040+0000","last_fullsized":"2024-03-06T07:26:05.735040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:16.582927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734885+0000","last_change":"2024-03-06T07:25:55.174623+0000","last_active":"2024-03-06T07:26:05.734885+0000","last_peered":"2024-03-06T07:26:05.734885+0000","last_clean":"2024-03-06T07:26:05.734885+0000","last_became_active":"2024-03-06T07:25:55.174336+0000","last_became_peered":"2024-03-06T07:25:55.174336+0000","last_unstale":"2024-03-06T07:26:05.734885+0000","last_undegraded":"2024-03-06T07:26:05.734885+0000","last_fullsized":"2024-03-06T07:26:05.734885+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:15.212900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735113+0000","last_change":"2024-03-06T07:26:04.231307+0000","last_active":"2024-03-06T07:26:05.735113+0000","last_peered":"2024-03-06T07:26:05.735113+0000","last_clean":"2024-03-06T07:26:05.735113+0000","last_became_active":"2024-03-06T07:26:04.229445+0000","last_became_peered":"2024-03-06T07:26:04.229445+0000","last_unstale":"2024-03-06T07:26:05.735113+0000","last_undegraded":"2024-03-06T07:26:05.735113+0000","last_fullsized":"2024-03-06T07:26:05.735113+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:53.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239940+0000","last_change":"2024-03-06T07:25:55.173469+0000","last_active":"2024-03-06T07:26:05.239940+0000","last_peered":"2024-03-06T07:26:05.239940+0000","last_clean":"2024-03-06T07:26:05.239940+0000","last_became_active":"2024-03-06T07:25:55.173211+0000","last_became_peered":"2024-03-06T07:25:55.173211+0000","last_unstale":"2024-03-06T07:26:05.239940+0000","last_undegraded":"2024-03-06T07:26:05.239940+0000","last_fullsized":"2024-03-06T07:26:05.239940+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:44.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239408+0000","last_change":"2024-03-06T07:26:04.229361+0000","last_active":"2024-03-06T07:26:05.239408+0000","last_peered":"2024-03-06T07:26:05.239408+0000","last_clean":"2024-03-06T07:26:05.239408+0000","last_became_active":"2024-03-06T07:26:04.229096+0000","last_became_peered":"2024-03-06T07:26:04.229096+0000","last_unstale":"2024-03-06T07:26:05.239408+0000","last_undegraded":"2024-03-06T07:26:05.239408+0000","last_fullsized":"2024-03-06T07:26:05.239408+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:53.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233146+0000","last_change":"2024-03-06T07:26:03.218174+0000","last_active":"2024-03-06T07:26:05.233146+0000","last_peered":"2024-03-06T07:26:05.233146+0000","last_clean":"2024-03-06T07:26:05.233146+0000","last_became_active":"2024-03-06T07:26:03.217787+0000","last_became_peered":"2024-03-06T07:26:03.217787+0000","last_unstale":"2024-03-06T07:26:05.233146+0000","last_undegraded":"2024-03-06T07:26:05.233146+0000","last_fullsized":"2024-03-06T07:26:05.233146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:05.900220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734952+0000","last_change":"2024-03-06T07:25:55.183313+0000","last_active":"2024-03-06T07:26:05.734952+0000","last_peered":"2024-03-06T07:26:05.734952+0000","last_clean":"2024-03-06T07:26:05.734952+0000","last_became_active":"2024-03-06T07:25:55.183007+0000","last_became_peered":"2024-03-06T07:25:55.183007+0000","last_unstale":"2024-03-06T07:26:05.734952+0000","last_undegraded":"2024-03-06T07:26:05.734952+0000","last_fullsized":"2024-03-06T07:26:05.734952+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:53.878454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"149'4","reported_seq":26,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239236+0000","last_change":"2024-03-06T07:25:54.177412+0000","last_active":"2024-03-06T07:26:05.239236+0000","last_peered":"2024-03-06T07:26:05.239236+0000","last_clean":"2024-03-06T07:26:05.239236+0000","last_became_active":"2024-03-06T07:25:54.177071+0000","last_became_peered":"2024-03-06T07:25:54.177071+0000","last_unstale":"2024-03-06T07:26:05.239236+0000","last_undegraded":"2024-03-06T07:26:05.239236+0000","last_fullsized":"2024-03-06T07:26:05.239236+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:58.975791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239854+0000","last_change":"2024-03-06T07:26:04.236127+0000","last_active":"2024-03-06T07:26:05.239854+0000","last_peered":"2024-03-06T07:26:05.239854+0000","last_clean":"2024-03-06T07:26:05.239854+0000","last_became_active":"2024-03-06T07:26:04.235069+0000","last_became_peered":"2024-03-06T07:26:04.235069+0000","last_unstale":"2024-03-06T07:26:05.239854+0000","last_undegraded":"2024-03-06T07:26:05.239854+0000","last_fullsized":"2024-03-06T07:26:05.239854+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:16.582927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734391+0000","last_change":"2024-03-06T07:26:03.225955+0000","last_active":"2024-03-06T07:26:05.734391+0000","last_peered":"2024-03-06T07:26:05.734391+0000","last_clean":"2024-03-06T07:26:05.734391+0000","last_became_active":"2024-03-06T07:26:03.225631+0000","last_became_peered":"2024-03-06T07:26:03.225631+0000","last_unstale":"2024-03-06T07:26:05.734391+0000","last_undegraded":"2024-03-06T07:26:05.734391+0000","last_fullsized":"2024-03-06T07:26:05.734391+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:20:01.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734676+0000","last_change":"2024-03-06T07:25:55.169574+0000","last_active":"2024-03-06T07:26:05.734676+0000","last_peered":"2024-03-06T07:26:05.734676+0000","last_clean":"2024-03-06T07:26:05.734676+0000","last_became_active":"2024-03-06T07:25:55.168987+0000","last_became_peered":"2024-03-06T07:25:55.168987+0000","last_unstale":"2024-03-06T07:26:05.734676+0000","last_undegraded":"2024-03-06T07:26:05.734676+0000","last_fullsized":"2024-03-06T07:26:05.734676+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:55.457043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233876+0000","last_change":"2024-03-06T07:25:54.178532+0000","last_active":"2024-03-06T07:26:05.233876+0000","last_peered":"2024-03-06T07:26:05.233876+0000","last_clean":"2024-03-06T07:26:05.233876+0000","last_became_active":"2024-03-06T07:25:54.178228+0000","last_became_peered":"2024-03-06T07:25:54.178228+0000","last_unstale":"2024-03-06T07:26:05.233876+0000","last_undegraded":"2024-03-06T07:26:05.233876+0000","last_fullsized":"2024-03-06T07:26:05.233876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:07.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734323+0000","last_change":"2024-03-06T07:26:04.230124+0000","last_active":"2024-03-06T07:26:05.734323+0000","last_peered":"2024-03-06T07:26:05.734323+0000","last_clean":"2024-03-06T07:26:05.734323+0000","last_became_active":"2024-03-06T07:26:04.229507+0000","last_became_peered":"2024-03-06T07:26:04.229507+0000","last_unstale":"2024-03-06T07:26:05.734323+0000","last_undegraded":"2024-03-06T07:26:05.734323+0000","last_fullsized":"2024-03-06T07:26:05.734323+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:12:11.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239088+0000","last_change":"2024-03-06T07:26:03.221631+0000","last_active":"2024-03-06T07:26:05.239088+0000","last_peered":"2024-03-06T07:26:05.239088+0000","last_clean":"2024-03-06T07:26:05.239088+0000","last_became_active":"2024-03-06T07:26:03.221345+0000","last_became_peered":"2024-03-06T07:26:03.221345+0000","last_unstale":"2024-03-06T07:26:05.239088+0000","last_undegraded":"2024-03-06T07:26:05.239088+0000","last_fullsized":"2024-03-06T07:26:05.239088+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:26.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233603+0000","last_change":"2024-03-06T07:25:55.181488+0000","last_active":"2024-03-06T07:26:05.233603+0000","last_peered":"2024-03-06T07:26:05.233603+0000","last_clean":"2024-03-06T07:26:05.233603+0000","last_became_active":"2024-03-06T07:25:55.181212+0000","last_became_peered":"2024-03-06T07:25:55.181212+0000","last_unstale":"2024-03-06T07:26:05.233603+0000","last_undegraded":"2024-03-06T07:26:05.233603+0000","last_fullsized":"2024-03-06T07:26:05.233603+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:07.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735614+0000","last_change":"2024-03-06T07:25:54.176834+0000","last_active":"2024-03-06T07:26:05.735614+0000","last_peered":"2024-03-06T07:26:05.735614+0000","last_clean":"2024-03-06T07:26:05.735614+0000","last_became_active":"2024-03-06T07:25:54.176517+0000","last_became_peered":"2024-03-06T07:25:54.176517+0000","last_unstale":"2024-03-06T07:26:05.735614+0000","last_undegraded":"2024-03-06T07:26:05.735614+0000","last_fullsized":"2024-03-06T07:26:05.735614+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:46.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734896+0000","last_change":"2024-03-06T07:26:04.232779+0000","last_active":"2024-03-06T07:26:05.734896+0000","last_peered":"2024-03-06T07:26:05.734896+0000","last_clean":"2024-03-06T07:26:05.734896+0000","last_became_active":"2024-03-06T07:26:04.232425+0000","last_became_peered":"2024-03-06T07:26:04.232425+0000","last_unstale":"2024-03-06T07:26:05.734896+0000","last_undegraded":"2024-03-06T07:26:05.734896+0000","last_fullsized":"2024-03-06T07:26:05.734896+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:05.373306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736405+0000","last_change":"2024-03-06T07:26:03.233595+0000","last_active":"2024-03-06T07:26:05.736405+0000","last_peered":"2024-03-06T07:26:05.736405+0000","last_clean":"2024-03-06T07:26:05.736405+0000","last_became_active":"2024-03-06T07:26:03.233343+0000","last_became_peered":"2024-03-06T07:26:03.233343+0000","last_unstale":"2024-03-06T07:26:05.736405+0000","last_undegraded":"2024-03-06T07:26:05.736405+0000","last_fullsized":"2024-03-06T07:26:05.736405+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:08.234461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735372+0000","last_change":"2024-03-06T07:25:55.186954+0000","last_active":"2024-03-06T07:26:05.735372+0000","last_peered":"2024-03-06T07:26:05.735372+0000","last_clean":"2024-03-06T07:26:05.735372+0000","last_became_active":"2024-03-06T07:25:55.186700+0000","last_became_peered":"2024-03-06T07:25:55.186700+0000","last_unstale":"2024-03-06T07:26:05.735372+0000","last_undegraded":"2024-03-06T07:26:05.735372+0000","last_fullsized":"2024-03-06T07:26:05.735372+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735762+0000","last_change":"2024-03-06T07:25:54.173923+0000","last_active":"2024-03-06T07:26:05.735762+0000","last_peered":"2024-03-06T07:26:05.735762+0000","last_clean":"2024-03-06T07:26:05.735762+0000","last_became_active":"2024-03-06T07:25:54.173628+0000","last_became_peered":"2024-03-06T07:25:54.173628+0000","last_unstale":"2024-03-06T07:26:05.735762+0000","last_undegraded":"2024-03-06T07:26:05.735762+0000","last_fullsized":"2024-03-06T07:26:05.735762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:18.074282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239446+0000","last_change":"2024-03-06T07:26:04.233045+0000","last_active":"2024-03-06T07:26:05.239446+0000","last_peered":"2024-03-06T07:26:05.239446+0000","last_clean":"2024-03-06T07:26:05.239446+0000","last_became_active":"2024-03-06T07:26:04.232782+0000","last_became_peered":"2024-03-06T07:26:04.232782+0000","last_unstale":"2024-03-06T07:26:05.239446+0000","last_undegraded":"2024-03-06T07:26:05.239446+0000","last_fullsized":"2024-03-06T07:26:05.239446+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:12.012177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233479+0000","last_change":"2024-03-06T07:26:03.226022+0000","last_active":"2024-03-06T07:26:05.233479+0000","last_peered":"2024-03-06T07:26:05.233479+0000","last_clean":"2024-03-06T07:26:05.233479+0000","last_became_active":"2024-03-06T07:26:03.225758+0000","last_became_peered":"2024-03-06T07:26:03.225758+0000","last_unstale":"2024-03-06T07:26:05.233479+0000","last_undegraded":"2024-03-06T07:26:05.233479+0000","last_fullsized":"2024-03-06T07:26:05.233479+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:11.655160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736835+0000","last_change":"2024-03-06T07:25:55.179023+0000","last_active":"2024-03-06T07:26:05.736835+0000","last_peered":"2024-03-06T07:26:05.736835+0000","last_clean":"2024-03-06T07:26:05.736835+0000","last_became_active":"2024-03-06T07:25:55.178604+0000","last_became_peered":"2024-03-06T07:25:55.178604+0000","last_unstale":"2024-03-06T07:26:05.736835+0000","last_undegraded":"2024-03-06T07:26:05.736835+0000","last_fullsized":"2024-03-06T07:26:05.736835+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736778+0000","last_change":"2024-03-06T07:25:54.179340+0000","last_active":"2024-03-06T07:26:05.736778+0000","last_peered":"2024-03-06T07:26:05.736778+0000","last_clean":"2024-03-06T07:26:05.736778+0000","last_became_active":"2024-03-06T07:25:54.179061+0000","last_became_peered":"2024-03-06T07:25:54.179061+0000","last_unstale":"2024-03-06T07:26:05.736778+0000","last_undegraded":"2024-03-06T07:26:05.736778+0000","last_fullsized":"2024-03-06T07:26:05.736778+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:24.779584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737406+0000","last_change":"2024-03-06T07:26:04.234152+0000","last_active":"2024-03-06T07:26:05.737406+0000","last_peered":"2024-03-06T07:26:05.737406+0000","last_clean":"2024-03-06T07:26:05.737406+0000","last_became_active":"2024-03-06T07:26:04.233622+0000","last_became_peered":"2024-03-06T07:26:04.233622+0000","last_unstale":"2024-03-06T07:26:05.737406+0000","last_undegraded":"2024-03-06T07:26:05.737406+0000","last_fullsized":"2024-03-06T07:26:05.737406+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:00.621820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238603+0000","last_change":"2024-03-06T07:26:03.227584+0000","last_active":"2024-03-06T07:26:05.238603+0000","last_peered":"2024-03-06T07:26:05.238603+0000","last_clean":"2024-03-06T07:26:05.238603+0000","last_became_active":"2024-03-06T07:26:03.227070+0000","last_became_peered":"2024-03-06T07:26:03.227070+0000","last_unstale":"2024-03-06T07:26:05.238603+0000","last_undegraded":"2024-03-06T07:26:05.238603+0000","last_fullsized":"2024-03-06T07:26:05.238603+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:28.273592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737141+0000","last_change":"2024-03-06T07:25:55.186844+0000","last_active":"2024-03-06T07:26:05.737141+0000","last_peered":"2024-03-06T07:26:05.737141+0000","last_clean":"2024-03-06T07:26:05.737141+0000","last_became_active":"2024-03-06T07:25:55.186558+0000","last_became_peered":"2024-03-06T07:25:55.186558+0000","last_unstale":"2024-03-06T07:26:05.737141+0000","last_undegraded":"2024-03-06T07:26:05.737141+0000","last_fullsized":"2024-03-06T07:26:05.737141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:40.318903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233699+0000","last_change":"2024-03-06T07:25:54.182713+0000","last_active":"2024-03-06T07:26:05.233699+0000","last_peered":"2024-03-06T07:26:05.233699+0000","last_clean":"2024-03-06T07:26:05.233699+0000","last_became_active":"2024-03-06T07:25:54.182447+0000","last_became_peered":"2024-03-06T07:25:54.182447+0000","last_unstale":"2024-03-06T07:26:05.233699+0000","last_undegraded":"2024-03-06T07:26:05.233699+0000","last_fullsized":"2024-03-06T07:26:05.233699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:17.509614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736123+0000","last_change":"2024-03-06T07:26:04.233107+0000","last_active":"2024-03-06T07:26:05.736123+0000","last_peered":"2024-03-06T07:26:05.736123+0000","last_clean":"2024-03-06T07:26:05.736123+0000","last_became_active":"2024-03-06T07:26:04.231842+0000","last_became_peered":"2024-03-06T07:26:04.231842+0000","last_unstale":"2024-03-06T07:26:05.736123+0000","last_undegraded":"2024-03-06T07:26:05.736123+0000","last_fullsized":"2024-03-06T07:26:05.736123+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:12.940025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735232+0000","last_change":"2024-03-06T07:26:03.234423+0000","last_active":"2024-03-06T07:26:05.735232+0000","last_peered":"2024-03-06T07:26:05.735232+0000","last_clean":"2024-03-06T07:26:05.735232+0000","last_became_active":"2024-03-06T07:26:03.234173+0000","last_became_peered":"2024-03-06T07:26:03.234173+0000","last_unstale":"2024-03-06T07:26:05.735232+0000","last_undegraded":"2024-03-06T07:26:05.735232+0000","last_fullsized":"2024-03-06T07:26:05.735232+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:33.445743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736225+0000","last_change":"2024-03-06T07:25:55.172223+0000","last_active":"2024-03-06T07:26:05.736225+0000","last_peered":"2024-03-06T07:26:05.736225+0000","last_clean":"2024-03-06T07:26:05.736225+0000","last_became_active":"2024-03-06T07:25:55.171895+0000","last_became_peered":"2024-03-06T07:25:55.171895+0000","last_unstale":"2024-03-06T07:26:05.736225+0000","last_undegraded":"2024-03-06T07:26:05.736225+0000","last_fullsized":"2024-03-06T07:26:05.736225+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:27.069827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735146+0000","last_change":"2024-03-06T07:25:54.172467+0000","last_active":"2024-03-06T07:26:05.735146+0000","last_peered":"2024-03-06T07:26:05.735146+0000","last_clean":"2024-03-06T07:26:05.735146+0000","last_became_active":"2024-03-06T07:25:54.172185+0000","last_became_peered":"2024-03-06T07:25:54.172185+0000","last_unstale":"2024-03-06T07:26:05.735146+0000","last_undegraded":"2024-03-06T07:26:05.735146+0000","last_fullsized":"2024-03-06T07:26:05.735146+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:58.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736072+0000","last_change":"2024-03-06T07:26:04.240395+0000","last_active":"2024-03-06T07:26:05.736072+0000","last_peered":"2024-03-06T07:26:05.736072+0000","last_clean":"2024-03-06T07:26:05.736072+0000","last_became_active":"2024-03-06T07:26:04.240115+0000","last_became_peered":"2024-03-06T07:26:04.240115+0000","last_unstale":"2024-03-06T07:26:05.736072+0000","last_undegraded":"2024-03-06T07:26:05.736072+0000","last_fullsized":"2024-03-06T07:26:05.736072+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:08.475164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737146+0000","last_change":"2024-03-06T07:26:03.230790+0000","last_active":"2024-03-06T07:26:05.737146+0000","last_peered":"2024-03-06T07:26:05.737146+0000","last_clean":"2024-03-06T07:26:05.737146+0000","last_became_active":"2024-03-06T07:26:03.230556+0000","last_became_peered":"2024-03-06T07:26:03.230556+0000","last_unstale":"2024-03-06T07:26:05.737146+0000","last_undegraded":"2024-03-06T07:26:05.737146+0000","last_fullsized":"2024-03-06T07:26:05.737146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:57.404566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239141+0000","last_change":"2024-03-06T07:25:55.175586+0000","last_active":"2024-03-06T07:26:05.239141+0000","last_peered":"2024-03-06T07:26:05.239141+0000","last_clean":"2024-03-06T07:26:05.239141+0000","last_became_active":"2024-03-06T07:25:55.175328+0000","last_became_peered":"2024-03-06T07:25:55.175328+0000","last_unstale":"2024-03-06T07:26:05.239141+0000","last_undegraded":"2024-03-06T07:26:05.239141+0000","last_fullsized":"2024-03-06T07:26:05.239141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:34.059703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734918+0000","last_change":"2024-03-06T07:25:54.175104+0000","last_active":"2024-03-06T07:26:05.734918+0000","last_peered":"2024-03-06T07:26:05.734918+0000","last_clean":"2024-03-06T07:26:05.734918+0000","last_became_active":"2024-03-06T07:25:54.174837+0000","last_became_peered":"2024-03-06T07:25:54.174837+0000","last_unstale":"2024-03-06T07:26:05.734918+0000","last_undegraded":"2024-03-06T07:26:05.734918+0000","last_fullsized":"2024-03-06T07:26:05.734918+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:15.768972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736224+0000","last_change":"2024-03-06T07:26:04.235419+0000","last_active":"2024-03-06T07:26:05.736224+0000","last_peered":"2024-03-06T07:26:05.736224+0000","last_clean":"2024-03-06T07:26:05.736224+0000","last_became_active":"2024-03-06T07:26:04.235181+0000","last_became_peered":"2024-03-06T07:26:04.235181+0000","last_unstale":"2024-03-06T07:26:05.736224+0000","last_undegraded":"2024-03-06T07:26:05.736224+0000","last_fullsized":"2024-03-06T07:26:05.736224+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:48:51.543206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239721+0000","last_change":"2024-03-06T07:26:03.226983+0000","last_active":"2024-03-06T07:26:05.239721+0000","last_peered":"2024-03-06T07:26:05.239721+0000","last_clean":"2024-03-06T07:26:05.239721+0000","last_became_active":"2024-03-06T07:26:03.226649+0000","last_became_peered":"2024-03-06T07:26:03.226649+0000","last_unstale":"2024-03-06T07:26:05.239721+0000","last_undegraded":"2024-03-06T07:26:05.239721+0000","last_fullsized":"2024-03-06T07:26:05.239721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:30.775606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735341+0000","last_change":"2024-03-06T07:25:55.183382+0000","last_active":"2024-03-06T07:26:05.735341+0000","last_peered":"2024-03-06T07:26:05.735341+0000","last_clean":"2024-03-06T07:26:05.735341+0000","last_became_active":"2024-03-06T07:25:55.182975+0000","last_became_peered":"2024-03-06T07:25:55.182975+0000","last_unstale":"2024-03-06T07:26:05.735341+0000","last_undegraded":"2024-03-06T07:26:05.735341+0000","last_fullsized":"2024-03-06T07:26:05.735341+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:04.379898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736000+0000","last_change":"2024-03-06T07:25:54.184577+0000","last_active":"2024-03-06T07:26:05.736000+0000","last_peered":"2024-03-06T07:26:05.736000+0000","last_clean":"2024-03-06T07:26:05.736000+0000","last_became_active":"2024-03-06T07:25:54.184068+0000","last_became_peered":"2024-03-06T07:25:54.184068+0000","last_unstale":"2024-03-06T07:26:05.736000+0000","last_undegraded":"2024-03-06T07:26:05.736000+0000","last_fullsized":"2024-03-06T07:26:05.736000+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:30.954235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238281+0000","last_change":"2024-03-06T07:26:04.228139+0000","last_active":"2024-03-06T07:26:05.238281+0000","last_peered":"2024-03-06T07:26:05.238281+0000","last_clean":"2024-03-06T07:26:05.238281+0000","last_became_active":"2024-03-06T07:26:04.227846+0000","last_became_peered":"2024-03-06T07:26:04.227846+0000","last_unstale":"2024-03-06T07:26:05.238281+0000","last_undegraded":"2024-03-06T07:26:05.238281+0000","last_fullsized":"2024-03-06T07:26:05.238281+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:29.622186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734724+0000","last_change":"2024-03-06T07:26:03.235295+0000","last_active":"2024-03-06T07:26:05.734724+0000","last_peered":"2024-03-06T07:26:05.734724+0000","last_clean":"2024-03-06T07:26:05.734724+0000","last_became_active":"2024-03-06T07:26:03.235026+0000","last_became_peered":"2024-03-06T07:26:03.235026+0000","last_unstale":"2024-03-06T07:26:05.734724+0000","last_undegraded":"2024-03-06T07:26:05.734724+0000","last_fullsized":"2024-03-06T07:26:05.734724+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:59.611849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735592+0000","last_change":"2024-03-06T07:25:55.175578+0000","last_active":"2024-03-06T07:26:05.735592+0000","last_peered":"2024-03-06T07:26:05.735592+0000","last_clean":"2024-03-06T07:26:05.735592+0000","last_became_active":"2024-03-06T07:25:55.175257+0000","last_became_peered":"2024-03-06T07:25:55.175257+0000","last_unstale":"2024-03-06T07:26:05.735592+0000","last_undegraded":"2024-03-06T07:26:05.735592+0000","last_fullsized":"2024-03-06T07:26:05.735592+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:56.174605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"149'3","reported_seq":25,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239243+0000","last_change":"2024-03-06T07:25:54.177422+0000","last_active":"2024-03-06T07:26:05.239243+0000","last_peered":"2024-03-06T07:26:05.239243+0000","last_clean":"2024-03-06T07:26:05.239243+0000","last_became_active":"2024-03-06T07:25:54.177048+0000","last_became_peered":"2024-03-06T07:25:54.177048+0000","last_unstale":"2024-03-06T07:26:05.239243+0000","last_undegraded":"2024-03-06T07:26:05.239243+0000","last_fullsized":"2024-03-06T07:26:05.239243+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:21.672283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735347+0000","last_change":"2024-03-06T07:26:04.229458+0000","last_active":"2024-03-06T07:26:05.735347+0000","last_peered":"2024-03-06T07:26:05.735347+0000","last_clean":"2024-03-06T07:26:05.735347+0000","last_became_active":"2024-03-06T07:26:04.229214+0000","last_became_peered":"2024-03-06T07:26:04.229214+0000","last_unstale":"2024-03-06T07:26:05.735347+0000","last_undegraded":"2024-03-06T07:26:05.735347+0000","last_fullsized":"2024-03-06T07:26:05.735347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:43.424540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233407+0000","last_change":"2024-03-06T07:26:03.225838+0000","last_active":"2024-03-06T07:26:05.233407+0000","last_peered":"2024-03-06T07:26:05.233407+0000","last_clean":"2024-03-06T07:26:05.233407+0000","last_became_active":"2024-03-06T07:26:03.225557+0000","last_became_peered":"2024-03-06T07:26:03.225557+0000","last_unstale":"2024-03-06T07:26:05.233407+0000","last_undegraded":"2024-03-06T07:26:05.233407+0000","last_fullsized":"2024-03-06T07:26:05.233407+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:27.860270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735515+0000","last_change":"2024-03-06T07:25:55.182193+0000","last_active":"2024-03-06T07:26:05.735515+0000","last_peered":"2024-03-06T07:26:05.735515+0000","last_clean":"2024-03-06T07:26:05.735515+0000","last_became_active":"2024-03-06T07:25:55.181940+0000","last_became_peered":"2024-03-06T07:25:55.181940+0000","last_unstale":"2024-03-06T07:26:05.735515+0000","last_undegraded":"2024-03-06T07:26:05.735515+0000","last_fullsized":"2024-03-06T07:26:05.735515+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:17.306156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"149'6","reported_seq":28,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736383+0000","last_change":"2024-03-06T07:25:54.184311+0000","last_active":"2024-03-06T07:26:05.736383+0000","last_peered":"2024-03-06T07:26:05.736383+0000","last_clean":"2024-03-06T07:26:05.736383+0000","last_became_active":"2024-03-06T07:25:54.183969+0000","last_became_peered":"2024-03-06T07:25:54.183969+0000","last_unstale":"2024-03-06T07:26:05.736383+0000","last_undegraded":"2024-03-06T07:26:05.736383+0000","last_fullsized":"2024-03-06T07:26:05.736383+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:08.451250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736206+0000","last_change":"2024-03-06T07:26:04.233830+0000","last_active":"2024-03-06T07:26:05.736206+0000","last_peered":"2024-03-06T07:26:05.736206+0000","last_clean":"2024-03-06T07:26:05.736206+0000","last_became_active":"2024-03-06T07:26:04.232811+0000","last_became_peered":"2024-03-06T07:26:04.232811+0000","last_unstale":"2024-03-06T07:26:05.736206+0000","last_undegraded":"2024-03-06T07:26:05.736206+0000","last_fullsized":"2024-03-06T07:26:05.736206+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:41.011166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735008+0000","last_change":"2024-03-06T07:26:03.226298+0000","last_active":"2024-03-06T07:26:05.735008+0000","last_peered":"2024-03-06T07:26:05.735008+0000","last_clean":"2024-03-06T07:26:05.735008+0000","last_became_active":"2024-03-06T07:26:03.226040+0000","last_became_peered":"2024-03-06T07:26:03.226040+0000","last_unstale":"2024-03-06T07:26:05.735008+0000","last_undegraded":"2024-03-06T07:26:05.735008+0000","last_fullsized":"2024-03-06T07:26:05.735008+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:16.749386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240045+0000","last_change":"2024-03-06T07:25:55.174144+0000","last_active":"2024-03-06T07:26:05.240045+0000","last_peered":"2024-03-06T07:26:05.240045+0000","last_clean":"2024-03-06T07:26:05.240045+0000","last_became_active":"2024-03-06T07:25:55.173885+0000","last_became_peered":"2024-03-06T07:25:55.173885+0000","last_unstale":"2024-03-06T07:26:05.240045+0000","last_undegraded":"2024-03-06T07:26:05.240045+0000","last_fullsized":"2024-03-06T07:26:05.240045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:59.981015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233588+0000","last_change":"2024-03-06T07:25:54.175000+0000","last_active":"2024-03-06T07:26:05.233588+0000","last_peered":"2024-03-06T07:26:05.233588+0000","last_clean":"2024-03-06T07:26:05.233588+0000","last_became_active":"2024-03-06T07:25:54.174717+0000","last_became_peered":"2024-03-06T07:25:54.174717+0000","last_unstale":"2024-03-06T07:26:05.233588+0000","last_undegraded":"2024-03-06T07:26:05.233588+0000","last_fullsized":"2024-03-06T07:26:05.233588+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:52.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239649+0000","last_change":"2024-03-06T07:26:04.232417+0000","last_active":"2024-03-06T07:26:05.239649+0000","last_peered":"2024-03-06T07:26:05.239649+0000","last_clean":"2024-03-06T07:26:05.239649+0000","last_became_active":"2024-03-06T07:26:04.232109+0000","last_became_peered":"2024-03-06T07:26:04.232109+0000","last_unstale":"2024-03-06T07:26:05.239649+0000","last_undegraded":"2024-03-06T07:26:05.239649+0000","last_fullsized":"2024-03-06T07:26:05.239649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:24.264607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238939+0000","last_change":"2024-03-06T07:26:03.225275+0000","last_active":"2024-03-06T07:26:05.238939+0000","last_peered":"2024-03-06T07:26:05.238939+0000","last_clean":"2024-03-06T07:26:05.238939+0000","last_became_active":"2024-03-06T07:26:03.224992+0000","last_became_peered":"2024-03-06T07:26:03.224992+0000","last_unstale":"2024-03-06T07:26:05.238939+0000","last_undegraded":"2024-03-06T07:26:05.238939+0000","last_fullsized":"2024-03-06T07:26:05.238939+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:12.421635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737052+0000","last_change":"2024-03-06T07:25:55.186488+0000","last_active":"2024-03-06T07:26:05.737052+0000","last_peered":"2024-03-06T07:26:05.737052+0000","last_clean":"2024-03-06T07:26:05.737052+0000","last_became_active":"2024-03-06T07:25:55.185803+0000","last_became_peered":"2024-03-06T07:25:55.185803+0000","last_unstale":"2024-03-06T07:26:05.737052+0000","last_undegraded":"2024-03-06T07:26:05.737052+0000","last_fullsized":"2024-03-06T07:26:05.737052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735682+0000","last_change":"2024-03-06T07:25:54.178593+0000","last_active":"2024-03-06T07:26:05.735682+0000","last_peered":"2024-03-06T07:26:05.735682+0000","last_clean":"2024-03-06T07:26:05.735682+0000","last_became_active":"2024-03-06T07:25:54.178229+0000","last_became_peered":"2024-03-06T07:25:54.178229+0000","last_unstale":"2024-03-06T07:26:05.735682+0000","last_undegraded":"2024-03-06T07:26:05.735682+0000","last_fullsized":"2024-03-06T07:26:05.735682+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:59.187499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735634+0000","last_change":"2024-03-06T07:26:04.233871+0000","last_active":"2024-03-06T07:26:05.735634+0000","last_peered":"2024-03-06T07:26:05.735634+0000","last_clean":"2024-03-06T07:26:05.735634+0000","last_became_active":"2024-03-06T07:26:04.233555+0000","last_became_peered":"2024-03-06T07:26:04.233555+0000","last_unstale":"2024-03-06T07:26:05.735634+0000","last_undegraded":"2024-03-06T07:26:05.735634+0000","last_fullsized":"2024-03-06T07:26:05.735634+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:41.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233893+0000","last_change":"2024-03-06T07:26:03.227103+0000","last_active":"2024-03-06T07:26:05.233893+0000","last_peered":"2024-03-06T07:26:05.233893+0000","last_clean":"2024-03-06T07:26:05.233893+0000","last_became_active":"2024-03-06T07:26:03.226866+0000","last_became_peered":"2024-03-06T07:26:03.226866+0000","last_unstale":"2024-03-06T07:26:05.233893+0000","last_undegraded":"2024-03-06T07:26:05.233893+0000","last_fullsized":"2024-03-06T07:26:05.233893+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:42.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736370+0000","last_change":"2024-03-06T07:25:55.175110+0000","last_active":"2024-03-06T07:26:05.736370+0000","last_peered":"2024-03-06T07:26:05.736370+0000","last_clean":"2024-03-06T07:26:05.736370+0000","last_became_active":"2024-03-06T07:25:55.174818+0000","last_became_peered":"2024-03-06T07:25:55.174818+0000","last_unstale":"2024-03-06T07:26:05.736370+0000","last_undegraded":"2024-03-06T07:26:05.736370+0000","last_fullsized":"2024-03-06T07:26:05.736370+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:18.191924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735019+0000","last_change":"2024-03-06T07:25:54.178997+0000","last_active":"2024-03-06T07:26:05.735019+0000","last_peered":"2024-03-06T07:26:05.735019+0000","last_clean":"2024-03-06T07:26:05.735019+0000","last_became_active":"2024-03-06T07:25:54.178699+0000","last_became_peered":"2024-03-06T07:25:54.178699+0000","last_unstale":"2024-03-06T07:26:05.735019+0000","last_undegraded":"2024-03-06T07:26:05.735019+0000","last_fullsized":"2024-03-06T07:26:05.735019+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:04.021145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736821+0000","last_change":"2024-03-06T07:26:04.230169+0000","last_active":"2024-03-06T07:26:05.736821+0000","last_peered":"2024-03-06T07:26:05.736821+0000","last_clean":"2024-03-06T07:26:05.736821+0000","last_became_active":"2024-03-06T07:26:04.229842+0000","last_became_peered":"2024-03-06T07:26:04.229842+0000","last_unstale":"2024-03-06T07:26:05.736821+0000","last_undegraded":"2024-03-06T07:26:05.736821+0000","last_fullsized":"2024-03-06T07:26:05.736821+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:56.769422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735301+0000","last_change":"2024-03-06T07:26:03.235702+0000","last_active":"2024-03-06T07:26:05.735301+0000","last_peered":"2024-03-06T07:26:05.735301+0000","last_clean":"2024-03-06T07:26:05.735301+0000","last_became_active":"2024-03-06T07:26:03.235369+0000","last_became_peered":"2024-03-06T07:26:03.235369+0000","last_unstale":"2024-03-06T07:26:05.735301+0000","last_undegraded":"2024-03-06T07:26:05.735301+0000","last_fullsized":"2024-03-06T07:26:05.735301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:24.176175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736143+0000","last_change":"2024-03-06T07:25:55.181498+0000","last_active":"2024-03-06T07:26:05.736143+0000","last_peered":"2024-03-06T07:26:05.736143+0000","last_clean":"2024-03-06T07:26:05.736143+0000","last_became_active":"2024-03-06T07:25:55.180900+0000","last_became_peered":"2024-03-06T07:25:55.180900+0000","last_unstale":"2024-03-06T07:26:05.736143+0000","last_undegraded":"2024-03-06T07:26:05.736143+0000","last_fullsized":"2024-03-06T07:26:05.736143+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:06:57.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"149'8","reported_seq":30,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735009+0000","last_change":"2024-03-06T07:25:54.177051+0000","last_active":"2024-03-06T07:26:05.735009+0000","last_peered":"2024-03-06T07:26:05.735009+0000","last_clean":"2024-03-06T07:26:05.735009+0000","last_became_active":"2024-03-06T07:25:54.176774+0000","last_became_peered":"2024-03-06T07:25:54.176774+0000","last_unstale":"2024-03-06T07:26:05.735009+0000","last_undegraded":"2024-03-06T07:26:05.735009+0000","last_fullsized":"2024-03-06T07:26:05.735009+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:14.207675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736360+0000","last_change":"2024-03-06T07:26:04.228270+0000","last_active":"2024-03-06T07:26:05.736360+0000","last_peered":"2024-03-06T07:26:05.736360+0000","last_clean":"2024-03-06T07:26:05.736360+0000","last_became_active":"2024-03-06T07:26:04.228032+0000","last_became_peered":"2024-03-06T07:26:04.228032+0000","last_unstale":"2024-03-06T07:26:05.736360+0000","last_undegraded":"2024-03-06T07:26:05.736360+0000","last_fullsized":"2024-03-06T07:26:05.736360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:38.167351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735262+0000","last_change":"2024-03-06T07:26:03.225140+0000","last_active":"2024-03-06T07:26:05.735262+0000","last_peered":"2024-03-06T07:26:05.735262+0000","last_clean":"2024-03-06T07:26:05.735262+0000","last_became_active":"2024-03-06T07:26:03.224872+0000","last_became_peered":"2024-03-06T07:26:03.224872+0000","last_unstale":"2024-03-06T07:26:05.735262+0000","last_undegraded":"2024-03-06T07:26:05.735262+0000","last_fullsized":"2024-03-06T07:26:05.735262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:32.154450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734821+0000","last_change":"2024-03-06T07:25:55.190742+0000","last_active":"2024-03-06T07:26:05.734821+0000","last_peered":"2024-03-06T07:26:05.734821+0000","last_clean":"2024-03-06T07:26:05.734821+0000","last_became_active":"2024-03-06T07:25:55.190479+0000","last_became_peered":"2024-03-06T07:25:55.190479+0000","last_unstale":"2024-03-06T07:26:05.734821+0000","last_undegraded":"2024-03-06T07:26:05.734821+0000","last_fullsized":"2024-03-06T07:26:05.734821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736183+0000","last_change":"2024-03-06T07:25:54.178661+0000","last_active":"2024-03-06T07:26:05.736183+0000","last_peered":"2024-03-06T07:26:05.736183+0000","last_clean":"2024-03-06T07:26:05.736183+0000","last_became_active":"2024-03-06T07:25:54.178379+0000","last_became_peered":"2024-03-06T07:25:54.178379+0000","last_unstale":"2024-03-06T07:26:05.736183+0000","last_undegraded":"2024-03-06T07:26:05.736183+0000","last_fullsized":"2024-03-06T07:26:05.736183+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:30.562792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734839+0000","last_change":"2024-03-06T07:26:04.234322+0000","last_active":"2024-03-06T07:26:05.734839+0000","last_peered":"2024-03-06T07:26:05.734839+0000","last_clean":"2024-03-06T07:26:05.734839+0000","last_became_active":"2024-03-06T07:26:04.234083+0000","last_became_peered":"2024-03-06T07:26:04.234083+0000","last_unstale":"2024-03-06T07:26:05.734839+0000","last_undegraded":"2024-03-06T07:26:05.734839+0000","last_fullsized":"2024-03-06T07:26:05.734839+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:40.831704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239653+0000","last_change":"2024-03-06T07:26:03.229781+0000","last_active":"2024-03-06T07:26:05.239653+0000","last_peered":"2024-03-06T07:26:05.239653+0000","last_clean":"2024-03-06T07:26:05.239653+0000","last_became_active":"2024-03-06T07:26:03.229551+0000","last_became_peered":"2024-03-06T07:26:03.229551+0000","last_unstale":"2024-03-06T07:26:05.239653+0000","last_undegraded":"2024-03-06T07:26:05.239653+0000","last_fullsized":"2024-03-06T07:26:05.239653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:46.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240477+0000","last_change":"2024-03-06T07:25:55.178452+0000","last_active":"2024-03-06T07:26:05.240477+0000","last_peered":"2024-03-06T07:26:05.240477+0000","last_clean":"2024-03-06T07:26:05.240477+0000","last_became_active":"2024-03-06T07:25:55.178193+0000","last_became_peered":"2024-03-06T07:25:55.178193+0000","last_unstale":"2024-03-06T07:26:05.240477+0000","last_undegraded":"2024-03-06T07:26:05.240477+0000","last_fullsized":"2024-03-06T07:26:05.240477+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:32.109891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737256+0000","last_change":"2024-03-06T07:25:54.173705+0000","last_active":"2024-03-06T07:26:05.737256+0000","last_peered":"2024-03-06T07:26:05.737256+0000","last_clean":"2024-03-06T07:26:05.737256+0000","last_became_active":"2024-03-06T07:25:54.173438+0000","last_became_peered":"2024-03-06T07:25:54.173438+0000","last_unstale":"2024-03-06T07:26:05.737256+0000","last_undegraded":"2024-03-06T07:26:05.737256+0000","last_fullsized":"2024-03-06T07:26:05.737256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:51.526470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239552+0000","last_change":"2024-03-06T07:26:04.231621+0000","last_active":"2024-03-06T07:26:05.239552+0000","last_peered":"2024-03-06T07:26:05.239552+0000","last_clean":"2024-03-06T07:26:05.239552+0000","last_became_active":"2024-03-06T07:26:04.231349+0000","last_became_peered":"2024-03-06T07:26:04.231349+0000","last_unstale":"2024-03-06T07:26:05.239552+0000","last_undegraded":"2024-03-06T07:26:05.239552+0000","last_fullsized":"2024-03-06T07:26:05.239552+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:25.825904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734908+0000","last_change":"2024-03-06T07:26:03.228059+0000","last_active":"2024-03-06T07:26:05.734908+0000","last_peered":"2024-03-06T07:26:05.734908+0000","last_clean":"2024-03-06T07:26:05.734908+0000","last_became_active":"2024-03-06T07:26:03.227604+0000","last_became_peered":"2024-03-06T07:26:03.227604+0000","last_unstale":"2024-03-06T07:26:05.734908+0000","last_undegraded":"2024-03-06T07:26:05.734908+0000","last_fullsized":"2024-03-06T07:26:05.734908+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:40.001196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735180+0000","last_change":"2024-03-06T07:25:55.172838+0000","last_active":"2024-03-06T07:26:05.735180+0000","last_peered":"2024-03-06T07:26:05.735180+0000","last_clean":"2024-03-06T07:26:05.735180+0000","last_became_active":"2024-03-06T07:25:55.172505+0000","last_became_peered":"2024-03-06T07:25:55.172505+0000","last_unstale":"2024-03-06T07:26:05.735180+0000","last_undegraded":"2024-03-06T07:26:05.735180+0000","last_fullsized":"2024-03-06T07:26:05.735180+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:25.784809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233712+0000","last_change":"2024-03-06T07:25:54.177100+0000","last_active":"2024-03-06T07:26:05.233712+0000","last_peered":"2024-03-06T07:26:05.233712+0000","last_clean":"2024-03-06T07:26:05.233712+0000","last_became_active":"2024-03-06T07:25:54.176822+0000","last_became_peered":"2024-03-06T07:25:54.176822+0000","last_unstale":"2024-03-06T07:26:05.233712+0000","last_undegraded":"2024-03-06T07:26:05.233712+0000","last_fullsized":"2024-03-06T07:26:05.233712+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.349070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238660+0000","last_change":"2024-03-06T07:26:04.231459+0000","last_active":"2024-03-06T07:26:05.238660+0000","last_peered":"2024-03-06T07:26:05.238660+0000","last_clean":"2024-03-06T07:26:05.238660+0000","last_became_active":"2024-03-06T07:26:04.231187+0000","last_became_peered":"2024-03-06T07:26:04.231187+0000","last_unstale":"2024-03-06T07:26:05.238660+0000","last_undegraded":"2024-03-06T07:26:05.238660+0000","last_fullsized":"2024-03-06T07:26:05.238660+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:56.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734520+0000","last_change":"2024-03-06T07:26:03.230950+0000","last_active":"2024-03-06T07:26:05.734520+0000","last_peered":"2024-03-06T07:26:05.734520+0000","last_clean":"2024-03-06T07:26:05.734520+0000","last_became_active":"2024-03-06T07:26:03.230671+0000","last_became_peered":"2024-03-06T07:26:03.230671+0000","last_unstale":"2024-03-06T07:26:05.734520+0000","last_undegraded":"2024-03-06T07:26:05.734520+0000","last_fullsized":"2024-03-06T07:26:05.734520+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:17.498212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735327+0000","last_change":"2024-03-06T07:25:55.183106+0000","last_active":"2024-03-06T07:26:05.735327+0000","last_peered":"2024-03-06T07:26:05.735327+0000","last_clean":"2024-03-06T07:26:05.735327+0000","last_became_active":"2024-03-06T07:25:55.182843+0000","last_became_peered":"2024-03-06T07:25:55.182843+0000","last_unstale":"2024-03-06T07:26:05.735327+0000","last_undegraded":"2024-03-06T07:26:05.735327+0000","last_fullsized":"2024-03-06T07:26:05.735327+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:52.531694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736933+0000","last_change":"2024-03-06T07:25:54.172691+0000","last_active":"2024-03-06T07:26:05.736933+0000","last_peered":"2024-03-06T07:26:05.736933+0000","last_clean":"2024-03-06T07:26:05.736933+0000","last_became_active":"2024-03-06T07:25:54.172355+0000","last_became_peered":"2024-03-06T07:25:54.172355+0000","last_unstale":"2024-03-06T07:26:05.736933+0000","last_undegraded":"2024-03-06T07:26:05.736933+0000","last_fullsized":"2024-03-06T07:26:05.736933+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.756681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735899+0000","last_change":"2024-03-06T07:26:04.232191+0000","last_active":"2024-03-06T07:26:05.735899+0000","last_peered":"2024-03-06T07:26:05.735899+0000","last_clean":"2024-03-06T07:26:05.735899+0000","last_became_active":"2024-03-06T07:26:04.231117+0000","last_became_peered":"2024-03-06T07:26:04.231117+0000","last_unstale":"2024-03-06T07:26:05.735899+0000","last_undegraded":"2024-03-06T07:26:05.735899+0000","last_fullsized":"2024-03-06T07:26:05.735899+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:04.508750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736119+0000","last_change":"2024-03-06T07:26:03.234210+0000","last_active":"2024-03-06T07:26:05.736119+0000","last_peered":"2024-03-06T07:26:05.736119+0000","last_clean":"2024-03-06T07:26:05.736119+0000","last_became_active":"2024-03-06T07:26:03.233949+0000","last_became_peered":"2024-03-06T07:26:03.233949+0000","last_unstale":"2024-03-06T07:26:05.736119+0000","last_undegraded":"2024-03-06T07:26:05.736119+0000","last_fullsized":"2024-03-06T07:26:05.736119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:48.748869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735094+0000","last_change":"2024-03-06T07:25:55.179180+0000","last_active":"2024-03-06T07:26:05.735094+0000","last_peered":"2024-03-06T07:26:05.735094+0000","last_clean":"2024-03-06T07:26:05.735094+0000","last_became_active":"2024-03-06T07:25:55.178890+0000","last_became_peered":"2024-03-06T07:25:55.178890+0000","last_unstale":"2024-03-06T07:26:05.735094+0000","last_undegraded":"2024-03-06T07:26:05.735094+0000","last_fullsized":"2024-03-06T07:26:05.735094+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:00.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233694+0000","last_change":"2024-03-06T07:25:54.181928+0000","last_active":"2024-03-06T07:26:05.233694+0000","last_peered":"2024-03-06T07:26:05.233694+0000","last_clean":"2024-03-06T07:26:05.233694+0000","last_became_active":"2024-03-06T07:25:54.181140+0000","last_became_peered":"2024-03-06T07:25:54.181140+0000","last_unstale":"2024-03-06T07:26:05.233694+0000","last_undegraded":"2024-03-06T07:26:05.233694+0000","last_fullsized":"2024-03-06T07:26:05.233694+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:57.291913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735985+0000","last_change":"2024-03-06T07:26:04.229256+0000","last_active":"2024-03-06T07:26:05.735985+0000","last_peered":"2024-03-06T07:26:05.735985+0000","last_clean":"2024-03-06T07:26:05.735985+0000","last_became_active":"2024-03-06T07:26:04.229001+0000","last_became_peered":"2024-03-06T07:26:04.229001+0000","last_unstale":"2024-03-06T07:26:05.735985+0000","last_undegraded":"2024-03-06T07:26:05.735985+0000","last_fullsized":"2024-03-06T07:26:05.735985+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:41.161598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736327+0000","last_change":"2024-03-06T07:26:03.232499+0000","last_active":"2024-03-06T07:26:05.736327+0000","last_peered":"2024-03-06T07:26:05.736327+0000","last_clean":"2024-03-06T07:26:05.736327+0000","last_became_active":"2024-03-06T07:26:03.232246+0000","last_became_peered":"2024-03-06T07:26:03.232246+0000","last_unstale":"2024-03-06T07:26:05.736327+0000","last_undegraded":"2024-03-06T07:26:05.736327+0000","last_fullsized":"2024-03-06T07:26:05.736327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:11:01.982286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233667+0000","last_change":"2024-03-06T07:25:55.192859+0000","last_active":"2024-03-06T07:26:05.233667+0000","last_peered":"2024-03-06T07:26:05.233667+0000","last_clean":"2024-03-06T07:26:05.233667+0000","last_became_active":"2024-03-06T07:25:55.192570+0000","last_became_peered":"2024-03-06T07:25:55.192570+0000","last_unstale":"2024-03-06T07:26:05.233667+0000","last_undegraded":"2024-03-06T07:26:05.233667+0000","last_fullsized":"2024-03-06T07:26:05.233667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:00.192724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735328+0000","last_change":"2024-03-06T07:25:54.180637+0000","last_active":"2024-03-06T07:26:05.735328+0000","last_peered":"2024-03-06T07:26:05.735328+0000","last_clean":"2024-03-06T07:26:05.735328+0000","last_became_active":"2024-03-06T07:25:54.180270+0000","last_became_peered":"2024-03-06T07:25:54.180270+0000","last_unstale":"2024-03-06T07:26:05.735328+0000","last_undegraded":"2024-03-06T07:26:05.735328+0000","last_fullsized":"2024-03-06T07:26:05.735328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:20.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735712+0000","last_change":"2024-03-06T07:26:04.228643+0000","last_active":"2024-03-06T07:26:05.735712+0000","last_peered":"2024-03-06T07:26:05.735712+0000","last_clean":"2024-03-06T07:26:05.735712+0000","last_became_active":"2024-03-06T07:26:04.228385+0000","last_became_peered":"2024-03-06T07:26:04.228385+0000","last_unstale":"2024-03-06T07:26:05.735712+0000","last_undegraded":"2024-03-06T07:26:05.735712+0000","last_fullsized":"2024-03-06T07:26:05.735712+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:53.397866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233946+0000","last_change":"2024-03-06T07:26:03.226146+0000","last_active":"2024-03-06T07:26:05.233946+0000","last_peered":"2024-03-06T07:26:05.233946+0000","last_clean":"2024-03-06T07:26:05.233946+0000","last_became_active":"2024-03-06T07:26:03.225906+0000","last_became_peered":"2024-03-06T07:26:03.225906+0000","last_unstale":"2024-03-06T07:26:05.233946+0000","last_undegraded":"2024-03-06T07:26:05.233946+0000","last_fullsized":"2024-03-06T07:26:05.233946+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:08.360132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233781+0000","last_change":"2024-03-06T07:25:55.194175+0000","last_active":"2024-03-06T07:26:05.233781+0000","last_peered":"2024-03-06T07:26:05.233781+0000","last_clean":"2024-03-06T07:26:05.233781+0000","last_became_active":"2024-03-06T07:25:55.192548+0000","last_became_peered":"2024-03-06T07:25:55.192548+0000","last_unstale":"2024-03-06T07:26:05.233781+0000","last_undegraded":"2024-03-06T07:26:05.233781+0000","last_fullsized":"2024-03-06T07:26:05.233781+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:09.456474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735363+0000","last_change":"2024-03-06T07:25:54.171781+0000","last_active":"2024-03-06T07:26:05.735363+0000","last_peered":"2024-03-06T07:26:05.735363+0000","last_clean":"2024-03-06T07:26:05.735363+0000","last_became_active":"2024-03-06T07:25:54.171466+0000","last_became_peered":"2024-03-06T07:25:54.171466+0000","last_unstale":"2024-03-06T07:26:05.735363+0000","last_undegraded":"2024-03-06T07:26:05.735363+0000","last_fullsized":"2024-03-06T07:26:05.735363+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:50.533388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737092+0000","last_change":"2024-03-06T07:26:04.227633+0000","last_active":"2024-03-06T07:26:05.737092+0000","last_peered":"2024-03-06T07:26:05.737092+0000","last_clean":"2024-03-06T07:26:05.737092+0000","last_became_active":"2024-03-06T07:26:04.227384+0000","last_became_peered":"2024-03-06T07:26:04.227384+0000","last_unstale":"2024-03-06T07:26:05.737092+0000","last_undegraded":"2024-03-06T07:26:05.737092+0000","last_fullsized":"2024-03-06T07:26:05.737092+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:34.455713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735997+0000","last_change":"2024-03-06T07:26:03.227238+0000","last_active":"2024-03-06T07:26:05.735997+0000","last_peered":"2024-03-06T07:26:05.735997+0000","last_clean":"2024-03-06T07:26:05.735997+0000","last_became_active":"2024-03-06T07:26:03.226984+0000","last_became_peered":"2024-03-06T07:26:03.226984+0000","last_unstale":"2024-03-06T07:26:05.735997+0000","last_undegraded":"2024-03-06T07:26:05.735997+0000","last_fullsized":"2024-03-06T07:26:05.735997+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:16.810241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736055+0000","last_change":"2024-03-06T07:25:55.173561+0000","last_active":"2024-03-06T07:26:05.736055+0000","last_peered":"2024-03-06T07:26:05.736055+0000","last_clean":"2024-03-06T07:26:05.736055+0000","last_became_active":"2024-03-06T07:25:55.173309+0000","last_became_peered":"2024-03-06T07:25:55.173309+0000","last_unstale":"2024-03-06T07:26:05.736055+0000","last_undegraded":"2024-03-06T07:26:05.736055+0000","last_fullsized":"2024-03-06T07:26:05.736055+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735126+0000","last_change":"2024-03-06T07:25:54.178755+0000","last_active":"2024-03-06T07:26:05.735126+0000","last_peered":"2024-03-06T07:26:05.735126+0000","last_clean":"2024-03-06T07:26:05.735126+0000","last_became_active":"2024-03-06T07:25:54.178492+0000","last_became_peered":"2024-03-06T07:25:54.178492+0000","last_unstale":"2024-03-06T07:26:05.735126+0000","last_undegraded":"2024-03-06T07:26:05.735126+0000","last_fullsized":"2024-03-06T07:26:05.735126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:18.439323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234211+0000","last_change":"2024-03-06T07:26:04.233227+0000","last_active":"2024-03-06T07:26:05.234211+0000","last_peered":"2024-03-06T07:26:05.234211+0000","last_clean":"2024-03-06T07:26:05.234211+0000","last_became_active":"2024-03-06T07:26:04.232960+0000","last_became_peered":"2024-03-06T07:26:04.232960+0000","last_unstale":"2024-03-06T07:26:05.234211+0000","last_undegraded":"2024-03-06T07:26:05.234211+0000","last_fullsized":"2024-03-06T07:26:05.234211+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:25.186145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734222+0000","last_change":"2024-03-06T07:26:03.231682+0000","last_active":"2024-03-06T07:26:05.734222+0000","last_peered":"2024-03-06T07:26:05.734222+0000","last_clean":"2024-03-06T07:26:05.734222+0000","last_became_active":"2024-03-06T07:26:03.231429+0000","last_became_peered":"2024-03-06T07:26:03.231429+0000","last_unstale":"2024-03-06T07:26:05.734222+0000","last_undegraded":"2024-03-06T07:26:05.734222+0000","last_fullsized":"2024-03-06T07:26:05.734222+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:32.804654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734523+0000","last_change":"2024-03-06T07:25:55.172471+0000","last_active":"2024-03-06T07:26:05.734523+0000","last_peered":"2024-03-06T07:26:05.734523+0000","last_clean":"2024-03-06T07:26:05.734523+0000","last_became_active":"2024-03-06T07:25:55.172189+0000","last_became_peered":"2024-03-06T07:25:55.172189+0000","last_unstale":"2024-03-06T07:26:05.734523+0000","last_undegraded":"2024-03-06T07:26:05.734523+0000","last_fullsized":"2024-03-06T07:26:05.734523+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:53.761906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234086+0000","last_change":"2024-03-06T07:25:54.171718+0000","last_active":"2024-03-06T07:26:05.234086+0000","last_peered":"2024-03-06T07:26:05.234086+0000","last_clean":"2024-03-06T07:26:05.234086+0000","last_became_active":"2024-03-06T07:25:54.171431+0000","last_became_peered":"2024-03-06T07:25:54.171431+0000","last_unstale":"2024-03-06T07:26:05.234086+0000","last_undegraded":"2024-03-06T07:26:05.234086+0000","last_fullsized":"2024-03-06T07:26:05.234086+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:51.464247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734939+0000","last_change":"2024-03-06T07:26:04.225352+0000","last_active":"2024-03-06T07:26:05.734939+0000","last_peered":"2024-03-06T07:26:05.734939+0000","last_clean":"2024-03-06T07:26:05.734939+0000","last_became_active":"2024-03-06T07:26:04.225106+0000","last_became_peered":"2024-03-06T07:26:04.225106+0000","last_unstale":"2024-03-06T07:26:05.734939+0000","last_undegraded":"2024-03-06T07:26:05.734939+0000","last_fullsized":"2024-03-06T07:26:05.734939+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:53:38.971076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239849+0000","last_change":"2024-03-06T07:26:03.226686+0000","last_active":"2024-03-06T07:26:05.239849+0000","last_peered":"2024-03-06T07:26:05.239849+0000","last_clean":"2024-03-06T07:26:05.239849+0000","last_became_active":"2024-03-06T07:26:03.226406+0000","last_became_peered":"2024-03-06T07:26:03.226406+0000","last_unstale":"2024-03-06T07:26:05.239849+0000","last_undegraded":"2024-03-06T07:26:05.239849+0000","last_fullsized":"2024-03-06T07:26:05.239849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:06.496358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735742+0000","last_change":"2024-03-06T07:25:55.184864+0000","last_active":"2024-03-06T07:26:05.735742+0000","last_peered":"2024-03-06T07:26:05.735742+0000","last_clean":"2024-03-06T07:26:05.735742+0000","last_became_active":"2024-03-06T07:25:55.184462+0000","last_became_peered":"2024-03-06T07:25:55.184462+0000","last_unstale":"2024-03-06T07:26:05.735742+0000","last_undegraded":"2024-03-06T07:26:05.735742+0000","last_fullsized":"2024-03-06T07:26:05.735742+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"150'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735866+0000","last_change":"2024-03-06T07:25:54.184297+0000","last_active":"2024-03-06T07:26:05.735866+0000","last_peered":"2024-03-06T07:26:05.735866+0000","last_clean":"2024-03-06T07:26:05.735866+0000","last_became_active":"2024-03-06T07:25:54.184000+0000","last_became_peered":"2024-03-06T07:25:54.184000+0000","last_unstale":"2024-03-06T07:26:05.735866+0000","last_undegraded":"2024-03-06T07:26:05.735866+0000","last_fullsized":"2024-03-06T07:26:05.735866+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:06.525995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734539+0000","last_change":"2024-03-06T07:26:04.228678+0000","last_active":"2024-03-06T07:26:05.734539+0000","last_peered":"2024-03-06T07:26:05.734539+0000","last_clean":"2024-03-06T07:26:05.734539+0000","last_became_active":"2024-03-06T07:26:04.228431+0000","last_became_peered":"2024-03-06T07:26:04.228431+0000","last_unstale":"2024-03-06T07:26:05.734539+0000","last_undegraded":"2024-03-06T07:26:05.734539+0000","last_fullsized":"2024-03-06T07:26:05.734539+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:12.665130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239204+0000","last_change":"2024-03-06T07:26:03.226063+0000","last_active":"2024-03-06T07:26:05.239204+0000","last_peered":"2024-03-06T07:26:05.239204+0000","last_clean":"2024-03-06T07:26:05.239204+0000","last_became_active":"2024-03-06T07:26:03.225610+0000","last_became_peered":"2024-03-06T07:26:03.225610+0000","last_unstale":"2024-03-06T07:26:05.239204+0000","last_undegraded":"2024-03-06T07:26:05.239204+0000","last_fullsized":"2024-03-06T07:26:05.239204+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:24.862106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735716+0000","last_change":"2024-03-06T07:25:55.173629+0000","last_active":"2024-03-06T07:26:05.735716+0000","last_peered":"2024-03-06T07:26:05.735716+0000","last_clean":"2024-03-06T07:26:05.735716+0000","last_became_active":"2024-03-06T07:25:55.173365+0000","last_became_peered":"2024-03-06T07:25:55.173365+0000","last_unstale":"2024-03-06T07:26:05.735716+0000","last_undegraded":"2024-03-06T07:26:05.735716+0000","last_fullsized":"2024-03-06T07:26:05.735716+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:24.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736321+0000","last_change":"2024-03-06T07:25:54.180119+0000","last_active":"2024-03-06T07:26:05.736321+0000","last_peered":"2024-03-06T07:26:05.736321+0000","last_clean":"2024-03-06T07:26:05.736321+0000","last_became_active":"2024-03-06T07:25:54.179853+0000","last_became_peered":"2024-03-06T07:25:54.179853+0000","last_unstale":"2024-03-06T07:26:05.736321+0000","last_undegraded":"2024-03-06T07:26:05.736321+0000","last_fullsized":"2024-03-06T07:26:05.736321+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:43.160051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233271+0000","last_change":"2024-03-06T07:26:04.232203+0000","last_active":"2024-03-06T07:26:05.233271+0000","last_peered":"2024-03-06T07:26:05.233271+0000","last_clean":"2024-03-06T07:26:05.233271+0000","last_became_active":"2024-03-06T07:26:04.231938+0000","last_became_peered":"2024-03-06T07:26:04.231938+0000","last_unstale":"2024-03-06T07:26:05.233271+0000","last_undegraded":"2024-03-06T07:26:05.233271+0000","last_fullsized":"2024-03-06T07:26:05.233271+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:16.379024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233291+0000","last_change":"2024-03-06T07:26:03.226739+0000","last_active":"2024-03-06T07:26:05.233291+0000","last_peered":"2024-03-06T07:26:05.233291+0000","last_clean":"2024-03-06T07:26:05.233291+0000","last_became_active":"2024-03-06T07:26:03.226483+0000","last_became_peered":"2024-03-06T07:26:03.226483+0000","last_unstale":"2024-03-06T07:26:05.233291+0000","last_undegraded":"2024-03-06T07:26:05.233291+0000","last_fullsized":"2024-03-06T07:26:05.233291+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:04.216343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736674+0000","last_change":"2024-03-06T07:25:55.184654+0000","last_active":"2024-03-06T07:26:05.736674+0000","last_peered":"2024-03-06T07:26:05.736674+0000","last_clean":"2024-03-06T07:26:05.736674+0000","last_became_active":"2024-03-06T07:25:55.184305+0000","last_became_peered":"2024-03-06T07:25:55.184305+0000","last_unstale":"2024-03-06T07:26:05.736674+0000","last_undegraded":"2024-03-06T07:26:05.736674+0000","last_fullsized":"2024-03-06T07:26:05.736674+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735859+0000","last_change":"2024-03-06T07:25:54.174742+0000","last_active":"2024-03-06T07:26:05.735859+0000","last_peered":"2024-03-06T07:26:05.735859+0000","last_clean":"2024-03-06T07:26:05.735859+0000","last_became_active":"2024-03-06T07:25:54.174462+0000","last_became_peered":"2024-03-06T07:25:54.174462+0000","last_unstale":"2024-03-06T07:26:05.735859+0000","last_undegraded":"2024-03-06T07:26:05.735859+0000","last_fullsized":"2024-03-06T07:26:05.735859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:47:43.050877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736288+0000","last_change":"2024-03-06T07:26:04.238292+0000","last_active":"2024-03-06T07:26:05.736288+0000","last_peered":"2024-03-06T07:26:05.736288+0000","last_clean":"2024-03-06T07:26:05.736288+0000","last_became_active":"2024-03-06T07:26:04.237876+0000","last_became_peered":"2024-03-06T07:26:04.237876+0000","last_unstale":"2024-03-06T07:26:05.736288+0000","last_undegraded":"2024-03-06T07:26:05.736288+0000","last_fullsized":"2024-03-06T07:26:05.736288+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:09.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240046+0000","last_change":"2024-03-06T07:26:03.233488+0000","last_active":"2024-03-06T07:26:05.240046+0000","last_peered":"2024-03-06T07:26:05.240046+0000","last_clean":"2024-03-06T07:26:05.240046+0000","last_became_active":"2024-03-06T07:26:03.233211+0000","last_became_peered":"2024-03-06T07:26:03.233211+0000","last_unstale":"2024-03-06T07:26:05.240046+0000","last_undegraded":"2024-03-06T07:26:05.240046+0000","last_fullsized":"2024-03-06T07:26:05.240046+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:27.486285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239291+0000","last_change":"2024-03-06T07:25:55.181956+0000","last_active":"2024-03-06T07:26:05.239291+0000","last_peered":"2024-03-06T07:26:05.239291+0000","last_clean":"2024-03-06T07:26:05.239291+0000","last_became_active":"2024-03-06T07:25:55.181658+0000","last_became_peered":"2024-03-06T07:25:55.181658+0000","last_unstale":"2024-03-06T07:26:05.239291+0000","last_undegraded":"2024-03-06T07:26:05.239291+0000","last_fullsized":"2024-03-06T07:26:05.239291+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:23.152358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240044+0000","last_change":"2024-03-06T07:25:54.172421+0000","last_active":"2024-03-06T07:26:05.240044+0000","last_peered":"2024-03-06T07:26:05.240044+0000","last_clean":"2024-03-06T07:26:05.240044+0000","last_became_active":"2024-03-06T07:25:54.172157+0000","last_became_peered":"2024-03-06T07:25:54.172157+0000","last_unstale":"2024-03-06T07:26:05.240044+0000","last_undegraded":"2024-03-06T07:26:05.240044+0000","last_fullsized":"2024-03-06T07:26:05.240044+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:07.283064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737486+0000","last_change":"2024-03-06T07:26:04.228518+0000","last_active":"2024-03-06T07:26:05.737486+0000","last_peered":"2024-03-06T07:26:05.737486+0000","last_clean":"2024-03-06T07:26:05.737486+0000","last_became_active":"2024-03-06T07:26:04.228278+0000","last_became_peered":"2024-03-06T07:26:04.228278+0000","last_unstale":"2024-03-06T07:26:05.737486+0000","last_undegraded":"2024-03-06T07:26:05.737486+0000","last_fullsized":"2024-03-06T07:26:05.737486+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:49.758839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735231+0000","last_change":"2024-03-06T07:26:03.226623+0000","last_active":"2024-03-06T07:26:05.735231+0000","last_peered":"2024-03-06T07:26:05.735231+0000","last_clean":"2024-03-06T07:26:05.735231+0000","last_became_active":"2024-03-06T07:26:03.226351+0000","last_became_peered":"2024-03-06T07:26:03.226351+0000","last_unstale":"2024-03-06T07:26:05.735231+0000","last_undegraded":"2024-03-06T07:26:05.735231+0000","last_fullsized":"2024-03-06T07:26:05.735231+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:06.338875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735662+0000","last_change":"2024-03-06T07:25:55.183461+0000","last_active":"2024-03-06T07:26:05.735662+0000","last_peered":"2024-03-06T07:26:05.735662+0000","last_clean":"2024-03-06T07:26:05.735662+0000","last_became_active":"2024-03-06T07:25:55.183161+0000","last_became_peered":"2024-03-06T07:25:55.183161+0000","last_unstale":"2024-03-06T07:26:05.735662+0000","last_undegraded":"2024-03-06T07:26:05.735662+0000","last_fullsized":"2024-03-06T07:26:05.735662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735767+0000","last_change":"2024-03-06T07:25:54.182221+0000","last_active":"2024-03-06T07:26:05.735767+0000","last_peered":"2024-03-06T07:26:05.735767+0000","last_clean":"2024-03-06T07:26:05.735767+0000","last_became_active":"2024-03-06T07:25:54.181966+0000","last_became_peered":"2024-03-06T07:25:54.181966+0000","last_unstale":"2024-03-06T07:26:05.735767+0000","last_undegraded":"2024-03-06T07:26:05.735767+0000","last_fullsized":"2024-03-06T07:26:05.735767+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:48.357604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735561+0000","last_change":"2024-03-06T07:26:04.231324+0000","last_active":"2024-03-06T07:26:05.735561+0000","last_peered":"2024-03-06T07:26:05.735561+0000","last_clean":"2024-03-06T07:26:05.735561+0000","last_became_active":"2024-03-06T07:26:04.231041+0000","last_became_peered":"2024-03-06T07:26:04.231041+0000","last_unstale":"2024-03-06T07:26:05.735561+0000","last_undegraded":"2024-03-06T07:26:05.735561+0000","last_fullsized":"2024-03-06T07:26:05.735561+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:09.032723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736198+0000","last_change":"2024-03-06T07:26:03.232763+0000","last_active":"2024-03-06T07:26:05.736198+0000","last_peered":"2024-03-06T07:26:05.736198+0000","last_clean":"2024-03-06T07:26:05.736198+0000","last_became_active":"2024-03-06T07:26:03.232424+0000","last_became_peered":"2024-03-06T07:26:03.232424+0000","last_unstale":"2024-03-06T07:26:05.736198+0000","last_undegraded":"2024-03-06T07:26:05.736198+0000","last_fullsized":"2024-03-06T07:26:05.736198+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:01.280176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735694+0000","last_change":"2024-03-06T07:25:55.178896+0000","last_active":"2024-03-06T07:26:05.735694+0000","last_peered":"2024-03-06T07:26:05.735694+0000","last_clean":"2024-03-06T07:26:05.735694+0000","last_became_active":"2024-03-06T07:25:55.178599+0000","last_became_peered":"2024-03-06T07:25:55.178599+0000","last_unstale":"2024-03-06T07:26:05.735694+0000","last_undegraded":"2024-03-06T07:26:05.735694+0000","last_fullsized":"2024-03-06T07:26:05.735694+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:34.372833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735130+0000","last_change":"2024-03-06T07:25:54.176751+0000","last_active":"2024-03-06T07:26:05.735130+0000","last_peered":"2024-03-06T07:26:05.735130+0000","last_clean":"2024-03-06T07:26:05.735130+0000","last_became_active":"2024-03-06T07:25:54.176461+0000","last_became_peered":"2024-03-06T07:25:54.176461+0000","last_unstale":"2024-03-06T07:26:05.735130+0000","last_undegraded":"2024-03-06T07:26:05.735130+0000","last_fullsized":"2024-03-06T07:26:05.735130+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:09.865297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735143+0000","last_change":"2024-03-06T07:26:04.239654+0000","last_active":"2024-03-06T07:26:05.735143+0000","last_peered":"2024-03-06T07:26:05.735143+0000","last_clean":"2024-03-06T07:26:05.735143+0000","last_became_active":"2024-03-06T07:26:04.239353+0000","last_became_peered":"2024-03-06T07:26:04.239353+0000","last_unstale":"2024-03-06T07:26:05.735143+0000","last_undegraded":"2024-03-06T07:26:05.735143+0000","last_fullsized":"2024-03-06T07:26:05.735143+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:27.236434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233614+0000","last_change":"2024-03-06T07:26:03.218889+0000","last_active":"2024-03-06T07:26:05.233614+0000","last_peered":"2024-03-06T07:26:05.233614+0000","last_clean":"2024-03-06T07:26:05.233614+0000","last_became_active":"2024-03-06T07:26:03.218638+0000","last_became_peered":"2024-03-06T07:26:03.218638+0000","last_unstale":"2024-03-06T07:26:05.233614+0000","last_undegraded":"2024-03-06T07:26:05.233614+0000","last_fullsized":"2024-03-06T07:26:05.233614+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.498780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736254+0000","last_change":"2024-03-06T07:25:55.184289+0000","last_active":"2024-03-06T07:26:05.736254+0000","last_peered":"2024-03-06T07:26:05.736254+0000","last_clean":"2024-03-06T07:26:05.736254+0000","last_became_active":"2024-03-06T07:25:55.184007+0000","last_became_peered":"2024-03-06T07:25:55.184007+0000","last_unstale":"2024-03-06T07:26:05.736254+0000","last_undegraded":"2024-03-06T07:26:05.736254+0000","last_fullsized":"2024-03-06T07:26:05.736254+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:09.480817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233157+0000","last_change":"2024-03-06T07:25:54.182027+0000","last_active":"2024-03-06T07:26:05.233157+0000","last_peered":"2024-03-06T07:26:05.233157+0000","last_clean":"2024-03-06T07:26:05.233157+0000","last_became_active":"2024-03-06T07:25:54.181755+0000","last_became_peered":"2024-03-06T07:25:54.181755+0000","last_unstale":"2024-03-06T07:26:05.233157+0000","last_undegraded":"2024-03-06T07:26:05.233157+0000","last_fullsized":"2024-03-06T07:26:05.233157+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:03.374674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240556+0000","last_change":"2024-03-06T07:26:04.241623+0000","last_active":"2024-03-06T07:26:05.240556+0000","last_peered":"2024-03-06T07:26:05.240556+0000","last_clean":"2024-03-06T07:26:05.240556+0000","last_became_active":"2024-03-06T07:26:04.241336+0000","last_became_peered":"2024-03-06T07:26:04.241336+0000","last_unstale":"2024-03-06T07:26:05.240556+0000","last_undegraded":"2024-03-06T07:26:05.240556+0000","last_fullsized":"2024-03-06T07:26:05.240556+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:39:01.583402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233871+0000","last_change":"2024-03-06T07:26:03.222524+0000","last_active":"2024-03-06T07:26:05.233871+0000","last_peered":"2024-03-06T07:26:05.233871+0000","last_clean":"2024-03-06T07:26:05.233871+0000","last_became_active":"2024-03-06T07:26:03.222275+0000","last_became_peered":"2024-03-06T07:26:03.222275+0000","last_unstale":"2024-03-06T07:26:05.233871+0000","last_undegraded":"2024-03-06T07:26:05.233871+0000","last_fullsized":"2024-03-06T07:26:05.233871+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:11.930055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734632+0000","last_change":"2024-03-06T07:25:55.176555+0000","last_active":"2024-03-06T07:26:05.734632+0000","last_peered":"2024-03-06T07:26:05.734632+0000","last_clean":"2024-03-06T07:26:05.734632+0000","last_became_active":"2024-03-06T07:25:55.176295+0000","last_became_peered":"2024-03-06T07:25:55.176295+0000","last_unstale":"2024-03-06T07:26:05.734632+0000","last_undegraded":"2024-03-06T07:26:05.734632+0000","last_fullsized":"2024-03-06T07:26:05.734632+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:18.514838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735777+0000","last_change":"2024-03-06T07:25:54.184373+0000","last_active":"2024-03-06T07:26:05.735777+0000","last_peered":"2024-03-06T07:26:05.735777+0000","last_clean":"2024-03-06T07:26:05.735777+0000","last_became_active":"2024-03-06T07:25:54.184109+0000","last_became_peered":"2024-03-06T07:25:54.184109+0000","last_unstale":"2024-03-06T07:26:05.735777+0000","last_undegraded":"2024-03-06T07:26:05.735777+0000","last_fullsized":"2024-03-06T07:26:05.735777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:01.955245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735233+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:26:05.735233+0000","last_peered":"2024-03-06T07:26:05.735233+0000","last_clean":"2024-03-06T07:26:05.735233+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:26:05.735233+0000","last_undegraded":"2024-03-06T07:26:05.735233+0000","last_fullsized":"2024-03-06T07:26:05.735233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2002944,"data_stored":1534716,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1941504,"data_stored":944418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":946106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":946758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1966080,"data_stored":946724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1941504,"data_stored":944930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":945960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1535548,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:08.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.514+0000 7f9e80b55640 1 Processor -- start 2024-03-06T07:26:08.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.514+0000 7f9e80b55640 1 -- start start 2024-03-06T07:26:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e80b55640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e7c159270 0x7f9e7c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e80b55640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7c0c73d0 con 0x7f9e7c0c88c0 2024-03-06T07:26:08.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e80b55640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7c0be2e0 con 0x7f9e7c14f360 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e80b55640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7c159700 con 0x7f9e7c159270 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e7ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9e7c159270 0x7f9e7c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e73fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9e7c0c88c0 0x7f9e7c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57316/0 (socket says 172.21.15.17:57316) 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.515+0000 7f9e73fff640 1 -- 172.21.15.17:0/3995687773 learned_addr learned my addr 172.21.15.17:0/3995687773 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.516+0000 7f9e7ad9d640 1 -- 172.21.15.17:0/3995687773 >> v1:172.21.15.163:6789/0 conn(0x7f9e7c14f360 legacy=0x7f9e7c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.516+0000 7f9e7ad9d640 1 -- 172.21.15.17:0/3995687773 >> v1:172.21.15.17:6789/0 conn(0x7f9e7c0c88c0 legacy=0x7f9e7c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.516+0000 7f9e7ad9d640 1 -- 172.21.15.17:0/3995687773 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c0c7670 con 0x7f9e7c159270 2024-03-06T07:26:08.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.516+0000 7f9e79d9b640 1 -- 172.21.15.17:0/3995687773 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525317700 0 0) 0x7f9e7c0c73d0 con 0x7f9e7c0c88c0 2024-03-06T07:26:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e7ad9d640 1 --2- 172.21.15.17:0/3995687773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c159270 0x7f9e7c155c10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009d90 tx=0x7f9e7402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea0c6dbe06b3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e79d9b640 1 -- 172.21.15.17:0/3995687773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74039070 con 0x7f9e7c159270 2024-03-06T07:26:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e79d9b640 1 -- 172.21.15.17:0/3995687773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7402cbd0 con 0x7f9e7c159270 2024-03-06T07:26:08.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e79d9b640 1 -- 172.21.15.17:0/3995687773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74034440 con 0x7f9e7c159270 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 -- 172.21.15.17:0/3995687773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c159270 msgr2=0x7f9e7c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3995687773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c159270 0x7f9e7c155c10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009d90 tx=0x7f9e7402be20 comp rx=0 tx=0).stop 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 -- 172.21.15.17:0/3995687773 shutdown_connections 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3995687773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c159270 0x7f9e7c155c10 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 -- 172.21.15.17:0/3995687773 >> 172.21.15.17:0/3995687773 conn(0x7f9e7c0ac570 msgr2=0x7f9e7c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 -- 172.21.15.17:0/3995687773 shutdown_connections 2024-03-06T07:26:08.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.517+0000 7f9e80b55640 1 -- 172.21.15.17:0/3995687773 wait complete. 2024-03-06T07:26:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.519+0000 7f9e80b55640 1 Processor -- start 2024-03-06T07:26:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.519+0000 7f9e80b55640 1 -- start start 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e7c0c88c0 0x7f9e7c1eb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c159270 0x7f9e7c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0c7c30 con 0x7f9e7c14f360 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0c5a50 con 0x7f9e7c159270 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e80b55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c0c5bd0 con 0x7f9e7c0c88c0 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39724/0 (socket says 172.21.15.17:39724) 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e73fff640 1 -- 172.21.15.17:0/3429980526 learned_addr learned my addr 172.21.15.17:0/3429980526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e7ad9d640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c159270 0x7f9e7c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.520+0000 7f9e7a59c640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e7c0c88c0 0x7f9e7c1eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 -- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e7c0c88c0 msgr2=0x7f9e7c1eb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e7c0c88c0 0x7f9e7c1eb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 -- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c159270 msgr2=0x7f9e7c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c159270 0x7f9e7c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 -- 172.21.15.17:0/3429980526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c1527a0 con 0x7f9e7c14f360 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e73fff640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c4b0 tx=0x7f9e6400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64018070 con 0x7f9e7c14f360 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e64007500 con 0x7f9e7c14f360 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7c1eb8c0 con 0x7f9e7c14f360 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e64004a20 con 0x7f9e7c14f360 2024-03-06T07:26:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.521+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c14f870 con 0x7f9e7c14f360 2024-03-06T07:26:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.522+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c04f330 con 0x7f9e7c14f360 2024-03-06T07:26:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.522+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e6401b070 con 0x7f9e7c14f360 2024-03-06T07:26:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.523+0000 7f9e72ffd640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 0x7f9e5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.523+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f9e640898d0 con 0x7f9e7c14f360 2024-03-06T07:26:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.523+0000 7f9e7a59c640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 0x7f9e5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.524+0000 7f9e7a59c640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 0x7f9e5c06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c004f60 tx=0x7f9e6c007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.527+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e7c1eb8c0 con 0x7f9e7c14f360 2024-03-06T07:26:08.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.704+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e7c0c8d50 con 0x7f9e5c06b4d0 2024-03-06T07:26:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.715+0000 7f9e72ffd640 1 -- 172.21.15.17:0/3429980526 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+491533 (secure 0 0 0) 0x7f9e7c0c8d50 con 0x7f9e5c06b4d0 2024-03-06T07:26:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:08.727 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:26:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 msgr2=0x7f9e5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 0x7f9e5c06d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c004f60 tx=0x7f9e6c007e30 comp rx=0 tx=0).stop 2024-03-06T07:26:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 msgr2=0x7f9e7c1519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9e6400c4b0 tx=0x7f9e6400c970 comp rx=0 tx=0).stop 2024-03-06T07:26:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 shutdown_connections 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e5c06b4d0 0x7f9e5c06d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e7c159270 0x7f9e7c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e7c14f360 0x7f9e7c1519a0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 --2- 172.21.15.17:0/3429980526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e7c0c88c0 0x7f9e7c1eb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 >> 172.21.15.17:0/3429980526 conn(0x7f9e7c0ac570 msgr2=0x7f9e7c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 shutdown_connections 2024-03-06T07:26:08.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.719+0000 7f9e80b55640 1 -- 172.21.15.17:0/3429980526 wait complete. 2024-03-06T07:26:08.732 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-06T07:26:07.421246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":249,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":357,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264860,"kb_used_data":15392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709860,"statfs":{"total":773094113280,"available":772822896640,"internally_reserved":0,"allocated":15761408,"data_stored":8745160,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.212984"},"pg_stats":[{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239997+0000","last_change":"2024-03-06T07:26:04.229926+0000","last_active":"2024-03-06T07:26:05.239997+0000","last_peered":"2024-03-06T07:26:05.239997+0000","last_clean":"2024-03-06T07:26:05.239997+0000","last_became_active":"2024-03-06T07:26:04.229626+0000","last_became_peered":"2024-03-06T07:26:04.229626+0000","last_unstale":"2024-03-06T07:26:05.239997+0000","last_undegraded":"2024-03-06T07:26:05.239997+0000","last_fullsized":"2024-03-06T07:26:05.239997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:34.836516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238917+0000","last_change":"2024-03-06T07:25:55.171050+0000","last_active":"2024-03-06T07:26:05.238917+0000","last_peered":"2024-03-06T07:26:05.238917+0000","last_clean":"2024-03-06T07:26:05.238917+0000","last_became_active":"2024-03-06T07:25:55.170737+0000","last_became_peered":"2024-03-06T07:25:55.170737+0000","last_unstale":"2024-03-06T07:26:05.238917+0000","last_undegraded":"2024-03-06T07:26:05.238917+0000","last_fullsized":"2024-03-06T07:26:05.238917+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:46.814853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234080+0000","last_change":"2024-03-06T07:26:04.232419+0000","last_active":"2024-03-06T07:26:05.234080+0000","last_peered":"2024-03-06T07:26:05.234080+0000","last_clean":"2024-03-06T07:26:05.234080+0000","last_became_active":"2024-03-06T07:26:04.230470+0000","last_became_peered":"2024-03-06T07:26:04.230470+0000","last_unstale":"2024-03-06T07:26:05.234080+0000","last_undegraded":"2024-03-06T07:26:05.234080+0000","last_fullsized":"2024-03-06T07:26:05.234080+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:34.455713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736021+0000","last_change":"2024-03-06T07:25:55.179421+0000","last_active":"2024-03-06T07:26:05.736021+0000","last_peered":"2024-03-06T07:26:05.736021+0000","last_clean":"2024-03-06T07:26:05.736021+0000","last_became_active":"2024-03-06T07:25:55.179145+0000","last_became_peered":"2024-03-06T07:25:55.179145+0000","last_unstale":"2024-03-06T07:26:05.736021+0000","last_undegraded":"2024-03-06T07:26:05.736021+0000","last_fullsized":"2024-03-06T07:26:05.736021+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736957+0000","last_change":"2024-03-06T07:26:04.234996+0000","last_active":"2024-03-06T07:26:05.736957+0000","last_peered":"2024-03-06T07:26:05.736957+0000","last_clean":"2024-03-06T07:26:05.736957+0000","last_became_active":"2024-03-06T07:26:04.234723+0000","last_became_peered":"2024-03-06T07:26:04.234723+0000","last_unstale":"2024-03-06T07:26:05.736957+0000","last_undegraded":"2024-03-06T07:26:05.736957+0000","last_fullsized":"2024-03-06T07:26:05.736957+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:16.379024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240371+0000","last_change":"2024-03-06T07:25:55.178609+0000","last_active":"2024-03-06T07:26:05.240371+0000","last_peered":"2024-03-06T07:26:05.240371+0000","last_clean":"2024-03-06T07:26:05.240371+0000","last_became_active":"2024-03-06T07:25:55.178334+0000","last_became_peered":"2024-03-06T07:25:55.178334+0000","last_unstale":"2024-03-06T07:26:05.240371+0000","last_undegraded":"2024-03-06T07:26:05.240371+0000","last_fullsized":"2024-03-06T07:26:05.240371+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:19.444548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737016+0000","last_change":"2024-03-06T07:26:04.237958+0000","last_active":"2024-03-06T07:26:05.737016+0000","last_peered":"2024-03-06T07:26:05.737016+0000","last_clean":"2024-03-06T07:26:05.737016+0000","last_became_active":"2024-03-06T07:26:04.237445+0000","last_became_peered":"2024-03-06T07:26:04.237445+0000","last_unstale":"2024-03-06T07:26:05.737016+0000","last_undegraded":"2024-03-06T07:26:05.737016+0000","last_fullsized":"2024-03-06T07:26:05.737016+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:47.243949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239188+0000","last_change":"2024-03-06T07:25:55.181844+0000","last_active":"2024-03-06T07:26:05.239188+0000","last_peered":"2024-03-06T07:26:05.239188+0000","last_clean":"2024-03-06T07:26:05.239188+0000","last_became_active":"2024-03-06T07:25:55.181282+0000","last_became_peered":"2024-03-06T07:25:55.181282+0000","last_unstale":"2024-03-06T07:26:05.239188+0000","last_undegraded":"2024-03-06T07:26:05.239188+0000","last_fullsized":"2024-03-06T07:26:05.239188+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:58.454621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735040+0000","last_change":"2024-03-06T07:26:04.229646+0000","last_active":"2024-03-06T07:26:05.735040+0000","last_peered":"2024-03-06T07:26:05.735040+0000","last_clean":"2024-03-06T07:26:05.735040+0000","last_became_active":"2024-03-06T07:26:04.229225+0000","last_became_peered":"2024-03-06T07:26:04.229225+0000","last_unstale":"2024-03-06T07:26:05.735040+0000","last_undegraded":"2024-03-06T07:26:05.735040+0000","last_fullsized":"2024-03-06T07:26:05.735040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:17.339996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233491+0000","last_change":"2024-03-06T07:25:55.188179+0000","last_active":"2024-03-06T07:26:05.233491+0000","last_peered":"2024-03-06T07:26:05.233491+0000","last_clean":"2024-03-06T07:26:05.233491+0000","last_became_active":"2024-03-06T07:25:55.187905+0000","last_became_peered":"2024-03-06T07:25:55.187905+0000","last_unstale":"2024-03-06T07:26:05.233491+0000","last_undegraded":"2024-03-06T07:26:05.233491+0000","last_fullsized":"2024-03-06T07:26:05.233491+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:31.959459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734882+0000","last_change":"2024-03-06T07:26:04.233412+0000","last_active":"2024-03-06T07:26:05.734882+0000","last_peered":"2024-03-06T07:26:05.734882+0000","last_clean":"2024-03-06T07:26:05.734882+0000","last_became_active":"2024-03-06T07:26:04.232888+0000","last_became_peered":"2024-03-06T07:26:04.232888+0000","last_unstale":"2024-03-06T07:26:05.734882+0000","last_undegraded":"2024-03-06T07:26:05.734882+0000","last_fullsized":"2024-03-06T07:26:05.734882+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:54.282221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734702+0000","last_change":"2024-03-06T07:25:55.179259+0000","last_active":"2024-03-06T07:26:05.734702+0000","last_peered":"2024-03-06T07:26:05.734702+0000","last_clean":"2024-03-06T07:26:05.734702+0000","last_became_active":"2024-03-06T07:25:55.178971+0000","last_became_peered":"2024-03-06T07:25:55.178971+0000","last_unstale":"2024-03-06T07:26:05.734702+0000","last_undegraded":"2024-03-06T07:26:05.734702+0000","last_fullsized":"2024-03-06T07:26:05.734702+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:52.469471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734671+0000","last_change":"2024-03-06T07:26:04.231613+0000","last_active":"2024-03-06T07:26:05.734671+0000","last_peered":"2024-03-06T07:26:05.734671+0000","last_clean":"2024-03-06T07:26:05.734671+0000","last_became_active":"2024-03-06T07:26:04.231368+0000","last_became_peered":"2024-03-06T07:26:04.231368+0000","last_unstale":"2024-03-06T07:26:05.734671+0000","last_undegraded":"2024-03-06T07:26:05.734671+0000","last_fullsized":"2024-03-06T07:26:05.734671+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:47:32.806030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735512+0000","last_change":"2024-03-06T07:25:55.189198+0000","last_active":"2024-03-06T07:26:05.735512+0000","last_peered":"2024-03-06T07:26:05.735512+0000","last_clean":"2024-03-06T07:26:05.735512+0000","last_became_active":"2024-03-06T07:25:55.188937+0000","last_became_peered":"2024-03-06T07:25:55.188937+0000","last_unstale":"2024-03-06T07:26:05.735512+0000","last_undegraded":"2024-03-06T07:26:05.735512+0000","last_fullsized":"2024-03-06T07:26:05.735512+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735116+0000","last_change":"2024-03-06T07:26:04.229595+0000","last_active":"2024-03-06T07:26:05.735116+0000","last_peered":"2024-03-06T07:26:05.735116+0000","last_clean":"2024-03-06T07:26:05.735116+0000","last_became_active":"2024-03-06T07:26:04.229347+0000","last_became_peered":"2024-03-06T07:26:04.229347+0000","last_unstale":"2024-03-06T07:26:05.735116+0000","last_undegraded":"2024-03-06T07:26:05.735116+0000","last_fullsized":"2024-03-06T07:26:05.735116+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:33:23.378545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734586+0000","last_change":"2024-03-06T07:25:55.174913+0000","last_active":"2024-03-06T07:26:05.734586+0000","last_peered":"2024-03-06T07:26:05.734586+0000","last_clean":"2024-03-06T07:26:05.734586+0000","last_became_active":"2024-03-06T07:25:55.174627+0000","last_became_peered":"2024-03-06T07:25:55.174627+0000","last_unstale":"2024-03-06T07:26:05.734586+0000","last_undegraded":"2024-03-06T07:26:05.734586+0000","last_fullsized":"2024-03-06T07:26:05.734586+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734949+0000","last_change":"2024-03-06T07:26:04.230891+0000","last_active":"2024-03-06T07:26:05.734949+0000","last_peered":"2024-03-06T07:26:05.734949+0000","last_clean":"2024-03-06T07:26:05.734949+0000","last_became_active":"2024-03-06T07:26:04.230580+0000","last_became_peered":"2024-03-06T07:26:04.230580+0000","last_unstale":"2024-03-06T07:26:05.734949+0000","last_undegraded":"2024-03-06T07:26:05.734949+0000","last_fullsized":"2024-03-06T07:26:05.734949+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:12.012177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737488+0000","last_change":"2024-03-06T07:25:55.175881+0000","last_active":"2024-03-06T07:26:05.737488+0000","last_peered":"2024-03-06T07:26:05.737488+0000","last_clean":"2024-03-06T07:26:05.737488+0000","last_became_active":"2024-03-06T07:25:55.175604+0000","last_became_peered":"2024-03-06T07:25:55.175604+0000","last_unstale":"2024-03-06T07:26:05.737488+0000","last_undegraded":"2024-03-06T07:26:05.737488+0000","last_fullsized":"2024-03-06T07:26:05.737488+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:18.989187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233654+0000","last_change":"2024-03-06T07:26:04.230741+0000","last_active":"2024-03-06T07:26:05.233654+0000","last_peered":"2024-03-06T07:26:05.233654+0000","last_clean":"2024-03-06T07:26:05.233654+0000","last_became_active":"2024-03-06T07:26:04.230058+0000","last_became_peered":"2024-03-06T07:26:04.230058+0000","last_unstale":"2024-03-06T07:26:05.233654+0000","last_undegraded":"2024-03-06T07:26:05.233654+0000","last_fullsized":"2024-03-06T07:26:05.233654+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:18:47.243949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737255+0000","last_change":"2024-03-06T07:25:55.174883+0000","last_active":"2024-03-06T07:26:05.737255+0000","last_peered":"2024-03-06T07:26:05.737255+0000","last_clean":"2024-03-06T07:26:05.737255+0000","last_became_active":"2024-03-06T07:25:55.174590+0000","last_became_peered":"2024-03-06T07:25:55.174590+0000","last_unstale":"2024-03-06T07:26:05.737255+0000","last_undegraded":"2024-03-06T07:26:05.737255+0000","last_fullsized":"2024-03-06T07:26:05.737255+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737375+0000","last_change":"2024-03-06T07:26:04.238866+0000","last_active":"2024-03-06T07:26:05.737375+0000","last_peered":"2024-03-06T07:26:05.737375+0000","last_clean":"2024-03-06T07:26:05.737375+0000","last_became_active":"2024-03-06T07:26:04.238193+0000","last_became_peered":"2024-03-06T07:26:04.238193+0000","last_unstale":"2024-03-06T07:26:05.737375+0000","last_undegraded":"2024-03-06T07:26:05.737375+0000","last_fullsized":"2024-03-06T07:26:05.737375+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:25.186145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735440+0000","last_change":"2024-03-06T07:25:55.182365+0000","last_active":"2024-03-06T07:26:05.735440+0000","last_peered":"2024-03-06T07:26:05.735440+0000","last_clean":"2024-03-06T07:26:05.735440+0000","last_became_active":"2024-03-06T07:25:55.182083+0000","last_became_peered":"2024-03-06T07:25:55.182083+0000","last_unstale":"2024-03-06T07:26:05.735440+0000","last_undegraded":"2024-03-06T07:26:05.735440+0000","last_fullsized":"2024-03-06T07:26:05.735440+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:24:53.751341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735925+0000","last_change":"2024-03-06T07:26:04.220512+0000","last_active":"2024-03-06T07:26:05.735925+0000","last_peered":"2024-03-06T07:26:05.735925+0000","last_clean":"2024-03-06T07:26:05.735925+0000","last_became_active":"2024-03-06T07:26:04.220222+0000","last_became_peered":"2024-03-06T07:26:04.220222+0000","last_unstale":"2024-03-06T07:26:05.735925+0000","last_undegraded":"2024-03-06T07:26:05.735925+0000","last_fullsized":"2024-03-06T07:26:05.735925+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:30:58.340820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735742+0000","last_change":"2024-03-06T07:25:55.179820+0000","last_active":"2024-03-06T07:26:05.735742+0000","last_peered":"2024-03-06T07:26:05.735742+0000","last_clean":"2024-03-06T07:26:05.735742+0000","last_became_active":"2024-03-06T07:25:55.179573+0000","last_became_peered":"2024-03-06T07:25:55.179573+0000","last_unstale":"2024-03-06T07:26:05.735742+0000","last_undegraded":"2024-03-06T07:26:05.735742+0000","last_fullsized":"2024-03-06T07:26:05.735742+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:49:53.238439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735695+0000","last_change":"2024-03-06T07:26:04.234781+0000","last_active":"2024-03-06T07:26:05.735695+0000","last_peered":"2024-03-06T07:26:05.735695+0000","last_clean":"2024-03-06T07:26:05.735695+0000","last_became_active":"2024-03-06T07:26:04.234368+0000","last_became_peered":"2024-03-06T07:26:04.234368+0000","last_unstale":"2024-03-06T07:26:05.735695+0000","last_undegraded":"2024-03-06T07:26:05.735695+0000","last_fullsized":"2024-03-06T07:26:05.735695+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:11.750221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734840+0000","last_change":"2024-03-06T07:25:55.178116+0000","last_active":"2024-03-06T07:26:05.734840+0000","last_peered":"2024-03-06T07:26:05.734840+0000","last_clean":"2024-03-06T07:26:05.734840+0000","last_became_active":"2024-03-06T07:25:55.177769+0000","last_became_peered":"2024-03-06T07:25:55.177769+0000","last_unstale":"2024-03-06T07:26:05.734840+0000","last_undegraded":"2024-03-06T07:26:05.734840+0000","last_fullsized":"2024-03-06T07:26:05.734840+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:19.818533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734678+0000","last_change":"2024-03-06T07:26:04.228971+0000","last_active":"2024-03-06T07:26:05.734678+0000","last_peered":"2024-03-06T07:26:05.734678+0000","last_clean":"2024-03-06T07:26:05.734678+0000","last_became_active":"2024-03-06T07:26:04.228716+0000","last_became_peered":"2024-03-06T07:26:04.228716+0000","last_unstale":"2024-03-06T07:26:05.734678+0000","last_undegraded":"2024-03-06T07:26:05.734678+0000","last_fullsized":"2024-03-06T07:26:05.734678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:20.406031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233509+0000","last_change":"2024-03-06T07:25:55.185952+0000","last_active":"2024-03-06T07:26:05.233509+0000","last_peered":"2024-03-06T07:26:05.233509+0000","last_clean":"2024-03-06T07:26:05.233509+0000","last_became_active":"2024-03-06T07:25:55.185677+0000","last_became_peered":"2024-03-06T07:25:55.185677+0000","last_unstale":"2024-03-06T07:26:05.233509+0000","last_undegraded":"2024-03-06T07:26:05.233509+0000","last_fullsized":"2024-03-06T07:26:05.233509+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:02.960470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238997+0000","last_change":"2024-03-06T07:26:04.230638+0000","last_active":"2024-03-06T07:26:05.238997+0000","last_peered":"2024-03-06T07:26:05.238997+0000","last_clean":"2024-03-06T07:26:05.238997+0000","last_became_active":"2024-03-06T07:26:04.230366+0000","last_became_peered":"2024-03-06T07:26:04.230366+0000","last_unstale":"2024-03-06T07:26:05.238997+0000","last_undegraded":"2024-03-06T07:26:05.238997+0000","last_fullsized":"2024-03-06T07:26:05.238997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:20:28.040894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734512+0000","last_change":"2024-03-06T07:25:55.180387+0000","last_active":"2024-03-06T07:26:05.734512+0000","last_peered":"2024-03-06T07:26:05.734512+0000","last_clean":"2024-03-06T07:26:05.734512+0000","last_became_active":"2024-03-06T07:25:55.179778+0000","last_became_peered":"2024-03-06T07:25:55.179778+0000","last_unstale":"2024-03-06T07:26:05.734512+0000","last_undegraded":"2024-03-06T07:26:05.734512+0000","last_fullsized":"2024-03-06T07:26:05.734512+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:16.774197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735486+0000","last_change":"2024-03-06T07:26:04.232126+0000","last_active":"2024-03-06T07:26:05.735486+0000","last_peered":"2024-03-06T07:26:05.735486+0000","last_clean":"2024-03-06T07:26:05.735486+0000","last_became_active":"2024-03-06T07:26:04.230221+0000","last_became_peered":"2024-03-06T07:26:04.230221+0000","last_unstale":"2024-03-06T07:26:05.735486+0000","last_undegraded":"2024-03-06T07:26:05.735486+0000","last_fullsized":"2024-03-06T07:26:05.735486+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:31.729215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238322+0000","last_change":"2024-03-06T07:25:55.180281+0000","last_active":"2024-03-06T07:26:05.238322+0000","last_peered":"2024-03-06T07:26:05.238322+0000","last_clean":"2024-03-06T07:26:05.238322+0000","last_became_active":"2024-03-06T07:25:55.180014+0000","last_became_peered":"2024-03-06T07:25:55.180014+0000","last_unstale":"2024-03-06T07:26:05.238322+0000","last_undegraded":"2024-03-06T07:26:05.238322+0000","last_fullsized":"2024-03-06T07:26:05.238322+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:08:51.191943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239078+0000","last_change":"2024-03-06T07:26:04.228988+0000","last_active":"2024-03-06T07:26:05.239078+0000","last_peered":"2024-03-06T07:26:05.239078+0000","last_clean":"2024-03-06T07:26:05.239078+0000","last_became_active":"2024-03-06T07:26:04.228619+0000","last_became_peered":"2024-03-06T07:26:04.228619+0000","last_unstale":"2024-03-06T07:26:05.239078+0000","last_undegraded":"2024-03-06T07:26:05.239078+0000","last_fullsized":"2024-03-06T07:26:05.239078+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:12.281654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737511+0000","last_change":"2024-03-06T07:25:55.180083+0000","last_active":"2024-03-06T07:26:05.737511+0000","last_peered":"2024-03-06T07:26:05.737511+0000","last_clean":"2024-03-06T07:26:05.737511+0000","last_became_active":"2024-03-06T07:25:55.179804+0000","last_became_peered":"2024-03-06T07:25:55.179804+0000","last_unstale":"2024-03-06T07:26:05.737511+0000","last_undegraded":"2024-03-06T07:26:05.737511+0000","last_fullsized":"2024-03-06T07:26:05.737511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:03.229947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735046+0000","last_change":"2024-03-06T07:26:04.226151+0000","last_active":"2024-03-06T07:26:05.735046+0000","last_peered":"2024-03-06T07:26:05.735046+0000","last_clean":"2024-03-06T07:26:05.735046+0000","last_became_active":"2024-03-06T07:26:04.225917+0000","last_became_peered":"2024-03-06T07:26:04.225917+0000","last_unstale":"2024-03-06T07:26:05.735046+0000","last_undegraded":"2024-03-06T07:26:05.735046+0000","last_fullsized":"2024-03-06T07:26:05.735046+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:00.625515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240221+0000","last_change":"2024-03-06T07:25:55.176450+0000","last_active":"2024-03-06T07:26:05.240221+0000","last_peered":"2024-03-06T07:26:05.240221+0000","last_clean":"2024-03-06T07:26:05.240221+0000","last_became_active":"2024-03-06T07:25:55.176194+0000","last_became_peered":"2024-03-06T07:25:55.176194+0000","last_unstale":"2024-03-06T07:26:05.240221+0000","last_undegraded":"2024-03-06T07:26:05.240221+0000","last_fullsized":"2024-03-06T07:26:05.240221+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:53.513334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735881+0000","last_change":"2024-03-06T07:26:04.224711+0000","last_active":"2024-03-06T07:26:05.735881+0000","last_peered":"2024-03-06T07:26:05.735881+0000","last_clean":"2024-03-06T07:26:05.735881+0000","last_became_active":"2024-03-06T07:26:04.224195+0000","last_became_peered":"2024-03-06T07:26:04.224195+0000","last_unstale":"2024-03-06T07:26:05.735881+0000","last_undegraded":"2024-03-06T07:26:05.735881+0000","last_fullsized":"2024-03-06T07:26:05.735881+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:48.963645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735464+0000","last_change":"2024-03-06T07:25:55.176884+0000","last_active":"2024-03-06T07:26:05.735464+0000","last_peered":"2024-03-06T07:26:05.735464+0000","last_clean":"2024-03-06T07:26:05.735464+0000","last_became_active":"2024-03-06T07:25:55.176546+0000","last_became_peered":"2024-03-06T07:25:55.176546+0000","last_unstale":"2024-03-06T07:26:05.735464+0000","last_undegraded":"2024-03-06T07:26:05.735464+0000","last_fullsized":"2024-03-06T07:26:05.735464+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:22.733869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240474+0000","last_change":"2024-03-06T07:26:04.229350+0000","last_active":"2024-03-06T07:26:05.240474+0000","last_peered":"2024-03-06T07:26:05.240474+0000","last_clean":"2024-03-06T07:26:05.240474+0000","last_became_active":"2024-03-06T07:26:04.229068+0000","last_became_peered":"2024-03-06T07:26:04.229068+0000","last_unstale":"2024-03-06T07:26:05.240474+0000","last_undegraded":"2024-03-06T07:26:05.240474+0000","last_fullsized":"2024-03-06T07:26:05.240474+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:18.508182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735759+0000","last_change":"2024-03-06T07:25:55.181620+0000","last_active":"2024-03-06T07:26:05.735759+0000","last_peered":"2024-03-06T07:26:05.735759+0000","last_clean":"2024-03-06T07:26:05.735759+0000","last_became_active":"2024-03-06T07:25:55.181336+0000","last_became_peered":"2024-03-06T07:25:55.181336+0000","last_unstale":"2024-03-06T07:26:05.735759+0000","last_undegraded":"2024-03-06T07:26:05.735759+0000","last_fullsized":"2024-03-06T07:26:05.735759+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:53:26.282224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734640+0000","last_change":"2024-03-06T07:26:04.234275+0000","last_active":"2024-03-06T07:26:05.734640+0000","last_peered":"2024-03-06T07:26:05.734640+0000","last_clean":"2024-03-06T07:26:05.734640+0000","last_became_active":"2024-03-06T07:26:04.234011+0000","last_became_peered":"2024-03-06T07:26:04.234011+0000","last_unstale":"2024-03-06T07:26:05.734640+0000","last_undegraded":"2024-03-06T07:26:05.734640+0000","last_fullsized":"2024-03-06T07:26:05.734640+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:19.072059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734938+0000","last_change":"2024-03-06T07:25:55.171658+0000","last_active":"2024-03-06T07:26:05.734938+0000","last_peered":"2024-03-06T07:26:05.734938+0000","last_clean":"2024-03-06T07:26:05.734938+0000","last_became_active":"2024-03-06T07:25:55.171366+0000","last_became_peered":"2024-03-06T07:25:55.171366+0000","last_unstale":"2024-03-06T07:26:05.734938+0000","last_undegraded":"2024-03-06T07:26:05.734938+0000","last_fullsized":"2024-03-06T07:26:05.734938+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:03.613423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734764+0000","last_change":"2024-03-06T07:26:04.235220+0000","last_active":"2024-03-06T07:26:05.734764+0000","last_peered":"2024-03-06T07:26:05.734764+0000","last_clean":"2024-03-06T07:26:05.734764+0000","last_became_active":"2024-03-06T07:26:04.234786+0000","last_became_peered":"2024-03-06T07:26:04.234786+0000","last_unstale":"2024-03-06T07:26:05.734764+0000","last_undegraded":"2024-03-06T07:26:05.734764+0000","last_fullsized":"2024-03-06T07:26:05.734764+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:09.370102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734708+0000","last_change":"2024-03-06T07:25:55.185023+0000","last_active":"2024-03-06T07:26:05.734708+0000","last_peered":"2024-03-06T07:26:05.734708+0000","last_clean":"2024-03-06T07:26:05.734708+0000","last_became_active":"2024-03-06T07:25:55.184708+0000","last_became_peered":"2024-03-06T07:25:55.184708+0000","last_unstale":"2024-03-06T07:26:05.734708+0000","last_undegraded":"2024-03-06T07:26:05.734708+0000","last_fullsized":"2024-03-06T07:26:05.734708+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:25.354294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736684+0000","last_change":"2024-03-06T07:26:04.239321+0000","last_active":"2024-03-06T07:26:05.736684+0000","last_peered":"2024-03-06T07:26:05.736684+0000","last_clean":"2024-03-06T07:26:05.736684+0000","last_became_active":"2024-03-06T07:26:04.238746+0000","last_became_peered":"2024-03-06T07:26:04.238746+0000","last_unstale":"2024-03-06T07:26:05.736684+0000","last_undegraded":"2024-03-06T07:26:05.736684+0000","last_fullsized":"2024-03-06T07:26:05.736684+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:46:44.443786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233321+0000","last_change":"2024-03-06T07:25:55.189044+0000","last_active":"2024-03-06T07:26:05.233321+0000","last_peered":"2024-03-06T07:26:05.233321+0000","last_clean":"2024-03-06T07:26:05.233321+0000","last_became_active":"2024-03-06T07:25:55.188772+0000","last_became_peered":"2024-03-06T07:25:55.188772+0000","last_unstale":"2024-03-06T07:26:05.233321+0000","last_undegraded":"2024-03-06T07:26:05.233321+0000","last_fullsized":"2024-03-06T07:26:05.233321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:44.165275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736084+0000","last_change":"2024-03-06T07:26:04.239527+0000","last_active":"2024-03-06T07:26:05.736084+0000","last_peered":"2024-03-06T07:26:05.736084+0000","last_clean":"2024-03-06T07:26:05.736084+0000","last_became_active":"2024-03-06T07:26:04.238895+0000","last_became_peered":"2024-03-06T07:26:04.238895+0000","last_unstale":"2024-03-06T07:26:05.736084+0000","last_undegraded":"2024-03-06T07:26:05.736084+0000","last_fullsized":"2024-03-06T07:26:05.736084+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:23:51.470414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238399+0000","last_change":"2024-03-06T07:25:55.179621+0000","last_active":"2024-03-06T07:26:05.238399+0000","last_peered":"2024-03-06T07:26:05.238399+0000","last_clean":"2024-03-06T07:26:05.238399+0000","last_became_active":"2024-03-06T07:25:55.179320+0000","last_became_peered":"2024-03-06T07:25:55.179320+0000","last_unstale":"2024-03-06T07:26:05.238399+0000","last_undegraded":"2024-03-06T07:26:05.238399+0000","last_fullsized":"2024-03-06T07:26:05.238399+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:38.124631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735672+0000","last_change":"2024-03-06T07:26:04.226226+0000","last_active":"2024-03-06T07:26:05.735672+0000","last_peered":"2024-03-06T07:26:05.735672+0000","last_clean":"2024-03-06T07:26:05.735672+0000","last_became_active":"2024-03-06T07:26:04.225948+0000","last_became_peered":"2024-03-06T07:26:04.225948+0000","last_unstale":"2024-03-06T07:26:05.735672+0000","last_undegraded":"2024-03-06T07:26:05.735672+0000","last_fullsized":"2024-03-06T07:26:05.735672+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:06.910190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736090+0000","last_change":"2024-03-06T07:25:55.183095+0000","last_active":"2024-03-06T07:26:05.736090+0000","last_peered":"2024-03-06T07:26:05.736090+0000","last_clean":"2024-03-06T07:26:05.736090+0000","last_became_active":"2024-03-06T07:25:55.182807+0000","last_became_peered":"2024-03-06T07:25:55.182807+0000","last_unstale":"2024-03-06T07:26:05.736090+0000","last_undegraded":"2024-03-06T07:26:05.736090+0000","last_fullsized":"2024-03-06T07:26:05.736090+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:02.698513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736473+0000","last_change":"2024-03-06T07:26:04.235785+0000","last_active":"2024-03-06T07:26:05.736473+0000","last_peered":"2024-03-06T07:26:05.736473+0000","last_clean":"2024-03-06T07:26:05.736473+0000","last_became_active":"2024-03-06T07:26:04.232700+0000","last_became_peered":"2024-03-06T07:26:04.232700+0000","last_unstale":"2024-03-06T07:26:05.736473+0000","last_undegraded":"2024-03-06T07:26:05.736473+0000","last_fullsized":"2024-03-06T07:26:05.736473+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:02.137613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736145+0000","last_change":"2024-03-06T07:25:55.180267+0000","last_active":"2024-03-06T07:26:05.736145+0000","last_peered":"2024-03-06T07:26:05.736145+0000","last_clean":"2024-03-06T07:26:05.736145+0000","last_became_active":"2024-03-06T07:25:55.180006+0000","last_became_peered":"2024-03-06T07:25:55.180006+0000","last_unstale":"2024-03-06T07:26:05.736145+0000","last_undegraded":"2024-03-06T07:26:05.736145+0000","last_fullsized":"2024-03-06T07:26:05.736145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:58:22.125383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735859+0000","last_change":"2024-03-06T07:26:04.226834+0000","last_active":"2024-03-06T07:26:05.735859+0000","last_peered":"2024-03-06T07:26:05.735859+0000","last_clean":"2024-03-06T07:26:05.735859+0000","last_became_active":"2024-03-06T07:26:04.226551+0000","last_became_peered":"2024-03-06T07:26:04.226551+0000","last_unstale":"2024-03-06T07:26:05.735859+0000","last_undegraded":"2024-03-06T07:26:05.735859+0000","last_fullsized":"2024-03-06T07:26:05.735859+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:56:02.565042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735547+0000","last_change":"2024-03-06T07:25:55.185010+0000","last_active":"2024-03-06T07:26:05.735547+0000","last_peered":"2024-03-06T07:26:05.735547+0000","last_clean":"2024-03-06T07:26:05.735547+0000","last_became_active":"2024-03-06T07:25:55.184723+0000","last_became_peered":"2024-03-06T07:25:55.184723+0000","last_unstale":"2024-03-06T07:26:05.735547+0000","last_undegraded":"2024-03-06T07:26:05.735547+0000","last_fullsized":"2024-03-06T07:26:05.735547+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736042+0000","last_change":"2024-03-06T07:26:04.239795+0000","last_active":"2024-03-06T07:26:05.736042+0000","last_peered":"2024-03-06T07:26:05.736042+0000","last_clean":"2024-03-06T07:26:05.736042+0000","last_became_active":"2024-03-06T07:26:04.239202+0000","last_became_peered":"2024-03-06T07:26:04.239202+0000","last_unstale":"2024-03-06T07:26:05.736042+0000","last_undegraded":"2024-03-06T07:26:05.736042+0000","last_fullsized":"2024-03-06T07:26:05.736042+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:18.508182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734876+0000","last_change":"2024-03-06T07:25:55.171489+0000","last_active":"2024-03-06T07:26:05.734876+0000","last_peered":"2024-03-06T07:26:05.734876+0000","last_clean":"2024-03-06T07:26:05.734876+0000","last_became_active":"2024-03-06T07:25:55.171184+0000","last_became_peered":"2024-03-06T07:25:55.171184+0000","last_unstale":"2024-03-06T07:26:05.734876+0000","last_undegraded":"2024-03-06T07:26:05.734876+0000","last_fullsized":"2024-03-06T07:26:05.734876+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:03.888318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734435+0000","last_change":"2024-03-06T07:26:04.231741+0000","last_active":"2024-03-06T07:26:05.734435+0000","last_peered":"2024-03-06T07:26:05.734435+0000","last_clean":"2024-03-06T07:26:05.734435+0000","last_became_active":"2024-03-06T07:26:04.231482+0000","last_became_peered":"2024-03-06T07:26:04.231482+0000","last_unstale":"2024-03-06T07:26:05.734435+0000","last_undegraded":"2024-03-06T07:26:05.734435+0000","last_fullsized":"2024-03-06T07:26:05.734435+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:35.442692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735237+0000","last_change":"2024-03-06T07:25:55.175885+0000","last_active":"2024-03-06T07:26:05.735237+0000","last_peered":"2024-03-06T07:26:05.735237+0000","last_clean":"2024-03-06T07:26:05.735237+0000","last_became_active":"2024-03-06T07:25:55.175607+0000","last_became_peered":"2024-03-06T07:25:55.175607+0000","last_unstale":"2024-03-06T07:26:05.735237+0000","last_undegraded":"2024-03-06T07:26:05.735237+0000","last_fullsized":"2024-03-06T07:26:05.735237+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734794+0000","last_change":"2024-03-06T07:26:04.239640+0000","last_active":"2024-03-06T07:26:05.734794+0000","last_peered":"2024-03-06T07:26:05.734794+0000","last_clean":"2024-03-06T07:26:05.734794+0000","last_became_active":"2024-03-06T07:26:04.239381+0000","last_became_peered":"2024-03-06T07:26:04.239381+0000","last_unstale":"2024-03-06T07:26:05.734794+0000","last_undegraded":"2024-03-06T07:26:05.734794+0000","last_fullsized":"2024-03-06T07:26:05.734794+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:14.833381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239631+0000","last_change":"2024-03-06T07:25:55.182544+0000","last_active":"2024-03-06T07:26:05.239631+0000","last_peered":"2024-03-06T07:26:05.239631+0000","last_clean":"2024-03-06T07:26:05.239631+0000","last_became_active":"2024-03-06T07:25:55.182215+0000","last_became_peered":"2024-03-06T07:25:55.182215+0000","last_unstale":"2024-03-06T07:26:05.239631+0000","last_undegraded":"2024-03-06T07:26:05.239631+0000","last_fullsized":"2024-03-06T07:26:05.239631+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:47.717714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735040+0000","last_change":"2024-03-06T07:26:04.234363+0000","last_active":"2024-03-06T07:26:05.735040+0000","last_peered":"2024-03-06T07:26:05.735040+0000","last_clean":"2024-03-06T07:26:05.735040+0000","last_became_active":"2024-03-06T07:26:04.234100+0000","last_became_peered":"2024-03-06T07:26:04.234100+0000","last_unstale":"2024-03-06T07:26:05.735040+0000","last_undegraded":"2024-03-06T07:26:05.735040+0000","last_fullsized":"2024-03-06T07:26:05.735040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:16.582927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734885+0000","last_change":"2024-03-06T07:25:55.174623+0000","last_active":"2024-03-06T07:26:05.734885+0000","last_peered":"2024-03-06T07:26:05.734885+0000","last_clean":"2024-03-06T07:26:05.734885+0000","last_became_active":"2024-03-06T07:25:55.174336+0000","last_became_peered":"2024-03-06T07:25:55.174336+0000","last_unstale":"2024-03-06T07:26:05.734885+0000","last_undegraded":"2024-03-06T07:26:05.734885+0000","last_fullsized":"2024-03-06T07:26:05.734885+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:15.212900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735113+0000","last_change":"2024-03-06T07:26:04.231307+0000","last_active":"2024-03-06T07:26:05.735113+0000","last_peered":"2024-03-06T07:26:05.735113+0000","last_clean":"2024-03-06T07:26:05.735113+0000","last_became_active":"2024-03-06T07:26:04.229445+0000","last_became_peered":"2024-03-06T07:26:04.229445+0000","last_unstale":"2024-03-06T07:26:05.735113+0000","last_undegraded":"2024-03-06T07:26:05.735113+0000","last_fullsized":"2024-03-06T07:26:05.735113+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:53.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239940+0000","last_change":"2024-03-06T07:25:55.173469+0000","last_active":"2024-03-06T07:26:05.239940+0000","last_peered":"2024-03-06T07:26:05.239940+0000","last_clean":"2024-03-06T07:26:05.239940+0000","last_became_active":"2024-03-06T07:25:55.173211+0000","last_became_peered":"2024-03-06T07:25:55.173211+0000","last_unstale":"2024-03-06T07:26:05.239940+0000","last_undegraded":"2024-03-06T07:26:05.239940+0000","last_fullsized":"2024-03-06T07:26:05.239940+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:44.346159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239408+0000","last_change":"2024-03-06T07:26:04.229361+0000","last_active":"2024-03-06T07:26:05.239408+0000","last_peered":"2024-03-06T07:26:05.239408+0000","last_clean":"2024-03-06T07:26:05.239408+0000","last_became_active":"2024-03-06T07:26:04.229096+0000","last_became_peered":"2024-03-06T07:26:04.229096+0000","last_unstale":"2024-03-06T07:26:05.239408+0000","last_undegraded":"2024-03-06T07:26:05.239408+0000","last_fullsized":"2024-03-06T07:26:05.239408+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:53.216983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233146+0000","last_change":"2024-03-06T07:26:03.218174+0000","last_active":"2024-03-06T07:26:05.233146+0000","last_peered":"2024-03-06T07:26:05.233146+0000","last_clean":"2024-03-06T07:26:05.233146+0000","last_became_active":"2024-03-06T07:26:03.217787+0000","last_became_peered":"2024-03-06T07:26:03.217787+0000","last_unstale":"2024-03-06T07:26:05.233146+0000","last_undegraded":"2024-03-06T07:26:05.233146+0000","last_fullsized":"2024-03-06T07:26:05.233146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:05.900220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734952+0000","last_change":"2024-03-06T07:25:55.183313+0000","last_active":"2024-03-06T07:26:05.734952+0000","last_peered":"2024-03-06T07:26:05.734952+0000","last_clean":"2024-03-06T07:26:05.734952+0000","last_became_active":"2024-03-06T07:25:55.183007+0000","last_became_peered":"2024-03-06T07:25:55.183007+0000","last_unstale":"2024-03-06T07:26:05.734952+0000","last_undegraded":"2024-03-06T07:26:05.734952+0000","last_fullsized":"2024-03-06T07:26:05.734952+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:53.878454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"149'4","reported_seq":26,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239236+0000","last_change":"2024-03-06T07:25:54.177412+0000","last_active":"2024-03-06T07:26:05.239236+0000","last_peered":"2024-03-06T07:26:05.239236+0000","last_clean":"2024-03-06T07:26:05.239236+0000","last_became_active":"2024-03-06T07:25:54.177071+0000","last_became_peered":"2024-03-06T07:25:54.177071+0000","last_unstale":"2024-03-06T07:26:05.239236+0000","last_undegraded":"2024-03-06T07:26:05.239236+0000","last_fullsized":"2024-03-06T07:26:05.239236+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:58.975791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239854+0000","last_change":"2024-03-06T07:26:04.236127+0000","last_active":"2024-03-06T07:26:05.239854+0000","last_peered":"2024-03-06T07:26:05.239854+0000","last_clean":"2024-03-06T07:26:05.239854+0000","last_became_active":"2024-03-06T07:26:04.235069+0000","last_became_peered":"2024-03-06T07:26:04.235069+0000","last_unstale":"2024-03-06T07:26:05.239854+0000","last_undegraded":"2024-03-06T07:26:05.239854+0000","last_fullsized":"2024-03-06T07:26:05.239854+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:16.582927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734391+0000","last_change":"2024-03-06T07:26:03.225955+0000","last_active":"2024-03-06T07:26:05.734391+0000","last_peered":"2024-03-06T07:26:05.734391+0000","last_clean":"2024-03-06T07:26:05.734391+0000","last_became_active":"2024-03-06T07:26:03.225631+0000","last_became_peered":"2024-03-06T07:26:03.225631+0000","last_unstale":"2024-03-06T07:26:05.734391+0000","last_undegraded":"2024-03-06T07:26:05.734391+0000","last_fullsized":"2024-03-06T07:26:05.734391+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:20:01.920191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734676+0000","last_change":"2024-03-06T07:25:55.169574+0000","last_active":"2024-03-06T07:26:05.734676+0000","last_peered":"2024-03-06T07:26:05.734676+0000","last_clean":"2024-03-06T07:26:05.734676+0000","last_became_active":"2024-03-06T07:25:55.168987+0000","last_became_peered":"2024-03-06T07:25:55.168987+0000","last_unstale":"2024-03-06T07:26:05.734676+0000","last_undegraded":"2024-03-06T07:26:05.734676+0000","last_fullsized":"2024-03-06T07:26:05.734676+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:42:55.457043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233876+0000","last_change":"2024-03-06T07:25:54.178532+0000","last_active":"2024-03-06T07:26:05.233876+0000","last_peered":"2024-03-06T07:26:05.233876+0000","last_clean":"2024-03-06T07:26:05.233876+0000","last_became_active":"2024-03-06T07:25:54.178228+0000","last_became_peered":"2024-03-06T07:25:54.178228+0000","last_unstale":"2024-03-06T07:26:05.233876+0000","last_undegraded":"2024-03-06T07:26:05.233876+0000","last_fullsized":"2024-03-06T07:26:05.233876+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:07.702424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734323+0000","last_change":"2024-03-06T07:26:04.230124+0000","last_active":"2024-03-06T07:26:05.734323+0000","last_peered":"2024-03-06T07:26:05.734323+0000","last_clean":"2024-03-06T07:26:05.734323+0000","last_became_active":"2024-03-06T07:26:04.229507+0000","last_became_peered":"2024-03-06T07:26:04.229507+0000","last_unstale":"2024-03-06T07:26:05.734323+0000","last_undegraded":"2024-03-06T07:26:05.734323+0000","last_fullsized":"2024-03-06T07:26:05.734323+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:12:11.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239088+0000","last_change":"2024-03-06T07:26:03.221631+0000","last_active":"2024-03-06T07:26:05.239088+0000","last_peered":"2024-03-06T07:26:05.239088+0000","last_clean":"2024-03-06T07:26:05.239088+0000","last_became_active":"2024-03-06T07:26:03.221345+0000","last_became_peered":"2024-03-06T07:26:03.221345+0000","last_unstale":"2024-03-06T07:26:05.239088+0000","last_undegraded":"2024-03-06T07:26:05.239088+0000","last_fullsized":"2024-03-06T07:26:05.239088+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:26.233661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233603+0000","last_change":"2024-03-06T07:25:55.181488+0000","last_active":"2024-03-06T07:26:05.233603+0000","last_peered":"2024-03-06T07:26:05.233603+0000","last_clean":"2024-03-06T07:26:05.233603+0000","last_became_active":"2024-03-06T07:25:55.181212+0000","last_became_peered":"2024-03-06T07:25:55.181212+0000","last_unstale":"2024-03-06T07:26:05.233603+0000","last_undegraded":"2024-03-06T07:26:05.233603+0000","last_fullsized":"2024-03-06T07:26:05.233603+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:07.531219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735614+0000","last_change":"2024-03-06T07:25:54.176834+0000","last_active":"2024-03-06T07:26:05.735614+0000","last_peered":"2024-03-06T07:26:05.735614+0000","last_clean":"2024-03-06T07:26:05.735614+0000","last_became_active":"2024-03-06T07:25:54.176517+0000","last_became_peered":"2024-03-06T07:25:54.176517+0000","last_unstale":"2024-03-06T07:26:05.735614+0000","last_undegraded":"2024-03-06T07:26:05.735614+0000","last_fullsized":"2024-03-06T07:26:05.735614+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:46.712490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734896+0000","last_change":"2024-03-06T07:26:04.232779+0000","last_active":"2024-03-06T07:26:05.734896+0000","last_peered":"2024-03-06T07:26:05.734896+0000","last_clean":"2024-03-06T07:26:05.734896+0000","last_became_active":"2024-03-06T07:26:04.232425+0000","last_became_peered":"2024-03-06T07:26:04.232425+0000","last_unstale":"2024-03-06T07:26:05.734896+0000","last_undegraded":"2024-03-06T07:26:05.734896+0000","last_fullsized":"2024-03-06T07:26:05.734896+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:05.373306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736405+0000","last_change":"2024-03-06T07:26:03.233595+0000","last_active":"2024-03-06T07:26:05.736405+0000","last_peered":"2024-03-06T07:26:05.736405+0000","last_clean":"2024-03-06T07:26:05.736405+0000","last_became_active":"2024-03-06T07:26:03.233343+0000","last_became_peered":"2024-03-06T07:26:03.233343+0000","last_unstale":"2024-03-06T07:26:05.736405+0000","last_undegraded":"2024-03-06T07:26:05.736405+0000","last_fullsized":"2024-03-06T07:26:05.736405+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:08.234461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735372+0000","last_change":"2024-03-06T07:25:55.186954+0000","last_active":"2024-03-06T07:26:05.735372+0000","last_peered":"2024-03-06T07:26:05.735372+0000","last_clean":"2024-03-06T07:26:05.735372+0000","last_became_active":"2024-03-06T07:25:55.186700+0000","last_became_peered":"2024-03-06T07:25:55.186700+0000","last_unstale":"2024-03-06T07:26:05.735372+0000","last_undegraded":"2024-03-06T07:26:05.735372+0000","last_fullsized":"2024-03-06T07:26:05.735372+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:08.707649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735762+0000","last_change":"2024-03-06T07:25:54.173923+0000","last_active":"2024-03-06T07:26:05.735762+0000","last_peered":"2024-03-06T07:26:05.735762+0000","last_clean":"2024-03-06T07:26:05.735762+0000","last_became_active":"2024-03-06T07:25:54.173628+0000","last_became_peered":"2024-03-06T07:25:54.173628+0000","last_unstale":"2024-03-06T07:26:05.735762+0000","last_undegraded":"2024-03-06T07:26:05.735762+0000","last_fullsized":"2024-03-06T07:26:05.735762+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:18.074282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239446+0000","last_change":"2024-03-06T07:26:04.233045+0000","last_active":"2024-03-06T07:26:05.239446+0000","last_peered":"2024-03-06T07:26:05.239446+0000","last_clean":"2024-03-06T07:26:05.239446+0000","last_became_active":"2024-03-06T07:26:04.232782+0000","last_became_peered":"2024-03-06T07:26:04.232782+0000","last_unstale":"2024-03-06T07:26:05.239446+0000","last_undegraded":"2024-03-06T07:26:05.239446+0000","last_fullsized":"2024-03-06T07:26:05.239446+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:12.012177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233479+0000","last_change":"2024-03-06T07:26:03.226022+0000","last_active":"2024-03-06T07:26:05.233479+0000","last_peered":"2024-03-06T07:26:05.233479+0000","last_clean":"2024-03-06T07:26:05.233479+0000","last_became_active":"2024-03-06T07:26:03.225758+0000","last_became_peered":"2024-03-06T07:26:03.225758+0000","last_unstale":"2024-03-06T07:26:05.233479+0000","last_undegraded":"2024-03-06T07:26:05.233479+0000","last_fullsized":"2024-03-06T07:26:05.233479+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:11.655160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736835+0000","last_change":"2024-03-06T07:25:55.179023+0000","last_active":"2024-03-06T07:26:05.736835+0000","last_peered":"2024-03-06T07:26:05.736835+0000","last_clean":"2024-03-06T07:26:05.736835+0000","last_became_active":"2024-03-06T07:25:55.178604+0000","last_became_peered":"2024-03-06T07:25:55.178604+0000","last_unstale":"2024-03-06T07:26:05.736835+0000","last_undegraded":"2024-03-06T07:26:05.736835+0000","last_fullsized":"2024-03-06T07:26:05.736835+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736778+0000","last_change":"2024-03-06T07:25:54.179340+0000","last_active":"2024-03-06T07:26:05.736778+0000","last_peered":"2024-03-06T07:26:05.736778+0000","last_clean":"2024-03-06T07:26:05.736778+0000","last_became_active":"2024-03-06T07:25:54.179061+0000","last_became_peered":"2024-03-06T07:25:54.179061+0000","last_unstale":"2024-03-06T07:26:05.736778+0000","last_undegraded":"2024-03-06T07:26:05.736778+0000","last_fullsized":"2024-03-06T07:26:05.736778+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:24.779584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737406+0000","last_change":"2024-03-06T07:26:04.234152+0000","last_active":"2024-03-06T07:26:05.737406+0000","last_peered":"2024-03-06T07:26:05.737406+0000","last_clean":"2024-03-06T07:26:05.737406+0000","last_became_active":"2024-03-06T07:26:04.233622+0000","last_became_peered":"2024-03-06T07:26:04.233622+0000","last_unstale":"2024-03-06T07:26:05.737406+0000","last_undegraded":"2024-03-06T07:26:05.737406+0000","last_fullsized":"2024-03-06T07:26:05.737406+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:00.621820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238603+0000","last_change":"2024-03-06T07:26:03.227584+0000","last_active":"2024-03-06T07:26:05.238603+0000","last_peered":"2024-03-06T07:26:05.238603+0000","last_clean":"2024-03-06T07:26:05.238603+0000","last_became_active":"2024-03-06T07:26:03.227070+0000","last_became_peered":"2024-03-06T07:26:03.227070+0000","last_unstale":"2024-03-06T07:26:05.238603+0000","last_undegraded":"2024-03-06T07:26:05.238603+0000","last_fullsized":"2024-03-06T07:26:05.238603+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:35:28.273592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737141+0000","last_change":"2024-03-06T07:25:55.186844+0000","last_active":"2024-03-06T07:26:05.737141+0000","last_peered":"2024-03-06T07:26:05.737141+0000","last_clean":"2024-03-06T07:26:05.737141+0000","last_became_active":"2024-03-06T07:25:55.186558+0000","last_became_peered":"2024-03-06T07:25:55.186558+0000","last_unstale":"2024-03-06T07:26:05.737141+0000","last_undegraded":"2024-03-06T07:26:05.737141+0000","last_fullsized":"2024-03-06T07:26:05.737141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:37:40.318903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233699+0000","last_change":"2024-03-06T07:25:54.182713+0000","last_active":"2024-03-06T07:26:05.233699+0000","last_peered":"2024-03-06T07:26:05.233699+0000","last_clean":"2024-03-06T07:26:05.233699+0000","last_became_active":"2024-03-06T07:25:54.182447+0000","last_became_peered":"2024-03-06T07:25:54.182447+0000","last_unstale":"2024-03-06T07:26:05.233699+0000","last_undegraded":"2024-03-06T07:26:05.233699+0000","last_fullsized":"2024-03-06T07:26:05.233699+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:17.509614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736123+0000","last_change":"2024-03-06T07:26:04.233107+0000","last_active":"2024-03-06T07:26:05.736123+0000","last_peered":"2024-03-06T07:26:05.736123+0000","last_clean":"2024-03-06T07:26:05.736123+0000","last_became_active":"2024-03-06T07:26:04.231842+0000","last_became_peered":"2024-03-06T07:26:04.231842+0000","last_unstale":"2024-03-06T07:26:05.736123+0000","last_undegraded":"2024-03-06T07:26:05.736123+0000","last_fullsized":"2024-03-06T07:26:05.736123+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:12.940025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735232+0000","last_change":"2024-03-06T07:26:03.234423+0000","last_active":"2024-03-06T07:26:05.735232+0000","last_peered":"2024-03-06T07:26:05.735232+0000","last_clean":"2024-03-06T07:26:05.735232+0000","last_became_active":"2024-03-06T07:26:03.234173+0000","last_became_peered":"2024-03-06T07:26:03.234173+0000","last_unstale":"2024-03-06T07:26:05.735232+0000","last_undegraded":"2024-03-06T07:26:05.735232+0000","last_fullsized":"2024-03-06T07:26:05.735232+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:33.445743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736225+0000","last_change":"2024-03-06T07:25:55.172223+0000","last_active":"2024-03-06T07:26:05.736225+0000","last_peered":"2024-03-06T07:26:05.736225+0000","last_clean":"2024-03-06T07:26:05.736225+0000","last_became_active":"2024-03-06T07:25:55.171895+0000","last_became_peered":"2024-03-06T07:25:55.171895+0000","last_unstale":"2024-03-06T07:26:05.736225+0000","last_undegraded":"2024-03-06T07:26:05.736225+0000","last_fullsized":"2024-03-06T07:26:05.736225+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:27.069827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735146+0000","last_change":"2024-03-06T07:25:54.172467+0000","last_active":"2024-03-06T07:26:05.735146+0000","last_peered":"2024-03-06T07:26:05.735146+0000","last_clean":"2024-03-06T07:26:05.735146+0000","last_became_active":"2024-03-06T07:25:54.172185+0000","last_became_peered":"2024-03-06T07:25:54.172185+0000","last_unstale":"2024-03-06T07:26:05.735146+0000","last_undegraded":"2024-03-06T07:26:05.735146+0000","last_fullsized":"2024-03-06T07:26:05.735146+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:58.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736072+0000","last_change":"2024-03-06T07:26:04.240395+0000","last_active":"2024-03-06T07:26:05.736072+0000","last_peered":"2024-03-06T07:26:05.736072+0000","last_clean":"2024-03-06T07:26:05.736072+0000","last_became_active":"2024-03-06T07:26:04.240115+0000","last_became_peered":"2024-03-06T07:26:04.240115+0000","last_unstale":"2024-03-06T07:26:05.736072+0000","last_undegraded":"2024-03-06T07:26:05.736072+0000","last_fullsized":"2024-03-06T07:26:05.736072+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:08.475164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737146+0000","last_change":"2024-03-06T07:26:03.230790+0000","last_active":"2024-03-06T07:26:05.737146+0000","last_peered":"2024-03-06T07:26:05.737146+0000","last_clean":"2024-03-06T07:26:05.737146+0000","last_became_active":"2024-03-06T07:26:03.230556+0000","last_became_peered":"2024-03-06T07:26:03.230556+0000","last_unstale":"2024-03-06T07:26:05.737146+0000","last_undegraded":"2024-03-06T07:26:05.737146+0000","last_fullsized":"2024-03-06T07:26:05.737146+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:57.404566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239141+0000","last_change":"2024-03-06T07:25:55.175586+0000","last_active":"2024-03-06T07:26:05.239141+0000","last_peered":"2024-03-06T07:26:05.239141+0000","last_clean":"2024-03-06T07:26:05.239141+0000","last_became_active":"2024-03-06T07:25:55.175328+0000","last_became_peered":"2024-03-06T07:25:55.175328+0000","last_unstale":"2024-03-06T07:26:05.239141+0000","last_undegraded":"2024-03-06T07:26:05.239141+0000","last_fullsized":"2024-03-06T07:26:05.239141+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:34.059703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734918+0000","last_change":"2024-03-06T07:25:54.175104+0000","last_active":"2024-03-06T07:26:05.734918+0000","last_peered":"2024-03-06T07:26:05.734918+0000","last_clean":"2024-03-06T07:26:05.734918+0000","last_became_active":"2024-03-06T07:25:54.174837+0000","last_became_peered":"2024-03-06T07:25:54.174837+0000","last_unstale":"2024-03-06T07:26:05.734918+0000","last_undegraded":"2024-03-06T07:26:05.734918+0000","last_fullsized":"2024-03-06T07:26:05.734918+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:15.768972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736224+0000","last_change":"2024-03-06T07:26:04.235419+0000","last_active":"2024-03-06T07:26:05.736224+0000","last_peered":"2024-03-06T07:26:05.736224+0000","last_clean":"2024-03-06T07:26:05.736224+0000","last_became_active":"2024-03-06T07:26:04.235181+0000","last_became_peered":"2024-03-06T07:26:04.235181+0000","last_unstale":"2024-03-06T07:26:05.736224+0000","last_undegraded":"2024-03-06T07:26:05.736224+0000","last_fullsized":"2024-03-06T07:26:05.736224+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:48:51.543206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239721+0000","last_change":"2024-03-06T07:26:03.226983+0000","last_active":"2024-03-06T07:26:05.239721+0000","last_peered":"2024-03-06T07:26:05.239721+0000","last_clean":"2024-03-06T07:26:05.239721+0000","last_became_active":"2024-03-06T07:26:03.226649+0000","last_became_peered":"2024-03-06T07:26:03.226649+0000","last_unstale":"2024-03-06T07:26:05.239721+0000","last_undegraded":"2024-03-06T07:26:05.239721+0000","last_fullsized":"2024-03-06T07:26:05.239721+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:38:30.775606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735341+0000","last_change":"2024-03-06T07:25:55.183382+0000","last_active":"2024-03-06T07:26:05.735341+0000","last_peered":"2024-03-06T07:26:05.735341+0000","last_clean":"2024-03-06T07:26:05.735341+0000","last_became_active":"2024-03-06T07:25:55.182975+0000","last_became_peered":"2024-03-06T07:25:55.182975+0000","last_unstale":"2024-03-06T07:26:05.735341+0000","last_undegraded":"2024-03-06T07:26:05.735341+0000","last_fullsized":"2024-03-06T07:26:05.735341+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:04.379898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736000+0000","last_change":"2024-03-06T07:25:54.184577+0000","last_active":"2024-03-06T07:26:05.736000+0000","last_peered":"2024-03-06T07:26:05.736000+0000","last_clean":"2024-03-06T07:26:05.736000+0000","last_became_active":"2024-03-06T07:25:54.184068+0000","last_became_peered":"2024-03-06T07:25:54.184068+0000","last_unstale":"2024-03-06T07:26:05.736000+0000","last_undegraded":"2024-03-06T07:26:05.736000+0000","last_fullsized":"2024-03-06T07:26:05.736000+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:30.954235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238281+0000","last_change":"2024-03-06T07:26:04.228139+0000","last_active":"2024-03-06T07:26:05.238281+0000","last_peered":"2024-03-06T07:26:05.238281+0000","last_clean":"2024-03-06T07:26:05.238281+0000","last_became_active":"2024-03-06T07:26:04.227846+0000","last_became_peered":"2024-03-06T07:26:04.227846+0000","last_unstale":"2024-03-06T07:26:05.238281+0000","last_undegraded":"2024-03-06T07:26:05.238281+0000","last_fullsized":"2024-03-06T07:26:05.238281+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:29.622186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734724+0000","last_change":"2024-03-06T07:26:03.235295+0000","last_active":"2024-03-06T07:26:05.734724+0000","last_peered":"2024-03-06T07:26:05.734724+0000","last_clean":"2024-03-06T07:26:05.734724+0000","last_became_active":"2024-03-06T07:26:03.235026+0000","last_became_peered":"2024-03-06T07:26:03.235026+0000","last_unstale":"2024-03-06T07:26:05.734724+0000","last_undegraded":"2024-03-06T07:26:05.734724+0000","last_fullsized":"2024-03-06T07:26:05.734724+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:04:59.611849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735592+0000","last_change":"2024-03-06T07:25:55.175578+0000","last_active":"2024-03-06T07:26:05.735592+0000","last_peered":"2024-03-06T07:26:05.735592+0000","last_clean":"2024-03-06T07:26:05.735592+0000","last_became_active":"2024-03-06T07:25:55.175257+0000","last_became_peered":"2024-03-06T07:25:55.175257+0000","last_unstale":"2024-03-06T07:26:05.735592+0000","last_undegraded":"2024-03-06T07:26:05.735592+0000","last_fullsized":"2024-03-06T07:26:05.735592+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:02:56.174605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"149'3","reported_seq":25,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239243+0000","last_change":"2024-03-06T07:25:54.177422+0000","last_active":"2024-03-06T07:26:05.239243+0000","last_peered":"2024-03-06T07:26:05.239243+0000","last_clean":"2024-03-06T07:26:05.239243+0000","last_became_active":"2024-03-06T07:25:54.177048+0000","last_became_peered":"2024-03-06T07:25:54.177048+0000","last_unstale":"2024-03-06T07:26:05.239243+0000","last_undegraded":"2024-03-06T07:26:05.239243+0000","last_fullsized":"2024-03-06T07:26:05.239243+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:21.672283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735347+0000","last_change":"2024-03-06T07:26:04.229458+0000","last_active":"2024-03-06T07:26:05.735347+0000","last_peered":"2024-03-06T07:26:05.735347+0000","last_clean":"2024-03-06T07:26:05.735347+0000","last_became_active":"2024-03-06T07:26:04.229214+0000","last_became_peered":"2024-03-06T07:26:04.229214+0000","last_unstale":"2024-03-06T07:26:05.735347+0000","last_undegraded":"2024-03-06T07:26:05.735347+0000","last_fullsized":"2024-03-06T07:26:05.735347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:43.424540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233407+0000","last_change":"2024-03-06T07:26:03.225838+0000","last_active":"2024-03-06T07:26:05.233407+0000","last_peered":"2024-03-06T07:26:05.233407+0000","last_clean":"2024-03-06T07:26:05.233407+0000","last_became_active":"2024-03-06T07:26:03.225557+0000","last_became_peered":"2024-03-06T07:26:03.225557+0000","last_unstale":"2024-03-06T07:26:05.233407+0000","last_undegraded":"2024-03-06T07:26:05.233407+0000","last_fullsized":"2024-03-06T07:26:05.233407+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:27.860270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735515+0000","last_change":"2024-03-06T07:25:55.182193+0000","last_active":"2024-03-06T07:26:05.735515+0000","last_peered":"2024-03-06T07:26:05.735515+0000","last_clean":"2024-03-06T07:26:05.735515+0000","last_became_active":"2024-03-06T07:25:55.181940+0000","last_became_peered":"2024-03-06T07:25:55.181940+0000","last_unstale":"2024-03-06T07:26:05.735515+0000","last_undegraded":"2024-03-06T07:26:05.735515+0000","last_fullsized":"2024-03-06T07:26:05.735515+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:06:17.306156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"149'6","reported_seq":28,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736383+0000","last_change":"2024-03-06T07:25:54.184311+0000","last_active":"2024-03-06T07:26:05.736383+0000","last_peered":"2024-03-06T07:26:05.736383+0000","last_clean":"2024-03-06T07:26:05.736383+0000","last_became_active":"2024-03-06T07:25:54.183969+0000","last_became_peered":"2024-03-06T07:25:54.183969+0000","last_unstale":"2024-03-06T07:26:05.736383+0000","last_undegraded":"2024-03-06T07:26:05.736383+0000","last_fullsized":"2024-03-06T07:26:05.736383+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:08.451250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736206+0000","last_change":"2024-03-06T07:26:04.233830+0000","last_active":"2024-03-06T07:26:05.736206+0000","last_peered":"2024-03-06T07:26:05.736206+0000","last_clean":"2024-03-06T07:26:05.736206+0000","last_became_active":"2024-03-06T07:26:04.232811+0000","last_became_peered":"2024-03-06T07:26:04.232811+0000","last_unstale":"2024-03-06T07:26:05.736206+0000","last_undegraded":"2024-03-06T07:26:05.736206+0000","last_fullsized":"2024-03-06T07:26:05.736206+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:41.011166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735008+0000","last_change":"2024-03-06T07:26:03.226298+0000","last_active":"2024-03-06T07:26:05.735008+0000","last_peered":"2024-03-06T07:26:05.735008+0000","last_clean":"2024-03-06T07:26:05.735008+0000","last_became_active":"2024-03-06T07:26:03.226040+0000","last_became_peered":"2024-03-06T07:26:03.226040+0000","last_unstale":"2024-03-06T07:26:05.735008+0000","last_undegraded":"2024-03-06T07:26:05.735008+0000","last_fullsized":"2024-03-06T07:26:05.735008+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:34:16.749386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240045+0000","last_change":"2024-03-06T07:25:55.174144+0000","last_active":"2024-03-06T07:26:05.240045+0000","last_peered":"2024-03-06T07:26:05.240045+0000","last_clean":"2024-03-06T07:26:05.240045+0000","last_became_active":"2024-03-06T07:25:55.173885+0000","last_became_peered":"2024-03-06T07:25:55.173885+0000","last_unstale":"2024-03-06T07:26:05.240045+0000","last_undegraded":"2024-03-06T07:26:05.240045+0000","last_fullsized":"2024-03-06T07:26:05.240045+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:59.981015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233588+0000","last_change":"2024-03-06T07:25:54.175000+0000","last_active":"2024-03-06T07:26:05.233588+0000","last_peered":"2024-03-06T07:26:05.233588+0000","last_clean":"2024-03-06T07:26:05.233588+0000","last_became_active":"2024-03-06T07:25:54.174717+0000","last_became_peered":"2024-03-06T07:25:54.174717+0000","last_unstale":"2024-03-06T07:26:05.233588+0000","last_undegraded":"2024-03-06T07:26:05.233588+0000","last_fullsized":"2024-03-06T07:26:05.233588+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:19:52.873229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239649+0000","last_change":"2024-03-06T07:26:04.232417+0000","last_active":"2024-03-06T07:26:05.239649+0000","last_peered":"2024-03-06T07:26:05.239649+0000","last_clean":"2024-03-06T07:26:05.239649+0000","last_became_active":"2024-03-06T07:26:04.232109+0000","last_became_peered":"2024-03-06T07:26:04.232109+0000","last_unstale":"2024-03-06T07:26:05.239649+0000","last_undegraded":"2024-03-06T07:26:05.239649+0000","last_fullsized":"2024-03-06T07:26:05.239649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:24.264607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238939+0000","last_change":"2024-03-06T07:26:03.225275+0000","last_active":"2024-03-06T07:26:05.238939+0000","last_peered":"2024-03-06T07:26:05.238939+0000","last_clean":"2024-03-06T07:26:05.238939+0000","last_became_active":"2024-03-06T07:26:03.224992+0000","last_became_peered":"2024-03-06T07:26:03.224992+0000","last_unstale":"2024-03-06T07:26:05.238939+0000","last_undegraded":"2024-03-06T07:26:05.238939+0000","last_fullsized":"2024-03-06T07:26:05.238939+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:12.421635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737052+0000","last_change":"2024-03-06T07:25:55.186488+0000","last_active":"2024-03-06T07:26:05.737052+0000","last_peered":"2024-03-06T07:26:05.737052+0000","last_clean":"2024-03-06T07:26:05.737052+0000","last_became_active":"2024-03-06T07:25:55.185803+0000","last_became_peered":"2024-03-06T07:25:55.185803+0000","last_unstale":"2024-03-06T07:26:05.737052+0000","last_undegraded":"2024-03-06T07:26:05.737052+0000","last_fullsized":"2024-03-06T07:26:05.737052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735682+0000","last_change":"2024-03-06T07:25:54.178593+0000","last_active":"2024-03-06T07:26:05.735682+0000","last_peered":"2024-03-06T07:26:05.735682+0000","last_clean":"2024-03-06T07:26:05.735682+0000","last_became_active":"2024-03-06T07:25:54.178229+0000","last_became_peered":"2024-03-06T07:25:54.178229+0000","last_unstale":"2024-03-06T07:26:05.735682+0000","last_undegraded":"2024-03-06T07:26:05.735682+0000","last_fullsized":"2024-03-06T07:26:05.735682+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:59.187499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735634+0000","last_change":"2024-03-06T07:26:04.233871+0000","last_active":"2024-03-06T07:26:05.735634+0000","last_peered":"2024-03-06T07:26:05.735634+0000","last_clean":"2024-03-06T07:26:05.735634+0000","last_became_active":"2024-03-06T07:26:04.233555+0000","last_became_peered":"2024-03-06T07:26:04.233555+0000","last_unstale":"2024-03-06T07:26:05.735634+0000","last_undegraded":"2024-03-06T07:26:05.735634+0000","last_fullsized":"2024-03-06T07:26:05.735634+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:04:41.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233893+0000","last_change":"2024-03-06T07:26:03.227103+0000","last_active":"2024-03-06T07:26:05.233893+0000","last_peered":"2024-03-06T07:26:05.233893+0000","last_clean":"2024-03-06T07:26:05.233893+0000","last_became_active":"2024-03-06T07:26:03.226866+0000","last_became_peered":"2024-03-06T07:26:03.226866+0000","last_unstale":"2024-03-06T07:26:05.233893+0000","last_undegraded":"2024-03-06T07:26:05.233893+0000","last_fullsized":"2024-03-06T07:26:05.233893+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:42.414570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736370+0000","last_change":"2024-03-06T07:25:55.175110+0000","last_active":"2024-03-06T07:26:05.736370+0000","last_peered":"2024-03-06T07:26:05.736370+0000","last_clean":"2024-03-06T07:26:05.736370+0000","last_became_active":"2024-03-06T07:25:55.174818+0000","last_became_peered":"2024-03-06T07:25:55.174818+0000","last_unstale":"2024-03-06T07:26:05.736370+0000","last_undegraded":"2024-03-06T07:26:05.736370+0000","last_fullsized":"2024-03-06T07:26:05.736370+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:10:18.191924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735019+0000","last_change":"2024-03-06T07:25:54.178997+0000","last_active":"2024-03-06T07:26:05.735019+0000","last_peered":"2024-03-06T07:26:05.735019+0000","last_clean":"2024-03-06T07:26:05.735019+0000","last_became_active":"2024-03-06T07:25:54.178699+0000","last_became_peered":"2024-03-06T07:25:54.178699+0000","last_unstale":"2024-03-06T07:26:05.735019+0000","last_undegraded":"2024-03-06T07:26:05.735019+0000","last_fullsized":"2024-03-06T07:26:05.735019+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:57:04.021145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736821+0000","last_change":"2024-03-06T07:26:04.230169+0000","last_active":"2024-03-06T07:26:05.736821+0000","last_peered":"2024-03-06T07:26:05.736821+0000","last_clean":"2024-03-06T07:26:05.736821+0000","last_became_active":"2024-03-06T07:26:04.229842+0000","last_became_peered":"2024-03-06T07:26:04.229842+0000","last_unstale":"2024-03-06T07:26:05.736821+0000","last_undegraded":"2024-03-06T07:26:05.736821+0000","last_fullsized":"2024-03-06T07:26:05.736821+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:56.769422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735301+0000","last_change":"2024-03-06T07:26:03.235702+0000","last_active":"2024-03-06T07:26:05.735301+0000","last_peered":"2024-03-06T07:26:05.735301+0000","last_clean":"2024-03-06T07:26:05.735301+0000","last_became_active":"2024-03-06T07:26:03.235369+0000","last_became_peered":"2024-03-06T07:26:03.235369+0000","last_unstale":"2024-03-06T07:26:05.735301+0000","last_undegraded":"2024-03-06T07:26:05.735301+0000","last_fullsized":"2024-03-06T07:26:05.735301+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:24.176175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736143+0000","last_change":"2024-03-06T07:25:55.181498+0000","last_active":"2024-03-06T07:26:05.736143+0000","last_peered":"2024-03-06T07:26:05.736143+0000","last_clean":"2024-03-06T07:26:05.736143+0000","last_became_active":"2024-03-06T07:25:55.180900+0000","last_became_peered":"2024-03-06T07:25:55.180900+0000","last_unstale":"2024-03-06T07:26:05.736143+0000","last_undegraded":"2024-03-06T07:26:05.736143+0000","last_fullsized":"2024-03-06T07:26:05.736143+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:06:57.524924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"149'8","reported_seq":30,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735009+0000","last_change":"2024-03-06T07:25:54.177051+0000","last_active":"2024-03-06T07:26:05.735009+0000","last_peered":"2024-03-06T07:26:05.735009+0000","last_clean":"2024-03-06T07:26:05.735009+0000","last_became_active":"2024-03-06T07:25:54.176774+0000","last_became_peered":"2024-03-06T07:25:54.176774+0000","last_unstale":"2024-03-06T07:26:05.735009+0000","last_undegraded":"2024-03-06T07:26:05.735009+0000","last_fullsized":"2024-03-06T07:26:05.735009+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:21:14.207675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736360+0000","last_change":"2024-03-06T07:26:04.228270+0000","last_active":"2024-03-06T07:26:05.736360+0000","last_peered":"2024-03-06T07:26:05.736360+0000","last_clean":"2024-03-06T07:26:05.736360+0000","last_became_active":"2024-03-06T07:26:04.228032+0000","last_became_peered":"2024-03-06T07:26:04.228032+0000","last_unstale":"2024-03-06T07:26:05.736360+0000","last_undegraded":"2024-03-06T07:26:05.736360+0000","last_fullsized":"2024-03-06T07:26:05.736360+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:38.167351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735262+0000","last_change":"2024-03-06T07:26:03.225140+0000","last_active":"2024-03-06T07:26:05.735262+0000","last_peered":"2024-03-06T07:26:05.735262+0000","last_clean":"2024-03-06T07:26:05.735262+0000","last_became_active":"2024-03-06T07:26:03.224872+0000","last_became_peered":"2024-03-06T07:26:03.224872+0000","last_unstale":"2024-03-06T07:26:05.735262+0000","last_undegraded":"2024-03-06T07:26:05.735262+0000","last_fullsized":"2024-03-06T07:26:05.735262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:32.154450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734821+0000","last_change":"2024-03-06T07:25:55.190742+0000","last_active":"2024-03-06T07:26:05.734821+0000","last_peered":"2024-03-06T07:26:05.734821+0000","last_clean":"2024-03-06T07:26:05.734821+0000","last_became_active":"2024-03-06T07:25:55.190479+0000","last_became_peered":"2024-03-06T07:25:55.190479+0000","last_unstale":"2024-03-06T07:26:05.734821+0000","last_undegraded":"2024-03-06T07:26:05.734821+0000","last_fullsized":"2024-03-06T07:26:05.734821+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:58.297138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736183+0000","last_change":"2024-03-06T07:25:54.178661+0000","last_active":"2024-03-06T07:26:05.736183+0000","last_peered":"2024-03-06T07:26:05.736183+0000","last_clean":"2024-03-06T07:26:05.736183+0000","last_became_active":"2024-03-06T07:25:54.178379+0000","last_became_peered":"2024-03-06T07:25:54.178379+0000","last_unstale":"2024-03-06T07:26:05.736183+0000","last_undegraded":"2024-03-06T07:26:05.736183+0000","last_fullsized":"2024-03-06T07:26:05.736183+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:30.562792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734839+0000","last_change":"2024-03-06T07:26:04.234322+0000","last_active":"2024-03-06T07:26:05.734839+0000","last_peered":"2024-03-06T07:26:05.734839+0000","last_clean":"2024-03-06T07:26:05.734839+0000","last_became_active":"2024-03-06T07:26:04.234083+0000","last_became_peered":"2024-03-06T07:26:04.234083+0000","last_unstale":"2024-03-06T07:26:05.734839+0000","last_undegraded":"2024-03-06T07:26:05.734839+0000","last_fullsized":"2024-03-06T07:26:05.734839+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:40.831704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239653+0000","last_change":"2024-03-06T07:26:03.229781+0000","last_active":"2024-03-06T07:26:05.239653+0000","last_peered":"2024-03-06T07:26:05.239653+0000","last_clean":"2024-03-06T07:26:05.239653+0000","last_became_active":"2024-03-06T07:26:03.229551+0000","last_became_peered":"2024-03-06T07:26:03.229551+0000","last_unstale":"2024-03-06T07:26:05.239653+0000","last_undegraded":"2024-03-06T07:26:05.239653+0000","last_fullsized":"2024-03-06T07:26:05.239653+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:46.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240477+0000","last_change":"2024-03-06T07:25:55.178452+0000","last_active":"2024-03-06T07:26:05.240477+0000","last_peered":"2024-03-06T07:26:05.240477+0000","last_clean":"2024-03-06T07:26:05.240477+0000","last_became_active":"2024-03-06T07:25:55.178193+0000","last_became_peered":"2024-03-06T07:25:55.178193+0000","last_unstale":"2024-03-06T07:26:05.240477+0000","last_undegraded":"2024-03-06T07:26:05.240477+0000","last_fullsized":"2024-03-06T07:26:05.240477+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:32.109891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737256+0000","last_change":"2024-03-06T07:25:54.173705+0000","last_active":"2024-03-06T07:26:05.737256+0000","last_peered":"2024-03-06T07:26:05.737256+0000","last_clean":"2024-03-06T07:26:05.737256+0000","last_became_active":"2024-03-06T07:25:54.173438+0000","last_became_peered":"2024-03-06T07:25:54.173438+0000","last_unstale":"2024-03-06T07:26:05.737256+0000","last_undegraded":"2024-03-06T07:26:05.737256+0000","last_fullsized":"2024-03-06T07:26:05.737256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:51.526470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239552+0000","last_change":"2024-03-06T07:26:04.231621+0000","last_active":"2024-03-06T07:26:05.239552+0000","last_peered":"2024-03-06T07:26:05.239552+0000","last_clean":"2024-03-06T07:26:05.239552+0000","last_became_active":"2024-03-06T07:26:04.231349+0000","last_became_peered":"2024-03-06T07:26:04.231349+0000","last_unstale":"2024-03-06T07:26:05.239552+0000","last_undegraded":"2024-03-06T07:26:05.239552+0000","last_fullsized":"2024-03-06T07:26:05.239552+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:29:25.825904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734908+0000","last_change":"2024-03-06T07:26:03.228059+0000","last_active":"2024-03-06T07:26:05.734908+0000","last_peered":"2024-03-06T07:26:05.734908+0000","last_clean":"2024-03-06T07:26:05.734908+0000","last_became_active":"2024-03-06T07:26:03.227604+0000","last_became_peered":"2024-03-06T07:26:03.227604+0000","last_unstale":"2024-03-06T07:26:05.734908+0000","last_undegraded":"2024-03-06T07:26:05.734908+0000","last_fullsized":"2024-03-06T07:26:05.734908+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:40:40.001196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735180+0000","last_change":"2024-03-06T07:25:55.172838+0000","last_active":"2024-03-06T07:26:05.735180+0000","last_peered":"2024-03-06T07:26:05.735180+0000","last_clean":"2024-03-06T07:26:05.735180+0000","last_became_active":"2024-03-06T07:25:55.172505+0000","last_became_peered":"2024-03-06T07:25:55.172505+0000","last_unstale":"2024-03-06T07:26:05.735180+0000","last_undegraded":"2024-03-06T07:26:05.735180+0000","last_fullsized":"2024-03-06T07:26:05.735180+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:45:25.784809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"149'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233712+0000","last_change":"2024-03-06T07:25:54.177100+0000","last_active":"2024-03-06T07:26:05.233712+0000","last_peered":"2024-03-06T07:26:05.233712+0000","last_clean":"2024-03-06T07:26:05.233712+0000","last_became_active":"2024-03-06T07:25:54.176822+0000","last_became_peered":"2024-03-06T07:25:54.176822+0000","last_unstale":"2024-03-06T07:26:05.233712+0000","last_undegraded":"2024-03-06T07:26:05.233712+0000","last_fullsized":"2024-03-06T07:26:05.233712+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.349070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.238660+0000","last_change":"2024-03-06T07:26:04.231459+0000","last_active":"2024-03-06T07:26:05.238660+0000","last_peered":"2024-03-06T07:26:05.238660+0000","last_clean":"2024-03-06T07:26:05.238660+0000","last_became_active":"2024-03-06T07:26:04.231187+0000","last_became_peered":"2024-03-06T07:26:04.231187+0000","last_unstale":"2024-03-06T07:26:05.238660+0000","last_undegraded":"2024-03-06T07:26:05.238660+0000","last_fullsized":"2024-03-06T07:26:05.238660+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:56.178408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734520+0000","last_change":"2024-03-06T07:26:03.230950+0000","last_active":"2024-03-06T07:26:05.734520+0000","last_peered":"2024-03-06T07:26:05.734520+0000","last_clean":"2024-03-06T07:26:05.734520+0000","last_became_active":"2024-03-06T07:26:03.230671+0000","last_became_peered":"2024-03-06T07:26:03.230671+0000","last_unstale":"2024-03-06T07:26:05.734520+0000","last_undegraded":"2024-03-06T07:26:05.734520+0000","last_fullsized":"2024-03-06T07:26:05.734520+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:17.498212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735327+0000","last_change":"2024-03-06T07:25:55.183106+0000","last_active":"2024-03-06T07:26:05.735327+0000","last_peered":"2024-03-06T07:26:05.735327+0000","last_clean":"2024-03-06T07:26:05.735327+0000","last_became_active":"2024-03-06T07:25:55.182843+0000","last_became_peered":"2024-03-06T07:25:55.182843+0000","last_unstale":"2024-03-06T07:26:05.735327+0000","last_undegraded":"2024-03-06T07:26:05.735327+0000","last_fullsized":"2024-03-06T07:26:05.735327+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:38:52.531694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736933+0000","last_change":"2024-03-06T07:25:54.172691+0000","last_active":"2024-03-06T07:26:05.736933+0000","last_peered":"2024-03-06T07:26:05.736933+0000","last_clean":"2024-03-06T07:26:05.736933+0000","last_became_active":"2024-03-06T07:25:54.172355+0000","last_became_peered":"2024-03-06T07:25:54.172355+0000","last_unstale":"2024-03-06T07:26:05.736933+0000","last_undegraded":"2024-03-06T07:26:05.736933+0000","last_fullsized":"2024-03-06T07:26:05.736933+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.756681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735899+0000","last_change":"2024-03-06T07:26:04.232191+0000","last_active":"2024-03-06T07:26:05.735899+0000","last_peered":"2024-03-06T07:26:05.735899+0000","last_clean":"2024-03-06T07:26:05.735899+0000","last_became_active":"2024-03-06T07:26:04.231117+0000","last_became_peered":"2024-03-06T07:26:04.231117+0000","last_unstale":"2024-03-06T07:26:05.735899+0000","last_undegraded":"2024-03-06T07:26:05.735899+0000","last_fullsized":"2024-03-06T07:26:05.735899+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:04.508750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736119+0000","last_change":"2024-03-06T07:26:03.234210+0000","last_active":"2024-03-06T07:26:05.736119+0000","last_peered":"2024-03-06T07:26:05.736119+0000","last_clean":"2024-03-06T07:26:05.736119+0000","last_became_active":"2024-03-06T07:26:03.233949+0000","last_became_peered":"2024-03-06T07:26:03.233949+0000","last_unstale":"2024-03-06T07:26:05.736119+0000","last_undegraded":"2024-03-06T07:26:05.736119+0000","last_fullsized":"2024-03-06T07:26:05.736119+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:48.748869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735094+0000","last_change":"2024-03-06T07:25:55.179180+0000","last_active":"2024-03-06T07:26:05.735094+0000","last_peered":"2024-03-06T07:26:05.735094+0000","last_clean":"2024-03-06T07:26:05.735094+0000","last_became_active":"2024-03-06T07:25:55.178890+0000","last_became_peered":"2024-03-06T07:25:55.178890+0000","last_unstale":"2024-03-06T07:26:05.735094+0000","last_undegraded":"2024-03-06T07:26:05.735094+0000","last_fullsized":"2024-03-06T07:26:05.735094+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:00.318395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233694+0000","last_change":"2024-03-06T07:25:54.181928+0000","last_active":"2024-03-06T07:26:05.233694+0000","last_peered":"2024-03-06T07:26:05.233694+0000","last_clean":"2024-03-06T07:26:05.233694+0000","last_became_active":"2024-03-06T07:25:54.181140+0000","last_became_peered":"2024-03-06T07:25:54.181140+0000","last_unstale":"2024-03-06T07:26:05.233694+0000","last_undegraded":"2024-03-06T07:26:05.233694+0000","last_fullsized":"2024-03-06T07:26:05.233694+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:54:57.291913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735985+0000","last_change":"2024-03-06T07:26:04.229256+0000","last_active":"2024-03-06T07:26:05.735985+0000","last_peered":"2024-03-06T07:26:05.735985+0000","last_clean":"2024-03-06T07:26:05.735985+0000","last_became_active":"2024-03-06T07:26:04.229001+0000","last_became_peered":"2024-03-06T07:26:04.229001+0000","last_unstale":"2024-03-06T07:26:05.735985+0000","last_undegraded":"2024-03-06T07:26:05.735985+0000","last_fullsized":"2024-03-06T07:26:05.735985+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:41.161598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736327+0000","last_change":"2024-03-06T07:26:03.232499+0000","last_active":"2024-03-06T07:26:05.736327+0000","last_peered":"2024-03-06T07:26:05.736327+0000","last_clean":"2024-03-06T07:26:05.736327+0000","last_became_active":"2024-03-06T07:26:03.232246+0000","last_became_peered":"2024-03-06T07:26:03.232246+0000","last_unstale":"2024-03-06T07:26:05.736327+0000","last_undegraded":"2024-03-06T07:26:05.736327+0000","last_fullsized":"2024-03-06T07:26:05.736327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:11:01.982286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233667+0000","last_change":"2024-03-06T07:25:55.192859+0000","last_active":"2024-03-06T07:26:05.233667+0000","last_peered":"2024-03-06T07:26:05.233667+0000","last_clean":"2024-03-06T07:26:05.233667+0000","last_became_active":"2024-03-06T07:25:55.192570+0000","last_became_peered":"2024-03-06T07:25:55.192570+0000","last_unstale":"2024-03-06T07:26:05.233667+0000","last_undegraded":"2024-03-06T07:26:05.233667+0000","last_fullsized":"2024-03-06T07:26:05.233667+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:00.192724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735328+0000","last_change":"2024-03-06T07:25:54.180637+0000","last_active":"2024-03-06T07:26:05.735328+0000","last_peered":"2024-03-06T07:26:05.735328+0000","last_clean":"2024-03-06T07:26:05.735328+0000","last_became_active":"2024-03-06T07:25:54.180270+0000","last_became_peered":"2024-03-06T07:25:54.180270+0000","last_unstale":"2024-03-06T07:26:05.735328+0000","last_undegraded":"2024-03-06T07:26:05.735328+0000","last_fullsized":"2024-03-06T07:26:05.735328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:28:20.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735712+0000","last_change":"2024-03-06T07:26:04.228643+0000","last_active":"2024-03-06T07:26:05.735712+0000","last_peered":"2024-03-06T07:26:05.735712+0000","last_clean":"2024-03-06T07:26:05.735712+0000","last_became_active":"2024-03-06T07:26:04.228385+0000","last_became_peered":"2024-03-06T07:26:04.228385+0000","last_unstale":"2024-03-06T07:26:05.735712+0000","last_undegraded":"2024-03-06T07:26:05.735712+0000","last_fullsized":"2024-03-06T07:26:05.735712+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:47:53.397866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233946+0000","last_change":"2024-03-06T07:26:03.226146+0000","last_active":"2024-03-06T07:26:05.233946+0000","last_peered":"2024-03-06T07:26:05.233946+0000","last_clean":"2024-03-06T07:26:05.233946+0000","last_became_active":"2024-03-06T07:26:03.225906+0000","last_became_peered":"2024-03-06T07:26:03.225906+0000","last_unstale":"2024-03-06T07:26:05.233946+0000","last_undegraded":"2024-03-06T07:26:05.233946+0000","last_fullsized":"2024-03-06T07:26:05.233946+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:08.360132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233781+0000","last_change":"2024-03-06T07:25:55.194175+0000","last_active":"2024-03-06T07:26:05.233781+0000","last_peered":"2024-03-06T07:26:05.233781+0000","last_clean":"2024-03-06T07:26:05.233781+0000","last_became_active":"2024-03-06T07:25:55.192548+0000","last_became_peered":"2024-03-06T07:25:55.192548+0000","last_unstale":"2024-03-06T07:26:05.233781+0000","last_undegraded":"2024-03-06T07:26:05.233781+0000","last_fullsized":"2024-03-06T07:26:05.233781+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:59:09.456474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735363+0000","last_change":"2024-03-06T07:25:54.171781+0000","last_active":"2024-03-06T07:26:05.735363+0000","last_peered":"2024-03-06T07:26:05.735363+0000","last_clean":"2024-03-06T07:26:05.735363+0000","last_became_active":"2024-03-06T07:25:54.171466+0000","last_became_peered":"2024-03-06T07:25:54.171466+0000","last_unstale":"2024-03-06T07:26:05.735363+0000","last_undegraded":"2024-03-06T07:26:05.735363+0000","last_fullsized":"2024-03-06T07:26:05.735363+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:56:50.533388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737092+0000","last_change":"2024-03-06T07:26:04.227633+0000","last_active":"2024-03-06T07:26:05.737092+0000","last_peered":"2024-03-06T07:26:05.737092+0000","last_clean":"2024-03-06T07:26:05.737092+0000","last_became_active":"2024-03-06T07:26:04.227384+0000","last_became_peered":"2024-03-06T07:26:04.227384+0000","last_unstale":"2024-03-06T07:26:05.737092+0000","last_undegraded":"2024-03-06T07:26:05.737092+0000","last_fullsized":"2024-03-06T07:26:05.737092+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:34.455713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735997+0000","last_change":"2024-03-06T07:26:03.227238+0000","last_active":"2024-03-06T07:26:05.735997+0000","last_peered":"2024-03-06T07:26:05.735997+0000","last_clean":"2024-03-06T07:26:05.735997+0000","last_became_active":"2024-03-06T07:26:03.226984+0000","last_became_peered":"2024-03-06T07:26:03.226984+0000","last_unstale":"2024-03-06T07:26:05.735997+0000","last_undegraded":"2024-03-06T07:26:05.735997+0000","last_fullsized":"2024-03-06T07:26:05.735997+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:21:16.810241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736055+0000","last_change":"2024-03-06T07:25:55.173561+0000","last_active":"2024-03-06T07:26:05.736055+0000","last_peered":"2024-03-06T07:26:05.736055+0000","last_clean":"2024-03-06T07:26:05.736055+0000","last_became_active":"2024-03-06T07:25:55.173309+0000","last_became_peered":"2024-03-06T07:25:55.173309+0000","last_unstale":"2024-03-06T07:26:05.736055+0000","last_undegraded":"2024-03-06T07:26:05.736055+0000","last_fullsized":"2024-03-06T07:26:05.736055+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.126701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735126+0000","last_change":"2024-03-06T07:25:54.178755+0000","last_active":"2024-03-06T07:26:05.735126+0000","last_peered":"2024-03-06T07:26:05.735126+0000","last_clean":"2024-03-06T07:26:05.735126+0000","last_became_active":"2024-03-06T07:25:54.178492+0000","last_became_peered":"2024-03-06T07:25:54.178492+0000","last_unstale":"2024-03-06T07:26:05.735126+0000","last_undegraded":"2024-03-06T07:26:05.735126+0000","last_fullsized":"2024-03-06T07:26:05.735126+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:18.439323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234211+0000","last_change":"2024-03-06T07:26:04.233227+0000","last_active":"2024-03-06T07:26:05.234211+0000","last_peered":"2024-03-06T07:26:05.234211+0000","last_clean":"2024-03-06T07:26:05.234211+0000","last_became_active":"2024-03-06T07:26:04.232960+0000","last_became_peered":"2024-03-06T07:26:04.232960+0000","last_unstale":"2024-03-06T07:26:05.234211+0000","last_undegraded":"2024-03-06T07:26:05.234211+0000","last_fullsized":"2024-03-06T07:26:05.234211+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:25.186145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734222+0000","last_change":"2024-03-06T07:26:03.231682+0000","last_active":"2024-03-06T07:26:05.734222+0000","last_peered":"2024-03-06T07:26:05.734222+0000","last_clean":"2024-03-06T07:26:05.734222+0000","last_became_active":"2024-03-06T07:26:03.231429+0000","last_became_peered":"2024-03-06T07:26:03.231429+0000","last_unstale":"2024-03-06T07:26:05.734222+0000","last_undegraded":"2024-03-06T07:26:05.734222+0000","last_fullsized":"2024-03-06T07:26:05.734222+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:32.804654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734523+0000","last_change":"2024-03-06T07:25:55.172471+0000","last_active":"2024-03-06T07:26:05.734523+0000","last_peered":"2024-03-06T07:26:05.734523+0000","last_clean":"2024-03-06T07:26:05.734523+0000","last_became_active":"2024-03-06T07:25:55.172189+0000","last_became_peered":"2024-03-06T07:25:55.172189+0000","last_unstale":"2024-03-06T07:26:05.734523+0000","last_undegraded":"2024-03-06T07:26:05.734523+0000","last_fullsized":"2024-03-06T07:26:05.734523+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:53.761906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"149'2","reported_seq":24,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.234086+0000","last_change":"2024-03-06T07:25:54.171718+0000","last_active":"2024-03-06T07:26:05.234086+0000","last_peered":"2024-03-06T07:26:05.234086+0000","last_clean":"2024-03-06T07:26:05.234086+0000","last_became_active":"2024-03-06T07:25:54.171431+0000","last_became_peered":"2024-03-06T07:25:54.171431+0000","last_unstale":"2024-03-06T07:26:05.234086+0000","last_undegraded":"2024-03-06T07:26:05.234086+0000","last_fullsized":"2024-03-06T07:26:05.234086+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:51.464247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734939+0000","last_change":"2024-03-06T07:26:04.225352+0000","last_active":"2024-03-06T07:26:05.734939+0000","last_peered":"2024-03-06T07:26:05.734939+0000","last_clean":"2024-03-06T07:26:05.734939+0000","last_became_active":"2024-03-06T07:26:04.225106+0000","last_became_peered":"2024-03-06T07:26:04.225106+0000","last_unstale":"2024-03-06T07:26:05.734939+0000","last_undegraded":"2024-03-06T07:26:05.734939+0000","last_fullsized":"2024-03-06T07:26:05.734939+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:53:38.971076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239849+0000","last_change":"2024-03-06T07:26:03.226686+0000","last_active":"2024-03-06T07:26:05.239849+0000","last_peered":"2024-03-06T07:26:05.239849+0000","last_clean":"2024-03-06T07:26:05.239849+0000","last_became_active":"2024-03-06T07:26:03.226406+0000","last_became_peered":"2024-03-06T07:26:03.226406+0000","last_unstale":"2024-03-06T07:26:05.239849+0000","last_undegraded":"2024-03-06T07:26:05.239849+0000","last_fullsized":"2024-03-06T07:26:05.239849+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:06.496358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735742+0000","last_change":"2024-03-06T07:25:55.184864+0000","last_active":"2024-03-06T07:26:05.735742+0000","last_peered":"2024-03-06T07:26:05.735742+0000","last_clean":"2024-03-06T07:26:05.735742+0000","last_became_active":"2024-03-06T07:25:55.184462+0000","last_became_peered":"2024-03-06T07:25:55.184462+0000","last_unstale":"2024-03-06T07:26:05.735742+0000","last_undegraded":"2024-03-06T07:26:05.735742+0000","last_fullsized":"2024-03-06T07:26:05.735742+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:36.253680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"150'1","reported_seq":23,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735866+0000","last_change":"2024-03-06T07:25:54.184297+0000","last_active":"2024-03-06T07:26:05.735866+0000","last_peered":"2024-03-06T07:26:05.735866+0000","last_clean":"2024-03-06T07:26:05.735866+0000","last_became_active":"2024-03-06T07:25:54.184000+0000","last_became_peered":"2024-03-06T07:25:54.184000+0000","last_unstale":"2024-03-06T07:26:05.735866+0000","last_undegraded":"2024-03-06T07:26:05.735866+0000","last_fullsized":"2024-03-06T07:26:05.735866+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:06.525995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734539+0000","last_change":"2024-03-06T07:26:04.228678+0000","last_active":"2024-03-06T07:26:05.734539+0000","last_peered":"2024-03-06T07:26:05.734539+0000","last_clean":"2024-03-06T07:26:05.734539+0000","last_became_active":"2024-03-06T07:26:04.228431+0000","last_became_peered":"2024-03-06T07:26:04.228431+0000","last_unstale":"2024-03-06T07:26:05.734539+0000","last_undegraded":"2024-03-06T07:26:05.734539+0000","last_fullsized":"2024-03-06T07:26:05.734539+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:12.665130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239204+0000","last_change":"2024-03-06T07:26:03.226063+0000","last_active":"2024-03-06T07:26:05.239204+0000","last_peered":"2024-03-06T07:26:05.239204+0000","last_clean":"2024-03-06T07:26:05.239204+0000","last_became_active":"2024-03-06T07:26:03.225610+0000","last_became_peered":"2024-03-06T07:26:03.225610+0000","last_unstale":"2024-03-06T07:26:05.239204+0000","last_undegraded":"2024-03-06T07:26:05.239204+0000","last_fullsized":"2024-03-06T07:26:05.239204+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:24.862106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735716+0000","last_change":"2024-03-06T07:25:55.173629+0000","last_active":"2024-03-06T07:26:05.735716+0000","last_peered":"2024-03-06T07:26:05.735716+0000","last_clean":"2024-03-06T07:26:05.735716+0000","last_became_active":"2024-03-06T07:25:55.173365+0000","last_became_peered":"2024-03-06T07:25:55.173365+0000","last_unstale":"2024-03-06T07:26:05.735716+0000","last_undegraded":"2024-03-06T07:26:05.735716+0000","last_fullsized":"2024-03-06T07:26:05.735716+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:24.989175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736321+0000","last_change":"2024-03-06T07:25:54.180119+0000","last_active":"2024-03-06T07:26:05.736321+0000","last_peered":"2024-03-06T07:26:05.736321+0000","last_clean":"2024-03-06T07:26:05.736321+0000","last_became_active":"2024-03-06T07:25:54.179853+0000","last_became_peered":"2024-03-06T07:25:54.179853+0000","last_unstale":"2024-03-06T07:26:05.736321+0000","last_undegraded":"2024-03-06T07:26:05.736321+0000","last_fullsized":"2024-03-06T07:26:05.736321+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:31:43.160051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233271+0000","last_change":"2024-03-06T07:26:04.232203+0000","last_active":"2024-03-06T07:26:05.233271+0000","last_peered":"2024-03-06T07:26:05.233271+0000","last_clean":"2024-03-06T07:26:05.233271+0000","last_became_active":"2024-03-06T07:26:04.231938+0000","last_became_peered":"2024-03-06T07:26:04.231938+0000","last_unstale":"2024-03-06T07:26:05.233271+0000","last_undegraded":"2024-03-06T07:26:05.233271+0000","last_fullsized":"2024-03-06T07:26:05.233271+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:43:16.379024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233291+0000","last_change":"2024-03-06T07:26:03.226739+0000","last_active":"2024-03-06T07:26:05.233291+0000","last_peered":"2024-03-06T07:26:05.233291+0000","last_clean":"2024-03-06T07:26:05.233291+0000","last_became_active":"2024-03-06T07:26:03.226483+0000","last_became_peered":"2024-03-06T07:26:03.226483+0000","last_unstale":"2024-03-06T07:26:05.233291+0000","last_undegraded":"2024-03-06T07:26:05.233291+0000","last_fullsized":"2024-03-06T07:26:05.233291+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:04.216343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736674+0000","last_change":"2024-03-06T07:25:55.184654+0000","last_active":"2024-03-06T07:26:05.736674+0000","last_peered":"2024-03-06T07:26:05.736674+0000","last_clean":"2024-03-06T07:26:05.736674+0000","last_became_active":"2024-03-06T07:25:55.184305+0000","last_became_peered":"2024-03-06T07:25:55.184305+0000","last_unstale":"2024-03-06T07:26:05.736674+0000","last_undegraded":"2024-03-06T07:26:05.736674+0000","last_fullsized":"2024-03-06T07:26:05.736674+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:52:20.570479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735859+0000","last_change":"2024-03-06T07:25:54.174742+0000","last_active":"2024-03-06T07:26:05.735859+0000","last_peered":"2024-03-06T07:26:05.735859+0000","last_clean":"2024-03-06T07:26:05.735859+0000","last_became_active":"2024-03-06T07:25:54.174462+0000","last_became_peered":"2024-03-06T07:25:54.174462+0000","last_unstale":"2024-03-06T07:26:05.735859+0000","last_undegraded":"2024-03-06T07:26:05.735859+0000","last_fullsized":"2024-03-06T07:26:05.735859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:47:43.050877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736288+0000","last_change":"2024-03-06T07:26:04.238292+0000","last_active":"2024-03-06T07:26:05.736288+0000","last_peered":"2024-03-06T07:26:05.736288+0000","last_clean":"2024-03-06T07:26:05.736288+0000","last_became_active":"2024-03-06T07:26:04.237876+0000","last_became_peered":"2024-03-06T07:26:04.237876+0000","last_unstale":"2024-03-06T07:26:05.736288+0000","last_undegraded":"2024-03-06T07:26:05.736288+0000","last_fullsized":"2024-03-06T07:26:05.736288+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:09.244431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240046+0000","last_change":"2024-03-06T07:26:03.233488+0000","last_active":"2024-03-06T07:26:05.240046+0000","last_peered":"2024-03-06T07:26:05.240046+0000","last_clean":"2024-03-06T07:26:05.240046+0000","last_became_active":"2024-03-06T07:26:03.233211+0000","last_became_peered":"2024-03-06T07:26:03.233211+0000","last_unstale":"2024-03-06T07:26:05.240046+0000","last_undegraded":"2024-03-06T07:26:05.240046+0000","last_fullsized":"2024-03-06T07:26:05.240046+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:53:27.486285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.239291+0000","last_change":"2024-03-06T07:25:55.181956+0000","last_active":"2024-03-06T07:26:05.239291+0000","last_peered":"2024-03-06T07:26:05.239291+0000","last_clean":"2024-03-06T07:26:05.239291+0000","last_became_active":"2024-03-06T07:25:55.181658+0000","last_became_peered":"2024-03-06T07:25:55.181658+0000","last_unstale":"2024-03-06T07:26:05.239291+0000","last_undegraded":"2024-03-06T07:26:05.239291+0000","last_fullsized":"2024-03-06T07:26:05.239291+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:27:23.152358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240044+0000","last_change":"2024-03-06T07:25:54.172421+0000","last_active":"2024-03-06T07:26:05.240044+0000","last_peered":"2024-03-06T07:26:05.240044+0000","last_clean":"2024-03-06T07:26:05.240044+0000","last_became_active":"2024-03-06T07:25:54.172157+0000","last_became_peered":"2024-03-06T07:25:54.172157+0000","last_unstale":"2024-03-06T07:26:05.240044+0000","last_undegraded":"2024-03-06T07:26:05.240044+0000","last_fullsized":"2024-03-06T07:26:05.240044+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:07:07.283064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.737486+0000","last_change":"2024-03-06T07:26:04.228518+0000","last_active":"2024-03-06T07:26:05.737486+0000","last_peered":"2024-03-06T07:26:05.737486+0000","last_clean":"2024-03-06T07:26:05.737486+0000","last_became_active":"2024-03-06T07:26:04.228278+0000","last_became_peered":"2024-03-06T07:26:04.228278+0000","last_unstale":"2024-03-06T07:26:05.737486+0000","last_undegraded":"2024-03-06T07:26:05.737486+0000","last_fullsized":"2024-03-06T07:26:05.737486+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:49.758839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735231+0000","last_change":"2024-03-06T07:26:03.226623+0000","last_active":"2024-03-06T07:26:05.735231+0000","last_peered":"2024-03-06T07:26:05.735231+0000","last_clean":"2024-03-06T07:26:05.735231+0000","last_became_active":"2024-03-06T07:26:03.226351+0000","last_became_peered":"2024-03-06T07:26:03.226351+0000","last_unstale":"2024-03-06T07:26:05.735231+0000","last_undegraded":"2024-03-06T07:26:05.735231+0000","last_fullsized":"2024-03-06T07:26:05.735231+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:55:06.338875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735662+0000","last_change":"2024-03-06T07:25:55.183461+0000","last_active":"2024-03-06T07:26:05.735662+0000","last_peered":"2024-03-06T07:26:05.735662+0000","last_clean":"2024-03-06T07:26:05.735662+0000","last_became_active":"2024-03-06T07:25:55.183161+0000","last_became_peered":"2024-03-06T07:25:55.183161+0000","last_unstale":"2024-03-06T07:26:05.735662+0000","last_undegraded":"2024-03-06T07:26:05.735662+0000","last_fullsized":"2024-03-06T07:26:05.735662+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:02.272014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735767+0000","last_change":"2024-03-06T07:25:54.182221+0000","last_active":"2024-03-06T07:26:05.735767+0000","last_peered":"2024-03-06T07:26:05.735767+0000","last_clean":"2024-03-06T07:26:05.735767+0000","last_became_active":"2024-03-06T07:25:54.181966+0000","last_became_peered":"2024-03-06T07:25:54.181966+0000","last_unstale":"2024-03-06T07:26:05.735767+0000","last_undegraded":"2024-03-06T07:26:05.735767+0000","last_fullsized":"2024-03-06T07:26:05.735767+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:48.357604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735561+0000","last_change":"2024-03-06T07:26:04.231324+0000","last_active":"2024-03-06T07:26:05.735561+0000","last_peered":"2024-03-06T07:26:05.735561+0000","last_clean":"2024-03-06T07:26:05.735561+0000","last_became_active":"2024-03-06T07:26:04.231041+0000","last_became_peered":"2024-03-06T07:26:04.231041+0000","last_unstale":"2024-03-06T07:26:05.735561+0000","last_undegraded":"2024-03-06T07:26:05.735561+0000","last_fullsized":"2024-03-06T07:26:05.735561+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:09.032723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736198+0000","last_change":"2024-03-06T07:26:03.232763+0000","last_active":"2024-03-06T07:26:05.736198+0000","last_peered":"2024-03-06T07:26:05.736198+0000","last_clean":"2024-03-06T07:26:05.736198+0000","last_became_active":"2024-03-06T07:26:03.232424+0000","last_became_peered":"2024-03-06T07:26:03.232424+0000","last_unstale":"2024-03-06T07:26:05.736198+0000","last_undegraded":"2024-03-06T07:26:05.736198+0000","last_fullsized":"2024-03-06T07:26:05.736198+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:01.280176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735694+0000","last_change":"2024-03-06T07:25:55.178896+0000","last_active":"2024-03-06T07:26:05.735694+0000","last_peered":"2024-03-06T07:26:05.735694+0000","last_clean":"2024-03-06T07:26:05.735694+0000","last_became_active":"2024-03-06T07:25:55.178599+0000","last_became_peered":"2024-03-06T07:25:55.178599+0000","last_unstale":"2024-03-06T07:26:05.735694+0000","last_undegraded":"2024-03-06T07:26:05.735694+0000","last_fullsized":"2024-03-06T07:26:05.735694+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:37:34.372833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735130+0000","last_change":"2024-03-06T07:25:54.176751+0000","last_active":"2024-03-06T07:26:05.735130+0000","last_peered":"2024-03-06T07:26:05.735130+0000","last_clean":"2024-03-06T07:26:05.735130+0000","last_became_active":"2024-03-06T07:25:54.176461+0000","last_became_peered":"2024-03-06T07:25:54.176461+0000","last_unstale":"2024-03-06T07:26:05.735130+0000","last_undegraded":"2024-03-06T07:26:05.735130+0000","last_fullsized":"2024-03-06T07:26:05.735130+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:09:09.865297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735143+0000","last_change":"2024-03-06T07:26:04.239654+0000","last_active":"2024-03-06T07:26:05.735143+0000","last_peered":"2024-03-06T07:26:05.735143+0000","last_clean":"2024-03-06T07:26:05.735143+0000","last_became_active":"2024-03-06T07:26:04.239353+0000","last_became_peered":"2024-03-06T07:26:04.239353+0000","last_unstale":"2024-03-06T07:26:05.735143+0000","last_undegraded":"2024-03-06T07:26:05.735143+0000","last_fullsized":"2024-03-06T07:26:05.735143+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:51:27.236434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233614+0000","last_change":"2024-03-06T07:26:03.218889+0000","last_active":"2024-03-06T07:26:05.233614+0000","last_peered":"2024-03-06T07:26:05.233614+0000","last_clean":"2024-03-06T07:26:05.233614+0000","last_became_active":"2024-03-06T07:26:03.218638+0000","last_became_peered":"2024-03-06T07:26:03.218638+0000","last_unstale":"2024-03-06T07:26:05.233614+0000","last_undegraded":"2024-03-06T07:26:05.233614+0000","last_fullsized":"2024-03-06T07:26:05.233614+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:03.498780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.736254+0000","last_change":"2024-03-06T07:25:55.184289+0000","last_active":"2024-03-06T07:26:05.736254+0000","last_peered":"2024-03-06T07:26:05.736254+0000","last_clean":"2024-03-06T07:26:05.736254+0000","last_became_active":"2024-03-06T07:25:55.184007+0000","last_became_peered":"2024-03-06T07:25:55.184007+0000","last_unstale":"2024-03-06T07:26:05.736254+0000","last_undegraded":"2024-03-06T07:26:05.736254+0000","last_fullsized":"2024-03-06T07:26:05.736254+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:52:09.480817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233157+0000","last_change":"2024-03-06T07:25:54.182027+0000","last_active":"2024-03-06T07:26:05.233157+0000","last_peered":"2024-03-06T07:26:05.233157+0000","last_clean":"2024-03-06T07:26:05.233157+0000","last_became_active":"2024-03-06T07:25:54.181755+0000","last_became_peered":"2024-03-06T07:25:54.181755+0000","last_unstale":"2024-03-06T07:26:05.233157+0000","last_undegraded":"2024-03-06T07:26:05.233157+0000","last_fullsized":"2024-03-06T07:26:05.233157+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:03.374674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.240556+0000","last_change":"2024-03-06T07:26:04.241623+0000","last_active":"2024-03-06T07:26:05.240556+0000","last_peered":"2024-03-06T07:26:05.240556+0000","last_clean":"2024-03-06T07:26:05.240556+0000","last_became_active":"2024-03-06T07:26:04.241336+0000","last_became_peered":"2024-03-06T07:26:04.241336+0000","last_unstale":"2024-03-06T07:26:05.240556+0000","last_undegraded":"2024-03-06T07:26:05.240556+0000","last_fullsized":"2024-03-06T07:26:05.240556+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:03.205056+0000","last_clean_scrub_stamp":"2024-03-06T07:26:03.205056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:39:01.583402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.233871+0000","last_change":"2024-03-06T07:26:03.222524+0000","last_active":"2024-03-06T07:26:05.233871+0000","last_peered":"2024-03-06T07:26:05.233871+0000","last_clean":"2024-03-06T07:26:05.233871+0000","last_became_active":"2024-03-06T07:26:03.222275+0000","last_became_peered":"2024-03-06T07:26:03.222275+0000","last_unstale":"2024-03-06T07:26:05.233871+0000","last_undegraded":"2024-03-06T07:26:05.233871+0000","last_fullsized":"2024-03-06T07:26:05.233871+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:02.195086+0000","last_clean_scrub_stamp":"2024-03-06T07:26:02.195086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:11.930055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.734632+0000","last_change":"2024-03-06T07:25:55.176555+0000","last_active":"2024-03-06T07:26:05.734632+0000","last_peered":"2024-03-06T07:26:05.734632+0000","last_clean":"2024-03-06T07:26:05.734632+0000","last_became_active":"2024-03-06T07:25:55.176295+0000","last_became_peered":"2024-03-06T07:25:55.176295+0000","last_unstale":"2024-03-06T07:26:05.734632+0000","last_undegraded":"2024-03-06T07:26:05.734632+0000","last_fullsized":"2024-03-06T07:26:05.734632+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:54.153349+0000","last_clean_scrub_stamp":"2024-03-06T07:25:54.153349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:12:18.514838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735777+0000","last_change":"2024-03-06T07:25:54.184373+0000","last_active":"2024-03-06T07:26:05.735777+0000","last_peered":"2024-03-06T07:26:05.735777+0000","last_clean":"2024-03-06T07:26:05.735777+0000","last_became_active":"2024-03-06T07:25:54.184109+0000","last_became_peered":"2024-03-06T07:25:54.184109+0000","last_unstale":"2024-03-06T07:26:05.735777+0000","last_undegraded":"2024-03-06T07:26:05.735777+0000","last_fullsized":"2024-03-06T07:26:05.735777+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:25:53.148124+0000","last_clean_scrub_stamp":"2024-03-06T07:25:53.148124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:39:01.955245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-03-06T07:26:05.735233+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:26:05.735233+0000","last_peered":"2024-03-06T07:26:05.735233+0000","last_clean":"2024-03-06T07:26:05.735233+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:26:05.735233+0000","last_undegraded":"2024-03-06T07:26:05.735233+0000","last_fullsized":"2024-03-06T07:26:05.735233+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2002944,"data_stored":1534716,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1941504,"data_stored":944418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1961984,"data_stored":946106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":946758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1966080,"data_stored":946724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1941504,"data_stored":944930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":1949696,"data_stored":945960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2027520,"data_stored":1535548,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:08.739 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:26:08.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:08.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.871+0000 7febb1755640 1 Processor -- start 2024-03-06T07:26:08.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.871+0000 7febb1755640 1 -- start start 2024-03-06T07:26:08.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febb1755640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac14f650 0x7febac071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febb1755640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febac152c20 0x7febac15c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febb1755640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac072440 con 0x7febac0c3150 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febb1755640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0c2b10 con 0x7febac152c20 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febb1755640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac152140 con 0x7febac14f650 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febab7fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febac152c20 0x7febac15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febaaffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac14f650 0x7febac071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febaa7fc640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7febac0c3150 0x7febac0c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:56050/0 (socket says 172.21.15.17:56050) 2024-03-06T07:26:08.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febaa7fc640 1 -- 172.21.15.17:0/2455652542 learned_addr learned my addr 172.21.15.17:0/2455652542 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febab7fe640 1 -- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac14f650 msgr2=0x7febac071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febab7fe640 1 --2- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac14f650 0x7febac071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febab7fe640 1 -- 172.21.15.17:0/2455652542 >> v1:172.21.15.163:6789/0 conn(0x7febac0c3150 legacy=0x7febac0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7febab7fe640 1 -- 172.21.15.17:0/2455652542 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac152420 con 0x7febac152c20 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.873+0000 7feba9ffb640 1 -- 172.21.15.17:0/2455652542 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184865738 0 0) 0x7feba0001110 con 0x7febac0c3150 2024-03-06T07:26:08.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7febab7fe640 1 --2- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac152c20 0x7febac15c810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feba4009d90 tx=0x7feba402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a654af1d6a3d975 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7feba9ffb640 1 -- 172.21.15.17:0/2455652542 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4039070 con 0x7febac152c20 2024-03-06T07:26:08.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7feba9ffb640 1 -- 172.21.15.17:0/2455652542 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba402cc50 con 0x7febac152c20 2024-03-06T07:26:08.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7feba9ffb640 1 -- 172.21.15.17:0/2455652542 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4034440 con 0x7febac152c20 2024-03-06T07:26:08.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7febb1755640 1 -- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac152c20 msgr2=0x7febac15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.875+0000 7febb1755640 1 --2- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac152c20 0x7febac15c810 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feba4009d90 tx=0x7feba402bea0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 -- 172.21.15.17:0/2455652542 shutdown_connections 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 --2- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac152c20 0x7febac15c810 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 --2- 172.21.15.17:0/2455652542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac14f650 0x7febac071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 -- 172.21.15.17:0/2455652542 >> 172.21.15.17:0/2455652542 conn(0x7febac0c4290 msgr2=0x7febac0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 -- 172.21.15.17:0/2455652542 shutdown_connections 2024-03-06T07:26:08.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.876+0000 7febb1755640 1 -- 172.21.15.17:0/2455652542 wait complete. 2024-03-06T07:26:08.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.877+0000 7febb1755640 1 Processor -- start 2024-03-06T07:26:08.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.878+0000 7febb1755640 1 -- start start 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac14f650 0x7febac06f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac152c20 0x7febac1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c40c0 con 0x7febac0c3150 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c3f40 con 0x7febac14f650 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febb1755640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0c5da0 con 0x7febac152c20 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39738/0 (socket says 172.21.15.17:39738) 2024-03-06T07:26:08.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 -- 172.21.15.17:0/1949324830 learned_addr learned my addr 172.21.15.17:0/1949324830 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febab7fe640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac152c20 0x7febac1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaa7fc640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac14f650 0x7febac06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 -- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac152c20 msgr2=0x7febac1478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac152c20 0x7febac1478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 -- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac14f650 msgr2=0x7febac06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac14f650 0x7febac06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 -- 172.21.15.17:0/1949324830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac148510 con 0x7febac0c3150 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7febaaffd640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00e9e0 tx=0x7feb9c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c017070 con 0x7febac0c3150 2024-03-06T07:26:08.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.879+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c00c9d0 con 0x7febac0c3150 2024-03-06T07:26:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.880+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac1deea0 con 0x7febac0c3150 2024-03-06T07:26:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.880+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c00cdc0 con 0x7febac0c3150 2024-03-06T07:26:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.880+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac06d710 con 0x7febac0c3150 2024-03-06T07:26:08.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.881+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac04f330 con 0x7febac0c3150 2024-03-06T07:26:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.881+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb9c01a070 con 0x7febac0c3150 2024-03-06T07:26:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.882+0000 7feb937fe640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 0x7feb9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.882+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7feb9c08a950 con 0x7febac0c3150 2024-03-06T07:26:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.882+0000 7febaa7fc640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 0x7feb9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:08.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.883+0000 7febaa7fc640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 0x7feb9806d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7febac0c6430 tx=0x7feba0007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:08.891+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb9c058d30 con 0x7febac0c3150 2024-03-06T07:26:09.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.109+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febac068460 con 0x7febac0c3150 2024-03-06T07:26:09.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.110+0000 7feb937fe640 1 -- 172.21.15.17:0/1949324830 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+6266 (secure 0 0 0) 0x7feb9c05cdd0 con 0x7febac0c3150 2024-03-06T07:26:09.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:09.112 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":141,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:25:56.622881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8781":{"gid":8781,"name":"c","rank":0,"incarnation":134,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2429596225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2429596225},{"type":"v1","addr":"172.21.15.17:6835","nonce":2429596225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8781,"qdb_cluster":[8781]},"id":11},{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:06.596155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8778":{"gid":8778,"name":"a","rank":0,"incarnation":137,"state":"up:active","state_seq":5,"addr":"172.21.15.17:6834/81289984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":81289984},{"type":"v1","addr":"172.21.15.17:6834","nonce":81289984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8778,"qdb_cluster":[8778]},"id":12}]} 2024-03-06T07:26:09.112 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 141 2024-03-06T07:26:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.111+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 msgr2=0x7feb9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.111+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 0x7feb9806d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7febac0c6430 tx=0x7feba0007b50 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.111+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 msgr2=0x7febac1519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.111+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00e9e0 tx=0x7feb9c00eea0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 shutdown_connections 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb9806b4d0 0x7feb9806d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febac152c20 0x7febac1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febac14f650 0x7febac06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 --2- 172.21.15.17:0/1949324830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febac0c3150 0x7febac1519f0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 >> 172.21.15.17:0/1949324830 conn(0x7febac0c4290 msgr2=0x7febac071410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 shutdown_connections 2024-03-06T07:26:09.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.112+0000 7febb1755640 1 -- 172.21.15.17:0/1949324830 wait complete. 2024-03-06T07:26:09.128 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 141, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:54.526748+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 136, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8781": { "addr": "172.21.15.17:6835/2429596225", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 2429596225, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 2429596225, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8781, "incarnation": 134, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:25:56.622881+0000", "qdb_cluster": [ 8781 ], "qdb_leader": 8781, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8781 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:03.654281+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 141, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8778": { "addr": "172.21.15.17:6834/81289984", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 81289984, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 81289984, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8778, "incarnation": 137, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:06.596155+0000", "qdb_cluster": [ 8778 ], "qdb_leader": 8778, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8778 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/91031835", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 91031835, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 91031835, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 133, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8200, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2502302602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2502302602, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2502302602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 133, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8203, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:09.128 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:26:09.129 INFO:tasks.cephfs.test_failover:current = [{'gid': 8778, 'name': 'a', 'rank': 0, 'incarnation': 137, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.17:6834/81289984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 81289984}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 81289984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8203, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 133}, {'gid': 8781, 'name': 'c', 'rank': 0, 'incarnation': 134, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2429596225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2429596225}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2429596225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8200, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/91031835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 91031835}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 91031835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 133}] 2024-03-06T07:26:09.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:09.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 Processor -- start 2024-03-06T07:26:09.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 -- start start 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f034c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f034c155470 0x7f034c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c14f890 con 0x7f034c14f4b0 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c0c2500 con 0x7f034c155470 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.244+0000 7f0350ba0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0bd2e0 con 0x7f034c0c3870 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f034c14f4b0 0x7f034c06f2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57324/0 (socket says 172.21.15.17:57324) 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034b7fe640 1 -- 172.21.15.17:0/3478432234 learned_addr learned my addr 172.21.15.17:0/3478432234 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034bfff640 1 --2- 172.21.15.17:0/3478432234 >> v2:172.21.15.163:3300/0 conn(0x7f034c155470 0x7f034c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034affd640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f034c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761264508 0 0) 0x7f034c14f890 con 0x7f034c14f4b0 2024-03-06T07:26:09.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0334003520 con 0x7f034c14f4b0 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988713776 0 0) 0x7f0334003520 con 0x7f034c14f4b0 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 msgr2=0x7f034c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f034c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 >> v2:172.21.15.163:3300/0 conn(0x7f034c155470 msgr2=0x7f034c15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 --2- 172.21.15.17:0/3478432234 >> v2:172.21.15.163:3300/0 conn(0x7f034c155470 0x7f034c15c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0c1f40 con 0x7f034c14f4b0 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f033c001670 con 0x7f034c14f4b0 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 >> v1:172.21.15.17:6789/0 conn(0x7f034c14f4b0 legacy=0x7f034c06f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0334003a70 0x7f0334003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f0334004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0334004d40 0x7f03340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c0bd2e0 con 0x7f0334004d40 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033400ba20 con 0x7f0334003a70 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033400bc20 con 0x7f034c0c3870 2024-03-06T07:26:09.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034bfff640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0334004d40 0x7f03340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034b7fe640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0334003a70 0x7f0334003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.245+0000 7f034affd640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f0334004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 -- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 msgr2=0x7f0334004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c0c3870 0x7f0334004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 -- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0334003a70 msgr2=0x7f0334003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0334003a70 0x7f0334003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 -- 172.21.15.17:0/3478432234 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334003520 con 0x7f0334004d40 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.246+0000 7f034bfff640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0334004d40 0x7f03340051c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f034c071120 tx=0x7f0334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e900693c8ffc2220 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344039070 con 0x7f0334004d40 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034402cbd0 con 0x7f0334004d40 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f034a7fc640 1 -- 172.21.15.17:0/3478432234 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344009c10 con 0x7f0334004d40 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 -- 172.21.15.17:0/3478432234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0334004d40 msgr2=0x7f03340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 --2- 172.21.15.17:0/3478432234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0334004d40 0x7f03340051c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f034c071120 tx=0x7f0334005830 comp rx=0 tx=0).stop 2024-03-06T07:26:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f034bfff640 1 -- 172.21.15.17:0/3478432234 reap_dead start 2024-03-06T07:26:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 -- 172.21.15.17:0/3478432234 shutdown_connections 2024-03-06T07:26:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 -- 172.21.15.17:0/3478432234 >> 172.21.15.17:0/3478432234 conn(0x7f034c0c4040 msgr2=0x7f034c0c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 -- 172.21.15.17:0/3478432234 shutdown_connections 2024-03-06T07:26:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.247+0000 7f0350ba0640 1 -- 172.21.15.17:0/3478432234 wait complete. 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.248+0000 7f0350ba0640 1 Processor -- start 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.248+0000 7f0350ba0640 1 -- start start 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c1dcef0 0x7f034c1dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f034c1eaa30 0x7f034c1eaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c06f9c0 con 0x7f034c150810 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c152090 con 0x7f034c1eaa30 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f0350ba0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c06fb40 con 0x7f034c1dcef0 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f034affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f034affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39758/0 (socket says 172.21.15.17:39758) 2024-03-06T07:26:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.249+0000 7f034affd640 1 -- 172.21.15.17:0/1274532985 learned_addr learned my addr 172.21.15.17:0/1274532985 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034b7fe640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c1dcef0 0x7f034c1dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034bfff640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f034c1eaa30 0x7f034c1eaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 -- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c1dcef0 msgr2=0x7f034c1dd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c1dcef0 0x7f034c1dd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 -- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f034c1eaa30 msgr2=0x7f034c1eaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f034c1eaa30 0x7f034c1eaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 -- 172.21.15.17:0/1274532985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c1ddf90 con 0x7f034c150810 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.250+0000 7f034affd640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f0340011940 tx=0x7f0340011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.252+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340019070 con 0x7f034c150810 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.252+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340004240 con 0x7f034c150810 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.252+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c071720 con 0x7f034c150810 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.252+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034000f7c0 con 0x7f034c150810 2024-03-06T07:26:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.252+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c072430 con 0x7f034c150810 2024-03-06T07:26:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.253+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c069430 con 0x7f034c150810 2024-03-06T07:26:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.254+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f034001c070 con 0x7f034c150810 2024-03-06T07:26:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.254+0000 7f0333fff640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.254+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f034008a9c0 con 0x7f034c150810 2024-03-06T07:26:09.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.254+0000 7f034b7fe640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.255+0000 7f034b7fe640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 0x7f033806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f033c004840 tx=0x7f033c007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.258+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f034008ce20 con 0x7f034c150810 2024-03-06T07:26:09.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.475+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f034c074490 con 0x7f034c150810 2024-03-06T07:26:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.476+0000 7f0333fff640 1 -- 172.21.15.17:0/1274532985 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+6266 (secure 0 0 0) 0x7f0340058da0 con 0x7f034c150810 2024-03-06T07:26:09.478 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:09.478 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":141,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8200,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":8203,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:25:56.622881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8781":{"gid":8781,"name":"c","rank":0,"incarnation":134,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2429596225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":2429596225},{"type":"v1","addr":"172.21.15.17:6835","nonce":2429596225}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8781,"qdb_cluster":[8781]},"id":11},{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:06.596155+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8778":{"gid":8778,"name":"a","rank":0,"incarnation":137,"state":"up:active","state_seq":5,"addr":"172.21.15.17:6834/81289984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":81289984},{"type":"v1","addr":"172.21.15.17:6834","nonce":81289984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8778,"qdb_cluster":[8778]},"id":12}]} 2024-03-06T07:26:09.478 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 141 2024-03-06T07:26:09.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 msgr2=0x7f033806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 0x7f033806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f033c004840 tx=0x7f033c007970 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 msgr2=0x7f034c150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f0340011940 tx=0x7f0340011d40 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 shutdown_connections 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f033806b4d0 0x7f033806d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f034c1eaa30 0x7f034c1eaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f034c150810 0x7f034c150c90 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 --2- 172.21.15.17:0/1274532985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f034c1dcef0 0x7f034c1dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 >> 172.21.15.17:0/1274532985 conn(0x7f034c0c4040 msgr2=0x7f034c14e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 shutdown_connections 2024-03-06T07:26:09.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.479+0000 7f0350ba0640 1 -- 172.21.15.17:0/1274532985 wait complete. 2024-03-06T07:26:09.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:09.484 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:09.494 INFO:tasks.cephfs.test_failover:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 141, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:54.526748+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 136, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8781": { "addr": "172.21.15.17:6835/2429596225", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 2429596225, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 2429596225, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8781, "incarnation": 134, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 3 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:25:56.622881+0000", "qdb_cluster": [ 8781 ], "qdb_leader": 8781, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8781 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:03.654281+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 141, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8778": { "addr": "172.21.15.17:6834/81289984", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 81289984, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 81289984, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8778, "incarnation": 137, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:06.596155+0000", "qdb_cluster": [ 8778 ], "qdb_leader": 8778, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8778 } } } ], "standbys": [ { "addr": "172.21.15.163:6836/91031835", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 91031835, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 91031835, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 133, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8200, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.163:6837/2502302602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2502302602, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2502302602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 133, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8203, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:09.495 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': -1, 'name': 'a'}, {'join_fscid': -1, 'name': 'b'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': -1, 'name': 'd'}] 2024-03-06T07:26:09.495 INFO:tasks.cephfs.test_failover:current = [{'gid': 8778, 'name': 'a', 'rank': 0, 'incarnation': 137, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.17:6834/81289984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 81289984}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 81289984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8203, 'name': 'b', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 133}, {'gid': 8781, 'name': 'c', 'rank': 0, 'incarnation': 134, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2429596225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 2429596225}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2429596225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 8200, 'name': 'd', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.163:6836/91031835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 91031835}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 91031835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 133}] 2024-03-06T07:26:09.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.d mds_join_fs cephfs2 2024-03-06T07:26:09.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.677+0000 7fc39c38c640 1 Processor -- start 2024-03-06T07:26:09.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.677+0000 7fc39c38c640 1 -- start start 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39c38c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3940c3150 0x7fc3940c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39c38c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc3940c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39c38c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc394072440 con 0x7fc39414f650 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39c38c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940c2b10 con 0x7fc3940c3150 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39c38c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc394152140 con 0x7fc394152c20 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3940c3150 0x7fc3940c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3940c3150 0x7fc3940c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39768/0 (socket says 172.21.15.17:39768) 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 -- 172.21.15.17:0/1641656351 learned_addr learned my addr 172.21.15.17:0/1641656351 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc39a929640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc3940c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 -- 172.21.15.17:0/1641656351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 msgr2=0x7fc3940c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:09.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc3940c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 -- 172.21.15.17:0/1641656351 >> v1:172.21.15.163:6789/0 conn(0x7fc39414f650 legacy=0x7fc394071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.678+0000 7fc399927640 1 -- 172.21.15.17:0/1641656351 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394152420 con 0x7fc3940c3150 2024-03-06T07:26:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.679+0000 7fc399927640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3940c3150 0x7fc3940c5690 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc384009870 tx=0x7fc38402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb1ca3905e167a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.679+0000 7fc399126640 1 -- 172.21.15.17:0/1641656351 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384034070 con 0x7fc3940c3150 2024-03-06T07:26:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.679+0000 7fc399126640 1 -- 172.21.15.17:0/1641656351 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384030840 con 0x7fc3940c3150 2024-03-06T07:26:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.679+0000 7fc399126640 1 -- 172.21.15.17:0/1641656351 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384030c10 con 0x7fc3940c3150 2024-03-06T07:26:09.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 -- 172.21.15.17:0/1641656351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3940c3150 msgr2=0x7fc3940c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3940c3150 0x7fc3940c5690 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fc384009870 tx=0x7fc38402b980 comp rx=0 tx=0).stop 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 -- 172.21.15.17:0/1641656351 shutdown_connections 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc3940c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 --2- 172.21.15.17:0/1641656351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3940c3150 0x7fc3940c5690 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 -- 172.21.15.17:0/1641656351 >> 172.21.15.17:0/1641656351 conn(0x7fc3940c4290 msgr2=0x7fc3940c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 -- 172.21.15.17:0/1641656351 shutdown_connections 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.680+0000 7fc39c38c640 1 -- 172.21.15.17:0/1641656351 wait complete. 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 Processor -- start 2024-03-06T07:26:09.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 -- start start 2024-03-06T07:26:09.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3940c3150 0x7fc39406b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc39406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394072110 con 0x7fc39414f650 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394071f90 con 0x7fc3940c3150 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.681+0000 7fc39c38c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940c3f40 con 0x7fc394152c20 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39778/0 (socket says 172.21.15.17:39778) 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 -- 172.21.15.17:0/3788293427 learned_addr learned my addr 172.21.15.17:0/3788293427 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc39a929640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc39406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc39a128640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3940c3150 0x7fc39406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 -- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 msgr2=0x7fc39406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc39406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 -- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3940c3150 msgr2=0x7fc39406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3940c3150 0x7fc39406b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 -- 172.21.15.17:0/3788293427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394145660 con 0x7fc39414f650 2024-03-06T07:26:09.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc399927640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc384000c00 tx=0x7fc384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384048070 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384030640 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc394145900 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384034040 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.682+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394151ce0 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.683+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc39404f330 con 0x7fc39414f650 2024-03-06T07:26:09.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.684+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc38404b070 con 0x7fc39414f650 2024-03-06T07:26:09.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.684+0000 7fc38affd640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 0x7fc37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.684+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fc3840ac570 con 0x7fc39414f650 2024-03-06T07:26:09.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.685+0000 7fc39a128640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 0x7fc37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:09.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.685+0000 7fc39a128640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 0x7fc37806d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc390001640 tx=0x7fc390009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:09.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.689+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc38407af10 con 0x7fc39414f650 2024-03-06T07:26:09.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.872+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7fc394068460 con 0x7fc39414f650 2024-03-06T07:26:09.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.875+0000 7fc38affd640 1 -- 172.21.15.17:0/3788293427 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v52)=0 v52) v1 ==== 117+0+0 (secure 0 0 0) 0x7fc384030810 con 0x7fc39414f650 2024-03-06T07:26:09.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 msgr2=0x7fc37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 0x7fc37806d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc390001640 tx=0x7fc390009040 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 msgr2=0x7fc39414c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fc384000c00 tx=0x7fc384004730 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 shutdown_connections 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc37806b4d0 0x7fc37806d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc394152c20 0x7fc39406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc39414f650 0x7fc39414c1b0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 --2- 172.21.15.17:0/3788293427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3940c3150 0x7fc39406b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 >> 172.21.15.17:0/3788293427 conn(0x7fc3940c4290 msgr2=0x7fc394155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 shutdown_connections 2024-03-06T07:26:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.878+0000 7fc39c38c640 1 -- 172.21.15.17:0/3788293427 wait complete. 2024-03-06T07:26:09.890 INFO:tasks.cephfs.test_failover:assigning affinity to cephfs2 for active mds (mds.a) 2024-03-06T07:26:09.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds.a mds_join_fs cephfs2 2024-03-06T07:26:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.996+0000 7f6c3ab93640 1 Processor -- start 2024-03-06T07:26:09.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.996+0000 7f6c3ab93640 1 -- start start 2024-03-06T07:26:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c3ab93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 0x7f6c340c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c3ab93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c34152c20 0x7f6c340c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c3ab93640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c34072440 con 0x7f6c3414f650 2024-03-06T07:26:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c3ab93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c340c2b10 con 0x7f6c34152c20 2024-03-06T07:26:09.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c3ab93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34152140 con 0x7f6c340c3150 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c34152c20 0x7f6c340c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6c34152c20 0x7f6c340c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39782/0 (socket says 172.21.15.17:39782) 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 -- 172.21.15.17:0/4254140138 learned_addr learned my addr 172.21.15.17:0/4254140138 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c33fff640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 0x7f6c340c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 -- 172.21.15.17:0/4254140138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 msgr2=0x7f6c340c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 0x7f6c340c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 -- 172.21.15.17:0/4254140138 >> v1:172.21.15.163:6789/0 conn(0x7f6c3414f650 legacy=0x7f6c34071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c39130640 1 -- 172.21.15.17:0/4254140138 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34152420 con 0x7f6c34152c20 2024-03-06T07:26:10.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.997+0000 7f6c33fff640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 0x7f6c340c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.998+0000 7f6c39130640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c34152c20 0x7f6c340c5bd0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d150 tx=0x7f6c2c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1726f465a052ef3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c337fe640 1 -- 172.21.15.17:0/4254140138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c02ea80 con 0x7f6c34152c20 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c337fe640 1 -- 172.21.15.17:0/4254140138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c02ec20 con 0x7f6c34152c20 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c337fe640 1 -- 172.21.15.17:0/4254140138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c009600 con 0x7f6c34152c20 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 -- 172.21.15.17:0/4254140138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c34152c20 msgr2=0x7f6c340c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c34152c20 0x7f6c340c5bd0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00d150 tx=0x7f6c2c02e0e0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 -- 172.21.15.17:0/4254140138 shutdown_connections 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c34152c20 0x7f6c340c5bd0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/4254140138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c340c3150 0x7f6c340c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 -- 172.21.15.17:0/4254140138 >> 172.21.15.17:0/4254140138 conn(0x7f6c340c4290 msgr2=0x7f6c340c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 -- 172.21.15.17:0/4254140138 shutdown_connections 2024-03-06T07:26:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:09.999+0000 7f6c3ab93640 1 -- 172.21.15.17:0/4254140138 wait complete. 2024-03-06T07:26:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.000+0000 7f6c3ab93640 1 Processor -- start 2024-03-06T07:26:10.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 -- start start 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c3414f650 0x7f6c341ead20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 0x7f6c341eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34072110 con 0x7f6c340c3150 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34071f90 con 0x7f6c34152c20 2024-03-06T07:26:10.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3ab93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340c3f40 con 0x7f6c3414f650 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3892f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.001+0000 7f6c3892f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39796/0 (socket says 172.21.15.17:39796) 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 -- 172.21.15.17:0/1270173323 learned_addr learned my addr 172.21.15.17:0/1270173323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c33fff640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c3414f650 0x7f6c341ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 -- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c3414f650 msgr2=0x7f6c341ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c3414f650 0x7f6c341ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 -- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 msgr2=0x7f6c341eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c39130640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 0x7f6c341eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 0x7f6c341eb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 -- 172.21.15.17:0/1270173323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c341eea80 con 0x7f6c340c3150 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c39130640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 0x7f6c341eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:10.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3892f640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c410 tx=0x7f6c2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28017070 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c280035f0 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c341eed60 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280039e0 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.002+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c340c1370 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.003+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c341de200 con 0x7f6c340c3150 2024-03-06T07:26:10.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.004+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c2801a070 con 0x7f6c340c3150 2024-03-06T07:26:10.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.004+0000 7f6c317fa640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.004+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f6c28089b80 con 0x7f6c340c3150 2024-03-06T07:26:10.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.004+0000 7f6c33fff640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.005+0000 7f6c33fff640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 0x7f6c1806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6c20008850 tx=0x7f6c20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.008+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c28058510 con 0x7f6c340c3150 2024-03-06T07:26:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.216+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_join_fs}] v 0) v1 -- 0x7f6c341ddce0 con 0x7f6c340c3150 2024-03-06T07:26:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.220+0000 7f6c317fa640 1 -- 172.21.15.17:0/1270173323 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_join_fs}]=0 v53)=0 v53) v1 ==== 117+0+0 (secure 0 0 0) 0x7f6c28057e10 con 0x7f6c340c3150 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 msgr2=0x7f6c1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 0x7f6c1806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6c20008850 tx=0x7f6c20008040 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 msgr2=0x7f6c340c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c410 tx=0x7f6c2800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 shutdown_connections 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6c1806b4d0 0x7f6c1806d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c34152c20 0x7f6c341eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c3414f650 0x7f6c341ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 --2- 172.21.15.17:0/1270173323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c340c3150 0x7f6c340c0e30 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 >> 172.21.15.17:0/1270173323 conn(0x7f6c340c4290 msgr2=0x7f6c340c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 shutdown_connections 2024-03-06T07:26:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.222+0000 7f6c3ab93640 1 -- 172.21.15.17:0/1270173323 wait complete. 2024-03-06T07:26:10.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 11:0 2024-03-06T07:26:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.351+0000 7f88c425c640 1 Processor -- start 2024-03-06T07:26:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.351+0000 7f88c425c640 1 -- start start 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c425c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88bc0c6dd0 0x7f88bc160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c425c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc155130 con 0x7f88bc0c69f0 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c425c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc0c7200 con 0x7f88bc0c8950 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c425c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc155440 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c27f9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88bc0c6dd0 0x7f88bc160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c1ff8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f88bc0c69f0 0x7f88bc0acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57338/0 (socket says 172.21.15.17:57338) 2024-03-06T07:26:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c1ff8640 1 -- 172.21.15.17:0/2454557794 learned_addr learned my addr 172.21.15.17:0/2454557794 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c27f9640 1 -- 172.21.15.17:0/2454557794 >> v1:172.21.15.163:6789/0 conn(0x7f88bc0c8950 legacy=0x7f88bc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c27f9640 1 -- 172.21.15.17:0/2454557794 >> v1:172.21.15.17:6789/0 conn(0x7f88bc0c69f0 legacy=0x7f88bc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c27f9640 1 -- 172.21.15.17:0/2454557794 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0ad480 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.352+0000 7f88c0ff6640 1 -- 172.21.15.17:0/2454557794 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620107731 0 0) 0x7f88ac001490 con 0x7f88bc0c69f0 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.353+0000 7f88c27f9640 1 --2- 172.21.15.17:0/2454557794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc160ac0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f88bc1555c0 tx=0x7f88b402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e05099661631fd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.353+0000 7f88c0ff6640 1 -- 172.21.15.17:0/2454557794 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40315a0 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.353+0000 7f88c0ff6640 1 -- 172.21.15.17:0/2454557794 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b4037070 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.353+0000 7f88c0ff6640 1 -- 172.21.15.17:0/2454557794 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4031960 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 -- 172.21.15.17:0/2454557794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 msgr2=0x7f88bc160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 --2- 172.21.15.17:0/2454557794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc160ac0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f88bc1555c0 tx=0x7f88b402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 -- 172.21.15.17:0/2454557794 shutdown_connections 2024-03-06T07:26:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 --2- 172.21.15.17:0/2454557794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc160ac0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 -- 172.21.15.17:0/2454557794 >> 172.21.15.17:0/2454557794 conn(0x7f88bc0cae70 msgr2=0x7f88bc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 -- 172.21.15.17:0/2454557794 shutdown_connections 2024-03-06T07:26:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.354+0000 7f88c425c640 1 -- 172.21.15.17:0/2454557794 wait complete. 2024-03-06T07:26:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.356+0000 7f88c425c640 1 Processor -- start 2024-03-06T07:26:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.356+0000 7f88c425c640 1 -- start start 2024-03-06T07:26:10.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc0c69f0 0x7f88bc0c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c8950 0x7f88bc1edd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c6310 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c6030 con 0x7f88bc0c69f0 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c425c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc155fe0 con 0x7f88bc0c8950 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39812/0 (socket says 172.21.15.17:39812) 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 -- 172.21.15.17:0/3253188692 learned_addr learned my addr 172.21.15.17:0/3253188692 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 -- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c8950 msgr2=0x7f88bc1edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c8950 0x7f88bc1edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c1ff8640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc0c69f0 0x7f88bc0c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 -- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc0c69f0 msgr2=0x7f88bc0c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc0c69f0 0x7f88bc0c27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 -- 172.21.15.17:0/3253188692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc1eb360 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88c17f7640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c430 tx=0x7f88b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.357+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0011790 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.358+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b0017070 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.358+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc1eb640 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.358+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0011bb0 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.358+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc159240 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.358+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc04f330 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.359+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88b001c460 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.360+0000 7f88ba7fc640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 0x7f88a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.360+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f88b008b030 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.360+0000 7f88c1ff8640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 0x7f88a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.361+0000 7f88c1ff8640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 0x7f88a806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0012e0 tx=0x7f88ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.364+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88b0059ac0 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.711+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "11:0"} v 0) v1 -- 0x7f88bc14fd60 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.713 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:26:10.712+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.713+0000 7f88ba7fc640 1 -- 172.21.15.17:0/3253188692 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "11:0"}]=0 failed mds gid 8781 v142) v1 ==== 98+0+0 (secure 0 0 0) 0x7f88b00593c0 con 0x7f88bc0c6dd0 2024-03-06T07:26:10.715 INFO:teuthology.orchestra.run.smithi017.stderr:failed mds gid 8781 2024-03-06T07:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 msgr2=0x7f88a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 0x7f88a806d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f88ac0012e0 tx=0x7f88ac009040 comp rx=0 tx=0).stop 2024-03-06T07:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 msgr2=0x7f88bc0c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f88b000c430 tx=0x7f88b000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 shutdown_connections 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b4d0 0x7f88a806d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c8950 0x7f88bc1edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c6dd0 0x7f88bc0c4330 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 --2- 172.21.15.17:0/3253188692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc0c69f0 0x7f88bc0c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 >> 172.21.15.17:0/3253188692 conn(0x7f88bc0cae70 msgr2=0x7f88bc150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 shutdown_connections 2024-03-06T07:26:10.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.717+0000 7f88c425c640 1 -- 172.21.15.17:0/3253188692 wait complete. 2024-03-06T07:26:10.720 INFO:tasks.ceph.mds.c.smithi017.stderr: -335> 2024-03-06T07:26:00.444+0000 7f8095cb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:10.726 INFO:tasks.ceph.mds.a.smithi017.stderr: -215> 2024-03-06T07:26:05.445+0000 7eff03eaa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:10.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:10.750 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:26:10.757 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:26:10.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.838+0000 7f4b0eaaf640 1 Processor -- start 2024-03-06T07:26:10.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.838+0000 7f4b0eaaf640 1 -- start start 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0eaaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0eaaf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4b080c3d60 0x7f4b080c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0eaaf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b08156f90 con 0x7f4b08156a00 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0eaaf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080c38d0 con 0x7f4b080c3d60 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0eaaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08157270 con 0x7f4b08072720 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0d04c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4b080c3d60 0x7f4b080c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43714/0 (socket says 172.21.15.17:43714) 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 -- 172.21.15.17:0/2229366314 learned_addr learned my addr 172.21.15.17:0/2229366314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b07fff640 1 --1- 172.21.15.17:0/2229366314 >> v1:172.21.15.163:6789/0 conn(0x7f4b08156a00 0x7f4b08150290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:41334/0 (socket says 172.21.15.17:41334) 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b077fe640 1 -- 172.21.15.17:0/2229366314 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765581758 0 0) 0x7f4b08156f90 con 0x7f4b08156a00 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 -- 172.21.15.17:0/2229366314 >> v2:172.21.15.163:3300/0 conn(0x7f4b080c3d60 msgr2=0x7f4b080c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 --2- 172.21.15.17:0/2229366314 >> v2:172.21.15.163:3300/0 conn(0x7f4b080c3d60 0x7f4b080c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 -- 172.21.15.17:0/2229366314 >> v1:172.21.15.163:6789/0 conn(0x7f4b08156a00 legacy=0x7f4b08150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.839+0000 7f4b0c84b640 1 -- 172.21.15.17:0/2229366314 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080c3280 con 0x7f4b08072720 2024-03-06T07:26:10.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.840+0000 7f4b0c84b640 1 --2- 172.21.15.17:0/2229366314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098a0 tx=0x7f4afc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df96943faf671056 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.840+0000 7f4b077fe640 1 -- 172.21.15.17:0/2229366314 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc035020 con 0x7f4b08072720 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.840+0000 7f4b077fe640 1 -- 172.21.15.17:0/2229366314 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc031640 con 0x7f4b08072720 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.840+0000 7f4b077fe640 1 -- 172.21.15.17:0/2229366314 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc02d040 con 0x7f4b08072720 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/2229366314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 msgr2=0x7f4b081564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/2229366314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0098a0 tx=0x7f4afc02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/2229366314 shutdown_connections 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/2229366314 >> v2:172.21.15.163:3300/0 conn(0x7f4b080c3d60 0x7f4b080c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/2229366314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b08072720 0x7f4b081564c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/2229366314 >> 172.21.15.17:0/2229366314 conn(0x7f4b080aa630 msgr2=0x7f4b080aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/2229366314 shutdown_connections 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.841+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/2229366314 wait complete. 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.842+0000 7f4b0eaaf640 1 Processor -- start 2024-03-06T07:26:10.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.842+0000 7f4b0eaaf640 1 -- start start 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b08072720 0x7f4b080ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b080c3d60 0x7f4b081ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081509a0 con 0x7f4b08156a00 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806f8f0 con 0x7f4b08072720 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08150b20 con 0x7f4b080c3d60 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36862/0 (socket says 172.21.15.17:36862) 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 -- 172.21.15.17:0/1388656958 learned_addr learned my addr 172.21.15.17:0/1388656958 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b07fff640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b080c3d60 0x7f4b081ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0c84b640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b08072720 0x7f4b080ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 -- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b080c3d60 msgr2=0x7f4b081ebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b080c3d60 0x7f4b081ebc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 -- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b08072720 msgr2=0x7f4b080ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b08072720 0x7f4b080ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 -- 172.21.15.17:0/1388656958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08153660 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0d04c640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000efc0 tx=0x7f4b0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00018020 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b00009b10 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b081ec540 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.843+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000146c0 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.844+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b081ec820 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.844+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0804f330 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.845+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b00005d10 con 0x7f4b08156a00 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.845+0000 7f4b057fa640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 0x7f4aec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:10.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.845+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f4b0008b0a0 con 0x7f4b08156a00 2024-03-06T07:26:10.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.845+0000 7f4b0c84b640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 0x7f4aec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:10.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.846+0000 7f4b0c84b640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 0x7f4aec06d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4afc036000 tx=0x7f4afc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:10.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:10.850+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b00022070 con 0x7f4b08156a00 2024-03-06T07:26:11.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.070+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b080aaab0 con 0x7f4b08156a00 2024-03-06T07:26:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.070+0000 7f4b057fa640 1 -- 172.21.15.17:0/1388656958 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4679 (secure 0 0 0) 0x7f4b00010040 con 0x7f4b08156a00 2024-03-06T07:26:11.072 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:11.072 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":143,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:10.717626+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":11},{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:10.717830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:replay","state_seq":6,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":12}]} 2024-03-06T07:26:11.073 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 143 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 msgr2=0x7f4aec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 0x7f4aec06d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4afc036000 tx=0x7f4afc008030 comp rx=0 tx=0).stop 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 msgr2=0x7f4b081556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000efc0 tx=0x7f4b0000c490 comp rx=0 tx=0).stop 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 shutdown_connections 2024-03-06T07:26:11.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4aec06b4d0 0x7f4aec06d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b08156a00 0x7f4b081556f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b080c3d60 0x7f4b081ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 --2- 172.21.15.17:0/1388656958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b08072720 0x7f4b080ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 >> 172.21.15.17:0/1388656958 conn(0x7f4b080aa630 msgr2=0x7f4b0814c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 shutdown_connections 2024-03-06T07:26:11.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:11.072+0000 7f4b0eaaf640 1 -- 172.21.15.17:0/1388656958 wait complete. 2024-03-06T07:26:11.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:54.526748+0000', 'modified': '2024-03-06T07:26:10.717626+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 155, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8203': {'gid': 8203, 'name': 'b', 'rank': 0, 'incarnation': 143, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:11.488 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8203:up:replay 2024-03-06T07:26:12.489 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:12.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.600+0000 7f5306881640 1 Processor -- start 2024-03-06T07:26:12.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 -- start start 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53001527b0 0x7f5300159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f53000c88b0 0x7f53000c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000c7c60 con 0x7f53000c7180 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5300159640 con 0x7f53000c88b0 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f5306881640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c8060 con 0x7f53001527b0 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f52ff7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f53000c7180 0x7f5300155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34984/0 (socket says 172.21.15.17:34984) 2024-03-06T07:26:12.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.601+0000 7f52ff7fe640 1 -- 172.21.15.17:0/1997154217 learned_addr learned my addr 172.21.15.17:0/1997154217 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f52fffff640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53001527b0 0x7f5300159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f5304e1e640 1 --2- 172.21.15.17:0/1997154217 >> v2:172.21.15.163:3300/0 conn(0x7f53000c88b0 0x7f53000c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468353779 0 0) 0x7f52f4001110 con 0x7f53000c7180 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e8003520 con 0x7f53000c7180 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 614644184 0 0) 0x7f52e8003520 con 0x7f53000c7180 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f5304e1e640 1 -- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53001527b0 msgr2=0x7f5300159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f5304e1e640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53001527b0 0x7f5300159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f5304e1e640 1 -- 172.21.15.17:0/1997154217 >> v1:172.21.15.17:6789/0 conn(0x7f53000c7180 legacy=0x7f5300155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.602+0000 7f5304e1e640 1 -- 172.21.15.17:0/1997154217 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000c76a0 con 0x7f53000c88b0 2024-03-06T07:26:12.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f5304e1e640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53000c88b0 0x7f53000c5330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68b33531f83ab23 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8039070 con 0x7f53000c88b0 2024-03-06T07:26:12.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f802c810 con 0x7f53000c88b0 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f52feffd640 1 -- 172.21.15.17:0/1997154217 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4001840 con 0x7f53000c88b0 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f5306881640 1 -- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53000c88b0 msgr2=0x7f53000c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.603+0000 7f5306881640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53000c88b0 0x7f53000c5330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009d90 tx=0x7f52f802bce0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 -- 172.21.15.17:0/1997154217 shutdown_connections 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53000c88b0 0x7f53000c5330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 --2- 172.21.15.17:0/1997154217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53001527b0 0x7f5300159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 -- 172.21.15.17:0/1997154217 >> 172.21.15.17:0/1997154217 conn(0x7f53000ac560 msgr2=0x7f53000ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 -- 172.21.15.17:0/1997154217 shutdown_connections 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.604+0000 7f5306881640 1 -- 172.21.15.17:0/1997154217 wait complete. 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.605+0000 7f5306881640 1 Processor -- start 2024-03-06T07:26:12.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.605+0000 7f5306881640 1 -- start start 2024-03-06T07:26:12.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53000c7180 0x7f5300143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53001527b0 0x7f5300157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300155600 con 0x7f53000c88b0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300155320 con 0x7f53001527b0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c8d60 con 0x7f53000c7180 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36876/0 (socket says 172.21.15.17:36876) 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 -- 172.21.15.17:0/162827308 learned_addr learned my addr 172.21.15.17:0/162827308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5304e1e640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53001527b0 0x7f5300157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52fffff640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53000c7180 0x7f5300143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 -- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53000c7180 msgr2=0x7f5300143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53000c7180 0x7f5300143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 -- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53001527b0 msgr2=0x7f5300157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53001527b0 0x7f5300157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 -- 172.21.15.17:0/162827308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000c1160 con 0x7f53000c88b0 2024-03-06T07:26:12.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52ff7fe640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c750 tx=0x7f52f400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4005ad0 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4005c70 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53000c1440 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.606+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40072b0 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.607+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5300158f40 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.608+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530004f330 con 0x7f53000c88b0 2024-03-06T07:26:12.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.608+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52f4007450 con 0x7f53000c88b0 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.608+0000 7f52fcff9640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.609+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f52f408a4a0 con 0x7f53000c88b0 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.609+0000 7f52fffff640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:12.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.609+0000 7f52fffff640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009aa0 tx=0x7f52f0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.613+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53000c7c60 con 0x7f53000c88b0 2024-03-06T07:26:12.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.835+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5300144660 con 0x7f53000c88b0 2024-03-06T07:26:12.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.836+0000 7f52fcff9640 1 -- 172.21.15.17:0/162827308 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+6262 (secure 0 0 0) 0x7f52f4058750 con 0x7f53000c88b0 2024-03-06T07:26:12.838 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:12.838 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":145,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:11.727000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":11},{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:11.726910+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":12}]} 2024-03-06T07:26:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 145 2024-03-06T07:26:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 msgr2=0x7f52ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009aa0 tx=0x7f52f0009190 comp rx=0 tx=0).stop 2024-03-06T07:26:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 msgr2=0x7f5300156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c750 tx=0x7f52f400cc10 comp rx=0 tx=0).stop 2024-03-06T07:26:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 shutdown_connections 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52ec06b4d0 0x7f52ec06d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53001527b0 0x7f5300157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53000c88b0 0x7f5300156fe0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 --2- 172.21.15.17:0/162827308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53000c7180 0x7f5300143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 >> 172.21.15.17:0/162827308 conn(0x7f53000ac560 msgr2=0x7f530014aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 shutdown_connections 2024-03-06T07:26:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:12.838+0000 7f5306881640 1 -- 172.21.15.17:0/162827308 wait complete. 2024-03-06T07:26:13.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 145, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:54.526748+0000', 'modified': '2024-03-06T07:26:11.727000+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 155, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8203': {'gid': 8203, 'name': 'b', 'rank': 0, 'incarnation': 143, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:13.253 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8203:up:rejoin 2024-03-06T07:26:14.254 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.366+0000 7fcc55084640 1 Processor -- start 2024-03-06T07:26:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.366+0000 7fcc55084640 1 -- start start 2024-03-06T07:26:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc55084640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc50158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc55084640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc500c7760 con 0x7fcc50159460 2024-03-06T07:26:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc55084640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc500c7a40 con 0x7fcc50152920 2024-03-06T07:26:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc55084640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501587b0 con 0x7fcc500c8a00 2024-03-06T07:26:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc4f59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcc50159460 0x7fcc500c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34994/0 (socket says 172.21.15.17:34994) 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc4f59e640 1 -- 172.21.15.17:0/2069712293 learned_addr learned my addr 172.21.15.17:0/2069712293 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc47fff640 1 --2- 172.21.15.17:0/2069712293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc50158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956366694 0 0) 0x7fcc500c7760 con 0x7fcc50159460 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc3c003520 con 0x7fcc50159460 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631266480 0 0) 0x7fcc500c7a40 con 0x7fcc50152920 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc47fff640 1 -- 172.21.15.17:0/2069712293 >> v1:172.21.15.163:6789/0 conn(0x7fcc50152920 legacy=0x7fcc50155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc47fff640 1 -- 172.21.15.17:0/2069712293 >> v1:172.21.15.17:6789/0 conn(0x7fcc50159460 legacy=0x7fcc500c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.367+0000 7fcc47fff640 1 -- 172.21.15.17:0/2069712293 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc500c7d20 con 0x7fcc500c8a00 2024-03-06T07:26:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.368+0000 7fcc47fff640 1 --2- 172.21.15.17:0/2069712293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc50158f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc380097d0 tx=0x7fcc3802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b59a8eb698f0e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.368+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38035020 con 0x7fcc500c8a00 2024-03-06T07:26:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.368+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38031800 con 0x7fcc500c8a00 2024-03-06T07:26:14.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.368+0000 7fcc4e59c640 1 -- 172.21.15.17:0/2069712293 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38031bd0 con 0x7fcc500c8a00 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 -- 172.21.15.17:0/2069712293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 msgr2=0x7fcc50158f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 --2- 172.21.15.17:0/2069712293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc50158f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc380097d0 tx=0x7fcc3802b8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 -- 172.21.15.17:0/2069712293 shutdown_connections 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 --2- 172.21.15.17:0/2069712293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc50158f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 -- 172.21.15.17:0/2069712293 >> 172.21.15.17:0/2069712293 conn(0x7fcc500ac560 msgr2=0x7fcc500ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 -- 172.21.15.17:0/2069712293 shutdown_connections 2024-03-06T07:26:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.369+0000 7fcc55084640 1 -- 172.21.15.17:0/2069712293 wait complete. 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.370+0000 7fcc55084640 1 Processor -- start 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.370+0000 7fcc55084640 1 -- start start 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc501578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc50152920 0x7fcc50152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500c5a40 con 0x7fcc50155f70 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5006d030 con 0x7fcc50152920 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc55084640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500c5bc0 con 0x7fcc500c8a00 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36886/0 (socket says 172.21.15.17:36886) 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 -- 172.21.15.17:0/1437109344 learned_addr learned my addr 172.21.15.17:0/1437109344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4ed9d640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc501578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc47fff640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc50152920 0x7fcc50152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 -- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 msgr2=0x7fcc501578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc501578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 -- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc50152920 msgr2=0x7fcc50152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc50152920 0x7fcc50152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.371+0000 7fcc4f59e640 1 -- 172.21.15.17:0/1437109344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc500c27e0 con 0x7fcc50155f70 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc4f59e640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800ca70 tx=0x7fcc4800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48017070 con 0x7fcc50155f70 2024-03-06T07:26:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4800abd0 con 0x7fcc50155f70 2024-03-06T07:26:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc500c4590 con 0x7fcc50155f70 2024-03-06T07:26:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48012400 con 0x7fcc50155f70 2024-03-06T07:26:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.372+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc500c2490 con 0x7fcc50155f70 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.373+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5006cb00 con 0x7fcc50155f70 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.373+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc4801a070 con 0x7fcc50155f70 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.374+0000 7fcc477fe640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 0x7fcc3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.374+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fcc4808a400 con 0x7fcc50155f70 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.374+0000 7fcc4ed9d640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 0x7fcc3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:14.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.374+0000 7fcc4ed9d640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 0x7fcc3006d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcc500be3c0 tx=0x7fcc40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:14.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.378+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcc5006cb00 con 0x7fcc50155f70 2024-03-06T07:26:14.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.599+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc5004f330 con 0x7fcc50155f70 2024-03-06T07:26:14.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.600+0000 7fcc477fe640 1 -- 172.21.15.17:0/1437109344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7fcc48058690 con 0x7fcc50155f70 2024-03-06T07:26:14.601 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:14.602 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.601+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 msgr2=0x7fcc3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.601+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 0x7fcc3006d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcc500be3c0 tx=0x7fcc40009040 comp rx=0 tx=0).stop 2024-03-06T07:26:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.601+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 msgr2=0x7fcc501edff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.601+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800ca70 tx=0x7fcc4800cf30 comp rx=0 tx=0).stop 2024-03-06T07:26:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 shutdown_connections 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc3006b4d0 0x7fcc3006d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc50155f70 0x7fcc501edff0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc50152920 0x7fcc50152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 --2- 172.21.15.17:0/1437109344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc500c8a00 0x7fcc501578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 >> 172.21.15.17:0/1437109344 conn(0x7fcc500ac560 msgr2=0x7fcc500c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 shutdown_connections 2024-03-06T07:26:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:14.602+0000 7fcc55084640 1 -- 172.21.15.17:0/1437109344 wait complete. 2024-03-06T07:26:15.016 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:25:54.526748+0000', 'modified': '2024-03-06T07:26:13.724498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 155, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8203}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8203': {'gid': 8203, 'name': 'b', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8203, 'qdb_cluster': [8203]} 2024-03-06T07:26:15.016 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:26:15.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-06T07:26:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.118+0000 7f5c2380f640 1 Processor -- start 2024-03-06T07:26:15.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.118+0000 7f5c2380f640 1 -- start start 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c2380f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 0x7f5c1c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c2380f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c2380f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c0c5140 con 0x7f5c1c1526c0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c2380f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c3790 con 0x7f5c1c0c2b20 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c2380f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c155bb0 con 0x7f5c1c1587e0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c215ab640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5c1c1526c0 0x7f5c1c0c8930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35006/0 (socket says 172.21.15.17:35006) 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c215ab640 1 -- 172.21.15.17:0/1684511459 learned_addr learned my addr 172.21.15.17:0/1684511459 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c21dac640 1 --2- 172.21.15.17:0/1684511459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c20daa640 1 --2- 172.21.15.17:0/1684511459 >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 0x7f5c1c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519853965 0 0) 0x7f5c1c0c5140 con 0x7f5c1c1526c0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c08003520 con 0x7f5c1c1526c0 2024-03-06T07:26:15.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163789057 0 0) 0x7f5c08003520 con 0x7f5c1c1526c0 2024-03-06T07:26:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c21dac640 1 -- 172.21.15.17:0/1684511459 >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 msgr2=0x7f5c1c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c21dac640 1 --2- 172.21.15.17:0/1684511459 >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 0x7f5c1c0c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c21dac640 1 -- 172.21.15.17:0/1684511459 >> v1:172.21.15.17:6789/0 conn(0x7f5c1c1526c0 legacy=0x7f5c1c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c21dac640 1 -- 172.21.15.17:0/1684511459 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c155e90 con 0x7f5c1c1587e0 2024-03-06T07:26:15.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.119+0000 7f5c20daa640 1 --2- 172.21.15.17:0/1684511459 >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 0x7f5c1c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.120+0000 7f5c21dac640 1 --2- 172.21.15.17:0/1684511459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c158c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009d90 tx=0x7f5c1402bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d096349fa85c3b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.120+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14039070 con 0x7f5c1c1587e0 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.120+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1402c810 con 0x7f5c1c1587e0 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.120+0000 7f5c13fff640 1 -- 172.21.15.17:0/1684511459 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18001670 con 0x7f5c1c1587e0 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 -- 172.21.15.17:0/1684511459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 msgr2=0x7f5c1c158c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 --2- 172.21.15.17:0/1684511459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c158c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009d90 tx=0x7f5c1402bce0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 -- 172.21.15.17:0/1684511459 shutdown_connections 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 --2- 172.21.15.17:0/1684511459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c158c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 --2- 172.21.15.17:0/1684511459 >> v2:172.21.15.163:3300/0 conn(0x7f5c1c0c2b20 0x7f5c1c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 -- 172.21.15.17:0/1684511459 >> 172.21.15.17:0/1684511459 conn(0x7f5c1c0aa810 msgr2=0x7f5c1c0aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 -- 172.21.15.17:0/1684511459 shutdown_connections 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.121+0000 7f5c2380f640 1 -- 172.21.15.17:0/1684511459 wait complete. 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 Processor -- start 2024-03-06T07:26:15.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 -- start start 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c1c0c2b20 0x7f5c1c1ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c9040 con 0x7f5c1c1526c0 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c152aa0 con 0x7f5c1c0c2b20 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.122+0000 7f5c2380f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c0c91c0 con 0x7f5c1c1587e0 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36888/0 (socket says 172.21.15.17:36888) 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 -- 172.21.15.17:0/3025941906 learned_addr learned my addr 172.21.15.17:0/3025941906 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c21dac640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c1c0c2b20 0x7f5c1c1ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 -- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 msgr2=0x7f5c1c153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 -- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c1c0c2b20 msgr2=0x7f5c1c1ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c1c0c2b20 0x7f5c1c1ee2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 -- 172.21.15.17:0/3025941906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c154e20 con 0x7f5c1c1526c0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c20daa640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ebe0 tx=0x7f5c0c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c11ffb640 1 -- 172.21.15.17:0/3025941906 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c017070 con 0x7f5c1c1526c0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c11ffb640 1 -- 172.21.15.17:0/3025941906 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c00cc30 con 0x7f5c1c1526c0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c155100 con 0x7f5c1c1526c0 2024-03-06T07:26:15.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.123+0000 7f5c11ffb640 1 -- 172.21.15.17:0/3025941906 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c013470 con 0x7f5c1c1526c0 2024-03-06T07:26:15.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.124+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c14f6e0 con 0x7f5c1c1526c0 2024-03-06T07:26:15.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.125+0000 7f5c11ffb640 1 -- 172.21.15.17:0/3025941906 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c0c01a070 con 0x7f5c1c1526c0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.125+0000 7f5c11ffb640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.125+0000 7f5c11ffb640 1 -- 172.21.15.17:0/3025941906 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f5c0c08a610 con 0x7f5c1c1526c0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.125+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.126+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 0x7f5c0006d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5c180013c0 tx=0x7f5c18001350 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.126+0000 7f5c263f0740 1 Processor -- start 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.126+0000 7f5c263f0740 1 -- start start 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459f86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f874b0 0x55c459f83680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459eed6b0 con 0x55c459f88f50 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459e96a60 con 0x55c459f89330 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c459ee9750 con 0x55c459f874b0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36894/0 (socket says 172.21.15.17:36894) 2024-03-06T07:26:15.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 -- 172.21.15.17:0/3293430719 learned_addr learned my addr 172.21.15.17:0/3293430719 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459f86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c21dac640 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f874b0 0x55c459f83680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 -- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f874b0 msgr2=0x55c459f83680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f874b0 0x55c459f83680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 -- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 msgr2=0x55c459f86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459f86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c215ab640 1 -- 172.21.15.17:0/3293430719 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c459eb4f30 con 0x55c459f88f50 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.127+0000 7f5c20daa640 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459f86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c215ab640 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005d70 tx=0x7f5c18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922cfa4ec5192952 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c06ffd640 1 -- 172.21.15.17:0/3293430719 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18026070 con 0x55c459f88f50 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c06ffd640 1 -- 172.21.15.17:0/3293430719 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180065d0 con 0x55c459f88f50 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c06ffd640 1 -- 172.21.15.17:0/3293430719 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18005e60 con 0x55c459f88f50 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 -- 172.21.15.17:0/3293430719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 msgr2=0x55c459f86a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005d70 tx=0x7f5c18012000 comp rx=0 tx=0).stop 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 -- 172.21.15.17:0/3293430719 shutdown_connections 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f874b0 0x55c459f83680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459f86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 --2- 172.21.15.17:0/3293430719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459f86a30 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 -- 172.21.15.17:0/3293430719 >> 172.21.15.17:0/3293430719 conn(0x55c459f857d0 msgr2=0x55c459faa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 -- 172.21.15.17:0/3293430719 shutdown_connections 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.128+0000 7f5c263f0740 1 -- 172.21.15.17:0/3293430719 wait complete. 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.129+0000 7f5c263f0740 1 -- start start 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.129+0000 7f5c263f0740 1 Processor -- start 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459facd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459fadd30 0x55c459fae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459fab230 con 0x55c459f88f50 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459ea9ea0 con 0x55c459f89330 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c459faac70 con 0x55c459fadd30 2024-03-06T07:26:15.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c263f0740 10 client.0 _ll_register_callbacks cb 0x55c459ea9870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36904/0 (socket says 172.21.15.17:36904) 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 -- 172.21.15.17:0/2626233158 learned_addr learned my addr 172.21.15.17:0/2626233158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c21dac640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459fadd30 0x55c459fae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459facd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 -- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459fadd30 msgr2=0x55c459fae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459fadd30 0x55c459fae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 -- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 msgr2=0x55c459facd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459facd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 -- 172.21.15.17:0/2626233158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c459eaa270 con 0x55c459f88f50 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5c215ab640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x55c459fc85d0 tx=0x7f5c18001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e7c2ab399e9ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.130+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18031060 con 0x55c459f88f50 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f5c18021bb0 con 0x55c459f88f50 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c180065d0 con 0x55c459f88f50 2024-03-06T07:26:15.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c18037050 con 0x55c459f88f50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 10 client.9117 fetch_fsmap learned FSMap version 146 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55c459c022d0 con 0x55c459f88f50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c180065d0 con 0x55c459f88f50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 146) v1 ==== 3929+0+0 (secure 0 0 0) 0x7f5c1803a020 con 0x55c459f88f50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 10 client.9117 fetch_fsmap finished waiting for FSMap version 146 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 10 client.9117 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 20 client.9117 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fabc50 0x55c459fac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 4 client.9117 mds_command: new command op to 8203 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 --> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c459efabf0 con 0x55c459fabc50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.131+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fabc50 0x55c459fac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.132+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fabc50 0x55c459fac030 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c4b1c449d2dd99f server_cookie=8372edf1cefdaeaf in_seq=0 out_seq=0 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.132+0000 7f5bfbfff640 10 client.9117 ms_handle_connect on v2:172.21.15.163:6835/2502302602 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.132+0000 7f5bfbfff640 1 -- 172.21.15.17:0/2626233158 <== mds.0 v2:172.21.15.163:6835/2502302602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c459efabf0 con 0x55c459fabc50 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.132+0000 7f5bfbfff640 10 client.9117 handle_command_reply: tid=0 2024-03-06T07:26:15.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 client.9117 shutdown 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 msgr2=0x55c459fafaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x55c459fc85d0 tx=0x7f5c18001e00 comp rx=0 tx=0).stop 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 shutdown_connections 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fabc50 0x55c459fac030 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459fadd30 0x55c459fae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459f89330 0x55c459facd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5bfbfff640 0 client.9117 ms_handle_reset on v2:172.21.15.163:6835/2502302602 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2626233158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f88f50 0x55c459fafaf0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 >> 172.21.15.17:0/2626233158 conn(0x55c459f874b0 msgr2=0x55c459f82510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 shutdown_connections 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 1 -- 172.21.15.17:0/2626233158 wait complete. 2024-03-06T07:26:15.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.133+0000 7f5c263f0740 20 client.9117 trim_cache size 0 max 16384 2024-03-06T07:26:15.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 Processor -- start 2024-03-06T07:26:15.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 -- start start 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459fc2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459f839c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f9d10 con 0x55c459f89330 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1facc0 con 0x55c459fabc50 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c06f2b0 con 0x55c459f88f50 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36918/0 (socket says 172.21.15.17:36918) 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c20daa640 1 -- 172.21.15.17:0/2524989308 learned_addr learned my addr 172.21.15.17:0/2524989308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c215ab640 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459fc2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.150+0000 7f5c21dac640 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459f839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 -- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 msgr2=0x55c459fc2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459fc2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 -- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 msgr2=0x55c459f839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459f839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 -- 172.21.15.17:0/2524989308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c459ccc6b0 con 0x55c459f89330 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c20daa640 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00f010 tx=0x7f5c0c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaecddb99df1c73b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c067fc640 1 -- 172.21.15.17:0/2524989308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c021040 con 0x55c459f89330 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c067fc640 1 -- 172.21.15.17:0/2524989308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c054f10 con 0x55c459f89330 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c067fc640 1 -- 172.21.15.17:0/2524989308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c057cf0 con 0x55c459f89330 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 -- 172.21.15.17:0/2524989308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 msgr2=0x55c459fd2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00f010 tx=0x7f5c0c01e040 comp rx=0 tx=0).stop 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 -- 172.21.15.17:0/2524989308 shutdown_connections 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459f839c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fd2a70 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 --2- 172.21.15.17:0/2524989308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459fc2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 -- 172.21.15.17:0/2524989308 >> 172.21.15.17:0/2524989308 conn(0x55c459f874b0 msgr2=0x55c459fdeef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 -- 172.21.15.17:0/2524989308 shutdown_connections 2024-03-06T07:26:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.151+0000 7f5c263f0740 1 -- 172.21.15.17:0/2524989308 wait complete. 2024-03-06T07:26:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.152+0000 7f5c263f0740 1 -- start start 2024-03-06T07:26:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.152+0000 7f5c263f0740 1 Processor -- start 2024-03-06T07:26:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459f86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459fe9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c459fabad0 con 0x55c459f89330 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c1f8f10 con 0x55c459fabc50 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c459fd59c0 con 0x55c459f88f50 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 10 client.0 _ll_register_callbacks cb 0x7f5c080036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36928/0 (socket says 172.21.15.17:36928) 2024-03-06T07:26:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 -- 172.21.15.17:0/212908714 learned_addr learned my addr 172.21.15.17:0/212908714 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c215ab640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459f86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c21dac640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459fe9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 -- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 msgr2=0x55c459f86770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459f86770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 -- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 msgr2=0x55c459fe9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459fe9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 -- 172.21.15.17:0/212908714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45a0365b0 con 0x55c459f89330 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c20daa640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009160 tx=0x7f5c0c08a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65288af0fcd32cac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c021040 con 0x55c459f89330 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f5c0c057ef0 con 0x55c459f89330 2024-03-06T07:26:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.153+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c0c054f10 con 0x55c459f89330 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c0c0886e0 con 0x55c459f89330 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 10 client.9123 fetch_fsmap learned FSMap version 146 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55c459f86f10 con 0x55c459f89330 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c054f10 con 0x55c459f89330 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 146) v1 ==== 3929+0+0 (secure 0 0 0) 0x7f5c0c024060 con 0x55c459f89330 2024-03-06T07:26:15.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 10 client.9123 fetch_fsmap finished waiting for FSMap version 146 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 10 client.9123 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 20 client.9123 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fadd30 0x55c45a038e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 4 client.9123 mds_command: new command op to 8203 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 --> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c459ef4f40 con 0x55c459fadd30 2024-03-06T07:26:15.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c215ab640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fadd30 0x55c45a038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5bfbfff640 10 client.9123 ms_handle_connect on v2:172.21.15.163:6835/2502302602 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.154+0000 7f5c215ab640 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fadd30 0x55c45a038e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247d84ef474f62d0 server_cookie=41c222f4da028239 in_seq=0 out_seq=0 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5bfbfff640 1 -- 172.21.15.17:0/212908714 <== mds.0 v2:172.21.15.163:6835/2502302602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55c459ef4f40 con 0x55c459fadd30 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5bfbfff640 10 client.9123 handle_command_reply: tid=0 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 client.9123 shutdown 2024-03-06T07:26:15.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 msgr2=0x55c459fe72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009160 tx=0x7f5c0c08a040 comp rx=0 tx=0).stop 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 shutdown_connections 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:6835/2502302602,v1:172.21.15.163:6837/2502302602] conn(0x55c459fadd30 0x55c45a038e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c459fabc50 0x55c459fe9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c459f89330 0x55c459fe72d0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5bfbfff640 0 client.9123 ms_handle_reset on v2:172.21.15.163:6835/2502302602 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 --2- 172.21.15.17:0/212908714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c459f88f50 0x55c459f86770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 >> 172.21.15.17:0/212908714 conn(0x55c459f874b0 msgr2=0x55c459fc2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 shutdown_connections 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 1 -- 172.21.15.17:0/212908714 wait complete. 2024-03-06T07:26:15.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c263f0740 20 client.9123 trim_cache size 0 max 16384 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 8203, 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:26:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.4315270460000002, 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 146, 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 156, 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 156, 2024-03-06T07:26:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25.073497756999998 2024-03-06T07:26:15.172 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 msgr2=0x7f5c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 0x7f5c0006d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5c180013c0 tx=0x7f5c18001350 comp rx=0 tx=0).stop 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 msgr2=0x7f5c1c1eea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00ebe0 tx=0x7f5c0c00c520 comp rx=0 tx=0).stop 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 shutdown_connections 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c1c1587e0 0x7f5c1c153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c1c1526c0 0x7f5c1c1eea30 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 --2- 172.21.15.17:0/3025941906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c1c0c2b20 0x7f5c1c1ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.155+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 >> 172.21.15.17:0/3025941906 conn(0x7f5c1c0aa810 msgr2=0x7f5c1c0a9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.156+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 shutdown_connections 2024-03-06T07:26:15.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.156+0000 7f5c2380f640 1 -- 172.21.15.17:0/3025941906 wait complete. 2024-03-06T07:26:15.176 DEBUG:tasks.cephfs.test_failover:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 11, "epoch": 146, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 11, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:25:54.526748+0000", "damaged": [], "data_pools": [ 24 ], "enabled": true, "epoch": 146, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8203": { "addr": "172.21.15.163:6837/2502302602", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6835", "nonce": 2502302602, "type": "v2" }, { "addr": "172.21.15.163:6837", "nonce": 2502302602, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8203, "incarnation": 143, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 155, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 23, "modified": "2024-03-06T07:26:13.724498+0000", "qdb_cluster": [ 8203 ], "qdb_leader": 8203, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8203 } } }, { "id": 12, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:26:03.654281+0000", "damaged": [], "data_pools": [ 26 ], "enabled": true, "epoch": 146, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs2", "in": [ 0 ], "info": { "gid_8200": { "addr": "172.21.15.163:6836/91031835", "addrs": { "addrvec": [ { "addr": "172.21.15.163:6834", "nonce": 91031835, "type": "v2" }, { "addr": "172.21.15.163:6836", "nonce": 91031835, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8200, "incarnation": 143, "join_fscid": 12, "name": "d", "rank": 0, "state": "up:active", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 156, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 25, "modified": "2024-03-06T07:26:13.724499+0000", "qdb_cluster": [ 8200 ], "qdb_leader": 8200, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8200 } } } ], "standbys": [ { "addr": "172.21.15.17:6833/4075955916", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 4075955916, "type": "v2" }, { "addr": "172.21.15.17:6833", "nonce": 4075955916, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 144, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 9084, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.17:6835/2508903849", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6834", "nonce": 2508903849, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 2508903849, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 144, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 9090, "incarnation": 0, "join_fscid": 12, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-06T07:26:15.178 INFO:tasks.cephfs.test_failover:target = [{'join_fscid': 12, 'name': 'a'}, {'join_fscid': -1, 'name': 'b', 'state': 'up:active'}, {'join_fscid': -1, 'name': 'c'}, {'join_fscid': 12, 'name': 'd'}] 2024-03-06T07:26:15.179 INFO:tasks.cephfs.test_failover:current = [{'gid': 9090, 'name': 'a', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6835/2508903849', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2508903849}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2508903849}]}, 'join_fscid': 12, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 144}, {'gid': 8203, 'name': 'b', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.163:6837/2502302602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 2502302602}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 2502302602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, {'gid': 9084, 'name': 'c', 'rank': -1, 'incarnation': 0, 'state': 'up:standby', 'state_seq': 1, 'addr': '172.21.15.17:6833/4075955916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 4075955916}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 4075955916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'epoch': 144}, {'gid': 8200, 'name': 'd', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.163:6836/91031835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 91031835}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 91031835}]}, 'join_fscid': 12, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}] 2024-03-06T07:26:15.179 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-06T07:26:15.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:15.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 Processor -- start 2024-03-06T07:26:15.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 -- start start 2024-03-06T07:26:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fe0072f70 0x7f7fe0154570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe00bdc30 con 0x7f7fe00bbf80 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe00bd690 con 0x7f7fe0072f70 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fe62eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00bd950 con 0x7f7fe00c6560 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.279+0000 7f7fdffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fe0072f70 0x7f7fe0154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fe0072f70 0x7f7fe0154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36930/0 (socket says 172.21.15.17:36930) 2024-03-06T07:26:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 -- 172.21.15.17:0/1253215885 learned_addr learned my addr 172.21.15.17:0/1253215885 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdf7fe640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 -- 172.21.15.17:0/1253215885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 msgr2=0x7f7fe0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 -- 172.21.15.17:0/1253215885 >> v1:172.21.15.163:6789/0 conn(0x7f7fe00bbf80 legacy=0x7f7fe0156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.280+0000 7f7fdffff640 1 -- 172.21.15.17:0/1253215885 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00c5540 con 0x7f7fe0072f70 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fdffff640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe0072f70 0x7f7fe0154570 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009870 tx=0x7f7fd002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c6478f95f1e7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fdeffd640 1 -- 172.21.15.17:0/1253215885 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0035020 con 0x7f7fe0072f70 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fdeffd640 1 -- 172.21.15.17:0/1253215885 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd0031640 con 0x7f7fe0072f70 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fdeffd640 1 -- 172.21.15.17:0/1253215885 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd002d040 con 0x7f7fe0072f70 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 -- 172.21.15.17:0/1253215885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe0072f70 msgr2=0x7f7fe0154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe0072f70 0x7f7fe0154570 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009870 tx=0x7f7fd002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 -- 172.21.15.17:0/1253215885 shutdown_connections 2024-03-06T07:26:15.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/1253215885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe0072f70 0x7f7fe0154570 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 -- 172.21.15.17:0/1253215885 >> 172.21.15.17:0/1253215885 conn(0x7f7fe006b1a0 msgr2=0x7f7fe006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 -- 172.21.15.17:0/1253215885 shutdown_connections 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.281+0000 7f7fe62eb640 1 -- 172.21.15.17:0/1253215885 wait complete. 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.282+0000 7f7fe62eb640 1 Processor -- start 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 -- start start 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0072f70 0x7f7fe00a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0073350 con 0x7f7fe00bbf80 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00bba60 con 0x7f7fe0072f70 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe62eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00734d0 con 0x7f7fe00c6560 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36936/0 (socket says 172.21.15.17:36936) 2024-03-06T07:26:15.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fdf7fe640 1 -- 172.21.15.17:0/2442959418 learned_addr learned my addr 172.21.15.17:0/2442959418 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fdffff640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0072f70 0x7f7fe00a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.283+0000 7f7fe4888640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 -- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 msgr2=0x7f7fe006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 -- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0072f70 msgr2=0x7f7fe00a7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0072f70 0x7f7fe00a7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 -- 172.21.15.17:0/2442959418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00ac7e0 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdf7fe640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9e0 tx=0x7f7fd400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4017070 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd400c9d0 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe0155b30 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400cdc0 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.284+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe01560f0 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.285+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe004f330 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.286+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fd401a070 con 0x7f7fe00bbf80 2024-03-06T07:26:15.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.286+0000 7f7fdcff9640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 0x7f7fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.286+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f7fd408a690 con 0x7f7fe00bbf80 2024-03-06T07:26:15.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.286+0000 7f7fdffff640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 0x7f7fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.287+0000 7f7fdffff640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 0x7f7fcc06d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0036000 tx=0x7f7fd003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.290+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fd40589c0 con 0x7f7fe00bbf80 2024-03-06T07:26:15.449 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:26:15.448+0000 7f8629c23640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:15.450 INFO:tasks.ceph.mds.d.smithi163.stderr:2024-03-06T07:26:15.448+0000 7f932869b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.510+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe00c6260 con 0x7f7fe00bbf80 2024-03-06T07:26:15.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.510+0000 7f7fdcff9640 1 -- 172.21.15.17:0/2442959418 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7f7fd405ca60 con 0x7f7fe00bbf80 2024-03-06T07:26:15.512 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:15.512 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:15.512 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 msgr2=0x7f7fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 0x7f7fcc06d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0036000 tx=0x7f7fd003c040 comp rx=0 tx=0).stop 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 msgr2=0x7f7fe006d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400e9e0 tx=0x7f7fd400eea0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 shutdown_connections 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fcc06b4d0 0x7f7fcc06d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fe00c6560 0x7f7fe006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fe00bbf80 0x7f7fe006d070 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 --2- 172.21.15.17:0/2442959418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fe0072f70 0x7f7fe00a7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 >> 172.21.15.17:0/2442959418 conn(0x7f7fe006b1a0 msgr2=0x7f7fe00c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 shutdown_connections 2024-03-06T07:26:15.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.512+0000 7f7fe62eb640 1 -- 172.21.15.17:0/2442959418 wait complete. 2024-03-06T07:26:15.525 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:26:15.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:15.573 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:15.616 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:26:15.616 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:26:15.616 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:15.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:26:15.631 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:26:15.631 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:15.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:26:15.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 Processor -- start 2024-03-06T07:26:15.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 -- start start 2024-03-06T07:26:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff28806d9f0 0x7ff2880a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2880c3710 0x7ff2881558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff288158800 con 0x7ff2880c3280 2024-03-06T07:26:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff288158ae0 con 0x7ff28806d9f0 2024-03-06T07:26:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.774+0000 7ff28e9b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff288158dc0 con 0x7ff2880c3710 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff287fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff28806d9f0 0x7ff2880a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff2877fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2880c3280 0x7ff2881551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35022/0 (socket says 172.21.15.17:35022) 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff2877fe640 1 -- 172.21.15.17:0/3881469553 learned_addr learned my addr 172.21.15.17:0/3881469553 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff28cf53640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2880c3710 0x7ff2881558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff287fff640 1 -- 172.21.15.17:0/3881469553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2880c3710 msgr2=0x7ff2881558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff287fff640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2880c3710 0x7ff2881558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff287fff640 1 -- 172.21.15.17:0/3881469553 >> v1:172.21.15.17:6789/0 conn(0x7ff2880c3280 legacy=0x7ff2881551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff287fff640 1 -- 172.21.15.17:0/3881469553 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2881590a0 con 0x7ff28806d9f0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.775+0000 7ff286ffd640 1 -- 172.21.15.17:0/3881469553 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985406439 0 0) 0x7ff288158800 con 0x7ff2880c3280 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff287fff640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28806d9f0 0x7ff2880a7cd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff2780097a0 tx=0x7ff27802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50860a735e197915 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff286ffd640 1 -- 172.21.15.17:0/3881469553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780315a0 con 0x7ff28806d9f0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff286ffd640 1 -- 172.21.15.17:0/3881469553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff278037070 con 0x7ff28806d9f0 2024-03-06T07:26:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff286ffd640 1 -- 172.21.15.17:0/3881469553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278031960 con 0x7ff28806d9f0 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 -- 172.21.15.17:0/3881469553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28806d9f0 msgr2=0x7ff2880a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28806d9f0 0x7ff2880a7cd0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff2780097a0 tx=0x7ff27802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 -- 172.21.15.17:0/3881469553 shutdown_connections 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2880c3710 0x7ff2881558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/3881469553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff28806d9f0 0x7ff2880a7cd0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 -- 172.21.15.17:0/3881469553 >> 172.21.15.17:0/3881469553 conn(0x7ff2880aa630 msgr2=0x7ff2880aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 -- 172.21.15.17:0/3881469553 shutdown_connections 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.776+0000 7ff28e9b6640 1 -- 172.21.15.17:0/3881469553 wait complete. 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.777+0000 7ff28e9b6640 1 Processor -- start 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 -- start start 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff28806d9f0 0x7ff2881ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2880c3710 0x7ff28809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28806ddd0 con 0x7ff2880c3280 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff288155e30 con 0x7ff2880c3710 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28e9b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28806df50 con 0x7ff28806d9f0 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36954/0 (socket says 172.21.15.17:36954) 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff2877fe640 1 -- 172.21.15.17:0/4022490403 learned_addr learned my addr 172.21.15.17:0/4022490403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff287fff640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff28806d9f0 0x7ff2881ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.778+0000 7ff28cf53640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2880c3710 0x7ff28809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 -- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff28806d9f0 msgr2=0x7ff2881ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff28806d9f0 0x7ff2881ea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 -- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2880c3710 msgr2=0x7ff28809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2880c3710 0x7ff28809ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 -- 172.21.15.17:0/4022490403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2880ae070 con 0x7ff2880c3280 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff2877fe640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c4d0 tx=0x7ff27c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c018070 con 0x7ff2880c3280 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27c007500 con 0x7ff2880c3280 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2880acf70 con 0x7ff2880c3280 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c004a20 con 0x7ff2880c3280 2024-03-06T07:26:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.779+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2880ad530 con 0x7ff2880c3280 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.780+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28804f330 con 0x7ff2880c3280 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.781+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff27c01b070 con 0x7ff2880c3280 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.781+0000 7ff284ff9640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 0x7ff27406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.781+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7ff27c089930 con 0x7ff2880c3280 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.781+0000 7ff287fff640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 0x7ff27406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.782+0000 7ff287fff640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 0x7ff27406d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:15.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:15.785+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff27c057c60 con 0x7ff2880c3280 2024-03-06T07:26:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.005+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2880c5f70 con 0x7ff2880c3280 2024-03-06T07:26:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.006+0000 7ff284ff9640 1 -- 172.21.15.17:0/4022490403 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7ff27c05bd00 con 0x7ff2880c3280 2024-03-06T07:26:16.007 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:16.007 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:16.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 msgr2=0x7ff27406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 0x7ff27406d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803d040 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 msgr2=0x7ff288156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00c4d0 tx=0x7ff27c00c990 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 shutdown_connections 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff27406b500 0x7ff27406d9c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2880c3710 0x7ff28809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2880c3280 0x7ff288156e90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 --2- 172.21.15.17:0/4022490403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff28806d9f0 0x7ff2881ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 >> 172.21.15.17:0/4022490403 conn(0x7ff2880aa630 msgr2=0x7ff2880c5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 shutdown_connections 2024-03-06T07:26:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.008+0000 7ff28e9b6640 1 -- 172.21.15.17:0/4022490403 wait complete. 2024-03-06T07:26:16.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:16.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.120+0000 7f68e613e640 1 Processor -- start 2024-03-06T07:26:16.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.120+0000 7f68e613e640 1 -- start start 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68e613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68e613e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e0072630 con 0x7f68e01592d0 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68e613e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e0072910 con 0x7f68e006da40 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68e613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e006de20 con 0x7f68e00c6780 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68dffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68e01592d0 0x7f68e00c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35026/0 (socket says 172.21.15.17:35026) 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68dffff640 1 -- 172.21.15.17:0/881326222 learned_addr learned my addr 172.21.15.17:0/881326222 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68deffd640 1 --2- 172.21.15.17:0/881326222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651414485 0 0) 0x7f68e0072630 con 0x7f68e01592d0 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c8003520 con 0x7f68e01592d0 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390474076 0 0) 0x7f68e0072910 con 0x7f68e006da40 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e0072630 con 0x7f68e006da40 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68deffd640 1 -- 172.21.15.17:0/881326222 >> v1:172.21.15.163:6789/0 conn(0x7f68e006da40 legacy=0x7f68e0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68deffd640 1 -- 172.21.15.17:0/881326222 >> v1:172.21.15.17:6789/0 conn(0x7f68e01592d0 legacy=0x7f68e00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.121+0000 7f68deffd640 1 -- 172.21.15.17:0/881326222 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0072bf0 con 0x7f68e00c6780 2024-03-06T07:26:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.122+0000 7f68deffd640 1 --2- 172.21.15.17:0/881326222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e0158d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097d0 tx=0x7f68d402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555cb0551b07d5ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.122+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4035020 con 0x7f68e00c6780 2024-03-06T07:26:16.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.122+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4031800 con 0x7f68e00c6780 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.122+0000 7f68de7fc640 1 -- 172.21.15.17:0/881326222 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4031bb0 con 0x7f68e00c6780 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 -- 172.21.15.17:0/881326222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 msgr2=0x7f68e0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 --2- 172.21.15.17:0/881326222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e0158d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097d0 tx=0x7f68d402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 -- 172.21.15.17:0/881326222 shutdown_connections 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 --2- 172.21.15.17:0/881326222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e0158d90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 -- 172.21.15.17:0/881326222 >> 172.21.15.17:0/881326222 conn(0x7f68e00aa630 msgr2=0x7f68e00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 -- 172.21.15.17:0/881326222 shutdown_connections 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.123+0000 7f68e613e640 1 -- 172.21.15.17:0/881326222 wait complete. 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.124+0000 7f68e613e640 1 Processor -- start 2024-03-06T07:26:16.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.124+0000 7f68e613e640 1 -- start start 2024-03-06T07:26:16.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 0x7f68e00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00c3b10 con 0x7f68e01592d0 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00c3990 con 0x7f68e006da40 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68e613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e00c7380 con 0x7f68e00c6780 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43884/0 (socket says 172.21.15.17:43884) 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68dffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36964/0 (socket says 172.21.15.17:36964) 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 -- 172.21.15.17:0/767168636 learned_addr learned my addr 172.21.15.17:0/767168636 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68df7fe640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 0x7f68e00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 -- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 msgr2=0x7f68e00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 0x7f68e00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 -- 172.21.15.17:0/767168636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 msgr2=0x7f68e00a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 -- 172.21.15.17:0/767168636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e01ea9f0 con 0x7f68e00c6780 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68df7fe640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 0x7f68e00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:16.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68dffff640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.125+0000 7f68deffd640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000c00 tx=0x7f68d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.126+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4041070 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.126+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4031e30 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.126+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e006fff0 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.126+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4035040 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.126+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e01eacb0 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.127+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e004f330 con 0x7f68e00c6780 2024-03-06T07:26:16.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.127+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68d404c060 con 0x7f68e00c6780 2024-03-06T07:26:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.127+0000 7f68c7fff640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 0x7f68cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.128+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f68d40abff0 con 0x7f68e00c6780 2024-03-06T07:26:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.128+0000 7f68df7fe640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 0x7f68cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.128+0000 7f68df7fe640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 0x7f68cc06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007fd0 tx=0x7f68d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.132+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68d407a320 con 0x7f68e00c6780 2024-03-06T07:26:16.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.312+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68e0073240 con 0x7f68e00c6780 2024-03-06T07:26:16.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.314+0000 7f68c7fff640 1 -- 172.21.15.17:0/767168636 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20857 (secure 0 0 0) 0x7f68d407e3c0 con 0x7f68e00c6780 2024-03-06T07:26:16.316 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:16.316 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":156,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:10.717640+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 msgr2=0x7f68cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 0x7f68cc06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f68d0007fd0 tx=0x7f68d0009040 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 msgr2=0x7f68e01eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000c00 tx=0x7f68d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 shutdown_connections 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68cc06b4d0 0x7f68cc06d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68e01592d0 0x7f68e00a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68e00c6780 0x7f68e01eb4b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 --2- 172.21.15.17:0/767168636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68e006da40 0x7f68e00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 >> 172.21.15.17:0/767168636 conn(0x7f68e00aa630 msgr2=0x7f68e00c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 shutdown_connections 2024-03-06T07:26:16.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.316+0000 7f68e613e640 1 -- 172.21.15.17:0/767168636 wait complete. 2024-03-06T07:26:16.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:25:52.790714+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:25:53.497871+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:26:02.038846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:26:02.549840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:16.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:16.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.431+0000 7f88c37b6640 1 Processor -- start 2024-03-06T07:26:16.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.431+0000 7f88c37b6640 1 -- start start 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c37b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc06d750 0x7f88bc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c37b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88bc0c3a70 0x7f88bc072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c37b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc156060 con 0x7f88bc06db30 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c37b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc155b30 con 0x7f88bc0c3a70 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c37b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0a8430 con 0x7f88bc06d750 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c0d51640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f88bc0c3a70 0x7f88bc072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c1d53640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f88bc06db30 0x7f88bc0c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35040/0 (socket says 172.21.15.17:35040) 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c1d53640 1 -- 172.21.15.17:0/1632214342 learned_addr learned my addr 172.21.15.17:0/1632214342 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.432+0000 7f88c1552640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc06d750 0x7f88bc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88c0d51640 1 -- 172.21.15.17:0/1632214342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc06d750 msgr2=0x7f88bc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88c0d51640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc06d750 0x7f88bc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88c0d51640 1 -- 172.21.15.17:0/1632214342 >> v1:172.21.15.17:6789/0 conn(0x7f88bc06db30 legacy=0x7f88bc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88c0d51640 1 -- 172.21.15.17:0/1632214342 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc155d80 con 0x7f88bc0c3a70 2024-03-06T07:26:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88c0d51640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c3a70 0x7f88bc072680 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f88b00097a0 tx=0x7f88b002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e000fa334079d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88bbfff640 1 -- 172.21.15.17:0/1632214342 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0039070 con 0x7f88bc0c3a70 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88bbfff640 1 -- 172.21.15.17:0/1632214342 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b00316f0 con 0x7f88bc0c3a70 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.433+0000 7f88bbfff640 1 -- 172.21.15.17:0/1632214342 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b0031ac0 con 0x7f88bc0c3a70 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 -- 172.21.15.17:0/1632214342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c3a70 msgr2=0x7f88bc072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c3a70 0x7f88bc072680 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f88b00097a0 tx=0x7f88b002b790 comp rx=0 tx=0).stop 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 -- 172.21.15.17:0/1632214342 shutdown_connections 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc0c3a70 0x7f88bc072680 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 --2- 172.21.15.17:0/1632214342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc06d750 0x7f88bc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 -- 172.21.15.17:0/1632214342 >> 172.21.15.17:0/1632214342 conn(0x7f88bc0c5560 msgr2=0x7f88bc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 -- 172.21.15.17:0/1632214342 shutdown_connections 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.434+0000 7f88c37b6640 1 -- 172.21.15.17:0/1632214342 wait complete. 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.435+0000 7f88c37b6640 1 Processor -- start 2024-03-06T07:26:16.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.435+0000 7f88c37b6640 1 -- start start 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06db30 0x7f88bc1eae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c3a70 0x7f88bc154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc072bc0 con 0x7f88bc06d750 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c27f0 con 0x7f88bc06db30 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c37b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc072d40 con 0x7f88bc0c3a70 2024-03-06T07:26:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36980/0 (socket says 172.21.15.17:36980) 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 -- 172.21.15.17:0/2576528648 learned_addr learned my addr 172.21.15.17:0/2576528648 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1d53640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c3a70 0x7f88bc154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c0d51640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06db30 0x7f88bc1eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 -- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c3a70 msgr2=0x7f88bc154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c3a70 0x7f88bc154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 -- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06db30 msgr2=0x7f88bc1eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06db30 0x7f88bc1eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 -- 172.21.15.17:0/2576528648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc1eb3c0 con 0x7f88bc06d750 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.436+0000 7f88c1552640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00efc0 tx=0x7f88ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.437+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac018020 con 0x7f88bc06d750 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.437+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac014480 con 0x7f88bc06d750 2024-03-06T07:26:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.437+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc14dd70 con 0x7f88bc06d750 2024-03-06T07:26:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.437+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac014850 con 0x7f88bc06d750 2024-03-06T07:26:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.437+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc070400 con 0x7f88bc06d750 2024-03-06T07:26:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.438+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc04f330 con 0x7f88bc06d750 2024-03-06T07:26:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.438+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88ac005d10 con 0x7f88bc06d750 2024-03-06T07:26:16.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.439+0000 7f88b9ffb640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.439+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f88ac08b170 con 0x7f88bc06d750 2024-03-06T07:26:16.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.439+0000 7f88c0d51640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.439+0000 7f88c0d51640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 0x7f88a806d9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f88b0036000 tx=0x7f88b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.443+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88ac022070 con 0x7f88bc06d750 2024-03-06T07:26:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.621+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88bc074700 con 0x7f88bc06d750 2024-03-06T07:26:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.622+0000 7f88b9ffb640 1 -- 172.21.15.17:0/2576528648 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88bc1eb3c0 con 0x7f88bc06d750 2024-03-06T07:26:16.624 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:26:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 msgr2=0x7f88a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 0x7f88a806d9c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f88b0036000 tx=0x7f88b0008030 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 msgr2=0x7f88bc1ee4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00efc0 tx=0x7f88ac00c490 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 shutdown_connections 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88bc0c3a70 0x7f88bc154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88bc06db30 0x7f88bc1eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 --2- 172.21.15.17:0/2576528648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88bc06d750 0x7f88bc1ee4d0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 >> 172.21.15.17:0/2576528648 conn(0x7f88bc0c5560 msgr2=0x7f88bc0a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 shutdown_connections 2024-03-06T07:26:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.624+0000 7f88c37b6640 1 -- 172.21.15.17:0/2576528648 wait complete. 2024-03-06T07:26:16.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:26:16.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.739+0000 7f49d113e640 1 Processor -- start 2024-03-06T07:26:16.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.739+0000 7f49d113e640 1 -- start start 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49d113e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c47e0 0x7f49cc0c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49d113e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49cc0c4bc0 0x7f49cc072670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49d113e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc0c8170 con 0x7f49cc0c7a90 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49d113e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0c7440 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49d113e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc153cb0 con 0x7f49cc0c47e0 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cb59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49cc0c4bc0 0x7f49cc072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49ca59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49cc0c7a90 0x7f49cc0c6d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35046/0 (socket says 172.21.15.17:35046) 2024-03-06T07:26:16.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49ca59c640 1 -- 172.21.15.17:0/2002896706 learned_addr learned my addr 172.21.15.17:0/2002896706 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cad9d640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c47e0 0x7f49cc0c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cb59e640 1 -- 172.21.15.17:0/2002896706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c47e0 msgr2=0x7f49cc0c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cb59e640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c47e0 0x7f49cc0c67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cb59e640 1 -- 172.21.15.17:0/2002896706 >> v1:172.21.15.17:6789/0 conn(0x7f49cc0c7a90 legacy=0x7f49cc0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.740+0000 7f49cb59e640 1 -- 172.21.15.17:0/2002896706 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc153f90 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.741+0000 7f49cb59e640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc072670 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009d90 tx=0x7f49c402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f282f7db213a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.741+0000 7f49c9d9b640 1 -- 172.21.15.17:0/2002896706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4039070 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.741+0000 7f49c9d9b640 1 -- 172.21.15.17:0/2002896706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c402cb40 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.741+0000 7f49c9d9b640 1 -- 172.21.15.17:0/2002896706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4034440 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 -- 172.21.15.17:0/2002896706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c4bc0 msgr2=0x7f49cc072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc072670 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009d90 tx=0x7f49c402bd90 comp rx=0 tx=0).stop 2024-03-06T07:26:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 -- 172.21.15.17:0/2002896706 shutdown_connections 2024-03-06T07:26:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc072670 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 --2- 172.21.15.17:0/2002896706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c47e0 0x7f49cc0c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 -- 172.21.15.17:0/2002896706 >> 172.21.15.17:0/2002896706 conn(0x7f49cc0aa6a0 msgr2=0x7f49cc0aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 -- 172.21.15.17:0/2002896706 shutdown_connections 2024-03-06T07:26:16.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.742+0000 7f49d113e640 1 -- 172.21.15.17:0/2002896706 wait complete. 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.743+0000 7f49d113e640 1 Processor -- start 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.743+0000 7f49d113e640 1 -- start start 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc14f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c7a90 0x7f49cc14f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc072bb0 con 0x7f49cc0c47e0 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc070a40 con 0x7f49cc0c4bc0 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49d113e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc072d30 con 0x7f49cc0c7a90 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37002/0 (socket says 172.21.15.17:37002) 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 -- 172.21.15.17:0/701945428 learned_addr learned my addr 172.21.15.17:0/701945428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cb59e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c7a90 0x7f49cc14f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49ca59c640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc14f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 -- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c7a90 msgr2=0x7f49cc14f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c7a90 0x7f49cc14f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 -- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49cc0c4bc0 msgr2=0x7f49cc14f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc14f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 -- 172.21.15.17:0/701945428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc1ee6c0 con 0x7f49cc0c47e0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.744+0000 7f49cad9d640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e8a0 tx=0x7f49c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.745+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49cc0c47e0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.745+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000c940 con 0x7f49cc0c47e0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.745+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc1ee9a0 con 0x7f49cc0c47e0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.745+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000cd30 con 0x7f49cc0c47e0 2024-03-06T07:26:16.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.745+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc1eef60 con 0x7f49cc0c47e0 2024-03-06T07:26:16.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.746+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc06c610 con 0x7f49cc0c47e0 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.746+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49c001a070 con 0x7f49cc0c47e0 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.746+0000 7f49b37fe640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.747+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f49c008a880 con 0x7f49cc0c47e0 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.747+0000 7f49ca59c640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.747+0000 7f49ca59c640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 0x7f49b806d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06f790 tx=0x7f49bc007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:16.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.751+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49cc06c610 con 0x7f49cc0c47e0 2024-03-06T07:26:16.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.930+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49cc04f330 con 0x7f49cc0c47e0 2024-03-06T07:26:16.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.931+0000 7f49b37fe640 1 -- 172.21.15.17:0/701945428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49c0058bb0 con 0x7f49cc0c47e0 2024-03-06T07:26:16.939 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 msgr2=0x7f49b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 0x7f49b806d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06f790 tx=0x7f49bc007e80 comp rx=0 tx=0).stop 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 msgr2=0x7f49cc06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e8a0 tx=0x7f49c000ed60 comp rx=0 tx=0).stop 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 shutdown_connections 2024-03-06T07:26:16.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f49b806b4d0 0x7f49b806d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49cc0c7a90 0x7f49cc14f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49cc0c4bc0 0x7f49cc14f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 --2- 172.21.15.17:0/701945428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0c47e0 0x7f49cc06fa30 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 >> 172.21.15.17:0/701945428 conn(0x7f49cc0aa6a0 msgr2=0x7f49cc0c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 shutdown_connections 2024-03-06T07:26:16.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:16.932+0000 7f49d113e640 1 -- 172.21.15.17:0/701945428 wait complete. 2024-03-06T07:26:16.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:26:17.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.048+0000 7f41108d4640 1 Processor -- start 2024-03-06T07:26:17.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.048+0000 7f41108d4640 1 -- start start 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f41108d4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f410c0c2aa0 0x7f410c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f41108d4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c0670c0 con 0x7f410c0c26c0 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f41108d4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410c06b0f0 con 0x7f410c0c0470 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f41108d4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f410c06b740 con 0x7f410c0c2aa0 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f4103fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f410c0c2aa0 0x7f410c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f410a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f410c0c26c0 0x7f410c06bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35052/0 (socket says 172.21.15.17:35052) 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f410a59c640 1 -- 172.21.15.17:0/3027923634 learned_addr learned my addr 172.21.15.17:0/3027923634 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:17.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f4103fff640 1 -- 172.21.15.17:0/3027923634 >> v1:172.21.15.163:6789/0 conn(0x7f410c0c0470 legacy=0x7f410c152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f4103fff640 1 -- 172.21.15.17:0/3027923634 >> v1:172.21.15.17:6789/0 conn(0x7f410c0c26c0 legacy=0x7f410c06bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f4109d9b640 1 -- 172.21.15.17:0/3027923634 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245904084 0 0) 0x7f410c0670c0 con 0x7f410c0c26c0 2024-03-06T07:26:17.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.049+0000 7f4103fff640 1 -- 172.21.15.17:0/3027923634 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c06b3d0 con 0x7f410c0c2aa0 2024-03-06T07:26:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.050+0000 7f4103fff640 1 --2- 172.21.15.17:0/3027923634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c2aa0 0x7f410c155bd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097a0 tx=0x7f40f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b063b16a015d1a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.050+0000 7f4109d9b640 1 -- 172.21.15.17:0/3027923634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40315a0 con 0x7f410c0c2aa0 2024-03-06T07:26:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.050+0000 7f4109d9b640 1 -- 172.21.15.17:0/3027923634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f4037070 con 0x7f410c0c2aa0 2024-03-06T07:26:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.050+0000 7f4109d9b640 1 -- 172.21.15.17:0/3027923634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4031960 con 0x7f410c0c2aa0 2024-03-06T07:26:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 -- 172.21.15.17:0/3027923634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c2aa0 msgr2=0x7f410c155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 --2- 172.21.15.17:0/3027923634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c2aa0 0x7f410c155bd0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097a0 tx=0x7f40f402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 -- 172.21.15.17:0/3027923634 shutdown_connections 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 --2- 172.21.15.17:0/3027923634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c2aa0 0x7f410c155bd0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 -- 172.21.15.17:0/3027923634 >> 172.21.15.17:0/3027923634 conn(0x7f410c0c3b40 msgr2=0x7f410c0c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 -- 172.21.15.17:0/3027923634 shutdown_connections 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.051+0000 7f41108d4640 1 -- 172.21.15.17:0/3027923634 wait complete. 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.052+0000 7f41108d4640 1 Processor -- start 2024-03-06T07:26:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.052+0000 7f41108d4640 1 -- start start 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 0x7f410c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c0c2aa0 0x7f410c1ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c20b0 con 0x7f410c0c0470 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c0c1f30 con 0x7f410c0c2aa0 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410c066d00 con 0x7f410c0c26c0 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37010/0 (socket says 172.21.15.17:37010) 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 -- 172.21.15.17:0/3686039791 learned_addr learned my addr 172.21.15.17:0/3686039791 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f4103fff640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 0x7f410c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410ad9d640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c0c2aa0 0x7f410c1ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 -- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 msgr2=0x7f410c0727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 0x7f410c0727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 -- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c0c2aa0 msgr2=0x7f410c1ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c0c2aa0 0x7f410c1ea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 -- 172.21.15.17:0/3686039791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0ac440 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f4103fff640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 0x7f410c0727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f410a59c640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c4d0 tx=0x7f40fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc018070 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40fc0074e0 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40fc004a20 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c0ac720 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.053+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c1ee040 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.054+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c04f330 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.055+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40fc01b070 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.055+0000 7f4102ffd640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 0x7f40ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.055+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f40fc0899b0 con 0x7f410c0c0470 2024-03-06T07:26:17.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.055+0000 7f4103fff640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 0x7f40ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.056+0000 7f4103fff640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 0x7f40ec06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f40f4036000 tx=0x7f40f403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.059+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40fc057ce0 con 0x7f410c0c0470 2024-03-06T07:26:17.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.241+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f410c068e00 con 0x7f410c0c0470 2024-03-06T07:26:17.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.241+0000 7f4102ffd640 1 -- 172.21.15.17:0/3686039791 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40fc05bd80 con 0x7f410c0c0470 2024-03-06T07:26:17.243 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 msgr2=0x7f40ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 0x7f40ec06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f40f4036000 tx=0x7f40f403d040 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 msgr2=0x7f410c1ee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00c4d0 tx=0x7f40fc00c990 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 shutdown_connections 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40ec06b4d0 0x7f40ec06d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f410c0c2aa0 0x7f410c1ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f410c0c26c0 0x7f410c0727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 --2- 172.21.15.17:0/3686039791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f410c0c0470 0x7f410c1ee560 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 >> 172.21.15.17:0/3686039791 conn(0x7f410c0c3b40 msgr2=0x7f410c152120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 shutdown_connections 2024-03-06T07:26:17.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.243+0000 7f41108d4640 1 -- 172.21.15.17:0/3686039791 wait complete. 2024-03-06T07:26:17.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:26:17.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.357+0000 7fe295655640 1 Processor -- start 2024-03-06T07:26:17.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.357+0000 7fe295655640 1 -- start start 2024-03-06T07:26:17.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe295655640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe290072cd0 0x7fe290155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe295655640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe295655640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe290152db0 con 0x7fe2901529d0 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe295655640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900c6dd0 con 0x7fe290072cd0 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe295655640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c6940 con 0x7fe290073290 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe28e7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe290072cd0 0x7fe290155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43930/0 (socket says 172.21.15.17:43930) 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe28f7fe640 1 -- 172.21.15.17:0/658923462 learned_addr learned my addr 172.21.15.17:0/658923462 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.358+0000 7fe28effd640 1 --1- 172.21.15.17:0/658923462 >> v1:172.21.15.163:6789/0 conn(0x7fe2901529d0 0x7fe2900c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:41380/0 (socket says 172.21.15.17:41380) 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28f7fe640 1 -- 172.21.15.17:0/658923462 >> v2:172.21.15.163:3300/0 conn(0x7fe290072cd0 msgr2=0x7fe290155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:17.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28f7fe640 1 --2- 172.21.15.17:0/658923462 >> v2:172.21.15.163:3300/0 conn(0x7fe290072cd0 0x7fe290155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28dffb640 1 -- 172.21.15.17:0/658923462 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552317913 0 0) 0x7fe288001110 con 0x7fe2901529d0 2024-03-06T07:26:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28f7fe640 1 -- 172.21.15.17:0/658923462 >> v1:172.21.15.163:6789/0 conn(0x7fe2901529d0 legacy=0x7fe2900c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28f7fe640 1 -- 172.21.15.17:0/658923462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2901590f0 con 0x7fe290073290 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.359+0000 7fe28f7fe640 1 --2- 172.21.15.17:0/658923462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe280009e50 tx=0x7fe28002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6626bf5b5e9402 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe28dffb640 1 -- 172.21.15.17:0/658923462 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28002c9e0 con 0x7fe290073290 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe28dffb640 1 -- 172.21.15.17:0/658923462 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28002cb80 con 0x7fe290073290 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe28dffb640 1 -- 172.21.15.17:0/658923462 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800343f0 con 0x7fe290073290 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 -- 172.21.15.17:0/658923462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 msgr2=0x7fe290155660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 --2- 172.21.15.17:0/658923462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe280009e50 tx=0x7fe28002be40 comp rx=0 tx=0).stop 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 -- 172.21.15.17:0/658923462 shutdown_connections 2024-03-06T07:26:17.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 --2- 172.21.15.17:0/658923462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290155660 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 --2- 172.21.15.17:0/658923462 >> v2:172.21.15.163:3300/0 conn(0x7fe290072cd0 0x7fe290155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 -- 172.21.15.17:0/658923462 >> 172.21.15.17:0/658923462 conn(0x7fe2900aa630 msgr2=0x7fe2900aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 -- 172.21.15.17:0/658923462 shutdown_connections 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.360+0000 7fe295655640 1 -- 172.21.15.17:0/658923462 wait complete. 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.361+0000 7fe295655640 1 Processor -- start 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 -- start start 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290072cd0 0x7fe29006d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290159370 con 0x7fe2901529d0 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290072720 con 0x7fe290072cd0 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe295655640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900723b0 con 0x7fe290073290 2024-03-06T07:26:17.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37018/0 (socket says 172.21.15.17:37018) 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe28f7fe640 1 -- 172.21.15.17:0/12732075 learned_addr learned my addr 172.21.15.17:0/12732075 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe28e7fc640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.362+0000 7fe28effd640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290072cd0 0x7fe29006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 -- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 msgr2=0x7fe290148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 -- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290072cd0 msgr2=0x7fe29006d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290072cd0 0x7fe29006d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 -- 172.21.15.17:0/12732075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900ae370 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe28f7fe640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe280036040 tx=0x7fe280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280043020 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28003e450 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28003e820 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe290156560 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.363+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2900acfa0 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.364+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29004f330 con 0x7fe2901529d0 2024-03-06T07:26:17.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.364+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe280046020 con 0x7fe2901529d0 2024-03-06T07:26:17.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.365+0000 7fe2777fe640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 0x7fe27c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.365+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fe2800acf90 con 0x7fe2901529d0 2024-03-06T07:26:17.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.365+0000 7fe28effd640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 0x7fe27c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.366+0000 7fe28effd640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 0x7fe27c06d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.369+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe28004d020 con 0x7fe2901529d0 2024-03-06T07:26:17.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.550+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2900c4a00 con 0x7fe2901529d0 2024-03-06T07:26:17.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.550+0000 7fe2777fe640 1 -- 172.21.15.17:0/12732075 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v156) v1 ==== 105+0+11 (secure 0 0 0) 0x7fe2901590f0 con 0x7fe2901529d0 2024-03-06T07:26:17.552 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:26:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 msgr2=0x7fe27c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 0x7fe27c06d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe288007de0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 msgr2=0x7fe2901edb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fe280036040 tx=0x7fe280004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 shutdown_connections 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe27c06b4d0 0x7fe27c06d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2901529d0 0x7fe2901edb60 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe290073290 0x7fe290148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 --2- 172.21.15.17:0/12732075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe290072cd0 0x7fe29006d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 >> 172.21.15.17:0/12732075 conn(0x7fe2900aa630 msgr2=0x7fe29006ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 shutdown_connections 2024-03-06T07:26:17.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.552+0000 7fe295655640 1 -- 172.21.15.17:0/12732075 wait complete. 2024-03-06T07:26:17.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:26:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.670+0000 7face33d6640 1 Processor -- start 2024-03-06T07:26:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.670+0000 7face33d6640 1 -- start start 2024-03-06T07:26:17.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face33d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face33d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facdc155530 0x7facdc1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face33d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc152dd0 con 0x7facdc1529f0 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face33d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc159710 con 0x7facdc155530 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face33d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c68b0 con 0x7facdc0c5370 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1973640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facdc155530 0x7facdc1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1172640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7facdc1529f0 0x7facdc0c88f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35060/0 (socket says 172.21.15.17:35060) 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1172640 1 -- 172.21.15.17:0/1944411114 learned_addr learned my addr 172.21.15.17:0/1944411114 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face0971640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1973640 1 -- 172.21.15.17:0/1944411114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 msgr2=0x7facdc0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1973640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1973640 1 -- 172.21.15.17:0/1944411114 >> v1:172.21.15.17:6789/0 conn(0x7facdc1529f0 legacy=0x7facdc0c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7face1973640 1 -- 172.21.15.17:0/1944411114 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc158b00 con 0x7facdc155530 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.671+0000 7facdbfff640 1 -- 172.21.15.17:0/1944411114 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637242346 0 0) 0x7facdc152dd0 con 0x7facdc1529f0 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.672+0000 7face1973640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1559c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7faccc0097a0 tx=0x7faccc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fef031629abdad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.672+0000 7facdbfff640 1 -- 172.21.15.17:0/1944411114 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0315a0 con 0x7facdc155530 2024-03-06T07:26:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.672+0000 7facdbfff640 1 -- 172.21.15.17:0/1944411114 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc037070 con 0x7facdc155530 2024-03-06T07:26:17.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.672+0000 7facdbfff640 1 -- 172.21.15.17:0/1944411114 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc031960 con 0x7facdc155530 2024-03-06T07:26:17.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 -- 172.21.15.17:0/1944411114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 msgr2=0x7facdc1559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1559c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7faccc0097a0 tx=0x7faccc02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 -- 172.21.15.17:0/1944411114 shutdown_connections 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1559c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 --2- 172.21.15.17:0/1944411114 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 -- 172.21.15.17:0/1944411114 >> 172.21.15.17:0/1944411114 conn(0x7facdc0ac580 msgr2=0x7facdc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 -- 172.21.15.17:0/1944411114 shutdown_connections 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.673+0000 7face33d6640 1 -- 172.21.15.17:0/1944411114 wait complete. 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.674+0000 7face33d6640 1 Processor -- start 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.674+0000 7face33d6640 1 -- start start 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.674+0000 7face33d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.674+0000 7face33d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facdc1529f0 0x7facdc0ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face33d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face33d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1590c0 con 0x7facdc155530 2024-03-06T07:26:17.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face33d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c9000 con 0x7facdc1529f0 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face33d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1593a0 con 0x7facdc0c5370 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37028/0 (socket says 172.21.15.17:37028) 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 -- 172.21.15.17:0/3736496031 learned_addr learned my addr 172.21.15.17:0/3736496031 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1172640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face0971640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facdc1529f0 0x7facdc0ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 -- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 msgr2=0x7facdc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 -- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facdc1529f0 msgr2=0x7facdc0ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facdc1529f0 0x7facdc0ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 -- 172.21.15.17:0/3736496031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc1eaba0 con 0x7facdc155530 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face1973640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098c0 tx=0x7faccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc049070 con 0x7facdc155530 2024-03-06T07:26:17.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc032430 con 0x7facdc155530 2024-03-06T07:26:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc1eae80 con 0x7facdc155530 2024-03-06T07:26:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.675+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc032800 con 0x7facdc155530 2024-03-06T07:26:17.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.676+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc1eb440 con 0x7facdc155530 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.676+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc04f330 con 0x7facdc155530 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.677+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faccc0329a0 con 0x7facdc155530 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.677+0000 7facd9ffb640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 0x7facc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.677+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7faccc02f020 con 0x7facdc155530 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.678+0000 7face1172640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 0x7facc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.678+0000 7face1172640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 0x7facc806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7facd40012e0 tx=0x7facd4001700 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.682+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faccc079f90 con 0x7facdc155530 2024-03-06T07:26:17.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.863+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7facdc0c4af0 con 0x7facdc155530 2024-03-06T07:26:17.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.863+0000 7facd9ffb640 1 -- 172.21.15.17:0/3736496031 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v156) v1 ==== 101+0+11 (secure 0 0 0) 0x7faccc07e030 con 0x7facdc155530 2024-03-06T07:26:17.865 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:26:17.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 msgr2=0x7facc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 0x7facc806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7facd40012e0 tx=0x7facd4001700 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 msgr2=0x7facdc1ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faccc0098c0 tx=0x7faccc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 shutdown_connections 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facc806b4d0 0x7facc806d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facdc155530 0x7facdc1ee070 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facdc1529f0 0x7facdc0ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 --2- 172.21.15.17:0/3736496031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facdc0c5370 0x7facdc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 >> 172.21.15.17:0/3736496031 conn(0x7facdc0ac580 msgr2=0x7facdc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 shutdown_connections 2024-03-06T07:26:17.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.866+0000 7face33d6640 1 -- 172.21.15.17:0/3736496031 wait complete. 2024-03-06T07:26:17.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.977+0000 7fee860eb640 1 Processor -- start 2024-03-06T07:26:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.977+0000 7fee860eb640 1 -- start start 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee8014f360 0x7fee80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee800c88c0 0x7fee800c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee800c5880 0x7fee80158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee800c8dd0 con 0x7fee800c88c0 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee800c73d0 con 0x7fee800c5880 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee860eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee800be2e0 con 0x7fee8014f360 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee7effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee800c88c0 0x7fee800c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee7effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fee800c88c0 0x7fee800c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37030/0 (socket says 172.21.15.17:37030) 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee7effd640 1 -- 172.21.15.17:0/1517086572 learned_addr learned my addr 172.21.15.17:0/1517086572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee7f7fe640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee8014f360 0x7fee80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.978+0000 7fee7ffff640 1 --2- 172.21.15.17:0/1517086572 >> v2:172.21.15.163:3300/0 conn(0x7fee800c5880 0x7fee80158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 -- 172.21.15.17:0/1517086572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee8014f360 msgr2=0x7fee80155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee8014f360 0x7fee80155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 -- 172.21.15.17:0/1517086572 >> v2:172.21.15.163:3300/0 conn(0x7fee800c5880 msgr2=0x7fee80158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 --2- 172.21.15.17:0/1517086572 >> v2:172.21.15.163:3300/0 conn(0x7fee800c5880 0x7fee80158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 -- 172.21.15.17:0/1517086572 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee800c7670 con 0x7fee800c88c0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7effd640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee800c5340 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fee8008cc10 tx=0x7fee7402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6406da286b9f3668 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.979+0000 7fee7e7fc640 1 -- 172.21.15.17:0/1517086572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7402e070 con 0x7fee800c88c0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee7e7fc640 1 -- 172.21.15.17:0/1517086572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee74037070 con 0x7fee800c88c0 2024-03-06T07:26:17.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee7e7fc640 1 -- 172.21.15.17:0/1517086572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74031a00 con 0x7fee800c88c0 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 -- 172.21.15.17:0/1517086572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 msgr2=0x7fee800c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee800c5340 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fee8008cc10 tx=0x7fee7402b880 comp rx=0 tx=0).stop 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 -- 172.21.15.17:0/1517086572 shutdown_connections 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 --2- 172.21.15.17:0/1517086572 >> v2:172.21.15.163:3300/0 conn(0x7fee800c5880 0x7fee80158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee800c5340 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 --2- 172.21.15.17:0/1517086572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee8014f360 0x7fee80155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 -- 172.21.15.17:0/1517086572 >> 172.21.15.17:0/1517086572 conn(0x7fee800ac570 msgr2=0x7fee800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:17.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 -- 172.21.15.17:0/1517086572 shutdown_connections 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.980+0000 7fee860eb640 1 -- 172.21.15.17:0/1517086572 wait complete. 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.981+0000 7fee860eb640 1 Processor -- start 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.981+0000 7fee860eb640 1 -- start start 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee800c5880 0x7fee80157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee8014f360 0x7fee80153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee801590a0 con 0x7fee800c88c0 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee800c7c30 con 0x7fee8014f360 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee860eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee80159220 con 0x7fee800c5880 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37032/0 (socket says 172.21.15.17:37032) 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 -- 172.21.15.17:0/3221626067 learned_addr learned my addr 172.21.15.17:0/3221626067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7f7fe640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee800c5880 0x7fee80157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7ffff640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee8014f360 0x7fee80153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 -- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee800c5880 msgr2=0x7fee80157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee800c5880 0x7fee80157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 -- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee8014f360 msgr2=0x7fee80153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee8014f360 0x7fee80153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 -- 172.21.15.17:0/3221626067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80154900 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.982+0000 7fee7effd640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fee7402b8c0 tx=0x7fee740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.983+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74032670 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.983+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee74032810 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.983+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74032c00 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.983+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee800beed0 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.983+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee80154ba0 con 0x7fee800c88c0 2024-03-06T07:26:17.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.984+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee800c69d0 con 0x7fee800c88c0 2024-03-06T07:26:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.984+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee740453f0 con 0x7fee800c88c0 2024-03-06T07:26:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.985+0000 7fee67fff640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 0x7fee6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.985+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fee740ac2c0 con 0x7fee800c88c0 2024-03-06T07:26:17.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.985+0000 7fee7f7fe640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 0x7fee6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:17.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.985+0000 7fee7f7fe640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 0x7fee6c06d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fee70009aa0 tx=0x7fee70009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:17.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:17.989+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fee800c69d0 con 0x7fee800c88c0 2024-03-06T07:26:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.208+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee800c56a0 con 0x7fee800c88c0 2024-03-06T07:26:18.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.208+0000 7fee67fff640 1 -- 172.21.15.17:0/3221626067 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7fee800c56a0 con 0x7fee800c88c0 2024-03-06T07:26:18.210 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:18.211 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:18.211 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:18.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 msgr2=0x7fee6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 0x7fee6c06d9c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fee70009aa0 tx=0x7fee70009190 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 msgr2=0x7fee80153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fee7402b8c0 tx=0x7fee740046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 shutdown_connections 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee6c06b500 0x7fee6c06d9c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee8014f360 0x7fee80153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee800c88c0 0x7fee80153400 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 --2- 172.21.15.17:0/3221626067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee800c5880 0x7fee80157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.210+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 >> 172.21.15.17:0/3221626067 conn(0x7fee800ac570 msgr2=0x7fee8014e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.211+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 shutdown_connections 2024-03-06T07:26:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.211+0000 7fee860eb640 1 -- 172.21.15.17:0/3221626067 wait complete. 2024-03-06T07:26:18.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 Processor -- start 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 -- start start 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f4c14f790 0x7f4f4c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f4c155a40 0x7f4f4c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0c28d0 con 0x7f4f4c0c1aa0 2024-03-06T07:26:18.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c2680 con 0x7f4f4c155a40 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f5188f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c2bb0 con 0x7f4f4c14f790 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f4b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f4c155a40 0x7f4f4c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f4b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f4c155a40 0x7f4f4c155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37042/0 (socket says 172.21.15.17:37042) 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f4b7fe640 1 -- 172.21.15.17:0/1950894378 learned_addr learned my addr 172.21.15.17:0/1950894378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.323+0000 7f4f4affd640 1 --2- 172.21.15.17:0/1950894378 >> v2:172.21.15.163:3300/0 conn(0x7f4f4c14f790 0x7f4f4c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.324+0000 7f4f4b7fe640 1 -- 172.21.15.17:0/1950894378 >> v2:172.21.15.163:3300/0 conn(0x7f4f4c14f790 msgr2=0x7f4f4c1529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.324+0000 7f4f4b7fe640 1 --2- 172.21.15.17:0/1950894378 >> v2:172.21.15.163:3300/0 conn(0x7f4f4c14f790 0x7f4f4c1529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.324+0000 7f4f4b7fe640 1 -- 172.21.15.17:0/1950894378 >> v1:172.21.15.163:6789/0 conn(0x7f4f4c0c1aa0 legacy=0x7f4f4c0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.324+0000 7f4f4b7fe640 1 -- 172.21.15.17:0/1950894378 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0c2e90 con 0x7f4f4c155a40 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.324+0000 7f4f4b7fe640 1 --2- 172.21.15.17:0/1950894378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c155a40 0x7f4f4c155e50 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0c3170 tx=0x7f4f4402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af1a7b012aa3839 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f49ffb640 1 -- 172.21.15.17:0/1950894378 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4402cb20 con 0x7f4f4c155a40 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f49ffb640 1 -- 172.21.15.17:0/1950894378 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4402ccc0 con 0x7f4f4c155a40 2024-03-06T07:26:18.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f49ffb640 1 -- 172.21.15.17:0/1950894378 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44034610 con 0x7f4f4c155a40 2024-03-06T07:26:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 -- 172.21.15.17:0/1950894378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c155a40 msgr2=0x7f4f4c155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 --2- 172.21.15.17:0/1950894378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c155a40 0x7f4f4c155e50 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0c3170 tx=0x7f4f4402bf80 comp rx=0 tx=0).stop 2024-03-06T07:26:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 -- 172.21.15.17:0/1950894378 shutdown_connections 2024-03-06T07:26:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 --2- 172.21.15.17:0/1950894378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c155a40 0x7f4f4c155e50 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 --2- 172.21.15.17:0/1950894378 >> v2:172.21.15.163:3300/0 conn(0x7f4f4c14f790 0x7f4f4c1529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 -- 172.21.15.17:0/1950894378 >> 172.21.15.17:0/1950894378 conn(0x7f4f4c0c6710 msgr2=0x7f4f4c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 -- 172.21.15.17:0/1950894378 shutdown_connections 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.325+0000 7f4f5188f640 1 -- 172.21.15.17:0/1950894378 wait complete. 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.326+0000 7f4f5188f640 1 Processor -- start 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.326+0000 7f4f5188f640 1 -- start start 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f4c0c1aa0 0x7f4f4c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f4c155a40 0x7f4f4c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c0d60 con 0x7f4f4c14f790 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c156390 con 0x7f4f4c155a40 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f5188f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0c5e30 con 0x7f4f4c0c1aa0 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37052/0 (socket says 172.21.15.17:37052) 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f4a7fc640 1 -- 172.21.15.17:0/2262135183 learned_addr learned my addr 172.21.15.17:0/2262135183 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:18.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f4affd640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f4c0c1aa0 0x7f4f4c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.327+0000 7f4f4b7fe640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f4c155a40 0x7f4f4c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 -- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f4c0c1aa0 msgr2=0x7f4f4c1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f4c0c1aa0 0x7f4f4c1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 -- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f4c155a40 msgr2=0x7f4f4c14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f4c155a40 0x7f4f4c14ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 -- 172.21.15.17:0/2262135183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c1eb7a0 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f4a7fc640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c6a0 tx=0x7f4f4000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f400052d0 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f40005490 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40001940 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c1eba80 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.328+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c1516f0 con 0x7f4f4c14f790 2024-03-06T07:26:18.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.329+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c04f330 con 0x7f4f4c14f790 2024-03-06T07:26:18.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.329+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f40001ae0 con 0x7f4f4c14f790 2024-03-06T07:26:18.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.330+0000 7f4f33fff640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 0x7f4f3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.330+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f4f4008cb50 con 0x7f4f4c14f790 2024-03-06T07:26:18.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.330+0000 7f4f4affd640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 0x7f4f3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.330+0000 7f4f4affd640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 0x7f4f3806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c1eb6d0 tx=0x7f4f3c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.334+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f40015030 con 0x7f4f4c14f790 2024-03-06T07:26:18.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.515+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f4c0ab5c0 con 0x7f4f4c14f790 2024-03-06T07:26:18.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.516+0000 7f4f33fff640 1 -- 172.21.15.17:0/2262135183 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20857 (secure 0 0 0) 0x7f4f4005adf0 con 0x7f4f4c14f790 2024-03-06T07:26:18.518 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:18.518 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":156,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:10.717640+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:18.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 msgr2=0x7f4f3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 0x7f4f3806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c1eb6d0 tx=0x7f4f3c009190 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 msgr2=0x7f4f4c14b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000c6a0 tx=0x7f4f4000cb60 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 shutdown_connections 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f3806b4d0 0x7f4f3806d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f4c155a40 0x7f4f4c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f4c14f790 0x7f4f4c14b9a0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 --2- 172.21.15.17:0/2262135183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f4c0c1aa0 0x7f4f4c1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 >> 172.21.15.17:0/2262135183 conn(0x7f4f4c0c6710 msgr2=0x7f4f4c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 shutdown_connections 2024-03-06T07:26:18.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.518+0000 7f4f5188f640 1 -- 172.21.15.17:0/2262135183 wait complete. 2024-03-06T07:26:18.531 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:26:18.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.631+0000 7fc1a5be9640 1 Processor -- start 2024-03-06T07:26:18.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.631+0000 7fc1a5be9640 1 -- start start 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc1a5be9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1a00c6780 0x7fc1a0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc1a5be9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a0072630 con 0x7fc1a01592d0 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc1a5be9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a0072910 con 0x7fc1a006da40 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc1a5be9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a006de20 con 0x7fc1a00c6780 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1a01592d0 0x7fc1a00c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35074/0 (socket says 172.21.15.17:35074) 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19ffff640 1 -- 172.21.15.17:0/2934941104 learned_addr learned my addr 172.21.15.17:0/2934941104 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19effd640 1 --2- 172.21.15.17:0/2934941104 >> v2:172.21.15.163:3300/0 conn(0x7fc1a00c6780 0x7fc1a0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19effd640 1 -- 172.21.15.17:0/2934941104 >> v1:172.21.15.163:6789/0 conn(0x7fc1a006da40 legacy=0x7fc1a0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19effd640 1 -- 172.21.15.17:0/2934941104 >> v1:172.21.15.17:6789/0 conn(0x7fc1a01592d0 legacy=0x7fc1a00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19effd640 1 -- 172.21.15.17:0/2934941104 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0072bf0 con 0x7fc1a00c6780 2024-03-06T07:26:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.632+0000 7fc19e7fc640 1 -- 172.21.15.17:0/2934941104 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212258910 0 0) 0x7fc1a0072630 con 0x7fc1a01592d0 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.633+0000 7fc19effd640 1 --2- 172.21.15.17:0/2934941104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a00c6780 0x7fc1a0158d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097a0 tx=0x7fc19402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ccc8ceedf4a1e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.633+0000 7fc19e7fc640 1 -- 172.21.15.17:0/2934941104 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194035020 con 0x7fc1a00c6780 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.633+0000 7fc19e7fc640 1 -- 172.21.15.17:0/2934941104 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194031830 con 0x7fc1a00c6780 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.633+0000 7fc19e7fc640 1 -- 172.21.15.17:0/2934941104 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194031c00 con 0x7fc1a00c6780 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2934941104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a00c6780 msgr2=0x7fc1a0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2934941104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a00c6780 0x7fc1a0158d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc1940097a0 tx=0x7fc19402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2934941104 shutdown_connections 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2934941104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a00c6780 0x7fc1a0158d90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2934941104 >> 172.21.15.17:0/2934941104 conn(0x7fc1a00aa630 msgr2=0x7fc1a00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2934941104 shutdown_connections 2024-03-06T07:26:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.634+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2934941104 wait complete. 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 Processor -- start 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 -- start start 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a006da40 0x7fc1a01edc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a01592d0 0x7fc1a00ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00c3b10 con 0x7fc1a00c6780 2024-03-06T07:26:18.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.635+0000 7fc1a5be9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00c3990 con 0x7fc1a01592d0 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc1a5be9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00c7380 con 0x7fc1a006da40 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37056/0 (socket says 172.21.15.17:37056) 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 -- 172.21.15.17:0/2330153814 learned_addr learned my addr 172.21.15.17:0/2330153814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19f7fe640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a006da40 0x7fc1a01edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19ffff640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a01592d0 0x7fc1a00ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 -- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a006da40 msgr2=0x7fc1a01edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a006da40 0x7fc1a01edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 -- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a01592d0 msgr2=0x7fc1a00ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a01592d0 0x7fc1a00ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 -- 172.21.15.17:0/2330153814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00ae6e0 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc19effd640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098c0 tx=0x7fc1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194041070 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19403aa30 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a00ab8c0 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194035040 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.636+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00ae120 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.637+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a004f330 con 0x7fc1a00c6780 2024-03-06T07:26:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.638+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc19404c060 con 0x7fc1a00c6780 2024-03-06T07:26:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.638+0000 7fc187fff640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 0x7fc18c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.638+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fc1940ac0f0 con 0x7fc1a00c6780 2024-03-06T07:26:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.638+0000 7fc19f7fe640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 0x7fc18c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.639+0000 7fc19f7fe640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 0x7fc18c06d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0152950 tx=0x7fc190009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.642+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc19407a390 con 0x7fc1a00c6780 2024-03-06T07:26:18.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.865+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1a00a9b20 con 0x7fc1a00c6780 2024-03-06T07:26:18.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.866+0000 7fc187fff640 1 -- 172.21.15.17:0/2330153814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7fc19407e430 con 0x7fc1a00c6780 2024-03-06T07:26:18.867 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:18.867 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:18.868 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 msgr2=0x7fc18c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 0x7fc18c06d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0152950 tx=0x7fc190009040 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 msgr2=0x7fc1a00a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fc1940098c0 tx=0x7fc1940046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 shutdown_connections 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc18c06b4d0 0x7fc18c06d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1a01592d0 0x7fc1a00ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a00c6780 0x7fc1a00a95e0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 --2- 172.21.15.17:0/2330153814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1a006da40 0x7fc1a01edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 >> 172.21.15.17:0/2330153814 conn(0x7fc1a00aa630 msgr2=0x7fc1a00c58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 shutdown_connections 2024-03-06T07:26:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.868+0000 7fc1a5be9640 1 -- 172.21.15.17:0/2330153814 wait complete. 2024-03-06T07:26:18.880 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 Processor -- start 2024-03-06T07:26:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 -- start start 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d0c0c5330 0x7f3d0c0c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0c155110 con 0x7f3d0c158db0 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0c0c8f40 con 0x7f3d0c0c88b0 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.980+0000 7f3d13b02640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c1559e0 con 0x7f3d0c0c5330 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1209f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3d0c158db0 0x7f3d0c15fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35080/0 (socket says 172.21.15.17:35080) 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1209f640 1 -- 172.21.15.17:0/3454888042 learned_addr learned my addr 172.21.15.17:0/3454888042 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1189e640 1 --2- 172.21.15.17:0/3454888042 >> v2:172.21.15.163:3300/0 conn(0x7f3d0c0c5330 0x7f3d0c0c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1089c640 1 -- 172.21.15.17:0/3454888042 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202328619 0 0) 0x7f3d04001490 con 0x7f3d0c158db0 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1089c640 1 -- 172.21.15.17:0/3454888042 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf8003520 con 0x7f3d0c158db0 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1189e640 1 -- 172.21.15.17:0/3454888042 >> v1:172.21.15.163:6789/0 conn(0x7f3d0c0c88b0 legacy=0x7f3d0c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1189e640 1 -- 172.21.15.17:0/3454888042 >> v1:172.21.15.17:6789/0 conn(0x7f3d0c158db0 legacy=0x7f3d0c15fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.981+0000 7f3d1189e640 1 -- 172.21.15.17:0/3454888042 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c155390 con 0x7f3d0c0c5330 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d1189e640 1 --2- 172.21.15.17:0/3454888042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c0c5710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097a0 tx=0x7f3d0802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2346a6f95c4e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d1089c640 1 -- 172.21.15.17:0/3454888042 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08035020 con 0x7f3d0c0c5330 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d1089c640 1 -- 172.21.15.17:0/3454888042 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08031830 con 0x7f3d0c0c5330 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d1089c640 1 -- 172.21.15.17:0/3454888042 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08031be0 con 0x7f3d0c0c5330 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d13b02640 1 -- 172.21.15.17:0/3454888042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c5330 msgr2=0x7f3d0c0c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.982+0000 7f3d13b02640 1 --2- 172.21.15.17:0/3454888042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c0c5710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097a0 tx=0x7f3d0802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.983+0000 7f3d13b02640 1 -- 172.21.15.17:0/3454888042 shutdown_connections 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.983+0000 7f3d13b02640 1 --2- 172.21.15.17:0/3454888042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c0c5710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.983+0000 7f3d13b02640 1 -- 172.21.15.17:0/3454888042 >> 172.21.15.17:0/3454888042 conn(0x7f3d0c0ac560 msgr2=0x7f3d0c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.983+0000 7f3d13b02640 1 -- 172.21.15.17:0/3454888042 shutdown_connections 2024-03-06T07:26:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.983+0000 7f3d13b02640 1 -- 172.21.15.17:0/3454888042 wait complete. 2024-03-06T07:26:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 Processor -- start 2024-03-06T07:26:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 -- start start 2024-03-06T07:26:18.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 0x7f3d0c0c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d0c1eac60 0x7f3d0c0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1529d0 con 0x7f3d0c0c5330 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c06d090 con 0x7f3d0c0c88b0 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d13b02640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c152b50 con 0x7f3d0c1eac60 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d1189e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d1189e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37060/0 (socket says 172.21.15.17:37060) 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.984+0000 7f3d1189e640 1 -- 172.21.15.17:0/168222949 learned_addr learned my addr 172.21.15.17:0/168222949 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1209f640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d0c1eac60 0x7f3d0c0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1109d640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 0x7f3d0c0c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 -- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d0c1eac60 msgr2=0x7f3d0c0c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d0c1eac60 0x7f3d0c0c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 -- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 msgr2=0x7f3d0c0c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 0x7f3d0c0c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 -- 172.21.15.17:0/168222949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c14ded0 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1109d640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 0x7f3d0c0c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d1189e640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097a0 tx=0x7f3d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08035070 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08004230 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c14e1b0 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0803a630 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.985+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c14e770 con 0x7f3d0c0c5330 2024-03-06T07:26:18.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.986+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c154f30 con 0x7f3d0c0c5330 2024-03-06T07:26:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.987+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d08038020 con 0x7f3d0c0c5330 2024-03-06T07:26:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.987+0000 7f3d027fc640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.987+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f3d080ac330 con 0x7f3d0c0c5330 2024-03-06T07:26:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.987+0000 7f3d1109d640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:18.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.988+0000 7f3d1109d640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 0x7f3cf006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0ca350 tx=0x7f3cfc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:18.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:18.991+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3d0c154f30 con 0x7f3d0c0c5330 2024-03-06T07:26:19.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.213+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d0c04f330 con 0x7f3d0c0c5330 2024-03-06T07:26:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.213+0000 7f3d027fc640 1 -- 172.21.15.17:0/168222949 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+6276 (secure 0 0 0) 0x7f3d08040070 con 0x7f3d0c0c5330 2024-03-06T07:26:19.215 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:19.215 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":146,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9090,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:25:54.526748+0000","modified":"2024-03-06T07:26:13.724498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":155,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8203},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8203":{"gid":8203,"name":"b","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6837/2502302602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":2502302602},{"type":"v1","addr":"172.21.15.163:6837","nonce":2502302602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8203,"qdb_cluster":[8203]},"id":11},{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:13.724499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":156,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8200":{"gid":8200,"name":"d","rank":0,"incarnation":143,"state":"up:active","state_seq":9,"addr":"172.21.15.163:6836/91031835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":91031835},{"type":"v1","addr":"172.21.15.163:6836","nonce":91031835}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8200,"qdb_cluster":[8200]},"id":12}]} 2024-03-06T07:26:19.215 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 146 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 msgr2=0x7f3cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 0x7f3cf006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0ca350 tx=0x7f3cfc009040 comp rx=0 tx=0).stop 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 msgr2=0x7f3d0c157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3d080097a0 tx=0x7f3d080046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 shutdown_connections 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cf006b4d0 0x7f3cf006d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d0c1eac60 0x7f3d0c0c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d0c0c88b0 0x7f3d0c0c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 --2- 172.21.15.17:0/168222949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d0c0c5330 0x7f3d0c157c70 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 >> 172.21.15.17:0/168222949 conn(0x7f3d0c0ac560 msgr2=0x7f3d0c14f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 shutdown_connections 2024-03-06T07:26:19.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.215+0000 7f3d13b02640 1 -- 172.21.15.17:0/168222949 wait complete. 2024-03-06T07:26:19.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:19.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.327+0000 7ff5e94df640 1 Processor -- start 2024-03-06T07:26:19.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.327+0000 7ff5e94df640 1 -- start start 2024-03-06T07:26:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e94df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5e4152920 0x7ff5e4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e94df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2620 0x7ff5e40c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e94df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e406afc0 con 0x7ff5e40c2190 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e94df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e4156160 con 0x7ff5e4152920 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e94df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e406b2a0 con 0x7ff5e40c2620 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5e4152920 0x7ff5e4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5e4152920 0x7ff5e4155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37062/0 (socket says 172.21.15.17:37062) 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e2ffd640 1 -- 172.21.15.17:0/274661605 learned_addr learned my addr 172.21.15.17:0/274661605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.328+0000 7ff5e37fe640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2620 0x7ff5e40c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e2ffd640 1 -- 172.21.15.17:0/274661605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2620 msgr2=0x7ff5e40c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e2ffd640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2620 0x7ff5e40c1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e2ffd640 1 -- 172.21.15.17:0/274661605 >> v1:172.21.15.163:6789/0 conn(0x7ff5e40c2190 legacy=0x7ff5e40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e2ffd640 1 -- 172.21.15.17:0/274661605 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e406b580 con 0x7ff5e4152920 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e2ffd640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e4155c20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009870 tx=0x7ff5dc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc41a6ec4368b848 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e1ffb640 1 -- 172.21.15.17:0/274661605 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc034070 con 0x7ff5e4152920 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e1ffb640 1 -- 172.21.15.17:0/274661605 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5dc030840 con 0x7ff5e4152920 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.329+0000 7ff5e1ffb640 1 -- 172.21.15.17:0/274661605 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc030c10 con 0x7ff5e4152920 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 -- 172.21.15.17:0/274661605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 msgr2=0x7ff5e4155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e4155c20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009870 tx=0x7ff5dc02b980 comp rx=0 tx=0).stop 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 -- 172.21.15.17:0/274661605 shutdown_connections 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2620 0x7ff5e40c1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 --2- 172.21.15.17:0/274661605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e4155c20 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 -- 172.21.15.17:0/274661605 >> 172.21.15.17:0/274661605 conn(0x7ff5e40c4290 msgr2=0x7ff5e40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 -- 172.21.15.17:0/274661605 shutdown_connections 2024-03-06T07:26:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.330+0000 7ff5e94df640 1 -- 172.21.15.17:0/274661605 wait complete. 2024-03-06T07:26:19.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.331+0000 7ff5e94df640 1 Processor -- start 2024-03-06T07:26:19.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.331+0000 7ff5e94df640 1 -- start start 2024-03-06T07:26:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2190 0x7ff5e40a7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e40c2620 0x7ff5e4071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40a7100 con 0x7ff5e4152920 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40c0d60 con 0x7ff5e40c2620 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e94df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e40a7280 con 0x7ff5e40c2190 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37072/0 (socket says 172.21.15.17:37072) 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 -- 172.21.15.17:0/4261846080 learned_addr learned my addr 172.21.15.17:0/4261846080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e2ffd640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2190 0x7ff5e40a7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e27fc640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e40c2620 0x7ff5e4071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 -- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2190 msgr2=0x7ff5e40a7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2190 0x7ff5e40a7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 -- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e40c2620 msgr2=0x7ff5e4071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e40c2620 0x7ff5e4071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.332+0000 7ff5e37fe640 1 -- 172.21.15.17:0/4261846080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4073090 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5e37fe640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400efc0 tx=0x7ff5d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4018020 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4014480 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4014870 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e40abd60 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.333+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e40ac000 con 0x7ff5e4152920 2024-03-06T07:26:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.334+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e404f330 con 0x7ff5e4152920 2024-03-06T07:26:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.334+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5d4005d10 con 0x7ff5e4152920 2024-03-06T07:26:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.335+0000 7ff5db7fe640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.335+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7ff5d408b110 con 0x7ff5e4152920 2024-03-06T07:26:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.335+0000 7ff5e2ffd640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.336+0000 7ff5e2ffd640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 0x7ff5c406d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc036000 tx=0x7ff5dc03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.339+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff5d4022070 con 0x7ff5e4152920 2024-03-06T07:26:19.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.521+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e406bca0 con 0x7ff5e4152920 2024-03-06T07:26:19.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.522+0000 7ff5db7fe640 1 -- 172.21.15.17:0/4261846080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+20857 (secure 0 0 0) 0x7ff5d4010040 con 0x7ff5e4152920 2024-03-06T07:26:19.524 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:19.525 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":156,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:10.717640+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:25:52.790714+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-03-06T07:25:53.497871+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:19.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 msgr2=0x7ff5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 0x7ff5c406d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc036000 tx=0x7ff5dc03b040 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 msgr2=0x7ff5e40ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400efc0 tx=0x7ff5d400c490 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 shutdown_connections 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff5c406b4d0 0x7ff5c406d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5e4152920 0x7ff5e40ac800 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e40c2620 0x7ff5e4071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 --2- 172.21.15.17:0/4261846080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e40c2190 0x7ff5e40a7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 >> 172.21.15.17:0/4261846080 conn(0x7ff5e40c4290 msgr2=0x7ff5e40c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 shutdown_connections 2024-03-06T07:26:19.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.525+0000 7ff5e94df640 1 -- 172.21.15.17:0/4261846080 wait complete. 2024-03-06T07:26:19.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:26:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.638+0000 7fe873c64640 1 Processor -- start 2024-03-06T07:26:19.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.638+0000 7fe873c64640 1 -- start start 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe873c64640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe86c0c6ab0 0x7fe86c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe873c64640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c0a9990 con 0x7fe86c14f110 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe873c64640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c1556c0 con 0x7fe86c1561a0 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe873c64640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86c155d10 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe871a00640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe86c0c6ab0 0x7fe86c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe8711ff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe86c1561a0 0x7fe86c0a9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:41420/0 (socket says 172.21.15.17:41420) 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe8711ff640 1 -- 172.21.15.17:0/2881948893 learned_addr learned my addr 172.21.15.17:0/2881948893 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.639+0000 7fe872201640 1 --1- 172.21.15.17:0/2881948893 >> v1:172.21.15.17:6789/0 conn(0x7fe86c14f110 0x7fe86c15c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35094/0 (socket says 172.21.15.17:35094) 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766708013 0 0) 0x7fe86c0a9990 con 0x7fe86c14f110 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe858005730 con 0x7fe86c14f110 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe871a00640 1 -- 172.21.15.17:0/2881948893 >> v1:172.21.15.163:6789/0 conn(0x7fe86c1561a0 legacy=0x7fe86c0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326975386 0 0) 0x7fe86c1556c0 con 0x7fe86c1561a0 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe871a00640 1 -- 172.21.15.17:0/2881948893 >> v1:172.21.15.17:6789/0 conn(0x7fe86c14f110 legacy=0x7fe86c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe871a00640 1 -- 172.21.15.17:0/2881948893 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c1559a0 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.640+0000 7fe871a00640 1 --2- 172.21.15.17:0/2881948893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c0c6ab0 0x7fe86c1529e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe868009d90 tx=0x7fe86802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5adae1c8c4d082e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868039070 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86802cc60 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe8709fe640 1 -- 172.21.15.17:0/2881948893 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868034660 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 -- 172.21.15.17:0/2881948893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c0c6ab0 msgr2=0x7fe86c1529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 --2- 172.21.15.17:0/2881948893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c0c6ab0 0x7fe86c1529e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe868009d90 tx=0x7fe86802beb0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 -- 172.21.15.17:0/2881948893 shutdown_connections 2024-03-06T07:26:19.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 --2- 172.21.15.17:0/2881948893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c0c6ab0 0x7fe86c1529e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 -- 172.21.15.17:0/2881948893 >> 172.21.15.17:0/2881948893 conn(0x7fe86c06b1a0 msgr2=0x7fe86c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 -- 172.21.15.17:0/2881948893 shutdown_connections 2024-03-06T07:26:19.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.641+0000 7fe873c64640 1 -- 172.21.15.17:0/2881948893 wait complete. 2024-03-06T07:26:19.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 Processor -- start 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 -- start start 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe86c0c6ab0 0x7fe86c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c14f110 0x7fe86c1467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0aa600 con 0x7fe86c1561a0 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0c2a90 con 0x7fe86c14f110 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe873c64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c1534e0 con 0x7fe86c0c6ab0 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe872201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe872201640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37080/0 (socket says 172.21.15.17:37080) 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe8711ff640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c14f110 0x7fe86c1467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe872201640 1 -- 172.21.15.17:0/1911401855 learned_addr learned my addr 172.21.15.17:0/1911401855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.643+0000 7fe871a00640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe86c0c6ab0 0x7fe86c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 -- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe86c0c6ab0 msgr2=0x7fe86c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe86c0c6ab0 0x7fe86c14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 -- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c14f110 msgr2=0x7fe86c1467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c14f110 0x7fe86c1467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 -- 172.21.15.17:0/1911401855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0a8d80 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe872201640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c430 tx=0x7fe86400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640074e0 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864007680 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c1e6ea0 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864004a20 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.644+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c14e3a0 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.645+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c070e80 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.645+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe86401a020 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.646+0000 7fe8627fc640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.646+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fe864089e70 con 0x7fe86c1561a0 2024-03-06T07:26:19.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.646+0000 7fe871a00640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.647+0000 7fe871a00640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 0x7fe85006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe868036000 tx=0x7fe868030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.650+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe86c070e80 con 0x7fe86c1561a0 2024-03-06T07:26:19.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.871+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe86c04f330 con 0x7fe86c1561a0 2024-03-06T07:26:19.873 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:26:19.872+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:19.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.873+0000 7fe8627fc640 1 -- 172.21.15.17:0/1911401855 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v147) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe864058100 con 0x7fe86c1561a0 2024-03-06T07:26:19.874 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:26:19.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 msgr2=0x7fe85006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 0x7fe85006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe868036000 tx=0x7fe868030000 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 msgr2=0x7fe86c146ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c430 tx=0x7fe86400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 shutdown_connections 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe86c1561a0 0x7fe86c146ce0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe86c14f110 0x7fe86c1467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 --2- 172.21.15.17:0/1911401855 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe86c0c6ab0 0x7fe86c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 >> 172.21.15.17:0/1911401855 conn(0x7fe86c06b1a0 msgr2=0x7fe86c148c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 shutdown_connections 2024-03-06T07:26:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.876+0000 7fe873c64640 1 -- 172.21.15.17:0/1911401855 wait complete. 2024-03-06T07:26:19.880 INFO:tasks.ceph.mds.b.smithi163.stderr: -146> 2024-03-06T07:26:15.448+0000 7f8629c23640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:19.888 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:26:19.905 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:26:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 Processor -- start 2024-03-06T07:26:19.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 -- start start 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 0x7f70580c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70580c58b0 0x7f7058155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70581596f0 con 0x7f70580c88f0 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70580c9130 con 0x7f70580c58b0 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.992+0000 7f705ca2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7058159260 con 0x7f70581527f0 2024-03-06T07:26:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70580c58b0 0x7f7058155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70580c58b0 0x7f7058155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37086/0 (socket says 172.21.15.17:37086) 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 -- 172.21.15.17:0/151169652 learned_addr learned my addr 172.21.15.17:0/151169652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f705659c640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 0x7f70580c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 -- 172.21.15.17:0/151169652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 msgr2=0x7f70580c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 0x7f70580c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 -- 172.21.15.17:0/151169652 >> v1:172.21.15.163:6789/0 conn(0x7f70580c88f0 legacy=0x7f7058155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7055d9b640 1 -- 172.21.15.17:0/151169652 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116251981 0 0) 0x7f70581596f0 con 0x7f70580c88f0 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f7056d9d640 1 -- 172.21.15.17:0/151169652 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70580c82a0 con 0x7f70580c58b0 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.993+0000 7f705659c640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 0x7f70580c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f7056d9d640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70580c58b0 0x7f7058155920 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7050009e50 tx=0x7f705002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996f8e8d70340655 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f7055d9b640 1 -- 172.21.15.17:0/151169652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705002c9e0 con 0x7f70580c58b0 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f7055d9b640 1 -- 172.21.15.17:0/151169652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705002cb80 con 0x7f70580c58b0 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f7055d9b640 1 -- 172.21.15.17:0/151169652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70500343f0 con 0x7f70580c58b0 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 -- 172.21.15.17:0/151169652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70580c58b0 msgr2=0x7f7058155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70580c58b0 0x7f7058155920 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f7050009e50 tx=0x7f705002be40 comp rx=0 tx=0).stop 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 -- 172.21.15.17:0/151169652 shutdown_connections 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70580c58b0 0x7f7058155920 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 --2- 172.21.15.17:0/151169652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70581527f0 0x7f70580c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 -- 172.21.15.17:0/151169652 >> 172.21.15.17:0/151169652 conn(0x7f70580ac580 msgr2=0x7f70580ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 -- 172.21.15.17:0/151169652 shutdown_connections 2024-03-06T07:26:19.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.994+0000 7f705ca2a640 1 -- 172.21.15.17:0/151169652 wait complete. 2024-03-06T07:26:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 Processor -- start 2024-03-06T07:26:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 -- start start 2024-03-06T07:26:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 0x7f70581490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70580c88f0 0x7f70581495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7058158c10 con 0x7f70581527f0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70580c6b90 con 0x7f70580c88f0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705ca2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7058158ef0 con 0x7f70580c58b0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f7056d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f7056d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37100/0 (socket says 172.21.15.17:37100) 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f7056d9d640 1 -- 172.21.15.17:0/58189386 learned_addr learned my addr 172.21.15.17:0/58189386 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f704ffff640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70580c88f0 0x7f70581495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.996+0000 7f705659c640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 0x7f70581490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 -- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 msgr2=0x7f70581490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 0x7f70581490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 -- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70580c88f0 msgr2=0x7f70581495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70580c88f0 0x7f70581495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 -- 172.21.15.17:0/58189386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70580c4490 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f705659c640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 0x7f70581490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f7056d9d640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7050009e50 tx=0x7f7050004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050043020 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705003e450 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050009510 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70580c4700 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.997+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70580aa780 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.998+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70580c8e50 con 0x7f70581527f0 2024-03-06T07:26:20.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.998+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7050046020 con 0x7f70581527f0 2024-03-06T07:26:20.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.999+0000 7f704effd640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 0x7f703806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:20.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.999+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f70500ace20 con 0x7f70581527f0 2024-03-06T07:26:20.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.999+0000 7f705659c640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 0x7f703806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:20.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:19.999+0000 7f705659c640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 0x7f703806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7048009af0 tx=0x7f7048008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:20.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.003+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70580c8e50 con 0x7f70581527f0 2024-03-06T07:26:20.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.221+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7058075300 con 0x7f70581527f0 2024-03-06T07:26:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.874+0000 7f704effd640 1 -- 172.21.15.17:0/58189386 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v148) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7058075300 con 0x7f70581527f0 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 msgr2=0x7f703806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 0x7f703806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7048009af0 tx=0x7f7048008040 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 msgr2=0x7f70581ee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7050009e50 tx=0x7f7050004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 shutdown_connections 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f703806b4d0 0x7f703806d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70581527f0 0x7f70581ee490 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70580c88f0 0x7f70581495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 --2- 172.21.15.17:0/58189386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70580c58b0 0x7f70581490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:20.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 >> 172.21.15.17:0/58189386 conn(0x7f70580ac580 msgr2=0x7f70580ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:20.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 shutdown_connections 2024-03-06T07:26:20.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:20.878+0000 7f705ca2a640 1 -- 172.21.15.17:0/58189386 wait complete. 2024-03-06T07:26:20.886 INFO:tasks.ceph.mds.d.smithi163.stderr: -216> 2024-03-06T07:26:15.448+0000 7f932869b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:20.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:26:20.910 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:26:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.007+0000 7f8395ad4640 1 Processor -- start 2024-03-06T07:26:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.007+0000 7f8395ad4640 1 -- start start 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f8395ad4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8390152f80 0x7f83900c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f8395ad4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8390156b50 con 0x7f8390156570 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f8395ad4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8390156e30 con 0x7f8390149a90 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f8395ad4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8390149e70 con 0x7f8390152f80 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8390152f80 0x7f83900c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8390152f80 0x7f83900c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59258/0 (socket says 172.21.15.17:59258) 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8390156570 0x7f83900c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48434/0 (socket says 172.21.15.17:48434) 2024-03-06T07:26:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 -- 172.21.15.17:0/3245733978 learned_addr learned my addr 172.21.15.17:0/3245733978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 -- 172.21.15.17:0/3245733978 >> v1:172.21.15.163:6789/0 conn(0x7f8390149a90 legacy=0x7f8390072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 -- 172.21.15.17:0/3245733978 >> v1:172.21.15.17:6789/0 conn(0x7f8390156570 legacy=0x7f83900c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.008+0000 7f838ffff640 1 -- 172.21.15.17:0/3245733978 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390157110 con 0x7f8390152f80 2024-03-06T07:26:21.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.009+0000 7f838ffff640 1 --2- 172.21.15.17:0/3245733978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c3f50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f8388009e50 tx=0x7f838802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38b6c9c1ee3ee0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.009+0000 7f838e7fc640 1 -- 172.21.15.17:0/3245733978 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838802cb20 con 0x7f8390152f80 2024-03-06T07:26:21.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.009+0000 7f838e7fc640 1 -- 172.21.15.17:0/3245733978 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838802ccc0 con 0x7f8390152f80 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.009+0000 7f838e7fc640 1 -- 172.21.15.17:0/3245733978 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388034610 con 0x7f8390152f80 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 -- 172.21.15.17:0/3245733978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 msgr2=0x7f83900c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3245733978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c3f50 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f8388009e50 tx=0x7f838802bf80 comp rx=0 tx=0).stop 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 -- 172.21.15.17:0/3245733978 shutdown_connections 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3245733978 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c3f50 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 -- 172.21.15.17:0/3245733978 >> 172.21.15.17:0/3245733978 conn(0x7f83900aa630 msgr2=0x7f83900aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 -- 172.21.15.17:0/3245733978 shutdown_connections 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.010+0000 7f8395ad4640 1 -- 172.21.15.17:0/3245733978 wait complete. 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.011+0000 7f8395ad4640 1 Processor -- start 2024-03-06T07:26:21.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.011+0000 7f8395ad4640 1 -- start start 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8390149a90 0x7f8390152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 0x7f8390144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c33b0 con 0x7f8390152f80 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8390153f00 con 0x7f8390156570 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f8395ad4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900c4490 con 0x7f8390149a90 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59270/0 (socket says 172.21.15.17:59270) 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 -- 172.21.15.17:0/3375269913 learned_addr learned my addr 172.21.15.17:0/3375269913 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838f7fe640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8390149a90 0x7f8390152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838ffff640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 0x7f8390144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 -- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8390149a90 msgr2=0x7f8390152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8390149a90 0x7f8390152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 -- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 msgr2=0x7f8390144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 0x7f8390144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838effd640 1 -- 172.21.15.17:0/3375269913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83901e9640 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.012+0000 7f838ffff640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 0x7f8390144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f838effd640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f838400c3c0 tx=0x7f838400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840074e0 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384007680 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901475b0 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384004a00 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.013+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83901e98e0 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.014+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83900c4dc0 con 0x7f8390152f80 2024-03-06T07:26:21.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.014+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f838401a020 con 0x7f8390152f80 2024-03-06T07:26:21.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.015+0000 7f8377fff640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:21.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.015+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 7301+0+0 (secure 0 0 0) 0x7f8384089e80 con 0x7f8390152f80 2024-03-06T07:26:21.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.015+0000 7f838f7fe640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:21.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.015+0000 7f838f7fe640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f83901e6a70 tx=0x7f8380007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:21.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.019+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83900c4dc0 con 0x7f8390152f80 2024-03-06T07:26:21.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.203+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83900c3e70 con 0x7f8390152f80 2024-03-06T07:26:21.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.883+0000 7f8377fff640 1 -- 172.21.15.17:0/3375269913 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v159) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8384058750 con 0x7f8390152f80 2024-03-06T07:26:21.885 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 msgr2=0x7f837c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 0x7f837c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f83901e6a70 tx=0x7f8380007e30 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 msgr2=0x7f83900c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f838400c3c0 tx=0x7f838400c880 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 shutdown_connections 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f837c06b4d0 0x7f837c06d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8390156570 0x7f8390144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8390152f80 0x7f83900c6420 secure :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f838400c3c0 tx=0x7f838400c880 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 --2- 172.21.15.17:0/3375269913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8390149a90 0x7f8390152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:21.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.887+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 >> 172.21.15.17:0/3375269913 conn(0x7f83900aa630 msgr2=0x7f83900a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:21.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.888+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 shutdown_connections 2024-03-06T07:26:21.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:21.888+0000 7f8395ad4640 1 -- 172.21.15.17:0/3375269913 wait complete. 2024-03-06T07:26:21.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:26:22.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.013+0000 7f74431ac640 1 Processor -- start 2024-03-06T07:26:22.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.013+0000 7f74431ac640 1 -- start start 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f74431ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f743c1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f74431ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f743c0c2920 0x7f743c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f74431ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c070430 con 0x7f743c0c1d10 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f74431ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c0711f0 con 0x7f743c0c2920 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f74431ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c070d60 con 0x7f743c0abb60 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f743bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f743c0c1d10 0x7f743c156440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48448/0 (socket says 172.21.15.17:48448) 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f743bfff640 1 -- 172.21.15.17:0/879771228 learned_addr learned my addr 172.21.15.17:0/879771228 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873103737 0 0) 0x7f743c070430 con 0x7f743c0c1d10 2024-03-06T07:26:22.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7428003520 con 0x7f743c0c1d10 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f7441749640 1 --2- 172.21.15.17:0/879771228 >> v2:172.21.15.163:3300/0 conn(0x7f743c0c2920 0x7f743c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.014+0000 7f7440f48640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f743c1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226606709 0 0) 0x7f7428003520 con 0x7f743c0c1d10 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 msgr2=0x7f743c1502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f743c1502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 >> v2:172.21.15.163:3300/0 conn(0x7f743c0c2920 msgr2=0x7f743c156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 --2- 172.21.15.17:0/879771228 >> v2:172.21.15.163:3300/0 conn(0x7f743c0c2920 0x7f743c156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c070710 con 0x7f743c0c1d10 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f742c001670 con 0x7f743c0c1d10 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 >> v1:172.21.15.17:6789/0 conn(0x7f743c0c1d10 legacy=0x7f743c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74280039b0 0x7f7428003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f74280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c070d60 con 0x7f74280039b0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742800b9c0 con 0x7f7428004800 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742800bb40 con 0x7f743c0abb60 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74280039b0 0x7f7428003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f743bfff640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7441749640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f74280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 -- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 msgr2=0x7f74280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f74280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 -- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7428004800 msgr2=0x7f7428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7428004800 0x7f7428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7440f48640 1 -- 172.21.15.17:0/879771228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428003520 con 0x7f74280039b0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.015+0000 7f7441749640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c0abb60 0x7f74280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.016+0000 7f7440f48640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74280039b0 0x7f7428003dd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f743400e170 tx=0x7f7428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcd56ec44d8956 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.016+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434039070 con 0x7f74280039b0 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.016+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743402cbf0 con 0x7f74280039b0 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.016+0000 7f743b7fe640 1 -- 172.21.15.17:0/879771228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434009c10 con 0x7f74280039b0 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 -- 172.21.15.17:0/879771228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74280039b0 msgr2=0x7f7428003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 --2- 172.21.15.17:0/879771228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74280039b0 0x7f7428003dd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f743400e170 tx=0x7f7428005830 comp rx=0 tx=0).stop 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f7441749640 1 -- 172.21.15.17:0/879771228 reap_dead start 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 -- 172.21.15.17:0/879771228 shutdown_connections 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 -- 172.21.15.17:0/879771228 >> 172.21.15.17:0/879771228 conn(0x7f743c0c65b0 msgr2=0x7f743c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 -- 172.21.15.17:0/879771228 shutdown_connections 2024-03-06T07:26:22.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.017+0000 7f74431ac640 1 -- 172.21.15.17:0/879771228 wait complete. 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.018+0000 7f74431ac640 1 Processor -- start 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.018+0000 7f74431ac640 1 -- start start 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74280039b0 0x7f743c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c143b20 0x7f743c14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0abf40 con 0x7f743c14c150 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c157090 con 0x7f74280039b0 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f74431ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0ac0c0 con 0x7f743c143b20 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59292/0 (socket says 172.21.15.17:59292) 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 -- 172.21.15.17:0/886553043 learned_addr learned my addr 172.21.15.17:0/886553043 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f743bfff640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c143b20 0x7f743c14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7440f48640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74280039b0 0x7f743c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 -- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c143b20 msgr2=0x7f743c14dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c143b20 0x7f743c14dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 -- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74280039b0 msgr2=0x7f743c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74280039b0 0x7f743c150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 -- 172.21.15.17:0/886553043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c149880 con 0x7f743c14c150 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.019+0000 7f7441749640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f743c0c2dd0 tx=0x7f7424009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.020+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7424003610 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.020+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742401b070 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.020+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c14fd70 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.020+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240039f0 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.020+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c14fa90 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.021+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c04f330 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.021+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7424003b90 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.021+0000 7f74397fa640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.021+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6863+0+0 (secure 0 0 0) 0x7f742408a1f0 con 0x7f743c14c150 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.022+0000 7f7440f48640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.022+0000 7f7440f48640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7434036000 tx=0x7f743402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:22.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.026+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7424024020 con 0x7f743c14c150 2024-03-06T07:26:22.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.207+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f743c152a70 con 0x7f743c14c150 2024-03-06T07:26:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.884+0000 7f74397fa640 1 -- 172.21.15.17:0/886553043 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v160) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7424058bf0 con 0x7f743c14c150 2024-03-06T07:26:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:26:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 msgr2=0x7f741c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 0x7f741c06d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7434036000 tx=0x7f743402e000 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 msgr2=0x7f743c14b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f743c0c2dd0 tx=0x7f7424009f90 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 shutdown_connections 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f741c06b4d0 0x7f741c06d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f743c14c150 0x7f743c14b0a0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f743c143b20 0x7f743c14dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 --2- 172.21.15.17:0/886553043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f74280039b0 0x7f743c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 >> 172.21.15.17:0/886553043 conn(0x7f743c0c65b0 msgr2=0x7f743c0c0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 shutdown_connections 2024-03-06T07:26:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:22.889+0000 7f74431ac640 1 -- 172.21.15.17:0/886553043 wait complete. 2024-03-06T07:26:22.906 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:23.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 Processor -- start 2024-03-06T07:26:23.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 -- start start 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f00c6780 0x7f42f0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 0x7f42f00c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f006de20 con 0x7f42f00c6780 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f0156050 con 0x7f42f01592d0 2024-03-06T07:26:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42f5866640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f0072630 con 0x7f42f006da40 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.016+0000 7f42e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f00c6780 0x7f42f0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f42f00c6780 0x7f42f0158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59306/0 (socket says 172.21.15.17:59306) 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 -- 172.21.15.17:0/2246413917 learned_addr learned my addr 172.21.15.17:0/2246413917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 -- 172.21.15.17:0/2246413917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 msgr2=0x7f42f0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42eeffd640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f0155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42ef7fe640 1 --2- 172.21.15.17:0/2246413917 >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 0x7f42f00c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 -- 172.21.15.17:0/2246413917 >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 msgr2=0x7f42f00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2246413917 >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 0x7f42f00c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42e7fff640 1 -- 172.21.15.17:0/2246413917 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f0072910 con 0x7f42f00c6780 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42eeffd640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.017+0000 7f42ef7fe640 1 --2- 172.21.15.17:0/2246413917 >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 0x7f42f00c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0158d90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f42f00a8360 tx=0x7f42d802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da51b19b698359e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42ee7fc640 1 -- 172.21.15.17:0/2246413917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80317d0 con 0x7f42f00c6780 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42ee7fc640 1 -- 172.21.15.17:0/2246413917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d8037070 con 0x7f42f00c6780 2024-03-06T07:26:23.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42ee7fc640 1 -- 172.21.15.17:0/2246413917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8031b90 con 0x7f42f00c6780 2024-03-06T07:26:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 -- 172.21.15.17:0/2246413917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 msgr2=0x7f42f0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0158d90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f42f00a8360 tx=0x7f42d802b870 comp rx=0 tx=0).stop 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 -- 172.21.15.17:0/2246413917 shutdown_connections 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 --2- 172.21.15.17:0/2246413917 >> v2:172.21.15.163:3300/0 conn(0x7f42f01592d0 0x7f42f00c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0158d90 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 --2- 172.21.15.17:0/2246413917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f0155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 -- 172.21.15.17:0/2246413917 >> 172.21.15.17:0/2246413917 conn(0x7f42f00aa630 msgr2=0x7f42f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 -- 172.21.15.17:0/2246413917 shutdown_connections 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.018+0000 7f42f5866640 1 -- 172.21.15.17:0/2246413917 wait complete. 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.019+0000 7f42f5866640 1 Processor -- start 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 -- start start 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f00adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f01592d0 0x7f42f006f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00c37c0 con 0x7f42f00c6780 2024-03-06T07:26:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f0158890 con 0x7f42f01592d0 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42f5866640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00c3940 con 0x7f42f006da40 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59320/0 (socket says 172.21.15.17:59320) 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 -- 172.21.15.17:0/2109126176 learned_addr learned my addr 172.21.15.17:0/2109126176 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 -- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 msgr2=0x7f42f00adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f00adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 -- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f01592d0 msgr2=0x7f42f006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f01592d0 0x7f42f006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.020+0000 7f42e7fff640 1 -- 172.21.15.17:0/2109126176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f01ebc60 con 0x7f42f00c6780 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42e7fff640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f42d802b8b0 tx=0x7f42d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8032870 con 0x7f42f00c6780 2024-03-06T07:26:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d8032a10 con 0x7f42f00c6780 2024-03-06T07:26:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8032e00 con 0x7f42f00c6780 2024-03-06T07:26:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f00ac180 con 0x7f42f00c6780 2024-03-06T07:26:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.021+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f00ae1d0 con 0x7f42f00c6780 2024-03-06T07:26:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.022+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f004f330 con 0x7f42f00c6780 2024-03-06T07:26:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.022+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42d80453f0 con 0x7f42f00c6780 2024-03-06T07:26:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.023+0000 7f42ecff9640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.023+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f42d80ac470 con 0x7f42f00c6780 2024-03-06T07:26:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.023+0000 7f42eeffd640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.024+0000 7f42eeffd640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 0x7f42d006d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006b10 tx=0x7f42e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.027+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f42d8040070 con 0x7f42f00c6780 2024-03-06T07:26:23.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.211+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f01580f0 con 0x7f42f00c6780 2024-03-06T07:26:23.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.212+0000 7f42ecff9640 1 -- 172.21.15.17:0/2109126176 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f42d807aff0 con 0x7f42f00c6780 2024-03-06T07:26:23.214 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:23.214 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":160,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:22.883804+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.163:6837/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.163:6835/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.163:6834/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 msgr2=0x7f42d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 0x7f42d006d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006b10 tx=0x7f42e0008040 comp rx=0 tx=0).stop 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 msgr2=0x7f42f0156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f42d802b8b0 tx=0x7f42d80046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 shutdown_connections 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42f01592d0 0x7f42f006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00c6780 0x7f42f0156630 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 --2- 172.21.15.17:0/2109126176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42f006da40 0x7f42f00adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 >> 172.21.15.17:0/2109126176 conn(0x7f42f00aa630 msgr2=0x7f42f00c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 shutdown_connections 2024-03-06T07:26:23.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.214+0000 7f42f5866640 1 -- 172.21.15.17:0/2109126176 wait complete. 2024-03-06T07:26:23.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs2_metadata', 'create_time': '2024-03-06T07:26:02.038846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs2_data', 'create_time': '2024-03-06T07:26:02.549840+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs2'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:23.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 Processor -- start 2024-03-06T07:26:23.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 -- start start 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 0x7f49180c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49180c57d0 0x7f4918155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4918152dd0 con 0x7f49180c5370 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4918159710 con 0x7f49180c57d0 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.330+0000 7f491f1ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4918156180 con 0x7f49181529f0 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491d749640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49180c57d0 0x7f4918155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f4917fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49180c5370 0x7f4918155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48456/0 (socket says 172.21.15.17:48456) 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f4917fff640 1 -- 172.21.15.17:0/429407289 learned_addr learned my addr 172.21.15.17:0/429407289 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491cf48640 1 --2- 172.21.15.17:0/429407289 >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 0x7f49180c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491d749640 1 -- 172.21.15.17:0/429407289 >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 msgr2=0x7f49180c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491d749640 1 --2- 172.21.15.17:0/429407289 >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 0x7f49180c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491d749640 1 -- 172.21.15.17:0/429407289 >> v1:172.21.15.17:6789/0 conn(0x7f49180c5370 legacy=0x7f4918155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f49177fe640 1 -- 172.21.15.17:0/429407289 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684219659 0 0) 0x7f4918152dd0 con 0x7f49180c5370 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491d749640 1 -- 172.21.15.17:0/429407289 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918158b00 con 0x7f49180c57d0 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.331+0000 7f491cf48640 1 --2- 172.21.15.17:0/429407289 >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 0x7f49180c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491d749640 1 --2- 172.21.15.17:0/429407289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49180c57d0 0x7f4918155c40 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f49180cba90 tx=0x7f491002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6fb739590f29fb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f49177fe640 1 -- 172.21.15.17:0/429407289 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910039070 con 0x7f49180c57d0 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f49177fe640 1 -- 172.21.15.17:0/429407289 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491002cbf0 con 0x7f49180c57d0 2024-03-06T07:26:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f49177fe640 1 -- 172.21.15.17:0/429407289 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910034440 con 0x7f49180c57d0 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 -- 172.21.15.17:0/429407289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49180c57d0 msgr2=0x7f4918155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 --2- 172.21.15.17:0/429407289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49180c57d0 0x7f4918155c40 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f49180cba90 tx=0x7f491002be40 comp rx=0 tx=0).stop 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 -- 172.21.15.17:0/429407289 shutdown_connections 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 --2- 172.21.15.17:0/429407289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49180c57d0 0x7f4918155c40 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 --2- 172.21.15.17:0/429407289 >> v2:172.21.15.163:3300/0 conn(0x7f49181529f0 0x7f49180c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 -- 172.21.15.17:0/429407289 >> 172.21.15.17:0/429407289 conn(0x7f49180ac580 msgr2=0x7f49180ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 -- 172.21.15.17:0/429407289 shutdown_connections 2024-03-06T07:26:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.332+0000 7f491f1ac640 1 -- 172.21.15.17:0/429407289 wait complete. 2024-03-06T07:26:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.333+0000 7f491f1ac640 1 Processor -- start 2024-03-06T07:26:23.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.333+0000 7f491f1ac640 1 -- start start 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49180c5370 0x7f4918148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49180c57d0 0x7f4918157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49181590c0 con 0x7f49181529f0 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180c68b0 con 0x7f49180c57d0 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491f1ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49181593a0 con 0x7f49180c5370 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491d749640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491d749640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59338/0 (socket says 172.21.15.17:59338) 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.334+0000 7f491d749640 1 -- 172.21.15.17:0/958916104 learned_addr learned my addr 172.21.15.17:0/958916104 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f4917fff640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49180c57d0 0x7f4918157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491cf48640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49180c5370 0x7f4918148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 -- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49180c5370 msgr2=0x7f4918148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49180c5370 0x7f4918148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 -- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49180c57d0 msgr2=0x7f4918157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49180c57d0 0x7f4918157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 -- 172.21.15.17:0/958916104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918148cd0 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491d749640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f491002c450 tx=0x7f4910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910039050 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910045070 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491814e3a0 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491003e830 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.335+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49181ea9f0 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.336+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4918075300 con 0x7f49181529f0 2024-03-06T07:26:23.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.336+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f491003e9d0 con 0x7f49181529f0 2024-03-06T07:26:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.337+0000 7f49157fa640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.337+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f49100ad650 con 0x7f49181529f0 2024-03-06T07:26:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.337+0000 7f491cf48640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.337+0000 7f491cf48640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b840 tx=0x7f490c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.341+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4918075300 con 0x7f49181529f0 2024-03-06T07:26:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.529+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49181eabf0 con 0x7f49181529f0 2024-03-06T07:26:23.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.529+0000 7f49157fa640 1 -- 172.21.15.17:0/958916104 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49181eabf0 con 0x7f49181529f0 2024-03-06T07:26:23.531 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:26:23.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 msgr2=0x7f48fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f490c00b840 tx=0x7f490c008040 comp rx=0 tx=0).stop 2024-03-06T07:26:23.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 msgr2=0x7f49181eaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f491002c450 tx=0x7f4910004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 shutdown_connections 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49181529f0 0x7f49181eaf10 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49180c57d0 0x7f4918157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 --2- 172.21.15.17:0/958916104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49180c5370 0x7f4918148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.531+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 >> 172.21.15.17:0/958916104 conn(0x7f49180ac580 msgr2=0x7f491814fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.532+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 shutdown_connections 2024-03-06T07:26:23.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.532+0000 7f491f1ac640 1 -- 172.21.15.17:0/958916104 wait complete. 2024-03-06T07:26:23.544 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_metadata pg_num 2024-03-06T07:26:23.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.647+0000 7fe051b83640 1 Processor -- start 2024-03-06T07:26:23.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.647+0000 7fe051b83640 1 -- start start 2024-03-06T07:26:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe051b83640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe04c1529d0 0x7fe04c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe051b83640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 0x7fe04c155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe051b83640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c152db0 con 0x7fe04c072cd0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe051b83640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c0c6dd0 con 0x7fe04c1529d0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe051b83640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c0c6940 con 0x7fe04c0c57e0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe04c1529d0 0x7fe04c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe04c1529d0 0x7fe04c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59346/0 (socket says 172.21.15.17:59346) 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 -- 172.21.15.17:0/3375746679 learned_addr learned my addr 172.21.15.17:0/3375746679 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 -- 172.21.15.17:0/3375746679 >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 msgr2=0x7fe04c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04affd640 1 --2- 172.21.15.17:0/3375746679 >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 0x7fe04c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 --2- 172.21.15.17:0/3375746679 >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 0x7fe04c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 -- 172.21.15.17:0/3375746679 >> v1:172.21.15.163:6789/0 conn(0x7fe04c072cd0 legacy=0x7fe04c155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04b7fe640 1 -- 172.21.15.17:0/3375746679 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c1590f0 con 0x7fe04c1529d0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.648+0000 7fe04affd640 1 --2- 172.21.15.17:0/3375746679 >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 0x7fe04c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.649+0000 7fe04b7fe640 1 --2- 172.21.15.17:0/3375746679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0c52a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0c7c90 tx=0x7fe03c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7899a235fa6c498f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.649+0000 7fe04a7fc640 1 -- 172.21.15.17:0/3375746679 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c038070 con 0x7fe04c1529d0 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.649+0000 7fe04a7fc640 1 -- 172.21.15.17:0/3375746679 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c02f050 con 0x7fe04c1529d0 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.649+0000 7fe04a7fc640 1 -- 172.21.15.17:0/3375746679 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c031970 con 0x7fe04c1529d0 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 -- 172.21.15.17:0/3375746679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 msgr2=0x7fe04c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 --2- 172.21.15.17:0/3375746679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0c52a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0c7c90 tx=0x7fe03c02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 -- 172.21.15.17:0/3375746679 shutdown_connections 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 --2- 172.21.15.17:0/3375746679 >> v2:172.21.15.163:3300/0 conn(0x7fe04c0c57e0 0x7fe04c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 --2- 172.21.15.17:0/3375746679 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0c52a0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 -- 172.21.15.17:0/3375746679 >> 172.21.15.17:0/3375746679 conn(0x7fe04c0aa630 msgr2=0x7fe04c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 -- 172.21.15.17:0/3375746679 shutdown_connections 2024-03-06T07:26:23.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.650+0000 7fe051b83640 1 -- 172.21.15.17:0/3375746679 wait complete. 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.651+0000 7fe051b83640 1 Processor -- start 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.651+0000 7fe051b83640 1 -- start start 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.651+0000 7fe051b83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe04c072cd0 0x7fe04c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.651+0000 7fe051b83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 0x7fe04c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c072720 con 0x7fe04c1529d0 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c155d70 con 0x7fe04c0c57e0 2024-03-06T07:26:23.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c159370 con 0x7fe04c072cd0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59352/0 (socket says 172.21.15.17:59352) 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 -- 172.21.15.17:0/979540098 learned_addr learned my addr 172.21.15.17:0/979540098 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04b7fe640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe04c072cd0 0x7fe04c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04affd640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 0x7fe04c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 -- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe04c072cd0 msgr2=0x7fe04c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe04c072cd0 0x7fe04c14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 -- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 msgr2=0x7fe04c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 0x7fe04c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 -- 172.21.15.17:0/979540098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c14fe50 con 0x7fe04c1529d0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04affd640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 0x7fe04c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe04bfff640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe04400ba10 tx=0x7fe04400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044019070 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04400cac0 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0ac7c0 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044014440 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.652+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c157300 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.653+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c157110 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.654+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe04401c070 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.654+0000 7fe033fff640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 0x7fe03806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.654+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fe04408a100 con 0x7fe04c1529d0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.654+0000 7fe04b7fe640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 0x7fe03806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.655+0000 7fe04b7fe640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 0x7fe03806d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe03c035000 tx=0x7fe03c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.658+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe044058c80 con 0x7fe04c1529d0 2024-03-06T07:26:23.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.837+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe04c04f330 con 0x7fe04c1529d0 2024-03-06T07:26:23.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.837+0000 7fe033fff640 1 -- 172.21.15.17:0/979540098 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_metadata", "var": "pg_num"}]=0 v160) v1 ==== 105+0+11 (secure 0 0 0) 0x7fe044058580 con 0x7fe04c1529d0 2024-03-06T07:26:23.839 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 msgr2=0x7fe03806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 0x7fe03806d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe03c035000 tx=0x7fe03c008030 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 msgr2=0x7fe04c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fe04400ba10 tx=0x7fe04400bed0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 shutdown_connections 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe03806b500 0x7fe03806d9c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe04c1529d0 0x7fe04c0adf10 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe04c0c57e0 0x7fe04c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 --2- 172.21.15.17:0/979540098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe04c072cd0 0x7fe04c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.839+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 >> 172.21.15.17:0/979540098 conn(0x7fe04c0aa630 msgr2=0x7fe04c0a9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.840+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 shutdown_connections 2024-03-06T07:26:23.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.840+0000 7fe051b83640 1 -- 172.21.15.17:0/979540098 wait complete. 2024-03-06T07:26:23.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs2_data pg_num 2024-03-06T07:26:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.982+0000 7f357359e640 1 Processor -- start 2024-03-06T07:26:23.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.982+0000 7f357359e640 1 -- start start 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357359e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3574152750 0x7f35740c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357359e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35740c1a80 0x7f357415f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357359e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740c0cc0 con 0x7f357406e6a0 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357359e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740c15f0 con 0x7f35740c1a80 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357359e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357406bb60 con 0x7f3574152750 2024-03-06T07:26:23.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35740c1a80 0x7f357415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3571d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f357406e6a0 0x7f3574158600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48470/0 (socket says 172.21.15.17:48470) 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3571d9b640 1 -- 172.21.15.17:0/938638959 learned_addr learned my addr 172.21.15.17:0/938638959 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35740c1a80 0x7f357415f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59364/0 (socket says 172.21.15.17:59364) 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f357259c640 1 --2- 172.21.15.17:0/938638959 >> v2:172.21.15.163:3300/0 conn(0x7f3574152750 0x7f35740c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 -- 172.21.15.17:0/938638959 >> v2:172.21.15.163:3300/0 conn(0x7f3574152750 msgr2=0x7f35740c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 --2- 172.21.15.17:0/938638959 >> v2:172.21.15.163:3300/0 conn(0x7f3574152750 0x7f35740c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 -- 172.21.15.17:0/938638959 >> v1:172.21.15.17:6789/0 conn(0x7f357406e6a0 legacy=0x7f3574158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.983+0000 7f3572d9d640 1 -- 172.21.15.17:0/938638959 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740c09e0 con 0x7f35740c1a80 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.984+0000 7f3572d9d640 1 --2- 172.21.15.17:0/938638959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35740c1a80 0x7f357415f990 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3574074dd0 tx=0x7f356c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c873930a15a9fcfe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.984+0000 7f357159a640 1 -- 172.21.15.17:0/938638959 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c039070 con 0x7f35740c1a80 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.984+0000 7f357159a640 1 -- 172.21.15.17:0/938638959 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c02cbf0 con 0x7f35740c1a80 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.984+0000 7f357159a640 1 -- 172.21.15.17:0/938638959 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c034440 con 0x7f35740c1a80 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 -- 172.21.15.17:0/938638959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35740c1a80 msgr2=0x7f357415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 --2- 172.21.15.17:0/938638959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35740c1a80 0x7f357415f990 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f3574074dd0 tx=0x7f356c02be40 comp rx=0 tx=0).stop 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 -- 172.21.15.17:0/938638959 shutdown_connections 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 --2- 172.21.15.17:0/938638959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35740c1a80 0x7f357415f990 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 --2- 172.21.15.17:0/938638959 >> v2:172.21.15.163:3300/0 conn(0x7f3574152750 0x7f35740c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 -- 172.21.15.17:0/938638959 >> 172.21.15.17:0/938638959 conn(0x7f3574067120 msgr2=0x7f35740a9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:23.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 -- 172.21.15.17:0/938638959 shutdown_connections 2024-03-06T07:26:23.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.985+0000 7f357359e640 1 -- 172.21.15.17:0/938638959 wait complete. 2024-03-06T07:26:23.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.986+0000 7f357359e640 1 Processor -- start 2024-03-06T07:26:23.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.986+0000 7f357359e640 1 -- start start 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35740c1a80 0x7f3574156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3574152750 0x7f3574156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357406b540 con 0x7f357406e6a0 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c2c60 con 0x7f3574152750 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357359e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3574158d10 con 0x7f35740c1a80 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59378/0 (socket says 172.21.15.17:59378) 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 -- 172.21.15.17:0/3283062134 learned_addr learned my addr 172.21.15.17:0/3283062134 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 -- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35740c1a80 msgr2=0x7f3574156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35740c1a80 0x7f3574156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 -- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3574152750 msgr2=0x7f3574156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3574152750 0x7f3574156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 -- 172.21.15.17:0/3283062134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574157210 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.987+0000 7f357259c640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f356400d620 tx=0x7f356400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.988+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400b620 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.988+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3564019070 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.988+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35741ec260 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.988+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400ba00 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.988+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35741579e0 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.989+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740677c0 con 0x7f357406e6a0 2024-03-06T07:26:23.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.990+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356400bba0 con 0x7f357406e6a0 2024-03-06T07:26:23.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.990+0000 7f3562ffd640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:23.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.990+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f356408a860 con 0x7f357406e6a0 2024-03-06T07:26:23.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.990+0000 7f3571d9b640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:23.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.991+0000 7f3571d9b640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 0x7f355406d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3568005880 tx=0x7f3568007aa0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:23.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:23.994+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35740677c0 con 0x7f357406e6a0 2024-03-06T07:26:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.174+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"} v 0) v1 -- 0x7f35740693c0 con 0x7f357406e6a0 2024-03-06T07:26:24.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.174+0000 7f3562ffd640 1 -- 172.21.15.17:0/3283062134 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs2_data", "var": "pg_num"}]=0 v160) v1 ==== 101+0+11 (secure 0 0 0) 0x7f35741ec260 con 0x7f357406e6a0 2024-03-06T07:26:24.176 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 msgr2=0x7f355406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 0x7f355406d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3568005880 tx=0x7f3568007aa0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 msgr2=0x7f35740c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f356400d620 tx=0x7f356400dae0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 shutdown_connections 2024-03-06T07:26:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f355406b4d0 0x7f355406d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3574152750 0x7f3574156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35740c1a80 0x7f3574156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 --2- 172.21.15.17:0/3283062134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357406e6a0 0x7f35740c6460 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 >> 172.21.15.17:0/3283062134 conn(0x7f3574067120 msgr2=0x7f35740711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 shutdown_connections 2024-03-06T07:26:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.176+0000 7f357359e640 1 -- 172.21.15.17:0/3283062134 wait complete. 2024-03-06T07:26:24.189 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.288+0000 7fb34399a640 1 Processor -- start 2024-03-06T07:26:24.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.288+0000 7fb34399a640 1 -- start start 2024-03-06T07:26:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c1527b0 0x7fb33c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb33c0c7180 0x7fb33c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 0x7fb33c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c159640 con 0x7fb33c0c7180 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0c8df0 con 0x7fb33c155b20 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb34399a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0c7c60 con 0x7fb33c1527b0 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb340f35640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb33c0c7180 0x7fb33c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb340f35640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb33c0c7180 0x7fb33c0c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59390/0 (socket says 172.21.15.17:59390) 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb340f35640 1 -- 172.21.15.17:0/1070495136 learned_addr learned my addr 172.21.15.17:0/1070495136 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb341f37640 1 --2- 172.21.15.17:0/1070495136 >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 0x7fb33c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.289+0000 7fb341736640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c1527b0 0x7fb33c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 -- 172.21.15.17:0/1070495136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c1527b0 msgr2=0x7fb33c159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c1527b0 0x7fb33c159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 -- 172.21.15.17:0/1070495136 >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 msgr2=0x7fb33c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 --2- 172.21.15.17:0/1070495136 >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 0x7fb33c0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 -- 172.21.15.17:0/1070495136 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0c8060 con 0x7fb33c0c7180 2024-03-06T07:26:24.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb341f37640 1 --2- 172.21.15.17:0/1070495136 >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 0x7fb33c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb340f35640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0c88b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb33c06b170 tx=0x7fb32c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe577e2e4af9c0d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.290+0000 7fb333fff640 1 -- 172.21.15.17:0/1070495136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c02e070 con 0x7fb33c0c7180 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb333fff640 1 -- 172.21.15.17:0/1070495136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c037070 con 0x7fb33c0c7180 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb333fff640 1 -- 172.21.15.17:0/1070495136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c031a00 con 0x7fb33c0c7180 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 -- 172.21.15.17:0/1070495136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 msgr2=0x7fb33c0c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0c88b0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb33c06b170 tx=0x7fb32c02b880 comp rx=0 tx=0).stop 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 -- 172.21.15.17:0/1070495136 shutdown_connections 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 --2- 172.21.15.17:0/1070495136 >> v2:172.21.15.163:3300/0 conn(0x7fb33c155b20 0x7fb33c0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0c88b0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 --2- 172.21.15.17:0/1070495136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c1527b0 0x7fb33c159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 -- 172.21.15.17:0/1070495136 >> 172.21.15.17:0/1070495136 conn(0x7fb33c0ac560 msgr2=0x7fb33c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 -- 172.21.15.17:0/1070495136 shutdown_connections 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.291+0000 7fb34399a640 1 -- 172.21.15.17:0/1070495136 wait complete. 2024-03-06T07:26:24.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.292+0000 7fb34399a640 1 Processor -- start 2024-03-06T07:26:24.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.292+0000 7fb34399a640 1 -- start start 2024-03-06T07:26:24.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb33c1527b0 0x7fb33c1eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c155b20 0x7fb33c1eb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c155320 con 0x7fb33c0c7180 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0c7980 con 0x7fb33c1527b0 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb34399a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c155600 con 0x7fb33c155b20 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59396/0 (socket says 172.21.15.17:59396) 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 -- 172.21.15.17:0/2888468628 learned_addr learned my addr 172.21.15.17:0/2888468628 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341f37640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c155b20 0x7fb33c1eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb340f35640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb33c1527b0 0x7fb33c1eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 -- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c155b20 msgr2=0x7fb33c1eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c155b20 0x7fb33c1eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 -- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb33c1527b0 msgr2=0x7fb33c1eaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb33c1527b0 0x7fb33c1eaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.293+0000 7fb341736640 1 -- 172.21.15.17:0/2888468628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c1eb970 con 0x7fb33c0c7180 2024-03-06T07:26:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb341736640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d800 tx=0x7fb33800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800b6c0 con 0x7fb33c0c7180 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338019070 con 0x7fb33c0c7180 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c1ea9f0 con 0x7fb33c0c7180 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800ba80 con 0x7fb33c0c7180 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.294+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c1ee740 con 0x7fb33c0c7180 2024-03-06T07:26:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.295+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c04f330 con 0x7fb33c0c7180 2024-03-06T07:26:24.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.296+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb33800bc20 con 0x7fb33c0c7180 2024-03-06T07:26:24.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.296+0000 7fb331ffb640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.296+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7fb33808a7c0 con 0x7fb33c0c7180 2024-03-06T07:26:24.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.296+0000 7fb340f35640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.297+0000 7fb340f35640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 0x7fb32006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb32c036000 tx=0x7fb32c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.300+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb33c1eb970 con 0x7fb33c0c7180 2024-03-06T07:26:24.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.519+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb33c06ce00 con 0x7fb33c0c7180 2024-03-06T07:26:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.519+0000 7fb331ffb640 1 -- 172.21.15.17:0/2888468628 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fb338088e20 con 0x7fb33c0c7180 2024-03-06T07:26:24.521 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:24.521 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":152,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/923114993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":923114993},{"type":"v1","addr":"172.21.15.163:6837","nonce":923114993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":8548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2327494506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2327494506},{"type":"v1","addr":"172.21.15.163:6836","nonce":2327494506}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:23.894566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9090":{"gid":9090,"name":"a","rank":0,"incarnation":149,"state":"up:active","state_seq":6,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9090,"qdb_cluster":[9090]},"id":12}]} 2024-03-06T07:26:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 152 2024-03-06T07:26:24.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 msgr2=0x7fb32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 0x7fb32006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb32c036000 tx=0x7fb32c03d040 comp rx=0 tx=0).stop 2024-03-06T07:26:24.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 msgr2=0x7fb33c0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d800 tx=0x7fb33800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 shutdown_connections 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb32006b4d0 0x7fb32006d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb33c155b20 0x7fb33c1eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb33c1527b0 0x7fb33c1eaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.521+0000 7fb34399a640 1 --2- 172.21.15.17:0/2888468628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb33c0c7180 0x7fb33c0ca1a0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.522+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 >> 172.21.15.17:0/2888468628 conn(0x7fb33c0ac560 msgr2=0x7fb33c0bdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.522+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 shutdown_connections 2024-03-06T07:26:24.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.522+0000 7fb34399a640 1 -- 172.21.15.17:0/2888468628 wait complete. 2024-03-06T07:26:24.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.633+0000 7f99b40df640 1 Processor -- start 2024-03-06T07:26:24.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.633+0000 7f99b40df640 1 -- start start 2024-03-06T07:26:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b40df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99ac159270 0x7f99ac155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b40df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0c73d0 con 0x7f99ac14f360 2024-03-06T07:26:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b40df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ac0be2e0 con 0x7f99ac0c88c0 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b40df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac159700 con 0x7f99ac159270 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b1e7b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f99ac14f360 0x7f99ac0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48476/0 (socket says 172.21.15.17:48476) 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b1e7b640 1 -- 172.21.15.17:0/3212141556 learned_addr learned my addr 172.21.15.17:0/3212141556 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107762061 0 0) 0x7f99ac0c73d0 con 0x7f99ac14f360 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9998003520 con 0x7f99ac14f360 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b267c640 1 --2- 172.21.15.17:0/3212141556 >> v2:172.21.15.163:3300/0 conn(0x7f99ac159270 0x7f99ac155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112508680 0 0) 0x7f9998003520 con 0x7f99ac14f360 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 >> v2:172.21.15.163:3300/0 conn(0x7f99ac159270 msgr2=0x7f99ac155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 --2- 172.21.15.17:0/3212141556 >> v2:172.21.15.163:3300/0 conn(0x7f99ac159270 0x7f99ac155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 >> v1:172.21.15.163:6789/0 conn(0x7f99ac0c88c0 legacy=0x7f99ac158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0c7670 con 0x7f99ac14f360 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f99a8001670 con 0x7f99ac14f360 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b267c640 1 --2- 172.21.15.17:0/3212141556 >> v2:172.21.15.163:3300/0 conn(0x7f99ac159270 0x7f99ac155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 >> v1:172.21.15.17:6789/0 conn(0x7f99ac14f360 legacy=0x7f99ac0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.634+0000 7f99b0e79640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b0e79640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b0e79640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99980052e0 0x7f999800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac159700 con 0x7f99980039b0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999800c1b0 con 0x7f9998004800 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999800c3b0 con 0x7f99980052e0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b167a640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b267c640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99980052e0 0x7f999800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 -- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99980052e0 msgr2=0x7f999800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99980052e0 0x7f999800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 -- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9998004800 msgr2=0x7f9998004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b1e7b640 1 -- 172.21.15.17:0/3212141556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998003520 con 0x7f99980039b0 2024-03-06T07:26:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b267c640 1 -- 172.21.15.17:0/3212141556 reap_dead start 2024-03-06T07:26:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.635+0000 7f99b167a640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9998004800 0x7f9998004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001eb0 tx=0x7f9998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d745c44ef06ea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8007d30 con 0x7f99980039b0 2024-03-06T07:26:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a8030070 con 0x7f99980039b0 2024-03-06T07:26:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b0e79640 1 -- 172.21.15.17:0/3212141556 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8005d70 con 0x7f99980039b0 2024-03-06T07:26:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b40df640 1 -- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 msgr2=0x7f9998003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b40df640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f99a8001eb0 tx=0x7f9998005830 comp rx=0 tx=0).stop 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b40df640 1 -- 172.21.15.17:0/3212141556 shutdown_connections 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b40df640 1 --2- 172.21.15.17:0/3212141556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99980039b0 0x7f9998003dd0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.636+0000 7f99b40df640 1 -- 172.21.15.17:0/3212141556 >> 172.21.15.17:0/3212141556 conn(0x7f99ac0ac570 msgr2=0x7f99ac0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.637+0000 7f99b40df640 1 -- 172.21.15.17:0/3212141556 shutdown_connections 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.637+0000 7f99b40df640 1 -- 172.21.15.17:0/3212141556 wait complete. 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 Processor -- start 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 -- start start 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980039b0 0x7f99ac14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 0x7f99ac1ef240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac0c7c30 con 0x7f99ac0af900 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac155c10 con 0x7f99980039b0 2024-03-06T07:26:24.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b40df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac155d90 con 0x7f99ac1524b0 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b267c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b267c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59414/0 (socket says 172.21.15.17:59414) 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.638+0000 7f99b267c640 1 -- 172.21.15.17:0/150318852 learned_addr learned my addr 172.21.15.17:0/150318852 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980039b0 0x7f99ac14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b167a640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 0x7f99ac1ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 -- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 msgr2=0x7f99ac1ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 0x7f99ac1ef240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 -- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980039b0 msgr2=0x7f99ac14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980039b0 0x7f99ac14aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 -- 172.21.15.17:0/150318852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac1516f0 con 0x7f99ac0af900 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b167a640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 0x7f99ac1ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b267c640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f99ac067c30 tx=0x7f99a4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4014410 con 0x7f99ac0af900 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a4019070 con 0x7f99ac0af900 2024-03-06T07:26:24.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40147d0 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac1519d0 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.639+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac151f90 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.640+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac158b60 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.641+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99a4014970 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.641+0000 7f99a27fc640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.641+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f99a408a960 con 0x7f99ac0af900 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.641+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.642+0000 7f99b1e7b640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.646+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99ac158b60 con 0x7f99ac0af900 2024-03-06T07:26:24.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.827+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99ac0c6cb0 con 0x7f99ac0af900 2024-03-06T07:26:24.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.829+0000 7f99a27fc640 1 -- 172.21.15.17:0/150318852 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f99ac0c6cb0 con 0x7f99ac0af900 2024-03-06T07:26:24.830 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:24.831 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":160,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:22.883804+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.163:6837/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.163:6835/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.163:6834/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:24.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.830+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 msgr2=0x7f999006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.830+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 0x7f999006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99a8034000 tx=0x7f99a8009030 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 msgr2=0x7f99ac0ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f99ac067c30 tx=0x7f99a4007700 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 shutdown_connections 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f999006b4d0 0x7f999006d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99ac0af900 0x7f99ac0ab7a0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99ac1524b0 0x7f99ac1ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 --2- 172.21.15.17:0/150318852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99980039b0 0x7f99ac14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 >> 172.21.15.17:0/150318852 conn(0x7f99ac0ac570 msgr2=0x7f99ac0c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 shutdown_connections 2024-03-06T07:26:24.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.831+0000 7f99b40df640 1 -- 172.21.15.17:0/150318852 wait complete. 2024-03-06T07:26:24.843 INFO:tasks.cephfs.filesystem:Destroying file system cephfs2 and related pools 2024-03-06T07:26:24.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:24.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.944+0000 7effedf7a640 1 Processor -- start 2024-03-06T07:26:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.944+0000 7effedf7a640 1 -- start start 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effedf7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7effe806ef30 0x7effe806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effedf7a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe80c1ae0 con 0x7effe8155c80 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effedf7a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe806e450 con 0x7effe8152520 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effedf7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80c1dc0 con 0x7effe806ef30 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7effe806ef30 0x7effe806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7effe8155c80 0x7effe80720a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48492/0 (socket says 172.21.15.17:48492) 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe6ffd640 1 -- 172.21.15.17:0/1107731960 learned_addr learned my addr 172.21.15.17:0/1107731960 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe7fff640 1 -- 172.21.15.17:0/1107731960 >> v1:172.21.15.163:6789/0 conn(0x7effe8152520 legacy=0x7effe80c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe7fff640 1 -- 172.21.15.17:0/1107731960 >> v1:172.21.15.17:6789/0 conn(0x7effe8155c80 legacy=0x7effe80720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe67fc640 1 -- 172.21.15.17:0/1107731960 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541057437 0 0) 0x7effe80c1ae0 con 0x7effe8155c80 2024-03-06T07:26:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.945+0000 7effe7fff640 1 -- 172.21.15.17:0/1107731960 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe806e730 con 0x7effe806ef30 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.946+0000 7effe7fff640 1 --2- 172.21.15.17:0/1107731960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe806ef30 0x7effe806afc0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1323d627def2b4d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.946+0000 7effe67fc640 1 -- 172.21.15.17:0/1107731960 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0039070 con 0x7effe806ef30 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.946+0000 7effe67fc640 1 -- 172.21.15.17:0/1107731960 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe002cc60 con 0x7effe806ef30 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.946+0000 7effe67fc640 1 -- 172.21.15.17:0/1107731960 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0034660 con 0x7effe806ef30 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 -- 172.21.15.17:0/1107731960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe806ef30 msgr2=0x7effe806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 --2- 172.21.15.17:0/1107731960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe806ef30 0x7effe806afc0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7effe0009d90 tx=0x7effe002beb0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 -- 172.21.15.17:0/1107731960 shutdown_connections 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 --2- 172.21.15.17:0/1107731960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe806ef30 0x7effe806afc0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 -- 172.21.15.17:0/1107731960 >> 172.21.15.17:0/1107731960 conn(0x7effe80c76b0 msgr2=0x7effe80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 -- 172.21.15.17:0/1107731960 shutdown_connections 2024-03-06T07:26:24.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.947+0000 7effedf7a640 1 -- 172.21.15.17:0/1107731960 wait complete. 2024-03-06T07:26:24.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.948+0000 7effedf7a640 1 Processor -- start 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.948+0000 7effedf7a640 1 -- start start 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effe806ef30 0x7effe81ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effe8152520 0x7effe814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe8152900 con 0x7effe8155c80 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80727b0 con 0x7effe8152520 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effedf7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe8152a80 con 0x7effe806ef30 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59438/0 (socket says 172.21.15.17:59438) 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 -- 172.21.15.17:0/2164456783 learned_addr learned my addr 172.21.15.17:0/2164456783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe77fe640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effe806ef30 0x7effe81ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe6ffd640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effe8152520 0x7effe814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 -- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effe806ef30 msgr2=0x7effe81ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effe806ef30 0x7effe81ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 -- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effe8152520 msgr2=0x7effe814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effe8152520 0x7effe814cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 -- 172.21.15.17:0/2164456783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe81eed20 con 0x7effe8155c80 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.949+0000 7effe7fff640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7effe0037040 tx=0x7effe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.950+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0044030 con 0x7effe8155c80 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.950+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe002ca60 con 0x7effe8155c80 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.950+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe81ef000 con 0x7effe8155c80 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.950+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0039040 con 0x7effe8155c80 2024-03-06T07:26:24.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.950+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe814d850 con 0x7effe8155c80 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.951+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8071c10 con 0x7effe8155c80 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.952+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effe0047070 con 0x7effe8155c80 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.952+0000 7effdffff640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 0x7effc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.952+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7effe00acda0 con 0x7effe8155c80 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.952+0000 7effe77fe640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 0x7effc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:24.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.953+0000 7effe77fe640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 0x7effc806d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7effd0006fd0 tx=0x7effd0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:24.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:24.956+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effe8071c10 con 0x7effe8155c80 2024-03-06T07:26:25.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.177+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe804f330 con 0x7effe8155c80 2024-03-06T07:26:25.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.177+0000 7effdffff640 1 -- 172.21.15.17:0/2164456783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4993 (secure 0 0 0) 0x7effe004e070 con 0x7effe8155c80 2024-03-06T07:26:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:25.179 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":152,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/923114993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":923114993},{"type":"v1","addr":"172.21.15.163:6837","nonce":923114993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":8548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2327494506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2327494506},{"type":"v1","addr":"172.21.15.163:6836","nonce":2327494506}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:23.894566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9090":{"gid":9090,"name":"a","rank":0,"incarnation":149,"state":"up:active","state_seq":6,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9090,"qdb_cluster":[9090]},"id":12}]} 2024-03-06T07:26:25.179 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 152 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 msgr2=0x7effc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 0x7effc806d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7effd0006fd0 tx=0x7effd0008040 comp rx=0 tx=0).stop 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 msgr2=0x7effe814d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7effe0037040 tx=0x7effe0004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 shutdown_connections 2024-03-06T07:26:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7effc806b4d0 0x7effc806d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe8155c80 0x7effe814d310 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7effe8152520 0x7effe814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 --2- 172.21.15.17:0/2164456783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7effe806ef30 0x7effe81ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 >> 172.21.15.17:0/2164456783 conn(0x7effe80c76b0 msgr2=0x7effe806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 shutdown_connections 2024-03-06T07:26:25.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.179+0000 7effedf7a640 1 -- 172.21.15.17:0/2164456783 wait complete. 2024-03-06T07:26:25.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:25.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.292+0000 7f460f581640 1 Processor -- start 2024-03-06T07:26:25.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.292+0000 7f460f581640 1 -- start start 2024-03-06T07:26:25.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460f581640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f460814f360 0x7f4608155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460f581640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080c73d0 con 0x7f46080c88c0 2024-03-06T07:26:25.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460f581640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080be2e0 con 0x7f4608158b60 2024-03-06T07:26:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460f581640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4608159700 con 0x7f460814f360 2024-03-06T07:26:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460d31d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f460814f360 0x7f4608155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460cb1c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46080c88c0 0x7f46080c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48500/0 (socket says 172.21.15.17:48500) 2024-03-06T07:26:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460cb1c640 1 -- 172.21.15.17:0/3148354107 learned_addr learned my addr 172.21.15.17:0/3148354107 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460d31d640 1 -- 172.21.15.17:0/3148354107 >> v1:172.21.15.163:6789/0 conn(0x7f4608158b60 legacy=0x7f4608158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460d31d640 1 -- 172.21.15.17:0/3148354107 >> v1:172.21.15.17:6789/0 conn(0x7f46080c88c0 legacy=0x7f46080c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f45fffff640 1 -- 172.21.15.17:0/3148354107 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447305462 0 0) 0x7f46080c73d0 con 0x7f46080c88c0 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.293+0000 7f460d31d640 1 -- 172.21.15.17:0/3148354107 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080c7670 con 0x7f460814f360 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.294+0000 7f460d31d640 1 --2- 172.21.15.17:0/3148354107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f460814f360 0x7f4608155c10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f46040097a0 tx=0x7f460402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c4a5d37fb48a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.294+0000 7f45fffff640 1 -- 172.21.15.17:0/3148354107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040315a0 con 0x7f460814f360 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.294+0000 7f45fffff640 1 -- 172.21.15.17:0/3148354107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4604037070 con 0x7f460814f360 2024-03-06T07:26:25.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.294+0000 7f45fffff640 1 -- 172.21.15.17:0/3148354107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604031960 con 0x7f460814f360 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 -- 172.21.15.17:0/3148354107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f460814f360 msgr2=0x7f4608155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 --2- 172.21.15.17:0/3148354107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f460814f360 0x7f4608155c10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f46040097a0 tx=0x7f460402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 -- 172.21.15.17:0/3148354107 shutdown_connections 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 --2- 172.21.15.17:0/3148354107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f460814f360 0x7f4608155c10 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 -- 172.21.15.17:0/3148354107 >> 172.21.15.17:0/3148354107 conn(0x7f46080ac570 msgr2=0x7f46080ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 -- 172.21.15.17:0/3148354107 shutdown_connections 2024-03-06T07:26:25.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.295+0000 7f460f581640 1 -- 172.21.15.17:0/3148354107 wait complete. 2024-03-06T07:26:25.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.296+0000 7f460f581640 1 Processor -- start 2024-03-06T07:26:25.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.296+0000 7f460f581640 1 -- start start 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 0x7f46081ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 0x7f46081eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080c7c30 con 0x7f46080c88c0 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080c5a50 con 0x7f4608158b60 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460f581640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080c5bd0 con 0x7f460814f360 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59452/0 (socket says 172.21.15.17:59452) 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 -- 172.21.15.17:0/4016136277 learned_addr learned my addr 172.21.15.17:0/4016136277 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460cb1c640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 0x7f46081ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460db1e640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 0x7f46081eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 -- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 msgr2=0x7f46081ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 0x7f46081ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 -- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 msgr2=0x7f46081eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 0x7f46081eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 -- 172.21.15.17:0/4016136277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46081514c0 con 0x7f46080c88c0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460db1e640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 0x7f46081eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460cb1c640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 0x7f46081ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.297+0000 7f460d31d640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f46040097a0 tx=0x7f46040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.298+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604032840 con 0x7f46080c88c0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.298+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040329e0 con 0x7f46080c88c0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.298+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46081ef280 con 0x7f46080c88c0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.298+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4604032dd0 con 0x7f46080c88c0 2024-03-06T07:26:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.298+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46081517b0 con 0x7f46080c88c0 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.299+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4608075300 con 0x7f46080c88c0 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.299+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f460403b3f0 con 0x7f46080c88c0 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.300+0000 7f45fdffb640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 0x7f45ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.300+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f460402f030 con 0x7f46080c88c0 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.300+0000 7f460cb1c640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 0x7f45ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.300+0000 7f460cb1c640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 0x7f45ec06d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f46080c94d0 tx=0x7f45f8007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.304+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4608075300 con 0x7f46080c88c0 2024-03-06T07:26:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.521+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460804f330 con 0x7f46080c88c0 2024-03-06T07:26:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.521+0000 7f45fdffb640 1 -- 172.21.15.17:0/4016136277 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f4604040070 con 0x7f46080c88c0 2024-03-06T07:26:25.523 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:25.523 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":152,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/923114993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":923114993},{"type":"v1","addr":"172.21.15.163:6837","nonce":923114993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":8548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2327494506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2327494506},{"type":"v1","addr":"172.21.15.163:6836","nonce":2327494506}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150},{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:03.654281+0000","modified":"2024-03-06T07:26:23.894566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":158,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9090},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9090":{"gid":9090,"name":"a","rank":0,"incarnation":149,"state":"up:active","state_seq":6,"addr":"172.21.15.17:6835/2508903849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2508903849},{"type":"v1","addr":"172.21.15.17:6835","nonce":2508903849}]},"join_fscid":12,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs2","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9090,"qdb_cluster":[9090]},"id":12}]} 2024-03-06T07:26:25.523 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 152 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 msgr2=0x7f45ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 0x7f45ec06d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f46080c94d0 tx=0x7f45f8007c10 comp rx=0 tx=0).stop 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 msgr2=0x7f46080bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f46040097a0 tx=0x7f46040046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 shutdown_connections 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f45ec06b4d0 0x7f45ec06d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4608158b60 0x7f46081eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f460814f360 0x7f46081ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 --2- 172.21.15.17:0/4016136277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46080c88c0 0x7f46080bef00 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 >> 172.21.15.17:0/4016136277 conn(0x7f46080ac570 msgr2=0x7f4608073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 shutdown_connections 2024-03-06T07:26:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.524+0000 7f460f581640 1 -- 172.21.15.17:0/4016136277 wait complete. 2024-03-06T07:26:25.537 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:25.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.638+0000 7f899de66640 1 Processor -- start 2024-03-06T07:26:25.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.638+0000 7f899de66640 1 -- start start 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.638+0000 7f899de66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f899814f360 0x7f8998155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.638+0000 7f899de66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f89980c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.638+0000 7f899de66640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f89980c5880 0x7f8998158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f899de66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980c8dd0 con 0x7f899814f360 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f899de66640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980c73d0 con 0x7f89980c5880 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f899de66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980be2e0 con 0x7f89980c88c0 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f899814f360 0x7f8998155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f899814f360 0x7f8998155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59466/0 (socket says 172.21.15.17:59466) 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 -- 172.21.15.17:0/2925202813 learned_addr learned my addr 172.21.15.17:0/2925202813 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f8996ffd640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f89980c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f8997fff640 1 --2- 172.21.15.17:0/2925202813 >> v2:172.21.15.163:3300/0 conn(0x7f89980c5880 0x7f8998158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 -- 172.21.15.17:0/2925202813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 msgr2=0x7f89980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f89980c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 -- 172.21.15.17:0/2925202813 >> v2:172.21.15.163:3300/0 conn(0x7f89980c5880 msgr2=0x7f8998158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:25.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 --2- 172.21.15.17:0/2925202813 >> v2:172.21.15.163:3300/0 conn(0x7f89980c5880 0x7f8998158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.639+0000 7f89977fe640 1 -- 172.21.15.17:0/2925202813 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980c7670 con 0x7f899814f360 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.640+0000 7f89977fe640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f8998155c10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f899808cc10 tx=0x7f898802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc3a4ac37d23f8c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.640+0000 7f89967fc640 1 -- 172.21.15.17:0/2925202813 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898802e070 con 0x7f899814f360 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.640+0000 7f89967fc640 1 -- 172.21.15.17:0/2925202813 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8988037070 con 0x7f899814f360 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.640+0000 7f89967fc640 1 -- 172.21.15.17:0/2925202813 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988031a00 con 0x7f899814f360 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 -- 172.21.15.17:0/2925202813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 msgr2=0x7f8998155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f8998155c10 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f899808cc10 tx=0x7f898802b880 comp rx=0 tx=0).stop 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 -- 172.21.15.17:0/2925202813 shutdown_connections 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 --2- 172.21.15.17:0/2925202813 >> v2:172.21.15.163:3300/0 conn(0x7f89980c5880 0x7f8998158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f89980c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 --2- 172.21.15.17:0/2925202813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f8998155c10 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 -- 172.21.15.17:0/2925202813 >> 172.21.15.17:0/2925202813 conn(0x7f89980ac570 msgr2=0x7f89980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 -- 172.21.15.17:0/2925202813 shutdown_connections 2024-03-06T07:26:25.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.641+0000 7f899de66640 1 -- 172.21.15.17:0/2925202813 wait complete. 2024-03-06T07:26:25.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.642+0000 7f899de66640 1 Processor -- start 2024-03-06T07:26:25.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.643+0000 7f899de66640 1 -- start start 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89980c5880 0x7f89981ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f8998151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89981590a0 con 0x7f899814f360 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89980c7c30 con 0x7f89980c5880 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f899de66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8998159220 con 0x7f89980c88c0 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59474/0 (socket says 172.21.15.17:59474) 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 -- 172.21.15.17:0/1055427840 learned_addr learned my addr 172.21.15.17:0/1055427840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f89977fe640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89980c5880 0x7f89981ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8996ffd640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f8998151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 -- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 msgr2=0x7f8998151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f8998151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 -- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89980c5880 msgr2=0x7f89981ee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89980c5880 0x7f89981ee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 -- 172.21.15.17:0/1055427840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980af980 con 0x7f899814f360 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.644+0000 7f8997fff640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8990007e60 tx=0x7f899000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.645+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990014620 con 0x7f899814f360 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.645+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990019070 con 0x7f899814f360 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.645+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990014a20 con 0x7f899814f360 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.645+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89980af6e0 con 0x7f899814f360 2024-03-06T07:26:25.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.645+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89981eec60 con 0x7f899814f360 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.646+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899804f330 con 0x7f899814f360 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.647+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8990014bc0 con 0x7f899814f360 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.647+0000 7f897ffff640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 0x7f898406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.647+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f899008ac50 con 0x7f899814f360 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.647+0000 7f89977fe640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 0x7f898406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.648+0000 7f89977fe640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 0x7f898406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8988036000 tx=0x7f898803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.651+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8990090020 con 0x7f899814f360 2024-03-06T07:26:25.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.834+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f899806cda0 con 0x7f899814f360 2024-03-06T07:26:25.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.835+0000 7f897ffff640 1 -- 172.21.15.17:0/1055427840 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f8990059100 con 0x7f899814f360 2024-03-06T07:26:25.837 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:25.837 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":160,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:22.883804+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs2_metadata","create_time":"2024-03-06T07:26:02.038846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs2_data","create_time":"2024-03-06T07:26:02.549840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs2"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6836/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.163:6837/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.163:6835/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.163:6834/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 msgr2=0x7f898406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 0x7f898406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8988036000 tx=0x7f898803d040 comp rx=0 tx=0).stop 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 msgr2=0x7f89981524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f8990007e60 tx=0x7f899000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 shutdown_connections 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f898406b4d0 0x7f898406d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f899814f360 0x7f89981524d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89980c88c0 0x7f8998151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 --2- 172.21.15.17:0/1055427840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89980c5880 0x7f89981ee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 >> 172.21.15.17:0/1055427840 conn(0x7f89980ac570 msgr2=0x7f899806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 shutdown_connections 2024-03-06T07:26:25.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.837+0000 7f899de66640 1 -- 172.21.15.17:0/1055427840 wait complete. 2024-03-06T07:26:25.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs2 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.952+0000 7f8b7adbc640 1 Processor -- start 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.952+0000 7f8b7adbc640 1 -- start start 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b7adbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b740c6dd0 0x7f8b74160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b7adbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b74155130 con 0x7f8b740c8950 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b7adbc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b740c7200 con 0x7f8b740c69f0 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b7adbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b74155440 con 0x7f8b740c6dd0 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b73fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8b740c8950 0x7f8b741558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48504/0 (socket says 172.21.15.17:48504) 2024-03-06T07:26:25.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b73fff640 1 -- 172.21.15.17:0/1773345187 learned_addr learned my addr 172.21.15.17:0/1773345187 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b79359640 1 --2- 172.21.15.17:0/1773345187 >> v2:172.21.15.163:3300/0 conn(0x7f8b740c6dd0 0x7f8b74160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805094923 0 0) 0x7f8b60001110 con 0x7f8b740c8950 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b64003520 con 0x7f8b740c8950 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313493253 0 0) 0x7f8b740c7200 con 0x7f8b740c69f0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b60001110 con 0x7f8b740c69f0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1563915122 0 0) 0x7f8b64003520 con 0x7f8b740c8950 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b79359640 1 -- 172.21.15.17:0/1773345187 >> v1:172.21.15.163:6789/0 conn(0x7f8b740c69f0 legacy=0x7f8b740acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b79359640 1 -- 172.21.15.17:0/1773345187 >> v1:172.21.15.17:6789/0 conn(0x7f8b740c8950 legacy=0x7f8b741558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.953+0000 7f8b79359640 1 -- 172.21.15.17:0/1773345187 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740ad480 con 0x7f8b740c6dd0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.954+0000 7f8b79359640 1 --2- 172.21.15.17:0/1773345187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c6dd0 0x7f8b74160ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8b741555c0 tx=0x7f8b6c02bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e25fe2c580c657ee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c039070 con 0x7f8b740c6dd0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c02c8a0 con 0x7f8b740c6dd0 2024-03-06T07:26:25.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b737fe640 1 -- 172.21.15.17:0/1773345187 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60001840 con 0x7f8b740c6dd0 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 -- 172.21.15.17:0/1773345187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c6dd0 msgr2=0x7f8b74160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/1773345187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c6dd0 0x7f8b74160ac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8b741555c0 tx=0x7f8b6c02bd70 comp rx=0 tx=0).stop 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 -- 172.21.15.17:0/1773345187 shutdown_connections 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/1773345187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c6dd0 0x7f8b74160ac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 -- 172.21.15.17:0/1773345187 >> 172.21.15.17:0/1773345187 conn(0x7f8b740cae70 msgr2=0x7f8b740cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 -- 172.21.15.17:0/1773345187 shutdown_connections 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.955+0000 7f8b7adbc640 1 -- 172.21.15.17:0/1773345187 wait complete. 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.956+0000 7f8b7adbc640 1 Processor -- start 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.956+0000 7f8b7adbc640 1 -- start start 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b740c69f0 0x7f8b74154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 0x7f8b741eba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c6310 con 0x7f8b740c6dd0 2024-03-06T07:26:25.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b740c6030 con 0x7f8b740c8950 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b7adbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74155fe0 con 0x7f8b740c69f0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b73fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59480/0 (socket says 172.21.15.17:59480) 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.957+0000 7f8b73fff640 1 -- 172.21.15.17:0/4026393990 learned_addr learned my addr 172.21.15.17:0/4026393990 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b79359640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 0x7f8b741eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b78b58640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b740c69f0 0x7f8b74154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 -- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b740c69f0 msgr2=0x7f8b74154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b740c69f0 0x7f8b74154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 -- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 msgr2=0x7f8b741eba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 0x7f8b741eba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 -- 172.21.15.17:0/4026393990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b741ebfb0 con 0x7f8b740c6dd0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b79359640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 0x7f8b741eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b73fff640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000caf0 tx=0x7f8b600018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60005ce0 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60007070 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60007460 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b741eb030 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.958+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b74157f30 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.959+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7404f330 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.960+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b60007600 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.960+0000 7f8b717fa640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 0x7f8b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.960+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6449+0+0 (secure 0 0 0) 0x7f8b6008a5a0 con 0x7f8b740c6dd0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.961+0000 7f8b78b58640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 0x7f8b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.961+0000 7f8b78b58640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 0x7f8b5806d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8b68007fd0 tx=0x7f8b68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:25.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:25.965+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b741ebfb0 con 0x7f8b740c6dd0 2024-03-06T07:26:26.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.184+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs2"} v 0) v1 -- 0x7f8b740ca650 con 0x7f8b740c6dd0 2024-03-06T07:26:26.186 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:26:26.184+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:26:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.185+0000 7f8b717fa640 1 -- 172.21.15.17:0/4026393990 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs2"}]=0 cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v153) v1 ==== 163+0+0 (secure 0 0 0) 0x7f8b74155130 con 0x7f8b740c6dd0 2024-03-06T07:26:26.187 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs2 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:26:26.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 msgr2=0x7f8b5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 0x7f8b5806d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8b68007fd0 tx=0x7f8b68009040 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 msgr2=0x7f8b741eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000caf0 tx=0x7f8b600018b0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 shutdown_connections 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8b5806b4d0 0x7f8b5806d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b740c8950 0x7f8b741eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8b740c6dd0 0x7f8b741eb530 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 --2- 172.21.15.17:0/4026393990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b740c69f0 0x7f8b74154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 >> 172.21.15.17:0/4026393990 conn(0x7f8b740cae70 msgr2=0x7f8b740acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 shutdown_connections 2024-03-06T07:26:26.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.188+0000 7f8b7adbc640 1 -- 172.21.15.17:0/4026393990 wait complete. 2024-03-06T07:26:26.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs2 --yes-i-really-mean-it 2024-03-06T07:26:26.214 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:26:26.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 Processor -- start 2024-03-06T07:26:26.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 -- start start 2024-03-06T07:26:26.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f720c0c4d10 0x7f720c0c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 0x7f720c1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720c155300 con 0x7f720c0c4930 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720c1555e0 con 0x7f720c0c4d10 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.303+0000 7f7213d13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1558c0 con 0x7f720c0c21c0 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f720c0c4d10 0x7f720c0c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f720c0c4d10 0x7f720c0c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59506/0 (socket says 172.21.15.17:59506) 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 -- 172.21.15.17:0/660771951 learned_addr learned my addr 172.21.15.17:0/660771951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72122b0640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 0x7f720c1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 -- 172.21.15.17:0/660771951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 msgr2=0x7f720c1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 0x7f720c1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 -- 172.21.15.17:0/660771951 >> v1:172.21.15.163:6789/0 conn(0x7f720c0c4930 legacy=0x7f720c0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72112ae640 1 -- 172.21.15.17:0/660771951 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c155ba0 con 0x7f720c0c4d10 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.304+0000 7f72122b0640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 0x7f720c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f72112ae640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4d10 0x7f720c0c1c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009870 tx=0x7f71fc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efff63518fa9325f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:26.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7210aad640 1 -- 172.21.15.17:0/660771951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0317c0 con 0x7f720c0c4d10 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7210aad640 1 -- 172.21.15.17:0/660771951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc037070 con 0x7f720c0c4d10 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7210aad640 1 -- 172.21.15.17:0/660771951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc031b80 con 0x7f720c0c4d10 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 -- 172.21.15.17:0/660771951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4d10 msgr2=0x7f720c0c1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4d10 0x7f720c0c1c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009870 tx=0x7f71fc02b860 comp rx=0 tx=0).stop 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 -- 172.21.15.17:0/660771951 shutdown_connections 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c21c0 0x7f720c1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 --2- 172.21.15.17:0/660771951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4d10 0x7f720c0c1c80 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 -- 172.21.15.17:0/660771951 >> 172.21.15.17:0/660771951 conn(0x7f720c0aa5c0 msgr2=0x7f720c0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 -- 172.21.15.17:0/660771951 shutdown_connections 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.305+0000 7f7213d13640 1 -- 172.21.15.17:0/660771951 wait complete. 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 Processor -- start 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 -- start start 2024-03-06T07:26:26.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f720c0c21c0 0x7f720c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c4d10 0x7f720c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c0c81c0 con 0x7f720c0c4930 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1560c0 con 0x7f720c0c21c0 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f7213d13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c0c8340 con 0x7f720c0c4d10 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f72112ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f72112ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59510/0 (socket says 172.21.15.17:59510) 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.307+0000 7f72112ae640 1 -- 172.21.15.17:0/3052830683 learned_addr learned my addr 172.21.15.17:0/3052830683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 -- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c4d10 msgr2=0x7f720c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72122b0640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c4d10 0x7f720c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f7211aaf640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f720c0c21c0 0x7f720c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c4d10 0x7f720c06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 -- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f720c0c21c0 msgr2=0x7f720c150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f720c0c21c0 0x7f720c150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 -- 172.21.15.17:0/3052830683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c06fd30 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72112ae640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f71fc02b8a0 tx=0x7f71fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc032860 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc032a00 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc032df0 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c150ae0 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.308+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c06f810 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.309+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c04f330 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.309+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71fc04c060 con 0x7f720c0c4930 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.310+0000 7f72027fc640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 0x7f71f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:26.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.310+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f71fc0abdc0 con 0x7f720c0c4930 2024-03-06T07:26:26.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.310+0000 7f7211aaf640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 0x7f71f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:26.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.310+0000 7f7211aaf640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 0x7f71f006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7204003d90 tx=0x7f7204009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:26.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.314+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f71fc040070 con 0x7f720c0c4930 2024-03-06T07:26:26.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:26.535+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f720c070110 con 0x7f720c0c4930 2024-03-06T07:26:27.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.187+0000 7f72027fc640 1 -- 172.21.15.17:0/3052830683 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs2", "yes_i_really_mean_it": true}]=0 v154) v1 ==== 105+0+0 (secure 0 0 0) 0x7f71fc07a990 con 0x7f720c0c4930 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 msgr2=0x7f71f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 0x7f71f006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7204003d90 tx=0x7f7204009040 comp rx=0 tx=0).stop 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 msgr2=0x7f720c1edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f71fc02b8a0 tx=0x7f71fc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 shutdown_connections 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f71f006b4d0 0x7f71f006d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f720c0c4d10 0x7f720c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f720c0c4930 0x7f720c1edf40 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 --2- 172.21.15.17:0/3052830683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f720c0c21c0 0x7f720c150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 >> 172.21.15.17:0/3052830683 conn(0x7f720c0aa5c0 msgr2=0x7f720c0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:27.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 shutdown_connections 2024-03-06T07:26:27.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.189+0000 7f7213d13640 1 -- 172.21.15.17:0/3052830683 wait complete. 2024-03-06T07:26:27.204 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_metadata cephfs2_metadata --yes-i-really-really-mean-it 2024-03-06T07:26:27.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.354+0000 7f798dbe9640 1 Processor -- start 2024-03-06T07:26:27.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.354+0000 7f798dbe9640 1 -- start start 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f798dbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7988151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f798dbe9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7988153390 0x7f7988153820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f798dbe9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f798806e850 con 0x7f79880c22b0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f798dbe9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798806d2b0 con 0x7f7988153390 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f798dbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798806d590 con 0x7f79880a9830 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f79877fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f79880c22b0 0x7f798814ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48518/0 (socket says 172.21.15.17:48518) 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f79877fe640 1 -- 172.21.15.17:0/4253670164 learned_addr learned my addr 172.21.15.17:0/4253670164 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f7986ffd640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7988151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f7987fff640 1 --2- 172.21.15.17:0/4253670164 >> v2:172.21.15.163:3300/0 conn(0x7f7988153390 0x7f7988153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034051505 0 0) 0x7f7978001110 con 0x7f79880c22b0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.355+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7970003520 con 0x7f79880c22b0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2344753288 0 0) 0x7f7970003520 con 0x7f79880c22b0 2024-03-06T07:26:27.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 msgr2=0x7f7988151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7988151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 >> v2:172.21.15.163:3300/0 conn(0x7f7988153390 msgr2=0x7f7988153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 --2- 172.21.15.17:0/4253670164 >> v2:172.21.15.163:3300/0 conn(0x7f7988153390 0x7f7988153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798806d870 con 0x7f79880c22b0 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7978001840 con 0x7f79880c22b0 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 >> v1:172.21.15.17:6789/0 conn(0x7f79880c22b0 legacy=0x7f798814ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79700039b0 0x7f7970003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7970004d40 0x7f79700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798806d2b0 con 0x7f79700039b0 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798806d590 con 0x7f7970004d40 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797000b9c0 con 0x7f79880a9830 2024-03-06T07:26:27.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79700039b0 0x7f7970003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f7986ffd640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f7987fff640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7970004d40 0x7f79700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 -- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 msgr2=0x7f7970004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f79880a9830 0x7f7970004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 -- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7970004d40 msgr2=0x7f79700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7970004d40 0x7f79700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.356+0000 7f79877fe640 1 -- 172.21.15.17:0/4253670164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7970003520 con 0x7f79700039b0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.357+0000 7f79877fe640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79700039b0 0x7f7970003dd0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f797800c4f0 tx=0x7f7970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdecd2a71656235 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.357+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800c0e0 con 0x7f79700039b0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.357+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978030070 con 0x7f79700039b0 2024-03-06T07:26:27.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.357+0000 7f79867fc640 1 -- 172.21.15.17:0/4253670164 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780058d0 con 0x7f79700039b0 2024-03-06T07:26:27.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 -- 172.21.15.17:0/4253670164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79700039b0 msgr2=0x7f7970003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 --2- 172.21.15.17:0/4253670164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79700039b0 0x7f7970003dd0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f797800c4f0 tx=0x7f7970005830 comp rx=0 tx=0).stop 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f7987fff640 1 -- 172.21.15.17:0/4253670164 reap_dead start 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 -- 172.21.15.17:0/4253670164 shutdown_connections 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 -- 172.21.15.17:0/4253670164 >> 172.21.15.17:0/4253670164 conn(0x7f79880c65b0 msgr2=0x7f79880c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 -- 172.21.15.17:0/4253670164 shutdown_connections 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.358+0000 7f798dbe9640 1 -- 172.21.15.17:0/4253670164 wait complete. 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.359+0000 7f798dbe9640 1 Processor -- start 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.359+0000 7f798dbe9640 1 -- start start 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79700039b0 0x7f79880c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798814bb30 0x7f79881e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798814f7e0 con 0x7f79881eb060 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798814f660 con 0x7f79700039b0 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c2690 con 0x7f798814bb30 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59536/0 (socket says 172.21.15.17:59536) 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 -- 172.21.15.17:0/3250869042 learned_addr learned my addr 172.21.15.17:0/3250869042 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f79877fe640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79700039b0 0x7f79880c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7986ffd640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798814bb30 0x7f79881e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 -- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798814bb30 msgr2=0x7f79881e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798814bb30 0x7f79881e1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 -- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79700039b0 msgr2=0x7f79880c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79700039b0 0x7f79880c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 -- 172.21.15.17:0/3250869042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79881ea880 con 0x7f79881eb060 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f7987fff640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7988070290 tx=0x7f798000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000fde0 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7980015430 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79881514c0 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980015820 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.360+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880ab2b0 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.361+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798804f330 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.362+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79800159c0 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.362+0000 7f796ffff640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.362+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f798008b2e0 con 0x7f79881eb060 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.362+0000 7f79877fe640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:27.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.363+0000 7f79877fe640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 0x7f797406d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7978034000 tx=0x7f7978007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:27.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.367+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7980019030 con 0x7f79881eb060 2024-03-06T07:26:27.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:27.547+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79881e2260 con 0x7f79881eb060 2024-03-06T07:26:28.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.192+0000 7f796ffff640 1 -- 172.21.15.17:0/3250869042 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_metadata", "pool2": "cephfs2_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_metadata' removed v162) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7980010040 con 0x7f79881eb060 2024-03-06T07:26:28.194 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_metadata' removed 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.195+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 msgr2=0x7f797406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.195+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 0x7f797406d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7978034000 tx=0x7f7978007000 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.195+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 msgr2=0x7f79881eddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.195+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7988070290 tx=0x7f798000f580 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 shutdown_connections 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79881eb060 0x7f79881eddf0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f798814bb30 0x7f79881e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 --2- 172.21.15.17:0/3250869042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f79700039b0 0x7f79880c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 >> 172.21.15.17:0/3250869042 conn(0x7f79880c65b0 msgr2=0x7f79881525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 shutdown_connections 2024-03-06T07:26:28.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.196+0000 7f798dbe9640 1 -- 172.21.15.17:0/3250869042 wait complete. 2024-03-06T07:26:28.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs2_data cephfs2_data --yes-i-really-really-mean-it 2024-03-06T07:26:28.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.323+0000 7ff758319640 1 Processor -- start 2024-03-06T07:26:28.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.323+0000 7ff758319640 1 -- start start 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff750152920 0x7ff750155970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750155eb0 0x7ff7500c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7500c80d0 0x7ff7500724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff750159740 con 0x7ff7500c80d0 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7501551d0 con 0x7ff750152920 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff758319640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c8ab0 con 0x7ff750155eb0 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7500c80d0 0x7ff7500724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff7500c80d0 0x7ff7500724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59544/0 (socket says 172.21.15.17:59544) 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 -- 172.21.15.17:0/3032963432 learned_addr learned my addr 172.21.15.17:0/3032963432 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:28.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7560b5640 1 --2- 172.21.15.17:0/3032963432 >> v2:172.21.15.163:3300/0 conn(0x7ff750152920 0x7ff750155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7558b4640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750155eb0 0x7ff7500c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 -- 172.21.15.17:0/3032963432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750155eb0 msgr2=0x7ff7500c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750155eb0 0x7ff7500c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 -- 172.21.15.17:0/3032963432 >> v2:172.21.15.163:3300/0 conn(0x7ff750152920 msgr2=0x7ff750155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 --2- 172.21.15.17:0/3032963432 >> v2:172.21.15.163:3300/0 conn(0x7ff750152920 0x7ff750155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.324+0000 7ff7568b6640 1 -- 172.21.15.17:0/3032963432 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500c8d90 con 0x7ff7500c80d0 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.325+0000 7ff7568b6640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff7500724f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff750073610 tx=0x7ff74802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7487cc152cb17cd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.325+0000 7ff7550b3640 1 -- 172.21.15.17:0/3032963432 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74802e070 con 0x7ff7500c80d0 2024-03-06T07:26:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.325+0000 7ff7550b3640 1 -- 172.21.15.17:0/3032963432 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748037070 con 0x7ff7500c80d0 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.325+0000 7ff7550b3640 1 -- 172.21.15.17:0/3032963432 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748031a00 con 0x7ff7500c80d0 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 -- 172.21.15.17:0/3032963432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 msgr2=0x7ff7500724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff7500724f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff750073610 tx=0x7ff74802b880 comp rx=0 tx=0).stop 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 -- 172.21.15.17:0/3032963432 shutdown_connections 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff7500724f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 --2- 172.21.15.17:0/3032963432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750155eb0 0x7ff7500c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 --2- 172.21.15.17:0/3032963432 >> v2:172.21.15.163:3300/0 conn(0x7ff750152920 0x7ff750155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 -- 172.21.15.17:0/3032963432 >> 172.21.15.17:0/3032963432 conn(0x7ff7500aa5c0 msgr2=0x7ff7500aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 -- 172.21.15.17:0/3032963432 shutdown_connections 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.326+0000 7ff758319640 1 -- 172.21.15.17:0/3032963432 wait complete. 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.327+0000 7ff758319640 1 Processor -- start 2024-03-06T07:26:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.327+0000 7ff758319640 1 -- start start 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750152920 0x7ff75006c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff750155eb0 0x7ff75009ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c86c0 con 0x7ff7500c80d0 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750155450 con 0x7ff750155eb0 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff758319640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c8840 con 0x7ff750152920 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59552/0 (socket says 172.21.15.17:59552) 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 -- 172.21.15.17:0/4241919700 learned_addr learned my addr 172.21.15.17:0/4241919700 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7558b4640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750152920 0x7ff75006c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7568b6640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff750155eb0 0x7ff75009ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 -- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750152920 msgr2=0x7ff75006c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750152920 0x7ff75006c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 -- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff750155eb0 msgr2=0x7ff75009ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff750155eb0 0x7ff75009ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 -- 172.21.15.17:0/4241919700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7501ec180 con 0x7ff7500c80d0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.328+0000 7ff7560b5640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff740007e90 tx=0x7ff74000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.329+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740014620 con 0x7ff7500c80d0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.329+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740019070 con 0x7ff7500c80d0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.329+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75006e9b0 con 0x7ff7500c80d0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.329+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740014a00 con 0x7ff7500c80d0 2024-03-06T07:26:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.329+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff750070450 con 0x7ff7500c80d0 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.330+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750153e60 con 0x7ff7500c80d0 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.330+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff740014ba0 con 0x7ff7500c80d0 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.330+0000 7ff74effd640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.330+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6095+0+0 (secure 0 0 0) 0x7ff74008aa30 con 0x7ff7500c80d0 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.331+0000 7ff7558b4640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:28.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.331+0000 7ff7558b4640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 0x7ff73c06d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff7500c2930 tx=0x7ff744008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.335+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff750153e60 con 0x7ff7500c80d0 2024-03-06T07:26:28.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:28.514+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff75004f330 con 0x7ff7500c80d0 2024-03-06T07:26:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.193+0000 7ff74effd640 1 -- 172.21.15.17:0/4241919700 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs2_data", "pool2": "cephfs2_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs2_data' removed v163) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff740059170 con 0x7ff7500c80d0 2024-03-06T07:26:29.195 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs2_data' removed 2024-03-06T07:26:29.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 msgr2=0x7ff73c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 0x7ff73c06d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff7500c2930 tx=0x7ff744008040 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 msgr2=0x7ff75006f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7ff740007e90 tx=0x7ff74000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 shutdown_connections 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff750155eb0 0x7ff75009ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff750152920 0x7ff75006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 --2- 172.21.15.17:0/4241919700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7500c80d0 0x7ff75006f490 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 >> 172.21.15.17:0/4241919700 conn(0x7ff7500aa5c0 msgr2=0x7ff750158330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 shutdown_connections 2024-03-06T07:26:29.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.198+0000 7ff758319640 1 -- 172.21.15.17:0/4241919700 wait complete. 2024-03-06T07:26:29.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.d mds_join_fs 2024-03-06T07:26:29.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 Processor -- start 2024-03-06T07:26:29.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 -- start start 2024-03-06T07:26:29.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa280c88f0 0x7ffa28155210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa2815fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa281596f0 con 0x7ffa281527f0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa280c9130 con 0x7ffa280c88f0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa2e66b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28159260 con 0x7ffa28155750 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa27fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffa281527f0 0x7ffa280c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48528/0 (socket says 172.21.15.17:48528) 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.322+0000 7ffa27fff640 1 -- 172.21.15.17:0/2634315459 learned_addr learned my addr 172.21.15.17:0/2634315459 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa2cc08640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa2815fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299034681 0 0) 0x7ffa281596f0 con 0x7ffa281527f0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa277fe640 1 --2- 172.21.15.17:0/2634315459 >> v2:172.21.15.163:3300/0 conn(0x7ffa280c88f0 0x7ffa28155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa10003520 con 0x7ffa281527f0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 302213832 0 0) 0x7ffa10003520 con 0x7ffa281527f0 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 msgr2=0x7ffa2815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa2815fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 >> v2:172.21.15.163:3300/0 conn(0x7ffa280c88f0 msgr2=0x7ffa28155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 --2- 172.21.15.17:0/2634315459 >> v2:172.21.15.163:3300/0 conn(0x7ffa280c88f0 0x7ffa28155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280c82a0 con 0x7ffa281527f0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ffa18001670 con 0x7ffa281527f0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 >> v1:172.21.15.17:6789/0 conn(0x7ffa281527f0 legacy=0x7ffa280c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa100039b0 0x7ffa10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa10004d40 0x7ffa100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c9130 con 0x7ffa100039b0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28159260 con 0x7ffa10004d40 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1000ba40 con 0x7ffa28155750 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa27fff640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa100039b0 0x7ffa10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa277fe640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa2cc08640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa10004d40 0x7ffa100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa27fff640 1 -- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 msgr2=0x7ffa10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.323+0000 7ffa27fff640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa27fff640 1 -- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa10004d40 msgr2=0x7ffa100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa27fff640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa10004d40 0x7ffa100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa27fff640 1 -- 172.21.15.17:0/2634315459 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10003520 con 0x7ffa100039b0 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa277fe640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa28155750 0x7ffa10004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa2cc08640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa10004d40 0x7ffa100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa27fff640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa100039b0 0x7ffa10003dd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001050 tx=0x7ffa10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2c8b2c661bd424 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18007d70 con 0x7ffa100039b0 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18030070 con 0x7ffa100039b0 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.324+0000 7ffa26ffd640 1 -- 172.21.15.17:0/2634315459 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1802cd80 con 0x7ffa100039b0 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 -- 172.21.15.17:0/2634315459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa100039b0 msgr2=0x7ffa10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/2634315459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa100039b0 0x7ffa10003dd0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7ffa18001050 tx=0x7ffa10005830 comp rx=0 tx=0).stop 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2cc08640 1 -- 172.21.15.17:0/2634315459 reap_dead start 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 -- 172.21.15.17:0/2634315459 shutdown_connections 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 -- 172.21.15.17:0/2634315459 >> 172.21.15.17:0/2634315459 conn(0x7ffa280ac580 msgr2=0x7ffa280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 -- 172.21.15.17:0/2634315459 shutdown_connections 2024-03-06T07:26:29.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.325+0000 7ffa2e66b640 1 -- 172.21.15.17:0/2634315459 wait complete. 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.326+0000 7ffa2e66b640 1 Processor -- start 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.326+0000 7ffa2e66b640 1 -- start start 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa100039b0 0x7ffa28157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa281ee850 0x7ffa28147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28158c10 con 0x7ffa280c4830 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c6b90 con 0x7ffa100039b0 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa2e66b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28158ef0 con 0x7ffa281ee850 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59572/0 (socket says 172.21.15.17:59572) 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 -- 172.21.15.17:0/1251075297 learned_addr learned my addr 172.21.15.17:0/1251075297 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:29.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 -- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa281ee850 msgr2=0x7ffa28147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa281ee850 0x7ffa28147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 -- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa100039b0 msgr2=0x7ffa28157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa100039b0 0x7ffa28157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.327+0000 7ffa277fe640 1 -- 172.21.15.17:0/1251075297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2814d3c0 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa277fe640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0116d0 tx=0x7ffa1c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00f940 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c00fae0 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2814d6a0 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c015460 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.328+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa28158160 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.329+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2804f330 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.329+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa1c015600 con 0x7ffa280c4830 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.329+0000 7ffa24ff9640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 0x7ffa1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.330+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7ffa1c08a970 con 0x7ffa280c4830 2024-03-06T07:26:29.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.330+0000 7ffa27fff640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 0x7ffa1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.330+0000 7ffa27fff640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 0x7ffa1406d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.334+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa1c019030 con 0x7ffa280c4830 2024-03-06T07:26:29.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.518+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"} v 0) v1 -- 0x7ffa280c8080 con 0x7ffa280c4830 2024-03-06T07:26:29.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.521+0000 7ffa24ff9640 1 -- 172.21.15.17:0/1251075297 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.d", "name": "mds_join_fs"}]=0 v54) v1 ==== 96+0+0 (secure 0 0 0) 0x7ffa1c0598a0 con 0x7ffa280c4830 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 msgr2=0x7ffa1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 0x7ffa1406d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ffa18034000 tx=0x7ffa1802e000 comp rx=0 tx=0).stop 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 msgr2=0x7ffa281ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0116d0 tx=0x7ffa1c011b90 comp rx=0 tx=0).stop 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 shutdown_connections 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffa1406b500 0x7ffa1406d9c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa281ee850 0x7ffa28147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa280c4830 0x7ffa281ee310 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 --2- 172.21.15.17:0/1251075297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa100039b0 0x7ffa28157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 >> 172.21.15.17:0/1251075297 conn(0x7ffa280ac580 msgr2=0x7ffa280c7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 shutdown_connections 2024-03-06T07:26:29.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.523+0000 7ffa2e66b640 1 -- 172.21.15.17:0/1251075297 wait complete. 2024-03-06T07:26:29.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:26:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.636+0000 7f931f7bd640 1 Processor -- start 2024-03-06T07:26:29.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.636+0000 7f931f7bd640 1 -- start start 2024-03-06T07:26:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931f7bd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9318155c30 0x7f931806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931f7bd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93180b0450 con 0x7f93180c5380 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931f7bd640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93181560e0 con 0x7f93180b0070 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931f7bd640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931806cfb0 con 0x7f9318155c30 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931dd5a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f93180c5380 0x7f931814e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48530/0 (socket says 172.21.15.17:48530) 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.637+0000 7f931dd5a640 1 -- 172.21.15.17:0/2211363992 learned_addr learned my addr 172.21.15.17:0/2211363992 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931cd58640 1 --2- 172.21.15.17:0/2211363992 >> v2:172.21.15.163:3300/0 conn(0x7f9318155c30 0x7f931806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116868531 0 0) 0x7f9308001490 con 0x7f93180c5380 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9300003520 con 0x7f93180c5380 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 370123274 0 0) 0x7f9300003520 con 0x7f93180c5380 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 >> v2:172.21.15.163:3300/0 conn(0x7f9318155c30 msgr2=0x7f931806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 --2- 172.21.15.17:0/2211363992 >> v2:172.21.15.163:3300/0 conn(0x7f9318155c30 0x7f931806ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 >> v1:172.21.15.163:6789/0 conn(0x7f93180b0070 legacy=0x7f93180c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180c77f0 con 0x7f93180c5380 2024-03-06T07:26:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9308001bc0 con 0x7f93180c5380 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 >> v1:172.21.15.17:6789/0 conn(0x7f93180c5380 legacy=0x7f931814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93000039b0 0x7f9300003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9300004800 0x7f9300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f930000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931806cfb0 con 0x7f93000051c0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930000c160 con 0x7f93000039b0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930000c360 con 0x7f9300004800 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f930000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931d559640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93000039b0 0x7f9300003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931cd58640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9300004800 0x7f9300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 -- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9300004800 msgr2=0x7f9300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9300004800 0x7f9300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 -- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93000039b0 msgr2=0x7f9300003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93000039b0 0x7f9300003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931dd5a640 1 -- 172.21.15.17:0/2211363992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9300003520 con 0x7f93000051c0 2024-03-06T07:26:29.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.638+0000 7f931d559640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93000039b0 0x7f9300003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.639+0000 7f931dd5a640 1 -- 172.21.15.17:0/2211363992 reap_dead start 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.639+0000 7f931dd5a640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f930000b9c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f930800aa30 tx=0x7f9300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6828f41410be7cde server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.639+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308038070 con 0x7f93000051c0 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.639+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9308033620 con 0x7f93000051c0 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.639+0000 7f9317fff640 1 -- 172.21.15.17:0/2211363992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310001370 con 0x7f93000051c0 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 -- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 msgr2=0x7f930000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f930000b9c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f930800aa30 tx=0x7f9300005830 comp rx=0 tx=0).stop 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 -- 172.21.15.17:0/2211363992 shutdown_connections 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 --2- 172.21.15.17:0/2211363992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f930000b9c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 -- 172.21.15.17:0/2211363992 >> 172.21.15.17:0/2211363992 conn(0x7f93180ac590 msgr2=0x7f93180ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 -- 172.21.15.17:0/2211363992 shutdown_connections 2024-03-06T07:26:29.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.640+0000 7f931f7bd640 1 -- 172.21.15.17:0/2211363992 wait complete. 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.641+0000 7f931f7bd640 1 Processor -- start 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.641+0000 7f931f7bd640 1 -- start start 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931814f090 0x7f9318140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931814f5e0 0x7f9318141120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931806c270 con 0x7f93000051c0 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931806c020 con 0x7f931814f5e0 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931806c550 con 0x7f931814f090 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59588/0 (socket says 172.21.15.17:59588) 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 -- 172.21.15.17:0/808350702 learned_addr learned my addr 172.21.15.17:0/808350702 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:29.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931dd5a640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931814f5e0 0x7f9318141120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931cd58640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931814f090 0x7f9318140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 -- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931814f090 msgr2=0x7f9318140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931814f090 0x7f9318140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 -- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931814f5e0 msgr2=0x7f9318141120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931814f5e0 0x7f9318141120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 -- 172.21.15.17:0/808350702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180c24e0 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931d559640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f931000f650 tx=0x7f931000fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100052e0 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310018070 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.642+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180c27c0 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.643+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310003830 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.643+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180c2d80 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.644+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931804f330 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.644+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93100039d0 con 0x7f93000051c0 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.644+0000 7f9315ffb640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 0x7f930406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.645+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f9310089ce0 con 0x7f93000051c0 2024-03-06T07:26:29.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.645+0000 7f931cd58640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 0x7f930406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.645+0000 7f931cd58640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 0x7f930406d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f93180c4d20 tx=0x7f930c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.649+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9310058c10 con 0x7f93000051c0 2024-03-06T07:26:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.830+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f931806d860 con 0x7f93000051c0 2024-03-06T07:26:29.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.833+0000 7f9315ffb640 1 -- 172.21.15.17:0/808350702 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9310058510 con 0x7f93000051c0 2024-03-06T07:26:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 msgr2=0x7f930406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 0x7f930406d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f93180c4d20 tx=0x7f930c009190 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 msgr2=0x7f93180c43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f931000f650 tx=0x7f931000fa50 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 shutdown_connections 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f930406b4d0 0x7f930406d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f931814f5e0 0x7f9318141120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f931814f090 0x7f9318140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 --2- 172.21.15.17:0/808350702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93000051c0 0x7f93180c43d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 >> 172.21.15.17:0/808350702 conn(0x7f93180ac590 msgr2=0x7f9318154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 shutdown_connections 2024-03-06T07:26:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.835+0000 7f931f7bd640 1 -- 172.21.15.17:0/808350702 wait complete. 2024-03-06T07:26:29.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds.a mds_join_fs 2024-03-06T07:26:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.949+0000 7feaf813a640 1 Processor -- start 2024-03-06T07:26:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.949+0000 7feaf813a640 1 -- start start 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf813a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feaf0152730 0x7feaf00c3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf813a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf813a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf0152b10 con 0x7feaf00c6710 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf813a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf0156110 con 0x7feaf0152730 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf813a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0158ad0 con 0x7feaf00c3750 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feaf0152730 0x7feaf00c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feaf0152730 0x7feaf00c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59604/0 (socket says 172.21.15.17:59604) 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 -- 172.21.15.17:0/4096888064 learned_addr learned my addr 172.21.15.17:0/4096888064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf66d7640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 -- 172.21.15.17:0/4096888064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 msgr2=0x7feaf0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf0155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 -- 172.21.15.17:0/4096888064 >> v1:172.21.15.163:6789/0 conn(0x7feaf00c6710 legacy=0x7feaf01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.950+0000 7feaf5ed6640 1 -- 172.21.15.17:0/4096888064 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0158db0 con 0x7feaf0152730 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf5ed6640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0152730 0x7feaf00c3210 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7feaec009870 tx=0x7feaec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e8ea616f45567d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf4ed4640 1 -- 172.21.15.17:0/4096888064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec035020 con 0x7feaf0152730 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf4ed4640 1 -- 172.21.15.17:0/4096888064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec031640 con 0x7feaf0152730 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf4ed4640 1 -- 172.21.15.17:0/4096888064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec02d040 con 0x7feaf0152730 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf813a640 1 -- 172.21.15.17:0/4096888064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0152730 msgr2=0x7feaf00c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.951+0000 7feaf813a640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0152730 0x7feaf00c3210 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7feaec009870 tx=0x7feaec02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 -- 172.21.15.17:0/4096888064 shutdown_connections 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf0155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 --2- 172.21.15.17:0/4096888064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0152730 0x7feaf00c3210 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 -- 172.21.15.17:0/4096888064 >> 172.21.15.17:0/4096888064 conn(0x7feaf00aa5c0 msgr2=0x7feaf00aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 -- 172.21.15.17:0/4096888064 shutdown_connections 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.952+0000 7feaf813a640 1 -- 172.21.15.17:0/4096888064 wait complete. 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 Processor -- start 2024-03-06T07:26:29.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 -- start start 2024-03-06T07:26:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf01eae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0152730 0x7feaf01eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf01586f0 con 0x7feaf00c6710 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0159370 con 0x7feaf0152730 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.953+0000 7feaf813a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0158870 con 0x7feaf00c3750 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59608/0 (socket says 172.21.15.17:59608) 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 -- 172.21.15.17:0/1531373556 learned_addr learned my addr 172.21.15.17:0/1531373556 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf5ed6640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf01eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf66d7640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0152730 0x7feaf01eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 -- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 msgr2=0x7feaf01eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf01eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 -- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0152730 msgr2=0x7feaf01eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0152730 0x7feaf01eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 -- 172.21.15.17:0/1531373556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf014c470 con 0x7feaf00c6710 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf56d5640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7feae000c470 tx=0x7feae0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00052d0 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0005470 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf006ebe0 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0007260 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.954+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf014c190 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.955+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf006ede0 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.956+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae0007400 con 0x7feaf00c6710 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.956+0000 7feae67fc640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 0x7fead406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:29.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.956+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7feae0089660 con 0x7feaf00c6710 2024-03-06T07:26:29.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.956+0000 7feaf5ed6640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 0x7fead406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:29.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.957+0000 7feaf5ed6640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 0x7fead406d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7feaec036000 tx=0x7feaec03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:29.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:29.960+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feaf006ede0 con 0x7feaf00c6710 2024-03-06T07:26:30.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.146+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"} v 0) v1 -- 0x7feaf004f330 con 0x7feaf00c6710 2024-03-06T07:26:30.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.149+0000 7feae67fc640 1 -- 172.21.15.17:0/1531373556 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds.a", "name": "mds_join_fs"}]=0 v56) v1 ==== 96+0+0 (secure 0 0 0) 0x7feae0058590 con 0x7feaf00c6710 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 msgr2=0x7fead406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 0x7fead406d9c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7feaec036000 tx=0x7feaec03c040 comp rx=0 tx=0).stop 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 msgr2=0x7feaf0151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7feae000c470 tx=0x7feae0001e70 comp rx=0 tx=0).stop 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 shutdown_connections 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fead406b500 0x7fead406d9c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feaf0152730 0x7feaf01eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf00c6710 0x7feaf0151210 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 --2- 172.21.15.17:0/1531373556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feaf00c3750 0x7feaf01eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 >> 172.21.15.17:0/1531373556 conn(0x7feaf00aa5c0 msgr2=0x7feaf00c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 shutdown_connections 2024-03-06T07:26:30.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.151+0000 7feaf813a640 1 -- 172.21.15.17:0/1531373556 wait complete. 2024-03-06T07:26:30.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla' 2024-03-06T07:26:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.263+0000 7f73dbf32640 1 Processor -- start 2024-03-06T07:26:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73dbf32640 1 -- start start 2024-03-06T07:26:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73dbf32640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f73d406d9f0 0x7f73d40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73dbf32640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d4158800 con 0x7f73d40c3710 2024-03-06T07:26:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73dbf32640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d4158ae0 con 0x7f73d40c3280 2024-03-06T07:26:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73dbf32640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d4158dc0 con 0x7f73d406d9f0 2024-03-06T07:26:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73d9cce640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f73d406d9f0 0x7f73d40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73da4cf640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f73d40c3710 0x7f73d41558f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48544/0 (socket says 172.21.15.17:48544) 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.264+0000 7f73da4cf640 1 -- 172.21.15.17:0/4166790621 learned_addr learned my addr 172.21.15.17:0/4166790621 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.265+0000 7f73d9cce640 1 -- 172.21.15.17:0/4166790621 >> v1:172.21.15.163:6789/0 conn(0x7f73d40c3280 legacy=0x7f73d41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.265+0000 7f73d9cce640 1 -- 172.21.15.17:0/4166790621 >> v1:172.21.15.17:6789/0 conn(0x7f73d40c3710 legacy=0x7f73d41558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.265+0000 7f73d8ccc640 1 -- 172.21.15.17:0/4166790621 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530231544 0 0) 0x7f73d4158800 con 0x7f73d40c3710 2024-03-06T07:26:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.265+0000 7f73d9cce640 1 -- 172.21.15.17:0/4166790621 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d41590a0 con 0x7f73d406d9f0 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.265+0000 7f73d9cce640 1 --2- 172.21.15.17:0/4166790621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d40a7cd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0097a0 tx=0x7f73cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83714f577a5d082a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73d8ccc640 1 -- 172.21.15.17:0/4166790621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0315a0 con 0x7f73d406d9f0 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73d8ccc640 1 -- 172.21.15.17:0/4166790621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc037070 con 0x7f73d406d9f0 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73d8ccc640 1 -- 172.21.15.17:0/4166790621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc031960 con 0x7f73d406d9f0 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 -- 172.21.15.17:0/4166790621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 msgr2=0x7f73d40a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 --2- 172.21.15.17:0/4166790621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d40a7cd0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0097a0 tx=0x7f73cc02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 -- 172.21.15.17:0/4166790621 shutdown_connections 2024-03-06T07:26:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 --2- 172.21.15.17:0/4166790621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d40a7cd0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 -- 172.21.15.17:0/4166790621 >> 172.21.15.17:0/4166790621 conn(0x7f73d40aa630 msgr2=0x7f73d40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:30.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 -- 172.21.15.17:0/4166790621 shutdown_connections 2024-03-06T07:26:30.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.266+0000 7f73dbf32640 1 -- 172.21.15.17:0/4166790621 wait complete. 2024-03-06T07:26:30.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.267+0000 7f73dbf32640 1 Processor -- start 2024-03-06T07:26:30.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.267+0000 7f73dbf32640 1 -- start start 2024-03-06T07:26:30.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 0x7f73d414af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d40c3710 0x7f73d414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d406df50 con 0x7f73d406d9f0 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d406ddd0 con 0x7f73d40c3280 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73dbf32640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d40a8210 con 0x7f73d40c3710 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59628/0 (socket says 172.21.15.17:59628) 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 -- 172.21.15.17:0/3602460297 learned_addr learned my addr 172.21.15.17:0/3602460297 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d94cd640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 0x7f73d414af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 -- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d40c3710 msgr2=0x7f73d414b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73da4cf640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d40c3710 0x7f73d414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d40c3710 0x7f73d414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 -- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 msgr2=0x7f73d414af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 0x7f73d414af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d9cce640 1 -- 172.21.15.17:0/3602460297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d406e350 con 0x7f73d406d9f0 2024-03-06T07:26:30.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.268+0000 7f73d94cd640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 0x7f73d414af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73d9cce640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0098c0 tx=0x7f73cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc032840 con 0x7f73d406d9f0 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc0329e0 con 0x7f73d406d9f0 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc032dd0 con 0x7f73d406d9f0 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d40a7f80 con 0x7f73d406d9f0 2024-03-06T07:26:30.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.269+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d41542e0 con 0x7f73d406d9f0 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.270+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d404f330 con 0x7f73d406d9f0 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.270+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73cc04d3f0 con 0x7f73d406d9f0 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.271+0000 7f73ca7fc640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 0x7f73b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.271+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f73cc02f030 con 0x7f73d406d9f0 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.271+0000 7f73d94cd640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 0x7f73b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:30.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.271+0000 7f73d94cd640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 0x7f73b806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f73d4072490 tx=0x7f73c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:30.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.275+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73cc040070 con 0x7f73d406d9f0 2024-03-06T07:26:30.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:30.524+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]} v 0) v1 -- 0x7f73d4073240 con 0x7f73d406d9f0 2024-03-06T07:26:31.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.199+0000 7f73ca7fc640 1 -- 172.21.15.17:0/3602460297 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterAffinity.test_join_fs_vanilla"]}]=0 v381) v1 ==== 146+0+0 (secure 0 0 0) 0x7f73cc07ada0 con 0x7f73d406d9f0 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 msgr2=0x7f73b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 0x7f73b806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f73d4072490 tx=0x7f73c4009040 comp rx=0 tx=0).stop 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 msgr2=0x7f73d414d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0098c0 tx=0x7f73cc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 shutdown_connections 2024-03-06T07:26:31.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f73b806b4d0 0x7f73b806d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73d40c3710 0x7f73d414b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73d40c3280 0x7f73d414af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 --2- 172.21.15.17:0/3602460297 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73d406d9f0 0x7f73d414d110 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 >> 172.21.15.17:0/3602460297 conn(0x7f73d40aa630 msgr2=0x7f73d40c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 shutdown_connections 2024-03-06T07:26:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.201+0000 7f73dbf32640 1 -- 172.21.15.17:0/3602460297 wait complete. 2024-03-06T07:26:31.216 INFO:tasks.cephfs_test_runner:That a vanilla standby is preferred over others with mds_join_fs set to another fs. ... ok 2024-03-06T07:26:31.217 INFO:tasks.cephfs_test_runner:Starting test: test_down (tasks.cephfs.test_failover.TestClusterResize) That down setting toggles and sets max_mds appropriately. 2024-03-06T07:26:31.217 INFO:tasks.cephfs_test_runner:test_down (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:26:31.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:31.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.323+0000 7f35dbfd7640 1 Processor -- start 2024-03-06T07:26:31.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35dbfd7640 1 -- start start 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35dbfd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35d4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35dbfd7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40c73d0 con 0x7f35d414f360 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35dbfd7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40be2e0 con 0x7f35d4158b60 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35dbfd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4159700 con 0x7f35d40c88c0 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.324+0000 7f35d9d73640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f35d414f360 0x7f35d40c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54230/0 (socket says 172.21.15.17:54230) 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d9d73640 1 -- 172.21.15.17:0/446832336 learned_addr learned my addr 172.21.15.17:0/446832336 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d9572640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35d4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934999418 0 0) 0x7f35d40c73d0 con 0x7f35d414f360 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c0003520 con 0x7f35d414f360 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313199606 0 0) 0x7f35d40be2e0 con 0x7f35d4158b60 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d40c73d0 con 0x7f35d4158b60 2024-03-06T07:26:31.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044007630 0 0) 0x7f35c0003520 con 0x7f35d414f360 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 msgr2=0x7f35d4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35d4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 >> v1:172.21.15.163:6789/0 conn(0x7f35d4158b60 legacy=0x7f35d4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40c7670 con 0x7f35d414f360 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f35d0001670 con 0x7f35d414f360 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 >> v1:172.21.15.17:6789/0 conn(0x7f35d414f360 legacy=0x7f35d40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c0003b70 0x7f35c0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35c00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4159700 con 0x7f35c0003b70 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c000bad0 con 0x7f35c0004800 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c000bcd0 con 0x7f35d40c88c0 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d9d73640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c0003b70 0x7f35c0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.325+0000 7f35d9572640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35da574640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35c00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 -- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 msgr2=0x7f35c00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35d40c88c0 0x7f35c00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 -- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35c0004800 msgr2=0x7f35c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35c0004800 0x7f35c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 -- 172.21.15.17:0/446832336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0003520 con 0x7f35c0003b70 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d9d73640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c0003b70 0x7f35c0003f70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c190 tx=0x7f35c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8d972769479af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0007ce0 con 0x7f35c0003b70 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d0030070 con 0x7f35c0003b70 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.326+0000 7f35d8d71640 1 -- 172.21.15.17:0/446832336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0005d50 con 0x7f35c0003b70 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 -- 172.21.15.17:0/446832336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c0003b70 msgr2=0x7f35c0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/446832336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35c0003b70 0x7f35c0003f70 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c190 tx=0x7f35c0005830 comp rx=0 tx=0).stop 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35da574640 1 -- 172.21.15.17:0/446832336 reap_dead start 2024-03-06T07:26:31.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 -- 172.21.15.17:0/446832336 shutdown_connections 2024-03-06T07:26:31.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 -- 172.21.15.17:0/446832336 >> 172.21.15.17:0/446832336 conn(0x7f35d40ac570 msgr2=0x7f35d40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 -- 172.21.15.17:0/446832336 shutdown_connections 2024-03-06T07:26:31.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.327+0000 7f35dbfd7640 1 -- 172.21.15.17:0/446832336 wait complete. 2024-03-06T07:26:31.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.328+0000 7f35dbfd7640 1 Processor -- start 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.328+0000 7f35dbfd7640 1 -- start start 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 0x7f35d41eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 0x7f35d4157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c7c30 con 0x7f35d40c4b60 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c5a50 con 0x7f35d4156d80 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35dbfd7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40c5bd0 con 0x7f35c0003b70 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49940/0 (socket says 172.21.15.17:49940) 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 -- 172.21.15.17:0/3076082927 learned_addr learned my addr 172.21.15.17:0/3076082927 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9d73640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 0x7f35d41eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35da574640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 0x7f35d4157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 -- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 msgr2=0x7f35d41eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 0x7f35d41eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 -- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 msgr2=0x7f35d4157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 0x7f35d4157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 -- 172.21.15.17:0/3076082927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d41ef540 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35da574640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 0x7f35d4157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9d73640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 0x7f35d41eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.329+0000 7f35d9572640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f35c4011560 tx=0x7f35c4011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.330+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4004220 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.330+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c40043c0 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.330+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40abca0 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.330+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c4015870 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.330+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40ac260 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.331+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d404f330 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.331+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35c4015a10 con 0x7f35d40c4b60 2024-03-06T07:26:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.332+0000 7f35ca7fc640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 0x7f35b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.332+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f35c408a3c0 con 0x7f35d40c4b60 2024-03-06T07:26:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.332+0000 7f35d9d73640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 0x7f35b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.332+0000 7f35d9d73640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 0x7f35b806d9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f35d0034000 tx=0x7f35d0007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.336+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35c4019030 con 0x7f35d40c4b60 2024-03-06T07:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.515+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35d40c8d50 con 0x7f35d40c4b60 2024-03-06T07:26:31.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.516+0000 7f35ca7fc640 1 -- 172.21.15.17:0/3076082927 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+13819 (secure 0 0 0) 0x7f35c40592f0 con 0x7f35d40c4b60 2024-03-06T07:26:31.518 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:31.518 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":163,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:29.192900+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/2508903849":"2024-03-07T07:26:26.185069+0000","172.21.15.163:6836/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.17:6835/2508903849":"2024-03-07T07:26:26.185069+0000","172.21.15.163:6837/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.163:6835/2502302602":"2024-03-07T07:26:19.872514+0000","172.21.15.17:6834/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.163:6834/91031835":"2024-03-07T07:26:20.878241+0000","172.21.15.17:6832/81289984":"2024-03-07T07:26:10.717582+0000","172.21.15.17:6835/2429596225":"2024-03-07T07:26:10.712917+0000","172.21.15.17:6833/2429596225":"2024-03-07T07:26:10.712917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 msgr2=0x7f35b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 0x7f35b806d9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f35d0034000 tx=0x7f35d0007000 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 msgr2=0x7f35d41ef000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f35c4011560 tx=0x7f35c4011a20 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 shutdown_connections 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f35b806b500 0x7f35b806d9c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35d4156d80 0x7f35d4157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35d40c4b60 0x7f35d41ef000 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 --2- 172.21.15.17:0/3076082927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35c0003b70 0x7f35d41eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 >> 172.21.15.17:0/3076082927 conn(0x7f35d40ac570 msgr2=0x7f35d414de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 shutdown_connections 2024-03-06T07:26:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.518+0000 7f35dbfd7640 1 -- 172.21.15.17:0/3076082927 wait complete. 2024-03-06T07:26:31.530 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:31.530 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 Processor -- start 2024-03-06T07:26:31.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 -- start start 2024-03-06T07:26:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5114155c30 0x7f511406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51140b0450 con 0x7f51140b0070 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51141560e0 con 0x7f51140c5380 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f51188c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511406cfb0 con 0x7f5114155c30 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f510bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5114155c30 0x7f511406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f511259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f51140b0070 0x7f51140c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54242/0 (socket says 172.21.15.17:54242) 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.633+0000 7f511259c640 1 -- 172.21.15.17:0/3545041382 learned_addr learned my addr 172.21.15.17:0/3545041382 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.634+0000 7f5111d9b640 1 -- 172.21.15.17:0/3545041382 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140249286 0 0) 0x7f5104001110 con 0x7f51140b0070 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.634+0000 7f5111d9b640 1 -- 172.21.15.17:0/3545041382 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5100003520 con 0x7f51140b0070 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.634+0000 7f510bfff640 1 -- 172.21.15.17:0/3545041382 >> v1:172.21.15.163:6789/0 conn(0x7f51140c5380 legacy=0x7f511414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.634+0000 7f510bfff640 1 -- 172.21.15.17:0/3545041382 >> v1:172.21.15.17:6789/0 conn(0x7f51140b0070 legacy=0x7f51140c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.634+0000 7f510bfff640 1 -- 172.21.15.17:0/3545041382 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140c77f0 con 0x7f5114155c30 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f510bfff640 1 --2- 172.21.15.17:0/3545041382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5114155c30 0x7f511406ca70 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71428bf7cade1a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f5111d9b640 1 -- 172.21.15.17:0/3545041382 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0315a0 con 0x7f5114155c30 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f5111d9b640 1 -- 172.21.15.17:0/3545041382 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc037070 con 0x7f5114155c30 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f5111d9b640 1 -- 172.21.15.17:0/3545041382 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc031960 con 0x7f5114155c30 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 -- 172.21.15.17:0/3545041382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5114155c30 msgr2=0x7f511406ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 --2- 172.21.15.17:0/3545041382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5114155c30 0x7f511406ca70 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 -- 172.21.15.17:0/3545041382 shutdown_connections 2024-03-06T07:26:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 --2- 172.21.15.17:0/3545041382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5114155c30 0x7f511406ca70 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 -- 172.21.15.17:0/3545041382 >> 172.21.15.17:0/3545041382 conn(0x7f51140ac590 msgr2=0x7f51140ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 -- 172.21.15.17:0/3545041382 shutdown_connections 2024-03-06T07:26:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.635+0000 7f51188c2640 1 -- 172.21.15.17:0/3545041382 wait complete. 2024-03-06T07:26:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 Processor -- start 2024-03-06T07:26:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 -- start start 2024-03-06T07:26:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51140b0070 0x7f511414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5114155c30 0x7f51141ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511406c270 con 0x7f51140c5380 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511406c020 con 0x7f51140b0070 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f51188c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511406c550 con 0x7f5114155c30 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f510bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f510bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49956/0 (socket says 172.21.15.17:49956) 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.637+0000 7f510bfff640 1 -- 172.21.15.17:0/2854386576 learned_addr learned my addr 172.21.15.17:0/2854386576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 -- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5114155c30 msgr2=0x7f51141ee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f5112d9d640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5114155c30 0x7f51141ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f511259c640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51140b0070 0x7f511414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5114155c30 0x7f51141ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 -- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51140b0070 msgr2=0x7f511414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51140b0070 0x7f511414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 -- 172.21.15.17:0/2854386576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140c4b00 con 0x7f51140c5380 2024-03-06T07:26:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510bfff640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f50fc000c00 tx=0x7f50fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032630 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc0327d0 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51140c4820 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032bc0 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.638+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51141ef1f0 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.639+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511404f330 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.639+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50fc04d3f0 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.640+0000 7f510affd640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 0x7f50f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.640+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f50fc02f030 con 0x7f51140c5380 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.640+0000 7f511259c640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 0x7f50f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.641+0000 7f511259c640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 0x7f50f406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51040012b0 tx=0x7f5104009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.644+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50fc040070 con 0x7f51140c5380 2024-03-06T07:26:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.825+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f511406d860 con 0x7f51140c5380 2024-03-06T07:26:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.826+0000 7f510affd640 1 -- 172.21.15.17:0/2854386576 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50fc07a3e0 con 0x7f51140c5380 2024-03-06T07:26:31.827 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:26:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.827+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 msgr2=0x7f50f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.827+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 0x7f50f406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f51040012b0 tx=0x7f5104009040 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.827+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 msgr2=0x7f51141ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f50fc000c00 tx=0x7f50fc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 shutdown_connections 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f50f406b500 0x7f50f406d9c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5114155c30 0x7f51141ee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51140c5380 0x7f51141ee3d0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 --2- 172.21.15.17:0/2854386576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51140b0070 0x7f511414e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 >> 172.21.15.17:0/2854386576 conn(0x7f51140ac590 msgr2=0x7f51140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 shutdown_connections 2024-03-06T07:26:31.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.828+0000 7f51188c2640 1 -- 172.21.15.17:0/2854386576 wait complete. 2024-03-06T07:26:31.840 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down' 2024-03-06T07:26:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.940+0000 7f8afafac640 1 Processor -- start 2024-03-06T07:26:31.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.940+0000 7f8afafac640 1 -- start start 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8afafac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8af40c5330 0x7f8af40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8afafac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8af4158db0 0x7f8af415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8afafac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af4155110 con 0x7f8af40c88b0 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8afafac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af41559e0 con 0x7f8af40c5330 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8afafac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af40c8f40 con 0x7f8af4158db0 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af8d48640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8af40c5330 0x7f8af40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8af40c88b0 0x7f8af41586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54256/0 (socket says 172.21.15.17:54256) 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af3fff640 1 -- 172.21.15.17:0/2187690598 learned_addr learned my addr 172.21.15.17:0/2187690598 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af9549640 1 --2- 172.21.15.17:0/2187690598 >> v2:172.21.15.163:3300/0 conn(0x7f8af4158db0 0x7f8af415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af8d48640 1 -- 172.21.15.17:0/2187690598 >> v2:172.21.15.163:3300/0 conn(0x7f8af4158db0 msgr2=0x7f8af415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af8d48640 1 --2- 172.21.15.17:0/2187690598 >> v2:172.21.15.163:3300/0 conn(0x7f8af4158db0 0x7f8af415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af8d48640 1 -- 172.21.15.17:0/2187690598 >> v1:172.21.15.17:6789/0 conn(0x7f8af40c88b0 legacy=0x7f8af41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.941+0000 7f8af8d48640 1 -- 172.21.15.17:0/2187690598 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4155390 con 0x7f8af40c5330 2024-03-06T07:26:31.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.942+0000 7f8af8d48640 1 --2- 172.21.15.17:0/2187690598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c5330 0x7f8af40c5710 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f8af40ca870 tx=0x7f8aec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5681951f6dfdb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.942+0000 7f8af37fe640 1 -- 172.21.15.17:0/2187690598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec039070 con 0x7f8af40c5330 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.942+0000 7f8af37fe640 1 -- 172.21.15.17:0/2187690598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec02cbf0 con 0x7f8af40c5330 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.942+0000 7f8af37fe640 1 -- 172.21.15.17:0/2187690598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec034440 con 0x7f8af40c5330 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 -- 172.21.15.17:0/2187690598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c5330 msgr2=0x7f8af40c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 --2- 172.21.15.17:0/2187690598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c5330 0x7f8af40c5710 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f8af40ca870 tx=0x7f8aec02be40 comp rx=0 tx=0).stop 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 -- 172.21.15.17:0/2187690598 shutdown_connections 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 --2- 172.21.15.17:0/2187690598 >> v2:172.21.15.163:3300/0 conn(0x7f8af4158db0 0x7f8af415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 --2- 172.21.15.17:0/2187690598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c5330 0x7f8af40c5710 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 -- 172.21.15.17:0/2187690598 >> 172.21.15.17:0/2187690598 conn(0x7f8af40ac560 msgr2=0x7f8af40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 -- 172.21.15.17:0/2187690598 shutdown_connections 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.943+0000 7f8afafac640 1 -- 172.21.15.17:0/2187690598 wait complete. 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.944+0000 7f8afafac640 1 Processor -- start 2024-03-06T07:26:31.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.944+0000 7f8afafac640 1 -- start start 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 0x7f8af4151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4158380 0x7f8af414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af41529d0 con 0x7f8af40c88b0 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af406d090 con 0x7f8af4158380 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8afafac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4152b50 con 0x7f8af40c5330 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49972/0 (socket says 172.21.15.17:49972) 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 -- 172.21.15.17:0/3406683030 learned_addr learned my addr 172.21.15.17:0/3406683030 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af8d48640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 0x7f8af4151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af9549640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4158380 0x7f8af414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 -- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 msgr2=0x7f8af4151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 0x7f8af4151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 -- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4158380 msgr2=0x7f8af414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4158380 0x7f8af414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 -- 172.21.15.17:0/3406683030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af41ee730 con 0x7f8af40c88b0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af8d48640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 0x7f8af4151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.945+0000 7f8af3fff640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c470 tx=0x7f8ae8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.946+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8017070 con 0x7f8af40c88b0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.946+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae8005ce0 con 0x7f8af40c88b0 2024-03-06T07:26:31.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.946+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af41eea10 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.946+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae80072f0 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.946+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41eefd0 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.947+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af4153460 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.947+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ae801a070 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.947+0000 7f8af17fa640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.948+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f8ae8089560 con 0x7f8af40c88b0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.948+0000 7f8af8d48640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:31.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.948+0000 7f8af8d48640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8aec036000 tx=0x7f8aec030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:31.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:31.952+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8af4153460 con 0x7f8af40c88b0 2024-03-06T07:26:32.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:32.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down"]} v 0) v1 -- 0x7f8af404f330 con 0x7f8af40c88b0 2024-03-06T07:26:33.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.205+0000 7f8af17fa640 1 -- 172.21.15.17:0/3406683030 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down"]}]=0 v383) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8af4155110 con 0x7f8af40c88b0 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 msgr2=0x7f8ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 0x7f8ae006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8aec036000 tx=0x7f8aec030000 comp rx=0 tx=0).stop 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 msgr2=0x7f8af414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800c470 tx=0x7f8ae8001e70 comp rx=0 tx=0).stop 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 shutdown_connections 2024-03-06T07:26:33.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ae006b4d0 0x7f8ae006d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4158380 0x7f8af414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8af40c88b0 0x7f8af414e670 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 --2- 172.21.15.17:0/3406683030 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40c5330 0x7f8af4151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 >> 172.21.15.17:0/3406683030 conn(0x7f8af40ac560 msgr2=0x7f8af41549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 shutdown_connections 2024-03-06T07:26:33.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.208+0000 7f8afafac640 1 -- 172.21.15.17:0/3406683030 wait complete. 2024-03-06T07:26:33.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:26:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.332+0000 7ff516e88640 1 Processor -- start 2024-03-06T07:26:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.332+0000 7ff516e88640 1 -- start start 2024-03-06T07:26:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff516e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5100c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff516e88640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff510155530 0x7ff5101559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff516e88640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff510152dd0 con 0x7ff5101529f0 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff516e88640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff510159710 con 0x7ff510155530 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff516e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c68b0 con 0x7ff5100c5370 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff510155530 0x7ff5101559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff510155530 0x7ff5101559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49986/0 (socket says 172.21.15.17:49986) 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 -- 172.21.15.17:0/439209644 learned_addr learned my addr 172.21.15.17:0/439209644 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff50ffff640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5100c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 -- 172.21.15.17:0/439209644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 msgr2=0x7ff5100c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5100c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 -- 172.21.15.17:0/439209644 >> v1:172.21.15.163:6789/0 conn(0x7ff5101529f0 legacy=0x7ff5100c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff515425640 1 -- 172.21.15.17:0/439209644 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510158b00 con 0x7ff510155530 2024-03-06T07:26:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.333+0000 7ff50ffff640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5100c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff515425640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff510155530 0x7ff5101559c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff508009e50 tx=0x7ff50802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab715a348fefadb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff50f7fe640 1 -- 172.21.15.17:0/439209644 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50802ca70 con 0x7ff510155530 2024-03-06T07:26:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff50f7fe640 1 -- 172.21.15.17:0/439209644 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50802cc10 con 0x7ff510155530 2024-03-06T07:26:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff50f7fe640 1 -- 172.21.15.17:0/439209644 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080343f0 con 0x7ff510155530 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff516e88640 1 -- 172.21.15.17:0/439209644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff510155530 msgr2=0x7ff5101559c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.334+0000 7ff516e88640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff510155530 0x7ff5101559c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff508009e50 tx=0x7ff50802bed0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 -- 172.21.15.17:0/439209644 shutdown_connections 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff510155530 0x7ff5101559c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 --2- 172.21.15.17:0/439209644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5100c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 -- 172.21.15.17:0/439209644 >> 172.21.15.17:0/439209644 conn(0x7ff5100ac580 msgr2=0x7ff5100ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 -- 172.21.15.17:0/439209644 shutdown_connections 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.335+0000 7ff516e88640 1 -- 172.21.15.17:0/439209644 wait complete. 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.336+0000 7ff516e88640 1 Processor -- start 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.336+0000 7ff516e88640 1 -- start start 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.336+0000 7ff516e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5101ee360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 0x7ff510149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101590c0 con 0x7ff5101529f0 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c9000 con 0x7ff510155530 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101593a0 con 0x7ff5100c5370 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49992/0 (socket says 172.21.15.17:49992) 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 -- 172.21.15.17:0/1643550727 learned_addr learned my addr 172.21.15.17:0/1643550727 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff514c24640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5101ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff515425640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 0x7ff510149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 -- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 msgr2=0x7ff5101ee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5101ee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 -- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 msgr2=0x7ff510149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 0x7ff510149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 -- 172.21.15.17:0/1643550727 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5101eec30 con 0x7ff5101529f0 2024-03-06T07:26:33.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff515425640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 0x7ff510149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff514c24640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5101ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50ffff640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e810 tx=0x7ff4fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00f040 con 0x7ff5101529f0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc004030 con 0x7ff5101529f0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5101ec170 con 0x7ff5101529f0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004400 con 0x7ff5101529f0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.337+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5101ec730 con 0x7ff5101529f0 2024-03-06T07:26:33.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.338+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51004f330 con 0x7ff5101529f0 2024-03-06T07:26:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.339+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4fc00c6d0 con 0x7ff5101529f0 2024-03-06T07:26:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.339+0000 7ff50d7fa640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.339+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7ff4fc089f30 con 0x7ff5101529f0 2024-03-06T07:26:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.340+0000 7ff514c24640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.340+0000 7ff514c24640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff504001680 tx=0x7ff504009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.344+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4fc017020 con 0x7ff5101529f0 2024-03-06T07:26:33.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.526+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff5100b0940 con 0x7ff5101529f0 2024-03-06T07:26:33.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.529+0000 7ff50d7fa640 1 -- 172.21.15.17:0/1643550727 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v57)=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff4fc058e60 con 0x7ff5101529f0 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 msgr2=0x7ff4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff504001680 tx=0x7ff504009040 comp rx=0 tx=0).stop 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 msgr2=0x7ff510148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e810 tx=0x7ff4fc00ecd0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 shutdown_connections 2024-03-06T07:26:33.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff510155530 0x7ff510149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5101529f0 0x7ff510148ce0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 --2- 172.21.15.17:0/1643550727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5100c5370 0x7ff5101ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 >> 172.21.15.17:0/1643550727 conn(0x7ff5100ac580 msgr2=0x7ff510153940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.531+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 shutdown_connections 2024-03-06T07:26:33.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.532+0000 7ff516e88640 1 -- 172.21.15.17:0/1643550727 wait complete. 2024-03-06T07:26:33.544 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:26:33.544 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:33.590 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:26:33.639 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:26:33.640 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:26:33.640 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:33.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:26:33.654 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:26:33.655 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:26:33.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:26:33.672 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:26:33.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:33.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.793+0000 7f611073e640 1 Processor -- start 2024-03-06T07:26:33.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.793+0000 7f611073e640 1 -- start start 2024-03-06T07:26:33.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f611073e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f610814f790 0x7f61081529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f611073e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61080c28d0 con 0x7f61080c1aa0 2024-03-06T07:26:33.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f611073e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61080c2bb0 con 0x7f6108155a40 2024-03-06T07:26:33.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f611073e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61080c2680 con 0x7f610814f790 2024-03-06T07:26:33.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610e4da640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f610814f790 0x7f61081529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610dcd9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f61080c1aa0 0x7f61080c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54262/0 (socket says 172.21.15.17:54262) 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610dcd9640 1 -- 172.21.15.17:0/3565284147 learned_addr learned my addr 172.21.15.17:0/3565284147 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610e4da640 1 -- 172.21.15.17:0/3565284147 >> v1:172.21.15.163:6789/0 conn(0x7f6108155a40 legacy=0x7f6108155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610e4da640 1 -- 172.21.15.17:0/3565284147 >> v1:172.21.15.17:6789/0 conn(0x7f61080c1aa0 legacy=0x7f61080c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610e4da640 1 -- 172.21.15.17:0/3565284147 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080c2e90 con 0x7f610814f790 2024-03-06T07:26:33.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.794+0000 7f610d4d8640 1 -- 172.21.15.17:0/3565284147 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896490594 0 0) 0x7f61080c28d0 con 0x7f61080c1aa0 2024-03-06T07:26:33.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.795+0000 7f610e4da640 1 --2- 172.21.15.17:0/3565284147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f610814f790 0x7f61081529d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f61080c2800 tx=0x7f610002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9f7915ab12b62e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.795+0000 7f610d4d8640 1 -- 172.21.15.17:0/3565284147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100035020 con 0x7f610814f790 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.795+0000 7f610d4d8640 1 -- 172.21.15.17:0/3565284147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6100031830 con 0x7f610814f790 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.795+0000 7f610d4d8640 1 -- 172.21.15.17:0/3565284147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6100031c00 con 0x7f610814f790 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 -- 172.21.15.17:0/3565284147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f610814f790 msgr2=0x7f61081529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 --2- 172.21.15.17:0/3565284147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f610814f790 0x7f61081529d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f61080c2800 tx=0x7f610002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 -- 172.21.15.17:0/3565284147 shutdown_connections 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 --2- 172.21.15.17:0/3565284147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f610814f790 0x7f61081529d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 -- 172.21.15.17:0/3565284147 >> 172.21.15.17:0/3565284147 conn(0x7f61080c6710 msgr2=0x7f61080c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 -- 172.21.15.17:0/3565284147 shutdown_connections 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.796+0000 7f611073e640 1 -- 172.21.15.17:0/3565284147 wait complete. 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.797+0000 7f611073e640 1 Processor -- start 2024-03-06T07:26:33.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.797+0000 7f611073e640 1 -- start start 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61080c1aa0 0x7f61081e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 0x7f6108150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080c5e30 con 0x7f6108155a40 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080c0d60 con 0x7f61080c1aa0 2024-03-06T07:26:33.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f611073e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61080c5fb0 con 0x7f610814f790 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50002/0 (socket says 172.21.15.17:50002) 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 -- 172.21.15.17:0/2216857456 learned_addr learned my addr 172.21.15.17:0/2216857456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610e4da640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61080c1aa0 0x7f61081e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610dcd9640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 0x7f6108150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 -- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 msgr2=0x7f6108150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 0x7f6108150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 -- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61080c1aa0 msgr2=0x7f61081e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61080c1aa0 0x7f61081e92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 -- 172.21.15.17:0/2216857456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6108150170 con 0x7f6108155a40 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610dcd9640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 0x7f6108150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:33.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f610ecdb640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b9c0 tx=0x7f60f800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.798+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8019070 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.799+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f800cba0 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.799+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6108150450 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.799+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8014440 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.799+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6108153cb0 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.800+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610804f330 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.800+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60f8007370 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.801+0000 7f6106ffd640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 0x7f60f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.801+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f60f8089d50 con 0x7f6108155a40 2024-03-06T07:26:33.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.801+0000 7f610e4da640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 0x7f60f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:33.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.801+0000 7f610e4da640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 0x7f60f406d9c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6100036000 tx=0x7f610003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:33.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:33.805+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f60f808d030 con 0x7f6108155a40 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.022+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61080c1f40 con 0x7f6108155a40 2024-03-06T07:26:34.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.023+0000 7f6106ffd640 1 -- 172.21.15.17:0/2216857456 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f60f8058c80 con 0x7f6108155a40 2024-03-06T07:26:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:34.025 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":154,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/923114993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":923114993},{"type":"v1","addr":"172.21.15.163:6837","nonce":923114993}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148},{"gid":8548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2327494506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2327494506},{"type":"v1","addr":"172.21.15.163:6836","nonce":2327494506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":154},{"gid":9084,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/4075955916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":4075955916},{"type":"v1","addr":"172.21.15.17:6833","nonce":4075955916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144},{"gid":9330,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/3002837435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":3002837435},{"type":"v1","addr":"172.21.15.17:6835","nonce":3002837435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":154}],"filesystems":[]} 2024-03-06T07:26:34.025 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 154 2024-03-06T07:26:34.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.024+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 msgr2=0x7f60f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.024+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 0x7f60f406d9c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6100036000 tx=0x7f610003c040 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.024+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 msgr2=0x7f6108149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.024+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b9c0 tx=0x7f60f800be80 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 shutdown_connections 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f60f406b500 0x7f60f406d9c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6108155a40 0x7f6108149f60 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f610814f790 0x7f6108150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 --2- 172.21.15.17:0/2216857456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61080c1aa0 0x7f61081e92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 >> 172.21.15.17:0/2216857456 conn(0x7f61080c6710 msgr2=0x7f6108155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 shutdown_connections 2024-03-06T07:26:34.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.025+0000 7f611073e640 1 -- 172.21.15.17:0/2216857456 wait complete. 2024-03-06T07:26:34.037 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:26:34.037 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:26:34.038 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:26:34.038 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:34.038 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:26:34.038 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:26:34.038 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:26:34.038 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:34.038 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:26:34.038 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:26:34.039 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:26:34.039 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:34.039 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:26:34.039 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:26:34.039 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:26:34.039 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:26:34.049 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:26:34.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:26:34.050 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:26:34.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:26:34.052 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:26:34.053 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:26:34.054 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:26:34.054 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:26:34.092 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:26:34.092 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:34.093 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:26:34.093 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:34.095 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:26:34.095 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:34.096 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:26:34.096 DEBUG:teuthology.parallel:result is None 2024-03-06T07:26:34.096 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:26:34.168 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:26:34.168 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:26:34.178 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:26:34.178 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:26:34.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.200+0000 7f4f8af7a640 1 Processor -- start 2024-03-06T07:26:34.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.200+0000 7f4f8af7a640 1 -- start start 2024-03-06T07:26:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f8af7a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 0x7f4f840c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f8af7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f840c8950 0x7f4f840acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f8af7a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f84155130 con 0x7f4f840ad2b0 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f8af7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f84155440 con 0x7f4f840c8950 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f8af7a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f840c5d50 con 0x7f4f840c69f0 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f840c8950 0x7f4f840acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4f840c8950 0x7f4f840acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50044/0 (socket says 172.21.15.17:50044) 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 -- 172.21.15.17:0/1939062422 learned_addr learned my addr 172.21.15.17:0/1939062422 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f88d16640 1 --2- 172.21.15.17:0/1939062422 >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 0x7f4f840c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 -- 172.21.15.17:0/1939062422 >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 msgr2=0x7f4f840c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 --2- 172.21.15.17:0/1939062422 >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 0x7f4f840c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 -- 172.21.15.17:0/1939062422 >> v1:172.21.15.163:6789/0 conn(0x7f4f840ad2b0 legacy=0x7f4f841558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f83fff640 1 -- 172.21.15.17:0/1939062422 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840c6030 con 0x7f4f840c8950 2024-03-06T07:26:34.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.201+0000 7f4f88d16640 1 --2- 172.21.15.17:0/1939062422 >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 0x7f4f840c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f83fff640 1 --2- 172.21.15.17:0/1939062422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c8950 0x7f4f840acd70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4f840aa430 tx=0x7f4f7002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca1556d755b71d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f837fe640 1 -- 172.21.15.17:0/1939062422 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7002e070 con 0x7f4f840c8950 2024-03-06T07:26:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f837fe640 1 -- 172.21.15.17:0/1939062422 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70037070 con 0x7f4f840c8950 2024-03-06T07:26:34.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f837fe640 1 -- 172.21.15.17:0/1939062422 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70031a00 con 0x7f4f840c8950 2024-03-06T07:26:34.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1939062422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c8950 msgr2=0x7f4f840acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1939062422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c8950 0x7f4f840acd70 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f4f840aa430 tx=0x7f4f7002b880 comp rx=0 tx=0).stop 2024-03-06T07:26:34.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.202+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1939062422 shutdown_connections 2024-03-06T07:26:34.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.203+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1939062422 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c8950 0x7f4f840acd70 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.203+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1939062422 >> v2:172.21.15.163:3300/0 conn(0x7f4f840c69f0 0x7f4f840c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.203+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1939062422 >> 172.21.15.17:0/1939062422 conn(0x7f4f840cae70 msgr2=0x7f4f840cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.203+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1939062422 shutdown_connections 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.203+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1939062422 wait complete. 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 Processor -- start 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 -- start start 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f840ad2b0 0x7f4f8406bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f840c8950 0x7f4f840c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84152a20 con 0x7f4f840c69f0 2024-03-06T07:26:34.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84155fe0 con 0x7f4f840ad2b0 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f8af7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84152ba0 con 0x7f4f840c8950 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.204+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50056/0 (socket says 172.21.15.17:50056) 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 -- 172.21.15.17:0/1712643247 learned_addr learned my addr 172.21.15.17:0/1712643247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f88d16640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f840ad2b0 0x7f4f8406bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f89517640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f840c8950 0x7f4f840c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 -- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f840c8950 msgr2=0x7f4f840c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f840c8950 0x7f4f840c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 -- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f840ad2b0 msgr2=0x7f4f8406bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f840ad2b0 0x7f4f8406bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 -- 172.21.15.17:0/1712643247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840adc00 con 0x7f4f840c69f0 2024-03-06T07:26:34.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f83fff640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002b8c0 tx=0x7f4f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70032880 con 0x7f4f840c69f0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70032a20 con 0x7f4f840c69f0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.205+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70032e10 con 0x7f4f840c69f0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.206+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f840adee0 con 0x7f4f840c69f0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.206+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f840ae4a0 con 0x7f4f840c69f0 2024-03-06T07:26:34.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.207+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8404f330 con 0x7f4f840c69f0 2024-03-06T07:26:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.210+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f7004c020 con 0x7f4f840c69f0 2024-03-06T07:26:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.210+0000 7f4f817fa640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 0x7f4f6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.210+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5679+0+0 (secure 0 0 0) 0x7f4f700ac910 con 0x7f4f840c69f0 2024-03-06T07:26:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.210+0000 7f4f88d16640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 0x7f4f6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:34.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.211+0000 7f4f88d16640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 0x7f4f6806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4f8414d0a0 tx=0x7f4f78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:34.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.213+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4f70053070 con 0x7f4f840c69f0 2024-03-06T07:26:34.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:34.409+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4f840ca650 con 0x7f4f840c69f0 2024-03-06T07:26:35.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.228+0000 7f4f817fa640 1 -- 172.21.15.17:0/1712643247 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v168) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4f70040070 con 0x7f4f840c69f0 2024-03-06T07:26:35.230 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:26:35.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.230+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 msgr2=0x7f4f6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.230+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 0x7f4f6806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4f8414d0a0 tx=0x7f4f78008040 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.230+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 msgr2=0x7f4f840c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.230+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f4f7002b8c0 tx=0x7f4f700046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 shutdown_connections 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4f6806b4d0 0x7f4f6806d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f840c8950 0x7f4f840c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f840c69f0 0x7f4f840c1cd0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 --2- 172.21.15.17:0/1712643247 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f840ad2b0 0x7f4f8406bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 >> 172.21.15.17:0/1712643247 conn(0x7f4f840cae70 msgr2=0x7f4f84150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 shutdown_connections 2024-03-06T07:26:35.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.231+0000 7f4f8af7a640 1 -- 172.21.15.17:0/1712643247 wait complete. 2024-03-06T07:26:35.246 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:26:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.353+0000 7f716c688640 1 Processor -- start 2024-03-06T07:26:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.353+0000 7f716c688640 1 -- start start 2024-03-06T07:26:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716c688640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f716406d9f0 0x7f71640a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716c688640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71640c3280 0x7f71640c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716c688640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7164158800 con 0x7f71641551e0 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716c688640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7164158ae0 con 0x7f71640c3280 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716c688640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7164158dc0 con 0x7f716406d9f0 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71640c3280 0x7f71640c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71640c3280 0x7f71640c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50060/0 (socket says 172.21.15.17:50060) 2024-03-06T07:26:35.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 -- 172.21.15.17:0/1214026504 learned_addr learned my addr 172.21.15.17:0/1214026504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f716a424640 1 --2- 172.21.15.17:0/1214026504 >> v2:172.21.15.163:3300/0 conn(0x7f716406d9f0 0x7f71640a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 -- 172.21.15.17:0/1214026504 >> v2:172.21.15.163:3300/0 conn(0x7f716406d9f0 msgr2=0x7f71640a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 --2- 172.21.15.17:0/1214026504 >> v2:172.21.15.163:3300/0 conn(0x7f716406d9f0 0x7f71640a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 -- 172.21.15.17:0/1214026504 >> v1:172.21.15.163:6789/0 conn(0x7f71641551e0 legacy=0x7f71641555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.354+0000 7f7169c23640 1 -- 172.21.15.17:0/1214026504 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71641590a0 con 0x7f71640c3280 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.355+0000 7f7169c23640 1 --2- 172.21.15.17:0/1214026504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71640c3280 0x7f71640c3710 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f71640c9820 tx=0x7f715402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60346837207befd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.355+0000 7f7169422640 1 -- 172.21.15.17:0/1214026504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715402e070 con 0x7f71640c3280 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.355+0000 7f7169422640 1 -- 172.21.15.17:0/1214026504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154037070 con 0x7f71640c3280 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.355+0000 7f7169422640 1 -- 172.21.15.17:0/1214026504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154031a00 con 0x7f71640c3280 2024-03-06T07:26:35.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 -- 172.21.15.17:0/1214026504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71640c3280 msgr2=0x7f71640c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 --2- 172.21.15.17:0/1214026504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71640c3280 0x7f71640c3710 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f71640c9820 tx=0x7f715402b880 comp rx=0 tx=0).stop 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 -- 172.21.15.17:0/1214026504 shutdown_connections 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 --2- 172.21.15.17:0/1214026504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71640c3280 0x7f71640c3710 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 --2- 172.21.15.17:0/1214026504 >> v2:172.21.15.163:3300/0 conn(0x7f716406d9f0 0x7f71640a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 -- 172.21.15.17:0/1214026504 >> 172.21.15.17:0/1214026504 conn(0x7f71640aa630 msgr2=0x7f71640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 -- 172.21.15.17:0/1214026504 shutdown_connections 2024-03-06T07:26:35.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.356+0000 7f716c688640 1 -- 172.21.15.17:0/1214026504 wait complete. 2024-03-06T07:26:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 Processor -- start 2024-03-06T07:26:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 -- start start 2024-03-06T07:26:35.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 0x7f716414cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641551e0 0x7f716414d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716406df50 con 0x7f716406d9f0 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716406ddd0 con 0x7f71640c3280 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.357+0000 7f716c688640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71640a8210 con 0x7f71641551e0 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50072/0 (socket says 172.21.15.17:50072) 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 -- 172.21.15.17:0/1901297439 learned_addr learned my addr 172.21.15.17:0/1901297439 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f7169c23640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 0x7f716414cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716ac25640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641551e0 0x7f716414d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 -- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641551e0 msgr2=0x7f716414d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641551e0 0x7f716414d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 -- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 msgr2=0x7f716414cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 0x7f716414cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 -- 172.21.15.17:0/1901297439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640a8040 con 0x7f716406d9f0 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f7169c23640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 0x7f716414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716a424640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f716000d830 tx=0x7f716000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000b6c0 con 0x7f716406d9f0 2024-03-06T07:26:35.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160019070 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716406e270 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000ba80 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.358+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716406ff20 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.359+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71641ee0f0 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.360+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f716000bc20 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.360+0000 7f715affd640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.360+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f716008a470 con 0x7f716406d9f0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.360+0000 7f7169c23640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.361+0000 7f7169c23640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 0x7f714806d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7154036000 tx=0x7f715403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.365+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7160059540 con 0x7f716406d9f0 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.615+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f716404f330 con 0x7f716406d9f0 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.615+0000 7f715affd640 1 -- 172.21.15.17:0/1901297439 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f7160058e40 con 0x7f716406d9f0 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:26:35.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.618 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:26:35.619 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:26:35.620 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.621 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.623 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.624 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.626 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.627 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.628 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:26:35.629 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.630 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:35.631 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:26:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:26:35.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 msgr2=0x7f714806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 0x7f714806d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7154036000 tx=0x7f715403d040 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 msgr2=0x7f71640c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f716000d830 tx=0x7f716000dcf0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 shutdown_connections 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71641551e0 0x7f716414d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71640c3280 0x7f716414cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 --2- 172.21.15.17:0/1901297439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f716406d9f0 0x7f71640c5580 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 >> 172.21.15.17:0/1901297439 conn(0x7f71640aa630 msgr2=0x7f71641555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 shutdown_connections 2024-03-06T07:26:35.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.617+0000 7f716c688640 1 -- 172.21.15.17:0/1901297439 wait complete. 2024-03-06T07:26:35.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:35.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.736+0000 7fb48bf56640 1 Processor -- start 2024-03-06T07:26:35.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.736+0000 7fb48bf56640 1 -- start start 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48bf56640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb484073250 0x7fb4840c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48bf56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb48415fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48bf56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4840bdc20 con 0x7fb484072e70 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48bf56640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48406c8e0 con 0x7fb484073250 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48bf56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48406ce80 con 0x7fb4840c3950 2024-03-06T07:26:35.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb489cf2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb484072e70 0x7fb4840bd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54286/0 (socket says 172.21.15.17:54286) 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb489cf2640 1 -- 172.21.15.17:0/691404755 learned_addr learned my addr 172.21.15.17:0/691404755 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb48a4f3640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb48415fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb4894f1640 1 --2- 172.21.15.17:0/691404755 >> v2:172.21.15.163:3300/0 conn(0x7fb484073250 0x7fb4840c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076219354 0 0) 0x7fb4840bdc20 con 0x7fb484072e70 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb470003520 con 0x7fb484072e70 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.737+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1409419561 0 0) 0x7fb470003520 con 0x7fb484072e70 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 msgr2=0x7fb48415fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb48415fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 >> v2:172.21.15.163:3300/0 conn(0x7fb484073250 msgr2=0x7fb4840c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 --2- 172.21.15.17:0/691404755 >> v2:172.21.15.163:3300/0 conn(0x7fb484073250 0x7fb4840c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4840c1b00 con 0x7fb484072e70 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb47c001a10 con 0x7fb484072e70 2024-03-06T07:26:35.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 >> v1:172.21.15.17:6789/0 conn(0x7fb484072e70 legacy=0x7fb4840bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4700039b0 0x7fb470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb470004800 0x7fb470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb4700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48406c8e0 con 0x7fb470004800 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48406ce80 con 0x7fb4700039b0 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47000b9f0 con 0x7fb4840c3950 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb470004800 0x7fb470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb48a4f3640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb4700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb489cf2640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4700039b0 0x7fb470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 -- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 msgr2=0x7fb4700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4840c3950 0x7fb4700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 -- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4700039b0 msgr2=0x7fb470003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4700039b0 0x7fb470003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.738+0000 7fb4894f1640 1 -- 172.21.15.17:0/691404755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470003520 con 0x7fb470004800 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb4894f1640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb470004800 0x7fb470004c80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fb474009a90 tx=0x7fb470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f07ba946cd8ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474039070 con 0x7fb470004800 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47402c6b0 con 0x7fb470004800 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb488cf0640 1 -- 172.21.15.17:0/691404755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474009690 con 0x7fb470004800 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb48bf56640 1 -- 172.21.15.17:0/691404755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb470004800 msgr2=0x7fb470004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb48bf56640 1 --2- 172.21.15.17:0/691404755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb470004800 0x7fb470004c80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fb474009a90 tx=0x7fb470005830 comp rx=0 tx=0).stop 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.739+0000 7fb48a4f3640 1 -- 172.21.15.17:0/691404755 reap_dead start 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.740+0000 7fb48bf56640 1 -- 172.21.15.17:0/691404755 shutdown_connections 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.740+0000 7fb48bf56640 1 -- 172.21.15.17:0/691404755 >> 172.21.15.17:0/691404755 conn(0x7fb48406b1a0 msgr2=0x7fb48406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.740+0000 7fb48bf56640 1 -- 172.21.15.17:0/691404755 shutdown_connections 2024-03-06T07:26:35.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.740+0000 7fb48bf56640 1 -- 172.21.15.17:0/691404755 wait complete. 2024-03-06T07:26:35.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 Processor -- start 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 -- start start 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 0x7fb4840a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4840bde40 0x7fb484071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840c6760 con 0x7fb48414dc80 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4840c4a00 con 0x7fb4840bde40 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48bf56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841eecb0 con 0x7fb470004800 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.741+0000 7fb48a4f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50090/0 (socket says 172.21.15.17:50090) 2024-03-06T07:26:35.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 -- 172.21.15.17:0/242450453 learned_addr learned my addr 172.21.15.17:0/242450453 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb4894f1640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4840bde40 0x7fb484071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb489cf2640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 0x7fb4840a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 -- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 msgr2=0x7fb4840a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 0x7fb4840a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 -- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4840bde40 msgr2=0x7fb484071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4840bde40 0x7fb484071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 -- 172.21.15.17:0/242450453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4840be9b0 con 0x7fb48414dc80 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb489cf2640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 0x7fb4840a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48a4f3640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb4840c3360 tx=0x7fb480011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480004240 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48001b070 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4841eef50 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000f7a0 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.742+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484071e70 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.743+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48404f330 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.744+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb480012040 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.744+0000 7fb47a7fc640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 0x7fb46806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.744+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb480089dc0 con 0x7fb48414dc80 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.744+0000 7fb489cf2640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 0x7fb46806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:35.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.745+0000 7fb489cf2640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 0x7fb46806d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb4840c4640 tx=0x7fb47c006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:35.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.748+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb480058d70 con 0x7fb48414dc80 2024-03-06T07:26:35.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.930+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4840c6340 con 0x7fb48414dc80 2024-03-06T07:26:35.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.931+0000 7fb47a7fc640 1 -- 172.21.15.17:0/242450453 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+13174 (secure 0 0 0) 0x7fb480058670 con 0x7fb48414dc80 2024-03-06T07:26:35.933 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:35.933 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":168,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:35.227373+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 msgr2=0x7fb46806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 0x7fb46806d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb4840c4640 tx=0x7fb47c006040 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 msgr2=0x7fb48414e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fb4840c3360 tx=0x7fb480011cf0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 shutdown_connections 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb46806b4d0 0x7fb46806d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb48414dc80 0x7fb48414e100 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4840bde40 0x7fb484071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 --2- 172.21.15.17:0/242450453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb470004800 0x7fb4840a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:35.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 >> 172.21.15.17:0/242450453 conn(0x7fb48406b1a0 msgr2=0x7fb4840c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:35.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 shutdown_connections 2024-03-06T07:26:35.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:35.933+0000 7fb48bf56640 1 -- 172.21.15.17:0/242450453 wait complete. 2024-03-06T07:26:35.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:26:35.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:26:36.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.050+0000 7f818b51e640 1 Processor -- start 2024-03-06T07:26:36.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.050+0000 7f818b51e640 1 -- start start 2024-03-06T07:26:36.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f818b51e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f818406bff0 0x7f81840c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f818b51e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81840c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f818b51e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81840b04b0 con 0x7f81840b00d0 2024-03-06T07:26:36.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f818b51e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8184156160 con 0x7f818406bff0 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f818b51e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c5710 con 0x7f81840c7210 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f8188ab9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f818406bff0 0x7f81840c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f81892ba640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f81840b00d0 0x7f818406b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54288/0 (socket says 172.21.15.17:54288) 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f81892ba640 1 -- 172.21.15.17:0/71951980 learned_addr learned my addr 172.21.15.17:0/71951980 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f8188ab9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f818406bff0 0x7f81840c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50100/0 (socket says 172.21.15.17:50100) 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.051+0000 7f8189abb640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81840c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8188ab9640 1 -- 172.21.15.17:0/71951980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 msgr2=0x7f81840c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8188ab9640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81840c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8188ab9640 1 -- 172.21.15.17:0/71951980 >> v1:172.21.15.17:6789/0 conn(0x7f81840b00d0 legacy=0x7f818406b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8183fff640 1 -- 172.21.15.17:0/71951980 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257737359 0 0) 0x7f81840b04b0 con 0x7f81840b00d0 2024-03-06T07:26:36.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8188ab9640 1 -- 172.21.15.17:0/71951980 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818406d210 con 0x7f818406bff0 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8188ab9640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f818406bff0 0x7f81840c8ff0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f81780097a0 tx=0x7f817802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826f20df11f3eecc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.052+0000 7f8183fff640 1 -- 172.21.15.17:0/71951980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178035020 con 0x7f818406bff0 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f8183fff640 1 -- 172.21.15.17:0/71951980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8178031830 con 0x7f818406bff0 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f8183fff640 1 -- 172.21.15.17:0/71951980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178031c00 con 0x7f818406bff0 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 -- 172.21.15.17:0/71951980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f818406bff0 msgr2=0x7f81840c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f818406bff0 0x7f81840c8ff0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f81780097a0 tx=0x7f817802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 -- 172.21.15.17:0/71951980 shutdown_connections 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81840c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 --2- 172.21.15.17:0/71951980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f818406bff0 0x7f81840c8ff0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 -- 172.21.15.17:0/71951980 >> 172.21.15.17:0/71951980 conn(0x7f81840ac810 msgr2=0x7f81840acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 -- 172.21.15.17:0/71951980 shutdown_connections 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.053+0000 7f818b51e640 1 -- 172.21.15.17:0/71951980 wait complete. 2024-03-06T07:26:36.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.054+0000 7f818b51e640 1 Processor -- start 2024-03-06T07:26:36.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.054+0000 7f818b51e640 1 -- start start 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818406bff0 0x7f81841ee270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81841574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c5ba0 con 0x7f81840b00d0 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8184152c50 con 0x7f818406bff0 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f818b51e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840c5d20 con 0x7f81840c7210 2024-03-06T07:26:36.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50102/0 (socket says 172.21.15.17:50102) 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 -- 172.21.15.17:0/445820621 learned_addr learned my addr 172.21.15.17:0/445820621 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f81892ba640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818406bff0 0x7f81841ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8189abb640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81841574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 -- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 msgr2=0x7f81841574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81841574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 -- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818406bff0 msgr2=0x7f81841ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818406bff0 0x7f81841ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 -- 172.21.15.17:0/445820621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81841eeab0 con 0x7f81840b00d0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8189abb640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81841574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.055+0000 7f8188ab9640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8178034040 tx=0x7f81780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.056+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178040070 con 0x7f81840b00d0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.056+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81780399a0 con 0x7f81840b00d0 2024-03-06T07:26:36.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.056+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81841eed90 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.056+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178035040 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.056+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81841ee7b0 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.057+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81840c47c0 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.057+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8178038020 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.057+0000 7f8181ffb640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 0x7f817006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.058+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f81780aba00 con 0x7f81840b00d0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.058+0000 7f81892ba640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 0x7f817006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.058+0000 7f81892ba640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 0x7f817006d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f817c0012e0 tx=0x7f817c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.062+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f817807aa50 con 0x7f81840b00d0 2024-03-06T07:26:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.243+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f818404f330 con 0x7f81840b00d0 2024-03-06T07:26:36.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.243+0000 7f8181ffb640 1 -- 172.21.15.17:0/445820621 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f817807a350 con 0x7f81840b00d0 2024-03-06T07:26:36.245 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 msgr2=0x7f817006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 0x7f817006d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f817c0012e0 tx=0x7f817c009040 comp rx=0 tx=0).stop 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 msgr2=0x7f8184156fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f8178034040 tx=0x7f81780046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 shutdown_connections 2024-03-06T07:26:36.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f817006b4d0 0x7f817006d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81840c7210 0x7f81841574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81840b00d0 0x7f8184156fb0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 --2- 172.21.15.17:0/445820621 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f818406bff0 0x7f81841ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 >> 172.21.15.17:0/445820621 conn(0x7f81840ac810 msgr2=0x7f81840ab250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 shutdown_connections 2024-03-06T07:26:36.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.245+0000 7f818b51e640 1 -- 172.21.15.17:0/445820621 wait complete. 2024-03-06T07:26:36.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:26:36.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.357+0000 7f347593e640 1 Processor -- start 2024-03-06T07:26:36.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.357+0000 7f347593e640 1 -- start start 2024-03-06T07:26:36.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3470152920 0x7f3470155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34700c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f34700c1280 0x7f34700c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3470156160 con 0x7f3470152920 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347006afc0 con 0x7f34700c1280 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f347593e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347006b2a0 con 0x7f34700c2190 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f346effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3470152920 0x7f3470155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f346effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3470152920 0x7f3470155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50116/0 (socket says 172.21.15.17:50116) 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.358+0000 7f346effd640 1 -- 172.21.15.17:0/2787816486 learned_addr learned my addr 172.21.15.17:0/2787816486 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346e7fc640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34700c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346f7fe640 1 --2- 172.21.15.17:0/2787816486 >> v2:172.21.15.163:3300/0 conn(0x7f34700c1280 0x7f34700c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 -- 172.21.15.17:0/2787816486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 msgr2=0x7f34700c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34700c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 -- 172.21.15.17:0/2787816486 >> v2:172.21.15.163:3300/0 conn(0x7f34700c1280 msgr2=0x7f34700c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 --2- 172.21.15.17:0/2787816486 >> v2:172.21.15.163:3300/0 conn(0x7f34700c1280 0x7f34700c1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 -- 172.21.15.17:0/2787816486 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347006b580 con 0x7f3470152920 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346effd640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f3470155c20 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f34700c6320 tx=0x7f346802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1157b772e66583b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.359+0000 7f346dffb640 1 -- 172.21.15.17:0/2787816486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468035020 con 0x7f3470152920 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f346dffb640 1 -- 172.21.15.17:0/2787816486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468031870 con 0x7f3470152920 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f346dffb640 1 -- 172.21.15.17:0/2787816486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468031c40 con 0x7f3470152920 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 -- 172.21.15.17:0/2787816486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 msgr2=0x7f3470155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f3470155c20 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f34700c6320 tx=0x7f346802b910 comp rx=0 tx=0).stop 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 -- 172.21.15.17:0/2787816486 shutdown_connections 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 --2- 172.21.15.17:0/2787816486 >> v2:172.21.15.163:3300/0 conn(0x7f34700c1280 0x7f34700c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34700c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 --2- 172.21.15.17:0/2787816486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f3470155c20 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 -- 172.21.15.17:0/2787816486 >> 172.21.15.17:0/2787816486 conn(0x7f34700c4290 msgr2=0x7f34700c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 -- 172.21.15.17:0/2787816486 shutdown_connections 2024-03-06T07:26:36.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.360+0000 7f347593e640 1 -- 172.21.15.17:0/2787816486 wait complete. 2024-03-06T07:26:36.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.361+0000 7f347593e640 1 Processor -- start 2024-03-06T07:26:36.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.361+0000 7f347593e640 1 -- start start 2024-03-06T07:26:36.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34700c1280 0x7f3470148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34701eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c1c50 con 0x7f3470152920 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c0d60 con 0x7f34700c1280 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f347593e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700c1dd0 con 0x7f34700c2190 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50118/0 (socket says 172.21.15.17:50118) 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 -- 172.21.15.17:0/1697747972 learned_addr learned my addr 172.21.15.17:0/1697747972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346effd640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34700c1280 0x7f3470148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346e7fc640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34701eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 -- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 msgr2=0x7f34701eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34701eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 -- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34700c1280 msgr2=0x7f3470148f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34700c1280 0x7f3470148f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346f7fe640 1 -- 172.21.15.17:0/1697747972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347014a3f0 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.362+0000 7f346e7fc640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34701eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f346f7fe640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f346000cc30 tx=0x7f346000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346000ce40 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3460003990 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347014a6d0 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460003d40 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.363+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34701eb390 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.364+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34701eb590 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.364+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3460004000 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.365+0000 7f3467fff640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 0x7f345006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.365+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f346008a460 con 0x7f3470152920 2024-03-06T07:26:36.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.365+0000 7f346effd640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 0x7f345006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.365+0000 7f346effd640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 0x7f345006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3468036000 tx=0x7f346803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.369+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f34701eb590 con 0x7f3470152920 2024-03-06T07:26:36.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.555+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f347004f330 con 0x7f3470152920 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.555+0000 7f3467fff640 1 -- 172.21.15.17:0/1697747972 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v168) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3460014030 con 0x7f3470152920 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:26:36.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:26:36.561 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:26:36.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 msgr2=0x7f345006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 0x7f345006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3468036000 tx=0x7f346803c040 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 msgr2=0x7f347014d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f346000cc30 tx=0x7f346000a5c0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 shutdown_connections 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f345006b4d0 0x7f345006d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3470152920 0x7f347014d510 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34700c2190 0x7f34701eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 --2- 172.21.15.17:0/1697747972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34700c1280 0x7f3470148f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 >> 172.21.15.17:0/1697747972 conn(0x7f34700c4290 msgr2=0x7f3470154a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 shutdown_connections 2024-03-06T07:26:36.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.557+0000 7f347593e640 1 -- 172.21.15.17:0/1697747972 wait complete. 2024-03-06T07:26:36.570 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:26:36.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:26:36.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 Processor -- start 2024-03-06T07:26:36.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 -- start start 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7ff0140c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff014159230 0x7ff0140c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140c8ea0 con 0x7ff0140c7470 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff014158da0 con 0x7ff014159230 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.674+0000 7ff01a281640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140c7a10 con 0x7ff014152860 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff0137fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff0140c7470 0x7ff014155d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54300/0 (socket says 172.21.15.17:54300) 2024-03-06T07:26:36.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff0137fe640 1 -- 172.21.15.17:0/3152840684 learned_addr learned my addr 172.21.15.17:0/3152840684 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff013fff640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7ff0140c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff01881e640 1 --2- 172.21.15.17:0/3152840684 >> v2:172.21.15.163:3300/0 conn(0x7ff014159230 0x7ff0140c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425967126 0 0) 0x7ff0140c8ea0 con 0x7ff0140c7470 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feffc003520 con 0x7ff0140c7470 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372440642 0 0) 0x7feffc003520 con 0x7ff0140c7470 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 msgr2=0x7ff0140c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7ff0140c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 >> v2:172.21.15.163:3300/0 conn(0x7ff014159230 msgr2=0x7ff0140c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 --2- 172.21.15.17:0/3152840684 >> v2:172.21.15.163:3300/0 conn(0x7ff014159230 0x7ff0140c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140c7cf0 con 0x7ff0140c7470 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff008001670 con 0x7ff0140c7470 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 >> v1:172.21.15.17:6789/0 conn(0x7ff0140c7470 legacy=0x7ff014155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7feffc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc003f30 0x7feffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feffc004d40 0x7feffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff014158da0 con 0x7feffc004d40 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140c7a10 con 0x7feffc003f30 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc00ba10 con 0x7ff014152860 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff01881e640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feffc004d40 0x7feffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff013fff640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7feffc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.675+0000 7ff0137fe640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc003f30 0x7feffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 -- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 msgr2=0x7feffc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff014152860 0x7feffc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 -- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc003f30 msgr2=0x7feffc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc003f30 0x7feffc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 -- 172.21.15.17:0/3152840684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc003520 con 0x7feffc004d40 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff0137fe640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc003f30 0x7feffc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.676+0000 7ff01881e640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feffc004d40 0x7feffc0051c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e170 tx=0x7feffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1fc4572ea99f5f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c039070 con 0x7feffc004d40 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c02cc20 con 0x7feffc004d40 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff012ffd640 1 -- 172.21.15.17:0/3152840684 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c009c10 con 0x7feffc004d40 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 -- 172.21.15.17:0/3152840684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feffc004d40 msgr2=0x7feffc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 --2- 172.21.15.17:0/3152840684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feffc004d40 0x7feffc0051c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00e170 tx=0x7feffc005830 comp rx=0 tx=0).stop 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01881e640 1 -- 172.21.15.17:0/3152840684 reap_dead start 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 -- 172.21.15.17:0/3152840684 shutdown_connections 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 -- 172.21.15.17:0/3152840684 >> 172.21.15.17:0/3152840684 conn(0x7ff0140ac630 msgr2=0x7ff0140aca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 -- 172.21.15.17:0/3152840684 shutdown_connections 2024-03-06T07:26:36.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.677+0000 7ff01a281640 1 -- 172.21.15.17:0/3152840684 wait complete. 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.678+0000 7ff01a281640 1 Processor -- start 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.678+0000 7ff01a281640 1 -- start start 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc004d40 0x7ff014155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0141ee670 0x7ff0141eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0141596c0 con 0x7ff0140be590 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff014152c40 con 0x7feffc004d40 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01a281640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff014159840 con 0x7ff0141ee670 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50146/0 (socket says 172.21.15.17:50146) 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff0137fe640 1 -- 172.21.15.17:0/1136768565 learned_addr learned my addr 172.21.15.17:0/1136768565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff013fff640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc004d40 0x7ff014155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff01881e640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0141ee670 0x7ff0141eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff0137fe640 1 -- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0141ee670 msgr2=0x7ff0141eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.679+0000 7ff0137fe640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0141ee670 0x7ff0141eeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff0137fe640 1 -- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc004d40 msgr2=0x7ff014155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff0137fe640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc004d40 0x7ff014155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff0137fe640 1 -- 172.21.15.17:0/1136768565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014157290 con 0x7ff0140be590 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff0137fe640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff00800e870 tx=0x7ff00800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800c870 con 0x7ff0140be590 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008018070 con 0x7ff0140be590 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff014157570 con 0x7ff0140be590 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800cc30 con 0x7ff0140be590 2024-03-06T07:26:36.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.680+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0141557d0 con 0x7ff0140be590 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.681+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01404f330 con 0x7ff0140be590 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.681+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff00801d460 con 0x7ff0140be590 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.682+0000 7ff010ff9640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 0x7ff00006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.682+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff008089e10 con 0x7ff0140be590 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.682+0000 7ff013fff640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 0x7ff00006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.682+0000 7ff013fff640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 0x7ff00006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff004009bc0 tx=0x7ff004009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:36.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.686+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff008058ee0 con 0x7ff0140be590 2024-03-06T07:26:36.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:36.866+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff0140c8110 con 0x7ff0140be590 2024-03-06T07:26:37.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.239+0000 7ff010ff9640 1 -- 172.21.15.17:0/1136768565 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v169) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff0080587e0 con 0x7ff0140be590 2024-03-06T07:26:37.241 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:26:37.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.241+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 msgr2=0x7ff00006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 0x7ff00006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff004009bc0 tx=0x7ff004009190 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 msgr2=0x7ff0141ec300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff00800e870 tx=0x7ff00800ec70 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 shutdown_connections 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff00006b4d0 0x7ff00006d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0141ee670 0x7ff0141eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0140be590 0x7ff0141ec300 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 --2- 172.21.15.17:0/1136768565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feffc004d40 0x7ff014155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 >> 172.21.15.17:0/1136768565 conn(0x7ff0140ac630 msgr2=0x7ff0140c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 shutdown_connections 2024-03-06T07:26:37.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.242+0000 7ff01a281640 1 -- 172.21.15.17:0/1136768565 wait complete. 2024-03-06T07:26:37.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:26:37.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.365+0000 7fcaf84f4640 1 Processor -- start 2024-03-06T07:26:37.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.365+0000 7fcaf84f4640 1 -- start start 2024-03-06T07:26:37.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf84f4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcaf01529f0 0x7fcaf00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf84f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 0x7fcaf0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf84f4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf0152dd0 con 0x7fcaf00c5370 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf84f4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf0159710 con 0x7fcaf01529f0 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf84f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0156180 con 0x7fcaf00c57d0 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6290640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcaf01529f0 0x7fcaf00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf5a8f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcaf00c5370 0x7fcaf0155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54308/0 (socket says 172.21.15.17:54308) 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf5a8f640 1 -- 172.21.15.17:0/2522648503 learned_addr learned my addr 172.21.15.17:0/2522648503 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6a91640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 0x7fcaf0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6290640 1 -- 172.21.15.17:0/2522648503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 msgr2=0x7fcaf0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 0x7fcaf0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6290640 1 -- 172.21.15.17:0/2522648503 >> v1:172.21.15.17:6789/0 conn(0x7fcaf00c5370 legacy=0x7fcaf0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6290640 1 -- 172.21.15.17:0/2522648503 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0158b00 con 0x7fcaf01529f0 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf528e640 1 -- 172.21.15.17:0/2522648503 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641778747 0 0) 0x7fcaf0152dd0 con 0x7fcaf00c5370 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.366+0000 7fcaf6a91640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 0x7fcaf0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf01529f0 0x7fcaf00c88f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097a0 tx=0x7fcaec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df20854223b1d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf528e640 1 -- 172.21.15.17:0/2522648503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0315a0 con 0x7fcaf01529f0 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf528e640 1 -- 172.21.15.17:0/2522648503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec037070 con 0x7fcaf01529f0 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf528e640 1 -- 172.21.15.17:0/2522648503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec031960 con 0x7fcaf01529f0 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2522648503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf01529f0 msgr2=0x7fcaf00c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.367+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf01529f0 0x7fcaf00c88f0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0097a0 tx=0x7fcaec02b840 comp rx=0 tx=0).stop 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2522648503 shutdown_connections 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c57d0 0x7fcaf0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2522648503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf01529f0 0x7fcaf00c88f0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2522648503 >> 172.21.15.17:0/2522648503 conn(0x7fcaf00ac580 msgr2=0x7fcaf00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2522648503 shutdown_connections 2024-03-06T07:26:37.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.368+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2522648503 wait complete. 2024-03-06T07:26:37.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 Processor -- start 2024-03-06T07:26:37.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 -- start start 2024-03-06T07:26:37.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 0x7fcaf01eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 0x7fcaf0154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01590c0 con 0x7fcaf00c57d0 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf00c68b0 con 0x7fcaf01529f0 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf84f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01593a0 con 0x7fcaf00c5370 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf5a8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf5a8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50172/0 (socket says 172.21.15.17:50172) 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.369+0000 7fcaf5a8f640 1 -- 172.21.15.17:0/2873504979 learned_addr learned my addr 172.21.15.17:0/2873504979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 0x7fcaf01eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf6a91640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 0x7fcaf0154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 -- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 msgr2=0x7fcaf01eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 0x7fcaf01eb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 -- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 msgr2=0x7fcaf0154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 0x7fcaf0154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 -- 172.21.15.17:0/2873504979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0157d80 con 0x7fcaf00c57d0 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf6a91640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 0x7fcaf0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 0x7fcaf01eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:37.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf5a8f640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c4a0 tx=0x7fcae000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0018070 con 0x7fcaf00c57d0 2024-03-06T07:26:37.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae00074e0 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf0158060 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0004a20 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.370+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf0156800 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.371+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0157b80 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.372+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcae0004bc0 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.372+0000 7fcae6ffd640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 0x7fcad406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.372+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fcae00894a0 con 0x7fcaf00c57d0 2024-03-06T07:26:37.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.373+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 0x7fcad406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:37.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.373+0000 7fcaf6290640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 0x7fcad406d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcaec036000 tx=0x7fcaec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:37.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.377+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcae0058380 con 0x7fcaf00c57d0 2024-03-06T07:26:37.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:37.561+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcaf004f330 con 0x7fcaf00c57d0 2024-03-06T07:26:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.244+0000 7fcae6ffd640 1 -- 172.21.15.17:0/2873504979 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v170) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcae005e080 con 0x7fcaf00c57d0 2024-03-06T07:26:38.246 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 msgr2=0x7fcad406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 0x7fcad406d9c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcaec036000 tx=0x7fcaec008030 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 msgr2=0x7fcaf0149d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c4a0 tx=0x7fcae000c960 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 shutdown_connections 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcad406b500 0x7fcad406d9c0 secure :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcaec036000 tx=0x7fcaec008030 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcaf01529f0 0x7fcaf0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcaf00c57d0 0x7fcaf0149d10 secure :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c4a0 tx=0x7fcae000c960 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 --2- 172.21.15.17:0/2873504979 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcaf00c5370 0x7fcaf01eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.247+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 >> 172.21.15.17:0/2873504979 conn(0x7fcaf00ac580 msgr2=0x7fcaf00c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.249+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 shutdown_connections 2024-03-06T07:26:38.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.251+0000 7fcaf84f4640 1 -- 172.21.15.17:0/2873504979 wait complete. 2024-03-06T07:26:38.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.380+0000 7f458cadf640 1 Processor -- start 2024-03-06T07:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 -- start start 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 0x7f4588156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45880c7f60 0x7f4588072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4588072a50 0x7f45880c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4588070b20 con 0x7f4588072a50 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4588073580 con 0x7f45881505f0 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f458cadf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45880701f0 con 0x7f45880c7f60 2024-03-06T07:26:38.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f4586d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4588072a50 0x7f45880c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f4586d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4588072a50 0x7f45880c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50184/0 (socket says 172.21.15.17:50184) 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.381+0000 7f4586d9d640 1 -- 172.21.15.17:0/2894235910 learned_addr learned my addr 172.21.15.17:0/2894235910 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f457ffff640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45880c7f60 0x7f4588072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f458659c640 1 --2- 172.21.15.17:0/2894235910 >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 0x7f4588156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 -- 172.21.15.17:0/2894235910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45880c7f60 msgr2=0x7f4588072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45880c7f60 0x7f4588072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 -- 172.21.15.17:0/2894235910 >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 msgr2=0x7f4588156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 --2- 172.21.15.17:0/2894235910 >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 0x7f4588156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 -- 172.21.15.17:0/2894235910 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880704d0 con 0x7f4588072a50 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f458659c640 1 --2- 172.21.15.17:0/2894235910 >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 0x7f4588156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.382+0000 7f4586d9d640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4588072a50 0x7f45880c6710 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4588072060 tx=0x7f458002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c904d3bd13c24a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f4585d9b640 1 -- 172.21.15.17:0/2894235910 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458002ca70 con 0x7f4588072a50 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f4585d9b640 1 -- 172.21.15.17:0/2894235910 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458002cc10 con 0x7f4588072a50 2024-03-06T07:26:38.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f4585d9b640 1 -- 172.21.15.17:0/2894235910 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45800343f0 con 0x7f4588072a50 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 -- 172.21.15.17:0/2894235910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4588072a50 msgr2=0x7f45880c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4588072a50 0x7f45880c6710 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4588072060 tx=0x7f458002bed0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 -- 172.21.15.17:0/2894235910 shutdown_connections 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4588072a50 0x7f45880c6710 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 --2- 172.21.15.17:0/2894235910 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45880c7f60 0x7f4588072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 --2- 172.21.15.17:0/2894235910 >> v2:172.21.15.163:3300/0 conn(0x7f45881505f0 0x7f4588156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 -- 172.21.15.17:0/2894235910 >> 172.21.15.17:0/2894235910 conn(0x7f45880aa5c0 msgr2=0x7f45880aa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:38.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 -- 172.21.15.17:0/2894235910 shutdown_connections 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.383+0000 7f458cadf640 1 -- 172.21.15.17:0/2894235910 wait complete. 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.384+0000 7f458cadf640 1 Processor -- start 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 -- start start 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4588072a50 0x7f45880adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45881505f0 0x7f4588144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881509d0 con 0x7f45880c7f60 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458806fec0 con 0x7f45881505f0 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f458cadf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4588150b50 con 0x7f4588072a50 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f457ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f457ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50198/0 (socket says 172.21.15.17:50198) 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.385+0000 7f457ffff640 1 -- 172.21.15.17:0/2132864032 learned_addr learned my addr 172.21.15.17:0/2132864032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 -- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4588072a50 msgr2=0x7f45880adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4588072a50 0x7f45880adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 -- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45881505f0 msgr2=0x7f4588144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45881505f0 0x7f4588144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 -- 172.21.15.17:0/2132864032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45881e9ef0 con 0x7f45880c7f60 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457ffff640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f457000e8a0 tx=0x7f457000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570017070 con 0x7f45880c7f60 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457000ee90 con 0x7f45880c7f60 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4588153c70 con 0x7f45880c7f60 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000cb80 con 0x7f45880c7f60 2024-03-06T07:26:38.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.386+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45881e9650 con 0x7f45880c7f60 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.387+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458804f330 con 0x7f45880c7f60 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.388+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f457001a070 con 0x7f45880c7f60 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.388+0000 7f457effd640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 0x7f456806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.388+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f457008a120 con 0x7f45880c7f60 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.388+0000 7f458659c640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 0x7f456806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:38.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.389+0000 7f458659c640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 0x7f456806d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4578006fd0 tx=0x7f4578008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:38.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.392+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4570058f00 con 0x7f45880c7f60 2024-03-06T07:26:38.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:38.612+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f45880c3c50 con 0x7f45880c7f60 2024-03-06T07:26:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.258+0000 7f457effd640 1 -- 172.21.15.17:0/2132864032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v156) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4570058800 con 0x7f45880c7f60 2024-03-06T07:26:39.260 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 27 and data pool 28 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 msgr2=0x7f456806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 0x7f456806d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4578006fd0 tx=0x7f4578008040 comp rx=0 tx=0).stop 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 msgr2=0x7f4588152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f457000e8a0 tx=0x7f457000ed60 comp rx=0 tx=0).stop 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 shutdown_connections 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f456806b4d0 0x7f456806d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f45881505f0 0x7f4588144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45880c7f60 0x7f4588152540 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 --2- 172.21.15.17:0/2132864032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4588072a50 0x7f45880adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.266+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 >> 172.21.15.17:0/2132864032 conn(0x7f45880aa5c0 msgr2=0x7f45880c3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.268+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 shutdown_connections 2024-03-06T07:26:39.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.268+0000 7f458cadf640 1 -- 172.21.15.17:0/2132864032 wait complete. 2024-03-06T07:26:39.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:39.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.395+0000 7fb48049d640 1 Processor -- start 2024-03-06T07:26:39.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.395+0000 7fb48049d640 1 -- start start 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.395+0000 7fb48049d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4780c1320 0x7fb4781566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.395+0000 7fb48049d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4780c6000 0x7fb478156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb48049d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb478153a60 con 0x7fb478150580 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb48049d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb478153d40 con 0x7fb4780c1320 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb48049d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806c2c0 con 0x7fb4780c6000 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4780c1320 0x7fb4781566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4780c1320 0x7fb4781566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50208/0 (socket says 172.21.15.17:50208) 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 -- 172.21.15.17:0/1539927692 learned_addr learned my addr 172.21.15.17:0/1539927692 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47ea3a640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4780c6000 0x7fb478156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 -- 172.21.15.17:0/1539927692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4780c6000 msgr2=0x7fb478156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4780c6000 0x7fb478156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 -- 172.21.15.17:0/1539927692 >> v1:172.21.15.163:6789/0 conn(0x7fb478150580 legacy=0x7fb47806dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.396+0000 7fb47da38640 1 -- 172.21.15.17:0/1539927692 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47806c5a0 con 0x7fb4780c1320 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb47da38640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb4781566b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb468009870 tx=0x7fb46802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32bb2ef0735c13e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb47d237640 1 -- 172.21.15.17:0/1539927692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468035020 con 0x7fb4780c1320 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb47d237640 1 -- 172.21.15.17:0/1539927692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468031640 con 0x7fb4780c1320 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb47d237640 1 -- 172.21.15.17:0/1539927692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46802d040 con 0x7fb4780c1320 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 -- 172.21.15.17:0/1539927692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 msgr2=0x7fb4781566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb4781566b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb468009870 tx=0x7fb46802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 -- 172.21.15.17:0/1539927692 shutdown_connections 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4780c6000 0x7fb478156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 --2- 172.21.15.17:0/1539927692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb4781566b0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.397+0000 7fb48049d640 1 -- 172.21.15.17:0/1539927692 >> 172.21.15.17:0/1539927692 conn(0x7fb4780c6710 msgr2=0x7fb4780c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.398+0000 7fb48049d640 1 -- 172.21.15.17:0/1539927692 shutdown_connections 2024-03-06T07:26:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.398+0000 7fb48049d640 1 -- 172.21.15.17:0/1539927692 wait complete. 2024-03-06T07:26:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 Processor -- start 2024-03-06T07:26:39.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 -- start start 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4780c6000 0x7fb4781e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb478150580 0x7fb4781e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806e5a0 con 0x7fb4780c1320 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47806e420 con 0x7fb4780c6000 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb48049d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780c0c40 con 0x7fb478150580 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb47e239640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb47e239640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50222/0 (socket says 172.21.15.17:50222) 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.399+0000 7fb47e239640 1 -- 172.21.15.17:0/181965696 learned_addr learned my addr 172.21.15.17:0/181965696 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:39.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 -- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb478150580 msgr2=0x7fb4781e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47ea3a640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb478150580 0x7fb4781e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47da38640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4780c6000 0x7fb4781e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb478150580 0x7fb4781e8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 -- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4780c6000 msgr2=0x7fb4781e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4780c6000 0x7fb4781e8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 -- 172.21.15.17:0/181965696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4781ebff0 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb47e239640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c3c0 tx=0x7fb47400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740052d0 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb474005490 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb478156320 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb474007260 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.400+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4781ec570 con 0x7fb4780c1320 2024-03-06T07:26:39.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.401+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb478067ad0 con 0x7fb4780c1320 2024-03-06T07:26:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.402+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb474007400 con 0x7fb4780c1320 2024-03-06T07:26:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.402+0000 7fb46effd640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 0x7fb45c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.402+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4740898a0 con 0x7fb4780c1320 2024-03-06T07:26:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.402+0000 7fb47da38640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 0x7fb45c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.403+0000 7fb47da38640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 0x7fb45c06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb468036000 tx=0x7fb468008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.406+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb478067ad0 con 0x7fb4780c1320 2024-03-06T07:26:39.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.590+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb47804f330 con 0x7fb4780c1320 2024-03-06T07:26:39.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.591+0000 7fb46effd640 1 -- 172.21.15.17:0/181965696 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16965 (secure 0 0 0) 0x7fb474058590 con 0x7fb4780c1320 2024-03-06T07:26:39.593 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:39.593 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":171,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:38.613685+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:39.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 msgr2=0x7fb45c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 0x7fb45c06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb468036000 tx=0x7fb468008030 comp rx=0 tx=0).stop 2024-03-06T07:26:39.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 msgr2=0x7fb478154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c3c0 tx=0x7fb47400c880 comp rx=0 tx=0).stop 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 shutdown_connections 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb45c06b4d0 0x7fb45c06d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb478150580 0x7fb4781e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4780c6000 0x7fb4781e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 --2- 172.21.15.17:0/181965696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4780c1320 0x7fb478154290 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 >> 172.21.15.17:0/181965696 conn(0x7fb4780c6710 msgr2=0x7fb4780c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 shutdown_connections 2024-03-06T07:26:39.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.593+0000 7fb48049d640 1 -- 172.21.15.17:0/181965696 wait complete. 2024-03-06T07:26:39.605 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:39.620 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:39.650 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:26:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.706+0000 7f5d49dbc640 1 Processor -- start 2024-03-06T07:26:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.706+0000 7f5d49dbc640 1 -- start start 2024-03-06T07:26:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d49dbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d44155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d49dbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 0x7f5d44158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d49dbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d440c73d0 con 0x7f5d440c88c0 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d49dbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d440c8dd0 con 0x7f5d44158b60 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d49dbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440be2e0 con 0x7f5d4414f360 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d42ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5d440c88c0 0x7f5d440c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54312/0 (socket says 172.21.15.17:54312) 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d42ffd640 1 -- 172.21.15.17:0/2366772742 learned_addr learned my addr 172.21.15.17:0/2366772742 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d437fe640 1 --2- 172.21.15.17:0/2366772742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d44155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d43fff640 1 --2- 172.21.15.17:0/2366772742 >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 0x7f5d44158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d427fc640 1 -- 172.21.15.17:0/2366772742 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879321229 0 0) 0x7f5d440c73d0 con 0x7f5d440c88c0 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d427fc640 1 -- 172.21.15.17:0/2366772742 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c003520 con 0x7f5d440c88c0 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d437fe640 1 -- 172.21.15.17:0/2366772742 >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 msgr2=0x7f5d44158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d437fe640 1 --2- 172.21.15.17:0/2366772742 >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 0x7f5d44158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d437fe640 1 -- 172.21.15.17:0/2366772742 >> v1:172.21.15.17:6789/0 conn(0x7f5d440c88c0 legacy=0x7f5d440c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d437fe640 1 -- 172.21.15.17:0/2366772742 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440c7670 con 0x7f5d4414f360 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.707+0000 7f5d43fff640 1 --2- 172.21.15.17:0/2366772742 >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 0x7f5d44158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.708+0000 7f5d437fe640 1 --2- 172.21.15.17:0/2366772742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d44155c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d380097a0 tx=0x7f5d3802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c341ca911bccd25 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.708+0000 7f5d427fc640 1 -- 172.21.15.17:0/2366772742 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031780 con 0x7f5d4414f360 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.708+0000 7f5d427fc640 1 -- 172.21.15.17:0/2366772742 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38037070 con 0x7f5d4414f360 2024-03-06T07:26:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.708+0000 7f5d427fc640 1 -- 172.21.15.17:0/2366772742 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38031b20 con 0x7f5d4414f360 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 -- 172.21.15.17:0/2366772742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 msgr2=0x7f5d44155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/2366772742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d44155c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5d380097a0 tx=0x7f5d3802b820 comp rx=0 tx=0).stop 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 -- 172.21.15.17:0/2366772742 shutdown_connections 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/2366772742 >> v2:172.21.15.163:3300/0 conn(0x7f5d44158b60 0x7f5d44158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/2366772742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d44155c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 -- 172.21.15.17:0/2366772742 >> 172.21.15.17:0/2366772742 conn(0x7f5d440ac570 msgr2=0x7f5d440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 -- 172.21.15.17:0/2366772742 shutdown_connections 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.709+0000 7f5d49dbc640 1 -- 172.21.15.17:0/2366772742 wait complete. 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.710+0000 7f5d49dbc640 1 Processor -- start 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.710+0000 7f5d49dbc640 1 -- start start 2024-03-06T07:26:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d440c88c0 0x7f5d440c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d440af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440c5a50 con 0x7f5d44158b60 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440c7c30 con 0x7f5d440c88c0 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d440c5bd0 con 0x7f5d4414f360 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50224/0 (socket says 172.21.15.17:50224) 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 -- 172.21.15.17:0/31173487 learned_addr learned my addr 172.21.15.17:0/31173487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d437fe640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d440c88c0 0x7f5d440c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d42ffd640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d440af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 -- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 msgr2=0x7f5d440af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d440af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 -- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d440c88c0 msgr2=0x7f5d440c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d440c88c0 0x7f5d440c10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 -- 172.21.15.17:0/31173487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440c17e0 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d43fff640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009f00 tx=0x7f5d3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c018030 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00ce50 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d441edc60 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003bb0 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.711+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d441ee220 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.712+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4404f330 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.713+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d3c01b070 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.713+0000 7f5d2bfff640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.713+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d3c08acf0 con 0x7f5d44158b60 2024-03-06T07:26:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.714+0000 7f5d437fe640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.714+0000 7f5d437fe640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 0x7f5d3006d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d3803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.718+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d3c010040 con 0x7f5d44158b60 2024-03-06T07:26:39.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.900+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d4406cda0 con 0x7f5d44158b60 2024-03-06T07:26:39.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.901+0000 7f5d2bfff640 1 -- 172.21.15.17:0/31173487 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16965 (secure 0 0 0) 0x7f5d3c059380 con 0x7f5d44158b60 2024-03-06T07:26:39.902 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:39.903 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":171,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:38.613685+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:39.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 msgr2=0x7f5d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 0x7f5d3006d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d3803d040 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 msgr2=0x7f5d4414bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009f00 tx=0x7f5d3c00c490 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 shutdown_connections 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3006b4d0 0x7f5d3006d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d44158b60 0x7f5d4414bce0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d4414f360 0x7f5d440af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 --2- 172.21.15.17:0/31173487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d440c88c0 0x7f5d440c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 >> 172.21.15.17:0/31173487 conn(0x7f5d440ac570 msgr2=0x7f5d4414cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 shutdown_connections 2024-03-06T07:26:39.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:39.903+0000 7f5d49dbc640 1 -- 172.21.15.17:0/31173487 wait complete. 2024-03-06T07:26:39.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:26:40.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.058+0000 7f43b4d5e640 1 Processor -- start 2024-03-06T07:26:40.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.058+0000 7f43b4d5e640 1 -- start start 2024-03-06T07:26:40.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43b4d5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43b4d5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 0x7f43b0072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43b4d5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00c62a0 con 0x7f43b00c4740 2024-03-06T07:26:40.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43b4d5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b0153f00 con 0x7f43b00c69a0 2024-03-06T07:26:40.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43b4d5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c4c00 con 0x7f43b00a8290 2024-03-06T07:26:40.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43add9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f43b00c4740 0x7f43b00701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54328/0 (socket says 172.21.15.17:54328) 2024-03-06T07:26:40.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43add9b640 1 -- 172.21.15.17:0/745859700 learned_addr learned my addr 172.21.15.17:0/745859700 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:40.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43ae59c640 1 --2- 172.21.15.17:0/745859700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.059+0000 7f43aed9d640 1 --2- 172.21.15.17:0/745859700 >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 0x7f43b0072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ad59a640 1 -- 172.21.15.17:0/745859700 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742143271 0 0) 0x7f43b00c62a0 con 0x7f43b00c4740 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ad59a640 1 -- 172.21.15.17:0/745859700 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f439c003520 con 0x7f43b00c4740 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ae59c640 1 -- 172.21.15.17:0/745859700 >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 msgr2=0x7f43b0072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ae59c640 1 --2- 172.21.15.17:0/745859700 >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 0x7f43b0072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ae59c640 1 -- 172.21.15.17:0/745859700 >> v1:172.21.15.17:6789/0 conn(0x7f43b00c4740 legacy=0x7f43b00701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43ae59c640 1 -- 172.21.15.17:0/745859700 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00c4ee0 con 0x7f43b00a8290 2024-03-06T07:26:40.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.060+0000 7f43aed9d640 1 --2- 172.21.15.17:0/745859700 >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 0x7f43b0072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:26:40.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43ae59c640 1 --2- 172.21.15.17:0/745859700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b01539c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43a80097a0 tx=0x7f43a802b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586469b8ef219bbe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43ad59a640 1 -- 172.21.15.17:0/745859700 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8038070 con 0x7f43b00a8290 2024-03-06T07:26:40.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43ad59a640 1 -- 172.21.15.17:0/745859700 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8031700 con 0x7f43b00a8290 2024-03-06T07:26:40.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43ad59a640 1 -- 172.21.15.17:0/745859700 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8031ad0 con 0x7f43b00a8290 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 -- 172.21.15.17:0/745859700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 msgr2=0x7f43b01539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/745859700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b01539c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43a80097a0 tx=0x7f43a802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 -- 172.21.15.17:0/745859700 shutdown_connections 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/745859700 >> v2:172.21.15.163:3300/0 conn(0x7f43b00c69a0 0x7f43b0072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/745859700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b01539c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 -- 172.21.15.17:0/745859700 >> 172.21.15.17:0/745859700 conn(0x7f43b00c5560 msgr2=0x7f43b00c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 -- 172.21.15.17:0/745859700 shutdown_connections 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.061+0000 7f43b4d5e640 1 -- 172.21.15.17:0/745859700 wait complete. 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 Processor -- start 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 -- start start 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b0149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b00c4740 0x7f43b00c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c6e50 con 0x7f43b00c69a0 2024-03-06T07:26:40.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b006fc80 con 0x7f43b00c4740 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43b4d5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0072bc0 con 0x7f43b00a8290 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43aed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.063+0000 7f43aed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50234/0 (socket says 172.21.15.17:50234) 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 -- 172.21.15.17:0/852892226 learned_addr learned my addr 172.21.15.17:0/852892226 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43ae59c640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b0149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43add9b640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b00c4740 0x7f43b00c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 -- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 msgr2=0x7f43b0149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b0149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 -- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b00c4740 msgr2=0x7f43b00c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b00c4740 0x7f43b00c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 -- 172.21.15.17:0/852892226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b014a3b0 con 0x7f43b00c69a0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43aed9d640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009e60 tx=0x7f43a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0018030 con 0x7f43b00c69a0 2024-03-06T07:26:40.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a000cdf0 con 0x7f43b00c69a0 2024-03-06T07:26:40.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b01544c0 con 0x7f43b00c69a0 2024-03-06T07:26:40.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0003bb0 con 0x7f43b00c69a0 2024-03-06T07:26:40.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.064+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b006e080 con 0x7f43b00c69a0 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.065+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b01542b0 con 0x7f43b00c69a0 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.066+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43a001b070 con 0x7f43b00c69a0 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.066+0000 7f43a6ffd640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 0x7f439006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.066+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43a008ab70 con 0x7f43b00c69a0 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.066+0000 7f43ae59c640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 0x7f439006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:40.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.067+0000 7f43ae59c640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 0x7f439006d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f43a8035000 tx=0x7f43a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:40.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.071+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f43a0010040 con 0x7f43b00c69a0 2024-03-06T07:26:40.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:40.295+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43b004f330 con 0x7f43b00c69a0 2024-03-06T07:26:41.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.266+0000 7f43a6ffd640 1 -- 172.21.15.17:0/852892226 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v159) v1 ==== 118+0+0 (secure 0 0 0) 0x7f43a00592a0 con 0x7f43b00c69a0 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 msgr2=0x7f439006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 0x7f439006d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f43a8035000 tx=0x7f43a8008030 comp rx=0 tx=0).stop 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 msgr2=0x7f43b01dd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009e60 tx=0x7f43a000c490 comp rx=0 tx=0).stop 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 shutdown_connections 2024-03-06T07:26:41.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f439006b4d0 0x7f439006d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b00c69a0 0x7f43b01dd530 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b00c4740 0x7f43b00c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 --2- 172.21.15.17:0/852892226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b00a8290 0x7f43b0149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 >> 172.21.15.17:0/852892226 conn(0x7f43b00c5560 msgr2=0x7f43b00a9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.269+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 shutdown_connections 2024-03-06T07:26:41.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.270+0000 7f43b4d5e640 1 -- 172.21.15.17:0/852892226 wait complete. 2024-03-06T07:26:41.282 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:26:41.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:26:41.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.384+0000 7fdeec3d7640 1 Processor -- start 2024-03-06T07:26:41.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.384+0000 7fdeec3d7640 1 -- start start 2024-03-06T07:26:41.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeec3d7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee40b0070 0x7fdee4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeec3d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 0x7fdee40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeec3d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee40b0450 con 0x7fdee406ca70 2024-03-06T07:26:41.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeec3d7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee406c020 con 0x7fdee40b0070 2024-03-06T07:26:41.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeec3d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee406cf20 con 0x7fdee40c5380 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeea173640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee40b0070 0x7fdee4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeea173640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee40b0070 0x7fdee4155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56720/0 (socket says 172.21.15.17:56720) 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.385+0000 7fdeea173640 1 -- 172.21.15.17:0/2404326586 learned_addr learned my addr 172.21.15.17:0/2404326586 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea974640 1 --2- 172.21.15.17:0/2404326586 >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 0x7fdee40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea173640 1 -- 172.21.15.17:0/2404326586 >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 msgr2=0x7fdee40c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea173640 1 --2- 172.21.15.17:0/2404326586 >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 0x7fdee40c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea173640 1 -- 172.21.15.17:0/2404326586 >> v1:172.21.15.163:6789/0 conn(0x7fdee406ca70 legacy=0x7fdee40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea173640 1 -- 172.21.15.17:0/2404326586 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40c77f0 con 0x7fdee40b0070 2024-03-06T07:26:41.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea974640 1 --2- 172.21.15.17:0/2404326586 >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 0x7fdee40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.386+0000 7fdeea173640 1 --2- 172.21.15.17:0/2404326586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee4155c30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fdee40be030 tx=0x7fdee002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466109637fdbf00d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdee9171640 1 -- 172.21.15.17:0/2404326586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee002e070 con 0x7fdee40b0070 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdee9171640 1 -- 172.21.15.17:0/2404326586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee0037070 con 0x7fdee40b0070 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdee9171640 1 -- 172.21.15.17:0/2404326586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0031a00 con 0x7fdee40b0070 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 -- 172.21.15.17:0/2404326586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 msgr2=0x7fdee4155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/2404326586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee4155c30 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fdee40be030 tx=0x7fdee002b880 comp rx=0 tx=0).stop 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 -- 172.21.15.17:0/2404326586 shutdown_connections 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/2404326586 >> v2:172.21.15.163:3300/0 conn(0x7fdee40c5380 0x7fdee40c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/2404326586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee4155c30 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 -- 172.21.15.17:0/2404326586 >> 172.21.15.17:0/2404326586 conn(0x7fdee40ac590 msgr2=0x7fdee40ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 -- 172.21.15.17:0/2404326586 shutdown_connections 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.387+0000 7fdeec3d7640 1 -- 172.21.15.17:0/2404326586 wait complete. 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.388+0000 7fdeec3d7640 1 Processor -- start 2024-03-06T07:26:41.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.388+0000 7fdeec3d7640 1 -- start start 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 0x7fdee4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee40c5380 0x7fdee41ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406c550 con 0x7fdee40b0070 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406c270 con 0x7fdee406ca70 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeec3d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406da80 con 0x7fdee40c5380 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56724/0 (socket says 172.21.15.17:56724) 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 -- 172.21.15.17:0/735587098 learned_addr learned my addr 172.21.15.17:0/735587098 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeea173640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 0x7fdee4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdeea974640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee40c5380 0x7fdee41ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 -- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee40c5380 msgr2=0x7fdee41ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee40c5380 0x7fdee41ee050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 -- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 msgr2=0x7fdee4157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.389+0000 7fdee9972640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 0x7fdee4157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fdee9972640 1 -- 172.21.15.17:0/735587098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4156a40 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fdeea173640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 0x7fdee4157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fdee9972640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fded800b3f0 tx=0x7fded800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8015020 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8011830 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4156d20 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8011c20 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.390+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee4158a40 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.391+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee404f330 con 0x7fdee40b0070 2024-03-06T07:26:41.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.391+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fded8018020 con 0x7fdee40b0070 2024-03-06T07:26:41.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.392+0000 7fded6ffd640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 0x7fdec806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:41.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.392+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fded8089fc0 con 0x7fdee40b0070 2024-03-06T07:26:41.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.392+0000 7fdeea173640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 0x7fdec806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:41.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.393+0000 7fdeea173640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 0x7fdec806d9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdee0036000 tx=0x7fdee003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:41.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.396+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fded801f030 con 0x7fdee40b0070 2024-03-06T07:26:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:41.797+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdee40ca380 con 0x7fdee40b0070 2024-03-06T07:26:42.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.272+0000 7fded6ffd640 1 -- 172.21.15.17:0/735587098 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v160) v1 ==== 122+0+0 (secure 0 0 0) 0x7fded80172c0 con 0x7fdee40b0070 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 msgr2=0x7fdec806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 0x7fdec806d9c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdee0036000 tx=0x7fdee003d040 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 msgr2=0x7fdee40c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fded800b3f0 tx=0x7fded800b8b0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 shutdown_connections 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdec806b500 0x7fdec806d9c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee40c5380 0x7fdee41ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee40b0070 0x7fdee40c4530 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 --2- 172.21.15.17:0/735587098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee406ca70 0x7fdee4157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 >> 172.21.15.17:0/735587098 conn(0x7fdee40ac590 msgr2=0x7fdee40ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 shutdown_connections 2024-03-06T07:26:42.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.274+0000 7fdeec3d7640 1 -- 172.21.15.17:0/735587098 wait complete. 2024-03-06T07:26:42.286 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:26:42.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.386+0000 7ff9e2a78640 1 Processor -- start 2024-03-06T07:26:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.386+0000 7ff9e2a78640 1 -- start start 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e2a78640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9dc14f790 0x7ff9dc1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e2a78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e2a78640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9dc0c28d0 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e2a78640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9dc0c2680 con 0x7ff9dc14f790 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e2a78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0c2bb0 con 0x7ff9dc0c1aa0 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9dc14f790 0x7ff9dc1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9dc14f790 0x7ff9dc1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56726/0 (socket says 172.21.15.17:56726) 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 -- 172.21.15.17:0/1353058164 learned_addr learned my addr 172.21.15.17:0/1353058164 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9dbfff640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 -- 172.21.15.17:0/1353058164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 msgr2=0x7ff9dc0c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc0c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 -- 172.21.15.17:0/1353058164 >> v1:172.21.15.163:6789/0 conn(0x7ff9dc0c5c60 legacy=0x7ff9dc155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.387+0000 7ff9e0814640 1 -- 172.21.15.17:0/1353058164 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc0c2e90 con 0x7ff9dc14f790 2024-03-06T07:26:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.388+0000 7ff9e0814640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1529d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009870 tx=0x7ff9cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b622eba8874c9d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.388+0000 7ff9db7fe640 1 -- 172.21.15.17:0/1353058164 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc034070 con 0x7ff9dc14f790 2024-03-06T07:26:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.388+0000 7ff9db7fe640 1 -- 172.21.15.17:0/1353058164 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc030840 con 0x7ff9dc14f790 2024-03-06T07:26:42.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.388+0000 7ff9db7fe640 1 -- 172.21.15.17:0/1353058164 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc030c10 con 0x7ff9dc14f790 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 -- 172.21.15.17:0/1353058164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc14f790 msgr2=0x7ff9dc1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1529d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009870 tx=0x7ff9cc02b980 comp rx=0 tx=0).stop 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 -- 172.21.15.17:0/1353058164 shutdown_connections 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc0c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/1353058164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1529d0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 -- 172.21.15.17:0/1353058164 >> 172.21.15.17:0/1353058164 conn(0x7ff9dc0c6710 msgr2=0x7ff9dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 -- 172.21.15.17:0/1353058164 shutdown_connections 2024-03-06T07:26:42.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.389+0000 7ff9e2a78640 1 -- 172.21.15.17:0/1353058164 wait complete. 2024-03-06T07:26:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.391+0000 7ff9e2a78640 1 Processor -- start 2024-03-06T07:26:42.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.391+0000 7ff9e2a78640 1 -- start start 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc06cc00 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc0c0d60 con 0x7ff9dc14f790 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e2a78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc06cd80 con 0x7ff9dc0c1aa0 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56742/0 (socket says 172.21.15.17:56742) 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 -- 172.21.15.17:0/357100146 learned_addr learned my addr 172.21.15.17:0/357100146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e0814640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9e1015640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 -- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 msgr2=0x7ff9dc14e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc14e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 -- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9dc14f790 msgr2=0x7ff9dc1e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.392+0000 7ff9dbfff640 1 -- 172.21.15.17:0/357100146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc1df350 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9dbfff640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000faa0 tx=0x7ff9d000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0015070 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d0007ce0 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc14ed70 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d001a6a0 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.393+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc1eb7c0 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.394+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc04f330 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.395+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9d0011450 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.395+0000 7ff9d97fa640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 0x7ff9c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.395+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9d008cad0 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.395+0000 7ff9e0814640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 0x7ff9c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.396+0000 7ff9e0814640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 0x7ff9c806d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc036000 tx=0x7ff9cc03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.399+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9dc14ed70 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.827+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9dc0c1f40 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.828+0000 7ff9d97fa640 1 -- 172.21.15.17:0/357100146 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+4993 (secure 0 0 0) 0x7ff9d005b200 con 0x7ff9dc0c5c60 2024-03-06T07:26:42.830 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:42.830 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":160,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:41.798922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9432,"qdb_cluster":[9432]},"id":13}]} 2024-03-06T07:26:42.830 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 160 2024-03-06T07:26:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 msgr2=0x7ff9c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 0x7ff9c806d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc036000 tx=0x7ff9cc03b040 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 msgr2=0x7ff9dc0a6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000faa0 tx=0x7ff9d000ff60 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 shutdown_connections 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff9c806b500 0x7ff9c806d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9dc14f790 0x7ff9dc1e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c5c60 0x7ff9dc0a6d60 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 --2- 172.21.15.17:0/357100146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9dc0c1aa0 0x7ff9dc14e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 >> 172.21.15.17:0/357100146 conn(0x7ff9dc0c6710 msgr2=0x7ff9dc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 shutdown_connections 2024-03-06T07:26:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.831+0000 7ff9e2a78640 1 -- 172.21.15.17:0/357100146 wait complete. 2024-03-06T07:26:42.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.943+0000 7fc642e83640 1 Processor -- start 2024-03-06T07:26:42.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.943+0000 7fc642e83640 1 -- start start 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc642e83640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 0x7fc63c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc642e83640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc63c0c1320 0x7fc63c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc642e83640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63c153a60 con 0x7fc63c0c6000 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc642e83640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63c153d40 con 0x7fc63c0c1320 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc642e83640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63c06c2c0 con 0x7fc63c150580 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc63c0c1320 0x7fc63c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc641420640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc63c0c6000 0x7fc63c1566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45754/0 (socket says 172.21.15.17:45754) 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc641420640 1 -- 172.21.15.17:0/1691519915 learned_addr learned my addr 172.21.15.17:0/1691519915 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:42.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc640c1f640 1 --2- 172.21.15.17:0/1691519915 >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 0x7fc63c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63bfff640 1 -- 172.21.15.17:0/1691519915 >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 msgr2=0x7fc63c06dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63bfff640 1 --2- 172.21.15.17:0/1691519915 >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 0x7fc63c06dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63bfff640 1 -- 172.21.15.17:0/1691519915 >> v1:172.21.15.17:6789/0 conn(0x7fc63c0c6000 legacy=0x7fc63c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63b7fe640 1 -- 172.21.15.17:0/1691519915 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711320107 0 0) 0x7fc63c153a60 con 0x7fc63c0c6000 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc63bfff640 1 -- 172.21.15.17:0/1691519915 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c06c5a0 con 0x7fc63c0c1320 2024-03-06T07:26:42.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.944+0000 7fc640c1f640 1 --2- 172.21.15.17:0/1691519915 >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 0x7fc63c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.945+0000 7fc63bfff640 1 --2- 172.21.15.17:0/1691519915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c0c1320 0x7fc63c06e250 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0a9550 tx=0x7fc62802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650d94aba9026bf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.945+0000 7fc63b7fe640 1 -- 172.21.15.17:0/1691519915 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280315a0 con 0x7fc63c0c1320 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.945+0000 7fc63b7fe640 1 -- 172.21.15.17:0/1691519915 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628037070 con 0x7fc63c0c1320 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.945+0000 7fc63b7fe640 1 -- 172.21.15.17:0/1691519915 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628031960 con 0x7fc63c0c1320 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 -- 172.21.15.17:0/1691519915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c0c1320 msgr2=0x7fc63c06e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 --2- 172.21.15.17:0/1691519915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c0c1320 0x7fc63c06e250 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0a9550 tx=0x7fc62802b840 comp rx=0 tx=0).stop 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 -- 172.21.15.17:0/1691519915 shutdown_connections 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 --2- 172.21.15.17:0/1691519915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c0c1320 0x7fc63c06e250 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 --2- 172.21.15.17:0/1691519915 >> v2:172.21.15.163:3300/0 conn(0x7fc63c150580 0x7fc63c06dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 -- 172.21.15.17:0/1691519915 >> 172.21.15.17:0/1691519915 conn(0x7fc63c0c6710 msgr2=0x7fc63c0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 -- 172.21.15.17:0/1691519915 shutdown_connections 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.946+0000 7fc642e83640 1 -- 172.21.15.17:0/1691519915 wait complete. 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.947+0000 7fc642e83640 1 Processor -- start 2024-03-06T07:26:42.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.947+0000 7fc642e83640 1 -- start start 2024-03-06T07:26:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc63c0c1320 0x7fc63c1eab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc63c0c6000 0x7fc63c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0c0dc0 con 0x7fc63c150580 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c0c0c40 con 0x7fc63c0c6000 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c156dc0 con 0x7fc63c0c1320 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56760/0 (socket says 172.21.15.17:56760) 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 -- 172.21.15.17:0/2030690340 learned_addr learned my addr 172.21.15.17:0/2030690340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc63bfff640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc63c0c6000 0x7fc63c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 -- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc63c0c1320 msgr2=0x7fc63c1eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc63c0c1320 0x7fc63c1eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 -- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc63c0c6000 msgr2=0x7fc63c1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc63c0c6000 0x7fc63c1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 -- 172.21.15.17:0/2030690340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c14e5c0 con 0x7fc63c150580 2024-03-06T07:26:42.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc641420640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c7a0 tx=0x7fc63400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634018070 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63400a640 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c152710 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400aab0 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.948+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c152cd0 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.949+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c04f330 con 0x7fc63c150580 2024-03-06T07:26:42.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.950+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc63401b070 con 0x7fc63c150580 2024-03-06T07:26:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.950+0000 7fc6397fa640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.951+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc634089940 con 0x7fc63c150580 2024-03-06T07:26:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.951+0000 7fc640c1f640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:42.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.951+0000 7fc640c1f640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 0x7fc62006d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc630008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:42.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:42.955+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc634058630 con 0x7fc63c150580 2024-03-06T07:26:43.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.138+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc63c153850 con 0x7fc63c150580 2024-03-06T07:26:43.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.139+0000 7fc6397fa640 1 -- 172.21.15.17:0/2030690340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16952 (secure 0 0 0) 0x7fc634057f30 con 0x7fc63c150580 2024-03-06T07:26:43.141 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:43.141 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":172,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:40.243399+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:26:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 msgr2=0x7fc62006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 0x7fc62006d9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc630006fd0 tx=0x7fc630008040 comp rx=0 tx=0).stop 2024-03-06T07:26:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 msgr2=0x7fc63c1ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c7a0 tx=0x7fc63400cc60 comp rx=0 tx=0).stop 2024-03-06T07:26:43.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 shutdown_connections 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc62006b500 0x7fc62006d9c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc63c150580 0x7fc63c1ee770 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc63c0c6000 0x7fc63c1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 --2- 172.21.15.17:0/2030690340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc63c0c1320 0x7fc63c1eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 >> 172.21.15.17:0/2030690340 conn(0x7fc63c0c6710 msgr2=0x7fc63c0c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 shutdown_connections 2024-03-06T07:26:43.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.141+0000 7fc642e83640 1 -- 172.21.15.17:0/2030690340 wait complete. 2024-03-06T07:26:43.155 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:26:43.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 Processor -- start 2024-03-06T07:26:43.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 -- start start 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0054152960 0x7f005406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 0x7f00541552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0054152d40 con 0x7f00540ac260 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0054066e30 con 0x7f0054152960 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0059d63640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00540c1dc0 con 0x7f00541592d0 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f00537fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0054152960 0x7f005406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.256+0000 7f0053fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f00540ac260 0x7f0054155810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45766/0 (socket says 172.21.15.17:45766) 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f0053fff640 1 -- 172.21.15.17:0/1547823803 learned_addr learned my addr 172.21.15.17:0/1547823803 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f00537fe640 1 -- 172.21.15.17:0/1547823803 >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 msgr2=0x7f00541552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f0052ffd640 1 --2- 172.21.15.17:0/1547823803 >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 0x7f00541552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f00537fe640 1 --2- 172.21.15.17:0/1547823803 >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 0x7f00541552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f00537fe640 1 -- 172.21.15.17:0/1547823803 >> v1:172.21.15.17:6789/0 conn(0x7f00540ac260 legacy=0x7f0054155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f00527fc640 1 -- 172.21.15.17:0/1547823803 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64666392 0 0) 0x7f0054152d40 con 0x7f00540ac260 2024-03-06T07:26:43.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f00537fe640 1 -- 172.21.15.17:0/1547823803 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540c1b50 con 0x7f0054152960 2024-03-06T07:26:43.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.257+0000 7f0052ffd640 1 --2- 172.21.15.17:0/1547823803 >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 0x7f00541552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f00537fe640 1 --2- 172.21.15.17:0/1547823803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f005406dc90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f005406ee90 tx=0x7f004402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a8ce83b2794f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f00527fc640 1 -- 172.21.15.17:0/1547823803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044035020 con 0x7f0054152960 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f00527fc640 1 -- 172.21.15.17:0/1547823803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044031830 con 0x7f0054152960 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f00527fc640 1 -- 172.21.15.17:0/1547823803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044031c00 con 0x7f0054152960 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 -- 172.21.15.17:0/1547823803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 msgr2=0x7f005406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 --2- 172.21.15.17:0/1547823803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f005406dc90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f005406ee90 tx=0x7f004402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 -- 172.21.15.17:0/1547823803 shutdown_connections 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 --2- 172.21.15.17:0/1547823803 >> v2:172.21.15.163:3300/0 conn(0x7f00541592d0 0x7f00541552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 --2- 172.21.15.17:0/1547823803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f005406dc90 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 -- 172.21.15.17:0/1547823803 >> 172.21.15.17:0/1547823803 conn(0x7f00540c70b0 msgr2=0x7f00540c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 -- 172.21.15.17:0/1547823803 shutdown_connections 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.258+0000 7f0059d63640 1 -- 172.21.15.17:0/1547823803 wait complete. 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.259+0000 7f0059d63640 1 Processor -- start 2024-03-06T07:26:43.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 -- start start 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00540ac260 0x7f0054071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0054154a60 0x7f0054157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540c20a0 con 0x7f0054152960 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054159760 con 0x7f00540ac260 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0059d63640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00540c2220 con 0x7f0054154a60 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0052ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0052ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56774/0 (socket says 172.21.15.17:56774) 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.260+0000 7f0052ffd640 1 -- 172.21.15.17:0/3937034837 learned_addr learned my addr 172.21.15.17:0/3937034837 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 -- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0054154a60 msgr2=0x7f0054157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0054154a60 0x7f0054157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 -- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00540ac260 msgr2=0x7f0054071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f00537fe640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00540ac260 0x7f0054071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00540ac260 0x7f0054071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 -- 172.21.15.17:0/3937034837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540722f0 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0052ffd640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c820 tx=0x7f003c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c015070 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c007a30 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00540725d0 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c007e00 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.261+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0054154f10 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.262+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005404f330 con 0x7f0054152960 2024-03-06T07:26:43.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.263+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f003c018070 con 0x7f0054152960 2024-03-06T07:26:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.263+0000 7f004bfff640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 0x7f003406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.263+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f003c08a2d0 con 0x7f0054152960 2024-03-06T07:26:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.263+0000 7f00537fe640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 0x7f003406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.264+0000 7f00537fe640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 0x7f003406d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0044036000 tx=0x7f0044008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.267+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f003c058fc0 con 0x7f0054152960 2024-03-06T07:26:43.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.446+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00540c4f50 con 0x7f003406b500 2024-03-06T07:26:43.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.453+0000 7f004bfff640 1 -- 172.21.15.17:0/3937034837 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252569 (secure 0 0 0) 0x7f00540c4f50 con 0x7f003406b500 2024-03-06T07:26:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:43.458 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 msgr2=0x7f003406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 0x7f003406d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0044036000 tx=0x7f0044008030 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 msgr2=0x7f00541eeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c820 tx=0x7f003c00cc20 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 shutdown_connections 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f003406b500 0x7f003406d9c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0054154a60 0x7f0054157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0054152960 0x7f00541eeeb0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 --2- 172.21.15.17:0/3937034837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00540ac260 0x7f0054071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 >> 172.21.15.17:0/3937034837 conn(0x7f00540c70b0 msgr2=0x7f00540aa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 shutdown_connections 2024-03-06T07:26:43.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.456+0000 7f0059d63640 1 -- 172.21.15.17:0/3937034837 wait complete. 2024-03-06T07:26:43.469 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-03-06T07:26:43.429917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270808,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703912,"statfs":{"total":773094113280,"available":772816805888,"internally_reserved":0,"allocated":17596416,"data_stored":9702972,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001144"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255796+0000","last_change":"2024-03-06T07:26:39.265235+0000","last_active":"2024-03-06T07:26:40.255796+0000","last_peered":"2024-03-06T07:26:40.255796+0000","last_clean":"2024-03-06T07:26:40.255796+0000","last_became_active":"2024-03-06T07:26:39.264159+0000","last_became_peered":"2024-03-06T07:26:39.264159+0000","last_unstale":"2024-03-06T07:26:40.255796+0000","last_undegraded":"2024-03-06T07:26:40.255796+0000","last_fullsized":"2024-03-06T07:26:40.255796+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:36.272472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249827+0000","last_change":"2024-03-06T07:26:39.267139+0000","last_active":"2024-03-06T07:26:40.249827+0000","last_peered":"2024-03-06T07:26:40.249827+0000","last_clean":"2024-03-06T07:26:40.249827+0000","last_became_active":"2024-03-06T07:26:39.266830+0000","last_became_peered":"2024-03-06T07:26:39.266830+0000","last_unstale":"2024-03-06T07:26:40.249827+0000","last_undegraded":"2024-03-06T07:26:40.249827+0000","last_fullsized":"2024-03-06T07:26:40.249827+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:21.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271395+0000","last_change":"2024-03-06T07:26:39.265218+0000","last_active":"2024-03-06T07:26:40.271395+0000","last_peered":"2024-03-06T07:26:40.271395+0000","last_clean":"2024-03-06T07:26:40.271395+0000","last_became_active":"2024-03-06T07:26:39.264926+0000","last_became_peered":"2024-03-06T07:26:39.264926+0000","last_unstale":"2024-03-06T07:26:40.271395+0000","last_undegraded":"2024-03-06T07:26:40.271395+0000","last_fullsized":"2024-03-06T07:26:40.271395+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:33.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310980+0000","last_change":"2024-03-06T07:26:39.262329+0000","last_active":"2024-03-06T07:26:41.310980+0000","last_peered":"2024-03-06T07:26:41.310980+0000","last_clean":"2024-03-06T07:26:41.310980+0000","last_became_active":"2024-03-06T07:26:39.262051+0000","last_became_peered":"2024-03-06T07:26:39.262051+0000","last_unstale":"2024-03-06T07:26:41.310980+0000","last_undegraded":"2024-03-06T07:26:41.310980+0000","last_fullsized":"2024-03-06T07:26:41.310980+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:05.819935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320499+0000","last_change":"2024-03-06T07:26:39.270056+0000","last_active":"2024-03-06T07:26:40.320499+0000","last_peered":"2024-03-06T07:26:40.320499+0000","last_clean":"2024-03-06T07:26:40.320499+0000","last_became_active":"2024-03-06T07:26:39.269621+0000","last_became_peered":"2024-03-06T07:26:39.269621+0000","last_unstale":"2024-03-06T07:26:40.320499+0000","last_undegraded":"2024-03-06T07:26:40.320499+0000","last_fullsized":"2024-03-06T07:26:40.320499+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:46.358864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310775+0000","last_change":"2024-03-06T07:26:39.266307+0000","last_active":"2024-03-06T07:26:41.310775+0000","last_peered":"2024-03-06T07:26:41.310775+0000","last_clean":"2024-03-06T07:26:41.310775+0000","last_became_active":"2024-03-06T07:26:39.265649+0000","last_became_peered":"2024-03-06T07:26:39.265649+0000","last_unstale":"2024-03-06T07:26:41.310775+0000","last_undegraded":"2024-03-06T07:26:41.310775+0000","last_fullsized":"2024-03-06T07:26:41.310775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:30.673499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320441+0000","last_change":"2024-03-06T07:26:39.267985+0000","last_active":"2024-03-06T07:26:40.320441+0000","last_peered":"2024-03-06T07:26:40.320441+0000","last_clean":"2024-03-06T07:26:40.320441+0000","last_became_active":"2024-03-06T07:26:39.267673+0000","last_became_peered":"2024-03-06T07:26:39.267673+0000","last_unstale":"2024-03-06T07:26:40.320441+0000","last_undegraded":"2024-03-06T07:26:40.320441+0000","last_fullsized":"2024-03-06T07:26:40.320441+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:20.340530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254550+0000","last_change":"2024-03-06T07:26:39.257459+0000","last_active":"2024-03-06T07:26:40.254550+0000","last_peered":"2024-03-06T07:26:40.254550+0000","last_clean":"2024-03-06T07:26:40.254550+0000","last_became_active":"2024-03-06T07:26:39.257193+0000","last_became_peered":"2024-03-06T07:26:39.257193+0000","last_unstale":"2024-03-06T07:26:40.254550+0000","last_undegraded":"2024-03-06T07:26:40.254550+0000","last_fullsized":"2024-03-06T07:26:40.254550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:06.900442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810346+0000","last_change":"2024-03-06T07:26:39.272838+0000","last_active":"2024-03-06T07:26:40.810346+0000","last_peered":"2024-03-06T07:26:40.810346+0000","last_clean":"2024-03-06T07:26:40.810346+0000","last_became_active":"2024-03-06T07:26:39.272564+0000","last_became_peered":"2024-03-06T07:26:39.272564+0000","last_unstale":"2024-03-06T07:26:40.810346+0000","last_undegraded":"2024-03-06T07:26:40.810346+0000","last_fullsized":"2024-03-06T07:26:40.810346+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:36.142892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255978+0000","last_change":"2024-03-06T07:26:39.269676+0000","last_active":"2024-03-06T07:26:40.255978+0000","last_peered":"2024-03-06T07:26:40.255978+0000","last_clean":"2024-03-06T07:26:40.255978+0000","last_became_active":"2024-03-06T07:26:39.269377+0000","last_became_peered":"2024-03-06T07:26:39.269377+0000","last_unstale":"2024-03-06T07:26:40.255978+0000","last_undegraded":"2024-03-06T07:26:40.255978+0000","last_fullsized":"2024-03-06T07:26:40.255978+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:41.945333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271443+0000","last_change":"2024-03-06T07:26:39.262387+0000","last_active":"2024-03-06T07:26:40.271443+0000","last_peered":"2024-03-06T07:26:40.271443+0000","last_clean":"2024-03-06T07:26:40.271443+0000","last_became_active":"2024-03-06T07:26:39.261697+0000","last_became_peered":"2024-03-06T07:26:39.261697+0000","last_unstale":"2024-03-06T07:26:40.271443+0000","last_undegraded":"2024-03-06T07:26:40.271443+0000","last_fullsized":"2024-03-06T07:26:40.271443+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:43.510306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320686+0000","last_change":"2024-03-06T07:26:39.265828+0000","last_active":"2024-03-06T07:26:40.320686+0000","last_peered":"2024-03-06T07:26:40.320686+0000","last_clean":"2024-03-06T07:26:40.320686+0000","last_became_active":"2024-03-06T07:26:39.265524+0000","last_became_peered":"2024-03-06T07:26:39.265524+0000","last_unstale":"2024-03-06T07:26:40.320686+0000","last_undegraded":"2024-03-06T07:26:40.320686+0000","last_fullsized":"2024-03-06T07:26:40.320686+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:03.076037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311106+0000","last_change":"2024-03-06T07:26:39.268793+0000","last_active":"2024-03-06T07:26:41.311106+0000","last_peered":"2024-03-06T07:26:41.311106+0000","last_clean":"2024-03-06T07:26:41.311106+0000","last_became_active":"2024-03-06T07:26:39.268258+0000","last_became_peered":"2024-03-06T07:26:39.268258+0000","last_unstale":"2024-03-06T07:26:41.311106+0000","last_undegraded":"2024-03-06T07:26:41.311106+0000","last_fullsized":"2024-03-06T07:26:41.311106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:29.776010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255739+0000","last_change":"2024-03-06T07:26:39.254939+0000","last_active":"2024-03-06T07:26:40.255739+0000","last_peered":"2024-03-06T07:26:40.255739+0000","last_clean":"2024-03-06T07:26:40.255739+0000","last_became_active":"2024-03-06T07:26:39.254669+0000","last_became_peered":"2024-03-06T07:26:39.254669+0000","last_unstale":"2024-03-06T07:26:40.255739+0000","last_undegraded":"2024-03-06T07:26:40.255739+0000","last_fullsized":"2024-03-06T07:26:40.255739+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:16.774222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310644+0000","last_change":"2024-03-06T07:26:39.264952+0000","last_active":"2024-03-06T07:26:41.310644+0000","last_peered":"2024-03-06T07:26:41.310644+0000","last_clean":"2024-03-06T07:26:41.310644+0000","last_became_active":"2024-03-06T07:26:39.264676+0000","last_became_peered":"2024-03-06T07:26:39.264676+0000","last_unstale":"2024-03-06T07:26:41.310644+0000","last_undegraded":"2024-03-06T07:26:41.310644+0000","last_fullsized":"2024-03-06T07:26:41.310644+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:18.459682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271200+0000","last_change":"2024-03-06T07:26:39.252696+0000","last_active":"2024-03-06T07:26:40.271200+0000","last_peered":"2024-03-06T07:26:40.271200+0000","last_clean":"2024-03-06T07:26:40.271200+0000","last_became_active":"2024-03-06T07:26:39.252405+0000","last_became_peered":"2024-03-06T07:26:39.252405+0000","last_unstale":"2024-03-06T07:26:40.271200+0000","last_undegraded":"2024-03-06T07:26:40.271200+0000","last_fullsized":"2024-03-06T07:26:40.271200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:33:58.413688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810359+0000","last_change":"2024-03-06T07:26:39.265006+0000","last_active":"2024-03-06T07:26:40.810359+0000","last_peered":"2024-03-06T07:26:40.810359+0000","last_clean":"2024-03-06T07:26:40.810359+0000","last_became_active":"2024-03-06T07:26:39.264684+0000","last_became_peered":"2024-03-06T07:26:39.264684+0000","last_unstale":"2024-03-06T07:26:40.810359+0000","last_undegraded":"2024-03-06T07:26:40.810359+0000","last_fullsized":"2024-03-06T07:26:40.810359+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:50.404141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810273+0000","last_change":"2024-03-06T07:26:39.271941+0000","last_active":"2024-03-06T07:26:40.810273+0000","last_peered":"2024-03-06T07:26:40.810273+0000","last_clean":"2024-03-06T07:26:40.810273+0000","last_became_active":"2024-03-06T07:26:39.271608+0000","last_became_peered":"2024-03-06T07:26:39.271608+0000","last_unstale":"2024-03-06T07:26:40.810273+0000","last_undegraded":"2024-03-06T07:26:40.810273+0000","last_fullsized":"2024-03-06T07:26:40.810273+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:18.287778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256266+0000","last_change":"2024-03-06T07:26:39.263669+0000","last_active":"2024-03-06T07:26:40.256266+0000","last_peered":"2024-03-06T07:26:40.256266+0000","last_clean":"2024-03-06T07:26:40.256266+0000","last_became_active":"2024-03-06T07:26:39.263176+0000","last_became_peered":"2024-03-06T07:26:39.263176+0000","last_unstale":"2024-03-06T07:26:40.256266+0000","last_undegraded":"2024-03-06T07:26:40.256266+0000","last_fullsized":"2024-03-06T07:26:40.256266+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:40.261455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255090+0000","last_change":"2024-03-06T07:26:39.269155+0000","last_active":"2024-03-06T07:26:40.255090+0000","last_peered":"2024-03-06T07:26:40.255090+0000","last_clean":"2024-03-06T07:26:40.255090+0000","last_became_active":"2024-03-06T07:26:39.268816+0000","last_became_peered":"2024-03-06T07:26:39.268816+0000","last_unstale":"2024-03-06T07:26:40.255090+0000","last_undegraded":"2024-03-06T07:26:40.255090+0000","last_fullsized":"2024-03-06T07:26:40.255090+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:05.278788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255540+0000","last_change":"2024-03-06T07:26:39.270600+0000","last_active":"2024-03-06T07:26:40.255540+0000","last_peered":"2024-03-06T07:26:40.255540+0000","last_clean":"2024-03-06T07:26:40.255540+0000","last_became_active":"2024-03-06T07:26:39.269961+0000","last_became_peered":"2024-03-06T07:26:39.269961+0000","last_unstale":"2024-03-06T07:26:40.255540+0000","last_undegraded":"2024-03-06T07:26:40.255540+0000","last_fullsized":"2024-03-06T07:26:40.255540+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:15.866846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256165+0000","last_change":"2024-03-06T07:26:39.254466+0000","last_active":"2024-03-06T07:26:40.256165+0000","last_peered":"2024-03-06T07:26:40.256165+0000","last_clean":"2024-03-06T07:26:40.256165+0000","last_became_active":"2024-03-06T07:26:39.254144+0000","last_became_peered":"2024-03-06T07:26:39.254144+0000","last_unstale":"2024-03-06T07:26:40.256165+0000","last_undegraded":"2024-03-06T07:26:40.256165+0000","last_fullsized":"2024-03-06T07:26:40.256165+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:47.975168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249749+0000","last_change":"2024-03-06T07:26:39.257785+0000","last_active":"2024-03-06T07:26:40.249749+0000","last_peered":"2024-03-06T07:26:40.249749+0000","last_clean":"2024-03-06T07:26:40.249749+0000","last_became_active":"2024-03-06T07:26:39.257539+0000","last_became_peered":"2024-03-06T07:26:39.257539+0000","last_unstale":"2024-03-06T07:26:40.249749+0000","last_undegraded":"2024-03-06T07:26:40.249749+0000","last_fullsized":"2024-03-06T07:26:40.249749+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:01.239394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320336+0000","last_change":"2024-03-06T07:26:39.254092+0000","last_active":"2024-03-06T07:26:40.320336+0000","last_peered":"2024-03-06T07:26:40.320336+0000","last_clean":"2024-03-06T07:26:40.320336+0000","last_became_active":"2024-03-06T07:26:39.253775+0000","last_became_peered":"2024-03-06T07:26:39.253775+0000","last_unstale":"2024-03-06T07:26:40.320336+0000","last_undegraded":"2024-03-06T07:26:40.320336+0000","last_fullsized":"2024-03-06T07:26:40.320336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:37.325289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810488+0000","last_change":"2024-03-06T07:26:39.272892+0000","last_active":"2024-03-06T07:26:40.810488+0000","last_peered":"2024-03-06T07:26:40.810488+0000","last_clean":"2024-03-06T07:26:40.810488+0000","last_became_active":"2024-03-06T07:26:39.272632+0000","last_became_peered":"2024-03-06T07:26:39.272632+0000","last_unstale":"2024-03-06T07:26:40.810488+0000","last_undegraded":"2024-03-06T07:26:40.810488+0000","last_fullsized":"2024-03-06T07:26:40.810488+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:22.390867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255697+0000","last_change":"2024-03-06T07:26:39.265364+0000","last_active":"2024-03-06T07:26:40.255697+0000","last_peered":"2024-03-06T07:26:40.255697+0000","last_clean":"2024-03-06T07:26:40.255697+0000","last_became_active":"2024-03-06T07:26:39.264432+0000","last_became_peered":"2024-03-06T07:26:39.264432+0000","last_unstale":"2024-03-06T07:26:40.255697+0000","last_undegraded":"2024-03-06T07:26:40.255697+0000","last_fullsized":"2024-03-06T07:26:40.255697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:10.356581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271526+0000","last_change":"2024-03-06T07:26:39.263268+0000","last_active":"2024-03-06T07:26:40.271526+0000","last_peered":"2024-03-06T07:26:40.271526+0000","last_clean":"2024-03-06T07:26:40.271526+0000","last_became_active":"2024-03-06T07:26:39.262966+0000","last_became_peered":"2024-03-06T07:26:39.262966+0000","last_unstale":"2024-03-06T07:26:40.271526+0000","last_undegraded":"2024-03-06T07:26:40.271526+0000","last_fullsized":"2024-03-06T07:26:40.271526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:24:17.414856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320737+0000","last_change":"2024-03-06T07:26:39.262772+0000","last_active":"2024-03-06T07:26:40.320737+0000","last_peered":"2024-03-06T07:26:40.320737+0000","last_clean":"2024-03-06T07:26:40.320737+0000","last_became_active":"2024-03-06T07:26:39.262388+0000","last_became_peered":"2024-03-06T07:26:39.262388+0000","last_unstale":"2024-03-06T07:26:40.320737+0000","last_undegraded":"2024-03-06T07:26:40.320737+0000","last_fullsized":"2024-03-06T07:26:40.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:47.316796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810051+0000","last_change":"2024-03-06T07:26:39.270056+0000","last_active":"2024-03-06T07:26:40.810051+0000","last_peered":"2024-03-06T07:26:40.810051+0000","last_clean":"2024-03-06T07:26:40.810051+0000","last_became_active":"2024-03-06T07:26:39.269740+0000","last_became_peered":"2024-03-06T07:26:39.269740+0000","last_unstale":"2024-03-06T07:26:40.810051+0000","last_undegraded":"2024-03-06T07:26:40.810051+0000","last_fullsized":"2024-03-06T07:26:40.810051+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:58.941044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310932+0000","last_change":"2024-03-06T07:26:39.271213+0000","last_active":"2024-03-06T07:26:41.310932+0000","last_peered":"2024-03-06T07:26:41.310932+0000","last_clean":"2024-03-06T07:26:41.310932+0000","last_became_active":"2024-03-06T07:26:39.270899+0000","last_became_peered":"2024-03-06T07:26:39.270899+0000","last_unstale":"2024-03-06T07:26:41.310932+0000","last_undegraded":"2024-03-06T07:26:41.310932+0000","last_fullsized":"2024-03-06T07:26:41.310932+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:46.785363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310749+0000","last_change":"2024-03-06T07:26:39.269155+0000","last_active":"2024-03-06T07:26:41.310749+0000","last_peered":"2024-03-06T07:26:41.310749+0000","last_clean":"2024-03-06T07:26:41.310749+0000","last_became_active":"2024-03-06T07:26:39.268877+0000","last_became_peered":"2024-03-06T07:26:39.268877+0000","last_unstale":"2024-03-06T07:26:41.310749+0000","last_undegraded":"2024-03-06T07:26:41.310749+0000","last_fullsized":"2024-03-06T07:26:41.310749+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:28.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310577+0000","last_change":"2024-03-06T07:26:39.265203+0000","last_active":"2024-03-06T07:26:41.310577+0000","last_peered":"2024-03-06T07:26:41.310577+0000","last_clean":"2024-03-06T07:26:41.310577+0000","last_became_active":"2024-03-06T07:26:39.264957+0000","last_became_peered":"2024-03-06T07:26:39.264957+0000","last_unstale":"2024-03-06T07:26:41.310577+0000","last_undegraded":"2024-03-06T07:26:41.310577+0000","last_fullsized":"2024-03-06T07:26:41.310577+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:02.114821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249642+0000","last_change":"2024-03-06T07:26:39.255177+0000","last_active":"2024-03-06T07:26:40.249642+0000","last_peered":"2024-03-06T07:26:40.249642+0000","last_clean":"2024-03-06T07:26:40.249642+0000","last_became_active":"2024-03-06T07:26:39.254844+0000","last_became_peered":"2024-03-06T07:26:39.254844+0000","last_unstale":"2024-03-06T07:26:40.249642+0000","last_undegraded":"2024-03-06T07:26:40.249642+0000","last_fullsized":"2024-03-06T07:26:40.249642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:19.212179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"171'4","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311074+0000","last_change":"2024-03-06T07:26:38.252288+0000","last_active":"2024-03-06T07:26:41.311074+0000","last_peered":"2024-03-06T07:26:41.311074+0000","last_clean":"2024-03-06T07:26:41.311074+0000","last_became_active":"2024-03-06T07:26:38.251978+0000","last_became_peered":"2024-03-06T07:26:38.251978+0000","last_unstale":"2024-03-06T07:26:41.311074+0000","last_undegraded":"2024-03-06T07:26:41.311074+0000","last_fullsized":"2024-03-06T07:26:41.311074+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:28.311316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255456+0000","last_change":"2024-03-06T07:26:39.264834+0000","last_active":"2024-03-06T07:26:40.255456+0000","last_peered":"2024-03-06T07:26:40.255456+0000","last_clean":"2024-03-06T07:26:40.255456+0000","last_became_active":"2024-03-06T07:26:39.264461+0000","last_became_peered":"2024-03-06T07:26:39.264461+0000","last_unstale":"2024-03-06T07:26:40.255456+0000","last_undegraded":"2024-03-06T07:26:40.255456+0000","last_fullsized":"2024-03-06T07:26:40.255456+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:15.442066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255608+0000","last_change":"2024-03-06T07:26:38.263737+0000","last_active":"2024-03-06T07:26:40.255608+0000","last_peered":"2024-03-06T07:26:40.255608+0000","last_clean":"2024-03-06T07:26:40.255608+0000","last_became_active":"2024-03-06T07:26:38.263193+0000","last_became_peered":"2024-03-06T07:26:38.263193+0000","last_unstale":"2024-03-06T07:26:40.255608+0000","last_undegraded":"2024-03-06T07:26:40.255608+0000","last_fullsized":"2024-03-06T07:26:40.255608+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:25.499508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256009+0000","last_change":"2024-03-06T07:26:39.255539+0000","last_active":"2024-03-06T07:26:40.256009+0000","last_peered":"2024-03-06T07:26:40.256009+0000","last_clean":"2024-03-06T07:26:40.256009+0000","last_became_active":"2024-03-06T07:26:39.255246+0000","last_became_peered":"2024-03-06T07:26:39.255246+0000","last_unstale":"2024-03-06T07:26:40.256009+0000","last_undegraded":"2024-03-06T07:26:40.256009+0000","last_fullsized":"2024-03-06T07:26:40.256009+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:39.543893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249648+0000","last_change":"2024-03-06T07:26:38.257296+0000","last_active":"2024-03-06T07:26:40.249648+0000","last_peered":"2024-03-06T07:26:40.249648+0000","last_clean":"2024-03-06T07:26:40.249648+0000","last_became_active":"2024-03-06T07:26:38.257010+0000","last_became_peered":"2024-03-06T07:26:38.257010+0000","last_unstale":"2024-03-06T07:26:40.249648+0000","last_undegraded":"2024-03-06T07:26:40.249648+0000","last_fullsized":"2024-03-06T07:26:40.249648+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:48.553025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320574+0000","last_change":"2024-03-06T07:26:39.253050+0000","last_active":"2024-03-06T07:26:40.320574+0000","last_peered":"2024-03-06T07:26:40.320574+0000","last_clean":"2024-03-06T07:26:40.320574+0000","last_became_active":"2024-03-06T07:26:39.252742+0000","last_became_peered":"2024-03-06T07:26:39.252742+0000","last_unstale":"2024-03-06T07:26:40.320574+0000","last_undegraded":"2024-03-06T07:26:40.320574+0000","last_fullsized":"2024-03-06T07:26:40.320574+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:53:04.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810072+0000","last_change":"2024-03-06T07:26:38.265662+0000","last_active":"2024-03-06T07:26:40.810072+0000","last_peered":"2024-03-06T07:26:40.810072+0000","last_clean":"2024-03-06T07:26:40.810072+0000","last_became_active":"2024-03-06T07:26:38.265394+0000","last_became_peered":"2024-03-06T07:26:38.265394+0000","last_unstale":"2024-03-06T07:26:40.810072+0000","last_undegraded":"2024-03-06T07:26:40.810072+0000","last_fullsized":"2024-03-06T07:26:40.810072+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:39.402401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311213+0000","last_change":"2024-03-06T07:26:39.267995+0000","last_active":"2024-03-06T07:26:41.311213+0000","last_peered":"2024-03-06T07:26:41.311213+0000","last_clean":"2024-03-06T07:26:41.311213+0000","last_became_active":"2024-03-06T07:26:39.267720+0000","last_became_peered":"2024-03-06T07:26:39.267720+0000","last_unstale":"2024-03-06T07:26:41.311213+0000","last_undegraded":"2024-03-06T07:26:41.311213+0000","last_fullsized":"2024-03-06T07:26:41.311213+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:26.578348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310921+0000","last_change":"2024-03-06T07:26:38.261164+0000","last_active":"2024-03-06T07:26:41.310921+0000","last_peered":"2024-03-06T07:26:41.310921+0000","last_clean":"2024-03-06T07:26:41.310921+0000","last_became_active":"2024-03-06T07:26:38.260678+0000","last_became_peered":"2024-03-06T07:26:38.260678+0000","last_unstale":"2024-03-06T07:26:41.310921+0000","last_undegraded":"2024-03-06T07:26:41.310921+0000","last_fullsized":"2024-03-06T07:26:41.310921+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:23.787933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311132+0000","last_change":"2024-03-06T07:26:39.269820+0000","last_active":"2024-03-06T07:26:41.311132+0000","last_peered":"2024-03-06T07:26:41.311132+0000","last_clean":"2024-03-06T07:26:41.311132+0000","last_became_active":"2024-03-06T07:26:39.269531+0000","last_became_peered":"2024-03-06T07:26:39.269531+0000","last_unstale":"2024-03-06T07:26:41.311132+0000","last_undegraded":"2024-03-06T07:26:41.311132+0000","last_fullsized":"2024-03-06T07:26:41.311132+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:26.505556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310799+0000","last_change":"2024-03-06T07:26:38.262646+0000","last_active":"2024-03-06T07:26:41.310799+0000","last_peered":"2024-03-06T07:26:41.310799+0000","last_clean":"2024-03-06T07:26:41.310799+0000","last_became_active":"2024-03-06T07:26:38.262377+0000","last_became_peered":"2024-03-06T07:26:38.262377+0000","last_unstale":"2024-03-06T07:26:41.310799+0000","last_undegraded":"2024-03-06T07:26:41.310799+0000","last_fullsized":"2024-03-06T07:26:41.310799+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:08.484807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810491+0000","last_change":"2024-03-06T07:26:39.264351+0000","last_active":"2024-03-06T07:26:40.810491+0000","last_peered":"2024-03-06T07:26:40.810491+0000","last_clean":"2024-03-06T07:26:40.810491+0000","last_became_active":"2024-03-06T07:26:39.264081+0000","last_became_peered":"2024-03-06T07:26:39.264081+0000","last_unstale":"2024-03-06T07:26:40.810491+0000","last_undegraded":"2024-03-06T07:26:40.810491+0000","last_fullsized":"2024-03-06T07:26:40.810491+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:49.559073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271186+0000","last_change":"2024-03-06T07:26:38.258205+0000","last_active":"2024-03-06T07:26:40.271186+0000","last_peered":"2024-03-06T07:26:40.271186+0000","last_clean":"2024-03-06T07:26:40.271186+0000","last_became_active":"2024-03-06T07:26:38.257947+0000","last_became_peered":"2024-03-06T07:26:38.257947+0000","last_unstale":"2024-03-06T07:26:40.271186+0000","last_undegraded":"2024-03-06T07:26:40.271186+0000","last_fullsized":"2024-03-06T07:26:40.271186+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:27.466247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310610+0000","last_change":"2024-03-06T07:26:39.266280+0000","last_active":"2024-03-06T07:26:41.310610+0000","last_peered":"2024-03-06T07:26:41.310610+0000","last_clean":"2024-03-06T07:26:41.310610+0000","last_became_active":"2024-03-06T07:26:39.266010+0000","last_became_peered":"2024-03-06T07:26:39.266010+0000","last_unstale":"2024-03-06T07:26:41.310610+0000","last_undegraded":"2024-03-06T07:26:41.310610+0000","last_fullsized":"2024-03-06T07:26:41.310610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:44.405245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320818+0000","last_change":"2024-03-06T07:26:38.260935+0000","last_active":"2024-03-06T07:26:40.320818+0000","last_peered":"2024-03-06T07:26:40.320818+0000","last_clean":"2024-03-06T07:26:40.320818+0000","last_became_active":"2024-03-06T07:26:38.260625+0000","last_became_peered":"2024-03-06T07:26:38.260625+0000","last_unstale":"2024-03-06T07:26:40.320818+0000","last_undegraded":"2024-03-06T07:26:40.320818+0000","last_fullsized":"2024-03-06T07:26:40.320818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:39:35.612496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271221+0000","last_change":"2024-03-06T07:26:39.256106+0000","last_active":"2024-03-06T07:26:40.271221+0000","last_peered":"2024-03-06T07:26:40.271221+0000","last_clean":"2024-03-06T07:26:40.271221+0000","last_became_active":"2024-03-06T07:26:39.255570+0000","last_became_peered":"2024-03-06T07:26:39.255570+0000","last_unstale":"2024-03-06T07:26:40.271221+0000","last_undegraded":"2024-03-06T07:26:40.271221+0000","last_fullsized":"2024-03-06T07:26:40.271221+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:02.271576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311356+0000","last_change":"2024-03-06T07:26:38.264961+0000","last_active":"2024-03-06T07:26:41.311356+0000","last_peered":"2024-03-06T07:26:41.311356+0000","last_clean":"2024-03-06T07:26:41.311356+0000","last_became_active":"2024-03-06T07:26:38.264360+0000","last_became_peered":"2024-03-06T07:26:38.264360+0000","last_unstale":"2024-03-06T07:26:41.311356+0000","last_undegraded":"2024-03-06T07:26:41.311356+0000","last_fullsized":"2024-03-06T07:26:41.311356+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:34.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810017+0000","last_change":"2024-03-06T07:26:39.265250+0000","last_active":"2024-03-06T07:26:40.810017+0000","last_peered":"2024-03-06T07:26:40.810017+0000","last_clean":"2024-03-06T07:26:40.810017+0000","last_became_active":"2024-03-06T07:26:39.264938+0000","last_became_peered":"2024-03-06T07:26:39.264938+0000","last_unstale":"2024-03-06T07:26:40.810017+0000","last_undegraded":"2024-03-06T07:26:40.810017+0000","last_fullsized":"2024-03-06T07:26:40.810017+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:01.314726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"171'3","reported_seq":17,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810178+0000","last_change":"2024-03-06T07:26:38.263099+0000","last_active":"2024-03-06T07:26:40.810178+0000","last_peered":"2024-03-06T07:26:40.810178+0000","last_clean":"2024-03-06T07:26:40.810178+0000","last_became_active":"2024-03-06T07:26:38.262764+0000","last_became_peered":"2024-03-06T07:26:38.262764+0000","last_unstale":"2024-03-06T07:26:40.810178+0000","last_undegraded":"2024-03-06T07:26:40.810178+0000","last_fullsized":"2024-03-06T07:26:40.810178+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:28.769962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320647+0000","last_change":"2024-03-06T07:26:39.263432+0000","last_active":"2024-03-06T07:26:40.320647+0000","last_peered":"2024-03-06T07:26:40.320647+0000","last_clean":"2024-03-06T07:26:40.320647+0000","last_became_active":"2024-03-06T07:26:39.263157+0000","last_became_peered":"2024-03-06T07:26:39.263157+0000","last_unstale":"2024-03-06T07:26:40.320647+0000","last_undegraded":"2024-03-06T07:26:40.320647+0000","last_fullsized":"2024-03-06T07:26:40.320647+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:31.213551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"171'6","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311032+0000","last_change":"2024-03-06T07:26:38.261487+0000","last_active":"2024-03-06T07:26:41.311032+0000","last_peered":"2024-03-06T07:26:41.311032+0000","last_clean":"2024-03-06T07:26:41.311032+0000","last_became_active":"2024-03-06T07:26:38.261216+0000","last_became_peered":"2024-03-06T07:26:38.261216+0000","last_unstale":"2024-03-06T07:26:41.311032+0000","last_undegraded":"2024-03-06T07:26:41.311032+0000","last_fullsized":"2024-03-06T07:26:41.311032+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:59.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249815+0000","last_change":"2024-03-06T07:26:39.255921+0000","last_active":"2024-03-06T07:26:40.249815+0000","last_peered":"2024-03-06T07:26:40.249815+0000","last_clean":"2024-03-06T07:26:40.249815+0000","last_became_active":"2024-03-06T07:26:39.255644+0000","last_became_peered":"2024-03-06T07:26:39.255644+0000","last_unstale":"2024-03-06T07:26:40.249815+0000","last_undegraded":"2024-03-06T07:26:40.249815+0000","last_fullsized":"2024-03-06T07:26:40.249815+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:45.948396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249765+0000","last_change":"2024-03-06T07:26:38.264768+0000","last_active":"2024-03-06T07:26:40.249765+0000","last_peered":"2024-03-06T07:26:40.249765+0000","last_clean":"2024-03-06T07:26:40.249765+0000","last_became_active":"2024-03-06T07:26:38.264507+0000","last_became_peered":"2024-03-06T07:26:38.264507+0000","last_unstale":"2024-03-06T07:26:40.249765+0000","last_undegraded":"2024-03-06T07:26:40.249765+0000","last_fullsized":"2024-03-06T07:26:40.249765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:38.785122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271593+0000","last_change":"2024-03-06T07:26:39.262631+0000","last_active":"2024-03-06T07:26:40.271593+0000","last_peered":"2024-03-06T07:26:40.271593+0000","last_clean":"2024-03-06T07:26:40.271593+0000","last_became_active":"2024-03-06T07:26:39.262118+0000","last_became_peered":"2024-03-06T07:26:39.262118+0000","last_unstale":"2024-03-06T07:26:40.271593+0000","last_undegraded":"2024-03-06T07:26:40.271593+0000","last_fullsized":"2024-03-06T07:26:40.271593+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:14.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271217+0000","last_change":"2024-03-06T07:26:38.254122+0000","last_active":"2024-03-06T07:26:40.271217+0000","last_peered":"2024-03-06T07:26:40.271217+0000","last_clean":"2024-03-06T07:26:40.271217+0000","last_became_active":"2024-03-06T07:26:38.253864+0000","last_became_peered":"2024-03-06T07:26:38.253864+0000","last_unstale":"2024-03-06T07:26:40.271217+0000","last_undegraded":"2024-03-06T07:26:40.271217+0000","last_fullsized":"2024-03-06T07:26:40.271217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:01.108773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810315+0000","last_change":"2024-03-06T07:26:39.271285+0000","last_active":"2024-03-06T07:26:40.810315+0000","last_peered":"2024-03-06T07:26:40.810315+0000","last_clean":"2024-03-06T07:26:40.810315+0000","last_became_active":"2024-03-06T07:26:39.270972+0000","last_became_peered":"2024-03-06T07:26:39.270972+0000","last_unstale":"2024-03-06T07:26:40.810315+0000","last_undegraded":"2024-03-06T07:26:40.810315+0000","last_fullsized":"2024-03-06T07:26:40.810315+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:43.077968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.809877+0000","last_change":"2024-03-06T07:26:38.264438+0000","last_active":"2024-03-06T07:26:40.809877+0000","last_peered":"2024-03-06T07:26:40.809877+0000","last_clean":"2024-03-06T07:26:40.809877+0000","last_became_active":"2024-03-06T07:26:38.264177+0000","last_became_peered":"2024-03-06T07:26:38.264177+0000","last_unstale":"2024-03-06T07:26:40.809877+0000","last_undegraded":"2024-03-06T07:26:40.809877+0000","last_fullsized":"2024-03-06T07:26:40.809877+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:04.813887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311247+0000","last_change":"2024-03-06T07:26:39.266877+0000","last_active":"2024-03-06T07:26:41.311247+0000","last_peered":"2024-03-06T07:26:41.311247+0000","last_clean":"2024-03-06T07:26:41.311247+0000","last_became_active":"2024-03-06T07:26:39.266605+0000","last_became_peered":"2024-03-06T07:26:39.266605+0000","last_unstale":"2024-03-06T07:26:41.311247+0000","last_undegraded":"2024-03-06T07:26:41.311247+0000","last_fullsized":"2024-03-06T07:26:41.311247+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:55.441174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"171'8","reported_seq":22,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320291+0000","last_change":"2024-03-06T07:26:38.258910+0000","last_active":"2024-03-06T07:26:40.320291+0000","last_peered":"2024-03-06T07:26:40.320291+0000","last_clean":"2024-03-06T07:26:40.320291+0000","last_became_active":"2024-03-06T07:26:38.258635+0000","last_became_peered":"2024-03-06T07:26:38.258635+0000","last_unstale":"2024-03-06T07:26:40.320291+0000","last_undegraded":"2024-03-06T07:26:40.320291+0000","last_fullsized":"2024-03-06T07:26:40.320291+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:36.832143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254809+0000","last_change":"2024-03-06T07:26:39.270147+0000","last_active":"2024-03-06T07:26:40.254809+0000","last_peered":"2024-03-06T07:26:40.254809+0000","last_clean":"2024-03-06T07:26:40.254809+0000","last_became_active":"2024-03-06T07:26:39.269481+0000","last_became_peered":"2024-03-06T07:26:39.269481+0000","last_unstale":"2024-03-06T07:26:40.254809+0000","last_undegraded":"2024-03-06T07:26:40.254809+0000","last_fullsized":"2024-03-06T07:26:40.254809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:54.509229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255696+0000","last_change":"2024-03-06T07:26:38.265919+0000","last_active":"2024-03-06T07:26:40.255696+0000","last_peered":"2024-03-06T07:26:40.255696+0000","last_clean":"2024-03-06T07:26:40.255696+0000","last_became_active":"2024-03-06T07:26:38.265582+0000","last_became_peered":"2024-03-06T07:26:38.265582+0000","last_unstale":"2024-03-06T07:26:40.255696+0000","last_undegraded":"2024-03-06T07:26:40.255696+0000","last_fullsized":"2024-03-06T07:26:40.255696+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:17.453635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255434+0000","last_change":"2024-03-06T07:26:39.260229+0000","last_active":"2024-03-06T07:26:40.255434+0000","last_peered":"2024-03-06T07:26:40.255434+0000","last_clean":"2024-03-06T07:26:40.255434+0000","last_became_active":"2024-03-06T07:26:39.259967+0000","last_became_peered":"2024-03-06T07:26:39.259967+0000","last_unstale":"2024-03-06T07:26:40.255434+0000","last_undegraded":"2024-03-06T07:26:40.255434+0000","last_fullsized":"2024-03-06T07:26:40.255434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:41.696337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255703+0000","last_change":"2024-03-06T07:26:38.267967+0000","last_active":"2024-03-06T07:26:40.255703+0000","last_peered":"2024-03-06T07:26:40.255703+0000","last_clean":"2024-03-06T07:26:40.255703+0000","last_became_active":"2024-03-06T07:26:38.267650+0000","last_became_peered":"2024-03-06T07:26:38.267650+0000","last_unstale":"2024-03-06T07:26:40.255703+0000","last_undegraded":"2024-03-06T07:26:40.255703+0000","last_fullsized":"2024-03-06T07:26:40.255703+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:25.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254942+0000","last_change":"2024-03-06T07:26:39.268823+0000","last_active":"2024-03-06T07:26:40.254942+0000","last_peered":"2024-03-06T07:26:40.254942+0000","last_clean":"2024-03-06T07:26:40.254942+0000","last_became_active":"2024-03-06T07:26:39.268430+0000","last_became_peered":"2024-03-06T07:26:39.268430+0000","last_unstale":"2024-03-06T07:26:40.254942+0000","last_undegraded":"2024-03-06T07:26:40.254942+0000","last_fullsized":"2024-03-06T07:26:40.254942+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:24.793981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249674+0000","last_change":"2024-03-06T07:26:38.260630+0000","last_active":"2024-03-06T07:26:40.249674+0000","last_peered":"2024-03-06T07:26:40.249674+0000","last_clean":"2024-03-06T07:26:40.249674+0000","last_became_active":"2024-03-06T07:26:38.260349+0000","last_became_peered":"2024-03-06T07:26:38.260349+0000","last_unstale":"2024-03-06T07:26:40.249674+0000","last_undegraded":"2024-03-06T07:26:40.249674+0000","last_fullsized":"2024-03-06T07:26:40.249674+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:54.003451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255672+0000","last_change":"2024-03-06T07:26:39.260197+0000","last_active":"2024-03-06T07:26:40.255672+0000","last_peered":"2024-03-06T07:26:40.255672+0000","last_clean":"2024-03-06T07:26:40.255672+0000","last_became_active":"2024-03-06T07:26:39.259915+0000","last_became_peered":"2024-03-06T07:26:39.259915+0000","last_unstale":"2024-03-06T07:26:40.255672+0000","last_undegraded":"2024-03-06T07:26:40.255672+0000","last_fullsized":"2024-03-06T07:26:40.255672+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:29.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320739+0000","last_change":"2024-03-06T07:26:38.261857+0000","last_active":"2024-03-06T07:26:40.320739+0000","last_peered":"2024-03-06T07:26:40.320739+0000","last_clean":"2024-03-06T07:26:40.320739+0000","last_became_active":"2024-03-06T07:26:38.261487+0000","last_became_peered":"2024-03-06T07:26:38.261487+0000","last_unstale":"2024-03-06T07:26:40.320739+0000","last_undegraded":"2024-03-06T07:26:40.320739+0000","last_fullsized":"2024-03-06T07:26:40.320739+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:01.272726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249841+0000","last_change":"2024-03-06T07:26:39.266307+0000","last_active":"2024-03-06T07:26:40.249841+0000","last_peered":"2024-03-06T07:26:40.249841+0000","last_clean":"2024-03-06T07:26:40.249841+0000","last_became_active":"2024-03-06T07:26:39.266035+0000","last_became_peered":"2024-03-06T07:26:39.266035+0000","last_unstale":"2024-03-06T07:26:40.249841+0000","last_undegraded":"2024-03-06T07:26:40.249841+0000","last_fullsized":"2024-03-06T07:26:40.249841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:43.243557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320629+0000","last_change":"2024-03-06T07:26:38.257000+0000","last_active":"2024-03-06T07:26:40.320629+0000","last_peered":"2024-03-06T07:26:40.320629+0000","last_clean":"2024-03-06T07:26:40.320629+0000","last_became_active":"2024-03-06T07:26:38.256724+0000","last_became_peered":"2024-03-06T07:26:38.256724+0000","last_unstale":"2024-03-06T07:26:40.320629+0000","last_undegraded":"2024-03-06T07:26:40.320629+0000","last_fullsized":"2024-03-06T07:26:40.320629+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:59.901171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311151+0000","last_change":"2024-03-06T07:26:39.272306+0000","last_active":"2024-03-06T07:26:41.311151+0000","last_peered":"2024-03-06T07:26:41.311151+0000","last_clean":"2024-03-06T07:26:41.311151+0000","last_became_active":"2024-03-06T07:26:39.272052+0000","last_became_peered":"2024-03-06T07:26:39.272052+0000","last_unstale":"2024-03-06T07:26:41.311151+0000","last_undegraded":"2024-03-06T07:26:41.311151+0000","last_fullsized":"2024-03-06T07:26:41.311151+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:40.408449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249621+0000","last_change":"2024-03-06T07:26:38.260016+0000","last_active":"2024-03-06T07:26:40.249621+0000","last_peered":"2024-03-06T07:26:40.249621+0000","last_clean":"2024-03-06T07:26:40.249621+0000","last_became_active":"2024-03-06T07:26:38.259723+0000","last_became_peered":"2024-03-06T07:26:38.259723+0000","last_unstale":"2024-03-06T07:26:40.249621+0000","last_undegraded":"2024-03-06T07:26:40.249621+0000","last_fullsized":"2024-03-06T07:26:40.249621+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:45.259421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810367+0000","last_change":"2024-03-06T07:26:39.263792+0000","last_active":"2024-03-06T07:26:40.810367+0000","last_peered":"2024-03-06T07:26:40.810367+0000","last_clean":"2024-03-06T07:26:40.810367+0000","last_became_active":"2024-03-06T07:26:39.263324+0000","last_became_peered":"2024-03-06T07:26:39.263324+0000","last_unstale":"2024-03-06T07:26:40.810367+0000","last_undegraded":"2024-03-06T07:26:40.810367+0000","last_fullsized":"2024-03-06T07:26:40.810367+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:21:41.110242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310763+0000","last_change":"2024-03-06T07:26:38.253562+0000","last_active":"2024-03-06T07:26:41.310763+0000","last_peered":"2024-03-06T07:26:41.310763+0000","last_clean":"2024-03-06T07:26:41.310763+0000","last_became_active":"2024-03-06T07:26:38.252639+0000","last_became_peered":"2024-03-06T07:26:38.252639+0000","last_unstale":"2024-03-06T07:26:41.310763+0000","last_undegraded":"2024-03-06T07:26:41.310763+0000","last_fullsized":"2024-03-06T07:26:41.310763+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:40.939285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256367+0000","last_change":"2024-03-06T07:26:39.255525+0000","last_active":"2024-03-06T07:26:40.256367+0000","last_peered":"2024-03-06T07:26:40.256367+0000","last_clean":"2024-03-06T07:26:40.256367+0000","last_became_active":"2024-03-06T07:26:39.255218+0000","last_became_peered":"2024-03-06T07:26:39.255218+0000","last_unstale":"2024-03-06T07:26:40.256367+0000","last_undegraded":"2024-03-06T07:26:40.256367+0000","last_fullsized":"2024-03-06T07:26:40.256367+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:03.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254783+0000","last_change":"2024-03-06T07:26:38.265988+0000","last_active":"2024-03-06T07:26:40.254783+0000","last_peered":"2024-03-06T07:26:40.254783+0000","last_clean":"2024-03-06T07:26:40.254783+0000","last_became_active":"2024-03-06T07:26:38.265493+0000","last_became_peered":"2024-03-06T07:26:38.265493+0000","last_unstale":"2024-03-06T07:26:40.254783+0000","last_undegraded":"2024-03-06T07:26:40.254783+0000","last_fullsized":"2024-03-06T07:26:40.254783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:46.694225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254957+0000","last_change":"2024-03-06T07:26:39.270436+0000","last_active":"2024-03-06T07:26:40.254957+0000","last_peered":"2024-03-06T07:26:40.254957+0000","last_clean":"2024-03-06T07:26:40.254957+0000","last_became_active":"2024-03-06T07:26:39.269858+0000","last_became_peered":"2024-03-06T07:26:39.269858+0000","last_unstale":"2024-03-06T07:26:40.254957+0000","last_undegraded":"2024-03-06T07:26:40.254957+0000","last_fullsized":"2024-03-06T07:26:40.254957+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:07:47.286326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810532+0000","last_change":"2024-03-06T07:26:38.260321+0000","last_active":"2024-03-06T07:26:40.810532+0000","last_peered":"2024-03-06T07:26:40.810532+0000","last_clean":"2024-03-06T07:26:40.810532+0000","last_became_active":"2024-03-06T07:26:38.260071+0000","last_became_peered":"2024-03-06T07:26:38.260071+0000","last_unstale":"2024-03-06T07:26:40.810532+0000","last_undegraded":"2024-03-06T07:26:40.810532+0000","last_fullsized":"2024-03-06T07:26:40.810532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:48.862475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271312+0000","last_change":"2024-03-06T07:26:39.255863+0000","last_active":"2024-03-06T07:26:40.271312+0000","last_peered":"2024-03-06T07:26:40.271312+0000","last_clean":"2024-03-06T07:26:40.271312+0000","last_became_active":"2024-03-06T07:26:39.255180+0000","last_became_peered":"2024-03-06T07:26:39.255180+0000","last_unstale":"2024-03-06T07:26:40.271312+0000","last_undegraded":"2024-03-06T07:26:40.271312+0000","last_fullsized":"2024-03-06T07:26:40.271312+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:23.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"172'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256077+0000","last_change":"2024-03-06T07:26:38.256677+0000","last_active":"2024-03-06T07:26:40.256077+0000","last_peered":"2024-03-06T07:26:40.256077+0000","last_clean":"2024-03-06T07:26:40.256077+0000","last_became_active":"2024-03-06T07:26:38.256360+0000","last_became_peered":"2024-03-06T07:26:38.256360+0000","last_unstale":"2024-03-06T07:26:40.256077+0000","last_undegraded":"2024-03-06T07:26:40.256077+0000","last_fullsized":"2024-03-06T07:26:40.256077+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:36.319241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810018+0000","last_change":"2024-03-06T07:26:39.259869+0000","last_active":"2024-03-06T07:26:40.810018+0000","last_peered":"2024-03-06T07:26:40.810018+0000","last_clean":"2024-03-06T07:26:40.810018+0000","last_became_active":"2024-03-06T07:26:39.259073+0000","last_became_peered":"2024-03-06T07:26:39.259073+0000","last_unstale":"2024-03-06T07:26:40.810018+0000","last_undegraded":"2024-03-06T07:26:40.810018+0000","last_fullsized":"2024-03-06T07:26:40.810018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:14.831639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310939+0000","last_change":"2024-03-06T07:26:38.261044+0000","last_active":"2024-03-06T07:26:41.310939+0000","last_peered":"2024-03-06T07:26:41.310939+0000","last_clean":"2024-03-06T07:26:41.310939+0000","last_became_active":"2024-03-06T07:26:38.260783+0000","last_became_peered":"2024-03-06T07:26:38.260783+0000","last_unstale":"2024-03-06T07:26:41.310939+0000","last_undegraded":"2024-03-06T07:26:41.310939+0000","last_fullsized":"2024-03-06T07:26:41.310939+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:15.768174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255088+0000","last_change":"2024-03-06T07:26:39.263856+0000","last_active":"2024-03-06T07:26:40.255088+0000","last_peered":"2024-03-06T07:26:40.255088+0000","last_clean":"2024-03-06T07:26:40.255088+0000","last_became_active":"2024-03-06T07:26:39.263117+0000","last_became_peered":"2024-03-06T07:26:39.263117+0000","last_unstale":"2024-03-06T07:26:40.255088+0000","last_undegraded":"2024-03-06T07:26:40.255088+0000","last_fullsized":"2024-03-06T07:26:40.255088+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:11:38.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255267+0000","last_change":"2024-03-06T07:26:38.266025+0000","last_active":"2024-03-06T07:26:40.255267+0000","last_peered":"2024-03-06T07:26:40.255267+0000","last_clean":"2024-03-06T07:26:40.255267+0000","last_became_active":"2024-03-06T07:26:38.265692+0000","last_became_peered":"2024-03-06T07:26:38.265692+0000","last_unstale":"2024-03-06T07:26:40.255267+0000","last_undegraded":"2024-03-06T07:26:40.255267+0000","last_fullsized":"2024-03-06T07:26:40.255267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:43.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255226+0000","last_change":"2024-03-06T07:26:39.256045+0000","last_active":"2024-03-06T07:26:40.255226+0000","last_peered":"2024-03-06T07:26:40.255226+0000","last_clean":"2024-03-06T07:26:40.255226+0000","last_became_active":"2024-03-06T07:26:39.255761+0000","last_became_peered":"2024-03-06T07:26:39.255761+0000","last_unstale":"2024-03-06T07:26:40.255226+0000","last_undegraded":"2024-03-06T07:26:40.255226+0000","last_fullsized":"2024-03-06T07:26:40.255226+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:54.107202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255717+0000","last_change":"2024-03-06T07:26:38.255242+0000","last_active":"2024-03-06T07:26:40.255717+0000","last_peered":"2024-03-06T07:26:40.255717+0000","last_clean":"2024-03-06T07:26:40.255717+0000","last_became_active":"2024-03-06T07:26:38.254954+0000","last_became_peered":"2024-03-06T07:26:38.254954+0000","last_unstale":"2024-03-06T07:26:40.255717+0000","last_undegraded":"2024-03-06T07:26:40.255717+0000","last_fullsized":"2024-03-06T07:26:40.255717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:19.334482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311234+0000","last_change":"2024-03-06T07:26:39.271443+0000","last_active":"2024-03-06T07:26:41.311234+0000","last_peered":"2024-03-06T07:26:41.311234+0000","last_clean":"2024-03-06T07:26:41.311234+0000","last_became_active":"2024-03-06T07:26:39.270882+0000","last_became_peered":"2024-03-06T07:26:39.270882+0000","last_unstale":"2024-03-06T07:26:41.311234+0000","last_undegraded":"2024-03-06T07:26:41.311234+0000","last_fullsized":"2024-03-06T07:26:41.311234+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:07.841172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810076+0000","last_change":"2024-03-06T07:26:38.266006+0000","last_active":"2024-03-06T07:26:40.810076+0000","last_peered":"2024-03-06T07:26:40.810076+0000","last_clean":"2024-03-06T07:26:40.810076+0000","last_became_active":"2024-03-06T07:26:38.265728+0000","last_became_peered":"2024-03-06T07:26:38.265728+0000","last_unstale":"2024-03-06T07:26:40.810076+0000","last_undegraded":"2024-03-06T07:26:40.810076+0000","last_fullsized":"2024-03-06T07:26:40.810076+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:54.435126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311371+0000","last_change":"2024-03-06T07:26:39.268286+0000","last_active":"2024-03-06T07:26:41.311371+0000","last_peered":"2024-03-06T07:26:41.311371+0000","last_clean":"2024-03-06T07:26:41.311371+0000","last_became_active":"2024-03-06T07:26:39.268027+0000","last_became_peered":"2024-03-06T07:26:39.268027+0000","last_unstale":"2024-03-06T07:26:41.311371+0000","last_undegraded":"2024-03-06T07:26:41.311371+0000","last_fullsized":"2024-03-06T07:26:41.311371+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:49.868523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255501+0000","last_change":"2024-03-06T07:26:38.262673+0000","last_active":"2024-03-06T07:26:40.255501+0000","last_peered":"2024-03-06T07:26:40.255501+0000","last_clean":"2024-03-06T07:26:40.255501+0000","last_became_active":"2024-03-06T07:26:38.260157+0000","last_became_peered":"2024-03-06T07:26:38.260157+0000","last_unstale":"2024-03-06T07:26:40.255501+0000","last_undegraded":"2024-03-06T07:26:40.255501+0000","last_fullsized":"2024-03-06T07:26:40.255501+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:45.779315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271488+0000","last_change":"2024-03-06T07:26:39.255450+0000","last_active":"2024-03-06T07:26:40.271488+0000","last_peered":"2024-03-06T07:26:40.271488+0000","last_clean":"2024-03-06T07:26:40.271488+0000","last_became_active":"2024-03-06T07:26:39.255136+0000","last_became_peered":"2024-03-06T07:26:39.255136+0000","last_unstale":"2024-03-06T07:26:40.271488+0000","last_undegraded":"2024-03-06T07:26:40.271488+0000","last_fullsized":"2024-03-06T07:26:40.271488+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:37.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320409+0000","last_change":"2024-03-06T07:26:38.255185+0000","last_active":"2024-03-06T07:26:40.320409+0000","last_peered":"2024-03-06T07:26:40.320409+0000","last_clean":"2024-03-06T07:26:40.320409+0000","last_became_active":"2024-03-06T07:26:38.254907+0000","last_became_peered":"2024-03-06T07:26:38.254907+0000","last_unstale":"2024-03-06T07:26:40.320409+0000","last_undegraded":"2024-03-06T07:26:40.320409+0000","last_fullsized":"2024-03-06T07:26:40.320409+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:39:05.814671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255455+0000","last_change":"2024-03-06T07:26:39.264397+0000","last_active":"2024-03-06T07:26:40.255455+0000","last_peered":"2024-03-06T07:26:40.255455+0000","last_clean":"2024-03-06T07:26:40.255455+0000","last_became_active":"2024-03-06T07:26:39.264081+0000","last_became_peered":"2024-03-06T07:26:39.264081+0000","last_unstale":"2024-03-06T07:26:40.255455+0000","last_undegraded":"2024-03-06T07:26:40.255455+0000","last_fullsized":"2024-03-06T07:26:40.255455+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:22:20.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810075+0000","last_change":"2024-03-06T07:26:38.262369+0000","last_active":"2024-03-06T07:26:40.810075+0000","last_peered":"2024-03-06T07:26:40.810075+0000","last_clean":"2024-03-06T07:26:40.810075+0000","last_became_active":"2024-03-06T07:26:38.262081+0000","last_became_peered":"2024-03-06T07:26:38.262081+0000","last_unstale":"2024-03-06T07:26:40.810075+0000","last_undegraded":"2024-03-06T07:26:40.810075+0000","last_fullsized":"2024-03-06T07:26:40.810075+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:06.835124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310554+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:26:41.310554+0000","last_peered":"2024-03-06T07:26:41.310554+0000","last_clean":"2024-03-06T07:26:41.310554+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:26:41.310554+0000","last_undegraded":"2024-03-06T07:26:41.310554+0000","last_fullsized":"2024-03-06T07:26:41.310554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2256896,"data_stored":1656727,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2191360,"data_stored":1066871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2183168,"data_stored":1064733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1065259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2187264,"data_stored":1064803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1064767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1064677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37536,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334304,"statfs":{"total":96636764160,"available":96598327296,"internally_reserved":0,"allocated":2244608,"data_stored":1655135,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:43.473 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:26:43.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.608+0000 7f8c8f11e640 1 Processor -- start 2024-03-06T07:26:43.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.609+0000 7f8c8f11e640 1 -- start start 2024-03-06T07:26:43.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8f11e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c8814f790 0x7f8c881529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8f11e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c88155a40 0x7f8c88155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8f11e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c881563d0 con 0x7f8c880c1aa0 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8f11e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880c2680 con 0x7f8c8814f790 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8f11e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c28d0 con 0x7f8c88155a40 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c8814f790 0x7f8c881529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c8814f790 0x7f8c881529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56776/0 (socket says 172.21.15.17:56776) 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 -- 172.21.15.17:0/257899185 learned_addr learned my addr 172.21.15.17:0/257899185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8d6bb640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c88155a40 0x7f8c88155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 -- 172.21.15.17:0/257899185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c88155a40 msgr2=0x7f8c88155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c88155a40 0x7f8c88155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 -- 172.21.15.17:0/257899185 >> v1:172.21.15.163:6789/0 conn(0x7f8c880c1aa0 legacy=0x7f8c880c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.610+0000 7f8c8ceba640 1 -- 172.21.15.17:0/257899185 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880c2bb0 con 0x7f8c8814f790 2024-03-06T07:26:43.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.611+0000 7f8c8ceba640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c8814f790 0x7f8c881529d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009870 tx=0x7f8c7802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe273a8e0a51af87 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.611+0000 7f8c877fe640 1 -- 172.21.15.17:0/257899185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78035020 con 0x7f8c8814f790 2024-03-06T07:26:43.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.611+0000 7f8c877fe640 1 -- 172.21.15.17:0/257899185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78031640 con 0x7f8c8814f790 2024-03-06T07:26:43.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.611+0000 7f8c877fe640 1 -- 172.21.15.17:0/257899185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7802d040 con 0x7f8c8814f790 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 -- 172.21.15.17:0/257899185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c8814f790 msgr2=0x7f8c881529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c8814f790 0x7f8c881529d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009870 tx=0x7f8c7802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 -- 172.21.15.17:0/257899185 shutdown_connections 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c88155a40 0x7f8c88155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/257899185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c8814f790 0x7f8c881529d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 -- 172.21.15.17:0/257899185 >> 172.21.15.17:0/257899185 conn(0x7f8c880c6710 msgr2=0x7f8c880c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 -- 172.21.15.17:0/257899185 shutdown_connections 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.612+0000 7f8c8f11e640 1 -- 172.21.15.17:0/257899185 wait complete. 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.613+0000 7f8c8f11e640 1 Processor -- start 2024-03-06T07:26:43.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.613+0000 7f8c8f11e640 1 -- start start 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c880c1aa0 0x7f8c88149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8814f790 0x7f8c8814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c0d60 con 0x7f8c88155a40 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88155520 con 0x7f8c8814f790 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.614+0000 7f8c8f11e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c5e30 con 0x7f8c880c1aa0 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56782/0 (socket says 172.21.15.17:56782) 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 -- 172.21.15.17:0/2667452905 learned_addr learned my addr 172.21.15.17:0/2667452905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8ceba640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c880c1aa0 0x7f8c88149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c87fff640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8814f790 0x7f8c8814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 -- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c880c1aa0 msgr2=0x7f8c88149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c880c1aa0 0x7f8c88149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 -- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8814f790 msgr2=0x7f8c8814a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8814f790 0x7f8c8814a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 -- 172.21.15.17:0/2667452905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88070ea0 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8d6bb640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000cc30 tx=0x7f8c8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000ce40 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80003990 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80003d40 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c88071180 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.615+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8814d300 con 0x7f8c88155a40 2024-03-06T07:26:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.616+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8804f330 con 0x7f8c88155a40 2024-03-06T07:26:43.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.617+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c80003ee0 con 0x7f8c88155a40 2024-03-06T07:26:43.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.617+0000 7f8c857fa640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 0x7f8c6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.617+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c8008a7e0 con 0x7f8c88155a40 2024-03-06T07:26:43.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.618+0000 7f8c8ceba640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 0x7f8c6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.618+0000 7f8c8ceba640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 0x7f8c6806d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8c78036000 tx=0x7f8c7803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.622+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c80014030 con 0x7f8c88155a40 2024-03-06T07:26:43.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.797+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c8814d9e0 con 0x7f8c6806b500 2024-03-06T07:26:43.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.803+0000 7f8c857fa640 1 -- 172.21.15.17:0/2667452905 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252569 (secure 0 0 0) 0x7f8c8814d9e0 con 0x7f8c6806b500 2024-03-06T07:26:43.807 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:43.808 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 msgr2=0x7f8c6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 0x7f8c6806d9c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8c78036000 tx=0x7f8c7803c040 comp rx=0 tx=0).stop 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 msgr2=0x7f8c881ec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000cc30 tx=0x7f8c8000a5c0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 shutdown_connections 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c6806b500 0x7f8c6806d9c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c88155a40 0x7f8c881ec2a0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c8814f790 0x7f8c8814a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 --2- 172.21.15.17:0/2667452905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c880c1aa0 0x7f8c88149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 >> 172.21.15.17:0/2667452905 conn(0x7f8c880c6710 msgr2=0x7f8c8806db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 shutdown_connections 2024-03-06T07:26:43.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.806+0000 7f8c8f11e640 1 -- 172.21.15.17:0/2667452905 wait complete. 2024-03-06T07:26:43.818 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-03-06T07:26:43.429917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270808,"kb_used_data":17184,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754703912,"statfs":{"total":773094113280,"available":772816805888,"internally_reserved":0,"allocated":17596416,"data_stored":9702972,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001144"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255796+0000","last_change":"2024-03-06T07:26:39.265235+0000","last_active":"2024-03-06T07:26:40.255796+0000","last_peered":"2024-03-06T07:26:40.255796+0000","last_clean":"2024-03-06T07:26:40.255796+0000","last_became_active":"2024-03-06T07:26:39.264159+0000","last_became_peered":"2024-03-06T07:26:39.264159+0000","last_unstale":"2024-03-06T07:26:40.255796+0000","last_undegraded":"2024-03-06T07:26:40.255796+0000","last_fullsized":"2024-03-06T07:26:40.255796+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:13:36.272472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249827+0000","last_change":"2024-03-06T07:26:39.267139+0000","last_active":"2024-03-06T07:26:40.249827+0000","last_peered":"2024-03-06T07:26:40.249827+0000","last_clean":"2024-03-06T07:26:40.249827+0000","last_became_active":"2024-03-06T07:26:39.266830+0000","last_became_peered":"2024-03-06T07:26:39.266830+0000","last_unstale":"2024-03-06T07:26:40.249827+0000","last_undegraded":"2024-03-06T07:26:40.249827+0000","last_fullsized":"2024-03-06T07:26:40.249827+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:27:21.412864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271395+0000","last_change":"2024-03-06T07:26:39.265218+0000","last_active":"2024-03-06T07:26:40.271395+0000","last_peered":"2024-03-06T07:26:40.271395+0000","last_clean":"2024-03-06T07:26:40.271395+0000","last_became_active":"2024-03-06T07:26:39.264926+0000","last_became_peered":"2024-03-06T07:26:39.264926+0000","last_unstale":"2024-03-06T07:26:40.271395+0000","last_undegraded":"2024-03-06T07:26:40.271395+0000","last_fullsized":"2024-03-06T07:26:40.271395+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:31:33.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310980+0000","last_change":"2024-03-06T07:26:39.262329+0000","last_active":"2024-03-06T07:26:41.310980+0000","last_peered":"2024-03-06T07:26:41.310980+0000","last_clean":"2024-03-06T07:26:41.310980+0000","last_became_active":"2024-03-06T07:26:39.262051+0000","last_became_peered":"2024-03-06T07:26:39.262051+0000","last_unstale":"2024-03-06T07:26:41.310980+0000","last_undegraded":"2024-03-06T07:26:41.310980+0000","last_fullsized":"2024-03-06T07:26:41.310980+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:05.819935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320499+0000","last_change":"2024-03-06T07:26:39.270056+0000","last_active":"2024-03-06T07:26:40.320499+0000","last_peered":"2024-03-06T07:26:40.320499+0000","last_clean":"2024-03-06T07:26:40.320499+0000","last_became_active":"2024-03-06T07:26:39.269621+0000","last_became_peered":"2024-03-06T07:26:39.269621+0000","last_unstale":"2024-03-06T07:26:40.320499+0000","last_undegraded":"2024-03-06T07:26:40.320499+0000","last_fullsized":"2024-03-06T07:26:40.320499+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:37:46.358864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310775+0000","last_change":"2024-03-06T07:26:39.266307+0000","last_active":"2024-03-06T07:26:41.310775+0000","last_peered":"2024-03-06T07:26:41.310775+0000","last_clean":"2024-03-06T07:26:41.310775+0000","last_became_active":"2024-03-06T07:26:39.265649+0000","last_became_peered":"2024-03-06T07:26:39.265649+0000","last_unstale":"2024-03-06T07:26:41.310775+0000","last_undegraded":"2024-03-06T07:26:41.310775+0000","last_fullsized":"2024-03-06T07:26:41.310775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:42:30.673499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320441+0000","last_change":"2024-03-06T07:26:39.267985+0000","last_active":"2024-03-06T07:26:40.320441+0000","last_peered":"2024-03-06T07:26:40.320441+0000","last_clean":"2024-03-06T07:26:40.320441+0000","last_became_active":"2024-03-06T07:26:39.267673+0000","last_became_peered":"2024-03-06T07:26:39.267673+0000","last_unstale":"2024-03-06T07:26:40.320441+0000","last_undegraded":"2024-03-06T07:26:40.320441+0000","last_fullsized":"2024-03-06T07:26:40.320441+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:20.340530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254550+0000","last_change":"2024-03-06T07:26:39.257459+0000","last_active":"2024-03-06T07:26:40.254550+0000","last_peered":"2024-03-06T07:26:40.254550+0000","last_clean":"2024-03-06T07:26:40.254550+0000","last_became_active":"2024-03-06T07:26:39.257193+0000","last_became_peered":"2024-03-06T07:26:39.257193+0000","last_unstale":"2024-03-06T07:26:40.254550+0000","last_undegraded":"2024-03-06T07:26:40.254550+0000","last_fullsized":"2024-03-06T07:26:40.254550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:41:06.900442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810346+0000","last_change":"2024-03-06T07:26:39.272838+0000","last_active":"2024-03-06T07:26:40.810346+0000","last_peered":"2024-03-06T07:26:40.810346+0000","last_clean":"2024-03-06T07:26:40.810346+0000","last_became_active":"2024-03-06T07:26:39.272564+0000","last_became_peered":"2024-03-06T07:26:39.272564+0000","last_unstale":"2024-03-06T07:26:40.810346+0000","last_undegraded":"2024-03-06T07:26:40.810346+0000","last_fullsized":"2024-03-06T07:26:40.810346+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:36.142892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255978+0000","last_change":"2024-03-06T07:26:39.269676+0000","last_active":"2024-03-06T07:26:40.255978+0000","last_peered":"2024-03-06T07:26:40.255978+0000","last_clean":"2024-03-06T07:26:40.255978+0000","last_became_active":"2024-03-06T07:26:39.269377+0000","last_became_peered":"2024-03-06T07:26:39.269377+0000","last_unstale":"2024-03-06T07:26:40.255978+0000","last_undegraded":"2024-03-06T07:26:40.255978+0000","last_fullsized":"2024-03-06T07:26:40.255978+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:41.945333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271443+0000","last_change":"2024-03-06T07:26:39.262387+0000","last_active":"2024-03-06T07:26:40.271443+0000","last_peered":"2024-03-06T07:26:40.271443+0000","last_clean":"2024-03-06T07:26:40.271443+0000","last_became_active":"2024-03-06T07:26:39.261697+0000","last_became_peered":"2024-03-06T07:26:39.261697+0000","last_unstale":"2024-03-06T07:26:40.271443+0000","last_undegraded":"2024-03-06T07:26:40.271443+0000","last_fullsized":"2024-03-06T07:26:40.271443+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:43.510306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320686+0000","last_change":"2024-03-06T07:26:39.265828+0000","last_active":"2024-03-06T07:26:40.320686+0000","last_peered":"2024-03-06T07:26:40.320686+0000","last_clean":"2024-03-06T07:26:40.320686+0000","last_became_active":"2024-03-06T07:26:39.265524+0000","last_became_peered":"2024-03-06T07:26:39.265524+0000","last_unstale":"2024-03-06T07:26:40.320686+0000","last_undegraded":"2024-03-06T07:26:40.320686+0000","last_fullsized":"2024-03-06T07:26:40.320686+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:21:03.076037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311106+0000","last_change":"2024-03-06T07:26:39.268793+0000","last_active":"2024-03-06T07:26:41.311106+0000","last_peered":"2024-03-06T07:26:41.311106+0000","last_clean":"2024-03-06T07:26:41.311106+0000","last_became_active":"2024-03-06T07:26:39.268258+0000","last_became_peered":"2024-03-06T07:26:39.268258+0000","last_unstale":"2024-03-06T07:26:41.311106+0000","last_undegraded":"2024-03-06T07:26:41.311106+0000","last_fullsized":"2024-03-06T07:26:41.311106+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:29.776010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255739+0000","last_change":"2024-03-06T07:26:39.254939+0000","last_active":"2024-03-06T07:26:40.255739+0000","last_peered":"2024-03-06T07:26:40.255739+0000","last_clean":"2024-03-06T07:26:40.255739+0000","last_became_active":"2024-03-06T07:26:39.254669+0000","last_became_peered":"2024-03-06T07:26:39.254669+0000","last_unstale":"2024-03-06T07:26:40.255739+0000","last_undegraded":"2024-03-06T07:26:40.255739+0000","last_fullsized":"2024-03-06T07:26:40.255739+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:16.774222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310644+0000","last_change":"2024-03-06T07:26:39.264952+0000","last_active":"2024-03-06T07:26:41.310644+0000","last_peered":"2024-03-06T07:26:41.310644+0000","last_clean":"2024-03-06T07:26:41.310644+0000","last_became_active":"2024-03-06T07:26:39.264676+0000","last_became_peered":"2024-03-06T07:26:39.264676+0000","last_unstale":"2024-03-06T07:26:41.310644+0000","last_undegraded":"2024-03-06T07:26:41.310644+0000","last_fullsized":"2024-03-06T07:26:41.310644+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:18.459682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271200+0000","last_change":"2024-03-06T07:26:39.252696+0000","last_active":"2024-03-06T07:26:40.271200+0000","last_peered":"2024-03-06T07:26:40.271200+0000","last_clean":"2024-03-06T07:26:40.271200+0000","last_became_active":"2024-03-06T07:26:39.252405+0000","last_became_peered":"2024-03-06T07:26:39.252405+0000","last_unstale":"2024-03-06T07:26:40.271200+0000","last_undegraded":"2024-03-06T07:26:40.271200+0000","last_fullsized":"2024-03-06T07:26:40.271200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:33:58.413688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810359+0000","last_change":"2024-03-06T07:26:39.265006+0000","last_active":"2024-03-06T07:26:40.810359+0000","last_peered":"2024-03-06T07:26:40.810359+0000","last_clean":"2024-03-06T07:26:40.810359+0000","last_became_active":"2024-03-06T07:26:39.264684+0000","last_became_peered":"2024-03-06T07:26:39.264684+0000","last_unstale":"2024-03-06T07:26:40.810359+0000","last_undegraded":"2024-03-06T07:26:40.810359+0000","last_fullsized":"2024-03-06T07:26:40.810359+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:57:50.404141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810273+0000","last_change":"2024-03-06T07:26:39.271941+0000","last_active":"2024-03-06T07:26:40.810273+0000","last_peered":"2024-03-06T07:26:40.810273+0000","last_clean":"2024-03-06T07:26:40.810273+0000","last_became_active":"2024-03-06T07:26:39.271608+0000","last_became_peered":"2024-03-06T07:26:39.271608+0000","last_unstale":"2024-03-06T07:26:40.810273+0000","last_undegraded":"2024-03-06T07:26:40.810273+0000","last_fullsized":"2024-03-06T07:26:40.810273+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:53:18.287778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256266+0000","last_change":"2024-03-06T07:26:39.263669+0000","last_active":"2024-03-06T07:26:40.256266+0000","last_peered":"2024-03-06T07:26:40.256266+0000","last_clean":"2024-03-06T07:26:40.256266+0000","last_became_active":"2024-03-06T07:26:39.263176+0000","last_became_peered":"2024-03-06T07:26:39.263176+0000","last_unstale":"2024-03-06T07:26:40.256266+0000","last_undegraded":"2024-03-06T07:26:40.256266+0000","last_fullsized":"2024-03-06T07:26:40.256266+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:40.261455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255090+0000","last_change":"2024-03-06T07:26:39.269155+0000","last_active":"2024-03-06T07:26:40.255090+0000","last_peered":"2024-03-06T07:26:40.255090+0000","last_clean":"2024-03-06T07:26:40.255090+0000","last_became_active":"2024-03-06T07:26:39.268816+0000","last_became_peered":"2024-03-06T07:26:39.268816+0000","last_unstale":"2024-03-06T07:26:40.255090+0000","last_undegraded":"2024-03-06T07:26:40.255090+0000","last_fullsized":"2024-03-06T07:26:40.255090+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:05.278788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255540+0000","last_change":"2024-03-06T07:26:39.270600+0000","last_active":"2024-03-06T07:26:40.255540+0000","last_peered":"2024-03-06T07:26:40.255540+0000","last_clean":"2024-03-06T07:26:40.255540+0000","last_became_active":"2024-03-06T07:26:39.269961+0000","last_became_peered":"2024-03-06T07:26:39.269961+0000","last_unstale":"2024-03-06T07:26:40.255540+0000","last_undegraded":"2024-03-06T07:26:40.255540+0000","last_fullsized":"2024-03-06T07:26:40.255540+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:15.866846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256165+0000","last_change":"2024-03-06T07:26:39.254466+0000","last_active":"2024-03-06T07:26:40.256165+0000","last_peered":"2024-03-06T07:26:40.256165+0000","last_clean":"2024-03-06T07:26:40.256165+0000","last_became_active":"2024-03-06T07:26:39.254144+0000","last_became_peered":"2024-03-06T07:26:39.254144+0000","last_unstale":"2024-03-06T07:26:40.256165+0000","last_undegraded":"2024-03-06T07:26:40.256165+0000","last_fullsized":"2024-03-06T07:26:40.256165+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:39:47.975168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249749+0000","last_change":"2024-03-06T07:26:39.257785+0000","last_active":"2024-03-06T07:26:40.249749+0000","last_peered":"2024-03-06T07:26:40.249749+0000","last_clean":"2024-03-06T07:26:40.249749+0000","last_became_active":"2024-03-06T07:26:39.257539+0000","last_became_peered":"2024-03-06T07:26:39.257539+0000","last_unstale":"2024-03-06T07:26:40.249749+0000","last_undegraded":"2024-03-06T07:26:40.249749+0000","last_fullsized":"2024-03-06T07:26:40.249749+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:01.239394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320336+0000","last_change":"2024-03-06T07:26:39.254092+0000","last_active":"2024-03-06T07:26:40.320336+0000","last_peered":"2024-03-06T07:26:40.320336+0000","last_clean":"2024-03-06T07:26:40.320336+0000","last_became_active":"2024-03-06T07:26:39.253775+0000","last_became_peered":"2024-03-06T07:26:39.253775+0000","last_unstale":"2024-03-06T07:26:40.320336+0000","last_undegraded":"2024-03-06T07:26:40.320336+0000","last_fullsized":"2024-03-06T07:26:40.320336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:37.325289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810488+0000","last_change":"2024-03-06T07:26:39.272892+0000","last_active":"2024-03-06T07:26:40.810488+0000","last_peered":"2024-03-06T07:26:40.810488+0000","last_clean":"2024-03-06T07:26:40.810488+0000","last_became_active":"2024-03-06T07:26:39.272632+0000","last_became_peered":"2024-03-06T07:26:39.272632+0000","last_unstale":"2024-03-06T07:26:40.810488+0000","last_undegraded":"2024-03-06T07:26:40.810488+0000","last_fullsized":"2024-03-06T07:26:40.810488+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:22.390867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255697+0000","last_change":"2024-03-06T07:26:39.265364+0000","last_active":"2024-03-06T07:26:40.255697+0000","last_peered":"2024-03-06T07:26:40.255697+0000","last_clean":"2024-03-06T07:26:40.255697+0000","last_became_active":"2024-03-06T07:26:39.264432+0000","last_became_peered":"2024-03-06T07:26:39.264432+0000","last_unstale":"2024-03-06T07:26:40.255697+0000","last_undegraded":"2024-03-06T07:26:40.255697+0000","last_fullsized":"2024-03-06T07:26:40.255697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:10.356581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271526+0000","last_change":"2024-03-06T07:26:39.263268+0000","last_active":"2024-03-06T07:26:40.271526+0000","last_peered":"2024-03-06T07:26:40.271526+0000","last_clean":"2024-03-06T07:26:40.271526+0000","last_became_active":"2024-03-06T07:26:39.262966+0000","last_became_peered":"2024-03-06T07:26:39.262966+0000","last_unstale":"2024-03-06T07:26:40.271526+0000","last_undegraded":"2024-03-06T07:26:40.271526+0000","last_fullsized":"2024-03-06T07:26:40.271526+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:24:17.414856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320737+0000","last_change":"2024-03-06T07:26:39.262772+0000","last_active":"2024-03-06T07:26:40.320737+0000","last_peered":"2024-03-06T07:26:40.320737+0000","last_clean":"2024-03-06T07:26:40.320737+0000","last_became_active":"2024-03-06T07:26:39.262388+0000","last_became_peered":"2024-03-06T07:26:39.262388+0000","last_unstale":"2024-03-06T07:26:40.320737+0000","last_undegraded":"2024-03-06T07:26:40.320737+0000","last_fullsized":"2024-03-06T07:26:40.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:47.316796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810051+0000","last_change":"2024-03-06T07:26:39.270056+0000","last_active":"2024-03-06T07:26:40.810051+0000","last_peered":"2024-03-06T07:26:40.810051+0000","last_clean":"2024-03-06T07:26:40.810051+0000","last_became_active":"2024-03-06T07:26:39.269740+0000","last_became_peered":"2024-03-06T07:26:39.269740+0000","last_unstale":"2024-03-06T07:26:40.810051+0000","last_undegraded":"2024-03-06T07:26:40.810051+0000","last_fullsized":"2024-03-06T07:26:40.810051+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:37:58.941044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310932+0000","last_change":"2024-03-06T07:26:39.271213+0000","last_active":"2024-03-06T07:26:41.310932+0000","last_peered":"2024-03-06T07:26:41.310932+0000","last_clean":"2024-03-06T07:26:41.310932+0000","last_became_active":"2024-03-06T07:26:39.270899+0000","last_became_peered":"2024-03-06T07:26:39.270899+0000","last_unstale":"2024-03-06T07:26:41.310932+0000","last_undegraded":"2024-03-06T07:26:41.310932+0000","last_fullsized":"2024-03-06T07:26:41.310932+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:46.785363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310749+0000","last_change":"2024-03-06T07:26:39.269155+0000","last_active":"2024-03-06T07:26:41.310749+0000","last_peered":"2024-03-06T07:26:41.310749+0000","last_clean":"2024-03-06T07:26:41.310749+0000","last_became_active":"2024-03-06T07:26:39.268877+0000","last_became_peered":"2024-03-06T07:26:39.268877+0000","last_unstale":"2024-03-06T07:26:41.310749+0000","last_undegraded":"2024-03-06T07:26:41.310749+0000","last_fullsized":"2024-03-06T07:26:41.310749+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:28.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310577+0000","last_change":"2024-03-06T07:26:39.265203+0000","last_active":"2024-03-06T07:26:41.310577+0000","last_peered":"2024-03-06T07:26:41.310577+0000","last_clean":"2024-03-06T07:26:41.310577+0000","last_became_active":"2024-03-06T07:26:39.264957+0000","last_became_peered":"2024-03-06T07:26:39.264957+0000","last_unstale":"2024-03-06T07:26:41.310577+0000","last_undegraded":"2024-03-06T07:26:41.310577+0000","last_fullsized":"2024-03-06T07:26:41.310577+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:02.114821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249642+0000","last_change":"2024-03-06T07:26:39.255177+0000","last_active":"2024-03-06T07:26:40.249642+0000","last_peered":"2024-03-06T07:26:40.249642+0000","last_clean":"2024-03-06T07:26:40.249642+0000","last_became_active":"2024-03-06T07:26:39.254844+0000","last_became_peered":"2024-03-06T07:26:39.254844+0000","last_unstale":"2024-03-06T07:26:40.249642+0000","last_undegraded":"2024-03-06T07:26:40.249642+0000","last_fullsized":"2024-03-06T07:26:40.249642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:47:19.212179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"171'4","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311074+0000","last_change":"2024-03-06T07:26:38.252288+0000","last_active":"2024-03-06T07:26:41.311074+0000","last_peered":"2024-03-06T07:26:41.311074+0000","last_clean":"2024-03-06T07:26:41.311074+0000","last_became_active":"2024-03-06T07:26:38.251978+0000","last_became_peered":"2024-03-06T07:26:38.251978+0000","last_unstale":"2024-03-06T07:26:41.311074+0000","last_undegraded":"2024-03-06T07:26:41.311074+0000","last_fullsized":"2024-03-06T07:26:41.311074+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:28.311316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255456+0000","last_change":"2024-03-06T07:26:39.264834+0000","last_active":"2024-03-06T07:26:40.255456+0000","last_peered":"2024-03-06T07:26:40.255456+0000","last_clean":"2024-03-06T07:26:40.255456+0000","last_became_active":"2024-03-06T07:26:39.264461+0000","last_became_peered":"2024-03-06T07:26:39.264461+0000","last_unstale":"2024-03-06T07:26:40.255456+0000","last_undegraded":"2024-03-06T07:26:40.255456+0000","last_fullsized":"2024-03-06T07:26:40.255456+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:33:15.442066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255608+0000","last_change":"2024-03-06T07:26:38.263737+0000","last_active":"2024-03-06T07:26:40.255608+0000","last_peered":"2024-03-06T07:26:40.255608+0000","last_clean":"2024-03-06T07:26:40.255608+0000","last_became_active":"2024-03-06T07:26:38.263193+0000","last_became_peered":"2024-03-06T07:26:38.263193+0000","last_unstale":"2024-03-06T07:26:40.255608+0000","last_undegraded":"2024-03-06T07:26:40.255608+0000","last_fullsized":"2024-03-06T07:26:40.255608+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:25.499508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256009+0000","last_change":"2024-03-06T07:26:39.255539+0000","last_active":"2024-03-06T07:26:40.256009+0000","last_peered":"2024-03-06T07:26:40.256009+0000","last_clean":"2024-03-06T07:26:40.256009+0000","last_became_active":"2024-03-06T07:26:39.255246+0000","last_became_peered":"2024-03-06T07:26:39.255246+0000","last_unstale":"2024-03-06T07:26:40.256009+0000","last_undegraded":"2024-03-06T07:26:40.256009+0000","last_fullsized":"2024-03-06T07:26:40.256009+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:43:39.543893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249648+0000","last_change":"2024-03-06T07:26:38.257296+0000","last_active":"2024-03-06T07:26:40.249648+0000","last_peered":"2024-03-06T07:26:40.249648+0000","last_clean":"2024-03-06T07:26:40.249648+0000","last_became_active":"2024-03-06T07:26:38.257010+0000","last_became_peered":"2024-03-06T07:26:38.257010+0000","last_unstale":"2024-03-06T07:26:40.249648+0000","last_undegraded":"2024-03-06T07:26:40.249648+0000","last_fullsized":"2024-03-06T07:26:40.249648+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:48.553025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320574+0000","last_change":"2024-03-06T07:26:39.253050+0000","last_active":"2024-03-06T07:26:40.320574+0000","last_peered":"2024-03-06T07:26:40.320574+0000","last_clean":"2024-03-06T07:26:40.320574+0000","last_became_active":"2024-03-06T07:26:39.252742+0000","last_became_peered":"2024-03-06T07:26:39.252742+0000","last_unstale":"2024-03-06T07:26:40.320574+0000","last_undegraded":"2024-03-06T07:26:40.320574+0000","last_fullsized":"2024-03-06T07:26:40.320574+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:53:04.657329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810072+0000","last_change":"2024-03-06T07:26:38.265662+0000","last_active":"2024-03-06T07:26:40.810072+0000","last_peered":"2024-03-06T07:26:40.810072+0000","last_clean":"2024-03-06T07:26:40.810072+0000","last_became_active":"2024-03-06T07:26:38.265394+0000","last_became_peered":"2024-03-06T07:26:38.265394+0000","last_unstale":"2024-03-06T07:26:40.810072+0000","last_undegraded":"2024-03-06T07:26:40.810072+0000","last_fullsized":"2024-03-06T07:26:40.810072+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:39.402401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311213+0000","last_change":"2024-03-06T07:26:39.267995+0000","last_active":"2024-03-06T07:26:41.311213+0000","last_peered":"2024-03-06T07:26:41.311213+0000","last_clean":"2024-03-06T07:26:41.311213+0000","last_became_active":"2024-03-06T07:26:39.267720+0000","last_became_peered":"2024-03-06T07:26:39.267720+0000","last_unstale":"2024-03-06T07:26:41.311213+0000","last_undegraded":"2024-03-06T07:26:41.311213+0000","last_fullsized":"2024-03-06T07:26:41.311213+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:26.578348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310921+0000","last_change":"2024-03-06T07:26:38.261164+0000","last_active":"2024-03-06T07:26:41.310921+0000","last_peered":"2024-03-06T07:26:41.310921+0000","last_clean":"2024-03-06T07:26:41.310921+0000","last_became_active":"2024-03-06T07:26:38.260678+0000","last_became_peered":"2024-03-06T07:26:38.260678+0000","last_unstale":"2024-03-06T07:26:41.310921+0000","last_undegraded":"2024-03-06T07:26:41.310921+0000","last_fullsized":"2024-03-06T07:26:41.310921+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:23.787933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311132+0000","last_change":"2024-03-06T07:26:39.269820+0000","last_active":"2024-03-06T07:26:41.311132+0000","last_peered":"2024-03-06T07:26:41.311132+0000","last_clean":"2024-03-06T07:26:41.311132+0000","last_became_active":"2024-03-06T07:26:39.269531+0000","last_became_peered":"2024-03-06T07:26:39.269531+0000","last_unstale":"2024-03-06T07:26:41.311132+0000","last_undegraded":"2024-03-06T07:26:41.311132+0000","last_fullsized":"2024-03-06T07:26:41.311132+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:26.505556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310799+0000","last_change":"2024-03-06T07:26:38.262646+0000","last_active":"2024-03-06T07:26:41.310799+0000","last_peered":"2024-03-06T07:26:41.310799+0000","last_clean":"2024-03-06T07:26:41.310799+0000","last_became_active":"2024-03-06T07:26:38.262377+0000","last_became_peered":"2024-03-06T07:26:38.262377+0000","last_unstale":"2024-03-06T07:26:41.310799+0000","last_undegraded":"2024-03-06T07:26:41.310799+0000","last_fullsized":"2024-03-06T07:26:41.310799+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:53:08.484807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810491+0000","last_change":"2024-03-06T07:26:39.264351+0000","last_active":"2024-03-06T07:26:40.810491+0000","last_peered":"2024-03-06T07:26:40.810491+0000","last_clean":"2024-03-06T07:26:40.810491+0000","last_became_active":"2024-03-06T07:26:39.264081+0000","last_became_peered":"2024-03-06T07:26:39.264081+0000","last_unstale":"2024-03-06T07:26:40.810491+0000","last_undegraded":"2024-03-06T07:26:40.810491+0000","last_fullsized":"2024-03-06T07:26:40.810491+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:29:49.559073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271186+0000","last_change":"2024-03-06T07:26:38.258205+0000","last_active":"2024-03-06T07:26:40.271186+0000","last_peered":"2024-03-06T07:26:40.271186+0000","last_clean":"2024-03-06T07:26:40.271186+0000","last_became_active":"2024-03-06T07:26:38.257947+0000","last_became_peered":"2024-03-06T07:26:38.257947+0000","last_unstale":"2024-03-06T07:26:40.271186+0000","last_undegraded":"2024-03-06T07:26:40.271186+0000","last_fullsized":"2024-03-06T07:26:40.271186+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:10:27.466247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310610+0000","last_change":"2024-03-06T07:26:39.266280+0000","last_active":"2024-03-06T07:26:41.310610+0000","last_peered":"2024-03-06T07:26:41.310610+0000","last_clean":"2024-03-06T07:26:41.310610+0000","last_became_active":"2024-03-06T07:26:39.266010+0000","last_became_peered":"2024-03-06T07:26:39.266010+0000","last_unstale":"2024-03-06T07:26:41.310610+0000","last_undegraded":"2024-03-06T07:26:41.310610+0000","last_fullsized":"2024-03-06T07:26:41.310610+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:44.405245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320818+0000","last_change":"2024-03-06T07:26:38.260935+0000","last_active":"2024-03-06T07:26:40.320818+0000","last_peered":"2024-03-06T07:26:40.320818+0000","last_clean":"2024-03-06T07:26:40.320818+0000","last_became_active":"2024-03-06T07:26:38.260625+0000","last_became_peered":"2024-03-06T07:26:38.260625+0000","last_unstale":"2024-03-06T07:26:40.320818+0000","last_undegraded":"2024-03-06T07:26:40.320818+0000","last_fullsized":"2024-03-06T07:26:40.320818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:39:35.612496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271221+0000","last_change":"2024-03-06T07:26:39.256106+0000","last_active":"2024-03-06T07:26:40.271221+0000","last_peered":"2024-03-06T07:26:40.271221+0000","last_clean":"2024-03-06T07:26:40.271221+0000","last_became_active":"2024-03-06T07:26:39.255570+0000","last_became_peered":"2024-03-06T07:26:39.255570+0000","last_unstale":"2024-03-06T07:26:40.271221+0000","last_undegraded":"2024-03-06T07:26:40.271221+0000","last_fullsized":"2024-03-06T07:26:40.271221+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:52:02.271576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311356+0000","last_change":"2024-03-06T07:26:38.264961+0000","last_active":"2024-03-06T07:26:41.311356+0000","last_peered":"2024-03-06T07:26:41.311356+0000","last_clean":"2024-03-06T07:26:41.311356+0000","last_became_active":"2024-03-06T07:26:38.264360+0000","last_became_peered":"2024-03-06T07:26:38.264360+0000","last_unstale":"2024-03-06T07:26:41.311356+0000","last_undegraded":"2024-03-06T07:26:41.311356+0000","last_fullsized":"2024-03-06T07:26:41.311356+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:34.650914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810017+0000","last_change":"2024-03-06T07:26:39.265250+0000","last_active":"2024-03-06T07:26:40.810017+0000","last_peered":"2024-03-06T07:26:40.810017+0000","last_clean":"2024-03-06T07:26:40.810017+0000","last_became_active":"2024-03-06T07:26:39.264938+0000","last_became_peered":"2024-03-06T07:26:39.264938+0000","last_unstale":"2024-03-06T07:26:40.810017+0000","last_undegraded":"2024-03-06T07:26:40.810017+0000","last_fullsized":"2024-03-06T07:26:40.810017+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:01.314726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"171'3","reported_seq":17,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810178+0000","last_change":"2024-03-06T07:26:38.263099+0000","last_active":"2024-03-06T07:26:40.810178+0000","last_peered":"2024-03-06T07:26:40.810178+0000","last_clean":"2024-03-06T07:26:40.810178+0000","last_became_active":"2024-03-06T07:26:38.262764+0000","last_became_peered":"2024-03-06T07:26:38.262764+0000","last_unstale":"2024-03-06T07:26:40.810178+0000","last_undegraded":"2024-03-06T07:26:40.810178+0000","last_fullsized":"2024-03-06T07:26:40.810178+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:28.769962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320647+0000","last_change":"2024-03-06T07:26:39.263432+0000","last_active":"2024-03-06T07:26:40.320647+0000","last_peered":"2024-03-06T07:26:40.320647+0000","last_clean":"2024-03-06T07:26:40.320647+0000","last_became_active":"2024-03-06T07:26:39.263157+0000","last_became_peered":"2024-03-06T07:26:39.263157+0000","last_unstale":"2024-03-06T07:26:40.320647+0000","last_undegraded":"2024-03-06T07:26:40.320647+0000","last_fullsized":"2024-03-06T07:26:40.320647+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:31.213551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"171'6","reported_seq":20,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311032+0000","last_change":"2024-03-06T07:26:38.261487+0000","last_active":"2024-03-06T07:26:41.311032+0000","last_peered":"2024-03-06T07:26:41.311032+0000","last_clean":"2024-03-06T07:26:41.311032+0000","last_became_active":"2024-03-06T07:26:38.261216+0000","last_became_peered":"2024-03-06T07:26:38.261216+0000","last_unstale":"2024-03-06T07:26:41.311032+0000","last_undegraded":"2024-03-06T07:26:41.311032+0000","last_fullsized":"2024-03-06T07:26:41.311032+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:56:59.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249815+0000","last_change":"2024-03-06T07:26:39.255921+0000","last_active":"2024-03-06T07:26:40.249815+0000","last_peered":"2024-03-06T07:26:40.249815+0000","last_clean":"2024-03-06T07:26:40.249815+0000","last_became_active":"2024-03-06T07:26:39.255644+0000","last_became_peered":"2024-03-06T07:26:39.255644+0000","last_unstale":"2024-03-06T07:26:40.249815+0000","last_undegraded":"2024-03-06T07:26:40.249815+0000","last_fullsized":"2024-03-06T07:26:40.249815+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:34:45.948396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249765+0000","last_change":"2024-03-06T07:26:38.264768+0000","last_active":"2024-03-06T07:26:40.249765+0000","last_peered":"2024-03-06T07:26:40.249765+0000","last_clean":"2024-03-06T07:26:40.249765+0000","last_became_active":"2024-03-06T07:26:38.264507+0000","last_became_peered":"2024-03-06T07:26:38.264507+0000","last_unstale":"2024-03-06T07:26:40.249765+0000","last_undegraded":"2024-03-06T07:26:40.249765+0000","last_fullsized":"2024-03-06T07:26:40.249765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:38.785122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271593+0000","last_change":"2024-03-06T07:26:39.262631+0000","last_active":"2024-03-06T07:26:40.271593+0000","last_peered":"2024-03-06T07:26:40.271593+0000","last_clean":"2024-03-06T07:26:40.271593+0000","last_became_active":"2024-03-06T07:26:39.262118+0000","last_became_peered":"2024-03-06T07:26:39.262118+0000","last_unstale":"2024-03-06T07:26:40.271593+0000","last_undegraded":"2024-03-06T07:26:40.271593+0000","last_fullsized":"2024-03-06T07:26:40.271593+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:39:14.760963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271217+0000","last_change":"2024-03-06T07:26:38.254122+0000","last_active":"2024-03-06T07:26:40.271217+0000","last_peered":"2024-03-06T07:26:40.271217+0000","last_clean":"2024-03-06T07:26:40.271217+0000","last_became_active":"2024-03-06T07:26:38.253864+0000","last_became_peered":"2024-03-06T07:26:38.253864+0000","last_unstale":"2024-03-06T07:26:40.271217+0000","last_undegraded":"2024-03-06T07:26:40.271217+0000","last_fullsized":"2024-03-06T07:26:40.271217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:01.108773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810315+0000","last_change":"2024-03-06T07:26:39.271285+0000","last_active":"2024-03-06T07:26:40.810315+0000","last_peered":"2024-03-06T07:26:40.810315+0000","last_clean":"2024-03-06T07:26:40.810315+0000","last_became_active":"2024-03-06T07:26:39.270972+0000","last_became_peered":"2024-03-06T07:26:39.270972+0000","last_unstale":"2024-03-06T07:26:40.810315+0000","last_undegraded":"2024-03-06T07:26:40.810315+0000","last_fullsized":"2024-03-06T07:26:40.810315+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:43.077968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.809877+0000","last_change":"2024-03-06T07:26:38.264438+0000","last_active":"2024-03-06T07:26:40.809877+0000","last_peered":"2024-03-06T07:26:40.809877+0000","last_clean":"2024-03-06T07:26:40.809877+0000","last_became_active":"2024-03-06T07:26:38.264177+0000","last_became_peered":"2024-03-06T07:26:38.264177+0000","last_unstale":"2024-03-06T07:26:40.809877+0000","last_undegraded":"2024-03-06T07:26:40.809877+0000","last_fullsized":"2024-03-06T07:26:40.809877+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:04.813887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311247+0000","last_change":"2024-03-06T07:26:39.266877+0000","last_active":"2024-03-06T07:26:41.311247+0000","last_peered":"2024-03-06T07:26:41.311247+0000","last_clean":"2024-03-06T07:26:41.311247+0000","last_became_active":"2024-03-06T07:26:39.266605+0000","last_became_peered":"2024-03-06T07:26:39.266605+0000","last_unstale":"2024-03-06T07:26:41.311247+0000","last_undegraded":"2024-03-06T07:26:41.311247+0000","last_fullsized":"2024-03-06T07:26:41.311247+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:55.441174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"171'8","reported_seq":22,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320291+0000","last_change":"2024-03-06T07:26:38.258910+0000","last_active":"2024-03-06T07:26:40.320291+0000","last_peered":"2024-03-06T07:26:40.320291+0000","last_clean":"2024-03-06T07:26:40.320291+0000","last_became_active":"2024-03-06T07:26:38.258635+0000","last_became_peered":"2024-03-06T07:26:38.258635+0000","last_unstale":"2024-03-06T07:26:40.320291+0000","last_undegraded":"2024-03-06T07:26:40.320291+0000","last_fullsized":"2024-03-06T07:26:40.320291+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:25:36.832143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254809+0000","last_change":"2024-03-06T07:26:39.270147+0000","last_active":"2024-03-06T07:26:40.254809+0000","last_peered":"2024-03-06T07:26:40.254809+0000","last_clean":"2024-03-06T07:26:40.254809+0000","last_became_active":"2024-03-06T07:26:39.269481+0000","last_became_peered":"2024-03-06T07:26:39.269481+0000","last_unstale":"2024-03-06T07:26:40.254809+0000","last_undegraded":"2024-03-06T07:26:40.254809+0000","last_fullsized":"2024-03-06T07:26:40.254809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:17:54.509229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255696+0000","last_change":"2024-03-06T07:26:38.265919+0000","last_active":"2024-03-06T07:26:40.255696+0000","last_peered":"2024-03-06T07:26:40.255696+0000","last_clean":"2024-03-06T07:26:40.255696+0000","last_became_active":"2024-03-06T07:26:38.265582+0000","last_became_peered":"2024-03-06T07:26:38.265582+0000","last_unstale":"2024-03-06T07:26:40.255696+0000","last_undegraded":"2024-03-06T07:26:40.255696+0000","last_fullsized":"2024-03-06T07:26:40.255696+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:38:17.453635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255434+0000","last_change":"2024-03-06T07:26:39.260229+0000","last_active":"2024-03-06T07:26:40.255434+0000","last_peered":"2024-03-06T07:26:40.255434+0000","last_clean":"2024-03-06T07:26:40.255434+0000","last_became_active":"2024-03-06T07:26:39.259967+0000","last_became_peered":"2024-03-06T07:26:39.259967+0000","last_unstale":"2024-03-06T07:26:40.255434+0000","last_undegraded":"2024-03-06T07:26:40.255434+0000","last_fullsized":"2024-03-06T07:26:40.255434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:16:41.696337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255703+0000","last_change":"2024-03-06T07:26:38.267967+0000","last_active":"2024-03-06T07:26:40.255703+0000","last_peered":"2024-03-06T07:26:40.255703+0000","last_clean":"2024-03-06T07:26:40.255703+0000","last_became_active":"2024-03-06T07:26:38.267650+0000","last_became_peered":"2024-03-06T07:26:38.267650+0000","last_unstale":"2024-03-06T07:26:40.255703+0000","last_undegraded":"2024-03-06T07:26:40.255703+0000","last_fullsized":"2024-03-06T07:26:40.255703+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:49:25.572300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254942+0000","last_change":"2024-03-06T07:26:39.268823+0000","last_active":"2024-03-06T07:26:40.254942+0000","last_peered":"2024-03-06T07:26:40.254942+0000","last_clean":"2024-03-06T07:26:40.254942+0000","last_became_active":"2024-03-06T07:26:39.268430+0000","last_became_peered":"2024-03-06T07:26:39.268430+0000","last_unstale":"2024-03-06T07:26:40.254942+0000","last_undegraded":"2024-03-06T07:26:40.254942+0000","last_fullsized":"2024-03-06T07:26:40.254942+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:26:24.793981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249674+0000","last_change":"2024-03-06T07:26:38.260630+0000","last_active":"2024-03-06T07:26:40.249674+0000","last_peered":"2024-03-06T07:26:40.249674+0000","last_clean":"2024-03-06T07:26:40.249674+0000","last_became_active":"2024-03-06T07:26:38.260349+0000","last_became_peered":"2024-03-06T07:26:38.260349+0000","last_unstale":"2024-03-06T07:26:40.249674+0000","last_undegraded":"2024-03-06T07:26:40.249674+0000","last_fullsized":"2024-03-06T07:26:40.249674+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:58:54.003451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255672+0000","last_change":"2024-03-06T07:26:39.260197+0000","last_active":"2024-03-06T07:26:40.255672+0000","last_peered":"2024-03-06T07:26:40.255672+0000","last_clean":"2024-03-06T07:26:40.255672+0000","last_became_active":"2024-03-06T07:26:39.259915+0000","last_became_peered":"2024-03-06T07:26:39.259915+0000","last_unstale":"2024-03-06T07:26:40.255672+0000","last_undegraded":"2024-03-06T07:26:40.255672+0000","last_fullsized":"2024-03-06T07:26:40.255672+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:29.317363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320739+0000","last_change":"2024-03-06T07:26:38.261857+0000","last_active":"2024-03-06T07:26:40.320739+0000","last_peered":"2024-03-06T07:26:40.320739+0000","last_clean":"2024-03-06T07:26:40.320739+0000","last_became_active":"2024-03-06T07:26:38.261487+0000","last_became_peered":"2024-03-06T07:26:38.261487+0000","last_unstale":"2024-03-06T07:26:40.320739+0000","last_undegraded":"2024-03-06T07:26:40.320739+0000","last_fullsized":"2024-03-06T07:26:40.320739+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:11:01.272726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249841+0000","last_change":"2024-03-06T07:26:39.266307+0000","last_active":"2024-03-06T07:26:40.249841+0000","last_peered":"2024-03-06T07:26:40.249841+0000","last_clean":"2024-03-06T07:26:40.249841+0000","last_became_active":"2024-03-06T07:26:39.266035+0000","last_became_peered":"2024-03-06T07:26:39.266035+0000","last_unstale":"2024-03-06T07:26:40.249841+0000","last_undegraded":"2024-03-06T07:26:40.249841+0000","last_fullsized":"2024-03-06T07:26:40.249841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:12:43.243557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320629+0000","last_change":"2024-03-06T07:26:38.257000+0000","last_active":"2024-03-06T07:26:40.320629+0000","last_peered":"2024-03-06T07:26:40.320629+0000","last_clean":"2024-03-06T07:26:40.320629+0000","last_became_active":"2024-03-06T07:26:38.256724+0000","last_became_peered":"2024-03-06T07:26:38.256724+0000","last_unstale":"2024-03-06T07:26:40.320629+0000","last_undegraded":"2024-03-06T07:26:40.320629+0000","last_fullsized":"2024-03-06T07:26:40.320629+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:14:59.901171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311151+0000","last_change":"2024-03-06T07:26:39.272306+0000","last_active":"2024-03-06T07:26:41.311151+0000","last_peered":"2024-03-06T07:26:41.311151+0000","last_clean":"2024-03-06T07:26:41.311151+0000","last_became_active":"2024-03-06T07:26:39.272052+0000","last_became_peered":"2024-03-06T07:26:39.272052+0000","last_unstale":"2024-03-06T07:26:41.311151+0000","last_undegraded":"2024-03-06T07:26:41.311151+0000","last_fullsized":"2024-03-06T07:26:41.311151+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:40.408449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.249621+0000","last_change":"2024-03-06T07:26:38.260016+0000","last_active":"2024-03-06T07:26:40.249621+0000","last_peered":"2024-03-06T07:26:40.249621+0000","last_clean":"2024-03-06T07:26:40.249621+0000","last_became_active":"2024-03-06T07:26:38.259723+0000","last_became_peered":"2024-03-06T07:26:38.259723+0000","last_unstale":"2024-03-06T07:26:40.249621+0000","last_undegraded":"2024-03-06T07:26:40.249621+0000","last_fullsized":"2024-03-06T07:26:40.249621+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:13:45.259421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810367+0000","last_change":"2024-03-06T07:26:39.263792+0000","last_active":"2024-03-06T07:26:40.810367+0000","last_peered":"2024-03-06T07:26:40.810367+0000","last_clean":"2024-03-06T07:26:40.810367+0000","last_became_active":"2024-03-06T07:26:39.263324+0000","last_became_peered":"2024-03-06T07:26:39.263324+0000","last_unstale":"2024-03-06T07:26:40.810367+0000","last_undegraded":"2024-03-06T07:26:40.810367+0000","last_fullsized":"2024-03-06T07:26:40.810367+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:21:41.110242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310763+0000","last_change":"2024-03-06T07:26:38.253562+0000","last_active":"2024-03-06T07:26:41.310763+0000","last_peered":"2024-03-06T07:26:41.310763+0000","last_clean":"2024-03-06T07:26:41.310763+0000","last_became_active":"2024-03-06T07:26:38.252639+0000","last_became_peered":"2024-03-06T07:26:38.252639+0000","last_unstale":"2024-03-06T07:26:41.310763+0000","last_undegraded":"2024-03-06T07:26:41.310763+0000","last_fullsized":"2024-03-06T07:26:41.310763+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:29:40.939285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256367+0000","last_change":"2024-03-06T07:26:39.255525+0000","last_active":"2024-03-06T07:26:40.256367+0000","last_peered":"2024-03-06T07:26:40.256367+0000","last_clean":"2024-03-06T07:26:40.256367+0000","last_became_active":"2024-03-06T07:26:39.255218+0000","last_became_peered":"2024-03-06T07:26:39.255218+0000","last_unstale":"2024-03-06T07:26:40.256367+0000","last_undegraded":"2024-03-06T07:26:40.256367+0000","last_fullsized":"2024-03-06T07:26:40.256367+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:03.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254783+0000","last_change":"2024-03-06T07:26:38.265988+0000","last_active":"2024-03-06T07:26:40.254783+0000","last_peered":"2024-03-06T07:26:40.254783+0000","last_clean":"2024-03-06T07:26:40.254783+0000","last_became_active":"2024-03-06T07:26:38.265493+0000","last_became_peered":"2024-03-06T07:26:38.265493+0000","last_unstale":"2024-03-06T07:26:40.254783+0000","last_undegraded":"2024-03-06T07:26:40.254783+0000","last_fullsized":"2024-03-06T07:26:40.254783+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:33:46.694225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.254957+0000","last_change":"2024-03-06T07:26:39.270436+0000","last_active":"2024-03-06T07:26:40.254957+0000","last_peered":"2024-03-06T07:26:40.254957+0000","last_clean":"2024-03-06T07:26:40.254957+0000","last_became_active":"2024-03-06T07:26:39.269858+0000","last_became_peered":"2024-03-06T07:26:39.269858+0000","last_unstale":"2024-03-06T07:26:40.254957+0000","last_undegraded":"2024-03-06T07:26:40.254957+0000","last_fullsized":"2024-03-06T07:26:40.254957+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:07:47.286326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"171'2","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810532+0000","last_change":"2024-03-06T07:26:38.260321+0000","last_active":"2024-03-06T07:26:40.810532+0000","last_peered":"2024-03-06T07:26:40.810532+0000","last_clean":"2024-03-06T07:26:40.810532+0000","last_became_active":"2024-03-06T07:26:38.260071+0000","last_became_peered":"2024-03-06T07:26:38.260071+0000","last_unstale":"2024-03-06T07:26:40.810532+0000","last_undegraded":"2024-03-06T07:26:40.810532+0000","last_fullsized":"2024-03-06T07:26:40.810532+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:48.862475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271312+0000","last_change":"2024-03-06T07:26:39.255863+0000","last_active":"2024-03-06T07:26:40.271312+0000","last_peered":"2024-03-06T07:26:40.271312+0000","last_clean":"2024-03-06T07:26:40.271312+0000","last_became_active":"2024-03-06T07:26:39.255180+0000","last_became_peered":"2024-03-06T07:26:39.255180+0000","last_unstale":"2024-03-06T07:26:40.271312+0000","last_undegraded":"2024-03-06T07:26:40.271312+0000","last_fullsized":"2024-03-06T07:26:40.271312+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:05:23.998787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"172'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.256077+0000","last_change":"2024-03-06T07:26:38.256677+0000","last_active":"2024-03-06T07:26:40.256077+0000","last_peered":"2024-03-06T07:26:40.256077+0000","last_clean":"2024-03-06T07:26:40.256077+0000","last_became_active":"2024-03-06T07:26:38.256360+0000","last_became_peered":"2024-03-06T07:26:38.256360+0000","last_unstale":"2024-03-06T07:26:40.256077+0000","last_undegraded":"2024-03-06T07:26:40.256077+0000","last_fullsized":"2024-03-06T07:26:40.256077+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:50:36.319241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810018+0000","last_change":"2024-03-06T07:26:39.259869+0000","last_active":"2024-03-06T07:26:40.810018+0000","last_peered":"2024-03-06T07:26:40.810018+0000","last_clean":"2024-03-06T07:26:40.810018+0000","last_became_active":"2024-03-06T07:26:39.259073+0000","last_became_peered":"2024-03-06T07:26:39.259073+0000","last_unstale":"2024-03-06T07:26:40.810018+0000","last_undegraded":"2024-03-06T07:26:40.810018+0000","last_fullsized":"2024-03-06T07:26:40.810018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:03:14.831639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310939+0000","last_change":"2024-03-06T07:26:38.261044+0000","last_active":"2024-03-06T07:26:41.310939+0000","last_peered":"2024-03-06T07:26:41.310939+0000","last_clean":"2024-03-06T07:26:41.310939+0000","last_became_active":"2024-03-06T07:26:38.260783+0000","last_became_peered":"2024-03-06T07:26:38.260783+0000","last_unstale":"2024-03-06T07:26:41.310939+0000","last_undegraded":"2024-03-06T07:26:41.310939+0000","last_fullsized":"2024-03-06T07:26:41.310939+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:05:15.768174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255088+0000","last_change":"2024-03-06T07:26:39.263856+0000","last_active":"2024-03-06T07:26:40.255088+0000","last_peered":"2024-03-06T07:26:40.255088+0000","last_clean":"2024-03-06T07:26:40.255088+0000","last_became_active":"2024-03-06T07:26:39.263117+0000","last_became_peered":"2024-03-06T07:26:39.263117+0000","last_unstale":"2024-03-06T07:26:40.255088+0000","last_undegraded":"2024-03-06T07:26:40.255088+0000","last_fullsized":"2024-03-06T07:26:40.255088+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:11:38.027398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255267+0000","last_change":"2024-03-06T07:26:38.266025+0000","last_active":"2024-03-06T07:26:40.255267+0000","last_peered":"2024-03-06T07:26:40.255267+0000","last_clean":"2024-03-06T07:26:40.255267+0000","last_became_active":"2024-03-06T07:26:38.265692+0000","last_became_peered":"2024-03-06T07:26:38.265692+0000","last_unstale":"2024-03-06T07:26:40.255267+0000","last_undegraded":"2024-03-06T07:26:40.255267+0000","last_fullsized":"2024-03-06T07:26:40.255267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:32:43.399197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255226+0000","last_change":"2024-03-06T07:26:39.256045+0000","last_active":"2024-03-06T07:26:40.255226+0000","last_peered":"2024-03-06T07:26:40.255226+0000","last_clean":"2024-03-06T07:26:40.255226+0000","last_became_active":"2024-03-06T07:26:39.255761+0000","last_became_peered":"2024-03-06T07:26:39.255761+0000","last_unstale":"2024-03-06T07:26:40.255226+0000","last_undegraded":"2024-03-06T07:26:40.255226+0000","last_fullsized":"2024-03-06T07:26:40.255226+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:16:54.107202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255717+0000","last_change":"2024-03-06T07:26:38.255242+0000","last_active":"2024-03-06T07:26:40.255717+0000","last_peered":"2024-03-06T07:26:40.255717+0000","last_clean":"2024-03-06T07:26:40.255717+0000","last_became_active":"2024-03-06T07:26:38.254954+0000","last_became_peered":"2024-03-06T07:26:38.254954+0000","last_unstale":"2024-03-06T07:26:40.255717+0000","last_undegraded":"2024-03-06T07:26:40.255717+0000","last_fullsized":"2024-03-06T07:26:40.255717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:19.334482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311234+0000","last_change":"2024-03-06T07:26:39.271443+0000","last_active":"2024-03-06T07:26:41.311234+0000","last_peered":"2024-03-06T07:26:41.311234+0000","last_clean":"2024-03-06T07:26:41.311234+0000","last_became_active":"2024-03-06T07:26:39.270882+0000","last_became_peered":"2024-03-06T07:26:39.270882+0000","last_unstale":"2024-03-06T07:26:41.311234+0000","last_undegraded":"2024-03-06T07:26:41.311234+0000","last_fullsized":"2024-03-06T07:26:41.311234+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:07.841172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810076+0000","last_change":"2024-03-06T07:26:38.266006+0000","last_active":"2024-03-06T07:26:40.810076+0000","last_peered":"2024-03-06T07:26:40.810076+0000","last_clean":"2024-03-06T07:26:40.810076+0000","last_became_active":"2024-03-06T07:26:38.265728+0000","last_became_peered":"2024-03-06T07:26:38.265728+0000","last_unstale":"2024-03-06T07:26:40.810076+0000","last_undegraded":"2024-03-06T07:26:40.810076+0000","last_fullsized":"2024-03-06T07:26:40.810076+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:54.435126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.311371+0000","last_change":"2024-03-06T07:26:39.268286+0000","last_active":"2024-03-06T07:26:41.311371+0000","last_peered":"2024-03-06T07:26:41.311371+0000","last_clean":"2024-03-06T07:26:41.311371+0000","last_became_active":"2024-03-06T07:26:39.268027+0000","last_became_peered":"2024-03-06T07:26:39.268027+0000","last_unstale":"2024-03-06T07:26:41.311371+0000","last_undegraded":"2024-03-06T07:26:41.311371+0000","last_fullsized":"2024-03-06T07:26:41.311371+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:02:49.868523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255501+0000","last_change":"2024-03-06T07:26:38.262673+0000","last_active":"2024-03-06T07:26:40.255501+0000","last_peered":"2024-03-06T07:26:40.255501+0000","last_clean":"2024-03-06T07:26:40.255501+0000","last_became_active":"2024-03-06T07:26:38.260157+0000","last_became_peered":"2024-03-06T07:26:38.260157+0000","last_unstale":"2024-03-06T07:26:40.255501+0000","last_undegraded":"2024-03-06T07:26:40.255501+0000","last_fullsized":"2024-03-06T07:26:40.255501+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:57:45.779315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.271488+0000","last_change":"2024-03-06T07:26:39.255450+0000","last_active":"2024-03-06T07:26:40.271488+0000","last_peered":"2024-03-06T07:26:40.271488+0000","last_clean":"2024-03-06T07:26:40.271488+0000","last_became_active":"2024-03-06T07:26:39.255136+0000","last_became_peered":"2024-03-06T07:26:39.255136+0000","last_unstale":"2024-03-06T07:26:40.271488+0000","last_undegraded":"2024-03-06T07:26:40.271488+0000","last_fullsized":"2024-03-06T07:26:40.271488+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:37.172755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.320409+0000","last_change":"2024-03-06T07:26:38.255185+0000","last_active":"2024-03-06T07:26:40.320409+0000","last_peered":"2024-03-06T07:26:40.320409+0000","last_clean":"2024-03-06T07:26:40.320409+0000","last_became_active":"2024-03-06T07:26:38.254907+0000","last_became_peered":"2024-03-06T07:26:38.254907+0000","last_unstale":"2024-03-06T07:26:40.320409+0000","last_undegraded":"2024-03-06T07:26:40.320409+0000","last_fullsized":"2024-03-06T07:26:40.320409+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:39:05.814671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.255455+0000","last_change":"2024-03-06T07:26:39.264397+0000","last_active":"2024-03-06T07:26:40.255455+0000","last_peered":"2024-03-06T07:26:40.255455+0000","last_clean":"2024-03-06T07:26:40.255455+0000","last_became_active":"2024-03-06T07:26:39.264081+0000","last_became_peered":"2024-03-06T07:26:39.264081+0000","last_unstale":"2024-03-06T07:26:40.255455+0000","last_undegraded":"2024-03-06T07:26:40.255455+0000","last_fullsized":"2024-03-06T07:26:40.255455+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:38.240198+0000","last_clean_scrub_stamp":"2024-03-06T07:26:38.240198+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:22:20.003461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"171'1","reported_seq":15,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:40.810075+0000","last_change":"2024-03-06T07:26:38.262369+0000","last_active":"2024-03-06T07:26:40.810075+0000","last_peered":"2024-03-06T07:26:40.810075+0000","last_clean":"2024-03-06T07:26:40.810075+0000","last_became_active":"2024-03-06T07:26:38.262081+0000","last_became_peered":"2024-03-06T07:26:38.262081+0000","last_unstale":"2024-03-06T07:26:40.810075+0000","last_undegraded":"2024-03-06T07:26:40.810075+0000","last_fullsized":"2024-03-06T07:26:40.810075+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:26:37.234151+0000","last_clean_scrub_stamp":"2024-03-06T07:26:37.234151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:06.835124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":451,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-03-06T07:26:41.310554+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:26:41.310554+0000","last_peered":"2024-03-06T07:26:41.310554+0000","last_clean":"2024-03-06T07:26:41.310554+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:26:41.310554+0000","last_undegraded":"2024-03-06T07:26:41.310554+0000","last_fullsized":"2024-03-06T07:26:41.310554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2256896,"data_stored":1656727,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33324,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2191360,"data_stored":1066871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2183168,"data_stored":1064733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1065259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2187264,"data_stored":1064803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1064767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1064677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37536,"kb_used_data":2192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334304,"statfs":{"total":96636764160,"available":96598327296,"internally_reserved":0,"allocated":2244608,"data_stored":1655135,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:26:43.822 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:26:43.823 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:26:43.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.966+0000 7f04c07ac640 1 Processor -- start 2024-03-06T07:26:43.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.966+0000 7f04c07ac640 1 -- start start 2024-03-06T07:26:43.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04c07ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04b81539c0 0x7f04b8072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04c07ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80c62a0 con 0x7f04b80a8290 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04c07ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80c4c00 con 0x7f04b80c4740 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04c07ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b80c4ee0 con 0x7f04b81539c0 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04bed49640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f04b81539c0 0x7f04b8072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04be548640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f04b80a8290 0x7f04b80701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45776/0 (socket says 172.21.15.17:45776) 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04be548640 1 -- 172.21.15.17:0/3258483018 learned_addr learned my addr 172.21.15.17:0/3258483018 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04bed49640 1 -- 172.21.15.17:0/3258483018 >> v1:172.21.15.163:6789/0 conn(0x7f04b80c4740 legacy=0x7f04b80c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04bed49640 1 -- 172.21.15.17:0/3258483018 >> v1:172.21.15.17:6789/0 conn(0x7f04b80a8290 legacy=0x7f04b80701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.967+0000 7f04bed49640 1 -- 172.21.15.17:0/3258483018 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b806f9a0 con 0x7f04b81539c0 2024-03-06T07:26:43.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04bed49640 1 --2- 172.21.15.17:0/3258483018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b81539c0 0x7f04b8072680 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009d90 tx=0x7f04b002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e383cf99b7594a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04bd546640 1 -- 172.21.15.17:0/3258483018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0039070 con 0x7f04b81539c0 2024-03-06T07:26:43.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04bd546640 1 -- 172.21.15.17:0/3258483018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b002cbd0 con 0x7f04b81539c0 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04bd546640 1 -- 172.21.15.17:0/3258483018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0034440 con 0x7f04b81539c0 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04c07ac640 1 -- 172.21.15.17:0/3258483018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b81539c0 msgr2=0x7f04b8072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.968+0000 7f04c07ac640 1 --2- 172.21.15.17:0/3258483018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b81539c0 0x7f04b8072680 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009d90 tx=0x7f04b002be20 comp rx=0 tx=0).stop 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.969+0000 7f04c07ac640 1 -- 172.21.15.17:0/3258483018 shutdown_connections 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.969+0000 7f04c07ac640 1 --2- 172.21.15.17:0/3258483018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b81539c0 0x7f04b8072680 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.969+0000 7f04c07ac640 1 -- 172.21.15.17:0/3258483018 >> 172.21.15.17:0/3258483018 conn(0x7f04b80c5560 msgr2=0x7f04b80c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.969+0000 7f04c07ac640 1 -- 172.21.15.17:0/3258483018 shutdown_connections 2024-03-06T07:26:43.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.969+0000 7f04c07ac640 1 -- 172.21.15.17:0/3258483018 wait complete. 2024-03-06T07:26:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.970+0000 7f04c07ac640 1 Processor -- start 2024-03-06T07:26:43.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.970+0000 7f04c07ac640 1 -- start start 2024-03-06T07:26:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 0x7f04b81e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 0x7f04b81dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8072bc0 con 0x7f04b80a8290 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8153e10 con 0x7f04b81539c0 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.971+0000 7f04c07ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b8072d40 con 0x7f04b80c4740 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56798/0 (socket says 172.21.15.17:56798) 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 -- 172.21.15.17:0/1273992751 learned_addr learned my addr 172.21.15.17:0/1273992751 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 -- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 msgr2=0x7f04b81e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04bed49640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 0x7f04b81dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04bdd47640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 0x7f04b81e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 0x7f04b81e8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 -- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 msgr2=0x7f04b81dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 0x7f04b81dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 -- 172.21.15.17:0/1273992751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80aece0 con 0x7f04b80a8290 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04bdd47640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 0x7f04b81e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04bed49640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 0x7f04b81dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04be548640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c4d0 tx=0x7f04b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4017070 con 0x7f04b80a8290 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4005ce0 con 0x7f04b80a8290 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b814ae20 con 0x7f04b80a8290 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4007270 con 0x7f04b80a8290 2024-03-06T07:26:43.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.972+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b81ec110 con 0x7f04b80a8290 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.973+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b804f330 con 0x7f04b80a8290 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.974+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04b401a070 con 0x7f04b80a8290 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.974+0000 7f04aeffd640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.974+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b4089540 con 0x7f04b80a8290 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.974+0000 7f04bdd47640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:43.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.975+0000 7f04bdd47640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f04b814e0d0 tx=0x7f04a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:43.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:43.978+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f04b4057c70 con 0x7f04b80a8290 2024-03-06T07:26:44.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.225+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f04b80735e0 con 0x7f04b80a8290 2024-03-06T07:26:44.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.227+0000 7f04aeffd640 1 -- 172.21.15.17:0/1273992751 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f04b405bd10 con 0x7f04b80a8290 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:26:44.229 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 msgr2=0x7f049c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f04b814e0d0 tx=0x7f04a8009040 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 msgr2=0x7f04b814db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c4d0 tx=0x7f04b400c990 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 shutdown_connections 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04b81539c0 0x7f04b81dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04b80c4740 0x7f04b81e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 --2- 172.21.15.17:0/1273992751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04b80a8290 0x7f04b814db90 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 >> 172.21.15.17:0/1273992751 conn(0x7f04b80c5560 msgr2=0x7f04b81dc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 shutdown_connections 2024-03-06T07:26:44.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.229+0000 7f04c07ac640 1 -- 172.21.15.17:0/1273992751 wait complete. 2024-03-06T07:26:44.241 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:44.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.354+0000 7f9cad2c9640 1 Processor -- start 2024-03-06T07:26:44.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.355+0000 7f9cad2c9640 1 -- start start 2024-03-06T07:26:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9cad2c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 0x7f9ca81558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9cad2c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8158800 con 0x7f9ca80c3280 2024-03-06T07:26:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9cad2c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8158ae0 con 0x7f9ca806d9f0 2024-03-06T07:26:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9cad2c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8158dc0 con 0x7f9ca80c3710 2024-03-06T07:26:44.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9ca80c3280 0x7f9ca81551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45782/0 (socket says 172.21.15.17:45782) 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca67fc640 1 -- 172.21.15.17:0/553626497 learned_addr learned my addr 172.21.15.17:0/553626497 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca77fe640 1 --2- 172.21.15.17:0/553626497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 0x7f9ca81558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260327451 0 0) 0x7f9ca8158800 con 0x7f9ca80c3280 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c94003520 con 0x7f9ca80c3280 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246884525 0 0) 0x7f9ca8158ae0 con 0x7f9ca806d9f0 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca8158800 con 0x7f9ca806d9f0 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca77fe640 1 -- 172.21.15.17:0/553626497 >> v1:172.21.15.163:6789/0 conn(0x7f9ca806d9f0 legacy=0x7f9ca80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca77fe640 1 -- 172.21.15.17:0/553626497 >> v1:172.21.15.17:6789/0 conn(0x7f9ca80c3280 legacy=0x7f9ca81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.356+0000 7f9ca77fe640 1 -- 172.21.15.17:0/553626497 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca81590a0 con 0x7f9ca80c3710 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.357+0000 7f9ca77fe640 1 --2- 172.21.15.17:0/553626497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 0x7f9ca81558f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009dc0 tx=0x7f9ca002be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54e52990ce849498 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.357+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0039070 con 0x7f9ca80c3710 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.357+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca002cc40 con 0x7f9ca80c3710 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9ca5ffb640 1 -- 172.21.15.17:0/553626497 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0034440 con 0x7f9ca80c3710 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 -- 172.21.15.17:0/553626497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 msgr2=0x7f9ca81558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/553626497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 0x7f9ca81558f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009dc0 tx=0x7f9ca002be90 comp rx=0 tx=0).stop 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 -- 172.21.15.17:0/553626497 shutdown_connections 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/553626497 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3710 0x7f9ca81558f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 -- 172.21.15.17:0/553626497 >> 172.21.15.17:0/553626497 conn(0x7f9ca80aa630 msgr2=0x7f9ca80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 -- 172.21.15.17:0/553626497 shutdown_connections 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.358+0000 7f9cad2c9640 1 -- 172.21.15.17:0/553626497 wait complete. 2024-03-06T07:26:44.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.359+0000 7f9cad2c9640 1 Processor -- start 2024-03-06T07:26:44.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.359+0000 7f9cad2c9640 1 -- start start 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ca806d9f0 0x7f9ca80ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 0x7f9ca81ea780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca806ddd0 con 0x7f9ca80c3710 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8155e30 con 0x7f9ca806d9f0 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9cad2c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca806df50 con 0x7f9ca80c3280 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56814/0 (socket says 172.21.15.17:56814) 2024-03-06T07:26:44.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 -- 172.21.15.17:0/2427047266 learned_addr learned my addr 172.21.15.17:0/2427047266 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca6ffd640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ca806d9f0 0x7f9ca80ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca67fc640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 0x7f9ca81ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 -- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 msgr2=0x7f9ca81ea780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 0x7f9ca81ea780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 -- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ca806d9f0 msgr2=0x7f9ca80ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ca806d9f0 0x7f9ca80ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.360+0000 7f9ca77fe640 1 -- 172.21.15.17:0/2427047266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca809b710 con 0x7f9ca80c3710 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9ca67fc640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 0x7f9ca81ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9ca77fe640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009ee0 tx=0x7f9ca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0042020 con 0x7f9ca80c3710 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca003a040 con 0x7f9ca80c3710 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca81eb200 con 0x7f9ca80c3710 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0039070 con 0x7f9ca80c3710 2024-03-06T07:26:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.361+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca806ec10 con 0x7f9ca80c3710 2024-03-06T07:26:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.362+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca804f330 con 0x7f9ca80c3710 2024-03-06T07:26:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.363+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ca0045020 con 0x7f9ca80c3710 2024-03-06T07:26:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.363+0000 7f9c9f7fe640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.364+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ca00ac1e0 con 0x7f9ca80c3710 2024-03-06T07:26:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.364+0000 7f9ca6ffd640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.364+0000 7f9ca6ffd640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 0x7f9c8806d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9c980011e0 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.368+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ca004c030 con 0x7f9ca80c3710 2024-03-06T07:26:44.536 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:26:44.535+0000 7f2ae7895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.640+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca8073240 con 0x7f9ca80c3710 2024-03-06T07:26:44.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.641+0000 7f9c9f7fe640 1 -- 172.21.15.17:0/2427047266 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f9ca007a870 con 0x7f9ca80c3710 2024-03-06T07:26:44.643 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:44.643 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":160,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:41.798922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9432,"qdb_cluster":[9432]},"id":13}]} 2024-03-06T07:26:44.643 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 160 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 msgr2=0x7f9c8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 0x7f9c8806d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9c980011e0 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 msgr2=0x7f9ca81eacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009ee0 tx=0x7f9ca0004a40 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 shutdown_connections 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c8806b500 0x7f9c8806d9c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ca80c3710 0x7f9ca81eacc0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ca80c3280 0x7f9ca81ea780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 --2- 172.21.15.17:0/2427047266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ca806d9f0 0x7f9ca80ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 >> 172.21.15.17:0/2427047266 conn(0x7f9ca80aa630 msgr2=0x7f9ca81532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 shutdown_connections 2024-03-06T07:26:44.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:44.643+0000 7f9cad2c9640 1 -- 172.21.15.17:0/2427047266 wait complete. 2024-03-06T07:26:45.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 160, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:41.798922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9432, 'qdb_cluster': [9432]} 2024-03-06T07:26:45.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:26:45.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:26:45.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.160+0000 7f7f1d89d640 1 Processor -- start 2024-03-06T07:26:45.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.160+0000 7f7f1d89d640 1 -- start start 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f1d89d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f180a7ff0 0x7f7f18155900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f1d89d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180733f0 con 0x7f7f18152b40 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f1d89d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f180c82c0 con 0x7f7f180724d0 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f1d89d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f18072d70 con 0x7f7f180a7ff0 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f16ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7f18152b40 0x7f7f180c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45792/0 (socket says 172.21.15.17:45792) 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.161+0000 7f7f16ffd640 1 -- 172.21.15.17:0/139111201 learned_addr learned my addr 172.21.15.17:0/139111201 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f177fe640 1 --2- 172.21.15.17:0/139111201 >> v2:172.21.15.163:3300/0 conn(0x7f7f180a7ff0 0x7f7f18155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500775273 0 0) 0x7f7f180733f0 con 0x7f7f18152b40 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f00003520 con 0x7f7f18152b40 2024-03-06T07:26:45.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945558708 0 0) 0x7f7f00003520 con 0x7f7f18152b40 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 >> v2:172.21.15.163:3300/0 conn(0x7f7f180a7ff0 msgr2=0x7f7f18155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 --2- 172.21.15.17:0/139111201 >> v2:172.21.15.163:3300/0 conn(0x7f7f180a7ff0 0x7f7f18155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 >> v1:172.21.15.163:6789/0 conn(0x7f7f180724d0 legacy=0x7f7f181551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1806fd30 con 0x7f7f18152b40 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7f10001670 con 0x7f7f18152b40 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 >> v1:172.21.15.17:6789/0 conn(0x7f7f18152b40 legacy=0x7f7f180c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f000043b0 0x7f7f00004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 0x7f7f0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f18072d70 con 0x7f7f00005120 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0000c060 con 0x7f7f000043b0 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0000c260 con 0x7f7f000039b0 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f177fe640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 0x7f7f0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.162+0000 7f7f167fc640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f000043b0 0x7f7f00004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 -- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f000039b0 msgr2=0x7f7f00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f000039b0 0x7f7f00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 -- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f000043b0 msgr2=0x7f7f00004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f000043b0 0x7f7f00004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 -- 172.21.15.17:0/139111201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00003520 con 0x7f7f00005120 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 -- 172.21.15.17:0/139111201 reap_dead start 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.163+0000 7f7f177fe640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 0x7f7f0000b9c0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fc0 tx=0x7f7f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d2cc06375a1cb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08009c10 con 0x7f7f00005120 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0802cca0 con 0x7f7f00005120 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f15ffb640 1 -- 172.21.15.17:0/139111201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08003bd0 con 0x7f7f00005120 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 -- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 msgr2=0x7f7f0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 0x7f7f0000b9c0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009fc0 tx=0x7f7f00005830 comp rx=0 tx=0).stop 2024-03-06T07:26:45.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 -- 172.21.15.17:0/139111201 shutdown_connections 2024-03-06T07:26:45.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/139111201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f00005120 0x7f7f0000b9c0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 -- 172.21.15.17:0/139111201 >> 172.21.15.17:0/139111201 conn(0x7f7f1806b970 msgr2=0x7f7f180aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 -- 172.21.15.17:0/139111201 shutdown_connections 2024-03-06T07:26:45.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.164+0000 7f7f1d89d640 1 -- 172.21.15.17:0/139111201 wait complete. 2024-03-06T07:26:45.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.165+0000 7f7f1d89d640 1 Processor -- start 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 -- start start 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f00005120 0x7f7f180a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f180acd00 0x7f7f180ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180c8810 con 0x7f7f181eaec0 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f18070490 con 0x7f7f00005120 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f1d89d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f180c8990 con 0x7f7f180acd00 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56836/0 (socket says 172.21.15.17:56836) 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f167fc640 1 -- 172.21.15.17:0/733005728 learned_addr learned my addr 172.21.15.17:0/733005728 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f177fe640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f180acd00 0x7f7f180ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.166+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f00005120 0x7f7f180a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 -- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f180acd00 msgr2=0x7f7f180ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f180acd00 0x7f7f180ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 -- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f00005120 msgr2=0x7f7f180a8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f00005120 0x7f7f180a8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 -- 172.21.15.17:0/733005728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f180a9ce0 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f167fc640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00f7c0 tx=0x7f7f0c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7efffff640 1 -- 172.21.15.17:0/733005728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00fe70 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7efffff640 1 -- 172.21.15.17:0/733005728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c018070 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7efffff640 1 -- 172.21.15.17:0/733005728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00d730 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f180a9fc0 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.167+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f181eb340 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.169+0000 7f7efffff640 1 -- 172.21.15.17:0/733005728 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f0c00d8d0 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.169+0000 7f7efffff640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 0x7f7f0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.169+0000 7f7efffff640 1 -- 172.21.15.17:0/733005728 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f0c08a2f0 con 0x7f7f181eaec0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.169+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 0x7f7f0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.170+0000 7f7f20563740 1 Processor -- start 2024-03-06T07:26:45.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.170+0000 7f7f20563740 1 -- start start 2024-03-06T07:26:45.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.170+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 0x7f7f0406d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7f10007f30 tx=0x7f7f10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 0x55765b4b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 0x55765b4b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b41e6b0 con 0x55765b4b9f50 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b3c7a60 con 0x55765b4b84b0 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55765b41a750 con 0x55765b4ba330 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56838/0 (socket says 172.21.15.17:56838) 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 -- 172.21.15.17:0/2486196666 learned_addr learned my addr 172.21.15.17:0/2486196666 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f177fe640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 0x55765b4b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f167fc640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 0x55765b4b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 -- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 msgr2=0x55765b4b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 0x55765b4b7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 -- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 msgr2=0x55765b4b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 0x55765b4b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 -- 172.21.15.17:0/2486196666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765b3e5f30 con 0x55765b4b9f50 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f177fe640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 0x55765b4b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f167fc640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 0x55765b4b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:45.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7f100017e0 tx=0x7f7f10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46affe5f7e0bb536 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7efd7fa640 1 -- 172.21.15.17:0/2486196666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10015bd0 con 0x55765b4b9f50 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7efd7fa640 1 -- 172.21.15.17:0/2486196666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10015d70 con 0x55765b4b9f50 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.171+0000 7f7efd7fa640 1 -- 172.21.15.17:0/2486196666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10023620 con 0x55765b4b9f50 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 -- 172.21.15.17:0/2486196666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 msgr2=0x55765b4b7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f7f100017e0 tx=0x7f7f10014000 comp rx=0 tx=0).stop 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 -- 172.21.15.17:0/2486196666 shutdown_connections 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b84b0 0x55765b4b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4ba330 0x55765b4b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 --2- 172.21.15.17:0/2486196666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4b7a30 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 -- 172.21.15.17:0/2486196666 >> 172.21.15.17:0/2486196666 conn(0x55765b4b67d0 msgr2=0x55765b4db3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 -- 172.21.15.17:0/2486196666 shutdown_connections 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.172+0000 7f7f20563740 1 -- 172.21.15.17:0/2486196666 wait complete. 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.173+0000 7f7f20563740 1 -- start start 2024-03-06T07:26:45.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.173+0000 7f7f20563740 1 Processor -- start 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 0x55765b4e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b4dc230 con 0x55765b4ba330 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b3daea0 con 0x55765b4b9f50 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55765b4dbc70 con 0x55765b4defa0 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 10 client.0 _ll_register_callbacks cb 0x55765b3da870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56842/0 (socket says 172.21.15.17:56842) 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 -- 172.21.15.17:0/1739169434 learned_addr learned my addr 172.21.15.17:0/1739169434 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f177fe640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 0x55765b4e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 -- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 msgr2=0x55765b4d6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4d6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 -- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 msgr2=0x55765b4e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 0x55765b4e0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 -- 172.21.15.17:0/1739169434 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765b3db270 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 0x55765b4e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f167fc640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c088630 tx=0x7f7f0c0590a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c5c3edf66dd117 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7ef67fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c070020 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f0c08a070 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f10015d70 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.174+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f0c075050 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 10 client.9579 fetch_fsmap learned FSMap version 160 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=2+}) v3 -- 0x55765b1332d0 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f10015d70 con 0x55765b4ba330 2024-03-06T07:26:45.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 160) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7f0c0772e0 con 0x55765b4ba330 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 10 client.9579 fetch_fsmap finished waiting for FSMap version 160 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 10 client.9579 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 20 client.9579 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b4dcee0 0x55765b515470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 4 client.9579 mds_command: new command op to 9432 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 --> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55765b42bbf0 con 0x55765b4dcee0 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b4dcee0 0x55765b515470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b4dcee0 0x55765b515470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c734107ab97ecfbb server_cookie=4dc2964c8d29edf9 in_seq=0 out_seq=0 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.175+0000 7f7ef67fc640 10 client.9579 ms_handle_connect on v2:172.21.15.17:6834/1264571498 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7ef67fc640 1 -- 172.21.15.17:0/1739169434 <== mds.0 v2:172.21.15.17:6834/1264571498 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55765b42bbf0 con 0x55765b4dcee0 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7ef67fc640 10 client.9579 handle_command_reply: tid=0 2024-03-06T07:26:45.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 client.9579 shutdown 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 msgr2=0x55765b509e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c088630 tx=0x7f7f0c0590a0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 shutdown_connections 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b4dcee0 0x55765b515470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4d6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b509e30 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7ef67fc640 0 client.9579 ms_handle_reset on v2:172.21.15.17:6834/1264571498 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 --2- 172.21.15.17:0/1739169434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4b9f50 0x55765b4e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 >> 172.21.15.17:0/1739169434 conn(0x55765b4b84b0 msgr2=0x55765b4db0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 shutdown_connections 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 1 -- 172.21.15.17:0/1739169434 wait complete. 2024-03-06T07:26:45.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.176+0000 7f7f20563740 20 client.9579 trim_cache size 0 max 16384 2024-03-06T07:26:45.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.193+0000 7f7f20563740 1 Processor -- start 2024-03-06T07:26:45.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.193+0000 7f7f20563740 1 -- start start 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4ba330 0x55765b4de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f181f7230 con 0x55765b4b9f50 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f181ee7e0 con 0x55765b4ba330 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f181f6c30 con 0x55765b4defa0 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56856/0 (socket says 172.21.15.17:56856) 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 -- 172.21.15.17:0/1517099246 learned_addr learned my addr 172.21.15.17:0/1517099246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f177fe640 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f167fc640 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4ba330 0x55765b4de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 -- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 msgr2=0x55765b4daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 -- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4ba330 msgr2=0x55765b4de1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4ba330 0x55765b4de1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 -- 172.21.15.17:0/1517099246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55765b1fd6b0 con 0x55765b4b9f50 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7f10018000 tx=0x7f7f10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775485213e9062d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.194+0000 7f7efcff9640 1 -- 172.21.15.17:0/1517099246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10015d30 con 0x55765b4b9f50 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7efcff9640 1 -- 172.21.15.17:0/1517099246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100362a0 con 0x55765b4b9f50 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7efcff9640 1 -- 172.21.15.17:0/1517099246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10036720 con 0x55765b4b9f50 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 -- 172.21.15.17:0/1517099246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 msgr2=0x55765b4f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7f10018000 tx=0x7f7f10016000 comp rx=0 tx=0).stop 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 -- 172.21.15.17:0/1517099246 shutdown_connections 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4defa0 0x55765b4daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4ba330 0x55765b4de1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 --2- 172.21.15.17:0/1517099246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4b9f50 0x55765b4f3920 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 -- 172.21.15.17:0/1517099246 >> 172.21.15.17:0/1517099246 conn(0x55765b4b84b0 msgr2=0x55765b503f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 -- 172.21.15.17:0/1517099246 shutdown_connections 2024-03-06T07:26:45.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.195+0000 7f7f20563740 1 -- 172.21.15.17:0/1517099246 wait complete. 2024-03-06T07:26:45.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.196+0000 7f7f20563740 1 -- start start 2024-03-06T07:26:45.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.196+0000 7f7f20563740 1 Processor -- start 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.196+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4b9f50 0x55765b5660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.196+0000 7f7f20563740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4defa0 0x55765b4d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b4f6d30 con 0x55765b4ba330 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55765b4ddf10 con 0x55765b4defa0 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55765b50a370 con 0x55765b4b9f50 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 10 client.0 _ll_register_callbacks cb 0x7f7f0c058240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56870/0 (socket says 172.21.15.17:56870) 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 -- 172.21.15.17:0/2810746884 learned_addr learned my addr 172.21.15.17:0/2810746884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f177fe640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4defa0 0x55765b4d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4b9f50 0x55765b5660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 -- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4b9f50 msgr2=0x55765b5660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4b9f50 0x55765b5660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 -- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4defa0 msgr2=0x55765b4d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4defa0 0x55765b4d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 -- 172.21.15.17:0/2810746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55765b4b3980 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f167fc640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c08a2c0 tx=0x7f7f0c088560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0563d28ba3cbcfc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7ef67fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c070040 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f0c08a070 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.197+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f100362a0 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f0c058420 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 10 client.9585 fetch_fsmap learned FSMap version 160 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=2+}) v3 -- 0x55765b566750 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f100362a0 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 160) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f7f0c00dbc0 con 0x55765b4ba330 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 10 client.9585 fetch_fsmap finished waiting for FSMap version 160 2024-03-06T07:26:45.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 10 client.9585 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 20 client.9585 populate_metadata read hostname 'smithi017' 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b563aa0 0x55765b4d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 4 client.9585 mds_command: new command op to 9432 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 --> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55765b4dd630 con 0x55765b563aa0 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b563aa0 0x55765b4d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7f16ffd640 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b563aa0 0x55765b4d2fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c618d299f70b180 server_cookie=83cd2c7597114d34 in_seq=0 out_seq=0 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.198+0000 7f7ef67fc640 10 client.9585 ms_handle_connect on v2:172.21.15.17:6834/1264571498 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7ef67fc640 1 -- 172.21.15.17:0/2810746884 <== mds.0 v2:172.21.15.17:6834/1264571498 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55765b4dd630 con 0x55765b563aa0 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7ef67fc640 10 client.9585 handle_command_reply: tid=0 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 client.9585 shutdown 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 msgr2=0x55765b4de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c08a2c0 tx=0x7f7f0c088560 comp rx=0 tx=0).stop 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 shutdown_connections 2024-03-06T07:26:45.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:6834/1264571498,v1:172.21.15.17:6835/1264571498] conn(0x55765b563aa0 0x55765b4d2fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55765b4defa0 0x55765b4d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55765b4ba330 0x55765b4de090 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7ef67fc640 0 client.9585 ms_handle_reset on v2:172.21.15.17:6834/1264571498 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 --2- 172.21.15.17:0/2810746884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55765b4b9f50 0x55765b5660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 >> 172.21.15.17:0/2810746884 conn(0x55765b4b84b0 msgr2=0x55765b4afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 shutdown_connections 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 1 -- 172.21.15.17:0/2810746884 wait complete. 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f20563740 20 client.9585 trim_cache size 0 max 16384 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:26:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9432, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 5.943192829, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 160, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 172, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 172, 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11.029772726999999 2024-03-06T07:26:45.207 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 msgr2=0x7f7f0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 0x7f7f0406d9c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7f10007f30 tx=0x7f7f10009040 comp rx=0 tx=0).stop 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 msgr2=0x7f7f18156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00f7c0 tx=0x7f7f0c00fc80 comp rx=0 tx=0).stop 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 shutdown_connections 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f0406b500 0x7f7f0406d9c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f180acd00 0x7f7f180ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f181eaec0 0x7f7f18156970 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 --2- 172.21.15.17:0/733005728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f00005120 0x7f7f180a8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.199+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 >> 172.21.15.17:0/733005728 conn(0x7f7f1806b970 msgr2=0x7f7f1806e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.200+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 shutdown_connections 2024-03-06T07:26:45.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.200+0000 7f7f1d89d640 1 -- 172.21.15.17:0/733005728 wait complete. 2024-03-06T07:26:45.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:26:45.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.315+0000 7f50939bf640 1 Processor -- start 2024-03-06T07:26:45.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.315+0000 7f50939bf640 1 -- start start 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f50939bf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f50939bf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f508c06dca0 0x7f508c06f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f50939bf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0c2d80 con 0x7f508c155520 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f50939bf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0c2b30 con 0x7f508c06dca0 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f50939bf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c3d70 con 0x7f508c14f580 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5090f5a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f508c06dca0 0x7f508c06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5091f5c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f508c155520 0x7f508c1559b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45804/0 (socket says 172.21.15.17:45804) 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5091f5c640 1 -- 172.21.15.17:0/142273636 learned_addr learned my addr 172.21.15.17:0/142273636 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5090f5a640 1 -- 172.21.15.17:0/142273636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 msgr2=0x7f508c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f509175b640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f508bfff640 1 -- 172.21.15.17:0/142273636 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664388077 0 0) 0x7f508c0c2d80 con 0x7f508c155520 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5090f5a640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5090f5a640 1 -- 172.21.15.17:0/142273636 >> v1:172.21.15.17:6789/0 conn(0x7f508c155520 legacy=0x7f508c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f5090f5a640 1 -- 172.21.15.17:0/142273636 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0c3a00 con 0x7f508c06dca0 2024-03-06T07:26:45.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.316+0000 7f509175b640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.317+0000 7f5090f5a640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c06dca0 0x7f508c06f700 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f508000ba40 tx=0x7f508002df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71316f1304605e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.317+0000 7f508bfff640 1 -- 172.21.15.17:0/142273636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508003b070 con 0x7f508c06dca0 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.317+0000 7f508bfff640 1 -- 172.21.15.17:0/142273636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508002ed30 con 0x7f508c06dca0 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.317+0000 7f508bfff640 1 -- 172.21.15.17:0/142273636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080036660 con 0x7f508c06dca0 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 -- 172.21.15.17:0/142273636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c06dca0 msgr2=0x7f508c06f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c06dca0 0x7f508c06f700 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f508000ba40 tx=0x7f508002df80 comp rx=0 tx=0).stop 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 -- 172.21.15.17:0/142273636 shutdown_connections 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c06dca0 0x7f508c06f700 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 --2- 172.21.15.17:0/142273636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 -- 172.21.15.17:0/142273636 >> 172.21.15.17:0/142273636 conn(0x7f508c0a94d0 msgr2=0x7f508c0a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 -- 172.21.15.17:0/142273636 shutdown_connections 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.318+0000 7f50939bf640 1 -- 172.21.15.17:0/142273636 wait complete. 2024-03-06T07:26:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.319+0000 7f50939bf640 1 Processor -- start 2024-03-06T07:26:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.319+0000 7f50939bf640 1 -- start start 2024-03-06T07:26:45.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f508c06dca0 0x7f508c1e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c3060 con 0x7f508c155520 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c14f960 con 0x7f508c06dca0 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c06fc40 con 0x7f508c14f580 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56892/0 (socket says 172.21.15.17:56892) 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 -- 172.21.15.17:0/3895607408 learned_addr learned my addr 172.21.15.17:0/3895607408 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5090f5a640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f509175b640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f508c06dca0 0x7f508c1e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 -- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 msgr2=0x7f508c152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 -- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f508c06dca0 msgr2=0x7f508c1e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f508c06dca0 0x7f508c1e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:45.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 -- 172.21.15.17:0/3895607408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c152cc0 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5091f5c640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c700 tx=0x7f507c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c018070 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c00ce10 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0ab920 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.320+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00a800 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.321+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0ab640 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.322+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c04f330 con 0x7f508c155520 2024-03-06T07:26:45.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.322+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507c01b070 con 0x7f508c155520 2024-03-06T07:26:45.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.322+0000 7f5089ffb640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:45.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.323+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f507c089840 con 0x7f508c155520 2024-03-06T07:26:45.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.323+0000 7f509175b640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:45.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.323+0000 7f509175b640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 0x7f507806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5084008850 tx=0x7f5084008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:45.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.327+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f507c0585c0 con 0x7f508c155520 2024-03-06T07:26:45.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:45.548+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f508c0ac880 con 0x7f508c155520 2024-03-06T07:26:46.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.284+0000 7f5089ffb640 1 -- 172.21.15.17:0/3895607408 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v161) v1 ==== 125+0+0 (secure 0 0 0) 0x7f507c057ec0 con 0x7f508c155520 2024-03-06T07:26:46.286 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked down. 2024-03-06T07:26:46.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 msgr2=0x7f507806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 0x7f507806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5084008850 tx=0x7f5084008040 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 msgr2=0x7f508c14c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c700 tx=0x7f507c00cbc0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 shutdown_connections 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b4d0 0x7f507806d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f508c155520 0x7f508c14c4d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f508c14f580 0x7f508c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 --2- 172.21.15.17:0/3895607408 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f508c06dca0 0x7f508c1e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 >> 172.21.15.17:0/3895607408 conn(0x7f508c0a94d0 msgr2=0x7f508c14edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 shutdown_connections 2024-03-06T07:26:46.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.287+0000 7f50939bf640 1 -- 172.21.15.17:0/3895607408 wait complete. 2024-03-06T07:26:46.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:46.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.400+0000 7fee3ea78640 1 Processor -- start 2024-03-06T07:26:46.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.400+0000 7fee3ea78640 1 -- start start 2024-03-06T07:26:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.400+0000 7fee3ea78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee380a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3ea78640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee380c3280 0x7fee380c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3ea78640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee38158800 con 0x7fee381551e0 2024-03-06T07:26:46.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3ea78640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee38158ae0 con 0x7fee380c3280 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3ea78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee38158dc0 con 0x7fee3806d9f0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3d015640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fee381551e0 0x7fee38155610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45818/0 (socket says 172.21.15.17:45818) 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3d015640 1 -- 172.21.15.17:0/1101607714 learned_addr learned my addr 172.21.15.17:0/1101607714 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3c814640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee380a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee37fff640 1 --2- 172.21.15.17:0/1101607714 >> v2:172.21.15.163:3300/0 conn(0x7fee380c3280 0x7fee380c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190719729 0 0) 0x7fee38158800 con 0x7fee381551e0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee28003520 con 0x7fee381551e0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2467454638 0 0) 0x7fee28003520 con 0x7fee381551e0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 msgr2=0x7fee380a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee380a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 >> v2:172.21.15.163:3300/0 conn(0x7fee380c3280 msgr2=0x7fee380c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 --2- 172.21.15.17:0/1101607714 >> v2:172.21.15.163:3300/0 conn(0x7fee380c3280 0x7fee380c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee381590a0 con 0x7fee381551e0 2024-03-06T07:26:46.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fee300019f0 con 0x7fee381551e0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 >> v1:172.21.15.17:6789/0 conn(0x7fee381551e0 legacy=0x7fee38155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee280039b0 0x7fee28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38158ae0 con 0x7fee28004d40 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee38158dc0 con 0x7fee280039b0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2800b9f0 con 0x7fee3806d9f0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.401+0000 7fee3d015640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3c814640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee280039b0 0x7fee28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee37fff640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 -- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 msgr2=0x7fee28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 -- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee280039b0 msgr2=0x7fee28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee280039b0 0x7fee28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 -- 172.21.15.17:0/1101607714 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28003520 con 0x7fee28004d40 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee37fff640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee3806d9f0 0x7fee28004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.402+0000 7fee3d015640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee280051c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee3000a7a0 tx=0x7fee28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75584318160f82d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000a3f0 con 0x7fee28004d40 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee30034630 con 0x7fee28004d40 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee377fe640 1 -- 172.21.15.17:0/1101607714 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30034a40 con 0x7fee28004d40 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 -- 172.21.15.17:0/1101607714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 msgr2=0x7fee280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1101607714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee280051c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee3000a7a0 tx=0x7fee28005830 comp rx=0 tx=0).stop 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3d015640 1 -- 172.21.15.17:0/1101607714 reap_dead start 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 -- 172.21.15.17:0/1101607714 shutdown_connections 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 -- 172.21.15.17:0/1101607714 >> 172.21.15.17:0/1101607714 conn(0x7fee380aa630 msgr2=0x7fee380aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 -- 172.21.15.17:0/1101607714 shutdown_connections 2024-03-06T07:26:46.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.403+0000 7fee3ea78640 1 -- 172.21.15.17:0/1101607714 wait complete. 2024-03-06T07:26:46.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.404+0000 7fee3ea78640 1 Processor -- start 2024-03-06T07:26:46.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.404+0000 7fee3ea78640 1 -- start start 2024-03-06T07:26:46.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 0x7fee3806f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 0x7fee3806f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3806df50 con 0x7fee28004d40 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3806ddd0 con 0x7fee3809b4f0 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3ea78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee38155d20 con 0x7fee381edd30 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56912/0 (socket says 172.21.15.17:56912) 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 -- 172.21.15.17:0/1232099260 learned_addr learned my addr 172.21.15.17:0/1232099260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 -- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 msgr2=0x7fee3806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3d015640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 0x7fee3806f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 0x7fee3806f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee37fff640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 0x7fee3806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 -- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 msgr2=0x7fee3806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 0x7fee3806f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3c814640 1 -- 172.21.15.17:0/1232099260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3809b9b0 con 0x7fee28004d40 2024-03-06T07:26:46.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee3d015640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 0x7fee3806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.405+0000 7fee37fff640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 0x7fee3806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee3c814640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0118e0 tx=0x7fee2c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00faf0 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00fc90 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3806e0d0 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c014440 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.406+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3806e690 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.407+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3804f330 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.407+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee2c012040 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.408+0000 7fee357fa640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.408+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee2c08aa40 con 0x7fee28004d40 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.408+0000 7fee37fff640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:46.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.409+0000 7fee37fff640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 0x7fee1c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fee381eee50 tx=0x7fee24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:46.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.412+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fee2c019030 con 0x7fee28004d40 2024-03-06T07:26:46.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.630+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee38073240 con 0x7fee28004d40 2024-03-06T07:26:46.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.630+0000 7fee357fa640 1 -- 172.21.15.17:0/1232099260 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fee2c0597c0 con 0x7fee28004d40 2024-03-06T07:26:46.632 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:46.633 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:46.633 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 msgr2=0x7fee1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 0x7fee1c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fee381eee50 tx=0x7fee24008040 comp rx=0 tx=0).stop 2024-03-06T07:26:46.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 msgr2=0x7fee380abaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0118e0 tx=0x7fee2c011da0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 shutdown_connections 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee1c06b4d0 0x7fee1c06d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee381edd30 0x7fee3806f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee3809b4f0 0x7fee3806f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 --2- 172.21.15.17:0/1232099260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee28004d40 0x7fee380abaa0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 >> 172.21.15.17:0/1232099260 conn(0x7fee380aa630 msgr2=0x7fee380c7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 shutdown_connections 2024-03-06T07:26:46.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:46.633+0000 7fee3ea78640 1 -- 172.21.15.17:0/1232099260 wait complete. 2024-03-06T07:26:47.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:47.047 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:48.048 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.161+0000 7f50362d4640 1 Processor -- start 2024-03-06T07:26:48.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.161+0000 7f50362d4640 1 -- start start 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f50362d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 0x7f5030155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f50362d4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50300c6780 0x7f5030158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f50362d4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5030156050 con 0x7f50301592d0 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f50362d4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503006de20 con 0x7f50300c6780 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f50362d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5030072630 con 0x7f503006da40 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50300c6780 0x7f5030158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f5034871640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50301592d0 0x7f50300c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45834/0 (socket says 172.21.15.17:45834) 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f5034871640 1 -- 172.21.15.17:0/3022506116 learned_addr learned my addr 172.21.15.17:0/3022506116 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502ffff640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 0x7f5030155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502f7fe640 1 -- 172.21.15.17:0/3022506116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 msgr2=0x7f5030155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502f7fe640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 0x7f5030155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502f7fe640 1 -- 172.21.15.17:0/3022506116 >> v1:172.21.15.17:6789/0 conn(0x7f50301592d0 legacy=0x7f50300c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502f7fe640 1 -- 172.21.15.17:0/3022506116 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030072910 con 0x7f50300c6780 2024-03-06T07:26:48.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.162+0000 7f502ffff640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 0x7f5030155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.163+0000 7f502f7fe640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50300c6780 0x7f5030158d90 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3982000425970d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.163+0000 7f502effd640 1 -- 172.21.15.17:0/3022506116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200315a0 con 0x7f50300c6780 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.163+0000 7f502effd640 1 -- 172.21.15.17:0/3022506116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020037070 con 0x7f50300c6780 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.163+0000 7f502effd640 1 -- 172.21.15.17:0/3022506116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020031960 con 0x7f50300c6780 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 -- 172.21.15.17:0/3022506116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50300c6780 msgr2=0x7f5030158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50300c6780 0x7f5030158d90 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f50200097a0 tx=0x7f502002b840 comp rx=0 tx=0).stop 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 -- 172.21.15.17:0/3022506116 shutdown_connections 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50300c6780 0x7f5030158d90 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 --2- 172.21.15.17:0/3022506116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f503006da40 0x7f5030155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 -- 172.21.15.17:0/3022506116 >> 172.21.15.17:0/3022506116 conn(0x7f50300aa630 msgr2=0x7f50300aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 -- 172.21.15.17:0/3022506116 shutdown_connections 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.164+0000 7f50362d4640 1 -- 172.21.15.17:0/3022506116 wait complete. 2024-03-06T07:26:48.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.165+0000 7f50362d4640 1 Processor -- start 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.165+0000 7f50362d4640 1 -- start start 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 0x7f503014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 0x7f503014c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300c3990 con 0x7f503006da40 2024-03-06T07:26:48.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5030158890 con 0x7f50300c6780 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f50362d4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50300c3b10 con 0x7f50301592d0 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56926/0 (socket says 172.21.15.17:56926) 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 -- 172.21.15.17:0/2373888201 learned_addr learned my addr 172.21.15.17:0/2373888201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f5034871640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 0x7f503014c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502f7fe640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 0x7f503014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 -- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 msgr2=0x7f503014c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 0x7f503014c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 -- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 msgr2=0x7f503014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 0x7f503014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 -- 172.21.15.17:0/2373888201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503014fa30 con 0x7f503006da40 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f5034871640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 0x7f503014c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502f7fe640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 0x7f503014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:48.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.166+0000 7f502ffff640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f502400c7f0 tx=0x7f502400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.167+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024015070 con 0x7f503006da40 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.167+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50240077c0 con 0x7f503006da40 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.167+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503014f750 con 0x7f503006da40 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.167+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024007b70 con 0x7f503006da40 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.167+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50301eb830 con 0x7f503006da40 2024-03-06T07:26:48.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.168+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503004f330 con 0x7f503006da40 2024-03-06T07:26:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.168+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5024018070 con 0x7f503006da40 2024-03-06T07:26:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.169+0000 7f502cff9640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 0x7f501c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.169+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f502408a370 con 0x7f503006da40 2024-03-06T07:26:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.169+0000 7f502f7fe640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 0x7f501c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:48.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.170+0000 7f502f7fe640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 0x7f501c06d9c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5020036000 tx=0x7f502003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:48.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.173+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50240590f0 con 0x7f503006da40 2024-03-06T07:26:48.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.396+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5030073240 con 0x7f503006da40 2024-03-06T07:26:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.397+0000 7f502cff9640 1 -- 172.21.15.17:0/2373888201 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f50240589f0 con 0x7f503006da40 2024-03-06T07:26:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:48.399 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 msgr2=0x7f501c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 0x7f501c06d9c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5020036000 tx=0x7f502003d040 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 msgr2=0x7f5030151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f502400c7f0 tx=0x7f502400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 shutdown_connections 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f501c06b500 0x7f501c06d9c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50301592d0 0x7f503014c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50300c6780 0x7f503014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 --2- 172.21.15.17:0/2373888201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f503006da40 0x7f5030151fc0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 >> 172.21.15.17:0/2373888201 conn(0x7f50300aa630 msgr2=0x7f50300c5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 shutdown_connections 2024-03-06T07:26:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:48.399+0000 7f50362d4640 1 -- 172.21.15.17:0/2373888201 wait complete. 2024-03-06T07:26:48.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:48.814 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:49.814 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.926+0000 7f3a40647640 1 Processor -- start 2024-03-06T07:26:49.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.926+0000 7f3a40647640 1 -- start start 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a3806afc0 0x7f3a3806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c20b0 0x7f3a38153180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3a380c25c0 0x7f3a381536c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380a9530 con 0x7f3a380c25c0 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380a9810 con 0x7f3a3806afc0 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a40647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380a9af0 con 0x7f3a380c20b0 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a3ebe4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3a380c25c0 0x7f3a381536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a3ebe4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3a380c25c0 0x7f3a381536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56930/0 (socket says 172.21.15.17:56930) 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a3ebe4640 1 -- 172.21.15.17:0/859677761 learned_addr learned my addr 172.21.15.17:0/859677761 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a3e3e3640 1 --2- 172.21.15.17:0/859677761 >> v2:172.21.15.163:3300/0 conn(0x7f3a3806afc0 0x7f3a3806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.927+0000 7f3a3dbe2640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c20b0 0x7f3a38153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 -- 172.21.15.17:0/859677761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c20b0 msgr2=0x7f3a38153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c20b0 0x7f3a38153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 -- 172.21.15.17:0/859677761 >> v2:172.21.15.163:3300/0 conn(0x7f3a3806afc0 msgr2=0x7f3a3806b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:49.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 --2- 172.21.15.17:0/859677761 >> v2:172.21.15.163:3300/0 conn(0x7f3a3806afc0 0x7f3a3806b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 -- 172.21.15.17:0/859677761 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380a9dd0 con 0x7f3a380c25c0 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3ebe4640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a380c25c0 0x7f3a381536c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f3a380c5180 tx=0x7f3a3402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5718004a754a9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.928+0000 7f3a3d3e1640 1 -- 172.21.15.17:0/859677761 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3402c950 con 0x7f3a380c25c0 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a3d3e1640 1 -- 172.21.15.17:0/859677761 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3402caf0 con 0x7f3a380c25c0 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a3d3e1640 1 -- 172.21.15.17:0/859677761 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a340343f0 con 0x7f3a380c25c0 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 -- 172.21.15.17:0/859677761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a380c25c0 msgr2=0x7f3a381536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a380c25c0 0x7f3a381536c0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f3a380c5180 tx=0x7f3a3402bdb0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 -- 172.21.15.17:0/859677761 shutdown_connections 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a380c25c0 0x7f3a381536c0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 --2- 172.21.15.17:0/859677761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c20b0 0x7f3a38153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 --2- 172.21.15.17:0/859677761 >> v2:172.21.15.163:3300/0 conn(0x7f3a3806afc0 0x7f3a3806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 -- 172.21.15.17:0/859677761 >> 172.21.15.17:0/859677761 conn(0x7f3a380c76b0 msgr2=0x7f3a380c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 -- 172.21.15.17:0/859677761 shutdown_connections 2024-03-06T07:26:49.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.929+0000 7f3a40647640 1 -- 172.21.15.17:0/859677761 wait complete. 2024-03-06T07:26:49.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.930+0000 7f3a40647640 1 Processor -- start 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.930+0000 7f3a40647640 1 -- start start 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a380c20b0 0x7f3a380c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c25c0 0x7f3a381ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38150b00 con 0x7f3a3806afc0 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38150980 con 0x7f3a380c20b0 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a40647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380a7d50 con 0x7f3a380c25c0 2024-03-06T07:26:49.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56942/0 (socket says 172.21.15.17:56942) 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 -- 172.21.15.17:0/12531133 learned_addr learned my addr 172.21.15.17:0/12531133 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 -- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c25c0 msgr2=0x7f3a381ecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c25c0 0x7f3a381ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 -- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a380c20b0 msgr2=0x7f3a380c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a380c20b0 0x7f3a380c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.931+0000 7f3a3e3e3640 1 -- 172.21.15.17:0/12531133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3814b840 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a3e3e3640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d5f0 tx=0x7f3a3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300125f0 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30019070 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300129d0 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a381df950 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.932+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a381dff10 con 0x7f3a3806afc0 2024-03-06T07:26:49.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.933+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3804f330 con 0x7f3a3806afc0 2024-03-06T07:26:49.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.933+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a30012b70 con 0x7f3a3806afc0 2024-03-06T07:26:49.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.934+0000 7f3a2effd640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:49.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.934+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a3008a710 con 0x7f3a3806afc0 2024-03-06T07:26:49.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.934+0000 7f3a3dbe2640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:49.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.934+0000 7f3a3dbe2640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3a380c7610 tx=0x7f3a28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:49.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:49.938+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a30059490 con 0x7f3a3806afc0 2024-03-06T07:26:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.156+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a381e0110 con 0x7f3a3806afc0 2024-03-06T07:26:50.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.157+0000 7f3a2effd640 1 -- 172.21.15.17:0/12531133 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f3a30058d90 con 0x7f3a3806afc0 2024-03-06T07:26:50.158 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:50.159 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:50.159 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 msgr2=0x7f3a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3a380c7610 tx=0x7f3a28008040 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 msgr2=0x7f3a381e9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000d5f0 tx=0x7f3a3000dab0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 shutdown_connections 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a380c25c0 0x7f3a381ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a380c20b0 0x7f3a380c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 --2- 172.21.15.17:0/12531133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a3806afc0 0x7f3a381e9100 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 >> 172.21.15.17:0/12531133 conn(0x7f3a380c76b0 msgr2=0x7f3a38149090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 shutdown_connections 2024-03-06T07:26:50.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:50.159+0000 7f3a40647640 1 -- 172.21.15.17:0/12531133 wait complete. 2024-03-06T07:26:50.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:50.574 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:51.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:51.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.687+0000 7f9462f1e640 1 Processor -- start 2024-03-06T07:26:51.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.687+0000 7f9462f1e640 1 -- start start 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.687+0000 7f9462f1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f945c06f7b0 0x7f945c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f9462f1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c1506f0 0x7f945c0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f9462f1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c157340 con 0x7f945c072720 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f9462f1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c156b40 con 0x7f945c06f7b0 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f9462f1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0c3740 con 0x7f945c1506f0 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f945c06f7b0 0x7f945c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f945c06f7b0 0x7f945c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53288/0 (socket says 172.21.15.17:53288) 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 -- 172.21.15.17:0/3446782195 learned_addr learned my addr 172.21.15.17:0/3446782195 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f94614bb640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c1506f0 0x7f945c0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 -- 172.21.15.17:0/3446782195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c1506f0 msgr2=0x7f945c0c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:51.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c1506f0 0x7f945c0c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 -- 172.21.15.17:0/3446782195 >> v1:172.21.15.163:6789/0 conn(0x7f945c072720 legacy=0x7f945c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:51.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.688+0000 7f945bfff640 1 -- 172.21.15.17:0/3446782195 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c156d80 con 0x7f945c06f7b0 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f945bfff640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c06f7b0 0x7f945c1501b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f944802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91fa92016b4fdeb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f945b7fe640 1 -- 172.21.15.17:0/3446782195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448035020 con 0x7f945c06f7b0 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f945b7fe640 1 -- 172.21.15.17:0/3446782195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448031640 con 0x7f945c06f7b0 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f945b7fe640 1 -- 172.21.15.17:0/3446782195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944802d040 con 0x7f945c06f7b0 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f9462f1e640 1 -- 172.21.15.17:0/3446782195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c06f7b0 msgr2=0x7f945c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.689+0000 7f9462f1e640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c06f7b0 0x7f945c1501b0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f9448009870 tx=0x7f944802b8f0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 -- 172.21.15.17:0/3446782195 shutdown_connections 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c1506f0 0x7f945c0c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 --2- 172.21.15.17:0/3446782195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c06f7b0 0x7f945c1501b0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 -- 172.21.15.17:0/3446782195 >> 172.21.15.17:0/3446782195 conn(0x7f945c0aa630 msgr2=0x7f945c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 -- 172.21.15.17:0/3446782195 shutdown_connections 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.690+0000 7f9462f1e640 1 -- 172.21.15.17:0/3446782195 wait complete. 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.691+0000 7f9462f1e640 1 Processor -- start 2024-03-06T07:26:51.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.691+0000 7f9462f1e640 1 -- start start 2024-03-06T07:26:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c06f7b0 0x7f945c1ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f945c1506f0 0x7f945c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c072b00 con 0x7f945c072720 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0c33d0 con 0x7f945c1506f0 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9462f1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c072c80 con 0x7f945c06f7b0 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53302/0 (socket says 172.21.15.17:53302) 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 -- 172.21.15.17:0/2710622769 learned_addr learned my addr 172.21.15.17:0/2710622769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f9460cba640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c06f7b0 0x7f945c1ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f94614bb640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f945c1506f0 0x7f945c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 -- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c06f7b0 msgr2=0x7f945c1ebc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c06f7b0 0x7f945c1ebc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 -- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f945c1506f0 msgr2=0x7f945c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f945c1506f0 0x7f945c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 -- 172.21.15.17:0/2710622769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c1ec910 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f945bfff640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f944802b930 tx=0x7f94480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448040070 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.692+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944803e070 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.693+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c150f40 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.693+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448035040 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.693+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c154270 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.693+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c04f330 con 0x7f945c072720 2024-03-06T07:26:51.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.694+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9448038020 con 0x7f945c072720 2024-03-06T07:26:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.695+0000 7f94597fa640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 0x7f944006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.695+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94480abbc0 con 0x7f945c072720 2024-03-06T07:26:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.695+0000 7f9460cba640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 0x7f944006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:51.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.695+0000 7f9460cba640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 0x7f944006d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f945c0c44d0 tx=0x7f9450009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:51.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.699+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f944807a2f0 con 0x7f945c072720 2024-03-06T07:26:51.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.921+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f945c14f240 con 0x7f945c072720 2024-03-06T07:26:51.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.921+0000 7f94597fa640 1 -- 172.21.15.17:0/2710622769 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f944807e390 con 0x7f945c072720 2024-03-06T07:26:51.923 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:51.923 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:51.923 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 msgr2=0x7f944006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 0x7f944006d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f945c0c44d0 tx=0x7f9450009040 comp rx=0 tx=0).stop 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 msgr2=0x7f945c154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f944802b930 tx=0x7f94480046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 shutdown_connections 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f944006b4d0 0x7f944006d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f945c1506f0 0x7f945c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f945c072720 0x7f945c154790 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 --2- 172.21.15.17:0/2710622769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f945c06f7b0 0x7f945c1ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:51.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 >> 172.21.15.17:0/2710622769 conn(0x7f945c0aa630 msgr2=0x7f945c14e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:51.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 shutdown_connections 2024-03-06T07:26:51.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:51.923+0000 7f9462f1e640 1 -- 172.21.15.17:0/2710622769 wait complete. 2024-03-06T07:26:52.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:52.339 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:53.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:53.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.453+0000 7efea65d7640 1 Processor -- start 2024-03-06T07:26:53.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.453+0000 7efea65d7640 1 -- start start 2024-03-06T07:26:53.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efea65d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efea00c5330 0x7efea00c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efea65d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea0159630 con 0x7efea0152920 2024-03-06T07:26:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efea65d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00c7760 con 0x7efea00c8a00 2024-03-06T07:26:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efea65d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea01587b0 con 0x7efea00c5330 2024-03-06T07:26:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efe9ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efea0152920 0x7efea0155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50164/0 (socket says 172.21.15.17:50164) 2024-03-06T07:26:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efe9ffff640 1 -- 172.21.15.17:0/707014612 learned_addr learned my addr 172.21.15.17:0/707014612 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302588727 0 0) 0x7efea0159630 con 0x7efea0152920 2024-03-06T07:26:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe88003520 con 0x7efea0152920 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.454+0000 7efea4b74640 1 --2- 172.21.15.17:0/707014612 >> v2:172.21.15.163:3300/0 conn(0x7efea00c5330 0x7efea00c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637560970 0 0) 0x7efe88003520 con 0x7efea0152920 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 >> v2:172.21.15.163:3300/0 conn(0x7efea00c5330 msgr2=0x7efea00c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 --2- 172.21.15.17:0/707014612 >> v2:172.21.15.163:3300/0 conn(0x7efea00c5330 0x7efea00c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 >> v1:172.21.15.163:6789/0 conn(0x7efea00c8a00 legacy=0x7efea0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00c7a40 con 0x7efea0152920 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7efe90001670 con 0x7efea0152920 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 >> v1:172.21.15.17:6789/0 conn(0x7efea0152920 legacy=0x7efea0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe880039b0 0x7efe88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe88004800 0x7efe88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 0x7efe8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea01587b0 con 0x7efe880051c0 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8800c160 con 0x7efe88004800 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8800c360 con 0x7efe880039b0 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 0x7efe8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9ffff640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe880039b0 0x7efe88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9f7fe640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe88004800 0x7efe88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 -- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe880039b0 msgr2=0x7efe88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe880039b0 0x7efe88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 -- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe88004800 msgr2=0x7efe88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe88004800 0x7efe88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efea4b74640 1 -- 172.21.15.17:0/707014612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe88003520 con 0x7efe880051c0 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.455+0000 7efe9f7fe640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe88004800 0x7efe88004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.456+0000 7efea4b74640 1 -- 172.21.15.17:0/707014612 reap_dead start 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.456+0000 7efea4b74640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 0x7efe8800b9c0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7efe98009fc0 tx=0x7efe88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8f4ab0d2a5825f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.456+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98009c10 con 0x7efe880051c0 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.456+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9802cca0 con 0x7efe880051c0 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.456+0000 7efe9effd640 1 -- 172.21.15.17:0/707014612 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98003bd0 con 0x7efe880051c0 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 -- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 msgr2=0x7efe8800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 0x7efe8800b9c0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7efe98009fc0 tx=0x7efe88005830 comp rx=0 tx=0).stop 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 -- 172.21.15.17:0/707014612 shutdown_connections 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 --2- 172.21.15.17:0/707014612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe880051c0 0x7efe8800b9c0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 -- 172.21.15.17:0/707014612 >> 172.21.15.17:0/707014612 conn(0x7efea00ac560 msgr2=0x7efea00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 -- 172.21.15.17:0/707014612 shutdown_connections 2024-03-06T07:26:53.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.457+0000 7efea65d7640 1 -- 172.21.15.17:0/707014612 wait complete. 2024-03-06T07:26:53.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.458+0000 7efea65d7640 1 Processor -- start 2024-03-06T07:26:53.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.458+0000 7efea65d7640 1 -- start start 2024-03-06T07:26:53.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe880051c0 0x7efea014b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea0150d40 0x7efea0147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea006d030 con 0x7efea0144eb0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0158a00 con 0x7efe880051c0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea65d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c7390 con 0x7efea0150d40 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53332/0 (socket says 172.21.15.17:53332) 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 -- 172.21.15.17:0/2498005542 learned_addr learned my addr 172.21.15.17:0/2498005542 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efe9ffff640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe880051c0 0x7efea014b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efe9f7fe640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea0150d40 0x7efea0147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 -- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea0150d40 msgr2=0x7efea0147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea0150d40 0x7efea0147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 -- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe880051c0 msgr2=0x7efea014b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe880051c0 0x7efea014b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 -- 172.21.15.17:0/2498005542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00c4a30 con 0x7efea0144eb0 2024-03-06T07:26:53.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efea4b74640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7efe9802c6d0 tx=0x7efe98009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98003990 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.459+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98044070 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.460+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea00c4470 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.460+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98003dd0 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.460+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea0156910 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.461+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea004f330 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.461+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe98005d10 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.461+0000 7efe9cff9640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 0x7efe8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.461+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe980ac950 con 0x7efea0144eb0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.462+0000 7efe9ffff640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 0x7efe8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:53.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.462+0000 7efe9ffff640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 0x7efe8c06d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efe90001cd0 tx=0x7efe90001c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:53.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.466+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efe9807b1a0 con 0x7efea0144eb0 2024-03-06T07:26:53.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.688+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efea0156390 con 0x7efea0144eb0 2024-03-06T07:26:53.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.689+0000 7efe9cff9640 1 -- 172.21.15.17:0/2498005542 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7efe9807f240 con 0x7efea0144eb0 2024-03-06T07:26:53.690 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:53.691 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:53.691 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 msgr2=0x7efe8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 0x7efe8c06d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efe90001cd0 tx=0x7efe90001c60 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 msgr2=0x7efea0145330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7efe9802c6d0 tx=0x7efe98009750 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 shutdown_connections 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe8c06b500 0x7efe8c06d9c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efea0144eb0 0x7efea0145330 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea0150d40 0x7efea0147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 --2- 172.21.15.17:0/2498005542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe880051c0 0x7efea014b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 >> 172.21.15.17:0/2498005542 conn(0x7efea00ac560 msgr2=0x7efea006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 shutdown_connections 2024-03-06T07:26:53.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:53.691+0000 7efea65d7640 1 -- 172.21.15.17:0/2498005542 wait complete. 2024-03-06T07:26:54.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:54.106 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:55.107 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.219+0000 7f5155c9d640 1 Processor -- start 2024-03-06T07:26:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.219+0000 7f5155c9d640 1 -- start start 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f5155c9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5150150580 0x7f515006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f5155c9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51500c1320 0x7f515006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f5155c9d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5150153a60 con 0x7f51500c6000 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f5155c9d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5150153d40 con 0x7f51500c1320 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f5155c9d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515006c2c0 con 0x7f5150150580 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51500c1320 0x7f515006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f51500c6000 0x7f51501566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50172/0 (socket says 172.21.15.17:50172) 2024-03-06T07:26:55.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514ffff640 1 -- 172.21.15.17:0/417460053 learned_addr learned my addr 172.21.15.17:0/417460053 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514f7fe640 1 --2- 172.21.15.17:0/417460053 >> v2:172.21.15.163:3300/0 conn(0x7f5150150580 0x7f515006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514effd640 1 -- 172.21.15.17:0/417460053 >> v2:172.21.15.163:3300/0 conn(0x7f5150150580 msgr2=0x7f515006dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514effd640 1 --2- 172.21.15.17:0/417460053 >> v2:172.21.15.163:3300/0 conn(0x7f5150150580 0x7f515006dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514effd640 1 -- 172.21.15.17:0/417460053 >> v1:172.21.15.17:6789/0 conn(0x7f51500c6000 legacy=0x7f51501566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514e7fc640 1 -- 172.21.15.17:0/417460053 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786715215 0 0) 0x7f5150153a60 con 0x7f51500c6000 2024-03-06T07:26:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.220+0000 7f514effd640 1 -- 172.21.15.17:0/417460053 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515006c5a0 con 0x7f51500c1320 2024-03-06T07:26:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.221+0000 7f514effd640 1 --2- 172.21.15.17:0/417460053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c1320 0x7f515006e250 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f51500a9550 tx=0x7f514402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd9b3ef0802b6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.221+0000 7f514e7fc640 1 -- 172.21.15.17:0/417460053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440315a0 con 0x7f51500c1320 2024-03-06T07:26:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.221+0000 7f514e7fc640 1 -- 172.21.15.17:0/417460053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144037070 con 0x7f51500c1320 2024-03-06T07:26:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.221+0000 7f514e7fc640 1 -- 172.21.15.17:0/417460053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144031960 con 0x7f51500c1320 2024-03-06T07:26:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 -- 172.21.15.17:0/417460053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c1320 msgr2=0x7f515006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 --2- 172.21.15.17:0/417460053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c1320 0x7f515006e250 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f51500a9550 tx=0x7f514402b840 comp rx=0 tx=0).stop 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 -- 172.21.15.17:0/417460053 shutdown_connections 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 --2- 172.21.15.17:0/417460053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c1320 0x7f515006e250 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 --2- 172.21.15.17:0/417460053 >> v2:172.21.15.163:3300/0 conn(0x7f5150150580 0x7f515006dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 -- 172.21.15.17:0/417460053 >> 172.21.15.17:0/417460053 conn(0x7f51500c6710 msgr2=0x7f51500c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 -- 172.21.15.17:0/417460053 shutdown_connections 2024-03-06T07:26:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.222+0000 7f5155c9d640 1 -- 172.21.15.17:0/417460053 wait complete. 2024-03-06T07:26:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.223+0000 7f5155c9d640 1 Processor -- start 2024-03-06T07:26:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.223+0000 7f5155c9d640 1 -- start start 2024-03-06T07:26:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51500c1320 0x7f5150155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 0x7f5150149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51500c0dc0 con 0x7f51500c6000 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51500c0c40 con 0x7f5150150580 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f5155c9d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5150156dc0 con 0x7f51500c1320 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53356/0 (socket says 172.21.15.17:53356) 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 -- 172.21.15.17:0/59595806 learned_addr learned my addr 172.21.15.17:0/59595806 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514f7fe640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51500c1320 0x7f5150155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514ffff640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 0x7f5150149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 -- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51500c1320 msgr2=0x7f5150155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51500c1320 0x7f5150155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 -- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 msgr2=0x7f5150149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 0x7f5150149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 -- 172.21.15.17:0/59595806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150070b00 con 0x7f51500c6000 2024-03-06T07:26:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514ffff640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 0x7f5150149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.224+0000 7f514effd640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5144009890 tx=0x7f51440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.225+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144032840 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.225+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144032a00 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.225+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51500c0840 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.225+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144032df0 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.225+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5150071930 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.226+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5150067ad0 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.226+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f514404c020 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.226+0000 7f5137fff640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.227+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f514402f020 con 0x7f51500c6000 2024-03-06T07:26:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.227+0000 7f514f7fe640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:55.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.227+0000 7f514f7fe640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 0x7f513c06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:55.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.231+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5150067ad0 con 0x7f51500c6000 2024-03-06T07:26:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.454+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51500702f0 con 0x7f51500c6000 2024-03-06T07:26:55.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.455+0000 7f5137fff640 1 -- 172.21.15.17:0/59595806 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f5144040070 con 0x7f51500c6000 2024-03-06T07:26:55.456 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:55.457 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:55.457 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 msgr2=0x7f513c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 0x7f513c06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5140006fd0 tx=0x7f5140008040 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 msgr2=0x7f5150149a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5144009890 tx=0x7f51440046c0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 shutdown_connections 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5150150580 0x7f5150149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51500c6000 0x7f5150149a10 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 --2- 172.21.15.17:0/59595806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51500c1320 0x7f5150155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 >> 172.21.15.17:0/59595806 conn(0x7f51500c6710 msgr2=0x7f51500c4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:55.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 shutdown_connections 2024-03-06T07:26:55.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:55.457+0000 7f5155c9d640 1 -- 172.21.15.17:0/59595806 wait complete. 2024-03-06T07:26:55.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:55.872 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:56.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:56.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.986+0000 7fafc8381640 1 Processor -- start 2024-03-06T07:26:56.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.986+0000 7fafc8381640 1 -- start start 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc8381640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fafc00c1aa0 0x7fafc01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc8381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 0x7fafc0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc8381640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc01563d0 con 0x7fafc014f790 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc8381640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00c2680 con 0x7fafc00c1aa0 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc8381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c28d0 con 0x7fafc0155a40 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc611d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fafc014f790 0x7fafc00c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50186/0 (socket says 172.21.15.17:50186) 2024-03-06T07:26:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc611d640 1 -- 172.21.15.17:0/951039691 learned_addr learned my addr 172.21.15.17:0/951039691 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc591c640 1 --2- 172.21.15.17:0/951039691 >> v2:172.21.15.17:3300/0 conn(0x7fafc00c1aa0 0x7fafc01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc691e640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 0x7fafc0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc511b640 1 -- 172.21.15.17:0/951039691 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511008204 0 0) 0x7fafc01563d0 con 0x7fafc014f790 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc511b640 1 -- 172.21.15.17:0/951039691 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafac003520 con 0x7fafc014f790 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc591c640 1 -- 172.21.15.17:0/951039691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 msgr2=0x7fafc0155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc591c640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 0x7fafc0155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc591c640 1 -- 172.21.15.17:0/951039691 >> v1:172.21.15.17:6789/0 conn(0x7fafc014f790 legacy=0x7fafc00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc591c640 1 -- 172.21.15.17:0/951039691 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00c2bb0 con 0x7fafc00c1aa0 2024-03-06T07:26:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.987+0000 7fafc691e640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 0x7fafc0155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:56.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.988+0000 7fafc591c640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00c1aa0 0x7fafc01529d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7b92698ba6fec0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.988+0000 7fafc511b640 1 -- 172.21.15.17:0/951039691 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0035020 con 0x7fafc00c1aa0 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.988+0000 7fafc511b640 1 -- 172.21.15.17:0/951039691 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0031830 con 0x7fafc00c1aa0 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.988+0000 7fafc511b640 1 -- 172.21.15.17:0/951039691 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0031be0 con 0x7fafc00c1aa0 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 -- 172.21.15.17:0/951039691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00c1aa0 msgr2=0x7fafc01529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00c1aa0 0x7fafc01529d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 -- 172.21.15.17:0/951039691 shutdown_connections 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc0155a40 0x7fafc0155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 --2- 172.21.15.17:0/951039691 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc00c1aa0 0x7fafc01529d0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 -- 172.21.15.17:0/951039691 >> 172.21.15.17:0/951039691 conn(0x7fafc00c6710 msgr2=0x7fafc00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 -- 172.21.15.17:0/951039691 shutdown_connections 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.989+0000 7fafc8381640 1 -- 172.21.15.17:0/951039691 wait complete. 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.990+0000 7fafc8381640 1 Processor -- start 2024-03-06T07:26:56.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.990+0000 7fafc8381640 1 -- start start 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 0x7fafc01deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 0x7fafc01ec260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c0d60 con 0x7fafc0155a40 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0155520 con 0x7fafc00c1aa0 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc8381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c5e30 con 0x7fafc014f790 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53376/0 (socket says 172.21.15.17:53376) 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 -- 172.21.15.17:0/1107501743 learned_addr learned my addr 172.21.15.17:0/1107501743 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc611d640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 0x7fafc01deeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc591c640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 0x7fafc01ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 -- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 msgr2=0x7fafc01ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 0x7fafc01ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 -- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 msgr2=0x7fafc01deeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 0x7fafc01deeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc691e640 1 -- 172.21.15.17:0/1107501743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0072970 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc611d640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 0x7fafc01deeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.991+0000 7fafc591c640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 0x7fafc01ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafc691e640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007df0 tx=0x7fafb800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801a070 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8003990 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc0072c10 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8003d80 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.992+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00705d0 con 0x7fafc0155a40 2024-03-06T07:26:56.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.993+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc004f330 con 0x7fafc0155a40 2024-03-06T07:26:56.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.993+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafb801d070 con 0x7fafc0155a40 2024-03-06T07:26:56.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.994+0000 7fafb6ffd640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 0x7fafa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:56.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.994+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fafb808a5d0 con 0x7fafc0155a40 2024-03-06T07:26:56.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.994+0000 7fafc611d640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 0x7fafa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:56.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.995+0000 7fafc611d640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 0x7fafa406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fafbc007fd0 tx=0x7fafbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:56.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:56.998+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fafb8010070 con 0x7fafc0155a40 2024-03-06T07:26:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.217+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafc00a5f60 con 0x7fafc0155a40 2024-03-06T07:26:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.217+0000 7fafb6ffd640 1 -- 172.21.15.17:0/1107501743 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7fafb8059300 con 0x7fafc0155a40 2024-03-06T07:26:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:57.219 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 msgr2=0x7fafa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 0x7fafa406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fafbc007fd0 tx=0x7fafbc009040 comp rx=0 tx=0).stop 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 msgr2=0x7fafc01ec7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fafb8007df0 tx=0x7fafb800d4a0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 shutdown_connections 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafa406b4d0 0x7fafa406d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafc0155a40 0x7fafc01ec7a0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafc014f790 0x7fafc01ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 --2- 172.21.15.17:0/1107501743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafc00c1aa0 0x7fafc01deeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 >> 172.21.15.17:0/1107501743 conn(0x7fafc00c6710 msgr2=0x7fafc00c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 shutdown_connections 2024-03-06T07:26:57.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:57.219+0000 7fafc8381640 1 -- 172.21.15.17:0/1107501743 wait complete. 2024-03-06T07:26:57.633 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:57.633 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:58.634 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:26:58.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.746+0000 7f246a55c640 1 Processor -- start 2024-03-06T07:26:58.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.746+0000 7f246a55c640 1 -- start start 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f246a55c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2464155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f246a55c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2464156050 con 0x7f24640c6780 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f246a55c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2464072630 con 0x7f24640c7380 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f246a55c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246406de20 con 0x7f246406da40 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f24637fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f24640c6780 0x7f2464158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:50200/0 (socket says 172.21.15.17:50200) 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f24637fe640 1 -- 172.21.15.17:0/3775094701 learned_addr learned my addr 172.21.15.17:0/3775094701 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:26:58.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2463fff640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2464155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195436971 0 0) 0x7f244c001110 con 0x7f24640c6780 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2450003520 con 0x7f24640c6780 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229684596 0 0) 0x7f2464072630 con 0x7f24640c7380 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c001110 con 0x7f24640c7380 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794474733 0 0) 0x7f2450003520 con 0x7f24640c6780 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 msgr2=0x7f2464155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2464155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 >> v1:172.21.15.163:6789/0 conn(0x7f24640c7380 legacy=0x7f24640c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464072910 con 0x7f24640c6780 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f244c001840 con 0x7f24640c6780 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.747+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 >> v1:172.21.15.17:6789/0 conn(0x7f24640c6780 legacy=0x7f2464158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2450003b70 0x7f2450003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2450004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2450004d40 0x7f24500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246406de20 con 0x7f2450003b70 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245000bb50 con 0x7f2450004d40 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245000bd20 con 0x7f246406da40 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2450003b70 0x7f2450003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f24637fe640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2450004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2468af9640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2450004d40 0x7f24500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 -- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 msgr2=0x7f2450004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f246406da40 0x7f2450004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 -- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2450004d40 msgr2=0x7f24500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2450004d40 0x7f24500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2463fff640 1 -- 172.21.15.17:0/3775094701 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450003520 con 0x7f2450003b70 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.748+0000 7f2468af9640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2450004d40 0x7f24500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:26:58.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f2463fff640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2450003b70 0x7f2450003f70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2454009af0 tx=0x7f2450005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655f2cc6016df376 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454039070 con 0x7f2450003b70 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245402c6b0 con 0x7f2450003b70 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f2462ffd640 1 -- 172.21.15.17:0/3775094701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454009690 con 0x7f2450003b70 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 -- 172.21.15.17:0/3775094701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2450003b70 msgr2=0x7f2450003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 --2- 172.21.15.17:0/3775094701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2450003b70 0x7f2450003f70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2454009af0 tx=0x7f2450005830 comp rx=0 tx=0).stop 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f2468af9640 1 -- 172.21.15.17:0/3775094701 reap_dead start 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 -- 172.21.15.17:0/3775094701 shutdown_connections 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 -- 172.21.15.17:0/3775094701 >> 172.21.15.17:0/3775094701 conn(0x7f24640aa630 msgr2=0x7f24640aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 -- 172.21.15.17:0/3775094701 shutdown_connections 2024-03-06T07:26:58.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.749+0000 7f246a55c640 1 -- 172.21.15.17:0/3775094701 wait complete. 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 Processor -- start 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 -- start start 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2450003b70 0x7f24640a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f246414eed0 0x7f24641ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24641594a0 con 0x7f24640ab620 2024-03-06T07:26:58.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2464158890 con 0x7f246414eed0 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.751+0000 7f246a55c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24640c3990 con 0x7f2450003b70 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53390/0 (socket says 172.21.15.17:53390) 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 -- 172.21.15.17:0/1190154947 learned_addr learned my addr 172.21.15.17:0/1190154947 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f2468af9640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f246414eed0 0x7f24641ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 -- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2450003b70 msgr2=0x7f24640a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f2463fff640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2450003b70 0x7f24640a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2450003b70 0x7f24640a9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 -- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f246414eed0 msgr2=0x7f24641ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f246414eed0 0x7f24641ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 -- 172.21.15.17:0/1190154947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24641eb680 con 0x7f24640ab620 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f24637fe640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f244c0019b0 tx=0x7f244c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00f9c0 con 0x7f24640ab620 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c018070 con 0x7f24640ab620 2024-03-06T07:26:58.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24641eb960 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00fe00 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.752+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24640aa3b0 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.753+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246406d140 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.754+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f244c01d460 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.754+0000 7f2460ff9640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.755+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f244c08a4f0 con 0x7f24640ab620 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.755+0000 7f2463fff640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:26:58.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.755+0000 7f2463fff640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 0x7f244406d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2454036000 tx=0x7f2454030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:26:58.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.759+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f246406d140 con 0x7f24640ab620 2024-03-06T07:26:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.977+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f246404f330 con 0x7f24640ab620 2024-03-06T07:26:58.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.978+0000 7f2460ff9640 1 -- 172.21.15.17:0/1190154947 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f2464156050 con 0x7f24640ab620 2024-03-06T07:26:58.980 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:26:58.980 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":162,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:46.287561+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9432},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9432":{"gid":9432,"name":"a","rank":0,"incarnation":156,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/1264571498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1264571498},{"type":"v1","addr":"172.21.15.17:6835","nonce":1264571498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:26:58.980 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 162 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 msgr2=0x7f244406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 0x7f244406d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2454036000 tx=0x7f2454030000 comp rx=0 tx=0).stop 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 msgr2=0x7f246414e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f244c0019b0 tx=0x7f244c007720 comp rx=0 tx=0).stop 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 shutdown_connections 2024-03-06T07:26:58.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f246414eed0 0x7f24641ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24640ab620 0x7f246414e990 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 --2- 172.21.15.17:0/1190154947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2450003b70 0x7f24640a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 >> 172.21.15.17:0/1190154947 conn(0x7f24640aa630 msgr2=0x7f24640a8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 shutdown_connections 2024-03-06T07:26:58.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:26:58.980+0000 7f246a55c640 1 -- 172.21.15.17:0/1190154947 wait complete. 2024-03-06T07:26:59.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 162, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:46.287561+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9432}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9432': {'gid': 9432, 'name': 'a', 'rank': 0, 'incarnation': 156, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/1264571498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1264571498}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1264571498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:26:59.395 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9432:up:stopping 2024-03-06T07:26:59.525 INFO:tasks.ceph.mds.a.smithi017.stderr: -660> 2024-03-06T07:26:44.535+0000 7f2ae7895640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:26:59.550 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:27:00.397 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:00.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.508+0000 7f384d71e640 1 Processor -- start 2024-03-06T07:27:00.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.508+0000 7f384d71e640 1 -- start start 2024-03-06T07:27:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f384d71e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f3848071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f384d71e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3848152c20 0x7f384815c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f384d71e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848072440 con 0x7f38480c3150 2024-03-06T07:27:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f384d71e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480c2b10 con 0x7f3848152c20 2024-03-06T07:27:00.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f384d71e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3848152140 con 0x7f384814f650 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f38477fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3848152c20 0x7f384815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f38467fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f38480c3150 0x7f38480c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53326/0 (socket says 172.21.15.17:53326) 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.509+0000 7f38467fc640 1 -- 172.21.15.17:0/2461703633 learned_addr learned my addr 172.21.15.17:0/2461703633 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f3846ffd640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f3848071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f38477fe640 1 -- 172.21.15.17:0/2461703633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 msgr2=0x7f3848071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f38477fe640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f3848071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f38477fe640 1 -- 172.21.15.17:0/2461703633 >> v1:172.21.15.17:6789/0 conn(0x7f38480c3150 legacy=0x7f38480c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f3845ffb640 1 -- 172.21.15.17:0/2461703633 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085651916 0 0) 0x7f3830001110 con 0x7f38480c3150 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f38477fe640 1 -- 172.21.15.17:0/2461703633 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848152420 con 0x7f3848152c20 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f3846ffd640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f3848071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:00.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.510+0000 7f38477fe640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f384815c810 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f383802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45403849e199e8e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f3845ffb640 1 -- 172.21.15.17:0/2461703633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838039070 con 0x7f3848152c20 2024-03-06T07:27:00.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f3845ffb640 1 -- 172.21.15.17:0/2461703633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383802cc50 con 0x7f3848152c20 2024-03-06T07:27:00.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f3845ffb640 1 -- 172.21.15.17:0/2461703633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838034440 con 0x7f3848152c20 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 -- 172.21.15.17:0/2461703633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 msgr2=0x7f384815c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f384815c810 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f383802bea0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 -- 172.21.15.17:0/2461703633 shutdown_connections 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f384815c810 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 --2- 172.21.15.17:0/2461703633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f3848071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 -- 172.21.15.17:0/2461703633 >> 172.21.15.17:0/2461703633 conn(0x7f38480c4290 msgr2=0x7f38480c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 -- 172.21.15.17:0/2461703633 shutdown_connections 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.511+0000 7f384d71e640 1 -- 172.21.15.17:0/2461703633 wait complete. 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 Processor -- start 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 -- start start 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38480c3150 0x7f384806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f38481474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480c40c0 con 0x7f3848152c20 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480c3f40 con 0x7f38480c3150 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f384d71e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480c5da0 con 0x7f384814f650 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f38477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f38477fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46050/0 (socket says 172.21.15.17:46050) 2024-03-06T07:27:00.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.513+0000 7f38477fe640 1 -- 172.21.15.17:0/966346787 learned_addr learned my addr 172.21.15.17:0/966346787 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38467fc640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f38481474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f3846ffd640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38480c3150 0x7f384806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 -- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 msgr2=0x7f38481474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f38481474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 -- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38480c3150 msgr2=0x7f384806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38480c3150 0x7f384806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 -- 172.21.15.17:0/966346787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848147fb0 con 0x7f3848152c20 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f38477fe640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f3838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838044020 con 0x7f3848152c20 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383803a040 con 0x7f3848152c20 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3848147cd0 con 0x7f3848152c20 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838039050 con 0x7f3848152c20 2024-03-06T07:27:00.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.514+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848153560 con 0x7f3848152c20 2024-03-06T07:27:00.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.515+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38481537a0 con 0x7f3848152c20 2024-03-06T07:27:00.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.516+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3838003990 con 0x7f3848152c20 2024-03-06T07:27:00.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.516+0000 7f383f7fe640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:00.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.516+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f38380ac890 con 0x7f3848152c20 2024-03-06T07:27:00.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.516+0000 7f3846ffd640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:00.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.517+0000 7f3846ffd640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3848151c00 tx=0x7f3840008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:00.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.521+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f383804e060 con 0x7f3848152c20 2024-03-06T07:27:00.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.742+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384804f330 con 0x7f3848152c20 2024-03-06T07:27:00.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.743+0000 7f383f7fe640 1 -- 172.21.15.17:0/966346787 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+4976 (secure 0 0 0) 0x7f383803d360 con 0x7f3848152c20 2024-03-06T07:27:00.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:00.745 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":164,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":164}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:59.172336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:00.747 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 164 2024-03-06T07:27:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 msgr2=0x7f382806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 0x7f382806d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3848151c00 tx=0x7f3840008040 comp rx=0 tx=0).stop 2024-03-06T07:27:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 msgr2=0x7f38481ead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f3838004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:00.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 shutdown_connections 2024-03-06T07:27:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382806b4d0 0x7f382806d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3848152c20 0x7f38481ead30 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f384814f650 0x7f38481474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 --2- 172.21.15.17:0/966346787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38480c3150 0x7f384806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:00.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 >> 172.21.15.17:0/966346787 conn(0x7f38480c4290 msgr2=0x7f38480c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 shutdown_connections 2024-03-06T07:27:00.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:00.745+0000 7f384d71e640 1 -- 172.21.15.17:0/966346787 wait complete. 2024-03-06T07:27:01.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 163, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:26:59.172336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [], 'up': {}, 'failed': [], 'damaged': [], 'stopped': [0], 'info': {}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:01.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/0 2024-03-06T07:27:01.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.264+0000 7ff72118a640 1 Processor -- start 2024-03-06T07:27:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.264+0000 7ff72118a640 1 -- start start 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff72118a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff71c0bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff72118a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff71c073250 0x7ff71c0c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff72118a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c06c8e0 con 0x7ff71c0c3950 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff72118a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c06ce80 con 0x7ff71c073250 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff72118a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c1b00 con 0x7ff71c072e70 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71b59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff71c0c3950 0x7ff71c1589b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53336/0 (socket says 172.21.15.17:53336) 2024-03-06T07:27:01.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71b59e640 1 -- 172.21.15.17:0/1921551015 learned_addr learned my addr 172.21.15.17:0/1921551015 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff71c0bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff713fff640 1 --2- 172.21.15.17:0/1921551015 >> v2:172.21.15.163:3300/0 conn(0x7ff71c073250 0x7ff71c0c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824897256 0 0) 0x7ff71c06c8e0 con 0x7ff71c0c3950 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff708003520 con 0x7ff71c0c3950 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1897672250 0 0) 0x7ff708003520 con 0x7ff71c0c3950 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 msgr2=0x7ff71c0bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff71c0bd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 >> v2:172.21.15.163:3300/0 conn(0x7ff71c073250 msgr2=0x7ff71c0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 --2- 172.21.15.17:0/1921551015 >> v2:172.21.15.163:3300/0 conn(0x7ff71c073250 0x7ff71c0c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c06cba0 con 0x7ff71c0c3950 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff7140019f0 con 0x7ff71c0c3950 2024-03-06T07:27:01.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 >> v1:172.21.15.17:6789/0 conn(0x7ff71c0c3950 legacy=0x7ff71c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.265+0000 7ff71a59c640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080039b0 0x7ff708003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71a59c640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff708004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71a59c640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff708004d40 0x7ff7080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c06ce80 con 0x7ff7080039b0 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c1b00 con 0x7ff708004d40 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800b9c0 con 0x7ff71c072e70 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080039b0 0x7ff708003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff713fff640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff708004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71b59e640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff708004d40 0x7ff7080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 -- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 msgr2=0x7ff708004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c072e70 0x7ff708004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 -- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff708004d40 msgr2=0x7ff7080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff708004d40 0x7ff7080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.266+0000 7ff71ad9d640 1 -- 172.21.15.17:0/1921551015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708003520 con 0x7ff7080039b0 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080039b0 0x7ff708003dd0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009ac0 tx=0x7ff708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515dbcf9fccde853 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c039070 con 0x7ff7080039b0 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c02c6b0 con 0x7ff7080039b0 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff71a59c640 1 -- 172.21.15.17:0/1921551015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c009690 con 0x7ff7080039b0 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 -- 172.21.15.17:0/1921551015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080039b0 msgr2=0x7ff708003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 --2- 172.21.15.17:0/1921551015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7080039b0 0x7ff708003dd0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009ac0 tx=0x7ff708005830 comp rx=0 tx=0).stop 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff71b59e640 1 -- 172.21.15.17:0/1921551015 reap_dead start 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 -- 172.21.15.17:0/1921551015 shutdown_connections 2024-03-06T07:27:01.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 -- 172.21.15.17:0/1921551015 >> 172.21.15.17:0/1921551015 conn(0x7ff71c06b1a0 msgr2=0x7ff71c06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 -- 172.21.15.17:0/1921551015 shutdown_connections 2024-03-06T07:27:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.267+0000 7ff72118a640 1 -- 172.21.15.17:0/1921551015 wait complete. 2024-03-06T07:27:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 Processor -- start 2024-03-06T07:27:01.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 -- start start 2024-03-06T07:27:01.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7080039b0 0x7ff71c154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c1532b0 0x7ff71c070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c4b80 con 0x7ff71c0be240 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0c4a00 con 0x7ff7080039b0 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff72118a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c06e490 con 0x7ff71c1532b0 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff71b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff71b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46066/0 (socket says 172.21.15.17:46066) 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff71b59e640 1 -- 172.21.15.17:0/2774672319 learned_addr learned my addr 172.21.15.17:0/2774672319 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7080039b0 0x7ff71c154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.269+0000 7ff713fff640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c1532b0 0x7ff71c070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 -- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c1532b0 msgr2=0x7ff71c070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c1532b0 0x7ff71c070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 -- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7080039b0 msgr2=0x7ff71c154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7080039b0 0x7ff71c154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 -- 172.21.15.17:0/2774672319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c153a40 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff71b59e640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff714002060 tx=0x7ff71400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400ee20 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714018070 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714013660 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c0a7900 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.270+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c153740 con 0x7ff71c0be240 2024-03-06T07:27:01.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.271+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c04f330 con 0x7ff71c0be240 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.272+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff714013800 con 0x7ff71c0be240 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.272+0000 7ff7137fe640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.272+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff71408a5a0 con 0x7ff71c0be240 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.272+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.273+0000 7ff71ad9d640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036000 tx=0x7ff70c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.276+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff714058c80 con 0x7ff71c0be240 2024-03-06T07:27:01.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.495+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff71c0c6340 con 0x7ff71c0be240 2024-03-06T07:27:01.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.495+0000 7ff7137fe640 1 -- 172.21.15.17:0/2774672319 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+4976 (secure 0 0 0) 0x7ff71405cd20 con 0x7ff71c0be240 2024-03-06T07:27:01.497 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:01.497 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":164,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9660,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":164}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:26:59.172336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:01.497 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 164 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 msgr2=0x7ff6fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036000 tx=0x7ff70c030000 comp rx=0 tx=0).stop 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 msgr2=0x7ff71c070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff714002060 tx=0x7ff71400e530 comp rx=0 tx=0).stop 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 shutdown_connections 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6fc06b4d0 0x7ff6fc06d990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff71c0be240 0x7ff71c070bb0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff71c1532b0 0x7ff71c070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 --2- 172.21.15.17:0/2774672319 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7080039b0 0x7ff71c154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 >> 172.21.15.17:0/2774672319 conn(0x7ff71c06b1a0 msgr2=0x7ff71c1554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:01.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 shutdown_connections 2024-03-06T07:27:01.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.497+0000 7ff72118a640 1 -- 172.21.15.17:0/2774672319 wait complete. 2024-03-06T07:27:01.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down false 2024-03-06T07:27:01.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.612+0000 7f94f795c640 1 Processor -- start 2024-03-06T07:27:01.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.612+0000 7f94f795c640 1 -- start start 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f795c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f01529f0 0x7f94f00c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f795c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f795c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f0152dd0 con 0x7f94f0155530 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f795c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f0159710 con 0x7f94f01529f0 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f795c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c68b0 con 0x7f94f00c5370 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f01529f0 0x7f94f00c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94f01529f0 0x7f94f00c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46082/0 (socket says 172.21.15.17:46082) 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 -- 172.21.15.17:0/1519892866 learned_addr learned my addr 172.21.15.17:0/1519892866 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 -- 172.21.15.17:0/1519892866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 msgr2=0x7f94f00c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 --2- 172.21.15.17:0/1519892866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 -- 172.21.15.17:0/1519892866 >> v1:172.21.15.163:6789/0 conn(0x7f94f0155530 legacy=0x7f94f01559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.613+0000 7f94f56f8640 1 -- 172.21.15.17:0/1519892866 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0158b00 con 0x7f94f01529f0 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.614+0000 7f94f56f8640 1 --2- 172.21.15.17:0/1519892866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f01529f0 0x7f94f00c88f0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009e50 tx=0x7f94e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7ce38c1a8e8037 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.614+0000 7f94effff640 1 -- 172.21.15.17:0/1519892866 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e002c9e0 con 0x7f94f01529f0 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.614+0000 7f94effff640 1 -- 172.21.15.17:0/1519892866 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e002cb80 con 0x7f94f01529f0 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.614+0000 7f94effff640 1 -- 172.21.15.17:0/1519892866 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00343f0 con 0x7f94f01529f0 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 -- 172.21.15.17:0/1519892866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f01529f0 msgr2=0x7f94f00c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 --2- 172.21.15.17:0/1519892866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f01529f0 0x7f94f00c88f0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009e50 tx=0x7f94e002be40 comp rx=0 tx=0).stop 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 -- 172.21.15.17:0/1519892866 shutdown_connections 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 --2- 172.21.15.17:0/1519892866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f00c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 --2- 172.21.15.17:0/1519892866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f01529f0 0x7f94f00c88f0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 -- 172.21.15.17:0/1519892866 >> 172.21.15.17:0/1519892866 conn(0x7f94f00ac580 msgr2=0x7f94f00ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:01.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 -- 172.21.15.17:0/1519892866 shutdown_connections 2024-03-06T07:27:01.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.615+0000 7f94f795c640 1 -- 172.21.15.17:0/1519892866 wait complete. 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.616+0000 7f94f795c640 1 Processor -- start 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.616+0000 7f94f795c640 1 -- start start 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f01581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94f01529f0 0x7f94f014bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01593a0 con 0x7f94f0155530 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01590c0 con 0x7f94f01529f0 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f795c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f01560d0 con 0x7f94f00c5370 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46090/0 (socket says 172.21.15.17:46090) 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 -- 172.21.15.17:0/343187746 learned_addr learned my addr 172.21.15.17:0/343187746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f56f8640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f01581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f4ef7640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94f01529f0 0x7f94f014bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 -- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 msgr2=0x7f94f01581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f01581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 -- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94f01529f0 msgr2=0x7f94f014bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94f01529f0 0x7f94f014bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 -- 172.21.15.17:0/343187746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f01ee060 con 0x7f94f0155530 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.617+0000 7f94f5ef9640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f94e800b490 tx=0x7f94e800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.618+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8015020 con 0x7f94f0155530 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.618+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e8011850 con 0x7f94f0155530 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.618+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f01ee340 con 0x7f94f0155530 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.618+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8011c40 con 0x7f94f0155530 2024-03-06T07:27:01.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.618+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f01ee900 con 0x7f94f0155530 2024-03-06T07:27:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.619+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f004f330 con 0x7f94f0155530 2024-03-06T07:27:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.619+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94e8018020 con 0x7f94f0155530 2024-03-06T07:27:01.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.619+0000 7f94edffb640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 0x7f94dc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.620+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f94e8089fe0 con 0x7f94f0155530 2024-03-06T07:27:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.620+0000 7f94f56f8640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 0x7f94dc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.620+0000 7f94f56f8640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 0x7f94dc06d9e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f94e0035000 tx=0x7f94e0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.624+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94e801f020 con 0x7f94f0155530 2024-03-06T07:27:01.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.842+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "false"} v 0) v1 -- 0x7f94f0158730 con 0x7f94f0155530 2024-03-06T07:27:01.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.854+0000 7f94edffb640 1 -- 172.21.15.17:0/343187746 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "false"}]=0 cephfs marked up, max_mds = 1 v165) v1 ==== 135+0+0 (secure 0 0 0) 0x7f94e8058df0 con 0x7f94f0155530 2024-03-06T07:27:01.856 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked up, max_mds = 1 2024-03-06T07:27:01.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 msgr2=0x7f94dc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 0x7f94dc06d9e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f94e0035000 tx=0x7f94e0030000 comp rx=0 tx=0).stop 2024-03-06T07:27:01.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 msgr2=0x7f94f014c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f94e800b490 tx=0x7f94e800b890 comp rx=0 tx=0).stop 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 shutdown_connections 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94dc06b520 0x7f94dc06d9e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94f0155530 0x7f94f014c050 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94f01529f0 0x7f94f014bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 --2- 172.21.15.17:0/343187746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94f00c5370 0x7f94f01581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 >> 172.21.15.17:0/343187746 conn(0x7f94f00ac580 msgr2=0x7f94f0147150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 shutdown_connections 2024-03-06T07:27:01.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.856+0000 7f94f795c640 1 -- 172.21.15.17:0/343187746 wait complete. 2024-03-06T07:27:01.870 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:01.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.969+0000 7f6afd7d0640 1 Processor -- start 2024-03-06T07:27:01.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.969+0000 7f6afd7d0640 1 -- start start 2024-03-06T07:27:01.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6afd7d0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6af814f360 0x7f6af8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6afd7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6af80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6afd7d0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80c73d0 con 0x7f6af80c5880 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6afd7d0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80c8dd0 con 0x7f6af814f360 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6afd7d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80be2e0 con 0x7f6af80c88c0 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6af80c5880 0x7f6af8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53350/0 (socket says 172.21.15.17:53350) 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af77fe640 1 -- 172.21.15.17:0/1387559330 learned_addr learned my addr 172.21.15.17:0/1387559330 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266654908 0 0) 0x7f6af80c73d0 con 0x7f6af80c5880 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae4003520 con 0x7f6af80c5880 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af67fc640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6af80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/1387559330 >> v2:172.21.15.163:3300/0 conn(0x7f6af814f360 0x7f6af8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.970+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 84349791 0 0) 0x7f6ae4003520 con 0x7f6af80c5880 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 msgr2=0x7f6af80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6af80c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 >> v2:172.21.15.163:3300/0 conn(0x7f6af814f360 msgr2=0x7f6af8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 --2- 172.21.15.17:0/1387559330 >> v2:172.21.15.163:3300/0 conn(0x7f6af814f360 0x7f6af8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80c7670 con 0x7f6af80c5880 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6af00019f0 con 0x7f6af80c5880 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 >> v1:172.21.15.17:6789/0 conn(0x7f6af80c5880 legacy=0x7f6af8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6ae40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae4003f30 0x7f6ae4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae4004d40 0x7f6ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80be2e0 con 0x7f6ae4004d40 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae400b9c0 con 0x7f6ae4003f30 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae400bb90 con 0x7f6af80c88c0 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae4004d40 0x7f6ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6ae40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af67fc640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae4003f30 0x7f6ae4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 -- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 msgr2=0x7f6ae40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6ae40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 -- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae4003f30 msgr2=0x7f6ae4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:01.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae4003f30 0x7f6ae4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af77fe640 1 -- 172.21.15.17:0/1387559330 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae4003520 con 0x7f6ae4004d40 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af67fc640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6ae4003f30 0x7f6ae4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.971+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af80c88c0 0x7f6ae40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6af77fe640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae4004d40 0x7f6ae40051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f6af000a890 tx=0x7f6ae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31383fd202eb9e7d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000a4e0 con 0x7f6ae4004d40 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0034650 con 0x7f6ae4004d40 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6af5ffb640 1 -- 172.21.15.17:0/1387559330 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0034a40 con 0x7f6ae4004d40 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6afd7d0640 1 -- 172.21.15.17:0/1387559330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae4004d40 msgr2=0x7f6ae40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/1387559330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ae4004d40 0x7f6ae40051c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f6af000a890 tx=0x7f6ae4005830 comp rx=0 tx=0).stop 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6af77fe640 1 -- 172.21.15.17:0/1387559330 reap_dead start 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6afd7d0640 1 -- 172.21.15.17:0/1387559330 shutdown_connections 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.972+0000 7f6afd7d0640 1 -- 172.21.15.17:0/1387559330 >> 172.21.15.17:0/1387559330 conn(0x7f6af80ac570 msgr2=0x7f6af80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.973+0000 7f6afd7d0640 1 -- 172.21.15.17:0/1387559330 shutdown_connections 2024-03-06T07:27:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.973+0000 7f6afd7d0640 1 -- 172.21.15.17:0/1387559330 wait complete. 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 Processor -- start 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 -- start start 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af8152470 0x7f6af81edc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af8158400 0x7f6af81ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c5340 con 0x7f6af80abce0 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c7c30 con 0x7f6af8158400 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6afd7d0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c54c0 con 0x7f6af8152470 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6af6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.974+0000 7f6af6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46100/0 (socket says 172.21.15.17:46100) 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 -- 172.21.15.17:0/2671272098 learned_addr learned my addr 172.21.15.17:0/2671272098 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 -- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af8152470 msgr2=0x7f6af81edc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af8152470 0x7f6af81edc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 -- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af8158400 msgr2=0x7f6af81ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af8158400 0x7f6af81ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 -- 172.21.15.17:0/2671272098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8157c40 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6af6ffd640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0011700 tx=0x7f6ae0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000f8e0 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae000fa80 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af8157f00 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000fe50 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.975+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af81578f0 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.976+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af804f330 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.977+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ae0015460 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.977+0000 7f6aef7fe640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.977+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6ae008ac90 con 0x7f6af80abce0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.977+0000 7f6af67fc640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:01.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.978+0000 7f6af67fc640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 0x7f6ad806d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6af81528b0 tx=0x7f6ae8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:01.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:01.981+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ae0019030 con 0x7f6af80abce0 2024-03-06T07:27:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.198+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af806cda0 con 0x7f6af80abce0 2024-03-06T07:27:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.198+0000 7f6aef7fe640 1 -- 172.21.15.17:0/2671272098 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+4988 (secure 0 0 0) 0x7f6ae0059930 con 0x7f6af80abce0 2024-03-06T07:27:02.200 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:02.200 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":165,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:01.853813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:starting","state_seq":1,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:02.200 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 165 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 msgr2=0x7f6ad806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 0x7f6ad806d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6af81528b0 tx=0x7f6ae8008040 comp rx=0 tx=0).stop 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 msgr2=0x7f6af80afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0011700 tx=0x7f6ae0011bc0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 shutdown_connections 2024-03-06T07:27:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ad806b500 0x7f6ad806d9c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af8158400 0x7f6af81ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6af8152470 0x7f6af81edc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 --2- 172.21.15.17:0/2671272098 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6af80abce0 0x7f6af80afcd0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 >> 172.21.15.17:0/2671272098 conn(0x7f6af80ac570 msgr2=0x7f6af80c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 shutdown_connections 2024-03-06T07:27:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.200+0000 7f6afd7d0640 1 -- 172.21.15.17:0/2671272098 wait complete. 2024-03-06T07:27:02.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:02.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 Processor -- start 2024-03-06T07:27:02.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 -- start start 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff40014f790 0x7ff4001529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff4000c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4000c5c60 0x7ff400155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4000c2680 con 0x7ff40014f790 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4000c28d0 con 0x7ff4000c5c60 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.314+0000 7ff406c17640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000c2bb0 con 0x7ff4000c1aa0 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff40014f790 0x7ff4001529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff40014f790 0x7ff4001529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46114/0 (socket says 172.21.15.17:46114) 2024-03-06T07:27:02.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 -- 172.21.15.17:0/2567070601 learned_addr learned my addr 172.21.15.17:0/2567070601 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 -- 172.21.15.17:0/2567070601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 msgr2=0x7ff4000c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4051b4640 1 --2- 172.21.15.17:0/2567070601 >> v2:172.21.15.163:3300/0 conn(0x7ff4000c5c60 0x7ff400155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff3fffff640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff4000c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff4000c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 -- 172.21.15.17:0/2567070601 >> v2:172.21.15.163:3300/0 conn(0x7ff4000c5c60 msgr2=0x7ff400155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 --2- 172.21.15.17:0/2567070601 >> v2:172.21.15.163:3300/0 conn(0x7ff4000c5c60 0x7ff400155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.315+0000 7ff4049b3640 1 -- 172.21.15.17:0/2567070601 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4000c2e90 con 0x7ff40014f790 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff4049b3640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001529d0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff40006c960 tx=0x7ff3f402db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9903e6d73a7e647 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff3ff7fe640 1 -- 172.21.15.17:0/2567070601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f402f8e0 con 0x7ff40014f790 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff3ff7fe640 1 -- 172.21.15.17:0/2567070601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f402fa80 con 0x7ff40014f790 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff3ff7fe640 1 -- 172.21.15.17:0/2567070601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f402fe50 con 0x7ff40014f790 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 -- 172.21.15.17:0/2567070601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 msgr2=0x7ff4001529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001529d0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7ff40006c960 tx=0x7ff3f402db20 comp rx=0 tx=0).stop 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 -- 172.21.15.17:0/2567070601 shutdown_connections 2024-03-06T07:27:02.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 --2- 172.21.15.17:0/2567070601 >> v2:172.21.15.163:3300/0 conn(0x7ff4000c5c60 0x7ff400155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff4000c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 --2- 172.21.15.17:0/2567070601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001529d0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 -- 172.21.15.17:0/2567070601 >> 172.21.15.17:0/2567070601 conn(0x7ff4000c6710 msgr2=0x7ff4000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 -- 172.21.15.17:0/2567070601 shutdown_connections 2024-03-06T07:27:02.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.316+0000 7ff406c17640 1 -- 172.21.15.17:0/2567070601 wait complete. 2024-03-06T07:27:02.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.317+0000 7ff406c17640 1 Processor -- start 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 -- start start 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff400148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 0x7ff40014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40006cc00 con 0x7ff40014f790 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000c0d60 con 0x7ff4000c5c60 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff406c17640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40006cd80 con 0x7ff4000c1aa0 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.318+0000 7ff4051b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46124/0 (socket says 172.21.15.17:46124) 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 -- 172.21.15.17:0/69515448 learned_addr learned my addr 172.21.15.17:0/69515448 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 -- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 msgr2=0x7ff400148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4049b3640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff400148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff3fffff640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 0x7ff40014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff400148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 -- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 msgr2=0x7ff40014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 0x7ff40014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 -- 172.21.15.17:0/69515448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400154350 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4049b3640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff400148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff3fffff640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 0x7ff40014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff4051b4640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800cda0 tx=0x7ff3f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003b90 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8003d30 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff400154630 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8004240 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.319+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff400150970 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.320+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40004f330 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.321+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3f80043e0 con 0x7ff40014f790 2024-03-06T07:27:02.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.321+0000 7ff3fd7fa640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.321+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff3f808af60 con 0x7ff40014f790 2024-03-06T07:27:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.321+0000 7ff4049b3640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:02.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.322+0000 7ff4049b3640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 0x7ff3e406d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4038000 tx=0x7ff3f4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:02.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.325+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff3f801d340 con 0x7ff40014f790 2024-03-06T07:27:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.545+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff40014d640 con 0x7ff40014f790 2024-03-06T07:27:02.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.546+0000 7ff3fd7fa640 1 -- 172.21.15.17:0/69515448 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+4988 (secure 0 0 0) 0x7ff3f8059c00 con 0x7ff40014f790 2024-03-06T07:27:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:02.548 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":165,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:01.853813+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:starting","state_seq":1,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":13}]} 2024-03-06T07:27:02.548 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 165 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 msgr2=0x7ff3e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 0x7ff3e406d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4038000 tx=0x7ff3f4008030 comp rx=0 tx=0).stop 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 msgr2=0x7ff4001ee110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800cda0 tx=0x7ff3f800a660 comp rx=0 tx=0).stop 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 shutdown_connections 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff3e406b4d0 0x7ff3e406d990 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff40014f790 0x7ff4001ee110 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4000c5c60 0x7ff40014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 --2- 172.21.15.17:0/69515448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4000c1aa0 0x7ff400148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 >> 172.21.15.17:0/69515448 conn(0x7ff4000c6710 msgr2=0x7ff400152410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 shutdown_connections 2024-03-06T07:27:02.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:02.548+0000 7ff406c17640 1 -- 172.21.15.17:0/69515448 wait complete. 2024-03-06T07:27:02.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 165, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:27:01.853813+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9660': {'gid': 9660, 'name': 'a', 'rank': 0, 'incarnation': 165, 'state': 'up:starting', 'state_seq': 1, 'addr': '172.21.15.17:6835/1440163522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1440163522}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1440163522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:02.963 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9660:up:starting 2024-03-06T07:27:03.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.064+0000 7fe8345ac640 1 Processor -- start 2024-03-06T07:27:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.064+0000 7fe8345ac640 1 -- start start 2024-03-06T07:27:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe82c14f360 0x7fe82c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 0x7fe82c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0c8dd0 con 0x7fe82c14f360 2024-03-06T07:27:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0c73d0 con 0x7fe82c0c5880 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe8345ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c0be2e0 con 0x7fe82c0c88c0 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe832348640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe82c14f360 0x7fe82c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe832348640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe82c14f360 0x7fe82c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46138/0 (socket says 172.21.15.17:46138) 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.065+0000 7fe832348640 1 -- 172.21.15.17:0/3473874472 learned_addr learned my addr 172.21.15.17:0/3473874472 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe831b47640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832b49640 1 --2- 172.21.15.17:0/3473874472 >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 0x7fe82c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 -- 172.21.15.17:0/3473874472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 msgr2=0x7fe82c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 -- 172.21.15.17:0/3473874472 >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 msgr2=0x7fe82c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 --2- 172.21.15.17:0/3473874472 >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 0x7fe82c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 -- 172.21.15.17:0/3473874472 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0c7670 con 0x7fe82c14f360 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe831b47640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832b49640 1 --2- 172.21.15.17:0/3473874472 >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 0x7fe82c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe832348640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c155c10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe82c08cc10 tx=0x7fe82802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb050f298673428 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.066+0000 7fe831346640 1 -- 172.21.15.17:0/3473874472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82802e070 con 0x7fe82c14f360 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe831346640 1 -- 172.21.15.17:0/3473874472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe828037070 con 0x7fe82c14f360 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe831346640 1 -- 172.21.15.17:0/3473874472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828031a00 con 0x7fe82c14f360 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 -- 172.21.15.17:0/3473874472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 msgr2=0x7fe82c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c155c10 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fe82c08cc10 tx=0x7fe82802b880 comp rx=0 tx=0).stop 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 -- 172.21.15.17:0/3473874472 shutdown_connections 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 --2- 172.21.15.17:0/3473874472 >> v2:172.21.15.163:3300/0 conn(0x7fe82c0c5880 0x7fe82c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 --2- 172.21.15.17:0/3473874472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c155c10 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 -- 172.21.15.17:0/3473874472 >> 172.21.15.17:0/3473874472 conn(0x7fe82c0ac570 msgr2=0x7fe82c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 -- 172.21.15.17:0/3473874472 shutdown_connections 2024-03-06T07:27:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.067+0000 7fe8345ac640 1 -- 172.21.15.17:0/3473874472 wait complete. 2024-03-06T07:27:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.068+0000 7fe8345ac640 1 Processor -- start 2024-03-06T07:27:04.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.068+0000 7fe8345ac640 1 -- start start 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 0x7fe82c0bf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c1ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c1590a0 con 0x7fe82c14f360 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c0c7c30 con 0x7fe82c0c5880 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe8345ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c159220 con 0x7fe82c0c88c0 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46154/0 (socket says 172.21.15.17:46154) 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 -- 172.21.15.17:0/888725299 learned_addr learned my addr 172.21.15.17:0/888725299 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832348640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 0x7fe82c0bf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe831b47640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c1ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 -- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 msgr2=0x7fe82c1ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c1ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 -- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 msgr2=0x7fe82c0bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 0x7fe82c0bf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832b49640 1 -- 172.21.15.17:0/888725299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0c0c60 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.069+0000 7fe832348640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 0x7fe82c0bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe832b49640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe82400cef0 tx=0x7fe82400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82401a070 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82400d840 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400dc30 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c0c0f40 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.070+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c0bf0a0 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.071+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c04f330 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.071+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe82401d070 con 0x7fe82c14f360 2024-03-06T07:27:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.072+0000 7fe822ffd640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 0x7fe81006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:04.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.072+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fe82408abe0 con 0x7fe82c14f360 2024-03-06T07:27:04.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.072+0000 7fe832348640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 0x7fe81006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:04.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.072+0000 7fe832348640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 0x7fe81006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe828036000 tx=0x7fe828008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.076+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe824059910 con 0x7fe82c14f360 2024-03-06T07:27:04.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.377+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe82c06cda0 con 0x7fe82c14f360 2024-03-06T07:27:04.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.378+0000 7fe822ffd640 1 -- 172.21.15.17:0/888725299 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fe824059210 con 0x7fe82c14f360 2024-03-06T07:27:04.379 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:04.380 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 msgr2=0x7fe81006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 0x7fe81006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe828036000 tx=0x7fe828008030 comp rx=0 tx=0).stop 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 msgr2=0x7fe82c0c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe82400cef0 tx=0x7fe82400a710 comp rx=0 tx=0).stop 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 shutdown_connections 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe81006b4d0 0x7fe81006d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c14f360 0x7fe82c0c4410 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe82c0c88c0 0x7fe82c1ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 --2- 172.21.15.17:0/888725299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe82c0c5880 0x7fe82c0bf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 >> 172.21.15.17:0/888725299 conn(0x7fe82c0ac570 msgr2=0x7fe82c153000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 shutdown_connections 2024-03-06T07:27:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.380+0000 7fe8345ac640 1 -- 172.21.15.17:0/888725299 wait complete. 2024-03-06T07:27:04.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 166, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:26:38.614061+0000', 'modified': '2024-03-06T07:27:02.855289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9660': {'gid': 9660, 'name': 'a', 'rank': 0, 'incarnation': 165, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.17:6835/1440163522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 1440163522}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1440163522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9660, 'qdb_cluster': [9660]} 2024-03-06T07:27:04.795 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:27:04.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:27:04.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 Processor -- start 2024-03-06T07:27:05.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 -- start start 2024-03-06T07:27:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb00158770 0x7feb00158b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb00072d20 0x7feb0015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb00159090 con 0x7feb00073270 2024-03-06T07:27:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000a7fd0 con 0x7feb00158770 2024-03-06T07:27:05.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.897+0000 7feb05015640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000c26c0 con 0x7feb00072d20 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb00158770 0x7feb00158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafe59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feb00073270 0x7feb000724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53356/0 (socket says 172.21.15.17:53356) 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafe59c640 1 -- 172.21.15.17:0/1100841218 learned_addr learned my addr 172.21.15.17:0/1100841218 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafed9d640 1 -- 172.21.15.17:0/1100841218 >> v2:172.21.15.163:3300/0 conn(0x7feb00072d20 msgr2=0x7feb0015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafed9d640 1 --2- 172.21.15.17:0/1100841218 >> v2:172.21.15.163:3300/0 conn(0x7feb00072d20 0x7feb0015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafed9d640 1 -- 172.21.15.17:0/1100841218 >> v1:172.21.15.17:6789/0 conn(0x7feb00073270 legacy=0x7feb000724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafdd9b640 1 -- 172.21.15.17:0/1100841218 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748940025 0 0) 0x7feb00159090 con 0x7feb00073270 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.898+0000 7feafed9d640 1 -- 172.21.15.17:0/1100841218 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0006f230 con 0x7feb00158770 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feafed9d640 1 --2- 172.21.15.17:0/1100841218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00158770 0x7feb00158b50 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7feb0006f770 tx=0x7feaf002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848b714fab4cc2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feafdd9b640 1 -- 172.21.15.17:0/1100841218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0038070 con 0x7feb00158770 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feafdd9b640 1 -- 172.21.15.17:0/1100841218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf0031710 con 0x7feb00158770 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feafdd9b640 1 -- 172.21.15.17:0/1100841218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0031ae0 con 0x7feb00158770 2024-03-06T07:27:05.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 -- 172.21.15.17:0/1100841218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00158770 msgr2=0x7feb00158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 --2- 172.21.15.17:0/1100841218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00158770 0x7feb00158b50 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7feb0006f770 tx=0x7feaf002b7b0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 -- 172.21.15.17:0/1100841218 shutdown_connections 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 --2- 172.21.15.17:0/1100841218 >> v2:172.21.15.163:3300/0 conn(0x7feb00072d20 0x7feb0015fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 --2- 172.21.15.17:0/1100841218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00158770 0x7feb00158b50 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 -- 172.21.15.17:0/1100841218 >> 172.21.15.17:0/1100841218 conn(0x7feb0006b970 msgr2=0x7feb000aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 -- 172.21.15.17:0/1100841218 shutdown_connections 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.899+0000 7feb05015640 1 -- 172.21.15.17:0/1100841218 wait complete. 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 Processor -- start 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 -- start start 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb00072d20 0x7feb001ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb00158770 0x7feb001e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000c2910 con 0x7feb00073270 2024-03-06T07:27:05.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000a8220 con 0x7feb00072d20 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.901+0000 7feb05015640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0006e2e0 con 0x7feb00158770 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46174/0 (socket says 172.21.15.17:46174) 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 -- 172.21.15.17:0/1982611798 learned_addr learned my addr 172.21.15.17:0/1982611798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafed9d640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb00072d20 0x7feb001ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feaff59e640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb00158770 0x7feb001e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 -- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb00158770 msgr2=0x7feb001e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb00158770 0x7feb001e05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 -- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb00072d20 msgr2=0x7feb001ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb00072d20 0x7feb001ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 -- 172.21.15.17:0/1982611798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb000adac0 con 0x7feb00073270 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feafe59c640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c460 tx=0x7feaf400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feae77fe640 1 -- 172.21.15.17:0/1982611798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4018070 con 0x7feb00073270 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feae77fe640 1 -- 172.21.15.17:0/1982611798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4007500 con 0x7feb00073270 2024-03-06T07:27:05.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb001eef40 con 0x7feb00073270 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feae77fe640 1 -- 172.21.15.17:0/1982611798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004a20 con 0x7feb00073270 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.902+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb001df8b0 con 0x7feb00073270 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.904+0000 7feae77fe640 1 -- 172.21.15.17:0/1982611798 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feaf401b070 con 0x7feb00073270 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.904+0000 7feae77fe640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 0x7feaec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.904+0000 7feae77fe640 1 -- 172.21.15.17:0/1982611798 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7feaf4089610 con 0x7feb00073270 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.904+0000 7feafed9d640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 0x7feaec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.905+0000 7feafed9d640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 0x7feaec06d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7feaf0035000 tx=0x7feaf0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.905+0000 7feb07bfe740 1 Processor -- start 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.905+0000 7feb07bfe740 1 -- start start 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d650f50 0x55694d64ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d64f4b0 0x55694d64b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d5b56b0 con 0x55694d651330 2024-03-06T07:27:05.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d55ea60 con 0x55694d64f4b0 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694d5b1750 con 0x55694d650f50 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46176/0 (socket says 172.21.15.17:46176) 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 -- 172.21.15.17:0/1467013565 learned_addr learned my addr 172.21.15.17:0/1467013565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafed9d640 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d650f50 0x55694d64ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feaff59e640 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d64f4b0 0x55694d64b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 -- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d650f50 msgr2=0x55694d64ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d650f50 0x55694d64ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 -- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d64f4b0 msgr2=0x55694d64b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d64f4b0 0x55694d64b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.906+0000 7feafe59c640 1 -- 172.21.15.17:0/1467013565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55694d57cf30 con 0x55694d651330 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feafe59c640 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7feaf4001120 tx=0x7feaf4057c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1a8f2c36f39408 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feae4ff9640 1 -- 172.21.15.17:0/1467013565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf405e070 con 0x55694d651330 2024-03-06T07:27:05.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feae4ff9640 1 -- 172.21.15.17:0/1467013565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4060350 con 0x55694d651330 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feae4ff9640 1 -- 172.21.15.17:0/1467013565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf40542b0 con 0x55694d651330 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 -- 172.21.15.17:0/1467013565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 msgr2=0x55694d64ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7feaf4001120 tx=0x7feaf4057c40 comp rx=0 tx=0).stop 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 -- 172.21.15.17:0/1467013565 shutdown_connections 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d64f4b0 0x55694d64b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d651330 0x55694d64ef70 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1467013565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d650f50 0x55694d64ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.907+0000 7feb07bfe740 1 -- 172.21.15.17:0/1467013565 >> 172.21.15.17:0/1467013565 conn(0x55694d64d7d0 msgr2=0x55694d6723c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.908+0000 7feb07bfe740 1 -- 172.21.15.17:0/1467013565 shutdown_connections 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.908+0000 7feb07bfe740 1 -- 172.21.15.17:0/1467013565 wait complete. 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.908+0000 7feb07bfe740 1 -- start start 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.908+0000 7feb07bfe740 1 Processor -- start 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d6a1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d674280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d673230 con 0x55694d650f50 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d571ea0 con 0x55694d651330 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694d672c70 con 0x55694d673e00 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feb07bfe740 10 client.0 _ll_register_callbacks cb 0x55694d571870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46180/0 (socket says 172.21.15.17:46180) 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 -- 172.21.15.17:0/1163629226 learned_addr learned my addr 172.21.15.17:0/1163629226 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafe59c640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d6a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feaff59e640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d674280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 -- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 msgr2=0x55694d674280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d674280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 -- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 msgr2=0x55694d6a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d6a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 -- 172.21.15.17:0/1163629226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55694d572270 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feafed9d640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7feaf002bdc0 tx=0x7feaf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181f67868b0e1ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.909+0000 7feaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf004f020 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7feaf004a910 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaf4060350 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf0038040 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 10 client.9714 fetch_fsmap learned FSMap version 166 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=2+}) v3 -- 0x55694d2ca2d0 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4060350 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 166) v1 ==== 3314+0+0 (secure 0 0 0) 0x7feaf004ab10 con 0x55694d650f50 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 10 client.9714 fetch_fsmap finished waiting for FSMap version 166 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 10 client.9714 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 20 client.9714 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:05.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d69e180 0x55694d6ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 4 client.9714 mds_command: new command op to 9660 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 --> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55694d5c2bf0 con 0x55694d69e180 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.910+0000 7feafe59c640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d69e180 0x55694d6ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feafe59c640 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d69e180 0x55694d6ac300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14bed839a7fb590 server_cookie=2361bdc3be5c8c9a in_seq=0 out_seq=0 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feaddffb640 10 client.9714 ms_handle_connect on v2:172.21.15.17:6834/1440163522 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feaddffb640 1 -- 172.21.15.17:0/1163629226 <== mds.0 v2:172.21.15.17:6834/1440163522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55694d5c2bf0 con 0x55694d69e180 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feaddffb640 10 client.9714 handle_command_reply: tid=0 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 client.9714 shutdown 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 msgr2=0x55694d6717b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7feaf002bdc0 tx=0x7feaf0004730 comp rx=0 tx=0).stop 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 shutdown_connections 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d674280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d69e180 0x55694d6ac300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d6a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1163629226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d6717b0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feaddffb640 0 client.9714 ms_handle_reset on v2:172.21.15.17:6834/1440163522 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.911+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 >> 172.21.15.17:0/1163629226 conn(0x55694d64f4b0 msgr2=0x55694d64f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.912+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 shutdown_connections 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.912+0000 7feb07bfe740 1 -- 172.21.15.17:0/1163629226 wait complete. 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.912+0000 7feb07bfe740 20 client.9714 trim_cache size 0 max 16384 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.928+0000 7feb07bfe740 1 Processor -- start 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.928+0000 7feb07bfe740 1 -- start start 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d651330 0x55694d651770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d673e00 0x55694d651cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb001f6ee0 con 0x55694d650f50 2024-03-06T07:27:05.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0014f020 con 0x55694d673e00 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb001f6910 con 0x55694d651330 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46184/0 (socket says 172.21.15.17:46184) 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 -- 172.21.15.17:0/1489613216 learned_addr learned my addr 172.21.15.17:0/1489613216 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 -- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d651330 msgr2=0x55694d651770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d651330 0x55694d651770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 -- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d673e00 msgr2=0x55694d651cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d673e00 0x55694d651cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 -- 172.21.15.17:0/1489613216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55694d3946b0 con 0x55694d650f50 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.929+0000 7feafed9d640 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7feaf00316f0 tx=0x7feaf0048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2400cb38d37e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feadffff640 1 -- 172.21.15.17:0/1489613216 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf004f040 con 0x55694d650f50 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feadffff640 1 -- 172.21.15.17:0/1489613216 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf004aa40 con 0x55694d650f50 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feadffff640 1 -- 172.21.15.17:0/1489613216 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf003d440 con 0x55694d650f50 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 -- 172.21.15.17:0/1489613216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 msgr2=0x55694d64a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7feaf00316f0 tx=0x7feaf0048000 comp rx=0 tx=0).stop 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 -- 172.21.15.17:0/1489613216 shutdown_connections 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d673e00 0x55694d651cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d651330 0x55694d651770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 --2- 172.21.15.17:0/1489613216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64a700 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 -- 172.21.15.17:0/1489613216 >> 172.21.15.17:0/1489613216 conn(0x55694d64f4b0 msgr2=0x55694d64f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 -- 172.21.15.17:0/1489613216 shutdown_connections 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.930+0000 7feb07bfe740 1 -- 172.21.15.17:0/1489613216 wait complete. 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 -- start start 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 Processor -- start 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d66f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d690650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d677bf0 con 0x55694d650f50 2024-03-06T07:27:05.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694d678e20 con 0x55694d651330 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694d672270 con 0x55694d673e00 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.931+0000 7feb07bfe740 10 client.0 _ll_register_callbacks cb 0x7feaf0031c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46196/0 (socket says 172.21.15.17:46196) 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 -- 172.21.15.17:0/426899225 learned_addr learned my addr 172.21.15.17:0/426899225 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaff59e640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d690650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafe59c640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d66f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 -- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 msgr2=0x55694d690650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d690650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 -- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 msgr2=0x55694d66f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d66f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 -- 172.21.15.17:0/426899225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55694d700830 con 0x55694d650f50 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafe59c640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d66f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:05.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feafed9d640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7feaf005f040 tx=0x7feaf0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72b10829c15473 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaddffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf006c060 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7feaf004a840 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaf004aa40 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=166) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaf004f040 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feb07bfe740 10 client.9720 fetch_fsmap learned FSMap version 166 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=166,monmap=2+}) v3 -- 0x55694d66ee80 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.932+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf004aa40 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 166) v1 ==== 3314+0+0 (secure 0 0 0) 0x7feaf0066070 con 0x55694d650f50 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 10 client.9720 fetch_fsmap finished waiting for FSMap version 166 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 10 client.9720 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 20 client.9720 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d690d90 0x55694d6fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 4 client.9720 mds_command: new command op to 9660 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:05.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 --> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55694d5bcf40 con 0x55694d690d90 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feafe59c640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d690d90 0x55694d6fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feaddffb640 10 client.9720 ms_handle_connect on v2:172.21.15.17:6834/1440163522 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feafe59c640 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d690d90 0x55694d6fd580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2bf2eba602f63e4 server_cookie=9b8d7563d9f88602 in_seq=0 out_seq=0 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feaddffb640 1 -- 172.21.15.17:0/426899225 <== mds.0 v2:172.21.15.17:6834/1440163522 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55694d5bcf40 con 0x55694d690d90 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feaddffb640 10 client.9720 handle_command_reply: tid=0 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 client.9720 shutdown 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 msgr2=0x55694d64e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7feaf005f040 tx=0x7feaf0009190 comp rx=0 tx=0).stop 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 shutdown_connections 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55694d673e00 0x55694d690650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:6834/1440163522,v1:172.21.15.17:6835/1440163522] conn(0x55694d690d90 0x55694d6fd580 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55694d651330 0x55694d66f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 --2- 172.21.15.17:0/426899225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55694d650f50 0x55694d64e9a0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feaddffb640 0 client.9720 ms_handle_reset on v2:172.21.15.17:6834/1440163522 2024-03-06T07:27:05.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.933+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 >> 172.21.15.17:0/426899225 conn(0x55694d64f4b0 msgr2=0x55694d64a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 shutdown_connections 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb07bfe740 1 -- 172.21.15.17:0/426899225 wait complete. 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb07bfe740 20 client.9720 trim_cache size 0 max 16384 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9660, 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 3.0788023390000001, 2024-03-06T07:27:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 166, 2024-03-06T07:27:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 174, 2024-03-06T07:27:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 174, 2024-03-06T07:27:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5.3829628539999996 2024-03-06T07:27:05.096 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 msgr2=0x7feaec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 0x7feaec06d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7feaf0035000 tx=0x7feaf0008030 comp rx=0 tx=0).stop 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 msgr2=0x7feb001e00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7feaf400c460 tx=0x7feaf400c920 comp rx=0 tx=0).stop 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 shutdown_connections 2024-03-06T07:27:05.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feaec06b4d0 0x7feaec06d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb00158770 0x7feb001e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb00073270 0x7feb001e00b0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 --2- 172.21.15.17:0/1982611798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb00072d20 0x7feb001ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 >> 172.21.15.17:0/1982611798 conn(0x7feb0006b970 msgr2=0x7feb00158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 shutdown_connections 2024-03-06T07:27:05.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:04.934+0000 7feb05015640 1 -- 172.21.15.17:0/1982611798 wait complete. 2024-03-06T07:27:05.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.198+0000 7fe40986f640 1 Processor -- start 2024-03-06T07:27:05.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.198+0000 7fe40986f640 1 -- start start 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe40986f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe404158db0 0x7fe4040c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe40986f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 0x7fe40415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe40986f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe404155110 con 0x7fe4040c5330 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe40986f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4041559e0 con 0x7fe404158db0 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe40986f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040c8f40 con 0x7fe4040c88b0 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe3fa7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe404158db0 0x7fe4040c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe3fa7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe404158db0 0x7fe4040c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46208/0 (socket says 172.21.15.17:46208) 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.199+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/1184231408 learned_addr learned my addr 172.21.15.17:0/1184231408 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe4037fe640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 0x7fe40415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/1184231408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 msgr2=0x7fe40415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe3fa7fc640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 0x7fe40415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/1184231408 >> v1:172.21.15.163:6789/0 conn(0x7fe4040c5330 legacy=0x7fe4041586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/1184231408 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe404155390 con 0x7fe404158db0 2024-03-06T07:27:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.200+0000 7fe4037fe640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 0x7fe40415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:05.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe3fa7fc640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe404158db0 0x7fe4040c5710 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d88876126f3197 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe4027fc640 1 -- 172.21.15.17:0/1184231408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0317c0 con 0x7fe404158db0 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe4027fc640 1 -- 172.21.15.17:0/1184231408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec037070 con 0x7fe404158db0 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe4027fc640 1 -- 172.21.15.17:0/1184231408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec031b80 con 0x7fe404158db0 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 -- 172.21.15.17:0/1184231408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe404158db0 msgr2=0x7fe4040c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe404158db0 0x7fe4040c5710 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec02b860 comp rx=0 tx=0).stop 2024-03-06T07:27:05.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 -- 172.21.15.17:0/1184231408 shutdown_connections 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c88b0 0x7fe40415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 --2- 172.21.15.17:0/1184231408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe404158db0 0x7fe4040c5710 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 -- 172.21.15.17:0/1184231408 >> 172.21.15.17:0/1184231408 conn(0x7fe4040ac560 msgr2=0x7fe4040ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 -- 172.21.15.17:0/1184231408 shutdown_connections 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.201+0000 7fe40986f640 1 -- 172.21.15.17:0/1184231408 wait complete. 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.202+0000 7fe40986f640 1 Processor -- start 2024-03-06T07:27:05.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 -- start start 2024-03-06T07:27:05.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 0x7fe40414edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 0x7fe4040c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041529d0 con 0x7fe4040c88b0 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40406d090 con 0x7fe40414f7d0 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe40986f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe404152b50 con 0x7fe4040c5330 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe3fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe3fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46222/0 (socket says 172.21.15.17:46222) 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.203+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/2231803803 learned_addr learned my addr 172.21.15.17:0/2231803803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe4037fe640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 0x7fe4040c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe402ffd640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 0x7fe40414edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 msgr2=0x7fe40414edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 0x7fe40414edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 msgr2=0x7fe4040c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 0x7fe4040c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 -- 172.21.15.17:0/2231803803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40414cd10 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe4037fe640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 0x7fe4040c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe402ffd640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 0x7fe40414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe3fa7fc640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032860 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec032a00 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032df0 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40414cf80 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.204+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40414f2f0 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.205+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40404f330 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.206+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ec04c060 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.206+0000 7fe400ff9640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 0x7fe3e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.206+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fe3ec0ac120 con 0x7fe4040c88b0 2024-03-06T07:27:05.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.206+0000 7fe402ffd640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 0x7fe3e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.207+0000 7fe402ffd640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 0x7fe3e406d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4003eb0 tx=0x7fe3f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.214+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe3ec040070 con 0x7fe4040c88b0 2024-03-06T07:27:05.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.434+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe404154b70 con 0x7fe4040c88b0 2024-03-06T07:27:05.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.435+0000 7fe400ff9640 1 -- 172.21.15.17:0/2231803803 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7fe404155110 con 0x7fe4040c88b0 2024-03-06T07:27:05.436 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:05.436 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:05.437 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:05.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 msgr2=0x7fe3e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 0x7fe3e406d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe3f4003eb0 tx=0x7fe3f4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 msgr2=0x7fe404157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 shutdown_connections 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe3e406b4d0 0x7fe3e406d990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe40414f7d0 0x7fe4040c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4040c88b0 0x7fe404157cc0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 --2- 172.21.15.17:0/2231803803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4040c5330 0x7fe40414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 >> 172.21.15.17:0/2231803803 conn(0x7fe4040ac560 msgr2=0x7fe4041531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 shutdown_connections 2024-03-06T07:27:05.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.437+0000 7fe40986f640 1 -- 172.21.15.17:0/2231803803 wait complete. 2024-03-06T07:27:05.449 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:27:05.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:05.488 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:05.537 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:27:05.537 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:27:05.538 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:05.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:27:05.553 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:27:05.553 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:05.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:27:05.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:05.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.691+0000 7f26768af640 1 Processor -- start 2024-03-06T07:27:05.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.691+0000 7f26768af640 1 -- start start 2024-03-06T07:27:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.692+0000 7f26768af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26701506b0 0x7f2670153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.692+0000 7f26768af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f26700702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f26768af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2670070800 con 0x7f26700acc00 2024-03-06T07:27:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f26768af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2670156450 con 0x7f26701506b0 2024-03-06T07:27:05.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f26768af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2670156a10 con 0x7f2670156ea0 2024-03-06T07:27:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266ffff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26701506b0 0x7f2670153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f26700702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f2674e4c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f26700acc00 0x7f26700c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:60578/0 (socket says 172.21.15.17:60578) 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f2674e4c640 1 -- 172.21.15.17:0/404007225 learned_addr learned my addr 172.21.15.17:0/404007225 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266effd640 1 -- 172.21.15.17:0/404007225 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411986184 0 0) 0x7f2670070800 con 0x7f26700acc00 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266effd640 1 -- 172.21.15.17:0/404007225 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2658003520 con 0x7f26700acc00 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266f7fe640 1 -- 172.21.15.17:0/404007225 >> v2:172.21.15.163:3300/0 conn(0x7f26701506b0 msgr2=0x7f2670153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266f7fe640 1 --2- 172.21.15.17:0/404007225 >> v2:172.21.15.163:3300/0 conn(0x7f26701506b0 0x7f2670153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266f7fe640 1 -- 172.21.15.17:0/404007225 >> v1:172.21.15.163:6789/0 conn(0x7f26700acc00 legacy=0x7f26700c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.693+0000 7f266f7fe640 1 -- 172.21.15.17:0/404007225 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670070aa0 con 0x7f2670156ea0 2024-03-06T07:27:05.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.694+0000 7f266f7fe640 1 --2- 172.21.15.17:0/404007225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f26700702c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26701565d0 tx=0x7f266402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5d00610c13fdfb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.694+0000 7f266effd640 1 -- 172.21.15.17:0/404007225 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664035020 con 0x7f2670156ea0 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f266effd640 1 -- 172.21.15.17:0/404007225 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664031800 con 0x7f2670156ea0 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f266effd640 1 -- 172.21.15.17:0/404007225 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664031bb0 con 0x7f2670156ea0 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 -- 172.21.15.17:0/404007225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 msgr2=0x7f26700702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 --2- 172.21.15.17:0/404007225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f26700702c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f26701565d0 tx=0x7f266402b8a0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 -- 172.21.15.17:0/404007225 shutdown_connections 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 --2- 172.21.15.17:0/404007225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f26700702c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 --2- 172.21.15.17:0/404007225 >> v2:172.21.15.163:3300/0 conn(0x7f26701506b0 0x7f2670153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 -- 172.21.15.17:0/404007225 >> 172.21.15.17:0/404007225 conn(0x7f267006b1a0 msgr2=0x7f267006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 -- 172.21.15.17:0/404007225 shutdown_connections 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.695+0000 7f26768af640 1 -- 172.21.15.17:0/404007225 wait complete. 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.696+0000 7f26768af640 1 Processor -- start 2024-03-06T07:27:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.696+0000 7f26768af640 1 -- start start 2024-03-06T07:27:05.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26700acc00 0x7f26701ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f2670145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26700c5bb0 con 0x7f26701506b0 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26701566a0 con 0x7f26700acc00 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f26768af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26700c5d30 con 0x7f2670156ea0 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46224/0 (socket says 172.21.15.17:46224) 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 -- 172.21.15.17:0/2554173938 learned_addr learned my addr 172.21.15.17:0/2554173938 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266ffff640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26700acc00 0x7f26701ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f2674e4c640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f2670145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 -- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 msgr2=0x7f2670145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f2670145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 -- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26700acc00 msgr2=0x7f26701ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26700acc00 0x7f26701ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.697+0000 7f266f7fe640 1 -- 172.21.15.17:0/2554173938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700c4f90 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f266f7fe640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f2664000c00 tx=0x7f26640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664035040 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664039cf0 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267006c530 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664039ea0 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.698+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26700c4cb0 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.699+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267004f330 con 0x7f26701506b0 2024-03-06T07:27:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.699+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2664038020 con 0x7f26701506b0 2024-03-06T07:27:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.700+0000 7f266cff9640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 0x7f265c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.700+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f26640ac420 con 0x7f26701506b0 2024-03-06T07:27:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.700+0000 7f266ffff640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 0x7f265c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.700+0000 7f266ffff640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 0x7f265c06d9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2660009b40 tx=0x7f2660009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.704+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f266407b040 con 0x7f26701506b0 2024-03-06T07:27:05.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.924+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26700700a0 con 0x7f26701506b0 2024-03-06T07:27:05.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.924+0000 7f266cff9640 1 -- 172.21.15.17:0/2554173938 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f266407a940 con 0x7f26701506b0 2024-03-06T07:27:05.926 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:05.926 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:05.927 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 msgr2=0x7f265c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 0x7f265c06d9c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2660009b40 tx=0x7f2660009190 comp rx=0 tx=0).stop 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 msgr2=0x7f26700c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f2664000c00 tx=0x7f26640046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 shutdown_connections 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f265c06b500 0x7f265c06d9c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2670156ea0 0x7f2670145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26701506b0 0x7f26700c7d80 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 --2- 172.21.15.17:0/2554173938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26700acc00 0x7f26701ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 >> 172.21.15.17:0/2554173938 conn(0x7f267006b1a0 msgr2=0x7f267014d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 shutdown_connections 2024-03-06T07:27:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:05.926+0000 7f26768af640 1 -- 172.21.15.17:0/2554173938 wait complete. 2024-03-06T07:27:05.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:06.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.038+0000 7f2f433c9640 1 Processor -- start 2024-03-06T07:27:06.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.038+0000 7f2f433c9640 1 -- start start 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f433c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f3c0c88c0 0x7f2f3c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f433c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f433c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c0c73d0 con 0x7f2f3c14f360 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f433c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c8dd0 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f433c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0be2e0 con 0x7f2f3c158b60 2024-03-06T07:27:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f3c0c88c0 0x7f2f3c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2f3c0c88c0 0x7f2f3c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46228/0 (socket says 172.21.15.17:46228) 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 -- 172.21.15.17:0/66188950 learned_addr learned my addr 172.21.15.17:0/66188950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f41966640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 -- 172.21.15.17:0/66188950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 msgr2=0x7f2f3c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 -- 172.21.15.17:0/66188950 >> v1:172.21.15.163:6789/0 conn(0x7f2f3c14f360 legacy=0x7f2f3c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.039+0000 7f2f40964640 1 -- 172.21.15.17:0/66188950 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0c7670 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.040+0000 7f2f40964640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c155c10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009870 tx=0x7f2f3002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72930eaa5f38010 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.040+0000 7f2f3bfff640 1 -- 172.21.15.17:0/66188950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30035020 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.040+0000 7f2f3bfff640 1 -- 172.21.15.17:0/66188950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30031640 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.040+0000 7f2f3bfff640 1 -- 172.21.15.17:0/66188950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3002d040 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 -- 172.21.15.17:0/66188950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 msgr2=0x7f2f3c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c155c10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009870 tx=0x7f2f3002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 -- 172.21.15.17:0/66188950 shutdown_connections 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 --2- 172.21.15.17:0/66188950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c155c10 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 -- 172.21.15.17:0/66188950 >> 172.21.15.17:0/66188950 conn(0x7f2f3c0ac570 msgr2=0x7f2f3c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 -- 172.21.15.17:0/66188950 shutdown_connections 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.041+0000 7f2f433c9640 1 -- 172.21.15.17:0/66188950 wait complete. 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.042+0000 7f2f433c9640 1 Processor -- start 2024-03-06T07:27:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.042+0000 7f2f433c9640 1 -- start start 2024-03-06T07:27:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 0x7f2f3c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c1ead10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c5a50 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c7c30 con 0x7f2f3c14f360 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f433c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c0c5bd0 con 0x7f2f3c158b60 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46244/0 (socket says 172.21.15.17:46244) 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 -- 172.21.15.17:0/2340871090 learned_addr learned my addr 172.21.15.17:0/2340871090 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f40964640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 0x7f2f3c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41966640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c1ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 -- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 msgr2=0x7f2f3c1ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c1ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 -- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 msgr2=0x7f2f3c14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 0x7f2f3c14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 -- 172.21.15.17:0/2340871090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c152590 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41966640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c1ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f40964640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 0x7f2f3c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f41165640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c760 tx=0x7f2f2c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.043+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c016020 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.044+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c006d50 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.044+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c003bb0 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.044+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c0c1b30 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.044+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c14e380 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.045+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c04f330 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.046+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f2c00a850 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.046+0000 7f2f39ffb640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.046+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2f2c089fc0 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.046+0000 7f2f40964640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.047+0000 7f2f40964640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2f30036000 tx=0x7f2f3003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.051+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f2c020020 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.230+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f3c155ed0 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.232+0000 7f2f39ffb640 1 -- 172.21.15.17:0/2340871090 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17097 (secure 0 0 0) 0x7f2f2c019310 con 0x7f2f3c0c88c0 2024-03-06T07:27:06.233 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:06.233 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":174,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:54.491601+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.0","mappings":[{"from":6,"to":4}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 msgr2=0x7f2f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2f30036000 tx=0x7f2f3003c040 comp rx=0 tx=0).stop 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 msgr2=0x7f2f3c158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c760 tx=0x7f2f2c00cc20 comp rx=0 tx=0).stop 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 shutdown_connections 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f3c158b60 0x7f2f3c1ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f3c14f360 0x7f2f3c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 --2- 172.21.15.17:0/2340871090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f3c0c88c0 0x7f2f3c158420 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 >> 172.21.15.17:0/2340871090 conn(0x7f2f3c0ac570 msgr2=0x7f2f3c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 shutdown_connections 2024-03-06T07:27:06.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.234+0000 7f2f433c9640 1 -- 172.21.15.17:0/2340871090 wait complete. 2024-03-06T07:27:06.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:26:36.885685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:26:37.580691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:06.248 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.350+0000 7f3cd92fb640 1 Processor -- start 2024-03-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.350+0000 7f3cd92fb640 1 -- start start 2024-03-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd92fb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3cd414bcc0 0x7f3cd4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd92fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cd40c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd92fb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40c75e0 con 0x7f3cd40c5910 2024-03-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd92fb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd40c7f10 con 0x7f3cd414bcc0 2024-03-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd92fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c78c0 con 0x7f3cd40be150 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3cd40c5910 0x7f3cd4158a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53366/0 (socket says 172.21.15.17:53366) 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1470067576 learned_addr learned my addr 172.21.15.17:0/1470067576 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cca7fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cd40c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459877756 0 0) 0x7f3cd40c75e0 con 0x7f3cd40c5910 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1470067576 >> v2:172.21.15.163:3300/0 conn(0x7f3cd414bcc0 0x7f3cd4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc0003520 con 0x7f3cd40c5910 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209493639 0 0) 0x7f3cc0003520 con 0x7f3cd40c5910 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 msgr2=0x7f3cd40c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cd40c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 >> v2:172.21.15.163:3300/0 conn(0x7f3cd414bcc0 msgr2=0x7f3cd4155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 --2- 172.21.15.17:0/1470067576 >> v2:172.21.15.163:3300/0 conn(0x7f3cd414bcc0 0x7f3cd4155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40c7ba0 con 0x7f3cd40c5910 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3ccc0019f0 con 0x7f3cd40c5910 2024-03-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 >> v1:172.21.15.17:6789/0 conn(0x7f3cd40c5910 legacy=0x7f3cd4158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cc00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc0003f30 0x7f3cc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c78c0 con 0x7f3cc0004d40 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000b9c0 con 0x7f3cc0003f30 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.351+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc000bb70 con 0x7f3cd40be150 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cc00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cca7fc640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc0003f30 0x7f3cc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 msgr2=0x7f3cc00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd40be150 0x7f3cc00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc0003f30 msgr2=0x7f3cc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cc0003f30 0x7f3cc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.352+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1470067576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0003520 con 0x7f3cc0004d40 2024-03-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00a8a0 tx=0x7f3cc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9210ffe2ec14a47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00a4c0 con 0x7f3cc0004d40 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc034630 con 0x7f3cc0004d40 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd27fc640 1 -- 172.21.15.17:0/1470067576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc034a20 con 0x7f3cc0004d40 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1470067576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc0004d40 msgr2=0x7f3cc00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1470067576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cc0004d40 0x7f3cc00051c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00a8a0 tx=0x7f3cc0005830 comp rx=0 tx=0).stop 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1470067576 reap_dead start 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1470067576 shutdown_connections 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1470067576 >> 172.21.15.17:0/1470067576 conn(0x7f3cd40ac600 msgr2=0x7f3cd40aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1470067576 shutdown_connections 2024-03-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.353+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1470067576 wait complete. 2024-03-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.354+0000 7f3cd92fb640 1 Processor -- start 2024-03-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 -- start start 2024-03-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 0x7f3cd41eb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 0x7f3cd414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c5550 con 0x7f3cd40c0e30 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c53d0 con 0x7f3cd414acd0 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd92fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40c56d0 con 0x7f3cd414c310 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46264/0 (socket says 172.21.15.17:46264) 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.355+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1361343329 learned_addr learned my addr 172.21.15.17:0/1361343329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cca7fc640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 0x7f3cd414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 0x7f3cd41eb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 msgr2=0x7f3cd41eb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 0x7f3cd41eb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 msgr2=0x7f3cd414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 0x7f3cd414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 -- 172.21.15.17:0/1361343329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40c17f0 con 0x7f3cd40c0e30 2024-03-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cca7fc640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 0x7f3cd414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 0x7f3cd41eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd37fe640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00a990 tx=0x7f3cd40c7d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc045070 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc03a040 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40bf670 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00a4c0 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.356+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40c0630 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.357+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd404f330 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.357+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ccc048070 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.358+0000 7f3ccbfff640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 0x7f3cb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.358+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3ccc0abde0 con 0x7f3cd40c0e30 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.358+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 0x7f3cb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.359+0000 7f3cd2ffd640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 0x7f3cb406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009b30 tx=0x7f3cc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.362+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ccc07a3b0 con 0x7f3cd40c0e30 2024-03-06T07:27:06.529 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:27:06.528+0000 7f5f2815b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.541+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cd40c83e0 con 0x7f3cd40c0e30 2024-03-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.542+0000 7f3ccbfff640 1 -- 172.21.15.17:0/1361343329 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ccc07e450 con 0x7f3cd40c0e30 2024-03-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 msgr2=0x7f3cb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 0x7f3cb406d9c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009b30 tx=0x7f3cc4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 msgr2=0x7f3cd40c12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00a990 tx=0x7f3cd40c7d70 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 shutdown_connections 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3cb406b500 0x7f3cb406d9c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3cd40c0e30 0x7f3cd40c12b0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3cd414acd0 0x7f3cd414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 --2- 172.21.15.17:0/1361343329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3cd414c310 0x7f3cd41eb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 >> 172.21.15.17:0/1361343329 conn(0x7f3cd40ac600 msgr2=0x7f3cd4157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 shutdown_connections 2024-03-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.544+0000 7f3cd92fb640 1 -- 172.21.15.17:0/1361343329 wait complete. 2024-03-06T07:27:06.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.669+0000 7f68d8a0e640 1 Processor -- start 2024-03-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.669+0000 7f68d8a0e640 1 -- start start 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d8a0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d40c67f0 0x7f68d4155220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d8a0e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 0x7f68d4155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d8a0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d4152cb0 con 0x7f68d41528d0 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d8a0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d4155ca0 con 0x7f68d40c67f0 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d8a0e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40c38b0 con 0x7f68d40c4ef0 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d40c67f0 0x7f68d4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68d41528d0 0x7f68d40724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53376/0 (socket says 172.21.15.17:53376) 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68d40c67f0 0x7f68d4155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46272/0 (socket says 172.21.15.17:46272) 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d259c640 1 -- 172.21.15.17:0/3248107110 learned_addr learned my addr 172.21.15.17:0/3248107110 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d2d9d640 1 --2- 172.21.15.17:0/3248107110 >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 0x7f68d4155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 -- 172.21.15.17:0/3248107110 >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 msgr2=0x7f68d4155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 --2- 172.21.15.17:0/3248107110 >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 0x7f68d4155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 -- 172.21.15.17:0/3248107110 >> v1:172.21.15.17:6789/0 conn(0x7f68d41528d0 legacy=0x7f68d40724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68cbfff640 1 -- 172.21.15.17:0/3248107110 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d41594f0 con 0x7f68d40c67f0 2024-03-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.670+0000 7f68d2d9d640 1 --2- 172.21.15.17:0/3248107110 >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 0x7f68d4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.671+0000 7f68cbfff640 1 --2- 172.21.15.17:0/3248107110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d4155220 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f68d40aa630 tx=0x7f68bc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28fff332d3991ae server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.671+0000 7f68d1d9b640 1 -- 172.21.15.17:0/3248107110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc034070 con 0x7f68d40c67f0 2024-03-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.671+0000 7f68d1d9b640 1 -- 172.21.15.17:0/3248107110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc030610 con 0x7f68d40c67f0 2024-03-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.671+0000 7f68d1d9b640 1 -- 172.21.15.17:0/3248107110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0309e0 con 0x7f68d40c67f0 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 -- 172.21.15.17:0/3248107110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 msgr2=0x7f68d4155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/3248107110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d4155220 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f68d40aa630 tx=0x7f68bc02b960 comp rx=0 tx=0).stop 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 -- 172.21.15.17:0/3248107110 shutdown_connections 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/3248107110 >> v2:172.21.15.163:3300/0 conn(0x7f68d40c4ef0 0x7f68d4155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/3248107110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d4155220 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 -- 172.21.15.17:0/3248107110 >> 172.21.15.17:0/3248107110 conn(0x7f68d40aa6a0 msgr2=0x7f68d40aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 -- 172.21.15.17:0/3248107110 shutdown_connections 2024-03-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.672+0000 7f68d8a0e640 1 -- 172.21.15.17:0/3248107110 wait complete. 2024-03-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.673+0000 7f68d8a0e640 1 Processor -- start 2024-03-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.673+0000 7f68d8a0e640 1 -- start start 2024-03-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 0x7f68d41eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41528d0 0x7f68d4151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40a8250 con 0x7f68d40c67f0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40c3390 con 0x7f68d41528d0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d8a0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41591a0 con 0x7f68d40c4ef0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46276/0 (socket says 172.21.15.17:46276) 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 -- 172.21.15.17:0/1465562892 learned_addr learned my addr 172.21.15.17:0/1465562892 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d259c640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 0x7f68d41eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d2d9d640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41528d0 0x7f68d4151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 -- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 msgr2=0x7f68d41eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 0x7f68d41eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 -- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41528d0 msgr2=0x7f68d4151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41528d0 0x7f68d4151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68cbfff640 1 -- 172.21.15.17:0/1465562892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d41eb9e0 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.674+0000 7f68d259c640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 0x7f68d41eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68cbfff640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009890 tx=0x7f68bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc046020 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc039a10 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d40ae300 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc034040 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.675+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d41eaca0 con 0x7f68d40c67f0 2024-03-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.676+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d41eaab0 con 0x7f68d40c67f0 2024-03-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.676+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68bc037070 con 0x7f68d40c67f0 2024-03-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.677+0000 7f68caffd640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 0x7f68b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.677+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f68bc0ac1d0 con 0x7f68d40c67f0 2024-03-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.677+0000 7f68d259c640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 0x7f68b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.678+0000 7f68d259c640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 0x7f68b406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000f60 tx=0x7f68c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.685+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68bc050020 con 0x7f68d40c67f0 2024-03-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.866+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68d404f330 con 0x7f68d40c67f0 2024-03-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.867+0000 7f68caffd640 1 -- 172.21.15.17:0/1465562892 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v174) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68bc039e20 con 0x7f68d40c67f0 2024-03-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 msgr2=0x7f68b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 0x7f68b406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000f60 tx=0x7f68c4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 msgr2=0x7f68d41ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009890 tx=0x7f68bc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 shutdown_connections 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f68b406b4d0 0x7f68b406d990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d41528d0 0x7f68d4151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68d40c67f0 0x7f68d41ee080 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 --2- 172.21.15.17:0/1465562892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d40c4ef0 0x7f68d41eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 >> 172.21.15.17:0/1465562892 conn(0x7f68d40aa6a0 msgr2=0x7f68d40c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 shutdown_connections 2024-03-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.870+0000 7f68d8a0e640 1 -- 172.21.15.17:0/1465562892 wait complete. 2024-03-06T07:27:06.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.984+0000 7fb4b34af640 1 Processor -- start 2024-03-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.984+0000 7fb4b34af640 1 -- start start 2024-03-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.985+0000 7fb4b34af640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b34af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4ac070a50 0x7fb4ac156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b34af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 0x7fb4ac0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b34af640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ac070ee0 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b34af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4ac156940 con 0x7fb4ac070a50 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b34af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac157310 con 0x7fb4ac152e40 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46290/0 (socket says 172.21.15.17:46290) 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 -- 172.21.15.17:0/1410201717 learned_addr learned my addr 172.21.15.17:0/1410201717 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b0a4a640 1 --2- 172.21.15.17:0/1410201717 >> v2:172.21.15.163:3300/0 conn(0x7fb4ac070a50 0x7fb4ac156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b1a4c640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 0x7fb4ac0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 -- 172.21.15.17:0/1410201717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 msgr2=0x7fb4ac0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 0x7fb4ac0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 -- 172.21.15.17:0/1410201717 >> v2:172.21.15.163:3300/0 conn(0x7fb4ac070a50 msgr2=0x7fb4ac156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 --2- 172.21.15.17:0/1410201717 >> v2:172.21.15.163:3300/0 conn(0x7fb4ac070a50 0x7fb4ac156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b124b640 1 -- 172.21.15.17:0/1410201717 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0c80b0 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.986+0000 7fb4b1a4c640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 0x7fb4ac0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.987+0000 7fb4b124b640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0a7be0 tx=0x7fb49c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0c427b359233bd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.987+0000 7fb4a3fff640 1 -- 172.21.15.17:0/1410201717 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c02e070 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.987+0000 7fb4a3fff640 1 -- 172.21.15.17:0/1410201717 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c037070 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.987+0000 7fb4a3fff640 1 -- 172.21.15.17:0/1410201717 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031a00 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 -- 172.21.15.17:0/1410201717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac0c4c60 msgr2=0x7fb4ac0c7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0a7be0 tx=0x7fb49c02b880 comp rx=0 tx=0).stop 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 -- 172.21.15.17:0/1410201717 shutdown_connections 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac152e40 0x7fb4ac0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 --2- 172.21.15.17:0/1410201717 >> v2:172.21.15.163:3300/0 conn(0x7fb4ac070a50 0x7fb4ac156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 --2- 172.21.15.17:0/1410201717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac0c4c60 0x7fb4ac0c7a20 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 -- 172.21.15.17:0/1410201717 >> 172.21.15.17:0/1410201717 conn(0x7fb4ac0aa6a0 msgr2=0x7fb4ac0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 -- 172.21.15.17:0/1410201717 shutdown_connections 2024-03-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.988+0000 7fb4b34af640 1 -- 172.21.15.17:0/1410201717 wait complete. 2024-03-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.989+0000 7fb4b34af640 1 Processor -- start 2024-03-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.990+0000 7fb4b34af640 1 -- start start 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4ac070a50 0x7fb4ac145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac0c4c60 0x7fb4ac146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac1532d0 con 0x7fb4ac152e40 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac1565d0 con 0x7fb4ac070a50 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac072a10 con 0x7fb4ac0c4c60 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46298/0 (socket says 172.21.15.17:46298) 2024-03-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 -- 172.21.15.17:0/858336844 learned_addr learned my addr 172.21.15.17:0/858336844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b0a4a640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac0c4c60 0x7fb4ac146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b124b640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4ac070a50 0x7fb4ac145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 -- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac0c4c60 msgr2=0x7fb4ac146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac0c4c60 0x7fb4ac146790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 -- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4ac070a50 msgr2=0x7fb4ac145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4ac070a50 0x7fb4ac145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 -- 172.21.15.17:0/858336844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac14bd30 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b1a4c640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007e70 tx=0x7fb4a400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a401a070 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a4003bb0 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac14ba50 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.991+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4004030 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.992+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac148c40 con 0x7fb4ac152e40 2024-03-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.992+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac04f330 con 0x7fb4ac152e40 2024-03-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.993+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4a401d070 con 0x7fb4ac152e40 2024-03-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.994+0000 7fb4a1ffb640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 0x7fb49006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.994+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb4a408a5e0 con 0x7fb4ac152e40 2024-03-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.994+0000 7fb4b124b640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 0x7fb49006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.994+0000 7fb4b124b640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 0x7fb49006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb49c036000 tx=0x7fb49c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:06.998+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb4a4010070 con 0x7fb4ac152e40 2024-03-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.178+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4ac0c2fd0 con 0x7fb4ac152e40 2024-03-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.179+0000 7fb4a1ffb640 1 -- 172.21.15.17:0/858336844 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v174) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb4a4059310 con 0x7fb4ac152e40 2024-03-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 msgr2=0x7fb49006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 0x7fb49006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb49c036000 tx=0x7fb49c03d040 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 msgr2=0x7fb4ac1e9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007e70 tx=0x7fb4a400d4a0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 shutdown_connections 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb49006b4d0 0x7fb49006d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4ac152e40 0x7fb4ac1e9010 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4ac0c4c60 0x7fb4ac146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 --2- 172.21.15.17:0/858336844 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4ac070a50 0x7fb4ac145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 >> 172.21.15.17:0/858336844 conn(0x7fb4ac0aa6a0 msgr2=0x7fb4ac070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 shutdown_connections 2024-03-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.181+0000 7fb4b34af640 1 -- 172.21.15.17:0/858336844 wait complete. 2024-03-06T07:27:07.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.296+0000 7f8c6aba0640 1 Processor -- start 2024-03-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.296+0000 7f8c6aba0640 1 -- start start 2024-03-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6aba0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c64152800 0x7f8c64072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6aba0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c6406fec0 0x7f8c640c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6aba0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c64158970 con 0x7f8c640c8b00 2024-03-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6aba0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c64158c50 con 0x7f8c6406fec0 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6aba0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c64158f30 con 0x7f8c64152800 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6939d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c6406fec0 0x7f8c640c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6a39f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8c640c8b00 0x7f8c641552e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53384/0 (socket says 172.21.15.17:53384) 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c6a39f640 1 -- 172.21.15.17:0/1517534907 learned_addr learned my addr 172.21.15.17:0/1517534907 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.297+0000 7f8c69b9e640 1 --2- 172.21.15.17:0/1517534907 >> v2:172.21.15.163:3300/0 conn(0x7f8c64152800 0x7f8c64072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c6939d640 1 -- 172.21.15.17:0/1517534907 >> v2:172.21.15.163:3300/0 conn(0x7f8c64152800 msgr2=0x7f8c64072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1517534907 >> v2:172.21.15.163:3300/0 conn(0x7f8c64152800 0x7f8c64072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c6939d640 1 -- 172.21.15.17:0/1517534907 >> v1:172.21.15.17:6789/0 conn(0x7f8c640c8b00 legacy=0x7f8c641552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c6939d640 1 -- 172.21.15.17:0/1517534907 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64159730 con 0x7f8c6406fec0 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1517534907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640c85c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406ed40 tx=0x7f8c5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2b67efb8bf50c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c68b9c640 1 -- 172.21.15.17:0/1517534907 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540315a0 con 0x7f8c6406fec0 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.298+0000 7f8c68b9c640 1 -- 172.21.15.17:0/1517534907 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54037070 con 0x7f8c6406fec0 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c68b9c640 1 -- 172.21.15.17:0/1517534907 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54031960 con 0x7f8c6406fec0 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1517534907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 msgr2=0x7f8c640c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1517534907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640c85c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406ed40 tx=0x7f8c5402b840 comp rx=0 tx=0).stop 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1517534907 shutdown_connections 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1517534907 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640c85c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1517534907 >> v2:172.21.15.163:3300/0 conn(0x7f8c64152800 0x7f8c64072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1517534907 >> 172.21.15.17:0/1517534907 conn(0x7f8c640aa4e0 msgr2=0x7f8c640aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1517534907 shutdown_connections 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.299+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1517534907 wait complete. 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.300+0000 7f8c6aba0640 1 Processor -- start 2024-03-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.300+0000 7f8c6aba0640 1 -- start start 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 0x7f8c640bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 0x7f8c640be120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64152be0 con 0x7f8c6406fec0 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6406f660 con 0x7f8c64152800 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6aba0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c64152d60 con 0x7f8c640c8b00 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46330/0 (socket says 172.21.15.17:46330) 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 -- 172.21.15.17:0/1613298747 learned_addr learned my addr 172.21.15.17:0/1613298747 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6a39f640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 0x7f8c640be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 0x7f8c640bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 -- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 msgr2=0x7f8c640bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 0x7f8c640bdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 -- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 msgr2=0x7f8c640be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 0x7f8c640be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c69b9e640 1 -- 172.21.15.17:0/1613298747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640bfe60 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6a39f640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 0x7f8c640be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.301+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 0x7f8c640bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c69b9e640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d620 tx=0x7f8c6000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000b620 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60019070 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c641ebe30 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000ba00 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.302+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c640bec70 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.303+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c64069430 con 0x7f8c6406fec0 2024-03-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.303+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c6000bba0 con 0x7f8c6406fec0 2024-03-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.304+0000 7f8c5a7fc640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.304+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f8c6008a790 con 0x7f8c6406fec0 2024-03-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.304+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.305+0000 7f8c6939d640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 0x7f8c4806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c54036000 tx=0x7f8c5403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.308+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c600594c0 con 0x7f8c6406fec0 2024-03-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.526+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c6406f220 con 0x7f8c6406fec0 2024-03-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.527+0000 7f8c5a7fc640 1 -- 172.21.15.17:0/1613298747 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f8c60058dc0 con 0x7f8c6406fec0 2024-03-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 msgr2=0x7f8c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 0x7f8c4806d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8c54036000 tx=0x7f8c5403d040 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 msgr2=0x7f8c640a9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d620 tx=0x7f8c6000dae0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 shutdown_connections 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c64152800 0x7f8c640be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c640c8b00 0x7f8c640bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 --2- 172.21.15.17:0/1613298747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c6406fec0 0x7f8c640a9100 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 >> 172.21.15.17:0/1613298747 conn(0x7f8c640aa4e0 msgr2=0x7f8c640c4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 shutdown_connections 2024-03-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.529+0000 7f8c6aba0640 1 -- 172.21.15.17:0/1613298747 wait complete. 2024-03-06T07:27:07.541 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.641+0000 7fed45466640 1 Processor -- start 2024-03-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.641+0000 7fed45466640 1 -- start start 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.642+0000 7fed45466640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed40152920 0x7fed40155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.642+0000 7fed45466640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 0x7fed40158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.642+0000 7fed45466640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed40156180 con 0x7fed40159460 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.642+0000 7fed45466640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed401587b0 con 0x7fed40152920 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.642+0000 7fed45466640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed400c7760 con 0x7fed400c8a00 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed40152920 0x7fed40155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3e7fc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 0x7fed40158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3f7fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fed40159460 0x7fed400c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:60590/0 (socket says 172.21.15.17:60590) 2024-03-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed40152920 0x7fed40155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:51538/0 (socket says 172.21.15.17:51538) 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3f7fe640 1 -- 172.21.15.17:0/4012271195 learned_addr learned my addr 172.21.15.17:0/4012271195 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3dffb640 1 -- 172.21.15.17:0/4012271195 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280805974 0 0) 0x7fed38001490 con 0x7fed40159460 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3dffb640 1 -- 172.21.15.17:0/4012271195 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed28003520 con 0x7fed40159460 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 -- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 msgr2=0x7fed40158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 0x7fed40158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 -- 172.21.15.17:0/4012271195 >> v1:172.21.15.163:6789/0 conn(0x7fed40159460 legacy=0x7fed400c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3effd640 1 -- 172.21.15.17:0/4012271195 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed400c7a40 con 0x7fed40152920 2024-03-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.643+0000 7fed3e7fc640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 0x7fed40158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.644+0000 7fed3effd640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed40152920 0x7fed40155c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5d534b4899c50f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.644+0000 7fed3dffb640 1 -- 172.21.15.17:0/4012271195 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300315a0 con 0x7fed40152920 2024-03-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed3dffb640 1 -- 172.21.15.17:0/4012271195 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30037070 con 0x7fed40152920 2024-03-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed3dffb640 1 -- 172.21.15.17:0/4012271195 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30031940 con 0x7fed40152920 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 -- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed40152920 msgr2=0x7fed40155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed40152920 0x7fed40155c40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fed300097a0 tx=0x7fed3002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 -- 172.21.15.17:0/4012271195 shutdown_connections 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed400c8a00 0x7fed40158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 --2- 172.21.15.17:0/4012271195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed40152920 0x7fed40155c40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 -- 172.21.15.17:0/4012271195 >> 172.21.15.17:0/4012271195 conn(0x7fed400ac560 msgr2=0x7fed400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 -- 172.21.15.17:0/4012271195 shutdown_connections 2024-03-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.645+0000 7fed45466640 1 -- 172.21.15.17:0/4012271195 wait complete. 2024-03-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.646+0000 7fed45466640 1 Processor -- start 2024-03-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.646+0000 7fed45466640 1 -- start start 2024-03-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed400c8a00 0x7fed401de820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed40152920 0x7fed4014b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4006d030 con 0x7fed401481c0 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed40158a00 con 0x7fed400c8a00 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed45466640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed400c5a40 con 0x7fed40152920 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46332/0 (socket says 172.21.15.17:46332) 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3f7fe640 1 -- 172.21.15.17:0/3121048540 learned_addr learned my addr 172.21.15.17:0/3121048540 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3effd640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed400c8a00 0x7fed401de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3e7fc640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed40152920 0x7fed4014b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3f7fe640 1 -- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed40152920 msgr2=0x7fed4014b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.647+0000 7fed3f7fe640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed40152920 0x7fed4014b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed3f7fe640 1 -- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed400c8a00 msgr2=0x7fed401de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed3f7fe640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed400c8a00 0x7fed401de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed3f7fe640 1 -- 172.21.15.17:0/3121048540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed401ebc70 con 0x7fed401481c0 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed3f7fe640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fed3800cad0 tx=0x7fed3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38017070 con 0x7fed401481c0 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3800ac30 con 0x7fed401481c0 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4014a5e0 con 0x7fed401481c0 2024-03-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38012400 con 0x7fed401481c0 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.648+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4014aba0 con 0x7fed401481c0 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.649+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4004f330 con 0x7fed401481c0 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.649+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed3801a070 con 0x7fed401481c0 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.650+0000 7fed277fe640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 0x7fed2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.650+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fed38089ee0 con 0x7fed401481c0 2024-03-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.650+0000 7fed3effd640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 0x7fed2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.651+0000 7fed3effd640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 0x7fed2c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fed30036000 tx=0x7fed3003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.654+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fed40156180 con 0x7fed401481c0 2024-03-06T07:27:07.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.938+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed400ac340 con 0x7fed401481c0 2024-03-06T07:27:07.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.939+0000 7fed277fe640 1 -- 172.21.15.17:0/3121048540 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17097 (secure 0 0 0) 0x7fed380585c0 con 0x7fed401481c0 2024-03-06T07:27:07.941 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:07.941 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":174,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:54.491601+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.0","mappings":[{"from":6,"to":4}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 msgr2=0x7fed2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 0x7fed2c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fed30036000 tx=0x7fed3003d040 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 msgr2=0x7fed40149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fed3800cad0 tx=0x7fed3800cf90 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 shutdown_connections 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fed2c06b4d0 0x7fed2c06d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fed401481c0 0x7fed40149060 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed40152920 0x7fed4014b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 --2- 172.21.15.17:0/3121048540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed400c8a00 0x7fed401de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 >> 172.21.15.17:0/3121048540 conn(0x7fed400ac560 msgr2=0x7fed40150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 shutdown_connections 2024-03-06T07:27:07.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:07.941+0000 7fed45466640 1 -- 172.21.15.17:0/3121048540 wait complete. 2024-03-06T07:27:07.953 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:27:07.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.086+0000 7f979ea39640 1 Processor -- start 2024-03-06T07:27:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.086+0000 7f979ea39640 1 -- start start 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f979ea39640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9798155ba0 0x7f97980c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f979ea39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97981586f0 0x7f9798158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f979ea39640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97981591c0 con 0x7f97980c5480 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f979ea39640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979806df40 con 0x7f9798155ba0 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f979ea39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979806fdb0 con 0x7f97981586f0 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f97977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9798155ba0 0x7f97980c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f97977fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9798155ba0 0x7f97980c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46348/0 (socket says 172.21.15.17:46348) 2024-03-06T07:27:08.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.087+0000 7f97977fe640 1 -- 172.21.15.17:0/3635061783 learned_addr learned my addr 172.21.15.17:0/3635061783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f97977fe640 1 -- 172.21.15.17:0/3635061783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97981586f0 msgr2=0x7f9798158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f979cfd6640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97981586f0 0x7f9798158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f97977fe640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97981586f0 0x7f9798158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f97977fe640 1 -- 172.21.15.17:0/3635061783 >> v1:172.21.15.163:6789/0 conn(0x7f97980c5480 legacy=0x7f979806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f97977fe640 1 -- 172.21.15.17:0/3635061783 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97981594a0 con 0x7f9798155ba0 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f97977fe640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97980c5860 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f9780009870 tx=0x7f978002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6356f93638d312b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.088+0000 7f9796ffd640 1 -- 172.21.15.17:0/3635061783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97800317c0 con 0x7f9798155ba0 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f9796ffd640 1 -- 172.21.15.17:0/3635061783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9780037070 con 0x7f9798155ba0 2024-03-06T07:27:08.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f9796ffd640 1 -- 172.21.15.17:0/3635061783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780031b80 con 0x7f9798155ba0 2024-03-06T07:27:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 -- 172.21.15.17:0/3635061783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 msgr2=0x7f97980c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97980c5860 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f9780009870 tx=0x7f978002b860 comp rx=0 tx=0).stop 2024-03-06T07:27:08.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 -- 172.21.15.17:0/3635061783 shutdown_connections 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97981586f0 0x7f9798158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 --2- 172.21.15.17:0/3635061783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97980c5860 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 -- 172.21.15.17:0/3635061783 >> 172.21.15.17:0/3635061783 conn(0x7f979806b1a0 msgr2=0x7f979806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 -- 172.21.15.17:0/3635061783 shutdown_connections 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.089+0000 7f979ea39640 1 -- 172.21.15.17:0/3635061783 wait complete. 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.090+0000 7f979ea39640 1 Processor -- start 2024-03-06T07:27:08.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.090+0000 7f979ea39640 1 -- start start 2024-03-06T07:27:08.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.091+0000 7f979ea39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97980c5480 0x7f97981ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.091+0000 7f979ea39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97981586f0 0x7f97981e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798155680 con 0x7f9798155ba0 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798156000 con 0x7f97981586f0 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9798152a20 con 0x7f97980c5480 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46362/0 (socket says 172.21.15.17:46362) 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 -- 172.21.15.17:0/3597686005 learned_addr learned my addr 172.21.15.17:0/3597686005 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979cfd6640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97981586f0 0x7f97981e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f9797fff640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97980c5480 0x7f97981ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 -- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97980c5480 msgr2=0x7f97981ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97980c5480 0x7f97981ebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 -- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97981586f0 msgr2=0x7f97981e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97981586f0 0x7f97981e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 -- 172.21.15.17:0/3597686005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97981ef820 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f97977fe640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f97800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780032860 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9780032a20 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980c4270 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9780032e10 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.092+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97981eb700 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.093+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9798155080 con 0x7f9798155ba0 2024-03-06T07:27:08.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.094+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f978004c060 con 0x7f9798155ba0 2024-03-06T07:27:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.094+0000 7f9794ff9640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 0x7f977806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.095+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f97800ac080 con 0x7f9798155ba0 2024-03-06T07:27:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.095+0000 7f9797fff640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 0x7f977806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.095+0000 7f9797fff640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 0x7f977806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9788004ea0 tx=0x7f9788009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.099+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9798155080 con 0x7f9798155ba0 2024-03-06T07:27:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.317+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979804f330 con 0x7f9798155ba0 2024-03-06T07:27:08.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.318+0000 7f9794ff9640 1 -- 172.21.15.17:0/3597686005 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f9780040070 con 0x7f9798155ba0 2024-03-06T07:27:08.320 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:08.320 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:08.320 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 msgr2=0x7f977806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 0x7f977806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9788004ea0 tx=0x7f9788009040 comp rx=0 tx=0).stop 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 msgr2=0x7f97981ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f97800046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 shutdown_connections 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f977806b500 0x7f977806d9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97981586f0 0x7f97981e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9798155ba0 0x7f97981ef2e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 --2- 172.21.15.17:0/3597686005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97980c5480 0x7f97981ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 >> 172.21.15.17:0/3597686005 conn(0x7f979806b1a0 msgr2=0x7f97980707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 shutdown_connections 2024-03-06T07:27:08.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.320+0000 7f979ea39640 1 -- 172.21.15.17:0/3597686005 wait complete. 2024-03-06T07:27:08.333 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.432+0000 7f7c9d73a640 1 Processor -- start 2024-03-06T07:27:08.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.432+0000 7f7c9d73a640 1 -- start start 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c981527b0 0x7f7c98159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 0x7f7c980c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c98155b20 0x7f7c980c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c98159640 con 0x7f7c981527b0 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c980c8df0 con 0x7f7c980c7180 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c9d73a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980c7c60 con 0x7f7c98155b20 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c981527b0 0x7f7c98159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c981527b0 0x7f7c98159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46368/0 (socket says 172.21.15.17:46368) 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 -- 172.21.15.17:0/550919080 learned_addr learned my addr 172.21.15.17:0/550919080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c967fc640 1 --2- 172.21.15.17:0/550919080 >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 0x7f7c980c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c977fe640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c98155b20 0x7f7c980c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 -- 172.21.15.17:0/550919080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c98155b20 msgr2=0x7f7c980c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:08.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c98155b20 0x7f7c980c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 -- 172.21.15.17:0/550919080 >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 msgr2=0x7f7c980c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/550919080 >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 0x7f7c980c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c96ffd640 1 -- 172.21.15.17:0/550919080 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980c8060 con 0x7f7c981527b0 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.433+0000 7f7c967fc640 1 --2- 172.21.15.17:0/550919080 >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 0x7f7c980c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.434+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c981527b0 0x7f7c98159100 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7c9808cac0 tx=0x7f7c8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada083d1d3dd4ffe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.434+0000 7f7c95ffb640 1 -- 172.21.15.17:0/550919080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c02e070 con 0x7f7c981527b0 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.434+0000 7f7c95ffb640 1 -- 172.21.15.17:0/550919080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c037070 con 0x7f7c981527b0 2024-03-06T07:27:08.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.434+0000 7f7c95ffb640 1 -- 172.21.15.17:0/550919080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c031a00 con 0x7f7c981527b0 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 -- 172.21.15.17:0/550919080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c981527b0 msgr2=0x7f7c98159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c981527b0 0x7f7c98159100 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7c9808cac0 tx=0x7f7c8c02b880 comp rx=0 tx=0).stop 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 -- 172.21.15.17:0/550919080 shutdown_connections 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c98155b20 0x7f7c980c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/550919080 >> v2:172.21.15.163:3300/0 conn(0x7f7c980c7180 0x7f7c980c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/550919080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c981527b0 0x7f7c98159100 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 -- 172.21.15.17:0/550919080 >> 172.21.15.17:0/550919080 conn(0x7f7c980ac560 msgr2=0x7f7c980ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 -- 172.21.15.17:0/550919080 shutdown_connections 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.435+0000 7f7c9d73a640 1 -- 172.21.15.17:0/550919080 wait complete. 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.436+0000 7f7c9d73a640 1 Processor -- start 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.436+0000 7f7c9d73a640 1 -- start start 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 0x7f7c98154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981527b0 0x7f7c981537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98155320 con 0x7f7c98155b20 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980c7980 con 0x7f7c981527b0 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98155600 con 0x7f7c980c7180 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46380/0 (socket says 172.21.15.17:46380) 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 -- 172.21.15.17:0/362419390 learned_addr learned my addr 172.21.15.17:0/362419390 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 0x7f7c98154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c967fc640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981527b0 0x7f7c981537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 -- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 msgr2=0x7f7c98154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 0x7f7c98154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 -- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981527b0 msgr2=0x7f7c981537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981527b0 0x7f7c981537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 -- 172.21.15.17:0/362419390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98150350 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 0x7f7c98154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c977fe640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007e90 tx=0x7f7c9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014420 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90019070 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c98157530 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014800 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.437+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c981eeb50 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.439+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9804f330 con 0x7f7c98155b20 2024-03-06T07:27:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.439+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c900149a0 con 0x7f7c98155b20 2024-03-06T07:27:08.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.439+0000 7f7c7f7fe640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.439+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f7c9008aac0 con 0x7f7c98155b20 2024-03-06T07:27:08.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.440+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.440+0000 7f7c96ffd640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 0x7f7c8406d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c036000 tx=0x7f7c8c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.444+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c98150350 con 0x7f7c98155b20 2024-03-06T07:27:08.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.713+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9806ce00 con 0x7f7c98155b20 2024-03-06T07:27:08.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.714+0000 7f7c7f7fe640 1 -- 172.21.15.17:0/362419390 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+4993 (secure 0 0 0) 0x7f7c900597f0 con 0x7f7c98155b20 2024-03-06T07:27:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:08.716 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":166,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:26:38.614061+0000","modified":"2024-03-06T07:27:02.855289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":165,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/1440163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1440163522},{"type":"v1","addr":"172.21.15.17:6835","nonce":1440163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9660,"qdb_cluster":[9660]},"id":13}]} 2024-03-06T07:27:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 166 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 msgr2=0x7f7c8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 0x7f7c8406d9c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c036000 tx=0x7f7c8c03d040 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 msgr2=0x7f7c981ee0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f7c90007e90 tx=0x7f7c9000e4b0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 shutdown_connections 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c98155b20 0x7f7c981ee0d0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7c981527b0 0x7f7c981537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 --2- 172.21.15.17:0/362419390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7c980c7180 0x7f7c98154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 >> 172.21.15.17:0/362419390 conn(0x7f7c980ac560 msgr2=0x7f7c980ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 shutdown_connections 2024-03-06T07:27:08.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.716+0000 7f7c9d73a640 1 -- 172.21.15.17:0/362419390 wait complete. 2024-03-06T07:27:08.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.832+0000 7f9920f32640 1 Processor -- start 2024-03-06T07:27:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.832+0000 7f9920f32640 1 -- start start 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f9920f32640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f991c0c1990 0x7f991c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f9920f32640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c0c1dc0 con 0x7f991c152920 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f9920f32640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06afc0 con 0x7f991c155c20 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f9920f32640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c06b2a0 con 0x7f991c0c1990 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f991c0c1990 0x7f991c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f991c152920 0x7f991c0c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53386/0 (socket says 172.21.15.17:53386) 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991a59c640 1 -- 172.21.15.17:0/4287493858 learned_addr learned my addr 172.21.15.17:0/4287493858 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991ad9d640 1 -- 172.21.15.17:0/4287493858 >> v1:172.21.15.163:6789/0 conn(0x7f991c155c20 legacy=0x7f991c0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991959a640 1 -- 172.21.15.17:0/4287493858 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149851230 0 0) 0x7f991c0c1dc0 con 0x7f991c152920 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991ad9d640 1 -- 172.21.15.17:0/4287493858 >> v1:172.21.15.17:6789/0 conn(0x7f991c152920 legacy=0x7f991c0c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.833+0000 7f991ad9d640 1 -- 172.21.15.17:0/4287493858 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c06b580 con 0x7f991c0c1990 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.834+0000 7f991ad9d640 1 --2- 172.21.15.17:0/4287493858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0c1990 0x7f991c160b30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d90 tx=0x7f991402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c216730f99ad0a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.834+0000 7f991959a640 1 -- 172.21.15.17:0/4287493858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914039070 con 0x7f991c0c1990 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.834+0000 7f991959a640 1 -- 172.21.15.17:0/4287493858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991402cd00 con 0x7f991c0c1990 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.834+0000 7f991959a640 1 -- 172.21.15.17:0/4287493858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914034660 con 0x7f991c0c1990 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 -- 172.21.15.17:0/4287493858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0c1990 msgr2=0x7f991c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 --2- 172.21.15.17:0/4287493858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0c1990 0x7f991c160b30 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f9914009d90 tx=0x7f991402bf50 comp rx=0 tx=0).stop 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 -- 172.21.15.17:0/4287493858 shutdown_connections 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 --2- 172.21.15.17:0/4287493858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c0c1990 0x7f991c160b30 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 -- 172.21.15.17:0/4287493858 >> 172.21.15.17:0/4287493858 conn(0x7f991c0c4290 msgr2=0x7f991c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 -- 172.21.15.17:0/4287493858 shutdown_connections 2024-03-06T07:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.835+0000 7f9920f32640 1 -- 172.21.15.17:0/4287493858 wait complete. 2024-03-06T07:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.836+0000 7f9920f32640 1 Processor -- start 2024-03-06T07:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.836+0000 7f9920f32640 1 -- start start 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 0x7f991c1df930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 0x7f991c1ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1560b0 con 0x7f991c155c20 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0c0d60 con 0x7f991c152920 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9920f32640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0c28a0 con 0x7f991c0c1990 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46406/0 (socket says 172.21.15.17:46406) 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 -- 172.21.15.17:0/1394224150 learned_addr learned my addr 172.21.15.17:0/1394224150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991a59c640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 0x7f991c1df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9919d9b640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 0x7f991c1ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 -- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 msgr2=0x7f991c1df930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 0x7f991c1df930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 -- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 msgr2=0x7f991c1ec2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 0x7f991c1ec2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991ad9d640 1 -- 172.21.15.17:0/1394224150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c151280 con 0x7f991c155c20 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f991a59c640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 0x7f991c1df930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.837+0000 7f9919d9b640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 0x7f991c1ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f991ad9d640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f9914037040 tx=0x7f9914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914039050 con 0x7f991c155c20 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991403a040 con 0x7f991c155c20 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991403e610 con 0x7f991c155c20 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c1debe0 con 0x7f991c155c20 2024-03-06T07:27:08.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.838+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c1eb320 con 0x7f991c155c20 2024-03-06T07:27:08.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.839+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c1eae30 con 0x7f991c155c20 2024-03-06T07:27:08.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.840+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9914047020 con 0x7f991c155c20 2024-03-06T07:27:08.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.840+0000 7f9912ffd640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:08.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.840+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f99140ac6d0 con 0x7f991c155c20 2024-03-06T07:27:08.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.840+0000 7f991a59c640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:08.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.841+0000 7f991a59c640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f60 tx=0x7f990c007c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:08.848+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f991404e020 con 0x7f991c155c20 2024-03-06T07:27:09.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.030+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f991c04f330 con 0x7f991c155c20 2024-03-06T07:27:09.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.031+0000 7f9912ffd640 1 -- 172.21.15.17:0/1394224150 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) v1 ==== 74+0+17097 (secure 0 0 0) 0x7f991407b4c0 con 0x7f991c155c20 2024-03-06T07:27:09.033 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:09.033 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":174,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:26:54.491601+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:26:36.885685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-03-06T07:26:37.580691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"28.0","mappings":[{"from":6,"to":4}]},{"pgid":"28.35","mappings":[{"from":6,"to":4}]},{"pgid":"28.36","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 msgr2=0x7f98fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f990c000f60 tx=0x7f990c007c50 comp rx=0 tx=0).stop 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 msgr2=0x7f991c1ec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f9914037040 tx=0x7f9914004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 shutdown_connections 2024-03-06T07:27:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f991c155c20 0x7f991c1ec7e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f991c152920 0x7f991c1ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 --2- 172.21.15.17:0/1394224150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991c0c1990 0x7f991c1df930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.033+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 >> 172.21.15.17:0/1394224150 conn(0x7f991c0c4290 msgr2=0x7f991c0c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.034+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 shutdown_connections 2024-03-06T07:27:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.034+0000 7f9920f32640 1 -- 172.21.15.17:0/1394224150 wait complete. 2024-03-06T07:27:09.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:27:09.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.151+0000 7ff948956640 1 Processor -- start 2024-03-06T07:27:09.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.151+0000 7ff948956640 1 -- start start 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff948956640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9440c7170 0x7ff9440c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff948956640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c6c40 0x7ff9440c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff948956640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9440a8390 con 0x7ff94406ff10 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff948956640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9440c7620 con 0x7ff9440c7170 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff948956640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440730f0 con 0x7ff9440c6c40 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff941d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff9440c7170 0x7ff9440c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff94259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff94406ff10 0x7ff944153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53394/0 (socket says 172.21.15.17:53394) 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff94259c640 1 -- 172.21.15.17:0/2397430956 learned_addr learned my addr 172.21.15.17:0/2397430956 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff942d9d640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c6c40 0x7ff9440c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff941d9b640 1 -- 172.21.15.17:0/2397430956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c6c40 msgr2=0x7ff9440c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:09.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff941d9b640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c6c40 0x7ff9440c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff941d9b640 1 -- 172.21.15.17:0/2397430956 >> v1:172.21.15.17:6789/0 conn(0x7ff94406ff10 legacy=0x7ff944153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff941d9b640 1 -- 172.21.15.17:0/2397430956 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9440733d0 con 0x7ff9440c7170 2024-03-06T07:27:09.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.152+0000 7ff94159a640 1 -- 172.21.15.17:0/2397430956 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346600552 0 0) 0x7ff9440a8390 con 0x7ff94406ff10 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.153+0000 7ff941d9b640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c7170 0x7ff9440c8810 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7ff9380097a0 tx=0x7ff93802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42becf00ef41360 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.153+0000 7ff94159a640 1 -- 172.21.15.17:0/2397430956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938038070 con 0x7ff9440c7170 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.153+0000 7ff94159a640 1 -- 172.21.15.17:0/2397430956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938031700 con 0x7ff9440c7170 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.153+0000 7ff94159a640 1 -- 172.21.15.17:0/2397430956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938031ad0 con 0x7ff9440c7170 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 -- 172.21.15.17:0/2397430956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c7170 msgr2=0x7ff9440c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c7170 0x7ff9440c8810 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7ff9380097a0 tx=0x7ff93802b7a0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 -- 172.21.15.17:0/2397430956 shutdown_connections 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c6c40 0x7ff9440c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 --2- 172.21.15.17:0/2397430956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c7170 0x7ff9440c8810 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 -- 172.21.15.17:0/2397430956 >> 172.21.15.17:0/2397430956 conn(0x7ff94406b970 msgr2=0x7ff9440aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 -- 172.21.15.17:0/2397430956 shutdown_connections 2024-03-06T07:27:09.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.154+0000 7ff948956640 1 -- 172.21.15.17:0/2397430956 wait complete. 2024-03-06T07:27:09.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.155+0000 7ff948956640 1 Processor -- start 2024-03-06T07:27:09.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.155+0000 7ff948956640 1 -- start start 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff94406ff10 0x7ff9440aa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c7170 0x7ff9441e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9441544b0 con 0x7ff9440c6c40 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff944154330 con 0x7ff94406ff10 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff948956640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9440702f0 con 0x7ff9440c7170 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff941d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46428/0 (socket says 172.21.15.17:46428) 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.156+0000 7ff941d9b640 1 -- 172.21.15.17:0/1368093089 learned_addr learned my addr 172.21.15.17:0/1368093089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 -- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c7170 msgr2=0x7ff9441e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c7170 0x7ff9441e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 -- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff94406ff10 msgr2=0x7ff9440aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff94259c640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff94406ff10 0x7ff9440aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff94406ff10 0x7ff9440aa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 -- 172.21.15.17:0/1368093089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9440aa6d0 con 0x7ff9440c6c40 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff941d9b640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff9380097a0 tx=0x7ff9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938038040 con 0x7ff9440c6c40 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938004250 con 0x7ff9440c6c40 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff944154dc0 con 0x7ff9440c6c40 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938033820 con 0x7ff9440c6c40 2024-03-06T07:27:09.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.157+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff944155380 con 0x7ff9440c6c40 2024-03-06T07:27:09.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.158+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94404f330 con 0x7ff9440c6c40 2024-03-06T07:27:09.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.159+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9380339c0 con 0x7ff9440c6c40 2024-03-06T07:27:09.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.159+0000 7ff932ffd640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 0x7ff92406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.159+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff9380ac7b0 con 0x7ff9440c6c40 2024-03-06T07:27:09.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.159+0000 7ff94259c640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 0x7ff92406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.160+0000 7ff94259c640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 0x7ff92406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff934000f60 tx=0x7ff934009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.163+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff938030070 con 0x7ff9440c6c40 2024-03-06T07:27:09.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.383+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff9440c2980 con 0x7ff9440c6c40 2024-03-06T07:27:09.385 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:27:09.384+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:27:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.385+0000 7ff932ffd640 1 -- 172.21.15.17:0/1368093089 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v167) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff93807b4e0 con 0x7ff9440c6c40 2024-03-06T07:27:09.387 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 msgr2=0x7ff92406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 0x7ff92406d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff934000f60 tx=0x7ff934009040 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 msgr2=0x7ff9440acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff9380097a0 tx=0x7ff9380046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 shutdown_connections 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff92406b4d0 0x7ff92406d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9440c7170 0x7ff9441e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9440c6c40 0x7ff9440acb10 secure :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7ff9380097a0 tx=0x7ff9380046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 --2- 172.21.15.17:0/1368093089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff94406ff10 0x7ff9440aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 >> 172.21.15.17:0/1368093089 conn(0x7ff94406b970 msgr2=0x7ff944152330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 shutdown_connections 2024-03-06T07:27:09.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.388+0000 7ff948956640 1 -- 172.21.15.17:0/1368093089 wait complete. 2024-03-06T07:27:09.391 INFO:tasks.ceph.mds.a.smithi017.stderr: -93> 2024-03-06T07:27:06.528+0000 7f5f2815b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:09.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:27:09.417 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:27:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.506+0000 7efeb76b8640 1 Processor -- start 2024-03-06T07:27:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.506+0000 7efeb76b8640 1 -- start start 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb76b8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efeb0155c30 0x7efeb006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb76b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00b0450 con 0x7efeb00c5380 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb76b8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb01560e0 con 0x7efeb00b0070 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb76b8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb006cfb0 con 0x7efeb0155c30 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb4c53640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efeb0155c30 0x7efeb006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb5c55640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efeb00c5380 0x7efeb014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53408/0 (socket says 172.21.15.17:53408) 2024-03-06T07:27:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.507+0000 7efeb5c55640 1 -- 172.21.15.17:0/2935680099 learned_addr learned my addr 172.21.15.17:0/2935680099 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeb4c53640 1 -- 172.21.15.17:0/2935680099 >> v1:172.21.15.163:6789/0 conn(0x7efeb00b0070 legacy=0x7efeb00c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeb4c53640 1 -- 172.21.15.17:0/2935680099 >> v1:172.21.15.17:6789/0 conn(0x7efeb00c5380 legacy=0x7efeb014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeb4c53640 1 -- 172.21.15.17:0/2935680099 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00c77f0 con 0x7efeb0155c30 2024-03-06T07:27:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeaffff640 1 -- 172.21.15.17:0/2935680099 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709893873 0 0) 0x7efea0001490 con 0x7efeb00c5380 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeb4c53640 1 --2- 172.21.15.17:0/2935680099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb006ca70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7efea40097a0 tx=0x7efea402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81009cfc9abc4a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeaffff640 1 -- 172.21.15.17:0/2935680099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4035020 con 0x7efeb0155c30 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeaffff640 1 -- 172.21.15.17:0/2935680099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea4031830 con 0x7efeb0155c30 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.508+0000 7efeaffff640 1 -- 172.21.15.17:0/2935680099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4031c00 con 0x7efeb0155c30 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 -- 172.21.15.17:0/2935680099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 msgr2=0x7efeb006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2935680099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb006ca70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7efea40097a0 tx=0x7efea402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 -- 172.21.15.17:0/2935680099 shutdown_connections 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2935680099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb006ca70 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 -- 172.21.15.17:0/2935680099 >> 172.21.15.17:0/2935680099 conn(0x7efeb00ac590 msgr2=0x7efeb00ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 -- 172.21.15.17:0/2935680099 shutdown_connections 2024-03-06T07:27:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.509+0000 7efeb76b8640 1 -- 172.21.15.17:0/2935680099 wait complete. 2024-03-06T07:27:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.510+0000 7efeb76b8640 1 Processor -- start 2024-03-06T07:27:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.510+0000 7efeb76b8640 1 -- start start 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efeb00b0070 0x7efeb0151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efeb00c5380 0x7efeb01542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006c270 con 0x7efeb0155c30 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006c020 con 0x7efeb00c5380 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006c550 con 0x7efeb00b0070 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46466/0 (socket says 172.21.15.17:46466) 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 -- 172.21.15.17:0/2145718807 learned_addr learned my addr 172.21.15.17:0/2145718807 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb4c53640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efeb00c5380 0x7efeb01542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5454640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efeb00b0070 0x7efeb0151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 -- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efeb00b0070 msgr2=0x7efeb0151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efeb00b0070 0x7efeb0151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 -- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efeb00c5380 msgr2=0x7efeb01542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efeb00c5380 0x7efeb01542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 -- 172.21.15.17:0/2145718807 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0159230 con 0x7efeb0155c30 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb5c55640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7efea000c7f0 tx=0x7efea000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0018070 con 0x7efeb0155c30 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea000a5e0 con 0x7efeb0155c30 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.511+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb0151af0 con 0x7efeb0155c30 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.512+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea000aa10 con 0x7efeb0155c30 2024-03-06T07:27:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.512+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00c4ce0 con 0x7efeb0155c30 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.513+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0159710 con 0x7efeb0155c30 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.513+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efea001b070 con 0x7efeb0155c30 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.513+0000 7efeadffb640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 0x7efe9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.513+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6274+0+0 (secure 0 0 0) 0x7efea00898d0 con 0x7efeb0155c30 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.513+0000 7efeb5454640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 0x7efe9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.514+0000 7efeb5454640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 0x7efe9c06d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efeb00ac500 tx=0x7efea8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:09.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.517+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efeb0159710 con 0x7efeb0155c30 2024-03-06T07:27:09.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:09.716 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:09.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:09.735+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efeb006d860 con 0x7efeb0155c30 2024-03-06T07:27:10.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.386+0000 7efeadffb640 1 -- 172.21.15.17:0/2145718807 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v168) v1 ==== 104+0+0 (secure 0 0 0) 0x7efeb00b0450 con 0x7efeb0155c30 2024-03-06T07:27:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 msgr2=0x7efe9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 0x7efe9c06d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efeb00ac500 tx=0x7efea8009040 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 msgr2=0x7efeb01ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7efea000c7f0 tx=0x7efea000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 shutdown_connections 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efe9c06b4d0 0x7efe9c06d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb0155c30 0x7efeb01ee0c0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efeb00c5380 0x7efeb01542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 --2- 172.21.15.17:0/2145718807 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efeb00b0070 0x7efeb0151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.388+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 >> 172.21.15.17:0/2145718807 conn(0x7efeb00ac590 msgr2=0x7efeb00aaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.389+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 shutdown_connections 2024-03-06T07:27:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.389+0000 7efeb76b8640 1 -- 172.21.15.17:0/2145718807 wait complete. 2024-03-06T07:27:10.405 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:27:10.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.515+0000 7f700a042640 1 Processor -- start 2024-03-06T07:27:10.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.515+0000 7f700a042640 1 -- start start 2024-03-06T07:27:10.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f700a042640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7004072fc0 0x7f70040724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f700a042640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7004072a10 0x7f70040c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f700a042640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040c6220 con 0x7f70040c5e40 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f700a042640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7004070210 con 0x7f7004072a10 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f700a042640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040704d0 con 0x7f7004072fc0 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f7003fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7004072a10 0x7f70040c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f70037fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f70040c5e40 0x7f7004153940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57448/0 (socket says 172.21.15.17:57448) 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.516+0000 7f70037fe640 1 -- 172.21.15.17:0/2898175546 learned_addr learned my addr 172.21.15.17:0/2898175546 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7002ffd640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7004072fc0 0x7f70040724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7003fff640 1 -- 172.21.15.17:0/2898175546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7004072fc0 msgr2=0x7f70040724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7003fff640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7004072fc0 0x7f70040724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7003fff640 1 -- 172.21.15.17:0/2898175546 >> v1:172.21.15.17:6789/0 conn(0x7f70040c5e40 legacy=0x7f7004153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7003fff640 1 -- 172.21.15.17:0/2898175546 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700406f610 con 0x7f7004072a10 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.517+0000 7f7003fff640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072a10 0x7f70040c46d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9799692d532d16b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f70027fc640 1 -- 172.21.15.17:0/2898175546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc039070 con 0x7f7004072a10 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f70027fc640 1 -- 172.21.15.17:0/2898175546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc02cbc0 con 0x7f7004072a10 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f70027fc640 1 -- 172.21.15.17:0/2898175546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc034440 con 0x7f7004072a10 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 -- 172.21.15.17:0/2898175546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072a10 msgr2=0x7f70040c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072a10 0x7f70040c46d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc02be10 comp rx=0 tx=0).stop 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 -- 172.21.15.17:0/2898175546 shutdown_connections 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072a10 0x7f70040c46d0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 --2- 172.21.15.17:0/2898175546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7004072fc0 0x7f70040724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 -- 172.21.15.17:0/2898175546 >> 172.21.15.17:0/2898175546 conn(0x7f70040c88f0 msgr2=0x7f70040c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:10.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 -- 172.21.15.17:0/2898175546 shutdown_connections 2024-03-06T07:27:10.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.518+0000 7f700a042640 1 -- 172.21.15.17:0/2898175546 wait complete. 2024-03-06T07:27:10.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.519+0000 7f700a042640 1 Processor -- start 2024-03-06T07:27:10.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 -- start start 2024-03-06T07:27:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7004072a10 0x7f700406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70040c5e40 0x7f70041ede80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70040c4c10 con 0x7f7004072fc0 2024-03-06T07:27:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406fbd0 con 0x7f7004072a10 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f700a042640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040c4d90 con 0x7f70040c5e40 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f7002ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f7002ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49530/0 (socket says 172.21.15.17:49530) 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.520+0000 7f7002ffd640 1 -- 172.21.15.17:0/1983972287 learned_addr learned my addr 172.21.15.17:0/1983972287 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f70037fe640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7004072a10 0x7f700406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 -- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70040c5e40 msgr2=0x7f70041ede80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70040c5e40 0x7f70041ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 -- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7004072a10 msgr2=0x7f700406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7004072a10 0x7f700406d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 -- 172.21.15.17:0/1983972287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700414d5d0 con 0x7f7004072fc0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f7002ffd640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800d800 tx=0x7f6ff800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800b6c0 con 0x7f7004072fc0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8019070 con 0x7f7004072fc0 2024-03-06T07:27:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70041515c0 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800baa0 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.521+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70040a9dd0 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.522+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700406dc30 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.522+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ff800bc40 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.523+0000 7f6febfff640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 0x7f6ff006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.523+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f6ff808a800 con 0x7f7004072fc0 2024-03-06T07:27:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.523+0000 7f70037fe640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 0x7f6ff006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:10.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.524+0000 7f70037fe640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 0x7f6ff006d9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f70040c3110 tx=0x7f6ff4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:10.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.527+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6ff80594d0 con 0x7f7004072fc0 2024-03-06T07:27:10.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:10.710+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f700404f330 con 0x7f7004072fc0 2024-03-06T07:27:11.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.392+0000 7f6febfff640 1 -- 172.21.15.17:0/1983972287 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v176) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6ff8058dd0 con 0x7f7004072fc0 2024-03-06T07:27:11.395 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 msgr2=0x7f6ff006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 0x7f6ff006d9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f70040c3110 tx=0x7f6ff4009040 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 msgr2=0x7f70040a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800d800 tx=0x7f6ff800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 shutdown_connections 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ff006b500 0x7f6ff006d9c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70040c5e40 0x7f70041ede80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7004072fc0 0x7f70040a9890 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 --2- 172.21.15.17:0/1983972287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7004072a10 0x7f700406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 >> 172.21.15.17:0/1983972287 conn(0x7f70040c88f0 msgr2=0x7f70040c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 shutdown_connections 2024-03-06T07:27:11.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.395+0000 7f700a042640 1 -- 172.21.15.17:0/1983972287 wait complete. 2024-03-06T07:27:11.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:27:11.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.525+0000 7f1e2f17a640 1 Processor -- start 2024-03-06T07:27:11.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.525+0000 7f1e2f17a640 1 -- start start 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2f17a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e280c8980 0x7f1e281551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2f17a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e280c8f60 con 0x7f1e2814f180 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2f17a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e280c9240 con 0x7f1e281566c0 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2f17a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e280c9520 con 0x7f1e280c8980 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2cf16640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1e2814f180 0x7f1e280ca000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57454/0 (socket says 172.21.15.17:57454) 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.526+0000 7f1e2cf16640 1 -- 172.21.15.17:0/2262916191 learned_addr learned my addr 172.21.15.17:0/2262916191 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e27fff640 1 --2- 172.21.15.17:0/2262916191 >> v2:172.21.15.163:3300/0 conn(0x7f1e280c8980 0x7f1e281551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575952745 0 0) 0x7f1e280c8f60 con 0x7f1e2814f180 2024-03-06T07:27:11.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e18003520 con 0x7f1e2814f180 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823927479 0 0) 0x7f1e18003520 con 0x7f1e2814f180 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 >> v2:172.21.15.163:3300/0 conn(0x7f1e280c8980 msgr2=0x7f1e281551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 --2- 172.21.15.17:0/2262916191 >> v2:172.21.15.163:3300/0 conn(0x7f1e280c8980 0x7f1e281551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 >> v1:172.21.15.163:6789/0 conn(0x7f1e281566c0 legacy=0x7f1e2806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280c9800 con 0x7f1e2814f180 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1e1c001670 con 0x7f1e2814f180 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 >> v1:172.21.15.17:6789/0 conn(0x7f1e2814f180 legacy=0x7f1e280ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 0x7f1e18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e18004800 0x7f1e18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e180051c0 0x7f1e1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280c9520 con 0x7f1e180039b0 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1800c060 con 0x7f1e180051c0 2024-03-06T07:27:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1800c260 con 0x7f1e18004800 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 0x7f1e18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2d717640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e180051c0 0x7f1e1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e27fff640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e18004800 0x7f1e18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 -- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e18004800 msgr2=0x7f1e18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e18004800 0x7f1e18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 -- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e180051c0 msgr2=0x7f1e1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e180051c0 0x7f1e1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2cf16640 1 -- 172.21.15.17:0/2262916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e18003520 con 0x7f1e180039b0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2d717640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e180051c0 0x7f1e1800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.527+0000 7f1e2d717640 1 -- 172.21.15.17:0/2262916191 reap_dead start 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.528+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 0x7f1e18003dd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c240 tx=0x7f1e18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb60abf922126bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.528+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c007d10 con 0x7f1e180039b0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.528+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c030070 con 0x7f1e180039b0 2024-03-06T07:27:11.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.528+0000 7f1e277fe640 1 -- 172.21.15.17:0/2262916191 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c02cdc0 con 0x7f1e180039b0 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 msgr2=0x7f1e18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 0x7f1e18003dd0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c00c240 tx=0x7f1e18005830 comp rx=0 tx=0).stop 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2262916191 shutdown_connections 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2262916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e180039b0 0x7f1e18003dd0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2262916191 >> 172.21.15.17:0/2262916191 conn(0x7f1e28072d20 msgr2=0x7f1e280ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2262916191 shutdown_connections 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.529+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2262916191 wait complete. 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.530+0000 7f1e2f17a640 1 Processor -- start 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.530+0000 7f1e2f17a640 1 -- start start 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e180039b0 0x7f1e281509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 0x7f1e280c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2814f560 con 0x7f1e280c5db0 2024-03-06T07:27:11.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28156af0 con 0x7f1e2814bf70 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2f17a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2814f6e0 con 0x7f1e180039b0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49552/0 (socket says 172.21.15.17:49552) 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 -- 172.21.15.17:0/2958663916 learned_addr learned my addr 172.21.15.17:0/2958663916 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e180039b0 0x7f1e281509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e27fff640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 0x7f1e280c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 -- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e180039b0 msgr2=0x7f1e281509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e180039b0 0x7f1e281509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 -- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 msgr2=0x7f1e280c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 0x7f1e280c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 -- 172.21.15.17:0/2958663916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28152430 con 0x7f1e280c5db0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e27fff640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 0x7f1e280c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.531+0000 7f1e2d717640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e4d0 tx=0x7f1e2000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.532+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20003b80 con 0x7f1e280c5db0 2024-03-06T07:27:11.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.532+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20018070 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.532+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e28152710 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.532+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200050d0 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.532+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e281eec80 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.533+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2804f330 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.533+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e20005270 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.533+0000 7f1e257fa640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 0x7f1e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.534+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f1e20089d40 con 0x7f1e280c5db0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.534+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 0x7f1e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:11.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.534+0000 7f1e2cf16640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 0x7f1e0c06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034000 tx=0x7f1e1c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:11.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.538+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e20058b40 con 0x7f1e280c5db0 2024-03-06T07:27:11.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:11.722+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e280c75d0 con 0x7f1e280c5db0 2024-03-06T07:27:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.396+0000 7f1e257fa640 1 -- 172.21.15.17:0/2958663916 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v177) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1e20058440 con 0x7f1e280c5db0 2024-03-06T07:27:12.399 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:27:12.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 msgr2=0x7f1e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 0x7f1e0c06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034000 tx=0x7f1e1c009030 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 msgr2=0x7f1e281ee740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000e4d0 tx=0x7f1e2000e990 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 shutdown_connections 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e0c06b4d0 0x7f1e0c06d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e280c5db0 0x7f1e281ee740 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e2814bf70 0x7f1e280c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 --2- 172.21.15.17:0/2958663916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e180039b0 0x7f1e281509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 >> 172.21.15.17:0/2958663916 conn(0x7f1e28072d20 msgr2=0x7f1e281e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 shutdown_connections 2024-03-06T07:27:12.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.401+0000 7f1e2f17a640 1 -- 172.21.15.17:0/2958663916 wait complete. 2024-03-06T07:27:12.418 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:27:12.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.525+0000 7f7708bc9640 1 Processor -- start 2024-03-06T07:27:12.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.525+0000 7f7708bc9640 1 -- start start 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7708bc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040724d0 0x7f7704155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7708bc9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77040c2170 0x7f77040728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7708bc9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77040c2c00 con 0x7f77040c8370 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7708bc9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7704155270 con 0x7f77040c2170 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7708bc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704155550 con 0x7f77040724d0 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77040c2170 0x7f77040728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f77040c2170 0x7f77040728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49556/0 (socket says 172.21.15.17:49556) 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 -- 172.21.15.17:0/1269867709 learned_addr learned my addr 172.21.15.17:0/1269867709 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 -- 172.21.15.17:0/1269867709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040724d0 msgr2=0x7f7704155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 --2- 172.21.15.17:0/1269867709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040724d0 0x7f7704155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 -- 172.21.15.17:0/1269867709 >> v1:172.21.15.163:6789/0 conn(0x7f77040c8370 legacy=0x7f7704158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:12.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.526+0000 7f7702d9d640 1 -- 172.21.15.17:0/1269867709 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704155830 con 0x7f77040c2170 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f7702d9d640 1 --2- 172.21.15.17:0/1269867709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77040728b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7704070540 tx=0x7f76fc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6331b58a9c460787 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f770159a640 1 -- 172.21.15.17:0/1269867709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc02c9c0 con 0x7f77040c2170 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f770159a640 1 -- 172.21.15.17:0/1269867709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc02cb60 con 0x7f77040c2170 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f770159a640 1 -- 172.21.15.17:0/1269867709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0343f0 con 0x7f77040c2170 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f7708bc9640 1 -- 172.21.15.17:0/1269867709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 msgr2=0x7f77040728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.527+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1269867709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77040728b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f7704070540 tx=0x7f76fc02be20 comp rx=0 tx=0).stop 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 -- 172.21.15.17:0/1269867709 shutdown_connections 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1269867709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77040728b0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1269867709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040724d0 0x7f7704155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 -- 172.21.15.17:0/1269867709 >> 172.21.15.17:0/1269867709 conn(0x7f77040aa5c0 msgr2=0x7f77040aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:12.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 -- 172.21.15.17:0/1269867709 shutdown_connections 2024-03-06T07:27:12.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.528+0000 7f7708bc9640 1 -- 172.21.15.17:0/1269867709 wait complete. 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.529+0000 7f7708bc9640 1 Processor -- start 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.529+0000 7f7708bc9640 1 -- start start 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 0x7f770406d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040c8370 0x7f770406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040c2880 con 0x7f77040c2170 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040c2ea0 con 0x7f77040724d0 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704072df0 con 0x7f77040c8370 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49568/0 (socket says 172.21.15.17:49568) 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 -- 172.21.15.17:0/1869130603 learned_addr learned my addr 172.21.15.17:0/1869130603 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:12.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7702d9d640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040c8370 0x7f770406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f770259c640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 0x7f770406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 -- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040c8370 msgr2=0x7f770406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040c8370 0x7f770406e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 -- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 msgr2=0x7f770406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 0x7f770406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 -- 172.21.15.17:0/1869130603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77041eee90 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f770259c640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 0x7f770406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7701d9b640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c410 tx=0x7f76f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80035f0 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77041518d0 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8003a20 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.530+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77041587e0 con 0x7f77040c2170 2024-03-06T07:27:12.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.531+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770404f330 con 0x7f77040c2170 2024-03-06T07:27:12.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.532+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76f8003bc0 con 0x7f77040c2170 2024-03-06T07:27:12.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.532+0000 7f76f2ffd640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.532+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f76f80893d0 con 0x7f77040c2170 2024-03-06T07:27:12.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.532+0000 7f770259c640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.533+0000 7f770259c640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 0x7f76e406d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f77041523d0 tx=0x7f76f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.537+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76f80583d0 con 0x7f77040c2170 2024-03-06T07:27:12.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.717+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f770406d3d0 con 0x7f77040c2170 2024-03-06T07:27:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.721+0000 7f76f2ffd640 1 -- 172.21.15.17:0/1869130603 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f76f80010c0 con 0x7f77040c2170 2024-03-06T07:27:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 msgr2=0x7f76e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 0x7f76e406d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f77041523d0 tx=0x7f76f4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 msgr2=0x7f77041582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c410 tx=0x7f76f800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 shutdown_connections 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77040c8370 0x7f770406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77040c2170 0x7f77041582a0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 --2- 172.21.15.17:0/1869130603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77040724d0 0x7f770406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 >> 172.21.15.17:0/1869130603 conn(0x7f77040aa5c0 msgr2=0x7f77040aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 shutdown_connections 2024-03-06T07:27:12.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.723+0000 7f7708bc9640 1 -- 172.21.15.17:0/1869130603 wait complete. 2024-03-06T07:27:12.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down' 2024-03-06T07:27:12.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 Processor -- start 2024-03-06T07:27:12.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 -- start start 2024-03-06T07:27:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0a40c3710 0x7fc0a41558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a4158800 con 0x7fc0a40c3280 2024-03-06T07:27:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a4158ae0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.835+0000 7fc0a8b9a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a4158dc0 con 0x7fc0a40c3710 2024-03-06T07:27:12.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc0a2d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0a40c3710 0x7fc0a41558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc099d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc0a40c3280 0x7fc0a41551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57462/0 (socket says 172.21.15.17:57462) 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc099d9b640 1 -- 172.21.15.17:0/3978596927 learned_addr learned my addr 172.21.15.17:0/3978596927 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc0a2d9d640 1 -- 172.21.15.17:0/3978596927 >> v1:172.21.15.163:6789/0 conn(0x7fc0a406d9f0 legacy=0x7fc0a40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc0a2d9d640 1 -- 172.21.15.17:0/3978596927 >> v1:172.21.15.17:6789/0 conn(0x7fc0a40c3280 legacy=0x7fc0a41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc0a1d9b640 1 -- 172.21.15.17:0/3978596927 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238186279 0 0) 0x7fc0a4158800 con 0x7fc0a40c3280 2024-03-06T07:27:12.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.836+0000 7fc0a2d9d640 1 -- 172.21.15.17:0/3978596927 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a41590a0 con 0x7fc0a40c3710 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a2d9d640 1 --2- 172.21.15.17:0/3978596927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40c3710 0x7fc0a41558f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ac8e530eb85733 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a1d9b640 1 -- 172.21.15.17:0/3978596927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c039070 con 0x7fc0a40c3710 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a1d9b640 1 -- 172.21.15.17:0/3978596927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c02cbd0 con 0x7fc0a40c3710 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a1d9b640 1 -- 172.21.15.17:0/3978596927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c034440 con 0x7fc0a40c3710 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/3978596927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40c3710 msgr2=0x7fc0a41558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/3978596927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40c3710 0x7fc0a41558f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fc09c009d90 tx=0x7fc09c02be20 comp rx=0 tx=0).stop 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/3978596927 shutdown_connections 2024-03-06T07:27:12.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/3978596927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a40c3710 0x7fc0a41558f0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/3978596927 >> 172.21.15.17:0/3978596927 conn(0x7fc0a40aa630 msgr2=0x7fc0a40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/3978596927 shutdown_connections 2024-03-06T07:27:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.837+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/3978596927 wait complete. 2024-03-06T07:27:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 Processor -- start 2024-03-06T07:27:12.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 -- start start 2024-03-06T07:27:12.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a40c3280 0x7fc0a41eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 0x7fc0a414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a406ddd0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a4155e30 con 0x7fc0a40c3280 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a8b9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a406df50 con 0x7fc0a40c3710 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49584/0 (socket says 172.21.15.17:49584) 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.839+0000 7fc0a259c640 1 -- 172.21.15.17:0/1236787935 learned_addr learned my addr 172.21.15.17:0/1236787935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc099d9b640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a40c3280 0x7fc0a41eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a2d9d640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 0x7fc0a414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 -- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 msgr2=0x7fc0a414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 0x7fc0a414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 -- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a40c3280 msgr2=0x7fc0a41eb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a40c3280 0x7fc0a41eb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 -- 172.21.15.17:0/1236787935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a40ab9d0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a2d9d640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 0x7fc0a414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a259c640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c430 tx=0x7fc09400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094011790 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094017070 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a41eea20 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094011bb0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.840+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a41eac90 con 0x7fc0a406d9f0 2024-03-06T07:27:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.841+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a41eaaa0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.842+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc09401c460 con 0x7fc0a406d9f0 2024-03-06T07:27:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.842+0000 7fc09b7fe640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.842+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc094089ef0 con 0x7fc0a406d9f0 2024-03-06T07:27:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.842+0000 7fc099d9b640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.843+0000 7fc099d9b640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc0a41df6b0 tx=0x7fc08c007b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:12.847+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc094058f70 con 0x7fc0a406d9f0 2024-03-06T07:27:13.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.100+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down"]} v 0) v1 -- 0x7fc0a404f330 con 0x7fc0a406d9f0 2024-03-06T07:27:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.398+0000 7fc09b7fe640 1 -- 172.21.15.17:0/1236787935 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down"]}]=0 v421) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc094058870 con 0x7fc0a406d9f0 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 msgr2=0x7fc08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc0a41df6b0 tx=0x7fc08c007b70 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 msgr2=0x7fc0a4157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc09400c430 tx=0x7fc09400c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 shutdown_connections 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc08406b4d0 0x7fc08406d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a40c3710 0x7fc0a414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a40c3280 0x7fc0a41eb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 --2- 172.21.15.17:0/1236787935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a406d9f0 0x7fc0a4157670 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 >> 172.21.15.17:0/1236787935 conn(0x7fc0a40aa630 msgr2=0x7fc0a4154860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 shutdown_connections 2024-03-06T07:27:13.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.400+0000 7fc0a8b9a640 1 -- 172.21.15.17:0/1236787935 wait complete. 2024-03-06T07:27:13.414 INFO:tasks.cephfs_test_runner:That down setting toggles and sets max_mds appropriately. ... ok 2024-03-06T07:27:13.415 INFO:tasks.cephfs_test_runner:Starting test: test_down_grow (tasks.cephfs.test_failover.TestClusterResize) That setting max_mds undoes down. 2024-03-06T07:27:13.415 INFO:tasks.cephfs_test_runner:test_down_grow (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:27:13.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 Processor -- start 2024-03-06T07:27:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 -- start start 2024-03-06T07:27:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa2140c88c0 0x7fa214155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa214158b60 0x7fa214158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2140c73d0 con 0x7fa21414f360 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2140c8dd0 con 0x7fa2140c88c0 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa21ad5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140be2e0 con 0x7fa214158b60 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa213fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa2140c88c0 0x7fa214155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa218af8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa21414f360 0x7fa2140c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57470/0 (socket says 172.21.15.17:57470) 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa213fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa2140c88c0 0x7fa214155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49598/0 (socket says 172.21.15.17:49598) 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.515+0000 7fa218af8640 1 -- 172.21.15.17:0/4211113288 learned_addr learned my addr 172.21.15.17:0/4211113288 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa2192f9640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa214158b60 0x7fa214158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa213fff640 1 -- 172.21.15.17:0/4211113288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa214158b60 msgr2=0x7fa214158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa213fff640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa214158b60 0x7fa214158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa213fff640 1 -- 172.21.15.17:0/4211113288 >> v1:172.21.15.17:6789/0 conn(0x7fa21414f360 legacy=0x7fa2140c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa213fff640 1 -- 172.21.15.17:0/4211113288 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140c7670 con 0x7fa2140c88c0 2024-03-06T07:27:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.516+0000 7fa2137fe640 1 -- 172.21.15.17:0/4211113288 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703879983 0 0) 0x7fa2140c73d0 con 0x7fa21414f360 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa213fff640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2140c88c0 0x7fa214155c10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa20002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41724f9969c796f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa2137fe640 1 -- 172.21.15.17:0/4211113288 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000315a0 con 0x7fa2140c88c0 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa2137fe640 1 -- 172.21.15.17:0/4211113288 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200037070 con 0x7fa2140c88c0 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa2137fe640 1 -- 172.21.15.17:0/4211113288 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200031960 con 0x7fa2140c88c0 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 -- 172.21.15.17:0/4211113288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2140c88c0 msgr2=0x7fa214155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2140c88c0 0x7fa214155c10 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa2000097a0 tx=0x7fa20002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 -- 172.21.15.17:0/4211113288 shutdown_connections 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa214158b60 0x7fa214158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/4211113288 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2140c88c0 0x7fa214155c10 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 -- 172.21.15.17:0/4211113288 >> 172.21.15.17:0/4211113288 conn(0x7fa2140ac570 msgr2=0x7fa2140ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 -- 172.21.15.17:0/4211113288 shutdown_connections 2024-03-06T07:27:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.517+0000 7fa21ad5c640 1 -- 172.21.15.17:0/4211113288 wait complete. 2024-03-06T07:27:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.518+0000 7fa21ad5c640 1 Processor -- start 2024-03-06T07:27:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.518+0000 7fa21ad5c640 1 -- start start 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2140c88c0 0x7fa214150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa214158b60 0x7fa21414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140c5a50 con 0x7fa21414f360 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140c7c30 con 0x7fa214158b60 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa21ad5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140c5bd0 con 0x7fa2140c88c0 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa213fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa213fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49606/0 (socket says 172.21.15.17:49606) 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.519+0000 7fa213fff640 1 -- 172.21.15.17:0/2183984270 learned_addr learned my addr 172.21.15.17:0/2183984270 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 -- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2140c88c0 msgr2=0x7fa214150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2140c88c0 0x7fa214150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 -- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa214158b60 msgr2=0x7fa21414a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa214158b60 0x7fa21414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 -- 172.21.15.17:0/2183984270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2141eef10 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa213fff640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa200034040 tx=0x7fa2000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200032630 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2000327d0 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200032ba0 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa214150360 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.520+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa214150640 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.521+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa214150150 con 0x7fa21414f360 2024-03-06T07:27:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.521+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa20004e430 con 0x7fa21414f360 2024-03-06T07:27:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.522+0000 7fa2117fa640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 0x7fa1f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.522+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa20002f030 con 0x7fa21414f360 2024-03-06T07:27:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.522+0000 7fa218af8640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 0x7fa1f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.523+0000 7fa218af8640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 0x7fa1f806d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.526+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa200040070 con 0x7fa21414f360 2024-03-06T07:27:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.710+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa21404f330 con 0x7fa21414f360 2024-03-06T07:27:13.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.711+0000 7fa2117fa640 1 -- 172.21.15.17:0/2183984270 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+13303 (secure 0 0 0) 0x7fa20007b000 con 0x7fa21414f360 2024-03-06T07:27:13.714 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:13.714 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":177,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:12.394269+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/1440163522":"2024-03-07T07:27:09.385011+0000","172.21.15.17:6834/1440163522":"2024-03-07T07:27:09.385011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 msgr2=0x7fa1f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 0x7fa1f806d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa208007fd0 tx=0x7fa208007960 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 msgr2=0x7fa2141ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa200034040 tx=0x7fa2000046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 shutdown_connections 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa1f806b4d0 0x7fa1f806d990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa214158b60 0x7fa21414a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa21414f360 0x7fa2141ee5d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 --2- 172.21.15.17:0/2183984270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2140c88c0 0x7fa214150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 >> 172.21.15.17:0/2183984270 conn(0x7fa2140ac570 msgr2=0x7fa21406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 shutdown_connections 2024-03-06T07:27:13.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.713+0000 7fa21ad5c640 1 -- 172.21.15.17:0/2183984270 wait complete. 2024-03-06T07:27:13.727 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:13.727 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:13.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.830+0000 7f5291b3e640 1 Processor -- start 2024-03-06T07:27:13.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.830+0000 7f5291b3e640 1 -- start start 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f5291b3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f528c0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f5291b3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f528c070a50 0x7f528c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f5291b3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c070eb0 con 0x7f528c152e40 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f5291b3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c156940 con 0x7f528c070a50 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f5291b3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c157310 con 0x7f528c0c4c60 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f528c152e40 0x7f528c0724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57484/0 (socket says 172.21.15.17:57484) 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528bfff640 1 -- 172.21.15.17:0/2403716981 learned_addr learned my addr 172.21.15.17:0/2403716981 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815470898 0 0) 0x7f528c070eb0 con 0x7f528c152e40 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528b7fe640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f528c0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5274003520 con 0x7f528c152e40 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528affd640 1 --2- 172.21.15.17:0/2403716981 >> v2:172.21.15.163:3300/0 conn(0x7f528c070a50 0x7f528c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4088690090 0 0) 0x7f5274003520 con 0x7f528c152e40 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 msgr2=0x7f528c0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f528c0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 >> v2:172.21.15.163:3300/0 conn(0x7f528c070a50 msgr2=0x7f528c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 --2- 172.21.15.17:0/2403716981 >> v2:172.21.15.163:3300/0 conn(0x7f528c070a50 0x7f528c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0c80b0 con 0x7f528c152e40 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f52840019f0 con 0x7f528c152e40 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 >> v1:172.21.15.17:6789/0 conn(0x7f528c152e40 legacy=0x7f528c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52740039b0 0x7f5274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f5274004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5274004d40 0x7f52740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c156940 con 0x7f5274004d40 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c157310 con 0x7f52740039b0 2024-03-06T07:27:13.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.831+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527400bb20 con 0x7f528c0c4c60 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5274004d40 0x7f52740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528b7fe640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52740039b0 0x7f5274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528affd640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f5274004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 -- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 msgr2=0x7f5274004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c0c4c60 0x7f5274004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 -- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52740039b0 msgr2=0x7f5274003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52740039b0 0x7f5274003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.832+0000 7f528bfff640 1 -- 172.21.15.17:0/2403716981 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274003520 con 0x7f5274004d40 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f528bfff640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5274004d40 0x7f52740051c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f528400a920 tx=0x7f5274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30e7932ca3738aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400a4c0 con 0x7f5274004d40 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284034630 con 0x7f5274004d40 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f528a7fc640 1 -- 172.21.15.17:0/2403716981 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284034a60 con 0x7f5274004d40 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 -- 172.21.15.17:0/2403716981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5274004d40 msgr2=0x7f52740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 --2- 172.21.15.17:0/2403716981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5274004d40 0x7f52740051c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f528400a920 tx=0x7f5274005830 comp rx=0 tx=0).stop 2024-03-06T07:27:13.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f528bfff640 1 -- 172.21.15.17:0/2403716981 reap_dead start 2024-03-06T07:27:13.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 -- 172.21.15.17:0/2403716981 shutdown_connections 2024-03-06T07:27:13.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 -- 172.21.15.17:0/2403716981 >> 172.21.15.17:0/2403716981 conn(0x7f528c0aa6a0 msgr2=0x7f528c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:13.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 -- 172.21.15.17:0/2403716981 shutdown_connections 2024-03-06T07:27:13.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.833+0000 7f5291b3e640 1 -- 172.21.15.17:0/2403716981 wait complete. 2024-03-06T07:27:13.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.834+0000 7f5291b3e640 1 Processor -- start 2024-03-06T07:27:13.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 -- start start 2024-03-06T07:27:13.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f528c155250 0x7f528c1def30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c155630 0x7f528c1ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1532d0 con 0x7f528c06d8b0 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1565d0 con 0x7f528c155250 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f5291b3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c072be0 con 0x7f528c155630 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f528bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f528bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49622/0 (socket says 172.21.15.17:49622) 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.835+0000 7f528bfff640 1 -- 172.21.15.17:0/3398433728 learned_addr learned my addr 172.21.15.17:0/3398433728 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 -- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c155630 msgr2=0x7f528c1ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c155630 0x7f528c1ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 -- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f528c155250 msgr2=0x7f528c1def30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f528c155250 0x7f528c1def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 -- 172.21.15.17:0/3398433728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c1eb420 con 0x7f528c06d8b0 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f528bfff640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f528400a920 tx=0x7f528c1568b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400a4c0 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284045070 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c1eb700 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284002000 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.836+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c155cf0 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.837+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c04f330 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.837+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5284034bf0 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.838+0000 7f5273fff640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 0x7f527806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.838+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f52840ac830 con 0x7f528c06d8b0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.838+0000 7f528b7fe640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 0x7f527806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:13.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.838+0000 7f528b7fe640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 0x7f527806d9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f527c006fd0 tx=0x7f527c0040a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:13.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:13.842+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f528407b8b0 con 0x7f528c06d8b0 2024-03-06T07:27:14.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.025+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f528c0c2fd0 con 0x7f528c06d8b0 2024-03-06T07:27:14.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.025+0000 7f5273fff640 1 -- 172.21.15.17:0/3398433728 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f528407b1b0 con 0x7f528c06d8b0 2024-03-06T07:27:14.028 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 msgr2=0x7f527806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 0x7f527806d9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f527c006fd0 tx=0x7f527c0040a0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 msgr2=0x7f528c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f528400a920 tx=0x7f528c1568b0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 shutdown_connections 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f527806b500 0x7f527806d9c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f528c06d8b0 0x7f528c06dd30 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f528c155630 0x7f528c1ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 --2- 172.21.15.17:0/3398433728 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f528c155250 0x7f528c1def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 >> 172.21.15.17:0/3398433728 conn(0x7f528c0aa6a0 msgr2=0x7f528c070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 shutdown_connections 2024-03-06T07:27:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.027+0000 7f5291b3e640 1 -- 172.21.15.17:0/3398433728 wait complete. 2024-03-06T07:27:14.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow' 2024-03-06T07:27:14.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.139+0000 7eff6aef4640 1 Processor -- start 2024-03-06T07:27:14.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.139+0000 7eff6aef4640 1 -- start start 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff6aef4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff64155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff6aef4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff64158b60 0x7eff64158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff6aef4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff640c73d0 con 0x7eff6414f360 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff6aef4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640c8dd0 con 0x7eff64158b60 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff6aef4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640be2e0 con 0x7eff640c88c0 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff64158b60 0x7eff64158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7eff64158b60 0x7eff64158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49636/0 (socket says 172.21.15.17:49636) 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 -- 172.21.15.17:0/2906390983 learned_addr learned my addr 172.21.15.17:0/2906390983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff63fff640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff64155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 -- 172.21.15.17:0/2906390983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 msgr2=0x7eff64155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff64155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.140+0000 7eff69491640 1 -- 172.21.15.17:0/2906390983 >> v1:172.21.15.163:6789/0 conn(0x7eff6414f360 legacy=0x7eff640c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.141+0000 7eff69491640 1 -- 172.21.15.17:0/2906390983 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640c7670 con 0x7eff64158b60 2024-03-06T07:27:14.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.141+0000 7eff69491640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff64158ff0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009e50 tx=0x7eff5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ed51ebcf889030 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.141+0000 7eff637fe640 1 -- 172.21.15.17:0/2906390983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c02ca70 con 0x7eff64158b60 2024-03-06T07:27:14.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.141+0000 7eff637fe640 1 -- 172.21.15.17:0/2906390983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c02cc10 con 0x7eff64158b60 2024-03-06T07:27:14.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.141+0000 7eff637fe640 1 -- 172.21.15.17:0/2906390983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0343f0 con 0x7eff64158b60 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 -- 172.21.15.17:0/2906390983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 msgr2=0x7eff64158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff64158ff0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009e50 tx=0x7eff5c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 -- 172.21.15.17:0/2906390983 shutdown_connections 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff64158ff0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 --2- 172.21.15.17:0/2906390983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff64155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 -- 172.21.15.17:0/2906390983 >> 172.21.15.17:0/2906390983 conn(0x7eff640ac570 msgr2=0x7eff640ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 -- 172.21.15.17:0/2906390983 shutdown_connections 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.142+0000 7eff6aef4640 1 -- 172.21.15.17:0/2906390983 wait complete. 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.143+0000 7eff6aef4640 1 Processor -- start 2024-03-06T07:27:14.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.143+0000 7eff6aef4640 1 -- start start 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff640c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 0x7eff641ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640c5a50 con 0x7eff64158b60 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640c7c30 con 0x7eff6414f360 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff6aef4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640c5bd0 con 0x7eff640c88c0 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49638/0 (socket says 172.21.15.17:49638) 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 -- 172.21.15.17:0/3389590597 learned_addr learned my addr 172.21.15.17:0/3389590597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff63fff640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 0x7eff641ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff68c90640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff640c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 -- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 msgr2=0x7eff640c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff640c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 -- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 msgr2=0x7eff641ec1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 0x7eff641ec1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 -- 172.21.15.17:0/3389590597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6414db90 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff63fff640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 0x7eff641ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff68c90640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff640c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.144+0000 7eff69491640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009f70 tx=0x7eff5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.145+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c044020 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.145+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c040690 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.145+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff640bf280 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.145+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c009510 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.145+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff640bf840 con 0x7eff64158b60 2024-03-06T07:27:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.146+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6404f330 con 0x7eff64158b60 2024-03-06T07:27:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.146+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff5c005d10 con 0x7eff64158b60 2024-03-06T07:27:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.146+0000 7eff617fa640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 0x7eff5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.146+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7eff5c0ac6c0 con 0x7eff64158b60 2024-03-06T07:27:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.147+0000 7eff68c90640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 0x7eff5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.147+0000 7eff68c90640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 0x7eff5006d9c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7eff540015e0 tx=0x7eff54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.151+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff5c04e020 con 0x7eff64158b60 2024-03-06T07:27:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:14.401+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]} v 0) v1 -- 0x7eff6406cda0 con 0x7eff64158b60 2024-03-06T07:27:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.404+0000 7eff617fa640 1 -- 172.21.15.17:0/3389590597 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]}]=0 v423) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff5c07b740 con 0x7eff64158b60 2024-03-06T07:27:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 msgr2=0x7eff5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 0x7eff5006d9c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7eff540015e0 tx=0x7eff54009040 comp rx=0 tx=0).stop 2024-03-06T07:27:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 msgr2=0x7eff641ec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009f70 tx=0x7eff5c004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 shutdown_connections 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff5006b500 0x7eff5006d9c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.406+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff64158b60 0x7eff641ec6e0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.407+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff6414f360 0x7eff641ec1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.407+0000 7eff6aef4640 1 --2- 172.21.15.17:0/3389590597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff640c88c0 0x7eff640c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.407+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 >> 172.21.15.17:0/3389590597 conn(0x7eff640ac570 msgr2=0x7eff6406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.407+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 shutdown_connections 2024-03-06T07:27:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.407+0000 7eff6aef4640 1 -- 172.21.15.17:0/3389590597 wait complete. 2024-03-06T07:27:15.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:27:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.536+0000 7f3bea2f4640 1 Processor -- start 2024-03-06T07:27:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.536+0000 7f3bea2f4640 1 -- start start 2024-03-06T07:27:15.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.536+0000 7f3bea2f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 0x7f3be40c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.536+0000 7f3bea2f4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3be4073190 0x7f3be4156530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3bea2f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be4072600 con 0x7f3be40c5c60 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3bea2f4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40723b0 con 0x7f3be4073190 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3bea2f4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40728e0 con 0x7f3be40c5880 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3be4073190 0x7f3be4156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3be4073190 0x7f3be4156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49648/0 (socket says 172.21.15.17:49648) 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 -- 172.21.15.17:0/3641475798 learned_addr learned my addr 172.21.15.17:0/3641475798 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be3fff640 1 --2- 172.21.15.17:0/3641475798 >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 0x7f3be40c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 -- 172.21.15.17:0/3641475798 >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 msgr2=0x7f3be40c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 --2- 172.21.15.17:0/3641475798 >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 0x7f3be40c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 -- 172.21.15.17:0/3641475798 >> v1:172.21.15.163:6789/0 conn(0x7f3be40c5c60 legacy=0x7f3be40c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be8891640 1 -- 172.21.15.17:0/3641475798 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be4153af0 con 0x7f3be4073190 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.537+0000 7f3be3fff640 1 --2- 172.21.15.17:0/3641475798 >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 0x7f3be40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.538+0000 7f3be8891640 1 --2- 172.21.15.17:0/3641475798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be4156530 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f3be4070f80 tx=0x7f3bdc02bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff45bf387b0707d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.538+0000 7f3be2ffd640 1 -- 172.21.15.17:0/3641475798 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc02c960 con 0x7f3be4073190 2024-03-06T07:27:15.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.538+0000 7f3be2ffd640 1 -- 172.21.15.17:0/3641475798 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc02cb00 con 0x7f3be4073190 2024-03-06T07:27:15.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.538+0000 7f3be2ffd640 1 -- 172.21.15.17:0/3641475798 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0343f0 con 0x7f3be4073190 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 -- 172.21.15.17:0/3641475798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 msgr2=0x7f3be4156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/3641475798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be4156530 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f3be4070f80 tx=0x7f3bdc02bdc0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 -- 172.21.15.17:0/3641475798 shutdown_connections 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/3641475798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be4156530 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/3641475798 >> v2:172.21.15.163:3300/0 conn(0x7f3be40c5880 0x7f3be40c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 -- 172.21.15.17:0/3641475798 >> 172.21.15.17:0/3641475798 conn(0x7f3be40aa630 msgr2=0x7f3be40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 -- 172.21.15.17:0/3641475798 shutdown_connections 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.539+0000 7f3bea2f4640 1 -- 172.21.15.17:0/3641475798 wait complete. 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.540+0000 7f3bea2f4640 1 Processor -- start 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.540+0000 7f3bea2f4640 1 -- start start 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 0x7f3be406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 0x7f3be406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c78f0 con 0x7f3be4073190 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40c6780 con 0x7f3be40c5880 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4156a70 con 0x7f3be40c5c60 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49652/0 (socket says 172.21.15.17:49652) 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 -- 172.21.15.17:0/2843200656 learned_addr learned my addr 172.21.15.17:0/2843200656 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be37fe640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 0x7f3be406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be8891640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 0x7f3be406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 -- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 msgr2=0x7f3be406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 0x7f3be406c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 -- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 msgr2=0x7f3be406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 0x7f3be406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 -- 172.21.15.17:0/2843200656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40c2f50 con 0x7f3be4073190 2024-03-06T07:27:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be37fe640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 0x7f3be406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be8891640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 0x7f3be406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be3fff640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c7f0 tx=0x7f3bd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4015070 con 0x7f3be4073190 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd4007a30 con 0x7f3be4073190 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4007e00 con 0x7f3be4073190 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.541+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be406d000 con 0x7f3be4073190 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.542+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be41eb870 con 0x7f3be4073190 2024-03-06T07:27:15.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.542+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be404f330 con 0x7f3be4073190 2024-03-06T07:27:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.543+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bd4018070 con 0x7f3be4073190 2024-03-06T07:27:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.543+0000 7f3be0ff9640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.543+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3bd4089fb0 con 0x7f3be4073190 2024-03-06T07:27:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.544+0000 7f3be37fe640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.544+0000 7f3be37fe640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 0x7f3bd006d9c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3be414fd70 tx=0x7f3bd8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:15.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.548+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bd4059030 con 0x7f3be4073190 2024-03-06T07:27:15.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.728+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3be4072ca0 con 0x7f3be4073190 2024-03-06T07:27:15.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.731+0000 7f3be0ff9640 1 -- 172.21.15.17:0/2843200656 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v59)=0 v59) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3bd4058930 con 0x7f3be4073190 2024-03-06T07:27:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 msgr2=0x7f3bd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 0x7f3bd006d9c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3be414fd70 tx=0x7f3bd8008040 comp rx=0 tx=0).stop 2024-03-06T07:27:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 msgr2=0x7f3be40c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c7f0 tx=0x7f3bd400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 shutdown_connections 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bd006b500 0x7f3bd006d9c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be40c5c60 0x7f3be406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be40c5880 0x7f3be406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 --2- 172.21.15.17:0/2843200656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3be4073190 0x7f3be40c1cb0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 >> 172.21.15.17:0/2843200656 conn(0x7f3be40aa630 msgr2=0x7f3be414f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 shutdown_connections 2024-03-06T07:27:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.733+0000 7f3bea2f4640 1 -- 172.21.15.17:0/2843200656 wait complete. 2024-03-06T07:27:15.746 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:27:15.747 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:15.797 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:15.839 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:27:15.839 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:27:15.839 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:15.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:27:15.854 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:27:15.854 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:15.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:27:15.897 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:27:15.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.995+0000 7f31185ac640 1 Processor -- start 2024-03-06T07:27:15.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.995+0000 7f31185ac640 1 -- start start 2024-03-06T07:27:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f31185ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31100c8a00 0x7f3110158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f31185ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31100c7760 con 0x7f3110159460 2024-03-06T07:27:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f31185ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31100c7a40 con 0x7f3110152920 2024-03-06T07:27:16.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f31185ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31101587b0 con 0x7f31100c8a00 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f3116b49640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3110159460 0x7f31100c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57490/0 (socket says 172.21.15.17:57490) 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f3116b49640 1 -- 172.21.15.17:0/1200706091 learned_addr learned my addr 172.21.15.17:0/1200706091 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663547348 0 0) 0x7f31100c7760 con 0x7f3110159460 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f8003520 con 0x7f3110159460 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.996+0000 7f3115b47640 1 --2- 172.21.15.17:0/1200706091 >> v2:172.21.15.163:3300/0 conn(0x7f31100c8a00 0x7f3110158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 29961672 0 0) 0x7f30f8003520 con 0x7f3110159460 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 >> v2:172.21.15.163:3300/0 conn(0x7f31100c8a00 msgr2=0x7f3110158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 --2- 172.21.15.17:0/1200706091 >> v2:172.21.15.163:3300/0 conn(0x7f31100c8a00 0x7f3110158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 >> v1:172.21.15.163:6789/0 conn(0x7f3110152920 legacy=0x7f3110155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100c7d20 con 0x7f3110159460 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f31000019f0 con 0x7f3110159460 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 >> v1:172.21.15.17:6789/0 conn(0x7f3110159460 legacy=0x7f31100c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80052e0 0x7f30f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101587b0 con 0x7f30f80039b0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c1b0 con 0x7f30f80052e0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c3b0 con 0x7f30f8004800 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116b49640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80052e0 0x7f30f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115b47640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 -- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 msgr2=0x7f30f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 -- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80052e0 msgr2=0x7f30f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80052e0 0x7f30f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116348640 1 -- 172.21.15.17:0/1200706091 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8003520 con 0x7f30f80039b0 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3116b49640 1 -- 172.21.15.17:0/1200706091 reap_dead start 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.997+0000 7f3115b47640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:16.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f3116348640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f3108001720 tx=0x7f30f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6518d0b36bf9de55 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108001370 con 0x7f30f80039b0 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080052d0 con 0x7f30f80039b0 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f3115346640 1 -- 172.21.15.17:0/1200706091 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108003830 con 0x7f30f80039b0 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f31185ac640 1 -- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 msgr2=0x7f30f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f31185ac640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f3108001720 tx=0x7f30f8005830 comp rx=0 tx=0).stop 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f31185ac640 1 -- 172.21.15.17:0/1200706091 shutdown_connections 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f31185ac640 1 --2- 172.21.15.17:0/1200706091 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.998+0000 7f31185ac640 1 -- 172.21.15.17:0/1200706091 >> 172.21.15.17:0/1200706091 conn(0x7f31100ac560 msgr2=0x7f31100ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.999+0000 7f31185ac640 1 -- 172.21.15.17:0/1200706091 shutdown_connections 2024-03-06T07:27:16.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:15.999+0000 7f31185ac640 1 -- 172.21.15.17:0/1200706091 wait complete. 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 Processor -- start 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 -- start start 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 0x7f31101eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31101496d0 0x7f31101eb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c7390 con 0x7f31100c1e60 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311006d030 con 0x7f30f80039b0 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f31185ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c7510 con 0x7f31101496d0 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f3116b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f3116b49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49664/0 (socket says 172.21.15.17:49664) 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.000+0000 7f3116b49640 1 -- 172.21.15.17:0/371771042 learned_addr learned my addr 172.21.15.17:0/371771042 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3115b47640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31101496d0 0x7f31101eb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116348640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 0x7f31101eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 -- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31101496d0 msgr2=0x7f31101eb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31101496d0 0x7f31101eb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 -- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 msgr2=0x7f31101eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 0x7f31101eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 -- 172.21.15.17:0/371771042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31101ebea0 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116348640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 0x7f31101eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f3116b49640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f31100c8960 tx=0x7f3100007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310000ebc0 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100018070 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100013420 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31101ec900 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.001+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31101581e0 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.002+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311006c8a0 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.003+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31000135c0 con 0x7f31100c1e60 2024-03-06T07:27:16.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.003+0000 7f310effd640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:16.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.003+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f310008a350 con 0x7f31100c1e60 2024-03-06T07:27:16.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.003+0000 7f3116348640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:16.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.004+0000 7f3116348640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3108033000 tx=0x7f3108009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:16.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.007+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f311006c8a0 con 0x7f31100c1e60 2024-03-06T07:27:16.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.225+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f311004f330 con 0x7f31100c1e60 2024-03-06T07:27:16.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.226+0000 7f310effd640 1 -- 172.21.15.17:0/371771042 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f31000593d0 con 0x7f31100c1e60 2024-03-06T07:27:16.228 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:16.229 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":168,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8668,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1779308370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":1779308370},{"type":"v1","addr":"172.21.15.163:6836","nonce":1779308370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":8671,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1514084118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1514084118},{"type":"v1","addr":"172.21.15.163:6837","nonce":1514084118}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/2769315010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2769315010},{"type":"v1","addr":"172.21.15.17:6833","nonce":2769315010}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":155},{"gid":9804,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/229855171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":229855171},{"type":"v1","addr":"172.21.15.17:6835","nonce":229855171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[]} 2024-03-06T07:27:16.229 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 168 2024-03-06T07:27:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.227+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 msgr2=0x7f30fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3108033000 tx=0x7f3108009030 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 msgr2=0x7f31101ec3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f31100c8960 tx=0x7f3100007bc0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 shutdown_connections 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30fc06b4d0 0x7f30fc06d990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31100c1e60 0x7f31101ec3c0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31101496d0 0x7f31101eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 --2- 172.21.15.17:0/371771042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80039b0 0x7f31101eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 >> 172.21.15.17:0/371771042 conn(0x7f31100ac560 msgr2=0x7f311014d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 shutdown_connections 2024-03-06T07:27:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:16.228+0000 7f31185ac640 1 -- 172.21.15.17:0/371771042 wait complete. 2024-03-06T07:27:16.241 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:27:16.241 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:27:16.241 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:27:16.242 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:16.242 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:27:16.242 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:27:16.242 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:27:16.242 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:16.242 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:27:16.242 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:27:16.243 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:27:16.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:16.243 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:27:16.243 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:27:16.243 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:27:16.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:27:16.251 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:27:16.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:27:16.254 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:27:16.254 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:27:16.255 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:27:16.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:27:16.293 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:27:16.294 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:16.297 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:27:16.297 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:16.297 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:27:16.298 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:16.367 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:27:16.379 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:27:16.379 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:27:16.379 INFO:tasks.ceph.mds.b.smithi163.stderr:2024-03-06T07:27:16.378+0000 7f3ee772d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-03-06T07:27:19.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-06T07:27:22.251 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:27:22.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:27:22.254 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:27:22.254 DEBUG:teuthology.parallel:result is None 2024-03-06T07:27:22.254 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:27:22.340 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:27:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.366+0000 7f57e815c640 1 Processor -- start 2024-03-06T07:27:22.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 -- start start 2024-03-06T07:27:22.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57e00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57e01558d0 0x7f57e0155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e0155130 con 0x7f57e00c8950 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e0155440 con 0x7f57e01558d0 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e815c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ad480 con 0x7f57e00c69f0 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e56f7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f57e00c8950 0x7f57e00acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60934/0 (socket says 172.21.15.17:60934) 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.367+0000 7f57e56f7640 1 -- 172.21.15.17:0/593069450 learned_addr learned my addr 172.21.15.17:0/593069450 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 --2- 172.21.15.17:0/593069450 >> v2:172.21.15.163:3300/0 conn(0x7f57e01558d0 0x7f57e0155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e5ef8640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57e00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785193953 0 0) 0x7f57dc001110 con 0x7f57e00c8950 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003520 con 0x7f57e00c8950 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205710222 0 0) 0x7f57cc003520 con 0x7f57e00c8950 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 msgr2=0x7f57e00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57e00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 >> v2:172.21.15.163:3300/0 conn(0x7f57e01558d0 msgr2=0x7f57e0155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 --2- 172.21.15.17:0/593069450 >> v2:172.21.15.163:3300/0 conn(0x7f57e01558d0 0x7f57e0155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00c5d50 con 0x7f57e00c8950 2024-03-06T07:27:22.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f57dc001840 con 0x7f57e00c8950 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 >> v1:172.21.15.17:6789/0 conn(0x7f57e00c8950 legacy=0x7f57e00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc003f30 0x7f57cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57cc004d40 0x7f57cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0155440 con 0x7f57cc004d40 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00ad480 con 0x7f57cc003f30 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00ba10 con 0x7f57e00c69f0 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57cc004d40 0x7f57cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e5ef8640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e56f7640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc003f30 0x7f57cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 -- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 msgr2=0x7f57cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e00c69f0 0x7f57cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 -- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc003f30 msgr2=0x7f57cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc003f30 0x7f57cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e66f9640 1 -- 172.21.15.17:0/593069450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc003520 con 0x7f57cc004d40 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.368+0000 7f57e56f7640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57cc003f30 0x7f57cc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.369+0000 7f57e66f9640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57cc004d40 0x7f57cc0051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e170 tx=0x7f57cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d35fa4f30b2801a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.369+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8039070 con 0x7f57cc004d40 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.369+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d802cc20 con 0x7f57cc004d40 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.369+0000 7f57e4ef6640 1 -- 172.21.15.17:0/593069450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8009c10 con 0x7f57cc004d40 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 -- 172.21.15.17:0/593069450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57cc004d40 msgr2=0x7f57cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 --2- 172.21.15.17:0/593069450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57cc004d40 0x7f57cc0051c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f57d800e170 tx=0x7f57cc005830 comp rx=0 tx=0).stop 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e66f9640 1 -- 172.21.15.17:0/593069450 reap_dead start 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 -- 172.21.15.17:0/593069450 shutdown_connections 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 -- 172.21.15.17:0/593069450 >> 172.21.15.17:0/593069450 conn(0x7f57e00cae70 msgr2=0x7f57e00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 -- 172.21.15.17:0/593069450 shutdown_connections 2024-03-06T07:27:22.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.370+0000 7f57e815c640 1 -- 172.21.15.17:0/593069450 wait complete. 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.371+0000 7f57e815c640 1 Processor -- start 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.371+0000 7f57e815c640 1 -- start start 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e006c820 0x7f57e014cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 0x7f57e014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0152a20 con 0x7f57e006cc00 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c6310 con 0x7f57e006c820 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e815c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0152ba0 con 0x7f57e014be90 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42598/0 (socket says 172.21.15.17:42598) 2024-03-06T07:27:22.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 -- 172.21.15.17:0/3597356144 learned_addr learned my addr 172.21.15.17:0/3597356144 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e5ef8640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e006c820 0x7f57e014cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e66f9640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 0x7f57e014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 -- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 msgr2=0x7f57e014c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 0x7f57e014c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 -- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e006c820 msgr2=0x7f57e014cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e006c820 0x7f57e014cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e56f7640 1 -- 172.21.15.17:0/3597356144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0157c80 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.372+0000 7f57e66f9640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 0x7f57e014c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57e56f7640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e8f0 tx=0x7f57dc00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00c8d0 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc018070 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e0159330 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00cc70 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.373+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0158200 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.374+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e004f330 con 0x7f57e006cc00 2024-03-06T07:27:22.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.374+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57dc013420 con 0x7f57e006cc00 2024-03-06T07:27:22.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.375+0000 7f57d67fc640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:22.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.375+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f57dc089bd0 con 0x7f57e006cc00 2024-03-06T07:27:22.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.375+0000 7f57e5ef8640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:22.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.376+0000 7f57e5ef8640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 0x7f57c406d9c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009bc0 tx=0x7f57d0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:22.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.379+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57e0155130 con 0x7f57e006cc00 2024-03-06T07:27:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:22.575+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f57e00ca650 con 0x7f57e006cc00 2024-03-06T07:27:23.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.506+0000 7f57d67fc640 1 -- 172.21.15.17:0/3597356144 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v182) v1 ==== 97+0+0 (secure 0 0 0) 0x7f57dc058c50 con 0x7f57e006cc00 2024-03-06T07:27:23.509 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:27:23.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 msgr2=0x7f57c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 0x7f57c406d9c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009bc0 tx=0x7f57d0009190 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 msgr2=0x7f57e0158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e8f0 tx=0x7f57dc00ecf0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 shutdown_connections 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57e014be90 0x7f57e014c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57e006cc00 0x7f57e0158df0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 --2- 172.21.15.17:0/3597356144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57e006c820 0x7f57e014cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 >> 172.21.15.17:0/3597356144 conn(0x7f57e00cae70 msgr2=0x7f57e0150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 shutdown_connections 2024-03-06T07:27:23.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.509+0000 7f57e815c640 1 -- 172.21.15.17:0/3597356144 wait complete. 2024-03-06T07:27:23.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:27:23.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.629+0000 7efc88c9d640 1 Processor -- start 2024-03-06T07:27:23.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.629+0000 7efc88c9d640 1 -- start start 2024-03-06T07:27:23.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc88c9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c8a00 0x7efc84158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc88c9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc84159460 0x7efc840c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc88c9d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840c7760 con 0x7efc84152920 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc88c9d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc841587b0 con 0x7efc84159460 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc88c9d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c7a40 con 0x7efc840c8a00 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc84159460 0x7efc840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc84159460 0x7efc840c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42606/0 (socket says 172.21.15.17:42606) 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 -- 172.21.15.17:0/2661491852 learned_addr learned my addr 172.21.15.17:0/2661491852 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 -- 172.21.15.17:0/2661491852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c8a00 msgr2=0x7efc84158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 --2- 172.21.15.17:0/2661491852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c8a00 0x7efc84158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 -- 172.21.15.17:0/2661491852 >> v1:172.21.15.163:6789/0 conn(0x7efc84152920 legacy=0x7efc84155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:23.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.630+0000 7efc82d9d640 1 -- 172.21.15.17:0/2661491852 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840c7d20 con 0x7efc84159460 2024-03-06T07:27:23.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc82d9d640 1 --2- 172.21.15.17:0/2661491852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84159460 0x7efc840c5330 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009e50 tx=0x7efc7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94d81bcf9cea705 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc8159a640 1 -- 172.21.15.17:0/2661491852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c02c9e0 con 0x7efc84159460 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc8159a640 1 -- 172.21.15.17:0/2661491852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c02cb80 con 0x7efc84159460 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc8159a640 1 -- 172.21.15.17:0/2661491852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0343f0 con 0x7efc84159460 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc88c9d640 1 -- 172.21.15.17:0/2661491852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84159460 msgr2=0x7efc840c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.631+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2661491852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84159460 0x7efc840c5330 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009e50 tx=0x7efc7c02be40 comp rx=0 tx=0).stop 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 -- 172.21.15.17:0/2661491852 shutdown_connections 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2661491852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84159460 0x7efc840c5330 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2661491852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc840c8a00 0x7efc84158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 -- 172.21.15.17:0/2661491852 >> 172.21.15.17:0/2661491852 conn(0x7efc840ac560 msgr2=0x7efc840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 -- 172.21.15.17:0/2661491852 shutdown_connections 2024-03-06T07:27:23.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.632+0000 7efc88c9d640 1 -- 172.21.15.17:0/2661491852 wait complete. 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.633+0000 7efc88c9d640 1 Processor -- start 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.633+0000 7efc88c9d640 1 -- start start 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152920 0x7efc841e0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 0x7efc841eb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c7390 con 0x7efc840c8a00 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8406d030 con 0x7efc841eb0c0 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c7510 con 0x7efc84152920 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42618/0 (socket says 172.21.15.17:42618) 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 -- 172.21.15.17:0/2181490891 learned_addr learned my addr 172.21.15.17:0/2181490891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc81d9b640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152920 0x7efc841e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc82d9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 0x7efc841eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 -- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152920 msgr2=0x7efc841e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152920 0x7efc841e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 -- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 msgr2=0x7efc841eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 0x7efc841eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 -- 172.21.15.17:0/2181490891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc841dfbf0 con 0x7efc840c8a00 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc82d9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 0x7efc841eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc8259c640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c410 tx=0x7efc7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74017070 con 0x7efc840c8a00 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc740035f0 con 0x7efc840c8a00 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc841ee860 con 0x7efc840c8a00 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc740039e0 con 0x7efc840c8a00 2024-03-06T07:27:23.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.634+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc841e0ca0 con 0x7efc840c8a00 2024-03-06T07:27:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.635+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8404f330 con 0x7efc840c8a00 2024-03-06T07:27:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.636+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc7401a070 con 0x7efc840c8a00 2024-03-06T07:27:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.636+0000 7efc72ffd640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 0x7efc6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.636+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc74089490 con 0x7efc840c8a00 2024-03-06T07:27:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.637+0000 7efc81d9b640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 0x7efc6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.637+0000 7efc81d9b640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 0x7efc6406d9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efc840be2c0 tx=0x7efc78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:23.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.641+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc740584c0 con 0x7efc840c8a00 2024-03-06T07:27:23.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.890+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efc84156390 con 0x7efc840c8a00 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.890+0000 7efc72ffd640 1 -- 172.21.15.17:0/2181490891 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+4724 (secure 0 0 0) 0x7efc740010c0 con 0x7efc840c8a00 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:27:23.893 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:27:23.894 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:27:23.896 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.897 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:27:23.903 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.904 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:27:23.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.906 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:23.906 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:27:23.906 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 msgr2=0x7efc6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 0x7efc6406d9c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efc840be2c0 tx=0x7efc78008040 comp rx=0 tx=0).stop 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 msgr2=0x7efc841dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c410 tx=0x7efc7400c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 shutdown_connections 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6406b500 0x7efc6406d9c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc841eb0c0 0x7efc841eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc84152920 0x7efc841e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 --2- 172.21.15.17:0/2181490891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c8a00 0x7efc841dfed0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 >> 172.21.15.17:0/2181490891 conn(0x7efc840ac560 msgr2=0x7efc84151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 shutdown_connections 2024-03-06T07:27:23.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:23.892+0000 7efc88c9d640 1 -- 172.21.15.17:0/2181490891 wait complete. 2024-03-06T07:27:23.908 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:24.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 Processor -- start 2024-03-06T07:27:24.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 -- start start 2024-03-06T07:27:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fabfc06da40 0x7fabfc155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc156050 con 0x7fabfc0c7380 2024-03-06T07:27:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabfc072630 con 0x7fabfc0c6780 2024-03-06T07:27:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.008+0000 7fac01050640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc06de20 con 0x7fabfc06da40 2024-03-06T07:27:24.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fabfc06da40 0x7fabfc155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfb59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fabfc0c7380 0x7fabfc0c3280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60938/0 (socket says 172.21.15.17:60938) 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfb59e640 1 -- 172.21.15.17:0/831491897 learned_addr learned my addr 172.21.15.17:0/831491897 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfad9d640 1 -- 172.21.15.17:0/831491897 >> v1:172.21.15.163:6789/0 conn(0x7fabfc0c6780 legacy=0x7fabfc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfad9d640 1 -- 172.21.15.17:0/831491897 >> v1:172.21.15.17:6789/0 conn(0x7fabfc0c7380 legacy=0x7fabfc0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfad9d640 1 -- 172.21.15.17:0/831491897 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc072910 con 0x7fabfc06da40 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.009+0000 7fabfa59c640 1 -- 172.21.15.17:0/831491897 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896700364 0 0) 0x7fabf4001490 con 0x7fabfc0c7380 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fabfad9d640 1 --2- 172.21.15.17:0/831491897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc06da40 0x7fabfc155b10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097a0 tx=0x7fabec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28f8a4af16bc79 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fabfa59c640 1 -- 172.21.15.17:0/831491897 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0315a0 con 0x7fabfc06da40 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fabfa59c640 1 -- 172.21.15.17:0/831491897 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec037070 con 0x7fabfc06da40 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fabfa59c640 1 -- 172.21.15.17:0/831491897 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec031960 con 0x7fabfc06da40 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 -- 172.21.15.17:0/831491897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc06da40 msgr2=0x7fabfc155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 --2- 172.21.15.17:0/831491897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc06da40 0x7fabfc155b10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097a0 tx=0x7fabec02b840 comp rx=0 tx=0).stop 2024-03-06T07:27:24.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 -- 172.21.15.17:0/831491897 shutdown_connections 2024-03-06T07:27:24.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 --2- 172.21.15.17:0/831491897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc06da40 0x7fabfc155b10 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 -- 172.21.15.17:0/831491897 >> 172.21.15.17:0/831491897 conn(0x7fabfc0aa630 msgr2=0x7fabfc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 -- 172.21.15.17:0/831491897 shutdown_connections 2024-03-06T07:27:24.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.010+0000 7fac01050640 1 -- 172.21.15.17:0/831491897 wait complete. 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 Processor -- start 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 -- start start 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabfc06da40 0x7fabfc1eb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabfc0c7380 0x7fabfc1eef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc1594a0 con 0x7fabfc0c6780 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc158890 con 0x7fabfc0c7380 2024-03-06T07:27:24.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fac01050640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0c3990 con 0x7fabfc06da40 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fabf3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42638/0 (socket says 172.21.15.17:42638) 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.012+0000 7fabf3fff640 1 -- 172.21.15.17:0/1005942377 learned_addr learned my addr 172.21.15.17:0/1005942377 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabfad9d640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabfc06da40 0x7fabfc1eb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabfb59e640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabfc0c7380 0x7fabfc1eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 -- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabfc06da40 msgr2=0x7fabfc1eb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabfc06da40 0x7fabfc1eb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 -- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabfc0c7380 msgr2=0x7fabfc1eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabfc0c7380 0x7fabfc1eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 -- 172.21.15.17:0/1005942377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc1dea60 con 0x7fabfc0c6780 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf3fff640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b420 tx=0x7fabe400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4011830 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe4016070 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4011c10 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc1ee720 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.013+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc156c10 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.014+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc04f330 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.014+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabe401b460 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.015+0000 7fabf37fe640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 0x7fabdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.015+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fabe4089aa0 con 0x7fabfc0c6780 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.015+0000 7fabfad9d640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 0x7fabdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.016+0000 7fabfad9d640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 0x7fabdc06d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fabec036000 tx=0x7fabec03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.019+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabe4020020 con 0x7fabfc0c6780 2024-03-06T07:27:24.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.200+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabfc073240 con 0x7fabfc0c6780 2024-03-06T07:27:24.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.201+0000 7fabf37fe640 1 -- 172.21.15.17:0/1005942377 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+13174 (secure 0 0 0) 0x7fabe4058b70 con 0x7fabfc0c6780 2024-03-06T07:27:24.204 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:24.204 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:23.502595+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 msgr2=0x7fabdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 0x7fabdc06d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fabec036000 tx=0x7fabec03d040 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 msgr2=0x7fabfc1566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fabe400b420 tx=0x7fabe400b8e0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 shutdown_connections 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabdc06b4d0 0x7fabdc06d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabfc0c7380 0x7fabfc1eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabfc0c6780 0x7fabfc1566d0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 --2- 172.21.15.17:0/1005942377 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabfc06da40 0x7fabfc1eb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 >> 172.21.15.17:0/1005942377 conn(0x7fabfc0aa630 msgr2=0x7fabfc0c7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 shutdown_connections 2024-03-06T07:27:24.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.203+0000 7fac01050640 1 -- 172.21.15.17:0/1005942377 wait complete. 2024-03-06T07:27:24.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:24.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.318+0000 7f1e7ea66640 1 Processor -- start 2024-03-06T07:27:24.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.318+0000 7f1e7ea66640 1 -- start start 2024-03-06T07:27:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e7806d750 0x7f1e780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1e780c3a70 0x7f1e78072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e7806db30 0x7f1e780c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78155b30 con 0x7f1e7806db30 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78156060 con 0x7f1e780c3a70 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7ea66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780a8430 con 0x7f1e7806d750 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7d003640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e7806db30 0x7f1e780c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7d003640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e7806db30 0x7f1e780c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42644/0 (socket says 172.21.15.17:42644) 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7d003640 1 -- 172.21.15.17:0/177820428 learned_addr learned my addr 172.21.15.17:0/177820428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.319+0000 7f1e7c802640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e7806d750 0x7f1e780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e77fff640 1 --2- 172.21.15.17:0/177820428 >> v2:172.21.15.163:3300/0 conn(0x7f1e780c3a70 0x7f1e78072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e7d003640 1 -- 172.21.15.17:0/177820428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e7806d750 msgr2=0x7f1e780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e7d003640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e7806d750 0x7f1e780732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e7d003640 1 -- 172.21.15.17:0/177820428 >> v2:172.21.15.163:3300/0 conn(0x7f1e780c3a70 msgr2=0x7f1e78072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e7d003640 1 --2- 172.21.15.17:0/177820428 >> v2:172.21.15.163:3300/0 conn(0x7f1e780c3a70 0x7f1e78072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.320+0000 7f1e7d003640 1 -- 172.21.15.17:0/177820428 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78155d80 con 0x7f1e7806db30 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7d003640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806db30 0x7f1e780c1e90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1e780ab7d0 tx=0x7f1e6002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f94e4e0123ed62 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e777fe640 1 -- 172.21.15.17:0/177820428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60038070 con 0x7f1e7806db30 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e777fe640 1 -- 172.21.15.17:0/177820428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6002f050 con 0x7f1e7806db30 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e777fe640 1 -- 172.21.15.17:0/177820428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60031970 con 0x7f1e7806db30 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 -- 172.21.15.17:0/177820428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806db30 msgr2=0x7f1e780c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806db30 0x7f1e780c1e90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1e780ab7d0 tx=0x7f1e6002b7f0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 -- 172.21.15.17:0/177820428 shutdown_connections 2024-03-06T07:27:24.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806db30 0x7f1e780c1e90 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/177820428 >> v2:172.21.15.163:3300/0 conn(0x7f1e780c3a70 0x7f1e78072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/177820428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e7806d750 0x7f1e780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 -- 172.21.15.17:0/177820428 >> 172.21.15.17:0/177820428 conn(0x7f1e780c5560 msgr2=0x7f1e780c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 -- 172.21.15.17:0/177820428 shutdown_connections 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.321+0000 7f1e7ea66640 1 -- 172.21.15.17:0/177820428 wait complete. 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.322+0000 7f1e7ea66640 1 Processor -- start 2024-03-06T07:27:24.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 -- start start 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e7806db30 0x7f1e78153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e780c3a70 0x7f1e780ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780a7cd0 con 0x7f1e7806d750 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78072e10 con 0x7f1e7806db30 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7ea66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e780a7e50 con 0x7f1e780c3a70 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7c802640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7c802640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42654/0 (socket says 172.21.15.17:42654) 2024-03-06T07:27:24.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.323+0000 7f1e7c802640 1 -- 172.21.15.17:0/2128909652 learned_addr learned my addr 172.21.15.17:0/2128909652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7d003640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e780c3a70 0x7f1e780ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e77fff640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e7806db30 0x7f1e78153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 -- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e780c3a70 msgr2=0x7f1e780ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e780c3a70 0x7f1e780ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 -- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e7806db30 msgr2=0x7f1e78153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e7806db30 0x7f1e78153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 -- 172.21.15.17:0/2128909652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e781edd30 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7c802640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000ef00 tx=0x7f1e7000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018020 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7000bd30 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780ac210 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70014620 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.324+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e780ad730 con 0x7f1e7806d750 2024-03-06T07:27:24.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.325+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7804f330 con 0x7f1e7806d750 2024-03-06T07:27:24.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.326+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e70005d10 con 0x7f1e7806d750 2024-03-06T07:27:24.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.326+0000 7f1e757fa640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.326+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1e7008a670 con 0x7f1e7806d750 2024-03-06T07:27:24.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.326+0000 7f1e77fff640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.327+0000 7f1e77fff640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 0x7f1e5806d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e78154d50 tx=0x7f1e68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.330+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1e70022070 con 0x7f1e7806d750 2024-03-06T07:27:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.512+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e780ae650 con 0x7f1e7806d750 2024-03-06T07:27:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.512+0000 7f1e757fa640 1 -- 172.21.15.17:0/2128909652 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v182) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e700596b0 con 0x7f1e7806d750 2024-03-06T07:27:24.516 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:27:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 msgr2=0x7f1e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 0x7f1e5806d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e78154d50 tx=0x7f1e68008040 comp rx=0 tx=0).stop 2024-03-06T07:27:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 msgr2=0x7f1e78154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000ef00 tx=0x7f1e7000b480 comp rx=0 tx=0).stop 2024-03-06T07:27:24.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 shutdown_connections 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1e5806b4d0 0x7f1e5806d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e780c3a70 0x7f1e780ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e7806db30 0x7f1e78153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 --2- 172.21.15.17:0/2128909652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e7806d750 0x7f1e78154ec0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 >> 172.21.15.17:0/2128909652 conn(0x7f1e780c5560 msgr2=0x7f1e78072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.514+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 shutdown_connections 2024-03-06T07:27:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.515+0000 7f1e7ea66640 1 -- 172.21.15.17:0/2128909652 wait complete. 2024-03-06T07:27:24.528 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:27:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.626+0000 7f45930d6640 1 Processor -- start 2024-03-06T07:27:24.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.626+0000 7f45930d6640 1 -- start start 2024-03-06T07:27:24.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f45930d6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f458c14f250 0x7f458c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f45930d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f458c0c7350 0x7f458c0c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f45930d6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c06c150 con 0x7f458c155c20 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f45930d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c06c480 con 0x7f458c0c7350 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f45930d6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c14f630 con 0x7f458c14f250 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f458c0c7350 0x7f458c0c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f458c0c7350 0x7f458c0c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42666/0 (socket says 172.21.15.17:42666) 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 -- 172.21.15.17:0/3572641716 learned_addr learned my addr 172.21.15.17:0/3572641716 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4590e72640 1 --2- 172.21.15.17:0/3572641716 >> v2:172.21.15.163:3300/0 conn(0x7f458c14f250 0x7f458c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 -- 172.21.15.17:0/3572641716 >> v2:172.21.15.163:3300/0 conn(0x7f458c14f250 msgr2=0x7f458c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 --2- 172.21.15.17:0/3572641716 >> v2:172.21.15.163:3300/0 conn(0x7f458c14f250 0x7f458c0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 -- 172.21.15.17:0/3572641716 >> v1:172.21.15.163:6789/0 conn(0x7f458c155c20 legacy=0x7f458c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.627+0000 7f4591673640 1 -- 172.21.15.17:0/3572641716 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0c5660 con 0x7f458c0c7350 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.628+0000 7f4591673640 1 --2- 172.21.15.17:0/3572641716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c0c7350 0x7f458c0c77c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ad2d0 tx=0x7f458402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30867ab0d7d3d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.628+0000 7f458b7fe640 1 -- 172.21.15.17:0/3572641716 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458402c960 con 0x7f458c0c7350 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.628+0000 7f458b7fe640 1 -- 172.21.15.17:0/3572641716 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458402cb00 con 0x7f458c0c7350 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.628+0000 7f458b7fe640 1 -- 172.21.15.17:0/3572641716 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840343f0 con 0x7f458c0c7350 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 -- 172.21.15.17:0/3572641716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c0c7350 msgr2=0x7f458c0c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 --2- 172.21.15.17:0/3572641716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c0c7350 0x7f458c0c77c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ad2d0 tx=0x7f458402bdc0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 -- 172.21.15.17:0/3572641716 shutdown_connections 2024-03-06T07:27:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 --2- 172.21.15.17:0/3572641716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c0c7350 0x7f458c0c77c0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 --2- 172.21.15.17:0/3572641716 >> v2:172.21.15.163:3300/0 conn(0x7f458c14f250 0x7f458c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 -- 172.21.15.17:0/3572641716 >> 172.21.15.17:0/3572641716 conn(0x7f458c0ac4d0 msgr2=0x7f458c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 -- 172.21.15.17:0/3572641716 shutdown_connections 2024-03-06T07:27:24.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.629+0000 7f45930d6640 1 -- 172.21.15.17:0/3572641716 wait complete. 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.630+0000 7f45930d6640 1 Processor -- start 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.630+0000 7f45930d6640 1 -- start start 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c0c7350 0x7f458c1eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c155c20 0x7f458c1eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0c5c20 con 0x7f458c14f250 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c06daf0 con 0x7f458c0c7350 2024-03-06T07:27:24.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f45930d6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c1560a0 con 0x7f458c155c20 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42672/0 (socket says 172.21.15.17:42672) 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 -- 172.21.15.17:0/1371770538 learned_addr learned my addr 172.21.15.17:0/1371770538 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f4590e72640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c0c7350 0x7f458c1eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f4591673640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c155c20 0x7f458c1eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 -- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c155c20 msgr2=0x7f458c1eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c155c20 0x7f458c1eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 -- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c0c7350 msgr2=0x7f458c1eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c0c7350 0x7f458c1eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 -- 172.21.15.17:0/1371770538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0c1bf0 con 0x7f458c14f250 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.631+0000 7f458bfff640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f458000b3f0 tx=0x7f458000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.632+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580015020 con 0x7f458c14f250 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.632+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4580011830 con 0x7f458c14f250 2024-03-06T07:27:24.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.632+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580011c20 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.632+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c0c1ed0 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.632+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c0c21b0 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.633+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c06bda0 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.633+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4580018020 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.634+0000 7f45897fa640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 0x7f457806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.634+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4580089b80 con 0x7f458c14f250 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.634+0000 7f4590e72640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 0x7f457806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.635+0000 7f4590e72640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 0x7f457806d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.638+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f458c06bda0 con 0x7f458c14f250 2024-03-06T07:27:24.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.819+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f458c04f330 con 0x7f458c14f250 2024-03-06T07:27:24.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.819+0000 7f45897fa640 1 -- 172.21.15.17:0/1371770538 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v182) v1 ==== 84+0+65 (secure 0 0 0) 0x7f458001f030 con 0x7f458c14f250 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:27:24.822 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 msgr2=0x7f457806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 0x7f457806d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f457c006fd0 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 msgr2=0x7f458c14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f458000b3f0 tx=0x7f458000b8b0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 shutdown_connections 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f457806b4d0 0x7f457806d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f458c155c20 0x7f458c1eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f458c14f250 0x7f458c14a180 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 --2- 172.21.15.17:0/1371770538 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458c0c7350 0x7f458c1eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 >> 172.21.15.17:0/1371770538 conn(0x7f458c0ac4d0 msgr2=0x7f458c14ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 shutdown_connections 2024-03-06T07:27:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.821+0000 7f45930d6640 1 -- 172.21.15.17:0/1371770538 wait complete. 2024-03-06T07:27:24.834 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:27:24.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:27:24.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.936+0000 7f52eb6df640 1 Processor -- start 2024-03-06T07:27:24.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 -- start start 2024-03-06T07:27:24.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f52e40c88e0 0x7f52e40c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52e40c7930 0x7f52e406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e40c5400 con 0x7f52e4152870 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e406d320 con 0x7f52e40c7930 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.937+0000 7f52eb6df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e406b8d0 con 0x7f52e40c88e0 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52e40c7930 0x7f52e406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52e40c7930 0x7f52e406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42682/0 (socket says 172.21.15.17:42682) 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 -- 172.21.15.17:0/2026506464 learned_addr learned my addr 172.21.15.17:0/2026506464 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 -- 172.21.15.17:0/2026506464 >> v2:172.21.15.163:3300/0 conn(0x7f52e40c88e0 msgr2=0x7f52e40c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 --2- 172.21.15.17:0/2026506464 >> v2:172.21.15.163:3300/0 conn(0x7f52e40c88e0 0x7f52e40c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 -- 172.21.15.17:0/2026506464 >> v1:172.21.15.163:6789/0 conn(0x7f52e4152870 legacy=0x7f52e4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:24.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.938+0000 7f52e9c7c640 1 -- 172.21.15.17:0/2026506464 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40c56e0 con 0x7f52e40c7930 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52e9c7c640 1 --2- 172.21.15.17:0/2026506464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e406bd60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f52e40ad5b0 tx=0x7f52dc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594cceec1e183810 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52dbfff640 1 -- 172.21.15.17:0/2026506464 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02c9f0 con 0x7f52e40c7930 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52dbfff640 1 -- 172.21.15.17:0/2026506464 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc02cb90 con 0x7f52e40c7930 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52dbfff640 1 -- 172.21.15.17:0/2026506464 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc0343f0 con 0x7f52e40c7930 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 -- 172.21.15.17:0/2026506464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 msgr2=0x7f52e406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 --2- 172.21.15.17:0/2026506464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e406bd60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f52e40ad5b0 tx=0x7f52dc02be50 comp rx=0 tx=0).stop 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 -- 172.21.15.17:0/2026506464 shutdown_connections 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 --2- 172.21.15.17:0/2026506464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e406bd60 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 --2- 172.21.15.17:0/2026506464 >> v2:172.21.15.163:3300/0 conn(0x7f52e40c88e0 0x7f52e40c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 -- 172.21.15.17:0/2026506464 >> 172.21.15.17:0/2026506464 conn(0x7f52e40ac5f0 msgr2=0x7f52e40aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 -- 172.21.15.17:0/2026506464 shutdown_connections 2024-03-06T07:27:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.939+0000 7f52eb6df640 1 -- 172.21.15.17:0/2026506464 wait complete. 2024-03-06T07:27:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.940+0000 7f52eb6df640 1 Processor -- start 2024-03-06T07:27:24.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.940+0000 7f52eb6df640 1 -- start start 2024-03-06T07:27:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52e40c88e0 0x7f52e40c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52e4152870 0x7f52e40c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c9470 con 0x7f52e40c7930 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c5ca0 con 0x7f52e4152870 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52eb6df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e40c95f0 con 0x7f52e40c88e0 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42692/0 (socket says 172.21.15.17:42692) 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 -- 172.21.15.17:0/225452033 learned_addr learned my addr 172.21.15.17:0/225452033 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 -- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52e40c88e0 msgr2=0x7f52e40c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52e40c88e0 0x7f52e40c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 -- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52e4152870 msgr2=0x7f52e40c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52e4152870 0x7f52e40c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.941+0000 7f52e947b640 1 -- 172.21.15.17:0/225452033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4159470 con 0x7f52e40c7930 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52e947b640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c410 tx=0x7f52e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0017070 con 0x7f52e40c7930 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e00035f0 con 0x7f52e40c7930 2024-03-06T07:27:24.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e41eb4f0 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0003a20 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.942+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e40c1e10 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.943+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e404f330 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.943+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52e001a070 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.944+0000 7f52d9ffb640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.944+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f52e0089490 con 0x7f52e40c7930 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.944+0000 7f52e8c7a640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:24.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.944+0000 7f52e8c7a640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f52e414cae0 tx=0x7f52d4006bf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:24.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:24.948+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f52e0058560 con 0x7f52e40c7930 2024-03-06T07:27:25.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.127+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f52e406cf70 con 0x7f52e40c7930 2024-03-06T07:27:25.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.511+0000 7f52d9ffb640 1 -- 172.21.15.17:0/225452033 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v183) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52e00010c0 con 0x7f52e40c7930 2024-03-06T07:27:25.514 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 msgr2=0x7f52c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 0x7f52c806d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f52e414cae0 tx=0x7f52d4006bf0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 msgr2=0x7f52e4148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c410 tx=0x7f52e000c8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 shutdown_connections 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f52c806b4d0 0x7f52c806d990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52e4152870 0x7f52e40c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52e40c88e0 0x7f52e40c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 --2- 172.21.15.17:0/225452033 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52e40c7930 0x7f52e4148070 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 >> 172.21.15.17:0/225452033 conn(0x7f52e40ac5f0 msgr2=0x7f52e40ab8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.515+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 shutdown_connections 2024-03-06T07:27:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.516+0000 7f52eb6df640 1 -- 172.21.15.17:0/225452033 wait complete. 2024-03-06T07:27:25.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:27:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.639+0000 7f40c354b640 1 Processor -- start 2024-03-06T07:27:25.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.639+0000 7f40c354b640 1 -- start start 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c354b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40bc06c900 0x7f40bc0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c354b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0c3ef0 con 0x7f40bc0ab170 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c354b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc1556d0 con 0x7f40bc0c5990 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c354b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc1559b0 con 0x7f40bc06c900 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c0ae6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40bc06c900 0x7f40bc0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c1ae8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f40bc0ab170 0x7f40bc0a9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60940/0 (socket says 172.21.15.17:60940) 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c0ae6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40bc06c900 0x7f40bc0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42694/0 (socket says 172.21.15.17:42694) 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c1ae8640 1 -- 172.21.15.17:0/3628654701 learned_addr learned my addr 172.21.15.17:0/3628654701 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c0ae6640 1 -- 172.21.15.17:0/3628654701 >> v1:172.21.15.163:6789/0 conn(0x7f40bc0c5990 legacy=0x7f40bc06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c0ae6640 1 -- 172.21.15.17:0/3628654701 >> v1:172.21.15.17:6789/0 conn(0x7f40bc0ab170 legacy=0x7f40bc0a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40c0ae6640 1 -- 172.21.15.17:0/3628654701 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc155c90 con 0x7f40bc06c900 2024-03-06T07:27:25.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.640+0000 7f40b3fff640 1 -- 172.21.15.17:0/3628654701 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405491395 0 0) 0x7f40bc0c3ef0 con 0x7f40bc0ab170 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40c0ae6640 1 --2- 172.21.15.17:0/3628654701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc0c0ab0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097d0 tx=0x7f40ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0243292905aeb6d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40b3fff640 1 -- 172.21.15.17:0/3628654701 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac035020 con 0x7f40bc06c900 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40b3fff640 1 -- 172.21.15.17:0/3628654701 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac031830 con 0x7f40bc06c900 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40b3fff640 1 -- 172.21.15.17:0/3628654701 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac031c00 con 0x7f40bc06c900 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40c354b640 1 -- 172.21.15.17:0/3628654701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 msgr2=0x7f40bc0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.641+0000 7f40c354b640 1 --2- 172.21.15.17:0/3628654701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc0c0ab0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097d0 tx=0x7f40ac02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.642+0000 7f40c354b640 1 -- 172.21.15.17:0/3628654701 shutdown_connections 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.642+0000 7f40c354b640 1 --2- 172.21.15.17:0/3628654701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc0c0ab0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.642+0000 7f40c354b640 1 -- 172.21.15.17:0/3628654701 >> 172.21.15.17:0/3628654701 conn(0x7f40bc0c6710 msgr2=0x7f40bc0c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.642+0000 7f40c354b640 1 -- 172.21.15.17:0/3628654701 shutdown_connections 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.642+0000 7f40c354b640 1 -- 172.21.15.17:0/3628654701 wait complete. 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 Processor -- start 2024-03-06T07:27:25.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 -- start start 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 0x7f40bc1e9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40bc0c5990 0x7f40bc0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc156610 con 0x7f40bc06c900 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc156490 con 0x7f40bc0ab170 2024-03-06T07:27:25.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c354b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c3af0 con 0x7f40bc0c5990 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c12e7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c12e7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42702/0 (socket says 172.21.15.17:42702) 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.643+0000 7f40c12e7640 1 -- 172.21.15.17:0/4187582445 learned_addr learned my addr 172.21.15.17:0/4187582445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c0ae6640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 0x7f40bc1e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c1ae8640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40bc0c5990 0x7f40bc0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 -- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40bc0c5990 msgr2=0x7f40bc0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40bc0c5990 0x7f40bc0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 -- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 msgr2=0x7f40bc1e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 0x7f40bc1e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 -- 172.21.15.17:0/4187582445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc1ebda0 con 0x7f40bc06c900 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c0ae6640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 0x7f40bc1e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c12e7640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f40b800b3f0 tx=0x7f40b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8015020 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8011830 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc070730 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8011c20 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.644+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc070a10 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.645+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc04f330 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.646+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40b8018020 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.646+0000 7f40b1ffb640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.646+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f40b8089de0 con 0x7f40bc06c900 2024-03-06T07:27:25.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.647+0000 7f40c0ae6640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:25.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.647+0000 7f40c0ae6640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f40ac036000 tx=0x7f40ac03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:25.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.651+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40b801f030 con 0x7f40bc06c900 2024-03-06T07:27:25.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:25.832+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f40bc0ab5e0 con 0x7f40bc06c900 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.514+0000 7f40b1ffb640 1 -- 172.21.15.17:0/4187582445 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v184) v1 ==== 170+0+0 (secure 0 0 0) 0x7f40b80172c0 con 0x7f40bc06c900 2024-03-06T07:27:26.517 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 msgr2=0x7f40a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f40ac036000 tx=0x7f40ac03c040 comp rx=0 tx=0).stop 2024-03-06T07:27:26.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 msgr2=0x7f40bc1ec840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f40b800b3f0 tx=0x7f40b800b8b0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 shutdown_connections 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a006b4d0 0x7f40a006d990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40bc0c5990 0x7f40bc0c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40bc0ab170 0x7f40bc1e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 --2- 172.21.15.17:0/4187582445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40bc06c900 0x7f40bc1ec840 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 >> 172.21.15.17:0/4187582445 conn(0x7f40bc0c6710 msgr2=0x7f40bc0c25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 shutdown_connections 2024-03-06T07:27:26.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.517+0000 7f40c354b640 1 -- 172.21.15.17:0/4187582445 wait complete. 2024-03-06T07:27:26.546 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:27:26.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.653+0000 7fe4fd193640 1 Processor -- start 2024-03-06T07:27:26.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.653+0000 7fe4fd193640 1 -- start start 2024-03-06T07:27:26.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4fd193640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 0x7fe4f806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4fd193640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4f80c20b0 0x7fe4f8153180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4fd193640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f8153dd0 con 0x7fe4f80c25c0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4fd193640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f80a9530 con 0x7fe4f80c20b0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4fd193640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f80a9810 con 0x7fe4f806afc0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4f80c20b0 0x7fe4f8153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4f80c20b0 0x7fe4f8153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42704/0 (socket says 172.21.15.17:42704) 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 -- 172.21.15.17:0/389497705 learned_addr learned my addr 172.21.15.17:0/389497705 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4f6d9d640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 0x7fe4f806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 -- 172.21.15.17:0/389497705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 msgr2=0x7fe4f806b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 0x7fe4f806b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 -- 172.21.15.17:0/389497705 >> v1:172.21.15.163:6789/0 conn(0x7fe4f80c25c0 legacy=0x7fe4f81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4effff640 1 -- 172.21.15.17:0/389497705 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f80a9af0 con 0x7fe4f80c20b0 2024-03-06T07:27:26.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.654+0000 7fe4f6d9d640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 0x7fe4f806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.655+0000 7fe4effff640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c20b0 0x7fe4f8153180 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009870 tx=0x7fe4e002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d3c7e9e68696b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.655+0000 7fe4f659c640 1 -- 172.21.15.17:0/389497705 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00317c0 con 0x7fe4f80c20b0 2024-03-06T07:27:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.655+0000 7fe4f659c640 1 -- 172.21.15.17:0/389497705 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e0037070 con 0x7fe4f80c20b0 2024-03-06T07:27:26.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.655+0000 7fe4f659c640 1 -- 172.21.15.17:0/389497705 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0031b80 con 0x7fe4f80c20b0 2024-03-06T07:27:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 -- 172.21.15.17:0/389497705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c20b0 msgr2=0x7fe4f8153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c20b0 0x7fe4f8153180 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009870 tx=0x7fe4e002b860 comp rx=0 tx=0).stop 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 -- 172.21.15.17:0/389497705 shutdown_connections 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c20b0 0x7fe4f8153180 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 --2- 172.21.15.17:0/389497705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f806afc0 0x7fe4f806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 -- 172.21.15.17:0/389497705 >> 172.21.15.17:0/389497705 conn(0x7fe4f80c76b0 msgr2=0x7fe4f80c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 -- 172.21.15.17:0/389497705 shutdown_connections 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.656+0000 7fe4fd193640 1 -- 172.21.15.17:0/389497705 wait complete. 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.657+0000 7fe4fd193640 1 Processor -- start 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.657+0000 7fe4fd193640 1 -- start start 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4f806afc0 0x7fe4f8154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f80c20b0 0x7fe4f8149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8150980 con 0x7fe4f80c25c0 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f80aa0b0 con 0x7fe4f806afc0 2024-03-06T07:27:26.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f8150b00 con 0x7fe4f80c20b0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42710/0 (socket says 172.21.15.17:42710) 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 -- 172.21.15.17:0/2892916190 learned_addr learned my addr 172.21.15.17:0/2892916190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4effff640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f80c20b0 0x7fe4f8149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f6d9d640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4f806afc0 0x7fe4f8154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 -- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f80c20b0 msgr2=0x7fe4f8149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f80c20b0 0x7fe4f8149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 -- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4f806afc0 msgr2=0x7fe4f8154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4f806afc0 0x7fe4f8154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 -- 172.21.15.17:0/2892916190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f8156330 con 0x7fe4f80c25c0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4f759e640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c7a0 tx=0x7fe4f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003b70 con 0x7fe4f80c25c0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0003d30 con 0x7fe4f80c25c0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f8156610 con 0x7fe4f80c25c0 2024-03-06T07:27:26.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.658+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000aa90 con 0x7fe4f80c25c0 2024-03-06T07:27:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.659+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f8156bd0 con 0x7fe4f80c25c0 2024-03-06T07:27:26.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.659+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f8156dd0 con 0x7fe4f80c25c0 2024-03-06T07:27:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.660+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4f000ac30 con 0x7fe4f80c25c0 2024-03-06T07:27:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.660+0000 7fe4ef7fe640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 0x7fe4d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.660+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe4f008a0d0 con 0x7fe4f80c25c0 2024-03-06T07:27:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.661+0000 7fe4f6d9d640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 0x7fe4d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:26.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.661+0000 7fe4f6d9d640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 0x7fe4d806d9c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8006e80 tx=0x7fe4e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:26.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.665+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4f8156dd0 con 0x7fe4f80c25c0 2024-03-06T07:27:26.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-06T07:27:26.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:26.883+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe4f804f330 con 0x7fe4f80c25c0 2024-03-06T07:27:27.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.535+0000 7fe4ef7fe640 1 -- 172.21.15.17:0/2892916190 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v171) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe4f00587d0 con 0x7fe4f80c25c0 2024-03-06T07:27:27.538 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 29 and data pool 30 2024-03-06T07:27:27.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.545+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 msgr2=0x7fe4d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.545+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 0x7fe4d806d9c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8006e80 tx=0x7fe4e8008040 comp rx=0 tx=0).stop 2024-03-06T07:27:27.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.545+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 msgr2=0x7fe4f81499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.545+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c7a0 tx=0x7fe4f000cc60 comp rx=0 tx=0).stop 2024-03-06T07:27:27.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 shutdown_connections 2024-03-06T07:27:27.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4d806b500 0x7fe4d806d9c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4f80c25c0 0x7fe4f81499d0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4f80c20b0 0x7fe4f8149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 --2- 172.21.15.17:0/2892916190 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4f806afc0 0x7fe4f8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.546+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 >> 172.21.15.17:0/2892916190 conn(0x7fe4f80c76b0 msgr2=0x7fe4f81525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.549+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 shutdown_connections 2024-03-06T07:27:27.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.550+0000 7fe4fd193640 1 -- 172.21.15.17:0/2892916190 wait complete. 2024-03-06T07:27:27.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.674+0000 7f7ee18cf640 1 Processor -- start 2024-03-06T07:27:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.674+0000 7f7ee18cf640 1 -- start start 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ee18cf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ee18cf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7edc0c3710 0x7f7edc1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ee18cf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc158800 con 0x7f7edc0c3280 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ee18cf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc158ae0 con 0x7f7edc06d9f0 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ee18cf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc158dc0 con 0x7f7edc0c3710 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7eda7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7edc0c3280 0x7f7edc1551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60952/0 (socket says 172.21.15.17:60952) 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7eda7fc640 1 -- 172.21.15.17:0/259588519 learned_addr learned my addr 172.21.15.17:0/259588519 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:27.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 --2- 172.21.15.17:0/259588519 >> v2:172.21.15.17:3300/0 conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42712/0 (socket says 172.21.15.17:42712) 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edb7fe640 1 --2- 172.21.15.17:0/259588519 >> v2:172.21.15.163:3300/0 conn(0x7f7edc0c3710 0x7f7edc1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 -- 172.21.15.17:0/259588519 >> v2:172.21.15.163:3300/0 conn(0x7f7edc0c3710 msgr2=0x7f7edc1558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 --2- 172.21.15.17:0/259588519 >> v2:172.21.15.163:3300/0 conn(0x7f7edc0c3710 0x7f7edc1558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 -- 172.21.15.17:0/259588519 >> v1:172.21.15.17:6789/0 conn(0x7f7edc0c3280 legacy=0x7f7edc1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7ed9ffb640 1 -- 172.21.15.17:0/259588519 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791366763 0 0) 0x7f7edc158800 con 0x7f7edc0c3280 2024-03-06T07:27:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.675+0000 7f7edaffd640 1 -- 172.21.15.17:0/259588519 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc1590a0 con 0x7f7edc06d9f0 2024-03-06T07:27:27.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.676+0000 7f7edaffd640 1 --2- 172.21.15.17:0/259588519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7edc159760 tx=0x7f7ed402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d04e4eb44b90bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.676+0000 7f7ed9ffb640 1 -- 172.21.15.17:0/259588519 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40315a0 con 0x7f7edc06d9f0 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.676+0000 7f7ed9ffb640 1 -- 172.21.15.17:0/259588519 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4037070 con 0x7f7edc06d9f0 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.676+0000 7f7ed9ffb640 1 -- 172.21.15.17:0/259588519 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4031960 con 0x7f7edc06d9f0 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 -- 172.21.15.17:0/259588519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc06d9f0 msgr2=0x7f7edc0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/259588519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7edc159760 tx=0x7f7ed402b840 comp rx=0 tx=0).stop 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 -- 172.21.15.17:0/259588519 shutdown_connections 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/259588519 >> v2:172.21.15.163:3300/0 conn(0x7f7edc0c3710 0x7f7edc1558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/259588519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc06d9f0 0x7f7edc0a7cd0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 -- 172.21.15.17:0/259588519 >> 172.21.15.17:0/259588519 conn(0x7f7edc0aa630 msgr2=0x7f7edc0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 -- 172.21.15.17:0/259588519 shutdown_connections 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.677+0000 7f7ee18cf640 1 -- 172.21.15.17:0/259588519 wait complete. 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.678+0000 7f7ee18cf640 1 Processor -- start 2024-03-06T07:27:27.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.678+0000 7f7ee18cf640 1 -- start start 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc06d9f0 0x7f7edc1ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0c3280 0x7f7edc0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06ddd0 con 0x7f7edc0c3710 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc155e30 con 0x7f7edc0c3280 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ee18cf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc06df50 con 0x7f7edc06d9f0 2024-03-06T07:27:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42720/0 (socket says 172.21.15.17:42720) 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 -- 172.21.15.17:0/2203137535 learned_addr learned my addr 172.21.15.17:0/2203137535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7eda7fc640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0c3280 0x7f7edc0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edaffd640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc06d9f0 0x7f7edc1ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 -- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc06d9f0 msgr2=0x7f7edc1ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc06d9f0 0x7f7edc1ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 -- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0c3280 msgr2=0x7f7edc0ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0c3280 0x7f7edc0ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 -- 172.21.15.17:0/2203137535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc151610 con 0x7f7edc0c3710 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7edb7fe640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007d20 tx=0x7f7ecc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc014420 con 0x7f7edc0c3710 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.679+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc019070 con 0x7f7edc0c3710 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.680+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc1518f0 con 0x7f7edc0c3710 2024-03-06T07:27:27.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.680+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc014820 con 0x7f7edc0c3710 2024-03-06T07:27:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.680+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc153a50 con 0x7f7edc0c3710 2024-03-06T07:27:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.681+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc04f330 con 0x7f7edc0c3710 2024-03-06T07:27:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.681+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ecc0149c0 con 0x7f7edc0c3710 2024-03-06T07:27:27.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.681+0000 7f7ec3fff640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.682+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ecc08ab90 con 0x7f7edc0c3710 2024-03-06T07:27:27.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.682+0000 7f7edaffd640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.682+0000 7f7edaffd640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 0x7f7ec806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4036000 tx=0x7f7ed4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.686+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ecc011260 con 0x7f7edc0c3710 2024-03-06T07:27:27.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.869+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7edc151ac0 con 0x7f7edc0c3710 2024-03-06T07:27:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.871+0000 7f7ec3fff640 1 -- 172.21.15.17:0/2203137535 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f7ecc059910 con 0x7f7edc0c3710 2024-03-06T07:27:27.873 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:27.874 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":185,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:26.885878+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 msgr2=0x7f7ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 0x7f7ec806d990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4036000 tx=0x7f7ed4008030 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 msgr2=0x7f7edc153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007d20 tx=0x7f7ecc00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 shutdown_connections 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7edc0c3710 0x7f7edc153510 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0c3280 0x7f7edc0ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 --2- 172.21.15.17:0/2203137535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc06d9f0 0x7f7edc1ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 >> 172.21.15.17:0/2203137535 conn(0x7f7edc0aa630 msgr2=0x7f7edc06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 shutdown_connections 2024-03-06T07:27:27.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.873+0000 7f7ee18cf640 1 -- 172.21.15.17:0/2203137535 wait complete. 2024-03-06T07:27:27.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:27.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.985+0000 7f9ef70b5640 1 Processor -- start 2024-03-06T07:27:27.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.985+0000 7f9ef70b5640 1 -- start start 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef70b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 0x7f9ef01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef70b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef70b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef01563d0 con 0x7f9ef00c1aa0 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef70b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef00c2680 con 0x7f9ef014f790 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef70b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c28d0 con 0x7f9ef0155a40 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef4e51640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 0x7f9ef01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:36350/0 (socket says 172.21.15.17:36350) 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 -- 172.21.15.17:0/3774175816 learned_addr learned my addr 172.21.15.17:0/3774175816 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 -- 172.21.15.17:0/3774175816 >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 msgr2=0x7f9ef01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 --2- 172.21.15.17:0/3774175816 >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 0x7f9ef01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 -- 172.21.15.17:0/3774175816 >> v1:172.21.15.163:6789/0 conn(0x7f9ef00c1aa0 legacy=0x7f9ef00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef5652640 1 -- 172.21.15.17:0/3774175816 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00c2bb0 con 0x7f9ef0155a40 2024-03-06T07:27:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.986+0000 7f9ef4e51640 1 --2- 172.21.15.17:0/3774175816 >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 0x7f9ef01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.987+0000 7f9ef5652640 1 --2- 172.21.15.17:0/3774175816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800d150 tx=0x7f9ee802e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77ccd92e7486ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.987+0000 7f9eef7fe640 1 -- 172.21.15.17:0/3774175816 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800b0d0 con 0x7f9ef0155a40 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.987+0000 7f9eef7fe640 1 -- 172.21.15.17:0/3774175816 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee800b270 con 0x7f9ef0155a40 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.987+0000 7f9eef7fe640 1 -- 172.21.15.17:0/3774175816 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8009600 con 0x7f9ef0155a40 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 -- 172.21.15.17:0/3774175816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 msgr2=0x7f9ef0155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/3774175816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800d150 tx=0x7f9ee802e160 comp rx=0 tx=0).stop 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 -- 172.21.15.17:0/3774175816 shutdown_connections 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/3774175816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef0155a40 0x7f9ef0155e90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/3774175816 >> v2:172.21.15.163:3300/0 conn(0x7f9ef014f790 0x7f9ef01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 -- 172.21.15.17:0/3774175816 >> 172.21.15.17:0/3774175816 conn(0x7f9ef00c6710 msgr2=0x7f9ef00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 -- 172.21.15.17:0/3774175816 shutdown_connections 2024-03-06T07:27:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.988+0000 7f9ef70b5640 1 -- 172.21.15.17:0/3774175816 wait complete. 2024-03-06T07:27:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.989+0000 7f9ef70b5640 1 Processor -- start 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.989+0000 7f9ef70b5640 1 -- start start 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ef00c1aa0 0x7f9ef0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef014f790 0x7f9ef0148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c0d60 con 0x7f9ef0155a40 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef0155520 con 0x7f9ef00c1aa0 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef70b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef00c5e30 con 0x7f9ef014f790 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42722/0 (socket says 172.21.15.17:42722) 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 -- 172.21.15.17:0/4060693862 learned_addr learned my addr 172.21.15.17:0/4060693862 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef4e51640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ef00c1aa0 0x7f9ef0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9eeffff640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef014f790 0x7f9ef0148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 -- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef014f790 msgr2=0x7f9ef0148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef014f790 0x7f9ef0148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 -- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ef00c1aa0 msgr2=0x7f9ef0153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ef00c1aa0 0x7f9ef0153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.990+0000 7f9ef5652640 1 -- 172.21.15.17:0/4060693862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0151b60 con 0x7f9ef0155a40 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9ef5652640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8038040 tx=0x7f9ee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8045020 con 0x7f9ef0155a40 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee8009e30 con 0x7f9ef0155a40 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef01e74e0 con 0x7f9ef0155a40 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800b0d0 con 0x7f9ef0155a40 2024-03-06T07:27:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.991+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef00a8070 con 0x7f9ef0155a40 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.992+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef0151d30 con 0x7f9ef0155a40 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.992+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ee8048020 con 0x7f9ef0155a40 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.993+0000 7f9eed7fa640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.993+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ee80aed30 con 0x7f9ef0155a40 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.993+0000 7f9ef4e51640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.993+0000 7f9ef4e51640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 0x7f9ed406d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00a79d0 tx=0x7f9ee4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:27.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:27.997+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ef0151d30 con 0x7f9ef0155a40 2024-03-06T07:27:28.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.178+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ef004f330 con 0x7f9ef0155a40 2024-03-06T07:27:28.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.179+0000 7f9eed7fa640 1 -- 172.21.15.17:0/4060693862 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f9ee804f030 con 0x7f9ef0155a40 2024-03-06T07:27:28.182 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:28.182 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":185,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:26.885878+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 msgr2=0x7f9ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 0x7f9ed406d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00a79d0 tx=0x7f9ee4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 msgr2=0x7f9ef0149300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8038040 tx=0x7f9ee8004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:28.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 shutdown_connections 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ef0155a40 0x7f9ef0149300 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ef014f790 0x7f9ef0148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 --2- 172.21.15.17:0/4060693862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ef00c1aa0 0x7f9ef0153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.181+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 >> 172.21.15.17:0/4060693862 conn(0x7f9ef00c6710 msgr2=0x7f9ef014dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.182+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 shutdown_connections 2024-03-06T07:27:28.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.182+0000 7f9ef70b5640 1 -- 172.21.15.17:0/4060693862 wait complete. 2024-03-06T07:27:28.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:27:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.297+0000 7f4d959b5640 1 Processor -- start 2024-03-06T07:27:28.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.297+0000 7f4d959b5640 1 -- start start 2024-03-06T07:27:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d959b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d9014bcc0 0x7f4d90155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d959b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 0x7f4d900c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d959b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d900c75e0 con 0x7f4d900c5910 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d959b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900c7f10 con 0x7f4d9014bcc0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d959b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d900c78c0 con 0x7f4d900be150 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d9014bcc0 0x7f4d90155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4d9014bcc0 0x7f4d90155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42728/0 (socket says 172.21.15.17:42728) 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 -- 172.21.15.17:0/925479602 learned_addr learned my addr 172.21.15.17:0/925479602 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d87fff640 1 --2- 172.21.15.17:0/925479602 >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 0x7f4d900c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 -- 172.21.15.17:0/925479602 >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 msgr2=0x7f4d900c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 --2- 172.21.15.17:0/925479602 >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 0x7f4d900c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 -- 172.21.15.17:0/925479602 >> v1:172.21.15.163:6789/0 conn(0x7f4d900c5910 legacy=0x7f4d90158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d8effd640 1 -- 172.21.15.17:0/925479602 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900c7ba0 con 0x7f4d9014bcc0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.298+0000 7f4d87fff640 1 --2- 172.21.15.17:0/925479602 >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 0x7f4d900c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.299+0000 7f4d8effd640 1 --2- 172.21.15.17:0/925479602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d9014bcc0 0x7f4d90155a00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f4d900c71d0 tx=0x7f4d8002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca70ff3ed74a655b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.299+0000 7f4d8e7fc640 1 -- 172.21.15.17:0/925479602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8002f850 con 0x7f4d9014bcc0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.299+0000 7f4d8e7fc640 1 -- 172.21.15.17:0/925479602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8002f9f0 con 0x7f4d9014bcc0 2024-03-06T07:27:28.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.299+0000 7f4d8e7fc640 1 -- 172.21.15.17:0/925479602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8002fdc0 con 0x7f4d9014bcc0 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 -- 172.21.15.17:0/925479602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d9014bcc0 msgr2=0x7f4d90155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 --2- 172.21.15.17:0/925479602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d9014bcc0 0x7f4d90155a00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f4d900c71d0 tx=0x7f4d8002ba30 comp rx=0 tx=0).stop 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 -- 172.21.15.17:0/925479602 shutdown_connections 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 --2- 172.21.15.17:0/925479602 >> v2:172.21.15.163:3300/0 conn(0x7f4d900be150 0x7f4d900c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 --2- 172.21.15.17:0/925479602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d9014bcc0 0x7f4d90155a00 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 -- 172.21.15.17:0/925479602 >> 172.21.15.17:0/925479602 conn(0x7f4d900ac600 msgr2=0x7f4d900aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 -- 172.21.15.17:0/925479602 shutdown_connections 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.300+0000 7f4d959b5640 1 -- 172.21.15.17:0/925479602 wait complete. 2024-03-06T07:27:28.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 Processor -- start 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 -- start start 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d900be150 0x7f4d90154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d9014bcc0 0x7f4d900c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d901592a0 con 0x7f4d900c5910 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d90159120 con 0x7f4d9014bcc0 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d959b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d90159420 con 0x7f4d900be150 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42736/0 (socket says 172.21.15.17:42736) 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.301+0000 7f4d87fff640 1 -- 172.21.15.17:0/1116823118 learned_addr learned my addr 172.21.15.17:0/1116823118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d8effd640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d900be150 0x7f4d90154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d8f7fe640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d9014bcc0 0x7f4d900c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 -- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d900be150 msgr2=0x7f4d90154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d900be150 0x7f4d90154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 -- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d9014bcc0 msgr2=0x7f4d900c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d9014bcc0 0x7f4d900c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 -- 172.21.15.17:0/1116823118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d901eb2a0 con 0x7f4d900c5910 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d87fff640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e810 tx=0x7f4d7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7800f040 con 0x7f4d900c5910 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78004030 con 0x7f4d900c5910 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d901eb580 con 0x7f4d900c5910 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78004400 con 0x7f4d900c5910 2024-03-06T07:27:28.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.302+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d901551b0 con 0x7f4d900c5910 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.303+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9004f330 con 0x7f4d900c5910 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.304+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d7800c670 con 0x7f4d900c5910 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.304+0000 7f4d8cff9640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 0x7f4d7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.304+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d78089ec0 con 0x7f4d900c5910 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.304+0000 7f4d8effd640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 0x7f4d7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.305+0000 7f4d8effd640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 0x7f4d7006d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4d80036000 tx=0x7f4d80008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.308+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4d78017020 con 0x7f4d900c5910 2024-03-06T07:27:28.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.529+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4d900c83e0 con 0x7f4d900c5910 2024-03-06T07:27:28.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.535+0000 7f4d8cff9640 1 -- 172.21.15.17:0/1116823118 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v172) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4d780585f0 con 0x7f4d900c5910 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 msgr2=0x7f4d7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 0x7f4d7006d990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4d80036000 tx=0x7f4d80008030 comp rx=0 tx=0).stop 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 msgr2=0x7f4d900c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800e810 tx=0x7f4d7800ecd0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 shutdown_connections 2024-03-06T07:27:28.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4d7006b4d0 0x7f4d7006d990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4d9014bcc0 0x7f4d900c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d900c5910 0x7f4d900c07a0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 --2- 172.21.15.17:0/1116823118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4d900be150 0x7f4d90154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 >> 172.21.15.17:0/1116823118 conn(0x7f4d900ac600 msgr2=0x7f4d90156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 shutdown_connections 2024-03-06T07:27:28.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.537+0000 7f4d959b5640 1 -- 172.21.15.17:0/1116823118 wait complete. 2024-03-06T07:27:28.552 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:27:28.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:27:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.654+0000 7f4e5cd3e640 1 Processor -- start 2024-03-06T07:27:28.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.654+0000 7f4e5cd3e640 1 -- start start 2024-03-06T07:27:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.654+0000 7f4e5cd3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4e580b0070 0x7f4e58155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.654+0000 7f4e5cd3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e580c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5cd3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580b0450 con 0x7f4e580c5380 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5cd3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5806c020 con 0x7f4e580b0070 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5cd3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806cf20 con 0x7f4e5806ca70 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e56d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e580c5380 0x7f4e5814e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60960/0 (socket says 172.21.15.17:60960) 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e56d9d640 1 -- 172.21.15.17:0/2050725103 learned_addr learned my addr 172.21.15.17:0/2050725103 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e55d9b640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e580c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> v2:172.21.15.163:3300/0 conn(0x7f4e580b0070 0x7f4e58155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855770714 0 0) 0x7f4e50001490 con 0x7f4e580c5380 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e40003520 con 0x7f4e580c5380 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3237909188 0 0) 0x7f4e40003520 con 0x7f4e580c5380 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 msgr2=0x7f4e580c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e580c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 >> v2:172.21.15.163:3300/0 conn(0x7f4e580b0070 msgr2=0x7f4e58155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 --2- 172.21.15.17:0/2050725103 >> v2:172.21.15.163:3300/0 conn(0x7f4e580b0070 0x7f4e58155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580c77f0 con 0x7f4e580c5380 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4e50001bc0 con 0x7f4e580c5380 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 >> v1:172.21.15.17:6789/0 conn(0x7f4e580c5380 legacy=0x7f4e5814e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e400039b0 0x7f4e40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806c020 con 0x7f4e400039b0 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806cf20 con 0x7f4e40004800 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4000b9f0 con 0x7f4e5806ca70 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.655+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> v2:172.21.15.163:3300/0 conn(0x7f4e580b0070 0x7f4e58155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e55d9b640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e400039b0 0x7f4e40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e56d9d640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 -- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 msgr2=0x7f4e400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e5806ca70 0x7f4e400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 -- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e40004800 msgr2=0x7f4e40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e40004800 0x7f4e40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.656+0000 7f4e5659c640 1 -- 172.21.15.17:0/2050725103 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40003520 con 0x7f4e400039b0 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.657+0000 7f4e5659c640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e400039b0 0x7f4e40003dd0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009c70 tx=0x7f4e40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4cc296bfc6f823 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.657+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48009840 con 0x7f4e400039b0 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.657+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48004020 con 0x7f4e400039b0 2024-03-06T07:27:28.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.657+0000 7f4e5559a640 1 -- 172.21.15.17:0/2050725103 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480043d0 con 0x7f4e400039b0 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/2050725103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e400039b0 msgr2=0x7f4e40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/2050725103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e400039b0 0x7f4e40003dd0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009c70 tx=0x7f4e40005830 comp rx=0 tx=0).stop 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e56d9d640 1 -- 172.21.15.17:0/2050725103 reap_dead start 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/2050725103 shutdown_connections 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/2050725103 >> 172.21.15.17:0/2050725103 conn(0x7f4e580ac590 msgr2=0x7f4e580ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/2050725103 shutdown_connections 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.658+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/2050725103 wait complete. 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.659+0000 7f4e5cd3e640 1 Processor -- start 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.659+0000 7f4e5cd3e640 1 -- start start 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e400039b0 0x7f4e581591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 0x7f4e581eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806c550 con 0x7f4e580c2e30 2024-03-06T07:27:28.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806c270 con 0x7f4e580c4b00 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.660+0000 7f4e5cd3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5806da80 con 0x7f4e400039b0 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42762/0 (socket says 172.21.15.17:42762) 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 -- 172.21.15.17:0/3451614370 learned_addr learned my addr 172.21.15.17:0/3451614370 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e55d9b640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 0x7f4e581eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e5659c640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e400039b0 0x7f4e581591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 -- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e400039b0 msgr2=0x7f4e581591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e400039b0 0x7f4e581591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 -- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 msgr2=0x7f4e581eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 0x7f4e581eefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 -- 172.21.15.17:0/3451614370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58158450 con 0x7f4e580c2e30 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e55d9b640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 0x7f4e581eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e56d9d640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b07c0 tx=0x7f4e5000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50017020 con 0x7f4e580c2e30 2024-03-06T07:27:28.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500078c0 con 0x7f4e580c2e30 2024-03-06T07:27:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e58158730 con 0x7f4e580c2e30 2024-03-06T07:27:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500133f0 con 0x7f4e580c2e30 2024-03-06T07:27:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.661+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e58158100 con 0x7f4e580c2e30 2024-03-06T07:27:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.662+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58159700 con 0x7f4e580c2e30 2024-03-06T07:27:28.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.663+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e5001a020 con 0x7f4e580c2e30 2024-03-06T07:27:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.663+0000 7f4e46ffd640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 0x7f4e3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.663+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e5008aa60 con 0x7f4e580c2e30 2024-03-06T07:27:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.664+0000 7f4e5659c640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 0x7f4e3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:28.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.664+0000 7f4e5659c640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 0x7f4e3806d9c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4e48035000 tx=0x7f4e48030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:28.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.667+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e50021020 con 0x7f4e580c2e30 2024-03-06T07:27:28.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:28.900+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4e581554e0 con 0x7f4e580c2e30 2024-03-06T07:27:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.543+0000 7f4e46ffd640 1 -- 172.21.15.17:0/3451614370 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v173) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e580b0450 con 0x7f4e580c2e30 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 msgr2=0x7f4e3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 0x7f4e3806d9c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4e48035000 tx=0x7f4e48030000 comp rx=0 tx=0).stop 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 msgr2=0x7f4e581eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f4e580b07c0 tx=0x7f4e5000af80 comp rx=0 tx=0).stop 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 shutdown_connections 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e3806b500 0x7f4e3806d9c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e580c2e30 0x7f4e581eb040 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e580c4b00 0x7f4e581eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 --2- 172.21.15.17:0/3451614370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e400039b0 0x7f4e581591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 >> 172.21.15.17:0/3451614370 conn(0x7f4e580ac590 msgr2=0x7f4e580af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 shutdown_connections 2024-03-06T07:27:29.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.545+0000 7f4e5cd3e640 1 -- 172.21.15.17:0/3451614370 wait complete. 2024-03-06T07:27:29.558 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:27:29.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:29.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.658+0000 7fd0c5302640 1 Processor -- start 2024-03-06T07:27:29.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.658+0000 7fd0c5302640 1 -- start start 2024-03-06T07:27:29.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0c5302640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd0c01550c0 0x7fd0c01554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0c5302640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00aa660 con 0x7fd0c00aa180 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0c5302640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00aa940 con 0x7fd0c01559e0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0c5302640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00c4610 con 0x7fd0c01550c0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0beffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd0c01550c0 0x7fd0c01554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0bf7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd0c00aa180 0x7fd0c00c5060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60968/0 (socket says 172.21.15.17:60968) 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0bf7fe640 1 -- 172.21.15.17:0/1259584503 learned_addr learned my addr 172.21.15.17:0/1259584503 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0beffd640 1 -- 172.21.15.17:0/1259584503 >> v1:172.21.15.163:6789/0 conn(0x7fd0c01559e0 legacy=0x7fd0c014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0beffd640 1 -- 172.21.15.17:0/1259584503 >> v1:172.21.15.17:6789/0 conn(0x7fd0c00aa180 legacy=0x7fd0c00c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0beffd640 1 -- 172.21.15.17:0/1259584503 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00c4860 con 0x7fd0c01550c0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.659+0000 7fd0be7fc640 1 -- 172.21.15.17:0/1259584503 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190515419 0 0) 0x7fd0c00aa660 con 0x7fd0c00aa180 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.660+0000 7fd0beffd640 1 --2- 172.21.15.17:0/1259584503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c01550c0 0x7fd0c01554a0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00c4790 tx=0x7fd0b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937e18ab29150543 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.660+0000 7fd0be7fc640 1 -- 172.21.15.17:0/1259584503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b00315a0 con 0x7fd0c01550c0 2024-03-06T07:27:29.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.660+0000 7fd0be7fc640 1 -- 172.21.15.17:0/1259584503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b0037070 con 0x7fd0c01550c0 2024-03-06T07:27:29.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.660+0000 7fd0be7fc640 1 -- 172.21.15.17:0/1259584503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0031960 con 0x7fd0c01550c0 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 -- 172.21.15.17:0/1259584503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c01550c0 msgr2=0x7fd0c01554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 --2- 172.21.15.17:0/1259584503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c01550c0 0x7fd0c01554a0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00c4790 tx=0x7fd0b002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 -- 172.21.15.17:0/1259584503 shutdown_connections 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 --2- 172.21.15.17:0/1259584503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c01550c0 0x7fd0c01554a0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 -- 172.21.15.17:0/1259584503 >> 172.21.15.17:0/1259584503 conn(0x7fd0c00c6710 msgr2=0x7fd0c00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 -- 172.21.15.17:0/1259584503 shutdown_connections 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.661+0000 7fd0c5302640 1 -- 172.21.15.17:0/1259584503 wait complete. 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 Processor -- start 2024-03-06T07:27:29.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 -- start start 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c01550c0 0x7fd0c00a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c01559e0 0x7fd0c01dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c014f490 con 0x7fd0c00aa180 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c014f310 con 0x7fd0c01550c0 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.662+0000 7fd0c5302640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c01de120 con 0x7fd0c01559e0 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42784/0 (socket says 172.21.15.17:42784) 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 -- 172.21.15.17:0/4148827903 learned_addr learned my addr 172.21.15.17:0/4148827903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0b7fff640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c01550c0 0x7fd0c00a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0bf7fe640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c01559e0 0x7fd0c01dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 -- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c01559e0 msgr2=0x7fd0c01dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c01559e0 0x7fd0c01dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 -- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c01550c0 msgr2=0x7fd0c00a81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c01550c0 0x7fd0c00a81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 -- 172.21.15.17:0/4148827903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0145f30 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0beffd640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034040 tx=0x7fd0b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004230 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b00043d0 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c01de360 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0032860 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.663+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c0149d10 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.664+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c004f330 con 0x7fd0c00aa180 2024-03-06T07:27:29.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.665+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0b0032a00 con 0x7fd0c00aa180 2024-03-06T07:27:29.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.665+0000 7fd0b77fe640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 0x7fd0a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:29.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.665+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0b002f030 con 0x7fd0c00aa180 2024-03-06T07:27:29.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.665+0000 7fd0b7fff640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 0x7fd0a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:29.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.666+0000 7fd0b7fff640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 0x7fd0a006d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0155620 tx=0x7fd0a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:29.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.669+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0b0041070 con 0x7fd0c00aa180 2024-03-06T07:27:29.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.947+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c0068450 con 0x7fd0c00aa180 2024-03-06T07:27:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.947+0000 7fd0b77fe640 1 -- 172.21.15.17:0/4148827903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+4990 (secure 0 0 0) 0x7fd0b007b340 con 0x7fd0c00aa180 2024-03-06T07:27:29.950 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:29.950 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":173,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:28.902635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9900,"qdb_cluster":[9900]},"id":14}]} 2024-03-06T07:27:29.950 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 173 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 msgr2=0x7fd0a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 0x7fd0a006d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0155620 tx=0x7fd0a8009040 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 msgr2=0x7fd0c0145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034040 tx=0x7fd0b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 shutdown_connections 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0a006b4d0 0x7fd0a006d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0c01559e0 0x7fd0c01dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0c01550c0 0x7fd0c00a81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 --2- 172.21.15.17:0/4148827903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0c00aa180 0x7fd0c0145480 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 >> 172.21.15.17:0/4148827903 conn(0x7fd0c00c6710 msgr2=0x7fd0c014e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:29.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 shutdown_connections 2024-03-06T07:27:29.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:29.950+0000 7fd0c5302640 1 -- 172.21.15.17:0/4148827903 wait complete. 2024-03-06T07:27:29.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:30.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.065+0000 7f036b55c640 1 Processor -- start 2024-03-06T07:27:30.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 -- start start 2024-03-06T07:27:30.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03641506b0 0x7f0364153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0364156ea0 0x7f03640702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03640acc00 0x7f03640c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0364156450 con 0x7f03641506b0 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0364156a10 con 0x7f0364156ea0 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f036b55c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0364070800 con 0x7f03640acc00 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.066+0000 7f03692f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03641506b0 0x7f0364153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f03641506b0 0x7f0364153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42790/0 (socket says 172.21.15.17:42790) 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 -- 172.21.15.17:0/3823898082 learned_addr learned my addr 172.21.15.17:0/3823898082 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f0368af7640 1 --2- 172.21.15.17:0/3823898082 >> v2:172.21.15.163:3300/0 conn(0x7f0364156ea0 0x7f03640702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 -- 172.21.15.17:0/3823898082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03640acc00 msgr2=0x7f03640c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 --2- 172.21.15.17:0/3823898082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03640acc00 0x7f03640c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 -- 172.21.15.17:0/3823898082 >> v2:172.21.15.163:3300/0 conn(0x7f0364156ea0 msgr2=0x7f03640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 --2- 172.21.15.17:0/3823898082 >> v2:172.21.15.163:3300/0 conn(0x7f0364156ea0 0x7f03640702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.067+0000 7f03692f8640 1 -- 172.21.15.17:0/3823898082 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364070aa0 con 0x7f03641506b0 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f03692f8640 1 --2- 172.21.15.17:0/3823898082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03641506b0 0x7f0364153990 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f036406bf40 tx=0x7f036002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f14a8c40e56be6e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f035bfff640 1 -- 172.21.15.17:0/3823898082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360035020 con 0x7f03641506b0 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f035bfff640 1 -- 172.21.15.17:0/3823898082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360031870 con 0x7f03641506b0 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f035bfff640 1 -- 172.21.15.17:0/3823898082 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360031c40 con 0x7f03641506b0 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 -- 172.21.15.17:0/3823898082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03641506b0 msgr2=0x7f0364153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 --2- 172.21.15.17:0/3823898082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03641506b0 0x7f0364153990 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f036406bf40 tx=0x7f036002b910 comp rx=0 tx=0).stop 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 -- 172.21.15.17:0/3823898082 shutdown_connections 2024-03-06T07:27:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 --2- 172.21.15.17:0/3823898082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03640acc00 0x7f03640c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 --2- 172.21.15.17:0/3823898082 >> v2:172.21.15.163:3300/0 conn(0x7f0364156ea0 0x7f03640702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 --2- 172.21.15.17:0/3823898082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03641506b0 0x7f0364153990 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 -- 172.21.15.17:0/3823898082 >> 172.21.15.17:0/3823898082 conn(0x7f036406b1a0 msgr2=0x7f036406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 -- 172.21.15.17:0/3823898082 shutdown_connections 2024-03-06T07:27:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.068+0000 7f036b55c640 1 -- 172.21.15.17:0/3823898082 wait complete. 2024-03-06T07:27:30.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.069+0000 7f036b55c640 1 Processor -- start 2024-03-06T07:27:30.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.070+0000 7f036b55c640 1 -- start start 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 0x7f03640c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 0x7f0364152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640c59e0 con 0x7f03640acc00 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03641566a0 con 0x7f0364156ea0 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03640c5b60 con 0x7f03641506b0 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42794/0 (socket says 172.21.15.17:42794) 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 -- 172.21.15.17:0/2198515645 learned_addr learned my addr 172.21.15.17:0/2198515645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 -- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 msgr2=0x7f03640c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0368af7640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 0x7f03640c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0369af9640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 0x7f0364152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 0x7f03640c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 -- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 msgr2=0x7f0364152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 0x7f0364152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 -- 172.21.15.17:0/2198515645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364155e80 con 0x7f03640acc00 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0369af9640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 0x7f0364152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0368af7640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 0x7f03640c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f03692f8640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f036002b950 tx=0x7f03600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360040070 con 0x7f03640acc00 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036003a420 con 0x7f03640acc00 2024-03-06T07:27:30.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03640c4040 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360035040 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.072+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0364152970 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.073+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03640c8230 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.073+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0360038020 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.073+0000 7f0359ffb640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 0x7f034806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.074+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03600ac360 con 0x7f03640acc00 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.074+0000 7f0368af7640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 0x7f034806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.074+0000 7f0368af7640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 0x7f034806d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03640c5160 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.078+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f036007b040 con 0x7f03640acc00 2024-03-06T07:27:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.261+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f036404f330 con 0x7f03640acc00 2024-03-06T07:27:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.262+0000 7f0359ffb640 1 -- 172.21.15.17:0/2198515645 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+16884 (secure 0 0 0) 0x7f036007a940 con 0x7f03640acc00 2024-03-06T07:27:30.265 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:30.265 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":186,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:28.536361+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 msgr2=0x7f034806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 0x7f034806d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03640c5160 tx=0x7f0354008040 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 msgr2=0x7f03640c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f036002b950 tx=0x7f03600046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 shutdown_connections 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f034806b500 0x7f034806d9c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0364156ea0 0x7f0364152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03641506b0 0x7f03640c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 --2- 172.21.15.17:0/2198515645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03640acc00 0x7f03640c2940 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 >> 172.21.15.17:0/2198515645 conn(0x7f036406b1a0 msgr2=0x7f03640acfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 shutdown_connections 2024-03-06T07:27:30.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.265+0000 7f036b55c640 1 -- 172.21.15.17:0/2198515645 wait complete. 2024-03-06T07:27:30.278 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:27:30.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.377+0000 7f6fa8b1e640 1 Processor -- start 2024-03-06T07:27:30.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.378+0000 7f6fa8b1e640 1 -- start start 2024-03-06T07:27:30.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.378+0000 7f6fa8b1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fa40c5480 0x7f6fa4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.378+0000 7f6fa8b1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40c5cc0 con 0x7f6fa41586f0 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.378+0000 7f6fa8b1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa41591c0 con 0x7f6fa40c5860 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.378+0000 7f6fa8b1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa406df40 con 0x7f6fa40c5480 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6fa40c5480 0x7f6fa4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa2d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6fa41586f0 0x7f6fa415f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60970/0 (socket says 172.21.15.17:60970) 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa2d9d640 1 -- 172.21.15.17:0/278384774 learned_addr learned my addr 172.21.15.17:0/278384774 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa259c640 1 -- 172.21.15.17:0/278384774 >> v1:172.21.15.163:6789/0 conn(0x7f6fa40c5860 legacy=0x7f6fa406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa259c640 1 -- 172.21.15.17:0/278384774 >> v1:172.21.15.17:6789/0 conn(0x7f6fa41586f0 legacy=0x7f6fa415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa259c640 1 -- 172.21.15.17:0/278384774 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa41594a0 con 0x7f6fa40c5480 2024-03-06T07:27:30.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.379+0000 7f6fa159a640 1 -- 172.21.15.17:0/278384774 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109531181 0 0) 0x7f6fa40c5cc0 con 0x7f6fa41586f0 2024-03-06T07:27:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.380+0000 7f6fa259c640 1 --2- 172.21.15.17:0/278384774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5480 0x7f6fa4155ba0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6f940097a0 tx=0x7f6f9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537a7c5c4f0ca477 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.380+0000 7f6fa159a640 1 -- 172.21.15.17:0/278384774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f940315a0 con 0x7f6fa40c5480 2024-03-06T07:27:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.380+0000 7f6fa159a640 1 -- 172.21.15.17:0/278384774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f94037070 con 0x7f6fa40c5480 2024-03-06T07:27:30.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.380+0000 7f6fa159a640 1 -- 172.21.15.17:0/278384774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94031960 con 0x7f6fa40c5480 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/278384774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5480 msgr2=0x7f6fa4155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/278384774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5480 0x7f6fa4155ba0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6f940097a0 tx=0x7f6f9402b840 comp rx=0 tx=0).stop 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/278384774 shutdown_connections 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/278384774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5480 0x7f6fa4155ba0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/278384774 >> 172.21.15.17:0/278384774 conn(0x7f6fa406b1a0 msgr2=0x7f6fa406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/278384774 shutdown_connections 2024-03-06T07:27:30.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.381+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/278384774 wait complete. 2024-03-06T07:27:30.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.383+0000 7f6fa8b1e640 1 Processor -- start 2024-03-06T07:27:30.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.383+0000 7f6fa8b1e640 1 -- start start 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 0x7f6fa414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa41586f0 0x7f6fa4153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4155680 con 0x7f6fa40c5860 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa406fdb0 con 0x7f6fa40c5480 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa8b1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4152a20 con 0x7f6fa41586f0 2024-03-06T07:27:30.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42824/0 (socket says 172.21.15.17:42824) 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa1d9b640 1 -- 172.21.15.17:0/732830665 learned_addr learned my addr 172.21.15.17:0/732830665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa259c640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 0x7f6fa414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa2d9d640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa41586f0 0x7f6fa4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.384+0000 7f6fa1d9b640 1 -- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa41586f0 msgr2=0x7f6fa4153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa1d9b640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa41586f0 0x7f6fa4153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa1d9b640 1 -- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 msgr2=0x7f6fa414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa1d9b640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 0x7f6fa414dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa1d9b640 1 -- 172.21.15.17:0/732830665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa41546b0 con 0x7f6fa40c5860 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa259c640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 0x7f6fa414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa1d9b640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c430 tx=0x7f6f9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98011790 con 0x7f6fa40c5860 2024-03-06T07:27:30.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f98017070 con 0x7f6fa40c5860 2024-03-06T07:27:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41eb900 con 0x7f6fa40c5860 2024-03-06T07:27:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f98011b70 con 0x7f6fa40c5860 2024-03-06T07:27:30.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.385+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa4153110 con 0x7f6fa40c5860 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.386+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa4153310 con 0x7f6fa40c5860 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.386+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f98011d10 con 0x7f6fa40c5860 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.387+0000 7f6f92ffd640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 0x7f6f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.387+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f9808a1c0 con 0x7f6fa40c5860 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.387+0000 7f6fa259c640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 0x7f6f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.388+0000 7f6fa259c640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 0x7f6f8406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6f94036000 tx=0x7f6f9403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.394+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fa4153310 con 0x7f6fa40c5860 2024-03-06T07:27:30.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.637+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6fa404f330 con 0x7f6f8406b4d0 2024-03-06T07:27:30.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.643+0000 7f6f92ffd640 1 -- 172.21.15.17:0/732830665 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247163 (secure 0 0 0) 0x7f6fa404f330 con 0x7f6f8406b4d0 2024-03-06T07:27:30.648 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:30.650 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:27:30.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.646+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 msgr2=0x7f6f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.646+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 0x7f6f8406d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6f94036000 tx=0x7f6f9403d040 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.646+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 msgr2=0x7f6fa40c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.646+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f6f9800c430 tx=0x7f6f9800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 shutdown_connections 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f8406b4d0 0x7f6f8406d990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6fa41586f0 0x7f6fa4153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fa40c5860 0x7f6fa40c89b0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 --2- 172.21.15.17:0/732830665 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6fa40c5480 0x7f6fa414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 >> 172.21.15.17:0/732830665 conn(0x7f6fa406b1a0 msgr2=0x7f6fa4151450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 shutdown_connections 2024-03-06T07:27:30.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.647+0000 7f6fa8b1e640 1 -- 172.21.15.17:0/732830665 wait complete. 2024-03-06T07:27:30.661 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-03-06T07:27:29.441527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271944,"kb_used_data":18336,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754702776,"statfs":{"total":773094113280,"available":772815642624,"internally_reserved":0,"allocated":18776064,"data_stored":10277864,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001166"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525776+0000","last_change":"2024-03-06T07:27:26.525776+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525776+0000","last_undegraded":"2024-03-06T07:27:26.525776+0000","last_fullsized":"2024-03-06T07:27:26.525776+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527532+0000","last_change":"2024-03-06T07:27:26.527532+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527532+0000","last_undegraded":"2024-03-06T07:27:26.527532+0000","last_fullsized":"2024-03-06T07:27:26.527532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525851+0000","last_change":"2024-03-06T07:27:26.525851+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525851+0000","last_undegraded":"2024-03-06T07:27:26.525851+0000","last_fullsized":"2024-03-06T07:27:26.525851+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.531856+0000","last_change":"2024-03-06T07:27:26.531856+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.531856+0000","last_undegraded":"2024-03-06T07:27:26.531856+0000","last_fullsized":"2024-03-06T07:27:26.531856+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524917+0000","last_change":"2024-03-06T07:27:26.524917+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524917+0000","last_undegraded":"2024-03-06T07:27:26.524917+0000","last_fullsized":"2024-03-06T07:27:26.524917+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522367+0000","last_change":"2024-03-06T07:27:26.522367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522367+0000","last_undegraded":"2024-03-06T07:27:26.522367+0000","last_fullsized":"2024-03-06T07:27:26.522367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525635+0000","last_change":"2024-03-06T07:27:26.525635+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525635+0000","last_undegraded":"2024-03-06T07:27:26.525635+0000","last_fullsized":"2024-03-06T07:27:26.525635+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524185+0000","last_change":"2024-03-06T07:27:26.524185+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524185+0000","last_undegraded":"2024-03-06T07:27:26.524185+0000","last_fullsized":"2024-03-06T07:27:26.524185+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525644+0000","last_change":"2024-03-06T07:27:26.525644+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525644+0000","last_undegraded":"2024-03-06T07:27:26.525644+0000","last_fullsized":"2024-03-06T07:27:26.525644+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525171+0000","last_change":"2024-03-06T07:27:26.525171+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525171+0000","last_undegraded":"2024-03-06T07:27:26.525171+0000","last_fullsized":"2024-03-06T07:27:26.525171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.533216+0000","last_change":"2024-03-06T07:27:26.533216+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.533216+0000","last_undegraded":"2024-03-06T07:27:26.533216+0000","last_fullsized":"2024-03-06T07:27:26.533216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525045+0000","last_change":"2024-03-06T07:27:26.525045+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525045+0000","last_undegraded":"2024-03-06T07:27:26.525045+0000","last_fullsized":"2024-03-06T07:27:26.525045+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523316+0000","last_change":"2024-03-06T07:27:26.523316+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523316+0000","last_undegraded":"2024-03-06T07:27:26.523316+0000","last_fullsized":"2024-03-06T07:27:26.523316+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524993+0000","last_change":"2024-03-06T07:27:26.524993+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524993+0000","last_undegraded":"2024-03-06T07:27:26.524993+0000","last_fullsized":"2024-03-06T07:27:26.524993+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522439+0000","last_change":"2024-03-06T07:27:26.522439+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522439+0000","last_undegraded":"2024-03-06T07:27:26.522439+0000","last_fullsized":"2024-03-06T07:27:26.522439+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523802+0000","last_change":"2024-03-06T07:27:26.523802+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523802+0000","last_undegraded":"2024-03-06T07:27:26.523802+0000","last_fullsized":"2024-03-06T07:27:26.523802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524473+0000","last_change":"2024-03-06T07:27:26.524473+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524473+0000","last_undegraded":"2024-03-06T07:27:26.524473+0000","last_fullsized":"2024-03-06T07:27:26.524473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524367+0000","last_change":"2024-03-06T07:27:26.524367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524367+0000","last_undegraded":"2024-03-06T07:27:26.524367+0000","last_fullsized":"2024-03-06T07:27:26.524367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526381+0000","last_change":"2024-03-06T07:27:26.526381+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526381+0000","last_undegraded":"2024-03-06T07:27:26.526381+0000","last_fullsized":"2024-03-06T07:27:26.526381+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524159+0000","last_change":"2024-03-06T07:27:26.524159+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524159+0000","last_undegraded":"2024-03-06T07:27:26.524159+0000","last_fullsized":"2024-03-06T07:27:26.524159+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523967+0000","last_change":"2024-03-06T07:27:26.523967+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523967+0000","last_undegraded":"2024-03-06T07:27:26.523967+0000","last_fullsized":"2024-03-06T07:27:26.523967+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523234+0000","last_change":"2024-03-06T07:27:26.523234+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523234+0000","last_undegraded":"2024-03-06T07:27:26.523234+0000","last_fullsized":"2024-03-06T07:27:26.523234+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524717+0000","last_change":"2024-03-06T07:27:26.524717+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524717+0000","last_undegraded":"2024-03-06T07:27:26.524717+0000","last_fullsized":"2024-03-06T07:27:26.524717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524868+0000","last_change":"2024-03-06T07:27:26.524868+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524868+0000","last_undegraded":"2024-03-06T07:27:26.524868+0000","last_fullsized":"2024-03-06T07:27:26.524868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525341+0000","last_change":"2024-03-06T07:27:26.525341+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525341+0000","last_undegraded":"2024-03-06T07:27:26.525341+0000","last_fullsized":"2024-03-06T07:27:26.525341+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523861+0000","last_change":"2024-03-06T07:27:26.523861+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523861+0000","last_undegraded":"2024-03-06T07:27:26.523861+0000","last_fullsized":"2024-03-06T07:27:26.523861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.532806+0000","last_change":"2024-03-06T07:27:26.532806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.532806+0000","last_undegraded":"2024-03-06T07:27:26.532806+0000","last_fullsized":"2024-03-06T07:27:26.532806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523869+0000","last_change":"2024-03-06T07:27:26.523869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523869+0000","last_undegraded":"2024-03-06T07:27:26.523869+0000","last_fullsized":"2024-03-06T07:27:26.523869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527024+0000","last_change":"2024-03-06T07:27:26.527024+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527024+0000","last_undegraded":"2024-03-06T07:27:26.527024+0000","last_fullsized":"2024-03-06T07:27:26.527024+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522869+0000","last_change":"2024-03-06T07:27:26.522869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522869+0000","last_undegraded":"2024-03-06T07:27:26.522869+0000","last_fullsized":"2024-03-06T07:27:26.522869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524162+0000","last_change":"2024-03-06T07:27:26.524162+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524162+0000","last_undegraded":"2024-03-06T07:27:26.524162+0000","last_fullsized":"2024-03-06T07:27:26.524162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523410+0000","last_change":"2024-03-06T07:27:26.523410+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523410+0000","last_undegraded":"2024-03-06T07:27:26.523410+0000","last_fullsized":"2024-03-06T07:27:26.523410+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523603+0000","last_change":"2024-03-06T07:27:26.523603+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523603+0000","last_undegraded":"2024-03-06T07:27:26.523603+0000","last_fullsized":"2024-03-06T07:27:26.523603+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537037+0000","last_change":"2024-03-06T07:27:26.536435+0000","last_active":"2024-03-06T07:27:26.537037+0000","last_peered":"2024-03-06T07:27:26.537037+0000","last_clean":"2024-03-06T07:27:26.537037+0000","last_became_active":"2024-03-06T07:27:26.536143+0000","last_became_peered":"2024-03-06T07:27:26.536143+0000","last_unstale":"2024-03-06T07:27:26.537037+0000","last_undegraded":"2024-03-06T07:27:26.537037+0000","last_fullsized":"2024-03-06T07:27:26.537037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:03.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522747+0000","last_change":"2024-03-06T07:27:26.522747+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522747+0000","last_undegraded":"2024-03-06T07:27:26.522747+0000","last_fullsized":"2024-03-06T07:27:26.522747+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541050+0000","last_change":"2024-03-06T07:27:26.535890+0000","last_active":"2024-03-06T07:27:26.541050+0000","last_peered":"2024-03-06T07:27:26.541050+0000","last_clean":"2024-03-06T07:27:26.541050+0000","last_became_active":"2024-03-06T07:27:26.535629+0000","last_became_peered":"2024-03-06T07:27:26.535629+0000","last_unstale":"2024-03-06T07:27:26.541050+0000","last_undegraded":"2024-03-06T07:27:26.541050+0000","last_fullsized":"2024-03-06T07:27:26.541050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:07.272784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523859+0000","last_change":"2024-03-06T07:27:26.523859+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523859+0000","last_undegraded":"2024-03-06T07:27:26.523859+0000","last_fullsized":"2024-03-06T07:27:26.523859+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540965+0000","last_change":"2024-03-06T07:27:26.536426+0000","last_active":"2024-03-06T07:27:26.540965+0000","last_peered":"2024-03-06T07:27:26.540965+0000","last_clean":"2024-03-06T07:27:26.540965+0000","last_became_active":"2024-03-06T07:27:26.536145+0000","last_became_peered":"2024-03-06T07:27:26.536145+0000","last_unstale":"2024-03-06T07:27:26.540965+0000","last_undegraded":"2024-03-06T07:27:26.540965+0000","last_fullsized":"2024-03-06T07:27:26.540965+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:20.645286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522778+0000","last_change":"2024-03-06T07:27:26.522778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522778+0000","last_undegraded":"2024-03-06T07:27:26.522778+0000","last_fullsized":"2024-03-06T07:27:26.522778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533721+0000","last_change":"2024-03-06T07:27:26.531896+0000","last_active":"2024-03-06T07:27:26.533721+0000","last_peered":"2024-03-06T07:27:26.533721+0000","last_clean":"2024-03-06T07:27:26.533721+0000","last_became_active":"2024-03-06T07:27:26.531604+0000","last_became_peered":"2024-03-06T07:27:26.531604+0000","last_unstale":"2024-03-06T07:27:26.533721+0000","last_undegraded":"2024-03-06T07:27:26.533721+0000","last_fullsized":"2024-03-06T07:27:26.533721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:27.060493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523730+0000","last_change":"2024-03-06T07:27:26.523730+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523730+0000","last_undegraded":"2024-03-06T07:27:26.523730+0000","last_fullsized":"2024-03-06T07:27:26.523730+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541104+0000","last_change":"2024-03-06T07:27:26.537136+0000","last_active":"2024-03-06T07:27:26.541104+0000","last_peered":"2024-03-06T07:27:26.541104+0000","last_clean":"2024-03-06T07:27:26.541104+0000","last_became_active":"2024-03-06T07:27:26.536848+0000","last_became_peered":"2024-03-06T07:27:26.536848+0000","last_unstale":"2024-03-06T07:27:26.541104+0000","last_undegraded":"2024-03-06T07:27:26.541104+0000","last_fullsized":"2024-03-06T07:27:26.541104+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:05.557102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522813+0000","last_change":"2024-03-06T07:27:26.522813+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522813+0000","last_undegraded":"2024-03-06T07:27:26.522813+0000","last_fullsized":"2024-03-06T07:27:26.522813+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541500+0000","last_change":"2024-03-06T07:27:26.539023+0000","last_active":"2024-03-06T07:27:26.541500+0000","last_peered":"2024-03-06T07:27:26.541500+0000","last_clean":"2024-03-06T07:27:26.541500+0000","last_became_active":"2024-03-06T07:27:26.538724+0000","last_became_peered":"2024-03-06T07:27:26.538724+0000","last_unstale":"2024-03-06T07:27:26.541500+0000","last_undegraded":"2024-03-06T07:27:26.541500+0000","last_fullsized":"2024-03-06T07:27:26.541500+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524997+0000","last_change":"2024-03-06T07:27:26.524997+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524997+0000","last_undegraded":"2024-03-06T07:27:26.524997+0000","last_fullsized":"2024-03-06T07:27:26.524997+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.528810+0000","last_change":"2024-03-06T07:27:26.527218+0000","last_active":"2024-03-06T07:27:26.528810+0000","last_peered":"2024-03-06T07:27:26.528810+0000","last_clean":"2024-03-06T07:27:26.528810+0000","last_became_active":"2024-03-06T07:27:26.526206+0000","last_became_peered":"2024-03-06T07:27:26.526206+0000","last_unstale":"2024-03-06T07:27:26.528810+0000","last_undegraded":"2024-03-06T07:27:26.528810+0000","last_fullsized":"2024-03-06T07:27:26.528810+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:11:41.576679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522046+0000","last_change":"2024-03-06T07:27:26.522046+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522046+0000","last_undegraded":"2024-03-06T07:27:26.522046+0000","last_fullsized":"2024-03-06T07:27:26.522046+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543830+0000","last_change":"2024-03-06T07:27:26.542376+0000","last_active":"2024-03-06T07:27:26.543830+0000","last_peered":"2024-03-06T07:27:26.543830+0000","last_clean":"2024-03-06T07:27:26.543830+0000","last_became_active":"2024-03-06T07:27:26.542091+0000","last_became_peered":"2024-03-06T07:27:26.542091+0000","last_unstale":"2024-03-06T07:27:26.543830+0000","last_undegraded":"2024-03-06T07:27:26.543830+0000","last_fullsized":"2024-03-06T07:27:26.543830+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:37.673464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522321+0000","last_change":"2024-03-06T07:27:26.522321+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522321+0000","last_undegraded":"2024-03-06T07:27:26.522321+0000","last_fullsized":"2024-03-06T07:27:26.522321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541606+0000","last_change":"2024-03-06T07:27:26.539198+0000","last_active":"2024-03-06T07:27:26.541606+0000","last_peered":"2024-03-06T07:27:26.541606+0000","last_clean":"2024-03-06T07:27:26.541606+0000","last_became_active":"2024-03-06T07:27:26.538712+0000","last_became_peered":"2024-03-06T07:27:26.538712+0000","last_unstale":"2024-03-06T07:27:26.541606+0000","last_undegraded":"2024-03-06T07:27:26.541606+0000","last_fullsized":"2024-03-06T07:27:26.541606+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:39.408799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522214+0000","last_change":"2024-03-06T07:27:26.522214+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522214+0000","last_undegraded":"2024-03-06T07:27:26.522214+0000","last_fullsized":"2024-03-06T07:27:26.522214+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541542+0000","last_change":"2024-03-06T07:27:26.539250+0000","last_active":"2024-03-06T07:27:26.541542+0000","last_peered":"2024-03-06T07:27:26.541542+0000","last_clean":"2024-03-06T07:27:26.541542+0000","last_became_active":"2024-03-06T07:27:26.538970+0000","last_became_peered":"2024-03-06T07:27:26.538970+0000","last_unstale":"2024-03-06T07:27:26.541542+0000","last_undegraded":"2024-03-06T07:27:26.541542+0000","last_fullsized":"2024-03-06T07:27:26.541542+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:30.347291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.534253+0000","last_change":"2024-03-06T07:27:26.534253+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.534253+0000","last_undegraded":"2024-03-06T07:27:26.534253+0000","last_fullsized":"2024-03-06T07:27:26.534253+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540930+0000","last_change":"2024-03-06T07:27:26.537269+0000","last_active":"2024-03-06T07:27:26.540930+0000","last_peered":"2024-03-06T07:27:26.540930+0000","last_clean":"2024-03-06T07:27:26.540930+0000","last_became_active":"2024-03-06T07:27:26.536990+0000","last_became_peered":"2024-03-06T07:27:26.536990+0000","last_unstale":"2024-03-06T07:27:26.540930+0000","last_undegraded":"2024-03-06T07:27:26.540930+0000","last_fullsized":"2024-03-06T07:27:26.540930+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.779629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526845+0000","last_change":"2024-03-06T07:27:26.526845+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526845+0000","last_undegraded":"2024-03-06T07:27:26.526845+0000","last_fullsized":"2024-03-06T07:27:26.526845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536170+0000","last_change":"2024-03-06T07:27:26.533164+0000","last_active":"2024-03-06T07:27:26.536170+0000","last_peered":"2024-03-06T07:27:26.536170+0000","last_clean":"2024-03-06T07:27:26.536170+0000","last_became_active":"2024-03-06T07:27:26.532233+0000","last_became_peered":"2024-03-06T07:27:26.532233+0000","last_unstale":"2024-03-06T07:27:26.536170+0000","last_undegraded":"2024-03-06T07:27:26.536170+0000","last_fullsized":"2024-03-06T07:27:26.536170+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:25.296721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522687+0000","last_change":"2024-03-06T07:27:26.522687+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522687+0000","last_undegraded":"2024-03-06T07:27:26.522687+0000","last_fullsized":"2024-03-06T07:27:26.522687+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537685+0000","last_change":"2024-03-06T07:27:26.536746+0000","last_active":"2024-03-06T07:27:26.537685+0000","last_peered":"2024-03-06T07:27:26.537685+0000","last_clean":"2024-03-06T07:27:26.537685+0000","last_became_active":"2024-03-06T07:27:26.536443+0000","last_became_peered":"2024-03-06T07:27:26.536443+0000","last_unstale":"2024-03-06T07:27:26.537685+0000","last_undegraded":"2024-03-06T07:27:26.537685+0000","last_fullsized":"2024-03-06T07:27:26.537685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:03.817951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522240+0000","last_change":"2024-03-06T07:27:26.522240+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522240+0000","last_undegraded":"2024-03-06T07:27:26.522240+0000","last_fullsized":"2024-03-06T07:27:26.522240+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540837+0000","last_change":"2024-03-06T07:27:26.535007+0000","last_active":"2024-03-06T07:27:26.540837+0000","last_peered":"2024-03-06T07:27:26.540837+0000","last_clean":"2024-03-06T07:27:26.540837+0000","last_became_active":"2024-03-06T07:27:26.534742+0000","last_became_peered":"2024-03-06T07:27:26.534742+0000","last_unstale":"2024-03-06T07:27:26.540837+0000","last_undegraded":"2024-03-06T07:27:26.540837+0000","last_fullsized":"2024-03-06T07:27:26.540837+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:11.268111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522063+0000","last_change":"2024-03-06T07:27:26.522063+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522063+0000","last_undegraded":"2024-03-06T07:27:26.522063+0000","last_fullsized":"2024-03-06T07:27:26.522063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543721+0000","last_change":"2024-03-06T07:27:26.539852+0000","last_active":"2024-03-06T07:27:26.543721+0000","last_peered":"2024-03-06T07:27:26.543721+0000","last_clean":"2024-03-06T07:27:26.543721+0000","last_became_active":"2024-03-06T07:27:26.539585+0000","last_became_peered":"2024-03-06T07:27:26.539585+0000","last_unstale":"2024-03-06T07:27:26.543721+0000","last_undegraded":"2024-03-06T07:27:26.543721+0000","last_fullsized":"2024-03-06T07:27:26.543721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:02.711389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527806+0000","last_change":"2024-03-06T07:27:26.527806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527806+0000","last_undegraded":"2024-03-06T07:27:26.527806+0000","last_fullsized":"2024-03-06T07:27:26.527806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.530613+0000","last_change":"2024-03-06T07:27:26.529704+0000","last_active":"2024-03-06T07:27:26.530613+0000","last_peered":"2024-03-06T07:27:26.530613+0000","last_clean":"2024-03-06T07:27:26.530613+0000","last_became_active":"2024-03-06T07:27:26.529322+0000","last_became_peered":"2024-03-06T07:27:26.529322+0000","last_unstale":"2024-03-06T07:27:26.530613+0000","last_undegraded":"2024-03-06T07:27:26.530613+0000","last_fullsized":"2024-03-06T07:27:26.530613+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:08:23.114407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525947+0000","last_change":"2024-03-06T07:27:26.525947+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525947+0000","last_undegraded":"2024-03-06T07:27:26.525947+0000","last_fullsized":"2024-03-06T07:27:26.525947+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533414+0000","last_change":"2024-03-06T07:27:26.531502+0000","last_active":"2024-03-06T07:27:26.533414+0000","last_peered":"2024-03-06T07:27:26.533414+0000","last_clean":"2024-03-06T07:27:26.533414+0000","last_became_active":"2024-03-06T07:27:26.531216+0000","last_became_peered":"2024-03-06T07:27:26.531216+0000","last_unstale":"2024-03-06T07:27:26.533414+0000","last_undegraded":"2024-03-06T07:27:26.533414+0000","last_fullsized":"2024-03-06T07:27:26.533414+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:41.376525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521721+0000","last_change":"2024-03-06T07:27:26.521721+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521721+0000","last_undegraded":"2024-03-06T07:27:26.521721+0000","last_fullsized":"2024-03-06T07:27:26.521721+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546585+0000","last_change":"2024-03-06T07:27:26.541505+0000","last_active":"2024-03-06T07:27:26.546585+0000","last_peered":"2024-03-06T07:27:26.546585+0000","last_clean":"2024-03-06T07:27:26.546585+0000","last_became_active":"2024-03-06T07:27:26.541235+0000","last_became_peered":"2024-03-06T07:27:26.541235+0000","last_unstale":"2024-03-06T07:27:26.546585+0000","last_undegraded":"2024-03-06T07:27:26.546585+0000","last_fullsized":"2024-03-06T07:27:26.546585+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522464+0000","last_change":"2024-03-06T07:27:26.522464+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522464+0000","last_undegraded":"2024-03-06T07:27:26.522464+0000","last_fullsized":"2024-03-06T07:27:26.522464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543780+0000","last_change":"2024-03-06T07:27:26.540712+0000","last_active":"2024-03-06T07:27:26.543780+0000","last_peered":"2024-03-06T07:27:26.543780+0000","last_clean":"2024-03-06T07:27:26.543780+0000","last_became_active":"2024-03-06T07:27:26.540456+0000","last_became_peered":"2024-03-06T07:27:26.540456+0000","last_unstale":"2024-03-06T07:27:26.543780+0000","last_undegraded":"2024-03-06T07:27:26.543780+0000","last_fullsized":"2024-03-06T07:27:26.543780+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:23.541795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522744+0000","last_change":"2024-03-06T07:27:26.522744+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522744+0000","last_undegraded":"2024-03-06T07:27:26.522744+0000","last_fullsized":"2024-03-06T07:27:26.522744+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535226+0000","last_change":"2024-03-06T07:27:26.530763+0000","last_active":"2024-03-06T07:27:26.535226+0000","last_peered":"2024-03-06T07:27:26.535226+0000","last_clean":"2024-03-06T07:27:26.535226+0000","last_became_active":"2024-03-06T07:27:26.530485+0000","last_became_peered":"2024-03-06T07:27:26.530485+0000","last_unstale":"2024-03-06T07:27:26.535226+0000","last_undegraded":"2024-03-06T07:27:26.535226+0000","last_fullsized":"2024-03-06T07:27:26.535226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:22.926285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525008+0000","last_change":"2024-03-06T07:27:26.525008+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525008+0000","last_undegraded":"2024-03-06T07:27:26.525008+0000","last_fullsized":"2024-03-06T07:27:26.525008+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533839+0000","last_change":"2024-03-06T07:27:26.532686+0000","last_active":"2024-03-06T07:27:26.533839+0000","last_peered":"2024-03-06T07:27:26.533839+0000","last_clean":"2024-03-06T07:27:26.533839+0000","last_became_active":"2024-03-06T07:27:26.532416+0000","last_became_peered":"2024-03-06T07:27:26.532416+0000","last_unstale":"2024-03-06T07:27:26.533839+0000","last_undegraded":"2024-03-06T07:27:26.533839+0000","last_fullsized":"2024-03-06T07:27:26.533839+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:01.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.528494+0000","last_change":"2024-03-06T07:27:26.528494+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.528494+0000","last_undegraded":"2024-03-06T07:27:26.528494+0000","last_fullsized":"2024-03-06T07:27:26.528494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540337+0000","last_change":"2024-03-06T07:27:26.539907+0000","last_active":"2024-03-06T07:27:26.540337+0000","last_peered":"2024-03-06T07:27:26.540337+0000","last_clean":"2024-03-06T07:27:26.540337+0000","last_became_active":"2024-03-06T07:27:26.539575+0000","last_became_peered":"2024-03-06T07:27:26.539575+0000","last_unstale":"2024-03-06T07:27:26.540337+0000","last_undegraded":"2024-03-06T07:27:26.540337+0000","last_fullsized":"2024-03-06T07:27:26.540337+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:44:36.880373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521242+0000","last_change":"2024-03-06T07:27:26.521242+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521242+0000","last_undegraded":"2024-03-06T07:27:26.521242+0000","last_fullsized":"2024-03-06T07:27:26.521242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.532649+0000","last_change":"2024-03-06T07:27:26.531365+0000","last_active":"2024-03-06T07:27:26.532649+0000","last_peered":"2024-03-06T07:27:26.532649+0000","last_clean":"2024-03-06T07:27:26.532649+0000","last_became_active":"2024-03-06T07:27:26.531079+0000","last_became_peered":"2024-03-06T07:27:26.531079+0000","last_unstale":"2024-03-06T07:27:26.532649+0000","last_undegraded":"2024-03-06T07:27:26.532649+0000","last_fullsized":"2024-03-06T07:27:26.532649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:52.548111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527383+0000","last_change":"2024-03-06T07:27:26.527383+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527383+0000","last_undegraded":"2024-03-06T07:27:26.527383+0000","last_fullsized":"2024-03-06T07:27:26.527383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546609+0000","last_change":"2024-03-06T07:27:26.542508+0000","last_active":"2024-03-06T07:27:26.546609+0000","last_peered":"2024-03-06T07:27:26.546609+0000","last_clean":"2024-03-06T07:27:26.546609+0000","last_became_active":"2024-03-06T07:27:26.542231+0000","last_became_peered":"2024-03-06T07:27:26.542231+0000","last_unstale":"2024-03-06T07:27:26.546609+0000","last_undegraded":"2024-03-06T07:27:26.546609+0000","last_fullsized":"2024-03-06T07:27:26.546609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525432+0000","last_change":"2024-03-06T07:27:26.525432+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525432+0000","last_undegraded":"2024-03-06T07:27:26.525432+0000","last_fullsized":"2024-03-06T07:27:26.525432+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.539884+0000","last_change":"2024-03-06T07:27:26.539371+0000","last_active":"2024-03-06T07:27:26.539884+0000","last_peered":"2024-03-06T07:27:26.539884+0000","last_clean":"2024-03-06T07:27:26.539884+0000","last_became_active":"2024-03-06T07:27:26.539052+0000","last_became_peered":"2024-03-06T07:27:26.539052+0000","last_unstale":"2024-03-06T07:27:26.539884+0000","last_undegraded":"2024-03-06T07:27:26.539884+0000","last_fullsized":"2024-03-06T07:27:26.539884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:18.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527778+0000","last_change":"2024-03-06T07:27:26.527778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527778+0000","last_undegraded":"2024-03-06T07:27:26.527778+0000","last_fullsized":"2024-03-06T07:27:26.527778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533750+0000","last_change":"2024-03-06T07:27:26.532169+0000","last_active":"2024-03-06T07:27:26.533750+0000","last_peered":"2024-03-06T07:27:26.533750+0000","last_clean":"2024-03-06T07:27:26.533750+0000","last_became_active":"2024-03-06T07:27:26.531854+0000","last_became_peered":"2024-03-06T07:27:26.531854+0000","last_unstale":"2024-03-06T07:27:26.533750+0000","last_undegraded":"2024-03-06T07:27:26.533750+0000","last_fullsized":"2024-03-06T07:27:26.533750+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:34.712123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526066+0000","last_change":"2024-03-06T07:27:26.526066+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526066+0000","last_undegraded":"2024-03-06T07:27:26.526066+0000","last_fullsized":"2024-03-06T07:27:26.526066+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535765+0000","last_change":"2024-03-06T07:27:26.534771+0000","last_active":"2024-03-06T07:27:26.535765+0000","last_peered":"2024-03-06T07:27:26.535765+0000","last_clean":"2024-03-06T07:27:26.535765+0000","last_became_active":"2024-03-06T07:27:26.534079+0000","last_became_peered":"2024-03-06T07:27:26.534079+0000","last_unstale":"2024-03-06T07:27:26.535765+0000","last_undegraded":"2024-03-06T07:27:26.535765+0000","last_fullsized":"2024-03-06T07:27:26.535765+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:28.965661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521229+0000","last_change":"2024-03-06T07:27:26.521229+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521229+0000","last_undegraded":"2024-03-06T07:27:26.521229+0000","last_fullsized":"2024-03-06T07:27:26.521229+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535262+0000","last_change":"2024-03-06T07:27:26.532514+0000","last_active":"2024-03-06T07:27:26.535262+0000","last_peered":"2024-03-06T07:27:26.535262+0000","last_clean":"2024-03-06T07:27:26.535262+0000","last_became_active":"2024-03-06T07:27:26.532219+0000","last_became_peered":"2024-03-06T07:27:26.532219+0000","last_unstale":"2024-03-06T07:27:26.535262+0000","last_undegraded":"2024-03-06T07:27:26.535262+0000","last_fullsized":"2024-03-06T07:27:26.535262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:27:12.063304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521463+0000","last_change":"2024-03-06T07:27:26.521463+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521463+0000","last_undegraded":"2024-03-06T07:27:26.521463+0000","last_fullsized":"2024-03-06T07:27:26.521463+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541466+0000","last_change":"2024-03-06T07:27:26.538820+0000","last_active":"2024-03-06T07:27:26.541466+0000","last_peered":"2024-03-06T07:27:26.541466+0000","last_clean":"2024-03-06T07:27:26.541466+0000","last_became_active":"2024-03-06T07:27:26.538519+0000","last_became_peered":"2024-03-06T07:27:26.538519+0000","last_unstale":"2024-03-06T07:27:26.541466+0000","last_undegraded":"2024-03-06T07:27:26.541466+0000","last_fullsized":"2024-03-06T07:27:26.541466+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524485+0000","last_change":"2024-03-06T07:27:26.524485+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524485+0000","last_undegraded":"2024-03-06T07:27:26.524485+0000","last_fullsized":"2024-03-06T07:27:26.524485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541511+0000","last_change":"2024-03-06T07:27:26.539830+0000","last_active":"2024-03-06T07:27:26.541511+0000","last_peered":"2024-03-06T07:27:26.541511+0000","last_clean":"2024-03-06T07:27:26.541511+0000","last_became_active":"2024-03-06T07:27:26.539557+0000","last_became_peered":"2024-03-06T07:27:26.539557+0000","last_unstale":"2024-03-06T07:27:26.541511+0000","last_undegraded":"2024-03-06T07:27:26.541511+0000","last_fullsized":"2024-03-06T07:27:26.541511+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:23.412215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522827+0000","last_change":"2024-03-06T07:27:26.522827+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522827+0000","last_undegraded":"2024-03-06T07:27:26.522827+0000","last_fullsized":"2024-03-06T07:27:26.522827+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543752+0000","last_change":"2024-03-06T07:27:26.540192+0000","last_active":"2024-03-06T07:27:26.543752+0000","last_peered":"2024-03-06T07:27:26.543752+0000","last_clean":"2024-03-06T07:27:26.543752+0000","last_became_active":"2024-03-06T07:27:26.539560+0000","last_became_peered":"2024-03-06T07:27:26.539560+0000","last_unstale":"2024-03-06T07:27:26.543752+0000","last_undegraded":"2024-03-06T07:27:26.543752+0000","last_fullsized":"2024-03-06T07:27:26.543752+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:57.625904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523231+0000","last_change":"2024-03-06T07:27:26.523231+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523231+0000","last_undegraded":"2024-03-06T07:27:26.523231+0000","last_fullsized":"2024-03-06T07:27:26.523231+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536438+0000","last_change":"2024-03-06T07:27:26.534849+0000","last_active":"2024-03-06T07:27:26.536438+0000","last_peered":"2024-03-06T07:27:26.536438+0000","last_clean":"2024-03-06T07:27:26.536438+0000","last_became_active":"2024-03-06T07:27:26.534566+0000","last_became_peered":"2024-03-06T07:27:26.534566+0000","last_unstale":"2024-03-06T07:27:26.536438+0000","last_undegraded":"2024-03-06T07:27:26.536438+0000","last_fullsized":"2024-03-06T07:27:26.536438+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:46.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":472,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.526237+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:27:26.526237+0000","last_peered":"2024-03-06T07:27:26.526237+0000","last_clean":"2024-03-06T07:27:26.526237+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:27:26.526237+0000","last_undegraded":"2024-03-06T07:27:26.526237+0000","last_fullsized":"2024-03-06T07:27:26.526237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:30.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.799+0000 7f9a04aa4640 1 Processor -- start 2024-03-06T07:27:30.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.799+0000 7f9a04aa4640 1 -- start start 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a0014f790 0x7f9a001529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9a000c5c60 0x7f9a00155a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000c2680 con 0x7f9a0014f790 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000c28d0 con 0x7f9a000c5c60 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f9a04aa4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000c2bb0 con 0x7f9a000c1aa0 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a0014f790 0x7f9a001529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a0014f790 0x7f9a001529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47096/0 (socket says 172.21.15.17:47096) 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 -- 172.21.15.17:0/528545441 learned_addr learned my addr 172.21.15.17:0/528545441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fed9d640 1 --2- 172.21.15.17:0/528545441 >> v2:172.21.15.163:3300/0 conn(0x7f9a000c5c60 0x7f9a00155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fdd9b640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 -- 172.21.15.17:0/528545441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 msgr2=0x7f9a000c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 -- 172.21.15.17:0/528545441 >> v2:172.21.15.163:3300/0 conn(0x7f9a000c5c60 msgr2=0x7f9a00155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 --2- 172.21.15.17:0/528545441 >> v2:172.21.15.163:3300/0 conn(0x7f9a000c5c60 0x7f9a00155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.801+0000 7f99fe59c640 1 -- 172.21.15.17:0/528545441 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000c2e90 con 0x7f9a0014f790 2024-03-06T07:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f99fe59c640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0014f790 0x7f9a001529d0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f9a0006c960 tx=0x7f99e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f75a2ba4a46db55 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f99fd59a640 1 -- 172.21.15.17:0/528545441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8035020 con 0x7f9a0014f790 2024-03-06T07:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f99fd59a640 1 -- 172.21.15.17:0/528545441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8031870 con 0x7f9a0014f790 2024-03-06T07:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f99fd59a640 1 -- 172.21.15.17:0/528545441 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8031c40 con 0x7f9a0014f790 2024-03-06T07:27:30.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f9a04aa4640 1 -- 172.21.15.17:0/528545441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0014f790 msgr2=0x7f9a001529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.802+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0014f790 0x7f9a001529d0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f9a0006c960 tx=0x7f99e802b910 comp rx=0 tx=0).stop 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 -- 172.21.15.17:0/528545441 shutdown_connections 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/528545441 >> v2:172.21.15.163:3300/0 conn(0x7f9a000c5c60 0x7f9a00155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/528545441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0014f790 0x7f9a001529d0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 -- 172.21.15.17:0/528545441 >> 172.21.15.17:0/528545441 conn(0x7f9a000c6710 msgr2=0x7f9a000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 -- 172.21.15.17:0/528545441 shutdown_connections 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.803+0000 7f9a04aa4640 1 -- 172.21.15.17:0/528545441 wait complete. 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.804+0000 7f9a04aa4640 1 Processor -- start 2024-03-06T07:27:30.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.804+0000 7f9a04aa4640 1 -- start start 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0014f790 0x7f9a001e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0006cc00 con 0x7f9a000c5c60 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000c0d60 con 0x7f9a0014f790 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0006cd80 con 0x7f9a000c1aa0 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47108/0 (socket says 172.21.15.17:47108) 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 -- 172.21.15.17:0/1262563771 learned_addr learned my addr 172.21.15.17:0/1262563771 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fe59c640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fed9d640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0014f790 0x7f9a001e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 -- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 msgr2=0x7f9a000702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 -- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0014f790 msgr2=0x7f9a001e0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0014f790 0x7f9a001e0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 -- 172.21.15.17:0/1262563771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a001e13e0 con 0x7f9a000c5c60 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99fdd9b640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f99f000d830 tx=0x7f99f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0018020 con 0x7f9a000c5c60 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f00096a0 con 0x7f9a000c5c60 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0009a70 con 0x7f9a000c5c60 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a00073420 con 0x7f9a000c5c60 2024-03-06T07:27:30.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.805+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a00073140 con 0x7f9a000c5c60 2024-03-06T07:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.807+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0004f330 con 0x7f9a000c5c60 2024-03-06T07:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.807+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99f001b020 con 0x7f9a000c5c60 2024-03-06T07:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.807+0000 7f99f6ffd640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 0x7f99e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.807+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99f008a710 con 0x7f9a000c5c60 2024-03-06T07:27:30.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.808+0000 7f99fe59c640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 0x7f99e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:30.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.808+0000 7f99fe59c640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 0x7f99e006d9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f99e8036000 tx=0x7f99e803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:30.812+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99f0022030 con 0x7f9a000c5c60 2024-03-06T07:27:31.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.121+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a000c1f40 con 0x7f99e006b500 2024-03-06T07:27:31.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.127+0000 7f99f6ffd640 1 -- 172.21.15.17:0/1262563771 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247163 (secure 0 0 0) 0x7f9a000c1f40 con 0x7f99e006b500 2024-03-06T07:27:31.132 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:31.134 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:27:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 msgr2=0x7f99e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 0x7f99e006d9c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f99e8036000 tx=0x7f99e803c040 comp rx=0 tx=0).stop 2024-03-06T07:27:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 msgr2=0x7f9a001ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f99f000d830 tx=0x7f99f000dcf0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 shutdown_connections 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f99e006b500 0x7f99e006d9c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9a0014f790 0x7f9a001e0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a000c5c60 0x7f9a001ee080 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 --2- 172.21.15.17:0/1262563771 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9a000c1aa0 0x7f9a000702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 >> 172.21.15.17:0/1262563771 conn(0x7f9a000c6710 msgr2=0x7f9a000c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 shutdown_connections 2024-03-06T07:27:31.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.130+0000 7f9a04aa4640 1 -- 172.21.15.17:0/1262563771 wait complete. 2024-03-06T07:27:31.144 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-03-06T07:27:29.441527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271944,"kb_used_data":18336,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754702776,"statfs":{"total":773094113280,"available":772815642624,"internally_reserved":0,"allocated":18776064,"data_stored":10277864,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001166"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525776+0000","last_change":"2024-03-06T07:27:26.525776+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525776+0000","last_undegraded":"2024-03-06T07:27:26.525776+0000","last_fullsized":"2024-03-06T07:27:26.525776+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527532+0000","last_change":"2024-03-06T07:27:26.527532+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527532+0000","last_undegraded":"2024-03-06T07:27:26.527532+0000","last_fullsized":"2024-03-06T07:27:26.527532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525851+0000","last_change":"2024-03-06T07:27:26.525851+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525851+0000","last_undegraded":"2024-03-06T07:27:26.525851+0000","last_fullsized":"2024-03-06T07:27:26.525851+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.531856+0000","last_change":"2024-03-06T07:27:26.531856+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.531856+0000","last_undegraded":"2024-03-06T07:27:26.531856+0000","last_fullsized":"2024-03-06T07:27:26.531856+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524917+0000","last_change":"2024-03-06T07:27:26.524917+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524917+0000","last_undegraded":"2024-03-06T07:27:26.524917+0000","last_fullsized":"2024-03-06T07:27:26.524917+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522367+0000","last_change":"2024-03-06T07:27:26.522367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522367+0000","last_undegraded":"2024-03-06T07:27:26.522367+0000","last_fullsized":"2024-03-06T07:27:26.522367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525635+0000","last_change":"2024-03-06T07:27:26.525635+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525635+0000","last_undegraded":"2024-03-06T07:27:26.525635+0000","last_fullsized":"2024-03-06T07:27:26.525635+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524185+0000","last_change":"2024-03-06T07:27:26.524185+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524185+0000","last_undegraded":"2024-03-06T07:27:26.524185+0000","last_fullsized":"2024-03-06T07:27:26.524185+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525644+0000","last_change":"2024-03-06T07:27:26.525644+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525644+0000","last_undegraded":"2024-03-06T07:27:26.525644+0000","last_fullsized":"2024-03-06T07:27:26.525644+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525171+0000","last_change":"2024-03-06T07:27:26.525171+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525171+0000","last_undegraded":"2024-03-06T07:27:26.525171+0000","last_fullsized":"2024-03-06T07:27:26.525171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.533216+0000","last_change":"2024-03-06T07:27:26.533216+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.533216+0000","last_undegraded":"2024-03-06T07:27:26.533216+0000","last_fullsized":"2024-03-06T07:27:26.533216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525045+0000","last_change":"2024-03-06T07:27:26.525045+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525045+0000","last_undegraded":"2024-03-06T07:27:26.525045+0000","last_fullsized":"2024-03-06T07:27:26.525045+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523316+0000","last_change":"2024-03-06T07:27:26.523316+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523316+0000","last_undegraded":"2024-03-06T07:27:26.523316+0000","last_fullsized":"2024-03-06T07:27:26.523316+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524993+0000","last_change":"2024-03-06T07:27:26.524993+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524993+0000","last_undegraded":"2024-03-06T07:27:26.524993+0000","last_fullsized":"2024-03-06T07:27:26.524993+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522439+0000","last_change":"2024-03-06T07:27:26.522439+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522439+0000","last_undegraded":"2024-03-06T07:27:26.522439+0000","last_fullsized":"2024-03-06T07:27:26.522439+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523802+0000","last_change":"2024-03-06T07:27:26.523802+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523802+0000","last_undegraded":"2024-03-06T07:27:26.523802+0000","last_fullsized":"2024-03-06T07:27:26.523802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524473+0000","last_change":"2024-03-06T07:27:26.524473+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524473+0000","last_undegraded":"2024-03-06T07:27:26.524473+0000","last_fullsized":"2024-03-06T07:27:26.524473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524367+0000","last_change":"2024-03-06T07:27:26.524367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524367+0000","last_undegraded":"2024-03-06T07:27:26.524367+0000","last_fullsized":"2024-03-06T07:27:26.524367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526381+0000","last_change":"2024-03-06T07:27:26.526381+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526381+0000","last_undegraded":"2024-03-06T07:27:26.526381+0000","last_fullsized":"2024-03-06T07:27:26.526381+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524159+0000","last_change":"2024-03-06T07:27:26.524159+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524159+0000","last_undegraded":"2024-03-06T07:27:26.524159+0000","last_fullsized":"2024-03-06T07:27:26.524159+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523967+0000","last_change":"2024-03-06T07:27:26.523967+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523967+0000","last_undegraded":"2024-03-06T07:27:26.523967+0000","last_fullsized":"2024-03-06T07:27:26.523967+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523234+0000","last_change":"2024-03-06T07:27:26.523234+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523234+0000","last_undegraded":"2024-03-06T07:27:26.523234+0000","last_fullsized":"2024-03-06T07:27:26.523234+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524717+0000","last_change":"2024-03-06T07:27:26.524717+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524717+0000","last_undegraded":"2024-03-06T07:27:26.524717+0000","last_fullsized":"2024-03-06T07:27:26.524717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524868+0000","last_change":"2024-03-06T07:27:26.524868+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524868+0000","last_undegraded":"2024-03-06T07:27:26.524868+0000","last_fullsized":"2024-03-06T07:27:26.524868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525341+0000","last_change":"2024-03-06T07:27:26.525341+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525341+0000","last_undegraded":"2024-03-06T07:27:26.525341+0000","last_fullsized":"2024-03-06T07:27:26.525341+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523861+0000","last_change":"2024-03-06T07:27:26.523861+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523861+0000","last_undegraded":"2024-03-06T07:27:26.523861+0000","last_fullsized":"2024-03-06T07:27:26.523861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.532806+0000","last_change":"2024-03-06T07:27:26.532806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.532806+0000","last_undegraded":"2024-03-06T07:27:26.532806+0000","last_fullsized":"2024-03-06T07:27:26.532806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523869+0000","last_change":"2024-03-06T07:27:26.523869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523869+0000","last_undegraded":"2024-03-06T07:27:26.523869+0000","last_fullsized":"2024-03-06T07:27:26.523869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527024+0000","last_change":"2024-03-06T07:27:26.527024+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527024+0000","last_undegraded":"2024-03-06T07:27:26.527024+0000","last_fullsized":"2024-03-06T07:27:26.527024+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522869+0000","last_change":"2024-03-06T07:27:26.522869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522869+0000","last_undegraded":"2024-03-06T07:27:26.522869+0000","last_fullsized":"2024-03-06T07:27:26.522869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524162+0000","last_change":"2024-03-06T07:27:26.524162+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524162+0000","last_undegraded":"2024-03-06T07:27:26.524162+0000","last_fullsized":"2024-03-06T07:27:26.524162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523410+0000","last_change":"2024-03-06T07:27:26.523410+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523410+0000","last_undegraded":"2024-03-06T07:27:26.523410+0000","last_fullsized":"2024-03-06T07:27:26.523410+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523603+0000","last_change":"2024-03-06T07:27:26.523603+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523603+0000","last_undegraded":"2024-03-06T07:27:26.523603+0000","last_fullsized":"2024-03-06T07:27:26.523603+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537037+0000","last_change":"2024-03-06T07:27:26.536435+0000","last_active":"2024-03-06T07:27:26.537037+0000","last_peered":"2024-03-06T07:27:26.537037+0000","last_clean":"2024-03-06T07:27:26.537037+0000","last_became_active":"2024-03-06T07:27:26.536143+0000","last_became_peered":"2024-03-06T07:27:26.536143+0000","last_unstale":"2024-03-06T07:27:26.537037+0000","last_undegraded":"2024-03-06T07:27:26.537037+0000","last_fullsized":"2024-03-06T07:27:26.537037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:03.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522747+0000","last_change":"2024-03-06T07:27:26.522747+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522747+0000","last_undegraded":"2024-03-06T07:27:26.522747+0000","last_fullsized":"2024-03-06T07:27:26.522747+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541050+0000","last_change":"2024-03-06T07:27:26.535890+0000","last_active":"2024-03-06T07:27:26.541050+0000","last_peered":"2024-03-06T07:27:26.541050+0000","last_clean":"2024-03-06T07:27:26.541050+0000","last_became_active":"2024-03-06T07:27:26.535629+0000","last_became_peered":"2024-03-06T07:27:26.535629+0000","last_unstale":"2024-03-06T07:27:26.541050+0000","last_undegraded":"2024-03-06T07:27:26.541050+0000","last_fullsized":"2024-03-06T07:27:26.541050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:07.272784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523859+0000","last_change":"2024-03-06T07:27:26.523859+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523859+0000","last_undegraded":"2024-03-06T07:27:26.523859+0000","last_fullsized":"2024-03-06T07:27:26.523859+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540965+0000","last_change":"2024-03-06T07:27:26.536426+0000","last_active":"2024-03-06T07:27:26.540965+0000","last_peered":"2024-03-06T07:27:26.540965+0000","last_clean":"2024-03-06T07:27:26.540965+0000","last_became_active":"2024-03-06T07:27:26.536145+0000","last_became_peered":"2024-03-06T07:27:26.536145+0000","last_unstale":"2024-03-06T07:27:26.540965+0000","last_undegraded":"2024-03-06T07:27:26.540965+0000","last_fullsized":"2024-03-06T07:27:26.540965+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:20.645286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522778+0000","last_change":"2024-03-06T07:27:26.522778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522778+0000","last_undegraded":"2024-03-06T07:27:26.522778+0000","last_fullsized":"2024-03-06T07:27:26.522778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533721+0000","last_change":"2024-03-06T07:27:26.531896+0000","last_active":"2024-03-06T07:27:26.533721+0000","last_peered":"2024-03-06T07:27:26.533721+0000","last_clean":"2024-03-06T07:27:26.533721+0000","last_became_active":"2024-03-06T07:27:26.531604+0000","last_became_peered":"2024-03-06T07:27:26.531604+0000","last_unstale":"2024-03-06T07:27:26.533721+0000","last_undegraded":"2024-03-06T07:27:26.533721+0000","last_fullsized":"2024-03-06T07:27:26.533721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:27.060493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523730+0000","last_change":"2024-03-06T07:27:26.523730+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523730+0000","last_undegraded":"2024-03-06T07:27:26.523730+0000","last_fullsized":"2024-03-06T07:27:26.523730+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541104+0000","last_change":"2024-03-06T07:27:26.537136+0000","last_active":"2024-03-06T07:27:26.541104+0000","last_peered":"2024-03-06T07:27:26.541104+0000","last_clean":"2024-03-06T07:27:26.541104+0000","last_became_active":"2024-03-06T07:27:26.536848+0000","last_became_peered":"2024-03-06T07:27:26.536848+0000","last_unstale":"2024-03-06T07:27:26.541104+0000","last_undegraded":"2024-03-06T07:27:26.541104+0000","last_fullsized":"2024-03-06T07:27:26.541104+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:05.557102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522813+0000","last_change":"2024-03-06T07:27:26.522813+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522813+0000","last_undegraded":"2024-03-06T07:27:26.522813+0000","last_fullsized":"2024-03-06T07:27:26.522813+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541500+0000","last_change":"2024-03-06T07:27:26.539023+0000","last_active":"2024-03-06T07:27:26.541500+0000","last_peered":"2024-03-06T07:27:26.541500+0000","last_clean":"2024-03-06T07:27:26.541500+0000","last_became_active":"2024-03-06T07:27:26.538724+0000","last_became_peered":"2024-03-06T07:27:26.538724+0000","last_unstale":"2024-03-06T07:27:26.541500+0000","last_undegraded":"2024-03-06T07:27:26.541500+0000","last_fullsized":"2024-03-06T07:27:26.541500+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524997+0000","last_change":"2024-03-06T07:27:26.524997+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524997+0000","last_undegraded":"2024-03-06T07:27:26.524997+0000","last_fullsized":"2024-03-06T07:27:26.524997+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.528810+0000","last_change":"2024-03-06T07:27:26.527218+0000","last_active":"2024-03-06T07:27:26.528810+0000","last_peered":"2024-03-06T07:27:26.528810+0000","last_clean":"2024-03-06T07:27:26.528810+0000","last_became_active":"2024-03-06T07:27:26.526206+0000","last_became_peered":"2024-03-06T07:27:26.526206+0000","last_unstale":"2024-03-06T07:27:26.528810+0000","last_undegraded":"2024-03-06T07:27:26.528810+0000","last_fullsized":"2024-03-06T07:27:26.528810+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:11:41.576679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522046+0000","last_change":"2024-03-06T07:27:26.522046+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522046+0000","last_undegraded":"2024-03-06T07:27:26.522046+0000","last_fullsized":"2024-03-06T07:27:26.522046+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543830+0000","last_change":"2024-03-06T07:27:26.542376+0000","last_active":"2024-03-06T07:27:26.543830+0000","last_peered":"2024-03-06T07:27:26.543830+0000","last_clean":"2024-03-06T07:27:26.543830+0000","last_became_active":"2024-03-06T07:27:26.542091+0000","last_became_peered":"2024-03-06T07:27:26.542091+0000","last_unstale":"2024-03-06T07:27:26.543830+0000","last_undegraded":"2024-03-06T07:27:26.543830+0000","last_fullsized":"2024-03-06T07:27:26.543830+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:37.673464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522321+0000","last_change":"2024-03-06T07:27:26.522321+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522321+0000","last_undegraded":"2024-03-06T07:27:26.522321+0000","last_fullsized":"2024-03-06T07:27:26.522321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541606+0000","last_change":"2024-03-06T07:27:26.539198+0000","last_active":"2024-03-06T07:27:26.541606+0000","last_peered":"2024-03-06T07:27:26.541606+0000","last_clean":"2024-03-06T07:27:26.541606+0000","last_became_active":"2024-03-06T07:27:26.538712+0000","last_became_peered":"2024-03-06T07:27:26.538712+0000","last_unstale":"2024-03-06T07:27:26.541606+0000","last_undegraded":"2024-03-06T07:27:26.541606+0000","last_fullsized":"2024-03-06T07:27:26.541606+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:39.408799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522214+0000","last_change":"2024-03-06T07:27:26.522214+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522214+0000","last_undegraded":"2024-03-06T07:27:26.522214+0000","last_fullsized":"2024-03-06T07:27:26.522214+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541542+0000","last_change":"2024-03-06T07:27:26.539250+0000","last_active":"2024-03-06T07:27:26.541542+0000","last_peered":"2024-03-06T07:27:26.541542+0000","last_clean":"2024-03-06T07:27:26.541542+0000","last_became_active":"2024-03-06T07:27:26.538970+0000","last_became_peered":"2024-03-06T07:27:26.538970+0000","last_unstale":"2024-03-06T07:27:26.541542+0000","last_undegraded":"2024-03-06T07:27:26.541542+0000","last_fullsized":"2024-03-06T07:27:26.541542+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:30.347291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.534253+0000","last_change":"2024-03-06T07:27:26.534253+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.534253+0000","last_undegraded":"2024-03-06T07:27:26.534253+0000","last_fullsized":"2024-03-06T07:27:26.534253+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540930+0000","last_change":"2024-03-06T07:27:26.537269+0000","last_active":"2024-03-06T07:27:26.540930+0000","last_peered":"2024-03-06T07:27:26.540930+0000","last_clean":"2024-03-06T07:27:26.540930+0000","last_became_active":"2024-03-06T07:27:26.536990+0000","last_became_peered":"2024-03-06T07:27:26.536990+0000","last_unstale":"2024-03-06T07:27:26.540930+0000","last_undegraded":"2024-03-06T07:27:26.540930+0000","last_fullsized":"2024-03-06T07:27:26.540930+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.779629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526845+0000","last_change":"2024-03-06T07:27:26.526845+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526845+0000","last_undegraded":"2024-03-06T07:27:26.526845+0000","last_fullsized":"2024-03-06T07:27:26.526845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536170+0000","last_change":"2024-03-06T07:27:26.533164+0000","last_active":"2024-03-06T07:27:26.536170+0000","last_peered":"2024-03-06T07:27:26.536170+0000","last_clean":"2024-03-06T07:27:26.536170+0000","last_became_active":"2024-03-06T07:27:26.532233+0000","last_became_peered":"2024-03-06T07:27:26.532233+0000","last_unstale":"2024-03-06T07:27:26.536170+0000","last_undegraded":"2024-03-06T07:27:26.536170+0000","last_fullsized":"2024-03-06T07:27:26.536170+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:25.296721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522687+0000","last_change":"2024-03-06T07:27:26.522687+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522687+0000","last_undegraded":"2024-03-06T07:27:26.522687+0000","last_fullsized":"2024-03-06T07:27:26.522687+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537685+0000","last_change":"2024-03-06T07:27:26.536746+0000","last_active":"2024-03-06T07:27:26.537685+0000","last_peered":"2024-03-06T07:27:26.537685+0000","last_clean":"2024-03-06T07:27:26.537685+0000","last_became_active":"2024-03-06T07:27:26.536443+0000","last_became_peered":"2024-03-06T07:27:26.536443+0000","last_unstale":"2024-03-06T07:27:26.537685+0000","last_undegraded":"2024-03-06T07:27:26.537685+0000","last_fullsized":"2024-03-06T07:27:26.537685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:03.817951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522240+0000","last_change":"2024-03-06T07:27:26.522240+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522240+0000","last_undegraded":"2024-03-06T07:27:26.522240+0000","last_fullsized":"2024-03-06T07:27:26.522240+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540837+0000","last_change":"2024-03-06T07:27:26.535007+0000","last_active":"2024-03-06T07:27:26.540837+0000","last_peered":"2024-03-06T07:27:26.540837+0000","last_clean":"2024-03-06T07:27:26.540837+0000","last_became_active":"2024-03-06T07:27:26.534742+0000","last_became_peered":"2024-03-06T07:27:26.534742+0000","last_unstale":"2024-03-06T07:27:26.540837+0000","last_undegraded":"2024-03-06T07:27:26.540837+0000","last_fullsized":"2024-03-06T07:27:26.540837+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:11.268111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522063+0000","last_change":"2024-03-06T07:27:26.522063+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522063+0000","last_undegraded":"2024-03-06T07:27:26.522063+0000","last_fullsized":"2024-03-06T07:27:26.522063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543721+0000","last_change":"2024-03-06T07:27:26.539852+0000","last_active":"2024-03-06T07:27:26.543721+0000","last_peered":"2024-03-06T07:27:26.543721+0000","last_clean":"2024-03-06T07:27:26.543721+0000","last_became_active":"2024-03-06T07:27:26.539585+0000","last_became_peered":"2024-03-06T07:27:26.539585+0000","last_unstale":"2024-03-06T07:27:26.543721+0000","last_undegraded":"2024-03-06T07:27:26.543721+0000","last_fullsized":"2024-03-06T07:27:26.543721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:02.711389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527806+0000","last_change":"2024-03-06T07:27:26.527806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527806+0000","last_undegraded":"2024-03-06T07:27:26.527806+0000","last_fullsized":"2024-03-06T07:27:26.527806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.530613+0000","last_change":"2024-03-06T07:27:26.529704+0000","last_active":"2024-03-06T07:27:26.530613+0000","last_peered":"2024-03-06T07:27:26.530613+0000","last_clean":"2024-03-06T07:27:26.530613+0000","last_became_active":"2024-03-06T07:27:26.529322+0000","last_became_peered":"2024-03-06T07:27:26.529322+0000","last_unstale":"2024-03-06T07:27:26.530613+0000","last_undegraded":"2024-03-06T07:27:26.530613+0000","last_fullsized":"2024-03-06T07:27:26.530613+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:08:23.114407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525947+0000","last_change":"2024-03-06T07:27:26.525947+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525947+0000","last_undegraded":"2024-03-06T07:27:26.525947+0000","last_fullsized":"2024-03-06T07:27:26.525947+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533414+0000","last_change":"2024-03-06T07:27:26.531502+0000","last_active":"2024-03-06T07:27:26.533414+0000","last_peered":"2024-03-06T07:27:26.533414+0000","last_clean":"2024-03-06T07:27:26.533414+0000","last_became_active":"2024-03-06T07:27:26.531216+0000","last_became_peered":"2024-03-06T07:27:26.531216+0000","last_unstale":"2024-03-06T07:27:26.533414+0000","last_undegraded":"2024-03-06T07:27:26.533414+0000","last_fullsized":"2024-03-06T07:27:26.533414+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:41.376525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521721+0000","last_change":"2024-03-06T07:27:26.521721+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521721+0000","last_undegraded":"2024-03-06T07:27:26.521721+0000","last_fullsized":"2024-03-06T07:27:26.521721+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546585+0000","last_change":"2024-03-06T07:27:26.541505+0000","last_active":"2024-03-06T07:27:26.546585+0000","last_peered":"2024-03-06T07:27:26.546585+0000","last_clean":"2024-03-06T07:27:26.546585+0000","last_became_active":"2024-03-06T07:27:26.541235+0000","last_became_peered":"2024-03-06T07:27:26.541235+0000","last_unstale":"2024-03-06T07:27:26.546585+0000","last_undegraded":"2024-03-06T07:27:26.546585+0000","last_fullsized":"2024-03-06T07:27:26.546585+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522464+0000","last_change":"2024-03-06T07:27:26.522464+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522464+0000","last_undegraded":"2024-03-06T07:27:26.522464+0000","last_fullsized":"2024-03-06T07:27:26.522464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543780+0000","last_change":"2024-03-06T07:27:26.540712+0000","last_active":"2024-03-06T07:27:26.543780+0000","last_peered":"2024-03-06T07:27:26.543780+0000","last_clean":"2024-03-06T07:27:26.543780+0000","last_became_active":"2024-03-06T07:27:26.540456+0000","last_became_peered":"2024-03-06T07:27:26.540456+0000","last_unstale":"2024-03-06T07:27:26.543780+0000","last_undegraded":"2024-03-06T07:27:26.543780+0000","last_fullsized":"2024-03-06T07:27:26.543780+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:23.541795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522744+0000","last_change":"2024-03-06T07:27:26.522744+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522744+0000","last_undegraded":"2024-03-06T07:27:26.522744+0000","last_fullsized":"2024-03-06T07:27:26.522744+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535226+0000","last_change":"2024-03-06T07:27:26.530763+0000","last_active":"2024-03-06T07:27:26.535226+0000","last_peered":"2024-03-06T07:27:26.535226+0000","last_clean":"2024-03-06T07:27:26.535226+0000","last_became_active":"2024-03-06T07:27:26.530485+0000","last_became_peered":"2024-03-06T07:27:26.530485+0000","last_unstale":"2024-03-06T07:27:26.535226+0000","last_undegraded":"2024-03-06T07:27:26.535226+0000","last_fullsized":"2024-03-06T07:27:26.535226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:22.926285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525008+0000","last_change":"2024-03-06T07:27:26.525008+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525008+0000","last_undegraded":"2024-03-06T07:27:26.525008+0000","last_fullsized":"2024-03-06T07:27:26.525008+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533839+0000","last_change":"2024-03-06T07:27:26.532686+0000","last_active":"2024-03-06T07:27:26.533839+0000","last_peered":"2024-03-06T07:27:26.533839+0000","last_clean":"2024-03-06T07:27:26.533839+0000","last_became_active":"2024-03-06T07:27:26.532416+0000","last_became_peered":"2024-03-06T07:27:26.532416+0000","last_unstale":"2024-03-06T07:27:26.533839+0000","last_undegraded":"2024-03-06T07:27:26.533839+0000","last_fullsized":"2024-03-06T07:27:26.533839+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:01.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.528494+0000","last_change":"2024-03-06T07:27:26.528494+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.528494+0000","last_undegraded":"2024-03-06T07:27:26.528494+0000","last_fullsized":"2024-03-06T07:27:26.528494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540337+0000","last_change":"2024-03-06T07:27:26.539907+0000","last_active":"2024-03-06T07:27:26.540337+0000","last_peered":"2024-03-06T07:27:26.540337+0000","last_clean":"2024-03-06T07:27:26.540337+0000","last_became_active":"2024-03-06T07:27:26.539575+0000","last_became_peered":"2024-03-06T07:27:26.539575+0000","last_unstale":"2024-03-06T07:27:26.540337+0000","last_undegraded":"2024-03-06T07:27:26.540337+0000","last_fullsized":"2024-03-06T07:27:26.540337+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:44:36.880373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521242+0000","last_change":"2024-03-06T07:27:26.521242+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521242+0000","last_undegraded":"2024-03-06T07:27:26.521242+0000","last_fullsized":"2024-03-06T07:27:26.521242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.532649+0000","last_change":"2024-03-06T07:27:26.531365+0000","last_active":"2024-03-06T07:27:26.532649+0000","last_peered":"2024-03-06T07:27:26.532649+0000","last_clean":"2024-03-06T07:27:26.532649+0000","last_became_active":"2024-03-06T07:27:26.531079+0000","last_became_peered":"2024-03-06T07:27:26.531079+0000","last_unstale":"2024-03-06T07:27:26.532649+0000","last_undegraded":"2024-03-06T07:27:26.532649+0000","last_fullsized":"2024-03-06T07:27:26.532649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:52.548111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527383+0000","last_change":"2024-03-06T07:27:26.527383+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527383+0000","last_undegraded":"2024-03-06T07:27:26.527383+0000","last_fullsized":"2024-03-06T07:27:26.527383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546609+0000","last_change":"2024-03-06T07:27:26.542508+0000","last_active":"2024-03-06T07:27:26.546609+0000","last_peered":"2024-03-06T07:27:26.546609+0000","last_clean":"2024-03-06T07:27:26.546609+0000","last_became_active":"2024-03-06T07:27:26.542231+0000","last_became_peered":"2024-03-06T07:27:26.542231+0000","last_unstale":"2024-03-06T07:27:26.546609+0000","last_undegraded":"2024-03-06T07:27:26.546609+0000","last_fullsized":"2024-03-06T07:27:26.546609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525432+0000","last_change":"2024-03-06T07:27:26.525432+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525432+0000","last_undegraded":"2024-03-06T07:27:26.525432+0000","last_fullsized":"2024-03-06T07:27:26.525432+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.539884+0000","last_change":"2024-03-06T07:27:26.539371+0000","last_active":"2024-03-06T07:27:26.539884+0000","last_peered":"2024-03-06T07:27:26.539884+0000","last_clean":"2024-03-06T07:27:26.539884+0000","last_became_active":"2024-03-06T07:27:26.539052+0000","last_became_peered":"2024-03-06T07:27:26.539052+0000","last_unstale":"2024-03-06T07:27:26.539884+0000","last_undegraded":"2024-03-06T07:27:26.539884+0000","last_fullsized":"2024-03-06T07:27:26.539884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:18.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527778+0000","last_change":"2024-03-06T07:27:26.527778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527778+0000","last_undegraded":"2024-03-06T07:27:26.527778+0000","last_fullsized":"2024-03-06T07:27:26.527778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533750+0000","last_change":"2024-03-06T07:27:26.532169+0000","last_active":"2024-03-06T07:27:26.533750+0000","last_peered":"2024-03-06T07:27:26.533750+0000","last_clean":"2024-03-06T07:27:26.533750+0000","last_became_active":"2024-03-06T07:27:26.531854+0000","last_became_peered":"2024-03-06T07:27:26.531854+0000","last_unstale":"2024-03-06T07:27:26.533750+0000","last_undegraded":"2024-03-06T07:27:26.533750+0000","last_fullsized":"2024-03-06T07:27:26.533750+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:34.712123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526066+0000","last_change":"2024-03-06T07:27:26.526066+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526066+0000","last_undegraded":"2024-03-06T07:27:26.526066+0000","last_fullsized":"2024-03-06T07:27:26.526066+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535765+0000","last_change":"2024-03-06T07:27:26.534771+0000","last_active":"2024-03-06T07:27:26.535765+0000","last_peered":"2024-03-06T07:27:26.535765+0000","last_clean":"2024-03-06T07:27:26.535765+0000","last_became_active":"2024-03-06T07:27:26.534079+0000","last_became_peered":"2024-03-06T07:27:26.534079+0000","last_unstale":"2024-03-06T07:27:26.535765+0000","last_undegraded":"2024-03-06T07:27:26.535765+0000","last_fullsized":"2024-03-06T07:27:26.535765+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:28.965661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521229+0000","last_change":"2024-03-06T07:27:26.521229+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521229+0000","last_undegraded":"2024-03-06T07:27:26.521229+0000","last_fullsized":"2024-03-06T07:27:26.521229+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535262+0000","last_change":"2024-03-06T07:27:26.532514+0000","last_active":"2024-03-06T07:27:26.535262+0000","last_peered":"2024-03-06T07:27:26.535262+0000","last_clean":"2024-03-06T07:27:26.535262+0000","last_became_active":"2024-03-06T07:27:26.532219+0000","last_became_peered":"2024-03-06T07:27:26.532219+0000","last_unstale":"2024-03-06T07:27:26.535262+0000","last_undegraded":"2024-03-06T07:27:26.535262+0000","last_fullsized":"2024-03-06T07:27:26.535262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:27:12.063304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521463+0000","last_change":"2024-03-06T07:27:26.521463+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521463+0000","last_undegraded":"2024-03-06T07:27:26.521463+0000","last_fullsized":"2024-03-06T07:27:26.521463+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541466+0000","last_change":"2024-03-06T07:27:26.538820+0000","last_active":"2024-03-06T07:27:26.541466+0000","last_peered":"2024-03-06T07:27:26.541466+0000","last_clean":"2024-03-06T07:27:26.541466+0000","last_became_active":"2024-03-06T07:27:26.538519+0000","last_became_peered":"2024-03-06T07:27:26.538519+0000","last_unstale":"2024-03-06T07:27:26.541466+0000","last_undegraded":"2024-03-06T07:27:26.541466+0000","last_fullsized":"2024-03-06T07:27:26.541466+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524485+0000","last_change":"2024-03-06T07:27:26.524485+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524485+0000","last_undegraded":"2024-03-06T07:27:26.524485+0000","last_fullsized":"2024-03-06T07:27:26.524485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541511+0000","last_change":"2024-03-06T07:27:26.539830+0000","last_active":"2024-03-06T07:27:26.541511+0000","last_peered":"2024-03-06T07:27:26.541511+0000","last_clean":"2024-03-06T07:27:26.541511+0000","last_became_active":"2024-03-06T07:27:26.539557+0000","last_became_peered":"2024-03-06T07:27:26.539557+0000","last_unstale":"2024-03-06T07:27:26.541511+0000","last_undegraded":"2024-03-06T07:27:26.541511+0000","last_fullsized":"2024-03-06T07:27:26.541511+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:23.412215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522827+0000","last_change":"2024-03-06T07:27:26.522827+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522827+0000","last_undegraded":"2024-03-06T07:27:26.522827+0000","last_fullsized":"2024-03-06T07:27:26.522827+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543752+0000","last_change":"2024-03-06T07:27:26.540192+0000","last_active":"2024-03-06T07:27:26.543752+0000","last_peered":"2024-03-06T07:27:26.543752+0000","last_clean":"2024-03-06T07:27:26.543752+0000","last_became_active":"2024-03-06T07:27:26.539560+0000","last_became_peered":"2024-03-06T07:27:26.539560+0000","last_unstale":"2024-03-06T07:27:26.543752+0000","last_undegraded":"2024-03-06T07:27:26.543752+0000","last_fullsized":"2024-03-06T07:27:26.543752+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:57.625904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523231+0000","last_change":"2024-03-06T07:27:26.523231+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523231+0000","last_undegraded":"2024-03-06T07:27:26.523231+0000","last_fullsized":"2024-03-06T07:27:26.523231+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536438+0000","last_change":"2024-03-06T07:27:26.534849+0000","last_active":"2024-03-06T07:27:26.536438+0000","last_peered":"2024-03-06T07:27:26.536438+0000","last_clean":"2024-03-06T07:27:26.536438+0000","last_became_active":"2024-03-06T07:27:26.534566+0000","last_became_peered":"2024-03-06T07:27:26.534566+0000","last_unstale":"2024-03-06T07:27:26.536438+0000","last_undegraded":"2024-03-06T07:27:26.536438+0000","last_fullsized":"2024-03-06T07:27:26.536438+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:46.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":472,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.526237+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:27:26.526237+0000","last_peered":"2024-03-06T07:27:26.526237+0000","last_clean":"2024-03-06T07:27:26.526237+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:27:26.526237+0000","last_undegraded":"2024-03-06T07:27:26.526237+0000","last_fullsized":"2024-03-06T07:27:26.526237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:31.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 Processor -- start 2024-03-06T07:27:31.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 -- start start 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dec156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dec070aa0 con 0x7f1dec1506b0 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dec070d80 con 0x7f1dec0acc00 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1df08eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec153e10 con 0x7f1dec153990 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1dea59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1dec1506b0 0x7f1dec0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54548/0 (socket says 172.21.15.17:54548) 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.283+0000 7f1dea59c640 1 -- 172.21.15.17:0/2865113421 learned_addr learned my addr 172.21.15.17:0/2865113421 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de3fff640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dec156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814013144 0 0) 0x7f1dec070aa0 con 0x7f1dec1506b0 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd8003520 con 0x7f1dec1506b0 2024-03-06T07:27:31.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125848933 0 0) 0x7f1dec070d80 con 0x7f1dec0acc00 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dec070aa0 con 0x7f1dec0acc00 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859427072 0 0) 0x7f1dd8003520 con 0x7f1dec1506b0 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 msgr2=0x7f1dec156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dec156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 >> v1:172.21.15.163:6789/0 conn(0x7f1dec0acc00 legacy=0x7f1dec0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec1566a0 con 0x7f1dec1506b0 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1ddc001670 con 0x7f1dec1506b0 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 >> v1:172.21.15.17:6789/0 conn(0x7f1dec1506b0 legacy=0x7f1dec0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd8003b70 0x7f1dd8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec153e10 con 0x7f1dd8003b70 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd800baa0 con 0x7f1dd8004800 2024-03-06T07:27:31.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd800bca0 con 0x7f1dec153990 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd8003b70 0x7f1dd8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1de3fff640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dead9d640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dea59c640 1 -- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 msgr2=0x7f1dd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dea59c640 1 -- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dd8004800 msgr2=0x7f1dd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.284+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dd8004800 0x7f1dd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1dea59c640 1 -- 172.21.15.17:0/2865113421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8003520 con 0x7f1dd8003b70 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1dead9d640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dec153990 0x7f1dd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd8003b70 0x7f1dd8003f70 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc007fc0 tx=0x7f1dd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c61ec8d505c2ec3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc007be0 con 0x7f1dd8003b70 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc030070 con 0x7f1dd8003b70 2024-03-06T07:27:31.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.285+0000 7f1de9d9b640 1 -- 172.21.15.17:0/2865113421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc005d10 con 0x7f1dd8003b70 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 -- 172.21.15.17:0/2865113421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd8003b70 msgr2=0x7f1dd8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2865113421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dd8003b70 0x7f1dd8003f70 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc007fc0 tx=0x7f1dd8005830 comp rx=0 tx=0).stop 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1dead9d640 1 -- 172.21.15.17:0/2865113421 reap_dead start 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 -- 172.21.15.17:0/2865113421 shutdown_connections 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 -- 172.21.15.17:0/2865113421 >> 172.21.15.17:0/2865113421 conn(0x7f1dec06b1a0 msgr2=0x7f1dec06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 -- 172.21.15.17:0/2865113421 shutdown_connections 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.286+0000 7f1df08eb640 1 -- 172.21.15.17:0/2865113421 wait complete. 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 Processor -- start 2024-03-06T07:27:31.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 -- start start 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dd8003b70 0x7f1dec1dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 0x7f1dec0aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec156a10 con 0x7f1dec154660 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec156450 con 0x7f1dec0aaa90 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1df08eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0ad070 con 0x7f1dd8003b70 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47126/0 (socket says 172.21.15.17:47126) 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 -- 172.21.15.17:0/2814882104 learned_addr learned my addr 172.21.15.17:0/2814882104 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:31.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dd8003b70 0x7f1dec1dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1de3fff640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 0x7f1dec0aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 -- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dd8003b70 msgr2=0x7f1dec1dfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dd8003b70 0x7f1dec1dfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 -- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 msgr2=0x7f1dec0aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 0x7f1dec0aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1dead9d640 1 -- 172.21.15.17:0/2814882104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec156280 con 0x7f1dec154660 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.288+0000 7f1de3fff640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 0x7f1dec0aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1dead9d640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1dec06be90 tx=0x7f1de400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4006b50 con 0x7f1dec154660 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de4006cf0 con 0x7f1dec154660 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003970 con 0x7f1dec154660 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec1ed150 con 0x7f1dec154660 2024-03-06T07:27:31.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.289+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec0a8e90 con 0x7f1dec154660 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.290+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec04f330 con 0x7f1dec154660 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.290+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1de401a020 con 0x7f1dec154660 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.291+0000 7f1de2ffd640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 0x7f1dcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.291+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1de40899c0 con 0x7f1dec154660 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.291+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 0x7f1dcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.291+0000 7f1dea59c640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 0x7f1dcc06d9c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc034000 tx=0x7f1ddc007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.296+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1de408be20 con 0x7f1dec154660 2024-03-06T07:27:31.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.473+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1dec1ecc20 con 0x7f1dcc06b500 2024-03-06T07:27:31.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.479+0000 7f1de2ffd640 1 -- 172.21.15.17:0/2814882104 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247163 (secure 0 0 0) 0x7f1dec1ecc20 con 0x7f1dcc06b500 2024-03-06T07:27:31.484 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:31.486 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 msgr2=0x7f1dcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 0x7f1dcc06d9c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc034000 tx=0x7f1ddc007000 comp rx=0 tx=0).stop 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 msgr2=0x7f1dec154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1dec06be90 tx=0x7f1de400eb90 comp rx=0 tx=0).stop 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 shutdown_connections 2024-03-06T07:27:31.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1dcc06b500 0x7f1dcc06d9c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1dec154660 0x7f1dec154b10 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1dec0aaa90 0x7f1dec0aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 --2- 172.21.15.17:0/2814882104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1dd8003b70 0x7f1dec1dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 >> 172.21.15.17:0/2814882104 conn(0x7f1dec06b1a0 msgr2=0x7f1dec151700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 shutdown_connections 2024-03-06T07:27:31.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.483+0000 7f1df08eb640 1 -- 172.21.15.17:0/2814882104 wait complete. 2024-03-06T07:27:31.495 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-03-06T07:27:31.441982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271944,"kb_used_data":18336,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754702776,"statfs":{"total":773094113280,"available":772815642624,"internally_reserved":0,"allocated":18776064,"data_stored":10277864,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.925093"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525776+0000","last_change":"2024-03-06T07:27:26.525776+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525776+0000","last_undegraded":"2024-03-06T07:27:26.525776+0000","last_fullsized":"2024-03-06T07:27:26.525776+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527532+0000","last_change":"2024-03-06T07:27:26.527532+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527532+0000","last_undegraded":"2024-03-06T07:27:26.527532+0000","last_fullsized":"2024-03-06T07:27:26.527532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525851+0000","last_change":"2024-03-06T07:27:26.525851+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525851+0000","last_undegraded":"2024-03-06T07:27:26.525851+0000","last_fullsized":"2024-03-06T07:27:26.525851+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.531856+0000","last_change":"2024-03-06T07:27:26.531856+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.531856+0000","last_undegraded":"2024-03-06T07:27:26.531856+0000","last_fullsized":"2024-03-06T07:27:26.531856+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524917+0000","last_change":"2024-03-06T07:27:26.524917+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524917+0000","last_undegraded":"2024-03-06T07:27:26.524917+0000","last_fullsized":"2024-03-06T07:27:26.524917+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522367+0000","last_change":"2024-03-06T07:27:26.522367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522367+0000","last_undegraded":"2024-03-06T07:27:26.522367+0000","last_fullsized":"2024-03-06T07:27:26.522367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525635+0000","last_change":"2024-03-06T07:27:26.525635+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525635+0000","last_undegraded":"2024-03-06T07:27:26.525635+0000","last_fullsized":"2024-03-06T07:27:26.525635+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524185+0000","last_change":"2024-03-06T07:27:26.524185+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524185+0000","last_undegraded":"2024-03-06T07:27:26.524185+0000","last_fullsized":"2024-03-06T07:27:26.524185+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525644+0000","last_change":"2024-03-06T07:27:26.525644+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525644+0000","last_undegraded":"2024-03-06T07:27:26.525644+0000","last_fullsized":"2024-03-06T07:27:26.525644+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525171+0000","last_change":"2024-03-06T07:27:26.525171+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525171+0000","last_undegraded":"2024-03-06T07:27:26.525171+0000","last_fullsized":"2024-03-06T07:27:26.525171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.533216+0000","last_change":"2024-03-06T07:27:26.533216+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.533216+0000","last_undegraded":"2024-03-06T07:27:26.533216+0000","last_fullsized":"2024-03-06T07:27:26.533216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525045+0000","last_change":"2024-03-06T07:27:26.525045+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525045+0000","last_undegraded":"2024-03-06T07:27:26.525045+0000","last_fullsized":"2024-03-06T07:27:26.525045+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523316+0000","last_change":"2024-03-06T07:27:26.523316+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523316+0000","last_undegraded":"2024-03-06T07:27:26.523316+0000","last_fullsized":"2024-03-06T07:27:26.523316+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524993+0000","last_change":"2024-03-06T07:27:26.524993+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524993+0000","last_undegraded":"2024-03-06T07:27:26.524993+0000","last_fullsized":"2024-03-06T07:27:26.524993+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522439+0000","last_change":"2024-03-06T07:27:26.522439+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522439+0000","last_undegraded":"2024-03-06T07:27:26.522439+0000","last_fullsized":"2024-03-06T07:27:26.522439+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523802+0000","last_change":"2024-03-06T07:27:26.523802+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523802+0000","last_undegraded":"2024-03-06T07:27:26.523802+0000","last_fullsized":"2024-03-06T07:27:26.523802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524473+0000","last_change":"2024-03-06T07:27:26.524473+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524473+0000","last_undegraded":"2024-03-06T07:27:26.524473+0000","last_fullsized":"2024-03-06T07:27:26.524473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524367+0000","last_change":"2024-03-06T07:27:26.524367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524367+0000","last_undegraded":"2024-03-06T07:27:26.524367+0000","last_fullsized":"2024-03-06T07:27:26.524367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526381+0000","last_change":"2024-03-06T07:27:26.526381+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526381+0000","last_undegraded":"2024-03-06T07:27:26.526381+0000","last_fullsized":"2024-03-06T07:27:26.526381+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524159+0000","last_change":"2024-03-06T07:27:26.524159+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524159+0000","last_undegraded":"2024-03-06T07:27:26.524159+0000","last_fullsized":"2024-03-06T07:27:26.524159+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523967+0000","last_change":"2024-03-06T07:27:26.523967+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523967+0000","last_undegraded":"2024-03-06T07:27:26.523967+0000","last_fullsized":"2024-03-06T07:27:26.523967+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523234+0000","last_change":"2024-03-06T07:27:26.523234+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523234+0000","last_undegraded":"2024-03-06T07:27:26.523234+0000","last_fullsized":"2024-03-06T07:27:26.523234+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524717+0000","last_change":"2024-03-06T07:27:26.524717+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524717+0000","last_undegraded":"2024-03-06T07:27:26.524717+0000","last_fullsized":"2024-03-06T07:27:26.524717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524868+0000","last_change":"2024-03-06T07:27:26.524868+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524868+0000","last_undegraded":"2024-03-06T07:27:26.524868+0000","last_fullsized":"2024-03-06T07:27:26.524868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525341+0000","last_change":"2024-03-06T07:27:26.525341+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525341+0000","last_undegraded":"2024-03-06T07:27:26.525341+0000","last_fullsized":"2024-03-06T07:27:26.525341+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523861+0000","last_change":"2024-03-06T07:27:26.523861+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523861+0000","last_undegraded":"2024-03-06T07:27:26.523861+0000","last_fullsized":"2024-03-06T07:27:26.523861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.532806+0000","last_change":"2024-03-06T07:27:26.532806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.532806+0000","last_undegraded":"2024-03-06T07:27:26.532806+0000","last_fullsized":"2024-03-06T07:27:26.532806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523869+0000","last_change":"2024-03-06T07:27:26.523869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523869+0000","last_undegraded":"2024-03-06T07:27:26.523869+0000","last_fullsized":"2024-03-06T07:27:26.523869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527024+0000","last_change":"2024-03-06T07:27:26.527024+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527024+0000","last_undegraded":"2024-03-06T07:27:26.527024+0000","last_fullsized":"2024-03-06T07:27:26.527024+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522869+0000","last_change":"2024-03-06T07:27:26.522869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522869+0000","last_undegraded":"2024-03-06T07:27:26.522869+0000","last_fullsized":"2024-03-06T07:27:26.522869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524162+0000","last_change":"2024-03-06T07:27:26.524162+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524162+0000","last_undegraded":"2024-03-06T07:27:26.524162+0000","last_fullsized":"2024-03-06T07:27:26.524162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523410+0000","last_change":"2024-03-06T07:27:26.523410+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523410+0000","last_undegraded":"2024-03-06T07:27:26.523410+0000","last_fullsized":"2024-03-06T07:27:26.523410+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523603+0000","last_change":"2024-03-06T07:27:26.523603+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523603+0000","last_undegraded":"2024-03-06T07:27:26.523603+0000","last_fullsized":"2024-03-06T07:27:26.523603+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537037+0000","last_change":"2024-03-06T07:27:26.536435+0000","last_active":"2024-03-06T07:27:26.537037+0000","last_peered":"2024-03-06T07:27:26.537037+0000","last_clean":"2024-03-06T07:27:26.537037+0000","last_became_active":"2024-03-06T07:27:26.536143+0000","last_became_peered":"2024-03-06T07:27:26.536143+0000","last_unstale":"2024-03-06T07:27:26.537037+0000","last_undegraded":"2024-03-06T07:27:26.537037+0000","last_fullsized":"2024-03-06T07:27:26.537037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:03.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522747+0000","last_change":"2024-03-06T07:27:26.522747+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522747+0000","last_undegraded":"2024-03-06T07:27:26.522747+0000","last_fullsized":"2024-03-06T07:27:26.522747+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541050+0000","last_change":"2024-03-06T07:27:26.535890+0000","last_active":"2024-03-06T07:27:26.541050+0000","last_peered":"2024-03-06T07:27:26.541050+0000","last_clean":"2024-03-06T07:27:26.541050+0000","last_became_active":"2024-03-06T07:27:26.535629+0000","last_became_peered":"2024-03-06T07:27:26.535629+0000","last_unstale":"2024-03-06T07:27:26.541050+0000","last_undegraded":"2024-03-06T07:27:26.541050+0000","last_fullsized":"2024-03-06T07:27:26.541050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:07.272784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523859+0000","last_change":"2024-03-06T07:27:26.523859+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523859+0000","last_undegraded":"2024-03-06T07:27:26.523859+0000","last_fullsized":"2024-03-06T07:27:26.523859+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540965+0000","last_change":"2024-03-06T07:27:26.536426+0000","last_active":"2024-03-06T07:27:26.540965+0000","last_peered":"2024-03-06T07:27:26.540965+0000","last_clean":"2024-03-06T07:27:26.540965+0000","last_became_active":"2024-03-06T07:27:26.536145+0000","last_became_peered":"2024-03-06T07:27:26.536145+0000","last_unstale":"2024-03-06T07:27:26.540965+0000","last_undegraded":"2024-03-06T07:27:26.540965+0000","last_fullsized":"2024-03-06T07:27:26.540965+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:20.645286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522778+0000","last_change":"2024-03-06T07:27:26.522778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522778+0000","last_undegraded":"2024-03-06T07:27:26.522778+0000","last_fullsized":"2024-03-06T07:27:26.522778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533721+0000","last_change":"2024-03-06T07:27:26.531896+0000","last_active":"2024-03-06T07:27:26.533721+0000","last_peered":"2024-03-06T07:27:26.533721+0000","last_clean":"2024-03-06T07:27:26.533721+0000","last_became_active":"2024-03-06T07:27:26.531604+0000","last_became_peered":"2024-03-06T07:27:26.531604+0000","last_unstale":"2024-03-06T07:27:26.533721+0000","last_undegraded":"2024-03-06T07:27:26.533721+0000","last_fullsized":"2024-03-06T07:27:26.533721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:27.060493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523730+0000","last_change":"2024-03-06T07:27:26.523730+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523730+0000","last_undegraded":"2024-03-06T07:27:26.523730+0000","last_fullsized":"2024-03-06T07:27:26.523730+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541104+0000","last_change":"2024-03-06T07:27:26.537136+0000","last_active":"2024-03-06T07:27:26.541104+0000","last_peered":"2024-03-06T07:27:26.541104+0000","last_clean":"2024-03-06T07:27:26.541104+0000","last_became_active":"2024-03-06T07:27:26.536848+0000","last_became_peered":"2024-03-06T07:27:26.536848+0000","last_unstale":"2024-03-06T07:27:26.541104+0000","last_undegraded":"2024-03-06T07:27:26.541104+0000","last_fullsized":"2024-03-06T07:27:26.541104+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:05.557102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522813+0000","last_change":"2024-03-06T07:27:26.522813+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522813+0000","last_undegraded":"2024-03-06T07:27:26.522813+0000","last_fullsized":"2024-03-06T07:27:26.522813+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541500+0000","last_change":"2024-03-06T07:27:26.539023+0000","last_active":"2024-03-06T07:27:26.541500+0000","last_peered":"2024-03-06T07:27:26.541500+0000","last_clean":"2024-03-06T07:27:26.541500+0000","last_became_active":"2024-03-06T07:27:26.538724+0000","last_became_peered":"2024-03-06T07:27:26.538724+0000","last_unstale":"2024-03-06T07:27:26.541500+0000","last_undegraded":"2024-03-06T07:27:26.541500+0000","last_fullsized":"2024-03-06T07:27:26.541500+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524997+0000","last_change":"2024-03-06T07:27:26.524997+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524997+0000","last_undegraded":"2024-03-06T07:27:26.524997+0000","last_fullsized":"2024-03-06T07:27:26.524997+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.528810+0000","last_change":"2024-03-06T07:27:26.527218+0000","last_active":"2024-03-06T07:27:26.528810+0000","last_peered":"2024-03-06T07:27:26.528810+0000","last_clean":"2024-03-06T07:27:26.528810+0000","last_became_active":"2024-03-06T07:27:26.526206+0000","last_became_peered":"2024-03-06T07:27:26.526206+0000","last_unstale":"2024-03-06T07:27:26.528810+0000","last_undegraded":"2024-03-06T07:27:26.528810+0000","last_fullsized":"2024-03-06T07:27:26.528810+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:11:41.576679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522046+0000","last_change":"2024-03-06T07:27:26.522046+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522046+0000","last_undegraded":"2024-03-06T07:27:26.522046+0000","last_fullsized":"2024-03-06T07:27:26.522046+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543830+0000","last_change":"2024-03-06T07:27:26.542376+0000","last_active":"2024-03-06T07:27:26.543830+0000","last_peered":"2024-03-06T07:27:26.543830+0000","last_clean":"2024-03-06T07:27:26.543830+0000","last_became_active":"2024-03-06T07:27:26.542091+0000","last_became_peered":"2024-03-06T07:27:26.542091+0000","last_unstale":"2024-03-06T07:27:26.543830+0000","last_undegraded":"2024-03-06T07:27:26.543830+0000","last_fullsized":"2024-03-06T07:27:26.543830+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:37.673464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522321+0000","last_change":"2024-03-06T07:27:26.522321+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522321+0000","last_undegraded":"2024-03-06T07:27:26.522321+0000","last_fullsized":"2024-03-06T07:27:26.522321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541606+0000","last_change":"2024-03-06T07:27:26.539198+0000","last_active":"2024-03-06T07:27:26.541606+0000","last_peered":"2024-03-06T07:27:26.541606+0000","last_clean":"2024-03-06T07:27:26.541606+0000","last_became_active":"2024-03-06T07:27:26.538712+0000","last_became_peered":"2024-03-06T07:27:26.538712+0000","last_unstale":"2024-03-06T07:27:26.541606+0000","last_undegraded":"2024-03-06T07:27:26.541606+0000","last_fullsized":"2024-03-06T07:27:26.541606+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:39.408799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522214+0000","last_change":"2024-03-06T07:27:26.522214+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522214+0000","last_undegraded":"2024-03-06T07:27:26.522214+0000","last_fullsized":"2024-03-06T07:27:26.522214+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541542+0000","last_change":"2024-03-06T07:27:26.539250+0000","last_active":"2024-03-06T07:27:26.541542+0000","last_peered":"2024-03-06T07:27:26.541542+0000","last_clean":"2024-03-06T07:27:26.541542+0000","last_became_active":"2024-03-06T07:27:26.538970+0000","last_became_peered":"2024-03-06T07:27:26.538970+0000","last_unstale":"2024-03-06T07:27:26.541542+0000","last_undegraded":"2024-03-06T07:27:26.541542+0000","last_fullsized":"2024-03-06T07:27:26.541542+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:30.347291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.534253+0000","last_change":"2024-03-06T07:27:26.534253+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.534253+0000","last_undegraded":"2024-03-06T07:27:26.534253+0000","last_fullsized":"2024-03-06T07:27:26.534253+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540930+0000","last_change":"2024-03-06T07:27:26.537269+0000","last_active":"2024-03-06T07:27:26.540930+0000","last_peered":"2024-03-06T07:27:26.540930+0000","last_clean":"2024-03-06T07:27:26.540930+0000","last_became_active":"2024-03-06T07:27:26.536990+0000","last_became_peered":"2024-03-06T07:27:26.536990+0000","last_unstale":"2024-03-06T07:27:26.540930+0000","last_undegraded":"2024-03-06T07:27:26.540930+0000","last_fullsized":"2024-03-06T07:27:26.540930+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.779629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526845+0000","last_change":"2024-03-06T07:27:26.526845+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526845+0000","last_undegraded":"2024-03-06T07:27:26.526845+0000","last_fullsized":"2024-03-06T07:27:26.526845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536170+0000","last_change":"2024-03-06T07:27:26.533164+0000","last_active":"2024-03-06T07:27:26.536170+0000","last_peered":"2024-03-06T07:27:26.536170+0000","last_clean":"2024-03-06T07:27:26.536170+0000","last_became_active":"2024-03-06T07:27:26.532233+0000","last_became_peered":"2024-03-06T07:27:26.532233+0000","last_unstale":"2024-03-06T07:27:26.536170+0000","last_undegraded":"2024-03-06T07:27:26.536170+0000","last_fullsized":"2024-03-06T07:27:26.536170+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:25.296721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522687+0000","last_change":"2024-03-06T07:27:26.522687+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522687+0000","last_undegraded":"2024-03-06T07:27:26.522687+0000","last_fullsized":"2024-03-06T07:27:26.522687+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537685+0000","last_change":"2024-03-06T07:27:26.536746+0000","last_active":"2024-03-06T07:27:26.537685+0000","last_peered":"2024-03-06T07:27:26.537685+0000","last_clean":"2024-03-06T07:27:26.537685+0000","last_became_active":"2024-03-06T07:27:26.536443+0000","last_became_peered":"2024-03-06T07:27:26.536443+0000","last_unstale":"2024-03-06T07:27:26.537685+0000","last_undegraded":"2024-03-06T07:27:26.537685+0000","last_fullsized":"2024-03-06T07:27:26.537685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:03.817951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522240+0000","last_change":"2024-03-06T07:27:26.522240+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522240+0000","last_undegraded":"2024-03-06T07:27:26.522240+0000","last_fullsized":"2024-03-06T07:27:26.522240+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540837+0000","last_change":"2024-03-06T07:27:26.535007+0000","last_active":"2024-03-06T07:27:26.540837+0000","last_peered":"2024-03-06T07:27:26.540837+0000","last_clean":"2024-03-06T07:27:26.540837+0000","last_became_active":"2024-03-06T07:27:26.534742+0000","last_became_peered":"2024-03-06T07:27:26.534742+0000","last_unstale":"2024-03-06T07:27:26.540837+0000","last_undegraded":"2024-03-06T07:27:26.540837+0000","last_fullsized":"2024-03-06T07:27:26.540837+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:11.268111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522063+0000","last_change":"2024-03-06T07:27:26.522063+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522063+0000","last_undegraded":"2024-03-06T07:27:26.522063+0000","last_fullsized":"2024-03-06T07:27:26.522063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543721+0000","last_change":"2024-03-06T07:27:26.539852+0000","last_active":"2024-03-06T07:27:26.543721+0000","last_peered":"2024-03-06T07:27:26.543721+0000","last_clean":"2024-03-06T07:27:26.543721+0000","last_became_active":"2024-03-06T07:27:26.539585+0000","last_became_peered":"2024-03-06T07:27:26.539585+0000","last_unstale":"2024-03-06T07:27:26.543721+0000","last_undegraded":"2024-03-06T07:27:26.543721+0000","last_fullsized":"2024-03-06T07:27:26.543721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:02.711389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527806+0000","last_change":"2024-03-06T07:27:26.527806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527806+0000","last_undegraded":"2024-03-06T07:27:26.527806+0000","last_fullsized":"2024-03-06T07:27:26.527806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.530613+0000","last_change":"2024-03-06T07:27:26.529704+0000","last_active":"2024-03-06T07:27:26.530613+0000","last_peered":"2024-03-06T07:27:26.530613+0000","last_clean":"2024-03-06T07:27:26.530613+0000","last_became_active":"2024-03-06T07:27:26.529322+0000","last_became_peered":"2024-03-06T07:27:26.529322+0000","last_unstale":"2024-03-06T07:27:26.530613+0000","last_undegraded":"2024-03-06T07:27:26.530613+0000","last_fullsized":"2024-03-06T07:27:26.530613+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:08:23.114407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525947+0000","last_change":"2024-03-06T07:27:26.525947+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525947+0000","last_undegraded":"2024-03-06T07:27:26.525947+0000","last_fullsized":"2024-03-06T07:27:26.525947+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533414+0000","last_change":"2024-03-06T07:27:26.531502+0000","last_active":"2024-03-06T07:27:26.533414+0000","last_peered":"2024-03-06T07:27:26.533414+0000","last_clean":"2024-03-06T07:27:26.533414+0000","last_became_active":"2024-03-06T07:27:26.531216+0000","last_became_peered":"2024-03-06T07:27:26.531216+0000","last_unstale":"2024-03-06T07:27:26.533414+0000","last_undegraded":"2024-03-06T07:27:26.533414+0000","last_fullsized":"2024-03-06T07:27:26.533414+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:41.376525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521721+0000","last_change":"2024-03-06T07:27:26.521721+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521721+0000","last_undegraded":"2024-03-06T07:27:26.521721+0000","last_fullsized":"2024-03-06T07:27:26.521721+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546585+0000","last_change":"2024-03-06T07:27:26.541505+0000","last_active":"2024-03-06T07:27:26.546585+0000","last_peered":"2024-03-06T07:27:26.546585+0000","last_clean":"2024-03-06T07:27:26.546585+0000","last_became_active":"2024-03-06T07:27:26.541235+0000","last_became_peered":"2024-03-06T07:27:26.541235+0000","last_unstale":"2024-03-06T07:27:26.546585+0000","last_undegraded":"2024-03-06T07:27:26.546585+0000","last_fullsized":"2024-03-06T07:27:26.546585+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522464+0000","last_change":"2024-03-06T07:27:26.522464+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522464+0000","last_undegraded":"2024-03-06T07:27:26.522464+0000","last_fullsized":"2024-03-06T07:27:26.522464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543780+0000","last_change":"2024-03-06T07:27:26.540712+0000","last_active":"2024-03-06T07:27:26.543780+0000","last_peered":"2024-03-06T07:27:26.543780+0000","last_clean":"2024-03-06T07:27:26.543780+0000","last_became_active":"2024-03-06T07:27:26.540456+0000","last_became_peered":"2024-03-06T07:27:26.540456+0000","last_unstale":"2024-03-06T07:27:26.543780+0000","last_undegraded":"2024-03-06T07:27:26.543780+0000","last_fullsized":"2024-03-06T07:27:26.543780+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:23.541795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522744+0000","last_change":"2024-03-06T07:27:26.522744+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522744+0000","last_undegraded":"2024-03-06T07:27:26.522744+0000","last_fullsized":"2024-03-06T07:27:26.522744+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535226+0000","last_change":"2024-03-06T07:27:26.530763+0000","last_active":"2024-03-06T07:27:26.535226+0000","last_peered":"2024-03-06T07:27:26.535226+0000","last_clean":"2024-03-06T07:27:26.535226+0000","last_became_active":"2024-03-06T07:27:26.530485+0000","last_became_peered":"2024-03-06T07:27:26.530485+0000","last_unstale":"2024-03-06T07:27:26.535226+0000","last_undegraded":"2024-03-06T07:27:26.535226+0000","last_fullsized":"2024-03-06T07:27:26.535226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:22.926285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525008+0000","last_change":"2024-03-06T07:27:26.525008+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525008+0000","last_undegraded":"2024-03-06T07:27:26.525008+0000","last_fullsized":"2024-03-06T07:27:26.525008+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533839+0000","last_change":"2024-03-06T07:27:26.532686+0000","last_active":"2024-03-06T07:27:26.533839+0000","last_peered":"2024-03-06T07:27:26.533839+0000","last_clean":"2024-03-06T07:27:26.533839+0000","last_became_active":"2024-03-06T07:27:26.532416+0000","last_became_peered":"2024-03-06T07:27:26.532416+0000","last_unstale":"2024-03-06T07:27:26.533839+0000","last_undegraded":"2024-03-06T07:27:26.533839+0000","last_fullsized":"2024-03-06T07:27:26.533839+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:01.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.528494+0000","last_change":"2024-03-06T07:27:26.528494+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.528494+0000","last_undegraded":"2024-03-06T07:27:26.528494+0000","last_fullsized":"2024-03-06T07:27:26.528494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540337+0000","last_change":"2024-03-06T07:27:26.539907+0000","last_active":"2024-03-06T07:27:26.540337+0000","last_peered":"2024-03-06T07:27:26.540337+0000","last_clean":"2024-03-06T07:27:26.540337+0000","last_became_active":"2024-03-06T07:27:26.539575+0000","last_became_peered":"2024-03-06T07:27:26.539575+0000","last_unstale":"2024-03-06T07:27:26.540337+0000","last_undegraded":"2024-03-06T07:27:26.540337+0000","last_fullsized":"2024-03-06T07:27:26.540337+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:44:36.880373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521242+0000","last_change":"2024-03-06T07:27:26.521242+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521242+0000","last_undegraded":"2024-03-06T07:27:26.521242+0000","last_fullsized":"2024-03-06T07:27:26.521242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.532649+0000","last_change":"2024-03-06T07:27:26.531365+0000","last_active":"2024-03-06T07:27:26.532649+0000","last_peered":"2024-03-06T07:27:26.532649+0000","last_clean":"2024-03-06T07:27:26.532649+0000","last_became_active":"2024-03-06T07:27:26.531079+0000","last_became_peered":"2024-03-06T07:27:26.531079+0000","last_unstale":"2024-03-06T07:27:26.532649+0000","last_undegraded":"2024-03-06T07:27:26.532649+0000","last_fullsized":"2024-03-06T07:27:26.532649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:52.548111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527383+0000","last_change":"2024-03-06T07:27:26.527383+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527383+0000","last_undegraded":"2024-03-06T07:27:26.527383+0000","last_fullsized":"2024-03-06T07:27:26.527383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546609+0000","last_change":"2024-03-06T07:27:26.542508+0000","last_active":"2024-03-06T07:27:26.546609+0000","last_peered":"2024-03-06T07:27:26.546609+0000","last_clean":"2024-03-06T07:27:26.546609+0000","last_became_active":"2024-03-06T07:27:26.542231+0000","last_became_peered":"2024-03-06T07:27:26.542231+0000","last_unstale":"2024-03-06T07:27:26.546609+0000","last_undegraded":"2024-03-06T07:27:26.546609+0000","last_fullsized":"2024-03-06T07:27:26.546609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525432+0000","last_change":"2024-03-06T07:27:26.525432+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525432+0000","last_undegraded":"2024-03-06T07:27:26.525432+0000","last_fullsized":"2024-03-06T07:27:26.525432+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.539884+0000","last_change":"2024-03-06T07:27:26.539371+0000","last_active":"2024-03-06T07:27:26.539884+0000","last_peered":"2024-03-06T07:27:26.539884+0000","last_clean":"2024-03-06T07:27:26.539884+0000","last_became_active":"2024-03-06T07:27:26.539052+0000","last_became_peered":"2024-03-06T07:27:26.539052+0000","last_unstale":"2024-03-06T07:27:26.539884+0000","last_undegraded":"2024-03-06T07:27:26.539884+0000","last_fullsized":"2024-03-06T07:27:26.539884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:18.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527778+0000","last_change":"2024-03-06T07:27:26.527778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527778+0000","last_undegraded":"2024-03-06T07:27:26.527778+0000","last_fullsized":"2024-03-06T07:27:26.527778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533750+0000","last_change":"2024-03-06T07:27:26.532169+0000","last_active":"2024-03-06T07:27:26.533750+0000","last_peered":"2024-03-06T07:27:26.533750+0000","last_clean":"2024-03-06T07:27:26.533750+0000","last_became_active":"2024-03-06T07:27:26.531854+0000","last_became_peered":"2024-03-06T07:27:26.531854+0000","last_unstale":"2024-03-06T07:27:26.533750+0000","last_undegraded":"2024-03-06T07:27:26.533750+0000","last_fullsized":"2024-03-06T07:27:26.533750+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:34.712123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526066+0000","last_change":"2024-03-06T07:27:26.526066+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526066+0000","last_undegraded":"2024-03-06T07:27:26.526066+0000","last_fullsized":"2024-03-06T07:27:26.526066+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535765+0000","last_change":"2024-03-06T07:27:26.534771+0000","last_active":"2024-03-06T07:27:26.535765+0000","last_peered":"2024-03-06T07:27:26.535765+0000","last_clean":"2024-03-06T07:27:26.535765+0000","last_became_active":"2024-03-06T07:27:26.534079+0000","last_became_peered":"2024-03-06T07:27:26.534079+0000","last_unstale":"2024-03-06T07:27:26.535765+0000","last_undegraded":"2024-03-06T07:27:26.535765+0000","last_fullsized":"2024-03-06T07:27:26.535765+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:28.965661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521229+0000","last_change":"2024-03-06T07:27:26.521229+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521229+0000","last_undegraded":"2024-03-06T07:27:26.521229+0000","last_fullsized":"2024-03-06T07:27:26.521229+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535262+0000","last_change":"2024-03-06T07:27:26.532514+0000","last_active":"2024-03-06T07:27:26.535262+0000","last_peered":"2024-03-06T07:27:26.535262+0000","last_clean":"2024-03-06T07:27:26.535262+0000","last_became_active":"2024-03-06T07:27:26.532219+0000","last_became_peered":"2024-03-06T07:27:26.532219+0000","last_unstale":"2024-03-06T07:27:26.535262+0000","last_undegraded":"2024-03-06T07:27:26.535262+0000","last_fullsized":"2024-03-06T07:27:26.535262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:27:12.063304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521463+0000","last_change":"2024-03-06T07:27:26.521463+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521463+0000","last_undegraded":"2024-03-06T07:27:26.521463+0000","last_fullsized":"2024-03-06T07:27:26.521463+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541466+0000","last_change":"2024-03-06T07:27:26.538820+0000","last_active":"2024-03-06T07:27:26.541466+0000","last_peered":"2024-03-06T07:27:26.541466+0000","last_clean":"2024-03-06T07:27:26.541466+0000","last_became_active":"2024-03-06T07:27:26.538519+0000","last_became_peered":"2024-03-06T07:27:26.538519+0000","last_unstale":"2024-03-06T07:27:26.541466+0000","last_undegraded":"2024-03-06T07:27:26.541466+0000","last_fullsized":"2024-03-06T07:27:26.541466+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524485+0000","last_change":"2024-03-06T07:27:26.524485+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524485+0000","last_undegraded":"2024-03-06T07:27:26.524485+0000","last_fullsized":"2024-03-06T07:27:26.524485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541511+0000","last_change":"2024-03-06T07:27:26.539830+0000","last_active":"2024-03-06T07:27:26.541511+0000","last_peered":"2024-03-06T07:27:26.541511+0000","last_clean":"2024-03-06T07:27:26.541511+0000","last_became_active":"2024-03-06T07:27:26.539557+0000","last_became_peered":"2024-03-06T07:27:26.539557+0000","last_unstale":"2024-03-06T07:27:26.541511+0000","last_undegraded":"2024-03-06T07:27:26.541511+0000","last_fullsized":"2024-03-06T07:27:26.541511+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:23.412215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522827+0000","last_change":"2024-03-06T07:27:26.522827+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522827+0000","last_undegraded":"2024-03-06T07:27:26.522827+0000","last_fullsized":"2024-03-06T07:27:26.522827+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543752+0000","last_change":"2024-03-06T07:27:26.540192+0000","last_active":"2024-03-06T07:27:26.543752+0000","last_peered":"2024-03-06T07:27:26.543752+0000","last_clean":"2024-03-06T07:27:26.543752+0000","last_became_active":"2024-03-06T07:27:26.539560+0000","last_became_peered":"2024-03-06T07:27:26.539560+0000","last_unstale":"2024-03-06T07:27:26.543752+0000","last_undegraded":"2024-03-06T07:27:26.543752+0000","last_fullsized":"2024-03-06T07:27:26.543752+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:57.625904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523231+0000","last_change":"2024-03-06T07:27:26.523231+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523231+0000","last_undegraded":"2024-03-06T07:27:26.523231+0000","last_fullsized":"2024-03-06T07:27:26.523231+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536438+0000","last_change":"2024-03-06T07:27:26.534849+0000","last_active":"2024-03-06T07:27:26.536438+0000","last_peered":"2024-03-06T07:27:26.536438+0000","last_clean":"2024-03-06T07:27:26.536438+0000","last_became_active":"2024-03-06T07:27:26.534566+0000","last_became_peered":"2024-03-06T07:27:26.534566+0000","last_unstale":"2024-03-06T07:27:26.536438+0000","last_undegraded":"2024-03-06T07:27:26.536438+0000","last_fullsized":"2024-03-06T07:27:26.536438+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:46.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":472,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.526237+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:27:26.526237+0000","last_peered":"2024-03-06T07:27:26.526237+0000","last_clean":"2024-03-06T07:27:26.526237+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:27:26.526237+0000","last_undegraded":"2024-03-06T07:27:26.526237+0000","last_fullsized":"2024-03-06T07:27:26.526237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3f is not active+clean 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525776+0000', 'last_change': '2024-03-06T07:27:26.525776+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525776+0000', 'last_undegraded': '2024-03-06T07:27:26.525776+0000', 'last_fullsized': '2024-03-06T07:27:26.525776+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3e is not active+clean 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.527532+0000', 'last_change': '2024-03-06T07:27:26.527532+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.527532+0000', 'last_undegraded': '2024-03-06T07:27:26.527532+0000', 'last_fullsized': '2024-03-06T07:27:26.527532+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3d is not active+clean 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525851+0000', 'last_change': '2024-03-06T07:27:26.525851+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525851+0000', 'last_undegraded': '2024-03-06T07:27:26.525851+0000', 'last_fullsized': '2024-03-06T07:27:26.525851+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3c is not active+clean 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.531856+0000', 'last_change': '2024-03-06T07:27:26.531856+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.531856+0000', 'last_undegraded': '2024-03-06T07:27:26.531856+0000', 'last_fullsized': '2024-03-06T07:27:26.531856+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:27:31.499 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3b is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524917+0000', 'last_change': '2024-03-06T07:27:26.524917+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524917+0000', 'last_undegraded': '2024-03-06T07:27:26.524917+0000', 'last_fullsized': '2024-03-06T07:27:26.524917+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3a is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522367+0000', 'last_change': '2024-03-06T07:27:26.522367+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522367+0000', 'last_undegraded': '2024-03-06T07:27:26.522367+0000', 'last_fullsized': '2024-03-06T07:27:26.522367+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.39 is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525635+0000', 'last_change': '2024-03-06T07:27:26.525635+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525635+0000', 'last_undegraded': '2024-03-06T07:27:26.525635+0000', 'last_fullsized': '2024-03-06T07:27:26.525635+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.38 is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524185+0000', 'last_change': '2024-03-06T07:27:26.524185+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524185+0000', 'last_undegraded': '2024-03-06T07:27:26.524185+0000', 'last_fullsized': '2024-03-06T07:27:26.524185+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.37 is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525644+0000', 'last_change': '2024-03-06T07:27:26.525644+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525644+0000', 'last_undegraded': '2024-03-06T07:27:26.525644+0000', 'last_fullsized': '2024-03-06T07:27:26.525644+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.36 is not active+clean 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525171+0000', 'last_change': '2024-03-06T07:27:26.525171+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525171+0000', 'last_undegraded': '2024-03-06T07:27:26.525171+0000', 'last_fullsized': '2024-03-06T07:27:26.525171+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.500 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.35 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.533216+0000', 'last_change': '2024-03-06T07:27:26.533216+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.533216+0000', 'last_undegraded': '2024-03-06T07:27:26.533216+0000', 'last_fullsized': '2024-03-06T07:27:26.533216+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.34 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525045+0000', 'last_change': '2024-03-06T07:27:26.525045+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525045+0000', 'last_undegraded': '2024-03-06T07:27:26.525045+0000', 'last_fullsized': '2024-03-06T07:27:26.525045+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.33 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523316+0000', 'last_change': '2024-03-06T07:27:26.523316+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523316+0000', 'last_undegraded': '2024-03-06T07:27:26.523316+0000', 'last_fullsized': '2024-03-06T07:27:26.523316+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.32 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524993+0000', 'last_change': '2024-03-06T07:27:26.524993+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524993+0000', 'last_undegraded': '2024-03-06T07:27:26.524993+0000', 'last_fullsized': '2024-03-06T07:27:26.524993+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.31 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522439+0000', 'last_change': '2024-03-06T07:27:26.522439+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522439+0000', 'last_undegraded': '2024-03-06T07:27:26.522439+0000', 'last_fullsized': '2024-03-06T07:27:26.522439+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.30 is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523802+0000', 'last_change': '2024-03-06T07:27:26.523802+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523802+0000', 'last_undegraded': '2024-03-06T07:27:26.523802+0000', 'last_fullsized': '2024-03-06T07:27:26.523802+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2f is not active+clean 2024-03-06T07:27:31.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524473+0000', 'last_change': '2024-03-06T07:27:26.524473+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524473+0000', 'last_undegraded': '2024-03-06T07:27:26.524473+0000', 'last_fullsized': '2024-03-06T07:27:26.524473+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2e is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524367+0000', 'last_change': '2024-03-06T07:27:26.524367+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524367+0000', 'last_undegraded': '2024-03-06T07:27:26.524367+0000', 'last_fullsized': '2024-03-06T07:27:26.524367+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2d is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.526381+0000', 'last_change': '2024-03-06T07:27:26.526381+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.526381+0000', 'last_undegraded': '2024-03-06T07:27:26.526381+0000', 'last_fullsized': '2024-03-06T07:27:26.526381+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2c is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524159+0000', 'last_change': '2024-03-06T07:27:26.524159+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524159+0000', 'last_undegraded': '2024-03-06T07:27:26.524159+0000', 'last_fullsized': '2024-03-06T07:27:26.524159+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2b is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523967+0000', 'last_change': '2024-03-06T07:27:26.523967+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523967+0000', 'last_undegraded': '2024-03-06T07:27:26.523967+0000', 'last_fullsized': '2024-03-06T07:27:26.523967+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2a is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523234+0000', 'last_change': '2024-03-06T07:27:26.523234+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523234+0000', 'last_undegraded': '2024-03-06T07:27:26.523234+0000', 'last_fullsized': '2024-03-06T07:27:26.523234+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.29 is not active+clean 2024-03-06T07:27:31.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524717+0000', 'last_change': '2024-03-06T07:27:26.524717+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524717+0000', 'last_undegraded': '2024-03-06T07:27:26.524717+0000', 'last_fullsized': '2024-03-06T07:27:26.524717+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.28 is not active+clean 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524868+0000', 'last_change': '2024-03-06T07:27:26.524868+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524868+0000', 'last_undegraded': '2024-03-06T07:27:26.524868+0000', 'last_fullsized': '2024-03-06T07:27:26.524868+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.27 is not active+clean 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525341+0000', 'last_change': '2024-03-06T07:27:26.525341+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525341+0000', 'last_undegraded': '2024-03-06T07:27:26.525341+0000', 'last_fullsized': '2024-03-06T07:27:26.525341+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.26 is not active+clean 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523861+0000', 'last_change': '2024-03-06T07:27:26.523861+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523861+0000', 'last_undegraded': '2024-03-06T07:27:26.523861+0000', 'last_fullsized': '2024-03-06T07:27:26.523861+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.25 is not active+clean 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.532806+0000', 'last_change': '2024-03-06T07:27:26.532806+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.532806+0000', 'last_undegraded': '2024-03-06T07:27:26.532806+0000', 'last_fullsized': '2024-03-06T07:27:26.532806+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:27:31.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.24 is not active+clean 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523869+0000', 'last_change': '2024-03-06T07:27:26.523869+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523869+0000', 'last_undegraded': '2024-03-06T07:27:26.523869+0000', 'last_fullsized': '2024-03-06T07:27:26.523869+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.23 is not active+clean 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.527024+0000', 'last_change': '2024-03-06T07:27:26.527024+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.527024+0000', 'last_undegraded': '2024-03-06T07:27:26.527024+0000', 'last_fullsized': '2024-03-06T07:27:26.527024+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.22 is not active+clean 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522869+0000', 'last_change': '2024-03-06T07:27:26.522869+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522869+0000', 'last_undegraded': '2024-03-06T07:27:26.522869+0000', 'last_fullsized': '2024-03-06T07:27:26.522869+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.21 is not active+clean 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524162+0000', 'last_change': '2024-03-06T07:27:26.524162+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524162+0000', 'last_undegraded': '2024-03-06T07:27:26.524162+0000', 'last_fullsized': '2024-03-06T07:27:26.524162+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.20 is not active+clean 2024-03-06T07:27:31.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523410+0000', 'last_change': '2024-03-06T07:27:26.523410+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523410+0000', 'last_undegraded': '2024-03-06T07:27:26.523410+0000', 'last_fullsized': '2024-03-06T07:27:26.523410+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1c is not active+clean 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523603+0000', 'last_change': '2024-03-06T07:27:26.523603+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523603+0000', 'last_undegraded': '2024-03-06T07:27:26.523603+0000', 'last_fullsized': '2024-03-06T07:27:26.523603+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1d is not active+clean 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522747+0000', 'last_change': '2024-03-06T07:27:26.522747+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522747+0000', 'last_undegraded': '2024-03-06T07:27:26.522747+0000', 'last_fullsized': '2024-03-06T07:27:26.522747+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1e is not active+clean 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523859+0000', 'last_change': '2024-03-06T07:27:26.523859+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523859+0000', 'last_undegraded': '2024-03-06T07:27:26.523859+0000', 'last_fullsized': '2024-03-06T07:27:26.523859+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.18 is not active+clean 2024-03-06T07:27:31.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522778+0000', 'last_change': '2024-03-06T07:27:26.522778+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522778+0000', 'last_undegraded': '2024-03-06T07:27:26.522778+0000', 'last_fullsized': '2024-03-06T07:27:26.522778+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.19 is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523730+0000', 'last_change': '2024-03-06T07:27:26.523730+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523730+0000', 'last_undegraded': '2024-03-06T07:27:26.523730+0000', 'last_fullsized': '2024-03-06T07:27:26.523730+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1a is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522813+0000', 'last_change': '2024-03-06T07:27:26.522813+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522813+0000', 'last_undegraded': '2024-03-06T07:27:26.522813+0000', 'last_fullsized': '2024-03-06T07:27:26.522813+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1b is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524997+0000', 'last_change': '2024-03-06T07:27:26.524997+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524997+0000', 'last_undegraded': '2024-03-06T07:27:26.524997+0000', 'last_fullsized': '2024-03-06T07:27:26.524997+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.14 is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522046+0000', 'last_change': '2024-03-06T07:27:26.522046+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522046+0000', 'last_undegraded': '2024-03-06T07:27:26.522046+0000', 'last_fullsized': '2024-03-06T07:27:26.522046+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.15 is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522321+0000', 'last_change': '2024-03-06T07:27:26.522321+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522321+0000', 'last_undegraded': '2024-03-06T07:27:26.522321+0000', 'last_fullsized': '2024-03-06T07:27:26.522321+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.16 is not active+clean 2024-03-06T07:27:31.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522214+0000', 'last_change': '2024-03-06T07:27:26.522214+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522214+0000', 'last_undegraded': '2024-03-06T07:27:26.522214+0000', 'last_fullsized': '2024-03-06T07:27:26.522214+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.17 is not active+clean 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.534253+0000', 'last_change': '2024-03-06T07:27:26.534253+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.534253+0000', 'last_undegraded': '2024-03-06T07:27:26.534253+0000', 'last_fullsized': '2024-03-06T07:27:26.534253+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.10 is not active+clean 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.526845+0000', 'last_change': '2024-03-06T07:27:26.526845+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.526845+0000', 'last_undegraded': '2024-03-06T07:27:26.526845+0000', 'last_fullsized': '2024-03-06T07:27:26.526845+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.11 is not active+clean 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522687+0000', 'last_change': '2024-03-06T07:27:26.522687+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522687+0000', 'last_undegraded': '2024-03-06T07:27:26.522687+0000', 'last_fullsized': '2024-03-06T07:27:26.522687+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.12 is not active+clean 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522240+0000', 'last_change': '2024-03-06T07:27:26.522240+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522240+0000', 'last_undegraded': '2024-03-06T07:27:26.522240+0000', 'last_fullsized': '2024-03-06T07:27:26.522240+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.13 is not active+clean 2024-03-06T07:27:31.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522063+0000', 'last_change': '2024-03-06T07:27:26.522063+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522063+0000', 'last_undegraded': '2024-03-06T07:27:26.522063+0000', 'last_fullsized': '2024-03-06T07:27:26.522063+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.c is not active+clean 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.527806+0000', 'last_change': '2024-03-06T07:27:26.527806+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.527806+0000', 'last_undegraded': '2024-03-06T07:27:26.527806+0000', 'last_fullsized': '2024-03-06T07:27:26.527806+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.d is not active+clean 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525947+0000', 'last_change': '2024-03-06T07:27:26.525947+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525947+0000', 'last_undegraded': '2024-03-06T07:27:26.525947+0000', 'last_fullsized': '2024-03-06T07:27:26.525947+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.e is not active+clean 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.521721+0000', 'last_change': '2024-03-06T07:27:26.521721+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.521721+0000', 'last_undegraded': '2024-03-06T07:27:26.521721+0000', 'last_fullsized': '2024-03-06T07:27:26.521721+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.f is not active+clean 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522464+0000', 'last_change': '2024-03-06T07:27:26.522464+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522464+0000', 'last_undegraded': '2024-03-06T07:27:26.522464+0000', 'last_fullsized': '2024-03-06T07:27:26.522464+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.8 is not active+clean 2024-03-06T07:27:31.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522744+0000', 'last_change': '2024-03-06T07:27:26.522744+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522744+0000', 'last_undegraded': '2024-03-06T07:27:26.522744+0000', 'last_fullsized': '2024-03-06T07:27:26.522744+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.9 is not active+clean 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525008+0000', 'last_change': '2024-03-06T07:27:26.525008+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525008+0000', 'last_undegraded': '2024-03-06T07:27:26.525008+0000', 'last_fullsized': '2024-03-06T07:27:26.525008+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.a is not active+clean 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.528494+0000', 'last_change': '2024-03-06T07:27:26.528494+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.528494+0000', 'last_undegraded': '2024-03-06T07:27:26.528494+0000', 'last_fullsized': '2024-03-06T07:27:26.528494+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.b is not active+clean 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.521242+0000', 'last_change': '2024-03-06T07:27:26.521242+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.521242+0000', 'last_undegraded': '2024-03-06T07:27:26.521242+0000', 'last_fullsized': '2024-03-06T07:27:26.521242+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.4 is not active+clean 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.527383+0000', 'last_change': '2024-03-06T07:27:26.527383+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.527383+0000', 'last_undegraded': '2024-03-06T07:27:26.527383+0000', 'last_fullsized': '2024-03-06T07:27:26.527383+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.5 is not active+clean 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.525432+0000', 'last_change': '2024-03-06T07:27:26.525432+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.525432+0000', 'last_undegraded': '2024-03-06T07:27:26.525432+0000', 'last_fullsized': '2024-03-06T07:27:26.525432+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-06T07:27:31.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.6 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.527778+0000', 'last_change': '2024-03-06T07:27:26.527778+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.527778+0000', 'last_undegraded': '2024-03-06T07:27:26.527778+0000', 'last_fullsized': '2024-03-06T07:27:26.527778+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.7 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.526066+0000', 'last_change': '2024-03-06T07:27:26.526066+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.526066+0000', 'last_undegraded': '2024-03-06T07:27:26.526066+0000', 'last_fullsized': '2024-03-06T07:27:26.526066+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.0 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.521229+0000', 'last_change': '2024-03-06T07:27:26.521229+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.521229+0000', 'last_undegraded': '2024-03-06T07:27:26.521229+0000', 'last_fullsized': '2024-03-06T07:27:26.521229+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.521463+0000', 'last_change': '2024-03-06T07:27:26.521463+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.521463+0000', 'last_undegraded': '2024-03-06T07:27:26.521463+0000', 'last_fullsized': '2024-03-06T07:27:26.521463+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.524485+0000', 'last_change': '2024-03-06T07:27:26.524485+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.524485+0000', 'last_undegraded': '2024-03-06T07:27:26.524485+0000', 'last_fullsized': '2024-03-06T07:27:26.524485+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3 is not active+clean 2024-03-06T07:27:31.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.522827+0000', 'last_change': '2024-03-06T07:27:26.522827+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.522827+0000', 'last_undegraded': '2024-03-06T07:27:26.522827+0000', 'last_fullsized': '2024-03-06T07:27:26.522827+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-06T07:27:31.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1f is not active+clean 2024-03-06T07:27:31.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 184, 'state': 'creating+peering', 'last_fresh': '2024-03-06T07:27:26.523231+0000', 'last_change': '2024-03-06T07:27:26.523231+0000', 'last_active': '2024-03-06T07:27:26.513551+0000', 'last_peered': '2024-03-06T07:27:26.513551+0000', 'last_clean': '2024-03-06T07:27:26.513551+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-06T07:27:26.523231+0000', 'last_undegraded': '2024-03-06T07:27:26.523231+0000', 'last_fullsized': '2024-03-06T07:27:26.523231+0000', 'mapping_epoch': 184, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 184, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'last_clean_scrub_stamp': '2024-03-06T07:27:26.513551+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-06T07:27:31.511 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-06T07:27:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 Processor -- start 2024-03-06T07:27:31.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 -- start start 2024-03-06T07:27:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70f00c6780 0x7f70f0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f0072630 con 0x7f70f006da40 2024-03-06T07:27:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f006de20 con 0x7f70f00c6780 2024-03-06T07:27:31.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.637+0000 7f70f598c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0072910 con 0x7f70f01592d0 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70f00c6780 0x7f70f0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70eeffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f70f006da40 0x7f70f0155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40776/0 (socket says 172.21.15.17:40776) 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70eeffd640 1 -- 172.21.15.17:0/2604049983 learned_addr learned my addr 172.21.15.17:0/2604049983 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70f00c6780 0x7f70f0158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56180/0 (socket says 172.21.15.17:56180) 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70ef7fe640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:48196/0 (socket says 172.21.15.17:48196) 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 -- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 msgr2=0x7f70f00c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 -- 172.21.15.17:0/2604049983 >> v1:172.21.15.163:6789/0 conn(0x7f70f006da40 legacy=0x7f70f0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70e7fff640 1 -- 172.21.15.17:0/2604049983 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0072bf0 con 0x7f70f00c6780 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70ef7fe640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.638+0000 7f70ee7fc640 1 -- 172.21.15.17:0/2604049983 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542441138 0 0) 0x7f70f0072630 con 0x7f70f006da40 2024-03-06T07:27:31.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.639+0000 7f70e7fff640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f00c6780 0x7f70f0158d90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f70d80097a0 tx=0x7f70d802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76099c870a0d6bf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.639+0000 7f70ee7fc640 1 -- 172.21.15.17:0/2604049983 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80315a0 con 0x7f70f00c6780 2024-03-06T07:27:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.639+0000 7f70ee7fc640 1 -- 172.21.15.17:0/2604049983 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8037070 con 0x7f70f00c6780 2024-03-06T07:27:31.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.639+0000 7f70ee7fc640 1 -- 172.21.15.17:0/2604049983 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8031960 con 0x7f70f00c6780 2024-03-06T07:27:31.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 -- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f00c6780 msgr2=0x7f70f0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f00c6780 0x7f70f0158d90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f70d80097a0 tx=0x7f70d802b840 comp rx=0 tx=0).stop 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 -- 172.21.15.17:0/2604049983 shutdown_connections 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f01592d0 0x7f70f00c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 --2- 172.21.15.17:0/2604049983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f00c6780 0x7f70f0158d90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 -- 172.21.15.17:0/2604049983 >> 172.21.15.17:0/2604049983 conn(0x7f70f00aa630 msgr2=0x7f70f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 -- 172.21.15.17:0/2604049983 shutdown_connections 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.640+0000 7f70f598c640 1 -- 172.21.15.17:0/2604049983 wait complete. 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 Processor -- start 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 -- start start 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 0x7f70f014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f00c6780 0x7f70f01e1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c3940 con 0x7f70f01592d0 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c37c0 con 0x7f70f006da40 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.641+0000 7f70f598c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c3ac0 con 0x7f70f00c6780 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47130/0 (socket says 172.21.15.17:47130) 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 -- 172.21.15.17:0/2373027963 learned_addr learned my addr 172.21.15.17:0/2373027963 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70e7fff640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f00c6780 0x7f70f01e1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70eeffd640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 0x7f70f014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 -- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f00c6780 msgr2=0x7f70f01e1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f00c6780 0x7f70f01e1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 -- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 msgr2=0x7f70f014e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 0x7f70f014e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 -- 172.21.15.17:0/2373027963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00ae810 con 0x7f70f01592d0 2024-03-06T07:27:31.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70eeffd640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 0x7f70f014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ef7fe640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f70e800efc0 tx=0x7f70e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8018030 con 0x7f70f01592d0 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e800cd30 con 0x7f70f01592d0 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f00ad260 con 0x7f70f01592d0 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8003bb0 con 0x7f70f01592d0 2024-03-06T07:27:31.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.642+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f01ebb30 con 0x7f70f01592d0 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.644+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f0156270 con 0x7f70f01592d0 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.644+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70e8005d10 con 0x7f70f01592d0 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.644+0000 7f70ecff9640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 0x7f70d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.644+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70e808ac30 con 0x7f70f01592d0 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.645+0000 7f70eeffd640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 0x7f70d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:31.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.645+0000 7f70eeffd640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 0x7f70d006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f70e0000f60 tx=0x7f70e0007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:31.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.648+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70f0156270 con 0x7f70f01592d0 2024-03-06T07:27:31.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.896+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f70f01550e0 con 0x7f70f01592d0 2024-03-06T07:27:31.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.896+0000 7f70ecff9640 1 -- 172.21.15.17:0/2373027963 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1458 (secure 0 0 0) 0x7f70e8022070 con 0x7f70f01592d0 2024-03-06T07:27:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:31.898 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":464,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":186,"num_osds":8,"num_up_osds":8,"osd_up_since":1709709597,"num_in_osds":8,"osd_in_since":1709709589,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":278470656,"bytes_avail":772815642624,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":173,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"a","status":"up:active","gid":9900}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-03-06T07:27:09.437224+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 msgr2=0x7f70d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 0x7f70d006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f70e0000f60 tx=0x7f70e0007ca0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 msgr2=0x7f70f01e2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f70e800efc0 tx=0x7f70e800c490 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 shutdown_connections 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70d006b4d0 0x7f70d006d990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70f01592d0 0x7f70f01e2170 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70f00c6780 0x7f70f01e1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 --2- 172.21.15.17:0/2373027963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70f006da40 0x7f70f014e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:31.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 >> 172.21.15.17:0/2373027963 conn(0x7f70f00aa630 msgr2=0x7f70f00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:31.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 shutdown_connections 2024-03-06T07:27:31.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:31.899+0000 7f70f598c640 1 -- 172.21.15.17:0/2373027963 wait complete. 2024-03-06T07:27:31.914 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-06T07:27:31.915 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.028+0000 7f08d3581640 1 Processor -- start 2024-03-06T07:27:32.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.028+0000 7f08d3581640 1 -- start start 2024-03-06T07:27:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d3581640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08cc14f360 0x7f08cc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d3581640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08cc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d3581640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08cc0c73d0 con 0x7f08cc0c5880 2024-03-06T07:27:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d3581640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc0c8dd0 con 0x7f08cc14f360 2024-03-06T07:27:32.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d3581640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0be2e0 con 0x7f08cc0c88c0 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d1b1e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f08cc0c5880 0x7f08cc158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54552/0 (socket says 172.21.15.17:54552) 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d1b1e640 1 -- 172.21.15.17:0/477882335 learned_addr learned my addr 172.21.15.17:0/477882335 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118263764 0 0) 0x7f08cc0c73d0 con 0x7f08cc0c5880 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.029+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> v2:172.21.15.163:3300/0 conn(0x7f08cc14f360 0x7f08cc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d0b1c640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08cc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b8003520 con 0x7f08cc0c5880 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220054802 0 0) 0x7f08b8003520 con 0x7f08cc0c5880 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 msgr2=0x7f08cc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08cc0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 >> v2:172.21.15.163:3300/0 conn(0x7f08cc14f360 msgr2=0x7f08cc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 --2- 172.21.15.17:0/477882335 >> v2:172.21.15.163:3300/0 conn(0x7f08cc14f360 0x7f08cc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0c7670 con 0x7f08cc0c5880 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f08c80019f0 con 0x7f08cc0c5880 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> v2:172.21.15.163:3300/0 conn(0x7f08cc14f360 0x7f08cc155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 >> v1:172.21.15.17:6789/0 conn(0x7f08cc0c5880 legacy=0x7f08cc158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08b80039b0 0x7f08b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b8004d40 0x7f08b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0be2e0 con 0x7f08b80039b0 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b800b9f0 con 0x7f08b8004d40 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b800bbf0 con 0x7f08cc0c88c0 2024-03-06T07:27:32.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08b80039b0 0x7f08b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d0b1c640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d1b1e640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b8004d40 0x7f08b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 -- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 msgr2=0x7f08b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c88c0 0x7f08b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 -- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b8004d40 msgr2=0x7f08b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b8004d40 0x7f08b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d131d640 1 -- 172.21.15.17:0/477882335 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8003520 con 0x7f08b80039b0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.030+0000 7f08d1b1e640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b8004d40 0x7f08b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.031+0000 7f08d131d640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08b80039b0 0x7f08b8003dd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f08c40098a0 tx=0x7f08b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10545b22630551e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.031+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400f040 con 0x7f08b80039b0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.031+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c4004230 con 0x7f08b80039b0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.031+0000 7f08c3fff640 1 -- 172.21.15.17:0/477882335 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4003610 con 0x7f08b80039b0 2024-03-06T07:27:32.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 -- 172.21.15.17:0/477882335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08b80039b0 msgr2=0x7f08b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 --2- 172.21.15.17:0/477882335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08b80039b0 0x7f08b8003dd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f08c40098a0 tx=0x7f08b8005830 comp rx=0 tx=0).stop 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d1b1e640 1 -- 172.21.15.17:0/477882335 reap_dead start 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 -- 172.21.15.17:0/477882335 shutdown_connections 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 -- 172.21.15.17:0/477882335 >> 172.21.15.17:0/477882335 conn(0x7f08cc0ac570 msgr2=0x7f08cc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 -- 172.21.15.17:0/477882335 shutdown_connections 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.032+0000 7f08d3581640 1 -- 172.21.15.17:0/477882335 wait complete. 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 Processor -- start 2024-03-06T07:27:32.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 -- start start 2024-03-06T07:27:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 0x7f08cc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c10c0 0x7f08cc1e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0c5340 con 0x7f08cc0bf540 2024-03-06T07:27:32.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0c7c30 con 0x7f08b80039b0 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.033+0000 7f08d3581640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0c54c0 con 0x7f08cc0c10c0 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47158/0 (socket says 172.21.15.17:47158) 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 -- 172.21.15.17:0/2572211170 learned_addr learned my addr 172.21.15.17:0/2572211170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d0b1c640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c10c0 0x7f08cc1e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d131d640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 0x7f08cc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 -- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c10c0 msgr2=0x7f08cc1e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c10c0 0x7f08cc1e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 -- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 msgr2=0x7f08cc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 0x7f08cc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 -- 172.21.15.17:0/2572211170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0c0460 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d131d640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 0x7f08cc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d1b1e640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ad360 tx=0x7f08c800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8007a80 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c800ece0 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc0c0740 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.034+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8014670 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.035+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1ec060 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.035+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc04f330 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.036+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08c8014810 con 0x7f08cc0bf540 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.036+0000 7f08c1ffb640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 0x7f08b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:32.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.036+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08c808a0d0 con 0x7f08cc0bf540 2024-03-06T07:27:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.037+0000 7f08d131d640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 0x7f08b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.037+0000 7f08d131d640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 0x7f08b006d9c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f08c4035000 tx=0x7f08c4039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:32.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.041+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f08c8058e50 con 0x7f08cc0bf540 2024-03-06T07:27:32.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.289+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08cc0ca5d0 con 0x7f08b006b500 2024-03-06T07:27:32.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.296+0000 7f08c1ffb640 1 -- 172.21.15.17:0/2572211170 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247163 (secure 0 0 0) 0x7f08cc0ca5d0 con 0x7f08b006b500 2024-03-06T07:27:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 msgr2=0x7f08b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 0x7f08b006d9c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f08c4035000 tx=0x7f08c4039040 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 msgr2=0x7f08cc1e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ad360 tx=0x7f08c800e530 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 shutdown_connections 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f08b006b500 0x7f08b006d9c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08cc0bf540 0x7f08cc1e7ff0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08cc0c10c0 0x7f08cc1e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 --2- 172.21.15.17:0/2572211170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b80039b0 0x7f08cc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 >> 172.21.15.17:0/2572211170 conn(0x7f08cc0ac570 msgr2=0x7f08cc14d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 shutdown_connections 2024-03-06T07:27:32.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:32.299+0000 7f08d3581640 1 -- 172.21.15.17:0/2572211170 wait complete. 2024-03-06T07:27:32.311 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-03-06T07:27:31.441982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271944,"kb_used_data":18336,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754702776,"statfs":{"total":773094113280,"available":772815642624,"internally_reserved":0,"allocated":18776064,"data_stored":10277864,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.925093"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525776+0000","last_change":"2024-03-06T07:27:26.525776+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525776+0000","last_undegraded":"2024-03-06T07:27:26.525776+0000","last_fullsized":"2024-03-06T07:27:26.525776+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527532+0000","last_change":"2024-03-06T07:27:26.527532+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527532+0000","last_undegraded":"2024-03-06T07:27:26.527532+0000","last_fullsized":"2024-03-06T07:27:26.527532+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525851+0000","last_change":"2024-03-06T07:27:26.525851+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525851+0000","last_undegraded":"2024-03-06T07:27:26.525851+0000","last_fullsized":"2024-03-06T07:27:26.525851+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.531856+0000","last_change":"2024-03-06T07:27:26.531856+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.531856+0000","last_undegraded":"2024-03-06T07:27:26.531856+0000","last_fullsized":"2024-03-06T07:27:26.531856+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524917+0000","last_change":"2024-03-06T07:27:26.524917+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524917+0000","last_undegraded":"2024-03-06T07:27:26.524917+0000","last_fullsized":"2024-03-06T07:27:26.524917+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522367+0000","last_change":"2024-03-06T07:27:26.522367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522367+0000","last_undegraded":"2024-03-06T07:27:26.522367+0000","last_fullsized":"2024-03-06T07:27:26.522367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525635+0000","last_change":"2024-03-06T07:27:26.525635+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525635+0000","last_undegraded":"2024-03-06T07:27:26.525635+0000","last_fullsized":"2024-03-06T07:27:26.525635+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524185+0000","last_change":"2024-03-06T07:27:26.524185+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524185+0000","last_undegraded":"2024-03-06T07:27:26.524185+0000","last_fullsized":"2024-03-06T07:27:26.524185+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525644+0000","last_change":"2024-03-06T07:27:26.525644+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525644+0000","last_undegraded":"2024-03-06T07:27:26.525644+0000","last_fullsized":"2024-03-06T07:27:26.525644+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525171+0000","last_change":"2024-03-06T07:27:26.525171+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525171+0000","last_undegraded":"2024-03-06T07:27:26.525171+0000","last_fullsized":"2024-03-06T07:27:26.525171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.533216+0000","last_change":"2024-03-06T07:27:26.533216+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.533216+0000","last_undegraded":"2024-03-06T07:27:26.533216+0000","last_fullsized":"2024-03-06T07:27:26.533216+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525045+0000","last_change":"2024-03-06T07:27:26.525045+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525045+0000","last_undegraded":"2024-03-06T07:27:26.525045+0000","last_fullsized":"2024-03-06T07:27:26.525045+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523316+0000","last_change":"2024-03-06T07:27:26.523316+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523316+0000","last_undegraded":"2024-03-06T07:27:26.523316+0000","last_fullsized":"2024-03-06T07:27:26.523316+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524993+0000","last_change":"2024-03-06T07:27:26.524993+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524993+0000","last_undegraded":"2024-03-06T07:27:26.524993+0000","last_fullsized":"2024-03-06T07:27:26.524993+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522439+0000","last_change":"2024-03-06T07:27:26.522439+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522439+0000","last_undegraded":"2024-03-06T07:27:26.522439+0000","last_fullsized":"2024-03-06T07:27:26.522439+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523802+0000","last_change":"2024-03-06T07:27:26.523802+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523802+0000","last_undegraded":"2024-03-06T07:27:26.523802+0000","last_fullsized":"2024-03-06T07:27:26.523802+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524473+0000","last_change":"2024-03-06T07:27:26.524473+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524473+0000","last_undegraded":"2024-03-06T07:27:26.524473+0000","last_fullsized":"2024-03-06T07:27:26.524473+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524367+0000","last_change":"2024-03-06T07:27:26.524367+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524367+0000","last_undegraded":"2024-03-06T07:27:26.524367+0000","last_fullsized":"2024-03-06T07:27:26.524367+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526381+0000","last_change":"2024-03-06T07:27:26.526381+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526381+0000","last_undegraded":"2024-03-06T07:27:26.526381+0000","last_fullsized":"2024-03-06T07:27:26.526381+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524159+0000","last_change":"2024-03-06T07:27:26.524159+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524159+0000","last_undegraded":"2024-03-06T07:27:26.524159+0000","last_fullsized":"2024-03-06T07:27:26.524159+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523967+0000","last_change":"2024-03-06T07:27:26.523967+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523967+0000","last_undegraded":"2024-03-06T07:27:26.523967+0000","last_fullsized":"2024-03-06T07:27:26.523967+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523234+0000","last_change":"2024-03-06T07:27:26.523234+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523234+0000","last_undegraded":"2024-03-06T07:27:26.523234+0000","last_fullsized":"2024-03-06T07:27:26.523234+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524717+0000","last_change":"2024-03-06T07:27:26.524717+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524717+0000","last_undegraded":"2024-03-06T07:27:26.524717+0000","last_fullsized":"2024-03-06T07:27:26.524717+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524868+0000","last_change":"2024-03-06T07:27:26.524868+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524868+0000","last_undegraded":"2024-03-06T07:27:26.524868+0000","last_fullsized":"2024-03-06T07:27:26.524868+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525341+0000","last_change":"2024-03-06T07:27:26.525341+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525341+0000","last_undegraded":"2024-03-06T07:27:26.525341+0000","last_fullsized":"2024-03-06T07:27:26.525341+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523861+0000","last_change":"2024-03-06T07:27:26.523861+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523861+0000","last_undegraded":"2024-03-06T07:27:26.523861+0000","last_fullsized":"2024-03-06T07:27:26.523861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.532806+0000","last_change":"2024-03-06T07:27:26.532806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.532806+0000","last_undegraded":"2024-03-06T07:27:26.532806+0000","last_fullsized":"2024-03-06T07:27:26.532806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523869+0000","last_change":"2024-03-06T07:27:26.523869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523869+0000","last_undegraded":"2024-03-06T07:27:26.523869+0000","last_fullsized":"2024-03-06T07:27:26.523869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527024+0000","last_change":"2024-03-06T07:27:26.527024+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527024+0000","last_undegraded":"2024-03-06T07:27:26.527024+0000","last_fullsized":"2024-03-06T07:27:26.527024+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522869+0000","last_change":"2024-03-06T07:27:26.522869+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522869+0000","last_undegraded":"2024-03-06T07:27:26.522869+0000","last_fullsized":"2024-03-06T07:27:26.522869+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524162+0000","last_change":"2024-03-06T07:27:26.524162+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524162+0000","last_undegraded":"2024-03-06T07:27:26.524162+0000","last_fullsized":"2024-03-06T07:27:26.524162+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523410+0000","last_change":"2024-03-06T07:27:26.523410+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523410+0000","last_undegraded":"2024-03-06T07:27:26.523410+0000","last_fullsized":"2024-03-06T07:27:26.523410+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523603+0000","last_change":"2024-03-06T07:27:26.523603+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523603+0000","last_undegraded":"2024-03-06T07:27:26.523603+0000","last_fullsized":"2024-03-06T07:27:26.523603+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537037+0000","last_change":"2024-03-06T07:27:26.536435+0000","last_active":"2024-03-06T07:27:26.537037+0000","last_peered":"2024-03-06T07:27:26.537037+0000","last_clean":"2024-03-06T07:27:26.537037+0000","last_became_active":"2024-03-06T07:27:26.536143+0000","last_became_peered":"2024-03-06T07:27:26.536143+0000","last_unstale":"2024-03-06T07:27:26.537037+0000","last_undegraded":"2024-03-06T07:27:26.537037+0000","last_fullsized":"2024-03-06T07:27:26.537037+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:21:03.136170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522747+0000","last_change":"2024-03-06T07:27:26.522747+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522747+0000","last_undegraded":"2024-03-06T07:27:26.522747+0000","last_fullsized":"2024-03-06T07:27:26.522747+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541050+0000","last_change":"2024-03-06T07:27:26.535890+0000","last_active":"2024-03-06T07:27:26.541050+0000","last_peered":"2024-03-06T07:27:26.541050+0000","last_clean":"2024-03-06T07:27:26.541050+0000","last_became_active":"2024-03-06T07:27:26.535629+0000","last_became_peered":"2024-03-06T07:27:26.535629+0000","last_unstale":"2024-03-06T07:27:26.541050+0000","last_undegraded":"2024-03-06T07:27:26.541050+0000","last_fullsized":"2024-03-06T07:27:26.541050+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:23:07.272784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523859+0000","last_change":"2024-03-06T07:27:26.523859+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523859+0000","last_undegraded":"2024-03-06T07:27:26.523859+0000","last_fullsized":"2024-03-06T07:27:26.523859+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540965+0000","last_change":"2024-03-06T07:27:26.536426+0000","last_active":"2024-03-06T07:27:26.540965+0000","last_peered":"2024-03-06T07:27:26.540965+0000","last_clean":"2024-03-06T07:27:26.540965+0000","last_became_active":"2024-03-06T07:27:26.536145+0000","last_became_peered":"2024-03-06T07:27:26.536145+0000","last_unstale":"2024-03-06T07:27:26.540965+0000","last_undegraded":"2024-03-06T07:27:26.540965+0000","last_fullsized":"2024-03-06T07:27:26.540965+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:20.645286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522778+0000","last_change":"2024-03-06T07:27:26.522778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522778+0000","last_undegraded":"2024-03-06T07:27:26.522778+0000","last_fullsized":"2024-03-06T07:27:26.522778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533721+0000","last_change":"2024-03-06T07:27:26.531896+0000","last_active":"2024-03-06T07:27:26.533721+0000","last_peered":"2024-03-06T07:27:26.533721+0000","last_clean":"2024-03-06T07:27:26.533721+0000","last_became_active":"2024-03-06T07:27:26.531604+0000","last_became_peered":"2024-03-06T07:27:26.531604+0000","last_unstale":"2024-03-06T07:27:26.533721+0000","last_undegraded":"2024-03-06T07:27:26.533721+0000","last_fullsized":"2024-03-06T07:27:26.533721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:27.060493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523730+0000","last_change":"2024-03-06T07:27:26.523730+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523730+0000","last_undegraded":"2024-03-06T07:27:26.523730+0000","last_fullsized":"2024-03-06T07:27:26.523730+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541104+0000","last_change":"2024-03-06T07:27:26.537136+0000","last_active":"2024-03-06T07:27:26.541104+0000","last_peered":"2024-03-06T07:27:26.541104+0000","last_clean":"2024-03-06T07:27:26.541104+0000","last_became_active":"2024-03-06T07:27:26.536848+0000","last_became_peered":"2024-03-06T07:27:26.536848+0000","last_unstale":"2024-03-06T07:27:26.541104+0000","last_undegraded":"2024-03-06T07:27:26.541104+0000","last_fullsized":"2024-03-06T07:27:26.541104+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:05.557102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522813+0000","last_change":"2024-03-06T07:27:26.522813+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522813+0000","last_undegraded":"2024-03-06T07:27:26.522813+0000","last_fullsized":"2024-03-06T07:27:26.522813+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541500+0000","last_change":"2024-03-06T07:27:26.539023+0000","last_active":"2024-03-06T07:27:26.541500+0000","last_peered":"2024-03-06T07:27:26.541500+0000","last_clean":"2024-03-06T07:27:26.541500+0000","last_became_active":"2024-03-06T07:27:26.538724+0000","last_became_peered":"2024-03-06T07:27:26.538724+0000","last_unstale":"2024-03-06T07:27:26.541500+0000","last_undegraded":"2024-03-06T07:27:26.541500+0000","last_fullsized":"2024-03-06T07:27:26.541500+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524997+0000","last_change":"2024-03-06T07:27:26.524997+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524997+0000","last_undegraded":"2024-03-06T07:27:26.524997+0000","last_fullsized":"2024-03-06T07:27:26.524997+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.528810+0000","last_change":"2024-03-06T07:27:26.527218+0000","last_active":"2024-03-06T07:27:26.528810+0000","last_peered":"2024-03-06T07:27:26.528810+0000","last_clean":"2024-03-06T07:27:26.528810+0000","last_became_active":"2024-03-06T07:27:26.526206+0000","last_became_peered":"2024-03-06T07:27:26.526206+0000","last_unstale":"2024-03-06T07:27:26.528810+0000","last_undegraded":"2024-03-06T07:27:26.528810+0000","last_fullsized":"2024-03-06T07:27:26.528810+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:11:41.576679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522046+0000","last_change":"2024-03-06T07:27:26.522046+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522046+0000","last_undegraded":"2024-03-06T07:27:26.522046+0000","last_fullsized":"2024-03-06T07:27:26.522046+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543830+0000","last_change":"2024-03-06T07:27:26.542376+0000","last_active":"2024-03-06T07:27:26.543830+0000","last_peered":"2024-03-06T07:27:26.543830+0000","last_clean":"2024-03-06T07:27:26.543830+0000","last_became_active":"2024-03-06T07:27:26.542091+0000","last_became_peered":"2024-03-06T07:27:26.542091+0000","last_unstale":"2024-03-06T07:27:26.543830+0000","last_undegraded":"2024-03-06T07:27:26.543830+0000","last_fullsized":"2024-03-06T07:27:26.543830+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:37.673464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522321+0000","last_change":"2024-03-06T07:27:26.522321+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522321+0000","last_undegraded":"2024-03-06T07:27:26.522321+0000","last_fullsized":"2024-03-06T07:27:26.522321+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541606+0000","last_change":"2024-03-06T07:27:26.539198+0000","last_active":"2024-03-06T07:27:26.541606+0000","last_peered":"2024-03-06T07:27:26.541606+0000","last_clean":"2024-03-06T07:27:26.541606+0000","last_became_active":"2024-03-06T07:27:26.538712+0000","last_became_peered":"2024-03-06T07:27:26.538712+0000","last_unstale":"2024-03-06T07:27:26.541606+0000","last_undegraded":"2024-03-06T07:27:26.541606+0000","last_fullsized":"2024-03-06T07:27:26.541606+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:39.408799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522214+0000","last_change":"2024-03-06T07:27:26.522214+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522214+0000","last_undegraded":"2024-03-06T07:27:26.522214+0000","last_fullsized":"2024-03-06T07:27:26.522214+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541542+0000","last_change":"2024-03-06T07:27:26.539250+0000","last_active":"2024-03-06T07:27:26.541542+0000","last_peered":"2024-03-06T07:27:26.541542+0000","last_clean":"2024-03-06T07:27:26.541542+0000","last_became_active":"2024-03-06T07:27:26.538970+0000","last_became_peered":"2024-03-06T07:27:26.538970+0000","last_unstale":"2024-03-06T07:27:26.541542+0000","last_undegraded":"2024-03-06T07:27:26.541542+0000","last_fullsized":"2024-03-06T07:27:26.541542+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:30.347291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.534253+0000","last_change":"2024-03-06T07:27:26.534253+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.534253+0000","last_undegraded":"2024-03-06T07:27:26.534253+0000","last_fullsized":"2024-03-06T07:27:26.534253+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540930+0000","last_change":"2024-03-06T07:27:26.537269+0000","last_active":"2024-03-06T07:27:26.540930+0000","last_peered":"2024-03-06T07:27:26.540930+0000","last_clean":"2024-03-06T07:27:26.540930+0000","last_became_active":"2024-03-06T07:27:26.536990+0000","last_became_peered":"2024-03-06T07:27:26.536990+0000","last_unstale":"2024-03-06T07:27:26.540930+0000","last_undegraded":"2024-03-06T07:27:26.540930+0000","last_fullsized":"2024-03-06T07:27:26.540930+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:30.779629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526845+0000","last_change":"2024-03-06T07:27:26.526845+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526845+0000","last_undegraded":"2024-03-06T07:27:26.526845+0000","last_fullsized":"2024-03-06T07:27:26.526845+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536170+0000","last_change":"2024-03-06T07:27:26.533164+0000","last_active":"2024-03-06T07:27:26.536170+0000","last_peered":"2024-03-06T07:27:26.536170+0000","last_clean":"2024-03-06T07:27:26.536170+0000","last_became_active":"2024-03-06T07:27:26.532233+0000","last_became_peered":"2024-03-06T07:27:26.532233+0000","last_unstale":"2024-03-06T07:27:26.536170+0000","last_undegraded":"2024-03-06T07:27:26.536170+0000","last_fullsized":"2024-03-06T07:27:26.536170+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:12:25.296721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522687+0000","last_change":"2024-03-06T07:27:26.522687+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522687+0000","last_undegraded":"2024-03-06T07:27:26.522687+0000","last_fullsized":"2024-03-06T07:27:26.522687+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.537685+0000","last_change":"2024-03-06T07:27:26.536746+0000","last_active":"2024-03-06T07:27:26.537685+0000","last_peered":"2024-03-06T07:27:26.537685+0000","last_clean":"2024-03-06T07:27:26.537685+0000","last_became_active":"2024-03-06T07:27:26.536443+0000","last_became_peered":"2024-03-06T07:27:26.536443+0000","last_unstale":"2024-03-06T07:27:26.537685+0000","last_undegraded":"2024-03-06T07:27:26.537685+0000","last_fullsized":"2024-03-06T07:27:26.537685+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:03.817951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522240+0000","last_change":"2024-03-06T07:27:26.522240+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522240+0000","last_undegraded":"2024-03-06T07:27:26.522240+0000","last_fullsized":"2024-03-06T07:27:26.522240+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540837+0000","last_change":"2024-03-06T07:27:26.535007+0000","last_active":"2024-03-06T07:27:26.540837+0000","last_peered":"2024-03-06T07:27:26.540837+0000","last_clean":"2024-03-06T07:27:26.540837+0000","last_became_active":"2024-03-06T07:27:26.534742+0000","last_became_peered":"2024-03-06T07:27:26.534742+0000","last_unstale":"2024-03-06T07:27:26.540837+0000","last_undegraded":"2024-03-06T07:27:26.540837+0000","last_fullsized":"2024-03-06T07:27:26.540837+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:11.268111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522063+0000","last_change":"2024-03-06T07:27:26.522063+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522063+0000","last_undegraded":"2024-03-06T07:27:26.522063+0000","last_fullsized":"2024-03-06T07:27:26.522063+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543721+0000","last_change":"2024-03-06T07:27:26.539852+0000","last_active":"2024-03-06T07:27:26.543721+0000","last_peered":"2024-03-06T07:27:26.543721+0000","last_clean":"2024-03-06T07:27:26.543721+0000","last_became_active":"2024-03-06T07:27:26.539585+0000","last_became_peered":"2024-03-06T07:27:26.539585+0000","last_unstale":"2024-03-06T07:27:26.543721+0000","last_undegraded":"2024-03-06T07:27:26.543721+0000","last_fullsized":"2024-03-06T07:27:26.543721+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:34:02.711389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527806+0000","last_change":"2024-03-06T07:27:26.527806+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527806+0000","last_undegraded":"2024-03-06T07:27:26.527806+0000","last_fullsized":"2024-03-06T07:27:26.527806+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.530613+0000","last_change":"2024-03-06T07:27:26.529704+0000","last_active":"2024-03-06T07:27:26.530613+0000","last_peered":"2024-03-06T07:27:26.530613+0000","last_clean":"2024-03-06T07:27:26.530613+0000","last_became_active":"2024-03-06T07:27:26.529322+0000","last_became_peered":"2024-03-06T07:27:26.529322+0000","last_unstale":"2024-03-06T07:27:26.530613+0000","last_undegraded":"2024-03-06T07:27:26.530613+0000","last_fullsized":"2024-03-06T07:27:26.530613+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:08:23.114407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525947+0000","last_change":"2024-03-06T07:27:26.525947+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525947+0000","last_undegraded":"2024-03-06T07:27:26.525947+0000","last_fullsized":"2024-03-06T07:27:26.525947+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533414+0000","last_change":"2024-03-06T07:27:26.531502+0000","last_active":"2024-03-06T07:27:26.533414+0000","last_peered":"2024-03-06T07:27:26.533414+0000","last_clean":"2024-03-06T07:27:26.533414+0000","last_became_active":"2024-03-06T07:27:26.531216+0000","last_became_peered":"2024-03-06T07:27:26.531216+0000","last_unstale":"2024-03-06T07:27:26.533414+0000","last_undegraded":"2024-03-06T07:27:26.533414+0000","last_fullsized":"2024-03-06T07:27:26.533414+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:17:41.376525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521721+0000","last_change":"2024-03-06T07:27:26.521721+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521721+0000","last_undegraded":"2024-03-06T07:27:26.521721+0000","last_fullsized":"2024-03-06T07:27:26.521721+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546585+0000","last_change":"2024-03-06T07:27:26.541505+0000","last_active":"2024-03-06T07:27:26.546585+0000","last_peered":"2024-03-06T07:27:26.546585+0000","last_clean":"2024-03-06T07:27:26.546585+0000","last_became_active":"2024-03-06T07:27:26.541235+0000","last_became_peered":"2024-03-06T07:27:26.541235+0000","last_unstale":"2024-03-06T07:27:26.546585+0000","last_undegraded":"2024-03-06T07:27:26.546585+0000","last_fullsized":"2024-03-06T07:27:26.546585+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522464+0000","last_change":"2024-03-06T07:27:26.522464+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522464+0000","last_undegraded":"2024-03-06T07:27:26.522464+0000","last_fullsized":"2024-03-06T07:27:26.522464+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543780+0000","last_change":"2024-03-06T07:27:26.540712+0000","last_active":"2024-03-06T07:27:26.543780+0000","last_peered":"2024-03-06T07:27:26.543780+0000","last_clean":"2024-03-06T07:27:26.543780+0000","last_became_active":"2024-03-06T07:27:26.540456+0000","last_became_peered":"2024-03-06T07:27:26.540456+0000","last_unstale":"2024-03-06T07:27:26.543780+0000","last_undegraded":"2024-03-06T07:27:26.543780+0000","last_fullsized":"2024-03-06T07:27:26.543780+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:14:23.541795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522744+0000","last_change":"2024-03-06T07:27:26.522744+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522744+0000","last_undegraded":"2024-03-06T07:27:26.522744+0000","last_fullsized":"2024-03-06T07:27:26.522744+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535226+0000","last_change":"2024-03-06T07:27:26.530763+0000","last_active":"2024-03-06T07:27:26.535226+0000","last_peered":"2024-03-06T07:27:26.535226+0000","last_clean":"2024-03-06T07:27:26.535226+0000","last_became_active":"2024-03-06T07:27:26.530485+0000","last_became_peered":"2024-03-06T07:27:26.530485+0000","last_unstale":"2024-03-06T07:27:26.535226+0000","last_undegraded":"2024-03-06T07:27:26.535226+0000","last_fullsized":"2024-03-06T07:27:26.535226+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:06:22.926285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525008+0000","last_change":"2024-03-06T07:27:26.525008+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525008+0000","last_undegraded":"2024-03-06T07:27:26.525008+0000","last_fullsized":"2024-03-06T07:27:26.525008+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533839+0000","last_change":"2024-03-06T07:27:26.532686+0000","last_active":"2024-03-06T07:27:26.533839+0000","last_peered":"2024-03-06T07:27:26.533839+0000","last_clean":"2024-03-06T07:27:26.533839+0000","last_became_active":"2024-03-06T07:27:26.532416+0000","last_became_peered":"2024-03-06T07:27:26.532416+0000","last_unstale":"2024-03-06T07:27:26.533839+0000","last_undegraded":"2024-03-06T07:27:26.533839+0000","last_fullsized":"2024-03-06T07:27:26.533839+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:01.114509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.528494+0000","last_change":"2024-03-06T07:27:26.528494+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.528494+0000","last_undegraded":"2024-03-06T07:27:26.528494+0000","last_fullsized":"2024-03-06T07:27:26.528494+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.540337+0000","last_change":"2024-03-06T07:27:26.539907+0000","last_active":"2024-03-06T07:27:26.540337+0000","last_peered":"2024-03-06T07:27:26.540337+0000","last_clean":"2024-03-06T07:27:26.540337+0000","last_became_active":"2024-03-06T07:27:26.539575+0000","last_became_peered":"2024-03-06T07:27:26.539575+0000","last_unstale":"2024-03-06T07:27:26.540337+0000","last_undegraded":"2024-03-06T07:27:26.540337+0000","last_fullsized":"2024-03-06T07:27:26.540337+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:44:36.880373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521242+0000","last_change":"2024-03-06T07:27:26.521242+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521242+0000","last_undegraded":"2024-03-06T07:27:26.521242+0000","last_fullsized":"2024-03-06T07:27:26.521242+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.532649+0000","last_change":"2024-03-06T07:27:26.531365+0000","last_active":"2024-03-06T07:27:26.532649+0000","last_peered":"2024-03-06T07:27:26.532649+0000","last_clean":"2024-03-06T07:27:26.532649+0000","last_became_active":"2024-03-06T07:27:26.531079+0000","last_became_peered":"2024-03-06T07:27:26.531079+0000","last_unstale":"2024-03-06T07:27:26.532649+0000","last_undegraded":"2024-03-06T07:27:26.532649+0000","last_fullsized":"2024-03-06T07:27:26.532649+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:54:52.548111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527383+0000","last_change":"2024-03-06T07:27:26.527383+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527383+0000","last_undegraded":"2024-03-06T07:27:26.527383+0000","last_fullsized":"2024-03-06T07:27:26.527383+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.546609+0000","last_change":"2024-03-06T07:27:26.542508+0000","last_active":"2024-03-06T07:27:26.546609+0000","last_peered":"2024-03-06T07:27:26.546609+0000","last_clean":"2024-03-06T07:27:26.546609+0000","last_became_active":"2024-03-06T07:27:26.542231+0000","last_became_peered":"2024-03-06T07:27:26.542231+0000","last_unstale":"2024-03-06T07:27:26.546609+0000","last_undegraded":"2024-03-06T07:27:26.546609+0000","last_fullsized":"2024-03-06T07:27:26.546609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:24.664474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.525432+0000","last_change":"2024-03-06T07:27:26.525432+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.525432+0000","last_undegraded":"2024-03-06T07:27:26.525432+0000","last_fullsized":"2024-03-06T07:27:26.525432+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.539884+0000","last_change":"2024-03-06T07:27:26.539371+0000","last_active":"2024-03-06T07:27:26.539884+0000","last_peered":"2024-03-06T07:27:26.539884+0000","last_clean":"2024-03-06T07:27:26.539884+0000","last_became_active":"2024-03-06T07:27:26.539052+0000","last_became_peered":"2024-03-06T07:27:26.539052+0000","last_unstale":"2024-03-06T07:27:26.539884+0000","last_undegraded":"2024-03-06T07:27:26.539884+0000","last_fullsized":"2024-03-06T07:27:26.539884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:18.596305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.527778+0000","last_change":"2024-03-06T07:27:26.527778+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.527778+0000","last_undegraded":"2024-03-06T07:27:26.527778+0000","last_fullsized":"2024-03-06T07:27:26.527778+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.533750+0000","last_change":"2024-03-06T07:27:26.532169+0000","last_active":"2024-03-06T07:27:26.533750+0000","last_peered":"2024-03-06T07:27:26.533750+0000","last_clean":"2024-03-06T07:27:26.533750+0000","last_became_active":"2024-03-06T07:27:26.531854+0000","last_became_peered":"2024-03-06T07:27:26.531854+0000","last_unstale":"2024-03-06T07:27:26.533750+0000","last_undegraded":"2024-03-06T07:27:26.533750+0000","last_fullsized":"2024-03-06T07:27:26.533750+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:34.712123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.526066+0000","last_change":"2024-03-06T07:27:26.526066+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.526066+0000","last_undegraded":"2024-03-06T07:27:26.526066+0000","last_fullsized":"2024-03-06T07:27:26.526066+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535765+0000","last_change":"2024-03-06T07:27:26.534771+0000","last_active":"2024-03-06T07:27:26.535765+0000","last_peered":"2024-03-06T07:27:26.535765+0000","last_clean":"2024-03-06T07:27:26.535765+0000","last_became_active":"2024-03-06T07:27:26.534079+0000","last_became_peered":"2024-03-06T07:27:26.534079+0000","last_unstale":"2024-03-06T07:27:26.535765+0000","last_undegraded":"2024-03-06T07:27:26.535765+0000","last_fullsized":"2024-03-06T07:27:26.535765+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:17:28.965661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521229+0000","last_change":"2024-03-06T07:27:26.521229+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521229+0000","last_undegraded":"2024-03-06T07:27:26.521229+0000","last_fullsized":"2024-03-06T07:27:26.521229+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.535262+0000","last_change":"2024-03-06T07:27:26.532514+0000","last_active":"2024-03-06T07:27:26.535262+0000","last_peered":"2024-03-06T07:27:26.535262+0000","last_clean":"2024-03-06T07:27:26.535262+0000","last_became_active":"2024-03-06T07:27:26.532219+0000","last_became_peered":"2024-03-06T07:27:26.532219+0000","last_unstale":"2024-03-06T07:27:26.535262+0000","last_undegraded":"2024-03-06T07:27:26.535262+0000","last_fullsized":"2024-03-06T07:27:26.535262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:27:12.063304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.521463+0000","last_change":"2024-03-06T07:27:26.521463+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.521463+0000","last_undegraded":"2024-03-06T07:27:26.521463+0000","last_fullsized":"2024-03-06T07:27:26.521463+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541466+0000","last_change":"2024-03-06T07:27:26.538820+0000","last_active":"2024-03-06T07:27:26.541466+0000","last_peered":"2024-03-06T07:27:26.541466+0000","last_clean":"2024-03-06T07:27:26.541466+0000","last_became_active":"2024-03-06T07:27:26.538519+0000","last_became_peered":"2024-03-06T07:27:26.538519+0000","last_unstale":"2024-03-06T07:27:26.541466+0000","last_undegraded":"2024-03-06T07:27:26.541466+0000","last_fullsized":"2024-03-06T07:27:26.541466+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:11.757516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.524485+0000","last_change":"2024-03-06T07:27:26.524485+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.524485+0000","last_undegraded":"2024-03-06T07:27:26.524485+0000","last_fullsized":"2024-03-06T07:27:26.524485+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.541511+0000","last_change":"2024-03-06T07:27:26.539830+0000","last_active":"2024-03-06T07:27:26.541511+0000","last_peered":"2024-03-06T07:27:26.541511+0000","last_clean":"2024-03-06T07:27:26.541511+0000","last_became_active":"2024-03-06T07:27:26.539557+0000","last_became_peered":"2024-03-06T07:27:26.539557+0000","last_unstale":"2024-03-06T07:27:26.541511+0000","last_undegraded":"2024-03-06T07:27:26.541511+0000","last_fullsized":"2024-03-06T07:27:26.541511+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:23.412215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.522827+0000","last_change":"2024-03-06T07:27:26.522827+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.522827+0000","last_undegraded":"2024-03-06T07:27:26.522827+0000","last_fullsized":"2024-03-06T07:27:26.522827+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.543752+0000","last_change":"2024-03-06T07:27:26.540192+0000","last_active":"2024-03-06T07:27:26.543752+0000","last_peered":"2024-03-06T07:27:26.543752+0000","last_clean":"2024-03-06T07:27:26.543752+0000","last_became_active":"2024-03-06T07:27:26.539560+0000","last_became_peered":"2024-03-06T07:27:26.539560+0000","last_unstale":"2024-03-06T07:27:26.543752+0000","last_undegraded":"2024-03-06T07:27:26.543752+0000","last_fullsized":"2024-03-06T07:27:26.543752+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:31:57.625904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":1,"reported_epoch":184,"state":"creating+peering","last_fresh":"2024-03-06T07:27:26.523231+0000","last_change":"2024-03-06T07:27:26.523231+0000","last_active":"2024-03-06T07:27:26.513551+0000","last_peered":"2024-03-06T07:27:26.513551+0000","last_clean":"2024-03-06T07:27:26.513551+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-06T07:27:26.523231+0000","last_undegraded":"2024-03-06T07:27:26.523231+0000","last_fullsized":"2024-03-06T07:27:26.523231+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.536438+0000","last_change":"2024-03-06T07:27:26.534849+0000","last_active":"2024-03-06T07:27:26.536438+0000","last_peered":"2024-03-06T07:27:26.536438+0000","last_clean":"2024-03-06T07:27:26.536438+0000","last_became_active":"2024-03-06T07:27:26.534566+0000","last_became_peered":"2024-03-06T07:27:26.534566+0000","last_unstale":"2024-03-06T07:27:26.536438+0000","last_undegraded":"2024-03-06T07:27:26.536438+0000","last_fullsized":"2024-03-06T07:27:26.536438+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:46.210368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":472,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-03-06T07:27:26.526237+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:27:26.526237+0000","last_peered":"2024-03-06T07:27:26.526237+0000","last_clean":"2024-03-06T07:27:26.526237+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:27:26.526237+0000","last_undegraded":"2024-03-06T07:27:26.526237+0000","last_fullsized":"2024-03-06T07:27:26.526237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2330624,"data_stored":1137141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2396160,"data_stored":1727509,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:32.503 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:27:32.501+0000 7fe2a188a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:35.316 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:27:35.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 Processor -- start 2024-03-06T07:27:35.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 -- start start 2024-03-06T07:27:35.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e4070aa0 con 0x7f70e41506b0 2024-03-06T07:27:35.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e4153e10 con 0x7f70e40acc00 2024-03-06T07:27:35.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.416+0000 7f70ebf1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4156450 con 0x7f70e4153990 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70ea4bb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70ea4bb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56222/0 (socket says 172.21.15.17:56222) 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:48242/0 (socket says 172.21.15.17:48242) 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70ea4bb640 1 -- 172.21.15.17:0/1093649071 learned_addr learned my addr 172.21.15.17:0/1093649071 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 -- 172.21.15.17:0/1093649071 >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 msgr2=0x7f70e40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 --2- 172.21.15.17:0/1093649071 >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 -- 172.21.15.17:0/1093649071 >> v1:172.21.15.163:6789/0 conn(0x7f70e41506b0 legacy=0x7f70e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70e94b9640 1 -- 172.21.15.17:0/1093649071 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4070d80 con 0x7f70e4153990 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.417+0000 7f70ea4bb640 1 --2- 172.21.15.17:0/1093649071 >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.418+0000 7f70e94b9640 1 --2- 172.21.15.17:0/1093649071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70d40097a0 tx=0x7f70d402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972a9df701297500 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.418+0000 7f70e8cb8640 1 -- 172.21.15.17:0/1093649071 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4035020 con 0x7f70e4153990 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.418+0000 7f70e8cb8640 1 -- 172.21.15.17:0/1093649071 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4031830 con 0x7f70e4153990 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.418+0000 7f70e8cb8640 1 -- 172.21.15.17:0/1093649071 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4031c00 con 0x7f70e4153990 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 -- 172.21.15.17:0/1093649071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 msgr2=0x7f70e4156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/1093649071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f70d40097a0 tx=0x7f70d402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 -- 172.21.15.17:0/1093649071 shutdown_connections 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/1093649071 >> v2:172.21.15.163:3300/0 conn(0x7f70e40acc00 0x7f70e40c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/1093649071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4156ea0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 -- 172.21.15.17:0/1093649071 >> 172.21.15.17:0/1093649071 conn(0x7f70e406b1a0 msgr2=0x7f70e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 -- 172.21.15.17:0/1093649071 shutdown_connections 2024-03-06T07:27:35.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.419+0000 7f70ebf1e640 1 -- 172.21.15.17:0/1093649071 wait complete. 2024-03-06T07:27:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.420+0000 7f70ebf1e640 1 Processor -- start 2024-03-06T07:27:35.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.420+0000 7f70ebf1e640 1 -- start start 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 0x7f70e40c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40ad070 con 0x7f70e41506b0 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4156a10 con 0x7f70e40acc00 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ebf1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40ad1f0 con 0x7f70e4153990 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e9cba640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 0x7f70e40c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47172/0 (socket says 172.21.15.17:47172) 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 -- 172.21.15.17:0/702600685 learned_addr learned my addr 172.21.15.17:0/702600685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70ea4bb640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 -- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 msgr2=0x7f70e4149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 -- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 msgr2=0x7f70e40c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 0x7f70e40c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e94b9640 1 -- 172.21.15.17:0/702600685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40a8a60 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.421+0000 7f70e9cba640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 0x7f70e40c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70e94b9640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f70d402b910 tx=0x7f70d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4040070 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d403a930 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e40ad990 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4035040 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.422+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e4151520 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.423+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e404f330 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.423+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70d404b060 con 0x7f70e41506b0 2024-03-06T07:27:35.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.423+0000 7f70da7fc640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 0x7f70c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.424+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70d40abaa0 con 0x7f70e41506b0 2024-03-06T07:27:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.424+0000 7f70e9cba640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 0x7f70c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.424+0000 7f70e9cba640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 0x7f70c806d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f70e0001b70 tx=0x7f70e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.428+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70e40a8a60 con 0x7f70e41506b0 2024-03-06T07:27:35.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.660+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70e4156230 con 0x7f70c806b4d0 2024-03-06T07:27:35.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.666+0000 7f70da7fc640 1 -- 172.21.15.17:0/702600685 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252566 (secure 0 0 0) 0x7f70e4156230 con 0x7f70c806b4d0 2024-03-06T07:27:35.670 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:35.672 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:27:35.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 msgr2=0x7f70c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 0x7f70c806d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f70e0001b70 tx=0x7f70e0009040 comp rx=0 tx=0).stop 2024-03-06T07:27:35.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 msgr2=0x7f70e4149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f70d402b910 tx=0x7f70d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 shutdown_connections 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f70c806b4d0 0x7f70c806d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70e4153990 0x7f70e4149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e41506b0 0x7f70e4149a40 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 --2- 172.21.15.17:0/702600685 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70e40acc00 0x7f70e40c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 >> 172.21.15.17:0/702600685 conn(0x7f70e406b1a0 msgr2=0x7f70e414b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 shutdown_connections 2024-03-06T07:27:35.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.669+0000 7f70ebf1e640 1 -- 172.21.15.17:0/702600685 wait complete. 2024-03-06T07:27:35.681 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-03-06T07:27:35.443004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272236,"kb_used_data":18592,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754702484,"statfs":{"total":773094113280,"available":772815343616,"internally_reserved":0,"allocated":19038208,"data_stored":10393556,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":259575348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002181"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543556+0000","last_change":"2024-03-06T07:27:27.543327+0000","last_active":"2024-03-06T07:27:28.543556+0000","last_peered":"2024-03-06T07:27:28.543556+0000","last_clean":"2024-03-06T07:27:28.543556+0000","last_became_active":"2024-03-06T07:27:27.542841+0000","last_became_peered":"2024-03-06T07:27:27.542841+0000","last_unstale":"2024-03-06T07:27:28.543556+0000","last_undegraded":"2024-03-06T07:27:28.543556+0000","last_fullsized":"2024-03-06T07:27:28.543556+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:36.786997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016140+0000","last_change":"2024-03-06T07:27:27.543659+0000","last_active":"2024-03-06T07:27:29.016140+0000","last_peered":"2024-03-06T07:27:29.016140+0000","last_clean":"2024-03-06T07:27:29.016140+0000","last_became_active":"2024-03-06T07:27:27.543373+0000","last_became_peered":"2024-03-06T07:27:27.543373+0000","last_unstale":"2024-03-06T07:27:29.016140+0000","last_undegraded":"2024-03-06T07:27:29.016140+0000","last_fullsized":"2024-03-06T07:27:29.016140+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:27:44.090062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542674+0000","last_change":"2024-03-06T07:27:27.551616+0000","last_active":"2024-03-06T07:27:28.542674+0000","last_peered":"2024-03-06T07:27:28.542674+0000","last_clean":"2024-03-06T07:27:28.542674+0000","last_became_active":"2024-03-06T07:27:27.551087+0000","last_became_peered":"2024-03-06T07:27:27.551087+0000","last_unstale":"2024-03-06T07:27:28.542674+0000","last_undegraded":"2024-03-06T07:27:28.542674+0000","last_fullsized":"2024-03-06T07:27:28.542674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:49:56.611620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.544170+0000","last_change":"2024-03-06T07:27:27.540916+0000","last_active":"2024-03-06T07:27:28.544170+0000","last_peered":"2024-03-06T07:27:28.544170+0000","last_clean":"2024-03-06T07:27:28.544170+0000","last_became_active":"2024-03-06T07:27:27.540655+0000","last_became_peered":"2024-03-06T07:27:27.540655+0000","last_unstale":"2024-03-06T07:27:28.544170+0000","last_undegraded":"2024-03-06T07:27:28.544170+0000","last_fullsized":"2024-03-06T07:27:28.544170+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:38.141876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543145+0000","last_change":"2024-03-06T07:27:27.546475+0000","last_active":"2024-03-06T07:27:28.543145+0000","last_peered":"2024-03-06T07:27:28.543145+0000","last_clean":"2024-03-06T07:27:28.543145+0000","last_became_active":"2024-03-06T07:27:27.545545+0000","last_became_peered":"2024-03-06T07:27:27.545545+0000","last_unstale":"2024-03-06T07:27:28.543145+0000","last_undegraded":"2024-03-06T07:27:28.543145+0000","last_fullsized":"2024-03-06T07:27:28.543145+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:35:34.221749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244907+0000","last_change":"2024-03-06T07:27:27.545192+0000","last_active":"2024-03-06T07:27:29.244907+0000","last_peered":"2024-03-06T07:27:29.244907+0000","last_clean":"2024-03-06T07:27:29.244907+0000","last_became_active":"2024-03-06T07:27:27.544805+0000","last_became_peered":"2024-03-06T07:27:27.544805+0000","last_unstale":"2024-03-06T07:27:29.244907+0000","last_undegraded":"2024-03-06T07:27:29.244907+0000","last_fullsized":"2024-03-06T07:27:29.244907+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:28.064523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542360+0000","last_change":"2024-03-06T07:27:27.547270+0000","last_active":"2024-03-06T07:27:28.542360+0000","last_peered":"2024-03-06T07:27:28.542360+0000","last_clean":"2024-03-06T07:27:28.542360+0000","last_became_active":"2024-03-06T07:27:27.546940+0000","last_became_peered":"2024-03-06T07:27:27.546940+0000","last_unstale":"2024-03-06T07:27:28.542360+0000","last_undegraded":"2024-03-06T07:27:28.542360+0000","last_fullsized":"2024-03-06T07:27:28.542360+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:22:35.931288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244635+0000","last_change":"2024-03-06T07:27:27.547611+0000","last_active":"2024-03-06T07:27:29.244635+0000","last_peered":"2024-03-06T07:27:29.244635+0000","last_clean":"2024-03-06T07:27:29.244635+0000","last_became_active":"2024-03-06T07:27:27.547251+0000","last_became_peered":"2024-03-06T07:27:27.547251+0000","last_unstale":"2024-03-06T07:27:29.244635+0000","last_undegraded":"2024-03-06T07:27:29.244635+0000","last_fullsized":"2024-03-06T07:27:29.244635+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:33:10.664219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244677+0000","last_change":"2024-03-06T07:27:27.546735+0000","last_active":"2024-03-06T07:27:29.244677+0000","last_peered":"2024-03-06T07:27:29.244677+0000","last_clean":"2024-03-06T07:27:29.244677+0000","last_became_active":"2024-03-06T07:27:27.545219+0000","last_became_peered":"2024-03-06T07:27:27.545219+0000","last_unstale":"2024-03-06T07:27:29.244677+0000","last_undegraded":"2024-03-06T07:27:29.244677+0000","last_fullsized":"2024-03-06T07:27:29.244677+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:44:39.687519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542814+0000","last_change":"2024-03-06T07:27:27.547185+0000","last_active":"2024-03-06T07:27:28.542814+0000","last_peered":"2024-03-06T07:27:28.542814+0000","last_clean":"2024-03-06T07:27:28.542814+0000","last_became_active":"2024-03-06T07:27:27.546644+0000","last_became_peered":"2024-03-06T07:27:27.546644+0000","last_unstale":"2024-03-06T07:27:28.542814+0000","last_undegraded":"2024-03-06T07:27:28.542814+0000","last_fullsized":"2024-03-06T07:27:28.542814+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:05:57.928751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.544275+0000","last_change":"2024-03-06T07:27:27.541814+0000","last_active":"2024-03-06T07:27:28.544275+0000","last_peered":"2024-03-06T07:27:28.544275+0000","last_clean":"2024-03-06T07:27:28.544275+0000","last_became_active":"2024-03-06T07:27:27.541448+0000","last_became_peered":"2024-03-06T07:27:27.541448+0000","last_unstale":"2024-03-06T07:27:28.544275+0000","last_undegraded":"2024-03-06T07:27:28.544275+0000","last_fullsized":"2024-03-06T07:27:28.544275+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:55:02.279571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744516+0000","last_change":"2024-03-06T07:27:27.546980+0000","last_active":"2024-03-06T07:27:28.744516+0000","last_peered":"2024-03-06T07:27:28.744516+0000","last_clean":"2024-03-06T07:27:28.744516+0000","last_became_active":"2024-03-06T07:27:27.546540+0000","last_became_peered":"2024-03-06T07:27:27.546540+0000","last_unstale":"2024-03-06T07:27:28.744516+0000","last_undegraded":"2024-03-06T07:27:28.744516+0000","last_fullsized":"2024-03-06T07:27:28.744516+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:45:24.416245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244830+0000","last_change":"2024-03-06T07:27:27.545079+0000","last_active":"2024-03-06T07:27:29.244830+0000","last_peered":"2024-03-06T07:27:29.244830+0000","last_clean":"2024-03-06T07:27:29.244830+0000","last_became_active":"2024-03-06T07:27:27.544791+0000","last_became_peered":"2024-03-06T07:27:27.544791+0000","last_unstale":"2024-03-06T07:27:29.244830+0000","last_undegraded":"2024-03-06T07:27:29.244830+0000","last_fullsized":"2024-03-06T07:27:29.244830+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:34.538822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744280+0000","last_change":"2024-03-06T07:27:27.555634+0000","last_active":"2024-03-06T07:27:28.744280+0000","last_peered":"2024-03-06T07:27:28.744280+0000","last_clean":"2024-03-06T07:27:28.744280+0000","last_became_active":"2024-03-06T07:27:27.553952+0000","last_became_peered":"2024-03-06T07:27:27.553952+0000","last_unstale":"2024-03-06T07:27:28.744280+0000","last_undegraded":"2024-03-06T07:27:28.744280+0000","last_fullsized":"2024-03-06T07:27:28.744280+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:06.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545440+0000","last_change":"2024-03-06T07:27:27.544406+0000","last_active":"2024-03-06T07:27:28.545440+0000","last_peered":"2024-03-06T07:27:28.545440+0000","last_clean":"2024-03-06T07:27:28.545440+0000","last_became_active":"2024-03-06T07:27:27.544134+0000","last_became_peered":"2024-03-06T07:27:27.544134+0000","last_unstale":"2024-03-06T07:27:28.545440+0000","last_undegraded":"2024-03-06T07:27:28.545440+0000","last_fullsized":"2024-03-06T07:27:28.545440+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:36:39.826541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244793+0000","last_change":"2024-03-06T07:27:27.551751+0000","last_active":"2024-03-06T07:27:29.244793+0000","last_peered":"2024-03-06T07:27:29.244793+0000","last_clean":"2024-03-06T07:27:29.244793+0000","last_became_active":"2024-03-06T07:27:27.551491+0000","last_became_peered":"2024-03-06T07:27:27.551491+0000","last_unstale":"2024-03-06T07:27:29.244793+0000","last_undegraded":"2024-03-06T07:27:29.244793+0000","last_fullsized":"2024-03-06T07:27:29.244793+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:43.282852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744686+0000","last_change":"2024-03-06T07:27:27.550517+0000","last_active":"2024-03-06T07:27:28.744686+0000","last_peered":"2024-03-06T07:27:28.744686+0000","last_clean":"2024-03-06T07:27:28.744686+0000","last_became_active":"2024-03-06T07:27:27.550159+0000","last_became_peered":"2024-03-06T07:27:27.550159+0000","last_unstale":"2024-03-06T07:27:28.744686+0000","last_undegraded":"2024-03-06T07:27:28.744686+0000","last_fullsized":"2024-03-06T07:27:28.744686+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:29.383595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542945+0000","last_change":"2024-03-06T07:27:27.547018+0000","last_active":"2024-03-06T07:27:28.542945+0000","last_peered":"2024-03-06T07:27:28.542945+0000","last_clean":"2024-03-06T07:27:28.542945+0000","last_became_active":"2024-03-06T07:27:27.546540+0000","last_became_peered":"2024-03-06T07:27:27.546540+0000","last_unstale":"2024-03-06T07:27:28.542945+0000","last_undegraded":"2024-03-06T07:27:28.542945+0000","last_fullsized":"2024-03-06T07:27:28.542945+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:22:30.957451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543418+0000","last_change":"2024-03-06T07:27:27.546209+0000","last_active":"2024-03-06T07:27:28.543418+0000","last_peered":"2024-03-06T07:27:28.543418+0000","last_clean":"2024-03-06T07:27:28.543418+0000","last_became_active":"2024-03-06T07:27:27.545184+0000","last_became_peered":"2024-03-06T07:27:27.545184+0000","last_unstale":"2024-03-06T07:27:28.543418+0000","last_undegraded":"2024-03-06T07:27:28.543418+0000","last_fullsized":"2024-03-06T07:27:28.543418+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:19.980358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.245164+0000","last_change":"2024-03-06T07:27:27.547871+0000","last_active":"2024-03-06T07:27:29.245164+0000","last_peered":"2024-03-06T07:27:29.245164+0000","last_clean":"2024-03-06T07:27:29.245164+0000","last_became_active":"2024-03-06T07:27:27.547626+0000","last_became_peered":"2024-03-06T07:27:27.547626+0000","last_unstale":"2024-03-06T07:27:29.245164+0000","last_undegraded":"2024-03-06T07:27:29.245164+0000","last_fullsized":"2024-03-06T07:27:29.245164+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:15:19.600334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542456+0000","last_change":"2024-03-06T07:27:27.549736+0000","last_active":"2024-03-06T07:27:28.542456+0000","last_peered":"2024-03-06T07:27:28.542456+0000","last_clean":"2024-03-06T07:27:28.542456+0000","last_became_active":"2024-03-06T07:27:27.549451+0000","last_became_peered":"2024-03-06T07:27:27.549451+0000","last_unstale":"2024-03-06T07:27:28.542456+0000","last_undegraded":"2024-03-06T07:27:28.542456+0000","last_fullsized":"2024-03-06T07:27:28.542456+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:09.945084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545443+0000","last_change":"2024-03-06T07:27:27.552706+0000","last_active":"2024-03-06T07:27:28.545443+0000","last_peered":"2024-03-06T07:27:28.545443+0000","last_clean":"2024-03-06T07:27:28.545443+0000","last_became_active":"2024-03-06T07:27:27.552405+0000","last_became_peered":"2024-03-06T07:27:27.552405+0000","last_unstale":"2024-03-06T07:27:28.545443+0000","last_undegraded":"2024-03-06T07:27:28.545443+0000","last_fullsized":"2024-03-06T07:27:28.545443+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:07:20.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244409+0000","last_change":"2024-03-06T07:27:27.547804+0000","last_active":"2024-03-06T07:27:29.244409+0000","last_peered":"2024-03-06T07:27:29.244409+0000","last_clean":"2024-03-06T07:27:29.244409+0000","last_became_active":"2024-03-06T07:27:27.547500+0000","last_became_peered":"2024-03-06T07:27:27.547500+0000","last_unstale":"2024-03-06T07:27:29.244409+0000","last_undegraded":"2024-03-06T07:27:29.244409+0000","last_fullsized":"2024-03-06T07:27:29.244409+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:35.058716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542911+0000","last_change":"2024-03-06T07:27:27.553636+0000","last_active":"2024-03-06T07:27:28.542911+0000","last_peered":"2024-03-06T07:27:28.542911+0000","last_clean":"2024-03-06T07:27:28.542911+0000","last_became_active":"2024-03-06T07:27:27.553353+0000","last_became_peered":"2024-03-06T07:27:27.553353+0000","last_unstale":"2024-03-06T07:27:28.542911+0000","last_undegraded":"2024-03-06T07:27:28.542911+0000","last_fullsized":"2024-03-06T07:27:28.542911+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:07.485532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543873+0000","last_change":"2024-03-06T07:27:27.554705+0000","last_active":"2024-03-06T07:27:28.543873+0000","last_peered":"2024-03-06T07:27:28.543873+0000","last_clean":"2024-03-06T07:27:28.543873+0000","last_became_active":"2024-03-06T07:27:27.554420+0000","last_became_peered":"2024-03-06T07:27:27.554420+0000","last_unstale":"2024-03-06T07:27:28.543873+0000","last_undegraded":"2024-03-06T07:27:28.543873+0000","last_fullsized":"2024-03-06T07:27:28.543873+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:10:18.912915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543026+0000","last_change":"2024-03-06T07:27:27.553709+0000","last_active":"2024-03-06T07:27:28.543026+0000","last_peered":"2024-03-06T07:27:28.543026+0000","last_clean":"2024-03-06T07:27:28.543026+0000","last_became_active":"2024-03-06T07:27:27.553348+0000","last_became_peered":"2024-03-06T07:27:27.553348+0000","last_unstale":"2024-03-06T07:27:28.543026+0000","last_undegraded":"2024-03-06T07:27:28.543026+0000","last_fullsized":"2024-03-06T07:27:28.543026+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:37:22.021180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.544287+0000","last_change":"2024-03-06T07:27:27.543759+0000","last_active":"2024-03-06T07:27:28.544287+0000","last_peered":"2024-03-06T07:27:28.544287+0000","last_clean":"2024-03-06T07:27:28.544287+0000","last_became_active":"2024-03-06T07:27:27.543511+0000","last_became_peered":"2024-03-06T07:27:27.543511+0000","last_unstale":"2024-03-06T07:27:28.544287+0000","last_undegraded":"2024-03-06T07:27:28.544287+0000","last_fullsized":"2024-03-06T07:27:28.544287+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:23.934011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744566+0000","last_change":"2024-03-06T07:27:27.547575+0000","last_active":"2024-03-06T07:27:28.744566+0000","last_peered":"2024-03-06T07:27:28.744566+0000","last_clean":"2024-03-06T07:27:28.744566+0000","last_became_active":"2024-03-06T07:27:27.546188+0000","last_became_peered":"2024-03-06T07:27:27.546188+0000","last_unstale":"2024-03-06T07:27:28.744566+0000","last_undegraded":"2024-03-06T07:27:28.744566+0000","last_fullsized":"2024-03-06T07:27:28.744566+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:58:38.677494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016187+0000","last_change":"2024-03-06T07:27:27.537639+0000","last_active":"2024-03-06T07:27:29.016187+0000","last_peered":"2024-03-06T07:27:29.016187+0000","last_clean":"2024-03-06T07:27:29.016187+0000","last_became_active":"2024-03-06T07:27:27.537234+0000","last_became_peered":"2024-03-06T07:27:27.537234+0000","last_unstale":"2024-03-06T07:27:29.016187+0000","last_undegraded":"2024-03-06T07:27:29.016187+0000","last_fullsized":"2024-03-06T07:27:29.016187+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:58.430041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244756+0000","last_change":"2024-03-06T07:27:27.550898+0000","last_active":"2024-03-06T07:27:29.244756+0000","last_peered":"2024-03-06T07:27:29.244756+0000","last_clean":"2024-03-06T07:27:29.244756+0000","last_became_active":"2024-03-06T07:27:27.550569+0000","last_became_peered":"2024-03-06T07:27:27.550569+0000","last_unstale":"2024-03-06T07:27:29.244756+0000","last_undegraded":"2024-03-06T07:27:29.244756+0000","last_fullsized":"2024-03-06T07:27:29.244756+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:37.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244480+0000","last_change":"2024-03-06T07:27:27.545612+0000","last_active":"2024-03-06T07:27:29.244480+0000","last_peered":"2024-03-06T07:27:29.244480+0000","last_clean":"2024-03-06T07:27:29.244480+0000","last_became_active":"2024-03-06T07:27:27.545286+0000","last_became_peered":"2024-03-06T07:27:27.545286+0000","last_unstale":"2024-03-06T07:27:29.244480+0000","last_undegraded":"2024-03-06T07:27:29.244480+0000","last_fullsized":"2024-03-06T07:27:29.244480+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:25:14.778909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244928+0000","last_change":"2024-03-06T07:27:27.552708+0000","last_active":"2024-03-06T07:27:29.244928+0000","last_peered":"2024-03-06T07:27:29.244928+0000","last_clean":"2024-03-06T07:27:29.244928+0000","last_became_active":"2024-03-06T07:27:27.552435+0000","last_became_peered":"2024-03-06T07:27:27.552435+0000","last_unstale":"2024-03-06T07:27:29.244928+0000","last_undegraded":"2024-03-06T07:27:29.244928+0000","last_fullsized":"2024-03-06T07:27:29.244928+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:15:35.716152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542568+0000","last_change":"2024-03-06T07:27:27.547865+0000","last_active":"2024-03-06T07:27:28.542568+0000","last_peered":"2024-03-06T07:27:28.542568+0000","last_clean":"2024-03-06T07:27:28.542568+0000","last_became_active":"2024-03-06T07:27:27.547559+0000","last_became_peered":"2024-03-06T07:27:27.547559+0000","last_unstale":"2024-03-06T07:27:28.542568+0000","last_undegraded":"2024-03-06T07:27:28.542568+0000","last_fullsized":"2024-03-06T07:27:28.542568+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:51.592789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"185'4","reported_seq":18,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244676+0000","last_change":"2024-03-06T07:27:26.536435+0000","last_active":"2024-03-06T07:27:29.244676+0000","last_peered":"2024-03-06T07:27:29.244676+0000","last_clean":"2024-03-06T07:27:29.244676+0000","last_became_active":"2024-03-06T07:27:26.536143+0000","last_became_peered":"2024-03-06T07:27:26.536143+0000","last_unstale":"2024-03-06T07:27:29.244676+0000","last_undegraded":"2024-03-06T07:27:29.244676+0000","last_fullsized":"2024-03-06T07:27:29.244676+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:44:26.813216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542658+0000","last_change":"2024-03-06T07:27:27.545106+0000","last_active":"2024-03-06T07:27:28.542658+0000","last_peered":"2024-03-06T07:27:28.542658+0000","last_clean":"2024-03-06T07:27:28.542658+0000","last_became_active":"2024-03-06T07:27:27.544790+0000","last_became_peered":"2024-03-06T07:27:27.544790+0000","last_unstale":"2024-03-06T07:27:28.542658+0000","last_undegraded":"2024-03-06T07:27:28.542658+0000","last_fullsized":"2024-03-06T07:27:28.542658+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:32:37.878799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744939+0000","last_change":"2024-03-06T07:27:26.535890+0000","last_active":"2024-03-06T07:27:28.744939+0000","last_peered":"2024-03-06T07:27:28.744939+0000","last_clean":"2024-03-06T07:27:28.744939+0000","last_became_active":"2024-03-06T07:27:26.535629+0000","last_became_peered":"2024-03-06T07:27:26.535629+0000","last_unstale":"2024-03-06T07:27:28.744939+0000","last_undegraded":"2024-03-06T07:27:28.744939+0000","last_fullsized":"2024-03-06T07:27:28.744939+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:27.677772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744816+0000","last_change":"2024-03-06T07:27:27.547064+0000","last_active":"2024-03-06T07:27:28.744816+0000","last_peered":"2024-03-06T07:27:28.744816+0000","last_clean":"2024-03-06T07:27:28.744816+0000","last_became_active":"2024-03-06T07:27:27.546774+0000","last_became_peered":"2024-03-06T07:27:27.546774+0000","last_unstale":"2024-03-06T07:27:28.744816+0000","last_undegraded":"2024-03-06T07:27:28.744816+0000","last_fullsized":"2024-03-06T07:27:28.744816+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:37.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244858+0000","last_change":"2024-03-06T07:27:26.536426+0000","last_active":"2024-03-06T07:27:29.244858+0000","last_peered":"2024-03-06T07:27:29.244858+0000","last_clean":"2024-03-06T07:27:29.244858+0000","last_became_active":"2024-03-06T07:27:26.536145+0000","last_became_peered":"2024-03-06T07:27:26.536145+0000","last_unstale":"2024-03-06T07:27:29.244858+0000","last_undegraded":"2024-03-06T07:27:29.244858+0000","last_fullsized":"2024-03-06T07:27:29.244858+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:04.684179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542575+0000","last_change":"2024-03-06T07:27:27.546100+0000","last_active":"2024-03-06T07:27:28.542575+0000","last_peered":"2024-03-06T07:27:28.542575+0000","last_clean":"2024-03-06T07:27:28.542575+0000","last_became_active":"2024-03-06T07:27:27.545769+0000","last_became_peered":"2024-03-06T07:27:27.545769+0000","last_unstale":"2024-03-06T07:27:28.542575+0000","last_undegraded":"2024-03-06T07:27:28.542575+0000","last_fullsized":"2024-03-06T07:27:28.542575+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:49:39.035110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545614+0000","last_change":"2024-03-06T07:27:26.531896+0000","last_active":"2024-03-06T07:27:28.545614+0000","last_peered":"2024-03-06T07:27:28.545614+0000","last_clean":"2024-03-06T07:27:28.545614+0000","last_became_active":"2024-03-06T07:27:26.531604+0000","last_became_peered":"2024-03-06T07:27:26.531604+0000","last_unstale":"2024-03-06T07:27:28.545614+0000","last_undegraded":"2024-03-06T07:27:28.545614+0000","last_fullsized":"2024-03-06T07:27:28.545614+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:30.512880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744105+0000","last_change":"2024-03-06T07:27:27.551150+0000","last_active":"2024-03-06T07:27:28.744105+0000","last_peered":"2024-03-06T07:27:28.744105+0000","last_clean":"2024-03-06T07:27:28.744105+0000","last_became_active":"2024-03-06T07:27:27.550843+0000","last_became_peered":"2024-03-06T07:27:27.550843+0000","last_unstale":"2024-03-06T07:27:28.744105+0000","last_undegraded":"2024-03-06T07:27:28.744105+0000","last_fullsized":"2024-03-06T07:27:28.744105+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:03.104992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543118+0000","last_change":"2024-03-06T07:27:26.537136+0000","last_active":"2024-03-06T07:27:28.543118+0000","last_peered":"2024-03-06T07:27:28.543118+0000","last_clean":"2024-03-06T07:27:28.543118+0000","last_became_active":"2024-03-06T07:27:26.536848+0000","last_became_peered":"2024-03-06T07:27:26.536848+0000","last_unstale":"2024-03-06T07:27:28.543118+0000","last_undegraded":"2024-03-06T07:27:28.543118+0000","last_fullsized":"2024-03-06T07:27:28.543118+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:30:36.828396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545480+0000","last_change":"2024-03-06T07:27:27.552029+0000","last_active":"2024-03-06T07:27:28.545480+0000","last_peered":"2024-03-06T07:27:28.545480+0000","last_clean":"2024-03-06T07:27:28.545480+0000","last_became_active":"2024-03-06T07:27:27.551751+0000","last_became_peered":"2024-03-06T07:27:27.551751+0000","last_unstale":"2024-03-06T07:27:28.545480+0000","last_undegraded":"2024-03-06T07:27:28.545480+0000","last_fullsized":"2024-03-06T07:27:28.545480+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:24:36.382269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543686+0000","last_change":"2024-03-06T07:27:26.539023+0000","last_active":"2024-03-06T07:27:28.543686+0000","last_peered":"2024-03-06T07:27:28.543686+0000","last_clean":"2024-03-06T07:27:28.543686+0000","last_became_active":"2024-03-06T07:27:26.538724+0000","last_became_peered":"2024-03-06T07:27:26.538724+0000","last_unstale":"2024-03-06T07:27:28.543686+0000","last_undegraded":"2024-03-06T07:27:28.543686+0000","last_fullsized":"2024-03-06T07:27:28.543686+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:17.942822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244542+0000","last_change":"2024-03-06T07:27:27.549644+0000","last_active":"2024-03-06T07:27:29.244542+0000","last_peered":"2024-03-06T07:27:29.244542+0000","last_clean":"2024-03-06T07:27:29.244542+0000","last_became_active":"2024-03-06T07:27:27.549376+0000","last_became_peered":"2024-03-06T07:27:27.549376+0000","last_unstale":"2024-03-06T07:27:29.244542+0000","last_undegraded":"2024-03-06T07:27:29.244542+0000","last_fullsized":"2024-03-06T07:27:29.244542+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:50:14.851701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542563+0000","last_change":"2024-03-06T07:27:26.527218+0000","last_active":"2024-03-06T07:27:28.542563+0000","last_peered":"2024-03-06T07:27:28.542563+0000","last_clean":"2024-03-06T07:27:28.542563+0000","last_became_active":"2024-03-06T07:27:26.526206+0000","last_became_peered":"2024-03-06T07:27:26.526206+0000","last_unstale":"2024-03-06T07:27:28.542563+0000","last_undegraded":"2024-03-06T07:27:28.542563+0000","last_fullsized":"2024-03-06T07:27:28.542563+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:58:49.601950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545625+0000","last_change":"2024-03-06T07:27:27.554315+0000","last_active":"2024-03-06T07:27:28.545625+0000","last_peered":"2024-03-06T07:27:28.545625+0000","last_clean":"2024-03-06T07:27:28.545625+0000","last_became_active":"2024-03-06T07:27:27.554039+0000","last_became_peered":"2024-03-06T07:27:27.554039+0000","last_unstale":"2024-03-06T07:27:28.545625+0000","last_undegraded":"2024-03-06T07:27:28.545625+0000","last_fullsized":"2024-03-06T07:27:28.545625+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:00.635639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543747+0000","last_change":"2024-03-06T07:27:26.542376+0000","last_active":"2024-03-06T07:27:28.543747+0000","last_peered":"2024-03-06T07:27:28.543747+0000","last_clean":"2024-03-06T07:27:28.543747+0000","last_became_active":"2024-03-06T07:27:26.542091+0000","last_became_peered":"2024-03-06T07:27:26.542091+0000","last_unstale":"2024-03-06T07:27:28.543747+0000","last_undegraded":"2024-03-06T07:27:28.543747+0000","last_fullsized":"2024-03-06T07:27:28.543747+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:28.379565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545730+0000","last_change":"2024-03-06T07:27:27.554724+0000","last_active":"2024-03-06T07:27:28.545730+0000","last_peered":"2024-03-06T07:27:28.545730+0000","last_clean":"2024-03-06T07:27:28.545730+0000","last_became_active":"2024-03-06T07:27:27.554417+0000","last_became_peered":"2024-03-06T07:27:27.554417+0000","last_unstale":"2024-03-06T07:27:28.545730+0000","last_undegraded":"2024-03-06T07:27:28.545730+0000","last_fullsized":"2024-03-06T07:27:28.545730+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:41.220039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.544362+0000","last_change":"2024-03-06T07:27:26.539198+0000","last_active":"2024-03-06T07:27:28.544362+0000","last_peered":"2024-03-06T07:27:28.544362+0000","last_clean":"2024-03-06T07:27:28.544362+0000","last_became_active":"2024-03-06T07:27:26.538712+0000","last_became_peered":"2024-03-06T07:27:26.538712+0000","last_unstale":"2024-03-06T07:27:28.544362+0000","last_undegraded":"2024-03-06T07:27:28.544362+0000","last_fullsized":"2024-03-06T07:27:28.544362+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:11:15.741619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543086+0000","last_change":"2024-03-06T07:27:27.552530+0000","last_active":"2024-03-06T07:27:28.543086+0000","last_peered":"2024-03-06T07:27:28.543086+0000","last_clean":"2024-03-06T07:27:28.543086+0000","last_became_active":"2024-03-06T07:27:27.552244+0000","last_became_peered":"2024-03-06T07:27:27.552244+0000","last_unstale":"2024-03-06T07:27:28.543086+0000","last_undegraded":"2024-03-06T07:27:28.543086+0000","last_fullsized":"2024-03-06T07:27:28.543086+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:41:34.639541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"185'3","reported_seq":17,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543530+0000","last_change":"2024-03-06T07:27:26.539250+0000","last_active":"2024-03-06T07:27:28.543530+0000","last_peered":"2024-03-06T07:27:28.543530+0000","last_clean":"2024-03-06T07:27:28.543530+0000","last_became_active":"2024-03-06T07:27:26.538970+0000","last_became_peered":"2024-03-06T07:27:26.538970+0000","last_unstale":"2024-03-06T07:27:28.543530+0000","last_undegraded":"2024-03-06T07:27:28.543530+0000","last_fullsized":"2024-03-06T07:27:28.543530+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:42.278822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.544487+0000","last_change":"2024-03-06T07:27:27.539425+0000","last_active":"2024-03-06T07:27:28.544487+0000","last_peered":"2024-03-06T07:27:28.544487+0000","last_clean":"2024-03-06T07:27:28.544487+0000","last_became_active":"2024-03-06T07:27:27.539155+0000","last_became_peered":"2024-03-06T07:27:27.539155+0000","last_unstale":"2024-03-06T07:27:28.544487+0000","last_undegraded":"2024-03-06T07:27:28.544487+0000","last_fullsized":"2024-03-06T07:27:28.544487+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:34:46.687041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"185'6","reported_seq":20,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.245091+0000","last_change":"2024-03-06T07:27:26.537269+0000","last_active":"2024-03-06T07:27:29.245091+0000","last_peered":"2024-03-06T07:27:29.245091+0000","last_clean":"2024-03-06T07:27:29.245091+0000","last_became_active":"2024-03-06T07:27:26.536990+0000","last_became_peered":"2024-03-06T07:27:26.536990+0000","last_unstale":"2024-03-06T07:27:29.245091+0000","last_undegraded":"2024-03-06T07:27:29.245091+0000","last_fullsized":"2024-03-06T07:27:29.245091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:23:24.442079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.015983+0000","last_change":"2024-03-06T07:27:27.536740+0000","last_active":"2024-03-06T07:27:29.015983+0000","last_peered":"2024-03-06T07:27:29.015983+0000","last_clean":"2024-03-06T07:27:29.015983+0000","last_became_active":"2024-03-06T07:27:27.536442+0000","last_became_peered":"2024-03-06T07:27:27.536442+0000","last_unstale":"2024-03-06T07:27:29.015983+0000","last_undegraded":"2024-03-06T07:27:29.015983+0000","last_fullsized":"2024-03-06T07:27:29.015983+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:00.635639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744351+0000","last_change":"2024-03-06T07:27:26.533164+0000","last_active":"2024-03-06T07:27:28.744351+0000","last_peered":"2024-03-06T07:27:28.744351+0000","last_clean":"2024-03-06T07:27:28.744351+0000","last_became_active":"2024-03-06T07:27:26.532233+0000","last_became_peered":"2024-03-06T07:27:26.532233+0000","last_unstale":"2024-03-06T07:27:28.744351+0000","last_undegraded":"2024-03-06T07:27:28.744351+0000","last_fullsized":"2024-03-06T07:27:28.744351+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:04:39.408799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744146+0000","last_change":"2024-03-06T07:27:27.546691+0000","last_active":"2024-03-06T07:27:28.744146+0000","last_peered":"2024-03-06T07:27:28.744146+0000","last_clean":"2024-03-06T07:27:28.744146+0000","last_became_active":"2024-03-06T07:27:27.546412+0000","last_became_peered":"2024-03-06T07:27:27.546412+0000","last_unstale":"2024-03-06T07:27:28.744146+0000","last_undegraded":"2024-03-06T07:27:28.744146+0000","last_fullsized":"2024-03-06T07:27:28.744146+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:49.588078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016331+0000","last_change":"2024-03-06T07:27:26.536746+0000","last_active":"2024-03-06T07:27:29.016331+0000","last_peered":"2024-03-06T07:27:29.016331+0000","last_clean":"2024-03-06T07:27:29.016331+0000","last_became_active":"2024-03-06T07:27:26.536443+0000","last_became_peered":"2024-03-06T07:27:26.536443+0000","last_unstale":"2024-03-06T07:27:29.016331+0000","last_undegraded":"2024-03-06T07:27:29.016331+0000","last_fullsized":"2024-03-06T07:27:29.016331+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:59:35.782968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542417+0000","last_change":"2024-03-06T07:27:27.549176+0000","last_active":"2024-03-06T07:27:28.542417+0000","last_peered":"2024-03-06T07:27:28.542417+0000","last_clean":"2024-03-06T07:27:28.542417+0000","last_became_active":"2024-03-06T07:27:27.548220+0000","last_became_peered":"2024-03-06T07:27:27.548220+0000","last_unstale":"2024-03-06T07:27:28.542417+0000","last_undegraded":"2024-03-06T07:27:28.542417+0000","last_fullsized":"2024-03-06T07:27:28.542417+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:44:24.328642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244547+0000","last_change":"2024-03-06T07:27:26.535007+0000","last_active":"2024-03-06T07:27:29.244547+0000","last_peered":"2024-03-06T07:27:29.244547+0000","last_clean":"2024-03-06T07:27:29.244547+0000","last_became_active":"2024-03-06T07:27:26.534742+0000","last_became_peered":"2024-03-06T07:27:26.534742+0000","last_unstale":"2024-03-06T07:27:29.244547+0000","last_undegraded":"2024-03-06T07:27:29.244547+0000","last_fullsized":"2024-03-06T07:27:29.244547+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:02.030286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545487+0000","last_change":"2024-03-06T07:27:27.548385+0000","last_active":"2024-03-06T07:27:28.545487+0000","last_peered":"2024-03-06T07:27:28.545487+0000","last_clean":"2024-03-06T07:27:28.545487+0000","last_became_active":"2024-03-06T07:27:27.547856+0000","last_became_peered":"2024-03-06T07:27:27.547856+0000","last_unstale":"2024-03-06T07:27:28.545487+0000","last_undegraded":"2024-03-06T07:27:28.545487+0000","last_fullsized":"2024-03-06T07:27:28.545487+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:00.527367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"185'8","reported_seq":22,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744079+0000","last_change":"2024-03-06T07:27:26.539852+0000","last_active":"2024-03-06T07:27:28.744079+0000","last_peered":"2024-03-06T07:27:28.744079+0000","last_clean":"2024-03-06T07:27:28.744079+0000","last_became_active":"2024-03-06T07:27:26.539585+0000","last_became_peered":"2024-03-06T07:27:26.539585+0000","last_unstale":"2024-03-06T07:27:28.744079+0000","last_undegraded":"2024-03-06T07:27:28.744079+0000","last_fullsized":"2024-03-06T07:27:28.744079+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:31:42.710497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016178+0000","last_change":"2024-03-06T07:27:27.548423+0000","last_active":"2024-03-06T07:27:29.016178+0000","last_peered":"2024-03-06T07:27:29.016178+0000","last_clean":"2024-03-06T07:27:29.016178+0000","last_became_active":"2024-03-06T07:27:27.548112+0000","last_became_peered":"2024-03-06T07:27:27.548112+0000","last_unstale":"2024-03-06T07:27:29.016178+0000","last_undegraded":"2024-03-06T07:27:29.016178+0000","last_fullsized":"2024-03-06T07:27:29.016178+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:32:41.220039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542872+0000","last_change":"2024-03-06T07:27:26.529704+0000","last_active":"2024-03-06T07:27:28.542872+0000","last_peered":"2024-03-06T07:27:28.542872+0000","last_clean":"2024-03-06T07:27:28.542872+0000","last_became_active":"2024-03-06T07:27:26.529322+0000","last_became_peered":"2024-03-06T07:27:26.529322+0000","last_unstale":"2024-03-06T07:27:28.542872+0000","last_undegraded":"2024-03-06T07:27:28.542872+0000","last_fullsized":"2024-03-06T07:27:28.542872+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:55:24.665378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543387+0000","last_change":"2024-03-06T07:27:27.543913+0000","last_active":"2024-03-06T07:27:28.543387+0000","last_peered":"2024-03-06T07:27:28.543387+0000","last_clean":"2024-03-06T07:27:28.543387+0000","last_became_active":"2024-03-06T07:27:27.543629+0000","last_became_peered":"2024-03-06T07:27:27.543629+0000","last_unstale":"2024-03-06T07:27:28.543387+0000","last_undegraded":"2024-03-06T07:27:28.543387+0000","last_fullsized":"2024-03-06T07:27:28.543387+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:13:31.516910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244524+0000","last_change":"2024-03-06T07:27:26.531502+0000","last_active":"2024-03-06T07:27:29.244524+0000","last_peered":"2024-03-06T07:27:29.244524+0000","last_clean":"2024-03-06T07:27:29.244524+0000","last_became_active":"2024-03-06T07:27:26.531216+0000","last_became_peered":"2024-03-06T07:27:26.531216+0000","last_unstale":"2024-03-06T07:27:29.244524+0000","last_undegraded":"2024-03-06T07:27:29.244524+0000","last_fullsized":"2024-03-06T07:27:29.244524+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:06:04.043545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543156+0000","last_change":"2024-03-06T07:27:27.546192+0000","last_active":"2024-03-06T07:27:28.543156+0000","last_peered":"2024-03-06T07:27:28.543156+0000","last_clean":"2024-03-06T07:27:28.543156+0000","last_became_active":"2024-03-06T07:27:27.545906+0000","last_became_peered":"2024-03-06T07:27:27.545906+0000","last_unstale":"2024-03-06T07:27:28.543156+0000","last_undegraded":"2024-03-06T07:27:28.543156+0000","last_fullsized":"2024-03-06T07:27:28.543156+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:43:14.505843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"185'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744616+0000","last_change":"2024-03-06T07:27:26.541505+0000","last_active":"2024-03-06T07:27:28.744616+0000","last_peered":"2024-03-06T07:27:28.744616+0000","last_clean":"2024-03-06T07:27:28.744616+0000","last_became_active":"2024-03-06T07:27:26.541235+0000","last_became_peered":"2024-03-06T07:27:26.541235+0000","last_unstale":"2024-03-06T07:27:28.744616+0000","last_undegraded":"2024-03-06T07:27:28.744616+0000","last_fullsized":"2024-03-06T07:27:28.744616+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:43:17.942822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744768+0000","last_change":"2024-03-06T07:27:27.553521+0000","last_active":"2024-03-06T07:27:28.744768+0000","last_peered":"2024-03-06T07:27:28.744768+0000","last_clean":"2024-03-06T07:27:28.744768+0000","last_became_active":"2024-03-06T07:27:27.553242+0000","last_became_peered":"2024-03-06T07:27:27.553242+0000","last_unstale":"2024-03-06T07:27:28.744768+0000","last_undegraded":"2024-03-06T07:27:28.744768+0000","last_fullsized":"2024-03-06T07:27:28.744768+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:33:10.664219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744729+0000","last_change":"2024-03-06T07:27:26.540712+0000","last_active":"2024-03-06T07:27:28.744729+0000","last_peered":"2024-03-06T07:27:28.744729+0000","last_clean":"2024-03-06T07:27:28.744729+0000","last_became_active":"2024-03-06T07:27:26.540456+0000","last_became_peered":"2024-03-06T07:27:26.540456+0000","last_unstale":"2024-03-06T07:27:28.744729+0000","last_undegraded":"2024-03-06T07:27:28.744729+0000","last_fullsized":"2024-03-06T07:27:28.744729+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:28:17.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744217+0000","last_change":"2024-03-06T07:27:27.550365+0000","last_active":"2024-03-06T07:27:28.744217+0000","last_peered":"2024-03-06T07:27:28.744217+0000","last_clean":"2024-03-06T07:27:28.744217+0000","last_became_active":"2024-03-06T07:27:27.550061+0000","last_became_peered":"2024-03-06T07:27:27.550061+0000","last_unstale":"2024-03-06T07:27:28.744217+0000","last_undegraded":"2024-03-06T07:27:28.744217+0000","last_fullsized":"2024-03-06T07:27:28.744217+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:38:47.214397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744489+0000","last_change":"2024-03-06T07:27:26.530763+0000","last_active":"2024-03-06T07:27:28.744489+0000","last_peered":"2024-03-06T07:27:28.744489+0000","last_clean":"2024-03-06T07:27:28.744489+0000","last_became_active":"2024-03-06T07:27:26.530485+0000","last_became_peered":"2024-03-06T07:27:26.530485+0000","last_unstale":"2024-03-06T07:27:28.744489+0000","last_undegraded":"2024-03-06T07:27:28.744489+0000","last_fullsized":"2024-03-06T07:27:28.744489+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:49.384144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543165+0000","last_change":"2024-03-06T07:27:27.544708+0000","last_active":"2024-03-06T07:27:28.543165+0000","last_peered":"2024-03-06T07:27:28.543165+0000","last_clean":"2024-03-06T07:27:28.543165+0000","last_became_active":"2024-03-06T07:27:27.544427+0000","last_became_peered":"2024-03-06T07:27:27.544427+0000","last_unstale":"2024-03-06T07:27:28.543165+0000","last_undegraded":"2024-03-06T07:27:28.543165+0000","last_fullsized":"2024-03-06T07:27:28.543165+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:09.686217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545557+0000","last_change":"2024-03-06T07:27:26.532686+0000","last_active":"2024-03-06T07:27:28.545557+0000","last_peered":"2024-03-06T07:27:28.545557+0000","last_clean":"2024-03-06T07:27:28.545557+0000","last_became_active":"2024-03-06T07:27:26.532416+0000","last_became_peered":"2024-03-06T07:27:26.532416+0000","last_unstale":"2024-03-06T07:27:28.545557+0000","last_undegraded":"2024-03-06T07:27:28.545557+0000","last_fullsized":"2024-03-06T07:27:28.545557+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:28:08.682187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016211+0000","last_change":"2024-03-06T07:27:27.549768+0000","last_active":"2024-03-06T07:27:29.016211+0000","last_peered":"2024-03-06T07:27:29.016211+0000","last_clean":"2024-03-06T07:27:29.016211+0000","last_became_active":"2024-03-06T07:27:27.549500+0000","last_became_peered":"2024-03-06T07:27:27.549500+0000","last_unstale":"2024-03-06T07:27:29.016211+0000","last_undegraded":"2024-03-06T07:27:29.016211+0000","last_fullsized":"2024-03-06T07:27:29.016211+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:05:12.799582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545428+0000","last_change":"2024-03-06T07:27:26.539907+0000","last_active":"2024-03-06T07:27:28.545428+0000","last_peered":"2024-03-06T07:27:28.545428+0000","last_clean":"2024-03-06T07:27:28.545428+0000","last_became_active":"2024-03-06T07:27:26.539575+0000","last_became_peered":"2024-03-06T07:27:26.539575+0000","last_unstale":"2024-03-06T07:27:28.545428+0000","last_undegraded":"2024-03-06T07:27:28.545428+0000","last_fullsized":"2024-03-06T07:27:28.545428+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:48:06.481503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542583+0000","last_change":"2024-03-06T07:27:27.553976+0000","last_active":"2024-03-06T07:27:28.542583+0000","last_peered":"2024-03-06T07:27:28.542583+0000","last_clean":"2024-03-06T07:27:28.542583+0000","last_became_active":"2024-03-06T07:27:27.553689+0000","last_became_peered":"2024-03-06T07:27:27.553689+0000","last_unstale":"2024-03-06T07:27:28.542583+0000","last_undegraded":"2024-03-06T07:27:28.542583+0000","last_fullsized":"2024-03-06T07:27:28.542583+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:21:41.861593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244335+0000","last_change":"2024-03-06T07:27:26.531365+0000","last_active":"2024-03-06T07:27:29.244335+0000","last_peered":"2024-03-06T07:27:29.244335+0000","last_clean":"2024-03-06T07:27:29.244335+0000","last_became_active":"2024-03-06T07:27:26.531079+0000","last_became_peered":"2024-03-06T07:27:26.531079+0000","last_unstale":"2024-03-06T07:27:29.244335+0000","last_undegraded":"2024-03-06T07:27:29.244335+0000","last_fullsized":"2024-03-06T07:27:29.244335+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:30:29.214656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.016289+0000","last_change":"2024-03-06T07:27:27.555188+0000","last_active":"2024-03-06T07:27:29.016289+0000","last_peered":"2024-03-06T07:27:29.016289+0000","last_clean":"2024-03-06T07:27:29.016289+0000","last_became_active":"2024-03-06T07:27:27.554916+0000","last_became_peered":"2024-03-06T07:27:27.554916+0000","last_unstale":"2024-03-06T07:27:29.016289+0000","last_undegraded":"2024-03-06T07:27:29.016289+0000","last_fullsized":"2024-03-06T07:27:29.016289+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:11:42.580709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744860+0000","last_change":"2024-03-06T07:27:26.542508+0000","last_active":"2024-03-06T07:27:28.744860+0000","last_peered":"2024-03-06T07:27:28.744860+0000","last_clean":"2024-03-06T07:27:28.744860+0000","last_became_active":"2024-03-06T07:27:26.542231+0000","last_became_peered":"2024-03-06T07:27:26.542231+0000","last_unstale":"2024-03-06T07:27:28.744860+0000","last_undegraded":"2024-03-06T07:27:28.744860+0000","last_fullsized":"2024-03-06T07:27:28.744860+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:37.137846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543550+0000","last_change":"2024-03-06T07:27:27.547313+0000","last_active":"2024-03-06T07:27:28.543550+0000","last_peered":"2024-03-06T07:27:28.543550+0000","last_clean":"2024-03-06T07:27:28.543550+0000","last_became_active":"2024-03-06T07:27:27.545936+0000","last_became_peered":"2024-03-06T07:27:27.545936+0000","last_unstale":"2024-03-06T07:27:28.543550+0000","last_undegraded":"2024-03-06T07:27:28.543550+0000","last_fullsized":"2024-03-06T07:27:28.543550+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:04.821981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"186'1","reported_seq":15,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545790+0000","last_change":"2024-03-06T07:27:26.539371+0000","last_active":"2024-03-06T07:27:28.545790+0000","last_peered":"2024-03-06T07:27:28.545790+0000","last_clean":"2024-03-06T07:27:28.545790+0000","last_became_active":"2024-03-06T07:27:26.539052+0000","last_became_peered":"2024-03-06T07:27:26.539052+0000","last_unstale":"2024-03-06T07:27:28.545790+0000","last_undegraded":"2024-03-06T07:27:28.545790+0000","last_fullsized":"2024-03-06T07:27:28.545790+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:35:33.217719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.245078+0000","last_change":"2024-03-06T07:27:27.550848+0000","last_active":"2024-03-06T07:27:29.245078+0000","last_peered":"2024-03-06T07:27:29.245078+0000","last_clean":"2024-03-06T07:27:29.245078+0000","last_became_active":"2024-03-06T07:27:27.550580+0000","last_became_peered":"2024-03-06T07:27:27.550580+0000","last_unstale":"2024-03-06T07:27:29.245078+0000","last_undegraded":"2024-03-06T07:27:29.245078+0000","last_fullsized":"2024-03-06T07:27:29.245078+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:02.118539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.545709+0000","last_change":"2024-03-06T07:27:26.532169+0000","last_active":"2024-03-06T07:27:28.545709+0000","last_peered":"2024-03-06T07:27:28.545709+0000","last_clean":"2024-03-06T07:27:28.545709+0000","last_became_active":"2024-03-06T07:27:26.531854+0000","last_became_peered":"2024-03-06T07:27:26.531854+0000","last_unstale":"2024-03-06T07:27:28.545709+0000","last_undegraded":"2024-03-06T07:27:28.545709+0000","last_fullsized":"2024-03-06T07:27:28.545709+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:20:48.508718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244580+0000","last_change":"2024-03-06T07:27:27.547027+0000","last_active":"2024-03-06T07:27:29.244580+0000","last_peered":"2024-03-06T07:27:29.244580+0000","last_clean":"2024-03-06T07:27:29.244580+0000","last_became_active":"2024-03-06T07:27:27.545535+0000","last_became_peered":"2024-03-06T07:27:27.545535+0000","last_unstale":"2024-03-06T07:27:29.244580+0000","last_undegraded":"2024-03-06T07:27:29.244580+0000","last_fullsized":"2024-03-06T07:27:29.244580+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:48:07.752281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744738+0000","last_change":"2024-03-06T07:27:26.534771+0000","last_active":"2024-03-06T07:27:28.744738+0000","last_peered":"2024-03-06T07:27:28.744738+0000","last_clean":"2024-03-06T07:27:28.744738+0000","last_became_active":"2024-03-06T07:27:26.534079+0000","last_became_peered":"2024-03-06T07:27:26.534079+0000","last_unstale":"2024-03-06T07:27:28.744738+0000","last_undegraded":"2024-03-06T07:27:28.744738+0000","last_fullsized":"2024-03-06T07:27:28.744738+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:53.089259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542355+0000","last_change":"2024-03-06T07:27:27.550343+0000","last_active":"2024-03-06T07:27:28.542355+0000","last_peered":"2024-03-06T07:27:28.542355+0000","last_clean":"2024-03-06T07:27:28.542355+0000","last_became_active":"2024-03-06T07:27:27.550061+0000","last_became_peered":"2024-03-06T07:27:27.550061+0000","last_unstale":"2024-03-06T07:27:28.542355+0000","last_undegraded":"2024-03-06T07:27:28.542355+0000","last_fullsized":"2024-03-06T07:27:28.542355+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:14.459905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744502+0000","last_change":"2024-03-06T07:27:26.532514+0000","last_active":"2024-03-06T07:27:28.744502+0000","last_peered":"2024-03-06T07:27:28.744502+0000","last_clean":"2024-03-06T07:27:28.744502+0000","last_became_active":"2024-03-06T07:27:26.532219+0000","last_became_peered":"2024-03-06T07:27:26.532219+0000","last_unstale":"2024-03-06T07:27:28.744502+0000","last_undegraded":"2024-03-06T07:27:28.744502+0000","last_fullsized":"2024-03-06T07:27:28.744502+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:11:15.741619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542393+0000","last_change":"2024-03-06T07:27:27.545560+0000","last_active":"2024-03-06T07:27:28.542393+0000","last_peered":"2024-03-06T07:27:28.542393+0000","last_clean":"2024-03-06T07:27:28.542393+0000","last_became_active":"2024-03-06T07:27:27.545086+0000","last_became_peered":"2024-03-06T07:27:27.545086+0000","last_unstale":"2024-03-06T07:27:28.542393+0000","last_undegraded":"2024-03-06T07:27:28.542393+0000","last_fullsized":"2024-03-06T07:27:28.542393+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:48:17.470721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543941+0000","last_change":"2024-03-06T07:27:26.538820+0000","last_active":"2024-03-06T07:27:28.543941+0000","last_peered":"2024-03-06T07:27:28.543941+0000","last_clean":"2024-03-06T07:27:28.543941+0000","last_became_active":"2024-03-06T07:27:26.538519+0000","last_became_peered":"2024-03-06T07:27:26.538519+0000","last_unstale":"2024-03-06T07:27:28.543941+0000","last_undegraded":"2024-03-06T07:27:28.543941+0000","last_fullsized":"2024-03-06T07:27:28.543941+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:08:49.384144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:29.244527+0000","last_change":"2024-03-06T07:27:27.552439+0000","last_active":"2024-03-06T07:27:29.244527+0000","last_peered":"2024-03-06T07:27:29.244527+0000","last_clean":"2024-03-06T07:27:29.244527+0000","last_became_active":"2024-03-06T07:27:27.552106+0000","last_became_peered":"2024-03-06T07:27:27.552106+0000","last_unstale":"2024-03-06T07:27:29.244527+0000","last_undegraded":"2024-03-06T07:27:29.244527+0000","last_fullsized":"2024-03-06T07:27:29.244527+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:25:01.475846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543019+0000","last_change":"2024-03-06T07:27:26.539830+0000","last_active":"2024-03-06T07:27:28.543019+0000","last_peered":"2024-03-06T07:27:28.543019+0000","last_clean":"2024-03-06T07:27:28.543019+0000","last_became_active":"2024-03-06T07:27:26.539557+0000","last_became_peered":"2024-03-06T07:27:26.539557+0000","last_unstale":"2024-03-06T07:27:28.543019+0000","last_undegraded":"2024-03-06T07:27:28.543019+0000","last_fullsized":"2024-03-06T07:27:28.543019+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:33:09.660190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744372+0000","last_change":"2024-03-06T07:27:27.543723+0000","last_active":"2024-03-06T07:27:28.744372+0000","last_peered":"2024-03-06T07:27:28.744372+0000","last_clean":"2024-03-06T07:27:28.744372+0000","last_became_active":"2024-03-06T07:27:27.543449+0000","last_became_peered":"2024-03-06T07:27:27.543449+0000","last_unstale":"2024-03-06T07:27:28.744372+0000","last_undegraded":"2024-03-06T07:27:28.744372+0000","last_fullsized":"2024-03-06T07:27:28.744372+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:35:31.351321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.744246+0000","last_change":"2024-03-06T07:27:26.540192+0000","last_active":"2024-03-06T07:27:28.744246+0000","last_peered":"2024-03-06T07:27:28.744246+0000","last_clean":"2024-03-06T07:27:28.744246+0000","last_became_active":"2024-03-06T07:27:26.539560+0000","last_became_peered":"2024-03-06T07:27:26.539560+0000","last_unstale":"2024-03-06T07:27:28.744246+0000","last_undegraded":"2024-03-06T07:27:28.744246+0000","last_fullsized":"2024-03-06T07:27:28.744246+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:55.110495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542820+0000","last_change":"2024-03-06T07:27:27.552561+0000","last_active":"2024-03-06T07:27:28.542820+0000","last_peered":"2024-03-06T07:27:28.542820+0000","last_clean":"2024-03-06T07:27:28.542820+0000","last_became_active":"2024-03-06T07:27:27.552278+0000","last_became_peered":"2024-03-06T07:27:27.552278+0000","last_unstale":"2024-03-06T07:27:28.542820+0000","last_undegraded":"2024-03-06T07:27:28.542820+0000","last_fullsized":"2024-03-06T07:27:28.542820+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:26.513551+0000","last_clean_scrub_stamp":"2024-03-06T07:27:26.513551+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:13:35.483684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"185'2","reported_seq":16,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.543257+0000","last_change":"2024-03-06T07:27:26.534849+0000","last_active":"2024-03-06T07:27:28.543257+0000","last_peered":"2024-03-06T07:27:28.543257+0000","last_clean":"2024-03-06T07:27:28.543257+0000","last_became_active":"2024-03-06T07:27:26.534566+0000","last_became_peered":"2024-03-06T07:27:26.534566+0000","last_unstale":"2024-03-06T07:27:28.543257+0000","last_undegraded":"2024-03-06T07:27:28.543257+0000","last_fullsized":"2024-03-06T07:27:28.543257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:27:25.509522+0000","last_clean_scrub_stamp":"2024-03-06T07:27:25.509522+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:14:33.534792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":476,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-03-06T07:27:28.542890+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:27:28.542890+0000","last_peered":"2024-03-06T07:27:28.542890+0000","last_clean":"2024-03-06T07:27:28.542890+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:27:28.542890+0000","last_undegraded":"2024-03-06T07:27:28.542890+0000","last_fullsized":"2024-03-06T07:27:28.542890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2428928,"data_stored":1742040,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33508,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338332,"statfs":{"total":96636764160,"available":96602451968,"internally_reserved":0,"allocated":2379776,"data_stored":1152738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33496,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338344,"statfs":{"total":96636764160,"available":96602464256,"internally_reserved":0,"allocated":2371584,"data_stored":1152682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2359296,"data_stored":1151090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":2359296,"data_stored":1151022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33496,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338344,"statfs":{"total":96636764160,"available":96602464256,"internally_reserved":0,"allocated":2363392,"data_stored":1151616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33484,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338356,"statfs":{"total":96636764160,"available":96602476544,"internally_reserved":0,"allocated":2355200,"data_stored":1151000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":2364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":2420736,"data_stored":1741368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:27:35.685 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:27:35.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.827+0000 7f069125c640 1 Processor -- start 2024-03-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.827+0000 7f069125c640 1 -- start start 2024-03-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f069125c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f068c0a8290 0x7f068c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f069125c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 0x7f068c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f069125c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c0c62a0 con 0x7f068c0c69a0 2024-03-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f069125c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c153f00 con 0x7f068c0a8290 2024-03-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f069125c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0706e0 con 0x7f068c0c4740 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f068c0a8290 0x7f068c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068a59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 0x7f068c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068ad9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f068c0a8290 0x7f068c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:56238/0 (socket says 172.21.15.17:56238) 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068b59e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f068c0c69a0 0x7f068c072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:40792/0 (socket says 172.21.15.17:40792) 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068a59c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 0x7f068c0701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:48254/0 (socket says 172.21.15.17:48254) 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.828+0000 7f068ad9d640 1 -- 172.21.15.17:0/1350697177 learned_addr learned my addr 172.21.15.17:0/1350697177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.829+0000 7f068ad9d640 1 -- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 msgr2=0x7f068c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.829+0000 7f068ad9d640 1 --2- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 0x7f068c0701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.829+0000 7f068ad9d640 1 -- 172.21.15.17:0/1350697177 >> v1:172.21.15.163:6789/0 conn(0x7f068c0c69a0 legacy=0x7f068c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.829+0000 7f068ad9d640 1 -- 172.21.15.17:0/1350697177 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0c4c00 con 0x7f068c0a8290 2024-03-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f068ad9d640 1 --2- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0a8290 0x7f068c1539c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f067c009e50 tx=0x7f067c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d132d9457dfe28e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f0689d9b640 1 -- 172.21.15.17:0/1350697177 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c02c9e0 con 0x7f068c0a8290 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f0689d9b640 1 -- 172.21.15.17:0/1350697177 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c02cb80 con 0x7f068c0a8290 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f0689d9b640 1 -- 172.21.15.17:0/1350697177 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c0343f0 con 0x7f068c0a8290 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 -- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0a8290 msgr2=0x7f068c1539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 --2- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0a8290 0x7f068c1539c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f067c009e50 tx=0x7f067c02be40 comp rx=0 tx=0).stop 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 -- 172.21.15.17:0/1350697177 shutdown_connections 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 --2- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0c4740 0x7f068c0701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 --2- 172.21.15.17:0/1350697177 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0a8290 0x7f068c1539c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 -- 172.21.15.17:0/1350697177 >> 172.21.15.17:0/1350697177 conn(0x7f068c0c5560 msgr2=0x7f068c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 -- 172.21.15.17:0/1350697177 shutdown_connections 2024-03-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.830+0000 7f069125c640 1 -- 172.21.15.17:0/1350697177 wait complete. 2024-03-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 Processor -- start 2024-03-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 -- start start 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0a8290 0x7f068c1ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0c4740 0x7f068c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c06fc80 con 0x7f068c0c69a0 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c06f9a0 con 0x7f068c0c4740 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f069125c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c072d90 con 0x7f068c0a8290 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.832+0000 7f068b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47186/0 (socket says 172.21.15.17:47186) 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 -- 172.21.15.17:0/2088221783 learned_addr learned my addr 172.21.15.17:0/2088221783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068a59c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0c4740 0x7f068c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068ad9d640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0a8290 0x7f068c1ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 -- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0a8290 msgr2=0x7f068c1ec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0a8290 0x7f068c1ec4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 -- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0c4740 msgr2=0x7f068c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0c4740 0x7f068c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 -- 172.21.15.17:0/2088221783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c1eca20 con 0x7f068c0c69a0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f068b59e640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f068400c390 tx=0x7f068400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840074e0 con 0x7f068c0c69a0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0684007680 con 0x7f068c0c69a0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c14b010 con 0x7f068c0c69a0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840049e0 con 0x7f068c0c69a0 2024-03-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.833+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c156f30 con 0x7f068c0c69a0 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.834+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c04f330 con 0x7f068c0c69a0 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.835+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f068401a020 con 0x7f068c0c69a0 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.835+0000 7f06737fe640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 0x7f067806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.835+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0684089a60 con 0x7f068c0c69a0 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.836+0000 7f068ad9d640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 0x7f067806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.836+0000 7f068ad9d640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 0x7f067806d9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f067c035000 tx=0x7f067c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:35.844+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0684010300 con 0x7f068c0c69a0 2024-03-06T07:27:36.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.094+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f068c0c8480 con 0x7f068c0c69a0 2024-03-06T07:27:36.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.096+0000 7f06737fe640 1 -- 172.21.15.17:0/2088221783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f06840587e0 con 0x7f068c0c69a0 2024-03-06T07:27:36.097 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:27:36.097 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:27:36.097 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:27:36.098 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:27:36.098 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:27:36.098 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 msgr2=0x7f067806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 0x7f067806d9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f067c035000 tx=0x7f067c030000 comp rx=0 tx=0).stop 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 msgr2=0x7f068c14f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f068400c390 tx=0x7f068400c850 comp rx=0 tx=0).stop 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 shutdown_connections 2024-03-06T07:27:36.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f067806b500 0x7f067806d9c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f068c0c69a0 0x7f068c14f190 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f068c0c4740 0x7f068c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 --2- 172.21.15.17:0/2088221783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f068c0a8290 0x7f068c1ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 >> 172.21.15.17:0/2088221783 conn(0x7f068c0c5560 msgr2=0x7f068c1527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 shutdown_connections 2024-03-06T07:27:36.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.098+0000 7f069125c640 1 -- 172.21.15.17:0/2088221783 wait complete. 2024-03-06T07:27:36.111 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:36.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.212+0000 7f5885ab5640 1 Processor -- start 2024-03-06T07:27:36.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.212+0000 7f5885ab5640 1 -- start start 2024-03-06T07:27:36.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f5885ab5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58800c7380 0x7f58800c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f5885ab5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880072630 con 0x7f588006da40 2024-03-06T07:27:36.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f5885ab5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5880072910 con 0x7f58800c6780 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f5885ab5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588006de20 con 0x7f58800c7380 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f587ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f58800c7380 0x7f58800c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f587f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f588006da40 0x7f5880155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54568/0 (socket says 172.21.15.17:54568) 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.213+0000 7f587f7fe640 1 -- 172.21.15.17:0/1460592797 learned_addr learned my addr 172.21.15.17:0/1460592797 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.214+0000 7f587ffff640 1 -- 172.21.15.17:0/1460592797 >> v1:172.21.15.163:6789/0 conn(0x7f58800c6780 legacy=0x7f5880158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.214+0000 7f587ffff640 1 -- 172.21.15.17:0/1460592797 >> v1:172.21.15.17:6789/0 conn(0x7f588006da40 legacy=0x7f5880155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.214+0000 7f587ffff640 1 -- 172.21.15.17:0/1460592797 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880072bf0 con 0x7f58800c7380 2024-03-06T07:27:36.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.214+0000 7f587e7fc640 1 -- 172.21.15.17:0/1460592797 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953382497 0 0) 0x7f5880072630 con 0x7f588006da40 2024-03-06T07:27:36.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.214+0000 7f587ffff640 1 --2- 172.21.15.17:0/1460592797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800c7380 0x7f58800c3280 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f5878009d90 tx=0x7f587802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed779e98dae180f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f587e7fc640 1 -- 172.21.15.17:0/1460592797 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878039070 con 0x7f58800c7380 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f587e7fc640 1 -- 172.21.15.17:0/1460592797 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587802cbd0 con 0x7f58800c7380 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f587e7fc640 1 -- 172.21.15.17:0/1460592797 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878034440 con 0x7f58800c7380 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 -- 172.21.15.17:0/1460592797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800c7380 msgr2=0x7f58800c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 --2- 172.21.15.17:0/1460592797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800c7380 0x7f58800c3280 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f5878009d90 tx=0x7f587802be20 comp rx=0 tx=0).stop 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 -- 172.21.15.17:0/1460592797 shutdown_connections 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 --2- 172.21.15.17:0/1460592797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58800c7380 0x7f58800c3280 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 -- 172.21.15.17:0/1460592797 >> 172.21.15.17:0/1460592797 conn(0x7f58800aa630 msgr2=0x7f58800aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 -- 172.21.15.17:0/1460592797 shutdown_connections 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.215+0000 7f5885ab5640 1 -- 172.21.15.17:0/1460592797 wait complete. 2024-03-06T07:27:36.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.216+0000 7f5885ab5640 1 Processor -- start 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.216+0000 7f5885ab5640 1 -- start start 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58800c6780 0x7f58801ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58800c7380 0x7f58800abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c37c0 con 0x7f588006da40 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58801594a0 con 0x7f58800c7380 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f5885ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c3940 con 0x7f58800c6780 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47190/0 (socket says 172.21.15.17:47190) 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 -- 172.21.15.17:0/3042719203 learned_addr learned my addr 172.21.15.17:0/3042719203 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587effd640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58800c6780 0x7f58801ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587ffff640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58800c7380 0x7f58800abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 -- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58800c6780 msgr2=0x7f58801ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58800c6780 0x7f58801ee7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 -- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58800c7380 msgr2=0x7f58800abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.217+0000 7f587f7fe640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58800c7380 0x7f58800abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f587f7fe640 1 -- 172.21.15.17:0/3042719203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58801df780 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f587f7fe640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f586800c430 tx=0x7f586800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58680074e0 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58680047e0 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58801ef000 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868004bd0 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.218+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58801eed00 con 0x7f588006da40 2024-03-06T07:27:36.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.219+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588004f330 con 0x7f588006da40 2024-03-06T07:27:36.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.219+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5868011070 con 0x7f588006da40 2024-03-06T07:27:36.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.220+0000 7f5877fff640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 0x7f586006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.220+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f586808a060 con 0x7f588006da40 2024-03-06T07:27:36.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.220+0000 7f587effd640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 0x7f586006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.221+0000 7f587effd640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 0x7f586006d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f58800c5880 tx=0x7f5870008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.228+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f586800e280 con 0x7f588006da40 2024-03-06T07:27:36.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.460+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58800c8150 con 0x7f588006da40 2024-03-06T07:27:36.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.460+0000 7f5877fff640 1 -- 172.21.15.17:0/3042719203 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+4990 (secure 0 0 0) 0x7f5868058cd0 con 0x7f588006da40 2024-03-06T07:27:36.462 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:36.462 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":173,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:28.902635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9900,"qdb_cluster":[9900]},"id":14}]} 2024-03-06T07:27:36.462 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 173 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 msgr2=0x7f586006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 0x7f586006d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f58800c5880 tx=0x7f5870008040 comp rx=0 tx=0).stop 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 msgr2=0x7f58800ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f586800c430 tx=0x7f586800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 shutdown_connections 2024-03-06T07:27:36.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f586006b4d0 0x7f586006d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f58800c7380 0x7f58800abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f58800c6780 0x7f58801ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 --2- 172.21.15.17:0/3042719203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f588006da40 0x7f58800ae2f0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 >> 172.21.15.17:0/3042719203 conn(0x7f58800aa630 msgr2=0x7f58800aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 shutdown_connections 2024-03-06T07:27:36.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.462+0000 7f5885ab5640 1 -- 172.21.15.17:0/3042719203 wait complete. 2024-03-06T07:27:36.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 173, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:28.902635+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9900, 'qdb_cluster': [9900]} 2024-03-06T07:27:36.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:27:36.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:27:36.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.978+0000 7fd38e4ed640 1 Processor -- start 2024-03-06T07:27:36.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.978+0000 7fd38e4ed640 1 -- start start 2024-03-06T07:27:36.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd38e4ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3880c8950 0x7fd3880c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd38e4ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd388155130 con 0x7fd3880c69f0 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd38e4ed640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880ad480 con 0x7fd3881558d0 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd38e4ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd388155440 con 0x7fd3880c8950 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd3877fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd3880c8950 0x7fd3880c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd387fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd3880c69f0 0x7fd3880acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54584/0 (socket says 172.21.15.17:54584) 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.979+0000 7fd387fff640 1 -- 172.21.15.17:0/3536047963 learned_addr learned my addr 172.21.15.17:0/3536047963 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.980+0000 7fd3877fe640 1 -- 172.21.15.17:0/3536047963 >> v1:172.21.15.163:6789/0 conn(0x7fd3881558d0 legacy=0x7fd38815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.980+0000 7fd3877fe640 1 -- 172.21.15.17:0/3536047963 >> v1:172.21.15.17:6789/0 conn(0x7fd3880c69f0 legacy=0x7fd3880acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.980+0000 7fd386ffd640 1 -- 172.21.15.17:0/3536047963 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409136393 0 0) 0x7fd378001110 con 0x7fd3880c69f0 2024-03-06T07:27:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.980+0000 7fd3877fe640 1 -- 172.21.15.17:0/3536047963 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880c5d50 con 0x7fd3880c8950 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.980+0000 7fd3877fe640 1 --2- 172.21.15.17:0/3536047963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3880c8950 0x7fd3880c6dd0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097a0 tx=0x7fd37c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c03ef7bfce3459 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd386ffd640 1 -- 172.21.15.17:0/3536047963 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0315a0 con 0x7fd3880c8950 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd386ffd640 1 -- 172.21.15.17:0/3536047963 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c037070 con 0x7fd3880c8950 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd386ffd640 1 -- 172.21.15.17:0/3536047963 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c031960 con 0x7fd3880c8950 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 -- 172.21.15.17:0/3536047963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3880c8950 msgr2=0x7fd3880c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/3536047963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3880c8950 0x7fd3880c6dd0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0097a0 tx=0x7fd37c02b840 comp rx=0 tx=0).stop 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 -- 172.21.15.17:0/3536047963 shutdown_connections 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/3536047963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3880c8950 0x7fd3880c6dd0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 -- 172.21.15.17:0/3536047963 >> 172.21.15.17:0/3536047963 conn(0x7fd3880cae70 msgr2=0x7fd3880cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 -- 172.21.15.17:0/3536047963 shutdown_connections 2024-03-06T07:27:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.981+0000 7fd38e4ed640 1 -- 172.21.15.17:0/3536047963 wait complete. 2024-03-06T07:27:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.982+0000 7fd38e4ed640 1 Processor -- start 2024-03-06T07:27:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.982+0000 7fd38e4ed640 1 -- start start 2024-03-06T07:27:36.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3880c69f0 0x7fd3880c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3880c8950 0x7fd3881eae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388152a20 con 0x7fd3881558d0 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3880c6310 con 0x7fd3880c69f0 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38e4ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd388152ba0 con 0x7fd3880c8950 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47196/0 (socket says 172.21.15.17:47196) 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2389913008 learned_addr learned my addr 172.21.15.17:0/2389913008 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd387fff640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3880c69f0 0x7fd3880c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3880c8950 msgr2=0x7fd3881eae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.983+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3880c8950 0x7fd3881eae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3880c69f0 msgr2=0x7fd3880c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3880c69f0 0x7fd3880c22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2389913008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3880c2000 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c780 tx=0x7fd38000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd384ff9640 1 -- 172.21.15.17:0/2389913008 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380004b60 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd384ff9640 1 -- 172.21.15.17:0/2389913008 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380004d00 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd384ff9640 1 -- 172.21.15.17:0/2389913008 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38000aa90 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd388157f80 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.984+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38806c7c0 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.985+0000 7fd384ff9640 1 -- 172.21.15.17:0/2389913008 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd38000ac30 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.986+0000 7fd384ff9640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.986+0000 7fd384ff9640 1 -- 172.21.15.17:0/2389913008 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd38008a640 con 0x7fd3881558d0 2024-03-06T07:27:36.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.986+0000 7fd387fff640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 Processor -- start 2024-03-06T07:27:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd387fff640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 0x7fd37406d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd378000f60 tx=0x7fd378009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 -- start start 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b618330 0x55cb0b615f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b6164b0 0x55cb0b612680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b57c6b0 con 0x55cb0b617f50 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b525a60 con 0x55cb0b6164b0 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.987+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b578750 con 0x55cb0b618330 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47200/0 (socket says 172.21.15.17:47200) 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 -- 172.21.15.17:0/1031382746 learned_addr learned my addr 172.21.15.17:0/1031382746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b6164b0 0x55cb0b612680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd3877fe640 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b618330 0x55cb0b615f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 -- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b618330 msgr2=0x55cb0b615f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b618330 0x55cb0b615f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 -- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b6164b0 msgr2=0x55cb0b612680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b6164b0 0x55cb0b612680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 -- 172.21.15.17:0/1031382746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb0b543f30 con 0x55cb0b617f50 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd387fff640 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd3780074d0 tx=0x7fd378010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df68973f19ac9fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd36dffb640 1 -- 172.21.15.17:0/1031382746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800da10 con 0x55cb0b617f50 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd36dffb640 1 -- 172.21.15.17:0/1031382746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800dbb0 con 0x55cb0b617f50 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.988+0000 7fd36dffb640 1 -- 172.21.15.17:0/1031382746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378020450 con 0x55cb0b617f50 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- 172.21.15.17:0/1031382746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 msgr2=0x55cb0b615a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fd3780074d0 tx=0x7fd378010000 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- 172.21.15.17:0/1031382746 shutdown_connections 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b6164b0 0x55cb0b612680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b618330 0x55cb0b615f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1031382746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b615a30 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- 172.21.15.17:0/1031382746 >> 172.21.15.17:0/1031382746 conn(0x55cb0b6147d0 msgr2=0x55cb0b6393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- 172.21.15.17:0/1031382746 shutdown_connections 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- 172.21.15.17:0/1031382746 wait complete. 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.989+0000 7fd3911b3740 1 -- start start 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 Processor -- start 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b63c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b63ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b63a230 con 0x55cb0b63fd80 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b538ea0 con 0x55cb0b618330 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b639c70 con 0x55cb0b617f50 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd3911b3740 10 client.0 _ll_register_callbacks cb 0x55cb0b538870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.990+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47208/0 (socket says 172.21.15.17:47208) 2024-03-06T07:27:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 -- 172.21.15.17:0/1784370262 learned_addr learned my addr 172.21.15.17:0/1784370262 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd3877fe640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b63ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd387fff640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b63c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 -- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 msgr2=0x55cb0b63c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b63c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 -- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 msgr2=0x55cb0b63ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b63ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 -- 172.21.15.17:0/1784370262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb0b539270 con 0x55cb0b63fd80 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd387fff640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b63c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fd380088a10 tx=0x7fd380059400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56668728f0ce6c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380070020 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd380010d90 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd37800dbb0 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd380061350 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd3911b3740 10 client.10053 fetch_fsmap learned FSMap version 173 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=2+}) v3 -- 0x55cb0b2912d0 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.991+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37800dbb0 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 173) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd3800552b0 con 0x55cb0b63fd80 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 10 client.10053 fetch_fsmap finished waiting for FSMap version 173 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 10 client.10053 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 20 client.10053 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b664aa0 0x55cb0b63af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 4 client.10053 mds_command: new command op to 9900 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 --> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb0b589bf0 con 0x55cb0b664aa0 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd387fff640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b664aa0 0x55cb0b63af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd366ffd640 10 client.10053 ms_handle_connect on v2:172.21.15.17:6834/445432625 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.992+0000 7fd387fff640 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b664aa0 0x55cb0b63af60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb2fbc9c09795bf server_cookie=e5cd133a5e223c5 in_seq=0 out_seq=0 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd366ffd640 1 -- 172.21.15.17:0/1784370262 <== mds.0 v2:172.21.15.17:6834/445432625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cb0b589bf0 con 0x55cb0b664aa0 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd366ffd640 10 client.10053 handle_command_reply: tid=0 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 client.10053 shutdown 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 msgr2=0x55cb0b634fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fd380088a10 tx=0x7fd380059400 comp rx=0 tx=0).stop 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 shutdown_connections 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b664aa0 0x55cb0b63af60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b63fd80 0x55cb0b634fc0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b63ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 --2- 172.21.15.17:0/1784370262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b63c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd366ffd640 0 client.10053 ms_handle_reset on v2:172.21.15.17:6834/445432625 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 >> 172.21.15.17:0/1784370262 conn(0x55cb0b6164b0 msgr2=0x55cb0b611510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 shutdown_connections 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 1 -- 172.21.15.17:0/1784370262 wait complete. 2024-03-06T07:27:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:36.993+0000 7fd3911b3740 20 client.10053 trim_cache size 0 max 16384 2024-03-06T07:27:37.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.026+0000 7fd3911b3740 1 Processor -- start 2024-03-06T07:27:37.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.026+0000 7fd3911b3740 1 -- start start 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b611510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b611a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881f6fd0 con 0x55cb0b653b50 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881eee60 con 0x55cb0b618330 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881f6a00 con 0x55cb0b617f50 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47220/0 (socket says 172.21.15.17:47220) 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2577225032 learned_addr learned my addr 172.21.15.17:0/2577225032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b611a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd387fff640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b611510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 msgr2=0x55cb0b611510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b611510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 msgr2=0x55cb0b611a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b611a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd38ca8a640 1 -- 172.21.15.17:0/2577225032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb0b35b6b0 con 0x55cb0b653b50 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd387fff640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b611510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.027+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b611a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd38005f010 tx=0x7fd38005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812145a15a41d389 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd36d7fa640 1 -- 172.21.15.17:0/2577225032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380070040 con 0x55cb0b653b50 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd36d7fa640 1 -- 172.21.15.17:0/2577225032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380010d90 con 0x55cb0b653b50 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd36d7fa640 1 -- 172.21.15.17:0/2577225032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380054ee0 con 0x55cb0b653b50 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 -- 172.21.15.17:0/2577225032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 msgr2=0x55cb0b653fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd38005f010 tx=0x7fd38005b000 comp rx=0 tx=0).stop 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 -- 172.21.15.17:0/2577225032 shutdown_connections 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b653b50 0x55cb0b653fd0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b611a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2577225032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b617f50 0x55cb0b611510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 -- 172.21.15.17:0/2577225032 >> 172.21.15.17:0/2577225032 conn(0x55cb0b6164b0 msgr2=0x55cb0b63e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 -- 172.21.15.17:0/2577225032 shutdown_connections 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.028+0000 7fd3911b3740 1 -- 172.21.15.17:0/2577225032 wait complete. 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.029+0000 7fd3911b3740 1 -- start start 2024-03-06T07:27:37.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.029+0000 7fd3911b3740 1 Processor -- start 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b6c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b653b50 0x55cb0b6c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b655020 con 0x55cb0b617f50 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b63a230 con 0x55cb0b618330 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb0b63d450 con 0x55cb0b653b50 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3911b3740 10 client.0 _ll_register_callbacks cb 0x7fd380058780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47222/0 (socket says 172.21.15.17:47222) 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 -- 172.21.15.17:0/2924456708 learned_addr learned my addr 172.21.15.17:0/2924456708 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b6c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd38ca8a640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b653b50 0x55cb0b6c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 -- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b653b50 msgr2=0x55cb0b6c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b653b50 0x55cb0b6c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 -- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 msgr2=0x55cb0b6c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b6c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 -- 172.21.15.17:0/2924456708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb0b612020 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd387fff640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fd378013000 tx=0x7fd378000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20a6bf3cc44daf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.030+0000 7fd366ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378035060 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd37800d990 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd380010d90 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd37803b050 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 10 client.10059 fetch_fsmap learned FSMap version 173 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=2+}) v3 -- 0x55cb0b635bd0 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380010d90 con 0x55cb0b617f50 2024-03-06T07:27:37.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 173) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fd37803e020 con 0x55cb0b617f50 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 10 client.10059 fetch_fsmap finished waiting for FSMap version 173 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 10 client.10059 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 20 client.10059 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b6c3530 0x55cb0b6506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 4 client.10059 mds_command: new command op to 9900 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 --> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cb0b583f40 con 0x55cb0b6c3530 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.031+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b6c3530 0x55cb0b6506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd366ffd640 10 client.10059 ms_handle_connect on v2:172.21.15.17:6834/445432625 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b6c3530 0x55cb0b6506d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec354f7e0f1a38d9 server_cookie=e0427e0c59bc4ba3 in_seq=0 out_seq=0 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd366ffd640 1 -- 172.21.15.17:0/2924456708 <== mds.0 v2:172.21.15.17:6834/445432625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55cb0b583f40 con 0x55cb0b6c3530 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd366ffd640 10 client.10059 handle_command_reply: tid=0 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 client.10059 shutdown 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 msgr2=0x55cb0b66e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fd378013000 tx=0x7fd378000ed0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 shutdown_connections 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:6834/445432625,v1:172.21.15.17:6835/445432625] conn(0x55cb0b6c3530 0x55cb0b6506d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55cb0b653b50 0x55cb0b6c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55cb0b618330 0x55cb0b6c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd366ffd640 0 client.10059 ms_handle_reset on v2:172.21.15.17:6834/445432625 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 --2- 172.21.15.17:0/2924456708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb0b617f50 0x55cb0b66e910 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 >> 172.21.15.17:0/2924456708 conn(0x55cb0b6164b0 msgr2=0x55cb0b63b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 shutdown_connections 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 1 -- 172.21.15.17:0/2924456708 wait complete. 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.032+0000 7fd3911b3740 20 client.10059 trim_cache size 0 max 16384 2024-03-06T07:27:37.040 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9900, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 9.4957320630000002, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 173, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 186, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 186, 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14.692092710000001 2024-03-06T07:27:37.041 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 msgr2=0x7fd37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 0x7fd37406d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd378000f60 tx=0x7fd378009040 comp rx=0 tx=0).stop 2024-03-06T07:27:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 msgr2=0x7fd38806c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c780 tx=0x7fd38000cc40 comp rx=0 tx=0).stop 2024-03-06T07:27:37.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 shutdown_connections 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd37406b4d0 0x7fd37406d990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3881558d0 0x7fd38806c280 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3880c8950 0x7fd3881eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 --2- 172.21.15.17:0/2389913008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3880c69f0 0x7fd3880c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 >> 172.21.15.17:0/2389913008 conn(0x7fd3880cae70 msgr2=0x7fd3880bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 shutdown_connections 2024-03-06T07:27:37.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.033+0000 7fd38e4ed640 1 -- 172.21.15.17:0/2389913008 wait complete. 2024-03-06T07:27:37.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:27:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.167+0000 7f280feeb640 1 Processor -- start 2024-03-06T07:27:37.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.167+0000 7f280feeb640 1 -- start start 2024-03-06T07:27:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280feeb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f28080c57d0 0x7f2808155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280feeb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2808152dd0 con 0x7f28080c5370 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280feeb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2808158b00 con 0x7f28081529f0 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280feeb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2808159710 con 0x7f28080c57d0 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280d486640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f28080c5370 0x7f2808155530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54590/0 (socket says 172.21.15.17:54590) 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280d486640 1 -- 172.21.15.17:0/1964583411 learned_addr learned my addr 172.21.15.17:0/1964583411 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280e488640 1 --2- 172.21.15.17:0/1964583411 >> v2:172.21.15.17:3300/0 conn(0x7f28080c57d0 0x7f2808155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280cc85640 1 -- 172.21.15.17:0/1964583411 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176729258 0 0) 0x7f2808152dd0 con 0x7f28080c5370 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280cc85640 1 -- 172.21.15.17:0/1964583411 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f4003520 con 0x7f28080c5370 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280e488640 1 -- 172.21.15.17:0/1964583411 >> v1:172.21.15.163:6789/0 conn(0x7f28081529f0 legacy=0x7f28080c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280e488640 1 -- 172.21.15.17:0/1964583411 >> v1:172.21.15.17:6789/0 conn(0x7f28080c5370 legacy=0x7f2808155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.168+0000 7f280e488640 1 -- 172.21.15.17:0/1964583411 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808158de0 con 0x7f28080c57d0 2024-03-06T07:27:37.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.169+0000 7f280e488640 1 --2- 172.21.15.17:0/1964583411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28080c57d0 0x7f2808155c40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a788a724db0ab716 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.169+0000 7f280cc85640 1 -- 172.21.15.17:0/1964583411 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800039070 con 0x7f28080c57d0 2024-03-06T07:27:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.169+0000 7f280cc85640 1 -- 172.21.15.17:0/1964583411 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280002cbd0 con 0x7f28080c57d0 2024-03-06T07:27:37.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.169+0000 7f280cc85640 1 -- 172.21.15.17:0/1964583411 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800034440 con 0x7f28080c57d0 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 -- 172.21.15.17:0/1964583411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28080c57d0 msgr2=0x7f2808155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 --2- 172.21.15.17:0/1964583411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28080c57d0 0x7f2808155c40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2800009d90 tx=0x7f280002be20 comp rx=0 tx=0).stop 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 -- 172.21.15.17:0/1964583411 shutdown_connections 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 --2- 172.21.15.17:0/1964583411 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28080c57d0 0x7f2808155c40 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 -- 172.21.15.17:0/1964583411 >> 172.21.15.17:0/1964583411 conn(0x7f28080ac580 msgr2=0x7f28080ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 -- 172.21.15.17:0/1964583411 shutdown_connections 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.170+0000 7f280feeb640 1 -- 172.21.15.17:0/1964583411 wait complete. 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 Processor -- start 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 -- start start 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28080c5370 0x7f2808151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28080c57d0 0x7f28081eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c68b0 con 0x7f28081529f0 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c9000 con 0x7f28080c57d0 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.171+0000 7f280feeb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28081593a0 con 0x7f28080c5370 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47252/0 (socket says 172.21.15.17:47252) 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 -- 172.21.15.17:0/183620376 learned_addr learned my addr 172.21.15.17:0/183620376 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 -- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28080c5370 msgr2=0x7f2808151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28080c5370 0x7f2808151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 -- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28080c57d0 msgr2=0x7f28081eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28080c57d0 0x7f28081eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 -- 172.21.15.17:0/183620376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28081544b0 con 0x7f28081529f0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280e488640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f280002be60 tx=0x7f2800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800039050 con 0x7f28081529f0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800045070 con 0x7f28081529f0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2808154790 con 0x7f28081529f0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28000046b0 con 0x7f28081529f0 2024-03-06T07:27:37.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.172+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2808155050 con 0x7f28081529f0 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.173+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2808068e00 con 0x7f28081529f0 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.174+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f280004a460 con 0x7f28081529f0 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.174+0000 7f27fe7fc640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 0x7f27ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.174+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28000accb0 con 0x7f28081529f0 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.174+0000 7f280dc87640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 0x7f27ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.175+0000 7f280dc87640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 0x7f27ec06d9c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2804006fd0 tx=0x7f2804006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.178+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2808068e00 con 0x7f28081529f0 2024-03-06T07:27:37.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.397+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f280804f330 con 0x7f28081529f0 2024-03-06T07:27:37.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.565+0000 7f27fe7fc640 1 -- 172.21.15.17:0/183620376 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v174) v1 ==== 125+0+0 (secure 0 0 0) 0x7f280007ba30 con 0x7f28081529f0 2024-03-06T07:27:37.567 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked down. 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 msgr2=0x7f27ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 0x7f27ec06d9c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f2804006fd0 tx=0x7f2804006bc0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 msgr2=0x7f28081ef270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f280002be60 tx=0x7f2800004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 shutdown_connections 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f27ec06b500 0x7f27ec06d9c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28081529f0 0x7f28081ef270 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28080c57d0 0x7f28081eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 --2- 172.21.15.17:0/183620376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28080c5370 0x7f2808151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 >> 172.21.15.17:0/183620376 conn(0x7f28080ac580 msgr2=0x7f28080c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 shutdown_connections 2024-03-06T07:27:37.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.568+0000 7f280feeb640 1 -- 172.21.15.17:0/183620376 wait complete. 2024-03-06T07:27:37.581 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.680+0000 7f46e4fac640 1 Processor -- start 2024-03-06T07:27:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.680+0000 7f46e4fac640 1 -- start start 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46e4fac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46e006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46e4fac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46e00ac260 0x7f46e01559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46e4fac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e0152d40 con 0x7f46e01592d0 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46e4fac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e0066e30 con 0x7f46e00ac260 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46e4fac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c1dc0 con 0x7f46e0152960 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46d7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46e01592d0 0x7f46e01552d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54606/0 (socket says 172.21.15.17:54606) 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46d7fff640 1 -- 172.21.15.17:0/2263707270 learned_addr learned my addr 172.21.15.17:0/2263707270 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46ded9d640 1 --2- 172.21.15.17:0/2263707270 >> v2:172.21.15.163:3300/0 conn(0x7f46e00ac260 0x7f46e01559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.681+0000 7f46de59c640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46e006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189749583 0 0) 0x7f46e0152d40 con 0x7f46e01592d0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46cc003520 con 0x7f46e01592d0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1571265910 0 0) 0x7f46cc003520 con 0x7f46e01592d0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 msgr2=0x7f46e006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46e006dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 >> v2:172.21.15.163:3300/0 conn(0x7f46e00ac260 msgr2=0x7f46e01559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 --2- 172.21.15.17:0/2263707270 >> v2:172.21.15.163:3300/0 conn(0x7f46e00ac260 0x7f46e01559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00c1b50 con 0x7f46e01592d0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f46c8001670 con 0x7f46e01592d0 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ded9d640 1 --2- 172.21.15.17:0/2263707270 >> v2:172.21.15.163:3300/0 conn(0x7f46e00ac260 0x7f46e01559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 >> v1:172.21.15.17:6789/0 conn(0x7f46e01592d0 legacy=0x7f46e01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0039b0 0x7f46cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46cc004800 0x7f46cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0066e30 con 0x7f46cc004800 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c1dc0 con 0x7f46cc0039b0 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc00b9c0 con 0x7f46e0152960 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46cc004800 0x7f46cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46de59c640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0039b0 0x7f46cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ded9d640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 -- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 msgr2=0x7f46cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 -- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0039b0 msgr2=0x7f46cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46cc0039b0 0x7f46cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46d7fff640 1 -- 172.21.15.17:0/2263707270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc003520 con 0x7f46cc004800 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.682+0000 7f46ded9d640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46e0152960 0x7f46cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.683+0000 7f46d7fff640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46cc004800 0x7f46cc004c80 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c210 tx=0x7f46cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a406741ab7e477 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.683+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8007d50 con 0x7f46cc004800 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.683+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8030070 con 0x7f46cc004800 2024-03-06T07:27:37.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.683+0000 7f46ddd9b640 1 -- 172.21.15.17:0/2263707270 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c802cd80 con 0x7f46cc004800 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 -- 172.21.15.17:0/2263707270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46cc004800 msgr2=0x7f46cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 --2- 172.21.15.17:0/2263707270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46cc004800 0x7f46cc004c80 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c210 tx=0x7f46cc005830 comp rx=0 tx=0).stop 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46ded9d640 1 -- 172.21.15.17:0/2263707270 reap_dead start 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 -- 172.21.15.17:0/2263707270 shutdown_connections 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 -- 172.21.15.17:0/2263707270 >> 172.21.15.17:0/2263707270 conn(0x7f46e00c70b0 msgr2=0x7f46e00c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 -- 172.21.15.17:0/2263707270 shutdown_connections 2024-03-06T07:27:37.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.684+0000 7f46e4fac640 1 -- 172.21.15.17:0/2263707270 wait complete. 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.685+0000 7f46e4fac640 1 Processor -- start 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.685+0000 7f46e4fac640 1 -- start start 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 0x7f46e014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 0x7f46e01eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c20a0 con 0x7f46e01ee480 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e0159760 con 0x7f46e014fed0 2024-03-06T07:27:37.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e00c2220 con 0x7f46cc004800 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47272/0 (socket says 172.21.15.17:47272) 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 -- 172.21.15.17:0/3799992167 learned_addr learned my addr 172.21.15.17:0/3799992167 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46de59c640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 0x7f46e014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46d7fff640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 0x7f46e01eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 -- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 msgr2=0x7f46e014f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 0x7f46e014f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 -- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 msgr2=0x7f46e01eb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 0x7f46e01eb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 -- 172.21.15.17:0/3799992167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e01ee9f0 con 0x7f46e01ee480 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46de59c640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 0x7f46e014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46d7fff640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 0x7f46e01eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46ded9d640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f46e0070510 tx=0x7f46d8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d801a020 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d8016470 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e01eecd0 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.686+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8016840 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.687+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e01ef290 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.687+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e004f330 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.688+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46d8005d10 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.688+0000 7f46d6ffd640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.688+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46d808b040 con 0x7f46e01ee480 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.689+0000 7f46de59c640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:37.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.689+0000 7f46de59c640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 0x7f46c006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f46e0153260 tx=0x7f46d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:37.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.693+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46d8024030 con 0x7f46e01ee480 2024-03-06T07:27:37.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.909+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e0072c70 con 0x7f46e01ee480 2024-03-06T07:27:37.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.910+0000 7f46d6ffd640 1 -- 172.21.15.17:0/3799992167 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f46d8059dc0 con 0x7f46e01ee480 2024-03-06T07:27:37.912 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:37.912 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:37.912 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:37.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 msgr2=0x7f46c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 0x7f46c006d9c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f46e0153260 tx=0x7f46d0008040 comp rx=0 tx=0).stop 2024-03-06T07:27:37.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 msgr2=0x7f46e01eb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:37.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f46e0070510 tx=0x7f46d8011890 comp rx=0 tx=0).stop 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 shutdown_connections 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46c006b500 0x7f46c006d9c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46e01ee480 0x7f46e01eb9e0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46e014fed0 0x7f46e01eb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 --2- 172.21.15.17:0/3799992167 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46cc004800 0x7f46e014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.911+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 >> 172.21.15.17:0/3799992167 conn(0x7f46e00c70b0 msgr2=0x7f46e00c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.912+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 shutdown_connections 2024-03-06T07:27:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:37.912+0000 7f46e4fac640 1 -- 172.21.15.17:0/3799992167 wait complete. 2024-03-06T07:27:38.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:38.326 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:39.327 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:39.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.439+0000 7f7e3ed35640 1 Processor -- start 2024-03-06T07:27:39.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.439+0000 7f7e3ed35640 1 -- start start 2024-03-06T07:27:39.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3ed35640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 0x7f7e381552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3ed35640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7e380ac260 0x7f7e38155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3ed35640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e38152d40 con 0x7f7e38152960 2024-03-06T07:27:39.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3ed35640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e38066e30 con 0x7f7e380ac260 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3ed35640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e38155d50 con 0x7f7e381592d0 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3d2d2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7e380ac260 0x7f7e38155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3d2d2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7e380ac260 0x7f7e38155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47284/0 (socket says 172.21.15.17:47284) 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.440+0000 7f7e3d2d2640 1 -- 172.21.15.17:0/2680941870 learned_addr learned my addr 172.21.15.17:0/2680941870 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e37fff640 1 --2- 172.21.15.17:0/2680941870 >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 0x7f7e381552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e3d2d2640 1 -- 172.21.15.17:0/2680941870 >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 msgr2=0x7f7e381552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e3d2d2640 1 --2- 172.21.15.17:0/2680941870 >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 0x7f7e381552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e3d2d2640 1 -- 172.21.15.17:0/2680941870 >> v1:172.21.15.163:6789/0 conn(0x7f7e38152960 legacy=0x7f7e3806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e3d2d2640 1 -- 172.21.15.17:0/2680941870 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380c1b50 con 0x7f7e380ac260 2024-03-06T07:27:39.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e37fff640 1 --2- 172.21.15.17:0/2680941870 >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 0x7f7e381552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.441+0000 7f7e3d2d2640 1 --2- 172.21.15.17:0/2680941870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e380ac260 0x7f7e38155810 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f7e3806ee90 tx=0x7f7e3002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2634d96716873743 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e377fe640 1 -- 172.21.15.17:0/2680941870 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3002ca80 con 0x7f7e380ac260 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e377fe640 1 -- 172.21.15.17:0/2680941870 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3002cc20 con 0x7f7e380ac260 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e377fe640 1 -- 172.21.15.17:0/2680941870 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e300343f0 con 0x7f7e380ac260 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2680941870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e380ac260 msgr2=0x7f7e38155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2680941870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e380ac260 0x7f7e38155810 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f7e3806ee90 tx=0x7f7e3002bee0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2680941870 shutdown_connections 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2680941870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e380ac260 0x7f7e38155810 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2680941870 >> v2:172.21.15.163:3300/0 conn(0x7f7e381592d0 0x7f7e381552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2680941870 >> 172.21.15.17:0/2680941870 conn(0x7f7e380c70b0 msgr2=0x7f7e380c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2680941870 shutdown_connections 2024-03-06T07:27:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.442+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2680941870 wait complete. 2024-03-06T07:27:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.443+0000 7f7e3ed35640 1 Processor -- start 2024-03-06T07:27:39.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.443+0000 7f7e3ed35640 1 -- start start 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e380ac260 0x7f7e381eed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e3814f360 0x7f7e381ead00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38159760 con 0x7f7e38152960 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380672c0 con 0x7f7e3814f360 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e3ed35640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3806e3a0 con 0x7f7e380ac260 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47292/0 (socket says 172.21.15.17:47292) 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.444+0000 7f7e37fff640 1 -- 172.21.15.17:0/2434011778 learned_addr learned my addr 172.21.15.17:0/2434011778 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e3cad1640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e380ac260 0x7f7e381eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e3d2d2640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e3814f360 0x7f7e381ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 -- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e380ac260 msgr2=0x7f7e381eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e380ac260 0x7f7e381eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 -- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e3814f360 msgr2=0x7f7e381ead00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e3814f360 0x7f7e381ead00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 -- 172.21.15.17:0/2434011778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e381ee270 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e37fff640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c7f0 tx=0x7f7e2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24015070 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24007820 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e38070ea0 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24007bd0 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.445+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e381ee510 con 0x7f7e38152960 2024-03-06T07:27:39.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.446+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38070c30 con 0x7f7e38152960 2024-03-06T07:27:39.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.447+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e24018070 con 0x7f7e38152960 2024-03-06T07:27:39.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.447+0000 7f7e357fa640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 0x7f7e1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:39.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.447+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e2408a400 con 0x7f7e38152960 2024-03-06T07:27:39.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.447+0000 7f7e3cad1640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 0x7f7e1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:39.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.448+0000 7f7e3cad1640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 0x7f7e1c06d9c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c004ea0 tx=0x7f7e2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:39.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.451+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e24059180 con 0x7f7e38152960 2024-03-06T07:27:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.671+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e3804f330 con 0x7f7e38152960 2024-03-06T07:27:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.671+0000 7f7e357fa640 1 -- 172.21.15.17:0/2434011778 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f7e24058a80 con 0x7f7e38152960 2024-03-06T07:27:39.673 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:39.673 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 msgr2=0x7f7e1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 0x7f7e1c06d9c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c004ea0 tx=0x7f7e2c009040 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 msgr2=0x7f7e381deba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c7f0 tx=0x7f7e2400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 shutdown_connections 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7e1c06b500 0x7f7e1c06d9c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7e3814f360 0x7f7e381ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7e38152960 0x7f7e381deba0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 --2- 172.21.15.17:0/2434011778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7e380ac260 0x7f7e381eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 >> 172.21.15.17:0/2434011778 conn(0x7f7e380c70b0 msgr2=0x7f7e3806dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 shutdown_connections 2024-03-06T07:27:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:39.673+0000 7f7e3ed35640 1 -- 172.21.15.17:0/2434011778 wait complete. 2024-03-06T07:27:40.033 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:40.035 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:27:40.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:40.088 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:41.089 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.201+0000 7f52355f7640 1 Processor -- start 2024-03-06T07:27:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.201+0000 7f52355f7640 1 -- start start 2024-03-06T07:27:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f52355f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52300c6780 0x7f5230158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f52355f7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52301592d0 0x7f52300c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f52355f7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5230072630 con 0x7f523006da40 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f52355f7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523006de20 con 0x7f52301592d0 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f52355f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5230072910 con 0x7f52300c6780 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f522f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52301592d0 0x7f52300c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f522f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f52301592d0 0x7f52300c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57910/0 (socket says 172.21.15.17:57910) 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.202+0000 7f522f7fe640 1 -- 172.21.15.17:0/3143557050 learned_addr learned my addr 172.21.15.17:0/3143557050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522e7fc640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52300c6780 0x7f5230158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522f7fe640 1 -- 172.21.15.17:0/3143557050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52300c6780 msgr2=0x7f5230158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522f7fe640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52300c6780 0x7f5230158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522f7fe640 1 -- 172.21.15.17:0/3143557050 >> v1:172.21.15.163:6789/0 conn(0x7f523006da40 legacy=0x7f5230155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522f7fe640 1 -- 172.21.15.17:0/3143557050 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230072bf0 con 0x7f52301592d0 2024-03-06T07:27:41.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.203+0000 7f522f7fe640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52301592d0 0x7f52300c3280 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f5228009870 tx=0x7f522802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4404c0a41936ff server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f522dffb640 1 -- 172.21.15.17:0/3143557050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280317c0 con 0x7f52301592d0 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f522dffb640 1 -- 172.21.15.17:0/3143557050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228037070 con 0x7f52301592d0 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f522dffb640 1 -- 172.21.15.17:0/3143557050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031b80 con 0x7f52301592d0 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 -- 172.21.15.17:0/3143557050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52301592d0 msgr2=0x7f52300c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52301592d0 0x7f52300c3280 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f5228009870 tx=0x7f522802b860 comp rx=0 tx=0).stop 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 -- 172.21.15.17:0/3143557050 shutdown_connections 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52301592d0 0x7f52300c3280 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 --2- 172.21.15.17:0/3143557050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52300c6780 0x7f5230158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 -- 172.21.15.17:0/3143557050 >> 172.21.15.17:0/3143557050 conn(0x7f52300aa630 msgr2=0x7f52300aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 -- 172.21.15.17:0/3143557050 shutdown_connections 2024-03-06T07:27:41.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.204+0000 7f52355f7640 1 -- 172.21.15.17:0/3143557050 wait complete. 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 Processor -- start 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 -- start start 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f523006da40 0x7f52300a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52301592d0 0x7f5230152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c3940 con 0x7f52300c6780 2024-03-06T07:27:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c37c0 con 0x7f523006da40 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f52355f7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c3ac0 con 0x7f52301592d0 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f522e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f522e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57912/0 (socket says 172.21.15.17:57912) 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.206+0000 7f522e7fc640 1 -- 172.21.15.17:0/198658413 learned_addr learned my addr 172.21.15.17:0/198658413 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522f7fe640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52301592d0 0x7f5230152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522effd640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f523006da40 0x7f52300a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 -- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52301592d0 msgr2=0x7f5230152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52301592d0 0x7f5230152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 -- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f523006da40 msgr2=0x7f52300a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f523006da40 0x7f52300a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 -- 172.21.15.17:0/198658413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300acff0 con 0x7f52300c6780 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f522e7fc640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f521800d800 tx=0x7f521800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800b660 con 0x7f52300c6780 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218019070 con 0x7f52300c6780 2024-03-06T07:27:41.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52301eee00 con 0x7f52300c6780 2024-03-06T07:27:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800ba40 con 0x7f52300c6780 2024-03-06T07:27:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.207+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52301dec20 con 0x7f52300c6780 2024-03-06T07:27:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.208+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523004f330 con 0x7f52300c6780 2024-03-06T07:27:41.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.209+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f521800bbe0 con 0x7f52300c6780 2024-03-06T07:27:41.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.209+0000 7f52277fe640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 0x7f521006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:41.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.209+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f521808a670 con 0x7f52300c6780 2024-03-06T07:27:41.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.209+0000 7f522effd640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 0x7f521006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:41.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.210+0000 7f522effd640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 0x7f521006d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5220009f20 tx=0x7f52200094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:41.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.214+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f52180593f0 con 0x7f52300c6780 2024-03-06T07:27:41.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.429+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5230073240 con 0x7f52300c6780 2024-03-06T07:27:41.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.430+0000 7f52277fe640 1 -- 172.21.15.17:0/198658413 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f5218058cf0 con 0x7f52300c6780 2024-03-06T07:27:41.432 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:41.432 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:41.432 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:41.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.431+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 msgr2=0x7f521006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.431+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 0x7f521006d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5220009f20 tx=0x7f52200094d0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.431+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 msgr2=0x7f5230151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.431+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f521800d800 tx=0x7f521800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 shutdown_connections 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f521006b4d0 0x7f521006d990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52301592d0 0x7f5230152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52300c6780 0x7f5230151b10 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 --2- 172.21.15.17:0/198658413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f523006da40 0x7f52300a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 >> 172.21.15.17:0/198658413 conn(0x7f52300aa630 msgr2=0x7f5230154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 shutdown_connections 2024-03-06T07:27:41.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:41.432+0000 7f52355f7640 1 -- 172.21.15.17:0/198658413 wait complete. 2024-03-06T07:27:41.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:41.846 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:42.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.959+0000 7f5724dbf640 1 Processor -- start 2024-03-06T07:27:42.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.959+0000 7f5724dbf640 1 -- start start 2024-03-06T07:27:42.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f5724dbf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 0x7f5720155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f5724dbf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5720158b60 0x7f5720158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f5724dbf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200c73d0 con 0x7f572014f360 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f5724dbf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57200c8dd0 con 0x7f5720158b60 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f5724dbf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57200be2e0 con 0x7f57200c88c0 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f571ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5720158b60 0x7f5720158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f571e59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f572014f360 0x7f57200c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41504/0 (socket says 172.21.15.17:41504) 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f571ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5720158b60 0x7f5720158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57926/0 (socket says 172.21.15.17:57926) 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.960+0000 7f571e59c640 1 -- 172.21.15.17:0/4111241773 learned_addr learned my addr 172.21.15.17:0/4111241773 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571dd9b640 1 --2- 172.21.15.17:0/4111241773 >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 0x7f5720155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571ed9d640 1 -- 172.21.15.17:0/4111241773 >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 msgr2=0x7f5720155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571ed9d640 1 --2- 172.21.15.17:0/4111241773 >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 0x7f5720155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571ed9d640 1 -- 172.21.15.17:0/4111241773 >> v1:172.21.15.17:6789/0 conn(0x7f572014f360 legacy=0x7f57200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571d59a640 1 -- 172.21.15.17:0/4111241773 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282934096 0 0) 0x7f57200c73d0 con 0x7f572014f360 2024-03-06T07:27:42.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571ed9d640 1 -- 172.21.15.17:0/4111241773 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200c7670 con 0x7f5720158b60 2024-03-06T07:27:42.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.961+0000 7f571dd9b640 1 --2- 172.21.15.17:0/4111241773 >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 0x7f5720155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f571ed9d640 1 --2- 172.21.15.17:0/4111241773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5720158b60 0x7f5720158ff0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f57200c84c0 tx=0x7f571002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a21a24b43ee2ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f571d59a640 1 -- 172.21.15.17:0/4111241773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710039070 con 0x7f5720158b60 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f571d59a640 1 -- 172.21.15.17:0/4111241773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571002cbf0 con 0x7f5720158b60 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f571d59a640 1 -- 172.21.15.17:0/4111241773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710034440 con 0x7f5720158b60 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 -- 172.21.15.17:0/4111241773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5720158b60 msgr2=0x7f5720158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 --2- 172.21.15.17:0/4111241773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5720158b60 0x7f5720158ff0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f57200c84c0 tx=0x7f571002be40 comp rx=0 tx=0).stop 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 -- 172.21.15.17:0/4111241773 shutdown_connections 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 --2- 172.21.15.17:0/4111241773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5720158b60 0x7f5720158ff0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 --2- 172.21.15.17:0/4111241773 >> v2:172.21.15.163:3300/0 conn(0x7f57200c88c0 0x7f5720155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 -- 172.21.15.17:0/4111241773 >> 172.21.15.17:0/4111241773 conn(0x7f57200ac570 msgr2=0x7f57200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 -- 172.21.15.17:0/4111241773 shutdown_connections 2024-03-06T07:27:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.962+0000 7f5724dbf640 1 -- 172.21.15.17:0/4111241773 wait complete. 2024-03-06T07:27:42.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 Processor -- start 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 -- start start 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f572014f360 0x7f5720148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5720158b60 0x7f57200c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c5a50 con 0x7f57200c88c0 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c7c30 con 0x7f5720158b60 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.964+0000 7f5724dbf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200c5bd0 con 0x7f572014f360 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57942/0 (socket says 172.21.15.17:57942) 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 -- 172.21.15.17:0/472677054 learned_addr learned my addr 172.21.15.17:0/472677054 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 -- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f572014f360 msgr2=0x7f5720148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f572014f360 0x7f5720148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 -- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5720158b60 msgr2=0x7f57200c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5720158b60 0x7f57200c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 -- 172.21.15.17:0/472677054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200b0630 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f571e59c640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f571800c4d0 tx=0x7f571800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718018070 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5718007500 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57200be690 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718004a20 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.965+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57200bec50 con 0x7f57200c88c0 2024-03-06T07:27:42.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.966+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572004f330 con 0x7f57200c88c0 2024-03-06T07:27:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.967+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f571801b070 con 0x7f57200c88c0 2024-03-06T07:27:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.967+0000 7f5716ffd640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 0x7f570006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.967+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57180895c0 con 0x7f57200c88c0 2024-03-06T07:27:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.968+0000 7f571dd9b640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 0x7f570006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:42.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.968+0000 7f571dd9b640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 0x7f570006d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:42.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:42.972+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5718058340 con 0x7f57200c88c0 2024-03-06T07:27:43.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.191+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f572006cda0 con 0x7f57200c88c0 2024-03-06T07:27:43.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.191+0000 7f5716ffd640 1 -- 172.21.15.17:0/472677054 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f5718057c40 con 0x7f57200c88c0 2024-03-06T07:27:43.193 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:43.193 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:43.194 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 msgr2=0x7f570006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 0x7f570006d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5708006fd0 tx=0x7f5708008040 comp rx=0 tx=0).stop 2024-03-06T07:27:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 msgr2=0x7f57200bf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f571800c4d0 tx=0x7f571800c990 comp rx=0 tx=0).stop 2024-03-06T07:27:43.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 shutdown_connections 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f570006b500 0x7f570006d9c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5720158b60 0x7f57200c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f572014f360 0x7f5720148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 --2- 172.21.15.17:0/472677054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f57200c88c0 0x7f57200bf730 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.193+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 >> 172.21.15.17:0/472677054 conn(0x7f57200ac570 msgr2=0x7f57200c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.194+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 shutdown_connections 2024-03-06T07:27:43.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:43.194+0000 7f5724dbf640 1 -- 172.21.15.17:0/472677054 wait complete. 2024-03-06T07:27:43.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:43.609 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:44.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.721+0000 7fbd4f365640 1 Processor -- start 2024-03-06T07:27:44.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.721+0000 7fbd4f365640 1 -- start start 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4f365640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd48155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4f365640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480c73d0 con 0x7fbd480c88c0 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4f365640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd480be2e0 con 0x7fbd48158b60 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4f365640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48159700 con 0x7fbd4814f360 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4c900640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbd480c88c0 0x7fbd480c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41518/0 (socket says 172.21.15.17:41518) 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.722+0000 7fbd4c900640 1 -- 172.21.15.17:0/1040420738 learned_addr learned my addr 172.21.15.17:0/1040420738 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd4d101640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd48155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266104579 0 0) 0x7fbd480c73d0 con 0x7fbd480c88c0 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30003520 con 0x7fbd480c88c0 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178754867 0 0) 0x7fbd30003520 con 0x7fbd480c88c0 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 msgr2=0x7fbd48155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd48155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 >> v1:172.21.15.163:6789/0 conn(0x7fbd48158b60 legacy=0x7fbd48158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480c7670 con 0x7fbd480c88c0 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbd3c001670 con 0x7fbd480c88c0 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 >> v1:172.21.15.17:6789/0 conn(0x7fbd480c88c0 legacy=0x7fbd480c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30003f20 0x7fbd30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30004d40 0x7fbd300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48159700 con 0x7fbd30003f20 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3000bad0 con 0x7fbd30004d40 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3000bcd0 con 0x7fbd4814f360 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd4c900640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30003f20 0x7fbd30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd4d101640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.723+0000 7fbd4d902640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30004d40 0x7fbd300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 -- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 msgr2=0x7fbd300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4814f360 0x7fbd300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 -- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30004d40 msgr2=0x7fbd300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30004d40 0x7fbd300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 -- 172.21.15.17:0/1040420738 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30003520 con 0x7fbd30003f20 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4d902640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30004d40 0x7fbd300051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.724+0000 7fbd4c900640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30003f20 0x7fbd30004800 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c160 tx=0x7fbd30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aea778ec300208 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c007cc0 con 0x7fbd30003f20 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c030070 con 0x7fbd30003f20 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd47fff640 1 -- 172.21.15.17:0/1040420738 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c005d50 con 0x7fbd30003f20 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 -- 172.21.15.17:0/1040420738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30003f20 msgr2=0x7fbd30004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 --2- 172.21.15.17:0/1040420738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30003f20 0x7fbd30004800 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c160 tx=0x7fbd30005830 comp rx=0 tx=0).stop 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4d902640 1 -- 172.21.15.17:0/1040420738 reap_dead start 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 -- 172.21.15.17:0/1040420738 shutdown_connections 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 -- 172.21.15.17:0/1040420738 >> 172.21.15.17:0/1040420738 conn(0x7fbd480ac570 msgr2=0x7fbd480ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 -- 172.21.15.17:0/1040420738 shutdown_connections 2024-03-06T07:27:44.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.725+0000 7fbd4f365640 1 -- 172.21.15.17:0/1040420738 wait complete. 2024-03-06T07:27:44.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.726+0000 7fbd4f365640 1 Processor -- start 2024-03-06T07:27:44.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.726+0000 7fbd4f365640 1 -- start start 2024-03-06T07:27:44.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd30003f20 0x7fbd48152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd480c3d50 0x7fbd480c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480c7c30 con 0x7fbd4814bd70 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480c5a50 con 0x7fbd480c3d50 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4f365640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd480c5bd0 con 0x7fbd30003f20 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4c900640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4c900640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57968/0 (socket says 172.21.15.17:57968) 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4c900640 1 -- 172.21.15.17:0/3052116124 learned_addr learned my addr 172.21.15.17:0/3052116124 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4d101640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd30003f20 0x7fbd48152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.727+0000 7fbd4d902640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd480c3d50 0x7fbd480c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 -- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd30003f20 msgr2=0x7fbd48152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd30003f20 0x7fbd48152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 -- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd480c3d50 msgr2=0x7fbd480c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd480c3d50 0x7fbd480c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 -- 172.21.15.17:0/3052116124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd481ebcb0 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4c900640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c02c400 tx=0x7fbd3c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c007cc0 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c02cbc0 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd481ebf90 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c007070 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.728+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd480adb40 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.729+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4804f330 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.730+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd3c048020 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.730+0000 7fbd45ffb640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 0x7fbd3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.730+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd3c0ab720 con 0x7fbd4814bd70 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.730+0000 7fbd4d101640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 0x7fbd3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:44.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.731+0000 7fbd4d101640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 0x7fbd3406d9c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009bc0 tx=0x7fbd40009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:44.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.734+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd3c07a560 con 0x7fbd4814bd70 2024-03-06T07:27:44.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.951+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd480c8d50 con 0x7fbd4814bd70 2024-03-06T07:27:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.951+0000 7fbd45ffb640 1 -- 172.21.15.17:0/3052116124 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7fbd3c079e60 con 0x7fbd4814bd70 2024-03-06T07:27:44.953 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:44.953 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:44.953 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 msgr2=0x7fbd3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 0x7fbd3406d9c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009bc0 tx=0x7fbd40009190 comp rx=0 tx=0).stop 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 msgr2=0x7fbd480ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c02c400 tx=0x7fbd3c0019c0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 shutdown_connections 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd3406b500 0x7fbd3406d9c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd480c3d50 0x7fbd480c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd4814bd70 0x7fbd480ad600 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 --2- 172.21.15.17:0/3052116124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd30003f20 0x7fbd48152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:44.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 >> 172.21.15.17:0/3052116124 conn(0x7fbd480ac570 msgr2=0x7fbd4814cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:44.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 shutdown_connections 2024-03-06T07:27:44.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:44.953+0000 7fbd4f365640 1 -- 172.21.15.17:0/3052116124 wait complete. 2024-03-06T07:27:45.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:45.368 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:46.369 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.480+0000 7ffb8f45c640 1 Processor -- start 2024-03-06T07:27:46.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.480+0000 7ffb8f45c640 1 -- start start 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8f45c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffb8814f360 0x7ffb88155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8f45c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880c73d0 con 0x7ffb88158b60 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8f45c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880be2e0 con 0x7ffb880c88c0 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8f45c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb88159700 con 0x7ffb8814f360 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8d9f9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffb88158b60 0x7ffb88158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41526/0 (socket says 172.21.15.17:41526) 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.481+0000 7ffb8d9f9640 1 -- 172.21.15.17:0/1087093370 learned_addr learned my addr 172.21.15.17:0/1087093370 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8d1f8640 1 --2- 172.21.15.17:0/1087093370 >> v2:172.21.15.163:3300/0 conn(0x7ffb8814f360 0x7ffb88155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090145560 0 0) 0x7ffb880c73d0 con 0x7ffb88158b60 2024-03-06T07:27:46.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb74003520 con 0x7ffb88158b60 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966106772 0 0) 0x7ffb74003520 con 0x7ffb88158b60 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 >> v2:172.21.15.163:3300/0 conn(0x7ffb8814f360 msgr2=0x7ffb88155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 --2- 172.21.15.17:0/1087093370 >> v2:172.21.15.163:3300/0 conn(0x7ffb8814f360 0x7ffb88155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 >> v1:172.21.15.163:6789/0 conn(0x7ffb880c88c0 legacy=0x7ffb880c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880c7670 con 0x7ffb88158b60 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ffb800019f0 con 0x7ffb88158b60 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 >> v1:172.21.15.17:6789/0 conn(0x7ffb88158b60 legacy=0x7ffb88158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb740039b0 0x7ffb74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88159700 con 0x7ffb74004800 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400c160 con 0x7ffb740051c0 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7400c360 con 0x7ffb740039b0 2024-03-06T07:27:46.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8d1f8640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb740039b0 0x7ffb74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8d9f9640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb740039b0 msgr2=0x7ffb74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb740039b0 0x7ffb74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb740051c0 msgr2=0x7ffb7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8d9f9640 1 -- 172.21.15.17:0/1087093370 reap_dead start 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.482+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1087093370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74003520 con 0x7ffb74004800 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.483+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7ffb78001720 tx=0x7ffb74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6aac2180ec9fb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.483+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78001370 con 0x7ffb74004800 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.483+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb780052d0 con 0x7ffb74004800 2024-03-06T07:27:46.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.483+0000 7ffb7ffff640 1 -- 172.21.15.17:0/1087093370 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003830 con 0x7ffb74004800 2024-03-06T07:27:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 msgr2=0x7ffb74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7ffb78001720 tx=0x7ffb74005830 comp rx=0 tx=0).stop 2024-03-06T07:27:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1087093370 shutdown_connections 2024-03-06T07:27:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1087093370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1087093370 >> 172.21.15.17:0/1087093370 conn(0x7ffb880ac570 msgr2=0x7ffb880ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1087093370 shutdown_connections 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.484+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1087093370 wait complete. 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.485+0000 7ffb8f45c640 1 Processor -- start 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.485+0000 7ffb8f45c640 1 -- start start 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb74004800 0x7ffb881ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb880c1070 0x7ffb8814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880c7c30 con 0x7ffb881ee950 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88155c10 con 0x7ffb880c1070 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8f45c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb88155d90 con 0x7ffb74004800 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57990/0 (socket says 172.21.15.17:57990) 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1205388020 learned_addr learned my addr 172.21.15.17:0/1205388020 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8d1f8640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb74004800 0x7ffb881ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8d9f9640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb880c1070 0x7ffb8814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb74004800 msgr2=0x7ffb881ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb74004800 0x7ffb881ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb880c1070 msgr2=0x7ffb8814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb880c1070 0x7ffb8814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.486+0000 7ffb8c9f7640 1 -- 172.21.15.17:0/1205388020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb881ec220 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb8c9f7640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ffb78005040 tx=0x7ffb78001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78041020 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78005480 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb881eee90 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78007a30 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.487+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb881ec7c0 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.488+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8804f330 con 0x7ffb881ee950 2024-03-06T07:27:46.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.488+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb78044020 con 0x7ffb881ee950 2024-03-06T07:27:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.489+0000 7ffb7dffb640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 0x7ffb6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.489+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb780abd20 con 0x7ffb881ee950 2024-03-06T07:27:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.489+0000 7ffb8d1f8640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 0x7ffb6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:46.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.490+0000 7ffb8d1f8640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 0x7ffb6c06d9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009b80 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:46.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.493+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb7804b030 con 0x7ffb881ee950 2024-03-06T07:27:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.715+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb880c8d50 con 0x7ffb881ee950 2024-03-06T07:27:46.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.716+0000 7ffb7dffb640 1 -- 172.21.15.17:0/1205388020 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7ffb780303c0 con 0x7ffb881ee950 2024-03-06T07:27:46.718 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:46.718 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:46.718 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:46.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 msgr2=0x7ffb6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 0x7ffb6c06d9c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009b80 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 msgr2=0x7ffb8814bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ffb78005040 tx=0x7ffb78001120 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 shutdown_connections 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffb6c06b500 0x7ffb6c06d9c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffb880c1070 0x7ffb8814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb881ee950 0x7ffb8814bbe0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 --2- 172.21.15.17:0/1205388020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffb74004800 0x7ffb881ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 >> 172.21.15.17:0/1205388020 conn(0x7ffb880ac570 msgr2=0x7ffb880aadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 shutdown_connections 2024-03-06T07:27:46.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:46.718+0000 7ffb8f45c640 1 -- 172.21.15.17:0/1205388020 wait complete. 2024-03-06T07:27:47.132 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:47.133 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:48.133 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:48.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.245+0000 7f5f1a834640 1 Processor -- start 2024-03-06T07:27:48.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.245+0000 7f5f1a834640 1 -- start start 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f1a834640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f140c6780 0x7f5f14158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f1a834640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f14072630 con 0x7f5f1406da40 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f1a834640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f14072910 con 0x7f5f141592d0 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f1a834640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1406de20 con 0x7f5f140c6780 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f137fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5f140c6780 0x7f5f14158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f13fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5f1406da40 0x7f5f14155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41542/0 (socket says 172.21.15.17:41542) 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f13fff640 1 -- 172.21.15.17:0/2003861454 learned_addr learned my addr 172.21.15.17:0/2003861454 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f137fe640 1 -- 172.21.15.17:0/2003861454 >> v1:172.21.15.163:6789/0 conn(0x7f5f141592d0 legacy=0x7f5f140c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:48.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f137fe640 1 -- 172.21.15.17:0/2003861454 >> v1:172.21.15.17:6789/0 conn(0x7f5f1406da40 legacy=0x7f5f14155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f137fe640 1 -- 172.21.15.17:0/2003861454 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14072bf0 con 0x7f5f140c6780 2024-03-06T07:27:48.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.246+0000 7f5f12ffd640 1 -- 172.21.15.17:0/2003861454 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378810776 0 0) 0x7f5f14072630 con 0x7f5f1406da40 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.247+0000 7f5f137fe640 1 --2- 172.21.15.17:0/2003861454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f140c6780 0x7f5f14158d90 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f5f040097a0 tx=0x7f5f0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad49d9bee1fc3a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.247+0000 7f5f12ffd640 1 -- 172.21.15.17:0/2003861454 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040315a0 con 0x7f5f140c6780 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.247+0000 7f5f12ffd640 1 -- 172.21.15.17:0/2003861454 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04037070 con 0x7f5f140c6780 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.247+0000 7f5f12ffd640 1 -- 172.21.15.17:0/2003861454 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04031960 con 0x7f5f140c6780 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 -- 172.21.15.17:0/2003861454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f140c6780 msgr2=0x7f5f14158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 --2- 172.21.15.17:0/2003861454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f140c6780 0x7f5f14158d90 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f5f040097a0 tx=0x7f5f0402b840 comp rx=0 tx=0).stop 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 -- 172.21.15.17:0/2003861454 shutdown_connections 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 --2- 172.21.15.17:0/2003861454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f140c6780 0x7f5f14158d90 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 -- 172.21.15.17:0/2003861454 >> 172.21.15.17:0/2003861454 conn(0x7f5f140aa630 msgr2=0x7f5f140aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 -- 172.21.15.17:0/2003861454 shutdown_connections 2024-03-06T07:27:48.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.248+0000 7f5f1a834640 1 -- 172.21.15.17:0/2003861454 wait complete. 2024-03-06T07:27:48.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.249+0000 7f5f1a834640 1 Processor -- start 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.249+0000 7f5f1a834640 1 -- start start 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f140c6780 0x7f5f1406c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f141592d0 0x7f5f140a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c3b10 con 0x7f5f1406da40 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c3990 con 0x7f5f141592d0 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f1a834640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140c7380 con 0x7f5f140c6780 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57994/0 (socket says 172.21.15.17:57994) 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 -- 172.21.15.17:0/11603387 learned_addr learned my addr 172.21.15.17:0/11603387 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f18dd1640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f141592d0 0x7f5f140a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f137fe640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f140c6780 0x7f5f1406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 -- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f140c6780 msgr2=0x7f5f1406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f140c6780 0x7f5f1406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 -- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f141592d0 msgr2=0x7f5f140a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f141592d0 0x7f5f140a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 -- 172.21.15.17:0/11603387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f140aa170 con 0x7f5f1406da40 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.250+0000 7f5f13fff640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.251+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08018070 con 0x7f5f1406da40 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.251+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08007580 con 0x7f5f1406da40 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.251+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f141ebfa0 con 0x7f5f1406da40 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.251+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08004a20 con 0x7f5f1406da40 2024-03-06T07:27:48.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.251+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f141ec240 con 0x7f5f1406da40 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.252+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1404f330 con 0x7f5f1406da40 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.252+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f0801b070 con 0x7f5f1406da40 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.252+0000 7f5f10ff9640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.253+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f08089650 con 0x7f5f1406da40 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.253+0000 7f5f137fe640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:48.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.253+0000 7f5f137fe640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 0x7f5f0006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5f04036000 tx=0x7f5f0403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:48.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.257+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5f080583d0 con 0x7f5f1406da40 2024-03-06T07:27:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.475+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f141541d0 con 0x7f5f1406da40 2024-03-06T07:27:48.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.476+0000 7f5f10ff9640 1 -- 172.21.15.17:0/11603387 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f5f08057cd0 con 0x7f5f1406da40 2024-03-06T07:27:48.477 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:48.478 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:48.478 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:48.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 msgr2=0x7f5f0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 0x7f5f0006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5f04036000 tx=0x7f5f0403d040 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 msgr2=0x7f5f1414e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c500 tx=0x7f5f0800c9c0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 shutdown_connections 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5f0006b4d0 0x7f5f0006d990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f141592d0 0x7f5f140a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f140c6780 0x7f5f1406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 --2- 172.21.15.17:0/11603387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1406da40 0x7f5f1414e640 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 >> 172.21.15.17:0/11603387 conn(0x7f5f140aa630 msgr2=0x7f5f141546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 shutdown_connections 2024-03-06T07:27:48.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:48.478+0000 7f5f1a834640 1 -- 172.21.15.17:0/11603387 wait complete. 2024-03-06T07:27:48.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:48.892 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:49.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.005+0000 7ff1e2b5e640 1 Processor -- start 2024-03-06T07:27:50.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.005+0000 7ff1e2b5e640 1 -- start start 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1e2b5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1dc0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1e2b5e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0c7c60 con 0x7ff1dc0c7180 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1e2b5e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0c8060 con 0x7ff1dc1527b0 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1e2b5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0c8df0 con 0x7ff1dc159100 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1dbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff1dc0c7180 0x7ff1dc0c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:41556/0 (socket says 172.21.15.17:41556) 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.006+0000 7ff1dbfff640 1 -- 172.21.15.17:0/2726571542 learned_addr learned my addr 172.21.15.17:0/2726571542 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e10fb640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1dc0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278701911 0 0) 0x7ff1d0001110 con 0x7ff1dc0c7180 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c4003520 con 0x7ff1dc0c7180 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089834206 0 0) 0x7ff1dc0c8060 con 0x7ff1dc1527b0 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d0001110 con 0x7ff1dc1527b0 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783285217 0 0) 0x7ff1c4003520 con 0x7ff1dc0c7180 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 msgr2=0x7ff1dc0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1dc0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 >> v1:172.21.15.163:6789/0 conn(0x7ff1dc1527b0 legacy=0x7ff1dc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc1595b0 con 0x7ff1dc0c7180 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff1d0001840 con 0x7ff1dc0c7180 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 >> v1:172.21.15.17:6789/0 conn(0x7ff1dc0c7180 legacy=0x7ff1dc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1c4003b70 0x7ff1c4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4004800 0x7ff1c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1c4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0c8df0 con 0x7ff1c4003b70 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c400bb50 con 0x7ff1c4004800 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c400bd20 con 0x7ff1dc159100 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1c4003b70 0x7ff1c4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1dbfff640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4004800 0x7ff1c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e10fb640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1c4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 -- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 msgr2=0x7ff1c4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc159100 0x7ff1c4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 -- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4004800 msgr2=0x7ff1c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4004800 0x7ff1c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.007+0000 7ff1e08fa640 1 -- 172.21.15.17:0/2726571542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c4003520 con 0x7ff1c4003b70 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.008+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1c4003b70 0x7ff1c4003f70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4002000 tx=0x7ff1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c5cc2956a80399 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.008+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4001bf0 con 0x7ff1c4003b70 2024-03-06T07:27:50.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.008+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d4003b70 con 0x7ff1c4003b70 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.008+0000 7ff1db7fe640 1 -- 172.21.15.17:0/2726571542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40050d0 con 0x7ff1c4003b70 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2726571542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1c4003b70 msgr2=0x7ff1c4003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2726571542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1c4003b70 0x7ff1c4003f70 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4002000 tx=0x7ff1c4005830 comp rx=0 tx=0).stop 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e10fb640 1 -- 172.21.15.17:0/2726571542 reap_dead start 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2726571542 shutdown_connections 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2726571542 >> 172.21.15.17:0/2726571542 conn(0x7ff1dc0ac560 msgr2=0x7ff1dc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2726571542 shutdown_connections 2024-03-06T07:27:50.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.009+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2726571542 wait complete. 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.010+0000 7ff1e2b5e640 1 Processor -- start 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.010+0000 7ff1e2b5e640 1 -- start start 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4003b70 0x7ff1dc1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc154c80 0x7ff1dc1ea560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc155320 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc0c7980 con 0x7ff1c4003b70 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e2b5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc155600 con 0x7ff1dc154c80 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58010/0 (socket says 172.21.15.17:58010) 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 -- 172.21.15.17:0/2082002421 learned_addr learned my addr 172.21.15.17:0/2082002421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:50.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1dbfff640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc154c80 0x7ff1dc1ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4003b70 0x7ff1dc1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 -- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc154c80 msgr2=0x7ff1dc1ea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc154c80 0x7ff1dc1ea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 -- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4003b70 msgr2=0x7ff1dc1513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4003b70 0x7ff1dc1513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.011+0000 7ff1e10fb640 1 -- 172.21.15.17:0/2082002421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc153f80 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1e10fb640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0aa480 tx=0x7ff1cc011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003610 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc01b070 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc1ee830 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0039f0 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.012+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc151900 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.013+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc04f330 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.013+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1cc003b90 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.014+0000 7ff1d97fa640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 0x7ff1c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:50.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.014+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1cc08a310 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.014+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 0x7ff1c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:50.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.015+0000 7ff1e08fa640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 0x7ff1c806d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4034000 tx=0x7ff1d4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:50.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.018+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1cc059000 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.240+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1dc1531d0 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.240+0000 7ff1d97fa640 1 -- 172.21.15.17:0/2082002421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7ff1cc058900 con 0x7ff1dc1eaaa0 2024-03-06T07:27:50.242 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:50.242 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:50.242 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:50.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 msgr2=0x7ff1c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 0x7ff1c806d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4034000 tx=0x7ff1d4009030 comp rx=0 tx=0).stop 2024-03-06T07:27:50.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 msgr2=0x7ff1dc14ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0aa480 tx=0x7ff1cc011920 comp rx=0 tx=0).stop 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 shutdown_connections 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.242+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff1c806b4d0 0x7ff1c806d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1dc1eaaa0 0x7ff1dc14ba40 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1dc154c80 0x7ff1dc1ea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 --2- 172.21.15.17:0/2082002421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1c4003b70 0x7ff1dc1513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 >> 172.21.15.17:0/2082002421 conn(0x7ff1dc0ac560 msgr2=0x7ff1dc1589b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 shutdown_connections 2024-03-06T07:27:50.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:50.243+0000 7ff1e2b5e640 1 -- 172.21.15.17:0/2082002421 wait complete. 2024-03-06T07:27:50.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:50.658 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:51.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.770+0000 7f4b9873e640 1 Processor -- start 2024-03-06T07:27:51.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.770+0000 7f4b9873e640 1 -- start start 2024-03-06T07:27:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b9873e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b900c3150 0x7f4b900c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b9873e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b90072440 con 0x7f4b90152c20 2024-03-06T07:27:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b9873e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900c2b10 con 0x7f4b9014f650 2024-03-06T07:27:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b9873e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b90152140 con 0x7f4b900c3150 2024-03-06T07:27:51.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b95cd9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4b900c3150 0x7f4b900c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b96cdb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4b90152c20 0x7f4b9015c810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53662/0 (socket says 172.21.15.17:53662) 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.771+0000 7f4b96cdb640 1 -- 172.21.15.17:0/2049783651 learned_addr learned my addr 172.21.15.17:0/2049783651 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.772+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2049783651 >> v1:172.21.15.163:6789/0 conn(0x7f4b9014f650 legacy=0x7f4b90071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.772+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2049783651 >> v1:172.21.15.17:6789/0 conn(0x7f4b90152c20 legacy=0x7f4b9015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.772+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2049783651 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b90152420 con 0x7f4b900c3150 2024-03-06T07:27:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.772+0000 7f4b954d8640 1 -- 172.21.15.17:0/2049783651 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543569598 0 0) 0x7f4b8c001490 con 0x7f4b90152c20 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.772+0000 7f4b95cd9640 1 --2- 172.21.15.17:0/2049783651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900c3150 0x7f4b900c5690 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c385eb1c0c23d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b954d8640 1 -- 172.21.15.17:0/2049783651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80035020 con 0x7f4b900c3150 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b954d8640 1 -- 172.21.15.17:0/2049783651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b80031830 con 0x7f4b900c3150 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b954d8640 1 -- 172.21.15.17:0/2049783651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80031c00 con 0x7f4b900c3150 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 -- 172.21.15.17:0/2049783651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900c3150 msgr2=0x7f4b900c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2049783651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900c3150 0x7f4b900c5690 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f4b800097a0 tx=0x7f4b8002b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 -- 172.21.15.17:0/2049783651 shutdown_connections 2024-03-06T07:27:51.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2049783651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b900c3150 0x7f4b900c5690 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 -- 172.21.15.17:0/2049783651 >> 172.21.15.17:0/2049783651 conn(0x7f4b900c4290 msgr2=0x7f4b900c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:51.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 -- 172.21.15.17:0/2049783651 shutdown_connections 2024-03-06T07:27:51.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.773+0000 7f4b9873e640 1 -- 172.21.15.17:0/2049783651 wait complete. 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.774+0000 7f4b9873e640 1 Processor -- start 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.774+0000 7f4b9873e640 1 -- start start 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 0x7f4b90149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b90152c20 0x7f4b9014bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90072110 con 0x7f4b9014f650 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b90071f90 con 0x7f4b900c3150 2024-03-06T07:27:51.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b9873e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900c3f40 con 0x7f4b90152c20 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b95cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b95cd9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60264/0 (socket says 172.21.15.17:60264) 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2805750431 learned_addr learned my addr 172.21.15.17:0/2805750431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b96cdb640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b90152c20 0x7f4b9014bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.775+0000 7f4b964da640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 0x7f4b90149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b90152c20 msgr2=0x7f4b9014bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b90152c20 0x7f4b9014bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 msgr2=0x7f4b90149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 0x7f4b90149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 -- 172.21.15.17:0/2805750431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b901ed0a0 con 0x7f4b9014f650 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b964da640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 0x7f4b90149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b95cd9640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098c0 tx=0x7f4b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80041070 con 0x7f4b9014f650 2024-03-06T07:27:51.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8003aa30 con 0x7f4b9014f650 2024-03-06T07:27:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b90154ff0 con 0x7f4b9014f650 2024-03-06T07:27:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b80035040 con 0x7f4b9014f650 2024-03-06T07:27:51.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.776+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b901ecdc0 con 0x7f4b9014f650 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.777+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9004f330 con 0x7f4b9014f650 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.777+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b80038020 con 0x7f4b9014f650 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.778+0000 7f4b86ffd640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 0x7f4b7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.778+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b800abd30 con 0x7f4b9014f650 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.778+0000 7f4b964da640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 0x7f4b7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:51.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.779+0000 7f4b964da640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 0x7f4b7406d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007fd0 tx=0x7f4b88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:51.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:51.782+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b8007a460 con 0x7f4b9014f650 2024-03-06T07:27:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.000+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b90068460 con 0x7f4b9014f650 2024-03-06T07:27:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.000+0000 7f4b86ffd640 1 -- 172.21.15.17:0/2805750431 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+4985 (secure 0 0 0) 0x7f4b8007e500 con 0x7f4b9014f650 2024-03-06T07:27:52.002 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:52.002 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":175,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:37.568556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0],"up":{"mds_0":9900},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9900":{"gid":9900,"name":"a","rank":0,"incarnation":171,"state":"up:stopping","state_seq":3,"addr":"172.21.15.17:6835/445432625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":445432625},{"type":"v1","addr":"172.21.15.17:6835","nonce":445432625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 175 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 msgr2=0x7f4b7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 0x7f4b7406d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007fd0 tx=0x7f4b88009040 comp rx=0 tx=0).stop 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 msgr2=0x7f4b900ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f4b800098c0 tx=0x7f4b800046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 shutdown_connections 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4b7406b4d0 0x7f4b7406d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:52.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4b90152c20 0x7f4b9014bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:52.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b9014f650 0x7f4b900ad550 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:52.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 --2- 172.21.15.17:0/2805750431 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4b900c3150 0x7f4b90149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:52.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 >> 172.21.15.17:0/2805750431 conn(0x7f4b900c4290 msgr2=0x7f4b9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:52.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 shutdown_connections 2024-03-06T07:27:52.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:52.002+0000 7f4b9873e640 1 -- 172.21.15.17:0/2805750431 wait complete. 2024-03-06T07:27:52.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 175, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:37.568556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0], 'up': {'mds_0': 9900}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9900': {'gid': 9900, 'name': 'a', 'rank': 0, 'incarnation': 171, 'state': 'up:stopping', 'state_seq': 3, 'addr': '172.21.15.17:6835/445432625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 445432625}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 445432625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:52.418 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9900:up:stopping 2024-03-06T07:27:52.869 INFO:tasks.ceph.mds.a.smithi017.stderr: -808> 2024-03-06T07:27:32.501+0000 7fe2a188a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:52.897 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:27:53.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.531+0000 7f80a4532640 1 Processor -- start 2024-03-06T07:27:53.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.531+0000 7f80a4532640 1 -- start start 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a4532640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f809c06d9f0 0x7f809c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a4532640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f809c0c3710 0x7f809c1558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a4532640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809c158800 con 0x7f809c0c3280 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a4532640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c158ae0 con 0x7f809c0c3710 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a4532640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809c158dc0 con 0x7f809c06d9f0 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a2acf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f809c0c3710 0x7f809c1558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a1acd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f809c0c3280 0x7f809c1551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53688/0 (socket says 172.21.15.17:53688) 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a1acd640 1 -- 172.21.15.17:0/4226695218 learned_addr learned my addr 172.21.15.17:0/4226695218 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a22ce640 1 --2- 172.21.15.17:0/4226695218 >> v2:172.21.15.163:3300/0 conn(0x7f809c06d9f0 0x7f809c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a2acf640 1 -- 172.21.15.17:0/4226695218 >> v2:172.21.15.163:3300/0 conn(0x7f809c06d9f0 msgr2=0x7f809c0a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a2acf640 1 --2- 172.21.15.17:0/4226695218 >> v2:172.21.15.163:3300/0 conn(0x7f809c06d9f0 0x7f809c0a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a2acf640 1 -- 172.21.15.17:0/4226695218 >> v1:172.21.15.17:6789/0 conn(0x7f809c0c3280 legacy=0x7f809c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a2acf640 1 -- 172.21.15.17:0/4226695218 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c1590a0 con 0x7f809c0c3710 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.532+0000 7f80a12cc640 1 -- 172.21.15.17:0/4226695218 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329119555 0 0) 0x7f809c158800 con 0x7f809c0c3280 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.533+0000 7f80a2acf640 1 --2- 172.21.15.17:0/4226695218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c0c3710 0x7f809c1558f0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f809c0c9820 tx=0x7f809402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac106e8aa5539138 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.533+0000 7f80a12cc640 1 -- 172.21.15.17:0/4226695218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094039070 con 0x7f809c0c3710 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.533+0000 7f80a12cc640 1 -- 172.21.15.17:0/4226695218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809402cbf0 con 0x7f809c0c3710 2024-03-06T07:27:53.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.533+0000 7f80a12cc640 1 -- 172.21.15.17:0/4226695218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094034440 con 0x7f809c0c3710 2024-03-06T07:27:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 -- 172.21.15.17:0/4226695218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c0c3710 msgr2=0x7f809c1558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 --2- 172.21.15.17:0/4226695218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c0c3710 0x7f809c1558f0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f809c0c9820 tx=0x7f809402be40 comp rx=0 tx=0).stop 2024-03-06T07:27:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 -- 172.21.15.17:0/4226695218 shutdown_connections 2024-03-06T07:27:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 --2- 172.21.15.17:0/4226695218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c0c3710 0x7f809c1558f0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 --2- 172.21.15.17:0/4226695218 >> v2:172.21.15.163:3300/0 conn(0x7f809c06d9f0 0x7f809c0a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 -- 172.21.15.17:0/4226695218 >> 172.21.15.17:0/4226695218 conn(0x7f809c0aa630 msgr2=0x7f809c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 -- 172.21.15.17:0/4226695218 shutdown_connections 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.534+0000 7f80a4532640 1 -- 172.21.15.17:0/4226695218 wait complete. 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.535+0000 7f80a4532640 1 Processor -- start 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.535+0000 7f80a4532640 1 -- start start 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 0x7f809c0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 0x7f809c1448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c06ddd0 con 0x7f809c06d9f0 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c155e30 con 0x7f809c0c3710 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c06df50 con 0x7f809c0c3280 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60298/0 (socket says 172.21.15.17:60298) 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 -- 172.21.15.17:0/4182687582 learned_addr learned my addr 172.21.15.17:0/4182687582 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a1acd640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 0x7f809c0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a2acf640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 0x7f809c1448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 -- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 msgr2=0x7f809c0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 0x7f809c0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 -- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 msgr2=0x7f809c1448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 0x7f809c1448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 -- 172.21.15.17:0/4182687582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c1ef040 con 0x7f809c06d9f0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a2acf640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 0x7f809c1448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a1acd640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 0x7f809c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a22ce640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f809800d800 tx=0x7f809800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800b6c0 con 0x7f809c06d9f0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8098019070 con 0x7f809c06d9f0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c0ab9c0 con 0x7f809c06d9f0 2024-03-06T07:27:53.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.536+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800bac0 con 0x7f809c06d9f0 2024-03-06T07:27:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.537+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c0ac820 con 0x7f809c06d9f0 2024-03-06T07:27:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.538+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c04f330 con 0x7f809c06d9f0 2024-03-06T07:27:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.538+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f809800bc60 con 0x7f809c06d9f0 2024-03-06T07:27:53.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.538+0000 7f8092ffd640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.539+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f809808a7b0 con 0x7f809c06d9f0 2024-03-06T07:27:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.539+0000 7f80a1acd640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:53.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.539+0000 7f80a1acd640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 0x7f808006d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f809c068eb0 tx=0x7f808c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:53.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.543+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f809c1ef040 con 0x7f809c06d9f0 2024-03-06T07:27:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.761+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f809c0aec10 con 0x7f809c06d9f0 2024-03-06T07:27:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.762+0000 7f8092ffd640 1 -- 172.21.15.17:0/4182687582 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+4184 (secure 0 0 0) 0x7f8098059530 con 0x7f809c06d9f0 2024-03-06T07:27:53.764 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:53.764 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":176,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:52.343462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:53.764 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 176 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 msgr2=0x7f808006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 0x7f808006d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f809c068eb0 tx=0x7f808c009040 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 msgr2=0x7f809c1eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f809800d800 tx=0x7f809800dcc0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 shutdown_connections 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808006b500 0x7f808006d9c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f809c0c3710 0x7f809c1448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f809c0c3280 0x7f809c0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 --2- 172.21.15.17:0/4182687582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f809c06d9f0 0x7f809c1eeb00 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 >> 172.21.15.17:0/4182687582 conn(0x7f809c0aa630 msgr2=0x7f809c153220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 shutdown_connections 2024-03-06T07:27:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:53.764+0000 7f80a4532640 1 -- 172.21.15.17:0/4182687582 wait complete. 2024-03-06T07:27:54.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 176, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:52.343462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [], 'up': {}, 'failed': [], 'damaged': [], 'stopped': [0], 'info': {}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:54.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/0 2024-03-06T07:27:54.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.281+0000 7fc7fcc66640 1 Processor -- start 2024-03-06T07:27:54.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.281+0000 7fc7fcc66640 1 -- start start 2024-03-06T07:27:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7fcc66640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc7f80c5330 0x7fc7f80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7fcc66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f8155110 con 0x7fc7f80c88b0 2024-03-06T07:27:54.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7fcc66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f80c8f40 con 0x7fc7f8158db0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7fcc66640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f81559e0 con 0x7fc7f80c5330 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7f5d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc7f80c88b0 0x7fc7f81586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53692/0 (socket says 172.21.15.17:53692) 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/1812684369 learned_addr learned my addr 172.21.15.17:0/1812684369 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7f659c640 1 --2- 172.21.15.17:0/1812684369 >> v2:172.21.15.163:3300/0 conn(0x7fc7f80c5330 0x7fc7f80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.282+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055820481 0 0) 0x7fc7ec001110 con 0x7fc7f80c88b0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e0003520 con 0x7fc7f80c88b0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3976165935 0 0) 0x7fc7e0003520 con 0x7fc7f80c88b0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 >> v2:172.21.15.163:3300/0 conn(0x7fc7f80c5330 msgr2=0x7fc7f80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 --2- 172.21.15.17:0/1812684369 >> v2:172.21.15.163:3300/0 conn(0x7fc7f80c5330 0x7fc7f80c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 >> v1:172.21.15.163:6789/0 conn(0x7fc7f8158db0 legacy=0x7fc7f815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8155390 con 0x7fc7f80c88b0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc7ec001840 con 0x7fc7f80c88b0 2024-03-06T07:27:54.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 >> v1:172.21.15.17:6789/0 conn(0x7fc7f80c88b0 legacy=0x7fc7f81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e00051c0 0x7fc7e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81559e0 con 0x7fc7e0004800 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e000c160 con 0x7fc7e00051c0 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e000c360 con 0x7fc7e00039b0 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f659c640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f6d9d640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e00051c0 0x7fc7e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7e00039b0 msgr2=0x7fc7e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e00051c0 msgr2=0x7fc7e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:54.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e00051c0 0x7fc7e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/1812684369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0003520 con 0x7fc7e0004800 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.283+0000 7fc7f6d9d640 1 -- 172.21.15.17:0/1812684369 reap_dead start 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.284+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c490 tx=0x7fc7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a12fea261e4928 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.284+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec00c0e0 con 0x7fc7e0004800 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.284+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec030070 con 0x7fc7e0004800 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.284+0000 7fc7f559a640 1 -- 172.21.15.17:0/1812684369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0058d0 con 0x7fc7e0004800 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 -- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 msgr2=0x7fc7e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c490 tx=0x7fc7e0005830 comp rx=0 tx=0).stop 2024-03-06T07:27:54.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 -- 172.21.15.17:0/1812684369 shutdown_connections 2024-03-06T07:27:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/1812684369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7e0004800 0x7fc7e0004c80 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 -- 172.21.15.17:0/1812684369 >> 172.21.15.17:0/1812684369 conn(0x7fc7f80ac560 msgr2=0x7fc7f80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 -- 172.21.15.17:0/1812684369 shutdown_connections 2024-03-06T07:27:54.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.285+0000 7fc7fcc66640 1 -- 172.21.15.17:0/1812684369 wait complete. 2024-03-06T07:27:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.286+0000 7fc7fcc66640 1 Processor -- start 2024-03-06T07:27:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.286+0000 7fc7fcc66640 1 -- start start 2024-03-06T07:27:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e0004800 0x7fc7f80c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7f80c1dc0 0x7fc7f80c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f81529d0 con 0x7fc7f8157a20 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f806d090 con 0x7fc7e0004800 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f8152b50 con 0x7fc7f80c1dc0 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60326/0 (socket says 172.21.15.17:60326) 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/2882064089 learned_addr learned my addr 172.21.15.17:0/2882064089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f659c640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e0004800 0x7fc7f80c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7f80c1dc0 msgr2=0x7fc7f80c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7f80c1dc0 0x7fc7f80c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e0004800 msgr2=0x7fc7f80c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e0004800 0x7fc7f80c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 -- 172.21.15.17:0/2882064089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80bde00 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7f5d9b640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c5b0 tx=0x7fc7f8155950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec004bf0 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec004d90 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f8156c60 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec007270 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.287+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f8157220 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.288+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f804f330 con 0x7fc7f8157a20 2024-03-06T07:27:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.289+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7ec007410 con 0x7fc7f8157a20 2024-03-06T07:27:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.289+0000 7fc7e6ffd640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 0x7fc7d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.289+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc7ec0abb40 con 0x7fc7f8157a20 2024-03-06T07:27:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.289+0000 7fc7f659c640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 0x7fc7d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.290+0000 7fc7f659c640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 0x7fc7d806d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81ef080 tx=0x7fc7f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.294+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc7f8155110 con 0x7fc7f8157a20 2024-03-06T07:27:54.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.511+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7f806cdc0 con 0x7fc7f8157a20 2024-03-06T07:27:54.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.512+0000 7fc7e6ffd640 1 -- 172.21.15.17:0/2882064089 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+4977 (secure 0 0 0) 0x7fc7ec07a850 con 0x7fc7f8157a20 2024-03-06T07:27:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:54.514 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":177,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":10140,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":177}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:52.343462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:54.514 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 177 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 msgr2=0x7fc7d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 0x7fc7d806d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc7f81ef080 tx=0x7fc7f0008040 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 msgr2=0x7fc7f81514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00c5b0 tx=0x7fc7f8155950 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 shutdown_connections 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc7d806b500 0x7fc7d806d9c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7f80c1dc0 0x7fc7f80c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc7f8157a20 0x7fc7f81514b0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 --2- 172.21.15.17:0/2882064089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7e0004800 0x7fc7f80c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 >> 172.21.15.17:0/2882064089 conn(0x7fc7f80ac560 msgr2=0x7fc7f80c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 shutdown_connections 2024-03-06T07:27:54.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.514+0000 7fc7fcc66640 1 -- 172.21.15.17:0/2882064089 wait complete. 2024-03-06T07:27:54.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-03-06T07:27:54.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 Processor -- start 2024-03-06T07:27:54.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 -- start start 2024-03-06T07:27:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b9014bcc0 0x7f5b90155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 0x7f5b900c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b900c5910 0x7f5b90158a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900c7f10 con 0x7f5b9014bcc0 2024-03-06T07:27:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b900c75e0 con 0x7f5b900c5910 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.630+0000 7f5b97f5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b900c78c0 con 0x7f5b900be150 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b9014bcc0 0x7f5b90155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b9014bcc0 0x7f5b90155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60340/0 (socket says 172.21.15.17:60340) 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 -- 172.21.15.17:0/4168568300 learned_addr learned my addr 172.21.15.17:0/4168568300 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b964fb640 1 --2- 172.21.15.17:0/4168568300 >> v2:172.21.15.163:3300/0 conn(0x7f5b900c5910 0x7f5b90158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b954f9640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 0x7f5b900c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 -- 172.21.15.17:0/4168568300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 msgr2=0x7f5b900c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 0x7f5b900c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 -- 172.21.15.17:0/4168568300 >> v2:172.21.15.163:3300/0 conn(0x7f5b900c5910 msgr2=0x7f5b90158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/4168568300 >> v2:172.21.15.163:3300/0 conn(0x7f5b900c5910 0x7f5b90158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b95cfa640 1 -- 172.21.15.17:0/4168568300 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b900c7ba0 con 0x7f5b9014bcc0 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.631+0000 7f5b954f9640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 0x7f5b900c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:54.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9014bcc0 0x7f5b90155a00 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5b900c7390 tx=0x7f5b8c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee331e11032def6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b94cf8640 1 -- 172.21.15.17:0/4168568300 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c034070 con 0x7f5b9014bcc0 2024-03-06T07:27:54.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b94cf8640 1 -- 172.21.15.17:0/4168568300 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c030860 con 0x7f5b9014bcc0 2024-03-06T07:27:54.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b94cf8640 1 -- 172.21.15.17:0/4168568300 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c030c30 con 0x7f5b9014bcc0 2024-03-06T07:27:54.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 -- 172.21.15.17:0/4168568300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9014bcc0 msgr2=0x7f5b90155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9014bcc0 0x7f5b90155a00 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f5b900c7390 tx=0x7f5b8c02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 -- 172.21.15.17:0/4168568300 shutdown_connections 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/4168568300 >> v2:172.21.15.163:3300/0 conn(0x7f5b900c5910 0x7f5b90158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b900be150 0x7f5b900c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/4168568300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9014bcc0 0x7f5b90155a00 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 -- 172.21.15.17:0/4168568300 >> 172.21.15.17:0/4168568300 conn(0x7f5b900ac600 msgr2=0x7f5b900aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 -- 172.21.15.17:0/4168568300 shutdown_connections 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.632+0000 7f5b97f5e640 1 -- 172.21.15.17:0/4168568300 wait complete. 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.633+0000 7f5b97f5e640 1 Processor -- start 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.633+0000 7f5b97f5e640 1 -- start start 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b900be150 0x7f5b901eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9014bcc0 0x7f5b90149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b901590d0 con 0x7f5b900c5910 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90158f50 con 0x7f5b900be150 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b97f5e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b90159250 con 0x7f5b9014bcc0 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b954f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b954f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60346/0 (socket says 172.21.15.17:60346) 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b954f9640 1 -- 172.21.15.17:0/315005713 learned_addr learned my addr 172.21.15.17:0/315005713 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b900be150 0x7f5b901eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.634+0000 7f5b964fb640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9014bcc0 0x7f5b90149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 -- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9014bcc0 msgr2=0x7f5b90149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9014bcc0 0x7f5b90149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 -- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b900be150 msgr2=0x7f5b901eb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b900be150 0x7f5b901eb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 -- 172.21.15.17:0/315005713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b901eaee0 con 0x7f5b900c5910 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b954f9640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c7f0 tx=0x7f5b8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80015070 con 0x7f5b900c5910 2024-03-06T07:27:54.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b800079d0 con 0x7f5b900c5910 2024-03-06T07:27:54.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b901eb1c0 con 0x7f5b900c5910 2024-03-06T07:27:54.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80007da0 con 0x7f5b900c5910 2024-03-06T07:27:54.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.635+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b900aace0 con 0x7f5b900c5910 2024-03-06T07:27:54.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.636+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9004f330 con 0x7f5b900c5910 2024-03-06T07:27:54.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.636+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b80018070 con 0x7f5b900c5910 2024-03-06T07:27:54.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.637+0000 7f5b867fc640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 0x7f5b7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:54.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.637+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5b8008a1a0 con 0x7f5b900c5910 2024-03-06T07:27:54.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.637+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 0x7f5b7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:54.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.637+0000 7f5b95cfa640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 0x7f5b7406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c036000 tx=0x7f5b8c03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:54.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.641+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b80058860 con 0x7f5b900c5910 2024-03-06T07:27:54.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.862+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f5b901595d0 con 0x7f5b900c5910 2024-03-06T07:27:54.874 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:27:54.873+0000 7f4455898640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:27:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.878+0000 7f5b867fc640 1 -- 172.21.15.17:0/315005713 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v178) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5b8005c900 con 0x7f5b900c5910 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 msgr2=0x7f5b7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 0x7f5b7406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c036000 tx=0x7f5b8c03b040 comp rx=0 tx=0).stop 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 msgr2=0x7f5b900aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f5b8000c7f0 tx=0x7f5b8000ccb0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 shutdown_connections 2024-03-06T07:27:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b7406b4d0 0x7f5b7406d990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9014bcc0 0x7f5b90149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b900c5910 0x7f5b900aa7a0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 --2- 172.21.15.17:0/315005713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b900be150 0x7f5b901eb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.881+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 >> 172.21.15.17:0/315005713 conn(0x7f5b900ac600 msgr2=0x7f5b900ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.882+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 shutdown_connections 2024-03-06T07:27:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.882+0000 7f5b97f5e640 1 -- 172.21.15.17:0/315005713 wait complete. 2024-03-06T07:27:54.898 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.998+0000 7f9245fa0640 1 Processor -- start 2024-03-06T07:27:55.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.998+0000 7f9245fa0640 1 -- start start 2024-03-06T07:27:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9245fa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f924006ef20 0x7f92400c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9245fa0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92400717f0 con 0x7f924014f3e0 2024-03-06T07:27:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9245fa0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9240071ad0 con 0x7f92400c2670 2024-03-06T07:27:55.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9245fa0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924014f7c0 con 0x7f924006ef20 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9244f9e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f924014f3e0 0x7f9240152550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53698/0 (socket says 172.21.15.17:53698) 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f9244f9e640 1 -- 172.21.15.17:0/4138058111 learned_addr learned my addr 172.21.15.17:0/4138058111 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:54.999+0000 7f924579f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f924006ef20 0x7f92400c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f923f7fe640 1 -- 172.21.15.17:0/4138058111 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72368931 0 0) 0x7f92400717f0 con 0x7f924014f3e0 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f923f7fe640 1 -- 172.21.15.17:0/4138058111 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9228003520 con 0x7f924014f3e0 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f924579f640 1 -- 172.21.15.17:0/4138058111 >> v1:172.21.15.163:6789/0 conn(0x7f92400c2670 legacy=0x7f924006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f924579f640 1 -- 172.21.15.17:0/4138058111 >> v1:172.21.15.17:6789/0 conn(0x7f924014f3e0 legacy=0x7f9240152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f924579f640 1 -- 172.21.15.17:0/4138058111 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240071db0 con 0x7f924006ef20 2024-03-06T07:27:55.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.000+0000 7f924579f640 1 --2- 172.21.15.17:0/4138058111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f92400c2b00 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f92300097a0 tx=0x7f923002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c71c7f07238d99 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f923f7fe640 1 -- 172.21.15.17:0/4138058111 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300315a0 con 0x7f924006ef20 2024-03-06T07:27:55.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f923f7fe640 1 -- 172.21.15.17:0/4138058111 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9230037070 con 0x7f924006ef20 2024-03-06T07:27:55.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f923f7fe640 1 -- 172.21.15.17:0/4138058111 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230031940 con 0x7f924006ef20 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 -- 172.21.15.17:0/4138058111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 msgr2=0x7f92400c2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 --2- 172.21.15.17:0/4138058111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f92400c2b00 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f92300097a0 tx=0x7f923002b840 comp rx=0 tx=0).stop 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 -- 172.21.15.17:0/4138058111 shutdown_connections 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 --2- 172.21.15.17:0/4138058111 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f92400c2b00 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 -- 172.21.15.17:0/4138058111 >> 172.21.15.17:0/4138058111 conn(0x7f924006c0d0 msgr2=0x7f924006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 -- 172.21.15.17:0/4138058111 shutdown_connections 2024-03-06T07:27:55.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.001+0000 7f9245fa0640 1 -- 172.21.15.17:0/4138058111 wait complete. 2024-03-06T07:27:55.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.002+0000 7f9245fa0640 1 Processor -- start 2024-03-06T07:27:55.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.002+0000 7f9245fa0640 1 -- start start 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92400c2670 0x7f92400ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 0x7f92401eba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240071450 con 0x7f924006ef20 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400c37d0 con 0x7f924014f3e0 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9245fa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92400715d0 con 0x7f92400c2670 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9244f9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9244f9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60354/0 (socket says 172.21.15.17:60354) 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f9244f9e640 1 -- 172.21.15.17:0/2007227872 learned_addr learned my addr 172.21.15.17:0/2007227872 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f924579f640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 0x7f92401eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.003+0000 7f923ffff640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92400c2670 0x7f92400ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 -- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92400c2670 msgr2=0x7f92400ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92400c2670 0x7f92400ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 -- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 msgr2=0x7f92401eba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 0x7f92401eba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 -- 172.21.15.17:0/2007227872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400aa660 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f924579f640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 0x7f92401eba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9244f9e640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f923800cbf0 tx=0x7f923800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238017070 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800ad40 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92401ec5b0 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238012620 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.004+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92400acd20 con 0x7f924006ef20 2024-03-06T07:27:55.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.005+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92400aab40 con 0x7f924006ef20 2024-03-06T07:27:55.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.006+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f923801a070 con 0x7f924006ef20 2024-03-06T07:27:55.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.006+0000 7f923d7fa640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 0x7f922c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:55.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.006+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f923808a190 con 0x7f924006ef20 2024-03-06T07:27:55.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.006+0000 7f923ffff640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 0x7f922c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:55.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.007+0000 7f923ffff640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 0x7f922c06d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9240152300 tx=0x7f9234009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:55.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.010+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f92400aab40 con 0x7f924006ef20 2024-03-06T07:27:55.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.229+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9240069430 con 0x7f924006ef20 2024-03-06T07:27:55.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.230+0000 7f923d7fa640 1 -- 172.21.15.17:0/2007227872 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+4991 (secure 0 0 0) 0x7f9238058ea0 con 0x7f924006ef20 2024-03-06T07:27:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:55.231 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":179,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":9882,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:54.881867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0],"up":{"mds_0":10140},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:starting","state_seq":1,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":14}]} 2024-03-06T07:27:55.232 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 179 2024-03-06T07:27:55.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 msgr2=0x7f922c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 0x7f922c06d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9240152300 tx=0x7f9234009040 comp rx=0 tx=0).stop 2024-03-06T07:27:55.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 msgr2=0x7f9240153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f923800cbf0 tx=0x7f923800a430 comp rx=0 tx=0).stop 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 shutdown_connections 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.231+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f922c06b4d0 0x7f922c06d990 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f924014f3e0 0x7f92401eba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f92400c2670 0x7f92400ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 --2- 172.21.15.17:0/2007227872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f924006ef20 0x7f9240153f10 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 >> 172.21.15.17:0/2007227872 conn(0x7f924006c0d0 msgr2=0x7f924006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 shutdown_connections 2024-03-06T07:27:55.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:55.232+0000 7f9245fa0640 1 -- 172.21.15.17:0/2007227872 wait complete. 2024-03-06T07:27:55.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 179, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:54.881867+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0], 'up': {'mds_0': 10140}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10140': {'gid': 10140, 'name': 'a', 'rank': 0, 'incarnation': 179, 'state': 'up:starting', 'state_seq': 1, 'addr': '172.21.15.17:6835/2561439920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2561439920}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2561439920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-06T07:27:55.646 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10140:up:starting 2024-03-06T07:27:56.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.747+0000 7faf50a5c640 1 Processor -- start 2024-03-06T07:27:56.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.747+0000 7faf50a5c640 1 -- start start 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf50a5c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faf4c153990 0x7faf4c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf50a5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c070aa0 con 0x7faf4c1506b0 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf50a5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c070d80 con 0x7faf4c0acc00 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf50a5c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c156450 con 0x7faf4c153990 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf4a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faf4c1506b0 0x7faf4c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53702/0 (socket says 172.21.15.17:53702) 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf4a59c640 1 -- 172.21.15.17:0/1570629680 learned_addr learned my addr 172.21.15.17:0/1570629680 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:56.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592473794 0 0) 0x7faf4c070aa0 con 0x7faf4c1506b0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf41d9b640 1 --2- 172.21.15.17:0/1570629680 >> v2:172.21.15.163:3300/0 conn(0x7faf4c153990 0x7faf4c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.748+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf38003520 con 0x7faf4c1506b0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2512450988 0 0) 0x7faf38003520 con 0x7faf4c1506b0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 >> v2:172.21.15.163:3300/0 conn(0x7faf4c153990 msgr2=0x7faf4c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 --2- 172.21.15.17:0/1570629680 >> v2:172.21.15.163:3300/0 conn(0x7faf4c153990 0x7faf4c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 >> v1:172.21.15.163:6789/0 conn(0x7faf4c0acc00 legacy=0x7faf4c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c1566a0 con 0x7faf4c1506b0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7faf3c001670 con 0x7faf4c1506b0 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 >> v1:172.21.15.17:6789/0 conn(0x7faf4c1506b0 legacy=0x7faf4c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380039b0 0x7faf38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf38004800 0x7faf38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 0x7faf3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c156450 con 0x7faf380051c0 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3800c060 con 0x7faf380039b0 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3800c260 con 0x7faf38004800 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 0x7faf3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4a59c640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380039b0 0x7faf38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf41d9b640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf38004800 0x7faf38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 -- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf38004800 msgr2=0x7faf38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf38004800 0x7faf38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 -- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380039b0 msgr2=0x7faf38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf380039b0 0x7faf38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.749+0000 7faf4ad9d640 1 -- 172.21.15.17:0/1570629680 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38003520 con 0x7faf380051c0 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.750+0000 7faf4ad9d640 1 -- 172.21.15.17:0/1570629680 reap_dead start 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.750+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 0x7faf3800b9c0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faf44001e50 tx=0x7faf38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ce11f87ee9586 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.750+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44001a70 con 0x7faf380051c0 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.750+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44003bb0 con 0x7faf380051c0 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.750+0000 7faf49d9b640 1 -- 172.21.15.17:0/1570629680 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440050d0 con 0x7faf380051c0 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 -- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 msgr2=0x7faf3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 0x7faf3800b9c0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7faf44001e50 tx=0x7faf38005830 comp rx=0 tx=0).stop 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 -- 172.21.15.17:0/1570629680 shutdown_connections 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 --2- 172.21.15.17:0/1570629680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf380051c0 0x7faf3800b9c0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 -- 172.21.15.17:0/1570629680 >> 172.21.15.17:0/1570629680 conn(0x7faf4c06b1a0 msgr2=0x7faf4c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 -- 172.21.15.17:0/1570629680 shutdown_connections 2024-03-06T07:27:56.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.751+0000 7faf50a5c640 1 -- 172.21.15.17:0/1570629680 wait complete. 2024-03-06T07:27:56.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.752+0000 7faf50a5c640 1 Processor -- start 2024-03-06T07:27:56.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.752+0000 7faf50a5c640 1 -- start start 2024-03-06T07:27:56.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf380051c0 0x7faf4c149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0aaf30 0x7faf4c1490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c156a10 con 0x7faf4c153fb0 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0ad210 con 0x7faf4c0aaf30 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf50a5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c156ea0 con 0x7faf380051c0 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60376/0 (socket says 172.21.15.17:60376) 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 -- 172.21.15.17:0/4019767272 learned_addr learned my addr 172.21.15.17:0/4019767272 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4a59c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf380051c0 0x7faf4c149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf41d9b640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0aaf30 0x7faf4c1490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 -- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf380051c0 msgr2=0x7faf4c149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf380051c0 0x7faf4c149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 -- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0aaf30 msgr2=0x7faf4c1490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0aaf30 0x7faf4c1490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 -- 172.21.15.17:0/4019767272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c149610 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.753+0000 7faf4ad9d640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7faf44001f70 tx=0x7faf440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.754+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44007db0 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.754+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44006d60 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.754+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c149940 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.754+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44007230 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.754+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c0a8d60 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.755+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c1e8970 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.755+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf440073d0 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.756+0000 7faf437fe640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.756+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7faf440ac1f0 con 0x7faf4c153fb0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.756+0000 7faf4a59c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:56.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.756+0000 7faf4a59c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faf4c1538d0 tx=0x7faf3c0078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:56.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.764+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf4c149610 con 0x7faf4c153fb0 2024-03-06T07:27:56.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.985+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4c04f330 con 0x7faf4c153fb0 2024-03-06T07:27:56.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.985+0000 7faf437fe640 1 -- 172.21.15.17:0/4019767272 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7faf44033030 con 0x7faf4c153fb0 2024-03-06T07:27:56.987 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:56.987 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:27:56.987 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:27:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 msgr2=0x7faf2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 0x7faf2c06d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faf4c1538d0 tx=0x7faf3c0078f0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 msgr2=0x7faf4c155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7faf44001f70 tx=0x7faf440014a0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 shutdown_connections 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7faf2c06b4d0 0x7faf2c06d990 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faf4c153fb0 0x7faf4c155a40 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf4c0aaf30 0x7faf4c1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 --2- 172.21.15.17:0/4019767272 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf380051c0 0x7faf4c149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.987+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 >> 172.21.15.17:0/4019767272 conn(0x7faf4c06b1a0 msgr2=0x7faf4c14d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.988+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 shutdown_connections 2024-03-06T07:27:56.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:56.988+0000 7faf50a5c640 1 -- 172.21.15.17:0/4019767272 wait complete. 2024-03-06T07:27:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 182, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:56.891501+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10140, 'mds_1': 9882}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9882': {'gid': 9882, 'name': 'c', 'rank': 1, 'incarnation': 181, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.17:6833/3930100029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3930100029}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 3930100029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10140': {'gid': 10140, 'name': 'a', 'rank': 0, 'incarnation': 179, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.17:6835/2561439920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2561439920}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2561439920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10140, 'qdb_cluster': [9882, 10140]} 2024-03-06T07:27:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:27:57.406 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:27:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.506+0000 7f22506eb640 1 Processor -- start 2024-03-06T07:27:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.506+0000 7f22506eb640 1 -- start start 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f22506eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22480c6dd0 0x7f2248160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f22506eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2248155130 con 0x7f22480c8950 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f22506eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22480c7200 con 0x7f22480c69f0 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f22506eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2248155440 con 0x7f22480c6dd0 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f224ec88640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f22480c6dd0 0x7f2248160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f224dc86640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f22480c8950 0x7f22481558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53704/0 (socket says 172.21.15.17:53704) 2024-03-06T07:27:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.507+0000 7f224dc86640 1 -- 172.21.15.17:0/4262705576 learned_addr learned my addr 172.21.15.17:0/4262705576 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.508+0000 7f224ec88640 1 -- 172.21.15.17:0/4262705576 >> v1:172.21.15.163:6789/0 conn(0x7f22480c69f0 legacy=0x7f22480acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.508+0000 7f224ec88640 1 -- 172.21.15.17:0/4262705576 >> v1:172.21.15.17:6789/0 conn(0x7f22480c8950 legacy=0x7f22481558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.508+0000 7f224d485640 1 -- 172.21.15.17:0/4262705576 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1920446160 0 0) 0x7f2238001110 con 0x7f22480c8950 2024-03-06T07:27:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.508+0000 7f224ec88640 1 -- 172.21.15.17:0/4262705576 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480ad480 con 0x7f22480c6dd0 2024-03-06T07:27:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f224ec88640 1 --2- 172.21.15.17:0/4262705576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c6dd0 0x7f2248160ac0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f22481555c0 tx=0x7f224402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4419e39aef1ef35 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f224d485640 1 -- 172.21.15.17:0/4262705576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244035020 con 0x7f22480c6dd0 2024-03-06T07:27:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f224d485640 1 -- 172.21.15.17:0/4262705576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244031830 con 0x7f22480c6dd0 2024-03-06T07:27:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f224d485640 1 -- 172.21.15.17:0/4262705576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244031c00 con 0x7f22480c6dd0 2024-03-06T07:27:57.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f22506eb640 1 -- 172.21.15.17:0/4262705576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c6dd0 msgr2=0x7f2248160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.509+0000 7f22506eb640 1 --2- 172.21.15.17:0/4262705576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c6dd0 0x7f2248160ac0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f22481555c0 tx=0x7f224402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.510+0000 7f22506eb640 1 -- 172.21.15.17:0/4262705576 shutdown_connections 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.510+0000 7f22506eb640 1 --2- 172.21.15.17:0/4262705576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c6dd0 0x7f2248160ac0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.510+0000 7f22506eb640 1 -- 172.21.15.17:0/4262705576 >> 172.21.15.17:0/4262705576 conn(0x7f22480cae70 msgr2=0x7f22480cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.510+0000 7f22506eb640 1 -- 172.21.15.17:0/4262705576 shutdown_connections 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.510+0000 7f22506eb640 1 -- 172.21.15.17:0/4262705576 wait complete. 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.511+0000 7f22506eb640 1 Processor -- start 2024-03-06T07:27:57.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.511+0000 7f22506eb640 1 -- start start 2024-03-06T07:27:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22480c6dd0 0x7f22480c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 0x7f2248149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480c6310 con 0x7f22480c69f0 2024-03-06T07:27:57.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480c6030 con 0x7f22480c8950 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f22506eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2248155fe0 con 0x7f22480c6dd0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f224e487640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f224e487640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60390/0 (socket says 172.21.15.17:60390) 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.512+0000 7f224e487640 1 -- 172.21.15.17:0/185455709 learned_addr learned my addr 172.21.15.17:0/185455709 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224ec88640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 0x7f2248149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224dc86640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22480c6dd0 0x7f22480c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 -- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22480c6dd0 msgr2=0x7f22480c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22480c6dd0 0x7f22480c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 -- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 msgr2=0x7f2248149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 0x7f2248149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 -- 172.21.15.17:0/185455709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224814f430 con 0x7f22480c69f0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224ec88640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 0x7f2248149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f224e487640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f224000c760 tx=0x7f224000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f223effd640 1 -- 172.21.15.17:0/185455709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240003b70 con 0x7f22480c69f0 2024-03-06T07:27:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f223effd640 1 -- 172.21.15.17:0/185455709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240003d30 con 0x7f22480c69f0 2024-03-06T07:27:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22481585b0 con 0x7f22480c69f0 2024-03-06T07:27:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f223effd640 1 -- 172.21.15.17:0/185455709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224000aa90 con 0x7f22480c69f0 2024-03-06T07:27:57.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.513+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248158870 con 0x7f22480c69f0 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.515+0000 7f223effd640 1 -- 172.21.15.17:0/185455709 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f224000ac30 con 0x7f22480c69f0 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.515+0000 7f223effd640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 0x7f222c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.515+0000 7f223effd640 1 -- 172.21.15.17:0/185455709 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f224008a220 con 0x7f22480c69f0 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.515+0000 7f224dc86640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 0x7f222c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.516+0000 7f224dc86640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 0x7f222c06d9e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f22480b0d20 tx=0x7f2238009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.516+0000 7f22533af740 1 Processor -- start 2024-03-06T07:27:57.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.516+0000 7f22533af740 1 -- start start 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba0da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba10330 0x55a1aba0df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ab9746b0 con 0x55a1aba0e4b0 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ab91da60 con 0x55a1aba10330 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1ab970750 con 0x55a1aba0ff50 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60406/0 (socket says 172.21.15.17:60406) 2024-03-06T07:27:57.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 -- 172.21.15.17:0/2275698741 learned_addr learned my addr 172.21.15.17:0/2275698741 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224dc86640 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba10330 0x55a1aba0df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224e487640 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba0da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 -- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 msgr2=0x55a1aba0da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba0da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 -- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba10330 msgr2=0x55a1aba0df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba10330 0x55a1aba0df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 -- 172.21.15.17:0/2275698741 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ab93bf30 con 0x55a1aba0e4b0 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.517+0000 7f224ec88640 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2244009890 tx=0x7f2244030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caed00cfeff00cc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f222bfff640 1 -- 172.21.15.17:0/2275698741 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244035040 con 0x55a1aba0e4b0 2024-03-06T07:27:57.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f222bfff640 1 -- 172.21.15.17:0/2275698741 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244031b70 con 0x55a1aba0e4b0 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f222bfff640 1 -- 172.21.15.17:0/2275698741 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224403a420 con 0x55a1aba0e4b0 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 -- 172.21.15.17:0/2275698741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 msgr2=0x55a1aba0a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2244009890 tx=0x7f2244030000 comp rx=0 tx=0).stop 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 -- 172.21.15.17:0/2275698741 shutdown_connections 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0e4b0 0x55a1aba0a680 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba10330 0x55a1aba0df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 --2- 172.21.15.17:0/2275698741 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba0da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 -- 172.21.15.17:0/2275698741 >> 172.21.15.17:0/2275698741 conn(0x55a1aba0c7d0 msgr2=0x55a1aba313c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 -- 172.21.15.17:0/2275698741 shutdown_connections 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.518+0000 7f22533af740 1 -- 172.21.15.17:0/2275698741 wait complete. 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.519+0000 7f22533af740 1 -- start start 2024-03-06T07:27:57.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.519+0000 7f22533af740 1 Processor -- start 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1aba32230 con 0x55a1aba10330 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1ab930ea0 con 0x55a1aba33990 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1aba31c70 con 0x55a1aba0ff50 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22533af740 10 client.0 _ll_register_callbacks cb 0x55a1ab930870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:57.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60422/0 (socket says 172.21.15.17:60422) 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 -- 172.21.15.17:0/2411407710 learned_addr learned my addr 172.21.15.17:0/2411407710 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224e487640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224ec88640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 -- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 msgr2=0x55a1aba307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 -- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 msgr2=0x55a1aba2cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba2cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 -- 172.21.15.17:0/2411407710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1ab931270 con 0x55a1aba10330 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f224dc86640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f2238005dd0 tx=0x7f2238000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228e0f652ef1c8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.520+0000 7f22297fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238026060 con 0x55a1aba10330 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2238005e00 con 0x55a1aba10330 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2244031b70 con 0x55a1aba10330 2024-03-06T07:27:57.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2238012620 con 0x55a1aba10330 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 10 client.10191 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55a1ab6892d0 con 0x55a1aba10330 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244031b70 con 0x55a1aba10330 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f22380127c0 con 0x55a1aba10330 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 10 client.10191 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 10 client.10191 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 20 client.10191 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba60600 0x55a1aba6b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 4 client.10191 mds_command: new command op to 9882 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 --> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1ab981bf0 con 0x55a1aba60600 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.521+0000 7f224e487640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba60600 0x55a1aba6b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22297fa640 10 client.10191 ms_handle_connect on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f224e487640 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba60600 0x55a1aba6b280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0c47f97061adf57 server_cookie=98b1853f7a826b06 in_seq=0 out_seq=0 2024-03-06T07:27:57.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22297fa640 1 -- 172.21.15.17:0/2411407710 <== mds.1 v2:172.21.15.17:6832/3930100029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a1ab981bf0 con 0x55a1aba60600 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22297fa640 10 client.10191 handle_command_reply: tid=0 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 client.10191 shutdown 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 msgr2=0x55a1aba34250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f2238005dd0 tx=0x7f2238000ed0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 shutdown_connections 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba60600 0x55a1aba6b280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba2cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba34250 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22297fa640 0 client.10191 ms_handle_reset on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.522+0000 7f22533af740 1 --2- 172.21.15.17:0/2411407710 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.523+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 >> 172.21.15.17:0/2411407710 conn(0x55a1aba0e4b0 msgr2=0x55a1aba0e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.523+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 shutdown_connections 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.523+0000 7f22533af740 1 -- 172.21.15.17:0/2411407710 wait complete. 2024-03-06T07:27:57.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.523+0000 7f22533af740 20 client.10191 trim_cache size 0 max 16384 2024-03-06T07:27:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 Processor -- start 2024-03-06T07:27:57.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 -- start start 2024-03-06T07:27:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22481f7f50 con 0x55a1aba10330 2024-03-06T07:27:57.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248157300 con 0x55a1aba33990 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22481f6a10 con 0x55a1aba0ff50 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f224dc86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f224dc86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60428/0 (socket says 172.21.15.17:60428) 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.540+0000 7f224dc86640 1 -- 172.21.15.17:0/267373137 learned_addr learned my addr 172.21.15.17:0/267373137 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224e487640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224ec88640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 -- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 msgr2=0x55a1aba702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 -- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 msgr2=0x55a1aba09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba09510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 -- 172.21.15.17:0/267373137 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1ab7536b0 con 0x55a1aba10330 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224ec88640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f224dc86640 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f2238008ff0 tx=0x7f2238019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e931245f8dd52370 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f222b7fe640 1 -- 172.21.15.17:0/267373137 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800dd50 con 0x55a1aba10330 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f222b7fe640 1 -- 172.21.15.17:0/267373137 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22380127c0 con 0x55a1aba10330 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.541+0000 7f222b7fe640 1 -- 172.21.15.17:0/267373137 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238012270 con 0x55a1aba10330 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- 172.21.15.17:0/267373137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 msgr2=0x55a1aba0e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f2238008ff0 tx=0x7f2238019000 comp rx=0 tx=0).stop 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- 172.21.15.17:0/267373137 shutdown_connections 2024-03-06T07:27:57.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1aba09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba10330 0x55a1aba0e890 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 --2- 172.21.15.17:0/267373137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba0ff50 0x55a1aba702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- 172.21.15.17:0/267373137 >> 172.21.15.17:0/267373137 conn(0x55a1aba0e4b0 msgr2=0x55a1aba05ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- 172.21.15.17:0/267373137 shutdown_connections 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- 172.21.15.17:0/267373137 wait complete. 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 -- start start 2024-03-06T07:27:57.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.542+0000 7f22533af740 1 Processor -- start 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba10330 0x55a1aba2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1abab8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1aba334f0 con 0x55a1aba0ff50 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1aba32f30 con 0x55a1aba33990 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1aba34dd0 con 0x55a1aba10330 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f22533af740 10 client.0 _ll_register_callbacks cb 0x7f2238005a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f224e487640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f224e487640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60436/0 (socket says 172.21.15.17:60436) 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f224e487640 1 -- 172.21.15.17:0/1660280906 learned_addr learned my addr 172.21.15.17:0/1660280906 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f224dc86640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba10330 0x55a1aba2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.543+0000 7f224ec88640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1abab8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 -- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba10330 msgr2=0x55a1aba2f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba10330 0x55a1aba2f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 -- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 msgr2=0x55a1abab8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1abab8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 -- 172.21.15.17:0/1660280906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1aba2efa0 con 0x55a1aba0ff50 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f224e487640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f224000a430 tx=0x7f2240005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b10fc8ec427f4d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240054ef0 con 0x55a1aba0ff50 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f22400701c0 con 0x55a1aba0ff50 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22380127c0 con 0x55a1aba0ff50 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2240055090 con 0x55a1aba0ff50 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 10 client.10197 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:57.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55a1aba0bdf0 con 0x55a1aba0ff50 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22380127c0 con 0x55a1aba0ff50 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f2240088170 con 0x55a1aba0ff50 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 10 client.10197 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 10 client.10197 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 20 client.10197 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba4f650 0x55a1aba4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 4 client.10197 mds_command: new command op to 9882 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.544+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 --> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1ab97bf40 con 0x55a1aba4f650 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f224dc86640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba4f650 0x55a1aba4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22297fa640 10 client.10197 ms_handle_connect on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f224dc86640 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba4f650 0x55a1aba4fa30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60cfadbd41f19b2e server_cookie=8a4839b76e6beeee in_seq=0 out_seq=0 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22297fa640 1 -- 172.21.15.17:0/1660280906 <== mds.1 v2:172.21.15.17:6832/3930100029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55a1ab97bf40 con 0x55a1aba4f650 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22297fa640 10 client.10197 handle_command_reply: tid=0 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 client.10197 shutdown 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 msgr2=0x55a1aba0f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f224000a430 tx=0x7f2240005040 comp rx=0 tx=0).stop 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 shutdown_connections 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a1aba4f650 0x55a1aba4fa30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a1aba33990 0x55a1abab8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a1aba10330 0x55a1aba2f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22297fa640 0 client.10197 ms_handle_reset on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 --2- 172.21.15.17:0/1660280906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a1aba0ff50 0x55a1aba0f2a0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.545+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 >> 172.21.15.17:0/1660280906 conn(0x55a1aba0e4b0 msgr2=0x55a1aba34f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 shutdown_connections 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22533af740 1 -- 172.21.15.17:0/1660280906 wait complete. 2024-03-06T07:27:57.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22533af740 20 client.10197 trim_cache size 0 max 16384 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9882, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 1.6543952200000001, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 182, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 188, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 188, 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 41.178561786000003 2024-03-06T07:27:57.555 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:57.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 msgr2=0x7f222c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 0x7f222c06d9e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f22480b0d20 tx=0x7f2238009040 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 msgr2=0x7f224814eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f224000c760 tx=0x7f224000cc20 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 shutdown_connections 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f222c06b520 0x7f222c06d9e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22480c8950 0x7f2248149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22480c6dd0 0x7f22480c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 --2- 172.21.15.17:0/185455709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22480c69f0 0x7f224814eb90 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 >> 172.21.15.17:0/185455709 conn(0x7f22480cae70 msgr2=0x7f2248150ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 shutdown_connections 2024-03-06T07:27:57.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.546+0000 7f22506eb640 1 -- 172.21.15.17:0/185455709 wait complete. 2024-03-06T07:27:57.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:27:57.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.675+0000 7fd501639640 1 Processor -- start 2024-03-06T07:27:57.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.675+0000 7fd501639640 1 -- start start 2024-03-06T07:27:57.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc06b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd4fc0c53d0 0x7fd4fc0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4fc159520 0x7fd4fc1552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc159090 con 0x7fd4fc159520 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc13e420 con 0x7fd4fc0c53d0 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.676+0000 7fd501639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc158a40 con 0x7fd4fc0c1150 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4fc159520 0x7fd4fc1552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4fc159520 0x7fd4fc1552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60450/0 (socket says 172.21.15.17:60450) 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/298446575 learned_addr learned my addr 172.21.15.17:0/298446575 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4faffd640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/298446575 >> v2:172.21.15.163:3300/0 conn(0x7fd4fc0c53d0 0x7fd4fc0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/298446575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 msgr2=0x7fd4fc06b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc06b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/298446575 >> v2:172.21.15.163:3300/0 conn(0x7fd4fc0c53d0 msgr2=0x7fd4fc0c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/298446575 >> v2:172.21.15.163:3300/0 conn(0x7fd4fc0c53d0 0x7fd4fc0c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.677+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/298446575 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc158d20 con 0x7fd4fc159520 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc1552c0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0cbab0 tx=0x7fd4ec02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3378e7253f3bb970 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd4f9ffb640 1 -- 172.21.15.17:0/298446575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec02c950 con 0x7fd4fc159520 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd4f9ffb640 1 -- 172.21.15.17:0/298446575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec02caf0 con 0x7fd4fc159520 2024-03-06T07:27:57.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd4f9ffb640 1 -- 172.21.15.17:0/298446575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0343f0 con 0x7fd4fc159520 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 -- 172.21.15.17:0/298446575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 msgr2=0x7fd4fc1552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc1552c0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc0cbab0 tx=0x7fd4ec02bdb0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 -- 172.21.15.17:0/298446575 shutdown_connections 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc1552c0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 --2- 172.21.15.17:0/298446575 >> v2:172.21.15.163:3300/0 conn(0x7fd4fc0c53d0 0x7fd4fc0c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 --2- 172.21.15.17:0/298446575 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc06b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.678+0000 7fd501639640 1 -- 172.21.15.17:0/298446575 >> 172.21.15.17:0/298446575 conn(0x7fd4fc0ac680 msgr2=0x7fd4fc0aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.679+0000 7fd501639640 1 -- 172.21.15.17:0/298446575 shutdown_connections 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.679+0000 7fd501639640 1 -- 172.21.15.17:0/298446575 wait complete. 2024-03-06T07:27:57.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 Processor -- start 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 -- start start 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4fc0c53d0 0x7fd4fc0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc13e7d0 con 0x7fd4fc159520 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c1530 con 0x7fd4fc0c53d0 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.680+0000 7fd501639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc13e950 con 0x7fd4fc0c1150 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60458/0 (socket says 172.21.15.17:60458) 2024-03-06T07:27:57.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/1418265318 learned_addr learned my addr 172.21.15.17:0/1418265318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4fc0c53d0 0x7fd4fc0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 msgr2=0x7fd4fc0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc0aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4fc0c53d0 msgr2=0x7fd4fc0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4fc0c53d0 0x7fd4fc0af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/1418265318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc1397a0 con 0x7fd4fc159520 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009f40 tx=0x7fd4ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4f37fe640 1 -- 172.21.15.17:0/1418265318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec043020 con 0x7fd4fc159520 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4f37fe640 1 -- 172.21.15.17:0/1418265318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec03e450 con 0x7fd4fc159520 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd4f37fe640 1 -- 172.21.15.17:0/1418265318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec009510 con 0x7fd4fc159520 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0ae010 con 0x7fd4fc159520 2024-03-06T07:27:57.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.681+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc0ae5d0 con 0x7fd4fc159520 2024-03-06T07:27:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.683+0000 7fd4f37fe640 1 -- 172.21.15.17:0/1418265318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4ec046020 con 0x7fd4fc159520 2024-03-06T07:27:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.683+0000 7fd4f37fe640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 0x7fd4dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.684+0000 7fd4f37fe640 1 -- 172.21.15.17:0/1418265318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd4ec0acd50 con 0x7fd4fc159520 2024-03-06T07:27:57.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.684+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 0x7fd4dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.684+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 0x7fd4dc06d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4006fd0 tx=0x7fd4f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.684+0000 7fd504229740 1 Processor -- start 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.684+0000 7fd504229740 1 -- start start 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6bf50 0x555e6ac69a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6a4b0 0x555e6ac66680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6abd06b0 con 0x555e6ac6c330 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6ab79a60 con 0x555e6ac6bf50 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e6abcc750 con 0x555e6ac6a4b0 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd4fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd4fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60462/0 (socket says 172.21.15.17:60462) 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1348231572 learned_addr learned my addr 172.21.15.17:0/1348231572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6bf50 0x555e6ac69a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.685+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6a4b0 0x555e6ac66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6a4b0 msgr2=0x555e6ac66680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6a4b0 0x555e6ac66680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6bf50 msgr2=0x555e6ac69a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6bf50 0x555e6ac69a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1348231572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e6ab97f30 con 0x555e6ac6c330 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007900 tx=0x7fd4e4007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0294818ebd6120e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4f0ff9640 1 -- 172.21.15.17:0/1348231572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018020 con 0x555e6ac6c330 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4f0ff9640 1 -- 172.21.15.17:0/1348231572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4004300 con 0x555e6ac6c330 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd4f0ff9640 1 -- 172.21.15.17:0/1348231572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010af0 con 0x555e6ac6c330 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd504229740 1 -- 172.21.15.17:0/1348231572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 msgr2=0x555e6ac69f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.686+0000 7fd504229740 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007900 tx=0x7fd4e4007dc0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 -- 172.21.15.17:0/1348231572 shutdown_connections 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6a4b0 0x555e6ac66680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac69f70 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 --2- 172.21.15.17:0/1348231572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6bf50 0x555e6ac69a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 -- 172.21.15.17:0/1348231572 >> 172.21.15.17:0/1348231572 conn(0x555e6ac687d0 msgr2=0x555e6ac8d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 -- 172.21.15.17:0/1348231572 shutdown_connections 2024-03-06T07:27:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 -- 172.21.15.17:0/1348231572 wait complete. 2024-03-06T07:27:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.687+0000 7fd504229740 1 -- start start 2024-03-06T07:27:57.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.688+0000 7fd504229740 1 Processor -- start 2024-03-06T07:27:57.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6acb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ac8a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6ac8e230 con 0x555e6ac91cd0 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6ab8cea0 con 0x555e6ac6c330 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e6ac8dc70 con 0x555e6ac6bf50 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd504229740 10 client.0 _ll_register_callbacks cb 0x555e6ab8c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4faffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6acb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60466/0 (socket says 172.21.15.17:60466) 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/92164834 learned_addr learned my addr 172.21.15.17:0/92164834 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ac8a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 msgr2=0x555e6acb8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6acb8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 msgr2=0x555e6ac8a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ac8a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 -- 172.21.15.17:0/92164834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e6ab8d270 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec07b110 tx=0x7fd4ec07b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb4125447720c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.689+0000 7fd4d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec02c960 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd4ec0835e0 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4e4004300 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4ec02cb00 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 10 client.10209 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x555e6a8e52d0 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4004300 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd4ec07a860 con 0x555e6ac91cd0 2024-03-06T07:27:57.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 10 client.10209 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 10 client.10209 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 20 client.10209 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6acbc710 0x555e6acc7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 4 client.10209 mds_command: new command op to 10140 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.690+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 --> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e6abddbf0 con 0x555e6acbc710 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd4faffd640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6acbc710 0x555e6acc7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd4faffd640 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6acbc710 0x555e6acc7500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8876d874c32086b2 server_cookie=345d1fa74a1e95ee in_seq=0 out_seq=0 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd4d9ffb640 10 client.10209 ms_handle_connect on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/92164834 <== mds.0 v2:172.21.15.17:6834/2561439920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e6abddbf0 con 0x555e6acbc710 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd4d9ffb640 10 client.10209 handle_command_reply: tid=0 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.691+0000 7fd504229740 1 client.10209 shutdown 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 msgr2=0x555e6ac92150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec07b110 tx=0x7fd4ec07b410 comp rx=0 tx=0).stop 2024-03-06T07:27:57.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 shutdown_connections 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6acbc710 0x555e6acc7500 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac91cd0 0x555e6ac92150 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ac8a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd4d9ffb640 0 client.10209 ms_handle_reset on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 --2- 172.21.15.17:0/92164834 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6acb8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 >> 172.21.15.17:0/92164834 conn(0x555e6ac6a4b0 msgr2=0x555e6ac67030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 shutdown_connections 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 1 -- 172.21.15.17:0/92164834 wait complete. 2024-03-06T07:27:57.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.692+0000 7fd504229740 20 client.10209 trim_cache size 0 max 16384 2024-03-06T07:27:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.723+0000 7fd504229740 1 Processor -- start 2024-03-06T07:27:57.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.723+0000 7fd504229740 1 -- start start 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6ac8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac91cd0 0x555e6ac695a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc1f7e90 con 0x555e6ac6c330 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc1f64d0 con 0x555e6ac91cd0 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc1f6a30 con 0x555e6ac6bf50 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60472/0 (socket says 172.21.15.17:60472) 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1242128403 learned_addr learned my addr 172.21.15.17:0/1242128403 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac91cd0 0x555e6ac695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6ac8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 msgr2=0x555e6ac8cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6ac8cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac91cd0 msgr2=0x555e6ac695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac91cd0 0x555e6ac695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 -- 172.21.15.17:0/1242128403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e6a9af6b0 con 0x555e6ac6c330 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007820 tx=0x7fd4e4009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e4a0e5ea6ef826 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4dbfff640 1 -- 172.21.15.17:0/1242128403 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018040 con 0x555e6ac6c330 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4dbfff640 1 -- 172.21.15.17:0/1242128403 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4004510 con 0x555e6ac6c330 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.724+0000 7fd4dbfff640 1 -- 172.21.15.17:0/1242128403 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010d50 con 0x555e6ac6c330 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 -- 172.21.15.17:0/1242128403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 msgr2=0x555e6ac65510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4007820 tx=0x7fd4e4009000 comp rx=0 tx=0).stop 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 -- 172.21.15.17:0/1242128403 shutdown_connections 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac91cd0 0x555e6ac695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6c330 0x555e6ac65510 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 --2- 172.21.15.17:0/1242128403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac6bf50 0x555e6ac8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 -- 172.21.15.17:0/1242128403 >> 172.21.15.17:0/1242128403 conn(0x555e6ac6a4b0 msgr2=0x555e6ac67b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 -- 172.21.15.17:0/1242128403 shutdown_connections 2024-03-06T07:27:57.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.725+0000 7fd504229740 1 -- 172.21.15.17:0/1242128403 wait complete. 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 -- start start 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 Processor -- start 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ad1bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac91cd0 0x555e6ad159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6ac8edc0 con 0x555e6ac6bf50 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6ac8e230 con 0x555e6ac6c330 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.726+0000 7fd504229740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e6ac8f610 con 0x555e6ac91cd0 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd504229740 10 client.0 _ll_register_callbacks cb 0x7fd4ec07af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60474/0 (socket says 172.21.15.17:60474) 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 -- 172.21.15.17:0/1247465059 learned_addr learned my addr 172.21.15.17:0/1247465059 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4fb7fe640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac91cd0 0x555e6ad159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ad1bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 -- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac91cd0 msgr2=0x555e6ad159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac91cd0 0x555e6ad159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 -- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 msgr2=0x555e6ad1bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ad1bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 -- 172.21.15.17:0/1247465059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e6ad193d0 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4faffd640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x555e6ac92790 tx=0x7fd4f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e09be93ab6f8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4031060 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd4f4005ba0 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4e4004510 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4f4037050 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd504229740 10 client.10215 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.727+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x555e6ac91a30 con 0x555e6ac6bf50 2024-03-06T07:27:57.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4004510 con 0x555e6ac6bf50 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd4f403a020 con 0x555e6ac6bf50 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 10 client.10215 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 10 client.10215 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 20 client.10215 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6ac6c710 0x555e6ac6caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 4 client.10215 mds_command: new command op to 10140 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 --> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e6ac94170 con 0x555e6ac6c710 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6ac6c710 0x555e6ac6caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4d9ffb640 10 client.10215 ms_handle_connect on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4fa7fc640 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6ac6c710 0x555e6ac6caf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b18adf7d62de16 server_cookie=ed39876502ed97aa in_seq=0 out_seq=0 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4d9ffb640 1 -- 172.21.15.17:0/1247465059 <== mds.0 v2:172.21.15.17:6834/2561439920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x555e6ac94170 con 0x555e6ac6c710 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd4d9ffb640 10 client.10215 handle_command_reply: tid=0 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.728+0000 7fd504229740 1 client.10215 shutdown 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 msgr2=0x555e6ac699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x555e6ac92790 tx=0x7fd4f4013000 comp rx=0 tx=0).stop 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 shutdown_connections 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x555e6ac6c710 0x555e6ac6caf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x555e6ac91cd0 0x555e6ad159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x555e6ac6c330 0x555e6ad1bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 --2- 172.21.15.17:0/1247465059 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e6ac6bf50 0x555e6ac699a0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 >> 172.21.15.17:0/1247465059 conn(0x555e6ac6a4b0 msgr2=0x555e6ac8cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd4d9ffb640 0 client.10215 ms_handle_reset on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 shutdown_connections 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 1 -- 172.21.15.17:0/1247465059 wait complete. 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd504229740 20 client.10215 trim_cache size 0 max 16384 2024-03-06T07:27:57.736 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 10140, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 2.8443076, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 182, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 188, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 188, 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 4.8314810609999999 2024-03-06T07:27:57.737 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 msgr2=0x7fd4dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 0x7fd4dc06d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4006fd0 tx=0x7fd4f4008040 comp rx=0 tx=0).stop 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 msgr2=0x7fd4fc0c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009f40 tx=0x7fd4ec004a40 comp rx=0 tx=0).stop 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 shutdown_connections 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd4dc06b4d0 0x7fd4dc06d990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4fc159520 0x7fd4fc0c1bb0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd4fc0c53d0 0x7fd4fc0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 --2- 172.21.15.17:0/1418265318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd4fc0c1150 0x7fd4fc0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 >> 172.21.15.17:0/1418265318 conn(0x7fd4fc0ac680 msgr2=0x7fd4fc0c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 shutdown_connections 2024-03-06T07:27:57.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.729+0000 7fd501639640 1 -- 172.21.15.17:0/1418265318 wait complete. 2024-03-06T07:27:57.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.843+0000 7fa542639640 1 Processor -- start 2024-03-06T07:27:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.843+0000 7fa542639640 1 -- start start 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa542639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa53c0c5480 0x7fa53c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa542639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa542639640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa53c0c5cc0 con 0x7fa53c0c5860 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa542639640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa53c06df40 con 0x7fa53c0c5480 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa542639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c06fdb0 con 0x7fa53c1586f0 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa53c0c5480 0x7fa53c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa53c0c5860 0x7fa53c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53712/0 (socket says 172.21.15.17:53712) 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53b7fe640 1 -- 172.21.15.17:0/788435744 learned_addr learned my addr 172.21.15.17:0/788435744 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa53c0c5480 0x7fa53c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60476/0 (socket says 172.21.15.17:60476) 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa540bd6640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 -- 172.21.15.17:0/788435744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 msgr2=0x7fa53c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 -- 172.21.15.17:0/788435744 >> v1:172.21.15.17:6789/0 conn(0x7fa53c0c5860 legacy=0x7fa53c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53bfff640 1 -- 172.21.15.17:0/788435744 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c1591c0 con 0x7fa53c0c5480 2024-03-06T07:27:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.844+0000 7fa53affd640 1 -- 172.21.15.17:0/788435744 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180311009 0 0) 0x7fa53c0c5cc0 con 0x7fa53c0c5860 2024-03-06T07:27:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.845+0000 7fa53bfff640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c155ba0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097a0 tx=0x7fa52c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e18bafffae17f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.845+0000 7fa53affd640 1 -- 172.21.15.17:0/788435744 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0315a0 con 0x7fa53c0c5480 2024-03-06T07:27:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.845+0000 7fa53affd640 1 -- 172.21.15.17:0/788435744 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c037070 con 0x7fa53c0c5480 2024-03-06T07:27:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.845+0000 7fa53affd640 1 -- 172.21.15.17:0/788435744 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c031960 con 0x7fa53c0c5480 2024-03-06T07:27:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 -- 172.21.15.17:0/788435744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 msgr2=0x7fa53c155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c155ba0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0097a0 tx=0x7fa52c02b840 comp rx=0 tx=0).stop 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 -- 172.21.15.17:0/788435744 shutdown_connections 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 --2- 172.21.15.17:0/788435744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c155ba0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 -- 172.21.15.17:0/788435744 >> 172.21.15.17:0/788435744 conn(0x7fa53c06b1a0 msgr2=0x7fa53c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 -- 172.21.15.17:0/788435744 shutdown_connections 2024-03-06T07:27:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.846+0000 7fa542639640 1 -- 172.21.15.17:0/788435744 wait complete. 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.847+0000 7fa542639640 1 Processor -- start 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.847+0000 7fa542639640 1 -- start start 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa53c0c5860 0x7fa53c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c155680 con 0x7fa53c0c5480 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c1553a0 con 0x7fa53c0c5860 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa542639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c152a20 con 0x7fa53c1586f0 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa53bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60492/0 (socket says 172.21.15.17:60492) 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.848+0000 7fa53bfff640 1 -- 172.21.15.17:0/2171717687 learned_addr learned my addr 172.21.15.17:0/2171717687 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa540bd6640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53b7fe640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa53c0c5860 0x7fa53c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 -- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 msgr2=0x7fa53c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c150c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 -- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa53c0c5860 msgr2=0x7fa53c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa53c0c5860 0x7fa53c0c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 -- 172.21.15.17:0/2171717687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c14ddd0 con 0x7fa53c0c5480 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa53bfff640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0098c0 tx=0x7fa52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c032630 con 0x7fa53c0c5480 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c0327d0 con 0x7fa53c0c5480 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c032ba0 con 0x7fa53c0c5480 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c14d8f0 con 0x7fa53c0c5480 2024-03-06T07:27:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.849+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c151830 con 0x7fa53c0c5480 2024-03-06T07:27:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.850+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c14fa70 con 0x7fa53c0c5480 2024-03-06T07:27:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.851+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa52c04e430 con 0x7fa53c0c5480 2024-03-06T07:27:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.851+0000 7fa538ff9640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 0x7fa52806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:57.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.851+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa52c02f030 con 0x7fa53c0c5480 2024-03-06T07:27:57.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.851+0000 7fa53b7fe640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 0x7fa52806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:57.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.852+0000 7fa53b7fe640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 0x7fa52806d9c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa530007fd0 tx=0x7fa530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:57.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:57.856+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa52c040070 con 0x7fa53c0c5480 2024-03-06T07:27:58.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.096+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa53c04f330 con 0x7fa53c0c5480 2024-03-06T07:27:58.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.097+0000 7fa538ff9640 1 -- 172.21.15.17:0/2171717687 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7fa52c07af60 con 0x7fa53c0c5480 2024-03-06T07:27:58.098 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:58.099 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:27:58.099 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 msgr2=0x7fa52806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 0x7fa52806d9c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa530007fd0 tx=0x7fa530009040 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 msgr2=0x7fa53c1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0098c0 tx=0x7fa52c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 shutdown_connections 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa52806b500 0x7fa52806d9c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa53c1586f0 0x7fa53c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa53c0c5860 0x7fa53c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 --2- 172.21.15.17:0/2171717687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa53c0c5480 0x7fa53c1501a0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 >> 172.21.15.17:0/2171717687 conn(0x7fa53c06b1a0 msgr2=0x7fa53c06cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 shutdown_connections 2024-03-06T07:27:58.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.100+0000 7fa542639640 1 -- 172.21.15.17:0/2171717687 wait complete. 2024-03-06T07:27:58.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 182, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:27:26.886176+0000', 'modified': '2024-03-06T07:27:56.891501+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10140, 'mds_1': 9882}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9882': {'gid': 9882, 'name': 'c', 'rank': 1, 'incarnation': 181, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.17:6833/3930100029', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3930100029}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 3930100029}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10140': {'gid': 10140, 'name': 'a', 'rank': 0, 'incarnation': 179, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.17:6835/2561439920', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2561439920}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2561439920}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10140, 'qdb_cluster': [9882, 10140]} 2024-03-06T07:27:58.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:27:58.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:27:58.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.619+0000 7efce3064640 1 Processor -- start 2024-03-06T07:27:58.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.619+0000 7efce3064640 1 -- start start 2024-03-06T07:27:58.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce3064640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 0x7efcdc0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce3064640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 0x7efcdc0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce3064640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0a8390 con 0x7efcdc06ff10 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce3064640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc0c7620 con 0x7efcdc0c6c40 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce3064640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0730f0 con 0x7efcdc0c7170 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce0e00640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efcdc06ff10 0x7efcdc153c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53716/0 (socket says 172.21.15.17:53716) 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce0e00640 1 -- 172.21.15.17:0/3375090514 learned_addr learned my addr 172.21.15.17:0/3375090514 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efcdbfff640 1 --2- 172.21.15.17:0/3375090514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 0x7efcdc0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efce1601640 1 --2- 172.21.15.17:0/3375090514 >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 0x7efcdc0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400521858 0 0) 0x7efcdc0a8390 con 0x7efcdc06ff10 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.620+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc8003520 con 0x7efcdc06ff10 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1953521077 0 0) 0x7efcc8003520 con 0x7efcdc06ff10 2024-03-06T07:27:58.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efcdbfff640 1 -- 172.21.15.17:0/3375090514 >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 msgr2=0x7efcdc0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efcdbfff640 1 --2- 172.21.15.17:0/3375090514 >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 0x7efcdc0c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efcdbfff640 1 -- 172.21.15.17:0/3375090514 >> v1:172.21.15.17:6789/0 conn(0x7efcdc06ff10 legacy=0x7efcdc153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efcdbfff640 1 -- 172.21.15.17:0/3375090514 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0733d0 con 0x7efcdc0c7170 2024-03-06T07:27:58.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.621+0000 7efce1601640 1 --2- 172.21.15.17:0/3375090514 >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 0x7efcdc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:27:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efcdbfff640 1 --2- 172.21.15.17:0/3375090514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 0x7efcdc0c8810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcc40097a0 tx=0x7efcc402b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70a38f45c199f805 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4037070 con 0x7efcdc0c7170 2024-03-06T07:27:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc4030510 con 0x7efcdc0c7170 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efcdb7fe640 1 -- 172.21.15.17:0/3375090514 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc402be60 con 0x7efcdc0c7170 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efce3064640 1 -- 172.21.15.17:0/3375090514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 msgr2=0x7efcdc0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efce3064640 1 --2- 172.21.15.17:0/3375090514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 0x7efcdc0c8810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efcc40097a0 tx=0x7efcc402b660 comp rx=0 tx=0).stop 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.622+0000 7efce3064640 1 -- 172.21.15.17:0/3375090514 shutdown_connections 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.623+0000 7efce3064640 1 --2- 172.21.15.17:0/3375090514 >> v2:172.21.15.163:3300/0 conn(0x7efcdc0c6c40 0x7efcdc0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.623+0000 7efce3064640 1 --2- 172.21.15.17:0/3375090514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c7170 0x7efcdc0c8810 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.623+0000 7efce3064640 1 -- 172.21.15.17:0/3375090514 >> 172.21.15.17:0/3375090514 conn(0x7efcdc06b970 msgr2=0x7efcdc0aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.623+0000 7efce3064640 1 -- 172.21.15.17:0/3375090514 shutdown_connections 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.623+0000 7efce3064640 1 -- 172.21.15.17:0/3375090514 wait complete. 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.624+0000 7efce3064640 1 Processor -- start 2024-03-06T07:27:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.624+0000 7efce3064640 1 -- start start 2024-03-06T07:27:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.624+0000 7efce3064640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.624+0000 7efce3064640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 0x7efcdc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 0x7efcdc14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1544b0 con 0x7efcdc06ff10 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc154330 con 0x7efcdc0c7170 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0702f0 con 0x7efcdc0c6c40 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60500/0 (socket says 172.21.15.17:60500) 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 -- 172.21.15.17:0/2150821569 learned_addr learned my addr 172.21.15.17:0/2150821569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce1601640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 0x7efcdc14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 0x7efcdc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 -- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 msgr2=0x7efcdc14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 0x7efcdc14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 -- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 msgr2=0x7efcdc14cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 0x7efcdc14cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 -- 172.21.15.17:0/2150821569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0aaa40 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce1601640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 0x7efcdc14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 0x7efcdc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce0e00640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cb20 tx=0x7efccc001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efcd97fa640 1 -- 172.21.15.17:0/2150821569 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc005ce0 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efcd97fa640 1 -- 172.21.15.17:0/2150821569 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc007070 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc154930 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efcd97fa640 1 -- 172.21.15.17:0/2150821569 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc007460 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.625+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc06c1b0 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.627+0000 7efcd97fa640 1 -- 172.21.15.17:0/2150821569 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efccc007600 con 0x7efcdc06ff10 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.627+0000 7efcd97fa640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.627+0000 7efcd97fa640 1 -- 172.21.15.17:0/2150821569 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7efccc08a3c0 con 0x7efcdc06ff10 2024-03-06T07:27:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.628+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.628+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 0x7efcbc06d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efcc4034000 tx=0x7efcc4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.629+0000 7efce5c54740 1 Processor -- start 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.629+0000 7efce5c54740 1 -- start start 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 0x55a60103ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60103bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a600fa26b0 con 0x55a60103c4b0 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a600f4ba60 con 0x55a60103df50 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a600f9e750 con 0x55a60103e330 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60502/0 (socket says 172.21.15.17:60502) 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 -- 172.21.15.17:0/2382389812 learned_addr learned my addr 172.21.15.17:0/2382389812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce0e00640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 0x55a60103ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60103bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 -- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 msgr2=0x55a60103bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60103bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 -- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 msgr2=0x55a60103ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 0x55a60103ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 -- 172.21.15.17:0/2382389812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a600f69f30 con 0x55a60103c4b0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce0e00640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 0x55a60103ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.630+0000 7efce1601640 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7efcd4007680 tx=0x7efcd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5809a4bfe2194067 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efcd27fc640 1 -- 172.21.15.17:0/2382389812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd401a020 con 0x55a60103c4b0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efcd27fc640 1 -- 172.21.15.17:0/2382389812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4015850 con 0x55a60103c4b0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efcd27fc640 1 -- 172.21.15.17:0/2382389812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4015c20 con 0x55a60103c4b0 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 -- 172.21.15.17:0/2382389812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 msgr2=0x55a601038680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7efcd4007680 tx=0x7efcd4009750 comp rx=0 tx=0).stop 2024-03-06T07:27:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 -- 172.21.15.17:0/2382389812 shutdown_connections 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103c4b0 0x55a601038680 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60103bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 --2- 172.21.15.17:0/2382389812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103df50 0x55a60103ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 -- 172.21.15.17:0/2382389812 >> 172.21.15.17:0/2382389812 conn(0x55a60103a7d0 msgr2=0x55a60105f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 -- 172.21.15.17:0/2382389812 shutdown_connections 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.631+0000 7efce5c54740 1 -- 172.21.15.17:0/2382389812 wait complete. 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.632+0000 7efce5c54740 1 -- start start 2024-03-06T07:27:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.632+0000 7efce5c54740 1 Processor -- start 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60105ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60108b3a0 0x55a601059fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a601060230 con 0x55a60103df50 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a600f5eea0 con 0x55a60108b3a0 2024-03-06T07:27:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a60105fc70 con 0x55a60103e330 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce5c54740 10 client.0 _ll_register_callbacks cb 0x55a600f5e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60518/0 (socket says 172.21.15.17:60518) 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 -- 172.21.15.17:0/383897275 learned_addr learned my addr 172.21.15.17:0/383897275 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efcdbfff640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60105ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce1601640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60108b3a0 0x55a601059fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 -- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 msgr2=0x55a60105ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60105ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 -- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60108b3a0 msgr2=0x55a601059fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60108b3a0 0x55a601059fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 -- 172.21.15.17:0/383897275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a600f5f270 con 0x55a60103df50 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.633+0000 7efce0e00640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7efccc058960 tx=0x7efccc058a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13a349063bb9431 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc070070 con 0x55a60103df50 2024-03-06T07:27:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7efccc060c50 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcd4015850 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7efccc060ea0 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 10 client.10236 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55a600cb72d0 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4015850 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7efccc057e20 con 0x55a60103df50 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 10 client.10236 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 10 client.10236 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 20 client.10236 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a60108e820 0x55a601099500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 4 client.10236 mds_command: new command op to 9882 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 --> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a600fafbf0 con 0x55a60108e820 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.634+0000 7efcdbfff640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a60108e820 0x55a601099500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efcbb7fe640 10 client.10236 ms_handle_connect on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efcdbfff640 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a60108e820 0x55a601099500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15a3a81d0055c73b server_cookie=4b2c2ccb55b26908 in_seq=0 out_seq=0 2024-03-06T07:27:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efcbb7fe640 1 -- 172.21.15.17:0/383897275 <== mds.1 v2:172.21.15.17:6832/3930100029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a600fafbf0 con 0x55a60108e820 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efcbb7fe640 10 client.10236 handle_command_reply: tid=0 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efce5c54740 1 client.10236 shutdown 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 msgr2=0x55a60105c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.635+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7efccc058960 tx=0x7efccc058a90 comp rx=0 tx=0).stop 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 shutdown_connections 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a60108e820 0x55a601099500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60108b3a0 0x55a601059fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103e330 0x55a60105ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efcbb7fe640 0 client.10236 ms_handle_reset on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 --2- 172.21.15.17:0/383897275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60103df50 0x55a60105c240 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 >> 172.21.15.17:0/383897275 conn(0x55a60103c4b0 msgr2=0x55a601038590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 shutdown_connections 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 1 -- 172.21.15.17:0/383897275 wait complete. 2024-03-06T07:27:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.636+0000 7efce5c54740 20 client.10236 trim_cache size 0 max 16384 2024-03-06T07:27:58.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.656+0000 7efce5c54740 1 Processor -- start 2024-03-06T07:27:58.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.656+0000 7efce5c54740 1 -- start start 2024-03-06T07:27:58.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60109e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601033c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1f7e60 con 0x55a60108b3a0 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc06db30 con 0x55a60103e330 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc1f6b30 con 0x55a60103df50 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60532/0 (socket says 172.21.15.17:60532) 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 -- 172.21.15.17:0/2884717129 learned_addr learned my addr 172.21.15.17:0/2884717129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601033c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce0e00640 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60109e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 -- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 msgr2=0x55a60109e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60109e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 -- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 msgr2=0x55a601033c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601033c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 -- 172.21.15.17:0/2884717129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a600d816b0 con 0x55a60108b3a0 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.657+0000 7efce1601640 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7efcd4019040 tx=0x7efcd4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad1c1db56d61b95 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efcd1ffb640 1 -- 172.21.15.17:0/2884717129 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd401a040 con 0x55a60108b3a0 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efcd1ffb640 1 -- 172.21.15.17:0/2884717129 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4009d20 con 0x55a60108b3a0 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efcd1ffb640 1 -- 172.21.15.17:0/2884717129 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd4009750 con 0x55a60108b3a0 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 -- 172.21.15.17:0/2884717129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 msgr2=0x55a601057130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7efcd4019040 tx=0x7efcd4015000 comp rx=0 tx=0).stop 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 -- 172.21.15.17:0/2884717129 shutdown_connections 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a601057130 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601033c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 --2- 172.21.15.17:0/2884717129 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60109e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 -- 172.21.15.17:0/2884717129 >> 172.21.15.17:0/2884717129 conn(0x55a60103c4b0 msgr2=0x55a60105c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 -- 172.21.15.17:0/2884717129 shutdown_connections 2024-03-06T07:27:58.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.658+0000 7efce5c54740 1 -- 172.21.15.17:0/2884717129 wait complete. 2024-03-06T07:27:58.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.659+0000 7efce5c54740 1 -- start start 2024-03-06T07:27:58.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.659+0000 7efce5c54740 1 Processor -- start 2024-03-06T07:27:58.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60107e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601057530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a60105e830 con 0x55a60108b3a0 2024-03-06T07:27:58.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a601061250 con 0x55a60103e330 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a60105f310 con 0x55a60103df50 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.660+0000 7efce5c54740 10 client.0 _ll_register_callbacks cb 0x7efccc0583c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60544/0 (socket says 172.21.15.17:60544) 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 -- 172.21.15.17:0/2407547623 learned_addr learned my addr 172.21.15.17:0/2407547623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce0e00640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60107e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efcdbfff640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601057530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 -- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 msgr2=0x55a60107e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60107e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 -- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 msgr2=0x55a601057530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601057530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 -- 172.21.15.17:0/2407547623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6010375b0 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce1601640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7efcd4007fd0 tx=0x7efcd4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890142063f287fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efcbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd401a070 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7efcd4015a60 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcd4009d20 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd403a050 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce5c54740 10 client.10242 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.661+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55a601094ec0 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd4009d20 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7efcd403d020 con 0x55a60108b3a0 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 10 client.10242 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 10 client.10242 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:27:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 20 client.10242 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a6010e7c10 0x55a601086a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 4 client.10242 mds_command: new command op to 9882 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 --> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a600fa9f40 con 0x55a6010e7c10 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce0e00640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a6010e7c10 0x55a601086a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efcbb7fe640 10 client.10242 ms_handle_connect on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efce0e00640 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a6010e7c10 0x55a601086a20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d50479675d6e7969 server_cookie=b63f8c71333f7571 in_seq=0 out_seq=0 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.662+0000 7efcbb7fe640 1 -- 172.21.15.17:0/2407547623 <== mds.1 v2:172.21.15.17:6832/3930100029 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a600fa9f40 con 0x55a6010e7c10 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efcbb7fe640 10 client.10242 handle_command_reply: tid=0 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 client.10242 shutdown 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 msgr2=0x55a6010e7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7efcd4007fd0 tx=0x7efcd4004ab0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 shutdown_connections 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:6832/3930100029,v1:172.21.15.17:6833/3930100029] conn(0x55a6010e7c10 0x55a601086a20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a60108b3a0 0x55a6010e7540 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a60103e330 0x55a601057530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 --2- 172.21.15.17:0/2407547623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a60103df50 0x55a60107e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 >> 172.21.15.17:0/2407547623 conn(0x55a60103c4b0 msgr2=0x55a60105c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efcbb7fe640 0 client.10242 ms_handle_reset on v2:172.21.15.17:6832/3930100029 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 shutdown_connections 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 1 -- 172.21.15.17:0/2407547623 wait complete. 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce5c54740 20 client.10242 trim_cache size 0 max 16384 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:58.671 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9882, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 2.771738085, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 182, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 188, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 188, 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 42.295904597000003 2024-03-06T07:27:58.672 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 msgr2=0x7efcbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 0x7efcbc06d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efcc4034000 tx=0x7efcc4008030 comp rx=0 tx=0).stop 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 msgr2=0x7efcdc0c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.663+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cb20 tx=0x7efccc001d50 comp rx=0 tx=0).stop 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 shutdown_connections 2024-03-06T07:27:58.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efcbc06b4d0 0x7efcbc06d990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcdc0c7170 0x7efcdc14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcdc0c6c40 0x7efcdc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 --2- 172.21.15.17:0/2150821569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcdc06ff10 0x7efcdc0c1f20 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 >> 172.21.15.17:0/2150821569 conn(0x7efcdc06b970 msgr2=0x7efcdc1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 shutdown_connections 2024-03-06T07:27:58.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.664+0000 7efce3064640 1 -- 172.21.15.17:0/2150821569 wait complete. 2024-03-06T07:27:58.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-06T07:27:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.781+0000 7f19a4caf640 1 Processor -- start 2024-03-06T07:27:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.781+0000 7f19a4caf640 1 -- start start 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f19a4caf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19a00c6dd0 0x7f19a0160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f19a4caf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a0155130 con 0x7f19a00c8950 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f19a4caf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00c7200 con 0x7f19a00c69f0 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f19a4caf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a0155440 con 0x7f19a00c6dd0 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f199ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f19a00c6dd0 0x7f19a0160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f199dd9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f19a00c8950 0x7f19a01558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53728/0 (socket says 172.21.15.17:53728) 2024-03-06T07:27:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.782+0000 7f199dd9b640 1 -- 172.21.15.17:0/2930460623 learned_addr learned my addr 172.21.15.17:0/2930460623 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.783+0000 7f199ed9d640 1 -- 172.21.15.17:0/2930460623 >> v1:172.21.15.163:6789/0 conn(0x7f19a00c69f0 legacy=0x7f19a00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.783+0000 7f199ed9d640 1 -- 172.21.15.17:0/2930460623 >> v1:172.21.15.17:6789/0 conn(0x7f19a00c8950 legacy=0x7f19a01558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.783+0000 7f199ed9d640 1 -- 172.21.15.17:0/2930460623 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00ad480 con 0x7f19a00c6dd0 2024-03-06T07:27:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.783+0000 7f199ed9d640 1 --2- 172.21.15.17:0/2930460623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0160ac0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f19a01555c0 tx=0x7f199002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887de9dde3504bd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f199d59a640 1 -- 172.21.15.17:0/2930460623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990035020 con 0x7f19a00c6dd0 2024-03-06T07:27:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f199d59a640 1 -- 172.21.15.17:0/2930460623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990031810 con 0x7f19a00c6dd0 2024-03-06T07:27:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f199d59a640 1 -- 172.21.15.17:0/2930460623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990031be0 con 0x7f19a00c6dd0 2024-03-06T07:27:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 -- 172.21.15.17:0/2930460623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 msgr2=0x7f19a0160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 --2- 172.21.15.17:0/2930460623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0160ac0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f19a01555c0 tx=0x7f199002b8b0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 -- 172.21.15.17:0/2930460623 shutdown_connections 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 --2- 172.21.15.17:0/2930460623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0160ac0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 -- 172.21.15.17:0/2930460623 >> 172.21.15.17:0/2930460623 conn(0x7f19a00cae70 msgr2=0x7f19a00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 -- 172.21.15.17:0/2930460623 shutdown_connections 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.784+0000 7f19a4caf640 1 -- 172.21.15.17:0/2930460623 wait complete. 2024-03-06T07:27:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.786+0000 7f19a4caf640 1 Processor -- start 2024-03-06T07:27:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.786+0000 7f19a4caf640 1 -- start start 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a00c69f0 0x7f19a00ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 0x7f19a00c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00c6310 con 0x7f19a00c6dd0 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00c6030 con 0x7f19a00c69f0 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f19a4caf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a0155fe0 con 0x7f19a00c8950 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60570/0 (socket says 172.21.15.17:60570) 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.787+0000 7f199dd9b640 1 -- 172.21.15.17:0/4150088492 learned_addr learned my addr 172.21.15.17:0/4150088492 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199e59c640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a00c69f0 0x7f19a00ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199ed9d640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 0x7f19a00c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 -- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 msgr2=0x7f19a00c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 0x7f19a00c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 -- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a00c69f0 msgr2=0x7f19a00ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a00c69f0 0x7f19a00ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 -- 172.21.15.17:0/4150088492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a01eef00 con 0x7f19a00c6dd0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199ed9d640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 0x7f19a00c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f199dd9b640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f199400cba0 tx=0x7f199400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f198effd640 1 -- 172.21.15.17:0/4150088492 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994017070 con 0x7f19a00c6dd0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f198effd640 1 -- 172.21.15.17:0/4150088492 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199400ace0 con 0x7f19a00c6dd0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a00c46a0 con 0x7f19a00c6dd0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f198effd640 1 -- 172.21.15.17:0/4150088492 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994012600 con 0x7f19a00c6dd0 2024-03-06T07:27:58.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.788+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a0158ea0 con 0x7f19a00c6dd0 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.791+0000 7f198effd640 1 -- 172.21.15.17:0/4150088492 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f199401a070 con 0x7f19a00c6dd0 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.791+0000 7f198effd640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 0x7f198006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.791+0000 7f19a7973740 1 Processor -- start 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.791+0000 7f198effd640 1 -- 172.21.15.17:0/4150088492 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1994089ff0 con 0x7f19a00c6dd0 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.791+0000 7f19a7973740 1 -- start start 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199e59c640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 0x7f198006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d39a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d39f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14ca06b0 con 0x55be14d3a4b0 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14c49a60 con 0x55be14d3bf50 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be14c9c750 con 0x55be14d3c330 2024-03-06T07:27:58.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60580/0 (socket says 172.21.15.17:60580) 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 -- 172.21.15.17:0/4218188634 learned_addr learned my addr 172.21.15.17:0/4218188634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199e59c640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 0x7f198006d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1998007fd0 tx=0x7f1998009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199dd9b640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d39f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199e59c640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d39a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 -- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 msgr2=0x55be14d39f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d39f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 -- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 msgr2=0x55be14d39a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d39a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199ed9d640 1 -- 172.21.15.17:0/4218188634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be14c67f30 con 0x55be14d3a4b0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.792+0000 7f199e59c640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d39a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f199ed9d640 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f1990034040 tx=0x7f1990030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9c25ca6279ea02 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f197ffff640 1 -- 172.21.15.17:0/4218188634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990035040 con 0x55be14d3a4b0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f197ffff640 1 -- 172.21.15.17:0/4218188634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990031ad0 con 0x55be14d3a4b0 2024-03-06T07:27:58.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f197ffff640 1 -- 172.21.15.17:0/4218188634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990044440 con 0x55be14d3a4b0 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 -- 172.21.15.17:0/4218188634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 msgr2=0x55be14d36680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f1990034040 tx=0x7f1990030000 comp rx=0 tx=0).stop 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 -- 172.21.15.17:0/4218188634 shutdown_connections 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3a4b0 0x55be14d36680 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d39f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 --2- 172.21.15.17:0/4218188634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d39a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.793+0000 7f19a7973740 1 -- 172.21.15.17:0/4218188634 >> 172.21.15.17:0/4218188634 conn(0x55be14d387d0 msgr2=0x55be14d5d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.794+0000 7f19a7973740 1 -- 172.21.15.17:0/4218188634 shutdown_connections 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.794+0000 7f19a7973740 1 -- 172.21.15.17:0/4218188634 wait complete. 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.794+0000 7f19a7973740 1 -- start start 2024-03-06T07:27:58.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.794+0000 7f19a7973740 1 Processor -- start 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14d5e230 con 0x55be14d5fb20 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14c5cea0 con 0x55be14d3bf50 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be14d5dc70 con 0x55be14d3c330 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f19a7973740 10 client.0 _ll_register_callbacks cb 0x55be14c5c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60586/0 (socket says 172.21.15.17:60586) 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199ed9d640 1 -- 172.21.15.17:0/632482424 learned_addr learned my addr 172.21.15.17:0/632482424 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199e59c640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199dd9b640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199ed9d640 1 -- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 msgr2=0x55be14d886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.795+0000 7f199ed9d640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f199ed9d640 1 -- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 msgr2=0x55be14d5a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f199ed9d640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f199ed9d640 1 -- 172.21.15.17:0/632482424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be14c5d270 con 0x55be14d5fb20 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f199ed9d640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f199002bec0 tx=0x7f1990004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fa075a360be67b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990035050 con 0x55be14d5fb20 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1990003810 con 0x55be14d5fb20 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1990031ad0 con 0x55be14d5fb20 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1990055050 con 0x55be14d5fb20 2024-03-06T07:27:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 10 client.10257 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55be149b52d0 con 0x55be14d5fb20 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990031ad0 con 0x55be14d5fb20 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f1990058020 con 0x55be14d5fb20 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 10 client.10257 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 10 client.10257 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.796+0000 7f19a7973740 20 client.10257 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14d5edb0 0x55be14d972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f19a7973740 4 client.10257 mds_command: new command op to 10140 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 --> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be14cadbf0 con 0x55be14d5edb0 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f199e59c640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14d5edb0 0x55be14d972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f197d7fa640 10 client.10257 ms_handle_connect on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.797+0000 7f199e59c640 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14d5edb0 0x55be14d972e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9182648049212f72 server_cookie=6d20656b9f450bc1 in_seq=0 out_seq=0 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f197d7fa640 1 -- 172.21.15.17:0/632482424 <== mds.0 v2:172.21.15.17:6834/2561439920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be14cadbf0 con 0x55be14d5edb0 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f197d7fa640 10 client.10257 handle_command_reply: tid=0 2024-03-06T07:27:58.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 client.10257 shutdown 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 msgr2=0x55be14d590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f199002bec0 tx=0x7f1990004730 comp rx=0 tx=0).stop 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 shutdown_connections 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14d5edb0 0x55be14d972e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14d590f0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f197d7fa640 0 client.10257 ms_handle_reset on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 --2- 172.21.15.17:0/632482424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 >> 172.21.15.17:0/632482424 conn(0x55be14d3a4b0 msgr2=0x55be14d3b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 shutdown_connections 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 1 -- 172.21.15.17:0/632482424 wait complete. 2024-03-06T07:27:58.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.798+0000 7f19a7973740 20 client.10257 trim_cache size 0 max 16384 2024-03-06T07:27:58.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 Processor -- start 2024-03-06T07:27:58.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 -- start start 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3bf50 0x55be14d85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 0x55be14d399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01f7e10 con 0x55be14d3c330 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01eb860 con 0x55be14d5fb20 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01f68f0 con 0x55be14d3bf50 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.815+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60602/0 (socket says 172.21.15.17:60602) 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 -- 172.21.15.17:0/29227890 learned_addr learned my addr 172.21.15.17:0/29227890 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199ed9d640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 0x55be14d399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199e59c640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3bf50 0x55be14d85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 -- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3bf50 msgr2=0x55be14d85fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3bf50 0x55be14d85fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 -- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 msgr2=0x55be14d399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 0x55be14d399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 -- 172.21.15.17:0/29227890 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be14a7f6b0 con 0x55be14d3c330 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199ed9d640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 0x55be14d399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f199dd9b640 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f1994058640 tx=0x7f19940549d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71594fac779809e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f197f7fe640 1 -- 172.21.15.17:0/29227890 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994086070 con 0x55be14d3c330 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f197f7fe640 1 -- 172.21.15.17:0/29227890 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994070030 con 0x55be14d3c330 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.816+0000 7f197f7fe640 1 -- 172.21.15.17:0/29227890 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994060880 con 0x55be14d3c330 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- 172.21.15.17:0/29227890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 msgr2=0x55be14d357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f1994058640 tx=0x7f19940549d0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- 172.21.15.17:0/29227890 shutdown_connections 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d5fb20 0x55be14d399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d3c330 0x55be14d357b0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 --2- 172.21.15.17:0/29227890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3bf50 0x55be14d85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- 172.21.15.17:0/29227890 >> 172.21.15.17:0/29227890 conn(0x55be14d3a4b0 msgr2=0x55be14d37b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- 172.21.15.17:0/29227890 shutdown_connections 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- 172.21.15.17:0/29227890 wait complete. 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 -- start start 2024-03-06T07:27:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.817+0000 7f19a7973740 1 Processor -- start 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14d61ad0 con 0x55be14d5fb20 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be14d61060 con 0x55be14d3bf50 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be14d60040 con 0x55be14d3c330 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f19a7973740 10 client.0 _ll_register_callbacks cb 0x7f1990031930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f199ed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60608/0 (socket says 172.21.15.17:60608) 2024-03-06T07:27:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f199ed9d640 1 -- 172.21.15.17:0/1518054174 learned_addr learned my addr 172.21.15.17:0/1518054174 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f199e59c640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.818+0000 7f199dd9b640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 -- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 msgr2=0x55be14d58e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d58e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 -- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 msgr2=0x55be14d5d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 -- 172.21.15.17:0/1518054174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be14d59cf0 con 0x55be14d5fb20 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f199ed9d640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f1990001eb0 tx=0x7f199003c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79c181153726107 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990035040 con 0x55be14d5fb20 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1990044440 con 0x55be14d5fb20 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1994070030 con 0x55be14d5fb20 2024-03-06T07:27:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=182) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1990063050 con 0x55be14d5fb20 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 10 client.10263 fetch_fsmap learned FSMap version 182 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=182,monmap=2+}) v3 -- 0x55be14deb800 con 0x55be14d5fb20 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994070030 con 0x55be14d5fb20 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 182) v1 ==== 3334+0+0 (secure 0 0 0) 0x7f1990066020 con 0x55be14d5fb20 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 10 client.10263 fetch_fsmap finished waiting for FSMap version 182 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 10 client.10263 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 20 client.10263 populate_metadata read hostname 'smithi017' 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14de5d20 0x55be14d74280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 4 client.10263 mds_command: new command op to 10140 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.819+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 --> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be14ca7f40 con 0x55be14de5d20 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f199e59c640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14de5d20 0x55be14d74280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f197d7fa640 10 client.10263 ms_handle_connect on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f199e59c640 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14de5d20 0x55be14d74280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c4591e6c5fce2e server_cookie=18b2c5fd9efe9069 in_seq=0 out_seq=0 2024-03-06T07:27:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f197d7fa640 1 -- 172.21.15.17:0/1518054174 <== mds.0 v2:172.21.15.17:6834/2561439920 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55be14ca7f40 con 0x55be14de5d20 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f197d7fa640 10 client.10263 handle_command_reply: tid=0 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 client.10263 shutdown 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 msgr2=0x55be14de55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f1990001eb0 tx=0x7f199003c040 comp rx=0 tx=0).stop 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 shutdown_connections 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:6834/2561439920,v1:172.21.15.17:6835/2561439920] conn(0x55be14de5d20 0x55be14d74280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55be14d5fb20 0x55be14de55f0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55be14d3c330 0x55be14d58e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 --2- 172.21.15.17:0/1518054174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55be14d3bf50 0x55be14d5d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f197d7fa640 0 client.10263 ms_handle_reset on v2:172.21.15.17:6834/2561439920 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 >> 172.21.15.17:0/1518054174 conn(0x55be14d3a4b0 msgr2=0x55be14d399a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 shutdown_connections 2024-03-06T07:27:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.820+0000 7f19a7973740 1 -- 172.21.15.17:0/1518054174 wait complete. 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a7973740 20 client.10263 trim_cache size 0 max 16384 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 10140, 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 3.936026027, 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 182, 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 188, 2024-03-06T07:27:58.829 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 188, 2024-03-06T07:27:58.830 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5.9231970629999999 2024-03-06T07:27:58.830 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 msgr2=0x7f198006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 0x7f198006d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1998007fd0 tx=0x7f1998009040 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 msgr2=0x7f19a0158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f199400cba0 tx=0x7f199400a430 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 shutdown_connections 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f198006b4d0 0x7f198006d990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19a00c8950 0x7f19a00c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19a00c6dd0 0x7f19a0158960 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 --2- 172.21.15.17:0/4150088492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19a00c69f0 0x7f19a00ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 >> 172.21.15.17:0/4150088492 conn(0x7f19a00cae70 msgr2=0x7f19a00bff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:58.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 shutdown_connections 2024-03-06T07:27:58.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:58.821+0000 7f19a4caf640 1 -- 172.21.15.17:0/4150088492 wait complete. 2024-03-06T07:27:58.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:27:58.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:58.880 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:27:58.881 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:27:58.880+0000 7f4fe65e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:27:58.881 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-06T07:27:58.880+0000 7f4fe65e3640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-06T07:27:58.933 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:27:58.933 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:27:58.933 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:58.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:27:58.947 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:27:58.947 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:27:58.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:27:58.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:27:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.090+0000 7f80ab647640 1 Processor -- start 2024-03-06T07:27:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.090+0000 7f80ab647640 1 -- start start 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80ab647640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80a41550c0 0x7f80a41554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80ab647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41559e0 0x7f80a414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80ab647640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a40aa660 con 0x7f80a40aa180 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80ab647640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a40c4610 con 0x7f80a41550c0 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80ab647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40aa940 con 0x7f80a41559e0 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80a93e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80a41550c0 0x7f80a41554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80a93e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80a41550c0 0x7f80a41554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60612/0 (socket says 172.21.15.17:60612) 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80a93e3640 1 -- 172.21.15.17:0/1322026000 learned_addr learned my addr 172.21.15.17:0/1322026000 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.091+0000 7f80a8be2640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41559e0 0x7f80a414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f80a93e3640 1 -- 172.21.15.17:0/1322026000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41559e0 msgr2=0x7f80a414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f80a93e3640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41559e0 0x7f80a414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f80a93e3640 1 -- 172.21.15.17:0/1322026000 >> v1:172.21.15.163:6789/0 conn(0x7f80a40aa180 legacy=0x7f80a40c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f80a93e3640 1 -- 172.21.15.17:0/1322026000 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40c4860 con 0x7f80a41550c0 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f80a93e3640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41550c0 0x7f80a41554a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f809c009870 tx=0x7f809c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3101799bebe0981 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f809bfff640 1 -- 172.21.15.17:0/1322026000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c0317c0 con 0x7f80a41550c0 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.092+0000 7f809bfff640 1 -- 172.21.15.17:0/1322026000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c037070 con 0x7f80a41550c0 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f809bfff640 1 -- 172.21.15.17:0/1322026000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c031b80 con 0x7f80a41550c0 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 -- 172.21.15.17:0/1322026000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41550c0 msgr2=0x7f80a41554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41550c0 0x7f80a41554a0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f809c009870 tx=0x7f809c02b860 comp rx=0 tx=0).stop 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 -- 172.21.15.17:0/1322026000 shutdown_connections 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41559e0 0x7f80a414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 --2- 172.21.15.17:0/1322026000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41550c0 0x7f80a41554a0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 -- 172.21.15.17:0/1322026000 >> 172.21.15.17:0/1322026000 conn(0x7f80a40c6710 msgr2=0x7f80a40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 -- 172.21.15.17:0/1322026000 shutdown_connections 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.093+0000 7f80ab647640 1 -- 172.21.15.17:0/1322026000 wait complete. 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.095+0000 7f80ab647640 1 Processor -- start 2024-03-06T07:27:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.095+0000 7f80ab647640 1 -- start start 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 0x7f80a41e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 0x7f80a40c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414f140 con 0x7f80a41559e0 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40c1520 con 0x7f80a40aa180 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80ab647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414f2c0 con 0x7f80a41550c0 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80a9be4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80a9be4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60614/0 (socket says 172.21.15.17:60614) 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80a9be4640 1 -- 172.21.15.17:0/887759070 learned_addr learned my addr 172.21.15.17:0/887759070 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80a8be2640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 0x7f80a40c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.096+0000 7f80a93e3640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 0x7f80a41e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 -- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 msgr2=0x7f80a40c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 0x7f80a40c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 -- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 msgr2=0x7f80a41e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 0x7f80a41e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 -- 172.21.15.17:0/887759070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a41493b0 con 0x7f80a41559e0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a93e3640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 0x7f80a41e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a8be2640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 0x7f80a40c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80a9be4640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c6c0 tx=0x7f80a000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0003b70 con 0x7f80a41559e0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a0003d10 con 0x7f80a41559e0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a00052f0 con 0x7f80a41559e0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a4149690 con 0x7f80a41559e0 2024-03-06T07:27:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.097+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a4147ee0 con 0x7f80a41559e0 2024-03-06T07:27:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.098+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a41480e0 con 0x7f80a41559e0 2024-03-06T07:27:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.099+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80a0005490 con 0x7f80a41559e0 2024-03-06T07:27:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.099+0000 7f8099ffb640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 0x7f808806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.099+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f80a008a200 con 0x7f80a41559e0 2024-03-06T07:27:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.099+0000 7f80a93e3640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 0x7f808806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.100+0000 7f80a93e3640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 0x7f808806d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f809c036000 tx=0x7f809c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.107+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f80a41480e0 con 0x7f80a41559e0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.325+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80a404f330 con 0x7f80a41559e0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.326+0000 7f8099ffb640 1 -- 172.21.15.17:0/887759070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7f80a00588c0 con 0x7f80a41559e0 2024-03-06T07:27:59.327 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:27:59.328 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 msgr2=0x7f808806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 0x7f808806d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f809c036000 tx=0x7f809c008030 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 msgr2=0x7f80a4148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f80a000c6c0 tx=0x7f80a000cb80 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 shutdown_connections 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f808806b4d0 0x7f808806d990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80a41559e0 0x7f80a4148400 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80a41550c0 0x7f80a40c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 --2- 172.21.15.17:0/887759070 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80a40aa180 0x7f80a41e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 >> 172.21.15.17:0/887759070 conn(0x7f80a40c6710 msgr2=0x7f80a40a9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 shutdown_connections 2024-03-06T07:27:59.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.328+0000 7f80ab647640 1 -- 172.21.15.17:0/887759070 wait complete. 2024-03-06T07:27:59.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:27:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.441+0000 7fe075239640 1 Processor -- start 2024-03-06T07:27:59.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.441+0000 7fe075239640 1 -- start start 2024-03-06T07:27:59.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe075239640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0700c7180 0x7fe070159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe075239640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700c7c60 con 0x7fe0700c88b0 2024-03-06T07:27:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe075239640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700c8060 con 0x7fe0701527b0 2024-03-06T07:27:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe075239640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe070159640 con 0x7fe0700c7180 2024-03-06T07:27:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe067fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe0700c7180 0x7fe070159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe06f59e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe0700c88b0 0x7fe0700c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53730/0 (socket says 172.21.15.17:53730) 2024-03-06T07:27:59.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe06f59e640 1 -- 172.21.15.17:0/3843634822 learned_addr learned my addr 172.21.15.17:0/3843634822 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe067fff640 1 -- 172.21.15.17:0/3843634822 >> v1:172.21.15.163:6789/0 conn(0x7fe0701527b0 legacy=0x7fe070155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe067fff640 1 -- 172.21.15.17:0/3843634822 >> v1:172.21.15.17:6789/0 conn(0x7fe0700c88b0 legacy=0x7fe0700c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.442+0000 7fe06e59c640 1 -- 172.21.15.17:0/3843634822 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522585509 0 0) 0x7fe068001490 con 0x7fe0700c88b0 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.443+0000 7fe067fff640 1 -- 172.21.15.17:0/3843634822 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700c76a0 con 0x7fe0700c7180 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe067fff640 1 --2- 172.21.15.17:0/3843634822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c7180 0x7fe070159100 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe0580097a0 tx=0x7fe05802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4efd9d369cea65d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe06e59c640 1 -- 172.21.15.17:0/3843634822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058035020 con 0x7fe0700c7180 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe06e59c640 1 -- 172.21.15.17:0/3843634822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe058031830 con 0x7fe0700c7180 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe06e59c640 1 -- 172.21.15.17:0/3843634822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058031c00 con 0x7fe0700c7180 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 -- 172.21.15.17:0/3843634822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c7180 msgr2=0x7fe070159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 --2- 172.21.15.17:0/3843634822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c7180 0x7fe070159100 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe0580097a0 tx=0x7fe05802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 -- 172.21.15.17:0/3843634822 shutdown_connections 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 --2- 172.21.15.17:0/3843634822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c7180 0x7fe070159100 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 -- 172.21.15.17:0/3843634822 >> 172.21.15.17:0/3843634822 conn(0x7fe0700ac560 msgr2=0x7fe0700ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 -- 172.21.15.17:0/3843634822 shutdown_connections 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.444+0000 7fe075239640 1 -- 172.21.15.17:0/3843634822 wait complete. 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 Processor -- start 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 -- start start 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 0x7fe07014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701527b0 0x7fe07014b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070155600 con 0x7fe0700c88b0 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe070155320 con 0x7fe0701527b0 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe075239640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe070152b90 con 0x7fe0700c7180 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe067fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe067fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60640/0 (socket says 172.21.15.17:60640) 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.446+0000 7fe067fff640 1 -- 172.21.15.17:0/4268497487 learned_addr learned my addr 172.21.15.17:0/4268497487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe06f59e640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701527b0 0x7fe07014b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe06ed9d640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 0x7fe07014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 -- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 msgr2=0x7fe07014d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 0x7fe07014d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 -- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701527b0 msgr2=0x7fe07014b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701527b0 0x7fe07014b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 -- 172.21.15.17:0/4268497487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700c3a10 con 0x7fe0700c88b0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe06ed9d640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 0x7fe07014d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe067fff640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe0580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058035050 con 0x7fe0700c88b0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe058004230 con 0x7fe0700c88b0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05803b740 con 0x7fe0700c88b0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0700c3cf0 con 0x7fe0700c88b0 2024-03-06T07:27:59.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.447+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0700c48c0 con 0x7fe0700c88b0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.449+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07004f330 con 0x7fe0700c88b0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.449+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe058038020 con 0x7fe0700c88b0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.450+0000 7fe0677fe640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 0x7fe05006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.450+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe0580ac100 con 0x7fe0700c88b0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.450+0000 7fe06ed9d640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 0x7fe05006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.450+0000 7fe06ed9d640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 0x7fe05006d9c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe060007fd0 tx=0x7fe060009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.454+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe058040070 con 0x7fe0700c88b0 2024-03-06T07:27:59.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.741+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe07006ce00 con 0x7fe0700c88b0 2024-03-06T07:27:59.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.742+0000 7fe0677fe640 1 -- 172.21.15.17:0/4268497487 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fe0580a3e20 con 0x7fe0700c88b0 2024-03-06T07:27:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:27:59.744 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":188,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:54.873995+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 msgr2=0x7fe05006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 0x7fe05006d9c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe060007fd0 tx=0x7fe060009040 comp rx=0 tx=0).stop 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 msgr2=0x7fe07014d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe0580046c0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 shutdown_connections 2024-03-06T07:27:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe05006b500 0x7fe05006d9c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0701527b0 0x7fe07014b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0700c88b0 0x7fe07014d9a0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 --2- 172.21.15.17:0/4268497487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0700c7180 0x7fe07014d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.744+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 >> 172.21.15.17:0/4268497487 conn(0x7fe0700ac560 msgr2=0x7fe07006b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.745+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 shutdown_connections 2024-03-06T07:27:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.745+0000 7fe075239640 1 -- 172.21.15.17:0/4268497487 wait complete. 2024-03-06T07:27:59.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:27:25.147974+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:27:25.852438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:27:59.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:27:59.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.875+0000 7fba8a1e9640 1 Processor -- start 2024-03-06T07:27:59.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.875+0000 7fba8a1e9640 1 -- start start 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba8a1e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba8a1e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba84156060 con 0x7fba8406db30 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba8a1e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba840a8430 con 0x7fba8406d750 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba8a1e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba84155b30 con 0x7fba840c3a70 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba83fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fba8406db30 0x7fba840c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53744/0 (socket says 172.21.15.17:53744) 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba83fff640 1 -- 172.21.15.17:0/4065592934 learned_addr learned my addr 172.21.15.17:0/4065592934 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba7bfff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266832203 0 0) 0x7fba84156060 con 0x7fba8406db30 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.876+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba70003520 con 0x7fba8406db30 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1874931149 0 0) 0x7fba70003520 con 0x7fba8406db30 2024-03-06T07:27:59.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 msgr2=0x7fba840732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba840732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 >> v1:172.21.15.163:6789/0 conn(0x7fba8406d750 legacy=0x7fba84072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84155d80 con 0x7fba8406db30 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fba7c0019f0 con 0x7fba8406db30 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 >> v1:172.21.15.17:6789/0 conn(0x7fba8406db30 legacy=0x7fba840c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70003f60 0x7fba70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba70004d40 0x7fba700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba84155b30 con 0x7fba70004d40 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7000bc00 con 0x7fba70003f60 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7000be00 con 0x7fba840c3a70 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba70004d40 0x7fba700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba7bfff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70003f60 0x7fba70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba837fe640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 -- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 msgr2=0x7fba700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba840c3a70 0x7fba700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 -- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70003f60 msgr2=0x7fba70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70003f60 0x7fba70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.877+0000 7fba83fff640 1 -- 172.21.15.17:0/4065592934 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70003520 con 0x7fba70004d40 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.878+0000 7fba83fff640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba70004d40 0x7fba700051c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00a840 tx=0x7fba70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7848a39bf0f20bc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.878+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00a460 con 0x7fba70004d40 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.878+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c034650 con 0x7fba70004d40 2024-03-06T07:27:59.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.878+0000 7fba82ffd640 1 -- 172.21.15.17:0/4065592934 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c034a80 con 0x7fba70004d40 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 -- 172.21.15.17:0/4065592934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba70004d40 msgr2=0x7fba700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/4065592934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba70004d40 0x7fba700051c0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00a840 tx=0x7fba70005830 comp rx=0 tx=0).stop 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba83fff640 1 -- 172.21.15.17:0/4065592934 reap_dead start 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 -- 172.21.15.17:0/4065592934 shutdown_connections 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 -- 172.21.15.17:0/4065592934 >> 172.21.15.17:0/4065592934 conn(0x7fba840c5560 msgr2=0x7fba840c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 -- 172.21.15.17:0/4065592934 shutdown_connections 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.879+0000 7fba8a1e9640 1 -- 172.21.15.17:0/4065592934 wait complete. 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.880+0000 7fba8a1e9640 1 Processor -- start 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.880+0000 7fba8a1e9640 1 -- start start 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70004d40 0x7fba84150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba841eb0b0 0x7fba8414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840c2720 con 0x7fba840ac830 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840c25a0 con 0x7fba70004d40 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba8a1e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840a7cd0 con 0x7fba841eb0b0 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60666/0 (socket says 172.21.15.17:60666) 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 -- 172.21.15.17:0/784561313 learned_addr learned my addr 172.21.15.17:0/784561313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba83fff640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba841eb0b0 0x7fba8414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba837fe640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70004d40 0x7fba84150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 -- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba841eb0b0 msgr2=0x7fba8414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba841eb0b0 0x7fba8414c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 -- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70004d40 msgr2=0x7fba84150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70004d40 0x7fba84150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:27:59.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 -- 172.21.15.17:0/784561313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba841ee750 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.881+0000 7fba7bfff640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011910 tx=0x7fba6c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.882+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c019070 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.882+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c004240 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.882+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00f7c0 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.882+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba841ee9f0 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.882+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8414d5c0 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.883+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8404f330 con 0x7fba840ac830 2024-03-06T07:27:59.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.883+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba6c01c070 con 0x7fba840ac830 2024-03-06T07:27:59.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.884+0000 7fba80ff9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:27:59.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.884+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba6c08a7e0 con 0x7fba840ac830 2024-03-06T07:27:59.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.884+0000 7fba837fe640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:27:59.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.885+0000 7fba837fe640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fba74007fd0 tx=0x7fba74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:27:59.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:27:59.890+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fba6c0594f0 con 0x7fba840ac830 2024-03-06T07:28:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.232+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba840c1e90 con 0x7fba840ac830 2024-03-06T07:28:00.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.232+0000 7fba80ff9640 1 -- 172.21.15.17:0/784561313 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba6c058df0 con 0x7fba840ac830 2024-03-06T07:28:00.234 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:28:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 msgr2=0x7fba6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 0x7fba6406d990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fba74007fd0 tx=0x7fba74009040 comp rx=0 tx=0).stop 2024-03-06T07:28:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 msgr2=0x7fba841eab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fba6c011910 tx=0x7fba6c011dd0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 shutdown_connections 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fba6406b4d0 0x7fba6406d990 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba841eb0b0 0x7fba8414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba840ac830 0x7fba841eab70 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 --2- 172.21.15.17:0/784561313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba70004d40 0x7fba84150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 >> 172.21.15.17:0/784561313 conn(0x7fba840c5560 msgr2=0x7fba840c28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 shutdown_connections 2024-03-06T07:28:00.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.234+0000 7fba8a1e9640 1 -- 172.21.15.17:0/784561313 wait complete. 2024-03-06T07:28:00.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:28:00.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 Processor -- start 2024-03-06T07:28:00.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 -- start start 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff60155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff6006ca70 0x7eff600c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600b0450 con 0x7eff600c5380 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6006c020 con 0x7eff6006ca70 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.351+0000 7eff66756640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600c77f0 con 0x7eff600b0070 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff64cf3640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7eff600c5380 0x7eff6014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:53758/0 (socket says 172.21.15.17:53758) 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff64cf3640 1 -- 172.21.15.17:0/4156631282 learned_addr learned my addr 172.21.15.17:0/4156631282 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5ffff640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff60155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/4156631282 >> v2:172.21.15.163:3300/0 conn(0x7eff6006ca70 0x7eff600c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844980768 0 0) 0x7eff58001490 con 0x7eff600c5380 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff48003520 con 0x7eff600c5380 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 662546743 0 0) 0x7eff48003520 con 0x7eff600c5380 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 msgr2=0x7eff60155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff60155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 >> v2:172.21.15.163:3300/0 conn(0x7eff6006ca70 msgr2=0x7eff600c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 --2- 172.21.15.17:0/4156631282 >> v2:172.21.15.163:3300/0 conn(0x7eff6006ca70 0x7eff600c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600c7ad0 con 0x7eff600c5380 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7eff58001bc0 con 0x7eff600c5380 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 >> v1:172.21.15.17:6789/0 conn(0x7eff600c5380 legacy=0x7eff6014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff480039b0 0x7eff48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6006c020 con 0x7eff48004800 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600c77f0 con 0x7eff480039b0 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4800ba10 con 0x7eff600b0070 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.352+0000 7eff5ffff640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff480039b0 0x7eff48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff64cf3640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 -- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 msgr2=0x7eff480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff600b0070 0x7eff480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 -- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff480039b0 msgr2=0x7eff48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff480039b0 0x7eff48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 -- 172.21.15.17:0/4156631282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff48003520 con 0x7eff48004800 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.353+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff48004c80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7eff54009af0 tx=0x7eff48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcf89b6dd772423 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54039070 con 0x7eff48004800 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5402c710 con 0x7eff48004800 2024-03-06T07:28:00.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff5effd640 1 -- 172.21.15.17:0/4156631282 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54009690 con 0x7eff48004800 2024-03-06T07:28:00.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 -- 172.21.15.17:0/4156631282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 msgr2=0x7eff48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 --2- 172.21.15.17:0/4156631282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff48004c80 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7eff54009af0 tx=0x7eff48005830 comp rx=0 tx=0).stop 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff64cf3640 1 -- 172.21.15.17:0/4156631282 reap_dead start 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 -- 172.21.15.17:0/4156631282 shutdown_connections 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 -- 172.21.15.17:0/4156631282 >> 172.21.15.17:0/4156631282 conn(0x7eff600ac590 msgr2=0x7eff600ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 -- 172.21.15.17:0/4156631282 shutdown_connections 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.354+0000 7eff66756640 1 -- 172.21.15.17:0/4156631282 wait complete. 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.355+0000 7eff66756640 1 Processor -- start 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 -- start start 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff601eb4f0 0x7eff60154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 0x7eff600c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6006cef0 con 0x7eff48004800 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6006c550 con 0x7eff600c2ae0 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff66756640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6006da80 con 0x7eff601eb4f0 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff5ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff5ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60684/0 (socket says 172.21.15.17:60684) 2024-03-06T07:28:00.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.356+0000 7eff5ffff640 1 -- 172.21.15.17:0/2130043647 learned_addr learned my addr 172.21.15.17:0/2130043647 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff64cf3640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 0x7eff600c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff601eb4f0 0x7eff60154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 -- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff601eb4f0 msgr2=0x7eff60154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff601eb4f0 0x7eff60154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 -- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 msgr2=0x7eff600c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 0x7eff600c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 -- 172.21.15.17:0/2130043647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600c34a0 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff64cf3640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 0x7eff600c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5ffff640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7eff500119a0 tx=0x7eff50011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000fa90 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000fc30 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff60159230 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50014440 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.357+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff60158410 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.358+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff60158200 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.359+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff50012040 con 0x7eff48004800 2024-03-06T07:28:00.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.359+0000 7eff5cff9640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 0x7eff4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.359+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7eff5008a940 con 0x7eff48004800 2024-03-06T07:28:00.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.360+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 0x7eff4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.360+0000 7eff5f7fe640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 0x7eff4c06d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7eff54036000 tx=0x7eff54030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.363+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff50019030 con 0x7eff48004800 2024-03-06T07:28:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.670+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff6004f330 con 0x7eff48004800 2024-03-06T07:28:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.671+0000 7eff5cff9640 1 -- 172.21.15.17:0/2130043647 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v188) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff500595c0 con 0x7eff48004800 2024-03-06T07:28:00.676 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 msgr2=0x7eff4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 0x7eff4c06d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7eff54036000 tx=0x7eff54030000 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 msgr2=0x7eff600c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7eff500119a0 tx=0x7eff50011e60 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 shutdown_connections 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7eff4c06b4d0 0x7eff4c06d990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff600c2ae0 0x7eff600c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff601eb4f0 0x7eff60154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 --2- 172.21.15.17:0/2130043647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff48004800 0x7eff600c4960 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 >> 172.21.15.17:0/2130043647 conn(0x7eff600ac590 msgr2=0x7eff600ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 shutdown_connections 2024-03-06T07:28:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.673+0000 7eff66756640 1 -- 172.21.15.17:0/2130043647 wait complete. 2024-03-06T07:28:00.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:28:00.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.789+0000 7f01533eb640 1 Processor -- start 2024-03-06T07:28:00.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.789+0000 7f01533eb640 1 -- start start 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.790+0000 7f01533eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f014c0c3a70 0x7f014c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.790+0000 7f01533eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06db30 0x7f014c0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.790+0000 7f01533eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c156060 con 0x7f014c06d750 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.790+0000 7f01533eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c155b30 con 0x7f014c0c3a70 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.790+0000 7f01533eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0a8430 con 0x7f014c06db30 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f014c0c3a70 0x7f014c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f014c0c3a70 0x7f014c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32890/0 (socket says 172.21.15.17:32890) 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 -- 172.21.15.17:0/2797827645 learned_addr learned my addr 172.21.15.17:0/2797827645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0151988640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06db30 0x7f014c0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 -- 172.21.15.17:0/2797827645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06db30 msgr2=0x7f014c0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06db30 0x7f014c0c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 -- 172.21.15.17:0/2797827645 >> v1:172.21.15.163:6789/0 conn(0x7f014c06d750 legacy=0x7f014c072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:00.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.791+0000 7f0150986640 1 -- 172.21.15.17:0/2797827645 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c155d80 con 0x7f014c0c3a70 2024-03-06T07:28:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f0150986640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c0c3a70 0x7f014c0732a0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f013c009870 tx=0x7f013c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9395583e173c16 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f0143fff640 1 -- 172.21.15.17:0/2797827645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c0317c0 con 0x7f014c0c3a70 2024-03-06T07:28:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f0143fff640 1 -- 172.21.15.17:0/2797827645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c037070 con 0x7f014c0c3a70 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f0143fff640 1 -- 172.21.15.17:0/2797827645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c031b80 con 0x7f014c0c3a70 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 -- 172.21.15.17:0/2797827645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c0c3a70 msgr2=0x7f014c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c0c3a70 0x7f014c0732a0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f013c009870 tx=0x7f013c02b860 comp rx=0 tx=0).stop 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 -- 172.21.15.17:0/2797827645 shutdown_connections 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06db30 0x7f014c0c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 --2- 172.21.15.17:0/2797827645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c0c3a70 0x7f014c0732a0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.792+0000 7f01533eb640 1 -- 172.21.15.17:0/2797827645 >> 172.21.15.17:0/2797827645 conn(0x7f014c0c5560 msgr2=0x7f014c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.793+0000 7f01533eb640 1 -- 172.21.15.17:0/2797827645 shutdown_connections 2024-03-06T07:28:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.793+0000 7f01533eb640 1 -- 172.21.15.17:0/2797827645 wait complete. 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 Processor -- start 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 -- start start 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 0x7f014c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c0c3a70 0x7f014c1e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0a7e50 con 0x7f014c06db30 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0a7cd0 con 0x7f014c0c3a70 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f01533eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c0a7fd0 con 0x7f014c06d750 2024-03-06T07:28:00.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f0150986640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f0150986640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32892/0 (socket says 172.21.15.17:32892) 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.794+0000 7f0150986640 1 -- 172.21.15.17:0/2317418712 learned_addr learned my addr 172.21.15.17:0/2317418712 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0151187640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 0x7f014c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0151988640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c0c3a70 0x7f014c1e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 -- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 msgr2=0x7f014c1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 0x7f014c1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 -- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c0c3a70 msgr2=0x7f014c1e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c0c3a70 0x7f014c1e1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 -- 172.21.15.17:0/2317418712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c1eb450 con 0x7f014c06db30 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0151187640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 0x7f014c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0150986640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f013c000c00 tx=0x7f013c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c032660 con 0x7f014c06db30 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c032800 con 0x7f014c06db30 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c1eb6d0 con 0x7f014c06db30 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c032bf0 con 0x7f014c06db30 2024-03-06T07:28:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.795+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c154720 con 0x7f014c06db30 2024-03-06T07:28:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.797+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c04f330 con 0x7f014c06db30 2024-03-06T07:28:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.797+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f013c04d3f0 con 0x7f014c06db30 2024-03-06T07:28:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.797+0000 7f0141ffb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.797+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f013c0abf10 con 0x7f014c06db30 2024-03-06T07:28:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.797+0000 7f0151187640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.798+0000 7f0151187640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 0x7f013006d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f01480015e0 tx=0x7f0148009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.802+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f014c1eb450 con 0x7f014c06db30 2024-03-06T07:28:00.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.986+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f014c0c2900 con 0x7f014c06db30 2024-03-06T07:28:00.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.987+0000 7f0141ffb640 1 -- 172.21.15.17:0/2317418712 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v188) v1 ==== 100+0+11 (secure 0 0 0) 0x7f013c040070 con 0x7f014c06db30 2024-03-06T07:28:00.988 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:28:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 msgr2=0x7f013006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 0x7f013006d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f01480015e0 tx=0x7f0148009040 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 msgr2=0x7f014c1e19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f013c000c00 tx=0x7f013c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 shutdown_connections 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f014c0c3a70 0x7f014c1e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f014c06db30 0x7f014c1e19d0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 --2- 172.21.15.17:0/2317418712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f014c06d750 0x7f014c1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 >> 172.21.15.17:0/2317418712 conn(0x7f014c0c5560 msgr2=0x7f014c0c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 shutdown_connections 2024-03-06T07:28:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:00.989+0000 7f01533eb640 1 -- 172.21.15.17:0/2317418712 wait complete. 2024-03-06T07:28:01.001 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:01.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.122+0000 7fc3f9525640 1 Processor -- start 2024-03-06T07:28:01.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.122+0000 7fc3f9525640 1 -- start start 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f9525640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3f4152920 0x7fc3f4155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f9525640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3f40c2620 0x7fc3f40c1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f9525640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f406afc0 con 0x7fc3f40c2190 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f9525640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f4156160 con 0x7fc3f40c2620 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f9525640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f406b2a0 con 0x7fc3f4152920 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3f40c2620 0x7fc3f40c1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3ebfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc3f40c2190 0x7fc3f40c1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40716/0 (socket says 172.21.15.17:40716) 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3ebfff640 1 -- 172.21.15.17:0/2184997951 learned_addr learned my addr 172.21.15.17:0/2184997951 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f2ffd640 1 --2- 172.21.15.17:0/2184997951 >> v2:172.21.15.163:3300/0 conn(0x7fc3f4152920 0x7fc3f4155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f37fe640 1 -- 172.21.15.17:0/2184997951 >> v2:172.21.15.163:3300/0 conn(0x7fc3f4152920 msgr2=0x7fc3f4155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f37fe640 1 --2- 172.21.15.17:0/2184997951 >> v2:172.21.15.163:3300/0 conn(0x7fc3f4152920 0x7fc3f4155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f37fe640 1 -- 172.21.15.17:0/2184997951 >> v1:172.21.15.17:6789/0 conn(0x7fc3f40c2190 legacy=0x7fc3f40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f27fc640 1 -- 172.21.15.17:0/2184997951 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225197445 0 0) 0x7fc3f406afc0 con 0x7fc3f40c2190 2024-03-06T07:28:01.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.124+0000 7fc3f37fe640 1 -- 172.21.15.17:0/2184997951 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f406b580 con 0x7fc3f40c2620 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.125+0000 7fc3f37fe640 1 --2- 172.21.15.17:0/2184997951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2620 0x7fc3f40c1990 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4070a90 tx=0x7fc3ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354fc972bd26ce3a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.125+0000 7fc3f27fc640 1 -- 172.21.15.17:0/2184997951 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec039070 con 0x7fc3f40c2620 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.125+0000 7fc3f27fc640 1 -- 172.21.15.17:0/2184997951 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec02cc80 con 0x7fc3f40c2620 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.125+0000 7fc3f27fc640 1 -- 172.21.15.17:0/2184997951 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec034660 con 0x7fc3f40c2620 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 -- 172.21.15.17:0/2184997951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2620 msgr2=0x7fc3f40c1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 --2- 172.21.15.17:0/2184997951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2620 0x7fc3f40c1990 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4070a90 tx=0x7fc3ec02bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 -- 172.21.15.17:0/2184997951 shutdown_connections 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 --2- 172.21.15.17:0/2184997951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2620 0x7fc3f40c1990 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 --2- 172.21.15.17:0/2184997951 >> v2:172.21.15.163:3300/0 conn(0x7fc3f4152920 0x7fc3f4155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 -- 172.21.15.17:0/2184997951 >> 172.21.15.17:0/2184997951 conn(0x7fc3f40c4290 msgr2=0x7fc3f40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 -- 172.21.15.17:0/2184997951 shutdown_connections 2024-03-06T07:28:01.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.126+0000 7fc3f9525640 1 -- 172.21.15.17:0/2184997951 wait complete. 2024-03-06T07:28:01.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.127+0000 7fc3f9525640 1 Processor -- start 2024-03-06T07:28:01.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.127+0000 7fc3f9525640 1 -- start start 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 0x7fc3f4072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 0x7fc3f414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40a7100 con 0x7fc3f40c2190 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40c0d60 con 0x7fc3f4152920 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f9525640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f40a7280 con 0x7fc3f40c2620 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32916/0 (socket says 172.21.15.17:32916) 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 -- 172.21.15.17:0/526306755 learned_addr learned my addr 172.21.15.17:0/526306755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f37fe640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 0x7fc3f414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3ebfff640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 0x7fc3f4072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 -- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 msgr2=0x7fc3f4072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 0x7fc3f4072a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 -- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 msgr2=0x7fc3f414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 0x7fc3f414ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3f2ffd640 1 -- 172.21.15.17:0/526306755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f41eb5c0 con 0x7fc3f40c2190 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.128+0000 7fc3ebfff640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 0x7fc3f4072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3f37fe640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 0x7fc3f414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3f2ffd640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400d5f0 tx=0x7fc3e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40125f0 con 0x7fc3f40c2190 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e4019070 con 0x7fc3f40c2190 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f41eb8a0 con 0x7fc3f40c2190 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40129d0 con 0x7fc3f40c2190 2024-03-06T07:28:01.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.129+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f414fa70 con 0x7fc3f40c2190 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.130+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f404f330 con 0x7fc3f40c2190 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.130+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3e4012b70 con 0x7fc3f40c2190 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.131+0000 7fc3eb7fe640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.131+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc3e408a810 con 0x7fc3f40c2190 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.131+0000 7fc3ebfff640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.132+0000 7fc3ebfff640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 0x7fc3d406d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4149bb0 tx=0x7fc3dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.135+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc3e4059490 con 0x7fc3f40c2190 2024-03-06T07:28:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.565+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f4068460 con 0x7fc3f40c2190 2024-03-06T07:28:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.565+0000 7fc3eb7fe640 1 -- 172.21.15.17:0/526306755 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7fc3e4058d90 con 0x7fc3f40c2190 2024-03-06T07:28:01.567 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:01.567 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:28:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 msgr2=0x7fc3d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 0x7fc3d406d990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4149bb0 tx=0x7fc3dc009040 comp rx=0 tx=0).stop 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 msgr2=0x7fc3f414cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400d5f0 tx=0x7fc3e400dab0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 shutdown_connections 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc3d406b4d0 0x7fc3d406d990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3f4152920 0x7fc3f414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3f40c2620 0x7fc3f4072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 --2- 172.21.15.17:0/526306755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3f40c2190 0x7fc3f414cbc0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 >> 172.21.15.17:0/526306755 conn(0x7fc3f40c4290 msgr2=0x7fc3f40c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 shutdown_connections 2024-03-06T07:28:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.568+0000 7fc3f9525640 1 -- 172.21.15.17:0/526306755 wait complete. 2024-03-06T07:28:01.581 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:01.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.685+0000 7f8f99993640 1 Processor -- start 2024-03-06T07:28:01.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 -- start start 2024-03-06T07:28:01.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 0x7f8f94158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f94156180 con 0x7f8f94159460 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f941587b0 con 0x7f8f940c8a00 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.686+0000 7f8f99993640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940c7760 con 0x7f8f94152920 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f927fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 0x7f8f94158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43240/0 (socket says 172.21.15.17:43240) 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 -- 172.21.15.17:0/3896303795 learned_addr learned my addr 172.21.15.17:0/3896303795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f927fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 0x7f8f94158f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50708/0 (socket says 172.21.15.17:50708) 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 -- 172.21.15.17:0/3896303795 >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 msgr2=0x7f8f94158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 --2- 172.21.15.17:0/3896303795 >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 0x7f8f94158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 -- 172.21.15.17:0/3896303795 >> v1:172.21.15.163:6789/0 conn(0x7f8f94159460 legacy=0x7f8f940c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f92ffd640 1 -- 172.21.15.17:0/3896303795 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940c7a40 con 0x7f8f94152920 2024-03-06T07:28:01.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.687+0000 7f8f91ffb640 1 -- 172.21.15.17:0/3896303795 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709527772 0 0) 0x7f8f84001110 con 0x7f8f94159460 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.688+0000 7f8f92ffd640 1 --2- 172.21.15.17:0/3896303795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009e50 tx=0x7f8f8c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5635f66008adb6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.688+0000 7f8f91ffb640 1 -- 172.21.15.17:0/3896303795 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c02ca70 con 0x7f8f94152920 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.688+0000 7f8f91ffb640 1 -- 172.21.15.17:0/3896303795 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c02cc10 con 0x7f8f94152920 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.688+0000 7f8f91ffb640 1 -- 172.21.15.17:0/3896303795 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0343f0 con 0x7f8f94152920 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 -- 172.21.15.17:0/3896303795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 msgr2=0x7f8f94155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 --2- 172.21.15.17:0/3896303795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009e50 tx=0x7f8f8c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 -- 172.21.15.17:0/3896303795 shutdown_connections 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 --2- 172.21.15.17:0/3896303795 >> v2:172.21.15.163:3300/0 conn(0x7f8f940c8a00 0x7f8f94158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 --2- 172.21.15.17:0/3896303795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f94152920 0x7f8f94155c40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 -- 172.21.15.17:0/3896303795 >> 172.21.15.17:0/3896303795 conn(0x7f8f940ac560 msgr2=0x7f8f940ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 -- 172.21.15.17:0/3896303795 shutdown_connections 2024-03-06T07:28:01.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.689+0000 7f8f99993640 1 -- 172.21.15.17:0/3896303795 wait complete. 2024-03-06T07:28:01.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.690+0000 7f8f99993640 1 Processor -- start 2024-03-06T07:28:01.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.690+0000 7f8f99993640 1 -- start start 2024-03-06T07:28:01.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f940c8a00 0x7f8f940c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f94152920 0x7f8f940c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9406d030 con 0x7f8f94153030 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94158a00 con 0x7f8f94152920 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f99993640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940c5a40 con 0x7f8f940c8a00 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f937fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32920/0 (socket says 172.21.15.17:32920) 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f937fe640 1 -- 172.21.15.17:0/2890219039 learned_addr learned my addr 172.21.15.17:0/2890219039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f92ffd640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f940c8a00 0x7f8f940c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.691+0000 7f8f927fc640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f94152920 0x7f8f940c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 -- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f940c8a00 msgr2=0x7f8f940c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f940c8a00 0x7f8f940c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 -- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f94152920 msgr2=0x7f8f940c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f94152920 0x7f8f940c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 -- 172.21.15.17:0/2890219039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9414ad70 con 0x7f8f94153030 2024-03-06T07:28:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f937fe640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c3f0 tx=0x7f8f8400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f84017070 con 0x7f8f94153030 2024-03-06T07:28:01.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f840035f0 con 0x7f8f94153030 2024-03-06T07:28:01.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f840039e0 con 0x7f8f94153030 2024-03-06T07:28:01.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f94153c50 con 0x7f8f94153030 2024-03-06T07:28:01.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.692+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9414d8b0 con 0x7f8f94153030 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.693+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94067910 con 0x7f8f94153030 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.694+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f8401a070 con 0x7f8f94153030 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.694+0000 7f8f7bfff640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 0x7f8f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.694+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f8f840897e0 con 0x7f8f94153030 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.694+0000 7f8f92ffd640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 0x7f8f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.695+0000 7f8f92ffd640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 0x7f8f8006d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c035000 tx=0x7f8f8c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:01.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.700+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f94067910 con 0x7f8f94153030 2024-03-06T07:28:01.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.947+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f9404f330 con 0x7f8f94153030 2024-03-06T07:28:01.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.948+0000 7f8f7bfff640 1 -- 172.21.15.17:0/2890219039 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17068 (secure 0 0 0) 0x7f8f94156180 con 0x7f8f94153030 2024-03-06T07:28:01.950 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:01.950 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":188,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:54.873995+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 msgr2=0x7f8f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 0x7f8f8006d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c035000 tx=0x7f8f8c030000 comp rx=0 tx=0).stop 2024-03-06T07:28:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 msgr2=0x7f8f941534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:01.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400c3f0 tx=0x7f8f8400c8b0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 shutdown_connections 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f8006b4d0 0x7f8f8006d990 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f94153030 0x7f8f941534e0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f94152920 0x7f8f940c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 --2- 172.21.15.17:0/2890219039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f940c8a00 0x7f8f940c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 >> 172.21.15.17:0/2890219039 conn(0x7f8f940ac560 msgr2=0x7f8f940ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 shutdown_connections 2024-03-06T07:28:01.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:01.950+0000 7f8f99993640 1 -- 172.21.15.17:0/2890219039 wait complete. 2024-03-06T07:28:01.966 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:28:01.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:02.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.067+0000 7f67d097a640 1 Processor -- start 2024-03-06T07:28:02.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.067+0000 7f67d097a640 1 -- start start 2024-03-06T07:28:02.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67d097a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 0x7f67cc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67d097a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 0x7f67cc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67d097a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67cc0c73d0 con 0x7f67cc14f360 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67d097a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67cc0c8dd0 con 0x7f67cc158b60 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67d097a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0be2e0 con 0x7f67cc0c88c0 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67ca59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f67cc14f360 0x7f67cc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40730/0 (socket says 172.21.15.17:40730) 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67ca59c640 1 -- 172.21.15.17:0/1528402600 learned_addr learned my addr 172.21.15.17:0/1528402600 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67cad9d640 1 --2- 172.21.15.17:0/1528402600 >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 0x7f67cc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.068+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/1528402600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 0x7f67cc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972267912 0 0) 0x7f67cc0c73d0 con 0x7f67cc14f360 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67b4003520 con 0x7f67cc14f360 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c9d9b640 1 -- 172.21.15.17:0/1528402600 >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 msgr2=0x7f67cc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/1528402600 >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 0x7f67cc158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1015146749 0 0) 0x7f67b4003520 con 0x7f67cc14f360 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c9d9b640 1 -- 172.21.15.17:0/1528402600 >> v1:172.21.15.17:6789/0 conn(0x7f67cc14f360 legacy=0x7f67cc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67c9d9b640 1 -- 172.21.15.17:0/1528402600 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc0c7670 con 0x7f67cc0c88c0 2024-03-06T07:28:02.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.069+0000 7f67cad9d640 1 --2- 172.21.15.17:0/1528402600 >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 0x7f67cc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:02.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.070+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/1528402600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 0x7f67cc155c10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67c00097a0 tx=0x7f67c002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e446101c5682b861 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.070+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0037070 con 0x7f67cc0c88c0 2024-03-06T07:28:02.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.070+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0030510 con 0x7f67cc0c88c0 2024-03-06T07:28:02.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.070+0000 7f67c959a640 1 -- 172.21.15.17:0/1528402600 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc001670 con 0x7f67cc0c88c0 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 -- 172.21.15.17:0/1528402600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 msgr2=0x7f67cc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 --2- 172.21.15.17:0/1528402600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 0x7f67cc155c10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67c00097a0 tx=0x7f67c002b710 comp rx=0 tx=0).stop 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 -- 172.21.15.17:0/1528402600 shutdown_connections 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 --2- 172.21.15.17:0/1528402600 >> v2:172.21.15.163:3300/0 conn(0x7f67cc158b60 0x7f67cc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 --2- 172.21.15.17:0/1528402600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc0c88c0 0x7f67cc155c10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 -- 172.21.15.17:0/1528402600 >> 172.21.15.17:0/1528402600 conn(0x7f67cc0ac570 msgr2=0x7f67cc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 -- 172.21.15.17:0/1528402600 shutdown_connections 2024-03-06T07:28:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.071+0000 7f67d097a640 1 -- 172.21.15.17:0/1528402600 wait complete. 2024-03-06T07:28:02.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.073+0000 7f67d097a640 1 Processor -- start 2024-03-06T07:28:02.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.073+0000 7f67d097a640 1 -- start start 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 0x7f67cc154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc158b60 0x7f67cc149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0c5a50 con 0x7f67cc14f360 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0c7c30 con 0x7f67cc0c88c0 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67d097a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67cc0c5bd0 con 0x7f67cc158b60 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32936/0 (socket says 172.21.15.17:32936) 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 -- 172.21.15.17:0/3088622612 learned_addr learned my addr 172.21.15.17:0/3088622612 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67ca59c640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 0x7f67cc154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67cad9d640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc158b60 0x7f67cc149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 -- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc158b60 msgr2=0x7f67cc149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc158b60 0x7f67cc149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 -- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 msgr2=0x7f67cc154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 0x7f67cc154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 -- 172.21.15.17:0/3088622612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc1ef1b0 con 0x7f67cc14f360 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67ca59c640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 0x7f67cc154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.074+0000 7f67c9d9b640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f67c002b750 tx=0x7f67c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.075+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0041020 con 0x7f67cc14f360 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.075+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c003c870 con 0x7f67cc14f360 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.075+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67cc158530 con 0x7f67cc14f360 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.075+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0037040 con 0x7f67cc14f360 2024-03-06T07:28:02.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.075+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67cc154d90 con 0x7f67cc14f360 2024-03-06T07:28:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.076+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67cc1eefc0 con 0x7f67cc14f360 2024-03-06T07:28:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.077+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67c0044020 con 0x7f67cc14f360 2024-03-06T07:28:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.077+0000 7f67baffd640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 0x7f67ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.077+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f67c00abe30 con 0x7f67cc14f360 2024-03-06T07:28:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.078+0000 7f67ca59c640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 0x7f67ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.078+0000 7f67ca59c640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 0x7f67ac06d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f67bc007ae0 tx=0x7f67bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.086+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f67c004b030 con 0x7f67cc14f360 2024-03-06T07:28:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.304+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67cc04f330 con 0x7f67cc14f360 2024-03-06T07:28:02.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.305+0000 7f67baffd640 1 -- 172.21.15.17:0/3088622612 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7f67c007aab0 con 0x7f67cc14f360 2024-03-06T07:28:02.307 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:02.307 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:28:02.307 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:28:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 msgr2=0x7f67ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 0x7f67ac06d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f67bc007ae0 tx=0x7f67bc009040 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 msgr2=0x7f67cc148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f67c002b750 tx=0x7f67c00046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 shutdown_connections 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f67ac06b4d0 0x7f67ac06d990 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67cc158b60 0x7f67cc149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67cc14f360 0x7f67cc148de0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 --2- 172.21.15.17:0/3088622612 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67cc0c88c0 0x7f67cc154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 >> 172.21.15.17:0/3088622612 conn(0x7f67cc0ac570 msgr2=0x7f67cc06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 shutdown_connections 2024-03-06T07:28:02.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.308+0000 7f67d097a640 1 -- 172.21.15.17:0/3088622612 wait complete. 2024-03-06T07:28:02.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.422+0000 7fdcdc99a640 1 Processor -- start 2024-03-06T07:28:02.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.422+0000 7fdcdc99a640 1 -- start start 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcdc99a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdcd806ed00 0x7fdcd8152940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcdc99a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcd80c7340 0x7fdcd815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcdc99a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd80733e0 con 0x7fdcd80729e0 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcdc99a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd814f9a0 con 0x7fdcd80c7340 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcdc99a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd806d3b0 con 0x7fdcd806ed00 2024-03-06T07:28:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcd6d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcd80c7340 0x7fdcd815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcd6d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcd80c7340 0x7fdcd815c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32950/0 (socket says 172.21.15.17:32950) 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcd6d9d640 1 -- 172.21.15.17:0/4127764502 learned_addr learned my addr 172.21.15.17:0/4127764502 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.423+0000 7fdcd5d9b640 1 --2- 172.21.15.17:0/4127764502 >> v2:172.21.15.163:3300/0 conn(0x7fdcd806ed00 0x7fdcd8152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd6d9d640 1 -- 172.21.15.17:0/4127764502 >> v2:172.21.15.163:3300/0 conn(0x7fdcd806ed00 msgr2=0x7fdcd8152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd6d9d640 1 --2- 172.21.15.17:0/4127764502 >> v2:172.21.15.163:3300/0 conn(0x7fdcd806ed00 0x7fdcd8152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd6d9d640 1 -- 172.21.15.17:0/4127764502 >> v1:172.21.15.163:6789/0 conn(0x7fdcd80729e0 legacy=0x7fdcd80c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd6d9d640 1 -- 172.21.15.17:0/4127764502 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8072f90 con 0x7fdcd80c7340 2024-03-06T07:28:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd6d9d640 1 --2- 172.21.15.17:0/4127764502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd80c7340 0x7fdcd815c760 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8070c60 tx=0x7fdcd002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814bc346a4d5d60c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd559a640 1 -- 172.21.15.17:0/4127764502 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd002ea90 con 0x7fdcd80c7340 2024-03-06T07:28:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.424+0000 7fdcd559a640 1 -- 172.21.15.17:0/4127764502 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd002ec30 con 0x7fdcd80c7340 2024-03-06T07:28:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcd559a640 1 -- 172.21.15.17:0/4127764502 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0009600 con 0x7fdcd80c7340 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 -- 172.21.15.17:0/4127764502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd80c7340 msgr2=0x7fdcd815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/4127764502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd80c7340 0x7fdcd815c760 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8070c60 tx=0x7fdcd002e0f0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 -- 172.21.15.17:0/4127764502 shutdown_connections 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/4127764502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd80c7340 0x7fdcd815c760 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/4127764502 >> v2:172.21.15.163:3300/0 conn(0x7fdcd806ed00 0x7fdcd8152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 -- 172.21.15.17:0/4127764502 >> 172.21.15.17:0/4127764502 conn(0x7fdcd80c7800 msgr2=0x7fdcd80c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 -- 172.21.15.17:0/4127764502 shutdown_connections 2024-03-06T07:28:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.425+0000 7fdcdc99a640 1 -- 172.21.15.17:0/4127764502 wait complete. 2024-03-06T07:28:02.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 Processor -- start 2024-03-06T07:28:02.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 -- start start 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd80c7340 0x7fdcd80c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcd8149830 0x7fdcd8151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80c1d10 con 0x7fdcd806ed00 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd806d690 con 0x7fdcd80c7340 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcdc99a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd80c1e90 con 0x7fdcd8149830 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcd659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcd659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32962/0 (socket says 172.21.15.17:32962) 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.427+0000 7fdcd659c640 1 -- 172.21.15.17:0/3079681172 learned_addr learned my addr 172.21.15.17:0/3079681172 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd6d9d640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcd8149830 0x7fdcd8151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd5d9b640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd80c7340 0x7fdcd80c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 -- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcd8149830 msgr2=0x7fdcd8151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcd8149830 0x7fdcd8151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 -- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd80c7340 msgr2=0x7fdcd80c09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd80c7340 0x7fdcd80c09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 -- 172.21.15.17:0/3079681172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80c6930 con 0x7fdcd806ed00 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcd659c640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c430 tx=0x7fdcc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80052d0 con 0x7fdcd806ed00 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd80c14c0 con 0x7fdcd806ed00 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc8005490 con 0x7fdcd806ed00 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8007260 con 0x7fdcd806ed00 2024-03-06T07:28:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.428+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd80c0480 con 0x7fdcd806ed00 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.429+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd80c0680 con 0x7fdcd806ed00 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.430+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcc8007400 con 0x7fdcd806ed00 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.431+0000 7fdcc6ffd640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.431+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fdcc80898f0 con 0x7fdcd806ed00 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.431+0000 7fdcd5d9b640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.432+0000 7fdcd5d9b640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 0x7fdcb806d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdcd814a560 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.435+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdcd80c0680 con 0x7fdcd806ed00 2024-03-06T07:28:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.742+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcd804f330 con 0x7fdcd806ed00 2024-03-06T07:28:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.743+0000 7fdcc6ffd640 1 -- 172.21.15.17:0/3079681172 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5018 (secure 0 0 0) 0x7fdcd80c6930 con 0x7fdcd806ed00 2024-03-06T07:28:02.745 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:02.745 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":182,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:27:26.886176+0000","modified":"2024-03-06T07:27:56.891501+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10140,"mds_1":9882},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9882":{"gid":9882,"name":"c","rank":1,"incarnation":181,"state":"up:active","state_seq":11,"addr":"172.21.15.17:6833/3930100029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3930100029},{"type":"v1","addr":"172.21.15.17:6833","nonce":3930100029}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10140":{"gid":10140,"name":"a","rank":0,"incarnation":179,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6835/2561439920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2561439920},{"type":"v1","addr":"172.21.15.17:6835","nonce":2561439920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10140,"qdb_cluster":[9882,10140]},"id":14}]} 2024-03-06T07:28:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 182 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 msgr2=0x7fdcb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 0x7fdcb806d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fdcd814a560 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 msgr2=0x7fdcd8149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800c430 tx=0x7fdcc800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 shutdown_connections 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdcb806b4d0 0x7fdcb806d990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdcd8149830 0x7fdcd8151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdcd80c7340 0x7fdcd80c09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 --2- 172.21.15.17:0/3079681172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcd806ed00 0x7fdcd8149d60 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 >> 172.21.15.17:0/3079681172 conn(0x7fdcd80c7800 msgr2=0x7fdcd80c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 shutdown_connections 2024-03-06T07:28:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.745+0000 7fdcdc99a640 1 -- 172.21.15.17:0/3079681172 wait complete. 2024-03-06T07:28:02.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.859+0000 7fc26fa39640 1 Processor -- start 2024-03-06T07:28:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.859+0000 7fc26fa39640 1 -- start start 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26fa39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 0x7fc26806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26fa39640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc26806b3a0 0x7fc268152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26fa39640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26806b800 con 0x7fc2681534f0 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26fa39640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc268153980 con 0x7fc26806b3a0 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26fa39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc268150a20 con 0x7fc26806afc0 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26cfd4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc26806b3a0 0x7fc268152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26dfd6640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc2681534f0 0x7fc2680c5690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40746/0 (socket says 172.21.15.17:40746) 2024-03-06T07:28:02.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26dfd6640 1 -- 172.21.15.17:0/588475404 learned_addr learned my addr 172.21.15.17:0/588475404 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26d7d5640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 0x7fc26806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26cfd4640 1 -- 172.21.15.17:0/588475404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 msgr2=0x7fc26806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26cfd4640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 0x7fc26806f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26cfd4640 1 -- 172.21.15.17:0/588475404 >> v1:172.21.15.17:6789/0 conn(0x7fc2681534f0 legacy=0x7fc2680c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc25ffff640 1 -- 172.21.15.17:0/588475404 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862376314 0 0) 0x7fc26806b800 con 0x7fc2681534f0 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26cfd4640 1 -- 172.21.15.17:0/588475404 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680a70f0 con 0x7fc26806b3a0 2024-03-06T07:28:02.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.860+0000 7fc26d7d5640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 0x7fc26806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26cfd4640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc268152fb0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fc2640097a0 tx=0x7fc26402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e55f1ee1fabf5b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc25ffff640 1 -- 172.21.15.17:0/588475404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264038070 con 0x7fc26806b3a0 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc25ffff640 1 -- 172.21.15.17:0/588475404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264031700 con 0x7fc26806b3a0 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc25ffff640 1 -- 172.21.15.17:0/588475404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264031ad0 con 0x7fc26806b3a0 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 -- 172.21.15.17:0/588475404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 msgr2=0x7fc268152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc268152fb0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fc2640097a0 tx=0x7fc26402b7a0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 -- 172.21.15.17:0/588475404 shutdown_connections 2024-03-06T07:28:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc268152fb0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 --2- 172.21.15.17:0/588475404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26806afc0 0x7fc26806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.862+0000 7fc26fa39640 1 -- 172.21.15.17:0/588475404 >> 172.21.15.17:0/588475404 conn(0x7fc2680c4290 msgr2=0x7fc2680c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.863+0000 7fc26fa39640 1 -- 172.21.15.17:0/588475404 shutdown_connections 2024-03-06T07:28:02.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.863+0000 7fc26fa39640 1 -- 172.21.15.17:0/588475404 wait complete. 2024-03-06T07:28:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.864+0000 7fc26fa39640 1 Processor -- start 2024-03-06T07:28:02.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.864+0000 7fc26fa39640 1 -- start start 2024-03-06T07:28:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26806afc0 0x7fc2681529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2681534f0 0x7fc26814ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268156ad0 con 0x7fc26806b3a0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2681565a0 con 0x7fc26806afc0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26fa39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc268156db0 con 0x7fc2681534f0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26cfd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.865+0000 7fc26cfd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32980/0 (socket says 172.21.15.17:32980) 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 -- 172.21.15.17:0/4196258417 learned_addr learned my addr 172.21.15.17:0/4196258417 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26d7d5640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26806afc0 0x7fc2681529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 -- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2681534f0 msgr2=0x7fc26814ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2681534f0 0x7fc26814ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 -- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26806afc0 msgr2=0x7fc2681529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26806afc0 0x7fc2681529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 -- 172.21.15.17:0/4196258417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2681e0730 con 0x7fc26806b3a0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26cfd4640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098c0 tx=0x7fc2640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264038040 con 0x7fc26806b3a0 2024-03-06T07:28:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264031500 con 0x7fc26806b3a0 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2681ec6e0 con 0x7fc26806b3a0 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.866+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264033bd0 con 0x7fc26806b3a0 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.867+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2681ecca0 con 0x7fc26806b3a0 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.868+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26804f330 con 0x7fc26806b3a0 2024-03-06T07:28:02.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.868+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc264045020 con 0x7fc26806b3a0 2024-03-06T07:28:02.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.869+0000 7fc25dffb640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 0x7fc24c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:02.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.869+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc2640ab530 con 0x7fc26806b3a0 2024-03-06T07:28:02.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.869+0000 7fc26d7d5640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 0x7fc24c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:02.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.870+0000 7fc26d7d5640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 0x7fc24c06d990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:02.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:02.877+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc2681e0730 con 0x7fc26806b3a0 2024-03-06T07:28:03.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.060+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc268068460 con 0x7fc26806b3a0 2024-03-06T07:28:03.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.061+0000 7fc25dffb640 1 -- 172.21.15.17:0/4196258417 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+17068 (secure 0 0 0) 0x7fc264030070 con 0x7fc26806b3a0 2024-03-06T07:28:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:03.063 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":188,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:27:54.873995+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:27:25.147974+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-03-06T07:27:25.852438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.10","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 msgr2=0x7fc24c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 0x7fc24c06d990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258008040 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 msgr2=0x7fc26814b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098c0 tx=0x7fc2640046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 shutdown_connections 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc24c06b4d0 0x7fc24c06d990 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2681534f0 0x7fc26814ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc26806b3a0 0x7fc26814b530 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 --2- 172.21.15.17:0/4196258417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26806afc0 0x7fc2681529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 >> 172.21.15.17:0/4196258417 conn(0x7fc2680c4290 msgr2=0x7fc2680c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 shutdown_connections 2024-03-06T07:28:03.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.064+0000 7fc26fa39640 1 -- 172.21.15.17:0/4196258417 wait complete. 2024-03-06T07:28:03.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:28:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.177+0000 7f481f05c640 1 Processor -- start 2024-03-06T07:28:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.177+0000 7f481f05c640 1 -- start start 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481f05c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48180c8950 0x7f48180c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481f05c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4818155130 con 0x7f48181558d0 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481f05c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180ad480 con 0x7f48180c69f0 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481f05c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4818155440 con 0x7f48180c8950 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f4817fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48180c8950 0x7f48180c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481d5f9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f48181558d0 0x7f481815fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40758/0 (socket says 172.21.15.17:40758) 2024-03-06T07:28:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f4817fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f48180c8950 0x7f48180c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32982/0 (socket says 172.21.15.17:32982) 2024-03-06T07:28:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f481d5f9640 1 -- 172.21.15.17:0/2010777605 learned_addr learned my addr 172.21.15.17:0/2010777605 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f4817fff640 1 -- 172.21.15.17:0/2010777605 >> v1:172.21.15.163:6789/0 conn(0x7f48180c69f0 legacy=0x7f48180acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f4817fff640 1 -- 172.21.15.17:0/2010777605 >> v1:172.21.15.17:6789/0 conn(0x7f48181558d0 legacy=0x7f481815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f48177fe640 1 -- 172.21.15.17:0/2010777605 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92323987 0 0) 0x7f4810001490 con 0x7f48181558d0 2024-03-06T07:28:03.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.178+0000 7f4817fff640 1 -- 172.21.15.17:0/2010777605 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180c5d50 con 0x7f48180c8950 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.179+0000 7f4817fff640 1 --2- 172.21.15.17:0/2010777605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f48180c6dd0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f48040097a0 tx=0x7f480402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05b87ee7109fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.179+0000 7f48177fe640 1 -- 172.21.15.17:0/2010777605 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804035020 con 0x7f48180c8950 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.179+0000 7f48177fe640 1 -- 172.21.15.17:0/2010777605 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4804031830 con 0x7f48180c8950 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.179+0000 7f48177fe640 1 -- 172.21.15.17:0/2010777605 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804031c00 con 0x7f48180c8950 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 -- 172.21.15.17:0/2010777605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 msgr2=0x7f48180c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 --2- 172.21.15.17:0/2010777605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f48180c6dd0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f48040097a0 tx=0x7f480402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 -- 172.21.15.17:0/2010777605 shutdown_connections 2024-03-06T07:28:03.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 --2- 172.21.15.17:0/2010777605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f48180c6dd0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 -- 172.21.15.17:0/2010777605 >> 172.21.15.17:0/2010777605 conn(0x7f48180cae70 msgr2=0x7f48180cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 -- 172.21.15.17:0/2010777605 shutdown_connections 2024-03-06T07:28:03.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.180+0000 7f481f05c640 1 -- 172.21.15.17:0/2010777605 wait complete. 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.181+0000 7f481f05c640 1 Processor -- start 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.181+0000 7f481f05c640 1 -- start start 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48180c69f0 0x7f481814ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 0x7f4818156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4818152a20 con 0x7f48180c8950 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180c6310 con 0x7f48181558d0 2024-03-06T07:28:03.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481f05c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4818152ba0 con 0x7f48180c69f0 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32988/0 (socket says 172.21.15.17:32988) 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 -- 172.21.15.17:0/3767765268 learned_addr learned my addr 172.21.15.17:0/3767765268 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481d5f9640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 0x7f4818156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481cdf8640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48180c69f0 0x7f481814ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 -- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48180c69f0 msgr2=0x7f481814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48180c69f0 0x7f481814ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 -- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 msgr2=0x7f4818156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 0x7f4818156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 -- 172.21.15.17:0/3767765268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180c47f0 con 0x7f48180c8950 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f481d5f9640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 0x7f4818156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.182+0000 7f4817fff640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f48040098c0 tx=0x7f48040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.183+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804041070 con 0x7f48180c8950 2024-03-06T07:28:03.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.183+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480403aa30 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.183+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48180c4ad0 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.183+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804035040 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.183+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48181ef200 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.184+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481804f330 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.184+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f480404c060 con 0x7f48180c8950 2024-03-06T07:28:03.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.185+0000 7f48157fa640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.185+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f48040abe70 con 0x7f48180c8950 2024-03-06T07:28:03.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.185+0000 7f481cdf8640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.185+0000 7f481cdf8640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f60 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.189+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f480407ab80 con 0x7f48180c8950 2024-03-06T07:28:03.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.409+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f48180ca650 con 0x7f48180c8950 2024-03-06T07:28:03.410 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-06T07:28:03.409+0000 7f4453093640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-06T07:28:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.411+0000 7f48157fa640 1 -- 172.21.15.17:0/3767765268 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v183) v1 ==== 161+0+0 (secure 0 0 0) 0x7f480407a480 con 0x7f48180c8950 2024-03-06T07:28:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 msgr2=0x7f47fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f60 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 msgr2=0x7f4818157d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f48040098c0 tx=0x7f48040046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 shutdown_connections 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48181558d0 0x7f4818156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48180c8950 0x7f4818157d20 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 --2- 172.21.15.17:0/3767765268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48180c69f0 0x7f481814ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 >> 172.21.15.17:0/3767765268 conn(0x7f48180cae70 msgr2=0x7f48180c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 shutdown_connections 2024-03-06T07:28:03.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.414+0000 7f481f05c640 1 -- 172.21.15.17:0/3767765268 wait complete. 2024-03-06T07:28:03.417 INFO:tasks.ceph.mds.a.smithi017.stderr: -182> 2024-03-06T07:27:58.880+0000 7f4fe65e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:03.417 INFO:tasks.ceph.mds.a.smithi017.stderr: -180> 2024-03-06T07:27:58.880+0000 7f4fe65e3640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-06T07:28:03.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:28:03.441 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:28:03.445 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:28:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.534+0000 7ff5104df640 1 Processor -- start 2024-03-06T07:28:03.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.534+0000 7ff5104df640 1 -- start start 2024-03-06T07:28:03.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.535+0000 7ff5104df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5080c26c0 0x7ff508155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.535+0000 7ff5104df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff50806bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.535+0000 7ff5104df640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50806c110 con 0x7ff5080c0470 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.535+0000 7ff5104df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50806b740 con 0x7ff5080c26c0 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.535+0000 7ff5104df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080670c0 con 0x7ff5080c2aa0 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5080c26c0 0x7ff508155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50da7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff50806bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50ea7c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff5080c0470 0x7ff508152350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:49322/0 (socket says 172.21.15.17:49322) 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50ea7c640 1 -- 172.21.15.17:0/60038955 learned_addr learned my addr 172.21.15.17:0/60038955 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5080c26c0 0x7ff508155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:50814/0 (socket says 172.21.15.17:50814) 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 -- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 msgr2=0x7ff50806bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 --2- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff50806bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 -- 172.21.15.17:0/60038955 >> v1:172.21.15.163:6789/0 conn(0x7ff5080c0470 legacy=0x7ff508152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50e27b640 1 -- 172.21.15.17:0/60038955 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50806b0f0 con 0x7ff5080c26c0 2024-03-06T07:28:03.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.536+0000 7ff50d279640 1 -- 172.21.15.17:0/60038955 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420713313 0 0) 0x7ff50806c110 con 0x7ff5080c0470 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff50e27b640 1 --2- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c26c0 0x7ff508155bd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097a0 tx=0x7ff50002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8249d73d72a578c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff50d279640 1 -- 172.21.15.17:0/60038955 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500038070 con 0x7ff5080c26c0 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff50d279640 1 -- 172.21.15.17:0/60038955 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500031700 con 0x7ff5080c26c0 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff50d279640 1 -- 172.21.15.17:0/60038955 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500031ad0 con 0x7ff5080c26c0 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff5104df640 1 -- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c26c0 msgr2=0x7ff508155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.537+0000 7ff5104df640 1 --2- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c26c0 0x7ff508155bd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097a0 tx=0x7ff50002b7a0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 -- 172.21.15.17:0/60038955 shutdown_connections 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 --2- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff50806bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 --2- 172.21.15.17:0/60038955 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c26c0 0x7ff508155bd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 -- 172.21.15.17:0/60038955 >> 172.21.15.17:0/60038955 conn(0x7ff5080c3b40 msgr2=0x7ff5080c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:03.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 -- 172.21.15.17:0/60038955 shutdown_connections 2024-03-06T07:28:03.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.538+0000 7ff5104df640 1 -- 172.21.15.17:0/60038955 wait complete. 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 Processor -- start 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 -- start start 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c0470 0x7ff5080c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff5081559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080c1f30 con 0x7ff5080c26c0 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080c0900 con 0x7ff5080c0470 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff5104df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080c20b0 con 0x7ff5080c2aa0 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff50da7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff50da7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33032/0 (socket says 172.21.15.17:33032) 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.539+0000 7ff50da7a640 1 -- 172.21.15.17:0/4177293930 learned_addr learned my addr 172.21.15.17:0/4177293930 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50ea7c640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff5081559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 -- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 msgr2=0x7ff5081559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff5081559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 -- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c0470 msgr2=0x7ff5080c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50e27b640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c0470 0x7ff5080c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c0470 0x7ff5080c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 -- 172.21.15.17:0/4177293930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50814cab0 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff50da7a640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e870 tx=0x7ff4fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc017070 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00c940 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00cd30 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50814cd90 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.540+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50814d350 con 0x7ff5080c26c0 2024-03-06T07:28:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.541+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50804f330 con 0x7ff5080c26c0 2024-03-06T07:28:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.542+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4fc01a070 con 0x7ff5080c26c0 2024-03-06T07:28:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.542+0000 7ff506ffd640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.542+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff4fc08a6a0 con 0x7ff5080c26c0 2024-03-06T07:28:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.543+0000 7ff50e27b640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:03.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.543+0000 7ff50e27b640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff500035000 tx=0x7ff500008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.546+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4fc059300 con 0x7ff5080c26c0 2024-03-06T07:28:03.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:03.767+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5081506d0 con 0x7ff5080c26c0 2024-03-06T07:28:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.416+0000 7ff506ffd640 1 -- 172.21.15.17:0/4177293930 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v184) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4fc058c00 con 0x7ff5080c26c0 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 msgr2=0x7ff4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff500035000 tx=0x7ff500008030 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 msgr2=0x7ff508155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e870 tx=0x7ff4fc00ed30 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 shutdown_connections 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4f406b4d0 0x7ff4f406d990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080c2aa0 0x7ff5081559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5080c26c0 0x7ff508155490 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 --2- 172.21.15.17:0/4177293930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080c0470 0x7ff5080c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 >> 172.21.15.17:0/4177293930 conn(0x7ff5080c3b40 msgr2=0x7ff5081508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 shutdown_connections 2024-03-06T07:28:04.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.419+0000 7ff5104df640 1 -- 172.21.15.17:0/4177293930 wait complete. 2024-03-06T07:28:04.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:28:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 Processor -- start 2024-03-06T07:28:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 -- start start 2024-03-06T07:28:04.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd32c0c6780 0x7fd32c072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c0a7d70 con 0x7fd32c06fe20 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c0a8050 con 0x7fd32c072a50 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.589+0000 7fd331a39640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0a8330 con 0x7fd32c0c6780 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd32affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd32c06fe20 0x7fd32c155c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40772/0 (socket says 172.21.15.17:40772) 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd32affd640 1 -- 172.21.15.17:0/3750818919 learned_addr learned my addr 172.21.15.17:0/3750818919 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd32a7fc640 1 --2- 172.21.15.17:0/3750818919 >> v2:172.21.15.163:3300/0 conn(0x7fd32c0c6780 0x7fd32c072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001016418 0 0) 0x7fd32c0a7d70 con 0x7fd32c06fe20 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd314003520 con 0x7fd32c06fe20 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2491081167 0 0) 0x7fd314003520 con 0x7fd32c06fe20 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 >> v2:172.21.15.163:3300/0 conn(0x7fd32c0c6780 msgr2=0x7fd32c072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 --2- 172.21.15.17:0/3750818919 >> v2:172.21.15.163:3300/0 conn(0x7fd32c0c6780 0x7fd32c072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 >> v1:172.21.15.163:6789/0 conn(0x7fd32c072a50 legacy=0x7fd32c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0c3320 con 0x7fd32c06fe20 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd31c001670 con 0x7fd32c06fe20 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd32a7fc640 1 --2- 172.21.15.17:0/3750818919 >> v2:172.21.15.163:3300/0 conn(0x7fd32c0c6780 0x7fd32c072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 >> v1:172.21.15.17:6789/0 conn(0x7fd32c06fe20 legacy=0x7fd32c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd314004800 0x7fd314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3140051c0 0x7fd31400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0a8330 con 0x7fd3140039b0 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31400c160 con 0x7fd3140051c0 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31400c360 con 0x7fd314004800 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.590+0000 7fd32affd640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32a7fc640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd314004800 0x7fd314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32b7fe640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3140051c0 0x7fd31400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 -- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd314004800 msgr2=0x7fd314004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd314004800 0x7fd314004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 -- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3140051c0 msgr2=0x7fd31400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd3140051c0 0x7fd31400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32b7fe640 1 -- 172.21.15.17:0/3750818919 reap_dead start 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 -- 172.21.15.17:0/3750818919 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd314003520 con 0x7fd3140039b0 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.591+0000 7fd32affd640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007fa0 tx=0x7fd314005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b3d1f1126bbded server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c007cf0 con 0x7fd3140039b0 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31c005af0 con 0x7fd3140039b0 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd329ffb640 1 -- 172.21.15.17:0/3750818919 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31c007070 con 0x7fd3140039b0 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 -- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 msgr2=0x7fd314003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fd31c007fa0 tx=0x7fd314005830 comp rx=0 tx=0).stop 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 -- 172.21.15.17:0/3750818919 shutdown_connections 2024-03-06T07:28:04.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 --2- 172.21.15.17:0/3750818919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3140039b0 0x7fd314003dd0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 -- 172.21.15.17:0/3750818919 >> 172.21.15.17:0/3750818919 conn(0x7fd32c0aa630 msgr2=0x7fd32c0aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 -- 172.21.15.17:0/3750818919 shutdown_connections 2024-03-06T07:28:04.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.592+0000 7fd331a39640 1 -- 172.21.15.17:0/3750818919 wait complete. 2024-03-06T07:28:04.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.593+0000 7fd331a39640 1 Processor -- start 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.593+0000 7fd331a39640 1 -- start start 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3140039b0 0x7fd32c148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd32c152a90 0x7fd32c1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c072510 con 0x7fd32c0ac210 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c070200 con 0x7fd32c152a90 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd331a39640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c072690 con 0x7fd3140039b0 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd32b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd32b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33062/0 (socket says 172.21.15.17:33062) 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd32b7fe640 1 -- 172.21.15.17:0/1495799851 learned_addr learned my addr 172.21.15.17:0/1495799851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd32a7fc640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd32c152a90 0x7fd32c1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.594+0000 7fd32affd640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3140039b0 0x7fd32c148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 -- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3140039b0 msgr2=0x7fd32c148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3140039b0 0x7fd32c148ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 -- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd32c152a90 msgr2=0x7fd32c1490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd32c152a90 0x7fd32c1490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 -- 172.21.15.17:0/1495799851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c0ae3f0 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd32b7fe640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0c6b60 tx=0x7fd32400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324019070 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd324006d60 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c0ab970 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324003b90 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.595+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0a96a0 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.596+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32c04f330 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.597+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32401c070 con 0x7fd32c0ac210 2024-03-06T07:28:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.597+0000 7fd313fff640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 0x7fd31806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.597+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd324089750 con 0x7fd32c0ac210 2024-03-06T07:28:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.597+0000 7fd32affd640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 0x7fd31806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.598+0000 7fd32affd640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 0x7fd31806d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd31c03c000 tx=0x7fd31c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.601+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd324057c40 con 0x7fd32c0ac210 2024-03-06T07:28:04.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:04.785+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd32c0c5160 con 0x7fd32c0ac210 2024-03-06T07:28:05.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.418+0000 7fd313fff640 1 -- 172.21.15.17:0/1495799851 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v190) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd32405bce0 con 0x7fd32c0ac210 2024-03-06T07:28:05.420 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:28:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 msgr2=0x7fd31806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 0x7fd31806d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd31c03c000 tx=0x7fd31c009030 comp rx=0 tx=0).stop 2024-03-06T07:28:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 msgr2=0x7fd32c0ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0c6b60 tx=0x7fd32400e8a0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 shutdown_connections 2024-03-06T07:28:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd31806b500 0x7fd31806d9c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd32c0ac210 0x7fd32c0ac690 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd32c152a90 0x7fd32c1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 --2- 172.21.15.17:0/1495799851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd3140039b0 0x7fd32c148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 >> 172.21.15.17:0/1495799851 conn(0x7fd32c0aa630 msgr2=0x7fd32c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 shutdown_connections 2024-03-06T07:28:05.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.421+0000 7fd331a39640 1 -- 172.21.15.17:0/1495799851 wait complete. 2024-03-06T07:28:05.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:28:05.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.547+0000 7efedac5c640 1 Processor -- start 2024-03-06T07:28:05.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.547+0000 7efedac5c640 1 -- start start 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.547+0000 7efedac5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efed4072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efedac5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40a7d70 con 0x7efed4072a50 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efedac5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40a8050 con 0x7efed406fe20 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efedac5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40a8330 con 0x7efed40c6780 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed91f9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efed4072a50 0x7efed4158780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40786/0 (socket says 172.21.15.17:40786) 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed91f9640 1 -- 172.21.15.17:0/2209436953 learned_addr learned my addr 172.21.15.17:0/2209436953 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:05.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417229798 0 0) 0x7efed40a7d70 con 0x7efed4072a50 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec4003520 con 0x7efed4072a50 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed3fff640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efed4072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2990114853 0 0) 0x7efec4003520 con 0x7efed4072a50 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 msgr2=0x7efed4072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efed4072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 >> v1:172.21.15.163:6789/0 conn(0x7efed406fe20 legacy=0x7efed4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40c3320 con 0x7efed4072a50 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7efecc0019f0 con 0x7efed4072a50 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 >> v1:172.21.15.17:6789/0 conn(0x7efed4072a50 legacy=0x7efed4158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec40039e0 0x7efec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec4004d40 0x7efec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40a8330 con 0x7efec4004d40 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec400bb20 con 0x7efec40039e0 2024-03-06T07:28:05.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.548+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec400bd20 con 0x7efed40c6780 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec4004d40 0x7efec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed89f8640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec40039e0 0x7efec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed3fff640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 -- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 msgr2=0x7efec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed40c6780 0x7efec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 -- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec40039e0 msgr2=0x7efec4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec40039e0 0x7efec4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.549+0000 7efed91f9640 1 -- 172.21.15.17:0/2209436953 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4003520 con 0x7efec4004d40 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efed91f9640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec4004d40 0x7efec40051c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7efecc00a840 tx=0x7efec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7865fd7cfe70c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00a460 con 0x7efec4004d40 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc034650 con 0x7efec4004d40 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efed37fe640 1 -- 172.21.15.17:0/2209436953 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc034a40 con 0x7efec4004d40 2024-03-06T07:28:05.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 -- 172.21.15.17:0/2209436953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec4004d40 msgr2=0x7efec40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 --2- 172.21.15.17:0/2209436953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec4004d40 0x7efec40051c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7efecc00a840 tx=0x7efec4005830 comp rx=0 tx=0).stop 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efed91f9640 1 -- 172.21.15.17:0/2209436953 reap_dead start 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 -- 172.21.15.17:0/2209436953 shutdown_connections 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 -- 172.21.15.17:0/2209436953 >> 172.21.15.17:0/2209436953 conn(0x7efed40aa630 msgr2=0x7efed40aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 -- 172.21.15.17:0/2209436953 shutdown_connections 2024-03-06T07:28:05.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.550+0000 7efedac5c640 1 -- 172.21.15.17:0/2209436953 wait complete. 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.551+0000 7efedac5c640 1 Processor -- start 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.551+0000 7efedac5c640 1 -- start start 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4004d40 0x7efed40a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed414b230 0x7efed414a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed4072510 con 0x7efed40ae9d0 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed4070200 con 0x7efec4004d40 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efedac5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed4072690 con 0x7efed414b230 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efed91f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efed91f9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33080/0 (socket says 172.21.15.17:33080) 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efed91f9640 1 -- 172.21.15.17:0/874235557 learned_addr learned my addr 172.21.15.17:0/874235557 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:05.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efed3fff640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed414b230 0x7efed414a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.552+0000 7efed89f8640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4004d40 0x7efed40a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 -- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed414b230 msgr2=0x7efed414a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed414b230 0x7efed414a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 -- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4004d40 msgr2=0x7efed40a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4004d40 0x7efed40a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 -- 172.21.15.17:0/874235557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4158660 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed91f9640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7efecc00a840 tx=0x7efed4155190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc045070 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc03a040 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed40ae340 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc001fb0 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.553+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed4158900 con 0x7efed40ae9d0 2024-03-06T07:28:05.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.554+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed404f330 con 0x7efed40ae9d0 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.555+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efecc048070 con 0x7efed40ae9d0 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.555+0000 7efed17fa640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 0x7efeb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.555+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5835+0+0 (secure 0 0 0) 0x7efecc0abd80 con 0x7efed40ae9d0 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.555+0000 7efed89f8640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 0x7efeb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.556+0000 7efed89f8640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 0x7efeb806d9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efec80017e0 tx=0x7efec8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:05.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.559+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efecc07a5c0 con 0x7efed40ae9d0 2024-03-06T07:28:05.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:05.739+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efed40c5160 con 0x7efed40ae9d0 2024-03-06T07:28:06.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.422+0000 7efed17fa640 1 -- 172.21.15.17:0/874235557 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v191) v1 ==== 169+0+0 (secure 0 0 0) 0x7efecc07e660 con 0x7efed40ae9d0 2024-03-06T07:28:06.425 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:28:06.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 msgr2=0x7efeb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 0x7efeb806d9c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efec80017e0 tx=0x7efec8009040 comp rx=0 tx=0).stop 2024-03-06T07:28:06.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 msgr2=0x7efed414ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7efecc00a840 tx=0x7efed4155190 comp rx=0 tx=0).stop 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 shutdown_connections 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efeb806b500 0x7efeb806d9c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efed40ae9d0 0x7efed414ab40 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed414b230 0x7efed414a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 --2- 172.21.15.17:0/874235557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efec4004d40 0x7efed40a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.426+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 >> 172.21.15.17:0/874235557 conn(0x7efed40aa630 msgr2=0x7efed40c1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.427+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 shutdown_connections 2024-03-06T07:28:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.427+0000 7efedac5c640 1 -- 172.21.15.17:0/874235557 wait complete. 2024-03-06T07:28:06.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:28:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.550+0000 7fb5d3feb640 1 Processor -- start 2024-03-06T07:28:06.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.550+0000 7fb5d3feb640 1 -- start start 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d3feb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1527f0 0x7fb5cc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d3feb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 0x7fb5cc155920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d3feb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc1596f0 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d3feb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0c9130 con 0x7fb5cc1527f0 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d3feb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc159260 con 0x7fb5cc0c58b0 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1d87640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb5cc1527f0 0x7fb5cc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1586640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb5cc0c88f0 0x7fb5cc155210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40802/0 (socket says 172.21.15.17:40802) 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1586640 1 -- 172.21.15.17:0/3046301473 learned_addr learned my addr 172.21.15.17:0/3046301473 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d2588640 1 --2- 172.21.15.17:0/3046301473 >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 0x7fb5cc155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1d87640 1 -- 172.21.15.17:0/3046301473 >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 msgr2=0x7fb5cc155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3046301473 >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 0x7fb5cc155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1d87640 1 -- 172.21.15.17:0/3046301473 >> v1:172.21.15.17:6789/0 conn(0x7fb5cc0c88f0 legacy=0x7fb5cc155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d1d87640 1 -- 172.21.15.17:0/3046301473 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0c82a0 con 0x7fb5cc1527f0 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d0d85640 1 -- 172.21.15.17:0/3046301473 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524304819 0 0) 0x7fb5cc1596f0 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.551+0000 7fb5d2588640 1 --2- 172.21.15.17:0/3046301473 >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 0x7fb5cc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:06.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.552+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3046301473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc0c5370 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0c4cf0 tx=0x7fb5c802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d4cbf67fe5721e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.552+0000 7fb5d0d85640 1 -- 172.21.15.17:0/3046301473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8038070 con 0x7fb5cc1527f0 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.552+0000 7fb5d0d85640 1 -- 172.21.15.17:0/3046301473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8031710 con 0x7fb5cc1527f0 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.552+0000 7fb5d0d85640 1 -- 172.21.15.17:0/3046301473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8031ae0 con 0x7fb5cc1527f0 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3046301473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1527f0 msgr2=0x7fb5cc0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3046301473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc0c5370 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0c4cf0 tx=0x7fb5c802b7b0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3046301473 shutdown_connections 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3046301473 >> v2:172.21.15.163:3300/0 conn(0x7fb5cc0c58b0 0x7fb5cc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3046301473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc0c5370 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3046301473 >> 172.21.15.17:0/3046301473 conn(0x7fb5cc0ac580 msgr2=0x7fb5cc0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3046301473 shutdown_connections 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.553+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3046301473 wait complete. 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 Processor -- start 2024-03-06T07:28:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 -- start start 2024-03-06T07:28:06.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 0x7fb5cc1460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc158c10 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.554+0000 7fb5d3feb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0c6b90 con 0x7fb5cc1527f0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d3feb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc158ef0 con 0x7fb5cc0c58b0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33098/0 (socket says 172.21.15.17:33098) 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 -- 172.21.15.17:0/3072823200 learned_addr learned my addr 172.21.15.17:0/3072823200 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 0x7fb5cc1460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d2588640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 -- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 msgr2=0x7fb5cc1460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 0x7fb5cc1460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 -- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 msgr2=0x7fb5cc14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 -- 172.21.15.17:0/3072823200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc0c2060 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 0x7fb5cc1460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d2588640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d1586640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c430 tx=0x7fb5bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0074e0 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0047e0 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc150690 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc004bd0 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.555+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc1eaf60 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.556+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc04f330 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.557+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5bc011070 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.557+0000 7fb5c27fc640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.557+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb5bc089d10 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.557+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.558+0000 7fb5d1d87640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8035000 tx=0x7fb5c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.561+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5bc058d40 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.743+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5cc068f00 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.746+0000 7fb5c27fc640 1 -- 172.21.15.17:0/3072823200 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5bc058640 con 0x7fb5cc0c88f0 2024-03-06T07:28:06.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 msgr2=0x7fb5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 0x7fb5b006d990 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8035000 tx=0x7fb5c8008030 comp rx=0 tx=0).stop 2024-03-06T07:28:06.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 msgr2=0x7fb5cc14d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c430 tx=0x7fb5bc00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 shutdown_connections 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb5b006b4d0 0x7fb5b006d990 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5cc1527f0 0x7fb5cc14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5cc0c88f0 0x7fb5cc14d860 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 --2- 172.21.15.17:0/3072823200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5cc0c58b0 0x7fb5cc1460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 >> 172.21.15.17:0/3072823200 conn(0x7fb5cc0ac580 msgr2=0x7fb5cc14f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 shutdown_connections 2024-03-06T07:28:06.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.748+0000 7fb5d3feb640 1 -- 172.21.15.17:0/3072823200 wait complete. 2024-03-06T07:28:06.761 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow' 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.860+0000 7ff4d7bc9640 1 Processor -- start 2024-03-06T07:28:06.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.860+0000 7ff4d7bc9640 1 -- start start 2024-03-06T07:28:06.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d7bc9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4d01527b0 0x7ff4d0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d7bc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d7bc9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d00c7c60 con 0x7ff4d0155b20 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d7bc9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d0159640 con 0x7ff4d01527b0 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d7bc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d00c8df0 con 0x7ff4d00c7180 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d5965640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff4d01527b0 0x7ff4d0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d5164640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d5164640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43440/0 (socket says 172.21.15.17:43440) 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.861+0000 7ff4d5164640 1 -- 172.21.15.17:0/85915916 learned_addr learned my addr 172.21.15.17:0/85915916 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d5965640 1 -- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 msgr2=0x7ff4d00c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d5965640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d5965640 1 -- 172.21.15.17:0/85915916 >> v1:172.21.15.163:6789/0 conn(0x7ff4d0155b20 legacy=0x7ff4d00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d5965640 1 -- 172.21.15.17:0/85915916 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d00c8060 con 0x7ff4d01527b0 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d4963640 1 -- 172.21.15.17:0/85915916 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655293952 0 0) 0x7ff4c0001490 con 0x7ff4d0155b20 2024-03-06T07:28:06.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.862+0000 7ff4d5164640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d5965640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d01527b0 0x7ff4d0159100 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80097a0 tx=0x7ff4c802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943875573c601560 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d4963640 1 -- 172.21.15.17:0/85915916 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80315a0 con 0x7ff4d01527b0 2024-03-06T07:28:06.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d4963640 1 -- 172.21.15.17:0/85915916 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c8037070 con 0x7ff4d01527b0 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d4963640 1 -- 172.21.15.17:0/85915916 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8031960 con 0x7ff4d01527b0 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d01527b0 msgr2=0x7ff4d0159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d01527b0 0x7ff4d0159100 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80097a0 tx=0x7ff4c802b840 comp rx=0 tx=0).stop 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/85915916 shutdown_connections 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d00c7180 0x7ff4d00c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/85915916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d01527b0 0x7ff4d0159100 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.863+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/85915916 >> 172.21.15.17:0/85915916 conn(0x7ff4d00ac560 msgr2=0x7ff4d00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.864+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/85915916 shutdown_connections 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.864+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/85915916 wait complete. 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 Processor -- start 2024-03-06T07:28:06.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 -- start start 2024-03-06T07:28:06.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d00c7180 0x7ff4d00bde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d0155b20 0x7ff4d00bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0155320 con 0x7ff4d01527b0 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d00c7980 con 0x7ff4d00c7180 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d7bc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0155600 con 0x7ff4d0155b20 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d5164640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d5164640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33112/0 (socket says 172.21.15.17:33112) 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.865+0000 7ff4d5164640 1 -- 172.21.15.17:0/1889035642 learned_addr learned my addr 172.21.15.17:0/1889035642 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d6166640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d0155b20 0x7ff4d00bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5965640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d00c7180 0x7ff4d00bde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 -- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d0155b20 msgr2=0x7ff4d00bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d0155b20 0x7ff4d00bfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 -- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d00c7180 msgr2=0x7ff4d00bde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d00c7180 0x7ff4d00bde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 -- 172.21.15.17:0/1889035642 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d01ee120 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d5164640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9d0 tx=0x7ff4c400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4017070 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c400ca00 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d01ee400 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400cdb0 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.866+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01ee9c0 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.867+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d004f330 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.868+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4c401a070 con 0x7ff4d01527b0 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.868+0000 7ff4ce7fc640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 0x7ff4bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:06.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.868+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff4c408a380 con 0x7ff4d01527b0 2024-03-06T07:28:06.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.868+0000 7ff4d5965640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 0x7ff4bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:06.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.869+0000 7ff4d5965640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 0x7ff4bc06d9c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8036000 tx=0x7ff4c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:06.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:06.872+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4c4059320 con 0x7ff4d01527b0 2024-03-06T07:28:07.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.133+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]} v 0) v1 -- 0x7ff4d006ce00 con 0x7ff4d01527b0 2024-03-06T07:28:07.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.424+0000 7ff4ce7fc640 1 -- 172.21.15.17:0/1889035642 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_grow"]}]=0 v468) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff4c4058c20 con 0x7ff4d01527b0 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 msgr2=0x7ff4bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 0x7ff4bc06d9c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8036000 tx=0x7ff4c8008030 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 msgr2=0x7ff4d0154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e9d0 tx=0x7ff4c400ee90 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 shutdown_connections 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff4bc06b500 0x7ff4bc06d9c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff4d0155b20 0x7ff4d00bfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d01527b0 0x7ff4d0154590 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 --2- 172.21.15.17:0/1889035642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4d00c7180 0x7ff4d00bde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 >> 172.21.15.17:0/1889035642 conn(0x7ff4d00ac560 msgr2=0x7ff4d006b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 shutdown_connections 2024-03-06T07:28:07.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.426+0000 7ff4d7bc9640 1 -- 172.21.15.17:0/1889035642 wait complete. 2024-03-06T07:28:07.439 INFO:tasks.cephfs_test_runner:That setting max_mds undoes down. ... ok 2024-03-06T07:28:07.440 INFO:tasks.cephfs_test_runner:Starting test: test_down_health (tasks.cephfs.test_failover.TestClusterResize) That marking a FS down does not generate a health warning 2024-03-06T07:28:07.440 INFO:tasks.cephfs_test_runner:test_down_health (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:28:07.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:07.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 Processor -- start 2024-03-06T07:28:07.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 -- start start 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1b814f790 0x7fa1b81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 0x7fa1b80c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80c28d0 con 0x7fa1b80c5c60 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80c2680 con 0x7fa1b814f790 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.542+0000 7fa1bcb0d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c2bb0 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1b814f790 0x7fa1b81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa1b80c5c60 0x7fa1b8155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40806/0 (socket says 172.21.15.17:40806) 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b6d9d640 1 -- 172.21.15.17:0/3119162944 learned_addr learned my addr 172.21.15.17:0/3119162944 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1affff640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 0x7fa1b80c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b659c640 1 -- 172.21.15.17:0/3119162944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 msgr2=0x7fa1b80c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b659c640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 0x7fa1b80c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b659c640 1 -- 172.21.15.17:0/3119162944 >> v1:172.21.15.17:6789/0 conn(0x7fa1b80c5c60 legacy=0x7fa1b8155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1b659c640 1 -- 172.21.15.17:0/3119162944 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80c2e90 con 0x7fa1b814f790 2024-03-06T07:28:07.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.543+0000 7fa1affff640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 0x7fa1b80c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1b659c640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b814f790 0x7fa1b81529d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80097a0 tx=0x7fa1a802dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9b5efc4685097c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1b5d9b640 1 -- 172.21.15.17:0/3119162944 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a802f8a0 con 0x7fa1b814f790 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1b5d9b640 1 -- 172.21.15.17:0/3119162944 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a802fa40 con 0x7fa1b814f790 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1b5d9b640 1 -- 172.21.15.17:0/3119162944 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a802fe10 con 0x7fa1b814f790 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/3119162944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b814f790 msgr2=0x7fa1b81529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b814f790 0x7fa1b81529d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80097a0 tx=0x7fa1a802dae0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/3119162944 shutdown_connections 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c1aa0 0x7fa1b80c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/3119162944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b814f790 0x7fa1b81529d0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/3119162944 >> 172.21.15.17:0/3119162944 conn(0x7fa1b80c6710 msgr2=0x7fa1b80c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/3119162944 shutdown_connections 2024-03-06T07:28:07.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.544+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/3119162944 wait complete. 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 Processor -- start 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 -- start start 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c5c60 0x7fa1b81eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 0x7fa1b81eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b806cc00 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c0d60 con 0x7fa1b814f790 2024-03-06T07:28:07.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1bcb0d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b806cd80 con 0x7fa1b80c5c60 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.546+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33128/0 (socket says 172.21.15.17:33128) 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 -- 172.21.15.17:0/1506360725 learned_addr learned my addr 172.21.15.17:0/1506360725 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1affff640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c5c60 0x7fa1b81eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b6d9d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 0x7fa1b81eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 -- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c5c60 msgr2=0x7fa1b81eb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c5c60 0x7fa1b81eb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 -- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 msgr2=0x7fa1b81eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 0x7fa1b81eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 -- 172.21.15.17:0/1506360725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b8073140 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b6d9d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 0x7fa1b81eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1b659c640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098c0 tx=0x7fa1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8047070 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8030ab0 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8042410 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b80728a0 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.547+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b8072e60 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.548+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b804f330 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.549+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1a8039400 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.549+0000 7fa1aeffd640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 0x7fa19806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.549+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa1a80ab980 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.549+0000 7fa1affff640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 0x7fa19806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.550+0000 7fa1affff640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 0x7fa19806d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa1b81ef0a0 tx=0x7fa1a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.554+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1a8035030 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.736+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b80c1f40 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.737+0000 7fa1aeffd640 1 -- 172.21.15.17:0/1506360725 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+13433 (secure 0 0 0) 0x7fa1a807a360 con 0x7fa1b80c1aa0 2024-03-06T07:28:07.739 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:07.739 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":191,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:06.419247+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6835/2561439920":"2024-03-07T07:28:03.410133+0000","172.21.15.17:6833/3930100029":"2024-03-07T07:28:03.410121+0000","172.21.15.17:6834/2561439920":"2024-03-07T07:28:03.410133+0000","172.21.15.17:6832/3930100029":"2024-03-07T07:28:03.410121+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 msgr2=0x7fa19806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 0x7fa19806d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa1b81ef0a0 tx=0x7fa1a0008040 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 msgr2=0x7fa1b80c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80098c0 tx=0x7fa1a80046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 shutdown_connections 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa19806b4d0 0x7fa19806d990 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b814f790 0x7fa1b81eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80c5c60 0x7fa1b81eb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 --2- 172.21.15.17:0/1506360725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1b80c1aa0 0x7fa1b80c35c0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 >> 172.21.15.17:0/1506360725 conn(0x7fa1b80c6710 msgr2=0x7fa1b80a6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 shutdown_connections 2024-03-06T07:28:07.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.739+0000 7fa1bcb0d640 1 -- 172.21.15.17:0/1506360725 wait complete. 2024-03-06T07:28:07.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:07.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.854+0000 7fec1200e640 1 Processor -- start 2024-03-06T07:28:07.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.854+0000 7fec1200e640 1 -- start start 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec1200e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0a8290 0x7fec0c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec1200e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec0c0c69a0 0x7fec0c072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec1200e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c0c62a0 con 0x7fec0c0c4740 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec1200e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c153f00 con 0x7fec0c0c69a0 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec1200e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0c4c00 con 0x7fec0c0a8290 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec0c0c69a0 0x7fec0c072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fec0c0c4740 0x7fec0c0701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40816/0 (socket says 172.21.15.17:40816) 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0affd640 1 -- 172.21.15.17:0/3905471071 learned_addr learned my addr 172.21.15.17:0/3905471071 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0a7fc640 1 -- 172.21.15.17:0/3905471071 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274670373 0 0) 0x7fec0c0c62a0 con 0x7fec0c0c4740 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0a7fc640 1 -- 172.21.15.17:0/3905471071 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf4003520 con 0x7fec0c0c4740 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0b7fe640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0a8290 0x7fec0c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.855+0000 7fec0bfff640 1 -- 172.21.15.17:0/3905471071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0a8290 msgr2=0x7fec0c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0bfff640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0a8290 0x7fec0c1539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0bfff640 1 -- 172.21.15.17:0/3905471071 >> v1:172.21.15.17:6789/0 conn(0x7fec0c0c4740 legacy=0x7fec0c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0bfff640 1 -- 172.21.15.17:0/3905471071 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0c4ee0 con 0x7fec0c0c69a0 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0bfff640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c072680 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec0402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a725fa8ac032042d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0a7fc640 1 -- 172.21.15.17:0/3905471071 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04039070 con 0x7fec0c0c69a0 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0a7fc640 1 -- 172.21.15.17:0/3905471071 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0402cb30 con 0x7fec0c0c69a0 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.856+0000 7fec0a7fc640 1 -- 172.21.15.17:0/3905471071 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04034440 con 0x7fec0c0c69a0 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 -- 172.21.15.17:0/3905471071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 msgr2=0x7fec0c072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c072680 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec0402bd80 comp rx=0 tx=0).stop 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 -- 172.21.15.17:0/3905471071 shutdown_connections 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c072680 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 --2- 172.21.15.17:0/3905471071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0a8290 0x7fec0c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 -- 172.21.15.17:0/3905471071 >> 172.21.15.17:0/3905471071 conn(0x7fec0c0c5560 msgr2=0x7fec0c0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 -- 172.21.15.17:0/3905471071 shutdown_connections 2024-03-06T07:28:07.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.857+0000 7fec1200e640 1 -- 172.21.15.17:0/3905471071 wait complete. 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.858+0000 7fec1200e640 1 Processor -- start 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.858+0000 7fec1200e640 1 -- start start 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 0x7fec0c06dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 0x7fec0c1ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0c6e50 con 0x7fec0c0c69a0 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06fc80 con 0x7fec0c0a8290 2024-03-06T07:28:07.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec1200e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c072bc0 con 0x7fec0c0c4740 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33144/0 (socket says 172.21.15.17:33144) 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 -- 172.21.15.17:0/4088626764 learned_addr learned my addr 172.21.15.17:0/4088626764 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0b7fe640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 0x7fec0c06dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0affd640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 0x7fec0c1ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 -- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 msgr2=0x7fec0c1ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 0x7fec0c1ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 -- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 msgr2=0x7fec0c06dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 0x7fec0c06dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 -- 172.21.15.17:0/4088626764 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c06cff0 con 0x7fec0c0c69a0 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0b7fe640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 0x7fec0c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:07.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0affd640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 0x7fec0c1ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.859+0000 7fec0bfff640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.860+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04039050 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.860+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec04045070 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.860+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c06d2d0 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.860+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04009450 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.860+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c1eef90 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.861+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c04f330 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.861+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec0403e850 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.861+0000 7febf3fff640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 0x7febf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.861+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fec040acf40 con 0x7fec0c0c69a0 2024-03-06T07:28:07.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.862+0000 7fec0b7fe640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 0x7febf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:07.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.862+0000 7fec0b7fe640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 0x7febf806d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7febfc00a9d0 tx=0x7febfc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:07.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:07.866+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fec04032e20 con 0x7fec0c0c69a0 2024-03-06T07:28:08.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.048+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec0c0c3140 con 0x7fec0c0c69a0 2024-03-06T07:28:08.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.048+0000 7febf3fff640 1 -- 172.21.15.17:0/4088626764 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec0407c090 con 0x7fec0c0c69a0 2024-03-06T07:28:08.050 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 msgr2=0x7febf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 0x7febf806d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7febfc00a9d0 tx=0x7febfc008040 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 msgr2=0x7fec0c152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fec04009d90 tx=0x7fec04004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 shutdown_connections 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7febf806b4d0 0x7febf806d990 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec0c0c69a0 0x7fec0c152330 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec0c0c4740 0x7fec0c1ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 --2- 172.21.15.17:0/4088626764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec0c0a8290 0x7fec0c06dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 >> 172.21.15.17:0/4088626764 conn(0x7fec0c0c5560 msgr2=0x7fec0c153370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 shutdown_connections 2024-03-06T07:28:08.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.050+0000 7fec1200e640 1 -- 172.21.15.17:0/4088626764 wait complete. 2024-03-06T07:28:08.062 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health' 2024-03-06T07:28:08.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.162+0000 7f6ccce66640 1 Processor -- start 2024-03-06T07:28:08.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.163+0000 7f6ccce66640 1 -- start start 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.163+0000 7f6ccce66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cc8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.163+0000 7f6ccce66640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80c7760 con 0x7f6cc8152920 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.163+0000 7f6ccce66640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80c7a40 con 0x7f6cc8159460 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.163+0000 7f6ccce66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81587b0 con 0x7f6cc80c8a00 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6cc8152920 0x7f6cc8155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40818/0 (socket says 172.21.15.17:40818) 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc659c640 1 -- 172.21.15.17:0/3258240660 learned_addr learned my addr 172.21.15.17:0/3258240660 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cc8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457089481 0 0) 0x7f6cc80c7760 con 0x7f6cc8152920 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb4003520 con 0x7f6cc8152920 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827568814 0 0) 0x7f6cb4003520 con 0x7f6cc8152920 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 msgr2=0x7f6cc8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cc8158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 >> v1:172.21.15.163:6789/0 conn(0x7f6cc8159460 legacy=0x7f6cc80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80c7d20 con 0x7f6cc8152920 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6cc00019f0 con 0x7f6cc8152920 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 >> v1:172.21.15.17:6789/0 conn(0x7f6cc8152920 legacy=0x7f6cc8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40039e0 0x7f6cb4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cb4004800 0x7f6cb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cb4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81587b0 con 0x7f6cb4004800 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400baf0 con 0x7f6cb40039e0 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400bcf0 con 0x7f6cc80c8a00 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cb4004800 0x7f6cb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc659c640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40039e0 0x7f6cb4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.164+0000 7f6cc6d9d640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cb4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 msgr2=0x7f6cb4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cc80c8a00 0x7f6cb4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40039e0 msgr2=0x7f6cb4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40039e0 0x7f6cb4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/3258240660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb4003520 con 0x7f6cb4004800 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cb4004800 0x7f6cb4004c80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009ac0 tx=0x7f6cb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17251f23afaac5cd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.165+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0039070 con 0x7f6cb4004800 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb002c6b0 con 0x7f6cb4004800 2024-03-06T07:28:08.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6cc559a640 1 -- 172.21.15.17:0/3258240660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0009690 con 0x7f6cb4004800 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 -- 172.21.15.17:0/3258240660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cb4004800 msgr2=0x7f6cb4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 --2- 172.21.15.17:0/3258240660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cb4004800 0x7f6cb4004c80 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009ac0 tx=0x7f6cb4005830 comp rx=0 tx=0).stop 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6cc6d9d640 1 -- 172.21.15.17:0/3258240660 reap_dead start 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 -- 172.21.15.17:0/3258240660 shutdown_connections 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 -- 172.21.15.17:0/3258240660 >> 172.21.15.17:0/3258240660 conn(0x7f6cc80ac560 msgr2=0x7f6cc80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 -- 172.21.15.17:0/3258240660 shutdown_connections 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.166+0000 7f6ccce66640 1 -- 172.21.15.17:0/3258240660 wait complete. 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.167+0000 7f6ccce66640 1 Processor -- start 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.167+0000 7f6ccce66640 1 -- start start 2024-03-06T07:28:08.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 0x7f6cc80abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cc81eb590 0x7f6cc8147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80c5a40 con 0x7f6cc81df930 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc806d030 con 0x7f6cc81eb590 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6ccce66640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80c5bc0 con 0x7f6cb4004800 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33166/0 (socket says 172.21.15.17:33166) 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/1333574128 learned_addr learned my addr 172.21.15.17:0/1333574128 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc6d9d640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cc81eb590 0x7f6cc8147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc659c640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 0x7f6cc80abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 msgr2=0x7f6cc80abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 0x7f6cc80abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cc81eb590 msgr2=0x7f6cc8147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cc81eb590 0x7f6cc8147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc5d9b640 1 -- 172.21.15.17:0/1333574128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc81e23e0 con 0x7f6cc81df930 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.168+0000 7f6cc659c640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 0x7f6cc80abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:08.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6cc5d9b640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0037040 tx=0x7f6cb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0042020 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb003a040 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc81e26c0 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0039070 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.169+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc80ac290 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.170+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc81eb9a0 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.170+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cb0045020 con 0x7f6cc81df930 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.171+0000 7f6cbeffd640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 0x7f6ca806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:08.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.171+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6cb00ab430 con 0x7f6cc81df930 2024-03-06T07:28:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.171+0000 7f6cc659c640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 0x7f6ca806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.171+0000 7f6cc659c640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 0x7f6ca806d9c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80bda70 tx=0x7f6cc0006040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:08.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.175+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6cb004c070 con 0x7f6cc81df930 2024-03-06T07:28:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:08.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]} v 0) v1 -- 0x7f6cc804f330 con 0x7f6cc81df930 2024-03-06T07:28:09.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.429+0000 7f6cbeffd640 1 -- 172.21.15.17:0/1333574128 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]}]=0 v470) v1 ==== 143+0+0 (secure 0 0 0) 0x7f6cb0079e10 con 0x7f6cc81df930 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 msgr2=0x7f6ca806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 0x7f6ca806d9c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6cc80bda70 tx=0x7f6cc0006040 comp rx=0 tx=0).stop 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 msgr2=0x7f6cc81eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0037040 tx=0x7f6cb0004730 comp rx=0 tx=0).stop 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 shutdown_connections 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ca806b500 0x7f6ca806d9c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cc81eb590 0x7f6cc8147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6cc81df930 0x7f6cc81eb050 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 --2- 172.21.15.17:0/1333574128 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4004800 0x7f6cc80abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 >> 172.21.15.17:0/1333574128 conn(0x7f6cc80ac560 msgr2=0x7f6cc80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 shutdown_connections 2024-03-06T07:28:09.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.432+0000 7f6ccce66640 1 -- 172.21.15.17:0/1333574128 wait complete. 2024-03-06T07:28:09.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:28:09.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.556+0000 7f026720e640 1 Processor -- start 2024-03-06T07:28:09.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.556+0000 7f026720e640 1 -- start start 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f026720e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f026720e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f026006cad0 0x7f0260154c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f026720e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02600c2790 con 0x7f02600c1820 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f026720e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02600c1ca0 con 0x7f026006cad0 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f026720e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02600c3090 con 0x7f026014ef70 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f026006cad0 0x7f0260154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f026006cad0 0x7f0260154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33176/0 (socket says 172.21.15.17:33176) 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 -- 172.21.15.17:0/755480072 learned_addr learned my addr 172.21.15.17:0/755480072 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f0264faa640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 -- 172.21.15.17:0/755480072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 msgr2=0x7f026006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026006c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 -- 172.21.15.17:0/755480072 >> v1:172.21.15.163:6789/0 conn(0x7f02600c1820 legacy=0x7f02600c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:09.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.557+0000 7f02657ab640 1 -- 172.21.15.17:0/755480072 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600c1540 con 0x7f026006cad0 2024-03-06T07:28:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.558+0000 7f02657ab640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026006cad0 0x7f0260154c30 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f025800d150 tx=0x7f025802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6b24a30abaadd server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.558+0000 7f025f7fe640 1 -- 172.21.15.17:0/755480072 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025802e9e0 con 0x7f026006cad0 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.558+0000 7f025f7fe640 1 -- 172.21.15.17:0/755480072 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025802eb80 con 0x7f026006cad0 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.558+0000 7f025f7fe640 1 -- 172.21.15.17:0/755480072 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258009600 con 0x7f026006cad0 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.558+0000 7f026720e640 1 -- 172.21.15.17:0/755480072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026006cad0 msgr2=0x7f0260154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026006cad0 0x7f0260154c30 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f025800d150 tx=0x7f025802e040 comp rx=0 tx=0).stop 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 -- 172.21.15.17:0/755480072 shutdown_connections 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f026006cad0 0x7f0260154c30 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 --2- 172.21.15.17:0/755480072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 -- 172.21.15.17:0/755480072 >> 172.21.15.17:0/755480072 conn(0x7f02600c6710 msgr2=0x7f02600c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 -- 172.21.15.17:0/755480072 shutdown_connections 2024-03-06T07:28:09.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.559+0000 7f026720e640 1 -- 172.21.15.17:0/755480072 wait complete. 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.560+0000 7f026720e640 1 Processor -- start 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.560+0000 7f026720e640 1 -- start start 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f026006cad0 0x7f0260071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026014da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026006cf60 con 0x7f02600c1820 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026014f350 con 0x7f026006cad0 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0260155750 con 0x7f026014ef70 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33186/0 (socket says 172.21.15.17:33186) 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 -- 172.21.15.17:0/1416123056 learned_addr learned my addr 172.21.15.17:0/1416123056 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f02657ab640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026014da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f0264faa640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f026006cad0 0x7f0260071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 -- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 msgr2=0x7f026014da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026014da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 -- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f026006cad0 msgr2=0x7f0260071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f026006cad0 0x7f0260071810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 -- 172.21.15.17:0/1416123056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260071df0 con 0x7f02600c1820 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f02657ab640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026014da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025ffff640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c430 tx=0x7f024c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c011790 con 0x7f02600c1820 2024-03-06T07:28:09.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c017070 con 0x7f02600c1820 2024-03-06T07:28:09.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02600720d0 con 0x7f02600c1820 2024-03-06T07:28:09.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c011bb0 con 0x7f02600c1820 2024-03-06T07:28:09.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.561+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02601ea1f0 con 0x7f02600c1820 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.562+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026004f330 con 0x7f02600c1820 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.563+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f024c01c460 con 0x7f02600c1820 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.563+0000 7f025d7fa640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.563+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f024c089fe0 con 0x7f02600c1820 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.564+0000 7f0264faa640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:09.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.564+0000 7f0264faa640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 0x7f024406d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f025400b7c0 tx=0x7f0254008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:09.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.568+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f024c058f70 con 0x7f02600c1820 2024-03-06T07:28:09.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.751+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02600c1110 con 0x7f02600c1820 2024-03-06T07:28:09.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.754+0000 7f025d7fa640 1 -- 172.21.15.17:0/1416123056 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v61)=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f024c058870 con 0x7f02600c1820 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 msgr2=0x7f024406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 0x7f024406d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f025400b7c0 tx=0x7f0254008040 comp rx=0 tx=0).stop 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 msgr2=0x7f02601e7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c430 tx=0x7f024c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 shutdown_connections 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f024406b4d0 0x7f024406d990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f026014ef70 0x7f026014da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02600c1820 0x7f02601e7280 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 --2- 172.21.15.17:0/1416123056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f026006cad0 0x7f0260071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:09.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.756+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 >> 172.21.15.17:0/1416123056 conn(0x7f02600c6710 msgr2=0x7f026014be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:09.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.757+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 shutdown_connections 2024-03-06T07:28:09.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:09.757+0000 7f026720e640 1 -- 172.21.15.17:0/1416123056 wait complete. 2024-03-06T07:28:09.769 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:28:09.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:09.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:09.870 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:28:09.870 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:28:09.870 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:09.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:28:09.885 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:28:09.885 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:09.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:28:09.902 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:28:09.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:10.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.024+0000 7f2e5b302640 1 Processor -- start 2024-03-06T07:28:10.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5b302640 1 -- start start 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5b302640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e54159100 0x7f2e540c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5b302640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e540c7c60 con 0x7f2e541527b0 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5b302640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e540c8060 con 0x7f2e540c7180 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5b302640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e540c8df0 con 0x7f2e54159100 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5989f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2e54159100 0x7f2e540c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5909e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2e541527b0 0x7f2e54155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:40820/0 (socket says 172.21.15.17:40820) 2024-03-06T07:28:10.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.025+0000 7f2e5909e640 1 -- 172.21.15.17:0/940227867 learned_addr learned my addr 172.21.15.17:0/940227867 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:10.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.026+0000 7f2e5989f640 1 -- 172.21.15.17:0/940227867 >> v1:172.21.15.163:6789/0 conn(0x7f2e540c7180 legacy=0x7f2e540c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.026+0000 7f2e5989f640 1 -- 172.21.15.17:0/940227867 >> v1:172.21.15.17:6789/0 conn(0x7f2e541527b0 legacy=0x7f2e54155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.026+0000 7f2e5989f640 1 -- 172.21.15.17:0/940227867 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540c76a0 con 0x7f2e54159100 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.026+0000 7f2e5989f640 1 --2- 172.21.15.17:0/940227867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e54159100 0x7f2e540c88b0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f2e440097a0 tx=0x7f2e4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e039ac24544b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e53fff640 1 -- 172.21.15.17:0/940227867 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440315a0 con 0x7f2e54159100 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e53fff640 1 -- 172.21.15.17:0/940227867 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e44037070 con 0x7f2e54159100 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e53fff640 1 -- 172.21.15.17:0/940227867 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44031960 con 0x7f2e54159100 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 -- 172.21.15.17:0/940227867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e54159100 msgr2=0x7f2e540c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 --2- 172.21.15.17:0/940227867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e54159100 0x7f2e540c88b0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f2e440097a0 tx=0x7f2e4402b840 comp rx=0 tx=0).stop 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 -- 172.21.15.17:0/940227867 shutdown_connections 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 --2- 172.21.15.17:0/940227867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e54159100 0x7f2e540c88b0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 -- 172.21.15.17:0/940227867 >> 172.21.15.17:0/940227867 conn(0x7f2e540ac560 msgr2=0x7f2e540ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 -- 172.21.15.17:0/940227867 shutdown_connections 2024-03-06T07:28:10.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.027+0000 7f2e5b302640 1 -- 172.21.15.17:0/940227867 wait complete. 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.028+0000 7f2e5b302640 1 Processor -- start 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.028+0000 7f2e5b302640 1 -- start start 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 0x7f2e5414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 0x7f2e5414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54155600 con 0x7f2e541527b0 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54155320 con 0x7f2e540c7180 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5b302640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54152b90 con 0x7f2e54159100 2024-03-06T07:28:10.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33204/0 (socket says 172.21.15.17:33204) 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 -- 172.21.15.17:0/149519206 learned_addr learned my addr 172.21.15.17:0/149519206 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5909e640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 0x7f2e5414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5989f640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 0x7f2e5414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 -- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 msgr2=0x7f2e5414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 0x7f2e5414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 -- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 msgr2=0x7f2e5414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 0x7f2e5414e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5889d640 1 -- 172.21.15.17:0/149519206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e540c3a00 con 0x7f2e541527b0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5989f640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 0x7f2e5414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.029+0000 7f2e5909e640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 0x7f2e5414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e5889d640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c410 tx=0x7f2e4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48017070 con 0x7f2e541527b0 2024-03-06T07:28:10.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e480035f0 con 0x7f2e541527b0 2024-03-06T07:28:10.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e540c3ce0 con 0x7f2e541527b0 2024-03-06T07:28:10.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48003a20 con 0x7f2e541527b0 2024-03-06T07:28:10.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.030+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5414ddc0 con 0x7f2e541527b0 2024-03-06T07:28:10.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.031+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5404f330 con 0x7f2e541527b0 2024-03-06T07:28:10.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.031+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e4801a070 con 0x7f2e541527b0 2024-03-06T07:28:10.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.031+0000 7f2e51ffb640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 0x7f2e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.032+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2e48089540 con 0x7f2e541527b0 2024-03-06T07:28:10.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.032+0000 7f2e5909e640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 0x7f2e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.032+0000 7f2e5909e640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 0x7f2e4006d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0012e0 tx=0x7f2e4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.036+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e48058570 con 0x7f2e541527b0 2024-03-06T07:28:10.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:10.173 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.255+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e5406ce00 con 0x7f2e541527b0 2024-03-06T07:28:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.255+0000 7f2e51ffb640 1 -- 172.21.15.17:0/149519206 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+3706 (secure 0 0 0) 0x7f2e48057e70 con 0x7f2e541527b0 2024-03-06T07:28:10.257 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:10.257 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":184,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/3446094037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":3446094037},{"type":"v1","addr":"172.21.15.163:6837","nonce":3446094037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":8899,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/4082854675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":4082854675},{"type":"v1","addr":"172.21.15.163:6836","nonce":4082854675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":169},{"gid":10347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/835911832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":835911832},{"type":"v1","addr":"172.21.15.17:6833","nonce":835911832}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184},{"gid":10350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/375958918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":375958918},{"type":"v1","addr":"172.21.15.17:6835","nonce":375958918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":184}],"filesystems":[]} 2024-03-06T07:28:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 184 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 msgr2=0x7f2e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 0x7f2e4006d990 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0012e0 tx=0x7f2e4c009040 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 msgr2=0x7f2e5414b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c410 tx=0x7f2e4800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 shutdown_connections 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e4006b4d0 0x7f2e4006d990 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e54159100 0x7f2e5414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e541527b0 0x7f2e5414b890 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 --2- 172.21.15.17:0/149519206 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e540c7180 0x7f2e5414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 >> 172.21.15.17:0/149519206 conn(0x7f2e540ac560 msgr2=0x7f2e540c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 shutdown_connections 2024-03-06T07:28:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.257+0000 7f2e5b302640 1 -- 172.21.15.17:0/149519206 wait complete. 2024-03-06T07:28:10.269 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:28:10.269 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:28:10.270 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:28:10.270 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.270 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:28:10.270 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:28:10.270 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:28:10.270 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.270 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:28:10.270 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:28:10.271 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:28:10.271 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.271 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:28:10.271 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:28:10.271 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:28:10.271 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:28:10.283 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:28:10.283 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:28:10.284 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:28:10.284 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:28:10.318 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:28:10.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:28:10.319 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:28:10.320 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:28:10.326 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:28:10.326 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.327 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:28:10.327 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.361 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:28:10.362 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.362 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:28:10.362 DEBUG:teuthology.parallel:result is None 2024-03-06T07:28:10.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:28:10.405 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:28:10.405 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:28:10.440 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:28:10.446 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:28:10.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.466+0000 7f835058c640 1 Processor -- start 2024-03-06T07:28:10.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.466+0000 7f835058c640 1 -- start start 2024-03-06T07:28:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f835058c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f83480c8950 0x7f83480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f835058c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83481558d0 0x7f8348155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f835058c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8348155130 con 0x7f83480c69f0 2024-03-06T07:28:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f835058c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8348155440 con 0x7f83480c8950 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f835058c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83480ad480 con 0x7f83481558d0 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f834db27640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f83480c8950 0x7f83480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f834db27640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f83480c8950 0x7f83480c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33248/0 (socket says 172.21.15.17:33248) 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f834db27640 1 -- 172.21.15.17:0/1476941079 learned_addr learned my addr 172.21.15.17:0/1476941079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.467+0000 7f834eb29640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83481558d0 0x7f8348155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.468+0000 7f834db27640 1 -- 172.21.15.17:0/1476941079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83481558d0 msgr2=0x7f8348155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.468+0000 7f834db27640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83481558d0 0x7f8348155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.468+0000 7f834db27640 1 -- 172.21.15.17:0/1476941079 >> v1:172.21.15.163:6789/0 conn(0x7f83480c69f0 legacy=0x7f83480acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.468+0000 7f834db27640 1 -- 172.21.15.17:0/1476941079 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480c5d50 con 0x7f83480c8950 2024-03-06T07:28:10.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.468+0000 7f834db27640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f83480c6dd0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f833c009870 tx=0x7f833c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d943f3be9386886 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f834d326640 1 -- 172.21.15.17:0/1476941079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c035020 con 0x7f83480c8950 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f834d326640 1 -- 172.21.15.17:0/1476941079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c031640 con 0x7f83480c8950 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f834d326640 1 -- 172.21.15.17:0/1476941079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c02d040 con 0x7f83480c8950 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 -- 172.21.15.17:0/1476941079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 msgr2=0x7f83480c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f83480c6dd0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f833c009870 tx=0x7f833c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 -- 172.21.15.17:0/1476941079 shutdown_connections 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83481558d0 0x7f8348155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 --2- 172.21.15.17:0/1476941079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f83480c6dd0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 -- 172.21.15.17:0/1476941079 >> 172.21.15.17:0/1476941079 conn(0x7f83480cae70 msgr2=0x7f83480cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 -- 172.21.15.17:0/1476941079 shutdown_connections 2024-03-06T07:28:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.469+0000 7f835058c640 1 -- 172.21.15.17:0/1476941079 wait complete. 2024-03-06T07:28:10.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.470+0000 7f835058c640 1 Processor -- start 2024-03-06T07:28:10.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 -- start start 2024-03-06T07:28:10.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 0x7f83480aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481558d0 0x7f834814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8348152a20 con 0x7f83480c8950 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83480c6310 con 0x7f83481558d0 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f835058c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8348152ba0 con 0x7f83480c69f0 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f834db27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f834db27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33250/0 (socket says 172.21.15.17:33250) 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f834db27640 1 -- 172.21.15.17:0/2898382198 learned_addr learned my addr 172.21.15.17:0/2898382198 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f834eb29640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481558d0 0x7f834814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.471+0000 7f834e328640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 0x7f83480aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 -- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 msgr2=0x7f83480aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 0x7f83480aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 -- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481558d0 msgr2=0x7f834814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481558d0 0x7f834814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 -- 172.21.15.17:0/2898382198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348156b10 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834e328640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 0x7f83480aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f834db27640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c040070 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c0399f0 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c035040 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83480ae290 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.472+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83480ad710 con 0x7f83480c8950 2024-03-06T07:28:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.473+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8348075300 con 0x7f83480c8950 2024-03-06T07:28:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.474+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f833c04b060 con 0x7f83480c8950 2024-03-06T07:28:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.474+0000 7f8346ffd640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 0x7f833406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.474+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f833c0ab890 con 0x7f83480c8950 2024-03-06T07:28:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.474+0000 7f834e328640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 0x7f833406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.475+0000 7f834e328640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 0x7f833406d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8340001660 tx=0x7f8340009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:10.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.478+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8348075300 con 0x7f83480c8950 2024-03-06T07:28:10.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:10.675+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f83480ca650 con 0x7f83480c8950 2024-03-06T07:28:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.441+0000 7f8346ffd640 1 -- 172.21.15.17:0/2898382198 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v195) v1 ==== 97+0+0 (secure 0 0 0) 0x7f833c07a830 con 0x7f83480c8950 2024-03-06T07:28:11.443 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 msgr2=0x7f833406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 0x7f833406d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8340001660 tx=0x7f8340009040 comp rx=0 tx=0).stop 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 msgr2=0x7f834814c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f833c000c00 tx=0x7f833c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 shutdown_connections 2024-03-06T07:28:11.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f833406b4d0 0x7f833406d990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83481558d0 0x7f834814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83480c8950 0x7f834814c950 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 --2- 172.21.15.17:0/2898382198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83480c69f0 0x7f83480aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 >> 172.21.15.17:0/2898382198 conn(0x7f83480cae70 msgr2=0x7f83480bfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 shutdown_connections 2024-03-06T07:28:11.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.444+0000 7f835058c640 1 -- 172.21.15.17:0/2898382198 wait complete. 2024-03-06T07:28:11.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:28:11.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.565+0000 7f20dd4c9640 1 Processor -- start 2024-03-06T07:28:11.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.565+0000 7f20dd4c9640 1 -- start start 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20dd4c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d80c5480 0x7f20d8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20dd4c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20d80c5860 0x7f20d806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20dd4c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d80c5cc0 con 0x7f20d81586f0 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20dd4c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d806df40 con 0x7f20d80c5480 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20dd4c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d81591c0 con 0x7f20d80c5860 2024-03-06T07:28:11.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20d6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d80c5480 0x7f20d8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20d6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20d80c5480 0x7f20d8155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50684/0 (socket says 172.21.15.17:50684) 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.566+0000 7f20d6ffd640 1 -- 172.21.15.17:0/1630319567 learned_addr learned my addr 172.21.15.17:0/1630319567 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d6ffd640 1 -- 172.21.15.17:0/1630319567 >> v2:172.21.15.163:3300/0 conn(0x7f20d80c5860 msgr2=0x7f20d806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d6ffd640 1 --2- 172.21.15.17:0/1630319567 >> v2:172.21.15.163:3300/0 conn(0x7f20d80c5860 0x7f20d806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d6ffd640 1 -- 172.21.15.17:0/1630319567 >> v1:172.21.15.163:6789/0 conn(0x7f20d81586f0 legacy=0x7f20d80724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d6ffd640 1 -- 172.21.15.17:0/1630319567 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d81594a0 con 0x7f20d80c5480 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d6ffd640 1 --2- 172.21.15.17:0/1630319567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d80c5480 0x7f20d8155ba0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f20d80c9770 tx=0x7f20d002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1017ab27436d0214 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.567+0000 7f20d5ffb640 1 -- 172.21.15.17:0/1630319567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d002e070 con 0x7f20d80c5480 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20d5ffb640 1 -- 172.21.15.17:0/1630319567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d0037070 con 0x7f20d80c5480 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20d5ffb640 1 -- 172.21.15.17:0/1630319567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0031a00 con 0x7f20d80c5480 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 -- 172.21.15.17:0/1630319567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d80c5480 msgr2=0x7f20d8155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/1630319567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d80c5480 0x7f20d8155ba0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f20d80c9770 tx=0x7f20d002b880 comp rx=0 tx=0).stop 2024-03-06T07:28:11.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 -- 172.21.15.17:0/1630319567 shutdown_connections 2024-03-06T07:28:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/1630319567 >> v2:172.21.15.163:3300/0 conn(0x7f20d80c5860 0x7f20d806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/1630319567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d80c5480 0x7f20d8155ba0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 -- 172.21.15.17:0/1630319567 >> 172.21.15.17:0/1630319567 conn(0x7f20d806b1a0 msgr2=0x7f20d806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 -- 172.21.15.17:0/1630319567 shutdown_connections 2024-03-06T07:28:11.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.568+0000 7f20dd4c9640 1 -- 172.21.15.17:0/1630319567 wait complete. 2024-03-06T07:28:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.569+0000 7f20dd4c9640 1 Processor -- start 2024-03-06T07:28:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.569+0000 7f20dd4c9640 1 -- start start 2024-03-06T07:28:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d80c5480 0x7f20d8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 0x7f20d8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8152a20 con 0x7f20d81586f0 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8155680 con 0x7f20d80c5480 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20dd4c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8155050 con 0x7f20d80c5860 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50696/0 (socket says 172.21.15.17:50696) 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20d77fe640 1 -- 172.21.15.17:0/2036451744 learned_addr learned my addr 172.21.15.17:0/2036451744 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20d6ffd640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d80c5480 0x7f20d8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.570+0000 7f20d67fc640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 0x7f20d8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 -- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 msgr2=0x7f20d8154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 0x7f20d8154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 -- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d80c5480 msgr2=0x7f20d8151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d80c5480 0x7f20d8151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 -- 172.21.15.17:0/2036451744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d81520f0 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d67fc640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 0x7f20d8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20d77fe640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ba10 tx=0x7f20c800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8019070 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c800cb20 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d8151910 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8014440 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.571+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d81eb320 con 0x7f20d81586f0 2024-03-06T07:28:11.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.572+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d80c7a50 con 0x7f20d81586f0 2024-03-06T07:28:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.572+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20c801c070 con 0x7f20d81586f0 2024-03-06T07:28:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.573+0000 7f20cf7fe640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 0x7f20b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.573+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f20c8089ba0 con 0x7f20d81586f0 2024-03-06T07:28:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.573+0000 7f20d6ffd640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 0x7f20b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.573+0000 7f20d6ffd640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 0x7f20b806d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f20d0036000 tx=0x7f20d003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.577+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20d80c7a50 con 0x7f20d81586f0 2024-03-06T07:28:11.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.828+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f20d804f330 con 0x7f20d81586f0 2024-03-06T07:28:11.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.828+0000 7f20cf7fe640 1 -- 172.21.15.17:0/2036451744 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+4724 (secure 0 0 0) 0x7f20d81520f0 con 0x7f20d81586f0 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.833 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.834 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:28:11.835 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.838 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:28:11.840 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:28:11.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 msgr2=0x7f20b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 0x7f20b806d990 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f20d0036000 tx=0x7f20d003d040 comp rx=0 tx=0).stop 2024-03-06T07:28:11.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 msgr2=0x7f20d8156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ba10 tx=0x7f20c800bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 shutdown_connections 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20b806b4d0 0x7f20b806d990 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20d81586f0 0x7f20d8156440 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20d80c5860 0x7f20d8154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 --2- 172.21.15.17:0/2036451744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20d80c5480 0x7f20d8151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 >> 172.21.15.17:0/2036451744 conn(0x7f20d806b1a0 msgr2=0x7f20d806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 shutdown_connections 2024-03-06T07:28:11.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.830+0000 7f20dd4c9640 1 -- 172.21.15.17:0/2036451744 wait complete. 2024-03-06T07:28:11.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.943+0000 7f3fa7d0c640 1 Processor -- start 2024-03-06T07:28:11.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.943+0000 7f3fa7d0c640 1 -- start start 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa7d0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3fa00c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa7d0c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa0155110 con 0x7f3fa0158db0 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa7d0c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa00c8f40 con 0x7f3fa00c5330 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa7d0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01559e0 con 0x7f3fa00c88b0 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa52a7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3fa0158db0 0x7f3fa015da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54972/0 (socket says 172.21.15.17:54972) 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa52a7640 1 -- 172.21.15.17:0/124449401 learned_addr learned my addr 172.21.15.17:0/124449401 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa62a9640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3fa00c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447363586 0 0) 0x7f3f90001110 con 0x7f3fa0158db0 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f8c003520 con 0x7f3fa0158db0 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019055108 0 0) 0x7f3fa00c8f40 con 0x7f3fa00c5330 2024-03-06T07:28:11.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f90001110 con 0x7f3fa00c5330 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771547660 0 0) 0x7f3f8c003520 con 0x7f3fa0158db0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 msgr2=0x7f3fa00c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3fa00c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 >> v1:172.21.15.163:6789/0 conn(0x7f3fa00c5330 legacy=0x7f3fa01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0155390 con 0x7f3fa0158db0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3f90001840 con 0x7f3fa0158db0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 >> v1:172.21.15.17:6789/0 conn(0x7f3fa0158db0 legacy=0x7f3fa015da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f8c003b70 0x7f3f8c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3f8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c004d40 0x7f3f8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01559e0 con 0x7f3f8c003b70 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c00bb20 con 0x7f3f8c004d40 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c00bd20 con 0x7f3fa00c88b0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.944+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f8c003b70 0x7f3f8c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa52a7640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3f8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa62a9640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c004d40 0x7f3f8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 -- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 msgr2=0x7f3f8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c88b0 0x7f3f8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 -- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c004d40 msgr2=0x7f3f8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c004d40 0x7f3f8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 -- 172.21.15.17:0/124449401 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c003520 con 0x7f3f8c003b70 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa62a9640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c004d40 0x7f3f8c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.945+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f8c003b70 0x7f3f8c003f90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0018d0 tx=0x7f3f8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1066a3ed44fa63 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0014f0 con 0x7f3f8c003b70 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c0052d0 con 0x7f3f8c003b70 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa4aa6640 1 -- 172.21.15.17:0/124449401 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c003810 con 0x7f3f8c003b70 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/124449401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f8c003b70 msgr2=0x7f3f8c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/124449401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f8c003b70 0x7f3f8c003f90 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0018d0 tx=0x7f3f8c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa62a9640 1 -- 172.21.15.17:0/124449401 reap_dead start 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/124449401 shutdown_connections 2024-03-06T07:28:11.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/124449401 >> 172.21.15.17:0/124449401 conn(0x7f3fa00ac560 msgr2=0x7f3fa00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:11.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/124449401 shutdown_connections 2024-03-06T07:28:11.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.946+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/124449401 wait complete. 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.947+0000 7f3fa7d0c640 1 Processor -- start 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.947+0000 7f3fa7d0c640 1 -- start start 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 0x7f3fa00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c1a80 0x7f3fa00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01529d0 con 0x7f3fa00bde30 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa006d090 con 0x7f3f8c003b70 2024-03-06T07:28:11.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa7d0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0152b50 con 0x7f3fa00c1a80 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50720/0 (socket says 172.21.15.17:50720) 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 -- 172.21.15.17:0/490427814 learned_addr learned my addr 172.21.15.17:0/490427814 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 0x7f3fa00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa62a9640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c1a80 0x7f3fa00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 -- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c1a80 msgr2=0x7f3fa00ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c1a80 0x7f3fa00ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 -- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 msgr2=0x7f3fa00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 0x7f3fa00b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa52a7640 1 -- 172.21.15.17:0/490427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00ae810 con 0x7f3fa00bde30 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.948+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 0x7f3fa00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3fa52a7640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000e8d0 tx=0x7f3f9000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90019070 con 0x7f3fa00bde30 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90005320 con 0x7f3fa00bde30 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa01eeaf0 con 0x7f3fa00bde30 2024-03-06T07:28:11.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f900047e0 con 0x7f3fa00bde30 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.949+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa00af890 con 0x7f3fa00bde30 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.950+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00af3a0 con 0x7f3fa00bde30 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.950+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f9001c070 con 0x7f3fa00bde30 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.950+0000 7f3f967fc640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 0x7f3f8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.951+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f900899a0 con 0x7f3fa00bde30 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.951+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 0x7f3f8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:11.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.951+0000 7f3fa5aa8640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 0x7f3f8406d9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c034000 tx=0x7f3f9c031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:11.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:11.955+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fa0155110 con 0x7f3fa00bde30 2024-03-06T07:28:12.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.135+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fa006cdc0 con 0x7f3fa00bde30 2024-03-06T07:28:12.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.135+0000 7f3f967fc640 1 -- 172.21.15.17:0/490427814 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+13174 (secure 0 0 0) 0x7f3f900589e0 con 0x7f3fa00bde30 2024-03-06T07:28:12.137 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:12.137 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":195,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:11.439392+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":184,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 msgr2=0x7f3f8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 0x7f3f8406d9c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c034000 tx=0x7f3f9c031000 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 msgr2=0x7f3fa0157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000e8d0 tx=0x7f3f9000ed90 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 shutdown_connections 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3f8406b500 0x7f3f8406d9c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3fa00c1a80 0x7f3fa00ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3fa00bde30 0x7f3fa0157c10 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 --2- 172.21.15.17:0/490427814 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f8c003b70 0x7f3fa00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 >> 172.21.15.17:0/490427814 conn(0x7f3fa00ac560 msgr2=0x7f3fa01549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 shutdown_connections 2024-03-06T07:28:12.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.137+0000 7f3fa7d0c640 1 -- 172.21.15.17:0/490427814 wait complete. 2024-03-06T07:28:12.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:12.150 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.253+0000 7ff2fba9a640 1 Processor -- start 2024-03-06T07:28:12.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.253+0000 7ff2fba9a640 1 -- start start 2024-03-06T07:28:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fba9a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2f40a8290 0x7ff2f41539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fba9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f40c62a0 con 0x7ff2f40c69a0 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fba9a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f40c4c00 con 0x7ff2f40c4740 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fba9a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f4153f00 con 0x7ff2f40a8290 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fa037640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2f40c69a0 0x7ff2f4072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54984/0 (socket says 172.21.15.17:54984) 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.254+0000 7ff2fa037640 1 -- 172.21.15.17:0/2196109997 learned_addr learned my addr 172.21.15.17:0/2196109997 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9836640 1 --2- 172.21.15.17:0/2196109997 >> v2:172.21.15.163:3300/0 conn(0x7ff2f40a8290 0x7ff2f41539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421898851 0 0) 0x7ff2f40c62a0 con 0x7ff2f40c69a0 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e0003520 con 0x7ff2f40c69a0 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3089428872 0 0) 0x7ff2e0003520 con 0x7ff2f40c69a0 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 >> v2:172.21.15.163:3300/0 conn(0x7ff2f40a8290 msgr2=0x7ff2f41539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 --2- 172.21.15.17:0/2196109997 >> v2:172.21.15.163:3300/0 conn(0x7ff2f40a8290 0x7ff2f41539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 >> v1:172.21.15.163:6789/0 conn(0x7ff2f40c4740 legacy=0x7ff2f40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40c4ee0 con 0x7ff2f40c69a0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff2f00019f0 con 0x7ff2f40c69a0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 >> v1:172.21.15.17:6789/0 conn(0x7ff2f40c69a0 legacy=0x7ff2f4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e00039b0 0x7ff2e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 0x7ff2e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2e00052e0 0x7ff2e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f4153f00 con 0x7ff2e0004800 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e000c1b0 con 0x7ff2e00039b0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e000c3b0 con 0x7ff2e00052e0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 0x7ff2e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9836640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e00039b0 0x7ff2e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2fa037640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2e00052e0 0x7ff2e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 -- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2e00052e0 msgr2=0x7ff2e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2e00052e0 0x7ff2e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 -- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e00039b0 msgr2=0x7ff2e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e00039b0 0x7ff2e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2f9035640 1 -- 172.21.15.17:0/2196109997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e0003520 con 0x7ff2e0004800 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2fa037640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2e00052e0 0x7ff2e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.255+0000 7ff2fa037640 1 -- 172.21.15.17:0/2196109997 reap_dead start 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.256+0000 7ff2f9035640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 0x7ff2e0004c80 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4001720 tx=0x7ff2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b576763112bc047d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.256+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4001370 con 0x7ff2e0004800 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.256+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e40052d0 con 0x7ff2e0004800 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.256+0000 7ff2f8834640 1 -- 172.21.15.17:0/2196109997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4003830 con 0x7ff2e0004800 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 -- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 msgr2=0x7ff2e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 0x7ff2e0004c80 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4001720 tx=0x7ff2e0005830 comp rx=0 tx=0).stop 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 -- 172.21.15.17:0/2196109997 shutdown_connections 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/2196109997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2e0004800 0x7ff2e0004c80 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 -- 172.21.15.17:0/2196109997 >> 172.21.15.17:0/2196109997 conn(0x7ff2f40c5560 msgr2=0x7ff2f40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 -- 172.21.15.17:0/2196109997 shutdown_connections 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.257+0000 7ff2fba9a640 1 -- 172.21.15.17:0/2196109997 wait complete. 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.258+0000 7ff2fba9a640 1 Processor -- start 2024-03-06T07:28:12.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.258+0000 7ff2fba9a640 1 -- start start 2024-03-06T07:28:12.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 0x7ff2f40c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41eeec0 0x7ff2f40c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40c6df0 con 0x7ff2f41dfce0 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f406fc80 con 0x7ff2e0004800 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40c6f70 con 0x7ff2f41eeec0 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50726/0 (socket says 172.21.15.17:50726) 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 -- 172.21.15.17:0/698547487 learned_addr learned my addr 172.21.15.17:0/698547487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9836640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 0x7ff2f40c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fa037640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41eeec0 0x7ff2f40c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 -- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41eeec0 msgr2=0x7ff2f40c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41eeec0 0x7ff2f40c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 -- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 msgr2=0x7ff2f40c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 0x7ff2f40c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 -- 172.21.15.17:0/698547487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40c1f90 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9836640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 0x7ff2f40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2f9035640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005040 tx=0x7ff2e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4041070 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e4005480 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f40ae0b0 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4007a30 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.259+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f40aba40 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.260+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f4067ad0 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.261+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2e4044070 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.261+0000 7ff2ea7fc640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 0x7ff2d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.261+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff2e40abc00 con 0x7ff2f41dfce0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.261+0000 7ff2f9836640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 0x7ff2d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.262+0000 7ff2f9836640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 0x7ff2d806d9c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009c40 tx=0x7ff2ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.265+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2f4067ad0 con 0x7ff2f41dfce0 2024-03-06T07:28:12.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.448+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2f4075300 con 0x7ff2f41dfce0 2024-03-06T07:28:12.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.448+0000 7ff2ea7fc640 1 -- 172.21.15.17:0/698547487 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2f4075300 con 0x7ff2f41dfce0 2024-03-06T07:28:12.450 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 msgr2=0x7ff2d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 0x7ff2d806d9c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009c40 tx=0x7ff2ec008040 comp rx=0 tx=0).stop 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 msgr2=0x7ff2f41ee980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005040 tx=0x7ff2e4001120 comp rx=0 tx=0).stop 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 shutdown_connections 2024-03-06T07:28:12.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff2d806b500 0x7ff2d806d9c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41eeec0 0x7ff2f40c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2f41dfce0 0x7ff2f41ee980 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 --2- 172.21.15.17:0/698547487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2e0004800 0x7ff2f40c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 >> 172.21.15.17:0/698547487 conn(0x7ff2f40c5560 msgr2=0x7ff2f4152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 shutdown_connections 2024-03-06T07:28:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.450+0000 7ff2fba9a640 1 -- 172.21.15.17:0/698547487 wait complete. 2024-03-06T07:28:12.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:28:12.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 Processor -- start 2024-03-06T07:28:12.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 -- start start 2024-03-06T07:28:12.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78d40c6000 0x7f78d4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d4153a60 con 0x7f78d4150580 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d4153d40 con 0x7f78d40c1320 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.563+0000 7f78db8ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d406c2c0 con 0x7f78d40c6000 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9e8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f78d40c6000 0x7f78d4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9689640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f78d4150580 0x7f78d406dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54992/0 (socket says 172.21.15.17:54992) 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9689640 1 -- 172.21.15.17:0/2963438865 learned_addr learned my addr 172.21.15.17:0/2963438865 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9e8a640 1 -- 172.21.15.17:0/2963438865 >> v1:172.21.15.163:6789/0 conn(0x7f78d40c1320 legacy=0x7f78d41566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9e8a640 1 -- 172.21.15.17:0/2963438865 >> v1:172.21.15.17:6789/0 conn(0x7f78d4150580 legacy=0x7f78d406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d3fff640 1 -- 172.21.15.17:0/2963438865 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2582340674 0 0) 0x7f78d4153a60 con 0x7f78d4150580 2024-03-06T07:28:12.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.564+0000 7f78d9e8a640 1 -- 172.21.15.17:0/2963438865 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d406c5a0 con 0x7f78d40c6000 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78d9e8a640 1 --2- 172.21.15.17:0/2963438865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d4156dc0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009d90 tx=0x7f78c402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c381733dc167214e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78d3fff640 1 -- 172.21.15.17:0/2963438865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4039070 con 0x7f78d40c6000 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78d3fff640 1 -- 172.21.15.17:0/2963438865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c402cbd0 con 0x7f78d40c6000 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78d3fff640 1 -- 172.21.15.17:0/2963438865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4034440 con 0x7f78d40c6000 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78db8ed640 1 -- 172.21.15.17:0/2963438865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 msgr2=0x7f78d4156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78db8ed640 1 --2- 172.21.15.17:0/2963438865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d4156dc0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009d90 tx=0x7f78c402be20 comp rx=0 tx=0).stop 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78db8ed640 1 -- 172.21.15.17:0/2963438865 shutdown_connections 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78db8ed640 1 --2- 172.21.15.17:0/2963438865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d4156dc0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.565+0000 7f78db8ed640 1 -- 172.21.15.17:0/2963438865 >> 172.21.15.17:0/2963438865 conn(0x7f78d40c6710 msgr2=0x7f78d40c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.566+0000 7f78db8ed640 1 -- 172.21.15.17:0/2963438865 shutdown_connections 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.566+0000 7f78db8ed640 1 -- 172.21.15.17:0/2963438865 wait complete. 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 Processor -- start 2024-03-06T07:28:12.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 -- start start 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78d40c1320 0x7f78d4149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78d4150580 0x7f78d40bfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d406e420 con 0x7f78d40c6000 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4150960 con 0x7f78d4150580 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78db8ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d406e5a0 con 0x7f78d40c1320 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78d8e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78d8e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50742/0 (socket says 172.21.15.17:50742) 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.567+0000 7f78d8e88640 1 -- 172.21.15.17:0/3897970725 learned_addr learned my addr 172.21.15.17:0/3897970725 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 -- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78d40c1320 msgr2=0x7f78d4149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78d40c1320 0x7f78d4149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 -- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78d4150580 msgr2=0x7f78d40bfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78d4150580 0x7f78d40bfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 -- 172.21.15.17:0/3897970725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d41eefd0 con 0x7f78d40c6000 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d8e88640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c430 tx=0x7f78c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8011790 con 0x7f78d40c6000 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c8017070 con 0x7f78d40c6000 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d40a8000 con 0x7f78d40c6000 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8011bb0 con 0x7f78d40c6000 2024-03-06T07:28:12.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.568+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d41565c0 con 0x7f78d40c6000 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.569+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d4067ad0 con 0x7f78d40c6000 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.570+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78c801c460 con 0x7f78d40c6000 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.570+0000 7f78d1ffb640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 0x7f78c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.570+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f78c8089e40 con 0x7f78d40c6000 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.570+0000 7f78d9689640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 0x7f78c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.571+0000 7f78d9689640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 0x7f78c006d9c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007fd0 tx=0x7f78cc007960 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.574+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78d4067ad0 con 0x7f78d40c6000 2024-03-06T07:28:12.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.755+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f78d404f330 con 0x7f78d40c6000 2024-03-06T07:28:12.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.756+0000 7f78d1ffb640 1 -- 172.21.15.17:0/3897970725 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v195) v1 ==== 84+0+65 (secure 0 0 0) 0x7f78c8058f10 con 0x7f78d40c6000 2024-03-06T07:28:12.757 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:28:12.758 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:28:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 msgr2=0x7f78c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 0x7f78c006d9c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007fd0 tx=0x7f78cc007960 comp rx=0 tx=0).stop 2024-03-06T07:28:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 msgr2=0x7f78d41ee600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f78c800c430 tx=0x7f78c800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 shutdown_connections 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f78c006b500 0x7f78c006d9c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.757+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78d4150580 0x7f78d40bfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.758+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78d40c6000 0x7f78d41ee600 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.758+0000 7f78db8ed640 1 --2- 172.21.15.17:0/3897970725 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78d40c1320 0x7f78d4149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.758+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 >> 172.21.15.17:0/3897970725 conn(0x7f78d40c6710 msgr2=0x7f78d40c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.758+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 shutdown_connections 2024-03-06T07:28:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.758+0000 7f78db8ed640 1 -- 172.21.15.17:0/3897970725 wait complete. 2024-03-06T07:28:12.770 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:28:12.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:28:12.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 Processor -- start 2024-03-06T07:28:12.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 -- start start 2024-03-06T07:28:12.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f10500c1d10 0x7f1050156440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10500c2920 0x7f1050156980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1050070430 con 0x7f10500abb60 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10500711f0 con 0x7f10500c2920 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1056f0d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1050070d60 con 0x7f10500c1d10 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f10554aa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10500c2920 0x7f1050156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1054ca9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10500abb60 0x7f10501502f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55002/0 (socket says 172.21.15.17:55002) 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.874+0000 7f1054ca9640 1 -- 172.21.15.17:0/3714282542 learned_addr learned my addr 172.21.15.17:0/3714282542 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f104ffff640 1 --2- 172.21.15.17:0/3714282542 >> v2:172.21.15.163:3300/0 conn(0x7f10500c1d10 0x7f1050156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f10554aa640 1 -- 172.21.15.17:0/3714282542 >> v2:172.21.15.163:3300/0 conn(0x7f10500c1d10 msgr2=0x7f1050156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f10554aa640 1 --2- 172.21.15.17:0/3714282542 >> v2:172.21.15.163:3300/0 conn(0x7f10500c1d10 0x7f1050156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f10554aa640 1 -- 172.21.15.17:0/3714282542 >> v1:172.21.15.17:6789/0 conn(0x7f10500abb60 legacy=0x7f10501502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f10554aa640 1 -- 172.21.15.17:0/3714282542 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050070710 con 0x7f10500c2920 2024-03-06T07:28:12.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f104f7fe640 1 -- 172.21.15.17:0/3714282542 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512992209 0 0) 0x7f1050070430 con 0x7f10500abb60 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.875+0000 7f10554aa640 1 --2- 172.21.15.17:0/3714282542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050156980 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f10500a9690 tx=0x7f104802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0417517b20e541 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f104f7fe640 1 -- 172.21.15.17:0/3714282542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048039070 con 0x7f10500c2920 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f104f7fe640 1 -- 172.21.15.17:0/3714282542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104802cb60 con 0x7f10500c2920 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f104f7fe640 1 -- 172.21.15.17:0/3714282542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1048034440 con 0x7f10500c2920 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 -- 172.21.15.17:0/3714282542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 msgr2=0x7f1050156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3714282542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050156980 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f10500a9690 tx=0x7f104802bdb0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 -- 172.21.15.17:0/3714282542 shutdown_connections 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3714282542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050156980 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3714282542 >> v2:172.21.15.163:3300/0 conn(0x7f10500c1d10 0x7f1050156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 -- 172.21.15.17:0/3714282542 >> 172.21.15.17:0/3714282542 conn(0x7f10500c65b0 msgr2=0x7f10500c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 -- 172.21.15.17:0/3714282542 shutdown_connections 2024-03-06T07:28:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.876+0000 7f1056f0d640 1 -- 172.21.15.17:0/3714282542 wait complete. 2024-03-06T07:28:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.877+0000 7f1056f0d640 1 Processor -- start 2024-03-06T07:28:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.877+0000 7f1056f0d640 1 -- start start 2024-03-06T07:28:12.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 0x7f1050155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10500bb550 0x7f1050149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050150b80 con 0x7f10500c2920 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050150a00 con 0x7f10500abb60 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1056f0d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1050156ec0 con 0x7f10500bb550 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f104ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f104ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50752/0 (socket says 172.21.15.17:50752) 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f104ffff640 1 -- 172.21.15.17:0/3874131742 learned_addr learned my addr 172.21.15.17:0/3874131742 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f1054ca9640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 0x7f1050155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.878+0000 7f10554aa640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10500bb550 0x7f1050149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 -- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10500bb550 msgr2=0x7f1050149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10500bb550 0x7f1050149b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 -- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 msgr2=0x7f1050155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 0x7f1050155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 -- 172.21.15.17:0/3874131742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10501522c0 con 0x7f10500c2920 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f1054ca9640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 0x7f1050155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:12.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104ffff640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d830 tx=0x7f103c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00b6c0 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c019070 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10501525a0 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00ba80 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.879+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10500bc1e0 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.880+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105004f330 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.881+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f103c01e460 con 0x7f10500c2920 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.881+0000 7f104d7fa640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:12.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.881+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f103c08a610 con 0x7f10500c2920 2024-03-06T07:28:12.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.881+0000 7f1054ca9640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:12.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.882+0000 7f1054ca9640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 0x7f103406d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f60 tx=0x7f1044007c30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:12.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:12.885+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f103c0596e0 con 0x7f10500c2920 2024-03-06T07:28:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.065+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f10500a8fb0 con 0x7f10500c2920 2024-03-06T07:28:13.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.446+0000 7f104d7fa640 1 -- 172.21.15.17:0/3874131742 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v196) v1 ==== 138+0+0 (secure 0 0 0) 0x7f103c058fe0 con 0x7f10500c2920 2024-03-06T07:28:13.448 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 msgr2=0x7f103406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 0x7f103406d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1044000f60 tx=0x7f1044007c30 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 msgr2=0x7f1050149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d830 tx=0x7f103c00dcf0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 shutdown_connections 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10500bb550 0x7f1050149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10500c2920 0x7f1050149640 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 --2- 172.21.15.17:0/3874131742 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10500abb60 0x7f1050155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.449+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 >> 172.21.15.17:0/3874131742 conn(0x7f10500c65b0 msgr2=0x7f10500c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.450+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 shutdown_connections 2024-03-06T07:28:13.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.450+0000 7f1056f0d640 1 -- 172.21.15.17:0/3874131742 wait complete. 2024-03-06T07:28:13.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:28:13.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.577+0000 7f0cb9ba2640 1 Processor -- start 2024-03-06T07:28:13.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.577+0000 7f0cb9ba2640 1 -- start start 2024-03-06T07:28:13.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb9ba2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0cb40c0290 0x7f0cb4071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb9ba2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c3f0 0x7f0cb41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb9ba2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb4158cf0 con 0x7f0cb406c010 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb9ba2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb406c880 con 0x7f0cb40c0290 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb9ba2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4155250 con 0x7f0cb406c3f0 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0cb40c0290 0x7f0cb4071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb3fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0cb40c0290 0x7f0cb4071b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:58810/0 (socket says 172.21.15.17:58810) 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.578+0000 7f0cb3fff640 1 -- 172.21.15.17:0/3006490981 learned_addr learned my addr 172.21.15.17:0/3006490981 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb93a1640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c3f0 0x7f0cb41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb3fff640 1 -- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c3f0 msgr2=0x7f0cb41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb3fff640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c3f0 0x7f0cb41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb3fff640 1 -- 172.21.15.17:0/3006490981 >> v1:172.21.15.163:6789/0 conn(0x7f0cb406c010 legacy=0x7f0cb41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb3fff640 1 -- 172.21.15.17:0/3006490981 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4158fd0 con 0x7f0cb40c0290 2024-03-06T07:28:13.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.579+0000 7f0cb37fe640 1 -- 172.21.15.17:0/3006490981 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212192526 0 0) 0x7f0cb4158cf0 con 0x7f0cb406c010 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb3fff640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb4071b80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009870 tx=0x7f0ca802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4467db7aeb566dac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb37fe640 1 -- 172.21.15.17:0/3006490981 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8034070 con 0x7f0cb40c0290 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb37fe640 1 -- 172.21.15.17:0/3006490981 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca8030840 con 0x7f0cb40c0290 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb37fe640 1 -- 172.21.15.17:0/3006490981 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8030c10 con 0x7f0cb40c0290 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 msgr2=0x7f0cb4071b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb4071b80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8009870 tx=0x7f0ca802b980 comp rx=0 tx=0).stop 2024-03-06T07:28:13.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/3006490981 shutdown_connections 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c3f0 0x7f0cb41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/3006490981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb4071b80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/3006490981 >> 172.21.15.17:0/3006490981 conn(0x7f0cb40c3960 msgr2=0x7f0cb40c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/3006490981 shutdown_connections 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.580+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/3006490981 wait complete. 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.581+0000 7f0cb9ba2640 1 Processor -- start 2024-03-06T07:28:13.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 -- start start 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 0x7f0cb4146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb40c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4155c20 con 0x7f0cb406c3f0 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4154eb0 con 0x7f0cb40c0290 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb9ba2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4155da0 con 0x7f0cb406c010 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50768/0 (socket says 172.21.15.17:50768) 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.582+0000 7f0cb3fff640 1 -- 172.21.15.17:0/232251553 learned_addr learned my addr 172.21.15.17:0/232251553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb93a1640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb40c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb8ba0640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 0x7f0cb4146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 -- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 msgr2=0x7f0cb4146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 0x7f0cb4146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 -- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 msgr2=0x7f0cb40c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb40c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 -- 172.21.15.17:0/232251553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb41ea310 con 0x7f0cb406c3f0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb8ba0640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 0x7f0cb4146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb3fff640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802b9c0 tx=0x7f0ca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8034040 con 0x7f0cb406c3f0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca8030640 con 0x7f0cb406c3f0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80423f0 con 0x7f0cb406c3f0 2024-03-06T07:28:13.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb41ea5b0 con 0x7f0cb406c3f0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.583+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb41eab70 con 0x7f0cb406c3f0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.584+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb40693c0 con 0x7f0cb406c3f0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.585+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ca8037070 con 0x7f0cb406c3f0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.585+0000 7f0cb17fa640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 0x7f0ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.585+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0ca80ac350 con 0x7f0cb406c3f0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.585+0000 7f0cb8ba0640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 0x7f0ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:13.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.586+0000 7f0cb8ba0640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 0x7f0ca006d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0cac007fd0 tx=0x7f0cac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:13.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.590+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0ca807b3d0 con 0x7f0cb406c3f0 2024-03-06T07:28:13.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:13.773+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0cb4074a90 con 0x7f0cb406c3f0 2024-03-06T07:28:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.454+0000 7f0cb17fa640 1 -- 172.21.15.17:0/232251553 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v197) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0ca807acd0 con 0x7f0cb406c3f0 2024-03-06T07:28:14.456 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:28:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.457+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 msgr2=0x7f0ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.457+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 0x7f0ca006d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0cac007fd0 tx=0x7f0cac009040 comp rx=0 tx=0).stop 2024-03-06T07:28:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.457+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 msgr2=0x7f0cb41585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.457+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f0ca802b9c0 tx=0x7f0ca80046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 shutdown_connections 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0ca006b4d0 0x7f0ca006d990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cb40c0290 0x7f0cb40c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb406c3f0 0x7f0cb41585d0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 --2- 172.21.15.17:0/232251553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cb406c010 0x7f0cb4146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 >> 172.21.15.17:0/232251553 conn(0x7f0cb40c3960 msgr2=0x7f0cb41533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.458+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 shutdown_connections 2024-03-06T07:28:14.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.459+0000 7f0cb9ba2640 1 -- 172.21.15.17:0/232251553 wait complete. 2024-03-06T07:28:14.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:28:14.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.593+0000 7f07edac2640 1 Processor -- start 2024-03-06T07:28:14.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.593+0000 7f07edac2640 1 -- start start 2024-03-06T07:28:14.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07edac2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07e8158db0 0x7f07e80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07edac2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07edac2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e8155110 con 0x7f07e80c5330 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07edac2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e81559e0 con 0x7f07e8158db0 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07edac2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e80c8f40 con 0x7f07e80c88b0 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07e6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07e8158db0 0x7f07e80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07e77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f07e80c5330 0x7f07e81586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55006/0 (socket says 172.21.15.17:55006) 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07e6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f07e8158db0 0x7f07e80c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50782/0 (socket says 172.21.15.17:50782) 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.594+0000 7f07e77fe640 1 -- 172.21.15.17:0/1131355594 learned_addr learned my addr 172.21.15.17:0/1131355594 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e7fff640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e6ffd640 1 -- 172.21.15.17:0/1131355594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 msgr2=0x7f07e815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e815fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e6ffd640 1 -- 172.21.15.17:0/1131355594 >> v1:172.21.15.17:6789/0 conn(0x7f07e80c5330 legacy=0x7f07e81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e6ffd640 1 -- 172.21.15.17:0/1131355594 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8155390 con 0x7f07e8158db0 2024-03-06T07:28:14.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e67fc640 1 -- 172.21.15.17:0/1131355594 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250309160 0 0) 0x7f07d8001110 con 0x7f07e80c5330 2024-03-06T07:28:14.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.595+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e8158db0 0x7f07e80c5710 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0097a0 tx=0x7f07dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b1e129adb84c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07e67fc640 1 -- 172.21.15.17:0/1131355594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0315a0 con 0x7f07e8158db0 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07e67fc640 1 -- 172.21.15.17:0/1131355594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc037070 con 0x7f07e8158db0 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07e67fc640 1 -- 172.21.15.17:0/1131355594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc031960 con 0x7f07e8158db0 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 -- 172.21.15.17:0/1131355594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e8158db0 msgr2=0x7f07e80c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e8158db0 0x7f07e80c5710 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0097a0 tx=0x7f07dc02b840 comp rx=0 tx=0).stop 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 -- 172.21.15.17:0/1131355594 shutdown_connections 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e815fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 --2- 172.21.15.17:0/1131355594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e8158db0 0x7f07e80c5710 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 -- 172.21.15.17:0/1131355594 >> 172.21.15.17:0/1131355594 conn(0x7f07e80ac560 msgr2=0x7f07e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 -- 172.21.15.17:0/1131355594 shutdown_connections 2024-03-06T07:28:14.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.596+0000 7f07edac2640 1 -- 172.21.15.17:0/1131355594 wait complete. 2024-03-06T07:28:14.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.597+0000 7f07edac2640 1 Processor -- start 2024-03-06T07:28:14.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 -- start start 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 0x7f07e814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e81529d0 con 0x7f07e80c5330 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e806d090 con 0x7f07e814dbc0 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07edac2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8152b50 con 0x7f07e80c88b0 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50790/0 (socket says 172.21.15.17:50790) 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.598+0000 7f07e77fe640 1 -- 172.21.15.17:0/1262486889 learned_addr learned my addr 172.21.15.17:0/1262486889 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e7fff640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 0x7f07e814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 -- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 msgr2=0x7f07e814d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e814d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 -- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 msgr2=0x7f07e814b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 0x7f07e814b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 -- 172.21.15.17:0/1262486889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e814e140 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e7fff640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 0x7f07e814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07e77fe640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c7b0 tx=0x7f07d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8018070 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80072d0 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e814e420 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80047e0 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.599+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e814f760 con 0x7f07e80c5330 2024-03-06T07:28:14.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.600+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e804f330 con 0x7f07e80c5330 2024-03-06T07:28:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.601+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07d801b070 con 0x7f07e80c5330 2024-03-06T07:28:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.601+0000 7f07cffff640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 0x7f07d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.601+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f07d8089590 con 0x7f07e80c5330 2024-03-06T07:28:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.601+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 0x7f07d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:14.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.602+0000 7f07e6ffd640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 0x7f07d406d990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f07dc036000 tx=0x7f07dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:14.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.606+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07e8155110 con 0x7f07e80c5330 2024-03-06T07:28:14.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:14.826+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f07e81ebff0 con 0x7f07e80c5330 2024-03-06T07:28:15.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.456+0000 7f07cffff640 1 -- 172.21.15.17:0/1262486889 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v187) v1 ==== 174+0+0 (secure 0 0 0) 0x7f07d8058370 con 0x7f07e80c5330 2024-03-06T07:28:15.458 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 31 and data pool 32 2024-03-06T07:28:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 msgr2=0x7f07d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 0x7f07d406d990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f07dc036000 tx=0x7f07dc008030 comp rx=0 tx=0).stop 2024-03-06T07:28:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 msgr2=0x7f07e814f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c7b0 tx=0x7f07d800cc70 comp rx=0 tx=0).stop 2024-03-06T07:28:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 shutdown_connections 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07d406b4d0 0x7f07d406d990 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07e814dbc0 0x7f07e814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07e80c88b0 0x7f07e814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.461+0000 7f07edac2640 1 --2- 172.21.15.17:0/1262486889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f07e80c5330 0x7f07e814f220 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.462+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 >> 172.21.15.17:0/1262486889 conn(0x7f07e80ac560 msgr2=0x7f07e8153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.462+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 shutdown_connections 2024-03-06T07:28:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.464+0000 7f07edac2640 1 -- 172.21.15.17:0/1262486889 wait complete. 2024-03-06T07:28:15.498 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 Processor -- start 2024-03-06T07:28:15.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 -- start start 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387c072630 con 0x7f387c06da40 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387c06de20 con 0x7f387c1592d0 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.602+0000 7f388384e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c072910 con 0x7f387c0c6780 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3881deb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f38815ea640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f387c06da40 0x7f387c155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:54262/0 (socket says 172.21.15.17:54262) 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3881deb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:58850/0 (socket says 172.21.15.17:58850) 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:35894/0 (socket says 172.21.15.17:35894) 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f38815ea640 1 -- 172.21.15.17:0/3949580066 learned_addr learned my addr 172.21.15.17:0/3949580066 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:15.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 -- 172.21.15.17:0/3949580066 >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 msgr2=0x7f387c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 --2- 172.21.15.17:0/3949580066 >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 -- 172.21.15.17:0/3949580066 >> v1:172.21.15.163:6789/0 conn(0x7f387c06da40 legacy=0x7f387c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3880de9640 1 -- 172.21.15.17:0/3949580066 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c072bf0 con 0x7f387c0c6780 2024-03-06T07:28:15.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.603+0000 7f3881deb640 1 --2- 172.21.15.17:0/3949580066 >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.604+0000 7f3880de9640 1 --2- 172.21.15.17:0/3949580066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38700097a0 tx=0x7f387002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e60d845d6aab70c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.604+0000 7f387bfff640 1 -- 172.21.15.17:0/3949580066 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700315a0 con 0x7f387c0c6780 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.604+0000 7f387bfff640 1 -- 172.21.15.17:0/3949580066 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870037070 con 0x7f387c0c6780 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.604+0000 7f387bfff640 1 -- 172.21.15.17:0/3949580066 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870031960 con 0x7f387c0c6780 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 -- 172.21.15.17:0/3949580066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 msgr2=0x7f387c158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 --2- 172.21.15.17:0/3949580066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f38700097a0 tx=0x7f387002b840 comp rx=0 tx=0).stop 2024-03-06T07:28:15.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 -- 172.21.15.17:0/3949580066 shutdown_connections 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 --2- 172.21.15.17:0/3949580066 >> v2:172.21.15.163:3300/0 conn(0x7f387c1592d0 0x7f387c0c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 --2- 172.21.15.17:0/3949580066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c0c6780 0x7f387c158d90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 -- 172.21.15.17:0/3949580066 >> 172.21.15.17:0/3949580066 conn(0x7f387c0aa630 msgr2=0x7f387c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 -- 172.21.15.17:0/3949580066 shutdown_connections 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.605+0000 7f388384e640 1 -- 172.21.15.17:0/3949580066 wait complete. 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.606+0000 7f388384e640 1 Processor -- start 2024-03-06T07:28:15.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.606+0000 7f388384e640 1 -- start start 2024-03-06T07:28:15.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.606+0000 7f388384e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c06da40 0x7f387c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.606+0000 7f388384e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.606+0000 7f388384e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c1592d0 0x7f387c0ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f388384e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c3940 con 0x7f387c0c6780 2024-03-06T07:28:15.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f388384e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c37c0 con 0x7f387c1592d0 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f388384e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c0c3ac0 con 0x7f387c06da40 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50802/0 (socket says 172.21.15.17:50802) 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 -- 172.21.15.17:0/510557224 learned_addr learned my addr 172.21.15.17:0/510557224 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f38815ea640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c06da40 0x7f387c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3881deb640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c1592d0 0x7f387c0ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 -- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c06da40 msgr2=0x7f387c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c06da40 0x7f387c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 -- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c1592d0 msgr2=0x7f387c0ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c1592d0 0x7f387c0ab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 -- 172.21.15.17:0/510557224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c0702e0 con 0x7f387c0c6780 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3880de9640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f3870034040 tx=0x7f38700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870032630 con 0x7f387c0c6780 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700327d0 con 0x7f387c0c6780 2024-03-06T07:28:15.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c1ee0a0 con 0x7f387c0c6780 2024-03-06T07:28:15.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.607+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870032bc0 con 0x7f387c0c6780 2024-03-06T07:28:15.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.608+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c0aa030 con 0x7f387c0c6780 2024-03-06T07:28:15.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.609+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c04f330 con 0x7f387c0c6780 2024-03-06T07:28:15.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.609+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f387004e430 con 0x7f387c0c6780 2024-03-06T07:28:15.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.609+0000 7f3879ffb640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 0x7f386806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.610+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f387002f030 con 0x7f387c0c6780 2024-03-06T07:28:15.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.610+0000 7f38815ea640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 0x7f386806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.610+0000 7f38815ea640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 0x7f386806d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3874000f60 tx=0x7f3874009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.614+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3870040070 con 0x7f387c0c6780 2024-03-06T07:28:15.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.796+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f387c073240 con 0x7f387c0c6780 2024-03-06T07:28:15.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.797+0000 7f3879ffb640 1 -- 172.21.15.17:0/510557224 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16965 (secure 0 0 0) 0x7f387007ae90 con 0x7f387c0c6780 2024-03-06T07:28:15.799 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:15.799 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":198,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:14.827315+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:15.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 msgr2=0x7f386806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 0x7f386806d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3874000f60 tx=0x7f3874009040 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 msgr2=0x7f387c0a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f3870034040 tx=0x7f38700046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 shutdown_connections 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f386806b4d0 0x7f386806d990 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f387c1592d0 0x7f387c0ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c0c6780 0x7f387c0a9af0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 --2- 172.21.15.17:0/510557224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f387c06da40 0x7f387c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 >> 172.21.15.17:0/510557224 conn(0x7f387c0aa630 msgr2=0x7f387c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 shutdown_connections 2024-03-06T07:28:15.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.800+0000 7f388384e640 1 -- 172.21.15.17:0/510557224 wait complete. 2024-03-06T07:28:15.813 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:15.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.912+0000 7f90dc684640 1 Processor -- start 2024-03-06T07:28:15.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.912+0000 7f90dc684640 1 -- start start 2024-03-06T07:28:15.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dc684640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d40c6780 0x7f90d4158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dc684640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90d41592d0 0x7f90d40c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dc684640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d4072630 con 0x7f90d406da40 2024-03-06T07:28:15.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dc684640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d406de20 con 0x7f90d41592d0 2024-03-06T07:28:15.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dc684640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d4072910 con 0x7f90d40c6780 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dac21640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90d41592d0 0x7f90d40c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90d9c1f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d40c6780 0x7f90d4158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dac21640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90d41592d0 0x7f90d40c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:58870/0 (socket says 172.21.15.17:58870) 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.913+0000 7f90dac21640 1 -- 172.21.15.17:0/167581378 learned_addr learned my addr 172.21.15.17:0/167581378 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.914+0000 7f90dac21640 1 -- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d40c6780 msgr2=0x7f90d4158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:15.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.914+0000 7f90dac21640 1 --2- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d40c6780 0x7f90d4158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.914+0000 7f90dac21640 1 -- 172.21.15.17:0/167581378 >> v1:172.21.15.163:6789/0 conn(0x7f90d406da40 legacy=0x7f90d4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.914+0000 7f90dac21640 1 -- 172.21.15.17:0/167581378 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d4072bf0 con 0x7f90d41592d0 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.914+0000 7f90dac21640 1 --2- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40c3280 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009e50 tx=0x7f90c402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3223c2ec4b161498 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90d941e640 1 -- 172.21.15.17:0/167581378 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c402ca70 con 0x7f90d41592d0 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90d941e640 1 -- 172.21.15.17:0/167581378 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c402cc10 con 0x7f90d41592d0 2024-03-06T07:28:15.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90d941e640 1 -- 172.21.15.17:0/167581378 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c40343f0 con 0x7f90d41592d0 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 -- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 msgr2=0x7f90d40c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 --2- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40c3280 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009e50 tx=0x7f90c402bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 -- 172.21.15.17:0/167581378 shutdown_connections 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 --2- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40c3280 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 --2- 172.21.15.17:0/167581378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d40c6780 0x7f90d4158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 -- 172.21.15.17:0/167581378 >> 172.21.15.17:0/167581378 conn(0x7f90d40aa630 msgr2=0x7f90d40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 -- 172.21.15.17:0/167581378 shutdown_connections 2024-03-06T07:28:15.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.915+0000 7f90dc684640 1 -- 172.21.15.17:0/167581378 wait complete. 2024-03-06T07:28:15.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.916+0000 7f90dc684640 1 Processor -- start 2024-03-06T07:28:15.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.916+0000 7f90dc684640 1 -- start start 2024-03-06T07:28:15.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d406da40 0x7f90d406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40c3940 con 0x7f90d40c6780 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40c37c0 con 0x7f90d41592d0 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dc684640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d40c3ac0 con 0x7f90d406da40 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90d9c1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90d9c1f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50806/0 (socket says 172.21.15.17:50806) 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90d9c1f640 1 -- 172.21.15.17:0/3440657610 learned_addr learned my addr 172.21.15.17:0/3440657610 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90da420640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d406da40 0x7f90d406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.917+0000 7f90dac21640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 -- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d406da40 msgr2=0x7f90d406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d406da40 0x7f90d406f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 -- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 msgr2=0x7f90d40abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 -- 172.21.15.17:0/3440657610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d41eb800 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d9c1f640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f90c800e8a0 tx=0x7f90c800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8017070 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c800ee90 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c800cb60 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d41de820 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.918+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d406fae0 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.919+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d404f330 con 0x7f90d40c6780 2024-03-06T07:28:15.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.919+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c801a070 con 0x7f90d40c6780 2024-03-06T07:28:15.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.920+0000 7f90d2ffd640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:15.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.920+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90c808a480 con 0x7f90d40c6780 2024-03-06T07:28:15.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.920+0000 7f90da420640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:15.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.920+0000 7f90da420640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f90d40c5430 tx=0x7f90cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:15.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:15.924+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f90c8058bb0 con 0x7f90d40c6780 2024-03-06T07:28:16.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.106+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d4073240 con 0x7f90d40c6780 2024-03-06T07:28:16.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.107+0000 7f90d2ffd640 1 -- 172.21.15.17:0/3440657610 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16965 (secure 0 0 0) 0x7f90c805cc50 con 0x7f90d40c6780 2024-03-06T07:28:16.109 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:16.109 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":198,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:14.827315+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 msgr2=0x7f90c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 0x7f90c006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f90d40c5430 tx=0x7f90cc009040 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 msgr2=0x7f90d40ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f90c800e8a0 tx=0x7f90c800ed60 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 shutdown_connections 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f90c006b4d0 0x7f90c006d990 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90d41592d0 0x7f90d40abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90d40c6780 0x7f90d40ab9b0 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 --2- 172.21.15.17:0/3440657610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90d406da40 0x7f90d406f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 >> 172.21.15.17:0/3440657610 conn(0x7f90d40aa630 msgr2=0x7f90d4154870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 shutdown_connections 2024-03-06T07:28:16.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.109+0000 7f90dc684640 1 -- 172.21.15.17:0/3440657610 wait complete. 2024-03-06T07:28:16.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:28:16.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 Processor -- start 2024-03-06T07:28:16.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 -- start start 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f390c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c152cb0 con 0x7f390c0c67f0 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c0c38b0 con 0x7f390c072a10 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.224+0000 7f39116c3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1594f0 con 0x7f390c1528d0 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f390b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f390c0c67f0 0x7f390c15fac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55012/0 (socket says 172.21.15.17:55012) 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f390b7fe640 1 -- 172.21.15.17:0/3546239712 learned_addr learned my addr 172.21.15.17:0/3546239712 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061963576 0 0) 0x7f390c152cb0 con 0x7f390c0c67f0 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f4003520 con 0x7f390c0c67f0 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f390affd640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f390c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495137156 0 0) 0x7f38f4003520 con 0x7f390c0c67f0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 msgr2=0x7f390c0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f390c0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 >> v1:172.21.15.163:6789/0 conn(0x7f390c072a10 legacy=0x7f390c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0a7f70 con 0x7f390c0c67f0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f39040019f0 con 0x7f390c0c67f0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 >> v1:172.21.15.17:6789/0 conn(0x7f390c0c67f0 legacy=0x7f390c15fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f40039e0 0x7f38f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f38f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38f4004d40 0x7f38f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1594f0 con 0x7f38f4004d40 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f400bbe0 con 0x7f38f40039e0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f400bdb0 con 0x7f390c1528d0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f390b7fe640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38f4004d40 0x7f38f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.225+0000 7f390affd640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f40039e0 0x7f38f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390a7fc640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f38f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 -- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 msgr2=0x7f38f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f38f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 -- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f40039e0 msgr2=0x7f38f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f40039e0 0x7f38f4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 -- 172.21.15.17:0/3546239712 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4003520 con 0x7f38f4004d40 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390a7fc640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c1528d0 0x7f38f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.226+0000 7f390b7fe640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38f4004d40 0x7f38f40051c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f390400a880 tx=0x7f38f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9473210dd53645 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390400a4a0 con 0x7f38f4004d40 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904034650 con 0x7f38f4004d40 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f3909ffb640 1 -- 172.21.15.17:0/3546239712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904034a40 con 0x7f38f4004d40 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 -- 172.21.15.17:0/3546239712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38f4004d40 msgr2=0x7f38f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 --2- 172.21.15.17:0/3546239712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38f4004d40 0x7f38f40051c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f390400a880 tx=0x7f38f4005830 comp rx=0 tx=0).stop 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f390b7fe640 1 -- 172.21.15.17:0/3546239712 reap_dead start 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 -- 172.21.15.17:0/3546239712 shutdown_connections 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 -- 172.21.15.17:0/3546239712 >> 172.21.15.17:0/3546239712 conn(0x7f390c0aa6a0 msgr2=0x7f390c0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 -- 172.21.15.17:0/3546239712 shutdown_connections 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.227+0000 7f39116c3640 1 -- 172.21.15.17:0/3546239712 wait complete. 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.228+0000 7f39116c3640 1 Processor -- start 2024-03-06T07:28:16.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.228+0000 7f39116c3640 1 -- start start 2024-03-06T07:28:16.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 0x7f390c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c148200 0x7f390c1505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c1591a0 con 0x7f390c151620 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0a8250 con 0x7f38f4004d40 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f39116c3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c0c4ef0 con 0x7f390c148200 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f390a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f390a7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50814/0 (socket says 172.21.15.17:50814) 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f390a7fc640 1 -- 172.21.15.17:0/1884083149 learned_addr learned my addr 172.21.15.17:0/1884083149 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f390affd640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 0x7f390c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.229+0000 7f390b7fe640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c148200 0x7f390c1505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 -- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c148200 msgr2=0x7f390c1505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c148200 0x7f390c1505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 -- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 msgr2=0x7f390c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 0x7f390c1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 -- 172.21.15.17:0/1884083149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c147be0 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390affd640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 0x7f390c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f390a7fc640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f390000e630 tx=0x7f390000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000052e0 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900018070 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c147e50 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900003810 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.230+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c1520c0 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.231+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c14cfc0 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.231+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39000039b0 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.232+0000 7f38f37fe640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 0x7f38f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.232+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f390008a040 con 0x7f390c151620 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.232+0000 7f390affd640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 0x7f38f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.232+0000 7f390affd640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 0x7f38f806d9c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f390c151c50 tx=0x7f38fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.236+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f390c14cfc0 con 0x7f390c151620 2024-03-06T07:28:16.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.456+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f390c075300 con 0x7f390c151620 2024-03-06T07:28:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.468+0000 7f38f37fe640 1 -- 172.21.15.17:0/1884083149 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v188) v1 ==== 118+0+0 (secure 0 0 0) 0x7f390c075300 con 0x7f390c151620 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 msgr2=0x7f38f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 0x7f38f806d9c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f390c151c50 tx=0x7f38fc008040 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 msgr2=0x7f390c150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f390000e630 tx=0x7f390000eaf0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 shutdown_connections 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f38f806b500 0x7f38f806d9c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390c148200 0x7f390c1505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c151620 0x7f390c150080 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 --2- 172.21.15.17:0/1884083149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f4004d40 0x7f390c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 >> 172.21.15.17:0/1884083149 conn(0x7f390c0aa6a0 msgr2=0x7f390c0c6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 shutdown_connections 2024-03-06T07:28:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.470+0000 7f39116c3640 1 -- 172.21.15.17:0/1884083149 wait complete. 2024-03-06T07:28:16.484 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:28:16.484 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:28:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.619+0000 7fe74b116640 1 Processor -- start 2024-03-06T07:28:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe74b116640 1 -- start start 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe74b116640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe74406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe74b116640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7440b0450 con 0x7fe7440b0070 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe74b116640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7441560e0 con 0x7fe7440c5380 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe74b116640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74406cfb0 con 0x7fe744155c30 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe748eb2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe7440b0070 0x7fe7440c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55014/0 (socket says 172.21.15.17:55014) 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.620+0000 7fe748eb2640 1 -- 172.21.15.17:0/4111474484 learned_addr learned my addr 172.21.15.17:0/4111474484 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe743fff640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe74406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265751641 0 0) 0x7fe738001110 con 0x7fe7440b0070 2024-03-06T07:28:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe734003520 con 0x7fe7440b0070 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 683671580 0 0) 0x7fe734003520 con 0x7fe7440b0070 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 msgr2=0x7fe74406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe74406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 >> v1:172.21.15.163:6789/0 conn(0x7fe7440c5380 legacy=0x7fe74414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440c77f0 con 0x7fe7440b0070 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe738001840 con 0x7fe7440b0070 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 >> v1:172.21.15.17:6789/0 conn(0x7fe7440b0070 legacy=0x7fe7440c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7340039e0 0x7fe734003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe734004390 0x7fe734004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe734005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74406cfb0 con 0x7fe734004390 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73400baa0 con 0x7fe7340039e0 2024-03-06T07:28:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73400bca0 con 0x7fe744155c30 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe743fff640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe734004390 0x7fe734004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe748eb2640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7340039e0 0x7fe734003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.621+0000 7fe7496b3640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe734005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 -- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 msgr2=0x7fe734005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe744155c30 0x7fe734005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 -- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7340039e0 msgr2=0x7fe734003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7340039e0 0x7fe734003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 -- 172.21.15.17:0/4111474484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe734003520 con 0x7fe734004390 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe743fff640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe734004390 0x7fe734004860 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975e9adcaa9a41b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730039070 con 0x7fe734004390 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73002c6b0 con 0x7fe734004390 2024-03-06T07:28:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.622+0000 7fe7437fe640 1 -- 172.21.15.17:0/4111474484 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730009690 con 0x7fe734004390 2024-03-06T07:28:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 -- 172.21.15.17:0/4111474484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe734004390 msgr2=0x7fe734004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 --2- 172.21.15.17:0/4111474484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe734004390 0x7fe734004860 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ac0 tx=0x7fe734005830 comp rx=0 tx=0).stop 2024-03-06T07:28:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe7496b3640 1 -- 172.21.15.17:0/4111474484 reap_dead start 2024-03-06T07:28:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 -- 172.21.15.17:0/4111474484 shutdown_connections 2024-03-06T07:28:16.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 -- 172.21.15.17:0/4111474484 >> 172.21.15.17:0/4111474484 conn(0x7fe7440ac590 msgr2=0x7fe7440ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 -- 172.21.15.17:0/4111474484 shutdown_connections 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.623+0000 7fe74b116640 1 -- 172.21.15.17:0/4111474484 wait complete. 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.624+0000 7fe74b116640 1 Processor -- start 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.624+0000 7fe74b116640 1 -- start start 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734004390 0x7fe7440c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7440bf370 0x7fe74414f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74406c270 con 0x7fe744156420 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74406c020 con 0x7fe734004390 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe74b116640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74406c550 con 0x7fe7440bf370 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50828/0 (socket says 172.21.15.17:50828) 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 -- 172.21.15.17:0/2388461634 learned_addr learned my addr 172.21.15.17:0/2388461634 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe7496b3640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7440bf370 0x7fe74414f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe748eb2640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734004390 0x7fe7440c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 -- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7440bf370 msgr2=0x7fe74414f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7440bf370 0x7fe74414f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 -- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734004390 msgr2=0x7fe7440c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734004390 0x7fe7440c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.625+0000 7fe743fff640 1 -- 172.21.15.17:0/2388461634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7440c2930 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe743fff640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fe730000c00 tx=0x7fe730004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730042020 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73003a040 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7440bece0 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe730039070 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.626+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7441ef160 con 0x7fe744156420 2024-03-06T07:28:16.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.627+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7440af6a0 con 0x7fe744156420 2024-03-06T07:28:16.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.627+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe730045020 con 0x7fe744156420 2024-03-06T07:28:16.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.628+0000 7fe7417fa640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 0x7fe72806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:16.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.628+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7300ab590 con 0x7fe744156420 2024-03-06T07:28:16.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.628+0000 7fe748eb2640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 0x7fe72806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:16.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.628+0000 7fe748eb2640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 0x7fe72806d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe738000ed0 tx=0x7fe738009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:16.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.636+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe7440af6a0 con 0x7fe744156420 2024-03-06T07:28:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:16.852+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe7440ad900 con 0x7fe744156420 2024-03-06T07:28:17.443 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:28:17.442+0000 7f2b01baf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:17.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.475+0000 7fe7417fa640 1 -- 172.21.15.17:0/2388461634 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v189) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7440ad900 con 0x7fe744156420 2024-03-06T07:28:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 msgr2=0x7fe72806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 0x7fe72806d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe738000ed0 tx=0x7fe738009040 comp rx=0 tx=0).stop 2024-03-06T07:28:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 msgr2=0x7fe7441ee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fe730000c00 tx=0x7fe730004730 comp rx=0 tx=0).stop 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 shutdown_connections 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe72806b4d0 0x7fe72806d990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7440bf370 0x7fe74414f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe744156420 0x7fe7441ee640 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 --2- 172.21.15.17:0/2388461634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe734004390 0x7fe7440c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.477+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 >> 172.21.15.17:0/2388461634 conn(0x7fe7440ac590 msgr2=0x7fe7440aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.478+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 shutdown_connections 2024-03-06T07:28:17.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.478+0000 7fe74b116640 1 -- 172.21.15.17:0/2388461634 wait complete. 2024-03-06T07:28:17.490 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:28:17.490 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.589+0000 7f5e31e6b640 1 Processor -- start 2024-03-06T07:28:17.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e31e6b640 1 -- start start 2024-03-06T07:28:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e31e6b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e2c1506b0 0x7f5e2c153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e31e6b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c070aa0 con 0x7f5e2c156ea0 2024-03-06T07:28:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e31e6b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e2c070d80 con 0x7f5e2c0acc00 2024-03-06T07:28:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e31e6b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c156450 con 0x7f5e2c1506b0 2024-03-06T07:28:17.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e2b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e2c1506b0 0x7f5e2c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e2affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5e2c156ea0 0x7f5e2c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55028/0 (socket says 172.21.15.17:55028) 2024-03-06T07:28:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.590+0000 7f5e2affd640 1 -- 172.21.15.17:0/3329729540 learned_addr learned my addr 172.21.15.17:0/3329729540 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.591+0000 7f5e2b7fe640 1 -- 172.21.15.17:0/3329729540 >> v1:172.21.15.163:6789/0 conn(0x7f5e2c0acc00 legacy=0x7f5e2c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.591+0000 7f5e2b7fe640 1 -- 172.21.15.17:0/3329729540 >> v1:172.21.15.17:6789/0 conn(0x7f5e2c156ea0 legacy=0x7f5e2c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.591+0000 7f5e2b7fe640 1 -- 172.21.15.17:0/3329729540 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c1566a0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.591+0000 7f5e2b7fe640 1 --2- 172.21.15.17:0/3329729540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c153990 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009d90 tx=0x7f5e1c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac261f6f4548328 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e2a7fc640 1 -- 172.21.15.17:0/3329729540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c039070 con 0x7f5e2c1506b0 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e2a7fc640 1 -- 172.21.15.17:0/3329729540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c02cc60 con 0x7f5e2c1506b0 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e2a7fc640 1 -- 172.21.15.17:0/3329729540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c034660 con 0x7f5e2c1506b0 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 -- 172.21.15.17:0/3329729540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 msgr2=0x7f5e2c153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/3329729540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c153990 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009d90 tx=0x7f5e1c02beb0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 -- 172.21.15.17:0/3329729540 shutdown_connections 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/3329729540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c153990 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 -- 172.21.15.17:0/3329729540 >> 172.21.15.17:0/3329729540 conn(0x7f5e2c06b1a0 msgr2=0x7f5e2c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 -- 172.21.15.17:0/3329729540 shutdown_connections 2024-03-06T07:28:17.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.592+0000 7f5e31e6b640 1 -- 172.21.15.17:0/3329729540 wait complete. 2024-03-06T07:28:17.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.593+0000 7f5e31e6b640 1 Processor -- start 2024-03-06T07:28:17.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.593+0000 7f5e31e6b640 1 -- start start 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e2c0acc00 0x7f5e2c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e2c156ea0 0x7f5e2c13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ad050 con 0x7f5e2c1506b0 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c156a10 con 0x7f5e2c0acc00 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e31e6b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ad1d0 con 0x7f5e2c156ea0 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50848/0 (socket says 172.21.15.17:50848) 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2affd640 1 -- 172.21.15.17:0/725777659 learned_addr learned my addr 172.21.15.17:0/725777659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2b7fe640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e2c0acc00 0x7f5e2c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2bfff640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e2c156ea0 0x7f5e2c13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.594+0000 7f5e2affd640 1 -- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e2c156ea0 msgr2=0x7f5e2c13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e2affd640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e2c156ea0 0x7f5e2c13de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e2affd640 1 -- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e2c0acc00 msgr2=0x7f5e2c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e2affd640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e2c0acc00 0x7f5e2c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e2affd640 1 -- 172.21.15.17:0/725777659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0c7840 con 0x7f5e2c1506b0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e2affd640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c440 tx=0x7f5e2000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20017070 con 0x7f5e2c1506b0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20005ce0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200072f0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0c7b20 con 0x7f5e2c1506b0 2024-03-06T07:28:17.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.595+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0c80e0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.596+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2c04f330 con 0x7f5e2c1506b0 2024-03-06T07:28:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.596+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2001a070 con 0x7f5e2c1506b0 2024-03-06T07:28:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.597+0000 7f5e13fff640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 0x7f5e1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.597+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e200896b0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.597+0000 7f5e2b7fe640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 0x7f5e1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.597+0000 7f5e2b7fe640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 0x7f5e1806d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c036000 tx=0x7f5e1c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.601+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5e200583a0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.820+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e2c0c2da0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.821+0000 7f5e13fff640 1 -- 172.21.15.17:0/725777659 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+4996 (secure 0 0 0) 0x7f5e20057ca0 con 0x7f5e2c1506b0 2024-03-06T07:28:17.822 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:17.822 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":189,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:16.854170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10443":{"gid":10443,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2666237101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2666237101},{"type":"v1","addr":"172.21.15.17:6835","nonce":2666237101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10443,"qdb_cluster":[10443]},"id":15}]} 2024-03-06T07:28:17.823 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 189 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 msgr2=0x7f5e1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 0x7f5e1806d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c036000 tx=0x7f5e1c030000 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 msgr2=0x7f5e2c13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c440 tx=0x7f5e2000c900 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 shutdown_connections 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1806b4d0 0x7f5e1806d990 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e2c156ea0 0x7f5e2c13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e2c1506b0 0x7f5e2c13d8f0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 --2- 172.21.15.17:0/725777659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e2c0acc00 0x7f5e2c155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 >> 172.21.15.17:0/725777659 conn(0x7f5e2c06b1a0 msgr2=0x7f5e2c0ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 shutdown_connections 2024-03-06T07:28:17.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.823+0000 7f5e31e6b640 1 -- 172.21.15.17:0/725777659 wait complete. 2024-03-06T07:28:17.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:17.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.942+0000 7f9d2a8c2640 1 Processor -- start 2024-03-06T07:28:17.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.942+0000 7f9d2a8c2640 1 -- start start 2024-03-06T07:28:17.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d2a8c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d2406b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d2a8c2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d240c25c0 0x7f9d24153890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d2a8c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d24153dd0 con 0x7f9d240c20b0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d2a8c2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240a9530 con 0x7f9d240c25c0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d2a8c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240a9810 con 0x7f9d2406afc0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d237fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9d240c20b0 0x7f9d24153180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55040/0 (socket says 172.21.15.17:55040) 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.943+0000 7f9d237fe640 1 -- 172.21.15.17:0/1188871275 learned_addr learned my addr 172.21.15.17:0/1188871275 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d2406b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d28e5f640 1 --2- 172.21.15.17:0/1188871275 >> v2:172.21.15.163:3300/0 conn(0x7f9d240c25c0 0x7f9d24153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458930889 0 0) 0x7f9d24153dd0 con 0x7f9d240c20b0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d0c003520 con 0x7f9d240c20b0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3812697772 0 0) 0x7f9d0c003520 con 0x7f9d240c20b0 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 msgr2=0x7f9d2406b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d2406b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 >> v2:172.21.15.163:3300/0 conn(0x7f9d240c25c0 msgr2=0x7f9d24153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 --2- 172.21.15.17:0/1188871275 >> v2:172.21.15.163:3300/0 conn(0x7f9d240c25c0 0x7f9d24153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240a9af0 con 0x7f9d240c20b0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9d14001670 con 0x7f9d240c20b0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 >> v1:172.21.15.17:6789/0 conn(0x7f9d240c20b0 legacy=0x7f9d24153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d0c004800 0x7f9d0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240a9530 con 0x7f9d0c0039b0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240a9810 con 0x7f9d0c004800 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c00ba40 con 0x7f9d2406afc0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d237fe640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d0c004800 0x7f9d0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d28e5f640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 -- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 msgr2=0x7f9d0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d2406afc0 0x7f9d0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 -- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d0c004800 msgr2=0x7f9d0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d0c004800 0x7f9d0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.944+0000 7f9d23fff640 1 -- 172.21.15.17:0/1188871275 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c003520 con 0x7f9d0c0039b0 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.945+0000 7f9d23fff640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d0c003dd0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009ac0 tx=0x7f9d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a075dfd6b3a02949 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.945+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18039070 con 0x7f9d0c0039b0 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.945+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1802c6b0 con 0x7f9d0c0039b0 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.945+0000 7f9d22ffd640 1 -- 172.21.15.17:0/1188871275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18009690 con 0x7f9d0c0039b0 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/1188871275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 msgr2=0x7f9d0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/1188871275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d0c003dd0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009ac0 tx=0x7f9d0c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d28e5f640 1 -- 172.21.15.17:0/1188871275 reap_dead start 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/1188871275 shutdown_connections 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/1188871275 >> 172.21.15.17:0/1188871275 conn(0x7f9d240c76b0 msgr2=0x7f9d240c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/1188871275 shutdown_connections 2024-03-06T07:28:17.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.946+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/1188871275 wait complete. 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.947+0000 7f9d2a8c2640 1 Processor -- start 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.947+0000 7f9d2a8c2640 1 -- start start 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 0x7f9d240accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 0x7f9d240ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24150980 con 0x7f9d0c0039b0 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240aa0b0 con 0x7f9d240c5cb0 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d2a8c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24150b00 con 0x7f9d240c6130 2024-03-06T07:28:17.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50866/0 (socket says 172.21.15.17:50866) 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 -- 172.21.15.17:0/4234678340 learned_addr learned my addr 172.21.15.17:0/4234678340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d28e5f640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 0x7f9d240ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d237fe640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 0x7f9d240accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 -- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 msgr2=0x7f9d240ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 0x7f9d240ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 -- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 msgr2=0x7f9d240accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 0x7f9d240accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 -- 172.21.15.17:0/4234678340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240aa760 con 0x7f9d0c0039b0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d28e5f640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 0x7f9d240ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d237fe640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 0x7f9d240accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.948+0000 7f9d23fff640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.949+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18042020 con 0x7f9d0c0039b0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.949+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1803a040 con 0x7f9d0c0039b0 2024-03-06T07:28:17.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.949+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2414cfe0 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.949+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d18039070 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.949+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d240ac840 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.950+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2404f330 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.950+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d18045020 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.951+0000 7f9d20ff9640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 0x7f9d1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.951+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d180ab830 con 0x7f9d0c0039b0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.951+0000 7f9d237fe640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 0x7f9d1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:17.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.951+0000 7f9d237fe640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 0x7f9d1006d9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9d14001120 tx=0x7f9d14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:17.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:17.955+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d1804c070 con 0x7f9d0c0039b0 2024-03-06T07:28:18.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.142+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d240c1a50 con 0x7f9d0c0039b0 2024-03-06T07:28:18.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.143+0000 7f9d20ff9640 1 -- 172.21.15.17:0/4234678340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16952 (secure 0 0 0) 0x7f9d1807a520 con 0x7f9d0c0039b0 2024-03-06T07:28:18.145 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:18.145 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":199,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:16.467861+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 msgr2=0x7f9d1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 0x7f9d1006d9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9d14001120 tx=0x7f9d14009040 comp rx=0 tx=0).stop 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 msgr2=0x7f9d240ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18004730 comp rx=0 tx=0).stop 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 shutdown_connections 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9d1006b500 0x7f9d1006d9c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d240c6130 0x7f9d240ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d240c5cb0 0x7f9d240accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 --2- 172.21.15.17:0/4234678340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d0c0039b0 0x7f9d240ab3e0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 >> 172.21.15.17:0/4234678340 conn(0x7f9d240c76b0 msgr2=0x7f9d24149cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 shutdown_connections 2024-03-06T07:28:18.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.146+0000 7f9d2a8c2640 1 -- 172.21.15.17:0/4234678340 wait complete. 2024-03-06T07:28:18.159 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:28:18.159 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.262+0000 7fe477295640 1 Processor -- start 2024-03-06T07:28:18.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.262+0000 7fe477295640 1 -- start start 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe477295640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe47014f790 0x7fe4701529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe477295640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe470155a40 0x7fe470155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe477295640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe470156410 con 0x7fe4700c1aa0 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe477295640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4700c2680 con 0x7fe470155a40 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe477295640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c28d0 con 0x7fe47014f790 2024-03-06T07:28:18.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe475832640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe470155a40 0x7fe470155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe475832640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe470155a40 0x7fe470155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50876/0 (socket says 172.21.15.17:50876) 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe475832640 1 -- 172.21.15.17:0/2810791884 learned_addr learned my addr 172.21.15.17:0/2810791884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe475832640 1 -- 172.21.15.17:0/2810791884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe47014f790 msgr2=0x7fe4701529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.263+0000 7fe475832640 1 --2- 172.21.15.17:0/2810791884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe47014f790 0x7fe4701529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe475832640 1 -- 172.21.15.17:0/2810791884 >> v1:172.21.15.163:6789/0 conn(0x7fe4700c1aa0 legacy=0x7fe4700c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe475832640 1 -- 172.21.15.17:0/2810791884 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4700c2bb0 con 0x7fe470155a40 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe475832640 1 --2- 172.21.15.17:0/2810791884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe470155a40 0x7fe470155ed0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fe468009e50 tx=0x7fe46802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2423b45eb800dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe463fff640 1 -- 172.21.15.17:0/2810791884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46802ca70 con 0x7fe470155a40 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe463fff640 1 -- 172.21.15.17:0/2810791884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46802cc10 con 0x7fe470155a40 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.264+0000 7fe463fff640 1 -- 172.21.15.17:0/2810791884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4680343f0 con 0x7fe470155a40 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 -- 172.21.15.17:0/2810791884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe470155a40 msgr2=0x7fe470155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 --2- 172.21.15.17:0/2810791884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe470155a40 0x7fe470155ed0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fe468009e50 tx=0x7fe46802bed0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 -- 172.21.15.17:0/2810791884 shutdown_connections 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 --2- 172.21.15.17:0/2810791884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe470155a40 0x7fe470155ed0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 --2- 172.21.15.17:0/2810791884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe47014f790 0x7fe4701529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 -- 172.21.15.17:0/2810791884 >> 172.21.15.17:0/2810791884 conn(0x7fe4700c6710 msgr2=0x7fe4700c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 -- 172.21.15.17:0/2810791884 shutdown_connections 2024-03-06T07:28:18.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.265+0000 7fe477295640 1 -- 172.21.15.17:0/2810791884 wait complete. 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.266+0000 7fe477295640 1 Processor -- start 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.266+0000 7fe477295640 1 -- start start 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4700c1aa0 0x7fe4700bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 0x7fe4701eb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c0d60 con 0x7fe47014f790 2024-03-06T07:28:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe470155520 con 0x7fe4700c1aa0 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe477295640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700c5e30 con 0x7fe470155a40 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50884/0 (socket says 172.21.15.17:50884) 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 -- 172.21.15.17:0/4007609284 learned_addr learned my addr 172.21.15.17:0/4007609284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe475031640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4700c1aa0 0x7fe4700bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe475832640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 0x7fe4701eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 -- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 msgr2=0x7fe4701eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 0x7fe4701eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 -- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4700c1aa0 msgr2=0x7fe4700bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4700c1aa0 0x7fe4700bfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 -- 172.21.15.17:0/4007609284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4701ea820 con 0x7fe47014f790 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe475832640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 0x7fe4701eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.267+0000 7fe474830640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c410 tx=0x7fe46c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.268+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c017070 con 0x7fe47014f790 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.268+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c0035f0 con 0x7fe47014f790 2024-03-06T07:28:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.268+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4701eaac0 con 0x7fe47014f790 2024-03-06T07:28:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.268+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c003a20 con 0x7fe47014f790 2024-03-06T07:28:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.268+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4701eb080 con 0x7fe47014f790 2024-03-06T07:28:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.269+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47004f330 con 0x7fe47014f790 2024-03-06T07:28:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.269+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe46c01a070 con 0x7fe47014f790 2024-03-06T07:28:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.270+0000 7fe461ffb640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.270+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe46c0896f0 con 0x7fe47014f790 2024-03-06T07:28:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.270+0000 7fe475031640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.271+0000 7fe475031640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 0x7fe45406d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe464009990 tx=0x7fe464009110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.274+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe46c058470 con 0x7fe47014f790 2024-03-06T07:28:18.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.452+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe470155e50 con 0x7fe45406b4d0 2024-03-06T07:28:18.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.459+0000 7fe461ffb640 1 -- 172.21.15.17:0/4007609284 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7fe470155e50 con 0x7fe45406b4d0 2024-03-06T07:28:18.463 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:18.465 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:28:18.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 msgr2=0x7fe45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 0x7fe45406d990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fe464009990 tx=0x7fe464009110 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 msgr2=0x7fe4700c00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c410 tx=0x7fe46c00c810 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 shutdown_connections 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.461+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe45406b4d0 0x7fe45406d990 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe470155a40 0x7fe4701eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47014f790 0x7fe4700c00c0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 --2- 172.21.15.17:0/4007609284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4700c1aa0 0x7fe4700bfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 >> 172.21.15.17:0/4007609284 conn(0x7fe4700c6710 msgr2=0x7fe4700c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 shutdown_connections 2024-03-06T07:28:18.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.462+0000 7fe477295640 1 -- 172.21.15.17:0/4007609284 wait complete. 2024-03-06T07:28:18.474 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-03-06T07:28:17.453109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281816,"kb_used_data":19880,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754692904,"statfs":{"total":773094113280,"available":772805533696,"internally_reserved":0,"allocated":20357120,"data_stored":11032284,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":268095028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.004368"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952947+0000","last_change":"2024-03-06T07:28:15.482261+0000","last_active":"2024-03-06T07:28:16.952947+0000","last_peered":"2024-03-06T07:28:16.952947+0000","last_clean":"2024-03-06T07:28:16.952947+0000","last_became_active":"2024-03-06T07:28:15.482000+0000","last_became_peered":"2024-03-06T07:28:15.482000+0000","last_unstale":"2024-03-06T07:28:16.952947+0000","last_undegraded":"2024-03-06T07:28:16.952947+0000","last_fullsized":"2024-03-06T07:28:16.952947+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:20.675073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953057+0000","last_change":"2024-03-06T07:28:15.486936+0000","last_active":"2024-03-06T07:28:16.953057+0000","last_peered":"2024-03-06T07:28:16.953057+0000","last_clean":"2024-03-06T07:28:16.953057+0000","last_became_active":"2024-03-06T07:28:15.486503+0000","last_became_peered":"2024-03-06T07:28:15.486503+0000","last_unstale":"2024-03-06T07:28:16.953057+0000","last_undegraded":"2024-03-06T07:28:16.953057+0000","last_fullsized":"2024-03-06T07:28:16.953057+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:50.968467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473978+0000","last_change":"2024-03-06T07:28:15.475549+0000","last_active":"2024-03-06T07:28:16.473978+0000","last_peered":"2024-03-06T07:28:16.473978+0000","last_clean":"2024-03-06T07:28:16.473978+0000","last_became_active":"2024-03-06T07:28:15.475253+0000","last_became_peered":"2024-03-06T07:28:15.475253+0000","last_unstale":"2024-03-06T07:28:16.473978+0000","last_undegraded":"2024-03-06T07:28:16.473978+0000","last_fullsized":"2024-03-06T07:28:16.473978+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:09.652727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473962+0000","last_change":"2024-03-06T07:28:15.477564+0000","last_active":"2024-03-06T07:28:16.473962+0000","last_peered":"2024-03-06T07:28:16.473962+0000","last_clean":"2024-03-06T07:28:16.473962+0000","last_became_active":"2024-03-06T07:28:15.477255+0000","last_became_peered":"2024-03-06T07:28:15.477255+0000","last_unstale":"2024-03-06T07:28:16.473962+0000","last_undegraded":"2024-03-06T07:28:16.473962+0000","last_fullsized":"2024-03-06T07:28:16.473962+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:02.439995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477064+0000","last_change":"2024-03-06T07:28:15.472674+0000","last_active":"2024-03-06T07:28:16.477064+0000","last_peered":"2024-03-06T07:28:16.477064+0000","last_clean":"2024-03-06T07:28:16.477064+0000","last_became_active":"2024-03-06T07:28:15.472254+0000","last_became_peered":"2024-03-06T07:28:15.472254+0000","last_unstale":"2024-03-06T07:28:16.477064+0000","last_undegraded":"2024-03-06T07:28:16.477064+0000","last_fullsized":"2024-03-06T07:28:16.477064+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:32.366142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952829+0000","last_change":"2024-03-06T07:28:15.487583+0000","last_active":"2024-03-06T07:28:16.952829+0000","last_peered":"2024-03-06T07:28:16.952829+0000","last_clean":"2024-03-06T07:28:16.952829+0000","last_became_active":"2024-03-06T07:28:15.487324+0000","last_became_peered":"2024-03-06T07:28:15.487324+0000","last_unstale":"2024-03-06T07:28:16.952829+0000","last_undegraded":"2024-03-06T07:28:16.952829+0000","last_fullsized":"2024-03-06T07:28:16.952829+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:53.553871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952301+0000","last_change":"2024-03-06T07:28:15.487249+0000","last_active":"2024-03-06T07:28:16.952301+0000","last_peered":"2024-03-06T07:28:16.952301+0000","last_clean":"2024-03-06T07:28:16.952301+0000","last_became_active":"2024-03-06T07:28:15.486946+0000","last_became_peered":"2024-03-06T07:28:15.486946+0000","last_unstale":"2024-03-06T07:28:16.952301+0000","last_undegraded":"2024-03-06T07:28:16.952301+0000","last_fullsized":"2024-03-06T07:28:16.952301+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:13.603559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476050+0000","last_change":"2024-03-06T07:28:15.478455+0000","last_active":"2024-03-06T07:28:16.476050+0000","last_peered":"2024-03-06T07:28:16.476050+0000","last_clean":"2024-03-06T07:28:16.476050+0000","last_became_active":"2024-03-06T07:28:15.478197+0000","last_became_peered":"2024-03-06T07:28:15.478197+0000","last_unstale":"2024-03-06T07:28:16.476050+0000","last_undegraded":"2024-03-06T07:28:16.476050+0000","last_fullsized":"2024-03-06T07:28:16.476050+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:31:14.145099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953123+0000","last_change":"2024-03-06T07:28:15.481324+0000","last_active":"2024-03-06T07:28:16.953123+0000","last_peered":"2024-03-06T07:28:16.953123+0000","last_clean":"2024-03-06T07:28:16.953123+0000","last_became_active":"2024-03-06T07:28:15.481042+0000","last_became_peered":"2024-03-06T07:28:15.481042+0000","last_unstale":"2024-03-06T07:28:16.953123+0000","last_undegraded":"2024-03-06T07:28:16.953123+0000","last_fullsized":"2024-03-06T07:28:16.953123+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:58.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474176+0000","last_change":"2024-03-06T07:28:15.476284+0000","last_active":"2024-03-06T07:28:16.474176+0000","last_peered":"2024-03-06T07:28:16.474176+0000","last_clean":"2024-03-06T07:28:16.474176+0000","last_became_active":"2024-03-06T07:28:15.476001+0000","last_became_peered":"2024-03-06T07:28:15.476001+0000","last_unstale":"2024-03-06T07:28:16.474176+0000","last_undegraded":"2024-03-06T07:28:16.474176+0000","last_fullsized":"2024-03-06T07:28:16.474176+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:14.556084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474050+0000","last_change":"2024-03-06T07:28:15.479448+0000","last_active":"2024-03-06T07:28:16.474050+0000","last_peered":"2024-03-06T07:28:16.474050+0000","last_clean":"2024-03-06T07:28:16.474050+0000","last_became_active":"2024-03-06T07:28:15.479090+0000","last_became_peered":"2024-03-06T07:28:15.479090+0000","last_unstale":"2024-03-06T07:28:16.474050+0000","last_undegraded":"2024-03-06T07:28:16.474050+0000","last_fullsized":"2024-03-06T07:28:16.474050+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:14.816311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953032+0000","last_change":"2024-03-06T07:28:15.485110+0000","last_active":"2024-03-06T07:28:16.953032+0000","last_peered":"2024-03-06T07:28:16.953032+0000","last_clean":"2024-03-06T07:28:16.953032+0000","last_became_active":"2024-03-06T07:28:15.484843+0000","last_became_peered":"2024-03-06T07:28:15.484843+0000","last_unstale":"2024-03-06T07:28:16.953032+0000","last_undegraded":"2024-03-06T07:28:16.953032+0000","last_fullsized":"2024-03-06T07:28:16.953032+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:10.396815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476822+0000","last_change":"2024-03-06T07:28:15.474580+0000","last_active":"2024-03-06T07:28:16.476822+0000","last_peered":"2024-03-06T07:28:16.476822+0000","last_clean":"2024-03-06T07:28:16.476822+0000","last_became_active":"2024-03-06T07:28:15.474312+0000","last_became_peered":"2024-03-06T07:28:15.474312+0000","last_unstale":"2024-03-06T07:28:16.476822+0000","last_undegraded":"2024-03-06T07:28:16.476822+0000","last_fullsized":"2024-03-06T07:28:16.476822+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.760693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952007+0000","last_change":"2024-03-06T07:28:15.470286+0000","last_active":"2024-03-06T07:28:16.952007+0000","last_peered":"2024-03-06T07:28:16.952007+0000","last_clean":"2024-03-06T07:28:16.952007+0000","last_became_active":"2024-03-06T07:28:15.470021+0000","last_became_peered":"2024-03-06T07:28:15.470021+0000","last_unstale":"2024-03-06T07:28:16.952007+0000","last_undegraded":"2024-03-06T07:28:16.952007+0000","last_fullsized":"2024-03-06T07:28:16.952007+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:09.955331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952759+0000","last_change":"2024-03-06T07:28:15.470875+0000","last_active":"2024-03-06T07:28:16.952759+0000","last_peered":"2024-03-06T07:28:16.952759+0000","last_clean":"2024-03-06T07:28:16.952759+0000","last_became_active":"2024-03-06T07:28:15.470585+0000","last_became_peered":"2024-03-06T07:28:15.470585+0000","last_unstale":"2024-03-06T07:28:16.952759+0000","last_undegraded":"2024-03-06T07:28:16.952759+0000","last_fullsized":"2024-03-06T07:28:16.952759+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:53.847237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473616+0000","last_change":"2024-03-06T07:28:15.476490+0000","last_active":"2024-03-06T07:28:16.473616+0000","last_peered":"2024-03-06T07:28:16.473616+0000","last_clean":"2024-03-06T07:28:16.473616+0000","last_became_active":"2024-03-06T07:28:15.476181+0000","last_became_peered":"2024-03-06T07:28:15.476181+0000","last_unstale":"2024-03-06T07:28:16.473616+0000","last_undegraded":"2024-03-06T07:28:16.473616+0000","last_fullsized":"2024-03-06T07:28:16.473616+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:06:45.862903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477217+0000","last_change":"2024-03-06T07:28:15.468645+0000","last_active":"2024-03-06T07:28:16.477217+0000","last_peered":"2024-03-06T07:28:16.477217+0000","last_clean":"2024-03-06T07:28:16.477217+0000","last_became_active":"2024-03-06T07:28:15.468355+0000","last_became_peered":"2024-03-06T07:28:15.468355+0000","last_unstale":"2024-03-06T07:28:16.477217+0000","last_undegraded":"2024-03-06T07:28:16.477217+0000","last_fullsized":"2024-03-06T07:28:16.477217+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:24.316421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474269+0000","last_change":"2024-03-06T07:28:15.474022+0000","last_active":"2024-03-06T07:28:16.474269+0000","last_peered":"2024-03-06T07:28:16.474269+0000","last_clean":"2024-03-06T07:28:16.474269+0000","last_became_active":"2024-03-06T07:28:15.473656+0000","last_became_peered":"2024-03-06T07:28:15.473656+0000","last_unstale":"2024-03-06T07:28:16.474269+0000","last_undegraded":"2024-03-06T07:28:16.474269+0000","last_fullsized":"2024-03-06T07:28:16.474269+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:17.346689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953109+0000","last_change":"2024-03-06T07:28:15.475546+0000","last_active":"2024-03-06T07:28:16.953109+0000","last_peered":"2024-03-06T07:28:16.953109+0000","last_clean":"2024-03-06T07:28:16.953109+0000","last_became_active":"2024-03-06T07:28:15.475268+0000","last_became_peered":"2024-03-06T07:28:15.475268+0000","last_unstale":"2024-03-06T07:28:16.953109+0000","last_undegraded":"2024-03-06T07:28:16.953109+0000","last_fullsized":"2024-03-06T07:28:16.953109+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:02.394056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952471+0000","last_change":"2024-03-06T07:28:15.469070+0000","last_active":"2024-03-06T07:28:16.952471+0000","last_peered":"2024-03-06T07:28:16.952471+0000","last_clean":"2024-03-06T07:28:16.952471+0000","last_became_active":"2024-03-06T07:28:15.468785+0000","last_became_peered":"2024-03-06T07:28:15.468785+0000","last_unstale":"2024-03-06T07:28:16.952471+0000","last_undegraded":"2024-03-06T07:28:16.952471+0000","last_fullsized":"2024-03-06T07:28:16.952471+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:53.622360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952381+0000","last_change":"2024-03-06T07:28:15.479526+0000","last_active":"2024-03-06T07:28:16.952381+0000","last_peered":"2024-03-06T07:28:16.952381+0000","last_clean":"2024-03-06T07:28:16.952381+0000","last_became_active":"2024-03-06T07:28:15.479224+0000","last_became_peered":"2024-03-06T07:28:15.479224+0000","last_unstale":"2024-03-06T07:28:16.952381+0000","last_undegraded":"2024-03-06T07:28:16.952381+0000","last_fullsized":"2024-03-06T07:28:16.952381+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:06.690342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476217+0000","last_change":"2024-03-06T07:28:15.476992+0000","last_active":"2024-03-06T07:28:16.476217+0000","last_peered":"2024-03-06T07:28:16.476217+0000","last_clean":"2024-03-06T07:28:16.476217+0000","last_became_active":"2024-03-06T07:28:15.476720+0000","last_became_peered":"2024-03-06T07:28:15.476720+0000","last_unstale":"2024-03-06T07:28:16.476217+0000","last_undegraded":"2024-03-06T07:28:16.476217+0000","last_fullsized":"2024-03-06T07:28:16.476217+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:31.930810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952881+0000","last_change":"2024-03-06T07:28:15.476073+0000","last_active":"2024-03-06T07:28:16.952881+0000","last_peered":"2024-03-06T07:28:16.952881+0000","last_clean":"2024-03-06T07:28:16.952881+0000","last_became_active":"2024-03-06T07:28:15.475737+0000","last_became_peered":"2024-03-06T07:28:15.475737+0000","last_unstale":"2024-03-06T07:28:16.952881+0000","last_undegraded":"2024-03-06T07:28:16.952881+0000","last_fullsized":"2024-03-06T07:28:16.952881+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:10.552429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475491+0000","last_change":"2024-03-06T07:28:15.476093+0000","last_active":"2024-03-06T07:28:16.475491+0000","last_peered":"2024-03-06T07:28:16.475491+0000","last_clean":"2024-03-06T07:28:16.475491+0000","last_became_active":"2024-03-06T07:28:15.475717+0000","last_became_peered":"2024-03-06T07:28:15.475717+0000","last_unstale":"2024-03-06T07:28:16.475491+0000","last_undegraded":"2024-03-06T07:28:16.475491+0000","last_fullsized":"2024-03-06T07:28:16.475491+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953272+0000","last_change":"2024-03-06T07:28:15.472416+0000","last_active":"2024-03-06T07:28:16.953272+0000","last_peered":"2024-03-06T07:28:16.953272+0000","last_clean":"2024-03-06T07:28:16.953272+0000","last_became_active":"2024-03-06T07:28:15.472156+0000","last_became_peered":"2024-03-06T07:28:15.472156+0000","last_unstale":"2024-03-06T07:28:16.953272+0000","last_undegraded":"2024-03-06T07:28:16.953272+0000","last_fullsized":"2024-03-06T07:28:16.953272+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:51.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952826+0000","last_change":"2024-03-06T07:28:15.487221+0000","last_active":"2024-03-06T07:28:16.952826+0000","last_peered":"2024-03-06T07:28:16.952826+0000","last_clean":"2024-03-06T07:28:16.952826+0000","last_became_active":"2024-03-06T07:28:15.486932+0000","last_became_peered":"2024-03-06T07:28:15.486932+0000","last_unstale":"2024-03-06T07:28:16.952826+0000","last_undegraded":"2024-03-06T07:28:16.952826+0000","last_fullsized":"2024-03-06T07:28:16.952826+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476773+0000","last_change":"2024-03-06T07:28:15.476748+0000","last_active":"2024-03-06T07:28:16.476773+0000","last_peered":"2024-03-06T07:28:16.476773+0000","last_clean":"2024-03-06T07:28:16.476773+0000","last_became_active":"2024-03-06T07:28:15.476462+0000","last_became_peered":"2024-03-06T07:28:15.476462+0000","last_unstale":"2024-03-06T07:28:16.476773+0000","last_undegraded":"2024-03-06T07:28:16.476773+0000","last_fullsized":"2024-03-06T07:28:16.476773+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:06.690342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953092+0000","last_change":"2024-03-06T07:28:15.483660+0000","last_active":"2024-03-06T07:28:16.953092+0000","last_peered":"2024-03-06T07:28:16.953092+0000","last_clean":"2024-03-06T07:28:16.953092+0000","last_became_active":"2024-03-06T07:28:15.483379+0000","last_became_peered":"2024-03-06T07:28:15.483379+0000","last_unstale":"2024-03-06T07:28:16.953092+0000","last_undegraded":"2024-03-06T07:28:16.953092+0000","last_fullsized":"2024-03-06T07:28:16.953092+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:40.262748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951948+0000","last_change":"2024-03-06T07:28:15.476365+0000","last_active":"2024-03-06T07:28:16.951948+0000","last_peered":"2024-03-06T07:28:16.951948+0000","last_clean":"2024-03-06T07:28:16.951948+0000","last_became_active":"2024-03-06T07:28:15.476090+0000","last_became_peered":"2024-03-06T07:28:15.476090+0000","last_unstale":"2024-03-06T07:28:16.951948+0000","last_undegraded":"2024-03-06T07:28:16.951948+0000","last_fullsized":"2024-03-06T07:28:16.951948+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:18.891603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952158+0000","last_change":"2024-03-06T07:28:15.477135+0000","last_active":"2024-03-06T07:28:16.952158+0000","last_peered":"2024-03-06T07:28:16.952158+0000","last_clean":"2024-03-06T07:28:16.952158+0000","last_became_active":"2024-03-06T07:28:15.476869+0000","last_became_peered":"2024-03-06T07:28:15.476869+0000","last_unstale":"2024-03-06T07:28:16.952158+0000","last_undegraded":"2024-03-06T07:28:16.952158+0000","last_fullsized":"2024-03-06T07:28:16.952158+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:02.439995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951970+0000","last_change":"2024-03-06T07:28:15.480910+0000","last_active":"2024-03-06T07:28:16.951970+0000","last_peered":"2024-03-06T07:28:16.951970+0000","last_clean":"2024-03-06T07:28:16.951970+0000","last_became_active":"2024-03-06T07:28:15.478466+0000","last_became_peered":"2024-03-06T07:28:15.478466+0000","last_unstale":"2024-03-06T07:28:16.951970+0000","last_undegraded":"2024-03-06T07:28:16.951970+0000","last_fullsized":"2024-03-06T07:28:16.951970+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:00.206292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952616+0000","last_change":"2024-03-06T07:28:15.476973+0000","last_active":"2024-03-06T07:28:16.952616+0000","last_peered":"2024-03-06T07:28:16.952616+0000","last_clean":"2024-03-06T07:28:16.952616+0000","last_became_active":"2024-03-06T07:28:15.476711+0000","last_became_peered":"2024-03-06T07:28:15.476711+0000","last_unstale":"2024-03-06T07:28:16.952616+0000","last_undegraded":"2024-03-06T07:28:16.952616+0000","last_fullsized":"2024-03-06T07:28:16.952616+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:40.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952005+0000","last_change":"2024-03-06T07:28:15.486635+0000","last_active":"2024-03-06T07:28:16.952005+0000","last_peered":"2024-03-06T07:28:16.952005+0000","last_clean":"2024-03-06T07:28:16.952005+0000","last_became_active":"2024-03-06T07:28:15.480805+0000","last_became_peered":"2024-03-06T07:28:15.480805+0000","last_unstale":"2024-03-06T07:28:16.952005+0000","last_undegraded":"2024-03-06T07:28:16.952005+0000","last_fullsized":"2024-03-06T07:28:16.952005+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:09.583467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"198'4","reported_seq":18,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952954+0000","last_change":"2024-03-06T07:28:14.471148+0000","last_active":"2024-03-06T07:28:16.952954+0000","last_peered":"2024-03-06T07:28:16.952954+0000","last_clean":"2024-03-06T07:28:16.952954+0000","last_became_active":"2024-03-06T07:28:14.470830+0000","last_became_peered":"2024-03-06T07:28:14.470830+0000","last_unstale":"2024-03-06T07:28:16.952954+0000","last_undegraded":"2024-03-06T07:28:16.952954+0000","last_fullsized":"2024-03-06T07:28:16.952954+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:49.743612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953065+0000","last_change":"2024-03-06T07:28:15.475096+0000","last_active":"2024-03-06T07:28:16.953065+0000","last_peered":"2024-03-06T07:28:16.953065+0000","last_clean":"2024-03-06T07:28:16.953065+0000","last_became_active":"2024-03-06T07:28:15.474822+0000","last_became_peered":"2024-03-06T07:28:15.474822+0000","last_unstale":"2024-03-06T07:28:16.953065+0000","last_undegraded":"2024-03-06T07:28:16.953065+0000","last_fullsized":"2024-03-06T07:28:16.953065+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:09:11.330344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476183+0000","last_change":"2024-03-06T07:28:14.474168+0000","last_active":"2024-03-06T07:28:16.476183+0000","last_peered":"2024-03-06T07:28:16.476183+0000","last_clean":"2024-03-06T07:28:16.476183+0000","last_became_active":"2024-03-06T07:28:14.473851+0000","last_became_peered":"2024-03-06T07:28:14.473851+0000","last_unstale":"2024-03-06T07:28:16.476183+0000","last_undegraded":"2024-03-06T07:28:16.476183+0000","last_fullsized":"2024-03-06T07:28:16.476183+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:14.997022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952789+0000","last_change":"2024-03-06T07:28:15.488767+0000","last_active":"2024-03-06T07:28:16.952789+0000","last_peered":"2024-03-06T07:28:16.952789+0000","last_clean":"2024-03-06T07:28:16.952789+0000","last_became_active":"2024-03-06T07:28:15.488299+0000","last_became_peered":"2024-03-06T07:28:15.488299+0000","last_unstale":"2024-03-06T07:28:16.952789+0000","last_undegraded":"2024-03-06T07:28:16.952789+0000","last_fullsized":"2024-03-06T07:28:16.952789+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:43.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952334+0000","last_change":"2024-03-06T07:28:14.474406+0000","last_active":"2024-03-06T07:28:16.952334+0000","last_peered":"2024-03-06T07:28:16.952334+0000","last_clean":"2024-03-06T07:28:16.952334+0000","last_became_active":"2024-03-06T07:28:14.474128+0000","last_became_peered":"2024-03-06T07:28:14.474128+0000","last_unstale":"2024-03-06T07:28:16.952334+0000","last_undegraded":"2024-03-06T07:28:16.952334+0000","last_fullsized":"2024-03-06T07:28:16.952334+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:38.525288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952182+0000","last_change":"2024-03-06T07:28:15.470696+0000","last_active":"2024-03-06T07:28:16.952182+0000","last_peered":"2024-03-06T07:28:16.952182+0000","last_clean":"2024-03-06T07:28:16.952182+0000","last_became_active":"2024-03-06T07:28:15.470373+0000","last_became_peered":"2024-03-06T07:28:15.470373+0000","last_unstale":"2024-03-06T07:28:16.952182+0000","last_undegraded":"2024-03-06T07:28:16.952182+0000","last_fullsized":"2024-03-06T07:28:16.952182+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:09.652727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953043+0000","last_change":"2024-03-06T07:28:14.471583+0000","last_active":"2024-03-06T07:28:16.953043+0000","last_peered":"2024-03-06T07:28:16.953043+0000","last_clean":"2024-03-06T07:28:16.953043+0000","last_became_active":"2024-03-06T07:28:14.471286+0000","last_became_peered":"2024-03-06T07:28:14.471286+0000","last_unstale":"2024-03-06T07:28:16.953043+0000","last_undegraded":"2024-03-06T07:28:16.953043+0000","last_fullsized":"2024-03-06T07:28:16.953043+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:18.349428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952909+0000","last_change":"2024-03-06T07:28:15.479577+0000","last_active":"2024-03-06T07:28:16.952909+0000","last_peered":"2024-03-06T07:28:16.952909+0000","last_clean":"2024-03-06T07:28:16.952909+0000","last_became_active":"2024-03-06T07:28:15.479308+0000","last_became_peered":"2024-03-06T07:28:15.479308+0000","last_unstale":"2024-03-06T07:28:16.952909+0000","last_undegraded":"2024-03-06T07:28:16.952909+0000","last_fullsized":"2024-03-06T07:28:16.952909+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:32.366142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952269+0000","last_change":"2024-03-06T07:28:14.479954+0000","last_active":"2024-03-06T07:28:16.952269+0000","last_peered":"2024-03-06T07:28:16.952269+0000","last_clean":"2024-03-06T07:28:16.952269+0000","last_became_active":"2024-03-06T07:28:14.479664+0000","last_became_peered":"2024-03-06T07:28:14.479664+0000","last_unstale":"2024-03-06T07:28:16.952269+0000","last_undegraded":"2024-03-06T07:28:16.952269+0000","last_fullsized":"2024-03-06T07:28:16.952269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:04.403220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474111+0000","last_change":"2024-03-06T07:28:15.483469+0000","last_active":"2024-03-06T07:28:16.474111+0000","last_peered":"2024-03-06T07:28:16.474111+0000","last_clean":"2024-03-06T07:28:16.474111+0000","last_became_active":"2024-03-06T07:28:15.483167+0000","last_became_peered":"2024-03-06T07:28:15.483167+0000","last_unstale":"2024-03-06T07:28:16.474111+0000","last_undegraded":"2024-03-06T07:28:16.474111+0000","last_fullsized":"2024-03-06T07:28:16.474111+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:21:57.359258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473598+0000","last_change":"2024-03-06T07:28:14.468330+0000","last_active":"2024-03-06T07:28:16.473598+0000","last_peered":"2024-03-06T07:28:16.473598+0000","last_clean":"2024-03-06T07:28:16.473598+0000","last_became_active":"2024-03-06T07:28:14.468029+0000","last_became_peered":"2024-03-06T07:28:14.468029+0000","last_unstale":"2024-03-06T07:28:16.473598+0000","last_undegraded":"2024-03-06T07:28:16.473598+0000","last_fullsized":"2024-03-06T07:28:16.473598+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:14.598213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477054+0000","last_change":"2024-03-06T07:28:15.476659+0000","last_active":"2024-03-06T07:28:16.477054+0000","last_peered":"2024-03-06T07:28:16.477054+0000","last_clean":"2024-03-06T07:28:16.477054+0000","last_became_active":"2024-03-06T07:28:15.476377+0000","last_became_peered":"2024-03-06T07:28:15.476377+0000","last_unstale":"2024-03-06T07:28:16.477054+0000","last_undegraded":"2024-03-06T07:28:16.477054+0000","last_fullsized":"2024-03-06T07:28:16.477054+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952706+0000","last_change":"2024-03-06T07:28:14.468094+0000","last_active":"2024-03-06T07:28:16.952706+0000","last_peered":"2024-03-06T07:28:16.952706+0000","last_clean":"2024-03-06T07:28:16.952706+0000","last_became_active":"2024-03-06T07:28:14.467785+0000","last_became_peered":"2024-03-06T07:28:14.467785+0000","last_unstale":"2024-03-06T07:28:16.952706+0000","last_undegraded":"2024-03-06T07:28:16.952706+0000","last_fullsized":"2024-03-06T07:28:16.952706+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:53:19.841681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952477+0000","last_change":"2024-03-06T07:28:15.479770+0000","last_active":"2024-03-06T07:28:16.952477+0000","last_peered":"2024-03-06T07:28:16.952477+0000","last_clean":"2024-03-06T07:28:16.952477+0000","last_became_active":"2024-03-06T07:28:15.478996+0000","last_became_peered":"2024-03-06T07:28:15.478996+0000","last_unstale":"2024-03-06T07:28:16.952477+0000","last_undegraded":"2024-03-06T07:28:16.952477+0000","last_fullsized":"2024-03-06T07:28:16.952477+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:09.366966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952185+0000","last_change":"2024-03-06T07:28:14.479963+0000","last_active":"2024-03-06T07:28:16.952185+0000","last_peered":"2024-03-06T07:28:16.952185+0000","last_clean":"2024-03-06T07:28:16.952185+0000","last_became_active":"2024-03-06T07:28:14.479166+0000","last_became_peered":"2024-03-06T07:28:14.479166+0000","last_unstale":"2024-03-06T07:28:16.952185+0000","last_undegraded":"2024-03-06T07:28:16.952185+0000","last_fullsized":"2024-03-06T07:28:16.952185+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:56.877583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475940+0000","last_change":"2024-03-06T07:28:15.477776+0000","last_active":"2024-03-06T07:28:16.475940+0000","last_peered":"2024-03-06T07:28:16.475940+0000","last_clean":"2024-03-06T07:28:16.475940+0000","last_became_active":"2024-03-06T07:28:15.477007+0000","last_became_peered":"2024-03-06T07:28:15.477007+0000","last_unstale":"2024-03-06T07:28:16.475940+0000","last_undegraded":"2024-03-06T07:28:16.475940+0000","last_fullsized":"2024-03-06T07:28:16.475940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.721149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474240+0000","last_change":"2024-03-06T07:28:14.474135+0000","last_active":"2024-03-06T07:28:16.474240+0000","last_peered":"2024-03-06T07:28:16.474240+0000","last_clean":"2024-03-06T07:28:16.474240+0000","last_became_active":"2024-03-06T07:28:14.473827+0000","last_became_peered":"2024-03-06T07:28:14.473827+0000","last_unstale":"2024-03-06T07:28:16.474240+0000","last_undegraded":"2024-03-06T07:28:16.474240+0000","last_fullsized":"2024-03-06T07:28:16.474240+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:52.845585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473720+0000","last_change":"2024-03-06T07:28:15.475645+0000","last_active":"2024-03-06T07:28:16.473720+0000","last_peered":"2024-03-06T07:28:16.473720+0000","last_clean":"2024-03-06T07:28:16.473720+0000","last_became_active":"2024-03-06T07:28:15.475347+0000","last_became_peered":"2024-03-06T07:28:15.475347+0000","last_unstale":"2024-03-06T07:28:16.473720+0000","last_undegraded":"2024-03-06T07:28:16.473720+0000","last_fullsized":"2024-03-06T07:28:16.473720+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:18.891603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"198'3","reported_seq":17,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477190+0000","last_change":"2024-03-06T07:28:14.476613+0000","last_active":"2024-03-06T07:28:16.477190+0000","last_peered":"2024-03-06T07:28:16.477190+0000","last_clean":"2024-03-06T07:28:16.477190+0000","last_became_active":"2024-03-06T07:28:14.476330+0000","last_became_peered":"2024-03-06T07:28:14.476330+0000","last_unstale":"2024-03-06T07:28:16.477190+0000","last_undegraded":"2024-03-06T07:28:16.477190+0000","last_fullsized":"2024-03-06T07:28:16.477190+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.459867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952329+0000","last_change":"2024-03-06T07:28:15.476982+0000","last_active":"2024-03-06T07:28:16.952329+0000","last_peered":"2024-03-06T07:28:16.952329+0000","last_clean":"2024-03-06T07:28:16.952329+0000","last_became_active":"2024-03-06T07:28:15.476715+0000","last_became_peered":"2024-03-06T07:28:15.476715+0000","last_unstale":"2024-03-06T07:28:16.952329+0000","last_undegraded":"2024-03-06T07:28:16.952329+0000","last_fullsized":"2024-03-06T07:28:16.952329+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:57.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"198'6","reported_seq":20,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953128+0000","last_change":"2024-03-06T07:28:14.480209+0000","last_active":"2024-03-06T07:28:16.953128+0000","last_peered":"2024-03-06T07:28:16.953128+0000","last_clean":"2024-03-06T07:28:16.953128+0000","last_became_active":"2024-03-06T07:28:14.479918+0000","last_became_peered":"2024-03-06T07:28:14.479918+0000","last_unstale":"2024-03-06T07:28:16.953128+0000","last_undegraded":"2024-03-06T07:28:16.953128+0000","last_fullsized":"2024-03-06T07:28:16.953128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:19.050940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952384+0000","last_change":"2024-03-06T07:28:15.486999+0000","last_active":"2024-03-06T07:28:16.952384+0000","last_peered":"2024-03-06T07:28:16.952384+0000","last_clean":"2024-03-06T07:28:16.952384+0000","last_became_active":"2024-03-06T07:28:15.486437+0000","last_became_peered":"2024-03-06T07:28:15.486437+0000","last_unstale":"2024-03-06T07:28:16.952384+0000","last_undegraded":"2024-03-06T07:28:16.952384+0000","last_fullsized":"2024-03-06T07:28:16.952384+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:13.380260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953098+0000","last_change":"2024-03-06T07:28:14.470176+0000","last_active":"2024-03-06T07:28:16.953098+0000","last_peered":"2024-03-06T07:28:16.953098+0000","last_clean":"2024-03-06T07:28:16.953098+0000","last_became_active":"2024-03-06T07:28:14.469841+0000","last_became_peered":"2024-03-06T07:28:14.469841+0000","last_unstale":"2024-03-06T07:28:16.953098+0000","last_undegraded":"2024-03-06T07:28:16.953098+0000","last_fullsized":"2024-03-06T07:28:16.953098+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:05.688690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952669+0000","last_change":"2024-03-06T07:28:15.474360+0000","last_active":"2024-03-06T07:28:16.952669+0000","last_peered":"2024-03-06T07:28:16.952669+0000","last_clean":"2024-03-06T07:28:16.952669+0000","last_became_active":"2024-03-06T07:28:15.473914+0000","last_became_peered":"2024-03-06T07:28:15.473914+0000","last_unstale":"2024-03-06T07:28:16.952669+0000","last_undegraded":"2024-03-06T07:28:16.952669+0000","last_fullsized":"2024-03-06T07:28:16.952669+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:19.351080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476483+0000","last_change":"2024-03-06T07:28:14.477371+0000","last_active":"2024-03-06T07:28:16.476483+0000","last_peered":"2024-03-06T07:28:16.476483+0000","last_clean":"2024-03-06T07:28:16.476483+0000","last_became_active":"2024-03-06T07:28:14.477123+0000","last_became_peered":"2024-03-06T07:28:14.477123+0000","last_unstale":"2024-03-06T07:28:16.476483+0000","last_undegraded":"2024-03-06T07:28:16.476483+0000","last_fullsized":"2024-03-06T07:28:16.476483+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:07.520579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952797+0000","last_change":"2024-03-06T07:28:15.472634+0000","last_active":"2024-03-06T07:28:16.952797+0000","last_peered":"2024-03-06T07:28:16.952797+0000","last_clean":"2024-03-06T07:28:16.952797+0000","last_became_active":"2024-03-06T07:28:15.471820+0000","last_became_peered":"2024-03-06T07:28:15.471820+0000","last_unstale":"2024-03-06T07:28:16.952797+0000","last_undegraded":"2024-03-06T07:28:16.952797+0000","last_fullsized":"2024-03-06T07:28:16.952797+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:15.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476221+0000","last_change":"2024-03-06T07:28:14.474122+0000","last_active":"2024-03-06T07:28:16.476221+0000","last_peered":"2024-03-06T07:28:16.476221+0000","last_clean":"2024-03-06T07:28:16.476221+0000","last_became_active":"2024-03-06T07:28:14.473820+0000","last_became_peered":"2024-03-06T07:28:14.473820+0000","last_unstale":"2024-03-06T07:28:16.476221+0000","last_undegraded":"2024-03-06T07:28:16.476221+0000","last_fullsized":"2024-03-06T07:28:16.476221+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:47.568139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477036+0000","last_change":"2024-03-06T07:28:15.467219+0000","last_active":"2024-03-06T07:28:16.477036+0000","last_peered":"2024-03-06T07:28:16.477036+0000","last_clean":"2024-03-06T07:28:16.477036+0000","last_became_active":"2024-03-06T07:28:15.466232+0000","last_became_peered":"2024-03-06T07:28:15.466232+0000","last_unstale":"2024-03-06T07:28:16.477036+0000","last_undegraded":"2024-03-06T07:28:16.477036+0000","last_fullsized":"2024-03-06T07:28:16.477036+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:09.366966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"198'8","reported_seq":22,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951895+0000","last_change":"2024-03-06T07:28:14.476089+0000","last_active":"2024-03-06T07:28:16.951895+0000","last_peered":"2024-03-06T07:28:16.951895+0000","last_clean":"2024-03-06T07:28:16.951895+0000","last_became_active":"2024-03-06T07:28:14.475792+0000","last_became_peered":"2024-03-06T07:28:14.475792+0000","last_unstale":"2024-03-06T07:28:16.951895+0000","last_undegraded":"2024-03-06T07:28:16.951895+0000","last_fullsized":"2024-03-06T07:28:16.951895+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:03.678148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475525+0000","last_change":"2024-03-06T07:28:15.472076+0000","last_active":"2024-03-06T07:28:16.475525+0000","last_peered":"2024-03-06T07:28:16.475525+0000","last_clean":"2024-03-06T07:28:16.475525+0000","last_became_active":"2024-03-06T07:28:15.471056+0000","last_became_peered":"2024-03-06T07:28:15.471056+0000","last_unstale":"2024-03-06T07:28:16.475525+0000","last_undegraded":"2024-03-06T07:28:16.475525+0000","last_fullsized":"2024-03-06T07:28:16.475525+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:13.603559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953354+0000","last_change":"2024-03-06T07:28:14.476332+0000","last_active":"2024-03-06T07:28:16.953354+0000","last_peered":"2024-03-06T07:28:16.953354+0000","last_clean":"2024-03-06T07:28:16.953354+0000","last_became_active":"2024-03-06T07:28:14.476061+0000","last_became_peered":"2024-03-06T07:28:14.476061+0000","last_unstale":"2024-03-06T07:28:16.953354+0000","last_undegraded":"2024-03-06T07:28:16.953354+0000","last_fullsized":"2024-03-06T07:28:16.953354+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:16:34.591052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476023+0000","last_change":"2024-03-06T07:28:15.471783+0000","last_active":"2024-03-06T07:28:16.476023+0000","last_peered":"2024-03-06T07:28:16.476023+0000","last_clean":"2024-03-06T07:28:16.476023+0000","last_became_active":"2024-03-06T07:28:15.471514+0000","last_became_peered":"2024-03-06T07:28:15.471514+0000","last_unstale":"2024-03-06T07:28:16.476023+0000","last_undegraded":"2024-03-06T07:28:16.476023+0000","last_fullsized":"2024-03-06T07:28:16.476023+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:57.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953204+0000","last_change":"2024-03-06T07:28:14.477580+0000","last_active":"2024-03-06T07:28:16.953204+0000","last_peered":"2024-03-06T07:28:16.953204+0000","last_clean":"2024-03-06T07:28:16.953204+0000","last_became_active":"2024-03-06T07:28:14.476526+0000","last_became_peered":"2024-03-06T07:28:14.476526+0000","last_unstale":"2024-03-06T07:28:16.953204+0000","last_undegraded":"2024-03-06T07:28:16.953204+0000","last_fullsized":"2024-03-06T07:28:16.953204+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:44:05.879351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951959+0000","last_change":"2024-03-06T07:28:15.476701+0000","last_active":"2024-03-06T07:28:16.951959+0000","last_peered":"2024-03-06T07:28:16.951959+0000","last_clean":"2024-03-06T07:28:16.951959+0000","last_became_active":"2024-03-06T07:28:15.476433+0000","last_became_peered":"2024-03-06T07:28:15.476433+0000","last_unstale":"2024-03-06T07:28:16.951959+0000","last_undegraded":"2024-03-06T07:28:16.951959+0000","last_fullsized":"2024-03-06T07:28:16.951959+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:38.479080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475755+0000","last_change":"2024-03-06T07:28:14.473266+0000","last_active":"2024-03-06T07:28:16.475755+0000","last_peered":"2024-03-06T07:28:16.475755+0000","last_clean":"2024-03-06T07:28:16.475755+0000","last_became_active":"2024-03-06T07:28:14.472995+0000","last_became_peered":"2024-03-06T07:28:14.472995+0000","last_unstale":"2024-03-06T07:28:16.475755+0000","last_undegraded":"2024-03-06T07:28:16.475755+0000","last_fullsized":"2024-03-06T07:28:16.475755+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:16:22.648652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473879+0000","last_change":"2024-03-06T07:28:15.476734+0000","last_active":"2024-03-06T07:28:16.473879+0000","last_peered":"2024-03-06T07:28:16.473879+0000","last_clean":"2024-03-06T07:28:16.473879+0000","last_became_active":"2024-03-06T07:28:15.476449+0000","last_became_peered":"2024-03-06T07:28:15.476449+0000","last_unstale":"2024-03-06T07:28:16.473879+0000","last_undegraded":"2024-03-06T07:28:16.473879+0000","last_fullsized":"2024-03-06T07:28:16.473879+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:09.955331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952731+0000","last_change":"2024-03-06T07:28:14.468828+0000","last_active":"2024-03-06T07:28:16.952731+0000","last_peered":"2024-03-06T07:28:16.952731+0000","last_clean":"2024-03-06T07:28:16.952731+0000","last_became_active":"2024-03-06T07:28:14.468552+0000","last_became_peered":"2024-03-06T07:28:14.468552+0000","last_unstale":"2024-03-06T07:28:16.952731+0000","last_undegraded":"2024-03-06T07:28:16.952731+0000","last_fullsized":"2024-03-06T07:28:16.952731+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:08.365314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473895+0000","last_change":"2024-03-06T07:28:15.488981+0000","last_active":"2024-03-06T07:28:16.473895+0000","last_peered":"2024-03-06T07:28:16.473895+0000","last_clean":"2024-03-06T07:28:16.473895+0000","last_became_active":"2024-03-06T07:28:15.488675+0000","last_became_peered":"2024-03-06T07:28:15.488675+0000","last_unstale":"2024-03-06T07:28:16.473895+0000","last_undegraded":"2024-03-06T07:28:16.473895+0000","last_fullsized":"2024-03-06T07:28:16.473895+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:22.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473815+0000","last_change":"2024-03-06T07:28:14.477668+0000","last_active":"2024-03-06T07:28:16.473815+0000","last_peered":"2024-03-06T07:28:16.473815+0000","last_clean":"2024-03-06T07:28:16.473815+0000","last_became_active":"2024-03-06T07:28:14.477351+0000","last_became_peered":"2024-03-06T07:28:14.477351+0000","last_unstale":"2024-03-06T07:28:16.473815+0000","last_undegraded":"2024-03-06T07:28:16.473815+0000","last_fullsized":"2024-03-06T07:28:16.473815+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:04.324747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952318+0000","last_change":"2024-03-06T07:28:15.480926+0000","last_active":"2024-03-06T07:28:16.952318+0000","last_peered":"2024-03-06T07:28:16.952318+0000","last_clean":"2024-03-06T07:28:16.952318+0000","last_became_active":"2024-03-06T07:28:15.480656+0000","last_became_peered":"2024-03-06T07:28:15.480656+0000","last_unstale":"2024-03-06T07:28:16.952318+0000","last_undegraded":"2024-03-06T07:28:16.952318+0000","last_fullsized":"2024-03-06T07:28:16.952318+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:14:19.717811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476324+0000","last_change":"2024-03-06T07:28:14.474364+0000","last_active":"2024-03-06T07:28:16.476324+0000","last_peered":"2024-03-06T07:28:16.476324+0000","last_clean":"2024-03-06T07:28:16.476324+0000","last_became_active":"2024-03-06T07:28:14.474054+0000","last_became_peered":"2024-03-06T07:28:14.474054+0000","last_unstale":"2024-03-06T07:28:16.476324+0000","last_undegraded":"2024-03-06T07:28:16.476324+0000","last_fullsized":"2024-03-06T07:28:16.476324+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:06.532834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476020+0000","last_change":"2024-03-06T07:28:15.467927+0000","last_active":"2024-03-06T07:28:16.476020+0000","last_peered":"2024-03-06T07:28:16.476020+0000","last_clean":"2024-03-06T07:28:16.476020+0000","last_became_active":"2024-03-06T07:28:15.467662+0000","last_became_peered":"2024-03-06T07:28:15.467662+0000","last_unstale":"2024-03-06T07:28:16.476020+0000","last_undegraded":"2024-03-06T07:28:16.476020+0000","last_fullsized":"2024-03-06T07:28:16.476020+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:06.847067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476060+0000","last_change":"2024-03-06T07:28:14.471522+0000","last_active":"2024-03-06T07:28:16.476060+0000","last_peered":"2024-03-06T07:28:16.476060+0000","last_clean":"2024-03-06T07:28:16.476060+0000","last_became_active":"2024-03-06T07:28:14.471248+0000","last_became_peered":"2024-03-06T07:28:14.471248+0000","last_unstale":"2024-03-06T07:28:16.476060+0000","last_undegraded":"2024-03-06T07:28:16.476060+0000","last_fullsized":"2024-03-06T07:28:16.476060+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:24.816902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476595+0000","last_change":"2024-03-06T07:28:15.471462+0000","last_active":"2024-03-06T07:28:16.476595+0000","last_peered":"2024-03-06T07:28:16.476595+0000","last_clean":"2024-03-06T07:28:16.476595+0000","last_became_active":"2024-03-06T07:28:15.471165+0000","last_became_peered":"2024-03-06T07:28:15.471165+0000","last_unstale":"2024-03-06T07:28:16.476595+0000","last_undegraded":"2024-03-06T07:28:16.476595+0000","last_fullsized":"2024-03-06T07:28:16.476595+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:01.241913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952578+0000","last_change":"2024-03-06T07:28:14.475474+0000","last_active":"2024-03-06T07:28:16.952578+0000","last_peered":"2024-03-06T07:28:16.952578+0000","last_clean":"2024-03-06T07:28:16.952578+0000","last_became_active":"2024-03-06T07:28:14.475178+0000","last_became_peered":"2024-03-06T07:28:14.475178+0000","last_unstale":"2024-03-06T07:28:16.952578+0000","last_undegraded":"2024-03-06T07:28:16.952578+0000","last_fullsized":"2024-03-06T07:28:16.952578+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:22.863787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952372+0000","last_change":"2024-03-06T07:28:15.487222+0000","last_active":"2024-03-06T07:28:16.952372+0000","last_peered":"2024-03-06T07:28:16.952372+0000","last_clean":"2024-03-06T07:28:16.952372+0000","last_became_active":"2024-03-06T07:28:15.486917+0000","last_became_peered":"2024-03-06T07:28:15.486917+0000","last_unstale":"2024-03-06T07:28:16.952372+0000","last_undegraded":"2024-03-06T07:28:16.952372+0000","last_fullsized":"2024-03-06T07:28:16.952372+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:31:14.145099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474041+0000","last_change":"2024-03-06T07:28:14.482310+0000","last_active":"2024-03-06T07:28:16.474041+0000","last_peered":"2024-03-06T07:28:16.474041+0000","last_clean":"2024-03-06T07:28:16.474041+0000","last_became_active":"2024-03-06T07:28:14.481984+0000","last_became_peered":"2024-03-06T07:28:14.481984+0000","last_unstale":"2024-03-06T07:28:16.474041+0000","last_undegraded":"2024-03-06T07:28:16.474041+0000","last_fullsized":"2024-03-06T07:28:16.474041+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:17.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952668+0000","last_change":"2024-03-06T07:28:15.470152+0000","last_active":"2024-03-06T07:28:16.952668+0000","last_peered":"2024-03-06T07:28:16.952668+0000","last_clean":"2024-03-06T07:28:16.952668+0000","last_became_active":"2024-03-06T07:28:15.469898+0000","last_became_peered":"2024-03-06T07:28:15.469898+0000","last_unstale":"2024-03-06T07:28:16.952668+0000","last_undegraded":"2024-03-06T07:28:16.952668+0000","last_fullsized":"2024-03-06T07:28:16.952668+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:18.153085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476867+0000","last_change":"2024-03-06T07:28:14.474756+0000","last_active":"2024-03-06T07:28:16.476867+0000","last_peered":"2024-03-06T07:28:16.476867+0000","last_clean":"2024-03-06T07:28:16.476867+0000","last_became_active":"2024-03-06T07:28:14.474454+0000","last_became_peered":"2024-03-06T07:28:14.474454+0000","last_unstale":"2024-03-06T07:28:16.476867+0000","last_undegraded":"2024-03-06T07:28:16.476867+0000","last_fullsized":"2024-03-06T07:28:16.476867+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.759041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473657+0000","last_change":"2024-03-06T07:28:15.477055+0000","last_active":"2024-03-06T07:28:16.473657+0000","last_peered":"2024-03-06T07:28:16.473657+0000","last_clean":"2024-03-06T07:28:16.473657+0000","last_became_active":"2024-03-06T07:28:15.475898+0000","last_became_peered":"2024-03-06T07:28:15.475898+0000","last_unstale":"2024-03-06T07:28:16.473657+0000","last_undegraded":"2024-03-06T07:28:16.473657+0000","last_fullsized":"2024-03-06T07:28:16.473657+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:23.417836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952255+0000","last_change":"2024-03-06T07:28:14.477036+0000","last_active":"2024-03-06T07:28:16.952255+0000","last_peered":"2024-03-06T07:28:16.952255+0000","last_clean":"2024-03-06T07:28:16.952255+0000","last_became_active":"2024-03-06T07:28:14.476402+0000","last_became_peered":"2024-03-06T07:28:14.476402+0000","last_unstale":"2024-03-06T07:28:16.952255+0000","last_undegraded":"2024-03-06T07:28:16.952255+0000","last_fullsized":"2024-03-06T07:28:16.952255+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:11.261142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477029+0000","last_change":"2024-03-06T07:28:15.470650+0000","last_active":"2024-03-06T07:28:16.477029+0000","last_peered":"2024-03-06T07:28:16.477029+0000","last_clean":"2024-03-06T07:28:16.477029+0000","last_became_active":"2024-03-06T07:28:15.470324+0000","last_became_peered":"2024-03-06T07:28:15.470324+0000","last_unstale":"2024-03-06T07:28:16.477029+0000","last_undegraded":"2024-03-06T07:28:16.477029+0000","last_fullsized":"2024-03-06T07:28:16.477029+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:19.351080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952362+0000","last_change":"2024-03-06T07:28:14.478503+0000","last_active":"2024-03-06T07:28:16.952362+0000","last_peered":"2024-03-06T07:28:16.952362+0000","last_clean":"2024-03-06T07:28:16.952362+0000","last_became_active":"2024-03-06T07:28:14.478064+0000","last_became_peered":"2024-03-06T07:28:14.478064+0000","last_unstale":"2024-03-06T07:28:16.952362+0000","last_undegraded":"2024-03-06T07:28:16.952362+0000","last_fullsized":"2024-03-06T07:28:16.952362+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:25.967609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952040+0000","last_change":"2024-03-06T07:28:15.480793+0000","last_active":"2024-03-06T07:28:16.952040+0000","last_peered":"2024-03-06T07:28:16.952040+0000","last_clean":"2024-03-06T07:28:16.952040+0000","last_became_active":"2024-03-06T07:28:15.480518+0000","last_became_peered":"2024-03-06T07:28:15.480518+0000","last_unstale":"2024-03-06T07:28:16.952040+0000","last_undegraded":"2024-03-06T07:28:16.952040+0000","last_fullsized":"2024-03-06T07:28:16.952040+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:22.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476584+0000","last_change":"2024-03-06T07:28:14.473260+0000","last_active":"2024-03-06T07:28:16.476584+0000","last_peered":"2024-03-06T07:28:16.476584+0000","last_clean":"2024-03-06T07:28:16.476584+0000","last_became_active":"2024-03-06T07:28:14.472982+0000","last_became_peered":"2024-03-06T07:28:14.472982+0000","last_unstale":"2024-03-06T07:28:16.476584+0000","last_undegraded":"2024-03-06T07:28:16.476584+0000","last_fullsized":"2024-03-06T07:28:16.476584+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:49.051038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952901+0000","last_change":"2024-03-06T07:28:15.479469+0000","last_active":"2024-03-06T07:28:16.952901+0000","last_peered":"2024-03-06T07:28:16.952901+0000","last_clean":"2024-03-06T07:28:16.952901+0000","last_became_active":"2024-03-06T07:28:15.479143+0000","last_became_peered":"2024-03-06T07:28:15.479143+0000","last_unstale":"2024-03-06T07:28:16.952901+0000","last_undegraded":"2024-03-06T07:28:16.952901+0000","last_fullsized":"2024-03-06T07:28:16.952901+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:58.942134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476582+0000","last_change":"2024-03-06T07:28:14.474422+0000","last_active":"2024-03-06T07:28:16.476582+0000","last_peered":"2024-03-06T07:28:16.476582+0000","last_clean":"2024-03-06T07:28:16.476582+0000","last_became_active":"2024-03-06T07:28:14.474156+0000","last_became_peered":"2024-03-06T07:28:14.474156+0000","last_unstale":"2024-03-06T07:28:16.476582+0000","last_undegraded":"2024-03-06T07:28:16.476582+0000","last_fullsized":"2024-03-06T07:28:16.476582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:23.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953198+0000","last_change":"2024-03-06T07:28:15.480000+0000","last_active":"2024-03-06T07:28:16.953198+0000","last_peered":"2024-03-06T07:28:16.953198+0000","last_clean":"2024-03-06T07:28:16.953198+0000","last_became_active":"2024-03-06T07:28:15.479711+0000","last_became_peered":"2024-03-06T07:28:15.479711+0000","last_unstale":"2024-03-06T07:28:16.953198+0000","last_undegraded":"2024-03-06T07:28:16.953198+0000","last_fullsized":"2024-03-06T07:28:16.953198+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:35.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473620+0000","last_change":"2024-03-06T07:28:14.473380+0000","last_active":"2024-03-06T07:28:16.473620+0000","last_peered":"2024-03-06T07:28:16.473620+0000","last_clean":"2024-03-06T07:28:16.473620+0000","last_became_active":"2024-03-06T07:28:14.473115+0000","last_became_peered":"2024-03-06T07:28:14.473115+0000","last_unstale":"2024-03-06T07:28:16.473620+0000","last_undegraded":"2024-03-06T07:28:16.473620+0000","last_fullsized":"2024-03-06T07:28:16.473620+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:09.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477096+0000","last_change":"2024-03-06T07:28:15.477608+0000","last_active":"2024-03-06T07:28:16.477096+0000","last_peered":"2024-03-06T07:28:16.477096+0000","last_clean":"2024-03-06T07:28:16.477096+0000","last_became_active":"2024-03-06T07:28:15.477333+0000","last_became_peered":"2024-03-06T07:28:15.477333+0000","last_unstale":"2024-03-06T07:28:16.477096+0000","last_undegraded":"2024-03-06T07:28:16.477096+0000","last_fullsized":"2024-03-06T07:28:16.477096+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:08.522231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475431+0000","last_change":"2024-03-06T07:28:14.474588+0000","last_active":"2024-03-06T07:28:16.475431+0000","last_peered":"2024-03-06T07:28:16.475431+0000","last_clean":"2024-03-06T07:28:16.475431+0000","last_became_active":"2024-03-06T07:28:14.474229+0000","last_became_peered":"2024-03-06T07:28:14.474229+0000","last_unstale":"2024-03-06T07:28:16.475431+0000","last_undegraded":"2024-03-06T07:28:16.475431+0000","last_fullsized":"2024-03-06T07:28:16.475431+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:21.471321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952624+0000","last_change":"2024-03-06T07:28:15.490269+0000","last_active":"2024-03-06T07:28:16.952624+0000","last_peered":"2024-03-06T07:28:16.952624+0000","last_clean":"2024-03-06T07:28:16.952624+0000","last_became_active":"2024-03-06T07:28:15.490009+0000","last_became_peered":"2024-03-06T07:28:15.490009+0000","last_unstale":"2024-03-06T07:28:16.952624+0000","last_undegraded":"2024-03-06T07:28:16.952624+0000","last_fullsized":"2024-03-06T07:28:16.952624+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:57.021055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952783+0000","last_change":"2024-03-06T07:28:14.472626+0000","last_active":"2024-03-06T07:28:16.952783+0000","last_peered":"2024-03-06T07:28:16.952783+0000","last_clean":"2024-03-06T07:28:16.952783+0000","last_became_active":"2024-03-06T07:28:14.472023+0000","last_became_peered":"2024-03-06T07:28:14.472023+0000","last_unstale":"2024-03-06T07:28:16.952783+0000","last_undegraded":"2024-03-06T07:28:16.952783+0000","last_fullsized":"2024-03-06T07:28:16.952783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:47.568139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":502,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952367+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:28:16.952367+0000","last_peered":"2024-03-06T07:28:16.952367+0000","last_clean":"2024-03-06T07:28:16.952367+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:28:16.952367+0000","last_undegraded":"2024-03-06T07:28:16.952367+0000","last_fullsized":"2024-03-06T07:28:16.952367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33716,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338124,"statfs":{"total":96636764160,"available":96602238976,"internally_reserved":0,"allocated":2596864,"data_stored":1823006,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33656,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338184,"statfs":{"total":96636764160,"available":96602300416,"internally_reserved":0,"allocated":2531328,"data_stored":1232626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":2523136,"data_stored":1232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2527232,"data_stored":1232584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2547712,"data_stored":1234764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2519040,"data_stored":1226474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2523136,"data_stored":1232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37868,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333972,"statfs":{"total":96636764160,"available":96597987328,"internally_reserved":0,"allocated":2588672,"data_stored":1817802,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:18.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.614+0000 7efc8c21c640 1 Processor -- start 2024-03-06T07:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.614+0000 7efc8c21c640 1 -- start start 2024-03-06T07:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc8c21c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc840c88c0 0x7efc84155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc8c21c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840c73d0 con 0x7efc8414f360 2024-03-06T07:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc8c21c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840be2e0 con 0x7efc84158b60 2024-03-06T07:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc8c21c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc84159700 con 0x7efc840c88c0 2024-03-06T07:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc897b7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efc840c88c0 0x7efc84155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc89fb8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efc8414f360 0x7efc840c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55046/0 (socket says 172.21.15.17:55046) 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.615+0000 7efc89fb8640 1 -- 172.21.15.17:0/408848836 learned_addr learned my addr 172.21.15.17:0/408848836 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.616+0000 7efc897b7640 1 -- 172.21.15.17:0/408848836 >> v1:172.21.15.163:6789/0 conn(0x7efc84158b60 legacy=0x7efc84158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.616+0000 7efc897b7640 1 -- 172.21.15.17:0/408848836 >> v1:172.21.15.17:6789/0 conn(0x7efc8414f360 legacy=0x7efc840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.616+0000 7efc897b7640 1 -- 172.21.15.17:0/408848836 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840c7670 con 0x7efc840c88c0 2024-03-06T07:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.616+0000 7efc88fb6640 1 -- 172.21.15.17:0/408848836 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496776120 0 0) 0x7efc840c73d0 con 0x7efc8414f360 2024-03-06T07:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.616+0000 7efc897b7640 1 --2- 172.21.15.17:0/408848836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c88c0 0x7efc84155c10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efc800097a0 tx=0x7efc8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce14d6da3c31edb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc88fb6640 1 -- 172.21.15.17:0/408848836 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc800315a0 con 0x7efc840c88c0 2024-03-06T07:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc88fb6640 1 -- 172.21.15.17:0/408848836 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc80037070 con 0x7efc840c88c0 2024-03-06T07:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc88fb6640 1 -- 172.21.15.17:0/408848836 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80031960 con 0x7efc840c88c0 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 -- 172.21.15.17:0/408848836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c88c0 msgr2=0x7efc84155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 --2- 172.21.15.17:0/408848836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c88c0 0x7efc84155c10 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efc800097a0 tx=0x7efc8002b840 comp rx=0 tx=0).stop 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 -- 172.21.15.17:0/408848836 shutdown_connections 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 --2- 172.21.15.17:0/408848836 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc840c88c0 0x7efc84155c10 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 -- 172.21.15.17:0/408848836 >> 172.21.15.17:0/408848836 conn(0x7efc840ac570 msgr2=0x7efc840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 -- 172.21.15.17:0/408848836 shutdown_connections 2024-03-06T07:28:18.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.617+0000 7efc8c21c640 1 -- 172.21.15.17:0/408848836 wait complete. 2024-03-06T07:28:18.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.618+0000 7efc8c21c640 1 Processor -- start 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.618+0000 7efc8c21c640 1 -- start start 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c88c0 0x7efc8414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc8414f360 0x7efc840c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c7c30 con 0x7efc84158b60 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c5a50 con 0x7efc840c88c0 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8c21c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c5bd0 con 0x7efc8414f360 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50890/0 (socket says 172.21.15.17:50890) 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 -- 172.21.15.17:0/3600938581 learned_addr learned my addr 172.21.15.17:0/3600938581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc897b7640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc8414f360 0x7efc840c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc89fb8640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c88c0 0x7efc8414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 -- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc8414f360 msgr2=0x7efc840c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc8414f360 0x7efc840c0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 -- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c88c0 msgr2=0x7efc8414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c88c0 0x7efc8414b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.619+0000 7efc8a7b9640 1 -- 172.21.15.17:0/3600938581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8414bf00 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc8a7b9640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b970 tx=0x7efc7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c019070 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7c00cb40 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c014440 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8414b6e0 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.620+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc840bf280 con 0x7efc84158b60 2024-03-06T07:28:18.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.621+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc8404f330 con 0x7efc84158b60 2024-03-06T07:28:18.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.622+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc7c01c070 con 0x7efc84158b60 2024-03-06T07:28:18.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.622+0000 7efc7a7fc640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 0x7efc6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.622+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efc7c089fb0 con 0x7efc84158b60 2024-03-06T07:28:18.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.622+0000 7efc89fb8640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 0x7efc6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.623+0000 7efc89fb8640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 0x7efc6806d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.626+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efc7c058ca0 con 0x7efc84158b60 2024-03-06T07:28:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.804+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc840c8d50 con 0x7efc6806b4d0 2024-03-06T07:28:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.810+0000 7efc7a7fc640 1 -- 172.21.15.17:0/3600938581 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252563 (secure 0 0 0) 0x7efc840c8d50 con 0x7efc6806b4d0 2024-03-06T07:28:18.814 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:18.815 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 msgr2=0x7efc6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 0x7efc6806d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efc74007fd0 tx=0x7efc74009040 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 msgr2=0x7efc8414aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b970 tx=0x7efc7c00be30 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 shutdown_connections 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efc6806b4d0 0x7efc6806d990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc84158b60 0x7efc8414aa20 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc8414f360 0x7efc840c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 --2- 172.21.15.17:0/3600938581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc840c88c0 0x7efc8414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 >> 172.21.15.17:0/3600938581 conn(0x7efc840ac570 msgr2=0x7efc8414d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 shutdown_connections 2024-03-06T07:28:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.813+0000 7efc8c21c640 1 -- 172.21.15.17:0/3600938581 wait complete. 2024-03-06T07:28:18.825 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-03-06T07:28:17.453109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281816,"kb_used_data":19880,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754692904,"statfs":{"total":773094113280,"available":772805533696,"internally_reserved":0,"allocated":20357120,"data_stored":11032284,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":268095028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.004368"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952947+0000","last_change":"2024-03-06T07:28:15.482261+0000","last_active":"2024-03-06T07:28:16.952947+0000","last_peered":"2024-03-06T07:28:16.952947+0000","last_clean":"2024-03-06T07:28:16.952947+0000","last_became_active":"2024-03-06T07:28:15.482000+0000","last_became_peered":"2024-03-06T07:28:15.482000+0000","last_unstale":"2024-03-06T07:28:16.952947+0000","last_undegraded":"2024-03-06T07:28:16.952947+0000","last_fullsized":"2024-03-06T07:28:16.952947+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:20.675073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953057+0000","last_change":"2024-03-06T07:28:15.486936+0000","last_active":"2024-03-06T07:28:16.953057+0000","last_peered":"2024-03-06T07:28:16.953057+0000","last_clean":"2024-03-06T07:28:16.953057+0000","last_became_active":"2024-03-06T07:28:15.486503+0000","last_became_peered":"2024-03-06T07:28:15.486503+0000","last_unstale":"2024-03-06T07:28:16.953057+0000","last_undegraded":"2024-03-06T07:28:16.953057+0000","last_fullsized":"2024-03-06T07:28:16.953057+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:40:50.968467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473978+0000","last_change":"2024-03-06T07:28:15.475549+0000","last_active":"2024-03-06T07:28:16.473978+0000","last_peered":"2024-03-06T07:28:16.473978+0000","last_clean":"2024-03-06T07:28:16.473978+0000","last_became_active":"2024-03-06T07:28:15.475253+0000","last_became_peered":"2024-03-06T07:28:15.475253+0000","last_unstale":"2024-03-06T07:28:16.473978+0000","last_undegraded":"2024-03-06T07:28:16.473978+0000","last_fullsized":"2024-03-06T07:28:16.473978+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:09.652727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473962+0000","last_change":"2024-03-06T07:28:15.477564+0000","last_active":"2024-03-06T07:28:16.473962+0000","last_peered":"2024-03-06T07:28:16.473962+0000","last_clean":"2024-03-06T07:28:16.473962+0000","last_became_active":"2024-03-06T07:28:15.477255+0000","last_became_peered":"2024-03-06T07:28:15.477255+0000","last_unstale":"2024-03-06T07:28:16.473962+0000","last_undegraded":"2024-03-06T07:28:16.473962+0000","last_fullsized":"2024-03-06T07:28:16.473962+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:02.439995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477064+0000","last_change":"2024-03-06T07:28:15.472674+0000","last_active":"2024-03-06T07:28:16.477064+0000","last_peered":"2024-03-06T07:28:16.477064+0000","last_clean":"2024-03-06T07:28:16.477064+0000","last_became_active":"2024-03-06T07:28:15.472254+0000","last_became_peered":"2024-03-06T07:28:15.472254+0000","last_unstale":"2024-03-06T07:28:16.477064+0000","last_undegraded":"2024-03-06T07:28:16.477064+0000","last_fullsized":"2024-03-06T07:28:16.477064+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:32.366142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952829+0000","last_change":"2024-03-06T07:28:15.487583+0000","last_active":"2024-03-06T07:28:16.952829+0000","last_peered":"2024-03-06T07:28:16.952829+0000","last_clean":"2024-03-06T07:28:16.952829+0000","last_became_active":"2024-03-06T07:28:15.487324+0000","last_became_peered":"2024-03-06T07:28:15.487324+0000","last_unstale":"2024-03-06T07:28:16.952829+0000","last_undegraded":"2024-03-06T07:28:16.952829+0000","last_fullsized":"2024-03-06T07:28:16.952829+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:39:53.553871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952301+0000","last_change":"2024-03-06T07:28:15.487249+0000","last_active":"2024-03-06T07:28:16.952301+0000","last_peered":"2024-03-06T07:28:16.952301+0000","last_clean":"2024-03-06T07:28:16.952301+0000","last_became_active":"2024-03-06T07:28:15.486946+0000","last_became_peered":"2024-03-06T07:28:15.486946+0000","last_unstale":"2024-03-06T07:28:16.952301+0000","last_undegraded":"2024-03-06T07:28:16.952301+0000","last_fullsized":"2024-03-06T07:28:16.952301+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:13.603559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476050+0000","last_change":"2024-03-06T07:28:15.478455+0000","last_active":"2024-03-06T07:28:16.476050+0000","last_peered":"2024-03-06T07:28:16.476050+0000","last_clean":"2024-03-06T07:28:16.476050+0000","last_became_active":"2024-03-06T07:28:15.478197+0000","last_became_peered":"2024-03-06T07:28:15.478197+0000","last_unstale":"2024-03-06T07:28:16.476050+0000","last_undegraded":"2024-03-06T07:28:16.476050+0000","last_fullsized":"2024-03-06T07:28:16.476050+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:31:14.145099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953123+0000","last_change":"2024-03-06T07:28:15.481324+0000","last_active":"2024-03-06T07:28:16.953123+0000","last_peered":"2024-03-06T07:28:16.953123+0000","last_clean":"2024-03-06T07:28:16.953123+0000","last_became_active":"2024-03-06T07:28:15.481042+0000","last_became_peered":"2024-03-06T07:28:15.481042+0000","last_unstale":"2024-03-06T07:28:16.953123+0000","last_undegraded":"2024-03-06T07:28:16.953123+0000","last_fullsized":"2024-03-06T07:28:16.953123+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:58.689652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474176+0000","last_change":"2024-03-06T07:28:15.476284+0000","last_active":"2024-03-06T07:28:16.474176+0000","last_peered":"2024-03-06T07:28:16.474176+0000","last_clean":"2024-03-06T07:28:16.474176+0000","last_became_active":"2024-03-06T07:28:15.476001+0000","last_became_peered":"2024-03-06T07:28:15.476001+0000","last_unstale":"2024-03-06T07:28:16.474176+0000","last_undegraded":"2024-03-06T07:28:16.474176+0000","last_fullsized":"2024-03-06T07:28:16.474176+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:37:14.556084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474050+0000","last_change":"2024-03-06T07:28:15.479448+0000","last_active":"2024-03-06T07:28:16.474050+0000","last_peered":"2024-03-06T07:28:16.474050+0000","last_clean":"2024-03-06T07:28:16.474050+0000","last_became_active":"2024-03-06T07:28:15.479090+0000","last_became_peered":"2024-03-06T07:28:15.479090+0000","last_unstale":"2024-03-06T07:28:16.474050+0000","last_undegraded":"2024-03-06T07:28:16.474050+0000","last_fullsized":"2024-03-06T07:28:16.474050+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:58:14.816311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953032+0000","last_change":"2024-03-06T07:28:15.485110+0000","last_active":"2024-03-06T07:28:16.953032+0000","last_peered":"2024-03-06T07:28:16.953032+0000","last_clean":"2024-03-06T07:28:16.953032+0000","last_became_active":"2024-03-06T07:28:15.484843+0000","last_became_peered":"2024-03-06T07:28:15.484843+0000","last_unstale":"2024-03-06T07:28:16.953032+0000","last_undegraded":"2024-03-06T07:28:16.953032+0000","last_fullsized":"2024-03-06T07:28:16.953032+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:37:10.396815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476822+0000","last_change":"2024-03-06T07:28:15.474580+0000","last_active":"2024-03-06T07:28:16.476822+0000","last_peered":"2024-03-06T07:28:16.476822+0000","last_clean":"2024-03-06T07:28:16.476822+0000","last_became_active":"2024-03-06T07:28:15.474312+0000","last_became_peered":"2024-03-06T07:28:15.474312+0000","last_unstale":"2024-03-06T07:28:16.476822+0000","last_undegraded":"2024-03-06T07:28:16.476822+0000","last_fullsized":"2024-03-06T07:28:16.476822+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:27.760693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952007+0000","last_change":"2024-03-06T07:28:15.470286+0000","last_active":"2024-03-06T07:28:16.952007+0000","last_peered":"2024-03-06T07:28:16.952007+0000","last_clean":"2024-03-06T07:28:16.952007+0000","last_became_active":"2024-03-06T07:28:15.470021+0000","last_became_peered":"2024-03-06T07:28:15.470021+0000","last_unstale":"2024-03-06T07:28:16.952007+0000","last_undegraded":"2024-03-06T07:28:16.952007+0000","last_fullsized":"2024-03-06T07:28:16.952007+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:09.955331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952759+0000","last_change":"2024-03-06T07:28:15.470875+0000","last_active":"2024-03-06T07:28:16.952759+0000","last_peered":"2024-03-06T07:28:16.952759+0000","last_clean":"2024-03-06T07:28:16.952759+0000","last_became_active":"2024-03-06T07:28:15.470585+0000","last_became_peered":"2024-03-06T07:28:15.470585+0000","last_unstale":"2024-03-06T07:28:16.952759+0000","last_undegraded":"2024-03-06T07:28:16.952759+0000","last_fullsized":"2024-03-06T07:28:16.952759+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:53.847237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473616+0000","last_change":"2024-03-06T07:28:15.476490+0000","last_active":"2024-03-06T07:28:16.473616+0000","last_peered":"2024-03-06T07:28:16.473616+0000","last_clean":"2024-03-06T07:28:16.473616+0000","last_became_active":"2024-03-06T07:28:15.476181+0000","last_became_peered":"2024-03-06T07:28:15.476181+0000","last_unstale":"2024-03-06T07:28:16.473616+0000","last_undegraded":"2024-03-06T07:28:16.473616+0000","last_fullsized":"2024-03-06T07:28:16.473616+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:06:45.862903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477217+0000","last_change":"2024-03-06T07:28:15.468645+0000","last_active":"2024-03-06T07:28:16.477217+0000","last_peered":"2024-03-06T07:28:16.477217+0000","last_clean":"2024-03-06T07:28:16.477217+0000","last_became_active":"2024-03-06T07:28:15.468355+0000","last_became_peered":"2024-03-06T07:28:15.468355+0000","last_unstale":"2024-03-06T07:28:16.477217+0000","last_undegraded":"2024-03-06T07:28:16.477217+0000","last_fullsized":"2024-03-06T07:28:16.477217+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:24.316421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474269+0000","last_change":"2024-03-06T07:28:15.474022+0000","last_active":"2024-03-06T07:28:16.474269+0000","last_peered":"2024-03-06T07:28:16.474269+0000","last_clean":"2024-03-06T07:28:16.474269+0000","last_became_active":"2024-03-06T07:28:15.473656+0000","last_became_peered":"2024-03-06T07:28:15.473656+0000","last_unstale":"2024-03-06T07:28:16.474269+0000","last_undegraded":"2024-03-06T07:28:16.474269+0000","last_fullsized":"2024-03-06T07:28:16.474269+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:17.346689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953109+0000","last_change":"2024-03-06T07:28:15.475546+0000","last_active":"2024-03-06T07:28:16.953109+0000","last_peered":"2024-03-06T07:28:16.953109+0000","last_clean":"2024-03-06T07:28:16.953109+0000","last_became_active":"2024-03-06T07:28:15.475268+0000","last_became_peered":"2024-03-06T07:28:15.475268+0000","last_unstale":"2024-03-06T07:28:16.953109+0000","last_undegraded":"2024-03-06T07:28:16.953109+0000","last_fullsized":"2024-03-06T07:28:16.953109+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:02.394056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952471+0000","last_change":"2024-03-06T07:28:15.469070+0000","last_active":"2024-03-06T07:28:16.952471+0000","last_peered":"2024-03-06T07:28:16.952471+0000","last_clean":"2024-03-06T07:28:16.952471+0000","last_became_active":"2024-03-06T07:28:15.468785+0000","last_became_peered":"2024-03-06T07:28:15.468785+0000","last_unstale":"2024-03-06T07:28:16.952471+0000","last_undegraded":"2024-03-06T07:28:16.952471+0000","last_fullsized":"2024-03-06T07:28:16.952471+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:25:53.622360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952381+0000","last_change":"2024-03-06T07:28:15.479526+0000","last_active":"2024-03-06T07:28:16.952381+0000","last_peered":"2024-03-06T07:28:16.952381+0000","last_clean":"2024-03-06T07:28:16.952381+0000","last_became_active":"2024-03-06T07:28:15.479224+0000","last_became_peered":"2024-03-06T07:28:15.479224+0000","last_unstale":"2024-03-06T07:28:16.952381+0000","last_undegraded":"2024-03-06T07:28:16.952381+0000","last_fullsized":"2024-03-06T07:28:16.952381+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:06.690342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476217+0000","last_change":"2024-03-06T07:28:15.476992+0000","last_active":"2024-03-06T07:28:16.476217+0000","last_peered":"2024-03-06T07:28:16.476217+0000","last_clean":"2024-03-06T07:28:16.476217+0000","last_became_active":"2024-03-06T07:28:15.476720+0000","last_became_peered":"2024-03-06T07:28:15.476720+0000","last_unstale":"2024-03-06T07:28:16.476217+0000","last_undegraded":"2024-03-06T07:28:16.476217+0000","last_fullsized":"2024-03-06T07:28:16.476217+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:31:31.930810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952881+0000","last_change":"2024-03-06T07:28:15.476073+0000","last_active":"2024-03-06T07:28:16.952881+0000","last_peered":"2024-03-06T07:28:16.952881+0000","last_clean":"2024-03-06T07:28:16.952881+0000","last_became_active":"2024-03-06T07:28:15.475737+0000","last_became_peered":"2024-03-06T07:28:15.475737+0000","last_unstale":"2024-03-06T07:28:16.952881+0000","last_undegraded":"2024-03-06T07:28:16.952881+0000","last_fullsized":"2024-03-06T07:28:16.952881+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:10.552429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475491+0000","last_change":"2024-03-06T07:28:15.476093+0000","last_active":"2024-03-06T07:28:16.475491+0000","last_peered":"2024-03-06T07:28:16.475491+0000","last_clean":"2024-03-06T07:28:16.475491+0000","last_became_active":"2024-03-06T07:28:15.475717+0000","last_became_peered":"2024-03-06T07:28:15.475717+0000","last_unstale":"2024-03-06T07:28:16.475491+0000","last_undegraded":"2024-03-06T07:28:16.475491+0000","last_fullsized":"2024-03-06T07:28:16.475491+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:52.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953272+0000","last_change":"2024-03-06T07:28:15.472416+0000","last_active":"2024-03-06T07:28:16.953272+0000","last_peered":"2024-03-06T07:28:16.953272+0000","last_clean":"2024-03-06T07:28:16.953272+0000","last_became_active":"2024-03-06T07:28:15.472156+0000","last_became_peered":"2024-03-06T07:28:15.472156+0000","last_unstale":"2024-03-06T07:28:16.953272+0000","last_undegraded":"2024-03-06T07:28:16.953272+0000","last_fullsized":"2024-03-06T07:28:16.953272+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:04:51.039144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952826+0000","last_change":"2024-03-06T07:28:15.487221+0000","last_active":"2024-03-06T07:28:16.952826+0000","last_peered":"2024-03-06T07:28:16.952826+0000","last_clean":"2024-03-06T07:28:16.952826+0000","last_became_active":"2024-03-06T07:28:15.486932+0000","last_became_peered":"2024-03-06T07:28:15.486932+0000","last_unstale":"2024-03-06T07:28:16.952826+0000","last_undegraded":"2024-03-06T07:28:16.952826+0000","last_fullsized":"2024-03-06T07:28:16.952826+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476773+0000","last_change":"2024-03-06T07:28:15.476748+0000","last_active":"2024-03-06T07:28:16.476773+0000","last_peered":"2024-03-06T07:28:16.476773+0000","last_clean":"2024-03-06T07:28:16.476773+0000","last_became_active":"2024-03-06T07:28:15.476462+0000","last_became_peered":"2024-03-06T07:28:15.476462+0000","last_unstale":"2024-03-06T07:28:16.476773+0000","last_undegraded":"2024-03-06T07:28:16.476773+0000","last_fullsized":"2024-03-06T07:28:16.476773+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:06.690342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953092+0000","last_change":"2024-03-06T07:28:15.483660+0000","last_active":"2024-03-06T07:28:16.953092+0000","last_peered":"2024-03-06T07:28:16.953092+0000","last_clean":"2024-03-06T07:28:16.953092+0000","last_became_active":"2024-03-06T07:28:15.483379+0000","last_became_peered":"2024-03-06T07:28:15.483379+0000","last_unstale":"2024-03-06T07:28:16.953092+0000","last_undegraded":"2024-03-06T07:28:16.953092+0000","last_fullsized":"2024-03-06T07:28:16.953092+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:54:40.262748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951948+0000","last_change":"2024-03-06T07:28:15.476365+0000","last_active":"2024-03-06T07:28:16.951948+0000","last_peered":"2024-03-06T07:28:16.951948+0000","last_clean":"2024-03-06T07:28:16.951948+0000","last_became_active":"2024-03-06T07:28:15.476090+0000","last_became_peered":"2024-03-06T07:28:15.476090+0000","last_unstale":"2024-03-06T07:28:16.951948+0000","last_undegraded":"2024-03-06T07:28:16.951948+0000","last_fullsized":"2024-03-06T07:28:16.951948+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:18.891603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952158+0000","last_change":"2024-03-06T07:28:15.477135+0000","last_active":"2024-03-06T07:28:16.952158+0000","last_peered":"2024-03-06T07:28:16.952158+0000","last_clean":"2024-03-06T07:28:16.952158+0000","last_became_active":"2024-03-06T07:28:15.476869+0000","last_became_peered":"2024-03-06T07:28:15.476869+0000","last_unstale":"2024-03-06T07:28:16.952158+0000","last_undegraded":"2024-03-06T07:28:16.952158+0000","last_fullsized":"2024-03-06T07:28:16.952158+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:44:02.439995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951970+0000","last_change":"2024-03-06T07:28:15.480910+0000","last_active":"2024-03-06T07:28:16.951970+0000","last_peered":"2024-03-06T07:28:16.951970+0000","last_clean":"2024-03-06T07:28:16.951970+0000","last_became_active":"2024-03-06T07:28:15.478466+0000","last_became_peered":"2024-03-06T07:28:15.478466+0000","last_unstale":"2024-03-06T07:28:16.951970+0000","last_undegraded":"2024-03-06T07:28:16.951970+0000","last_fullsized":"2024-03-06T07:28:16.951970+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:07:00.206292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952616+0000","last_change":"2024-03-06T07:28:15.476973+0000","last_active":"2024-03-06T07:28:16.952616+0000","last_peered":"2024-03-06T07:28:16.952616+0000","last_clean":"2024-03-06T07:28:16.952616+0000","last_became_active":"2024-03-06T07:28:15.476711+0000","last_became_peered":"2024-03-06T07:28:15.476711+0000","last_unstale":"2024-03-06T07:28:16.952616+0000","last_undegraded":"2024-03-06T07:28:16.952616+0000","last_fullsized":"2024-03-06T07:28:16.952616+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:40.416289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952005+0000","last_change":"2024-03-06T07:28:15.486635+0000","last_active":"2024-03-06T07:28:16.952005+0000","last_peered":"2024-03-06T07:28:16.952005+0000","last_clean":"2024-03-06T07:28:16.952005+0000","last_became_active":"2024-03-06T07:28:15.480805+0000","last_became_peered":"2024-03-06T07:28:15.480805+0000","last_unstale":"2024-03-06T07:28:16.952005+0000","last_undegraded":"2024-03-06T07:28:16.952005+0000","last_fullsized":"2024-03-06T07:28:16.952005+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:33:09.583467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"198'4","reported_seq":18,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952954+0000","last_change":"2024-03-06T07:28:14.471148+0000","last_active":"2024-03-06T07:28:16.952954+0000","last_peered":"2024-03-06T07:28:16.952954+0000","last_clean":"2024-03-06T07:28:16.952954+0000","last_became_active":"2024-03-06T07:28:14.470830+0000","last_became_peered":"2024-03-06T07:28:14.470830+0000","last_unstale":"2024-03-06T07:28:16.952954+0000","last_undegraded":"2024-03-06T07:28:16.952954+0000","last_fullsized":"2024-03-06T07:28:16.952954+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:30:49.743612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953065+0000","last_change":"2024-03-06T07:28:15.475096+0000","last_active":"2024-03-06T07:28:16.953065+0000","last_peered":"2024-03-06T07:28:16.953065+0000","last_clean":"2024-03-06T07:28:16.953065+0000","last_became_active":"2024-03-06T07:28:15.474822+0000","last_became_peered":"2024-03-06T07:28:15.474822+0000","last_unstale":"2024-03-06T07:28:16.953065+0000","last_undegraded":"2024-03-06T07:28:16.953065+0000","last_fullsized":"2024-03-06T07:28:16.953065+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:09:11.330344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476183+0000","last_change":"2024-03-06T07:28:14.474168+0000","last_active":"2024-03-06T07:28:16.476183+0000","last_peered":"2024-03-06T07:28:16.476183+0000","last_clean":"2024-03-06T07:28:16.476183+0000","last_became_active":"2024-03-06T07:28:14.473851+0000","last_became_peered":"2024-03-06T07:28:14.473851+0000","last_unstale":"2024-03-06T07:28:16.476183+0000","last_undegraded":"2024-03-06T07:28:16.476183+0000","last_fullsized":"2024-03-06T07:28:16.476183+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:15:14.997022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952789+0000","last_change":"2024-03-06T07:28:15.488767+0000","last_active":"2024-03-06T07:28:16.952789+0000","last_peered":"2024-03-06T07:28:16.952789+0000","last_clean":"2024-03-06T07:28:16.952789+0000","last_became_active":"2024-03-06T07:28:15.488299+0000","last_became_peered":"2024-03-06T07:28:15.488299+0000","last_unstale":"2024-03-06T07:28:16.952789+0000","last_undegraded":"2024-03-06T07:28:16.952789+0000","last_fullsized":"2024-03-06T07:28:16.952789+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:52:43.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952334+0000","last_change":"2024-03-06T07:28:14.474406+0000","last_active":"2024-03-06T07:28:16.952334+0000","last_peered":"2024-03-06T07:28:16.952334+0000","last_clean":"2024-03-06T07:28:16.952334+0000","last_became_active":"2024-03-06T07:28:14.474128+0000","last_became_peered":"2024-03-06T07:28:14.474128+0000","last_unstale":"2024-03-06T07:28:16.952334+0000","last_undegraded":"2024-03-06T07:28:16.952334+0000","last_fullsized":"2024-03-06T07:28:16.952334+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:38.525288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952182+0000","last_change":"2024-03-06T07:28:15.470696+0000","last_active":"2024-03-06T07:28:16.952182+0000","last_peered":"2024-03-06T07:28:16.952182+0000","last_clean":"2024-03-06T07:28:16.952182+0000","last_became_active":"2024-03-06T07:28:15.470373+0000","last_became_peered":"2024-03-06T07:28:15.470373+0000","last_unstale":"2024-03-06T07:28:16.952182+0000","last_undegraded":"2024-03-06T07:28:16.952182+0000","last_fullsized":"2024-03-06T07:28:16.952182+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:41:09.652727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953043+0000","last_change":"2024-03-06T07:28:14.471583+0000","last_active":"2024-03-06T07:28:16.953043+0000","last_peered":"2024-03-06T07:28:16.953043+0000","last_clean":"2024-03-06T07:28:16.953043+0000","last_became_active":"2024-03-06T07:28:14.471286+0000","last_became_peered":"2024-03-06T07:28:14.471286+0000","last_unstale":"2024-03-06T07:28:16.953043+0000","last_undegraded":"2024-03-06T07:28:16.953043+0000","last_fullsized":"2024-03-06T07:28:16.953043+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:18.349428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952909+0000","last_change":"2024-03-06T07:28:15.479577+0000","last_active":"2024-03-06T07:28:16.952909+0000","last_peered":"2024-03-06T07:28:16.952909+0000","last_clean":"2024-03-06T07:28:16.952909+0000","last_became_active":"2024-03-06T07:28:15.479308+0000","last_became_peered":"2024-03-06T07:28:15.479308+0000","last_unstale":"2024-03-06T07:28:16.952909+0000","last_undegraded":"2024-03-06T07:28:16.952909+0000","last_fullsized":"2024-03-06T07:28:16.952909+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:32.366142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952269+0000","last_change":"2024-03-06T07:28:14.479954+0000","last_active":"2024-03-06T07:28:16.952269+0000","last_peered":"2024-03-06T07:28:16.952269+0000","last_clean":"2024-03-06T07:28:16.952269+0000","last_became_active":"2024-03-06T07:28:14.479664+0000","last_became_peered":"2024-03-06T07:28:14.479664+0000","last_unstale":"2024-03-06T07:28:16.952269+0000","last_undegraded":"2024-03-06T07:28:16.952269+0000","last_fullsized":"2024-03-06T07:28:16.952269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:49:04.403220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474111+0000","last_change":"2024-03-06T07:28:15.483469+0000","last_active":"2024-03-06T07:28:16.474111+0000","last_peered":"2024-03-06T07:28:16.474111+0000","last_clean":"2024-03-06T07:28:16.474111+0000","last_became_active":"2024-03-06T07:28:15.483167+0000","last_became_peered":"2024-03-06T07:28:15.483167+0000","last_unstale":"2024-03-06T07:28:16.474111+0000","last_undegraded":"2024-03-06T07:28:16.474111+0000","last_fullsized":"2024-03-06T07:28:16.474111+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:21:57.359258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473598+0000","last_change":"2024-03-06T07:28:14.468330+0000","last_active":"2024-03-06T07:28:16.473598+0000","last_peered":"2024-03-06T07:28:16.473598+0000","last_clean":"2024-03-06T07:28:16.473598+0000","last_became_active":"2024-03-06T07:28:14.468029+0000","last_became_peered":"2024-03-06T07:28:14.468029+0000","last_unstale":"2024-03-06T07:28:16.473598+0000","last_undegraded":"2024-03-06T07:28:16.473598+0000","last_fullsized":"2024-03-06T07:28:16.473598+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:14.598213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477054+0000","last_change":"2024-03-06T07:28:15.476659+0000","last_active":"2024-03-06T07:28:16.477054+0000","last_peered":"2024-03-06T07:28:16.477054+0000","last_clean":"2024-03-06T07:28:16.477054+0000","last_became_active":"2024-03-06T07:28:15.476377+0000","last_became_peered":"2024-03-06T07:28:15.476377+0000","last_unstale":"2024-03-06T07:28:16.477054+0000","last_undegraded":"2024-03-06T07:28:16.477054+0000","last_fullsized":"2024-03-06T07:28:16.477054+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:36:16.978101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952706+0000","last_change":"2024-03-06T07:28:14.468094+0000","last_active":"2024-03-06T07:28:16.952706+0000","last_peered":"2024-03-06T07:28:16.952706+0000","last_clean":"2024-03-06T07:28:16.952706+0000","last_became_active":"2024-03-06T07:28:14.467785+0000","last_became_peered":"2024-03-06T07:28:14.467785+0000","last_unstale":"2024-03-06T07:28:16.952706+0000","last_undegraded":"2024-03-06T07:28:16.952706+0000","last_fullsized":"2024-03-06T07:28:16.952706+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:53:19.841681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952477+0000","last_change":"2024-03-06T07:28:15.479770+0000","last_active":"2024-03-06T07:28:16.952477+0000","last_peered":"2024-03-06T07:28:16.952477+0000","last_clean":"2024-03-06T07:28:16.952477+0000","last_became_active":"2024-03-06T07:28:15.478996+0000","last_became_peered":"2024-03-06T07:28:15.478996+0000","last_unstale":"2024-03-06T07:28:16.952477+0000","last_undegraded":"2024-03-06T07:28:16.952477+0000","last_fullsized":"2024-03-06T07:28:16.952477+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:09.366966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952185+0000","last_change":"2024-03-06T07:28:14.479963+0000","last_active":"2024-03-06T07:28:16.952185+0000","last_peered":"2024-03-06T07:28:16.952185+0000","last_clean":"2024-03-06T07:28:16.952185+0000","last_became_active":"2024-03-06T07:28:14.479166+0000","last_became_peered":"2024-03-06T07:28:14.479166+0000","last_unstale":"2024-03-06T07:28:16.952185+0000","last_undegraded":"2024-03-06T07:28:16.952185+0000","last_fullsized":"2024-03-06T07:28:16.952185+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:43:56.877583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475940+0000","last_change":"2024-03-06T07:28:15.477776+0000","last_active":"2024-03-06T07:28:16.475940+0000","last_peered":"2024-03-06T07:28:16.475940+0000","last_clean":"2024-03-06T07:28:16.475940+0000","last_became_active":"2024-03-06T07:28:15.477007+0000","last_became_peered":"2024-03-06T07:28:15.477007+0000","last_unstale":"2024-03-06T07:28:16.475940+0000","last_undegraded":"2024-03-06T07:28:16.475940+0000","last_fullsized":"2024-03-06T07:28:16.475940+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:00:24.721149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474240+0000","last_change":"2024-03-06T07:28:14.474135+0000","last_active":"2024-03-06T07:28:16.474240+0000","last_peered":"2024-03-06T07:28:16.474240+0000","last_clean":"2024-03-06T07:28:16.474240+0000","last_became_active":"2024-03-06T07:28:14.473827+0000","last_became_peered":"2024-03-06T07:28:14.473827+0000","last_unstale":"2024-03-06T07:28:16.474240+0000","last_undegraded":"2024-03-06T07:28:16.474240+0000","last_fullsized":"2024-03-06T07:28:16.474240+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:05:52.845585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473720+0000","last_change":"2024-03-06T07:28:15.475645+0000","last_active":"2024-03-06T07:28:16.473720+0000","last_peered":"2024-03-06T07:28:16.473720+0000","last_clean":"2024-03-06T07:28:16.473720+0000","last_became_active":"2024-03-06T07:28:15.475347+0000","last_became_peered":"2024-03-06T07:28:15.475347+0000","last_unstale":"2024-03-06T07:28:16.473720+0000","last_undegraded":"2024-03-06T07:28:16.473720+0000","last_fullsized":"2024-03-06T07:28:16.473720+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:23:18.891603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"198'3","reported_seq":17,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477190+0000","last_change":"2024-03-06T07:28:14.476613+0000","last_active":"2024-03-06T07:28:16.477190+0000","last_peered":"2024-03-06T07:28:16.477190+0000","last_clean":"2024-03-06T07:28:16.477190+0000","last_became_active":"2024-03-06T07:28:14.476330+0000","last_became_peered":"2024-03-06T07:28:14.476330+0000","last_unstale":"2024-03-06T07:28:16.477190+0000","last_undegraded":"2024-03-06T07:28:16.477190+0000","last_fullsized":"2024-03-06T07:28:16.477190+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:48:47.459867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952329+0000","last_change":"2024-03-06T07:28:15.476982+0000","last_active":"2024-03-06T07:28:16.952329+0000","last_peered":"2024-03-06T07:28:16.952329+0000","last_clean":"2024-03-06T07:28:16.952329+0000","last_became_active":"2024-03-06T07:28:15.476715+0000","last_became_peered":"2024-03-06T07:28:15.476715+0000","last_unstale":"2024-03-06T07:28:16.952329+0000","last_undegraded":"2024-03-06T07:28:16.952329+0000","last_fullsized":"2024-03-06T07:28:16.952329+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:57.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"198'6","reported_seq":20,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953128+0000","last_change":"2024-03-06T07:28:14.480209+0000","last_active":"2024-03-06T07:28:16.953128+0000","last_peered":"2024-03-06T07:28:16.953128+0000","last_clean":"2024-03-06T07:28:16.953128+0000","last_became_active":"2024-03-06T07:28:14.479918+0000","last_became_peered":"2024-03-06T07:28:14.479918+0000","last_unstale":"2024-03-06T07:28:16.953128+0000","last_undegraded":"2024-03-06T07:28:16.953128+0000","last_fullsized":"2024-03-06T07:28:16.953128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:19.050940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952384+0000","last_change":"2024-03-06T07:28:15.486999+0000","last_active":"2024-03-06T07:28:16.952384+0000","last_peered":"2024-03-06T07:28:16.952384+0000","last_clean":"2024-03-06T07:28:16.952384+0000","last_became_active":"2024-03-06T07:28:15.486437+0000","last_became_peered":"2024-03-06T07:28:15.486437+0000","last_unstale":"2024-03-06T07:28:16.952384+0000","last_undegraded":"2024-03-06T07:28:16.952384+0000","last_fullsized":"2024-03-06T07:28:16.952384+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:24:13.380260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953098+0000","last_change":"2024-03-06T07:28:14.470176+0000","last_active":"2024-03-06T07:28:16.953098+0000","last_peered":"2024-03-06T07:28:16.953098+0000","last_clean":"2024-03-06T07:28:16.953098+0000","last_became_active":"2024-03-06T07:28:14.469841+0000","last_became_peered":"2024-03-06T07:28:14.469841+0000","last_unstale":"2024-03-06T07:28:16.953098+0000","last_undegraded":"2024-03-06T07:28:16.953098+0000","last_fullsized":"2024-03-06T07:28:16.953098+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:44:05.688690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952669+0000","last_change":"2024-03-06T07:28:15.474360+0000","last_active":"2024-03-06T07:28:16.952669+0000","last_peered":"2024-03-06T07:28:16.952669+0000","last_clean":"2024-03-06T07:28:16.952669+0000","last_became_active":"2024-03-06T07:28:15.473914+0000","last_became_peered":"2024-03-06T07:28:15.473914+0000","last_unstale":"2024-03-06T07:28:16.952669+0000","last_undegraded":"2024-03-06T07:28:16.952669+0000","last_fullsized":"2024-03-06T07:28:16.952669+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:19.351080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476483+0000","last_change":"2024-03-06T07:28:14.477371+0000","last_active":"2024-03-06T07:28:16.476483+0000","last_peered":"2024-03-06T07:28:16.476483+0000","last_clean":"2024-03-06T07:28:16.476483+0000","last_became_active":"2024-03-06T07:28:14.477123+0000","last_became_peered":"2024-03-06T07:28:14.477123+0000","last_unstale":"2024-03-06T07:28:16.476483+0000","last_undegraded":"2024-03-06T07:28:16.476483+0000","last_fullsized":"2024-03-06T07:28:16.476483+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:07.520579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952797+0000","last_change":"2024-03-06T07:28:15.472634+0000","last_active":"2024-03-06T07:28:16.952797+0000","last_peered":"2024-03-06T07:28:16.952797+0000","last_clean":"2024-03-06T07:28:16.952797+0000","last_became_active":"2024-03-06T07:28:15.471820+0000","last_became_peered":"2024-03-06T07:28:15.471820+0000","last_unstale":"2024-03-06T07:28:16.952797+0000","last_undegraded":"2024-03-06T07:28:16.952797+0000","last_fullsized":"2024-03-06T07:28:16.952797+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:03:15.599866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476221+0000","last_change":"2024-03-06T07:28:14.474122+0000","last_active":"2024-03-06T07:28:16.476221+0000","last_peered":"2024-03-06T07:28:16.476221+0000","last_clean":"2024-03-06T07:28:16.476221+0000","last_became_active":"2024-03-06T07:28:14.473820+0000","last_became_peered":"2024-03-06T07:28:14.473820+0000","last_unstale":"2024-03-06T07:28:16.476221+0000","last_undegraded":"2024-03-06T07:28:16.476221+0000","last_fullsized":"2024-03-06T07:28:16.476221+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:47.568139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477036+0000","last_change":"2024-03-06T07:28:15.467219+0000","last_active":"2024-03-06T07:28:16.477036+0000","last_peered":"2024-03-06T07:28:16.477036+0000","last_clean":"2024-03-06T07:28:16.477036+0000","last_became_active":"2024-03-06T07:28:15.466232+0000","last_became_peered":"2024-03-06T07:28:15.466232+0000","last_unstale":"2024-03-06T07:28:16.477036+0000","last_undegraded":"2024-03-06T07:28:16.477036+0000","last_fullsized":"2024-03-06T07:28:16.477036+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:09.366966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"198'8","reported_seq":22,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951895+0000","last_change":"2024-03-06T07:28:14.476089+0000","last_active":"2024-03-06T07:28:16.951895+0000","last_peered":"2024-03-06T07:28:16.951895+0000","last_clean":"2024-03-06T07:28:16.951895+0000","last_became_active":"2024-03-06T07:28:14.475792+0000","last_became_peered":"2024-03-06T07:28:14.475792+0000","last_unstale":"2024-03-06T07:28:16.951895+0000","last_undegraded":"2024-03-06T07:28:16.951895+0000","last_fullsized":"2024-03-06T07:28:16.951895+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:12:03.678148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475525+0000","last_change":"2024-03-06T07:28:15.472076+0000","last_active":"2024-03-06T07:28:16.475525+0000","last_peered":"2024-03-06T07:28:16.475525+0000","last_clean":"2024-03-06T07:28:16.475525+0000","last_became_active":"2024-03-06T07:28:15.471056+0000","last_became_peered":"2024-03-06T07:28:15.471056+0000","last_unstale":"2024-03-06T07:28:16.475525+0000","last_undegraded":"2024-03-06T07:28:16.475525+0000","last_fullsized":"2024-03-06T07:28:16.475525+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:56:13.603559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953354+0000","last_change":"2024-03-06T07:28:14.476332+0000","last_active":"2024-03-06T07:28:16.953354+0000","last_peered":"2024-03-06T07:28:16.953354+0000","last_clean":"2024-03-06T07:28:16.953354+0000","last_became_active":"2024-03-06T07:28:14.476061+0000","last_became_peered":"2024-03-06T07:28:14.476061+0000","last_unstale":"2024-03-06T07:28:16.953354+0000","last_undegraded":"2024-03-06T07:28:16.953354+0000","last_fullsized":"2024-03-06T07:28:16.953354+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:16:34.591052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476023+0000","last_change":"2024-03-06T07:28:15.471783+0000","last_active":"2024-03-06T07:28:16.476023+0000","last_peered":"2024-03-06T07:28:16.476023+0000","last_clean":"2024-03-06T07:28:16.476023+0000","last_became_active":"2024-03-06T07:28:15.471514+0000","last_became_peered":"2024-03-06T07:28:15.471514+0000","last_unstale":"2024-03-06T07:28:16.476023+0000","last_undegraded":"2024-03-06T07:28:16.476023+0000","last_fullsized":"2024-03-06T07:28:16.476023+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:48:57.996115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953204+0000","last_change":"2024-03-06T07:28:14.477580+0000","last_active":"2024-03-06T07:28:16.953204+0000","last_peered":"2024-03-06T07:28:16.953204+0000","last_clean":"2024-03-06T07:28:16.953204+0000","last_became_active":"2024-03-06T07:28:14.476526+0000","last_became_peered":"2024-03-06T07:28:14.476526+0000","last_unstale":"2024-03-06T07:28:16.953204+0000","last_undegraded":"2024-03-06T07:28:16.953204+0000","last_fullsized":"2024-03-06T07:28:16.953204+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:44:05.879351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.951959+0000","last_change":"2024-03-06T07:28:15.476701+0000","last_active":"2024-03-06T07:28:16.951959+0000","last_peered":"2024-03-06T07:28:16.951959+0000","last_clean":"2024-03-06T07:28:16.951959+0000","last_became_active":"2024-03-06T07:28:15.476433+0000","last_became_peered":"2024-03-06T07:28:15.476433+0000","last_unstale":"2024-03-06T07:28:16.951959+0000","last_undegraded":"2024-03-06T07:28:16.951959+0000","last_fullsized":"2024-03-06T07:28:16.951959+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:38.479080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475755+0000","last_change":"2024-03-06T07:28:14.473266+0000","last_active":"2024-03-06T07:28:16.475755+0000","last_peered":"2024-03-06T07:28:16.475755+0000","last_clean":"2024-03-06T07:28:16.475755+0000","last_became_active":"2024-03-06T07:28:14.472995+0000","last_became_peered":"2024-03-06T07:28:14.472995+0000","last_unstale":"2024-03-06T07:28:16.475755+0000","last_undegraded":"2024-03-06T07:28:16.475755+0000","last_fullsized":"2024-03-06T07:28:16.475755+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:16:22.648652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473879+0000","last_change":"2024-03-06T07:28:15.476734+0000","last_active":"2024-03-06T07:28:16.473879+0000","last_peered":"2024-03-06T07:28:16.473879+0000","last_clean":"2024-03-06T07:28:16.473879+0000","last_became_active":"2024-03-06T07:28:15.476449+0000","last_became_peered":"2024-03-06T07:28:15.476449+0000","last_unstale":"2024-03-06T07:28:16.473879+0000","last_undegraded":"2024-03-06T07:28:16.473879+0000","last_fullsized":"2024-03-06T07:28:16.473879+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:38:09.955331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952731+0000","last_change":"2024-03-06T07:28:14.468828+0000","last_active":"2024-03-06T07:28:16.952731+0000","last_peered":"2024-03-06T07:28:16.952731+0000","last_clean":"2024-03-06T07:28:16.952731+0000","last_became_active":"2024-03-06T07:28:14.468552+0000","last_became_peered":"2024-03-06T07:28:14.468552+0000","last_unstale":"2024-03-06T07:28:16.952731+0000","last_undegraded":"2024-03-06T07:28:16.952731+0000","last_fullsized":"2024-03-06T07:28:16.952731+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:06:08.365314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473895+0000","last_change":"2024-03-06T07:28:15.488981+0000","last_active":"2024-03-06T07:28:16.473895+0000","last_peered":"2024-03-06T07:28:16.473895+0000","last_clean":"2024-03-06T07:28:16.473895+0000","last_became_active":"2024-03-06T07:28:15.488675+0000","last_became_peered":"2024-03-06T07:28:15.488675+0000","last_unstale":"2024-03-06T07:28:16.473895+0000","last_undegraded":"2024-03-06T07:28:16.473895+0000","last_fullsized":"2024-03-06T07:28:16.473895+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:22.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473815+0000","last_change":"2024-03-06T07:28:14.477668+0000","last_active":"2024-03-06T07:28:16.473815+0000","last_peered":"2024-03-06T07:28:16.473815+0000","last_clean":"2024-03-06T07:28:16.473815+0000","last_became_active":"2024-03-06T07:28:14.477351+0000","last_became_peered":"2024-03-06T07:28:14.477351+0000","last_unstale":"2024-03-06T07:28:16.473815+0000","last_undegraded":"2024-03-06T07:28:16.473815+0000","last_fullsized":"2024-03-06T07:28:16.473815+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:04.324747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952318+0000","last_change":"2024-03-06T07:28:15.480926+0000","last_active":"2024-03-06T07:28:16.952318+0000","last_peered":"2024-03-06T07:28:16.952318+0000","last_clean":"2024-03-06T07:28:16.952318+0000","last_became_active":"2024-03-06T07:28:15.480656+0000","last_became_peered":"2024-03-06T07:28:15.480656+0000","last_unstale":"2024-03-06T07:28:16.952318+0000","last_undegraded":"2024-03-06T07:28:16.952318+0000","last_fullsized":"2024-03-06T07:28:16.952318+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:14:19.717811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476324+0000","last_change":"2024-03-06T07:28:14.474364+0000","last_active":"2024-03-06T07:28:16.476324+0000","last_peered":"2024-03-06T07:28:16.476324+0000","last_clean":"2024-03-06T07:28:16.476324+0000","last_became_active":"2024-03-06T07:28:14.474054+0000","last_became_peered":"2024-03-06T07:28:14.474054+0000","last_unstale":"2024-03-06T07:28:16.476324+0000","last_undegraded":"2024-03-06T07:28:16.476324+0000","last_fullsized":"2024-03-06T07:28:16.476324+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:16:06.532834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476020+0000","last_change":"2024-03-06T07:28:15.467927+0000","last_active":"2024-03-06T07:28:16.476020+0000","last_peered":"2024-03-06T07:28:16.476020+0000","last_clean":"2024-03-06T07:28:16.476020+0000","last_became_active":"2024-03-06T07:28:15.467662+0000","last_became_peered":"2024-03-06T07:28:15.467662+0000","last_unstale":"2024-03-06T07:28:16.476020+0000","last_undegraded":"2024-03-06T07:28:16.476020+0000","last_fullsized":"2024-03-06T07:28:16.476020+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:11:06.847067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476060+0000","last_change":"2024-03-06T07:28:14.471522+0000","last_active":"2024-03-06T07:28:16.476060+0000","last_peered":"2024-03-06T07:28:16.476060+0000","last_clean":"2024-03-06T07:28:16.476060+0000","last_became_active":"2024-03-06T07:28:14.471248+0000","last_became_peered":"2024-03-06T07:28:14.471248+0000","last_unstale":"2024-03-06T07:28:16.476060+0000","last_undegraded":"2024-03-06T07:28:16.476060+0000","last_fullsized":"2024-03-06T07:28:16.476060+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:45:24.816902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476595+0000","last_change":"2024-03-06T07:28:15.471462+0000","last_active":"2024-03-06T07:28:16.476595+0000","last_peered":"2024-03-06T07:28:16.476595+0000","last_clean":"2024-03-06T07:28:16.476595+0000","last_became_active":"2024-03-06T07:28:15.471165+0000","last_became_peered":"2024-03-06T07:28:15.471165+0000","last_unstale":"2024-03-06T07:28:16.476595+0000","last_undegraded":"2024-03-06T07:28:16.476595+0000","last_fullsized":"2024-03-06T07:28:16.476595+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:01.241913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952578+0000","last_change":"2024-03-06T07:28:14.475474+0000","last_active":"2024-03-06T07:28:16.952578+0000","last_peered":"2024-03-06T07:28:16.952578+0000","last_clean":"2024-03-06T07:28:16.952578+0000","last_became_active":"2024-03-06T07:28:14.475178+0000","last_became_peered":"2024-03-06T07:28:14.475178+0000","last_unstale":"2024-03-06T07:28:16.952578+0000","last_undegraded":"2024-03-06T07:28:16.952578+0000","last_fullsized":"2024-03-06T07:28:16.952578+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:22.863787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952372+0000","last_change":"2024-03-06T07:28:15.487222+0000","last_active":"2024-03-06T07:28:16.952372+0000","last_peered":"2024-03-06T07:28:16.952372+0000","last_clean":"2024-03-06T07:28:16.952372+0000","last_became_active":"2024-03-06T07:28:15.486917+0000","last_became_peered":"2024-03-06T07:28:15.486917+0000","last_unstale":"2024-03-06T07:28:16.952372+0000","last_undegraded":"2024-03-06T07:28:16.952372+0000","last_fullsized":"2024-03-06T07:28:16.952372+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:31:14.145099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"198'2","reported_seq":16,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.474041+0000","last_change":"2024-03-06T07:28:14.482310+0000","last_active":"2024-03-06T07:28:16.474041+0000","last_peered":"2024-03-06T07:28:16.474041+0000","last_clean":"2024-03-06T07:28:16.474041+0000","last_became_active":"2024-03-06T07:28:14.481984+0000","last_became_peered":"2024-03-06T07:28:14.481984+0000","last_unstale":"2024-03-06T07:28:16.474041+0000","last_undegraded":"2024-03-06T07:28:16.474041+0000","last_fullsized":"2024-03-06T07:28:16.474041+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:17.151433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952668+0000","last_change":"2024-03-06T07:28:15.470152+0000","last_active":"2024-03-06T07:28:16.952668+0000","last_peered":"2024-03-06T07:28:16.952668+0000","last_clean":"2024-03-06T07:28:16.952668+0000","last_became_active":"2024-03-06T07:28:15.469898+0000","last_became_peered":"2024-03-06T07:28:15.469898+0000","last_unstale":"2024-03-06T07:28:16.952668+0000","last_undegraded":"2024-03-06T07:28:16.952668+0000","last_fullsized":"2024-03-06T07:28:16.952668+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:18.153085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476867+0000","last_change":"2024-03-06T07:28:14.474756+0000","last_active":"2024-03-06T07:28:16.476867+0000","last_peered":"2024-03-06T07:28:16.476867+0000","last_clean":"2024-03-06T07:28:16.476867+0000","last_became_active":"2024-03-06T07:28:14.474454+0000","last_became_peered":"2024-03-06T07:28:14.474454+0000","last_unstale":"2024-03-06T07:28:16.476867+0000","last_undegraded":"2024-03-06T07:28:16.476867+0000","last_fullsized":"2024-03-06T07:28:16.476867+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:37:26.759041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473657+0000","last_change":"2024-03-06T07:28:15.477055+0000","last_active":"2024-03-06T07:28:16.473657+0000","last_peered":"2024-03-06T07:28:16.473657+0000","last_clean":"2024-03-06T07:28:16.473657+0000","last_became_active":"2024-03-06T07:28:15.475898+0000","last_became_peered":"2024-03-06T07:28:15.475898+0000","last_unstale":"2024-03-06T07:28:16.473657+0000","last_undegraded":"2024-03-06T07:28:16.473657+0000","last_fullsized":"2024-03-06T07:28:16.473657+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:14:23.417836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952255+0000","last_change":"2024-03-06T07:28:14.477036+0000","last_active":"2024-03-06T07:28:16.952255+0000","last_peered":"2024-03-06T07:28:16.952255+0000","last_clean":"2024-03-06T07:28:16.952255+0000","last_became_active":"2024-03-06T07:28:14.476402+0000","last_became_peered":"2024-03-06T07:28:14.476402+0000","last_unstale":"2024-03-06T07:28:16.952255+0000","last_undegraded":"2024-03-06T07:28:16.952255+0000","last_fullsized":"2024-03-06T07:28:16.952255+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:45:11.261142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477029+0000","last_change":"2024-03-06T07:28:15.470650+0000","last_active":"2024-03-06T07:28:16.477029+0000","last_peered":"2024-03-06T07:28:16.477029+0000","last_clean":"2024-03-06T07:28:16.477029+0000","last_became_active":"2024-03-06T07:28:15.470324+0000","last_became_peered":"2024-03-06T07:28:15.470324+0000","last_unstale":"2024-03-06T07:28:16.477029+0000","last_undegraded":"2024-03-06T07:28:16.477029+0000","last_fullsized":"2024-03-06T07:28:16.477029+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:24:19.351080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952362+0000","last_change":"2024-03-06T07:28:14.478503+0000","last_active":"2024-03-06T07:28:16.952362+0000","last_peered":"2024-03-06T07:28:16.952362+0000","last_clean":"2024-03-06T07:28:16.952362+0000","last_became_active":"2024-03-06T07:28:14.478064+0000","last_became_peered":"2024-03-06T07:28:14.478064+0000","last_unstale":"2024-03-06T07:28:16.952362+0000","last_undegraded":"2024-03-06T07:28:16.952362+0000","last_fullsized":"2024-03-06T07:28:16.952362+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:50:25.967609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952040+0000","last_change":"2024-03-06T07:28:15.480793+0000","last_active":"2024-03-06T07:28:16.952040+0000","last_peered":"2024-03-06T07:28:16.952040+0000","last_clean":"2024-03-06T07:28:16.952040+0000","last_became_active":"2024-03-06T07:28:15.480518+0000","last_became_peered":"2024-03-06T07:28:15.480518+0000","last_unstale":"2024-03-06T07:28:16.952040+0000","last_undegraded":"2024-03-06T07:28:16.952040+0000","last_fullsized":"2024-03-06T07:28:16.952040+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:42:22.573692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476584+0000","last_change":"2024-03-06T07:28:14.473260+0000","last_active":"2024-03-06T07:28:16.476584+0000","last_peered":"2024-03-06T07:28:16.476584+0000","last_clean":"2024-03-06T07:28:16.476584+0000","last_became_active":"2024-03-06T07:28:14.472982+0000","last_became_peered":"2024-03-06T07:28:14.472982+0000","last_unstale":"2024-03-06T07:28:16.476584+0000","last_undegraded":"2024-03-06T07:28:16.476584+0000","last_fullsized":"2024-03-06T07:28:16.476584+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:59:49.051038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952901+0000","last_change":"2024-03-06T07:28:15.479469+0000","last_active":"2024-03-06T07:28:16.952901+0000","last_peered":"2024-03-06T07:28:16.952901+0000","last_clean":"2024-03-06T07:28:16.952901+0000","last_became_active":"2024-03-06T07:28:15.479143+0000","last_became_peered":"2024-03-06T07:28:15.479143+0000","last_unstale":"2024-03-06T07:28:16.952901+0000","last_undegraded":"2024-03-06T07:28:16.952901+0000","last_fullsized":"2024-03-06T07:28:16.952901+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:19:58.942134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.476582+0000","last_change":"2024-03-06T07:28:14.474422+0000","last_active":"2024-03-06T07:28:16.476582+0000","last_peered":"2024-03-06T07:28:16.476582+0000","last_clean":"2024-03-06T07:28:16.476582+0000","last_became_active":"2024-03-06T07:28:14.474156+0000","last_became_peered":"2024-03-06T07:28:14.474156+0000","last_unstale":"2024-03-06T07:28:16.476582+0000","last_undegraded":"2024-03-06T07:28:16.476582+0000","last_fullsized":"2024-03-06T07:28:16.476582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:25:23.314768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.953198+0000","last_change":"2024-03-06T07:28:15.480000+0000","last_active":"2024-03-06T07:28:16.953198+0000","last_peered":"2024-03-06T07:28:16.953198+0000","last_clean":"2024-03-06T07:28:16.953198+0000","last_became_active":"2024-03-06T07:28:15.479711+0000","last_became_peered":"2024-03-06T07:28:15.479711+0000","last_unstale":"2024-03-06T07:28:16.953198+0000","last_undegraded":"2024-03-06T07:28:16.953198+0000","last_fullsized":"2024-03-06T07:28:16.953198+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:54:35.361349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.473620+0000","last_change":"2024-03-06T07:28:14.473380+0000","last_active":"2024-03-06T07:28:16.473620+0000","last_peered":"2024-03-06T07:28:16.473620+0000","last_clean":"2024-03-06T07:28:16.473620+0000","last_became_active":"2024-03-06T07:28:14.473115+0000","last_became_peered":"2024-03-06T07:28:14.473115+0000","last_unstale":"2024-03-06T07:28:16.473620+0000","last_undegraded":"2024-03-06T07:28:16.473620+0000","last_fullsized":"2024-03-06T07:28:16.473620+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:14:09.550777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.477096+0000","last_change":"2024-03-06T07:28:15.477608+0000","last_active":"2024-03-06T07:28:16.477096+0000","last_peered":"2024-03-06T07:28:16.477096+0000","last_clean":"2024-03-06T07:28:16.477096+0000","last_became_active":"2024-03-06T07:28:15.477333+0000","last_became_peered":"2024-03-06T07:28:15.477333+0000","last_unstale":"2024-03-06T07:28:16.477096+0000","last_undegraded":"2024-03-06T07:28:16.477096+0000","last_fullsized":"2024-03-06T07:28:16.477096+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:08:08.522231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":14,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.475431+0000","last_change":"2024-03-06T07:28:14.474588+0000","last_active":"2024-03-06T07:28:16.475431+0000","last_peered":"2024-03-06T07:28:16.475431+0000","last_clean":"2024-03-06T07:28:16.475431+0000","last_became_active":"2024-03-06T07:28:14.474229+0000","last_became_peered":"2024-03-06T07:28:14.474229+0000","last_unstale":"2024-03-06T07:28:16.475431+0000","last_undegraded":"2024-03-06T07:28:16.475431+0000","last_fullsized":"2024-03-06T07:28:16.475431+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:15:21.471321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":12,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952624+0000","last_change":"2024-03-06T07:28:15.490269+0000","last_active":"2024-03-06T07:28:16.952624+0000","last_peered":"2024-03-06T07:28:16.952624+0000","last_clean":"2024-03-06T07:28:16.952624+0000","last_became_active":"2024-03-06T07:28:15.490009+0000","last_became_peered":"2024-03-06T07:28:15.490009+0000","last_unstale":"2024-03-06T07:28:16.952624+0000","last_undegraded":"2024-03-06T07:28:16.952624+0000","last_fullsized":"2024-03-06T07:28:16.952624+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:14.447703+0000","last_clean_scrub_stamp":"2024-03-06T07:28:14.447703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:57.021055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"198'1","reported_seq":15,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952783+0000","last_change":"2024-03-06T07:28:14.472626+0000","last_active":"2024-03-06T07:28:16.952783+0000","last_peered":"2024-03-06T07:28:16.952783+0000","last_clean":"2024-03-06T07:28:16.952783+0000","last_became_active":"2024-03-06T07:28:14.472023+0000","last_became_peered":"2024-03-06T07:28:14.472023+0000","last_unstale":"2024-03-06T07:28:16.952783+0000","last_undegraded":"2024-03-06T07:28:16.952783+0000","last_fullsized":"2024-03-06T07:28:16.952783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:28:13.446051+0000","last_clean_scrub_stamp":"2024-03-06T07:28:13.446051+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:47.568139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":502,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-03-06T07:28:16.952367+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:28:16.952367+0000","last_peered":"2024-03-06T07:28:16.952367+0000","last_clean":"2024-03-06T07:28:16.952367+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:28:16.952367+0000","last_undegraded":"2024-03-06T07:28:16.952367+0000","last_fullsized":"2024-03-06T07:28:16.952367+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33716,"kb_used_data":2536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338124,"statfs":{"total":96636764160,"available":96602238976,"internally_reserved":0,"allocated":2596864,"data_stored":1823006,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33656,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338184,"statfs":{"total":96636764160,"available":96602300416,"internally_reserved":0,"allocated":2531328,"data_stored":1232626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":2523136,"data_stored":1232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33652,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338188,"statfs":{"total":96636764160,"available":96602304512,"internally_reserved":0,"allocated":2527232,"data_stored":1232584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2547712,"data_stored":1234764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2519040,"data_stored":1226474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37808,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334032,"statfs":{"total":96636764160,"available":96598048768,"internally_reserved":0,"allocated":2523136,"data_stored":1232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37868,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333972,"statfs":{"total":96636764160,"available":96597987328,"internally_reserved":0,"allocated":2588672,"data_stored":1817802,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:28:18.829 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:28:18.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:28:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 Processor -- start 2024-03-06T07:28:18.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 -- start start 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcfcc0b0070 0x7fcfcc155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcfcc0c5380 0x7fcfcc0c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0b0450 con 0x7fcfcc06ca70 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc06c020 con 0x7fcfcc0c5380 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfd1983640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc06cf20 con 0x7fcfcc0b0070 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfcb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcfcc0c5380 0x7fcfcc0c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfcb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcfcc0c5380 0x7fcfcc0c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50898/0 (socket says 172.21.15.17:50898) 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.976+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/1173780926 learned_addr learned my addr 172.21.15.17:0/1173780926 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcaffd640 1 --2- 172.21.15.17:0/1173780926 >> v2:172.21.15.163:3300/0 conn(0x7fcfcc0b0070 0x7fcfcc155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/1173780926 >> v2:172.21.15.163:3300/0 conn(0x7fcfcc0b0070 msgr2=0x7fcfcc155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcb7fe640 1 --2- 172.21.15.17:0/1173780926 >> v2:172.21.15.163:3300/0 conn(0x7fcfcc0b0070 0x7fcfcc155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/1173780926 >> v1:172.21.15.163:6789/0 conn(0x7fcfcc06ca70 legacy=0x7fcfcc0c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:18.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/1173780926 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0c77f0 con 0x7fcfcc0c5380 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.977+0000 7fcfcb7fe640 1 --2- 172.21.15.17:0/1173780926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc0c5790 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0aa3d0 tx=0x7fcfc402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b105c857213330eb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfca7fc640 1 -- 172.21.15.17:0/1173780926 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc402c9f0 con 0x7fcfcc0c5380 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfca7fc640 1 -- 172.21.15.17:0/1173780926 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc402cb90 con 0x7fcfcc0c5380 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfca7fc640 1 -- 172.21.15.17:0/1173780926 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40343f0 con 0x7fcfcc0c5380 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 -- 172.21.15.17:0/1173780926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 msgr2=0x7fcfcc0c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 --2- 172.21.15.17:0/1173780926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc0c5790 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0aa3d0 tx=0x7fcfc402be50 comp rx=0 tx=0).stop 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 -- 172.21.15.17:0/1173780926 shutdown_connections 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 --2- 172.21.15.17:0/1173780926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc0c5790 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 --2- 172.21.15.17:0/1173780926 >> v2:172.21.15.163:3300/0 conn(0x7fcfcc0b0070 0x7fcfcc155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 -- 172.21.15.17:0/1173780926 >> 172.21.15.17:0/1173780926 conn(0x7fcfcc0ac590 msgr2=0x7fcfcc0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:18.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 -- 172.21.15.17:0/1173780926 shutdown_connections 2024-03-06T07:28:18.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.978+0000 7fcfd1983640 1 -- 172.21.15.17:0/1173780926 wait complete. 2024-03-06T07:28:18.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.979+0000 7fcfd1983640 1 Processor -- start 2024-03-06T07:28:18.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.979+0000 7fcfd1983640 1 -- start start 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfcc06ca70 0x7fcfcc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfcc0b0070 0x7fcfcc1eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc06c550 con 0x7fcfcc0c5380 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc06c270 con 0x7fcfcc06ca70 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfd1983640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc06da80 con 0x7fcfcc0b0070 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50908/0 (socket says 172.21.15.17:50908) 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/2309649603 learned_addr learned my addr 172.21.15.17:0/2309649603 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcaffd640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfcc06ca70 0x7fcfcc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfcc0b0070 msgr2=0x7fcfcc1eb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfcc0b0070 0x7fcfcc1eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfcc06ca70 msgr2=0x7fcfcc1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfcc06ca70 0x7fcfcc1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.980+0000 7fcfcb7fe640 1 -- 172.21.15.17:0/2309649603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0c4750 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfcb7fe640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009f40 tx=0x7fcfc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4043020 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc403e450 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0c4a30 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009510 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.981+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc154820 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.982+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc04f330 con 0x7fcfcc0c5380 2024-03-06T07:28:18.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.982+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfc4046020 con 0x7fcfcc0c5380 2024-03-06T07:28:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.983+0000 7fcfc8ff9640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 0x7fcfac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.983+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcfc40ace80 con 0x7fcfcc0c5380 2024-03-06T07:28:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.983+0000 7fcfcaffd640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 0x7fcfac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:18.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.984+0000 7fcfcaffd640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 0x7fcfac06d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006fd0 tx=0x7fcfbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:18.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:18.987+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfc404d020 con 0x7fcfcc0c5380 2024-03-06T07:28:19.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.237+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcfcc1596d0 con 0x7fcfcc0c5380 2024-03-06T07:28:19.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.238+0000 7fcfc8ff9640 1 -- 172.21.15.17:0/2309649603 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+147 (secure 0 0 0) 0x7fcfcc0b0450 con 0x7fcfcc0c5380 2024-03-06T07:28:19.240 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:28:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:28:19.240 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:28:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:28:19.241 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:28:19.241 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 msgr2=0x7fcfac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 0x7fcfac06d990 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc006fd0 tx=0x7fcfbc008040 comp rx=0 tx=0).stop 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 msgr2=0x7fcfcc1542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009f40 tx=0x7fcfc4004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 shutdown_connections 2024-03-06T07:28:19.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcfac06b4d0 0x7fcfac06d990 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfcc0c5380 0x7fcfcc1542e0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfcc0b0070 0x7fcfcc1eb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 --2- 172.21.15.17:0/2309649603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfcc06ca70 0x7fcfcc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 >> 172.21.15.17:0/2309649603 conn(0x7fcfcc0ac590 msgr2=0x7fcfcc0c93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 shutdown_connections 2024-03-06T07:28:19.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.241+0000 7fcfd1983640 1 -- 172.21.15.17:0/2309649603 wait complete. 2024-03-06T07:28:19.254 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:19.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.354+0000 7f20be10c640 1 Processor -- start 2024-03-06T07:28:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.354+0000 7f20be10c640 1 -- start start 2024-03-06T07:28:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.354+0000 7f20be10c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b8153970 0x7f20b80abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20be10c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20b80a9450 0x7f20b80a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20be10c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b80ac480 con 0x7f20b81503c0 2024-03-06T07:28:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20be10c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b8153e70 con 0x7f20b80a9450 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20be10c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80c33b0 con 0x7f20b8153970 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20b80a9450 0x7f20b80a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f20b81503c0 0x7f20b80ab830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55050/0 (socket says 172.21.15.17:55050) 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b77fe640 1 -- 172.21.15.17:0/2759424119 learned_addr learned my addr 172.21.15.17:0/2759424119 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b6ffd640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b8153970 0x7f20b80abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b7fff640 1 -- 172.21.15.17:0/2759424119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b8153970 msgr2=0x7f20b80abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b7fff640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b8153970 0x7f20b80abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b7fff640 1 -- 172.21.15.17:0/2759424119 >> v1:172.21.15.17:6789/0 conn(0x7f20b81503c0 legacy=0x7f20b80ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b7fff640 1 -- 172.21.15.17:0/2759424119 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80c3690 con 0x7f20b80a9450 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.355+0000 7f20b67fc640 1 -- 172.21.15.17:0/2759424119 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740423974 0 0) 0x7f20a8001110 con 0x7f20b81503c0 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20b7fff640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b80a98a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d90 tx=0x7f20b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48071986f41ec681 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20b67fc640 1 -- 172.21.15.17:0/2759424119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0039070 con 0x7f20b80a9450 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20b67fc640 1 -- 172.21.15.17:0/2759424119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b002cbc0 con 0x7f20b80a9450 2024-03-06T07:28:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20b67fc640 1 -- 172.21.15.17:0/2759424119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0034440 con 0x7f20b80a9450 2024-03-06T07:28:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 -- 172.21.15.17:0/2759424119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 msgr2=0x7f20b80a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b80a98a0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d90 tx=0x7f20b002be10 comp rx=0 tx=0).stop 2024-03-06T07:28:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 -- 172.21.15.17:0/2759424119 shutdown_connections 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b80a98a0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 --2- 172.21.15.17:0/2759424119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b8153970 0x7f20b80abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.356+0000 7f20be10c640 1 -- 172.21.15.17:0/2759424119 >> 172.21.15.17:0/2759424119 conn(0x7f20b806bf00 msgr2=0x7f20b80c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.357+0000 7f20be10c640 1 -- 172.21.15.17:0/2759424119 shutdown_connections 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.357+0000 7f20be10c640 1 -- 172.21.15.17:0/2759424119 wait complete. 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.358+0000 7f20be10c640 1 Processor -- start 2024-03-06T07:28:19.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.358+0000 7f20be10c640 1 -- start start 2024-03-06T07:28:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.359+0000 7f20be10c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.359+0000 7f20be10c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b81503c0 0x7f20b81e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.359+0000 7f20be10c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20b8153970 0x7f20b81ea340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20be10c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b8150920 con 0x7f20b80a9450 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20be10c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81507a0 con 0x7f20b8153970 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20be10c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b806eda0 con 0x7f20b81503c0 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50918/0 (socket says 172.21.15.17:50918) 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 -- 172.21.15.17:0/377228860 learned_addr learned my addr 172.21.15.17:0/377228860 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b6ffd640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b81503c0 0x7f20b81e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b7fff640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20b8153970 0x7f20b81ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 -- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b81503c0 msgr2=0x7f20b81e9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b81503c0 0x7f20b81e9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 -- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20b8153970 msgr2=0x7f20b81ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20b8153970 0x7f20b81ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 -- 172.21.15.17:0/377228860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b81542a0 con 0x7f20b80a9450 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20b77fe640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c780 tx=0x7f20a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8018070 con 0x7f20b80a9450 2024-03-06T07:28:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80072d0 con 0x7f20b80a9450 2024-03-06T07:28:19.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a80047e0 con 0x7f20b80a9450 2024-03-06T07:28:19.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b81ec930 con 0x7f20b80a9450 2024-03-06T07:28:19.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.360+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b8154820 con 0x7f20b80a9450 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.362+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b804f330 con 0x7f20b80a9450 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.362+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20a8004980 con 0x7f20b80a9450 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.362+0000 7f209ffff640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.363+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20a80896a0 con 0x7f20b80a9450 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.363+0000 7f20b6ffd640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:19.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.363+0000 7f20b6ffd640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f20b81534f0 tx=0x7f20ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:19.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.367+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f20b80ac480 con 0x7f20b80a9450 2024-03-06T07:28:19.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.589+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20b8068460 con 0x7f20b80a9450 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.589+0000 7f209ffff640 1 -- 172.21.15.17:0/377228860 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+4996 (secure 0 0 0) 0x7f20a8004c70 con 0x7f20b80a9450 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:19.591 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":189,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:16.854170+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10443":{"gid":10443,"name":"c","rank":0,"incarnation":187,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6835/2666237101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2666237101},{"type":"v1","addr":"172.21.15.17:6835","nonce":2666237101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10443,"qdb_cluster":[10443]},"id":15}]} 2024-03-06T07:28:19.592 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 189 2024-03-06T07:28:19.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 msgr2=0x7f20a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 0x7f20a406d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f20b81534f0 tx=0x7f20ac008040 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 msgr2=0x7f20b814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c780 tx=0x7f20a800cc40 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 shutdown_connections 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f20a406b4d0 0x7f20a406d990 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20b8153970 0x7f20b81ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20b81503c0 0x7f20b81e9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 --2- 172.21.15.17:0/377228860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20b80a9450 0x7f20b814b0c0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 >> 172.21.15.17:0/377228860 conn(0x7f20b806bf00 msgr2=0x7f20b80ab780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:19.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 shutdown_connections 2024-03-06T07:28:19.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:19.592+0000 7f20be10c640 1 -- 172.21.15.17:0/377228860 wait complete. 2024-03-06T07:28:20.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 189, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:28:14.827630+0000', 'modified': '2024-03-06T07:28:16.854170+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10443': {'gid': 10443, 'name': 'c', 'rank': 0, 'incarnation': 187, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6835/2666237101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2666237101}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2666237101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10443, 'qdb_cluster': [10443]} 2024-03-06T07:28:20.008 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:28:20.008 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:28:20.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.113+0000 7f080fe9d640 1 Processor -- start 2024-03-06T07:28:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.113+0000 7f080fe9d640 1 -- start start 2024-03-06T07:28:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080fe9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f08080c7170 0x7f08080c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080fe9d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08080c6c40 0x7f08080c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080fe9d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08080a8390 con 0x7f080806ff10 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080fe9d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080c7620 con 0x7f08080c7170 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080fe9d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08080730f0 con 0x7f08080c6c40 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f08080c7170 0x7f08080c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f08080c7170 0x7f08080c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50932/0 (socket says 172.21.15.17:50932) 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 -- 172.21.15.17:0/3419258954 learned_addr learned my addr 172.21.15.17:0/3419258954 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 -- 172.21.15.17:0/3419258954 >> v2:172.21.15.163:3300/0 conn(0x7f08080c6c40 msgr2=0x7f08080c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080e43a640 1 --2- 172.21.15.17:0/3419258954 >> v2:172.21.15.163:3300/0 conn(0x7f08080c6c40 0x7f08080c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 --2- 172.21.15.17:0/3419258954 >> v2:172.21.15.163:3300/0 conn(0x7f08080c6c40 0x7f08080c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 -- 172.21.15.17:0/3419258954 >> v1:172.21.15.163:6789/0 conn(0x7f080806ff10 legacy=0x7f0808153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.114+0000 7f080d438640 1 -- 172.21.15.17:0/3419258954 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080733d0 con 0x7f08080c7170 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.115+0000 7f080d438640 1 --2- 172.21.15.17:0/3419258954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c7170 0x7f08080c8810 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f0808157910 tx=0x7f07f802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f632e2fb34e01de0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.115+0000 7f080cc37640 1 -- 172.21.15.17:0/3419258954 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8038070 con 0x7f08080c7170 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.115+0000 7f080cc37640 1 -- 172.21.15.17:0/3419258954 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f802f050 con 0x7f08080c7170 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.115+0000 7f080cc37640 1 -- 172.21.15.17:0/3419258954 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8031970 con 0x7f08080c7170 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 -- 172.21.15.17:0/3419258954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c7170 msgr2=0x7f08080c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 --2- 172.21.15.17:0/3419258954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c7170 0x7f08080c8810 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f0808157910 tx=0x7f07f802b7f0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 -- 172.21.15.17:0/3419258954 shutdown_connections 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 --2- 172.21.15.17:0/3419258954 >> v2:172.21.15.163:3300/0 conn(0x7f08080c6c40 0x7f08080c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 --2- 172.21.15.17:0/3419258954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c7170 0x7f08080c8810 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 -- 172.21.15.17:0/3419258954 >> 172.21.15.17:0/3419258954 conn(0x7f080806b970 msgr2=0x7f08080aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 -- 172.21.15.17:0/3419258954 shutdown_connections 2024-03-06T07:28:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.116+0000 7f080fe9d640 1 -- 172.21.15.17:0/3419258954 wait complete. 2024-03-06T07:28:20.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.117+0000 7f080fe9d640 1 Processor -- start 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.117+0000 7f080fe9d640 1 -- start start 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080806ff10 0x7f08080c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08080c7170 0x7f08081eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08081544b0 con 0x7f08080c6c40 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0808154330 con 0x7f080806ff10 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08080702f0 con 0x7f08080c7170 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50938/0 (socket says 172.21.15.17:50938) 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 -- 172.21.15.17:0/582703504 learned_addr learned my addr 172.21.15.17:0/582703504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080e43a640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08080c7170 0x7f08081eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080dc39640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080806ff10 0x7f08080c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 -- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08080c7170 msgr2=0x7f08081eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08080c7170 0x7f08081eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 -- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080806ff10 msgr2=0x7f08080c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080806ff10 0x7f08080c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 -- 172.21.15.17:0/582703504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808155360 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080d438640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f07fe7fc640 1 -- 172.21.15.17:0/582703504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8042070 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f07fe7fc640 1 -- 172.21.15.17:0/582703504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f803d890 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f07fe7fc640 1 -- 172.21.15.17:0/582703504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8038040 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080806d920 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.118+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080806dee0 con 0x7f08080c6c40 2024-03-06T07:28:20.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.120+0000 7f07fe7fc640 1 -- 172.21.15.17:0/582703504 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07f8045070 con 0x7f08080c6c40 2024-03-06T07:28:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.120+0000 7f07fe7fc640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 0x7f07ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.120+0000 7f07fe7fc640 1 -- 172.21.15.17:0/582703504 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07f80ac170 con 0x7f08080c6c40 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.121+0000 7f080dc39640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 0x7f07ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.121+0000 7f080dc39640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 0x7f07ec06d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0804007fd0 tx=0x7f0804009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.121+0000 7f0812b64740 1 Processor -- start 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.121+0000 7f0812b64740 1 -- start start 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add1f50 0x56200adcfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add04b0 0x56200adcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200ad366b0 con 0x56200add2330 2024-03-06T07:28:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200acdfa60 con 0x56200add1f50 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56200ad32750 con 0x56200add04b0 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50948/0 (socket says 172.21.15.17:50948) 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 -- 172.21.15.17:0/1635972102 learned_addr learned my addr 172.21.15.17:0/1635972102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080dc39640 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add1f50 0x56200adcfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080e43a640 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add04b0 0x56200adcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 -- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add04b0 msgr2=0x56200adcc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add04b0 0x56200adcc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 -- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add1f50 msgr2=0x56200adcfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add1f50 0x56200adcfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.122+0000 7f080d438640 1 -- 172.21.15.17:0/1635972102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56200acfdf30 con 0x56200add2330 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f080d438640 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009280 tx=0x7f07f807a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2f8845cc7c0682 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f07eb7fe640 1 -- 172.21.15.17:0/1635972102 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f804c070 con 0x56200add2330 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f07eb7fe640 1 -- 172.21.15.17:0/1635972102 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f802f050 con 0x56200add2330 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f07eb7fe640 1 -- 172.21.15.17:0/1635972102 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f807e940 con 0x56200add2330 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 -- 172.21.15.17:0/1635972102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 msgr2=0x56200adcff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009280 tx=0x7f07f807a280 comp rx=0 tx=0).stop 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 -- 172.21.15.17:0/1635972102 shutdown_connections 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add04b0 0x56200adcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200adcff70 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 --2- 172.21.15.17:0/1635972102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add1f50 0x56200adcfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.123+0000 7f0812b64740 1 -- 172.21.15.17:0/1635972102 >> 172.21.15.17:0/1635972102 conn(0x56200adce7d0 msgr2=0x56200adf33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.124+0000 7f0812b64740 1 -- 172.21.15.17:0/1635972102 shutdown_connections 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.124+0000 7f0812b64740 1 -- 172.21.15.17:0/1635972102 wait complete. 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.124+0000 7f0812b64740 1 -- start start 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.124+0000 7f0812b64740 1 Processor -- start 2024-03-06T07:28:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adf9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200adf5b90 0x56200adf6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200ad366b0 con 0x56200add1f50 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200acf2ea0 con 0x56200add2330 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56200adf4230 con 0x56200adf5b90 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f0812b64740 10 client.0 _ll_register_callbacks cb 0x56200acf2870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50950/0 (socket says 172.21.15.17:50950) 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 -- 172.21.15.17:0/1733401983 learned_addr learned my addr 172.21.15.17:0/1733401983 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080e43a640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200adf5b90 0x56200adf6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080d438640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adf9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 -- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200adf5b90 msgr2=0x56200adf6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200adf5b90 0x56200adf6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 -- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 msgr2=0x56200adf9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adf9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.125+0000 7f080dc39640 1 -- 172.21.15.17:0/1733401983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56200acf3270 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f080dc39640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f080400ea60 tx=0x7f0804001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedb2d11b36dbc1b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804027060 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0804010040 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07f802f050 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0804003d80 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 10 client.10572 fetch_fsmap learned FSMap version 189 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=2+}) v3 -- 0x56200aa4b2d0 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f802f050 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 189) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0804013420 con 0x56200add1f50 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 10 client.10572 fetch_fsmap finished waiting for FSMap version 189 2024-03-06T07:28:20.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 10 client.10572 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 20 client.10572 populate_metadata read hostname 'smithi017' 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae225c0 0x56200ae2d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 4 client.10572 mds_command: new command op to 10443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.126+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 --> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56200ad43bf0 con 0x56200ae225c0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.127+0000 7f080d438640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae225c0 0x56200ae2d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.127+0000 7f07e8ff9640 10 client.10572 ms_handle_connect on v2:172.21.15.17:6834/2666237101 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.127+0000 7f080d438640 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae225c0 0x56200ae2d110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20d3ad1bcef3a88 server_cookie=c2a27a186774eb12 in_seq=0 out_seq=0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.127+0000 7f07e8ff9640 1 -- 172.21.15.17:0/1733401983 <== mds.0 v2:172.21.15.17:6834/2666237101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56200ad43bf0 con 0x56200ae225c0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.127+0000 7f07e8ff9640 10 client.10572 handle_command_reply: tid=0 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 client.10572 shutdown 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 msgr2=0x56200adef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f080400ea60 tx=0x7f0804001020 comp rx=0 tx=0).stop 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 shutdown_connections 2024-03-06T07:28:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae225c0 0x56200ae2d110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200adf5b90 0x56200adf6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adf9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f07e8ff9640 0 client.10572 ms_handle_reset on v2:172.21.15.17:6834/2666237101 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 --2- 172.21.15.17:0/1733401983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add1f50 0x56200adef480 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 >> 172.21.15.17:0/1733401983 conn(0x56200add04b0 msgr2=0x56200add08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 shutdown_connections 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 1 -- 172.21.15.17:0/1733401983 wait complete. 2024-03-06T07:28:20.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.128+0000 7f0812b64740 20 client.10572 trim_cache size 0 max 16384 2024-03-06T07:28:20.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 Processor -- start 2024-03-06T07:28:20.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 -- start start 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200adcb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08081f6f50 con 0x56200adf5b90 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080814a080 con 0x56200add2330 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08081f6950 con 0x56200add1f50 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f080e43a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f080e43a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50954/0 (socket says 172.21.15.17:50954) 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.145+0000 7f080e43a640 1 -- 172.21.15.17:0/697057249 learned_addr learned my addr 172.21.15.17:0/697057249 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080d438640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080dc39640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200adcb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 -- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 msgr2=0x56200adcb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200adcb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 -- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 msgr2=0x56200adef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 -- 172.21.15.17:0/697057249 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56200ab156b0 con 0x56200adf5b90 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080d438640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f080e43a640 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f080000f560 tx=0x7f0800007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbafc56732df512 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f07eaffd640 1 -- 172.21.15.17:0/697057249 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080001e070 con 0x56200adf5b90 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f07eaffd640 1 -- 172.21.15.17:0/697057249 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080000e920 con 0x56200adf5b90 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.146+0000 7f07eaffd640 1 -- 172.21.15.17:0/697057249 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080000ecf0 con 0x56200adf5b90 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- 172.21.15.17:0/697057249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 msgr2=0x56200adef7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f080000f560 tx=0x7f0800007f90 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- 172.21.15.17:0/697057249 shutdown_connections 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200adf5b90 0x56200adef7d0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200add2330 0x56200adef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 --2- 172.21.15.17:0/697057249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200adcb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- 172.21.15.17:0/697057249 >> 172.21.15.17:0/697057249 conn(0x56200add04b0 msgr2=0x56200adc7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- 172.21.15.17:0/697057249 shutdown_connections 2024-03-06T07:28:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- 172.21.15.17:0/697057249 wait complete. 2024-03-06T07:28:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 -- start start 2024-03-06T07:28:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.147+0000 7f0812b64740 1 Processor -- start 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200add1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200adf5b90 0x56200ae7b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200adf6970 con 0x56200add2330 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56200adf8130 con 0x56200adf5b90 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56200ae1f270 con 0x56200add1f50 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f0812b64740 10 client.0 _ll_register_callbacks cb 0x7f08040053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f080d438640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50958/0 (socket says 172.21.15.17:50958) 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f080d438640 1 -- 172.21.15.17:0/995193704 learned_addr learned my addr 172.21.15.17:0/995193704 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f080dc39640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200add1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.148+0000 7f080e43a640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200adf5b90 0x56200ae7b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 -- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 msgr2=0x56200add1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200add1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 -- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200adf5b90 msgr2=0x56200ae7b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200adf5b90 0x56200ae7b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 -- 172.21.15.17:0/995193704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56200ae7e0b0 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f080d438640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f07f80aa5f0 tx=0x7f07f803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e70a0ffcef2e10f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8078af0 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07f8093dc0 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f080000e920 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=189) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07f8076e60 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 10 client.10578 fetch_fsmap learned FSMap version 189 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=189,monmap=2+}) v3 -- 0x56200adcbab0 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080000e920 con 0x56200add2330 2024-03-06T07:28:20.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 189) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f07f80921d0 con 0x56200add2330 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 10 client.10578 fetch_fsmap finished waiting for FSMap version 189 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 10 client.10578 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 20 client.10578 populate_metadata read hostname 'smithi017' 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae7b8c0 0x56200adcc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 4 client.10578 mds_command: new command op to 10443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.149+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 --> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56200ad3df40 con 0x56200ae7b8c0 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f080dc39640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae7b8c0 0x56200adcc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f080dc39640 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae7b8c0 0x56200adcc200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce69d6c6d3834ebe server_cookie=239125498e0a0b87 in_seq=0 out_seq=0 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f07e8ff9640 10 client.10578 ms_handle_connect on v2:172.21.15.17:6834/2666237101 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f07e8ff9640 1 -- 172.21.15.17:0/995193704 <== mds.0 v2:172.21.15.17:6834/2666237101 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x56200ad3df40 con 0x56200ae7b8c0 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f07e8ff9640 10 client.10578 handle_command_reply: tid=0 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 client.10578 shutdown 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 msgr2=0x56200ae81c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f07f80aa5f0 tx=0x7f07f803f000 comp rx=0 tx=0).stop 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 shutdown_connections 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56200adf5b90 0x56200ae7b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56200add2330 0x56200ae81c40 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.17:6834/2666237101,v1:172.21.15.17:6835/2666237101] conn(0x56200ae7b8c0 0x56200adcc200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 --2- 172.21.15.17:0/995193704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56200add1f50 0x56200add1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f07e8ff9640 0 client.10578 ms_handle_reset on v2:172.21.15.17:6834/2666237101 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 >> 172.21.15.17:0/995193704 conn(0x56200add04b0 msgr2=0x56200ae32350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 shutdown_connections 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 1 -- 172.21.15.17:0/995193704 wait complete. 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.150+0000 7f0812b64740 20 client.10578 trim_cache size 0 max 16384 2024-03-06T07:28:20.158 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 10443, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.6936983540000004, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 189, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 199, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 199, 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 9.7036750529999996 2024-03-06T07:28:20.159 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:28:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 msgr2=0x7f07ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 0x7f07ec06d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f0804007fd0 tx=0x7f0804009040 comp rx=0 tx=0).stop 2024-03-06T07:28:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 msgr2=0x7f08081eb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000c00 tx=0x7f07f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 shutdown_connections 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f07ec06b4d0 0x7f07ec06d990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08080c7170 0x7f08081eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08080c6c40 0x7f08081eb1b0 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 --2- 172.21.15.17:0/582703504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f080806ff10 0x7f08080c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 >> 172.21.15.17:0/582703504 conn(0x7f080806b970 msgr2=0x7f0808150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 shutdown_connections 2024-03-06T07:28:20.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.151+0000 7f080fe9d640 1 -- 172.21.15.17:0/582703504 wait complete. 2024-03-06T07:28:20.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:28:20.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 Processor -- start 2024-03-06T07:28:20.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 -- start start 2024-03-06T07:28:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1fcc06d750 0x7f1fcc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1fcc06db30 0x7f1fcc0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc156060 con 0x7f1fcc0c3a70 2024-03-06T07:28:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc155b30 con 0x7f1fcc06d750 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.267+0000 7f1fd23b6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0a8430 con 0x7f1fcc06db30 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1fcc06d750 0x7f1fcc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1fcc0c3a70 0x7f1fcc072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55052/0 (socket says 172.21.15.17:55052) 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcb7fe640 1 -- 172.21.15.17:0/1352552241 learned_addr learned my addr 172.21.15.17:0/1352552241 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1fcc06d750 0x7f1fcc0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50966/0 (socket says 172.21.15.17:50966) 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 -- 172.21.15.17:0/1352552241 >> v2:172.21.15.163:3300/0 conn(0x7f1fcc06db30 msgr2=0x7f1fcc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 --2- 172.21.15.17:0/1352552241 >> v2:172.21.15.163:3300/0 conn(0x7f1fcc06db30 0x7f1fcc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 -- 172.21.15.17:0/1352552241 >> v1:172.21.15.17:6789/0 conn(0x7f1fcc0c3a70 legacy=0x7f1fcc072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.268+0000 7f1fcbfff640 1 -- 172.21.15.17:0/1352552241 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc155d80 con 0x7f1fcc06d750 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fcbfff640 1 --2- 172.21.15.17:0/1352552241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0732a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc068860 tx=0x7f1fbc02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8926b63df474c026 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fcaffd640 1 -- 172.21.15.17:0/1352552241 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc038070 con 0x7f1fcc06d750 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fcaffd640 1 -- 172.21.15.17:0/1352552241 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc031740 con 0x7f1fcc06d750 2024-03-06T07:28:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fcaffd640 1 -- 172.21.15.17:0/1352552241 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02f050 con 0x7f1fcc06d750 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 -- 172.21.15.17:0/1352552241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 msgr2=0x7f1fcc0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/1352552241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0732a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc068860 tx=0x7f1fbc02b7e0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 -- 172.21.15.17:0/1352552241 shutdown_connections 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/1352552241 >> v2:172.21.15.163:3300/0 conn(0x7f1fcc06db30 0x7f1fcc0c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/1352552241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0732a0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 -- 172.21.15.17:0/1352552241 >> 172.21.15.17:0/1352552241 conn(0x7f1fcc0c5560 msgr2=0x7f1fcc0c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 -- 172.21.15.17:0/1352552241 shutdown_connections 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.269+0000 7f1fd23b6640 1 -- 172.21.15.17:0/1352552241 wait complete. 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.270+0000 7f1fd23b6640 1 Processor -- start 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 -- start start 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fcc06db30 0x7f1fcc0a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fcc0c3a70 0x7f1fcc0abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0a7e50 con 0x7f1fcc06d750 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0a7cd0 con 0x7f1fcc0c3a70 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fd23b6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0a7fd0 con 0x7f1fcc06db30 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50978/0 (socket says 172.21.15.17:50978) 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.271+0000 7f1fcbfff640 1 -- 172.21.15.17:0/195021393 learned_addr learned my addr 172.21.15.17:0/195021393 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 -- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fcc06db30 msgr2=0x7f1fcc0a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fcc06db30 0x7f1fcc0a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 -- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fcc0c3a70 msgr2=0x7f1fcc0abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fcc0c3a70 0x7f1fcc0abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 -- 172.21.15.17:0/195021393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc14a340 con 0x7f1fcc06d750 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fcbfff640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc000c00 tx=0x7f1fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc042070 con 0x7f1fcc06d750 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0339a0 con 0x7f1fcc06d750 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc1edd50 con 0x7f1fcc06d750 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc038040 con 0x7f1fcc06d750 2024-03-06T07:28:20.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.272+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc1ee5f0 con 0x7f1fcc06d750 2024-03-06T07:28:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.273+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc04f330 con 0x7f1fcc06d750 2024-03-06T07:28:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.273+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fbc045070 con 0x7f1fcc06d750 2024-03-06T07:28:20.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.274+0000 7f1fc8ff9640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 0x7f1fb806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.274+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1fbc0ac080 con 0x7f1fcc06d750 2024-03-06T07:28:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.274+0000 7f1fcb7fe640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 0x7f1fb806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:20.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.275+0000 7f1fcb7fe640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 0x7f1fb806d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000f60 tx=0x7f1fc0007a80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:20.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.278+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1fbc04c030 con 0x7f1fcc06d750 2024-03-06T07:28:20.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:20.497+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7f1fcc0aa570 con 0x7f1fcc06d750 2024-03-06T07:28:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.488+0000 7f1fc8ff9640 1 -- 172.21.15.17:0/195021393 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v190) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1fbc07ad60 con 0x7f1fcc06d750 2024-03-06T07:28:21.490 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked down. 2024-03-06T07:28:21.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 msgr2=0x7f1fb806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 0x7f1fb806d9c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0000f60 tx=0x7f1fc0007a80 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 msgr2=0x7f1fcc0ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc000c00 tx=0x7f1fbc0046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 shutdown_connections 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1fb806b500 0x7f1fb806d9c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.490+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fcc0c3a70 0x7f1fcc0abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.491+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fcc06db30 0x7f1fcc0a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.491+0000 7f1fd23b6640 1 --2- 172.21.15.17:0/195021393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1fcc06d750 0x7f1fcc0ae4e0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.491+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 >> 172.21.15.17:0/195021393 conn(0x7f1fcc0c5560 msgr2=0x7f1fcc152f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.491+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 shutdown_connections 2024-03-06T07:28:21.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.491+0000 7f1fd23b6640 1 -- 172.21.15.17:0/195021393 wait complete. 2024-03-06T07:28:21.504 INFO:tasks.ceph_test_case:waiting 30s for health warning matching 2024-03-06T07:28:21.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:21.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 Processor -- start 2024-03-06T07:28:21.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 -- start start 2024-03-06T07:28:21.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f97080c5330 0x7f97080c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9708155110 con 0x7f97080c88b0 2024-03-06T07:28:21.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080c8f40 con 0x7f9708158db0 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.603+0000 7f970e888640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97081559e0 con 0x7f97080c5330 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f9707fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f97080c5330 0x7f97080c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f97077fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f97080c88b0 0x7f97081586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52680/0 (socket says 172.21.15.17:52680) 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f97077fe640 1 -- 172.21.15.17:0/1437486911 learned_addr learned my addr 172.21.15.17:0/1437486911 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f9707fff640 1 -- 172.21.15.17:0/1437486911 >> v1:172.21.15.163:6789/0 conn(0x7f9708158db0 legacy=0x7f970815fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f9707fff640 1 -- 172.21.15.17:0/1437486911 >> v1:172.21.15.17:6789/0 conn(0x7f97080c88b0 legacy=0x7f97081586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.604+0000 7f9707fff640 1 -- 172.21.15.17:0/1437486911 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708155390 con 0x7f97080c5330 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f9707fff640 1 --2- 172.21.15.17:0/1437486911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97080c5330 0x7f97080c5710 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f96f80097a0 tx=0x7f96f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b84a3cc145722c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f9706ffd640 1 -- 172.21.15.17:0/1437486911 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80315a0 con 0x7f97080c5330 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f9706ffd640 1 -- 172.21.15.17:0/1437486911 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8037070 con 0x7f97080c5330 2024-03-06T07:28:21.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f9706ffd640 1 -- 172.21.15.17:0/1437486911 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8031960 con 0x7f97080c5330 2024-03-06T07:28:21.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 -- 172.21.15.17:0/1437486911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97080c5330 msgr2=0x7f97080c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 --2- 172.21.15.17:0/1437486911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97080c5330 0x7f97080c5710 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f96f80097a0 tx=0x7f96f802b840 comp rx=0 tx=0).stop 2024-03-06T07:28:21.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 -- 172.21.15.17:0/1437486911 shutdown_connections 2024-03-06T07:28:21.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 --2- 172.21.15.17:0/1437486911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f97080c5330 0x7f97080c5710 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 -- 172.21.15.17:0/1437486911 >> 172.21.15.17:0/1437486911 conn(0x7f97080ac560 msgr2=0x7f97080ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 -- 172.21.15.17:0/1437486911 shutdown_connections 2024-03-06T07:28:21.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.605+0000 7f970e888640 1 -- 172.21.15.17:0/1437486911 wait complete. 2024-03-06T07:28:21.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.606+0000 7f970e888640 1 Processor -- start 2024-03-06T07:28:21.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 -- start start 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080c5330 0x7f97081580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97080c88b0 0x7f970814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97081529d0 con 0x7f970814e300 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970806d090 con 0x7f97080c5330 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970e888640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9708152b50 con 0x7f97080c88b0 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970ce25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970ce25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42204/0 (socket says 172.21.15.17:42204) 2024-03-06T07:28:21.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.607+0000 7f970ce25640 1 -- 172.21.15.17:0/4254690887 learned_addr learned my addr 172.21.15.17:0/4254690887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 -- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97080c88b0 msgr2=0x7f970814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97080c88b0 0x7f970814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f9707fff640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080c5330 0x7f97081580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 -- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080c5330 msgr2=0x7f97081580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080c5330 0x7f97081580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 -- 172.21.15.17:0/4254690887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708152140 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970ce25640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f970000b9e0 tx=0x7f970000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9700003b70 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9700003d10 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9708156fc0 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970000cb60 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.608+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9708157580 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.609+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970804f330 con 0x7f970814e300 2024-03-06T07:28:21.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.610+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f970000cd00 con 0x7f970814e300 2024-03-06T07:28:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.610+0000 7f9704ff9640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 0x7f96f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.610+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f970008a130 con 0x7f970814e300 2024-03-06T07:28:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.610+0000 7f9707fff640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 0x7f96f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:21.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.611+0000 7f9707fff640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 0x7f96f406d9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f96f8036000 tx=0x7f96f8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.614+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9700058eb0 con 0x7f970814e300 2024-03-06T07:28:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.833+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f970806cdc0 con 0x7f970814e300 2024-03-06T07:28:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.833+0000 7f9704ff9640 1 -- 172.21.15.17:0/4254690887 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f97000587b0 con 0x7f970814e300 2024-03-06T07:28:21.835 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:21.835 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 msgr2=0x7f96f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 0x7f96f406d9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f96f8036000 tx=0x7f96f8008030 comp rx=0 tx=0).stop 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 msgr2=0x7f9708150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f970000b9e0 tx=0x7f970000bea0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 shutdown_connections 2024-03-06T07:28:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f96f406b500 0x7f96f406d9c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f970814e300 0x7f9708150c40 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97080c88b0 0x7f970814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 --2- 172.21.15.17:0/4254690887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97080c5330 0x7f97081580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 >> 172.21.15.17:0/4254690887 conn(0x7f97080ac560 msgr2=0x7f97080ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.835+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 shutdown_connections 2024-03-06T07:28:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:21.836+0000 7f970e888640 1 -- 172.21.15.17:0/4254690887 wait complete. 2024-03-06T07:28:21.848 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:21.848 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:26.849 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.961+0000 7f2405381640 1 Processor -- start 2024-03-06T07:28:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.961+0000 7f2405381640 1 -- start start 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f2405381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f240006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f2405381640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24000c1ae0 con 0x7f2400155c80 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f2405381640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240006e450 con 0x7f24000720a0 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f2405381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24000c1dc0 con 0x7f2400152520 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fe7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2400155c80 0x7f24000c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52682/0 (socket says 172.21.15.17:52682) 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fe7fc640 1 -- 172.21.15.17:0/3827060770 learned_addr learned my addr 172.21.15.17:0/3827060770 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23feffd640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f240006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677655748 0 0) 0x7f24000c1ae0 con 0x7f2400155c80 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e8003520 con 0x7f2400155c80 2024-03-06T07:28:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 235544412 0 0) 0x7f23e8003520 con 0x7f2400155c80 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 msgr2=0x7f240006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f240006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 >> v1:172.21.15.163:6789/0 conn(0x7f24000720a0 legacy=0x7f240006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240006e730 con 0x7f2400155c80 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f23f4001670 con 0x7f2400155c80 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 >> v1:172.21.15.17:6789/0 conn(0x7f2400155c80 legacy=0x7f24000c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e80039e0 0x7f23e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.962+0000 7f23fdffb640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f23e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24000c1dc0 con 0x7f23e80039e0 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e800ba70 con 0x7f23e8004800 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e800bc20 con 0x7f2400152520 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e80039e0 0x7f23e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23fe7fc640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23ff7fe640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f23e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 -- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 msgr2=0x7f23e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f23e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 -- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23e8004800 msgr2=0x7f23e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23feffd640 1 -- 172.21.15.17:0/3827060770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e8003520 con 0x7f23e80039e0 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.963+0000 7f23ff7fe640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2400152520 0x7f23e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f23feffd640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e80039e0 0x7f23e8003dc0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009f90 tx=0x7f23e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a760e0f3752059c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0039070 con 0x7f23e80039e0 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f002cbc0 con 0x7f23e80039e0 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f23fdffb640 1 -- 172.21.15.17:0/3827060770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0009c10 con 0x7f23e80039e0 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 -- 172.21.15.17:0/3827060770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e80039e0 msgr2=0x7f23e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 --2- 172.21.15.17:0/3827060770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23e80039e0 0x7f23e8003dc0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009f90 tx=0x7f23e8005830 comp rx=0 tx=0).stop 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f23ff7fe640 1 -- 172.21.15.17:0/3827060770 reap_dead start 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 -- 172.21.15.17:0/3827060770 shutdown_connections 2024-03-06T07:28:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 -- 172.21.15.17:0/3827060770 >> 172.21.15.17:0/3827060770 conn(0x7f24000c76b0 msgr2=0x7f24000c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 -- 172.21.15.17:0/3827060770 shutdown_connections 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.964+0000 7f2405381640 1 -- 172.21.15.17:0/3827060770 wait complete. 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 Processor -- start 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 -- start start 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23e80039e0 0x7f240014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24001ee6b0 0x7f24000a8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240006b850 con 0x7f24001e0060 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240006b6d0 con 0x7f24001ee6b0 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.966+0000 7f2405381640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2400152900 con 0x7f23e80039e0 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42216/0 (socket says 172.21.15.17:42216) 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 -- 172.21.15.17:0/329425426 learned_addr learned my addr 172.21.15.17:0/329425426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23feffd640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23e80039e0 0x7f240014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23ff7fe640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24001ee6b0 0x7f24000a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 -- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23e80039e0 msgr2=0x7f240014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23e80039e0 0x7f240014fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 -- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24001ee6b0 msgr2=0x7f24000a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24001ee6b0 0x7f24000a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 -- 172.21.15.17:0/329425426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24001e04e0 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23fe7fc640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007340 tx=0x7f23f4007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f401a070 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f400f9c0 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24001e1ee0 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f400fd90 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.967+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24001e2180 con 0x7f24001e0060 2024-03-06T07:28:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.968+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240004f330 con 0x7f24001e0060 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.969+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23f401d070 con 0x7f24001e0060 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.969+0000 7f23e77fe640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.969+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23f4089cb0 con 0x7f24001e0060 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.969+0000 7f23feffd640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.970+0000 7f23feffd640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 0x7f23ec06d990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f23f0036000 tx=0x7f23f002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:26.974+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24001e04e0 con 0x7f24001e0060 2024-03-06T07:28:27.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.192+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f24000c0b70 con 0x7f24001e0060 2024-03-06T07:28:27.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.192+0000 7f23e77fe640 1 -- 172.21.15.17:0/329425426 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f23f40583e0 con 0x7f24001e0060 2024-03-06T07:28:27.194 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:27.194 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:27.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 msgr2=0x7f23ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:27.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 0x7f23ec06d990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f23f0036000 tx=0x7f23f002e000 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 msgr2=0x7f24000c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f23f4007340 tx=0x7f23f4007740 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 shutdown_connections 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24001ee6b0 0x7f24000a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24001e0060 0x7f24000c27c0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 --2- 172.21.15.17:0/329425426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23e80039e0 0x7f240014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 >> 172.21.15.17:0/329425426 conn(0x7f24000c76b0 msgr2=0x7f240006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 shutdown_connections 2024-03-06T07:28:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:27.195+0000 7f2405381640 1 -- 172.21.15.17:0/329425426 wait complete. 2024-03-06T07:28:27.207 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:27.207 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:32.208 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.319+0000 7f6b07d19640 1 Processor -- start 2024-03-06T07:28:32.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.319+0000 7f6b07d19640 1 -- start start 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b07d19640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b001592d0 0x7f6b001552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b07d19640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b00152d40 con 0x7f6b00152960 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b07d19640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000c1b50 con 0x7f6b000ac260 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b07d19640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b00066e30 con 0x7f6b001592d0 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b052b4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6b001592d0 0x7f6b001552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b05ab5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6b00152960 0x7f6b0006dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60766/0 (socket says 172.21.15.17:60766) 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b05ab5640 1 -- 172.21.15.17:0/3471306240 learned_addr learned my addr 172.21.15.17:0/3471306240 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b052b4640 1 -- 172.21.15.17:0/3471306240 >> v1:172.21.15.163:6789/0 conn(0x7f6b000ac260 legacy=0x7f6b00155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b04ab3640 1 -- 172.21.15.17:0/3471306240 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526250804 0 0) 0x7f6b00152d40 con 0x7f6b00152960 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b052b4640 1 -- 172.21.15.17:0/3471306240 >> v1:172.21.15.17:6789/0 conn(0x7f6b00152960 legacy=0x7f6b0006dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.320+0000 7f6b052b4640 1 -- 172.21.15.17:0/3471306240 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000c1dc0 con 0x7f6b001592d0 2024-03-06T07:28:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.321+0000 7f6b052b4640 1 --2- 172.21.15.17:0/3471306240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b001592d0 0x7f6b001552d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b574eb5fc840f112 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.321+0000 7f6b04ab3640 1 -- 172.21.15.17:0/3471306240 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0034070 con 0x7f6b001592d0 2024-03-06T07:28:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.321+0000 7f6b04ab3640 1 -- 172.21.15.17:0/3471306240 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0030610 con 0x7f6b001592d0 2024-03-06T07:28:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.321+0000 7f6b04ab3640 1 -- 172.21.15.17:0/3471306240 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00309e0 con 0x7f6b001592d0 2024-03-06T07:28:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 -- 172.21.15.17:0/3471306240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b001592d0 msgr2=0x7f6b001552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3471306240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b001592d0 0x7f6b001552d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f6af00097a0 tx=0x7f6af002b960 comp rx=0 tx=0).stop 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 -- 172.21.15.17:0/3471306240 shutdown_connections 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3471306240 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b001592d0 0x7f6b001552d0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 -- 172.21.15.17:0/3471306240 >> 172.21.15.17:0/3471306240 conn(0x7f6b000c70b0 msgr2=0x7f6b000c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 -- 172.21.15.17:0/3471306240 shutdown_connections 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.322+0000 7f6b07d19640 1 -- 172.21.15.17:0/3471306240 wait complete. 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.323+0000 7f6b07d19640 1 Processor -- start 2024-03-06T07:28:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.323+0000 7f6b07d19640 1 -- start start 2024-03-06T07:28:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.323+0000 7f6b07d19640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b000ac260 0x7f6b001eb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.323+0000 7f6b07d19640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b00152960 0x7f6b0014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.323+0000 7f6b07d19640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b07d19640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0006e3a0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b07d19640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00159760 con 0x7f6b00152960 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b07d19640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0006e520 con 0x7f6b000ac260 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42446/0 (socket says 172.21.15.17:42446) 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 -- 172.21.15.17:0/3438488105 learned_addr learned my addr 172.21.15.17:0/3438488105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b052b4640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b00152960 0x7f6b0014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b05ab5640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b000ac260 0x7f6b001eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 -- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b000ac260 msgr2=0x7f6b001eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b000ac260 0x7f6b001eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 -- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b00152960 msgr2=0x7f6b0014c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b00152960 0x7f6b0014c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 -- 172.21.15.17:0/3438488105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0014e0d0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b062b6640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c3a0 tx=0x7f6afc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc017070 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0052d0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b00152df0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc003830 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.324+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b001533b0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.326+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0004f330 con 0x7f6b0014ecf0 2024-03-06T07:28:32.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.326+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6afc01a070 con 0x7f6b0014ecf0 2024-03-06T07:28:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.326+0000 7f6af67fc640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 0x7f6ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.326+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6afc089680 con 0x7f6b0014ecf0 2024-03-06T07:28:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.327+0000 7f6b05ab5640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 0x7f6ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:32.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.327+0000 7f6b05ab5640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 0x7f6ae406d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6af80012e0 tx=0x7f6af8007f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:32.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.331+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6afc0584c0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.549+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6b0006ebc0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.550+0000 7f6af67fc640 1 -- 172.21.15.17:0/3438488105 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6afc057dc0 con 0x7f6b0014ecf0 2024-03-06T07:28:32.551 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:32.552 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 msgr2=0x7f6ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 0x7f6ae406d990 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6af80012e0 tx=0x7f6af8007f30 comp rx=0 tx=0).stop 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 msgr2=0x7f6b0014f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00c3a0 tx=0x7f6afc00c860 comp rx=0 tx=0).stop 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 shutdown_connections 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6ae406b4d0 0x7f6ae406d990 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b0014ecf0 0x7f6b0014f1a0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b00152960 0x7f6b0014c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 --2- 172.21.15.17:0/3438488105 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b000ac260 0x7f6b001eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:32.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 >> 172.21.15.17:0/3438488105 conn(0x7f6b000c70b0 msgr2=0x7f6b0006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:32.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 shutdown_connections 2024-03-06T07:28:32.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:32.552+0000 7f6b07d19640 1 -- 172.21.15.17:0/3438488105 wait complete. 2024-03-06T07:28:32.564 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:32.564 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:35.892 INFO:tasks.ceph.mds.c.smithi017.stderr: -787> 2024-03-06T07:28:17.442+0000 7f2b01baf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:28:35.920 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:28:37.565 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:37.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.676+0000 7f30df884640 1 Processor -- start 2024-03-06T07:28:37.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.676+0000 7f30df884640 1 -- start start 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 0x7f30d8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 0x7f30d8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30d8159460 0x7f30d80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d81587b0 con 0x7f30d8159460 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d8156180 con 0x7f30d80c8a00 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30df884640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80c7760 con 0x7f30d8152920 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30d8159460 0x7f30d80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f30d8159460 0x7f30d80c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42466/0 (socket says 172.21.15.17:42466) 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 -- 172.21.15.17:0/849007935 learned_addr learned my addr 172.21.15.17:0/849007935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dd620640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 0x7f30d8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dce1f640 1 --2- 172.21.15.17:0/849007935 >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 0x7f30d8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 -- 172.21.15.17:0/849007935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 msgr2=0x7f30d8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 0x7f30d8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 -- 172.21.15.17:0/849007935 >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 msgr2=0x7f30d8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 --2- 172.21.15.17:0/849007935 >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 0x7f30d8158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dde21640 1 -- 172.21.15.17:0/849007935 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80c7a40 con 0x7f30d8159460 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dd620640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 0x7f30d8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:37.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.677+0000 7f30dce1f640 1 --2- 172.21.15.17:0/849007935 >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 0x7f30d8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:37.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.678+0000 7f30dde21640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d8159460 0x7f30d80c5330 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f30d808c6e0 tx=0x7f30d002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee3c18b39e42c6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.678+0000 7f30cbfff640 1 -- 172.21.15.17:0/849007935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d002c9e0 con 0x7f30d8159460 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.678+0000 7f30cbfff640 1 -- 172.21.15.17:0/849007935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d002cb80 con 0x7f30d8159460 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.678+0000 7f30cbfff640 1 -- 172.21.15.17:0/849007935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d00343f0 con 0x7f30d8159460 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 -- 172.21.15.17:0/849007935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d8159460 msgr2=0x7f30d80c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d8159460 0x7f30d80c5330 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f30d808c6e0 tx=0x7f30d002be40 comp rx=0 tx=0).stop 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 -- 172.21.15.17:0/849007935 shutdown_connections 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d8159460 0x7f30d80c5330 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 --2- 172.21.15.17:0/849007935 >> v2:172.21.15.163:3300/0 conn(0x7f30d80c8a00 0x7f30d8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 --2- 172.21.15.17:0/849007935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8152920 0x7f30d8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 -- 172.21.15.17:0/849007935 >> 172.21.15.17:0/849007935 conn(0x7f30d80ac560 msgr2=0x7f30d80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 -- 172.21.15.17:0/849007935 shutdown_connections 2024-03-06T07:28:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.679+0000 7f30df884640 1 -- 172.21.15.17:0/849007935 wait complete. 2024-03-06T07:28:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.680+0000 7f30df884640 1 Processor -- start 2024-03-06T07:28:37.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.680+0000 7f30df884640 1 -- start start 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 0x7f30d80c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 0x7f30d81ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d806d030 con 0x7f30d80c8a00 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8158a00 con 0x7f30d8152920 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30df884640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d80c7390 con 0x7f30d8155b00 2024-03-06T07:28:37.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42474/0 (socket says 172.21.15.17:42474) 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 -- 172.21.15.17:0/1596199182 learned_addr learned my addr 172.21.15.17:0/1596199182 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dce1f640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 0x7f30d80c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dde21640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 0x7f30d81ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 -- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 msgr2=0x7f30d81ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 0x7f30d81ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 -- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 msgr2=0x7f30d80c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 0x7f30d80c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 -- 172.21.15.17:0/1596199182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80c2530 con 0x7f30d80c8a00 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dde21640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 0x7f30d81ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dce1f640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 0x7f30d80c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.681+0000 7f30dd620640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d7d0 tx=0x7f30cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.682+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc004030 con 0x7f30d80c8a00 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.682+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0041d0 con 0x7f30d80c8a00 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.682+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc0096a0 con 0x7f30d80c8a00 2024-03-06T07:28:37.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.682+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d81eee90 con 0x7f30d80c8a00 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.682+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d80c3a20 con 0x7f30d80c8a00 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.683+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d804f330 con 0x7f30d80c8a00 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.683+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30cc009840 con 0x7f30d80c8a00 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.684+0000 7f30c9ffb640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.684+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30cc08a590 con 0x7f30d80c8a00 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.684+0000 7f30dce1f640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:37.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.685+0000 7f30dce1f640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 0x7f30bc06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006fd0 tx=0x7f30d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:37.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.688+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f30cc059310 con 0x7f30d80c8a00 2024-03-06T07:28:37.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.912+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f30d80c69d0 con 0x7f30d80c8a00 2024-03-06T07:28:37.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.912+0000 7f30c9ffb640 1 -- 172.21.15.17:0/1596199182 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f30cc058c10 con 0x7f30d80c8a00 2024-03-06T07:28:37.914 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:37.914 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.914+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 msgr2=0x7f30bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.914+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 0x7f30bc06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006fd0 tx=0x7f30d4008040 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.914+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 msgr2=0x7f30d80c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.914+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d7d0 tx=0x7f30cc00dc90 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 shutdown_connections 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30bc06b4d0 0x7f30bc06d990 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30d8155b00 0x7f30d81ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30d8152920 0x7f30d80c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 --2- 172.21.15.17:0/1596199182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d80c8a00 0x7f30d80c4470 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 >> 172.21.15.17:0/1596199182 conn(0x7f30d80ac560 msgr2=0x7f30d8150140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 shutdown_connections 2024-03-06T07:28:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:37.915+0000 7f30df884640 1 -- 172.21.15.17:0/1596199182 wait complete. 2024-03-06T07:28:37.927 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:37.927 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:40.225 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:40.228 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:28:42.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:43.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.039+0000 7fdb99639640 1 Processor -- start 2024-03-06T07:28:43.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.039+0000 7fdb99639640 1 -- start start 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb99639640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdb94152920 0x7fdb94155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb99639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb940c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb99639640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb94156180 con 0x7fdb940c8a00 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb99639640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb941587b0 con 0x7fdb94152920 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb99639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94159630 con 0x7fdb940c5330 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb927fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdb940c8a00 0x7fdb94158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45886/0 (socket says 172.21.15.17:45886) 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb927fc640 1 -- 172.21.15.17:0/4026038659 learned_addr learned my addr 172.21.15.17:0/4026038659 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/4026038659 >> v2:172.21.15.163:3300/0 conn(0x7fdb94152920 0x7fdb94155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413530764 0 0) 0x7fdb88001110 con 0x7fdb940c8a00 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb7c003520 con 0x7fdb940c8a00 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb937fe640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb940c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703524955 0 0) 0x7fdb7c003520 con 0x7fdb940c8a00 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 msgr2=0x7fdb940c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb940c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 >> v2:172.21.15.163:3300/0 conn(0x7fdb94152920 msgr2=0x7fdb94155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 --2- 172.21.15.17:0/4026038659 >> v2:172.21.15.163:3300/0 conn(0x7fdb94152920 0x7fdb94155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940c7760 con 0x7fdb940c8a00 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fdb88001840 con 0x7fdb940c8a00 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.040+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 >> v1:172.21.15.17:6789/0 conn(0x7fdb940c8a00 legacy=0x7fdb94158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7c003f30 0x7fdb7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb7c004d40 0x7fdb7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941587b0 con 0x7fdb7c003f30 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94159630 con 0x7fdb7c004d40 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c00ba10 con 0x7fdb940c5330 2024-03-06T07:28:43.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7c003f30 0x7fdb7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb937fe640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb7c004d40 0x7fdb7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 -- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 msgr2=0x7fdb7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb940c5330 0x7fdb7c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 -- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb7c004d40 msgr2=0x7fdb7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb7c004d40 0x7fdb7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.041+0000 7fdb927fc640 1 -- 172.21.15.17:0/4026038659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c003520 con 0x7fdb7c003f30 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.042+0000 7fdb927fc640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7c003f30 0x7fdb7c004800 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c4c0 tx=0x7fdb7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a91ce738b03b05 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.042+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800c0e0 con 0x7fdb7c003f30 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.042+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88030070 con 0x7fdb7c003f30 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.042+0000 7fdb91ffb640 1 -- 172.21.15.17:0/4026038659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880058d0 con 0x7fdb7c003f30 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 -- 172.21.15.17:0/4026038659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7c003f30 msgr2=0x7fdb7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 --2- 172.21.15.17:0/4026038659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7c003f30 0x7fdb7c004800 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800c4c0 tx=0x7fdb7c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb937fe640 1 -- 172.21.15.17:0/4026038659 reap_dead start 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 -- 172.21.15.17:0/4026038659 shutdown_connections 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 -- 172.21.15.17:0/4026038659 >> 172.21.15.17:0/4026038659 conn(0x7fdb940ac560 msgr2=0x7fdb940ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:43.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 -- 172.21.15.17:0/4026038659 shutdown_connections 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.043+0000 7fdb99639640 1 -- 172.21.15.17:0/4026038659 wait complete. 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.044+0000 7fdb99639640 1 Processor -- start 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.044+0000 7fdb99639640 1 -- start start 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 0x7fdb9414bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 0x7fdb941eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94158a00 con 0x7fdb940b02a0 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c7d20 con 0x7fdb941ee6c0 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9406d030 con 0x7fdb7c003f30 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52254/0 (socket says 172.21.15.17:52254) 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 -- 172.21.15.17:0/3847637795 learned_addr learned my addr 172.21.15.17:0/3847637795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:43.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 0x7fdb9414bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb937fe640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 0x7fdb941eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 -- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 msgr2=0x7fdb9414bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 0x7fdb9414bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 -- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 msgr2=0x7fdb941eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 0x7fdb941eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 -- 172.21.15.17:0/3847637795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9414c550 con 0x7fdb940b02a0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb937fe640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 0x7fdb941eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 0x7fdb9414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb927fc640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fdb88001b40 tx=0x7fdb88000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004bf0 con 0x7fdb940b02a0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88004db0 con 0x7fdb940b02a0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb940c4310 con 0x7fdb940b02a0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.045+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880072b0 con 0x7fdb940b02a0 2024-03-06T07:28:43.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.046+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb940c48d0 con 0x7fdb940b02a0 2024-03-06T07:28:43.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.046+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9404f330 con 0x7fdb940b02a0 2024-03-06T07:28:43.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.047+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb88007450 con 0x7fdb940b02a0 2024-03-06T07:28:43.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.047+0000 7fdb7b7fe640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:43.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.047+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdb880ab9a0 con 0x7fdb940b02a0 2024-03-06T07:28:43.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.048+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:43.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.048+0000 7fdb92ffd640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007440 tx=0x7fdb8400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:43.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.052+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb94156180 con 0x7fdb940b02a0 2024-03-06T07:28:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.269+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdb9406cda0 con 0x7fdb940b02a0 2024-03-06T07:28:43.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.270+0000 7fdb7b7fe640 1 -- 172.21.15.17:0/3847637795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdb88032320 con 0x7fdb940b02a0 2024-03-06T07:28:43.272 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:43.272 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 msgr2=0x7fdb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fdb84007440 tx=0x7fdb8400b040 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 msgr2=0x7fdb9414c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fdb88001b40 tx=0x7fdb88000fe0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 shutdown_connections 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb941ee6c0 0x7fdb941eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b02a0 0x7fdb9414c010 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 --2- 172.21.15.17:0/3847637795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb7c003f30 0x7fdb9414bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 >> 172.21.15.17:0/3847637795 conn(0x7fdb940ac560 msgr2=0x7fdb940abc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 shutdown_connections 2024-03-06T07:28:43.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:43.272+0000 7fdb99639640 1 -- 172.21.15.17:0/3847637795 wait complete. 2024-03-06T07:28:43.284 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:43.284 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:48.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.397+0000 7f09604fa640 1 Processor -- start 2024-03-06T07:28:48.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.397+0000 7f09604fa640 1 -- start start 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f09604fa640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0958155c80 0x7f095806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f09604fa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580c1ae0 con 0x7f09580720a0 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f09604fa640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095806e450 con 0x7f0958152520 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f09604fa640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580c1dc0 con 0x7f0958155c80 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f095ea97640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f09580720a0 0x7f095806afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45898/0 (socket says 172.21.15.17:45898) 2024-03-06T07:28:48.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f095ea97640 1 -- 172.21.15.17:0/2583535405 learned_addr learned my addr 172.21.15.17:0/2583535405 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.398+0000 7f095da95640 1 --2- 172.21.15.17:0/2583535405 >> v2:172.21.15.163:3300/0 conn(0x7f0958155c80 0x7f095806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.399+0000 7f095d294640 1 -- 172.21.15.17:0/2583535405 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926347325 0 0) 0x7f09580c1ae0 con 0x7f09580720a0 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.399+0000 7f095d294640 1 -- 172.21.15.17:0/2583535405 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0940003520 con 0x7f09580720a0 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.399+0000 7f095da95640 1 -- 172.21.15.17:0/2583535405 >> v1:172.21.15.163:6789/0 conn(0x7f0958152520 legacy=0x7f09580c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.399+0000 7f095da95640 1 -- 172.21.15.17:0/2583535405 >> v1:172.21.15.17:6789/0 conn(0x7f09580720a0 legacy=0x7f095806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.399+0000 7f095da95640 1 -- 172.21.15.17:0/2583535405 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095806e730 con 0x7f0958155c80 2024-03-06T07:28:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f095da95640 1 --2- 172.21.15.17:0/2583535405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958155c80 0x7f095806ef30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097a0 tx=0x7f094c02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af2ea0d57fca7f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f095d294640 1 -- 172.21.15.17:0/2583535405 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c034070 con 0x7f0958155c80 2024-03-06T07:28:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f095d294640 1 -- 172.21.15.17:0/2583535405 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c030610 con 0x7f0958155c80 2024-03-06T07:28:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f095d294640 1 -- 172.21.15.17:0/2583535405 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0309c0 con 0x7f0958155c80 2024-03-06T07:28:48.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 -- 172.21.15.17:0/2583535405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958155c80 msgr2=0x7f095806ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 --2- 172.21.15.17:0/2583535405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958155c80 0x7f095806ef30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097a0 tx=0x7f094c02b960 comp rx=0 tx=0).stop 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 -- 172.21.15.17:0/2583535405 shutdown_connections 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 --2- 172.21.15.17:0/2583535405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958155c80 0x7f095806ef30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 -- 172.21.15.17:0/2583535405 >> 172.21.15.17:0/2583535405 conn(0x7f09580c76b0 msgr2=0x7f09580c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.400+0000 7f09604fa640 1 -- 172.21.15.17:0/2583535405 shutdown_connections 2024-03-06T07:28:48.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.401+0000 7f09604fa640 1 -- 172.21.15.17:0/2583535405 wait complete. 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 Processor -- start 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 -- start start 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09580720a0 0x7f09581590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958152520 0x7f095814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095806b850 con 0x7f0958155c80 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095806b6d0 con 0x7f0958152520 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f09604fa640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0958152900 con 0x7f09580720a0 2024-03-06T07:28:48.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f095ea97640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f095ea97640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52262/0 (socket says 172.21.15.17:52262) 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.402+0000 7f095ea97640 1 -- 172.21.15.17:0/1162320019 learned_addr learned my addr 172.21.15.17:0/1162320019 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095da95640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958152520 0x7f095814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095e296640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09580720a0 0x7f09581590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 -- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09580720a0 msgr2=0x7f09581590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09580720a0 0x7f09581590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 -- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958152520 msgr2=0x7f095814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958152520 0x7f095814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 -- 172.21.15.17:0/1162320019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09581eeae0 con 0x7f0958155c80 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f095ea97640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f094800caa0 tx=0x7f094800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948017070 con 0x7f0958155c80 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094800ac00 con 0x7f0958155c80 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0958149d10 con 0x7f0958155c80 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948012400 con 0x7f0958155c80 2024-03-06T07:28:48.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.403+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958157020 con 0x7f0958155c80 2024-03-06T07:28:48.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.404+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095804f330 con 0x7f0958155c80 2024-03-06T07:28:48.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.405+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f094801a070 con 0x7f0958155c80 2024-03-06T07:28:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.405+0000 7f0956ffd640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.405+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f094808a100 con 0x7f0958155c80 2024-03-06T07:28:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.405+0000 7f095e296640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:48.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.406+0000 7f095e296640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 0x7f094406d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f09500016e0 tx=0x7f0950009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:48.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.409+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f09480587b0 con 0x7f0958155c80 2024-03-06T07:28:48.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.629+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f09580c0b70 con 0x7f0958155c80 2024-03-06T07:28:48.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.629+0000 7f0956ffd640 1 -- 172.21.15.17:0/1162320019 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f094805c850 con 0x7f0958155c80 2024-03-06T07:28:48.631 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:48.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:48.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.631+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 msgr2=0x7f094406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.631+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 0x7f094406d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f09500016e0 tx=0x7f0950009040 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.631+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 msgr2=0x7f095814ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.631+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f094800caa0 tx=0x7f094800cf60 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 shutdown_connections 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f094406b4d0 0x7f094406d990 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0958155c80 0x7f095814ba20 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0958152520 0x7f095814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 --2- 172.21.15.17:0/1162320019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09580720a0 0x7f09581590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 >> 172.21.15.17:0/1162320019 conn(0x7f09580c76b0 msgr2=0x7f09580724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 shutdown_connections 2024-03-06T07:28:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:48.632+0000 7f09604fa640 1 -- 172.21.15.17:0/1162320019 wait complete. 2024-03-06T07:28:48.644 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:48.644 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2024-03-06T07:28:53.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-06T07:28:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.756+0000 7f98fa24e640 1 Processor -- start 2024-03-06T07:28:53.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.756+0000 7f98fa24e640 1 -- start start 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98fa24e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4152960 0x7f98f406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98fa24e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98f40ac260 0x7f98f41559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98fa24e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f4152d40 con 0x7f98f41592d0 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98fa24e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f4066e30 con 0x7f98f40ac260 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98fa24e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40c1dc0 con 0x7f98f4152960 2024-03-06T07:28:53.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98f40ac260 0x7f98f41559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f98f40ac260 0x7f98f41559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55620/0 (socket says 172.21.15.17:55620) 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 -- 172.21.15.17:0/2776141480 learned_addr learned my addr 172.21.15.17:0/2776141480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f37fe640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4152960 0x7f98f406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 -- 172.21.15.17:0/2776141480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4152960 msgr2=0x7f98f406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4152960 0x7f98f406dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.757+0000 7f98f3fff640 1 -- 172.21.15.17:0/2776141480 >> v1:172.21.15.163:6789/0 conn(0x7f98f41592d0 legacy=0x7f98f41552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:53.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.758+0000 7f98f3fff640 1 -- 172.21.15.17:0/2776141480 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40c1b50 con 0x7f98f40ac260 2024-03-06T07:28:53.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.758+0000 7f98f3fff640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f40ac260 0x7f98f41559e0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e50 tx=0x7f98ec02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6750bf0b2f1b8c66 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.758+0000 7f98f27fc640 1 -- 172.21.15.17:0/2776141480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec02cb00 con 0x7f98f40ac260 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.758+0000 7f98f27fc640 1 -- 172.21.15.17:0/2776141480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec02cca0 con 0x7f98f40ac260 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98f27fc640 1 -- 172.21.15.17:0/2776141480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec034610 con 0x7f98f40ac260 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 -- 172.21.15.17:0/2776141480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f40ac260 msgr2=0x7f98f41559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f40ac260 0x7f98f41559e0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e50 tx=0x7f98ec02bf60 comp rx=0 tx=0).stop 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 -- 172.21.15.17:0/2776141480 shutdown_connections 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f40ac260 0x7f98f41559e0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 --2- 172.21.15.17:0/2776141480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f4152960 0x7f98f406dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 -- 172.21.15.17:0/2776141480 >> 172.21.15.17:0/2776141480 conn(0x7f98f40c70b0 msgr2=0x7f98f40c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:53.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 -- 172.21.15.17:0/2776141480 shutdown_connections 2024-03-06T07:28:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.759+0000 7f98fa24e640 1 -- 172.21.15.17:0/2776141480 wait complete. 2024-03-06T07:28:53.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.760+0000 7f98fa24e640 1 Processor -- start 2024-03-06T07:28:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.760+0000 7f98fa24e640 1 -- start start 2024-03-06T07:28:53.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 0x7f98f4072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 0x7f98f41eb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40c20a0 con 0x7f98f4158220 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4159760 con 0x7f98f4152960 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98fa24e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f40c2220 con 0x7f98f40ac260 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98f3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98f3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55636/0 (socket says 172.21.15.17:55636) 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.761+0000 7f98f3fff640 1 -- 172.21.15.17:0/1929177281 learned_addr learned my addr 172.21.15.17:0/1929177281 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f2ffd640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 0x7f98f41eb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f37fe640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 0x7f98f4072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 -- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 msgr2=0x7f98f4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 0x7f98f4072350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 -- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 msgr2=0x7f98f41eb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 0x7f98f41eb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 -- 172.21.15.17:0/1929177281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f41ef080 con 0x7f98f4158220 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f2ffd640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 0x7f98f41eb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f37fe640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 0x7f98f4072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98f3fff640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036040 tx=0x7f98ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec043020 con 0x7f98f4158220 2024-03-06T07:28:53.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec02ce50 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f41ef360 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec009510 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.762+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f414cf70 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.763+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f4075300 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.764+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98ec005d10 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.764+0000 7f98dbfff640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.764+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98ec0acc40 con 0x7f98f4158220 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.764+0000 7f98f37fe640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:53.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.765+0000 7f98f37fe640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 0x7f98e006d9c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f98e4006fd0 tx=0x7f98e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:53.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.769+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98f4075300 con 0x7f98f4158220 2024-03-06T07:28:53.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.990+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f98f404f330 con 0x7f98f4158220 2024-03-06T07:28:53.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.991+0000 7f98dbfff640 1 -- 172.21.15.17:0/1929177281 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f98ec0333a0 con 0x7f98f4158220 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:53.993 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 msgr2=0x7f98e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 0x7f98e006d9c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f98e4006fd0 tx=0x7f98e4008040 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 msgr2=0x7f98f41eb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036040 tx=0x7f98ec004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 shutdown_connections 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f98e006b500 0x7f98e006d9c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f4158220 0x7f98f41eb710 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98f4152960 0x7f98f41eb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 --2- 172.21.15.17:0/1929177281 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98f40ac260 0x7f98f4072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 >> 172.21.15.17:0/1929177281 conn(0x7f98f40c70b0 msgr2=0x7f98f40c5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 shutdown_connections 2024-03-06T07:28:53.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:53.993+0000 7f98fa24e640 1 -- 172.21.15.17:0/1929177281 wait complete. 2024-03-06T07:28:54.005 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-03-06T07:28:54.005 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:28:54.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:54.042 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:28:54.086 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:28:54.086 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:28:54.086 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:54.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:28:54.102 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:28:54.102 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:28:54.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:28:54.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:54.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 Processor -- start 2024-03-06T07:28:54.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 -- start start 2024-03-06T07:28:54.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4fe00c88b0 0x7f4fe015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe0155110 con 0x7f4fe00c5330 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe01559e0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.244+0000 7f4fe6556640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe00c8f40 con 0x7f4fe0158db0 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fe4af3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4fe00c88b0 0x7f4fe015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:49632/0 (socket says 172.21.15.17:49632) 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/413373005 learned_addr learned my addr 172.21.15.17:0/413373005 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/413373005 >> v2:172.21.15.163:3300/0 conn(0x7f4fe00c88b0 msgr2=0x7f4fe015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 --2- 172.21.15.17:0/413373005 >> v2:172.21.15.163:3300/0 conn(0x7f4fe00c88b0 0x7f4fe015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/413373005 >> v1:172.21.15.163:6789/0 conn(0x7f4fe00c5330 legacy=0x7f4fe01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.245+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/413373005 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0155390 con 0x7f4fe0158db0 2024-03-06T07:28:54.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fdf7fe640 1 --2- 172.21.15.17:0/413373005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009870 tx=0x7f4fd402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf7fcde30187867 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fdeffd640 1 -- 172.21.15.17:0/413373005 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035020 con 0x7f4fe0158db0 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fdeffd640 1 -- 172.21.15.17:0/413373005 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4031640 con 0x7f4fe0158db0 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fdeffd640 1 -- 172.21.15.17:0/413373005 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd402d040 con 0x7f4fe0158db0 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 -- 172.21.15.17:0/413373005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 msgr2=0x7f4fe00c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 --2- 172.21.15.17:0/413373005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009870 tx=0x7f4fd402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 -- 172.21.15.17:0/413373005 shutdown_connections 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 --2- 172.21.15.17:0/413373005 >> v2:172.21.15.163:3300/0 conn(0x7f4fe00c88b0 0x7f4fe015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 --2- 172.21.15.17:0/413373005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe0158db0 0x7f4fe00c5710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 -- 172.21.15.17:0/413373005 >> 172.21.15.17:0/413373005 conn(0x7f4fe00ac560 msgr2=0x7f4fe00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:54.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 -- 172.21.15.17:0/413373005 shutdown_connections 2024-03-06T07:28:54.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.246+0000 7f4fe6556640 1 -- 172.21.15.17:0/413373005 wait complete. 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.247+0000 7f4fe6556640 1 Processor -- start 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 -- start start 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe00c5330 0x7f4fe01489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fe01ebca0 0x7f4fe01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe01529d0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe006d090 con 0x7f4fe01ebca0 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fe6556640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe0152b50 con 0x7f4fe00c5330 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fdf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55650/0 (socket says 172.21.15.17:55650) 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.248+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/495597637 learned_addr learned my addr 172.21.15.17:0/495597637 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:54.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdffff640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe00c5330 0x7f4fe01489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fe4af3640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fe01ebca0 0x7f4fe01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe00c5330 msgr2=0x7f4fe01489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe00c5330 0x7f4fe01489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fe01ebca0 msgr2=0x7f4fe01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fe01ebca0 0x7f4fe01562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 -- 172.21.15.17:0/495597637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0157f10 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdf7fe640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4040070 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd40399f0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe01eb1c0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035040 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.249+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe00c20d0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.250+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe00c2390 con 0x7f4fe00c88b0 2024-03-06T07:28:54.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.251+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fd404b060 con 0x7f4fe00c88b0 2024-03-06T07:28:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.251+0000 7f4fdcff9640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 0x7f4fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.251+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4fd40ab9f0 con 0x7f4fe00c88b0 2024-03-06T07:28:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.251+0000 7f4fdffff640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 0x7f4fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.252+0000 7f4fdffff640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 0x7f4fcc06d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001780 tx=0x7f4fd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.255+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4fe00c2390 con 0x7f4fe00c88b0 2024-03-06T07:28:54.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.477+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fe004f330 con 0x7f4fe00c88b0 2024-03-06T07:28:54.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.478+0000 7f4fdcff9640 1 -- 172.21.15.17:0/495597637 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4975 (secure 0 0 0) 0x7f4fd407a080 con 0x7f4fe00c88b0 2024-03-06T07:28:54.479 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:54.480 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":193,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2194011003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2194011003},{"type":"v1","addr":"172.21.15.17:6835","nonce":2194011003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:35.450088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:28:54.480 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 193 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 msgr2=0x7f4fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 0x7f4fcc06d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0001780 tx=0x7f4fd0009040 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 msgr2=0x7f4fe0151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd40046c0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 shutdown_connections 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4fcc06b4d0 0x7f4fcc06d990 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4fe01ebca0 0x7f4fe01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fe00c88b0 0x7f4fe0151240 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 --2- 172.21.15.17:0/495597637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4fe00c5330 0x7f4fe01489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 >> 172.21.15.17:0/495597637 conn(0x7f4fe00ac560 msgr2=0x7f4fe00ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 shutdown_connections 2024-03-06T07:28:54.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.480+0000 7f4fe6556640 1 -- 172.21.15.17:0/495597637 wait complete. 2024-03-06T07:28:54.492 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.592+0000 7f26bbfdd640 1 Processor -- start 2024-03-06T07:28:54.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.592+0000 7f26bbfdd640 1 -- start start 2024-03-06T07:28:54.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26bbfdd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26b40c88c0 0x7f26b4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26bbfdd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40c73d0 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26bbfdd640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40be2e0 con 0x7f26b414f360 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26bbfdd640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b4159700 con 0x7f26b40c88c0 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26ba57a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f26b4158b60 0x7f26b4158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42016/0 (socket says 172.21.15.17:42016) 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.593+0000 7f26ba57a640 1 -- 172.21.15.17:0/2214364370 learned_addr learned my addr 172.21.15.17:0/2214364370 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9578640 1 --2- 172.21.15.17:0/2214364370 >> v2:172.21.15.163:3300/0 conn(0x7f26b40c88c0 0x7f26b4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991024611 0 0) 0x7f26b40c73d0 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003520 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3499353897 0 0) 0x7f269c003520 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 >> v2:172.21.15.163:3300/0 conn(0x7f26b40c88c0 msgr2=0x7f26b4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 --2- 172.21.15.17:0/2214364370 >> v2:172.21.15.163:3300/0 conn(0x7f26b40c88c0 0x7f26b4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 >> v1:172.21.15.163:6789/0 conn(0x7f26b414f360 legacy=0x7f26b40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b40c7670 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f26ac001670 con 0x7f26b4158b60 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 >> v1:172.21.15.17:6789/0 conn(0x7f26b4158b60 legacy=0x7f26b4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 0x7f269c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4159700 con 0x7f269c0039b0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c160 con 0x7f269c0051c0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c00c360 con 0x7f269c004800 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 0x7f269c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26ba57a640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9578640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 -- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c004800 msgr2=0x7f269c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c004800 0x7f269c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 -- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269c0051c0 msgr2=0x7f269c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f269c0051c0 0x7f269c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26b9d79640 1 -- 172.21.15.17:0/2214364370 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c003520 con 0x7f269c0039b0 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.594+0000 7f26ba57a640 1 -- 172.21.15.17:0/2214364370 reap_dead start 2024-03-06T07:28:54.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.595+0000 7f26b9d79640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 0x7f269c003dd0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f26a4001e50 tx=0x7f269c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c4b03b9f0a089c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.595+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4001a70 con 0x7f269c0039b0 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.595+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4003b70 con 0x7f269c0039b0 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.595+0000 7f26b8d77640 1 -- 172.21.15.17:0/2214364370 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40050d0 con 0x7f269c0039b0 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 -- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 msgr2=0x7f269c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 0x7f269c003dd0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f26a4001e50 tx=0x7f269c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 -- 172.21.15.17:0/2214364370 shutdown_connections 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/2214364370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f269c0039b0 0x7f269c003dd0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 -- 172.21.15.17:0/2214364370 >> 172.21.15.17:0/2214364370 conn(0x7f26b40ac570 msgr2=0x7f26b40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 -- 172.21.15.17:0/2214364370 shutdown_connections 2024-03-06T07:28:54.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.596+0000 7f26bbfdd640 1 -- 172.21.15.17:0/2214364370 wait complete. 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.597+0000 7f26bbfdd640 1 Processor -- start 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.597+0000 7f26bbfdd640 1 -- start start 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c0039b0 0x7f26b41510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26b414bc70 0x7f26b414c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b40c7c30 con 0x7f26b40afde0 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4155c10 con 0x7f26b414bc70 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4155d90 con 0x7f269c0039b0 2024-03-06T07:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55680/0 (socket says 172.21.15.17:55680) 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 -- 172.21.15.17:0/3639981670 learned_addr learned my addr 172.21.15.17:0/3639981670 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9d79640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c0039b0 0x7f26b41510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26ba57a640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26b414bc70 0x7f26b414c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 -- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c0039b0 msgr2=0x7f26b41510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c0039b0 0x7f26b41510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 -- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26b414bc70 msgr2=0x7f26b414c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26b414bc70 0x7f26b414c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 -- 172.21.15.17:0/3639981670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b414c630 con 0x7f26b40afde0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b9578640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011960 tx=0x7f26a8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8019070 con 0x7f26b40afde0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8004240 con 0x7f26b40afde0 2024-03-06T07:28:54.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800f880 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b4151610 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.598+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b4150580 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.599+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b4075300 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.600+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26a801c070 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.600+0000 7f26b27fc640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 0x7f26a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.600+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f26a808a720 con 0x7f26b40afde0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.600+0000 7f26b9d79640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 0x7f26a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.601+0000 7f26b9d79640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 0x7f26a006d9c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f26a4034000 tx=0x7f26a4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.604+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f26b4075300 con 0x7f26b40afde0 2024-03-06T07:28:54.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.787+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26b404f330 con 0x7f26b40afde0 2024-03-06T07:28:54.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.788+0000 7f26b27fc640 1 -- 172.21.15.17:0/3639981670 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17002 (secure 0 0 0) 0x7f26a8059480 con 0x7f26b40afde0 2024-03-06T07:28:54.791 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:54.791 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":200,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:53.926402+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 msgr2=0x7f26a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 0x7f26a006d9c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f26a4034000 tx=0x7f26a4009030 comp rx=0 tx=0).stop 2024-03-06T07:28:54.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 msgr2=0x7f26b4151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f26a8011960 tx=0x7f26a8011e20 comp rx=0 tx=0).stop 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 shutdown_connections 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f26a006b500 0x7f26a006d9c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26b414bc70 0x7f26b414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26b40afde0 0x7f26b4151fc0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 --2- 172.21.15.17:0/3639981670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f269c0039b0 0x7f26b41510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 >> 172.21.15.17:0/3639981670 conn(0x7f26b40ac570 msgr2=0x7f26b40c9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 shutdown_connections 2024-03-06T07:28:54.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.790+0000 7f26bbfdd640 1 -- 172.21.15.17:0/3639981670 wait complete. 2024-03-06T07:28:54.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:28:13.085360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:28:13.792683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:28:54.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:28:54.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.905+0000 7f509b6a4640 1 Processor -- start 2024-03-06T07:28:54.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.905+0000 7f509b6a4640 1 -- start start 2024-03-06T07:28:54.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f509b6a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50940c4c60 0x7f50940c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f509b6a4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5094152e40 0x7f50940724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f509b6a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5094070eb0 con 0x7f5094070a50 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f509b6a4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5094156940 con 0x7f5094152e40 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f509b6a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50940c80b0 con 0x7f50940c4c60 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5099c41640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5094152e40 0x7f50940724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5098c3f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5094070a50 0x7f5094156dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42032/0 (socket says 172.21.15.17:42032) 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5098c3f640 1 -- 172.21.15.17:0/2268246715 learned_addr learned my addr 172.21.15.17:0/2268246715 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5099c41640 1 -- 172.21.15.17:0/2268246715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50940c4c60 msgr2=0x7f50940c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5099c41640 1 --2- 172.21.15.17:0/2268246715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50940c4c60 0x7f50940c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5099c41640 1 -- 172.21.15.17:0/2268246715 >> v1:172.21.15.17:6789/0 conn(0x7f5094070a50 legacy=0x7f5094156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f5099c41640 1 -- 172.21.15.17:0/2268246715 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940c8390 con 0x7f5094152e40 2024-03-06T07:28:54.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.906+0000 7f508bfff640 1 -- 172.21.15.17:0/2268246715 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386413326 0 0) 0x7f5094070eb0 con 0x7f5094070a50 2024-03-06T07:28:54.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f5099c41640 1 --2- 172.21.15.17:0/2268246715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5094152e40 0x7f50940724d0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d90 tx=0x7f508c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddc99ca4477d6ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f508bfff640 1 -- 172.21.15.17:0/2268246715 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c039070 con 0x7f5094152e40 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f508bfff640 1 -- 172.21.15.17:0/2268246715 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c02cbc0 con 0x7f5094152e40 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f508bfff640 1 -- 172.21.15.17:0/2268246715 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c034440 con 0x7f5094152e40 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f509b6a4640 1 -- 172.21.15.17:0/2268246715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5094152e40 msgr2=0x7f50940724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.907+0000 7f509b6a4640 1 --2- 172.21.15.17:0/2268246715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5094152e40 0x7f50940724d0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d90 tx=0x7f508c02be10 comp rx=0 tx=0).stop 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 -- 172.21.15.17:0/2268246715 shutdown_connections 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 --2- 172.21.15.17:0/2268246715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5094152e40 0x7f50940724d0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 --2- 172.21.15.17:0/2268246715 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50940c4c60 0x7f50940c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 -- 172.21.15.17:0/2268246715 >> 172.21.15.17:0/2268246715 conn(0x7f50940aa6a0 msgr2=0x7f50940aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 -- 172.21.15.17:0/2268246715 shutdown_connections 2024-03-06T07:28:54.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.908+0000 7f509b6a4640 1 -- 172.21.15.17:0/2268246715 wait complete. 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 Processor -- start 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 -- start start 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5094070a50 0x7f50941ec940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5094152e40 0x7f50941dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5094072a10 con 0x7f50940c4c60 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50941532f0 con 0x7f5094152e40 2024-03-06T07:28:54.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f509b6a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5094072b90 con 0x7f5094070a50 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f5098c3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f5098c3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55688/0 (socket says 172.21.15.17:55688) 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.909+0000 7f5098c3f640 1 -- 172.21.15.17:0/116486947 learned_addr learned my addr 172.21.15.17:0/116486947 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 -- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5094070a50 msgr2=0x7f50941ec940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5094070a50 0x7f50941ec940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 -- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5094152e40 msgr2=0x7f50941dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5094152e40 0x7f50941dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 -- 172.21.15.17:0/116486947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50941e8990 con 0x7f50940c4c60 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5098c3f640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f508400c500 tx=0x7f508400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084018070 con 0x7f50940c4c60 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084007580 con 0x7f50940c4c60 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50940c2d90 con 0x7f50940c4c60 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084004a20 con 0x7f50940c4c60 2024-03-06T07:28:54.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.910+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5094156000 con 0x7f50940c4c60 2024-03-06T07:28:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.911+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509404f330 con 0x7f50940c4c60 2024-03-06T07:28:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.912+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f508401b070 con 0x7f50940c4c60 2024-03-06T07:28:54.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.912+0000 7f5089ffb640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 0x7f507806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.912+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5084089600 con 0x7f50940c4c60 2024-03-06T07:28:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.912+0000 7f5099440640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 0x7f507806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:54.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.913+0000 7f5099440640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 0x7f507806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5090009710 tx=0x7f5090006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:54.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:54.916+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5084058360 con 0x7f50940c4c60 2024-03-06T07:28:55.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.097+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5094073520 con 0x7f50940c4c60 2024-03-06T07:28:55.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.097+0000 7f5089ffb640 1 -- 172.21.15.17:0/116486947 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v201) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5084057c60 con 0x7f50940c4c60 2024-03-06T07:28:55.099 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:28:55.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.099+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 msgr2=0x7f507806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.099+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 0x7f507806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f5090009710 tx=0x7f5090006c90 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.099+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 msgr2=0x7f50941e9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.099+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f508400c500 tx=0x7f508400c9c0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 shutdown_connections 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f507806b500 0x7f507806d9c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5094152e40 0x7f50941dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50940c4c60 0x7f50941e9440 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 --2- 172.21.15.17:0/116486947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5094070a50 0x7f50941ec940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 >> 172.21.15.17:0/116486947 conn(0x7f50940aa6a0 msgr2=0x7f50940c3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 shutdown_connections 2024-03-06T07:28:55.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.100+0000 7f509b6a4640 1 -- 172.21.15.17:0/116486947 wait complete. 2024-03-06T07:28:55.112 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:28:55.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.214+0000 7f6b0c064640 1 Processor -- start 2024-03-06T07:28:55.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.215+0000 7f6b0c064640 1 -- start start 2024-03-06T07:28:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.215+0000 7f6b0c064640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6b0406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.215+0000 7f6b0c064640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b040b0450 con 0x7f6b040b0070 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.215+0000 7f6b0c064640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b041560e0 con 0x7f6b040c5380 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.215+0000 7f6b0c064640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406cfb0 con 0x7f6b04155c30 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b09e00640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6b040b0070 0x7f6b040c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42046/0 (socket says 172.21.15.17:42046) 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b09e00640 1 -- 172.21.15.17:0/239592851 learned_addr learned my addr 172.21.15.17:0/239592851 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b095ff640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6b0406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351977675 0 0) 0x7f6af4001490 con 0x7f6b040b0070 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aec003520 con 0x7f6b040b0070 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1429385798 0 0) 0x7f6aec003520 con 0x7f6b040b0070 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 msgr2=0x7f6b0406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6b0406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 >> v1:172.21.15.163:6789/0 conn(0x7f6b040c5380 legacy=0x7f6b0414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040c77f0 con 0x7f6b040b0070 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6af4001bc0 con 0x7f6b040b0070 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 >> v1:172.21.15.17:6789/0 conn(0x7f6b040b0070 legacy=0x7f6b040c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aec0039e0 0x7f6aec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aec004390 0x7f6aec004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6aec005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406cfb0 con 0x7f6aec0039e0 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00baa0 con 0x7f6aec004390 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec00bca0 con 0x7f6b04155c30 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.216+0000 7f6b09e00640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aec0039e0 0x7f6aec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b095ff640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aec004390 0x7f6aec004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b0a601640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6aec005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 -- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 msgr2=0x7f6aec005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6aec005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 -- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aec004390 msgr2=0x7f6aec004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aec004390 0x7f6aec004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 -- 172.21.15.17:0/239592851 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec003520 con 0x7f6aec0039e0 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b095ff640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6aec004390 0x7f6aec004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b0a601640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b04155c30 0x7f6aec005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.217+0000 7f6b09e00640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aec0039e0 0x7f6aec003dc0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6af400aa50 tx=0x7f6aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8be5df4311dbecf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af4038070 con 0x7f6aec0039e0 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af4033630 con 0x7f6aec0039e0 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b08dfe640 1 -- 172.21.15.17:0/239592851 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400a670 con 0x7f6aec0039e0 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 -- 172.21.15.17:0/239592851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aec0039e0 msgr2=0x7f6aec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 --2- 172.21.15.17:0/239592851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aec0039e0 0x7f6aec003dc0 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f6af400aa50 tx=0x7f6aec005830 comp rx=0 tx=0).stop 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0a601640 1 -- 172.21.15.17:0/239592851 reap_dead start 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 -- 172.21.15.17:0/239592851 shutdown_connections 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 -- 172.21.15.17:0/239592851 >> 172.21.15.17:0/239592851 conn(0x7f6b040ac590 msgr2=0x7f6b040ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 -- 172.21.15.17:0/239592851 shutdown_connections 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.218+0000 7f6b0c064640 1 -- 172.21.15.17:0/239592851 wait complete. 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.219+0000 7f6b0c064640 1 Processor -- start 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 -- start start 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6aec0039e0 0x7f6b0414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b04158210 0x7f6b04158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406c270 con 0x7f6b04154970 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406c020 con 0x7f6b04158210 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0c064640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0406c550 con 0x7f6aec0039e0 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0a601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0a601640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55708/0 (socket says 172.21.15.17:55708) 2024-03-06T07:28:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b0a601640 1 -- 172.21.15.17:0/1856007722 learned_addr learned my addr 172.21.15.17:0/1856007722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b095ff640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b04158210 0x7f6b04158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.220+0000 7f6b09e00640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6aec0039e0 0x7f6b0414f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 -- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6aec0039e0 msgr2=0x7f6b0414f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6aec0039e0 0x7f6b0414f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 -- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b04158210 msgr2=0x7f6b04158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b04158210 0x7f6b04158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 -- 172.21.15.17:0/1856007722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04159500 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0a601640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6b040cba60 tx=0x7f6afc00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc018070 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0035f0 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b04157730 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0039e0 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.221+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04154df0 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.222+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0404f330 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.222+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6afc01b070 con 0x7f6b04154970 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.223+0000 7f6b027fc640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.223+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6afc0899b0 con 0x7f6b04154970 2024-03-06T07:28:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.223+0000 7f6b09e00640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.224+0000 7f6b09e00640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 0x7f6af006d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6af4035000 tx=0x7f6af4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.227+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6afc0585f0 con 0x7f6b04154970 2024-03-06T07:28:55.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.410+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b04152720 con 0x7f6b04154970 2024-03-06T07:28:55.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.411+0000 7f6b027fc640 1 -- 172.21.15.17:0/1856007722 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v201) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6afc057ef0 con 0x7f6b04154970 2024-03-06T07:28:55.413 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:28:55.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 msgr2=0x7f6af006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 0x7f6af006d990 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6af4035000 tx=0x7f6af4030000 comp rx=0 tx=0).stop 2024-03-06T07:28:55.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 msgr2=0x7f6b040c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f6b040cba60 tx=0x7f6afc00d810 comp rx=0 tx=0).stop 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 shutdown_connections 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6af006b4d0 0x7f6af006d990 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b04154970 0x7f6b040c21e0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b04158210 0x7f6b04158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 --2- 172.21.15.17:0/1856007722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6aec0039e0 0x7f6b0414f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 >> 172.21.15.17:0/1856007722 conn(0x7f6b040ac590 msgr2=0x7f6b040ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 shutdown_connections 2024-03-06T07:28:55.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.413+0000 7f6b0c064640 1 -- 172.21.15.17:0/1856007722 wait complete. 2024-03-06T07:28:55.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:28:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 Processor -- start 2024-03-06T07:28:55.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 -- start start 2024-03-06T07:28:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e3806afc0 0x7f5e3806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e380c2120 0x7f5e38158c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e380c5840 con 0x7f5e380c2580 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e38158790 con 0x7f5e380c2120 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.530+0000 7f5e3f2f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c5b20 con 0x7f5e3806afc0 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e380c2120 0x7f5e38158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5e380c2120 0x7f5e38158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55710/0 (socket says 172.21.15.17:55710) 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 -- 172.21.15.17:0/3195357472 learned_addr learned my addr 172.21.15.17:0/3195357472 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3d090640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e3806afc0 0x7f5e3806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 -- 172.21.15.17:0/3195357472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e3806afc0 msgr2=0x7f5e3806b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e3806afc0 0x7f5e3806b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 -- 172.21.15.17:0/3195357472 >> v1:172.21.15.163:6789/0 conn(0x7f5e380c2580 legacy=0x7f5e38159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.531+0000 7f5e3c88f640 1 -- 172.21.15.17:0/3195357472 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380c5e00 con 0x7f5e380c2120 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3c88f640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380c2120 0x7f5e38158c20 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009870 tx=0x7f5e2802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21847b28fb523a88 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e2ffff640 1 -- 172.21.15.17:0/3195357472 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280317c0 con 0x7f5e380c2120 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e2ffff640 1 -- 172.21.15.17:0/3195357472 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28037070 con 0x7f5e380c2120 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e2ffff640 1 -- 172.21.15.17:0/3195357472 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28031b80 con 0x7f5e380c2120 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/3195357472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380c2120 msgr2=0x7f5e38158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380c2120 0x7f5e38158c20 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009870 tx=0x7f5e2802b860 comp rx=0 tx=0).stop 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/3195357472 shutdown_connections 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e380c2120 0x7f5e38158c20 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/3195357472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e3806afc0 0x7f5e3806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/3195357472 >> 172.21.15.17:0/3195357472 conn(0x7f5e380c4220 msgr2=0x7f5e380c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/3195357472 shutdown_connections 2024-03-06T07:28:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.532+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/3195357472 wait complete. 2024-03-06T07:28:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.533+0000 7f5e3f2f4640 1 Processor -- start 2024-03-06T07:28:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 -- start start 2024-03-06T07:28:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e380c2120 0x7f5e380716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e380c2580 0x7f5e3806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c2b30 con 0x7f5e3806afc0 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38155220 con 0x7f5e380c2120 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3f2f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380c2cb0 con 0x7f5e380c2580 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55722/0 (socket says 172.21.15.17:55722) 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 -- 172.21.15.17:0/293014651 learned_addr learned my addr 172.21.15.17:0/293014651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 -- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e380c2580 msgr2=0x7f5e3806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e380c2580 0x7f5e3806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 -- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e380c2120 msgr2=0x7f5e380716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.534+0000 7f5e3d090640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e380c2120 0x7f5e380716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e3d090640 1 -- 172.21.15.17:0/293014651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3806d810 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e3d090640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400d830 tx=0x7f5e3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400b6c0 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34019070 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38072090 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400ba80 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.535+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e38072650 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.536+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3804f330 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.536+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e3401e460 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.537+0000 7f5e2dffb640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.537+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5e3408a9e0 con 0x7f5e3806afc0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.537+0000 7f5e3c88f640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.538+0000 7f5e3c88f640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5e28036000 tx=0x7f5e28030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.541+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5e34059740 con 0x7f5e3806afc0 2024-03-06T07:28:55.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.721+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e380c6730 con 0x7f5e3806afc0 2024-03-06T07:28:55.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.721+0000 7f5e2dffb640 1 -- 172.21.15.17:0/293014651 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v201) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e34059040 con 0x7f5e3806afc0 2024-03-06T07:28:55.723 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 msgr2=0x7f5e1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5e28036000 tx=0x7f5e28030000 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 msgr2=0x7f5e381ebcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400d830 tx=0x7f5e3400dcf0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 shutdown_connections 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5e1c06b500 0x7f5e1c06d9c0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5e380c2580 0x7f5e3806e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5e380c2120 0x7f5e380716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 --2- 172.21.15.17:0/293014651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e3806afc0 0x7f5e381ebcd0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 >> 172.21.15.17:0/293014651 conn(0x7f5e380c4220 msgr2=0x7f5e38158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 shutdown_connections 2024-03-06T07:28:55.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.723+0000 7f5e3f2f4640 1 -- 172.21.15.17:0/293014651 wait complete. 2024-03-06T07:28:55.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.835+0000 7fdf480ed640 1 Processor -- start 2024-03-06T07:28:55.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.835+0000 7fdf480ed640 1 -- start start 2024-03-06T07:28:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf480ed640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdf40152920 0x7fdf40155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf480ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdf400c5330 0x7fdf400c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf480ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf40156180 con 0x7fdf400c8a00 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf480ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf401587b0 con 0x7fdf400c5330 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf480ed640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf40159630 con 0x7fdf40152920 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf45688640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdf400c8a00 0x7fdf40158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42052/0 (socket says 172.21.15.17:42052) 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf45688640 1 -- 172.21.15.17:0/1698822815 learned_addr learned my addr 172.21.15.17:0/1698822815 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.836+0000 7fdf4668a640 1 --2- 172.21.15.17:0/1698822815 >> v2:172.21.15.17:3300/0 conn(0x7fdf400c5330 0x7fdf400c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf45e89640 1 --2- 172.21.15.17:0/1698822815 >> v2:172.21.15.163:3300/0 conn(0x7fdf40152920 0x7fdf40155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf44e87640 1 -- 172.21.15.17:0/1698822815 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107993074 0 0) 0x7fdf34001110 con 0x7fdf400c8a00 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf44e87640 1 -- 172.21.15.17:0/1698822815 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf28003520 con 0x7fdf400c8a00 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf4668a640 1 -- 172.21.15.17:0/1698822815 >> v2:172.21.15.163:3300/0 conn(0x7fdf40152920 msgr2=0x7fdf40155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf4668a640 1 --2- 172.21.15.17:0/1698822815 >> v2:172.21.15.163:3300/0 conn(0x7fdf40152920 0x7fdf40155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf4668a640 1 -- 172.21.15.17:0/1698822815 >> v1:172.21.15.17:6789/0 conn(0x7fdf400c8a00 legacy=0x7fdf40158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf4668a640 1 -- 172.21.15.17:0/1698822815 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400c7760 con 0x7fdf400c5330 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.837+0000 7fdf4668a640 1 --2- 172.21.15.17:0/1698822815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf400c5330 0x7fdf400c57a0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fdf400be490 tx=0x7fdf3002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127e631c05421a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf44e87640 1 -- 172.21.15.17:0/1698822815 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30039070 con 0x7fdf400c5330 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf44e87640 1 -- 172.21.15.17:0/1698822815 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3002cbf0 con 0x7fdf400c5330 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf44e87640 1 -- 172.21.15.17:0/1698822815 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30034440 con 0x7fdf400c5330 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 -- 172.21.15.17:0/1698822815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf400c5330 msgr2=0x7fdf400c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 --2- 172.21.15.17:0/1698822815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf400c5330 0x7fdf400c57a0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fdf400be490 tx=0x7fdf3002be40 comp rx=0 tx=0).stop 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 -- 172.21.15.17:0/1698822815 shutdown_connections 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 --2- 172.21.15.17:0/1698822815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf400c5330 0x7fdf400c57a0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 --2- 172.21.15.17:0/1698822815 >> v2:172.21.15.163:3300/0 conn(0x7fdf40152920 0x7fdf40155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 -- 172.21.15.17:0/1698822815 >> 172.21.15.17:0/1698822815 conn(0x7fdf400ac560 msgr2=0x7fdf400ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:55.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 -- 172.21.15.17:0/1698822815 shutdown_connections 2024-03-06T07:28:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.838+0000 7fdf480ed640 1 -- 172.21.15.17:0/1698822815 wait complete. 2024-03-06T07:28:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.839+0000 7fdf480ed640 1 Processor -- start 2024-03-06T07:28:55.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.839+0000 7fdf480ed640 1 -- start start 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf400c5330 0x7fdf4014e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf400c8a00 0x7fdf401477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40158a00 con 0x7fdf40152920 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400c7d20 con 0x7fdf400c8a00 2024-03-06T07:28:55.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf480ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4006d030 con 0x7fdf400c5330 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55742/0 (socket says 172.21.15.17:55742) 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 -- 172.21.15.17:0/458557264 learned_addr learned my addr 172.21.15.17:0/458557264 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf45e89640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf400c5330 0x7fdf4014e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf45688640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf400c8a00 0x7fdf401477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 -- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf400c5330 msgr2=0x7fdf4014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf400c5330 0x7fdf4014e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 -- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf400c8a00 msgr2=0x7fdf401477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf400c8a00 0x7fdf401477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.840+0000 7fdf4668a640 1 -- 172.21.15.17:0/458557264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40148c30 con 0x7fdf40152920 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf4668a640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009e80 tx=0x7fdf30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30039050 con 0x7fdf40152920 2024-03-06T07:28:55.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30045070 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4014b760 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009450 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.841+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf400bea40 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.842+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4004f330 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.842+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf3003e850 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.842+0000 7fdf3e7fc640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 0x7fdf2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.843+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fdf300ac950 con 0x7fdf40152920 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.843+0000 7fdf45e89640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 0x7fdf2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:55.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.843+0000 7fdf45e89640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 0x7fdf2c06d9c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:55.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:55.847+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf3004e020 con 0x7fdf40152920 2024-03-06T07:28:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.066+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf4006cda0 con 0x7fdf40152920 2024-03-06T07:28:56.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.067+0000 7fdf3e7fc640 1 -- 172.21.15.17:0/458557264 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4975 (secure 0 0 0) 0x7fdf40148c30 con 0x7fdf40152920 2024-03-06T07:28:56.068 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:56.069 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":193,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2194011003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2194011003},{"type":"v1","addr":"172.21.15.17:6835","nonce":2194011003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:35.450088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:28:56.069 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 193 2024-03-06T07:28:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 msgr2=0x7fdf2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 0x7fdf2c06d9c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 msgr2=0x7fdf40147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009e80 tx=0x7fdf30004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 shutdown_connections 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf2c06b500 0x7fdf2c06d9c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf40152920 0x7fdf40147d20 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf400c8a00 0x7fdf401477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 --2- 172.21.15.17:0/458557264 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf400c5330 0x7fdf4014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.068+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 >> 172.21.15.17:0/458557264 conn(0x7fdf400ac560 msgr2=0x7fdf400c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.069+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 shutdown_connections 2024-03-06T07:28:56.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.069+0000 7fdf480ed640 1 -- 172.21.15.17:0/458557264 wait complete. 2024-03-06T07:28:56.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.180+0000 7f3f188b5640 1 Processor -- start 2024-03-06T07:28:56.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.180+0000 7f3f188b5640 1 -- start start 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f188b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f14152920 0x7f3f14155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f188b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f14158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f188b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f14156180 con 0x7f3f14159460 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f188b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f141587b0 con 0x7f3f14152920 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f188b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c7760 con 0x7f3f140c8a00 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f12d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3f14159460 0x7f3f140c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42066/0 (socket says 172.21.15.17:42066) 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f12d9d640 1 -- 172.21.15.17:0/4127073696 learned_addr learned my addr 172.21.15.17:0/4127073696 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f14158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f1259c640 1 --2- 172.21.15.17:0/4127073696 >> v2:172.21.15.163:3300/0 conn(0x7f3f14152920 0x7f3f14155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700201971 0 0) 0x7f3f04001490 con 0x7f3f14159460 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.181+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f00003520 con 0x7f3f14159460 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612155659 0 0) 0x7f3f00003520 con 0x7f3f14159460 2024-03-06T07:28:56.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 msgr2=0x7f3f14158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f14158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 >> v2:172.21.15.163:3300/0 conn(0x7f3f14152920 msgr2=0x7f3f14155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 --2- 172.21.15.17:0/4127073696 >> v2:172.21.15.163:3300/0 conn(0x7f3f14152920 0x7f3f14155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f140c7a40 con 0x7f3f14159460 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3f04001bc0 con 0x7f3f14159460 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 >> v1:172.21.15.17:6789/0 conn(0x7f3f14159460 legacy=0x7f3f140c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f000039b0 0x7f3f00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f141587b0 con 0x7f3f00004800 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c7760 con 0x7f3f000039b0 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000b9c0 con 0x7f3f140c8a00 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1259c640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f000039b0 0x7f3f00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f12d9d640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 -- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 msgr2=0x7f3f000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 -- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f000039b0 msgr2=0x7f3f00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f000039b0 0x7f3f00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f11d9b640 1 -- 172.21.15.17:0/4127073696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00003520 con 0x7f3f00004800 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f12d9d640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f140c8a00 0x7f3f000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.182+0000 7f3f1259c640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f000039b0 0x7f3f00003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f00004c80 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009af0 tx=0x7f3f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a2403b7dc55a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc039070 con 0x7f3f00004800 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc02c6b0 con 0x7f3f00004800 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f1159a640 1 -- 172.21.15.17:0/4127073696 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc009690 con 0x7f3f00004800 2024-03-06T07:28:56.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f188b5640 1 -- 172.21.15.17:0/4127073696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 msgr2=0x7f3f00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f188b5640 1 --2- 172.21.15.17:0/4127073696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f00004c80 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009af0 tx=0x7f3f00005830 comp rx=0 tx=0).stop 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.183+0000 7f3f12d9d640 1 -- 172.21.15.17:0/4127073696 reap_dead start 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.184+0000 7f3f188b5640 1 -- 172.21.15.17:0/4127073696 shutdown_connections 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.184+0000 7f3f188b5640 1 -- 172.21.15.17:0/4127073696 >> 172.21.15.17:0/4127073696 conn(0x7f3f140ac560 msgr2=0x7f3f140ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.184+0000 7f3f188b5640 1 -- 172.21.15.17:0/4127073696 shutdown_connections 2024-03-06T07:28:56.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.184+0000 7f3f188b5640 1 -- 172.21.15.17:0/4127073696 wait complete. 2024-03-06T07:28:56.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.185+0000 7f3f188b5640 1 Processor -- start 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.185+0000 7f3f188b5640 1 -- start start 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.185+0000 7f3f188b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.185+0000 7f3f188b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f14153150 0x7f3f141eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f14157f00 0x7f3f14156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1406d030 con 0x7f3f00004800 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14158a00 con 0x7f3f14153150 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f140c5a40 con 0x7f3f14157f00 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55760/0 (socket says 172.21.15.17:55760) 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 -- 172.21.15.17:0/2481652250 learned_addr learned my addr 172.21.15.17:0/2481652250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 -- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f14157f00 msgr2=0x7f3f14156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f14157f00 0x7f3f14156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 -- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f14153150 msgr2=0x7f3f141eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f14153150 0x7f3f141eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f14153150 0x7f3f141eaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 -- 172.21.15.17:0/2481652250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f141eb7a0 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f1259c640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c011680 tx=0x7f3f0c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00f9c0 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00fb60 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f141eba80 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c015460 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.186+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f141565b0 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.187+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f140c8580 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.188+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f0c015600 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.188+0000 7f3f0affd640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.188+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3f0c08ad40 con 0x7f3f00004800 2024-03-06T07:28:56.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.188+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.189+0000 7f3f11d9b640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 0x7f3ef406d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3efc036000 tx=0x7f3efc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.193+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3f140c8580 con 0x7f3f00004800 2024-03-06T07:28:56.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.374+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f1404f330 con 0x7f3f00004800 2024-03-06T07:28:56.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.375+0000 7f3f0affd640 1 -- 172.21.15.17:0/2481652250 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+17002 (secure 0 0 0) 0x7f3f0c019030 con 0x7f3f00004800 2024-03-06T07:28:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:56.377 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":201,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:54.933424+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:56.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.377+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 msgr2=0x7f3ef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.377+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 0x7f3ef406d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3efc036000 tx=0x7f3efc030000 comp rx=0 tx=0).stop 2024-03-06T07:28:56.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.377+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 msgr2=0x7f3f1414b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.377+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c011680 tx=0x7f3f0c011b40 comp rx=0 tx=0).stop 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 shutdown_connections 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ef406b4d0 0x7f3ef406d990 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f14157f00 0x7f3f14156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f14153150 0x7f3f141eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 --2- 172.21.15.17:0/2481652250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f00004800 0x7f3f1414b330 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 >> 172.21.15.17:0/2481652250 conn(0x7f3f140ac560 msgr2=0x7f3f140c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 shutdown_connections 2024-03-06T07:28:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.378+0000 7f3f188b5640 1 -- 172.21.15.17:0/2481652250 wait complete. 2024-03-06T07:28:56.390 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:28:56.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:56.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.490+0000 7f7f3b4e1640 1 Processor -- start 2024-03-06T07:28:56.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.490+0000 7f7f3b4e1640 1 -- start start 2024-03-06T07:28:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f3b4e1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f340c5330 0x7f7f340c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f3b4e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f34159630 con 0x7f7f340c8a00 2024-03-06T07:28:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f3b4e1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340c7760 con 0x7f7f34152920 2024-03-06T07:28:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f3b4e1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f341587b0 con 0x7f7f340c5330 2024-03-06T07:28:56.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f38a7c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7f340c8a00 0x7f7f34158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42074/0 (socket says 172.21.15.17:42074) 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f38a7c640 1 -- 172.21.15.17:0/4142209286 learned_addr learned my addr 172.21.15.17:0/4142209286 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f39a7e640 1 --2- 172.21.15.17:0/4142209286 >> v2:172.21.15.163:3300/0 conn(0x7f7f340c5330 0x7f7f340c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715856694 0 0) 0x7f7f34159630 con 0x7f7f340c8a00 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.491+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f1c003520 con 0x7f7f340c8a00 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2662542290 0 0) 0x7f7f1c003520 con 0x7f7f340c8a00 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 >> v2:172.21.15.163:3300/0 conn(0x7f7f340c5330 msgr2=0x7f7f340c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 --2- 172.21.15.17:0/4142209286 >> v2:172.21.15.163:3300/0 conn(0x7f7f340c5330 0x7f7f340c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 >> v1:172.21.15.163:6789/0 conn(0x7f7f34152920 legacy=0x7f7f34155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340c7a40 con 0x7f7f340c8a00 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7f28001670 con 0x7f7f340c8a00 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 >> v1:172.21.15.17:6789/0 conn(0x7f7f340c8a00 legacy=0x7f7f34158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f1c005210 0x7f7f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341587b0 con 0x7f7f1c003a00 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c00c160 con 0x7f7f1c004800 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c00c360 con 0x7f7f1c005210 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f38a7c640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f39a7e640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f1c005210 0x7f7f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 -- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f1c005210 msgr2=0x7f7f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f1c005210 0x7f7f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 -- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c004800 msgr2=0x7f7f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f3927d640 1 -- 172.21.15.17:0/4142209286 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c003520 con 0x7f7f1c003a00 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f38a7c640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c004800 0x7f7f1c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:56.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f39a7e640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f1c005210 0x7f7f1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.492+0000 7f7f39a7e640 1 -- 172.21.15.17:0/4142209286 reap_dead start 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f3927d640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e20 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c001720 tx=0x7f7f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d538542ef55fcdc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c001370 con 0x7f7f1c003a00 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0052d0 con 0x7f7f1c003a00 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f33fff640 1 -- 172.21.15.17:0/4142209286 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c003830 con 0x7f7f1c003a00 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 msgr2=0x7f7f1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.493+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e20 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c001720 tx=0x7f7f1c005830 comp rx=0 tx=0).stop 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.494+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/4142209286 shutdown_connections 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.494+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/4142209286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f1c003a00 0x7f7f1c003e20 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.494+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/4142209286 >> 172.21.15.17:0/4142209286 conn(0x7f7f340ac560 msgr2=0x7f7f340ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.494+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/4142209286 shutdown_connections 2024-03-06T07:28:56.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.494+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/4142209286 wait complete. 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 Processor -- start 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 -- start start 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c003a00 0x7f7f3414b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f340ae5a0 0x7f7f341ee270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3406d030 con 0x7f7f3414bf30 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34158a00 con 0x7f7f1c003a00 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f3b4e1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340c7390 con 0x7f7f340ae5a0 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.495+0000 7f7f38a7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55788/0 (socket says 172.21.15.17:55788) 2024-03-06T07:28:56.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 -- 172.21.15.17:0/2415634027 learned_addr learned my addr 172.21.15.17:0/2415634027 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f3927d640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c003a00 0x7f7f3414b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f39a7e640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f340ae5a0 0x7f7f341ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 -- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f340ae5a0 msgr2=0x7f7f341ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f340ae5a0 0x7f7f341ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 -- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c003a00 msgr2=0x7f7f3414b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c003a00 0x7f7f3414b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 -- 172.21.15.17:0/2415634027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3414c3b0 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f38a7c640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f2800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800c630 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28018070 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f340b0040 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800ca10 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.496+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f34155d80 con 0x7f7f3414bf30 2024-03-06T07:28:56.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.498+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f340be2a0 con 0x7f7f3414bf30 2024-03-06T07:28:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.498+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f2801d460 con 0x7f7f3414bf30 2024-03-06T07:28:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.498+0000 7f7f31ffb640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 0x7f7f2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.498+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f7f2808a280 con 0x7f7f3414bf30 2024-03-06T07:28:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.498+0000 7f7f3927d640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 0x7f7f2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.499+0000 7f7f3927d640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 0x7f7f2006d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c033000 tx=0x7f7f2c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.502+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7f28058990 con 0x7f7f3414bf30 2024-03-06T07:28:56.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.722+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f3404f330 con 0x7f7f3414bf30 2024-03-06T07:28:56.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.722+0000 7f7f31ffb640 1 -- 172.21.15.17:0/2415634027 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4975 (secure 0 0 0) 0x7f7f2805ca30 con 0x7f7f3414bf30 2024-03-06T07:28:56.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:56.724 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":193,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2194011003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2194011003},{"type":"v1","addr":"172.21.15.17:6835","nonce":2194011003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:35.450088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:28:56.725 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 193 2024-03-06T07:28:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 msgr2=0x7f7f2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 0x7f7f2006d9c0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c033000 tx=0x7f7f2c009030 comp rx=0 tx=0).stop 2024-03-06T07:28:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 msgr2=0x7f7f340ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f7f28000c00 tx=0x7f7f2800eb90 comp rx=0 tx=0).stop 2024-03-06T07:28:56.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 shutdown_connections 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7f2006b500 0x7f7f2006d9c0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f340ae5a0 0x7f7f341ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7f3414bf30 0x7f7f340ae060 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 --2- 172.21.15.17:0/2415634027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f1c003a00 0x7f7f3414b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 >> 172.21.15.17:0/2415634027 conn(0x7f7f340ac560 msgr2=0x7f7f3406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 shutdown_connections 2024-03-06T07:28:56.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.724+0000 7f7f3b4e1640 1 -- 172.21.15.17:0/2415634027 wait complete. 2024-03-06T07:28:56.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:28:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.836+0000 7f4befdae640 1 Processor -- start 2024-03-06T07:28:56.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.836+0000 7f4befdae640 1 -- start start 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4befdae640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 0x7f4be8156430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4befdae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4be81508c0 0x7f4be80c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4befdae640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be8157340 con 0x7f4be806f7b0 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4befdae640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be80c3740 con 0x7f4be81508c0 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4befdae640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be8156d80 con 0x7f4be8072720 2024-03-06T07:28:56.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4bed349640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4be806f7b0 0x7f4be81501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42084/0 (socket says 172.21.15.17:42084) 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4bee34b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4be81508c0 0x7f4be80c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.837+0000 7f4bed349640 1 -- 172.21.15.17:0/1225115536 learned_addr learned my addr 172.21.15.17:0/1225115536 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bedb4a640 1 --2- 172.21.15.17:0/1225115536 >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 0x7f4be8156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4becb48640 1 -- 172.21.15.17:0/1225115536 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700658752 0 0) 0x7f4be8157340 con 0x7f4be806f7b0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4becb48640 1 -- 172.21.15.17:0/1225115536 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd4003520 con 0x7f4be806f7b0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bee34b640 1 -- 172.21.15.17:0/1225115536 >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 msgr2=0x7f4be8156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bee34b640 1 --2- 172.21.15.17:0/1225115536 >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 0x7f4be8156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bee34b640 1 -- 172.21.15.17:0/1225115536 >> v1:172.21.15.17:6789/0 conn(0x7f4be806f7b0 legacy=0x7f4be81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bee34b640 1 -- 172.21.15.17:0/1225115536 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8157060 con 0x7f4be81508c0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bedb4a640 1 --2- 172.21.15.17:0/1225115536 >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 0x7f4be8156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.838+0000 7f4bee34b640 1 --2- 172.21.15.17:0/1225115536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80c3bd0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f4be8071ec0 tx=0x7f4be002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a45a451db5145f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4becb48640 1 -- 172.21.15.17:0/1225115536 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0039070 con 0x7f4be81508c0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4becb48640 1 -- 172.21.15.17:0/1225115536 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be002cbf0 con 0x7f4be81508c0 2024-03-06T07:28:56.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4becb48640 1 -- 172.21.15.17:0/1225115536 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0034440 con 0x7f4be81508c0 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 -- 172.21.15.17:0/1225115536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 msgr2=0x7f4be80c3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 --2- 172.21.15.17:0/1225115536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80c3bd0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f4be8071ec0 tx=0x7f4be002be40 comp rx=0 tx=0).stop 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 -- 172.21.15.17:0/1225115536 shutdown_connections 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 --2- 172.21.15.17:0/1225115536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80c3bd0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 --2- 172.21.15.17:0/1225115536 >> v2:172.21.15.163:3300/0 conn(0x7f4be8072720 0x7f4be8156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 -- 172.21.15.17:0/1225115536 >> 172.21.15.17:0/1225115536 conn(0x7f4be80aa630 msgr2=0x7f4be80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:56.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 -- 172.21.15.17:0/1225115536 shutdown_connections 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.839+0000 7f4befdae640 1 -- 172.21.15.17:0/1225115536 wait complete. 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.840+0000 7f4befdae640 1 Processor -- start 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.840+0000 7f4befdae640 1 -- start start 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4be806f7b0 0x7f4be80c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4be8072720 0x7f4be80c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8156af0 con 0x7f4be81508c0 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8156970 con 0x7f4be806f7b0 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4befdae640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8072b00 con 0x7f4be8072720 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55814/0 (socket says 172.21.15.17:55814) 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 -- 172.21.15.17:0/3804887682 learned_addr learned my addr 172.21.15.17:0/3804887682 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 -- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4be8072720 msgr2=0x7f4be80c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bed349640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4be8072720 0x7f4be80c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bedb4a640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4be806f7b0 0x7f4be80c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4be8072720 0x7f4be80c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 -- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4be806f7b0 msgr2=0x7f4be80c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4be806f7b0 0x7f4be80c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.841+0000 7f4bee34b640 1 -- 172.21.15.17:0/3804887682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80ac080 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4bee34b640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f4be002c450 tx=0x7f4be0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0039050 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0045070 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be81eea10 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0009450 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.842+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be80c5630 con 0x7f4be81508c0 2024-03-06T07:28:56.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.843+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be804f330 con 0x7f4be81508c0 2024-03-06T07:28:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.843+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4be003e850 con 0x7f4be81508c0 2024-03-06T07:28:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.844+0000 7f4bde7fc640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.844+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4be00ac9b0 con 0x7f4be81508c0 2024-03-06T07:28:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.844+0000 7f4bedb4a640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:56.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.844+0000 7f4bedb4a640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4be4006fd0 tx=0x7f4be4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:56.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:56.848+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4be004e020 con 0x7f4be81508c0 2024-03-06T07:28:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.067+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be80aac20 con 0x7f4be81508c0 2024-03-06T07:28:57.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.068+0000 7f4bde7fc640 1 -- 172.21.15.17:0/3804887682 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4975 (secure 0 0 0) 0x7f4be007b680 con 0x7f4be81508c0 2024-03-06T07:28:57.070 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:57.070 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":193,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2194011003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2194011003},{"type":"v1","addr":"172.21.15.17:6835","nonce":2194011003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":193}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:28:14.827630+0000","modified":"2024-03-06T07:28:35.450088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[0],"info":{},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":15}]} 2024-03-06T07:28:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 193 2024-03-06T07:28:57.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 msgr2=0x7f4bcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4be4006fd0 tx=0x7f4be4008040 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 msgr2=0x7f4be80acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f4be002c450 tx=0x7f4be0004a40 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 shutdown_connections 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4bcc06b500 0x7f4bcc06d9c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4be81508c0 0x7f4be80acb40 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4be8072720 0x7f4be80c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 --2- 172.21.15.17:0/3804887682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4be806f7b0 0x7f4be80c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 >> 172.21.15.17:0/3804887682 conn(0x7f4be80aa630 msgr2=0x7f4be806fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 shutdown_connections 2024-03-06T07:28:57.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.070+0000 7f4befdae640 1 -- 172.21.15.17:0/3804887682 wait complete. 2024-03-06T07:28:57.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:28:57.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.182+0000 7f3167565640 1 Processor -- start 2024-03-06T07:28:57.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.182+0000 7f3167565640 1 -- start start 2024-03-06T07:28:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3167565640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31601550c0 0x7f31601554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3167565640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600aa660 con 0x7f31601559e0 2024-03-06T07:28:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3167565640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31600aa940 con 0x7f31600aa180 2024-03-06T07:28:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3167565640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31600c4610 con 0x7f31601550c0 2024-03-06T07:28:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3164b00640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f31601559e0 0x7f316014ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42090/0 (socket says 172.21.15.17:42090) 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3164b00640 1 -- 172.21.15.17:0/1539743089 learned_addr learned my addr 172.21.15.17:0/1539743089 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f3165301640 1 --2- 172.21.15.17:0/1539743089 >> v2:172.21.15.163:3300/0 conn(0x7f31601550c0 0x7f31601554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404981104 0 0) 0x7f31600aa660 con 0x7f31601559e0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.183+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3148005730 con 0x7f31601559e0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946106625 0 0) 0x7f3148005730 con 0x7f31601559e0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 >> v2:172.21.15.163:3300/0 conn(0x7f31601550c0 msgr2=0x7f31601554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 --2- 172.21.15.17:0/1539743089 >> v2:172.21.15.163:3300/0 conn(0x7f31601550c0 0x7f31601554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 >> v1:172.21.15.163:6789/0 conn(0x7f31600aa180 legacy=0x7f31600c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600c4860 con 0x7f31601559e0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f3154001670 con 0x7f31601559e0 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 >> v1:172.21.15.17:6789/0 conn(0x7f31601559e0 legacy=0x7f316014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148005bc0 0x7f3148006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 0x7f3148006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480074f0 0x7f314800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600c4610 con 0x7f3148006a10 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314800e3c0 con 0x7f3148005bc0 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314800e5c0 con 0x7f31480074f0 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 0x7f3148006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3165301640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148005bc0 0x7f3148006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3165b02640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480074f0 0x7f314800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 -- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480074f0 msgr2=0x7f314800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480074f0 0x7f314800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 -- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148005bc0 msgr2=0x7f3148006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148005bc0 0x7f3148006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3164b00640 1 -- 172.21.15.17:0/1539743089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148005730 con 0x7f3148006a10 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3165b02640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31480074f0 0x7f314800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.184+0000 7f3165b02640 1 -- 172.21.15.17:0/1539743089 reap_dead start 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f3164b00640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 0x7f3148006e90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f315400c240 tx=0x7f3148007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b0181cab3f2ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154007d10 con 0x7f3148006a10 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154030070 con 0x7f3148006a10 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f315ffff640 1 -- 172.21.15.17:0/1539743089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315402cdc0 con 0x7f3148006a10 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f3167565640 1 -- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 msgr2=0x7f3148006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.185+0000 7f3167565640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 0x7f3148006e90 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f315400c240 tx=0x7f3148007a40 comp rx=0 tx=0).stop 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.186+0000 7f3167565640 1 -- 172.21.15.17:0/1539743089 shutdown_connections 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.186+0000 7f3167565640 1 --2- 172.21.15.17:0/1539743089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3148006a10 0x7f3148006e90 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.186+0000 7f3167565640 1 -- 172.21.15.17:0/1539743089 >> 172.21.15.17:0/1539743089 conn(0x7f31600c6710 msgr2=0x7f31600c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:57.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.186+0000 7f3167565640 1 -- 172.21.15.17:0/1539743089 shutdown_connections 2024-03-06T07:28:57.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.186+0000 7f3167565640 1 -- 172.21.15.17:0/1539743089 wait complete. 2024-03-06T07:28:57.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 Processor -- start 2024-03-06T07:28:57.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 -- start start 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148006a10 0x7f31600c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f316006d700 0x7f3160072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31601554a0 con 0x7f316006d280 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31600c1520 con 0x7f3148006a10 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3167565640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3160155620 con 0x7f316006d700 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3164b00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.187+0000 7f3164b00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55842/0 (socket says 172.21.15.17:55842) 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 -- 172.21.15.17:0/1132455950 learned_addr learned my addr 172.21.15.17:0/1132455950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3165b02640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f316006d700 0x7f3160072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3165301640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148006a10 0x7f31600c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 -- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f316006d700 msgr2=0x7f3160072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f316006d700 0x7f3160072d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 -- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148006a10 msgr2=0x7f31600c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148006a10 0x7f31600c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 -- 172.21.15.17:0/1132455950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31601edf90 con 0x7f316006d280 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3164b00640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f3154001970 tx=0x7f3160152940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154045070 con 0x7f316006d280 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3154000fe0 con 0x7f316006d280 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31601ee270 con 0x7f316006d280 2024-03-06T07:28:57.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154004c30 con 0x7f316006d280 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.188+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31601ee830 con 0x7f316006d280 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.189+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316004f330 con 0x7f316006d280 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.190+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3154007070 con 0x7f316006d280 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.190+0000 7f315dffb640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.190+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f31540ab080 con 0x7f316006d280 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.190+0000 7f3165301640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.191+0000 7f3165301640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 0x7f314c06d9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f316006d210 tx=0x7f3158008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.194+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3154079de0 con 0x7f316006d280 2024-03-06T07:28:57.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.379+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f316014c140 con 0x7f316006d280 2024-03-06T07:28:57.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.380+0000 7f315dffb640 1 -- 172.21.15.17:0/1132455950 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+17002 (secure 0 0 0) 0x7f31540796e0 con 0x7f316006d280 2024-03-06T07:28:57.382 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:28:57.382 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":201,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:28:54.933424+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:28:13.085360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2024-03-06T07:28:13.792683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:28:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 msgr2=0x7f314c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 0x7f314c06d9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f316006d210 tx=0x7f3158008040 comp rx=0 tx=0).stop 2024-03-06T07:28:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 msgr2=0x7f3160072800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f3154001970 tx=0x7f3160152940 comp rx=0 tx=0).stop 2024-03-06T07:28:57.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 shutdown_connections 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f314c06b500 0x7f314c06d9c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f316006d700 0x7f3160072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f316006d280 0x7f3160072800 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 --2- 172.21.15.17:0/1132455950 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3148006a10 0x7f31600c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 >> 172.21.15.17:0/1132455950 conn(0x7f31600c6710 msgr2=0x7f31600a9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 shutdown_connections 2024-03-06T07:28:57.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.382+0000 7f3167565640 1 -- 172.21.15.17:0/1132455950 wait complete. 2024-03-06T07:28:57.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:28:57.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 Processor -- start 2024-03-06T07:28:57.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 -- start start 2024-03-06T07:28:57.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 0x7f40b81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f40b8072cd0 0x7f40b806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b80733f0 con 0x7f40b8152720 2024-03-06T07:28:57.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b8155ca0 con 0x7f40b8072cd0 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40bd4b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b81561d0 con 0x7f40b806f550 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40b6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f40b8152720 0x7f40b80c77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42104/0 (socket says 172.21.15.17:42104) 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.497+0000 7f40b6ffd640 1 -- 172.21.15.17:0/28913810 learned_addr learned my addr 172.21.15.17:0/28913810 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b77fe640 1 --2- 172.21.15.17:0/28913810 >> v2:172.21.15.17:3300/0 conn(0x7f40b8072cd0 0x7f40b806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b5ffb640 1 -- 172.21.15.17:0/28913810 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251607033 0 0) 0x7f40b80733f0 con 0x7f40b8152720 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b5ffb640 1 -- 172.21.15.17:0/28913810 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a0003520 con 0x7f40b8152720 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b67fc640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 0x7f40b81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b77fe640 1 -- 172.21.15.17:0/28913810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 msgr2=0x7f40b81593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b77fe640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 0x7f40b81593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b77fe640 1 -- 172.21.15.17:0/28913810 >> v1:172.21.15.17:6789/0 conn(0x7f40b8152720 legacy=0x7f40b80c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b77fe640 1 -- 172.21.15.17:0/28913810 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8155ef0 con 0x7f40b8072cd0 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.498+0000 7f40b67fc640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 0x7f40b81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:57.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40b77fe640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b806fd90 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009d90 tx=0x7f40b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefdd56e799cd05 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40b5ffb640 1 -- 172.21.15.17:0/28913810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0039070 con 0x7f40b8072cd0 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40b5ffb640 1 -- 172.21.15.17:0/28913810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b002cbc0 con 0x7f40b8072cd0 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40b5ffb640 1 -- 172.21.15.17:0/28913810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0034440 con 0x7f40b8072cd0 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 -- 172.21.15.17:0/28913810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 msgr2=0x7f40b806fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b806fd90 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009d90 tx=0x7f40b002be10 comp rx=0 tx=0).stop 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 -- 172.21.15.17:0/28913810 shutdown_connections 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b806fd90 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/28913810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b806f550 0x7f40b81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 -- 172.21.15.17:0/28913810 >> 172.21.15.17:0/28913810 conn(0x7f40b806ea40 msgr2=0x7f40b806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 -- 172.21.15.17:0/28913810 shutdown_connections 2024-03-06T07:28:57.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.499+0000 7f40bd4b8640 1 -- 172.21.15.17:0/28913810 wait complete. 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.500+0000 7f40bd4b8640 1 Processor -- start 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 -- start start 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 0x7f40b80a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b8152720 0x7f40b814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8158ea0 con 0x7f40b8072cd0 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8158bc0 con 0x7f40b806f550 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40bd4b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80c7ef0 con 0x7f40b8152720 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40b67fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55862/0 (socket says 172.21.15.17:55862) 2024-03-06T07:28:57.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40b67fc640 1 -- 172.21.15.17:0/354083027 learned_addr learned my addr 172.21.15.17:0/354083027 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40b77fe640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b8152720 0x7f40b814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.501+0000 7f40b6ffd640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 0x7f40b80a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 -- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b8152720 msgr2=0x7f40b814b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b8152720 0x7f40b814b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 -- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 msgr2=0x7f40b80a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 0x7f40b80a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 -- 172.21.15.17:0/354083027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80c4c30 con 0x7f40b8072cd0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b6ffd640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 0x7f40b80a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40b67fc640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c820 tx=0x7f40ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac015070 con 0x7f40b8072cd0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac007a30 con 0x7f40b8072cd0 2024-03-06T07:28:57.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8158570 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac007e00 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.502+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b8157170 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.503+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b814c590 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.503+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40ac018070 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.504+0000 7f409f7fe640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 0x7f40a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.504+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f40ac08a190 con 0x7f40b8072cd0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.504+0000 7f40b6ffd640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 0x7f40a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:57.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.505+0000 7f40b6ffd640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 0x7f40a406d9c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007fd0 tx=0x7f40a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:57.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.508+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f40b814c590 con 0x7f40b8072cd0 2024-03-06T07:28:57.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.725+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f40b8150710 con 0x7f40b8072cd0 2024-03-06T07:28:57.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.952+0000 7f409f7fe640 1 -- 172.21.15.17:0/354083027 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v194) v1 ==== 161+0+0 (secure 0 0 0) 0x7f40b8150710 con 0x7f40b8072cd0 2024-03-06T07:28:57.954 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:28:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 msgr2=0x7f40a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 0x7f40a406d9c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007fd0 tx=0x7f40a8009040 comp rx=0 tx=0).stop 2024-03-06T07:28:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 msgr2=0x7f40b814ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c820 tx=0x7f40ac00cce0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 shutdown_connections 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f40a406b500 0x7f40a406d9c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f40b8152720 0x7f40b814b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b8072cd0 0x7f40b814ab30 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 --2- 172.21.15.17:0/354083027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40b806f550 0x7f40b80a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 >> 172.21.15.17:0/354083027 conn(0x7f40b806ea40 msgr2=0x7f40b814ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 shutdown_connections 2024-03-06T07:28:57.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:57.954+0000 7f40bd4b8640 1 -- 172.21.15.17:0/354083027 wait complete. 2024-03-06T07:28:57.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:28:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.069+0000 7f3bc34df640 1 Processor -- start 2024-03-06T07:28:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 -- start start 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c26c0 0x7f3bbc155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 0x7f3bbc152350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc06b740 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc06c110 con 0x7f3bbc0c0470 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.070+0000 7f3bc34df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0670c0 con 0x7f3bbc0c26c0 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55870/0 (socket says 172.21.15.17:55870) 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 -- 172.21.15.17:0/1362407594 learned_addr learned my addr 172.21.15.17:0/1362407594 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 -- 172.21.15.17:0/1362407594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c26c0 msgr2=0x7f3bbc155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1362407594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c26c0 0x7f3bbc155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 -- 172.21.15.17:0/1362407594 >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 msgr2=0x7f3bbc152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc1a7c640 1 --2- 172.21.15.17:0/1362407594 >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 0x7f3bbc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1362407594 >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 0x7f3bbc152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc0a7a640 1 -- 172.21.15.17:0/1362407594 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc06b0f0 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.071+0000 7f3bc1a7c640 1 --2- 172.21.15.17:0/1362407594 >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 0x7f3bbc152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:58.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1362407594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0c49f0 tx=0x7f3bac02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c78c75d28f658a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bb3fff640 1 -- 172.21.15.17:0/1362407594 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac038070 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bb3fff640 1 -- 172.21.15.17:0/1362407594 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac02f050 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bb3fff640 1 -- 172.21.15.17:0/1362407594 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac031970 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 -- 172.21.15.17:0/1362407594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c2aa0 msgr2=0x7f3bbc06bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1362407594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0c49f0 tx=0x7f3bac02b7f0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 -- 172.21.15.17:0/1362407594 shutdown_connections 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1362407594 >> v2:172.21.15.163:3300/0 conn(0x7f3bbc0c0470 0x7f3bbc152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1362407594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c2aa0 0x7f3bbc06bbd0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1362407594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c26c0 0x7f3bbc155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 -- 172.21.15.17:0/1362407594 >> 172.21.15.17:0/1362407594 conn(0x7f3bbc0c3b40 msgr2=0x7f3bbc0c3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:58.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.072+0000 7f3bc34df640 1 -- 172.21.15.17:0/1362407594 shutdown_connections 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.073+0000 7f3bc34df640 1 -- 172.21.15.17:0/1362407594 wait complete. 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 Processor -- start 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 -- start start 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bbc0c26c0 0x7f3bbc0ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c2aa0 0x7f3bbc1ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c1f30 con 0x7f3bbc0c0470 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c0900 con 0x7f3bbc0c26c0 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc34df640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c20b0 con 0x7f3bbc0c2aa0 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc127b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc127b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55872/0 (socket says 172.21.15.17:55872) 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc127b640 1 -- 172.21.15.17:0/1007764251 learned_addr learned my addr 172.21.15.17:0/1007764251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.074+0000 7f3bc1a7c640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c2aa0 0x7f3bbc1ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bbc0c26c0 0x7f3bbc0ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 -- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c2aa0 msgr2=0x7f3bbc1ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c2aa0 0x7f3bbc1ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 -- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bbc0c26c0 msgr2=0x7f3bbc0ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bbc0c26c0 0x7f3bbc0ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 -- 172.21.15.17:0/1007764251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0ac290 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc127b640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b4a0 tx=0x7f3bb800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8014070 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8010610 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8010a00 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0ac570 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.075+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc0aae50 con 0x7f3bbc0c0470 2024-03-06T07:28:58.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.076+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc04f330 con 0x7f3bbc0c0470 2024-03-06T07:28:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.076+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bb8017070 con 0x7f3bbc0c0470 2024-03-06T07:28:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.077+0000 7f3bb1ffb640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 0x7f3ba006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.077+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3bb8089d60 con 0x7f3bbc0c0470 2024-03-06T07:28:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.077+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 0x7f3ba006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:58.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.078+0000 7f3bc0a7a640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 0x7f3ba006d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3bac035000 tx=0x7f3bac008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:58.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.081+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bb8058ac0 con 0x7f3bbc0c0470 2024-03-06T07:28:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.299+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3bbc150e70 con 0x7f3bbc0c0470 2024-03-06T07:28:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.959+0000 7f3bb1ffb640 1 -- 172.21.15.17:0/1007764251 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v195) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3bb80583c0 con 0x7f3bbc0c0470 2024-03-06T07:28:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 msgr2=0x7f3ba006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 0x7f3ba006d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3bac035000 tx=0x7f3bac008030 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 msgr2=0x7f3bbc152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b4a0 tx=0x7f3bb800b960 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 shutdown_connections 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3ba006b4d0 0x7f3ba006d990 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bbc0c2aa0 0x7f3bbc1ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bbc0c26c0 0x7f3bbc0ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 --2- 172.21.15.17:0/1007764251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bbc0c0470 0x7f3bbc152f10 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 >> 172.21.15.17:0/1007764251 conn(0x7f3bbc0c3b40 msgr2=0x7f3bbc14c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 shutdown_connections 2024-03-06T07:28:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:58.962+0000 7f3bc34df640 1 -- 172.21.15.17:0/1007764251 wait complete. 2024-03-06T07:28:58.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:28:59.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.087+0000 7f8c49dac640 1 Processor -- start 2024-03-06T07:28:59.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.087+0000 7f8c49dac640 1 -- start start 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c44155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c440c6780 0x7f8c44072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 0x7f8c44158780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c44156150 con 0x7f8c440c6780 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c440a7d70 con 0x7f8c44072a50 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c49dac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440a8050 con 0x7f8c4406fe20 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c440c6780 0x7f8c44072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c440c6780 0x7f8c44072510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55884/0 (socket says 172.21.15.17:55884) 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 -- 172.21.15.17:0/395323865 learned_addr learned my addr 172.21.15.17:0/395323865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c437fe640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c44155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c43fff640 1 --2- 172.21.15.17:0/395323865 >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 0x7f8c44158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 -- 172.21.15.17:0/395323865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 msgr2=0x7f8c44155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c44155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 -- 172.21.15.17:0/395323865 >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 msgr2=0x7f8c44158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 --2- 172.21.15.17:0/395323865 >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 0x7f8c44158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c42ffd640 1 -- 172.21.15.17:0/395323865 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440a8330 con 0x7f8c440c6780 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.088+0000 7f8c43fff640 1 --2- 172.21.15.17:0/395323865 >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 0x7f8c44158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.089+0000 7f8c42ffd640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44072510 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f8c440722d0 tx=0x7f8c3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d168e0bcb354d347 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.089+0000 7f8c427fc640 1 -- 172.21.15.17:0/395323865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3802e070 con 0x7f8c440c6780 2024-03-06T07:28:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.089+0000 7f8c427fc640 1 -- 172.21.15.17:0/395323865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38037070 con 0x7f8c440c6780 2024-03-06T07:28:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.089+0000 7f8c427fc640 1 -- 172.21.15.17:0/395323865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38031a00 con 0x7f8c440c6780 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 -- 172.21.15.17:0/395323865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 msgr2=0x7f8c44072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44072510 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f8c440722d0 tx=0x7f8c3802b880 comp rx=0 tx=0).stop 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 -- 172.21.15.17:0/395323865 shutdown_connections 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 --2- 172.21.15.17:0/395323865 >> v2:172.21.15.163:3300/0 conn(0x7f8c44072a50 0x7f8c44158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44072510 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 --2- 172.21.15.17:0/395323865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c44155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 -- 172.21.15.17:0/395323865 >> 172.21.15.17:0/395323865 conn(0x7f8c440aa630 msgr2=0x7f8c440aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 -- 172.21.15.17:0/395323865 shutdown_connections 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.090+0000 7f8c49dac640 1 -- 172.21.15.17:0/395323865 wait complete. 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.091+0000 7f8c49dac640 1 Processor -- start 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.091+0000 7f8c49dac640 1 -- start start 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c441eaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c44072a50 0x7f8c441eb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c44070200 con 0x7f8c440c6780 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440c3600 con 0x7f8c44072a50 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c49dac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c44158cc0 con 0x7f8c4406fe20 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55890/0 (socket says 172.21.15.17:55890) 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 -- 172.21.15.17:0/1827677997 learned_addr learned my addr 172.21.15.17:0/1827677997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c42ffd640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c44072a50 0x7f8c441eb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c437fe640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c441eaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 -- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 msgr2=0x7f8c441eaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c441eaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 -- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c44072a50 msgr2=0x7f8c441eb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c44072a50 0x7f8c441eb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 -- 172.21.15.17:0/1827677997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c441ebc30 con 0x7f8c440c6780 2024-03-06T07:28:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.092+0000 7f8c43fff640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00ce40 tx=0x7f8c3c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.093+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01a070 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.093+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00d840 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.093+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c441ef0b0 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.093+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00dc30 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.093+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c441585d0 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.094+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4404f330 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.094+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c3c01d070 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.094+0000 7f8c2bfff640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 0x7f8c3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.095+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f8c3c08aae0 con 0x7f8c440c6780 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.095+0000 7f8c437fe640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 0x7f8c3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:28:59.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.095+0000 7f8c437fe640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 0x7f8c3006d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006fd0 tx=0x7f8c34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:28:59.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.099+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c3c059150 con 0x7f8c440c6780 2024-03-06T07:28:59.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.278+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c440730d0 con 0x7f8c440c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.966+0000 7f8c2bfff640 1 -- 172.21.15.17:0/1827677997 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c3c05d1f0 con 0x7f8c440c6780 2024-03-06T07:28:59.969 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 msgr2=0x7f8c3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 0x7f8c3006d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006fd0 tx=0x7f8c34008040 comp rx=0 tx=0).stop 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 msgr2=0x7f8c44158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00ce40 tx=0x7f8c3c00a710 comp rx=0 tx=0).stop 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 shutdown_connections 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8c3006b4d0 0x7f8c3006d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c440c6780 0x7f8c44158090 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c44072a50 0x7f8c441eb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 --2- 172.21.15.17:0/1827677997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c4406fe20 0x7f8c441eaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:28:59.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 >> 172.21.15.17:0/1827677997 conn(0x7f8c440aa630 msgr2=0x7f8c4406edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:28:59.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 shutdown_connections 2024-03-06T07:28:59.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:28:59.970+0000 7f8c49dac640 1 -- 172.21.15.17:0/1827677997 wait complete. 2024-03-06T07:28:59.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:29:00.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.095+0000 7fc5ff65c640 1 Processor -- start 2024-03-06T07:29:00.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.095+0000 7fc5ff65c640 1 -- start start 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5ff65c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f813dc90 0x7fc5f8159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5ff65c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f80c69f0 0x7fc5f80c6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5ff65c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f813e070 con 0x7fc5f80c53f0 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5ff65c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f806c190 con 0x7fc5f80c69f0 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5ff65c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f806bde0 con 0x7fc5f813dc90 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fdbf9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc5f80c69f0 0x7fc5f80c6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fcbf7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc5f80c53f0 0x7fc5f806b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42118/0 (socket says 172.21.15.17:42118) 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fcbf7640 1 -- 172.21.15.17:0/3569585802 learned_addr learned my addr 172.21.15.17:0/3569585802 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:00.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fdbf9640 1 -- 172.21.15.17:0/3569585802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f813dc90 msgr2=0x7fc5f8159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:00.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fd3f8640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f813dc90 0x7fc5f8159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fdbf9640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f813dc90 0x7fc5f8159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5effff640 1 -- 172.21.15.17:0/3569585802 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6502914 0 0) 0x7fc5f813e070 con 0x7fc5f80c53f0 2024-03-06T07:29:00.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fdbf9640 1 -- 172.21.15.17:0/3569585802 >> v1:172.21.15.17:6789/0 conn(0x7fc5f80c53f0 legacy=0x7fc5f806b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.096+0000 7fc5fdbf9640 1 -- 172.21.15.17:0/3569585802 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f813e420 con 0x7fc5f80c69f0 2024-03-06T07:29:00.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.097+0000 7fc5fdbf9640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f80c6ea0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009d90 tx=0x7fc5f002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10141f74f217acc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.097+0000 7fc5effff640 1 -- 172.21.15.17:0/3569585802 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0039070 con 0x7fc5f80c69f0 2024-03-06T07:29:00.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.097+0000 7fc5effff640 1 -- 172.21.15.17:0/3569585802 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f002cbc0 con 0x7fc5f80c69f0 2024-03-06T07:29:00.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.097+0000 7fc5effff640 1 -- 172.21.15.17:0/3569585802 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0034440 con 0x7fc5f80c69f0 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 -- 172.21.15.17:0/3569585802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 msgr2=0x7fc5f80c6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f80c6ea0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009d90 tx=0x7fc5f002be10 comp rx=0 tx=0).stop 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 -- 172.21.15.17:0/3569585802 shutdown_connections 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f80c6ea0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/3569585802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f813dc90 0x7fc5f8159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 -- 172.21.15.17:0/3569585802 >> 172.21.15.17:0/3569585802 conn(0x7fc5f80caa70 msgr2=0x7fc5f80cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 -- 172.21.15.17:0/3569585802 shutdown_connections 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.098+0000 7fc5ff65c640 1 -- 172.21.15.17:0/3569585802 wait complete. 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.099+0000 7fc5ff65c640 1 Processor -- start 2024-03-06T07:29:00.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.099+0000 7fc5ff65c640 1 -- start start 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c53f0 0x7fc5f81390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 0x7fc5f81ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c61f0 con 0x7fc5f80c69f0 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f813e9e0 con 0x7fc5f813dc90 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8159770 con 0x7fc5f80c53f0 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55918/0 (socket says 172.21.15.17:55918) 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 -- 172.21.15.17:0/957206290 learned_addr learned my addr 172.21.15.17:0/957206290 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fd3f8640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c53f0 0x7fc5f81390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fdbf9640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 0x7fc5f81ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 -- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c53f0 msgr2=0x7fc5f81390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c53f0 0x7fc5f81390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 -- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 msgr2=0x7fc5f81ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 0x7fc5f81ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 -- 172.21.15.17:0/957206290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f81ebd60 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fdbf9640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 0x7fc5f81ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5fcbf7640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c4e0 tx=0x7fc5e800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8018070 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8007500 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f81526d0 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.100+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8004a20 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.101+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f81ec020 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.102+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f804f330 con 0x7fc5f80c69f0 2024-03-06T07:29:00.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.102+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5e801b070 con 0x7fc5f80c69f0 2024-03-06T07:29:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.102+0000 7fc5edffb640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.102+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5692+0+0 (secure 0 0 0) 0x7fc5e8089490 con 0x7fc5f80c69f0 2024-03-06T07:29:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.102+0000 7fc5fd3f8640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:00.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.103+0000 7fc5fd3f8640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4006fd0 tx=0x7fc5f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:00.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.107+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5e8058470 con 0x7fc5f80c69f0 2024-03-06T07:29:00.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.287+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc5f80ca5d0 con 0x7fc5f80c69f0 2024-03-06T07:29:00.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.973+0000 7fc5edffb640 1 -- 172.21.15.17:0/957206290 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc5e8057d70 con 0x7fc5f80c69f0 2024-03-06T07:29:00.976 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:29:00.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 msgr2=0x7fc5dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4006fd0 tx=0x7fc5f4008040 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 msgr2=0x7fc5f8139610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c4e0 tx=0x7fc5e800c9a0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 shutdown_connections 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc5dc06b4d0 0x7fc5dc06d990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc5f813dc90 0x7fc5f81ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5f80c69f0 0x7fc5f8139610 secure :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c4e0 tx=0x7fc5e800c9a0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.976+0000 7fc5ff65c640 1 --2- 172.21.15.17:0/957206290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc5f80c53f0 0x7fc5f81390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.977+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 >> 172.21.15.17:0/957206290 conn(0x7fc5f80caa70 msgr2=0x7fc5f806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.977+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 shutdown_connections 2024-03-06T07:29:00.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:00.977+0000 7fc5ff65c640 1 -- 172.21.15.17:0/957206290 wait complete. 2024-03-06T07:29:00.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:29:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.100+0000 7f2ce1983640 1 Processor -- start 2024-03-06T07:29:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 -- start start 2024-03-06T07:29:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc155300 con 0x7f2cdc0c4930 2024-03-06T07:29:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc1555e0 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.101+0000 7f2ce1983640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1558c0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cdb7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:35858/0 (socket says 172.21.15.17:35858) 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cdaffd640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2cdc0c4930 0x7f2cdc0c7ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46360/0 (socket says 172.21.15.17:46360) 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cdb7fe640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:53284/0 (socket says 172.21.15.17:53284) 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 -- 172.21.15.17:0/2933158540 learned_addr learned my addr 172.21.15.17:0/2933158540 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 -- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 msgr2=0x7f2cdc1587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 -- 172.21.15.17:0/2933158540 >> v1:172.21.15.163:6789/0 conn(0x7f2cdc0c4930 legacy=0x7f2cdc0c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cda7fc640 1 -- 172.21.15.17:0/2933158540 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc155ba0 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.102+0000 7f2cdb7fe640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.103+0000 7f2cda7fc640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009870 tx=0x7f2cd002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811b5dabd2da260c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.103+0000 7f2cd9ffb640 1 -- 172.21.15.17:0/2933158540 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0035020 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.103+0000 7f2cd9ffb640 1 -- 172.21.15.17:0/2933158540 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd0031640 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.103+0000 7f2cd9ffb640 1 -- 172.21.15.17:0/2933158540 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd002d040 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 -- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4d10 msgr2=0x7f2cdc0c1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009870 tx=0x7f2cd002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 -- 172.21.15.17:0/2933158540 shutdown_connections 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c21c0 0x7f2cdc1587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2933158540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4d10 0x7f2cdc0c1c80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 -- 172.21.15.17:0/2933158540 >> 172.21.15.17:0/2933158540 conn(0x7f2cdc0aa5c0 msgr2=0x7f2cdc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 -- 172.21.15.17:0/2933158540 shutdown_connections 2024-03-06T07:29:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.104+0000 7f2ce1983640 1 -- 172.21.15.17:0/2933158540 wait complete. 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.105+0000 7f2ce1983640 1 Processor -- start 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.105+0000 7f2ce1983640 1 -- start start 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4930 0x7f2cdc1dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c4d10 0x7f2cdc1e0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c81c0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1560c0 con 0x7f2cdc0c4930 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2ce1983640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc0c8340 con 0x7f2cdc0c4d10 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38674/0 (socket says 172.21.15.17:38674) 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 -- 172.21.15.17:0/2859875751 learned_addr learned my addr 172.21.15.17:0/2859875751 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cda7fc640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4930 0x7f2cdc1dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdb7fe640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c4d10 0x7f2cdc1e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 -- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c4d10 msgr2=0x7f2cdc1e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c4d10 0x7f2cdc1e0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 -- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4930 msgr2=0x7f2cdc1dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4930 0x7f2cdc1dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.106+0000 7f2cdaffd640 1 -- 172.21.15.17:0/2859875751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc1eb920 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2cdaffd640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c400 tx=0x7f2ccc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0074e0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc007680 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1db600 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc004a00 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.107+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc0adc60 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.108+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc1db3f0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.108+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ccc01a020 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.109+0000 7f2cc3fff640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 0x7f2cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.109+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2ccc089710 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.109+0000 7f2cda7fc640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 0x7f2cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.110+0000 7f2cda7fc640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 0x7f2cc806d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0036000 tx=0x7f2cd003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.113+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2ccc0587e0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.294+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2cdc04f330 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.297+0000 7f2cc3fff640 1 -- 172.21.15.17:0/2859875751 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ccc0580e0 con 0x7f2cdc0c21c0 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 msgr2=0x7f2cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 0x7f2cc806d990 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0036000 tx=0x7f2cd003c040 comp rx=0 tx=0).stop 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 msgr2=0x7f2cdc1dc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c400 tx=0x7f2ccc00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 shutdown_connections 2024-03-06T07:29:01.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2cc806b4d0 0x7f2cc806d990 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cdc0c4d10 0x7f2cdc1e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2cdc0c4930 0x7f2cdc1dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 --2- 172.21.15.17:0/2859875751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cdc0c21c0 0x7f2cdc1dc050 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.299+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 >> 172.21.15.17:0/2859875751 conn(0x7f2cdc0aa5c0 msgr2=0x7f2cdc0aa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.300+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 shutdown_connections 2024-03-06T07:29:01.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.300+0000 7f2ce1983640 1 -- 172.21.15.17:0/2859875751 wait complete. 2024-03-06T07:29:01.313 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health' 2024-03-06T07:29:01.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.417+0000 7fae27701640 1 Processor -- start 2024-03-06T07:29:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.417+0000 7fae27701640 1 -- start start 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae27701640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae201550c0 0x7fae201554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae27701640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae201559e0 0x7fae2014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae27701640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae200aa660 con 0x7fae200aa180 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae27701640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae200c4610 con 0x7fae201559e0 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae27701640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae200aa940 con 0x7fae201550c0 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae201559e0 0x7fae2014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fae201559e0 0x7fae2014ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38678/0 (socket says 172.21.15.17:38678) 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 -- 172.21.15.17:0/111203389 learned_addr learned my addr 172.21.15.17:0/111203389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 -- 172.21.15.17:0/111203389 >> v2:172.21.15.163:3300/0 conn(0x7fae201550c0 msgr2=0x7fae201554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 --2- 172.21.15.17:0/111203389 >> v2:172.21.15.163:3300/0 conn(0x7fae201550c0 0x7fae201554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 -- 172.21.15.17:0/111203389 >> v1:172.21.15.163:6789/0 conn(0x7fae200aa180 legacy=0x7fae200c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:01.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.418+0000 7fae24c9c640 1 -- 172.21.15.17:0/111203389 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200c4860 con 0x7fae201559e0 2024-03-06T07:29:01.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae24c9c640 1 --2- 172.21.15.17:0/111203389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae2014ec00 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fae200c3fc0 tx=0x7fae1002dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c52fcc9076d844 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae17fff640 1 -- 172.21.15.17:0/111203389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10034070 con 0x7fae201559e0 2024-03-06T07:29:01.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae17fff640 1 -- 172.21.15.17:0/111203389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1002eb50 con 0x7fae201559e0 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae17fff640 1 -- 172.21.15.17:0/111203389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10038400 con 0x7fae201559e0 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae27701640 1 -- 172.21.15.17:0/111203389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 msgr2=0x7fae2014ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.419+0000 7fae27701640 1 --2- 172.21.15.17:0/111203389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae2014ec00 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fae200c3fc0 tx=0x7fae1002dda0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 -- 172.21.15.17:0/111203389 shutdown_connections 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 --2- 172.21.15.17:0/111203389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae2014ec00 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 --2- 172.21.15.17:0/111203389 >> v2:172.21.15.163:3300/0 conn(0x7fae201550c0 0x7fae201554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 -- 172.21.15.17:0/111203389 >> 172.21.15.17:0/111203389 conn(0x7fae200c6710 msgr2=0x7fae200c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 -- 172.21.15.17:0/111203389 shutdown_connections 2024-03-06T07:29:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.420+0000 7fae27701640 1 -- 172.21.15.17:0/111203389 wait complete. 2024-03-06T07:29:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 Processor -- start 2024-03-06T07:29:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 -- start start 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200aa180 0x7fae20154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 0x7fae201ee040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2014f140 con 0x7fae201559e0 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200c1560 con 0x7fae200aa180 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae27701640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2014f2c0 con 0x7fae201550c0 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae25c9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae25c9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38682/0 (socket says 172.21.15.17:38682) 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.421+0000 7fae25c9e640 1 -- 172.21.15.17:0/1758615793 learned_addr learned my addr 172.21.15.17:0/1758615793 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae2549d640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200aa180 0x7fae20154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae24c9c640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 0x7fae201ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 -- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 msgr2=0x7fae201ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 0x7fae201ee040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 -- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200aa180 msgr2=0x7fae20154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200aa180 0x7fae20154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 -- 172.21.15.17:0/1758615793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200bfda0 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae24c9c640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 0x7fae201ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae25c9e640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fae1800c780 tx=0x7fae1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18017070 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18003da0 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180052b0 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae20143c00 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.422+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae200a6870 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.423+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2004f330 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.424+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae1801a070 con 0x7fae201559e0 2024-03-06T07:29:01.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.424+0000 7fae15ffb640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 0x7fae0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.424+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fae18089610 con 0x7fae201559e0 2024-03-06T07:29:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.424+0000 7fae2549d640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 0x7fae0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.425+0000 7fae2549d640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 0x7fae0406d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009890 tx=0x7fae1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:01.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.428+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae180586e0 con 0x7fae201559e0 2024-03-06T07:29:01.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.683+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]} v 0) v1 -- 0x7fae200a63c0 con 0x7fae201559e0 2024-03-06T07:29:01.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.974+0000 7fae15ffb640 1 -- 172.21.15.17:0/1758615793 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_health"]}]=0 v511) v1 ==== 140+0+0 (secure 0 0 0) 0x7fae18057fe0 con 0x7fae201559e0 2024-03-06T07:29:01.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.976+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 msgr2=0x7fae0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.976+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 0x7fae0406d990 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009890 tx=0x7fae1c008040 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 msgr2=0x7fae201ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fae1800c780 tx=0x7fae1800cc40 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 shutdown_connections 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fae0406b4d0 0x7fae0406d990 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae201559e0 0x7fae201ee580 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae201550c0 0x7fae201ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 --2- 172.21.15.17:0/1758615793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae200aa180 0x7fae20154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 >> 172.21.15.17:0/1758615793 conn(0x7fae200c6710 msgr2=0x7fae20152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 shutdown_connections 2024-03-06T07:29:01.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:01.977+0000 7fae27701640 1 -- 172.21.15.17:0/1758615793 wait complete. 2024-03-06T07:29:01.991 INFO:tasks.cephfs_test_runner:That marking a FS down does not generate a health warning ... ok 2024-03-06T07:29:01.991 INFO:tasks.cephfs_test_runner:Starting test: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:29:01.991 INFO:tasks.cephfs_test_runner:test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:29:01.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.092+0000 7fc68106f640 1 Processor -- start 2024-03-06T07:29:02.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.092+0000 7fc68106f640 1 -- start start 2024-03-06T07:29:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.092+0000 7fc68106f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc67c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.092+0000 7fc68106f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc67c0c7380 0x7fc67c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc68106f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67c156050 con 0x7fc67c0c6780 2024-03-06T07:29:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc68106f640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67c06de20 con 0x7fc67c0c7380 2024-03-06T07:29:02.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc68106f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c072630 con 0x7fc67c06da40 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc673fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc67c0c6780 0x7fc67c158d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45126/0 (socket says 172.21.15.17:45126) 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc673fff640 1 -- 172.21.15.17:0/1342419438 learned_addr learned my addr 172.21.15.17:0/1342419438 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67b59e640 1 --2- 172.21.15.17:0/1342419438 >> v2:172.21.15.163:3300/0 conn(0x7fc67c0c7380 0x7fc67c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc67c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012175624 0 0) 0x7fc664001110 con 0x7fc67c0c6780 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc668003520 con 0x7fc67c0c6780 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1630497910 0 0) 0x7fc668003520 con 0x7fc67c0c6780 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 msgr2=0x7fc67c155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc67c155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 >> v2:172.21.15.163:3300/0 conn(0x7fc67c0c7380 msgr2=0x7fc67c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 --2- 172.21.15.17:0/1342419438 >> v2:172.21.15.163:3300/0 conn(0x7fc67c0c7380 0x7fc67c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c072910 con 0x7fc67c0c6780 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc664001840 con 0x7fc67c0c6780 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 >> v1:172.21.15.17:6789/0 conn(0x7fc67c0c6780 legacy=0x7fc67c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6680039b0 0x7fc668003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc668004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc6680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c06de20 con 0x7fc668004d40 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c072630 con 0x7fc6680039b0 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66800b9c0 con 0x7fc67c06da40 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67b59e640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc6680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6680039b0 0x7fc668003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.093+0000 7fc673fff640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc668004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 -- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 msgr2=0x7fc668004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c06da40 0x7fc668004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 -- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6680039b0 msgr2=0x7fc668003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6680039b0 0x7fc668003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 -- 172.21.15.17:0/1342419438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668003520 con 0x7fc668004d40 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6680039b0 0x7fc668003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.094+0000 7fc67b59e640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc6680051c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fc674009f90 tx=0x7fc668005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f88048797a3fdd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674039070 con 0x7fc668004d40 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67402cc20 con 0x7fc668004d40 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc67a59c640 1 -- 172.21.15.17:0/1342419438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674009c10 con 0x7fc668004d40 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 -- 172.21.15.17:0/1342419438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 msgr2=0x7fc6680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 --2- 172.21.15.17:0/1342419438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc6680051c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fc674009f90 tx=0x7fc668005830 comp rx=0 tx=0).stop 2024-03-06T07:29:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc67b59e640 1 -- 172.21.15.17:0/1342419438 reap_dead start 2024-03-06T07:29:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 -- 172.21.15.17:0/1342419438 shutdown_connections 2024-03-06T07:29:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 -- 172.21.15.17:0/1342419438 >> 172.21.15.17:0/1342419438 conn(0x7fc67c0aa630 msgr2=0x7fc67c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 -- 172.21.15.17:0/1342419438 shutdown_connections 2024-03-06T07:29:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.095+0000 7fc68106f640 1 -- 172.21.15.17:0/1342419438 wait complete. 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.096+0000 7fc68106f640 1 Processor -- start 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.096+0000 7fc68106f640 1 -- start start 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 0x7fc67c14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c153160 0x7fc67c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c1594a0 con 0x7fc668004d40 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c158890 con 0x7fc67c152c30 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc68106f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c0c37c0 con 0x7fc67c153160 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38696/0 (socket says 172.21.15.17:38696) 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 -- 172.21.15.17:0/890802438 learned_addr learned my addr 172.21.15.17:0/890802438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc673fff640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 0x7fc67c14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67b59e640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c153160 0x7fc67c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 -- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c153160 msgr2=0x7fc67c14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c153160 0x7fc67c14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 -- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 msgr2=0x7fc67c14eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 0x7fc67c14eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc67ad9d640 1 -- 172.21.15.17:0/890802438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c1dfc10 con 0x7fc668004d40 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.097+0000 7fc673fff640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 0x7fc67c14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc67ad9d640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fc66c011570 tx=0x7fc66c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c015870 con 0x7fc668004d40 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c015a10 con 0x7fc668004d40 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67c1dfef0 con 0x7fc668004d40 2024-03-06T07:29:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c015dc0 con 0x7fc668004d40 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.098+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67c0ab750 con 0x7fc668004d40 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.099+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67c0c4bf0 con 0x7fc668004d40 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.099+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc66c00f460 con 0x7fc668004d40 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.100+0000 7fc6737fe640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 0x7fc65c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.100+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc66c08a880 con 0x7fc668004d40 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.100+0000 7fc673fff640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 0x7fc65c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.100+0000 7fc673fff640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 0x7fc65c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc6640012f0 tx=0x7fc664009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.104+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc67c0c4bf0 con 0x7fc668004d40 2024-03-06T07:29:02.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.283+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc67c04f330 con 0x7fc668004d40 2024-03-06T07:29:02.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.284+0000 7fc6737fe640 1 -- 172.21.15.17:0/890802438 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+13174 (secure 0 0 0) 0x7fc66c019030 con 0x7fc668004d40 2024-03-06T07:29:02.287 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:02.287 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":203,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:29:00.968856+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 msgr2=0x7fc65c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 0x7fc65c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc6640012f0 tx=0x7fc664009040 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 msgr2=0x7fc67c157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fc66c011570 tx=0x7fc66c011a30 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 shutdown_connections 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc65c06b4d0 0x7fc65c06d990 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc67c153160 0x7fc67c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc67c152c30 0x7fc67c14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 --2- 172.21.15.17:0/890802438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc668004d40 0x7fc67c157690 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 >> 172.21.15.17:0/890802438 conn(0x7fc67c0aa630 msgr2=0x7fc67c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 shutdown_connections 2024-03-06T07:29:02.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.286+0000 7fc68106f640 1 -- 172.21.15.17:0/890802438 wait complete. 2024-03-06T07:29:02.300 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:29:02.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:29:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.401+0000 7f904e793640 1 Processor -- start 2024-03-06T07:29:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.401+0000 7f904e793640 1 -- start start 2024-03-06T07:29:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904e793640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90481529d0 0x7f90480c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904e793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048072cd0 0x7f9048155830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904e793640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9048152db0 con 0x7f90480c57e0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904e793640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90480c6dd0 con 0x7f90481529d0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904e793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90480c6940 con 0x7f9048072cd0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90481529d0 0x7f90480c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90481529d0 0x7f90480c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38698/0 (socket says 172.21.15.17:38698) 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 -- 172.21.15.17:0/1454599043 learned_addr learned my addr 172.21.15.17:0/1454599043 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f904cd30640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048072cd0 0x7f9048155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 -- 172.21.15.17:0/1454599043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048072cd0 msgr2=0x7f9048155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048072cd0 0x7f9048155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 -- 172.21.15.17:0/1454599043 >> v1:172.21.15.163:6789/0 conn(0x7f90480c57e0 legacy=0x7f9048155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.402+0000 7f9047fff640 1 -- 172.21.15.17:0/1454599043 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90481590f0 con 0x7f90481529d0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.403+0000 7f9047fff640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90481529d0 0x7f90480c52a0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f9038009870 tx=0x7f903802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae29681f885cddf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.403+0000 7f9046ffd640 1 -- 172.21.15.17:0/1454599043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038038070 con 0x7f90481529d0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.403+0000 7f9046ffd640 1 -- 172.21.15.17:0/1454599043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038031740 con 0x7f90481529d0 2024-03-06T07:29:02.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.403+0000 7f9046ffd640 1 -- 172.21.15.17:0/1454599043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903802f050 con 0x7f90481529d0 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 -- 172.21.15.17:0/1454599043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90481529d0 msgr2=0x7f90480c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90481529d0 0x7f90480c52a0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f9038009870 tx=0x7f903802b7e0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 -- 172.21.15.17:0/1454599043 shutdown_connections 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9048072cd0 0x7f9048155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 --2- 172.21.15.17:0/1454599043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90481529d0 0x7f90480c52a0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 -- 172.21.15.17:0/1454599043 >> 172.21.15.17:0/1454599043 conn(0x7f90480aa630 msgr2=0x7f90480aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 -- 172.21.15.17:0/1454599043 shutdown_connections 2024-03-06T07:29:02.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.404+0000 7f904e793640 1 -- 172.21.15.17:0/1454599043 wait complete. 2024-03-06T07:29:02.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.405+0000 7f904e793640 1 Processor -- start 2024-03-06T07:29:02.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.405+0000 7f904e793640 1 -- start start 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90480c57e0 0x7f9048154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90481529d0 0x7f9048157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048072720 con 0x7f9048072cd0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048155d70 con 0x7f90480c57e0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904e793640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9048159370 con 0x7f90481529d0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38712/0 (socket says 172.21.15.17:38712) 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 -- 172.21.15.17:0/2183926581 learned_addr learned my addr 172.21.15.17:0/2183926581 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f904cd30640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90481529d0 0x7f9048157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f90477fe640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90480c57e0 0x7f9048154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 -- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90481529d0 msgr2=0x7f9048157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90481529d0 0x7f9048157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 -- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90480c57e0 msgr2=0x7f9048154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90480c57e0 0x7f9048154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.406+0000 7f9047fff640 1 -- 172.21.15.17:0/2183926581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904814e870 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f9047fff640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f90380097c0 tx=0x7f90380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038042070 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038031ec0 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038038040 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9048154a70 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.407+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90481580a0 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.408+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904804f330 con 0x7f9048072cd0 2024-03-06T07:29:02.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.408+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9038045070 con 0x7f9048072cd0 2024-03-06T07:29:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.409+0000 7f9044ff9640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 0x7f902806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.409+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f90380abaf0 con 0x7f9048072cd0 2024-03-06T07:29:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.409+0000 7f90477fe640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 0x7f902806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.409+0000 7f90477fe640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 0x7f902806d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f90481ebf40 tx=0x7f9030007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.413+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f903804c030 con 0x7f9048072cd0 2024-03-06T07:29:02.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.595+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9048153d60 con 0x7f9048072cd0 2024-03-06T07:29:02.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.595+0000 7f9044ff9640 1 -- 172.21.15.17:0/2183926581 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v203) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90481590f0 con 0x7f9048072cd0 2024-03-06T07:29:02.598 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 msgr2=0x7f902806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 0x7f902806d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f90481ebf40 tx=0x7f9030007e30 comp rx=0 tx=0).stop 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 msgr2=0x7f904814bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f90380097c0 tx=0x7f90380046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 shutdown_connections 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f902806b4d0 0x7f902806d990 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90481529d0 0x7f9048157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90480c57e0 0x7f9048154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 --2- 172.21.15.17:0/2183926581 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9048072cd0 0x7f904814bb10 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 >> 172.21.15.17:0/2183926581 conn(0x7f90480aa630 msgr2=0x7f90480a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 shutdown_connections 2024-03-06T07:29:02.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.597+0000 7f904e793640 1 -- 172.21.15.17:0/2183926581 wait complete. 2024-03-06T07:29:02.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice' 2024-03-06T07:29:02.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.710+0000 7fc432e84640 1 Processor -- start 2024-03-06T07:29:02.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.710+0000 7fc432e84640 1 -- start start 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.710+0000 7fc432e84640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc42c072720 0x7fc42c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc432e84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c156f90 con 0x7fc42c0c3d60 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc432e84640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc42c157270 con 0x7fc42c156a00 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc432e84640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc42c0c38d0 con 0x7fc42c072720 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc431421640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc42c0c3d60 0x7fc42c0c41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45138/0 (socket says 172.21.15.17:45138) 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc431421640 1 -- 172.21.15.17:0/3315136115 learned_addr learned my addr 172.21.15.17:0/3315136115 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505047333 0 0) 0x7fc42c156f90 con 0x7fc42c0c3d60 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc41c003520 con 0x7fc42c0c3d60 2024-03-06T07:29:02.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc430c20640 1 --2- 172.21.15.17:0/3315136115 >> v2:172.21.15.163:3300/0 conn(0x7fc42c072720 0x7fc42c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394263223 0 0) 0x7fc41c003520 con 0x7fc42c0c3d60 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 >> v2:172.21.15.163:3300/0 conn(0x7fc42c072720 msgr2=0x7fc42c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 --2- 172.21.15.17:0/3315136115 >> v2:172.21.15.163:3300/0 conn(0x7fc42c072720 0x7fc42c1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 >> v1:172.21.15.163:6789/0 conn(0x7fc42c156a00 legacy=0x7fc42c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c0c3280 con 0x7fc42c0c3d60 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc4240019f0 con 0x7fc42c0c3d60 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 >> v1:172.21.15.17:6789/0 conn(0x7fc42c0c3d60 legacy=0x7fc42c0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 0x7fc41c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc41c005210 0x7fc41c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c0c38d0 con 0x7fc41c004800 2024-03-06T07:29:02.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00c160 con 0x7fc41c005210 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c00c360 con 0x7fc41c003a00 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.711+0000 7fc42bfff640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 0x7fc41c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc430c20640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc431421640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc41c005210 0x7fc41c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 -- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c003a00 msgr2=0x7fc41c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 -- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc41c005210 msgr2=0x7fc41c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc41c005210 0x7fc41c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 -- 172.21.15.17:0/3315136115 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c003520 con 0x7fc41c004800 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc431421640 1 -- 172.21.15.17:0/3315136115 reap_dead start 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc430c20640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c003a00 0x7fc41c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.712+0000 7fc42bfff640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 0x7fc41c004c80 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fc418001720 tx=0x7fc41c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77abd9af8816921 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418001370 con 0x7fc41c004800 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4180052d0 con 0x7fc41c004800 2024-03-06T07:29:02.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc42b7fe640 1 -- 172.21.15.17:0/3315136115 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418003830 con 0x7fc41c004800 2024-03-06T07:29:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 -- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 msgr2=0x7fc41c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 0x7fc41c004c80 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fc418001720 tx=0x7fc41c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 -- 172.21.15.17:0/3315136115 shutdown_connections 2024-03-06T07:29:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 --2- 172.21.15.17:0/3315136115 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc41c004800 0x7fc41c004c80 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 -- 172.21.15.17:0/3315136115 >> 172.21.15.17:0/3315136115 conn(0x7fc42c0aa630 msgr2=0x7fc42c0aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 -- 172.21.15.17:0/3315136115 shutdown_connections 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.713+0000 7fc432e84640 1 -- 172.21.15.17:0/3315136115 wait complete. 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.714+0000 7fc432e84640 1 Processor -- start 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.714+0000 7fc432e84640 1 -- start start 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c004800 0x7fc42c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1eab70 0x7fc42c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c072b00 con 0x7fc42c06c560 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c06f8f0 con 0x7fc42c1eab70 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc432e84640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42c072c80 con 0x7fc41c004800 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38736/0 (socket says 172.21.15.17:38736) 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 -- 172.21.15.17:0/657535768 learned_addr learned my addr 172.21.15.17:0/657535768 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 -- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c004800 msgr2=0x7fc42c155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc430c20640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c004800 0x7fc42c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc42bfff640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1eab70 0x7fc42c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c004800 0x7fc42c155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 -- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1eab70 msgr2=0x7fc42c06d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1eab70 0x7fc42c06d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.715+0000 7fc431421640 1 -- 172.21.15.17:0/657535768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42c154230 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc431421640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0721a0 tx=0x7fc424007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42400e9e0 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc424018070 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42c1eb0e0 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42400edc0 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.716+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc42c1eb6a0 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.717+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42c04f330 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.717+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc42401d460 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.718+0000 7fc4297fa640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 0x7fc41006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.718+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc42408a330 con 0x7fc42c06c560 2024-03-06T07:29:02.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.718+0000 7fc430c20640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 0x7fc41006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:02.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.718+0000 7fc430c20640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 0x7fc41006d990 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc42c069cc0 tx=0x7fc420008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:02.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.722+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc42c154230 con 0x7fc42c06c560 2024-03-06T07:29:02.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.971+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]} v 0) v1 -- 0x7fc42c150ad0 con 0x7fc42c06c560 2024-03-06T07:29:02.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.976+0000 7fc4297fa640 1 -- 172.21.15.17:0/657535768 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]}]=0 v512) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc424059370 con 0x7fc42c06c560 2024-03-06T07:29:02.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.977+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 msgr2=0x7fc41006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.977+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 0x7fc41006d990 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc42c069cc0 tx=0x7fc420008040 comp rx=0 tx=0).stop 2024-03-06T07:29:02.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.977+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 msgr2=0x7fc42c06c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:02.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.977+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fc42c0721a0 tx=0x7fc424007af0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 shutdown_connections 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc41006b4d0 0x7fc41006d990 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc42c06c560 0x7fc42c06c9e0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc42c1eab70 0x7fc42c06d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 --2- 172.21.15.17:0/657535768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc41c004800 0x7fc42c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 >> 172.21.15.17:0/657535768 conn(0x7fc42c0aa630 msgr2=0x7fc42c0c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 shutdown_connections 2024-03-06T07:29:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:02.978+0000 7fc432e84640 1 -- 172.21.15.17:0/657535768 wait complete. 2024-03-06T07:29:02.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-06T07:29:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.092+0000 7f7137bc7640 1 Processor -- start 2024-03-06T07:29:03.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 -- start start 2024-03-06T07:29:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71300b0070 0x7f7130155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f713006ca70 0x7f71300c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71300b0450 con 0x7f71300c5380 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713006c020 con 0x7f71300b0070 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7137bc7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713006cf20 con 0x7f713006ca70 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7135963640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f71300b0070 0x7f7130155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7136164640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f71300c5380 0x7f713014e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45142/0 (socket says 172.21.15.17:45142) 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.093+0000 7f7136164640 1 -- 172.21.15.17:0/1350386355 learned_addr learned my addr 172.21.15.17:0/1350386355 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7135162640 1 --2- 172.21.15.17:0/1350386355 >> v2:172.21.15.163:3300/0 conn(0x7f713006ca70 0x7f71300c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7135963640 1 -- 172.21.15.17:0/1350386355 >> v2:172.21.15.163:3300/0 conn(0x7f713006ca70 msgr2=0x7f71300c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7135963640 1 --2- 172.21.15.17:0/1350386355 >> v2:172.21.15.163:3300/0 conn(0x7f713006ca70 0x7f71300c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7135963640 1 -- 172.21.15.17:0/1350386355 >> v1:172.21.15.17:6789/0 conn(0x7f71300c5380 legacy=0x7f713014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7134961640 1 -- 172.21.15.17:0/1350386355 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293937121 0 0) 0x7f7120001490 con 0x7f71300c5380 2024-03-06T07:29:03.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.094+0000 7f7135963640 1 -- 172.21.15.17:0/1350386355 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300c77f0 con 0x7f71300b0070 2024-03-06T07:29:03.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7135963640 1 --2- 172.21.15.17:0/1350386355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71300b0070 0x7f7130155c30 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f713006d4c0 tx=0x7f712802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9fe19a14a9b8aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7134961640 1 -- 172.21.15.17:0/1350386355 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71280315a0 con 0x7f71300b0070 2024-03-06T07:29:03.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7134961640 1 -- 172.21.15.17:0/1350386355 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128037070 con 0x7f71300b0070 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7134961640 1 -- 172.21.15.17:0/1350386355 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128031960 con 0x7f71300b0070 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 -- 172.21.15.17:0/1350386355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71300b0070 msgr2=0x7f7130155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 --2- 172.21.15.17:0/1350386355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71300b0070 0x7f7130155c30 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f713006d4c0 tx=0x7f712802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 -- 172.21.15.17:0/1350386355 shutdown_connections 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 --2- 172.21.15.17:0/1350386355 >> v2:172.21.15.163:3300/0 conn(0x7f713006ca70 0x7f71300c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 --2- 172.21.15.17:0/1350386355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71300b0070 0x7f7130155c30 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 -- 172.21.15.17:0/1350386355 >> 172.21.15.17:0/1350386355 conn(0x7f71300ac590 msgr2=0x7f71300ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 -- 172.21.15.17:0/1350386355 shutdown_connections 2024-03-06T07:29:03.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.095+0000 7f7137bc7640 1 -- 172.21.15.17:0/1350386355 wait complete. 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.096+0000 7f7137bc7640 1 Processor -- start 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.096+0000 7f7137bc7640 1 -- start start 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 0x7f71300c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 0x7f7130154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713006c550 con 0x7f713006ca70 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713006c270 con 0x7f71300c5380 2024-03-06T07:29:03.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7137bc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713006da80 con 0x7f71300b0070 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38754/0 (socket says 172.21.15.17:38754) 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 -- 172.21.15.17:0/218927792 learned_addr learned my addr 172.21.15.17:0/218927792 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135162640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 0x7f71300c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7136164640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 0x7f7130154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 -- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 msgr2=0x7f71300c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 0x7f71300c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 -- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 msgr2=0x7f7130154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 0x7f7130154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.097+0000 7f7135963640 1 -- 172.21.15.17:0/218927792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713014e760 con 0x7f713006ca70 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f7135162640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 0x7f71300c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f7136164640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 0x7f7130154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f7135963640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f7128009890 tx=0x7f71280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128032840 con 0x7f713006ca70 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7128032a00 con 0x7f713006ca70 2024-03-06T07:29:03.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300c4200 con 0x7f713006ca70 2024-03-06T07:29:03.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128032df0 con 0x7f713006ca70 2024-03-06T07:29:03.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.098+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300ca510 con 0x7f713006ca70 2024-03-06T07:29:03.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.099+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713004f330 con 0x7f713006ca70 2024-03-06T07:29:03.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.099+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f712804c020 con 0x7f713006ca70 2024-03-06T07:29:03.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.100+0000 7f712e7fc640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.100+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f712802f020 con 0x7f713006ca70 2024-03-06T07:29:03.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.100+0000 7f7135162640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.101+0000 7f7135162640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 0x7f711c06d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.104+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7128040070 con 0x7f713006ca70 2024-03-06T07:29:03.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.285+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f713006d860 con 0x7f713006ca70 2024-03-06T07:29:03.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.288+0000 7f712e7fc640 1 -- 172.21.15.17:0/218927792 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v63)=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f712807af30 con 0x7f713006ca70 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 msgr2=0x7f711c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 0x7f711c06d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124008040 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 msgr2=0x7f71300c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f7128009890 tx=0x7f71280046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 shutdown_connections 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f711c06b4d0 0x7f711c06d990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71300c5380 0x7f7130154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71300b0070 0x7f71300c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 --2- 172.21.15.17:0/218927792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f713006ca70 0x7f71300c3460 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 >> 172.21.15.17:0/218927792 conn(0x7f71300ac590 msgr2=0x7f7130151330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 shutdown_connections 2024-03-06T07:29:03.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.290+0000 7f7137bc7640 1 -- 172.21.15.17:0/218927792 wait complete. 2024-03-06T07:29:03.303 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:29:03.303 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:29:03.342 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:29:03.387 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:29:03.387 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:29:03.388 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:29:03.402 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:29:03.402 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:29:03.402 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:29:03.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:29:03.419 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-03-06T07:29:03.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.541+0000 7fc26bdc9640 1 Processor -- start 2024-03-06T07:29:03.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 -- start start 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc264155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc264158b60 0x7fc264158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2640c73d0 con 0x7fc2640c88c0 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2640c8dd0 con 0x7fc264158b60 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc26bdc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640be2e0 con 0x7fc26414f360 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc269364640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc2640c88c0 0x7fc2640c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45148/0 (socket says 172.21.15.17:45148) 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.542+0000 7fc269364640 1 -- 172.21.15.17:0/2802507183 learned_addr learned my addr 172.21.15.17:0/2802507183 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc26a366640 1 --2- 172.21.15.17:0/2802507183 >> v2:172.21.15.163:3300/0 conn(0x7fc264158b60 0x7fc264158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269b65640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc264155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050826393 0 0) 0x7fc2640c73d0 con 0x7fc2640c88c0 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003520 con 0x7fc2640c88c0 2024-03-06T07:29:03.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3178134201 0 0) 0x7fc24c003520 con 0x7fc2640c88c0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 msgr2=0x7fc264155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc264155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 >> v2:172.21.15.163:3300/0 conn(0x7fc264158b60 msgr2=0x7fc264158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 --2- 172.21.15.17:0/2802507183 >> v2:172.21.15.163:3300/0 conn(0x7fc264158b60 0x7fc264158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640c7670 con 0x7fc2640c88c0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc258001670 con 0x7fc2640c88c0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 >> v1:172.21.15.17:6789/0 conn(0x7fc2640c88c0 legacy=0x7fc2640c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc24c0039b0 0x7fc24c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc24c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc24c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640be2e0 con 0x7fc24c004800 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c00b9c0 con 0x7fc24c0039b0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c00bb40 con 0x7fc26414f360 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc24c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269b65640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc24c0039b0 0x7fc24c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc26a366640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc24c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 -- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 msgr2=0x7fc24c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc26414f360 0x7fc24c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 -- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc24c0039b0 msgr2=0x7fc24c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc24c0039b0 0x7fc24c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269364640 1 -- 172.21.15.17:0/2802507183 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c003520 con 0x7fc24c004800 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.543+0000 7fc269b65640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc24c0039b0 0x7fc24c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.544+0000 7fc269364640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc24c004c80 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fc2640c8d20 tx=0x7fc24c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1117ebf82ee5adb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.544+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258007d70 con 0x7fc24c004800 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.544+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258030070 con 0x7fc24c004800 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.544+0000 7fc268b63640 1 -- 172.21.15.17:0/2802507183 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258005d30 con 0x7fc24c004800 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2802507183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 msgr2=0x7fc24c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2802507183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc24c004c80 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7fc2640c8d20 tx=0x7fc24c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26a366640 1 -- 172.21.15.17:0/2802507183 reap_dead start 2024-03-06T07:29:03.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2802507183 shutdown_connections 2024-03-06T07:29:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2802507183 >> 172.21.15.17:0/2802507183 conn(0x7fc2640ac570 msgr2=0x7fc2640ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2802507183 shutdown_connections 2024-03-06T07:29:03.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.545+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2802507183 wait complete. 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.546+0000 7fc26bdc9640 1 Processor -- start 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.546+0000 7fc26bdc9640 1 -- start start 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26414c3e0 0x7fc26414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 0x7fc26414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640c5a50 con 0x7fc24c004800 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640c7c30 con 0x7fc26414c3e0 2024-03-06T07:29:03.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640c5bd0 con 0x7fc2640aeb50 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38774/0 (socket says 172.21.15.17:38774) 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 -- 172.21.15.17:0/2697087537 learned_addr learned my addr 172.21.15.17:0/2697087537 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269364640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26414c3e0 0x7fc26414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26a366640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 0x7fc26414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 -- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 msgr2=0x7fc26414bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 0x7fc26414bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 -- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26414c3e0 msgr2=0x7fc26414b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26414c3e0 0x7fc26414b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 -- 172.21.15.17:0/2697087537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2640ad570 con 0x7fc24c004800 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26a366640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 0x7fc26414bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc269b65640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fc25c011aa0 tx=0x7fc25c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00fbe0 con 0x7fc24c004800 2024-03-06T07:29:03.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c00fd80 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c014640 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2640ad850 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.547+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2640ade10 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.548+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26404f330 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.549+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc25c012040 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.549+0000 7fc2627fc640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 0x7fc25006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.549+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc25c08a6b0 con 0x7fc24c004800 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.549+0000 7fc269364640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 0x7fc25006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:03.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.550+0000 7fc269364640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 0x7fc25006d9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc258034000 tx=0x7fc258007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:03.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.554+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc25c019030 con 0x7fc24c004800 2024-03-06T07:29:03.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.772+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc26406cda0 con 0x7fc24c004800 2024-03-06T07:29:03.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.772+0000 7fc2627fc640 1 -- 172.21.15.17:0/2697087537 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+3709 (secure 0 0 0) 0x7fc25c059780 con 0x7fc24c004800 2024-03-06T07:29:03.775 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:03.775 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":195,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7970,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/4049652050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":4049652050},{"type":"v1","addr":"172.21.15.163:6837","nonce":4049652050}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/212856078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":212856078},{"type":"v1","addr":"172.21.15.163:6836","nonce":212856078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10440,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3063315355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3063315355},{"type":"v1","addr":"172.21.15.17:6833","nonce":3063315355}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":185},{"gid":10617,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2194011003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2194011003},{"type":"v1","addr":"172.21.15.17:6835","nonce":2194011003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":193}],"filesystems":[]} 2024-03-06T07:29:03.775 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 195 2024-03-06T07:29:03.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 msgr2=0x7fc25006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 0x7fc25006d9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc258034000 tx=0x7fc258007000 comp rx=0 tx=0).stop 2024-03-06T07:29:03.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 msgr2=0x7fc2640ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:03.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fc25c011aa0 tx=0x7fc25c011f60 comp rx=0 tx=0).stop 2024-03-06T07:29:03.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 shutdown_connections 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc25006b500 0x7fc25006d9c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2640aeb50 0x7fc26414bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc26414c3e0 0x7fc26414b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 --2- 172.21.15.17:0/2697087537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc24c004800 0x7fc2640ae610 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 >> 172.21.15.17:0/2697087537 conn(0x7fc2640ac570 msgr2=0x7fc26406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 shutdown_connections 2024-03-06T07:29:03.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:03.774+0000 7fc26bdc9640 1 -- 172.21.15.17:0/2697087537 wait complete. 2024-03-06T07:29:03.788 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-06T07:29:03.788 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-06T07:29:03.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:29:03.788 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:03.788 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-06T07:29:03.788 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-06T07:29:03.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:29:03.789 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:03.789 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-06T07:29:03.789 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-06T07:29:03.789 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:29:03.789 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:03.789 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-06T07:29:03.789 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-06T07:29:03.790 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:29:03.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:29:03.797 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:29:03.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-06T07:29:03.798 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:29:03.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-06T07:29:03.801 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:29:03.802 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-06T07:29:03.839 INFO:tasks.ceph.mds.a:Started 2024-03-06T07:29:03.839 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:03.840 INFO:tasks.ceph.mds.c:Started 2024-03-06T07:29:03.840 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:03.844 INFO:tasks.ceph.mds.b:Started 2024-03-06T07:29:03.844 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:03.914 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:29:03.914 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-06T07:29:03.925 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2024-03-06T07:29:07.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-06T07:29:09.802 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:29:09.802 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-06T07:29:09.805 INFO:tasks.ceph.mds.d:Started 2024-03-06T07:29:09.805 DEBUG:teuthology.parallel:result is None 2024-03-06T07:29:09.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-06T07:29:09.888 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2024-03-06T07:29:09.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 Processor -- start 2024-03-06T07:29:09.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 -- start start 2024-03-06T07:29:09.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64200c0fa0 0x7f6420153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 0x7f642006de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6420070550 con 0x7f6420072f00 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642006e350 con 0x7f64200c0fa0 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.918+0000 7f6424913640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6420070830 con 0x7f642006d990 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641ed9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 0x7f642006de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641e59c640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6420072f00 0x7f64200c6840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:46434/0 (socket says 172.21.15.17:46434) 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641e59c640 1 -- 172.21.15.17:0/3384982016 learned_addr learned my addr 172.21.15.17:0/3384982016 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f6417fff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64200c0fa0 0x7f6420153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641ed9d640 1 -- 172.21.15.17:0/3384982016 >> v2:172.21.15.163:3300/0 conn(0x7f64200c0fa0 msgr2=0x7f6420153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641ed9d640 1 --2- 172.21.15.17:0/3384982016 >> v2:172.21.15.163:3300/0 conn(0x7f64200c0fa0 0x7f6420153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641ed9d640 1 -- 172.21.15.17:0/3384982016 >> v1:172.21.15.163:6789/0 conn(0x7f6420072f00 legacy=0x7f64200c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.919+0000 7f641ed9d640 1 -- 172.21.15.17:0/3384982016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420070b10 con 0x7f642006d990 2024-03-06T07:29:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f641ed9d640 1 --2- 172.21.15.17:0/3384982016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 0x7f642006de10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6418009d90 tx=0x7f641802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5839cb4513e710e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f641dd9b640 1 -- 172.21.15.17:0/3384982016 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418039070 con 0x7f642006d990 2024-03-06T07:29:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f641dd9b640 1 -- 172.21.15.17:0/3384982016 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641802cc50 con 0x7f642006d990 2024-03-06T07:29:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f641dd9b640 1 -- 172.21.15.17:0/3384982016 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418034440 con 0x7f642006d990 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f6424913640 1 -- 172.21.15.17:0/3384982016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 msgr2=0x7f642006de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.920+0000 7f6424913640 1 --2- 172.21.15.17:0/3384982016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 0x7f642006de10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6418009d90 tx=0x7f641802bea0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 -- 172.21.15.17:0/3384982016 shutdown_connections 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 --2- 172.21.15.17:0/3384982016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f642006d990 0x7f642006de10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 --2- 172.21.15.17:0/3384982016 >> v2:172.21.15.163:3300/0 conn(0x7f64200c0fa0 0x7f6420153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 -- 172.21.15.17:0/3384982016 >> 172.21.15.17:0/3384982016 conn(0x7f642006b1a0 msgr2=0x7f642006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 -- 172.21.15.17:0/3384982016 shutdown_connections 2024-03-06T07:29:09.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.921+0000 7f6424913640 1 -- 172.21.15.17:0/3384982016 wait complete. 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.922+0000 7f6424913640 1 Processor -- start 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.922+0000 7f6424913640 1 -- start start 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.922+0000 7f6424913640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.922+0000 7f6424913640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6420072f00 0x7f6420156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.922+0000 7f6424913640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64200c0fa0 0x7f642014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6424913640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200c70d0 con 0x7f642006d990 2024-03-06T07:29:09.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6424913640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200c6f50 con 0x7f64200c0fa0 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6424913640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6420070df0 con 0x7f6420072f00 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38812/0 (socket says 172.21.15.17:38812) 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 -- 172.21.15.17:0/3207815683 learned_addr learned my addr 172.21.15.17:0/3207815683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6417fff640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6420072f00 0x7f6420156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641ed9d640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64200c0fa0 0x7f642014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 -- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6420072f00 msgr2=0x7f6420156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6420072f00 0x7f6420156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 -- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64200c0fa0 msgr2=0x7f642014f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64200c0fa0 0x7f642014f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 -- 172.21.15.17:0/3207815683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64201dfe00 con 0x7f642006d990 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f641e59c640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f641000c400 tx=0x7f641000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410017070 con 0x7f642006d990 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6410005ce0 con 0x7f642006d990 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64201dfb20 con 0x7f642006d990 2024-03-06T07:29:09.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6410007310 con 0x7f642006d990 2024-03-06T07:29:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.923+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64201dd020 con 0x7f642006d990 2024-03-06T07:29:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.925+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642004f330 con 0x7f642006d990 2024-03-06T07:29:09.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.925+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f641001a070 con 0x7f642006d990 2024-03-06T07:29:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.925+0000 7f6416ffd640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 0x7f640006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.926+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6410089320 con 0x7f642006d990 2024-03-06T07:29:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.926+0000 7f6417fff640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 0x7f640006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:09.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.926+0000 7f6417fff640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 0x7f640006d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f64201e6b50 tx=0x7f6408009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:09.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:09.930+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6410057da0 con 0x7f642006d990 2024-03-06T07:29:10.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.128+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f64200c6270 con 0x7f642006d990 2024-03-06T07:29:10.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:10.293 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:10.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.992+0000 7f6416ffd640 1 -- 172.21.15.17:0/3207815683 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v208) v1 ==== 97+0+0 (secure 0 0 0) 0x7f641005be40 con 0x7f642006d990 2024-03-06T07:29:10.995 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-06T07:29:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 msgr2=0x7f640006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 0x7f640006d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f64201e6b50 tx=0x7f6408009190 comp rx=0 tx=0).stop 2024-03-06T07:29:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 msgr2=0x7f64201ec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:10.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f641000c400 tx=0x7f641000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 shutdown_connections 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f640006b4d0 0x7f640006d990 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64200c0fa0 0x7f642014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6420072f00 0x7f6420156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.995+0000 7f6424913640 1 --2- 172.21.15.17:0/3207815683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f642006d990 0x7f64201ec280 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.996+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 >> 172.21.15.17:0/3207815683 conn(0x7f642006b1a0 msgr2=0x7f64200c5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.996+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 shutdown_connections 2024-03-06T07:29:10.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:10.996+0000 7f6424913640 1 -- 172.21.15.17:0/3207815683 wait complete. 2024-03-06T07:29:11.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-06T07:29:11.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.116+0000 7efd58df7640 1 Processor -- start 2024-03-06T07:29:11.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.116+0000 7efd58df7640 1 -- start start 2024-03-06T07:29:11.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd58df7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd58df7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd54156180 con 0x7efd540c5330 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd58df7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd54159630 con 0x7efd540c8a00 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd58df7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd541587b0 con 0x7efd54152920 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd52d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efd540c5330 0x7efd5415fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60544/0 (socket says 172.21.15.17:60544) 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd52d9d640 1 -- 172.21.15.17:0/845106443 learned_addr learned my addr 172.21.15.17:0/845106443 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd5259c640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211196331 0 0) 0x7efd4c001110 con 0x7efd540c5330 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.117+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd40003520 con 0x7efd540c5330 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181604872 0 0) 0x7efd40003520 con 0x7efd540c5330 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 msgr2=0x7efd54155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd54155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 >> v1:172.21.15.163:6789/0 conn(0x7efd540c8a00 legacy=0x7efd54158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540c7760 con 0x7efd540c5330 2024-03-06T07:29:11.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7efd4c001840 con 0x7efd540c5330 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 >> v1:172.21.15.17:6789/0 conn(0x7efd540c5330 legacy=0x7efd5415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd40003f20 0x7efd40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd40004d40 0x7efd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd541587b0 con 0x7efd40003f20 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000bad0 con 0x7efd40004d40 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4000bcd0 con 0x7efd54152920 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd40003f20 0x7efd40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd5259c640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd52d9d640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd40004d40 0x7efd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 -- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 msgr2=0x7efd400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd54152920 0x7efd400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 -- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd40004d40 msgr2=0x7efd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd40004d40 0x7efd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.118+0000 7efd51d9b640 1 -- 172.21.15.17:0/845106443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40003520 con 0x7efd40003f20 2024-03-06T07:29:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.119+0000 7efd51d9b640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd40003f20 0x7efd40004800 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001910 tx=0x7efd40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deaf981239b2d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.119+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0014e0 con 0x7efd40003f20 2024-03-06T07:29:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.119+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c037070 con 0x7efd40003f20 2024-03-06T07:29:11.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.119+0000 7efd5159a640 1 -- 172.21.15.17:0/845106443 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c032880 con 0x7efd40003f20 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 -- 172.21.15.17:0/845106443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd40003f20 msgr2=0x7efd40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 --2- 172.21.15.17:0/845106443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd40003f20 0x7efd40004800 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001910 tx=0x7efd40005830 comp rx=0 tx=0).stop 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd52d9d640 1 -- 172.21.15.17:0/845106443 reap_dead start 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 -- 172.21.15.17:0/845106443 shutdown_connections 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 -- 172.21.15.17:0/845106443 >> 172.21.15.17:0/845106443 conn(0x7efd540ac560 msgr2=0x7efd540ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 -- 172.21.15.17:0/845106443 shutdown_connections 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.120+0000 7efd58df7640 1 -- 172.21.15.17:0/845106443 wait complete. 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.121+0000 7efd58df7640 1 Processor -- start 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.121+0000 7efd58df7640 1 -- start start 2024-03-06T07:29:11.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 0x7efd541eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 0x7efd541ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd54158a00 con 0x7efd5414e260 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540c7d20 con 0x7efd540c1a80 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd58df7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5406d030 con 0x7efd40003f20 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47194/0 (socket says 172.21.15.17:47194) 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 -- 172.21.15.17:0/3523812015 learned_addr learned my addr 172.21.15.17:0/3523812015 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd51d9b640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 0x7efd541ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd5259c640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 0x7efd541eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 -- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 msgr2=0x7efd541eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 0x7efd541eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 -- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 msgr2=0x7efd541ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 0x7efd541ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 -- 172.21.15.17:0/3523812015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54155900 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd51d9b640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 0x7efd541ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd5259c640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 0x7efd541eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.122+0000 7efd52d9d640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7efd540b0b20 tx=0x7efd4c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.123+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c01a070 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.123+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c00f620 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.123+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd54155be0 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.123+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00fa10 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.123+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd541ee430 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.124+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5404f330 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.124+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd4c01d070 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.124+0000 7efd4affd640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.125+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efd4c089520 con 0x7efd5414e260 2024-03-06T07:29:11.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.125+0000 7efd5259c640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.125+0000 7efd5259c640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efd440074d0 tx=0x7efd4400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.129+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efd54156180 con 0x7efd5414e260 2024-03-06T07:29:11.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.376+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efd54158590 con 0x7efd5414e260 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.377+0000 7efd4affd640 1 -- 172.21.15.17:0/3523812015 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4724 (secure 0 0 0) 0x7efd4c0585f0 con 0x7efd5414e260 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlliuIJBAAGTIUHlsjxZQdPvRCU2OK2A==", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-06T07:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlw/faMBAA0OCtIAojXIMnJK6UlRVL3w==", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.c", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl73XYKxAA1EKWfHoCv1MosL82p+xzdQ==", 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.d", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlT38tOBAAro6upUp2uqWQs8GEg9T0cA==", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDwGOhlhTUCFxAAqg8c4rAtC/BtHHOwEB8S4g==", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDzGOhlbg21FRAAjnuEL3DS+hp+MWivmSj7tA==", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD2GOhlPvi1ExAAq1XzB46+O91vaYt+X9dfgw==", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-06T07:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5GOhl9sS9EhAA28jr9on4arR+WzqKsUGwFA==", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD/GOhlCh9UGhAA9yg4H/FwSPUJUq2Z4+d1IQ==", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQACGehlsC7TGBAAt/HnTJLVfpJXjhPNEpr8sQ==", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAFGehl/S/WFhAAnrebIhvXef+9cbPAAO9T6Q==", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAIGehl/mSAFBAAff+P0GugBWcbIj6DKT+Iww==", 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA==", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow r", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.1", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDvGOhlDDREBBAAtzAnsUH/ACcDpsQjY4BVpw==", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-06T07:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhlwJ0uCBAA27dqniE6BkbTrfzmnQaZaQ==", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhldacqIRAArfE7rtNcYRdx6sKx0AK+Og==", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQDuGOhl5NtaHRAA8M5pGdwTjMeFVZkpDyeXbQ==", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-06T07:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-06T07:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-06T07:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:29:11.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 msgr2=0x7efd3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7efd440074d0 tx=0x7efd4400b040 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 msgr2=0x7efd540c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7efd540b0b20 tx=0x7efd4c007430 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 shutdown_connections 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efd3406b500 0x7efd3406d9c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd5414e260 0x7efd540c2530 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efd540c1a80 0x7efd541ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 --2- 172.21.15.17:0/3523812015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efd40003f20 0x7efd541eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 >> 172.21.15.17:0/3523812015 conn(0x7efd540ac560 msgr2=0x7efd540cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 shutdown_connections 2024-03-06T07:29:11.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.378+0000 7efd58df7640 1 -- 172.21.15.17:0/3523812015 wait complete. 2024-03-06T07:29:11.394 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.492+0000 7fe4e0356640 1 Processor -- start 2024-03-06T07:29:11.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.492+0000 7fe4e0356640 1 -- start start 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.493+0000 7fe4e0356640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.493+0000 7fe4e0356640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.493+0000 7fe4e0356640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d80c73d0 con 0x7fe4d80c88c0 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.493+0000 7fe4e0356640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d80c8dd0 con 0x7fe4d8158b60 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.493+0000 7fe4e0356640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80be2e0 con 0x7fe4d814f360 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de8f3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de8f3640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:60962/0 (socket says 172.21.15.17:60962) 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43850/0 (socket says 172.21.15.17:43850) 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de8f3640 1 -- 172.21.15.17:0/2153778326 learned_addr learned my addr 172.21.15.17:0/2153778326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 -- 172.21.15.17:0/2153778326 >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 msgr2=0x7fe4d8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 --2- 172.21.15.17:0/2153778326 >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 -- 172.21.15.17:0/2153778326 >> v1:172.21.15.163:6789/0 conn(0x7fe4d80c88c0 legacy=0x7fe4d80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de0f2640 1 -- 172.21.15.17:0/2153778326 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80c7670 con 0x7fe4d814f360 2024-03-06T07:29:11.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.494+0000 7fe4de8f3640 1 --2- 172.21.15.17:0/2153778326 >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.495+0000 7fe4de0f2640 1 --2- 172.21.15.17:0/2153778326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bbdaf01956507e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.495+0000 7fe4dd0f0640 1 -- 172.21.15.17:0/2153778326 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40315a0 con 0x7fe4d814f360 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.495+0000 7fe4dd0f0640 1 -- 172.21.15.17:0/2153778326 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4037070 con 0x7fe4d814f360 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.495+0000 7fe4dd0f0640 1 -- 172.21.15.17:0/2153778326 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4031960 con 0x7fe4d814f360 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 -- 172.21.15.17:0/2153778326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 msgr2=0x7fe4d8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 --2- 172.21.15.17:0/2153778326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40097a0 tx=0x7fe4d402b840 comp rx=0 tx=0).stop 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 -- 172.21.15.17:0/2153778326 shutdown_connections 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 --2- 172.21.15.17:0/2153778326 >> v2:172.21.15.163:3300/0 conn(0x7fe4d8158b60 0x7fe4d8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 --2- 172.21.15.17:0/2153778326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d814f360 0x7fe4d8155c10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 -- 172.21.15.17:0/2153778326 >> 172.21.15.17:0/2153778326 conn(0x7fe4d80ac570 msgr2=0x7fe4d80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 -- 172.21.15.17:0/2153778326 shutdown_connections 2024-03-06T07:29:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.496+0000 7fe4e0356640 1 -- 172.21.15.17:0/2153778326 wait complete. 2024-03-06T07:29:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 Processor -- start 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 -- start start 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d80c88c0 0x7fe4d81ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d8158b60 0x7fe4d814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80c5a50 con 0x7fe4d814f360 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80c7c30 con 0x7fe4d80c88c0 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4e0356640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80c5bd0 con 0x7fe4d8158b60 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.497+0000 7fe4dd8f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47206/0 (socket says 172.21.15.17:47206) 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 -- 172.21.15.17:0/1610006795 learned_addr learned my addr 172.21.15.17:0/1610006795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4de8f3640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d8158b60 0x7fe4d814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4de0f2640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d80c88c0 0x7fe4d81ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 -- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d8158b60 msgr2=0x7fe4d814b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d8158b60 0x7fe4d814b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 -- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d80c88c0 msgr2=0x7fe4d81ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d80c88c0 0x7fe4d81ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 -- 172.21.15.17:0/1610006795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d8151da0 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4dd8f1640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c470 tx=0x7fe4c800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80052d0 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8005490 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8007260 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d8152080 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.498+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d80ad860 con 0x7fe4d814f360 2024-03-06T07:29:11.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.499+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d80ada60 con 0x7fe4d814f360 2024-03-06T07:29:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.500+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4c8007400 con 0x7fe4d814f360 2024-03-06T07:29:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.500+0000 7fe4ceffd640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 0x7fe4bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.500+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe4c8089490 con 0x7fe4d814f360 2024-03-06T07:29:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.500+0000 7fe4de0f2640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 0x7fe4bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.501+0000 7fe4de0f2640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 0x7fe4bc06d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4036000 tx=0x7fe4d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.504+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4d80ada60 con 0x7fe4d814f360 2024-03-06T07:29:11.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.686+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d804f330 con 0x7fe4d814f360 2024-03-06T07:29:11.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.686+0000 7fe4ceffd640 1 -- 172.21.15.17:0/1610006795 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+13174 (secure 0 0 0) 0x7fe4c8057e00 con 0x7fe4d814f360 2024-03-06T07:29:11.689 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:11.689 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":208,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:29:10.993208+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":197,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 msgr2=0x7fe4bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 0x7fe4bc06d990 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4036000 tx=0x7fe4d403d040 comp rx=0 tx=0).stop 2024-03-06T07:29:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 msgr2=0x7fe4d80ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c470 tx=0x7fe4c800c930 comp rx=0 tx=0).stop 2024-03-06T07:29:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 shutdown_connections 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe4bc06b4d0 0x7fe4bc06d990 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d8158b60 0x7fe4d814b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4d814f360 0x7fe4d80ab2b0 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 --2- 172.21.15.17:0/1610006795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d80c88c0 0x7fe4d81ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 >> 172.21.15.17:0/1610006795 conn(0x7fe4d80ac570 msgr2=0x7fe4d814de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 shutdown_connections 2024-03-06T07:29:11.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.688+0000 7fe4e0356640 1 -- 172.21.15.17:0/1610006795 wait complete. 2024-03-06T07:29:11.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:29:11.702 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:29:11.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.803+0000 7f2b2e0af640 1 Processor -- start 2024-03-06T07:29:11.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.803+0000 7f2b2e0af640 1 -- start start 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b2e0af640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b281529f0 0x7f2b280c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b2e0af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 0x7f2b28155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b2e0af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b28152dd0 con 0x7f2b280c5370 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b2e0af640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b28159710 con 0x7f2b281529f0 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b2e0af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b28156180 con 0x7f2b280c57d0 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b281529f0 0x7f2b280c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2b281529f0 0x7f2b280c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47222/0 (socket says 172.21.15.17:47222) 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 -- 172.21.15.17:0/559531465 learned_addr learned my addr 172.21.15.17:0/559531465 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:11.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b27fff640 1 --2- 172.21.15.17:0/559531465 >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 0x7f2b28155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 -- 172.21.15.17:0/559531465 >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 msgr2=0x7f2b28155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 --2- 172.21.15.17:0/559531465 >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 0x7f2b28155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 -- 172.21.15.17:0/559531465 >> v1:172.21.15.163:6789/0 conn(0x7f2b280c5370 legacy=0x7f2b28155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b277fe640 1 -- 172.21.15.17:0/559531465 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28158b00 con 0x7f2b281529f0 2024-03-06T07:29:11.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.804+0000 7f2b27fff640 1 --2- 172.21.15.17:0/559531465 >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 0x7f2b28155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.805+0000 7f2b277fe640 1 --2- 172.21.15.17:0/559531465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b281529f0 0x7f2b280c88f0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2b280735c0 tx=0x7f2b1802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64615a60d58fc24 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.805+0000 7f2b267fc640 1 -- 172.21.15.17:0/559531465 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1802e070 con 0x7f2b281529f0 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.805+0000 7f2b267fc640 1 -- 172.21.15.17:0/559531465 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18037070 con 0x7f2b281529f0 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.805+0000 7f2b267fc640 1 -- 172.21.15.17:0/559531465 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18031a00 con 0x7f2b281529f0 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 -- 172.21.15.17:0/559531465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b281529f0 msgr2=0x7f2b280c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/559531465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b281529f0 0x7f2b280c88f0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f2b280735c0 tx=0x7f2b1802b880 comp rx=0 tx=0).stop 2024-03-06T07:29:11.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 -- 172.21.15.17:0/559531465 shutdown_connections 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/559531465 >> v2:172.21.15.163:3300/0 conn(0x7f2b280c57d0 0x7f2b28155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/559531465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b281529f0 0x7f2b280c88f0 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 -- 172.21.15.17:0/559531465 >> 172.21.15.17:0/559531465 conn(0x7f2b280ac580 msgr2=0x7f2b280ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 -- 172.21.15.17:0/559531465 shutdown_connections 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.806+0000 7f2b2e0af640 1 -- 172.21.15.17:0/559531465 wait complete. 2024-03-06T07:29:11.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.807+0000 7f2b2e0af640 1 Processor -- start 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.807+0000 7f2b2e0af640 1 -- start start 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280c5370 0x7f2b280ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b281529f0 0x7f2b28149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b281590c0 con 0x7f2b280c57d0 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280c68b0 con 0x7f2b280c5370 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b2e0af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b281593a0 con 0x7f2b281529f0 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47236/0 (socket says 172.21.15.17:47236) 2024-03-06T07:29:11.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 -- 172.21.15.17:0/764401161 learned_addr learned my addr 172.21.15.17:0/764401161 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b27fff640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b281529f0 0x7f2b28149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b277fe640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280c5370 0x7f2b280ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 -- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b281529f0 msgr2=0x7f2b28149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b281529f0 0x7f2b28149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 -- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280c5370 msgr2=0x7f2b280ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280c5370 0x7f2b280ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 -- 172.21.15.17:0/764401161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28149880 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b26ffd640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00c430 tx=0x7f2b1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.808+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011790 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.809+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c017070 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.809+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b281ee9d0 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.809+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011bb0 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.809+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2814a4e0 con 0x7f2b280c57d0 2024-03-06T07:29:11.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.809+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28069460 con 0x7f2b280c57d0 2024-03-06T07:29:11.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.810+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b1c01c460 con 0x7f2b280c57d0 2024-03-06T07:29:11.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.810+0000 7f2b0ffff640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:11.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.810+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2b1c089dd0 con 0x7f2b280c57d0 2024-03-06T07:29:11.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.810+0000 7f2b277fe640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:11.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.811+0000 7f2b277fe640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2b18036000 tx=0x7f2b1803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:11.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.815+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2b28069460 con 0x7f2b280c57d0 2024-03-06T07:29:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.995+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b2804f330 con 0x7f2b280c57d0 2024-03-06T07:29:11.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.995+0000 7f2b0ffff640 1 -- 172.21.15.17:0/764401161 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v208) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b1c08b070 con 0x7f2b280c57d0 2024-03-06T07:29:11.998 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 msgr2=0x7f2b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 0x7f2b1406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2b18036000 tx=0x7f2b1803d040 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 msgr2=0x7f2b28148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00c430 tx=0x7f2b1c00c8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 shutdown_connections 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.997+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2b1406b4d0 0x7f2b1406d990 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b281529f0 0x7f2b28149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b280c57d0 0x7f2b28148ae0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 --2- 172.21.15.17:0/764401161 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b280c5370 0x7f2b280ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 >> 172.21.15.17:0/764401161 conn(0x7f2b280ac580 msgr2=0x7f2b28151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 shutdown_connections 2024-03-06T07:29:12.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:11.998+0000 7f2b2e0af640 1 -- 172.21.15.17:0/764401161 wait complete. 2024-03-06T07:29:12.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 Processor -- start 2024-03-06T07:29:12.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 -- start start 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe406d9f0 0x7f2fe40a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fe40c3280 0x7f2fe40c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fe41551e0 0x7f2fe4155610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4158800 con 0x7f2fe41551e0 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe4158ae0 con 0x7f2fe40c3280 2024-03-06T07:29:12.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.110+0000 7f2fe92fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4158dc0 con 0x7f2fe406d9f0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fe41551e0 0x7f2fe4155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2fe41551e0 0x7f2fe4155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47244/0 (socket says 172.21.15.17:47244) 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 -- 172.21.15.17:0/4291542336 learned_addr learned my addr 172.21.15.17:0/4291542336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fdbfff640 1 --2- 172.21.15.17:0/4291542336 >> v2:172.21.15.163:3300/0 conn(0x7f2fe40c3280 0x7f2fe40c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe2ffd640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe406d9f0 0x7f2fe40a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 -- 172.21.15.17:0/4291542336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe406d9f0 msgr2=0x7f2fe40a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe406d9f0 0x7f2fe40a7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 -- 172.21.15.17:0/4291542336 >> v2:172.21.15.163:3300/0 conn(0x7f2fe40c3280 msgr2=0x7f2fe40c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 --2- 172.21.15.17:0/4291542336 >> v2:172.21.15.163:3300/0 conn(0x7f2fe40c3280 0x7f2fe40c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.111+0000 7f2fe37fe640 1 -- 172.21.15.17:0/4291542336 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe41590a0 con 0x7f2fe41551e0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe37fe640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe41551e0 0x7f2fe4155610 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c98d0 tx=0x7f2fdc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab27e49ee478ba server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe27fc640 1 -- 172.21.15.17:0/4291542336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc02c9e0 con 0x7f2fe41551e0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe27fc640 1 -- 172.21.15.17:0/4291542336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc02cb80 con 0x7f2fe41551e0 2024-03-06T07:29:12.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe27fc640 1 -- 172.21.15.17:0/4291542336 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0343f0 con 0x7f2fe41551e0 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 -- 172.21.15.17:0/4291542336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe41551e0 msgr2=0x7f2fe4155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe41551e0 0x7f2fe4155610 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c98d0 tx=0x7f2fdc02be40 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 -- 172.21.15.17:0/4291542336 shutdown_connections 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe41551e0 0x7f2fe4155610 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/4291542336 >> v2:172.21.15.163:3300/0 conn(0x7f2fe40c3280 0x7f2fe40c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/4291542336 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe406d9f0 0x7f2fe40a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 -- 172.21.15.17:0/4291542336 >> 172.21.15.17:0/4291542336 conn(0x7f2fe40aa630 msgr2=0x7f2fe40aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 -- 172.21.15.17:0/4291542336 shutdown_connections 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.112+0000 7f2fe92fb640 1 -- 172.21.15.17:0/4291542336 wait complete. 2024-03-06T07:29:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.113+0000 7f2fe92fb640 1 Processor -- start 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.113+0000 7f2fe92fb640 1 -- start start 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 0x7f2fe406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe41551e0 0x7f2fe40a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe406df50 con 0x7f2fe40c3280 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe406ddd0 con 0x7f2fe406d9f0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe92fb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40a8210 con 0x7f2fe41551e0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47252/0 (socket says 172.21.15.17:47252) 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fdbfff640 1 -- 172.21.15.17:0/1281143074 learned_addr learned my addr 172.21.15.17:0/1281143074 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fdbfff640 1 -- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe41551e0 msgr2=0x7f2fe40a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe37fe640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe41551e0 0x7f2fe40a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.114+0000 7f2fe2ffd640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 0x7f2fe406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdbfff640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe41551e0 0x7f2fe40a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdbfff640 1 -- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 msgr2=0x7f2fe406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdbfff640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 0x7f2fe406e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdbfff640 1 -- 172.21.15.17:0/1281143074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40c8610 con 0x7f2fe40c3280 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fe2ffd640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 0x7f2fe406e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdbfff640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00d620 tx=0x7f2fcc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00b620 con 0x7f2fe40c3280 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc019070 con 0x7f2fe40c3280 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00b9e0 con 0x7f2fe40c3280 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe41edf80 con 0x7f2fe40c3280 2024-03-06T07:29:12.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.115+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe40ae420 con 0x7f2fe40c3280 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.116+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe404f330 con 0x7f2fe40c3280 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.116+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fcc00bb80 con 0x7f2fe40c3280 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.117+0000 7f2fdb7fe640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 0x7f2fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.117+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2fcc08a400 con 0x7f2fe40c3280 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.117+0000 7f2fe2ffd640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 0x7f2fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.117+0000 7f2fe2ffd640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 0x7f2fc406d990 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4006fd0 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.121+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2fe40c8610 con 0x7f2fe40c3280 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.303+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2fe4073240 con 0x7f2fe40c3280 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.303+0000 7f2fdb7fe640 1 -- 172.21.15.17:0/1281143074 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v208) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2fcc0593b0 con 0x7f2fe40c3280 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-06T07:29:12.306 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-06T07:29:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 msgr2=0x7f2fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 0x7f2fc406d990 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4006fd0 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2024-03-06T07:29:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 msgr2=0x7f2fe40ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00d620 tx=0x7f2fcc00dae0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 shutdown_connections 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2fc406b4d0 0x7f2fc406d990 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe41551e0 0x7f2fe40a98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe40c3280 0x7f2fe40ac3f0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 --2- 172.21.15.17:0/1281143074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe406d9f0 0x7f2fe406e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 >> 172.21.15.17:0/1281143074 conn(0x7f2fe40aa630 msgr2=0x7f2fe406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.305+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 shutdown_connections 2024-03-06T07:29:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.306+0000 7f2fe92fb640 1 -- 172.21.15.17:0/1281143074 wait complete. 2024-03-06T07:29:12.319 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-06T07:29:12.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-06T07:29:12.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.421+0000 7fb76bd77640 1 Processor -- start 2024-03-06T07:29:12.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 -- start start 2024-03-06T07:29:12.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7640c6780 0x7fb764072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb764156150 con 0x7fb764072a50 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7640a7d70 con 0x7fb7640c6780 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb76bd77640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7640a8050 con 0x7fb76406fe20 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb769312640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7640c6780 0x7fb764072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb769312640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7640c6780 0x7fb764072510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47256/0 (socket says 172.21.15.17:47256) 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.422+0000 7fb769312640 1 -- 172.21.15.17:0/3999167565 learned_addr learned my addr 172.21.15.17:0/3999167565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769b13640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769312640 1 -- 172.21.15.17:0/3999167565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 msgr2=0x7fb764155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769312640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769312640 1 -- 172.21.15.17:0/3999167565 >> v1:172.21.15.163:6789/0 conn(0x7fb764072a50 legacy=0x7fb764158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769312640 1 -- 172.21.15.17:0/3999167565 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7640a8330 con 0x7fb7640c6780 2024-03-06T07:29:12.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769b13640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.423+0000 7fb769312640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7640c6780 0x7fb764072510 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fb754009870 tx=0x7fb75402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179d21a73ba4cacb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb768b11640 1 -- 172.21.15.17:0/3999167565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7540317c0 con 0x7fb7640c6780 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb768b11640 1 -- 172.21.15.17:0/3999167565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754037070 con 0x7fb7640c6780 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb768b11640 1 -- 172.21.15.17:0/3999167565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754031b80 con 0x7fb7640c6780 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 -- 172.21.15.17:0/3999167565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7640c6780 msgr2=0x7fb764072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7640c6780 0x7fb764072510 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fb754009870 tx=0x7fb75402b860 comp rx=0 tx=0).stop 2024-03-06T07:29:12.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 -- 172.21.15.17:0/3999167565 shutdown_connections 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7640c6780 0x7fb764072510 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 --2- 172.21.15.17:0/3999167565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 -- 172.21.15.17:0/3999167565 >> 172.21.15.17:0/3999167565 conn(0x7fb7640aa630 msgr2=0x7fb7640aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 -- 172.21.15.17:0/3999167565 shutdown_connections 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.424+0000 7fb76bd77640 1 -- 172.21.15.17:0/3999167565 wait complete. 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.425+0000 7fb76bd77640 1 Processor -- start 2024-03-06T07:29:12.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.425+0000 7fb76bd77640 1 -- start start 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7640c6780 0x7fb76414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764070200 con 0x7fb764072a50 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7640c3600 con 0x7fb7640c6780 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76bd77640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb764158e90 con 0x7fb76406fe20 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47264/0 (socket says 172.21.15.17:47264) 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 -- 172.21.15.17:0/422915822 learned_addr learned my addr 172.21.15.17:0/422915822 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb76a314640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7640c6780 0x7fb76414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769b13640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 -- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 msgr2=0x7fb764151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 -- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7640c6780 msgr2=0x7fb76414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7640c6780 0x7fb76414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.426+0000 7fb769312640 1 -- 172.21.15.17:0/422915822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7641eadc0 con 0x7fb764072a50 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb769312640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fb754009870 tx=0x7fb7540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754032860 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754032a00 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7641eb0a0 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754032df0 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.427+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7641eb660 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.428+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76406c550 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.428+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb75404c060 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.429+0000 7fb75a7fc640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 0x7fb74806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.429+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb7540abd30 con 0x7fb764072a50 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.429+0000 7fb769b13640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 0x7fb74806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.429+0000 7fb769b13640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 0x7fb74806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb76000a990 tx=0x7fb760008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:12.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.433+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb76406c550 con 0x7fb764072a50 2024-03-06T07:29:12.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:12.613+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb76404f330 con 0x7fb764072a50 2024-03-06T07:29:13.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.002+0000 7fb75a7fc640 1 -- 172.21.15.17:0/422915822 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v209) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb754040070 con 0x7fb764072a50 2024-03-06T07:29:13.005 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 msgr2=0x7fb74806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 0x7fb74806d990 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb76000a990 tx=0x7fb760008040 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 msgr2=0x7fb7640abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fb754009870 tx=0x7fb7540046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 shutdown_connections 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb74806b4d0 0x7fb74806d990 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7640c6780 0x7fb76414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb764072a50 0x7fb7640abfc0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 --2- 172.21.15.17:0/422915822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76406fe20 0x7fb764151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 >> 172.21.15.17:0/422915822 conn(0x7fb7640aa630 msgr2=0x7fb76406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 shutdown_connections 2024-03-06T07:29:13.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.005+0000 7fb76bd77640 1 -- 172.21.15.17:0/422915822 wait complete. 2024-03-06T07:29:13.023 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-06T07:29:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.130+0000 7f466384e640 1 Processor -- start 2024-03-06T07:29:13.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.130+0000 7f466384e640 1 -- start start 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f466384e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f465c1529f0 0x7f465c0c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f466384e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f465c0c5370 0x7f465c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f466384e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c152dd0 con 0x7f465c155530 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f466384e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c159710 con 0x7f465c1529f0 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f466384e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c0c68b0 con 0x7f465c0c5370 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f46615ea640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f465c1529f0 0x7f465c0c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f4661deb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f465c155530 0x7f465c1559c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60548/0 (socket says 172.21.15.17:60548) 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f4661deb640 1 -- 172.21.15.17:0/1185666130 learned_addr learned my addr 172.21.15.17:0/1185666130 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f46615ea640 1 -- 172.21.15.17:0/1185666130 >> v2:172.21.15.163:3300/0 conn(0x7f465c0c5370 msgr2=0x7f465c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f46615ea640 1 --2- 172.21.15.17:0/1185666130 >> v2:172.21.15.163:3300/0 conn(0x7f465c0c5370 0x7f465c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f46615ea640 1 -- 172.21.15.17:0/1185666130 >> v1:172.21.15.17:6789/0 conn(0x7f465c155530 legacy=0x7f465c1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.131+0000 7f46615ea640 1 -- 172.21.15.17:0/1185666130 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c158b00 con 0x7f465c1529f0 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.132+0000 7f46615ea640 1 --2- 172.21.15.17:0/1185666130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c1529f0 0x7f465c0c88f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f465c08c9b0 tx=0x7f465402b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78386e5520c768c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.132+0000 7f465bfff640 1 -- 172.21.15.17:0/1185666130 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654037070 con 0x7f465c1529f0 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.132+0000 7f465bfff640 1 -- 172.21.15.17:0/1185666130 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654030710 con 0x7f465c1529f0 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.132+0000 7f465bfff640 1 -- 172.21.15.17:0/1185666130 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654030ae0 con 0x7f465c1529f0 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 -- 172.21.15.17:0/1185666130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c1529f0 msgr2=0x7f465c0c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 --2- 172.21.15.17:0/1185666130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c1529f0 0x7f465c0c88f0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f465c08c9b0 tx=0x7f465402b680 comp rx=0 tx=0).stop 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 -- 172.21.15.17:0/1185666130 shutdown_connections 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 --2- 172.21.15.17:0/1185666130 >> v2:172.21.15.163:3300/0 conn(0x7f465c0c5370 0x7f465c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 --2- 172.21.15.17:0/1185666130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c1529f0 0x7f465c0c88f0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 -- 172.21.15.17:0/1185666130 >> 172.21.15.17:0/1185666130 conn(0x7f465c0ac580 msgr2=0x7f465c0ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:13.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 -- 172.21.15.17:0/1185666130 shutdown_connections 2024-03-06T07:29:13.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.133+0000 7f466384e640 1 -- 172.21.15.17:0/1185666130 wait complete. 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.134+0000 7f466384e640 1 Processor -- start 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.134+0000 7f466384e640 1 -- start start 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 0x7f465c1eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f465c1529f0 0x7f465c0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1593a0 con 0x7f465c155530 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1590c0 con 0x7f465c1529f0 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f466384e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c1560d0 con 0x7f465c0c5370 2024-03-06T07:29:13.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47282/0 (socket says 172.21.15.17:47282) 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 -- 172.21.15.17:0/4241229067 learned_addr learned my addr 172.21.15.17:0/4241229067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4660de9640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f465c1529f0 0x7f465c0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f46615ea640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 0x7f465c1eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 -- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 msgr2=0x7f465c1eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 0x7f465c1eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 -- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f465c1529f0 msgr2=0x7f465c0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f465c1529f0 0x7f465c0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 -- 172.21.15.17:0/4241229067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0ae580 con 0x7f465c155530 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f46615ea640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 0x7f465c1eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.135+0000 7f4661deb640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c910 tx=0x7f464c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.136+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c018070 con 0x7f465c155530 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.136+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c00a700 con 0x7f465c155530 2024-03-06T07:29:13.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.136+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c00ab30 con 0x7f465c155530 2024-03-06T07:29:13.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.136+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c1ee110 con 0x7f465c155530 2024-03-06T07:29:13.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.136+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c14cf60 con 0x7f465c155530 2024-03-06T07:29:13.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.137+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c075300 con 0x7f465c155530 2024-03-06T07:29:13.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.137+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f464c01b070 con 0x7f465c155530 2024-03-06T07:29:13.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.138+0000 7f4659ffb640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:13.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.138+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f464c0899c0 con 0x7f465c155530 2024-03-06T07:29:13.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.138+0000 7f46615ea640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:13.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.138+0000 7f46615ea640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f4654034000 tx=0x7f4654008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:13.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.142+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f465c075300 con 0x7f465c155530 2024-03-06T07:29:13.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:13.322+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f465c04f330 con 0x7f465c155530 2024-03-06T07:29:13.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-06T07:29:14.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.008+0000 7f4659ffb640 1 -- 172.21.15.17:0/4241229067 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v210) v1 ==== 170+0+0 (secure 0 0 0) 0x7f464c08be20 con 0x7f465c155530 2024-03-06T07:29:14.011 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.011+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 msgr2=0x7f464806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.011+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f4654034000 tx=0x7f4654008030 comp rx=0 tx=0).stop 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.011+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 msgr2=0x7f465c1540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.011+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f464c00c910 tx=0x7f464c00cdd0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 shutdown_connections 2024-03-06T07:29:14.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f464806b4d0 0x7f464806d990 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f465c155530 0x7f465c1540a0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f465c1529f0 0x7f465c0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 --2- 172.21.15.17:0/4241229067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f465c0c5370 0x7f465c1eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 >> 172.21.15.17:0/4241229067 conn(0x7f465c0ac580 msgr2=0x7f465c06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 shutdown_connections 2024-03-06T07:29:14.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.012+0000 7f466384e640 1 -- 172.21.15.17:0/4241229067 wait complete. 2024-03-06T07:29:14.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-06T07:29:14.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.146+0000 7f8473f13640 1 Processor -- start 2024-03-06T07:29:14.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.146+0000 7f8473f13640 1 -- start start 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8473f13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f846c0a8ff0 0x7f846c0ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8473f13640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 0x7f846c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8473f13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c0c1f90 con 0x7f846c0ab730 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8473f13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c14f850 con 0x7f846c0a8ff0 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8473f13640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c156260 con 0x7f846c0a6290 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f846c0a8ff0 0x7f846c0ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f846c0a8ff0 0x7f846c0ab1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47286/0 (socket says 172.21.15.17:47286) 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 -- 172.21.15.17:0/1575611344 learned_addr learned my addr 172.21.15.17:0/1575611344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f84714ae640 1 --2- 172.21.15.17:0/1575611344 >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 0x7f846c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 -- 172.21.15.17:0/1575611344 >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 msgr2=0x7f846c0a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 --2- 172.21.15.17:0/1575611344 >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 0x7f846c0a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 -- 172.21.15.17:0/1575611344 >> v1:172.21.15.163:6789/0 conn(0x7f846c0ab730 legacy=0x7f846c0aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f8471caf640 1 -- 172.21.15.17:0/1575611344 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0c11d0 con 0x7f846c0a8ff0 2024-03-06T07:29:14.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.147+0000 7f84714ae640 1 --2- 172.21.15.17:0/1575611344 >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 0x7f846c0a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:14.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.148+0000 7f8471caf640 1 --2- 172.21.15.17:0/1575611344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0ab1f0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f846c06e560 tx=0x7f846402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35c010eed26c719 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.148+0000 7f8470cad640 1 -- 172.21.15.17:0/1575611344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846402e070 con 0x7f846c0a8ff0 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.148+0000 7f8470cad640 1 -- 172.21.15.17:0/1575611344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464037070 con 0x7f846c0a8ff0 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.148+0000 7f8470cad640 1 -- 172.21.15.17:0/1575611344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464031a00 con 0x7f846c0a8ff0 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 -- 172.21.15.17:0/1575611344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 msgr2=0x7f846c0ab1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 --2- 172.21.15.17:0/1575611344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0ab1f0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f846c06e560 tx=0x7f846402b880 comp rx=0 tx=0).stop 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 -- 172.21.15.17:0/1575611344 shutdown_connections 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 --2- 172.21.15.17:0/1575611344 >> v2:172.21.15.163:3300/0 conn(0x7f846c0a6290 0x7f846c0a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 --2- 172.21.15.17:0/1575611344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0ab1f0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 -- 172.21.15.17:0/1575611344 >> 172.21.15.17:0/1575611344 conn(0x7f846c06afc0 msgr2=0x7f846c06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 -- 172.21.15.17:0/1575611344 shutdown_connections 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.149+0000 7f8473f13640 1 -- 172.21.15.17:0/1575611344 wait complete. 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.150+0000 7f8473f13640 1 Processor -- start 2024-03-06T07:29:14.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.150+0000 7f8473f13640 1 -- start start 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0ab730 0x7f846c143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 0x7f846c143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0c1a70 con 0x7f846c0a8ff0 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0c1790 con 0x7f846c0ab730 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8473f13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c155cf0 con 0x7f846c1477b0 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47296/0 (socket says 172.21.15.17:47296) 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 -- 172.21.15.17:0/1987734044 learned_addr learned my addr 172.21.15.17:0/1987734044 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f84714ae640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0ab730 0x7f846c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f84724b0640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 0x7f846c143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 -- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 msgr2=0x7f846c143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 0x7f846c143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 -- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0ab730 msgr2=0x7f846c143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:14.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0ab730 0x7f846c143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 -- 172.21.15.17:0/1987734044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c14c780 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f84724b0640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 0x7f846c143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.151+0000 7f8471caf640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f846402b8c0 tx=0x7f84640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464032880 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464032a20 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464032e10 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c1eb740 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c0c0280 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.152+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c04f330 con 0x7f846c0a8ff0 2024-03-06T07:29:14.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.153+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84640453f0 con 0x7f846c0a8ff0 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.153+0000 7f84627fc640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 0x7f845006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.154+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f84640ac190 con 0x7f846c0a8ff0 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.154+0000 7f84714ae640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 0x7f845006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:14.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.154+0000 7f84714ae640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 0x7f845006d9c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f846c0c76f0 tx=0x7f845c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:14.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.158+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8464040070 con 0x7f846c0a8ff0 2024-03-06T07:29:14.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:14.375+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f846c0c24e0 con 0x7f846c0a8ff0 2024-03-06T07:29:15.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.010+0000 7f84627fc640 1 -- 172.21.15.17:0/1987734044 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v198) v1 ==== 174+0+0 (secure 0 0 0) 0x7f846407a9b0 con 0x7f846c0a8ff0 2024-03-06T07:29:15.013 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 33 and data pool 34 2024-03-06T07:29:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.013+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 msgr2=0x7f845006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.013+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 0x7f845006d9c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f846c0c76f0 tx=0x7f845c008040 comp rx=0 tx=0).stop 2024-03-06T07:29:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.013+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 msgr2=0x7f846c0bfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.013+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f846402b8c0 tx=0x7f84640046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 shutdown_connections 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f845006b500 0x7f845006d9c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f846c1477b0 0x7f846c143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f846c0ab730 0x7f846c143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 --2- 172.21.15.17:0/1987734044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f846c0a8ff0 0x7f846c0bfd40 secure :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f846402b8c0 tx=0x7f84640046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.014+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 >> 172.21.15.17:0/1987734044 conn(0x7f846c06afc0 msgr2=0x7f846c153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.015+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 shutdown_connections 2024-03-06T07:29:15.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.015+0000 7f8473f13640 1 -- 172.21.15.17:0/1987734044 wait complete. 2024-03-06T07:29:15.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:15.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 Processor -- start 2024-03-06T07:29:15.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 -- start start 2024-03-06T07:29:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb86c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c0c73d0 con 0x7fb86c14f360 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb86c0be2e0 con 0x7fb86c158b60 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb872bb6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c159700 con 0x7fb86c0c88c0 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb870952640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb86c14f360 0x7fb86c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60560/0 (socket says 172.21.15.17:60560) 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.150+0000 7fb870952640 1 -- 172.21.15.17:0/1005347051 learned_addr learned my addr 172.21.15.17:0/1005347051 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb86c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427529514 0 0) 0x7fb86c0c73d0 con 0x7fb86c14f360 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb854003520 con 0x7fb86c14f360 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484893202 0 0) 0x7fb854003520 con 0x7fb86c14f360 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 msgr2=0x7fb86c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb86c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 >> v1:172.21.15.163:6789/0 conn(0x7fb86c158b60 legacy=0x7fb86c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c0c7670 con 0x7fb86c14f360 2024-03-06T07:29:15.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb8640019f0 con 0x7fb86c14f360 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 >> v1:172.21.15.17:6789/0 conn(0x7fb86c14f360 legacy=0x7fb86c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb854003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb854004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb854004d40 0x7fb8540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c159700 con 0x7fb8540039e0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85400baf0 con 0x7fb854004d40 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85400bcf0 con 0x7fb86c0c88c0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb854003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb854004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb871153640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb854004d40 0x7fb8540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 -- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 msgr2=0x7fb854004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb854004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 -- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb854004d40 msgr2=0x7fb8540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb854004d40 0x7fb8540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb870952640 1 -- 172.21.15.17:0/1005347051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb854003520 con 0x7fb8540039e0 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c0c88c0 0x7fb854004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.151+0000 7fb871153640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb854004d40 0x7fb8540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.152+0000 7fb870952640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb854003e00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fb86400a850 tx=0x7fb854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e322f7c58ce96ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.152+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86400a4a0 con 0x7fb8540039e0 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.152+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864034650 con 0x7fb8540039e0 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.152+0000 7fb86b7fe640 1 -- 172.21.15.17:0/1005347051 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864034a40 con 0x7fb8540039e0 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 -- 172.21.15.17:0/1005347051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 msgr2=0x7fb854003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1005347051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb854003e00 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7fb86400a850 tx=0x7fb854005830 comp rx=0 tx=0).stop 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb871153640 1 -- 172.21.15.17:0/1005347051 reap_dead start 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 -- 172.21.15.17:0/1005347051 shutdown_connections 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 -- 172.21.15.17:0/1005347051 >> 172.21.15.17:0/1005347051 conn(0x7fb86c0ac570 msgr2=0x7fb86c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 -- 172.21.15.17:0/1005347051 shutdown_connections 2024-03-06T07:29:15.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.153+0000 7fb872bb6640 1 -- 172.21.15.17:0/1005347051 wait complete. 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.154+0000 7fb872bb6640 1 Processor -- start 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.154+0000 7fb872bb6640 1 -- start start 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1523b0 0x7fb86c0c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 0x7fb86c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c7c30 con 0x7fb8540039e0 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c5a50 con 0x7fb86c0bedd0 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb872bb6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c0c5bd0 con 0x7fb86c1523b0 2024-03-06T07:29:15.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47324/0 (socket says 172.21.15.17:47324) 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 -- 172.21.15.17:0/1311371407 learned_addr learned my addr 172.21.15.17:0/1311371407 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1523b0 0x7fb86c0c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb871153640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 0x7fb86c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 -- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1523b0 msgr2=0x7fb86c0c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1523b0 0x7fb86c0c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 -- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 msgr2=0x7fb86c151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 0x7fb86c151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 -- 172.21.15.17:0/1311371407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c1520d0 con 0x7fb8540039e0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb871153640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 0x7fb86c151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb870952640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fb864001cf0 tx=0x7fb864001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.155+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8640019c0 con 0x7fb8540039e0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.156+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864004b60 con 0x7fb8540039e0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.156+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c1eaf50 con 0x7fb8540039e0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.156+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864034800 con 0x7fb8540039e0 2024-03-06T07:29:15.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.156+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c0c3540 con 0x7fb8540039e0 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.157+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c04f330 con 0x7fb8540039e0 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.157+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8640349a0 con 0x7fb8540039e0 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.157+0000 7fb8697fa640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.157+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8640acc70 con 0x7fb8540039e0 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.158+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.158+0000 7fb86bfff640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 0x7fb85806d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0c5440 tx=0x7fb860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.162+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb86407b950 con 0x7fb8540039e0 2024-03-06T07:29:15.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.343+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb86c0c8d50 con 0x7fb8540039e0 2024-03-06T07:29:15.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.344+0000 7fb8697fa640 1 -- 172.21.15.17:0/1311371407 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fb86407b250 con 0x7fb8540039e0 2024-03-06T07:29:15.347 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:15.347 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:29:14.377699+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 msgr2=0x7fb85806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 0x7fb85806d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0c5440 tx=0x7fb860008040 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 msgr2=0x7fb86c1eba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7fb864001cf0 tx=0x7fb864001dc0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 shutdown_connections 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb85806b4d0 0x7fb85806d990 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb86c0bedd0 0x7fb86c151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb86c1523b0 0x7fb86c0c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 --2- 172.21.15.17:0/1311371407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8540039e0 0x7fb86c1eba30 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 >> 172.21.15.17:0/1311371407 conn(0x7fb86c0ac570 msgr2=0x7fb86c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 shutdown_connections 2024-03-06T07:29:15.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.346+0000 7fb872bb6640 1 -- 172.21.15.17:0/1311371407 wait complete. 2024-03-06T07:29:15.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.458+0000 7f802be4e640 1 Processor -- start 2024-03-06T07:29:15.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.458+0000 7f802be4e640 1 -- start start 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802be4e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8024155ba0 0x7f80240c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802be4e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80241591c0 con 0x7f80241586f0 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802be4e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80241594a0 con 0x7f80240c5480 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802be4e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802406df40 con 0x7f8024155ba0 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f80293e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8024155ba0 0x7f80240c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802a3eb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f80241586f0 0x7f802415f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60568/0 (socket says 172.21.15.17:60568) 2024-03-06T07:29:15.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f802a3eb640 1 -- 172.21.15.17:0/2273322648 learned_addr learned my addr 172.21.15.17:0/2273322648 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.459+0000 7f80293e9640 1 -- 172.21.15.17:0/2273322648 >> v1:172.21.15.163:6789/0 conn(0x7f80240c5480 legacy=0x7f802406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.460+0000 7f80293e9640 1 -- 172.21.15.17:0/2273322648 >> v1:172.21.15.17:6789/0 conn(0x7f80241586f0 legacy=0x7f802415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.460+0000 7f8028be8640 1 -- 172.21.15.17:0/2273322648 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575848916 0 0) 0x7f80241591c0 con 0x7f80241586f0 2024-03-06T07:29:15.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.460+0000 7f80293e9640 1 -- 172.21.15.17:0/2273322648 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80241553a0 con 0x7f8024155ba0 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.460+0000 7f80293e9640 1 --2- 172.21.15.17:0/2273322648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80240c5860 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f80140097a0 tx=0x7f801402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e437cafe87c65e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.460+0000 7f8028be8640 1 -- 172.21.15.17:0/2273322648 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014035020 con 0x7f8024155ba0 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f8028be8640 1 -- 172.21.15.17:0/2273322648 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014031830 con 0x7f8024155ba0 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f8028be8640 1 -- 172.21.15.17:0/2273322648 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014031c00 con 0x7f8024155ba0 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 -- 172.21.15.17:0/2273322648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 msgr2=0x7f80240c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 --2- 172.21.15.17:0/2273322648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80240c5860 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f80140097a0 tx=0x7f801402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 -- 172.21.15.17:0/2273322648 shutdown_connections 2024-03-06T07:29:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 --2- 172.21.15.17:0/2273322648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80240c5860 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 -- 172.21.15.17:0/2273322648 >> 172.21.15.17:0/2273322648 conn(0x7f802406b1a0 msgr2=0x7f802406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 -- 172.21.15.17:0/2273322648 shutdown_connections 2024-03-06T07:29:15.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.461+0000 7f802be4e640 1 -- 172.21.15.17:0/2273322648 wait complete. 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.462+0000 7f802be4e640 1 Processor -- start 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.462+0000 7f802be4e640 1 -- start start 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240c5480 0x7f80241eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80241586f0 0x7f80241ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8024156000 con 0x7f8024155ba0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802406fdb0 con 0x7f80241586f0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802be4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8024152a20 con 0x7f80240c5480 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47348/0 (socket says 172.21.15.17:47348) 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 -- 172.21.15.17:0/824838733 learned_addr learned my addr 172.21.15.17:0/824838733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f802a3eb640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80241586f0 0x7f80241ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f8029bea640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240c5480 0x7f80241eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 -- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240c5480 msgr2=0x7f80241eab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240c5480 0x7f80241eab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 -- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80241586f0 msgr2=0x7f80241ee840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80241586f0 0x7f80241ee840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.463+0000 7f80293e9640 1 -- 172.21.15.17:0/824838733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802414c440 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f80293e9640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f8014000c00 tx=0x7f80140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014035050 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014004230 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8024158040 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801403b740 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.464+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80241ee320 con 0x7f8024155ba0 2024-03-06T07:29:15.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.465+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802404f330 con 0x7f8024155ba0 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.465+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8014038020 con 0x7f8024155ba0 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.466+0000 7f801a7fc640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 0x7f800806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.466+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80140ac170 con 0x7f8024155ba0 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.466+0000 7f8029bea640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 0x7f800806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.466+0000 7f8029bea640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 0x7f800806d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.470+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f802414c440 con 0x7f8024155ba0 2024-03-06T07:29:15.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.652+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802406e0c0 con 0x7f8024155ba0 2024-03-06T07:29:15.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.653+0000 7f801a7fc640 1 -- 172.21.15.17:0/824838733 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f8014040070 con 0x7f8024155ba0 2024-03-06T07:29:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:15.656 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:29:14.377699+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 msgr2=0x7f800806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 0x7f800806d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c009040 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 msgr2=0x7f80241542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f8014000c00 tx=0x7f80140046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 shutdown_connections 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f800806b4d0 0x7f800806d990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80241586f0 0x7f80241ee840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8024155ba0 0x7f80241542b0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 --2- 172.21.15.17:0/824838733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80240c5480 0x7f80241eab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 >> 172.21.15.17:0/824838733 conn(0x7f802406b1a0 msgr2=0x7f802406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 shutdown_connections 2024-03-06T07:29:15.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.655+0000 7f802be4e640 1 -- 172.21.15.17:0/824838733 wait complete. 2024-03-06T07:29:15.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-06T07:29:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.770+0000 7f1d705d7640 1 Processor -- start 2024-03-06T07:29:15.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.770+0000 7f1d705d7640 1 -- start start 2024-03-06T07:29:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d705d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d705d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680c3ef0 con 0x7f1d680c5990 2024-03-06T07:29:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d705d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d681556d0 con 0x7f1d680ab170 2024-03-06T07:29:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d705d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d681559b0 con 0x7f1d6806c900 2024-03-06T07:29:15.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6e373640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1d680c5990 0x7f1d6806c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60584/0 (socket says 172.21.15.17:60584) 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6e373640 1 -- 172.21.15.17:0/1577198811 learned_addr learned my addr 172.21.15.17:0/1577198811 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6db72640 1 --2- 172.21.15.17:0/1577198811 >> v2:172.21.15.163:3300/0 conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435356157 0 0) 0x7f1d680c3ef0 con 0x7f1d680c5990 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d54003520 con 0x7f1d680c5990 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641093672 0 0) 0x7f1d681556d0 con 0x7f1d680ab170 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d680c3ef0 con 0x7f1d680ab170 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066535244 0 0) 0x7f1d54003520 con 0x7f1d680c5990 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 >> v2:172.21.15.163:3300/0 conn(0x7f1d6806c900 msgr2=0x7f1d680c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 --2- 172.21.15.17:0/1577198811 >> v2:172.21.15.163:3300/0 conn(0x7f1d6806c900 0x7f1d680c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 >> v1:172.21.15.163:6789/0 conn(0x7f1d680ab170 legacy=0x7f1d680a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68155c90 con 0x7f1d680c5990 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1d64001670 con 0x7f1d680c5990 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 >> v1:172.21.15.17:6789/0 conn(0x7f1d680c5990 legacy=0x7f1d6806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54003b90 0x7f1d54003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 0x7f1d54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d540051c0 0x7f1d5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681559b0 con 0x7f1d54004800 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5400c090 con 0x7f1d540051c0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.771+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5400c290 con 0x7f1d54003b90 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 0x7f1d54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6e373640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54003b90 0x7f1d54003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6eb74640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d540051c0 0x7f1d5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 -- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54003b90 msgr2=0x7f1d54003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54003b90 0x7f1d54003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 -- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d540051c0 msgr2=0x7f1d5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d540051c0 0x7f1d5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6db72640 1 -- 172.21.15.17:0/1577198811 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54003520 con 0x7f1d54004800 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6eb74640 1 -- 172.21.15.17:0/1577198811 reap_dead start 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.772+0000 7f1d6e373640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54003b90 0x7f1d54003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d6db72640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 0x7f1d54004c80 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ac0 tx=0x7f1d54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38f395b11c3e0dc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58039070 con 0x7f1d54004800 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5802c6b0 con 0x7f1d54004800 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d6d371640 1 -- 172.21.15.17:0/1577198811 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58009690 con 0x7f1d54004800 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 -- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 msgr2=0x7f1d54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 0x7f1d54004c80 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009ac0 tx=0x7f1d54005830 comp rx=0 tx=0).stop 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 -- 172.21.15.17:0/1577198811 shutdown_connections 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 --2- 172.21.15.17:0/1577198811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d54004800 0x7f1d54004c80 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 -- 172.21.15.17:0/1577198811 >> 172.21.15.17:0/1577198811 conn(0x7f1d680c6710 msgr2=0x7f1d680c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 -- 172.21.15.17:0/1577198811 shutdown_connections 2024-03-06T07:29:15.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.773+0000 7f1d705d7640 1 -- 172.21.15.17:0/1577198811 wait complete. 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.774+0000 7f1d705d7640 1 Processor -- start 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.774+0000 7f1d705d7640 1 -- start start 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54004800 0x7f1d6814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 0x7f1d6814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68156610 con 0x7f1d68148c30 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68156490 con 0x7f1d681deed0 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d705d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680c3af0 con 0x7f1d54004800 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47362/0 (socket says 172.21.15.17:47362) 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 -- 172.21.15.17:0/3091559924 learned_addr learned my addr 172.21.15.17:0/3091559924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6db72640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 0x7f1d6814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6e373640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54004800 0x7f1d6814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 -- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54004800 msgr2=0x7f1d6814ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54004800 0x7f1d6814ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 -- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 msgr2=0x7f1d6814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 0x7f1d6814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6eb74640 1 -- 172.21.15.17:0/3091559924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68073250 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.775+0000 7f1d6db72640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 0x7f1d6814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d6eb74640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1d6806d960 tx=0x7f1d6000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60019070 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d60006b50 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68072920 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60003970 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.776+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d68072ee0 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.777+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6804f330 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.777+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d6001c070 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.778+0000 7f1d5effd640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 0x7f1d4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.778+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d60089a30 con 0x7f1d68148c30 2024-03-06T07:29:15.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.778+0000 7f1d6e373640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 0x7f1d4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.778+0000 7f1d6e373640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 0x7f1d4c06d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1d64001050 tx=0x7f1d64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:15.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:15.782+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1d600580d0 con 0x7f1d68148c30 2024-03-06T07:29:16.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.003+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d680c1740 con 0x7f1d68148c30 2024-03-06T07:29:16.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.027+0000 7f1d5effd640 1 -- 172.21.15.17:0/3091559924 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v199) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1d6005c170 con 0x7f1d68148c30 2024-03-06T07:29:16.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 msgr2=0x7f1d4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 0x7f1d4c06d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1d64001050 tx=0x7f1d64009040 comp rx=0 tx=0).stop 2024-03-06T07:29:16.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 msgr2=0x7f1d681490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f1d6806d960 tx=0x7f1d6000eb90 comp rx=0 tx=0).stop 2024-03-06T07:29:16.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 shutdown_connections 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1d4c06b4d0 0x7f1d4c06d990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d68148c30 0x7f1d681490b0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d681deed0 0x7f1d6814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 --2- 172.21.15.17:0/3091559924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d54004800 0x7f1d6814ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 >> 172.21.15.17:0/3091559924 conn(0x7f1d680c6710 msgr2=0x7f1d680c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 shutdown_connections 2024-03-06T07:29:16.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.029+0000 7f1d705d7640 1 -- 172.21.15.17:0/3091559924 wait complete. 2024-03-06T07:29:16.046 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-06T07:29:16.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.149+0000 7fabfcec2640 1 Processor -- start 2024-03-06T07:29:16.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.149+0000 7fabfcec2640 1 -- start start 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabfcec2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fabf806c240 0x7fabf8070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabfcec2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf8155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabfcec2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf8151f70 con 0x7fabf8152750 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabfcec2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf806ee00 con 0x7fabf806c240 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabfcec2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806f0e0 con 0x7fabf80acb80 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf659c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fabf806c240 0x7fabf8070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fabf8152750 0x7fabf814f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60594/0 (socket says 172.21.15.17:60594) 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf6d9d640 1 -- 172.21.15.17:0/2605980916 learned_addr learned my addr 172.21.15.17:0/2605980916 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf5d9b640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf8155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf659c640 1 -- 172.21.15.17:0/2605980916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 msgr2=0x7fabf8155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf659c640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf8155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf659c640 1 -- 172.21.15.17:0/2605980916 >> v1:172.21.15.17:6789/0 conn(0x7fabf8152750 legacy=0x7fabf814f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf659c640 1 -- 172.21.15.17:0/2605980916 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf806f3c0 con 0x7fabf806c240 2024-03-06T07:29:16.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.150+0000 7fabf5d9b640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf8155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabf659c640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf8070bb0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097a0 tx=0x7fabe802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bb8f90f6b272c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabf559a640 1 -- 172.21.15.17:0/2605980916 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80315a0 con 0x7fabf806c240 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabf559a640 1 -- 172.21.15.17:0/2605980916 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8037070 con 0x7fabf806c240 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabf559a640 1 -- 172.21.15.17:0/2605980916 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8031960 con 0x7fabf806c240 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 -- 172.21.15.17:0/2605980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 msgr2=0x7fabf8070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf8070bb0 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097a0 tx=0x7fabe802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:16.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 -- 172.21.15.17:0/2605980916 shutdown_connections 2024-03-06T07:29:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf8155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 --2- 172.21.15.17:0/2605980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf8070bb0 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 -- 172.21.15.17:0/2605980916 >> 172.21.15.17:0/2605980916 conn(0x7fabf80c3b40 msgr2=0x7fabf80c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 -- 172.21.15.17:0/2605980916 shutdown_connections 2024-03-06T07:29:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.152+0000 7fabfcec2640 1 -- 172.21.15.17:0/2605980916 wait complete. 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 Processor -- start 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 -- start start 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf81dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf8152750 0x7fabf80c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806c7a0 con 0x7fabf806c240 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806c620 con 0x7fabf8152750 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabfcec2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8152be0 con 0x7fabf80acb80 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabf659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabf659c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47378/0 (socket says 172.21.15.17:47378) 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.154+0000 7fabf659c640 1 -- 172.21.15.17:0/1648950703 learned_addr learned my addr 172.21.15.17:0/1648950703 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:16.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 -- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 msgr2=0x7fabf81dc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf81dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 -- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf8152750 msgr2=0x7fabf80c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf8152750 0x7fabf80c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 -- 172.21.15.17:0/1648950703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf81514e0 con 0x7fabf806c240 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabf659c640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fabe8034040 tx=0x7fabe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8032840 con 0x7fabf806c240 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8032a00 con 0x7fabf806c240 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf81517c0 con 0x7fabf806c240 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8032df0 con 0x7fabf806c240 2024-03-06T07:29:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.155+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf80aad10 con 0x7fabf806c240 2024-03-06T07:29:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.156+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf804f330 con 0x7fabf806c240 2024-03-06T07:29:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.157+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fabe804e430 con 0x7fabf806c240 2024-03-06T07:29:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.157+0000 7fabe6ffd640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 0x7fabd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.157+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fabe802f030 con 0x7fabf806c240 2024-03-06T07:29:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.157+0000 7fabf5d9b640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 0x7fabd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.158+0000 7fabf5d9b640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 0x7fabd806d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fabec006fd0 tx=0x7fabec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:16.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.161+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabe8040070 con 0x7fabf806c240 2024-03-06T07:29:16.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:16.478+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fabf81eedb0 con 0x7fabf806c240 2024-03-06T07:29:17.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.031+0000 7fabe6ffd640 1 -- 172.21.15.17:0/1648950703 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v200) v1 ==== 122+0+0 (secure 0 0 0) 0x7fabe807afd0 con 0x7fabf806c240 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 msgr2=0x7fabd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 0x7fabd806d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fabec006fd0 tx=0x7fabec008040 comp rx=0 tx=0).stop 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 msgr2=0x7fabf81ee870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fabe8034040 tx=0x7fabe80046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 shutdown_connections 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fabd806b4d0 0x7fabd806d990 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fabf8152750 0x7fabf80c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fabf80acb80 0x7fabf81dc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 --2- 172.21.15.17:0/1648950703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabf806c240 0x7fabf81ee870 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 >> 172.21.15.17:0/1648950703 conn(0x7fabf80c3b40 msgr2=0x7fabf80c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 shutdown_connections 2024-03-06T07:29:17.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.033+0000 7fabfcec2640 1 -- 172.21.15.17:0/1648950703 wait complete. 2024-03-06T07:29:17.047 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-06T07:29:17.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:17.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.147+0000 7f6b9ea17640 1 Processor -- start 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.147+0000 7f6b9ea17640 1 -- start start 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9ea17640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 0x7f6b98155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9ea17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9ea17640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b980c73d0 con 0x7f6b9814f360 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9ea17640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b980c8dd0 con 0x7f6b980c88c0 2024-03-06T07:29:17.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9ea17640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980be2e0 con 0x7f6b98158b60 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9cfb4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b977fe640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 0x7f6b98155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9cfb4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:43988/0 (socket says 172.21.15.17:43988) 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3739138265 learned_addr learned my addr 172.21.15.17:0/3739138265 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b96ffd640 1 -- 172.21.15.17:0/3739138265 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56250712 0 0) 0x7f6b980c73d0 con 0x7f6b9814f360 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.148+0000 7f6b96ffd640 1 -- 172.21.15.17:0/3739138265 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b84003520 con 0x7f6b9814f360 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3739138265 >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 msgr2=0x7f6b98155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b9cfb4640 1 --2- 172.21.15.17:0/3739138265 >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 0x7f6b98155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3739138265 >> v1:172.21.15.163:6789/0 conn(0x7f6b9814f360 legacy=0x7f6b980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3739138265 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980c7670 con 0x7f6b98158b60 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b977fe640 1 --2- 172.21.15.17:0/3739138265 >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 0x7f6b98155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.149+0000 7f6b9cfb4640 1 --2- 172.21.15.17:0/3739138265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009e50 tx=0x7f6b9002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c566242e5439fa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b96ffd640 1 -- 172.21.15.17:0/3739138265 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9002ca70 con 0x7f6b98158b60 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b96ffd640 1 -- 172.21.15.17:0/3739138265 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9002cc10 con 0x7f6b98158b60 2024-03-06T07:29:17.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b96ffd640 1 -- 172.21.15.17:0/3739138265 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b900343f0 con 0x7f6b98158b60 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3739138265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 msgr2=0x7f6b98158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3739138265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009e50 tx=0x7f6b9002bed0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3739138265 shutdown_connections 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3739138265 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b98158b60 0x7f6b98158ff0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3739138265 >> v2:172.21.15.163:3300/0 conn(0x7f6b980c88c0 0x7f6b98155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3739138265 >> 172.21.15.17:0/3739138265 conn(0x7f6b980ac570 msgr2=0x7f6b980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3739138265 shutdown_connections 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.150+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3739138265 wait complete. 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.151+0000 7f6b9ea17640 1 Processor -- start 2024-03-06T07:29:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.151+0000 7f6b9ea17640 1 -- start start 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b980c88c0 0x7f6b981eb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b9814f360 0x7f6b98157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980c5a50 con 0x7f6b98158b60 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980c7c30 con 0x7f6b980c88c0 2024-03-06T07:29:17.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9ea17640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980c5bd0 con 0x7f6b9814f360 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47392/0 (socket says 172.21.15.17:47392) 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3609162035 learned_addr learned my addr 172.21.15.17:0/3609162035 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b977fe640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b9814f360 0x7f6b98157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b97fff640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b980c88c0 0x7f6b981eb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b9814f360 msgr2=0x7f6b98157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b9814f360 0x7f6b98157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b980c88c0 msgr2=0x7f6b981eb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b980c88c0 0x7f6b981eb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.152+0000 7f6b9cfb4640 1 -- 172.21.15.17:0/3609162035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980bf540 con 0x7f6b98158b60 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b9cfb4640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6b90036040 tx=0x7f6b90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90044020 con 0x7f6b98158b60 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b90040670 con 0x7f6b98158b60 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b980bf820 con 0x7f6b98158b60 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90009510 con 0x7f6b98158b60 2024-03-06T07:29:17.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.153+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b980bfde0 con 0x7f6b98158b60 2024-03-06T07:29:17.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.154+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9804f330 con 0x7f6b98158b60 2024-03-06T07:29:17.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.154+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b90047020 con 0x7f6b98158b60 2024-03-06T07:29:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.155+0000 7f6b94ff9640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 0x7f6b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.155+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b900acb10 con 0x7f6b98158b60 2024-03-06T07:29:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.155+0000 7f6b97fff640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 0x7f6b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.156+0000 7f6b97fff640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 0x7f6b7806d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6b88007fd0 tx=0x7f6b88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.159+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b9004e030 con 0x7f6b98158b60 2024-03-06T07:29:17.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.481+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b9806cda0 con 0x7f6b98158b60 2024-03-06T07:29:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.482+0000 7f6b94ff9640 1 -- 172.21.15.17:0/3609162035 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f6b9007b240 con 0x7f6b98158b60 2024-03-06T07:29:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:17.484 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":200,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:16.480212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:17.485 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 200 2024-03-06T07:29:17.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.483+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 msgr2=0x7f6b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.483+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 0x7f6b7806d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6b88007fd0 tx=0x7f6b88009040 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.483+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 msgr2=0x7f6b980c05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.483+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f6b90036040 tx=0x7f6b90004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 shutdown_connections 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b7806b4d0 0x7f6b7806d990 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b98158b60 0x7f6b980c05e0 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b9814f360 0x7f6b98157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 --2- 172.21.15.17:0/3609162035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b980c88c0 0x7f6b981eb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 >> 172.21.15.17:0/3609162035 conn(0x7f6b980ac570 msgr2=0x7f6b9814d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 shutdown_connections 2024-03-06T07:29:17.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.484+0000 7f6b9ea17640 1 -- 172.21.15.17:0/3609162035 wait complete. 2024-03-06T07:29:17.497 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:29:17.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 Processor -- start 2024-03-06T07:29:17.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 -- start start 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6680c5480 0x7ff668155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff668158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680c5cc0 con 0x7ff6680c5860 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66806df40 con 0x7ff6680c5480 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.601+0000 7ff66da42640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66806fdb0 con 0x7ff6681586f0 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff666ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff6680c5480 0x7ff668155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff6667fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff6680c5860 0x7ff66806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60608/0 (socket says 172.21.15.17:60608) 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff6667fc640 1 -- 172.21.15.17:0/3617566564 learned_addr learned my addr 172.21.15.17:0/3617566564 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff665ffb640 1 -- 172.21.15.17:0/3617566564 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912873941 0 0) 0x7ff6680c5cc0 con 0x7ff6680c5860 2024-03-06T07:29:17.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff6677fe640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff668158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff665ffb640 1 -- 172.21.15.17:0/3617566564 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff650003520 con 0x7ff6680c5860 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff666ffd640 1 -- 172.21.15.17:0/3617566564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 msgr2=0x7ff668158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff666ffd640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff668158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff666ffd640 1 -- 172.21.15.17:0/3617566564 >> v1:172.21.15.17:6789/0 conn(0x7ff6680c5860 legacy=0x7ff66806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff666ffd640 1 -- 172.21.15.17:0/3617566564 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6681591c0 con 0x7ff6680c5480 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.602+0000 7ff6677fe640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff668158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:17.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff666ffd640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff668155ba0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7ff6580097a0 tx=0x7ff65802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93b4e3dc431d7a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff665ffb640 1 -- 172.21.15.17:0/3617566564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580315a0 con 0x7ff6680c5480 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff665ffb640 1 -- 172.21.15.17:0/3617566564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658037070 con 0x7ff6680c5480 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff665ffb640 1 -- 172.21.15.17:0/3617566564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658031960 con 0x7ff6680c5480 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 -- 172.21.15.17:0/3617566564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 msgr2=0x7ff668155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff668155ba0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7ff6580097a0 tx=0x7ff65802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:17.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 -- 172.21.15.17:0/3617566564 shutdown_connections 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff668158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 --2- 172.21.15.17:0/3617566564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff668155ba0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 -- 172.21.15.17:0/3617566564 >> 172.21.15.17:0/3617566564 conn(0x7ff66806b1a0 msgr2=0x7ff66806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 -- 172.21.15.17:0/3617566564 shutdown_connections 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.603+0000 7ff66da42640 1 -- 172.21.15.17:0/3617566564 wait complete. 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 Processor -- start 2024-03-06T07:29:17.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 -- start start 2024-03-06T07:29:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6680c5860 0x7ff6681eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff6680c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668155680 con 0x7ff6680c5480 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6681553a0 con 0x7ff6680c5860 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff66da42640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff668152a20 con 0x7ff6681586f0 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff666ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff666ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47408/0 (socket says 172.21.15.17:47408) 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.605+0000 7ff666ffd640 1 -- 172.21.15.17:0/2332028577 learned_addr learned my addr 172.21.15.17:0/2332028577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff6677fe640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff6680c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff6667fc640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6680c5860 0x7ff6681eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 -- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 msgr2=0x7ff6680c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff6680c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 -- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6680c5860 msgr2=0x7ff6681eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6680c5860 0x7ff6681eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 -- 172.21.15.17:0/2332028577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668156ef0 con 0x7ff6680c5480 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff666ffd640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098c0 tx=0x7ff6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658032630 con 0x7ff6680c5480 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6580327d0 con 0x7ff6680c5480 2024-03-06T07:29:17.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6681571d0 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658032bc0 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.606+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff668158200 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.607+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66804f330 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.607+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff65804e430 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.608+0000 7ff64ffff640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 0x7ff65406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.608+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff65802f030 con 0x7ff6680c5480 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.608+0000 7ff6667fc640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 0x7ff65406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.609+0000 7ff6667fc640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 0x7ff65406d9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff6680c4bd0 tx=0x7ff65c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.612+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff658040070 con 0x7ff6680c5480 2024-03-06T07:29:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.799+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff66806e0c0 con 0x7ff6680c5480 2024-03-06T07:29:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.800+0000 7ff64ffff640 1 -- 172.21.15.17:0/2332028577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+16888 (secure 0 0 0) 0x7ff65807ae90 con 0x7ff6680c5480 2024-03-06T07:29:17.803 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:17.803 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":212,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:29:16.021740+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 msgr2=0x7ff65406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 0x7ff65406d9c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff6680c4bd0 tx=0x7ff65c009040 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 msgr2=0x7ff6681ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098c0 tx=0x7ff6580046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 shutdown_connections 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff65406b500 0x7ff65406d9c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6681586f0 0x7ff6680c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6680c5860 0x7ff6681eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 --2- 172.21.15.17:0/2332028577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6680c5480 0x7ff6681ee710 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 >> 172.21.15.17:0/2332028577 conn(0x7ff66806b1a0 msgr2=0x7ff6680c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 shutdown_connections 2024-03-06T07:29:17.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.803+0000 7ff66da42640 1 -- 172.21.15.17:0/2332028577 wait complete. 2024-03-06T07:29:17.816 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-06T07:29:17.817 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:29:17.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.915+0000 7f74555b5640 1 Processor -- start 2024-03-06T07:29:17.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.915+0000 7f74555b5640 1 -- start start 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f74555b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f7450155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f74555b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7450158b60 0x7f7450158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f74555b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74500c73d0 con 0x7f745014f360 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f74555b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74500c8dd0 con 0x7f7450158b60 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f74555b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500be2e0 con 0x7f74500c88c0 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7450158b60 0x7f7450158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7450158b60 0x7f7450158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47418/0 (socket says 172.21.15.17:47418) 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 -- 172.21.15.17:0/752586887 learned_addr learned my addr 172.21.15.17:0/752586887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f7447fff640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f7450155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 -- 172.21.15.17:0/752586887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 msgr2=0x7f7450155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f7450155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 -- 172.21.15.17:0/752586887 >> v1:172.21.15.163:6789/0 conn(0x7f745014f360 legacy=0x7f74500c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.916+0000 7f744f7fe640 1 -- 172.21.15.17:0/752586887 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500c7670 con 0x7f7450158b60 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.917+0000 7f744f7fe640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7450158b60 0x7f7450158ff0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f7448009e50 tx=0x7f744802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67a57fe5a687a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.917+0000 7f744e7fc640 1 -- 172.21.15.17:0/752586887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744802ca80 con 0x7f7450158b60 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.917+0000 7f744e7fc640 1 -- 172.21.15.17:0/752586887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744802cc20 con 0x7f7450158b60 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.917+0000 7f744e7fc640 1 -- 172.21.15.17:0/752586887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480343f0 con 0x7f7450158b60 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 -- 172.21.15.17:0/752586887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7450158b60 msgr2=0x7f7450158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7450158b60 0x7f7450158ff0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f7448009e50 tx=0x7f744802bee0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 -- 172.21.15.17:0/752586887 shutdown_connections 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7450158b60 0x7f7450158ff0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 --2- 172.21.15.17:0/752586887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f7450155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 -- 172.21.15.17:0/752586887 >> 172.21.15.17:0/752586887 conn(0x7f74500ac570 msgr2=0x7f74500ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 -- 172.21.15.17:0/752586887 shutdown_connections 2024-03-06T07:29:17.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.918+0000 7f74555b5640 1 -- 172.21.15.17:0/752586887 wait complete. 2024-03-06T07:29:17.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.919+0000 7f74555b5640 1 Processor -- start 2024-03-06T07:29:17.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.919+0000 7f74555b5640 1 -- start start 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f74500c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7450158b60 0x7f74500bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500c5a50 con 0x7f745014f360 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74500c7c30 con 0x7f7450158b60 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f74555b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500c5bd0 con 0x7f74500c88c0 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f7447fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f7447fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47428/0 (socket says 172.21.15.17:47428) 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f7447fff640 1 -- 172.21.15.17:0/1546577830 learned_addr learned my addr 172.21.15.17:0/1546577830 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:17.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.920+0000 7f744f7fe640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7450158b60 0x7f74500bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f744effd640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f74500c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 -- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 msgr2=0x7f74500c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f74500c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 -- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7450158b60 msgr2=0x7f74500bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7450158b60 0x7f74500bf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 -- 172.21.15.17:0/1546577830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500c0230 con 0x7f745014f360 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f7447fff640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f743800e840 tx=0x7f743800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438017070 con 0x7f745014f360 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800c940 con 0x7f745014f360 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800cd10 con 0x7f745014f360 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74500be490 con 0x7f745014f360 2024-03-06T07:29:17.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.921+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74500bea50 con 0x7f745014f360 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.922+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745004f330 con 0x7f745014f360 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.923+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f743801a070 con 0x7f745014f360 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.923+0000 7f74477fe640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 0x7f743006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.923+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f743808a180 con 0x7f745014f360 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.923+0000 7f744effd640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 0x7f743006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:17.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.924+0000 7f744effd640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 0x7f743006d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7440007fd0 tx=0x7f7440009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:17.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:17.927+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7438058f00 con 0x7f745014f360 2024-03-06T07:29:18.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.105+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7450152ac0 con 0x7f743006b4d0 2024-03-06T07:29:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.111+0000 7f74477fe640 1 -- 172.21.15.17:0/1546577830 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252562 (secure 0 0 0) 0x7f74400050d0 con 0x7f743006b4d0 2024-03-06T07:29:18.117 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 msgr2=0x7f743006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 0x7f743006d990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7440007fd0 tx=0x7f7440009040 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 msgr2=0x7f74501eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f743800e840 tx=0x7f743800ed00 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 shutdown_connections 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f743006b4d0 0x7f743006d990 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7450158b60 0x7f74500bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745014f360 0x7f74501eb750 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 --2- 172.21.15.17:0/1546577830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f74500c88c0 0x7f74500c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 >> 172.21.15.17:0/1546577830 conn(0x7f74500ac570 msgr2=0x7f745006b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 shutdown_connections 2024-03-06T07:29:18.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.115+0000 7f74555b5640 1 -- 172.21.15.17:0/1546577830 wait complete. 2024-03-06T07:29:18.128 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-03-06T07:29:17.468105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283116,"kb_used_data":21168,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691604,"statfs":{"total":773094113280,"available":772804202496,"internally_reserved":0,"allocated":21676032,"data_stored":11650124,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":268095028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.461740"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034707+0000","last_change":"2024-03-06T07:29:15.033803+0000","last_active":"2024-03-06T07:29:16.034707+0000","last_peered":"2024-03-06T07:29:16.034707+0000","last_clean":"2024-03-06T07:29:16.034707+0000","last_became_active":"2024-03-06T07:29:15.033282+0000","last_became_peered":"2024-03-06T07:29:15.033282+0000","last_unstale":"2024-03-06T07:29:16.034707+0000","last_undegraded":"2024-03-06T07:29:16.034707+0000","last_fullsized":"2024-03-06T07:29:16.034707+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:44.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030891+0000","last_change":"2024-03-06T07:29:15.040115+0000","last_active":"2024-03-06T07:29:16.030891+0000","last_peered":"2024-03-06T07:29:16.030891+0000","last_clean":"2024-03-06T07:29:16.030891+0000","last_became_active":"2024-03-06T07:29:15.039740+0000","last_became_peered":"2024-03-06T07:29:15.039740+0000","last_unstale":"2024-03-06T07:29:16.030891+0000","last_undegraded":"2024-03-06T07:29:16.030891+0000","last_fullsized":"2024-03-06T07:29:16.030891+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:24.207873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039684+0000","last_change":"2024-03-06T07:29:15.035672+0000","last_active":"2024-03-06T07:29:15.039684+0000","last_peered":"2024-03-06T07:29:15.039684+0000","last_clean":"2024-03-06T07:29:15.039684+0000","last_became_active":"2024-03-06T07:29:15.035401+0000","last_became_peered":"2024-03-06T07:29:15.035401+0000","last_unstale":"2024-03-06T07:29:15.039684+0000","last_undegraded":"2024-03-06T07:29:15.039684+0000","last_fullsized":"2024-03-06T07:29:15.039684+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:54:14.834987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571236+0000","last_change":"2024-03-06T07:29:15.033488+0000","last_active":"2024-03-06T07:29:16.571236+0000","last_peered":"2024-03-06T07:29:16.571236+0000","last_clean":"2024-03-06T07:29:16.571236+0000","last_became_active":"2024-03-06T07:29:15.033098+0000","last_became_peered":"2024-03-06T07:29:15.033098+0000","last_unstale":"2024-03-06T07:29:16.571236+0000","last_undegraded":"2024-03-06T07:29:16.571236+0000","last_fullsized":"2024-03-06T07:29:16.571236+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:55.521589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.043594+0000","last_change":"2024-03-06T07:29:15.042500+0000","last_active":"2024-03-06T07:29:15.043594+0000","last_peered":"2024-03-06T07:29:15.043594+0000","last_clean":"2024-03-06T07:29:15.043594+0000","last_became_active":"2024-03-06T07:29:15.042254+0000","last_became_peered":"2024-03-06T07:29:15.042254+0000","last_unstale":"2024-03-06T07:29:15.043594+0000","last_undegraded":"2024-03-06T07:29:15.043594+0000","last_fullsized":"2024-03-06T07:29:15.043594+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:53:43.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019819+0000","last_change":"2024-03-06T07:29:15.038746+0000","last_active":"2024-03-06T07:29:17.019819+0000","last_peered":"2024-03-06T07:29:17.019819+0000","last_clean":"2024-03-06T07:29:17.019819+0000","last_became_active":"2024-03-06T07:29:15.038349+0000","last_became_peered":"2024-03-06T07:29:15.038349+0000","last_unstale":"2024-03-06T07:29:17.019819+0000","last_undegraded":"2024-03-06T07:29:17.019819+0000","last_fullsized":"2024-03-06T07:29:17.019819+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.370419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029918+0000","last_change":"2024-03-06T07:29:15.025643+0000","last_active":"2024-03-06T07:29:15.029918+0000","last_peered":"2024-03-06T07:29:15.029918+0000","last_clean":"2024-03-06T07:29:15.029918+0000","last_became_active":"2024-03-06T07:29:15.025375+0000","last_became_peered":"2024-03-06T07:29:15.025375+0000","last_unstale":"2024-03-06T07:29:15.029918+0000","last_undegraded":"2024-03-06T07:29:15.029918+0000","last_fullsized":"2024-03-06T07:29:15.029918+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:56.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570869+0000","last_change":"2024-03-06T07:29:15.036351+0000","last_active":"2024-03-06T07:29:16.570869+0000","last_peered":"2024-03-06T07:29:16.570869+0000","last_clean":"2024-03-06T07:29:16.570869+0000","last_became_active":"2024-03-06T07:29:15.035792+0000","last_became_peered":"2024-03-06T07:29:15.035792+0000","last_unstale":"2024-03-06T07:29:16.570869+0000","last_undegraded":"2024-03-06T07:29:16.570869+0000","last_fullsized":"2024-03-06T07:29:16.570869+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:19:33.416598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020308+0000","last_change":"2024-03-06T07:29:15.034693+0000","last_active":"2024-03-06T07:29:17.020308+0000","last_peered":"2024-03-06T07:29:17.020308+0000","last_clean":"2024-03-06T07:29:17.020308+0000","last_became_active":"2024-03-06T07:29:15.034382+0000","last_became_peered":"2024-03-06T07:29:15.034382+0000","last_unstale":"2024-03-06T07:29:17.020308+0000","last_undegraded":"2024-03-06T07:29:17.020308+0000","last_fullsized":"2024-03-06T07:29:17.020308+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:19.609882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034424+0000","last_change":"2024-03-06T07:29:15.027164+0000","last_active":"2024-03-06T07:29:16.034424+0000","last_peered":"2024-03-06T07:29:16.034424+0000","last_clean":"2024-03-06T07:29:16.034424+0000","last_became_active":"2024-03-06T07:29:15.025973+0000","last_became_peered":"2024-03-06T07:29:15.025973+0000","last_unstale":"2024-03-06T07:29:16.034424+0000","last_undegraded":"2024-03-06T07:29:16.034424+0000","last_fullsized":"2024-03-06T07:29:16.034424+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:26.952702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034747+0000","last_change":"2024-03-06T07:29:15.026581+0000","last_active":"2024-03-06T07:29:16.034747+0000","last_peered":"2024-03-06T07:29:16.034747+0000","last_clean":"2024-03-06T07:29:16.034747+0000","last_became_active":"2024-03-06T07:29:15.026304+0000","last_became_peered":"2024-03-06T07:29:15.026304+0000","last_unstale":"2024-03-06T07:29:16.034747+0000","last_undegraded":"2024-03-06T07:29:16.034747+0000","last_fullsized":"2024-03-06T07:29:16.034747+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:45.420192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020105+0000","last_change":"2024-03-06T07:29:15.031760+0000","last_active":"2024-03-06T07:29:17.020105+0000","last_peered":"2024-03-06T07:29:17.020105+0000","last_clean":"2024-03-06T07:29:17.020105+0000","last_became_active":"2024-03-06T07:29:15.030268+0000","last_became_peered":"2024-03-06T07:29:15.030268+0000","last_unstale":"2024-03-06T07:29:17.020105+0000","last_undegraded":"2024-03-06T07:29:17.020105+0000","last_fullsized":"2024-03-06T07:29:17.020105+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:08.244634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571287+0000","last_change":"2024-03-06T07:29:15.039963+0000","last_active":"2024-03-06T07:29:16.571287+0000","last_peered":"2024-03-06T07:29:16.571287+0000","last_clean":"2024-03-06T07:29:16.571287+0000","last_became_active":"2024-03-06T07:29:15.039690+0000","last_became_peered":"2024-03-06T07:29:15.039690+0000","last_unstale":"2024-03-06T07:29:16.571287+0000","last_undegraded":"2024-03-06T07:29:16.571287+0000","last_fullsized":"2024-03-06T07:29:16.571287+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:52.397315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030823+0000","last_change":"2024-03-06T07:29:15.044872+0000","last_active":"2024-03-06T07:29:16.030823+0000","last_peered":"2024-03-06T07:29:16.030823+0000","last_clean":"2024-03-06T07:29:16.030823+0000","last_became_active":"2024-03-06T07:29:15.044362+0000","last_became_peered":"2024-03-06T07:29:15.044362+0000","last_unstale":"2024-03-06T07:29:16.030823+0000","last_undegraded":"2024-03-06T07:29:16.030823+0000","last_fullsized":"2024-03-06T07:29:16.030823+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:22.901814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030600+0000","last_change":"2024-03-06T07:29:15.040549+0000","last_active":"2024-03-06T07:29:16.030600+0000","last_peered":"2024-03-06T07:29:16.030600+0000","last_clean":"2024-03-06T07:29:16.030600+0000","last_became_active":"2024-03-06T07:29:15.040240+0000","last_became_peered":"2024-03-06T07:29:15.040240+0000","last_unstale":"2024-03-06T07:29:16.030600+0000","last_undegraded":"2024-03-06T07:29:16.030600+0000","last_fullsized":"2024-03-06T07:29:16.030600+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:58.499423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034310+0000","last_change":"2024-03-06T07:29:15.030886+0000","last_active":"2024-03-06T07:29:16.034310+0000","last_peered":"2024-03-06T07:29:16.034310+0000","last_clean":"2024-03-06T07:29:16.034310+0000","last_became_active":"2024-03-06T07:29:15.030592+0000","last_became_peered":"2024-03-06T07:29:15.030592+0000","last_unstale":"2024-03-06T07:29:16.034310+0000","last_undegraded":"2024-03-06T07:29:16.034310+0000","last_fullsized":"2024-03-06T07:29:16.034310+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:55.325606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034350+0000","last_change":"2024-03-06T07:29:15.037166+0000","last_active":"2024-03-06T07:29:16.034350+0000","last_peered":"2024-03-06T07:29:16.034350+0000","last_clean":"2024-03-06T07:29:16.034350+0000","last_became_active":"2024-03-06T07:29:15.036848+0000","last_became_peered":"2024-03-06T07:29:15.036848+0000","last_unstale":"2024-03-06T07:29:16.034350+0000","last_undegraded":"2024-03-06T07:29:16.034350+0000","last_fullsized":"2024-03-06T07:29:16.034350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.019571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017775+0000","last_change":"2024-03-06T07:29:15.037532+0000","last_active":"2024-03-06T07:29:17.017775+0000","last_peered":"2024-03-06T07:29:17.017775+0000","last_clean":"2024-03-06T07:29:17.017775+0000","last_became_active":"2024-03-06T07:29:15.037030+0000","last_became_peered":"2024-03-06T07:29:15.037030+0000","last_unstale":"2024-03-06T07:29:17.017775+0000","last_undegraded":"2024-03-06T07:29:17.017775+0000","last_fullsized":"2024-03-06T07:29:17.017775+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:11.609877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033745+0000","last_change":"2024-03-06T07:29:15.020745+0000","last_active":"2024-03-06T07:29:16.033745+0000","last_peered":"2024-03-06T07:29:16.033745+0000","last_clean":"2024-03-06T07:29:16.033745+0000","last_became_active":"2024-03-06T07:29:15.020457+0000","last_became_peered":"2024-03-06T07:29:15.020457+0000","last_unstale":"2024-03-06T07:29:16.033745+0000","last_undegraded":"2024-03-06T07:29:16.033745+0000","last_fullsized":"2024-03-06T07:29:16.033745+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:15.926071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020108+0000","last_change":"2024-03-06T07:29:15.038239+0000","last_active":"2024-03-06T07:29:17.020108+0000","last_peered":"2024-03-06T07:29:17.020108+0000","last_clean":"2024-03-06T07:29:17.020108+0000","last_became_active":"2024-03-06T07:29:15.037893+0000","last_became_peered":"2024-03-06T07:29:15.037893+0000","last_unstale":"2024-03-06T07:29:17.020108+0000","last_undegraded":"2024-03-06T07:29:17.020108+0000","last_fullsized":"2024-03-06T07:29:17.020108+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:37.781015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042659+0000","last_change":"2024-03-06T07:29:15.041455+0000","last_active":"2024-03-06T07:29:15.042659+0000","last_peered":"2024-03-06T07:29:15.042659+0000","last_clean":"2024-03-06T07:29:15.042659+0000","last_became_active":"2024-03-06T07:29:15.041182+0000","last_became_peered":"2024-03-06T07:29:15.041182+0000","last_unstale":"2024-03-06T07:29:15.042659+0000","last_undegraded":"2024-03-06T07:29:15.042659+0000","last_fullsized":"2024-03-06T07:29:15.042659+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:56.578344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033944+0000","last_change":"2024-03-06T07:29:15.031960+0000","last_active":"2024-03-06T07:29:16.033944+0000","last_peered":"2024-03-06T07:29:16.033944+0000","last_clean":"2024-03-06T07:29:16.033944+0000","last_became_active":"2024-03-06T07:29:15.030103+0000","last_became_peered":"2024-03-06T07:29:15.030103+0000","last_unstale":"2024-03-06T07:29:16.033944+0000","last_undegraded":"2024-03-06T07:29:16.033944+0000","last_fullsized":"2024-03-06T07:29:16.033944+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:11.820083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030556+0000","last_change":"2024-03-06T07:29:15.036298+0000","last_active":"2024-03-06T07:29:16.030556+0000","last_peered":"2024-03-06T07:29:16.030556+0000","last_clean":"2024-03-06T07:29:16.030556+0000","last_became_active":"2024-03-06T07:29:15.035470+0000","last_became_peered":"2024-03-06T07:29:15.035470+0000","last_unstale":"2024-03-06T07:29:16.030556+0000","last_undegraded":"2024-03-06T07:29:16.030556+0000","last_fullsized":"2024-03-06T07:29:16.030556+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:53.111160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034044+0000","last_change":"2024-03-06T07:29:15.028230+0000","last_active":"2024-03-06T07:29:16.034044+0000","last_peered":"2024-03-06T07:29:16.034044+0000","last_clean":"2024-03-06T07:29:16.034044+0000","last_became_active":"2024-03-06T07:29:15.027926+0000","last_became_peered":"2024-03-06T07:29:15.027926+0000","last_unstale":"2024-03-06T07:29:16.034044+0000","last_undegraded":"2024-03-06T07:29:16.034044+0000","last_fullsized":"2024-03-06T07:29:16.034044+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:57.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042531+0000","last_change":"2024-03-06T07:29:15.038176+0000","last_active":"2024-03-06T07:29:15.042531+0000","last_peered":"2024-03-06T07:29:15.042531+0000","last_clean":"2024-03-06T07:29:15.042531+0000","last_became_active":"2024-03-06T07:29:15.037908+0000","last_became_peered":"2024-03-06T07:29:15.037908+0000","last_unstale":"2024-03-06T07:29:15.042531+0000","last_undegraded":"2024-03-06T07:29:15.042531+0000","last_fullsized":"2024-03-06T07:29:15.042531+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:54:59.988732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031017+0000","last_change":"2024-03-06T07:29:15.032343+0000","last_active":"2024-03-06T07:29:16.031017+0000","last_peered":"2024-03-06T07:29:16.031017+0000","last_clean":"2024-03-06T07:29:16.031017+0000","last_became_active":"2024-03-06T07:29:15.032060+0000","last_became_peered":"2024-03-06T07:29:15.032060+0000","last_unstale":"2024-03-06T07:29:16.031017+0000","last_undegraded":"2024-03-06T07:29:16.031017+0000","last_fullsized":"2024-03-06T07:29:16.031017+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:45.267220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034795+0000","last_change":"2024-03-06T07:29:15.032781+0000","last_active":"2024-03-06T07:29:16.034795+0000","last_peered":"2024-03-06T07:29:16.034795+0000","last_clean":"2024-03-06T07:29:16.034795+0000","last_became_active":"2024-03-06T07:29:15.031276+0000","last_became_peered":"2024-03-06T07:29:15.031276+0000","last_unstale":"2024-03-06T07:29:16.034795+0000","last_undegraded":"2024-03-06T07:29:16.034795+0000","last_fullsized":"2024-03-06T07:29:16.034795+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:18.448892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034308+0000","last_change":"2024-03-06T07:29:15.031915+0000","last_active":"2024-03-06T07:29:16.034308+0000","last_peered":"2024-03-06T07:29:16.034308+0000","last_clean":"2024-03-06T07:29:16.034308+0000","last_became_active":"2024-03-06T07:29:15.031381+0000","last_became_peered":"2024-03-06T07:29:15.031381+0000","last_unstale":"2024-03-06T07:29:16.034308+0000","last_undegraded":"2024-03-06T07:29:16.034308+0000","last_fullsized":"2024-03-06T07:29:16.034308+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:39.084230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019898+0000","last_change":"2024-03-06T07:29:15.039048+0000","last_active":"2024-03-06T07:29:17.019898+0000","last_peered":"2024-03-06T07:29:17.019898+0000","last_clean":"2024-03-06T07:29:17.019898+0000","last_became_active":"2024-03-06T07:29:15.038533+0000","last_became_peered":"2024-03-06T07:29:15.038533+0000","last_unstale":"2024-03-06T07:29:17.019898+0000","last_undegraded":"2024-03-06T07:29:17.019898+0000","last_fullsized":"2024-03-06T07:29:17.019898+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:43.165397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030713+0000","last_change":"2024-03-06T07:29:15.038354+0000","last_active":"2024-03-06T07:29:16.030713+0000","last_peered":"2024-03-06T07:29:16.030713+0000","last_clean":"2024-03-06T07:29:16.030713+0000","last_became_active":"2024-03-06T07:29:15.037386+0000","last_became_peered":"2024-03-06T07:29:15.037386+0000","last_unstale":"2024-03-06T07:29:16.030713+0000","last_undegraded":"2024-03-06T07:29:16.030713+0000","last_fullsized":"2024-03-06T07:29:16.030713+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:39.820037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030448+0000","last_change":"2024-03-06T07:29:15.039364+0000","last_active":"2024-03-06T07:29:16.030448+0000","last_peered":"2024-03-06T07:29:16.030448+0000","last_clean":"2024-03-06T07:29:16.030448+0000","last_became_active":"2024-03-06T07:29:15.039092+0000","last_became_peered":"2024-03-06T07:29:15.039092+0000","last_unstale":"2024-03-06T07:29:16.030448+0000","last_undegraded":"2024-03-06T07:29:16.030448+0000","last_fullsized":"2024-03-06T07:29:16.030448+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:56.578344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033857+0000","last_change":"2024-03-06T07:29:15.027471+0000","last_active":"2024-03-06T07:29:16.033857+0000","last_peered":"2024-03-06T07:29:16.033857+0000","last_clean":"2024-03-06T07:29:16.033857+0000","last_became_active":"2024-03-06T07:29:15.027170+0000","last_became_peered":"2024-03-06T07:29:15.027170+0000","last_unstale":"2024-03-06T07:29:16.033857+0000","last_undegraded":"2024-03-06T07:29:16.033857+0000","last_fullsized":"2024-03-06T07:29:16.033857+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:50.867428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"211'4","reported_seq":18,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017769+0000","last_change":"2024-03-06T07:29:14.032355+0000","last_active":"2024-03-06T07:29:17.017769+0000","last_peered":"2024-03-06T07:29:17.017769+0000","last_clean":"2024-03-06T07:29:17.017769+0000","last_became_active":"2024-03-06T07:29:14.032092+0000","last_became_peered":"2024-03-06T07:29:14.032092+0000","last_unstale":"2024-03-06T07:29:17.017769+0000","last_undegraded":"2024-03-06T07:29:17.017769+0000","last_fullsized":"2024-03-06T07:29:17.017769+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:30.483240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017680+0000","last_change":"2024-03-06T07:29:15.043032+0000","last_active":"2024-03-06T07:29:17.017680+0000","last_peered":"2024-03-06T07:29:17.017680+0000","last_clean":"2024-03-06T07:29:17.017680+0000","last_became_active":"2024-03-06T07:29:15.042762+0000","last_became_peered":"2024-03-06T07:29:15.042762+0000","last_unstale":"2024-03-06T07:29:17.017680+0000","last_undegraded":"2024-03-06T07:29:17.017680+0000","last_fullsized":"2024-03-06T07:29:17.017680+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:00.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034209+0000","last_change":"2024-03-06T07:29:14.020709+0000","last_active":"2024-03-06T07:29:16.034209+0000","last_peered":"2024-03-06T07:29:16.034209+0000","last_clean":"2024-03-06T07:29:16.034209+0000","last_became_active":"2024-03-06T07:29:14.020430+0000","last_became_peered":"2024-03-06T07:29:14.020430+0000","last_unstale":"2024-03-06T07:29:16.034209+0000","last_undegraded":"2024-03-06T07:29:16.034209+0000","last_fullsized":"2024-03-06T07:29:16.034209+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017771+0000","last_change":"2024-03-06T07:29:15.038256+0000","last_active":"2024-03-06T07:29:17.017771+0000","last_peered":"2024-03-06T07:29:17.017771+0000","last_clean":"2024-03-06T07:29:17.017771+0000","last_became_active":"2024-03-06T07:29:15.037847+0000","last_became_peered":"2024-03-06T07:29:15.037847+0000","last_unstale":"2024-03-06T07:29:17.017771+0000","last_undegraded":"2024-03-06T07:29:17.017771+0000","last_fullsized":"2024-03-06T07:29:17.017771+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:30.072150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034929+0000","last_change":"2024-03-06T07:29:14.031928+0000","last_active":"2024-03-06T07:29:16.034929+0000","last_peered":"2024-03-06T07:29:16.034929+0000","last_clean":"2024-03-06T07:29:16.034929+0000","last_became_active":"2024-03-06T07:29:14.031631+0000","last_became_peered":"2024-03-06T07:29:14.031631+0000","last_unstale":"2024-03-06T07:29:16.034929+0000","last_undegraded":"2024-03-06T07:29:16.034929+0000","last_fullsized":"2024-03-06T07:29:16.034929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:15.530531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031193+0000","last_change":"2024-03-06T07:29:15.033634+0000","last_active":"2024-03-06T07:29:16.031193+0000","last_peered":"2024-03-06T07:29:16.031193+0000","last_clean":"2024-03-06T07:29:16.031193+0000","last_became_active":"2024-03-06T07:29:15.033375+0000","last_became_peered":"2024-03-06T07:29:15.033375+0000","last_unstale":"2024-03-06T07:29:16.031193+0000","last_undegraded":"2024-03-06T07:29:16.031193+0000","last_fullsized":"2024-03-06T07:29:16.031193+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:47.983876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034072+0000","last_change":"2024-03-06T07:29:14.034963+0000","last_active":"2024-03-06T07:29:16.034072+0000","last_peered":"2024-03-06T07:29:16.034072+0000","last_clean":"2024-03-06T07:29:16.034072+0000","last_became_active":"2024-03-06T07:29:14.034677+0000","last_became_peered":"2024-03-06T07:29:14.034677+0000","last_unstale":"2024-03-06T07:29:16.034072+0000","last_undegraded":"2024-03-06T07:29:16.034072+0000","last_fullsized":"2024-03-06T07:29:16.034072+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:13.368742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030506+0000","last_change":"2024-03-06T07:29:15.035450+0000","last_active":"2024-03-06T07:29:16.030506+0000","last_peered":"2024-03-06T07:29:16.030506+0000","last_clean":"2024-03-06T07:29:16.030506+0000","last_became_active":"2024-03-06T07:29:15.034828+0000","last_became_peered":"2024-03-06T07:29:15.034828+0000","last_unstale":"2024-03-06T07:29:16.030506+0000","last_undegraded":"2024-03-06T07:29:16.030506+0000","last_fullsized":"2024-03-06T07:29:16.030506+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:53:43.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"211'1","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038565+0000","last_change":"2024-03-06T07:29:14.031147+0000","last_active":"2024-03-06T07:29:15.038565+0000","last_peered":"2024-03-06T07:29:15.038565+0000","last_clean":"2024-03-06T07:29:15.038565+0000","last_became_active":"2024-03-06T07:29:14.029748+0000","last_became_peered":"2024-03-06T07:29:14.029748+0000","last_unstale":"2024-03-06T07:29:15.038565+0000","last_undegraded":"2024-03-06T07:29:15.038565+0000","last_fullsized":"2024-03-06T07:29:15.038565+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:11.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030755+0000","last_change":"2024-03-06T07:29:15.029842+0000","last_active":"2024-03-06T07:29:16.030755+0000","last_peered":"2024-03-06T07:29:16.030755+0000","last_clean":"2024-03-06T07:29:16.030755+0000","last_became_active":"2024-03-06T07:29:15.029540+0000","last_became_peered":"2024-03-06T07:29:15.029540+0000","last_unstale":"2024-03-06T07:29:16.030755+0000","last_undegraded":"2024-03-06T07:29:16.030755+0000","last_fullsized":"2024-03-06T07:29:16.030755+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:58.246941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031054+0000","last_change":"2024-03-06T07:29:14.022007+0000","last_active":"2024-03-06T07:29:16.031054+0000","last_peered":"2024-03-06T07:29:16.031054+0000","last_clean":"2024-03-06T07:29:16.031054+0000","last_became_active":"2024-03-06T07:29:14.021716+0000","last_became_peered":"2024-03-06T07:29:14.021716+0000","last_unstale":"2024-03-06T07:29:16.031054+0000","last_undegraded":"2024-03-06T07:29:16.031054+0000","last_fullsized":"2024-03-06T07:29:16.031054+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:07.239775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.034989+0000","last_change":"2024-03-06T07:29:15.032188+0000","last_active":"2024-03-06T07:29:15.034989+0000","last_peered":"2024-03-06T07:29:15.034989+0000","last_clean":"2024-03-06T07:29:15.034989+0000","last_became_active":"2024-03-06T07:29:15.031911+0000","last_became_peered":"2024-03-06T07:29:15.031911+0000","last_unstale":"2024-03-06T07:29:15.034989+0000","last_undegraded":"2024-03-06T07:29:15.034989+0000","last_fullsized":"2024-03-06T07:29:15.034989+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:14.113373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033659+0000","last_change":"2024-03-06T07:29:14.028532+0000","last_active":"2024-03-06T07:29:16.033659+0000","last_peered":"2024-03-06T07:29:16.033659+0000","last_clean":"2024-03-06T07:29:16.033659+0000","last_became_active":"2024-03-06T07:29:14.028226+0000","last_became_peered":"2024-03-06T07:29:14.028226+0000","last_unstale":"2024-03-06T07:29:16.033659+0000","last_undegraded":"2024-03-06T07:29:16.033659+0000","last_fullsized":"2024-03-06T07:29:16.033659+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:21.126122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571419+0000","last_change":"2024-03-06T07:29:15.038779+0000","last_active":"2024-03-06T07:29:16.571419+0000","last_peered":"2024-03-06T07:29:16.571419+0000","last_clean":"2024-03-06T07:29:16.571419+0000","last_became_active":"2024-03-06T07:29:15.038353+0000","last_became_peered":"2024-03-06T07:29:15.038353+0000","last_unstale":"2024-03-06T07:29:16.571419+0000","last_undegraded":"2024-03-06T07:29:16.571419+0000","last_fullsized":"2024-03-06T07:29:16.571419+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:23.216683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.014354+0000","last_change":"2024-03-06T07:29:14.029645+0000","last_active":"2024-03-06T07:29:15.014354+0000","last_peered":"2024-03-06T07:29:15.014354+0000","last_clean":"2024-03-06T07:29:15.014354+0000","last_became_active":"2024-03-06T07:29:14.029367+0000","last_became_peered":"2024-03-06T07:29:14.029367+0000","last_unstale":"2024-03-06T07:29:15.014354+0000","last_undegraded":"2024-03-06T07:29:15.014354+0000","last_fullsized":"2024-03-06T07:29:15.014354+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:01.067942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571205+0000","last_change":"2024-03-06T07:29:15.041825+0000","last_active":"2024-03-06T07:29:16.571205+0000","last_peered":"2024-03-06T07:29:16.571205+0000","last_clean":"2024-03-06T07:29:16.571205+0000","last_became_active":"2024-03-06T07:29:15.041550+0000","last_became_peered":"2024-03-06T07:29:15.041550+0000","last_unstale":"2024-03-06T07:29:16.571205+0000","last_undegraded":"2024-03-06T07:29:16.571205+0000","last_fullsized":"2024-03-06T07:29:16.571205+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:44.310590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571323+0000","last_change":"2024-03-06T07:29:14.032990+0000","last_active":"2024-03-06T07:29:16.571323+0000","last_peered":"2024-03-06T07:29:16.571323+0000","last_clean":"2024-03-06T07:29:16.571323+0000","last_became_active":"2024-03-06T07:29:14.032701+0000","last_became_peered":"2024-03-06T07:29:14.032701+0000","last_unstale":"2024-03-06T07:29:16.571323+0000","last_undegraded":"2024-03-06T07:29:16.571323+0000","last_fullsized":"2024-03-06T07:29:16.571323+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:18:17.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017686+0000","last_change":"2024-03-06T07:29:15.036708+0000","last_active":"2024-03-06T07:29:17.017686+0000","last_peered":"2024-03-06T07:29:17.017686+0000","last_clean":"2024-03-06T07:29:17.017686+0000","last_became_active":"2024-03-06T07:29:15.036199+0000","last_became_peered":"2024-03-06T07:29:15.036199+0000","last_unstale":"2024-03-06T07:29:17.017686+0000","last_undegraded":"2024-03-06T07:29:17.017686+0000","last_fullsized":"2024-03-06T07:29:17.017686+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:31.581503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034316+0000","last_change":"2024-03-06T07:29:14.026059+0000","last_active":"2024-03-06T07:29:16.034316+0000","last_peered":"2024-03-06T07:29:16.034316+0000","last_clean":"2024-03-06T07:29:16.034316+0000","last_became_active":"2024-03-06T07:29:14.025771+0000","last_became_peered":"2024-03-06T07:29:14.025771+0000","last_unstale":"2024-03-06T07:29:16.034316+0000","last_undegraded":"2024-03-06T07:29:16.034316+0000","last_fullsized":"2024-03-06T07:29:16.034316+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:08.205157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020254+0000","last_change":"2024-03-06T07:29:15.038816+0000","last_active":"2024-03-06T07:29:17.020254+0000","last_peered":"2024-03-06T07:29:17.020254+0000","last_clean":"2024-03-06T07:29:17.020254+0000","last_became_active":"2024-03-06T07:29:15.038487+0000","last_became_peered":"2024-03-06T07:29:15.038487+0000","last_unstale":"2024-03-06T07:29:17.020254+0000","last_undegraded":"2024-03-06T07:29:17.020254+0000","last_fullsized":"2024-03-06T07:29:17.020254+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:50.302554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"211'3","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571265+0000","last_change":"2024-03-06T07:29:14.033683+0000","last_active":"2024-03-06T07:29:16.571265+0000","last_peered":"2024-03-06T07:29:16.571265+0000","last_clean":"2024-03-06T07:29:16.571265+0000","last_became_active":"2024-03-06T07:29:14.033401+0000","last_became_peered":"2024-03-06T07:29:14.033401+0000","last_unstale":"2024-03-06T07:29:16.571265+0000","last_undegraded":"2024-03-06T07:29:16.571265+0000","last_fullsized":"2024-03-06T07:29:16.571265+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:56.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034767+0000","last_change":"2024-03-06T07:29:15.023045+0000","last_active":"2024-03-06T07:29:16.034767+0000","last_peered":"2024-03-06T07:29:16.034767+0000","last_clean":"2024-03-06T07:29:16.034767+0000","last_became_active":"2024-03-06T07:29:15.022709+0000","last_became_peered":"2024-03-06T07:29:15.022709+0000","last_unstale":"2024-03-06T07:29:16.034767+0000","last_undegraded":"2024-03-06T07:29:16.034767+0000","last_fullsized":"2024-03-06T07:29:16.034767+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:15:22.975125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"211'6","reported_seq":20,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017564+0000","last_change":"2024-03-06T07:29:14.031198+0000","last_active":"2024-03-06T07:29:17.017564+0000","last_peered":"2024-03-06T07:29:17.017564+0000","last_clean":"2024-03-06T07:29:17.017564+0000","last_became_active":"2024-03-06T07:29:14.030920+0000","last_became_peered":"2024-03-06T07:29:14.030920+0000","last_unstale":"2024-03-06T07:29:17.017564+0000","last_undegraded":"2024-03-06T07:29:17.017564+0000","last_fullsized":"2024-03-06T07:29:17.017564+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:51.308563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.020014+0000","last_change":"2024-03-06T07:29:15.019023+0000","last_active":"2024-03-06T07:29:15.020014+0000","last_peered":"2024-03-06T07:29:15.020014+0000","last_clean":"2024-03-06T07:29:15.020014+0000","last_became_active":"2024-03-06T07:29:15.018734+0000","last_became_peered":"2024-03-06T07:29:15.018734+0000","last_unstale":"2024-03-06T07:29:15.020014+0000","last_undegraded":"2024-03-06T07:29:15.020014+0000","last_fullsized":"2024-03-06T07:29:15.020014+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:58:55.748350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"211'2","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032015+0000","last_change":"2024-03-06T07:29:14.033296+0000","last_active":"2024-03-06T07:29:15.032015+0000","last_peered":"2024-03-06T07:29:15.032015+0000","last_clean":"2024-03-06T07:29:15.032015+0000","last_became_active":"2024-03-06T07:29:14.032954+0000","last_became_peered":"2024-03-06T07:29:14.032954+0000","last_unstale":"2024-03-06T07:29:15.032015+0000","last_undegraded":"2024-03-06T07:29:15.032015+0000","last_fullsized":"2024-03-06T07:29:15.032015+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:36.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034342+0000","last_change":"2024-03-06T07:29:15.030828+0000","last_active":"2024-03-06T07:29:16.034342+0000","last_peered":"2024-03-06T07:29:16.034342+0000","last_clean":"2024-03-06T07:29:16.034342+0000","last_became_active":"2024-03-06T07:29:15.030563+0000","last_became_peered":"2024-03-06T07:29:15.030563+0000","last_unstale":"2024-03-06T07:29:16.034342+0000","last_undegraded":"2024-03-06T07:29:16.034342+0000","last_fullsized":"2024-03-06T07:29:16.034342+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:26.526550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571158+0000","last_change":"2024-03-06T07:29:14.029215+0000","last_active":"2024-03-06T07:29:16.571158+0000","last_peered":"2024-03-06T07:29:16.571158+0000","last_clean":"2024-03-06T07:29:16.571158+0000","last_became_active":"2024-03-06T07:29:14.028947+0000","last_became_peered":"2024-03-06T07:29:14.028947+0000","last_unstale":"2024-03-06T07:29:16.571158+0000","last_undegraded":"2024-03-06T07:29:16.571158+0000","last_fullsized":"2024-03-06T07:29:16.571158+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:13.969646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031422+0000","last_change":"2024-03-06T07:29:15.023744+0000","last_active":"2024-03-06T07:29:15.031422+0000","last_peered":"2024-03-06T07:29:15.031422+0000","last_clean":"2024-03-06T07:29:15.031422+0000","last_became_active":"2024-03-06T07:29:15.023435+0000","last_became_peered":"2024-03-06T07:29:15.023435+0000","last_unstale":"2024-03-06T07:29:15.031422+0000","last_undegraded":"2024-03-06T07:29:15.031422+0000","last_fullsized":"2024-03-06T07:29:15.031422+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:57.386206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570965+0000","last_change":"2024-03-06T07:29:14.022144+0000","last_active":"2024-03-06T07:29:16.570965+0000","last_peered":"2024-03-06T07:29:16.570965+0000","last_clean":"2024-03-06T07:29:16.570965+0000","last_became_active":"2024-03-06T07:29:14.021872+0000","last_became_peered":"2024-03-06T07:29:14.021872+0000","last_unstale":"2024-03-06T07:29:16.570965+0000","last_undegraded":"2024-03-06T07:29:16.570965+0000","last_fullsized":"2024-03-06T07:29:16.570965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:22:47.392808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.035517+0000","last_change":"2024-03-06T07:29:15.034474+0000","last_active":"2024-03-06T07:29:15.035517+0000","last_peered":"2024-03-06T07:29:15.035517+0000","last_clean":"2024-03-06T07:29:15.035517+0000","last_became_active":"2024-03-06T07:29:15.034202+0000","last_became_peered":"2024-03-06T07:29:15.034202+0000","last_unstale":"2024-03-06T07:29:15.035517+0000","last_undegraded":"2024-03-06T07:29:15.035517+0000","last_fullsized":"2024-03-06T07:29:15.035517+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:39.973578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"211'8","reported_seq":22,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570980+0000","last_change":"2024-03-06T07:29:14.030216+0000","last_active":"2024-03-06T07:29:16.570980+0000","last_peered":"2024-03-06T07:29:16.570980+0000","last_clean":"2024-03-06T07:29:16.570980+0000","last_became_active":"2024-03-06T07:29:14.029939+0000","last_became_peered":"2024-03-06T07:29:14.029939+0000","last_unstale":"2024-03-06T07:29:16.570980+0000","last_undegraded":"2024-03-06T07:29:16.570980+0000","last_fullsized":"2024-03-06T07:29:16.570980+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:38.320550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034485+0000","last_change":"2024-03-06T07:29:15.032073+0000","last_active":"2024-03-06T07:29:16.034485+0000","last_peered":"2024-03-06T07:29:16.034485+0000","last_clean":"2024-03-06T07:29:16.034485+0000","last_became_active":"2024-03-06T07:29:15.031746+0000","last_became_peered":"2024-03-06T07:29:15.031746+0000","last_unstale":"2024-03-06T07:29:16.034485+0000","last_undegraded":"2024-03-06T07:29:16.034485+0000","last_fullsized":"2024-03-06T07:29:16.034485+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:29.353034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020375+0000","last_change":"2024-03-06T07:29:14.031533+0000","last_active":"2024-03-06T07:29:17.020375+0000","last_peered":"2024-03-06T07:29:17.020375+0000","last_clean":"2024-03-06T07:29:17.020375+0000","last_became_active":"2024-03-06T07:29:14.031212+0000","last_became_peered":"2024-03-06T07:29:14.031212+0000","last_unstale":"2024-03-06T07:29:17.020375+0000","last_undegraded":"2024-03-06T07:29:17.020375+0000","last_fullsized":"2024-03-06T07:29:17.020375+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:54:13.830128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034911+0000","last_change":"2024-03-06T07:29:15.028265+0000","last_active":"2024-03-06T07:29:16.034911+0000","last_peered":"2024-03-06T07:29:16.034911+0000","last_clean":"2024-03-06T07:29:16.034911+0000","last_became_active":"2024-03-06T07:29:15.027970+0000","last_became_peered":"2024-03-06T07:29:15.027970+0000","last_unstale":"2024-03-06T07:29:16.034911+0000","last_undegraded":"2024-03-06T07:29:16.034911+0000","last_fullsized":"2024-03-06T07:29:16.034911+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:22.130981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017839+0000","last_change":"2024-03-06T07:29:14.031677+0000","last_active":"2024-03-06T07:29:17.017839+0000","last_peered":"2024-03-06T07:29:17.017839+0000","last_clean":"2024-03-06T07:29:17.017839+0000","last_became_active":"2024-03-06T07:29:14.031373+0000","last_became_peered":"2024-03-06T07:29:14.031373+0000","last_unstale":"2024-03-06T07:29:17.017839+0000","last_undegraded":"2024-03-06T07:29:17.017839+0000","last_fullsized":"2024-03-06T07:29:17.017839+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.074661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571286+0000","last_change":"2024-03-06T07:29:15.044773+0000","last_active":"2024-03-06T07:29:16.571286+0000","last_peered":"2024-03-06T07:29:16.571286+0000","last_clean":"2024-03-06T07:29:16.571286+0000","last_became_active":"2024-03-06T07:29:15.044497+0000","last_became_peered":"2024-03-06T07:29:15.044497+0000","last_unstale":"2024-03-06T07:29:16.571286+0000","last_undegraded":"2024-03-06T07:29:16.571286+0000","last_fullsized":"2024-03-06T07:29:16.571286+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:23.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"211'1","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046990+0000","last_change":"2024-03-06T07:29:14.027538+0000","last_active":"2024-03-06T07:29:15.046990+0000","last_peered":"2024-03-06T07:29:15.046990+0000","last_clean":"2024-03-06T07:29:15.046990+0000","last_became_active":"2024-03-06T07:29:14.027011+0000","last_became_peered":"2024-03-06T07:29:14.027011+0000","last_unstale":"2024-03-06T07:29:15.046990+0000","last_undegraded":"2024-03-06T07:29:15.046990+0000","last_fullsized":"2024-03-06T07:29:15.046990+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:21.126122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034863+0000","last_change":"2024-03-06T07:29:15.033590+0000","last_active":"2024-03-06T07:29:16.034863+0000","last_peered":"2024-03-06T07:29:16.034863+0000","last_clean":"2024-03-06T07:29:16.034863+0000","last_became_active":"2024-03-06T07:29:15.032877+0000","last_became_peered":"2024-03-06T07:29:15.032877+0000","last_unstale":"2024-03-06T07:29:16.034863+0000","last_undegraded":"2024-03-06T07:29:16.034863+0000","last_fullsized":"2024-03-06T07:29:16.034863+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:39:09.512620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017655+0000","last_change":"2024-03-06T07:29:14.031829+0000","last_active":"2024-03-06T07:29:17.017655+0000","last_peered":"2024-03-06T07:29:17.017655+0000","last_clean":"2024-03-06T07:29:17.017655+0000","last_became_active":"2024-03-06T07:29:14.031533+0000","last_became_peered":"2024-03-06T07:29:14.031533+0000","last_unstale":"2024-03-06T07:29:17.017655+0000","last_undegraded":"2024-03-06T07:29:17.017655+0000","last_fullsized":"2024-03-06T07:29:17.017655+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.273579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030712+0000","last_change":"2024-03-06T07:29:15.035684+0000","last_active":"2024-03-06T07:29:16.030712+0000","last_peered":"2024-03-06T07:29:16.030712+0000","last_clean":"2024-03-06T07:29:16.030712+0000","last_became_active":"2024-03-06T07:29:15.035179+0000","last_became_peered":"2024-03-06T07:29:15.035179+0000","last_unstale":"2024-03-06T07:29:16.030712+0000","last_undegraded":"2024-03-06T07:29:16.030712+0000","last_fullsized":"2024-03-06T07:29:16.030712+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:09.954104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"211'2","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.047350+0000","last_change":"2024-03-06T07:29:14.030048+0000","last_active":"2024-03-06T07:29:15.047350+0000","last_peered":"2024-03-06T07:29:15.047350+0000","last_clean":"2024-03-06T07:29:15.047350+0000","last_became_active":"2024-03-06T07:29:14.029389+0000","last_became_peered":"2024-03-06T07:29:14.029389+0000","last_unstale":"2024-03-06T07:29:15.047350+0000","last_undegraded":"2024-03-06T07:29:15.047350+0000","last_fullsized":"2024-03-06T07:29:15.047350+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:56.272946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034505+0000","last_change":"2024-03-06T07:29:15.035454+0000","last_active":"2024-03-06T07:29:16.034505+0000","last_peered":"2024-03-06T07:29:16.034505+0000","last_clean":"2024-03-06T07:29:16.034505+0000","last_became_active":"2024-03-06T07:29:15.035152+0000","last_became_peered":"2024-03-06T07:29:15.035152+0000","last_unstale":"2024-03-06T07:29:16.034505+0000","last_undegraded":"2024-03-06T07:29:16.034505+0000","last_fullsized":"2024-03-06T07:29:16.034505+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:25.370240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.014912+0000","last_change":"2024-03-06T07:29:14.033999+0000","last_active":"2024-03-06T07:29:15.014912+0000","last_peered":"2024-03-06T07:29:15.014912+0000","last_clean":"2024-03-06T07:29:15.014912+0000","last_became_active":"2024-03-06T07:29:14.033721+0000","last_became_peered":"2024-03-06T07:29:14.033721+0000","last_unstale":"2024-03-06T07:29:15.014912+0000","last_undegraded":"2024-03-06T07:29:15.014912+0000","last_fullsized":"2024-03-06T07:29:15.014912+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:35.701125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571120+0000","last_change":"2024-03-06T07:29:15.041084+0000","last_active":"2024-03-06T07:29:16.571120+0000","last_peered":"2024-03-06T07:29:16.571120+0000","last_clean":"2024-03-06T07:29:16.571120+0000","last_became_active":"2024-03-06T07:29:15.040773+0000","last_became_peered":"2024-03-06T07:29:15.040773+0000","last_unstale":"2024-03-06T07:29:16.571120+0000","last_undegraded":"2024-03-06T07:29:16.571120+0000","last_fullsized":"2024-03-06T07:29:16.571120+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:50.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030863+0000","last_change":"2024-03-06T07:29:14.031920+0000","last_active":"2024-03-06T07:29:16.030863+0000","last_peered":"2024-03-06T07:29:16.030863+0000","last_clean":"2024-03-06T07:29:16.030863+0000","last_became_active":"2024-03-06T07:29:14.031588+0000","last_became_peered":"2024-03-06T07:29:14.031588+0000","last_unstale":"2024-03-06T07:29:16.030863+0000","last_undegraded":"2024-03-06T07:29:16.030863+0000","last_fullsized":"2024-03-06T07:29:16.030863+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:49.297694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030650+0000","last_change":"2024-03-06T07:29:15.040031+0000","last_active":"2024-03-06T07:29:16.030650+0000","last_peered":"2024-03-06T07:29:16.030650+0000","last_clean":"2024-03-06T07:29:16.030650+0000","last_became_active":"2024-03-06T07:29:15.039736+0000","last_became_peered":"2024-03-06T07:29:15.039736+0000","last_unstale":"2024-03-06T07:29:16.030650+0000","last_undegraded":"2024-03-06T07:29:16.030650+0000","last_fullsized":"2024-03-06T07:29:16.030650+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:20.232362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570995+0000","last_change":"2024-03-06T07:29:14.021470+0000","last_active":"2024-03-06T07:29:16.570995+0000","last_peered":"2024-03-06T07:29:16.570995+0000","last_clean":"2024-03-06T07:29:16.570995+0000","last_became_active":"2024-03-06T07:29:14.021182+0000","last_became_peered":"2024-03-06T07:29:14.021182+0000","last_unstale":"2024-03-06T07:29:16.570995+0000","last_undegraded":"2024-03-06T07:29:16.570995+0000","last_fullsized":"2024-03-06T07:29:16.570995+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:27.551838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020273+0000","last_change":"2024-03-06T07:29:15.039331+0000","last_active":"2024-03-06T07:29:17.020273+0000","last_peered":"2024-03-06T07:29:17.020273+0000","last_clean":"2024-03-06T07:29:17.020273+0000","last_became_active":"2024-03-06T07:29:15.038714+0000","last_became_peered":"2024-03-06T07:29:15.038714+0000","last_unstale":"2024-03-06T07:29:17.020273+0000","last_undegraded":"2024-03-06T07:29:17.020273+0000","last_fullsized":"2024-03-06T07:29:17.020273+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:17:35.149993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571297+0000","last_change":"2024-03-06T07:29:14.026448+0000","last_active":"2024-03-06T07:29:16.571297+0000","last_peered":"2024-03-06T07:29:16.571297+0000","last_clean":"2024-03-06T07:29:16.571297+0000","last_became_active":"2024-03-06T07:29:14.026082+0000","last_became_peered":"2024-03-06T07:29:14.026082+0000","last_unstale":"2024-03-06T07:29:16.571297+0000","last_undegraded":"2024-03-06T07:29:16.571297+0000","last_fullsized":"2024-03-06T07:29:16.571297+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.523431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020113+0000","last_change":"2024-03-06T07:29:15.035814+0000","last_active":"2024-03-06T07:29:17.020113+0000","last_peered":"2024-03-06T07:29:17.020113+0000","last_clean":"2024-03-06T07:29:17.020113+0000","last_became_active":"2024-03-06T07:29:15.035541+0000","last_became_peered":"2024-03-06T07:29:15.035541+0000","last_unstale":"2024-03-06T07:29:17.020113+0000","last_undegraded":"2024-03-06T07:29:17.020113+0000","last_fullsized":"2024-03-06T07:29:17.020113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:19.801951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"212'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019364+0000","last_change":"2024-03-06T07:29:14.030770+0000","last_active":"2024-03-06T07:29:17.019364+0000","last_peered":"2024-03-06T07:29:17.019364+0000","last_clean":"2024-03-06T07:29:17.019364+0000","last_became_active":"2024-03-06T07:29:14.030460+0000","last_became_peered":"2024-03-06T07:29:14.030460+0000","last_unstale":"2024-03-06T07:29:17.019364+0000","last_undegraded":"2024-03-06T07:29:17.019364+0000","last_fullsized":"2024-03-06T07:29:17.019364+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:22.868851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.035064+0000","last_change":"2024-03-06T07:29:15.033841+0000","last_active":"2024-03-06T07:29:15.035064+0000","last_peered":"2024-03-06T07:29:15.035064+0000","last_clean":"2024-03-06T07:29:15.035064+0000","last_became_active":"2024-03-06T07:29:15.033563+0000","last_became_peered":"2024-03-06T07:29:15.033563+0000","last_unstale":"2024-03-06T07:29:15.035064+0000","last_undegraded":"2024-03-06T07:29:15.035064+0000","last_fullsized":"2024-03-06T07:29:15.035064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:16.903978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024013+0000","last_change":"2024-03-06T07:29:14.030874+0000","last_active":"2024-03-06T07:29:15.024013+0000","last_peered":"2024-03-06T07:29:15.024013+0000","last_clean":"2024-03-06T07:29:15.024013+0000","last_became_active":"2024-03-06T07:29:14.030607+0000","last_became_peered":"2024-03-06T07:29:14.030607+0000","last_unstale":"2024-03-06T07:29:15.024013+0000","last_undegraded":"2024-03-06T07:29:15.024013+0000","last_fullsized":"2024-03-06T07:29:15.024013+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020348+0000","last_change":"2024-03-06T07:29:15.027929+0000","last_active":"2024-03-06T07:29:17.020348+0000","last_peered":"2024-03-06T07:29:17.020348+0000","last_clean":"2024-03-06T07:29:17.020348+0000","last_became_active":"2024-03-06T07:29:15.024578+0000","last_became_peered":"2024-03-06T07:29:15.024578+0000","last_unstale":"2024-03-06T07:29:17.020348+0000","last_undegraded":"2024-03-06T07:29:17.020348+0000","last_fullsized":"2024-03-06T07:29:17.020348+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:20.400622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024749+0000","last_change":"2024-03-06T07:29:14.026706+0000","last_active":"2024-03-06T07:29:15.024749+0000","last_peered":"2024-03-06T07:29:15.024749+0000","last_clean":"2024-03-06T07:29:15.024749+0000","last_became_active":"2024-03-06T07:29:14.026337+0000","last_became_peered":"2024-03-06T07:29:14.026337+0000","last_unstale":"2024-03-06T07:29:15.024749+0000","last_undegraded":"2024-03-06T07:29:15.024749+0000","last_fullsized":"2024-03-06T07:29:15.024749+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:08.205157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570866+0000","last_change":"2024-03-06T07:29:15.040449+0000","last_active":"2024-03-06T07:29:16.570866+0000","last_peered":"2024-03-06T07:29:16.570866+0000","last_clean":"2024-03-06T07:29:16.570866+0000","last_became_active":"2024-03-06T07:29:15.037756+0000","last_became_peered":"2024-03-06T07:29:15.037756+0000","last_unstale":"2024-03-06T07:29:16.570866+0000","last_undegraded":"2024-03-06T07:29:16.570866+0000","last_fullsized":"2024-03-06T07:29:16.570866+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:17.967222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017588+0000","last_change":"2024-03-06T07:29:14.025980+0000","last_active":"2024-03-06T07:29:17.017588+0000","last_peered":"2024-03-06T07:29:17.017588+0000","last_clean":"2024-03-06T07:29:17.017588+0000","last_became_active":"2024-03-06T07:29:14.025697+0000","last_became_peered":"2024-03-06T07:29:14.025697+0000","last_unstale":"2024-03-06T07:29:17.017588+0000","last_undegraded":"2024-03-06T07:29:17.017588+0000","last_fullsized":"2024-03-06T07:29:17.017588+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:26.313123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030871+0000","last_change":"2024-03-06T07:29:15.037674+0000","last_active":"2024-03-06T07:29:16.030871+0000","last_peered":"2024-03-06T07:29:16.030871+0000","last_clean":"2024-03-06T07:29:16.030871+0000","last_became_active":"2024-03-06T07:29:15.036597+0000","last_became_peered":"2024-03-06T07:29:15.036597+0000","last_unstale":"2024-03-06T07:29:16.030871+0000","last_undegraded":"2024-03-06T07:29:16.030871+0000","last_fullsized":"2024-03-06T07:29:16.030871+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:18.023292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017595+0000","last_change":"2024-03-06T07:29:14.032273+0000","last_active":"2024-03-06T07:29:17.017595+0000","last_peered":"2024-03-06T07:29:17.017595+0000","last_clean":"2024-03-06T07:29:17.017595+0000","last_became_active":"2024-03-06T07:29:14.032006+0000","last_became_peered":"2024-03-06T07:29:14.032006+0000","last_unstale":"2024-03-06T07:29:17.017595+0000","last_undegraded":"2024-03-06T07:29:17.017595+0000","last_fullsized":"2024-03-06T07:29:17.017595+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:59.794344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031136+0000","last_change":"2024-03-06T07:29:15.036506+0000","last_active":"2024-03-06T07:29:16.031136+0000","last_peered":"2024-03-06T07:29:16.031136+0000","last_clean":"2024-03-06T07:29:16.031136+0000","last_became_active":"2024-03-06T07:29:15.036231+0000","last_became_peered":"2024-03-06T07:29:15.036231+0000","last_unstale":"2024-03-06T07:29:16.031136+0000","last_undegraded":"2024-03-06T07:29:16.031136+0000","last_fullsized":"2024-03-06T07:29:16.031136+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:47.183696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.025129+0000","last_change":"2024-03-06T07:29:14.026703+0000","last_active":"2024-03-06T07:29:15.025129+0000","last_peered":"2024-03-06T07:29:15.025129+0000","last_clean":"2024-03-06T07:29:15.025129+0000","last_became_active":"2024-03-06T07:29:14.026373+0000","last_became_peered":"2024-03-06T07:29:14.026373+0000","last_unstale":"2024-03-06T07:29:15.025129+0000","last_undegraded":"2024-03-06T07:29:15.025129+0000","last_fullsized":"2024-03-06T07:29:15.025129+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:13.368742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034044+0000","last_change":"2024-03-06T07:29:15.037909+0000","last_active":"2024-03-06T07:29:16.034044+0000","last_peered":"2024-03-06T07:29:16.034044+0000","last_clean":"2024-03-06T07:29:16.034044+0000","last_became_active":"2024-03-06T07:29:15.037534+0000","last_became_peered":"2024-03-06T07:29:15.037534+0000","last_unstale":"2024-03-06T07:29:16.034044+0000","last_undegraded":"2024-03-06T07:29:16.034044+0000","last_fullsized":"2024-03-06T07:29:16.034044+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:26:30.847956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017633+0000","last_change":"2024-03-06T07:29:14.026763+0000","last_active":"2024-03-06T07:29:17.017633+0000","last_peered":"2024-03-06T07:29:17.017633+0000","last_clean":"2024-03-06T07:29:17.017633+0000","last_became_active":"2024-03-06T07:29:14.026491+0000","last_became_peered":"2024-03-06T07:29:14.026491+0000","last_unstale":"2024-03-06T07:29:17.017633+0000","last_undegraded":"2024-03-06T07:29:17.017633+0000","last_fullsized":"2024-03-06T07:29:17.017633+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:05.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571316+0000","last_change":"2024-03-06T07:29:15.036098+0000","last_active":"2024-03-06T07:29:16.571316+0000","last_peered":"2024-03-06T07:29:16.571316+0000","last_clean":"2024-03-06T07:29:16.571316+0000","last_became_active":"2024-03-06T07:29:15.035774+0000","last_became_peered":"2024-03-06T07:29:15.035774+0000","last_unstale":"2024-03-06T07:29:16.571316+0000","last_undegraded":"2024-03-06T07:29:16.571316+0000","last_fullsized":"2024-03-06T07:29:16.571316+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:01.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":527,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019809+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:29:17.019809+0000","last_peered":"2024-03-06T07:29:17.019809+0000","last_clean":"2024-03-06T07:29:17.019809+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:29:17.019809+0000","last_undegraded":"2024-03-06T07:29:17.019809+0000","last_fullsized":"2024-03-06T07:29:17.019809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33872,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337968,"statfs":{"total":96636764160,"available":96602079232,"internally_reserved":0,"allocated":2752512,"data_stored":1895697,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2695168,"data_stored":1311451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33808,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338032,"statfs":{"total":96636764160,"available":96602144768,"internally_reserved":0,"allocated":2682880,"data_stored":1304795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2703360,"data_stored":1313029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2695168,"data_stored":1311507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2699264,"data_stored":1306941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37964,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333876,"statfs":{"total":96636764160,"available":96597889024,"internally_reserved":0,"allocated":2686976,"data_stored":1304849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38040,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333800,"statfs":{"total":96636764160,"available":96597811200,"internally_reserved":0,"allocated":2760704,"data_stored":1901855,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:29:18.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:29:18.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.266+0000 7f1972e88640 1 Processor -- start 2024-03-06T07:29:18.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 -- start start 2024-03-06T07:29:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f196c0c88c0 0x7f196c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0c73d0 con 0x7f196c14f360 2024-03-06T07:29:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c0c8dd0 con 0x7f196c0c88c0 2024-03-06T07:29:18.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.267+0000 7f1972e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0be2e0 con 0x7f196c158b60 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:44044/0 (socket says 172.21.15.17:44044) 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f196bfff640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f196c0c88c0 0x7f196c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 -- 172.21.15.17:0/61887530 learned_addr learned my addr 172.21.15.17:0/61887530 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 -- 172.21.15.17:0/61887530 >> v2:172.21.15.163:3300/0 conn(0x7f196c0c88c0 msgr2=0x7f196c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 --2- 172.21.15.17:0/61887530 >> v2:172.21.15.163:3300/0 conn(0x7f196c0c88c0 0x7f196c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 -- 172.21.15.17:0/61887530 >> v1:172.21.15.163:6789/0 conn(0x7f196c14f360 legacy=0x7f196c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.268+0000 7f1971425640 1 -- 172.21.15.17:0/61887530 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0c7670 con 0x7f196c158b60 2024-03-06T07:29:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.269+0000 7f1971425640 1 --2- 172.21.15.17:0/61887530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1964009e50 tx=0x7f196402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7976cfdee8b8f84c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.269+0000 7f196b7fe640 1 -- 172.21.15.17:0/61887530 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196402ca70 con 0x7f196c158b60 2024-03-06T07:29:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.269+0000 7f196b7fe640 1 -- 172.21.15.17:0/61887530 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196402cc10 con 0x7f196c158b60 2024-03-06T07:29:18.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.269+0000 7f196b7fe640 1 -- 172.21.15.17:0/61887530 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640343f0 con 0x7f196c158b60 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 -- 172.21.15.17:0/61887530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 msgr2=0x7f196c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 --2- 172.21.15.17:0/61887530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1964009e50 tx=0x7f196402bed0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 -- 172.21.15.17:0/61887530 shutdown_connections 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 --2- 172.21.15.17:0/61887530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c158b60 0x7f196c158ff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 --2- 172.21.15.17:0/61887530 >> v2:172.21.15.163:3300/0 conn(0x7f196c0c88c0 0x7f196c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 -- 172.21.15.17:0/61887530 >> 172.21.15.17:0/61887530 conn(0x7f196c0ac570 msgr2=0x7f196c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 -- 172.21.15.17:0/61887530 shutdown_connections 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.270+0000 7f1972e88640 1 -- 172.21.15.17:0/61887530 wait complete. 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.271+0000 7f1972e88640 1 Processor -- start 2024-03-06T07:29:18.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.271+0000 7f1972e88640 1 -- start start 2024-03-06T07:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c0c88c0 0x7f196c0c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c158b60 0x7f196c14bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c5a50 con 0x7f196c14f360 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c7c30 con 0x7f196c158b60 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c5bd0 con 0x7f196c0c88c0 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47434/0 (socket says 172.21.15.17:47434) 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 -- 172.21.15.17:0/2673739311 learned_addr learned my addr 172.21.15.17:0/2673739311 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1971425640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c158b60 0x7f196c14bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1970c24640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c0c88c0 0x7f196c0c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 -- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c0c88c0 msgr2=0x7f196c0c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c0c88c0 0x7f196c0c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 -- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c158b60 msgr2=0x7f196c14bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c158b60 0x7f196c14bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 -- 172.21.15.17:0/2673739311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c1523b0 con 0x7f196c14f360 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f196bfff640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f195800e870 tx=0x7f195800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958017070 con 0x7f196c14f360 2024-03-06T07:29:18.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195800c940 con 0x7f196c14f360 2024-03-06T07:29:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c14f870 con 0x7f196c14f360 2024-03-06T07:29:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195800cd10 con 0x7f196c14f360 2024-03-06T07:29:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.272+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c0bff70 con 0x7f196c14f360 2024-03-06T07:29:18.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.274+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c06cb00 con 0x7f196c14f360 2024-03-06T07:29:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.274+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f195801a070 con 0x7f196c14f360 2024-03-06T07:29:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.274+0000 7f19697fa640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.274+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f195808a180 con 0x7f196c14f360 2024-03-06T07:29:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.275+0000 7f1970c24640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.275+0000 7f1970c24640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f19600016e0 tx=0x7f1960009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.279+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f196c06cb00 con 0x7f196c14f360 2024-03-06T07:29:18.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.627+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f196c0bfd10 con 0x7f195006b500 2024-03-06T07:29:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.633+0000 7f19697fa640 1 -- 172.21.15.17:0/2673739311 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252562 (secure 0 0 0) 0x7f1960001cc0 con 0x7f195006b500 2024-03-06T07:29:18.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:29:18.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 msgr2=0x7f195006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f19600016e0 tx=0x7f1960009040 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 msgr2=0x7f196c14b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f195800e870 tx=0x7f195800ed30 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 shutdown_connections 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f195006b500 0x7f195006d9c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f196c158b60 0x7f196c14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f196c14f360 0x7f196c14b9c0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 --2- 172.21.15.17:0/2673739311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f196c0c88c0 0x7f196c0c07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 >> 172.21.15.17:0/2673739311 conn(0x7f196c0ac570 msgr2=0x7f196c06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 shutdown_connections 2024-03-06T07:29:18.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.636+0000 7f1972e88640 1 -- 172.21.15.17:0/2673739311 wait complete. 2024-03-06T07:29:18.651 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-03-06T07:29:17.468105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":283116,"kb_used_data":21168,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754691604,"statfs":{"total":773094113280,"available":772804202496,"internally_reserved":0,"allocated":21676032,"data_stored":11650124,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":268095028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.461740"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034707+0000","last_change":"2024-03-06T07:29:15.033803+0000","last_active":"2024-03-06T07:29:16.034707+0000","last_peered":"2024-03-06T07:29:16.034707+0000","last_clean":"2024-03-06T07:29:16.034707+0000","last_became_active":"2024-03-06T07:29:15.033282+0000","last_became_peered":"2024-03-06T07:29:15.033282+0000","last_unstale":"2024-03-06T07:29:16.034707+0000","last_undegraded":"2024-03-06T07:29:16.034707+0000","last_fullsized":"2024-03-06T07:29:16.034707+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:51:44.103061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030891+0000","last_change":"2024-03-06T07:29:15.040115+0000","last_active":"2024-03-06T07:29:16.030891+0000","last_peered":"2024-03-06T07:29:16.030891+0000","last_clean":"2024-03-06T07:29:16.030891+0000","last_became_active":"2024-03-06T07:29:15.039740+0000","last_became_peered":"2024-03-06T07:29:15.039740+0000","last_unstale":"2024-03-06T07:29:16.030891+0000","last_undegraded":"2024-03-06T07:29:16.030891+0000","last_fullsized":"2024-03-06T07:29:16.030891+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:49:24.207873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.039684+0000","last_change":"2024-03-06T07:29:15.035672+0000","last_active":"2024-03-06T07:29:15.039684+0000","last_peered":"2024-03-06T07:29:15.039684+0000","last_clean":"2024-03-06T07:29:15.039684+0000","last_became_active":"2024-03-06T07:29:15.035401+0000","last_became_peered":"2024-03-06T07:29:15.035401+0000","last_unstale":"2024-03-06T07:29:15.039684+0000","last_undegraded":"2024-03-06T07:29:15.039684+0000","last_fullsized":"2024-03-06T07:29:15.039684+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:54:14.834987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571236+0000","last_change":"2024-03-06T07:29:15.033488+0000","last_active":"2024-03-06T07:29:16.571236+0000","last_peered":"2024-03-06T07:29:16.571236+0000","last_clean":"2024-03-06T07:29:16.571236+0000","last_became_active":"2024-03-06T07:29:15.033098+0000","last_became_peered":"2024-03-06T07:29:15.033098+0000","last_unstale":"2024-03-06T07:29:16.571236+0000","last_undegraded":"2024-03-06T07:29:16.571236+0000","last_fullsized":"2024-03-06T07:29:16.571236+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:55.521589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.043594+0000","last_change":"2024-03-06T07:29:15.042500+0000","last_active":"2024-03-06T07:29:15.043594+0000","last_peered":"2024-03-06T07:29:15.043594+0000","last_clean":"2024-03-06T07:29:15.043594+0000","last_became_active":"2024-03-06T07:29:15.042254+0000","last_became_peered":"2024-03-06T07:29:15.042254+0000","last_unstale":"2024-03-06T07:29:15.043594+0000","last_undegraded":"2024-03-06T07:29:15.043594+0000","last_fullsized":"2024-03-06T07:29:15.043594+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:53:43.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019819+0000","last_change":"2024-03-06T07:29:15.038746+0000","last_active":"2024-03-06T07:29:17.019819+0000","last_peered":"2024-03-06T07:29:17.019819+0000","last_clean":"2024-03-06T07:29:17.019819+0000","last_became_active":"2024-03-06T07:29:15.038349+0000","last_became_peered":"2024-03-06T07:29:15.038349+0000","last_unstale":"2024-03-06T07:29:17.019819+0000","last_undegraded":"2024-03-06T07:29:17.019819+0000","last_fullsized":"2024-03-06T07:29:17.019819+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:36.370419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.029918+0000","last_change":"2024-03-06T07:29:15.025643+0000","last_active":"2024-03-06T07:29:15.029918+0000","last_peered":"2024-03-06T07:29:15.029918+0000","last_clean":"2024-03-06T07:29:15.029918+0000","last_became_active":"2024-03-06T07:29:15.025375+0000","last_became_peered":"2024-03-06T07:29:15.025375+0000","last_unstale":"2024-03-06T07:29:15.029918+0000","last_undegraded":"2024-03-06T07:29:15.029918+0000","last_fullsized":"2024-03-06T07:29:15.029918+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:22:56.916547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570869+0000","last_change":"2024-03-06T07:29:15.036351+0000","last_active":"2024-03-06T07:29:16.570869+0000","last_peered":"2024-03-06T07:29:16.570869+0000","last_clean":"2024-03-06T07:29:16.570869+0000","last_became_active":"2024-03-06T07:29:15.035792+0000","last_became_peered":"2024-03-06T07:29:15.035792+0000","last_unstale":"2024-03-06T07:29:16.570869+0000","last_undegraded":"2024-03-06T07:29:16.570869+0000","last_fullsized":"2024-03-06T07:29:16.570869+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:19:33.416598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020308+0000","last_change":"2024-03-06T07:29:15.034693+0000","last_active":"2024-03-06T07:29:17.020308+0000","last_peered":"2024-03-06T07:29:17.020308+0000","last_clean":"2024-03-06T07:29:17.020308+0000","last_became_active":"2024-03-06T07:29:15.034382+0000","last_became_peered":"2024-03-06T07:29:15.034382+0000","last_unstale":"2024-03-06T07:29:17.020308+0000","last_undegraded":"2024-03-06T07:29:17.020308+0000","last_fullsized":"2024-03-06T07:29:17.020308+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:38:19.609882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034424+0000","last_change":"2024-03-06T07:29:15.027164+0000","last_active":"2024-03-06T07:29:16.034424+0000","last_peered":"2024-03-06T07:29:16.034424+0000","last_clean":"2024-03-06T07:29:16.034424+0000","last_became_active":"2024-03-06T07:29:15.025973+0000","last_became_peered":"2024-03-06T07:29:15.025973+0000","last_unstale":"2024-03-06T07:29:16.034424+0000","last_undegraded":"2024-03-06T07:29:16.034424+0000","last_fullsized":"2024-03-06T07:29:16.034424+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:12:26.952702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034747+0000","last_change":"2024-03-06T07:29:15.026581+0000","last_active":"2024-03-06T07:29:16.034747+0000","last_peered":"2024-03-06T07:29:16.034747+0000","last_clean":"2024-03-06T07:29:16.034747+0000","last_became_active":"2024-03-06T07:29:15.026304+0000","last_became_peered":"2024-03-06T07:29:15.026304+0000","last_unstale":"2024-03-06T07:29:16.034747+0000","last_undegraded":"2024-03-06T07:29:16.034747+0000","last_fullsized":"2024-03-06T07:29:16.034747+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:07:45.420192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020105+0000","last_change":"2024-03-06T07:29:15.031760+0000","last_active":"2024-03-06T07:29:17.020105+0000","last_peered":"2024-03-06T07:29:17.020105+0000","last_clean":"2024-03-06T07:29:17.020105+0000","last_became_active":"2024-03-06T07:29:15.030268+0000","last_became_peered":"2024-03-06T07:29:15.030268+0000","last_unstale":"2024-03-06T07:29:17.020105+0000","last_undegraded":"2024-03-06T07:29:17.020105+0000","last_fullsized":"2024-03-06T07:29:17.020105+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:08.244634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571287+0000","last_change":"2024-03-06T07:29:15.039963+0000","last_active":"2024-03-06T07:29:16.571287+0000","last_peered":"2024-03-06T07:29:16.571287+0000","last_clean":"2024-03-06T07:29:16.571287+0000","last_became_active":"2024-03-06T07:29:15.039690+0000","last_became_peered":"2024-03-06T07:29:15.039690+0000","last_unstale":"2024-03-06T07:29:16.571287+0000","last_undegraded":"2024-03-06T07:29:16.571287+0000","last_fullsized":"2024-03-06T07:29:16.571287+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:47:52.397315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030823+0000","last_change":"2024-03-06T07:29:15.044872+0000","last_active":"2024-03-06T07:29:16.030823+0000","last_peered":"2024-03-06T07:29:16.030823+0000","last_clean":"2024-03-06T07:29:16.030823+0000","last_became_active":"2024-03-06T07:29:15.044362+0000","last_became_peered":"2024-03-06T07:29:15.044362+0000","last_unstale":"2024-03-06T07:29:16.030823+0000","last_undegraded":"2024-03-06T07:29:16.030823+0000","last_fullsized":"2024-03-06T07:29:16.030823+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:21:22.901814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030600+0000","last_change":"2024-03-06T07:29:15.040549+0000","last_active":"2024-03-06T07:29:16.030600+0000","last_peered":"2024-03-06T07:29:16.030600+0000","last_clean":"2024-03-06T07:29:16.030600+0000","last_became_active":"2024-03-06T07:29:15.040240+0000","last_became_peered":"2024-03-06T07:29:15.040240+0000","last_unstale":"2024-03-06T07:29:16.030600+0000","last_undegraded":"2024-03-06T07:29:16.030600+0000","last_fullsized":"2024-03-06T07:29:16.030600+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:20:58.499423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034310+0000","last_change":"2024-03-06T07:29:15.030886+0000","last_active":"2024-03-06T07:29:16.034310+0000","last_peered":"2024-03-06T07:29:16.034310+0000","last_clean":"2024-03-06T07:29:16.034310+0000","last_became_active":"2024-03-06T07:29:15.030592+0000","last_became_peered":"2024-03-06T07:29:15.030592+0000","last_unstale":"2024-03-06T07:29:16.034310+0000","last_undegraded":"2024-03-06T07:29:16.034310+0000","last_fullsized":"2024-03-06T07:29:16.034310+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:03:55.325606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034350+0000","last_change":"2024-03-06T07:29:15.037166+0000","last_active":"2024-03-06T07:29:16.034350+0000","last_peered":"2024-03-06T07:29:16.034350+0000","last_clean":"2024-03-06T07:29:16.034350+0000","last_became_active":"2024-03-06T07:29:15.036848+0000","last_became_peered":"2024-03-06T07:29:15.036848+0000","last_unstale":"2024-03-06T07:29:16.034350+0000","last_undegraded":"2024-03-06T07:29:16.034350+0000","last_fullsized":"2024-03-06T07:29:16.034350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:52.019571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017775+0000","last_change":"2024-03-06T07:29:15.037532+0000","last_active":"2024-03-06T07:29:17.017775+0000","last_peered":"2024-03-06T07:29:17.017775+0000","last_clean":"2024-03-06T07:29:17.017775+0000","last_became_active":"2024-03-06T07:29:15.037030+0000","last_became_peered":"2024-03-06T07:29:15.037030+0000","last_unstale":"2024-03-06T07:29:17.017775+0000","last_undegraded":"2024-03-06T07:29:17.017775+0000","last_fullsized":"2024-03-06T07:29:17.017775+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:10:11.609877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033745+0000","last_change":"2024-03-06T07:29:15.020745+0000","last_active":"2024-03-06T07:29:16.033745+0000","last_peered":"2024-03-06T07:29:16.033745+0000","last_clean":"2024-03-06T07:29:16.033745+0000","last_became_active":"2024-03-06T07:29:15.020457+0000","last_became_peered":"2024-03-06T07:29:15.020457+0000","last_unstale":"2024-03-06T07:29:16.033745+0000","last_undegraded":"2024-03-06T07:29:16.033745+0000","last_fullsized":"2024-03-06T07:29:16.033745+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:26:15.926071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020108+0000","last_change":"2024-03-06T07:29:15.038239+0000","last_active":"2024-03-06T07:29:17.020108+0000","last_peered":"2024-03-06T07:29:17.020108+0000","last_clean":"2024-03-06T07:29:17.020108+0000","last_became_active":"2024-03-06T07:29:15.037893+0000","last_became_peered":"2024-03-06T07:29:15.037893+0000","last_unstale":"2024-03-06T07:29:17.020108+0000","last_undegraded":"2024-03-06T07:29:17.020108+0000","last_fullsized":"2024-03-06T07:29:17.020108+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:09:37.781015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042659+0000","last_change":"2024-03-06T07:29:15.041455+0000","last_active":"2024-03-06T07:29:15.042659+0000","last_peered":"2024-03-06T07:29:15.042659+0000","last_clean":"2024-03-06T07:29:15.042659+0000","last_became_active":"2024-03-06T07:29:15.041182+0000","last_became_peered":"2024-03-06T07:29:15.041182+0000","last_unstale":"2024-03-06T07:29:15.042659+0000","last_undegraded":"2024-03-06T07:29:15.042659+0000","last_fullsized":"2024-03-06T07:29:15.042659+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:56.578344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033944+0000","last_change":"2024-03-06T07:29:15.031960+0000","last_active":"2024-03-06T07:29:16.033944+0000","last_peered":"2024-03-06T07:29:16.033944+0000","last_clean":"2024-03-06T07:29:16.033944+0000","last_became_active":"2024-03-06T07:29:15.030103+0000","last_became_peered":"2024-03-06T07:29:15.030103+0000","last_unstale":"2024-03-06T07:29:16.033944+0000","last_undegraded":"2024-03-06T07:29:16.033944+0000","last_fullsized":"2024-03-06T07:29:16.033944+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:46:11.820083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030556+0000","last_change":"2024-03-06T07:29:15.036298+0000","last_active":"2024-03-06T07:29:16.030556+0000","last_peered":"2024-03-06T07:29:16.030556+0000","last_clean":"2024-03-06T07:29:16.030556+0000","last_became_active":"2024-03-06T07:29:15.035470+0000","last_became_peered":"2024-03-06T07:29:15.035470+0000","last_unstale":"2024-03-06T07:29:16.030556+0000","last_undegraded":"2024-03-06T07:29:16.030556+0000","last_fullsized":"2024-03-06T07:29:16.030556+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:40:53.111160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034044+0000","last_change":"2024-03-06T07:29:15.028230+0000","last_active":"2024-03-06T07:29:16.034044+0000","last_peered":"2024-03-06T07:29:16.034044+0000","last_clean":"2024-03-06T07:29:16.034044+0000","last_became_active":"2024-03-06T07:29:15.027926+0000","last_became_peered":"2024-03-06T07:29:15.027926+0000","last_unstale":"2024-03-06T07:29:16.034044+0000","last_undegraded":"2024-03-06T07:29:16.034044+0000","last_fullsized":"2024-03-06T07:29:16.034044+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:44:57.436525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.042531+0000","last_change":"2024-03-06T07:29:15.038176+0000","last_active":"2024-03-06T07:29:15.042531+0000","last_peered":"2024-03-06T07:29:15.042531+0000","last_clean":"2024-03-06T07:29:15.042531+0000","last_became_active":"2024-03-06T07:29:15.037908+0000","last_became_peered":"2024-03-06T07:29:15.037908+0000","last_unstale":"2024-03-06T07:29:15.042531+0000","last_undegraded":"2024-03-06T07:29:15.042531+0000","last_fullsized":"2024-03-06T07:29:15.042531+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:54:59.988732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031017+0000","last_change":"2024-03-06T07:29:15.032343+0000","last_active":"2024-03-06T07:29:16.031017+0000","last_peered":"2024-03-06T07:29:16.031017+0000","last_clean":"2024-03-06T07:29:16.031017+0000","last_became_active":"2024-03-06T07:29:15.032060+0000","last_became_peered":"2024-03-06T07:29:15.032060+0000","last_unstale":"2024-03-06T07:29:16.031017+0000","last_undegraded":"2024-03-06T07:29:16.031017+0000","last_fullsized":"2024-03-06T07:29:16.031017+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:59:45.267220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034795+0000","last_change":"2024-03-06T07:29:15.032781+0000","last_active":"2024-03-06T07:29:16.034795+0000","last_peered":"2024-03-06T07:29:16.034795+0000","last_clean":"2024-03-06T07:29:16.034795+0000","last_became_active":"2024-03-06T07:29:15.031276+0000","last_became_peered":"2024-03-06T07:29:15.031276+0000","last_unstale":"2024-03-06T07:29:16.034795+0000","last_undegraded":"2024-03-06T07:29:16.034795+0000","last_fullsized":"2024-03-06T07:29:16.034795+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:24:18.448892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034308+0000","last_change":"2024-03-06T07:29:15.031915+0000","last_active":"2024-03-06T07:29:16.034308+0000","last_peered":"2024-03-06T07:29:16.034308+0000","last_clean":"2024-03-06T07:29:16.034308+0000","last_became_active":"2024-03-06T07:29:15.031381+0000","last_became_peered":"2024-03-06T07:29:15.031381+0000","last_unstale":"2024-03-06T07:29:16.034308+0000","last_undegraded":"2024-03-06T07:29:16.034308+0000","last_fullsized":"2024-03-06T07:29:16.034308+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:56:39.084230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019898+0000","last_change":"2024-03-06T07:29:15.039048+0000","last_active":"2024-03-06T07:29:17.019898+0000","last_peered":"2024-03-06T07:29:17.019898+0000","last_clean":"2024-03-06T07:29:17.019898+0000","last_became_active":"2024-03-06T07:29:15.038533+0000","last_became_peered":"2024-03-06T07:29:15.038533+0000","last_unstale":"2024-03-06T07:29:17.019898+0000","last_undegraded":"2024-03-06T07:29:17.019898+0000","last_fullsized":"2024-03-06T07:29:17.019898+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:43.165397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030713+0000","last_change":"2024-03-06T07:29:15.038354+0000","last_active":"2024-03-06T07:29:16.030713+0000","last_peered":"2024-03-06T07:29:16.030713+0000","last_clean":"2024-03-06T07:29:16.030713+0000","last_became_active":"2024-03-06T07:29:15.037386+0000","last_became_peered":"2024-03-06T07:29:15.037386+0000","last_unstale":"2024-03-06T07:29:16.030713+0000","last_undegraded":"2024-03-06T07:29:16.030713+0000","last_fullsized":"2024-03-06T07:29:16.030713+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:55:39.820037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030448+0000","last_change":"2024-03-06T07:29:15.039364+0000","last_active":"2024-03-06T07:29:16.030448+0000","last_peered":"2024-03-06T07:29:16.030448+0000","last_clean":"2024-03-06T07:29:16.030448+0000","last_became_active":"2024-03-06T07:29:15.039092+0000","last_became_peered":"2024-03-06T07:29:15.039092+0000","last_unstale":"2024-03-06T07:29:16.030448+0000","last_undegraded":"2024-03-06T07:29:16.030448+0000","last_fullsized":"2024-03-06T07:29:16.030448+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:23:56.578344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033857+0000","last_change":"2024-03-06T07:29:15.027471+0000","last_active":"2024-03-06T07:29:16.033857+0000","last_peered":"2024-03-06T07:29:16.033857+0000","last_clean":"2024-03-06T07:29:16.033857+0000","last_became_active":"2024-03-06T07:29:15.027170+0000","last_became_peered":"2024-03-06T07:29:15.027170+0000","last_unstale":"2024-03-06T07:29:16.033857+0000","last_undegraded":"2024-03-06T07:29:16.033857+0000","last_fullsized":"2024-03-06T07:29:16.033857+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:57:50.867428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"211'4","reported_seq":18,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017769+0000","last_change":"2024-03-06T07:29:14.032355+0000","last_active":"2024-03-06T07:29:17.017769+0000","last_peered":"2024-03-06T07:29:17.017769+0000","last_clean":"2024-03-06T07:29:17.017769+0000","last_became_active":"2024-03-06T07:29:14.032092+0000","last_became_peered":"2024-03-06T07:29:14.032092+0000","last_unstale":"2024-03-06T07:29:17.017769+0000","last_undegraded":"2024-03-06T07:29:17.017769+0000","last_fullsized":"2024-03-06T07:29:17.017769+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:32:30.483240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017680+0000","last_change":"2024-03-06T07:29:15.043032+0000","last_active":"2024-03-06T07:29:17.017680+0000","last_peered":"2024-03-06T07:29:17.017680+0000","last_clean":"2024-03-06T07:29:17.017680+0000","last_became_active":"2024-03-06T07:29:15.042762+0000","last_became_peered":"2024-03-06T07:29:15.042762+0000","last_unstale":"2024-03-06T07:29:17.017680+0000","last_undegraded":"2024-03-06T07:29:17.017680+0000","last_fullsized":"2024-03-06T07:29:17.017680+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:07:00.291023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034209+0000","last_change":"2024-03-06T07:29:14.020709+0000","last_active":"2024-03-06T07:29:16.034209+0000","last_peered":"2024-03-06T07:29:16.034209+0000","last_clean":"2024-03-06T07:29:16.034209+0000","last_became_active":"2024-03-06T07:29:14.020430+0000","last_became_peered":"2024-03-06T07:29:14.020430+0000","last_unstale":"2024-03-06T07:29:16.034209+0000","last_undegraded":"2024-03-06T07:29:16.034209+0000","last_fullsized":"2024-03-06T07:29:16.034209+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017771+0000","last_change":"2024-03-06T07:29:15.038256+0000","last_active":"2024-03-06T07:29:17.017771+0000","last_peered":"2024-03-06T07:29:17.017771+0000","last_clean":"2024-03-06T07:29:17.017771+0000","last_became_active":"2024-03-06T07:29:15.037847+0000","last_became_peered":"2024-03-06T07:29:15.037847+0000","last_unstale":"2024-03-06T07:29:17.017771+0000","last_undegraded":"2024-03-06T07:29:17.017771+0000","last_fullsized":"2024-03-06T07:29:17.017771+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:13:30.072150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034929+0000","last_change":"2024-03-06T07:29:14.031928+0000","last_active":"2024-03-06T07:29:16.034929+0000","last_peered":"2024-03-06T07:29:16.034929+0000","last_clean":"2024-03-06T07:29:16.034929+0000","last_became_active":"2024-03-06T07:29:14.031631+0000","last_became_peered":"2024-03-06T07:29:14.031631+0000","last_unstale":"2024-03-06T07:29:16.034929+0000","last_undegraded":"2024-03-06T07:29:16.034929+0000","last_fullsized":"2024-03-06T07:29:16.034929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T17:37:15.530531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031193+0000","last_change":"2024-03-06T07:29:15.033634+0000","last_active":"2024-03-06T07:29:16.031193+0000","last_peered":"2024-03-06T07:29:16.031193+0000","last_clean":"2024-03-06T07:29:16.031193+0000","last_became_active":"2024-03-06T07:29:15.033375+0000","last_became_peered":"2024-03-06T07:29:15.033375+0000","last_unstale":"2024-03-06T07:29:16.031193+0000","last_undegraded":"2024-03-06T07:29:16.031193+0000","last_fullsized":"2024-03-06T07:29:16.031193+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:47.983876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034072+0000","last_change":"2024-03-06T07:29:14.034963+0000","last_active":"2024-03-06T07:29:16.034072+0000","last_peered":"2024-03-06T07:29:16.034072+0000","last_clean":"2024-03-06T07:29:16.034072+0000","last_became_active":"2024-03-06T07:29:14.034677+0000","last_became_peered":"2024-03-06T07:29:14.034677+0000","last_unstale":"2024-03-06T07:29:16.034072+0000","last_undegraded":"2024-03-06T07:29:16.034072+0000","last_fullsized":"2024-03-06T07:29:16.034072+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:13.368742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030506+0000","last_change":"2024-03-06T07:29:15.035450+0000","last_active":"2024-03-06T07:29:16.030506+0000","last_peered":"2024-03-06T07:29:16.030506+0000","last_clean":"2024-03-06T07:29:16.030506+0000","last_became_active":"2024-03-06T07:29:15.034828+0000","last_became_peered":"2024-03-06T07:29:15.034828+0000","last_unstale":"2024-03-06T07:29:16.030506+0000","last_undegraded":"2024-03-06T07:29:16.030506+0000","last_fullsized":"2024-03-06T07:29:16.030506+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:53:43.369926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"211'1","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.038565+0000","last_change":"2024-03-06T07:29:14.031147+0000","last_active":"2024-03-06T07:29:15.038565+0000","last_peered":"2024-03-06T07:29:15.038565+0000","last_clean":"2024-03-06T07:29:15.038565+0000","last_became_active":"2024-03-06T07:29:14.029748+0000","last_became_peered":"2024-03-06T07:29:14.029748+0000","last_unstale":"2024-03-06T07:29:15.038565+0000","last_undegraded":"2024-03-06T07:29:15.038565+0000","last_fullsized":"2024-03-06T07:29:15.038565+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:05:11.831935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030755+0000","last_change":"2024-03-06T07:29:15.029842+0000","last_active":"2024-03-06T07:29:16.030755+0000","last_peered":"2024-03-06T07:29:16.030755+0000","last_clean":"2024-03-06T07:29:16.030755+0000","last_became_active":"2024-03-06T07:29:15.029540+0000","last_became_peered":"2024-03-06T07:29:15.029540+0000","last_unstale":"2024-03-06T07:29:16.030755+0000","last_undegraded":"2024-03-06T07:29:16.030755+0000","last_fullsized":"2024-03-06T07:29:16.030755+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:25:58.246941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031054+0000","last_change":"2024-03-06T07:29:14.022007+0000","last_active":"2024-03-06T07:29:16.031054+0000","last_peered":"2024-03-06T07:29:16.031054+0000","last_clean":"2024-03-06T07:29:16.031054+0000","last_became_active":"2024-03-06T07:29:14.021716+0000","last_became_peered":"2024-03-06T07:29:14.021716+0000","last_unstale":"2024-03-06T07:29:16.031054+0000","last_undegraded":"2024-03-06T07:29:16.031054+0000","last_fullsized":"2024-03-06T07:29:16.031054+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:33:07.239775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.034989+0000","last_change":"2024-03-06T07:29:15.032188+0000","last_active":"2024-03-06T07:29:15.034989+0000","last_peered":"2024-03-06T07:29:15.034989+0000","last_clean":"2024-03-06T07:29:15.034989+0000","last_became_active":"2024-03-06T07:29:15.031911+0000","last_became_peered":"2024-03-06T07:29:15.031911+0000","last_unstale":"2024-03-06T07:29:15.034989+0000","last_undegraded":"2024-03-06T07:29:15.034989+0000","last_fullsized":"2024-03-06T07:29:15.034989+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:38:14.113373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.033659+0000","last_change":"2024-03-06T07:29:14.028532+0000","last_active":"2024-03-06T07:29:16.033659+0000","last_peered":"2024-03-06T07:29:16.033659+0000","last_clean":"2024-03-06T07:29:16.033659+0000","last_became_active":"2024-03-06T07:29:14.028226+0000","last_became_peered":"2024-03-06T07:29:14.028226+0000","last_unstale":"2024-03-06T07:29:16.033659+0000","last_undegraded":"2024-03-06T07:29:16.033659+0000","last_fullsized":"2024-03-06T07:29:16.033659+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:21.126122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571419+0000","last_change":"2024-03-06T07:29:15.038779+0000","last_active":"2024-03-06T07:29:16.571419+0000","last_peered":"2024-03-06T07:29:16.571419+0000","last_clean":"2024-03-06T07:29:16.571419+0000","last_became_active":"2024-03-06T07:29:15.038353+0000","last_became_peered":"2024-03-06T07:29:15.038353+0000","last_unstale":"2024-03-06T07:29:16.571419+0000","last_undegraded":"2024-03-06T07:29:16.571419+0000","last_fullsized":"2024-03-06T07:29:16.571419+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:22:23.216683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.014354+0000","last_change":"2024-03-06T07:29:14.029645+0000","last_active":"2024-03-06T07:29:15.014354+0000","last_peered":"2024-03-06T07:29:15.014354+0000","last_clean":"2024-03-06T07:29:15.014354+0000","last_became_active":"2024-03-06T07:29:14.029367+0000","last_became_peered":"2024-03-06T07:29:14.029367+0000","last_unstale":"2024-03-06T07:29:15.014354+0000","last_undegraded":"2024-03-06T07:29:15.014354+0000","last_fullsized":"2024-03-06T07:29:15.014354+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:06:01.067942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571205+0000","last_change":"2024-03-06T07:29:15.041825+0000","last_active":"2024-03-06T07:29:16.571205+0000","last_peered":"2024-03-06T07:29:16.571205+0000","last_clean":"2024-03-06T07:29:16.571205+0000","last_became_active":"2024-03-06T07:29:15.041550+0000","last_became_peered":"2024-03-06T07:29:15.041550+0000","last_unstale":"2024-03-06T07:29:16.571205+0000","last_undegraded":"2024-03-06T07:29:16.571205+0000","last_fullsized":"2024-03-06T07:29:16.571205+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:53:44.310590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571323+0000","last_change":"2024-03-06T07:29:14.032990+0000","last_active":"2024-03-06T07:29:16.571323+0000","last_peered":"2024-03-06T07:29:16.571323+0000","last_clean":"2024-03-06T07:29:16.571323+0000","last_became_active":"2024-03-06T07:29:14.032701+0000","last_became_peered":"2024-03-06T07:29:14.032701+0000","last_unstale":"2024-03-06T07:29:16.571323+0000","last_undegraded":"2024-03-06T07:29:16.571323+0000","last_fullsized":"2024-03-06T07:29:16.571323+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:18:17.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017686+0000","last_change":"2024-03-06T07:29:15.036708+0000","last_active":"2024-03-06T07:29:17.017686+0000","last_peered":"2024-03-06T07:29:17.017686+0000","last_clean":"2024-03-06T07:29:17.017686+0000","last_became_active":"2024-03-06T07:29:15.036199+0000","last_became_peered":"2024-03-06T07:29:15.036199+0000","last_unstale":"2024-03-06T07:29:17.017686+0000","last_undegraded":"2024-03-06T07:29:17.017686+0000","last_fullsized":"2024-03-06T07:29:17.017686+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:29:31.581503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034316+0000","last_change":"2024-03-06T07:29:14.026059+0000","last_active":"2024-03-06T07:29:16.034316+0000","last_peered":"2024-03-06T07:29:16.034316+0000","last_clean":"2024-03-06T07:29:16.034316+0000","last_became_active":"2024-03-06T07:29:14.025771+0000","last_became_peered":"2024-03-06T07:29:14.025771+0000","last_unstale":"2024-03-06T07:29:16.034316+0000","last_undegraded":"2024-03-06T07:29:16.034316+0000","last_fullsized":"2024-03-06T07:29:16.034316+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:08.205157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020254+0000","last_change":"2024-03-06T07:29:15.038816+0000","last_active":"2024-03-06T07:29:17.020254+0000","last_peered":"2024-03-06T07:29:17.020254+0000","last_clean":"2024-03-06T07:29:17.020254+0000","last_became_active":"2024-03-06T07:29:15.038487+0000","last_became_peered":"2024-03-06T07:29:15.038487+0000","last_unstale":"2024-03-06T07:29:17.020254+0000","last_undegraded":"2024-03-06T07:29:17.020254+0000","last_fullsized":"2024-03-06T07:29:17.020254+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:50.302554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"211'3","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571265+0000","last_change":"2024-03-06T07:29:14.033683+0000","last_active":"2024-03-06T07:29:16.571265+0000","last_peered":"2024-03-06T07:29:16.571265+0000","last_clean":"2024-03-06T07:29:16.571265+0000","last_became_active":"2024-03-06T07:29:14.033401+0000","last_became_peered":"2024-03-06T07:29:14.033401+0000","last_unstale":"2024-03-06T07:29:16.571265+0000","last_undegraded":"2024-03-06T07:29:16.571265+0000","last_fullsized":"2024-03-06T07:29:16.571265+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:14:56.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034767+0000","last_change":"2024-03-06T07:29:15.023045+0000","last_active":"2024-03-06T07:29:16.034767+0000","last_peered":"2024-03-06T07:29:16.034767+0000","last_clean":"2024-03-06T07:29:16.034767+0000","last_became_active":"2024-03-06T07:29:15.022709+0000","last_became_peered":"2024-03-06T07:29:15.022709+0000","last_unstale":"2024-03-06T07:29:16.034767+0000","last_undegraded":"2024-03-06T07:29:16.034767+0000","last_fullsized":"2024-03-06T07:29:16.034767+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:15:22.975125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"211'6","reported_seq":20,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017564+0000","last_change":"2024-03-06T07:29:14.031198+0000","last_active":"2024-03-06T07:29:17.017564+0000","last_peered":"2024-03-06T07:29:17.017564+0000","last_clean":"2024-03-06T07:29:17.017564+0000","last_became_active":"2024-03-06T07:29:14.030920+0000","last_became_peered":"2024-03-06T07:29:14.030920+0000","last_unstale":"2024-03-06T07:29:17.017564+0000","last_undegraded":"2024-03-06T07:29:17.017564+0000","last_fullsized":"2024-03-06T07:29:17.017564+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:34:51.308563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.020014+0000","last_change":"2024-03-06T07:29:15.019023+0000","last_active":"2024-03-06T07:29:15.020014+0000","last_peered":"2024-03-06T07:29:15.020014+0000","last_clean":"2024-03-06T07:29:15.020014+0000","last_became_active":"2024-03-06T07:29:15.018734+0000","last_became_peered":"2024-03-06T07:29:15.018734+0000","last_unstale":"2024-03-06T07:29:15.020014+0000","last_undegraded":"2024-03-06T07:29:15.020014+0000","last_fullsized":"2024-03-06T07:29:15.020014+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:58:55.748350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"211'2","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.032015+0000","last_change":"2024-03-06T07:29:14.033296+0000","last_active":"2024-03-06T07:29:15.032015+0000","last_peered":"2024-03-06T07:29:15.032015+0000","last_clean":"2024-03-06T07:29:15.032015+0000","last_became_active":"2024-03-06T07:29:14.032954+0000","last_became_peered":"2024-03-06T07:29:14.032954+0000","last_unstale":"2024-03-06T07:29:15.032015+0000","last_undegraded":"2024-03-06T07:29:15.032015+0000","last_fullsized":"2024-03-06T07:29:15.032015+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:08:36.069733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034342+0000","last_change":"2024-03-06T07:29:15.030828+0000","last_active":"2024-03-06T07:29:16.034342+0000","last_peered":"2024-03-06T07:29:16.034342+0000","last_clean":"2024-03-06T07:29:16.034342+0000","last_became_active":"2024-03-06T07:29:15.030563+0000","last_became_peered":"2024-03-06T07:29:15.030563+0000","last_unstale":"2024-03-06T07:29:16.034342+0000","last_undegraded":"2024-03-06T07:29:16.034342+0000","last_fullsized":"2024-03-06T07:29:16.034342+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:51:26.526550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571158+0000","last_change":"2024-03-06T07:29:14.029215+0000","last_active":"2024-03-06T07:29:16.571158+0000","last_peered":"2024-03-06T07:29:16.571158+0000","last_clean":"2024-03-06T07:29:16.571158+0000","last_became_active":"2024-03-06T07:29:14.028947+0000","last_became_peered":"2024-03-06T07:29:14.028947+0000","last_unstale":"2024-03-06T07:29:16.571158+0000","last_undegraded":"2024-03-06T07:29:16.571158+0000","last_fullsized":"2024-03-06T07:29:16.571158+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:36:13.969646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.031422+0000","last_change":"2024-03-06T07:29:15.023744+0000","last_active":"2024-03-06T07:29:15.031422+0000","last_peered":"2024-03-06T07:29:15.031422+0000","last_clean":"2024-03-06T07:29:15.031422+0000","last_became_active":"2024-03-06T07:29:15.023435+0000","last_became_peered":"2024-03-06T07:29:15.023435+0000","last_unstale":"2024-03-06T07:29:15.031422+0000","last_undegraded":"2024-03-06T07:29:15.031422+0000","last_fullsized":"2024-03-06T07:29:15.031422+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:27:57.386206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570965+0000","last_change":"2024-03-06T07:29:14.022144+0000","last_active":"2024-03-06T07:29:16.570965+0000","last_peered":"2024-03-06T07:29:16.570965+0000","last_clean":"2024-03-06T07:29:16.570965+0000","last_became_active":"2024-03-06T07:29:14.021872+0000","last_became_peered":"2024-03-06T07:29:14.021872+0000","last_unstale":"2024-03-06T07:29:16.570965+0000","last_undegraded":"2024-03-06T07:29:16.570965+0000","last_fullsized":"2024-03-06T07:29:16.570965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:22:47.392808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.035517+0000","last_change":"2024-03-06T07:29:15.034474+0000","last_active":"2024-03-06T07:29:15.035517+0000","last_peered":"2024-03-06T07:29:15.035517+0000","last_clean":"2024-03-06T07:29:15.035517+0000","last_became_active":"2024-03-06T07:29:15.034202+0000","last_became_peered":"2024-03-06T07:29:15.034202+0000","last_unstale":"2024-03-06T07:29:15.035517+0000","last_undegraded":"2024-03-06T07:29:15.035517+0000","last_fullsized":"2024-03-06T07:29:15.035517+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:46:39.973578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"211'8","reported_seq":22,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570980+0000","last_change":"2024-03-06T07:29:14.030216+0000","last_active":"2024-03-06T07:29:16.570980+0000","last_peered":"2024-03-06T07:29:16.570980+0000","last_clean":"2024-03-06T07:29:16.570980+0000","last_became_active":"2024-03-06T07:29:14.029939+0000","last_became_peered":"2024-03-06T07:29:14.029939+0000","last_unstale":"2024-03-06T07:29:16.570980+0000","last_undegraded":"2024-03-06T07:29:16.570980+0000","last_fullsized":"2024-03-06T07:29:16.570980+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:48:38.320550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034485+0000","last_change":"2024-03-06T07:29:15.032073+0000","last_active":"2024-03-06T07:29:16.034485+0000","last_peered":"2024-03-06T07:29:16.034485+0000","last_clean":"2024-03-06T07:29:16.034485+0000","last_became_active":"2024-03-06T07:29:15.031746+0000","last_became_peered":"2024-03-06T07:29:15.031746+0000","last_unstale":"2024-03-06T07:29:16.034485+0000","last_undegraded":"2024-03-06T07:29:16.034485+0000","last_fullsized":"2024-03-06T07:29:16.034485+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:23:29.353034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"211'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020375+0000","last_change":"2024-03-06T07:29:14.031533+0000","last_active":"2024-03-06T07:29:17.020375+0000","last_peered":"2024-03-06T07:29:17.020375+0000","last_clean":"2024-03-06T07:29:17.020375+0000","last_became_active":"2024-03-06T07:29:14.031212+0000","last_became_peered":"2024-03-06T07:29:14.031212+0000","last_unstale":"2024-03-06T07:29:17.020375+0000","last_undegraded":"2024-03-06T07:29:17.020375+0000","last_fullsized":"2024-03-06T07:29:17.020375+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:54:13.830128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034911+0000","last_change":"2024-03-06T07:29:15.028265+0000","last_active":"2024-03-06T07:29:16.034911+0000","last_peered":"2024-03-06T07:29:16.034911+0000","last_clean":"2024-03-06T07:29:16.034911+0000","last_became_active":"2024-03-06T07:29:15.027970+0000","last_became_peered":"2024-03-06T07:29:15.027970+0000","last_unstale":"2024-03-06T07:29:16.034911+0000","last_undegraded":"2024-03-06T07:29:16.034911+0000","last_fullsized":"2024-03-06T07:29:16.034911+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:22.130981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017839+0000","last_change":"2024-03-06T07:29:14.031677+0000","last_active":"2024-03-06T07:29:17.017839+0000","last_peered":"2024-03-06T07:29:17.017839+0000","last_clean":"2024-03-06T07:29:17.017839+0000","last_became_active":"2024-03-06T07:29:14.031373+0000","last_became_peered":"2024-03-06T07:29:14.031373+0000","last_unstale":"2024-03-06T07:29:17.017839+0000","last_undegraded":"2024-03-06T07:29:17.017839+0000","last_fullsized":"2024-03-06T07:29:17.017839+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T11:09:07.074661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571286+0000","last_change":"2024-03-06T07:29:15.044773+0000","last_active":"2024-03-06T07:29:16.571286+0000","last_peered":"2024-03-06T07:29:16.571286+0000","last_clean":"2024-03-06T07:29:16.571286+0000","last_became_active":"2024-03-06T07:29:15.044497+0000","last_became_peered":"2024-03-06T07:29:15.044497+0000","last_unstale":"2024-03-06T07:29:16.571286+0000","last_undegraded":"2024-03-06T07:29:16.571286+0000","last_fullsized":"2024-03-06T07:29:16.571286+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:49:23.879288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"211'1","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.046990+0000","last_change":"2024-03-06T07:29:14.027538+0000","last_active":"2024-03-06T07:29:15.046990+0000","last_peered":"2024-03-06T07:29:15.046990+0000","last_clean":"2024-03-06T07:29:15.046990+0000","last_became_active":"2024-03-06T07:29:14.027011+0000","last_became_peered":"2024-03-06T07:29:14.027011+0000","last_unstale":"2024-03-06T07:29:15.046990+0000","last_undegraded":"2024-03-06T07:29:15.046990+0000","last_fullsized":"2024-03-06T07:29:15.046990+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:43:21.126122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034863+0000","last_change":"2024-03-06T07:29:15.033590+0000","last_active":"2024-03-06T07:29:16.034863+0000","last_peered":"2024-03-06T07:29:16.034863+0000","last_clean":"2024-03-06T07:29:16.034863+0000","last_became_active":"2024-03-06T07:29:15.032877+0000","last_became_peered":"2024-03-06T07:29:15.032877+0000","last_unstale":"2024-03-06T07:29:16.034863+0000","last_undegraded":"2024-03-06T07:29:16.034863+0000","last_fullsized":"2024-03-06T07:29:16.034863+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:39:09.512620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017655+0000","last_change":"2024-03-06T07:29:14.031829+0000","last_active":"2024-03-06T07:29:17.017655+0000","last_peered":"2024-03-06T07:29:17.017655+0000","last_clean":"2024-03-06T07:29:17.017655+0000","last_became_active":"2024-03-06T07:29:14.031533+0000","last_became_peered":"2024-03-06T07:29:14.031533+0000","last_unstale":"2024-03-06T07:29:17.017655+0000","last_undegraded":"2024-03-06T07:29:17.017655+0000","last_fullsized":"2024-03-06T07:29:17.017655+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:01:23.273579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030712+0000","last_change":"2024-03-06T07:29:15.035684+0000","last_active":"2024-03-06T07:29:16.030712+0000","last_peered":"2024-03-06T07:29:16.030712+0000","last_clean":"2024-03-06T07:29:16.030712+0000","last_became_active":"2024-03-06T07:29:15.035179+0000","last_became_peered":"2024-03-06T07:29:15.035179+0000","last_unstale":"2024-03-06T07:29:16.030712+0000","last_undegraded":"2024-03-06T07:29:16.030712+0000","last_fullsized":"2024-03-06T07:29:16.030712+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T09:38:09.954104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"211'2","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.047350+0000","last_change":"2024-03-06T07:29:14.030048+0000","last_active":"2024-03-06T07:29:15.047350+0000","last_peered":"2024-03-06T07:29:15.047350+0000","last_clean":"2024-03-06T07:29:15.047350+0000","last_became_active":"2024-03-06T07:29:14.029389+0000","last_became_peered":"2024-03-06T07:29:14.029389+0000","last_unstale":"2024-03-06T07:29:15.047350+0000","last_undegraded":"2024-03-06T07:29:15.047350+0000","last_fullsized":"2024-03-06T07:29:15.047350+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:18:56.272946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034505+0000","last_change":"2024-03-06T07:29:15.035454+0000","last_active":"2024-03-06T07:29:16.034505+0000","last_peered":"2024-03-06T07:29:16.034505+0000","last_clean":"2024-03-06T07:29:16.034505+0000","last_became_active":"2024-03-06T07:29:15.035152+0000","last_became_peered":"2024-03-06T07:29:15.035152+0000","last_unstale":"2024-03-06T07:29:16.034505+0000","last_undegraded":"2024-03-06T07:29:16.034505+0000","last_fullsized":"2024-03-06T07:29:16.034505+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:34:25.370240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.014912+0000","last_change":"2024-03-06T07:29:14.033999+0000","last_active":"2024-03-06T07:29:15.014912+0000","last_peered":"2024-03-06T07:29:15.014912+0000","last_clean":"2024-03-06T07:29:15.014912+0000","last_became_active":"2024-03-06T07:29:14.033721+0000","last_became_peered":"2024-03-06T07:29:14.033721+0000","last_unstale":"2024-03-06T07:29:15.014912+0000","last_undegraded":"2024-03-06T07:29:15.014912+0000","last_fullsized":"2024-03-06T07:29:15.014912+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:38:35.701125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571120+0000","last_change":"2024-03-06T07:29:15.041084+0000","last_active":"2024-03-06T07:29:16.571120+0000","last_peered":"2024-03-06T07:29:16.571120+0000","last_clean":"2024-03-06T07:29:16.571120+0000","last_became_active":"2024-03-06T07:29:15.040773+0000","last_became_peered":"2024-03-06T07:29:15.040773+0000","last_unstale":"2024-03-06T07:29:16.571120+0000","last_undegraded":"2024-03-06T07:29:16.571120+0000","last_fullsized":"2024-03-06T07:29:16.571120+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T10:33:50.510478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030863+0000","last_change":"2024-03-06T07:29:14.031920+0000","last_active":"2024-03-06T07:29:16.030863+0000","last_peered":"2024-03-06T07:29:16.030863+0000","last_clean":"2024-03-06T07:29:16.030863+0000","last_became_active":"2024-03-06T07:29:14.031588+0000","last_became_peered":"2024-03-06T07:29:14.031588+0000","last_unstale":"2024-03-06T07:29:16.030863+0000","last_undegraded":"2024-03-06T07:29:16.030863+0000","last_fullsized":"2024-03-06T07:29:16.030863+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:31:49.297694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030650+0000","last_change":"2024-03-06T07:29:15.040031+0000","last_active":"2024-03-06T07:29:16.030650+0000","last_peered":"2024-03-06T07:29:16.030650+0000","last_clean":"2024-03-06T07:29:16.030650+0000","last_became_active":"2024-03-06T07:29:15.039736+0000","last_became_peered":"2024-03-06T07:29:15.039736+0000","last_unstale":"2024-03-06T07:29:16.030650+0000","last_undegraded":"2024-03-06T07:29:16.030650+0000","last_fullsized":"2024-03-06T07:29:16.030650+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:22:20.232362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570995+0000","last_change":"2024-03-06T07:29:14.021470+0000","last_active":"2024-03-06T07:29:16.570995+0000","last_peered":"2024-03-06T07:29:16.570995+0000","last_clean":"2024-03-06T07:29:16.570995+0000","last_became_active":"2024-03-06T07:29:14.021182+0000","last_became_peered":"2024-03-06T07:29:14.021182+0000","last_unstale":"2024-03-06T07:29:16.570995+0000","last_undegraded":"2024-03-06T07:29:16.570995+0000","last_fullsized":"2024-03-06T07:29:16.570995+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T07:45:27.551838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020273+0000","last_change":"2024-03-06T07:29:15.039331+0000","last_active":"2024-03-06T07:29:17.020273+0000","last_peered":"2024-03-06T07:29:17.020273+0000","last_clean":"2024-03-06T07:29:17.020273+0000","last_became_active":"2024-03-06T07:29:15.038714+0000","last_became_peered":"2024-03-06T07:29:15.038714+0000","last_unstale":"2024-03-06T07:29:17.020273+0000","last_undegraded":"2024-03-06T07:29:17.020273+0000","last_fullsized":"2024-03-06T07:29:17.020273+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:17:35.149993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"211'2","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571297+0000","last_change":"2024-03-06T07:29:14.026448+0000","last_active":"2024-03-06T07:29:16.571297+0000","last_peered":"2024-03-06T07:29:16.571297+0000","last_clean":"2024-03-06T07:29:16.571297+0000","last_became_active":"2024-03-06T07:29:14.026082+0000","last_became_peered":"2024-03-06T07:29:14.026082+0000","last_unstale":"2024-03-06T07:29:16.571297+0000","last_undegraded":"2024-03-06T07:29:16.571297+0000","last_fullsized":"2024-03-06T07:29:16.571297+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:08:48.523431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020113+0000","last_change":"2024-03-06T07:29:15.035814+0000","last_active":"2024-03-06T07:29:17.020113+0000","last_peered":"2024-03-06T07:29:17.020113+0000","last_clean":"2024-03-06T07:29:17.020113+0000","last_became_active":"2024-03-06T07:29:15.035541+0000","last_became_peered":"2024-03-06T07:29:15.035541+0000","last_unstale":"2024-03-06T07:29:17.020113+0000","last_undegraded":"2024-03-06T07:29:17.020113+0000","last_fullsized":"2024-03-06T07:29:17.020113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:23:19.801951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"212'1","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019364+0000","last_change":"2024-03-06T07:29:14.030770+0000","last_active":"2024-03-06T07:29:17.019364+0000","last_peered":"2024-03-06T07:29:17.019364+0000","last_clean":"2024-03-06T07:29:17.019364+0000","last_became_active":"2024-03-06T07:29:14.030460+0000","last_became_peered":"2024-03-06T07:29:14.030460+0000","last_unstale":"2024-03-06T07:29:17.019364+0000","last_undegraded":"2024-03-06T07:29:17.019364+0000","last_fullsized":"2024-03-06T07:29:17.019364+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:26:22.868851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.035064+0000","last_change":"2024-03-06T07:29:15.033841+0000","last_active":"2024-03-06T07:29:15.035064+0000","last_peered":"2024-03-06T07:29:15.035064+0000","last_clean":"2024-03-06T07:29:15.035064+0000","last_became_active":"2024-03-06T07:29:15.033563+0000","last_became_peered":"2024-03-06T07:29:15.033563+0000","last_unstale":"2024-03-06T07:29:15.035064+0000","last_undegraded":"2024-03-06T07:29:15.035064+0000","last_fullsized":"2024-03-06T07:29:15.035064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:07:16.903978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024013+0000","last_change":"2024-03-06T07:29:14.030874+0000","last_active":"2024-03-06T07:29:15.024013+0000","last_peered":"2024-03-06T07:29:15.024013+0000","last_clean":"2024-03-06T07:29:15.024013+0000","last_became_active":"2024-03-06T07:29:14.030607+0000","last_became_peered":"2024-03-06T07:29:14.030607+0000","last_unstale":"2024-03-06T07:29:15.024013+0000","last_undegraded":"2024-03-06T07:29:15.024013+0000","last_fullsized":"2024-03-06T07:29:15.024013+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:45:00.992425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.020348+0000","last_change":"2024-03-06T07:29:15.027929+0000","last_active":"2024-03-06T07:29:17.020348+0000","last_peered":"2024-03-06T07:29:17.020348+0000","last_clean":"2024-03-06T07:29:17.020348+0000","last_became_active":"2024-03-06T07:29:15.024578+0000","last_became_peered":"2024-03-06T07:29:15.024578+0000","last_unstale":"2024-03-06T07:29:17.020348+0000","last_undegraded":"2024-03-06T07:29:17.020348+0000","last_fullsized":"2024-03-06T07:29:17.020348+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:54:20.400622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.024749+0000","last_change":"2024-03-06T07:29:14.026706+0000","last_active":"2024-03-06T07:29:15.024749+0000","last_peered":"2024-03-06T07:29:15.024749+0000","last_clean":"2024-03-06T07:29:15.024749+0000","last_became_active":"2024-03-06T07:29:14.026337+0000","last_became_peered":"2024-03-06T07:29:14.026337+0000","last_unstale":"2024-03-06T07:29:15.024749+0000","last_undegraded":"2024-03-06T07:29:15.024749+0000","last_fullsized":"2024-03-06T07:29:15.024749+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:42:08.205157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.570866+0000","last_change":"2024-03-06T07:29:15.040449+0000","last_active":"2024-03-06T07:29:16.570866+0000","last_peered":"2024-03-06T07:29:16.570866+0000","last_clean":"2024-03-06T07:29:16.570866+0000","last_became_active":"2024-03-06T07:29:15.037756+0000","last_became_peered":"2024-03-06T07:29:15.037756+0000","last_unstale":"2024-03-06T07:29:16.570866+0000","last_undegraded":"2024-03-06T07:29:16.570866+0000","last_fullsized":"2024-03-06T07:29:16.570866+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:29:17.967222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017588+0000","last_change":"2024-03-06T07:29:14.025980+0000","last_active":"2024-03-06T07:29:17.017588+0000","last_peered":"2024-03-06T07:29:17.017588+0000","last_clean":"2024-03-06T07:29:17.017588+0000","last_became_active":"2024-03-06T07:29:14.025697+0000","last_became_peered":"2024-03-06T07:29:14.025697+0000","last_unstale":"2024-03-06T07:29:17.017588+0000","last_undegraded":"2024-03-06T07:29:17.017588+0000","last_fullsized":"2024-03-06T07:29:17.017588+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:38:26.313123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.030871+0000","last_change":"2024-03-06T07:29:15.037674+0000","last_active":"2024-03-06T07:29:16.030871+0000","last_peered":"2024-03-06T07:29:16.030871+0000","last_clean":"2024-03-06T07:29:16.030871+0000","last_became_active":"2024-03-06T07:29:15.036597+0000","last_became_peered":"2024-03-06T07:29:15.036597+0000","last_unstale":"2024-03-06T07:29:16.030871+0000","last_undegraded":"2024-03-06T07:29:16.030871+0000","last_fullsized":"2024-03-06T07:29:16.030871+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T18:11:18.023292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017595+0000","last_change":"2024-03-06T07:29:14.032273+0000","last_active":"2024-03-06T07:29:17.017595+0000","last_peered":"2024-03-06T07:29:17.017595+0000","last_clean":"2024-03-06T07:29:17.017595+0000","last_became_active":"2024-03-06T07:29:14.032006+0000","last_became_peered":"2024-03-06T07:29:14.032006+0000","last_unstale":"2024-03-06T07:29:17.017595+0000","last_undegraded":"2024-03-06T07:29:17.017595+0000","last_fullsized":"2024-03-06T07:29:17.017595+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T19:13:59.794344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.031136+0000","last_change":"2024-03-06T07:29:15.036506+0000","last_active":"2024-03-06T07:29:16.031136+0000","last_peered":"2024-03-06T07:29:16.031136+0000","last_clean":"2024-03-06T07:29:16.031136+0000","last_became_active":"2024-03-06T07:29:15.036231+0000","last_became_peered":"2024-03-06T07:29:15.036231+0000","last_unstale":"2024-03-06T07:29:16.031136+0000","last_undegraded":"2024-03-06T07:29:16.031136+0000","last_fullsized":"2024-03-06T07:29:16.031136+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:47:47.183696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-03-06T07:29:15.025129+0000","last_change":"2024-03-06T07:29:14.026703+0000","last_active":"2024-03-06T07:29:15.025129+0000","last_peered":"2024-03-06T07:29:15.025129+0000","last_clean":"2024-03-06T07:29:15.025129+0000","last_became_active":"2024-03-06T07:29:14.026373+0000","last_became_peered":"2024-03-06T07:29:14.026373+0000","last_unstale":"2024-03-06T07:29:15.025129+0000","last_undegraded":"2024-03-06T07:29:15.025129+0000","last_fullsized":"2024-03-06T07:29:15.025129+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T14:59:13.368742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.034044+0000","last_change":"2024-03-06T07:29:15.037909+0000","last_active":"2024-03-06T07:29:16.034044+0000","last_peered":"2024-03-06T07:29:16.034044+0000","last_clean":"2024-03-06T07:29:16.034044+0000","last_became_active":"2024-03-06T07:29:15.037534+0000","last_became_peered":"2024-03-06T07:29:15.037534+0000","last_unstale":"2024-03-06T07:29:16.034044+0000","last_undegraded":"2024-03-06T07:29:16.034044+0000","last_fullsized":"2024-03-06T07:29:16.034044+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T16:26:30.847956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.017633+0000","last_change":"2024-03-06T07:29:14.026763+0000","last_active":"2024-03-06T07:29:17.017633+0000","last_peered":"2024-03-06T07:29:17.017633+0000","last_clean":"2024-03-06T07:29:17.017633+0000","last_became_active":"2024-03-06T07:29:14.026491+0000","last_became_peered":"2024-03-06T07:29:14.026491+0000","last_unstale":"2024-03-06T07:29:17.017633+0000","last_undegraded":"2024-03-06T07:29:17.017633+0000","last_fullsized":"2024-03-06T07:29:17.017633+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:13.000133+0000","last_clean_scrub_stamp":"2024-03-06T07:29:13.000133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T15:12:05.399497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":12,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:16.571316+0000","last_change":"2024-03-06T07:29:15.036098+0000","last_active":"2024-03-06T07:29:16.571316+0000","last_peered":"2024-03-06T07:29:16.571316+0000","last_clean":"2024-03-06T07:29:16.571316+0000","last_became_active":"2024-03-06T07:29:15.035774+0000","last_became_peered":"2024-03-06T07:29:15.035774+0000","last_unstale":"2024-03-06T07:29:16.571316+0000","last_undegraded":"2024-03-06T07:29:16.571316+0000","last_fullsized":"2024-03-06T07:29:16.571316+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:29:14.004992+0000","last_clean_scrub_stamp":"2024-03-06T07:29:14.004992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T12:31:01.007773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":527,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-03-06T07:29:17.019809+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:29:17.019809+0000","last_peered":"2024-03-06T07:29:17.019809+0000","last_clean":"2024-03-06T07:29:17.019809+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:29:17.019809+0000","last_undegraded":"2024-03-06T07:29:17.019809+0000","last_fullsized":"2024-03-06T07:29:17.019809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33872,"kb_used_data":2688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337968,"statfs":{"total":96636764160,"available":96602079232,"internally_reserved":0,"allocated":2752512,"data_stored":1895697,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2695168,"data_stored":1311451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33808,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338032,"statfs":{"total":96636764160,"available":96602144768,"internally_reserved":0,"allocated":2682880,"data_stored":1304795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2703360,"data_stored":1313029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37976,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333864,"statfs":{"total":96636764160,"available":96597876736,"internally_reserved":0,"allocated":2695168,"data_stored":1311507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33816,"kb_used_data":2636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338024,"statfs":{"total":96636764160,"available":96602136576,"internally_reserved":0,"allocated":2699264,"data_stored":1306941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37964,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333876,"statfs":{"total":96636764160,"available":96597889024,"internally_reserved":0,"allocated":2686976,"data_stored":1304849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38040,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333800,"statfs":{"total":96636764160,"available":96597811200,"internally_reserved":0,"allocated":2760704,"data_stored":1901855,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:29:18.654 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-06T07:29:18.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-06T07:29:18.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.800+0000 7ff7f235e640 1 Processor -- start 2024-03-06T07:29:18.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.800+0000 7ff7f235e640 1 -- start start 2024-03-06T07:29:18.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f235e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff7ec0c2700 0x7ff7ec155230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f235e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 0x7ff7ec155770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f235e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ec0c4480 con 0x7ff7ec152690 2024-03-06T07:29:18.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f235e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ec0c2c30 con 0x7ff7ec0c2700 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f235e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0c4db0 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7f08fb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 0x7ff7ec155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7ebfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff7ec152690 0x7ff7ec0c7900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:38636/0 (socket says 172.21.15.17:38636) 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7ebfff640 1 -- 172.21.15.17:0/2965737318 learned_addr learned my addr 172.21.15.17:0/2965737318 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.801+0000 7ff7eb7fe640 1 --2- 172.21.15.17:0/2965737318 >> v2:172.21.15.163:3300/0 conn(0x7ff7ec0c2700 0x7ff7ec155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7f08fb640 1 -- 172.21.15.17:0/2965737318 >> v2:172.21.15.163:3300/0 conn(0x7ff7ec0c2700 msgr2=0x7ff7ec155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7f08fb640 1 --2- 172.21.15.17:0/2965737318 >> v2:172.21.15.163:3300/0 conn(0x7ff7ec0c2700 0x7ff7ec155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7f08fb640 1 -- 172.21.15.17:0/2965737318 >> v1:172.21.15.163:6789/0 conn(0x7ff7ec152690 legacy=0x7ff7ec0c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7f08fb640 1 -- 172.21.15.17:0/2965737318 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0c4760 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7eaffd640 1 -- 172.21.15.17:0/2965737318 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 239463548 0 0) 0x7ff7ec0c4480 con 0x7ff7ec152690 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.802+0000 7ff7f08fb640 1 --2- 172.21.15.17:0/2965737318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 0x7ff7ec155770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009d90 tx=0x7ff7e402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97d78119ab5e06d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7eaffd640 1 -- 172.21.15.17:0/2965737318 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4039070 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7eaffd640 1 -- 172.21.15.17:0/2965737318 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e402cc50 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7eaffd640 1 -- 172.21.15.17:0/2965737318 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4034440 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 -- 172.21.15.17:0/2965737318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 msgr2=0x7ff7ec155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 --2- 172.21.15.17:0/2965737318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 0x7ff7ec155770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009d90 tx=0x7ff7e402bea0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 -- 172.21.15.17:0/2965737318 shutdown_connections 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 --2- 172.21.15.17:0/2965737318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec0c1fa0 0x7ff7ec155770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 --2- 172.21.15.17:0/2965737318 >> v2:172.21.15.163:3300/0 conn(0x7ff7ec0c2700 0x7ff7ec155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 -- 172.21.15.17:0/2965737318 >> 172.21.15.17:0/2965737318 conn(0x7ff7ec0aa550 msgr2=0x7ff7ec0aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 -- 172.21.15.17:0/2965737318 shutdown_connections 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.803+0000 7ff7f235e640 1 -- 172.21.15.17:0/2965737318 wait complete. 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.804+0000 7ff7f235e640 1 Processor -- start 2024-03-06T07:29:18.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.804+0000 7ff7f235e640 1 -- start start 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7ec0c1fa0 0x7ff7ec0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec152690 0x7ff7ec13dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec152bf0 con 0x7ff7ec0c2700 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec152a70 con 0x7ff7ec0c1fa0 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f235e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec0c9060 con 0x7ff7ec152690 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7eb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47448/0 (socket says 172.21.15.17:47448) 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7eb7fe640 1 -- 172.21.15.17:0/3106136995 learned_addr learned my addr 172.21.15.17:0/3106136995 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:18.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7ebfff640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7ec0c1fa0 0x7ff7ec0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.805+0000 7ff7f08fb640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec152690 0x7ff7ec13dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 -- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec152690 msgr2=0x7ff7ec13dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec152690 0x7ff7ec13dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 -- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7ec0c1fa0 msgr2=0x7ff7ec0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7ec0c1fa0 0x7ff7ec0acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 -- 172.21.15.17:0/3106136995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0add90 con 0x7ff7ec0c2700 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7eb7fe640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000e9b0 tx=0x7ff7e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0017070 con 0x7ff7ec0c2700 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e000c9a0 con 0x7ff7ec0c2700 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec1ebb90 con 0x7ff7ec0c2700 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e000cd90 con 0x7ff7ec0c2700 2024-03-06T07:29:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.806+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec14e4e0 con 0x7ff7ec0c2700 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.807+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec04f330 con 0x7ff7ec0c2700 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.808+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7e001a070 con 0x7ff7ec0c2700 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.808+0000 7ff7e8ff9640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 0x7ff7d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.808+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7e008a240 con 0x7ff7ec0c2700 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.808+0000 7ff7ebfff640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 0x7ff7d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:18.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.809+0000 7ff7ebfff640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 0x7ff7d806d9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc007fd0 tx=0x7ff7dc007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:18.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:18.814+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7ec0add90 con 0x7ff7ec0c2700 2024-03-06T07:29:19.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.063+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff7ec0c5f50 con 0x7ff7ec0c2700 2024-03-06T07:29:19.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.065+0000 7ff7e8ff9640 1 -- 172.21.15.17:0/3106136995 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff7e0058970 con 0x7ff7ec0c2700 2024-03-06T07:29:19.067 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-06T07:29:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvGOhlzLPcABAAHpiIN/dqtpTwYvxTpoDDFA== 2024-03-06T07:29:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-06T07:29:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-06T07:29:19.068 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-06T07:29:19.068 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 msgr2=0x7ff7d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 0x7ff7d806d9c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc007fd0 tx=0x7ff7dc007de0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 msgr2=0x7ff7ec13d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000e9b0 tx=0x7ff7e000ee70 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 shutdown_connections 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff7d806b500 0x7ff7d806d9c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7ec152690 0x7ff7ec13dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7ec0c2700 0x7ff7ec13d700 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 --2- 172.21.15.17:0/3106136995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7ec0c1fa0 0x7ff7ec0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 >> 172.21.15.17:0/3106136995 conn(0x7ff7ec0aa550 msgr2=0x7ff7ec0c85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 shutdown_connections 2024-03-06T07:29:19.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.067+0000 7ff7f235e640 1 -- 172.21.15.17:0/3106136995 wait complete. 2024-03-06T07:29:19.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:19.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 Processor -- start 2024-03-06T07:29:19.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 -- start start 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f310c153990 0x7f310c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c070aa0 con 0x7f310c0acc00 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c070d80 con 0x7f310c1506b0 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f311477a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c156450 con 0x7f310c153990 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3112d17640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f310c0acc00 0x7f310c0c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60612/0 (socket says 172.21.15.17:60612) 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3112d17640 1 -- 172.21.15.17:0/363414178 learned_addr learned my addr 172.21.15.17:0/363414178 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3111d15640 1 --2- 172.21.15.17:0/363414178 >> v2:172.21.15.163:3300/0 conn(0x7f310c153990 0x7f310c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854936206 0 0) 0x7f310c070aa0 con 0x7f310c0acc00 2024-03-06T07:29:19.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30f8003520 con 0x7f310c0acc00 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.184+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512228687 0 0) 0x7f30f8003520 con 0x7f310c0acc00 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 >> v2:172.21.15.163:3300/0 conn(0x7f310c153990 msgr2=0x7f310c156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 --2- 172.21.15.17:0/363414178 >> v2:172.21.15.163:3300/0 conn(0x7f310c153990 0x7f310c156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 >> v1:172.21.15.163:6789/0 conn(0x7f310c1506b0 legacy=0x7f310c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c1566a0 con 0x7f310c0acc00 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f31040019f0 con 0x7f310c0acc00 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 >> v1:172.21.15.17:6789/0 conn(0x7f310c0acc00 legacy=0x7f310c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80051c0 0x7f30f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c156450 con 0x7f30f80039b0 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c060 con 0x7f30f80051c0 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f800c260 con 0x7f30f8004800 2024-03-06T07:29:19.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3111d15640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112d17640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80051c0 0x7f30f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 -- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 msgr2=0x7f30f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30f8004800 0x7f30f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 -- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80051c0 msgr2=0x7f30f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30f80051c0 0x7f30f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112516640 1 -- 172.21.15.17:0/363414178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f8003520 con 0x7f30f80039b0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.185+0000 7f3112d17640 1 -- 172.21.15.17:0/363414178 reap_dead start 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f3112516640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f3108001750 tx=0x7f30f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7061ac58f76f183f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108001370 con 0x7f30f80039b0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080052d0 con 0x7f30f80039b0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f3111514640 1 -- 172.21.15.17:0/363414178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108003830 con 0x7f30f80039b0 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 -- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 msgr2=0x7f30f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f3108001750 tx=0x7f30f8005830 comp rx=0 tx=0).stop 2024-03-06T07:29:19.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 -- 172.21.15.17:0/363414178 shutdown_connections 2024-03-06T07:29:19.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 --2- 172.21.15.17:0/363414178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f30f8003dd0 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 -- 172.21.15.17:0/363414178 >> 172.21.15.17:0/363414178 conn(0x7f310c06b1a0 msgr2=0x7f310c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 -- 172.21.15.17:0/363414178 shutdown_connections 2024-03-06T07:29:19.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.186+0000 7f311477a640 1 -- 172.21.15.17:0/363414178 wait complete. 2024-03-06T07:29:19.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.188+0000 7f311477a640 1 Processor -- start 2024-03-06T07:29:19.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.188+0000 7f311477a640 1 -- start start 2024-03-06T07:29:19.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f310c0c2ad0 0x7f310c149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 0x7f310c14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c156a10 con 0x7f30f80039b0 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0ad210 con 0x7f310c0c2ad0 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f311477a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c156ea0 con 0x7f310c0c6520 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47462/0 (socket says 172.21.15.17:47462) 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 -- 172.21.15.17:0/3225262600 learned_addr learned my addr 172.21.15.17:0/3225262600 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3111d15640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f310c0c2ad0 0x7f310c149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112d17640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 0x7f310c14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 -- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 msgr2=0x7f310c14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 0x7f310c14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 -- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f310c0c2ad0 msgr2=0x7f310c149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f310c0c2ad0 0x7f310c149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112516640 1 -- 172.21.15.17:0/3225262600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c14c7e0 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.189+0000 7f3112d17640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 0x7f310c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f3112516640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108041020 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3108005480 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c1ec920 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108007a30 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.190+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c1ecc00 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.191+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c04f330 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.191+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3108044020 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.192+0000 7f3102ffd640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.192+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31080abc80 con 0x7f30f80039b0 2024-03-06T07:29:19.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.192+0000 7f3111d15640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.192+0000 7f3111d15640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 0x7f30f006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f310c14c330 tx=0x7f30fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.196+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f310804b030 con 0x7f30f80039b0 2024-03-06T07:29:19.299 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:29:19.296+0000 7f56818ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:29:19.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.418+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f310c0700a0 con 0x7f30f80039b0 2024-03-06T07:29:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.418+0000 7f3102ffd640 1 -- 172.21.15.17:0/3225262600 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f310807aa00 con 0x7f30f80039b0 2024-03-06T07:29:19.421 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:19.421 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":200,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:16.480212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:19.422 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 200 2024-03-06T07:29:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.420+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 msgr2=0x7f30f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.420+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 0x7f30f006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f310c14c330 tx=0x7f30fc008040 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 msgr2=0x7f310c0c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108001120 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 shutdown_connections 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f310c0c6520 0x7f310c14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f310c0c2ad0 0x7f310c149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 --2- 172.21.15.17:0/3225262600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80039b0 0x7f310c0c4a60 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 >> 172.21.15.17:0/3225262600 conn(0x7f310c06b1a0 msgr2=0x7f310c0c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 shutdown_connections 2024-03-06T07:29:19.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.421+0000 7f311477a640 1 -- 172.21.15.17:0/3225262600 wait complete. 2024-03-06T07:29:19.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 200, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:16.480212+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10791}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:19.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-06T07:29:19.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:29:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 Processor -- start 2024-03-06T07:29:19.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 -- start start 2024-03-06T07:29:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 0x7f173406fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17340733f0 con 0x7f173406f550 2024-03-06T07:29:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1734155ca0 con 0x7f1734072cd0 2024-03-06T07:29:19.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f173a33a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17341561d0 con 0x7f1734152720 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f17388d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 0x7f173406fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:44090/0 (socket says 172.21.15.17:44090) 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f17337fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f173406f550 0x7f17340c77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:38668/0 (socket says 172.21.15.17:38668) 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 -- 172.21.15.17:0/4250412590 learned_addr learned my addr 172.21.15.17:0/4250412590 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 -- 172.21.15.17:0/4250412590 >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 msgr2=0x7f173406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 --2- 172.21.15.17:0/4250412590 >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 0x7f173406fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 -- 172.21.15.17:0/4250412590 >> v1:172.21.15.163:6789/0 conn(0x7f173406f550 legacy=0x7f17340c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f1733fff640 1 -- 172.21.15.17:0/4250412590 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734155ef0 con 0x7f1734152720 2024-03-06T07:29:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.939+0000 7f17388d7640 1 --2- 172.21.15.17:0/4250412590 >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 0x7f173406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.940+0000 7f1733fff640 1 --2- 172.21.15.17:0/4250412590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1724009870 tx=0x7f172402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46783ae564e58413 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.940+0000 7f1732ffd640 1 -- 172.21.15.17:0/4250412590 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724035020 con 0x7f1734152720 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.940+0000 7f1732ffd640 1 -- 172.21.15.17:0/4250412590 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724031640 con 0x7f1734152720 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.940+0000 7f1732ffd640 1 -- 172.21.15.17:0/4250412590 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172402d040 con 0x7f1734152720 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 -- 172.21.15.17:0/4250412590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 msgr2=0x7f17341593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 --2- 172.21.15.17:0/4250412590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1724009870 tx=0x7f172402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 -- 172.21.15.17:0/4250412590 shutdown_connections 2024-03-06T07:29:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 --2- 172.21.15.17:0/4250412590 >> v2:172.21.15.163:3300/0 conn(0x7f1734072cd0 0x7f173406fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 --2- 172.21.15.17:0/4250412590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734152720 0x7f17341593c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 -- 172.21.15.17:0/4250412590 >> 172.21.15.17:0/4250412590 conn(0x7f173406ea40 msgr2=0x7f173406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 -- 172.21.15.17:0/4250412590 shutdown_connections 2024-03-06T07:29:19.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.941+0000 7f173a33a640 1 -- 172.21.15.17:0/4250412590 wait complete. 2024-03-06T07:29:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.942+0000 7f173a33a640 1 Processor -- start 2024-03-06T07:29:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.942+0000 7f173a33a640 1 -- start start 2024-03-06T07:29:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734072cd0 0x7f173406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 0x7f1734144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1734158ea0 con 0x7f173406f550 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1734158bc0 con 0x7f1734152720 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f173a33a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17340c7ef0 con 0x7f1734072cd0 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f1733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f1733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47470/0 (socket says 172.21.15.17:47470) 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.943+0000 7f1733fff640 1 -- 172.21.15.17:0/3186722053 learned_addr learned my addr 172.21.15.17:0/3186722053 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f17388d7640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 0x7f1734144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f17337fe640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734072cd0 0x7f173406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 -- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734072cd0 msgr2=0x7f173406d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734072cd0 0x7f173406d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 -- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 msgr2=0x7f1734144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 0x7f1734144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 -- 172.21.15.17:0/3186722053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734149ba0 con 0x7f173406f550 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f17388d7640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 0x7f1734144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1733fff640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f17240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1730ff9640 1 -- 172.21.15.17:0/3186722053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724040070 con 0x7f173406f550 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1730ff9640 1 -- 172.21.15.17:0/3186722053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17240399f0 con 0x7f173406f550 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1734149e80 con 0x7f173406f550 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f1730ff9640 1 -- 172.21.15.17:0/3186722053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724035040 con 0x7f173406f550 2024-03-06T07:29:19.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.944+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173414a440 con 0x7f173406f550 2024-03-06T07:29:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.946+0000 7f1730ff9640 1 -- 172.21.15.17:0/3186722053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f172404b060 con 0x7f173406f550 2024-03-06T07:29:19.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.946+0000 7f1730ff9640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 0x7f172006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.946+0000 7f1730ff9640 1 -- 172.21.15.17:0/3186722053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17240abb90 con 0x7f173406f550 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.946+0000 7f17337fe640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 0x7f172006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.947+0000 7f17337fe640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 0x7f172006d9c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f173414efa0 tx=0x7f1728007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.947+0000 7f173cff7740 1 Processor -- start 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.947+0000 7f173cff7740 1 -- start start 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f764f50 0x56017f762a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 0x56017f75f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f6c96b0 con 0x56017f765330 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f672a60 con 0x56017f7634b0 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56017f6c5750 con 0x56017f764f50 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47482/0 (socket says 172.21.15.17:47482) 2024-03-06T07:29:19.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 -- 172.21.15.17:0/630655790 learned_addr learned my addr 172.21.15.17:0/630655790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f1733fff640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f764f50 0x56017f762a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17388d7640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 0x56017f75f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 -- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f764f50 msgr2=0x56017f762a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f764f50 0x56017f762a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 -- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 msgr2=0x56017f75f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 0x56017f75f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 -- 172.21.15.17:0/630655790 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017f690f30 con 0x56017f765330 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17388d7640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 0x56017f75f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f17337fe640 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f173406f990 tx=0x7f1728011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe31691ec33abc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f1719ffb640 1 -- 172.21.15.17:0/630655790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728001c00 con 0x56017f765330 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f1719ffb640 1 -- 172.21.15.17:0/630655790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280047e0 con 0x56017f765330 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.948+0000 7f1719ffb640 1 -- 172.21.15.17:0/630655790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728004b90 con 0x56017f765330 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- 172.21.15.17:0/630655790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 msgr2=0x56017f762f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f173406f990 tx=0x7f1728011000 comp rx=0 tx=0).stop 2024-03-06T07:29:19.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- 172.21.15.17:0/630655790 shutdown_connections 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f7634b0 0x56017f75f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f762f70 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 --2- 172.21.15.17:0/630655790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f764f50 0x56017f762a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- 172.21.15.17:0/630655790 >> 172.21.15.17:0/630655790 conn(0x56017f7617d0 msgr2=0x56017f7863c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- 172.21.15.17:0/630655790 shutdown_connections 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- 172.21.15.17:0/630655790 wait complete. 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.949+0000 7f173cff7740 1 -- start start 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 Processor -- start 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f79ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f789e30 0x56017f788060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f787230 con 0x56017f764f50 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f685ea0 con 0x56017f789e30 2024-03-06T07:29:19.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.950+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56017f786c70 con 0x56017f765330 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f173cff7740 10 client.0 _ll_register_callbacks cb 0x56017f685870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47484/0 (socket says 172.21.15.17:47484) 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 -- 172.21.15.17:0/1902461284 learned_addr learned my addr 172.21.15.17:0/1902461284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f17388d7640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f789e30 0x56017f788060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f17337fe640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f79ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 -- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 msgr2=0x56017f79ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f79ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 -- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f789e30 msgr2=0x56017f788060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f789e30 0x56017f788060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 -- 172.21.15.17:0/1902461284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56017f686270 con 0x56017f764f50 2024-03-06T07:29:19.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f17337fe640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f79ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1733fff640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f1724009870 tx=0x7f172407e2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3816bdcfdc2a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724078490 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1724082410 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17280047e0 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1724082630 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f173cff7740 10 client.10917 fetch_fsmap learned FSMap version 200 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) v3 -- 0x56017f3de2d0 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.951+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280047e0 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 200) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f17240795e0 con 0x56017f764f50 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 10 client.10917 fetch_fsmap finished waiting for FSMap version 200 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 10 client.10917 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 20 client.10917 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f7b5370 0x56017f7c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 4 client.10917 mds_command: new command op to 10791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 --> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56017f6d6bf0 con 0x56017f7b5370 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f17337fe640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f7b5370 0x56017f7c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f1712ffd640 10 client.10917 ms_handle_connect on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.952+0000 7f17337fe640 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f7b5370 0x56017f7c00e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858d63c5f4f2d34 server_cookie=d2e1702b519417e0 in_seq=0 out_seq=0 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f1712ffd640 1 -- 172.21.15.17:0/1902461284 <== mds.0 v2:172.21.15.17:6832/2434034079 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56017f6d6bf0 con 0x56017f7b5370 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f1712ffd640 10 client.10917 handle_command_reply: tid=0 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 client.10917 shutdown 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 msgr2=0x56017f7892b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f1724009870 tx=0x7f172407e2e0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 shutdown_connections 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f7b5370 0x56017f7c00e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f789e30 0x56017f788060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f1712ffd640 0 client.10917 ms_handle_reset on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:19.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f79ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 --2- 172.21.15.17:0/1902461284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f764f50 0x56017f7892b0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 >> 172.21.15.17:0/1902461284 conn(0x56017f7634b0 msgr2=0x56017f7638e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 shutdown_connections 2024-03-06T07:29:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 1 -- 172.21.15.17:0/1902461284 wait complete. 2024-03-06T07:29:19.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.953+0000 7f173cff7740 20 client.10917 trim_cache size 0 max 16384 2024-03-06T07:29:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 Processor -- start 2024-03-06T07:29:19.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 -- start start 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f79ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f784ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17341f6e40 con 0x56017f789e30 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17341eecf0 con 0x56017f764f50 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.970+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17341f6840 con 0x56017f765330 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47500/0 (socket says 172.21.15.17:47500) 2024-03-06T07:29:19.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 -- 172.21.15.17:0/2563194923 learned_addr learned my addr 172.21.15.17:0/2563194923 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17337fe640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f784ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f1733fff640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f79ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 -- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 msgr2=0x56017f784ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f784ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 -- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 msgr2=0x56017f79ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f79ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 -- 172.21.15.17:0/2563194923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56017f4a86b0 con 0x56017f789e30 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f1733fff640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f79ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17388d7640 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f172c0076b0 tx=0x7f172c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f413714180970dee server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17197fa640 1 -- 172.21.15.17:0/2563194923 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c01d070 con 0x56017f789e30 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17197fa640 1 -- 172.21.15.17:0/2563194923 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c0166a0 con 0x56017f789e30 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.971+0000 7f17197fa640 1 -- 172.21.15.17:0/2563194923 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c016a90 con 0x56017f789e30 2024-03-06T07:29:19.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 -- 172.21.15.17:0/2563194923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 msgr2=0x56017f75e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f172c0076b0 tx=0x7f172c009750 comp rx=0 tx=0).stop 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 -- 172.21.15.17:0/2563194923 shutdown_connections 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f789e30 0x56017f75e510 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f765330 0x56017f784ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 --2- 172.21.15.17:0/2563194923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f79ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 -- 172.21.15.17:0/2563194923 >> 172.21.15.17:0/2563194923 conn(0x56017f7634b0 msgr2=0x56017f7aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 -- 172.21.15.17:0/2563194923 shutdown_connections 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.972+0000 7f173cff7740 1 -- 172.21.15.17:0/2563194923 wait complete. 2024-03-06T07:29:19.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.973+0000 7f173cff7740 1 -- start start 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.973+0000 7f173cff7740 1 Processor -- start 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f785140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f789e30 0x56017f784290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f7886f0 con 0x56017f765330 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56017f787230 con 0x56017f764f50 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56017f77f790 con 0x56017f789e30 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f173cff7740 10 client.0 _ll_register_callbacks cb 0x7f172407a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47506/0 (socket says 172.21.15.17:47506) 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 -- 172.21.15.17:0/2380392108 learned_addr learned my addr 172.21.15.17:0/2380392108 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17388d7640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f789e30 0x56017f784290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f1733fff640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f785140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 -- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f789e30 msgr2=0x56017f784290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f789e30 0x56017f784290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 -- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 msgr2=0x56017f785140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f785140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.974+0000 7f17337fe640 1 -- 172.21.15.17:0/2380392108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56017f75f9e0 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f17337fe640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f1728000f60 tx=0x7f1728013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe855067cae4d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728010040 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1728004c90 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f172c0166a0 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=200) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17280047e0 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 10 client.10923 fetch_fsmap learned FSMap version 200 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=200,monmap=2+}) v3 -- 0x56017f7847d0 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c0166a0 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 200) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f172802c070 con 0x56017f765330 2024-03-06T07:29:19.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 10 client.10923 fetch_fsmap finished waiting for FSMap version 200 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 10 client.10923 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 20 client.10923 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f80dc20 0x56017f80e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 4 client.10923 mds_command: new command op to 10791 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 --> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56017f6d0f40 con 0x56017f80dc20 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.975+0000 7f1733fff640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f80dc20 0x56017f80e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f1712ffd640 10 client.10923 ms_handle_connect on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f1733fff640 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f80dc20 0x56017f80e000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9df152d825fc78 server_cookie=3a7bde534a93bded in_seq=0 out_seq=0 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f1712ffd640 1 -- 172.21.15.17:0/2380392108 <== mds.0 v2:172.21.15.17:6832/2434034079 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x56017f6d0f40 con 0x56017f80dc20 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f1712ffd640 10 client.10923 handle_command_reply: tid=0 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 client.10923 shutdown 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 msgr2=0x56017f7603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7f1728000f60 tx=0x7f1728013000 comp rx=0 tx=0).stop 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 shutdown_connections 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x56017f80dc20 0x56017f80e000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56017f789e30 0x56017f784290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56017f765330 0x56017f7603a0 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f1712ffd640 0 client.10923 ms_handle_reset on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 --2- 172.21.15.17:0/2380392108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56017f764f50 0x56017f785140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 >> 172.21.15.17:0/2380392108 conn(0x56017f7634b0 msgr2=0x56017f7660d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 shutdown_connections 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 1 -- 172.21.15.17:0/2380392108 wait complete. 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.976+0000 7f173cff7740 20 client.10923 trim_cache size 0 max 16384 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:29:19.984 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 10791, 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 4.9654344369999999, 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 200, 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 212, 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 212, 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 16.062717181 2024-03-06T07:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 msgr2=0x7f172006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 0x7f172006d9c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f173414efa0 tx=0x7f1728007cc0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 msgr2=0x7f173414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f1724000c00 tx=0x7f17240046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 shutdown_connections 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f172006b500 0x7f172006d9c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1734152720 0x7f1734144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1734072cd0 0x7f173406d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 --2- 172.21.15.17:0/3186722053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f173406f550 0x7f173414acf0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:19.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 >> 172.21.15.17:0/3186722053 conn(0x7f173406ea40 msgr2=0x7f173414f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:19.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 shutdown_connections 2024-03-06T07:29:19.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:19.977+0000 7f173a33a640 1 -- 172.21.15.17:0/3186722053 wait complete. 2024-03-06T07:29:19.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.093+0000 7f36ecf5c640 1 Processor -- start 2024-03-06T07:29:20.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.093+0000 7f36ecf5c640 1 -- start start 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36ecf5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 0x7f36e81552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36ecf5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e8152d40 con 0x7f36e80ac260 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36ecf5c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80c1b50 con 0x7f36e8152960 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36ecf5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8066e30 con 0x7f36e81592d0 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36e6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f36e80ac260 0x7f36e8155810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60616/0 (socket says 172.21.15.17:60616) 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.094+0000 7f36e6d9d640 1 -- 172.21.15.17:0/988420881 learned_addr learned my addr 172.21.15.17:0/988420881 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e5d9b640 1 --2- 172.21.15.17:0/988420881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 0x7f36e81552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081212336 0 0) 0x7f36e8152d40 con 0x7f36e80ac260 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d4003520 con 0x7f36e80ac260 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913791975 0 0) 0x7f36e80c1b50 con 0x7f36e8152960 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e8152d40 con 0x7f36e8152960 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e5d9b640 1 -- 172.21.15.17:0/988420881 >> v1:172.21.15.163:6789/0 conn(0x7f36e8152960 legacy=0x7f36e806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e5d9b640 1 -- 172.21.15.17:0/988420881 >> v1:172.21.15.17:6789/0 conn(0x7f36e80ac260 legacy=0x7f36e8155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.095+0000 7f36e5d9b640 1 -- 172.21.15.17:0/988420881 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80c1dc0 con 0x7f36e81592d0 2024-03-06T07:29:20.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36e5d9b640 1 --2- 172.21.15.17:0/988420881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 0x7f36e81552d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36d80097d0 tx=0x7f36d802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4900940065e85b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8035020 con 0x7f36e81592d0 2024-03-06T07:29:20.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8031890 con 0x7f36e81592d0 2024-03-06T07:29:20.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36e559a640 1 -- 172.21.15.17:0/988420881 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8031c60 con 0x7f36e81592d0 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 -- 172.21.15.17:0/988420881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 msgr2=0x7f36e81552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/988420881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 0x7f36e81552d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36d80097d0 tx=0x7f36d802b930 comp rx=0 tx=0).stop 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 -- 172.21.15.17:0/988420881 shutdown_connections 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/988420881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e81592d0 0x7f36e81552d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 -- 172.21.15.17:0/988420881 >> 172.21.15.17:0/988420881 conn(0x7f36e80c70b0 msgr2=0x7f36e80c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 -- 172.21.15.17:0/988420881 shutdown_connections 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.097+0000 7f36ecf5c640 1 -- 172.21.15.17:0/988420881 wait complete. 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 Processor -- start 2024-03-06T07:29:20.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 -- start start 2024-03-06T07:29:20.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e80ac260 0x7f36e81ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e8152960 0x7f36e81de930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e3a0 con 0x7f36e81eea20 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8159760 con 0x7f36e80ac260 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36ecf5c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e520 con 0x7f36e8152960 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.098+0000 7f36e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47512/0 (socket says 172.21.15.17:47512) 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 -- 172.21.15.17:0/2092226835 learned_addr learned my addr 172.21.15.17:0/2092226835 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e659c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e80ac260 0x7f36e81ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e5d9b640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e8152960 0x7f36e81de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 -- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e8152960 msgr2=0x7f36e81de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e8152960 0x7f36e81de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 -- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e80ac260 msgr2=0x7f36e81ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e80ac260 0x7f36e81ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 -- 172.21.15.17:0/2092226835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8156cd0 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36e6d9d640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f36e000cb70 tx=0x7f36e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0017070 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e000ace0 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0012620 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81eeed0 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.099+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e8155110 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.100+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e804f330 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.101+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36e001a070 con 0x7f36e81eea20 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.101+0000 7f36deffd640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.101+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36e008a0f0 con 0x7f36e81eea20 2024-03-06T07:29:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.102+0000 7f36e659c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.102+0000 7f36e659c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001650 tx=0x7f36d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.107+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36e0058e70 con 0x7f36e81eea20 2024-03-06T07:29:20.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.323+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36e8156270 con 0x7f36e81eea20 2024-03-06T07:29:20.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.323+0000 7f36deffd640 1 -- 172.21.15.17:0/2092226835 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+4995 (secure 0 0 0) 0x7f36e0058770 con 0x7f36e81eea20 2024-03-06T07:29:20.326 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:20.326 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":200,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":197}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:16.480212+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:20.327 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 200 2024-03-06T07:29:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 msgr2=0x7f36c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001650 tx=0x7f36d0009040 comp rx=0 tx=0).stop 2024-03-06T07:29:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 msgr2=0x7f36e8140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f36e000cb70 tx=0x7f36e000a430 comp rx=0 tx=0).stop 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 shutdown_connections 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e81eea20 0x7f36e8140780 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e8152960 0x7f36e81de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 --2- 172.21.15.17:0/2092226835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e80ac260 0x7f36e81ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 >> 172.21.15.17:0/2092226835 conn(0x7f36e80c70b0 msgr2=0x7f36e80c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 shutdown_connections 2024-03-06T07:29:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.325+0000 7f36ecf5c640 1 -- 172.21.15.17:0/2092226835 wait complete. 2024-03-06T07:29:20.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-03-06T07:29:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 Processor -- start 2024-03-06T07:29:20.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 -- start start 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7b0c0b0070 0x7f7b0c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b0c06ca70 0x7f7b0c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0b0450 con 0x7f7b0c0c5380 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c06c020 con 0x7f7b0c06ca70 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.442+0000 7f7b10fbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c06cf20 con 0x7f7b0c0b0070 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b09d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7b0c06ca70 0x7f7b0c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b0ad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7b0c0c5380 0x7f7b0c14e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60622/0 (socket says 172.21.15.17:60622) 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b0ad9d640 1 -- 172.21.15.17:0/3821950585 learned_addr learned my addr 172.21.15.17:0/3821950585 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b0a59c640 1 --2- 172.21.15.17:0/3821950585 >> v2:172.21.15.163:3300/0 conn(0x7f7b0c0b0070 0x7f7b0c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b09d9b640 1 -- 172.21.15.17:0/3821950585 >> v2:172.21.15.163:3300/0 conn(0x7f7b0c0b0070 msgr2=0x7f7b0c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b09d9b640 1 --2- 172.21.15.17:0/3821950585 >> v2:172.21.15.163:3300/0 conn(0x7f7b0c0b0070 0x7f7b0c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b09d9b640 1 -- 172.21.15.17:0/3821950585 >> v1:172.21.15.17:6789/0 conn(0x7f7b0c0c5380 legacy=0x7f7b0c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b0959a640 1 -- 172.21.15.17:0/3821950585 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371951975 0 0) 0x7f7b04001490 con 0x7f7b0c0c5380 2024-03-06T07:29:20.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.443+0000 7f7b09d9b640 1 -- 172.21.15.17:0/3821950585 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0c77f0 con 0x7f7b0c06ca70 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b09d9b640 1 --2- 172.21.15.17:0/3821950585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c06ca70 0x7f7b0c0c8900 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0aa3d0 tx=0x7f7af402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d517b646bb247098 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b0959a640 1 -- 172.21.15.17:0/3821950585 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40315a0 con 0x7f7b0c06ca70 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b0959a640 1 -- 172.21.15.17:0/3821950585 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4037070 con 0x7f7b0c06ca70 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b0959a640 1 -- 172.21.15.17:0/3821950585 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4031960 con 0x7f7b0c06ca70 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 -- 172.21.15.17:0/3821950585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c06ca70 msgr2=0x7f7b0c0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/3821950585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c06ca70 0x7f7b0c0c8900 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0aa3d0 tx=0x7f7af402b840 comp rx=0 tx=0).stop 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 -- 172.21.15.17:0/3821950585 shutdown_connections 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/3821950585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c06ca70 0x7f7b0c0c8900 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/3821950585 >> v2:172.21.15.163:3300/0 conn(0x7f7b0c0b0070 0x7f7b0c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.444+0000 7f7b10fbc640 1 -- 172.21.15.17:0/3821950585 >> 172.21.15.17:0/3821950585 conn(0x7f7b0c0ac590 msgr2=0x7f7b0c0ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:20.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.445+0000 7f7b10fbc640 1 -- 172.21.15.17:0/3821950585 shutdown_connections 2024-03-06T07:29:20.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.445+0000 7f7b10fbc640 1 -- 172.21.15.17:0/3821950585 wait complete. 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 Processor -- start 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 -- start start 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b0c06ca70 0x7f7b0c1eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b0c0c5380 0x7f7b0c0c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06c550 con 0x7f7b0c0b0070 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06c270 con 0x7f7b0c0c5380 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.446+0000 7f7b10fbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c06da80 con 0x7f7b0c06ca70 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47524/0 (socket says 172.21.15.17:47524) 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 -- 172.21.15.17:0/316921668 learned_addr learned my addr 172.21.15.17:0/316921668 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b0ad9d640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b0c0c5380 0x7f7b0c0c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b0a59c640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b0c06ca70 0x7f7b0c1eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 -- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b0c06ca70 msgr2=0x7f7b0c1eb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b0c06ca70 0x7f7b0c1eb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 -- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b0c0c5380 msgr2=0x7f7b0c0c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b0c0c5380 0x7f7b0c0c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 -- 172.21.15.17:0/316921668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0c30c0 con 0x7f7b0c0b0070 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b09d9b640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004230 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40043d0 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c156fb0 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003850 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.447+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c157570 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.448+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c04f330 con 0x7f7b0c0b0070 2024-03-06T07:29:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.449+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7af40039f0 con 0x7f7b0c0b0070 2024-03-06T07:29:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.449+0000 7f7b02ffd640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 0x7f7aec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.449+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7af402f030 con 0x7f7b0c0b0070 2024-03-06T07:29:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.450+0000 7f7b0a59c640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 0x7f7aec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.450+0000 7f7b0a59c640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 0x7f7aec06d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006fd0 tx=0x7f7afc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.454+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7af4041070 con 0x7f7b0c0b0070 2024-03-06T07:29:20.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:20.674+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f7b0c0c4af0 con 0x7f7b0c0b0070 2024-03-06T07:29:21.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.068+0000 7f7b02ffd640 1 -- 172.21.15.17:0/316921668 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v201) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7af407b010 con 0x7f7b0c0b0070 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 msgr2=0x7f7aec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 0x7f7aec06d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f7afc006fd0 tx=0x7f7afc008040 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 msgr2=0x7f7b0c153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af40046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 shutdown_connections 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aec06b4d0 0x7f7aec06d990 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7b0c0c5380 0x7f7b0c0c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b0c0b0070 0x7f7b0c153ee0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 --2- 172.21.15.17:0/316921668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7b0c06ca70 0x7f7b0c1eb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 >> 172.21.15.17:0/316921668 conn(0x7f7b0c0ac590 msgr2=0x7f7b0c0c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 shutdown_connections 2024-03-06T07:29:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.071+0000 7f7b10fbc640 1 -- 172.21.15.17:0/316921668 wait complete. 2024-03-06T07:29:21.085 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:21.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.186+0000 7f4ad5ef4640 1 Processor -- start 2024-03-06T07:29:21.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.186+0000 7f4ad5ef4640 1 -- start start 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ad5ef4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ad00c5480 0x7f4ad0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ad5ef4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ad00c5860 0x7f4ad006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ad5ef4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00c5cc0 con 0x7f4ad01586f0 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ad5ef4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad006df40 con 0x7f4ad00c5480 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ad5ef4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad01591c0 con 0x7f4ad00c5860 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acf7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ad00c5480 0x7f4ad0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4ad01586f0 0x7f4ad00724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59612/0 (socket says 172.21.15.17:59612) 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acffff640 1 -- 172.21.15.17:0/511624832 learned_addr learned my addr 172.21.15.17:0/511624832 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acf7fe640 1 -- 172.21.15.17:0/511624832 >> v2:172.21.15.163:3300/0 conn(0x7f4ad00c5860 msgr2=0x7f4ad006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acf7fe640 1 --2- 172.21.15.17:0/511624832 >> v2:172.21.15.163:3300/0 conn(0x7f4ad00c5860 0x7f4ad006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acf7fe640 1 -- 172.21.15.17:0/511624832 >> v1:172.21.15.17:6789/0 conn(0x7f4ad01586f0 legacy=0x7f4ad00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4acf7fe640 1 -- 172.21.15.17:0/511624832 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad01594a0 con 0x7f4ad00c5480 2024-03-06T07:29:21.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.187+0000 7f4ace7fc640 1 -- 172.21.15.17:0/511624832 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625782656 0 0) 0x7f4ad00c5cc0 con 0x7f4ad01586f0 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4acf7fe640 1 --2- 172.21.15.17:0/511624832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad0155ba0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00c9770 tx=0x7f4ac002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0018fc1ad12e4f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ace7fc640 1 -- 172.21.15.17:0/511624832 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00315a0 con 0x7f4ad00c5480 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ace7fc640 1 -- 172.21.15.17:0/511624832 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4ad00c5480 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ace7fc640 1 -- 172.21.15.17:0/511624832 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0031960 con 0x7f4ad00c5480 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/511624832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 msgr2=0x7f4ad0155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/511624832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad0155ba0 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00c9770 tx=0x7f4ac002b840 comp rx=0 tx=0).stop 2024-03-06T07:29:21.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/511624832 shutdown_connections 2024-03-06T07:29:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/511624832 >> v2:172.21.15.163:3300/0 conn(0x7f4ad00c5860 0x7f4ad006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/511624832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad0155ba0 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/511624832 >> 172.21.15.17:0/511624832 conn(0x7f4ad006b1a0 msgr2=0x7f4ad006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/511624832 shutdown_connections 2024-03-06T07:29:21.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.189+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/511624832 wait complete. 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.190+0000 7f4ad5ef4640 1 Processor -- start 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 -- start start 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ad00c5860 0x7f4ad014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ad01586f0 0x7f4ad00ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0152a20 con 0x7f4ad00c5480 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0155680 con 0x7f4ad00c5860 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4ad5ef4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0155050 con 0x7f4ad01586f0 2024-03-06T07:29:21.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4acf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59406/0 (socket says 172.21.15.17:59406) 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4acf7fe640 1 -- 172.21.15.17:0/1106231337 learned_addr learned my addr 172.21.15.17:0/1106231337 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4aceffd640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ad00c5860 0x7f4ad014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.191+0000 7f4acffff640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ad01586f0 0x7f4ad00ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 -- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ad01586f0 msgr2=0x7f4ad00ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ad01586f0 0x7f4ad00ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 -- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ad00c5860 msgr2=0x7f4ad014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ad00c5860 0x7f4ad014bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 -- 172.21.15.17:0/1106231337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad014f180 con 0x7f4ad00c5480 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4acf7fe640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009890 tx=0x7f4ac00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0032840 con 0x7f4ad00c5480 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0032a00 con 0x7f4ad00c5480 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad00ab7e0 con 0x7f4ad00c5480 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0032df0 con 0x7f4ad00c5480 2024-03-06T07:29:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.192+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad014c830 con 0x7f4ad00c5480 2024-03-06T07:29:21.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.193+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad004f330 con 0x7f4ad00c5480 2024-03-06T07:29:21.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.194+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac004c020 con 0x7f4ad00c5480 2024-03-06T07:29:21.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.194+0000 7f4ab7fff640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:21.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.194+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ac002f020 con 0x7f4ad00c5480 2024-03-06T07:29:21.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.194+0000 7f4aceffd640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:21.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.195+0000 7f4aceffd640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ac2c0 tx=0x7f4ac4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.198+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ac0040070 con 0x7f4ad00c5480 2024-03-06T07:29:21.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.419+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ad00c2670 con 0x7f4ad00c5480 2024-03-06T07:29:21.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.420+0000 7f4ab7fff640 1 -- 172.21.15.17:0/1106231337 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f4ac007b0a0 con 0x7f4ad00c5480 2024-03-06T07:29:21.421 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:21.422 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":202,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:21.071407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:creating","state_seq":1,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:21.422 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 202 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 msgr2=0x7f4abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ac2c0 tx=0x7f4ac4008040 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 msgr2=0x7f4ad01eabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009890 tx=0x7f4ac00046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 shutdown_connections 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ad01586f0 0x7f4ad00ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ad00c5860 0x7f4ad014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 --2- 172.21.15.17:0/1106231337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ad00c5480 0x7f4ad01eabd0 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:21.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 >> 172.21.15.17:0/1106231337 conn(0x7f4ad006b1a0 msgr2=0x7f4ad0070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 shutdown_connections 2024-03-06T07:29:21.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:21.422+0000 7f4ad5ef4640 1 -- 172.21.15.17:0/1106231337 wait complete. 2024-03-06T07:29:21.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 202, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:21.071407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:21.837 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:creating 2024-03-06T07:29:22.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:22.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.938+0000 7f5d5b633640 1 Processor -- start 2024-03-06T07:29:22.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.939+0000 7f5d5b633640 1 -- start start 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d5b633640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5d5414f360 0x7f5d54155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d5b633640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540c73d0 con 0x7f5d540c88c0 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d5b633640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540be2e0 con 0x7f5d54158b60 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d5b633640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d54159700 con 0x7f5d5414f360 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d58bce640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5d540c88c0 0x7f5d540c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59622/0 (socket says 172.21.15.17:59622) 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d58bce640 1 -- 172.21.15.17:0/1458310596 learned_addr learned my addr 172.21.15.17:0/1458310596 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d593cf640 1 --2- 172.21.15.17:0/1458310596 >> v2:172.21.15.163:3300/0 conn(0x7f5d5414f360 0x7f5d54155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181354348 0 0) 0x7f5d540c73d0 con 0x7f5d540c88c0 2024-03-06T07:29:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d40003520 con 0x7f5d540c88c0 2024-03-06T07:29:22.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183712631 0 0) 0x7f5d540be2e0 con 0x7f5d54158b60 2024-03-06T07:29:22.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d593cf640 1 -- 172.21.15.17:0/1458310596 >> v1:172.21.15.163:6789/0 conn(0x7f5d54158b60 legacy=0x7f5d54158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:22.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d593cf640 1 -- 172.21.15.17:0/1458310596 >> v1:172.21.15.17:6789/0 conn(0x7f5d540c88c0 legacy=0x7f5d540c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:22.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.940+0000 7f5d593cf640 1 -- 172.21.15.17:0/1458310596 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540c7670 con 0x7f5d5414f360 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.941+0000 7f5d593cf640 1 --2- 172.21.15.17:0/1458310596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d54155c10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097a0 tx=0x7f5d4802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e6830001ad2eca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.941+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480315a0 con 0x7f5d5414f360 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.941+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d48037070 con 0x7f5d5414f360 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.941+0000 7f5d47fff640 1 -- 172.21.15.17:0/1458310596 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48031960 con 0x7f5d5414f360 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 -- 172.21.15.17:0/1458310596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 msgr2=0x7f5d54155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 --2- 172.21.15.17:0/1458310596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d54155c10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5d480097a0 tx=0x7f5d4802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 -- 172.21.15.17:0/1458310596 shutdown_connections 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 --2- 172.21.15.17:0/1458310596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d54155c10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 -- 172.21.15.17:0/1458310596 >> 172.21.15.17:0/1458310596 conn(0x7f5d540ac570 msgr2=0x7f5d540ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 -- 172.21.15.17:0/1458310596 shutdown_connections 2024-03-06T07:29:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.942+0000 7f5d5b633640 1 -- 172.21.15.17:0/1458310596 wait complete. 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.943+0000 7f5d5b633640 1 Processor -- start 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.943+0000 7f5d5b633640 1 -- start start 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d540c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d54158b60 0x7f5d540c3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c7c30 con 0x7f5d540c88c0 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c5a50 con 0x7f5d5414f360 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d5b633640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c5bd0 con 0x7f5d54158b60 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59412/0 (socket says 172.21.15.17:59412) 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 -- 172.21.15.17:0/3187284435 learned_addr learned my addr 172.21.15.17:0/3187284435 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d58bce640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d540c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:22.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d59bd0640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d54158b60 0x7f5d540c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 -- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d54158b60 msgr2=0x7f5d540c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d54158b60 0x7f5d540c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 -- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 msgr2=0x7f5d540c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d540c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 -- 172.21.15.17:0/3187284435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540c0450 con 0x7f5d540c88c0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d58bce640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d540c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.944+0000 7f5d593cf640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034040 tx=0x7f5d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.945+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48004020 con 0x7f5d540c88c0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.945+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d480041e0 con 0x7f5d540c88c0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.945+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d541ef010 con 0x7f5d540c88c0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.945+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003610 con 0x7f5d540c88c0 2024-03-06T07:29:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.945+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c2c20 con 0x7f5d540c88c0 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.946+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5404f330 con 0x7f5d540c88c0 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.946+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d480037b0 con 0x7f5d540c88c0 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.947+0000 7f5d45ffb640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.947+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d4802f030 con 0x7f5d540c88c0 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.947+0000 7f5d58bce640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.947+0000 7f5d58bce640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 0x7f5d3806d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c0110 tx=0x7f5d50007a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:22.953+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d48040070 con 0x7f5d540c88c0 2024-03-06T07:29:23.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.235+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d540c8d50 con 0x7f5d540c88c0 2024-03-06T07:29:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.235+0000 7f5d45ffb640 1 -- 172.21.15.17:0/3187284435 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+5014 (secure 0 0 0) 0x7f5d4807a0b0 con 0x7f5d540c88c0 2024-03-06T07:29:23.237 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:23.237 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":203,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:22.075585+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[9556,10791]},"id":16}]} 2024-03-06T07:29:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 203 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 msgr2=0x7f5d3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 0x7f5d3806d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f5d540c0110 tx=0x7f5d50007a10 comp rx=0 tx=0).stop 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 msgr2=0x7f5d541ee620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034040 tx=0x7f5d480046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 shutdown_connections 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5d54158b60 0x7f5d540c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5d5414f360 0x7f5d540c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 --2- 172.21.15.17:0/3187284435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d540c88c0 0x7f5d541ee620 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 >> 172.21.15.17:0/3187284435 conn(0x7f5d540ac570 msgr2=0x7f5d5414dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 shutdown_connections 2024-03-06T07:29:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.238+0000 7f5d5b633640 1 -- 172.21.15.17:0/3187284435 wait complete. 2024-03-06T07:29:23.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 203, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:22.075585+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [9556, 10791]} 2024-03-06T07:29:23.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2024-03-06T07:29:23.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-06T07:29:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.753+0000 7fe9c2e50640 1 Processor -- start 2024-03-06T07:29:23.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.753+0000 7fe9c2e50640 1 -- start start 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c2e50640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe9bc0c69f0 0x7fe9bc0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c2e50640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 0x7fe9bc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c2e50640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc155130 con 0x7fe9bc0ad2b0 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c2e50640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc155440 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c2e50640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0c5d50 con 0x7fe9bc0c8950 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c0bec640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe9bc0c69f0 0x7fe9bc0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c13ed640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe9bc0ad2b0 0x7fe9bc1558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:51742/0 (socket says 172.21.15.17:51742) 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9c13ed640 1 -- 172.21.15.17:0/934189279 learned_addr learned my addr 172.21.15.17:0/934189279 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.754+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 0x7fe9bc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9bb7fe640 1 -- 172.21.15.17:0/934189279 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551581647 0 0) 0x7fe9b4001490 con 0x7fe9bc0ad2b0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9bb7fe640 1 -- 172.21.15.17:0/934189279 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac003520 con 0x7fe9bc0ad2b0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9c0bec640 1 -- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 msgr2=0x7fe9bc0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 0x7fe9bc0acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9c0bec640 1 -- 172.21.15.17:0/934189279 >> v1:172.21.15.163:6789/0 conn(0x7fe9bc0ad2b0 legacy=0x7fe9bc1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9c0bec640 1 -- 172.21.15.17:0/934189279 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0c6030 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 0x7fe9bc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.755+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0c69f0 0x7fe9bc0c6dd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00097a0 tx=0x7fe9b002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b34939d44beaec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9bb7fe640 1 -- 172.21.15.17:0/934189279 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00315a0 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9bb7fe640 1 -- 172.21.15.17:0/934189279 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0037070 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9bb7fe640 1 -- 172.21.15.17:0/934189279 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0031960 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 -- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0c69f0 msgr2=0x7fe9bc0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0c69f0 0x7fe9bc0c6dd0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00097a0 tx=0x7fe9b002b840 comp rx=0 tx=0).stop 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 -- 172.21.15.17:0/934189279 shutdown_connections 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c8950 0x7fe9bc0acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/934189279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0c69f0 0x7fe9bc0c6dd0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 -- 172.21.15.17:0/934189279 >> 172.21.15.17:0/934189279 conn(0x7fe9bc0cae70 msgr2=0x7fe9bc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 -- 172.21.15.17:0/934189279 shutdown_connections 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.756+0000 7fe9c2e50640 1 -- 172.21.15.17:0/934189279 wait complete. 2024-03-06T07:29:23.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.757+0000 7fe9c2e50640 1 Processor -- start 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.757+0000 7fe9c2e50640 1 -- start start 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0ad2b0 0x7fe9bc0aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c69f0 0x7fe9bc154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc152a20 con 0x7fe9bc0c8950 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc155fe0 con 0x7fe9bc0ad2b0 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c2e50640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc152ba0 con 0x7fe9bc0c69f0 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c13ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c13ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59422/0 (socket says 172.21.15.17:59422) 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c0bec640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0ad2b0 0x7fe9bc0aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9c13ed640 1 -- 172.21.15.17:0/1371662540 learned_addr learned my addr 172.21.15.17:0/1371662540 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.758+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c69f0 0x7fe9bc154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 -- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c69f0 msgr2=0x7fe9bc154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c69f0 0x7fe9bc154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 -- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0ad2b0 msgr2=0x7fe9bc0aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0ad2b0 0x7fe9bc0aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 -- 172.21.15.17:0/1371662540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc1eeb90 con 0x7fe9bc0c8950 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400caa0 tx=0x7fe9b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9b97fa640 1 -- 172.21.15.17:0/1371662540 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4017070 con 0x7fe9bc0c8950 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9b97fa640 1 -- 172.21.15.17:0/1371662540 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b400ac00 con 0x7fe9bc0c8950 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0adbc0 con 0x7fe9bc0c8950 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9b97fa640 1 -- 172.21.15.17:0/1371662540 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4012400 con 0x7fe9bc0c8950 2024-03-06T07:29:23.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.759+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc154650 con 0x7fe9bc0c8950 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.761+0000 7fe9b97fa640 1 -- 172.21.15.17:0/1371662540 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b401a070 con 0x7fe9bc0c8950 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.761+0000 7fe9b97fa640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.761+0000 7fe9b97fa640 1 -- 172.21.15.17:0/1371662540 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9b408a0f0 con 0x7fe9bc0c8950 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.761+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 0x7fe9a006d990 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036000 tx=0x7fe9b003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 Processor -- start 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 -- start start 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c055330 0x56348c052f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c0534b0 0x56348c04f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348bfb96b0 con 0x56348c054f50 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348bf62a60 con 0x56348c055330 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.762+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56348bfb5750 con 0x56348c0534b0 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59426/0 (socket says 172.21.15.17:59426) 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 -- 172.21.15.17:0/1081150237 learned_addr learned my addr 172.21.15.17:0/1081150237 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c055330 0x56348c052f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c0534b0 0x56348c04f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 -- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c0534b0 msgr2=0x56348c04f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c0534b0 0x56348c04f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 -- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c055330 msgr2=0x56348c052f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c055330 0x56348c052f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 -- 172.21.15.17:0/1081150237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56348bf80f30 con 0x56348c054f50 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0034040 tx=0x7fe9b0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9772f7a19f9987 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9a67fc640 1 -- 172.21.15.17:0/1081150237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0054070 con 0x56348c054f50 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9a67fc640 1 -- 172.21.15.17:0/1081150237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0037040 con 0x56348c054f50 2024-03-06T07:29:23.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.763+0000 7fe9a67fc640 1 -- 172.21.15.17:0/1081150237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0032b00 con 0x56348c054f50 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1081150237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 msgr2=0x56348c052a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0034040 tx=0x7fe9b0030000 comp rx=0 tx=0).stop 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1081150237 shutdown_connections 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c0534b0 0x56348c04f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c055330 0x56348c052f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1081150237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c054f50 0x56348c052a30 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1081150237 >> 172.21.15.17:0/1081150237 conn(0x56348c0517d0 msgr2=0x56348c0763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1081150237 shutdown_connections 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.764+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1081150237 wait complete. 2024-03-06T07:29:23.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.765+0000 7fe9c5a31740 1 -- start start 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.765+0000 7fe9c5a31740 1 Processor -- start 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c0a2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348c077230 con 0x56348c055330 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348bf75ea0 con 0x56348c07aa30 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56348c076c70 con 0x56348c054f50 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 10 client.0 _ll_register_callbacks cb 0x56348bf75870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59430/0 (socket says 172.21.15.17:59430) 2024-03-06T07:29:23.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 -- 172.21.15.17:0/1269478567 learned_addr learned my addr 172.21.15.17:0/1269478567 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c0a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 -- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 msgr2=0x56348c0a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c0a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 -- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 msgr2=0x56348c071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 -- 172.21.15.17:0/1269478567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56348bf76270 con 0x56348c055330 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8014750 tx=0x7fe9a8014c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8f0ddbe973f35e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe99b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8022060 con 0x56348c055330 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9a8012b20 con 0x56348c055330 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.766+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b0037040 con 0x56348c055330 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a8028050 con 0x56348c055330 2024-03-06T07:29:23.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 10 client.10962 fetch_fsmap learned FSMap version 203 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x56348bcce2d0 con 0x56348c055330 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0037040 con 0x56348c055330 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 203) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fe9a802b020 con 0x56348c055330 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 10 client.10962 fetch_fsmap finished waiting for FSMap version 203 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 10 client.10962 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 20 client.10962 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c0781b0 0x56348c0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 4 client.10962 mds_command: new command op to 9556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 --> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56348bfc6bf0 con 0x56348c0781b0 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c0781b0 0x56348c0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe99b7fe640 10 client.10962 ms_handle_connect on v2:172.21.15.163:6836/370957199 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.767+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c0781b0 0x56348c0b0680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d69d3ac27eae260 server_cookie=368065ce7536d465 in_seq=0 out_seq=0 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe99b7fe640 1 -- 172.21.15.17:0/1269478567 <== mds.1 v2:172.21.15.163:6836/370957199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56348bfc6bf0 con 0x56348c0781b0 2024-03-06T07:29:23.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe99b7fe640 10 client.10962 handle_command_reply: tid=0 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 client.10962 shutdown 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 msgr2=0x56348c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8014750 tx=0x7fe9a8014c10 comp rx=0 tx=0).stop 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 shutdown_connections 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c0781b0 0x56348c0b0680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c079200 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe99b7fe640 0 client.10962 ms_handle_reset on v2:172.21.15.163:6836/370957199 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/1269478567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c0a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 >> 172.21.15.17:0/1269478567 conn(0x56348c0534b0 msgr2=0x56348c091b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 shutdown_connections 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.768+0000 7fe9c5a31740 1 -- 172.21.15.17:0/1269478567 wait complete. 2024-03-06T07:29:23.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.769+0000 7fe9c5a31740 20 client.10962 trim_cache size 0 max 16384 2024-03-06T07:29:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.785+0000 7fe9c5a31740 1 Processor -- start 2024-03-06T07:29:23.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.785+0000 7fe9c5a31740 1 -- start start 2024-03-06T07:29:23.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c055c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f7c20 con 0x56348c055330 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f6250 con 0x56348c07aa30 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f6750 con 0x56348c054f50 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59434/0 (socket says 172.21.15.17:59434) 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 -- 172.21.15.17:0/271685535 learned_addr learned my addr 172.21.15.17:0/271685535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c055c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 -- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 msgr2=0x56348c079200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c079200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 -- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 msgr2=0x56348c055c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c055c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 -- 172.21.15.17:0/271685535 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56348bd986b0 con 0x56348c055330 2024-03-06T07:29:23.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c079200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.786+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8000c00 tx=0x7fe9a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d821539d076ef091 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9a5ffb640 1 -- 172.21.15.17:0/271685535 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8005990 con 0x56348c055330 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9a5ffb640 1 -- 172.21.15.17:0/271685535 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8005b30 con 0x56348c055330 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9a5ffb640 1 -- 172.21.15.17:0/271685535 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8006040 con 0x56348c055330 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 -- 172.21.15.17:0/271685535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 msgr2=0x56348c055710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8000c00 tx=0x7fe9a8017000 comp rx=0 tx=0).stop 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 -- 172.21.15.17:0/271685535 shutdown_connections 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c07aa30 0x56348c055c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c055710 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/271685535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c054f50 0x56348c079200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 -- 172.21.15.17:0/271685535 >> 172.21.15.17:0/271685535 conn(0x56348c0534b0 msgr2=0x56348c091b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 -- 172.21.15.17:0/271685535 shutdown_connections 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.787+0000 7fe9c5a31740 1 -- 172.21.15.17:0/271685535 wait complete. 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.788+0000 7fe9c5a31740 1 -- start start 2024-03-06T07:29:23.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.788+0000 7fe9c5a31740 1 Processor -- start 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c054f50 0x56348c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c07aa30 0x56348c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348c079cc0 con 0x56348c055330 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x56348c0766a0 con 0x56348c054f50 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x56348c077600 con 0x56348c07aa30 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 10 client.0 _ll_register_callbacks cb 0x7fe9a80143b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59450/0 (socket says 172.21.15.17:59450) 2024-03-06T07:29:23.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 -- 172.21.15.17:0/3550545871 learned_addr learned my addr 172.21.15.17:0/3550545871 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c13ed640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c07aa30 0x56348c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c054f50 0x56348c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 -- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c07aa30 msgr2=0x56348c0fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c07aa30 0x56348c0fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 -- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c054f50 msgr2=0x56348c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c054f50 0x56348c0b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 -- 172.21.15.17:0/3550545871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56348c075240 con 0x56348c055330 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9bbfff640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586027d3402e1ee3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe99b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8039060 con 0x56348c055330 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9a8004040 con 0x56348c055330 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9a8005b30 con 0x56348c055330 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a803f050 con 0x56348c055330 2024-03-06T07:29:23.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 10 client.10968 fetch_fsmap learned FSMap version 203 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.789+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x56348c075870 con 0x56348c055330 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8005b30 con 0x56348c055330 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 203) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fe9a8042020 con 0x56348c055330 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 10 client.10968 fetch_fsmap finished waiting for FSMap version 203 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 10 client.10968 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 20 client.10968 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c08d5d0 0x56348c08d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 4 client.10968 mds_command: new command op to 9556 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 --> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56348bfc0f40 con 0x56348c08d5d0 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c08d5d0 0x56348c08d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe9c0bec640 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c08d5d0 0x56348c08d9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ebdfe76fc23cac4 server_cookie=37c0b8683c138432 in_seq=0 out_seq=0 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe99b7fe640 10 client.10968 ms_handle_connect on v2:172.21.15.163:6836/370957199 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe99b7fe640 1 -- 172.21.15.17:0/3550545871 <== mds.1 v2:172.21.15.163:6836/370957199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56348bfc0f40 con 0x56348c08d5d0 2024-03-06T07:29:23.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.790+0000 7fe99b7fe640 10 client.10968 handle_command_reply: tid=0 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 client.10968 shutdown 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 msgr2=0x56348c054330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80098c0 tx=0x7fe9a80046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 shutdown_connections 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:6836/370957199,v1:172.21.15.163:6837/370957199] conn(0x56348c08d5d0 0x56348c08d9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x56348c07aa30 0x56348c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56348c055330 0x56348c054330 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe99b7fe640 0 client.10968 ms_handle_reset on v2:172.21.15.163:6836/370957199 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 --2- 172.21.15.17:0/3550545871 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x56348c054f50 0x56348c0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 >> 172.21.15.17:0/3550545871 conn(0x56348c0534b0 msgr2=0x56348c091b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 shutdown_connections 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 1 -- 172.21.15.17:0/3550545871 wait complete. 2024-03-06T07:29:23.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c5a31740 20 client.10968 trim_cache size 0 max 16384 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 1, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 9556, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 2.7153992069999999, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 203, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 212, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 212, 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13.901947908 2024-03-06T07:29:23.799 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:29:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 msgr2=0x7fe9a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 0x7fe9a006d990 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0036000 tx=0x7fe9b003d040 comp rx=0 tx=0).stop 2024-03-06T07:29:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 msgr2=0x7fe9bc14bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400caa0 tx=0x7fe9b400cf60 comp rx=0 tx=0).stop 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 shutdown_connections 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe9a006b4d0 0x7fe9a006d990 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9bc0c8950 0x7fe9bc14bd90 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9bc0c69f0 0x7fe9bc154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 --2- 172.21.15.17:0/1371662540 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9bc0ad2b0 0x7fe9bc0aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.791+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 >> 172.21.15.17:0/1371662540 conn(0x7fe9bc0cae70 msgr2=0x7fe9bc150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.792+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 shutdown_connections 2024-03-06T07:29:23.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.792+0000 7fe9c2e50640 1 -- 172.21.15.17:0/1371662540 wait complete. 2024-03-06T07:29:23.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.910+0000 7fd577df6640 1 Processor -- start 2024-03-06T07:29:23.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.910+0000 7fd577df6640 1 -- start start 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd577df6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5700c8950 0x7fd5700c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd577df6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5701558d0 0x7fd570155d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd577df6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd570155130 con 0x7fd5700c69f0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd577df6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd570155440 con 0x7fd5700c8950 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd577df6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700ad480 con 0x7fd5701558d0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd575391640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5700c8950 0x7fd5700c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd575b92640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd5700c69f0 0x7fd5700acd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59632/0 (socket says 172.21.15.17:59632) 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd575391640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5700c8950 0x7fd5700c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59466/0 (socket says 172.21.15.17:59466) 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd575391640 1 -- 172.21.15.17:0/2215208467 learned_addr learned my addr 172.21.15.17:0/2215208467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.911+0000 7fd576393640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5701558d0 0x7fd570155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd575391640 1 -- 172.21.15.17:0/2215208467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5701558d0 msgr2=0x7fd570155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd575391640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5701558d0 0x7fd570155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd575391640 1 -- 172.21.15.17:0/2215208467 >> v1:172.21.15.17:6789/0 conn(0x7fd5700c69f0 legacy=0x7fd5700acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd575391640 1 -- 172.21.15.17:0/2215208467 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700c5d50 con 0x7fd5700c8950 2024-03-06T07:29:23.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd574b90640 1 -- 172.21.15.17:0/2215208467 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449078611 0 0) 0x7fd56c001110 con 0x7fd5700c69f0 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.912+0000 7fd575391640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5700c8950 0x7fd5700c6dd0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4c5f9b4100a79e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd574b90640 1 -- 172.21.15.17:0/2215208467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560035020 con 0x7fd5700c8950 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd574b90640 1 -- 172.21.15.17:0/2215208467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560031640 con 0x7fd5700c8950 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd574b90640 1 -- 172.21.15.17:0/2215208467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56002d040 con 0x7fd5700c8950 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 -- 172.21.15.17:0/2215208467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5700c8950 msgr2=0x7fd5700c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5700c8950 0x7fd5700c6dd0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 -- 172.21.15.17:0/2215208467 shutdown_connections 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5701558d0 0x7fd570155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 --2- 172.21.15.17:0/2215208467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5700c8950 0x7fd5700c6dd0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 -- 172.21.15.17:0/2215208467 >> 172.21.15.17:0/2215208467 conn(0x7fd5700cae70 msgr2=0x7fd5700cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 -- 172.21.15.17:0/2215208467 shutdown_connections 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.913+0000 7fd577df6640 1 -- 172.21.15.17:0/2215208467 wait complete. 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.914+0000 7fd577df6640 1 Processor -- start 2024-03-06T07:29:23.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.914+0000 7fd577df6640 1 -- start start 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5700c69f0 0x7fd57014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5700c8950 0x7fd570153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570152a20 con 0x7fd5701558d0 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c6310 con 0x7fd5700c8950 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd577df6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd570152ba0 con 0x7fd5700c69f0 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59478/0 (socket says 172.21.15.17:59478) 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 -- 172.21.15.17:0/1891175911 learned_addr learned my addr 172.21.15.17:0/1891175911 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd575b92640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5700c69f0 0x7fd57014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd575391640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5700c8950 0x7fd570153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 -- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5700c69f0 msgr2=0x7fd57014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5700c69f0 0x7fd57014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 -- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5700c8950 msgr2=0x7fd570153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5700c8950 0x7fd570153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.915+0000 7fd576393640 1 -- 172.21.15.17:0/1891175911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5701ea5c0 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd576393640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fd56800efc0 tx=0x7fd56800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd5667fc640 1 -- 172.21.15.17:0/1891175911 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568018020 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd5667fc640 1 -- 172.21.15.17:0/1891175911 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568014480 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd570154af0 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd5667fc640 1 -- 172.21.15.17:0/1891175911 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568014870 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.916+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd570158bc0 con 0x7fd5701558d0 2024-03-06T07:29:23.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.917+0000 7fd5667fc640 1 -- 172.21.15.17:0/1891175911 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd568005d10 con 0x7fd5701558d0 2024-03-06T07:29:23.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.918+0000 7fd5667fc640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.918+0000 7fd5667fc640 1 -- 172.21.15.17:0/1891175911 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd56808af80 con 0x7fd5701558d0 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.918+0000 7fd575b92640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.919+0000 7fd575b92640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd56c007fd0 tx=0x7fd56c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.919+0000 7fd57a9d6740 1 Processor -- start 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.919+0000 7fd57a9d6740 1 -- start start 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da07f50 0x557f6da05a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da08330 0x557f6da05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6d96c6b0 con 0x557f6da064b0 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6d915a60 con 0x557f6da07f50 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f6d968750 con 0x557f6da08330 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59490/0 (socket says 172.21.15.17:59490) 2024-03-06T07:29:23.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 -- 172.21.15.17:0/2554160079 learned_addr learned my addr 172.21.15.17:0/2554160079 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd575391640 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da08330 0x557f6da05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd575b92640 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da07f50 0x557f6da05a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 -- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da08330 msgr2=0x557f6da05f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da08330 0x557f6da05f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 -- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da07f50 msgr2=0x557f6da05a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da07f50 0x557f6da05a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 -- 172.21.15.17:0/2554160079 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f6d933f30 con 0x557f6da064b0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd576393640 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fd568089320 tx=0x7fd5680595b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416288d9d2842509 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd5537fe640 1 -- 172.21.15.17:0/2554160079 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568087070 con 0x557f6da064b0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd5537fe640 1 -- 172.21.15.17:0/2554160079 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5680579b0 con 0x557f6da064b0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.920+0000 7fd5537fe640 1 -- 172.21.15.17:0/2554160079 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568071070 con 0x557f6da064b0 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 -- 172.21.15.17:0/2554160079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 msgr2=0x557f6da02680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7fd568089320 tx=0x7fd5680595b0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 -- 172.21.15.17:0/2554160079 shutdown_connections 2024-03-06T07:29:23.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da064b0 0x557f6da02680 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da08330 0x557f6da05f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/2554160079 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da07f50 0x557f6da05a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 -- 172.21.15.17:0/2554160079 >> 172.21.15.17:0/2554160079 conn(0x557f6da047d0 msgr2=0x557f6da293c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 -- 172.21.15.17:0/2554160079 shutdown_connections 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.921+0000 7fd57a9d6740 1 -- 172.21.15.17:0/2554160079 wait complete. 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.922+0000 7fd57a9d6740 1 -- start start 2024-03-06T07:29:23.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.922+0000 7fd57a9d6740 1 Processor -- start 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6da2a230 con 0x557f6da08330 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6d928ea0 con 0x557f6da57dc0 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f6da29c70 con 0x557f6da07f50 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd57a9d6740 10 client.0 _ll_register_callbacks cb 0x557f6d928870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575b92640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59492/0 (socket says 172.21.15.17:59492) 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 -- 172.21.15.17:0/1974129927 learned_addr learned my addr 172.21.15.17:0/1974129927 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd576393640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 -- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 msgr2=0x557f6da3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da3c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 -- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 msgr2=0x557f6da26070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da26070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 -- 172.21.15.17:0/1974129927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f6d929270 con 0x557f6da08330 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd575391640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fd560009990 tx=0x7fd560004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae5985d03576830 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.923+0000 7fd550ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56004a060 con 0x557f6da08330 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd56003a830 con 0x557f6da08330 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5680579b0 con 0x557f6da08330 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd560050050 con 0x557f6da08330 2024-03-06T07:29:23.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 10 client.10983 fetch_fsmap learned FSMap version 203 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x557f6d6812d0 con 0x557f6da08330 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5680579b0 con 0x557f6da08330 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 203) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd560053020 con 0x557f6da08330 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 10 client.10983 fetch_fsmap finished waiting for FSMap version 203 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 10 client.10983 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 20 client.10983 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6da3bf40 0x557f6da63460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 4 client.10983 mds_command: new command op to 10791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 --> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f6d979bf0 con 0x557f6da3bf40 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.924+0000 7fd575b92640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6da3bf40 0x557f6da63460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd550ff9640 10 client.10983 ms_handle_connect on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd575b92640 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6da3bf40 0x557f6da63460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6484a519eed5b6de server_cookie=ace2e16612ef1803 in_seq=0 out_seq=0 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd550ff9640 1 -- 172.21.15.17:0/1974129927 <== mds.0 v2:172.21.15.17:6832/2434034079 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557f6d979bf0 con 0x557f6da3bf40 2024-03-06T07:29:23.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd550ff9640 10 client.10983 handle_command_reply: tid=0 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 client.10983 shutdown 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 msgr2=0x557f6da55310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7fd560009990 tx=0x7fd560004730 comp rx=0 tx=0).stop 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 shutdown_connections 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6da3bf40 0x557f6da63460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da26070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da55310 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd550ff9640 0 client.10983 ms_handle_reset on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/1974129927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 >> 172.21.15.17:0/1974129927 conn(0x557f6da064b0 msgr2=0x557f6da068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 shutdown_connections 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.925+0000 7fd57a9d6740 1 -- 172.21.15.17:0/1974129927 wait complete. 2024-03-06T07:29:23.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.926+0000 7fd57a9d6740 20 client.10983 trim_cache size 0 max 16384 2024-03-06T07:29:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.942+0000 7fd57a9d6740 1 Processor -- start 2024-03-06T07:29:23.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.942+0000 7fd57a9d6740 1 -- start start 2024-03-06T07:29:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da05790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701f7de0 con 0x557f6da08330 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701f6270 con 0x557f6da57dc0 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701f6870 con 0x557f6da07f50 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59504/0 (socket says 172.21.15.17:59504) 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 -- 172.21.15.17:0/454434589 learned_addr learned my addr 172.21.15.17:0/454434589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd576393640 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575b92640 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da05790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 -- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 msgr2=0x557f6da05790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da05790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 -- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 msgr2=0x557f6da39a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da39a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 -- 172.21.15.17:0/454434589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f6d74b6b0 con 0x557f6da08330 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.943+0000 7fd575391640 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fd5600090b0 tx=0x7fd560041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66462bbff8ad96a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd552ffd640 1 -- 172.21.15.17:0/454434589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560059070 con 0x557f6da08330 2024-03-06T07:29:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd552ffd640 1 -- 172.21.15.17:0/454434589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560004030 con 0x557f6da08330 2024-03-06T07:29:23.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd552ffd640 1 -- 172.21.15.17:0/454434589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560035040 con 0x557f6da08330 2024-03-06T07:29:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 -- 172.21.15.17:0/454434589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 msgr2=0x557f6da05cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7fd5600090b0 tx=0x7fd560041000 comp rx=0 tx=0).stop 2024-03-06T07:29:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 -- 172.21.15.17:0/454434589 shutdown_connections 2024-03-06T07:29:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da57dc0 0x557f6da39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da08330 0x557f6da05cd0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/454434589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da07f50 0x557f6da05790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 -- 172.21.15.17:0/454434589 >> 172.21.15.17:0/454434589 conn(0x557f6da064b0 msgr2=0x557f6da06890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 -- 172.21.15.17:0/454434589 shutdown_connections 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.944+0000 7fd57a9d6740 1 -- 172.21.15.17:0/454434589 wait complete. 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.945+0000 7fd57a9d6740 1 -- start start 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.945+0000 7fd57a9d6740 1 Processor -- start 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da08330 0x557f6da39ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da57dc0 0x557f6dab1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6da58200 con 0x557f6da07f50 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6da54b10 con 0x557f6da08330 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f6da3b3c0 con 0x557f6da57dc0 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd57a9d6740 10 client.0 _ll_register_callbacks cb 0x7fd560031820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-06T07:29:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59506/0 (socket says 172.21.15.17:59506) 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 -- 172.21.15.17:0/509951364 learned_addr learned my addr 172.21.15.17:0/509951364 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575391640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da08330 0x557f6da39ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd576393640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da57dc0 0x557f6dab1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 -- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da57dc0 msgr2=0x557f6dab1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da57dc0 0x557f6dab1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 -- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da08330 msgr2=0x557f6da39ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da08330 0x557f6da39ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 -- 172.21.15.17:0/509951364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f6da292b0 con 0x557f6da07f50 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd575b92640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x557f6da3e080 tx=0x7fd56c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4a7abb0a58fffa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.946+0000 7fd550ff9640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c029d00 con 0x557f6da07f50 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd56c0223f0 con 0x557f6da07f50 2024-03-06T07:29:23.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd560004030 con 0x557f6da07f50 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd56c037050 con 0x557f6da07f50 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 10 client.10989 fetch_fsmap learned FSMap version 203 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=203,monmap=2+}) v3 -- 0x557f6da3a7e0 con 0x557f6da07f50 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560004030 con 0x557f6da07f50 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 203) v1 ==== 3334+0+0 (secure 0 0 0) 0x7fd56c03a020 con 0x557f6da07f50 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 10 client.10989 fetch_fsmap finished waiting for FSMap version 203 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 10 client.10989 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 20 client.10989 populate_metadata read hostname 'smithi017' 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6dab7a60 0x557f6dab7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 4 client.10989 mds_command: new command op to 10791 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 --> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f6d973f40 con 0x557f6dab7a60 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.947+0000 7fd575391640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6dab7a60 0x557f6dab7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd575391640 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6dab7a60 0x557f6dab7e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b293e862ceae99fc server_cookie=612c89bbe15c6e9e in_seq=0 out_seq=0 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd550ff9640 10 client.10989 ms_handle_connect on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:23.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd550ff9640 1 -- 172.21.15.17:0/509951364 <== mds.0 v2:172.21.15.17:6832/2434034079 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x557f6d973f40 con 0x557f6dab7a60 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd550ff9640 10 client.10989 handle_command_reply: tid=0 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 client.10989 shutdown 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 msgr2=0x557f6da67d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x557f6da3e080 tx=0x7fd56c013000 comp rx=0 tx=0).stop 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 shutdown_connections 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:6832/2434034079,v1:172.21.15.17:6834/2434034079] conn(0x557f6dab7a60 0x557f6dab7e40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x557f6da57dc0 0x557f6dab1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd550ff9640 0 client.10989 ms_handle_reset on v2:172.21.15.17:6832/2434034079 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x557f6da08330 0x557f6da39ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 --2- 172.21.15.17:0/509951364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557f6da07f50 0x557f6da67d90 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 >> 172.21.15.17:0/509951364 conn(0x557f6da064b0 msgr2=0x557f6da06fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 shutdown_connections 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 1 -- 172.21.15.17:0/509951364 wait complete. 2024-03-06T07:29:23.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.948+0000 7fd57a9d6740 20 client.10989 trim_cache size 0 max 16384 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5fd8a41a-f84b-4e64-ac3e-5a25cd35578f", 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 10791, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 8.9363060250000004, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 203, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 212, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 212, 2024-03-06T07:29:23.956 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20.033585131999999 2024-03-06T07:29:23.957 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:29:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 msgr2=0x7fd55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd56c007fd0 tx=0x7fd56c007de0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 msgr2=0x7fd5700c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fd56800efc0 tx=0x7fd56800c490 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 shutdown_connections 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5701558d0 0x7fd5700c4380 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5700c8950 0x7fd570153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 --2- 172.21.15.17:0/1891175911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5700c69f0 0x7fd57014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 >> 172.21.15.17:0/1891175911 conn(0x7fd5700cae70 msgr2=0x7fd5700acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 shutdown_connections 2024-03-06T07:29:23.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:23.949+0000 7fd577df6640 1 -- 172.21.15.17:0/1891175911 wait complete. 2024-03-06T07:29:23.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs down true 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.067+0000 7fa02e8c2640 1 Processor -- start 2024-03-06T07:29:24.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.067+0000 7fa02e8c2640 1 -- start start 2024-03-06T07:29:24.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.068+0000 7fa02e8c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0280a8290 0x7fa0281539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.068+0000 7fa02e8c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280c62a0 con 0x7fa0280c4740 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.068+0000 7fa02e8c2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0280c4c00 con 0x7fa0280c69a0 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.068+0000 7fa02e8c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa028153f00 con 0x7fa0280a8290 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0280a8290 0x7fa0281539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0280a8290 0x7fa0281539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59512/0 (socket says 172.21.15.17:59512) 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 -- 172.21.15.17:0/1959714339 learned_addr learned my addr 172.21.15.17:0/1959714339 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa0277fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa0280c4740 0x7fa0280701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59640/0 (socket says 172.21.15.17:59640) 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 -- 172.21.15.17:0/1959714339 >> v1:172.21.15.163:6789/0 conn(0x7fa0280c69a0 legacy=0x7fa028072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa026ffd640 1 -- 172.21.15.17:0/1959714339 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895680583 0 0) 0x7fa0280c4c00 con 0x7fa0280c69a0 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 -- 172.21.15.17:0/1959714339 >> v1:172.21.15.17:6789/0 conn(0x7fa0280c4740 legacy=0x7fa0280701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.069+0000 7fa027fff640 1 -- 172.21.15.17:0/1959714339 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0280c4ee0 con 0x7fa0280a8290 2024-03-06T07:29:24.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.070+0000 7fa027fff640 1 --2- 172.21.15.17:0/1959714339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa0281539c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa01802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95789df86db63832 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.070+0000 7fa026ffd640 1 -- 172.21.15.17:0/1959714339 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018035020 con 0x7fa0280a8290 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.070+0000 7fa026ffd640 1 -- 172.21.15.17:0/1959714339 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018031870 con 0x7fa0280a8290 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.070+0000 7fa026ffd640 1 -- 172.21.15.17:0/1959714339 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018031c40 con 0x7fa0280a8290 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 -- 172.21.15.17:0/1959714339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 msgr2=0x7fa0281539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/1959714339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa0281539c0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa01802b910 comp rx=0 tx=0).stop 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 -- 172.21.15.17:0/1959714339 shutdown_connections 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/1959714339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa0281539c0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 -- 172.21.15.17:0/1959714339 >> 172.21.15.17:0/1959714339 conn(0x7fa0280c5560 msgr2=0x7fa0280c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 -- 172.21.15.17:0/1959714339 shutdown_connections 2024-03-06T07:29:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.071+0000 7fa02e8c2640 1 -- 172.21.15.17:0/1959714339 wait complete. 2024-03-06T07:29:24.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.072+0000 7fa02e8c2640 1 Processor -- start 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.072+0000 7fa02e8c2640 1 -- start start 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0280c4740 0x7fa02814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0280c69a0 0x7fa0281ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028072d90 con 0x7fa0280a8290 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02806fc80 con 0x7fa0280c4740 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02e8c2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa028072f10 con 0x7fa0280c69a0 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59520/0 (socket says 172.21.15.17:59520) 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 -- 172.21.15.17:0/4191559648 learned_addr learned my addr 172.21.15.17:0/4191559648 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa0277fe640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0280c4740 0x7fa02814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa02ce5f640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0280c69a0 0x7fa0281ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 -- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0280c69a0 msgr2=0x7fa0281ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0280c69a0 0x7fa0281ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 -- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0280c4740 msgr2=0x7fa02814d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0280c4740 0x7fa02814d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 -- 172.21.15.17:0/4191559648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02806c840 con 0x7fa0280a8290 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.073+0000 7fa027fff640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa0180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.074+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018040070 con 0x7fa0280a8290 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.074+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01803a420 con 0x7fa0280a8290 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.074+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018035040 con 0x7fa0280a8290 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.074+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02814cfe0 con 0x7fa0280a8290 2024-03-06T07:29:24.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.074+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02806cd90 con 0x7fa0280a8290 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.075+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02804f330 con 0x7fa0280a8290 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.075+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa01804b060 con 0x7fa0280a8290 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.075+0000 7fa024ff9640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 0x7fa01406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.076+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0180ac120 con 0x7fa0280a8290 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.076+0000 7fa0277fe640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 0x7fa01406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:24.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.076+0000 7fa0277fe640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 0x7fa01406d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa0281e05d0 tx=0x7fa01c007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:24.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.080+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa01807a850 con 0x7fa0280a8290 2024-03-06T07:29:24.299 INFO:tasks.ceph.mds.c.smithi017.stderr:2024-03-06T07:29:24.297+0000 7f56818ff640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-06T07:29:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:24.300+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"} v 0) v1 -- 0x7fa0280c8480 con 0x7fa0280a8290 2024-03-06T07:29:25.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.087+0000 7fa024ff9640 1 -- 172.21.15.17:0/4191559648 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "down", "val": "true"}]=0 cephfs marked down. v204) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa01807e8f0 con 0x7fa0280a8290 2024-03-06T07:29:25.089 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked down. 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 msgr2=0x7fa01406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 0x7fa01406d990 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa0281e05d0 tx=0x7fa01c007e80 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 msgr2=0x7fa02814d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa0180046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 shutdown_connections 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa01406b4d0 0x7fa01406d990 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0280c69a0 0x7fa0281ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0280c4740 0x7fa02814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 --2- 172.21.15.17:0/4191559648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0280a8290 0x7fa02814d470 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 >> 172.21.15.17:0/4191559648 conn(0x7fa0280c5560 msgr2=0x7fa028152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:25.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 shutdown_connections 2024-03-06T07:29:25.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.090+0000 7fa02e8c2640 1 -- 172.21.15.17:0/4191559648 wait complete. 2024-03-06T07:29:25.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:25.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.202+0000 7f547785c640 1 Processor -- start 2024-03-06T07:29:25.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.202+0000 7f547785c640 1 -- start start 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f547785c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f5470159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f547785c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700c7c60 con 0x7f54700c88b0 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f547785c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54700c8060 con 0x7f54700c7180 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f547785c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5470159640 con 0x7f54701527b0 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f5475df9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f54700c88b0 0x7f54700c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59642/0 (socket says 172.21.15.17:59642) 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.203+0000 7f5475df9640 1 -- 172.21.15.17:0/3032245469 learned_addr learned my addr 172.21.15.17:0/3032245469 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f5470159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813993574 0 0) 0x7f5468001490 con 0x7f54700c88b0 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f545c003520 con 0x7f54700c88b0 2024-03-06T07:29:25.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115664250 0 0) 0x7f54700c8060 con 0x7f54700c7180 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5468001490 con 0x7f54700c7180 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066631401 0 0) 0x7f545c003520 con 0x7f54700c88b0 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 msgr2=0x7f5470159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f5470159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 >> v1:172.21.15.163:6789/0 conn(0x7f54700c7180 legacy=0x7f5470155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700c76a0 con 0x7f54700c88b0 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f5468001bc0 con 0x7f54700c88b0 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f5470159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 >> v1:172.21.15.17:6789/0 conn(0x7f54700c88b0 legacy=0x7f54700c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f545c003b70 0x7f545c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f545c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c004d40 0x7f545c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470159640 con 0x7f545c003b70 2024-03-06T07:29:25.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c00baa0 con 0x7f545c004d40 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c00bca0 con 0x7f54701527b0 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f545c003b70 0x7f545c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5475df9640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c004d40 0x7f545c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5474df7640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f545c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 -- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 msgr2=0x7f545c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f545c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 -- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c004d40 msgr2=0x7f545c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f545c004d40 0x7f545c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f54755f8640 1 -- 172.21.15.17:0/3032245469 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c003520 con 0x7f545c003b70 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.204+0000 7f5474df7640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54701527b0 0x7f545c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.205+0000 7f54755f8640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f545c003b70 0x7f545c003f70 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f546c009c90 tx=0x7f545c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67e60c0ac0e274c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.205+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c039070 con 0x7f545c003b70 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.205+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c02c8f0 con 0x7f545c003b70 2024-03-06T07:29:25.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.205+0000 7f5467fff640 1 -- 172.21.15.17:0/3032245469 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c009860 con 0x7f545c003b70 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 -- 172.21.15.17:0/3032245469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f545c003b70 msgr2=0x7f545c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 --2- 172.21.15.17:0/3032245469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f545c003b70 0x7f545c003f70 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f546c009c90 tx=0x7f545c005830 comp rx=0 tx=0).stop 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f5475df9640 1 -- 172.21.15.17:0/3032245469 reap_dead start 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 -- 172.21.15.17:0/3032245469 shutdown_connections 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 -- 172.21.15.17:0/3032245469 >> 172.21.15.17:0/3032245469 conn(0x7f54700ac560 msgr2=0x7f54700ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:25.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 -- 172.21.15.17:0/3032245469 shutdown_connections 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.206+0000 7f547785c640 1 -- 172.21.15.17:0/3032245469 wait complete. 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.207+0000 7f547785c640 1 Processor -- start 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.207+0000 7f547785c640 1 -- start start 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f545c003b70 0x7f5470150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54700c1a80 0x7f54700ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470155600 con 0x7f54700cad50 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470155320 con 0x7f54700c1a80 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54700c8d60 con 0x7f545c003b70 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59534/0 (socket says 172.21.15.17:59534) 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 -- 172.21.15.17:0/3244656176 learned_addr learned my addr 172.21.15.17:0/3244656176 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f54755f8640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f545c003b70 0x7f5470150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5474df7640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54700c1a80 0x7f54700ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 -- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f545c003b70 msgr2=0x7f5470150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f545c003b70 0x7f5470150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 -- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54700c1a80 msgr2=0x7f54700ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54700c1a80 0x7f54700ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 -- 172.21.15.17:0/3244656176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54701514d0 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5475df9640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f54700c4cb0 tx=0x7f5468007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468017020 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5468007a70 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54701eb050 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.208+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468013630 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.209+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547014f8f0 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.210+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547014faf0 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.210+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f546801a020 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.211+0000 7f5465ffb640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 0x7f545406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.211+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f546808aa50 con 0x7f54700cad50 2024-03-06T07:29:25.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.211+0000 7f54755f8640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 0x7f545406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:25.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.211+0000 7f54755f8640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 0x7f545406d9c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f546c036000 tx=0x7f546c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:25.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.215+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f547014faf0 con 0x7f54700cad50 2024-03-06T07:29:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.437+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547004f330 con 0x7f54700cad50 2024-03-06T07:29:25.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.437+0000 7f5465ffb640 1 -- 172.21.15.17:0/3244656176 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f5468021020 con 0x7f54700cad50 2024-03-06T07:29:25.440 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:25.440 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:25.440 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.439+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 msgr2=0x7f545406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.439+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 0x7f545406d9c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f546c036000 tx=0x7f546c030000 comp rx=0 tx=0).stop 2024-03-06T07:29:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.439+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 msgr2=0x7f547014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:25.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.439+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f54700c4cb0 tx=0x7f5468007430 comp rx=0 tx=0).stop 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.439+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 shutdown_connections 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f545406b500 0x7f545406d9c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f54700cad50 0x7f547014fe10 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54700c1a80 0x7f54700ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 --2- 172.21.15.17:0/3244656176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f545c003b70 0x7f5470150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 >> 172.21.15.17:0/3244656176 conn(0x7f54700ac560 msgr2=0x7f547014ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 shutdown_connections 2024-03-06T07:29:25.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:25.440+0000 7f547785c640 1 -- 172.21.15.17:0/3244656176 wait complete. 2024-03-06T07:29:25.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:25.855 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:26.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:26.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.955+0000 7fa61e46d640 1 Processor -- start 2024-03-06T07:29:26.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.955+0000 7fa61e46d640 1 -- start start 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa61e46d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa61814f360 0x7fa618155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa61e46d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6180c73d0 con 0x7fa6180c88c0 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa61e46d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6180be2e0 con 0x7fa618158b60 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa61e46d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa618159700 con 0x7fa61814f360 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa617fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa61814f360 0x7fa618155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa6177fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa6180c88c0 0x7fa6180c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59658/0 (socket says 172.21.15.17:59658) 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa6177fe640 1 -- 172.21.15.17:0/2455558440 learned_addr learned my addr 172.21.15.17:0/2455558440 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa617fff640 1 -- 172.21.15.17:0/2455558440 >> v1:172.21.15.163:6789/0 conn(0x7fa618158b60 legacy=0x7fa618158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa617fff640 1 -- 172.21.15.17:0/2455558440 >> v1:172.21.15.17:6789/0 conn(0x7fa6180c88c0 legacy=0x7fa6180c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa617fff640 1 -- 172.21.15.17:0/2455558440 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6180c7670 con 0x7fa61814f360 2024-03-06T07:29:26.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.956+0000 7fa616ffd640 1 -- 172.21.15.17:0/2455558440 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731004101 0 0) 0x7fa6180c73d0 con 0x7fa6180c88c0 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.957+0000 7fa617fff640 1 --2- 172.21.15.17:0/2455558440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61814f360 0x7fa618155c10 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097a0 tx=0x7fa60802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9507b7900825a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.957+0000 7fa616ffd640 1 -- 172.21.15.17:0/2455558440 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080315a0 con 0x7fa61814f360 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.957+0000 7fa616ffd640 1 -- 172.21.15.17:0/2455558440 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608037070 con 0x7fa61814f360 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.957+0000 7fa616ffd640 1 -- 172.21.15.17:0/2455558440 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608031960 con 0x7fa61814f360 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 -- 172.21.15.17:0/2455558440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61814f360 msgr2=0x7fa618155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 --2- 172.21.15.17:0/2455558440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61814f360 0x7fa618155c10 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097a0 tx=0x7fa60802b840 comp rx=0 tx=0).stop 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 -- 172.21.15.17:0/2455558440 shutdown_connections 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 --2- 172.21.15.17:0/2455558440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61814f360 0x7fa618155c10 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 -- 172.21.15.17:0/2455558440 >> 172.21.15.17:0/2455558440 conn(0x7fa6180ac570 msgr2=0x7fa6180ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 -- 172.21.15.17:0/2455558440 shutdown_connections 2024-03-06T07:29:26.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.958+0000 7fa61e46d640 1 -- 172.21.15.17:0/2455558440 wait complete. 2024-03-06T07:29:26.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.959+0000 7fa61e46d640 1 Processor -- start 2024-03-06T07:29:26.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.959+0000 7fa61e46d640 1 -- start start 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa61814f360 0x7fa6181ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 0x7fa6181eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c7c30 con 0x7fa6180c88c0 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c5a50 con 0x7fa61814f360 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180c5bd0 con 0x7fa618158b60 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59552/0 (socket says 172.21.15.17:59552) 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 -- 172.21.15.17:0/3583892399 learned_addr learned my addr 172.21.15.17:0/3583892399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa6177fe640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa61814f360 0x7fa6181ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61ca0a640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 0x7fa6181eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 -- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 msgr2=0x7fa6181eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 0x7fa6181eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 -- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa61814f360 msgr2=0x7fa6181ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa61814f360 0x7fa6181ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 -- 172.21.15.17:0/3583892399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6181514c0 con 0x7fa6180c88c0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61ca0a640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 0x7fa6181eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa617fff640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fa608034040 tx=0x7fa6080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608004230 con 0x7fa6180c88c0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6080043f0 con 0x7fa6180c88c0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6181ef280 con 0x7fa6180c88c0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608032880 con 0x7fa6180c88c0 2024-03-06T07:29:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.960+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6181517b0 con 0x7fa6180c88c0 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.961+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa618075300 con 0x7fa6180c88c0 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.962+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa608032a20 con 0x7fa6180c88c0 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.962+0000 7fa614ff9640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 0x7fa60406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.962+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa60802f030 con 0x7fa6180c88c0 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.963+0000 7fa6177fe640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 0x7fa60406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:26.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.963+0000 7fa6177fe640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 0x7fa60406d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa6180c94d0 tx=0x7fa60c007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:26.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:26.967+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa618075300 con 0x7fa6180c88c0 2024-03-06T07:29:27.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.274+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa61804f330 con 0x7fa6180c88c0 2024-03-06T07:29:27.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.274+0000 7fa614ff9640 1 -- 172.21.15.17:0/3583892399 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fa608041070 con 0x7fa6180c88c0 2024-03-06T07:29:27.276 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:27.276 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 msgr2=0x7fa60406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 0x7fa60406d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa6180c94d0 tx=0x7fa60c007d50 comp rx=0 tx=0).stop 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 msgr2=0x7fa6180bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7fa608034040 tx=0x7fa6080046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 shutdown_connections 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa60406b4d0 0x7fa60406d990 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa618158b60 0x7fa6181eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa61814f360 0x7fa6181ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 --2- 172.21.15.17:0/3583892399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6180c88c0 0x7fa6180bef00 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 >> 172.21.15.17:0/3583892399 conn(0x7fa6180ac570 msgr2=0x7fa618073e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 shutdown_connections 2024-03-06T07:29:27.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:27.276+0000 7fa61e46d640 1 -- 172.21.15.17:0/3583892399 wait complete. 2024-03-06T07:29:27.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:27.691 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:28.692 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:28.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.793+0000 7f7641572640 1 Processor -- start 2024-03-06T07:29:28.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.793+0000 7f7641572640 1 -- start start 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f7641572640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f763c0c5480 0x7f763c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f7641572640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f763c0c5860 0x7f763c06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f7641572640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0c5cc0 con 0x7f763c1586f0 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f7641572640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c06df40 con 0x7f763c0c5860 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f7641572640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c1591c0 con 0x7f763c0c5480 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f763c0c5860 0x7f763c06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f763c0c5860 0x7f763c06f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59558/0 (socket says 172.21.15.17:59558) 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 -- 172.21.15.17:0/300867772 learned_addr learned my addr 172.21.15.17:0/300867772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 -- 172.21.15.17:0/300867772 >> v2:172.21.15.163:3300/0 conn(0x7f763c0c5480 msgr2=0x7f763c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 --2- 172.21.15.17:0/300867772 >> v2:172.21.15.163:3300/0 conn(0x7f763c0c5480 0x7f763c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 -- 172.21.15.17:0/300867772 >> v1:172.21.15.163:6789/0 conn(0x7f763c1586f0 legacy=0x7f763c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.794+0000 7f763a7fc640 1 -- 172.21.15.17:0/300867772 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c1594a0 con 0x7f763c0c5860 2024-03-06T07:29:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.795+0000 7f763a7fc640 1 --2- 172.21.15.17:0/300867772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5860 0x7f763c06f6a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f763c0c1cf0 tx=0x7f763002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf744d693cfd9b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.795+0000 7f7639ffb640 1 -- 172.21.15.17:0/300867772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763002e070 con 0x7f763c0c5860 2024-03-06T07:29:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.795+0000 7f7639ffb640 1 -- 172.21.15.17:0/300867772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7630037070 con 0x7f763c0c5860 2024-03-06T07:29:28.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.795+0000 7f7639ffb640 1 -- 172.21.15.17:0/300867772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7630031a00 con 0x7f763c0c5860 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 -- 172.21.15.17:0/300867772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5860 msgr2=0x7f763c06f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 --2- 172.21.15.17:0/300867772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5860 0x7f763c06f6a0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f763c0c1cf0 tx=0x7f763002b880 comp rx=0 tx=0).stop 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 -- 172.21.15.17:0/300867772 shutdown_connections 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 --2- 172.21.15.17:0/300867772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5860 0x7f763c06f6a0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 --2- 172.21.15.17:0/300867772 >> v2:172.21.15.163:3300/0 conn(0x7f763c0c5480 0x7f763c155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 -- 172.21.15.17:0/300867772 >> 172.21.15.17:0/300867772 conn(0x7f763c06b1a0 msgr2=0x7f763c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 -- 172.21.15.17:0/300867772 shutdown_connections 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.796+0000 7f7641572640 1 -- 172.21.15.17:0/300867772 wait complete. 2024-03-06T07:29:28.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.798+0000 7f7641572640 1 Processor -- start 2024-03-06T07:29:28.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.798+0000 7f7641572640 1 -- start start 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 0x7f763c1eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f763c1586f0 0x7f763c1eb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c152a20 con 0x7f763c0c5480 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c155680 con 0x7f763c0c5860 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f7641572640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c155050 con 0x7f763c1586f0 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59562/0 (socket says 172.21.15.17:59562) 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 -- 172.21.15.17:0/2098719908 learned_addr learned my addr 172.21.15.17:0/2098719908 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763a7fc640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 0x7f763c1eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763b7fe640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f763c1586f0 0x7f763c1eb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 -- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f763c1586f0 msgr2=0x7f763c1eb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f763c1586f0 0x7f763c1eb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 -- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 msgr2=0x7f763c1eb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 0x7f763c1eb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 -- 172.21.15.17:0/2098719908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c1ee8f0 con 0x7f763c0c5480 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763a7fc640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 0x7f763c1eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f763affd640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f762c00ba50 tx=0x7f762c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.799+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c019070 con 0x7f763c0c5480 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.800+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c00cc00 con 0x7f763c0c5480 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.800+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c1eebd0 con 0x7f763c0c5480 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.800+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c014440 con 0x7f763c0c5480 2024-03-06T07:29:28.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.800+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c1ef190 con 0x7f763c0c5480 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.801+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c04f330 con 0x7f763c0c5480 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.801+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f762c01c070 con 0x7f763c0c5480 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.802+0000 7f76237fe640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 0x7f762806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.802+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f762c08a170 con 0x7f763c0c5480 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.802+0000 7f763a7fc640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 0x7f762806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:28.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.802+0000 7f763a7fc640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 0x7f762806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f7630008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:28.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:28.806+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f762c058e60 con 0x7f763c0c5480 2024-03-06T07:29:29.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.026+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f763c06e0c0 con 0x7f763c0c5480 2024-03-06T07:29:29.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.027+0000 7f76237fe640 1 -- 172.21.15.17:0/2098719908 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f762c058760 con 0x7f763c0c5480 2024-03-06T07:29:29.028 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:29.029 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:29.029 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:29.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 msgr2=0x7f762806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 0x7f762806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7630036000 tx=0x7f7630008030 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 msgr2=0x7f763c0c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f762c00ba50 tx=0x7f762c00bf10 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 shutdown_connections 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f762806b4d0 0x7f762806d990 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f763c1586f0 0x7f763c1eb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f763c0c5860 0x7f763c1eb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 --2- 172.21.15.17:0/2098719908 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f763c0c5480 0x7f763c0c4780 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 >> 172.21.15.17:0/2098719908 conn(0x7f763c06b1a0 msgr2=0x7f763c0c8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 shutdown_connections 2024-03-06T07:29:29.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:29.029+0000 7f7641572640 1 -- 172.21.15.17:0/2098719908 wait complete. 2024-03-06T07:29:29.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:29.444 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:30.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:30.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.544+0000 7f0fa4102640 1 Processor -- start 2024-03-06T07:29:30.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.544+0000 7f0fa4102640 1 -- start start 2024-03-06T07:29:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa4102640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0f9c14f360 0x7f0f9c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa4102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 0x7f0f9c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa4102640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9c0c73d0 con 0x7f0f9c0c5880 2024-03-06T07:29:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa4102640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9c0c8dd0 con 0x7f0f9c14f360 2024-03-06T07:29:30.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa4102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0be2e0 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa269f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0f9c0c5880 0x7f0f9c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35764/0 (socket says 172.21.15.17:35764) 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa269f640 1 -- 172.21.15.17:0/1025358667 learned_addr learned my addr 172.21.15.17:0/1025358667 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa1e9e640 1 --2- 172.21.15.17:0/1025358667 >> v2:172.21.15.163:3300/0 conn(0x7f0f9c14f360 0x7f0f9c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa169d640 1 --2- 172.21.15.17:0/1025358667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 0x7f0f9c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200879348 0 0) 0x7f0f9c0c73d0 con 0x7f0f9c0c5880 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f88003520 con 0x7f0f9c0c5880 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa169d640 1 -- 172.21.15.17:0/1025358667 >> v2:172.21.15.163:3300/0 conn(0x7f0f9c14f360 msgr2=0x7f0f9c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa169d640 1 --2- 172.21.15.17:0/1025358667 >> v2:172.21.15.163:3300/0 conn(0x7f0f9c14f360 0x7f0f9c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa169d640 1 -- 172.21.15.17:0/1025358667 >> v1:172.21.15.17:6789/0 conn(0x7f0f9c0c5880 legacy=0x7f0f9c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2777586492 0 0) 0x7f0f88003520 con 0x7f0f9c0c5880 2024-03-06T07:29:30.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.545+0000 7f0fa169d640 1 -- 172.21.15.17:0/1025358667 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0c7670 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.546+0000 7f0fa169d640 1 --2- 172.21.15.17:0/1025358667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 0x7f0f9c0c5340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0097a0 tx=0x7f0f8c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a1ae2246d95cef server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.546+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c037070 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.546+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c030510 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.546+0000 7f0fa0e9c640 1 -- 172.21.15.17:0/1025358667 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940019f0 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 -- 172.21.15.17:0/1025358667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 msgr2=0x7f0f9c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 --2- 172.21.15.17:0/1025358667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 0x7f0f9c0c5340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0097a0 tx=0x7f0f8c02b710 comp rx=0 tx=0).stop 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 -- 172.21.15.17:0/1025358667 shutdown_connections 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 --2- 172.21.15.17:0/1025358667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c88c0 0x7f0f9c0c5340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 --2- 172.21.15.17:0/1025358667 >> v2:172.21.15.163:3300/0 conn(0x7f0f9c14f360 0x7f0f9c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 -- 172.21.15.17:0/1025358667 >> 172.21.15.17:0/1025358667 conn(0x7f0f9c0ac570 msgr2=0x7f0f9c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 -- 172.21.15.17:0/1025358667 shutdown_connections 2024-03-06T07:29:30.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.547+0000 7f0fa4102640 1 -- 172.21.15.17:0/1025358667 wait complete. 2024-03-06T07:29:30.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.549+0000 7f0fa4102640 1 Processor -- start 2024-03-06T07:29:30.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.549+0000 7f0fa4102640 1 -- start start 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c5880 0x7f0f9c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f9c14f360 0x7f0f9c0be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c159270 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c0c7c30 con 0x7f0f9c14f360 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa4102640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c1593f0 con 0x7f0f9c0c5880 2024-03-06T07:29:30.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59340/0 (socket says 172.21.15.17:59340) 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 -- 172.21.15.17:0/2503919073 learned_addr learned my addr 172.21.15.17:0/2503919073 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa1e9e640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c5880 0x7f0f9c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa269f640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f9c14f360 0x7f0f9c0be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 -- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c5880 msgr2=0x7f0f9c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c5880 0x7f0f9c0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 -- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f9c14f360 msgr2=0x7f0f9c0be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f9c14f360 0x7f0f9c0be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.550+0000 7f0fa169d640 1 -- 172.21.15.17:0/2503919073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0b02a0 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0fa169d640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000c00 tx=0x7f0f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c041020 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c03c870 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c0c4c60 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c037040 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.551+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c0ad710 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.552+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c04f330 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.552+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f8c044020 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.553+0000 7f0f927fc640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.553+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f8c0abe60 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.553+0000 7f0fa1e9e640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.554+0000 7f0fa1e9e640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0c4eb0 tx=0x7f0f98008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.557+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f8c04b030 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.779+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f9c06cda0 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.779+0000 7f0f927fc640 1 -- 172.21.15.17:0/2503919073 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f0f8c07ab40 con 0x7f0f9c0c88c0 2024-03-06T07:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:30.782 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 msgr2=0x7f0f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0c4eb0 tx=0x7f0f98008040 comp rx=0 tx=0).stop 2024-03-06T07:29:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 msgr2=0x7f0f9c157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c000c00 tx=0x7f0f8c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 shutdown_connections 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f9c14f360 0x7f0f9c0be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f9c0c88c0 0x7f0f9c157b70 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 --2- 172.21.15.17:0/2503919073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f9c0c5880 0x7f0f9c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 >> 172.21.15.17:0/2503919073 conn(0x7f0f9c0ac570 msgr2=0x7f0f9c0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 shutdown_connections 2024-03-06T07:29:30.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:30.782+0000 7f0fa4102640 1 -- 172.21.15.17:0/2503919073 wait complete. 2024-03-06T07:29:31.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:31.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:32.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:32.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.298+0000 7f7daa3d7640 1 Processor -- start 2024-03-06T07:29:32.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.298+0000 7f7daa3d7640 1 -- start start 2024-03-06T07:29:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7daa3d7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7da40c5860 0x7f7da41609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7daa3d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da41591c0 con 0x7f7da4155ba0 2024-03-06T07:29:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7daa3d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da41594a0 con 0x7f7da40c5480 2024-03-06T07:29:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7daa3d7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da406df40 con 0x7f7da40c5860 2024-03-06T07:29:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7da37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7da4155ba0 0x7f7da41586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35766/0 (socket says 172.21.15.17:35766) 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7da37fe640 1 -- 172.21.15.17:0/1231824404 learned_addr learned my addr 172.21.15.17:0/1231824404 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7da8974640 1 --2- 172.21.15.17:0/1231824404 >> v2:172.21.15.163:3300/0 conn(0x7f7da40c5860 0x7f7da41609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260163432 0 0) 0x7f7da41591c0 con 0x7f7da4155ba0 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.299+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d90003520 con 0x7f7da4155ba0 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.300+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3852686162 0 0) 0x7f7d90003520 con 0x7f7da4155ba0 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.300+0000 7f7da8974640 1 -- 172.21.15.17:0/1231824404 >> v1:172.21.15.163:6789/0 conn(0x7f7da40c5480 legacy=0x7f7da406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.300+0000 7f7da8974640 1 -- 172.21.15.17:0/1231824404 >> v1:172.21.15.17:6789/0 conn(0x7f7da4155ba0 legacy=0x7f7da41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.300+0000 7f7da8974640 1 -- 172.21.15.17:0/1231824404 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da41553a0 con 0x7f7da40c5860 2024-03-06T07:29:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.300+0000 7f7da8974640 1 --2- 172.21.15.17:0/1231824404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da40c5860 0x7f7da41609e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d217e9387744bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c039070 con 0x7f7da40c5860 2024-03-06T07:29:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c02c8b0 con 0x7f7da40c5860 2024-03-06T07:29:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7da2ffd640 1 -- 172.21.15.17:0/1231824404 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c001670 con 0x7f7da40c5860 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 -- 172.21.15.17:0/1231824404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da40c5860 msgr2=0x7f7da41609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/1231824404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da40c5860 0x7f7da41609e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009d90 tx=0x7f7d9c02bd80 comp rx=0 tx=0).stop 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 -- 172.21.15.17:0/1231824404 shutdown_connections 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/1231824404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da40c5860 0x7f7da41609e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 -- 172.21.15.17:0/1231824404 >> 172.21.15.17:0/1231824404 conn(0x7f7da406b1a0 msgr2=0x7f7da406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 -- 172.21.15.17:0/1231824404 shutdown_connections 2024-03-06T07:29:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.301+0000 7f7daa3d7640 1 -- 172.21.15.17:0/1231824404 wait complete. 2024-03-06T07:29:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.302+0000 7f7daa3d7640 1 Processor -- start 2024-03-06T07:29:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 -- start start 2024-03-06T07:29:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7da40c5860 0x7f7da414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 0x7f7da41de6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c5cf0 con 0x7f7da40c5480 2024-03-06T07:29:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da406fdb0 con 0x7f7da4155ba0 2024-03-06T07:29:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7daa3d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da4156000 con 0x7f7da40c5860 2024-03-06T07:29:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7da3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59346/0 (socket says 172.21.15.17:59346) 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.303+0000 7f7da3fff640 1 -- 172.21.15.17:0/870394044 learned_addr learned my addr 172.21.15.17:0/870394044 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da8974640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 0x7f7da41de6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da37fe640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7da40c5860 0x7f7da414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 -- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7da40c5860 msgr2=0x7f7da414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7da40c5860 0x7f7da414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 -- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 msgr2=0x7f7da41de6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 0x7f7da41de6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 -- 172.21.15.17:0/870394044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da414f210 con 0x7f7da40c5480 2024-03-06T07:29:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da8974640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 0x7f7da41de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:32.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da3fff640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c450 tx=0x7f7d9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:32.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d940052d0 con 0x7f7da40c5480 2024-03-06T07:29:32.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d94005470 con 0x7f7da40c5480 2024-03-06T07:29:32.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da41eae40 con 0x7f7da40c5480 2024-03-06T07:29:32.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d94012870 con 0x7f7da40c5480 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.304+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da41eb400 con 0x7f7da40c5480 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.305+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da404f330 con 0x7f7da40c5480 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.306+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d94012a10 con 0x7f7da40c5480 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.306+0000 7f7da0ff9640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 0x7f7d8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.306+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d94089f40 con 0x7f7da40c5480 2024-03-06T07:29:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.306+0000 7f7da37fe640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 0x7f7d8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.307+0000 7f7da37fe640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 0x7f7d8406d990 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005880 tx=0x7f7d8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.310+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d94058cc0 con 0x7f7da40c5480 2024-03-06T07:29:32.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.531+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7da406e0c0 con 0x7f7da40c5480 2024-03-06T07:29:32.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.532+0000 7f7da0ff9640 1 -- 172.21.15.17:0/870394044 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f7d940585c0 con 0x7f7da40c5480 2024-03-06T07:29:32.533 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:32.534 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:32.534 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 msgr2=0x7f7d8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 0x7f7d8406d990 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c005880 tx=0x7f7d8c009040 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 msgr2=0x7f7da41571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400c450 tx=0x7f7d9400c910 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 shutdown_connections 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d8406b4d0 0x7f7d8406d990 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7da4155ba0 0x7f7da41de6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7da40c5860 0x7f7da414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 --2- 172.21.15.17:0/870394044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7da40c5480 0x7f7da41571d0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 >> 172.21.15.17:0/870394044 conn(0x7f7da406b1a0 msgr2=0x7f7da40c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 shutdown_connections 2024-03-06T07:29:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:32.534+0000 7f7daa3d7640 1 -- 172.21.15.17:0/870394044 wait complete. 2024-03-06T07:29:32.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:32.950 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:33.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.051+0000 7fadca884640 1 Processor -- start 2024-03-06T07:29:34.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.051+0000 7fadca884640 1 -- start start 2024-03-06T07:29:34.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadca884640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadc40c5330 0x7fadc40c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadca884640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc4155110 con 0x7fadc4158db0 2024-03-06T07:29:34.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadca884640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc40c8f40 con 0x7fadc40c88b0 2024-03-06T07:29:34.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadca884640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc41559e0 con 0x7fadc40c5330 2024-03-06T07:29:34.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadc8e21640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fadc4158db0 0x7fadc415fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35780/0 (socket says 172.21.15.17:35780) 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadc8e21640 1 -- 172.21.15.17:0/217582601 learned_addr learned my addr 172.21.15.17:0/217582601 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadc3fff640 1 --2- 172.21.15.17:0/217582601 >> v2:172.21.15.163:3300/0 conn(0x7fadc40c5330 0x7fadc40c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758673214 0 0) 0x7fadbc001490 con 0x7fadc4158db0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.052+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb0003520 con 0x7fadc4158db0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006141209 0 0) 0x7fadc40c8f40 con 0x7fadc40c88b0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadbc001490 con 0x7fadc40c88b0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512703667 0 0) 0x7fadb0003520 con 0x7fadc4158db0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 >> v2:172.21.15.163:3300/0 conn(0x7fadc40c5330 msgr2=0x7fadc40c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 --2- 172.21.15.17:0/217582601 >> v2:172.21.15.163:3300/0 conn(0x7fadc40c5330 0x7fadc40c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 >> v1:172.21.15.163:6789/0 conn(0x7fadc40c88b0 legacy=0x7fadc41586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4155390 con 0x7fadc4158db0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fadbc001bc0 con 0x7fadc4158db0 2024-03-06T07:29:34.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc3fff640 1 --2- 172.21.15.17:0/217582601 >> v2:172.21.15.163:3300/0 conn(0x7fadc40c5330 0x7fadc40c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 >> v1:172.21.15.17:6789/0 conn(0x7fadc4158db0 legacy=0x7fadc415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb0003b40 0x7fadb0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadb0004800 0x7fadb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 0x7fadb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41559e0 con 0x7fadb00051c0 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb000c0f0 con 0x7fadb0003b40 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb000c2f0 con 0x7fadb0004800 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 0x7fadb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc3fff640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb0003b40 0x7fadb0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc37fe640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadb0004800 0x7fadb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 -- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadb0004800 msgr2=0x7fadb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadb0004800 0x7fadb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 -- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb0003b40 msgr2=0x7fadb0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb0003b40 0x7fadb0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.053+0000 7fadc8e21640 1 -- 172.21.15.17:0/217582601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0003520 con 0x7fadb00051c0 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.054+0000 7fadc8e21640 1 -- 172.21.15.17:0/217582601 reap_dead start 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.054+0000 7fadc8e21640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 0x7fadb000b9c0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00a9a0 tx=0x7fadb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5216c8286303f002 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.054+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00a5f0 con 0x7fadb00051c0 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.054+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc007290 con 0x7fadb00051c0 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.054+0000 7fadc2ffd640 1 -- 172.21.15.17:0/217582601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc007680 con 0x7fadb00051c0 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 -- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 msgr2=0x7fadb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 0x7fadb000b9c0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00a9a0 tx=0x7fadb0005830 comp rx=0 tx=0).stop 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 -- 172.21.15.17:0/217582601 shutdown_connections 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 --2- 172.21.15.17:0/217582601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadb00051c0 0x7fadb000b9c0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 -- 172.21.15.17:0/217582601 >> 172.21.15.17:0/217582601 conn(0x7fadc40ac560 msgr2=0x7fadc40ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 -- 172.21.15.17:0/217582601 shutdown_connections 2024-03-06T07:29:34.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.055+0000 7fadca884640 1 -- 172.21.15.17:0/217582601 wait complete. 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.056+0000 7fadca884640 1 Processor -- start 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.056+0000 7fadca884640 1 -- start start 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb00051c0 0x7fadc41ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 0x7fadc414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41529d0 con 0x7fadc40c0720 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc406d090 con 0x7fadb00051c0 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadca884640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc4152b50 con 0x7fadc40be2c0 2024-03-06T07:29:34.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59362/0 (socket says 172.21.15.17:59362) 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 -- 172.21.15.17:0/1239169019 learned_addr learned my addr 172.21.15.17:0/1239169019 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc3fff640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb00051c0 0x7fadc41ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc8e21640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 0x7fadc414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 -- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 msgr2=0x7fadc414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 0x7fadc414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 -- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb00051c0 msgr2=0x7fadc41ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb00051c0 0x7fadc41ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc37fe640 1 -- 172.21.15.17:0/1239169019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc4158750 con 0x7fadc40c0720 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.057+0000 7fadc8e21640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 0x7fadc414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadc37fe640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fadb400f7d0 tx=0x7fadb400fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb401a070 con 0x7fadc40c0720 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4005110 con 0x7fadc40c0720 2024-03-06T07:29:34.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4007050 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc4158a30 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.058+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc41ee180 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.059+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc404f330 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.060+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadb401d070 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.060+0000 7fadc0ff9640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 0x7fada406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.060+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadb4089a90 con 0x7fadc40c0720 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.060+0000 7fadc3fff640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 0x7fada406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:34.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.061+0000 7fadc3fff640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 0x7fada406d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fadac0093d0 tx=0x7fadac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:34.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.068+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fadb4058130 con 0x7fadc40c0720 2024-03-06T07:29:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.285+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadc40cb5c0 con 0x7fadc40c0720 2024-03-06T07:29:34.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.286+0000 7fadc0ff9640 1 -- 172.21.15.17:0/1239169019 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fadb405c1d0 con 0x7fadc40c0720 2024-03-06T07:29:34.287 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:34.287 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:34.288 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.287+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 msgr2=0x7fada406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.287+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 0x7fada406d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fadac0093d0 tx=0x7fadac009190 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 msgr2=0x7fadc414d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fadb400f7d0 tx=0x7fadb400fc90 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 shutdown_connections 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fada406b4d0 0x7fada406d990 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadc40be2c0 0x7fadc414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadc40c0720 0x7fadc414d670 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 --2- 172.21.15.17:0/1239169019 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadb00051c0 0x7fadc41ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 >> 172.21.15.17:0/1239169019 conn(0x7fadc40ac560 msgr2=0x7fadc40c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 shutdown_connections 2024-03-06T07:29:34.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:34.288+0000 7fadca884640 1 -- 172.21.15.17:0/1239169019 wait complete. 2024-03-06T07:29:34.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:34.703 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:35.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:35.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.812+0000 7f72bff65640 1 Processor -- start 2024-03-06T07:29:35.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.812+0000 7f72bff65640 1 -- start start 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bff65640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72b80c88c0 0x7f72b8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bff65640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80c73d0 con 0x7f72b814f360 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bff65640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80be2e0 con 0x7f72b8158b60 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bff65640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b8159700 con 0x7f72b80c88c0 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bdd01640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f72b814f360 0x7f72b80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35796/0 (socket says 172.21.15.17:35796) 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.813+0000 7f72bdd01640 1 -- 172.21.15.17:0/2258717923 learned_addr learned my addr 172.21.15.17:0/2258717923 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bd500640 1 --2- 172.21.15.17:0/2258717923 >> v2:172.21.15.163:3300/0 conn(0x7f72b80c88c0 0x7f72b8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12378299 0 0) 0x7f72b80c73d0 con 0x7f72b814f360 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a0003520 con 0x7f72b814f360 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149181298 0 0) 0x7f72a0003520 con 0x7f72b814f360 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 >> v2:172.21.15.163:3300/0 conn(0x7f72b80c88c0 msgr2=0x7f72b8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 --2- 172.21.15.17:0/2258717923 >> v2:172.21.15.163:3300/0 conn(0x7f72b80c88c0 0x7f72b8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 >> v1:172.21.15.163:6789/0 conn(0x7f72b8158b60 legacy=0x7f72b8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80c7670 con 0x7f72b814f360 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f72b0001670 con 0x7f72b814f360 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 >> v1:172.21.15.17:6789/0 conn(0x7f72b814f360 legacy=0x7f72b80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a00043b0 0x7f72a0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a0005120 0x7f72a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8159700 con 0x7f72a00039b0 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c060 con 0x7f72a00043b0 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a000c260 con 0x7f72a0005120 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bd500640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a00043b0 0x7f72a0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72be502640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a0005120 0x7f72a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 -- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a0005120 msgr2=0x7f72a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72a0005120 0x7f72a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 -- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a00043b0 msgr2=0x7f72a0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72a00043b0 0x7f72a0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72bdd01640 1 -- 172.21.15.17:0/2258717923 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a0003520 con 0x7f72a00039b0 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.814+0000 7f72be502640 1 -- 172.21.15.17:0/2258717923 reap_dead start 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.815+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72a0003d90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c290 tx=0x7f72a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55af89b9986ca60 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.815+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007d50 con 0x7f72a00039b0 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.815+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0030070 con 0x7f72a00039b0 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.815+0000 7f72bccff640 1 -- 172.21.15.17:0/2258717923 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b002cdc0 con 0x7f72a00039b0 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 -- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 msgr2=0x7f72a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72a0003d90 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c290 tx=0x7f72a0005830 comp rx=0 tx=0).stop 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 -- 172.21.15.17:0/2258717923 shutdown_connections 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 --2- 172.21.15.17:0/2258717923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72a0003d90 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 -- 172.21.15.17:0/2258717923 >> 172.21.15.17:0/2258717923 conn(0x7f72b80ac570 msgr2=0x7f72b80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 -- 172.21.15.17:0/2258717923 shutdown_connections 2024-03-06T07:29:35.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.816+0000 7f72bff65640 1 -- 172.21.15.17:0/2258717923 wait complete. 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.817+0000 7f72bff65640 1 Processor -- start 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.817+0000 7f72bff65640 1 -- start start 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 0x7f72b80af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 0x7f72b80af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80c7c30 con 0x7f72a00039b0 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8155c10 con 0x7f72b80c0ff0 2024-03-06T07:29:35.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b8155d90 con 0x7f72b814bab0 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59380/0 (socket says 172.21.15.17:59380) 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 -- 172.21.15.17:0/2958452380 learned_addr learned my addr 172.21.15.17:0/2958452380 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bd500640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 0x7f72b80af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72be502640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 0x7f72b80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 -- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 msgr2=0x7f72b80af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 0x7f72b80af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 -- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 msgr2=0x7f72b80af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 0x7f72b80af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 -- 172.21.15.17:0/2958452380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b81eb8b0 con 0x7f72a00039b0 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72be502640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 0x7f72b80af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:35.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bd500640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 0x7f72b80af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bdd01640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c3b0 tx=0x7f72b002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007d50 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0004bf0 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.818+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b80be530 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.819+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0007070 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.819+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81eb540 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.820+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b80c64c0 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.820+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72b0007210 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.820+0000 7f72b67fc640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 0x7f72a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.820+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72b00ab7c0 con 0x7f72a00039b0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.821+0000 7f72bd500640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 0x7f72a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:35.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.821+0000 7f72bd500640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 0x7f72a406d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f72b80c9f10 tx=0x7f72ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:35.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:35.825+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72b80c64c0 con 0x7f72a00039b0 2024-03-06T07:29:36.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.042+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72b804f330 con 0x7f72a00039b0 2024-03-06T07:29:36.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.042+0000 7f72b67fc640 1 -- 172.21.15.17:0/2958452380 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f72b007a620 con 0x7f72a00039b0 2024-03-06T07:29:36.044 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:36.044 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:36.044 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:36.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 msgr2=0x7f72a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:36.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 0x7f72a406d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f72b80c9f10 tx=0x7f72ac008040 comp rx=0 tx=0).stop 2024-03-06T07:29:36.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 msgr2=0x7f72b80c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:36.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c3b0 tx=0x7f72b002e070 comp rx=0 tx=0).stop 2024-03-06T07:29:36.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 shutdown_connections 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f72a406b500 0x7f72a406d9c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72b814bab0 0x7f72b80af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72b80c0ff0 0x7f72b80af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 --2- 172.21.15.17:0/2958452380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72a00039b0 0x7f72b80c0ab0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 >> 172.21.15.17:0/2958452380 conn(0x7f72b80ac570 msgr2=0x7f72b814e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 shutdown_connections 2024-03-06T07:29:36.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:36.044+0000 7f72bff65640 1 -- 172.21.15.17:0/2958452380 wait complete. 2024-03-06T07:29:36.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:36.462 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:37.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:37.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.602+0000 7fc0a06b8640 1 Processor -- start 2024-03-06T07:29:37.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.602+0000 7fc0a06b8640 1 -- start start 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc0a06b8640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0980c88c0 0x7fc098155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc0a06b8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980c73d0 con 0x7fc09814f360 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc0a06b8640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980be2e0 con 0x7fc098158b60 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc0a06b8640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc098159700 con 0x7fc0980c88c0 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc09e454640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc09814f360 0x7fc0980c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:35812/0 (socket says 172.21.15.17:35812) 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.603+0000 7fc09e454640 1 -- 172.21.15.17:0/1489178845 learned_addr learned my addr 172.21.15.17:0/1489178845 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564074424 0 0) 0x7fc0980c73d0 con 0x7fc09814f360 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 --2- 172.21.15.17:0/1489178845 >> v2:172.21.15.163:3300/0 conn(0x7fc0980c88c0 0x7fc098155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc084003520 con 0x7fc09814f360 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1456221101 0 0) 0x7fc084003520 con 0x7fc09814f360 2024-03-06T07:29:37.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 >> v2:172.21.15.163:3300/0 conn(0x7fc0980c88c0 msgr2=0x7fc098155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 --2- 172.21.15.17:0/1489178845 >> v2:172.21.15.163:3300/0 conn(0x7fc0980c88c0 0x7fc098155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 >> v1:172.21.15.163:6789/0 conn(0x7fc098158b60 legacy=0x7fc098158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980c7670 con 0x7fc09814f360 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc090001670 con 0x7fc09814f360 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 >> v1:172.21.15.17:6789/0 conn(0x7fc09814f360 legacy=0x7fc0980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0840039b0 0x7fc084003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840051c0 0x7fc08400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098159700 con 0x7fc084004800 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08400c160 con 0x7fc0840039b0 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08400c360 con 0x7fc0840051c0 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09e454640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0840039b0 0x7fc084003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09ec55640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840051c0 0x7fc08400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 -- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840051c0 msgr2=0x7fc08400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840051c0 0x7fc08400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 -- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0840039b0 msgr2=0x7fc084003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0840039b0 0x7fc084003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09dc53640 1 -- 172.21.15.17:0/1489178845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084003520 con 0x7fc084004800 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.604+0000 7fc09ec55640 1 -- 172.21.15.17:0/1489178845 reap_dead start 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.605+0000 7fc09dc53640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc084004c80 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fc088009a90 tx=0x7fc084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c13801db833d821 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.605+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088039070 con 0x7fc084004800 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.605+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08802c680 con 0x7fc084004800 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.605+0000 7fc09d452640 1 -- 172.21.15.17:0/1489178845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088009690 con 0x7fc084004800 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 -- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 msgr2=0x7fc084004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc084004c80 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fc088009a90 tx=0x7fc084005830 comp rx=0 tx=0).stop 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 -- 172.21.15.17:0/1489178845 shutdown_connections 2024-03-06T07:29:37.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/1489178845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc084004c80 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 -- 172.21.15.17:0/1489178845 >> 172.21.15.17:0/1489178845 conn(0x7fc0980ac570 msgr2=0x7fc0980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:37.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 -- 172.21.15.17:0/1489178845 shutdown_connections 2024-03-06T07:29:37.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.606+0000 7fc0a06b8640 1 -- 172.21.15.17:0/1489178845 wait complete. 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.607+0000 7fc0a06b8640 1 Processor -- start 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.607+0000 7fc0a06b8640 1 -- start start 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980abb70 0x7fc098151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0981ee770 0x7fc0981e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980c7c30 con 0x7fc084004800 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098155c10 con 0x7fc0980abb70 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc0a06b8640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc098155d90 con 0x7fc0981ee770 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59404/0 (socket says 172.21.15.17:59404) 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 -- 172.21.15.17:0/3062739514 learned_addr learned my addr 172.21.15.17:0/3062739514 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09dc53640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980abb70 0x7fc098151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09ec55640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0981ee770 0x7fc0981e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 -- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0981ee770 msgr2=0x7fc0981e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0981ee770 0x7fc0981e7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 -- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980abb70 msgr2=0x7fc098151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:37.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980abb70 0x7fc098151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 -- 172.21.15.17:0/3062739514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980ae010 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc09e454640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fc090007290 tx=0x7fc090007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.608+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000f960 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.609+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090018070 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.609+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000fd20 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.609+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980ae2f0 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.609+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980b0600 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.610+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09804f330 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.610+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc09001d460 con 0x7fc084004800 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.610+0000 7fc08effd640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:37.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.610+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc09008a380 con 0x7fc084004800 2024-03-06T07:29:37.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.610+0000 7fc09dc53640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:37.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.611+0000 7fc09dc53640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 0x7fc07c06d9c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc088036000 tx=0x7fc088030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:37.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:37.615+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc090058ab0 con 0x7fc084004800 2024-03-06T07:29:38.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.036+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0980c8d50 con 0x7fc084004800 2024-03-06T07:29:38.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.036+0000 7fc08effd640 1 -- 172.21.15.17:0/3062739514 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fc09005cb50 con 0x7fc084004800 2024-03-06T07:29:38.038 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:38.038 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:38.039 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:38.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 msgr2=0x7fc07c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 0x7fc07c06d9c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc088036000 tx=0x7fc088030000 comp rx=0 tx=0).stop 2024-03-06T07:29:38.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 msgr2=0x7fc0981ebdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:38.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fc090007290 tx=0x7fc090007750 comp rx=0 tx=0).stop 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 shutdown_connections 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0981ee770 0x7fc0981e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0980abb70 0x7fc098151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 --2- 172.21.15.17:0/3062739514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc084004800 0x7fc0981ebdd0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 >> 172.21.15.17:0/3062739514 conn(0x7fc0980ac570 msgr2=0x7fc09814cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 shutdown_connections 2024-03-06T07:29:38.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:38.038+0000 7fc0a06b8640 1 -- 172.21.15.17:0/3062739514 wait complete. 2024-03-06T07:29:38.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:38.453 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:39.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:39.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.554+0000 7fe1eec4e640 1 Processor -- start 2024-03-06T07:29:39.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 -- start start 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1e80c88c0 0x7fe1e8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80c73d0 con 0x7fe1e814f360 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e80c8dd0 con 0x7fe1e80c88c0 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1eec4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80be2e0 con 0x7fe1e8158b60 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1e80c88c0 0x7fe1e8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1e80c88c0 0x7fe1e8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59420/0 (socket says 172.21.15.17:59420) 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.555+0000 7fe1e7fff640 1 -- 172.21.15.17:0/3206957561 learned_addr learned my addr 172.21.15.17:0/3206957561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.556+0000 7fe1ed1eb640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.556+0000 7fe1e7fff640 1 -- 172.21.15.17:0/3206957561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 msgr2=0x7fe1e8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:39.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.556+0000 7fe1e7fff640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.556+0000 7fe1e7fff640 1 -- 172.21.15.17:0/3206957561 >> v1:172.21.15.163:6789/0 conn(0x7fe1e814f360 legacy=0x7fe1e80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:39.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.556+0000 7fe1e7fff640 1 -- 172.21.15.17:0/3206957561 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80c7670 con 0x7fe1e80c88c0 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.557+0000 7fe1e7fff640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8155c10 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009870 tx=0x7fe1d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda83f606b1489b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.557+0000 7fe1e77fe640 1 -- 172.21.15.17:0/3206957561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0035020 con 0x7fe1e80c88c0 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.557+0000 7fe1e77fe640 1 -- 172.21.15.17:0/3206957561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0031640 con 0x7fe1e80c88c0 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.557+0000 7fe1e77fe640 1 -- 172.21.15.17:0/3206957561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d002d040 con 0x7fe1e80c88c0 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 -- 172.21.15.17:0/3206957561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e80c88c0 msgr2=0x7fe1e8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8155c10 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009870 tx=0x7fe1d002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 -- 172.21.15.17:0/3206957561 shutdown_connections 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/3206957561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8155c10 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 -- 172.21.15.17:0/3206957561 >> 172.21.15.17:0/3206957561 conn(0x7fe1e80ac570 msgr2=0x7fe1e80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 -- 172.21.15.17:0/3206957561 shutdown_connections 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.558+0000 7fe1eec4e640 1 -- 172.21.15.17:0/3206957561 wait complete. 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.559+0000 7fe1eec4e640 1 Processor -- start 2024-03-06T07:29:39.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.559+0000 7fe1eec4e640 1 -- start start 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.559+0000 7fe1eec4e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.559+0000 7fe1eec4e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e81ead10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c5a50 con 0x7fe1e814f360 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c7c30 con 0x7fe1e80c88c0 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80c5bd0 con 0x7fe1e8158b60 2024-03-06T07:29:39.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59432/0 (socket says 172.21.15.17:59432) 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 -- 172.21.15.17:0/334789924 learned_addr learned my addr 172.21.15.17:0/334789924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1ed1eb640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e81ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1ec9ea640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 -- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 msgr2=0x7fe1e81ead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e81ead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 -- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80c88c0 msgr2=0x7fe1e8158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 -- 172.21.15.17:0/334789924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8152590 con 0x7fe1e814f360 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e7fff640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe1d002b930 tx=0x7fe1d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:39.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0040070 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d003e070 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e80c1b30 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0035040 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.560+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e814e380 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.562+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e804f330 con 0x7fe1e814f360 2024-03-06T07:29:39.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.562+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1d0038020 con 0x7fe1e814f360 2024-03-06T07:29:39.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.562+0000 7fe1e57fa640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 0x7fe1c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:39.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.562+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1d00abd60 con 0x7fe1e814f360 2024-03-06T07:29:39.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.562+0000 7fe1ec9ea640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 0x7fe1c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:39.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.563+0000 7fe1ec9ea640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 0x7fe1c806d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:39.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.566+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1d007aa60 con 0x7fe1e814f360 2024-03-06T07:29:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.785+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1e8155ed0 con 0x7fe1e814f360 2024-03-06T07:29:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.786+0000 7fe1e57fa640 1 -- 172.21.15.17:0/334789924 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fe1d007a360 con 0x7fe1e814f360 2024-03-06T07:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 msgr2=0x7fe1c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 0x7fe1c806d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8009040 comp rx=0 tx=0).stop 2024-03-06T07:29:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 msgr2=0x7fe1e814de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7fe1d002b930 tx=0x7fe1d00046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 shutdown_connections 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1c806b4d0 0x7fe1c806d990 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1e8158b60 0x7fe1e81ead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1e814f360 0x7fe1e814de40 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 --2- 172.21.15.17:0/334789924 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1e80c88c0 0x7fe1e8158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 >> 172.21.15.17:0/334789924 conn(0x7fe1e80ac570 msgr2=0x7fe1e80c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 shutdown_connections 2024-03-06T07:29:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:39.788+0000 7fe1eec4e640 1 -- 172.21.15.17:0/334789924 wait complete. 2024-03-06T07:29:40.204 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:40.205 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:40.353 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:40.356 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:29:41.205 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:41.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.306+0000 7f5b6e3d7640 1 Processor -- start 2024-03-06T07:29:41.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.306+0000 7f5b6e3d7640 1 -- start start 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6e3d7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b68159100 0x7f5b680c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6e3d7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680c7c60 con 0x7f5b680c7180 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6e3d7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680c8060 con 0x7f5b681527b0 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6e3d7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680c8df0 con 0x7f5b68159100 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6c974640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5b68159100 0x7f5b680c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b677fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5b680c7180 0x7f5b680c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59634/0 (socket says 172.21.15.17:59634) 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b677fe640 1 -- 172.21.15.17:0/3171624927 learned_addr learned my addr 172.21.15.17:0/3171624927 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:41.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6c974640 1 -- 172.21.15.17:0/3171624927 >> v1:172.21.15.163:6789/0 conn(0x7f5b681527b0 legacy=0x7f5b68155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:41.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6c974640 1 -- 172.21.15.17:0/3171624927 >> v1:172.21.15.17:6789/0 conn(0x7f5b680c7180 legacy=0x7f5b680c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.307+0000 7f5b6c974640 1 -- 172.21.15.17:0/3171624927 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680c76a0 con 0x7f5b68159100 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.308+0000 7f5b6c974640 1 --2- 172.21.15.17:0/3171624927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b680c88b0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009d90 tx=0x7f5b6002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870c7593e9c70eda server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.308+0000 7f5b66ffd640 1 -- 172.21.15.17:0/3171624927 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60039070 con 0x7f5b68159100 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.308+0000 7f5b66ffd640 1 -- 172.21.15.17:0/3171624927 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6002cbd0 con 0x7f5b68159100 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.308+0000 7f5b66ffd640 1 -- 172.21.15.17:0/3171624927 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60034440 con 0x7f5b68159100 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/3171624927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 msgr2=0x7f5b680c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/3171624927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b680c88b0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009d90 tx=0x7f5b6002be20 comp rx=0 tx=0).stop 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/3171624927 shutdown_connections 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/3171624927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b680c88b0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/3171624927 >> 172.21.15.17:0/3171624927 conn(0x7f5b680ac560 msgr2=0x7f5b680ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/3171624927 shutdown_connections 2024-03-06T07:29:41.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.309+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/3171624927 wait complete. 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.310+0000 7f5b6e3d7640 1 Processor -- start 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.310+0000 7f5b6e3d7640 1 -- start start 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b680c7180 0x7f5b681eb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 0x7f5b6806be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:41.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68155600 con 0x7f5b68159100 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68155320 con 0x7f5b680c7180 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6e3d7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68152b90 con 0x7f5b681527b0 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46288/0 (socket says 172.21.15.17:46288) 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 -- 172.21.15.17:0/1102848628 learned_addr learned my addr 172.21.15.17:0/1102848628 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b67fff640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b680c7180 0x7f5b681eb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b677fe640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 0x7f5b6806be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 -- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 msgr2=0x7f5b6806be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 0x7f5b6806be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 -- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b680c7180 msgr2=0x7f5b681eb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b680c7180 0x7f5b681eb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 -- 172.21.15.17:0/1102848628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680c43e0 con 0x7f5b68159100 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b677fe640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 0x7f5b6806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b6c974640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009eb0 tx=0x7f5b60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60044020 con 0x7f5b68159100 2024-03-06T07:29:41.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.311+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6003f470 con 0x7f5b68159100 2024-03-06T07:29:41.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.312+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b680c3e20 con 0x7f5b68159100 2024-03-06T07:29:41.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.312+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60039040 con 0x7f5b68159100 2024-03-06T07:29:41.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.312+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6806c360 con 0x7f5b68159100 2024-03-06T07:29:41.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.312+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6804f330 con 0x7f5b68159100 2024-03-06T07:29:41.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.313+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b60047020 con 0x7f5b68159100 2024-03-06T07:29:41.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.314+0000 7f5b64ff9640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 0x7f5b5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:41.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.314+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b600acc10 con 0x7f5b68159100 2024-03-06T07:29:41.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.314+0000 7f5b67fff640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 0x7f5b5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:41.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.315+0000 7f5b67fff640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 0x7f5b5406d9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5b580017f0 tx=0x7f5b58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:41.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.318+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b6004e070 con 0x7f5b68159100 2024-03-06T07:29:41.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.536+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b6806ce00 con 0x7f5b68159100 2024-03-06T07:29:41.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.537+0000 7f5b64ff9640 1 -- 172.21.15.17:0/1102848628 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f5b6007b990 con 0x7f5b68159100 2024-03-06T07:29:41.538 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:41.539 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:41.539 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.538+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 msgr2=0x7f5b5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.538+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 0x7f5b5406d9c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f5b580017f0 tx=0x7f5b58009040 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.538+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 msgr2=0x7f5b681ee4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.538+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009eb0 tx=0x7f5b60004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 shutdown_connections 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b5406b500 0x7f5b5406d9c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b68159100 0x7f5b681ee4e0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b681527b0 0x7f5b6806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 --2- 172.21.15.17:0/1102848628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b680c7180 0x7f5b681eb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 >> 172.21.15.17:0/1102848628 conn(0x7f5b680ac560 msgr2=0x7f5b6814fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 shutdown_connections 2024-03-06T07:29:41.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:41.539+0000 7f5b6e3d7640 1 -- 172.21.15.17:0/1102848628 wait complete. 2024-03-06T07:29:41.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:41.954 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:42.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.081+0000 7fa538adf640 1 Processor -- start 2024-03-06T07:29:43.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.081+0000 7fa538adf640 1 -- start start 2024-03-06T07:29:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa538adf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa534155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa538adf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340c73d0 con 0x7fa5340c88c0 2024-03-06T07:29:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa538adf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340be2e0 con 0x7fa53414f360 2024-03-06T07:29:43.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa538adf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa534159700 con 0x7fa534159270 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa52bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa5340c88c0 0x7fa534158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59640/0 (socket says 172.21.15.17:59640) 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa52bfff640 1 -- 172.21.15.17:0/2070273344 learned_addr learned my addr 172.21.15.17:0/2070273344 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa532d9d640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa534155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290817951 0 0) 0x7fa5340c73d0 con 0x7fa5340c88c0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa520003520 con 0x7fa5340c88c0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2618321409 0 0) 0x7fa520003520 con 0x7fa5340c88c0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 msgr2=0x7fa534155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa534155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 >> v1:172.21.15.163:6789/0 conn(0x7fa53414f360 legacy=0x7fa5340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5340c7670 con 0x7fa5340c88c0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fa51c001670 con 0x7fa5340c88c0 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 >> v1:172.21.15.17:6789/0 conn(0x7fa5340c88c0 legacy=0x7fa534158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5200039e0 0x7fa520003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa520004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa520004d40 0x7fa5200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534159700 con 0x7fa5200039e0 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.082+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000bb20 con 0x7fa520004d40 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000bd20 con 0x7fa534159270 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5200039e0 0x7fa520003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa532d9d640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa520004d40 0x7fa5200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa52bfff640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa520004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 -- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 msgr2=0x7fa520004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa520004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 -- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa520004d40 msgr2=0x7fa5200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa520004d40 0x7fa5200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa53259c640 1 -- 172.21.15.17:0/2070273344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520003520 con 0x7fa5200039e0 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa532d9d640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa520004d40 0x7fa5200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.083+0000 7fa52bfff640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa534159270 0x7fa520004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa53259c640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5200039e0 0x7fa520003e00 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fa524001750 tx=0x7fa520005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac56250d65df5c5e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524001370 con 0x7fa5200039e0 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5240052d0 con 0x7fa5200039e0 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa531d9b640 1 -- 172.21.15.17:0/2070273344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524003830 con 0x7fa5200039e0 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 -- 172.21.15.17:0/2070273344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5200039e0 msgr2=0x7fa520003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 --2- 172.21.15.17:0/2070273344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5200039e0 0x7fa520003e00 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fa524001750 tx=0x7fa520005830 comp rx=0 tx=0).stop 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa532d9d640 1 -- 172.21.15.17:0/2070273344 reap_dead start 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 -- 172.21.15.17:0/2070273344 shutdown_connections 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 -- 172.21.15.17:0/2070273344 >> 172.21.15.17:0/2070273344 conn(0x7fa5340ac570 msgr2=0x7fa5340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 -- 172.21.15.17:0/2070273344 shutdown_connections 2024-03-06T07:29:43.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.084+0000 7fa538adf640 1 -- 172.21.15.17:0/2070273344 wait complete. 2024-03-06T07:29:43.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.085+0000 7fa538adf640 1 Processor -- start 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 -- start start 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5200039e0 0x7fa534150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa534150da0 0x7fa53414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c7c30 con 0x7fa5340c9aa0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c5a50 con 0x7fa534150da0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa538adf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c5bd0 con 0x7fa5200039e0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46312/0 (socket says 172.21.15.17:46312) 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.086+0000 7fa52bfff640 1 -- 172.21.15.17:0/1794591119 learned_addr learned my addr 172.21.15.17:0/1794591119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa532d9d640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa534150da0 0x7fa53414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa53259c640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5200039e0 0x7fa534150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 -- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5200039e0 msgr2=0x7fa534150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5200039e0 0x7fa534150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 -- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa534150da0 msgr2=0x7fa53414fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa534150da0 0x7fa53414fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 -- 172.21.15.17:0/1794591119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5341ec580 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52bfff640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e730 tx=0x7fa51c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00ee40 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c018070 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5340c4780 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00ca70 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.087+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5341eb7c0 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.088+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53404f330 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.089+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa51c00cc10 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.089+0000 7fa52affd640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 0x7fa51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.089+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa51c08a1d0 con 0x7fa5340c9aa0 2024-03-06T07:29:43.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.089+0000 7fa53259c640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 0x7fa51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.090+0000 7fa53259c640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 0x7fa51406d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa524033000 tx=0x7fa524009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:43.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.094+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa51c058f50 con 0x7fa5340c9aa0 2024-03-06T07:29:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.310+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5340c8d50 con 0x7fa5340c9aa0 2024-03-06T07:29:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.311+0000 7fa52affd640 1 -- 172.21.15.17:0/1794591119 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fa51c058850 con 0x7fa5340c9aa0 2024-03-06T07:29:43.313 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:43.313 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:43.313 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:43.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.312+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 msgr2=0x7fa51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.312+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 0x7fa51406d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fa524033000 tx=0x7fa524009030 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.312+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 msgr2=0x7fa53414f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.312+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00e730 tx=0x7fa51c00ebf0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 shutdown_connections 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa51406b4d0 0x7fa51406d990 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa534150da0 0x7fa53414fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5340c9aa0 0x7fa53414f740 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 --2- 172.21.15.17:0/1794591119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5200039e0 0x7fa534150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 >> 172.21.15.17:0/1794591119 conn(0x7fa5340ac570 msgr2=0x7fa53406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 shutdown_connections 2024-03-06T07:29:43.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:43.313+0000 7fa538adf640 1 -- 172.21.15.17:0/1794591119 wait complete. 2024-03-06T07:29:43.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:43.728 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:44.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:44.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.828+0000 7f7fe215c640 1 Processor -- start 2024-03-06T07:29:44.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.828+0000 7f7fe215c640 1 -- start start 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fe215c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fdc06e490 0x7f7fdc160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fe215c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0bdc20 con 0x7f7fdc073250 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fe215c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc06c8e0 con 0x7f7fdc072e70 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fe215c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc06ce80 con 0x7f7fdc06e490 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fdaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7fdc073250 0x7f7fdc0c3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59654/0 (socket says 172.21.15.17:59654) 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fdbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7fdc06e490 0x7f7fdc160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fdaffd640 1 -- 172.21.15.17:0/3134120652 learned_addr learned my addr 172.21.15.17:0/3134120652 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fdbfff640 1 -- 172.21.15.17:0/3134120652 >> v1:172.21.15.163:6789/0 conn(0x7f7fdc072e70 legacy=0x7f7fdc0bd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.829+0000 7f7fdbfff640 1 -- 172.21.15.17:0/3134120652 >> v1:172.21.15.17:6789/0 conn(0x7f7fdc073250 legacy=0x7f7fdc0c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fda7fc640 1 -- 172.21.15.17:0/3134120652 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852043228 0 0) 0x7f7fdc0bdc20 con 0x7f7fdc073250 2024-03-06T07:29:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fdbfff640 1 -- 172.21.15.17:0/3134120652 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0c1b00 con 0x7f7fdc06e490 2024-03-06T07:29:44.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fdbfff640 1 --2- 172.21.15.17:0/3134120652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc160cf0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009d90 tx=0x7f7fd402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44577cf88678cc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fda7fc640 1 -- 172.21.15.17:0/3134120652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4039070 con 0x7f7fdc06e490 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fda7fc640 1 -- 172.21.15.17:0/3134120652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd402cc60 con 0x7f7fdc06e490 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.830+0000 7f7fda7fc640 1 -- 172.21.15.17:0/3134120652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4034660 con 0x7f7fdc06e490 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 -- 172.21.15.17:0/3134120652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 msgr2=0x7f7fdc160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 --2- 172.21.15.17:0/3134120652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc160cf0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009d90 tx=0x7f7fd402beb0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 -- 172.21.15.17:0/3134120652 shutdown_connections 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 --2- 172.21.15.17:0/3134120652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc160cf0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 -- 172.21.15.17:0/3134120652 >> 172.21.15.17:0/3134120652 conn(0x7f7fdc06b1a0 msgr2=0x7f7fdc06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 -- 172.21.15.17:0/3134120652 shutdown_connections 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.831+0000 7f7fe215c640 1 -- 172.21.15.17:0/3134120652 wait complete. 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.832+0000 7f7fe215c640 1 Processor -- start 2024-03-06T07:29:44.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.832+0000 7f7fe215c640 1 -- start start 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fdc072e70 0x7f7fdc157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fdc073250 0x7f7fdc0c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c3ca0 con 0x7f7fdc06e490 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c3b20 con 0x7f7fdc073250 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fe215c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0c6760 con 0x7f7fdc072e70 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46336/0 (socket says 172.21.15.17:46336) 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 -- 172.21.15.17:0/1335792778 learned_addr learned my addr 172.21.15.17:0/1335792778 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdaffd640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fdc072e70 0x7f7fdc157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdbfff640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fdc073250 0x7f7fdc0c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 -- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fdc072e70 msgr2=0x7f7fdc157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fdc072e70 0x7f7fdc157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 -- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fdc073250 msgr2=0x7f7fdc0c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fdc073250 0x7f7fdc0c03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.833+0000 7f7fdb7fe640 1 -- 172.21.15.17:0/1335792778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc1eb9a0 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fdb7fe640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00c450 tx=0x7f7fcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0052d0 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc005470 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc1eb100 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc012870 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.834+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc1eb6c0 con 0x7f7fdc06e490 2024-03-06T07:29:44.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.835+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc06baa0 con 0x7f7fdc06e490 2024-03-06T07:29:44.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.836+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fcc012a10 con 0x7f7fdc06e490 2024-03-06T07:29:44.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.836+0000 7f7fd3fff640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 0x7f7fbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:44.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.836+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fcc089e10 con 0x7f7fdc06e490 2024-03-06T07:29:44.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.837+0000 7f7fdaffd640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 0x7f7fbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:44.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.837+0000 7f7fdaffd640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 0x7f7fbc06d9c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0acaf0 tx=0x7f7fc4007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:44.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:44.842+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fdc06baa0 con 0x7f7fdc06e490 2024-03-06T07:29:45.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.064+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fdc04f330 con 0x7f7fdc06e490 2024-03-06T07:29:45.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.065+0000 7f7fd3fff640 1 -- 172.21.15.17:0/1335792778 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f7fcc0584c0 con 0x7f7fdc06e490 2024-03-06T07:29:45.066 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:45.066 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:45.067 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 msgr2=0x7f7fbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 0x7f7fbc06d9c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0acaf0 tx=0x7f7fc4007e40 comp rx=0 tx=0).stop 2024-03-06T07:29:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 msgr2=0x7f7fdc1eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00c450 tx=0x7f7fcc00c910 comp rx=0 tx=0).stop 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 shutdown_connections 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7fbc06b500 0x7f7fbc06d9c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7fdc073250 0x7f7fdc0c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7fdc072e70 0x7f7fdc157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.066+0000 7f7fe215c640 1 --2- 172.21.15.17:0/1335792778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7fdc06e490 0x7f7fdc1eaa00 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.067+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 >> 172.21.15.17:0/1335792778 conn(0x7f7fdc06b1a0 msgr2=0x7f7fdc0bc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.067+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 shutdown_connections 2024-03-06T07:29:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:45.067+0000 7f7fe215c640 1 -- 172.21.15.17:0/1335792778 wait complete. 2024-03-06T07:29:45.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:45.482 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:46.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:46.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 Processor -- start 2024-03-06T07:29:46.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 -- start start 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa73414f360 0x7fa734155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa734158b60 0x7fa734158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340c73d0 con 0x7fa7340c88c0 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340c8dd0 con 0x7fa734158b60 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa73b0ec640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340be2e0 con 0x7fa73414f360 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa739689640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa734158b60 0x7fa734158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa739689640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa734158b60 0x7fa734158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46338/0 (socket says 172.21.15.17:46338) 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.588+0000 7fa739689640 1 -- 172.21.15.17:0/1672938854 learned_addr learned my addr 172.21.15.17:0/1672938854 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:46.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa738e88640 1 --2- 172.21.15.17:0/1672938854 >> v2:172.21.15.163:3300/0 conn(0x7fa73414f360 0x7fa734155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa739689640 1 -- 172.21.15.17:0/1672938854 >> v2:172.21.15.163:3300/0 conn(0x7fa73414f360 msgr2=0x7fa734155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa739689640 1 --2- 172.21.15.17:0/1672938854 >> v2:172.21.15.163:3300/0 conn(0x7fa73414f360 0x7fa734155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa739689640 1 -- 172.21.15.17:0/1672938854 >> v1:172.21.15.163:6789/0 conn(0x7fa7340c88c0 legacy=0x7fa7340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa739689640 1 -- 172.21.15.17:0/1672938854 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340c7670 con 0x7fa734158b60 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.589+0000 7fa739689640 1 --2- 172.21.15.17:0/1672938854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa734158b60 0x7fa734158ff0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fa73408c840 tx=0x7fa72c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b225482492b40a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa7337fe640 1 -- 172.21.15.17:0/1672938854 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c035020 con 0x7fa734158b60 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa7337fe640 1 -- 172.21.15.17:0/1672938854 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c031870 con 0x7fa734158b60 2024-03-06T07:29:46.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa7337fe640 1 -- 172.21.15.17:0/1672938854 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c031c40 con 0x7fa734158b60 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 -- 172.21.15.17:0/1672938854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa734158b60 msgr2=0x7fa734158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/1672938854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa734158b60 0x7fa734158ff0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7fa73408c840 tx=0x7fa72c02b910 comp rx=0 tx=0).stop 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 -- 172.21.15.17:0/1672938854 shutdown_connections 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/1672938854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa734158b60 0x7fa734158ff0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/1672938854 >> v2:172.21.15.163:3300/0 conn(0x7fa73414f360 0x7fa734155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 -- 172.21.15.17:0/1672938854 >> 172.21.15.17:0/1672938854 conn(0x7fa7340ac570 msgr2=0x7fa7340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 -- 172.21.15.17:0/1672938854 shutdown_connections 2024-03-06T07:29:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.590+0000 7fa73b0ec640 1 -- 172.21.15.17:0/1672938854 wait complete. 2024-03-06T07:29:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.591+0000 7fa73b0ec640 1 Processor -- start 2024-03-06T07:29:46.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 -- start start 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7340c88c0 0x7fa7340bee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa734158b60 0x7fa7340ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c5a50 con 0x7fa73414f360 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c7c30 con 0x7fa734158b60 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa73b0ec640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c5bd0 con 0x7fa7340c88c0 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.592+0000 7fa733fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46350/0 (socket says 172.21.15.17:46350) 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 -- 172.21.15.17:0/2744589351 learned_addr learned my addr 172.21.15.17:0/2744589351 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa738e88640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7340c88c0 0x7fa7340bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 -- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7340c88c0 msgr2=0x7fa7340bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7340c88c0 0x7fa7340bee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 -- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa734158b60 msgr2=0x7fa7340ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa734158b60 0x7fa7340ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 -- 172.21.15.17:0/2744589351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340ad4c0 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa733fff640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c440 tx=0x7fa72000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200052d0 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720005470 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7340ad7a0 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720007260 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.593+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340add60 con 0x7fa73414f360 2024-03-06T07:29:46.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.594+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73404f330 con 0x7fa73414f360 2024-03-06T07:29:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.595+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7200074c0 con 0x7fa73414f360 2024-03-06T07:29:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.595+0000 7fa7317fa640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 0x7fa71806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.595+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa720089750 con 0x7fa73414f360 2024-03-06T07:29:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.595+0000 7fa738e88640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 0x7fa71806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:46.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.596+0000 7fa738e88640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 0x7fa71806d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7340c4de0 tx=0x7fa72800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:46.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.600+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7200584d0 con 0x7fa73414f360 2024-03-06T07:29:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.818+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73414cd20 con 0x7fa73414f360 2024-03-06T07:29:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.819+0000 7fa7317fa640 1 -- 172.21.15.17:0/2744589351 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fa720057dd0 con 0x7fa73414f360 2024-03-06T07:29:46.820 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:46.820 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:46.821 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.820+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 msgr2=0x7fa71806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.820+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 0x7fa71806d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa7340c4de0 tx=0x7fa72800b040 comp rx=0 tx=0).stop 2024-03-06T07:29:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.820+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 msgr2=0x7fa7340afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.820+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c440 tx=0x7fa72000c900 comp rx=0 tx=0).stop 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 shutdown_connections 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa71806b4d0 0x7fa71806d990 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa734158b60 0x7fa7340ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa73414f360 0x7fa7340afc30 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 --2- 172.21.15.17:0/2744589351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7340c88c0 0x7fa7340bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 >> 172.21.15.17:0/2744589351 conn(0x7fa7340ac570 msgr2=0x7fa7340c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 shutdown_connections 2024-03-06T07:29:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:46.821+0000 7fa73b0ec640 1 -- 172.21.15.17:0/2744589351 wait complete. 2024-03-06T07:29:47.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:47.236 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:48.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.337+0000 7feb3f042640 1 Processor -- start 2024-03-06T07:29:48.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.337+0000 7feb3f042640 1 -- start start 2024-03-06T07:29:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3f042640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb3814f790 0x7feb381529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3f042640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb380c1aa0 0x7feb380c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3f042640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380c28d0 con 0x7feb380c5c60 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3f042640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380c2680 con 0x7feb3814f790 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3f042640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380c2bb0 con 0x7feb380c1aa0 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb3814f790 0x7feb381529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb3814f790 0x7feb381529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46366/0 (socket says 172.21.15.17:46366) 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 -- 172.21.15.17:0/4195136758 learned_addr learned my addr 172.21.15.17:0/4195136758 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3d5df640 1 --1- 172.21.15.17:0/4195136758 >> v1:172.21.15.17:6789/0 conn(0x7feb380c5c60 0x7feb38155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59670/0 (socket says 172.21.15.17:59670) 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 -- 172.21.15.17:0/4195136758 >> v2:172.21.15.163:3300/0 conn(0x7feb380c1aa0 msgr2=0x7feb380c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 --2- 172.21.15.17:0/4195136758 >> v2:172.21.15.163:3300/0 conn(0x7feb380c1aa0 0x7feb380c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 -- 172.21.15.17:0/4195136758 >> v1:172.21.15.17:6789/0 conn(0x7feb380c5c60 legacy=0x7feb38155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb3cdde640 1 -- 172.21.15.17:0/4195136758 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380c2e90 con 0x7feb3814f790 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.338+0000 7feb377fe640 1 -- 172.21.15.17:0/4195136758 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454360085 0 0) 0x7feb380c28d0 con 0x7feb380c5c60 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.339+0000 7feb3cdde640 1 --2- 172.21.15.17:0/4195136758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb3814f790 0x7feb381529d0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7feb38070280 tx=0x7feb2c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd824efc77d1ac server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.339+0000 7feb377fe640 1 -- 172.21.15.17:0/4195136758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c034070 con 0x7feb3814f790 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.339+0000 7feb377fe640 1 -- 172.21.15.17:0/4195136758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c030860 con 0x7feb3814f790 2024-03-06T07:29:48.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.339+0000 7feb377fe640 1 -- 172.21.15.17:0/4195136758 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c030c30 con 0x7feb3814f790 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 -- 172.21.15.17:0/4195136758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb3814f790 msgr2=0x7feb381529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 --2- 172.21.15.17:0/4195136758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb3814f790 0x7feb381529d0 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7feb38070280 tx=0x7feb2c02b9a0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 -- 172.21.15.17:0/4195136758 shutdown_connections 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 --2- 172.21.15.17:0/4195136758 >> v2:172.21.15.163:3300/0 conn(0x7feb380c1aa0 0x7feb380c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 --2- 172.21.15.17:0/4195136758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb3814f790 0x7feb381529d0 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 -- 172.21.15.17:0/4195136758 >> 172.21.15.17:0/4195136758 conn(0x7feb380c6710 msgr2=0x7feb380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 -- 172.21.15.17:0/4195136758 shutdown_connections 2024-03-06T07:29:48.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.340+0000 7feb3f042640 1 -- 172.21.15.17:0/4195136758 wait complete. 2024-03-06T07:29:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.341+0000 7feb3f042640 1 Processor -- start 2024-03-06T07:29:48.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.342+0000 7feb3f042640 1 -- start start 2024-03-06T07:29:48.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 0x7feb3814d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 0x7feb381ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806cc00 con 0x7feb380c1aa0 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c0d60 con 0x7feb3814f790 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3f042640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806cd80 con 0x7feb380c5c60 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46382/0 (socket says 172.21.15.17:46382) 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 -- 172.21.15.17:0/840911393 learned_addr learned my addr 172.21.15.17:0/840911393 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb37fff640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 0x7feb3814d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3d5df640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 0x7feb381ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 -- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 msgr2=0x7feb3814d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 0x7feb3814d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 -- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 msgr2=0x7feb381ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 0x7feb381ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 -- 172.21.15.17:0/840911393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb381dbb30 con 0x7feb380c1aa0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3d5df640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 0x7feb381ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb37fff640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 0x7feb3814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.343+0000 7feb3cdde640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0090b0 tx=0x7feb2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.344+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c034040 con 0x7feb380c1aa0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.344+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c03ab90 con 0x7feb380c1aa0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.344+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb381dbe10 con 0x7feb380c1aa0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.344+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c004030 con 0x7feb380c1aa0 2024-03-06T07:29:48.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.344+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb380729f0 con 0x7feb380c1aa0 2024-03-06T07:29:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.345+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38072500 con 0x7feb380c1aa0 2024-03-06T07:29:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.345+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb2c04b060 con 0x7feb380c1aa0 2024-03-06T07:29:48.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.346+0000 7feb357fa640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 0x7feb1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.346+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feb2c0ac460 con 0x7feb380c1aa0 2024-03-06T07:29:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.346+0000 7feb37fff640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 0x7feb1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:48.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.346+0000 7feb37fff640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 0x7feb1c06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feb3806d750 tx=0x7feb24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:48.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.354+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb2c03f070 con 0x7feb380c1aa0 2024-03-06T07:29:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.573+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb3804f330 con 0x7feb380c1aa0 2024-03-06T07:29:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.574+0000 7feb357fa640 1 -- 172.21.15.17:0/840911393 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7feb2c038e90 con 0x7feb380c1aa0 2024-03-06T07:29:48.575 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:48.576 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 msgr2=0x7feb1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 0x7feb1c06d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7feb3806d750 tx=0x7feb24008040 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 msgr2=0x7feb38072f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0090b0 tx=0x7feb2c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 shutdown_connections 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7feb1c06b4d0 0x7feb1c06d990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb3814f790 0x7feb381ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb380c5c60 0x7feb3814d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 --2- 172.21.15.17:0/840911393 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb380c1aa0 0x7feb38072f10 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 >> 172.21.15.17:0/840911393 conn(0x7feb380c6710 msgr2=0x7feb380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 shutdown_connections 2024-03-06T07:29:48.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:48.576+0000 7feb3f042640 1 -- 172.21.15.17:0/840911393 wait complete. 2024-03-06T07:29:48.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:48.991 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:49.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.092+0000 7f184c242640 1 Processor -- start 2024-03-06T07:29:50.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.092+0000 7f184c242640 1 -- start start 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.092+0000 7f184c242640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1844155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f184c242640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440c73d0 con 0x7f18440c88c0 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f184c242640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440be2e0 con 0x7f1844158b60 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f184c242640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1844159700 con 0x7f184414f360 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f18497dd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f18440c88c0 0x7f18440c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59686/0 (socket says 172.21.15.17:59686) 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f18497dd640 1 -- 172.21.15.17:0/699478596 learned_addr learned my addr 172.21.15.17:0/699478596 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995318452 0 0) 0x7f18440c73d0 con 0x7f18440c88c0 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1830003520 con 0x7f18440c88c0 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1849fde640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1844155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 960239541 0 0) 0x7f1830003520 con 0x7f18440c88c0 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 msgr2=0x7f1844155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:50.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1844155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 >> v1:172.21.15.163:6789/0 conn(0x7f1844158b60 legacy=0x7f1844158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440c7670 con 0x7f18440c88c0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1840001670 con 0x7f18440c88c0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 >> v1:172.21.15.17:6789/0 conn(0x7f18440c88c0 legacy=0x7f18440c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18300039e0 0x7f1830003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830004800 0x7f1830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1830005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844159700 con 0x7f1830004800 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830014300 con 0x7f18300039e0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830014500 con 0x7f184414f360 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.093+0000 7f18497dd640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830004800 0x7f1830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f1849fde640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18300039e0 0x7f1830003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f184a7df640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1830005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 -- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 msgr2=0x7f1830005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:50.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184414f360 0x7f1830005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 -- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18300039e0 msgr2=0x7f1830003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18300039e0 0x7f1830003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 -- 172.21.15.17:0/699478596 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830003520 con 0x7f1830004800 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.094+0000 7f18497dd640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830004800 0x7f1830004c80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f18440c8d20 tx=0x7f1830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b782a67866f691c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840007d30 con 0x7f1830004800 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840030070 con 0x7f1830004800 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f1848fdc640 1 -- 172.21.15.17:0/699478596 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840005d30 con 0x7f1830004800 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 -- 172.21.15.17:0/699478596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830004800 msgr2=0x7f1830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 --2- 172.21.15.17:0/699478596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830004800 0x7f1830004c80 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f18440c8d20 tx=0x7f1830005830 comp rx=0 tx=0).stop 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184a7df640 1 -- 172.21.15.17:0/699478596 reap_dead start 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 -- 172.21.15.17:0/699478596 shutdown_connections 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 -- 172.21.15.17:0/699478596 >> 172.21.15.17:0/699478596 conn(0x7f18440ac570 msgr2=0x7f18440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 -- 172.21.15.17:0/699478596 shutdown_connections 2024-03-06T07:29:50.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.095+0000 7f184c242640 1 -- 172.21.15.17:0/699478596 wait complete. 2024-03-06T07:29:50.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.096+0000 7f184c242640 1 Processor -- start 2024-03-06T07:29:50.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.096+0000 7f184c242640 1 -- start start 2024-03-06T07:29:50.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1830004800 0x7f18440c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1844147b80 0x7f1844143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c7c30 con 0x7f1844158550 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c5a50 con 0x7f1830004800 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184c242640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440c5bd0 con 0x7f1844147b80 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f18497dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f18497dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46414/0 (socket says 172.21.15.17:46414) 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f18497dd640 1 -- 172.21.15.17:0/2884191366 learned_addr learned my addr 172.21.15.17:0/2884191366 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f184a7df640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1844147b80 0x7f1844143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.097+0000 7f1849fde640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1830004800 0x7f18440c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 -- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1844147b80 msgr2=0x7f1844143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1844147b80 0x7f1844143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 -- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1830004800 msgr2=0x7f18440c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1830004800 0x7f18440c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 -- 172.21.15.17:0/2884191366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18441ea6d0 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f18497dd640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f184000c300 tx=0x7f1840001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840045070 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840000ed0 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840004df0 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440ca390 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.098+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1844156900 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.099+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184404f330 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.100+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1840007070 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.100+0000 7f183a7fc640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 0x7f182806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.100+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18400ab2b0 con 0x7f1844158550 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.100+0000 7f1849fde640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 0x7f182806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:50.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.101+0000 7f1849fde640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 0x7f182806d9c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1834009bc0 tx=0x7f1834009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:50.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.108+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1840079af0 con 0x7f1844158550 2024-03-06T07:29:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.433+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18440c8d50 con 0x7f1844158550 2024-03-06T07:29:50.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.433+0000 7f183a7fc640 1 -- 172.21.15.17:0/2884191366 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7f184007db90 con 0x7f1844158550 2024-03-06T07:29:50.435 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:50.435 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:50.436 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.436+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 msgr2=0x7f182806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.436+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 0x7f182806d9c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1834009bc0 tx=0x7f1834009190 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.436+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 msgr2=0x7f1844147640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.436+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f184000c300 tx=0x7f1840001a40 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 shutdown_connections 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f182806b500 0x7f182806d9c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1844147b80 0x7f1844143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1844158550 0x7f1844147640 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 --2- 172.21.15.17:0/2884191366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1830004800 0x7f18440c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 >> 172.21.15.17:0/2884191366 conn(0x7f18440ac570 msgr2=0x7f18440abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 shutdown_connections 2024-03-06T07:29:50.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:50.437+0000 7f184c242640 1 -- 172.21.15.17:0/2884191366 wait complete. 2024-03-06T07:29:50.854 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:50.855 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:51.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:51.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.956+0000 7fe2ef95c640 1 Processor -- start 2024-03-06T07:29:51.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.956+0000 7fe2ef95c640 1 -- start start 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ef95c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe2e8153990 0x7fe2e8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ef95c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e80c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ef95c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e8070aa0 con 0x7fe2e81506b0 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ef95c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e8156450 con 0x7fe2e8153990 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ef95c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8156a10 con 0x7fe2e80acc00 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ed6f8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe2e81506b0 0x7fe2e80702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42668/0 (socket says 172.21.15.17:42668) 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ed6f8640 1 -- 172.21.15.17:0/132206687 learned_addr learned my addr 172.21.15.17:0/132206687 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2ecef7640 1 --2- 172.21.15.17:0/132206687 >> v2:172.21.15.163:3300/0 conn(0x7fe2e8153990 0x7fe2e8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2edef9640 1 --2- 172.21.15.17:0/132206687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e80c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523641713 0 0) 0x7fe2e8070aa0 con 0x7fe2e81506b0 2024-03-06T07:29:51.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d4003520 con 0x7fe2e81506b0 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433069338 0 0) 0x7fe2d4003520 con 0x7fe2e81506b0 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2edef9640 1 -- 172.21.15.17:0/132206687 >> v2:172.21.15.163:3300/0 conn(0x7fe2e8153990 msgr2=0x7fe2e8156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2edef9640 1 --2- 172.21.15.17:0/132206687 >> v2:172.21.15.163:3300/0 conn(0x7fe2e8153990 0x7fe2e8156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2edef9640 1 -- 172.21.15.17:0/132206687 >> v1:172.21.15.17:6789/0 conn(0x7fe2e81506b0 legacy=0x7fe2e80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.957+0000 7fe2edef9640 1 -- 172.21.15.17:0/132206687 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8070d80 con 0x7fe2e80acc00 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.958+0000 7fe2edef9640 1 --2- 172.21.15.17:0/132206687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e80c54a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009d90 tx=0x7fe2e002bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1fe043eaf9c9d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0039070 con 0x7fe2e80acc00 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e002c8a0 con 0x7fe2e80acc00 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2dffff640 1 -- 172.21.15.17:0/132206687 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4001690 con 0x7fe2e80acc00 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 -- 172.21.15.17:0/132206687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 msgr2=0x7fe2e80c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/132206687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e80c54a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009d90 tx=0x7fe2e002bd70 comp rx=0 tx=0).stop 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 -- 172.21.15.17:0/132206687 shutdown_connections 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/132206687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e80c54a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/132206687 >> v2:172.21.15.163:3300/0 conn(0x7fe2e8153990 0x7fe2e8156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 -- 172.21.15.17:0/132206687 >> 172.21.15.17:0/132206687 conn(0x7fe2e806b1a0 msgr2=0x7fe2e806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:51.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 -- 172.21.15.17:0/132206687 shutdown_connections 2024-03-06T07:29:51.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.959+0000 7fe2ef95c640 1 -- 172.21.15.17:0/132206687 wait complete. 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.960+0000 7fe2ef95c640 1 Processor -- start 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.960+0000 7fe2ef95c640 1 -- start start 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e81506b0 0x7fe2e8148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ad1d0 con 0x7fe2e8153990 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ad050 con 0x7fe2e81506b0 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ef95c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ad350 con 0x7fe2e80acc00 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34732/0 (socket says 172.21.15.17:34732) 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 -- 172.21.15.17:0/1954605108 learned_addr learned my addr 172.21.15.17:0/1954605108 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ed6f8640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2ecef7640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e81506b0 0x7fe2e8148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 -- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 msgr2=0x7fe2e81487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e81487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 -- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e81506b0 msgr2=0x7fe2e8148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e81506b0 0x7fe2e8148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.961+0000 7fe2edef9640 1 -- 172.21.15.17:0/1954605108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e81ec680 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2edef9640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009eb0 tx=0x7fe2e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0039050 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0045070 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e81ec920 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009450 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.962+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e8146e20 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.963+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e8146960 con 0x7fe2e8153990 2024-03-06T07:29:51.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.963+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2e0040690 con 0x7fe2e8153990 2024-03-06T07:29:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.964+0000 7fe2ddffb640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.964+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe2e00aca30 con 0x7fe2e8153990 2024-03-06T07:29:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.964+0000 7fe2ed6f8640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:51.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.964+0000 7fe2ed6f8640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001800 tx=0x7fe2e4001250 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:51.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:51.968+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe2e8146960 con 0x7fe2e8153990 2024-03-06T07:29:52.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.188+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e804f330 con 0x7fe2e8153990 2024-03-06T07:29:52.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.189+0000 7fe2ddffb640 1 -- 172.21.15.17:0/1954605108 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fe2e007b160 con 0x7fe2e8153990 2024-03-06T07:29:52.191 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:52.191 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:52.191 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:52.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 msgr2=0x7fe2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001800 tx=0x7fe2e4001250 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 msgr2=0x7fe2e81e5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0009eb0 tx=0x7fe2e0004a40 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 shutdown_connections 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe2cc06b4d0 0x7fe2cc06d990 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2e8153990 0x7fe2e81e5810 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2e81506b0 0x7fe2e8148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 --2- 172.21.15.17:0/1954605108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2e80acc00 0x7fe2e81487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 >> 172.21.15.17:0/1954605108 conn(0x7fe2e806b1a0 msgr2=0x7fe2e80c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 shutdown_connections 2024-03-06T07:29:52.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:52.191+0000 7fe2ef95c640 1 -- 172.21.15.17:0/1954605108 wait complete. 2024-03-06T07:29:52.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:52.606 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:53.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.707+0000 7fdee65a1640 1 Processor -- start 2024-03-06T07:29:53.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.707+0000 7fdee65a1640 1 -- start start 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.708+0000 7fdee65a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.708+0000 7fdee65a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee0158b60 0x7fdee0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee65a1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00c73d0 con 0x7fdee00c88c0 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee65a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00c8dd0 con 0x7fdee0158b60 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee65a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00be2e0 con 0x7fdee014f360 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee0158b60 0x7fdee0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdee0158b60 0x7fdee0158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34740/0 (socket says 172.21.15.17:34740) 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 -- 172.21.15.17:0/1804735499 learned_addr learned my addr 172.21.15.17:0/1804735499 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdedffff640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 -- 172.21.15.17:0/1804735499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 msgr2=0x7fdee0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 -- 172.21.15.17:0/1804735499 >> v1:172.21.15.163:6789/0 conn(0x7fdee00c88c0 legacy=0x7fdee00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:53.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.709+0000 7fdee4b3e640 1 -- 172.21.15.17:0/1804735499 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00c7670 con 0x7fdee0158b60 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee4b3e640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee0158b60 0x7fdee0158ff0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fded8009e50 tx=0x7fded802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebffb76d92cbd25d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdedeffd640 1 -- 172.21.15.17:0/1804735499 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded802c9e0 con 0x7fdee0158b60 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdedeffd640 1 -- 172.21.15.17:0/1804735499 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded802cb80 con 0x7fdee0158b60 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdedeffd640 1 -- 172.21.15.17:0/1804735499 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80343f0 con 0x7fdee0158b60 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 -- 172.21.15.17:0/1804735499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee0158b60 msgr2=0x7fdee0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee0158b60 0x7fdee0158ff0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7fded8009e50 tx=0x7fded802be40 comp rx=0 tx=0).stop 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 -- 172.21.15.17:0/1804735499 shutdown_connections 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee0158b60 0x7fdee0158ff0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 --2- 172.21.15.17:0/1804735499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.710+0000 7fdee65a1640 1 -- 172.21.15.17:0/1804735499 >> 172.21.15.17:0/1804735499 conn(0x7fdee00ac570 msgr2=0x7fdee00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.711+0000 7fdee65a1640 1 -- 172.21.15.17:0/1804735499 shutdown_connections 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.711+0000 7fdee65a1640 1 -- 172.21.15.17:0/1804735499 wait complete. 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 Processor -- start 2024-03-06T07:29:53.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 -- start start 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 0x7fdee01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c5a50 con 0x7fdee00c88c0 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c7c30 con 0x7fdee0158b60 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.712+0000 7fdee65a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c5bd0 con 0x7fdee014f360 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34748/0 (socket says 172.21.15.17:34748) 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 -- 172.21.15.17:0/2298653893 learned_addr learned my addr 172.21.15.17:0/2298653893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedf7fe640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdee4b3e640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 0x7fdee01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 -- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 msgr2=0x7fdee0151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 -- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 msgr2=0x7fdee01506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 0x7fdee01506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 -- 172.21.15.17:0/2298653893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee014f8e0 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdee4b3e640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 0x7fdee01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedffff640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fded000d830 tx=0x7fded000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000b6c0 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0019070 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee014fbc0 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000ba80 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.713+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee0151290 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.714+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00ca040 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.715+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fded001e460 con 0x7fdee00c88c0 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.715+0000 7fdedcff9640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 0x7fdecc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:53.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.715+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fded008aa30 con 0x7fdee00c88c0 2024-03-06T07:29:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.716+0000 7fdedf7fe640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 0x7fdecc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:53.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.716+0000 7fdedf7fe640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 0x7fdecc06d990 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:53.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.720+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdee00ca040 con 0x7fdee00c88c0 2024-03-06T07:29:53.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.937+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee004f330 con 0x7fdee00c88c0 2024-03-06T07:29:53.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.938+0000 7fdedcff9640 1 -- 172.21.15.17:0/2298653893 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fded0059720 con 0x7fdee00c88c0 2024-03-06T07:29:53.940 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:53.940 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:53.940 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 msgr2=0x7fdecc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 0x7fdecc06d990 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4008040 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 msgr2=0x7fdee014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7fded000d830 tx=0x7fded000dcf0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 shutdown_connections 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdecc06b4d0 0x7fdecc06d990 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdee0158b60 0x7fdee01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdee014f360 0x7fdee0151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 --2- 172.21.15.17:0/2298653893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdee00c88c0 0x7fdee014b790 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:53.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 >> 172.21.15.17:0/2298653893 conn(0x7fdee00ac570 msgr2=0x7fdee01550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 shutdown_connections 2024-03-06T07:29:53.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:53.940+0000 7fdee65a1640 1 -- 172.21.15.17:0/2298653893 wait complete. 2024-03-06T07:29:54.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:54.355 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:55.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:55.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 Processor -- start 2024-03-06T07:29:55.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 -- start start 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdeb81506b0 0x7fdeb8153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdeb8156ea0 0x7fdeb80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb8070800 con 0x7fdeb80acc00 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb8156450 con 0x7fdeb8156ea0 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdebe6b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb8156a10 con 0x7fdeb81506b0 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.460+0000 7fdeb77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdeb8156ea0 0x7fdeb80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdeb8156ea0 0x7fdeb80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34754/0 (socket says 172.21.15.17:34754) 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 -- 172.21.15.17:0/2208784018 learned_addr learned my addr 172.21.15.17:0/2208784018 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb7fff640 1 --2- 172.21.15.17:0/2208784018 >> v2:172.21.15.163:3300/0 conn(0x7fdeb81506b0 0x7fdeb8153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 -- 172.21.15.17:0/2208784018 >> v2:172.21.15.163:3300/0 conn(0x7fdeb81506b0 msgr2=0x7fdeb8153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 --2- 172.21.15.17:0/2208784018 >> v2:172.21.15.163:3300/0 conn(0x7fdeb81506b0 0x7fdeb8153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 -- 172.21.15.17:0/2208784018 >> v1:172.21.15.163:6789/0 conn(0x7fdeb80acc00 legacy=0x7fdeb80c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:55.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.461+0000 7fdeb77fe640 1 -- 172.21.15.17:0/2208784018 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8070aa0 con 0x7fdeb8156ea0 2024-03-06T07:29:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.462+0000 7fdeb77fe640 1 --2- 172.21.15.17:0/2208784018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb8156ea0 0x7fdeb80702c0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fdeb81565d0 tx=0x7fdeac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc2cf61c072edbc server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.462+0000 7fdeb6ffd640 1 -- 172.21.15.17:0/2208784018 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac035020 con 0x7fdeb8156ea0 2024-03-06T07:29:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.462+0000 7fdeb6ffd640 1 -- 172.21.15.17:0/2208784018 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac031870 con 0x7fdeb8156ea0 2024-03-06T07:29:55.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.462+0000 7fdeb6ffd640 1 -- 172.21.15.17:0/2208784018 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac031c40 con 0x7fdeb8156ea0 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 -- 172.21.15.17:0/2208784018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb8156ea0 msgr2=0x7fdeb80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/2208784018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb8156ea0 0x7fdeb80702c0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7fdeb81565d0 tx=0x7fdeac02b910 comp rx=0 tx=0).stop 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 -- 172.21.15.17:0/2208784018 shutdown_connections 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/2208784018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb8156ea0 0x7fdeb80702c0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/2208784018 >> v2:172.21.15.163:3300/0 conn(0x7fdeb81506b0 0x7fdeb8153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 -- 172.21.15.17:0/2208784018 >> 172.21.15.17:0/2208784018 conn(0x7fdeb806b1a0 msgr2=0x7fdeb806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 -- 172.21.15.17:0/2208784018 shutdown_connections 2024-03-06T07:29:55.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.463+0000 7fdebe6b5640 1 -- 172.21.15.17:0/2208784018 wait complete. 2024-03-06T07:29:55.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.464+0000 7fdebe6b5640 1 Processor -- start 2024-03-06T07:29:55.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.464+0000 7fdebe6b5640 1 -- start start 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb81506b0 0x7fdeb8143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 0x7fdeb8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80c5bb0 con 0x7fdeb80acc00 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb81566a0 con 0x7fdeb81506b0 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebe6b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80c5d30 con 0x7fdeb8156ea0 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34758/0 (socket says 172.21.15.17:34758) 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 -- 172.21.15.17:0/148003348 learned_addr learned my addr 172.21.15.17:0/148003348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb77fe640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb81506b0 0x7fdeb8143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebcc52640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 0x7fdeb8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 -- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 msgr2=0x7fdeb8143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 0x7fdeb8143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 -- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb81506b0 msgr2=0x7fdeb8143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb81506b0 0x7fdeb8143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 -- 172.21.15.17:0/148003348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb81e8aa0 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdebcc52640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 0x7fdeb8143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb7fff640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fdea800c7f0 tx=0x7fdea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.465+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8015070 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.466+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea8004240 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.466+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8007620 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.466+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb80ab170 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.466+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb81ecda0 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.467+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb804f330 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.467+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdea80079d0 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.468+0000 7fdeb4ff9640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 0x7fdea406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.468+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fdea808a300 con 0x7fdeb80acc00 2024-03-06T07:29:55.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.468+0000 7fdeb77fe640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 0x7fdea406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:55.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.468+0000 7fdeb77fe640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 0x7fdea406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdeac036000 tx=0x7fdeac03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:55.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.472+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdea80589e0 con 0x7fdeb80acc00 2024-03-06T07:29:55.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.902+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdeb8068020 con 0x7fdeb80acc00 2024-03-06T07:29:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.902+0000 7fdeb4ff9640 1 -- 172.21.15.17:0/148003348 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5011 (secure 0 0 0) 0x7fdea805ca80 con 0x7fdeb80acc00 2024-03-06T07:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":205,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196},{"gid":9547,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:25.090279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9556":{"gid":9556,"name":"d","rank":1,"incarnation":202,"state":"up:stopping","state_seq":4,"addr":"172.21.15.163:6837/370957199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":370957199},{"type":"v1","addr":"172.21.15.163:6837","nonce":370957199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:55.904 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 205 2024-03-06T07:29:55.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 msgr2=0x7fdea406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 0x7fdea406d9c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdeac036000 tx=0x7fdeac03c040 comp rx=0 tx=0).stop 2024-03-06T07:29:55.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 msgr2=0x7fdeb8155250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:55.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7fdea800c7f0 tx=0x7fdea800ccb0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 shutdown_connections 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdea406b500 0x7fdea406d9c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdeb8156ea0 0x7fdeb8143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdeb81506b0 0x7fdeb8143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 --2- 172.21.15.17:0/148003348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdeb80acc00 0x7fdeb8155250 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 >> 172.21.15.17:0/148003348 conn(0x7fdeb806b1a0 msgr2=0x7fdeb814e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 shutdown_connections 2024-03-06T07:29:55.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:55.904+0000 7fdebe6b5640 1 -- 172.21.15.17:0/148003348 wait complete. 2024-03-06T07:29:56.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 205, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:25.090279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9556': {'gid': 9556, 'name': 'd', 'rank': 1, 'incarnation': 202, 'state': 'up:stopping', 'state_seq': 4, 'addr': '172.21.15.163:6837/370957199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6836', 'nonce': 370957199}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 370957199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:56.320 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9556:up:stopping 2024-03-06T07:29:57.320 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:57.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.424+0000 7f564e756640 1 Processor -- start 2024-03-06T07:29:57.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.424+0000 7f564e756640 1 -- start start 2024-03-06T07:29:57.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564e756640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 0x7f5648155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564e756640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5648158b60 0x7f5648158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564e756640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56480c73d0 con 0x7f564814f360 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564e756640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480c8dd0 con 0x7f5648158b60 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564e756640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480be2e0 con 0x7f56480c88c0 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564ccf3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5648158b60 0x7f5648158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564ccf3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5648158b60 0x7f5648158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34774/0 (socket says 172.21.15.17:34774) 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.425+0000 7f564ccf3640 1 -- 172.21.15.17:0/217407431 learned_addr learned my addr 172.21.15.17:0/217407431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f564ccf3640 1 -- 172.21.15.17:0/217407431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 msgr2=0x7f5648155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f56477fe640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 0x7f5648155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f564ccf3640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 0x7f5648155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f564ccf3640 1 -- 172.21.15.17:0/217407431 >> v1:172.21.15.163:6789/0 conn(0x7f564814f360 legacy=0x7f56480c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f564ccf3640 1 -- 172.21.15.17:0/217407431 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480c7670 con 0x7f5648158b60 2024-03-06T07:29:57.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f56477fe640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 0x7f5648155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.426+0000 7f564ccf3640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5648158b60 0x7f5648158ff0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f5640009e50 tx=0x7f564002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c4703548364171 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f5646ffd640 1 -- 172.21.15.17:0/217407431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564002c9e0 con 0x7f5648158b60 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f5646ffd640 1 -- 172.21.15.17:0/217407431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564002cb80 con 0x7f5648158b60 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f5646ffd640 1 -- 172.21.15.17:0/217407431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400343f0 con 0x7f5648158b60 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 -- 172.21.15.17:0/217407431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5648158b60 msgr2=0x7f5648158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5648158b60 0x7f5648158ff0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f5640009e50 tx=0x7f564002be40 comp rx=0 tx=0).stop 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 -- 172.21.15.17:0/217407431 shutdown_connections 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5648158b60 0x7f5648158ff0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 --2- 172.21.15.17:0/217407431 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56480c88c0 0x7f5648155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 -- 172.21.15.17:0/217407431 >> 172.21.15.17:0/217407431 conn(0x7f56480ac570 msgr2=0x7f56480ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 -- 172.21.15.17:0/217407431 shutdown_connections 2024-03-06T07:29:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.427+0000 7f564e756640 1 -- 172.21.15.17:0/217407431 wait complete. 2024-03-06T07:29:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.428+0000 7f564e756640 1 Processor -- start 2024-03-06T07:29:57.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 -- start start 2024-03-06T07:29:57.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 0x7f56480c0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 0x7f56480bf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c5a50 con 0x7f56480c88c0 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c7c30 con 0x7f5648158b60 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f564e756640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56480c5bd0 con 0x7f564814f360 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f5647fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f5647fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34786/0 (socket says 172.21.15.17:34786) 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.429+0000 7f5647fff640 1 -- 172.21.15.17:0/2376250266 learned_addr learned my addr 172.21.15.17:0/2376250266 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f564ccf3640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 0x7f56480bf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f56477fe640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 0x7f56480c0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 -- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 msgr2=0x7f56480c0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 0x7f56480c0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 -- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 msgr2=0x7f56480bf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:57.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 0x7f56480bf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 -- 172.21.15.17:0/2376250266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480be5c0 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f564ccf3640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 0x7f56480bf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f56477fe640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 0x7f56480c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5647fff640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b420 tx=0x7f563c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c011830 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c016070 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56480be8a0 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c011c10 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.430+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56480bee60 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.431+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564804f330 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.432+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f563c01b460 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.432+0000 7f5644ff9640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 0x7f563406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.432+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f563c089ed0 con 0x7f56480c88c0 2024-03-06T07:29:57.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.432+0000 7f56477fe640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 0x7f563406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:57.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.433+0000 7f56477fe640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 0x7f563406d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5638006fd0 tx=0x7f5638008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:57.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.440+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f563c020020 con 0x7f56480c88c0 2024-03-06T07:29:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.658+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56481554a0 con 0x7f56480c88c0 2024-03-06T07:29:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.659+0000 7f5644ff9640 1 -- 172.21.15.17:0/2376250266 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+4219 (secure 0 0 0) 0x7f563c058ba0 con 0x7f56480c88c0 2024-03-06T07:29:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:57.661 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":206,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:56.878381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:replay","state_seq":1,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 206 2024-03-06T07:29:57.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.660+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 msgr2=0x7f563406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.660+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 0x7f563406d990 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5638006fd0 tx=0x7f5638008040 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.660+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 msgr2=0x7f56481eb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.660+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f563c00b420 tx=0x7f563c00b8e0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 shutdown_connections 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f563406b4d0 0x7f563406d990 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5648158b60 0x7f56480bf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f564814f360 0x7f56480c0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 --2- 172.21.15.17:0/2376250266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f56480c88c0 0x7f56481eb770 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 >> 172.21.15.17:0/2376250266 conn(0x7f56480ac570 msgr2=0x7f5648153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 shutdown_connections 2024-03-06T07:29:57.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:57.661+0000 7f564e756640 1 -- 172.21.15.17:0/2376250266 wait complete. 2024-03-06T07:29:58.075 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 206, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:56.878381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:58.076 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:replay 2024-03-06T07:29:59.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:29:59.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.177+0000 7f378a29d640 1 Processor -- start 2024-03-06T07:29:59.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.177+0000 7f378a29d640 1 -- start start 2024-03-06T07:29:59.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f378a29d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f378414f360 0x7f3784155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f378a29d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 0x7f37840c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f378a29d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37840c73d0 con 0x7f37840c5880 2024-03-06T07:29:59.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f378a29d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37840c8dd0 con 0x7f378414f360 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f378a29d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37840be2e0 con 0x7f37840c88c0 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f3783fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f378414f360 0x7f3784155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f3783fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f378414f360 0x7f3784155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34792/0 (socket says 172.21.15.17:34792) 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f3783fff640 1 -- 172.21.15.17:0/2558156265 learned_addr learned my addr 172.21.15.17:0/2558156265 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.178+0000 7f37837fe640 1 --2- 172.21.15.17:0/2558156265 >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 0x7f37840c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3783fff640 1 -- 172.21.15.17:0/2558156265 >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 msgr2=0x7f37840c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3783fff640 1 --2- 172.21.15.17:0/2558156265 >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 0x7f37840c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3783fff640 1 -- 172.21.15.17:0/2558156265 >> v1:172.21.15.163:6789/0 conn(0x7f37840c5880 legacy=0x7f3784158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3783fff640 1 -- 172.21.15.17:0/2558156265 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840c7670 con 0x7f378414f360 2024-03-06T07:29:59.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f37837fe640 1 --2- 172.21.15.17:0/2558156265 >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 0x7f37840c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3783fff640 1 --2- 172.21.15.17:0/2558156265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378414f360 0x7f3784155c10 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f37840c84c0 tx=0x7f377402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af15c400bfa2a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.179+0000 7f3782ffd640 1 -- 172.21.15.17:0/2558156265 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774035020 con 0x7f378414f360 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f3782ffd640 1 -- 172.21.15.17:0/2558156265 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3774031870 con 0x7f378414f360 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f3782ffd640 1 -- 172.21.15.17:0/2558156265 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031c40 con 0x7f378414f360 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 -- 172.21.15.17:0/2558156265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378414f360 msgr2=0x7f3784155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 --2- 172.21.15.17:0/2558156265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378414f360 0x7f3784155c10 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f37840c84c0 tx=0x7f377402b910 comp rx=0 tx=0).stop 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 -- 172.21.15.17:0/2558156265 shutdown_connections 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 --2- 172.21.15.17:0/2558156265 >> v2:172.21.15.163:3300/0 conn(0x7f37840c88c0 0x7f37840c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 --2- 172.21.15.17:0/2558156265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378414f360 0x7f3784155c10 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 -- 172.21.15.17:0/2558156265 >> 172.21.15.17:0/2558156265 conn(0x7f37840ac570 msgr2=0x7f37840ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 -- 172.21.15.17:0/2558156265 shutdown_connections 2024-03-06T07:29:59.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.180+0000 7f378a29d640 1 -- 172.21.15.17:0/2558156265 wait complete. 2024-03-06T07:29:59.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.182+0000 7f378a29d640 1 Processor -- start 2024-03-06T07:29:59.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.182+0000 7f378a29d640 1 -- start start 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37840c88c0 0x7f37840bed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f378414f360 0x7f37840bf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784159270 con 0x7f37840c5880 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37840c7c30 con 0x7f37840c88c0 2024-03-06T07:29:59.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f378a29d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37841593f0 con 0x7f378414f360 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f3783fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f3783fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34804/0 (socket says 172.21.15.17:34804) 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.183+0000 7f3783fff640 1 -- 172.21.15.17:0/2710324706 learned_addr learned my addr 172.21.15.17:0/2710324706 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f378883a640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f378414f360 0x7f37840bf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f37837fe640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37840c88c0 0x7f37840bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 -- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f378414f360 msgr2=0x7f37840bf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f378414f360 0x7f37840bf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 -- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37840c88c0 msgr2=0x7f37840bed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37840c88c0 0x7f37840bed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 -- 172.21.15.17:0/2710324706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37840afaa0 con 0x7f37840c5880 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3783fff640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f377402b950 tx=0x7f37740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774040070 con 0x7f37840c5880 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3774030d00 con 0x7f37840c5880 2024-03-06T07:29:59.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774035040 con 0x7f37840c5880 2024-03-06T07:29:59.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37841eebf0 con 0x7f37840c5880 2024-03-06T07:29:59.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.184+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37841ee8f0 con 0x7f37840c5880 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.185+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378404f330 con 0x7f37840c5880 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.186+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f377404b060 con 0x7f37840c5880 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.186+0000 7f3780ff9640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.186+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f37740ac1e0 con 0x7f37840c5880 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.186+0000 7f37837fe640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:29:59.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.187+0000 7f37837fe640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f37840abdd0 tx=0x7f376c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:29:59.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.194+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f377407ada0 con 0x7f37840c5880 2024-03-06T07:29:59.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.412+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f378406cda0 con 0x7f37840c5880 2024-03-06T07:29:59.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.412+0000 7f3780ff9640 1 -- 172.21.15.17:0/2710324706 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+4223 (secure 0 0 0) 0x7f377407a6a0 con 0x7f37840c5880 2024-03-06T07:29:59.414 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:29:59.414 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":208,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:29:57.884199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:reconnect","state_seq":16,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:29:59.415 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 208 2024-03-06T07:29:59.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 msgr2=0x7f376406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f37840abdd0 tx=0x7f376c008040 comp rx=0 tx=0).stop 2024-03-06T07:29:59.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 msgr2=0x7f37841ee3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:29:59.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f377402b950 tx=0x7f37740046c0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 shutdown_connections 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f378414f360 0x7f37840bf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37840c88c0 0x7f37840bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 --2- 172.21.15.17:0/2710324706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37840c5880 0x7f37841ee3b0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 >> 172.21.15.17:0/2710324706 conn(0x7f37840ac570 msgr2=0x7f378414cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.414+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 shutdown_connections 2024-03-06T07:29:59.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:29:59.415+0000 7f378a29d640 1 -- 172.21.15.17:0/2710324706 wait complete. 2024-03-06T07:29:59.829 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 208, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:29:57.884199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:reconnect', 'state_seq': 16, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:29:59.830 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:reconnect 2024-03-06T07:30:00.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.932+0000 7fe1e1d65640 1 Processor -- start 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.932+0000 7fe1e1d65640 1 -- start start 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1e1d65640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1dc156ea0 0x7fe1dc0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1e1d65640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0ad310 con 0x7fe1dc1506b0 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1e1d65640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc1572a0 con 0x7fe1dc153990 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1e1d65640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc156450 con 0x7fe1dc156ea0 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe1dc156ea0 0x7fe1dc0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1db7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe1dc1506b0 0x7fe1dc0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46638/0 (socket says 172.21.15.17:46638) 2024-03-06T07:30:00.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1db7fe640 1 -- 172.21.15.17:0/3333211425 learned_addr learned my addr 172.21.15.17:0/3333211425 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:00.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1dbfff640 1 -- 172.21.15.17:0/3333211425 >> v1:172.21.15.163:6789/0 conn(0x7fe1dc153990 legacy=0x7fe1dc0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:00.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1dbfff640 1 -- 172.21.15.17:0/3333211425 >> v1:172.21.15.17:6789/0 conn(0x7fe1dc1506b0 legacy=0x7fe1dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1dbfff640 1 -- 172.21.15.17:0/3333211425 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc070aa0 con 0x7fe1dc156ea0 2024-03-06T07:30:00.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.933+0000 7fe1da7fc640 1 -- 172.21.15.17:0/3333211425 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625949519 0 0) 0x7fe1dc0ad310 con 0x7fe1dc1506b0 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.934+0000 7fe1dbfff640 1 --2- 172.21.15.17:0/3333211425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0c54a0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009d90 tx=0x7fe1d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3616f74e633162f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.934+0000 7fe1da7fc640 1 -- 172.21.15.17:0/3333211425 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4039070 con 0x7fe1dc156ea0 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.934+0000 7fe1da7fc640 1 -- 172.21.15.17:0/3333211425 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d402cc60 con 0x7fe1dc156ea0 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.934+0000 7fe1da7fc640 1 -- 172.21.15.17:0/3333211425 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4034660 con 0x7fe1dc156ea0 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 -- 172.21.15.17:0/3333211425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc156ea0 msgr2=0x7fe1dc0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/3333211425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0c54a0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009d90 tx=0x7fe1d402beb0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 -- 172.21.15.17:0/3333211425 shutdown_connections 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/3333211425 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0c54a0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 -- 172.21.15.17:0/3333211425 >> 172.21.15.17:0/3333211425 conn(0x7fe1dc06b1a0 msgr2=0x7fe1dc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 -- 172.21.15.17:0/3333211425 shutdown_connections 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.935+0000 7fe1e1d65640 1 -- 172.21.15.17:0/3333211425 wait complete. 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.937+0000 7fe1e1d65640 1 Processor -- start 2024-03-06T07:30:00.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.937+0000 7fe1e1d65640 1 -- start start 2024-03-06T07:30:00.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1dc1506b0 0x7fe1dc1ec730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc1566a0 con 0x7fe1dc153990 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc156a10 con 0x7fe1dc156ea0 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1e1d65640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c59e0 con 0x7fe1dc1506b0 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1daffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43800/0 (socket says 172.21.15.17:43800) 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.938+0000 7fe1daffd640 1 -- 172.21.15.17:0/1377837707 learned_addr learned my addr 172.21.15.17:0/1377837707 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1dbfff640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1db7fe640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1dc1506b0 0x7fe1dc1ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 -- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1dc1506b0 msgr2=0x7fe1dc1ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1dc1506b0 0x7fe1dc1ec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 -- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 msgr2=0x7fe1dc0a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 -- 172.21.15.17:0/1377837707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc1ecc70 con 0x7fe1dc153990 2024-03-06T07:30:00.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1dbfff640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1daffd640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c460 tx=0x7fe1cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc016020 con 0x7fe1dc153990 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc012870 con 0x7fe1dc153990 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc012c60 con 0x7fe1dc153990 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0aaf60 con 0x7fe1dc153990 2024-03-06T07:30:00.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.939+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc0ab520 con 0x7fe1dc153990 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.940+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc067fd0 con 0x7fe1dc153990 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.941+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1cc019020 con 0x7fe1dc153990 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.942+0000 7fe1d3fff640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.942+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fe1cc08a4e0 con 0x7fe1dc153990 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.942+0000 7fe1db7fe640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:00.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.942+0000 7fe1db7fe640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4000f60 tx=0x7fe1c4007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:00.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:00.946+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1dc067fd0 con 0x7fe1dc153990 2024-03-06T07:30:01.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.167+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1dc04f330 con 0x7fe1dc153990 2024-03-06T07:30:01.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.167+0000 7fe1d3fff640 1 -- 172.21.15.17:0/1377837707 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fe1cc020020 con 0x7fe1dc153990 2024-03-06T07:30:01.169 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:01.169 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:01.169 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 msgr2=0x7fe1bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4000f60 tx=0x7fe1c4007b50 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 msgr2=0x7fe1dc155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c460 tx=0x7fe1cc00c920 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 shutdown_connections 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe1bc06b4d0 0x7fe1bc06d990 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1dc156ea0 0x7fe1dc0a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1dc153990 0x7fe1dc155640 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 --2- 172.21.15.17:0/1377837707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1dc1506b0 0x7fe1dc1ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 >> 172.21.15.17:0/1377837707 conn(0x7fe1dc06b1a0 msgr2=0x7fe1dc0702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:01.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 shutdown_connections 2024-03-06T07:30:01.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:01.169+0000 7fe1e1d65640 1 -- 172.21.15.17:0/1377837707 wait complete. 2024-03-06T07:30:01.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:01.587 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:02.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.697+0000 7f209e55e640 1 Processor -- start 2024-03-06T07:30:02.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.697+0000 7f209e55e640 1 -- start start 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209e55e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 0x7f20980c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209e55e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20980c7350 0x7f20980c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209e55e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209806c150 con 0x7f2098155c20 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209e55e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209806c480 con 0x7f20980c7350 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209e55e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209814f630 con 0x7f209814f250 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20980c7350 0x7f20980c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f20980c7350 0x7f20980c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43802/0 (socket says 172.21.15.17:43802) 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 -- 172.21.15.17:0/3115336260 learned_addr learned my addr 172.21.15.17:0/3115336260 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f2097fff640 1 --2- 172.21.15.17:0/3115336260 >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 0x7f20980c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 -- 172.21.15.17:0/3115336260 >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 msgr2=0x7f20980c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 --2- 172.21.15.17:0/3115336260 >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 0x7f20980c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 -- 172.21.15.17:0/3115336260 >> v1:172.21.15.163:6789/0 conn(0x7f2098155c20 legacy=0x7f209806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.698+0000 7f209cafb640 1 -- 172.21.15.17:0/3115336260 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980c5660 con 0x7f20980c7350 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.699+0000 7f2097fff640 1 --2- 172.21.15.17:0/3115336260 >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 0x7f20980c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.699+0000 7f209cafb640 1 --2- 172.21.15.17:0/3115336260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20980c7350 0x7f20980c77c0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad2d0 tx=0x7f209002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29006cde91d6012c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.699+0000 7f2096ffd640 1 -- 172.21.15.17:0/3115336260 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209002c960 con 0x7f20980c7350 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.699+0000 7f2096ffd640 1 -- 172.21.15.17:0/3115336260 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209002cb00 con 0x7f20980c7350 2024-03-06T07:30:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.699+0000 7f2096ffd640 1 -- 172.21.15.17:0/3115336260 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900343f0 con 0x7f20980c7350 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 -- 172.21.15.17:0/3115336260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20980c7350 msgr2=0x7f20980c77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 --2- 172.21.15.17:0/3115336260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20980c7350 0x7f20980c77c0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f20980ad2d0 tx=0x7f209002bdc0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 -- 172.21.15.17:0/3115336260 shutdown_connections 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 --2- 172.21.15.17:0/3115336260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20980c7350 0x7f20980c77c0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 --2- 172.21.15.17:0/3115336260 >> v2:172.21.15.163:3300/0 conn(0x7f209814f250 0x7f20980c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 -- 172.21.15.17:0/3115336260 >> 172.21.15.17:0/3115336260 conn(0x7f20980ac4d0 msgr2=0x7f20980ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 -- 172.21.15.17:0/3115336260 shutdown_connections 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.700+0000 7f209e55e640 1 -- 172.21.15.17:0/3115336260 wait complete. 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.701+0000 7f209e55e640 1 Processor -- start 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.701+0000 7f209e55e640 1 -- start start 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980c7350 0x7f209814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 0x7f20981589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20980c5c20 con 0x7f2098155c20 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209806daf0 con 0x7f20980c7350 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209e55e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20981560a0 con 0x7f209814f250 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43804/0 (socket says 172.21.15.17:43804) 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 -- 172.21.15.17:0/4090302006 learned_addr learned my addr 172.21.15.17:0/4090302006 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f2097fff640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980c7350 0x7f209814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f20977fe640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 0x7f20981589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 -- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 msgr2=0x7f20981589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 0x7f20981589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 -- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980c7350 msgr2=0x7f209814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980c7350 0x7f209814ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f209cafb640 1 -- 172.21.15.17:0/4090302006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980c2930 con 0x7f2098155c20 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.702+0000 7f20977fe640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 0x7f20981589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f209cafb640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f2090036040 tx=0x7f2090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090043020 con 0x7f2098155c20 2024-03-06T07:30:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209003e450 con 0x7f2098155c20 2024-03-06T07:30:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209003e840 con 0x7f2098155c20 2024-03-06T07:30:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20980c1bb0 con 0x7f2098155c20 2024-03-06T07:30:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.703+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098150780 con 0x7f2098155c20 2024-03-06T07:30:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.704+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209804f330 con 0x7f2098155c20 2024-03-06T07:30:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.704+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2090046020 con 0x7f2098155c20 2024-03-06T07:30:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.705+0000 7f2094ff9640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.705+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f20900acce0 con 0x7f2098155c20 2024-03-06T07:30:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.705+0000 7f2097fff640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.706+0000 7f2097fff640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ab60 tx=0x7f208c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.709+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f209004d030 con 0x7f2098155c20 2024-03-06T07:30:02.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.927+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f209806d860 con 0x7f2098155c20 2024-03-06T07:30:02.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.927+0000 7f2094ff9640 1 -- 172.21.15.17:0/4090302006 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f209007b9b0 con 0x7f2098155c20 2024-03-06T07:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:02.929 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:02.930 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.929+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 msgr2=0x7f208406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.929+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 0x7f208406d990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ab60 tx=0x7f208c008040 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.929+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 msgr2=0x7f20981ebcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.929+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f2090036040 tx=0x7f2090004a40 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 shutdown_connections 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f208406b4d0 0x7f208406d990 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2098155c20 0x7f20981ebcf0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f209814f250 0x7f20981589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 --2- 172.21.15.17:0/4090302006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980c7350 0x7f209814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 >> 172.21.15.17:0/4090302006 conn(0x7f20980ac4d0 msgr2=0x7f20980aa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 shutdown_connections 2024-03-06T07:30:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:02.930+0000 7f209e55e640 1 -- 172.21.15.17:0/4090302006 wait complete. 2024-03-06T07:30:03.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:03.344 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:04.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:04.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.444+0000 7fbcaaf3e640 1 Processor -- start 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.444+0000 7fbcaaf3e640 1 -- start start 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbcaaf3e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbca40c88c0 0x7fbca4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbcaaf3e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca40c73d0 con 0x7fbca4158b60 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbcaaf3e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca40be2e0 con 0x7fbca414f360 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbcaaf3e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca4159700 con 0x7fbca40c88c0 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbca94db640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbca4158b60 0x7fbca4158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46652/0 (socket says 172.21.15.17:46652) 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbca94db640 1 -- 172.21.15.17:0/1568879140 learned_addr learned my addr 172.21.15.17:0/1568879140 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.445+0000 7fbca3fff640 1 --2- 172.21.15.17:0/1568879140 >> v2:172.21.15.163:3300/0 conn(0x7fbca40c88c0 0x7fbca4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381150380 0 0) 0x7fbca40c73d0 con 0x7fbca4158b60 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc94003520 con 0x7fbca4158b60 2024-03-06T07:30:04.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628001514 0 0) 0x7fbc94003520 con 0x7fbca4158b60 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 >> v2:172.21.15.163:3300/0 conn(0x7fbca40c88c0 msgr2=0x7fbca4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 --2- 172.21.15.17:0/1568879140 >> v2:172.21.15.163:3300/0 conn(0x7fbca40c88c0 0x7fbca4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 >> v1:172.21.15.163:6789/0 conn(0x7fbca414f360 legacy=0x7fbca40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca40c7670 con 0x7fbca4158b60 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbc9c0019f0 con 0x7fbca4158b60 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 >> v1:172.21.15.17:6789/0 conn(0x7fbca4158b60 legacy=0x7fbca4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc940039b0 0x7fbc94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc94004800 0x7fbc94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbc9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca4159700 con 0x7fbc940051c0 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c160 con 0x7fbc94004800 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9400c360 con 0x7fbc940039b0 2024-03-06T07:30:04.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca3fff640 1 --2- 172.21.15.17:0/1568879140 >> v2:172.21.15.163:3300/0 conn(0x7fbca40c88c0 0x7fbca4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbc9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca8cda640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc940039b0 0x7fbc94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca3fff640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc94004800 0x7fbc94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 -- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc940039b0 msgr2=0x7fbc94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc940039b0 0x7fbc94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 -- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc94004800 msgr2=0x7fbc94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc94004800 0x7fbc94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.446+0000 7fbca94db640 1 -- 172.21.15.17:0/1568879140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc94003520 con 0x7fbc940051c0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.447+0000 7fbca94db640 1 -- 172.21.15.17:0/1568879140 reap_dead start 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.447+0000 7fbca94db640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbc9400b9c0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00a850 tx=0x7fbc94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640a7e6bc9bb3c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.447+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c00a4a0 con 0x7fbc940051c0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.447+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c034650 con 0x7fbc940051c0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.447+0000 7fbca37fe640 1 -- 172.21.15.17:0/1568879140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c034a40 con 0x7fbc940051c0 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 msgr2=0x7fbc9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbc9400b9c0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00a850 tx=0x7fbc94005830 comp rx=0 tx=0).stop 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/1568879140 shutdown_connections 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/1568879140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbc9400b9c0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/1568879140 >> 172.21.15.17:0/1568879140 conn(0x7fbca40ac570 msgr2=0x7fbca40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/1568879140 shutdown_connections 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.448+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/1568879140 wait complete. 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.449+0000 7fbcaaf3e640 1 Processor -- start 2024-03-06T07:30:04.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.450+0000 7fbcaaf3e640 1 -- start start 2024-03-06T07:30:04.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbca41509f0 0x7fbca4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbca414acd0 0x7fbca414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca40c7c30 con 0x7fbc940051c0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca4155c10 con 0x7fbca414acd0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbcaaf3e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca4155d90 con 0x7fbca41509f0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43828/0 (socket says 172.21.15.17:43828) 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 -- 172.21.15.17:0/3191788061 learned_addr learned my addr 172.21.15.17:0/3191788061 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 -- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbca41509f0 msgr2=0x7fbca4157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca3fff640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbca41509f0 0x7fbca4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca94db640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbca414acd0 0x7fbca414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbca41509f0 0x7fbca4157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 -- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbca414acd0 msgr2=0x7fbca414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbca414acd0 0x7fbca414b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 -- 172.21.15.17:0/3191788061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca414f990 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.451+0000 7fbca8cda640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800e480 tx=0x7fbc9800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.452+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc980052e0 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.452+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc98018070 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.452+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98003830 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.452+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca414fc70 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.452+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca4150230 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.453+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca404f330 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.453+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc980039d0 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.453+0000 7fbca17fa640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.453+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fbc9808a090 con 0x7fbc940051c0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.454+0000 7fbca3fff640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:04.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.454+0000 7fbca3fff640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 0x7fbc8806d9c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbca414d1e0 tx=0x7fbc90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:04.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.458+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc98058cd0 con 0x7fbc940051c0 2024-03-06T07:30:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.678+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbca40c8d50 con 0x7fbc940051c0 2024-03-06T07:30:04.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.678+0000 7fbca17fa640 1 -- 172.21.15.17:0/3191788061 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fbc980585d0 con 0x7fbc940051c0 2024-03-06T07:30:04.680 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:04.681 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:04.681 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 msgr2=0x7fbc8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 0x7fbc8806d9c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbca414d1e0 tx=0x7fbc90008040 comp rx=0 tx=0).stop 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 msgr2=0x7fbca40b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fbc9800e480 tx=0x7fbc9800e940 comp rx=0 tx=0).stop 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 shutdown_connections 2024-03-06T07:30:04.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbca414acd0 0x7fbca414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbca41509f0 0x7fbca4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 --2- 172.21.15.17:0/3191788061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc940051c0 0x7fbca40b0120 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 >> 172.21.15.17:0/3191788061 conn(0x7fbca40ac570 msgr2=0x7fbca414e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 shutdown_connections 2024-03-06T07:30:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:04.680+0000 7fbcaaf3e640 1 -- 172.21.15.17:0/3191788061 wait complete. 2024-03-06T07:30:05.095 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:05.095 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:06.096 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.196+0000 7f46aaf1e640 1 Processor -- start 2024-03-06T07:30:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.196+0000 7f46aaf1e640 1 -- start start 2024-03-06T07:30:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46aaf1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f46a406ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46aaf1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a40c1ae0 con 0x7f46a4152520 2024-03-06T07:30:06.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46aaf1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a406e450 con 0x7f46a40720a0 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46aaf1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a40c1dc0 con 0x7f46a4155c80 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a8cba640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f46a4152520 0x7f46a40c12b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46666/0 (socket says 172.21.15.17:46666) 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a8cba640 1 -- 172.21.15.17:0/2223478042 learned_addr learned my addr 172.21.15.17:0/2223478042 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045942548 0 0) 0x7f46a40c1ae0 con 0x7f46a4152520 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4694003520 con 0x7f46a4152520 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a3fff640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f46a406ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3923224087 0 0) 0x7f4694003520 con 0x7f46a4152520 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 msgr2=0x7f46a406ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f46a406ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 >> v1:172.21.15.163:6789/0 conn(0x7f46a40720a0 legacy=0x7f46a406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a406e730 con 0x7f46a4152520 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f4698001670 con 0x7f46a4152520 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 >> v1:172.21.15.17:6789/0 conn(0x7f46a4152520 legacy=0x7f46a40c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f4694003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f4694004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4694004d40 0x7f46940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a40c1dc0 con 0x7f46940039e0 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469400bb50 con 0x7f4694004d40 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.197+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469400bd20 con 0x7f46a4155c80 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f4694003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a3fff640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f4694004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a94bb640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4694004d40 0x7f46940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 -- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 msgr2=0x7f4694004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a4155c80 0x7f4694004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 -- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4694004d40 msgr2=0x7f46940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4694004d40 0x7f46940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.198+0000 7f46a8cba640 1 -- 172.21.15.17:0/2223478042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694003520 con 0x7f46940039e0 2024-03-06T07:30:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46a8cba640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f4694003e00 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f469800c270 tx=0x7f4694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b55061268c9beb server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698007cf0 con 0x7f46940039e0 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698030070 con 0x7f46940039e0 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46a37fe640 1 -- 172.21.15.17:0/2223478042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698005d30 con 0x7f46940039e0 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 -- 172.21.15.17:0/2223478042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 msgr2=0x7f4694003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/2223478042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f4694003e00 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f469800c270 tx=0x7f4694005830 comp rx=0 tx=0).stop 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46a94bb640 1 -- 172.21.15.17:0/2223478042 reap_dead start 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 -- 172.21.15.17:0/2223478042 shutdown_connections 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 -- 172.21.15.17:0/2223478042 >> 172.21.15.17:0/2223478042 conn(0x7f46a40c76b0 msgr2=0x7f46a40c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 -- 172.21.15.17:0/2223478042 shutdown_connections 2024-03-06T07:30:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.199+0000 7f46aaf1e640 1 -- 172.21.15.17:0/2223478042 wait complete. 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.200+0000 7f46aaf1e640 1 Processor -- start 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.200+0000 7f46aaf1e640 1 -- start start 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a41590c0 0x7f46a40a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a40a7210 0x7f46a414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a406b850 con 0x7f46940039e0 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a406b6d0 con 0x7f46a40a7210 2024-03-06T07:30:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46aaf1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4152900 con 0x7f46a41590c0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46a8cba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46a8cba640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43848/0 (socket says 172.21.15.17:43848) 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.201+0000 7f46a8cba640 1 -- 172.21.15.17:0/329828806 learned_addr learned my addr 172.21.15.17:0/329828806 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a3fff640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a41590c0 0x7f46a40a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a94bb640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a40a7210 0x7f46a414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 -- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a41590c0 msgr2=0x7f46a40a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a41590c0 0x7f46a40a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 -- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a40a7210 msgr2=0x7f46a414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a40a7210 0x7f46a414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 -- 172.21.15.17:0/329828806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a4150ea0 con 0x7f46940039e0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a8cba640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f469800c210 tx=0x7f46a4154fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698045070 con 0x7f46940039e0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698004bf0 con 0x7f46940039e0 2024-03-06T07:30:06.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a41ee140 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698007cf0 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.202+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a41e2430 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.203+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a40a75f0 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.203+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4698007070 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.204+0000 7f46a17fa640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 0x7f468806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.204+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f46980ab3a0 con 0x7f46940039e0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.204+0000 7f46a3fff640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 0x7f468806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.205+0000 7f46a3fff640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 0x7f468806d9c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f46a4157a30 tx=0x7f4690008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.208+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46a40a75f0 con 0x7f46940039e0 2024-03-06T07:30:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.427+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46a404f330 con 0x7f46940039e0 2024-03-06T07:30:06.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.428+0000 7f46a17fa640 1 -- 172.21.15.17:0/329828806 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f469807a150 con 0x7f46940039e0 2024-03-06T07:30:06.430 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:06.430 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:06.430 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 msgr2=0x7f468806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 0x7f468806d9c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f46a4157a30 tx=0x7f4690008040 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 msgr2=0x7f46a41e1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f469800c210 tx=0x7f46a4154fe0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 shutdown_connections 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f468806b500 0x7f468806d9c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46a40a7210 0x7f46a414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46a41590c0 0x7f46a40a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 --2- 172.21.15.17:0/329828806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46940039e0 0x7f46a41e1ef0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:06.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 >> 172.21.15.17:0/329828806 conn(0x7f46a40c76b0 msgr2=0x7f46a4154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 shutdown_connections 2024-03-06T07:30:06.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:06.430+0000 7f46aaf1e640 1 -- 172.21.15.17:0/329828806 wait complete. 2024-03-06T07:30:06.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:06.845 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:07.846 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:07.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.946+0000 7f9f3f28d640 1 Processor -- start 2024-03-06T07:30:07.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 -- start start 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f380c5480 0x7f9f38155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f380c5860 0x7f9f3806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f380c5cc0 con 0x7f9f381586f0 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3806df40 con 0x7f9f380c5480 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3f28d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f381591c0 con 0x7f9f380c5860 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3d029640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f380c5480 0x7f9f38155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3d029640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9f380c5480 0x7f9f38155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43856/0 (socket says 172.21.15.17:43856) 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3d82a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9f381586f0 0x7f9f380724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46670/0 (socket says 172.21.15.17:46670) 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.947+0000 7f9f3d029640 1 -- 172.21.15.17:0/1578212638 learned_addr learned my addr 172.21.15.17:0/1578212638 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.948+0000 7f9f3d029640 1 -- 172.21.15.17:0/1578212638 >> v2:172.21.15.163:3300/0 conn(0x7f9f380c5860 msgr2=0x7f9f3806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.948+0000 7f9f3d029640 1 --2- 172.21.15.17:0/1578212638 >> v2:172.21.15.163:3300/0 conn(0x7f9f380c5860 0x7f9f3806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.948+0000 7f9f3d029640 1 -- 172.21.15.17:0/1578212638 >> v1:172.21.15.17:6789/0 conn(0x7f9f381586f0 legacy=0x7f9f380724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.948+0000 7f9f3d029640 1 -- 172.21.15.17:0/1578212638 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f381594a0 con 0x7f9f380c5480 2024-03-06T07:30:07.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.948+0000 7f9f2ffff640 1 -- 172.21.15.17:0/1578212638 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175844635 0 0) 0x7f9f380c5cc0 con 0x7f9f381586f0 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.949+0000 7f9f3d029640 1 --2- 172.21.15.17:0/1578212638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f38155ba0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9f380c9770 tx=0x7f9f3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b870ac16f8a6de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.949+0000 7f9f2ffff640 1 -- 172.21.15.17:0/1578212638 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34035020 con 0x7f9f380c5480 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.949+0000 7f9f2ffff640 1 -- 172.21.15.17:0/1578212638 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34031870 con 0x7f9f380c5480 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.949+0000 7f9f2ffff640 1 -- 172.21.15.17:0/1578212638 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34031c40 con 0x7f9f380c5480 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 -- 172.21.15.17:0/1578212638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 msgr2=0x7f9f38155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/1578212638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f38155ba0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f9f380c9770 tx=0x7f9f3402b910 comp rx=0 tx=0).stop 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 -- 172.21.15.17:0/1578212638 shutdown_connections 2024-03-06T07:30:07.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/1578212638 >> v2:172.21.15.163:3300/0 conn(0x7f9f380c5860 0x7f9f3806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/1578212638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f38155ba0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 -- 172.21.15.17:0/1578212638 >> 172.21.15.17:0/1578212638 conn(0x7f9f3806b1a0 msgr2=0x7f9f3806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 -- 172.21.15.17:0/1578212638 shutdown_connections 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.950+0000 7f9f3f28d640 1 -- 172.21.15.17:0/1578212638 wait complete. 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 Processor -- start 2024-03-06T07:30:07.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 -- start start 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f380c5860 0x7f9f380ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f381586f0 0x7f9f381e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38152a20 con 0x7f9f380c5480 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38155680 con 0x7f9f380c5860 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3f28d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38155050 con 0x7f9f381586f0 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3d029640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.951+0000 7f9f3d029640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43864/0 (socket says 172.21.15.17:43864) 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 -- 172.21.15.17:0/2159067510 learned_addr learned my addr 172.21.15.17:0/2159067510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 -- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f381586f0 msgr2=0x7f9f381e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f381586f0 0x7f9f381e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 -- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f380c5860 msgr2=0x7f9f380ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f380c5860 0x7f9f380ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 -- 172.21.15.17:0/2159067510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38153ed0 con 0x7f9f380c5480 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3d029640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3403ad60 con 0x7f9f380c5480 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340443f0 con 0x7f9f380c5480 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f381541b0 con 0x7f9f380c5480 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340447e0 con 0x7f9f380c5480 2024-03-06T07:30:07.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.952+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f38154770 con 0x7f9f380c5480 2024-03-06T07:30:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.953+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3804f330 con 0x7f9f380c5480 2024-03-06T07:30:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.954+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f34038020 con 0x7f9f380c5480 2024-03-06T07:30:07.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.954+0000 7f9f2dffb640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 0x7f9f1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.954+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f9f340ac7d0 con 0x7f9f380c5480 2024-03-06T07:30:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.955+0000 7f9f3c828640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 0x7f9f1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:07.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.955+0000 7f9f3c828640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 0x7f9f1c06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f380aaca0 tx=0x7f9f28006c90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:07.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:07.959+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9f3403f070 con 0x7f9f380c5480 2024-03-06T07:30:08.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.177+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f3806e0c0 con 0x7f9f380c5480 2024-03-06T07:30:08.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.177+0000 7f9f2dffb640 1 -- 172.21.15.17:0/2159067510 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f9f3407b300 con 0x7f9f380c5480 2024-03-06T07:30:08.179 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:08.179 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:08.179 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 msgr2=0x7f9f1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 0x7f9f1c06d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9f380aaca0 tx=0x7f9f28006c90 comp rx=0 tx=0).stop 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 msgr2=0x7f9f381eddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f9f34000c00 tx=0x7f9f340046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 shutdown_connections 2024-03-06T07:30:08.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9f1c06b4d0 0x7f9f1c06d990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f381586f0 0x7f9f381e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f380c5860 0x7f9f380ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 --2- 172.21.15.17:0/2159067510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f380c5480 0x7f9f381eddc0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 >> 172.21.15.17:0/2159067510 conn(0x7f9f3806b1a0 msgr2=0x7f9f380c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 shutdown_connections 2024-03-06T07:30:08.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:08.179+0000 7f9f3f28d640 1 -- 172.21.15.17:0/2159067510 wait complete. 2024-03-06T07:30:08.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:08.594 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:09.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.694+0000 7f6bb6088640 1 Processor -- start 2024-03-06T07:30:09.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.694+0000 7f6bb6088640 1 -- start start 2024-03-06T07:30:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6bb6088640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6bb00c8a00 0x7f6bb0158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6bb6088640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6bb0159460 0x7f6bb00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6bb6088640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bb00c7760 con 0x7f6bb0152920 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6bb6088640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb01587b0 con 0x7f6bb0159460 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6bb6088640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00c7a40 con 0x7f6bb00c8a00 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6bb0159460 0x7f6bb00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6bb0159460 0x7f6bb00c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43880/0 (socket says 172.21.15.17:43880) 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 -- 172.21.15.17:0/3555502124 learned_addr learned my addr 172.21.15.17:0/3555502124 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baeffd640 1 --2- 172.21.15.17:0/3555502124 >> v2:172.21.15.163:3300/0 conn(0x7f6bb00c8a00 0x7f6bb0158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 -- 172.21.15.17:0/3555502124 >> v2:172.21.15.163:3300/0 conn(0x7f6bb00c8a00 msgr2=0x7f6bb0158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 --2- 172.21.15.17:0/3555502124 >> v2:172.21.15.163:3300/0 conn(0x7f6bb00c8a00 0x7f6bb0158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.695+0000 7f6baffff640 1 -- 172.21.15.17:0/3555502124 >> v1:172.21.15.163:6789/0 conn(0x7f6bb0152920 legacy=0x7f6bb0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.696+0000 7f6baffff640 1 -- 172.21.15.17:0/3555502124 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00c7d20 con 0x7f6bb0159460 2024-03-06T07:30:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6baffff640 1 --2- 172.21.15.17:0/3555502124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb0159460 0x7f6bb00c5330 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0067c30 tx=0x7f6ba802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1796ff49167d11f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bae7fc640 1 -- 172.21.15.17:0/3555502124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba802ca80 con 0x7f6bb0159460 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bae7fc640 1 -- 172.21.15.17:0/3555502124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba802cc20 con 0x7f6bb0159460 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bae7fc640 1 -- 172.21.15.17:0/3555502124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80343f0 con 0x7f6bb0159460 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 -- 172.21.15.17:0/3555502124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb0159460 msgr2=0x7f6bb00c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 --2- 172.21.15.17:0/3555502124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb0159460 0x7f6bb00c5330 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0067c30 tx=0x7f6ba802bee0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 -- 172.21.15.17:0/3555502124 shutdown_connections 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 --2- 172.21.15.17:0/3555502124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb0159460 0x7f6bb00c5330 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 --2- 172.21.15.17:0/3555502124 >> v2:172.21.15.163:3300/0 conn(0x7f6bb00c8a00 0x7f6bb0158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 -- 172.21.15.17:0/3555502124 >> 172.21.15.17:0/3555502124 conn(0x7f6bb00ac560 msgr2=0x7f6bb00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 -- 172.21.15.17:0/3555502124 shutdown_connections 2024-03-06T07:30:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.697+0000 7f6bb6088640 1 -- 172.21.15.17:0/3555502124 wait complete. 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.698+0000 7f6bb6088640 1 Processor -- start 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 -- start start 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bb00c8a00 0x7f6bb014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 0x7f6bb014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00c7390 con 0x7f6bb00c33a0 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb006d030 con 0x7f6bb00c8a00 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6bb6088640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00c7510 con 0x7f6bb0152920 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6baffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6baffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43896/0 (socket says 172.21.15.17:43896) 2024-03-06T07:30:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6baffff640 1 -- 172.21.15.17:0/985553995 learned_addr learned my addr 172.21.15.17:0/985553995 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.699+0000 7f6baf7fe640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bb00c8a00 0x7f6bb014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baeffd640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 0x7f6bb014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 -- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 msgr2=0x7f6bb014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 0x7f6bb014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 -- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bb00c8a00 msgr2=0x7f6bb014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bb00c8a00 0x7f6bb014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 -- 172.21.15.17:0/985553995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00c4430 con 0x7f6bb00c33a0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baeffd640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 0x7f6bb014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6baffff640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009f40 tx=0x7f6ba8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8043020 con 0x7f6bb00c33a0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba803e450 con 0x7f6bb00c33a0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb00c48c0 con 0x7f6bb00c33a0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8009510 con 0x7f6bb00c33a0 2024-03-06T07:30:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.700+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb00c2e80 con 0x7f6bb00c33a0 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.701+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb00c3080 con 0x7f6bb00c33a0 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.702+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ba8046020 con 0x7f6bb00c33a0 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.702+0000 7f6ba7fff640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 0x7f6b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.702+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f6ba80ace60 con 0x7f6bb00c33a0 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.702+0000 7f6baf7fe640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 0x7f6b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.703+0000 7f6baf7fe640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 0x7f6b9006d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001780 tx=0x7f6b98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:09.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:09.706+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6bb00c3080 con 0x7f6bb00c33a0 2024-03-06T07:30:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.069+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bb004f330 con 0x7f6bb00c33a0 2024-03-06T07:30:10.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.070+0000 7f6ba7fff640 1 -- 172.21.15.17:0/985553995 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f6ba804d020 con 0x7f6bb00c33a0 2024-03-06T07:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:10.072 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.071+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 msgr2=0x7f6b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.071+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 0x7f6b9006d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001780 tx=0x7f6b98009040 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.071+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 msgr2=0x7f6bb00c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.071+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009f40 tx=0x7f6ba8004a40 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 shutdown_connections 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b9006b4d0 0x7f6b9006d990 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6bb00c33a0 0x7f6bb00c3830 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bb0152920 0x7f6bb014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 --2- 172.21.15.17:0/985553995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bb00c8a00 0x7f6bb014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 >> 172.21.15.17:0/985553995 conn(0x7f6bb00ac560 msgr2=0x7f6bb00ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:10.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 shutdown_connections 2024-03-06T07:30:10.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:10.072+0000 7f6bb6088640 1 -- 172.21.15.17:0/985553995 wait complete. 2024-03-06T07:30:10.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:10.478 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:10.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:10.486 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:11.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.588+0000 7ff23a081640 1 Processor -- start 2024-03-06T07:30:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.588+0000 7ff23a081640 1 -- start start 2024-03-06T07:30:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.588+0000 7ff23a081640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff23414f790 0x7ff2341529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.588+0000 7ff23a081640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff234155a40 0x7ff234155e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff23a081640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2341563d0 con 0x7ff2340c1aa0 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff23a081640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2340c2680 con 0x7ff23414f790 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff23a081640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c28d0 con 0x7ff234155a40 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff23414f790 0x7ff2341529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff23414f790 0x7ff2341529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48664/0 (socket says 172.21.15.17:48664) 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff232ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff2340c1aa0 0x7ff2340c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59576/0 (socket says 172.21.15.17:59576) 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 -- 172.21.15.17:0/2490132912 learned_addr learned my addr 172.21.15.17:0/2490132912 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff233fff640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff234155a40 0x7ff234155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 -- 172.21.15.17:0/2490132912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff234155a40 msgr2=0x7ff234155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff234155a40 0x7ff234155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 -- 172.21.15.17:0/2490132912 >> v1:172.21.15.17:6789/0 conn(0x7ff2340c1aa0 legacy=0x7ff2340c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2337fe640 1 -- 172.21.15.17:0/2490132912 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340c2bb0 con 0x7ff23414f790 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.589+0000 7ff2327fc640 1 -- 172.21.15.17:0/2490132912 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791557493 0 0) 0x7ff2341563d0 con 0x7ff2340c1aa0 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff2337fe640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff23414f790 0x7ff2341529d0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff224009870 tx=0x7ff22402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d68d290240f3c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff2327fc640 1 -- 172.21.15.17:0/2490132912 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224034070 con 0x7ff23414f790 2024-03-06T07:30:11.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff2327fc640 1 -- 172.21.15.17:0/2490132912 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff224030840 con 0x7ff23414f790 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff2327fc640 1 -- 172.21.15.17:0/2490132912 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff224030c10 con 0x7ff23414f790 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 -- 172.21.15.17:0/2490132912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff23414f790 msgr2=0x7ff2341529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff23414f790 0x7ff2341529d0 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7ff224009870 tx=0x7ff22402b980 comp rx=0 tx=0).stop 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 -- 172.21.15.17:0/2490132912 shutdown_connections 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff234155a40 0x7ff234155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 --2- 172.21.15.17:0/2490132912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff23414f790 0x7ff2341529d0 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.590+0000 7ff23a081640 1 -- 172.21.15.17:0/2490132912 >> 172.21.15.17:0/2490132912 conn(0x7ff2340c6710 msgr2=0x7ff2340c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.591+0000 7ff23a081640 1 -- 172.21.15.17:0/2490132912 shutdown_connections 2024-03-06T07:30:11.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.591+0000 7ff23a081640 1 -- 172.21.15.17:0/2490132912 wait complete. 2024-03-06T07:30:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 Processor -- start 2024-03-06T07:30:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 -- start start 2024-03-06T07:30:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 0x7ff2341eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 0x7ff234071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c0d60 con 0x7ff234155a40 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234155520 con 0x7ff23414f790 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.592+0000 7ff23a081640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2340c5e30 con 0x7ff2340c1aa0 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48670/0 (socket says 172.21.15.17:48670) 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 -- 172.21.15.17:0/1555128417 learned_addr learned my addr 172.21.15.17:0/1555128417 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 -- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 msgr2=0x7ff2341eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff2337fe640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 0x7ff2341eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff232ffd640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 0x7ff234071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 0x7ff2341eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 -- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 msgr2=0x7ff234071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 0x7ff234071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 -- 172.21.15.17:0/1555128417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2341eec30 con 0x7ff234155a40 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff2337fe640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 0x7ff2341eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff232ffd640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 0x7ff234071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff233fff640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007e60 tx=0x7ff22c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c014420 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c019070 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff234072f40 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c014820 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.593+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2341eab30 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.594+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23404f330 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.595+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff22c0149c0 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.595+0000 7ff21bfff640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.595+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7ff22c08aca0 con 0x7ff234155a40 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.595+0000 7ff2337fe640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:11.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.596+0000 7ff2337fe640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 0x7ff22006d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff224036000 tx=0x7ff22403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:11.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.604+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff22c0598d0 con 0x7ff234155a40 2024-03-06T07:30:11.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.821+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2340c1f40 con 0x7ff234155a40 2024-03-06T07:30:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.821+0000 7ff21bfff640 1 -- 172.21.15.17:0/1555128417 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7ff22c0591d0 con 0x7ff234155a40 2024-03-06T07:30:11.823 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:11.823 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:11.823 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 msgr2=0x7ff22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 0x7ff22006d990 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff224036000 tx=0x7ff22403b040 comp rx=0 tx=0).stop 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 msgr2=0x7ff2340c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7ff22c007e60 tx=0x7ff22c00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 shutdown_connections 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff22006b4d0 0x7ff22006d990 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff234155a40 0x7ff2340c4610 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff23414f790 0x7ff234071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 --2- 172.21.15.17:0/1555128417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2340c1aa0 0x7ff2341eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 >> 172.21.15.17:0/1555128417 conn(0x7ff2340c6710 msgr2=0x7ff2340c5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 shutdown_connections 2024-03-06T07:30:11.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:11.823+0000 7ff23a081640 1 -- 172.21.15.17:0/1555128417 wait complete. 2024-03-06T07:30:12.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:12.238 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:13.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:13.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.338+0000 7fd578b93640 1 Processor -- start 2024-03-06T07:30:13.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.338+0000 7fd578b93640 1 -- start start 2024-03-06T07:30:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd578b93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5740c88c0 0x7fd574155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd578b93640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd574158b60 0x7fd574158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd578b93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5740c73d0 con 0x7fd57414f360 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd578b93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5740c8dd0 con 0x7fd5740c88c0 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd578b93640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5740be2e0 con 0x7fd574158b60 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd571d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5740c88c0 0x7fd574155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd571d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd5740c88c0 0x7fd574155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48676/0 (socket says 172.21.15.17:48676) 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd571d9b640 1 -- 172.21.15.17:0/4079365764 learned_addr learned my addr 172.21.15.17:0/4079365764 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.339+0000 7fd57259c640 1 --1- 172.21.15.17:0/4079365764 >> v1:172.21.15.17:6789/0 conn(0x7fd57414f360 0x7fd5740c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59584/0 (socket says 172.21.15.17:59584) 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd571d9b640 1 -- 172.21.15.17:0/4079365764 >> v2:172.21.15.163:3300/0 conn(0x7fd574158b60 msgr2=0x7fd574158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd571d9b640 1 --2- 172.21.15.17:0/4079365764 >> v2:172.21.15.163:3300/0 conn(0x7fd574158b60 0x7fd574158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd571d9b640 1 -- 172.21.15.17:0/4079365764 >> v1:172.21.15.17:6789/0 conn(0x7fd57414f360 legacy=0x7fd5740c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd57159a640 1 -- 172.21.15.17:0/4079365764 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041470652 0 0) 0x7fd5740c73d0 con 0x7fd57414f360 2024-03-06T07:30:13.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd571d9b640 1 -- 172.21.15.17:0/4079365764 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740c7670 con 0x7fd5740c88c0 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.340+0000 7fd571d9b640 1 --2- 172.21.15.17:0/4079365764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5740c88c0 0x7fd574155c10 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd574152d60 tx=0x7fd56802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f08e249a45ebfe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd57159a640 1 -- 172.21.15.17:0/4079365764 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568035020 con 0x7fd5740c88c0 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd57159a640 1 -- 172.21.15.17:0/4079365764 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568031870 con 0x7fd5740c88c0 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd57159a640 1 -- 172.21.15.17:0/4079365764 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568031c40 con 0x7fd5740c88c0 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 -- 172.21.15.17:0/4079365764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5740c88c0 msgr2=0x7fd574155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 --2- 172.21.15.17:0/4079365764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5740c88c0 0x7fd574155c10 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fd574152d60 tx=0x7fd56802b910 comp rx=0 tx=0).stop 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 -- 172.21.15.17:0/4079365764 shutdown_connections 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 --2- 172.21.15.17:0/4079365764 >> v2:172.21.15.163:3300/0 conn(0x7fd574158b60 0x7fd574158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 --2- 172.21.15.17:0/4079365764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5740c88c0 0x7fd574155c10 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 -- 172.21.15.17:0/4079365764 >> 172.21.15.17:0/4079365764 conn(0x7fd5740ac570 msgr2=0x7fd5740ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 -- 172.21.15.17:0/4079365764 shutdown_connections 2024-03-06T07:30:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.341+0000 7fd578b93640 1 -- 172.21.15.17:0/4079365764 wait complete. 2024-03-06T07:30:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.342+0000 7fd578b93640 1 Processor -- start 2024-03-06T07:30:13.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 -- start start 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 0x7fd574150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd57414f360 0x7fd5741ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c5a50 con 0x7fd574158b60 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c7c30 con 0x7fd57414f360 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd578b93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740c5bd0 con 0x7fd5740c88c0 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd572d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd572d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48688/0 (socket says 172.21.15.17:48688) 2024-03-06T07:30:13.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd572d9d640 1 -- 172.21.15.17:0/2052537561 learned_addr learned my addr 172.21.15.17:0/2052537561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.343+0000 7fd57259c640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 0x7fd574150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd571d9b640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd57414f360 0x7fd5741ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 -- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 msgr2=0x7fd574150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 0x7fd574150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 -- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd57414f360 msgr2=0x7fd5741ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd57414f360 0x7fd5741ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 -- 172.21.15.17:0/2052537561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5741ec730 con 0x7fd574158b60 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd57259c640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 0x7fd574150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd572d9d640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b9a0 tx=0x7fd56400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564019070 con 0x7fd574158b60 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56400cbc0 con 0x7fd574158b60 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5741eee30 con 0x7fd574158b60 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564014440 con 0x7fd574158b60 2024-03-06T07:30:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.344+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5741504d0 con 0x7fd574158b60 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.345+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57414d1a0 con 0x7fd574158b60 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.346+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56401c070 con 0x7fd574158b60 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.346+0000 7fd562ffd640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.346+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fd56408a0c0 con 0x7fd574158b60 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.346+0000 7fd57259c640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.347+0000 7fd57259c640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.350+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd57414d1a0 con 0x7fd574158b60 2024-03-06T07:30:13.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.573+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd57404f330 con 0x7fd574158b60 2024-03-06T07:30:13.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.573+0000 7fd562ffd640 1 -- 172.21.15.17:0/2052537561 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fd564058d90 con 0x7fd574158b60 2024-03-06T07:30:13.575 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:13.575 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:13.576 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 msgr2=0x7fd55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd56c000f60 tx=0x7fd56c007c10 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 msgr2=0x7fd5741ee560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fd56400b9a0 tx=0x7fd56400be60 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 shutdown_connections 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd55406b4d0 0x7fd55406d990 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd574158b60 0x7fd5741ee560 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd57414f360 0x7fd5741ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 --2- 172.21.15.17:0/2052537561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5740c88c0 0x7fd574150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 >> 172.21.15.17:0/2052537561 conn(0x7fd5740ac570 msgr2=0x7fd5740ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:13.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 shutdown_connections 2024-03-06T07:30:13.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:13.576+0000 7fd578b93640 1 -- 172.21.15.17:0/2052537561 wait complete. 2024-03-06T07:30:13.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:13.991 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:14.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.096+0000 7f0a8f2c9640 1 Processor -- start 2024-03-06T07:30:15.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.096+0000 7f0a8f2c9640 1 -- start start 2024-03-06T07:30:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.097+0000 7f0a8f2c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a8806cae0 0x7f0a8806e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8f2c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a88155740 con 0x7f0a8814e940 2024-03-06T07:30:15.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8f2c9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8806d9d0 con 0x7f0a88154a50 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8f2c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a88153060 con 0x7f0a8806cae0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a8806cae0 0x7f0a8806e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0a8806cae0 0x7f0a8806e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48692/0 (socket says 172.21.15.17:48692) 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 -- 172.21.15.17:0/1269375560 learned_addr learned my addr 172.21.15.17:0/1269375560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 -- 172.21.15.17:0/1269375560 >> v1:172.21.15.163:6789/0 conn(0x7f0a88154a50 legacy=0x7f0a88154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 -- 172.21.15.17:0/1269375560 >> v1:172.21.15.17:6789/0 conn(0x7f0a8814e940 legacy=0x7f0a88152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a7ffff640 1 -- 172.21.15.17:0/1269375560 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690531967 0 0) 0x7f0a88155740 con 0x7f0a8814e940 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.098+0000 7f0a8c864640 1 -- 172.21.15.17:0/1269375560 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8806dcb0 con 0x7f0a8806cae0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8c864640 1 --2- 172.21.15.17:0/1269375560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a8806e790 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800bb00 tx=0x7f0a7802dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b190da0a753216 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a7ffff640 1 -- 172.21.15.17:0/1269375560 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78034070 con 0x7f0a8806cae0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a7ffff640 1 -- 172.21.15.17:0/1269375560 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7802ec50 con 0x7f0a8806cae0 2024-03-06T07:30:15.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a7ffff640 1 -- 172.21.15.17:0/1269375560 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78038400 con 0x7f0a8806cae0 2024-03-06T07:30:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1269375560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 msgr2=0x7f0a8806e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1269375560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a8806e790 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800bb00 tx=0x7f0a7802dea0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1269375560 shutdown_connections 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1269375560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a8806e790 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.099+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1269375560 >> 172.21.15.17:0/1269375560 conn(0x7f0a8806bf00 msgr2=0x7f0a880c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.100+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1269375560 shutdown_connections 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.100+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1269375560 wait complete. 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 Processor -- start 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 -- start start 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a8814e940 0x7f0a880aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a88154a50 0x7f0a880c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880c15a0 con 0x7f0a8806cae0 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8806e270 con 0x7f0a88154a50 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8f2c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8814ed20 con 0x7f0a8814e940 2024-03-06T07:30:15.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8d065640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.101+0000 7f0a8d065640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48700/0 (socket says 172.21.15.17:48700) 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 -- 172.21.15.17:0/1891898592 learned_addr learned my addr 172.21.15.17:0/1891898592 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 -- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a8814e940 msgr2=0x7f0a880aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a8814e940 0x7f0a880aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 -- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a88154a50 msgr2=0x7f0a880c1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a88154a50 0x7f0a880c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 -- 172.21.15.17:0/1891898592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880c4b30 con 0x7f0a8806cae0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8d065640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c4b0 tx=0x7f0a8400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84017070 con 0x7f0a8806cae0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84005480 con 0x7f0a8806cae0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a880c4e10 con 0x7f0a8806cae0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84003810 con 0x7f0a8806cae0 2024-03-06T07:30:15.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.102+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a880c53d0 con 0x7f0a8806cae0 2024-03-06T07:30:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.103+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a88074ac0 con 0x7f0a8806cae0 2024-03-06T07:30:15.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.104+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a8401a070 con 0x7f0a8806cae0 2024-03-06T07:30:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.104+0000 7f0a7dffb640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 0x7f0a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.104+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f0a84089820 con 0x7f0a8806cae0 2024-03-06T07:30:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.104+0000 7f0a8c864640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 0x7f0a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:15.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.105+0000 7f0a8c864640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 0x7f0a6c06d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0a78039000 tx=0x7f0a78030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:15.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.108+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0a88074ac0 con 0x7f0a8806cae0 2024-03-06T07:30:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.327+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a8804f330 con 0x7f0a8806cae0 2024-03-06T07:30:15.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.328+0000 7f0a7dffb640 1 -- 172.21.15.17:0/1891898592 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f0a84058460 con 0x7f0a8806cae0 2024-03-06T07:30:15.329 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:15.330 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:15.330 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 msgr2=0x7f0a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 0x7f0a6c06d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0a78039000 tx=0x7f0a78030000 comp rx=0 tx=0).stop 2024-03-06T07:30:15.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 msgr2=0x7f0a881e6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400c4b0 tx=0x7f0a8400c970 comp rx=0 tx=0).stop 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 shutdown_connections 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0a6c06b4d0 0x7f0a6c06d990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a88154a50 0x7f0a880c1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a8814e940 0x7f0a880aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 --2- 172.21.15.17:0/1891898592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a8806cae0 0x7f0a881e6450 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.329+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 >> 172.21.15.17:0/1891898592 conn(0x7f0a8806bf00 msgr2=0x7f0a881439c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.330+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 shutdown_connections 2024-03-06T07:30:15.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:15.330+0000 7f0a8f2c9640 1 -- 172.21.15.17:0/1891898592 wait complete. 2024-03-06T07:30:15.744 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:15.744 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:16.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:16.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.846+0000 7fc92f1c7640 1 Processor -- start 2024-03-06T07:30:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.846+0000 7fc92f1c7640 1 -- start start 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92f1c7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9280c1aa0 0x7fc9281529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92f1c7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9280c28d0 con 0x7fc92814f790 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92f1c7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9280c2bb0 con 0x7fc928155a40 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92f1c7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9280c2680 con 0x7fc9280c1aa0 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc927fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9280c1aa0 0x7fc9281529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92cf63640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc92814f790 0x7fc9280c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59602/0 (socket says 172.21.15.17:59602) 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc92cf63640 1 -- 172.21.15.17:0/1367619790 learned_addr learned my addr 172.21.15.17:0/1367619790 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc927fff640 1 -- 172.21.15.17:0/1367619790 >> v1:172.21.15.163:6789/0 conn(0x7fc928155a40 legacy=0x7fc928155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc927fff640 1 -- 172.21.15.17:0/1367619790 >> v1:172.21.15.17:6789/0 conn(0x7fc92814f790 legacy=0x7fc9280c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc927fff640 1 -- 172.21.15.17:0/1367619790 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9280c2e90 con 0x7fc9280c1aa0 2024-03-06T07:30:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.847+0000 7fc9277fe640 1 -- 172.21.15.17:0/1367619790 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079461800 0 0) 0x7fc9280c28d0 con 0x7fc92814f790 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc927fff640 1 --2- 172.21.15.17:0/1367619790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9280c1aa0 0x7fc9281529d0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fc9280c2800 tx=0x7fc91402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b230557f3dbbb953 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc9277fe640 1 -- 172.21.15.17:0/1367619790 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914035020 con 0x7fc9280c1aa0 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc9277fe640 1 -- 172.21.15.17:0/1367619790 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914031830 con 0x7fc9280c1aa0 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc9277fe640 1 -- 172.21.15.17:0/1367619790 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914031c00 con 0x7fc9280c1aa0 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc92f1c7640 1 -- 172.21.15.17:0/1367619790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9280c1aa0 msgr2=0x7fc9281529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/1367619790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9280c1aa0 0x7fc9281529d0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7fc9280c2800 tx=0x7fc91402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc92f1c7640 1 -- 172.21.15.17:0/1367619790 shutdown_connections 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/1367619790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9280c1aa0 0x7fc9281529d0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.848+0000 7fc92f1c7640 1 -- 172.21.15.17:0/1367619790 >> 172.21.15.17:0/1367619790 conn(0x7fc9280c6710 msgr2=0x7fc9280c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.849+0000 7fc92f1c7640 1 -- 172.21.15.17:0/1367619790 shutdown_connections 2024-03-06T07:30:16.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.849+0000 7fc92f1c7640 1 -- 172.21.15.17:0/1367619790 wait complete. 2024-03-06T07:30:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 Processor -- start 2024-03-06T07:30:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 -- start start 2024-03-06T07:30:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9280c1aa0 0x7fc9281eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc928155a40 0x7fc9281eb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280c5e30 con 0x7fc92814f790 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280c0d60 con 0x7fc928155a40 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.850+0000 7fc92f1c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9280c5fb0 con 0x7fc9280c1aa0 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48714/0 (socket says 172.21.15.17:48714) 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 -- 172.21.15.17:0/3236688346 learned_addr learned my addr 172.21.15.17:0/3236688346 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 -- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9280c1aa0 msgr2=0x7fc9281eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9280c1aa0 0x7fc9281eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 -- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc928155a40 msgr2=0x7fc9281eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc928155a40 0x7fc9281eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 -- 172.21.15.17:0/3236688346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928071bd0 con 0x7fc92814f790 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc927fff640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fc914009890 tx=0x7fc9140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914004230 con 0x7fc92814f790 2024-03-06T07:30:16.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9140043d0 con 0x7fc92814f790 2024-03-06T07:30:16.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91403cad0 con 0x7fc92814f790 2024-03-06T07:30:16.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc928071eb0 con 0x7fc92814f790 2024-03-06T07:30:16.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.851+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc928153d30 con 0x7fc92814f790 2024-03-06T07:30:16.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.852+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92804f330 con 0x7fc92814f790 2024-03-06T07:30:16.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.853+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc914038020 con 0x7fc92814f790 2024-03-06T07:30:16.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.853+0000 7fc9257fa640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:16.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.853+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fc9140ac680 con 0x7fc92814f790 2024-03-06T07:30:16.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.853+0000 7fc92cf63640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:16.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.854+0000 7fc92cf63640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 0x7fc90c06d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc91c007fd0 tx=0x7fc91c007920 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:16.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:16.858+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc914041070 con 0x7fc92814f790 2024-03-06T07:30:17.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.078+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9280a5f60 con 0x7fc92814f790 2024-03-06T07:30:17.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.079+0000 7fc9257fa640 1 -- 172.21.15.17:0/3236688346 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fc91407b350 con 0x7fc92814f790 2024-03-06T07:30:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:17.081 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:17.081 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 msgr2=0x7fc90c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 0x7fc90c06d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc91c007fd0 tx=0x7fc91c007920 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 msgr2=0x7fc9281dfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fc914009890 tx=0x7fc9140046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 shutdown_connections 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc928155a40 0x7fc9281eb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92814f790 0x7fc9281dfc10 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 --2- 172.21.15.17:0/3236688346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9280c1aa0 0x7fc9281eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 >> 172.21.15.17:0/3236688346 conn(0x7fc9280c6710 msgr2=0x7fc928072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 shutdown_connections 2024-03-06T07:30:17.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:17.081+0000 7fc92f1c7640 1 -- 172.21.15.17:0/3236688346 wait complete. 2024-03-06T07:30:17.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:17.496 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:18.497 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.598+0000 7f93b67a0640 1 Processor -- start 2024-03-06T07:30:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.598+0000 7f93b67a0640 1 -- start start 2024-03-06T07:30:18.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b67a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f93b0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b67a0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b0158970 con 0x7f93b006fec0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b67a0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b0158c50 con 0x7f93b00c8cd0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b67a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0158f30 con 0x7f93b0152800 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b4f9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f93b006fec0 0x7f93b00c85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59604/0 (socket says 172.21.15.17:59604) 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.599+0000 7f93b4f9d640 1 -- 172.21.15.17:0/911656546 learned_addr learned my addr 172.21.15.17:0/911656546 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93b579e640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f93b0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562765917 0 0) 0x7f93b0158970 con 0x7f93b006fec0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939c003520 con 0x7f93b006fec0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150537264 0 0) 0x7f939c003520 con 0x7f93b006fec0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 msgr2=0x7f93b0072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f93b0072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 >> v1:172.21.15.163:6789/0 conn(0x7f93b00c8cd0 legacy=0x7f93b01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0159730 con 0x7f93b006fec0 2024-03-06T07:30:18.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f93a0001670 con 0x7f93b006fec0 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 >> v1:172.21.15.17:6789/0 conn(0x7f93b006fec0 legacy=0x7f93b00c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f939c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f939c004020 0x7f939c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004d40 0x7f939c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0158f30 con 0x7f939c004020 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c00bc00 con 0x7f939c004d40 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c00be00 con 0x7f93b0152800 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f939c004020 0x7f939c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93b579e640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f939c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.600+0000 7f93b5f9f640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004d40 0x7f939c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 -- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 msgr2=0x7f939c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b0152800 0x7f939c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 -- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004d40 msgr2=0x7f939c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004d40 0x7f939c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 -- 172.21.15.17:0/911656546 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c003520 con 0x7f939c004020 2024-03-06T07:30:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f939c004020 0x7f939c004800 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c210 tx=0x7f939c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d426cf8d2f9476b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0007d30 con 0x7f939c004020 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0030070 con 0x7f939c004020 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.601+0000 7f93a7fff640 1 -- 172.21.15.17:0/911656546 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a002cda0 con 0x7f939c004020 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 -- 172.21.15.17:0/911656546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f939c004020 msgr2=0x7f939c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 --2- 172.21.15.17:0/911656546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f939c004020 0x7f939c004800 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c210 tx=0x7f939c005830 comp rx=0 tx=0).stop 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b5f9f640 1 -- 172.21.15.17:0/911656546 reap_dead start 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 -- 172.21.15.17:0/911656546 shutdown_connections 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 -- 172.21.15.17:0/911656546 >> 172.21.15.17:0/911656546 conn(0x7f93b00aa4e0 msgr2=0x7f93b00aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 -- 172.21.15.17:0/911656546 shutdown_connections 2024-03-06T07:30:18.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.602+0000 7f93b67a0640 1 -- 172.21.15.17:0/911656546 wait complete. 2024-03-06T07:30:18.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.603+0000 7f93b67a0640 1 Processor -- start 2024-03-06T07:30:18.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.603+0000 7f93b67a0640 1 -- start start 2024-03-06T07:30:18.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004020 0x7f93b01ee150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 0x7f93b01ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0152be0 con 0x7f93b00bfe30 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b006f660 con 0x7f939c004020 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b67a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0152d60 con 0x7f93b01ee690 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48726/0 (socket says 172.21.15.17:48726) 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 -- 172.21.15.17:0/4090366152 learned_addr learned my addr 172.21.15.17:0/4090366152 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b579e640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004020 0x7f93b01ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 0x7f93b01ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 -- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 msgr2=0x7f93b01ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 0x7f93b01ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 -- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004020 msgr2=0x7f93b01ee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004020 0x7f93b01ee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 -- 172.21.15.17:0/4090366152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b01eeb90 con 0x7f93b00bfe30 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b4f9d640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 0x7f93b01ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.604+0000 7f93b5f9f640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f93b0156440 tx=0x7f93ac00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.605+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00ee90 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.605+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac018070 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.605+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b00ab680 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.605+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac003bb0 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.605+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b00be8e0 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.606+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b0069430 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.606+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93ac0050d0 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.606+0000 7f93a5ffb640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 0x7f939406db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.607+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f93ac089fe0 con 0x7f93b00bfe30 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.607+0000 7f93b579e640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 0x7f939406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.607+0000 7f93b579e640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 0x7f939406db60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f93b00ab4e0 tx=0x7f93a8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:18.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.611+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f93ac058cb0 con 0x7f93b00bfe30 2024-03-06T07:30:18.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.830+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93b006f220 con 0x7f93b00bfe30 2024-03-06T07:30:18.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.830+0000 7f93a5ffb640 1 -- 172.21.15.17:0/4090366152 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f93ac0585b0 con 0x7f93b00bfe30 2024-03-06T07:30:18.832 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:18.833 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:18.833 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 msgr2=0x7f939406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 0x7f939406db60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f93b00ab4e0 tx=0x7f93a8009190 comp rx=0 tx=0).stop 2024-03-06T07:30:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 msgr2=0x7f93b01ec620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f93b0156440 tx=0x7f93ac00eb50 comp rx=0 tx=0).stop 2024-03-06T07:30:18.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 shutdown_connections 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f939406b6a0 0x7f939406db60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93b00bfe30 0x7f93b01ec620 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f93b01ee690 0x7f93b01ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 --2- 172.21.15.17:0/4090366152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f939c004020 0x7f93b01ee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 >> 172.21.15.17:0/4090366152 conn(0x7f93b00aa4e0 msgr2=0x7f93b0072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 shutdown_connections 2024-03-06T07:30:18.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:18.832+0000 7f93b67a0640 1 -- 172.21.15.17:0/4090366152 wait complete. 2024-03-06T07:30:19.248 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:19.248 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:20.249 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:20.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.349+0000 7f9ff326b640 1 Processor -- start 2024-03-06T07:30:20.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff326b640 1 -- start start 2024-03-06T07:30:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff326b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fec155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff326b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0c73d0 con 0x7f9fec0c88c0 2024-03-06T07:30:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff326b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec0be2e0 con 0x7f9fec14f360 2024-03-06T07:30:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff326b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec159700 con 0x7f9fec159270 2024-03-06T07:30:20.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff0806640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9fec0c88c0 0x7f9fec158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:59608/0 (socket says 172.21.15.17:59608) 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.350+0000 7f9ff0806640 1 -- 172.21.15.17:0/1025151024 learned_addr learned my addr 172.21.15.17:0/1025151024 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fec155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128752732 0 0) 0x7f9fec0c73d0 con 0x7f9fec0c88c0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd8003520 con 0x7f9fec0c88c0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784958110 0 0) 0x7f9fd8003520 con 0x7f9fec0c88c0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 msgr2=0x7f9fec155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fec155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 >> v1:172.21.15.163:6789/0 conn(0x7f9fec14f360 legacy=0x7f9fec0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0c7670 con 0x7f9fec0c88c0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9fdc001670 con 0x7f9fec0c88c0 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 >> v1:172.21.15.17:6789/0 conn(0x7f9fec0c88c0 legacy=0x7f9fec158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fd80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8003f20 0x7f9fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec159700 con 0x7f9fd8004d40 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800bad0 con 0x7f9fd8003f20 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd800bcd0 con 0x7f9fec159270 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff0806640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8003f20 0x7f9fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1007640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fd80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 -- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 msgr2=0x7f9fd80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fd80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 -- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8003f20 msgr2=0x7f9fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd8003f20 0x7f9fd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1808640 1 -- 172.21.15.17:0/1025151024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd8003520 con 0x7f9fd8004d40 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.351+0000 7f9ff1007640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fec159270 0x7f9fd80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff1808640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fc0 tx=0x7f9fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30bd9f7f1fa0099 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:20.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8039070 con 0x7f9fd8004d40 2024-03-06T07:30:20.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe802cc20 con 0x7f9fd8004d40 2024-03-06T07:30:20.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9fe3fff640 1 -- 172.21.15.17:0/1025151024 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8009c10 con 0x7f9fd8004d40 2024-03-06T07:30:20.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 -- 172.21.15.17:0/1025151024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd8004d40 msgr2=0x7f9fd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 --2- 172.21.15.17:0/1025151024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009fc0 tx=0x7f9fd8005830 comp rx=0 tx=0).stop 2024-03-06T07:30:20.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff1808640 1 -- 172.21.15.17:0/1025151024 reap_dead start 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 -- 172.21.15.17:0/1025151024 shutdown_connections 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 -- 172.21.15.17:0/1025151024 >> 172.21.15.17:0/1025151024 conn(0x7f9fec0ac570 msgr2=0x7f9fec0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 -- 172.21.15.17:0/1025151024 shutdown_connections 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.353+0000 7f9ff326b640 1 -- 172.21.15.17:0/1025151024 wait complete. 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.354+0000 7f9ff326b640 1 Processor -- start 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.354+0000 7f9ff326b640 1 -- start start 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8004d40 0x7f9fec0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fec1ec980 0x7f9fec1edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0c7c30 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0c5a50 con 0x7f9fec1ec980 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff326b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0c5bd0 con 0x7f9fd8004d40 2024-03-06T07:30:20.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff0806640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff0806640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48730/0 (socket says 172.21.15.17:48730) 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff0806640 1 -- 172.21.15.17:0/3218757995 learned_addr learned my addr 172.21.15.17:0/3218757995 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff1808640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fec1ec980 0x7f9fec1edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff1007640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8004d40 0x7f9fec0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.355+0000 7f9ff0806640 1 -- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8004d40 msgr2=0x7f9fec0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff0806640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8004d40 0x7f9fec0afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff0806640 1 -- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fec1ec980 msgr2=0x7f9fec1edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff0806640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fec1ec980 0x7f9fec1edfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff0806640 1 -- 172.21.15.17:0/3218757995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0ae210 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff0806640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc007310 tx=0x7f9fdc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00f940 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc018070 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec0ae4b0 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00fd60 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.356+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec0aea70 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.357+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec04f330 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.358+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fdc01d460 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.358+0000 7f9fe1ffb640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 0x7f9fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.358+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f9fdc08a460 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.358+0000 7f9ff1007640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 0x7f9fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.359+0000 7f9ff1007640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 0x7f9fd006d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9fec152ae0 tx=0x7f9fe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:20.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.362+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9fdc0591f0 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.599+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fec0c8d50 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.600+0000 7f9fe1ffb640 1 -- 172.21.15.17:0/3218757995 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f9fdc058af0 con 0x7f9fec1ebfc0 2024-03-06T07:30:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:20.602 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:20.602 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 msgr2=0x7f9fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 0x7f9fd006d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9fec152ae0 tx=0x7f9fe4009040 comp rx=0 tx=0).stop 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 msgr2=0x7f9fec1ec440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc007310 tx=0x7f9fdc0077d0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 shutdown_connections 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9fd006b4d0 0x7f9fd006d990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fec1ec980 0x7f9fec1edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9fec1ebfc0 0x7f9fec1ec440 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 --2- 172.21.15.17:0/3218757995 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8004d40 0x7f9fec0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 >> 172.21.15.17:0/3218757995 conn(0x7f9fec0ac570 msgr2=0x7f9fec14e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 shutdown_connections 2024-03-06T07:30:20.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:20.602+0000 7f9ff326b640 1 -- 172.21.15.17:0/3218757995 wait complete. 2024-03-06T07:30:21.017 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:21.017 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:22.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:22.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.118+0000 7facf0fa0640 1 Processor -- start 2024-03-06T07:30:22.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.118+0000 7facf0fa0640 1 -- start start 2024-03-06T07:30:22.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7facf0fa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facec0c8cd0 0x7facec072e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7facf0fa0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 0x7facec1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7facf0fa0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec158970 con 0x7facec152800 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7facf0fa0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec158c50 con 0x7facec0c8cd0 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7facf0fa0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec158f30 con 0x7facec06fec0 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7faceaffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7facec0c8cd0 0x7facec072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7faceb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7facec152800 0x7facec0c85c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47760/0 (socket says 172.21.15.17:47760) 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.119+0000 7faceb7fe640 1 -- 172.21.15.17:0/3092901689 learned_addr learned my addr 172.21.15.17:0/3092901689 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7facebfff640 1 --2- 172.21.15.17:0/3092901689 >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 0x7facec1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7faceaffd640 1 -- 172.21.15.17:0/3092901689 >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 msgr2=0x7facec1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7faceaffd640 1 --2- 172.21.15.17:0/3092901689 >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 0x7facec1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7faceaffd640 1 -- 172.21.15.17:0/3092901689 >> v1:172.21.15.17:6789/0 conn(0x7facec152800 legacy=0x7facec0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7faceaffd640 1 -- 172.21.15.17:0/3092901689 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec159730 con 0x7facec0c8cd0 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7facea7fc640 1 -- 172.21.15.17:0/3092901689 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267077191 0 0) 0x7facec158970 con 0x7facec152800 2024-03-06T07:30:22.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7facebfff640 1 --2- 172.21.15.17:0/3092901689 >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 0x7facec1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.120+0000 7faceaffd640 1 --2- 172.21.15.17:0/3092901689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec072e00 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7facec0c3d40 tx=0x7face002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab5f881d6c5bf39 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facea7fc640 1 -- 172.21.15.17:0/3092901689 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00315a0 con 0x7facec0c8cd0 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facea7fc640 1 -- 172.21.15.17:0/3092901689 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0037070 con 0x7facec0c8cd0 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facea7fc640 1 -- 172.21.15.17:0/3092901689 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0031960 con 0x7facec0c8cd0 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 -- 172.21.15.17:0/3092901689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 msgr2=0x7facec072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3092901689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec072e00 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7facec0c3d40 tx=0x7face002b840 comp rx=0 tx=0).stop 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 -- 172.21.15.17:0/3092901689 shutdown_connections 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3092901689 >> v2:172.21.15.163:3300/0 conn(0x7facec06fec0 0x7facec1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3092901689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec072e00 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 -- 172.21.15.17:0/3092901689 >> 172.21.15.17:0/3092901689 conn(0x7facec0aa4e0 msgr2=0x7facec0aa8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:22.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 -- 172.21.15.17:0/3092901689 shutdown_connections 2024-03-06T07:30:22.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.121+0000 7facf0fa0640 1 -- 172.21.15.17:0/3092901689 wait complete. 2024-03-06T07:30:22.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.122+0000 7facf0fa0640 1 Processor -- start 2024-03-06T07:30:22.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.122+0000 7facf0fa0640 1 -- start start 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 0x7facec1ee230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 0x7facec154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec152be0 con 0x7facec0c8cd0 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec06f660 con 0x7facec152800 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facf0fa0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec152d60 con 0x7facec06fec0 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7faceaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7faceaffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41926/0 (socket says 172.21.15.17:41926) 2024-03-06T07:30:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7faceaffd640 1 -- 172.21.15.17:0/3476407987 learned_addr learned my addr 172.21.15.17:0/3476407987 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7faceb7fe640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 0x7facec1ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.123+0000 7facebfff640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 0x7facec154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 -- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 msgr2=0x7facec1ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 0x7facec1ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 -- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 msgr2=0x7facec154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 0x7facec154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 -- 172.21.15.17:0/3476407987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec1eed40 con 0x7facec0c8cd0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceb7fe640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 0x7facec1ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facebfff640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 0x7facec154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7faceaffd640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7face0009890 tx=0x7face00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0032840 con 0x7facec0c8cd0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face00329e0 con 0x7facec0c8cd0 2024-03-06T07:30:22.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec0bfdf0 con 0x7facec0c8cd0 2024-03-06T07:30:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0032dd0 con 0x7facec0c8cd0 2024-03-06T07:30:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.124+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec06d9c0 con 0x7facec0c8cd0 2024-03-06T07:30:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.125+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0bfbe0 con 0x7facec0c8cd0 2024-03-06T07:30:22.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.126+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7face004c020 con 0x7facec0c8cd0 2024-03-06T07:30:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.126+0000 7facd3fff640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.127+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7face002f020 con 0x7facec0c8cd0 2024-03-06T07:30:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.127+0000 7faceb7fe640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:22.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.127+0000 7faceb7fe640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 0x7facd806d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facdc0012e0 tx=0x7facdc005bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:22.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.131+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7face0040070 con 0x7facec0c8cd0 2024-03-06T07:30:22.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.354+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facec069430 con 0x7facec0c8cd0 2024-03-06T07:30:22.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.354+0000 7facd3fff640 1 -- 172.21.15.17:0/3476407987 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7face007adf0 con 0x7facec0c8cd0 2024-03-06T07:30:22.356 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:22.356 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:22.357 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:22.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.357+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 msgr2=0x7facd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.357+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 0x7facd806d990 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facdc0012e0 tx=0x7facdc005bb0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.357+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 msgr2=0x7facec1548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.357+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7face0009890 tx=0x7face00046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 shutdown_connections 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7facec152800 0x7facec154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7facec0c8cd0 0x7facec1548f0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 --2- 172.21.15.17:0/3476407987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7facec06fec0 0x7facec1ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 >> 172.21.15.17:0/3476407987 conn(0x7facec0aa4e0 msgr2=0x7facec0729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 shutdown_connections 2024-03-06T07:30:22.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:22.358+0000 7facf0fa0640 1 -- 172.21.15.17:0/3476407987 wait complete. 2024-03-06T07:30:22.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:22.776 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:23.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:23.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.876+0000 7f9c4a69a640 1 Processor -- start 2024-03-06T07:30:23.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.876+0000 7f9c4a69a640 1 -- start start 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c4a69a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9c4406afc0 0x7f9c4406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c4a69a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c440a70f0 con 0x7f9c441536c0 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c4a69a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c441567f0 con 0x7f9c4406b3a0 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c4a69a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4406b830 con 0x7f9c4406afc0 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9c4406afc0 0x7f9c4406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9c4406afc0 0x7f9c4406f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41932/0 (socket says 172.21.15.17:41932) 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c48c37640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9c441536c0 0x7f9c4415d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47762/0 (socket says 172.21.15.17:47762) 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 -- 172.21.15.17:0/3189579504 learned_addr learned my addr 172.21.15.17:0/3189579504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:23.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 -- 172.21.15.17:0/3189579504 >> v1:172.21.15.163:6789/0 conn(0x7f9c4406b3a0 legacy=0x7f9c44152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 -- 172.21.15.17:0/3189579504 >> v1:172.21.15.17:6789/0 conn(0x7f9c441536c0 legacy=0x7f9c4415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c43fff640 1 -- 172.21.15.17:0/3189579504 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44156ad0 con 0x7f9c4406afc0 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.877+0000 7f9c42ffd640 1 -- 172.21.15.17:0/3189579504 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593869825 0 0) 0x7f9c440a70f0 con 0x7f9c441536c0 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.878+0000 7f9c43fff640 1 --2- 172.21.15.17:0/3189579504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c4406f100 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0098a0 tx=0x7f9c2c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803aa49b93317686 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.878+0000 7f9c42ffd640 1 -- 172.21.15.17:0/3189579504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0317c0 con 0x7f9c4406afc0 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.878+0000 7f9c42ffd640 1 -- 172.21.15.17:0/3189579504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c037070 con 0x7f9c4406afc0 2024-03-06T07:30:23.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.878+0000 7f9c42ffd640 1 -- 172.21.15.17:0/3189579504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c031b80 con 0x7f9c4406afc0 2024-03-06T07:30:23.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3189579504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 msgr2=0x7f9c4406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3189579504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c4406f100 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0098a0 tx=0x7f9c2c02b860 comp rx=0 tx=0).stop 2024-03-06T07:30:23.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3189579504 shutdown_connections 2024-03-06T07:30:23.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3189579504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c4406f100 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3189579504 >> 172.21.15.17:0/3189579504 conn(0x7f9c440c4290 msgr2=0x7f9c440c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3189579504 shutdown_connections 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.879+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3189579504 wait complete. 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.880+0000 7f9c4a69a640 1 Processor -- start 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.880+0000 7f9c4a69a640 1 -- start start 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4406b3a0 0x7f9c441ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c441536c0 0x7f9c440c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c441565a0 con 0x7f9c4406afc0 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44150a20 con 0x7f9c4406b3a0 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c4a69a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440c7360 con 0x7f9c441536c0 2024-03-06T07:30:23.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41938/0 (socket says 172.21.15.17:41938) 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 -- 172.21.15.17:0/3788532636 learned_addr learned my addr 172.21.15.17:0/3788532636 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c48c37640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c441536c0 0x7f9c440c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c437fe640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4406b3a0 0x7f9c441ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 -- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c441536c0 msgr2=0x7f9c440c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c441536c0 0x7f9c440c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 -- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4406b3a0 msgr2=0x7f9c441ebaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4406b3a0 0x7f9c441ebaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 -- 172.21.15.17:0/3788532636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44152d40 con 0x7f9c4406afc0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.881+0000 7f9c43fff640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.882+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c032860 con 0x7f9c4406afc0 2024-03-06T07:30:23.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.882+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c032a20 con 0x7f9c4406afc0 2024-03-06T07:30:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.882+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c032e10 con 0x7f9c4406afc0 2024-03-06T07:30:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.882+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c44153020 con 0x7f9c4406afc0 2024-03-06T07:30:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.882+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c440c3410 con 0x7f9c4406afc0 2024-03-06T07:30:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.883+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c44067ad0 con 0x7f9c4406afc0 2024-03-06T07:30:23.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.883+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c2c04c060 con 0x7f9c4406afc0 2024-03-06T07:30:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.883+0000 7f9c40ff9640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 0x7f9c2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.884+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f9c2c0abc40 con 0x7f9c4406afc0 2024-03-06T07:30:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.884+0000 7f9c437fe640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 0x7f9c2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.884+0000 7f9c437fe640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 0x7f9c2406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007fd0 tx=0x7f9c34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:23.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:23.888+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c44067ad0 con 0x7f9c4406afc0 2024-03-06T07:30:24.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.112+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c4404f330 con 0x7f9c4406afc0 2024-03-06T07:30:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.112+0000 7f9c40ff9640 1 -- 172.21.15.17:0/3788532636 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f9c2c040070 con 0x7f9c4406afc0 2024-03-06T07:30:24.114 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:24.114 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 msgr2=0x7f9c2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 0x7f9c2406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f9c34007fd0 tx=0x7f9c34009040 comp rx=0 tx=0).stop 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 msgr2=0x7f9c440aa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 shutdown_connections 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c2406b500 0x7f9c2406d9c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c441536c0 0x7f9c440c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4406b3a0 0x7f9c441ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:24.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 --2- 172.21.15.17:0/3788532636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c4406afc0 0x7f9c440aa630 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 >> 172.21.15.17:0/3788532636 conn(0x7f9c440c4290 msgr2=0x7f9c440a8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 shutdown_connections 2024-03-06T07:30:24.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:24.114+0000 7f9c4a69a640 1 -- 172.21.15.17:0/3788532636 wait complete. 2024-03-06T07:30:24.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:24.532 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:25.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:25.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.672+0000 7fee5daeb640 1 Processor -- start 2024-03-06T07:30:25.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.672+0000 7fee5daeb640 1 -- start start 2024-03-06T07:30:25.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee5daeb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee5daeb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee581572f0 con 0x7fee58153990 2024-03-06T07:30:25.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee5daeb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee580ad310 con 0x7fee581506b0 2024-03-06T07:30:25.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee5daeb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee58156450 con 0x7fee58156ea0 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee56ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fee58153990 0x7fee580acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47768/0 (socket says 172.21.15.17:47768) 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee56ffd640 1 -- 172.21.15.17:0/1545123709 learned_addr learned my addr 172.21.15.17:0/1545123709 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee57fff640 1 --2- 172.21.15.17:0/1545123709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083036963 0 0) 0x7fee581572f0 con 0x7fee58153990 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.673+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee44003520 con 0x7fee58153990 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203639682 0 0) 0x7fee580ad310 con 0x7fee581506b0 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee581572f0 con 0x7fee581506b0 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3864470691 0 0) 0x7fee44003520 con 0x7fee58153990 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee57fff640 1 -- 172.21.15.17:0/1545123709 >> v1:172.21.15.163:6789/0 conn(0x7fee581506b0 legacy=0x7fee580702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee57fff640 1 -- 172.21.15.17:0/1545123709 >> v1:172.21.15.17:6789/0 conn(0x7fee58153990 legacy=0x7fee580acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.674+0000 7fee57fff640 1 -- 172.21.15.17:0/1545123709 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58070aa0 con 0x7fee58156ea0 2024-03-06T07:30:25.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee57fff640 1 --2- 172.21.15.17:0/1545123709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c54a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee50009dc0 tx=0x7fee5002bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f714c8640e90e87c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50039070 con 0x7fee58156ea0 2024-03-06T07:30:25.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5002c910 con 0x7fee58156ea0 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee567fc640 1 -- 172.21.15.17:0/1545123709 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40001690 con 0x7fee58156ea0 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee5daeb640 1 -- 172.21.15.17:0/1545123709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 msgr2=0x7fee580c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee5daeb640 1 --2- 172.21.15.17:0/1545123709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c54a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fee50009dc0 tx=0x7fee5002bde0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee5daeb640 1 -- 172.21.15.17:0/1545123709 shutdown_connections 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee5daeb640 1 --2- 172.21.15.17:0/1545123709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c54a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.675+0000 7fee5daeb640 1 -- 172.21.15.17:0/1545123709 >> 172.21.15.17:0/1545123709 conn(0x7fee5806b1a0 msgr2=0x7fee5806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.676+0000 7fee5daeb640 1 -- 172.21.15.17:0/1545123709 shutdown_connections 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.676+0000 7fee5daeb640 1 -- 172.21.15.17:0/1545123709 wait complete. 2024-03-06T07:30:25.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.677+0000 7fee5daeb640 1 Processor -- start 2024-03-06T07:30:25.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.677+0000 7fee5daeb640 1 -- start start 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 0x7fee580c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee581566a0 con 0x7fee581506b0 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58156a10 con 0x7fee58153990 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.678+0000 7fee5daeb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee580c59e0 con 0x7fee58156ea0 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41942/0 (socket says 172.21.15.17:41942) 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 -- 172.21.15.17:0/2988556904 learned_addr learned my addr 172.21.15.17:0/2988556904 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee56ffd640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 0x7fee580c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee57fff640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 -- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 msgr2=0x7fee580c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 -- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 msgr2=0x7fee580c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 0x7fee580c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 -- 172.21.15.17:0/2988556904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580c46d0 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee56ffd640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 0x7fee580c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee577fe640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c490 tx=0x7fee4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee480052d0 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48005470 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48012850 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581473e0 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.679+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee58155dd0 con 0x7fee581506b0 2024-03-06T07:30:25.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.680+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5804f330 con 0x7fee581506b0 2024-03-06T07:30:25.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.681+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee480129f0 con 0x7fee581506b0 2024-03-06T07:30:25.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.681+0000 7fee4ffff640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:25.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.681+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fee4808a090 con 0x7fee581506b0 2024-03-06T07:30:25.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.681+0000 7fee56ffd640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:25.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.682+0000 7fee56ffd640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 0x7fee3806d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fee580ad820 tx=0x7fee400015f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:25.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.685+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fee48058d60 con 0x7fee581506b0 2024-03-06T07:30:25.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.902+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee580700a0 con 0x7fee581506b0 2024-03-06T07:30:25.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.902+0000 7fee4ffff640 1 -- 172.21.15.17:0/2988556904 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fee48058660 con 0x7fee581506b0 2024-03-06T07:30:25.904 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:25.904 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:25.905 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 msgr2=0x7fee3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 0x7fee3806d990 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fee580ad820 tx=0x7fee400015f0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 msgr2=0x7fee581465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7fee4800c490 tx=0x7fee4800c950 comp rx=0 tx=0).stop 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 shutdown_connections 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fee3806b4d0 0x7fee3806d990 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee58156ea0 0x7fee580c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee58153990 0x7fee580c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 --2- 172.21.15.17:0/2988556904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee581506b0 0x7fee581465a0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 >> 172.21.15.17:0/2988556904 conn(0x7fee5806b1a0 msgr2=0x7fee580c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 shutdown_connections 2024-03-06T07:30:25.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:25.905+0000 7fee5daeb640 1 -- 172.21.15.17:0/2988556904 wait complete. 2024-03-06T07:30:26.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:26.322 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:27.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:27.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.424+0000 7f1c7b8a4640 1 Processor -- start 2024-03-06T07:30:27.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.424+0000 7f1c7b8a4640 1 -- start start 2024-03-06T07:30:27.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c7b8a4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c7414f360 0x7f1c74155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c7b8a4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c74158b60 0x7f1c74158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c7b8a4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740c73d0 con 0x7f1c740c88c0 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c7b8a4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740c8dd0 con 0x7f1c7414f360 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c7b8a4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740be2e0 con 0x7f1c74158b60 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c79640640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c7414f360 0x7f1c74155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c79640640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1c7414f360 0x7f1c74155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41954/0 (socket says 172.21.15.17:41954) 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.425+0000 7f1c79640640 1 -- 172.21.15.17:0/3379400292 learned_addr learned my addr 172.21.15.17:0/3379400292 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c79640640 1 -- 172.21.15.17:0/3379400292 >> v2:172.21.15.163:3300/0 conn(0x7f1c74158b60 msgr2=0x7f1c74158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c79640640 1 --2- 172.21.15.17:0/3379400292 >> v2:172.21.15.163:3300/0 conn(0x7f1c74158b60 0x7f1c74158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c79640640 1 -- 172.21.15.17:0/3379400292 >> v1:172.21.15.163:6789/0 conn(0x7f1c740c88c0 legacy=0x7f1c740c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c79640640 1 -- 172.21.15.17:0/3379400292 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740c7670 con 0x7f1c7414f360 2024-03-06T07:30:27.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c79640640 1 --2- 172.21.15.17:0/3379400292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c7414f360 0x7f1c74155c10 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f1c74152d60 tx=0x7f1c7002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ea0ea8032c070c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.426+0000 7f1c6bfff640 1 -- 172.21.15.17:0/3379400292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7002e070 con 0x7f1c7414f360 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c6bfff640 1 -- 172.21.15.17:0/3379400292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70037070 con 0x7f1c7414f360 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c6bfff640 1 -- 172.21.15.17:0/3379400292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70031a00 con 0x7f1c7414f360 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3379400292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c7414f360 msgr2=0x7f1c74155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3379400292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c7414f360 0x7f1c74155c10 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f1c74152d60 tx=0x7f1c7002b880 comp rx=0 tx=0).stop 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3379400292 shutdown_connections 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3379400292 >> v2:172.21.15.163:3300/0 conn(0x7f1c74158b60 0x7f1c74158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3379400292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c7414f360 0x7f1c74155c10 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3379400292 >> 172.21.15.17:0/3379400292 conn(0x7f1c740ac570 msgr2=0x7f1c740ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:27.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3379400292 shutdown_connections 2024-03-06T07:30:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.427+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3379400292 wait complete. 2024-03-06T07:30:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.428+0000 7f1c7b8a4640 1 Processor -- start 2024-03-06T07:30:27.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.428+0000 7f1c7b8a4640 1 -- start start 2024-03-06T07:30:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740c88c0 0x7f1c740ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7414f360 0x7f1c741eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c5a50 con 0x7f1c74158b60 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c7c30 con 0x7f1c740c88c0 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c7b8a4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c5bd0 con 0x7f1c7414f360 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41960/0 (socket says 172.21.15.17:41960) 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 -- 172.21.15.17:0/3044716903 learned_addr learned my addr 172.21.15.17:0/3044716903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79640640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740c88c0 0x7f1c740ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c78e3f640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7414f360 0x7f1c741eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 -- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7414f360 msgr2=0x7f1c741eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7414f360 0x7f1c741eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 -- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740c88c0 msgr2=0x7f1c740ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740c88c0 0x7f1c740ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.429+0000 7f1c79e41640 1 -- 172.21.15.17:0/3044716903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c741ef260 con 0x7f1c74158b60 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c79e41640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b970 tx=0x7f1c6c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:27.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c019070 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00cb60 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c014440 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c740c0da0 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.430+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740abd10 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.431+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7404f330 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.431+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c6c01c070 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.432+0000 7f1c69ffb640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 0x7f1c5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.432+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f1c6c089fe0 con 0x7f1c74158b60 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.432+0000 7f1c79640640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 0x7f1c5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:27.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.432+0000 7f1c79640640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 0x7f1c5806d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1c70036000 tx=0x7f1c7003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:27.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.436+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c6c058660 con 0x7f1c74158b60 2024-03-06T07:30:27.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.723+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c7406cda0 con 0x7f1c74158b60 2024-03-06T07:30:27.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.723+0000 7f1c69ffb640 1 -- 172.21.15.17:0/3044716903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f1c6c05c700 con 0x7f1c74158b60 2024-03-06T07:30:27.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:27.725 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:27.726 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:27.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 msgr2=0x7f1c5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 0x7f1c5806d990 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1c70036000 tx=0x7f1c7003d040 comp rx=0 tx=0).stop 2024-03-06T07:30:27.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 msgr2=0x7f1c741eb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:27.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00b970 tx=0x7f1c6c00be30 comp rx=0 tx=0).stop 2024-03-06T07:30:27.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 shutdown_connections 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1c5806b4d0 0x7f1c5806d990 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c74158b60 0x7f1c741eb920 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7414f360 0x7f1c741eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 --2- 172.21.15.17:0/3044716903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740c88c0 0x7f1c740ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 >> 172.21.15.17:0/3044716903 conn(0x7f1c740ac570 msgr2=0x7f1c740ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 shutdown_connections 2024-03-06T07:30:27.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:27.725+0000 7f1c7b8a4640 1 -- 172.21.15.17:0/3044716903 wait complete. 2024-03-06T07:30:28.140 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:28.140 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:29.141 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:29.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.241+0000 7f2741e6f640 1 Processor -- start 2024-03-06T07:30:29.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.241+0000 7f2741e6f640 1 -- start start 2024-03-06T07:30:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f2741e6f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c1527b0 0x7f273c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f2741e6f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 0x7f273c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f2741e6f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0c7c60 con 0x7f273c155b20 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f2741e6f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c159640 con 0x7f273c1527b0 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f2741e6f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0c8df0 con 0x7f273c0c7180 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c1527b0 0x7f273c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f273c1527b0 0x7f273c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41970/0 (socket says 172.21.15.17:41970) 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 -- 172.21.15.17:0/2899164052 learned_addr learned my addr 172.21.15.17:0/2899164052 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273affd640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 0x7f273c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 -- 172.21.15.17:0/2899164052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 msgr2=0x7f273c0c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:29.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 0x7f273c0c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 -- 172.21.15.17:0/2899164052 >> v1:172.21.15.17:6789/0 conn(0x7f273c155b20 legacy=0x7f273c0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273b7fe640 1 -- 172.21.15.17:0/2899164052 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0c8060 con 0x7f273c1527b0 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.242+0000 7f273affd640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 0x7f273c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f273b7fe640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1527b0 0x7f273c159100 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228fc17e8d9250d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f273a7fc640 1 -- 172.21.15.17:0/2899164052 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c035020 con 0x7f273c1527b0 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f273a7fc640 1 -- 172.21.15.17:0/2899164052 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c031640 con 0x7f273c1527b0 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f273a7fc640 1 -- 172.21.15.17:0/2899164052 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c02d040 con 0x7f273c1527b0 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 -- 172.21.15.17:0/2899164052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1527b0 msgr2=0x7f273c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1527b0 0x7f273c159100 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f272c009870 tx=0x7f272c02b8f0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 -- 172.21.15.17:0/2899164052 shutdown_connections 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c0c7180 0x7f273c0c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2899164052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c1527b0 0x7f273c159100 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.243+0000 7f2741e6f640 1 -- 172.21.15.17:0/2899164052 >> 172.21.15.17:0/2899164052 conn(0x7f273c0ac560 msgr2=0x7f273c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:29.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.244+0000 7f2741e6f640 1 -- 172.21.15.17:0/2899164052 shutdown_connections 2024-03-06T07:30:29.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.244+0000 7f2741e6f640 1 -- 172.21.15.17:0/2899164052 wait complete. 2024-03-06T07:30:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 Processor -- start 2024-03-06T07:30:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 -- start start 2024-03-06T07:30:29.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 0x7f273c1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 0x7f273c1eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c155320 con 0x7f273c0c7180 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0c7980 con 0x7f273c1527b0 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.245+0000 7f2741e6f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c155600 con 0x7f273c155b20 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41982/0 (socket says 172.21.15.17:41982) 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 -- 172.21.15.17:0/2258433063 learned_addr learned my addr 172.21.15.17:0/2258433063 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273bfff640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 0x7f273c1eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273affd640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 0x7f273c1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 -- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 msgr2=0x7f273c1eb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 0x7f273c1eb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 -- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 msgr2=0x7f273c1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 0x7f273c1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 -- 172.21.15.17:0/2258433063 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c154bc0 con 0x7f273c0c7180 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273affd640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 0x7f273c1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273bfff640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 0x7f273c1eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f273b7fe640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c03aa60 con 0x7f273c0c7180 2024-03-06T07:30:29.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c03ac00 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c154ea0 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c0443f0 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.246+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c1585f0 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.247+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c1ea990 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.248+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f272c038020 con 0x7f273c0c7180 2024-03-06T07:30:29.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.248+0000 7f2723fff640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 0x7f272806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:29.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.248+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f272c0ac0e0 con 0x7f273c0c7180 2024-03-06T07:30:29.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.248+0000 7f273affd640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 0x7f272806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:29.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.249+0000 7f273affd640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 0x7f272806d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2730006fd0 tx=0x7f2730008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:29.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.253+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f272c03f070 con 0x7f273c0c7180 2024-03-06T07:30:29.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.472+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f273c04f330 con 0x7f273c0c7180 2024-03-06T07:30:29.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.472+0000 7f2723fff640 1 -- 172.21.15.17:0/2258433063 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7f272c07aed0 con 0x7f273c0c7180 2024-03-06T07:30:29.474 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:29.474 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:29.474 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:29.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 msgr2=0x7f272806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 0x7f272806d990 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f2730006fd0 tx=0x7f2730008040 comp rx=0 tx=0).stop 2024-03-06T07:30:29.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 msgr2=0x7f273c1ee990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:29.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f272c000c00 tx=0x7f272c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 shutdown_connections 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f272806b4d0 0x7f272806d990 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f273c155b20 0x7f273c1eb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f273c1527b0 0x7f273c1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 --2- 172.21.15.17:0/2258433063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f273c0c7180 0x7f273c1ee990 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 >> 172.21.15.17:0/2258433063 conn(0x7f273c0ac560 msgr2=0x7f273c0ab080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 shutdown_connections 2024-03-06T07:30:29.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:29.474+0000 7f2741e6f640 1 -- 172.21.15.17:0/2258433063 wait complete. 2024-03-06T07:30:29.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:29.889 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:30.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:30.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.990+0000 7fa14560f640 1 Processor -- start 2024-03-06T07:30:30.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.990+0000 7fa14560f640 1 -- start start 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa14560f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1401506b0 0x7fa140153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa14560f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 0x7fa1400c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa14560f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa140070aa0 con 0x7fa140156ea0 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa14560f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa140156450 con 0x7fa1401506b0 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa14560f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa140156a10 con 0x7fa1400acc00 2024-03-06T07:30:30.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa13effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1401506b0 0x7fa140153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa13effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa1401506b0 0x7fa140153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36700/0 (socket says 172.21.15.17:36700) 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa13effd640 1 -- 172.21.15.17:0/2454907456 learned_addr learned my addr 172.21.15.17:0/2454907456 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.991+0000 7fa13f7fe640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 0x7fa1400c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13effd640 1 -- 172.21.15.17:0/2454907456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 msgr2=0x7fa1400c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13effd640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 0x7fa1400c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13effd640 1 -- 172.21.15.17:0/2454907456 >> v1:172.21.15.163:6789/0 conn(0x7fa140156ea0 legacy=0x7fa1400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13effd640 1 -- 172.21.15.17:0/2454907456 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140070d80 con 0x7fa1401506b0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13f7fe640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 0x7fa1400c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.992+0000 7fa13effd640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1401506b0 0x7fa140153990 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa130009870 tx=0x7fa13002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c420822f540a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa13dffb640 1 -- 172.21.15.17:0/2454907456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130034070 con 0x7fa1401506b0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa13dffb640 1 -- 172.21.15.17:0/2454907456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa130030840 con 0x7fa1401506b0 2024-03-06T07:30:30.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa13dffb640 1 -- 172.21.15.17:0/2454907456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130030c10 con 0x7fa1401506b0 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 -- 172.21.15.17:0/2454907456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1401506b0 msgr2=0x7fa140153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1401506b0 0x7fa140153990 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fa130009870 tx=0x7fa13002b980 comp rx=0 tx=0).stop 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 -- 172.21.15.17:0/2454907456 shutdown_connections 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1400acc00 0x7fa1400c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 --2- 172.21.15.17:0/2454907456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1401506b0 0x7fa140153990 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 -- 172.21.15.17:0/2454907456 >> 172.21.15.17:0/2454907456 conn(0x7fa14006b1a0 msgr2=0x7fa14006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 -- 172.21.15.17:0/2454907456 shutdown_connections 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.993+0000 7fa14560f640 1 -- 172.21.15.17:0/2454907456 wait complete. 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.994+0000 7fa14560f640 1 Processor -- start 2024-03-06T07:30:30.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.994+0000 7fa14560f640 1 -- start start 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400acc00 0x7fa1400aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1401506b0 0x7fa1400a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400ad1d0 con 0x7fa140156ea0 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400ad050 con 0x7fa1400acc00 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa14560f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400ad350 con 0x7fa1401506b0 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36714/0 (socket says 172.21.15.17:36714) 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13f7fe640 1 -- 172.21.15.17:0/3733741164 learned_addr learned my addr 172.21.15.17:0/3733741164 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13effd640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400acc00 0x7fa1400aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:31.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13e7fc640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1401506b0 0x7fa1400a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.995+0000 7fa13f7fe640 1 -- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1401506b0 msgr2=0x7fa1400a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa13f7fe640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1401506b0 0x7fa1400a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa13f7fe640 1 -- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400acc00 msgr2=0x7fa1400aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa13f7fe640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400acc00 0x7fa1400aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa13f7fe640 1 -- 172.21.15.17:0/3733741164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1401ea0e0 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa13f7fe640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fa138007de0 tx=0x7fa13800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13801a070 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa138003990 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1400a89b0 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138003d80 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.996+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1400a83f0 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.997+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1400c62f0 con 0x7fa140156ea0 2024-03-06T07:30:31.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.997+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa13801d070 con 0x7fa140156ea0 2024-03-06T07:30:31.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.998+0000 7fa1377fe640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:31.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.998+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6274+0+0 (secure 0 0 0) 0x7fa13808a690 con 0x7fa140156ea0 2024-03-06T07:30:31.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.998+0000 7fa13effd640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:31.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:30.999+0000 7fa13effd640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 0x7fa12006d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa130036000 tx=0x7fa130008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:31.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.002+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa1400c62f0 con 0x7fa140156ea0 2024-03-06T07:30:31.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.223+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa14004f330 con 0x7fa140156ea0 2024-03-06T07:30:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.223+0000 7fa1377fe640 1 -- 172.21.15.17:0/3733741164 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) v1 ==== 95+0+4222 (secure 0 0 0) 0x7fa138010070 con 0x7fa140156ea0 2024-03-06T07:30:31.225 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:31.225 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":211,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8162,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":196}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:00.887057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":215,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":9547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9547":{"gid":9547,"name":"b","rank":1,"incarnation":206,"state":"up:stopping","state_seq":18,"addr":"172.21.15.163:6835/3635018115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3635018115},{"type":"v1","addr":"172.21.15.163:6835","nonce":3635018115}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:31.225 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 211 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 msgr2=0x7fa12006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 0x7fa12006d990 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa130036000 tx=0x7fa130008030 comp rx=0 tx=0).stop 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 msgr2=0x7fa1400a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 secure :-1 s=READY pgs=2460 cs=0 l=1 rev1=1 crypto rx=0x7fa138007de0 tx=0x7fa13800d4a0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 shutdown_connections 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa12006b4d0 0x7fa12006d990 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa140156ea0 0x7fa1400a93a0 unknown :-1 s=CLOSED pgs=2460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1401506b0 0x7fa1400a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 --2- 172.21.15.17:0/3733741164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1400acc00 0x7fa1400aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 >> 172.21.15.17:0/3733741164 conn(0x7fa14006b1a0 msgr2=0x7fa14014d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 shutdown_connections 2024-03-06T07:30:31.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:31.225+0000 7fa14560f640 1 -- 172.21.15.17:0/3733741164 wait complete. 2024-03-06T07:30:31.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 211, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:00.887057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 215, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 9547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9547': {'gid': 9547, 'name': 'b', 'rank': 1, 'incarnation': 206, 'state': 'up:stopping', 'state_seq': 18, 'addr': '172.21.15.163:6835/3635018115', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 3635018115}, {'type': 'v1', 'addr': '172.21.15.163:6835', 'nonce': 3635018115}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:31.640 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9547:up:stopping 2024-03-06T07:30:32.641 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:32.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.741+0000 7f4a8a80e640 1 Processor -- start 2024-03-06T07:30:32.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.741+0000 7f4a8a80e640 1 -- start start 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 0x7f4a84159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 0x7f4a840c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a84155b20 0x7f4a840c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a84159640 con 0x7f4a84155b20 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840c8df0 con 0x7f4a841527b0 2024-03-06T07:30:32.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a8a80e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c7c60 con 0x7f4a840c7180 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a84155b20 0x7f4a840c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4a84155b20 0x7f4a840c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36730/0 (socket says 172.21.15.17:36730) 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 -- 172.21.15.17:0/2738012199 learned_addr learned my addr 172.21.15.17:0/2738012199 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a837fe640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 0x7f4a840c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a83fff640 1 --2- 172.21.15.17:0/2738012199 >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 0x7f4a84159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 -- 172.21.15.17:0/2738012199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 msgr2=0x7f4a840c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 0x7f4a840c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 -- 172.21.15.17:0/2738012199 >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 msgr2=0x7f4a84159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 --2- 172.21.15.17:0/2738012199 >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 0x7f4a84159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a88dab640 1 -- 172.21.15.17:0/2738012199 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840c8060 con 0x7f4a84155b20 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a837fe640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 0x7f4a840c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:32.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.742+0000 7f4a83fff640 1 --2- 172.21.15.17:0/2738012199 >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 0x7f4a84159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:32.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.743+0000 7f4a88dab640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a84155b20 0x7f4a840c5330 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4a840cba80 tx=0x7f4a7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22e022dec6da3ca server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.743+0000 7f4a82ffd640 1 -- 172.21.15.17:0/2738012199 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c02c9e0 con 0x7f4a84155b20 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.743+0000 7f4a82ffd640 1 -- 172.21.15.17:0/2738012199 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7c02cb80 con 0x7f4a84155b20 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.743+0000 7f4a82ffd640 1 -- 172.21.15.17:0/2738012199 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c0343f0 con 0x7f4a84155b20 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2738012199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a84155b20 msgr2=0x7f4a840c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a84155b20 0x7f4a840c5330 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f4a840cba80 tx=0x7f4a7c02be40 comp rx=0 tx=0).stop 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2738012199 shutdown_connections 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a84155b20 0x7f4a840c5330 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2738012199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a840c7180 0x7f4a840c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2738012199 >> v2:172.21.15.163:3300/0 conn(0x7f4a841527b0 0x7f4a84159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2738012199 >> 172.21.15.17:0/2738012199 conn(0x7f4a840ac560 msgr2=0x7f4a840ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2738012199 shutdown_connections 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.744+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2738012199 wait complete. 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.745+0000 7f4a8a80e640 1 Processor -- start 2024-03-06T07:30:32.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.745+0000 7f4a8a80e640 1 -- start start 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a841527b0 0x7f4a840aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a84155b20 0x7f4a840afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84155320 con 0x7f4a840c7180 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c7980 con 0x7f4a841527b0 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84155600 con 0x7f4a84155b20 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36732/0 (socket says 172.21.15.17:36732) 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 -- 172.21.15.17:0/2669908901 learned_addr learned my addr 172.21.15.17:0/2669908901 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a88dab640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a84155b20 0x7f4a840afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a837fe640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a841527b0 0x7f4a840aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 -- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a84155b20 msgr2=0x7f4a840afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a84155b20 0x7f4a840afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 -- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a841527b0 msgr2=0x7f4a840aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a841527b0 0x7f4a840aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 -- 172.21.15.17:0/2669908901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840adae0 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a83fff640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c7f0 tx=0x7f4a7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74015070 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74007a30 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.746+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840addc0 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.747+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74007e00 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.747+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a840ae380 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.748+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8404f330 con 0x7f4a840c7180 2024-03-06T07:30:32.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.748+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a74018070 con 0x7f4a840c7180 2024-03-06T07:30:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.748+0000 7f4a80ff9640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 0x7f4a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.749+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4a7408a3e0 con 0x7f4a840c7180 2024-03-06T07:30:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.749+0000 7f4a837fe640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 0x7f4a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:32.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.749+0000 7f4a837fe640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 0x7f4a7006d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4a8414cb00 tx=0x7f4a78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:32.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:32.753+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a74058fd0 con 0x7f4a840c7180 2024-03-06T07:30:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.016+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a8406ce00 con 0x7f4a840c7180 2024-03-06T07:30:33.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.016+0000 7f4a80ff9640 1 -- 172.21.15.17:0/2669908901 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 213 v213) v1 ==== 95+0+3427 (secure 0 0 0) 0x7f4a740588d0 con 0x7f4a840c7180 2024-03-06T07:30:33.018 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:33.019 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":213,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":213,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:31.895900+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:resolve","state_seq":23,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:33.019 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 213 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 msgr2=0x7f4a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 0x7f4a7006d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4a8414cb00 tx=0x7f4a78008040 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 msgr2=0x7f4a841ea640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400c7f0 tx=0x7f4a7400cbf0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 shutdown_connections 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4a7006b4d0 0x7f4a7006d990 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a84155b20 0x7f4a840afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a841527b0 0x7f4a840aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 --2- 172.21.15.17:0/2669908901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a840c7180 0x7f4a841ea640 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 >> 172.21.15.17:0/2669908901 conn(0x7f4a840ac560 msgr2=0x7f4a840cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 shutdown_connections 2024-03-06T07:30:33.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:33.019+0000 7f4a8a80e640 1 -- 172.21.15.17:0/2669908901 wait complete. 2024-03-06T07:30:33.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 213, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:31.895900+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:resolve', 'state_seq': 23, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:33.437 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:resolve 2024-03-06T07:30:34.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 Processor -- start 2024-03-06T07:30:34.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 -- start start 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faff00c88c0 0x7faff0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff00c73d0 con 0x7faff014f360 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff00be2e0 con 0x7faff0158b60 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.538+0000 7faff6e64640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff0159700 con 0x7faff00c88c0 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7fafeffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faff00c88c0 0x7faff0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7faff4c00640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7faff014f360 0x7faff00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33464/0 (socket says 172.21.15.17:33464) 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7faff4c00640 1 -- 172.21.15.17:0/778787089 learned_addr learned my addr 172.21.15.17:0/778787089 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7fafeffff640 1 -- 172.21.15.17:0/778787089 >> v1:172.21.15.163:6789/0 conn(0x7faff0158b60 legacy=0x7faff0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7fafeffff640 1 -- 172.21.15.17:0/778787089 >> v1:172.21.15.17:6789/0 conn(0x7faff014f360 legacy=0x7faff00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.539+0000 7fafeffff640 1 -- 172.21.15.17:0/778787089 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00c7670 con 0x7faff00c88c0 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7fafeffff640 1 --2- 172.21.15.17:0/778787089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff00c88c0 0x7faff0155c10 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fafe40097a0 tx=0x7fafe402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc68c35f526f976 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7fafef7fe640 1 -- 172.21.15.17:0/778787089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40315a0 con 0x7faff00c88c0 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7fafef7fe640 1 -- 172.21.15.17:0/778787089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4037070 con 0x7faff00c88c0 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7fafef7fe640 1 -- 172.21.15.17:0/778787089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4031960 con 0x7faff00c88c0 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 -- 172.21.15.17:0/778787089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff00c88c0 msgr2=0x7faff0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 --2- 172.21.15.17:0/778787089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff00c88c0 0x7faff0155c10 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fafe40097a0 tx=0x7fafe402b840 comp rx=0 tx=0).stop 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 -- 172.21.15.17:0/778787089 shutdown_connections 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 --2- 172.21.15.17:0/778787089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff00c88c0 0x7faff0155c10 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 -- 172.21.15.17:0/778787089 >> 172.21.15.17:0/778787089 conn(0x7faff00ac570 msgr2=0x7faff00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:34.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 -- 172.21.15.17:0/778787089 shutdown_connections 2024-03-06T07:30:34.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.540+0000 7faff6e64640 1 -- 172.21.15.17:0/778787089 wait complete. 2024-03-06T07:30:34.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.541+0000 7faff6e64640 1 Processor -- start 2024-03-06T07:30:34.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 -- start start 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff00c88c0 0x7faff00be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff014f360 0x7faff01ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff00c7c30 con 0x7faff0158b60 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff00c5a50 con 0x7faff014f360 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff6e64640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff00c5bd0 con 0x7faff00c88c0 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.542+0000 7faff5401640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36756/0 (socket says 172.21.15.17:36756) 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 -- 172.21.15.17:0/3327679882 learned_addr learned my addr 172.21.15.17:0/3327679882 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff4c00640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff00c88c0 0x7faff00be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7fafeffff640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff014f360 0x7faff01ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:34.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 -- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff00c88c0 msgr2=0x7faff00be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff00c88c0 0x7faff00be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 -- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff014f360 msgr2=0x7faff01ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff014f360 0x7faff01ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 -- 172.21.15.17:0/3327679882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00c11c0 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff5401640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fafe800b970 tx=0x7fafe800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8019070 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe800cae0 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8014440 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff00bf0a0 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.543+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff00c0de0 con 0x7faff0158b60 2024-03-06T07:30:34.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.544+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff004f330 con 0x7faff0158b60 2024-03-06T07:30:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.545+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafe801c070 con 0x7faff0158b60 2024-03-06T07:30:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.545+0000 7fafed7fa640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.545+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fafe8089ff0 con 0x7faff0158b60 2024-03-06T07:30:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.545+0000 7faff4c00640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:34.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.546+0000 7faff4c00640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 0x7fafdc06d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000f60 tx=0x7fafe0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:34.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.550+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fafe8058c70 con 0x7faff0158b60 2024-03-06T07:30:34.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.767+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faff00c8d50 con 0x7faff0158b60 2024-03-06T07:30:34.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.767+0000 7fafed7fa640 1 -- 172.21.15.17:0/3327679882 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+3429 (secure 0 0 0) 0x7fafe8058570 con 0x7faff0158b60 2024-03-06T07:30:34.770 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:34.770 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":214,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:32.888410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 214 2024-03-06T07:30:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 msgr2=0x7fafdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 0x7fafdc06d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000f60 tx=0x7fafe0009040 comp rx=0 tx=0).stop 2024-03-06T07:30:34.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 msgr2=0x7faff01eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fafe800b970 tx=0x7fafe800be30 comp rx=0 tx=0).stop 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 shutdown_connections 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faff0158b60 0x7faff01eea10 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff014f360 0x7faff01ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 --2- 172.21.15.17:0/3327679882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff00c88c0 0x7faff00be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 >> 172.21.15.17:0/3327679882 conn(0x7faff00ac570 msgr2=0x7faff014d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 shutdown_connections 2024-03-06T07:30:34.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:34.769+0000 7faff6e64640 1 -- 172.21.15.17:0/3327679882 wait complete. 2024-03-06T07:30:35.184 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 214, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:32.888410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:reconnect', 'state_seq': 24, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:35.184 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:reconnect 2024-03-06T07:30:36.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.292+0000 7f5051f13640 1 Processor -- start 2024-03-06T07:30:36.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.292+0000 7f5051f13640 1 -- start start 2024-03-06T07:30:36.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5051f13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f504c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5051f13640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0c5cc0 con 0x7f504c0c5860 2024-03-06T07:30:36.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5051f13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c1591c0 con 0x7f504c1586f0 2024-03-06T07:30:36.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5051f13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c06df40 con 0x7f504c0c5480 2024-03-06T07:30:36.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5043fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f504c0c5860 0x7f504c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33470/0 (socket says 172.21.15.17:33470) 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f5043fff640 1 -- 172.21.15.17:0/1568972906 learned_addr learned my addr 172.21.15.17:0/1568972906 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f504b7fe640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f504c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539717004 0 0) 0x7f504c0c5cc0 con 0x7f504c0c5860 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.293+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5038003520 con 0x7f504c0c5860 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1293845202 0 0) 0x7f5038003520 con 0x7f504c0c5860 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 msgr2=0x7f504c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f504c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 >> v1:172.21.15.163:6789/0 conn(0x7f504c1586f0 legacy=0x7f504c15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c1594a0 con 0x7f504c0c5860 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f5034001670 con 0x7f504c0c5860 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 >> v1:172.21.15.17:6789/0 conn(0x7f504c0c5860 legacy=0x7f504c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50380039e0 0x7f5038003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f5038004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5038004d40 0x7f50380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c06df40 con 0x7f5038004d40 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503800baf0 con 0x7f50380039e0 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503800bcf0 con 0x7f504c0c5480 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5038004d40 0x7f50380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f5043fff640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f5038004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504b7fe640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50380039e0 0x7f5038003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 -- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 msgr2=0x7f5038004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f504c0c5480 0x7f5038004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 -- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50380039e0 msgr2=0x7f5038003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50380039e0 0x7f5038003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.294+0000 7f504bfff640 1 -- 172.21.15.17:0/1568972906 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5038003520 con 0x7f5038004d40 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.295+0000 7f504bfff640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5038004d40 0x7f50380051c0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5044001e50 tx=0x7f5038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cd189ef6e3d88b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044001a70 con 0x7f5038004d40 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044003b70 con 0x7f5038004d40 2024-03-06T07:30:36.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f504affd640 1 -- 172.21.15.17:0/1568972906 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50440050d0 con 0x7f5038004d40 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 -- 172.21.15.17:0/1568972906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5038004d40 msgr2=0x7f50380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 --2- 172.21.15.17:0/1568972906 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5038004d40 0x7f50380051c0 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f5044001e50 tx=0x7f5038005830 comp rx=0 tx=0).stop 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f504bfff640 1 -- 172.21.15.17:0/1568972906 reap_dead start 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 -- 172.21.15.17:0/1568972906 shutdown_connections 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 -- 172.21.15.17:0/1568972906 >> 172.21.15.17:0/1568972906 conn(0x7f504c06b1a0 msgr2=0x7f504c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 -- 172.21.15.17:0/1568972906 shutdown_connections 2024-03-06T07:30:36.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.296+0000 7f5051f13640 1 -- 172.21.15.17:0/1568972906 wait complete. 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.297+0000 7f5051f13640 1 Processor -- start 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.297+0000 7f5051f13640 1 -- start start 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 0x7f504c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 0x7f504c1ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c155680 con 0x7f504c14da70 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c06fdb0 con 0x7f504c0c45b0 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5051f13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c152a20 con 0x7f5038004d40 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5043fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5043fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36776/0 (socket says 172.21.15.17:36776) 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5043fff640 1 -- 172.21.15.17:0/4051424652 learned_addr learned my addr 172.21.15.17:0/4051424652 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f504bfff640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 0x7f504c1ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f504b7fe640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 0x7f504c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5043fff640 1 -- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 msgr2=0x7f504c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.298+0000 7f5043fff640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 0x7f504c14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5043fff640 1 -- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 msgr2=0x7f504c1ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5043fff640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 0x7f504c1ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5043fff640 1 -- 172.21.15.17:0/4051424652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0c4a30 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f504bfff640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 0x7f504c1ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f504b7fe640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 0x7f504c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5043fff640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f50340072f0 tx=0x7f50340077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400f960 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034018070 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c0ac060 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400fd60 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.299+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c14ded0 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.300+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c0c3980 con 0x7f504c14da70 2024-03-06T07:30:36.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.300+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f503401d460 con 0x7f504c14da70 2024-03-06T07:30:36.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.301+0000 7f5048ff9640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 0x7f502c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:36.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.301+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f503408a490 con 0x7f504c14da70 2024-03-06T07:30:36.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.301+0000 7f504b7fe640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 0x7f502c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:36.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.301+0000 7f504b7fe640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 0x7f502c06d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f503c009ab0 tx=0x7f503c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:36.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.305+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5034059110 con 0x7f504c14da70 2024-03-06T07:30:36.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.522+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f504c04f330 con 0x7f504c14da70 2024-03-06T07:30:36.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.523+0000 7f5048ff9640 1 -- 172.21.15.17:0/4051424652 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f5034058a10 con 0x7f504c14da70 2024-03-06T07:30:36.524 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:36.525 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:36.525 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:36.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 msgr2=0x7f502c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 0x7f502c06d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f503c009ab0 tx=0x7f503c008040 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 msgr2=0x7f504c0c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f50340072f0 tx=0x7f50340077b0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 shutdown_connections 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f502c06b500 0x7f502c06d9c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f504c0c45b0 0x7f504c1ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c14da70 0x7f504c0c4070 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 --2- 172.21.15.17:0/4051424652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5038004d40 0x7f504c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 >> 172.21.15.17:0/4051424652 conn(0x7f504c06b1a0 msgr2=0x7f504c06cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 shutdown_connections 2024-03-06T07:30:36.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:36.525+0000 7f5051f13640 1 -- 172.21.15.17:0/4051424652 wait complete. 2024-03-06T07:30:36.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:36.943 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:37.944 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:38.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 Processor -- start 2024-03-06T07:30:38.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 -- start start 2024-03-06T07:30:38.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3a40c88c0 0x7ff3a4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a4158b60 0x7ff3a4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a40c73d0 con 0x7ff3a414f360 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a40c8dd0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.045+0000 7ff3a971e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40be2e0 con 0x7ff3a4158b60 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3a40c88c0 0x7ff3a4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff3a40c88c0 0x7ff3a4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36782/0 (socket says 172.21.15.17:36782) 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 -- 172.21.15.17:0/3315270284 learned_addr learned my addr 172.21.15.17:0/3315270284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff3a37fe640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a4158b60 0x7ff3a4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 -- 172.21.15.17:0/3315270284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a4158b60 msgr2=0x7ff3a4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a4158b60 0x7ff3a4158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 -- 172.21.15.17:0/3315270284 >> v1:172.21.15.163:6789/0 conn(0x7ff3a414f360 legacy=0x7ff3a40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:38.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.046+0000 7ff39bfff640 1 -- 172.21.15.17:0/3315270284 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40c7670 con 0x7ff3a40c88c0 2024-03-06T07:30:38.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.047+0000 7ff39bfff640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a4155c10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d805fc0a81b7a89c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:38.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.047+0000 7ff3a27fc640 1 -- 172.21.15.17:0/3315270284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0317c0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.047+0000 7ff3a27fc640 1 -- 172.21.15.17:0/3315270284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c037070 con 0x7ff3a40c88c0 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.047+0000 7ff3a27fc640 1 -- 172.21.15.17:0/3315270284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c031b80 con 0x7ff3a40c88c0 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 -- 172.21.15.17:0/3315270284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 msgr2=0x7ff3a4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a4155c10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02b860 comp rx=0 tx=0).stop 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 -- 172.21.15.17:0/3315270284 shutdown_connections 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a4158b60 0x7ff3a4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 --2- 172.21.15.17:0/3315270284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a4155c10 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 -- 172.21.15.17:0/3315270284 >> 172.21.15.17:0/3315270284 conn(0x7ff3a40ac570 msgr2=0x7ff3a40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 -- 172.21.15.17:0/3315270284 shutdown_connections 2024-03-06T07:30:38.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.048+0000 7ff3a971e640 1 -- 172.21.15.17:0/3315270284 wait complete. 2024-03-06T07:30:38.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.049+0000 7ff3a971e640 1 Processor -- start 2024-03-06T07:30:38.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.049+0000 7ff3a971e640 1 -- start start 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a414f360 0x7ff3a41e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 0x7ff3a4156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c5a50 con 0x7ff3a40c88c0 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c7c30 con 0x7ff3a4158b60 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a40c5bd0 con 0x7ff3a414f360 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36794/0 (socket says 172.21.15.17:36794) 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 -- 172.21.15.17:0/1656481874 learned_addr learned my addr 172.21.15.17:0/1656481874 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff39bfff640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a414f360 0x7ff3a41e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 -- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a414f360 msgr2=0x7ff3a41e20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a37fe640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 0x7ff3a4156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a414f360 0x7ff3a41e20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 -- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 msgr2=0x7ff3a4156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 0x7ff3a4156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 -- 172.21.15.17:0/1656481874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a41e13f0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a37fe640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 0x7ff3a4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a2ffd640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b420 tx=0x7ff39400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394011830 con 0x7ff3a40c88c0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff394016070 con 0x7ff3a40c88c0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394011c10 con 0x7ff3a40c88c0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a41ee0d0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.050+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a41ee690 con 0x7ff3a40c88c0 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.051+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a404f330 con 0x7ff3a40c88c0 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.052+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff39401b460 con 0x7ff3a40c88c0 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.052+0000 7ff39b7fe640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 0x7ff38406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.052+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7ff394089fb0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.052+0000 7ff39bfff640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 0x7ff38406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:38.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.053+0000 7ff39bfff640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 0x7ff38406d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff38c036000 tx=0x7ff38c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:38.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.057+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff394020020 con 0x7ff3a40c88c0 2024-03-06T07:30:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.277+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3a40c81c0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.277+0000 7ff39b7fe640 1 -- 172.21.15.17:0/1656481874 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7ff394058ba0 con 0x7ff3a40c88c0 2024-03-06T07:30:38.279 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:38.279 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:38.279 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 msgr2=0x7ff38406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 0x7ff38406d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff38c036000 tx=0x7ff38c03d040 comp rx=0 tx=0).stop 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 msgr2=0x7ff3a41e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b420 tx=0x7ff39400b8e0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 shutdown_connections 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff38406b4d0 0x7ff38406d990 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3a4158b60 0x7ff3a4156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3a414f360 0x7ff3a41e20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 --2- 172.21.15.17:0/1656481874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3a40c88c0 0x7ff3a41e0970 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 >> 172.21.15.17:0/1656481874 conn(0x7ff3a40ac570 msgr2=0x7ff3a40ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 shutdown_connections 2024-03-06T07:30:38.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:38.279+0000 7ff3a971e640 1 -- 172.21.15.17:0/1656481874 wait complete. 2024-03-06T07:30:38.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:38.694 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:39.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.796+0000 7f7d3c393640 1 Processor -- start 2024-03-06T07:30:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.796+0000 7f7d3c393640 1 -- start start 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.796+0000 7f7d3c393640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7d3414f360 0x7f7d34155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3c393640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340c73d0 con 0x7f7d340c88c0 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3c393640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340be2e0 con 0x7f7d34158b60 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3c393640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d34159700 con 0x7f7d3414f360 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3992e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7d340c88c0 0x7f7d340c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:33476/0 (socket says 172.21.15.17:33476) 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3992e640 1 -- 172.21.15.17:0/3271397921 learned_addr learned my addr 172.21.15.17:0/3271397921 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922229817 0 0) 0x7f7d340c73d0 con 0x7f7d340c88c0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d20003520 con 0x7f7d340c88c0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/3271397921 >> v2:172.21.15.163:3300/0 conn(0x7f7d3414f360 0x7f7d34155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095789478 0 0) 0x7f7d20003520 con 0x7f7d340c88c0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 >> v2:172.21.15.163:3300/0 conn(0x7f7d3414f360 msgr2=0x7f7d34155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 --2- 172.21.15.17:0/3271397921 >> v2:172.21.15.163:3300/0 conn(0x7f7d3414f360 0x7f7d34155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 >> v1:172.21.15.163:6789/0 conn(0x7f7d34158b60 legacy=0x7f7d34158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340c7670 con 0x7f7d340c88c0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f7d24001670 con 0x7f7d340c88c0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 >> v1:172.21.15.17:6789/0 conn(0x7f7d340c88c0 legacy=0x7f7d340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 0x7f7d20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d20004800 0x7f7d20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200051c0 0x7f7d2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34159700 con 0x7f7d200039b0 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2000c160 con 0x7f7d20004800 2024-03-06T07:30:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2000c360 con 0x7f7d200051c0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.797+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 0x7f7d20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3992e640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d20004800 0x7f7d20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a930640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200051c0 0x7f7d2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 -- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200051c0 msgr2=0x7f7d2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200051c0 0x7f7d2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 -- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d20004800 msgr2=0x7f7d20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d20004800 0x7f7d20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 -- 172.21.15.17:0/3271397921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d20003520 con 0x7f7d200039b0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a930640 1 -- 172.21.15.17:0/3271397921 reap_dead start 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3992e640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d20004800 0x7f7d20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.798+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 0x7f7d20003dd0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fc0 tx=0x7f7d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1f1406123ef328 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c039070 con 0x7f7d200039b0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c02cc20 con 0x7f7d200039b0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3912d640 1 -- 172.21.15.17:0/3271397921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c009c10 con 0x7f7d200039b0 2024-03-06T07:30:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 -- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 msgr2=0x7f7d20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 0x7f7d20003dd0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fc0 tx=0x7f7d20005830 comp rx=0 tx=0).stop 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 -- 172.21.15.17:0/3271397921 shutdown_connections 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 --2- 172.21.15.17:0/3271397921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d200039b0 0x7f7d20003dd0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 -- 172.21.15.17:0/3271397921 >> 172.21.15.17:0/3271397921 conn(0x7f7d340ac570 msgr2=0x7f7d340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 -- 172.21.15.17:0/3271397921 shutdown_connections 2024-03-06T07:30:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.799+0000 7f7d3c393640 1 -- 172.21.15.17:0/3271397921 wait complete. 2024-03-06T07:30:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.800+0000 7f7d3c393640 1 Processor -- start 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.800+0000 7f7d3c393640 1 -- start start 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200039b0 0x7f7d34152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 0x7f7d3414bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340c7c30 con 0x7f7d34150e90 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34155c10 con 0x7f7d3414bb70 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3c393640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d34155d90 con 0x7f7d200039b0 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36820/0 (socket says 172.21.15.17:36820) 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 -- 172.21.15.17:0/172924573 learned_addr learned my addr 172.21.15.17:0/172924573 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3a930640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 0x7f7d3414bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200039b0 0x7f7d34152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 -- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200039b0 msgr2=0x7f7d34152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200039b0 0x7f7d34152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 -- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 msgr2=0x7f7d3414bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 0x7f7d3414bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.801+0000 7f7d3992e640 1 -- 172.21.15.17:0/172924573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d341547e0 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d3a930640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 0x7f7d3414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d3992e640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e810 tx=0x7f7d2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400c870 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24018070 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d340b0030 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400cc50 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.802+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d34154500 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.803+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3404f330 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.803+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d2401d460 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.804+0000 7f7d2affd640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 0x7f7d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.804+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7d2408a2a0 con 0x7f7d34150e90 2024-03-06T07:30:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.804+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 0x7f7d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.804+0000 7f7d3a12f640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 0x7f7d1806d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c036000 tx=0x7f7d2c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:39.812+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d24058e00 con 0x7f7d34150e90 2024-03-06T07:30:40.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.031+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d340c8d50 con 0x7f7d34150e90 2024-03-06T07:30:40.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.031+0000 7f7d2affd640 1 -- 172.21.15.17:0/172924573 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f7d24058700 con 0x7f7d34150e90 2024-03-06T07:30:40.033 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:40.033 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:40.033 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 msgr2=0x7f7d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 0x7f7d1806d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c036000 tx=0x7f7d2c02e000 comp rx=0 tx=0).stop 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 msgr2=0x7f7d341512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400e810 tx=0x7f7d2400ecd0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 shutdown_connections 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7d1806b4d0 0x7f7d1806d990 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7d3414bb70 0x7f7d3414bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7d34150e90 0x7f7d341512f0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 --2- 172.21.15.17:0/172924573 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7d200039b0 0x7f7d34152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 >> 172.21.15.17:0/172924573 conn(0x7f7d340ac570 msgr2=0x7f7d3414d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 shutdown_connections 2024-03-06T07:30:40.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:40.033+0000 7f7d3c393640 1 -- 172.21.15.17:0/172924573 wait complete. 2024-03-06T07:30:40.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:40.448 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:40.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:40.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:30:41.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.549+0000 7f5e029c9640 1 Processor -- start 2024-03-06T07:30:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.549+0000 7f5e029c9640 1 -- start start 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5e029c9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dfc14f360 0x7f5dfc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5e029c9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5dfc158b60 0x7f5dfc158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5e029c9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc0c73d0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5e029c9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc0c8dd0 con 0x7f5dfc14f360 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5e029c9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc0be2e0 con 0x7f5dfc158b60 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dfc14f360 0x7f5dfc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5dfc0c88c0 0x7f5dfc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36980/0 (socket says 172.21.15.17:36980) 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfb7fe640 1 -- 172.21.15.17:0/781714439 learned_addr learned my addr 172.21.15.17:0/781714439 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfbfff640 1 -- 172.21.15.17:0/781714439 >> v2:172.21.15.163:3300/0 conn(0x7f5dfc158b60 msgr2=0x7f5dfc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfbfff640 1 --2- 172.21.15.17:0/781714439 >> v2:172.21.15.163:3300/0 conn(0x7f5dfc158b60 0x7f5dfc158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfbfff640 1 -- 172.21.15.17:0/781714439 >> v1:172.21.15.17:6789/0 conn(0x7f5dfc0c88c0 legacy=0x7f5dfc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfaffd640 1 -- 172.21.15.17:0/781714439 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940944518 0 0) 0x7f5dfc0c73d0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.550+0000 7f5dfbfff640 1 -- 172.21.15.17:0/781714439 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc0c7670 con 0x7f5dfc14f360 2024-03-06T07:30:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.551+0000 7f5dfbfff640 1 --2- 172.21.15.17:0/781714439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc14f360 0x7f5dfc155c10 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc152d60 tx=0x7f5dec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e1895c907ed40b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.551+0000 7f5dfaffd640 1 -- 172.21.15.17:0/781714439 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec0315a0 con 0x7f5dfc14f360 2024-03-06T07:30:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.551+0000 7f5dfaffd640 1 -- 172.21.15.17:0/781714439 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec037070 con 0x7f5dfc14f360 2024-03-06T07:30:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.551+0000 7f5dfaffd640 1 -- 172.21.15.17:0/781714439 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031960 con 0x7f5dfc14f360 2024-03-06T07:30:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 -- 172.21.15.17:0/781714439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc14f360 msgr2=0x7f5dfc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 --2- 172.21.15.17:0/781714439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc14f360 0x7f5dfc155c10 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc152d60 tx=0x7f5dec02b840 comp rx=0 tx=0).stop 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 -- 172.21.15.17:0/781714439 shutdown_connections 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 --2- 172.21.15.17:0/781714439 >> v2:172.21.15.163:3300/0 conn(0x7f5dfc158b60 0x7f5dfc158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 --2- 172.21.15.17:0/781714439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc14f360 0x7f5dfc155c10 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 -- 172.21.15.17:0/781714439 >> 172.21.15.17:0/781714439 conn(0x7f5dfc0ac570 msgr2=0x7f5dfc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 -- 172.21.15.17:0/781714439 shutdown_connections 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.552+0000 7f5e029c9640 1 -- 172.21.15.17:0/781714439 wait complete. 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.553+0000 7f5e029c9640 1 Processor -- start 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.553+0000 7f5e029c9640 1 -- start start 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 0x7f5dfc0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 0x7f5dfc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc0c5a50 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc0c7c30 con 0x7f5dfc158b60 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e029c9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc0c5bd0 con 0x7f5dfc14f360 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45032/0 (socket says 172.21.15.17:45032) 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 -- 172.21.15.17:0/2745104051 learned_addr learned my addr 172.21.15.17:0/2745104051 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfb7fe640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 0x7f5dfc0c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e00f66640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 0x7f5dfc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 -- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 msgr2=0x7f5dfc0c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 0x7f5dfc0c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 -- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 msgr2=0x7f5dfc157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 0x7f5dfc157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 -- 172.21.15.17:0/2745104051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc0c3530 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfb7fe640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 0x7f5dfc0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5e00f66640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 0x7f5dfc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5dfbfff640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f5dec034040 tx=0x7f5dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.554+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec032840 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.555+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec0329e0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.555+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc0c48b0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.555+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec032dd0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.555+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc0c2820 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.556+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc04f330 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.556+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5dec03b3f0 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.556+0000 7f5df8ff9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.557+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5dec02f030 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.557+0000 7f5dfb7fe640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:41.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.557+0000 7f5dfb7fe640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 0x7f5de806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc152d60 tx=0x7f5df0007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:41.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.561+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5dec040070 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.782+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dfc0ac950 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.782+0000 7f5df8ff9640 1 -- 172.21.15.17:0/2745104051 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f5dec07ad80 con 0x7f5dfc0c88c0 2024-03-06T07:30:41.784 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:41.784 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:41.785 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.784+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 msgr2=0x7f5de806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.784+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 0x7f5de806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc152d60 tx=0x7f5df0007ca0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.784+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 msgr2=0x7f5dfc14bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.784+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f5dec034040 tx=0x7f5dec0046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 shutdown_connections 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5de806b4d0 0x7f5de806d990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dfc158b60 0x7f5dfc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dfc14f360 0x7f5dfc0c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 --2- 172.21.15.17:0/2745104051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dfc0c88c0 0x7f5dfc14bf70 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 >> 172.21.15.17:0/2745104051 conn(0x7f5dfc0ac570 msgr2=0x7f5dfc14cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 shutdown_connections 2024-03-06T07:30:41.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:41.785+0000 7f5e029c9640 1 -- 172.21.15.17:0/2745104051 wait complete. 2024-03-06T07:30:42.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:42.199 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:43.200 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:43.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.300+0000 7fea9faed640 1 Processor -- start 2024-03-06T07:30:43.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.300+0000 7fea9faed640 1 -- start start 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.300+0000 7fea9faed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea9814f360 0x7fea98155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9faed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea98158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9faed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea980c73d0 con 0x7fea980c88c0 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9faed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea980c8dd0 con 0x7fea9814f360 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9faed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980be2e0 con 0x7fea98158b60 2024-03-06T07:30:43.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d889640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea9814f360 0x7fea98155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d088640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fea980c88c0 0x7fea980c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36982/0 (socket says 172.21.15.17:36982) 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d088640 1 -- 172.21.15.17:0/3118135915 learned_addr learned my addr 172.21.15.17:0/3118135915 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9e08a640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea98158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d889640 1 -- 172.21.15.17:0/3118135915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 msgr2=0x7fea98158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d889640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea98158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d889640 1 -- 172.21.15.17:0/3118135915 >> v1:172.21.15.17:6789/0 conn(0x7fea980c88c0 legacy=0x7fea980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9c887640 1 -- 172.21.15.17:0/3118135915 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516846720 0 0) 0x7fea980c73d0 con 0x7fea980c88c0 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9d889640 1 -- 172.21.15.17:0/3118135915 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980c7670 con 0x7fea9814f360 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.301+0000 7fea9e08a640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea98158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9d889640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98155c10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fea940097a0 tx=0x7fea9402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f5f9ec36214e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9c887640 1 -- 172.21.15.17:0/3118135915 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940315a0 con 0x7fea9814f360 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9c887640 1 -- 172.21.15.17:0/3118135915 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea94037070 con 0x7fea9814f360 2024-03-06T07:30:43.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9c887640 1 -- 172.21.15.17:0/3118135915 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94031960 con 0x7fea9814f360 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 -- 172.21.15.17:0/3118135915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 msgr2=0x7fea98155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98155c10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fea940097a0 tx=0x7fea9402b840 comp rx=0 tx=0).stop 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 -- 172.21.15.17:0/3118135915 shutdown_connections 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea98158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 --2- 172.21.15.17:0/3118135915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98155c10 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.302+0000 7fea9faed640 1 -- 172.21.15.17:0/3118135915 >> 172.21.15.17:0/3118135915 conn(0x7fea980ac570 msgr2=0x7fea980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.303+0000 7fea9faed640 1 -- 172.21.15.17:0/3118135915 shutdown_connections 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.303+0000 7fea9faed640 1 -- 172.21.15.17:0/3118135915 wait complete. 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 Processor -- start 2024-03-06T07:30:43.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 -- start start 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 0x7fea980ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea981e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980c5a50 con 0x7fea9814f360 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980c7c30 con 0x7fea980c88c0 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9faed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980c5bd0 con 0x7fea98158b60 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.304+0000 7fea9d088640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45050/0 (socket says 172.21.15.17:45050) 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 -- 172.21.15.17:0/2936468956 learned_addr learned my addr 172.21.15.17:0/2936468956 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9e08a640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea981e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d889640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 0x7fea980ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 -- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 msgr2=0x7fea981e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea981e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 -- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 msgr2=0x7fea980ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 0x7fea980ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 -- 172.21.15.17:0/2936468956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea981eb840 con 0x7fea9814f360 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9e08a640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea981e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d889640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 0x7fea980ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9d088640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c410 tx=0x7fea8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea880074e0 con 0x7fea9814f360 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea880047e0 con 0x7fea9814f360 2024-03-06T07:30:43.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea981eacc0 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88004bd0 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.305+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea981eb280 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.306+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9804f330 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.307+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea88011070 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.308+0000 7fea8e7fc640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 0x7fea7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.308+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fea88089fe0 con 0x7fea9814f360 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.308+0000 7fea9d889640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 0x7fea7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:43.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.309+0000 7fea9d889640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 0x7fea7c06d9c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fea94036000 tx=0x7fea9403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:43.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.312+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea8800e280 con 0x7fea9814f360 2024-03-06T07:30:43.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.531+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea981eb620 con 0x7fea9814f360 2024-03-06T07:30:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.531+0000 7fea8e7fc640 1 -- 172.21.15.17:0/2936468956 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7fea88058be0 con 0x7fea9814f360 2024-03-06T07:30:43.533 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:43.533 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:43.533 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.533+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 msgr2=0x7fea7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.533+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 0x7fea7c06d9c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fea94036000 tx=0x7fea9403d040 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 msgr2=0x7fea98151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c410 tx=0x7fea8800c8d0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 shutdown_connections 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fea7c06b500 0x7fea7c06d9c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea98158b60 0x7fea981e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea9814f360 0x7fea98151350 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 --2- 172.21.15.17:0/2936468956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea980c88c0 0x7fea980ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 >> 172.21.15.17:0/2936468956 conn(0x7fea980ac570 msgr2=0x7fea980ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 shutdown_connections 2024-03-06T07:30:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:43.534+0000 7fea9faed640 1 -- 172.21.15.17:0/2936468956 wait complete. 2024-03-06T07:30:43.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:43.948 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:44.949 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:45.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.065+0000 7fe90a023640 1 Processor -- start 2024-03-06T07:30:45.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.065+0000 7fe90a023640 1 -- start start 2024-03-06T07:30:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe90a023640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 0x7fe904155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe90a023640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe904158b60 0x7fe904158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe90a023640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040c73d0 con 0x7fe9040c88c0 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe90a023640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040c8dd0 con 0x7fe904158b60 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe90a023640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040be2e0 con 0x7fe90414f360 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe903fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe904158b60 0x7fe904158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe903fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe904158b60 0x7fe904158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45052/0 (socket says 172.21.15.17:45052) 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe903fff640 1 -- 172.21.15.17:0/677427028 learned_addr learned my addr 172.21.15.17:0/677427028 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.066+0000 7fe9037fe640 1 --2- 172.21.15.17:0/677427028 >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 0x7fe904155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe903fff640 1 -- 172.21.15.17:0/677427028 >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 msgr2=0x7fe904155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe903fff640 1 --2- 172.21.15.17:0/677427028 >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 0x7fe904155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe903fff640 1 -- 172.21.15.17:0/677427028 >> v1:172.21.15.163:6789/0 conn(0x7fe9040c88c0 legacy=0x7fe9040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe903fff640 1 -- 172.21.15.17:0/677427028 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040c7670 con 0x7fe904158b60 2024-03-06T07:30:45.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe9037fe640 1 --2- 172.21.15.17:0/677427028 >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 0x7fe904155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe903fff640 1 --2- 172.21.15.17:0/677427028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe904158b60 0x7fe904158ff0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fe90408c840 tx=0x7fe8fc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d54c50cbb8fda1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe9027fc640 1 -- 172.21.15.17:0/677427028 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc02c9f0 con 0x7fe904158b60 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe9027fc640 1 -- 172.21.15.17:0/677427028 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc02cb90 con 0x7fe904158b60 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.067+0000 7fe9027fc640 1 -- 172.21.15.17:0/677427028 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0343f0 con 0x7fe904158b60 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 -- 172.21.15.17:0/677427028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe904158b60 msgr2=0x7fe904158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 --2- 172.21.15.17:0/677427028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe904158b60 0x7fe904158ff0 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fe90408c840 tx=0x7fe8fc02be50 comp rx=0 tx=0).stop 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 -- 172.21.15.17:0/677427028 shutdown_connections 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 --2- 172.21.15.17:0/677427028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe904158b60 0x7fe904158ff0 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 --2- 172.21.15.17:0/677427028 >> v2:172.21.15.163:3300/0 conn(0x7fe90414f360 0x7fe904155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 -- 172.21.15.17:0/677427028 >> 172.21.15.17:0/677427028 conn(0x7fe9040ac570 msgr2=0x7fe9040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 -- 172.21.15.17:0/677427028 shutdown_connections 2024-03-06T07:30:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.068+0000 7fe90a023640 1 -- 172.21.15.17:0/677427028 wait complete. 2024-03-06T07:30:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.069+0000 7fe90a023640 1 Processor -- start 2024-03-06T07:30:45.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.069+0000 7fe90a023640 1 -- start start 2024-03-06T07:30:45.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9040c88c0 0x7fe9040c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe904158b60 0x7fe904151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c5a50 con 0x7fe90414f360 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c7c30 con 0x7fe904158b60 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe90a023640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c5bd0 con 0x7fe9040c88c0 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45068/0 (socket says 172.21.15.17:45068) 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 -- 172.21.15.17:0/380296344 learned_addr learned my addr 172.21.15.17:0/380296344 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe903fff640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe904158b60 0x7fe904151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe9037fe640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9040c88c0 0x7fe9040c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 -- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9040c88c0 msgr2=0x7fe9040c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9040c88c0 0x7fe9040c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 -- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe904158b60 msgr2=0x7fe904151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:45.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe904158b60 0x7fe904151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.070+0000 7fe902ffd640 1 -- 172.21.15.17:0/380296344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90414f990 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe902ffd640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b3c0 tx=0x7fe8ec00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec011830 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec016070 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90414fc70 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec011c50 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.071+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe904150230 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.072+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe904075300 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.072+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8ec01b460 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.073+0000 7fe8fbfff640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.073+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe8ec089f60 con 0x7fe90414f360 2024-03-06T07:30:45.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.073+0000 7fe9037fe640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:45.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.073+0000 7fe9037fe640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe9040c4e50 tx=0x7fe8f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:45.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.077+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe904075300 con 0x7fe90414f360 2024-03-06T07:30:45.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.293+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe90404f330 con 0x7fe90414f360 2024-03-06T07:30:45.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.294+0000 7fe8fbfff640 1 -- 172.21.15.17:0/380296344 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7fe8ec058590 con 0x7fe90414f360 2024-03-06T07:30:45.296 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:45.296 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:45.296 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 msgr2=0x7fe8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe9040c4e50 tx=0x7fe8f4008040 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 msgr2=0x7fe9040beac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00b3c0 tx=0x7fe8ec00b880 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 shutdown_connections 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe904158b60 0x7fe904151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe90414f360 0x7fe9040beac0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 --2- 172.21.15.17:0/380296344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9040c88c0 0x7fe9040c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 >> 172.21.15.17:0/380296344 conn(0x7fe9040ac570 msgr2=0x7fe9040ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 shutdown_connections 2024-03-06T07:30:45.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:45.296+0000 7fe90a023640 1 -- 172.21.15.17:0/380296344 wait complete. 2024-03-06T07:30:45.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:45.710 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:46.711 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.810+0000 7fd0d0846640 1 Processor -- start 2024-03-06T07:30:46.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.810+0000 7fd0d0846640 1 -- start start 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0d0846640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0cc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0d0846640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0c73d0 con 0x7fd0cc158b60 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0d0846640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0be2e0 con 0x7fd0cc14f360 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0d0846640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc159700 con 0x7fd0cc0c88c0 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0cad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd0cc158b60 0x7fd0cc158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36990/0 (socket says 172.21.15.17:36990) 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3267723051 learned_addr learned my addr 172.21.15.17:0/3267723051 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0cc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.811+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537060557 0 0) 0x7fd0cc0c73d0 con 0x7fd0cc158b60 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b4003520 con 0x7fd0cc158b60 2024-03-06T07:30:46.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2539136112 0 0) 0x7fd0b4003520 con 0x7fd0cc158b60 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 msgr2=0x7fd0cc155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0cc155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 >> v1:172.21.15.163:6789/0 conn(0x7fd0cc14f360 legacy=0x7fd0cc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0c7670 con 0x7fd0cc158b60 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd0bc0019f0 con 0x7fd0cc158b60 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 >> v1:172.21.15.17:6789/0 conn(0x7fd0cc158b60 legacy=0x7fd0cc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b40039e0 0x7fd0b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4004d40 0x7fd0b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc159700 con 0x7fd0b40039e0 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b400baf0 con 0x7fd0b4004d40 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b400bcf0 con 0x7fd0cc0c88c0 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b40039e0 0x7fd0b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4004d40 0x7fd0b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 msgr2=0x7fd0b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4004d40 msgr2=0x7fd0b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0b4004d40 0x7fd0b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0ca59c640 1 -- 172.21.15.17:0/3267723051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4003520 con 0x7fd0b40039e0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.812+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0cc0c88c0 0x7fd0b4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.813+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b40039e0 0x7fd0b4003e00 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4001750 tx=0x7fd0b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf38d6309c49fe4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.813+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4001370 con 0x7fd0b40039e0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.813+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c40052d0 con 0x7fd0b40039e0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.813+0000 7fd0c959a640 1 -- 172.21.15.17:0/3267723051 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4003830 con 0x7fd0b40039e0 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 -- 172.21.15.17:0/3267723051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b40039e0 msgr2=0x7fd0b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 --2- 172.21.15.17:0/3267723051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b40039e0 0x7fd0b4003e00 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4001750 tx=0x7fd0b4005830 comp rx=0 tx=0).stop 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0cad9d640 1 -- 172.21.15.17:0/3267723051 reap_dead start 2024-03-06T07:30:46.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 -- 172.21.15.17:0/3267723051 shutdown_connections 2024-03-06T07:30:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 -- 172.21.15.17:0/3267723051 >> 172.21.15.17:0/3267723051 conn(0x7fd0cc0ac570 msgr2=0x7fd0cc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 -- 172.21.15.17:0/3267723051 shutdown_connections 2024-03-06T07:30:46.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.814+0000 7fd0d0846640 1 -- 172.21.15.17:0/3267723051 wait complete. 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.816+0000 7fd0d0846640 1 Processor -- start 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.816+0000 7fd0d0846640 1 -- start start 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0b40039e0 0x7fd0cc1ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0c4a00 0x7fd0cc150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c7c30 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c5a50 con 0x7fd0cc0c4a00 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0d0846640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0c5bd0 con 0x7fd0b40039e0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45094/0 (socket says 172.21.15.17:45094) 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 -- 172.21.15.17:0/2660600768 learned_addr learned my addr 172.21.15.17:0/2660600768 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0cad9d640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0c4a00 0x7fd0cc150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0b40039e0 0x7fd0cc1ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 -- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0b40039e0 msgr2=0x7fd0cc1ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0b40039e0 0x7fd0cc1ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 -- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0c4a00 msgr2=0x7fd0cc150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0c4a00 0x7fd0cc150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.817+0000 7fd0c9d9b640 1 -- 172.21.15.17:0/2660600768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc14c550 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0c9d9b640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011610 tx=0x7fd0c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000f8f0 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c000fa90 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc151f10 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000fe40 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.818+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1524d0 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.819+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc04f330 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.819+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0c0015460 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.819+0000 7fd0baffd640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.820+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd0c008acf0 con 0x7fd0cc0c24e0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.820+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.820+0000 7fd0ca59c640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4033000 tx=0x7fd0c4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:46.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:46.824+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0c008f070 con 0x7fd0cc0c24e0 2024-03-06T07:30:47.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.043+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0cc0c8d50 con 0x7fd0cc0c24e0 2024-03-06T07:30:47.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.044+0000 7fd0baffd640 1 -- 172.21.15.17:0/2660600768 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7fd0c0019030 con 0x7fd0cc0c24e0 2024-03-06T07:30:47.046 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:47.046 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:47.047 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 msgr2=0x7fd0ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4033000 tx=0x7fd0c4009030 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 msgr2=0x7fd0cc154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0011610 tx=0x7fd0c0011ad0 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 shutdown_connections 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd0ac06b4d0 0x7fd0ac06d990 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0cc0c4a00 0x7fd0cc150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0cc0c24e0 0x7fd0cc154890 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 --2- 172.21.15.17:0/2660600768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd0b40039e0 0x7fd0cc1ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 >> 172.21.15.17:0/2660600768 conn(0x7fd0cc0ac570 msgr2=0x7fd0cc06b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 shutdown_connections 2024-03-06T07:30:47.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:47.045+0000 7fd0d0846640 1 -- 172.21.15.17:0/2660600768 wait complete. 2024-03-06T07:30:47.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:47.462 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:48.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:48.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.562+0000 7f1f3b181640 1 Processor -- start 2024-03-06T07:30:48.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.562+0000 7f1f3b181640 1 -- start start 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.562+0000 7f1f3b181640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f340c88b0 0x7f1f340c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.562+0000 7f1f3b181640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f34155110 con 0x7f1f340c5330 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.562+0000 7f1f3b181640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f340c8f40 con 0x7f1f34158db0 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f3b181640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f341559e0 con 0x7f1f340c88b0 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f38f1d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1f340c5330 0x7f1f341586a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37006/0 (socket says 172.21.15.17:37006) 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f38f1d640 1 -- 172.21.15.17:0/2584390925 learned_addr learned my addr 172.21.15.17:0/2584390925 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f3971e640 1 --2- 172.21.15.17:0/2584390925 >> v2:172.21.15.163:3300/0 conn(0x7f1f340c88b0 0x7f1f340c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782051395 0 0) 0x7f1f24001110 con 0x7f1f340c5330 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c003520 con 0x7f1f340c5330 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210310240 0 0) 0x7f1f1c003520 con 0x7f1f340c5330 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 >> v2:172.21.15.163:3300/0 conn(0x7f1f340c88b0 msgr2=0x7f1f340c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:48.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 --2- 172.21.15.17:0/2584390925 >> v2:172.21.15.163:3300/0 conn(0x7f1f340c88b0 0x7f1f340c5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 >> v1:172.21.15.163:6789/0 conn(0x7f1f34158db0 legacy=0x7f1f3415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34155390 con 0x7f1f340c5330 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f1f24001840 con 0x7f1f340c5330 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 >> v1:172.21.15.17:6789/0 conn(0x7f1f340c5330 legacy=0x7f1f341586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 0x7f1f1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c004800 0x7f1f1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f1c0051c0 0x7f1f1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341559e0 con 0x7f1f1c0039b0 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c00c160 con 0x7f1f1c004800 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c00c360 con 0x7f1f1c0051c0 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.563+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 0x7f1f1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f33fff640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c004800 0x7f1f1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f3971e640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f1c0051c0 0x7f1f1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 -- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f1c0051c0 msgr2=0x7f1f1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:48.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f1c0051c0 0x7f1f1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 -- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c004800 msgr2=0x7f1f1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c004800 0x7f1f1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 -- 172.21.15.17:0/2584390925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c003520 con 0x7f1f1c0039b0 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f3971e640 1 -- 172.21.15.17:0/2584390925 reap_dead start 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 0x7f1f1c003dd0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c4c0 tx=0x7f1f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aba372884101576 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.564+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400c0e0 con 0x7f1f1c0039b0 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24030070 con 0x7f1f1c0039b0 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f337fe640 1 -- 172.21.15.17:0/2584390925 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240058d0 con 0x7f1f1c0039b0 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 -- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 msgr2=0x7f1f1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 0x7f1f1c003dd0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400c4c0 tx=0x7f1f1c005830 comp rx=0 tx=0).stop 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 -- 172.21.15.17:0/2584390925 shutdown_connections 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2584390925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f1c0039b0 0x7f1f1c003dd0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 -- 172.21.15.17:0/2584390925 >> 172.21.15.17:0/2584390925 conn(0x7f1f340ac560 msgr2=0x7f1f340ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:48.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 -- 172.21.15.17:0/2584390925 shutdown_connections 2024-03-06T07:30:48.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.565+0000 7f1f3b181640 1 -- 172.21.15.17:0/2584390925 wait complete. 2024-03-06T07:30:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.566+0000 7f1f3b181640 1 Processor -- start 2024-03-06T07:30:48.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.566+0000 7f1f3b181640 1 -- start start 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 0x7f1f3414e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f340c1a80 0x7f1f341eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f341529d0 con 0x7f1f341ebc00 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3406d090 con 0x7f1f1c0039b0 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3b181640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34152b50 con 0x7f1f340c1a80 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45112/0 (socket says 172.21.15.17:45112) 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 -- 172.21.15.17:0/2734407464 learned_addr learned my addr 172.21.15.17:0/2734407464 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f33fff640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f340c1a80 0x7f1f341eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 0x7f1f3414e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 -- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f340c1a80 msgr2=0x7f1f341eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f340c1a80 0x7f1f341eb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 -- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 msgr2=0x7f1f3414e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 0x7f1f3414e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.567+0000 7f1f3971e640 1 -- 172.21.15.17:0/2734407464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f341ef300 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 0x7f1f3414e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f3971e640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f1f340aa480 tx=0x7f1f2c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c011de0 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c01b070 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f34158290 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c003990 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.568+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3414eae0 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.569+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f341ef090 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.569+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f2c003b30 con 0x7f1f341ebc00 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.570+0000 7f1f317fa640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 0x7f1f2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:48.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.570+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f1f2c08ab30 con 0x7f1f341ebc00 2024-03-06T07:30:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.570+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 0x7f1f2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.571+0000 7f1f38f1d640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 0x7f1f2006d9c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1f24034000 tx=0x7f1f24007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:48.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.574+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f2c003e20 con 0x7f1f341ebc00 2024-03-06T07:30:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.792+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f3404f330 con 0x7f1f341ebc00 2024-03-06T07:30:48.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.793+0000 7f1f317fa640 1 -- 172.21.15.17:0/2734407464 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f1f2c0597b0 con 0x7f1f341ebc00 2024-03-06T07:30:48.796 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:48.796 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:48.796 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:48.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 msgr2=0x7f1f2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 0x7f1f2006d9c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1f24034000 tx=0x7f1f24007000 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 msgr2=0x7f1f3414b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7f1f340aa480 tx=0x7f1f2c0115a0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 shutdown_connections 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f1f2006b500 0x7f1f2006d9c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f341ebc00 0x7f1f3414b9b0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f340c1a80 0x7f1f341eb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 --2- 172.21.15.17:0/2734407464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f1c0039b0 0x7f1f3414e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 >> 172.21.15.17:0/2734407464 conn(0x7f1f340ac560 msgr2=0x7f1f340c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 shutdown_connections 2024-03-06T07:30:48.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:48.795+0000 7f1f3b181640 1 -- 172.21.15.17:0/2734407464 wait complete. 2024-03-06T07:30:49.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:49.211 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:50.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.310+0000 7fe240adf640 1 Processor -- start 2024-03-06T07:30:50.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.310+0000 7fe240adf640 1 -- start start 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe240adf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe23c14f360 0x7fe23c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe240adf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe23c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe240adf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c0c73d0 con 0x7fe23c0c5880 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe240adf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0c8dd0 con 0x7fe23c14f360 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe240adf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0be2e0 con 0x7fe23c0c88c0 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe23ad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe23c0c5880 0x7fe23c158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37020/0 (socket says 172.21.15.17:37020) 2024-03-06T07:30:50.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.311+0000 7fe23ad9d640 1 -- 172.21.15.17:0/3580510367 learned_addr learned my addr 172.21.15.17:0/3580510367 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe233fff640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe23c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23a59c640 1 --2- 172.21.15.17:0/3580510367 >> v2:172.21.15.163:3300/0 conn(0x7fe23c14f360 0x7fe23c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604080199 0 0) 0x7fe23c0c73d0 con 0x7fe23c0c5880 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe228003520 con 0x7fe23c0c5880 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811195323 0 0) 0x7fe228003520 con 0x7fe23c0c5880 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 msgr2=0x7fe23c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe23c0c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 >> v2:172.21.15.163:3300/0 conn(0x7fe23c14f360 msgr2=0x7fe23c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 --2- 172.21.15.17:0/3580510367 >> v2:172.21.15.163:3300/0 conn(0x7fe23c14f360 0x7fe23c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0c7670 con 0x7fe23c0c5880 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe2340019f0 con 0x7fe23c0c5880 2024-03-06T07:30:50.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 >> v1:172.21.15.17:6789/0 conn(0x7fe23c0c5880 legacy=0x7fe23c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2280039b0 0x7fe228003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228004800 0x7fe228004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe2280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0be2e0 con 0x7fe2280039b0 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22800b9c0 con 0x7fe228004800 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22800bb40 con 0x7fe23c0c88c0 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23a59c640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2280039b0 0x7fe228003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe233fff640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228004800 0x7fe228004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23ad9d640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe2280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23a59c640 1 -- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 msgr2=0x7fe2280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23a59c640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0c88c0 0x7fe2280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.312+0000 7fe23a59c640 1 -- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228004800 msgr2=0x7fe228004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe23a59c640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe228004800 0x7fe228004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe23a59c640 1 -- 172.21.15.17:0/3580510367 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228003520 con 0x7fe2280039b0 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe23a59c640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2280039b0 0x7fe228003dd0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009af0 tx=0x7fe228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeca90362e80ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c039070 con 0x7fe2280039b0 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c02c710 con 0x7fe2280039b0 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.313+0000 7fe239d9b640 1 -- 172.21.15.17:0/3580510367 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c009690 con 0x7fe2280039b0 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 -- 172.21.15.17:0/3580510367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2280039b0 msgr2=0x7fe228003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 --2- 172.21.15.17:0/3580510367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2280039b0 0x7fe228003dd0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009af0 tx=0x7fe228005830 comp rx=0 tx=0).stop 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe23ad9d640 1 -- 172.21.15.17:0/3580510367 reap_dead start 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 -- 172.21.15.17:0/3580510367 shutdown_connections 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 -- 172.21.15.17:0/3580510367 >> 172.21.15.17:0/3580510367 conn(0x7fe23c0ac570 msgr2=0x7fe23c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:50.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 -- 172.21.15.17:0/3580510367 shutdown_connections 2024-03-06T07:30:50.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.314+0000 7fe240adf640 1 -- 172.21.15.17:0/3580510367 wait complete. 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.315+0000 7fe240adf640 1 Processor -- start 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.315+0000 7fe240adf640 1 -- start start 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe23c0ab6a0 0x7fe23c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0bdd90 0x7fe23c14b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0c5340 con 0x7fe23c152230 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0c7c30 con 0x7fe23c0ab6a0 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe240adf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0c54c0 con 0x7fe23c0bdd90 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45124/0 (socket says 172.21.15.17:45124) 2024-03-06T07:30:50.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23ad9d640 1 -- 172.21.15.17:0/2604630733 learned_addr learned my addr 172.21.15.17:0/2604630733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe233fff640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0bdd90 0x7fe23c14b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23a59c640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe23c0ab6a0 0x7fe23c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23ad9d640 1 -- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0bdd90 msgr2=0x7fe23c14b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.316+0000 7fe23ad9d640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0bdd90 0x7fe23c14b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe23ad9d640 1 -- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe23c0ab6a0 msgr2=0x7fe23c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe23ad9d640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe23c0ab6a0 0x7fe23c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe23ad9d640 1 -- 172.21.15.17:0/2604630733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0c1350 con 0x7fe23c152230 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe23ad9d640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0c1860 tx=0x7fe23400a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400ae50 con 0x7fe23c152230 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234019070 con 0x7fe23c152230 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c0aec60 con 0x7fe23c152230 2024-03-06T07:30:50.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234013840 con 0x7fe23c152230 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.317+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c0af220 con 0x7fe23c152230 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.318+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c04f330 con 0x7fe23c152230 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.318+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2340139e0 con 0x7fe23c152230 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.319+0000 7fe232ffd640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 0x7fe21c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.319+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe23408ae60 con 0x7fe23c152230 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.319+0000 7fe23a59c640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 0x7fe21c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.320+0000 7fe23a59c640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 0x7fe21c06db60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:50.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.323+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe234024250 con 0x7fe23c152230 2024-03-06T07:30:50.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.542+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23c06cda0 con 0x7fe23c152230 2024-03-06T07:30:50.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.543+0000 7fe232ffd640 1 -- 172.21.15.17:0/2604630733 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7fe234059ae0 con 0x7fe23c152230 2024-03-06T07:30:50.545 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:50.545 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:50.546 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 msgr2=0x7fe21c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 0x7fe21c06db60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c030000 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 msgr2=0x7fe23c1ee170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0c1860 tx=0x7fe23400a790 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 shutdown_connections 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21c06b6a0 0x7fe21c06db60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c152230 0x7fe23c1ee170 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe23c0bdd90 0x7fe23c14b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 --2- 172.21.15.17:0/2604630733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe23c0ab6a0 0x7fe23c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 >> 172.21.15.17:0/2604630733 conn(0x7fe23c0ac570 msgr2=0x7fe23c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:50.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 shutdown_connections 2024-03-06T07:30:50.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:50.545+0000 7fe240adf640 1 -- 172.21.15.17:0/2604630733 wait complete. 2024-03-06T07:30:50.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:50.961 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:51.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.077+0000 7f536ba8f640 1 Processor -- start 2024-03-06T07:30:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.077+0000 7f536ba8f640 1 -- start start 2024-03-06T07:30:52.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536ba8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5364156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536ba8f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5364070aa0 con 0x7f53641506b0 2024-03-06T07:30:52.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536ba8f640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5364070d80 con 0x7f53640acc00 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536ba8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5364153e10 con 0x7f5364153990 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536982b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f53641506b0 0x7f53640702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34344/0 (socket says 172.21.15.17:34344) 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.078+0000 7f536982b640 1 -- 172.21.15.17:0/1262317874 learned_addr learned my addr 172.21.15.17:0/1262317874 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109172506 0 0) 0x7f5364070aa0 con 0x7f53641506b0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536902a640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5364156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5350003520 con 0x7f53641506b0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018896854 0 0) 0x7f5350003520 con 0x7f53641506b0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 msgr2=0x7f5364156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5364156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 >> v1:172.21.15.163:6789/0 conn(0x7f53640acc00 legacy=0x7f53640c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53641566a0 con 0x7f53641506b0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f535c001670 con 0x7f53641506b0 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 >> v1:172.21.15.17:6789/0 conn(0x7f53641506b0 legacy=0x7f53640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53500039e0 0x7f5350003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5350004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5350004d40 0x7f53500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5364153e10 con 0x7f53500039e0 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535000baa0 con 0x7f5350004d40 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535000bca0 con 0x7f5364153990 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53500039e0 0x7f5350003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536902a640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5350004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536a02c640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5350004d40 0x7f53500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 -- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 msgr2=0x7f5350004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5350004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 -- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5350004d40 msgr2=0x7f53500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5350004d40 0x7f53500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536982b640 1 -- 172.21.15.17:0/1262317874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350003520 con 0x7f53500039e0 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.079+0000 7f536902a640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5364153990 0x7f5350004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:52.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.080+0000 7f536982b640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53500039e0 0x7f5350003e00 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c210 tx=0x7f5350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d22487c1729df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.080+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c007cf0 con 0x7f53500039e0 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.080+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c030070 con 0x7f53500039e0 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.080+0000 7f5368829640 1 -- 172.21.15.17:0/1262317874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005d30 con 0x7f53500039e0 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 -- 172.21.15.17:0/1262317874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53500039e0 msgr2=0x7f5350003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 --2- 172.21.15.17:0/1262317874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53500039e0 0x7f5350003e00 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c210 tx=0x7f5350005830 comp rx=0 tx=0).stop 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536a02c640 1 -- 172.21.15.17:0/1262317874 reap_dead start 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 -- 172.21.15.17:0/1262317874 shutdown_connections 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 -- 172.21.15.17:0/1262317874 >> 172.21.15.17:0/1262317874 conn(0x7f536406b1a0 msgr2=0x7f536406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 -- 172.21.15.17:0/1262317874 shutdown_connections 2024-03-06T07:30:52.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.081+0000 7f536ba8f640 1 -- 172.21.15.17:0/1262317874 wait complete. 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.082+0000 7f536ba8f640 1 Processor -- start 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.082+0000 7f536ba8f640 1 -- start start 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500039e0 0x7f53641e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5364154120 0x7f53641eca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5364156a10 con 0x7f5364144050 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5364156450 con 0x7f5364154120 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53640ad070 con 0x7f53500039e0 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38702/0 (socket says 172.21.15.17:38702) 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 -- 172.21.15.17:0/682566623 learned_addr learned my addr 172.21.15.17:0/682566623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536982b640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500039e0 0x7f53641e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536a02c640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5364154120 0x7f53641eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 -- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500039e0 msgr2=0x7f53641e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500039e0 0x7f53641e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 -- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5364154120 msgr2=0x7f53641eca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5364154120 0x7f53641eca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 -- 172.21.15.17:0/682566623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53641561d0 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536902a640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f5354011730 tx=0x7f5354011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535400f9c0 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535400fb60 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53641e90c0 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354015460 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.083+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5364155ef0 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.084+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536404f330 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.085+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5354015600 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.085+0000 7f535a7fc640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 0x7f534806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.085+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f535408ada0 con 0x7f5364144050 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.085+0000 7f536982b640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 0x7f534806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:52.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.086+0000 7f536982b640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 0x7f534806d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f535c034000 tx=0x7f535c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:52.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.090+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5354019030 con 0x7f5364144050 2024-03-06T07:30:52.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.309+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53640700a0 con 0x7f5364144050 2024-03-06T07:30:52.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.309+0000 7f535a7fc640 1 -- 172.21.15.17:0/682566623 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f5354059a20 con 0x7f5364144050 2024-03-06T07:30:52.312 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:52.312 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:52.312 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 msgr2=0x7f534806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 0x7f534806d990 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f535c034000 tx=0x7f535c007000 comp rx=0 tx=0).stop 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 msgr2=0x7f53641496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f5354011730 tx=0x7f5354011bf0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 shutdown_connections 2024-03-06T07:30:52.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f534806b4d0 0x7f534806d990 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5364154120 0x7f53641eca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5364144050 0x7f53641496a0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 --2- 172.21.15.17:0/682566623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53500039e0 0x7f53641e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 >> 172.21.15.17:0/682566623 conn(0x7f536406b1a0 msgr2=0x7f5364151000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 shutdown_connections 2024-03-06T07:30:52.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:52.311+0000 7f536ba8f640 1 -- 172.21.15.17:0/682566623 wait complete. 2024-03-06T07:30:52.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:52.728 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:53.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:53.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.828+0000 7f9ead855640 1 Processor -- start 2024-03-06T07:30:53.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 -- start start 2024-03-06T07:30:53.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8152960 0x7f9ea806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea80ac260 0x7f9ea81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea8152d40 con 0x7f9ea81592d0 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea8066e30 con 0x7f9ea80ac260 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ead855640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80c1dc0 con 0x7f9ea8152960 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ea77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9ea80ac260 0x7f9ea81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ea67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9ea81592d0 0x7f9ea81552d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34352/0 (socket says 172.21.15.17:34352) 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.829+0000 7f9ea67fc640 1 -- 172.21.15.17:0/748814404 learned_addr learned my addr 172.21.15.17:0/748814404 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.830+0000 7f9ea6ffd640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8152960 0x7f9ea806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.830+0000 7f9ea77fe640 1 -- 172.21.15.17:0/748814404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8152960 msgr2=0x7f9ea806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.830+0000 7f9ea77fe640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8152960 0x7f9ea806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.830+0000 7f9ea77fe640 1 -- 172.21.15.17:0/748814404 >> v1:172.21.15.17:6789/0 conn(0x7f9ea81592d0 legacy=0x7f9ea81552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.830+0000 7f9ea77fe640 1 -- 172.21.15.17:0/748814404 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80c1b50 con 0x7f9ea80ac260 2024-03-06T07:30:53.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.831+0000 7f9ea77fe640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea81559e0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d90 tx=0x7f9ea002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6735005674800e9d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.831+0000 7f9ea5ffb640 1 -- 172.21.15.17:0/748814404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0039070 con 0x7f9ea80ac260 2024-03-06T07:30:53.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.831+0000 7f9ea5ffb640 1 -- 172.21.15.17:0/748814404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea002cc50 con 0x7f9ea80ac260 2024-03-06T07:30:53.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.831+0000 7f9ea5ffb640 1 -- 172.21.15.17:0/748814404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0034440 con 0x7f9ea80ac260 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 -- 172.21.15.17:0/748814404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 msgr2=0x7f9ea81559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea81559e0 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d90 tx=0x7f9ea002bea0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 -- 172.21.15.17:0/748814404 shutdown_connections 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea81559e0 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 --2- 172.21.15.17:0/748814404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea8152960 0x7f9ea806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 -- 172.21.15.17:0/748814404 >> 172.21.15.17:0/748814404 conn(0x7f9ea80c70b0 msgr2=0x7f9ea80c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 -- 172.21.15.17:0/748814404 shutdown_connections 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.832+0000 7f9ead855640 1 -- 172.21.15.17:0/748814404 wait complete. 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.833+0000 7f9ead855640 1 Processor -- start 2024-03-06T07:30:53.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.833+0000 7f9ead855640 1 -- start start 2024-03-06T07:30:53.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8152960 0x7f9ea8153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea81eaf60 0x7f9ea8153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80c20a0 con 0x7f9ea80ac260 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8159760 con 0x7f9ea8152960 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80c2220 con 0x7f9ea81eaf60 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38732/0 (socket says 172.21.15.17:38732) 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 -- 172.21.15.17:0/3098064192 learned_addr learned my addr 172.21.15.17:0/3098064192 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea77fe640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea81eaf60 0x7f9ea8153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea67fc640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8152960 0x7f9ea8153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 -- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea81eaf60 msgr2=0x7f9ea8153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea81eaf60 0x7f9ea8153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 -- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8152960 msgr2=0x7f9ea8153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8152960 0x7f9ea8153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 -- 172.21.15.17:0/3098064192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea81eb440 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ea6ffd640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c820 tx=0x7f9e9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98015070 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98004240 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81eb720 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800ce10 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.834+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81ebce0 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.835+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea804f330 con 0x7f9ea80ac260 2024-03-06T07:30:53.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.836+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e98007830 con 0x7f9ea80ac260 2024-03-06T07:30:53.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.836+0000 7f9e8ffff640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 0x7f9e9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:53.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.836+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f9e9808a370 con 0x7f9ea80ac260 2024-03-06T07:30:53.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.837+0000 7f9ea67fc640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 0x7f9e9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:53.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.837+0000 7f9ea67fc640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 0x7f9e9406d990 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80c5950 tx=0x7f9e9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:53.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:53.841+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e98058f50 con 0x7f9ea80ac260 2024-03-06T07:30:54.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.061+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ea8156270 con 0x7f9ea80ac260 2024-03-06T07:30:54.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.062+0000 7f9e8ffff640 1 -- 172.21.15.17:0/3098064192 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f9e98058850 con 0x7f9ea80ac260 2024-03-06T07:30:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:54.065 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:54.065 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 msgr2=0x7f9e9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 0x7f9e9406d990 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80c5950 tx=0x7f9e9c009040 comp rx=0 tx=0).stop 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 msgr2=0x7f9ea8071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c820 tx=0x7f9e9800cce0 comp rx=0 tx=0).stop 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 shutdown_connections 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9e9406b4d0 0x7f9e9406d990 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ea81eaf60 0x7f9ea8153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ea8152960 0x7f9ea8153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:54.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 --2- 172.21.15.17:0/3098064192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ea80ac260 0x7f9ea8071f10 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 >> 172.21.15.17:0/3098064192 conn(0x7f9ea80c70b0 msgr2=0x7f9ea806bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 shutdown_connections 2024-03-06T07:30:54.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:54.064+0000 7f9ead855640 1 -- 172.21.15.17:0/3098064192 wait complete. 2024-03-06T07:30:54.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:54.480 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:55.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:55.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.581+0000 7f470a2e1640 1 Processor -- start 2024-03-06T07:30:55.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.581+0000 7f470a2e1640 1 -- start start 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f470a2e1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47040c88c0 0x7f4704155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f470a2e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040c73d0 con 0x7f470414f360 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f470a2e1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47040be2e0 con 0x7f4704158b60 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f470a2e1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4704159700 con 0x7f47040c88c0 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f47037fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47040c88c0 0x7f4704155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f4703fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f470414f360 0x7f47040c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34366/0 (socket says 172.21.15.17:34366) 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f4703fff640 1 -- 172.21.15.17:0/2993271074 learned_addr learned my addr 172.21.15.17:0/2993271074 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:55.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f47037fe640 1 -- 172.21.15.17:0/2993271074 >> v1:172.21.15.163:6789/0 conn(0x7f4704158b60 legacy=0x7f4704158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f47037fe640 1 -- 172.21.15.17:0/2993271074 >> v1:172.21.15.17:6789/0 conn(0x7f470414f360 legacy=0x7f47040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f4702ffd640 1 -- 172.21.15.17:0/2993271074 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766424077 0 0) 0x7f47040c73d0 con 0x7f470414f360 2024-03-06T07:30:55.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.582+0000 7f47037fe640 1 -- 172.21.15.17:0/2993271074 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040c7670 con 0x7f47040c88c0 2024-03-06T07:30:55.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f47037fe640 1 --2- 172.21.15.17:0/2993271074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47040c88c0 0x7f4704155c10 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1fb48b468ffc76 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f4702ffd640 1 -- 172.21.15.17:0/2993271074 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80315a0 con 0x7f47040c88c0 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f4702ffd640 1 -- 172.21.15.17:0/2993271074 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f8037070 con 0x7f47040c88c0 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f4702ffd640 1 -- 172.21.15.17:0/2993271074 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8031960 con 0x7f47040c88c0 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f470a2e1640 1 -- 172.21.15.17:0/2993271074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47040c88c0 msgr2=0x7f4704155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f470a2e1640 1 --2- 172.21.15.17:0/2993271074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47040c88c0 0x7f4704155c10 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097a0 tx=0x7f46f802b840 comp rx=0 tx=0).stop 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f470a2e1640 1 -- 172.21.15.17:0/2993271074 shutdown_connections 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.583+0000 7f470a2e1640 1 --2- 172.21.15.17:0/2993271074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47040c88c0 0x7f4704155c10 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.584+0000 7f470a2e1640 1 -- 172.21.15.17:0/2993271074 >> 172.21.15.17:0/2993271074 conn(0x7f47040ac570 msgr2=0x7f47040ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.584+0000 7f470a2e1640 1 -- 172.21.15.17:0/2993271074 shutdown_connections 2024-03-06T07:30:55.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.584+0000 7f470a2e1640 1 -- 172.21.15.17:0/2993271074 wait complete. 2024-03-06T07:30:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 Processor -- start 2024-03-06T07:30:55.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 -- start start 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47040c88c0 0x7f47040be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4704158b60 0x7f4704154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c7c30 con 0x7f470414f360 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c5a50 con 0x7f47040c88c0 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.585+0000 7f470a2e1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47040c5bd0 con 0x7f4704158b60 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38736/0 (socket says 172.21.15.17:38736) 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 -- 172.21.15.17:0/3813141770 learned_addr learned my addr 172.21.15.17:0/3813141770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:55.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f470887e640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4704158b60 0x7f4704154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f4703fff640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47040c88c0 0x7f47040be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 -- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4704158b60 msgr2=0x7f4704154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4704158b60 0x7f4704154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 -- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47040c88c0 msgr2=0x7f47040be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47040c88c0 0x7f47040be760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 -- 172.21.15.17:0/3813141770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704153980 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f47037fe640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f46f8034040 tx=0x7f46f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8032840 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f80329e0 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4704158530 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8032dd0 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.586+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4704153c40 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.587+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470404f330 con 0x7f470414f360 2024-03-06T07:30:55.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.588+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f46f803b3f0 con 0x7f470414f360 2024-03-06T07:30:55.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.589+0000 7f4700ff9640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:55.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.589+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f46f802f030 con 0x7f470414f360 2024-03-06T07:30:55.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.589+0000 7f4703fff640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:55.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.589+0000 7f4703fff640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 0x7f46f006d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007fd0 tx=0x7f46f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:55.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.593+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f46f8040070 con 0x7f470414f360 2024-03-06T07:30:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.810+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47040c8d50 con 0x7f470414f360 2024-03-06T07:30:55.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.811+0000 7f4700ff9640 1 -- 172.21.15.17:0/3813141770 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f46f807b010 con 0x7f470414f360 2024-03-06T07:30:55.813 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:55.814 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:55.814 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 msgr2=0x7f46f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 0x7f46f006d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f46f4007fd0 tx=0x7f46f4009040 comp rx=0 tx=0).stop 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 msgr2=0x7f4704154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f46f8034040 tx=0x7f46f80046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 shutdown_connections 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f46f006b4d0 0x7f46f006d990 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4704158b60 0x7f4704154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f470414f360 0x7f4704154440 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.812+0000 7f470a2e1640 1 --2- 172.21.15.17:0/3813141770 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47040c88c0 0x7f47040be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.813+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 >> 172.21.15.17:0/3813141770 conn(0x7f47040ac570 msgr2=0x7f470414cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.813+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 shutdown_connections 2024-03-06T07:30:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:55.813+0000 7f470a2e1640 1 -- 172.21.15.17:0/3813141770 wait complete. 2024-03-06T07:30:56.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:56.229 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:57.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:57.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.329+0000 7f25d0422640 1 Processor -- start 2024-03-06T07:30:57.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.329+0000 7f25d0422640 1 -- start start 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25d0422640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25c80c88c0 0x7f25c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25d0422640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c8158b60 0x7f25c8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25d0422640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c80c73d0 con 0x7f25c814f360 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25d0422640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c80c8dd0 con 0x7f25c80c88c0 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25d0422640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80be2e0 con 0x7f25c8158b60 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25cd9bd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f25c80c88c0 0x7f25c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25ce1be640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f25c814f360 0x7f25c80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34370/0 (socket says 172.21.15.17:34370) 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25ce1be640 1 -- 172.21.15.17:0/29876296 learned_addr learned my addr 172.21.15.17:0/29876296 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.330+0000 7f25ce9bf640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c8158b60 0x7f25c8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.331+0000 7f25cd9bd640 1 -- 172.21.15.17:0/29876296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c8158b60 msgr2=0x7f25c8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:57.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.331+0000 7f25cd9bd640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c8158b60 0x7f25c8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.331+0000 7f25cd9bd640 1 -- 172.21.15.17:0/29876296 >> v1:172.21.15.17:6789/0 conn(0x7f25c814f360 legacy=0x7f25c80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.331+0000 7f25cd1bc640 1 -- 172.21.15.17:0/29876296 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228780456 0 0) 0x7f25c80c73d0 con 0x7f25c814f360 2024-03-06T07:30:57.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.331+0000 7f25cd9bd640 1 -- 172.21.15.17:0/29876296 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80c7670 con 0x7f25c80c88c0 2024-03-06T07:30:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25cd9bd640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c80c88c0 0x7f25c8155c10 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f25b80097a0 tx=0x7f25b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc905c8e614f909 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25cd1bc640 1 -- 172.21.15.17:0/29876296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80315a0 con 0x7f25c80c88c0 2024-03-06T07:30:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25cd1bc640 1 -- 172.21.15.17:0/29876296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8037070 con 0x7f25c80c88c0 2024-03-06T07:30:57.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25cd1bc640 1 -- 172.21.15.17:0/29876296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8031960 con 0x7f25c80c88c0 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 -- 172.21.15.17:0/29876296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c80c88c0 msgr2=0x7f25c8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c80c88c0 0x7f25c8155c10 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f25b80097a0 tx=0x7f25b802b840 comp rx=0 tx=0).stop 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 -- 172.21.15.17:0/29876296 shutdown_connections 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c8158b60 0x7f25c8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 --2- 172.21.15.17:0/29876296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c80c88c0 0x7f25c8155c10 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 -- 172.21.15.17:0/29876296 >> 172.21.15.17:0/29876296 conn(0x7f25c80ac570 msgr2=0x7f25c80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 -- 172.21.15.17:0/29876296 shutdown_connections 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.332+0000 7f25d0422640 1 -- 172.21.15.17:0/29876296 wait complete. 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 Processor -- start 2024-03-06T07:30:57.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 -- start start 2024-03-06T07:30:57.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 0x7f25c8150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 0x7f25c814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c5a50 con 0x7f25c814f360 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c7c30 con 0x7f25c8158b60 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25d0422640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c5bd0 con 0x7f25c80c88c0 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25cd9bd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25cd9bd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38750/0 (socket says 172.21.15.17:38750) 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.334+0000 7f25cd9bd640 1 -- 172.21.15.17:0/3670094572 learned_addr learned my addr 172.21.15.17:0/3670094572 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25ce1be640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 0x7f25c8150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25ce9bf640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 0x7f25c814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 -- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 msgr2=0x7f25c8150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 0x7f25c8150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 -- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 msgr2=0x7f25c814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 0x7f25c814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 -- 172.21.15.17:0/3670094572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c81eef10 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25ce1be640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 0x7f25c8150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25ce9bf640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 0x7f25c814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25cd9bd640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f25b8034040 tx=0x7f25b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8032630 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b80327d0 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c8150360 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8032ba0 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.335+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c8150640 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.336+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c8150150 con 0x7f25c814f360 2024-03-06T07:30:57.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.337+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25b804e430 con 0x7f25c814f360 2024-03-06T07:30:57.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.337+0000 7f25beffd640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 0x7f25ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:57.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.337+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f25b802f030 con 0x7f25c814f360 2024-03-06T07:30:57.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.337+0000 7f25ce1be640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 0x7f25ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:57.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.338+0000 7f25ce1be640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 0x7f25ac06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f60 tx=0x7f25c4007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:57.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.342+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f25b8040070 con 0x7f25c814f360 2024-03-06T07:30:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.562+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25c804f330 con 0x7f25c814f360 2024-03-06T07:30:57.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.563+0000 7f25beffd640 1 -- 172.21.15.17:0/3670094572 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f25b807b000 con 0x7f25c814f360 2024-03-06T07:30:57.566 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:57.566 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:57.566 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 msgr2=0x7f25ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 0x7f25ac06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f25c4000f60 tx=0x7f25c4007cc0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 msgr2=0x7f25c81ee5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f25b8034040 tx=0x7f25b80046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 shutdown_connections 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f25ac06b4d0 0x7f25ac06d990 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25c8158b60 0x7f25c814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25c814f360 0x7f25c81ee5d0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 --2- 172.21.15.17:0/3670094572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25c80c88c0 0x7f25c8150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 >> 172.21.15.17:0/3670094572 conn(0x7f25c80ac570 msgr2=0x7f25c806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 shutdown_connections 2024-03-06T07:30:57.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:57.565+0000 7f25d0422640 1 -- 172.21.15.17:0/3670094572 wait complete. 2024-03-06T07:30:57.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:57.981 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:30:58.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:30:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.093+0000 7f5bb9c37640 1 Processor -- start 2024-03-06T07:30:59.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.093+0000 7f5bb9c37640 1 -- start start 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.094+0000 7f5bb9c37640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bb414f360 0x7f5bb4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.094+0000 7f5bb9c37640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5bb40c88c0 0x7f5bb40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb9c37640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb40c73d0 con 0x7f5bb40c5880 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb9c37640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb40c8dd0 con 0x7f5bb414f360 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb9c37640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb40be2e0 con 0x7f5bb40c88c0 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bb414f360 0x7f5bb4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5bb414f360 0x7f5bb4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38762/0 (socket says 172.21.15.17:38762) 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 -- 172.21.15.17:0/3651609515 learned_addr learned my addr 172.21.15.17:0/3651609515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 -- 172.21.15.17:0/3651609515 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40c88c0 msgr2=0x7f5bb40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 --2- 172.21.15.17:0/3651609515 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40c88c0 0x7f5bb40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 -- 172.21.15.17:0/3651609515 >> v1:172.21.15.163:6789/0 conn(0x7f5bb40c5880 legacy=0x7f5bb4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:30:59.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.095+0000 7f5bb37fe640 1 -- 172.21.15.17:0/3651609515 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb40c7670 con 0x7f5bb414f360 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.096+0000 7f5bb37fe640 1 --2- 172.21.15.17:0/3651609515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb414f360 0x7f5bb4155c10 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c84c0 tx=0x7f5ba402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffab19b275c699 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.096+0000 7f5bb27fc640 1 -- 172.21.15.17:0/3651609515 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba402e070 con 0x7f5bb414f360 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.096+0000 7f5bb27fc640 1 -- 172.21.15.17:0/3651609515 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4037070 con 0x7f5bb414f360 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.096+0000 7f5bb27fc640 1 -- 172.21.15.17:0/3651609515 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4031a00 con 0x7f5bb414f360 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 -- 172.21.15.17:0/3651609515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb414f360 msgr2=0x7f5bb4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/3651609515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb414f360 0x7f5bb4155c10 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c84c0 tx=0x7f5ba402b880 comp rx=0 tx=0).stop 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 -- 172.21.15.17:0/3651609515 shutdown_connections 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/3651609515 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40c88c0 0x7f5bb40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/3651609515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb414f360 0x7f5bb4155c10 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 -- 172.21.15.17:0/3651609515 >> 172.21.15.17:0/3651609515 conn(0x7f5bb40ac570 msgr2=0x7f5bb40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 -- 172.21.15.17:0/3651609515 shutdown_connections 2024-03-06T07:30:59.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.097+0000 7f5bb9c37640 1 -- 172.21.15.17:0/3651609515 wait complete. 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.098+0000 7f5bb9c37640 1 Processor -- start 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.098+0000 7f5bb9c37640 1 -- start start 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bb40c88c0 0x7f5bb40c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb414f360 0x7f5bb40be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4159270 con 0x7f5bb40c5880 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40c7c30 con 0x7f5bb414f360 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41593f0 con 0x7f5bb40c88c0 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38766/0 (socket says 172.21.15.17:38766) 2024-03-06T07:30:59.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 -- 172.21.15.17:0/2193298623 learned_addr learned my addr 172.21.15.17:0/2193298623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb3fff640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb414f360 0x7f5bb40be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb2ffd640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bb40c88c0 0x7f5bb40c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 -- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bb40c88c0 msgr2=0x7f5bb40c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bb40c88c0 0x7f5bb40c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 -- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb414f360 msgr2=0x7f5bb40be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb414f360 0x7f5bb40be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 -- 172.21.15.17:0/2193298623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb41ebe40 con 0x7f5bb40c5880 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb37fe640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f5ba402b8c0 tx=0x7f5ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4032880 con 0x7f5bb40c5880 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4032a20 con 0x7f5bb40c5880 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb40ad940 con 0x7f5bb40c5880 2024-03-06T07:30:59.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4032e10 con 0x7f5bb40c5880 2024-03-06T07:30:59.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.099+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb40b0140 con 0x7f5bb40c5880 2024-03-06T07:30:59.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.100+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb40c1240 con 0x7f5bb40c5880 2024-03-06T07:30:59.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.101+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ba40453f0 con 0x7f5bb40c5880 2024-03-06T07:30:59.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.101+0000 7f5babfff640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 0x7f5b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:30:59.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.101+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5ba40ac450 con 0x7f5bb40c5880 2024-03-06T07:30:59.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.101+0000 7f5bb2ffd640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 0x7f5b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:30:59.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.102+0000 7f5bb2ffd640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 0x7f5b9406d990 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c1700 tx=0x7f5b9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:30:59.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.106+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5bb40c1240 con 0x7f5bb40c5880 2024-03-06T07:30:59.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.329+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bb404f330 con 0x7f5bb40c5880 2024-03-06T07:30:59.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.329+0000 7f5babfff640 1 -- 172.21.15.17:0/2193298623 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f5ba4040070 con 0x7f5bb40c5880 2024-03-06T07:30:59.332 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:30:59.332 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:30:59.332 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 msgr2=0x7f5b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 0x7f5b9406d990 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40c1700 tx=0x7f5b9c008040 comp rx=0 tx=0).stop 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 msgr2=0x7f5bb41eb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 secure :-1 s=READY pgs=2504 cs=0 l=1 rev1=1 crypto rx=0x7f5ba402b8c0 tx=0x7f5ba40046c0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 shutdown_connections 2024-03-06T07:30:59.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5b9406b4d0 0x7f5b9406d990 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb414f360 0x7f5bb40be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5bb40c88c0 0x7f5bb40c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 --2- 172.21.15.17:0/2193298623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bb40c5880 0x7f5bb41eb900 unknown :-1 s=CLOSED pgs=2504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 >> 172.21.15.17:0/2193298623 conn(0x7f5bb40ac570 msgr2=0x7f5bb40c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 shutdown_connections 2024-03-06T07:30:59.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:30:59.331+0000 7f5bb9c37640 1 -- 172.21.15.17:0/2193298623 wait complete. 2024-03-06T07:30:59.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:30:59.748 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:00.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:00.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.847+0000 7f94ee3a2640 1 Processor -- start 2024-03-06T07:31:00.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.847+0000 7f94ee3a2640 1 -- start start 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ee3a2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e814f450 0x7f94e806cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ee3a2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e806d580 con 0x7f94e8151c80 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ee3a2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e806d860 con 0x7f94e8153550 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ee3a2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e8155040 con 0x7f94e814f450 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e814f450 0x7f94e806cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f94e814f450 0x7f94e806cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47896/0 (socket says 172.21.15.17:47896) 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 -- 172.21.15.17:0/4041262609 learned_addr learned my addr 172.21.15.17:0/4041262609 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 -- 172.21.15.17:0/4041262609 >> v1:172.21.15.163:6789/0 conn(0x7f94e8153550 legacy=0x7f94e81539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 -- 172.21.15.17:0/4041262609 >> v1:172.21.15.17:6789/0 conn(0x7f94e8151c80 legacy=0x7f94e80c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94ed3a0640 1 -- 172.21.15.17:0/4041262609 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e806db40 con 0x7f94e814f450 2024-03-06T07:31:00.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.848+0000 7f94e7fff640 1 -- 172.21.15.17:0/4041262609 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835646598 0 0) 0x7f94e806d580 con 0x7f94e8151c80 2024-03-06T07:31:00.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.849+0000 7f94ed3a0640 1 --2- 172.21.15.17:0/4041262609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e814f450 0x7f94e806cfa0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009e50 tx=0x7f94d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd688d55408e6ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.849+0000 7f94e7fff640 1 -- 172.21.15.17:0/4041262609 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d802ca80 con 0x7f94e814f450 2024-03-06T07:31:00.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.849+0000 7f94e7fff640 1 -- 172.21.15.17:0/4041262609 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d802cc20 con 0x7f94e814f450 2024-03-06T07:31:00.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.849+0000 7f94e7fff640 1 -- 172.21.15.17:0/4041262609 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d80343f0 con 0x7f94e814f450 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 -- 172.21.15.17:0/4041262609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e814f450 msgr2=0x7f94e806cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/4041262609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e814f450 0x7f94e806cfa0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009e50 tx=0x7f94d802bee0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 -- 172.21.15.17:0/4041262609 shutdown_connections 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/4041262609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e814f450 0x7f94e806cfa0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 -- 172.21.15.17:0/4041262609 >> 172.21.15.17:0/4041262609 conn(0x7f94e80c2e90 msgr2=0x7f94e80c3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 -- 172.21.15.17:0/4041262609 shutdown_connections 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.850+0000 7f94ee3a2640 1 -- 172.21.15.17:0/4041262609 wait complete. 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.851+0000 7f94ee3a2640 1 Processor -- start 2024-03-06T07:31:00.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.851+0000 7f94ee3a2640 1 -- start start 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 0x7f94e81eb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94e8153550 0x7f94e80bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80c25b0 con 0x7f94e8151c80 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e806e480 con 0x7f94e814f450 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80c2730 con 0x7f94e8153550 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47910/0 (socket says 172.21.15.17:47910) 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 -- 172.21.15.17:0/985584039 learned_addr learned my addr 172.21.15.17:0/985584039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 -- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94e8153550 msgr2=0x7f94e80bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94e8153550 0x7f94e80bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 -- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 msgr2=0x7f94e81eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ed3a0640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 0x7f94e81eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 0x7f94e81eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 -- 172.21.15.17:0/985584039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80c7190 con 0x7f94e8151c80 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ed3a0640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 0x7f94e81eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ecb9f640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c490 tx=0x7f94dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc005480 con 0x7f94e8151c80 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc0035f0 con 0x7f94e8151c80 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.852+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e80c68f0 con 0x7f94e8151c80 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.853+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0039c0 con 0x7f94e8151c80 2024-03-06T07:31:00.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.853+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e80c6eb0 con 0x7f94e8151c80 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.854+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e8069430 con 0x7f94e8151c80 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.854+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94dc003b60 con 0x7f94e8151c80 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.854+0000 7f94e5ffb640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 0x7f94d406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.854+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f94dc08a0f0 con 0x7f94e8151c80 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.855+0000 7f94ed3a0640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 0x7f94d406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:00.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.855+0000 7f94ed3a0640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 0x7f94d406db30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f94d8035000 tx=0x7f94d8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:00.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:00.859+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94dc0182b0 con 0x7f94e8151c80 2024-03-06T07:31:01.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.260+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94e8154230 con 0x7f94e8151c80 2024-03-06T07:31:01.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.260+0000 7f94e5ffb640 1 -- 172.21.15.17:0/985584039 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f94dc058cd0 con 0x7f94e8151c80 2024-03-06T07:31:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:01.263 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:01.263 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 msgr2=0x7f94d406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 0x7f94d406db30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f94d8035000 tx=0x7f94d8030000 comp rx=0 tx=0).stop 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 msgr2=0x7f94e814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c490 tx=0x7f94dc00c950 comp rx=0 tx=0).stop 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 shutdown_connections 2024-03-06T07:31:01.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f94d406b670 0x7f94d406db30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94e8153550 0x7f94e80bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94e8151c80 0x7f94e814e210 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 --2- 172.21.15.17:0/985584039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f94e814f450 0x7f94e81eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 >> 172.21.15.17:0/985584039 conn(0x7f94e80c2e90 msgr2=0x7f94e8072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 shutdown_connections 2024-03-06T07:31:01.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:01.262+0000 7f94ee3a2640 1 -- 172.21.15.17:0/985584039 wait complete. 2024-03-06T07:31:01.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:01.681 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:02.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.784+0000 7fdfe7c71640 1 Processor -- start 2024-03-06T07:31:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.784+0000 7fdfe7c71640 1 -- start start 2024-03-06T07:31:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe7c71640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdfe01506b0 0x7fdfe0153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe7c71640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfe00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe7c71640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe0070800 con 0x7fdfe00acc00 2024-03-06T07:31:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe7c71640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe0156450 con 0x7fdfe01506b0 2024-03-06T07:31:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe7c71640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0156a10 con 0x7fdfe0156ea0 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe620e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdfe00acc00 0x7fdfe00c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43096/0 (socket says 172.21.15.17:43096) 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe620e640 1 -- 172.21.15.17:0/3236890066 learned_addr learned my addr 172.21.15.17:0/3236890066 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> v2:172.21.15.163:3300/0 conn(0x7fdfe01506b0 0x7fdfe0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.785+0000 7fdfe520c640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfe00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248938951 0 0) 0x7fdfe0070800 con 0x7fdfe00acc00 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfcc003520 con 0x7fdfe00acc00 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 882875646 0 0) 0x7fdfcc003520 con 0x7fdfe00acc00 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 msgr2=0x7fdfe00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfe00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 >> v2:172.21.15.163:3300/0 conn(0x7fdfe01506b0 msgr2=0x7fdfe0153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 --2- 172.21.15.17:0/3236890066 >> v2:172.21.15.163:3300/0 conn(0x7fdfe01506b0 0x7fdfe0153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0070aa0 con 0x7fdfe00acc00 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fdfd80019f0 con 0x7fdfe00acc00 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> v2:172.21.15.163:3300/0 conn(0x7fdfe01506b0 0x7fdfe0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 >> v1:172.21.15.17:6789/0 conn(0x7fdfe00acc00 legacy=0x7fdfe00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfcc0039b0 0x7fdfcc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfcc004800 0x7fdfcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0156450 con 0x7fdfcc0039b0 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0156a10 con 0x7fdfcc004800 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc00b9c0 con 0x7fdfe0156ea0 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfcc0039b0 0x7fdfcc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe620e640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe520c640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfcc004800 0x7fdfcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 -- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 msgr2=0x7fdfcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0156ea0 0x7fdfcc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 -- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfcc004800 msgr2=0x7fdfcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfcc004800 0x7fdfcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.786+0000 7fdfe5a0d640 1 -- 172.21.15.17:0/3236890066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc003520 con 0x7fdfcc0039b0 2024-03-06T07:31:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.787+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfcc0039b0 0x7fdfcc003df0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009c90 tx=0x7fdfcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27739c1a104d1948 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.787+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc039070 con 0x7fdfcc0039b0 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.787+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc02c980 con 0x7fdfcc0039b0 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.787+0000 7fdfe4a0b640 1 -- 172.21.15.17:0/3236890066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc009860 con 0x7fdfcc0039b0 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3236890066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfcc0039b0 msgr2=0x7fdfcc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3236890066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfcc0039b0 0x7fdfcc003df0 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009c90 tx=0x7fdfcc005830 comp rx=0 tx=0).stop 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe620e640 1 -- 172.21.15.17:0/3236890066 reap_dead start 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3236890066 shutdown_connections 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3236890066 >> 172.21.15.17:0/3236890066 conn(0x7fdfe006b1a0 msgr2=0x7fdfe006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3236890066 shutdown_connections 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.788+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3236890066 wait complete. 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.790+0000 7fdfe7c71640 1 Processor -- start 2024-03-06T07:31:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.790+0000 7fdfe7c71640 1 -- start start 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe01ec280 0x7fdfe014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00ab260 0x7fdfe014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00c5bb0 con 0x7fdfe014fc70 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe01566a0 con 0x7fdfe00ab260 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe7c71640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00c5d30 con 0x7fdfe01ec280 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47924/0 (socket says 172.21.15.17:47924) 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 -- 172.21.15.17:0/3589330773 learned_addr learned my addr 172.21.15.17:0/3589330773 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe01ec280 0x7fdfe014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe520c640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00ab260 0x7fdfe014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 -- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe01ec280 msgr2=0x7fdfe014deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe01ec280 0x7fdfe014deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 -- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00ab260 msgr2=0x7fdfe014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00ab260 0x7fdfe014f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.791+0000 7fdfe620e640 1 -- 172.21.15.17:0/3589330773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe014d4a0 con 0x7fdfe014fc70 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfe620e640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00735e0 tx=0x7fdfd80079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800aa20 con 0x7fdfe014fc70 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd8018070 con 0x7fdfe014fc70 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe01ecbc0 con 0x7fdfe014fc70 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800ae00 con 0x7fdfe014fc70 2024-03-06T07:31:02.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.792+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe0147560 con 0x7fdfe014fc70 2024-03-06T07:31:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.793+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe004f330 con 0x7fdfe014fc70 2024-03-06T07:31:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.793+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfd801d460 con 0x7fdfe014fc70 2024-03-06T07:31:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.793+0000 7fdfd67fc640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 0x7fdfc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.794+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdfd808a7b0 con 0x7fdfe014fc70 2024-03-06T07:31:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.794+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 0x7fdfc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:02.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.794+0000 7fdfe5a0d640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 0x7fdfc406d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc036000 tx=0x7fdfdc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:02.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:02.798+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdfd8059430 con 0x7fdfe014fc70 2024-03-06T07:31:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.043+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe014dc10 con 0x7fdfe014fc70 2024-03-06T07:31:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.044+0000 7fdfd67fc640 1 -- 172.21.15.17:0/3589330773 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7fdfd8058d30 con 0x7fdfe014fc70 2024-03-06T07:31:03.046 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:03.047 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 msgr2=0x7fdfc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 0x7fdfc406d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc036000 tx=0x7fdfdc030000 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 msgr2=0x7fdfe0146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fdfe00735e0 tx=0x7fdfd80079d0 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 shutdown_connections 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfc406b4d0 0x7fdfc406d990 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfe014fc70 0x7fdfe0146a10 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00ab260 0x7fdfe014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 --2- 172.21.15.17:0/3589330773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe01ec280 0x7fdfe014deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 >> 172.21.15.17:0/3589330773 conn(0x7fdfe006b1a0 msgr2=0x7fdfe00724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 shutdown_connections 2024-03-06T07:31:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:03.046+0000 7fdfe7c71640 1 -- 172.21.15.17:0/3589330773 wait complete. 2024-03-06T07:31:03.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:03.461 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:04.462 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.561+0000 7f8125217640 1 Processor -- start 2024-03-06T07:31:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.561+0000 7f8125217640 1 -- start start 2024-03-06T07:31:04.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8125217640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81200c5480 0x7f8120155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8125217640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f812006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8125217640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81200c5cc0 con 0x7f81201586f0 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8125217640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812006df40 con 0x7f81200c5480 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8125217640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81201591c0 con 0x7f81200c5860 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81200c5480 0x7f8120155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81200c5480 0x7f8120155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47938/0 (socket says 172.21.15.17:47938) 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 -- 172.21.15.17:0/3404985067 learned_addr learned my addr 172.21.15.17:0/3404985067 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f8117fff640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f812006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 -- 172.21.15.17:0/3404985067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 msgr2=0x7f812006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f812006f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 -- 172.21.15.17:0/3404985067 >> v1:172.21.15.163:6789/0 conn(0x7f81201586f0 legacy=0x7f81200724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.562+0000 7f811ed9d640 1 -- 172.21.15.17:0/3404985067 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81201594a0 con 0x7f81200c5480 2024-03-06T07:31:04.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.563+0000 7f811ed9d640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81200c5480 0x7f8120155ba0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f8110009870 tx=0x7f811002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8a4d2d597b684a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.563+0000 7f811e59c640 1 -- 172.21.15.17:0/3404985067 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110035020 con 0x7f81200c5480 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.563+0000 7f811e59c640 1 -- 172.21.15.17:0/3404985067 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8110031640 con 0x7f81200c5480 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.563+0000 7f811e59c640 1 -- 172.21.15.17:0/3404985067 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811002d040 con 0x7f81200c5480 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 -- 172.21.15.17:0/3404985067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81200c5480 msgr2=0x7f8120155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81200c5480 0x7f8120155ba0 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7f8110009870 tx=0x7f811002b8f0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 -- 172.21.15.17:0/3404985067 shutdown_connections 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f812006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 --2- 172.21.15.17:0/3404985067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81200c5480 0x7f8120155ba0 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 -- 172.21.15.17:0/3404985067 >> 172.21.15.17:0/3404985067 conn(0x7f812006b1a0 msgr2=0x7f812006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 -- 172.21.15.17:0/3404985067 shutdown_connections 2024-03-06T07:31:04.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.564+0000 7f8125217640 1 -- 172.21.15.17:0/3404985067 wait complete. 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.565+0000 7f8125217640 1 Processor -- start 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.565+0000 7f8125217640 1 -- start start 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81200c5480 0x7f81201e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f81200abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120152a20 con 0x7f81201586f0 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120155680 con 0x7f81200c5480 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8125217640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8120155050 con 0x7f81200c5860 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47952/0 (socket says 172.21.15.17:47952) 2024-03-06T07:31:04.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 -- 172.21.15.17:0/758052786 learned_addr learned my addr 172.21.15.17:0/758052786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f8117fff640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f81200abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811ed9d640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81200c5480 0x7f81201e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 -- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 msgr2=0x7f81200abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f81200abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 -- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81200c5480 msgr2=0x7f81201e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81200c5480 0x7f81201e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.566+0000 7f811f59e640 1 -- 172.21.15.17:0/758052786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200aad80 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f811f59e640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f811800c730 tx=0x7f811800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118003b70 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118003d30 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118005310 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81201eed40 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.567+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81200c4870 con 0x7f81201586f0 2024-03-06T07:31:04.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.568+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81201ef260 con 0x7f81201586f0 2024-03-06T07:31:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.568+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81180054b0 con 0x7f81201586f0 2024-03-06T07:31:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.569+0000 7f81177fe640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 0x7f810006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.569+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f811808a120 con 0x7f81201586f0 2024-03-06T07:31:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.569+0000 7f811ed9d640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 0x7f810006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:04.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.570+0000 7f811ed9d640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 0x7f810006d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8110036000 tx=0x7f811003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:04.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.573+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81201ef260 con 0x7f81201586f0 2024-03-06T07:31:04.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.791+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81200c4a70 con 0x7f81201586f0 2024-03-06T07:31:04.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.792+0000 7f81177fe640 1 -- 172.21.15.17:0/758052786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f8118058750 con 0x7f81201586f0 2024-03-06T07:31:04.795 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:04.795 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:04.795 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 msgr2=0x7f810006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 0x7f810006d990 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8110036000 tx=0x7f811003c040 comp rx=0 tx=0).stop 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 msgr2=0x7f81200ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7f811800c730 tx=0x7f811800cbf0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 shutdown_connections 2024-03-06T07:31:04.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f810006b4d0 0x7f810006d990 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81201586f0 0x7f81200ac000 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f81200c5860 0x7f81200abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 --2- 172.21.15.17:0/758052786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f81200c5480 0x7f81201e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 >> 172.21.15.17:0/758052786 conn(0x7f812006b1a0 msgr2=0x7f812006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 shutdown_connections 2024-03-06T07:31:04.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:04.794+0000 7f8125217640 1 -- 172.21.15.17:0/758052786 wait complete. 2024-03-06T07:31:05.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:05.210 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:06.211 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:06.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.310+0000 7f5c48caf640 1 Processor -- start 2024-03-06T07:31:06.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.310+0000 7f5c48caf640 1 -- start start 2024-03-06T07:31:06.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c441506b0 0x7f5c44153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c44156ea0 0x7f5c440702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5c440acc00 0x7f5c440c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c44157320 con 0x7f5c440acc00 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c44156450 con 0x7f5c44156ea0 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c48caf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44156a10 con 0x7f5c441506b0 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5c440acc00 0x7f5c440c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5c440acc00 0x7f5c440c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47954/0 (socket says 172.21.15.17:47954) 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 -- 172.21.15.17:0/2268181492 learned_addr learned my addr 172.21.15.17:0/2268181492 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c4259c640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c441506b0 0x7f5c44153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c3bfff640 1 --2- 172.21.15.17:0/2268181492 >> v2:172.21.15.163:3300/0 conn(0x7f5c44156ea0 0x7f5c440702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 -- 172.21.15.17:0/2268181492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c441506b0 msgr2=0x7f5c44153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c441506b0 0x7f5c44153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 -- 172.21.15.17:0/2268181492 >> v2:172.21.15.163:3300/0 conn(0x7f5c44156ea0 msgr2=0x7f5c440702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:06.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 --2- 172.21.15.17:0/2268181492 >> v2:172.21.15.163:3300/0 conn(0x7f5c44156ea0 0x7f5c440702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.311+0000 7f5c42d9d640 1 -- 172.21.15.17:0/2268181492 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44070800 con 0x7f5c440acc00 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c42d9d640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c440acc00 0x7f5c440c54a0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f5c440c62c0 tx=0x7f5c3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ffa8be0540aa2c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c41d9b640 1 -- 172.21.15.17:0/2268181492 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c02ca70 con 0x7f5c440acc00 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c41d9b640 1 -- 172.21.15.17:0/2268181492 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c02cc10 con 0x7f5c440acc00 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c41d9b640 1 -- 172.21.15.17:0/2268181492 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0343f0 con 0x7f5c440acc00 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 -- 172.21.15.17:0/2268181492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c440acc00 msgr2=0x7f5c440c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c440acc00 0x7f5c440c54a0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7f5c440c62c0 tx=0x7f5c3c02bed0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 -- 172.21.15.17:0/2268181492 shutdown_connections 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c440acc00 0x7f5c440c54a0 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 --2- 172.21.15.17:0/2268181492 >> v2:172.21.15.163:3300/0 conn(0x7f5c44156ea0 0x7f5c440702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 --2- 172.21.15.17:0/2268181492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c441506b0 0x7f5c44153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 -- 172.21.15.17:0/2268181492 >> 172.21.15.17:0/2268181492 conn(0x7f5c4406b1a0 msgr2=0x7f5c4406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 -- 172.21.15.17:0/2268181492 shutdown_connections 2024-03-06T07:31:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.313+0000 7f5c48caf640 1 -- 172.21.15.17:0/2268181492 wait complete. 2024-03-06T07:31:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.315+0000 7f5c48caf640 1 Processor -- start 2024-03-06T07:31:06.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.315+0000 7f5c48caf640 1 -- start start 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440acc00 0x7f5c441ec030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c44156ea0 0x7f5c44152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c441566a0 con 0x7f5c441506b0 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c44070d80 con 0x7f5c440acc00 2024-03-06T07:31:06.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c48caf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440ad050 con 0x7f5c44156ea0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c3bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c3bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47958/0 (socket says 172.21.15.17:47958) 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c3bfff640 1 -- 172.21.15.17:0/3801548409 learned_addr learned my addr 172.21.15.17:0/3801548409 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.316+0000 7f5c4259c640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440acc00 0x7f5c441ec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c42d9d640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c44156ea0 0x7f5c44152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 -- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c44156ea0 msgr2=0x7f5c44152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c44156ea0 0x7f5c44152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 -- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440acc00 msgr2=0x7f5c441ec030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440acc00 0x7f5c441ec030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 -- 172.21.15.17:0/3801548409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44150e90 con 0x7f5c441506b0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3bfff640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00e870 tx=0x7f5c2c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c017070 con 0x7f5c441506b0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00c940 con 0x7f5c441506b0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00cd30 con 0x7f5c441506b0 2024-03-06T07:31:06.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c44151170 con 0x7f5c441506b0 2024-03-06T07:31:06.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.317+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c44151730 con 0x7f5c441506b0 2024-03-06T07:31:06.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.318+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4404f330 con 0x7f5c441506b0 2024-03-06T07:31:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.319+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c2c01a070 con 0x7f5c441506b0 2024-03-06T07:31:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.319+0000 7f5c3affd640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 0x7f5c2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.319+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f5c2c08a5d0 con 0x7f5c441506b0 2024-03-06T07:31:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.319+0000 7f5c4259c640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 0x7f5c2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:06.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.320+0000 7f5c4259c640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 0x7f5c2406d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5c441ed0f0 tx=0x7f5c34009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:06.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.323+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5c2c058c00 con 0x7f5c441506b0 2024-03-06T07:31:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.542+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c440700a0 con 0x7f5c441506b0 2024-03-06T07:31:06.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.543+0000 7f5c3affd640 1 -- 172.21.15.17:0/3801548409 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+3428 (secure 0 0 0) 0x7f5c2c05cca0 con 0x7f5c441506b0 2024-03-06T07:31:06.546 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:06.546 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":217,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:30:35.900516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:06.546 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 217 2024-03-06T07:31:06.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 msgr2=0x7f5c2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 0x7f5c2406d990 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5c441ed0f0 tx=0x7f5c34009190 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 msgr2=0x7f5c44149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00e870 tx=0x7f5c2c00ed30 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 shutdown_connections 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f5c2406b4d0 0x7f5c2406d990 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c44156ea0 0x7f5c44152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c441506b0 0x7f5c44149020 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 --2- 172.21.15.17:0/3801548409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440acc00 0x7f5c441ec030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.544+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 >> 172.21.15.17:0/3801548409 conn(0x7f5c4406b1a0 msgr2=0x7f5c440c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.545+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 shutdown_connections 2024-03-06T07:31:06.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:06.545+0000 7f5c48caf640 1 -- 172.21.15.17:0/3801548409 wait complete. 2024-03-06T07:31:06.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 217, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:30:35.900516+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:06.961 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:07.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 Processor -- start 2024-03-06T07:31:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 -- start start 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0d014f360 0x7fa0d0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d00c73d0 con 0x7fa0d00c88c0 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d00be2e0 con 0x7fa0d0158b60 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.061+0000 7fa0d6f55640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d0159700 con 0x7fa0d014f360 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0d4cf1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa0d014f360 0x7fa0d0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0cffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa0d00c88c0 0x7fa0d00c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43112/0 (socket says 172.21.15.17:43112) 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0cffff640 1 -- 172.21.15.17:0/4191736850 learned_addr learned my addr 172.21.15.17:0/4191736850 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0d4cf1640 1 -- 172.21.15.17:0/4191736850 >> v1:172.21.15.163:6789/0 conn(0x7fa0d0158b60 legacy=0x7fa0d0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0d4cf1640 1 -- 172.21.15.17:0/4191736850 >> v1:172.21.15.17:6789/0 conn(0x7fa0d00c88c0 legacy=0x7fa0d00c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0d4cf1640 1 -- 172.21.15.17:0/4191736850 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d00c7670 con 0x7fa0d014f360 2024-03-06T07:31:08.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.062+0000 7fa0cf7fe640 1 -- 172.21.15.17:0/4191736850 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399038580 0 0) 0x7fa0d00c73d0 con 0x7fa0d00c88c0 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.063+0000 7fa0d4cf1640 1 --2- 172.21.15.17:0/4191736850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d014f360 0x7fa0d0155c10 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40097a0 tx=0x7fa0c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b682f797c9ebb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.063+0000 7fa0cf7fe640 1 -- 172.21.15.17:0/4191736850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4035020 con 0x7fa0d014f360 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.063+0000 7fa0cf7fe640 1 -- 172.21.15.17:0/4191736850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4031830 con 0x7fa0d014f360 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.063+0000 7fa0cf7fe640 1 -- 172.21.15.17:0/4191736850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4031c00 con 0x7fa0d014f360 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 -- 172.21.15.17:0/4191736850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d014f360 msgr2=0x7fa0d0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/4191736850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d014f360 0x7fa0d0155c10 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40097a0 tx=0x7fa0c402b8d0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 -- 172.21.15.17:0/4191736850 shutdown_connections 2024-03-06T07:31:08.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/4191736850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d014f360 0x7fa0d0155c10 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 -- 172.21.15.17:0/4191736850 >> 172.21.15.17:0/4191736850 conn(0x7fa0d00ac570 msgr2=0x7fa0d00ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:08.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 -- 172.21.15.17:0/4191736850 shutdown_connections 2024-03-06T07:31:08.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.064+0000 7fa0d6f55640 1 -- 172.21.15.17:0/4191736850 wait complete. 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.065+0000 7fa0d6f55640 1 Processor -- start 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.065+0000 7fa0d6f55640 1 -- start start 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0d00c88c0 0x7fa0d01eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0d014f360 0x7fa0d014c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:08.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c7c30 con 0x7fa0d0158b60 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c5a50 con 0x7fa0d00c88c0 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00c5bd0 con 0x7fa0d014f360 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47986/0 (socket says 172.21.15.17:47986) 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 -- 172.21.15.17:0/120690922 learned_addr learned my addr 172.21.15.17:0/120690922 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d4cf1640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0d00c88c0 0x7fa0d01eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0cffff640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0d014f360 0x7fa0d014c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 -- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0d014f360 msgr2=0x7fa0d014c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0d014f360 0x7fa0d014c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 -- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0d00c88c0 msgr2=0x7fa0d01eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0d00c88c0 0x7fa0d01eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 -- 172.21.15.17:0/120690922 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d01eb3b0 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d54f2640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800c6b0 tx=0x7fa0c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8003b70 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c8003d30 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.066+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d01eb690 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.067+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8005310 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.067+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d01ebc50 con 0x7fa0d0158b60 2024-03-06T07:31:08.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.068+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d004f330 con 0x7fa0d0158b60 2024-03-06T07:31:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.068+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0c80054b0 con 0x7fa0d0158b60 2024-03-06T07:31:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.069+0000 7fa0cd7fa640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.069+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fa0c808a260 con 0x7fa0d0158b60 2024-03-06T07:31:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.069+0000 7fa0d4cf1640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:08.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.069+0000 7fa0d4cf1640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 0x7fa0b406d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036000 tx=0x7fa0c4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:08.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.073+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa0c8058ee0 con 0x7fa0d0158b60 2024-03-06T07:31:08.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.290+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0d00c8d50 con 0x7fa0d0158b60 2024-03-06T07:31:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.290+0000 7fa0cd7fa640 1 -- 172.21.15.17:0/120690922 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fa0c80587e0 con 0x7fa0d0158b60 2024-03-06T07:31:08.293 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:08.293 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:08.293 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:08.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 msgr2=0x7fa0b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 0x7fa0b406d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036000 tx=0x7fa0c4008030 comp rx=0 tx=0).stop 2024-03-06T07:31:08.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 msgr2=0x7fa0d00afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:08.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800c6b0 tx=0x7fa0c800cb70 comp rx=0 tx=0).stop 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 shutdown_connections 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa0b406b4d0 0x7fa0b406d990 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0d0158b60 0x7fa0d00afc30 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0d014f360 0x7fa0d014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 --2- 172.21.15.17:0/120690922 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0d00c88c0 0x7fa0d01eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 >> 172.21.15.17:0/120690922 conn(0x7fa0d00ac570 msgr2=0x7fa0d00c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 shutdown_connections 2024-03-06T07:31:08.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:08.292+0000 7fa0d6f55640 1 -- 172.21.15.17:0/120690922 wait complete. 2024-03-06T07:31:08.709 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:08.709 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:09.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:09.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 Processor -- start 2024-03-06T07:31:09.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 -- start start 2024-03-06T07:31:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdea00c3280 0x7fdea00c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fdea0155630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea0158800 con 0x7fdea006d9f0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea0158ae0 con 0x7fdea00c3280 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.812+0000 7fdea7ec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0158dc0 con 0x7fdea01551e0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea5c65640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdea006d9f0 0x7fdea00a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43128/0 (socket says 172.21.15.17:43128) 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea5c65640 1 -- 172.21.15.17:0/1907235 learned_addr learned my addr 172.21.15.17:0/1907235 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea6466640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fdea0155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea5464640 1 --2- 172.21.15.17:0/1907235 >> v2:172.21.15.163:3300/0 conn(0x7fdea00c3280 0x7fdea00c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819868050 0 0) 0x7fdea0158800 con 0x7fdea006d9f0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde88003520 con 0x7fdea006d9f0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3209203354 0 0) 0x7fde88003520 con 0x7fdea006d9f0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 msgr2=0x7fdea0155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fdea0155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 >> v2:172.21.15.163:3300/0 conn(0x7fdea00c3280 msgr2=0x7fdea00c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 --2- 172.21.15.17:0/1907235 >> v2:172.21.15.163:3300/0 conn(0x7fdea00c3280 0x7fdea00c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea01590a0 con 0x7fdea006d9f0 2024-03-06T07:31:09.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fde900019f0 con 0x7fdea006d9f0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 >> v1:172.21.15.17:6789/0 conn(0x7fdea006d9f0 legacy=0x7fdea00a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde880039b0 0x7fde88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fde88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde88004d40 0x7fde880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0158ae0 con 0x7fde88004d40 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea0158dc0 con 0x7fde880039b0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8800b9c0 con 0x7fdea01551e0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea5c65640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde880039b0 0x7fde88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea6466640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde88004d40 0x7fde880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.813+0000 7fdea5464640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fde88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.814+0000 7fdea6466640 1 -- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 msgr2=0x7fde88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.814+0000 7fdea6466640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01551e0 0x7fde88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.814+0000 7fdea6466640 1 -- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde880039b0 msgr2=0x7fde88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:09.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.814+0000 7fdea6466640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde880039b0 0x7fde88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.814+0000 7fdea6466640 1 -- 172.21.15.17:0/1907235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88003520 con 0x7fde88004d40 2024-03-06T07:31:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.815+0000 7fdea6466640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde88004d40 0x7fde880051c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fde98009ac0 tx=0x7fde88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd2f7daffd21b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.815+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98039070 con 0x7fde88004d40 2024-03-06T07:31:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.815+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9802c6b0 con 0x7fde88004d40 2024-03-06T07:31:09.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.815+0000 7fdea4c63640 1 -- 172.21.15.17:0/1907235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98009690 con 0x7fde88004d40 2024-03-06T07:31:09.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 -- 172.21.15.17:0/1907235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde88004d40 msgr2=0x7fde880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/1907235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde88004d40 0x7fde880051c0 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fde98009ac0 tx=0x7fde88005830 comp rx=0 tx=0).stop 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea6466640 1 -- 172.21.15.17:0/1907235 reap_dead start 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 -- 172.21.15.17:0/1907235 shutdown_connections 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 -- 172.21.15.17:0/1907235 >> 172.21.15.17:0/1907235 conn(0x7fdea00aa630 msgr2=0x7fdea00aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 -- 172.21.15.17:0/1907235 shutdown_connections 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.816+0000 7fdea7ec9640 1 -- 172.21.15.17:0/1907235 wait complete. 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.817+0000 7fdea7ec9640 1 Processor -- start 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.818+0000 7fdea7ec9640 1 -- start start 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01ee1f0 0x7fdea01ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 0x7fdea01e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea006df50 con 0x7fdea014efd0 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea006ddd0 con 0x7fdea014f520 2024-03-06T07:31:09.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00c6910 con 0x7fdea01ee1f0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48004/0 (socket says 172.21.15.17:48004) 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 -- 172.21.15.17:0/249599337 learned_addr learned my addr 172.21.15.17:0/249599337 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5c65640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01ee1f0 0x7fdea01ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea6466640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 0x7fdea01e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 -- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01ee1f0 msgr2=0x7fdea01ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01ee1f0 0x7fdea01ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 -- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 msgr2=0x7fdea01e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 0x7fdea01e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 -- 172.21.15.17:0/249599337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea00ab7a0 con 0x7fdea014efd0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea6466640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 0x7fdea01e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea5464640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fde94011aa0 tx=0x7fde94011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400fac0 con 0x7fdea014efd0 2024-03-06T07:31:09.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400fc60 con 0x7fdea014efd0 2024-03-06T07:31:09.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94014440 con 0x7fdea014efd0 2024-03-06T07:31:09.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea01eec70 con 0x7fdea014efd0 2024-03-06T07:31:09.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.819+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea00703c0 con 0x7fdea014efd0 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.821+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea0070680 con 0x7fdea014efd0 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.821+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde94012040 con 0x7fdea014efd0 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.821+0000 7fde9e7fc640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 0x7fde8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.822+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fde9408ab00 con 0x7fdea014efd0 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.822+0000 7fdea5c65640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 0x7fde8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:09.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.822+0000 7fdea5c65640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 0x7fde8c06d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fde90004bc0 tx=0x7fde9000a0e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:09.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:09.825+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fde94019030 con 0x7fdea014efd0 2024-03-06T07:31:10.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.242+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdea004f330 con 0x7fdea014efd0 2024-03-06T07:31:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.242+0000 7fde9e7fc640 1 -- 172.21.15.17:0/249599337 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fde940596f0 con 0x7fdea014efd0 2024-03-06T07:31:10.244 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:10.244 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 msgr2=0x7fde8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 0x7fde8c06d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fde90004bc0 tx=0x7fde9000a0e0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 msgr2=0x7fdea01e19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fde94011aa0 tx=0x7fde94011ea0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 shutdown_connections 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fde8c06b4d0 0x7fde8c06d990 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdea014f520 0x7fdea01e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea014efd0 0x7fdea01e19a0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 --2- 172.21.15.17:0/249599337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdea01ee1f0 0x7fdea01ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:10.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 >> 172.21.15.17:0/249599337 conn(0x7fdea00aa630 msgr2=0x7fdea0154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:10.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 shutdown_connections 2024-03-06T07:31:10.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:10.245+0000 7fdea7ec9640 1 -- 172.21.15.17:0/249599337 wait complete. 2024-03-06T07:31:10.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:10.660 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:10.677 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:10.680 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:11.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:11.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.760+0000 7fcc4bafb640 1 Processor -- start 2024-03-06T07:31:11.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.760+0000 7fcc4bafb640 1 -- start start 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4bafb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc4414f360 0x7fcc44155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4bafb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcc440c88c0 0x7fcc440c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4bafb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc440c73d0 con 0x7fcc440c5880 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4bafb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc440c8dd0 con 0x7fcc440c88c0 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4bafb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc440be2e0 con 0x7fcc4414f360 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc49096640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcc440c88c0 0x7fcc440c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4a098640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcc440c5880 0x7fcc44158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34644/0 (socket says 172.21.15.17:34644) 2024-03-06T07:31:11.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc49096640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcc440c88c0 0x7fcc440c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34586/0 (socket says 172.21.15.17:34586) 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc4a098640 1 -- 172.21.15.17:0/130929127 learned_addr learned my addr 172.21.15.17:0/130929127 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.761+0000 7fcc49897640 1 --2- 172.21.15.17:0/130929127 >> v2:172.21.15.163:3300/0 conn(0x7fcc4414f360 0x7fcc44155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc49096640 1 -- 172.21.15.17:0/130929127 >> v2:172.21.15.163:3300/0 conn(0x7fcc4414f360 msgr2=0x7fcc44155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc49096640 1 --2- 172.21.15.17:0/130929127 >> v2:172.21.15.163:3300/0 conn(0x7fcc4414f360 0x7fcc44155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc49096640 1 -- 172.21.15.17:0/130929127 >> v1:172.21.15.17:6789/0 conn(0x7fcc440c5880 legacy=0x7fcc44158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc49096640 1 -- 172.21.15.17:0/130929127 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440c7670 con 0x7fcc440c88c0 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc49096640 1 --2- 172.21.15.17:0/130929127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc440c88c0 0x7fcc440c5340 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fcc4408c840 tx=0x7fcc3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5461958e53ef7ec server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc48895640 1 -- 172.21.15.17:0/130929127 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340315a0 con 0x7fcc440c88c0 2024-03-06T07:31:11.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc48895640 1 -- 172.21.15.17:0/130929127 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc34037070 con 0x7fcc440c88c0 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.762+0000 7fcc48895640 1 -- 172.21.15.17:0/130929127 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34031960 con 0x7fcc440c88c0 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 -- 172.21.15.17:0/130929127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc440c88c0 msgr2=0x7fcc440c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/130929127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc440c88c0 0x7fcc440c5340 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7fcc4408c840 tx=0x7fcc3402b840 comp rx=0 tx=0).stop 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 -- 172.21.15.17:0/130929127 shutdown_connections 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/130929127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc440c88c0 0x7fcc440c5340 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/130929127 >> v2:172.21.15.163:3300/0 conn(0x7fcc4414f360 0x7fcc44155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 -- 172.21.15.17:0/130929127 >> 172.21.15.17:0/130929127 conn(0x7fcc440ac570 msgr2=0x7fcc440ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 -- 172.21.15.17:0/130929127 shutdown_connections 2024-03-06T07:31:11.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.763+0000 7fcc4bafb640 1 -- 172.21.15.17:0/130929127 wait complete. 2024-03-06T07:31:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.764+0000 7fcc4bafb640 1 Processor -- start 2024-03-06T07:31:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.764+0000 7fcc4bafb640 1 -- start start 2024-03-06T07:31:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc440c5880 0x7fcc4414b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc440c88c0 0x7fcc4414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc44159270 con 0x7fcc4414f360 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc440c7c30 con 0x7fcc440c88c0 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4bafb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc441593f0 con 0x7fcc440c5880 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34590/0 (socket says 172.21.15.17:34590) 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 -- 172.21.15.17:0/791734783 learned_addr learned my addr 172.21.15.17:0/791734783 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc49096640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc440c88c0 0x7fcc4414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc49897640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc440c5880 0x7fcc4414b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 -- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc440c5880 msgr2=0x7fcc4414b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc440c5880 0x7fcc4414b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 -- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc440c88c0 msgr2=0x7fcc4414bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc440c88c0 0x7fcc4414bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.765+0000 7fcc4a098640 1 -- 172.21.15.17:0/791734783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440c1470 con 0x7fcc4414f360 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc4a098640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00cbc0 tx=0x7fcc3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c017070 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c00ad40 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc440af750 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c012600 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.766+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc441eecd0 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.767+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc440afa40 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.767+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc3c01a070 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.768+0000 7fcc3a7fc640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 0x7fcc2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.768+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fcc3c08a070 con 0x7fcc4414f360 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.768+0000 7fcc49897640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 0x7fcc2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:11.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.768+0000 7fcc49897640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 0x7fcc2806d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006fd0 tx=0x7fcc40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:11.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.772+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcc3c0586a0 con 0x7fcc4414f360 2024-03-06T07:31:11.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.992+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc4404f330 con 0x7fcc4414f360 2024-03-06T07:31:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.992+0000 7fcc3a7fc640 1 -- 172.21.15.17:0/791734783 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fcc3c05c740 con 0x7fcc4414f360 2024-03-06T07:31:11.994 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:11.994 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:11.994 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 msgr2=0x7fcc2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 0x7fcc2806d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006fd0 tx=0x7fcc40008040 comp rx=0 tx=0).stop 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 msgr2=0x7fcc441ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00cbc0 tx=0x7fcc3c00a430 comp rx=0 tx=0).stop 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 shutdown_connections 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcc2806b4d0 0x7fcc2806d990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc4414f360 0x7fcc441ee070 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc440c88c0 0x7fcc4414bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 --2- 172.21.15.17:0/791734783 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc440c5880 0x7fcc4414b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 >> 172.21.15.17:0/791734783 conn(0x7fcc440ac570 msgr2=0x7fcc440c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 shutdown_connections 2024-03-06T07:31:11.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:11.994+0000 7fcc4bafb640 1 -- 172.21.15.17:0/791734783 wait complete. 2024-03-06T07:31:12.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:12.408 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:13.410 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.510+0000 7f8ee2683640 1 Processor -- start 2024-03-06T07:31:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.510+0000 7f8ee2683640 1 -- start start 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee2683640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 0x7f8edc156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee2683640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8edc0acc00 0x7f8edc0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee2683640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc070aa0 con 0x7f8edc1506b0 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee2683640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc156450 con 0x7f8edc0acc00 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee2683640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8edc156a10 con 0x7f8edc153990 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8edbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8edc1506b0 0x7f8edc0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34660/0 (socket says 172.21.15.17:34660) 2024-03-06T07:31:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8edbfff640 1 -- 172.21.15.17:0/1872939223 learned_addr learned my addr 172.21.15.17:0/1872939223 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.511+0000 7f8ee0c20640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8edc0acc00 0x7f8edc0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1872939223 >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 msgr2=0x7f8edc156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8edb7fe640 1 --2- 172.21.15.17:0/1872939223 >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 0x7f8edc156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8ee0c20640 1 --2- 172.21.15.17:0/1872939223 >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 0x7f8edc156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8edaffd640 1 -- 172.21.15.17:0/1872939223 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014310928 0 0) 0x7f8edc070aa0 con 0x7f8edc1506b0 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1872939223 >> v1:172.21.15.17:6789/0 conn(0x7f8edc1506b0 legacy=0x7f8edc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1872939223 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc070d80 con 0x7f8edc0acc00 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8edb7fe640 1 --2- 172.21.15.17:0/1872939223 >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 0x7f8edc156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8ee0c20640 1 --2- 172.21.15.17:0/1872939223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c54a0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f8edc150a90 tx=0x7f8ed402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398096e96c65da47 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.512+0000 7f8edaffd640 1 -- 172.21.15.17:0/1872939223 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4039070 con 0x7f8edc0acc00 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8edaffd640 1 -- 172.21.15.17:0/1872939223 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed402cc80 con 0x7f8edc0acc00 2024-03-06T07:31:13.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8edaffd640 1 -- 172.21.15.17:0/1872939223 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4034660 con 0x7f8edc0acc00 2024-03-06T07:31:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 -- 172.21.15.17:0/1872939223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0acc00 msgr2=0x7f8edc0c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1872939223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c54a0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f8edc150a90 tx=0x7f8ed402bed0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 -- 172.21.15.17:0/1872939223 shutdown_connections 2024-03-06T07:31:13.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1872939223 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c54a0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1872939223 >> v2:172.21.15.163:3300/0 conn(0x7f8edc153990 0x7f8edc156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 -- 172.21.15.17:0/1872939223 >> 172.21.15.17:0/1872939223 conn(0x7f8edc06b1a0 msgr2=0x7f8edc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 -- 172.21.15.17:0/1872939223 shutdown_connections 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.513+0000 7f8ee2683640 1 -- 172.21.15.17:0/1872939223 wait complete. 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.515+0000 7f8ee2683640 1 Processor -- start 2024-03-06T07:31:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.515+0000 7f8ee2683640 1 -- start start 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 0x7f8edc0c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0c5b60 con 0x7f8edc153990 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0c59e0 con 0x7f8edc0acc00 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee2683640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc0c5ce0 con 0x7f8edc1506b0 2024-03-06T07:31:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee0c20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee0c20640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34616/0 (socket says 172.21.15.17:34616) 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1578339598 learned_addr learned my addr 172.21.15.17:0/1578339598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8edbfff640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.516+0000 7f8edb7fe640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 0x7f8edc0c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 msgr2=0x7f8edc0c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 0x7f8edc0c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0acc00 msgr2=0x7f8edc0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 -- 172.21.15.17:0/1578339598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0c4f00 con 0x7f8edc153990 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8edb7fe640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 0x7f8edc0c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee0c20640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009e80 tx=0x7f8ed4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4044020 con 0x7f8edc153990 2024-03-06T07:31:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed403a040 con 0x7f8edc153990 2024-03-06T07:31:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4039050 con 0x7f8edc153990 2024-03-06T07:31:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc1e8900 con 0x7f8edc153990 2024-03-06T07:31:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.517+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1e8ec0 con 0x7f8edc153990 2024-03-06T07:31:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.518+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc04f330 con 0x7f8edc153990 2024-03-06T07:31:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.518+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ed4003b90 con 0x7f8edc153990 2024-03-06T07:31:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.519+0000 7f8ed8ff9640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.519+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8ed40acbe0 con 0x7f8edc153990 2024-03-06T07:31:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.519+0000 7f8edbfff640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.520+0000 7f8edbfff640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000f60 tx=0x7f8ecc007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.523+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8ed404e020 con 0x7f8edc153990 2024-03-06T07:31:13.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.741+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8edc0700a0 con 0x7f8edc153990 2024-03-06T07:31:13.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.742+0000 7f8ed8ff9640 1 -- 172.21.15.17:0/1578339598 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f8ed407b860 con 0x7f8edc153990 2024-03-06T07:31:13.743 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:13.743 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:13.743 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:13.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 msgr2=0x7f8ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc000f60 tx=0x7f8ecc007d50 comp rx=0 tx=0).stop 2024-03-06T07:31:13.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 msgr2=0x7f8edc148ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009e80 tx=0x7f8ed4004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 shutdown_connections 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ebc06b4d0 0x7f8ebc06d990 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8edc153990 0x7f8edc148ce0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8edc1506b0 0x7f8edc0c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 --2- 172.21.15.17:0/1578339598 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8edc0acc00 0x7f8edc0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.743+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 >> 172.21.15.17:0/1578339598 conn(0x7f8edc06b1a0 msgr2=0x7f8edc14c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.744+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 shutdown_connections 2024-03-06T07:31:13.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:13.744+0000 7f8ee2683640 1 -- 172.21.15.17:0/1578339598 wait complete. 2024-03-06T07:31:14.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:14.162 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:15.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:15.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.263+0000 7f75d135c640 1 Processor -- start 2024-03-06T07:31:15.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.263+0000 7f75d135c640 1 -- start start 2024-03-06T07:31:15.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75d135c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f75cc158930 0x7f75cc1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75d135c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f75cc152c30 0x7f75cc158d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75d135c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0c2aa0 con 0x7f75cc159250 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75d135c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc0c3490 con 0x7f75cc158930 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75d135c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc0c3770 con 0x7f75cc152c30 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75caffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f75cc158930 0x7f75cc1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75cb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f75cc159250 0x7f75cc072950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:34670/0 (socket says 172.21.15.17:34670) 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75cb7fe640 1 -- 172.21.15.17:0/3567624004 learned_addr learned my addr 172.21.15.17:0/3567624004 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75ca7fc640 1 --2- 172.21.15.17:0/3567624004 >> v2:172.21.15.163:3300/0 conn(0x7f75cc152c30 0x7f75cc158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75caffd640 1 -- 172.21.15.17:0/3567624004 >> v2:172.21.15.163:3300/0 conn(0x7f75cc152c30 msgr2=0x7f75cc158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75caffd640 1 --2- 172.21.15.17:0/3567624004 >> v2:172.21.15.163:3300/0 conn(0x7f75cc152c30 0x7f75cc158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75caffd640 1 -- 172.21.15.17:0/3567624004 >> v1:172.21.15.17:6789/0 conn(0x7f75cc159250 legacy=0x7f75cc072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75c9ffb640 1 -- 172.21.15.17:0/3567624004 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497522970 0 0) 0x7f75cc0c2aa0 con 0x7f75cc159250 2024-03-06T07:31:15.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.264+0000 7f75caffd640 1 -- 172.21.15.17:0/3567624004 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0c2730 con 0x7f75cc158930 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.265+0000 7f75caffd640 1 --2- 172.21.15.17:0/3567624004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1526f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0c8890 tx=0x7f75c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8152c3c8bda138 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.265+0000 7f75c9ffb640 1 -- 172.21.15.17:0/3567624004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00315a0 con 0x7f75cc158930 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.265+0000 7f75c9ffb640 1 -- 172.21.15.17:0/3567624004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0037070 con 0x7f75cc158930 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.265+0000 7f75c9ffb640 1 -- 172.21.15.17:0/3567624004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0031960 con 0x7f75cc158930 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 -- 172.21.15.17:0/3567624004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 msgr2=0x7f75cc1526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 --2- 172.21.15.17:0/3567624004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1526f0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0c8890 tx=0x7f75c002b840 comp rx=0 tx=0).stop 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 -- 172.21.15.17:0/3567624004 shutdown_connections 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 --2- 172.21.15.17:0/3567624004 >> v2:172.21.15.163:3300/0 conn(0x7f75cc152c30 0x7f75cc158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 --2- 172.21.15.17:0/3567624004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1526f0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 -- 172.21.15.17:0/3567624004 >> 172.21.15.17:0/3567624004 conn(0x7f75cc0aa6a0 msgr2=0x7f75cc0aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:15.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 -- 172.21.15.17:0/3567624004 shutdown_connections 2024-03-06T07:31:15.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.266+0000 7f75d135c640 1 -- 172.21.15.17:0/3567624004 wait complete. 2024-03-06T07:31:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.267+0000 7f75d135c640 1 Processor -- start 2024-03-06T07:31:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.268+0000 7f75d135c640 1 -- start start 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75cc152c30 0x7f75cc14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 0x7f75cc14b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc159840 con 0x7f75cc158930 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1596c0 con 0x7f75cc159250 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc073060 con 0x7f75cc152c30 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34636/0 (socket says 172.21.15.17:34636) 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 -- 172.21.15.17:0/1554600473 learned_addr learned my addr 172.21.15.17:0/1554600473 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75caffd640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75cc152c30 0x7f75cc14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75cb7fe640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 0x7f75cc14b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 -- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75cc152c30 msgr2=0x7f75cc14c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75cc152c30 0x7f75cc14c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 -- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 msgr2=0x7f75cc14b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 0x7f75cc14b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 -- 172.21.15.17:0/1554600473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc1ef1b0 con 0x7f75cc158930 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75cb7fe640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 0x7f75cc14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75ca7fc640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00d800 tx=0x7f75bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00b6c0 con 0x7f75cc158930 2024-03-06T07:31:15.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc019070 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.269+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc1555c0 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.270+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc00ba80 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.270+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1eecb0 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.270+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc0c4a90 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.271+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75bc00bc20 con 0x7f75cc158930 2024-03-06T07:31:15.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.271+0000 7f75b37fe640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.271+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f75bc08a8f0 con 0x7f75cc158930 2024-03-06T07:31:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.272+0000 7f75caffd640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:15.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.272+0000 7f75caffd640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 0x7f75b806d990 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f75c0036000 tx=0x7f75c0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:15.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.276+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f75cc0c4a90 con 0x7f75cc158930 2024-03-06T07:31:15.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.494+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75cc1eef10 con 0x7f75cc158930 2024-03-06T07:31:15.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.494+0000 7f75b37fe640 1 -- 172.21.15.17:0/1554600473 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f75bc0594e0 con 0x7f75cc158930 2024-03-06T07:31:15.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:15.496 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:15.496 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:15.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 msgr2=0x7f75b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 0x7f75b806d990 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f75c0036000 tx=0x7f75c0008030 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 msgr2=0x7f75cc1ee770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00d800 tx=0x7f75bc00dcc0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 shutdown_connections 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75cc159250 0x7f75cc14b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75cc158930 0x7f75cc1ee770 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 --2- 172.21.15.17:0/1554600473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75cc152c30 0x7f75cc14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 >> 172.21.15.17:0/1554600473 conn(0x7f75cc0aa6a0 msgr2=0x7f75cc0c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 shutdown_connections 2024-03-06T07:31:15.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:15.497+0000 7f75d135c640 1 -- 172.21.15.17:0/1554600473 wait complete. 2024-03-06T07:31:15.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:15.915 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:16.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 Processor -- start 2024-03-06T07:31:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 -- start start 2024-03-06T07:31:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f218014f360 0x7f2180155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f21800c88c0 0x7f21800c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21800c73d0 con 0x7f21800c5880 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800c8dd0 con 0x7f21800c88c0 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f2186df6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21800be2e0 con 0x7f218014f360 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.017+0000 7f217ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f21800c88c0 0x7f21800c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f21800c88c0 0x7f21800c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34646/0 (socket says 172.21.15.17:34646) 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 -- 172.21.15.17:0/1338368967 learned_addr learned my addr 172.21.15.17:0/1338368967 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f2184b92640 1 --2- 172.21.15.17:0/1338368967 >> v2:172.21.15.163:3300/0 conn(0x7f218014f360 0x7f2180155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 -- 172.21.15.17:0/1338368967 >> v2:172.21.15.163:3300/0 conn(0x7f218014f360 msgr2=0x7f2180155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 --2- 172.21.15.17:0/1338368967 >> v2:172.21.15.163:3300/0 conn(0x7f218014f360 0x7f2180155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 -- 172.21.15.17:0/1338368967 >> v1:172.21.15.163:6789/0 conn(0x7f21800c5880 legacy=0x7f2180158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.018+0000 7f217ffff640 1 -- 172.21.15.17:0/1338368967 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800c7670 con 0x7f21800c88c0 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f217ffff640 1 --2- 172.21.15.17:0/1338368967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c88c0 0x7f21800c5340 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f218008c840 tx=0x7f217402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c17fe62661e5ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f217f7fe640 1 -- 172.21.15.17:0/1338368967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217402e070 con 0x7f21800c88c0 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f217f7fe640 1 -- 172.21.15.17:0/1338368967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2174037070 con 0x7f21800c88c0 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f217f7fe640 1 -- 172.21.15.17:0/1338368967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2174031a00 con 0x7f21800c88c0 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 -- 172.21.15.17:0/1338368967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c88c0 msgr2=0x7f21800c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 --2- 172.21.15.17:0/1338368967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c88c0 0x7f21800c5340 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f218008c840 tx=0x7f217402b880 comp rx=0 tx=0).stop 2024-03-06T07:31:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 -- 172.21.15.17:0/1338368967 shutdown_connections 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 --2- 172.21.15.17:0/1338368967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c88c0 0x7f21800c5340 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 --2- 172.21.15.17:0/1338368967 >> v2:172.21.15.163:3300/0 conn(0x7f218014f360 0x7f2180155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 -- 172.21.15.17:0/1338368967 >> 172.21.15.17:0/1338368967 conn(0x7f21800ac570 msgr2=0x7f21800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 -- 172.21.15.17:0/1338368967 shutdown_connections 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.019+0000 7f2186df6640 1 -- 172.21.15.17:0/1338368967 wait complete. 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.020+0000 7f2186df6640 1 Processor -- start 2024-03-06T07:31:17.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.020+0000 7f2186df6640 1 -- start start 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 0x7f21800aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 0x7f2180153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2180159270 con 0x7f21800c5880 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800c7c30 con 0x7f218014f360 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2186df6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21801593f0 con 0x7f21800c88c0 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2184b92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2184b92640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34656/0 (socket says 172.21.15.17:34656) 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.021+0000 7f2184b92640 1 -- 172.21.15.17:0/3801611857 learned_addr learned my addr 172.21.15.17:0/3801611857 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f217ffff640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 0x7f21800aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2185393640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 0x7f2180153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 -- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 msgr2=0x7f21800aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 0x7f21800aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 -- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 msgr2=0x7f2180153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 0x7f2180153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 -- 172.21.15.17:0/3801611857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800af380 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f217ffff640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 0x7f21800aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2185393640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 0x7f2180153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2184b92640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f2178007e40 tx=0x7f217800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178014620 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2178019070 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178014a00 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2180153330 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.022+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21800aff80 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.023+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218004f330 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.024+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2178014ba0 con 0x7f21800c5880 2024-03-06T07:31:17.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.024+0000 7f217d7fa640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.024+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f217808ad30 con 0x7f21800c5880 2024-03-06T07:31:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.024+0000 7f217ffff640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:17.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.025+0000 7f217ffff640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 0x7f216c06d990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2174036000 tx=0x7f217403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:17.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.032+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2178059920 con 0x7f21800c5880 2024-03-06T07:31:17.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.249+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f218006cda0 con 0x7f21800c5880 2024-03-06T07:31:17.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.249+0000 7f217d7fa640 1 -- 172.21.15.17:0/3801611857 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f2178059220 con 0x7f21800c5880 2024-03-06T07:31:17.251 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:17.251 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:17.252 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 msgr2=0x7f216c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 0x7f216c06d990 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2174036000 tx=0x7f217403d040 comp rx=0 tx=0).stop 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 msgr2=0x7f21801eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f2178007e40 tx=0x7f217800e4b0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 shutdown_connections 2024-03-06T07:31:17.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f216c06b4d0 0x7f216c06d990 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f218014f360 0x7f2180153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21800c88c0 0x7f21800aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 --2- 172.21.15.17:0/3801611857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21800c5880 0x7f21801eed00 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 >> 172.21.15.17:0/3801611857 conn(0x7f21800ac570 msgr2=0x7f218006c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 shutdown_connections 2024-03-06T07:31:17.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:17.251+0000 7f2186df6640 1 -- 172.21.15.17:0/3801611857 wait complete. 2024-03-06T07:31:17.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:17.666 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:18.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:18.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.767+0000 7f065cdac640 1 Processor -- start 2024-03-06T07:31:18.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.767+0000 7f065cdac640 1 -- start start 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f065cdac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06580c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f065cdac640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06581586f0 0x7f0658158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f065cdac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06581591c0 con 0x7f06580c5480 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f065cdac640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065806df40 con 0x7f06581586f0 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f065cdac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065806fdb0 con 0x7f0658155ba0 2024-03-06T07:31:18.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06581586f0 0x7f0658158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06581586f0 0x7f0658158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34658/0 (socket says 172.21.15.17:34658) 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 -- 172.21.15.17:0/3959116510 learned_addr learned my addr 172.21.15.17:0/3959116510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f064ffff640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06580c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 -- 172.21.15.17:0/3959116510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 msgr2=0x7f06580c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06580c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 -- 172.21.15.17:0/3959116510 >> v1:172.21.15.163:6789/0 conn(0x7f06580c5480 legacy=0x7f065806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.768+0000 7f0656d9d640 1 -- 172.21.15.17:0/3959116510 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06581594a0 con 0x7f06581586f0 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.769+0000 7f0656d9d640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f0658158b80 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f0650009e50 tx=0x7f065002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f83620597242c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.769+0000 7f0655d9b640 1 -- 172.21.15.17:0/3959116510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065002ca70 con 0x7f06581586f0 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.769+0000 7f0655d9b640 1 -- 172.21.15.17:0/3959116510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065002cc10 con 0x7f06581586f0 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.769+0000 7f0655d9b640 1 -- 172.21.15.17:0/3959116510 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06500343f0 con 0x7f06581586f0 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 -- 172.21.15.17:0/3959116510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 msgr2=0x7f0658158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f0658158b80 secure :-1 s=READY pgs=2532 cs=0 l=1 rev1=1 crypto rx=0x7f0650009e50 tx=0x7f065002bed0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 -- 172.21.15.17:0/3959116510 shutdown_connections 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f0658158b80 unknown :-1 s=CLOSED pgs=2532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 --2- 172.21.15.17:0/3959116510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06580c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 -- 172.21.15.17:0/3959116510 >> 172.21.15.17:0/3959116510 conn(0x7f065806b1a0 msgr2=0x7f065806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 -- 172.21.15.17:0/3959116510 shutdown_connections 2024-03-06T07:31:18.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.770+0000 7f065cdac640 1 -- 172.21.15.17:0/3959116510 wait complete. 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.771+0000 7f065cdac640 1 Processor -- start 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.771+0000 7f065cdac640 1 -- start start 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 0x7f06580ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06581e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658155680 con 0x7f06581586f0 2024-03-06T07:31:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658156000 con 0x7f06580c5480 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065cdac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0658152a20 con 0x7f0658155ba0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34674/0 (socket says 172.21.15.17:34674) 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 -- 172.21.15.17:0/2365201796 learned_addr learned my addr 172.21.15.17:0/2365201796 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f064ffff640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06581e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065659c640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 0x7f06580ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 -- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 msgr2=0x7f06581e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06581e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 -- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 msgr2=0x7f06580ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 0x7f06580ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f0656d9d640 1 -- 172.21.15.17:0/2365201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580c4940 con 0x7f06581586f0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f064ffff640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06581e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.772+0000 7f065659c640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 0x7f06580ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f0656d9d640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0650009e50 tx=0x7f0650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650044020 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650040670 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06581ef470 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650009510 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.773+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06581e20a0 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.774+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065806d4a0 con 0x7f06581586f0 2024-03-06T07:31:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.774+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0650047020 con 0x7f06581586f0 2024-03-06T07:31:18.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.775+0000 7f064effd640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 0x7f063806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:18.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.775+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f06500ace30 con 0x7f06581586f0 2024-03-06T07:31:18.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.775+0000 7f065659c640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 0x7f063806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:18.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.775+0000 7f065659c640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 0x7f063806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0648001680 tx=0x7f0648009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:18.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:18.783+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f065806d4a0 con 0x7f06581586f0 2024-03-06T07:31:19.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.002+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f065804f330 con 0x7f06581586f0 2024-03-06T07:31:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.003+0000 7f064effd640 1 -- 172.21.15.17:0/2365201796 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f065004e030 con 0x7f06581586f0 2024-03-06T07:31:19.005 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:19.005 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:19.005 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 msgr2=0x7f063806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 0x7f063806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0648001680 tx=0x7f0648009040 comp rx=0 tx=0).stop 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 msgr2=0x7f06581ebcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7f0650009e50 tx=0x7f0650004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 shutdown_connections 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f063806b4d0 0x7f063806d990 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06581586f0 0x7f06581ebcd0 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0658155ba0 0x7f06581e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 --2- 172.21.15.17:0/2365201796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06580c5480 0x7f06580ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 >> 172.21.15.17:0/2365201796 conn(0x7f065806b1a0 msgr2=0x7f0658152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 shutdown_connections 2024-03-06T07:31:19.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:19.005+0000 7f065cdac640 1 -- 172.21.15.17:0/2365201796 wait complete. 2024-03-06T07:31:19.422 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:19.422 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:20.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.526+0000 7f31e49ac640 1 Processor -- start 2024-03-06T07:31:20.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 -- start start 2024-03-06T07:31:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 0x7f31e0155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 0x7f31e0158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e00c5cc0 con 0x7f31e00c5860 2024-03-06T07:31:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e006df40 con 0x7f31e00c5480 2024-03-06T07:31:20.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.527+0000 7f31e49ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e006fdb0 con 0x7f31e01586f0 2024-03-06T07:31:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31de59c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 0x7f31e0155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ded9d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 0x7f31e0158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ddd9b640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f31e00c5860 0x7f31e006f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:37088/0 (socket says 172.21.15.17:37088) 2024-03-06T07:31:20.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ddd9b640 1 -- 172.21.15.17:0/3862443819 learned_addr learned my addr 172.21.15.17:0/3862443819 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ded9d640 1 -- 172.21.15.17:0/3862443819 >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 msgr2=0x7f31e0155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ded9d640 1 --2- 172.21.15.17:0/3862443819 >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 0x7f31e0155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ded9d640 1 -- 172.21.15.17:0/3862443819 >> v1:172.21.15.163:6789/0 conn(0x7f31e00c5860 legacy=0x7f31e006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31ded9d640 1 -- 172.21.15.17:0/3862443819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e01591c0 con 0x7f31e01586f0 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.528+0000 7f31de59c640 1 --2- 172.21.15.17:0/3862443819 >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 0x7f31e0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.529+0000 7f31ded9d640 1 --2- 172.21.15.17:0/3862443819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 0x7f31e0158b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009d90 tx=0x7f31d802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78ad5cf198e8ae3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.529+0000 7f31dd59a640 1 -- 172.21.15.17:0/3862443819 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8039070 con 0x7f31e01586f0 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.529+0000 7f31dd59a640 1 -- 172.21.15.17:0/3862443819 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d802cbc0 con 0x7f31e01586f0 2024-03-06T07:31:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.529+0000 7f31dd59a640 1 -- 172.21.15.17:0/3862443819 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8034440 con 0x7f31e01586f0 2024-03-06T07:31:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 -- 172.21.15.17:0/3862443819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 msgr2=0x7f31e0158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 --2- 172.21.15.17:0/3862443819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 0x7f31e0158b80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009d90 tx=0x7f31d802be10 comp rx=0 tx=0).stop 2024-03-06T07:31:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 -- 172.21.15.17:0/3862443819 shutdown_connections 2024-03-06T07:31:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 --2- 172.21.15.17:0/3862443819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e01586f0 0x7f31e0158b80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 --2- 172.21.15.17:0/3862443819 >> v2:172.21.15.163:3300/0 conn(0x7f31e00c5480 0x7f31e0155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 -- 172.21.15.17:0/3862443819 >> 172.21.15.17:0/3862443819 conn(0x7f31e006b1a0 msgr2=0x7f31e006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 -- 172.21.15.17:0/3862443819 shutdown_connections 2024-03-06T07:31:20.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.530+0000 7f31e49ac640 1 -- 172.21.15.17:0/3862443819 wait complete. 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.531+0000 7f31e49ac640 1 Processor -- start 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.531+0000 7f31e49ac640 1 -- start start 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e00c5860 0x7f31e0156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31e01586f0 0x7f31e0156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0155680 con 0x7f31e00c5480 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e01553a0 con 0x7f31e01586f0 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31e49ac640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e0152a20 con 0x7f31e00c5860 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40572/0 (socket says 172.21.15.17:40572) 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 -- 172.21.15.17:0/485383804 learned_addr learned my addr 172.21.15.17:0/485383804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:20.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31ded9d640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31e01586f0 0x7f31e0156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31ddd9b640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e00c5860 0x7f31e0156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 -- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e00c5860 msgr2=0x7f31e0156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e00c5860 0x7f31e0156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 -- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31e01586f0 msgr2=0x7f31e0156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31e01586f0 0x7f31e0156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.532+0000 7f31de59c640 1 -- 172.21.15.17:0/485383804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e01eb900 con 0x7f31e00c5480 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31de59c640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f31d000ebe0 tx=0x7f31d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0017070 con 0x7f31e00c5480 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d000cc30 con 0x7f31e00c5480 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0013470 con 0x7f31e00c5480 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e01ebbe0 con 0x7f31e00c5480 2024-03-06T07:31:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.533+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e00c4d60 con 0x7f31e00c5480 2024-03-06T07:31:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.534+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e004f330 con 0x7f31e00c5480 2024-03-06T07:31:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.534+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31d001a070 con 0x7f31e00c5480 2024-03-06T07:31:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.535+0000 7f31ceffd640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.535+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f31d008a6d0 con 0x7f31e00c5480 2024-03-06T07:31:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.535+0000 7f31ddd9b640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.536+0000 7f31ddd9b640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 0x7f31c006d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f31e00c5080 tx=0x7f31d4007e80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:20.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.539+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31d0058c70 con 0x7f31e00c5480 2024-03-06T07:31:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.971+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31e006e0c0 con 0x7f31e00c5480 2024-03-06T07:31:20.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.971+0000 7f31ceffd640 1 -- 172.21.15.17:0/485383804 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f31d005cd10 con 0x7f31e00c5480 2024-03-06T07:31:20.973 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:20.973 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 msgr2=0x7f31c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 0x7f31c006d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f31e00c5080 tx=0x7f31d4007e80 comp rx=0 tx=0).stop 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 msgr2=0x7f31e00c4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f31d000ebe0 tx=0x7f31d000c520 comp rx=0 tx=0).stop 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 shutdown_connections 2024-03-06T07:31:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f31c006b4d0 0x7f31c006d990 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f31e01586f0 0x7f31e0156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f31e00c5860 0x7f31e0156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 --2- 172.21.15.17:0/485383804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31e00c5480 0x7f31e00c4820 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 >> 172.21.15.17:0/485383804 conn(0x7f31e006b1a0 msgr2=0x7f31e0070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 shutdown_connections 2024-03-06T07:31:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:20.973+0000 7f31e49ac640 1 -- 172.21.15.17:0/485383804 wait complete. 2024-03-06T07:31:21.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:21.388 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:22.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:22.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.490+0000 7f358a49d640 1 Processor -- start 2024-03-06T07:31:22.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.490+0000 7f358a49d640 1 -- start start 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f358a49d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35841506b0 0x7f3584153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f358a49d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35840702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f358a49d640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3584070800 con 0x7f35840acc00 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f358a49d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3584156450 con 0x7f35841506b0 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f358a49d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3584156a10 con 0x7f3584156ea0 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35841506b0 0x7f3584153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f35841506b0 0x7f3584153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40582/0 (socket says 172.21.15.17:40582) 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 -- 172.21.15.17:0/2057139268 learned_addr learned my addr 172.21.15.17:0/2057139268 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f35837fe640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35840702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 -- 172.21.15.17:0/2057139268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 msgr2=0x7f35840702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35840702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.491+0000 7f3583fff640 1 -- 172.21.15.17:0/2057139268 >> v1:172.21.15.163:6789/0 conn(0x7f35840acc00 legacy=0x7f35840c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.492+0000 7f3583fff640 1 -- 172.21.15.17:0/2057139268 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584070aa0 con 0x7f35841506b0 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.492+0000 7f3583fff640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35841506b0 0x7f3584153990 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f35841565d0 tx=0x7f356c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0c739706c26a44 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.492+0000 7f3582ffd640 1 -- 172.21.15.17:0/2057139268 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c034070 con 0x7f35841506b0 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.492+0000 7f3582ffd640 1 -- 172.21.15.17:0/2057139268 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c030840 con 0x7f35841506b0 2024-03-06T07:31:22.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.492+0000 7f3582ffd640 1 -- 172.21.15.17:0/2057139268 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c030c10 con 0x7f35841506b0 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 -- 172.21.15.17:0/2057139268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35841506b0 msgr2=0x7f3584153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35841506b0 0x7f3584153990 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f35841565d0 tx=0x7f356c02b980 comp rx=0 tx=0).stop 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 -- 172.21.15.17:0/2057139268 shutdown_connections 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35840702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 --2- 172.21.15.17:0/2057139268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35841506b0 0x7f3584153990 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 -- 172.21.15.17:0/2057139268 >> 172.21.15.17:0/2057139268 conn(0x7f358406b1a0 msgr2=0x7f358406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 -- 172.21.15.17:0/2057139268 shutdown_connections 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.493+0000 7f358a49d640 1 -- 172.21.15.17:0/2057139268 wait complete. 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.494+0000 7f358a49d640 1 Processor -- start 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.494+0000 7f358a49d640 1 -- start start 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35841506b0 0x7f3584151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35841ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35840c5bb0 con 0x7f35840acc00 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35841566a0 con 0x7f35841506b0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f358a49d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35840c5d30 con 0x7f3584156ea0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40586/0 (socket says 172.21.15.17:40586) 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 -- 172.21.15.17:0/231837089 learned_addr learned my addr 172.21.15.17:0/231837089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f35837fe640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35841506b0 0x7f3584151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3588a3a640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35841ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 -- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 msgr2=0x7f35841ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35841ebb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 -- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35841506b0 msgr2=0x7f3584151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35841506b0 0x7f3584151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.495+0000 7f3583fff640 1 -- 172.21.15.17:0/231837089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584152c30 con 0x7f35840acc00 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f3583fff640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f356c02b9c0 tx=0x7f356c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c048070 con 0x7f35840acc00 2024-03-06T07:31:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c030640 con 0x7f35840acc00 2024-03-06T07:31:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3584151a90 con 0x7f35840acc00 2024-03-06T07:31:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c034040 con 0x7f35840acc00 2024-03-06T07:31:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.496+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3584155f10 con 0x7f35840acc00 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.497+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358404f330 con 0x7f35840acc00 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.497+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356c04b070 con 0x7f35840acc00 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.498+0000 7f3580ff9640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.498+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f356c0ac2f0 con 0x7f35840acc00 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.498+0000 7f35837fe640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.498+0000 7f35837fe640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 0x7f356406d9c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3574006fd0 tx=0x7f3574008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:22.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.502+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3584152c30 con 0x7f35840acc00 2024-03-06T07:31:22.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.777+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35840700a0 con 0x7f35840acc00 2024-03-06T07:31:22.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.778+0000 7f3580ff9640 1 -- 172.21.15.17:0/231837089 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f356c07af70 con 0x7f35840acc00 2024-03-06T07:31:22.780 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:22.780 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:22.780 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 msgr2=0x7f356406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 0x7f356406d9c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3574006fd0 tx=0x7f3574008040 comp rx=0 tx=0).stop 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 msgr2=0x7f35841526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f356c02b9c0 tx=0x7f356c004730 comp rx=0 tx=0).stop 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 shutdown_connections 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3584156ea0 0x7f35841ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35841506b0 0x7f3584151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 --2- 172.21.15.17:0/231837089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35840acc00 0x7f35841526f0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 >> 172.21.15.17:0/231837089 conn(0x7f358406b1a0 msgr2=0x7f35840c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 shutdown_connections 2024-03-06T07:31:22.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:22.780+0000 7f358a49d640 1 -- 172.21.15.17:0/231837089 wait complete. 2024-03-06T07:31:23.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:23.195 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:24.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.295+0000 7f68637b5640 1 Processor -- start 2024-03-06T07:31:24.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.296+0000 7f68637b5640 1 -- start start 2024-03-06T07:31:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f68637b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c155ba0 0x7f685c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f68637b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c1591c0 con 0x7f685c0c5480 2024-03-06T07:31:24.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f68637b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c1594a0 con 0x7f685c1586f0 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f68637b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06df40 con 0x7f685c155ba0 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6861551640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f685c0c5480 0x7f685c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54470/0 (socket says 172.21.15.17:54470) 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6861551640 1 -- 172.21.15.17:0/1392782424 learned_addr learned my addr 172.21.15.17:0/1392782424 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6860d50640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c155ba0 0x7f685c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461194809 0 0) 0x7f685c1591c0 con 0x7f685c0c5480 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6848003520 con 0x7f685c0c5480 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.297+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620141770 0 0) 0x7f685c1594a0 con 0x7f685c1586f0 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c1591c0 con 0x7f685c1586f0 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569020010 0 0) 0x7f6848003520 con 0x7f685c0c5480 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c155ba0 msgr2=0x7f685c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c155ba0 0x7f685c0c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 >> v1:172.21.15.163:6789/0 conn(0x7f685c1586f0 legacy=0x7f685c15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c1553a0 con 0x7f685c0c5480 2024-03-06T07:31:24.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6858001670 con 0x7f685c0c5480 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 >> v1:172.21.15.17:6789/0 conn(0x7f685c0c5480 legacy=0x7f685c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6848003b70 0x7f6848003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6848004800 0x7f6848004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68480051c0 0x7f684800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06df40 con 0x7f68480051c0 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c030 con 0x7f6848003b70 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c230 con 0x7f6848004800 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6860d50640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c155ba0 0x7f685c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68480051c0 0x7f684800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861551640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6848003b70 0x7f6848003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6860d50640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6848004800 0x7f6848004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 -- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6848004800 msgr2=0x7f6848004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6848004800 0x7f6848004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 -- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6848003b70 msgr2=0x7f6848003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6848003b70 0x7f6848003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.298+0000 7f6861d52640 1 -- 172.21.15.17:0/1392782424 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848003520 con 0x7f68480051c0 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f6861d52640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68480051c0 0x7f684800b9c0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6854001fd0 tx=0x7f6848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c0fb9eb28b7508 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854001bf0 con 0x7f68480051c0 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854003b70 con 0x7f68480051c0 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f6853fff640 1 -- 172.21.15.17:0/1392782424 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540050d0 con 0x7f68480051c0 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f68637b5640 1 -- 172.21.15.17:0/1392782424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68480051c0 msgr2=0x7f684800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f68637b5640 1 --2- 172.21.15.17:0/1392782424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68480051c0 0x7f684800b9c0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f6854001fd0 tx=0x7f6848005830 comp rx=0 tx=0).stop 2024-03-06T07:31:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f6861d52640 1 -- 172.21.15.17:0/1392782424 reap_dead start 2024-03-06T07:31:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f68637b5640 1 -- 172.21.15.17:0/1392782424 shutdown_connections 2024-03-06T07:31:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.299+0000 7f68637b5640 1 -- 172.21.15.17:0/1392782424 >> 172.21.15.17:0/1392782424 conn(0x7f685c06b1a0 msgr2=0x7f685c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.300+0000 7f68637b5640 1 -- 172.21.15.17:0/1392782424 shutdown_connections 2024-03-06T07:31:24.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.300+0000 7f68637b5640 1 -- 172.21.15.17:0/1392782424 wait complete. 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 Processor -- start 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 -- start start 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c1ee880 0x7f685c0ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 0x7f685c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c156000 con 0x7f685c152ec0 2024-03-06T07:31:24.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c06fdb0 con 0x7f685c1eec60 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.301+0000 7f68637b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c152a20 con 0x7f685c1ee880 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40598/0 (socket says 172.21.15.17:40598) 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 -- 172.21.15.17:0/4128410262 learned_addr learned my addr 172.21.15.17:0/4128410262 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6861d52640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 0x7f685c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6861551640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c1ee880 0x7f685c0ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 -- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c1ee880 msgr2=0x7f685c0ab560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c1ee880 0x7f685c0ab560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 -- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 msgr2=0x7f685c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 0x7f685c14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 -- 172.21.15.17:0/4128410262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c153870 con 0x7f685c152ec0 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6861d52640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 0x7f685c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6860d50640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f684c011830 tx=0x7f684c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:24.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00f9c0 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00fb60 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c1e1510 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.302+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c015460 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.303+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c153b10 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.304+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c04f330 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.304+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f684c015600 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.305+0000 7f6851ffb640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.305+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f684c08ae60 con 0x7f685c152ec0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.305+0000 7f6861551640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.306+0000 7f6861551640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 0x7f684006d9c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6858001eb0 tx=0x7f6858009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:24.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.309+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f684c019030 con 0x7f685c152ec0 2024-03-06T07:31:24.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.532+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685c06e0c0 con 0x7f685c152ec0 2024-03-06T07:31:24.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.533+0000 7f6851ffb640 1 -- 172.21.15.17:0/4128410262 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f684c059ae0 con 0x7f685c152ec0 2024-03-06T07:31:24.535 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:24.535 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:24.535 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 msgr2=0x7f684006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 0x7f684006d9c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6858001eb0 tx=0x7f6858009040 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 msgr2=0x7f685c14f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f684c011830 tx=0x7f684c011cf0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 shutdown_connections 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f684006b500 0x7f684006d9c0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c1eec60 0x7f685c14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c152ec0 0x7f685c14f750 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 --2- 172.21.15.17:0/4128410262 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f685c1ee880 0x7f685c0ab560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 >> 172.21.15.17:0/4128410262 conn(0x7f685c06b1a0 msgr2=0x7f685c0c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 shutdown_connections 2024-03-06T07:31:24.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:24.535+0000 7f68637b5640 1 -- 172.21.15.17:0/4128410262 wait complete. 2024-03-06T07:31:24.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:24.952 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:25.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:26.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 Processor -- start 2024-03-06T07:31:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 -- start start 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd73c14f360 0x7fd73c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c0c88c0 0x7fd73c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 0x7fd73c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73c0c8dd0 con 0x7fd73c14f360 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd73c0c73d0 con 0x7fd73c0c5880 2024-03-06T07:31:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.055+0000 7fd741b1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0be2e0 con 0x7fd73c0c88c0 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd73c14f360 0x7fd73c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd73c14f360 0x7fd73c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40608/0 (socket says 172.21.15.17:40608) 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 -- 172.21.15.17:0/3117852488 learned_addr learned my addr 172.21.15.17:0/3117852488 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73bfff640 1 --2- 172.21.15.17:0/3117852488 >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 0x7fd73c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73affd640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c0c88c0 0x7fd73c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 -- 172.21.15.17:0/3117852488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c0c88c0 msgr2=0x7fd73c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c0c88c0 0x7fd73c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 -- 172.21.15.17:0/3117852488 >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 msgr2=0x7fd73c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/3117852488 >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 0x7fd73c158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73b7fe640 1 -- 172.21.15.17:0/3117852488 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0c7670 con 0x7fd73c14f360 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.056+0000 7fd73bfff640 1 --2- 172.21.15.17:0/3117852488 >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 0x7fd73c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c14f360 0x7fd73c155c10 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fd73c08cc10 tx=0x7fd72c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38764f7b5cdfe96a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd73a7fc640 1 -- 172.21.15.17:0/3117852488 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c02e070 con 0x7fd73c14f360 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd73a7fc640 1 -- 172.21.15.17:0/3117852488 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c037070 con 0x7fd73c14f360 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd73a7fc640 1 -- 172.21.15.17:0/3117852488 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c031a00 con 0x7fd73c14f360 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 -- 172.21.15.17:0/3117852488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c14f360 msgr2=0x7fd73c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c14f360 0x7fd73c155c10 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fd73c08cc10 tx=0x7fd72c02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 -- 172.21.15.17:0/3117852488 shutdown_connections 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 --2- 172.21.15.17:0/3117852488 >> v2:172.21.15.163:3300/0 conn(0x7fd73c0c5880 0x7fd73c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c0c88c0 0x7fd73c0c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 --2- 172.21.15.17:0/3117852488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c14f360 0x7fd73c155c10 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 -- 172.21.15.17:0/3117852488 >> 172.21.15.17:0/3117852488 conn(0x7fd73c0ac570 msgr2=0x7fd73c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 -- 172.21.15.17:0/3117852488 shutdown_connections 2024-03-06T07:31:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.057+0000 7fd741b1e640 1 -- 172.21.15.17:0/3117852488 wait complete. 2024-03-06T07:31:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 Processor -- start 2024-03-06T07:31:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 -- start start 2024-03-06T07:31:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 0x7fd73c1ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c14f360 0x7fd73c1ec140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c1590a0 con 0x7fd73c0c5880 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c0c7c30 con 0x7fd73c0c88c0 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd741b1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c159220 con 0x7fd73c14f360 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.059+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40616/0 (socket says 172.21.15.17:40616) 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 -- 172.21.15.17:0/468568130 learned_addr learned my addr 172.21.15.17:0/468568130 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73affd640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 0x7fd73c1ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73bfff640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c14f360 0x7fd73c1ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 -- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c14f360 msgr2=0x7fd73c1ec140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c14f360 0x7fd73c1ec140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 -- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 msgr2=0x7fd73c1ebc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 0x7fd73c1ebc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 -- 172.21.15.17:0/468568130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c0bf0a0 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73affd640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 0x7fd73c1ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd73b7fe640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000c00 tx=0x7fd72c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c032880 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72c032a20 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c1ee9f0 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72c032e10 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.060+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c0bf620 con 0x7fd73c0c5880 2024-03-06T07:31:26.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.062+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c0beeb0 con 0x7fd73c0c5880 2024-03-06T07:31:26.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.062+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd72c04c060 con 0x7fd73c0c5880 2024-03-06T07:31:26.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.063+0000 7fd723fff640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 0x7fd72806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:26.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.063+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd72c0ac050 con 0x7fd73c0c5880 2024-03-06T07:31:26.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.063+0000 7fd73affd640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 0x7fd72806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:26.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.064+0000 7fd73affd640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 0x7fd72806d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0c53b0 tx=0x7fd730008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.067+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd72c040070 con 0x7fd73c0c5880 2024-03-06T07:31:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.286+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd73c04f330 con 0x7fd73c0c5880 2024-03-06T07:31:26.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.287+0000 7fd723fff640 1 -- 172.21.15.17:0/468568130 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fd72c04c350 con 0x7fd73c0c5880 2024-03-06T07:31:26.289 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:26.289 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:26.289 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 msgr2=0x7fd72806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 0x7fd72806d990 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0c53b0 tx=0x7fd730008040 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 msgr2=0x7fd73c1edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000c00 tx=0x7fd72c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 shutdown_connections 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd72806b4d0 0x7fd72806d990 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd73c14f360 0x7fd73c1ec140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd73c0c88c0 0x7fd73c1ebc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 --2- 172.21.15.17:0/468568130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd73c0c5880 0x7fd73c1edf70 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 >> 172.21.15.17:0/468568130 conn(0x7fd73c0ac570 msgr2=0x7fd73c153000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 shutdown_connections 2024-03-06T07:31:26.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:26.289+0000 7fd741b1e640 1 -- 172.21.15.17:0/468568130 wait complete. 2024-03-06T07:31:26.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:26.704 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:27.705 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:27.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.805+0000 7f4e4290c640 1 Processor -- start 2024-03-06T07:31:27.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.805+0000 7f4e4290c640 1 -- start start 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e4290c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c14f360 0x7f4e3c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e4290c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e4290c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3c0c73d0 con 0x7f4e3c0c5880 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e4290c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c8dd0 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e4290c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0be2e0 con 0x7f4e3c14f360 2024-03-06T07:31:27.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40618/0 (socket says 172.21.15.17:40618) 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 -- 172.21.15.17:0/687062487 learned_addr learned my addr 172.21.15.17:0/687062487 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 -- 172.21.15.17:0/687062487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c14f360 msgr2=0x7f4e3c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 --2- 172.21.15.17:0/687062487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c14f360 0x7f4e3c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 -- 172.21.15.17:0/687062487 >> v1:172.21.15.163:6789/0 conn(0x7f4e3c0c5880 legacy=0x7f4e3c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:27.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.806+0000 7f4e3b7fe640 1 -- 172.21.15.17:0/687062487 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c0c7670 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.807+0000 7f4e3b7fe640 1 --2- 172.21.15.17:0/687062487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009870 tx=0x7f4e2402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2881937f03ef4c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.807+0000 7f4e3affd640 1 -- 172.21.15.17:0/687062487 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e240317c0 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.807+0000 7f4e3affd640 1 -- 172.21.15.17:0/687062487 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e24037070 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.807+0000 7f4e3affd640 1 -- 172.21.15.17:0/687062487 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e24031b80 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 -- 172.21.15.17:0/687062487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0c88c0 msgr2=0x7f4e3c0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 --2- 172.21.15.17:0/687062487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009870 tx=0x7f4e2402b860 comp rx=0 tx=0).stop 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 -- 172.21.15.17:0/687062487 shutdown_connections 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 --2- 172.21.15.17:0/687062487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c0c88c0 0x7f4e3c0c5340 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 --2- 172.21.15.17:0/687062487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c14f360 0x7f4e3c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 -- 172.21.15.17:0/687062487 >> 172.21.15.17:0/687062487 conn(0x7f4e3c0ac570 msgr2=0x7f4e3c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 -- 172.21.15.17:0/687062487 shutdown_connections 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.808+0000 7f4e4290c640 1 -- 172.21.15.17:0/687062487 wait complete. 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.809+0000 7f4e4290c640 1 Processor -- start 2024-03-06T07:31:27.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.809+0000 7f4e4290c640 1 -- start start 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e3c0c5880 0x7f4e3c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c0c88c0 0x7f4e3c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c159270 con 0x7f4e3c14f360 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c7c30 con 0x7f4e3c0c5880 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1593f0 con 0x7f4e3c0c88c0 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40624/0 (socket says 172.21.15.17:40624) 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 -- 172.21.15.17:0/1172735002 learned_addr learned my addr 172.21.15.17:0/1172735002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:27.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e3b7fe640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c0c88c0 0x7f4e3c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e3bfff640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e3c0c5880 0x7f4e3c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 -- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c0c88c0 msgr2=0x7f4e3c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c0c88c0 0x7f4e3c157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 -- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e3c0c5880 msgr2=0x7f4e3c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e3c0c5880 0x7f4e3c0afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 -- 172.21.15.17:0/1172735002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c1eac10 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e40ea9640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400c7b0 tx=0x7f4e3400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34017070 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e34003da0 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c150580 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340052b0 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.810+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1eb1b0 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.812+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c04f330 con 0x7f4e3c14f360 2024-03-06T07:31:27.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.812+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e34005450 con 0x7f4e3c14f360 2024-03-06T07:31:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.812+0000 7f4e38ff9640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.812+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4e34089a70 con 0x7f4e3c14f360 2024-03-06T07:31:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.813+0000 7f4e3bfff640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:27.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.813+0000 7f4e3bfff640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c1ee0e0 tx=0x7f4e2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:27.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:27.821+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4e34058670 con 0x7f4e3c14f360 2024-03-06T07:31:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.042+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e3c0c69f0 con 0x7f4e3c14f360 2024-03-06T07:31:28.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.042+0000 7f4e38ff9640 1 -- 172.21.15.17:0/1172735002 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f4e34057f70 con 0x7f4e3c14f360 2024-03-06T07:31:28.044 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:28.045 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:28.045 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:28.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.044+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 msgr2=0x7f4e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.044+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c1ee0e0 tx=0x7f4e2c008040 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.044+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 msgr2=0x7f4e3c1ec080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.044+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400c7b0 tx=0x7f4e3400cc70 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 shutdown_connections 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4e1c06b4d0 0x7f4e1c06d990 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e3c14f360 0x7f4e3c1ec080 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e3c0c88c0 0x7f4e3c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 --2- 172.21.15.17:0/1172735002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e3c0c5880 0x7f4e3c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 >> 172.21.15.17:0/1172735002 conn(0x7f4e3c0ac570 msgr2=0x7f4e3c0c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 shutdown_connections 2024-03-06T07:31:28.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:28.045+0000 7f4e4290c640 1 -- 172.21.15.17:0/1172735002 wait complete. 2024-03-06T07:31:28.463 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:28.463 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:29.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.567+0000 7f3849ea3640 1 Processor -- start 2024-03-06T07:31:29.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.567+0000 7f3849ea3640 1 -- start start 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f3849ea3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f384414f310 0x7f38440c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f3849ea3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3844155bb0 con 0x7f38440aa180 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f3849ea3640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38440aa660 con 0x7f38441550c0 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f3849ea3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38440c4610 con 0x7f384414f310 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f3843fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f384414f310 0x7f38440c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f383bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f38440aa180 0x7f384414ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54472/0 (socket says 172.21.15.17:54472) 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.568+0000 7f383bfff640 1 -- 172.21.15.17:0/223164624 learned_addr learned my addr 172.21.15.17:0/223164624 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:29.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.569+0000 7f3843fff640 1 -- 172.21.15.17:0/223164624 >> v1:172.21.15.163:6789/0 conn(0x7f38441550c0 legacy=0x7f38441554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.569+0000 7f3843fff640 1 -- 172.21.15.17:0/223164624 >> v1:172.21.15.17:6789/0 conn(0x7f38440aa180 legacy=0x7f384414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.569+0000 7f3843fff640 1 -- 172.21.15.17:0/223164624 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38440aa940 con 0x7f384414f310 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.569+0000 7f3842ffd640 1 -- 172.21.15.17:0/223164624 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834129366 0 0) 0x7f3844155bb0 con 0x7f38440aa180 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.569+0000 7f3843fff640 1 --2- 172.21.15.17:0/223164624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f38440c5060 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f383c009d90 tx=0x7f383c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ca641905ba1655 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3842ffd640 1 -- 172.21.15.17:0/223164624 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c039070 con 0x7f384414f310 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3842ffd640 1 -- 172.21.15.17:0/223164624 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c02cb40 con 0x7f384414f310 2024-03-06T07:31:29.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3842ffd640 1 -- 172.21.15.17:0/223164624 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c034440 con 0x7f384414f310 2024-03-06T07:31:29.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 -- 172.21.15.17:0/223164624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 msgr2=0x7f38440c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 --2- 172.21.15.17:0/223164624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f38440c5060 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f383c009d90 tx=0x7f383c02bd90 comp rx=0 tx=0).stop 2024-03-06T07:31:29.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 -- 172.21.15.17:0/223164624 shutdown_connections 2024-03-06T07:31:29.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 --2- 172.21.15.17:0/223164624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f38440c5060 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 -- 172.21.15.17:0/223164624 >> 172.21.15.17:0/223164624 conn(0x7f38440c6710 msgr2=0x7f38440c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:29.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 -- 172.21.15.17:0/223164624 shutdown_connections 2024-03-06T07:31:29.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.570+0000 7f3849ea3640 1 -- 172.21.15.17:0/223164624 wait complete. 2024-03-06T07:31:29.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.571+0000 7f3849ea3640 1 Processor -- start 2024-03-06T07:31:29.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.572+0000 7f3849ea3640 1 -- start start 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38440aa180 0x7f38441510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38441550c0 0x7f38440702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440c1490 con 0x7f384414f310 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38440c1310 con 0x7f38440aa180 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38440c3210 con 0x7f38441550c0 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40640/0 (socket says 172.21.15.17:40640) 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 -- 172.21.15.17:0/1856388210 learned_addr learned my addr 172.21.15.17:0/1856388210 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:29.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f38437fe640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38440aa180 0x7f38441510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3843fff640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38441550c0 0x7f38440702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 -- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38441550c0 msgr2=0x7f38440702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38441550c0 0x7f38440702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 -- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38440aa180 msgr2=0x7f38441510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38440aa180 0x7f38441510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 -- 172.21.15.17:0/1856388210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844072150 con 0x7f384414f310 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f383bfff640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c4b0 tx=0x7f382c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c018070 con 0x7f384414f310 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c007500 con 0x7f384414f310 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3844151d60 con 0x7f384414f310 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c004a20 con 0x7f384414f310 2024-03-06T07:31:29.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.573+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3844072d00 con 0x7f384414f310 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.575+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384404f330 con 0x7f384414f310 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.575+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f382c01b070 con 0x7f384414f310 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.575+0000 7f3840ff9640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.575+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f382c0896c0 con 0x7f384414f310 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.576+0000 7f38437fe640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:29.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.576+0000 7f38437fe640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 0x7f382406d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:29.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.579+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f382c058400 con 0x7f384414f310 2024-03-06T07:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.990+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3844068450 con 0x7f384414f310 2024-03-06T07:31:29.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.990+0000 7f3840ff9640 1 -- 172.21.15.17:0/1856388210 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f382c057d00 con 0x7f384414f310 2024-03-06T07:31:29.992 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:29.993 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:29.993 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:29.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.992+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 msgr2=0x7f382406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.992+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 0x7f382406d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834008040 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 msgr2=0x7f3844151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c4b0 tx=0x7f382c00c970 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 shutdown_connections 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f382406b4d0 0x7f382406d990 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38441550c0 0x7f38440702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f384414f310 0x7f3844151610 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 --2- 172.21.15.17:0/1856388210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38440aa180 0x7f38441510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 >> 172.21.15.17:0/1856388210 conn(0x7f38440c6710 msgr2=0x7f38440c2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 shutdown_connections 2024-03-06T07:31:29.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:29.993+0000 7f3849ea3640 1 -- 172.21.15.17:0/1856388210 wait complete. 2024-03-06T07:31:30.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:30.408 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:31.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.510+0000 7fd6d553e640 1 Processor -- start 2024-03-06T07:31:31.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.510+0000 7fd6d553e640 1 -- start start 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6d553e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6d00c5330 0x7fd6d00c5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6d553e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6d0158db0 0x7fd6d015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6d553e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d0155110 con 0x7fd6d00c88b0 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6d553e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d01559e0 con 0x7fd6d00c5330 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6d553e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d00c8f40 con 0x7fd6d0158db0 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6d00c5330 0x7fd6d00c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd6d00c5330 0x7fd6d00c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43056/0 (socket says 172.21.15.17:43056) 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 -- 172.21.15.17:0/802882262 learned_addr learned my addr 172.21.15.17:0/802882262 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6cf7fe640 1 --2- 172.21.15.17:0/802882262 >> v2:172.21.15.163:3300/0 conn(0x7fd6d0158db0 0x7fd6d015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 -- 172.21.15.17:0/802882262 >> v2:172.21.15.163:3300/0 conn(0x7fd6d0158db0 msgr2=0x7fd6d015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 --2- 172.21.15.17:0/802882262 >> v2:172.21.15.163:3300/0 conn(0x7fd6d0158db0 0x7fd6d015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 -- 172.21.15.17:0/802882262 >> v1:172.21.15.163:6789/0 conn(0x7fd6d00c88b0 legacy=0x7fd6d01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.511+0000 7fd6ceffd640 1 -- 172.21.15.17:0/802882262 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d0155390 con 0x7fd6d00c5330 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.512+0000 7fd6ceffd640 1 --2- 172.21.15.17:0/802882262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d00c5710 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ca870 tx=0x7fd6c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866fd34d95761258 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.512+0000 7fd6ce7fc640 1 -- 172.21.15.17:0/802882262 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0035020 con 0x7fd6d00c5330 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.512+0000 7fd6ce7fc640 1 -- 172.21.15.17:0/802882262 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0031870 con 0x7fd6d00c5330 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.512+0000 7fd6ce7fc640 1 -- 172.21.15.17:0/802882262 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0031c40 con 0x7fd6d00c5330 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 -- 172.21.15.17:0/802882262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 msgr2=0x7fd6d00c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 --2- 172.21.15.17:0/802882262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d00c5710 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00ca870 tx=0x7fd6c002b910 comp rx=0 tx=0).stop 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 -- 172.21.15.17:0/802882262 shutdown_connections 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 --2- 172.21.15.17:0/802882262 >> v2:172.21.15.163:3300/0 conn(0x7fd6d0158db0 0x7fd6d015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 --2- 172.21.15.17:0/802882262 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d00c5710 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 -- 172.21.15.17:0/802882262 >> 172.21.15.17:0/802882262 conn(0x7fd6d00ac560 msgr2=0x7fd6d00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 -- 172.21.15.17:0/802882262 shutdown_connections 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.513+0000 7fd6d553e640 1 -- 172.21.15.17:0/802882262 wait complete. 2024-03-06T07:31:31.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.514+0000 7fd6d553e640 1 Processor -- start 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.514+0000 7fd6d553e640 1 -- start start 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6d00c88b0 0x7fd6d014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0151de0 0x7fd6d014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01529d0 con 0x7fd6d00c5330 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d006d090 con 0x7fd6d0151de0 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6d553e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0152b50 con 0x7fd6d00c88b0 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43064/0 (socket says 172.21.15.17:43064) 2024-03-06T07:31:31.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 -- 172.21.15.17:0/993724833 learned_addr learned my addr 172.21.15.17:0/993724833 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6c67fc640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6d00c88b0 0x7fd6d014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6cf7fe640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0151de0 0x7fd6d014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 -- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6d00c88b0 msgr2=0x7fd6d014bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6d00c88b0 0x7fd6d014bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 -- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0151de0 msgr2=0x7fd6d014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0151de0 0x7fd6d014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 -- 172.21.15.17:0/993724833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00c1a80 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.515+0000 7fd6ceffd640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fd6c002b950 tx=0x7fd6c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.516+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0040070 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.516+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0030d00 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.516+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0035040 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.516+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01eb430 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.516+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01506e0 con 0x7fd6d00c5330 2024-03-06T07:31:31.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.517+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d01501f0 con 0x7fd6d00c5330 2024-03-06T07:31:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.518+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c0038020 con 0x7fd6d00c5330 2024-03-06T07:31:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.518+0000 7fd6c7fff640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 0x7fd6b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.518+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd6c00ac250 con 0x7fd6d00c5330 2024-03-06T07:31:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.518+0000 7fd6c67fc640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 0x7fd6b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:31.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.519+0000 7fd6c67fc640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 0x7fd6b006d9c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00c1950 tx=0x7fd6b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:31.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.522+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd6c007aed0 con 0x7fd6d00c5330 2024-03-06T07:31:31.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.740+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6d006cdc0 con 0x7fd6d00c5330 2024-03-06T07:31:31.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.740+0000 7fd6c7fff640 1 -- 172.21.15.17:0/993724833 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fd6c007a7d0 con 0x7fd6d00c5330 2024-03-06T07:31:31.742 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:31.742 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:31.742 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:31.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 msgr2=0x7fd6b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 0x7fd6b006d9c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00c1950 tx=0x7fd6b8009040 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 msgr2=0x7fd6d0157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7fd6c002b950 tx=0x7fd6c00046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 shutdown_connections 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd6b006b500 0x7fd6b006d9c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6d0151de0 0x7fd6d014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6d00c88b0 0x7fd6d014bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 --2- 172.21.15.17:0/993724833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6d00c5330 0x7fd6d0157710 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 >> 172.21.15.17:0/993724833 conn(0x7fd6d00ac560 msgr2=0x7fd6d0154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 shutdown_connections 2024-03-06T07:31:31.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:31.742+0000 7fd6d553e640 1 -- 172.21.15.17:0/993724833 wait complete. 2024-03-06T07:31:32.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:32.157 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:33.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:33.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.258+0000 7f370bf55640 1 Processor -- start 2024-03-06T07:31:33.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.258+0000 7f370bf55640 1 -- start start 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37041527b0 0x7f3704159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37040c7180 0x7f37040c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3704155b20 0x7f37040c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3704159640 con 0x7f37040c7180 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040c8df0 con 0x7f3704155b20 2024-03-06T07:31:33.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370bf55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c7c60 con 0x7f37041527b0 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37040c7180 0x7f37040c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f37040c7180 0x7f37040c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43080/0 (socket says 172.21.15.17:43080) 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 -- 172.21.15.17:0/4015768000 learned_addr learned my addr 172.21.15.17:0/4015768000 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f3709cf1640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37041527b0 0x7f3704159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f370a4f2640 1 --2- 172.21.15.17:0/4015768000 >> v2:172.21.15.163:3300/0 conn(0x7f3704155b20 0x7f37040c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 -- 172.21.15.17:0/4015768000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37041527b0 msgr2=0x7f3704159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37041527b0 0x7f3704159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 -- 172.21.15.17:0/4015768000 >> v2:172.21.15.163:3300/0 conn(0x7f3704155b20 msgr2=0x7f37040c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 --2- 172.21.15.17:0/4015768000 >> v2:172.21.15.163:3300/0 conn(0x7f3704155b20 0x7f37040c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.259+0000 7f37094f0640 1 -- 172.21.15.17:0/4015768000 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040c8060 con 0x7f37040c7180 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.260+0000 7f37094f0640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37040c88b0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f370406b170 tx=0x7f36f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a6262a478f1fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.260+0000 7f3708cef640 1 -- 172.21.15.17:0/4015768000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f402e070 con 0x7f37040c7180 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.260+0000 7f3708cef640 1 -- 172.21.15.17:0/4015768000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f4037070 con 0x7f37040c7180 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.260+0000 7f3708cef640 1 -- 172.21.15.17:0/4015768000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4031a00 con 0x7f37040c7180 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 -- 172.21.15.17:0/4015768000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 msgr2=0x7f37040c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37040c88b0 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f370406b170 tx=0x7f36f402b880 comp rx=0 tx=0).stop 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 -- 172.21.15.17:0/4015768000 shutdown_connections 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 --2- 172.21.15.17:0/4015768000 >> v2:172.21.15.163:3300/0 conn(0x7f3704155b20 0x7f37040c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37040c88b0 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 --2- 172.21.15.17:0/4015768000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37041527b0 0x7f3704159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 -- 172.21.15.17:0/4015768000 >> 172.21.15.17:0/4015768000 conn(0x7f37040ac560 msgr2=0x7f37040ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 -- 172.21.15.17:0/4015768000 shutdown_connections 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.261+0000 7f370bf55640 1 -- 172.21.15.17:0/4015768000 wait complete. 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.262+0000 7f370bf55640 1 Processor -- start 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.262+0000 7f370bf55640 1 -- start start 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 0x7f37040c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 0x7f3704148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3704155320 con 0x7f37040c7180 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c7980 con 0x7f37041527b0 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704155600 con 0x7f3704155b20 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43088/0 (socket says 172.21.15.17:43088) 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 -- 172.21.15.17:0/405542948 learned_addr learned my addr 172.21.15.17:0/405542948 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f37094f0640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 0x7f37040c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370a4f2640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 0x7f3704148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 -- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 msgr2=0x7f3704148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 0x7f3704148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 -- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 msgr2=0x7f37040c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 0x7f37040c0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 -- 172.21.15.17:0/405542948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040be680 con 0x7f37040c7180 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f37094f0640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 0x7f37040c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370a4f2640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 0x7f3704148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f3709cf1640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d800 tx=0x7f36fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00b6c0 con 0x7f37040c7180 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc019070 con 0x7f37040c7180 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.263+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37040be920 con 0x7f37040c7180 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.264+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00ba80 con 0x7f37040c7180 2024-03-06T07:31:33.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.264+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37040beee0 con 0x7f37040c7180 2024-03-06T07:31:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.264+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370404f330 con 0x7f37040c7180 2024-03-06T07:31:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.265+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36fc00bc20 con 0x7f37040c7180 2024-03-06T07:31:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.265+0000 7f36fa7fc640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 0x7f36e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.265+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f36fc08a8b0 con 0x7f37040c7180 2024-03-06T07:31:33.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.266+0000 7f37094f0640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 0x7f36e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:33.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.266+0000 7f37094f0640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 0x7f36e806d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f36f4036000 tx=0x7f36f4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:33.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.270+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36fc059530 con 0x7f37040c7180 2024-03-06T07:31:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.490+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370406ce00 con 0x7f37040c7180 2024-03-06T07:31:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.491+0000 7f36fa7fc640 1 -- 172.21.15.17:0/405542948 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f36fc058e30 con 0x7f37040c7180 2024-03-06T07:31:33.492 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:33.493 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:33.493 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 msgr2=0x7f36e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 0x7f36e806d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f36f4036000 tx=0x7f36f4008030 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 msgr2=0x7f37041eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d800 tx=0x7f36fc00dcc0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 shutdown_connections 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36e806b4d0 0x7f36e806d990 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3704155b20 0x7f3704148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37041527b0 0x7f37040c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 --2- 172.21.15.17:0/405542948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37040c7180 0x7f37041eb4a0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 >> 172.21.15.17:0/405542948 conn(0x7f37040ac560 msgr2=0x7f3704152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 shutdown_connections 2024-03-06T07:31:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:33.493+0000 7f370bf55640 1 -- 172.21.15.17:0/405542948 wait complete. 2024-03-06T07:31:33.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:33.909 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:34.909 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:35.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.010+0000 7fdf9230c640 1 Processor -- start 2024-03-06T07:31:35.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.010+0000 7fdf9230c640 1 -- start start 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf9230c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdf8c153990 0x7fdf8c156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf9230c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8c070aa0 con 0x7fdf8c1506b0 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf9230c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8c070d80 con 0x7fdf8c0acc00 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf9230c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8c156450 con 0x7fdf8c153990 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdf8c153990 0x7fdf8c156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdf8c1506b0 0x7fdf8c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37336/0 (socket says 172.21.15.17:37336) 2024-03-06T07:31:35.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8bfff640 1 -- 172.21.15.17:0/916591834 learned_addr learned my addr 172.21.15.17:0/916591834 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8affd640 1 -- 172.21.15.17:0/916591834 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006429823 0 0) 0x7fdf8c070aa0 con 0x7fdf8c1506b0 2024-03-06T07:31:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8affd640 1 -- 172.21.15.17:0/916591834 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf74003520 con 0x7fdf8c1506b0 2024-03-06T07:31:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8b7fe640 1 -- 172.21.15.17:0/916591834 >> v1:172.21.15.163:6789/0 conn(0x7fdf8c0acc00 legacy=0x7fdf8c0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8b7fe640 1 -- 172.21.15.17:0/916591834 >> v1:172.21.15.17:6789/0 conn(0x7fdf8c1506b0 legacy=0x7fdf8c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.011+0000 7fdf8b7fe640 1 -- 172.21.15.17:0/916591834 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c1566a0 con 0x7fdf8c153990 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.012+0000 7fdf8b7fe640 1 --2- 172.21.15.17:0/916591834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c153990 0x7fdf8c156ea0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fdf800097a0 tx=0x7fdf8002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef28ef4514847d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.012+0000 7fdf8affd640 1 -- 172.21.15.17:0/916591834 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80034070 con 0x7fdf8c153990 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.012+0000 7fdf8affd640 1 -- 172.21.15.17:0/916591834 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80030610 con 0x7fdf8c153990 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.012+0000 7fdf8affd640 1 -- 172.21.15.17:0/916591834 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800309e0 con 0x7fdf8c153990 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 -- 172.21.15.17:0/916591834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c153990 msgr2=0x7fdf8c156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 --2- 172.21.15.17:0/916591834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c153990 0x7fdf8c156ea0 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fdf800097a0 tx=0x7fdf8002b960 comp rx=0 tx=0).stop 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 -- 172.21.15.17:0/916591834 shutdown_connections 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 --2- 172.21.15.17:0/916591834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c153990 0x7fdf8c156ea0 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 -- 172.21.15.17:0/916591834 >> 172.21.15.17:0/916591834 conn(0x7fdf8c06b1a0 msgr2=0x7fdf8c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:35.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 -- 172.21.15.17:0/916591834 shutdown_connections 2024-03-06T07:31:35.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.013+0000 7fdf9230c640 1 -- 172.21.15.17:0/916591834 wait complete. 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.014+0000 7fdf9230c640 1 Processor -- start 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.014+0000 7fdf9230c640 1 -- start start 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 0x7fdf8c1e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 0x7fdf8c1490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0ad000 con 0x7fdf8c0acc00 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c156a10 con 0x7fdf8c153990 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf9230c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0ad180 con 0x7fdf8c1506b0 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43110/0 (socket says 172.21.15.17:43110) 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 -- 172.21.15.17:0/1666430733 learned_addr learned my addr 172.21.15.17:0/1666430733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf908a9640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 0x7fdf8c1490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8b7fe640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 0x7fdf8c1e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 -- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 msgr2=0x7fdf8c1e8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 0x7fdf8c1e8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 -- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 msgr2=0x7fdf8c1490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 0x7fdf8c1490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 -- 172.21.15.17:0/1666430733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c1ecbb0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf908a9640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 0x7fdf8c1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8b7fe640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 0x7fdf8c1e8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.015+0000 7fdf8bfff640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c500 tx=0x7fdf7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.016+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c017070 con 0x7fdf8c0acc00 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.016+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c005ce0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.016+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c007330 con 0x7fdf8c0acc00 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.016+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c14aaf0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.016+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c149610 con 0x7fdf8c0acc00 2024-03-06T07:31:35.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.017+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c04f330 con 0x7fdf8c0acc00 2024-03-06T07:31:35.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.017+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf7c01a070 con 0x7fdf8c0acc00 2024-03-06T07:31:35.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.017+0000 7fdf88ff9640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 0x7fdf7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:35.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.018+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdf7c0898c0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.018+0000 7fdf8b7fe640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 0x7fdf7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:35.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.018+0000 7fdf8b7fe640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 0x7fdf7806d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdf80036000 tx=0x7fdf80008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:35.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.022+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdf8c1ecbb0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.240+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf8c0c4ee0 con 0x7fdf8c0acc00 2024-03-06T07:31:35.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.241+0000 7fdf88ff9640 1 -- 172.21.15.17:0/1666430733 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fdf7c058600 con 0x7fdf8c0acc00 2024-03-06T07:31:35.243 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:35.243 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:35.243 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 msgr2=0x7fdf7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 0x7fdf7806d990 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdf80036000 tx=0x7fdf80008030 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 msgr2=0x7fdf8c1528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c500 tx=0x7fdf7c00c9c0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 shutdown_connections 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdf7806b4d0 0x7fdf7806d990 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdf8c153990 0x7fdf8c1490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdf8c1506b0 0x7fdf8c1e8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 --2- 172.21.15.17:0/1666430733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf8c0acc00 0x7fdf8c1528c0 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:35.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 >> 172.21.15.17:0/1666430733 conn(0x7fdf8c06b1a0 msgr2=0x7fdf8c0c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:35.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 shutdown_connections 2024-03-06T07:31:35.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:35.243+0000 7fdf9230c640 1 -- 172.21.15.17:0/1666430733 wait complete. 2024-03-06T07:31:35.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:35.659 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:36.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.760+0000 7f0e76c6b640 1 Processor -- start 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.760+0000 7f0e76c6b640 1 -- start start 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e76c6b640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e7014f360 0x7f0e70155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e76c6b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e700c73d0 con 0x7f0e70158b60 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e76c6b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e700be2e0 con 0x7f0e700c88c0 2024-03-06T07:31:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e76c6b640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e70159700 con 0x7f0e7014f360 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e75208640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0e70158b60 0x7f0e70158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37346/0 (socket says 172.21.15.17:37346) 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e75208640 1 -- 172.21.15.17:0/111504585 learned_addr learned my addr 172.21.15.17:0/111504585 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e74a07640 1 --2- 172.21.15.17:0/111504585 >> v2:172.21.15.163:3300/0 conn(0x7f0e7014f360 0x7f0e70155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013685910 0 0) 0x7f0e700c73d0 con 0x7f0e70158b60 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e60003520 con 0x7f0e70158b60 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465228545 0 0) 0x7f0e700be2e0 con 0x7f0e700c88c0 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e700c73d0 con 0x7f0e700c88c0 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e74a07640 1 -- 172.21.15.17:0/111504585 >> v1:172.21.15.163:6789/0 conn(0x7f0e700c88c0 legacy=0x7f0e700c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857838975 0 0) 0x7f0e60003520 con 0x7f0e70158b60 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e74a07640 1 -- 172.21.15.17:0/111504585 >> v1:172.21.15.17:6789/0 conn(0x7f0e70158b60 legacy=0x7f0e70158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.761+0000 7f0e74a07640 1 -- 172.21.15.17:0/111504585 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700c7670 con 0x7f0e7014f360 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.762+0000 7f0e74a07640 1 --2- 172.21.15.17:0/111504585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70155c10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0e640097a0 tx=0x7f0e6402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8364b2ee861910 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.762+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64037070 con 0x7f0e7014f360 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.762+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e64030510 con 0x7f0e7014f360 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.762+0000 7f0e6f7fe640 1 -- 172.21.15.17:0/111504585 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680019f0 con 0x7f0e7014f360 2024-03-06T07:31:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 -- 172.21.15.17:0/111504585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 msgr2=0x7f0e70155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/111504585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70155c10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f0e640097a0 tx=0x7f0e6402b790 comp rx=0 tx=0).stop 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 -- 172.21.15.17:0/111504585 shutdown_connections 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/111504585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70155c10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 -- 172.21.15.17:0/111504585 >> 172.21.15.17:0/111504585 conn(0x7f0e700ac570 msgr2=0x7f0e700ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 -- 172.21.15.17:0/111504585 shutdown_connections 2024-03-06T07:31:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.763+0000 7f0e76c6b640 1 -- 172.21.15.17:0/111504585 wait complete. 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.764+0000 7f0e76c6b640 1 Processor -- start 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.764+0000 7f0e76c6b640 1 -- start start 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e70158b60 0x7f0e7014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e700c7c30 con 0x7f0e700c88c0 2024-03-06T07:31:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e700c5a50 con 0x7f0e7014f360 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e76c6b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e700c5bd0 con 0x7f0e70158b60 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43118/0 (socket says 172.21.15.17:43118) 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 -- 172.21.15.17:0/2356051251 learned_addr learned my addr 172.21.15.17:0/2356051251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e6ffff640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e75208640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e70158b60 0x7f0e7014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 -- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e70158b60 msgr2=0x7f0e7014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e70158b60 0x7f0e7014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 -- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 msgr2=0x7f0e70157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.765+0000 7f0e74a07640 1 -- 172.21.15.17:0/2356051251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700b0660 con 0x7f0e700c88c0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e74a07640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f0e6402b7d0 tx=0x7f0e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64042070 con 0x7f0e700c88c0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6403e890 con 0x7f0e700c88c0 2024-03-06T07:31:36.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e700b00a0 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64037040 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.766+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e701eafc0 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.767+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e701eadc0 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.767+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e64045070 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.768+0000 7f0e6d7fa640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.768+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f0e640ac170 con 0x7f0e700c88c0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.768+0000 7f0e6ffff640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:36.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.769+0000 7f0e6ffff640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 0x7f0e5406d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0011e0 tx=0x7f0e5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:36.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.772+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e700b0660 con 0x7f0e700c88c0 2024-03-06T07:31:36.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.994+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e7004f330 con 0x7f0e700c88c0 2024-03-06T07:31:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.994+0000 7f0e6d7fa640 1 -- 172.21.15.17:0/2356051251 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f0e6407ae80 con 0x7f0e700c88c0 2024-03-06T07:31:36.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:36.996 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 msgr2=0x7f0e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 0x7f0e5406d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0011e0 tx=0x7f0e5c009040 comp rx=0 tx=0).stop 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 msgr2=0x7f0e700c1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f0e6402b7d0 tx=0x7f0e640046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 shutdown_connections 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0e5406b4d0 0x7f0e5406d990 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e70158b60 0x7f0e7014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7014f360 0x7f0e70157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 --2- 172.21.15.17:0/2356051251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e700c88c0 0x7f0e700c1030 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 >> 172.21.15.17:0/2356051251 conn(0x7f0e700ac570 msgr2=0x7f0e7014dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 shutdown_connections 2024-03-06T07:31:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:36.996+0000 7f0e76c6b640 1 -- 172.21.15.17:0/2356051251 wait complete. 2024-03-06T07:31:37.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:37.412 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:38.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:38.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.512+0000 7f3bd350c640 1 Processor -- start 2024-03-06T07:31:38.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 -- start start 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bcc1527b0 0x7f3bcc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3bcc0c88b0 0x7f3bcc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc0c7c60 con 0x7f3bcc0c7180 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc159640 con 0x7f3bcc1527b0 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd350c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c8060 con 0x7f3bcc0c88b0 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd12a8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bcc1527b0 0x7f3bcc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd12a8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3bcc1527b0 0x7f3bcc159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43134/0 (socket says 172.21.15.17:43134) 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.513+0000 7f3bd12a8640 1 -- 172.21.15.17:0/4102871583 learned_addr learned my addr 172.21.15.17:0/4102871583 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd1aa9640 1 --2- 172.21.15.17:0/4102871583 >> v2:172.21.15.163:3300/0 conn(0x7f3bcc0c88b0 0x7f3bcc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd12a8640 1 -- 172.21.15.17:0/4102871583 >> v2:172.21.15.163:3300/0 conn(0x7f3bcc0c88b0 msgr2=0x7f3bcc0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:38.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd12a8640 1 --2- 172.21.15.17:0/4102871583 >> v2:172.21.15.163:3300/0 conn(0x7f3bcc0c88b0 0x7f3bcc0c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd12a8640 1 -- 172.21.15.17:0/4102871583 >> v1:172.21.15.163:6789/0 conn(0x7f3bcc0c7180 legacy=0x7f3bcc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd12a8640 1 -- 172.21.15.17:0/4102871583 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0c76a0 con 0x7f3bcc1527b0 2024-03-06T07:31:38.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.514+0000 7f3bd12a8640 1 --2- 172.21.15.17:0/4102871583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc159100 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0ad600 tx=0x7f3bc402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805fe17b2c1b6dc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bcbfff640 1 -- 172.21.15.17:0/4102871583 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc402ca80 con 0x7f3bcc1527b0 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bcbfff640 1 -- 172.21.15.17:0/4102871583 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc402cc20 con 0x7f3bcc1527b0 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bcbfff640 1 -- 172.21.15.17:0/4102871583 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40343f0 con 0x7f3bcc1527b0 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 -- 172.21.15.17:0/4102871583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc1527b0 msgr2=0x7f3bcc159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 --2- 172.21.15.17:0/4102871583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc159100 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0ad600 tx=0x7f3bc402bee0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 -- 172.21.15.17:0/4102871583 shutdown_connections 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 --2- 172.21.15.17:0/4102871583 >> v2:172.21.15.163:3300/0 conn(0x7f3bcc0c88b0 0x7f3bcc0c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 --2- 172.21.15.17:0/4102871583 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc159100 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 -- 172.21.15.17:0/4102871583 >> 172.21.15.17:0/4102871583 conn(0x7f3bcc0ac560 msgr2=0x7f3bcc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 -- 172.21.15.17:0/4102871583 shutdown_connections 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.515+0000 7f3bd350c640 1 -- 172.21.15.17:0/4102871583 wait complete. 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.516+0000 7f3bd350c640 1 Processor -- start 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.516+0000 7f3bd350c640 1 -- start start 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 0x7f3bcc153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc155600 con 0x7f3bcc0c7180 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc155320 con 0x7f3bcc1527b0 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd350c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc152b90 con 0x7f3bcc0c88b0 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43150/0 (socket says 172.21.15.17:43150) 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 -- 172.21.15.17:0/3737255039 learned_addr learned my addr 172.21.15.17:0/3737255039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd1aa9640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd0aa7640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 0x7f3bcc153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 -- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 msgr2=0x7f3bcc153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 0x7f3bcc153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 -- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 msgr2=0x7f3bcc153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd12a8640 1 -- 172.21.15.17:0/3737255039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0caab0 con 0x7f3bcc0c7180 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd1aa9640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.517+0000 7f3bd0aa7640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 0x7f3bcc153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bd12a8640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000f80 tx=0x7f3bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4043020 con 0x7f3bcc0c7180 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc403e450 con 0x7f3bcc0c7180 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0ab6c0 con 0x7f3bcc0c7180 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4009510 con 0x7f3bcc0c7180 2024-03-06T07:31:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.518+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc154e80 con 0x7f3bcc0c7180 2024-03-06T07:31:38.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.519+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc04f330 con 0x7f3bcc0c7180 2024-03-06T07:31:38.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.520+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bc4046020 con 0x7f3bcc0c7180 2024-03-06T07:31:38.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.520+0000 7f3bc9ffb640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:38.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.520+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f3bc40accc0 con 0x7f3bcc0c7180 2024-03-06T07:31:38.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.520+0000 7f3bd0aa7640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:38.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.521+0000 7f3bd0aa7640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 0x7f3bb806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc1ef190 tx=0x7f3bc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:38.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.525+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bc404d030 con 0x7f3bcc0c7180 2024-03-06T07:31:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.746+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bcc06ce00 con 0x7f3bcc0c7180 2024-03-06T07:31:38.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.747+0000 7f3bc9ffb640 1 -- 172.21.15.17:0/3737255039 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f3bc407b8b0 con 0x7f3bcc0c7180 2024-03-06T07:31:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:38.749 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:38.749 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 msgr2=0x7f3bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 0x7f3bb806d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc1ef190 tx=0x7f3bc0009040 comp rx=0 tx=0).stop 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 msgr2=0x7f3bcc154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000f80 tx=0x7f3bc4004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 shutdown_connections 2024-03-06T07:31:38.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f3bb806b4d0 0x7f3bb806d990 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3bcc1527b0 0x7f3bcc153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3bcc0c88b0 0x7f3bcc153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 --2- 172.21.15.17:0/3737255039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3bcc0c7180 0x7f3bcc154940 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 >> 172.21.15.17:0/3737255039 conn(0x7f3bcc0ac560 msgr2=0x7f3bcc0cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 shutdown_connections 2024-03-06T07:31:38.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:38.749+0000 7f3bd350c640 1 -- 172.21.15.17:0/3737255039 wait complete. 2024-03-06T07:31:39.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:39.163 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:40.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:40.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.264+0000 7fc92addd640 1 Processor -- start 2024-03-06T07:31:40.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.264+0000 7fc92addd640 1 -- start start 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.264+0000 7fc92addd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc92406d9f0 0x7fc9240a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc92addd640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9240c3710 0x7fc9241558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc92addd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc924158800 con 0x7fc9240c3280 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc92addd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc924158ae0 con 0x7fc92406d9f0 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc92addd640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc924158dc0 con 0x7fc9240c3710 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc928b79640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc92406d9f0 0x7fc9240a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc923fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc9240c3280 0x7fc9241551e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:37360/0 (socket says 172.21.15.17:37360) 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc923fff640 1 -- 172.21.15.17:0/3348654255 learned_addr learned my addr 172.21.15.17:0/3348654255 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc92937a640 1 --2- 172.21.15.17:0/3348654255 >> v2:172.21.15.163:3300/0 conn(0x7fc9240c3710 0x7fc9241558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc928b79640 1 -- 172.21.15.17:0/3348654255 >> v2:172.21.15.163:3300/0 conn(0x7fc9240c3710 msgr2=0x7fc9241558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:40.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc928b79640 1 --2- 172.21.15.17:0/3348654255 >> v2:172.21.15.163:3300/0 conn(0x7fc9240c3710 0x7fc9241558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc928b79640 1 -- 172.21.15.17:0/3348654255 >> v1:172.21.15.17:6789/0 conn(0x7fc9240c3280 legacy=0x7fc9241551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc928b79640 1 -- 172.21.15.17:0/3348654255 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9241590a0 con 0x7fc92406d9f0 2024-03-06T07:31:40.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.265+0000 7fc9237fe640 1 -- 172.21.15.17:0/3348654255 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444194849 0 0) 0x7fc924158800 con 0x7fc9240c3280 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc928b79640 1 --2- 172.21.15.17:0/3348654255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9240a7cd0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fc924159760 tx=0x7fc91802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea10d11543ddf14c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc9237fe640 1 -- 172.21.15.17:0/3348654255 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9180315a0 con 0x7fc92406d9f0 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc9237fe640 1 -- 172.21.15.17:0/3348654255 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918037070 con 0x7fc92406d9f0 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc9237fe640 1 -- 172.21.15.17:0/3348654255 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918031960 con 0x7fc92406d9f0 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 -- 172.21.15.17:0/3348654255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 msgr2=0x7fc9240a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 --2- 172.21.15.17:0/3348654255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9240a7cd0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7fc924159760 tx=0x7fc91802b840 comp rx=0 tx=0).stop 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 -- 172.21.15.17:0/3348654255 shutdown_connections 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 --2- 172.21.15.17:0/3348654255 >> v2:172.21.15.163:3300/0 conn(0x7fc9240c3710 0x7fc9241558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 --2- 172.21.15.17:0/3348654255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9240a7cd0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.266+0000 7fc92addd640 1 -- 172.21.15.17:0/3348654255 >> 172.21.15.17:0/3348654255 conn(0x7fc9240aa630 msgr2=0x7fc9240aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.267+0000 7fc92addd640 1 -- 172.21.15.17:0/3348654255 shutdown_connections 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.267+0000 7fc92addd640 1 -- 172.21.15.17:0/3348654255 wait complete. 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 Processor -- start 2024-03-06T07:31:40.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 -- start start 2024-03-06T07:31:40.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9240c3280 0x7fc9240ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9240c3710 0x7fc924153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92406ddd0 con 0x7fc92406d9f0 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924155e30 con 0x7fc9240c3280 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.268+0000 7fc92addd640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92406df50 con 0x7fc9240c3710 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43170/0 (socket says 172.21.15.17:43170) 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 -- 172.21.15.17:0/4203453170 learned_addr learned my addr 172.21.15.17:0/4203453170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc92937a640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9240c3710 0x7fc924153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc923fff640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9240c3280 0x7fc9240ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 -- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9240c3710 msgr2=0x7fc924153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9240c3710 0x7fc924153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 -- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9240c3280 msgr2=0x7fc9240ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9240c3280 0x7fc9240ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 -- 172.21.15.17:0/4203453170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924151610 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc928b79640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fc918009890 tx=0x7fc9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918032840 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9241518f0 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918032a00 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918032df0 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.269+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc924153a50 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.270+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc924151af0 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.271+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc91804c020 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.271+0000 7fc9217fa640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 0x7fc90806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.271+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc91802f020 con 0x7fc92406d9f0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.271+0000 7fc923fff640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 0x7fc90806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:40.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.272+0000 7fc923fff640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 0x7fc90806d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc924151500 tx=0x7fc910007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:40.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.275+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc924151af0 con 0x7fc92406d9f0 2024-03-06T07:31:40.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.493+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc92404f330 con 0x7fc92406d9f0 2024-03-06T07:31:40.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.494+0000 7fc9217fa640 1 -- 172.21.15.17:0/4203453170 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fc918040070 con 0x7fc92406d9f0 2024-03-06T07:31:40.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:40.496 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:40.496 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:40.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.495+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 msgr2=0x7fc90806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.495+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 0x7fc90806d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc924151500 tx=0x7fc910007b50 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.495+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 msgr2=0x7fc9241ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.495+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7fc918009890 tx=0x7fc9180046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 shutdown_connections 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc90806b4d0 0x7fc90806d990 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9240c3710 0x7fc924153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9240c3280 0x7fc9240ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 --2- 172.21.15.17:0/4203453170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc92406d9f0 0x7fc9241ee880 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 >> 172.21.15.17:0/4203453170 conn(0x7fc9240aa630 msgr2=0x7fc92406c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 shutdown_connections 2024-03-06T07:31:40.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:40.496+0000 7fc92addd640 1 -- 172.21.15.17:0/4203453170 wait complete. 2024-03-06T07:31:40.765 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:40.767 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:31:40.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:40.913 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:41.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.014+0000 7fbb6399a640 1 Processor -- start 2024-03-06T07:31:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.014+0000 7fbb6399a640 1 -- start start 2024-03-06T07:31:42.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb6399a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb5c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb6399a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c0c73d0 con 0x7fbb5c14f360 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb6399a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c0be2e0 con 0x7fbb5c0c88c0 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb6399a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c159700 con 0x7fbb5c159270 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb61736640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbb5c14f360 0x7fbb5c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32940/0 (socket says 172.21.15.17:32940) 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb61736640 1 -- 172.21.15.17:0/560622704 learned_addr learned my addr 172.21.15.17:0/560622704 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.015+0000 7fbb61f37640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb5c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1463050480 0 0) 0x7fbb5c0c73d0 con 0x7fbb5c14f360 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb48003520 con 0x7fbb5c14f360 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347600438 0 0) 0x7fbb48003520 con 0x7fbb5c14f360 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 msgr2=0x7fbb5c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb5c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 >> v1:172.21.15.163:6789/0 conn(0x7fbb5c0c88c0 legacy=0x7fbb5c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0c7670 con 0x7fbb5c14f360 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbb58001670 con 0x7fbb5c14f360 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 >> v1:172.21.15.17:6789/0 conn(0x7fbb5c14f360 legacy=0x7fbb5c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb48003f20 0x7fbb48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb48004d40 0x7fbb480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c159700 con 0x7fbb48004d40 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4800bad0 con 0x7fbb48003f20 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4800bcd0 con 0x7fbb5c159270 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb48004d40 0x7fbb480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61736640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb60f35640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb48003f20 0x7fbb48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 -- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 msgr2=0x7fbb480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c159270 0x7fbb480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 -- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb48003f20 msgr2=0x7fbb48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb48003f20 0x7fbb48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb61f37640 1 -- 172.21.15.17:0/560622704 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48003520 con 0x7fbb48004d40 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.016+0000 7fbb60f35640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb48003f20 0x7fbb48004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.017+0000 7fbb61f37640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb48004d40 0x7fbb480051c0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99868282b5ad2be4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.017+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54039070 con 0x7fbb48004d40 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.017+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5402cc20 con 0x7fbb48004d40 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.017+0000 7fbb4ffff640 1 -- 172.21.15.17:0/560622704 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54009c10 con 0x7fbb48004d40 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 -- 172.21.15.17:0/560622704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb48004d40 msgr2=0x7fbb480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 --2- 172.21.15.17:0/560622704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb48004d40 0x7fbb480051c0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009fc0 tx=0x7fbb48005830 comp rx=0 tx=0).stop 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb61f37640 1 -- 172.21.15.17:0/560622704 reap_dead start 2024-03-06T07:31:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 -- 172.21.15.17:0/560622704 shutdown_connections 2024-03-06T07:31:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 -- 172.21.15.17:0/560622704 >> 172.21.15.17:0/560622704 conn(0x7fbb5c0ac570 msgr2=0x7fbb5c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 -- 172.21.15.17:0/560622704 shutdown_connections 2024-03-06T07:31:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.018+0000 7fbb6399a640 1 -- 172.21.15.17:0/560622704 wait complete. 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.020+0000 7fbb6399a640 1 Processor -- start 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.020+0000 7fbb6399a640 1 -- start start 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 0x7fbb5c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c14c020 0x7fbb5c1ece20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0c7c30 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0c5a50 con 0x7fbb5c152190 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb6399a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0c5bd0 con 0x7fbb5c14c020 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48614/0 (socket says 172.21.15.17:48614) 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 -- 172.21.15.17:0/1097900181 learned_addr learned my addr 172.21.15.17:0/1097900181 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61f37640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c14c020 0x7fbb5c1ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb60f35640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 0x7fbb5c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 -- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c14c020 msgr2=0x7fbb5c1ece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c14c020 0x7fbb5c1ece20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 -- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 msgr2=0x7fbb5c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 0x7fbb5c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 -- 172.21.15.17:0/1097900181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c1ec0d0 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb60f35640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 0x7fbb5c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb61736640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007290 tx=0x7fbb58007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.021+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800f9c0 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.022+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb58018070 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.022+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800fd80 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.022+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5c1ec340 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.022+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c1eecc0 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.023+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5c04f330 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.023+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb5801d460 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.023+0000 7fbb4dffb640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 0x7fbb4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.024+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fbb5808a3c0 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.024+0000 7fbb60f35640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 0x7fbb4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:42.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.024+0000 7fbb60f35640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 0x7fbb4006d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b07e0 tx=0x7fbb50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:42.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.028+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb58059070 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.247+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb5c0c8d50 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.248+0000 7fbb4dffb640 1 -- 172.21.15.17:0/1097900181 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fbb58058970 con 0x7fbb5c1ee3a0 2024-03-06T07:31:42.250 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:42.250 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:42.250 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:42.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 msgr2=0x7fbb4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 0x7fbb4006d9c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0b07e0 tx=0x7fbb50009040 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 msgr2=0x7fbb5c1ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007290 tx=0x7fbb58007750 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 shutdown_connections 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbb4006b500 0x7fbb4006d9c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb5c14c020 0x7fbb5c1ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb5c152190 0x7fbb5c14bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 --2- 172.21.15.17:0/1097900181 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbb5c1ee3a0 0x7fbb5c1ee780 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 >> 172.21.15.17:0/1097900181 conn(0x7fbb5c0ac570 msgr2=0x7fbb5c0aa3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 shutdown_connections 2024-03-06T07:31:42.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:42.250+0000 7fbb6399a640 1 -- 172.21.15.17:0/1097900181 wait complete. 2024-03-06T07:31:42.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:42.664 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:43.665 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.765+0000 7f6b29ce1640 1 Processor -- start 2024-03-06T07:31:43.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.765+0000 7f6b29ce1640 1 -- start start 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b29ce1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6b241592d0 0x7f6b241552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b29ce1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b24152d40 con 0x7f6b24152960 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b29ce1640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b240c1b50 con 0x7f6b240ac260 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b29ce1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b24066e30 con 0x7f6b241592d0 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b237fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f6b24152960 0x7f6b2406dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32954/0 (socket says 172.21.15.17:32954) 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.766+0000 7f6b237fe640 1 -- 172.21.15.17:0/3277700215 learned_addr learned my addr 172.21.15.17:0/3277700215 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745330516 0 0) 0x7f6b24152d40 con 0x7f6b24152960 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b22ffd640 1 --2- 172.21.15.17:0/3277700215 >> v2:172.21.15.163:3300/0 conn(0x7f6b241592d0 0x7f6b241552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b10003520 con 0x7f6b24152960 2024-03-06T07:31:43.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178971254 0 0) 0x7f6b10003520 con 0x7f6b24152960 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 >> v2:172.21.15.163:3300/0 conn(0x7f6b241592d0 msgr2=0x7f6b241552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 --2- 172.21.15.17:0/3277700215 >> v2:172.21.15.163:3300/0 conn(0x7f6b241592d0 0x7f6b241552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 >> v1:172.21.15.163:6789/0 conn(0x7f6b240ac260 legacy=0x7f6b24155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b240c1dc0 con 0x7f6b24152960 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6b0c001670 con 0x7f6b24152960 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 >> v1:172.21.15.17:6789/0 conn(0x7f6b24152960 legacy=0x7f6b2406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 0x7f6b10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b10004800 0x7f6b10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100051c0 0x7f6b1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24066e30 con 0x7f6b100039b0 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1000c060 con 0x7f6b100051c0 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1000c260 con 0x7f6b10004800 2024-03-06T07:31:43.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 0x7f6b10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b23fff640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100051c0 0x7f6b1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b22ffd640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b10004800 0x7f6b10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 -- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b10004800 msgr2=0x7f6b10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b10004800 0x7f6b10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 -- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100051c0 msgr2=0x7f6b1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100051c0 0x7f6b1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b237fe640 1 -- 172.21.15.17:0/3277700215 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10003520 con 0x7f6b100039b0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b23fff640 1 -- 172.21.15.17:0/3277700215 reap_dead start 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.767+0000 7f6b22ffd640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b10004800 0x7f6b10004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.768+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 0x7f6b10003dd0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c210 tx=0x7f6b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4a7744dc903043 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.768+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c007d10 con 0x7f6b100039b0 2024-03-06T07:31:43.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.768+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c030070 con 0x7f6b100039b0 2024-03-06T07:31:43.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.768+0000 7f6b227fc640 1 -- 172.21.15.17:0/3277700215 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c005d30 con 0x7f6b100039b0 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 msgr2=0x7f6b10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 0x7f6b10003dd0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00c210 tx=0x7f6b10005830 comp rx=0 tx=0).stop 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3277700215 shutdown_connections 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3277700215 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b100039b0 0x7f6b10003dd0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3277700215 >> 172.21.15.17:0/3277700215 conn(0x7f6b240c70b0 msgr2=0x7f6b240c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3277700215 shutdown_connections 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.769+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3277700215 wait complete. 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.770+0000 7f6b29ce1640 1 Processor -- start 2024-03-06T07:31:43.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.770+0000 7f6b29ce1640 1 -- start start 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100039b0 0x7f6b241e0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b241ee660 0x7f6b2414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240c20a0 con 0x7f6b241eeae0 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b24159760 con 0x7f6b100039b0 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b29ce1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240c2220 con 0x7f6b241ee660 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48624/0 (socket says 172.21.15.17:48624) 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 -- 172.21.15.17:0/3758919920 learned_addr learned my addr 172.21.15.17:0/3758919920 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100039b0 0x7f6b241e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b22ffd640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b241ee660 0x7f6b2414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 -- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b241ee660 msgr2=0x7f6b2414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b241ee660 0x7f6b2414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 -- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100039b0 msgr2=0x7f6b241e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100039b0 0x7f6b241e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 -- 172.21.15.17:0/3758919920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b241ef310 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.771+0000 7f6b23fff640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f6b240687f0 tx=0x7f6b1c00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.772+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c01a070 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.772+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c006d60 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.772+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c003bb0 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.772+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b241df500 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.772+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b241eefa0 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.773+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b24072f40 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.773+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b1c01d070 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.774+0000 7f6b1bfff640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 0x7f6b0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.774+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f6b1c089ae0 con 0x7f6b241eeae0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.774+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 0x7f6b0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:43.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.775+0000 7f6b237fe640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 0x7f6b0406d990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c034000 tx=0x7f6b0c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:43.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.778+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6b24072f40 con 0x7f6b241eeae0 2024-03-06T07:31:43.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.997+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b2404f330 con 0x7f6b241eeae0 2024-03-06T07:31:43.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.997+0000 7f6b1bfff640 1 -- 172.21.15.17:0/3758919920 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f6b1c058070 con 0x7f6b241eeae0 2024-03-06T07:31:43.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:43.999 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:43.999 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 msgr2=0x7f6b0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 0x7f6b0406d990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c034000 tx=0x7f6b0c007000 comp rx=0 tx=0).stop 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 msgr2=0x7f6b2414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f6b240687f0 tx=0x7f6b1c00fc00 comp rx=0 tx=0).stop 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 shutdown_connections 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6b0406b4d0 0x7f6b0406d990 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b241eeae0 0x7f6b2414f0d0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b241ee660 0x7f6b2414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 --2- 172.21.15.17:0/3758919920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b100039b0 0x7f6b241e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:44.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 >> 172.21.15.17:0/3758919920 conn(0x7f6b240c70b0 msgr2=0x7f6b240c5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:44.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 shutdown_connections 2024-03-06T07:31:44.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:43.999+0000 7f6b29ce1640 1 -- 172.21.15.17:0/3758919920 wait complete. 2024-03-06T07:31:44.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:44.414 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:45.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:45.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.518+0000 7f453982a640 1 Processor -- start 2024-03-06T07:31:45.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.518+0000 7f453982a640 1 -- start start 2024-03-06T07:31:45.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f453982a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f453414f360 0x7f4534155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f453982a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4534158b60 0x7f4534158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f453982a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45340c73d0 con 0x7f45340c88c0 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f453982a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340c8dd0 con 0x7f453414f360 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f453982a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340be2e0 con 0x7f4534158b60 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f4532ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f453414f360 0x7f4534155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f45327fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f45340c88c0 0x7f45340c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32958/0 (socket says 172.21.15.17:32958) 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.519+0000 7f45327fc640 1 -- 172.21.15.17:0/2783334205 learned_addr learned my addr 172.21.15.17:0/2783334205 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f45337fe640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4534158b60 0x7f4534158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4532ffd640 1 -- 172.21.15.17:0/2783334205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4534158b60 msgr2=0x7f4534158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4532ffd640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4534158b60 0x7f4534158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4532ffd640 1 -- 172.21.15.17:0/2783334205 >> v1:172.21.15.17:6789/0 conn(0x7f45340c88c0 legacy=0x7f45340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4531ffb640 1 -- 172.21.15.17:0/2783334205 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243876085 0 0) 0x7f45340c73d0 con 0x7f45340c88c0 2024-03-06T07:31:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4532ffd640 1 -- 172.21.15.17:0/2783334205 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340c7670 con 0x7f453414f360 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.520+0000 7f4532ffd640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f453414f360 0x7f4534155c10 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f4524009d90 tx=0x7f452402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3dda395c90c6e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f4531ffb640 1 -- 172.21.15.17:0/2783334205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524039070 con 0x7f453414f360 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f4531ffb640 1 -- 172.21.15.17:0/2783334205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452402cbc0 con 0x7f453414f360 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f4531ffb640 1 -- 172.21.15.17:0/2783334205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524034440 con 0x7f453414f360 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 -- 172.21.15.17:0/2783334205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f453414f360 msgr2=0x7f4534155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f453414f360 0x7f4534155c10 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f4524009d90 tx=0x7f452402be10 comp rx=0 tx=0).stop 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 -- 172.21.15.17:0/2783334205 shutdown_connections 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4534158b60 0x7f4534158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 --2- 172.21.15.17:0/2783334205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f453414f360 0x7f4534155c10 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 -- 172.21.15.17:0/2783334205 >> 172.21.15.17:0/2783334205 conn(0x7f45340ac570 msgr2=0x7f45340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 -- 172.21.15.17:0/2783334205 shutdown_connections 2024-03-06T07:31:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.521+0000 7f453982a640 1 -- 172.21.15.17:0/2783334205 wait complete. 2024-03-06T07:31:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.522+0000 7f453982a640 1 Processor -- start 2024-03-06T07:31:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 -- start start 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c88c0 0x7f45341ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453414f360 0x7f453414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c5a50 con 0x7f4534158b60 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c7c30 con 0x7f453414f360 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f453982a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340c5bd0 con 0x7f45340c88c0 2024-03-06T07:31:45.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f45337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f45337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48644/0 (socket says 172.21.15.17:48644) 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f45337fe640 1 -- 172.21.15.17:0/713408384 learned_addr learned my addr 172.21.15.17:0/713408384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.523+0000 7f4532ffd640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c88c0 0x7f45341ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45327fc640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453414f360 0x7f453414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 -- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c88c0 msgr2=0x7f45341ee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c88c0 0x7f45341ee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 -- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453414f360 msgr2=0x7f453414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453414f360 0x7f453414ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 -- 172.21.15.17:0/713408384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45341eec40 con 0x7f4534158b60 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f45337fe640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c7f0 tx=0x7f452c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c015070 con 0x7f4534158b60 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c004220 con 0x7f4534158b60 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45341ebfd0 con 0x7f4534158b60 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c007620 con 0x7f4534158b60 2024-03-06T07:31:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.524+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45341ec590 con 0x7f4534158b60 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.525+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45341ec790 con 0x7f4534158b60 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.526+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f452c0077c0 con 0x7f4534158b60 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.526+0000 7f451bfff640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 0x7f452006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.526+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f452c08a1c0 con 0x7f4534158b60 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.526+0000 7f4532ffd640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 0x7f452006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.527+0000 7f4532ffd640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 0x7f452006d990 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4524036000 tx=0x7f4524030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:45.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.531+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f45341ec790 con 0x7f4534158b60 2024-03-06T07:31:45.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.748+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f453404f330 con 0x7f4534158b60 2024-03-06T07:31:45.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.749+0000 7f451bfff640 1 -- 172.21.15.17:0/713408384 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f452c0587f0 con 0x7f4534158b60 2024-03-06T07:31:45.751 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:45.751 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:45.751 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 msgr2=0x7f452006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 0x7f452006d990 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4524036000 tx=0x7f4524030000 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 msgr2=0x7f453414bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c7f0 tx=0x7f452c00ccb0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 shutdown_connections 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f452006b4d0 0x7f452006d990 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4534158b60 0x7f453414bf90 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f453414f360 0x7f453414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 --2- 172.21.15.17:0/713408384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f45340c88c0 0x7f45341ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 >> 172.21.15.17:0/713408384 conn(0x7f45340ac570 msgr2=0x7f453406b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 shutdown_connections 2024-03-06T07:31:45.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:45.751+0000 7f453982a640 1 -- 172.21.15.17:0/713408384 wait complete. 2024-03-06T07:31:46.166 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:46.166 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:47.167 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:47.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.266+0000 7fb3d338a640 1 Processor -- start 2024-03-06T07:31:47.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.266+0000 7fb3d338a640 1 -- start start 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d338a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3cc0c3150 0x7fb3cc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d338a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3cc152c20 0x7fb3cc0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d338a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3cc072440 con 0x7fb3cc14f650 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d338a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc0c2b10 con 0x7fb3cc0c3150 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d338a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3cc152140 con 0x7fb3cc152c20 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3cc0c3150 0x7fb3cc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb3cc0c3150 0x7fb3cc0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48648/0 (socket says 172.21.15.17:48648) 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 -- 172.21.15.17:0/3155523958 learned_addr learned my addr 172.21.15.17:0/3155523958 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d1927640 1 --2- 172.21.15.17:0/3155523958 >> v2:172.21.15.163:3300/0 conn(0x7fb3cc152c20 0x7fb3cc0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 -- 172.21.15.17:0/3155523958 >> v2:172.21.15.163:3300/0 conn(0x7fb3cc152c20 msgr2=0x7fb3cc0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 --2- 172.21.15.17:0/3155523958 >> v2:172.21.15.163:3300/0 conn(0x7fb3cc152c20 0x7fb3cc0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 -- 172.21.15.17:0/3155523958 >> v1:172.21.15.163:6789/0 conn(0x7fb3cc14f650 legacy=0x7fb3cc071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.267+0000 7fb3d0925640 1 -- 172.21.15.17:0/3155523958 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc152420 con 0x7fb3cc0c3150 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.268+0000 7fb3d0925640 1 --2- 172.21.15.17:0/3155523958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc0c5690 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc073690 tx=0x7fb3c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85bc1da7ebc4714 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.268+0000 7fb3bffff640 1 -- 172.21.15.17:0/3155523958 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8035020 con 0x7fb3cc0c3150 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.268+0000 7fb3bffff640 1 -- 172.21.15.17:0/3155523958 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c8031870 con 0x7fb3cc0c3150 2024-03-06T07:31:47.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.268+0000 7fb3bffff640 1 -- 172.21.15.17:0/3155523958 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8031c40 con 0x7fb3cc0c3150 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 -- 172.21.15.17:0/3155523958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 msgr2=0x7fb3cc0c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 --2- 172.21.15.17:0/3155523958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc0c5690 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc073690 tx=0x7fb3c802b910 comp rx=0 tx=0).stop 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 -- 172.21.15.17:0/3155523958 shutdown_connections 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 --2- 172.21.15.17:0/3155523958 >> v2:172.21.15.163:3300/0 conn(0x7fb3cc152c20 0x7fb3cc0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 --2- 172.21.15.17:0/3155523958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc0c5690 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 -- 172.21.15.17:0/3155523958 >> 172.21.15.17:0/3155523958 conn(0x7fb3cc0c4290 msgr2=0x7fb3cc0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 -- 172.21.15.17:0/3155523958 shutdown_connections 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.269+0000 7fb3d338a640 1 -- 172.21.15.17:0/3155523958 wait complete. 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.270+0000 7fb3d338a640 1 Processor -- start 2024-03-06T07:31:47.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.270+0000 7fb3d338a640 1 -- start start 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc14f650 0x7fb3cc0abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3cc152c20 0x7fb3cc0ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc072110 con 0x7fb3cc0c3150 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc071f90 con 0x7fb3cc14f650 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0c3f40 con 0x7fb3cc152c20 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48662/0 (socket says 172.21.15.17:48662) 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 -- 172.21.15.17:0/907705423 learned_addr learned my addr 172.21.15.17:0/907705423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d0925640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc14f650 0x7fb3cc0abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1927640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3cc152c20 0x7fb3cc0ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 -- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3cc152c20 msgr2=0x7fb3cc0ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:47.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3cc152c20 0x7fb3cc0ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 -- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc14f650 msgr2=0x7fb3cc0abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc14f650 0x7fb3cc0abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 -- 172.21.15.17:0/907705423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc149080 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d1126640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c430 tx=0x7fb3c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00052d0 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0005490 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc148da0 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.271+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0007260 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.272+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc153af0 con 0x7fb3cc0c3150 2024-03-06T07:31:47.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.273+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc04f330 con 0x7fb3cc0c3150 2024-03-06T07:31:47.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.273+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3c0007400 con 0x7fb3cc0c3150 2024-03-06T07:31:47.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.273+0000 7fb3bdffb640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 0x7fb3b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:47.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.273+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fb3c00898f0 con 0x7fb3cc0c3150 2024-03-06T07:31:47.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.274+0000 7fb3d0925640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 0x7fb3b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:47.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.274+0000 7fb3d0925640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 0x7fb3b006d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8036000 tx=0x7fb3c803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:47.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.278+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb3cc072440 con 0x7fb3cc0c3150 2024-03-06T07:31:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.557+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3cc068460 con 0x7fb3cc0c3150 2024-03-06T07:31:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.557+0000 7fb3bdffb640 1 -- 172.21.15.17:0/907705423 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fb3c0058570 con 0x7fb3cc0c3150 2024-03-06T07:31:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:47.559 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:47.559 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 msgr2=0x7fb3b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 0x7fb3b006d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8036000 tx=0x7fb3c803c040 comp rx=0 tx=0).stop 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 msgr2=0x7fb3cc155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c430 tx=0x7fb3c000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 shutdown_connections 2024-03-06T07:31:47.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb3b006b4d0 0x7fb3b006d990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3cc152c20 0x7fb3cc0ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3cc14f650 0x7fb3cc0abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 --2- 172.21.15.17:0/907705423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3cc0c3150 0x7fb3cc155b40 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 >> 172.21.15.17:0/907705423 conn(0x7fb3cc0c4290 msgr2=0x7fb3cc14d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 shutdown_connections 2024-03-06T07:31:47.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:47.559+0000 7fb3d338a640 1 -- 172.21.15.17:0/907705423 wait complete. 2024-03-06T07:31:47.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:47.975 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:48.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:49.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.076+0000 7f36b161c640 1 Processor -- start 2024-03-06T07:31:49.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.076+0000 7f36b161c640 1 -- start start 2024-03-06T07:31:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36b161c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f36ac0c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36b161c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac1572f0 con 0x7f36ac1506b0 2024-03-06T07:31:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36b161c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0ad310 con 0x7f36ac153990 2024-03-06T07:31:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36b161c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac156450 con 0x7f36ac156ea0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aaffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f36ac1506b0 0x7f36ac0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:32970/0 (socket says 172.21.15.17:32970) 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aaffd640 1 -- 172.21.15.17:0/382699622 learned_addr learned my addr 172.21.15.17:0/382699622 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f36ac0c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534006305 0 0) 0x7f36ac1572f0 con 0x7f36ac1506b0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698003520 con 0x7f36ac1506b0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108769973 0 0) 0x7f3698003520 con 0x7f36ac1506b0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 msgr2=0x7f36ac0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f36ac0c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 >> v1:172.21.15.163:6789/0 conn(0x7f36ac153990 legacy=0x7f36ac0acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac070aa0 con 0x7f36ac1506b0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f369c001690 con 0x7f36ac1506b0 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 >> v1:172.21.15.17:6789/0 conn(0x7f36ac1506b0 legacy=0x7f36ac0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36980039e0 0x7f3698003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f3698004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3698004d40 0x7f36980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac156450 con 0x7f36980039e0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369800b9c0 con 0x7f3698004d40 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.077+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369800bbc0 con 0x7f36ac156ea0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36980039e0 0x7f3698003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3698004d40 0x7f36980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36a3fff640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f3698004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 -- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 msgr2=0x7f3698004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac156ea0 0x7f3698004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 -- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3698004d40 msgr2=0x7f36980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3698004d40 0x7f36980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36aaffd640 1 -- 172.21.15.17:0/382699622 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3698003520 con 0x7f36980039e0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.078+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3698004d40 0x7f36980051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36aaffd640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36980039e0 0x7f3698003dc0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f369c00c210 tx=0x7f3698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5335c1b58fab3aea server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:49.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c007d30 con 0x7f36980039e0 2024-03-06T07:31:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c030070 con 0x7f36980039e0 2024-03-06T07:31:49.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36aa7fc640 1 -- 172.21.15.17:0/382699622 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c005d50 con 0x7f36980039e0 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36b161c640 1 -- 172.21.15.17:0/382699622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36980039e0 msgr2=0x7f3698003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36b161c640 1 --2- 172.21.15.17:0/382699622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36980039e0 0x7f3698003dc0 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f369c00c210 tx=0x7f3698005830 comp rx=0 tx=0).stop 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36ab7fe640 1 -- 172.21.15.17:0/382699622 reap_dead start 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36b161c640 1 -- 172.21.15.17:0/382699622 shutdown_connections 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36b161c640 1 -- 172.21.15.17:0/382699622 >> 172.21.15.17:0/382699622 conn(0x7f36ac06b1a0 msgr2=0x7f36ac06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.079+0000 7f36b161c640 1 -- 172.21.15.17:0/382699622 shutdown_connections 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.080+0000 7f36b161c640 1 -- 172.21.15.17:0/382699622 wait complete. 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 Processor -- start 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 -- start start 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36980039e0 0x7f36ac1ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac1534a0 0x7f36ac0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1566a0 con 0x7f36ac14a950 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac156a10 con 0x7f36980039e0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36b161c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0c54a0 con 0x7f36ac1534a0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36ab7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48682/0 (socket says 172.21.15.17:48682) 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36ab7fe640 1 -- 172.21.15.17:0/826307751 learned_addr learned my addr 172.21.15.17:0/826307751 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36a3fff640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac1534a0 0x7f36ac0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.081+0000 7f36aaffd640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36980039e0 0x7f36ac1ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 -- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac1534a0 msgr2=0x7f36ac0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac1534a0 0x7f36ac0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 -- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36980039e0 msgr2=0x7f36ac1ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36980039e0 0x7f36ac1ec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 -- 172.21.15.17:0/826307751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac155d80 con 0x7f36ac14a950 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36ab7fe640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f36ac06be90 tx=0x7f36a4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a401a030 con 0x7f36ac14a950 2024-03-06T07:31:49.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a4010e30 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac156060 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003bb0 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.082+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac0c82c0 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.083+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac04f330 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.084+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a4005d10 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.084+0000 7f36a37fe640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.084+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f36a408acb0 con 0x7f36ac14a950 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.085+0000 7f36aaffd640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:49.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.085+0000 7f36aaffd640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f369c034000 tx=0x7f369c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:49.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.089+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36a4024070 con 0x7f36ac14a950 2024-03-06T07:31:49.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.306+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36ac0700a0 con 0x7f36ac14a950 2024-03-06T07:31:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.306+0000 7f36a37fe640 1 -- 172.21.15.17:0/826307751 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f36a40592e0 con 0x7f36ac14a950 2024-03-06T07:31:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:49.308 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:49.308 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 msgr2=0x7f368c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f369c034000 tx=0x7f369c007000 comp rx=0 tx=0).stop 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 msgr2=0x7f36ac1e8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f36ac06be90 tx=0x7f36a4010590 comp rx=0 tx=0).stop 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 shutdown_connections 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36ac14a950 0x7f36ac1e8790 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36ac1534a0 0x7f36ac0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 --2- 172.21.15.17:0/826307751 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36980039e0 0x7f36ac1ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 >> 172.21.15.17:0/826307751 conn(0x7f36ac06b1a0 msgr2=0x7f36ac14f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 shutdown_connections 2024-03-06T07:31:49.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:49.308+0000 7f36b161c640 1 -- 172.21.15.17:0/826307751 wait complete. 2024-03-06T07:31:49.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:49.723 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:50.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:50.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.824+0000 7fb43ad93640 1 Processor -- start 2024-03-06T07:31:50.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.824+0000 7fb43ad93640 1 -- start start 2024-03-06T07:31:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb43ad93640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4341527b0 0x7fb434159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb43ad93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4340c88b0 0x7fb4340c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb43ad93640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4340c7c60 con 0x7fb4340c7180 2024-03-06T07:31:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb43ad93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb434159640 con 0x7fb4340c88b0 2024-03-06T07:31:50.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb43ad93640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4340c8060 con 0x7fb4341527b0 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4340c88b0 0x7fb4340c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb4340c88b0 0x7fb4340c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51974/0 (socket says 172.21.15.17:51974) 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 -- 172.21.15.17:0/2335928502 learned_addr learned my addr 172.21.15.17:0/2335928502 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb438b2f640 1 --2- 172.21.15.17:0/2335928502 >> v2:172.21.15.163:3300/0 conn(0x7fb4341527b0 0x7fb434159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 -- 172.21.15.17:0/2335928502 >> v2:172.21.15.163:3300/0 conn(0x7fb4341527b0 msgr2=0x7fb434159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 --2- 172.21.15.17:0/2335928502 >> v2:172.21.15.163:3300/0 conn(0x7fb4341527b0 0x7fb434159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 -- 172.21.15.17:0/2335928502 >> v1:172.21.15.163:6789/0 conn(0x7fb4340c7180 legacy=0x7fb434155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.825+0000 7fb439330640 1 -- 172.21.15.17:0/2335928502 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340c76a0 con 0x7fb4340c88b0 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.826+0000 7fb439330640 1 --2- 172.21.15.17:0/2335928502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4340c88b0 0x7fb4340c5330 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fb4340cba80 tx=0x7fb42c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61132040d93599 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb4337fe640 1 -- 172.21.15.17:0/2335928502 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c02ca80 con 0x7fb4340c88b0 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb4337fe640 1 -- 172.21.15.17:0/2335928502 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c02cc20 con 0x7fb4340c88b0 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb4337fe640 1 -- 172.21.15.17:0/2335928502 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0343f0 con 0x7fb4340c88b0 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 -- 172.21.15.17:0/2335928502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4340c88b0 msgr2=0x7fb4340c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 --2- 172.21.15.17:0/2335928502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4340c88b0 0x7fb4340c5330 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7fb4340cba80 tx=0x7fb42c02bee0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 -- 172.21.15.17:0/2335928502 shutdown_connections 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 --2- 172.21.15.17:0/2335928502 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4340c88b0 0x7fb4340c5330 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 --2- 172.21.15.17:0/2335928502 >> v2:172.21.15.163:3300/0 conn(0x7fb4341527b0 0x7fb434159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 -- 172.21.15.17:0/2335928502 >> 172.21.15.17:0/2335928502 conn(0x7fb4340ac560 msgr2=0x7fb4340ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 -- 172.21.15.17:0/2335928502 shutdown_connections 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.827+0000 7fb43ad93640 1 -- 172.21.15.17:0/2335928502 wait complete. 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.828+0000 7fb43ad93640 1 Processor -- start 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.828+0000 7fb43ad93640 1 -- start start 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 0x7fb4340c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4340c88b0 0x7fb4340c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434155600 con 0x7fb4341527b0 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434155320 con 0x7fb4340c7180 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb43ad93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb434152b90 con 0x7fb4340c88b0 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb439330640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb439330640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51980/0 (socket says 172.21.15.17:51980) 2024-03-06T07:31:50.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb439330640 1 -- 172.21.15.17:0/1184889522 learned_addr learned my addr 172.21.15.17:0/1184889522 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb438b2f640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 0x7fb4340c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.829+0000 7fb433fff640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4340c88b0 0x7fb4340c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 -- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4340c88b0 msgr2=0x7fb4340c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4340c88b0 0x7fb4340c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 -- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 msgr2=0x7fb4340c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 0x7fb4340c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 -- 172.21.15.17:0/1184889522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4341ee860 con 0x7fb4341527b0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb438b2f640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 0x7fb4340c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb439330640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c043020 con 0x7fb4341527b0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42c03e450 con 0x7fb4341527b0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4340c20a0 con 0x7fb4341527b0 2024-03-06T07:31:50.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42c009510 con 0x7fb4341527b0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.830+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4340c4630 con 0x7fb4341527b0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.831+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4340c1e90 con 0x7fb4341527b0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.831+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb42c046020 con 0x7fb4341527b0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.832+0000 7fb4317fa640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 0x7fb41806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.832+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fb42c0acc70 con 0x7fb4341527b0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.832+0000 7fb438b2f640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 0x7fb41806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.833+0000 7fb438b2f640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 0x7fb41806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb4340681b0 tx=0x7fb428008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:50.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:50.836+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb42c04d030 con 0x7fb4341527b0 2024-03-06T07:31:51.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.131+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb43406ce00 con 0x7fb4341527b0 2024-03-06T07:31:51.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.132+0000 7fb4317fa640 1 -- 172.21.15.17:0/1184889522 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fb42c07ba10 con 0x7fb4341527b0 2024-03-06T07:31:51.133 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:51.134 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:51.134 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 msgr2=0x7fb41806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 0x7fb41806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb4340681b0 tx=0x7fb428008040 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 msgr2=0x7fb434147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fb42c036040 tx=0x7fb42c004a40 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 shutdown_connections 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb41806b4d0 0x7fb41806d990 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4341527b0 0x7fb434147c00 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4340c88b0 0x7fb4340c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 --2- 172.21.15.17:0/1184889522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4340c7180 0x7fb4340c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 >> 172.21.15.17:0/1184889522 conn(0x7fb4340ac560 msgr2=0x7fb4340aa640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 shutdown_connections 2024-03-06T07:31:51.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:51.134+0000 7fb43ad93640 1 -- 172.21.15.17:0/1184889522 wait complete. 2024-03-06T07:31:51.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:51.549 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:52.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:52.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.651+0000 7f1659aaf640 1 Processor -- start 2024-03-06T07:31:52.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.651+0000 7f1659aaf640 1 -- start start 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f1654155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16540c88c0 0x7f16540c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16540c5880 0x7f1654158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16540c8dd0 con 0x7f16540c5880 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16540c73d0 con 0x7f16540c88c0 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1659aaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540be2e0 con 0x7f165414f360 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1653fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16540c5880 0x7f1654158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1653fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f16540c5880 0x7f1654158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51986/0 (socket says 172.21.15.17:51986) 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1653fff640 1 -- 172.21.15.17:0/2130859349 learned_addr learned my addr 172.21.15.17:0/2130859349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:52.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f1652ffd640 1 --2- 172.21.15.17:0/2130859349 >> v2:172.21.15.163:3300/0 conn(0x7f16540c88c0 0x7f16540c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.652+0000 7f16537fe640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f1654155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 -- 172.21.15.17:0/2130859349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 msgr2=0x7f1654155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f1654155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 -- 172.21.15.17:0/2130859349 >> v2:172.21.15.163:3300/0 conn(0x7f16540c88c0 msgr2=0x7f16540c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 --2- 172.21.15.17:0/2130859349 >> v2:172.21.15.163:3300/0 conn(0x7f16540c88c0 0x7f16540c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 -- 172.21.15.17:0/2130859349 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540c7670 con 0x7f16540c5880 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f1653fff640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f1654158b60 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f16540cbac0 tx=0x7f164c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0ae8b73c40ac1a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f16527fc640 1 -- 172.21.15.17:0/2130859349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c02e070 con 0x7f16540c5880 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f16527fc640 1 -- 172.21.15.17:0/2130859349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c037070 con 0x7f16540c5880 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.653+0000 7f16527fc640 1 -- 172.21.15.17:0/2130859349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c031a00 con 0x7f16540c5880 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 -- 172.21.15.17:0/2130859349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 msgr2=0x7f1654158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f1654158b60 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f16540cbac0 tx=0x7f164c02b880 comp rx=0 tx=0).stop 2024-03-06T07:31:52.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 -- 172.21.15.17:0/2130859349 shutdown_connections 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f1654158b60 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 --2- 172.21.15.17:0/2130859349 >> v2:172.21.15.163:3300/0 conn(0x7f16540c88c0 0x7f16540c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 --2- 172.21.15.17:0/2130859349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f1654155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 -- 172.21.15.17:0/2130859349 >> 172.21.15.17:0/2130859349 conn(0x7f16540ac570 msgr2=0x7f16540ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 -- 172.21.15.17:0/2130859349 shutdown_connections 2024-03-06T07:31:52.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.654+0000 7f1659aaf640 1 -- 172.21.15.17:0/2130859349 wait complete. 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.655+0000 7f1659aaf640 1 Processor -- start 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.655+0000 7f1659aaf640 1 -- start start 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16540c88c0 0x7f16540ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f16540b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16541590a0 con 0x7f16540c5880 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540c7c30 con 0x7f16540c88c0 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1659aaf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654159220 con 0x7f165414f360 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51988/0 (socket says 172.21.15.17:51988) 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 -- 172.21.15.17:0/3097172032 learned_addr learned my addr 172.21.15.17:0/3097172032 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1652ffd640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16540c88c0 0x7f16540ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f1653fff640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f16540b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 -- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 msgr2=0x7f16540b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f16540b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 -- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16540c88c0 msgr2=0x7f16540ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16540c88c0 0x7f16540ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.656+0000 7f16537fe640 1 -- 172.21.15.17:0/3097172032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540afcd0 con 0x7f16540c5880 2024-03-06T07:31:52.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f16537fe640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f164400efc0 tx=0x7f164400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644018020 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1644014480 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644014850 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16541eecb0 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.657+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541512f0 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.658+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165404f330 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.658+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1644005d10 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.659+0000 7f163bfff640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 0x7f164006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.659+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f164408ad40 con 0x7f16540c5880 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.659+0000 7f1652ffd640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 0x7f164006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.659+0000 7f1652ffd640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 0x7f164006d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f16540ae210 tx=0x7f1648009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:52.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.663+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1644022070 con 0x7f16540c5880 2024-03-06T07:31:52.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.879+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1654153940 con 0x7f16540c5880 2024-03-06T07:31:52.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.880+0000 7f163bfff640 1 -- 172.21.15.17:0/3097172032 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f1644010040 con 0x7f16540c5880 2024-03-06T07:31:52.882 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:52.882 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:52.882 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 msgr2=0x7f164006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 0x7f164006d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f16540ae210 tx=0x7f1648009190 comp rx=0 tx=0).stop 2024-03-06T07:31:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 msgr2=0x7f16540adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f164400efc0 tx=0x7f164400c490 comp rx=0 tx=0).stop 2024-03-06T07:31:52.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 shutdown_connections 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f164006b4d0 0x7f164006d990 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f165414f360 0x7f16540b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16540c88c0 0x7f16540ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 --2- 172.21.15.17:0/3097172032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16540c5880 0x7f16540adcd0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 >> 172.21.15.17:0/3097172032 conn(0x7f16540ac570 msgr2=0x7f16540c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 shutdown_connections 2024-03-06T07:31:52.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:52.882+0000 7f1659aaf640 1 -- 172.21.15.17:0/3097172032 wait complete. 2024-03-06T07:31:53.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:53.297 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:54.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:54.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.398+0000 7fc49e9e9640 1 Processor -- start 2024-03-06T07:31:54.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.398+0000 7fc49e9e9640 1 -- start start 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.399+0000 7fc49e9e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc498155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.399+0000 7fc49e9e9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 0x7fc498158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.399+0000 7fc49e9e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980c73d0 con 0x7fc4980c88c0 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.399+0000 7fc49e9e9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980c8dd0 con 0x7fc498158b60 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.399+0000 7fc49e9e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980be2e0 con 0x7fc49814f360 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc497fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc498155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc49cf86640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 0x7fc498158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc4977fe640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc4980c88c0 0x7fc4980c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57818/0 (socket says 172.21.15.17:57818) 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc4977fe640 1 -- 172.21.15.17:0/2666841692 learned_addr learned my addr 172.21.15.17:0/2666841692 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:54.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc497fff640 1 -- 172.21.15.17:0/2666841692 >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 msgr2=0x7fc498158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc497fff640 1 --2- 172.21.15.17:0/2666841692 >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 0x7fc498158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc497fff640 1 -- 172.21.15.17:0/2666841692 >> v1:172.21.15.163:6789/0 conn(0x7fc4980c88c0 legacy=0x7fc4980c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc497fff640 1 -- 172.21.15.17:0/2666841692 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980c7670 con 0x7fc49814f360 2024-03-06T07:31:54.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.400+0000 7fc49cf86640 1 --2- 172.21.15.17:0/2666841692 >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 0x7fc498158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:31:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc497fff640 1 --2- 172.21.15.17:0/2666841692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc498155c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4880097a0 tx=0x7fc48802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe2cc8a063bcaf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc496ffd640 1 -- 172.21.15.17:0/2666841692 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880315a0 con 0x7fc49814f360 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc496ffd640 1 -- 172.21.15.17:0/2666841692 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488037070 con 0x7fc49814f360 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc496ffd640 1 -- 172.21.15.17:0/2666841692 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488031960 con 0x7fc49814f360 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2666841692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 msgr2=0x7fc498155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2666841692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc498155c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4880097a0 tx=0x7fc48802b840 comp rx=0 tx=0).stop 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2666841692 shutdown_connections 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2666841692 >> v2:172.21.15.163:3300/0 conn(0x7fc498158b60 0x7fc498158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2666841692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc498155c10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2666841692 >> 172.21.15.17:0/2666841692 conn(0x7fc4980ac570 msgr2=0x7fc4980ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2666841692 shutdown_connections 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.401+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2666841692 wait complete. 2024-03-06T07:31:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.402+0000 7fc49e9e9640 1 Processor -- start 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 -- start start 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc4981542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc498158b60 0x7fc498156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c5a50 con 0x7fc4980c88c0 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c7c30 con 0x7fc498158b60 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc49e9e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980c5bd0 con 0x7fc49814f360 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc497fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc497fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52004/0 (socket says 172.21.15.17:52004) 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.403+0000 7fc497fff640 1 -- 172.21.15.17:0/2943672552 learned_addr learned my addr 172.21.15.17:0/2943672552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc49cf86640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc498158b60 0x7fc498156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc4977fe640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc4981542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 -- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 msgr2=0x7fc4981542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc4981542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 -- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc498158b60 msgr2=0x7fc498156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc498158b60 0x7fc498156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 -- 172.21.15.17:0/2943672552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980af310 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc497fff640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fc4880098c0 tx=0x7fc4880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488032630 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4880327f0 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4980af5f0 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488032be0 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.404+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc498156900 con 0x7fc4980c88c0 2024-03-06T07:31:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.405+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49804f330 con 0x7fc4980c88c0 2024-03-06T07:31:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.406+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc48804e430 con 0x7fc4980c88c0 2024-03-06T07:31:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.406+0000 7fc494ff9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 0x7fc48406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.406+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc48802f030 con 0x7fc4980c88c0 2024-03-06T07:31:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.406+0000 7fc4977fe640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 0x7fc48406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:54.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.407+0000 7fc4977fe640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 0x7fc48406d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc498158850 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:54.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.411+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc488040070 con 0x7fc4980c88c0 2024-03-06T07:31:54.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.628+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc49806cda0 con 0x7fc4980c88c0 2024-03-06T07:31:54.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.629+0000 7fc494ff9640 1 -- 172.21.15.17:0/2943672552 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fc48807af30 con 0x7fc4980c88c0 2024-03-06T07:31:54.630 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:54.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:54.631 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:54.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 msgr2=0x7fc48406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 0x7fc48406d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc498158850 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 msgr2=0x7fc4980ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fc4880098c0 tx=0x7fc4880046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 shutdown_connections 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc48406b4d0 0x7fc48406d990 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc498158b60 0x7fc498156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc49814f360 0x7fc4981542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 --2- 172.21.15.17:0/2943672552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4980c88c0 0x7fc4980ad390 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 >> 172.21.15.17:0/2943672552 conn(0x7fc4980ac570 msgr2=0x7fc49814cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 shutdown_connections 2024-03-06T07:31:54.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:54.631+0000 7fc49e9e9640 1 -- 172.21.15.17:0/2943672552 wait complete. 2024-03-06T07:31:55.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:55.046 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:56.048 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.148+0000 7f9928266640 1 Processor -- start 2024-03-06T07:31:56.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.149+0000 7f9928266640 1 -- start start 2024-03-06T07:31:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.149+0000 7f9928266640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f9920155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.149+0000 7f9928266640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99200c73d0 con 0x7f9920158b60 2024-03-06T07:31:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.149+0000 7f9928266640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99200be2e0 con 0x7f99200c88c0 2024-03-06T07:31:56.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.149+0000 7f9928266640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9920159700 con 0x7f992014f360 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9926803640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9920158b60 0x7f9920158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:60040/0 (socket says 172.21.15.17:60040) 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9926803640 1 -- 172.21.15.17:0/1601347341 learned_addr learned my addr 172.21.15.17:0/1601347341 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9926002640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f9920155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776213599 0 0) 0x7f99200c73d0 con 0x7f9920158b60 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f990c003520 con 0x7f9920158b60 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234081548 0 0) 0x7f99200be2e0 con 0x7f99200c88c0 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99200c73d0 con 0x7f99200c88c0 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285274976 0 0) 0x7f990c003520 con 0x7f9920158b60 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 msgr2=0x7f9920155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f9920155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 >> v1:172.21.15.163:6789/0 conn(0x7f99200c88c0 legacy=0x7f99200c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200c7670 con 0x7f9920158b60 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f99180019f0 con 0x7f9920158b60 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 >> v1:172.21.15.17:6789/0 conn(0x7f9920158b60 legacy=0x7f9920158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f990c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f990c0040d0 0x7f990c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f990c004d40 0x7f990c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9920159700 con 0x7f990c0040d0 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c00bb50 con 0x7f990c004d40 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c00bd20 con 0x7f992014f360 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.150+0000 7f9925801640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f990c0040d0 0x7f990c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9926002640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f990c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9926803640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f990c004d40 0x7f990c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9925801640 1 -- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 msgr2=0x7f990c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9925801640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f992014f360 0x7f990c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9925801640 1 -- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f990c004d40 msgr2=0x7f990c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9925801640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f990c004d40 0x7f990c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9925801640 1 -- 172.21.15.17:0/1601347341 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c003520 con 0x7f990c0040d0 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.151+0000 7f9926803640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f990c004d40 0x7f990c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.152+0000 7f9925801640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f990c0040d0 0x7f990c004800 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f991c0018a0 tx=0x7f990c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39078aaa58241295 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.152+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c037070 con 0x7f990c0040d0 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.152+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0052d0 con 0x7f990c0040d0 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.152+0000 7f9925000640 1 -- 172.21.15.17:0/1601347341 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0014f0 con 0x7f990c0040d0 2024-03-06T07:31:56.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 -- 172.21.15.17:0/1601347341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f990c0040d0 msgr2=0x7f990c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 --2- 172.21.15.17:0/1601347341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f990c0040d0 0x7f990c004800 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f991c0018a0 tx=0x7f990c005830 comp rx=0 tx=0).stop 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9926803640 1 -- 172.21.15.17:0/1601347341 reap_dead start 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 -- 172.21.15.17:0/1601347341 shutdown_connections 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 -- 172.21.15.17:0/1601347341 >> 172.21.15.17:0/1601347341 conn(0x7f99200ac570 msgr2=0x7f99200ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 -- 172.21.15.17:0/1601347341 shutdown_connections 2024-03-06T07:31:56.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.153+0000 7f9928266640 1 -- 172.21.15.17:0/1601347341 wait complete. 2024-03-06T07:31:56.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.154+0000 7f9928266640 1 Processor -- start 2024-03-06T07:31:56.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.154+0000 7f9928266640 1 -- start start 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f990c0040d0 0x7f99201dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99200c21e0 0x7f9920157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200c7c30 con 0x7f99201eb850 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99200c5a50 con 0x7f99200c21e0 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9928266640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99200c5bd0 con 0x7f990c0040d0 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52018/0 (socket says 172.21.15.17:52018) 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 -- 172.21.15.17:0/1641618396 learned_addr learned my addr 172.21.15.17:0/1641618396 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926002640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f990c0040d0 0x7f99201dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9925801640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99200c21e0 0x7f9920157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 -- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f990c0040d0 msgr2=0x7f99201dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f990c0040d0 0x7f99201dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 -- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99200c21e0 msgr2=0x7f9920157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99200c21e0 0x7f9920157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 -- 172.21.15.17:0/1641618396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99200c1f90 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f9926803640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f9918000f80 tx=0x7f991800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.155+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800eb30 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.156+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918018070 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.156+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9920158540 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.156+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918013420 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.156+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99200c4920 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.157+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920155c10 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.157+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99180135c0 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.157+0000 7f99167fc640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.158+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f991808a5f0 con 0x7f99201eb850 2024-03-06T07:31:56.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.158+0000 7f9926002640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:56.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.158+0000 7f9926002640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 0x7f990406d9c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9920155a50 tx=0x7f9910009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:56.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.162+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9920155c10 con 0x7f99201eb850 2024-03-06T07:31:56.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.379+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f992014de60 con 0x7f99201eb850 2024-03-06T07:31:56.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.379+0000 7f99167fc640 1 -- 172.21.15.17:0/1641618396 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f992014de60 con 0x7f99201eb850 2024-03-06T07:31:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:56.381 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:56.381 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:56.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 msgr2=0x7f990406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 0x7f990406d9c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f9920155a50 tx=0x7f9910009190 comp rx=0 tx=0).stop 2024-03-06T07:31:56.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 msgr2=0x7f992014bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:56.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f9918000f80 tx=0x7f991800e530 comp rx=0 tx=0).stop 2024-03-06T07:31:56.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 shutdown_connections 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99201eb850 0x7f992014bbe0 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99200c21e0 0x7f9920157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 --2- 172.21.15.17:0/1641618396 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f990c0040d0 0x7f99201dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 >> 172.21.15.17:0/1641618396 conn(0x7f99200ac570 msgr2=0x7f992014d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 shutdown_connections 2024-03-06T07:31:56.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:56.381+0000 7f9928266640 1 -- 172.21.15.17:0/1641618396 wait complete. 2024-03-06T07:31:56.796 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:56.796 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:57.797 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:57.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.896+0000 7f42e7d77640 1 Processor -- start 2024-03-06T07:31:57.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.896+0000 7f42e7d77640 1 -- start start 2024-03-06T07:31:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.897+0000 7f42e7d77640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42e00c7180 0x7f42e0159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.897+0000 7f42e7d77640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e00c88b0 0x7f42e00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.897+0000 7f42e7d77640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00c7c60 con 0x7f42e01527b0 2024-03-06T07:31:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.897+0000 7f42e7d77640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e0159640 con 0x7f42e00c7180 2024-03-06T07:31:57.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.897+0000 7f42e7d77640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00c8060 con 0x7f42e00c88b0 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42e00c7180 0x7f42e0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e6314640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e00c88b0 0x7f42e00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5b13640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f42e01527b0 0x7f42e0155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:57844/0 (socket says 172.21.15.17:57844) 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5b13640 1 -- 172.21.15.17:0/2055465357 learned_addr learned my addr 172.21.15.17:0/2055465357 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42e00c7180 0x7f42e0159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:34032/0 (socket says 172.21.15.17:34032) 2024-03-06T07:31:57.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 -- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e00c88b0 msgr2=0x7f42e00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 --2- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e00c88b0 0x7f42e00c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 -- 172.21.15.17:0/2055465357 >> v1:172.21.15.163:6789/0 conn(0x7f42e01527b0 legacy=0x7f42e0155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e5312640 1 -- 172.21.15.17:0/2055465357 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00c76a0 con 0x7f42e00c7180 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.898+0000 7f42e4b11640 1 -- 172.21.15.17:0/2055465357 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631484799 0 0) 0x7f42d8001110 con 0x7f42e01527b0 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.899+0000 7f42e5312640 1 --2- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e0159100 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f42d40097a0 tx=0x7f42d402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be026c9eaa4b930e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.899+0000 7f42e4b11640 1 -- 172.21.15.17:0/2055465357 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40315a0 con 0x7f42e00c7180 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.899+0000 7f42e4b11640 1 -- 172.21.15.17:0/2055465357 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4037070 con 0x7f42e00c7180 2024-03-06T07:31:57.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.899+0000 7f42e4b11640 1 -- 172.21.15.17:0/2055465357 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4031960 con 0x7f42e00c7180 2024-03-06T07:31:57.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 -- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 msgr2=0x7f42e0159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 --2- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e0159100 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f42d40097a0 tx=0x7f42d402b840 comp rx=0 tx=0).stop 2024-03-06T07:31:57.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 -- 172.21.15.17:0/2055465357 shutdown_connections 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 --2- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e00c88b0 0x7f42e00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 --2- 172.21.15.17:0/2055465357 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e0159100 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 -- 172.21.15.17:0/2055465357 >> 172.21.15.17:0/2055465357 conn(0x7f42e00ac560 msgr2=0x7f42e00ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 -- 172.21.15.17:0/2055465357 shutdown_connections 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.900+0000 7f42e7d77640 1 -- 172.21.15.17:0/2055465357 wait complete. 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.901+0000 7f42e7d77640 1 Processor -- start 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.901+0000 7f42e7d77640 1 -- start start 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e01df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e01527b0 0x7f42e00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0155600 con 0x7f42e00c88b0 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0155320 con 0x7f42e00c7180 2024-03-06T07:31:57.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e7d77640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00c8d60 con 0x7f42e01527b0 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52034/0 (socket says 172.21.15.17:52034) 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 -- 172.21.15.17:0/5656827 learned_addr learned my addr 172.21.15.17:0/5656827 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5b13640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e01df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e6314640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e01527b0 0x7f42e00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 -- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e01527b0 msgr2=0x7f42e00affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e01527b0 0x7f42e00affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 -- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 msgr2=0x7f42e01df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e01df0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5312640 1 -- 172.21.15.17:0/5656827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e01e0b40 con 0x7f42e00c88b0 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.902+0000 7f42e5b13640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e01df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:31:57.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42e5312640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f42d4034040 tx=0x7f42d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4032630 con 0x7f42e00c88b0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d40327d0 con 0x7f42e00c88b0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e0154360 con 0x7f42e00c88b0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4032bc0 con 0x7f42e00c88b0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.903+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e01e0de0 con 0x7f42e00c88b0 2024-03-06T07:31:57.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.904+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e004f330 con 0x7f42e00c88b0 2024-03-06T07:31:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.904+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42d404e430 con 0x7f42e00c88b0 2024-03-06T07:31:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.905+0000 7f42de7fc640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.905+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f42d402f030 con 0x7f42e00c88b0 2024-03-06T07:31:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.905+0000 7f42e5b13640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:57.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.905+0000 7f42e5b13640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000f60 tx=0x7f42d8007a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:57.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:57.909+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f42d4040070 con 0x7f42e00c88b0 2024-03-06T07:31:58.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.129+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42e01e0910 con 0x7f42e00c88b0 2024-03-06T07:31:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.129+0000 7f42de7fc640 1 -- 172.21.15.17:0/5656827 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f42d407aec0 con 0x7f42e00c88b0 2024-03-06T07:31:58.131 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:31:58.131 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:31:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 msgr2=0x7f42cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000f60 tx=0x7f42d8007a10 comp rx=0 tx=0).stop 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 msgr2=0x7f42e0153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f42d4034040 tx=0x7f42d40046c0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 shutdown_connections 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f42cc06b4d0 0x7f42cc06d990 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42e01527b0 0x7f42e00affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42e00c88b0 0x7f42e0153d10 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 --2- 172.21.15.17:0/5656827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42e00c7180 0x7f42e01df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 >> 172.21.15.17:0/5656827 conn(0x7f42e00ac560 msgr2=0x7f42e0158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 shutdown_connections 2024-03-06T07:31:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:58.131+0000 7f42e7d77640 1 -- 172.21.15.17:0/5656827 wait complete. 2024-03-06T07:31:58.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:31:58.546 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:31:59.548 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:31:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.649+0000 7f6f7cde9640 1 Processor -- start 2024-03-06T07:31:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.649+0000 7f6f7cde9640 1 -- start start 2024-03-06T07:31:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f7cde9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f780c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f7cde9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6f781586f0 0x7f6f78158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f7cde9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f781591c0 con 0x7f6f780c5480 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f7cde9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7806df40 con 0x7f6f781586f0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f7cde9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7806fdb0 con 0x7f6f78155ba0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6f781586f0 0x7f6f78158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f6f781586f0 0x7f6f78158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52044/0 (socket says 172.21.15.17:52044) 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 -- 172.21.15.17:0/3314830434 learned_addr learned my addr 172.21.15.17:0/3314830434 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f6ffff640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f780c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 -- 172.21.15.17:0/3314830434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 msgr2=0x7f6f780c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f780c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 -- 172.21.15.17:0/3314830434 >> v1:172.21.15.163:6789/0 conn(0x7f6f780c5480 legacy=0x7f6f7806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.650+0000 7f6f76d9d640 1 -- 172.21.15.17:0/3314830434 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f781594a0 con 0x7f6f781586f0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.651+0000 7f6f76d9d640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f78158b80 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895d84030f980a50 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:59.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.651+0000 7f6f75d9b640 1 -- 172.21.15.17:0/3314830434 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7002ca70 con 0x7f6f781586f0 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.651+0000 7f6f75d9b640 1 -- 172.21.15.17:0/3314830434 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7002cc10 con 0x7f6f781586f0 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.651+0000 7f6f75d9b640 1 -- 172.21.15.17:0/3314830434 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700343f0 con 0x7f6f781586f0 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 -- 172.21.15.17:0/3314830434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 msgr2=0x7f6f78158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f78158b80 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f7002bed0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 -- 172.21.15.17:0/3314830434 shutdown_connections 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f78158b80 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/3314830434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f780c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 -- 172.21.15.17:0/3314830434 >> 172.21.15.17:0/3314830434 conn(0x7f6f7806b1a0 msgr2=0x7f6f7806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 -- 172.21.15.17:0/3314830434 shutdown_connections 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.652+0000 7f6f7cde9640 1 -- 172.21.15.17:0/3314830434 wait complete. 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.653+0000 7f6f7cde9640 1 Processor -- start 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.653+0000 7f6f7cde9640 1 -- start start 2024-03-06T07:31:59.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f780c5480 0x7f6f781df010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f78157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78155680 con 0x7f6f781586f0 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78156000 con 0x7f6f780c5480 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7cde9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78152a20 con 0x7f6f78155ba0 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52052/0 (socket says 172.21.15.17:52052) 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 -- 172.21.15.17:0/79903496 learned_addr learned my addr 172.21.15.17:0/79903496 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f7659c640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f780c5480 0x7f6f781df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f6ffff640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f78157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 -- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 msgr2=0x7f6f78157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f78157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 -- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f780c5480 msgr2=0x7f6f781df010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f780c5480 0x7f6f781df010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 -- 172.21.15.17:0/79903496 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780c4d40 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.654+0000 7f6f76d9d640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.655+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70044020 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.655+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f700406b0 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.655+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f781eb270 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.655+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70009510 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.655+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f781eb830 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.656+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f781eba30 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.656+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f70047020 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.656+0000 7f6f6effd640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 0x7f6f5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.656+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f6f700acd80 con 0x7f6f781586f0 2024-03-06T07:31:59.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.657+0000 7f6f7659c640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 0x7f6f5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:31:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.657+0000 7f6f7659c640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 0x7f6f5806d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6f780aaf00 tx=0x7f6f68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:31:59.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:31:59.661+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f781eba30 con 0x7f6f781586f0 2024-03-06T07:32:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.040+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f781eb060 con 0x7f6f781586f0 2024-03-06T07:32:00.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.040+0000 7f6f6effd640 1 -- 172.21.15.17:0/79903496 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f6f7004e030 con 0x7f6f781586f0 2024-03-06T07:32:00.042 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:00.042 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:00.043 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.043+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 msgr2=0x7f6f5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.043+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 0x7f6f5806d990 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f6f780aaf00 tx=0x7f6f68009040 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.043+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 msgr2=0x7f6f781ee630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.043+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009e50 tx=0x7f6f70004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 shutdown_connections 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f6f5806b4d0 0x7f6f5806d990 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f781586f0 0x7f6f781ee630 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f78155ba0 0x7f6f78157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 --2- 172.21.15.17:0/79903496 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f780c5480 0x7f6f781df010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 >> 172.21.15.17:0/79903496 conn(0x7f6f7806b1a0 msgr2=0x7f6f78152ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:00.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 shutdown_connections 2024-03-06T07:32:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:00.044+0000 7f6f7cde9640 1 -- 172.21.15.17:0/79903496 wait complete. 2024-03-06T07:32:00.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:00.459 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:01.460 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:01.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.561+0000 7f234a8b5640 1 Processor -- start 2024-03-06T07:32:01.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.561+0000 7f234a8b5640 1 -- start start 2024-03-06T07:32:01.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f234a8b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f2344159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f234a8b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23440c7c60 con 0x7f23440c7180 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f234a8b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23440c8060 con 0x7f23440c88b0 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f234a8b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2344159640 con 0x7f23441527b0 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f23437fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f23440c7180 0x7f2344155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42790/0 (socket says 172.21.15.17:42790) 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f23437fe640 1 -- 172.21.15.17:0/1190462870 learned_addr learned my addr 172.21.15.17:0/1190462870 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f2343fff640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f2344159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440113390 0 0) 0x7f2338001110 con 0x7f23440c7180 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.562+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f232c003520 con 0x7f23440c7180 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525523610 0 0) 0x7f232c003520 con 0x7f23440c7180 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 msgr2=0x7f2344159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f2344159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 >> v1:172.21.15.163:6789/0 conn(0x7f23440c88b0 legacy=0x7f23440c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440c76a0 con 0x7f23440c7180 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f2338001840 con 0x7f23440c7180 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 >> v1:172.21.15.17:6789/0 conn(0x7f23440c7180 legacy=0x7f2344155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f232c0039e0 0x7f232c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f232c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f232c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344159640 con 0x7f232c004d40 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232c00ba70 con 0x7f232c0039e0 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232c00bc70 con 0x7f23441527b0 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f232c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2343fff640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f232c0039e0 0x7f232c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f23437fe640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f232c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 -- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 msgr2=0x7f232c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23441527b0 0x7f232c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 -- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f232c0039e0 msgr2=0x7f232c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f232c0039e0 0x7f232c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.563+0000 7f2348e52640 1 -- 172.21.15.17:0/1190462870 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c003520 con 0x7f232c004d40 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.564+0000 7f2348e52640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f232c0051c0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f233c002010 tx=0x7f232c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff9c3f6eda77e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.564+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c039070 con 0x7f232c004d40 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.564+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c02c750 con 0x7f232c004d40 2024-03-06T07:32:01.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.564+0000 7f2342ffd640 1 -- 172.21.15.17:0/1190462870 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c001be0 con 0x7f232c004d40 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 -- 172.21.15.17:0/1190462870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 msgr2=0x7f232c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 --2- 172.21.15.17:0/1190462870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f232c0051c0 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7f233c002010 tx=0x7f232c005830 comp rx=0 tx=0).stop 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f2348e52640 1 -- 172.21.15.17:0/1190462870 reap_dead start 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 -- 172.21.15.17:0/1190462870 shutdown_connections 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 -- 172.21.15.17:0/1190462870 >> 172.21.15.17:0/1190462870 conn(0x7f23440ac560 msgr2=0x7f23440ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 -- 172.21.15.17:0/1190462870 shutdown_connections 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.565+0000 7f234a8b5640 1 -- 172.21.15.17:0/1190462870 wait complete. 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.567+0000 7f234a8b5640 1 Processor -- start 2024-03-06T07:32:01.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.567+0000 7f234a8b5640 1 -- start start 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2344151a20 0x7f23441eb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 0x7f234406b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344155600 con 0x7f232c004d40 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344155320 con 0x7f23441eddc0 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f234a8b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23440c8d60 con 0x7f2344151a20 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32902/0 (socket says 172.21.15.17:32902) 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 -- 172.21.15.17:0/4110159209 learned_addr learned my addr 172.21.15.17:0/4110159209 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 -- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2344151a20 msgr2=0x7f23441eb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2344151a20 0x7f23441eb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 -- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 msgr2=0x7f234406b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2348e52640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 0x7f234406b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 0x7f234406b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2343fff640 1 -- 172.21.15.17:0/4110159209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23440c1b10 con 0x7f232c004d40 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.568+0000 7f2348e52640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 0x7f234406b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f2343fff640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f23340119a0 tx=0x7f2334011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400faf0 con 0x7f232c004d40 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400fc90 con 0x7f232c004d40 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23441eb7a0 con 0x7f232c004d40 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334014440 con 0x7f232c004d40 2024-03-06T07:32:01.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.569+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23441eaa60 con 0x7f232c004d40 2024-03-06T07:32:01.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.570+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23440c4870 con 0x7f232c004d40 2024-03-06T07:32:01.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.570+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2334012040 con 0x7f232c004d40 2024-03-06T07:32:01.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.571+0000 7f2340ff9640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:01.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.571+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f233408aae0 con 0x7f232c004d40 2024-03-06T07:32:01.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.571+0000 7f23437fe640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:01.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.571+0000 7f23437fe640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 0x7f233006d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2338001e40 tx=0x7f2338001cf0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:01.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.575+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23440c4870 con 0x7f232c004d40 2024-03-06T07:32:01.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.793+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23441ea850 con 0x7f232c004d40 2024-03-06T07:32:01.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.794+0000 7f2340ff9640 1 -- 172.21.15.17:0/4110159209 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f2334019030 con 0x7f232c004d40 2024-03-06T07:32:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:01.796 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:01.796 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 msgr2=0x7f233006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 0x7f233006d990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2338001e40 tx=0x7f2338001cf0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 msgr2=0x7f23440c1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f23340119a0 tx=0x7f2334011e60 comp rx=0 tx=0).stop 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 shutdown_connections 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f233006b4d0 0x7f233006d990 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23441eddc0 0x7f234406b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2344151a20 0x7f23441eb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 --2- 172.21.15.17:0/4110159209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f232c004d40 0x7f23440c1390 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 >> 172.21.15.17:0/4110159209 conn(0x7f23440ac560 msgr2=0x7f234414c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 shutdown_connections 2024-03-06T07:32:01.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:01.796+0000 7f234a8b5640 1 -- 172.21.15.17:0/4110159209 wait complete. 2024-03-06T07:32:02.210 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:02.211 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:03.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.311+0000 7ff43bbe9640 1 Processor -- start 2024-03-06T07:32:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff43bbe9640 1 -- start start 2024-03-06T07:32:03.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff43bbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff43bbe9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340c73d0 con 0x7ff4340c88c0 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff43bbe9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340be2e0 con 0x7ff43414f360 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff43bbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff434159700 con 0x7ff434159270 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff439184640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff4340c88c0 0x7ff434158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42806/0 (socket says 172.21.15.17:42806) 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.312+0000 7ff439184640 1 -- 172.21.15.17:0/503442572 learned_addr learned my addr 172.21.15.17:0/503442572 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff43a186640 1 --2- 172.21.15.17:0/503442572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900268468 0 0) 0x7ff4340c73d0 con 0x7ff4340c88c0 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff41c003520 con 0x7ff4340c88c0 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023024416 0 0) 0x7ff4340be2e0 con 0x7ff43414f360 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4340c73d0 con 0x7ff43414f360 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff43a186640 1 -- 172.21.15.17:0/503442572 >> v1:172.21.15.163:6789/0 conn(0x7ff43414f360 legacy=0x7ff4340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff43a186640 1 -- 172.21.15.17:0/503442572 >> v1:172.21.15.17:6789/0 conn(0x7ff4340c88c0 legacy=0x7ff434158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793181651 0 0) 0x7ff41c003520 con 0x7ff4340c88c0 2024-03-06T07:32:03.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.313+0000 7ff43a186640 1 -- 172.21.15.17:0/503442572 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340c7670 con 0x7ff434159270 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.314+0000 7ff43a186640 1 --2- 172.21.15.17:0/503442572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434155c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0097d0 tx=0x7ff42c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e88fd7a554f18561 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.314+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c037070 con 0x7ff434159270 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c030510 con 0x7ff434159270 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff438983640 1 -- 172.21.15.17:0/503442572 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428001670 con 0x7ff434159270 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 -- 172.21.15.17:0/503442572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 msgr2=0x7ff434155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/503442572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434155c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0097d0 tx=0x7ff42c02b770 comp rx=0 tx=0).stop 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 -- 172.21.15.17:0/503442572 shutdown_connections 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/503442572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434155c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 -- 172.21.15.17:0/503442572 >> 172.21.15.17:0/503442572 conn(0x7ff4340ac570 msgr2=0x7ff4340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 -- 172.21.15.17:0/503442572 shutdown_connections 2024-03-06T07:32:03.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.315+0000 7ff43bbe9640 1 -- 172.21.15.17:0/503442572 wait complete. 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.316+0000 7ff43bbe9640 1 Processor -- start 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.316+0000 7ff43bbe9640 1 -- start start 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c88c0 0x7ff4340af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c7c30 con 0x7ff43414f360 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c5a50 con 0x7ff4340c88c0 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43bbe9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c5bd0 con 0x7ff434159270 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32912/0 (socket says 172.21.15.17:32912) 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 -- 172.21.15.17:0/2720223970 learned_addr learned my addr 172.21.15.17:0/2720223970 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:03.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff43a186640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439985640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c88c0 0x7ff4340af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 -- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 msgr2=0x7ff434157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 -- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c88c0 msgr2=0x7ff4340af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c88c0 0x7ff4340af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 -- 172.21.15.17:0/2720223970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340abe70 con 0x7ff43414f360 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.317+0000 7ff439184640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7ff428001eb0 tx=0x7ff42800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.318+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428005ad0 con 0x7ff43414f360 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.318+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff428005c70 con 0x7ff43414f360 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.318+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4280072b0 con 0x7ff43414f360 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.318+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff434157e90 con 0x7ff43414f360 2024-03-06T07:32:03.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.318+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4340c43a0 con 0x7ff43414f360 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.319+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4341ee7e0 con 0x7ff43414f360 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.319+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff428007450 con 0x7ff43414f360 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.319+0000 7ff4327fc640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 0x7ff42006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.320+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7ff42808a240 con 0x7ff43414f360 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.320+0000 7ff439985640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 0x7ff42006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:03.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.320+0000 7ff439985640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 0x7ff42006d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff424001d50 tx=0x7ff424009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:03.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.324+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff428010300 con 0x7ff43414f360 2024-03-06T07:32:03.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.561+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43404f330 con 0x7ff43414f360 2024-03-06T07:32:03.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.562+0000 7ff4327fc640 1 -- 172.21.15.17:0/2720223970 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7ff428005e20 con 0x7ff43414f360 2024-03-06T07:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:03.564 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:03.564 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 msgr2=0x7ff42006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 0x7ff42006d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7ff424001d50 tx=0x7ff424009040 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 msgr2=0x7ff4340b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7ff428001eb0 tx=0x7ff42800cbe0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 shutdown_connections 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff42006b4d0 0x7ff42006d990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434159270 0x7ff434157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff43414f360 0x7ff4340b0100 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 --2- 172.21.15.17:0/2720223970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c88c0 0x7ff4340af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 >> 172.21.15.17:0/2720223970 conn(0x7ff4340ac570 msgr2=0x7ff4340ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 shutdown_connections 2024-03-06T07:32:03.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:03.565+0000 7ff43bbe9640 1 -- 172.21.15.17:0/2720223970 wait complete. 2024-03-06T07:32:03.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:03.983 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:04.983 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 Processor -- start 2024-03-06T07:32:05.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 -- start start 2024-03-06T07:32:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f5148155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51480c8a00 0x7f5148158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5148156180 con 0x7f5148159460 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51481587b0 con 0x7f51480c8a00 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.099+0000 7f5150777640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480c7760 con 0x7f5148152920 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514ed14640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5148159460 0x7f51480c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42822/0 (socket says 172.21.15.17:42822) 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514ed14640 1 -- 172.21.15.17:0/3803058214 learned_addr learned my addr 172.21.15.17:0/3803058214 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514e513640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f5148155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894723653 0 0) 0x7f5140001490 con 0x7f5148159460 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134003520 con 0x7f5148159460 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> v2:172.21.15.163:3300/0 conn(0x7f51480c8a00 0x7f5148158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506444187 0 0) 0x7f5134003520 con 0x7f5148159460 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 msgr2=0x7f5148155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f5148155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 >> v2:172.21.15.163:3300/0 conn(0x7f51480c8a00 msgr2=0x7f5148158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 --2- 172.21.15.17:0/3803058214 >> v2:172.21.15.163:3300/0 conn(0x7f51480c8a00 0x7f5148158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480c7a40 con 0x7f5148159460 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f5140001bc0 con 0x7f5148159460 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> v2:172.21.15.163:3300/0 conn(0x7f51480c8a00 0x7f5148158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 >> v1:172.21.15.17:6789/0 conn(0x7f5148159460 legacy=0x7f51480c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f51340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5134003f30 0x7f5134004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51481587b0 con 0x7f5134003f30 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480c7760 con 0x7f5134004d40 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513400bac0 con 0x7f5148152920 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.100+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5134003f30 0x7f5134004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514e513640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f51340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514ed14640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514dd12640 1 -- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 msgr2=0x7f51340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5148152920 0x7f51340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514dd12640 1 -- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134004d40 msgr2=0x7f51340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514dd12640 1 -- 172.21.15.17:0/3803058214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134003520 con 0x7f5134003f30 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.101+0000 7f514ed14640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134004d40 0x7f51340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f514dd12640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5134003f30 0x7f5134004800 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f51380098d0 tx=0x7f5134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a6135a33e9a0f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800f040 con 0x7f5134003f30 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138004230 con 0x7f5134003f30 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f514d511640 1 -- 172.21.15.17:0/3803058214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138003610 con 0x7f5134003f30 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 -- 172.21.15.17:0/3803058214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5134003f30 msgr2=0x7f5134004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 --2- 172.21.15.17:0/3803058214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5134003f30 0x7f5134004800 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f51380098d0 tx=0x7f5134005830 comp rx=0 tx=0).stop 2024-03-06T07:32:05.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f514ed14640 1 -- 172.21.15.17:0/3803058214 reap_dead start 2024-03-06T07:32:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 -- 172.21.15.17:0/3803058214 shutdown_connections 2024-03-06T07:32:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 -- 172.21.15.17:0/3803058214 >> 172.21.15.17:0/3803058214 conn(0x7f51480ac560 msgr2=0x7f51480ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 -- 172.21.15.17:0/3803058214 shutdown_connections 2024-03-06T07:32:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.102+0000 7f5150777640 1 -- 172.21.15.17:0/3803058214 wait complete. 2024-03-06T07:32:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.103+0000 7f5150777640 1 Processor -- start 2024-03-06T07:32:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.103+0000 7f5150777640 1 -- start start 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134003f30 0x7f51480ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51480cac20 0x7f51481ee690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514806d030 con 0x7f51480afdb0 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148158a00 con 0x7f5134003f30 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f5150777640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480c5a40 con 0x7f51480cac20 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32934/0 (socket says 172.21.15.17:32934) 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 -- 172.21.15.17:0/104526109 learned_addr learned my addr 172.21.15.17:0/104526109 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514e513640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134003f30 0x7f51480ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 -- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51480cac20 msgr2=0x7f51481ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51480cac20 0x7f51481ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 -- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134003f30 msgr2=0x7f51480ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134003f30 0x7f51480ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.104+0000 7f514ed14640 1 -- 172.21.15.17:0/104526109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148155f20 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f514ed14640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f51480cba80 tx=0x7f5140007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000aba0 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000ad40 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5148155c70 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140014670 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.105+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51481ef1b0 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.106+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51480c8580 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.106+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5140014810 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.107+0000 7f513effd640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 0x7f512c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.107+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f514008a3e0 con 0x7f51480afdb0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.107+0000 7f514e513640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 0x7f512c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:05.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.108+0000 7f514e513640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 0x7f512c06d9c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5148155ed0 tx=0x7f5144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.111+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51480c8580 con 0x7f51480afdb0 2024-03-06T07:32:05.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.331+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f514804f330 con 0x7f51480afdb0 2024-03-06T07:32:05.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.331+0000 7f513effd640 1 -- 172.21.15.17:0/104526109 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f5148156180 con 0x7f51480afdb0 2024-03-06T07:32:05.333 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:05.333 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:05.333 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 msgr2=0x7f512c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 0x7f512c06d9c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5148155ed0 tx=0x7f5144008040 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 msgr2=0x7f51480ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7f51480cba80 tx=0x7f5140007bc0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 shutdown_connections 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f512c06b500 0x7f512c06d9c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51480afdb0 0x7f51480ae020 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51480cac20 0x7f51481ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 --2- 172.21.15.17:0/104526109 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5134003f30 0x7f51480ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 >> 172.21.15.17:0/104526109 conn(0x7f51480ac560 msgr2=0x7f51480c5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 shutdown_connections 2024-03-06T07:32:05.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:05.333+0000 7f5150777640 1 -- 172.21.15.17:0/104526109 wait complete. 2024-03-06T07:32:05.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:05.748 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:06.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:06.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.848+0000 7f9232902640 1 Processor -- start 2024-03-06T07:32:06.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.848+0000 7f9232902640 1 -- start start 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9232902640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f922c14f360 0x7f922c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9232902640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f922c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9232902640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c0c73d0 con 0x7f922c0c88c0 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9232902640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0c8dd0 con 0x7f922c14f360 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9232902640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0be2e0 con 0x7f922c158b60 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f922b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f922c0c88c0 0x7f922c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42836/0 (socket says 172.21.15.17:42836) 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f922b7fe640 1 -- 172.21.15.17:0/1175196417 learned_addr learned my addr 172.21.15.17:0/1175196417 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f922bfff640 1 --2- 172.21.15.17:0/1175196417 >> v2:172.21.15.163:3300/0 conn(0x7f922c14f360 0x7f922c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f9230e9f640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f922c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934678580 0 0) 0x7f922c0c73d0 con 0x7f922c0c88c0 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.849+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9214003520 con 0x7f922c0c88c0 2024-03-06T07:32:06.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3236025498 0 0) 0x7f9214003520 con 0x7f922c0c88c0 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 msgr2=0x7f922c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f922c158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 >> v2:172.21.15.163:3300/0 conn(0x7f922c14f360 msgr2=0x7f922c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 --2- 172.21.15.17:0/1175196417 >> v2:172.21.15.163:3300/0 conn(0x7f922c14f360 0x7f922c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0c7670 con 0x7f922c0c88c0 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f921c001670 con 0x7f922c0c88c0 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 >> v1:172.21.15.17:6789/0 conn(0x7f922c0c88c0 legacy=0x7f922c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92140039b0 0x7f9214003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f9214004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9214004d40 0x7f92140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0be2e0 con 0x7f9214004d40 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921400b9c0 con 0x7f92140039b0 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921400bb70 con 0x7f922c158b60 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9214004d40 0x7f92140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922b7fe640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f9214004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f922bfff640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92140039b0 0x7f9214003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 -- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 msgr2=0x7f9214004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c158b60 0x7f9214004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 -- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92140039b0 msgr2=0x7f9214003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f92140039b0 0x7f9214003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.850+0000 7f9230e9f640 1 -- 172.21.15.17:0/1175196417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9214003520 con 0x7f9214004d40 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.851+0000 7f9230e9f640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9214004d40 0x7f92140051c0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f9224009fc0 tx=0x7f9214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad90d9cf4b56e3b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.851+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224039070 con 0x7f9214004d40 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.851+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922402cc20 con 0x7f9214004d40 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.851+0000 7f922affd640 1 -- 172.21.15.17:0/1175196417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009c10 con 0x7f9214004d40 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 -- 172.21.15.17:0/1175196417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9214004d40 msgr2=0x7f92140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 --2- 172.21.15.17:0/1175196417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9214004d40 0x7f92140051c0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7f9224009fc0 tx=0x7f9214005830 comp rx=0 tx=0).stop 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9230e9f640 1 -- 172.21.15.17:0/1175196417 reap_dead start 2024-03-06T07:32:06.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 -- 172.21.15.17:0/1175196417 shutdown_connections 2024-03-06T07:32:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 -- 172.21.15.17:0/1175196417 >> 172.21.15.17:0/1175196417 conn(0x7f922c0ac570 msgr2=0x7f922c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 -- 172.21.15.17:0/1175196417 shutdown_connections 2024-03-06T07:32:06.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.852+0000 7f9232902640 1 -- 172.21.15.17:0/1175196417 wait complete. 2024-03-06T07:32:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.853+0000 7f9232902640 1 Processor -- start 2024-03-06T07:32:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.854+0000 7f9232902640 1 -- start start 2024-03-06T07:32:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 0x7f922c1ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c0c1a90 0x7f922c1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c5a50 con 0x7f922c1497e0 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c7c30 con 0x7f922c1ee3a0 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9232902640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c5bd0 con 0x7f922c0c1a90 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32948/0 (socket says 172.21.15.17:32948) 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 -- 172.21.15.17:0/2835354813 learned_addr learned my addr 172.21.15.17:0/2835354813 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f922b7fe640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c0c1a90 0x7f922c1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f922bfff640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 0x7f922c1ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 -- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c0c1a90 msgr2=0x7f922c1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c0c1a90 0x7f922c1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 -- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 msgr2=0x7f922c1ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 0x7f922c1ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:06.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 -- 172.21.15.17:0/2835354813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c154d20 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f922bfff640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 0x7f922c1ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.855+0000 7f9230e9f640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f922400e230 tx=0x7f9224009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.856+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224039030 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.856+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9224043070 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.856+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c14a530 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.856+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224009c10 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.856+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c14a250 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.857+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c04f330 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.857+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9224005d10 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.858+0000 7f9228ff9640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.858+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f92240ac750 con 0x7f922c1497e0 2024-03-06T07:32:06.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.858+0000 7f922bfff640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:06.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.858+0000 7f922bfff640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 0x7f921806d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9220009bc0 tx=0x7f9220008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:06.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:06.862+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f922404d020 con 0x7f922c1497e0 2024-03-06T07:32:07.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.084+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f922c06cda0 con 0x7f922c1497e0 2024-03-06T07:32:07.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.084+0000 7f9228ff9640 1 -- 172.21.15.17:0/2835354813 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f922407b400 con 0x7f922c1497e0 2024-03-06T07:32:07.086 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:07.086 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:07.087 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 msgr2=0x7f921806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 0x7f921806d990 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f9220009bc0 tx=0x7f9220008040 comp rx=0 tx=0).stop 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 msgr2=0x7f922c153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f922400e230 tx=0x7f9224009750 comp rx=0 tx=0).stop 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 shutdown_connections 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f921806b4d0 0x7f921806d990 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f922c1497e0 0x7f922c153bd0 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f922c0c1a90 0x7f922c1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:07.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 --2- 172.21.15.17:0/2835354813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f922c1ee3a0 0x7f922c1ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 >> 172.21.15.17:0/2835354813 conn(0x7f922c0ac570 msgr2=0x7f922c14d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 shutdown_connections 2024-03-06T07:32:07.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:07.087+0000 7f9232902640 1 -- 172.21.15.17:0/2835354813 wait complete. 2024-03-06T07:32:07.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:07.502 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:08.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:08.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 Processor -- start 2024-03-06T07:32:08.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 -- start start 2024-03-06T07:32:08.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f586814f360 0x7f5868155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 0x7f5868158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680c73d0 con 0x7f58680c88c0 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58680c8dd0 con 0x7f586814f360 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586ebc9640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58680be2e0 con 0x7f5868158b60 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f586c965640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f586814f360 0x7f5868155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.604+0000 7f5867fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f58680c88c0 0x7f58680c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42846/0 (socket says 172.21.15.17:42846) 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f5867fff640 1 -- 172.21.15.17:0/1164856855 learned_addr learned my addr 172.21.15.17:0/1164856855 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586d166640 1 --2- 172.21.15.17:0/1164856855 >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 0x7f5868158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586c965640 1 -- 172.21.15.17:0/1164856855 >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 msgr2=0x7f5868158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586c965640 1 --2- 172.21.15.17:0/1164856855 >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 0x7f5868158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586c965640 1 -- 172.21.15.17:0/1164856855 >> v1:172.21.15.17:6789/0 conn(0x7f58680c88c0 legacy=0x7f58680c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586c965640 1 -- 172.21.15.17:0/1164856855 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58680c7670 con 0x7f586814f360 2024-03-06T07:32:08.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.605+0000 7f586d166640 1 --2- 172.21.15.17:0/1164856855 >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 0x7f5868158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586c965640 1 --2- 172.21.15.17:0/1164856855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586814f360 0x7f5868155c10 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f5868152d60 tx=0x7f586002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c87ea02cc5013 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f58677fe640 1 -- 172.21.15.17:0/1164856855 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860039070 con 0x7f586814f360 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f58677fe640 1 -- 172.21.15.17:0/1164856855 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586002cbf0 con 0x7f586814f360 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f58677fe640 1 -- 172.21.15.17:0/1164856855 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860034440 con 0x7f586814f360 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 -- 172.21.15.17:0/1164856855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586814f360 msgr2=0x7f5868155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 --2- 172.21.15.17:0/1164856855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586814f360 0x7f5868155c10 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f5868152d60 tx=0x7f586002be40 comp rx=0 tx=0).stop 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 -- 172.21.15.17:0/1164856855 shutdown_connections 2024-03-06T07:32:08.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 --2- 172.21.15.17:0/1164856855 >> v2:172.21.15.163:3300/0 conn(0x7f5868158b60 0x7f5868158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 --2- 172.21.15.17:0/1164856855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f586814f360 0x7f5868155c10 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 -- 172.21.15.17:0/1164856855 >> 172.21.15.17:0/1164856855 conn(0x7f58680ac570 msgr2=0x7f58680ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 -- 172.21.15.17:0/1164856855 shutdown_connections 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.606+0000 7f586ebc9640 1 -- 172.21.15.17:0/1164856855 wait complete. 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.608+0000 7f586ebc9640 1 Processor -- start 2024-03-06T07:32:08.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.608+0000 7f586ebc9640 1 -- start start 2024-03-06T07:32:08.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 0x7f58681eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5868158b60 0x7f5868151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680c5a50 con 0x7f58680c88c0 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680c7c30 con 0x7f586814f360 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586ebc9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58680c5bd0 con 0x7f5868158b60 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586c965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586c965640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32964/0 (socket says 172.21.15.17:32964) 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.609+0000 7f586c965640 1 -- 172.21.15.17:0/4223138566 learned_addr learned my addr 172.21.15.17:0/4223138566 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586d166640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5868158b60 0x7f5868151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f5867fff640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 0x7f58681eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 -- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5868158b60 msgr2=0x7f5868151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5868158b60 0x7f5868151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 -- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 msgr2=0x7f58681eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 0x7f58681eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 -- 172.21.15.17:0/4223138566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868152480 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f5867fff640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 0x7f58681eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586c965640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f5860037040 tx=0x7f5860004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860039050 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5860045070 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586003ea50 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58681508c0 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.610+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5868151ec0 con 0x7f58680c88c0 2024-03-06T07:32:08.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.611+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586804f330 con 0x7f58680c88c0 2024-03-06T07:32:08.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.611+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f586002ca70 con 0x7f58680c88c0 2024-03-06T07:32:08.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.612+0000 7f58657fa640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 0x7f585406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:08.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.612+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f58600ad240 con 0x7f58680c88c0 2024-03-06T07:32:08.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.612+0000 7f5867fff640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 0x7f585406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:08.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.613+0000 7f5867fff640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 0x7f585406d9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f58680c1790 tx=0x7f585c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:08.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.616+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f586007b990 con 0x7f58680c88c0 2024-03-06T07:32:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.835+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f586814ea20 con 0x7f58680c88c0 2024-03-06T07:32:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.835+0000 7f58657fa640 1 -- 172.21.15.17:0/4223138566 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f586007fa30 con 0x7f58680c88c0 2024-03-06T07:32:08.837 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:08.837 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 msgr2=0x7f585406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 0x7f585406d9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f58680c1790 tx=0x7f585c009040 comp rx=0 tx=0).stop 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 msgr2=0x7f58681ebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f5860037040 tx=0x7f5860004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 shutdown_connections 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f585406b500 0x7f585406d9c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5868158b60 0x7f5868151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f586814f360 0x7f58681eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 --2- 172.21.15.17:0/4223138566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58680c88c0 0x7f58681ebd80 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 >> 172.21.15.17:0/4223138566 conn(0x7f58680ac570 msgr2=0x7f58680c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 shutdown_connections 2024-03-06T07:32:08.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:08.837+0000 7f586ebc9640 1 -- 172.21.15.17:0/4223138566 wait complete. 2024-03-06T07:32:09.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:09.252 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:10.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.352+0000 7f8cc3393640 1 Processor -- start 2024-03-06T07:32:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.352+0000 7f8cc3393640 1 -- start start 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc3393640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cbc0c5860 0x7f8cbc1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc3393640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc1591c0 con 0x7f8cbc155ba0 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc3393640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc1594a0 con 0x7f8cbc0c5480 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc3393640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc06df40 con 0x7f8cbc0c5860 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc1930640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8cbc0c5860 0x7f8cbc1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc092e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8cbc155ba0 0x7f8cbc1586f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42860/0 (socket says 172.21.15.17:42860) 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc092e640 1 -- 172.21.15.17:0/3626591817 learned_addr learned my addr 172.21.15.17:0/3626591817 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc1930640 1 -- 172.21.15.17:0/3626591817 >> v1:172.21.15.163:6789/0 conn(0x7f8cbc0c5480 legacy=0x7f8cbc06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc1930640 1 -- 172.21.15.17:0/3626591817 >> v1:172.21.15.17:6789/0 conn(0x7f8cbc155ba0 legacy=0x7f8cbc1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.353+0000 7f8cc1930640 1 -- 172.21.15.17:0/3626591817 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc1553a0 con 0x7f8cbc0c5860 2024-03-06T07:32:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.354+0000 7f8cc1930640 1 --2- 172.21.15.17:0/3626591817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5860 0x7f8cbc1609e0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb402bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fcb644a455773e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.354+0000 7f8cb3fff640 1 -- 172.21.15.17:0/3626591817 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4039070 con 0x7f8cbc0c5860 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.354+0000 7f8cb3fff640 1 -- 172.21.15.17:0/3626591817 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb402c990 con 0x7f8cbc0c5860 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.354+0000 7f8cb3fff640 1 -- 172.21.15.17:0/3626591817 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb402cd60 con 0x7f8cbc0c5860 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 -- 172.21.15.17:0/3626591817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5860 msgr2=0x7f8cbc1609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 --2- 172.21.15.17:0/3626591817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5860 0x7f8cbc1609e0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009d90 tx=0x7f8cb402bc60 comp rx=0 tx=0).stop 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 -- 172.21.15.17:0/3626591817 shutdown_connections 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 --2- 172.21.15.17:0/3626591817 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5860 0x7f8cbc1609e0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 -- 172.21.15.17:0/3626591817 >> 172.21.15.17:0/3626591817 conn(0x7f8cbc06b1a0 msgr2=0x7f8cbc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 -- 172.21.15.17:0/3626591817 shutdown_connections 2024-03-06T07:32:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.355+0000 7f8cc3393640 1 -- 172.21.15.17:0/3626591817 wait complete. 2024-03-06T07:32:10.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.356+0000 7f8cc3393640 1 Processor -- start 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.356+0000 7f8cc3393640 1 -- start start 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cbc0c5860 0x7f8cbc14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc155ba0 0x7f8cbc14b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0c5cf0 con 0x7f8cbc0c5480 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc06fdb0 con 0x7f8cbc155ba0 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc3393640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc156000 con 0x7f8cbc0c5860 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:32996/0 (socket says 172.21.15.17:32996) 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 -- 172.21.15.17:0/2656430365 learned_addr learned my addr 172.21.15.17:0/2656430365 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc1930640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc155ba0 0x7f8cbc14b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc092e640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cbc0c5860 0x7f8cbc14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 -- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cbc0c5860 msgr2=0x7f8cbc14e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cbc0c5860 0x7f8cbc14e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 -- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc155ba0 msgr2=0x7f8cbc14b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc155ba0 0x7f8cbc14b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 -- 172.21.15.17:0/2656430365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc1e10b0 con 0x7f8cbc0c5480 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cc112f640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c460 tx=0x7f8cb800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.357+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8016020 con 0x7f8cbc0c5480 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.358+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8012870 con 0x7f8cbc0c5480 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.358+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc1e1390 con 0x7f8cbc0c5480 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.358+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8012c60 con 0x7f8cbc0c5480 2024-03-06T07:32:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.358+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc14fa60 con 0x7f8cbc0c5480 2024-03-06T07:32:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.359+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc04f330 con 0x7f8cbc0c5480 2024-03-06T07:32:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.359+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cb8019020 con 0x7f8cbc0c5480 2024-03-06T07:32:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.359+0000 7f8cb1ffb640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.360+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8cb808a250 con 0x7f8cbc0c5480 2024-03-06T07:32:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.360+0000 7f8cc092e640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:10.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.360+0000 7f8cc092e640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06bae0 tx=0x7f8cac007c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:10.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.364+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8cb8020030 con 0x7f8cbc0c5480 2024-03-06T07:32:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.701+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cbc14df00 con 0x7f8cbc0c5480 2024-03-06T07:32:10.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.702+0000 7f8cb1ffb640 1 -- 172.21.15.17:0/2656430365 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f8cb8058da0 con 0x7f8cbc0c5480 2024-03-06T07:32:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:10.704 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:10.704 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.703+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 msgr2=0x7f8ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.703+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 0x7f8ca006d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06bae0 tx=0x7f8cac007c60 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.703+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 msgr2=0x7f8cbc1eaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.703+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800c460 tx=0x7f8cb800c920 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 shutdown_connections 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8ca006b4d0 0x7f8ca006d990 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8cbc155ba0 0x7f8cbc14b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8cbc0c5860 0x7f8cbc14e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 --2- 172.21.15.17:0/2656430365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8cbc0c5480 0x7f8cbc1eaf30 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 >> 172.21.15.17:0/2656430365 conn(0x7f8cbc06b1a0 msgr2=0x7f8cbc06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 shutdown_connections 2024-03-06T07:32:10.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:10.704+0000 7f8cc3393640 1 -- 172.21.15.17:0/2656430365 wait complete. 2024-03-06T07:32:10.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:10.830 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:11.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:11.119 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:12.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 Processor -- start 2024-03-06T07:32:12.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 -- start start 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe934155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe934158b60 0x7fe934158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9340c73d0 con 0x7fe93414f360 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9340c8dd0 con 0x7fe934158b60 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.220+0000 7fe93c49a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340be2e0 con 0x7fe9340c88c0 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe93a236640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe93414f360 0x7fe9340c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51248/0 (socket says 172.21.15.17:51248) 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe93a236640 1 -- 172.21.15.17:0/1528741178 learned_addr learned my addr 172.21.15.17:0/1528741178 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939a35640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe934155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe93aa37640 1 --2- 172.21.15.17:0/1528741178 >> v2:172.21.15.163:3300/0 conn(0x7fe934158b60 0x7fe934158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068715001 0 0) 0x7fe9340c73d0 con 0x7fe93414f360 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe91c003520 con 0x7fe93414f360 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370475630 0 0) 0x7fe91c003520 con 0x7fe93414f360 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 msgr2=0x7fe934155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe934155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 >> v2:172.21.15.163:3300/0 conn(0x7fe934158b60 msgr2=0x7fe934158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 --2- 172.21.15.17:0/1528741178 >> v2:172.21.15.163:3300/0 conn(0x7fe934158b60 0x7fe934158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340c7670 con 0x7fe93414f360 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fe92c001670 con 0x7fe93414f360 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 >> v1:172.21.15.17:6789/0 conn(0x7fe93414f360 legacy=0x7fe9340c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe91c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe91c003f30 0x7fe91c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe91c004d40 0x7fe91c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340be2e0 con 0x7fe91c003f30 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c00b9c0 con 0x7fe91c004d40 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c00bb90 con 0x7fe9340c88c0 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe939a35640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe91c003f30 0x7fe91c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe93a236640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe91c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.221+0000 7fe93aa37640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe91c004d40 0x7fe91c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 -- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 msgr2=0x7fe91c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9340c88c0 0x7fe91c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 -- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe91c004d40 msgr2=0x7fe91c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe91c004d40 0x7fe91c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 -- 172.21.15.17:0/1528741178 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c003520 con 0x7fe91c003f30 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.222+0000 7fe939a35640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe91c003f30 0x7fe91c004800 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fe928009ac0 tx=0x7fe91c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee40349a397dc909 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928039070 con 0x7fe91c003f30 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92802c680 con 0x7fe91c003f30 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe939234640 1 -- 172.21.15.17:0/1528741178 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928009690 con 0x7fe91c003f30 2024-03-06T07:32:12.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 -- 172.21.15.17:0/1528741178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe91c003f30 msgr2=0x7fe91c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 --2- 172.21.15.17:0/1528741178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe91c003f30 0x7fe91c004800 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fe928009ac0 tx=0x7fe91c005830 comp rx=0 tx=0).stop 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93aa37640 1 -- 172.21.15.17:0/1528741178 reap_dead start 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 -- 172.21.15.17:0/1528741178 shutdown_connections 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 -- 172.21.15.17:0/1528741178 >> 172.21.15.17:0/1528741178 conn(0x7fe9340ac570 msgr2=0x7fe9340ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 -- 172.21.15.17:0/1528741178 shutdown_connections 2024-03-06T07:32:12.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.223+0000 7fe93c49a640 1 -- 172.21.15.17:0/1528741178 wait complete. 2024-03-06T07:32:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.224+0000 7fe93c49a640 1 Processor -- start 2024-03-06T07:32:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.224+0000 7fe93c49a640 1 -- start start 2024-03-06T07:32:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe91c003f30 0x7fe934157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340b0350 0x7fe9341eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c5a50 con 0x7fe9341eadf0 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c7c30 con 0x7fe9340b0350 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93c49a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c5bd0 con 0x7fe91c003f30 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51482/0 (socket says 172.21.15.17:51482) 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 -- 172.21.15.17:0/3291309576 learned_addr learned my addr 172.21.15.17:0/3291309576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93a236640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe91c003f30 0x7fe934157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe93aa37640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340b0350 0x7fe9341eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 -- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe91c003f30 msgr2=0x7fe934157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe91c003f30 0x7fe934157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 -- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340b0350 msgr2=0x7fe9341eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340b0350 0x7fe9341eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 -- 172.21.15.17:0/3291309576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340ad700 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.225+0000 7fe939a35640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fe928000c00 tx=0x7fe928004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.226+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928042020 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.226+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92803a040 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.226+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340ad9e0 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.226+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928039070 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.226+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9340adfa0 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.227+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9340ca5e0 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.227+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe928045020 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.227+0000 7fe932ffd640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 0x7fe92006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.228+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe9280ab880 con 0x7fe9341eadf0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.228+0000 7fe93a236640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 0x7fe92006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:12.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.228+0000 7fe93a236640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 0x7fe92006d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe9341e8c30 tx=0x7fe92c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:12.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.232+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe9340ca5e0 con 0x7fe9341eadf0 2024-03-06T07:32:12.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.452+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe93404f330 con 0x7fe9341eadf0 2024-03-06T07:32:12.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.452+0000 7fe932ffd640 1 -- 172.21.15.17:0/3291309576 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fe92804c070 con 0x7fe9341eadf0 2024-03-06T07:32:12.454 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:12.454 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:12.455 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:12.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.454+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 msgr2=0x7fe92006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 0x7fe92006d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe9341e8c30 tx=0x7fe92c009040 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 msgr2=0x7fe9341ee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7fe928000c00 tx=0x7fe928004730 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 shutdown_connections 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe92006b4d0 0x7fe92006d990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe9340b0350 0x7fe9341eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9341eadf0 0x7fe9341ee920 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 --2- 172.21.15.17:0/3291309576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe91c003f30 0x7fe934157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 >> 172.21.15.17:0/3291309576 conn(0x7fe9340ac570 msgr2=0x7fe93414c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 shutdown_connections 2024-03-06T07:32:12.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:12.455+0000 7fe93c49a640 1 -- 172.21.15.17:0/3291309576 wait complete. 2024-03-06T07:32:12.869 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:12.869 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:13.871 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:13.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.970+0000 7f4458ab5640 1 Processor -- start 2024-03-06T07:32:13.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 -- start start 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4454158b60 0x7f4454158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540c73d0 con 0x7f445414f360 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540c8dd0 con 0x7f44540c88c0 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f4458ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540be2e0 con 0x7f4454158b60 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f444bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f445259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f445414f360 0x7f44540c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51260/0 (socket says 172.21.15.17:51260) 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.971+0000 7f445259c640 1 -- 172.21.15.17:0/4028511116 learned_addr learned my addr 172.21.15.17:0/4028511116 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f4452d9d640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4454158b60 0x7f4454158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f444bfff640 1 -- 172.21.15.17:0/4028511116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4454158b60 msgr2=0x7f4454158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f444bfff640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4454158b60 0x7f4454158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f444bfff640 1 -- 172.21.15.17:0/4028511116 >> v1:172.21.15.17:6789/0 conn(0x7f445414f360 legacy=0x7f44540c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f444bfff640 1 -- 172.21.15.17:0/4028511116 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540c7670 con 0x7f44540c88c0 2024-03-06T07:32:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.972+0000 7f4451d9b640 1 -- 172.21.15.17:0/4028511116 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178208434 0 0) 0x7f44540c73d0 con 0x7f445414f360 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.973+0000 7f444bfff640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f4454155c10 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097a0 tx=0x7f443c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40de7fe39bc3ab66 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.973+0000 7f4451d9b640 1 -- 172.21.15.17:0/4028511116 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0315a0 con 0x7f44540c88c0 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.973+0000 7f4451d9b640 1 -- 172.21.15.17:0/4028511116 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c037070 con 0x7f44540c88c0 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.973+0000 7f4451d9b640 1 -- 172.21.15.17:0/4028511116 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c031960 con 0x7f44540c88c0 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 -- 172.21.15.17:0/4028511116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 msgr2=0x7f4454155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f4454155c10 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097a0 tx=0x7f443c02b840 comp rx=0 tx=0).stop 2024-03-06T07:32:13.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 -- 172.21.15.17:0/4028511116 shutdown_connections 2024-03-06T07:32:13.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4454158b60 0x7f4454158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 --2- 172.21.15.17:0/4028511116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f4454155c10 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 -- 172.21.15.17:0/4028511116 >> 172.21.15.17:0/4028511116 conn(0x7f44540ac570 msgr2=0x7f44540ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:13.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 -- 172.21.15.17:0/4028511116 shutdown_connections 2024-03-06T07:32:13.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.974+0000 7f4458ab5640 1 -- 172.21.15.17:0/4028511116 wait complete. 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.975+0000 7f4458ab5640 1 Processor -- start 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.975+0000 7f4458ab5640 1 -- start start 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 0x7f4454157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 0x7f44540c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c5a50 con 0x7f44540c88c0 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c7c30 con 0x7f4454158b60 2024-03-06T07:32:13.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540c5bd0 con 0x7f445414f360 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51500/0 (socket says 172.21.15.17:51500) 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 -- 172.21.15.17:0/1083699512 learned_addr learned my addr 172.21.15.17:0/1083699512 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f444bfff640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 0x7f4454157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4452d9d640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 0x7f44540c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 -- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 msgr2=0x7f4454157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 0x7f4454157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 -- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 msgr2=0x7f44540c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 0x7f44540c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 -- 172.21.15.17:0/1083699512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540c0930 con 0x7f44540c88c0 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4452d9d640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 0x7f44540c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f444bfff640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 0x7f4454157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:13.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f445259c640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f444400c4d0 tx=0x7f444400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444018070 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4444007500 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.976+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540c0c10 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.977+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444004a20 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.977+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44541581c0 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.977+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445404f330 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.978+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f444401b070 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.978+0000 7f444affd640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 0x7f443406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.979+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f4444089860 con 0x7f44540c88c0 2024-03-06T07:32:13.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.979+0000 7f444bfff640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 0x7f443406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:13.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.979+0000 7f444bfff640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 0x7f443406d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f443c036000 tx=0x7f443c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:13.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:13.987+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44440583c0 con 0x7f44540c88c0 2024-03-06T07:32:14.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.288+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f445406cda0 con 0x7f44540c88c0 2024-03-06T07:32:14.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.289+0000 7f444affd640 1 -- 172.21.15.17:0/1083699512 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f4444057cc0 con 0x7f44540c88c0 2024-03-06T07:32:14.291 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:14.291 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:14.291 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 msgr2=0x7f443406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 0x7f443406d990 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f443c036000 tx=0x7f443c008030 comp rx=0 tx=0).stop 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 msgr2=0x7f44541ee740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 secure :-1 s=READY pgs=2608 cs=0 l=1 rev1=1 crypto rx=0x7f444400c4d0 tx=0x7f444400c990 comp rx=0 tx=0).stop 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 shutdown_connections 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f443406b4d0 0x7f443406d990 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:14.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4454158b60 0x7f44540c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:14.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f445414f360 0x7f4454157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:14.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 --2- 172.21.15.17:0/1083699512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44540c88c0 0x7f44541ee740 unknown :-1 s=CLOSED pgs=2608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:14.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 >> 172.21.15.17:0/1083699512 conn(0x7f44540ac570 msgr2=0x7f44540c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:14.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 shutdown_connections 2024-03-06T07:32:14.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:14.291+0000 7f4458ab5640 1 -- 172.21.15.17:0/1083699512 wait complete. 2024-03-06T07:32:14.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:14.707 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:15.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.807+0000 7fbedf781640 1 Processor -- start 2024-03-06T07:32:15.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbedf781640 1 -- start start 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbedf781640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbed8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbedf781640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed80c73d0 con 0x7fbed8158b60 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbedf781640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed80be2e0 con 0x7fbed80c88c0 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbedf781640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8159700 con 0x7fbed814f360 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbeddd1e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbed8158b60 0x7fbed8158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:51262/0 (socket says 172.21.15.17:51262) 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.808+0000 7fbeddd1e640 1 -- 172.21.15.17:0/3205115735 learned_addr learned my addr 172.21.15.17:0/3205115735 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125169609 0 0) 0x7fbed80c73d0 con 0x7fbed8158b60 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbedd51d640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbed8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec0003520 con 0x7fbed8158b60 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539688641 0 0) 0x7fbec0003520 con 0x7fbed8158b60 2024-03-06T07:32:15.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 msgr2=0x7fbed8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbed8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 >> v1:172.21.15.163:6789/0 conn(0x7fbed80c88c0 legacy=0x7fbed80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80c7670 con 0x7fbed8158b60 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fbed0001670 con 0x7fbed8158b60 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 >> v1:172.21.15.17:6789/0 conn(0x7fbed8158b60 legacy=0x7fbed8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbec0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbec0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec0004d40 0x7fbec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8159700 con 0x7fbec00039e0 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec000baf0 con 0x7fbec0004d40 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec000bcf0 con 0x7fbed814f360 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbedd51d640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbec0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbedcd1c640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbec0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbeddd1e640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec0004d40 0x7fbec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.809+0000 7fbedd51d640 1 -- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 msgr2=0x7fbec0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbedd51d640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed814f360 0x7fbec0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbedd51d640 1 -- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec0004d40 msgr2=0x7fbec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbedd51d640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbec0004d40 0x7fbec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbedd51d640 1 -- 172.21.15.17:0/3205115735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec0003520 con 0x7fbec00039e0 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbedd51d640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbec0003e00 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009fc0 tx=0x7fbec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6a271672b28f13 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8039070 con 0x7fbec00039e0 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec802cb40 con 0x7fbec00039e0 2024-03-06T07:32:15.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.810+0000 7fbed7fff640 1 -- 172.21.15.17:0/3205115735 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8009c10 con 0x7fbec00039e0 2024-03-06T07:32:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 -- 172.21.15.17:0/3205115735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 msgr2=0x7fbec0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 --2- 172.21.15.17:0/3205115735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbec0003e00 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009fc0 tx=0x7fbec0005830 comp rx=0 tx=0).stop 2024-03-06T07:32:15.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbeddd1e640 1 -- 172.21.15.17:0/3205115735 reap_dead start 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 -- 172.21.15.17:0/3205115735 shutdown_connections 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 -- 172.21.15.17:0/3205115735 >> 172.21.15.17:0/3205115735 conn(0x7fbed80ac570 msgr2=0x7fbed80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 -- 172.21.15.17:0/3205115735 shutdown_connections 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.811+0000 7fbedf781640 1 -- 172.21.15.17:0/3205115735 wait complete. 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.812+0000 7fbedf781640 1 Processor -- start 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.812+0000 7fbedf781640 1 -- start start 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed80be760 0x7fbed814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed814b210 0x7fbed81565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c7c30 con 0x7fbec00039e0 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c5a50 con 0x7fbed814b210 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedf781640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed80c5bd0 con 0x7fbed80be760 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51520/0 (socket says 172.21.15.17:51520) 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 -- 172.21.15.17:0/2851447000 learned_addr learned my addr 172.21.15.17:0/2851447000 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbeddd1e640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed814b210 0x7fbed81565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedcd1c640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed80be760 0x7fbed814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 -- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed80be760 msgr2=0x7fbed814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed80be760 0x7fbed814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 -- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed814b210 msgr2=0x7fbed81565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed814b210 0x7fbed81565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 -- 172.21.15.17:0/2851447000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80c0420 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.813+0000 7fbedd51d640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fbec800e170 tx=0x7fbec8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.814+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8039050 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.814+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec803a040 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.814+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed80c0700 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.814+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8009c10 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.814+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed80c0cc0 con 0x7fbec00039e0 2024-03-06T07:32:15.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.815+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed804f330 con 0x7fbec00039e0 2024-03-06T07:32:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.815+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbec8047030 con 0x7fbec00039e0 2024-03-06T07:32:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.816+0000 7fbed5ffb640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.816+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fbec80ac740 con 0x7fbec00039e0 2024-03-06T07:32:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.816+0000 7fbedcd1c640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:15.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.816+0000 7fbedcd1c640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fbed80be1d0 tx=0x7fbecc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:15.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:15.820+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbec804e060 con 0x7fbec00039e0 2024-03-06T07:32:16.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.038+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed80c9ed0 con 0x7fbec00039e0 2024-03-06T07:32:16.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.038+0000 7fbed5ffb640 1 -- 172.21.15.17:0/2851447000 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fbed80c9ed0 con 0x7fbec00039e0 2024-03-06T07:32:16.040 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:16.040 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:16.041 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 msgr2=0x7fbec406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 0x7fbec406d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fbed80be1d0 tx=0x7fbecc009040 comp rx=0 tx=0).stop 2024-03-06T07:32:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 msgr2=0x7fbed814ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fbec800e170 tx=0x7fbec8009750 comp rx=0 tx=0).stop 2024-03-06T07:32:16.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 shutdown_connections 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbec406b4d0 0x7fbec406d990 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbed814b210 0x7fbed81565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbed80be760 0x7fbed814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 --2- 172.21.15.17:0/2851447000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec00039e0 0x7fbed814ba20 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 >> 172.21.15.17:0/2851447000 conn(0x7fbed80ac570 msgr2=0x7fbed814d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 shutdown_connections 2024-03-06T07:32:16.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:16.040+0000 7fbedf781640 1 -- 172.21.15.17:0/2851447000 wait complete. 2024-03-06T07:32:16.455 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:16.455 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:17.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.557+0000 7fcf59d0c640 1 Processor -- start 2024-03-06T07:32:17.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.557+0000 7fcf59d0c640 1 -- start start 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf59d0c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcf541592d0 0x7fcf541552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf59d0c640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf540ac260 0x7fcf54155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf59d0c640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf54152d40 con 0x7fcf54152960 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf59d0c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf54066e30 con 0x7fcf541592d0 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf59d0c640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf54155d50 con 0x7fcf540ac260 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcf541592d0 0x7fcf541552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcf541592d0 0x7fcf541552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51536/0 (socket says 172.21.15.17:51536) 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 -- 172.21.15.17:0/2901715314 learned_addr learned my addr 172.21.15.17:0/2901715314 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf53fff640 1 --2- 172.21.15.17:0/2901715314 >> v2:172.21.15.163:3300/0 conn(0x7fcf540ac260 0x7fcf54155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 -- 172.21.15.17:0/2901715314 >> v2:172.21.15.163:3300/0 conn(0x7fcf540ac260 msgr2=0x7fcf54155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 --2- 172.21.15.17:0/2901715314 >> v2:172.21.15.163:3300/0 conn(0x7fcf540ac260 0x7fcf54155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 -- 172.21.15.17:0/2901715314 >> v1:172.21.15.163:6789/0 conn(0x7fcf54152960 legacy=0x7fcf5406dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.558+0000 7fcf52ffd640 1 -- 172.21.15.17:0/2901715314 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540c1b50 con 0x7fcf541592d0 2024-03-06T07:32:17.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.559+0000 7fcf52ffd640 1 --2- 172.21.15.17:0/2901715314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf541592d0 0x7fcf541552d0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fcf5406b380 tx=0x7fcf4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5ad376167a47b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.559+0000 7fcf527fc640 1 -- 172.21.15.17:0/2901715314 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44035020 con 0x7fcf541592d0 2024-03-06T07:32:17.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.559+0000 7fcf527fc640 1 -- 172.21.15.17:0/2901715314 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44031870 con 0x7fcf541592d0 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.559+0000 7fcf527fc640 1 -- 172.21.15.17:0/2901715314 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44031c40 con 0x7fcf541592d0 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2901715314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf541592d0 msgr2=0x7fcf541552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2901715314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf541592d0 0x7fcf541552d0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fcf5406b380 tx=0x7fcf4402b910 comp rx=0 tx=0).stop 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2901715314 shutdown_connections 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2901715314 >> v2:172.21.15.163:3300/0 conn(0x7fcf540ac260 0x7fcf54155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2901715314 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf541592d0 0x7fcf541552d0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2901715314 >> 172.21.15.17:0/2901715314 conn(0x7fcf540c70b0 msgr2=0x7fcf540c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2901715314 shutdown_connections 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.560+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2901715314 wait complete. 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.561+0000 7fcf59d0c640 1 Processor -- start 2024-03-06T07:32:17.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.561+0000 7fcf59d0c640 1 -- start start 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf540ac260 0x7fcf541ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf54152960 0x7fcf541ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54159760 con 0x7fcf5414bf60 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540672c0 con 0x7fcf540ac260 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf59d0c640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5406e3a0 con 0x7fcf54152960 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51538/0 (socket says 172.21.15.17:51538) 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 -- 172.21.15.17:0/2257234775 learned_addr learned my addr 172.21.15.17:0/2257234775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf52ffd640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf54152960 0x7fcf541ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf537fe640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf540ac260 0x7fcf541ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 -- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf54152960 msgr2=0x7fcf541ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf54152960 0x7fcf541ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 -- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf540ac260 msgr2=0x7fcf541ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf540ac260 0x7fcf541ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 -- 172.21.15.17:0/2257234775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf541ec2e0 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.562+0000 7fcf53fff640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c007e40 tx=0x7fcf4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.563+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c014620 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.563+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c019070 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.563+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf54072810 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.563+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c014a20 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.563+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf541eb5a0 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.564+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54069460 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.564+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf4c014bc0 con 0x7fcf5414bf60 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.564+0000 7fcf3bfff640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 0x7fcf4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:17.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.564+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fcf4c08aca0 con 0x7fcf5414bf60 2024-03-06T07:32:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.565+0000 7fcf537fe640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 0x7fcf4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:17.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.565+0000 7fcf537fe640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 0x7fcf4006d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcf48004f60 tx=0x7fcf48007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:17.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.569+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcf54069460 con 0x7fcf5414bf60 2024-03-06T07:32:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.790+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf540c2c70 con 0x7fcf5414bf60 2024-03-06T07:32:17.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.791+0000 7fcf3bfff640 1 -- 172.21.15.17:0/2257234775 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fcf540c2c70 con 0x7fcf5414bf60 2024-03-06T07:32:17.793 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:17.793 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:17.793 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 msgr2=0x7fcf4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 0x7fcf4006d990 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcf48004f60 tx=0x7fcf48007e30 comp rx=0 tx=0).stop 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 msgr2=0x7fcf541dfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c007e40 tx=0x7fcf4c00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 shutdown_connections 2024-03-06T07:32:17.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fcf4006b4d0 0x7fcf4006d990 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf5414bf60 0x7fcf541dfe00 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf54152960 0x7fcf541ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 --2- 172.21.15.17:0/2257234775 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf540ac260 0x7fcf541ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 >> 172.21.15.17:0/2257234775 conn(0x7fcf540c70b0 msgr2=0x7fcf540c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 shutdown_connections 2024-03-06T07:32:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:17.793+0000 7fcf59d0c640 1 -- 172.21.15.17:0/2257234775 wait complete. 2024-03-06T07:32:18.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:18.208 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:19.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:19.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.312+0000 7f80e8d8a640 1 Processor -- start 2024-03-06T07:32:19.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.312+0000 7f80e8d8a640 1 -- start start 2024-03-06T07:32:19.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e8d8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80e41536c0 0x7f80e406f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e8d8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e415d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e8d8a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e40a70f0 con 0x7f80e406afc0 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e8d8a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e4150a20 con 0x7f80e41536c0 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e8d8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e41565a0 con 0x7f80e406b3a0 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80e41536c0 0x7f80e406f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f80e41536c0 0x7f80e406f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51546/0 (socket says 172.21.15.17:51546) 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 -- 172.21.15.17:0/3274065561 learned_addr learned my addr 172.21.15.17:0/3274065561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80e2d9d640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 -- 172.21.15.17:0/3274065561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 msgr2=0x7f80e415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 -- 172.21.15.17:0/3274065561 >> v1:172.21.15.163:6789/0 conn(0x7f80e406afc0 legacy=0x7f80e4152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.313+0000 7f80dbfff640 1 -- 172.21.15.17:0/3274065561 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e41567f0 con 0x7f80e41536c0 2024-03-06T07:32:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.314+0000 7f80dbfff640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e406f100 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009870 tx=0x7f80cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b02b768357ca8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.314+0000 7f80e1d9b640 1 -- 172.21.15.17:0/3274065561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0317c0 con 0x7f80e41536c0 2024-03-06T07:32:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.314+0000 7f80e1d9b640 1 -- 172.21.15.17:0/3274065561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc037070 con 0x7f80e41536c0 2024-03-06T07:32:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.314+0000 7f80e1d9b640 1 -- 172.21.15.17:0/3274065561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc031b80 con 0x7f80e41536c0 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3274065561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 msgr2=0x7f80e406f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e406f100 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009870 tx=0x7f80cc02b860 comp rx=0 tx=0).stop 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3274065561 shutdown_connections 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3274065561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e406f100 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3274065561 >> 172.21.15.17:0/3274065561 conn(0x7f80e40c4290 msgr2=0x7f80e40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3274065561 shutdown_connections 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.315+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3274065561 wait complete. 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.316+0000 7f80e8d8a640 1 Processor -- start 2024-03-06T07:32:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.316+0000 7f80e8d8a640 1 -- start start 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 0x7f80e41e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e4156db0 con 0x7f80e41536c0 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e406b810 con 0x7f80e406afc0 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e8d8a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e40c7360 con 0x7f80e406b3a0 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.317+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51556/0 (socket says 172.21.15.17:51556) 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 -- 172.21.15.17:0/3855610766 learned_addr learned my addr 172.21.15.17:0/3855610766 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80dbfff640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e259c640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 0x7f80e41e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 -- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 msgr2=0x7f80e406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 -- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 msgr2=0x7f80e41e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 0x7f80e41e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 -- 172.21.15.17:0/3855610766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e40ad040 con 0x7f80e41536c0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80dbfff640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e259c640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 0x7f80e41e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e2d9d640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007e60 tx=0x7f80dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc014420 con 0x7f80e41536c0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc019070 con 0x7f80e41536c0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc014820 con 0x7f80e41536c0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e4155ab0 con 0x7f80e41536c0 2024-03-06T07:32:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.318+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e40acd60 con 0x7f80e41536c0 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.319+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e40a7b60 con 0x7f80e41536c0 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.320+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80dc0149c0 con 0x7f80e41536c0 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.320+0000 7f80daffd640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 0x7f80c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.320+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f80dc08ac10 con 0x7f80e41536c0 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.320+0000 7f80e259c640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 0x7f80c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:19.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.321+0000 7f80e259c640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 0x7f80c406d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f80e4155610 tx=0x7f80d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:19.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.329+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f80e40a7b60 con 0x7f80e41536c0 2024-03-06T07:32:19.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.549+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80e404f330 con 0x7f80e41536c0 2024-03-06T07:32:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.549+0000 7f80daffd640 1 -- 172.21.15.17:0/3855610766 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f80e4155ab0 con 0x7f80e41536c0 2024-03-06T07:32:19.551 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:19.551 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:19.551 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 msgr2=0x7f80c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 0x7f80c406d9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f80e4155610 tx=0x7f80d4009040 comp rx=0 tx=0).stop 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 msgr2=0x7f80e41ecca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007e60 tx=0x7f80dc00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 shutdown_connections 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f80c406b500 0x7f80c406d9c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80e41536c0 0x7f80e41ecca0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80e406b3a0 0x7f80e406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 --2- 172.21.15.17:0/3855610766 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80e406afc0 0x7f80e41e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 >> 172.21.15.17:0/3855610766 conn(0x7f80e40c4290 msgr2=0x7f80e40a8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 shutdown_connections 2024-03-06T07:32:19.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:19.551+0000 7f80e8d8a640 1 -- 172.21.15.17:0/3855610766 wait complete. 2024-03-06T07:32:19.966 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:19.966 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:20.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.067+0000 7f38b84af640 1 Processor -- start 2024-03-06T07:32:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.067+0000 7f38b84af640 1 -- start start 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b84af640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f38b0156ea0 0x7f38b00c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b84af640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b00ad310 con 0x7f38b0153990 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b84af640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b01572a0 con 0x7f38b01506b0 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b84af640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b0156450 con 0x7f38b0156ea0 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b5a4a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f38b0153990 0x7f38b00acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45442/0 (socket says 172.21.15.17:45442) 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.068+0000 7f38b5a4a640 1 -- 172.21.15.17:0/670462709 learned_addr learned my addr 172.21.15.17:0/670462709 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b6a4c640 1 --2- 172.21.15.17:0/670462709 >> v2:172.21.15.163:3300/0 conn(0x7f38b0156ea0 0x7f38b00c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214549400 0 0) 0x7f38b00ad310 con 0x7f38b0153990 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3898003520 con 0x7f38b0153990 2024-03-06T07:32:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918999092 0 0) 0x7f3898003520 con 0x7f38b0153990 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 >> v2:172.21.15.163:3300/0 conn(0x7f38b0156ea0 msgr2=0x7f38b00c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 --2- 172.21.15.17:0/670462709 >> v2:172.21.15.163:3300/0 conn(0x7f38b0156ea0 0x7f38b00c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 >> v1:172.21.15.163:6789/0 conn(0x7f38b01506b0 legacy=0x7f38b00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0070aa0 con 0x7f38b0153990 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f38a4001670 con 0x7f38b0153990 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 >> v1:172.21.15.17:6789/0 conn(0x7f38b0153990 legacy=0x7f38b00acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38980039b0 0x7f3898003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 0x7f3898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38980051c0 0x7f389800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0156450 con 0x7f3898004800 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389800c060 con 0x7f38980051c0 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389800c260 con 0x7f38980039b0 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 0x7f3898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b624b640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38980039b0 0x7f3898003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.069+0000 7f38b6a4c640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38980051c0 0x7f389800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 -- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38980039b0 msgr2=0x7f3898003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38980039b0 0x7f3898003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 -- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38980051c0 msgr2=0x7f389800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38980051c0 0x7f389800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 -- 172.21.15.17:0/670462709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3898003520 con 0x7f3898004800 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b6a4c640 1 -- 172.21.15.17:0/670462709 reap_dead start 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 0x7f3898004c80 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c240 tx=0x7f3898005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f309b31e378dab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4007cf0 con 0x7f3898004800 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4030070 con 0x7f3898004800 2024-03-06T07:32:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.070+0000 7f38b5249640 1 -- 172.21.15.17:0/670462709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005d70 con 0x7f3898004800 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 -- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 msgr2=0x7f3898004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 0x7f3898004c80 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c240 tx=0x7f3898005830 comp rx=0 tx=0).stop 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 -- 172.21.15.17:0/670462709 shutdown_connections 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 --2- 172.21.15.17:0/670462709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3898004800 0x7f3898004c80 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 -- 172.21.15.17:0/670462709 >> 172.21.15.17:0/670462709 conn(0x7f38b006b1a0 msgr2=0x7f38b006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 -- 172.21.15.17:0/670462709 shutdown_connections 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.071+0000 7f38b84af640 1 -- 172.21.15.17:0/670462709 wait complete. 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 Processor -- start 2024-03-06T07:32:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 -- start start 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 0x7f38b0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 0x7f38b01ec360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01566a0 con 0x7f38b01ec8a0 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0156a10 con 0x7f38b01ebf00 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.073+0000 7f38b84af640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00702c0 con 0x7f3898004800 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54542/0 (socket says 172.21.15.17:54542) 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 -- 172.21.15.17:0/4122352508 learned_addr learned my addr 172.21.15.17:0/4122352508 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b624b640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 0x7f38b0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 0x7f38b01ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 -- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 msgr2=0x7f38b0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 0x7f38b0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 -- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 msgr2=0x7f38b01ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 0x7f38b01ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 -- 172.21.15.17:0/4122352508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0143c00 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b624b640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 0x7f38b0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b5a4a640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 0x7f38b01ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b6a4c640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f38b00c6270 tx=0x7f38a8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8019020 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8004240 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b00ab6b0 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8003610 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.074+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b0149320 con 0x7f38b01ec8a0 2024-03-06T07:32:21.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.075+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b0070650 con 0x7f38b01ec8a0 2024-03-06T07:32:21.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.076+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38a801c020 con 0x7f38b01ec8a0 2024-03-06T07:32:21.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.076+0000 7f38aeffd640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:21.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.076+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f38a808a620 con 0x7f38b01ec8a0 2024-03-06T07:32:21.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.077+0000 7f38b624b640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:21.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.077+0000 7f38b624b640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 0x7f389c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000f80 tx=0x7f38a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:21.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.081+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38b0070650 con 0x7f38b01ec8a0 2024-03-06T07:32:21.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.300+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38b004f330 con 0x7f38b01ec8a0 2024-03-06T07:32:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.301+0000 7f38aeffd640 1 -- 172.21.15.17:0/4122352508 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f38a8023020 con 0x7f38b01ec8a0 2024-03-06T07:32:21.303 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:21.303 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 msgr2=0x7f389c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 0x7f389c06d990 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f38a0000f80 tx=0x7f38a0009040 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 msgr2=0x7f38b014d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 secure :-1 s=READY pgs=2618 cs=0 l=1 rev1=1 crypto rx=0x7f38b00c6270 tx=0x7f38a8011ac0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 shutdown_connections 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f389c06b4d0 0x7f389c06d990 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38b01ec8a0 0x7f38b014d1e0 unknown :-1 s=CLOSED pgs=2618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38b01ebf00 0x7f38b01ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 --2- 172.21.15.17:0/4122352508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3898004800 0x7f38b0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 >> 172.21.15.17:0/4122352508 conn(0x7f38b006b1a0 msgr2=0x7f38b00c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 shutdown_connections 2024-03-06T07:32:21.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:21.303+0000 7f38b84af640 1 -- 172.21.15.17:0/4122352508 wait complete. 2024-03-06T07:32:21.718 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:21.718 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:22.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:22.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.818+0000 7f9182a78640 1 Processor -- start 2024-03-06T07:32:22.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.818+0000 7f9182a78640 1 -- start start 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9182a78640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 0x7f917c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9182a78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 0x7f917c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9182a78640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f917c0c73d0 con 0x7f917c0c88c0 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9182a78640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917c0c8dd0 con 0x7f917c14f360 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9182a78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0be2e0 con 0x7f917c158b60 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9181015640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 0x7f917c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f9180814640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 0x7f917c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f917bfff640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f917c0c88c0 0x7f917c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:41904/0 (socket says 172.21.15.17:41904) 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.819+0000 7f917bfff640 1 -- 172.21.15.17:0/502753040 learned_addr learned my addr 172.21.15.17:0/502753040 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f917b7fe640 1 -- 172.21.15.17:0/502753040 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973096728 0 0) 0x7f917c0c73d0 con 0x7f917c0c88c0 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f917b7fe640 1 -- 172.21.15.17:0/502753040 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916c003520 con 0x7f917c0c88c0 2024-03-06T07:32:22.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f9181015640 1 -- 172.21.15.17:0/502753040 >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 msgr2=0x7f917c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f9181015640 1 --2- 172.21.15.17:0/502753040 >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 0x7f917c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f9181015640 1 -- 172.21.15.17:0/502753040 >> v1:172.21.15.163:6789/0 conn(0x7f917c0c88c0 legacy=0x7f917c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f9181015640 1 -- 172.21.15.17:0/502753040 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0c7670 con 0x7f917c158b60 2024-03-06T07:32:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.820+0000 7f9180814640 1 --2- 172.21.15.17:0/502753040 >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 0x7f917c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:22.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9181015640 1 --2- 172.21.15.17:0/502753040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 0x7f917c158ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d90 tx=0x7f917402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6293acae103f9c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f917b7fe640 1 -- 172.21.15.17:0/502753040 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174039070 con 0x7f917c158b60 2024-03-06T07:32:22.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f917b7fe640 1 -- 172.21.15.17:0/502753040 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917402cbc0 con 0x7f917c158b60 2024-03-06T07:32:22.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f917b7fe640 1 -- 172.21.15.17:0/502753040 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174034440 con 0x7f917c158b60 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 -- 172.21.15.17:0/502753040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 msgr2=0x7f917c158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 --2- 172.21.15.17:0/502753040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 0x7f917c158ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9174009d90 tx=0x7f917402be10 comp rx=0 tx=0).stop 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 -- 172.21.15.17:0/502753040 shutdown_connections 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 --2- 172.21.15.17:0/502753040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c158b60 0x7f917c158ff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 --2- 172.21.15.17:0/502753040 >> v2:172.21.15.163:3300/0 conn(0x7f917c14f360 0x7f917c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 -- 172.21.15.17:0/502753040 >> 172.21.15.17:0/502753040 conn(0x7f917c0ac570 msgr2=0x7f917c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 -- 172.21.15.17:0/502753040 shutdown_connections 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.821+0000 7f9182a78640 1 -- 172.21.15.17:0/502753040 wait complete. 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.822+0000 7f9182a78640 1 Processor -- start 2024-03-06T07:32:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 -- start start 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c0c88c0 0x7f917c1ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f917c14f360 0x7f917c1ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c5a50 con 0x7f917c158b60 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c7c30 con 0x7f917c14f360 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9182a78640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917c0c5bd0 con 0x7f917c0c88c0 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.823+0000 7f9181015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54556/0 (socket says 172.21.15.17:54556) 2024-03-06T07:32:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 -- 172.21.15.17:0/563761252 learned_addr learned my addr 172.21.15.17:0/563761252 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9180814640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c0c88c0 0x7f917c1ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f917bfff640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f917c14f360 0x7f917c1ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 -- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c0c88c0 msgr2=0x7f917c1ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c0c88c0 0x7f917c1ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 -- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f917c14f360 msgr2=0x7f917c1ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f917c14f360 0x7f917c1ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 -- 172.21.15.17:0/563761252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0be8b0 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9181015640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f9174009eb0 tx=0x7f9174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174039050 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9174045070 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917c158710 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174009450 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.824+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917c1523c0 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.825+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917c04f330 con 0x7f917c158b60 2024-03-06T07:32:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.826+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f917403e830 con 0x7f917c158b60 2024-03-06T07:32:22.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.826+0000 7f91797fa640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 0x7f916006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:22.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.826+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f91740ad400 con 0x7f917c158b60 2024-03-06T07:32:22.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.826+0000 7f9180814640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 0x7f916006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:22.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.827+0000 7f9180814640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 0x7f916006d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9170009ac0 tx=0x7f9170008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:22.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:22.831+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9174032e20 con 0x7f917c158b60 2024-03-06T07:32:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.049+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f917c0ac250 con 0x7f917c158b60 2024-03-06T07:32:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.050+0000 7f91797fa640 1 -- 172.21.15.17:0/563761252 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f917407c1d0 con 0x7f917c158b60 2024-03-06T07:32:23.051 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:23.052 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:23.052 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.051+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 msgr2=0x7f916006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.051+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 0x7f916006d990 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9170009ac0 tx=0x7f9170008040 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.051+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 msgr2=0x7f917c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.051+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f9174009eb0 tx=0x7f9174004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 shutdown_connections 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f916006b4d0 0x7f916006d990 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f917c158b60 0x7f917c150a10 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f917c14f360 0x7f917c1ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 --2- 172.21.15.17:0/563761252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f917c0c88c0 0x7f917c1ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 >> 172.21.15.17:0/563761252 conn(0x7f917c0ac570 msgr2=0x7f917c0ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 shutdown_connections 2024-03-06T07:32:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:23.052+0000 7f9182a78640 1 -- 172.21.15.17:0/563761252 wait complete. 2024-03-06T07:32:23.466 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:23.466 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:24.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:24.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.567+0000 7f2da7d81640 1 Processor -- start 2024-03-06T07:32:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.567+0000 7f2da7d81640 1 -- start start 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da7d81640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2da0155c80 0x7f2da006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da7d81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2da00720a0 0x7f2da006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da7d81640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00c1ae0 con 0x7f2da0152520 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da7d81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da00c1dc0 con 0x7f2da00720a0 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da7d81640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da006e450 con 0x7f2da0155c80 2024-03-06T07:32:24.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2da00720a0 0x7f2da006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2da00720a0 0x7f2da006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54558/0 (socket says 172.21.15.17:54558) 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 -- 172.21.15.17:0/637718294 learned_addr learned my addr 172.21.15.17:0/637718294 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da531c640 1 --2- 172.21.15.17:0/637718294 >> v2:172.21.15.163:3300/0 conn(0x7f2da0155c80 0x7f2da006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 -- 172.21.15.17:0/637718294 >> v2:172.21.15.163:3300/0 conn(0x7f2da0155c80 msgr2=0x7f2da006ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 --2- 172.21.15.17:0/637718294 >> v2:172.21.15.163:3300/0 conn(0x7f2da0155c80 0x7f2da006ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 -- 172.21.15.17:0/637718294 >> v1:172.21.15.163:6789/0 conn(0x7f2da0152520 legacy=0x7f2da00c12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.568+0000 7f2da631e640 1 -- 172.21.15.17:0/637718294 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da006e730 con 0x7f2da00720a0 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.569+0000 7f2da631e640 1 --2- 172.21.15.17:0/637718294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da006afc0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f2da0073750 tx=0x7f2d9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcc61663eef1d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.569+0000 7f2da4b1b640 1 -- 172.21.15.17:0/637718294 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90034070 con 0x7f2da00720a0 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.569+0000 7f2da4b1b640 1 -- 172.21.15.17:0/637718294 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90030860 con 0x7f2da00720a0 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.569+0000 7f2da4b1b640 1 -- 172.21.15.17:0/637718294 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90030c30 con 0x7f2da00720a0 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 -- 172.21.15.17:0/637718294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 msgr2=0x7f2da006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 --2- 172.21.15.17:0/637718294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da006afc0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f2da0073750 tx=0x7f2d9002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 -- 172.21.15.17:0/637718294 shutdown_connections 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 --2- 172.21.15.17:0/637718294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da006afc0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 --2- 172.21.15.17:0/637718294 >> v2:172.21.15.163:3300/0 conn(0x7f2da0155c80 0x7f2da006ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 -- 172.21.15.17:0/637718294 >> 172.21.15.17:0/637718294 conn(0x7f2da00c76b0 msgr2=0x7f2da00c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:24.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 -- 172.21.15.17:0/637718294 shutdown_connections 2024-03-06T07:32:24.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.570+0000 7f2da7d81640 1 -- 172.21.15.17:0/637718294 wait complete. 2024-03-06T07:32:24.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.571+0000 7f2da7d81640 1 Processor -- start 2024-03-06T07:32:24.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.571+0000 7f2da7d81640 1 -- start start 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2da0152520 0x7f2da01de640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2da0155c80 0x7f2da01deb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0152a80 con 0x7f2da00720a0 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0152900 con 0x7f2da0152520 2024-03-06T07:32:24.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da006b500 con 0x7f2da0155c80 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54560/0 (socket says 172.21.15.17:54560) 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 -- 172.21.15.17:0/444720308 learned_addr learned my addr 172.21.15.17:0/444720308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da531c640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2da0152520 0x7f2da01de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 -- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2da0155c80 msgr2=0x7f2da01deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2da0155c80 0x7f2da01deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 -- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2da0152520 msgr2=0x7f2da01de640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2da0152520 0x7f2da01de640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 -- 172.21.15.17:0/444720308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da014fb90 con 0x7f2da00720a0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da5b1d640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c790 tx=0x7f2d9800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98016020 con 0x7f2da00720a0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98006d50 con 0x7f2da00720a0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da01ee670 con 0x7f2da00720a0 2024-03-06T07:32:24.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.572+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98003bb0 con 0x7f2da00720a0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.573+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da01ee370 con 0x7f2da00720a0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.574+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da004f330 con 0x7f2da00720a0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.574+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d9800a850 con 0x7f2da00720a0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.574+0000 7f2d9e7fc640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 0x7f2d8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.574+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f2d9808a120 con 0x7f2da00720a0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.575+0000 7f2da531c640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 0x7f2d8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:24.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.575+0000 7f2da531c640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 0x7f2d8c06d9c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2da0150000 tx=0x7f2d94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:24.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.579+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d98020020 con 0x7f2da00720a0 2024-03-06T07:32:24.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.796+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2da0074700 con 0x7f2da00720a0 2024-03-06T07:32:24.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.796+0000 7f2d9e7fc640 1 -- 172.21.15.17:0/444720308 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f2d98019310 con 0x7f2da00720a0 2024-03-06T07:32:24.798 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:24.798 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:24.798 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 msgr2=0x7f2d8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 0x7f2d8c06d9c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2da0150000 tx=0x7f2d94008040 comp rx=0 tx=0).stop 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 msgr2=0x7f2da01ede30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800c790 tx=0x7f2d9800cc50 comp rx=0 tx=0).stop 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 shutdown_connections 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2d8c06b500 0x7f2d8c06d9c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2da0155c80 0x7f2da01deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2da0152520 0x7f2da01de640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 --2- 172.21.15.17:0/444720308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2da00720a0 0x7f2da01ede30 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 >> 172.21.15.17:0/444720308 conn(0x7f2da00c76b0 msgr2=0x7f2da00c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 shutdown_connections 2024-03-06T07:32:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:24.798+0000 7f2da7d81640 1 -- 172.21.15.17:0/444720308 wait complete. 2024-03-06T07:32:25.212 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:25.213 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:26.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.349+0000 7fdfd03c7640 1 Processor -- start 2024-03-06T07:32:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.349+0000 7fdfd03c7640 1 -- start start 2024-03-06T07:32:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.350+0000 7fdfd03c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfc8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.350+0000 7fdfd03c7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8156180 con 0x7fdfc80c5330 2024-03-06T07:32:26.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfd03c7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc8159630 con 0x7fdfc80c8a00 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfd03c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81587b0 con 0x7fdfc8152920 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfce964640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdfc80c5330 0x7fdfc815fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45456/0 (socket says 172.21.15.17:45456) 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfce964640 1 -- 172.21.15.17:0/2036321362 learned_addr learned my addr 172.21.15.17:0/2036321362 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfce163640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfc8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198967344 0 0) 0x7fdfb8001110 con 0x7fdfc80c5330 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb0003520 con 0x7fdfc80c5330 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711331411 0 0) 0x7fdfc8159630 con 0x7fdfc80c8a00 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8001110 con 0x7fdfc80c8a00 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066694067 0 0) 0x7fdfb0003520 con 0x7fdfc80c5330 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 msgr2=0x7fdfc8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfc8155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 >> v1:172.21.15.163:6789/0 conn(0x7fdfc80c8a00 legacy=0x7fdfc8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80c7760 con 0x7fdfc80c5330 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fdfb8001840 con 0x7fdfc80c5330 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 >> v1:172.21.15.17:6789/0 conn(0x7fdfc80c5330 legacy=0x7fdfc815fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb0003b70 0x7fdfb0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfb0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc81587b0 con 0x7fdfb0004800 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb000bb50 con 0x7fdfb0003b70 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb000bd20 con 0x7fdfc8152920 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.351+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfce163640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb0003b70 0x7fdfb0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfce964640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfb0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 -- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 msgr2=0x7fdfb0005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc8152920 0x7fdfb0005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 -- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb0003b70 msgr2=0x7fdfb0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfb0003b70 0x7fdfb0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 -- 172.21.15.17:0/2036321362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0003520 con 0x7fdfb0004800 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.352+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfb0004c80 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc002000 tx=0x7fdfb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d8b6a4041c7367 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc001bf0 con 0x7fdfb0004800 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc003b70 con 0x7fdfb0004800 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfcd161640 1 -- 172.21.15.17:0/2036321362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0050d0 con 0x7fdfb0004800 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2036321362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 msgr2=0x7fdfb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2036321362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfb0004c80 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc002000 tx=0x7fdfb0005830 comp rx=0 tx=0).stop 2024-03-06T07:32:26.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfce964640 1 -- 172.21.15.17:0/2036321362 reap_dead start 2024-03-06T07:32:26.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2036321362 shutdown_connections 2024-03-06T07:32:26.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2036321362 >> 172.21.15.17:0/2036321362 conn(0x7fdfc80ac560 msgr2=0x7fdfc80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:26.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2036321362 shutdown_connections 2024-03-06T07:32:26.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.353+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2036321362 wait complete. 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.354+0000 7fdfd03c7640 1 Processor -- start 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.354+0000 7fdfd03c7640 1 -- start start 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc81ee010 0x7fdfc81532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 0x7fdfc80c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc8158a00 con 0x7fdfb0004800 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc80c7d20 con 0x7fdfc80c3920 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfd03c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc806d030 con 0x7fdfc81ee010 2024-03-06T07:32:26.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54572/0 (socket says 172.21.15.17:54572) 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 -- 172.21.15.17:0/2552184274 learned_addr learned my addr 172.21.15.17:0/2552184274 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc81ee010 0x7fdfc81532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce964640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 0x7fdfc80c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 -- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc81ee010 msgr2=0x7fdfc81532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc81ee010 0x7fdfc81532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 -- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 msgr2=0x7fdfc80c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 0x7fdfc80c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce163640 1 -- 172.21.15.17:0/2552184274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80c2440 con 0x7fdfb0004800 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.355+0000 7fdfce964640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 0x7fdfc80c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfce163640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00118e0 tx=0x7fdfc0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000faf0 con 0x7fdfb0004800 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000fc90 con 0x7fdfb0004800 2024-03-06T07:32:26.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc81ee850 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0014440 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.356+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc80c26e0 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.357+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc804f330 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.358+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfc0012040 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.358+0000 7fdfc6ffd640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 0x7fdfb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.358+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fdfc008a9e0 con 0x7fdfb0004800 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.358+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 0x7fdfb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.359+0000 7fdfcd962640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 0x7fdfb406d9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc034000 tx=0x7fdfbc009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:26.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.362+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdfc0019030 con 0x7fdfb0004800 2024-03-06T07:32:26.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.756+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc806cda0 con 0x7fdfb0004800 2024-03-06T07:32:26.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.756+0000 7fdfc6ffd640 1 -- 172.21.15.17:0/2552184274 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fdfc0059660 con 0x7fdfb0004800 2024-03-06T07:32:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:26.758 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:26.758 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 msgr2=0x7fdfb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 0x7fdfb406d9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc034000 tx=0x7fdfbc009030 comp rx=0 tx=0).stop 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 msgr2=0x7fdfc80c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00118e0 tx=0x7fdfc0011da0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 shutdown_connections 2024-03-06T07:32:26.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfb406b500 0x7fdfb406d9c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfc80c3920 0x7fdfc80c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfc81ee010 0x7fdfc81532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 --2- 172.21.15.17:0/2552184274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb0004800 0x7fdfc80c2ee0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 >> 172.21.15.17:0/2552184274 conn(0x7fdfc80ac560 msgr2=0x7fdfc8150ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 shutdown_connections 2024-03-06T07:32:26.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:26.758+0000 7fdfd03c7640 1 -- 172.21.15.17:0/2552184274 wait complete. 2024-03-06T07:32:27.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:27.173 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:28.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:28.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 Processor -- start 2024-03-06T07:32:28.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 -- start start 2024-03-06T07:32:28.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc0c88c0 0x7faffc155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:28.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0c73d0 con 0x7faffc14f360 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0be2e0 con 0x7faffc158b60 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb002d93640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc159700 con 0x7faffc0c88c0 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7faffbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc0c88c0 0x7faffc155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7faffbfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7faffc0c88c0 0x7faffc155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54584/0 (socket says 172.21.15.17:54584) 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7faffbfff640 1 -- 172.21.15.17:0/2429040632 learned_addr learned my addr 172.21.15.17:0/2429040632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.275+0000 7fb000b2f640 1 --1- 172.21.15.17:0/2429040632 >> v1:172.21.15.17:6789/0 conn(0x7faffc14f360 0x7faffc0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45472/0 (socket says 172.21.15.17:45472) 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.276+0000 7faffbfff640 1 -- 172.21.15.17:0/2429040632 >> v1:172.21.15.163:6789/0 conn(0x7faffc158b60 legacy=0x7faffc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.276+0000 7faffbfff640 1 -- 172.21.15.17:0/2429040632 >> v1:172.21.15.17:6789/0 conn(0x7faffc14f360 legacy=0x7faffc0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.276+0000 7faffb7fe640 1 -- 172.21.15.17:0/2429040632 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018675668 0 0) 0x7faffc0c73d0 con 0x7faffc14f360 2024-03-06T07:32:28.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.276+0000 7faffbfff640 1 -- 172.21.15.17:0/2429040632 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0c7670 con 0x7faffc0c88c0 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.277+0000 7faffbfff640 1 --2- 172.21.15.17:0/2429040632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc0c88c0 0x7faffc155c10 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7faff0009870 tx=0x7faff002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb20b311d3502a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.277+0000 7faffb7fe640 1 -- 172.21.15.17:0/2429040632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0035020 con 0x7faffc0c88c0 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.277+0000 7faffb7fe640 1 -- 172.21.15.17:0/2429040632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff0031870 con 0x7faffc0c88c0 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.277+0000 7faffb7fe640 1 -- 172.21.15.17:0/2429040632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0031c40 con 0x7faffc0c88c0 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 -- 172.21.15.17:0/2429040632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc0c88c0 msgr2=0x7faffc155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 --2- 172.21.15.17:0/2429040632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc0c88c0 0x7faffc155c10 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7faff0009870 tx=0x7faff002b910 comp rx=0 tx=0).stop 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 -- 172.21.15.17:0/2429040632 shutdown_connections 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 --2- 172.21.15.17:0/2429040632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc0c88c0 0x7faffc155c10 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 -- 172.21.15.17:0/2429040632 >> 172.21.15.17:0/2429040632 conn(0x7faffc0ac570 msgr2=0x7faffc0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:28.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 -- 172.21.15.17:0/2429040632 shutdown_connections 2024-03-06T07:32:28.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.278+0000 7fb002d93640 1 -- 172.21.15.17:0/2429040632 wait complete. 2024-03-06T07:32:28.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.279+0000 7fb002d93640 1 Processor -- start 2024-03-06T07:32:28.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.279+0000 7fb002d93640 1 -- start start 2024-03-06T07:32:28.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faffc0c88c0 0x7faffc0be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faffc14f360 0x7faffc154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c7c30 con 0x7faffc158b60 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c5a50 con 0x7faffc0c88c0 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0c5bd0 con 0x7faffc14f360 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54598/0 (socket says 172.21.15.17:54598) 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 -- 172.21.15.17:0/3841231123 learned_addr learned my addr 172.21.15.17:0/3841231123 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb000b2f640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faffc0c88c0 0x7faffc0be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7faffbfff640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faffc14f360 0x7faffc154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 -- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faffc14f360 msgr2=0x7faffc154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faffc14f360 0x7faffc154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 -- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faffc0c88c0 msgr2=0x7faffc0be760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faffc0c88c0 0x7faffc0be760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 -- 172.21.15.17:0/3841231123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc153980 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb001330640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fafec00b490 tx=0x7fafec00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec015020 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec011850 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc158530 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.280+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec011c40 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.281+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc153c40 con 0x7faffc158b60 2024-03-06T07:32:28.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.282+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc04f330 con 0x7faffc158b60 2024-03-06T07:32:28.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.283+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafec018020 con 0x7faffc158b60 2024-03-06T07:32:28.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.283+0000 7faff97fa640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 0x7fafe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:28.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.283+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fafec08a380 con 0x7faffc158b60 2024-03-06T07:32:28.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.283+0000 7fb000b2f640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 0x7fafe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:28.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.284+0000 7fb000b2f640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 0x7fafe806d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faff40012e0 tx=0x7faff4007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:28.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.287+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fafec01f020 con 0x7faffc158b60 2024-03-06T07:32:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.565+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faffc0c8d50 con 0x7faffc158b60 2024-03-06T07:32:28.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.566+0000 7faff97fa640 1 -- 172.21.15.17:0/3841231123 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fafec0172c0 con 0x7faffc158b60 2024-03-06T07:32:28.568 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:28.568 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:28.568 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 msgr2=0x7fafe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 0x7fafe806d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faff40012e0 tx=0x7faff4007ed0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 msgr2=0x7faffc154980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7fafec00b490 tx=0x7fafec00b950 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 shutdown_connections 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fafe806b4d0 0x7fafe806d990 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faffc158b60 0x7faffc154980 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faffc14f360 0x7faffc154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 --2- 172.21.15.17:0/3841231123 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faffc0c88c0 0x7faffc0be760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.568+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 >> 172.21.15.17:0/3841231123 conn(0x7faffc0ac570 msgr2=0x7faffc14cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.569+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 shutdown_connections 2024-03-06T07:32:28.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:28.569+0000 7fb002d93640 1 -- 172.21.15.17:0/3841231123 wait complete. 2024-03-06T07:32:28.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:28.987 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:29.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:30.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.088+0000 7fced14b5640 1 Processor -- start 2024-03-06T07:32:30.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.088+0000 7fced14b5640 1 -- start start 2024-03-06T07:32:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fced14b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcecc1592d0 0x7fcecc1552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fced14b5640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcecc0ac260 0x7fcecc155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fced14b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc152d40 con 0x7fcecc152960 2024-03-06T07:32:30.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fced14b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc066e30 con 0x7fcecc1592d0 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fced14b5640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc155d50 con 0x7fcecc0ac260 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcecc1592d0 0x7fcecc1552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcecc1592d0 0x7fcecc1552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54614/0 (socket says 172.21.15.17:54614) 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 -- 172.21.15.17:0/3450544205 learned_addr learned my addr 172.21.15.17:0/3450544205 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcecb7fe640 1 --2- 172.21.15.17:0/3450544205 >> v2:172.21.15.163:3300/0 conn(0x7fcecc0ac260 0x7fcecc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 -- 172.21.15.17:0/3450544205 >> v2:172.21.15.163:3300/0 conn(0x7fcecc0ac260 msgr2=0x7fcecc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 --2- 172.21.15.17:0/3450544205 >> v2:172.21.15.163:3300/0 conn(0x7fcecc0ac260 0x7fcecc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 -- 172.21.15.17:0/3450544205 >> v1:172.21.15.163:6789/0 conn(0x7fcecc152960 legacy=0x7fcecc06dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.089+0000 7fcec3fff640 1 -- 172.21.15.17:0/3450544205 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0c1b50 con 0x7fcecc1592d0 2024-03-06T07:32:30.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.090+0000 7fcec3fff640 1 --2- 172.21.15.17:0/3450544205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc1592d0 0x7fcecc1552d0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fcecc06b380 tx=0x7fceb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63db103bf6a31404 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.090+0000 7fceca7fc640 1 -- 172.21.15.17:0/3450544205 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4035020 con 0x7fcecc1592d0 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.090+0000 7fceca7fc640 1 -- 172.21.15.17:0/3450544205 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4031870 con 0x7fcecc1592d0 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.090+0000 7fceca7fc640 1 -- 172.21.15.17:0/3450544205 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4031c40 con 0x7fcecc1592d0 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 -- 172.21.15.17:0/3450544205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc1592d0 msgr2=0x7fcecc1552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 --2- 172.21.15.17:0/3450544205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc1592d0 0x7fcecc1552d0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fcecc06b380 tx=0x7fceb402b910 comp rx=0 tx=0).stop 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 -- 172.21.15.17:0/3450544205 shutdown_connections 2024-03-06T07:32:30.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 --2- 172.21.15.17:0/3450544205 >> v2:172.21.15.163:3300/0 conn(0x7fcecc0ac260 0x7fcecc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 --2- 172.21.15.17:0/3450544205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc1592d0 0x7fcecc1552d0 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 -- 172.21.15.17:0/3450544205 >> 172.21.15.17:0/3450544205 conn(0x7fcecc0c70b0 msgr2=0x7fcecc0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 -- 172.21.15.17:0/3450544205 shutdown_connections 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.091+0000 7fced14b5640 1 -- 172.21.15.17:0/3450544205 wait complete. 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.092+0000 7fced14b5640 1 Processor -- start 2024-03-06T07:32:30.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.093+0000 7fced14b5640 1 -- start start 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0ac260 0x7fcecc1ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 0x7fcecc1dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc159760 con 0x7fcecc152960 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0672c0 con 0x7fcecc0ac260 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc06e3a0 con 0x7fcecc14bf60 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54624/0 (socket says 172.21.15.17:54624) 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 -- 172.21.15.17:0/3698798369 learned_addr learned my addr 172.21.15.17:0/3698798369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcecaffd640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0ac260 0x7fcecc1ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcecb7fe640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 0x7fcecc1dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 -- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 msgr2=0x7fcecc1dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 0x7fcecc1dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 -- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0ac260 msgr2=0x7fcecc1ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0ac260 0x7fcecc1ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 -- 172.21.15.17:0/3698798369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc1ec2e0 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcecb7fe640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 0x7fcecc1dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec3fff640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fceb402b950 tx=0x7fceb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4040070 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4030d00 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc072810 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4035040 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.094+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc1eb5a0 con 0x7fcecc152960 2024-03-06T07:32:30.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.095+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc069460 con 0x7fcecc152960 2024-03-06T07:32:30.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.096+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fceb4038020 con 0x7fcecc152960 2024-03-06T07:32:30.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.096+0000 7fcec37fe640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 0x7fceac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:30.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.096+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fceb40ac320 con 0x7fcecc152960 2024-03-06T07:32:30.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.097+0000 7fcecaffd640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 0x7fceac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:30.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.097+0000 7fcecaffd640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 0x7fceac06d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcebc004ea0 tx=0x7fcebc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:30.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.105+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcecc069460 con 0x7fcecc152960 2024-03-06T07:32:30.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.325+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcecc0c2c70 con 0x7fcecc152960 2024-03-06T07:32:30.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.326+0000 7fcec37fe640 1 -- 172.21.15.17:0/3698798369 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fcecc0c2c70 con 0x7fcecc152960 2024-03-06T07:32:30.328 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:30.328 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:30.328 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:30.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 msgr2=0x7fceac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 0x7fceac06d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcebc004ea0 tx=0x7fcebc009040 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 msgr2=0x7fcecc1ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fceb402b950 tx=0x7fceb40046c0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 shutdown_connections 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fceac06b4d0 0x7fceac06d990 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc14bf60 0x7fcecc1dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcecc152960 0x7fcecc1ebda0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 --2- 172.21.15.17:0/3698798369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0ac260 0x7fcecc1ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 >> 172.21.15.17:0/3698798369 conn(0x7fcecc0c70b0 msgr2=0x7fcecc0c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 shutdown_connections 2024-03-06T07:32:30.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:30.328+0000 7fced14b5640 1 -- 172.21.15.17:0/3698798369 wait complete. 2024-03-06T07:32:30.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:30.743 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:31.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.844+0000 7fc4e27e9640 1 Processor -- start 2024-03-06T07:32:31.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.844+0000 7fc4e27e9640 1 -- start start 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e27e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4dc156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e27e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc070aa0 con 0x7fc4dc0acc00 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e27e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc070d80 con 0x7fc4dc1506b0 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e27e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc153e10 con 0x7fc4dc153990 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e0d86640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc4dc0acc00 0x7fc4dc0c54a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36344/0 (socket says 172.21.15.17:36344) 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1495117131 learned_addr learned my addr 172.21.15.17:0/1495117131 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4db7fe640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4dc156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718735156 0 0) 0x7fc4dc070aa0 con 0x7fc4dc0acc00 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c4003520 con 0x7fc4dc0acc00 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4271588426 0 0) 0x7fc4c4003520 con 0x7fc4dc0acc00 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 msgr2=0x7fc4dc156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4dc156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 >> v1:172.21.15.163:6789/0 conn(0x7fc4dc1506b0 legacy=0x7fc4dc0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc1566a0 con 0x7fc4dc0acc00 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fc4d40019f0 con 0x7fc4dc0acc00 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 >> v1:172.21.15.17:6789/0 conn(0x7fc4dc0acc00 legacy=0x7fc4dc0c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c4003f20 0x7fc4c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c4004d40 0x7fc4c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc153e10 con 0x7fc4c4004d40 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c400ba20 con 0x7fc4c4003f20 2024-03-06T07:32:31.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.846+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c400bc20 con 0x7fc4dc153990 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c4004d40 0x7fc4c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4db7fe640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c4003f20 0x7fc4c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4dbfff640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 msgr2=0x7fc4c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc153990 0x7fc4c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c4003f20 msgr2=0x7fc4c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4c4003f20 0x7fc4c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.847+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1495117131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4003520 con 0x7fc4c4004d40 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.848+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c4004d40 0x7fc4c40051c0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400a860 tx=0x7fc4c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7f1d0c749c8a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.848+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400a480 con 0x7fc4c4004d40 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.848+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d4034630 con 0x7fc4c4004d40 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.848+0000 7fc4daffd640 1 -- 172.21.15.17:0/1495117131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4034a80 con 0x7fc4c4004d40 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1495117131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c4004d40 msgr2=0x7fc4c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1495117131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c4004d40 0x7fc4c40051c0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400a860 tx=0x7fc4c4005830 comp rx=0 tx=0).stop 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1495117131 reap_dead start 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1495117131 shutdown_connections 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1495117131 >> 172.21.15.17:0/1495117131 conn(0x7fc4dc06b1a0 msgr2=0x7fc4dc06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1495117131 shutdown_connections 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.849+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1495117131 wait complete. 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.850+0000 7fc4e27e9640 1 Processor -- start 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.850+0000 7fc4e27e9640 1 -- start start 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc0c7910 0x7fc4dc1e9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 0x7fc4dc1e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc156a10 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc156450 con 0x7fc4dc14b3f0 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e27e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0ad070 con 0x7fc4dc0c7910 2024-03-06T07:32:31.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42920/0 (socket says 172.21.15.17:42920) 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1954412397 learned_addr learned my addr 172.21.15.17:0/1954412397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4dbfff640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc0c7910 0x7fc4dc1e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4db7fe640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 0x7fc4dc1e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc0c7910 msgr2=0x7fc4dc1e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc0c7910 0x7fc4dc1e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 msgr2=0x7fc4dc1e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 0x7fc4dc1e98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 -- 172.21.15.17:0/1954412397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc14d970 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4db7fe640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 0x7fc4dc1e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.851+0000 7fc4e0d86640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000f80 tx=0x7fc4dc071060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.852+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400a480 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.852+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d4045070 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.852+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc14b860 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.852+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4001fd0 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.852+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc14be20 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.853+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc153f90 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.853+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4d4034bf0 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.854+0000 7fc4d8ff9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 0x7fc4c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.854+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc4d40ac780 con 0x7fc4dc0aafa0 2024-03-06T07:32:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.854+0000 7fc4dbfff640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 0x7fc4c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:31.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.855+0000 7fc4dbfff640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 0x7fc4c806d9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc1e05b0 tx=0x7fc4cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:31.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:31.858+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc4dc153f90 con 0x7fc4dc0aafa0 2024-03-06T07:32:32.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.076+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4dc04f330 con 0x7fc4dc0aafa0 2024-03-06T07:32:32.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.077+0000 7fc4d8ff9640 1 -- 172.21.15.17:0/1954412397 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fc4d407b520 con 0x7fc4dc0aafa0 2024-03-06T07:32:32.079 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:32.079 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:32.079 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:32.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.078+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 msgr2=0x7fc4c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.078+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 0x7fc4c806d9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc1e05b0 tx=0x7fc4cc009040 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.078+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 msgr2=0x7fc4dc1ec6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.078+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000f80 tx=0x7fc4dc071060 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 shutdown_connections 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc4c806b500 0x7fc4c806d9c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4dc0aafa0 0x7fc4dc1ec6b0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4dc14b3f0 0x7fc4dc1e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 --2- 172.21.15.17:0/1954412397 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4dc0c7910 0x7fc4dc1e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 >> 172.21.15.17:0/1954412397 conn(0x7fc4dc06b1a0 msgr2=0x7fc4dc152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 shutdown_connections 2024-03-06T07:32:32.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:32.079+0000 7fc4e27e9640 1 -- 172.21.15.17:0/1954412397 wait complete. 2024-03-06T07:32:32.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:32.493 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:33.494 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.595+0000 7fd1b0681640 1 Processor -- start 2024-03-06T07:32:33.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.595+0000 7fd1b0681640 1 -- start start 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1b0681640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a80c88c0 0x7fd1a8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1b0681640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1a8158b60 0x7fd1a8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1b0681640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a80c73d0 con 0x7fd1a814f360 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1b0681640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a80c8dd0 con 0x7fd1a8158b60 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1b0681640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80be2e0 con 0x7fd1a80c88c0 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1a8158b60 0x7fd1a8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd1a8158b60 0x7fd1a8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42934/0 (socket says 172.21.15.17:42934) 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 -- 172.21.15.17:0/3072374589 learned_addr learned my addr 172.21.15.17:0/3072374589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1adc1c640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a80c88c0 0x7fd1a8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 -- 172.21.15.17:0/3072374589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a80c88c0 msgr2=0x7fd1a8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:33.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a80c88c0 0x7fd1a8155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 -- 172.21.15.17:0/3072374589 >> v1:172.21.15.163:6789/0 conn(0x7fd1a814f360 legacy=0x7fd1a80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.596+0000 7fd1aec1e640 1 -- 172.21.15.17:0/3072374589 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80c7670 con 0x7fd1a8158b60 2024-03-06T07:32:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1aec1e640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a8158ff0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736c9a977081f4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1ad41b640 1 -- 172.21.15.17:0/3072374589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a002ca70 con 0x7fd1a8158b60 2024-03-06T07:32:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1ad41b640 1 -- 172.21.15.17:0/3072374589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a002cc10 con 0x7fd1a8158b60 2024-03-06T07:32:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1ad41b640 1 -- 172.21.15.17:0/3072374589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a00343f0 con 0x7fd1a8158b60 2024-03-06T07:32:33.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1b0681640 1 -- 172.21.15.17:0/3072374589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 msgr2=0x7fd1a8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.597+0000 7fd1b0681640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a8158ff0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a002bed0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 -- 172.21.15.17:0/3072374589 shutdown_connections 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a8158ff0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 --2- 172.21.15.17:0/3072374589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a80c88c0 0x7fd1a8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 -- 172.21.15.17:0/3072374589 >> 172.21.15.17:0/3072374589 conn(0x7fd1a80ac570 msgr2=0x7fd1a80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 -- 172.21.15.17:0/3072374589 shutdown_connections 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.598+0000 7fd1b0681640 1 -- 172.21.15.17:0/3072374589 wait complete. 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 Processor -- start 2024-03-06T07:32:33.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 -- start start 2024-03-06T07:32:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 0x7fd1a80c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a814f360 0x7fd1a81eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c5a50 con 0x7fd1a8158b60 2024-03-06T07:32:33.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c7c30 con 0x7fd1a80c88c0 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.599+0000 7fd1b0681640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a80c5bd0 con 0x7fd1a814f360 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42942/0 (socket says 172.21.15.17:42942) 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 -- 172.21.15.17:0/2534750577 learned_addr learned my addr 172.21.15.17:0/2534750577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1adc1c640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a814f360 0x7fd1a81eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1ae41d640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 0x7fd1a80c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 -- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a814f360 msgr2=0x7fd1a81eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a814f360 0x7fd1a81eb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 -- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 msgr2=0x7fd1a80c21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 0x7fd1a80c21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 -- 172.21.15.17:0/2534750577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80c0230 con 0x7fd1a8158b60 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1ae41d640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 0x7fd1a80c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:33.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1aec1e640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0044020 con 0x7fd1a8158b60 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a0040670 con 0x7fd1a8158b60 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0009510 con 0x7fd1a8158b60 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a80be490 con 0x7fd1a8158b60 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.600+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a80bea50 con 0x7fd1a8158b60 2024-03-06T07:32:33.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.602+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a804f330 con 0x7fd1a8158b60 2024-03-06T07:32:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.602+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1a0047020 con 0x7fd1a8158b60 2024-03-06T07:32:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.603+0000 7fd19effd640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.603+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd1a00acea0 con 0x7fd1a8158b60 2024-03-06T07:32:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.603+0000 7fd1ae41d640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:33.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.604+0000 7fd1ae41d640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 0x7fd18c06d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4001660 tx=0x7fd1a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:33.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.607+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1a004e030 con 0x7fd1a8158b60 2024-03-06T07:32:33.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.823+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1a806cda0 con 0x7fd1a8158b60 2024-03-06T07:32:33.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.824+0000 7fd19effd640 1 -- 172.21.15.17:0/2534750577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fd1a007bb20 con 0x7fd1a8158b60 2024-03-06T07:32:33.826 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:33.826 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:33.826 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 msgr2=0x7fd18c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 0x7fd18c06d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4001660 tx=0x7fd1a4009040 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 msgr2=0x7fd1a80bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009e50 tx=0x7fd1a0004a40 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 shutdown_connections 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd18c06b4d0 0x7fd18c06d990 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1a8158b60 0x7fd1a80bf530 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1a814f360 0x7fd1a81eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 --2- 172.21.15.17:0/2534750577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1a80c88c0 0x7fd1a80c21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.826+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 >> 172.21.15.17:0/2534750577 conn(0x7fd1a80ac570 msgr2=0x7fd1a806b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.827+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 shutdown_connections 2024-03-06T07:32:33.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:33.827+0000 7fd1b0681640 1 -- 172.21.15.17:0/2534750577 wait complete. 2024-03-06T07:32:34.241 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:34.241 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:35.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:35.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.342+0000 7f9deb3e9640 1 Processor -- start 2024-03-06T07:32:35.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.342+0000 7f9deb3e9640 1 -- start start 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9deb3e9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9de4159270 0x7f9de4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9deb3e9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de40c73d0 con 0x7f9de40c88c0 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9deb3e9640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de40be2e0 con 0x7f9de414f360 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9deb3e9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de4159700 con 0x7f9de4159270 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de9986640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9de4159270 0x7f9de4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:35.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de8984640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9de40c88c0 0x7f9de4158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36346/0 (socket says 172.21.15.17:36346) 2024-03-06T07:32:35.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de8984640 1 -- 172.21.15.17:0/540942572 learned_addr learned my addr 172.21.15.17:0/540942572 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:35.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de9986640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9de4159270 0x7f9de4155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42956/0 (socket says 172.21.15.17:42956) 2024-03-06T07:32:35.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de9986640 1 -- 172.21.15.17:0/540942572 >> v1:172.21.15.163:6789/0 conn(0x7f9de414f360 legacy=0x7f9de40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:35.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de9986640 1 -- 172.21.15.17:0/540942572 >> v1:172.21.15.17:6789/0 conn(0x7f9de40c88c0 legacy=0x7f9de4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.343+0000 7f9de9986640 1 -- 172.21.15.17:0/540942572 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40c7670 con 0x7f9de4159270 2024-03-06T07:32:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.344+0000 7f9de9986640 1 --2- 172.21.15.17:0/540942572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de4159270 0x7f9de4155c10 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009d90 tx=0x7f9ddc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78413b89245455d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.344+0000 7f9ddbfff640 1 -- 172.21.15.17:0/540942572 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc039070 con 0x7f9de4159270 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.344+0000 7f9ddbfff640 1 -- 172.21.15.17:0/540942572 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc02cbd0 con 0x7f9de4159270 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.344+0000 7f9ddbfff640 1 -- 172.21.15.17:0/540942572 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc034440 con 0x7f9de4159270 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 -- 172.21.15.17:0/540942572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de4159270 msgr2=0x7f9de4155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/540942572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de4159270 0x7f9de4155c10 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009d90 tx=0x7f9ddc02be20 comp rx=0 tx=0).stop 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 -- 172.21.15.17:0/540942572 shutdown_connections 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/540942572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de4159270 0x7f9de4155c10 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 -- 172.21.15.17:0/540942572 >> 172.21.15.17:0/540942572 conn(0x7f9de40ac570 msgr2=0x7f9de40ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 -- 172.21.15.17:0/540942572 shutdown_connections 2024-03-06T07:32:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.345+0000 7f9deb3e9640 1 -- 172.21.15.17:0/540942572 wait complete. 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.346+0000 7f9deb3e9640 1 Processor -- start 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.346+0000 7f9deb3e9640 1 -- start start 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.346+0000 7f9deb3e9640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9de40c88c0 0x7f9de41542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9de4159270 0x7f9de4149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40c7c30 con 0x7f9de414f360 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40c5a50 con 0x7f9de40c88c0 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40c5bd0 con 0x7f9de4159270 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42962/0 (socket says 172.21.15.17:42962) 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 -- 172.21.15.17:0/1250266640 learned_addr learned my addr 172.21.15.17:0/1250266640 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 -- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9de4159270 msgr2=0x7f9de4149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de9986640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9de4159270 0x7f9de4149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de9185640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9de40c88c0 0x7f9de41542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9de4159270 0x7f9de4149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 -- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9de40c88c0 msgr2=0x7f9de41542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9de40c88c0 0x7f9de41542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 -- 172.21.15.17:0/1250266640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4157ab0 con 0x7f9de414f360 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9de8984640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c440 tx=0x7f9dd4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4017070 con 0x7f9de414f360 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4005ce0 con 0x7f9de414f360 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd40072f0 con 0x7f9de414f360 2024-03-06T07:32:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.347+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de41ef0b0 con 0x7f9de414f360 2024-03-06T07:32:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.348+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de4149fd0 con 0x7f9de414f360 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.348+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de404f330 con 0x7f9de414f360 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.349+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dd401a070 con 0x7f9de414f360 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.349+0000 7f9dd9ffb640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 0x7f9dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.350+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f9dd40897f0 con 0x7f9de414f360 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.350+0000 7f9de9185640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 0x7f9dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:35.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.350+0000 7f9de9185640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 0x7f9dc806d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f9de0003d90 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:35.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.354+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9dd4058470 con 0x7f9de414f360 2024-03-06T07:32:35.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.573+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9de40c8d50 con 0x7f9de414f360 2024-03-06T07:32:35.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.574+0000 7f9dd9ffb640 1 -- 172.21.15.17:0/1250266640 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f9dd4057d70 con 0x7f9de414f360 2024-03-06T07:32:35.577 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:35.577 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:35.577 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:35.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 msgr2=0x7f9dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 0x7f9dc806d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f9de0003d90 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 msgr2=0x7f9de4149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c440 tx=0x7f9dd4001e70 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 shutdown_connections 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9dc806b4d0 0x7f9dc806d990 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9de4159270 0x7f9de4149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9de414f360 0x7f9de4149550 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 --2- 172.21.15.17:0/1250266640 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9de40c88c0 0x7f9de41542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 >> 172.21.15.17:0/1250266640 conn(0x7f9de40ac570 msgr2=0x7f9de40c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 shutdown_connections 2024-03-06T07:32:35.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:35.576+0000 7f9deb3e9640 1 -- 172.21.15.17:0/1250266640 wait complete. 2024-03-06T07:32:35.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:35.992 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:36.993 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.092+0000 7fc93fb25640 1 Processor -- start 2024-03-06T07:32:37.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.092+0000 7fc93fb25640 1 -- start start 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93fb25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 0x7fc9381554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93fb25640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 0x7fc9380c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93fb25640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380aa660 con 0x7fc9381559e0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93fb25640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380c4610 con 0x7fc9380aa180 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93fb25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380aa940 con 0x7fc9381550c0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93d0c0640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc9381559e0 0x7fc93814ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36350/0 (socket says 172.21.15.17:36350) 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93d0c0640 1 -- 172.21.15.17:0/3168027677 learned_addr learned my addr 172.21.15.17:0/3168027677 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/3168027677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 0x7fc9381554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.093+0000 7fc93e0c2640 1 --2- 172.21.15.17:0/3168027677 >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 0x7fc9380c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13163867 0 0) 0x7fc9380aa660 con 0x7fc9381559e0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc924003520 con 0x7fc9381559e0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3128478534 0 0) 0x7fc924003520 con 0x7fc9381559e0 2024-03-06T07:32:37.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93d8c1640 1 -- 172.21.15.17:0/3168027677 >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 msgr2=0x7fc9380c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:37.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/3168027677 >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 0x7fc9380c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93d8c1640 1 -- 172.21.15.17:0/3168027677 >> v1:172.21.15.17:6789/0 conn(0x7fc9381559e0 legacy=0x7fc93814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93d8c1640 1 -- 172.21.15.17:0/3168027677 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380c4860 con 0x7fc9381550c0 2024-03-06T07:32:37.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.094+0000 7fc93e0c2640 1 --2- 172.21.15.17:0/3168027677 >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 0x7fc9380c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/3168027677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 0x7fc9381554a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a940 tx=0x7fc92802cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a6866322e9d747 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9340019f0 con 0x7fc9381550c0 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92802e670 con 0x7fc9381550c0 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93c8bf640 1 -- 172.21.15.17:0/3168027677 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800b040 con 0x7fc9381550c0 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 -- 172.21.15.17:0/3168027677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 msgr2=0x7fc9381554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 --2- 172.21.15.17:0/3168027677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 0x7fc9381554a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc92800a940 tx=0x7fc92802cb30 comp rx=0 tx=0).stop 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 -- 172.21.15.17:0/3168027677 shutdown_connections 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 --2- 172.21.15.17:0/3168027677 >> v2:172.21.15.163:3300/0 conn(0x7fc9380aa180 0x7fc9380c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 --2- 172.21.15.17:0/3168027677 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9381550c0 0x7fc9381554a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.095+0000 7fc93fb25640 1 -- 172.21.15.17:0/3168027677 >> 172.21.15.17:0/3168027677 conn(0x7fc9380c6710 msgr2=0x7fc9380c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.096+0000 7fc93fb25640 1 -- 172.21.15.17:0/3168027677 shutdown_connections 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.096+0000 7fc93fb25640 1 -- 172.21.15.17:0/3168027677 wait complete. 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 Processor -- start 2024-03-06T07:32:37.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 -- start start 2024-03-06T07:32:37.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 0x7fc93806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 0x7fc9381dff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93814f310 con 0x7fc9381550c0 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c1520 con 0x7fc9381559e0 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93fb25640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93814f490 con 0x7fc9380aa180 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93d0c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93d0c0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42964/0 (socket says 172.21.15.17:42964) 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.097+0000 7fc93d0c0640 1 -- 172.21.15.17:0/576143722 learned_addr learned my addr 172.21.15.17:0/576143722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 -- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 msgr2=0x7fc93806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93e0c2640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 0x7fc9381dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 0x7fc93806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 0x7fc93806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 -- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 msgr2=0x7fc9381dff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 0x7fc9381dff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 -- 172.21.15.17:0/576143722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93806ca00 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93e0c2640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 0x7fc9381dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 0x7fc93806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93d0c0640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fc93400cc10 tx=0x7fc93400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934017070 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93400ce20 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934012620 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc938153e10 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.098+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9381543d0 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.099+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93804f330 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.099+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc93401a070 con 0x7fc9381550c0 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.100+0000 7fc92e7fc640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 0x7fc91c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:37.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.100+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fc93408a3e0 con 0x7fc9381550c0 2024-03-06T07:32:37.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.100+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 0x7fc91c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:37.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.101+0000 7fc93d8c1640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 0x7fc91c06d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc928037000 tx=0x7fc92800c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:37.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.104+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc93806ca00 con 0x7fc9381550c0 2024-03-06T07:32:37.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.489+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9380c3aa0 con 0x7fc9381550c0 2024-03-06T07:32:37.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.490+0000 7fc92e7fc640 1 -- 172.21.15.17:0/576143722 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fc934058fd0 con 0x7fc9381550c0 2024-03-06T07:32:37.493 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:37.493 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 msgr2=0x7fc91c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 0x7fc91c06d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc928037000 tx=0x7fc92800c040 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 msgr2=0x7fc93806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fc93400cc10 tx=0x7fc93400a430 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 shutdown_connections 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc91c06b4d0 0x7fc91c06d990 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9381559e0 0x7fc9381dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9381550c0 0x7fc93806e310 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 --2- 172.21.15.17:0/576143722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9380aa180 0x7fc93806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.492+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 >> 172.21.15.17:0/576143722 conn(0x7fc9380c6710 msgr2=0x7fc9380bff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.493+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 shutdown_connections 2024-03-06T07:32:37.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:37.493+0000 7fc93fb25640 1 -- 172.21.15.17:0/576143722 wait complete. 2024-03-06T07:32:37.908 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:37.909 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:38.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:39.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.012+0000 7efcbed7a640 1 Processor -- start 2024-03-06T07:32:39.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.012+0000 7efcbed7a640 1 -- start start 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbed7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcb80c88c0 0x7efcb8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbed7a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efcb8158b60 0x7efcb8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbed7a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb80c73d0 con 0x7efcb814f360 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbed7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80c8dd0 con 0x7efcb80c88c0 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbed7a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80be2e0 con 0x7efcb8158b60 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcb7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcb80c88c0 0x7efcb8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbcb16640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efcb814f360 0x7efcb80c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36358/0 (socket says 172.21.15.17:36358) 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbcb16640 1 -- 172.21.15.17:0/347538966 learned_addr learned my addr 172.21.15.17:0/347538966 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcb7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcb80c88c0 0x7efcb8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42978/0 (socket says 172.21.15.17:42978) 2024-03-06T07:32:39.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcb7fff640 1 -- 172.21.15.17:0/347538966 >> v2:172.21.15.163:3300/0 conn(0x7efcb8158b60 msgr2=0x7efcb8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.013+0000 7efcbd317640 1 --2- 172.21.15.17:0/347538966 >> v2:172.21.15.163:3300/0 conn(0x7efcb8158b60 0x7efcb8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb7fff640 1 --2- 172.21.15.17:0/347538966 >> v2:172.21.15.163:3300/0 conn(0x7efcb8158b60 0x7efcb8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb77fe640 1 -- 172.21.15.17:0/347538966 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210681122 0 0) 0x7efcb80c73d0 con 0x7efcb814f360 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb7fff640 1 -- 172.21.15.17:0/347538966 >> v1:172.21.15.17:6789/0 conn(0x7efcb814f360 legacy=0x7efcb80c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb7fff640 1 -- 172.21.15.17:0/347538966 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80c7670 con 0x7efcb80c88c0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb7fff640 1 --2- 172.21.15.17:0/347538966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb80c88c0 0x7efcb8155c10 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7efcb8152d60 tx=0x7efcac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a765992d60d914b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb77fe640 1 -- 172.21.15.17:0/347538966 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0315a0 con 0x7efcb80c88c0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb77fe640 1 -- 172.21.15.17:0/347538966 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac037070 con 0x7efcb80c88c0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.014+0000 7efcb77fe640 1 -- 172.21.15.17:0/347538966 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac031960 con 0x7efcb80c88c0 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 -- 172.21.15.17:0/347538966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb80c88c0 msgr2=0x7efcb8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 --2- 172.21.15.17:0/347538966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb80c88c0 0x7efcb8155c10 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7efcb8152d60 tx=0x7efcac02b840 comp rx=0 tx=0).stop 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 -- 172.21.15.17:0/347538966 shutdown_connections 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 --2- 172.21.15.17:0/347538966 >> v2:172.21.15.163:3300/0 conn(0x7efcb8158b60 0x7efcb8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 --2- 172.21.15.17:0/347538966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb80c88c0 0x7efcb8155c10 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 -- 172.21.15.17:0/347538966 >> 172.21.15.17:0/347538966 conn(0x7efcb80ac570 msgr2=0x7efcb80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 -- 172.21.15.17:0/347538966 shutdown_connections 2024-03-06T07:32:39.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.015+0000 7efcbed7a640 1 -- 172.21.15.17:0/347538966 wait complete. 2024-03-06T07:32:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.016+0000 7efcbed7a640 1 Processor -- start 2024-03-06T07:32:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.016+0000 7efcbed7a640 1 -- start start 2024-03-06T07:32:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 0x7efcb80afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 0x7efcb8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c5a50 con 0x7efcb8158b60 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c7c30 con 0x7efcb814f360 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbed7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c5bd0 con 0x7efcb80c88c0 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42986/0 (socket says 172.21.15.17:42986) 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 -- 172.21.15.17:0/146243969 learned_addr learned my addr 172.21.15.17:0/146243969 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcb7fff640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 0x7efcb8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbcb16640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 0x7efcb80afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 -- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 msgr2=0x7efcb80afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 0x7efcb80afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 -- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 msgr2=0x7efcb8151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 0x7efcb8151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbd317640 1 -- 172.21.15.17:0/146243969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80ad660 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcb7fff640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 0x7efcb8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.017+0000 7efcbcb16640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 0x7efcb80afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcbd317640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7efcb000cda0 tx=0x7efcb000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0003b90 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb0003d30 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004240 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb81ebdf0 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.018+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb80adc00 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.019+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb804f330 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.019+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcb00043e0 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.020+0000 7efcb57fa640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 0x7efca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.020+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7efcb008af00 con 0x7efcb8158b60 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.020+0000 7efcbcb16640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 0x7efca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:39.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.020+0000 7efcbcb16640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 0x7efca406d990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efca8000f60 tx=0x7efca8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:39.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.024+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efcb0059b80 con 0x7efcb8158b60 2024-03-06T07:32:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.244+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcb806cda0 con 0x7efcb8158b60 2024-03-06T07:32:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.245+0000 7efcb57fa640 1 -- 172.21.15.17:0/146243969 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7efcb0059480 con 0x7efcb8158b60 2024-03-06T07:32:39.247 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:39.248 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:39.248 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 msgr2=0x7efca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 0x7efca406d990 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7efca8000f60 tx=0x7efca8009040 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 msgr2=0x7efcb81517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7efcb000cda0 tx=0x7efcb000a660 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 shutdown_connections 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7efca406b4d0 0x7efca406d990 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcb8158b60 0x7efcb81517b0 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efcb814f360 0x7efcb8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 --2- 172.21.15.17:0/146243969 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efcb80c88c0 0x7efcb80afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 >> 172.21.15.17:0/146243969 conn(0x7efcb80ac570 msgr2=0x7efcb80c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 shutdown_connections 2024-03-06T07:32:39.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:39.247+0000 7efcbed7a640 1 -- 172.21.15.17:0/146243969 wait complete. 2024-03-06T07:32:39.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:39.663 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:40.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:40.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.762+0000 7fd7ba5c7640 1 Processor -- start 2024-03-06T07:32:40.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 -- start start 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7b40c5480 0x7fd7b4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd7b406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b40c5cc0 con 0x7fd7b41586f0 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b406df40 con 0x7fd7b40c5480 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7ba5c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41591c0 con 0x7fd7b40c5860 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7b8b64640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd7b41586f0 0x7fd7b40724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55908/0 (socket says 172.21.15.17:55908) 2024-03-06T07:32:40.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.763+0000 7fd7b8b64640 1 -- 172.21.15.17:0/1834618860 learned_addr learned my addr 172.21.15.17:0/1834618860 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd7b406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/1834618860 >> v2:172.21.15.163:3300/0 conn(0x7fd7b40c5480 0x7fd7b4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927691489 0 0) 0x7fd7b40c5cc0 con 0x7fd7b41586f0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd79c003520 con 0x7fd7b41586f0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3072113535 0 0) 0x7fd79c003520 con 0x7fd7b41586f0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 msgr2=0x7fd7b406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd7b406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 >> v2:172.21.15.163:3300/0 conn(0x7fd7b40c5480 msgr2=0x7fd7b4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 --2- 172.21.15.17:0/1834618860 >> v2:172.21.15.163:3300/0 conn(0x7fd7b40c5480 0x7fd7b4155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b41594a0 con 0x7fd7b41586f0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd7ac0019f0 con 0x7fd7b41586f0 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 >> v1:172.21.15.17:6789/0 conn(0x7fd7b41586f0 legacy=0x7fd7b40724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd79c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c004800 0x7fd79c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd79c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b406df40 con 0x7fd79c0039b0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41591c0 con 0x7fd79c004800 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c00b9c0 con 0x7fd7b40c5860 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd79c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b8b64640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd79c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.764+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c004800 0x7fd79c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 -- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 msgr2=0x7fd79c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b40c5860 0x7fd79c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 -- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c004800 msgr2=0x7fd79c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd79c004800 0x7fd79c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 -- 172.21.15.17:0/1834618860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c003520 con 0x7fd79c0039b0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd79c003dd0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009af0 tx=0x7fd79c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c80caf185280a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:40.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4039070 con 0x7fd79c0039b0 2024-03-06T07:32:40.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a402c710 con 0x7fd79c0039b0 2024-03-06T07:32:40.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.765+0000 7fd7b2ffd640 1 -- 172.21.15.17:0/1834618860 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4009690 con 0x7fd79c0039b0 2024-03-06T07:32:40.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/1834618860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 msgr2=0x7fd79c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/1834618860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd79c003dd0 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009af0 tx=0x7fd79c005830 comp rx=0 tx=0).stop 2024-03-06T07:32:40.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7b8b64640 1 -- 172.21.15.17:0/1834618860 reap_dead start 2024-03-06T07:32:40.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/1834618860 shutdown_connections 2024-03-06T07:32:40.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/1834618860 >> 172.21.15.17:0/1834618860 conn(0x7fd7b406b1a0 msgr2=0x7fd7b406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/1834618860 shutdown_connections 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.766+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/1834618860 wait complete. 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.767+0000 7fd7ba5c7640 1 Processor -- start 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.767+0000 7fd7ba5c7640 1 -- start start 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 0x7fd7b40c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b41ee270 0x7fd7b41ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4152a20 con 0x7fd79c0039b0 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4155680 con 0x7fd7b41e2200 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7ba5c7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4155050 con 0x7fd7b41ee270 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58406/0 (socket says 172.21.15.17:58406) 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 -- 172.21.15.17:0/508760464 learned_addr learned my addr 172.21.15.17:0/508760464 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:40.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 0x7fd7b40c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b8b64640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b41ee270 0x7fd7b41ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 -- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b41ee270 msgr2=0x7fd7b41ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b41ee270 0x7fd7b41ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 -- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 msgr2=0x7fd7b40c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 0x7fd7b40c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b3fff640 1 -- 172.21.15.17:0/508760464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40a8930 con 0x7fd79c0039b0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.768+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 0x7fd7b40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7b3fff640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009af0 tx=0x7fd7a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4039050 con 0x7fd79c0039b0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a4045070 con 0x7fd79c0039b0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b40a7db0 con 0x7fd79c0039b0 2024-03-06T07:32:40.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4003a50 con 0x7fd79c0039b0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.769+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b40c48f0 con 0x7fd79c0039b0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.770+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b404f330 con 0x7fd79c0039b0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.770+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7a4003bf0 con 0x7fd79c0039b0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.771+0000 7fd7b0ff9640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.771+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd7a40abf00 con 0x7fd79c0039b0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.771+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.771+0000 7fd7b37fe640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 0x7fd7a006d9c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40abd80 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:40.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.775+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7a404e020 con 0x7fd79c0039b0 2024-03-06T07:32:40.984 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:40.987 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:32:40.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.994+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b406e0c0 con 0x7fd79c0039b0 2024-03-06T07:32:40.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.995+0000 7fd7b0ff9640 1 -- 172.21.15.17:0/508760464 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fd7a407aaf0 con 0x7fd79c0039b0 2024-03-06T07:32:40.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:40.998 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:40.998 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 msgr2=0x7fd7a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 0x7fd7a006d9c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40abd80 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 msgr2=0x7fd7b414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009af0 tx=0x7fd7a4004730 comp rx=0 tx=0).stop 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 shutdown_connections 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7b41ee270 0x7fd7b41ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7b41e2200 0x7fd7b40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 --2- 172.21.15.17:0/508760464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd79c0039b0 0x7fd7b414f6e0 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:41.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 >> 172.21.15.17:0/508760464 conn(0x7fd7b406b1a0 msgr2=0x7fd7b40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:41.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 shutdown_connections 2024-03-06T07:32:41.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:40.997+0000 7fd7ba5c7640 1 -- 172.21.15.17:0/508760464 wait complete. 2024-03-06T07:32:41.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:41.414 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:42.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 Processor -- start 2024-03-06T07:32:42.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 -- start start 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7800c88c0 0x7fa780155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa780158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7800c73d0 con 0x7fa78014f360 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7800c8dd0 con 0x7fa7800c88c0 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.515+0000 7fa785fb6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800be2e0 con 0x7fa780158b60 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa78014f360 0x7fa7800c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55920/0 (socket says 172.21.15.17:55920) 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77f7fe640 1 -- 172.21.15.17:0/2373451841 learned_addr learned my addr 172.21.15.17:0/2373451841 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77ffff640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa780158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> v2:172.21.15.163:3300/0 conn(0x7fa7800c88c0 0x7fa780155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176469344 0 0) 0x7fa7800c73d0 con 0x7fa78014f360 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa768003520 con 0x7fa78014f360 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124925467 0 0) 0x7fa768003520 con 0x7fa78014f360 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 msgr2=0x7fa780158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa780158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 >> v2:172.21.15.163:3300/0 conn(0x7fa7800c88c0 msgr2=0x7fa780155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 --2- 172.21.15.17:0/2373451841 >> v2:172.21.15.163:3300/0 conn(0x7fa7800c88c0 0x7fa780155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7800c7670 con 0x7fa78014f360 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fa770001670 con 0x7fa78014f360 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77ffff640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa780158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> v2:172.21.15.163:3300/0 conn(0x7fa7800c88c0 0x7fa780155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 >> v1:172.21.15.17:6789/0 conn(0x7fa78014f360 legacy=0x7fa7800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa7680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa768003fa0 0x7fa768004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa768004d40 0x7fa7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800be2e0 con 0x7fa768003fa0 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800baa0 con 0x7fa768004d40 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800bca0 con 0x7fa780158b60 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa768003fa0 0x7fa768004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77f7fe640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa7680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.516+0000 7fa77ffff640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa768004d40 0x7fa7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 -- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 msgr2=0x7fa7680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa780158b60 0x7fa7680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 -- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa768004d40 msgr2=0x7fa7680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa768004d40 0x7fa7680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 -- 172.21.15.17:0/2373451841 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768003520 con 0x7fa768003fa0 2024-03-06T07:32:42.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.517+0000 7fa77effd640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa768003fa0 0x7fa768004800 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fa7800c8d20 tx=0x7fa768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d446436b50ee5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77400f040 con 0x7fa768003fa0 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774004230 con 0x7fa768003fa0 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa77e7fc640 1 -- 172.21.15.17:0/2373451841 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774003610 con 0x7fa768003fa0 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 -- 172.21.15.17:0/2373451841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa768003fa0 msgr2=0x7fa768004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 --2- 172.21.15.17:0/2373451841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa768003fa0 0x7fa768004800 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7fa7800c8d20 tx=0x7fa768005830 comp rx=0 tx=0).stop 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa77ffff640 1 -- 172.21.15.17:0/2373451841 reap_dead start 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 -- 172.21.15.17:0/2373451841 shutdown_connections 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 -- 172.21.15.17:0/2373451841 >> 172.21.15.17:0/2373451841 conn(0x7fa7800ac570 msgr2=0x7fa7800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 -- 172.21.15.17:0/2373451841 shutdown_connections 2024-03-06T07:32:42.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.518+0000 7fa785fb6640 1 -- 172.21.15.17:0/2373451841 wait complete. 2024-03-06T07:32:42.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.519+0000 7fa785fb6640 1 Processor -- start 2024-03-06T07:32:42.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.519+0000 7fa785fb6640 1 -- start start 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa768003fa0 0x7fa7801ee060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7801e8110 0x7fa780150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800c5a50 con 0x7fa7800c0ac0 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800c7c30 con 0x7fa7801e8110 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa785fb6640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7800c5bd0 con 0x7fa768003fa0 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58410/0 (socket says 172.21.15.17:58410) 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 -- 172.21.15.17:0/3172148068 learned_addr learned my addr 172.21.15.17:0/3172148068 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77f7fe640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa768003fa0 0x7fa7801ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 -- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa768003fa0 msgr2=0x7fa7801ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa768003fa0 0x7fa7801ee060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 -- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7801e8110 msgr2=0x7fa780150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77ffff640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7801e8110 0x7fa780150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7801e8110 0x7fa780150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.520+0000 7fa77effd640 1 -- 172.21.15.17:0/3172148068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78014fc10 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa77effd640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fa77402bfc0 tx=0x7fa774004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774048070 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774030460 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774030850 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7800bdec0 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.521+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7801eec80 con 0x7fa7800c0ac0 2024-03-06T07:32:42.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.522+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7801e84f0 con 0x7fa7800c0ac0 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.523+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa77404b070 con 0x7fa7800c0ac0 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.523+0000 7fa767fff640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 0x7fa76c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.523+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fa7740abff0 con 0x7fa7800c0ac0 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.523+0000 7fa77f7fe640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 0x7fa76c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.524+0000 7fa77f7fe640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 0x7fa76c06d9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa770001430 tx=0x7fa770009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:42.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.528+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa774034020 con 0x7fa7800c0ac0 2024-03-06T07:32:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.852+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa78004f330 con 0x7fa7800c0ac0 2024-03-06T07:32:42.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.852+0000 7fa767fff640 1 -- 172.21.15.17:0/3172148068 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fa77407ac70 con 0x7fa7800c0ac0 2024-03-06T07:32:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:42.855 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:42.856 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 msgr2=0x7fa76c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 0x7fa76c06d9c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa770001430 tx=0x7fa770009040 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 msgr2=0x7fa7801e7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7fa77402bfc0 tx=0x7fa774004730 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 shutdown_connections 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.855+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa76c06b500 0x7fa76c06d9c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7801e8110 0x7fa780150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7800c0ac0 0x7fa7801e7bd0 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 --2- 172.21.15.17:0/3172148068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa768003fa0 0x7fa7801ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 >> 172.21.15.17:0/3172148068 conn(0x7fa7800ac570 msgr2=0x7fa7800c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 shutdown_connections 2024-03-06T07:32:42.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:42.856+0000 7fa785fb6640 1 -- 172.21.15.17:0/3172148068 wait complete. 2024-03-06T07:32:43.271 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:43.271 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:44.272 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:44.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.371+0000 7fe00fe88640 1 Processor -- start 2024-03-06T07:32:44.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.371+0000 7fe00fe88640 1 -- start start 2024-03-06T07:32:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00fe88640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe008156ea0 0x7fe0080c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00fe88640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080ad310 con 0x7fe008153990 2024-03-06T07:32:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00fe88640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0081572a0 con 0x7fe0081506b0 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00fe88640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe008156450 con 0x7fe008156ea0 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00d423640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe008153990 0x7fe0080acc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55928/0 (socket says 172.21.15.17:55928) 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00d423640 1 -- 172.21.15.17:0/3952482089 learned_addr learned my addr 172.21.15.17:0/3952482089 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559652575 0 0) 0x7fe0080ad310 con 0x7fe008153990 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.372+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff4003520 con 0x7fe008153990 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00e425640 1 --2- 172.21.15.17:0/3952482089 >> v2:172.21.15.163:3300/0 conn(0x7fe008156ea0 0x7fe0080c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257527260 0 0) 0x7fdff4003520 con 0x7fe008153990 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 >> v2:172.21.15.163:3300/0 conn(0x7fe008156ea0 msgr2=0x7fe0080c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 --2- 172.21.15.17:0/3952482089 >> v2:172.21.15.163:3300/0 conn(0x7fe008156ea0 0x7fe0080c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 >> v1:172.21.15.163:6789/0 conn(0x7fe0081506b0 legacy=0x7fe0080702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008070aa0 con 0x7fe008153990 2024-03-06T07:32:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fdff8001670 con 0x7fe008153990 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 >> v1:172.21.15.17:6789/0 conn(0x7fe008153990 legacy=0x7fe0080acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdff40039b0 0x7fdff4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fdff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdff40051c0 0x7fdff400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008156450 con 0x7fdff4004800 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff400c110 con 0x7fdff40051c0 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff400c310 con 0x7fdff40039b0 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fdff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00dc24640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdff40039b0 0x7fdff4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00e425640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdff40051c0 0x7fdff400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 -- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdff40039b0 msgr2=0x7fdff4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdff40039b0 0x7fdff4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 -- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdff40051c0 msgr2=0x7fdff400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdff40051c0 0x7fdff400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00d423640 1 -- 172.21.15.17:0/3952482089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4003520 con 0x7fdff4004800 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.373+0000 7fe00e425640 1 -- 172.21.15.17:0/3952482089 reap_dead start 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.374+0000 7fe00d423640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fdff4004c80 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c240 tx=0x7fdff4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be78c5a4dc2a900 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.374+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8007cf0 con 0x7fdff4004800 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.374+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff8030070 con 0x7fdff4004800 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.374+0000 7fe00cc22640 1 -- 172.21.15.17:0/3952482089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8005d70 con 0x7fdff4004800 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 -- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 msgr2=0x7fdff4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fdff4004c80 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c240 tx=0x7fdff4005830 comp rx=0 tx=0).stop 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 -- 172.21.15.17:0/3952482089 shutdown_connections 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 --2- 172.21.15.17:0/3952482089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fdff4004c80 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 -- 172.21.15.17:0/3952482089 >> 172.21.15.17:0/3952482089 conn(0x7fe00806b1a0 msgr2=0x7fe00806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 -- 172.21.15.17:0/3952482089 shutdown_connections 2024-03-06T07:32:44.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.375+0000 7fe00fe88640 1 -- 172.21.15.17:0/3952482089 wait complete. 2024-03-06T07:32:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.376+0000 7fe00fe88640 1 Processor -- start 2024-03-06T07:32:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.376+0000 7fe00fe88640 1 -- start start 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 0x7fe008155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 0x7fe0081eba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0081566a0 con 0x7fdff4004800 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008156a10 con 0x7fe00814bec0 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080702c0 con 0x7fe0081ef490 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58428/0 (socket says 172.21.15.17:58428) 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 -- 172.21.15.17:0/2136452375 learned_addr learned my addr 172.21.15.17:0/2136452375 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00d423640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 0x7fe008155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 -- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 msgr2=0x7fe0081eba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00e425640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 0x7fe0081eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 0x7fe0081eba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 -- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 msgr2=0x7fe008155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 0x7fe008155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 -- 172.21.15.17:0/2136452375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00814c3f0 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00e425640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 0x7fe0081eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00d423640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 0x7fe008155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00dc24640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fe00400f850 tx=0x7fe00400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400fe40 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe004018070 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00814c6d0 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400d730 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.377+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0081eef70 con 0x7fdff4004800 2024-03-06T07:32:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.378+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008070650 con 0x7fdff4004800 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.379+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe00400d8d0 con 0x7fdff4004800 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.379+0000 7fdffe7fc640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 0x7fdfec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.379+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe00408a350 con 0x7fdff4004800 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.379+0000 7fe00d423640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 0x7fdfec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.380+0000 7fe00d423640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 0x7fdfec06d9c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdff8034000 tx=0x7fdff8039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.384+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe008070650 con 0x7fdff4004800 2024-03-06T07:32:44.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.603+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe00804f330 con 0x7fdff4004800 2024-03-06T07:32:44.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.603+0000 7fdffe7fc640 1 -- 172.21.15.17:0/2136452375 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fe004058ea0 con 0x7fdff4004800 2024-03-06T07:32:44.606 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:44.606 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:44.606 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:44.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 msgr2=0x7fdfec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 0x7fdfec06d9c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdff8034000 tx=0x7fdff8039040 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 msgr2=0x7fe0081f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7fe00400f850 tx=0x7fe00400fd10 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 shutdown_connections 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fdfec06b500 0x7fdfec06d9c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0081ef490 0x7fe0081eba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe00814bec0 0x7fe008155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 --2- 172.21.15.17:0/2136452375 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdff4004800 0x7fe0081f2170 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 >> 172.21.15.17:0/2136452375 conn(0x7fe00806b1a0 msgr2=0x7fe0080c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.605+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 shutdown_connections 2024-03-06T07:32:44.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:44.606+0000 7fe00fe88640 1 -- 172.21.15.17:0/2136452375 wait complete. 2024-03-06T07:32:45.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:45.024 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:46.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.146+0000 7fbd0277a640 1 Processor -- start 2024-03-06T07:32:46.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.146+0000 7fbd0277a640 1 -- start start 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd0277a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc1527b0 0x7fbcfc159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd0277a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbcfc0c88b0 0x7fbcfc0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd0277a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc0c7c60 con 0x7fbcfc0c7180 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd0277a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc159640 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd0277a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c8060 con 0x7fbcfc1527b0 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbcfb7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbcfc0c7180 0x7fbcfc155b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55934/0 (socket says 172.21.15.17:55934) 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd00d17640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbcfc0c88b0 0x7fbcfc0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbcfb7fe640 1 -- 172.21.15.17:0/1678081320 learned_addr learned my addr 172.21.15.17:0/1678081320 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbcfbfff640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc1527b0 0x7fbcfc159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:46.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbcfaffd640 1 -- 172.21.15.17:0/1678081320 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349502541 0 0) 0x7fbcf0001110 con 0x7fbcfc0c7180 2024-03-06T07:32:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbcfaffd640 1 -- 172.21.15.17:0/1678081320 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce4003520 con 0x7fbcfc0c7180 2024-03-06T07:32:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd00d17640 1 -- 172.21.15.17:0/1678081320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc1527b0 msgr2=0x7fbcfc159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd00d17640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc1527b0 0x7fbcfc159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd00d17640 1 -- 172.21.15.17:0/1678081320 >> v1:172.21.15.17:6789/0 conn(0x7fbcfc0c7180 legacy=0x7fbcfc155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.147+0000 7fbd00d17640 1 -- 172.21.15.17:0/1678081320 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0c76a0 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.148+0000 7fbd00d17640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c88b0 0x7fbcfc0c5330 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009d90 tx=0x7fbcf402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a077d3e5b77e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.148+0000 7fbcfaffd640 1 -- 172.21.15.17:0/1678081320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4039070 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.148+0000 7fbcfaffd640 1 -- 172.21.15.17:0/1678081320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf402cbc0 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.148+0000 7fbcfaffd640 1 -- 172.21.15.17:0/1678081320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4034440 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 -- 172.21.15.17:0/1678081320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c88b0 msgr2=0x7fbcfc0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c88b0 0x7fbcfc0c5330 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009d90 tx=0x7fbcf402be10 comp rx=0 tx=0).stop 2024-03-06T07:32:46.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 -- 172.21.15.17:0/1678081320 shutdown_connections 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c88b0 0x7fbcfc0c5330 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 --2- 172.21.15.17:0/1678081320 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc1527b0 0x7fbcfc159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 -- 172.21.15.17:0/1678081320 >> 172.21.15.17:0/1678081320 conn(0x7fbcfc0ac560 msgr2=0x7fbcfc0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 -- 172.21.15.17:0/1678081320 shutdown_connections 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.149+0000 7fbd0277a640 1 -- 172.21.15.17:0/1678081320 wait complete. 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.151+0000 7fbd0277a640 1 Processor -- start 2024-03-06T07:32:46.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.151+0000 7fbd0277a640 1 -- start start 2024-03-06T07:32:46.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc0c88b0 0x7fbcfc1eaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcfc1527b0 0x7fbcfc1eb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc155600 con 0x7fbcfc0c7180 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc155320 con 0x7fbcfc1527b0 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbd0277a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c8d60 con 0x7fbcfc0c88b0 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58440/0 (socket says 172.21.15.17:58440) 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 -- 172.21.15.17:0/2054832650 learned_addr learned my addr 172.21.15.17:0/2054832650 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 -- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc0c88b0 msgr2=0x7fbcfc1eaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc0c88b0 0x7fbcfc1eaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 -- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcfc1527b0 msgr2=0x7fbcfc1eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcfc1527b0 0x7fbcfc1eb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.152+0000 7fbcfbfff640 1 -- 172.21.15.17:0/2054832650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc0c1a80 con 0x7fbcfc0c7180 2024-03-06T07:32:46.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbcfbfff640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e8a0 tx=0x7fbcec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec017070 con 0x7fbcfc0c7180 2024-03-06T07:32:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec00c940 con 0x7fbcfc0c7180 2024-03-06T07:32:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00cd30 con 0x7fbcfc0c7180 2024-03-06T07:32:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc0c1d40 con 0x7fbcfc0c7180 2024-03-06T07:32:46.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.153+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc143fa0 con 0x7fbcfc0c7180 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.154+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc0acea0 con 0x7fbcfc0c7180 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.154+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcec01a070 con 0x7fbcfc0c7180 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.155+0000 7fbcf8ff9640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.155+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fbcec08a230 con 0x7fbcfc0c7180 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.155+0000 7fbcfb7fe640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:46.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.155+0000 7fbcfb7fe640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000f60 tx=0x7fbcf0001110 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:46.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.159+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbcfc0acea0 con 0x7fbcfc0c7180 2024-03-06T07:32:46.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.379+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcfc04f330 con 0x7fbcfc0c7180 2024-03-06T07:32:46.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.379+0000 7fbcf8ff9640 1 -- 172.21.15.17:0/2054832650 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fbcec058eb0 con 0x7fbcfc0c7180 2024-03-06T07:32:46.382 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:46.382 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:46.382 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:46.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 msgr2=0x7fbce806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0000f60 tx=0x7fbcf0001110 comp rx=0 tx=0).stop 2024-03-06T07:32:46.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 msgr2=0x7fbcfc143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:46.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e8a0 tx=0x7fbcec00ed60 comp rx=0 tx=0).stop 2024-03-06T07:32:46.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 shutdown_connections 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbce806b500 0x7fbce806d9c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbcfc1527b0 0x7fbcfc1eb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbcfc0c88b0 0x7fbcfc1eaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 --2- 172.21.15.17:0/2054832650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcfc0c7180 0x7fbcfc143700 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 >> 172.21.15.17:0/2054832650 conn(0x7fbcfc0ac560 msgr2=0x7fbcfc0aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 shutdown_connections 2024-03-06T07:32:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:46.381+0000 7fbd0277a640 1 -- 172.21.15.17:0/2054832650 wait complete. 2024-03-06T07:32:46.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:46.797 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:47.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:47.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.898+0000 7fe234c0e640 1 Processor -- start 2024-03-06T07:32:47.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.898+0000 7fe234c0e640 1 -- start start 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe234c0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2300c5480 0x7fe230155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe234c0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe230158b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe234c0e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2300c5cc0 con 0x7fe2300c5860 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe234c0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23006df40 con 0x7fe2300c5480 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe234c0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23006fdb0 con 0x7fe2301586f0 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2300c5480 0x7fe230155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2300c5480 0x7fe230155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58452/0 (socket says 172.21.15.17:58452) 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 -- 172.21.15.17:0/3399654308 learned_addr learned my addr 172.21.15.17:0/3399654308 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22ed9d640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe230158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 -- 172.21.15.17:0/3399654308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 msgr2=0x7fe230158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:47.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe230158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 -- 172.21.15.17:0/3399654308 >> v1:172.21.15.163:6789/0 conn(0x7fe2300c5860 legacy=0x7fe23006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.900+0000 7fe22e59c640 1 -- 172.21.15.17:0/3399654308 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2301591c0 con 0x7fe2300c5480 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.901+0000 7fe22e59c640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5480 0x7fe230155ba0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe224009870 tx=0x7fe22402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d52c1a6f30627a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.901+0000 7fe22d59a640 1 -- 172.21.15.17:0/3399654308 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224035020 con 0x7fe2300c5480 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.901+0000 7fe22d59a640 1 -- 172.21.15.17:0/3399654308 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224031640 con 0x7fe2300c5480 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.901+0000 7fe22d59a640 1 -- 172.21.15.17:0/3399654308 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22402d040 con 0x7fe2300c5480 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 -- 172.21.15.17:0/3399654308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5480 msgr2=0x7fe230155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5480 0x7fe230155ba0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7fe224009870 tx=0x7fe22402b8f0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 -- 172.21.15.17:0/3399654308 shutdown_connections 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe230158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 --2- 172.21.15.17:0/3399654308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5480 0x7fe230155ba0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 -- 172.21.15.17:0/3399654308 >> 172.21.15.17:0/3399654308 conn(0x7fe23006b1a0 msgr2=0x7fe23006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:47.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 -- 172.21.15.17:0/3399654308 shutdown_connections 2024-03-06T07:32:47.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.902+0000 7fe234c0e640 1 -- 172.21.15.17:0/3399654308 wait complete. 2024-03-06T07:32:47.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.904+0000 7fe234c0e640 1 Processor -- start 2024-03-06T07:32:47.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.904+0000 7fe234c0e640 1 -- start start 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2300c5480 0x7fe2300c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe2300c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230155680 con 0x7fe2300c5860 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301553a0 con 0x7fe2300c5480 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe234c0e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe230152a20 con 0x7fe2301586f0 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58462/0 (socket says 172.21.15.17:58462) 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 -- 172.21.15.17:0/508657455 learned_addr learned my addr 172.21.15.17:0/508657455 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22e59c640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2300c5480 0x7fe2300c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22ed9d640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe2300c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 -- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 msgr2=0x7fe2300c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe2300c3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 -- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2300c5480 msgr2=0x7fe2300c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2300c5480 0x7fe2300c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 -- 172.21.15.17:0/508657455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2301ee250 con 0x7fe2300c5860 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.905+0000 7fe22dd9b640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c400 tx=0x7fe22000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.906+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200052d0 con 0x7fe2300c5860 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.906+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe220005490 con 0x7fe2300c5860 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.906+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe230151b80 con 0x7fe2300c5860 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.906+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220007260 con 0x7fe2300c5860 2024-03-06T07:32:47.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.906+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2301ee510 con 0x7fe2300c5860 2024-03-06T07:32:47.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.907+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23004f330 con 0x7fe2300c5860 2024-03-06T07:32:47.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.907+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe220007400 con 0x7fe2300c5860 2024-03-06T07:32:47.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.908+0000 7fe21effd640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:47.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.908+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fe220089880 con 0x7fe2300c5860 2024-03-06T07:32:47.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.908+0000 7fe22e59c640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:47.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.908+0000 7fe22e59c640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe224036000 tx=0x7fe22403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:47.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:47.912+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe220057e20 con 0x7fe2300c5860 2024-03-06T07:32:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.130+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23006e0c0 con 0x7fe2300c5860 2024-03-06T07:32:48.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.131+0000 7fe21effd640 1 -- 172.21.15.17:0/508657455 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fe22005bec0 con 0x7fe2300c5860 2024-03-06T07:32:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:48.134 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:48.134 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:48.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 msgr2=0x7fe21006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 0x7fe21006d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe224036000 tx=0x7fe22403c040 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 msgr2=0x7fe2300c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c400 tx=0x7fe22000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 shutdown_connections 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fe21006b500 0x7fe21006d9c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2301586f0 0x7fe2300c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2300c5860 0x7fe2300c2fc0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 --2- 172.21.15.17:0/508657455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2300c5480 0x7fe2300c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 >> 172.21.15.17:0/508657455 conn(0x7fe23006b1a0 msgr2=0x7fe2300c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 shutdown_connections 2024-03-06T07:32:48.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:48.133+0000 7fe234c0e640 1 -- 172.21.15.17:0/508657455 wait complete. 2024-03-06T07:32:48.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:48.549 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:49.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:49.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.650+0000 7f32dd53e640 1 Processor -- start 2024-03-06T07:32:49.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.650+0000 7f32dd53e640 1 -- start start 2024-03-06T07:32:49.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32dd53e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f32d80c6560 0x7f32d81530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32dd53e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32d8154c80 0x7f32d81535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32dd53e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d80bdc30 con 0x7f32d8072f70 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32dd53e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d80bd690 con 0x7f32d8154c80 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32dd53e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d80bd950 con 0x7f32d80c6560 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32d77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32d8154c80 0x7f32d81535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32d77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32d8154c80 0x7f32d81535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58464/0 (socket says 172.21.15.17:58464) 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.651+0000 7f32d77fe640 1 -- 172.21.15.17:0/2829853775 learned_addr learned my addr 172.21.15.17:0/2829853775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d67fc640 1 --2- 172.21.15.17:0/2829853775 >> v2:172.21.15.163:3300/0 conn(0x7f32d80c6560 0x7f32d81530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d77fe640 1 -- 172.21.15.17:0/2829853775 >> v2:172.21.15.163:3300/0 conn(0x7f32d80c6560 msgr2=0x7f32d81530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d77fe640 1 --2- 172.21.15.17:0/2829853775 >> v2:172.21.15.163:3300/0 conn(0x7f32d80c6560 0x7f32d81530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d77fe640 1 -- 172.21.15.17:0/2829853775 >> v1:172.21.15.163:6789/0 conn(0x7f32d8072f70 legacy=0x7f32d8154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:49.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d77fe640 1 -- 172.21.15.17:0/2829853775 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d80c5540 con 0x7f32d8154c80 2024-03-06T07:32:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.652+0000 7f32d77fe640 1 --2- 172.21.15.17:0/2829853775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8154c80 0x7f32d81535e0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c4650 tx=0x7f32d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bb995a07b52306 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32d5ffb640 1 -- 172.21.15.17:0/2829853775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d002ca80 con 0x7f32d8154c80 2024-03-06T07:32:49.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32d5ffb640 1 -- 172.21.15.17:0/2829853775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d002cc20 con 0x7f32d8154c80 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32d5ffb640 1 -- 172.21.15.17:0/2829853775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00343f0 con 0x7f32d8154c80 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 -- 172.21.15.17:0/2829853775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8154c80 msgr2=0x7f32d81535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2829853775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8154c80 0x7f32d81535e0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f32d80c4650 tx=0x7f32d002bee0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 -- 172.21.15.17:0/2829853775 shutdown_connections 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2829853775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8154c80 0x7f32d81535e0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2829853775 >> v2:172.21.15.163:3300/0 conn(0x7f32d80c6560 0x7f32d81530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 -- 172.21.15.17:0/2829853775 >> 172.21.15.17:0/2829853775 conn(0x7f32d806b1a0 msgr2=0x7f32d806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 -- 172.21.15.17:0/2829853775 shutdown_connections 2024-03-06T07:32:49.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.653+0000 7f32dd53e640 1 -- 172.21.15.17:0/2829853775 wait complete. 2024-03-06T07:32:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.655+0000 7f32dd53e640 1 Processor -- start 2024-03-06T07:32:49.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.655+0000 7f32dd53e640 1 -- start start 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d80c6560 0x7f32d8071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d8154c80 0x7f32d8149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d8073350 con 0x7f32d8072f70 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80bba60 con 0x7f32d80c6560 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32dd53e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d80734d0 con 0x7f32d8154c80 2024-03-06T07:32:49.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.656+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58480/0 (socket says 172.21.15.17:58480) 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 -- 172.21.15.17:0/2436253218 learned_addr learned my addr 172.21.15.17:0/2436253218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d67fc640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d80c6560 0x7f32d8071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d77fe640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d8154c80 0x7f32d8149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 -- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d8154c80 msgr2=0x7f32d8149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d8154c80 0x7f32d8149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 -- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d80c6560 msgr2=0x7f32d8071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d80c6560 0x7f32d8071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 -- 172.21.15.17:0/2436253218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8153800 con 0x7f32d8072f70 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32d6ffd640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c400 tx=0x7f32c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00052d0 con 0x7f32d8072f70 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0005490 con 0x7f32d8072f70 2024-03-06T07:32:49.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d8153aa0 con 0x7f32d8072f70 2024-03-06T07:32:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0007260 con 0x7f32d8072f70 2024-03-06T07:32:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.657+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d80722e0 con 0x7f32d8072f70 2024-03-06T07:32:49.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.658+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d804f330 con 0x7f32d8072f70 2024-03-06T07:32:49.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.659+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c0007400 con 0x7f32d8072f70 2024-03-06T07:32:49.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.659+0000 7f32cf7fe640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:49.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.659+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f32c0089910 con 0x7f32d8072f70 2024-03-06T07:32:49.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.659+0000 7f32d67fc640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:49.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.660+0000 7f32d67fc640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f32d814bca0 tx=0x7f32c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:49.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.663+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f32c0058590 con 0x7f32d8072f70 2024-03-06T07:32:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.880+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d80aa020 con 0x7f32d8072f70 2024-03-06T07:32:49.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.880+0000 7f32cf7fe640 1 -- 172.21.15.17:0/2436253218 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f32c0057e90 con 0x7f32d8072f70 2024-03-06T07:32:49.884 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:49.884 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:49.884 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 msgr2=0x7f32b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f32d814bca0 tx=0x7f32c8008040 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 msgr2=0x7f32d80aa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c400 tx=0x7f32c000c8c0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 shutdown_connections 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f32d8154c80 0x7f32d8149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f32d80c6560 0x7f32d8071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 --2- 172.21.15.17:0/2436253218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32d8072f70 0x7f32d80aa240 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:49.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 >> 172.21.15.17:0/2436253218 conn(0x7f32d806b1a0 msgr2=0x7f32d80c5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:49.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 shutdown_connections 2024-03-06T07:32:49.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:49.883+0000 7f32dd53e640 1 -- 172.21.15.17:0/2436253218 wait complete. 2024-03-06T07:32:50.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:50.299 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:51.301 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.399+0000 7f36ec46b640 1 Processor -- start 2024-03-06T07:32:51.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 -- start start 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36e41506b0 0x7f36e4153990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e4156ea0 0x7f36e40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e4070800 con 0x7f36e40acc00 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e4156450 con 0x7f36e41506b0 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.400+0000 7f36ec46b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4156a10 con 0x7f36e4156ea0 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36e41506b0 0x7f36e4153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36e41506b0 0x7f36e4153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36650/0 (socket says 172.21.15.17:36650) 2024-03-06T07:32:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 -- 172.21.15.17:0/80030392 learned_addr learned my addr 172.21.15.17:0/80030392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36e9a06640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e4156ea0 0x7f36e40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 -- 172.21.15.17:0/80030392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e4156ea0 msgr2=0x7f36e40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e4156ea0 0x7f36e40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 -- 172.21.15.17:0/80030392 >> v1:172.21.15.163:6789/0 conn(0x7f36e40acc00 legacy=0x7f36e40c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:51.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.401+0000 7f36ea207640 1 -- 172.21.15.17:0/80030392 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4070aa0 con 0x7f36e41506b0 2024-03-06T07:32:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ea207640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e41506b0 0x7f36e4153990 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f36e41565d0 tx=0x7f36e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf743fe5d44ffc0f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36e9205640 1 -- 172.21.15.17:0/80030392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0034070 con 0x7f36e41506b0 2024-03-06T07:32:51.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36e9205640 1 -- 172.21.15.17:0/80030392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e0030840 con 0x7f36e41506b0 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36e9205640 1 -- 172.21.15.17:0/80030392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0030c10 con 0x7f36e41506b0 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 -- 172.21.15.17:0/80030392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e41506b0 msgr2=0x7f36e4153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e41506b0 0x7f36e4153990 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7f36e41565d0 tx=0x7f36e002b980 comp rx=0 tx=0).stop 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 -- 172.21.15.17:0/80030392 shutdown_connections 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e4156ea0 0x7f36e40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 --2- 172.21.15.17:0/80030392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e41506b0 0x7f36e4153990 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 -- 172.21.15.17:0/80030392 >> 172.21.15.17:0/80030392 conn(0x7f36e406b1a0 msgr2=0x7f36e406b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 -- 172.21.15.17:0/80030392 shutdown_connections 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.402+0000 7f36ec46b640 1 -- 172.21.15.17:0/80030392 wait complete. 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.404+0000 7f36ec46b640 1 Processor -- start 2024-03-06T07:32:51.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.404+0000 7f36ec46b640 1 -- start start 2024-03-06T07:32:51.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e40acc00 0x7f36e41e8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e41506b0 0x7f36e40aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40c5bb0 con 0x7f36e4156ea0 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e41566a0 con 0x7f36e40acc00 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ec46b640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e40c5d30 con 0x7f36e41506b0 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36652/0 (socket says 172.21.15.17:36652) 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 -- 172.21.15.17:0/898743928 learned_addr learned my addr 172.21.15.17:0/898743928 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36ea207640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e40acc00 0x7f36e41e8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36e9a06640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e41506b0 0x7f36e40aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 -- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e41506b0 msgr2=0x7f36e40aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e41506b0 0x7f36e40aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 -- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e40acc00 msgr2=0x7f36e41e8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e40acc00 0x7f36e41e8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.405+0000 7f36eaa08640 1 -- 172.21.15.17:0/898743928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e40aadf0 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36eaa08640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c760 tx=0x7f36dc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc003b70 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc003d30 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e41e9730 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc00aa90 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.406+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e4155c00 con 0x7f36e4156ea0 2024-03-06T07:32:51.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.407+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e4155750 con 0x7f36e4156ea0 2024-03-06T07:32:51.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.407+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36dc00ac30 con 0x7f36e4156ea0 2024-03-06T07:32:51.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.408+0000 7f36daffd640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:51.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.408+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f36dc08a320 con 0x7f36e4156ea0 2024-03-06T07:32:51.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.408+0000 7f36ea207640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:51.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.409+0000 7f36ea207640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f36e0036000 tx=0x7f36e003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:51.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.412+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f36dc058f00 con 0x7f36e4156ea0 2024-03-06T07:32:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.670+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36e404f330 con 0x7f36e4156ea0 2024-03-06T07:32:51.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.671+0000 7f36daffd640 1 -- 172.21.15.17:0/898743928 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f36dc058800 con 0x7f36e4156ea0 2024-03-06T07:32:51.674 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:51.674 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:51.674 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 msgr2=0x7f36c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f36e0036000 tx=0x7f36e003b040 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 msgr2=0x7f36e40aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c760 tx=0x7f36dc00cc20 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 shutdown_connections 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f36c806b4d0 0x7f36c806d990 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36e4156ea0 0x7f36e40aa8b0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e41506b0 0x7f36e40aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 --2- 172.21.15.17:0/898743928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e40acc00 0x7f36e41e8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 >> 172.21.15.17:0/898743928 conn(0x7f36e406b1a0 msgr2=0x7f36e406f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:51.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 shutdown_connections 2024-03-06T07:32:51.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:51.673+0000 7f36ec46b640 1 -- 172.21.15.17:0/898743928 wait complete. 2024-03-06T07:32:52.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:52.088 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:53.089 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.192+0000 7f479c439640 1 Processor -- start 2024-03-06T07:32:53.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.192+0000 7f479c439640 1 -- start start 2024-03-06T07:32:53.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479c439640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f479414f790 0x7f47941529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479c439640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4794155a40 0x7f4794155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479c439640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47940c28d0 con 0x7f47940c1aa0 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479c439640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47940c2680 con 0x7f479414f790 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479c439640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47940c2bb0 con 0x7f4794155a40 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f479414f790 0x7f47941529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f479414f790 0x7f47941529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36658/0 (socket says 172.21.15.17:36658) 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f47999d4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f47940c1aa0 0x7f47940c5720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57860/0 (socket says 172.21.15.17:57860) 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 -- 172.21.15.17:0/1277327521 learned_addr learned my addr 172.21.15.17:0/1277327521 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a9d6640 1 --2- 172.21.15.17:0/1277327521 >> v2:172.21.15.163:3300/0 conn(0x7f4794155a40 0x7f4794155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 -- 172.21.15.17:0/1277327521 >> v2:172.21.15.163:3300/0 conn(0x7f4794155a40 msgr2=0x7f4794155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 --2- 172.21.15.17:0/1277327521 >> v2:172.21.15.163:3300/0 conn(0x7f4794155a40 0x7f4794155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 -- 172.21.15.17:0/1277327521 >> v1:172.21.15.17:6789/0 conn(0x7f47940c1aa0 legacy=0x7f47940c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f47991d3640 1 -- 172.21.15.17:0/1277327521 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396292453 0 0) 0x7f47940c28d0 con 0x7f47940c1aa0 2024-03-06T07:32:53.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.193+0000 7f479a1d5640 1 -- 172.21.15.17:0/1277327521 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47940c2e90 con 0x7f479414f790 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.194+0000 7f479a1d5640 1 --2- 172.21.15.17:0/1277327521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f479414f790 0x7f47941529d0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f479406c960 tx=0x7f479002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538749d05fe2640c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f47991d3640 1 -- 172.21.15.17:0/1277327521 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790034070 con 0x7f479414f790 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f47991d3640 1 -- 172.21.15.17:0/1277327521 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790030860 con 0x7f479414f790 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f47991d3640 1 -- 172.21.15.17:0/1277327521 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790030c30 con 0x7f479414f790 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 -- 172.21.15.17:0/1277327521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f479414f790 msgr2=0x7f47941529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 --2- 172.21.15.17:0/1277327521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f479414f790 0x7f47941529d0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7f479406c960 tx=0x7f479002b9a0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 -- 172.21.15.17:0/1277327521 shutdown_connections 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 --2- 172.21.15.17:0/1277327521 >> v2:172.21.15.163:3300/0 conn(0x7f4794155a40 0x7f4794155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 --2- 172.21.15.17:0/1277327521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f479414f790 0x7f47941529d0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 -- 172.21.15.17:0/1277327521 >> 172.21.15.17:0/1277327521 conn(0x7f47940c6710 msgr2=0x7f47940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 -- 172.21.15.17:0/1277327521 shutdown_connections 2024-03-06T07:32:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.195+0000 7f479c439640 1 -- 172.21.15.17:0/1277327521 wait complete. 2024-03-06T07:32:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.196+0000 7f479c439640 1 Processor -- start 2024-03-06T07:32:53.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.196+0000 7f479c439640 1 -- start start 2024-03-06T07:32:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47940c1aa0 0x7f4794071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f479414f790 0x7f47941dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47940c0d60 con 0x7f4794155a40 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4794156390 con 0x7f47940c1aa0 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479c439640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47940c5e30 con 0x7f479414f790 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479a9d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479a9d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36660/0 (socket says 172.21.15.17:36660) 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479a9d6640 1 -- 172.21.15.17:0/2739676887 learned_addr learned my addr 172.21.15.17:0/2739676887 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f47999d4640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f479414f790 0x7f47941dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.197+0000 7f479a1d5640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47940c1aa0 0x7f4794071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 -- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f479414f790 msgr2=0x7f47941dfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f479414f790 0x7f47941dfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 -- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47940c1aa0 msgr2=0x7f4794071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47940c1aa0 0x7f4794071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 -- 172.21.15.17:0/2739676887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479414e320 con 0x7f4794155a40 2024-03-06T07:32:53.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.198+0000 7f479a9d6640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f478c007e90 tx=0x7f478c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.199+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c014420 con 0x7f4794155a40 2024-03-06T07:32:53.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.199+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c019070 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.199+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479414e660 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.199+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c014800 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.200+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47940702f0 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.200+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479404f330 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.201+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f478c0149a0 con 0x7f4794155a40 2024-03-06T07:32:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.201+0000 7f4786ffd640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.201+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f478c08ab60 con 0x7f4794155a40 2024-03-06T07:32:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.202+0000 7f479a1d5640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.202+0000 7f479a1d5640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 0x7f477806d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f4790008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:53.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.206+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f479414e660 con 0x7f4794155a40 2024-03-06T07:32:53.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.423+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4794156170 con 0x7f4794155a40 2024-03-06T07:32:53.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.424+0000 7f4786ffd640 1 -- 172.21.15.17:0/2739676887 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f478c059100 con 0x7f4794155a40 2024-03-06T07:32:53.427 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:53.427 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:53.427 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:53.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 msgr2=0x7f477806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 0x7f477806d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4790036000 tx=0x7f4790008030 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 msgr2=0x7f47941eac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7f478c007e90 tx=0x7f478c00e4b0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 shutdown_connections 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4794155a40 0x7f47941eac80 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f479414f790 0x7f47941dfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 --2- 172.21.15.17:0/2739676887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47940c1aa0 0x7f4794071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 >> 172.21.15.17:0/2739676887 conn(0x7f47940c6710 msgr2=0x7f47940c30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 shutdown_connections 2024-03-06T07:32:53.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:53.426+0000 7f479c439640 1 -- 172.21.15.17:0/2739676887 wait complete. 2024-03-06T07:32:53.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:53.845 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:54.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:54.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.950+0000 7fd7feb13640 1 Processor -- start 2024-03-06T07:32:54.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.950+0000 7fd7feb13640 1 -- start start 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7feb13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 0x7fd7f8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7feb13640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7f8159460 0x7fd7f80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7feb13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80c7760 con 0x7fd7f8152920 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7feb13640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f81587b0 con 0x7fd7f8159460 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7feb13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c7a40 con 0x7fd7f80c8a00 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fd0b0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd7f8159460 0x7fd7f80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7f7fff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 0x7fd7f8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fc8af640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd7f8152920 0x7fd7f8155c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:39436/0 (socket says 172.21.15.17:39436) 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fc8af640 1 -- 172.21.15.17:0/3445456655 learned_addr learned my addr 172.21.15.17:0/3445456655 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:32:54.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fd0b0640 1 -- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 msgr2=0x7fd7f8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:54.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fd0b0640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 0x7fd7f8158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fd0b0640 1 -- 172.21.15.17:0/3445456655 >> v1:172.21.15.163:6789/0 conn(0x7fd7f8152920 legacy=0x7fd7f8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7fd0b0640 1 -- 172.21.15.17:0/3445456655 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80c7d20 con 0x7fd7f8159460 2024-03-06T07:32:54.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.951+0000 7fd7f7fff640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 0x7fd7f8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:32:54.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.952+0000 7fd7fd0b0640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f8159460 0x7fd7f80c5330 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009d90 tx=0x7fd7f002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47eb6e4f4381a034 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.952+0000 7fd7f77fe640 1 -- 172.21.15.17:0/3445456655 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0039070 con 0x7fd7f8159460 2024-03-06T07:32:54.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.952+0000 7fd7f77fe640 1 -- 172.21.15.17:0/3445456655 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f002cbc0 con 0x7fd7f8159460 2024-03-06T07:32:54.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.952+0000 7fd7f77fe640 1 -- 172.21.15.17:0/3445456655 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0034440 con 0x7fd7f8159460 2024-03-06T07:32:54.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 -- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f8159460 msgr2=0x7fd7f80c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f8159460 0x7fd7f80c5330 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009d90 tx=0x7fd7f002be10 comp rx=0 tx=0).stop 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 -- 172.21.15.17:0/3445456655 shutdown_connections 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f8159460 0x7fd7f80c5330 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 --2- 172.21.15.17:0/3445456655 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f80c8a00 0x7fd7f8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 -- 172.21.15.17:0/3445456655 >> 172.21.15.17:0/3445456655 conn(0x7fd7f80ac560 msgr2=0x7fd7f80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 -- 172.21.15.17:0/3445456655 shutdown_connections 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.953+0000 7fd7feb13640 1 -- 172.21.15.17:0/3445456655 wait complete. 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.954+0000 7fd7feb13640 1 Processor -- start 2024-03-06T07:32:54.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.954+0000 7fd7feb13640 1 -- start start 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f8152920 0x7fd7f8147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f81578c0 0x7fd7f81480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c7390 con 0x7fd7f80c8a00 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f806d030 con 0x7fd7f81578c0 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7feb13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c7510 con 0x7fd7f8152920 2024-03-06T07:32:54.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36676/0 (socket says 172.21.15.17:36676) 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 -- 172.21.15.17:0/1381863805 learned_addr learned my addr 172.21.15.17:0/1381863805 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7f7fff640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f8152920 0x7fd7f8147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fd0b0640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f81578c0 0x7fd7f81480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 -- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f8152920 msgr2=0x7fd7f8147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f8152920 0x7fd7f8147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 -- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f81578c0 msgr2=0x7fd7f81480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f81578c0 0x7fd7f81480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 -- 172.21.15.17:0/1381863805 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8157660 con 0x7fd7f80c8a00 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7fc8af640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c400 tx=0x7fd7ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.955+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec017070 con 0x7fd7f80c8a00 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.956+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec005ce0 con 0x7fd7f80c8a00 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.956+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f80c4610 con 0x7fd7f80c8a00 2024-03-06T07:32:54.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.956+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0072f0 con 0x7fd7f80c8a00 2024-03-06T07:32:54.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.956+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f8156e60 con 0x7fd7f80c8a00 2024-03-06T07:32:54.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.957+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f804f330 con 0x7fd7f80c8a00 2024-03-06T07:32:54.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.957+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7ec01a070 con 0x7fd7f80c8a00 2024-03-06T07:32:54.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.957+0000 7fd7f57fa640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 0x7fd7dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:54.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.958+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd7ec089820 con 0x7fd7f80c8a00 2024-03-06T07:32:54.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.958+0000 7fd7f7fff640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 0x7fd7dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:54.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.958+0000 7fd7f7fff640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 0x7fd7dc06d9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd7f814e850 tx=0x7fd7e4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:54.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:54.962+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd7ec057d30 con 0x7fd7f80c8a00 2024-03-06T07:32:55.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.283+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7f806cda0 con 0x7fd7f80c8a00 2024-03-06T07:32:55.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.284+0000 7fd7f57fa640 1 -- 172.21.15.17:0/1381863805 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fd7ec05bdd0 con 0x7fd7f80c8a00 2024-03-06T07:32:55.287 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:55.287 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:55.287 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 msgr2=0x7fd7dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 0x7fd7dc06d9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fd7f814e850 tx=0x7fd7e4009190 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 msgr2=0x7fd7f80c1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c400 tx=0x7fd7ec00c8c0 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 shutdown_connections 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd7dc06b500 0x7fd7dc06d9c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7f81578c0 0x7fd7f81480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7f8152920 0x7fd7f8147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 --2- 172.21.15.17:0/1381863805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f80c8a00 0x7fd7f80c1cf0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 >> 172.21.15.17:0/1381863805 conn(0x7fd7f80ac560 msgr2=0x7fd7f81503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 shutdown_connections 2024-03-06T07:32:55.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:55.287+0000 7fd7feb13640 1 -- 172.21.15.17:0/1381863805 wait complete. 2024-03-06T07:32:55.702 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:55.703 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:56.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:56.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 Processor -- start 2024-03-06T07:32:56.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 -- start start 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffae014f790 0x7ffae01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffae0155a40 0x7ffae0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae00c28d0 con 0x7ffae00c1aa0 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae00c2680 con 0x7ffae0155a40 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae7b13640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae00c2bb0 con 0x7ffae014f790 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffae0155a40 0x7ffae0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffae0155a40 0x7ffae0155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36684/0 (socket says 172.21.15.17:36684) 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 -- 172.21.15.17:0/4280022387 learned_addr learned my addr 172.21.15.17:0/4280022387 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:56.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae58af640 1 --2- 172.21.15.17:0/4280022387 >> v2:172.21.15.163:3300/0 conn(0x7ffae014f790 0x7ffae01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 -- 172.21.15.17:0/4280022387 >> v2:172.21.15.163:3300/0 conn(0x7ffae014f790 msgr2=0x7ffae01529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 --2- 172.21.15.17:0/4280022387 >> v2:172.21.15.163:3300/0 conn(0x7ffae014f790 0x7ffae01529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 -- 172.21.15.17:0/4280022387 >> v1:172.21.15.163:6789/0 conn(0x7ffae00c1aa0 legacy=0x7ffae00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.806+0000 7ffae60b0640 1 -- 172.21.15.17:0/4280022387 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae00c2e90 con 0x7ffae0155a40 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.807+0000 7ffae60b0640 1 --2- 172.21.15.17:0/4280022387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae0155a40 0x7ffae0155e50 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7ffae00c3170 tx=0x7ffad802e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8b1117c817886 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.807+0000 7ffae48ad640 1 -- 172.21.15.17:0/4280022387 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad800b0d0 con 0x7ffae0155a40 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.807+0000 7ffae48ad640 1 -- 172.21.15.17:0/4280022387 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad800b270 con 0x7ffae0155a40 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.807+0000 7ffae48ad640 1 -- 172.21.15.17:0/4280022387 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8009600 con 0x7ffae0155a40 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 -- 172.21.15.17:0/4280022387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae0155a40 msgr2=0x7ffae0155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 --2- 172.21.15.17:0/4280022387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae0155a40 0x7ffae0155e50 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7ffae00c3170 tx=0x7ffad802e180 comp rx=0 tx=0).stop 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 -- 172.21.15.17:0/4280022387 shutdown_connections 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 --2- 172.21.15.17:0/4280022387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae0155a40 0x7ffae0155e50 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 --2- 172.21.15.17:0/4280022387 >> v2:172.21.15.163:3300/0 conn(0x7ffae014f790 0x7ffae01529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 -- 172.21.15.17:0/4280022387 >> 172.21.15.17:0/4280022387 conn(0x7ffae00c6710 msgr2=0x7ffae00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 -- 172.21.15.17:0/4280022387 shutdown_connections 2024-03-06T07:32:56.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.808+0000 7ffae7b13640 1 -- 172.21.15.17:0/4280022387 wait complete. 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 Processor -- start 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 -- start start 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae00c1aa0 0x7ffae014bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 0x7ffae0149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.809+0000 7ffae7b13640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00c0d60 con 0x7ffae014f790 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae7b13640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae0156390 con 0x7ffae0155a40 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae7b13640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00c5e30 con 0x7ffae00c1aa0 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36694/0 (socket says 172.21.15.17:36694) 2024-03-06T07:32:56.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 -- 172.21.15.17:0/3096355607 learned_addr learned my addr 172.21.15.17:0/3096355607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae60b0640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 0x7ffae0149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae58af640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae00c1aa0 0x7ffae014bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 -- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae00c1aa0 msgr2=0x7ffae014bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae00c1aa0 0x7ffae014bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 -- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 msgr2=0x7ffae0149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 0x7ffae0149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 -- 172.21.15.17:0/3096355607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae01ee940 con 0x7ffae014f790 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae60b0640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 0x7ffae0149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae50ae640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c470 tx=0x7ffad0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00052d0 con 0x7ffae014f790 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad0005470 con 0x7ffae014f790 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0007260 con 0x7ffae014f790 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae01eec20 con 0x7ffae014f790 2024-03-06T07:32:56.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.810+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae01ef1e0 con 0x7ffae014f790 2024-03-06T07:32:56.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.811+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae004f330 con 0x7ffae014f790 2024-03-06T07:32:56.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.812+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffad0007400 con 0x7ffae014f790 2024-03-06T07:32:56.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.812+0000 7ffad67fc640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 0x7ffac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:56.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.812+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7ffad00899c0 con 0x7ffae014f790 2024-03-06T07:32:56.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.812+0000 7ffae58af640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 0x7ffac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:56.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.813+0000 7ffae58af640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 0x7ffac406d990 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffadc006fd0 tx=0x7ffadc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:56.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:56.817+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffad008be20 con 0x7ffae014f790 2024-03-06T07:32:57.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.035+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae00a5f60 con 0x7ffae014f790 2024-03-06T07:32:57.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.035+0000 7ffad67fc640 1 -- 172.21.15.17:0/3096355607 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7ffad0058520 con 0x7ffae014f790 2024-03-06T07:32:57.039 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:57.039 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:57.039 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 msgr2=0x7ffac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 0x7ffac406d990 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffadc006fd0 tx=0x7ffadc008040 comp rx=0 tx=0).stop 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 msgr2=0x7ffae0154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7ffad000c470 tx=0x7ffad0001e70 comp rx=0 tx=0).stop 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 shutdown_connections 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ffac406b4d0 0x7ffac406d990 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae0155a40 0x7ffae0149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffae014f790 0x7ffae0154b30 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 --2- 172.21.15.17:0/3096355607 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae00c1aa0 0x7ffae014bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:57.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 >> 172.21.15.17:0/3096355607 conn(0x7ffae00c6710 msgr2=0x7ffae00c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:57.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 shutdown_connections 2024-03-06T07:32:57.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:57.037+0000 7ffae7b13640 1 -- 172.21.15.17:0/3096355607 wait complete. 2024-03-06T07:32:57.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:57.453 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:32:58.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:32:58.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.554+0000 7fa86c0ed640 1 Processor -- start 2024-03-06T07:32:58.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.554+0000 7fa86c0ed640 1 -- start start 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8640c88c0 0x7fa8640c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8640c5880 0x7fa864158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8640c8dd0 con 0x7fa8640c5880 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8640c73d0 con 0x7fa8640c88c0 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86c0ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640be2e0 con 0x7fa86414f360 2024-03-06T07:32:58.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8640c5880 0x7fa864158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa8640c5880 0x7fa864158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36696/0 (socket says 172.21.15.17:36696) 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 -- 172.21.15.17:0/2780871412 learned_addr learned my addr 172.21.15.17:0/2780871412 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa869e89640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa869688640 1 --2- 172.21.15.17:0/2780871412 >> v2:172.21.15.163:3300/0 conn(0x7fa8640c88c0 0x7fa8640c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 -- 172.21.15.17:0/2780871412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 msgr2=0x7fa864155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 -- 172.21.15.17:0/2780871412 >> v2:172.21.15.163:3300/0 conn(0x7fa8640c88c0 msgr2=0x7fa8640c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 --2- 172.21.15.17:0/2780871412 >> v2:172.21.15.163:3300/0 conn(0x7fa8640c88c0 0x7fa8640c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.555+0000 7fa86a68a640 1 -- 172.21.15.17:0/2780871412 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8640c7670 con 0x7fa8640c5880 2024-03-06T07:32:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.556+0000 7fa86a68a640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c5880 0x7fa864158b60 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fa8640cbac0 tx=0x7fa85c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f3db5ad5bd63f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.556+0000 7fa868e87640 1 -- 172.21.15.17:0/2780871412 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c02c9e0 con 0x7fa8640c5880 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.556+0000 7fa868e87640 1 -- 172.21.15.17:0/2780871412 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c02cb80 con 0x7fa8640c5880 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.556+0000 7fa868e87640 1 -- 172.21.15.17:0/2780871412 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0343f0 con 0x7fa8640c5880 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 -- 172.21.15.17:0/2780871412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c5880 msgr2=0x7fa864158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c5880 0x7fa864158b60 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fa8640cbac0 tx=0x7fa85c02be40 comp rx=0 tx=0).stop 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 -- 172.21.15.17:0/2780871412 shutdown_connections 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c5880 0x7fa864158b60 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/2780871412 >> v2:172.21.15.163:3300/0 conn(0x7fa8640c88c0 0x7fa8640c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/2780871412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 -- 172.21.15.17:0/2780871412 >> 172.21.15.17:0/2780871412 conn(0x7fa8640ac570 msgr2=0x7fa8640ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 -- 172.21.15.17:0/2780871412 shutdown_connections 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.557+0000 7fa86c0ed640 1 -- 172.21.15.17:0/2780871412 wait complete. 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.558+0000 7fa86c0ed640 1 Processor -- start 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.558+0000 7fa86c0ed640 1 -- start start 2024-03-06T07:32:58.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 0x7fa8641ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8641590a0 con 0x7fa8640c88c0 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640c7c30 con 0x7fa8640c5880 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa864159220 con 0x7fa86414f360 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36712/0 (socket says 172.21.15.17:36712) 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 -- 172.21.15.17:0/4252878828 learned_addr learned my addr 172.21.15.17:0/4252878828 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86a68a640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869e89640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 0x7fa8641ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 -- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 msgr2=0x7fa864156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 -- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 msgr2=0x7fa8641ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 0x7fa8641ee780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 -- 172.21.15.17:0/4252878828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8640b0560 con 0x7fa8640c88c0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869e89640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 0x7fa8641ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa869688640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c7f0 tx=0x7fa85400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854015070 con 0x7fa8640c88c0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854004220 con 0x7fa8640c88c0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640ac2d0 con 0x7fa8640c88c0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854007620 con 0x7fa8640c88c0 2024-03-06T07:32:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.559+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa864157d60 con 0x7fa8640c88c0 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.560+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86404f330 con 0x7fa8640c88c0 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.561+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8540077c0 con 0x7fa8640c88c0 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.562+0000 7fa85a7fc640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.562+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fa85408a120 con 0x7fa8640c88c0 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.562+0000 7fa869e89640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:32:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.563+0000 7fa869e89640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 0x7fa84806d990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fa860006fd0 tx=0x7fa860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:32:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.566+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa854058750 con 0x7fa8640c88c0 2024-03-06T07:32:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.783+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa864158b00 con 0x7fa8640c88c0 2024-03-06T07:32:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.784+0000 7fa85a7fc640 1 -- 172.21.15.17:0/4252878828 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7fa85405c7f0 con 0x7fa8640c88c0 2024-03-06T07:32:58.786 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:32:58.786 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:32:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 msgr2=0x7fa84806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 0x7fa84806d990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fa860006fd0 tx=0x7fa860008040 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 msgr2=0x7fa8640ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c7f0 tx=0x7fa85400ccb0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 shutdown_connections 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fa84806b4d0 0x7fa84806d990 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa86414f360 0x7fa864156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8640c88c0 0x7fa8640ab7d0 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 --2- 172.21.15.17:0/4252878828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8640c5880 0x7fa8641ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 >> 172.21.15.17:0/4252878828 conn(0x7fa8640ac570 msgr2=0x7fa8640ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 shutdown_connections 2024-03-06T07:32:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:32:58.786+0000 7fa86c0ed640 1 -- 172.21.15.17:0/4252878828 wait complete. 2024-03-06T07:32:59.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 218, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-06T07:29:14.378013+0000', 'modified': '2024-03-06T07:31:06.887130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 216, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 0, 'in': [0, 1], 'up': {'mds_0': 10791, 'mds_1': 8162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8162': {'gid': 8162, 'name': 'a', 'rank': 1, 'incarnation': 212, 'state': 'up:stopping', 'state_seq': 26, 'addr': '172.21.15.17:6835/1774857824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6833', 'nonce': 1774857824}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 1774857824}]}, 'join_fscid': -1, 'laggy_since': '2024-03-06T07:31:06.887130+0000', 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10791': {'gid': 10791, 'name': 'c', 'rank': 0, 'incarnation': 198, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.17:6834/2434034079', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 2434034079}, {'type': 'v1', 'addr': '172.21.15.17:6834', 'nonce': 2434034079}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10791, 'qdb_cluster': [10791]} 2024-03-06T07:32:59.201 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8162:up:stopping 2024-03-06T07:33:00.202 DEBUG:tasks.cephfs.filesystem:status = { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 16, "epoch": 218, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 16, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-06T07:29:14.378013+0000", "damaged": [], "data_pools": [ 34 ], "enabled": true, "epoch": 218, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_10791": { "addr": "172.21.15.17:6834/2434034079", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6832", "nonce": 2434034079, "type": "v2" }, { "addr": "172.21.15.17:6834", "nonce": 2434034079, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 10791, "incarnation": 198, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_8162": { "addr": "172.21.15.17:6835/1774857824", "addrs": { "addrvec": [ { "addr": "172.21.15.17:6833", "nonce": 1774857824, "type": "v2" }, { "addr": "172.21.15.17:6835", "nonce": 1774857824, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8162, "incarnation": 212, "join_fscid": -1, "laggy_since": "2024-03-06T07:31:06.887130+0000", "name": "a", "rank": 1, "state": "up:stopping", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 216, "max_file_size": 1099511627776, "max_mds": 0, "max_xattr_size": 65536, "metadata_pool": 33, "modified": "2024-03-06T07:31:06.887130+0000", "qdb_cluster": [ 10791 ], "qdb_leader": 10791, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 10791, "mds_1": 8162 } } } ], "standbys": [] } 2024-03-06T07:33:00.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-06T07:33:00.203 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:33:00.242 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-06T07:33:00.286 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-06T07:33:00.286 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:33:00.286 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:00.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:33:00.301 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:33:00.301 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:00.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:33:00.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 Processor -- start 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 -- start start 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac814f360 0x7f7ac8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ac80c88c0 0x7f7ac80c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80c73d0 con 0x7f7ac80c5880 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80c8dd0 con 0x7f7ac80c88c0 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acf165640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80be2e0 con 0x7f7ac814f360 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7ac7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7ac80c88c0 0x7f7ac80c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.444+0000 7f7acd702640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7ac80c5880 0x7f7ac8158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:57864/0 (socket says 172.21.15.17:57864) 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7acd702640 1 -- 172.21.15.17:0/1403430968 learned_addr learned my addr 172.21.15.17:0/1403430968 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7accf01640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac814f360 0x7f7ac8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7ac7fff640 1 -- 172.21.15.17:0/1403430968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac814f360 msgr2=0x7f7ac8155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7ac7fff640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac814f360 0x7f7ac8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7ac7fff640 1 -- 172.21.15.17:0/1403430968 >> v1:172.21.15.17:6789/0 conn(0x7f7ac80c5880 legacy=0x7f7ac8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7ac7fff640 1 -- 172.21.15.17:0/1403430968 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80c7670 con 0x7f7ac80c88c0 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.445+0000 7f7ac77fe640 1 -- 172.21.15.17:0/1403430968 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892377390 0 0) 0x7f7ac80c73d0 con 0x7f7ac80c5880 2024-03-06T07:33:00.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7ac7fff640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac80c88c0 0x7f7ac80c5340 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80097a0 tx=0x7f7ab802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890f76ba883d203 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7ac77fe640 1 -- 172.21.15.17:0/1403430968 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80315a0 con 0x7f7ac80c88c0 2024-03-06T07:33:00.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7ac77fe640 1 -- 172.21.15.17:0/1403430968 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8037070 con 0x7f7ac80c88c0 2024-03-06T07:33:00.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7ac77fe640 1 -- 172.21.15.17:0/1403430968 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8031960 con 0x7f7ac80c88c0 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 -- 172.21.15.17:0/1403430968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac80c88c0 msgr2=0x7f7ac80c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac80c88c0 0x7f7ac80c5340 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80097a0 tx=0x7f7ab802b840 comp rx=0 tx=0).stop 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 -- 172.21.15.17:0/1403430968 shutdown_connections 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac80c88c0 0x7f7ac80c5340 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 --2- 172.21.15.17:0/1403430968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac814f360 0x7f7ac8155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 -- 172.21.15.17:0/1403430968 >> 172.21.15.17:0/1403430968 conn(0x7f7ac80ac570 msgr2=0x7f7ac80ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 -- 172.21.15.17:0/1403430968 shutdown_connections 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.446+0000 7f7acf165640 1 -- 172.21.15.17:0/1403430968 wait complete. 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.448+0000 7f7acf165640 1 Processor -- start 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.448+0000 7f7acf165640 1 -- start start 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac80c5880 0x7f7ac814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac80c88c0 0x7f7ac814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac8159270 con 0x7f7ac814f360 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c7c30 con 0x7f7ac80c88c0 2024-03-06T07:33:00.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acf165640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81593f0 con 0x7f7ac80c5880 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36724/0 (socket says 172.21.15.17:36724) 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 -- 172.21.15.17:0/473510340 learned_addr learned my addr 172.21.15.17:0/473510340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 -- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac80c5880 msgr2=0x7f7ac814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac80c5880 0x7f7ac814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 -- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac80c88c0 msgr2=0x7f7ac814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac80c88c0 0x7f7ac814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.449+0000 7f7acd702640 1 -- 172.21.15.17:0/473510340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac814c5f0 con 0x7f7ac814f360 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7acd702640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000c7a0 tx=0x7f7ac000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0018070 con 0x7f7ac814f360 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000a640 con 0x7f7ac814f360 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac81edf70 con 0x7f7ac814f360 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000aab0 con 0x7f7ac814f360 2024-03-06T07:33:00.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.450+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac81ee530 con 0x7f7ac814f360 2024-03-06T07:33:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.451+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac804f330 con 0x7f7ac814f360 2024-03-06T07:33:00.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.451+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ac001b070 con 0x7f7ac814f360 2024-03-06T07:33:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.452+0000 7f7ac57fa640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 0x7f7aa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.452+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7ac0089a20 con 0x7f7ac814f360 2024-03-06T07:33:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.452+0000 7f7accf01640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 0x7f7aa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.453+0000 7f7accf01640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 0x7f7aa806d9c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000a810 tx=0x7f7ab0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.456+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ac00586a0 con 0x7f7ac814f360 2024-03-06T07:33:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.674+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ac806cda0 con 0x7f7ac814f360 2024-03-06T07:33:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.675+0000 7f7ac57fa640 1 -- 172.21.15.17:0/473510340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f7ac0057fa0 con 0x7f7ac814f360 2024-03-06T07:33:00.676 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:00.677 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:33:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 msgr2=0x7f7aa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 0x7f7aa806d9c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000a810 tx=0x7f7ab0008040 comp rx=0 tx=0).stop 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 msgr2=0x7f7ac814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000c7a0 tx=0x7f7ac000cc60 comp rx=0 tx=0).stop 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 shutdown_connections 2024-03-06T07:33:00.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.676+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7aa806b500 0x7f7aa806d9c0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ac814f360 0x7f7ac814c0b0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7ac80c88c0 0x7f7ac814bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 --2- 172.21.15.17:0/473510340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7ac80c5880 0x7f7ac814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 >> 172.21.15.17:0/473510340 conn(0x7f7ac80ac570 msgr2=0x7f7ac80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 shutdown_connections 2024-03-06T07:33:00.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.677+0000 7f7acf165640 1 -- 172.21.15.17:0/473510340 wait complete. 2024-03-06T07:33:00.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.792+0000 7f5920f56640 1 Processor -- start 2024-03-06T07:33:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.792+0000 7f5920f56640 1 -- start start 2024-03-06T07:33:00.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5920f56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f591c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5920f56640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c0c73d0 con 0x7f591c0c88c0 2024-03-06T07:33:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5920f56640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c0be2e0 con 0x7f591c158b60 2024-03-06T07:33:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5920f56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c159700 con 0x7f591c14f360 2024-03-06T07:33:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5919d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f591c0c88c0 0x7f591c0c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43812/0 (socket says 172.21.15.17:43812) 2024-03-06T07:33:00.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f5919d9b640 1 -- 172.21.15.17:0/776967582 learned_addr learned my addr 172.21.15.17:0/776967582 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561464583 0 0) 0x7f591c0c73d0 con 0x7f591c0c88c0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5908003520 con 0x7f591c0c88c0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.793+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f591c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3750483491 0 0) 0x7f5908003520 con 0x7f591c0c88c0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 msgr2=0x7f591c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f591c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 >> v1:172.21.15.163:6789/0 conn(0x7f591c158b60 legacy=0x7f591c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0c7670 con 0x7f591c0c88c0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f590c001670 con 0x7f591c0c88c0 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f591c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:00.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 >> v1:172.21.15.17:6789/0 conn(0x7f591c0c88c0 legacy=0x7f591c0c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f5908003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5908004800 0x7f5908004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f590800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c159700 con 0x7f59080039e0 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59080054e0 con 0x7f5908004800 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590800bf00 con 0x7f591c14f360 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f5908003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f5919d9b640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5908004800 0x7f5908004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591ad9d640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f590800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 -- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 msgr2=0x7f590800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c14f360 0x7f590800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 -- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5908004800 msgr2=0x7f5908004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5908004800 0x7f5908004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.794+0000 7f591a59c640 1 -- 172.21.15.17:0/776967582 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5908003520 con 0x7f59080039e0 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.795+0000 7f591a59c640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f5908003e00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f59040098a0 tx=0x7f5908005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e582ee74594e6e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.795+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400f040 con 0x7f59080039e0 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.795+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904004230 con 0x7f59080039e0 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f591959a640 1 -- 172.21.15.17:0/776967582 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904003610 con 0x7f59080039e0 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 -- 172.21.15.17:0/776967582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 msgr2=0x7f5908003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 --2- 172.21.15.17:0/776967582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f5908003e00 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f59040098a0 tx=0x7f5908005830 comp rx=0 tx=0).stop 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f591ad9d640 1 -- 172.21.15.17:0/776967582 reap_dead start 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 -- 172.21.15.17:0/776967582 shutdown_connections 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 -- 172.21.15.17:0/776967582 >> 172.21.15.17:0/776967582 conn(0x7f591c0ac570 msgr2=0x7f591c0ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 -- 172.21.15.17:0/776967582 shutdown_connections 2024-03-06T07:33:00.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.796+0000 7f5920f56640 1 -- 172.21.15.17:0/776967582 wait complete. 2024-03-06T07:33:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.797+0000 7f5920f56640 1 Processor -- start 2024-03-06T07:33:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.797+0000 7f5920f56640 1 -- start start 2024-03-06T07:33:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c152160 0x7f591c1ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 0x7f591c0c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0c7c30 con 0x7f59080039e0 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0c5a50 con 0x7f591c1eef30 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f5920f56640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0c5bd0 con 0x7f591c152160 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41882/0 (socket says 172.21.15.17:41882) 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 -- 172.21.15.17:0/1333577155 learned_addr learned my addr 172.21.15.17:0/1333577155 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 -- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c152160 msgr2=0x7f591c1ee9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c152160 0x7f591c1ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591ad9d640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 0x7f591c0c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.798+0000 7f591a59c640 1 -- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 msgr2=0x7f591c0c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f591a59c640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 0x7f591c0c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f591a59c640 1 -- 172.21.15.17:0/1333577155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0ae370 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f591ad9d640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 0x7f591c0c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f591a59c640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f59040099c0 tx=0x7f5904004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904048070 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904030430 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c0ae650 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904030800 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.799+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c0aec10 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.800+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c0cb830 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.800+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f590404b070 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.801+0000 7f5912ffd640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.801+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f59040abd70 con 0x7f59080039e0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.801+0000 7f5919d9b640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:00.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.802+0000 7f5919d9b640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 0x7f58fc06d990 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c001050 tx=0x7f590c001b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:00.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.805+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f591c0cb830 con 0x7f59080039e0 2024-03-06T07:33:00.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.985+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f591c04f330 con 0x7f59080039e0 2024-03-06T07:33:00.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.987+0000 7f5912ffd640 1 -- 172.21.15.17:0/1333577155 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f5904034020 con 0x7f59080039e0 2024-03-06T07:33:00.988 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:00.989 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":216,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:30:31.883062+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.13","mappings":[{"from":0,"to":2}]},{"pgid":"34.19","mappings":[{"from":0,"to":1}]},{"pgid":"34.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6834/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6837/370957199":"2024-03-07T07:29:56.878364+0000","172.21.15.163:6836/370957199":"2024-03-07T07:29:56.878364+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:00.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 msgr2=0x7f58fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 0x7f58fc06d990 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f590c001050 tx=0x7f590c001b60 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 msgr2=0x7f591c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f59040099c0 tx=0x7f5904004730 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 shutdown_connections 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f58fc06b4d0 0x7f58fc06d990 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f591c1eef30 0x7f591c0c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f591c152160 0x7f591c1ee9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 --2- 172.21.15.17:0/1333577155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59080039e0 0x7f591c151c20 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 >> 172.21.15.17:0/1333577155 conn(0x7f591c0ac570 msgr2=0x7f591c0c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 shutdown_connections 2024-03-06T07:33:00.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:00.989+0000 7f5920f56640 1 -- 172.21.15.17:0/1333577155 wait complete. 2024-03-06T07:33:01.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-06T07:19:57.384221+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-06T07:29:12.634238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2024-03-06T07:29:13.343125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-06T07:33:01.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-06T07:33:01.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.104+0000 7f70bc13a640 1 Processor -- start 2024-03-06T07:33:01.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.104+0000 7f70bc13a640 1 -- start start 2024-03-06T07:33:01.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70bc13a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70b40a8290 0x7f70b41539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70bc13a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70b40c69a0 0x7f70b4072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70bc13a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40c62a0 con 0x7f70b40c4740 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70bc13a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b4153f00 con 0x7f70b40a8290 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70bc13a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40c4c00 con 0x7f70b40c69a0 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70b40a8290 0x7f70b41539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f70b40a8290 0x7f70b41539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41892/0 (socket says 172.21.15.17:41892) 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3104685646 learned_addr learned my addr 172.21.15.17:0/3104685646 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70ba6d7640 1 --2- 172.21.15.17:0/3104685646 >> v2:172.21.15.163:3300/0 conn(0x7f70b40c69a0 0x7f70b4072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3104685646 >> v2:172.21.15.163:3300/0 conn(0x7f70b40c69a0 msgr2=0x7f70b4072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 --2- 172.21.15.17:0/3104685646 >> v2:172.21.15.163:3300/0 conn(0x7f70b40c69a0 0x7f70b4072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3104685646 >> v1:172.21.15.163:6789/0 conn(0x7f70b40c4740 legacy=0x7f70b40701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.105+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3104685646 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40c4ee0 con 0x7f70b40a8290 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.106+0000 7f70b9ed6640 1 --2- 172.21.15.17:0/3104685646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b41539c0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f70b40722a0 tx=0x7f70b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17705ed8c704cf6c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.106+0000 7f70b8ed4640 1 -- 172.21.15.17:0/3104685646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b002e070 con 0x7f70b40a8290 2024-03-06T07:33:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.106+0000 7f70b8ed4640 1 -- 172.21.15.17:0/3104685646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0037070 con 0x7f70b40a8290 2024-03-06T07:33:01.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.106+0000 7f70b8ed4640 1 -- 172.21.15.17:0/3104685646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0031a00 con 0x7f70b40a8290 2024-03-06T07:33:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 -- 172.21.15.17:0/3104685646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 msgr2=0x7f70b41539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3104685646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b41539c0 secure :-1 s=READY pgs=2675 cs=0 l=1 rev1=1 crypto rx=0x7f70b40722a0 tx=0x7f70b002b880 comp rx=0 tx=0).stop 2024-03-06T07:33:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 -- 172.21.15.17:0/3104685646 shutdown_connections 2024-03-06T07:33:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3104685646 >> v2:172.21.15.163:3300/0 conn(0x7f70b40c69a0 0x7f70b4072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3104685646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b41539c0 unknown :-1 s=CLOSED pgs=2675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 -- 172.21.15.17:0/3104685646 >> 172.21.15.17:0/3104685646 conn(0x7f70b40c5560 msgr2=0x7f70b40c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 -- 172.21.15.17:0/3104685646 shutdown_connections 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.107+0000 7f70bc13a640 1 -- 172.21.15.17:0/3104685646 wait complete. 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 Processor -- start 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 -- start start 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 0x7f70b40c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 0x7f70b414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c6df0 con 0x7f70b40a8290 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.108+0000 7f70bc13a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b406fc80 con 0x7f70b40c69a0 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70bc13a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c6f70 con 0x7f70b40c4740 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41896/0 (socket says 172.21.15.17:41896) 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3946438406 learned_addr learned my addr 172.21.15.17:0/3946438406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70ba6d7640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 0x7f70b414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b96d5640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 0x7f70b40c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 msgr2=0x7f70b40c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 0x7f70b40c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 msgr2=0x7f70b414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 0x7f70b414dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 -- 172.21.15.17:0/3946438406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b41562d0 con 0x7f70b40a8290 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b96d5640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 0x7f70b40c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70ba6d7640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 0x7f70b414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70b9ed6640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f70b002b8c0 tx=0x7f70b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0032880 con 0x7f70b40a8290 2024-03-06T07:33:01.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0032a40 con 0x7f70b40a8290 2024-03-06T07:33:01.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0032e30 con 0x7f70b40a8290 2024-03-06T07:33:01.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b41565b0 con 0x7f70b40a8290 2024-03-06T07:33:01.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.109+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b414e990 con 0x7f70b40a8290 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.111+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b404f330 con 0x7f70b40a8290 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.111+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70b004c060 con 0x7f70b40a8290 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.111+0000 7f70aa7fc640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 0x7f709806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.111+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f70b00ac0b0 con 0x7f70b40a8290 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.112+0000 7f70b96d5640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 0x7f709806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.112+0000 7f70b96d5640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 0x7f709806d990 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f70b41ec050 tx=0x7f70a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.116+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70b0040070 con 0x7f70b40a8290 2024-03-06T07:33:01.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.296+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70b40c8480 con 0x7f70b40a8290 2024-03-06T07:33:01.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.296+0000 7f70aa7fc640 1 -- 172.21.15.17:0/3946438406 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v216) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70b007ac10 con 0x7f70b40a8290 2024-03-06T07:33:01.298 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 msgr2=0x7f709806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 0x7f709806d990 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f70b41ec050 tx=0x7f70a4009040 comp rx=0 tx=0).stop 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 msgr2=0x7f70b40c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f70b002b8c0 tx=0x7f70b00046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 shutdown_connections 2024-03-06T07:33:01.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f709806b4d0 0x7f709806d990 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40c69a0 0x7f70b414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40c4740 0x7f70b40c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 --2- 172.21.15.17:0/3946438406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70b40a8290 0x7f70b40c2100 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 >> 172.21.15.17:0/3946438406 conn(0x7f70b40c5560 msgr2=0x7f70b4072580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 shutdown_connections 2024-03-06T07:33:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.298+0000 7f70bc13a640 1 -- 172.21.15.17:0/3946438406 wait complete. 2024-03-06T07:33:01.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 Processor -- start 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 -- start start 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda700b0450 con 0x7fda700b0070 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7006cfb0 con 0x7fda700c5380 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.414+0000 7fda76f7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7006c020 con 0x7fda70155c30 2024-03-06T07:33:01.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda75517640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda74d16640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fda700b0070 0x7fda700c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:47170/0 (socket says 172.21.15.17:47170) 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda75517640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.17:40150/0 (socket says 172.21.15.17:40150) 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda74d16640 1 -- 172.21.15.17:0/1825189737 learned_addr learned my addr 172.21.15.17:0/1825189737 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.17:60238/0 (socket says 172.21.15.17:60238) 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 -- 172.21.15.17:0/1825189737 >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 msgr2=0x7fda700c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 --2- 172.21.15.17:0/1825189737 >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 -- 172.21.15.17:0/1825189737 >> v1:172.21.15.163:6789/0 conn(0x7fda700b0070 legacy=0x7fda700c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6f7fe640 1 -- 172.21.15.17:0/1825189737 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924728569 0 0) 0x7fda64001110 con 0x7fda700b0070 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda6ffff640 1 -- 172.21.15.17:0/1825189737 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda701560e0 con 0x7fda70155c30 2024-03-06T07:33:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.415+0000 7fda75517640 1 --2- 172.21.15.17:0/1825189737 >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.416+0000 7fda6ffff640 1 --2- 172.21.15.17:0/1825189737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0097a0 tx=0x7fda5c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f37be1dec68ad1f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.416+0000 7fda6f7fe640 1 -- 172.21.15.17:0/1825189737 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0315a0 con 0x7fda70155c30 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.416+0000 7fda6f7fe640 1 -- 172.21.15.17:0/1825189737 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c037070 con 0x7fda70155c30 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.416+0000 7fda6f7fe640 1 -- 172.21.15.17:0/1825189737 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c031960 con 0x7fda70155c30 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 -- 172.21.15.17:0/1825189737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 msgr2=0x7fda7006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1825189737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0097a0 tx=0x7fda5c02b840 comp rx=0 tx=0).stop 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 -- 172.21.15.17:0/1825189737 shutdown_connections 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1825189737 >> v2:172.21.15.163:3300/0 conn(0x7fda700c5380 0x7fda700c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1825189737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda70155c30 0x7fda7006ca70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 -- 172.21.15.17:0/1825189737 >> 172.21.15.17:0/1825189737 conn(0x7fda700ac590 msgr2=0x7fda700ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 -- 172.21.15.17:0/1825189737 shutdown_connections 2024-03-06T07:33:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.417+0000 7fda76f7a640 1 -- 172.21.15.17:0/1825189737 wait complete. 2024-03-06T07:33:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.418+0000 7fda76f7a640 1 Processor -- start 2024-03-06T07:33:01.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.418+0000 7fda76f7a640 1 -- start start 2024-03-06T07:33:01.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 0x7fda70158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda70155c30 0x7fda7014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7006c270 con 0x7fda700c5380 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700c7ad0 con 0x7fda70155c30 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda76f7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7006c550 con 0x7fda700b0070 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 0x7fda70158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda6ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41900/0 (socket says 172.21.15.17:41900) 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda6ffff640 1 -- 172.21.15.17:0/1630629499 learned_addr learned my addr 172.21.15.17:0/1630629499 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda75517640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda70155c30 0x7fda7014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 -- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda70155c30 msgr2=0x7fda7014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda70155c30 0x7fda7014f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 -- 172.21.15.17:0/1630629499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 msgr2=0x7fda7014f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda74d16640 1 -- 172.21.15.17:0/1630629499 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700c47b0 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.419+0000 7fda6ffff640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda74d16640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 0x7fda70158730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c7b0 tx=0x7fda6400cbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64018070 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640070d0 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda70157db0 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640074c0 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.420+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda70158c70 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.421+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7004f330 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.421+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda6401b070 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.422+0000 7fda6d7fa640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 0x7fda5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.422+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fda64089540 con 0x7fda700b0070 2024-03-06T07:33:01.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.422+0000 7fda6ffff640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 0x7fda5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.423+0000 7fda6ffff640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 0x7fda5406d990 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02b880 tx=0x7fda5c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.427+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda700b0450 con 0x7fda700b0070 2024-03-06T07:33:01.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.607+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda7006d860 con 0x7fda700b0070 2024-03-06T07:33:01.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.607+0000 7fda6d7fa640 1 -- 172.21.15.17:0/1630629499 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v216) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda64057b70 con 0x7fda700b0070 2024-03-06T07:33:01.609 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-06T07:33:01.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 msgr2=0x7fda5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 0x7fda5406d990 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fda5c02b880 tx=0x7fda5c03d040 comp rx=0 tx=0).stop 2024-03-06T07:33:01.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 msgr2=0x7fda70158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 0x7fda70158730 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c7b0 tx=0x7fda6400cbb0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 shutdown_connections 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fda5406b4d0 0x7fda5406d990 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda70155c30 0x7fda7014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda700c5380 0x7fda7014f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 --2- 172.21.15.17:0/1630629499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda700b0070 0x7fda70158730 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 >> 172.21.15.17:0/1630629499 conn(0x7fda700ac590 msgr2=0x7fda700c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 shutdown_connections 2024-03-06T07:33:01.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.609+0000 7fda76f7a640 1 -- 172.21.15.17:0/1630629499 wait complete. 2024-03-06T07:33:01.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-06T07:33:01.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.724+0000 7f8f39781640 1 Processor -- start 2024-03-06T07:33:01.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.724+0000 7f8f39781640 1 -- start start 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f39781640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f340c4740 0x7f8f341539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f39781640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340c62a0 con 0x7f8f340c69a0 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f39781640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340c4c00 con 0x7f8f340a8290 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f39781640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f34153f00 con 0x7f8f340c4740 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f2a7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f340c4740 0x7f8f341539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f337fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8f340c69a0 0x7f8f34072680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43828/0 (socket says 172.21.15.17:43828) 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f337fe640 1 -- 172.21.15.17:0/1186704753 learned_addr learned my addr 172.21.15.17:0/1186704753 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f327fc640 1 -- 172.21.15.17:0/1186704753 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367497886 0 0) 0x7f8f340c62a0 con 0x7f8f340c69a0 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f327fc640 1 -- 172.21.15.17:0/1186704753 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f20003520 con 0x7f8f340c69a0 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f2a7fc640 1 -- 172.21.15.17:0/1186704753 >> v1:172.21.15.163:6789/0 conn(0x7f8f340a8290 legacy=0x7f8f340701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f2a7fc640 1 -- 172.21.15.17:0/1186704753 >> v1:172.21.15.17:6789/0 conn(0x7f8f340c69a0 legacy=0x7f8f34072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.725+0000 7f8f2a7fc640 1 -- 172.21.15.17:0/1186704753 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340c4ee0 con 0x7f8f340c4740 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.726+0000 7f8f2a7fc640 1 --2- 172.21.15.17:0/1186704753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c4740 0x7f8f341539c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0097a0 tx=0x7f8f1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb6ef3ae74f197c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.726+0000 7f8f327fc640 1 -- 172.21.15.17:0/1186704753 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0315a0 con 0x7f8f340c4740 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.726+0000 7f8f327fc640 1 -- 172.21.15.17:0/1186704753 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c037070 con 0x7f8f340c4740 2024-03-06T07:33:01.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.726+0000 7f8f327fc640 1 -- 172.21.15.17:0/1186704753 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c031960 con 0x7f8f340c4740 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 -- 172.21.15.17:0/1186704753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c4740 msgr2=0x7f8f341539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 --2- 172.21.15.17:0/1186704753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c4740 0x7f8f341539c0 secure :-1 s=READY pgs=2678 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0097a0 tx=0x7f8f1c02b840 comp rx=0 tx=0).stop 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 -- 172.21.15.17:0/1186704753 shutdown_connections 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 --2- 172.21.15.17:0/1186704753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c4740 0x7f8f341539c0 unknown :-1 s=CLOSED pgs=2678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 -- 172.21.15.17:0/1186704753 >> 172.21.15.17:0/1186704753 conn(0x7f8f340c5560 msgr2=0x7f8f340c5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 -- 172.21.15.17:0/1186704753 shutdown_connections 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.727+0000 7f8f39781640 1 -- 172.21.15.17:0/1186704753 wait complete. 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.729+0000 7f8f39781640 1 Processor -- start 2024-03-06T07:33:01.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.729+0000 7f8f39781640 1 -- start start 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f340a8290 0x7f8f3414a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f340c4740 0x7f8f341e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34072d90 con 0x7f8f340c69a0 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3406fc80 con 0x7f8f340a8290 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f39781640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34072f10 con 0x7f8f340c4740 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41920/0 (socket says 172.21.15.17:41920) 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 -- 172.21.15.17:0/4239404080 learned_addr learned my addr 172.21.15.17:0/4239404080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f2a7fc640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f340c4740 0x7f8f341e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f32ffd640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f340a8290 0x7f8f3414a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 -- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f340c4740 msgr2=0x7f8f341e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f340c4740 0x7f8f341e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 -- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f340a8290 msgr2=0x7f8f3414a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f340a8290 0x7f8f3414a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.730+0000 7f8f337fe640 1 -- 172.21.15.17:0/4239404080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f341ec860 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f337fe640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00caa0 tx=0x7f8f2c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c017070 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c00ac00 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f341ecb40 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c012400 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.731+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f341ed100 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.732+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f34075300 con 0x7f8f340c69a0 2024-03-06T07:33:01.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.732+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f2c01a070 con 0x7f8f340c69a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.733+0000 7f8f2bfff640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.733+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f8f2c08a290 con 0x7f8f340c69a0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.733+0000 7f8f32ffd640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:01.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.733+0000 7f8f32ffd640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 0x7f8f1406d990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f8f24007fd0 tx=0x7f8f24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:01.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.741+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f34075300 con 0x7f8f340c69a0 2024-03-06T07:33:01.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.920+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f3404f330 con 0x7f8f340c69a0 2024-03-06T07:33:01.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.920+0000 7f8f2bfff640 1 -- 172.21.15.17:0/4239404080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v216) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f2c058e80 con 0x7f8f340c69a0 2024-03-06T07:33:01.922 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-06T07:33:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 msgr2=0x7f8f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 0x7f8f1406d990 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f8f24007fd0 tx=0x7f8f24009040 comp rx=0 tx=0).stop 2024-03-06T07:33:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 msgr2=0x7f8f341e9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00caa0 tx=0x7f8f2c00cf60 comp rx=0 tx=0).stop 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 shutdown_connections 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f1406b4d0 0x7f8f1406d990 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f340c69a0 0x7f8f341e9850 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f340c4740 0x7f8f341e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 --2- 172.21.15.17:0/4239404080 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f340a8290 0x7f8f3414a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 >> 172.21.15.17:0/4239404080 conn(0x7f8f340c5560 msgr2=0x7f8f340c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 shutdown_connections 2024-03-06T07:33:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:01.922+0000 7f8f39781640 1 -- 172.21.15.17:0/4239404080 wait complete. 2024-03-06T07:33:01.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:02.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.035+0000 7f2e9624e640 1 Processor -- start 2024-03-06T07:33:02.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.035+0000 7f2e9624e640 1 -- start start 2024-03-06T07:33:02.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e9624e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e90153970 0x7f2e900abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e9624e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e900ac480 con 0x7f2e900a9450 2024-03-06T07:33:02.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e9624e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e900c33b0 con 0x7f2e901503c0 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e9624e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e90153e70 con 0x7f2e90153970 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e8ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2e900a9450 0x7f2e900a98c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43838/0 (socket says 172.21.15.17:43838) 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e8ffff640 1 -- 172.21.15.17:0/1565212141 learned_addr learned my addr 172.21.15.17:0/1565212141 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e87fff640 1 --2- 172.21.15.17:0/1565212141 >> v2:172.21.15.163:3300/0 conn(0x7f2e90153970 0x7f2e900abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768350631 0 0) 0x7f2e88001490 con 0x7f2e900a9450 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.036+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e7c003520 con 0x7f2e900a9450 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888295274 0 0) 0x7f2e7c003520 con 0x7f2e900a9450 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 >> v2:172.21.15.163:3300/0 conn(0x7f2e90153970 msgr2=0x7f2e900abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 --2- 172.21.15.17:0/1565212141 >> v2:172.21.15.163:3300/0 conn(0x7f2e90153970 0x7f2e900abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 >> v1:172.21.15.163:6789/0 conn(0x7f2e901503c0 legacy=0x7f2e900ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900c3690 con 0x7f2e900a9450 2024-03-06T07:33:02.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f2e88001bc0 con 0x7f2e900a9450 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 >> v1:172.21.15.17:6789/0 conn(0x7f2e900a9450 legacy=0x7f2e900a98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c0039b0 0x7f2e7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e7c0043b0 0x7f2e7c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 0x7f2e7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90153e70 con 0x7f2e7c005120 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c00c060 con 0x7f2e7c0039b0 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7c00c260 con 0x7f2e7c0043b0 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 0x7f2e7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8f7fe640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c0039b0 0x7f2e7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e87fff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e7c0043b0 0x7f2e7c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 -- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e7c0043b0 msgr2=0x7f2e7c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e7c0043b0 0x7f2e7c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 -- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c0039b0 msgr2=0x7f2e7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c0039b0 0x7f2e7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e8ffff640 1 -- 172.21.15.17:0/1565212141 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c003520 con 0x7f2e7c005120 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.037+0000 7f2e87fff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e7c0043b0 0x7f2e7c004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.038+0000 7f2e8ffff640 1 -- 172.21.15.17:0/1565212141 reap_dead start 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.038+0000 7f2e8ffff640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 0x7f2e7c00b9c0 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800a9d0 tx=0x7f2e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65a8721898be3d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.038+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800a5f0 con 0x7f2e7c005120 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.038+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e880072b0 con 0x7f2e7c005120 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.038+0000 7f2e8effd640 1 -- 172.21.15.17:0/1565212141 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880076a0 con 0x7f2e7c005120 2024-03-06T07:33:02.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 -- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 msgr2=0x7f2e7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 0x7f2e7c00b9c0 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800a9d0 tx=0x7f2e7c005830 comp rx=0 tx=0).stop 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 -- 172.21.15.17:0/1565212141 shutdown_connections 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 --2- 172.21.15.17:0/1565212141 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e7c005120 0x7f2e7c00b9c0 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 -- 172.21.15.17:0/1565212141 >> 172.21.15.17:0/1565212141 conn(0x7f2e9006bf00 msgr2=0x7f2e900c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 -- 172.21.15.17:0/1565212141 shutdown_connections 2024-03-06T07:33:02.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.039+0000 7f2e9624e640 1 -- 172.21.15.17:0/1565212141 wait complete. 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.040+0000 7f2e9624e640 1 Processor -- start 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.040+0000 7f2e9624e640 1 -- start start 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.040+0000 7f2e9624e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c005120 0x7f2e901ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.040+0000 7f2e9624e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e90144d90 0x7f2e901eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e90150920 con 0x7f2e90145530 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901507a0 con 0x7f2e7c005120 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9006eda0 con 0x7f2e90144d90 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41934/0 (socket says 172.21.15.17:41934) 2024-03-06T07:33:02.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 -- 172.21.15.17:0/747920150 learned_addr learned my addr 172.21.15.17:0/747920150 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e8f7fe640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c005120 0x7f2e901ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e8ffff640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e90144d90 0x7f2e901eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 -- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e90144d90 msgr2=0x7f2e901eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e90144d90 0x7f2e901eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 -- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c005120 msgr2=0x7f2e901ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c005120 0x7f2e901ec4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 -- 172.21.15.17:0/747920150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e90156430 con 0x7f2e90145530 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e87fff640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f2e78011540 tx=0x7f2e78011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78015870 con 0x7f2e90145530 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78015a10 con 0x7f2e90145530 2024-03-06T07:33:02.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78015dc0 con 0x7f2e90145530 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901ecce0 con 0x7f2e90145530 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.041+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901ea5b0 con 0x7f2e90145530 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.042+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9004f330 con 0x7f2e90145530 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.043+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e7800f460 con 0x7f2e90145530 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.043+0000 7f2e8cff9640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 0x7f2e7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.043+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f2e7808acd0 con 0x7f2e90145530 2024-03-06T07:33:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.043+0000 7f2e8f7fe640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 0x7f2e7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.044+0000 7f2e8f7fe640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 0x7f2e7006d9c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2e80001890 tx=0x7f2e80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.048+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2e90156430 con 0x7f2e90145530 2024-03-06T07:33:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.267+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e90068460 con 0x7f2e90145530 2024-03-06T07:33:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.267+0000 7f2e8cff9640 1 -- 172.21.15.17:0/747920150 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f2e78019030 con 0x7f2e90145530 2024-03-06T07:33:02.269 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:02.269 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:33:02.269 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 msgr2=0x7f2e7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 0x7f2e7006d9c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2e80001890 tx=0x7f2e80009040 comp rx=0 tx=0).stop 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 msgr2=0x7f2e90144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f2e78011540 tx=0x7f2e78011a00 comp rx=0 tx=0).stop 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 shutdown_connections 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f2e7006b500 0x7f2e7006d9c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e90144d90 0x7f2e901eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e90145530 0x7f2e90144850 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 --2- 172.21.15.17:0/747920150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e7c005120 0x7f2e901ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 >> 172.21.15.17:0/747920150 conn(0x7f2e9006bf00 msgr2=0x7f2e900c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 shutdown_connections 2024-03-06T07:33:02.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.269+0000 7f2e9624e640 1 -- 172.21.15.17:0/747920150 wait complete. 2024-03-06T07:33:02.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 Processor -- start 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 -- start start 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd8301527b0 0x7fd830159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd8300c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300c7c60 con 0x7fd830155b20 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd830159640 con 0x7fd8301527b0 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd835a2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300c8df0 con 0x7fd8300c7180 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd82f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd830155b20 0x7fd8300c5330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43840/0 (socket says 172.21.15.17:43840) 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.382+0000 7fd82f7fe640 1 -- 172.21.15.17:0/2759681225 learned_addr learned my addr 172.21.15.17:0/2759681225 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:02.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd8300c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82effd640 1 --2- 172.21.15.17:0/2759681225 >> v2:172.21.15.163:3300/0 conn(0x7fd8301527b0 0x7fd830159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878927087 0 0) 0x7fd820001490 con 0x7fd830155b20 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003520 con 0x7fd830155b20 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1161891177 0 0) 0x7fd81c003520 con 0x7fd830155b20 2024-03-06T07:33:02.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 msgr2=0x7fd8300c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd8300c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 >> v2:172.21.15.163:3300/0 conn(0x7fd8301527b0 msgr2=0x7fd830159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 --2- 172.21.15.17:0/2759681225 >> v2:172.21.15.163:3300/0 conn(0x7fd8301527b0 0x7fd830159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300c8060 con 0x7fd830155b20 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd820001bc0 con 0x7fd830155b20 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd8300c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:02.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 >> v1:172.21.15.17:6789/0 conn(0x7fd830155b20 legacy=0x7fd8300c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd81c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830159640 con 0x7fd81c004800 2024-03-06T07:33:02.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300c8df0 con 0x7fd81c0039b0 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c00b9c0 con 0x7fd8300c7180 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82effd640 1 --2- 172.21.15.17:0/2759681225 >> v2:172.21.15.163:3300/0 conn(0x7fd8301527b0 0x7fd830159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82f7fe640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd81c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.383+0000 7fd82effd640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 -- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 msgr2=0x7fd81c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8300c7180 0x7fd81c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 -- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c0039b0 msgr2=0x7fd81c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 -- 172.21.15.17:0/2759681225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c003520 con 0x7fd81c004800 2024-03-06T07:33:02.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82effd640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd81c0039b0 0x7fd81c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fd818009c90 tx=0x7fd81c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735fd58ee3fda6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818039070 con 0x7fd81c004800 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81802c8f0 con 0x7fd81c004800 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.384+0000 7fd82dffb640 1 -- 172.21.15.17:0/2759681225 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818009860 con 0x7fd81c004800 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 -- 172.21.15.17:0/2759681225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c004800 msgr2=0x7fd81c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 --2- 172.21.15.17:0/2759681225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd81c004800 0x7fd81c004c80 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7fd818009c90 tx=0x7fd81c005830 comp rx=0 tx=0).stop 2024-03-06T07:33:02.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd82f7fe640 1 -- 172.21.15.17:0/2759681225 reap_dead start 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 -- 172.21.15.17:0/2759681225 shutdown_connections 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 -- 172.21.15.17:0/2759681225 >> 172.21.15.17:0/2759681225 conn(0x7fd8300ac560 msgr2=0x7fd8300ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 -- 172.21.15.17:0/2759681225 shutdown_connections 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.385+0000 7fd835a2a640 1 -- 172.21.15.17:0/2759681225 wait complete. 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.386+0000 7fd835a2a640 1 Processor -- start 2024-03-06T07:33:02.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.386+0000 7fd835a2a640 1 -- start start 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd81c004800 0x7fd8300c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8300c4550 0x7fd8301eb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830155320 con 0x7fd8301542b0 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300c7980 con 0x7fd8300c4550 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd835a2a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd830155600 con 0x7fd81c004800 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41960/0 (socket says 172.21.15.17:41960) 2024-03-06T07:33:02.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 -- 172.21.15.17:0/1418409387 learned_addr learned my addr 172.21.15.17:0/1418409387 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82e7fc640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8300c4550 0x7fd8301eb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82effd640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd81c004800 0x7fd8300c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 -- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd81c004800 msgr2=0x7fd8300c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd81c004800 0x7fd8300c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 -- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8300c4550 msgr2=0x7fd8301eb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8300c4550 0x7fd8301eb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.387+0000 7fd82f7fe640 1 -- 172.21.15.17:0/1418409387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300ca210 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd82f7fe640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fd8300c4cb0 tx=0x7fd82000aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820017020 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8200078c0 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200133f0 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8300ca760 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.388+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300ca480 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.389+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83004f330 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.389+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd82001a020 con 0x7fd8301542b0 2024-03-06T07:33:02.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.390+0000 7fd827fff640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 0x7fd81006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.390+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7fd82008acd0 con 0x7fd8301542b0 2024-03-06T07:33:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.390+0000 7fd82effd640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 0x7fd81006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.391+0000 7fd82effd640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 0x7fd81006d990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd830155290 tx=0x7fd828008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.394+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd820021020 con 0x7fd8301542b0 2024-03-06T07:33:02.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.580+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8300ac970 con 0x7fd8301542b0 2024-03-06T07:33:02.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.581+0000 7fd827fff640 1 -- 172.21.15.17:0/1418409387 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+17291 (secure 0 0 0) 0x7fd8300c7c60 con 0x7fd8301542b0 2024-03-06T07:33:02.583 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:02.583 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":216,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:30:31.883062+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.13","mappings":[{"from":0,"to":2}]},{"pgid":"34.19","mappings":[{"from":0,"to":1}]},{"pgid":"34.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6834/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6837/370957199":"2024-03-07T07:29:56.878364+0000","172.21.15.163:6836/370957199":"2024-03-07T07:29:56.878364+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 msgr2=0x7fd81006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 0x7fd81006d990 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd830155290 tx=0x7fd828008040 comp rx=0 tx=0).stop 2024-03-06T07:33:02.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 msgr2=0x7fd8300c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7fd8300c4cb0 tx=0x7fd82000aec0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 shutdown_connections 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd81006b4d0 0x7fd81006d990 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8301542b0 0x7fd8300c9880 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd8300c4550 0x7fd8301eb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 --2- 172.21.15.17:0/1418409387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd81c004800 0x7fd8300c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.583+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 >> 172.21.15.17:0/1418409387 conn(0x7fd8300ac560 msgr2=0x7fd83006b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.584+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 shutdown_connections 2024-03-06T07:33:02.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.584+0000 7fd835a2a640 1 -- 172.21.15.17:0/1418409387 wait complete. 2024-03-06T07:33:02.596 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:33:02.597 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.696+0000 7f687558a640 1 Processor -- start 2024-03-06T07:33:02.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.696+0000 7f687558a640 1 -- start start 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.697+0000 7f687558a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f687014f360 0x7f6870155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.697+0000 7f687558a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f6870158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.697+0000 7f687558a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700c73d0 con 0x7f68700c88c0 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.697+0000 7f687558a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68700c8dd0 con 0x7f687014f360 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.697+0000 7f687558a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700be2e0 con 0x7f6870158b60 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686e7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68700c88c0 0x7f68700c5340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43842/0 (socket says 172.21.15.17:43842) 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686e7fc640 1 -- 172.21.15.17:0/1664254012 learned_addr learned my addr 172.21.15.17:0/1664254012 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:02.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686f7fe640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f6870158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734274526 0 0) 0x7f68700c73d0 con 0x7f68700c88c0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686effd640 1 --2- 172.21.15.17:0/1664254012 >> v2:172.21.15.163:3300/0 conn(0x7f687014f360 0x7f6870155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c003520 con 0x7f68700c88c0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013951031 0 0) 0x7f685c003520 con 0x7f68700c88c0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 msgr2=0x7f6870158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f6870158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 >> v2:172.21.15.163:3300/0 conn(0x7f687014f360 msgr2=0x7f6870155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 --2- 172.21.15.17:0/1664254012 >> v2:172.21.15.163:3300/0 conn(0x7f687014f360 0x7f6870155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700c7670 con 0x7f68700c88c0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f6858001670 con 0x7f68700c88c0 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 >> v1:172.21.15.17:6789/0 conn(0x7f68700c88c0 legacy=0x7f68700c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c0039b0 0x7f685c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f685c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700be2e0 con 0x7f685c004d40 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c00b9c0 con 0x7f685c0039b0 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c00bb70 con 0x7f6870158b60 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686f7fe640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686effd640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c0039b0 0x7f685c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.698+0000 7f686e7fc640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f685c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 -- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 msgr2=0x7f685c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f685c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 -- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c0039b0 msgr2=0x7f685c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c0039b0 0x7f685c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 -- 172.21.15.17:0/1664254012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c003520 con 0x7f685c004d40 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686e7fc640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6870158b60 0x7f685c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686f7fe640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f6868009fc0 tx=0x7f685c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ed166234ff53aa server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.699+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868039070 con 0x7f685c004d40 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686802cc20 con 0x7f685c004d40 2024-03-06T07:33:02.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f686dffb640 1 -- 172.21.15.17:0/1664254012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868009c10 con 0x7f685c004d40 2024-03-06T07:33:02.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 -- 172.21.15.17:0/1664254012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c004d40 msgr2=0x7f685c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 --2- 172.21.15.17:0/1664254012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685c004d40 0x7f685c0051c0 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f6868009fc0 tx=0x7f685c005830 comp rx=0 tx=0).stop 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f686f7fe640 1 -- 172.21.15.17:0/1664254012 reap_dead start 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 -- 172.21.15.17:0/1664254012 shutdown_connections 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 -- 172.21.15.17:0/1664254012 >> 172.21.15.17:0/1664254012 conn(0x7f68700ac570 msgr2=0x7f68700ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 -- 172.21.15.17:0/1664254012 shutdown_connections 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.700+0000 7f687558a640 1 -- 172.21.15.17:0/1664254012 wait complete. 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.701+0000 7f687558a640 1 Processor -- start 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.701+0000 7f687558a640 1 -- start start 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 0x7f68700afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68701ee930 0x7f68700c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c5a50 con 0x7f68701ee4b0 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c7c30 con 0x7f685c004d40 2024-03-06T07:33:02.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f687558a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c5bd0 con 0x7f68701ee930 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41980/0 (socket says 172.21.15.17:41980) 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 -- 172.21.15.17:0/3618124970 learned_addr learned my addr 172.21.15.17:0/3618124970 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686effd640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 0x7f68700afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686f7fe640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68701ee930 0x7f68700c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 -- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68701ee930 msgr2=0x7f68700c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68701ee930 0x7f68700c2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 -- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 msgr2=0x7f68700afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 0x7f68700afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686e7fc640 1 -- 172.21.15.17:0/3618124970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700c3510 con 0x7f68701ee4b0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.702+0000 7f686effd640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 0x7f68700afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f686e7fc640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f685800e910 tx=0x7f685800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858019070 con 0x7f68701ee4b0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6858004ab0 con 0x7f68701ee4b0 2024-03-06T07:33:02.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68701eeea0 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858005050 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.703+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68700c3bc0 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.704+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68700c3dc0 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.704+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68580051f0 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.705+0000 7f68677fe640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 0x7f685006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.705+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f6858089cc0 con 0x7f68701ee4b0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.705+0000 7f686effd640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 0x7f685006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:02.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.706+0000 7f686effd640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 0x7f685006d9c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f687014eca0 tx=0x7f6860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.709+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68700c3dc0 con 0x7f68701ee4b0 2024-03-06T07:33:02.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.927+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f687004f330 con 0x7f68701ee4b0 2024-03-06T07:33:02.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.928+0000 7f68677fe640 1 -- 172.21.15.17:0/3618124970 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f68580582f0 con 0x7f68701ee4b0 2024-03-06T07:33:02.930 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:02.930 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:33:02.930 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:33:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 msgr2=0x7f685006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 0x7f685006d9c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f687014eca0 tx=0x7f6860008040 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 msgr2=0x7f68700c2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7f685800e910 tx=0x7f685800edd0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 shutdown_connections 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f685006b500 0x7f685006d9c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68701ee930 0x7f68700c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68701ee4b0 0x7f68700c2a90 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 --2- 172.21.15.17:0/3618124970 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f685c004d40 0x7f68700afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.929+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 >> 172.21.15.17:0/3618124970 conn(0x7f68700ac570 msgr2=0x7f687014de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.930+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 shutdown_connections 2024-03-06T07:33:02.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:02.930+0000 7f687558a640 1 -- 172.21.15.17:0/3618124970 wait complete. 2024-03-06T07:33:02.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:03.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.041+0000 7f51f8ac2640 1 Processor -- start 2024-03-06T07:33:03.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 -- start start 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 0x7f51f40c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51f40c17c0 0x7f51f415fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f406afc0 con 0x7f51f4152920 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f406b2a0 con 0x7f51f40c17c0 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.042+0000 7f51f8ac2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f406b580 con 0x7f51f4155c20 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f2d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f51f40c17c0 0x7f51f415fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f259c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f51f4152920 0x7f51f40c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43846/0 (socket says 172.21.15.17:43846) 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f259c640 1 -- 172.21.15.17:0/44848047 learned_addr learned my addr 172.21.15.17:0/44848047 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f1d9b640 1 --2- 172.21.15.17:0/44848047 >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 0x7f51f40c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f2d9d640 1 -- 172.21.15.17:0/44848047 >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 msgr2=0x7f51f40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f2d9d640 1 --2- 172.21.15.17:0/44848047 >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 0x7f51f40c1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f2d9d640 1 -- 172.21.15.17:0/44848047 >> v1:172.21.15.17:6789/0 conn(0x7f51f4152920 legacy=0x7f51f40c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f2d9d640 1 -- 172.21.15.17:0/44848047 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f406b860 con 0x7f51f40c17c0 2024-03-06T07:33:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f159a640 1 -- 172.21.15.17:0/44848047 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183209188 0 0) 0x7f51f406afc0 con 0x7f51f4152920 2024-03-06T07:33:03.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.043+0000 7f51f1d9b640 1 --2- 172.21.15.17:0/44848047 >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 0x7f51f40c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.044+0000 7f51f2d9d640 1 --2- 172.21.15.17:0/44848047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f40c17c0 0x7f51f415fb20 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f51f4159830 tx=0x7f51ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63adff9f5a1bef73 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.044+0000 7f51f159a640 1 -- 172.21.15.17:0/44848047 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec039070 con 0x7f51f40c17c0 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.044+0000 7f51f159a640 1 -- 172.21.15.17:0/44848047 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec02cc80 con 0x7f51f40c17c0 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.044+0000 7f51f159a640 1 -- 172.21.15.17:0/44848047 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec034660 con 0x7f51f40c17c0 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 -- 172.21.15.17:0/44848047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f40c17c0 msgr2=0x7f51f415fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/44848047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f40c17c0 0x7f51f415fb20 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f51f4159830 tx=0x7f51ec02bed0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 -- 172.21.15.17:0/44848047 shutdown_connections 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/44848047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f40c17c0 0x7f51f415fb20 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/44848047 >> v2:172.21.15.163:3300/0 conn(0x7f51f4155c20 0x7f51f40c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 -- 172.21.15.17:0/44848047 >> 172.21.15.17:0/44848047 conn(0x7f51f40c4290 msgr2=0x7f51f40c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 -- 172.21.15.17:0/44848047 shutdown_connections 2024-03-06T07:33:03.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.045+0000 7f51f8ac2640 1 -- 172.21.15.17:0/44848047 wait complete. 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.046+0000 7f51f8ac2640 1 Processor -- start 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.046+0000 7f51f8ac2640 1 -- start start 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f40c17c0 0x7f51f414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f4155c20 0x7f51f4145a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c28a0 con 0x7f51f4152920 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41560b0 con 0x7f51f40c17c0 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f8ac2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f40c2a20 con 0x7f51f4155c20 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41994/0 (socket says 172.21.15.17:41994) 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 -- 172.21.15.17:0/3373168295 learned_addr learned my addr 172.21.15.17:0/3373168295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f259c640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f40c17c0 0x7f51f414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f2d9d640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f4155c20 0x7f51f4145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 -- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f4155c20 msgr2=0x7f51f4145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f4155c20 0x7f51f4145a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 -- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f40c17c0 msgr2=0x7f51f414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f40c17c0 0x7f51f414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 -- 172.21.15.17:0/3373168295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f40ab2e0 con 0x7f51f4152920 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.047+0000 7f51f1d9b640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f51e800d7d0 tx=0x7f51e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.048+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8004030 con 0x7f51f4152920 2024-03-06T07:33:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.048+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e80041d0 con 0x7f51f4152920 2024-03-06T07:33:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.048+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e800b660 con 0x7f51f4152920 2024-03-06T07:33:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.048+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f40aad20 con 0x7f51f4152920 2024-03-06T07:33:03.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.048+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41eb0d0 con 0x7f51f4152920 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.049+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f41eaec0 con 0x7f51f4152920 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.049+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51e800b800 con 0x7f51f4152920 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.050+0000 7f51e2ffd640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 0x7f51d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.050+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f51e808a4d0 con 0x7f51f4152920 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.050+0000 7f51f259c640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 0x7f51d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.050+0000 7f51f259c640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 0x7f51d406d9c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f51e4000f60 tx=0x7f51e4007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.054+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f51f40ab2e0 con 0x7f51f4152920 2024-03-06T07:33:03.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.272+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51f404f330 con 0x7f51f4152920 2024-03-06T07:33:03.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.273+0000 7f51e2ffd640 1 -- 172.21.15.17:0/3373168295 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+3476 (secure 0 0 0) 0x7f51e8058a60 con 0x7f51f4152920 2024-03-06T07:33:03.275 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:03.275 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":218,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":218,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-06T07:29:14.378013+0000","modified":"2024-03-06T07:31:06.887130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":216,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":0,"in":[0,1],"up":{"mds_0":10791,"mds_1":8162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8162":{"gid":8162,"name":"a","rank":1,"incarnation":212,"state":"up:stopping","state_seq":26,"addr":"172.21.15.17:6835/1774857824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6833","nonce":1774857824},{"type":"v1","addr":"172.21.15.17:6835","nonce":1774857824}]},"join_fscid":-1,"laggy_since":"2024-03-06T07:31:06.887130+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10791":{"gid":10791,"name":"c","rank":0,"incarnation":198,"state":"up:active","state_seq":4,"addr":"172.21.15.17:6834/2434034079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2434034079},{"type":"v1","addr":"172.21.15.17:6834","nonce":2434034079}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10791,"qdb_cluster":[10791]},"id":16}]} 2024-03-06T07:33:03.275 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 218 2024-03-06T07:33:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 msgr2=0x7f51d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 0x7f51d406d9c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f51e4000f60 tx=0x7f51e4007d50 comp rx=0 tx=0).stop 2024-03-06T07:33:03.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 msgr2=0x7f51f41eb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f51e800d7d0 tx=0x7f51e800dc90 comp rx=0 tx=0).stop 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 shutdown_connections 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f51d406b500 0x7f51d406d9c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51f4155c20 0x7f51f4145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51f4152920 0x7f51f41eb5f0 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 --2- 172.21.15.17:0/3373168295 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51f40c17c0 0x7f51f414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 >> 172.21.15.17:0/3373168295 conn(0x7f51f40c4290 msgr2=0x7f51f4151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 shutdown_connections 2024-03-06T07:33:03.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.275+0000 7f51f8ac2640 1 -- 172.21.15.17:0/3373168295 wait complete. 2024-03-06T07:33:03.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:03.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 Processor -- start 2024-03-06T07:33:03.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 -- start start 2024-03-06T07:33:03.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f754c155c20 0x7f754c0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c06afc0 con 0x7f754c152920 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c06b2a0 con 0x7f754c0c17c0 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f7552acf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c06b580 con 0x7f754c155c20 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.388+0000 7f754bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f754c155c20 0x7f754c0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f755086b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f754c152920 0x7f754c0c2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43858/0 (socket says 172.21.15.17:43858) 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f754bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f754c155c20 0x7f754c0c1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42006/0 (socket says 172.21.15.17:42006) 2024-03-06T07:33:03.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f755086b640 1 -- 172.21.15.17:0/1091200093 learned_addr learned my addr 172.21.15.17:0/1091200093 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f754bfff640 1 -- 172.21.15.17:0/1091200093 >> v1:172.21.15.163:6789/0 conn(0x7f754c0c17c0 legacy=0x7f754c15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f754bfff640 1 -- 172.21.15.17:0/1091200093 >> v1:172.21.15.17:6789/0 conn(0x7f754c152920 legacy=0x7f754c0c2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f754bfff640 1 -- 172.21.15.17:0/1091200093 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c06b860 con 0x7f754c155c20 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.389+0000 7f754b7fe640 1 -- 172.21.15.17:0/1091200093 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198418229 0 0) 0x7f754c06afc0 con 0x7f754c152920 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f754bfff640 1 --2- 172.21.15.17:0/1091200093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c155c20 0x7f754c0c1280 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f75380097a0 tx=0x7f753802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ea904d0ba5c252 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f754b7fe640 1 -- 172.21.15.17:0/1091200093 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538035020 con 0x7f754c155c20 2024-03-06T07:33:03.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f754b7fe640 1 -- 172.21.15.17:0/1091200093 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7538031830 con 0x7f754c155c20 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f754b7fe640 1 -- 172.21.15.17:0/1091200093 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538031c00 con 0x7f754c155c20 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 -- 172.21.15.17:0/1091200093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c155c20 msgr2=0x7f754c0c1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 --2- 172.21.15.17:0/1091200093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c155c20 0x7f754c0c1280 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f75380097a0 tx=0x7f753802b8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 -- 172.21.15.17:0/1091200093 shutdown_connections 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 --2- 172.21.15.17:0/1091200093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c155c20 0x7f754c0c1280 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 -- 172.21.15.17:0/1091200093 >> 172.21.15.17:0/1091200093 conn(0x7f754c0c4290 msgr2=0x7f754c0c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 -- 172.21.15.17:0/1091200093 shutdown_connections 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.390+0000 7f7552acf640 1 -- 172.21.15.17:0/1091200093 wait complete. 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.391+0000 7f7552acf640 1 Processor -- start 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.391+0000 7f7552acf640 1 -- start start 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 0x7f754c1ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754c155c20 0x7f754c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c28a0 con 0x7f754c0c17c0 2024-03-06T07:33:03.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c1560b0 con 0x7f754c152920 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f7552acf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0c2a20 con 0x7f754c155c20 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f755086b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f755086b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42020/0 (socket says 172.21.15.17:42020) 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f755086b640 1 -- 172.21.15.17:0/2876134684 learned_addr learned my addr 172.21.15.17:0/2876134684 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f754bfff640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 0x7f754c1ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.392+0000 7f755106c640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754c155c20 0x7f754c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 -- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754c155c20 msgr2=0x7f754c156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754c155c20 0x7f754c156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 -- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 msgr2=0x7f754c1ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 0x7f754c1ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 -- 172.21.15.17:0/2876134684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c153ac0 con 0x7f754c0c17c0 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f754bfff640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 0x7f754c1ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f755086b640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f754000c4d0 tx=0x7f754000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540018070 con 0x7f754c0c17c0 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540007500 con 0x7f754c0c17c0 2024-03-06T07:33:03.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c151e70 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540004a20 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.393+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c153d60 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.394+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c1524d0 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.395+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f754001b070 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.395+0000 7f75497fa640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 0x7f753006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.395+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f7540089790 con 0x7f754c0c17c0 2024-03-06T07:33:03.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.395+0000 7f754bfff640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 0x7f753006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.396+0000 7f754bfff640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 0x7f753006d990 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f7538036000 tx=0x7f753803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.399+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f754c1524d0 con 0x7f754c0c17c0 2024-03-06T07:33:03.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.581+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f754c0c7550 con 0x7f754c0c17c0 2024-03-06T07:33:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.582+0000 7f75497fa640 1 -- 172.21.15.17:0/2876134684 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f754c0c7550 con 0x7f754c0c17c0 2024-03-06T07:33:03.585 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:03.585 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":216,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:30:31.883062+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2024-03-06T07:29:12.634238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"cephfs_data","create_time":"2024-03-06T07:29:13.343125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.13","mappings":[{"from":0,"to":2}]},{"pgid":"34.19","mappings":[{"from":0,"to":1}]},{"pgid":"34.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6835/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6834/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6837/370957199":"2024-03-07T07:29:56.878364+0000","172.21.15.163:6836/370957199":"2024-03-07T07:29:56.878364+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 msgr2=0x7f753006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 0x7f753006d990 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f7538036000 tx=0x7f753803c040 comp rx=0 tx=0).stop 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 msgr2=0x7f754c154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f754000c4d0 tx=0x7f754000c990 comp rx=0 tx=0).stop 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 shutdown_connections 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f753006b4d0 0x7f753006d990 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f754c155c20 0x7f754c156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f754c152920 0x7f754c1ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 --2- 172.21.15.17:0/2876134684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f754c0c17c0 0x7f754c154560 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 >> 172.21.15.17:0/2876134684 conn(0x7f754c0c4290 msgr2=0x7f754c0c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 shutdown_connections 2024-03-06T07:33:03.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.585+0000 7f7552acf640 1 -- 172.21.15.17:0/2876134684 wait complete. 2024-03-06T07:33:03.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-06T07:33:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.698+0000 7f109de9a640 1 Processor -- start 2024-03-06T07:33:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.698+0000 7f109de9a640 1 -- start start 2024-03-06T07:33:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f109de9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f1098153c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f109de9a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10980c8f20 0x7f1098154160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f109de9a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980a8390 con 0x7f10980c7170 2024-03-06T07:33:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f109de9a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980c7620 con 0x7f10980c8f20 2024-03-06T07:33:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f109de9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980730f0 con 0x7f109806ff10 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f1097fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10980c8f20 0x7f1098154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f1096ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10980c7170 0x7f10980c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43870/0 (socket says 172.21.15.17:43870) 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.699+0000 7f1096ffd640 1 -- 172.21.15.17:0/447101038 learned_addr learned my addr 172.21.15.17:0/447101038 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f10977fe640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f1098153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f1097fff640 1 -- 172.21.15.17:0/447101038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 msgr2=0x7f1098153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f1097fff640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f1098153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f1097fff640 1 -- 172.21.15.17:0/447101038 >> v1:172.21.15.17:6789/0 conn(0x7f10980c7170 legacy=0x7f10980c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f10967fc640 1 -- 172.21.15.17:0/447101038 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915222254 0 0) 0x7f10980a8390 con 0x7f10980c7170 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f1097fff640 1 -- 172.21.15.17:0/447101038 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980733d0 con 0x7f10980c8f20 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.700+0000 7f1097fff640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f1098154160 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f109002bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3e09fc350c2998 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f10967fc640 1 -- 172.21.15.17:0/447101038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090039070 con 0x7f10980c8f20 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f10967fc640 1 -- 172.21.15.17:0/447101038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109002cb30 con 0x7f10980c8f20 2024-03-06T07:33:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f10967fc640 1 -- 172.21.15.17:0/447101038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090034440 con 0x7f10980c8f20 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 -- 172.21.15.17:0/447101038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 msgr2=0x7f1098154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f1098154160 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7f1090009d90 tx=0x7f109002bd80 comp rx=0 tx=0).stop 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 -- 172.21.15.17:0/447101038 shutdown_connections 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f1098154160 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 --2- 172.21.15.17:0/447101038 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f1098153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 -- 172.21.15.17:0/447101038 >> 172.21.15.17:0/447101038 conn(0x7f109806b970 msgr2=0x7f10980aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 -- 172.21.15.17:0/447101038 shutdown_connections 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.701+0000 7f109de9a640 1 -- 172.21.15.17:0/447101038 wait complete. 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.703+0000 7f109de9a640 1 Processor -- start 2024-03-06T07:33:03.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.703+0000 7f109de9a640 1 -- start start 2024-03-06T07:33:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f10981546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10980c7170 0x7f10980a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098070470 con 0x7f10980c8f20 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980702f0 con 0x7f10980c7170 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f109de9a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980c6860 con 0x7f109806ff10 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f1097fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f1097fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42042/0 (socket says 172.21.15.17:42042) 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f1097fff640 1 -- 172.21.15.17:0/3350214623 learned_addr learned my addr 172.21.15.17:0/3350214623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f1097fff640 1 -- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 msgr2=0x7f10981546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.704+0000 7f1097fff640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f10981546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f1097fff640 1 -- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10980c7170 msgr2=0x7f10980a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f1097fff640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10980c7170 0x7f10980a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f1097fff640 1 -- 172.21.15.17:0/3350214623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1098156770 con 0x7f10980c8f20 2024-03-06T07:33:03.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f1097fff640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f1090009eb0 tx=0x7f1090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090039050 con 0x7f10980c8f20 2024-03-06T07:33:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1090045070 con 0x7f10980c8f20 2024-03-06T07:33:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1098156490 con 0x7f10980c8f20 2024-03-06T07:33:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109003e850 con 0x7f10980c8f20 2024-03-06T07:33:03.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.705+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1098157560 con 0x7f10980c8f20 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.706+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109804f330 con 0x7f10980c8f20 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.706+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f109003e9f0 con 0x7f10980c8f20 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.707+0000 7f108ffff640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 0x7f107806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.707+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6360+0+0 (secure 0 0 0) 0x7f10900ad410 con 0x7f10980c8f20 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.707+0000 7f10977fe640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 0x7f107806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:03.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.708+0000 7f10977fe640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 0x7f107806d990 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f10880098f0 tx=0x7f1088008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.715+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f109007c120 con 0x7f10980c8f20 2024-03-06T07:33:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.932+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f109806fa80 con 0x7f10980c8f20 2024-03-06T07:33:03.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.934+0000 7f108ffff640 1 -- 172.21.15.17:0/3350214623 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v219) v1 ==== 161+0+0 (secure 0 0 0) 0x7f109007ba20 con 0x7f10980c8f20 2024-03-06T07:33:03.936 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-06T07:33:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 msgr2=0x7f107806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 0x7f107806d990 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f10880098f0 tx=0x7f1088008040 comp rx=0 tx=0).stop 2024-03-06T07:33:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 msgr2=0x7f109814c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7f1090009eb0 tx=0x7f1090004a40 comp rx=0 tx=0).stop 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 shutdown_connections 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f107806b4d0 0x7f107806d990 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10980c8f20 0x7f109814c980 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10980c7170 0x7f10980a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 --2- 172.21.15.17:0/3350214623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f109806ff10 0x7f10981546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 >> 172.21.15.17:0/3350214623 conn(0x7f109806b970 msgr2=0x7f10980c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 shutdown_connections 2024-03-06T07:33:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:03.937+0000 7f109de9a640 1 -- 172.21.15.17:0/3350214623 wait complete. 2024-03-06T07:33:03.942 INFO:tasks.ceph.mds.c.smithi017.stderr: -8586> 2024-03-06T07:29:19.296+0000 7f56818ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-06T07:33:03.942 INFO:tasks.ceph.mds.c.smithi017.stderr: -8236> 2024-03-06T07:29:24.297+0000 7f56818ff640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-06T07:33:03.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-06T07:33:03.974 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-03-06T07:33:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.054+0000 7f119b839640 1 Processor -- start 2024-03-06T07:33:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 -- start start 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1194155c30 0x7f119406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f11940c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940b0450 con 0x7f11940b0070 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119406cfb0 con 0x7f1194155c30 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f119b839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119406c020 con 0x7f11940c5380 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f11995d5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f11940b0070 0x7f11940c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43882/0 (socket says 172.21.15.17:43882) 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.055+0000 7f11995d5640 1 -- 172.21.15.17:0/4233085668 learned_addr learned my addr 172.21.15.17:0/4233085668 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1199dd6640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f11940c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 --2- 172.21.15.17:0/4233085668 >> v2:172.21.15.163:3300/0 conn(0x7f1194155c30 0x7f119406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671790212 0 0) 0x7f118c001110 con 0x7f11940b0070 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117c003520 con 0x7f11940b0070 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181035140 0 0) 0x7f117c003520 con 0x7f11940b0070 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 msgr2=0x7f11940c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f11940c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 >> v2:172.21.15.163:3300/0 conn(0x7f1194155c30 msgr2=0x7f119406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 --2- 172.21.15.17:0/4233085668 >> v2:172.21.15.163:3300/0 conn(0x7f1194155c30 0x7f119406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11941560e0 con 0x7f11940b0070 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f118c001840 con 0x7f11940b0070 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 >> v1:172.21.15.17:6789/0 conn(0x7f11940b0070 legacy=0x7f11940c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f117c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117c003f30 0x7f117c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f117c004d40 0x7f117c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119406cfb0 con 0x7f117c003f30 2024-03-06T07:33:04.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119406c020 con 0x7f117c004d40 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c00ba10 con 0x7f11940c5380 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117c003f30 0x7f117c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f11995d5640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f117c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1199dd6640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f117c004d40 0x7f117c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 -- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 msgr2=0x7f117c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c5380 0x7f117c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 -- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f117c004d40 msgr2=0x7f117c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f117c004d40 0x7f117c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.056+0000 7f1198dd4640 1 -- 172.21.15.17:0/4233085668 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c003520 con 0x7f117c003f30 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.057+0000 7f1198dd4640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117c003f30 0x7f117c004800 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f1188009af0 tx=0x7f117c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447ffbafcacadc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.057+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188039070 con 0x7f117c003f30 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.057+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118802c710 con 0x7f117c003f30 2024-03-06T07:33:04.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.057+0000 7f1193fff640 1 -- 172.21.15.17:0/4233085668 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188009690 con 0x7f117c003f30 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 -- 172.21.15.17:0/4233085668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117c003f30 msgr2=0x7f117c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 --2- 172.21.15.17:0/4233085668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f117c003f30 0x7f117c004800 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7f1188009af0 tx=0x7f117c005830 comp rx=0 tx=0).stop 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f1199dd6640 1 -- 172.21.15.17:0/4233085668 reap_dead start 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 -- 172.21.15.17:0/4233085668 shutdown_connections 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 -- 172.21.15.17:0/4233085668 >> 172.21.15.17:0/4233085668 conn(0x7f11940ac590 msgr2=0x7f11940ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 -- 172.21.15.17:0/4233085668 shutdown_connections 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.058+0000 7f119b839640 1 -- 172.21.15.17:0/4233085668 wait complete. 2024-03-06T07:33:04.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.059+0000 7f119b839640 1 Processor -- start 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.059+0000 7f119b839640 1 -- start start 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 0x7f1194154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c3af0 0x7f11940c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119406c270 con 0x7f11940c4b80 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940c7ad0 con 0x7f11940c8a70 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119406c550 con 0x7f11940c3af0 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42074/0 (socket says 172.21.15.17:42074) 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 -- 172.21.15.17:0/1581962974 learned_addr learned my addr 172.21.15.17:0/1581962974 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1199dd6640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c3af0 0x7f11940c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f11995d5640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 0x7f1194154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 -- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c3af0 msgr2=0x7f11940c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c3af0 0x7f11940c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 -- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 msgr2=0x7f1194154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 0x7f1194154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 -- 172.21.15.17:0/1581962974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11941530b0 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f11995d5640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 0x7f1194154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1198dd4640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f1188037040 tx=0x7f1188004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188042020 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118803a040 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1194153390 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.060+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188039070 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.061+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1194153950 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.062+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1194158170 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.062+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1188045020 con 0x7f11940c4b80 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.062+0000 7f1191ffb640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:04.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.062+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f11880ab950 con 0x7f11940c4b80 2024-03-06T07:33:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.063+0000 7f11995d5640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.063+0000 7f11995d5640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 0x7f118006d990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f118c004840 tx=0x7f118c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:04.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.067+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f118804c070 con 0x7f11940c4b80 2024-03-06T07:33:04.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.290+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f119404f330 con 0x7f11940c4b80 2024-03-06T07:33:04.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.936+0000 7f1191ffb640 1 -- 172.21.15.17:0/1581962974 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v220) v1 ==== 104+0+0 (secure 0 0 0) 0x7f118807a520 con 0x7f11940c4b80 2024-03-06T07:33:04.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.938+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 msgr2=0x7f118006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.938+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 0x7f118006d990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f118c004840 tx=0x7f118c009040 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.938+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 msgr2=0x7f11941548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.938+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f1188037040 tx=0x7f1188004730 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 shutdown_connections 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f118006b4d0 0x7f118006d990 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11940c3af0 0x7f11940c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11940c4b80 0x7f11941548c0 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 --2- 172.21.15.17:0/1581962974 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11940c8a70 0x7f1194154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 >> 172.21.15.17:0/1581962974 conn(0x7f11940ac590 msgr2=0x7f11941de810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 shutdown_connections 2024-03-06T07:33:04.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:04.939+0000 7f119b839640 1 -- 172.21.15.17:0/1581962974 wait complete. 2024-03-06T07:33:04.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-06T07:33:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.100+0000 7fd177639640 1 Processor -- start 2024-03-06T07:33:05.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.100+0000 7fd177639640 1 -- start start 2024-03-06T07:33:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd177639640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd170152900 0x7fd1700c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd177639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 0x7fd170155900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd177639640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700c9140 con 0x7fd1700c8900 2024-03-06T07:33:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd177639640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700c6bc0 con 0x7fd170152900 2024-03-06T07:33:05.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd177639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701588a0 con 0x7fd1700c58c0 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd1753d5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd170152900 0x7fd1700c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd174bd4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd1700c8900 0x7fd1701551f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43886/0 (socket says 172.21.15.17:43886) 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd174bd4640 1 -- 172.21.15.17:0/2752442512 learned_addr learned my addr 172.21.15.17:0/2752442512 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd1753d5640 1 -- 172.21.15.17:0/2752442512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 msgr2=0x7fd170155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd175bd6640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 0x7fd170155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd1753d5640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 0x7fd170155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd1753d5640 1 -- 172.21.15.17:0/2752442512 >> v1:172.21.15.17:6789/0 conn(0x7fd1700c8900 legacy=0x7fd1701551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.101+0000 7fd167fff640 1 -- 172.21.15.17:0/2752442512 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786050593 0 0) 0x7fd160001110 con 0x7fd1700c8900 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd1753d5640 1 -- 172.21.15.17:0/2752442512 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170158b80 con 0x7fd170152900 2024-03-06T07:33:05.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd175bd6640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 0x7fd170155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd1753d5640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd170152900 0x7fd1700c5380 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd1680097a0 tx=0x7fd16802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750c8903bfbe6777 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd167fff640 1 -- 172.21.15.17:0/2752442512 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1680315a0 con 0x7fd170152900 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd167fff640 1 -- 172.21.15.17:0/2752442512 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168037070 con 0x7fd170152900 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.102+0000 7fd167fff640 1 -- 172.21.15.17:0/2752442512 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168031960 con 0x7fd170152900 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 -- 172.21.15.17:0/2752442512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd170152900 msgr2=0x7fd1700c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd170152900 0x7fd1700c5380 secure :-1 s=READY pgs=2704 cs=0 l=1 rev1=1 crypto rx=0x7fd1680097a0 tx=0x7fd16802b840 comp rx=0 tx=0).stop 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 -- 172.21.15.17:0/2752442512 shutdown_connections 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1700c58c0 0x7fd170155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 --2- 172.21.15.17:0/2752442512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd170152900 0x7fd1700c5380 unknown :-1 s=CLOSED pgs=2704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 -- 172.21.15.17:0/2752442512 >> 172.21.15.17:0/2752442512 conn(0x7fd1700728d0 msgr2=0x7fd170072ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 -- 172.21.15.17:0/2752442512 shutdown_connections 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.103+0000 7fd177639640 1 -- 172.21.15.17:0/2752442512 wait complete. 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.104+0000 7fd177639640 1 Processor -- start 2024-03-06T07:33:05.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.104+0000 7fd177639640 1 -- start start 2024-03-06T07:33:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1700c8900 0x7fd170147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 0x7fd170148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17006cba0 con 0x7fd1700c58c0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170159660 con 0x7fd1700c8900 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700c8e10 con 0x7fd170152900 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42096/0 (socket says 172.21.15.17:42096) 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 -- 172.21.15.17:0/3847019542 learned_addr learned my addr 172.21.15.17:0/3847019542 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd175bd6640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 0x7fd170148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd174bd4640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1700c8900 0x7fd170147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 -- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 msgr2=0x7fd170148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 0x7fd170148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 -- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1700c8900 msgr2=0x7fd170147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1700c8900 0x7fd170147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 -- 172.21.15.17:0/3847019542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1701eaa00 con 0x7fd1700c58c0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd175bd6640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 0x7fd170148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd1753d5640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fd1680098c0 tx=0x7fd1680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168032840 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168032a00 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1701eace0 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.105+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168032df0 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.106+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd170154d90 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.107+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17004f330 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.107+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd16804e430 con 0x7fd1700c58c0 2024-03-06T07:33:05.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.107+0000 7fd165ffb640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 0x7fd15406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.108+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd16802f030 con 0x7fd1700c58c0 2024-03-06T07:33:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.108+0000 7fd174bd4640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 0x7fd15406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:05.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.108+0000 7fd174bd4640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 0x7fd15406d9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd170157010 tx=0x7fd160007970 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:05.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.112+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd168040070 con 0x7fd1700c58c0 2024-03-06T07:33:05.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:05.293+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd17006ce00 con 0x7fd1700c58c0 2024-03-06T07:33:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.232+0000 7fd165ffb640 1 -- 172.21.15.17:0/3847019542 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v218) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd16807b080 con 0x7fd1700c58c0 2024-03-06T07:33:06.234 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 msgr2=0x7fd15406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 0x7fd15406d9c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd170157010 tx=0x7fd160007970 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 msgr2=0x7fd170154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7fd1680098c0 tx=0x7fd1680046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 shutdown_connections 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd15406b500 0x7fd15406d9c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd170152900 0x7fd170148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1700c8900 0x7fd170147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 --2- 172.21.15.17:0/3847019542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1700c58c0 0x7fd170154850 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 >> 172.21.15.17:0/3847019542 conn(0x7fd1700728d0 msgr2=0x7fd1700cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 shutdown_connections 2024-03-06T07:33:06.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.236+0000 7fd177639640 1 -- 172.21.15.17:0/3847019542 wait complete. 2024-03-06T07:33:06.252 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-06T07:33:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.362+0000 7f7a8aadf640 1 Processor -- start 2024-03-06T07:33:06.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.362+0000 7f7a8aadf640 1 -- start start 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8aadf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a84152870 0x7f7a84155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8aadf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a8406bd60 0x7f7a8406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8aadf640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a84156100 con 0x7f7a840c88e0 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8aadf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8406d320 con 0x7f7a8406bd60 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8aadf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8406b8d0 con 0x7f7a84152870 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a8406bd60 0x7f7a8406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7a8406bd60 0x7f7a8406c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42110/0 (socket says 172.21.15.17:42110) 2024-03-06T07:33:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 -- 172.21.15.17:0/76029635 learned_addr learned my addr 172.21.15.17:0/76029635 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 -- 172.21.15.17:0/76029635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a84152870 msgr2=0x7f7a84155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 --2- 172.21.15.17:0/76029635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a84152870 0x7f7a84155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 -- 172.21.15.17:0/76029635 >> v1:172.21.15.163:6789/0 conn(0x7f7a840c88e0 legacy=0x7f7a840c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:06.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.363+0000 7f7a8907c640 1 -- 172.21.15.17:0/76029635 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840c5400 con 0x7f7a8406bd60 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.364+0000 7f7a8907c640 1 --2- 172.21.15.17:0/76029635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a8406bd60 0x7f7a8406c1f0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009870 tx=0x7f7a7402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96673c1ecd9da3c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.364+0000 7f7a837fe640 1 -- 172.21.15.17:0/76029635 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a740317c0 con 0x7f7a8406bd60 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.364+0000 7f7a837fe640 1 -- 172.21.15.17:0/76029635 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74037070 con 0x7f7a8406bd60 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.364+0000 7f7a837fe640 1 -- 172.21.15.17:0/76029635 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74031b80 con 0x7f7a8406bd60 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 -- 172.21.15.17:0/76029635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a8406bd60 msgr2=0x7f7a8406c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/76029635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a8406bd60 0x7f7a8406c1f0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009870 tx=0x7f7a7402b860 comp rx=0 tx=0).stop 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 -- 172.21.15.17:0/76029635 shutdown_connections 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/76029635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a8406bd60 0x7f7a8406c1f0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/76029635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a84152870 0x7f7a84155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 -- 172.21.15.17:0/76029635 >> 172.21.15.17:0/76029635 conn(0x7f7a840ac5f0 msgr2=0x7f7a840aca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:06.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 -- 172.21.15.17:0/76029635 shutdown_connections 2024-03-06T07:33:06.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.365+0000 7f7a8aadf640 1 -- 172.21.15.17:0/76029635 wait complete. 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.366+0000 7f7a8aadf640 1 Processor -- start 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.366+0000 7f7a8aadf640 1 -- start start 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a8406bd60 0x7f7a841eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a84152870 0x7f7a8414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840c5ca0 con 0x7f7a840c88e0 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840c59c0 con 0x7f7a84152870 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8aadf640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840c7b00 con 0x7f7a8406bd60 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42112/0 (socket says 172.21.15.17:42112) 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 -- 172.21.15.17:0/2303217369 learned_addr learned my addr 172.21.15.17:0/2303217369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8887b640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a8406bd60 0x7f7a841eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a8907c640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a84152870 0x7f7a8414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 -- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a8406bd60 msgr2=0x7f7a841eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a8406bd60 0x7f7a841eeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 -- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a84152870 msgr2=0x7f7a8414e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a84152870 0x7f7a8414e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 -- 172.21.15.17:0/2303217369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840c3e80 con 0x7f7a840c88e0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.367+0000 7f7a83fff640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c430 tx=0x7f7a7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.368+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78017070 con 0x7f7a840c88e0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.368+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780035f0 con 0x7f7a840c88e0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.368+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a841569c0 con 0x7f7a840c88e0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.368+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780039e0 con 0x7f7a840c88e0 2024-03-06T07:33:06.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.368+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a841eb060 con 0x7f7a840c88e0 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.369+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a840c7fe0 con 0x7f7a840c88e0 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.369+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a7801a070 con 0x7f7a840c88e0 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.369+0000 7f7a817fa640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 0x7f7a7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.370+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6094+0+0 (secure 0 0 0) 0x7f7a780895b0 con 0x7f7a840c88e0 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.370+0000 7f7a8887b640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 0x7f7a7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:06.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.370+0000 7f7a8887b640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 0x7f7a7006d9c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f20 tx=0x7f7a7c0094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:06.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.374+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7a840c7fe0 con 0x7f7a840c88e0 2024-03-06T07:33:06.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:06.554+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a841eab70 con 0x7f7a840c88e0 2024-03-06T07:33:07.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.234+0000 7f7a817fa640 1 -- 172.21.15.17:0/2303217369 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v219) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7a78057cf0 con 0x7f7a840c88e0 2024-03-06T07:33:07.235 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 msgr2=0x7f7a7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 0x7f7a7006d9c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009f20 tx=0x7f7a7c0094f0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 msgr2=0x7f7a840c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 secure :-1 s=READY pgs=2707 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c430 tx=0x7f7a7800c8f0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 shutdown_connections 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f7a7006b500 0x7f7a7006d9c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a84152870 0x7f7a8414e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a840c88e0 0x7f7a840c9ea0 unknown :-1 s=CLOSED pgs=2707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 --2- 172.21.15.17:0/2303217369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a8406bd60 0x7f7a841eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 >> 172.21.15.17:0/2303217369 conn(0x7f7a840ac5f0 msgr2=0x7f7a840abdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:07.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 shutdown_connections 2024-03-06T07:33:07.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.238+0000 7f7a8aadf640 1 -- 172.21.15.17:0/2303217369 wait complete. 2024-03-06T07:33:07.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-06T07:33:07.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.363+0000 7f640caa4640 1 Processor -- start 2024-03-06T07:33:07.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.363+0000 7f640caa4640 1 -- start start 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.363+0000 7f640caa4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f64080ab310 0x7f64080ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640caa4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64080a97c0 0x7f64080c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640caa4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6408154330 con 0x7f64081538c0 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640caa4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080c46a0 con 0x7f64080a97c0 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640caa4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6408153dc0 con 0x7f64080ab310 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6406d9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f64080a97c0 0x7f64080c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6405d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f64081538c0 0x7f64081494a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43900/0 (socket says 172.21.15.17:43900) 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6405d9b640 1 -- 172.21.15.17:0/2215207876 learned_addr learned my addr 172.21.15.17:0/2215207876 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640659c640 1 --2- 172.21.15.17:0/2215207876 >> v2:172.21.15.163:3300/0 conn(0x7f64080ab310 0x7f64080ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6406d9d640 1 -- 172.21.15.17:0/2215207876 >> v2:172.21.15.163:3300/0 conn(0x7f64080ab310 msgr2=0x7f64080ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:07.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6406d9d640 1 --2- 172.21.15.17:0/2215207876 >> v2:172.21.15.163:3300/0 conn(0x7f64080ab310 0x7f64080ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6406d9d640 1 -- 172.21.15.17:0/2215207876 >> v1:172.21.15.17:6789/0 conn(0x7f64081538c0 legacy=0x7f64081494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f640559a640 1 -- 172.21.15.17:0/2215207876 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517542648 0 0) 0x7f6408154330 con 0x7f64081538c0 2024-03-06T07:33:07.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.364+0000 7f6406d9d640 1 -- 172.21.15.17:0/2215207876 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080c5d30 con 0x7f64080a97c0 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f6406d9d640 1 --2- 172.21.15.17:0/2215207876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080a97c0 0x7f64080c4b30 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f64080a9550 tx=0x7f640002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeadcfa16f87d96 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f640559a640 1 -- 172.21.15.17:0/2215207876 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400039070 con 0x7f64080a97c0 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f640559a640 1 -- 172.21.15.17:0/2215207876 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640002cbf0 con 0x7f64080a97c0 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f640559a640 1 -- 172.21.15.17:0/2215207876 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400034440 con 0x7f64080a97c0 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f640caa4640 1 -- 172.21.15.17:0/2215207876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080a97c0 msgr2=0x7f64080c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.365+0000 7f640caa4640 1 --2- 172.21.15.17:0/2215207876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080a97c0 0x7f64080c4b30 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f64080a9550 tx=0x7f640002be40 comp rx=0 tx=0).stop 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 -- 172.21.15.17:0/2215207876 shutdown_connections 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 --2- 172.21.15.17:0/2215207876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080a97c0 0x7f64080c4b30 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 --2- 172.21.15.17:0/2215207876 >> v2:172.21.15.163:3300/0 conn(0x7f64080ab310 0x7f64080ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 -- 172.21.15.17:0/2215207876 >> 172.21.15.17:0/2215207876 conn(0x7f64080c6710 msgr2=0x7f64080c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:07.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 -- 172.21.15.17:0/2215207876 shutdown_connections 2024-03-06T07:33:07.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.366+0000 7f640caa4640 1 -- 172.21.15.17:0/2215207876 wait complete. 2024-03-06T07:33:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.367+0000 7f640caa4640 1 Processor -- start 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.367+0000 7f640caa4640 1 -- start start 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.367+0000 7f640caa4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64080a97c0 0x7f6408152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64081538c0 0x7f64081e8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080abc30 con 0x7f64080ab310 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c62f0 con 0x7f64081538c0 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408149bb0 con 0x7f64080a97c0 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42142/0 (socket says 172.21.15.17:42142) 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 -- 172.21.15.17:0/1159373564 learned_addr learned my addr 172.21.15.17:0/1159373564 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6406d9d640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64081538c0 0x7f64081e8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 -- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64080a97c0 msgr2=0x7f6408152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64080a97c0 0x7f6408152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 -- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64081538c0 msgr2=0x7f64081e8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64081538c0 0x7f64081e8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 -- 172.21.15.17:0/1159373564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64081ec890 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f6405d9b640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c410 tx=0x7f63fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0074e0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc0047e0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc004bd0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64081469a0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.368+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6408072ed0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.369+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080acab0 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.370+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63fc011070 con 0x7f64080ab310 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.370+0000 7f63f6ffd640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 0x7f63e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.370+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f63fc089ce0 con 0x7f64080ab310 2024-03-06T07:33:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.370+0000 7f640659c640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 0x7f63e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.371+0000 7f640659c640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 0x7f63e806d990 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.375+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64080acab0 con 0x7f64080ab310 2024-03-06T07:33:07.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.558+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f640804f330 con 0x7f64080ab310 2024-03-06T07:33:07.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.561+0000 7f63f6ffd640 1 -- 172.21.15.17:0/1159373564 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7f63fc00e280 con 0x7f64080ab310 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 msgr2=0x7f63e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 0x7f63e806d990 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006fd0 tx=0x7f63f8008040 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 msgr2=0x7f64081e89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c410 tx=0x7f63fc00c8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 shutdown_connections 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f63e806b4d0 0x7f63e806d990 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f64081538c0 0x7f64081e8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ab310 0x7f64081e89e0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 --2- 172.21.15.17:0/1159373564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f64080a97c0 0x7f6408152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 >> 172.21.15.17:0/1159373564 conn(0x7f64080c6710 msgr2=0x7f64080c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 shutdown_connections 2024-03-06T07:33:07.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.564+0000 7f640caa4640 1 -- 172.21.15.17:0/1159373564 wait complete. 2024-03-06T07:33:07.576 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice' 2024-03-06T07:33:07.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.675+0000 7f698429a640 1 Processor -- start 2024-03-06T07:33:07.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.675+0000 7f698429a640 1 -- start start 2024-03-06T07:33:07.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f698429a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f697c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f698429a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c156180 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f698429a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c159630 con 0x7f697c0c5330 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f698429a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1587b0 con 0x7f697c152920 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f6981835640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f697c0c8a00 0x7f697c158f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43906/0 (socket says 172.21.15.17:43906) 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f6981835640 1 -- 172.21.15.17:0/3451832676 learned_addr learned my addr 172.21.15.17:0/3451832676 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f6982036640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f697c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811282716 0 0) 0x7f696c001110 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.676+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6968003520 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184990940 0 0) 0x7f6968003520 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 msgr2=0x7f697c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f697c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 >> v1:172.21.15.163:6789/0 conn(0x7f697c0c5330 legacy=0x7f697c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0c7760 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f696c001840 con 0x7f697c0c8a00 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 >> v1:172.21.15.17:6789/0 conn(0x7f697c0c8a00 legacy=0x7f697c158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69680039e0 0x7f6968003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f6968004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968004d40 0x7f69680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1587b0 con 0x7f69680039e0 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696800ba70 con 0x7f6968004d40 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696800bc70 con 0x7f697c152920 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69680039e0 0x7f6968003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6981835640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f6968004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982837640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968004d40 0x7f69680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 -- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 msgr2=0x7f6968004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f697c152920 0x7f6968004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 -- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968004d40 msgr2=0x7f69680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6968004d40 0x7f69680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.677+0000 7f6982036640 1 -- 172.21.15.17:0/3451832676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968003520 con 0x7f69680039e0 2024-03-06T07:33:07.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f6982036640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69680039e0 0x7f6968003dc0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f6978009ac0 tx=0x7f6968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725432702227a1e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978039070 con 0x7f69680039e0 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697802c6b0 con 0x7f69680039e0 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f6981034640 1 -- 172.21.15.17:0/3451832676 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978009690 con 0x7f69680039e0 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f698429a640 1 -- 172.21.15.17:0/3451832676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69680039e0 msgr2=0x7f6968003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f698429a640 1 --2- 172.21.15.17:0/3451832676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69680039e0 0x7f6968003dc0 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7f6978009ac0 tx=0x7f6968005830 comp rx=0 tx=0).stop 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.678+0000 7f6982837640 1 -- 172.21.15.17:0/3451832676 reap_dead start 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.679+0000 7f698429a640 1 -- 172.21.15.17:0/3451832676 shutdown_connections 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.679+0000 7f698429a640 1 -- 172.21.15.17:0/3451832676 >> 172.21.15.17:0/3451832676 conn(0x7f697c0ac560 msgr2=0x7f697c0ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.679+0000 7f698429a640 1 -- 172.21.15.17:0/3451832676 shutdown_connections 2024-03-06T07:33:07.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.679+0000 7f698429a640 1 -- 172.21.15.17:0/3451832676 wait complete. 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 Processor -- start 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 -- start start 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 0x7f697c14bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 0x7f697c0c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c158a00 con 0x7f697c155a10 2024-03-06T07:33:07.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0c7d20 con 0x7f697c155590 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.680+0000 7f698429a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c06d030 con 0x7f69680039e0 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42156/0 (socket says 172.21.15.17:42156) 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 -- 172.21.15.17:0/1720377283 learned_addr learned my addr 172.21.15.17:0/1720377283 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982036640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 0x7f697c14bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6981835640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 0x7f697c0c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 -- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 msgr2=0x7f697c14bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 0x7f697c14bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 -- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 msgr2=0x7f697c0c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 0x7f697c0c20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 -- 172.21.15.17:0/1720377283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c14c0c0 con 0x7f697c155a10 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982036640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 0x7f697c14bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6981835640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 0x7f697c0c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:07.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6982837640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f697c0be490 tx=0x7f697400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400ee50 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974018070 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c153b70 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974003dc0 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.681+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c154130 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.682+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0caed0 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.683+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69740050d0 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.683+0000 7f6972ffd640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 0x7f696006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.683+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f6974089f40 con 0x7f697c155a10 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.683+0000 7f6982036640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 0x7f696006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:07.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.684+0000 7f6982036640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 0x7f696006d9c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6978036000 tx=0x7f6978030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:07.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.687+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f697c0caed0 con 0x7f697c155a10 2024-03-06T07:33:07.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:07.939+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]} v 0) v1 -- 0x7f697c04f330 con 0x7f697c155a10 2024-03-06T07:33:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.247+0000 7f6972ffd640 1 -- 172.21.15.17:0/1720377283 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_failover.TestClusterResize.test_down_twice"]}]=0 v710) v1 ==== 139+0+0 (secure 0 0 0) 0x7f6974058e30 con 0x7f697c155a10 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.249+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 msgr2=0x7f696006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.249+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 0x7f696006d9c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6978036000 tx=0x7f6978030000 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 msgr2=0x7f697c0c2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f697c0be490 tx=0x7f697400ec00 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 shutdown_connections 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f696006b500 0x7f696006d9c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f697c155a10 0x7f697c0c2620 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f697c155590 0x7f697c0c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 --2- 172.21.15.17:0/1720377283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f69680039e0 0x7f697c14bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 >> 172.21.15.17:0/1720377283 conn(0x7f697c0ac560 msgr2=0x7f697c150bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:08.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 shutdown_connections 2024-03-06T07:33:08.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:08.250+0000 7f698429a640 1 -- 172.21.15.17:0/1720377283 wait complete. 2024-03-06T07:33:08.263 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. ... ERROR 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:ERROR: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/test_failover.py", line 274, in test_down_twice 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-03-06T07:33:08.264 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:Ran 12 tests in 744.660s 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:ERROR: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:That marking a FS down twice does not wipe old_max_mds. 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/test_failover.py", line 274, in test_down_twice 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: self.fs.wait_for_daemons() 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/filesystem.py", line 1240, in wait_for_daemons 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: raise RuntimeError("Timed out waiting for MDS daemons to become healthy") 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out waiting for MDS daemons to become healthy 2024-03-06T07:33:08.265 INFO:tasks.cephfs_test_runner: 2024-03-06T07:33:08.266 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:08.455 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a393e455f36c42c5907734f5d79b1903 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize) 2024-03-06T07:33:08.456 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-06T07:33:08.465 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-06T07:33:08.496 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-06T07:33:08.496 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-06T07:33:08.496 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:08.516 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057147,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=529 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-06T07:33:08.516 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2024-03-06T07:33:08.516 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-06T07:33:08.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056368,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19781 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248840k,nr_inodes=812210,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-03-06T07:33:08.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.535 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-06T07:33:08.571 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-06T07:33:08.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-06T07:33:08.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-06T07:33:08.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.732 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2024-03-06T07:33:08.751 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-06T07:33:08.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-06T07:33:08.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-06T07:33:08.823 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2024-03-06T07:33:08.922 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-06T07:33:08.931 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-06T07:33:08.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:09.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.032+0000 7f9c2b6f4640 1 Processor -- start 2024-03-06T07:33:09.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 -- start start 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 0x7f9c241529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c240c1aa0 0x7f9c240c5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240c28d0 con 0x7f9c240c5c60 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c240c2680 con 0x7f9c240c1aa0 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.033+0000 7f9c2b6f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240c2bb0 con 0x7f9c2414f790 2024-03-06T07:33:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c29490640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 0x7f9c241529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c29c91640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f9c240c5c60 0x7f9c24155a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:47244/0 (socket says 172.21.15.17:47244) 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c28c8f640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c240c1aa0 0x7f9c240c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c29c91640 1 -- 172.21.15.17:0/1693774193 learned_addr learned my addr 172.21.15.17:0/1693774193 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c1bfff640 1 -- 172.21.15.17:0/1693774193 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885589312 0 0) 0x7f9c240c28d0 con 0x7f9c240c5c60 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c1bfff640 1 -- 172.21.15.17:0/1693774193 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c10003520 con 0x7f9c240c5c60 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c28c8f640 1 -- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 msgr2=0x7f9c241529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c28c8f640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 0x7f9c241529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c28c8f640 1 -- 172.21.15.17:0/1693774193 >> v1:172.21.15.163:6789/0 conn(0x7f9c240c5c60 legacy=0x7f9c24155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c28c8f640 1 -- 172.21.15.17:0/1693774193 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240c2e90 con 0x7f9c240c1aa0 2024-03-06T07:33:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.034+0000 7f9c29490640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 0x7f9c241529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:09.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c28c8f640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c240c5720 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9c140097a0 tx=0x7f9c1402dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78215a594ae1add7 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c1bfff640 1 -- 172.21.15.17:0/1693774193 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402f8a0 con 0x7f9c240c1aa0 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c1bfff640 1 -- 172.21.15.17:0/1693774193 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1402fa40 con 0x7f9c240c1aa0 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c1bfff640 1 -- 172.21.15.17:0/1693774193 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402fe10 con 0x7f9c240c1aa0 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 msgr2=0x7f9c240c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.035+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c240c5720 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9c140097a0 tx=0x7f9c1402dae0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/1693774193 shutdown_connections 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c240c5720 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/1693774193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c2414f790 0x7f9c241529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/1693774193 >> 172.21.15.17:0/1693774193 conn(0x7f9c240c6710 msgr2=0x7f9c240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/1693774193 shutdown_connections 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.036+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/1693774193 wait complete. 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.037+0000 7f9c2b6f4640 1 Processor -- start 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.037+0000 7f9c2b6f4640 1 -- start start 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.037+0000 7f9c2b6f4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c24072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.037+0000 7f9c2b6f4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c240c5c60 0x7f9c241543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.037+0000 7f9c2b6f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c2b6f4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2406cc00 con 0x7f9c2414f790 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c2b6f4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240c0d60 con 0x7f9c240c1aa0 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c2b6f4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2406cd80 con 0x7f9c240c5c60 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42162/0 (socket says 172.21.15.17:42162) 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 -- 172.21.15.17:0/3206875050 learned_addr learned my addr 172.21.15.17:0/3206875050 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c28c8f640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c240c5c60 0x7f9c241543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29490640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c24072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 -- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c240c5c60 msgr2=0x7f9c241543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c240c5c60 0x7f9c241543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 -- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 msgr2=0x7f9c24072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c24072f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 -- 172.21.15.17:0/3206875050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c24072760 con 0x7f9c2414f790 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c29c91640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00cb40 tx=0x7f9c1c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c017070 con 0x7f9c2414f790 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00aca0 con 0x7f9c2414f790 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c012620 con 0x7f9c2414f790 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.038+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24073460 con 0x7f9c2414f790 2024-03-06T07:33:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.039+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c24071f50 con 0x7f9c2414f790 2024-03-06T07:33:09.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.040+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2404f330 con 0x7f9c2414f790 2024-03-06T07:33:09.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.040+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c1c01a070 con 0x7f9c2414f790 2024-03-06T07:33:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.040+0000 7f9c19ffb640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 0x7f9c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.040+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f9c1c089ed0 con 0x7f9c2414f790 2024-03-06T07:33:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.041+0000 7f9c29490640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 0x7f9c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.041+0000 7f9c29490640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 0x7f9c0806d990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009560 tx=0x7f9c20009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.045+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c1c058780 con 0x7f9c2414f790 2024-03-06T07:33:09.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.222+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c240c1f40 con 0x7f9c0806b4d0 2024-03-06T07:33:09.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.224+0000 7f9c19ffb640 1 -- 172.21.15.17:0/3206875050 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13555 (secure 0 0 0) 0x7f9c240c1f40 con 0x7f9c0806b4d0 2024-03-06T07:33:09.226 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:09.226 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 msgr2=0x7f9c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 0x7f9c0806d990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009560 tx=0x7f9c20009190 comp rx=0 tx=0).stop 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 msgr2=0x7f9c241e6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00cb40 tx=0x7f9c1c00a060 comp rx=0 tx=0).stop 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 shutdown_connections 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f9c0806b4d0 0x7f9c0806d990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2414f790 0x7f9c241e6240 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c240c5c60 0x7f9c241543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 --2- 172.21.15.17:0/3206875050 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c240c1aa0 0x7f9c24072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 >> 172.21.15.17:0/3206875050 conn(0x7f9c240c6710 msgr2=0x7f9c240c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 shutdown_connections 2024-03-06T07:33:09.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.226+0000 7f9c2b6f4640 1 -- 172.21.15.17:0/3206875050 wait complete. 2024-03-06T07:33:09.238 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":607,"stamp":"2024-03-06T07:33:07.530294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":287852,"kb_used_data":21760,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686868,"statfs":{"total":773094113280,"available":772799352832,"internally_reserved":0,"allocated":22282240,"data_stored":11972046,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-55,"num_read_kb":-61,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001344"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":541,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-03-06T07:33:06.333805+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:33:06.333805+0000","last_peered":"2024-03-06T07:33:06.333805+0000","last_clean":"2024-03-06T07:33:06.333805+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:33:06.333805+0000","last_undegraded":"2024-03-06T07:33:06.333805+0000","last_fullsized":"2024-03-06T07:33:06.333805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33948,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337892,"statfs":{"total":96636764160,"available":96602001408,"internally_reserved":0,"allocated":2834432,"data_stored":1941514,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38040,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333800,"statfs":{"total":96636764160,"available":96597811200,"internally_reserved":0,"allocated":2764800,"data_stored":1345168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38044,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333796,"statfs":{"total":96636764160,"available":96597807104,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1345846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2826240,"data_stored":1934934,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:09.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.339+0000 7f803398a640 1 Processor -- start 2024-03-06T07:33:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.339+0000 7f803398a640 1 -- start start 2024-03-06T07:33:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f803398a640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f802c0c5480 0x7f802c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f803398a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0c5cc0 con 0x7f802c1586f0 2024-03-06T07:33:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f803398a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c1591c0 con 0x7f802c0c5860 2024-03-06T07:33:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f803398a640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c06df40 con 0x7f802c0c5480 2024-03-06T07:33:09.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f8031f27640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f802c1586f0 0x7f802c15f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43910/0 (socket says 172.21.15.17:43910) 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f8031f27640 1 -- 172.21.15.17:0/2564306705 learned_addr learned my addr 172.21.15.17:0/2564306705 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.340+0000 7f8031726640 1 --2- 172.21.15.17:0/2564306705 >> v2:172.21.15.163:3300/0 conn(0x7f802c0c5480 0x7f802c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608559202 0 0) 0x7f802c0c5cc0 con 0x7f802c1586f0 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8014003520 con 0x7f802c1586f0 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864419091 0 0) 0x7f802c1591c0 con 0x7f802c0c5860 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c0c5cc0 con 0x7f802c0c5860 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3742588727 0 0) 0x7f8014003520 con 0x7f802c1586f0 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 >> v2:172.21.15.163:3300/0 conn(0x7f802c0c5480 msgr2=0x7f802c155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 --2- 172.21.15.17:0/2564306705 >> v2:172.21.15.163:3300/0 conn(0x7f802c0c5480 0x7f802c155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 >> v1:172.21.15.163:6789/0 conn(0x7f802c0c5860 legacy=0x7f802c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c1594a0 con 0x7f802c1586f0 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f801c0019f0 con 0x7f802c1586f0 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 >> v1:172.21.15.17:6789/0 conn(0x7f802c1586f0 legacy=0x7f802c15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8014003b40 0x7f8014003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 0x7f8014004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80140051c0 0x7f801400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c06df40 con 0x7f8014004800 2024-03-06T07:33:09.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801400c0f0 con 0x7f80140051c0 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801400c2f0 con 0x7f8014003b40 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 0x7f8014004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8031f27640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80140051c0 0x7f801400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8031726640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8014003b40 0x7f8014003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 -- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8014003b40 msgr2=0x7f8014003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8014003b40 0x7f8014003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 -- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80140051c0 msgr2=0x7f801400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80140051c0 0x7f801400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8031f27640 1 -- 172.21.15.17:0/2564306705 reap_dead start 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.341+0000 7f8030f25640 1 -- 172.21.15.17:0/2564306705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014003520 con 0x7f8014004800 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.342+0000 7f8030f25640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 0x7f8014004c80 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f80200018d0 tx=0x7f8014005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6667dafcbf64ad server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.342+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020037070 con 0x7f8014004800 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.342+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80200052d0 con 0x7f8014004800 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.342+0000 7f802bfff640 1 -- 172.21.15.17:0/2564306705 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80200014f0 con 0x7f8014004800 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 -- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 msgr2=0x7f8014004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 0x7f8014004c80 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7f80200018d0 tx=0x7f8014005830 comp rx=0 tx=0).stop 2024-03-06T07:33:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 -- 172.21.15.17:0/2564306705 shutdown_connections 2024-03-06T07:33:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 --2- 172.21.15.17:0/2564306705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8014004800 0x7f8014004c80 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 -- 172.21.15.17:0/2564306705 >> 172.21.15.17:0/2564306705 conn(0x7f802c06b1a0 msgr2=0x7f802c06b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 -- 172.21.15.17:0/2564306705 shutdown_connections 2024-03-06T07:33:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.343+0000 7f803398a640 1 -- 172.21.15.17:0/2564306705 wait complete. 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.344+0000 7f803398a640 1 Processor -- start 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.344+0000 7f803398a640 1 -- start start 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8014004800 0x7f802c0ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c06ff30 0x7f802c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c155680 con 0x7f802c0acc40 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c06fdb0 con 0x7f8014004800 2024-03-06T07:33:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c152a20 con 0x7f802c06ff30 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42182/0 (socket says 172.21.15.17:42182) 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 -- 172.21.15.17:0/4281151347 learned_addr learned my addr 172.21.15.17:0/4281151347 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8030f25640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c06ff30 0x7f802c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031726640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8014004800 0x7f802c0ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 -- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c06ff30 msgr2=0x7f802c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c06ff30 0x7f802c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 -- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8014004800 msgr2=0x7f802c0ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8014004800 0x7f802c0ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 -- 172.21.15.17:0/4281151347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0c3210 con 0x7f802c0acc40 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8031f27640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f802c0c2f40 tx=0x7f801c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00aa90 con 0x7f802c0acc40 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c018070 con 0x7f802c0acc40 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c0133f0 con 0x7f802c0acc40 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.345+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0ad440 con 0x7f802c0acc40 2024-03-06T07:33:09.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.346+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c158460 con 0x7f802c0acc40 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.346+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c152c30 con 0x7f802c0acc40 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.347+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f801c013590 con 0x7f802c0acc40 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.347+0000 7f8029ffb640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 0x7f801806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.347+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f801c08a6a0 con 0x7f802c0acc40 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.347+0000 7f8031726640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 0x7f801806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.348+0000 7f8031726640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 0x7f801806d9c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f80240099a0 tx=0x7f8024008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.352+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f802c152c30 con 0x7f802c0acc40 2024-03-06T07:33:09.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.533+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f802c0c2fc0 con 0x7f801806b500 2024-03-06T07:33:09.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.534+0000 7f8029ffb640 1 -- 172.21.15.17:0/4281151347 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13555 (secure 0 0 0) 0x7f802c0c2fc0 con 0x7f801806b500 2024-03-06T07:33:09.536 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:09.537 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:33:09.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 msgr2=0x7f801806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 0x7f801806d9c0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f80240099a0 tx=0x7f8024008040 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 msgr2=0x7f802c1ee990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7f802c0c2f40 tx=0x7f801c007a90 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 shutdown_connections 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f801806b500 0x7f801806d9c0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f802c0acc40 0x7f802c1ee990 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c06ff30 0x7f802c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 --2- 172.21.15.17:0/4281151347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8014004800 0x7f802c0ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 >> 172.21.15.17:0/4281151347 conn(0x7f802c06b1a0 msgr2=0x7f802c1515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 shutdown_connections 2024-03-06T07:33:09.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.536+0000 7f803398a640 1 -- 172.21.15.17:0/4281151347 wait complete. 2024-03-06T07:33:09.549 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-03-06T07:33:09.530733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":287852,"kb_used_data":21760,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686868,"statfs":{"total":773094113280,"available":772799352832,"internally_reserved":0,"allocated":22282240,"data_stored":11972046,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-55,"num_read_kb":-61,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001444"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":541,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-03-06T07:33:06.333805+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:33:06.333805+0000","last_peered":"2024-03-06T07:33:06.333805+0000","last_clean":"2024-03-06T07:33:06.333805+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:33:06.333805+0000","last_undegraded":"2024-03-06T07:33:06.333805+0000","last_fullsized":"2024-03-06T07:33:06.333805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33948,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337892,"statfs":{"total":96636764160,"available":96602001408,"internally_reserved":0,"allocated":2834432,"data_stored":1941514,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38040,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333800,"statfs":{"total":96636764160,"available":96597811200,"internally_reserved":0,"allocated":2764800,"data_stored":1345168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38044,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333796,"statfs":{"total":96636764160,"available":96597807104,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1345846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2826240,"data_stored":1934934,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:09.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-06T07:33:09.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.649+0000 7fd69c3a1640 1 Processor -- start 2024-03-06T07:33:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.649+0000 7fd69c3a1640 1 -- start start 2024-03-06T07:33:09.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69c3a1640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd694150580 0x7fd69406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69c3a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd69406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69c3a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd694153a60 con 0x7fd6940c6000 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69c3a1640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd694153d40 con 0x7fd694150580 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69c3a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69406c2c0 con 0x7fd6940c1320 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69a93e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd6940c6000 0x7fd6941566b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43912/0 (socket says 172.21.15.17:43912) 2024-03-06T07:33:09.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69a93e640 1 -- 172.21.15.17:0/1682417456 learned_addr learned my addr 172.21.15.17:0/1682417456 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69993c640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd69406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69a13d640 1 --2- 172.21.15.17:0/1682417456 >> v2:172.21.15.163:3300/0 conn(0x7fd694150580 0x7fd69406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224722724 0 0) 0x7fd694153a60 con 0x7fd6940c6000 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003520 con 0x7fd6940c6000 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991527440 0 0) 0x7fd680003520 con 0x7fd6940c6000 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 msgr2=0x7fd69406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd69406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 >> v2:172.21.15.163:3300/0 conn(0x7fd694150580 msgr2=0x7fd69406dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 --2- 172.21.15.17:0/1682417456 >> v2:172.21.15.163:3300/0 conn(0x7fd694150580 0x7fd69406dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69406c5a0 con 0x7fd6940c6000 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fd68c0019f0 con 0x7fd6940c6000 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69a13d640 1 --2- 172.21.15.17:0/1682417456 >> v2:172.21.15.163:3300/0 conn(0x7fd694150580 0x7fd69406dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 >> v1:172.21.15.17:6789/0 conn(0x7fd6940c6000 legacy=0x7fd6941566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd680004800 0x7fd680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd6800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694153d40 con 0x7fd680004800 2024-03-06T07:33:09.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69406c2c0 con 0x7fd6800039b0 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.650+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000ba70 con 0x7fd6940c1320 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd680004800 0x7fd680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69a13d640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69a93e640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd6800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 -- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 msgr2=0x7fd6800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd6800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 -- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6800039b0 msgr2=0x7fd680003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69993c640 1 -- 172.21.15.17:0/1682417456 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680003520 con 0x7fd680004800 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.651+0000 7fd69a93e640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6940c1320 0x7fd6800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69993c640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd680004800 0x7fd680004c80 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fd684009a90 tx=0x7fd680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5988fd517b3b507f server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684039070 con 0x7fd680004800 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68402c5a0 con 0x7fd680004800 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69913b640 1 -- 172.21.15.17:0/1682417456 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684009690 con 0x7fd680004800 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 -- 172.21.15.17:0/1682417456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd680004800 msgr2=0x7fd680004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/1682417456 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd680004800 0x7fd680004c80 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7fd684009a90 tx=0x7fd680005830 comp rx=0 tx=0).stop 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69a93e640 1 -- 172.21.15.17:0/1682417456 reap_dead start 2024-03-06T07:33:09.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 -- 172.21.15.17:0/1682417456 shutdown_connections 2024-03-06T07:33:09.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 -- 172.21.15.17:0/1682417456 >> 172.21.15.17:0/1682417456 conn(0x7fd6940c6710 msgr2=0x7fd6940c6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 -- 172.21.15.17:0/1682417456 shutdown_connections 2024-03-06T07:33:09.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.652+0000 7fd69c3a1640 1 -- 172.21.15.17:0/1682417456 wait complete. 2024-03-06T07:33:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.653+0000 7fd69c3a1640 1 Processor -- start 2024-03-06T07:33:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.653+0000 7fd69c3a1640 1 -- start start 2024-03-06T07:33:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd680004800 0x7fd694146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd694150e50 0x7fd694151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c0dc0 con 0x7fd694155f00 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c0c40 con 0x7fd680004800 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69c3a1640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69406e250 con 0x7fd694150e50 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42200/0 (socket says 172.21.15.17:42200) 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 -- 172.21.15.17:0/754571896 learned_addr learned my addr 172.21.15.17:0/754571896 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 -- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd694150e50 msgr2=0x7fd694151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd694150e50 0x7fd694151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 -- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd680004800 msgr2=0x7fd694146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd680004800 0x7fd694146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.654+0000 7fd69993c640 1 -- 172.21.15.17:0/754571896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69414e460 con 0x7fd694155f00 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd69993c640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fd684037040 tx=0x7fd684004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684042020 con 0x7fd694155f00 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68403a040 con 0x7fd694155f00 2024-03-06T07:33:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd694154a00 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684039070 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.655+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6941ec890 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.656+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69404f330 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.656+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd684045020 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.657+0000 7fd68affd640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 0x7fd67806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.657+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fd6840ab380 con 0x7fd694155f00 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.657+0000 7fd69a13d640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 0x7fd67806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.657+0000 7fd69a13d640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 0x7fd67806d9c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd690006a90 tx=0x7fd690006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.661+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd68404c070 con 0x7fd694155f00 2024-03-06T07:33:09.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.839+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd694068450 con 0x7fd67806b500 2024-03-06T07:33:09.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.840+0000 7fd68affd640 1 -- 172.21.15.17:0/754571896 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13555 (secure 0 0 0) 0x7fd694068450 con 0x7fd67806b500 2024-03-06T07:33:09.842 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:09.842 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 msgr2=0x7fd67806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 0x7fd67806d9c0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd690006a90 tx=0x7fd690006a20 comp rx=0 tx=0).stop 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 msgr2=0x7fd6941ebd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7fd684037040 tx=0x7fd684004730 comp rx=0 tx=0).stop 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 shutdown_connections 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fd67806b500 0x7fd67806d9c0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd694150e50 0x7fd694151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd694155f00 0x7fd6941ebd90 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 --2- 172.21.15.17:0/754571896 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd680004800 0x7fd694146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 >> 172.21.15.17:0/754571896 conn(0x7fd6940c6710 msgr2=0x7fd6940c4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 shutdown_connections 2024-03-06T07:33:09.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.842+0000 7fd69c3a1640 1 -- 172.21.15.17:0/754571896 wait complete. 2024-03-06T07:33:09.854 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2024-03-06T07:33:09.530733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":287852,"kb_used_data":21760,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686868,"statfs":{"total":773094113280,"available":772799352832,"internally_reserved":0,"allocated":22282240,"data_stored":11972046,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-55,"num_read_kb":-61,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001444"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":541,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-03-06T07:33:06.333805+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:33:06.333805+0000","last_peered":"2024-03-06T07:33:06.333805+0000","last_clean":"2024-03-06T07:33:06.333805+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:33:06.333805+0000","last_undegraded":"2024-03-06T07:33:06.333805+0000","last_fullsized":"2024-03-06T07:33:06.333805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575008,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33948,"kb_used_data":2768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337892,"statfs":{"total":96636764160,"available":96602001408,"internally_reserved":0,"allocated":2834432,"data_stored":1941514,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575008,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575009,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575009,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33888,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337952,"statfs":{"total":96636764160,"available":96602062848,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38040,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333800,"statfs":{"total":96636764160,"available":96597811200,"internally_reserved":0,"allocated":2764800,"data_stored":1345168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38044,"kb_used_data":2704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333796,"statfs":{"total":96636764160,"available":96597807104,"internally_reserved":0,"allocated":2768896,"data_stored":1351146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575008,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2781184,"data_stored":1345846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38100,"kb_used_data":2760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333740,"statfs":{"total":96636764160,"available":96597749760,"internally_reserved":0,"allocated":2826240,"data_stored":1934934,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:09.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-06T07:33:09.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.954+0000 7ff542b7a640 1 Processor -- start 2024-03-06T07:33:09.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.954+0000 7ff542b7a640 1 -- start start 2024-03-06T07:33:09.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff542b7a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff53c06afc0 0x7ff53c06b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff542b7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c0c20b0 0x7ff53c153180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff542b7a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c153dd0 con 0x7ff53c0c25c0 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff542b7a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c0a9530 con 0x7ff53c06afc0 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff542b7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0a9810 con 0x7ff53c0c20b0 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff540916640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff53c06afc0 0x7ff53c06b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff541117640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff53c0c25c0 0x7ff53c1536c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43918/0 (socket says 172.21.15.17:43918) 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff541117640 1 -- 172.21.15.17:0/1869180957 learned_addr learned my addr 172.21.15.17:0/1869180957 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff53bfff640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c0c20b0 0x7ff53c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff540916640 1 -- 172.21.15.17:0/1869180957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c0c20b0 msgr2=0x7ff53c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff540916640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c0c20b0 0x7ff53c153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff540916640 1 -- 172.21.15.17:0/1869180957 >> v1:172.21.15.17:6789/0 conn(0x7ff53c0c25c0 legacy=0x7ff53c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff53b7fe640 1 -- 172.21.15.17:0/1869180957 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496243593 0 0) 0x7ff53c153dd0 con 0x7ff53c0c25c0 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.955+0000 7ff540916640 1 -- 172.21.15.17:0/1869180957 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0a9af0 con 0x7ff53c06afc0 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.956+0000 7ff540916640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c06afc0 0x7ff53c06b3a0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7ff534009d90 tx=0x7ff53402bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb426f8fa04dbce6 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.956+0000 7ff53b7fe640 1 -- 172.21.15.17:0/1869180957 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534039070 con 0x7ff53c06afc0 2024-03-06T07:33:09.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.956+0000 7ff53b7fe640 1 -- 172.21.15.17:0/1869180957 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53402cb30 con 0x7ff53c06afc0 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.956+0000 7ff53b7fe640 1 -- 172.21.15.17:0/1869180957 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534034440 con 0x7ff53c06afc0 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 -- 172.21.15.17:0/1869180957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c06afc0 msgr2=0x7ff53c06b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c06afc0 0x7ff53c06b3a0 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7ff534009d90 tx=0x7ff53402bd80 comp rx=0 tx=0).stop 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 -- 172.21.15.17:0/1869180957 shutdown_connections 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c0c20b0 0x7ff53c153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1869180957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c06afc0 0x7ff53c06b3a0 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 -- 172.21.15.17:0/1869180957 >> 172.21.15.17:0/1869180957 conn(0x7ff53c0c76b0 msgr2=0x7ff53c0c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 -- 172.21.15.17:0/1869180957 shutdown_connections 2024-03-06T07:33:09.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.957+0000 7ff542b7a640 1 -- 172.21.15.17:0/1869180957 wait complete. 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.958+0000 7ff542b7a640 1 Processor -- start 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.958+0000 7ff542b7a640 1 -- start start 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c06afc0 0x7ff53c1ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff53c0c25c0 0x7ff53c1eb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c150980 con 0x7ff53c0c20b0 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0aa0b0 con 0x7ff53c0c25c0 2024-03-06T07:33:09.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff542b7a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c150b00 con 0x7ff53c06afc0 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42218/0 (socket says 172.21.15.17:42218) 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 -- 172.21.15.17:0/1977243428 learned_addr learned my addr 172.21.15.17:0/1977243428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff540916640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c06afc0 0x7ff53c1ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff541117640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff53c0c25c0 0x7ff53c1eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 -- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c06afc0 msgr2=0x7ff53c1ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c06afc0 0x7ff53c1ef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 -- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff53c0c25c0 msgr2=0x7ff53c1eb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff53c0c25c0 0x7ff53c1eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 -- 172.21.15.17:0/1977243428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0c6330 con 0x7ff53c0c20b0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.959+0000 7ff53bfff640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7ff53000d7d0 tx=0x7ff53000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.960+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000b6c0 con 0x7ff53c0c20b0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.960+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530019070 con 0x7ff53c0c20b0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.960+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c0c6610 con 0x7ff53c0c20b0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.960+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000bac0 con 0x7ff53c0c20b0 2024-03-06T07:33:09.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.960+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c0c6bd0 con 0x7ff53c0c20b0 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.961+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c04f330 con 0x7ff53c0c20b0 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.961+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff53000bc60 con 0x7ff53c0c20b0 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.961+0000 7ff5397fa640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.962+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7ff53008a670 con 0x7ff53c0c20b0 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.962+0000 7ff540916640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:09.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.962+0000 7ff540916640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 0x7ff52806d990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff534036000 tx=0x7ff534030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:09.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:09.966+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff53001c360 con 0x7ff53c0c20b0 2024-03-06T07:33:10.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.147+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff53c151500 con 0x7ff53c0c20b0 2024-03-06T07:33:10.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.148+0000 7ff5397fa640 1 -- 172.21.15.17:0/1977243428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v219) v1 ==== 74+0+13695 (secure 0 0 0) 0x7ff530059560 con 0x7ff53c0c20b0 2024-03-06T07:33:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:10.150 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":219,"fsid":"5fd8a41a-f84b-4e64-ac3e-5a25cd35578f","created":"2024-03-06T07:19:47.076487+0000","modified":"2024-03-06T07:33:07.233130+0000","last_up_change":"2024-03-06T07:19:57.335777+0000","last_in_change":"2024-03-06T07:19:49.329550+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-06T07:19:57.384221+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bdfaea27-aff4-4c67-9a67-1b61c249c50e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6801","nonce":3406119864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6805","nonce":3406119864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6813","nonce":3406119864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":3406119864},{"type":"v1","addr":"172.21.15.17:6809","nonce":3406119864}]},"public_addr":"172.21.15.17:6801/3406119864","cluster_addr":"172.21.15.17:6805/3406119864","heartbeat_back_addr":"172.21.15.17:6813/3406119864","heartbeat_front_addr":"172.21.15.17:6809/3406119864","state":["exists","up"]},{"osd":1,"uuid":"685f40a3-4f8f-4cf6-97a2-824b13a8c8af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6825","nonce":1161291487}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6827","nonce":1161291487}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6831","nonce":1161291487}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":1161291487},{"type":"v1","addr":"172.21.15.17:6829","nonce":1161291487}]},"public_addr":"172.21.15.17:6825/1161291487","cluster_addr":"172.21.15.17:6827/1161291487","heartbeat_back_addr":"172.21.15.17:6831/1161291487","heartbeat_front_addr":"172.21.15.17:6829/1161291487","state":["exists","up"]},{"osd":2,"uuid":"19c31cbd-e62a-420a-8e92-af8cb68e07ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":213,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6804","nonce":1741224553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6808","nonce":1741224553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6815","nonce":1741224553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":1741224553},{"type":"v1","addr":"172.21.15.17:6812","nonce":1741224553}]},"public_addr":"172.21.15.17:6804/1741224553","cluster_addr":"172.21.15.17:6808/1741224553","heartbeat_back_addr":"172.21.15.17:6815/1741224553","heartbeat_front_addr":"172.21.15.17:6812/1741224553","state":["exists","up"]},{"osd":3,"uuid":"476f7afe-1b29-400e-af39-a61efe750209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6817","nonce":3152244895}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6819","nonce":3152244895}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6823","nonce":3152244895}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3152244895},{"type":"v1","addr":"172.21.15.17:6821","nonce":3152244895}]},"public_addr":"172.21.15.17:6817/3152244895","cluster_addr":"172.21.15.17:6819/3152244895","heartbeat_back_addr":"172.21.15.17:6823/3152244895","heartbeat_front_addr":"172.21.15.17:6821/3152244895","state":["exists","up"]},{"osd":4,"uuid":"04f85f0d-7f27-43d7-a6c1-54d2b957e248","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6825","nonce":2660678220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6827","nonce":2660678220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6831","nonce":2660678220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":2660678220},{"type":"v1","addr":"172.21.15.163:6829","nonce":2660678220}]},"public_addr":"172.21.15.163:6825/2660678220","cluster_addr":"172.21.15.163:6827/2660678220","heartbeat_back_addr":"172.21.15.163:6831/2660678220","heartbeat_front_addr":"172.21.15.163:6829/2660678220","state":["exists","up"]},{"osd":5,"uuid":"dd5c8061-c103-41cf-9c03-5446a836d0af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6801","nonce":3569452097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6803","nonce":3569452097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6807","nonce":3569452097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":3569452097},{"type":"v1","addr":"172.21.15.163:6805","nonce":3569452097}]},"public_addr":"172.21.15.163:6801/3569452097","cluster_addr":"172.21.15.163:6803/3569452097","heartbeat_back_addr":"172.21.15.163:6807/3569452097","heartbeat_front_addr":"172.21.15.163:6805/3569452097","state":["exists","up"]},{"osd":6,"uuid":"46156205-4bb9-4cdf-bd12-9aa85d03da13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6809","nonce":1305126403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6811","nonce":1305126403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6815","nonce":1305126403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":1305126403},{"type":"v1","addr":"172.21.15.163:6813","nonce":1305126403}]},"public_addr":"172.21.15.163:6809/1305126403","cluster_addr":"172.21.15.163:6811/1305126403","heartbeat_back_addr":"172.21.15.163:6815/1305126403","heartbeat_front_addr":"172.21.15.163:6813/1305126403","state":["exists","up"]},{"osd":7,"uuid":"05b429af-7fbc-4a72-89c2-b4091bb1cfde","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6817","nonce":145909431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6819","nonce":145909431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6823","nonce":145909431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":145909431},{"type":"v1","addr":"172.21.15.163:6821","nonce":145909431}]},"public_addr":"172.21.15.163:6817/145909431","cluster_addr":"172.21.15.163:6819/145909431","heartbeat_back_addr":"172.21.15.163:6823/145909431","heartbeat_front_addr":"172.21.15.163:6821/145909431","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.504675+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.550967+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.472299+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.533882+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.571958+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.425413+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.491755+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-06T07:19:54.511812+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6834/2434034079":"2024-03-07T07:33:03.934109+0000","172.21.15.17:6832/2434034079":"2024-03-07T07:33:03.934109+0000","172.21.15.17:6835/1774857824":"2024-03-07T07:33:03.934093+0000","172.21.15.17:6833/1774857824":"2024-03-07T07:33:03.934093+0000","172.21.15.163:6835/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6834/3635018115":"2024-03-07T07:30:31.883023+0000","172.21.15.163:6837/370957199":"2024-03-07T07:29:56.878364+0000","172.21.15.163:6836/370957199":"2024-03-07T07:29:56.878364+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-06T07:33:10.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 msgr2=0x7ff52806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 0x7ff52806d990 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff534036000 tx=0x7ff534030000 comp rx=0 tx=0).stop 2024-03-06T07:33:10.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 msgr2=0x7ff53c0c0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:10.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7ff53000d7d0 tx=0x7ff53000dc90 comp rx=0 tx=0).stop 2024-03-06T07:33:10.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 shutdown_connections 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff53c0c25c0 0x7ff53c1eb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff53c0c20b0 0x7ff53c0c0d50 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 --2- 172.21.15.17:0/1977243428 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff53c06afc0 0x7ff53c1ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 >> 172.21.15.17:0/1977243428 conn(0x7ff53c0c76b0 msgr2=0x7ff53c152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 shutdown_connections 2024-03-06T07:33:10.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:10.150+0000 7ff542b7a640 1 -- 172.21.15.17:0/1977243428 wait complete. 2024-03-06T07:33:11.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:11.050 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:11.163 INFO:tasks.ceph:Scrubbing osd.0 2024-03-06T07:33:11.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.267+0000 7f0b43dc7640 1 Processor -- start 2024-03-06T07:33:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.267+0000 7f0b43dc7640 1 -- start start 2024-03-06T07:33:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c14ebb0 0x7f0b3c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b3c0a9fb0 0x7f0b3c06dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c155530 con 0x7f0b3c0a9980 2024-03-06T07:33:11.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c155a60 con 0x7f0b3c0a9fb0 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b43dc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c2e10 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51116/0 (socket says 172.21.15.17:51116) 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 -- 172.21.15.17:0/3760013042 learned_addr learned my addr 172.21.15.17:0/3760013042 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b42364640 1 --2- 172.21.15.17:0/3760013042 >> v2:172.21.15.163:3300/0 conn(0x7f0b3c0a9fb0 0x7f0b3c06dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41b63640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c14ebb0 0x7f0b3c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 -- 172.21.15.17:0/3760013042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c14ebb0 msgr2=0x7f0b3c1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c14ebb0 0x7f0b3c1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 -- 172.21.15.17:0/3760013042 >> v2:172.21.15.163:3300/0 conn(0x7f0b3c0a9fb0 msgr2=0x7f0b3c06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 --2- 172.21.15.17:0/3760013042 >> v2:172.21.15.163:3300/0 conn(0x7f0b3c0a9fb0 0x7f0b3c06dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.268+0000 7f0b41362640 1 -- 172.21.15.17:0/3760013042 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c14ef90 con 0x7f0b3c0a9980 2024-03-06T07:33:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.269+0000 7f0b41362640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0c7610 tx=0x7f0b3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f20e998d70d32cf server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b40b61640 1 -- 172.21.15.17:0/3760013042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3802e070 con 0x7f0b3c0a9980 2024-03-06T07:33:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b40b61640 1 -- 172.21.15.17:0/3760013042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38037070 con 0x7f0b3c0a9980 2024-03-06T07:33:11.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b40b61640 1 -- 172.21.15.17:0/3760013042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38031a00 con 0x7f0b3c0a9980 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 -- 172.21.15.17:0/3760013042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c0a9980 msgr2=0x7f0b3c0c28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0c7610 tx=0x7f0b3802b880 comp rx=0 tx=0).stop 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 -- 172.21.15.17:0/3760013042 shutdown_connections 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/3760013042 >> v2:172.21.15.163:3300/0 conn(0x7f0b3c0a9fb0 0x7f0b3c06dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c0a9980 0x7f0b3c0c28d0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/3760013042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c14ebb0 0x7f0b3c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 -- 172.21.15.17:0/3760013042 >> 172.21.15.17:0/3760013042 conn(0x7f0b3c0c65b0 msgr2=0x7f0b3c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 -- 172.21.15.17:0/3760013042 shutdown_connections 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.270+0000 7f0b43dc7640 1 -- 172.21.15.17:0/3760013042 wait complete. 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.271+0000 7f0b43dc7640 1 Processor -- start 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.271+0000 7f0b43dc7640 1 -- start start 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c0a9980 0x7f0b3c14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b3c0a9fb0 0x7f0b3c0bb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 0x7f0b3c14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c1519b0 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c1516d0 con 0x7f0b3c0a9fb0 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b43dc7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c151c90 con 0x7f0b3c0a9980 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b42364640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 0x7f0b3c14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b42364640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 0x7f0b3c14b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51120/0 (socket says 172.21.15.17:51120) 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b42364640 1 -- 172.21.15.17:0/4226788661 learned_addr learned my addr 172.21.15.17:0/4226788661 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b41362640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b3c0a9fb0 0x7f0b3c0bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.272+0000 7f0b41b63640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c0a9980 0x7f0b3c14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 -- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c0a9980 msgr2=0x7f0b3c14f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b3c0a9980 0x7f0b3c14f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 -- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b3c0a9fb0 msgr2=0x7f0b3c0bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b3c0a9fb0 0x7f0b3c0bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0a8830 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b42364640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 0x7f0b3c14b5f0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e00 tx=0x7f0b340075e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34019070 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34003b90 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34004030 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0a8590 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.273+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c1e8d00 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.274+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b3c04f330 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b34005d10 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b2006b4d0 0x7f0b2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f0b3408a7b0 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f0b20071b50 0x7f0b20073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b3400ee90 con 0x7f0b20071b50 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b3408ac40 con 0x7f0b3c14ebb0 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b41b63640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b2006b4d0 0x7f0b2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.275+0000 7f0b41362640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f0b20071b50 0x7f0b20073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.276+0000 7f0b41362640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f0b20071b50 0x7f0b20073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.276+0000 7f0b41b63640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b2006b4d0 0x7f0b2006d990 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.276+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== osd.0 v2:172.21.15.17:6800/3406119864 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0b3400ee90 con 0x7f0b20071b50 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 --> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b3c0a8bd0 con 0x7f0b20071b50 2024-03-06T07:33:11.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b2e7fc640 1 -- 172.21.15.17:0/4226788661 <== osd.0 v2:172.21.15.17:6800/3406119864 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0b3c0a8bd0 con 0x7f0b20071b50 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:11.294 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f0b20071b50 msgr2=0x7f0b20073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:6800/3406119864,v1:172.21.15.17:6801/3406119864] conn(0x7f0b20071b50 0x7f0b20073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b2006b4d0 msgr2=0x7f0b2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0b2006b4d0 0x7f0b2006d990 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006fd0 tx=0x7f0b30008040 comp rx=0 tx=0).stop 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 msgr2=0x7f0b3c14b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 --2- 172.21.15.17:0/4226788661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b3c14ebb0 0x7f0b3c14b5f0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009e00 tx=0x7f0b340075e0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b42364640 1 -- 172.21.15.17:0/4226788661 reap_dead start 2024-03-06T07:33:11.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 shutdown_connections 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 >> 172.21.15.17:0/4226788661 conn(0x7f0b3c0c65b0 msgr2=0x7f0b3c06d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 shutdown_connections 2024-03-06T07:33:11.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.290+0000 7f0b43dc7640 1 -- 172.21.15.17:0/4226788661 wait complete. 2024-03-06T07:33:11.302 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-06T07:33:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 Processor -- start 2024-03-06T07:33:11.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 -- start start 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ec8155240 0x7f4ec80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec80c8e30 con 0x7f4ec8152900 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec80c9110 con 0x7f4ec80c8950 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.404+0000 7f4ecd3b5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec8152ce0 con 0x7f4ec8155240 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4ec8152900 0x7f4ec80bf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36216/0 (socket says 172.21.15.17:36216) 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec6ffd640 1 -- 172.21.15.17:0/1774698416 learned_addr learned my addr 172.21.15.17:0/1774698416 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec77fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4ec8155240 0x7f4ec80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec77fe640 1 -- 172.21.15.17:0/1774698416 >> v1:172.21.15.163:6789/0 conn(0x7f4ec80c8950 legacy=0x7f4ec80c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec77fe640 1 -- 172.21.15.17:0/1774698416 >> v1:172.21.15.17:6789/0 conn(0x7f4ec8152900 legacy=0x7f4ec80bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec5ffb640 1 -- 172.21.15.17:0/1774698416 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184414304 0 0) 0x7f4ec80c8e30 con 0x7f4ec8152900 2024-03-06T07:33:11.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.405+0000 7f4ec77fe640 1 -- 172.21.15.17:0/1774698416 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80c93f0 con 0x7f4ec8155240 2024-03-06T07:33:11.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ec77fe640 1 --2- 172.21.15.17:0/1774698416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec80c6c00 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7e15a19584ea3b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ec5ffb640 1 -- 172.21.15.17:0/1774698416 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0039070 con 0x7f4ec8155240 2024-03-06T07:33:11.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ec5ffb640 1 -- 172.21.15.17:0/1774698416 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec002cb40 con 0x7f4ec8155240 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ec5ffb640 1 -- 172.21.15.17:0/1774698416 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0034440 con 0x7f4ec8155240 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/1774698416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 msgr2=0x7f4ec80c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/1774698416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec80c6c00 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002bd90 comp rx=0 tx=0).stop 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/1774698416 shutdown_connections 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/1774698416 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec80c6c00 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/1774698416 >> 172.21.15.17:0/1774698416 conn(0x7f4ec80ac860 msgr2=0x7f4ec80acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/1774698416 shutdown_connections 2024-03-06T07:33:11.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.406+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/1774698416 wait complete. 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 Processor -- start 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 -- start start 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 0x7f4ec80c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ec8152900 0x7f4ec80c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80c0000 con 0x7f4ec8155240 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80c7140 con 0x7f4ec80c8950 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ecd3b5640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80c0180 con 0x7f4ec8152900 2024-03-06T07:33:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ec77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ec77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51150/0 (socket says 172.21.15.17:51150) 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.408+0000 7f4ec77fe640 1 -- 172.21.15.17:0/3479454399 learned_addr learned my addr 172.21.15.17:0/3479454399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec6ffd640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 0x7f4ec80c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec67fc640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ec8152900 0x7f4ec80c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 -- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ec8152900 msgr2=0x7f4ec80c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ec8152900 0x7f4ec80c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 -- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 msgr2=0x7f4ec80c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 0x7f4ec80c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 -- 172.21.15.17:0/3479454399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec81eb3a0 con 0x7f4ec8155240 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec6ffd640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 0x7f4ec80c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ec77fe640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0037040 tx=0x7f4ec0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0044030 con 0x7f4ec8155240 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec002c940 con 0x7f4ec8155240 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec814e0f0 con 0x7f4ec8155240 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0039040 con 0x7f4ec8155240 2024-03-06T07:33:11.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.409+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec81eb920 con 0x7f4ec8155240 2024-03-06T07:33:11.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.410+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec8075300 con 0x7f4ec8155240 2024-03-06T07:33:11.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.410+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ec003f470 con 0x7f4ec8155240 2024-03-06T07:33:11.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.411+0000 7f4eaf7fe640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.411+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f4ec00acce0 con 0x7f4ec8155240 2024-03-06T07:33:11.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.411+0000 7f4ec6ffd640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.412+0000 7f4ec6ffd640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 0x7f4eb406d990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000f60 tx=0x7f4eb8007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.415+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4ec8075300 con 0x7f4ec8155240 2024-03-06T07:33:11.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.596+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec804f330 con 0x7f4eb406b4d0 2024-03-06T07:33:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.597+0000 7f4eaf7fe640 1 -- 172.21.15.17:0/3479454399 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ec804f330 con 0x7f4eb406b4d0 2024-03-06T07:33:11.598 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 msgr2=0x7f4eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 0x7f4eb406d990 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8000f60 tx=0x7f4eb8007c10 comp rx=0 tx=0).stop 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 msgr2=0x7f4ec8145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0037040 tx=0x7f4ec0004a40 comp rx=0 tx=0).stop 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 shutdown_connections 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f4eb406b4d0 0x7f4eb406d990 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec8155240 0x7f4ec8145200 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ec8152900 0x7f4ec80c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 --2- 172.21.15.17:0/3479454399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ec80c8950 0x7f4ec80c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 >> 172.21.15.17:0/3479454399 conn(0x7f4ec80ac860 msgr2=0x7f4ec80c55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 shutdown_connections 2024-03-06T07:33:11.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.599+0000 7f4ecd3b5640 1 -- 172.21.15.17:0/3479454399 wait complete. 2024-03-06T07:33:11.611 INFO:tasks.ceph:Scrubbing osd.1 2024-03-06T07:33:11.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:11.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 Processor -- start 2024-03-06T07:33:11.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 -- start start 2024-03-06T07:33:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06240abb60 0x7f06241502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 0x7f0624156440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0624070430 con 0x7f06240c2920 2024-03-06T07:33:11.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06240711f0 con 0x7f06240abb60 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.716+0000 7f0629dbc640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0624070d60 con 0x7f06240c1d10 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06240abb60 0x7f06241502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06240abb60 0x7f06241502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51162/0 (socket says 172.21.15.17:51162) 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 -- 172.21.15.17:0/2504410399 learned_addr learned my addr 172.21.15.17:0/2504410399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f0622ffd640 1 --2- 172.21.15.17:0/2504410399 >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 0x7f0624156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 -- 172.21.15.17:0/2504410399 >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 msgr2=0x7f0624156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 --2- 172.21.15.17:0/2504410399 >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 0x7f0624156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 -- 172.21.15.17:0/2504410399 >> v1:172.21.15.163:6789/0 conn(0x7f06240c2920 legacy=0x7f0624156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f06237fe640 1 -- 172.21.15.17:0/2504410399 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624070710 con 0x7f06240abb60 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.717+0000 7f0622ffd640 1 --2- 172.21.15.17:0/2504410399 >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 0x7f0624156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f06237fe640 1 --2- 172.21.15.17:0/2504410399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06240abb60 0x7f06241502f0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f06240a9690 tx=0x7f061402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9339ea9fb1cfd32a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f06227fc640 1 -- 172.21.15.17:0/2504410399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614038070 con 0x7f06240abb60 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f06227fc640 1 -- 172.21.15.17:0/2504410399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061402f050 con 0x7f06240abb60 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f06227fc640 1 -- 172.21.15.17:0/2504410399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614031970 con 0x7f06240abb60 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 -- 172.21.15.17:0/2504410399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06240abb60 msgr2=0x7f06241502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 --2- 172.21.15.17:0/2504410399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06240abb60 0x7f06241502f0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f06240a9690 tx=0x7f061402b7f0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 -- 172.21.15.17:0/2504410399 shutdown_connections 2024-03-06T07:33:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 --2- 172.21.15.17:0/2504410399 >> v2:172.21.15.163:3300/0 conn(0x7f06240c1d10 0x7f0624156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 --2- 172.21.15.17:0/2504410399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06240abb60 0x7f06241502f0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 -- 172.21.15.17:0/2504410399 >> 172.21.15.17:0/2504410399 conn(0x7f06240c65b0 msgr2=0x7f06240c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 -- 172.21.15.17:0/2504410399 shutdown_connections 2024-03-06T07:33:11.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.718+0000 7f0629dbc640 1 -- 172.21.15.17:0/2504410399 wait complete. 2024-03-06T07:33:11.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.719+0000 7f0629dbc640 1 Processor -- start 2024-03-06T07:33:11.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 -- start start 2024-03-06T07:33:11.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06240abb60 0x7f062414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06240c2920 0x7f06241e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 0x7f0624155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240abf40 con 0x7f06241dfa40 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624157090 con 0x7f06240abb60 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0629dbc640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06240ac0c0 con 0x7f06240c2920 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0623fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 0x7f0624155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0623fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 0x7f0624155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51170/0 (socket says 172.21.15.17:51170) 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0623fff640 1 -- 172.21.15.17:0/238170736 learned_addr learned my addr 172.21.15.17:0/238170736 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f0622ffd640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06240c2920 0x7f06241e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.720+0000 7f06237fe640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06240abb60 0x7f062414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 -- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06240c2920 msgr2=0x7f06241e96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f06240c2920 0x7f06241e96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 -- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06240abb60 msgr2=0x7f062414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f06240abb60 0x7f062414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624152f50 con 0x7f06241dfa40 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0623fff640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 0x7f0624155980 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b970 tx=0x7f061c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c019070 con 0x7f06241dfa40 2024-03-06T07:33:11.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c00cae0 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0624153230 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c014440 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.721+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0624153a60 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.722+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0624153e40 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.722+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f061c01c070 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f060bfff640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f061006b500 0x7f061006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f061c089d30 con 0x7f06241dfa40 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f060bfff640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f0610071b80 0x7f0610073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0610074690 con 0x7f0610071b80 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061c08c430 con 0x7f06241dfa40 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f06237fe640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f061006b500 0x7f061006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f0622ffd640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f0610071b80 0x7f0610073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.723+0000 7f0622ffd640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f0610071b80 0x7f0610073fc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.724+0000 7f06237fe640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f061006b500 0x7f061006d9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0614035000 tx=0x7f0614008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.724+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== osd.1 v2:172.21.15.17:6824/1161291487 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0610074690 con 0x7f0610071b80 2024-03-06T07:33:11.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.737+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 --> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f062404f330 con 0x7f0610071b80 2024-03-06T07:33:11.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.737+0000 7f060bfff640 1 -- 172.21.15.17:0/238170736 <== osd.1 v2:172.21.15.17:6824/1161291487 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f062404f330 con 0x7f0610071b80 2024-03-06T07:33:11.739 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:11.739 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:11.739 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:11.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f0610071b80 msgr2=0x7f0610073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:6824/1161291487,v1:172.21.15.17:6825/1161291487] conn(0x7f0610071b80 0x7f0610073fc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f061006b500 msgr2=0x7f061006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f061006b500 0x7f061006d9c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0614035000 tx=0x7f0614008030 comp rx=0 tx=0).stop 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 msgr2=0x7f0624155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 --2- 172.21.15.17:0/238170736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06241dfa40 0x7f0624155980 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b970 tx=0x7f061c00be30 comp rx=0 tx=0).stop 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0623fff640 1 -- 172.21.15.17:0/238170736 reap_dead start 2024-03-06T07:33:11.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 shutdown_connections 2024-03-06T07:33:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 >> 172.21.15.17:0/238170736 conn(0x7f06240c65b0 msgr2=0x7f06241500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 shutdown_connections 2024-03-06T07:33:11.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.738+0000 7f0629dbc640 1 -- 172.21.15.17:0/238170736 wait complete. 2024-03-06T07:33:11.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-06T07:33:11.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.850+0000 7fc469e81640 1 Processor -- start 2024-03-06T07:33:11.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.850+0000 7fc469e81640 1 -- start start 2024-03-06T07:33:11.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc469e81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc464152900 0x7fc464155240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc469e81640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4640c8950 0x7fc4640bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc469e81640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4640c8e30 con 0x7fc4640bfe30 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc469e81640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc464152ce0 con 0x7fc464152900 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc469e81640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4640c9110 con 0x7fc4640c8950 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4637fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc464152900 0x7fc464155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc463fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc4640bfe30 0x7fc4640c64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36222/0 (socket says 172.21.15.17:36222) 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc463fff640 1 -- 172.21.15.17:0/2922862343 learned_addr learned my addr 172.21.15.17:0/2922862343 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc462ffd640 1 --2- 172.21.15.17:0/2922862343 >> v2:172.21.15.163:3300/0 conn(0x7fc4640c8950 0x7fc4640bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4637fe640 1 -- 172.21.15.17:0/2922862343 >> v2:172.21.15.163:3300/0 conn(0x7fc4640c8950 msgr2=0x7fc4640bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4637fe640 1 --2- 172.21.15.17:0/2922862343 >> v2:172.21.15.163:3300/0 conn(0x7fc4640c8950 0x7fc4640bf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4637fe640 1 -- 172.21.15.17:0/2922862343 >> v1:172.21.15.17:6789/0 conn(0x7fc4640bfe30 legacy=0x7fc4640c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4627fc640 1 -- 172.21.15.17:0/2922862343 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790911072 0 0) 0x7fc4640c8e30 con 0x7fc4640bfe30 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.851+0000 7fc4637fe640 1 -- 172.21.15.17:0/2922862343 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640c93f0 con 0x7fc464152900 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc4637fe640 1 --2- 172.21.15.17:0/2922862343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc464152900 0x7fc464155240 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fc4640c5e90 tx=0x7fc45402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af55e4d6a2f0aa8b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc4627fc640 1 -- 172.21.15.17:0/2922862343 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454038070 con 0x7fc464152900 2024-03-06T07:33:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc4627fc640 1 -- 172.21.15.17:0/2922862343 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454031710 con 0x7fc464152900 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc4627fc640 1 -- 172.21.15.17:0/2922862343 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454031ae0 con 0x7fc464152900 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc469e81640 1 -- 172.21.15.17:0/2922862343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc464152900 msgr2=0x7fc464155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.852+0000 7fc469e81640 1 --2- 172.21.15.17:0/2922862343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc464152900 0x7fc464155240 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7fc4640c5e90 tx=0x7fc45402b7b0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 -- 172.21.15.17:0/2922862343 shutdown_connections 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 --2- 172.21.15.17:0/2922862343 >> v2:172.21.15.163:3300/0 conn(0x7fc4640c8950 0x7fc4640bf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 --2- 172.21.15.17:0/2922862343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc464152900 0x7fc464155240 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 -- 172.21.15.17:0/2922862343 >> 172.21.15.17:0/2922862343 conn(0x7fc4640ac860 msgr2=0x7fc4640acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:11.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 -- 172.21.15.17:0/2922862343 shutdown_connections 2024-03-06T07:33:11.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.853+0000 7fc469e81640 1 -- 172.21.15.17:0/2922862343 wait complete. 2024-03-06T07:33:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.854+0000 7fc469e81640 1 Processor -- start 2024-03-06T07:33:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.854+0000 7fc469e81640 1 -- start start 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.854+0000 7fc469e81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.854+0000 7fc469e81640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4640c8950 0x7fc464149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc464152900 0x7fc46414a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640c5770 con 0x7fc4640bfe30 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640c55f0 con 0x7fc464152900 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640c6c00 con 0x7fc4640c8950 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51188/0 (socket says 172.21.15.17:51188) 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 -- 172.21.15.17:0/2319636437 learned_addr learned my addr 172.21.15.17:0/2319636437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc463fff640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc464152900 0x7fc46414a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc462ffd640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4640c8950 0x7fc464149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 -- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4640c8950 msgr2=0x7fc464149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4640c8950 0x7fc464149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 -- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc464152900 msgr2=0x7fc46414a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc464152900 0x7fc46414a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 -- 172.21.15.17:0/2319636437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4641edf90 con 0x7fc4640bfe30 2024-03-06T07:33:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc4637fe640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fc454009890 tx=0x7fc4540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454038040 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454004230 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45403d670 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4641ee270 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.855+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641ee830 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.856+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46404f330 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.857+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc45403b070 con 0x7fc4640bfe30 2024-03-06T07:33:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.857+0000 7fc44bfff640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.857+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fc4540abbc0 con 0x7fc4640bfe30 2024-03-06T07:33:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.857+0000 7fc462ffd640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.858+0000 7fc462ffd640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc4640c4b20 tx=0x7fc458008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:11.862+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc454042020 con 0x7fc4640bfe30 2024-03-06T07:33:12.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.045+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4640c53d0 con 0x7fc45006b4d0 2024-03-06T07:33:12.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.046+0000 7fc44bfff640 1 -- 172.21.15.17:0/2319636437 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc4640c53d0 con 0x7fc45006b4d0 2024-03-06T07:33:12.047 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 msgr2=0x7fc45006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 0x7fc45006d990 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fc4640c4b20 tx=0x7fc458008040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 msgr2=0x7fc464154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 secure :-1 s=READY pgs=2733 cs=0 l=1 rev1=1 crypto rx=0x7fc454009890 tx=0x7fc4540046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 shutdown_connections 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fc45006b4d0 0x7fc45006d990 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc464152900 0x7fc46414a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4640c8950 0x7fc464149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 --2- 172.21.15.17:0/2319636437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4640bfe30 0x7fc464154370 unknown :-1 s=CLOSED pgs=2733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 >> 172.21.15.17:0/2319636437 conn(0x7fc4640ac860 msgr2=0x7fc4641519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 shutdown_connections 2024-03-06T07:33:12.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.048+0000 7fc469e81640 1 -- 172.21.15.17:0/2319636437 wait complete. 2024-03-06T07:33:12.060 INFO:tasks.ceph:Scrubbing osd.2 2024-03-06T07:33:12.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:12.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.164+0000 7f524e1a0640 1 Processor -- start 2024-03-06T07:33:12.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 -- start start 2024-03-06T07:33:12.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5248155d50 0x7f52480c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 0x7f524806b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52480c5840 con 0x7f52481529a0 2024-03-06T07:33:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524806ce60 con 0x7f5248155d50 2024-03-06T07:33:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.165+0000 7f524e1a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52480c5610 con 0x7f524806b1e0 2024-03-06T07:33:12.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524d19e640 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f52481529a0 0x7f52480c8650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.17:50624/0 (socket says 172.21.15.17:50624) 2024-03-06T07:33:12.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524c99d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5248155d50 0x7f52480c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524d19e640 1 -- 172.21.15.17:0/4109257928 learned_addr learned my addr 172.21.15.17:0/4109257928 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524d99f640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 0x7f524806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f5247fff640 1 -- 172.21.15.17:0/4109257928 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925411256 0 0) 0x7f52480c5840 con 0x7f52481529a0 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f5247fff640 1 -- 172.21.15.17:0/4109257928 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5230003520 con 0x7f52481529a0 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524c99d640 1 -- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 msgr2=0x7f524806b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524c99d640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 0x7f524806b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524c99d640 1 -- 172.21.15.17:0/4109257928 >> v1:172.21.15.163:6789/0 conn(0x7f52481529a0 legacy=0x7f52480c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524c99d640 1 -- 172.21.15.17:0/4109257928 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248155270 con 0x7f5248155d50 2024-03-06T07:33:12.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.166+0000 7f524d99f640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 0x7f524806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524c99d640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f52480c50d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097a0 tx=0x7f523c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b7fa9e1a6ef720 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f5247fff640 1 -- 172.21.15.17:0/4109257928 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c035020 con 0x7f5248155d50 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f5247fff640 1 -- 172.21.15.17:0/4109257928 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c031830 con 0x7f5248155d50 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f5247fff640 1 -- 172.21.15.17:0/4109257928 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c031be0 con 0x7f5248155d50 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 -- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 msgr2=0x7f52480c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f52480c50d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f523c0097a0 tx=0x7f523c02b8d0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 -- 172.21.15.17:0/4109257928 shutdown_connections 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f524806b1e0 0x7f524806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 --2- 172.21.15.17:0/4109257928 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f52480c50d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.167+0000 7f524e1a0640 1 -- 172.21.15.17:0/4109257928 >> 172.21.15.17:0/4109257928 conn(0x7f52480ca750 msgr2=0x7f52480cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.168+0000 7f524e1a0640 1 -- 172.21.15.17:0/4109257928 shutdown_connections 2024-03-06T07:33:12.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.168+0000 7f524e1a0640 1 -- 172.21.15.17:0/4109257928 wait complete. 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 Processor -- start 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 -- start start 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 0x7f52480aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52481529a0 0x7f52481564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f524814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52480c8d60 con 0x7f524806b1e0 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52481558c0 con 0x7f5248155d50 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524e1a0640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52480c5b20 con 0x7f52481529a0 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524d19e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 0x7f52480aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524d19e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 0x7f52480aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51192/0 (socket says 172.21.15.17:51192) 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.169+0000 7f524d19e640 1 -- 172.21.15.17:0/767244606 learned_addr learned my addr 172.21.15.17:0/767244606 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d99f640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f524814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 -- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52481529a0 msgr2=0x7f52481564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52481529a0 0x7f52481564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 -- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 msgr2=0x7f524814b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f524814b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52481ead10 con 0x7f524806b1e0 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d99f640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5248155d50 0x7f524814b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524d19e640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 0x7f52480aa900 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f524000c430 tx=0x7f524000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240017070 con 0x7f524806b1e0 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5240005ce0 con 0x7f524806b1e0 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524814e930 con 0x7f524806b1e0 2024-03-06T07:33:12.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400072f0 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.170+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524814f0d0 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.171+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52480684b0 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f524001a070 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f523406b4d0 0x7f523406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f52400896e0 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f5234071b50 0x7f5234073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5240010e50 con 0x7f5234071b50 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5240090050 con 0x7f524806b1e0 2024-03-06T07:33:12.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.172+0000 7f524c99d640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f523406b4d0 0x7f523406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.173+0000 7f524d99f640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f5234071b50 0x7f5234073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.173+0000 7f524d99f640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f5234071b50 0x7f5234073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.173+0000 7f524c99d640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f523406b4d0 0x7f523406d990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f523c036000 tx=0x7f523c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.173+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== osd.2 v2:172.21.15.17:6802/1741224553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5240010e50 con 0x7f5234071b50 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 --> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f52480693c0 con 0x7f5234071b50 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f5245ffb640 1 -- 172.21.15.17:0/767244606 <== osd.2 v2:172.21.15.17:6802/1741224553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f52480693c0 con 0x7f5234071b50 2024-03-06T07:33:12.188 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:12.189 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f5234071b50 msgr2=0x7f5234073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:6802/1741224553,v1:172.21.15.17:6804/1741224553] conn(0x7f5234071b50 0x7f5234073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f523406b4d0 msgr2=0x7f523406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f523406b4d0 0x7f523406d990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f523c036000 tx=0x7f523c030000 comp rx=0 tx=0).stop 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 msgr2=0x7f52480aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 --2- 172.21.15.17:0/767244606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f524806b1e0 0x7f52480aa900 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f524000c430 tx=0x7f524000c8f0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524d99f640 1 -- 172.21.15.17:0/767244606 reap_dead start 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 shutdown_connections 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 >> 172.21.15.17:0/767244606 conn(0x7f52480ca750 msgr2=0x7f52481e06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 shutdown_connections 2024-03-06T07:33:12.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.187+0000 7f524e1a0640 1 -- 172.21.15.17:0/767244606 wait complete. 2024-03-06T07:33:12.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-06T07:33:12.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.299+0000 7f6661ef4640 1 Processor -- start 2024-03-06T07:33:12.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.299+0000 7f6661ef4640 1 -- start start 2024-03-06T07:33:12.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f6661ef4640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f665c0c35b0 0x7f665c0c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f6661ef4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c06b6d0 con 0x7f665c1526d0 2024-03-06T07:33:12.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f6661ef4640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c06feb0 con 0x7f665c0c25f0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f6661ef4640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c06b960 con 0x7f665c0c35b0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f665c1526d0 0x7f665c06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36228/0 (socket says 172.21.15.17:36228) 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665b7fe640 1 -- 172.21.15.17:0/1528796567 learned_addr learned my addr 172.21.15.17:0/1528796567 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665affd640 1 --2- 172.21.15.17:0/1528796567 >> v2:172.21.15.163:3300/0 conn(0x7f665c0c35b0 0x7f665c0c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485756541 0 0) 0x7f665c06b6d0 con 0x7f665c1526d0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6644003520 con 0x7f665c1526d0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220656353 0 0) 0x7f665c06feb0 con 0x7f665c0c25f0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f665c06b6d0 con 0x7f665c0c25f0 2024-03-06T07:33:12.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243374465 0 0) 0x7f6644003520 con 0x7f665c1526d0 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 >> v2:172.21.15.163:3300/0 conn(0x7f665c0c35b0 msgr2=0x7f665c0c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 --2- 172.21.15.17:0/1528796567 >> v2:172.21.15.163:3300/0 conn(0x7f665c0c35b0 0x7f665c0c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 >> v1:172.21.15.163:6789/0 conn(0x7f665c0c25f0 legacy=0x7f665c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c06f0f0 con 0x7f665c1526d0 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f664c001670 con 0x7f665c1526d0 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665affd640 1 --2- 172.21.15.17:0/1528796567 >> v2:172.21.15.163:3300/0 conn(0x7f665c0c35b0 0x7f665c0c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 >> v1:172.21.15.17:6789/0 conn(0x7f665c1526d0 legacy=0x7f665c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 0x7f6644003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6644004800 0x7f6644004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644005210 0x7f664400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.300+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c06b960 con 0x7f6644003b90 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664400c140 con 0x7f6644004800 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664400c340 con 0x7f6644005210 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 0x7f6644003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665affd640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6644004800 0x7f6644004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665bfff640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644005210 0x7f664400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 -- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644005210 msgr2=0x7f664400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644005210 0x7f664400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 -- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6644004800 msgr2=0x7f6644004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6644004800 0x7f6644004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665b7fe640 1 -- 172.21.15.17:0/1528796567 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644003520 con 0x7f6644003b90 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665bfff640 1 -- 172.21.15.17:0/1528796567 reap_dead start 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.301+0000 7f665affd640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6644004800 0x7f6644004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f665b7fe640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 0x7f6644003f90 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c0e0 tx=0x7f6644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9f40162dd5810c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c007c00 con 0x7f6644003b90 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c030070 con 0x7f6644003b90 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f665a7fc640 1 -- 172.21.15.17:0/1528796567 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c005d10 con 0x7f6644003b90 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 -- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 msgr2=0x7f6644003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 0x7f6644003f90 secure :-1 s=READY pgs=2736 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c0e0 tx=0x7f6644005830 comp rx=0 tx=0).stop 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 -- 172.21.15.17:0/1528796567 shutdown_connections 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 --2- 172.21.15.17:0/1528796567 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6644003b90 0x7f6644003f90 unknown :-1 s=CLOSED pgs=2736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 -- 172.21.15.17:0/1528796567 >> 172.21.15.17:0/1528796567 conn(0x7f665c0c41b0 msgr2=0x7f665c0c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 -- 172.21.15.17:0/1528796567 shutdown_connections 2024-03-06T07:33:12.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.302+0000 7f6661ef4640 1 -- 172.21.15.17:0/1528796567 wait complete. 2024-03-06T07:33:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.303+0000 7f6661ef4640 1 Processor -- start 2024-03-06T07:33:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 -- start start 2024-03-06T07:33:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644003b90 0x7f665c0a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 0x7f665c0a8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c06f990 con 0x7f665c0a8db0 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c06f6b0 con 0x7f665c0a9270 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f6661ef4640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c152ab0 con 0x7f6644003b90 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f665bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f665bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51220/0 (socket says 172.21.15.17:51220) 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f665bfff640 1 -- 172.21.15.17:0/2292406711 learned_addr learned my addr 172.21.15.17:0/2292406711 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f665affd640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 0x7f665c0a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.304+0000 7f665b7fe640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644003b90 0x7f665c0a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 -- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644003b90 msgr2=0x7f665c0a9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644003b90 0x7f665c0a9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 -- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 msgr2=0x7f665c0a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 0x7f665c0a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 -- 172.21.15.17:0/2292406711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c153470 con 0x7f665c0a8db0 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665affd640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 0x7f665c0a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f665bfff640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f665c06e5f0 tx=0x7f665400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654003b80 con 0x7f665c0a8db0 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6654018070 con 0x7f665c0a8db0 2024-03-06T07:33:12.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c153750 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540050d0 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.305+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c153d10 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.306+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c04f330 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.306+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6654005270 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.307+0000 7f6643fff640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.307+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f6654089f20 con 0x7f665c0a8db0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.307+0000 7f665b7fe640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.308+0000 7f665b7fe640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f664c034000 tx=0x7f664c039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.311+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6654058ea0 con 0x7f665c0a8db0 2024-03-06T07:33:12.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.493+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f665c153220 con 0x7f664806b4d0 2024-03-06T07:33:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.493+0000 7f6643fff640 1 -- 172.21.15.17:0/2292406711 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f665c153220 con 0x7f664806b4d0 2024-03-06T07:33:12.495 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2024-03-06T07:33:12.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 msgr2=0x7f664806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 0x7f664806d990 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f664c034000 tx=0x7f664c039040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 msgr2=0x7f665c145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f665c06e5f0 tx=0x7f665400ecd0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 shutdown_connections 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f664806b4d0 0x7f664806d990 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f665c0a8db0 0x7f665c145ee0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f665c0a9270 0x7f665c0a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 --2- 172.21.15.17:0/2292406711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6644003b90 0x7f665c0a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 >> 172.21.15.17:0/2292406711 conn(0x7f665c0c41b0 msgr2=0x7f665c150740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 shutdown_connections 2024-03-06T07:33:12.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.495+0000 7f6661ef4640 1 -- 172.21.15.17:0/2292406711 wait complete. 2024-03-06T07:33:12.507 INFO:tasks.ceph:Scrubbing osd.3 2024-03-06T07:33:12.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 Processor -- start 2024-03-06T07:33:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 -- start start 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2d00abb60 0x7fb2d01502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d00c1d10 0x7fb2d0156440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d0070430 con 0x7fb2d00c2920 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d00711f0 con 0x7fb2d00abb60 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.613+0000 7fb2d80eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0070d60 con 0x7fb2d00c1d10 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d6688640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb2d00c2920 0x7fb2d0156980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36240/0 (socket says 172.21.15.17:36240) 2024-03-06T07:33:12.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d6688640 1 -- 172.21.15.17:0/3670972283 learned_addr learned my addr 172.21.15.17:0/3670972283 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d5e87640 1 --2- 172.21.15.17:0/3670972283 >> v2:172.21.15.163:3300/0 conn(0x7fb2d00abb60 0x7fb2d01502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d5686640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d00c1d10 0x7fb2d0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875063078 0 0) 0x7fb2d0070430 con 0x7fb2d00c2920 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2bc003520 con 0x7fb2d00c2920 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281036155 0 0) 0x7fb2bc003520 con 0x7fb2d00c2920 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d00c1d10 msgr2=0x7fb2d0156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d00c1d10 0x7fb2d0156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 >> v2:172.21.15.163:3300/0 conn(0x7fb2d00abb60 msgr2=0x7fb2d01502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 --2- 172.21.15.17:0/3670972283 >> v2:172.21.15.163:3300/0 conn(0x7fb2d00abb60 0x7fb2d01502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0070710 con 0x7fb2d00c2920 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fb2c80019f0 con 0x7fb2d00c2920 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 >> v1:172.21.15.17:6789/0 conn(0x7fb2d00c2920 legacy=0x7fb2d0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2bc0039b0 0x7fb2bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2bc004800 0x7fb2bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2bc005180 0x7fb2bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0070d60 con 0x7fb2bc005180 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0045b0 con 0x7fb2bc0039b0 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc00bf00 con 0x7fb2bc004800 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d6688640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2bc005180 0x7fb2bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d5686640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2bc004800 0x7fb2bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.614+0000 7fb2d5e87640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2bc0039b0 0x7fb2bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 -- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2bc004800 msgr2=0x7fb2bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2bc004800 0x7fb2bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 -- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2bc0039b0 msgr2=0x7fb2bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2bc0039b0 0x7fb2bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 -- 172.21.15.17:0/3670972283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc003520 con 0x7fb2bc005180 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.615+0000 7fb2d6688640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2bc005180 0x7fb2bc00b9c0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00c2dd0 tx=0x7fb2bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e8081f5f3d3a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800a4c0 con 0x7fb2bc005180 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8034630 con 0x7fb2bc005180 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d4e85640 1 -- 172.21.15.17:0/3670972283 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8034a20 con 0x7fb2bc005180 2024-03-06T07:33:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 -- 172.21.15.17:0/3670972283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2bc005180 msgr2=0x7fb2bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 --2- 172.21.15.17:0/3670972283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2bc005180 0x7fb2bc00b9c0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00c2dd0 tx=0x7fb2bc005830 comp rx=0 tx=0).stop 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d6688640 1 -- 172.21.15.17:0/3670972283 reap_dead start 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 -- 172.21.15.17:0/3670972283 shutdown_connections 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 -- 172.21.15.17:0/3670972283 >> 172.21.15.17:0/3670972283 conn(0x7fb2d00c65b0 msgr2=0x7fb2d00c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 -- 172.21.15.17:0/3670972283 shutdown_connections 2024-03-06T07:33:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.616+0000 7fb2d80eb640 1 -- 172.21.15.17:0/3670972283 wait complete. 2024-03-06T07:33:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.617+0000 7fb2d80eb640 1 Processor -- start 2024-03-06T07:33:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.617+0000 7fb2d80eb640 1 -- start start 2024-03-06T07:33:12.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0146c10 0x7fb2d00c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 0x7fb2d0144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01eb910 0x7fb2d01ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00abf40 con 0x7fb2d01449d0 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0157090 con 0x7fb2d0146c10 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d80eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00ac0c0 con 0x7fb2d01eb910 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 0x7fb2d0144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 0x7fb2d0144e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51230/0 (socket says 172.21.15.17:51230) 2024-03-06T07:33:12.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 -- 172.21.15.17:0/2954968528 learned_addr learned my addr 172.21.15.17:0/2954968528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d6688640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01eb910 0x7fb2d01ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5e87640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0146c10 0x7fb2d00c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 -- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01eb910 msgr2=0x7fb2d01ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01eb910 0x7fb2d01ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 -- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0146c10 msgr2=0x7fb2d00c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0146c10 0x7fb2d00c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00c49c0 con 0x7fb2d01449d0 2024-03-06T07:33:12.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.618+0000 7fb2d5686640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 0x7fb2d0144e50 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00119a0 tx=0x7fb2c0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000f9d0 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c000fb70 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d014b240 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0014440 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d01ec2d0 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.619+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2d014aff0 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.620+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2c0012040 con 0x7fb2d01449d0 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.620+0000 7fb2c67fc640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb2b406b500 0x7fb2b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.620+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fb2c008a680 con 0x7fb2d01449d0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2c67fc640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7fb2b4071b80 0x7fb2b4073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2c0014e50 con 0x7fb2b4071b80 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c0091050 con 0x7fb2d01449d0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2d5e87640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb2b406b500 0x7fb2b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2d6688640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7fb2b4071b80 0x7fb2b4073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2d6688640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7fb2b4071b80 0x7fb2b4073fc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.621+0000 7fb2d5e87640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb2b406b500 0x7fb2b406d9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bc0 tx=0x7fb2cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.622+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== osd.3 v2:172.21.15.17:6816/3152244895 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb2c0014e50 con 0x7fb2b4071b80 2024-03-06T07:33:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 --> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb2d004f330 con 0x7fb2b4071b80 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2c67fc640 1 -- 172.21.15.17:0/2954968528 <== osd.3 v2:172.21.15.17:6816/3152244895 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb2d004f330 con 0x7fb2b4071b80 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:12.637 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7fb2b4071b80 msgr2=0x7fb2b4073fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:6816/3152244895,v1:172.21.15.17:6817/3152244895] conn(0x7fb2b4071b80 0x7fb2b4073fc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb2b406b500 msgr2=0x7fb2b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fb2b406b500 0x7fb2b406d9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bc0 tx=0x7fb2cc009190 comp rx=0 tx=0).stop 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 msgr2=0x7fb2d0144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 --2- 172.21.15.17:0/2954968528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2d01449d0 0x7fb2d0144e50 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00119a0 tx=0x7fb2c0011da0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d6688640 1 -- 172.21.15.17:0/2954968528 reap_dead start 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.635+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 shutdown_connections 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.636+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 >> 172.21.15.17:0/2954968528 conn(0x7fb2d00c65b0 msgr2=0x7fb2d00c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.636+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 shutdown_connections 2024-03-06T07:33:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.636+0000 7fb2d80eb640 1 -- 172.21.15.17:0/2954968528 wait complete. 2024-03-06T07:33:12.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-06T07:33:12.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.748+0000 7f4830264640 1 Processor -- start 2024-03-06T07:33:12.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.748+0000 7f4830264640 1 -- start start 2024-03-06T07:33:12.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f4830264640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f4828160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f4830264640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48280c71e0 con 0x7f4828155120 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f4830264640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4828159590 con 0x7f48280bfe90 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f4830264640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4828158710 con 0x7f48280c6d70 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482d7ff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4828155120 0x7f48281522d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36248/0 (socket says 172.21.15.17:36248) 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482d7ff640 1 -- 172.21.15.17:0/367891150 learned_addr learned my addr 172.21.15.17:0/367891150 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482e801640 1 --2- 172.21.15.17:0/367891150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f4828160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515924117 0 0) 0x7f48280c71e0 con 0x7f4828155120 2024-03-06T07:33:12.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4814003520 con 0x7f4828155120 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.749+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761178458 0 0) 0x7f4828159590 con 0x7f48280bfe90 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.750+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48280c71e0 con 0x7f48280bfe90 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.750+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139752675 0 0) 0x7f4814003520 con 0x7f4828155120 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.750+0000 7f482e801640 1 -- 172.21.15.17:0/367891150 >> v1:172.21.15.163:6789/0 conn(0x7f48280bfe90 legacy=0x7f4828158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.750+0000 7f482e801640 1 -- 172.21.15.17:0/367891150 >> v1:172.21.15.17:6789/0 conn(0x7f4828155120 legacy=0x7f48281522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.750+0000 7f482e801640 1 -- 172.21.15.17:0/367891150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280c59d0 con 0x7f48280c6d70 2024-03-06T07:33:12.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f482e801640 1 --2- 172.21.15.17:0/367891150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f4828160ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4820009dc0 tx=0x7f482002bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d33016f2a7bef57 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820039070 con 0x7f48280c6d70 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482002c880 con 0x7f48280c6d70 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f482cffe640 1 -- 172.21.15.17:0/367891150 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818001670 con 0x7f48280c6d70 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f4830264640 1 -- 172.21.15.17:0/367891150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 msgr2=0x7f4828160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f4830264640 1 --2- 172.21.15.17:0/367891150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f4828160ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4820009dc0 tx=0x7f482002bd50 comp rx=0 tx=0).stop 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f4830264640 1 -- 172.21.15.17:0/367891150 shutdown_connections 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f4830264640 1 --2- 172.21.15.17:0/367891150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f4828160ab0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.751+0000 7f4830264640 1 -- 172.21.15.17:0/367891150 >> 172.21.15.17:0/367891150 conn(0x7f48280acaa0 msgr2=0x7f48280aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.752+0000 7f4830264640 1 -- 172.21.15.17:0/367891150 shutdown_connections 2024-03-06T07:33:12.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.752+0000 7f4830264640 1 -- 172.21.15.17:0/367891150 wait complete. 2024-03-06T07:33:12.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 Processor -- start 2024-03-06T07:33:12.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 -- start start 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f482806c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4828155120 0x7f48280aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4828158960 con 0x7f48280bfe90 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48280c5f90 con 0x7f4828155120 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f4830264640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48280ca700 con 0x7f48280c6d70 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f482e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f482e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51242/0 (socket says 172.21.15.17:51242) 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.753+0000 7f482e000640 1 -- 172.21.15.17:0/3710662503 learned_addr learned my addr 172.21.15.17:0/3710662503 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e801640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4828155120 0x7f48280aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482d7ff640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f482806c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 -- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 msgr2=0x7f482806c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f482806c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 -- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4828155120 msgr2=0x7f48280aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4828155120 0x7f48280aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 -- 172.21.15.17:0/3710662503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48280aa4b0 con 0x7f48280bfe90 2024-03-06T07:33:12.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482d7ff640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f482806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f482e000640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f482400c490 tx=0x7f482400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240052d0 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824005470 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48281eb450 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824012850 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.754+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48281eba10 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.755+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482804f330 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.756+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48240129f0 con 0x7f48280bfe90 2024-03-06T07:33:12.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.756+0000 7f481e7fc640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 0x7f480c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:12.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.756+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f4824089d80 con 0x7f48280bfe90 2024-03-06T07:33:12.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.756+0000 7f482d7ff640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 0x7f480c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:12.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.757+0000 7f482d7ff640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 0x7f480c06d9c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f48180017e0 tx=0x7f4818009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:12.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.760+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4824058d00 con 0x7f48280bfe90 2024-03-06T07:33:12.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.940+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f48280c53d0 con 0x7f480c06b500 2024-03-06T07:33:12.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.940+0000 7f481e7fc640 1 -- 172.21.15.17:0/3710662503 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f48280c53d0 con 0x7f480c06b500 2024-03-06T07:33:12.942 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2024-03-06T07:33:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 msgr2=0x7f480c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 0x7f480c06d9c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f48180017e0 tx=0x7f4818009040 comp rx=0 tx=0).stop 2024-03-06T07:33:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 msgr2=0x7f48281eae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:12.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f482400c490 tx=0x7f482400c950 comp rx=0 tx=0).stop 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 shutdown_connections 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f480c06b500 0x7f480c06d9c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4828155120 0x7f48280aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48280c6d70 0x7f482806c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 --2- 172.21.15.17:0/3710662503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48280bfe90 0x7f48281eae40 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 >> 172.21.15.17:0/3710662503 conn(0x7f48280acaa0 msgr2=0x7f4828152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 shutdown_connections 2024-03-06T07:33:12.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:12.942+0000 7f4830264640 1 -- 172.21.15.17:0/3710662503 wait complete. 2024-03-06T07:33:12.954 INFO:tasks.ceph:Scrubbing osd.4 2024-03-06T07:33:12.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:13.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.058+0000 7f0c2c233640 1 Processor -- start 2024-03-06T07:33:13.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.058+0000 7f0c2c233640 1 -- start start 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c2c233640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c24151e10 0x7f0c24153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c2c233640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2406e850 con 0x7f0c240a9830 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c2c233640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2406d2b0 con 0x7f0c240c22b0 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c2c233640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2406d590 con 0x7f0c24151e10 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c297ce640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0c240a9830 0x7f0c24153390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36256/0 (socket says 172.21.15.17:36256) 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c297ce640 1 -- 172.21.15.17:0/3501574108 learned_addr learned my addr 172.21.15.17:0/3501574108 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:13.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> v2:172.21.15.163:3300/0 conn(0x7f0c24151e10 0x7f0c24153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934211240 0 0) 0x7f0c18001110 con 0x7f0c240a9830 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c0c005730 con 0x7f0c240a9830 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 237033681 0 0) 0x7f0c0c005730 con 0x7f0c240a9830 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 >> v2:172.21.15.163:3300/0 conn(0x7f0c24151e10 msgr2=0x7f0c24153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 --2- 172.21.15.17:0/3501574108 >> v2:172.21.15.163:3300/0 conn(0x7f0c24151e10 0x7f0c24153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 >> v1:172.21.15.163:6789/0 conn(0x7f0c240c22b0 legacy=0x7f0c2414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2406d870 con 0x7f0c240a9830 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.059+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f0c18001840 con 0x7f0c240a9830 2024-03-06T07:33:13.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> v2:172.21.15.163:3300/0 conn(0x7f0c24151e10 0x7f0c24153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 >> v1:172.21.15.17:6789/0 conn(0x7f0c240a9830 legacy=0x7f0c24153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c0c005bc0 0x7f0c0c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0c006a10 0x7f0c0c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c0c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2406d590 con 0x7f0c0c0073d0 2024-03-06T07:33:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c00e320 con 0x7f0c0c005bc0 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c00e520 con 0x7f0c0c006a10 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c0c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c29fcf640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c0c005bc0 0x7f0c0c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c297ce640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0c006a10 0x7f0c0c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 -- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0c006a10 msgr2=0x7f0c0c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c0c006a10 0x7f0c0c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 -- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c0c005bc0 msgr2=0x7f0c0c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c0c005bc0 0x7f0c0c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.060+0000 7f0c2a7d0640 1 -- 172.21.15.17:0/3501574108 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c005730 con 0x7f0c0c0073d0 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c2a7d0640 1 -- 172.21.15.17:0/3501574108 reap_dead start 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c0c00dbd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f0c140073e0 tx=0x7f0c0c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cf9a0d71da3c30 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14039070 con 0x7f0c0c0073d0 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1402cdc0 con 0x7f0c0c0073d0 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c28fcd640 1 -- 172.21.15.17:0/3501574108 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14009c80 con 0x7f0c0c0073d0 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c2c233640 1 -- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 msgr2=0x7f0c0c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.061+0000 7f0c2c233640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c0c00dbd0 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f0c140073e0 tx=0x7f0c0c007a40 comp rx=0 tx=0).stop 2024-03-06T07:33:13.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.062+0000 7f0c2c233640 1 -- 172.21.15.17:0/3501574108 shutdown_connections 2024-03-06T07:33:13.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.062+0000 7f0c2c233640 1 --2- 172.21.15.17:0/3501574108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c0c00dbd0 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.062+0000 7f0c2c233640 1 -- 172.21.15.17:0/3501574108 >> 172.21.15.17:0/3501574108 conn(0x7f0c240c65b0 msgr2=0x7f0c240c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.062+0000 7f0c2c233640 1 -- 172.21.15.17:0/3501574108 shutdown_connections 2024-03-06T07:33:13.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.062+0000 7f0c2c233640 1 -- 172.21.15.17:0/3501574108 wait complete. 2024-03-06T07:33:13.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 Processor -- start 2024-03-06T07:33:13.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 -- start start 2024-03-06T07:33:13.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c240c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2414e620 0x7f0c2414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c2414bc10 0x7f0c2414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240c2810 con 0x7f0c0c0073d0 2024-03-06T07:33:13.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240c2690 con 0x7f0c2414e620 2024-03-06T07:33:13.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c2c233640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2414ef50 con 0x7f0c2414bc10 2024-03-06T07:33:13.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c29fcf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c240c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c29fcf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c240c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51248/0 (socket says 172.21.15.17:51248) 2024-03-06T07:33:13.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.063+0000 7f0c29fcf640 1 -- 172.21.15.17:0/3717393504 learned_addr learned my addr 172.21.15.17:0/3717393504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c297ce640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2414e620 0x7f0c2414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c2414bc10 0x7f0c2414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 -- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c2414bc10 msgr2=0x7f0c2414b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c2414bc10 0x7f0c2414b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 -- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2414e620 msgr2=0x7f0c2414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2414e620 0x7f0c2414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2414c160 con 0x7f0c0c0073d0 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c297ce640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c2414e620 0x7f0c2414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c29fcf640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c240c5440 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00f650 tx=0x7f0c1c00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0050d0 con 0x7f0c0c0073d0 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c018070 con 0x7f0c0c0073d0 2024-03-06T07:33:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c241e7170 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c005490 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.064+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2414c830 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.065+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c2404f330 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.065+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c1c00d730 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c227fc640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c1006b4d0 0x7f0c1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f0c1c089eb0 con 0x7f0c0c0073d0 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c227fc640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7f0c10071b50 0x7f0c10073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c1c007630 con 0x7f0c10071b50 2024-03-06T07:33:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c297ce640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c1006b4d0 0x7f0c1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c1c08c5b0 con 0x7f0c0c0073d0 2024-03-06T07:33:13.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7f0c10071b50 0x7f0c10073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.066+0000 7f0c297ce640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c1006b4d0 0x7f0c1006d990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c2414c380 tx=0x7f0c18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.067+0000 7f0c2a7d0640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7f0c10071b50 0x7f0c10073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.067+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== osd.4 v2:172.21.15.163:6824/2660678220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0c1c007630 con 0x7f0c10071b50 2024-03-06T07:33:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 --> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0c240c1360 con 0x7f0c10071b50 2024-03-06T07:33:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c227fc640 1 -- 172.21.15.17:0/3717393504 <== osd.4 v2:172.21.15.163:6824/2660678220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0c240c1360 con 0x7f0c10071b50 2024-03-06T07:33:13.084 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:13.084 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:13.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7f0c10071b50 msgr2=0x7f0c10073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6824/2660678220,v1:172.21.15.163:6825/2660678220] conn(0x7f0c10071b50 0x7f0c10073f90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c1006b4d0 msgr2=0x7f0c1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f0c1006b4d0 0x7f0c1006d990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0c2414c380 tx=0x7f0c18009040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 msgr2=0x7f0c240c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.081+0000 7f0c2c233640 1 --2- 172.21.15.17:0/3717393504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c0c0073d0 0x7f0c240c5440 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00f650 tx=0x7f0c1c00fb10 comp rx=0 tx=0).stop 2024-03-06T07:33:13.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.082+0000 7f0c2a7d0640 1 -- 172.21.15.17:0/3717393504 reap_dead start 2024-03-06T07:33:13.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.082+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 shutdown_connections 2024-03-06T07:33:13.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.082+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 >> 172.21.15.17:0/3717393504 conn(0x7f0c240c65b0 msgr2=0x7f0c24151380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.082+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 shutdown_connections 2024-03-06T07:33:13.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.082+0000 7f0c2c233640 1 -- 172.21.15.17:0/3717393504 wait complete. 2024-03-06T07:33:13.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-06T07:33:13.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.194+0000 7f5075481640 1 Processor -- start 2024-03-06T07:33:13.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.194+0000 7f5075481640 1 -- start start 2024-03-06T07:33:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.194+0000 7f5075481640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50700bfe90 0x7f5070155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.194+0000 7f5075481640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50700c6d70 0x7f5070158e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f5075481640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700c7200 con 0x7f50701593c0 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f5075481640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5070158710 con 0x7f50700c6d70 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f5075481640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50700c59d0 con 0x7f50700bfe90 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506e7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50700c6d70 0x7f5070158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f50701593c0 0x7f50701522d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36258/0 (socket says 172.21.15.17:36258) 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506f7fe640 1 -- 172.21.15.17:0/3012434493 learned_addr learned my addr 172.21.15.17:0/3012434493 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506effd640 1 --2- 172.21.15.17:0/3012434493 >> v2:172.21.15.163:3300/0 conn(0x7f50700bfe90 0x7f5070155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506e7fc640 1 -- 172.21.15.17:0/3012434493 >> v2:172.21.15.163:3300/0 conn(0x7f50700bfe90 msgr2=0x7f5070155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506e7fc640 1 --2- 172.21.15.17:0/3012434493 >> v2:172.21.15.163:3300/0 conn(0x7f50700bfe90 0x7f5070155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506e7fc640 1 -- 172.21.15.17:0/3012434493 >> v1:172.21.15.17:6789/0 conn(0x7f50701593c0 legacy=0x7f50701522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506dffb640 1 -- 172.21.15.17:0/3012434493 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 788735065 0 0) 0x7f5068001490 con 0x7f50701593c0 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.195+0000 7f506e7fc640 1 -- 172.21.15.17:0/3012434493 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700c5cb0 con 0x7f50700c6d70 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f506e7fc640 1 --2- 172.21.15.17:0/3012434493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c6d70 0x7f5070158e80 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f50700c8530 tx=0x7f506402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1986c6b158a29fa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f506dffb640 1 -- 172.21.15.17:0/3012434493 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640315a0 con 0x7f50700c6d70 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f506dffb640 1 -- 172.21.15.17:0/3012434493 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5064037070 con 0x7f50700c6d70 2024-03-06T07:33:13.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f506dffb640 1 -- 172.21.15.17:0/3012434493 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064031960 con 0x7f50700c6d70 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 -- 172.21.15.17:0/3012434493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c6d70 msgr2=0x7f5070158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 --2- 172.21.15.17:0/3012434493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c6d70 0x7f5070158e80 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f50700c8530 tx=0x7f506402b840 comp rx=0 tx=0).stop 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 -- 172.21.15.17:0/3012434493 shutdown_connections 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 --2- 172.21.15.17:0/3012434493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50700c6d70 0x7f5070158e80 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 --2- 172.21.15.17:0/3012434493 >> v2:172.21.15.163:3300/0 conn(0x7f50700bfe90 0x7f5070155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.196+0000 7f5075481640 1 -- 172.21.15.17:0/3012434493 >> 172.21.15.17:0/3012434493 conn(0x7f50700acaa0 msgr2=0x7f50700aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.197+0000 7f5075481640 1 -- 172.21.15.17:0/3012434493 shutdown_connections 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.197+0000 7f5075481640 1 -- 172.21.15.17:0/3012434493 wait complete. 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 Processor -- start 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 -- start start 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700bfe90 0x7f50700c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c6d70 0x7f50700c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700ca700 con 0x7f5070150d60 2024-03-06T07:33:13.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070158960 con 0x7f50700bfe90 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.198+0000 7f5075481640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700ca880 con 0x7f50700c6d70 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51272/0 (socket says 172.21.15.17:51272) 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 -- 172.21.15.17:0/3500084804 learned_addr learned my addr 172.21.15.17:0/3500084804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506e7fc640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c6d70 0x7f50700c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506effd640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700bfe90 0x7f50700c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 -- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c6d70 msgr2=0x7f50700c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c6d70 0x7f50700c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 -- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700bfe90 msgr2=0x7f50700c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700bfe90 0x7f50700c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 -- 172.21.15.17:0/3500084804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700c3c50 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f506f7fe640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f506800c700 tx=0x7f506800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068018070 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800ce10 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800a860 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507006cfa0 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.199+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50701ebb70 con 0x7f5070150d60 2024-03-06T07:33:13.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.200+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50701540f0 con 0x7f5070150d60 2024-03-06T07:33:13.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.201+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f506800aa00 con 0x7f5070150d60 2024-03-06T07:33:13.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.201+0000 7f50577fe640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 0x7f505c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.201+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f5068089780 con 0x7f5070150d60 2024-03-06T07:33:13.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.201+0000 7f506effd640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 0x7f505c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.202+0000 7f506effd640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 0x7f505c06d990 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f507006cbe0 tx=0x7f5060008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.205+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f50701540f0 con 0x7f5070150d60 2024-03-06T07:33:13.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.384+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f50701ebdb0 con 0x7f505c06b4d0 2024-03-06T07:33:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.385+0000 7f50577fe640 1 -- 172.21.15.17:0/3500084804 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5060004020 con 0x7f505c06b4d0 2024-03-06T07:33:13.386 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2024-03-06T07:33:13.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 msgr2=0x7f505c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 0x7f505c06d990 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f507006cbe0 tx=0x7f5060008040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 msgr2=0x7f50701ee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f506800c700 tx=0x7f506800cbc0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 shutdown_connections 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.386+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f505c06b4d0 0x7f505c06d990 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5070150d60 0x7f50701ee380 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50700c6d70 0x7f50700c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 --2- 172.21.15.17:0/3500084804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50700bfe90 0x7f50700c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 >> 172.21.15.17:0/3500084804 conn(0x7f50700acaa0 msgr2=0x7f50700ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 shutdown_connections 2024-03-06T07:33:13.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.387+0000 7f5075481640 1 -- 172.21.15.17:0/3500084804 wait complete. 2024-03-06T07:33:13.399 INFO:tasks.ceph:Scrubbing osd.5 2024-03-06T07:33:13.399 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:13.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.503+0000 7fab7f647640 1 Processor -- start 2024-03-06T07:33:13.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.503+0000 7fab7f647640 1 -- start start 2024-03-06T07:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7f647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab78155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7f647640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab78156150 con 0x7fab780c6780 2024-03-06T07:33:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7f647640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab780a7d70 con 0x7fab78158780 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7f647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab780a8050 con 0x7fab7806fe20 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7cbe2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fab780c6780 0x7fab78072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36264/0 (socket says 172.21.15.17:36264) 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7cbe2640 1 -- 172.21.15.17:0/3385232859 learned_addr learned my addr 172.21.15.17:0/3385232859 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab7d3e3640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab78155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218709957 0 0) 0x7fab6c001110 con 0x7fab780c6780 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.504+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab60003520 con 0x7fab780c6780 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1910920623 0 0) 0x7fab60003520 con 0x7fab780c6780 2024-03-06T07:33:13.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 msgr2=0x7fab78155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab78155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 >> v1:172.21.15.163:6789/0 conn(0x7fab78158780 legacy=0x7fab78158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780a8330 con 0x7fab780c6780 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7fab6c001840 con 0x7fab780c6780 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 >> v1:172.21.15.17:6789/0 conn(0x7fab780c6780 legacy=0x7fab78072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab60004d40 0x7fab600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab780a8050 con 0x7fab60003f60 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6000bb50 con 0x7fab60004d40 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6000bd50 con 0x7fab7806fe20 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7d3e3640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab60004d40 0x7fab600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 -- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 msgr2=0x7fab600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab7806fe20 0x7fab600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 -- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab60004d40 msgr2=0x7fab600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab60004d40 0x7fab600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7cbe2640 1 -- 172.21.15.17:0/3385232859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab60003520 con 0x7fab60003f60 2024-03-06T07:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.505+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab60004d40 0x7fab600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.506+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab60004800 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c4c0 tx=0x7fab60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f91500bbbfeb6e server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.506+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c00c0e0 con 0x7fab60003f60 2024-03-06T07:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.506+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c030070 con 0x7fab60003f60 2024-03-06T07:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.506+0000 7fab77fff640 1 -- 172.21.15.17:0/3385232859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0058d0 con 0x7fab60003f60 2024-03-06T07:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 -- 172.21.15.17:0/3385232859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 msgr2=0x7fab60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 --2- 172.21.15.17:0/3385232859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab60004800 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c4c0 tx=0x7fab60005830 comp rx=0 tx=0).stop 2024-03-06T07:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7dbe4640 1 -- 172.21.15.17:0/3385232859 reap_dead start 2024-03-06T07:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 -- 172.21.15.17:0/3385232859 shutdown_connections 2024-03-06T07:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 -- 172.21.15.17:0/3385232859 >> 172.21.15.17:0/3385232859 conn(0x7fab780aa630 msgr2=0x7fab780aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 -- 172.21.15.17:0/3385232859 shutdown_connections 2024-03-06T07:33:13.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.507+0000 7fab7f647640 1 -- 172.21.15.17:0/3385232859 wait complete. 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 Processor -- start 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 -- start start 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab780c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab780ad350 0x7fab78154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780ad7d0 0x7fab78154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78070200 con 0x7fab60003f60 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab780c3600 con 0x7fab780ad350 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.508+0000 7fab7f647640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab78072c20 con 0x7fab780ad7d0 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab780c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab780c24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51292/0 (socket says 172.21.15.17:51292) 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 -- 172.21.15.17:0/1280494163 learned_addr learned my addr 172.21.15.17:0/1280494163 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab780ad350 0x7fab78154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780ad7d0 0x7fab78154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 -- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780ad7d0 msgr2=0x7fab78154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780ad7d0 0x7fab78154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 -- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab780ad350 msgr2=0x7fab78154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab780ad350 0x7fab78154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab78154ed0 con 0x7fab60003f60 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab780ad7d0 0x7fab78154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7d3e3640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab780c24c0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fab68011e30 tx=0x7fab6800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68015430 con 0x7fab60003f60 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab680155d0 con 0x7fab60003f60 2024-03-06T07:33:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab781eb2b0 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680159a0 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.509+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab781eb840 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.510+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab781ebf00 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.511+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab68015b40 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.511+0000 7fab75ffb640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab6406b4d0 0x7fab6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.511+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fab6808b1e0 con 0x7fab60003f60 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab75ffb640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7fab64071b50 0x7fab64073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab64074660 con 0x7fab64071b50 2024-03-06T07:33:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab68092050 con 0x7fab60003f60 2024-03-06T07:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab6406b4d0 0x7fab6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7fab64071b50 0x7fab64073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab7cbe2640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab6406b4d0 0x7fab6406d990 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c4c0 tx=0x7fab6c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.512+0000 7fab7dbe4640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7fab64071b50 0x7fab64073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.513+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== osd.5 v2:172.21.15.163:6800/3569452097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab64074660 con 0x7fab64071b50 2024-03-06T07:33:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.526+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 --> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab780c3140 con 0x7fab64071b50 2024-03-06T07:33:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab75ffb640 1 -- 172.21.15.17:0/1280494163 <== osd.5 v2:172.21.15.163:6800/3569452097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fab780c3140 con 0x7fab64071b50 2024-03-06T07:33:13.529 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:13.530 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:13.530 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7fab64071b50 msgr2=0x7fab64073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6800/3569452097,v1:172.21.15.163:6801/3569452097] conn(0x7fab64071b50 0x7fab64073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab6406b4d0 msgr2=0x7fab6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fab6406b4d0 0x7fab6406d990 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00c4c0 tx=0x7fab6c007000 comp rx=0 tx=0).stop 2024-03-06T07:33:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 msgr2=0x7fab780c24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 --2- 172.21.15.17:0/1280494163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab60003f60 0x7fab780c24c0 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7fab68011e30 tx=0x7fab6800f580 comp rx=0 tx=0).stop 2024-03-06T07:33:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7dbe4640 1 -- 172.21.15.17:0/1280494163 reap_dead start 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 shutdown_connections 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 >> 172.21.15.17:0/1280494163 conn(0x7fab780aa630 msgr2=0x7fab7806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 shutdown_connections 2024-03-06T07:33:13.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.527+0000 7fab7f647640 1 -- 172.21.15.17:0/1280494163 wait complete. 2024-03-06T07:33:13.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-06T07:33:13.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.640+0000 7fbd63ff7640 1 Processor -- start 2024-03-06T07:33:13.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.640+0000 7fbd63ff7640 1 -- start start 2024-03-06T07:33:13.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd63ff7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd63ff7640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1587e0 0x7fbd5c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd63ff7640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0c5140 con 0x7fbd5c1526c0 2024-03-06T07:33:13.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd63ff7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c3790 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd63ff7640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c155bb0 con 0x7fbd5c1587e0 2024-03-06T07:33:13.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51302/0 (socket says 172.21.15.17:51302) 2024-03-06T07:33:13.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 -- 172.21.15.17:0/1319681680 learned_addr learned my addr 172.21.15.17:0/1319681680 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd62594640 1 --2- 172.21.15.17:0/1319681680 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1587e0 0x7fbd5c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 -- 172.21.15.17:0/1319681680 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1587e0 msgr2=0x7fbd5c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 --2- 172.21.15.17:0/1319681680 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1587e0 0x7fbd5c158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 -- 172.21.15.17:0/1319681680 >> v1:172.21.15.163:6789/0 conn(0x7fbd5c1526c0 legacy=0x7fbd5c0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.641+0000 7fbd61592640 1 -- 172.21.15.17:0/1319681680 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c155e90 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.642+0000 7fbd61592640 1 --2- 172.21.15.17:0/1319681680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0a7df0 tx=0x7fbd4c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1db228dec09189d server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.642+0000 7fbd60d91640 1 -- 172.21.15.17:0/1319681680 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c035020 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.642+0000 7fbd60d91640 1 -- 172.21.15.17:0/1319681680 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c031870 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.642+0000 7fbd60d91640 1 -- 172.21.15.17:0/1319681680 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c031c40 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 -- 172.21.15.17:0/1319681680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c0c2b20 msgr2=0x7fbd5c0c69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/1319681680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0a7df0 tx=0x7fbd4c02b910 comp rx=0 tx=0).stop 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 -- 172.21.15.17:0/1319681680 shutdown_connections 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/1319681680 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1587e0 0x7fbd5c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/1319681680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c0c69b0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 -- 172.21.15.17:0/1319681680 >> 172.21.15.17:0/1319681680 conn(0x7fbd5c0aa810 msgr2=0x7fbd5c0aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 -- 172.21.15.17:0/1319681680 shutdown_connections 2024-03-06T07:33:13.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.643+0000 7fbd63ff7640 1 -- 172.21.15.17:0/1319681680 wait complete. 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 Processor -- start 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 -- start start 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1526c0 0x7fbd5c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c9040 con 0x7fbd5c1587e0 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c152aa0 con 0x7fbd5c0c2b20 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.644+0000 7fbd63ff7640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c91c0 con 0x7fbd5c1526c0 2024-03-06T07:33:13.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51312/0 (socket says 172.21.15.17:51312) 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 -- 172.21.15.17:0/929448602 learned_addr learned my addr 172.21.15.17:0/929448602 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 -- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1526c0 msgr2=0x7fbd5c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1526c0 0x7fbd5c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 -- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0c2b20 msgr2=0x7fbd5c06d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 -- 172.21.15.17:0/929448602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c1ea6a0 con 0x7fbd5c1587e0 2024-03-06T07:33:13.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd62594640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007e90 tx=0x7fbd5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54014420 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54019070 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c1ea940 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54014800 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.645+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c1eaf00 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.646+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c04f330 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.647+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd540149a0 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.647+0000 7fbd527fc640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.647+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fbd5408a960 con 0x7fbd5c1587e0 2024-03-06T07:33:13.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.647+0000 7fbd61d93640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.648+0000 7fbd61d93640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 0x7fbd4006d990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f60 tx=0x7fbd580079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.651+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd540598e0 con 0x7fbd5c1587e0 2024-03-06T07:33:13.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.830+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd5c155620 con 0x7fbd4006b4d0 2024-03-06T07:33:13.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.831+0000 7fbd527fc640 1 -- 172.21.15.17:0/929448602 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd5c155620 con 0x7fbd4006b4d0 2024-03-06T07:33:13.832 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2024-03-06T07:33:13.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.832+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 msgr2=0x7fbd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.832+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 0x7fbd4006d990 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000f60 tx=0x7fbd580079d0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 msgr2=0x7fbd5c151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7fbd54007e90 tx=0x7fbd5400e4b0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 shutdown_connections 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd5c1587e0 0x7fbd5c151560 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1526c0 0x7fbd5c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 --2- 172.21.15.17:0/929448602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0c2b20 0x7fbd5c06d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 >> 172.21.15.17:0/929448602 conn(0x7fbd5c0aa810 msgr2=0x7fbd5c158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 shutdown_connections 2024-03-06T07:33:13.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.833+0000 7fbd63ff7640 1 -- 172.21.15.17:0/929448602 wait complete. 2024-03-06T07:33:13.845 INFO:tasks.ceph:Scrubbing osd.6 2024-03-06T07:33:13.845 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:13.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.948+0000 7f968b5e2640 1 Processor -- start 2024-03-06T07:33:13.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.948+0000 7f968b5e2640 1 -- start start 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968b5e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f968414ebb0 0x7f96841521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968b5e2640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96840a9980 0x7f96840c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968b5e2640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968414ef90 con 0x7f96840a9fb0 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968b5e2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9684155530 con 0x7f968414ebb0 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968b5e2640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9684155a60 con 0x7f96840a9980 2024-03-06T07:33:13.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f968414ebb0 0x7f96841521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f968414ebb0 0x7f96841521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51320/0 (socket says 172.21.15.17:51320) 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 -- 172.21.15.17:0/3084004423 learned_addr learned my addr 172.21.15.17:0/3084004423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 -- 172.21.15.17:0/3084004423 >> v2:172.21.15.163:3300/0 conn(0x7f96840a9980 msgr2=0x7f96840c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 --2- 172.21.15.17:0/3084004423 >> v2:172.21.15.163:3300/0 conn(0x7f96840a9980 0x7f96840c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 -- 172.21.15.17:0/3084004423 >> v1:172.21.15.163:6789/0 conn(0x7f96840a9fb0 legacy=0x7f968406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:13.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.949+0000 7f968937e640 1 -- 172.21.15.17:0/3084004423 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684155780 con 0x7f968414ebb0 2024-03-06T07:33:13.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.950+0000 7f968937e640 1 --2- 172.21.15.17:0/3084004423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f968414ebb0 0x7f96841521b0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f96840c74a0 tx=0x7f967c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee8e83d3e2e1fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.950+0000 7f9683fff640 1 -- 172.21.15.17:0/3084004423 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c02e070 con 0x7f968414ebb0 2024-03-06T07:33:13.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.950+0000 7f9683fff640 1 -- 172.21.15.17:0/3084004423 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c037070 con 0x7f968414ebb0 2024-03-06T07:33:13.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.950+0000 7f9683fff640 1 -- 172.21.15.17:0/3084004423 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c031a00 con 0x7f968414ebb0 2024-03-06T07:33:13.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 -- 172.21.15.17:0/3084004423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f968414ebb0 msgr2=0x7f96841521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 --2- 172.21.15.17:0/3084004423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f968414ebb0 0x7f96841521b0 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f96840c74a0 tx=0x7f967c02b880 comp rx=0 tx=0).stop 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 -- 172.21.15.17:0/3084004423 shutdown_connections 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 --2- 172.21.15.17:0/3084004423 >> v2:172.21.15.163:3300/0 conn(0x7f96840a9980 0x7f96840c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 --2- 172.21.15.17:0/3084004423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f968414ebb0 0x7f96841521b0 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 -- 172.21.15.17:0/3084004423 >> 172.21.15.17:0/3084004423 conn(0x7f96840c65b0 msgr2=0x7f96840c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 -- 172.21.15.17:0/3084004423 shutdown_connections 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.951+0000 7f968b5e2640 1 -- 172.21.15.17:0/3084004423 wait complete. 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.952+0000 7f968b5e2640 1 Processor -- start 2024-03-06T07:33:13.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.952+0000 7f968b5e2640 1 -- start start 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 0x7f9684149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96840a9fb0 0x7f96841defa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f968414ebb0 0x7f96841edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96841519b0 con 0x7f96840a9980 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c2e10 con 0x7f96840a9fb0 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684151c90 con 0x7f968414ebb0 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 0x7f9684149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 0x7f9684149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51322/0 (socket says 172.21.15.17:51322) 2024-03-06T07:33:13.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 -- 172.21.15.17:0/432814666 learned_addr learned my addr 172.21.15.17:0/432814666 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f9688b7d640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96840a9fb0 0x7f96841defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f9689b7f640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f968414ebb0 0x7f96841edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 -- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f968414ebb0 msgr2=0x7f96841edec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f968414ebb0 0x7f96841edec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 -- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96840a9fb0 msgr2=0x7f96841defa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96840a9fb0 0x7f96841defa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840c5980 con 0x7f96840a9980 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968937e640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 0x7f9684149e20 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f967c02b8c0 tx=0x7f967c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c032880 con 0x7f96840a9980 2024-03-06T07:33:13.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c032a20 con 0x7f96840a9980 2024-03-06T07:33:13.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968414a360 con 0x7f96840a9980 2024-03-06T07:33:13.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.953+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c032e10 con 0x7f96840a9980 2024-03-06T07:33:13.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.954+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96841eaad0 con 0x7f96840a9980 2024-03-06T07:33:13.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.954+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96841eb190 con 0x7f96840a9980 2024-03-06T07:33:13.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.955+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f967c0453f0 con 0x7f96840a9980 2024-03-06T07:33:13.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.955+0000 7f9681ffb640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f967006b4d0 0x7f967006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.955+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f967c0ac270 con 0x7f96840a9980 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9681ffb640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7f9670071b50 0x7f9670073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9688b7d640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f967006b4d0 0x7f967006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f967c043ec0 con 0x7f9670071b50 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967c0b3050 con 0x7f96840a9980 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9689b7f640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7f9670071b50 0x7f9670073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9688b7d640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f967006b4d0 0x7f967006d990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f96841e8770 tx=0x7f9678008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.956+0000 7f9689b7f640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7f9670071b50 0x7f9670073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:13.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.957+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== osd.6 v2:172.21.15.163:6808/1305126403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f967c043ec0 con 0x7f9670071b50 2024-03-06T07:33:13.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.970+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 --> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f968404f330 con 0x7f9670071b50 2024-03-06T07:33:13.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f9681ffb640 1 -- 172.21.15.17:0/432814666 <== osd.6 v2:172.21.15.163:6808/1305126403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f968404f330 con 0x7f9670071b50 2024-03-06T07:33:13.972 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:13.973 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:13.973 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7f9670071b50 msgr2=0x7f9670073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6808/1305126403,v1:172.21.15.163:6809/1305126403] conn(0x7f9670071b50 0x7f9670073f90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f967006b4d0 msgr2=0x7f967006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f967006b4d0 0x7f967006d990 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f96841e8770 tx=0x7f9678008040 comp rx=0 tx=0).stop 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 msgr2=0x7f9684149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 --2- 172.21.15.17:0/432814666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96840a9980 0x7f9684149e20 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f967c02b8c0 tx=0x7f967c0046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:13.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f9689b7f640 1 -- 172.21.15.17:0/432814666 reap_dead start 2024-03-06T07:33:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 shutdown_connections 2024-03-06T07:33:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 >> 172.21.15.17:0/432814666 conn(0x7f96840c65b0 msgr2=0x7f968414cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 shutdown_connections 2024-03-06T07:33:13.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:13.971+0000 7f968b5e2640 1 -- 172.21.15.17:0/432814666 wait complete. 2024-03-06T07:33:13.983 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-06T07:33:14.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.083+0000 7ff6f7d1e640 1 Processor -- start 2024-03-06T07:33:14.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.083+0000 7ff6f7d1e640 1 -- start start 2024-03-06T07:33:14.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f7d1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6f0160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f7d1e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f0155130 con 0x7ff6f00c8950 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f7d1e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f00ad480 con 0x7ff6f00c69f0 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f7d1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f0155440 con 0x7ff6f00c6dd0 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f52b9640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff6f00c8950 0x7ff6f01558d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36270/0 (socket says 172.21.15.17:36270) 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.084+0000 7ff6f52b9640 1 -- 172.21.15.17:0/952542853 learned_addr learned my addr 172.21.15.17:0/952542853 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6f0160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184854636 0 0) 0x7ff6e0001110 con 0x7ff6f00c8950 2024-03-06T07:33:14.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6dc003520 con 0x7ff6f00c8950 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119412553 0 0) 0x7ff6dc003520 con 0x7ff6f00c8950 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 msgr2=0x7ff6f0160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6f0160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 >> v1:172.21.15.163:6789/0 conn(0x7ff6f00c69f0 legacy=0x7ff6f00acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00c5d50 con 0x7ff6f00c8950 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7ff6e0001840 con 0x7ff6f00c8950 2024-03-06T07:33:14.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 >> v1:172.21.15.17:6789/0 conn(0x7ff6f00c8950 legacy=0x7ff6f01558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6dc003f20 0x7ff6dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc004d40 0x7ff6dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f0155440 con 0x7ff6dc004d40 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00bad0 con 0x7ff6dc003f20 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00bcd0 con 0x7ff6f00c6dd0 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc004d40 0x7ff6dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f52b9640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6dc003f20 0x7ff6dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f5aba640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 -- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 msgr2=0x7ff6dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6f00c6dd0 0x7ff6dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 -- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6dc003f20 msgr2=0x7ff6dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6dc003f20 0x7ff6dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.085+0000 7ff6f62bb640 1 -- 172.21.15.17:0/952542853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc003520 con 0x7ff6dc004d40 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.086+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc004d40 0x7ff6dc0051c0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009fc0 tx=0x7ff6dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285f4739ecfdf17a server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.086+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8039070 con 0x7ff6dc004d40 2024-03-06T07:33:14.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.086+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e802cc20 con 0x7ff6dc004d40 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.086+0000 7ff6f4ab8640 1 -- 172.21.15.17:0/952542853 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8009c10 con 0x7ff6dc004d40 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/952542853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc004d40 msgr2=0x7ff6dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/952542853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6dc004d40 0x7ff6dc0051c0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009fc0 tx=0x7ff6dc005830 comp rx=0 tx=0).stop 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f62bb640 1 -- 172.21.15.17:0/952542853 reap_dead start 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/952542853 shutdown_connections 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/952542853 >> 172.21.15.17:0/952542853 conn(0x7ff6f00cae70 msgr2=0x7ff6f00cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/952542853 shutdown_connections 2024-03-06T07:33:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.087+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/952542853 wait complete. 2024-03-06T07:33:14.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.088+0000 7ff6f7d1e640 1 Processor -- start 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.088+0000 7ff6f7d1e640 1 -- start start 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6dc004d40 0x7ff6f00c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 0x7ff6f01e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00c6310 con 0x7ff6f01eb030 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00c7240 con 0x7ff6f00c1a90 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f7d1e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f0155fe0 con 0x7ff6dc004d40 2024-03-06T07:33:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51338/0 (socket says 172.21.15.17:51338) 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 -- 172.21.15.17:0/242477932 learned_addr learned my addr 172.21.15.17:0/242477932 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f5aba640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6dc004d40 0x7ff6f00c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f52b9640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 0x7ff6f01e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 -- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6dc004d40 msgr2=0x7ff6f00c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6dc004d40 0x7ff6f00c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 -- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 msgr2=0x7ff6f01e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 0x7ff6f01e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 -- 172.21.15.17:0/242477932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f01e8390 con 0x7ff6f01eb030 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f52b9640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 0x7ff6f01e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.089+0000 7ff6f62bb640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e230 tx=0x7ff6e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8045070 con 0x7ff6f01eb030 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e803a040 con 0x7ff6f01eb030 2024-03-06T07:33:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8039070 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f00c4740 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f014b450 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.090+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f004f330 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.091+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6e8003bb0 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.091+0000 7ff6e67fc640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 0x7ff6d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.092+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7ff6e80abc20 con 0x7ff6f01eb030 2024-03-06T07:33:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.092+0000 7ff6f5aba640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 0x7ff6d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.092+0000 7ff6f5aba640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 0x7ff6d406d9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec003640 tx=0x7ff6ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.096+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff6e807a4c0 con 0x7ff6f01eb030 2024-03-06T07:33:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.275+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f00c37b0 con 0x7ff6d406b500 2024-03-06T07:33:14.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.276+0000 7ff6e67fc640 1 -- 172.21.15.17:0/242477932 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff6f00c37b0 con 0x7ff6d406b500 2024-03-06T07:33:14.278 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 6 to deep-scrub 2024-03-06T07:33:14.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.277+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 msgr2=0x7ff6d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.277+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 0x7ff6d406d9c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec003640 tx=0x7ff6ec009040 comp rx=0 tx=0).stop 2024-03-06T07:33:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 msgr2=0x7ff6f0153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e230 tx=0x7ff6e8009750 comp rx=0 tx=0).stop 2024-03-06T07:33:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 shutdown_connections 2024-03-06T07:33:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7ff6d406b500 0x7ff6d406d9c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6f01eb030 0x7ff6f0153ff0 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6f00c1a90 0x7ff6f01e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 --2- 172.21.15.17:0/242477932 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6dc004d40 0x7ff6f00c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 >> 172.21.15.17:0/242477932 conn(0x7ff6f00cae70 msgr2=0x7ff6f014fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 shutdown_connections 2024-03-06T07:33:14.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.278+0000 7ff6f7d1e640 1 -- 172.21.15.17:0/242477932 wait complete. 2024-03-06T07:33:14.290 INFO:tasks.ceph:Scrubbing osd.7 2024-03-06T07:33:14.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-06T07:33:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.394+0000 7fef3593a640 1 Processor -- start 2024-03-06T07:33:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.394+0000 7fef3593a640 1 -- start start 2024-03-06T07:33:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef3593a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 0x7fef30158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef3593a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef300a7d70 con 0x7fef300c6780 2024-03-06T07:33:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef3593a640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef300a8050 con 0x7fef3006fe20 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef3593a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300a8330 con 0x7fef30158780 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef27fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fef300c6780 0x7fef30072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36278/0 (socket says 172.21.15.17:36278) 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef27fff640 1 -- 172.21.15.17:0/1404898866 learned_addr learned my addr 172.21.15.17:0/1404898866 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/1404898866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 0x7fef30158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127005130 0 0) 0x7fef300a7d70 con 0x7fef300c6780 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef1c003520 con 0x7fef300c6780 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275856093 0 0) 0x7fef300a8050 con 0x7fef3006fe20 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef300a7d70 con 0x7fef3006fe20 2024-03-06T07:33:14.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2f7fe640 1 -- 172.21.15.17:0/1404898866 >> v1:172.21.15.163:6789/0 conn(0x7fef3006fe20 legacy=0x7fef30155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2f7fe640 1 -- 172.21.15.17:0/1404898866 >> v1:172.21.15.17:6789/0 conn(0x7fef300c6780 legacy=0x7fef30072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.395+0000 7fef2f7fe640 1 -- 172.21.15.17:0/1404898866 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300c3320 con 0x7fef30158780 2024-03-06T07:33:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.396+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/1404898866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 0x7fef30158bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef28009dc0 tx=0x7fef2802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8951fcc0a60be98b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.396+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28039070 con 0x7fef30158780 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.396+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2802cc40 con 0x7fef30158780 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.396+0000 7fef2e7fc640 1 -- 172.21.15.17:0/1404898866 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28034440 con 0x7fef30158780 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 -- 172.21.15.17:0/1404898866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 msgr2=0x7fef30158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 --2- 172.21.15.17:0/1404898866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 0x7fef30158bd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef28009dc0 tx=0x7fef2802be90 comp rx=0 tx=0).stop 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 -- 172.21.15.17:0/1404898866 shutdown_connections 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 --2- 172.21.15.17:0/1404898866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef30158780 0x7fef30158bd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 -- 172.21.15.17:0/1404898866 >> 172.21.15.17:0/1404898866 conn(0x7fef300aa630 msgr2=0x7fef300aaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 -- 172.21.15.17:0/1404898866 shutdown_connections 2024-03-06T07:33:14.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.397+0000 7fef3593a640 1 -- 172.21.15.17:0/1404898866 wait complete. 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.398+0000 7fef3593a640 1 Processor -- start 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.398+0000 7fef3593a640 1 -- start start 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 0x7fef301542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef300c6780 0x7fef30150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef30158780 0x7fef30151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30072c20 con 0x7fef3006fe20 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30070200 con 0x7fef30158780 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef30072da0 con 0x7fef300c6780 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 0x7fef301542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 0x7fef301542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51348/0 (socket says 172.21.15.17:51348) 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 -- 172.21.15.17:0/2160646772 learned_addr learned my addr 172.21.15.17:0/2160646772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef27fff640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef300c6780 0x7fef30150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef30158780 0x7fef30151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 -- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef300c6780 msgr2=0x7fef30150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef300c6780 0x7fef30150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 -- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef30158780 msgr2=0x7fef30151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef30158780 0x7fef30151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300ae390 con 0x7fef3006fe20 2024-03-06T07:33:14.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef30158780 0x7fef30151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:14.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2effd640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 0x7fef301542c0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fef2000c530 tx=0x7fef2000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef200052d0 con 0x7fef3006fe20 2024-03-06T07:33:14.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef20005490 con 0x7fef3006fe20 2024-03-06T07:33:14.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef300ae630 con 0x7fef3006fe20 2024-03-06T07:33:14.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20012870 con 0x7fef3006fe20 2024-03-06T07:33:14.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.399+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef301517d0 con 0x7fef3006fe20 2024-03-06T07:33:14.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.400+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef3004f330 con 0x7fef3006fe20 2024-03-06T07:33:14.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef20012a10 con 0x7fef3006fe20 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef1006b4d0 0x7fef1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7fef20089e90 con 0x7fef3006fe20 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7fef10071b50 0x7fef10073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef10074660 con 0x7fef10071b50 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.401+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=219) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef2008f460 con 0x7fef3006fe20 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.402+0000 7fef27fff640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef1006b4d0 0x7fef1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.402+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7fef10071b50 0x7fef10073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.402+0000 7fef27fff640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef1006b4d0 0x7fef1006d990 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fef301ef010 tx=0x7fef18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.402+0000 7fef2f7fe640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7fef10071b50 0x7fef10073f90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.403+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== osd.7 v2:172.21.15.163:6816/145909431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fef10074660 con 0x7fef10071b50 2024-03-06T07:33:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.416+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 --> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fef30151bf0 con 0x7fef10071b50 2024-03-06T07:33:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.416+0000 7fef2cff9640 1 -- 172.21.15.17:0/2160646772 <== osd.7 v2:172.21.15.163:6816/145909431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fef30151bf0 con 0x7fef10071b50 2024-03-06T07:33:14.418 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-06T07:33:14.419 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-06T07:33:14.419 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7fef10071b50 msgr2=0x7fef10073f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6816/145909431,v1:172.21.15.163:6817/145909431] conn(0x7fef10071b50 0x7fef10073f90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef1006b4d0 msgr2=0x7fef1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7fef1006b4d0 0x7fef1006d990 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fef301ef010 tx=0x7fef18009040 comp rx=0 tx=0).stop 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 msgr2=0x7fef301542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 --2- 172.21.15.17:0/2160646772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fef3006fe20 0x7fef301542c0 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7fef2000c530 tx=0x7fef2000c9f0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef2f7fe640 1 -- 172.21.15.17:0/2160646772 reap_dead start 2024-03-06T07:33:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 shutdown_connections 2024-03-06T07:33:14.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 >> 172.21.15.17:0/2160646772 conn(0x7fef300aa630 msgr2=0x7fef3006e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 shutdown_connections 2024-03-06T07:33:14.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.417+0000 7fef3593a640 1 -- 172.21.15.17:0/2160646772 wait complete. 2024-03-06T07:33:14.429 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-06T07:33:14.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 Processor -- start 2024-03-06T07:33:14.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 -- start start 2024-03-06T07:33:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0240072f40 0x7f02400c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02400c1140 con 0x7f0240152060 2024-03-06T07:33:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02400c1420 con 0x7f024006d680 2024-03-06T07:33:14.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f024484e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02400c2e30 con 0x7f0240072f40 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f0237fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0240072f40 0x7f02400c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f023ed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0240152060 0x7f024014e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36280/0 (socket says 172.21.15.17:36280) 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.529+0000 7f023ed9d640 1 -- 172.21.15.17:0/3545914230 learned_addr learned my addr 172.21.15.17:0/3545914230 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.530+0000 7f0237fff640 1 -- 172.21.15.17:0/3545914230 >> v1:172.21.15.163:6789/0 conn(0x7f024006d680 legacy=0x7f02400c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.530+0000 7f0237fff640 1 -- 172.21.15.17:0/3545914230 >> v1:172.21.15.17:6789/0 conn(0x7f0240152060 legacy=0x7f024014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.530+0000 7f023dd9b640 1 -- 172.21.15.17:0/3545914230 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213662367 0 0) 0x7f02400c1140 con 0x7f0240152060 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.530+0000 7f0237fff640 1 -- 172.21.15.17:0/3545914230 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400c1700 con 0x7f0240072f40 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.530+0000 7f0237fff640 1 --2- 172.21.15.17:0/3545914230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f02400c28f0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f022800a9a0 tx=0x7f022802ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449e257dc2e75bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f023dd9b640 1 -- 172.21.15.17:0/3545914230 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022803a070 con 0x7f0240072f40 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f023dd9b640 1 -- 172.21.15.17:0/3545914230 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022802daa0 con 0x7f0240072f40 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f023dd9b640 1 -- 172.21.15.17:0/3545914230 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022800b040 con 0x7f0240072f40 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 -- 172.21.15.17:0/3545914230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 msgr2=0x7f02400c28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 --2- 172.21.15.17:0/3545914230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f02400c28f0 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f022800a9a0 tx=0x7f022802ccf0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 -- 172.21.15.17:0/3545914230 shutdown_connections 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 --2- 172.21.15.17:0/3545914230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f02400c28f0 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 -- 172.21.15.17:0/3545914230 >> 172.21.15.17:0/3545914230 conn(0x7f024006b1a0 msgr2=0x7f024006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 -- 172.21.15.17:0/3545914230 shutdown_connections 2024-03-06T07:33:14.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.531+0000 7f024484e640 1 -- 172.21.15.17:0/3545914230 wait complete. 2024-03-06T07:33:14.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.532+0000 7f024484e640 1 Processor -- start 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 -- start start 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f024006d680 0x7f024014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0240152060 0x7f024014b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02401524f0 con 0x7f0240072f40 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02400c1d50 con 0x7f024006d680 2024-03-06T07:33:14.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f024484e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024006da60 con 0x7f0240152060 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f0237fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f0237fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51370/0 (socket says 172.21.15.17:51370) 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f0237fff640 1 -- 172.21.15.17:0/2950998683 learned_addr learned my addr 172.21.15.17:0/2950998683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f023ed9d640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0240152060 0x7f024014b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.533+0000 7f023e59c640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f024006d680 0x7f024014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 -- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0240152060 msgr2=0x7f024014b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0240152060 0x7f024014b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 -- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f024006d680 msgr2=0x7f024014d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f024006d680 0x7f024014d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 -- 172.21.15.17:0/2950998683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02401d9800 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0237fff640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f022800a8c0 tx=0x7f02280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228042020 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0228035b50 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024014e290 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022803a040 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.534+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02401d9aa0 con 0x7f0240072f40 2024-03-06T07:33:14.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.535+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024004f330 con 0x7f0240072f40 2024-03-06T07:33:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.535+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0228045020 con 0x7f0240072f40 2024-03-06T07:33:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.536+0000 7f0236ffd640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 0x7f022006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.536+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f02280abb80 con 0x7f0240072f40 2024-03-06T07:33:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.536+0000 7f023e59c640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 0x7f022006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.536+0000 7f023e59c640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 0x7f022006d990 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0230004ea0 tx=0x7f0230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.540+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f022804c030 con 0x7f0240072f40 2024-03-06T07:33:14.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.720+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f024014b890 con 0x7f022006b4d0 2024-03-06T07:33:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.720+0000 7f0236ffd640 1 -- 172.21.15.17:0/2950998683 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0230005420 con 0x7f022006b4d0 2024-03-06T07:33:14.722 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 7 to deep-scrub 2024-03-06T07:33:14.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 msgr2=0x7f022006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 0x7f022006d990 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0230004ea0 tx=0x7f0230009040 comp rx=0 tx=0).stop 2024-03-06T07:33:14.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 msgr2=0x7f024014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f022800a8c0 tx=0x7f02280046c0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 shutdown_connections 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f022006b4d0 0x7f022006d990 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0240152060 0x7f024014b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0240072f40 0x7f024014ae10 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 --2- 172.21.15.17:0/2950998683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f024006d680 0x7f024014d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 >> 172.21.15.17:0/2950998683 conn(0x7f024006b1a0 msgr2=0x7f02400c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 shutdown_connections 2024-03-06T07:33:14.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.722+0000 7f024484e640 1 -- 172.21.15.17:0/2950998683 wait complete. 2024-03-06T07:33:14.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.834+0000 7f138613e640 1 Processor -- start 2024-03-06T07:33:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.834+0000 7f138613e640 1 -- start start 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f138613e640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f138014f360 0x7f1380155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f138613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13800c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f138613e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13800c73d0 con 0x7f13800c5880 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f138613e640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13800c8dd0 con 0x7f138014f360 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f138613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800be2e0 con 0x7f13800c88c0 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f137ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f13800c5880 0x7f1380158b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36288/0 (socket says 172.21.15.17:36288) 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f137ffff640 1 -- 172.21.15.17:0/634235542 learned_addr learned my addr 172.21.15.17:0/634235542 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801515171 0 0) 0x7f13800c73d0 con 0x7f13800c5880 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.835+0000 7f137effd640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13800c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1368003520 con 0x7f13800c5880 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> v2:172.21.15.163:3300/0 conn(0x7f138014f360 0x7f1380155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3010696056 0 0) 0x7f1368003520 con 0x7f13800c5880 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 msgr2=0x7f13800c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13800c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 >> v2:172.21.15.163:3300/0 conn(0x7f138014f360 msgr2=0x7f1380155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 --2- 172.21.15.17:0/634235542 >> v2:172.21.15.163:3300/0 conn(0x7f138014f360 0x7f1380155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800c7670 con 0x7f13800c5880 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f13780019f0 con 0x7f13800c5880 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> v2:172.21.15.163:3300/0 conn(0x7f138014f360 0x7f1380155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 >> v1:172.21.15.17:6789/0 conn(0x7f13800c5880 legacy=0x7f1380158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13680039b0 0x7f1368003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1368004800 0x7f1368004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800be2e0 con 0x7f13680039b0 2024-03-06T07:33:14.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136800b9f0 con 0x7f1368004800 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136800bbf0 con 0x7f13800c88c0 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13680039b0 0x7f1368003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137effd640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1368004800 0x7f1368004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137ffff640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 -- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 msgr2=0x7f13680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13800c88c0 0x7f13680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 -- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1368004800 msgr2=0x7f1368004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:14.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1368004800 0x7f1368004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.836+0000 7f137f7fe640 1 -- 172.21.15.17:0/634235542 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1368003520 con 0x7f13680039b0 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.837+0000 7f137f7fe640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13680039b0 0x7f1368003dd0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c411431385a3c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.837+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000f040 con 0x7f13680039b0 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.837+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370004230 con 0x7f13680039b0 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.837+0000 7f137e7fc640 1 -- 172.21.15.17:0/634235542 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137002d040 con 0x7f13680039b0 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 -- 172.21.15.17:0/634235542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13680039b0 msgr2=0x7f1368003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 --2- 172.21.15.17:0/634235542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13680039b0 0x7f1368003dd0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f1370006fd0 tx=0x7f1368005830 comp rx=0 tx=0).stop 2024-03-06T07:33:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f137ffff640 1 -- 172.21.15.17:0/634235542 reap_dead start 2024-03-06T07:33:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 -- 172.21.15.17:0/634235542 shutdown_connections 2024-03-06T07:33:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 -- 172.21.15.17:0/634235542 >> 172.21.15.17:0/634235542 conn(0x7f13800ac570 msgr2=0x7f13800ac980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 -- 172.21.15.17:0/634235542 shutdown_connections 2024-03-06T07:33:14.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.838+0000 7f138613e640 1 -- 172.21.15.17:0/634235542 wait complete. 2024-03-06T07:33:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.839+0000 7f138613e640 1 Processor -- start 2024-03-06T07:33:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.839+0000 7f138613e640 1 -- start start 2024-03-06T07:33:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13680039b0 0x7f13800afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 0x7f13800c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c5340 con 0x7f13801eaa90 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c7c30 con 0x7f13800c3a30 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c54c0 con 0x7f13680039b0 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51390/0 (socket says 172.21.15.17:51390) 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 -- 172.21.15.17:0/3974088498 learned_addr learned my addr 172.21.15.17:0/3974088498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137f7fe640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13680039b0 0x7f13800afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137ffff640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 0x7f13800c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 -- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13680039b0 msgr2=0x7f13800afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13680039b0 0x7f13800afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 -- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 msgr2=0x7f13800c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 0x7f13800c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 -- 172.21.15.17:0/3974088498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800c1ce0 con 0x7f13801eaa90 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137ffff640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 0x7f13800c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f137effd640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f1374011560 tx=0x7f1374011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374004220 con 0x7f13801eaa90 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13740043c0 con 0x7f13801eaa90 2024-03-06T07:33:14.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13800c1fc0 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374015870 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.840+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13800c2580 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.841+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13800c1a70 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.842+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1374015a10 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.842+0000 7f1367fff640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.843+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f137408a750 con 0x7f13801eaa90 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.843+0000 7f137f7fe640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:14.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.843+0000 7f137f7fe640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 0x7f136c06d9c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1370035000 tx=0x7f1370031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:14.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:14.847+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1374019030 con 0x7f13801eaa90 2024-03-06T07:33:15.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.024+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13800c64c0 con 0x7f136c06b500 2024-03-06T07:33:15.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.025+0000 7f1367fff640 1 -- 172.21.15.17:0/3974088498 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13511 (secure 0 0 0) 0x7f13800c64c0 con 0x7f136c06b500 2024-03-06T07:33:15.027 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:15.027 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:33:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 msgr2=0x7f136c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 0x7f136c06d9c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f1370035000 tx=0x7f1370031000 comp rx=0 tx=0).stop 2024-03-06T07:33:15.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 msgr2=0x7f13800ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 secure :-1 s=READY pgs=2766 cs=0 l=1 rev1=1 crypto rx=0x7f1374011560 tx=0x7f1374011a20 comp rx=0 tx=0).stop 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 shutdown_connections 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f136c06b500 0x7f136c06d9c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13800c3a30 0x7f13800c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13801eaa90 0x7f13800ad660 unknown :-1 s=CLOSED pgs=2766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 --2- 172.21.15.17:0/3974088498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13680039b0 0x7f13800afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 >> 172.21.15.17:0/3974088498 conn(0x7f13800ac570 msgr2=0x7f138014cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.027+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 shutdown_connections 2024-03-06T07:33:15.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:15.028+0000 7f138613e640 1 -- 172.21.15.17:0/3974088498 wait complete. 2024-03-06T07:33:15.039 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":610,"stamp":"2024-03-06T07:33:13.531408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287988,"kb_used_data":21888,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686732,"statfs":{"total":773094113280,"available":772799213568,"internally_reserved":0,"allocated":22413312,"data_stored":12040032,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3816,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-55,"num_read_kb":-61,"num_write":-85,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001556"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":543,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-03-06T07:33:08.293918+0000","last_change":"2024-03-06T07:19:59.406929+0000","last_active":"2024-03-06T07:33:08.293918+0000","last_peered":"2024-03-06T07:33:08.293918+0000","last_clean":"2024-03-06T07:33:08.293918+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:33:08.293918+0000","last_undegraded":"2024-03-06T07:33:08.293918+0000","last_fullsized":"2024-03-06T07:33:08.293918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-06T07:19:58.338366+0000","last_clean_scrub_stamp":"2024-03-06T07:19:58.338366+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T13:36:49.552184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575009,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33968,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337872,"statfs":{"total":96636764160,"available":96601980928,"internally_reserved":0,"allocated":2850816,"data_stored":1947780,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575010,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575010,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38128,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333712,"statfs":{"total":96636764160,"available":96597721088,"internally_reserved":0,"allocated":2850816,"data_stored":1947780,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:15.042 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-06T07:19:58.338366+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=6, tm_hour=7, tm_min=19, tm_sec=58, tm_wday=2, tm_yday=66, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=6, tm_hour=7, tm_min=33, tm_sec=10, tm_wday=2, tm_yday=66, tm_isdst=0) 2024-03-06T07:33:15.042 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-06T07:33:35.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-06T07:33:35.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.155+0000 7f98ae75e640 1 Processor -- start 2024-03-06T07:33:35.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.155+0000 7f98ae75e640 1 -- start start 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98ae75e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98a80c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98ae75e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a8158800 con 0x7f98a806d9f0 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98ae75e640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a8158ae0 con 0x7f98a81551e0 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98ae75e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8158dc0 con 0x7f98a80c3280 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98a7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f98a806d9f0 0x7f98a80a7cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:39616/0 (socket says 172.21.15.17:39616) 2024-03-06T07:33:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98a7fff640 1 -- 172.21.15.17:0/2943357964 learned_addr learned my addr 172.21.15.17:0/2943357964 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98a77fe640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98a80c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.156+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047388025 0 0) 0x7f98a8158800 con 0x7f98a806d9f0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9894003520 con 0x7f98a806d9f0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455607852 0 0) 0x7f98a8158ae0 con 0x7f98a81551e0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a8158800 con 0x7f98a81551e0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574236791 0 0) 0x7f9894003520 con 0x7f98a806d9f0 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 msgr2=0x7f98a80c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98a80c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 >> v1:172.21.15.163:6789/0 conn(0x7f98a81551e0 legacy=0x7f98a8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a81590a0 con 0x7f98a806d9f0 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1673420281 0 0) 0x7f9898001670 con 0x7f98a806d9f0 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 >> v1:172.21.15.17:6789/0 conn(0x7f98a806d9f0 legacy=0x7f98a80a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f9894003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9894004800 0x7f9894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98940051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8158dc0 con 0x7f9894003b70 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989400bad0 con 0x7f9894004800 2024-03-06T07:33:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989400bcd0 con 0x7f98a80c3280 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f9894003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a77fe640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9894004800 0x7f9894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98accfb640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98940051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 -- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 msgr2=0x7f98940051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80c3280 0x7f98940051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 -- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9894004800 msgr2=0x7f9894004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9894004800 0x7f9894004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.157+0000 7f98a7fff640 1 -- 172.21.15.17:0/2943357964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894003520 con 0x7f9894003b70 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.158+0000 7f98a7fff640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f9894003f70 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f989800c190 tx=0x7f9894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ef207082bcbb4b server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.158+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898007cc0 con 0x7f9894003b70 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.158+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898030070 con 0x7f9894003b70 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.158+0000 7f98a6ffd640 1 -- 172.21.15.17:0/2943357964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898005d30 con 0x7f9894003b70 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 -- 172.21.15.17:0/2943357964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 msgr2=0x7f9894003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 --2- 172.21.15.17:0/2943357964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f9894003f70 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f989800c190 tx=0x7f9894005830 comp rx=0 tx=0).stop 2024-03-06T07:33:35.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98accfb640 1 -- 172.21.15.17:0/2943357964 reap_dead start 2024-03-06T07:33:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 -- 172.21.15.17:0/2943357964 shutdown_connections 2024-03-06T07:33:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 -- 172.21.15.17:0/2943357964 >> 172.21.15.17:0/2943357964 conn(0x7f98a80aa630 msgr2=0x7f98a80aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 -- 172.21.15.17:0/2943357964 shutdown_connections 2024-03-06T07:33:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.159+0000 7f98ae75e640 1 -- 172.21.15.17:0/2943357964 wait complete. 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.160+0000 7f98ae75e640 1 Processor -- start 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.160+0000 7f98ae75e640 1 -- start start 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 0x7f98a81e2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a806e0d0 0x7f98a806e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a806df50 con 0x7f9894003b70 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a806ddd0 con 0x7f98a806e0d0 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98ae75e640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8155d20 con 0x7f98a80ae3e0 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38080/0 (socket says 172.21.15.17:38080) 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 -- 172.21.15.17:0/980800984 learned_addr learned my addr 172.21.15.17:0/980800984 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98accfb640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a806e0d0 0x7f98a806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a77fe640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 0x7f98a81e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 -- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 msgr2=0x7f98a81e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 0x7f98a81e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 -- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a806e0d0 msgr2=0x7f98a806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a806e0d0 0x7f98a806e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a7fff640 1 -- 172.21.15.17:0/980800984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a81df3f0 con 0x7f9894003b70 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.161+0000 7f98a77fe640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 0x7f98a81e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98a7fff640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f98980018f0 tx=0x7f98a8155070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989802c9c0 con 0x7f9894003b70 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989802cb60 con 0x7f9894003b70 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a80abc90 con 0x7f9894003b70 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898007070 con 0x7f9894003b70 2024-03-06T07:33:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.162+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a81eb670 con 0x7f9894003b70 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.163+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a804f330 con 0x7f9894003b70 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.163+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9898007210 con 0x7f9894003b70 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.164+0000 7f98a4ff9640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.164+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f98980ab710 con 0x7f9894003b70 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.164+0000 7f98a77fe640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.164+0000 7f98a77fe640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 0x7f988806d990 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f98a80ae800 tx=0x7f9890008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.168+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98a80abc90 con 0x7f9894003b70 2024-03-06T07:33:35.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.346+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 --> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98a80c3700 con 0x7f988806b4d0 2024-03-06T07:33:35.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.347+0000 7f98a4ff9640 1 -- 172.21.15.17:0/980800984 <== mgr.4103 v2:172.21.15.163:6832/1478765449 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13463 (secure 0 0 0) 0x7f98a80c3700 con 0x7f988806b4d0 2024-03-06T07:33:35.349 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:35.350 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-06T07:33:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 msgr2=0x7f988806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 0x7f988806d990 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f98a80ae800 tx=0x7f9890008040 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 msgr2=0x7f98a81e1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f98980018f0 tx=0x7f98a8155070 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 shutdown_connections 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f988806b4d0 0x7f988806d990 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98a806e0d0 0x7f98a806e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98a80ae3e0 0x7f98a81e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 --2- 172.21.15.17:0/980800984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9894003b70 0x7f98a81e1be0 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 >> 172.21.15.17:0/980800984 conn(0x7f98a80aa630 msgr2=0x7f98a8154940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 shutdown_connections 2024-03-06T07:33:35.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.349+0000 7f98ae75e640 1 -- 172.21.15.17:0/980800984 wait complete. 2024-03-06T07:33:35.362 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":620,"stamp":"2024-03-06T07:33:33.535656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287972,"kb_used_data":21888,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754686748,"statfs":{"total":773094113280,"available":772799229952,"internally_reserved":0,"allocated":22413312,"data_stored":12040032,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12748,"internal_metadata":272354868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002555"},"pg_stats":[{"pgid":"1.0","version":"219'119","reported_seq":554,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-03-06T07:33:14.891396+0000","last_change":"2024-03-06T07:33:14.891430+0000","last_active":"2024-03-06T07:33:14.891396+0000","last_peered":"2024-03-06T07:33:14.891396+0000","last_clean":"2024-03-06T07:33:14.891396+0000","last_became_active":"2024-03-06T07:19:59.405869+0000","last_became_peered":"2024-03-06T07:19:59.405869+0000","last_unstale":"2024-03-06T07:33:14.891396+0000","last_undegraded":"2024-03-06T07:33:14.891396+0000","last_fullsized":"2024-03-06T07:33:14.891396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"219'119","last_scrub_stamp":"2024-03-06T07:33:14.891316+0000","last_deep_scrub":"219'119","last_deep_scrub_stamp":"2024-03-06T07:33:14.891316+0000","last_clean_scrub_stamp":"2024-03-06T07:33:14.891316+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-07T08:26:24.103007+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65040,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33960,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337880,"statfs":{"total":96636764160,"available":96601989120,"internally_reserved":0,"allocated":2850816,"data_stored":1947780,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33900,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38064,"kb_used_data":2720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2785280,"data_stored":1357412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38120,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333720,"statfs":{"total":96636764160,"available":96597729280,"internally_reserved":0,"allocated":2850816,"data_stored":1947780,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32520,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-06T07:33:35.363 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-06T07:33:35.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.462+0000 7f8f402eb640 1 Processor -- start 2024-03-06T07:33:35.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.462+0000 7f8f402eb640 1 -- start start 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f402eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f3814bcc0 0x7f8f38155a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f402eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f380c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f402eb640 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f380c75e0 con 0x7f8f380c5910 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f402eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f380c7f10 con 0x7f8f3814bcc0 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f402eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380c78c0 con 0x7f8f380be150 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f3814bcc0 0x7f8f38155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f3814bcc0 0x7f8f38155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38092/0 (socket says 172.21.15.17:38092) 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 -- 172.21.15.17:0/3555741591 learned_addr learned my addr 172.21.15.17:0/3555741591 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 -- 172.21.15.17:0/3555741591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 msgr2=0x7f8f380c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 --2- 172.21.15.17:0/3555741591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f380c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 -- 172.21.15.17:0/3555741591 >> v1:172.21.15.163:6789/0 conn(0x7f8f380c5910 legacy=0x7f8f38158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.463+0000 7f8f3e087640 1 -- 172.21.15.17:0/3555741591 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380c7ba0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.464+0000 7f8f3e087640 1 --2- 172.21.15.17:0/3555741591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f38155a00 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27c1abab9908164 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.464+0000 7f8f3d085640 1 -- 172.21.15.17:0/3555741591 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30034070 con 0x7f8f3814bcc0 2024-03-06T07:33:35.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.464+0000 7f8f3d085640 1 -- 172.21.15.17:0/3555741591 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30030840 con 0x7f8f3814bcc0 2024-03-06T07:33:35.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.464+0000 7f8f3d085640 1 -- 172.21.15.17:0/3555741591 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30030c10 con 0x7f8f3814bcc0 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 -- 172.21.15.17:0/3555741591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 msgr2=0x7f8f38155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 --2- 172.21.15.17:0/3555741591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f38155a00 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002b980 comp rx=0 tx=0).stop 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 -- 172.21.15.17:0/3555741591 shutdown_connections 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 --2- 172.21.15.17:0/3555741591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f380c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 --2- 172.21.15.17:0/3555741591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f38155a00 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 -- 172.21.15.17:0/3555741591 >> 172.21.15.17:0/3555741591 conn(0x7f8f380ac600 msgr2=0x7f8f380aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 -- 172.21.15.17:0/3555741591 shutdown_connections 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.465+0000 7f8f402eb640 1 -- 172.21.15.17:0/3555741591 wait complete. 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.466+0000 7f8f402eb640 1 Processor -- start 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.466+0000 7f8f402eb640 1 -- start start 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f381df120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 0x7f8f381df660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381592a0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38159120 con 0x7f8f380c5910 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38159420 con 0x7f8f380be150 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38094/0 (socket says 172.21.15.17:38094) 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 -- 172.21.15.17:0/950414410 learned_addr learned my addr 172.21.15.17:0/950414410 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3d886640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 0x7f8f381df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e087640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f381df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 -- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 msgr2=0x7f8f381df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f381df120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 -- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 msgr2=0x7f8f381df660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 0x7f8f381df660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 -- 172.21.15.17:0/950414410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380c49b0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3d886640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 0x7f8f381df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f3e888640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800c7b0 tx=0x7f8f2800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28017070 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f28003da0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f38154dc0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280052b0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.467+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f380c1c30 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.468+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3804f330 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.469+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f2801a070 con 0x7f8f3814bcc0 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.469+0000 7f8f36ffd640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 0x7f8f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.469+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f8f28089730 con 0x7f8f3814bcc0 2024-03-06T07:33:35.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.469+0000 7f8f3e087640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 0x7f8f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.470+0000 7f8f3e087640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 0x7f8f2406d990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8f30036000 tx=0x7f8f3003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.474+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8f280586b0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.657+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8f3806c560 con 0x7f8f3814bcc0 2024-03-06T07:33:35.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.660+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v65)=0 v65) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8f28057fb0 con 0x7f8f3814bcc0 2024-03-06T07:33:35.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.661+0000 7f8f36ffd640 1 -- 172.21.15.17:0/950414410 <== mon.0 v2:172.21.15.17:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8f2805c100 con 0x7f8f3814bcc0 2024-03-06T07:33:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 msgr2=0x7f8f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 0x7f8f2406d990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8f30036000 tx=0x7f8f3003b040 comp rx=0 tx=0).stop 2024-03-06T07:33:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 msgr2=0x7f8f381eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800c7b0 tx=0x7f8f2800cc70 comp rx=0 tx=0).stop 2024-03-06T07:33:35.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 shutdown_connections 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f8f2406b4d0 0x7f8f2406d990 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3814bcc0 0x7f8f381eb320 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f380c5910 0x7f8f381df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 --2- 172.21.15.17:0/950414410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f380be150 0x7f8f381df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 >> 172.21.15.17:0/950414410 conn(0x7f8f380ac600 msgr2=0x7f8f380abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 shutdown_connections 2024-03-06T07:33:35.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.662+0000 7f8f402eb640 1 -- 172.21.15.17:0/950414410 wait complete. 2024-03-06T07:33:35.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-06T07:33:35.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-06T07:33:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.774+0000 7f66ede6d640 1 Processor -- start 2024-03-06T07:33:35.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.774+0000 7f66ede6d640 1 -- start start 2024-03-06T07:33:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f66e8152920 0x7f66e8155c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e80c8a00 0x7f66e8158f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f66e8159460 0x7f66e80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e81587b0 con 0x7f66e8159460 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e8156180 con 0x7f66e8152920 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66ede6d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80c7760 con 0x7f66e80c8a00 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f66e8159460 0x7f66e80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f66e8159460 0x7f66e80c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38106/0 (socket says 172.21.15.17:38106) 2024-03-06T07:33:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 -- 172.21.15.17:0/3268049674 learned_addr learned my addr 172.21.15.17:0/3268049674 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 -- 172.21.15.17:0/3268049674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e80c8a00 msgr2=0x7f66e8158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 --2- 172.21.15.17:0/3268049674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e80c8a00 0x7f66e8158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 -- 172.21.15.17:0/3268049674 >> v2:172.21.15.163:3300/0 conn(0x7f66e8152920 msgr2=0x7f66e8155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 --2- 172.21.15.17:0/3268049674 >> v2:172.21.15.163:3300/0 conn(0x7f66e8152920 0x7f66e8155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.775+0000 7f66e7fff640 1 -- 172.21.15.17:0/3268049674 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e80c7a40 con 0x7f66e8159460 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.776+0000 7f66e7fff640 1 --2- 172.21.15.17:0/3268049674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e8159460 0x7f66e80c5330 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f66e80be490 tx=0x7f66e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7c7e6310a9e28c server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.776+0000 7f66e67fc640 1 -- 172.21.15.17:0/3268049674 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e002c9e0 con 0x7f66e8159460 2024-03-06T07:33:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.776+0000 7f66e67fc640 1 -- 172.21.15.17:0/3268049674 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f66e002cb80 con 0x7f66e8159460 2024-03-06T07:33:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.776+0000 7f66e67fc640 1 -- 172.21.15.17:0/3268049674 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00343f0 con 0x7f66e8159460 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 -- 172.21.15.17:0/3268049674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e8159460 msgr2=0x7f66e80c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 --2- 172.21.15.17:0/3268049674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e8159460 0x7f66e80c5330 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f66e80be490 tx=0x7f66e002be40 comp rx=0 tx=0).stop 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 -- 172.21.15.17:0/3268049674 shutdown_connections 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 --2- 172.21.15.17:0/3268049674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e8159460 0x7f66e80c5330 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 --2- 172.21.15.17:0/3268049674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e80c8a00 0x7f66e8158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 --2- 172.21.15.17:0/3268049674 >> v2:172.21.15.163:3300/0 conn(0x7f66e8152920 0x7f66e8155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 -- 172.21.15.17:0/3268049674 >> 172.21.15.17:0/3268049674 conn(0x7f66e80ac560 msgr2=0x7f66e80ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 -- 172.21.15.17:0/3268049674 shutdown_connections 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.777+0000 7f66ede6d640 1 -- 172.21.15.17:0/3268049674 wait complete. 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.778+0000 7f66ede6d640 1 Processor -- start 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.778+0000 7f66ede6d640 1 -- start start 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 0x7f66e8149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e8152920 0x7f66e81ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e806d030 con 0x7f66e814a050 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8158a00 con 0x7f66e80c8a00 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e80c7390 con 0x7f66e8152920 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38112/0 (socket says 172.21.15.17:38112) 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 -- 172.21.15.17:0/2874431284 learned_addr learned my addr 172.21.15.17:0/2874431284 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e77fe640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 0x7f66e8149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e6ffd640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e8152920 0x7f66e81ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 -- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e8152920 msgr2=0x7f66e81ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e8152920 0x7f66e81ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 -- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 msgr2=0x7f66e8149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 0x7f66e8149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 -- 172.21.15.17:0/2874431284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8149960 con 0x7f66e814a050 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e77fe640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 0x7f66e8149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66e7fff640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f66e0036040 tx=0x7f66e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0043020 con 0x7f66e814a050 2024-03-06T07:33:35.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f66e003e450 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e814e3d0 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.779+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e003e860 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.780+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e814eee0 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.780+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e814e1e0 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.781+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66e0046020 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.781+0000 7f66cffff640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.781+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 5593+0+0 (secure 0 0 0) 0x7f66e00acbb0 con 0x7f66e814a050 2024-03-06T07:33:35.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.781+0000 7f66e77fe640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-06T07:33:35.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.782+0000 7f66e77fe640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009890 tx=0x7f66d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-06T07:33:35.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:35.786+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66e004d030 con 0x7f66e814a050 2024-03-06T07:33:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.004+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66e814ecd0 con 0x7f66e814a050 2024-03-06T07:33:36.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.004+0000 7f66cffff640 1 -- 172.21.15.17:0/2874431284 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+1329 (secure 0 0 0) 0x7f66e007ba80 con 0x7f66e814a050 2024-03-06T07:33:36.006 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-06T07:33:36.006 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":220,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6834/2129501505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":2129501505},{"type":"v1","addr":"172.21.15.17:6834","nonce":2129501505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":220}],"filesystems":[]} 2024-03-06T07:33:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 220 2024-03-06T07:33:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 msgr2=0x7f66d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 0x7f66d406d990 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009890 tx=0x7f66d8008040 comp rx=0 tx=0).stop 2024-03-06T07:33:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 msgr2=0x7f66e814b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-06T07:33:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f66e0036040 tx=0x7f66e0004a40 comp rx=0 tx=0).stop 2024-03-06T07:33:36.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 shutdown_connections 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:6832/1478765449,v1:172.21.15.163:6833/1478765449] conn(0x7f66d406b4d0 0x7f66d406d990 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66e814a050 0x7f66e814b320 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66e8152920 0x7f66e81ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 --2- 172.21.15.17:0/2874431284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66e80c8a00 0x7f66e8149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 >> 172.21.15.17:0/2874431284 conn(0x7f66e80ac560 msgr2=0x7f66e80c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 shutdown_connections 2024-03-06T07:33:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-06T07:33:36.006+0000 7f66ede6d640 1 -- 172.21.15.17:0/2874431284 wait complete. 2024-03-06T07:33:36.018 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-06T07:33:36.019 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-06T07:33:36.019 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-06T07:33:36.019 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.028 INFO:tasks.ceph.mds.a:Stopped 2024-03-06T07:33:36.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-06T07:33:36.028 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.078 INFO:tasks.ceph.mds.c:Stopped 2024-03-06T07:33:36.078 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-06T07:33:36.078 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.090 INFO:tasks.ceph.mds.b:Stopped 2024-03-06T07:33:36.090 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-06T07:33:36.090 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.142 INFO:tasks.ceph.mds.d:Stopped 2024-03-06T07:33:36.143 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-06T07:33:36.143 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-06T07:33:36.143 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.167 INFO:tasks.ceph.osd.0:Stopped 2024-03-06T07:33:36.167 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-06T07:33:36.167 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.233 INFO:tasks.ceph.osd.1:Stopped 2024-03-06T07:33:36.233 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-06T07:33:36.233 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.305 INFO:tasks.ceph.osd.2:Stopped 2024-03-06T07:33:36.305 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-06T07:33:36.305 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.371 INFO:tasks.ceph.osd.3:Stopped 2024-03-06T07:33:36.371 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-06T07:33:36.371 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.415 INFO:tasks.ceph.osd.4:Stopped 2024-03-06T07:33:36.415 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-06T07:33:36.415 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.449 INFO:tasks.ceph.osd.5:Stopped 2024-03-06T07:33:36.450 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-06T07:33:36.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.518 INFO:tasks.ceph.osd.6:Stopped 2024-03-06T07:33:36.518 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-06T07:33:36.518 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.589 INFO:tasks.ceph.osd.7:Stopped 2024-03-06T07:33:36.589 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-06T07:33:36.589 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-06T07:33:36.589 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.614 INFO:tasks.ceph.mgr.y:Stopped 2024-03-06T07:33:36.614 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-06T07:33:36.614 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.658 INFO:tasks.ceph.mgr.x:Stopped 2024-03-06T07:33:36.659 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-06T07:33:36.659 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-06T07:33:36.659 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.670 INFO:tasks.ceph.mon.a:Stopped 2024-03-06T07:33:36.670 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-06T07:33:36.670 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.712 INFO:tasks.ceph.mon.b:Stopped 2024-03-06T07:33:36.713 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-06T07:33:36.713 INFO:teuthology.orchestra.run:waiting for 300 2024-03-06T07:33:36.765 INFO:tasks.ceph.mon.c:Stopped 2024-03-06T07:33:36.766 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-06T07:33:36.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v '\(MDS_INSUFFICIENT_STANDBY\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(FS_DEGRADED\)' | head -n 1 2024-03-06T07:33:36.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:33:36.800 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-06T07:33:36.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:33:36.982 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-06T07:33:37.073 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:33:37.073 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-06T07:33:37.154 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:33:37.154 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-06T07:33:37.205 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:33:37.205 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-06T07:33:37.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:33:37.381 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-06T07:33:37.484 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:33:37.485 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-06T07:33:37.549 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:33:37.550 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-06T07:33:37.600 INFO:tasks.ceph:Archiving mon data... 2024-03-06T07:33:37.601 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/data/mon.a.tgz 2024-03-06T07:33:37.601 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-06T07:33:37.617 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.EFtHa2xkxo 2024-03-06T07:33:37.617 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EFtHa2xkxo 2024-03-06T07:33:37.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.EFtHa2xkxo 2024-03-06T07:33:37.892 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.EFtHa2xkxo is 559KB 2024-03-06T07:33:37.991 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.EFtHa2xkxo 2024-03-06T07:33:38.005 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/data/mon.b.tgz 2024-03-06T07:33:38.005 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-03-06T07:33:38.023 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.156J9XIAlC 2024-03-06T07:33:38.023 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.156J9XIAlC 2024-03-06T07:33:38.217 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.156J9XIAlC 2024-03-06T07:33:38.301 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.156J9XIAlC is 571KB 2024-03-06T07:33:38.400 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.156J9XIAlC 2024-03-06T07:33:38.414 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/data/mon.c.tgz 2024-03-06T07:33:38.415 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2024-03-06T07:33:38.469 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.rAS55Df8CK 2024-03-06T07:33:38.470 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rAS55Df8CK 2024-03-06T07:33:38.643 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.rAS55Df8CK 2024-03-06T07:33:38.729 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.rAS55Df8CK is 572KB 2024-03-06T07:33:38.829 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.rAS55Df8CK 2024-03-06T07:33:38.844 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-06T07:33:38.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-06T07:33:38.847 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-06T07:33:38.933 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-06T07:33:38.965 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-06T07:33:38.966 INFO:tasks.ceph:Archiving crash dumps... 2024-03-06T07:33:38.967 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi017/crash 2024-03-06T07:33:38.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-06T07:33:39.002 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi163/crash 2024-03-06T07:33:39.016 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-06T07:33:39.042 INFO:tasks.ceph:Shutting down logrotate 2024-03-06T07:33:39.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:39.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:39.135 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:39.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-06T07:33:39.191 INFO:tasks.ceph:Compressing logs... 2024-03-06T07:33:39.191 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-06T07:33:39.207 DEBUG:teuthology.orchestra.run.smithi163:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42189.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37963.log 2024-03-06T07:33:39.289 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.1.42189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37723.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42189.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37683.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37723.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37963.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr: 87.2%/var/log/ceph/ceph-client.admin.37683.log: -- replaced with /var/log/ceph/ceph-client.admin.37723.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37683.log.gz 2024-03-06T07:33:39.290 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37843.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37923.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37843.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37843.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.7.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37923.log.gz 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37883.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37763.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37883.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-06T07:33:39.291 INFO:teuthology.orchestra.run.smithi163.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37883.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mgr.x.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37763.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-client.admin.37803.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-06T07:33:39.292 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mon.b.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-06T07:33:39.293 INFO:teuthology.orchestra.run.smithi163.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-06T07:33:39.350 INFO:teuthology.orchestra.run.smithi163.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-06T07:33:39.374 INFO:teuthology.orchestra.run.smithi163.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-06T07:33:39.471 INFO:teuthology.orchestra.run.smithi163.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-06T07:33:39.664 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75523.log 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66714.log 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89110.log 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84203.log 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75523.log.gz 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89110.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66714.log.gz 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67069.log 2024-03-06T07:33:39.665 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89110.log.gz 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83765.log 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84203.log.gz 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76644.log 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83765.log: gzip -5 --verbose -- 89.5% /var/log/ceph/ceph-client.admin.45826.log 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67069.log.gz 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80459.log 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83765.log.gz 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45826.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76644.log.gz 2024-03-06T07:33:39.666 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61475.log 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80459.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45826.log.gz 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43936.log 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80459.log.gz 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67688.log 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61475.log.gz 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70087.log 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67688.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43936.log.gzgzip 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68298.log 2024-03-06T07:33:39.667 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53974.log 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67688.log.gz 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70087.log.gz 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68298.log.gz 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87879.log 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54180.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53974.log.gz 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81359.log 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83681.log 2024-03-06T07:33:39.668 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87879.log.gz 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66127.log 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81359.log.gz 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66127.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83681.log.gz 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53623.log 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66127.log.gz 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72609.log 2024-03-06T07:33:39.669 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53623.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62184.log 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53623.log.gz 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72609.log.gz 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86853.log 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84768.log 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86853.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62184.log.gz 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85430.log 2024-03-06T07:33:39.670 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58141.log 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86853.log.gz 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84768.log.gz 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87021.log 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4%/var/log/ceph/ceph-client.admin.58141.log: -- replaced with /var/log/ceph/ceph-client.admin.85430.log.gz 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79682.log 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58141.log.gz 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84119.log 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71819.log 2024-03-06T07:33:39.671 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87021.log.gz 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84119.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79682.log.gz 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57953.log 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84119.log.gz 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76415.log 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74721.log 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71819.log.gz 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76415.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57953.log.gz 2024-03-06T07:33:39.672 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66840.log 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76415.log.gz 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51767.log 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48410.log 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74721.log.gz 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51767.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66840.log.gz 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51725.log 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51767.log.gz 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.48410.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65533.log 2024-03-06T07:33:39.673 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51725.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48410.log.gz 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52362.log 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51725.log.gz 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47843.log 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52362.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65533.log.gz 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71424.log 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47843.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52362.log.gz 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76019.log 2024-03-06T07:33:39.674 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47843.log.gz 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62472.log 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76019.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71424.log.gz 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73693.log 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62472.log: gzip -5 89.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.76019.log.gz 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.83255.log 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73693.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62472.log.gz 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59265.log 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83255.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.61601.log 2024-03-06T07:33:39.675 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73693.log.gz 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59265.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83255.log.gz 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59643.log 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59265.log.gz 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65701.log 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61601.log.gz 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75211.log 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59643.log.gz 2024-03-06T07:33:39.676 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65701.log.gz 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59391.log 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6%/var/log/ceph/ceph-client.admin.45532.log: -- replaced with /var/log/ceph/ceph-client.admin.75211.log.gz 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87669.log 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59391.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73464.log 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59391.log.gz 2024-03-06T07:33:39.677 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55087.log 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87669.log.gz 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46943.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73464.log.gz 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50715.log 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88047.log 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55087.log.gz 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2024-03-06T07:33:39.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88047.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50715.log.gz 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82979.log 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88047.log.gz/var/log/ceph/ceph-client.admin.51893.log: 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54587.log 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82979.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88774.log 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54587.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82979.log.gz 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78248.log 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88774.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54587.log.gz 2024-03-06T07:33:39.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87399.log 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78248.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88774.log.gz 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65180.log 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87399.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78248.log.gz 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47633.log 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65180.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87399.log.gz 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87315.log 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65180.log.gz 2024-03-06T07:33:39.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55927.log 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47633.log.gz 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81905.log 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87315.log.gz 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65054.log 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81905.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55927.log.gz 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58911.log 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65054.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81905.log.gz 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71466.log 2024-03-06T07:33:39.681 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65054.log.gz 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60975.log 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71466.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58911.log.gz 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60975.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.71466.log.gz -- 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.57827.log 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55465.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60975.log.gz 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78205.log 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7%/var/log/ceph/ceph-client.admin.57827.log: -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74150.log 2024-03-06T07:33:39.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78205.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57827.log.gz 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56862.log 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9%/var/log/ceph/ceph-client.admin.74150.log: -- replaced with /var/log/ceph/ceph-client.admin.78205.log.gz 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56862.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74150.log.gz 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62264.log 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56862.log.gz 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65893.log 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-03-06T07:33:39.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66672.log 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65893.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62264.log.gz 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48830.log 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66672.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65893.log.gz 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83297.log 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1%/var/log/ceph/ceph-client.admin.48830.log: -- replaced with /var/log/ceph/ceph-client.admin.66672.log.gz 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53093.log 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5%/var/log/ceph/ceph-client.admin.83297.log: -- replaced with /var/log/ceph/ceph-client.admin.48830.log.gz 2024-03-06T07:33:39.684 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75977.log 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83297.log.gz 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53093.log.gz 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66313.log 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51454.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75977.log.gz 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66313.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90036.log 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66313.log.gz 2024-03-06T07:33:39.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63269.log 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90036.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81033.log 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63269.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90036.log.gz 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51935.log 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43774.log 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63269.log.gz 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81033.log.gz 2024-03-06T07:33:39.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47213.log 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56988.log 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43774.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51935.log.gz 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42308.log 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43774.log.gz 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47213.log.gz 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80277.log 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42308.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56988.log.gz 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81821.log 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42308.log.gz 2024-03-06T07:33:39.687 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81443.log 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80277.log.gz 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54713.log 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81443.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81821.log.gz 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89586.log 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54713.log: gzip 89.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.81443.log.gz 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.71945.log 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89586.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54713.log.gz 2024-03-06T07:33:39.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70339.log 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71552.log 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89586.log.gz 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70339.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71945.log.gz 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68574.log 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70339.log.gz 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51202.log 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86415.log 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71552.log.gz 2024-03-06T07:33:39.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51202.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68574.log.gz 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63035.log 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51202.log.gz 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86979.log 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86415.log.gz 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63035.log.gz 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75463.log 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47297.log 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2024-03-06T07:33:39.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75463.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86979.log.gz 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66984.log 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47297.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75463.log.gz 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57306.log 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75660.log 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57306.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.66984.log.gz 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64393.log 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75660.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57306.log.gz 2024-03-06T07:33:39.691 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44104.log 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75660.log.gz 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43690.log 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64393.log.gz 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44104.log.gz 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67195.log 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76771.log 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5%/var/log/ceph/ceph-client.admin.67195.log: -- replaced with /var/log/ceph/ceph-client.admin.43690.log.gz 2024-03-06T07:33:39.692 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87837.log 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67195.log.gz 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77575.log 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87837.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76771.log.gz 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87837.log.gz 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72525.log 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77575.log.gz 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61895.log 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72525.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-03-06T07:33:39.693 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.58079.log 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72525.log.gz 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46859.log 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61895.log.gz 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54461.log 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46859.log: gzip -5 --verbose 89.4% -- -- replaced with /var/log/ceph/ceph-client.admin.58079.log.gz /var/log/ceph/ceph-client.admin.72693.log 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46859.log.gz 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67363.log 2024-03-06T07:33:39.694 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54461.log.gz 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79512.log 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67363.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72693.log.gz 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60094.log 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67363.log.gz 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83471.log 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79512.log.gz 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65096.log 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83471.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60094.log.gz 2024-03-06T07:33:39.695 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83471.log.gz 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66546.log 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65096.log.gz 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57432.log 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66546.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76518.log 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66546.log.gz 2024-03-06T07:33:39.696 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61811.log 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57432.log.gz 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50508.log 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72945.log 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76518.log.gz 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61811.log.gz 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54377.log 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52448.log 2024-03-06T07:33:39.697 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54377.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.72945.log.gz -5 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61289.log 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54377.log.gz 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89152.log 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52448.log.gz 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53848.log 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89152.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61289.log.gz 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52883.log 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89152.log.gz 2024-03-06T07:33:39.698 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49580.log 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53848.log.gz 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56210.log 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49580.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52883.log.gz 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81779.log 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56210.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84936.log 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81779.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2024-03-06T07:33:39.699 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84936.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81779.log.gz 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89994.log 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84936.log.gz 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62105.log 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89994.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67944.log 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69605.log 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89994.log.gz 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62105.log.gz 2024-03-06T07:33:39.700 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73124.log 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67944.log.gz 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62388.log 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72903.log 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69605.log.gz 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73124.log.gz 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50865.log 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62388.log.gz 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69334.log 2024-03-06T07:33:39.701 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50865.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72903.log.gz 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85157.log 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50865.log.gz 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49124.log 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69334.log.gz 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58868.log 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49124.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85157.log.gz 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49124.log.gz 2024-03-06T07:33:39.702 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77869.log 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45574.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58868.log.gz 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58430.log 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55675.log 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58430.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77869.log.gz 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60306.log 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55675.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58430.log.gz 2024-03-06T07:33:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72295.log 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60306.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55675.log.gz 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75421.log 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60306.log.gz 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0%/var/log/ceph/ceph-client.admin.75421.log: -- replaced with /var/log/ceph/ceph-client.admin.72295.log.gz 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75421.log.gz 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78571.log 2024-03-06T07:33:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44870.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63462.log 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78571.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84894.log 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78571.log.gz 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62021.log 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63462.log.gz 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69961.log 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70938.log 2024-03-06T07:33:39.705 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84894.log.gz 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62021.log.gz 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58304.log 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70938.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69961.log.gz 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63641.log 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.70938.log.gz 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58304.log.gz 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54252.log 2024-03-06T07:33:39.706 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67902.log 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63641.log.gz 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54252.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79850.log 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67646.log 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67902.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60741.log 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67902.log.gz 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79850.log.gz 2024-03-06T07:33:39.707 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85556.log 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60741.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67646.log.gz 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51160.log 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85556.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60741.log.gz 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87063.log 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85556.log.gz 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78911.log 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73987.log 2024-03-06T07:33:39.708 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51160.log.gz 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87063.log.gz 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60657.log 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46120.log 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60657.log: 87.4% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.73987.log.gz 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78911.log.gz 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77138.log 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60657.log.gz 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79182.log 2024-03-06T07:33:39.709 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46120.log.gz 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79182.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.73903.log 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77138.log.gz 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79182.log.gz/var/log/ceph/ceph-client.admin.67730.log: 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70260.log 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73903.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85850.log 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73903.log.gz 2024-03-06T07:33:39.710 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82073.log 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70260.log.gz 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74661.log 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82073.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.85850.log.gz -5 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85328.log 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82073.log.gz 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56508.log 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74661.log.gz 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69876.log 2024-03-06T07:33:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52585.log 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85328.log.gz 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69876.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56508.log.gz 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67986.log 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69876.log.gz 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72089.log 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60443.log 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52585.log.gz 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67986.log.gz 2024-03-06T07:33:39.712 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85244.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72089.log.gz 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64099.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88942.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85244.log.gz 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58388.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64099.log.gz 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53718.log 2024-03-06T07:33:39.713 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88942.log.gz 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58388.log.gz 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65268.log 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46901.log 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7%/var/log/ceph/ceph-client.admin.53718.log: -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73548.log 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53718.log.gz 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46901.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65268.log.gz 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76145.log 2024-03-06T07:33:39.714 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46901.log.gz 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73548.log.gz 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70419.log 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49537.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.76145.log.gz /var/log/ceph/ceph-client.admin.42178.log 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80757.log 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70419.log.gz 2024-03-06T07:33:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45446.log 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80757.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42178.log.gz /var/log/ceph/ceph-client.admin.78121.log 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80757.log.gz 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50316.log 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45446.log.gz 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47129.log 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50316.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.78121.log.gz -5 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86793.log 2024-03-06T07:33:39.716 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72379.log 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6%/var/log/ceph/ceph-client.admin.86793.log: -- replaced with /var/log/ceph/ceph-client.admin.47129.log.gz 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61102.log 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%/var/log/ceph/ceph-client.admin.72379.log: -- replaced with /var/log/ceph/ceph-client.admin.86793.log.gz 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43464.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61102.log.gz 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72379.log.gz 2024-03-06T07:33:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83849.log 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88173.log 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gzgzip 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80799.log 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88173.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.83849.log.gz -5 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.88858.log 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80799.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88173.log.gz 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58183.log 2024-03-06T07:33:39.718 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80799.log.gz 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76355.log 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88858.log.gz 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82586.log 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58183.log.gz 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82031.log 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76355.log.gz 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44873.log 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59307.log 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82586.log.gz 2024-03-06T07:33:39.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44873.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82031.log.gz 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68172.log 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44873.log.gz 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49874.log 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54797.log 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59307.log.gz 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49874.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68172.log.gz 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86060.log 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49874.log.gz 2024-03-06T07:33:39.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86060.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54797.log.gz 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52673.log 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74312.log 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86060.log.gz 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88485.log 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74312.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52673.log.gz 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81737.log 2024-03-06T07:33:39.721 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74312.log.gz 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80547.log 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81737.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.88485.log.gz --verbose 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.58766.log 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80547.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81737.log.gz 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68256.log 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58766.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80547.log.gz 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50130.log 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9%/var/log/ceph/ceph-client.admin.68256.log: -- replaced with /var/log/ceph/ceph-client.admin.58766.log.gz 2024-03-06T07:33:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72131.log 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68256.log.gz 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45426.log 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68658.log 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50130.log.gz 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72131.log.gz 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45426.log.gz 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2024-03-06T07:33:39.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65617.log 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68658.log.gz 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47507.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84540.log 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65617.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52277.log 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84540.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65617.log.gz 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61517.log 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84540.log.gz 2024-03-06T07:33:39.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56168.log 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64930.log 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56168.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61517.log.gz 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48011.log 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56168.log.gz 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82115.log 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41845.log 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64930.log.gz 2024-03-06T07:33:39.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48011.log.gz 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61937.log 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82115.log.gz 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75702.log 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61937.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41845.log.gz 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89381.log 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75702.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.61937.log.gz 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70214.log 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89381.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.75702.log.gz -- 2024-03-06T07:33:39.726 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.45700.log 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70214.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89381.log.gz 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64057.log 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9%/var/log/ceph/ceph-client.admin.45700.log: -- replaced with /var/log/ceph/ceph-client.admin.70214.log.gz 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51286.log 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64057.log: gzip -5 --verbose 88.7% -- /var/log/ceph/ceph-client.admin.61979.log -- replaced with /var/log/ceph/ceph-client.admin.45700.log.gz 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64057.log.gz 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78374.log 2024-03-06T07:33:39.727 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51286.log.gz 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64183.log 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78374.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.61979.log.gz 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47717.log 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64183.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78374.log.gz 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73250.log 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64183.log.gz 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47087.log 2024-03-06T07:33:39.728 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85199.log 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47717.log.gz 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73250.log.gz 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66924.log 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47087.log.gz 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55129.log 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66924.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.44530.log 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.85199.log.gz 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55129.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66924.log.gz 2024-03-06T07:33:39.729 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55129.log.gz 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44531.log 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90204.log 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68616.log 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44530.log.gz 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90204.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44531.log.gz 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61247.log 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90204.log.gz 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2024-03-06T07:33:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77827.log 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68616.log.gz 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45109.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61247.log.gz 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84372.log 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73777.log 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73334.log 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77827.log.gz 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73777.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84372.log.gz 2024-03-06T07:33:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74071.log 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73334.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73777.log.gz 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83807.log 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74071.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.73334.log.gz 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86583.log 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74071.log.gz 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65935.log 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57180.log 2024-03-06T07:33:39.732 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83807.log.gz 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86583.log.gz 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66756.log 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46374.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57180.log.gz 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr: 91.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.65935.log.gz -5 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49166.log 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.66756.log: --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-03-06T07:33:39.733 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80321.log 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66756.log.gz 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42394.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.49166.log.gz --verbose 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.69731.log 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46688.log 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80321.log.gz 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77224.log 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.0.46688.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69731.log.gz 2024-03-06T07:33:39.734 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88359.log 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46688.log.gz 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69773.log 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88359.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77224.log.gzgzip 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52543.log 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79098.log 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88359.log.gz 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69773.log.gz 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88900.log 2024-03-06T07:33:39.735 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52543.log.gz 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67237.log 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62063.log 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79098.log.gz 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88900.log.gz 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61331.log 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67237.log.gz 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-03-06T07:33:39.736 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68850.log 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62063.log.gz 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61331.log.gz 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53667.log 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88401.log 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56380.log 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53667.log.gz 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54755.log 2024-03-06T07:33:39.737 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68850.log.gz 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57869.log 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56380.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88401.log.gz 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57911.log 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56380.log.gz 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54755.log.gz 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64351.log 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57869.log.gz 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-03-06T07:33:39.738 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57911.log.gz 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90460.log 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64351.log.gz 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62602.log 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90460.log: gzip 88.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz -- 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.73506.log 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62602.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90460.log.gzgzip 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57739.log 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62602.log.gz 2024-03-06T07:33:39.739 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73506.log.gz 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70892.log 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51496.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57739.log.gz 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78845.log 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86937.log 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78845.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70892.log.gz 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79350.log 2024-03-06T07:33:39.740 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58556.log 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79350.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.78845.log.gz 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.86937.log.gz -- 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.56252.log 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79350.log.gz 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49962.log 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56252.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58556.log.gz 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82542.log 2024-03-06T07:33:39.741 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56252.log.gz 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53009.log 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82811.log 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49962.log.gz 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53009.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82542.log.gz 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82811.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53009.log.gz -5 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85062.log 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82811.log.gz 2024-03-06T07:33:39.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84245.log 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48122.log 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85062.log: /var/log/ceph/ceph-client.admin.84245.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85062.log.gz 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61643.log 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84245.log.gz 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65012.log 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48122.log: /var/log/ceph/ceph-client.admin.61643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46246.log 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48122.log.gz 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61643.log.gz 2024-03-06T07:33:39.743 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53369.log 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64687.log 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46246.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65012.log.gz 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84035.log 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53369.log.gz 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71106.log 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46246.log.gz 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68742.log 2024-03-06T07:33:39.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84035.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64687.log.gz 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71106.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.84035.log.gz 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71106.log.gz 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71987.log 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57560.log 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43648.log: /var/log/ceph/ceph-client.admin.71987.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68742.log.gz 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63143.log 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2024-03-06T07:33:39.745 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71987.log.gz 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.57560.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43380.log 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55003.log 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57560.log.gz 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63143.log.gz 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55003.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43380.log.gz 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42496.log 2024-03-06T07:33:39.746 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87753.log 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44865.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42496.log.gz 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76271.log 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42664.log 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60615.log 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87753.log.gz 2024-03-06T07:33:39.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42664.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76271.log.gz 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77403.log 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42664.log.gz 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38055.log 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58724.log 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60615.log.gz 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77403.log.gz 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71777.log 2024-03-06T07:33:39.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90330.log 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.38055.log.gz 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71777.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58724.log.gz 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.86288.log 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90330.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71777.log.gz 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42350.log 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86288.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90330.log.gz 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61559.log 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42350.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86288.log.gz 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72445.log 2024-03-06T07:33:39.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61559.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.45910.log 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42350.log.gz 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72445.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61559.log.gz 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67279.log 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72445.log.gz 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52492.log 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56946.log 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75608.log 2024-03-06T07:33:39.750 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67279.log.gz 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56946.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.61433.log 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56946.log.gz 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58598.log 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48494.log 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75608.log.gz 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61433.log.gz 2024-03-06T07:33:39.751 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85766.log 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58598.log.gz 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45108.log 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85766.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48494.log.gz 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59475.log 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85766.log.gz 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45108.log.gz 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42580.log 2024-03-06T07:33:39.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53201.log 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59475.log.gz 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42580.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68214.log 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42580.log.gz 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.53201.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56466.log 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65449.log 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53201.log.gz 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56466.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68214.log.gz 2024-03-06T07:33:39.753 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65743.log 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56466.log.gz 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66798.log 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65449.log.gz 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81094.log 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65743.log.gz 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56676.log 2024-03-06T07:33:39.754 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66798.log.gz 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56424.log 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56676.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.81094.log.gz 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79598.log 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65138.log 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56676.log.gz 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79598.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56424.log.gz 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53285.log 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79598.log.gz 2024-03-06T07:33:39.755 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81401.log 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71508.log 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65138.log.gz 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81401.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53285.log.gz 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84810.log 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81401.log.gz 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82374.log 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84810.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71508.log.gz 2024-03-06T07:33:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82374.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.84810.log.gz -- 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.72337.log 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82374.log.gz 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79224.log 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72337.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78332.log 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79224.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72337.log.gz 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58037.log 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%/var/log/ceph/ceph-client.admin.78332.log: -- replaced with /var/log/ceph/ceph-client.admin.79224.log.gz 2024-03-06T07:33:39.757 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88653.log 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78332.log.gz 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76686.log 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88653.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.58037.log.gz --verbose 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.57474.log 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76686.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.88653.log.gz 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89423.log 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76686.log.gz 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2024-03-06T07:33:39.758 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57474.log: /var/log/ceph/ceph-client.admin.89423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64477.log 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57474.log.gz 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89423.log.gz 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42136.log 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59139.log 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73590.log 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64477.log.gz 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59139.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42136.log.gzgzip 2024-03-06T07:33:39.759 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77268.log 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73590.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59139.log.gzgzip 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67442.log 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73590.log.gz 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56086.log 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77268.log.gz 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73166.log 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50592.log 2024-03-06T07:33:39.760 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67442.log.gz 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56086.log.gz 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82765.log 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73166.log.gz 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70129.log 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86895.log 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50592.log.gz 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70129.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82765.log.gz 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80589.log 2024-03-06T07:33:39.761 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70129.log.gz 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.86895.log: --verbose -- /var/log/ceph/ceph-client.admin.71232.log 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60531.log 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86895.log.gz 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80589.log.gz 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89628.log 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51370.log 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89628.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.71232.log.gz 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62783.log 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60531.log.gz 2024-03-06T07:33:39.762 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89628.log.gz 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67027.log 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62783.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51370.log.gz 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.79640.log 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62783.log.gz 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85472.log 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79640.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67027.log.gz 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88443.log 2024-03-06T07:33:39.763 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85472.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79640.log.gz 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85472.log.gz 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76850.log 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44062.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88443.log.gz 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81527.log 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43816.log 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76850.log.gz 2024-03-06T07:33:39.764 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70980.log 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43816.log: gzip -5 88.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.81527.log.gz 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.71731.log 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43816.log.gz 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89254.log 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71731.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.70980.log.gz 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.50088.log 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63889.log 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71731.log.gz 2024-03-06T07:33:39.765 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.89254.log.gz/var/log/ceph/ceph-client.admin.50088.log: 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50088.log.gz 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79266.log 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44537.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63889.log.gz 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55759.log 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79266.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62514.log 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6%/var/log/ceph/ceph-client.admin.55759.log: -- replaced with /var/log/ceph/ceph-client.admin.79266.log.gz 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64267.log 2024-03-06T07:33:39.766 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55759.log.gz 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62514.log.gz 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71298.log 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64267.log.gz 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89826.log 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84684.log 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71298.log.gz 2024-03-06T07:33:39.767 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80673.log 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89826.log.gz 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80673.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.84684.log.gz -5 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.90542.log 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43732.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80673.log.gz 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69066.log 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-03-06T07:33:39.768 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69066.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90542.log.gz 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44871.log 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69066.log.gz 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50907.log 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44871.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz -5 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49208.log 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44871.log.gz 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52841.log 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60891.log 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52841.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49208.log.gz 2024-03-06T07:33:39.769 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50907.log.gz 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80991.log 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52841.log.gz 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54545.log 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55549.log 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60891.log.gz 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77012.log 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80991.log.gz 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54545.log.gz 2024-03-06T07:33:39.770 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45784.log 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55549.log.gz 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72251.log 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77491.log 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77012.log.gz 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45784.log.gz 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73029.log 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72251.log.gz 2024-03-06T07:33:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89670.log 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73029.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77491.log.gz -5 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59013.log 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73029.log.gz 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47549.log 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%/var/log/ceph/ceph-client.admin.59013.log: -- replaced with /var/log/ceph/ceph-client.admin.89670.log.gz 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57995.log 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59013.log.gz 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80949.log 2024-03-06T07:33:39.772 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47549.log.gz 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78290.log 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77659.log 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57995.log.gz 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78290.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80949.log.gz 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82500.log 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78290.log.gz 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82416.log 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49748.log 2024-03-06T07:33:39.773 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77659.log.gz 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82500.log.gz 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82416.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82416.log.gz 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78019.log 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89068.log 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49748.log.gz 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66588.log 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78019.log.gz 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73945.log 2024-03-06T07:33:39.774 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89068.log.gz 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47591.log 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66588.log: /var/log/ceph/ceph-client.admin.73945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79392.log 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66588.log.gz 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47591.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73945.log.gz 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6%/var/log/ceph/ceph-client.admin.79392.log: -- replaced with /var/log/ceph/ceph-client.admin.47591.log.gz 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65575.log 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79392.log.gz 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81196.log 2024-03-06T07:33:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50442.log: /var/log/ceph/ceph-client.admin.65575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85934.log 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81196.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.65575.log.gz -5 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74480.log 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85934.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81196.log.gz 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82681.log 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74480.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.85934.log.gz -- 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.59055.log 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82681.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74480.log.gz 2024-03-06T07:33:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77617.log 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82681.log.gz 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67772.log 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59055.log: /var/log/ceph/ceph-client.admin.77617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68808.log 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59055.log.gz 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67772.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77617.log.gz 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44872.log 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78626.log 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68808.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67772.log.gz 2024-03-06T07:33:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44872.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68808.log.gz -5 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.89910.log 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44872.log.gz 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80715.log 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78626.log: /var/log/ceph/ceph-client.admin.89910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61769.log 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78626.log.gz 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80715.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89910.log.gz 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66882.log 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80715.log.gz 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75295.log 2024-03-06T07:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56634.log 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66882.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61769.log.gz 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79056.log 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66882.log.gz 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75295.log.gz 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84642.log 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56634.log: /var/log/ceph/ceph-client.admin.79056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56634.log.gz 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79056.log.gz 2024-03-06T07:33:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59223.log 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48290.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84642.log.gz 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89868.log 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59223.log: /var/log/ceph/ceph-client.admin.89868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89544.log 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59223.log.gz 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43978.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89868.log.gz 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46819.log 2024-03-06T07:33:39.780 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69024.log 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89544.log: /var/log/ceph/ceph-client.admin.46819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81947.log 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89544.log.gz 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46819.log.gz 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48578.log 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69024.log: /var/log/ceph/ceph-client.admin.81947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73380.log 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.69024.log.gz 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48578.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81947.log.gz 2024-03-06T07:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45209.log 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48578.log.gz 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54899.log 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77449.log 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45209.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73380.log.gz 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54899.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45209.log.gz 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46985.log 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54899.log.gz 2024-03-06T07:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81317.log 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77449.log: /var/log/ceph/ceph-client.admin.46985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48788.log 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77449.log.gz 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81317.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46985.log.gz 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90078.log 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81317.log.gz 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63185.log 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62909.log 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90078.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48788.log.gz 2024-03-06T07:33:39.783 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84077.log 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90078.log.gz 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63185.log.gz 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87189.log 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62909.log.gz 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50757.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84077.log.gz 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88569.log 2024-03-06T07:33:39.784 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47381.log 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81569.log 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87189.log.gz 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48746.log 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88569.log.gz 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47381.log: /var/log/ceph/ceph-client.admin.81569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56760.log 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47381.log.gz 2024-03-06T07:33:39.785 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81569.log.gz 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82458.log 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56760.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48746.log.gz 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.56760.log.gz /var/log/ceph/ceph-client.admin.58262.log 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45425.log 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82458.log: /var/log/ceph/ceph-client.admin.58262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68508.log 2024-03-06T07:33:39.786 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82458.log.gz 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89026.log 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45425.log.gz 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72819.log 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42772.log 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89026.log: /var/log/ceph/ceph-client.admin.72819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83951.log 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68508.log.gz 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.89026.log.gz 2024-03-06T07:33:39.787 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.72819.log.gz 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38333.log 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83951.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42772.log.gz 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57516.log 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44146.log: /var/log/ceph/ceph-client.admin.38333.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83951.log.gz 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57390.log 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2024-03-06T07:33:39.788 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38333.log.gz 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74767.log 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57516.log.gz 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48452.log 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57390.log: /var/log/ceph/ceph-client.admin.74767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86018.log 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57390.log.gz 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48452.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74767.log.gz 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53411.log 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48452.log.gz 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73735.log 2024-03-06T07:33:39.789 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86018.log: /var/log/ceph/ceph-client.admin.53411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86018.log.gz 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73735.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.53411.log.gz --verbose 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.44230.log 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73735.log.gz 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62737.log 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48536.log 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62737.log: gzip -5 88.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44230.log.gz /var/log/ceph/ceph-client.admin.83339.log 2024-03-06T07:33:39.790 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62737.log.gz 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75856.log 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48536.log.gz 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88089.log 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83339.log: /var/log/ceph/ceph-client.admin.75856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80501.log 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88089.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75856.log.gz 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71647.log 2024-03-06T07:33:39.791 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88089.log.gz 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71861.log 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83339.log.gz 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80501.log: /var/log/ceph/ceph-client.admin.71647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60933.log 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80501.log.gz 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71861.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71647.log.gz 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86709.log 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54960.log 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.71861.log.gz 2024-03-06T07:33:39.792 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86709.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60933.log.gz 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51118.log 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86709.log.gz 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64850.log 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54960.log: /var/log/ceph/ceph-client.admin.51118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87921.log 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54960.log.gz 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51118.log.gz 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64850.log.gz 2024-03-06T07:33:39.793 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89741.log 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87921.log: /var/log/ceph/ceph-client.admin.42007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74893.log 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87921.log.gz 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89741.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53243.log 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89741.log.gz 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51328.log 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74893.log: /var/log/ceph/ceph-client.admin.53243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72777.log 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74893.log.gz 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51328.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53243.log.gz 2024-03-06T07:33:39.794 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76560.log 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51328.log.gz /var/log/ceph/ceph-client.admin.47171.log 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72777.log: /var/log/ceph/ceph-client.admin.76560.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72777.log.gz 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83639.log 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76560.log.gz 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54671.log 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47171.log: /var/log/ceph/ceph-client.admin.83639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57604.log 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47171.log.gz 2024-03-06T07:33:39.795 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83639.log.gz 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54671.log.gz 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61685.log 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78519.log 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45206.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57604.log.gz 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87627.log 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61685.log.gz 2024-03-06T07:33:39.796 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71064.log 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78519.log: /var/log/ceph/ceph-client.admin.87627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78519.log.gz 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71064.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87627.log.gz 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63786.log 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71064.log.gz 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45207.log 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2024-03-06T07:33:39.797 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71022.log 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63786.log: /var/log/ceph/ceph-client.admin.45207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63599.log 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45207.log.gz 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57348.log 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86330.log 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.63786.log.gz 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63599.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71022.log.gz 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57348.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.63599.log.gz 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72651.log 2024-03-06T07:33:39.798 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57348.log.gz 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72029.log 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86330.log: /var/log/ceph/ceph-client.admin.72651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86330.log.gz 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3%/var/log/ceph/ceph-client.admin.72029.log: -- replaced with /var/log/ceph/ceph-client.admin.72651.log.gz 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74977.log 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43188.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72029.log.gz 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90414.log 2024-03-06T07:33:39.799 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80841.log 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74977.log: /var/log/ceph/ceph-client.admin.90414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85514.log 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74977.log.gz 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90414.log.gz 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86246.log 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64141.log 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85514.log: /var/log/ceph/ceph-client.admin.86246.log: 92.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85514.log.gz 2024-03-06T07:33:39.800 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80841.log.gz 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83105.log 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86246.log.gz 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose 87.7% -- -- replaced with /var/log/ceph/ceph-client.admin.64141.log.gz /var/log/ceph/ceph-client.admin.64225.log 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48332.log: /var/log/ceph/ceph-client.admin.83105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62558.log 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69918.log 2024-03-06T07:33:39.801 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5%/var/log/ceph/ceph-client.admin.62558.log: -- replaced with /var/log/ceph/ceph-client.admin.64225.log.gz 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48662.log 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.83105.log.gz 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69918.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62558.log.gz 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60401.log 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52056.log 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48662.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69918.log.gz 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85598.log 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48662.log.gz 2024-03-06T07:33:39.802 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52056.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60401.log.gz 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55423.log 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85598.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52056.log.gz 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87795.log 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85598.log.gz 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70850.log 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55423.log.gz 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86667.log 2024-03-06T07:33:39.803 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87795.log: /var/log/ceph/ceph-client.admin.70850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59349.log 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87795.log.gz 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70850.log.gz 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66198.log 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70003.log 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59349.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86667.log.gz 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42222.log 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59349.log.gz 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51412.log 2024-03-06T07:33:39.804 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70003.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66198.log.gz 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42222.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70003.log.gz -5 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60485.log 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42222.log.gz 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85286.log 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51412.log: /var/log/ceph/ceph-client.admin.60485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62346.log 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51412.log.gz 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60485.log.gz 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83597.log 2024-03-06T07:33:39.805 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.85286.log.gz --verbose 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-osd.3.log 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62346.log: /var/log/ceph/ceph-client.admin.83597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62346.log.gz 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83597.log.gz 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63396.log 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85892.log 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64309.log 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85892.log: gzip 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-03-06T07:33:39.806 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.90500.log 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64309.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63396.log.gz 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85892.log.gz 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78671.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64309.log.gz 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75337.log 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90500.log.gz 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66241.log 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83555.log 2024-03-06T07:33:39.807 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75337.log.gz 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66241.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.78671.log.gz 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50973.log 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66241.log.gz 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60783.log 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86541.log 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50973.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83555.log.gz 2024-03-06T07:33:39.808 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74354.log 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50973.log.gz 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62653.log 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60783.log.gz 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86541.log.gz 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48998.log 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62653.log: /var/log/ceph/ceph-client.admin.74438.log 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74354.log.gz 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69563.log 2024-03-06T07:33:39.809 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62653.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.74438.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53890.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48998.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69563.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.74438.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83021.log 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69563.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81863.log 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53890.log: /var/log/ceph/ceph-client.admin.83021.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53890.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47465.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83021.log.gz 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.810 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85640.log 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81863.log.gz 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52627.log 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85640.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47465.log.gz 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55507.log 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85640.log.gz 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59517.log 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52627.log.gz 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73208.log 2024-03-06T07:33:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59517.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55507.log.gz 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59517.log.gz 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49372.log 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73208.log.gz 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63931.log 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55885.log 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49372.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-03-06T07:33:39.812 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82937.log 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49372.log.gz 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63931.log.gz 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83405.log 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64729.log 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55885.log.gz 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83405.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82937.log.gz 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81154.log 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56550.log 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83405.log.gz 2024-03-06T07:33:39.813 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64729.log.gz 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80192.log 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68466.log 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56550.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81154.log.gz 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87273.log 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56550.log.gz 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68466.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80192.log.gz 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.80365.log 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68466.log.gz 2024-03-06T07:33:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79140.log 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87273.log.gz 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45742.log 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80365.log: /var/log/ceph/ceph-client.admin.79140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71689.log 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80365.log.gz 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79140.log.gz 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70669.log 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60573.log 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45742.log.gz 2024-03-06T07:33:39.815 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87147.log 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71689.log.gz 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60573.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70669.log.gz 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90246.log 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60573.log.gz 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65809.log 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87147.log: /var/log/ceph/ceph-client.admin.90246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49706.log 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87147.log.gz 2024-03-06T07:33:39.816 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65809.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90246.log.gz 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87585.log 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65809.log.gz 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56904.log 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75127.log 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49706.log.gz 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87585.log: /var/log/ceph/ceph-client.admin.56904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77743.log 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87585.log.gz 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.75127.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68958.log 2024-03-06T07:33:39.817 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56904.log.gz 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75127.log.gz 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.77743.log.gz --verbose 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.68130.log 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68958.log: /var/log/ceph/ceph-client.admin.56294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52757.log 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68130.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59433.log 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr: 90.7%/var/log/ceph/ceph-client.admin.52757.log: -- replaced with /var/log/ceph/ceph-client.admin.68958.log.gz 2024-03-06T07:33:39.818 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68130.log.gz 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58971.log 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52757.log.gz 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58971.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59433.log.gz 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62430.log 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46036.log 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45616.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58971.log.gz 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62430.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-03-06T07:33:39.819 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76061.log 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62430.log.gz 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46204.log 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46036.log: /var/log/ceph/ceph-client.admin.76061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71190.log 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46036.log.gz 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87483.log 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76061.log.gz 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46204.log.gz 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59559.log 2024-03-06T07:33:39.820 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71190.log.gz 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44868.log 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87483.log: /var/log/ceph/ceph-client.admin.59559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77533.log 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87483.log.gz 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44868.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59559.log.gz 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65659.log 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44868.log.gz 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48620.log 2024-03-06T07:33:39.821 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77533.log.gz 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80235.log 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65659.log: /var/log/ceph/ceph-client.admin.48620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65659.log.gz 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48620.log.gz 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53135.log 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80235.log.gz 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45110.log 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55339.log: /var/log/ceph/ceph-client.admin.53135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83513.log 2024-03-06T07:33:39.822 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49664.log 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45110.log.gz 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50046.log 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53135.log.gz 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.49664.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83513.log.gz 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47423.log 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50046.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49664.log.gz 2024-03-06T07:33:39.823 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66630.log 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50046.log.gz 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87711.log 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75379.log 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66630.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47423.log.gz 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87711.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66630.log.gz 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74230.log 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87711.log.gz 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.75379.log: --verbose -- /var/log/ceph/ceph-client.admin.69231.log 2024-03-06T07:33:39.824 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80150.log 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75379.log.gz 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69231.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74230.log.gz 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70585.log 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77096.log 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80150.log.gz 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70585.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69231.log.gz 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70585.log.gz 2024-03-06T07:33:39.825 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87231.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70171.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77096.log.gz 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87231.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83171.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87231.log.gz 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72861.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83171.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70171.log.gz 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86162.log 2024-03-06T07:33:39.826 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83171.log.gz 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72861.log.gz 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52967.log 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85388.log 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53760.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86162.log.gz 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57072.log 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-03-06T07:33:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60220.log 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52967.log.gz 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57072.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85388.log.gz 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85682.log 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82630.log 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60220.log: /var/log/ceph/ceph-client.admin.85682.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57072.log.gz 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69419.log 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60220.log.gz 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82630.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85682.log.gz 2024-03-06T07:33:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50004.log 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82630.log.gz 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61144.log 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50004.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69419.log.gz 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50004.log.gz 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67321.log 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79766.log 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61144.log.gz 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67321.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2024-03-06T07:33:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57138.log 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88257.log 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67321.log.gz 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47885.log 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79766.log.gz 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88257.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57138.log.gz 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55717.log 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47885.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88257.log.gz 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59853.log 2024-03-06T07:33:39.830 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47885.log.gz 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61727.log 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55717.log: /var/log/ceph/ceph-client.admin.59853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62695.log 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55717.log.gz 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59853.log.gz 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79556.log 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62695.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61727.log.gz 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53051.log 2024-03-06T07:33:39.831 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62695.log.gz 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84414.log 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52235.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79556.log.gz 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47801.log 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84414.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.53051.log.gz 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.85020.log 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84414.log.gz -5 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48704.log 2024-03-06T07:33:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47801.log: /var/log/ceph/ceph-client.admin.85020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88317.log 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47801.log.gz 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85020.log.gz 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58808.log 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.88317.log: --verbose -- /var/log/ceph/ceph-client.admin.75019.log 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48704.log.gz 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70713.log 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88317.log.gz 2024-03-06T07:33:39.833 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58808.log.gz 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59932.log 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49832.log 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75019.log.gz 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46162.log 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70713.log.gz 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59932.log.gz 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82895.log 2024-03-06T07:33:39.834 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49832.log.gz 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63744.log 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46162.log: /var/log/ceph/ceph-client.admin.82895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66504.log 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46162.log.gz 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3%/var/log/ceph/ceph-client.admin.63744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89194.log 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82895.log.gz 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79014.log 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66504.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63744.log.gz 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89194.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66504.log.gz 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77180.log 2024-03-06T07:33:39.835 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84582.log 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89194.log.gz 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79014.log: /var/log/ceph/ceph-client.admin.77180.log: gzip -5 --verbose 88.8% -- /var/log/ceph/ceph-client.admin.48956.log 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79014.log.gz 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77180.log.gz 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84582.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48956.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84582.log.gz 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74029.log 2024-03-06T07:33:39.836 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48956.log.gz 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63354.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64771.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74029.log: /var/log/ceph/ceph-client.admin.63354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80027.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74029.log.gz 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76728.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63354.log.gz 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75566.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64771.log.gz 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80027.log: /var/log/ceph/ceph-client.admin.76728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80108.log 2024-03-06T07:33:39.837 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73071.log 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76728.log.gz 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80027.log.gz 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80108.log: gzip 89.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.82723.log 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75566.log.gz 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80108.log.gz 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86499.log 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.82723.log: 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.73071.log.gz -- 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.86457.log 2024-03-06T07:33:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86499.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82723.log.gz 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.47255.log 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86457.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86499.log.gz 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55633.log 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86457.log.gz 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85808.log 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55633.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47255.log.gz 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.87105.log 2024-03-06T07:33:39.839 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85808.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53932.log 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85808.log.gz 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81611.log 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53932.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.87105.log.gz 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.87441.log 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81611.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53932.log.gz 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88131.log 2024-03-06T07:33:39.840 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81611.log.gz 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59769.log 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88131.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87441.log.gz 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.79434.log 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59769.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88131.log.gz 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49250.log 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59769.log.gz 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67111.log 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49250.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79434.log.gz 2024-03-06T07:33:39.841 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55297.log 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67111.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49250.log.gz 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67816.log 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67111.log.gz 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64435.log 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67816.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55297.log.gz 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64015.log 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64435.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67816.log.gz 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49453.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64435.log.gz 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79308.log 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%/var/log/ceph/ceph-client.admin.49453.log: -- replaced with /var/log/ceph/ceph-client.admin.64015.log.gz 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59601.log 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49453.log.gz 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53453.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54503.log 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79308.log.gz 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59601.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz -- 2024-03-06T07:33:39.843 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.88695.log 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65851.log 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59601.log.gz 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88695.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54503.log.gz 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81238.log 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88695.log.gz 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44536.log 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67153.log 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81238.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65851.log.gz 2024-03-06T07:33:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44536.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.81238.log.gz 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.82314.log 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67153.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44536.log.gz 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81653.log 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68382.log 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67153.log.gz 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44874.log 88.2% 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82314.log.gz 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81653.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58472.log 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68382.log: 2024-03-06T07:33:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44874.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68382.log.gz 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52154.log 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68088.log 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44874.log.gz 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58472.log: /var/log/ceph/ceph-client.admin.52154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70808.log 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58472.log.gz 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4%/var/log/ceph/ceph-client.admin.68088.log: gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.52154.log.gz --verbose 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.43590.log 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90288.log 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68088.log.gz 2024-03-06T07:33:39.846 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43590.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70808.log.gz 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84287.log 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43590.log.gz 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76187.log 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.90288.log: /var/log/ceph/ceph-client.admin.84287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65403.log 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7%/var/log/ceph/ceph-client.admin.76187.log: -- replaced with /var/log/ceph/ceph-client.admin.90288.log.gz 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.84287.log.gz --verbose 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.51683.log 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65403.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76187.log.gz 2024-03-06T07:33:39.847 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78163.log 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65403.log.gz 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72987.log 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78163.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51683.log.gz 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73422.log 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78163.log.gz 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54335.log 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72987.log: /var/log/ceph/ceph-client.admin.73422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70543.log 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72987.log.gz 2024-03-06T07:33:39.848 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54335.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73422.log.gz 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.79808.log 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70543.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54335.log.gz 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51244.log 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70543.log.gz 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59685.log 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51244.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79808.log.gz 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76602.log 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77361.log 2024-03-06T07:33:39.849 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51244.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59685.log: /var/log/ceph/ceph-client.admin.76602.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59685.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76602.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76950.log 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77361.log: /var/log/ceph/ceph-client.admin.45994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57222.log 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77361.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90372.log 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57222.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76950.log.gz 2024-03-06T07:33:39.850 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42538.log 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57222.log.gz 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56592.log/var/log/ceph/ceph-client.admin.90372.log: 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67604.log 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90372.log.gz 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58640.log 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42538.log.gz 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67604.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56592.log.gz 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90120.log 2024-03-06T07:33:39.851 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67604.log.gz 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86751.log 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58640.log: /var/log/ceph/ceph-client.admin.90120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58346.log 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58640.log.gz 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86751.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90120.log.gz 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57030.log 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.58346.log: --verbose -- /var/log/ceph/ceph-client.admin.44188.log 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86751.log.gz 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57030.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58346.log.gz 2024-03-06T07:33:39.852 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87525.log 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57030.log.gz 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85976.log 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44188.log: /var/log/ceph/ceph-client.admin.87525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68424.log 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44188.log.gz 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85976.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87525.log.gz 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55255.log 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68424.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.85976.log.gz 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51977.log 2024-03-06T07:33:39.853 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68424.log.gz 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87357.log 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55255.log: /var/log/ceph/ceph-client.admin.51977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84726.log 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55255.log.gz 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.87357.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71596.log 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51977.log.gz 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87357.log.gz 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42266.log 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.84726.log.gz -5 2024-03-06T07:33:39.854 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59097.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71596.log: /var/log/ceph/ceph-client.admin.42266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89297.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71596.log.gz 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42266.log.gz 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75790.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.8%/var/log/ceph/ceph-client.admin.89297.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.59097.log.gz --verbose 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-osd.0.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75790.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89297.log.gz 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65491.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82853.log 2024-03-06T07:33:39.855 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.65491.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75790.log.gz 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77977.log 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82853.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65491.log.gz 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71382.log 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82853.log.gz 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77977.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69833.log 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71382.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77977.log.gz 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81695.log 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9%/var/log/ceph/ceph-client.admin.69833.log: -- replaced with /var/log/ceph/ceph-client.admin.71382.log.gz 2024-03-06T07:33:39.856 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69833.log.gz 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69689.log 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57785.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81695.log.gz 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86625.log 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.69689.log: -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.60262.log 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69689.log.gz 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69377.log 2024-03-06T07:33:39.857 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86625.log.gz 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63227.log 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60262.log.gz 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.32106.log 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69377.log.gz 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77701.log 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63227.log.gz 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.tmp-client.admin.32106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58514.log 2024-03-06T07:33:39.858 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.32106.log.gz 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78434.log 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58514.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77701.log.gz 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53495.log 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78434.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58514.log.gz 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53579.log 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78434.log.gz 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75935.log 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53495.log.gz 2024-03-06T07:33:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77319.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53579.log.gz 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78061.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75935.log.gz 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77319.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77319.log.gz 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49918.log 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75169.log 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78061.log.gz 2024-03-06T07:33:39.860 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74619.log 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75169.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49918.log.gz 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.75169.log.gz 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88611.log 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48914.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74619.log.gz 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58682.log 88.1% 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-03-06T07:33:39.861 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62993.log 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88611.log.gz 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.58682.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76103.log 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62993.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58682.log.gz 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72173.log 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62993.log.gz 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47759.log 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76103.log.gz 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2024-03-06T07:33:39.862 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72173.log.gz 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43548.log 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66061.log 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47759.log.gz 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43548.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43548.log.gz 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.43506.log 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77054.log 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83909.log 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43506.log.gz 2024-03-06T07:33:39.863 INFO:teuthology.orchestra.run.smithi017.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.66061.log.gz 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76313.log 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77054.log.gz/var/log/ceph/ceph-client.admin.83909.log: 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74524.log 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76313.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.78737.log 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83909.log.gz 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76313.log.gz 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose --/var/log/ceph/ceph-client.admin.74524.log: /var/log/ceph/ceph-client.admin.74851.log 2024-03-06T07:33:39.864 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78737.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74524.log.gz 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54118.log 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74851.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.78737.log.gz 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89952.log 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74851.log.gz 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66019.log 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89952.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54118.log.gz 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66019.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89952.log.gz 2024-03-06T07:33:39.865 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64645.log 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66019.log.gz 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.70045.log 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83213.log 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64645.log.gz 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70501.log 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70045.log.gz 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56006.log 2024-03-06T07:33:39.866 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83213.log.gz 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76476.log 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70501.log: /var/log/ceph/ceph-client.admin.56006.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70501.log.gz 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56006.log.gz 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44272.log 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64603.log 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44272.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76476.log.gz 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60699.log 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-06T07:33:39.867 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44272.log.gz 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71903.log 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64603.log: /var/log/ceph/ceph-client.admin.60699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59727.log 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64603.log.gz 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60699.log.gz 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.71903.log: /var/log/ceph/ceph-client.admin.59727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53327.log 88.6% 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59727.log.gz 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71903.log.gz 2024-03-06T07:33:39.868 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73650.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62951.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53327.log.gz 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73861.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73650.log.gz 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67522.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62951.log.gz 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69647.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73861.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8%/var/log/ceph/ceph-client.admin.67522.log: -- replaced with /var/log/ceph/ceph-client.admin.73861.log.gz 2024-03-06T07:33:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69647.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69647.log.gz 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61853.log 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mds.c.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67522.log.gz 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56802.log 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68916.log 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61853.log.gz 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56802.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56802.log.gz 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77785.log 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74809.log 2024-03-06T07:33:39.870 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68916.log.gz 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77785.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77785.log.gz 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84456.log 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74396.log 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74809.log.gz 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84456.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84456.log.gz 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70757.log 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69189.log 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74396.log.gz 2024-03-06T07:33:39.871 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70757.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70757.log.gz 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89339.log 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69189.log.gz 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83063.log 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89339.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89339.log.gz 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70627.log 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89783.log 2024-03-06T07:33:39.872 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83063.log.gz 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88816.log 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.89783.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70627.log.gz 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67860.log 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89783.log.gz 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.88816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57655.log 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67860.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88816.log.gz 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67860.log.gz 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73292.log 2024-03-06T07:33:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90162.log 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57655.log: /var/log/ceph/ceph-client.admin.73292.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57655.log.gz 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73292.log.gz 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76229.log 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88527.log 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76229.log: /var/log/ceph/ceph-client.admin.90162.log: 88.7% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90162.log.gz 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76229.log.gz 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68700.log 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.88527.log: --verbose -- /var/log/ceph/ceph-client.admin.62825.log 2024-03-06T07:33:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68700.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88527.log.gz 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81485.log 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62825.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68700.log.gz 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62825.log.gz 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55843.log 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.81485.log: -- /var/log/ceph/ceph-client.admin.49790.log 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81485.log.gz 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54058.log 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49790.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55843.log.gz 2024-03-06T07:33:39.875 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49790.log.gz 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63557.log 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54058.log.gz 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51641.log 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63557.log.gz 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59181.log 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-03-06T07:33:39.876 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59181.log: /var/log/ceph/ceph-client.admin.51641.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51641.log.gz 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59181.log.gz 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87963.log 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56718.log 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57697.log: /var/log/ceph/ceph-client.admin.87963.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87963.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74935.log 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72735.log 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56718.log: /var/log/ceph/ceph-client.admin.74935.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74935.log.gz 2024-03-06T07:33:39.877 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56718.log.gz 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85104.log 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73819.log 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72735.log.gz 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57264.log 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85104.log.gz 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73819.log.gz 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71340.log 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65361.log 2024-03-06T07:33:39.878 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57264.log: /var/log/ceph/ceph-client.admin.71340.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57264.log.gz 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60825.log 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71340.log.gz 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51538.log 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60825.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.65361.log.gz 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78803.log 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60825.log.gz 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63077.log 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78803.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66401.log 2024-03-06T07:33:39.879 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78803.log.gz 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80417.log 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66401.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63077.log.gz 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66401.log.gz 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42052.log 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.80417.log: --verbose -- /var/log/ceph/ceph-client.admin.80907.log 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80417.log.gz 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49622.log 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80907.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42052.log.gz 2024-03-06T07:33:39.880 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72567.log 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.80907.log.gz 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49622.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49622.log.gz 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60012.log 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69503.log 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72567.log: /var/log/ceph/ceph-client.admin.60012.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72567.log.gz 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60012.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55045.log 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.881 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61373.log 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69503.log: /var/log/ceph/ceph-client.admin.55045.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55045.log.gz 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69503.log.gz 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79724.log 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61373.log.gz 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55213.log 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79724.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79724.log.gz 2024-03-06T07:33:39.882 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81989.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55213.log: -- /var/log/ceph/ceph-client.admin.52404.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55213.log.gz 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.81989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84498.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52404.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81989.log.gz 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52404.log.gz 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59811.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.84498.log: -- /var/log/ceph/ceph-client.admin.52715.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84498.log.gz 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49292.log 2024-03-06T07:33:39.883 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59811.log.gz 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52715.log.gz 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78477.log 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65224.log 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49292.log: /var/log/ceph/ceph-client.admin.78477.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49292.log.gz 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78477.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88984.log 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53537.log 2024-03-06T07:33:39.884 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65224.log: /var/log/ceph/ceph-client.admin.88984.log: 87.5% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88984.log.gz 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65224.log.gz 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69108.log 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60178.log 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53537.log.gz 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64519.log 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60178.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60178.log.gz 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69108.log.gz 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84329.log 2024-03-06T07:33:39.885 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64519.log.gz 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82194.log 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84329.log.gz 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84852.log 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.82194.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82194.log.gz 2024-03-06T07:33:39.886 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43422.log 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55381.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84852.log.gz 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63515.log 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54629.log 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43422.log: /var/log/ceph/ceph-client.admin.63515.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43422.log.gz 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63515.log.gzgzip 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75748.log 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52925.log 2024-03-06T07:33:39.887 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75748.log: /var/log/ceph/ceph-client.admin.54629.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54629.log.gz 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75748.log.gz 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88005.log 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52925.log: /var/log/ceph/ceph-client.admin.88005.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52925.log.gz 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88005.log.gz 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74568.log 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86373.log 2024-03-06T07:33:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74568.log: /var/log/ceph/ceph-client.admin.47045.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74568.log.gz 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66355.log 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60350.log 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86373.log: /var/log/ceph/ceph-client.admin.66355.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86373.log.gz 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66355.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46078.log 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60350.log: /var/log/ceph/ceph-client.admin.42622.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60350.log.gz 2024-03-06T07:33:39.889 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83993.log 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68028.log 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83993.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46078.log.gz 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83993.log.gz 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88215.log 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83723.log 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68028.log: /var/log/ceph/ceph-client.admin.88215.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68028.log.gz 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88215.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2024-03-06T07:33:39.890 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56336.log 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.83723.log: /var/log/ceph/ceph-client.admin.50400.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83723.log.gz 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52799.log 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75253.log 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56336.log: /var/log/ceph/ceph-client.admin.52799.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56336.log.gz 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85724.log 2024-03-06T07:33:39.891 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52799.log.gz 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62867.log 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.85724.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75253.log.gz 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85724.log.gz 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50274.log 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84161.log 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62867.log: /var/log/ceph/ceph-client.admin.50274.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62867.log.gz 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50274.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-03-06T07:33:39.892 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50358.log 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84161.log: /var/log/ceph/ceph-client.admin.43858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79929.log 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50358.log: 88.0% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50358.log.gz 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.84161.log.gz 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86120.log 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65319.log 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86120.log: /var/log/ceph/ceph-client.admin.79929.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79929.log.gz 2024-03-06T07:33:39.893 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86120.log.gz 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44314.log 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48872.log 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.65319.log: /var/log/ceph/ceph-client.admin.44314.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65319.log.gz 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44314.log.gz 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69461.log 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51851.log: /var/log/ceph/ceph-client.admin.48872.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48872.log.gz 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-03-06T07:33:39.894 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86204.log 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69461.log: -- /var/log/ceph/ceph-client.admin.80631.log 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69461.log.gz 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.86204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64561.log 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.80631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86204.log.gz 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80631.log.gz 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55801.log 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64561.log: /var/log/ceph/ceph-mds.a.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64561.log.gz 2024-03-06T07:33:39.895 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68340.log 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55801.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55801.log.gz 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63973.log 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68340.log: /var/log/ceph/ceph-client.admin.63973.log: 89.3% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68340.log.gz 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63973.log.gz 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75061.log 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60136.log 2024-03-06T07:33:39.896 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42094.log: /var/log/ceph/ceph-client.admin.75061.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77911.log 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60136.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.75061.log.gz 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60136.log.gz 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71148.log 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44869.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77911.log.gz 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84978.log 2024-03-06T07:33:39.897 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.71148.log: --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71148.log.gz 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.84978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50823.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84978.log.gz 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61017.log 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54016.log 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50634.log: /var/log/ceph/ceph-client.admin.61017.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.61017.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-03-06T07:33:39.898 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54016.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54016.log.gz 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47675.log 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53802.log: /var/log/ceph/ceph-client.admin.49040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-03-06T07:33:39.899 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47675.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47675.log.gz 2024-03-06T07:33:39.900 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-06T07:33:39.900 INFO:teuthology.orchestra.run.smithi163.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-06T07:33:39.900 INFO:teuthology.orchestra.run.smithi017.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-06T07:33:40.565 INFO:teuthology.orchestra.run.smithi017.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-06T07:33:40.607 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-06T07:33:40.704 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-06T07:33:40.843 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-06T07:33:40.862 INFO:teuthology.orchestra.run.smithi163.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-06T07:33:40.864 INFO:teuthology.orchestra.run.smithi163.stderr: 2024-03-06T07:33:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:real 0m1.613s 2024-03-06T07:33:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:user 0m7.148s 2024-03-06T07:33:40.864 INFO:teuthology.orchestra.run.smithi163.stderr:sys 0m0.275s 2024-03-06T07:33:40.871 INFO:teuthology.orchestra.run.smithi017.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-06T07:33:40.883 INFO:teuthology.orchestra.run.smithi017.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-06T07:33:41.007 INFO:teuthology.orchestra.run.smithi017.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-06T07:33:41.327 INFO:teuthology.orchestra.run.smithi017.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-06T07:33:41.329 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-06T07:33:41.329 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m2.111s 2024-03-06T07:33:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m8.121s 2024-03-06T07:33:41.330 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.786s 2024-03-06T07:33:41.330 INFO:tasks.ceph:Archiving logs... 2024-03-06T07:33:41.330 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi017/log 2024-03-06T07:33:41.331 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-06T07:33:43.254 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi163/log 2024-03-06T07:33:43.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-06T07:33:44.238 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-06T07:33:44.247 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-06T07:33:44.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-06T07:33:44.250 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-06T07:33:44.343 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-06T07:33:44.352 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-06T07:33:44.353 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:33:44.355 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-06T07:33:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:33:44.371 INFO:teuthology.orchestra.run.smithi163.stderr:bash: line 1: ntpq: command not found 2024-03-06T07:33:44.374 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:33:44.374 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-03-06T07:33:44.374 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 +1844us[+1815us] +/- 99ms 2024-03-06T07:33:44.375 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv02.front.sepia.ceph.com 3 6 377 20 -1059us[-1088us] +/- 54ms 2024-03-06T07:33:44.375 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +83us[ +54us] +/- 90ms 2024-03-06T07:33:44.375 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:33:44.376 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-06T07:33:44.376 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2024-03-06T07:33:44.376 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +1866us[+1855us] +/- 99ms 2024-03-06T07:33:44.377 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -1040us[-1051us] +/- 54ms 2024-03-06T07:33:44.377 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +96us[ +84us] +/- 90ms 2024-03-06T07:33:44.377 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-06T07:33:44.377 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-06T07:33:44.386 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-06T07:33:44.386 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-06T07:33:44.409 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-06T07:33:44.517 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-06T07:33:44.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:33:44.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:44.622 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-06T07:33:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:44.653 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-06T07:33:44.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-06T07:33:44.683 INFO:teuthology.task.internal:Duration was 1320.439989 seconds 2024-03-06T07:33:44.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-06T07:33:44.717 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-06T07:33:44.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-06T07:33:44.720 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-06T07:33:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:33:44.763 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-06T07:33:45.009 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-06T07:33:45.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-03-06T07:33:45.009 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-06T07:33:45.028 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2024-03-06T07:33:45.029 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-06T07:33:45.048 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-06T07:33:45.049 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-06T07:33:45.071 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-06T07:33:45.116 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-06T07:33:45.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-06T07:33:45.139 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-06T07:33:45.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-06T07:33:45.287 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-06T07:33:45.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-06T07:33:45.313 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-06T07:33:45.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-06T07:33:45.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:45.356 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:45.378 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-03-06T07:33:45.407 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2024-03-06T07:33:45.422 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:45.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:45.448 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-06T07:33:45.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-06T07:33:45.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-06T07:33:45.486 INFO:teuthology.task.internal:Transferring archived files... 2024-03-06T07:33:45.487 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi017 2024-03-06T07:33:45.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-06T07:33:45.525 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049/remote/smithi163 2024-03-06T07:33:45.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-06T07:33:45.560 INFO:teuthology.task.internal:Removing archive directory... 2024-03-06T07:33:45.560 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-06T07:33:45.562 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-06T07:33:45.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-06T07:33:45.620 INFO:teuthology.task.internal:Not uploading archives. 2024-03-06T07:33:45.620 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-06T07:33:45.655 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-06T07:33:45.656 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-06T07:33:45.658 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-06T07:33:45.669 INFO:teuthology.orchestra.run.smithi017.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 6 07:33 /home/ubuntu/cephtest 2024-03-06T07:33:45.672 INFO:teuthology.orchestra.run.smithi163.stdout: 265300 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 6 07:33 /home/ubuntu/cephtest 2024-03-06T07:33:45.673 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-06T07:33:45.730 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-06T07:33:45.785 INFO:teuthology.nuke:Checking targets against current locks 2024-03-06T07:33:45.810 DEBUG:teuthology.nuke:shortname: smithi017 2024-03-06T07:33:45.810 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:33:45.832 DEBUG:teuthology.nuke:shortname: smithi163 2024-03-06T07:33:45.832 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-06T07:33:45.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:55.667680', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAcrt7eHGyP45YOFjqvGwfgQruR9VE8zc0jHwe2MqzBnRuLZJfuLrloiAoLCPGAIdrYHw1ZthpXAFtdi5L8irSc='} 2024-03-06T07:33:45.858 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-06_07:00:27-fs-wip-lusov-qdb-distro-default-smithi/7583049', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-06 07:01:55.669376', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJTmalFYrhoj0FNk44EdEqJPl48O2hb4I1sAh3U3CBL03a7DTQ+SCm7f6Ki9qDFzw+00R9Y1L051xGFBKzR/JZw='} 2024-03-06T07:33:45.888 INFO:teuthology.orchestra.console.smithi017:Power off 2024-03-06T07:33:45.888 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-06T07:33:45.912 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-03-06T07:33:45.912 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:45.929 INFO:teuthology.orchestra.console.smithi163:Power off 2024-03-06T07:33:45.929 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-06T07:33:45.939 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-06T07:33:45.952 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2024-03-06T07:33:45.952 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:45.974 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-06T07:33:49.941 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:49.976 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:50.052 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-06T07:33:50.084 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2024-03-06T07:33:54.056 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:54.085 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-06T07:33:54.167 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-03-06T07:33:54.194 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2024-03-06T07:33:54.268 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-03-06T07:33:54.295 INFO:teuthology.orchestra.console.smithi163:Power off completed 2024-03-06T07:33:54.400 INFO:teuthology.run:Summary data: description: fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/failover} duration: 1320.4399888515472 failure_reason: 'Test failure: test_down_twice (tasks.cephfs.test_failover.TestClusterResize)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a393e455f36c42c5907734f5d79b1903 status: fail success: false 2024-03-06T07:33:54.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-06T07:33:54.475 INFO:teuthology.run:FAIL